Index: projects/netbsd-tests-update-12/Makefile =================================================================== --- projects/netbsd-tests-update-12/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/Makefile (revision 305172) @@ -1,585 +1,590 @@ # # $FreeBSD$ # # The user-driven targets are: # # universe - *Really* build *everything* (buildworld and # all kernels on all architectures). # tinderbox - Same as universe, but presents a list of failed build # targets and exits with an error if there were any. # buildworld - Rebuild *everything*, including glue to help do # upgrades. # installworld - Install everything built by "buildworld". # world - buildworld + installworld, no kernel. # buildkernel - Rebuild the kernel and the kernel-modules. # installkernel - Install the kernel and the kernel-modules. # installkernel.debug # reinstallkernel - Reinstall the kernel and the kernel-modules. # reinstallkernel.debug # kernel - buildkernel + installkernel. # kernel-toolchain - Builds the subset of world necessary to build a kernel # kernel-toolchains - Build kernel-toolchain for all universe targets. # doxygen - Build API documentation of the kernel, needs doxygen. # update - Convenient way to update your source tree(s). # checkworld - Run test suite on installed world. # check-old - List obsolete directories/files/libraries. # check-old-dirs - List obsolete directories. # check-old-files - List obsolete files. # check-old-libs - List obsolete libraries. # delete-old - Delete obsolete directories/files. # delete-old-dirs - Delete obsolete directories. # delete-old-files - Delete obsolete files. # delete-old-libs - Delete obsolete libraries. # targets - Print a list of supported TARGET/TARGET_ARCH pairs # for world and kernel targets. # toolchains - Build a toolchain for all world and kernel targets. # xdev - xdev-build + xdev-install for the architecture # specified with XDEV and XDEV_ARCH. # xdev-build - Build cross-development tools. # xdev-install - Install cross-development tools. # xdev-links - Create traditional links in /usr/bin for cc, etc # native-xtools - Create host binaries that produce target objects # for use in qemu user-mode jails. # # "quick" way to test all kernel builds: # _jflag=`sysctl -n hw.ncpu` # _jflag=$(($_jflag * 2)) # [ $_jflag -gt 12 ] && _jflag=12 # make universe -DMAKE_JUST_KERNELS JFLAG=-j${_jflag} # # This makefile is simple by design. The FreeBSD make automatically reads # the /usr/share/mk/sys.mk unless the -m argument is specified on the # command line. By keeping this makefile simple, it doesn't matter too # much how different the installed mk files are from those in the source # tree. This makefile executes a child make process, forcing it to use # the mk files from the source tree which are supposed to DTRT. # # Most of the user-driven targets (as listed above) are implemented in # Makefile.inc1. The exceptions are universe, tinderbox and targets. # # If you want to build your system from source be sure that /usr/obj has # at least 6GB of diskspace available. A complete 'universe' build requires # about 100GB of space. # # For individuals wanting to build from the sources currently on their # system, the simple instructions are: # # 1. `cd /usr/src' (or to the directory containing your source tree). # 2. Define `HISTORICAL_MAKE_WORLD' variable (see README). # 3. `make world' # # For individuals wanting to upgrade their sources (even if only a # delta of a few days): # # 1. `cd /usr/src' (or to the directory containing your source tree). # 2. `make buildworld' # 3. `make buildkernel KERNCONF=YOUR_KERNEL_HERE' (default is GENERIC). # 4. `make installkernel KERNCONF=YOUR_KERNEL_HERE' (default is GENERIC). # [steps 3. & 4. can be combined by using the "kernel" target] # 5. `reboot' (in single user mode: boot -s from the loader prompt). # 6. `mergemaster -p' # 7. `make installworld' # 8. `mergemaster' (you may wish to use -i, along with -U or -F). # 9. `make delete-old' # 10. `reboot' # 11. `make delete-old-libs' (in case no 3rd party program uses them anymore) # # See src/UPDATING `COMMON ITEMS' for more complete information. # # If TARGET=machine (e.g. powerpc, sparc64, ...) is specified you can # cross build world for other machine types using the buildworld target, # and once the world is built you can cross build a kernel using the # buildkernel target. # # Define the user-driven targets. These are listed here in alphabetical # order, but that's not important. # # Targets that begin with underscore are internal targets intended for # developer convenience only. They are intentionally not documented and # completely subject to change without notice. # # For more information, see the build(7) manual page. # # This is included so CC is set to ccache for -V, and COMPILER_TYPE/VERSION # can be cached for sub-makes. .if ${MAKE_VERSION} >= 20140620 && defined(.PARSEDIR) .include .endif # Note: we use this awkward construct to be compatible with FreeBSD's # old make used in 10.0 and 9.2 and earlier. -.if defined(MK_DIRDEPS_BUILD) && ${MK_DIRDEPS_BUILD} == "yes" && !make(showconfig) +.if defined(MK_DIRDEPS_BUILD) && ${MK_DIRDEPS_BUILD} == "yes" && \ + !make(showconfig) && !make(print-dir) # targets/Makefile plays the role of top-level .include "targets/Makefile" .else TGTS= all all-man buildenv buildenvvars buildkernel buildworld \ check check-old check-old-dirs check-old-files check-old-libs \ checkdpadd checkworld clean cleandepend cleandir cleanworld \ delete-old delete-old-dirs delete-old-files delete-old-libs \ depend distribute distributekernel distributekernel.debug \ distributeworld distrib-dirs distribution doxygen \ everything hier hierarchy install installcheck installkernel \ installkernel.debug packagekernel packageworld \ reinstallkernel reinstallkernel.debug \ installworld kernel-toolchain libraries lint maninstall \ obj objlink rerelease showconfig tags toolchain update \ _worldtmp _legacy _bootstrap-tools _cleanobj _obj \ _build-tools _cross-tools _includes _libraries \ build32 distribute32 install32 buildsoft distributesoft installsoft \ builddtb xdev xdev-build xdev-install \ xdev-links native-xtools stageworld stagekernel stage-packages \ create-world-packages create-kernel-packages create-packages \ packages installconfig real-packages sign-packages package-pkg \ - test-system-compiler + print-dir test-system-compiler # XXX: r156740: This can't work since bsd.subdir.mk is not included ever. # It will only work for SUBDIR_TARGETS in make.conf. TGTS+= ${SUBDIR_TARGETS} BITGTS= files includes BITGTS:=${BITGTS} ${BITGTS:S/^/build/} ${BITGTS:S/^/install/} TGTS+= ${BITGTS} # Only some targets are allowed to use meta mode. Others get it # disabled. In some cases, such as 'install', meta mode can be dangerous # as a cookie may be used to prevent redundant installations (such as # for WORLDTMP staging). For DESTDIR=/ we always want to install though. # For other cases, such as delete-old-libs, meta mode may break # the interactive tty prompt. The safest route is to just whitelist # the ones that benefit from it. META_TGT_WHITELIST+= \ _* build32 buildfiles buildincludes buildkernel buildsoft \ buildworld everything kernel-toolchain kernel-toolchains kernel \ kernels libraries native-xtools showconfig test-system-compiler \ tinderbox toolchain \ toolchains universe world worlds xdev xdev-build .ORDER: buildworld installworld .ORDER: buildworld distributeworld .ORDER: buildworld buildkernel .ORDER: installworld distribution .ORDER: installworld installkernel .ORDER: buildkernel installkernel .ORDER: buildkernel installkernel.debug .ORDER: buildkernel reinstallkernel .ORDER: buildkernel reinstallkernel.debug PATH= /sbin:/bin:/usr/sbin:/usr/bin MAKEOBJDIRPREFIX?= /usr/obj _MAKEOBJDIRPREFIX!= /usr/bin/env -i PATH=${PATH} MK_AUTO_OBJ=no ${MAKE} \ ${.MAKEFLAGS:MMAKEOBJDIRPREFIX=*} __MAKE_CONF=${__MAKE_CONF} \ -f /dev/null -V MAKEOBJDIRPREFIX dummy .if !empty(_MAKEOBJDIRPREFIX) .error MAKEOBJDIRPREFIX can only be set in environment, not as a global\ (in make.conf(5)) or command-line variable. .endif # We often need to use the tree's version of make to build it. # Choices add to complexity though. # We cannot blindly use a make which may not be the one we want # so be exlicit - until all choice is removed. WANT_MAKE= bmake .if !empty(.MAKE.MODE:Mmeta) # 20160604 - support missing-meta,missing-filemon and performance improvements WANT_MAKE_VERSION= 20160604 .else # 20160220 - support .dinclude for FAST_DEPEND. WANT_MAKE_VERSION= 20160220 .endif MYMAKE= ${MAKEOBJDIRPREFIX}${.CURDIR}/make.${MACHINE}/${WANT_MAKE} .if defined(.PARSEDIR) HAVE_MAKE= bmake .else HAVE_MAKE= fmake .endif .if ${HAVE_MAKE} != ${WANT_MAKE} || \ (defined(WANT_MAKE_VERSION) && ${MAKE_VERSION} < ${WANT_MAKE_VERSION}) NEED_MAKE_UPGRADE= t .endif .if exists(${MYMAKE}) SUB_MAKE:= ${MYMAKE} -m ${.CURDIR}/share/mk .elif defined(NEED_MAKE_UPGRADE) # It may not exist yet but we may cause it to. # In the case of fmake, upgrade_checks may cause a newer version to be built. SUB_MAKE= `test -x ${MYMAKE} && echo ${MYMAKE} || echo ${MAKE}` \ -m ${.CURDIR}/share/mk .else SUB_MAKE= ${MAKE} -m ${.CURDIR}/share/mk .endif _MAKE= PATH=${PATH} MAKE_CMD=${MAKE} ${SUB_MAKE} -f Makefile.inc1 \ TARGET=${_TARGET} TARGET_ARCH=${_TARGET_ARCH} # Only allow meta mode for the whitelisted targets. See META_TGT_WHITELIST # above. .for _tgt in ${META_TGT_WHITELIST} .if make(${_tgt}) _CAN_USE_META_MODE?= yes .endif .endfor .if !defined(_CAN_USE_META_MODE) _MAKE+= MK_META_MODE=no .if defined(.PARSEDIR) .unexport META_MODE .endif .elif defined(MK_META_MODE) && ${MK_META_MODE} == "yes" .if !exists(/dev/filemon) && !defined(NO_FILEMON) && !make(showconfig) # Require filemon be loaded to provide a working incremental build .error ${.newline}ERROR: The filemon module (/dev/filemon) is not loaded. \ ${.newline}ERROR: WITH_META_MODE is enabled but requires filemon for an incremental build. \ ${.newline}ERROR: 'kldload filemon' or pass -DNO_FILEMON to suppress this error. .endif # !exists(/dev/filemon) && !defined(NO_FILEMON) .endif # !defined(_CAN_USE_META_MODE) # Guess machine architecture from machine type, and vice versa. .if !defined(TARGET_ARCH) && defined(TARGET) _TARGET_ARCH= ${TARGET:S/pc98/i386/:S/arm64/aarch64/} .elif !defined(TARGET) && defined(TARGET_ARCH) && \ ${TARGET_ARCH} != ${MACHINE_ARCH} _TARGET= ${TARGET_ARCH:C/mips(n32|64)?(el)?/mips/:C/arm(v6)?(eb)?/arm/:C/aarch64/arm64/:C/powerpc64/powerpc/:C/riscv64/riscv/} .endif .if defined(TARGET) && !defined(_TARGET) _TARGET=${TARGET} .endif .if defined(TARGET_ARCH) && !defined(_TARGET_ARCH) _TARGET_ARCH=${TARGET_ARCH} .endif # for historical compatibility for xdev targets .if defined(XDEV) _TARGET= ${XDEV} .endif .if defined(XDEV_ARCH) _TARGET_ARCH= ${XDEV_ARCH} .endif # Otherwise, default to current machine type and architecture. _TARGET?= ${MACHINE} _TARGET_ARCH?= ${MACHINE_ARCH} + +.if make(print-dir) +.SILENT: +.endif # # Make sure we have an up-to-date make(1). Only world and buildworld # should do this as those are the initial targets used for upgrades. # The user can define ALWAYS_CHECK_MAKE to have this check performed # for all targets. # .if defined(ALWAYS_CHECK_MAKE) || !defined(.PARSEDIR) ${TGTS}: upgrade_checks .else buildworld: upgrade_checks .endif # # Handle the user-driven targets, using the source relative mk files. # tinderbox toolchains kernel-toolchains: .MAKE ${TGTS}: .PHONY .MAKE ${_+_}@cd ${.CURDIR}; ${_MAKE} ${.TARGET} # The historic default "all" target creates files which may cause stale # or (in the cross build case) unlinkable results. Fail with an error # when no target is given. The users can explicitly specify "all" # if they want the historic behavior. .MAIN: _guard _guard: .PHONY @echo @echo "Explicit target required. Likely \"${SUBDIR_OVERRIDE:Dall:Ubuildworld}\" is wanted. See build(7)." @echo @false STARTTIME!= LC_ALL=C date CHECK_TIME!= find ${.CURDIR}/sys/sys/param.h -mtime -0s ; echo .if !empty(CHECK_TIME) .error check your date/time: ${STARTTIME} .endif .if defined(HISTORICAL_MAKE_WORLD) || defined(DESTDIR) # # world # # Attempt to rebuild and reinstall everything. This target is not to be # used for upgrading an existing FreeBSD system, because the kernel is # not included. One can argue that this target doesn't build everything # then. # world: upgrade_checks .PHONY @echo "--------------------------------------------------------------" @echo ">>> make world started on ${STARTTIME}" @echo "--------------------------------------------------------------" .if target(pre-world) @echo @echo "--------------------------------------------------------------" @echo ">>> Making 'pre-world' target" @echo "--------------------------------------------------------------" ${_+_}@cd ${.CURDIR}; ${_MAKE} pre-world .endif ${_+_}@cd ${.CURDIR}; ${_MAKE} buildworld ${_+_}@cd ${.CURDIR}; ${_MAKE} installworld MK_META_MODE=no .if target(post-world) @echo @echo "--------------------------------------------------------------" @echo ">>> Making 'post-world' target" @echo "--------------------------------------------------------------" ${_+_}@cd ${.CURDIR}; ${_MAKE} post-world .endif @echo @echo "--------------------------------------------------------------" @echo ">>> make world completed on `LC_ALL=C date`" @echo " (started ${STARTTIME})" @echo "--------------------------------------------------------------" .else world: .PHONY @echo "WARNING: make world will overwrite your existing FreeBSD" @echo "installation without also building and installing a new" @echo "kernel. This can be dangerous. Please read the handbook," @echo "'Rebuilding world', for how to upgrade your system." @echo "Define DESTDIR to where you want to install FreeBSD," @echo "including /, to override this warning and proceed as usual." @echo "" @echo "Bailing out now..." @false .endif # # kernel # # Short hand for `make buildkernel installkernel' # kernel: buildkernel installkernel .PHONY # # Perform a few tests to determine if the installed tools are adequate # for building the world. # upgrade_checks: .PHONY .if defined(NEED_MAKE_UPGRADE) @${_+_}(cd ${.CURDIR} && ${MAKE} ${WANT_MAKE:S,^f,,}) .endif # # Upgrade make(1) to the current version using the installed # headers, libraries and tools. Also, allow the location of # the system bsdmake-like utility to be overridden. # MMAKEENV= MAKEOBJDIRPREFIX=${MYMAKE:H} \ DESTDIR= \ INSTALL="sh ${.CURDIR}/tools/install.sh" MMAKE= ${MMAKEENV} ${MAKE} \ MAN= -DNO_SHARED \ -DNO_CPU_CFLAGS -DNO_WERROR \ -DNO_SUBDIR \ DESTDIR= PROGNAME=${MYMAKE:T} bmake: .PHONY @echo @echo "--------------------------------------------------------------" @echo ">>> Building an up-to-date ${.TARGET}(1)" @echo "--------------------------------------------------------------" ${_+_}@cd ${.CURDIR}/usr.bin/${.TARGET}; \ ${MMAKE} obj; \ ${MMAKE} depend; \ ${MMAKE} all; \ ${MMAKE} install DESTDIR=${MYMAKE:H} BINDIR= regress: .PHONY @echo "'make regress' has been renamed 'make check'" | /usr/bin/fmt @false tinderbox toolchains kernel-toolchains kernels worlds: upgrade_checks tinderbox: .PHONY @cd ${.CURDIR}; ${SUB_MAKE} DOING_TINDERBOX=YES universe toolchains: .PHONY @cd ${.CURDIR}; ${SUB_MAKE} UNIVERSE_TARGET=toolchain universe kernel-toolchains: .PHONY @cd ${.CURDIR}; ${SUB_MAKE} UNIVERSE_TARGET=kernel-toolchain universe kernels: .PHONY @cd ${.CURDIR}; ${SUB_MAKE} UNIVERSE_TARGET=buildkernel universe worlds: .PHONY @cd ${.CURDIR}; ${SUB_MAKE} UNIVERSE_TARGET=buildworld universe # # universe # # Attempt to rebuild *everything* for all supported architectures, # with a reasonable chance of success, regardless of how old your # existing system is. # .if make(universe) || make(universe_kernels) || make(tinderbox) || make(targets) TARGETS?=amd64 arm arm64 i386 mips pc98 powerpc sparc64 _UNIVERSE_TARGETS= ${TARGETS} TARGET_ARCHES_arm?= arm armeb armv6 TARGET_ARCHES_arm64?= aarch64 TARGET_ARCHES_mips?= mipsel mips mips64el mips64 mipsn32 TARGET_ARCHES_powerpc?= powerpc powerpc64 TARGET_ARCHES_pc98?= i386 .for target in ${TARGETS} TARGET_ARCHES_${target}?= ${target} .endfor # XXX Remove arm64 from universe if the required binutils package is missing. # It does not build with the in-tree linker. .if !exists(/usr/local/aarch64-freebsd/bin/ld) && ${TARGETS:Marm64} _UNIVERSE_TARGETS:= ${_UNIVERSE_TARGETS:Narm64} universe: universe_arm64_skip .PHONY universe_epilogue: universe_arm64_skip .PHONY universe_arm64_skip: universe_prologue .PHONY @echo ">> arm64 skipped - install aarch64-binutils port or package to build" .endif .if defined(UNIVERSE_TARGET) MAKE_JUST_WORLDS= YES .else UNIVERSE_TARGET?= buildworld .endif KERNSRCDIR?= ${.CURDIR}/sys targets: .PHONY @echo "Supported TARGET/TARGET_ARCH pairs for world and kernel targets" .for target in ${TARGETS} .for target_arch in ${TARGET_ARCHES_${target}} @echo " ${target}/${target_arch}" .endfor .endfor .if defined(DOING_TINDERBOX) FAILFILE=${.CURDIR}/_.tinderbox.failed MAKEFAIL=tee -a ${FAILFILE} .else MAKEFAIL=cat .endif universe_prologue: upgrade_checks universe: universe_prologue universe_prologue: .PHONY @echo "--------------------------------------------------------------" @echo ">>> make universe started on ${STARTTIME}" @echo "--------------------------------------------------------------" .if defined(DOING_TINDERBOX) @rm -f ${FAILFILE} .endif .for target in ${_UNIVERSE_TARGETS} universe: universe_${target} universe_epilogue: universe_${target} universe_${target}: universe_${target}_prologue .PHONY universe_${target}_prologue: universe_prologue .PHONY @echo ">> ${target} started on `LC_ALL=C date`" universe_${target}_worlds: .PHONY .if !defined(MAKE_JUST_KERNELS) universe_${target}_done: universe_${target}_worlds .PHONY .for target_arch in ${TARGET_ARCHES_${target}} universe_${target}_worlds: universe_${target}_${target_arch} .PHONY universe_${target}_${target_arch}: universe_${target}_prologue .MAKE .PHONY @echo ">> ${target}.${target_arch} ${UNIVERSE_TARGET} started on `LC_ALL=C date`" @(cd ${.CURDIR} && env __MAKE_CONF=/dev/null \ ${SUB_MAKE} ${JFLAG} ${UNIVERSE_TARGET} \ TARGET=${target} \ TARGET_ARCH=${target_arch} \ > _.${target}.${target_arch}.${UNIVERSE_TARGET} 2>&1 || \ (echo "${target}.${target_arch} ${UNIVERSE_TARGET} failed," \ "check _.${target}.${target_arch}.${UNIVERSE_TARGET} for details" | \ ${MAKEFAIL})) @echo ">> ${target}.${target_arch} ${UNIVERSE_TARGET} completed on `LC_ALL=C date`" .endfor .endif # !MAKE_JUST_KERNELS .if !defined(MAKE_JUST_WORLDS) universe_${target}_done: universe_${target}_kernels .PHONY universe_${target}_kernels: universe_${target}_worlds .PHONY universe_${target}_kernels: universe_${target}_prologue .MAKE .PHONY .if exists(${KERNSRCDIR}/${target}/conf/NOTES) @(cd ${KERNSRCDIR}/${target}/conf && env __MAKE_CONF=/dev/null \ ${SUB_MAKE} LINT > ${.CURDIR}/_.${target}.makeLINT 2>&1 || \ (echo "${target} 'make LINT' failed," \ "check _.${target}.makeLINT for details"| ${MAKEFAIL})) .endif @cd ${.CURDIR}; ${SUB_MAKE} ${.MAKEFLAGS} TARGET=${target} \ universe_kernels .endif # !MAKE_JUST_WORLDS # Tell the user the worlds and kernels have completed universe_${target}: universe_${target}_done universe_${target}_done: @echo ">> ${target} completed on `LC_ALL=C date`" .endfor universe_kernels: universe_kernconfs .PHONY .if !defined(TARGET) TARGET!= uname -m .endif .if defined(MAKE_ALL_KERNELS) _THINNER=cat .else _THINNER=xargs grep -L "^.NO_UNIVERSE" || true .endif KERNCONFS!= cd ${KERNSRCDIR}/${TARGET}/conf && \ find [[:upper:][:digit:]]*[[:upper:][:digit:]] \ -type f -maxdepth 0 \ ! -name DEFAULTS ! -name NOTES | \ ${_THINNER} universe_kernconfs: .PHONY .for kernel in ${KERNCONFS} TARGET_ARCH_${kernel}!= cd ${KERNSRCDIR}/${TARGET}/conf && \ config -m ${KERNSRCDIR}/${TARGET}/conf/${kernel} 2> /dev/null | \ grep -v WARNING: | cut -f 2 .if empty(TARGET_ARCH_${kernel}) .error "Target architecture for ${TARGET}/conf/${kernel} unknown. config(8) likely too old." .endif universe_kernconfs: universe_kernconf_${TARGET}_${kernel} universe_kernconf_${TARGET}_${kernel}: .MAKE @(cd ${.CURDIR} && env __MAKE_CONF=/dev/null \ ${SUB_MAKE} ${JFLAG} buildkernel \ TARGET=${TARGET} \ TARGET_ARCH=${TARGET_ARCH_${kernel}} \ KERNCONF=${kernel} \ > _.${TARGET}.${kernel} 2>&1 || \ (echo "${TARGET} ${kernel} kernel failed," \ "check _.${TARGET}.${kernel} for details"| ${MAKEFAIL})) .endfor universe: universe_epilogue universe_epilogue: .PHONY @echo "--------------------------------------------------------------" @echo ">>> make universe completed on `LC_ALL=C date`" @echo " (started ${STARTTIME})" @echo "--------------------------------------------------------------" .if defined(DOING_TINDERBOX) @if [ -e ${FAILFILE} ] ; then \ echo "Tinderbox failed:" ;\ cat ${FAILFILE} ;\ exit 1 ;\ fi .endif .endif buildLINT: .PHONY ${MAKE} -C ${.CURDIR}/sys/${_TARGET}/conf LINT .if defined(.PARSEDIR) # This makefile does not run in meta mode .MAKE.MODE= normal # Normally the things we run from here don't either. # Using -DWITH_META_MODE # we can buildworld with meta files created which are useful # for debugging, but without any of the rest of a meta mode build. MK_DIRDEPS_BUILD= no MK_STAGING= no # tell meta.autodep.mk to not even think about updating anything. UPDATE_DEPENDFILE= NO .if !make(showconfig) .export MK_DIRDEPS_BUILD MK_STAGING UPDATE_DEPENDFILE .endif .if make(universe) # we do not want a failure of one branch abort all. MAKE_JOB_ERROR_TOKEN= no .export MAKE_JOB_ERROR_TOKEN .endif .endif # bmake .endif # DIRDEPS_BUILD Index: projects/netbsd-tests-update-12/bin/df/df.c =================================================================== --- projects/netbsd-tests-update-12/bin/df/df.c (revision 305171) +++ projects/netbsd-tests-update-12/bin/df/df.c (revision 305172) @@ -1,662 +1,665 @@ /*- * Copyright (c) 1980, 1990, 1993, 1994 * The Regents of the University of California. All rights reserved. * (c) UNIX System Laboratories, Inc. * All or some portions of this file are derived from material licensed * to the University of California by American Telephone and Telegraph * Co. or Unix System Laboratories, Inc. and are reproduced herein with * the permission of UNIX System Laboratories, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #if 0 #ifndef lint static const char copyright[] = "@(#) Copyright (c) 1980, 1990, 1993, 1994\n\ The Regents of the University of California. All rights reserved.\n"; #endif /* not lint */ #ifndef lint static char sccsid[] = "@(#)df.c 8.9 (Berkeley) 5/8/95"; #endif /* not lint */ #endif #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "extern.h" #define UNITS_SI 1 #define UNITS_2 2 /* Maximum widths of various fields. */ struct maxwidths { int mntfrom; int fstype; int total; int used; int avail; int iused; int ifree; }; static void addstat(struct statfs *, struct statfs *); static char *getmntpt(const char *); static int int64width(int64_t); static char *makenetvfslist(void); static void prthuman(const struct statfs *, int64_t); static void prthumanval(const char *, int64_t); static intmax_t fsbtoblk(int64_t, uint64_t, u_long); static void prtstat(struct statfs *, struct maxwidths *); static size_t regetmntinfo(struct statfs **, long, const char **); static void update_maxwidths(struct maxwidths *, const struct statfs *); static void usage(void); static __inline int imax(int a, int b) { return (a > b ? a : b); } static int aflag = 0, cflag, hflag, iflag, kflag, lflag = 0, nflag, Tflag; static int thousands; static struct ufs_args mdev; int main(int argc, char *argv[]) { struct stat stbuf; struct statfs statfsbuf, totalbuf; struct maxwidths maxwidths; struct statfs *mntbuf; const char *fstype; char *mntpath, *mntpt; const char **vfslist; int i, mntsize; int ch, rv; fstype = "ufs"; (void)setlocale(LC_ALL, ""); memset(&maxwidths, 0, sizeof(maxwidths)); memset(&totalbuf, 0, sizeof(totalbuf)); totalbuf.f_bsize = DEV_BSIZE; strlcpy(totalbuf.f_mntfromname, "total", MNAMELEN); vfslist = NULL; argc = xo_parse_args(argc, argv); if (argc < 0) exit(1); while ((ch = getopt(argc, argv, "abcgHhiklmnPt:T,")) != -1) switch (ch) { case 'a': aflag = 1; break; case 'b': /* FALLTHROUGH */ case 'P': /* * POSIX specifically discusses the behavior of * both -k and -P. It states that the blocksize should * be set to 1024. Thus, if this occurs, simply break * rather than clobbering the old blocksize. */ if (kflag) break; setenv("BLOCKSIZE", "512", 1); hflag = 0; break; case 'c': cflag = 1; break; case 'g': setenv("BLOCKSIZE", "1g", 1); hflag = 0; break; case 'H': hflag = UNITS_SI; break; case 'h': hflag = UNITS_2; break; case 'i': iflag = 1; break; case 'k': kflag++; setenv("BLOCKSIZE", "1024", 1); hflag = 0; break; case 'l': + /* Ignore duplicate -l */ + if (lflag) + break; if (vfslist != NULL) xo_errx(1, "-l and -t are mutually exclusive."); vfslist = makevfslist(makenetvfslist()); lflag = 1; break; case 'm': setenv("BLOCKSIZE", "1m", 1); hflag = 0; break; case 'n': nflag = 1; break; case 't': if (lflag) xo_errx(1, "-l and -t are mutually exclusive."); if (vfslist != NULL) xo_errx(1, "only one -t option may be specified"); fstype = optarg; vfslist = makevfslist(optarg); break; case 'T': Tflag = 1; break; case ',': thousands = 1; break; case '?': default: usage(); } argc -= optind; argv += optind; rv = 0; if (!*argv) { /* everything (modulo -t) */ mntsize = getmntinfo(&mntbuf, MNT_NOWAIT); mntsize = regetmntinfo(&mntbuf, mntsize, vfslist); } else { /* just the filesystems specified on the command line */ mntbuf = malloc(argc * sizeof(*mntbuf)); if (mntbuf == NULL) xo_err(1, "malloc()"); mntsize = 0; /* continued in for loop below */ } xo_open_container("storage-system-information"); xo_open_list("filesystem"); /* iterate through specified filesystems */ for (; *argv; argv++) { if (stat(*argv, &stbuf) < 0) { if ((mntpt = getmntpt(*argv)) == NULL) { xo_warn("%s", *argv); rv = 1; continue; } } else if (S_ISCHR(stbuf.st_mode)) { if ((mntpt = getmntpt(*argv)) == NULL) { mdev.fspec = *argv; mntpath = strdup("/tmp/df.XXXXXX"); if (mntpath == NULL) { xo_warn("strdup failed"); rv = 1; continue; } mntpt = mkdtemp(mntpath); if (mntpt == NULL) { xo_warn("mkdtemp(\"%s\") failed", mntpath); rv = 1; free(mntpath); continue; } if (mount(fstype, mntpt, MNT_RDONLY, &mdev) != 0) { xo_warn("%s", *argv); rv = 1; (void)rmdir(mntpt); free(mntpath); continue; } else if (statfs(mntpt, &statfsbuf) == 0) { statfsbuf.f_mntonname[0] = '\0'; prtstat(&statfsbuf, &maxwidths); if (cflag) addstat(&totalbuf, &statfsbuf); } else { xo_warn("%s", *argv); rv = 1; } (void)unmount(mntpt, 0); (void)rmdir(mntpt); free(mntpath); continue; } } else mntpt = *argv; /* * Statfs does not take a `wait' flag, so we cannot * implement nflag here. */ if (statfs(mntpt, &statfsbuf) < 0) { xo_warn("%s", mntpt); rv = 1; continue; } /* * Check to make sure the arguments we've been given are * satisfied. Return an error if we have been asked to * list a mount point that does not match the other args * we've been given (-l, -t, etc.). */ if (checkvfsname(statfsbuf.f_fstypename, vfslist)) { rv = 1; continue; } /* the user asked for it, so ignore the ignore flag */ statfsbuf.f_flags &= ~MNT_IGNORE; /* add to list */ mntbuf[mntsize++] = statfsbuf; } memset(&maxwidths, 0, sizeof(maxwidths)); for (i = 0; i < mntsize; i++) { if (aflag || (mntbuf[i].f_flags & MNT_IGNORE) == 0) { update_maxwidths(&maxwidths, &mntbuf[i]); if (cflag) addstat(&totalbuf, &mntbuf[i]); } } for (i = 0; i < mntsize; i++) if (aflag || (mntbuf[i].f_flags & MNT_IGNORE) == 0) prtstat(&mntbuf[i], &maxwidths); xo_close_list("filesystem"); if (cflag) prtstat(&totalbuf, &maxwidths); xo_close_container("storage-system-information"); xo_finish(); exit(rv); } static char * getmntpt(const char *name) { size_t mntsize, i; struct statfs *mntbuf; mntsize = getmntinfo(&mntbuf, MNT_NOWAIT); for (i = 0; i < mntsize; i++) { if (!strcmp(mntbuf[i].f_mntfromname, name)) return (mntbuf[i].f_mntonname); } return (NULL); } /* * Make a pass over the file system info in ``mntbuf'' filtering out * file system types not in vfslist and possibly re-stating to get * current (not cached) info. Returns the new count of valid statfs bufs. */ static size_t regetmntinfo(struct statfs **mntbufp, long mntsize, const char **vfslist) { int error, i, j; struct statfs *mntbuf; if (vfslist == NULL) return (nflag ? mntsize : getmntinfo(mntbufp, MNT_WAIT)); mntbuf = *mntbufp; for (j = 0, i = 0; i < mntsize; i++) { if (checkvfsname(mntbuf[i].f_fstypename, vfslist)) continue; /* * XXX statfs(2) can fail for various reasons. It may be * possible that the user does not have access to the * pathname, if this happens, we will fall back on * "stale" filesystem statistics. */ error = statfs(mntbuf[i].f_mntonname, &mntbuf[j]); if (nflag || error < 0) if (i != j) { if (error < 0) xo_warnx("%s stats possibly stale", mntbuf[i].f_mntonname); mntbuf[j] = mntbuf[i]; } j++; } return (j); } static void prthuman(const struct statfs *sfsp, int64_t used) { prthumanval(" {:blocks/%6s}", sfsp->f_blocks * sfsp->f_bsize); prthumanval(" {:used/%6s}", used * sfsp->f_bsize); prthumanval(" {:available/%6s}", sfsp->f_bavail * sfsp->f_bsize); } static void prthumanval(const char *fmt, int64_t bytes) { char buf[6]; int flags; flags = HN_B | HN_NOSPACE | HN_DECIMAL; if (hflag == UNITS_SI) flags |= HN_DIVISOR_1000; humanize_number(buf, sizeof(buf) - (bytes < 0 ? 0 : 1), bytes, "", HN_AUTOSCALE, flags); xo_attr("value", "%lld", (long long) bytes); xo_emit(fmt, buf); } /* * Print an inode count in "human-readable" format. */ static void prthumanvalinode(const char *fmt, int64_t bytes) { char buf[6]; int flags; flags = HN_NOSPACE | HN_DECIMAL | HN_DIVISOR_1000; humanize_number(buf, sizeof(buf) - (bytes < 0 ? 0 : 1), bytes, "", HN_AUTOSCALE, flags); xo_attr("value", "%lld", (long long) bytes); xo_emit(fmt, buf); } /* * Convert statfs returned file system size into BLOCKSIZE units. */ static intmax_t fsbtoblk(int64_t num, uint64_t fsbs, u_long bs) { return (num * (intmax_t) fsbs / (int64_t) bs); } /* * Print out status about a file system. */ static void prtstat(struct statfs *sfsp, struct maxwidths *mwp) { static long blocksize; static int headerlen, timesthrough = 0; static const char *header; int64_t used, availblks, inodes; const char *format; if (++timesthrough == 1) { mwp->mntfrom = imax(mwp->mntfrom, (int)strlen("Filesystem")); mwp->fstype = imax(mwp->fstype, (int)strlen("Type")); if (thousands) { /* make space for commas */ mwp->total += (mwp->total - 1) / 3; mwp->used += (mwp->used - 1) / 3; mwp->avail += (mwp->avail - 1) / 3; mwp->iused += (mwp->iused - 1) / 3; mwp->ifree += (mwp->ifree - 1) / 3; } if (hflag) { header = " Size"; mwp->total = mwp->used = mwp->avail = (int)strlen(header); } else { header = getbsize(&headerlen, &blocksize); mwp->total = imax(mwp->total, headerlen); } mwp->used = imax(mwp->used, (int)strlen("Used")); mwp->avail = imax(mwp->avail, (int)strlen("Avail")); xo_emit("{T:/%-*s}", mwp->mntfrom, "Filesystem"); if (Tflag) xo_emit(" {T:/%-*s}", mwp->fstype, "Type"); xo_emit(" {T:/%*s} {T:/%*s} {T:/%*s} Capacity", mwp->total, header, mwp->used, "Used", mwp->avail, "Avail"); if (iflag) { mwp->iused = imax(hflag ? 0 : mwp->iused, (int)strlen(" iused")); mwp->ifree = imax(hflag ? 0 : mwp->ifree, (int)strlen("ifree")); xo_emit(" {T:/%*s} {T:/%*s} {T:\%iused}", mwp->iused - 2, "iused", mwp->ifree, "ifree"); } xo_emit(" {T:Mounted on}\n"); } xo_open_instance("filesystem"); /* Check for 0 block size. Can this happen? */ if (sfsp->f_bsize == 0) { xo_warnx ("File system %s does not have a block size, assuming 512.", sfsp->f_mntonname); sfsp->f_bsize = 512; } xo_emit("{tk:name/%-*s}", mwp->mntfrom, sfsp->f_mntfromname); if (Tflag) xo_emit(" {:type/%-*s}", mwp->fstype, sfsp->f_fstypename); used = sfsp->f_blocks - sfsp->f_bfree; availblks = sfsp->f_bavail + used; if (hflag) { prthuman(sfsp, used); } else { if (thousands) format = " {t:total-blocks/%*j'd} {t:used-blocks/%*j'd} " "{t:available-blocks/%*j'd}"; else format = " {t:total-blocks/%*jd} {t:used-blocks/%*jd} " "{t:available-blocks/%*jd}"; xo_emit(format, mwp->total, fsbtoblk(sfsp->f_blocks, sfsp->f_bsize, blocksize), mwp->used, fsbtoblk(used, sfsp->f_bsize, blocksize), mwp->avail, fsbtoblk(sfsp->f_bavail, sfsp->f_bsize, blocksize)); } xo_emit(" {:used-percent/%5.0f}{U:%%}", availblks == 0 ? 100.0 : (double)used / (double)availblks * 100.0); if (iflag) { inodes = sfsp->f_files; used = inodes - sfsp->f_ffree; if (hflag) { xo_emit(" "); prthumanvalinode(" {:inodes-used/%5s}", used); prthumanvalinode(" {:inodes-free/%5s}", sfsp->f_ffree); } else { if (thousands) format = " {:inodes-used/%*j'd} {:inodes-free/%*j'd}"; else format = " {:inodes-used/%*jd} {:inodes-free/%*jd}"; xo_emit(format, mwp->iused, (intmax_t)used, mwp->ifree, (intmax_t)sfsp->f_ffree); } xo_emit(" {:inodes-used-percent/%4.0f}{U:%%} ", inodes == 0 ? 100.0 : (double)used / (double)inodes * 100.0); } else xo_emit(" "); if (strncmp(sfsp->f_mntfromname, "total", MNAMELEN) != 0) xo_emit(" {:mounted-on}", sfsp->f_mntonname); xo_emit("\n"); xo_close_instance("filesystem"); } static void addstat(struct statfs *totalfsp, struct statfs *statfsp) { uint64_t bsize; bsize = statfsp->f_bsize / totalfsp->f_bsize; totalfsp->f_blocks += statfsp->f_blocks * bsize; totalfsp->f_bfree += statfsp->f_bfree * bsize; totalfsp->f_bavail += statfsp->f_bavail * bsize; totalfsp->f_files += statfsp->f_files; totalfsp->f_ffree += statfsp->f_ffree; } /* * Update the maximum field-width information in `mwp' based on * the file system specified by `sfsp'. */ static void update_maxwidths(struct maxwidths *mwp, const struct statfs *sfsp) { static long blocksize = 0; int dummy; if (blocksize == 0) getbsize(&dummy, &blocksize); mwp->mntfrom = imax(mwp->mntfrom, (int)strlen(sfsp->f_mntfromname)); mwp->fstype = imax(mwp->fstype, (int)strlen(sfsp->f_fstypename)); mwp->total = imax(mwp->total, int64width( fsbtoblk((int64_t)sfsp->f_blocks, sfsp->f_bsize, blocksize))); mwp->used = imax(mwp->used, int64width(fsbtoblk((int64_t)sfsp->f_blocks - (int64_t)sfsp->f_bfree, sfsp->f_bsize, blocksize))); mwp->avail = imax(mwp->avail, int64width(fsbtoblk(sfsp->f_bavail, sfsp->f_bsize, blocksize))); mwp->iused = imax(mwp->iused, int64width((int64_t)sfsp->f_files - sfsp->f_ffree)); mwp->ifree = imax(mwp->ifree, int64width(sfsp->f_ffree)); } /* Return the width in characters of the specified value. */ static int int64width(int64_t val) { int len; len = 0; /* Negative or zero values require one extra digit. */ if (val <= 0) { val = -val; len++; } while (val > 0) { len++; val /= 10; } return (len); } static void usage(void) { xo_error( "usage: df [-b | -g | -H | -h | -k | -m | -P] [-acilnT] [-t type] [-,]\n" " [file | filesystem ...]\n"); exit(EX_USAGE); } static char * makenetvfslist(void) { char *str, *strptr, **listptr; struct xvfsconf *xvfsp, *keep_xvfsp; size_t buflen; int cnt, i, maxvfsconf; if (sysctlbyname("vfs.conflist", NULL, &buflen, NULL, 0) < 0) { xo_warn("sysctl(vfs.conflist)"); return (NULL); } xvfsp = malloc(buflen); if (xvfsp == NULL) { xo_warnx("malloc failed"); return (NULL); } keep_xvfsp = xvfsp; if (sysctlbyname("vfs.conflist", xvfsp, &buflen, NULL, 0) < 0) { xo_warn("sysctl(vfs.conflist)"); free(keep_xvfsp); return (NULL); } maxvfsconf = buflen / sizeof(struct xvfsconf); if ((listptr = malloc(sizeof(char*) * maxvfsconf)) == NULL) { xo_warnx("malloc failed"); free(keep_xvfsp); return (NULL); } for (cnt = 0, i = 0; i < maxvfsconf; i++) { if (xvfsp->vfc_flags & VFCF_NETWORK) { listptr[cnt++] = strdup(xvfsp->vfc_name); if (listptr[cnt-1] == NULL) { xo_warnx("malloc failed"); free(listptr); free(keep_xvfsp); return (NULL); } } xvfsp++; } if (cnt == 0 || (str = malloc(sizeof(char) * (32 * cnt + cnt + 2))) == NULL) { if (cnt > 0) xo_warnx("malloc failed"); free(listptr); free(keep_xvfsp); return (NULL); } *str = 'n'; *(str + 1) = 'o'; for (i = 0, strptr = str + 2; i < cnt; i++, strptr++) { strlcpy(strptr, listptr[i], 32); strptr += strlen(listptr[i]); *strptr = ','; free(listptr[i]); } *(--strptr) = '\0'; free(keep_xvfsp); free(listptr); return (str); } Index: projects/netbsd-tests-update-12/cddl/contrib/opensolaris/common/ctf/ctf_types.c =================================================================== --- projects/netbsd-tests-update-12/cddl/contrib/opensolaris/common/ctf/ctf_types.c (revision 305171) +++ projects/netbsd-tests-update-12/cddl/contrib/opensolaris/common/ctf/ctf_types.c (revision 305172) @@ -1,870 +1,887 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License, Version 1.0 only * (the "License"). You may not use this file except in compliance * with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright 2006 Sun Microsystems, Inc. All rights reserved. * Use is subject to license terms. */ #include ssize_t ctf_get_ctt_size(const ctf_file_t *fp, const ctf_type_t *tp, ssize_t *sizep, ssize_t *incrementp) { ssize_t size, increment; if (fp->ctf_version > CTF_VERSION_1 && tp->ctt_size == CTF_LSIZE_SENT) { size = CTF_TYPE_LSIZE(tp); increment = sizeof (ctf_type_t); } else { size = tp->ctt_size; increment = sizeof (ctf_stype_t); } if (sizep) *sizep = size; if (incrementp) *incrementp = increment; return (size); } /* * Iterate over the members of a STRUCT or UNION. We pass the name, member * type, and offset of each member to the specified callback function. */ int ctf_member_iter(ctf_file_t *fp, ctf_id_t type, ctf_member_f *func, void *arg) { ctf_file_t *ofp = fp; const ctf_type_t *tp; ssize_t size, increment; uint_t kind, n; int rc; if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (CTF_ERR); /* errno is set for us */ if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ (void) ctf_get_ctt_size(fp, tp, &size, &increment); kind = LCTF_INFO_KIND(fp, tp->ctt_info); if (kind != CTF_K_STRUCT && kind != CTF_K_UNION) return (ctf_set_errno(ofp, ECTF_NOTSOU)); if (fp->ctf_version == CTF_VERSION_1 || size < CTF_LSTRUCT_THRESH) { const ctf_member_t *mp = (const ctf_member_t *) ((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, mp++) { const char *name = ctf_strptr(fp, mp->ctm_name); if ((rc = func(name, mp->ctm_type, mp->ctm_offset, arg)) != 0) return (rc); } } else { const ctf_lmember_t *lmp = (const ctf_lmember_t *) ((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, lmp++) { const char *name = ctf_strptr(fp, lmp->ctlm_name); if ((rc = func(name, lmp->ctlm_type, (ulong_t)CTF_LMEM_OFFSET(lmp), arg)) != 0) return (rc); } } return (0); } /* * Iterate over the members of an ENUM. We pass the string name and associated * integer value of each enum element to the specified callback function. */ int ctf_enum_iter(ctf_file_t *fp, ctf_id_t type, ctf_enum_f *func, void *arg) { ctf_file_t *ofp = fp; const ctf_type_t *tp; const ctf_enum_t *ep; ssize_t increment; uint_t n; int rc; if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (CTF_ERR); /* errno is set for us */ if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ if (LCTF_INFO_KIND(fp, tp->ctt_info) != CTF_K_ENUM) return (ctf_set_errno(ofp, ECTF_NOTENUM)); (void) ctf_get_ctt_size(fp, tp, NULL, &increment); ep = (const ctf_enum_t *)((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, ep++) { const char *name = ctf_strptr(fp, ep->cte_name); if ((rc = func(name, ep->cte_value, arg)) != 0) return (rc); } return (0); } /* * Iterate over every root (user-visible) type in the given CTF container. * We pass the type ID of each type to the specified callback function. */ int ctf_type_iter(ctf_file_t *fp, ctf_type_f *func, void *arg) { ctf_id_t id, max = fp->ctf_typemax; int rc, child = (fp->ctf_flags & LCTF_CHILD); for (id = 1; id <= max; id++) { const ctf_type_t *tp = LCTF_INDEX_TO_TYPEPTR(fp, id); if (CTF_INFO_ISROOT(tp->ctt_info) && (rc = func(CTF_INDEX_TO_TYPE(id, child), arg)) != 0) return (rc); } return (0); } /* * Follow a given type through the graph for TYPEDEF, VOLATILE, CONST, and * RESTRICT nodes until we reach a "base" type node. This is useful when * we want to follow a type ID to a node that has members or a size. To guard * against infinite loops, we implement simplified cycle detection and check * each link against itself, the previous node, and the topmost node. */ ctf_id_t ctf_type_resolve(ctf_file_t *fp, ctf_id_t type) { ctf_id_t prev = type, otype = type; ctf_file_t *ofp = fp; const ctf_type_t *tp; while ((tp = ctf_lookup_by_id(&fp, type)) != NULL) { switch (LCTF_INFO_KIND(fp, tp->ctt_info)) { case CTF_K_TYPEDEF: case CTF_K_VOLATILE: case CTF_K_CONST: case CTF_K_RESTRICT: if (tp->ctt_type == type || tp->ctt_type == otype || tp->ctt_type == prev) { ctf_dprintf("type %ld cycle detected\n", otype); return (ctf_set_errno(ofp, ECTF_CORRUPT)); } prev = type; type = tp->ctt_type; break; default: return (type); } } return (CTF_ERR); /* errno is set for us */ } /* * Lookup the given type ID and print a string name for it into buf. Return * the actual number of bytes (not including \0) needed to format the name. */ static ssize_t ctf_type_qlname(ctf_file_t *fp, ctf_id_t type, char *buf, size_t len, const char *qname) { ctf_decl_t cd; ctf_decl_node_t *cdp; ctf_decl_prec_t prec, lp, rp; int ptr, arr; uint_t k; if (fp == NULL && type == CTF_ERR) return (-1); /* simplify caller code by permitting CTF_ERR */ ctf_decl_init(&cd, buf, len); ctf_decl_push(&cd, fp, type); if (cd.cd_err != 0) { ctf_decl_fini(&cd); return (ctf_set_errno(fp, cd.cd_err)); } /* * If the type graph's order conflicts with lexical precedence order * for pointers or arrays, then we need to surround the declarations at * the corresponding lexical precedence with parentheses. This can * result in either a parenthesized pointer (*) as in int (*)() or * int (*)[], or in a parenthesized pointer and array as in int (*[])(). */ ptr = cd.cd_order[CTF_PREC_POINTER] > CTF_PREC_POINTER; arr = cd.cd_order[CTF_PREC_ARRAY] > CTF_PREC_ARRAY; rp = arr ? CTF_PREC_ARRAY : ptr ? CTF_PREC_POINTER : -1; lp = ptr ? CTF_PREC_POINTER : arr ? CTF_PREC_ARRAY : -1; k = CTF_K_POINTER; /* avoid leading whitespace (see below) */ for (prec = CTF_PREC_BASE; prec < CTF_PREC_MAX; prec++) { for (cdp = ctf_list_next(&cd.cd_nodes[prec]); cdp != NULL; cdp = ctf_list_next(cdp)) { ctf_file_t *rfp = fp; const ctf_type_t *tp = ctf_lookup_by_id(&rfp, cdp->cd_type); const char *name = ctf_strptr(rfp, tp->ctt_name); if (k != CTF_K_POINTER && k != CTF_K_ARRAY) ctf_decl_sprintf(&cd, " "); if (lp == prec) { ctf_decl_sprintf(&cd, "("); lp = -1; } switch (cdp->cd_kind) { case CTF_K_INTEGER: case CTF_K_FLOAT: case CTF_K_TYPEDEF: if (qname != NULL) ctf_decl_sprintf(&cd, "%s`", qname); ctf_decl_sprintf(&cd, "%s", name); break; case CTF_K_POINTER: ctf_decl_sprintf(&cd, "*"); break; case CTF_K_ARRAY: ctf_decl_sprintf(&cd, "[%u]", cdp->cd_n); break; case CTF_K_FUNCTION: ctf_decl_sprintf(&cd, "()"); break; case CTF_K_STRUCT: case CTF_K_FORWARD: ctf_decl_sprintf(&cd, "struct "); if (qname != NULL) ctf_decl_sprintf(&cd, "%s`", qname); ctf_decl_sprintf(&cd, "%s", name); break; case CTF_K_UNION: ctf_decl_sprintf(&cd, "union "); if (qname != NULL) ctf_decl_sprintf(&cd, "%s`", qname); ctf_decl_sprintf(&cd, "%s", name); break; case CTF_K_ENUM: ctf_decl_sprintf(&cd, "enum "); if (qname != NULL) ctf_decl_sprintf(&cd, "%s`", qname); ctf_decl_sprintf(&cd, "%s", name); break; case CTF_K_VOLATILE: ctf_decl_sprintf(&cd, "volatile"); break; case CTF_K_CONST: ctf_decl_sprintf(&cd, "const"); break; case CTF_K_RESTRICT: ctf_decl_sprintf(&cd, "restrict"); break; } k = cdp->cd_kind; } if (rp == prec) ctf_decl_sprintf(&cd, ")"); } if (cd.cd_len >= len) (void) ctf_set_errno(fp, ECTF_NAMELEN); ctf_decl_fini(&cd); return (cd.cd_len); } ssize_t ctf_type_lname(ctf_file_t *fp, ctf_id_t type, char *buf, size_t len) { return (ctf_type_qlname(fp, type, buf, len, NULL)); } /* * Lookup the given type ID and print a string name for it into buf. If buf * is too small, return NULL: the ECTF_NAMELEN error is set on 'fp' for us. */ char * ctf_type_name(ctf_file_t *fp, ctf_id_t type, char *buf, size_t len) { ssize_t rv = ctf_type_qlname(fp, type, buf, len, NULL); return (rv >= 0 && rv < len ? buf : NULL); } char * ctf_type_qname(ctf_file_t *fp, ctf_id_t type, char *buf, size_t len, const char *qname) { ssize_t rv = ctf_type_qlname(fp, type, buf, len, qname); return (rv >= 0 && rv < len ? buf : NULL); } /* * Resolve the type down to a base type node, and then return the size * of the type storage in bytes. */ ssize_t ctf_type_size(ctf_file_t *fp, ctf_id_t type) { const ctf_type_t *tp; ssize_t size; ctf_arinfo_t ar; if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (-1); /* errno is set for us */ if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (-1); /* errno is set for us */ switch (LCTF_INFO_KIND(fp, tp->ctt_info)) { case CTF_K_POINTER: return (fp->ctf_dmodel->ctd_pointer); case CTF_K_FUNCTION: return (0); /* function size is only known by symtab */ case CTF_K_ENUM: return (fp->ctf_dmodel->ctd_int); case CTF_K_ARRAY: /* * Array size is not directly returned by stabs data. Instead, * it defines the element type and requires the user to perform * the multiplication. If ctf_get_ctt_size() returns zero, the * current version of ctfconvert does not compute member sizes * and we compute the size here on its behalf. */ if ((size = ctf_get_ctt_size(fp, tp, NULL, NULL)) > 0) return (size); if (ctf_array_info(fp, type, &ar) == CTF_ERR || (size = ctf_type_size(fp, ar.ctr_contents)) == CTF_ERR) return (-1); /* errno is set for us */ return (size * ar.ctr_nelems); default: return (ctf_get_ctt_size(fp, tp, NULL, NULL)); } } /* * Resolve the type down to a base type node, and then return the alignment * needed for the type storage in bytes. */ ssize_t ctf_type_align(ctf_file_t *fp, ctf_id_t type) { const ctf_type_t *tp; ctf_arinfo_t r; if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (-1); /* errno is set for us */ if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (-1); /* errno is set for us */ switch (LCTF_INFO_KIND(fp, tp->ctt_info)) { case CTF_K_POINTER: case CTF_K_FUNCTION: return (fp->ctf_dmodel->ctd_pointer); case CTF_K_ARRAY: if (ctf_array_info(fp, type, &r) == CTF_ERR) return (-1); /* errno is set for us */ return (ctf_type_align(fp, r.ctr_contents)); case CTF_K_STRUCT: case CTF_K_UNION: { uint_t n = LCTF_INFO_VLEN(fp, tp->ctt_info); ssize_t size, increment; size_t align = 0; const void *vmp; (void) ctf_get_ctt_size(fp, tp, &size, &increment); vmp = (uchar_t *)tp + increment; if (LCTF_INFO_KIND(fp, tp->ctt_info) == CTF_K_STRUCT) n = MIN(n, 1); /* only use first member for structs */ if (fp->ctf_version == CTF_VERSION_1 || size < CTF_LSTRUCT_THRESH) { const ctf_member_t *mp = vmp; for (; n != 0; n--, mp++) { ssize_t am = ctf_type_align(fp, mp->ctm_type); align = MAX(align, am); } } else { const ctf_lmember_t *lmp = vmp; for (; n != 0; n--, lmp++) { ssize_t am = ctf_type_align(fp, lmp->ctlm_type); align = MAX(align, am); } } return (align); } case CTF_K_ENUM: return (fp->ctf_dmodel->ctd_int); default: return (ctf_get_ctt_size(fp, tp, NULL, NULL)); } } /* * Return the kind (CTF_K_* constant) for the specified type ID. */ int ctf_type_kind(ctf_file_t *fp, ctf_id_t type) { const ctf_type_t *tp; if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ return (LCTF_INFO_KIND(fp, tp->ctt_info)); } /* * If the type is one that directly references another type (such as POINTER), * then return the ID of the type to which it refers. */ ctf_id_t ctf_type_reference(ctf_file_t *fp, ctf_id_t type) { ctf_file_t *ofp = fp; const ctf_type_t *tp; if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ switch (LCTF_INFO_KIND(fp, tp->ctt_info)) { case CTF_K_POINTER: case CTF_K_TYPEDEF: case CTF_K_VOLATILE: case CTF_K_CONST: case CTF_K_RESTRICT: return (tp->ctt_type); default: return (ctf_set_errno(ofp, ECTF_NOTREF)); } } /* * Find a pointer to type by looking in fp->ctf_ptrtab. If we can't find a * pointer to the given type, see if we can compute a pointer to the type * resulting from resolving the type down to its base type and use that * instead. This helps with cases where the CTF data includes "struct foo *" * but not "foo_t *" and the user accesses "foo_t *" in the debugger. */ ctf_id_t ctf_type_pointer(ctf_file_t *fp, ctf_id_t type) { ctf_file_t *ofp = fp; ctf_id_t ntype; if (ctf_lookup_by_id(&fp, type) == NULL) return (CTF_ERR); /* errno is set for us */ if ((ntype = fp->ctf_ptrtab[CTF_TYPE_TO_INDEX(type)]) != 0) return (CTF_INDEX_TO_TYPE(ntype, (fp->ctf_flags & LCTF_CHILD))); if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (ctf_set_errno(ofp, ECTF_NOTYPE)); if (ctf_lookup_by_id(&fp, type) == NULL) return (ctf_set_errno(ofp, ECTF_NOTYPE)); if ((ntype = fp->ctf_ptrtab[CTF_TYPE_TO_INDEX(type)]) != 0) return (CTF_INDEX_TO_TYPE(ntype, (fp->ctf_flags & LCTF_CHILD))); return (ctf_set_errno(ofp, ECTF_NOTYPE)); } /* * Return the encoding for the specified INTEGER or FLOAT. */ int ctf_type_encoding(ctf_file_t *fp, ctf_id_t type, ctf_encoding_t *ep) { ctf_file_t *ofp = fp; const ctf_type_t *tp; ssize_t increment; uint_t data; if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ (void) ctf_get_ctt_size(fp, tp, NULL, &increment); switch (LCTF_INFO_KIND(fp, tp->ctt_info)) { case CTF_K_INTEGER: data = *(const uint_t *)((uintptr_t)tp + increment); ep->cte_format = CTF_INT_ENCODING(data); ep->cte_offset = CTF_INT_OFFSET(data); ep->cte_bits = CTF_INT_BITS(data); break; case CTF_K_FLOAT: data = *(const uint_t *)((uintptr_t)tp + increment); ep->cte_format = CTF_FP_ENCODING(data); ep->cte_offset = CTF_FP_OFFSET(data); ep->cte_bits = CTF_FP_BITS(data); break; default: return (ctf_set_errno(ofp, ECTF_NOTINTFP)); } return (0); } int ctf_type_cmp(ctf_file_t *lfp, ctf_id_t ltype, ctf_file_t *rfp, ctf_id_t rtype) { int rval; if (ltype < rtype) rval = -1; else if (ltype > rtype) rval = 1; else rval = 0; if (lfp == rfp) return (rval); if (CTF_TYPE_ISPARENT(ltype) && lfp->ctf_parent != NULL) lfp = lfp->ctf_parent; if (CTF_TYPE_ISPARENT(rtype) && rfp->ctf_parent != NULL) rfp = rfp->ctf_parent; if (lfp < rfp) return (-1); if (lfp > rfp) return (1); return (rval); } /* * Return a boolean value indicating if two types are compatible integers or * floating-pointer values. This function returns true if the two types are * the same, or if they have the same ASCII name and encoding properties. * This function could be extended to test for compatibility for other kinds. */ int ctf_type_compat(ctf_file_t *lfp, ctf_id_t ltype, ctf_file_t *rfp, ctf_id_t rtype) { const ctf_type_t *ltp, *rtp; ctf_encoding_t le, re; ctf_arinfo_t la, ra; uint_t lkind, rkind; if (ctf_type_cmp(lfp, ltype, rfp, rtype) == 0) return (1); ltype = ctf_type_resolve(lfp, ltype); lkind = ctf_type_kind(lfp, ltype); rtype = ctf_type_resolve(rfp, rtype); rkind = ctf_type_kind(rfp, rtype); if (lkind != rkind || (ltp = ctf_lookup_by_id(&lfp, ltype)) == NULL || (rtp = ctf_lookup_by_id(&rfp, rtype)) == NULL || strcmp(ctf_strptr(lfp, ltp->ctt_name), ctf_strptr(rfp, rtp->ctt_name)) != 0) return (0); switch (lkind) { case CTF_K_INTEGER: case CTF_K_FLOAT: return (ctf_type_encoding(lfp, ltype, &le) == 0 && ctf_type_encoding(rfp, rtype, &re) == 0 && bcmp(&le, &re, sizeof (ctf_encoding_t)) == 0); case CTF_K_POINTER: return (ctf_type_compat(lfp, ctf_type_reference(lfp, ltype), rfp, ctf_type_reference(rfp, rtype))); case CTF_K_ARRAY: return (ctf_array_info(lfp, ltype, &la) == 0 && ctf_array_info(rfp, rtype, &ra) == 0 && la.ctr_nelems == ra.ctr_nelems && ctf_type_compat( lfp, la.ctr_contents, rfp, ra.ctr_contents) && ctf_type_compat(lfp, la.ctr_index, rfp, ra.ctr_index)); case CTF_K_STRUCT: case CTF_K_UNION: return (ctf_type_size(lfp, ltype) == ctf_type_size(rfp, rtype)); case CTF_K_ENUM: case CTF_K_FORWARD: return (1); /* no other checks required for these type kinds */ default: return (0); /* should not get here since we did a resolve */ } } -/* - * Return the type and offset for a given member of a STRUCT or UNION. - */ -int -ctf_member_info(ctf_file_t *fp, ctf_id_t type, const char *name, +static int +_ctf_member_info(ctf_file_t *fp, ctf_id_t type, const char *name, ulong_t off, ctf_membinfo_t *mip) { ctf_file_t *ofp = fp; const ctf_type_t *tp; ssize_t size, increment; uint_t kind, n; if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (CTF_ERR); /* errno is set for us */ if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ (void) ctf_get_ctt_size(fp, tp, &size, &increment); kind = LCTF_INFO_KIND(fp, tp->ctt_info); if (kind != CTF_K_STRUCT && kind != CTF_K_UNION) return (ctf_set_errno(ofp, ECTF_NOTSOU)); if (fp->ctf_version == CTF_VERSION_1 || size < CTF_LSTRUCT_THRESH) { const ctf_member_t *mp = (const ctf_member_t *) ((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, mp++) { + if (mp->ctm_name == 0 && + _ctf_member_info(fp, mp->ctm_type, name, + mp->ctm_offset + off, mip) == 0) + return (0); if (strcmp(ctf_strptr(fp, mp->ctm_name), name) == 0) { mip->ctm_type = mp->ctm_type; - mip->ctm_offset = mp->ctm_offset; + mip->ctm_offset = mp->ctm_offset + off; return (0); } } } else { const ctf_lmember_t *lmp = (const ctf_lmember_t *) ((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, lmp++) { + if (lmp->ctlm_name == 0 && + _ctf_member_info(fp, lmp->ctlm_name, name, + (ulong_t)CTF_LMEM_OFFSET(lmp) + off, mip) == 0) + return (0); if (strcmp(ctf_strptr(fp, lmp->ctlm_name), name) == 0) { mip->ctm_type = lmp->ctlm_type; - mip->ctm_offset = (ulong_t)CTF_LMEM_OFFSET(lmp); + mip->ctm_offset = + (ulong_t)CTF_LMEM_OFFSET(lmp) + off; return (0); } } } return (ctf_set_errno(ofp, ECTF_NOMEMBNAM)); +} + +/* + * Return the type and offset for a given member of a STRUCT or UNION. + */ +int +ctf_member_info(ctf_file_t *fp, ctf_id_t type, const char *name, + ctf_membinfo_t *mip) +{ + + return (_ctf_member_info(fp, type, name, 0, mip)); } /* * Return the array type, index, and size information for the specified ARRAY. */ int ctf_array_info(ctf_file_t *fp, ctf_id_t type, ctf_arinfo_t *arp) { ctf_file_t *ofp = fp; const ctf_type_t *tp; const ctf_array_t *ap; ssize_t increment; if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ if (LCTF_INFO_KIND(fp, tp->ctt_info) != CTF_K_ARRAY) return (ctf_set_errno(ofp, ECTF_NOTARRAY)); (void) ctf_get_ctt_size(fp, tp, NULL, &increment); ap = (const ctf_array_t *)((uintptr_t)tp + increment); arp->ctr_contents = ap->cta_contents; arp->ctr_index = ap->cta_index; arp->ctr_nelems = ap->cta_nelems; return (0); } /* * Convert the specified value to the corresponding enum member name, if a * matching name can be found. Otherwise NULL is returned. */ const char * ctf_enum_name(ctf_file_t *fp, ctf_id_t type, int value) { ctf_file_t *ofp = fp; const ctf_type_t *tp; const ctf_enum_t *ep; ssize_t increment; uint_t n; if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (NULL); /* errno is set for us */ if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (NULL); /* errno is set for us */ if (LCTF_INFO_KIND(fp, tp->ctt_info) != CTF_K_ENUM) { (void) ctf_set_errno(ofp, ECTF_NOTENUM); return (NULL); } (void) ctf_get_ctt_size(fp, tp, NULL, &increment); ep = (const ctf_enum_t *)((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, ep++) { if (ep->cte_value == value) return (ctf_strptr(fp, ep->cte_name)); } (void) ctf_set_errno(ofp, ECTF_NOENUMNAM); return (NULL); } /* * Convert the specified enum tag name to the corresponding value, if a * matching name can be found. Otherwise CTF_ERR is returned. */ int ctf_enum_value(ctf_file_t *fp, ctf_id_t type, const char *name, int *valp) { ctf_file_t *ofp = fp; const ctf_type_t *tp; const ctf_enum_t *ep; ssize_t size, increment; uint_t n; if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (CTF_ERR); /* errno is set for us */ if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ if (LCTF_INFO_KIND(fp, tp->ctt_info) != CTF_K_ENUM) { (void) ctf_set_errno(ofp, ECTF_NOTENUM); return (CTF_ERR); } (void) ctf_get_ctt_size(fp, tp, &size, &increment); ep = (const ctf_enum_t *)((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, ep++) { if (strcmp(ctf_strptr(fp, ep->cte_name), name) == 0) { if (valp != NULL) *valp = ep->cte_value; return (0); } } (void) ctf_set_errno(ofp, ECTF_NOENUMNAM); return (CTF_ERR); } /* * Recursively visit the members of any type. This function is used as the * engine for ctf_type_visit, below. We resolve the input type, recursively * invoke ourself for each type member if the type is a struct or union, and * then invoke the callback function on the current type. If any callback * returns non-zero, we abort and percolate the error code back up to the top. */ static int ctf_type_rvisit(ctf_file_t *fp, ctf_id_t type, ctf_visit_f *func, void *arg, const char *name, ulong_t offset, int depth) { ctf_id_t otype = type; const ctf_type_t *tp; ssize_t size, increment; uint_t kind, n; int rc; if ((type = ctf_type_resolve(fp, type)) == CTF_ERR) return (CTF_ERR); /* errno is set for us */ if ((tp = ctf_lookup_by_id(&fp, type)) == NULL) return (CTF_ERR); /* errno is set for us */ if ((rc = func(name, otype, offset, depth, arg)) != 0) return (rc); kind = LCTF_INFO_KIND(fp, tp->ctt_info); if (kind != CTF_K_STRUCT && kind != CTF_K_UNION) return (0); (void) ctf_get_ctt_size(fp, tp, &size, &increment); if (fp->ctf_version == CTF_VERSION_1 || size < CTF_LSTRUCT_THRESH) { const ctf_member_t *mp = (const ctf_member_t *) ((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, mp++) { if ((rc = ctf_type_rvisit(fp, mp->ctm_type, func, arg, ctf_strptr(fp, mp->ctm_name), offset + mp->ctm_offset, depth + 1)) != 0) return (rc); } } else { const ctf_lmember_t *lmp = (const ctf_lmember_t *) ((uintptr_t)tp + increment); for (n = LCTF_INFO_VLEN(fp, tp->ctt_info); n != 0; n--, lmp++) { if ((rc = ctf_type_rvisit(fp, lmp->ctlm_type, func, arg, ctf_strptr(fp, lmp->ctlm_name), offset + (ulong_t)CTF_LMEM_OFFSET(lmp), depth + 1)) != 0) return (rc); } } return (0); } /* * Recursively visit the members of any type. We pass the name, member * type, and offset of each member to the specified callback function. */ int ctf_type_visit(ctf_file_t *fp, ctf_id_t type, ctf_visit_f *func, void *arg) { return (ctf_type_rvisit(fp, type, func, arg, "", 0, 0)); } Index: projects/netbsd-tests-update-12/cddl/contrib/opensolaris =================================================================== --- projects/netbsd-tests-update-12/cddl/contrib/opensolaris (revision 305171) +++ projects/netbsd-tests-update-12/cddl/contrib/opensolaris (revision 305172) Property changes on: projects/netbsd-tests-update-12/cddl/contrib/opensolaris ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/cddl/contrib/opensolaris:r304816-305170 Index: projects/netbsd-tests-update-12/cddl/usr.sbin/dtrace/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/cddl/usr.sbin/dtrace/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/cddl/usr.sbin/dtrace/Makefile.depend (revision 305172) @@ -1,27 +1,29 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ cddl/lib/libctf \ cddl/lib/libdtrace \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/libc \ lib/libcompiler_rt \ lib/libcxxrt \ lib/libelf \ + lib/libkvm \ lib/libproc \ + lib/libprocstat \ lib/librtld_db \ lib/libthr \ lib/libutil \ lib/libz \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/cddl/usr.sbin/lockstat/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/cddl/usr.sbin/lockstat/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/cddl/usr.sbin/lockstat/Makefile.depend (revision 305172) @@ -1,28 +1,30 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ cddl/lib/libctf \ cddl/lib/libdtrace \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/libc \ lib/libcompiler_rt \ lib/libcxxrt \ lib/libelf \ + lib/libkvm \ lib/libproc \ + lib/libprocstat \ lib/librt \ lib/librtld_db \ lib/libthr \ lib/libutil \ lib/libz \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/cddl/usr.sbin/plockstat/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/cddl/usr.sbin/plockstat/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/cddl/usr.sbin/plockstat/Makefile.depend (revision 305172) @@ -1,27 +1,29 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ cddl/lib/libctf \ cddl/lib/libdtrace \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/libc \ lib/libcompiler_rt \ lib/libcxxrt \ lib/libelf \ + lib/libkvm \ lib/libproc \ + lib/libprocstat \ lib/librtld_db \ lib/libthr \ lib/libutil \ lib/libz \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/cddl/usr.sbin/zfsd/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/cddl/usr.sbin/zfsd/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/cddl/usr.sbin/zfsd/Makefile.depend (revision 305172) @@ -0,0 +1,35 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + cddl/lib/libavl \ + cddl/lib/libnvpair \ + cddl/lib/libumem \ + cddl/lib/libuutil \ + cddl/lib/libzfs \ + cddl/lib/libzfs_core \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + include/xlocale \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libc++ \ + lib/libcompiler_rt \ + lib/libcxxrt \ + lib/libdevdctl \ + lib/libexpat \ + lib/libgeom \ + lib/libmd \ + lib/libsbuf \ + lib/libthr \ + lib/libutil \ + lib/libz \ + lib/msun \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/cddl/usr.sbin/zfsd/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/cddl =================================================================== --- projects/netbsd-tests-update-12/cddl (revision 305171) +++ projects/netbsd-tests-update-12/cddl (revision 305172) Property changes on: projects/netbsd-tests-update-12/cddl ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/cddl:r305021-305170 Index: projects/netbsd-tests-update-12/contrib/elftoolchain/common/elfdefinitions.h =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/common/elfdefinitions.h (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/common/elfdefinitions.h (revision 305172) @@ -1,2889 +1,2892 @@ /*- * Copyright (c) 2010 Joseph Koshy * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * - * $Id: elfdefinitions.h 3455 2016-05-09 13:47:29Z emaste $ + * $Id: elfdefinitions.h 3485 2016-08-18 13:38:52Z emaste $ */ /* * These definitions are based on: * - The public specification of the ELF format as defined in the * October 2009 draft of System V ABI. * See: http://www.sco.com/developers/gabi/latest/ch4.intro.html * - The May 1998 (version 1.5) draft of "The ELF-64 object format". * - Processor-specific ELF ABI definitions for sparc, i386, amd64, mips, * ia64, and powerpc processors. * - The "Linkers and Libraries Guide", from Sun Microsystems. */ #ifndef _ELFDEFINITIONS_H_ #define _ELFDEFINITIONS_H_ #include /* * Types of capabilities. */ #define _ELF_DEFINE_CAPABILITIES() \ _ELF_DEFINE_CA(CA_SUNW_NULL, 0, "ignored") \ _ELF_DEFINE_CA(CA_SUNW_HW_1, 1, "hardware capability") \ _ELF_DEFINE_CA(CA_SUNW_SW_1, 2, "software capability") #undef _ELF_DEFINE_CA #define _ELF_DEFINE_CA(N, V, DESCR) N = V , enum { _ELF_DEFINE_CAPABILITIES() CA__LAST__ }; /* * Flags used with dynamic linking entries. */ #define _ELF_DEFINE_DYN_FLAGS() \ _ELF_DEFINE_DF(DF_ORIGIN, 0x1, \ "object being loaded may refer to $ORIGIN") \ _ELF_DEFINE_DF(DF_SYMBOLIC, 0x2, \ "search library for references before executable") \ _ELF_DEFINE_DF(DF_TEXTREL, 0x4, \ "relocation entries may modify text segment") \ _ELF_DEFINE_DF(DF_BIND_NOW, 0x8, \ "process relocation entries at load time") \ _ELF_DEFINE_DF(DF_STATIC_TLS, 0x10, \ "uses static thread-local storage") #undef _ELF_DEFINE_DF #define _ELF_DEFINE_DF(N, V, DESCR) N = V , enum { _ELF_DEFINE_DYN_FLAGS() DF__LAST__ }; /* * Dynamic linking entry types. */ #define _ELF_DEFINE_DYN_TYPES() \ _ELF_DEFINE_DT(DT_NULL, 0, "end of array") \ _ELF_DEFINE_DT(DT_NEEDED, 1, "names a needed library") \ _ELF_DEFINE_DT(DT_PLTRELSZ, 2, \ "size in bytes of associated relocation entries") \ _ELF_DEFINE_DT(DT_PLTGOT, 3, \ "address associated with the procedure linkage table") \ _ELF_DEFINE_DT(DT_HASH, 4, \ "address of the symbol hash table") \ _ELF_DEFINE_DT(DT_STRTAB, 5, \ "address of the string table") \ _ELF_DEFINE_DT(DT_SYMTAB, 6, \ "address of the symbol table") \ _ELF_DEFINE_DT(DT_RELA, 7, \ "address of the relocation table") \ _ELF_DEFINE_DT(DT_RELASZ, 8, "size of the DT_RELA table") \ _ELF_DEFINE_DT(DT_RELAENT, 9, "size of each DT_RELA entry") \ _ELF_DEFINE_DT(DT_STRSZ, 10, "size of the string table") \ _ELF_DEFINE_DT(DT_SYMENT, 11, \ "size of a symbol table entry") \ _ELF_DEFINE_DT(DT_INIT, 12, \ "address of the initialization function") \ _ELF_DEFINE_DT(DT_FINI, 13, \ "address of the finalization function") \ _ELF_DEFINE_DT(DT_SONAME, 14, "names the shared object") \ _ELF_DEFINE_DT(DT_RPATH, 15, \ "runtime library search path") \ _ELF_DEFINE_DT(DT_SYMBOLIC, 16, \ "alter symbol resolution algorithm") \ _ELF_DEFINE_DT(DT_REL, 17, \ "address of the DT_REL table") \ _ELF_DEFINE_DT(DT_RELSZ, 18, "size of the DT_REL table") \ _ELF_DEFINE_DT(DT_RELENT, 19, "size of each DT_REL entry") \ _ELF_DEFINE_DT(DT_PLTREL, 20, \ "type of relocation entry in the procedure linkage table") \ _ELF_DEFINE_DT(DT_DEBUG, 21, "used for debugging") \ _ELF_DEFINE_DT(DT_TEXTREL, 22, \ "text segment may be written to during relocation") \ _ELF_DEFINE_DT(DT_JMPREL, 23, \ "address of relocation entries associated with the procedure linkage table") \ _ELF_DEFINE_DT(DT_BIND_NOW, 24, \ "bind symbols at loading time") \ _ELF_DEFINE_DT(DT_INIT_ARRAY, 25, \ "pointers to initialization functions") \ _ELF_DEFINE_DT(DT_FINI_ARRAY, 26, \ "pointers to termination functions") \ _ELF_DEFINE_DT(DT_INIT_ARRAYSZ, 27, "size of the DT_INIT_ARRAY") \ _ELF_DEFINE_DT(DT_FINI_ARRAYSZ, 28, "size of the DT_FINI_ARRAY") \ _ELF_DEFINE_DT(DT_RUNPATH, 29, \ "index of library search path string") \ _ELF_DEFINE_DT(DT_FLAGS, 30, \ "flags specific to the object being loaded") \ _ELF_DEFINE_DT(DT_ENCODING, 32, "standard semantics") \ _ELF_DEFINE_DT(DT_PREINIT_ARRAY, 32, \ "pointers to pre-initialization functions") \ _ELF_DEFINE_DT(DT_PREINIT_ARRAYSZ, 33, \ "size of pre-initialization array") \ _ELF_DEFINE_DT(DT_MAXPOSTAGS, 34, \ "the number of positive tags") \ _ELF_DEFINE_DT(DT_LOOS, 0x6000000DUL, \ "start of OS-specific types") \ _ELF_DEFINE_DT(DT_SUNW_AUXILIARY, 0x6000000DUL, \ "offset of string naming auxiliary filtees") \ _ELF_DEFINE_DT(DT_SUNW_RTLDINF, 0x6000000EUL, "rtld internal use") \ _ELF_DEFINE_DT(DT_SUNW_FILTER, 0x6000000FUL, \ "offset of string naming standard filtees") \ _ELF_DEFINE_DT(DT_SUNW_CAP, 0x60000010UL, \ "address of hardware capabilities section") \ _ELF_DEFINE_DT(DT_HIOS, 0x6FFFF000UL, \ "end of OS-specific types") \ _ELF_DEFINE_DT(DT_VALRNGLO, 0x6FFFFD00UL, \ "start of range using the d_val field") \ _ELF_DEFINE_DT(DT_GNU_PRELINKED, 0x6FFFFDF5UL, \ "prelinking timestamp") \ _ELF_DEFINE_DT(DT_GNU_CONFLICTSZ, 0x6FFFFDF6UL, \ "size of conflict section") \ _ELF_DEFINE_DT(DT_GNU_LIBLISTSZ, 0x6FFFFDF7UL, \ "size of library list") \ _ELF_DEFINE_DT(DT_CHECKSUM, 0x6FFFFDF8UL, \ "checksum for the object") \ _ELF_DEFINE_DT(DT_PLTPADSZ, 0x6FFFFDF9UL, \ "size of PLT padding") \ _ELF_DEFINE_DT(DT_MOVEENT, 0x6FFFFDFAUL, \ "size of DT_MOVETAB entries") \ _ELF_DEFINE_DT(DT_MOVESZ, 0x6FFFFDFBUL, \ "total size of the MOVETAB table") \ _ELF_DEFINE_DT(DT_FEATURE, 0x6FFFFDFCUL, "feature values") \ _ELF_DEFINE_DT(DT_POSFLAG_1, 0x6FFFFDFDUL, \ "dynamic position flags") \ _ELF_DEFINE_DT(DT_SYMINSZ, 0x6FFFFDFEUL, \ "size of the DT_SYMINFO table") \ _ELF_DEFINE_DT(DT_SYMINENT, 0x6FFFFDFFUL, \ "size of a DT_SYMINFO entry") \ _ELF_DEFINE_DT(DT_VALRNGHI, 0x6FFFFDFFUL, \ "end of range using the d_val field") \ _ELF_DEFINE_DT(DT_ADDRRNGLO, 0x6FFFFE00UL, \ "start of range using the d_ptr field") \ _ELF_DEFINE_DT(DT_GNU_HASH, 0x6FFFFEF5UL, \ "GNU style hash tables") \ _ELF_DEFINE_DT(DT_TLSDESC_PLT, 0x6FFFFEF6UL, \ "location of PLT entry for TLS descriptor resolver calls") \ _ELF_DEFINE_DT(DT_TLSDESC_GOT, 0x6FFFFEF7UL, \ "location of GOT entry used by TLS descriptor resolver PLT entry") \ _ELF_DEFINE_DT(DT_GNU_CONFLICT, 0x6FFFFEF8UL, \ "address of conflict section") \ _ELF_DEFINE_DT(DT_GNU_LIBLIST, 0x6FFFFEF9UL, \ "address of conflict section") \ _ELF_DEFINE_DT(DT_CONFIG, 0x6FFFFEFAUL, \ "configuration file") \ _ELF_DEFINE_DT(DT_DEPAUDIT, 0x6FFFFEFBUL, \ "string defining audit libraries") \ _ELF_DEFINE_DT(DT_AUDIT, 0x6FFFFEFCUL, \ "string defining audit libraries") \ _ELF_DEFINE_DT(DT_PLTPAD, 0x6FFFFEFDUL, "PLT padding") \ _ELF_DEFINE_DT(DT_MOVETAB, 0x6FFFFEFEUL, \ "address of a move table") \ _ELF_DEFINE_DT(DT_SYMINFO, 0x6FFFFEFFUL, \ "address of the symbol information table") \ _ELF_DEFINE_DT(DT_ADDRRNGHI, 0x6FFFFEFFUL, \ "end of range using the d_ptr field") \ _ELF_DEFINE_DT(DT_VERSYM, 0x6FFFFFF0UL, \ "address of the version section") \ _ELF_DEFINE_DT(DT_RELACOUNT, 0x6FFFFFF9UL, \ "count of RELA relocations") \ _ELF_DEFINE_DT(DT_RELCOUNT, 0x6FFFFFFAUL, \ "count of REL relocations") \ _ELF_DEFINE_DT(DT_FLAGS_1, 0x6FFFFFFBUL, "flag values") \ _ELF_DEFINE_DT(DT_VERDEF, 0x6FFFFFFCUL, \ "address of the version definition segment") \ _ELF_DEFINE_DT(DT_VERDEFNUM, 0x6FFFFFFDUL, \ "the number of version definition entries") \ _ELF_DEFINE_DT(DT_VERNEED, 0x6FFFFFFEUL, \ "address of section with needed versions") \ _ELF_DEFINE_DT(DT_VERNEEDNUM, 0x6FFFFFFFUL, \ "the number of version needed entries") \ _ELF_DEFINE_DT(DT_LOPROC, 0x70000000UL, \ "start of processor-specific types") \ _ELF_DEFINE_DT(DT_ARM_SYMTABSZ, 0x70000001UL, \ "number of entries in the dynamic symbol table") \ _ELF_DEFINE_DT(DT_SPARC_REGISTER, 0x70000001UL, \ "index of an STT_SPARC_REGISTER symbol") \ _ELF_DEFINE_DT(DT_ARM_PREEMPTMAP, 0x70000002UL, \ "address of the preemption map") \ _ELF_DEFINE_DT(DT_MIPS_RLD_VERSION, 0x70000001UL, \ "version ID for runtime linker interface") \ _ELF_DEFINE_DT(DT_MIPS_TIME_STAMP, 0x70000002UL, \ "timestamp") \ _ELF_DEFINE_DT(DT_MIPS_ICHECKSUM, 0x70000003UL, \ "checksum of all external strings and common sizes") \ _ELF_DEFINE_DT(DT_MIPS_IVERSION, 0x70000004UL, \ "string table index of a version string") \ _ELF_DEFINE_DT(DT_MIPS_FLAGS, 0x70000005UL, \ "MIPS-specific flags") \ _ELF_DEFINE_DT(DT_MIPS_BASE_ADDRESS, 0x70000006UL, \ "base address for the executable/DSO") \ _ELF_DEFINE_DT(DT_MIPS_CONFLICT, 0x70000008UL, \ "address of .conflict section") \ _ELF_DEFINE_DT(DT_MIPS_LIBLIST, 0x70000009UL, \ "address of .liblist section") \ _ELF_DEFINE_DT(DT_MIPS_LOCAL_GOTNO, 0x7000000AUL, \ "number of local GOT entries") \ _ELF_DEFINE_DT(DT_MIPS_CONFLICTNO, 0x7000000BUL, \ "number of entries in the .conflict section") \ _ELF_DEFINE_DT(DT_MIPS_LIBLISTNO, 0x70000010UL, \ "number of entries in the .liblist section") \ _ELF_DEFINE_DT(DT_MIPS_SYMTABNO, 0x70000011UL, \ "number of entries in the .dynsym section") \ _ELF_DEFINE_DT(DT_MIPS_UNREFEXTNO, 0x70000012UL, \ "index of first external dynamic symbol not ref'ed locally") \ _ELF_DEFINE_DT(DT_MIPS_GOTSYM, 0x70000013UL, \ "index of first dynamic symbol corresponds to a GOT entry") \ _ELF_DEFINE_DT(DT_MIPS_HIPAGENO, 0x70000014UL, \ "number of page table entries in GOT") \ _ELF_DEFINE_DT(DT_MIPS_RLD_MAP, 0x70000016UL, \ "address of runtime linker map") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_CLASS, 0x70000017UL, \ "Delta C++ class definition") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_CLASS_NO, 0x70000018UL, \ "number of entries in DT_MIPS_DELTA_CLASS") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_INSTANCE, 0x70000019UL, \ "Delta C++ class instances") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_INSTANCE_NO, 0x7000001AUL, \ "number of entries in DT_MIPS_DELTA_INSTANCE") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_RELOC, 0x7000001BUL, \ "Delta relocations") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_RELOC_NO, 0x7000001CUL, \ "number of entries in DT_MIPS_DELTA_RELOC") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_SYM, 0x7000001DUL, \ "Delta symbols referred by Delta relocations") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_SYM_NO, 0x7000001EUL, \ "number of entries in DT_MIPS_DELTA_SYM") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_CLASSSYM, 0x70000020UL, \ "Delta symbols for class declarations") \ _ELF_DEFINE_DT(DT_MIPS_DELTA_CLASSSYM_NO, 0x70000021UL, \ "number of entries in DT_MIPS_DELTA_CLASSSYM") \ _ELF_DEFINE_DT(DT_MIPS_CXX_FLAGS, 0x70000022UL, \ "C++ flavor flags") \ _ELF_DEFINE_DT(DT_MIPS_PIXIE_INIT, 0x70000023UL, \ "address of an initialization routine created by pixie") \ _ELF_DEFINE_DT(DT_MIPS_SYMBOL_LIB, 0x70000024UL, \ "address of .MIPS.symlib section") \ _ELF_DEFINE_DT(DT_MIPS_LOCALPAGE_GOTIDX, 0x70000025UL, \ "GOT index of first page table entry for a segment") \ _ELF_DEFINE_DT(DT_MIPS_LOCAL_GOTIDX, 0x70000026UL, \ "GOT index of first page table entry for a local symbol") \ _ELF_DEFINE_DT(DT_MIPS_HIDDEN_GOTIDX, 0x70000027UL, \ "GOT index of first page table entry for a hidden symbol") \ _ELF_DEFINE_DT(DT_MIPS_PROTECTED_GOTIDX, 0x70000028UL, \ "GOT index of first page table entry for a protected symbol") \ _ELF_DEFINE_DT(DT_MIPS_OPTIONS, 0x70000029UL, \ "address of .MIPS.options section") \ _ELF_DEFINE_DT(DT_MIPS_INTERFACE, 0x7000002AUL, \ "address of .MIPS.interface section") \ _ELF_DEFINE_DT(DT_MIPS_DYNSTR_ALIGN, 0x7000002BUL, "???") \ _ELF_DEFINE_DT(DT_MIPS_INTERFACE_SIZE, 0x7000002CUL, \ "size of .MIPS.interface section") \ _ELF_DEFINE_DT(DT_MIPS_RLD_TEXT_RESOLVE_ADDR, 0x7000002DUL, \ "address of _rld_text_resolve in GOT") \ _ELF_DEFINE_DT(DT_MIPS_PERF_SUFFIX, 0x7000002EUL, \ "default suffix of DSO to be appended by dlopen") \ _ELF_DEFINE_DT(DT_MIPS_COMPACT_SIZE, 0x7000002FUL, \ "size of a ucode compact relocation record (o32)") \ _ELF_DEFINE_DT(DT_MIPS_GP_VALUE, 0x70000030UL, \ "GP value of a specified GP relative range") \ _ELF_DEFINE_DT(DT_MIPS_AUX_DYNAMIC, 0x70000031UL, \ "address of an auxiliary dynamic table") \ _ELF_DEFINE_DT(DT_MIPS_PLTGOT, 0x70000032UL, \ "address of the PLTGOT") \ _ELF_DEFINE_DT(DT_MIPS_RLD_OBJ_UPDATE, 0x70000033UL, \ "object list update callback") \ _ELF_DEFINE_DT(DT_MIPS_RWPLT, 0x70000034UL, \ "address of a writable PLT") \ _ELF_DEFINE_DT(DT_PPC_GOT, 0x70000000UL, \ "value of _GLOBAL_OFFSET_TABLE_") \ _ELF_DEFINE_DT(DT_PPC_TLSOPT, 0x70000001UL, \ "TLS descriptor should be optimized") \ _ELF_DEFINE_DT(DT_PPC64_GLINK, 0x70000000UL, \ "address of .glink section") \ _ELF_DEFINE_DT(DT_PPC64_OPD, 0x70000001UL, \ "address of .opd section") \ _ELF_DEFINE_DT(DT_PPC64_OPDSZ, 0x70000002UL, \ "size of .opd section") \ _ELF_DEFINE_DT(DT_PPC64_TLSOPT, 0x70000003UL, \ "TLS descriptor should be optimized") \ _ELF_DEFINE_DT(DT_AUXILIARY, 0x7FFFFFFDUL, \ "offset of string naming auxiliary filtees") \ _ELF_DEFINE_DT(DT_USED, 0x7FFFFFFEUL, "ignored") \ _ELF_DEFINE_DT(DT_FILTER, 0x7FFFFFFFUL, \ "index of string naming filtees") \ _ELF_DEFINE_DT(DT_HIPROC, 0x7FFFFFFFUL, \ "end of processor-specific types") #undef _ELF_DEFINE_DT #define _ELF_DEFINE_DT(N, V, DESCR) N = V , enum { _ELF_DEFINE_DYN_TYPES() DT__LAST__ = DT_HIPROC }; #define DT_DEPRECATED_SPARC_REGISTER DT_SPARC_REGISTER /* * Flags used in the executable header (field: e_flags). */ #define _ELF_DEFINE_EHDR_FLAGS() \ _ELF_DEFINE_EF(EF_ARM_RELEXEC, 0x00000001UL, \ "dynamic segment describes only how to relocate segments") \ _ELF_DEFINE_EF(EF_ARM_HASENTRY, 0x00000002UL, \ "e_entry contains a program entry point") \ _ELF_DEFINE_EF(EF_ARM_SYMSARESORTED, 0x00000004UL, \ "subsection of symbol table is sorted by symbol value") \ _ELF_DEFINE_EF(EF_ARM_DYNSYMSUSESEGIDX, 0x00000008UL, \ "dynamic symbol st_shndx = containing segment index + 1") \ _ELF_DEFINE_EF(EF_ARM_MAPSYMSFIRST, 0x00000010UL, \ "mapping symbols precede other local symbols in symtab") \ _ELF_DEFINE_EF(EF_ARM_BE8, 0x00800000UL, \ "file contains BE-8 code") \ _ELF_DEFINE_EF(EF_ARM_LE8, 0x00400000UL, \ "file contains LE-8 code") \ _ELF_DEFINE_EF(EF_ARM_EABIMASK, 0xFF000000UL, \ "mask for ARM EABI version number (0 denotes GNU or unknown)") \ _ELF_DEFINE_EF(EF_ARM_EABI_UNKNOWN, 0x00000000UL, \ "Unknown or GNU ARM EABI version number") \ _ELF_DEFINE_EF(EF_ARM_EABI_VER1, 0x01000000UL, \ "ARM EABI version 1") \ _ELF_DEFINE_EF(EF_ARM_EABI_VER2, 0x02000000UL, \ "ARM EABI version 2") \ _ELF_DEFINE_EF(EF_ARM_EABI_VER3, 0x03000000UL, \ "ARM EABI version 3") \ _ELF_DEFINE_EF(EF_ARM_EABI_VER4, 0x04000000UL, \ "ARM EABI version 4") \ _ELF_DEFINE_EF(EF_ARM_EABI_VER5, 0x05000000UL, \ "ARM EABI version 5") \ _ELF_DEFINE_EF(EF_ARM_INTERWORK, 0x00000004UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_APCS_26, 0x00000008UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_APCS_FLOAT, 0x00000010UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_PIC, 0x00000020UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_ALIGN8, 0x00000040UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_NEW_ABI, 0x00000080UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_OLD_ABI, 0x00000100UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_SOFT_FLOAT, 0x00000200UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_VFP_FLOAT, 0x00000400UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_ARM_MAVERICK_FLOAT, 0x00000800UL, \ "GNU EABI extension") \ _ELF_DEFINE_EF(EF_MIPS_NOREORDER, 0x00000001UL, \ "at least one .noreorder directive appeared in the source") \ _ELF_DEFINE_EF(EF_MIPS_PIC, 0x00000002UL, \ "file contains position independent code") \ _ELF_DEFINE_EF(EF_MIPS_CPIC, 0x00000004UL, \ "file's code uses standard conventions for calling PIC") \ _ELF_DEFINE_EF(EF_MIPS_UCODE, 0x00000010UL, \ "file contains UCODE (obsolete)") \ _ELF_DEFINE_EF(EF_MIPS_ABI2, 0x00000020UL, \ "file follows MIPS III 32-bit ABI") \ _ELF_DEFINE_EF(EF_MIPS_OPTIONS_FIRST, 0x00000080UL, \ "ld(1) should process .MIPS.options section first") \ _ELF_DEFINE_EF(EF_MIPS_ARCH_ASE, 0x0F000000UL, \ "file uses application-specific architectural extensions") \ _ELF_DEFINE_EF(EF_MIPS_ARCH_ASE_MDMX, 0x08000000UL, \ "file uses MDMX multimedia extensions") \ _ELF_DEFINE_EF(EF_MIPS_ARCH_ASE_M16, 0x04000000UL, \ "file uses MIPS-16 ISA extensions") \ _ELF_DEFINE_EF(EF_MIPS_ARCH, 0xF0000000UL, \ "4-bit MIPS architecture field") \ _ELF_DEFINE_EF(EF_PPC_EMB, 0x80000000UL, \ "Embedded PowerPC flag") \ _ELF_DEFINE_EF(EF_PPC_RELOCATABLE, 0x00010000UL, \ "-mrelocatable flag") \ _ELF_DEFINE_EF(EF_PPC_RELOCATABLE_LIB, 0x00008000UL, \ "-mrelocatable-lib flag") \ _ELF_DEFINE_EF(EF_SPARC_EXT_MASK, 0x00ffff00UL, \ "Vendor Extension mask") \ _ELF_DEFINE_EF(EF_SPARC_32PLUS, 0x00000100UL, \ "Generic V8+ features") \ _ELF_DEFINE_EF(EF_SPARC_SUN_US1, 0x00000200UL, \ "Sun UltraSPARCTM 1 Extensions") \ _ELF_DEFINE_EF(EF_SPARC_HAL_R1, 0x00000400UL, "HAL R1 Extensions") \ _ELF_DEFINE_EF(EF_SPARC_SUN_US3, 0x00000800UL, \ "Sun UltraSPARC 3 Extensions") \ _ELF_DEFINE_EF(EF_SPARCV9_MM, 0x00000003UL, \ "Mask for Memory Model") \ _ELF_DEFINE_EF(EF_SPARCV9_TSO, 0x00000000UL, \ "Total Store Ordering") \ _ELF_DEFINE_EF(EF_SPARCV9_PSO, 0x00000001UL, \ "Partial Store Ordering") \ _ELF_DEFINE_EF(EF_SPARCV9_RMO, 0x00000002UL, \ "Relaxed Memory Ordering") #undef _ELF_DEFINE_EF #define _ELF_DEFINE_EF(N, V, DESCR) N = V , enum { _ELF_DEFINE_EHDR_FLAGS() EF__LAST__ }; /* * Offsets in the `ei_ident[]` field of an ELF executable header. */ #define _ELF_DEFINE_EI_OFFSETS() \ _ELF_DEFINE_EI(EI_MAG0, 0, "magic number") \ _ELF_DEFINE_EI(EI_MAG1, 1, "magic number") \ _ELF_DEFINE_EI(EI_MAG2, 2, "magic number") \ _ELF_DEFINE_EI(EI_MAG3, 3, "magic number") \ _ELF_DEFINE_EI(EI_CLASS, 4, "file class") \ _ELF_DEFINE_EI(EI_DATA, 5, "data encoding") \ _ELF_DEFINE_EI(EI_VERSION, 6, "file version") \ _ELF_DEFINE_EI(EI_OSABI, 7, "OS ABI kind") \ _ELF_DEFINE_EI(EI_ABIVERSION, 8, "OS ABI version") \ _ELF_DEFINE_EI(EI_PAD, 9, "padding start") \ _ELF_DEFINE_EI(EI_NIDENT, 16, "total size") #undef _ELF_DEFINE_EI #define _ELF_DEFINE_EI(N, V, DESCR) N = V , enum { _ELF_DEFINE_EI_OFFSETS() EI__LAST__ }; /* * The ELF class of an object. */ #define _ELF_DEFINE_ELFCLASS() \ _ELF_DEFINE_EC(ELFCLASSNONE, 0, "Unknown ELF class") \ _ELF_DEFINE_EC(ELFCLASS32, 1, "32 bit objects") \ _ELF_DEFINE_EC(ELFCLASS64, 2, "64 bit objects") #undef _ELF_DEFINE_EC #define _ELF_DEFINE_EC(N, V, DESCR) N = V , enum { _ELF_DEFINE_ELFCLASS() EC__LAST__ }; /* * Endianness of data in an ELF object. */ #define _ELF_DEFINE_ELF_DATA_ENDIANNESS() \ _ELF_DEFINE_ED(ELFDATANONE, 0, "Unknown data endianness") \ _ELF_DEFINE_ED(ELFDATA2LSB, 1, "little endian") \ _ELF_DEFINE_ED(ELFDATA2MSB, 2, "big endian") #undef _ELF_DEFINE_ED #define _ELF_DEFINE_ED(N, V, DESCR) N = V , enum { _ELF_DEFINE_ELF_DATA_ENDIANNESS() ED__LAST__ }; /* * Values of the magic numbers used in identification array. */ #define _ELF_DEFINE_ELF_MAGIC() \ _ELF_DEFINE_EMAG(ELFMAG0, 0x7FU) \ _ELF_DEFINE_EMAG(ELFMAG1, 'E') \ _ELF_DEFINE_EMAG(ELFMAG2, 'L') \ _ELF_DEFINE_EMAG(ELFMAG3, 'F') #undef _ELF_DEFINE_EMAG #define _ELF_DEFINE_EMAG(N, V) N = V , enum { _ELF_DEFINE_ELF_MAGIC() ELFMAG__LAST__ }; /* * ELF OS ABI field. */ #define _ELF_DEFINE_ELF_OSABI() \ _ELF_DEFINE_EABI(ELFOSABI_NONE, 0, \ "No extensions or unspecified") \ _ELF_DEFINE_EABI(ELFOSABI_SYSV, 0, "SYSV") \ _ELF_DEFINE_EABI(ELFOSABI_HPUX, 1, "Hewlett-Packard HP-UX") \ _ELF_DEFINE_EABI(ELFOSABI_NETBSD, 2, "NetBSD") \ _ELF_DEFINE_EABI(ELFOSABI_GNU, 3, "GNU") \ _ELF_DEFINE_EABI(ELFOSABI_HURD, 4, "GNU/HURD") \ _ELF_DEFINE_EABI(ELFOSABI_86OPEN, 5, "86Open Common ABI") \ _ELF_DEFINE_EABI(ELFOSABI_SOLARIS, 6, "Sun Solaris") \ _ELF_DEFINE_EABI(ELFOSABI_AIX, 7, "AIX") \ _ELF_DEFINE_EABI(ELFOSABI_IRIX, 8, "IRIX") \ _ELF_DEFINE_EABI(ELFOSABI_FREEBSD, 9, "FreeBSD") \ _ELF_DEFINE_EABI(ELFOSABI_TRU64, 10, "Compaq TRU64 UNIX") \ _ELF_DEFINE_EABI(ELFOSABI_MODESTO, 11, "Novell Modesto") \ _ELF_DEFINE_EABI(ELFOSABI_OPENBSD, 12, "Open BSD") \ _ELF_DEFINE_EABI(ELFOSABI_OPENVMS, 13, "Open VMS") \ _ELF_DEFINE_EABI(ELFOSABI_NSK, 14, \ "Hewlett-Packard Non-Stop Kernel") \ _ELF_DEFINE_EABI(ELFOSABI_AROS, 15, "Amiga Research OS") \ _ELF_DEFINE_EABI(ELFOSABI_FENIXOS, 16, \ "The FenixOS highly scalable multi-core OS") \ _ELF_DEFINE_EABI(ELFOSABI_CLOUDABI, 17, "Nuxi CloudABI") \ _ELF_DEFINE_EABI(ELFOSABI_ARM_AEABI, 64, \ "ARM specific symbol versioning extensions") \ _ELF_DEFINE_EABI(ELFOSABI_ARM, 97, "ARM ABI") \ _ELF_DEFINE_EABI(ELFOSABI_STANDALONE, 255, \ "Standalone (embedded) application") #undef _ELF_DEFINE_EABI #define _ELF_DEFINE_EABI(N, V, DESCR) N = V , enum { _ELF_DEFINE_ELF_OSABI() ELFOSABI__LAST__ }; #define ELFOSABI_LINUX ELFOSABI_GNU /* * ELF Machine types: (EM_*). */ #define _ELF_DEFINE_ELF_MACHINES() \ _ELF_DEFINE_EM(EM_NONE, 0, "No machine") \ _ELF_DEFINE_EM(EM_M32, 1, "AT&T WE 32100") \ _ELF_DEFINE_EM(EM_SPARC, 2, "SPARC") \ _ELF_DEFINE_EM(EM_386, 3, "Intel 80386") \ _ELF_DEFINE_EM(EM_68K, 4, "Motorola 68000") \ _ELF_DEFINE_EM(EM_88K, 5, "Motorola 88000") \ _ELF_DEFINE_EM(EM_IAMCU, 6, "Intel MCU") \ _ELF_DEFINE_EM(EM_860, 7, "Intel 80860") \ _ELF_DEFINE_EM(EM_MIPS, 8, "MIPS I Architecture") \ _ELF_DEFINE_EM(EM_S370, 9, "IBM System/370 Processor") \ _ELF_DEFINE_EM(EM_MIPS_RS3_LE, 10, "MIPS RS3000 Little-endian") \ _ELF_DEFINE_EM(EM_PARISC, 15, "Hewlett-Packard PA-RISC") \ _ELF_DEFINE_EM(EM_VPP500, 17, "Fujitsu VPP500") \ _ELF_DEFINE_EM(EM_SPARC32PLUS, 18, \ "Enhanced instruction set SPARC") \ _ELF_DEFINE_EM(EM_960, 19, "Intel 80960") \ _ELF_DEFINE_EM(EM_PPC, 20, "PowerPC") \ _ELF_DEFINE_EM(EM_PPC64, 21, "64-bit PowerPC") \ _ELF_DEFINE_EM(EM_S390, 22, "IBM System/390 Processor") \ _ELF_DEFINE_EM(EM_SPU, 23, "IBM SPU/SPC") \ _ELF_DEFINE_EM(EM_V800, 36, "NEC V800") \ _ELF_DEFINE_EM(EM_FR20, 37, "Fujitsu FR20") \ _ELF_DEFINE_EM(EM_RH32, 38, "TRW RH-32") \ _ELF_DEFINE_EM(EM_RCE, 39, "Motorola RCE") \ _ELF_DEFINE_EM(EM_ARM, 40, "Advanced RISC Machines ARM") \ _ELF_DEFINE_EM(EM_ALPHA, 41, "Digital Alpha") \ _ELF_DEFINE_EM(EM_SH, 42, "Hitachi SH") \ _ELF_DEFINE_EM(EM_SPARCV9, 43, "SPARC Version 9") \ _ELF_DEFINE_EM(EM_TRICORE, 44, \ "Siemens TriCore embedded processor") \ _ELF_DEFINE_EM(EM_ARC, 45, \ "Argonaut RISC Core, Argonaut Technologies Inc.") \ _ELF_DEFINE_EM(EM_H8_300, 46, "Hitachi H8/300") \ _ELF_DEFINE_EM(EM_H8_300H, 47, "Hitachi H8/300H") \ _ELF_DEFINE_EM(EM_H8S, 48, "Hitachi H8S") \ _ELF_DEFINE_EM(EM_H8_500, 49, "Hitachi H8/500") \ _ELF_DEFINE_EM(EM_IA_64, 50, \ "Intel IA-64 processor architecture") \ _ELF_DEFINE_EM(EM_MIPS_X, 51, "Stanford MIPS-X") \ _ELF_DEFINE_EM(EM_COLDFIRE, 52, "Motorola ColdFire") \ _ELF_DEFINE_EM(EM_68HC12, 53, "Motorola M68HC12") \ _ELF_DEFINE_EM(EM_MMA, 54, \ "Fujitsu MMA Multimedia Accelerator") \ _ELF_DEFINE_EM(EM_PCP, 55, "Siemens PCP") \ _ELF_DEFINE_EM(EM_NCPU, 56, \ "Sony nCPU embedded RISC processor") \ _ELF_DEFINE_EM(EM_NDR1, 57, "Denso NDR1 microprocessor") \ _ELF_DEFINE_EM(EM_STARCORE, 58, "Motorola Star*Core processor") \ _ELF_DEFINE_EM(EM_ME16, 59, "Toyota ME16 processor") \ _ELF_DEFINE_EM(EM_ST100, 60, \ "STMicroelectronics ST100 processor") \ _ELF_DEFINE_EM(EM_TINYJ, 61, \ "Advanced Logic Corp. TinyJ embedded processor family") \ _ELF_DEFINE_EM(EM_X86_64, 62, "AMD x86-64 architecture") \ _ELF_DEFINE_EM(EM_PDSP, 63, "Sony DSP Processor") \ _ELF_DEFINE_EM(EM_PDP10, 64, \ "Digital Equipment Corp. PDP-10") \ _ELF_DEFINE_EM(EM_PDP11, 65, \ "Digital Equipment Corp. PDP-11") \ _ELF_DEFINE_EM(EM_FX66, 66, "Siemens FX66 microcontroller") \ _ELF_DEFINE_EM(EM_ST9PLUS, 67, \ "STMicroelectronics ST9+ 8/16 bit microcontroller") \ _ELF_DEFINE_EM(EM_ST7, 68, \ "STMicroelectronics ST7 8-bit microcontroller") \ _ELF_DEFINE_EM(EM_68HC16, 69, \ "Motorola MC68HC16 Microcontroller") \ _ELF_DEFINE_EM(EM_68HC11, 70, \ "Motorola MC68HC11 Microcontroller") \ _ELF_DEFINE_EM(EM_68HC08, 71, \ "Motorola MC68HC08 Microcontroller") \ _ELF_DEFINE_EM(EM_68HC05, 72, \ "Motorola MC68HC05 Microcontroller") \ _ELF_DEFINE_EM(EM_SVX, 73, "Silicon Graphics SVx") \ _ELF_DEFINE_EM(EM_ST19, 74, \ "STMicroelectronics ST19 8-bit microcontroller") \ _ELF_DEFINE_EM(EM_VAX, 75, "Digital VAX") \ _ELF_DEFINE_EM(EM_CRIS, 76, \ "Axis Communications 32-bit embedded processor") \ _ELF_DEFINE_EM(EM_JAVELIN, 77, \ "Infineon Technologies 32-bit embedded processor") \ _ELF_DEFINE_EM(EM_FIREPATH, 78, \ "Element 14 64-bit DSP Processor") \ _ELF_DEFINE_EM(EM_ZSP, 79, \ "LSI Logic 16-bit DSP Processor") \ _ELF_DEFINE_EM(EM_MMIX, 80, \ "Donald Knuth's educational 64-bit processor") \ _ELF_DEFINE_EM(EM_HUANY, 81, \ "Harvard University machine-independent object files") \ _ELF_DEFINE_EM(EM_PRISM, 82, "SiTera Prism") \ _ELF_DEFINE_EM(EM_AVR, 83, \ "Atmel AVR 8-bit microcontroller") \ _ELF_DEFINE_EM(EM_FR30, 84, "Fujitsu FR30") \ _ELF_DEFINE_EM(EM_D10V, 85, "Mitsubishi D10V") \ _ELF_DEFINE_EM(EM_D30V, 86, "Mitsubishi D30V") \ _ELF_DEFINE_EM(EM_V850, 87, "NEC v850") \ _ELF_DEFINE_EM(EM_M32R, 88, "Mitsubishi M32R") \ _ELF_DEFINE_EM(EM_MN10300, 89, "Matsushita MN10300") \ _ELF_DEFINE_EM(EM_MN10200, 90, "Matsushita MN10200") \ _ELF_DEFINE_EM(EM_PJ, 91, "picoJava") \ _ELF_DEFINE_EM(EM_OPENRISC, 92, \ "OpenRISC 32-bit embedded processor") \ _ELF_DEFINE_EM(EM_ARC_COMPACT, 93, \ "ARC International ARCompact processor") \ _ELF_DEFINE_EM(EM_XTENSA, 94, \ "Tensilica Xtensa Architecture") \ _ELF_DEFINE_EM(EM_VIDEOCORE, 95, \ "Alphamosaic VideoCore processor") \ _ELF_DEFINE_EM(EM_TMM_GPP, 96, \ "Thompson Multimedia General Purpose Processor") \ _ELF_DEFINE_EM(EM_NS32K, 97, \ "National Semiconductor 32000 series") \ _ELF_DEFINE_EM(EM_TPC, 98, "Tenor Network TPC processor") \ _ELF_DEFINE_EM(EM_SNP1K, 99, "Trebia SNP 1000 processor") \ _ELF_DEFINE_EM(EM_ST200, 100, \ "STMicroelectronics (www.st.com) ST200 microcontroller") \ _ELF_DEFINE_EM(EM_IP2K, 101, \ "Ubicom IP2xxx microcontroller family") \ _ELF_DEFINE_EM(EM_MAX, 102, "MAX Processor") \ _ELF_DEFINE_EM(EM_CR, 103, \ "National Semiconductor CompactRISC microprocessor") \ _ELF_DEFINE_EM(EM_F2MC16, 104, "Fujitsu F2MC16") \ _ELF_DEFINE_EM(EM_MSP430, 105, \ "Texas Instruments embedded microcontroller msp430") \ _ELF_DEFINE_EM(EM_BLACKFIN, 106, \ "Analog Devices Blackfin (DSP) processor") \ _ELF_DEFINE_EM(EM_SE_C33, 107, \ "S1C33 Family of Seiko Epson processors") \ _ELF_DEFINE_EM(EM_SEP, 108, \ "Sharp embedded microprocessor") \ _ELF_DEFINE_EM(EM_ARCA, 109, "Arca RISC Microprocessor") \ _ELF_DEFINE_EM(EM_UNICORE, 110, \ "Microprocessor series from PKU-Unity Ltd. and MPRC of Peking University") \ _ELF_DEFINE_EM(EM_EXCESS, 111, \ "eXcess: 16/32/64-bit configurable embedded CPU") \ _ELF_DEFINE_EM(EM_DXP, 112, \ "Icera Semiconductor Inc. Deep Execution Processor") \ _ELF_DEFINE_EM(EM_ALTERA_NIOS2, 113, \ "Altera Nios II soft-core processor") \ _ELF_DEFINE_EM(EM_CRX, 114, \ "National Semiconductor CompactRISC CRX microprocessor") \ _ELF_DEFINE_EM(EM_XGATE, 115, \ "Motorola XGATE embedded processor") \ _ELF_DEFINE_EM(EM_C166, 116, \ "Infineon C16x/XC16x processor") \ _ELF_DEFINE_EM(EM_M16C, 117, \ "Renesas M16C series microprocessors") \ _ELF_DEFINE_EM(EM_DSPIC30F, 118, \ "Microchip Technology dsPIC30F Digital Signal Controller") \ _ELF_DEFINE_EM(EM_CE, 119, \ "Freescale Communication Engine RISC core") \ _ELF_DEFINE_EM(EM_M32C, 120, \ "Renesas M32C series microprocessors") \ _ELF_DEFINE_EM(EM_TSK3000, 131, "Altium TSK3000 core") \ _ELF_DEFINE_EM(EM_RS08, 132, \ "Freescale RS08 embedded processor") \ _ELF_DEFINE_EM(EM_SHARC, 133, \ "Analog Devices SHARC family of 32-bit DSP processors") \ _ELF_DEFINE_EM(EM_ECOG2, 134, \ "Cyan Technology eCOG2 microprocessor") \ _ELF_DEFINE_EM(EM_SCORE7, 135, \ "Sunplus S+core7 RISC processor") \ _ELF_DEFINE_EM(EM_DSP24, 136, \ "New Japan Radio (NJR) 24-bit DSP Processor") \ _ELF_DEFINE_EM(EM_VIDEOCORE3, 137, \ "Broadcom VideoCore III processor") \ _ELF_DEFINE_EM(EM_LATTICEMICO32, 138, \ "RISC processor for Lattice FPGA architecture") \ _ELF_DEFINE_EM(EM_SE_C17, 139, "Seiko Epson C17 family") \ _ELF_DEFINE_EM(EM_TI_C6000, 140, \ "The Texas Instruments TMS320C6000 DSP family") \ _ELF_DEFINE_EM(EM_TI_C2000, 141, \ "The Texas Instruments TMS320C2000 DSP family") \ _ELF_DEFINE_EM(EM_TI_C5500, 142, \ "The Texas Instruments TMS320C55x DSP family") \ _ELF_DEFINE_EM(EM_MMDSP_PLUS, 160, \ "STMicroelectronics 64bit VLIW Data Signal Processor") \ _ELF_DEFINE_EM(EM_CYPRESS_M8C, 161, "Cypress M8C microprocessor") \ _ELF_DEFINE_EM(EM_R32C, 162, \ "Renesas R32C series microprocessors") \ _ELF_DEFINE_EM(EM_TRIMEDIA, 163, \ "NXP Semiconductors TriMedia architecture family") \ _ELF_DEFINE_EM(EM_QDSP6, 164, "QUALCOMM DSP6 Processor") \ _ELF_DEFINE_EM(EM_8051, 165, "Intel 8051 and variants") \ _ELF_DEFINE_EM(EM_STXP7X, 166, \ "STMicroelectronics STxP7x family of configurable and extensible RISC processors") \ _ELF_DEFINE_EM(EM_NDS32, 167, \ "Andes Technology compact code size embedded RISC processor family") \ _ELF_DEFINE_EM(EM_ECOG1, 168, \ "Cyan Technology eCOG1X family") \ _ELF_DEFINE_EM(EM_ECOG1X, 168, \ "Cyan Technology eCOG1X family") \ _ELF_DEFINE_EM(EM_MAXQ30, 169, \ "Dallas Semiconductor MAXQ30 Core Micro-controllers") \ _ELF_DEFINE_EM(EM_XIMO16, 170, \ "New Japan Radio (NJR) 16-bit DSP Processor") \ _ELF_DEFINE_EM(EM_MANIK, 171, \ "M2000 Reconfigurable RISC Microprocessor") \ _ELF_DEFINE_EM(EM_CRAYNV2, 172, \ "Cray Inc. NV2 vector architecture") \ _ELF_DEFINE_EM(EM_RX, 173, "Renesas RX family") \ _ELF_DEFINE_EM(EM_METAG, 174, \ "Imagination Technologies META processor architecture") \ _ELF_DEFINE_EM(EM_MCST_ELBRUS, 175, \ "MCST Elbrus general purpose hardware architecture") \ _ELF_DEFINE_EM(EM_ECOG16, 176, \ "Cyan Technology eCOG16 family") \ _ELF_DEFINE_EM(EM_CR16, 177, \ "National Semiconductor CompactRISC CR16 16-bit microprocessor") \ _ELF_DEFINE_EM(EM_ETPU, 178, \ "Freescale Extended Time Processing Unit") \ _ELF_DEFINE_EM(EM_SLE9X, 179, \ "Infineon Technologies SLE9X core") \ _ELF_DEFINE_EM(EM_AARCH64, 183, \ "AArch64 (64-bit ARM)") \ _ELF_DEFINE_EM(EM_AVR32, 185, \ "Atmel Corporation 32-bit microprocessor family") \ _ELF_DEFINE_EM(EM_STM8, 186, \ "STMicroeletronics STM8 8-bit microcontroller") \ _ELF_DEFINE_EM(EM_TILE64, 187, \ "Tilera TILE64 multicore architecture family") \ _ELF_DEFINE_EM(EM_TILEPRO, 188, \ "Tilera TILEPro multicore architecture family") \ _ELF_DEFINE_EM(EM_MICROBLAZE, 189, \ "Xilinx MicroBlaze 32-bit RISC soft processor core") \ _ELF_DEFINE_EM(EM_CUDA, 190, "NVIDIA CUDA architecture") \ _ELF_DEFINE_EM(EM_TILEGX, 191, \ "Tilera TILE-Gx multicore architecture family") \ _ELF_DEFINE_EM(EM_CLOUDSHIELD, 192, \ "CloudShield architecture family") \ _ELF_DEFINE_EM(EM_COREA_1ST, 193, \ "KIPO-KAIST Core-A 1st generation processor family") \ _ELF_DEFINE_EM(EM_COREA_2ND, 194, \ "KIPO-KAIST Core-A 2nd generation processor family") \ _ELF_DEFINE_EM(EM_ARC_COMPACT2, 195, "Synopsys ARCompact V2") \ _ELF_DEFINE_EM(EM_OPEN8, 196, \ "Open8 8-bit RISC soft processor core") \ _ELF_DEFINE_EM(EM_RL78, 197, "Renesas RL78 family") \ _ELF_DEFINE_EM(EM_VIDEOCORE5, 198, "Broadcom VideoCore V processor") \ _ELF_DEFINE_EM(EM_78KOR, 199, "Renesas 78KOR family") \ _ELF_DEFINE_EM(EM_56800EX, 200, \ "Freescale 56800EX Digital Signal Controller") \ _ELF_DEFINE_EM(EM_BA1, 201, "Beyond BA1 CPU architecture") \ _ELF_DEFINE_EM(EM_BA2, 202, "Beyond BA2 CPU architecture") \ _ELF_DEFINE_EM(EM_XCORE, 203, "XMOS xCORE processor family") \ _ELF_DEFINE_EM(EM_MCHP_PIC, 204, "Microchip 8-bit PIC(r) family") \ _ELF_DEFINE_EM(EM_INTEL205, 205, "Reserved by Intel") \ _ELF_DEFINE_EM(EM_INTEL206, 206, "Reserved by Intel") \ _ELF_DEFINE_EM(EM_INTEL207, 207, "Reserved by Intel") \ _ELF_DEFINE_EM(EM_INTEL208, 208, "Reserved by Intel") \ _ELF_DEFINE_EM(EM_INTEL209, 209, "Reserved by Intel") \ _ELF_DEFINE_EM(EM_KM32, 210, "KM211 KM32 32-bit processor") \ _ELF_DEFINE_EM(EM_KMX32, 211, "KM211 KMX32 32-bit processor") \ _ELF_DEFINE_EM(EM_KMX16, 212, "KM211 KMX16 16-bit processor") \ _ELF_DEFINE_EM(EM_KMX8, 213, "KM211 KMX8 8-bit processor") \ _ELF_DEFINE_EM(EM_KVARC, 214, "KM211 KMX32 KVARC processor") \ _ELF_DEFINE_EM(EM_RISCV, 243, "RISC-V") #undef _ELF_DEFINE_EM #define _ELF_DEFINE_EM(N, V, DESCR) N = V , enum { _ELF_DEFINE_ELF_MACHINES() EM__LAST__ }; /* Other synonyms. */ #define EM_AMD64 EM_X86_64 #define EM_ARC_A5 EM_ARC_COMPACT /* * ELF file types: (ET_*). */ #define _ELF_DEFINE_ELF_TYPES() \ _ELF_DEFINE_ET(ET_NONE, 0, "No file type") \ _ELF_DEFINE_ET(ET_REL, 1, "Relocatable object") \ _ELF_DEFINE_ET(ET_EXEC, 2, "Executable") \ _ELF_DEFINE_ET(ET_DYN, 3, "Shared object") \ _ELF_DEFINE_ET(ET_CORE, 4, "Core file") \ _ELF_DEFINE_ET(ET_LOOS, 0xFE00U, "Begin OS-specific range") \ _ELF_DEFINE_ET(ET_HIOS, 0xFEFFU, "End OS-specific range") \ _ELF_DEFINE_ET(ET_LOPROC, 0xFF00U, "Begin processor-specific range") \ _ELF_DEFINE_ET(ET_HIPROC, 0xFFFFU, "End processor-specific range") #undef _ELF_DEFINE_ET #define _ELF_DEFINE_ET(N, V, DESCR) N = V , enum { _ELF_DEFINE_ELF_TYPES() ET__LAST__ }; /* ELF file format version numbers. */ #define EV_NONE 0 #define EV_CURRENT 1 /* * Flags for section groups. */ #define GRP_COMDAT 0x1 /* COMDAT semantics */ #define GRP_MASKOS 0x0ff00000 /* OS-specific flags */ #define GRP_MASKPROC 0xf0000000 /* processor-specific flags */ /* * Flags / mask for .gnu.versym sections. */ #define VERSYM_VERSION 0x7fff #define VERSYM_HIDDEN 0x8000 /* * Flags used by program header table entries. */ #define _ELF_DEFINE_PHDR_FLAGS() \ _ELF_DEFINE_PF(PF_X, 0x1, "Execute") \ _ELF_DEFINE_PF(PF_W, 0x2, "Write") \ _ELF_DEFINE_PF(PF_R, 0x4, "Read") \ _ELF_DEFINE_PF(PF_MASKOS, 0x0ff00000, "OS-specific flags") \ _ELF_DEFINE_PF(PF_MASKPROC, 0xf0000000, "Processor-specific flags") \ _ELF_DEFINE_PF(PF_ARM_SB, 0x10000000, \ "segment contains the location addressed by the static base") \ _ELF_DEFINE_PF(PF_ARM_PI, 0x20000000, \ "segment is position-independent") \ _ELF_DEFINE_PF(PF_ARM_ABS, 0x40000000, \ "segment must be loaded at its base address") #undef _ELF_DEFINE_PF #define _ELF_DEFINE_PF(N, V, DESCR) N = V , enum { _ELF_DEFINE_PHDR_FLAGS() PF__LAST__ }; /* * Types of program header table entries. */ #define _ELF_DEFINE_PHDR_TYPES() \ _ELF_DEFINE_PT(PT_NULL, 0, "ignored entry") \ _ELF_DEFINE_PT(PT_LOAD, 1, "loadable segment") \ _ELF_DEFINE_PT(PT_DYNAMIC, 2, \ "contains dynamic linking information") \ _ELF_DEFINE_PT(PT_INTERP, 3, "names an interpreter") \ _ELF_DEFINE_PT(PT_NOTE, 4, "auxiliary information") \ _ELF_DEFINE_PT(PT_SHLIB, 5, "reserved") \ _ELF_DEFINE_PT(PT_PHDR, 6, \ "describes the program header itself") \ _ELF_DEFINE_PT(PT_TLS, 7, "thread local storage") \ _ELF_DEFINE_PT(PT_LOOS, 0x60000000UL, \ "start of OS-specific range") \ _ELF_DEFINE_PT(PT_SUNW_UNWIND, 0x6464E550UL, \ "Solaris/amd64 stack unwind tables") \ _ELF_DEFINE_PT(PT_GNU_EH_FRAME, 0x6474E550UL, \ "GCC generated .eh_frame_hdr segment") \ _ELF_DEFINE_PT(PT_GNU_STACK, 0x6474E551UL, \ "Stack flags") \ _ELF_DEFINE_PT(PT_GNU_RELRO, 0x6474E552UL, \ "Segment becomes read-only after relocation") \ _ELF_DEFINE_PT(PT_SUNWBSS, 0x6FFFFFFAUL, \ "A Solaris .SUNW_bss section") \ _ELF_DEFINE_PT(PT_SUNWSTACK, 0x6FFFFFFBUL, \ "A Solaris process stack") \ _ELF_DEFINE_PT(PT_SUNWDTRACE, 0x6FFFFFFCUL, \ "Used by dtrace(1)") \ _ELF_DEFINE_PT(PT_SUNWCAP, 0x6FFFFFFDUL, \ "Special hardware capability requirements") \ _ELF_DEFINE_PT(PT_HIOS, 0x6FFFFFFFUL, \ "end of OS-specific range") \ _ELF_DEFINE_PT(PT_LOPROC, 0x70000000UL, \ "start of processor-specific range") \ _ELF_DEFINE_PT(PT_ARM_ARCHEXT, 0x70000000UL, \ "platform architecture compatibility information") \ _ELF_DEFINE_PT(PT_ARM_EXIDX, 0x70000001UL, \ "exception unwind tables") \ _ELF_DEFINE_PT(PT_MIPS_REGINFO, 0x70000000UL, \ "register usage information") \ _ELF_DEFINE_PT(PT_MIPS_RTPROC, 0x70000001UL, \ "runtime procedure table") \ _ELF_DEFINE_PT(PT_MIPS_OPTIONS, 0x70000002UL, \ "options segment") \ _ELF_DEFINE_PT(PT_HIPROC, 0x7FFFFFFFUL, \ "end of processor-specific range") #undef _ELF_DEFINE_PT #define _ELF_DEFINE_PT(N, V, DESCR) N = V , enum { _ELF_DEFINE_PHDR_TYPES() PT__LAST__ = PT_HIPROC }; /* synonyms. */ #define PT_ARM_UNWIND PT_ARM_EXIDX #define PT_HISUNW PT_HIOS #define PT_LOSUNW PT_SUNWBSS /* * Section flags. */ #define _ELF_DEFINE_SECTION_FLAGS() \ _ELF_DEFINE_SHF(SHF_WRITE, 0x1, \ "writable during program execution") \ _ELF_DEFINE_SHF(SHF_ALLOC, 0x2, \ "occupies memory during program execution") \ _ELF_DEFINE_SHF(SHF_EXECINSTR, 0x4, "executable instructions") \ _ELF_DEFINE_SHF(SHF_MERGE, 0x10, \ "may be merged to prevent duplication") \ _ELF_DEFINE_SHF(SHF_STRINGS, 0x20, \ "NUL-terminated character strings") \ _ELF_DEFINE_SHF(SHF_INFO_LINK, 0x40, \ "the sh_info field holds a link") \ _ELF_DEFINE_SHF(SHF_LINK_ORDER, 0x80, \ "special ordering requirements during linking") \ _ELF_DEFINE_SHF(SHF_OS_NONCONFORMING, 0x100, \ "requires OS-specific processing during linking") \ _ELF_DEFINE_SHF(SHF_GROUP, 0x200, \ "member of a section group") \ _ELF_DEFINE_SHF(SHF_TLS, 0x400, \ "holds thread-local storage") \ _ELF_DEFINE_SHF(SHF_COMPRESSED, 0x800, \ "holds compressed data") \ _ELF_DEFINE_SHF(SHF_MASKOS, 0x0FF00000UL, \ "bits reserved for OS-specific semantics") \ _ELF_DEFINE_SHF(SHF_AMD64_LARGE, 0x10000000UL, \ "section uses large code model") \ _ELF_DEFINE_SHF(SHF_ENTRYSECT, 0x10000000UL, \ "section contains an entry point (ARM)") \ _ELF_DEFINE_SHF(SHF_COMDEF, 0x80000000UL, \ "section may be multiply defined in input to link step (ARM)") \ _ELF_DEFINE_SHF(SHF_MIPS_GPREL, 0x10000000UL, \ "section must be part of global data area") \ _ELF_DEFINE_SHF(SHF_MIPS_MERGE, 0x20000000UL, \ "section data should be merged to eliminate duplication") \ _ELF_DEFINE_SHF(SHF_MIPS_ADDR, 0x40000000UL, \ "section data is addressed by default") \ _ELF_DEFINE_SHF(SHF_MIPS_STRING, 0x80000000UL, \ "section data is string data by default") \ _ELF_DEFINE_SHF(SHF_MIPS_NOSTRIP, 0x08000000UL, \ "section data may not be stripped") \ _ELF_DEFINE_SHF(SHF_MIPS_LOCAL, 0x04000000UL, \ "section data local to process") \ _ELF_DEFINE_SHF(SHF_MIPS_NAMES, 0x02000000UL, \ "linker must generate implicit hidden weak names") \ _ELF_DEFINE_SHF(SHF_MIPS_NODUPE, 0x01000000UL, \ "linker must retain only one copy") \ _ELF_DEFINE_SHF(SHF_ORDERED, 0x40000000UL, \ "section is ordered with respect to other sections") \ _ELF_DEFINE_SHF(SHF_EXCLUDE, 0x80000000UL, \ "section is excluded from executables and shared objects") \ _ELF_DEFINE_SHF(SHF_MASKPROC, 0xF0000000UL, \ "bits reserved for processor-specific semantics") #undef _ELF_DEFINE_SHF #define _ELF_DEFINE_SHF(N, V, DESCR) N = V , enum { _ELF_DEFINE_SECTION_FLAGS() SHF__LAST__ }; /* * Special section indices. */ #define _ELF_DEFINE_SECTION_INDICES() \ _ELF_DEFINE_SHN(SHN_UNDEF, 0, "undefined section") \ _ELF_DEFINE_SHN(SHN_LORESERVE, 0xFF00U, "start of reserved area") \ _ELF_DEFINE_SHN(SHN_LOPROC, 0xFF00U, \ "start of processor-specific range") \ _ELF_DEFINE_SHN(SHN_BEFORE, 0xFF00U, "used for section ordering") \ _ELF_DEFINE_SHN(SHN_AFTER, 0xFF01U, "used for section ordering") \ _ELF_DEFINE_SHN(SHN_AMD64_LCOMMON, 0xFF02U, "large common block label") \ _ELF_DEFINE_SHN(SHN_MIPS_ACOMMON, 0xFF00U, \ "allocated common symbols in a DSO") \ _ELF_DEFINE_SHN(SHN_MIPS_TEXT, 0xFF01U, "Reserved (obsolete)") \ _ELF_DEFINE_SHN(SHN_MIPS_DATA, 0xFF02U, "Reserved (obsolete)") \ _ELF_DEFINE_SHN(SHN_MIPS_SCOMMON, 0xFF03U, \ "gp-addressable common symbols") \ _ELF_DEFINE_SHN(SHN_MIPS_SUNDEFINED, 0xFF04U, \ "gp-addressable undefined symbols") \ _ELF_DEFINE_SHN(SHN_MIPS_LCOMMON, 0xFF05U, "local common symbols") \ _ELF_DEFINE_SHN(SHN_MIPS_LUNDEFINED, 0xFF06U, \ "local undefined symbols") \ _ELF_DEFINE_SHN(SHN_HIPROC, 0xFF1FU, \ "end of processor-specific range") \ _ELF_DEFINE_SHN(SHN_LOOS, 0xFF20U, \ "start of OS-specific range") \ _ELF_DEFINE_SHN(SHN_SUNW_IGNORE, 0xFF3FU, "used by dtrace") \ _ELF_DEFINE_SHN(SHN_HIOS, 0xFF3FU, \ "end of OS-specific range") \ _ELF_DEFINE_SHN(SHN_ABS, 0xFFF1U, "absolute references") \ _ELF_DEFINE_SHN(SHN_COMMON, 0xFFF2U, "references to COMMON areas") \ _ELF_DEFINE_SHN(SHN_XINDEX, 0xFFFFU, "extended index") \ _ELF_DEFINE_SHN(SHN_HIRESERVE, 0xFFFFU, "end of reserved area") #undef _ELF_DEFINE_SHN #define _ELF_DEFINE_SHN(N, V, DESCR) N = V , enum { _ELF_DEFINE_SECTION_INDICES() SHN__LAST__ }; /* * Section types. */ #define _ELF_DEFINE_SECTION_TYPES() \ _ELF_DEFINE_SHT(SHT_NULL, 0, "inactive header") \ _ELF_DEFINE_SHT(SHT_PROGBITS, 1, "program defined information") \ _ELF_DEFINE_SHT(SHT_SYMTAB, 2, "symbol table") \ _ELF_DEFINE_SHT(SHT_STRTAB, 3, "string table") \ _ELF_DEFINE_SHT(SHT_RELA, 4, \ "relocation entries with addends") \ _ELF_DEFINE_SHT(SHT_HASH, 5, "symbol hash table") \ _ELF_DEFINE_SHT(SHT_DYNAMIC, 6, \ "information for dynamic linking") \ _ELF_DEFINE_SHT(SHT_NOTE, 7, "additional notes") \ _ELF_DEFINE_SHT(SHT_NOBITS, 8, "section occupying no space") \ _ELF_DEFINE_SHT(SHT_REL, 9, \ "relocation entries without addends") \ _ELF_DEFINE_SHT(SHT_SHLIB, 10, "reserved") \ _ELF_DEFINE_SHT(SHT_DYNSYM, 11, "symbol table") \ _ELF_DEFINE_SHT(SHT_INIT_ARRAY, 14, \ "pointers to initialization functions") \ _ELF_DEFINE_SHT(SHT_FINI_ARRAY, 15, \ "pointers to termination functions") \ _ELF_DEFINE_SHT(SHT_PREINIT_ARRAY, 16, \ "pointers to functions called before initialization") \ _ELF_DEFINE_SHT(SHT_GROUP, 17, "defines a section group") \ _ELF_DEFINE_SHT(SHT_SYMTAB_SHNDX, 18, \ "used for extended section numbering") \ _ELF_DEFINE_SHT(SHT_LOOS, 0x60000000UL, \ "start of OS-specific range") \ _ELF_DEFINE_SHT(SHT_SUNW_dof, 0x6FFFFFF4UL, \ "used by dtrace") \ _ELF_DEFINE_SHT(SHT_SUNW_cap, 0x6FFFFFF5UL, \ "capability requirements") \ _ELF_DEFINE_SHT(SHT_GNU_ATTRIBUTES, 0x6FFFFFF5UL, \ "object attributes") \ _ELF_DEFINE_SHT(SHT_SUNW_SIGNATURE, 0x6FFFFFF6UL, \ "module verification signature") \ _ELF_DEFINE_SHT(SHT_GNU_HASH, 0x6FFFFFF6UL, \ "GNU Hash sections") \ _ELF_DEFINE_SHT(SHT_GNU_LIBLIST, 0x6FFFFFF7UL, \ "List of libraries to be prelinked") \ _ELF_DEFINE_SHT(SHT_SUNW_ANNOTATE, 0x6FFFFFF7UL, \ "special section where unresolved references are allowed") \ _ELF_DEFINE_SHT(SHT_SUNW_DEBUGSTR, 0x6FFFFFF8UL, \ "debugging information") \ _ELF_DEFINE_SHT(SHT_CHECKSUM, 0x6FFFFFF8UL, \ "checksum for dynamic shared objects") \ _ELF_DEFINE_SHT(SHT_SUNW_DEBUG, 0x6FFFFFF9UL, \ "debugging information") \ _ELF_DEFINE_SHT(SHT_SUNW_move, 0x6FFFFFFAUL, \ "information to handle partially initialized symbols") \ _ELF_DEFINE_SHT(SHT_SUNW_COMDAT, 0x6FFFFFFBUL, \ "section supporting merging of multiple copies of data") \ _ELF_DEFINE_SHT(SHT_SUNW_syminfo, 0x6FFFFFFCUL, \ "additional symbol information") \ _ELF_DEFINE_SHT(SHT_SUNW_verdef, 0x6FFFFFFDUL, \ "symbol versioning information") \ _ELF_DEFINE_SHT(SHT_SUNW_verneed, 0x6FFFFFFEUL, \ "symbol versioning requirements") \ _ELF_DEFINE_SHT(SHT_SUNW_versym, 0x6FFFFFFFUL, \ "symbol versioning table") \ _ELF_DEFINE_SHT(SHT_HIOS, 0x6FFFFFFFUL, \ "end of OS-specific range") \ _ELF_DEFINE_SHT(SHT_LOPROC, 0x70000000UL, \ "start of processor-specific range") \ _ELF_DEFINE_SHT(SHT_ARM_EXIDX, 0x70000001UL, \ "exception index table") \ _ELF_DEFINE_SHT(SHT_ARM_PREEMPTMAP, 0x70000002UL, \ "BPABI DLL dynamic linking preemption map") \ _ELF_DEFINE_SHT(SHT_ARM_ATTRIBUTES, 0x70000003UL, \ "object file compatibility attributes") \ _ELF_DEFINE_SHT(SHT_ARM_DEBUGOVERLAY, 0x70000004UL, \ "overlay debug information") \ _ELF_DEFINE_SHT(SHT_ARM_OVERLAYSECTION, 0x70000005UL, \ "overlay debug information") \ _ELF_DEFINE_SHT(SHT_MIPS_LIBLIST, 0x70000000UL, \ "DSO library information used in link") \ _ELF_DEFINE_SHT(SHT_MIPS_MSYM, 0x70000001UL, \ "MIPS symbol table extension") \ _ELF_DEFINE_SHT(SHT_MIPS_CONFLICT, 0x70000002UL, \ "symbol conflicting with DSO-defined symbols ") \ _ELF_DEFINE_SHT(SHT_MIPS_GPTAB, 0x70000003UL, \ "global pointer table") \ _ELF_DEFINE_SHT(SHT_MIPS_UCODE, 0x70000004UL, \ "reserved") \ _ELF_DEFINE_SHT(SHT_MIPS_DEBUG, 0x70000005UL, \ "reserved (obsolete debug information)") \ _ELF_DEFINE_SHT(SHT_MIPS_REGINFO, 0x70000006UL, \ "register usage information") \ _ELF_DEFINE_SHT(SHT_MIPS_PACKAGE, 0x70000007UL, \ "OSF reserved") \ _ELF_DEFINE_SHT(SHT_MIPS_PACKSYM, 0x70000008UL, \ "OSF reserved") \ _ELF_DEFINE_SHT(SHT_MIPS_RELD, 0x70000009UL, \ "dynamic relocation") \ _ELF_DEFINE_SHT(SHT_MIPS_IFACE, 0x7000000BUL, \ "subprogram interface information") \ _ELF_DEFINE_SHT(SHT_MIPS_CONTENT, 0x7000000CUL, \ "section content classification") \ _ELF_DEFINE_SHT(SHT_MIPS_OPTIONS, 0x7000000DUL, \ "general options") \ _ELF_DEFINE_SHT(SHT_MIPS_DELTASYM, 0x7000001BUL, \ "Delta C++: symbol table") \ _ELF_DEFINE_SHT(SHT_MIPS_DELTAINST, 0x7000001CUL, \ "Delta C++: instance table") \ _ELF_DEFINE_SHT(SHT_MIPS_DELTACLASS, 0x7000001DUL, \ "Delta C++: class table") \ _ELF_DEFINE_SHT(SHT_MIPS_DWARF, 0x7000001EUL, \ "DWARF debug information") \ _ELF_DEFINE_SHT(SHT_MIPS_DELTADECL, 0x7000001FUL, \ "Delta C++: declarations") \ _ELF_DEFINE_SHT(SHT_MIPS_SYMBOL_LIB, 0x70000020UL, \ "symbol-to-library mapping") \ _ELF_DEFINE_SHT(SHT_MIPS_EVENTS, 0x70000021UL, \ "event locations") \ _ELF_DEFINE_SHT(SHT_MIPS_TRANSLATE, 0x70000022UL, \ "???") \ _ELF_DEFINE_SHT(SHT_MIPS_PIXIE, 0x70000023UL, \ "special pixie sections") \ _ELF_DEFINE_SHT(SHT_MIPS_XLATE, 0x70000024UL, \ "address translation table") \ _ELF_DEFINE_SHT(SHT_MIPS_XLATE_DEBUG, 0x70000025UL, \ "SGI internal address translation table") \ _ELF_DEFINE_SHT(SHT_MIPS_WHIRL, 0x70000026UL, \ "intermediate code") \ _ELF_DEFINE_SHT(SHT_MIPS_EH_REGION, 0x70000027UL, \ "C++ exception handling region info") \ _ELF_DEFINE_SHT(SHT_MIPS_XLATE_OLD, 0x70000028UL, \ "obsolete") \ _ELF_DEFINE_SHT(SHT_MIPS_PDR_EXCEPTION, 0x70000029UL, \ "runtime procedure descriptor table exception information") \ _ELF_DEFINE_SHT(SHT_MIPS_ABIFLAGS, 0x7000002AUL, \ "ABI flags") \ _ELF_DEFINE_SHT(SHT_SPARC_GOTDATA, 0x70000000UL, \ "SPARC-specific data") \ _ELF_DEFINE_SHT(SHT_X86_64_UNWIND, 0x70000001UL, \ "unwind tables for the AMD64") \ _ELF_DEFINE_SHT(SHT_ORDERED, 0x7FFFFFFFUL, \ "sort entries in the section") \ _ELF_DEFINE_SHT(SHT_HIPROC, 0x7FFFFFFFUL, \ "end of processor-specific range") \ _ELF_DEFINE_SHT(SHT_LOUSER, 0x80000000UL, \ "start of application-specific range") \ _ELF_DEFINE_SHT(SHT_HIUSER, 0xFFFFFFFFUL, \ "end of application-specific range") #undef _ELF_DEFINE_SHT #define _ELF_DEFINE_SHT(N, V, DESCR) N = V , enum { _ELF_DEFINE_SECTION_TYPES() SHT__LAST__ = SHT_HIUSER }; /* Aliases for section types. */ #define SHT_AMD64_UNWIND SHT_X86_64_UNWIND #define SHT_GNU_verdef SHT_SUNW_verdef #define SHT_GNU_verneed SHT_SUNW_verneed #define SHT_GNU_versym SHT_SUNW_versym /* * Symbol binding information. */ #define _ELF_DEFINE_SYMBOL_BINDING() \ _ELF_DEFINE_STB(STB_LOCAL, 0, \ "not visible outside defining object file") \ _ELF_DEFINE_STB(STB_GLOBAL, 1, \ "visible across all object files being combined") \ _ELF_DEFINE_STB(STB_WEAK, 2, \ "visible across all object files but with low precedence") \ _ELF_DEFINE_STB(STB_LOOS, 10, "start of OS-specific range") \ _ELF_DEFINE_STB(STB_GNU_UNIQUE, 10, "unique symbol (GNU)") \ _ELF_DEFINE_STB(STB_HIOS, 12, "end of OS-specific range") \ _ELF_DEFINE_STB(STB_LOPROC, 13, \ "start of processor-specific range") \ _ELF_DEFINE_STB(STB_HIPROC, 15, \ "end of processor-specific range") #undef _ELF_DEFINE_STB #define _ELF_DEFINE_STB(N, V, DESCR) N = V , enum { _ELF_DEFINE_SYMBOL_BINDING() STB__LAST__ }; /* * Symbol types */ #define _ELF_DEFINE_SYMBOL_TYPES() \ _ELF_DEFINE_STT(STT_NOTYPE, 0, "unspecified type") \ _ELF_DEFINE_STT(STT_OBJECT, 1, "data object") \ _ELF_DEFINE_STT(STT_FUNC, 2, "executable code") \ _ELF_DEFINE_STT(STT_SECTION, 3, "section") \ _ELF_DEFINE_STT(STT_FILE, 4, "source file") \ _ELF_DEFINE_STT(STT_COMMON, 5, "uninitialized common block") \ _ELF_DEFINE_STT(STT_TLS, 6, "thread local storage") \ _ELF_DEFINE_STT(STT_LOOS, 10, "start of OS-specific types") \ _ELF_DEFINE_STT(STT_GNU_IFUNC, 10, "indirect function") \ _ELF_DEFINE_STT(STT_HIOS, 12, "end of OS-specific types") \ _ELF_DEFINE_STT(STT_LOPROC, 13, \ "start of processor-specific types") \ _ELF_DEFINE_STT(STT_ARM_TFUNC, 13, "Thumb function (GNU)") \ _ELF_DEFINE_STT(STT_ARM_16BIT, 15, "Thumb label (GNU)") \ _ELF_DEFINE_STT(STT_SPARC_REGISTER, 13, "SPARC register information") \ _ELF_DEFINE_STT(STT_HIPROC, 15, \ "end of processor-specific types") #undef _ELF_DEFINE_STT #define _ELF_DEFINE_STT(N, V, DESCR) N = V , enum { _ELF_DEFINE_SYMBOL_TYPES() STT__LAST__ }; /* * Symbol binding. */ #define _ELF_DEFINE_SYMBOL_BINDING_KINDS() \ _ELF_DEFINE_SYB(SYMINFO_BT_SELF, 0xFFFFU, \ "bound to self") \ _ELF_DEFINE_SYB(SYMINFO_BT_PARENT, 0xFFFEU, \ "bound to parent") \ _ELF_DEFINE_SYB(SYMINFO_BT_NONE, 0xFFFDU, \ "no special binding") #undef _ELF_DEFINE_SYB #define _ELF_DEFINE_SYB(N, V, DESCR) N = V , enum { _ELF_DEFINE_SYMBOL_BINDING_KINDS() SYMINFO__LAST__ }; /* * Symbol visibility. */ #define _ELF_DEFINE_SYMBOL_VISIBILITY() \ _ELF_DEFINE_STV(STV_DEFAULT, 0, \ "as specified by symbol type") \ _ELF_DEFINE_STV(STV_INTERNAL, 1, \ "as defined by processor semantics") \ _ELF_DEFINE_STV(STV_HIDDEN, 2, \ "hidden from other components") \ _ELF_DEFINE_STV(STV_PROTECTED, 3, \ "local references are not preemptable") #undef _ELF_DEFINE_STV #define _ELF_DEFINE_STV(N, V, DESCR) N = V , enum { _ELF_DEFINE_SYMBOL_VISIBILITY() STV__LAST__ }; /* * Symbol flags. */ #define _ELF_DEFINE_SYMBOL_FLAGS() \ _ELF_DEFINE_SYF(SYMINFO_FLG_DIRECT, 0x01, \ "directly assocated reference") \ _ELF_DEFINE_SYF(SYMINFO_FLG_COPY, 0x04, \ "definition by copy-relocation") \ _ELF_DEFINE_SYF(SYMINFO_FLG_LAZYLOAD, 0x08, \ "object should be lazily loaded") \ _ELF_DEFINE_SYF(SYMINFO_FLG_DIRECTBIND, 0x10, \ "reference should be directly bound") \ _ELF_DEFINE_SYF(SYMINFO_FLG_NOEXTDIRECT, 0x20, \ "external references not allowed to bind to definition") #undef _ELF_DEFINE_SYF #define _ELF_DEFINE_SYF(N, V, DESCR) N = V , enum { _ELF_DEFINE_SYMBOL_FLAGS() SYMINFO_FLG__LAST__ }; /* * Version dependencies. */ #define _ELF_DEFINE_VERSIONING_DEPENDENCIES() \ _ELF_DEFINE_VERD(VER_NDX_LOCAL, 0, "local scope") \ _ELF_DEFINE_VERD(VER_NDX_GLOBAL, 1, "global scope") #undef _ELF_DEFINE_VERD #define _ELF_DEFINE_VERD(N, V, DESCR) N = V , enum { _ELF_DEFINE_VERSIONING_DEPENDENCIES() VER_NDX__LAST__ }; /* * Version flags. */ #define _ELF_DEFINE_VERSIONING_FLAGS() \ _ELF_DEFINE_VERF(VER_FLG_BASE, 0x1, "file version") \ _ELF_DEFINE_VERF(VER_FLG_WEAK, 0x2, "weak version") #undef _ELF_DEFINE_VERF #define _ELF_DEFINE_VERF(N, V, DESCR) N = V , enum { _ELF_DEFINE_VERSIONING_FLAGS() VER_FLG__LAST__ }; /* * Version needs */ #define _ELF_DEFINE_VERSIONING_NEEDS() \ _ELF_DEFINE_VRN(VER_NEED_NONE, 0, "invalid version") \ _ELF_DEFINE_VRN(VER_NEED_CURRENT, 1, "current version") #undef _ELF_DEFINE_VRN #define _ELF_DEFINE_VRN(N, V, DESCR) N = V , enum { _ELF_DEFINE_VERSIONING_NEEDS() VER_NEED__LAST__ }; /* * Version numbers. */ #define _ELF_DEFINE_VERSIONING_NUMBERS() \ _ELF_DEFINE_VRNU(VER_DEF_NONE, 0, "invalid version") \ _ELF_DEFINE_VRNU(VER_DEF_CURRENT, 1, "current version") #undef _ELF_DEFINE_VRNU #define _ELF_DEFINE_VRNU(N, V, DESCR) N = V , enum { _ELF_DEFINE_VERSIONING_NUMBERS() VER_DEF__LAST__ }; /** ** Relocation types. **/ #define _ELF_DEFINE_386_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_386_NONE, 0) \ _ELF_DEFINE_RELOC(R_386_32, 1) \ _ELF_DEFINE_RELOC(R_386_PC32, 2) \ _ELF_DEFINE_RELOC(R_386_GOT32, 3) \ _ELF_DEFINE_RELOC(R_386_PLT32, 4) \ _ELF_DEFINE_RELOC(R_386_COPY, 5) \ _ELF_DEFINE_RELOC(R_386_GLOB_DAT, 6) \ _ELF_DEFINE_RELOC(R_386_JUMP_SLOT, 7) \ _ELF_DEFINE_RELOC(R_386_RELATIVE, 8) \ _ELF_DEFINE_RELOC(R_386_GOTOFF, 9) \ _ELF_DEFINE_RELOC(R_386_GOTPC, 10) \ _ELF_DEFINE_RELOC(R_386_32PLT, 11) \ _ELF_DEFINE_RELOC(R_386_TLS_TPOFF, 14) \ _ELF_DEFINE_RELOC(R_386_TLS_IE, 15) \ _ELF_DEFINE_RELOC(R_386_TLS_GOTIE, 16) \ _ELF_DEFINE_RELOC(R_386_TLS_LE, 17) \ _ELF_DEFINE_RELOC(R_386_TLS_GD, 18) \ _ELF_DEFINE_RELOC(R_386_TLS_LDM, 19) \ _ELF_DEFINE_RELOC(R_386_16, 20) \ _ELF_DEFINE_RELOC(R_386_PC16, 21) \ _ELF_DEFINE_RELOC(R_386_8, 22) \ _ELF_DEFINE_RELOC(R_386_PC8, 23) \ _ELF_DEFINE_RELOC(R_386_TLS_GD_32, 24) \ _ELF_DEFINE_RELOC(R_386_TLS_GD_PUSH, 25) \ _ELF_DEFINE_RELOC(R_386_TLS_GD_CALL, 26) \ _ELF_DEFINE_RELOC(R_386_TLS_GD_POP, 27) \ _ELF_DEFINE_RELOC(R_386_TLS_LDM_32, 28) \ _ELF_DEFINE_RELOC(R_386_TLS_LDM_PUSH, 29) \ _ELF_DEFINE_RELOC(R_386_TLS_LDM_CALL, 30) \ _ELF_DEFINE_RELOC(R_386_TLS_LDM_POP, 31) \ _ELF_DEFINE_RELOC(R_386_TLS_LDO_32, 32) \ _ELF_DEFINE_RELOC(R_386_TLS_IE_32, 33) \ _ELF_DEFINE_RELOC(R_386_TLS_LE_32, 34) \ _ELF_DEFINE_RELOC(R_386_TLS_DTPMOD32, 35) \ _ELF_DEFINE_RELOC(R_386_TLS_DTPOFF32, 36) \ _ELF_DEFINE_RELOC(R_386_TLS_TPOFF32, 37) \ _ELF_DEFINE_RELOC(R_386_SIZE32, 38) \ _ELF_DEFINE_RELOC(R_386_TLS_GOTDESC, 39) \ _ELF_DEFINE_RELOC(R_386_TLS_DESC_CALL, 40) \ _ELF_DEFINE_RELOC(R_386_TLS_DESC, 41) \ _ELF_DEFINE_RELOC(R_386_IRELATIVE, 42) \ _ELF_DEFINE_RELOC(R_386_GOT32X, 43) /* */ #define _ELF_DEFINE_AARCH64_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_AARCH64_NONE, 0) \ _ELF_DEFINE_RELOC(R_AARCH64_ABS64, 257) \ _ELF_DEFINE_RELOC(R_AARCH64_ABS32, 258) \ _ELF_DEFINE_RELOC(R_AARCH64_ABS16, 259) \ _ELF_DEFINE_RELOC(R_AARCH64_PREL64, 260) \ _ELF_DEFINE_RELOC(R_AARCH64_PREL32, 261) \ _ELF_DEFINE_RELOC(R_AARCH64_PREL16, 262) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_UABS_G0, 263) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_UABS_G0_NC, 264) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_UABS_G1, 265) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_UABS_G1_NC, 266) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_UABS_G2, 267) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_UABS_G2_NC, 268) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_UABS_G3, 269) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_SABS_G0, 270) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_SABS_G1, 271) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_SABS_G2, 272) \ _ELF_DEFINE_RELOC(R_AARCH64_LD_PREL_LO19, 273) \ _ELF_DEFINE_RELOC(R_AARCH64_ADR_PREL_LO21, 274) \ _ELF_DEFINE_RELOC(R_AARCH64_ADR_PREL_PG_HI21, 275) \ _ELF_DEFINE_RELOC(R_AARCH64_ADR_PREL_PG_HI21_NC, 276) \ _ELF_DEFINE_RELOC(R_AARCH64_ADD_ABS_LO12_NC, 277) \ _ELF_DEFINE_RELOC(R_AARCH64_LDST8_ABS_LO12_NC, 278) \ _ELF_DEFINE_RELOC(R_AARCH64_TSTBR14, 279) \ _ELF_DEFINE_RELOC(R_AARCH64_CONDBR19, 280) \ _ELF_DEFINE_RELOC(R_AARCH64_JUMP26, 282) \ _ELF_DEFINE_RELOC(R_AARCH64_CALL26, 283) \ _ELF_DEFINE_RELOC(R_AARCH64_LDST16_ABS_LO12_NC, 284) \ _ELF_DEFINE_RELOC(R_AARCH64_LDST32_ABS_LO12_NC, 285) \ _ELF_DEFINE_RELOC(R_AARCH64_LDST64_ABS_LO12_NC, 286) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_PREL_G0, 287) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_PREL_G0_NC, 288) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_PREL_G1, 289) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_PREL_G1_NC, 290) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_PREL_G2, 291) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_PREL_G2_NC, 292) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_PREL_G3, 293) \ _ELF_DEFINE_RELOC(R_AARCH64_LDST128_ABS_LO12_NC, 299) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_GOTOFF_G0, 300) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_GOTOFF_G0_NC, 301) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_GOTOFF_G1, 302) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_GOTOFF_G1_NC, 303) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_GOTOFF_G2, 304) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_GOTOFF_G2_NC, 305) \ _ELF_DEFINE_RELOC(R_AARCH64_MOVW_GOTOFF_G3, 306) \ _ELF_DEFINE_RELOC(R_AARCH64_GOTREL64, 307) \ _ELF_DEFINE_RELOC(R_AARCH64_GOTREL32, 308) \ _ELF_DEFINE_RELOC(R_AARCH64_GOT_LD_PREL19, 309) \ _ELF_DEFINE_RELOC(R_AARCH64_LD64_GOTOFF_LO15, 310) \ _ELF_DEFINE_RELOC(R_AARCH64_ADR_GOT_PAGE, 311) \ _ELF_DEFINE_RELOC(R_AARCH64_LD64_GOT_LO12_NC, 312) \ _ELF_DEFINE_RELOC(R_AARCH64_LD64_GOTPAGE_LO15, 313) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSGD_ADR_PREL21, 512) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSGD_ADR_PAGE21, 513) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSGD_ADD_LO12_NC, 514) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSGD_MOVW_G1, 515) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSGD_MOVW_G0_NC, 516) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_ADR_PREL21, 517) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_ADR_PAGE21, 518) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_ADD_LO12_NC, 519) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_MOVW_G1, 520) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_MOVW_G0_NC, 521) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LD_PREL19, 522) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_MOVW_DTPREL_G2, 523) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_MOVW_DTPREL_G1, 524) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_MOVW_DTPREL_G1_NC, 525) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_MOVW_DTPREL_G0, 526) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_MOVW_DTPREL_G0_NC, 527) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_ADD_DTPREL_HI12, 529) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_ADD_DTPREL_LO12_NC, 530) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST8_DTPREL_LO12, 531) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST8_DTPREL_LO12_NC, 532) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST16_DTPREL_LO12, 533) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST16_DTPREL_LO12_NC, 534) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST32_DTPREL_LO12, 535) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST32_DTPREL_LO12_NC, 536) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST64_DTPREL_LO12, 537) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST64_DTPREL_LO12_NC, 538) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSIE_MOVW_GOTTPREL_G1, 539) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSIE_MOVW_GOTTPREL_G0_NC, 540) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21, 541) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC, 542) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSIE_LD_GOTTPREL_PREL19, 543) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_MOVW_TPREL_G2, 544) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_MOVW_TPREL_G1, 545) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_MOVW_TPREL_G1_NC, 546) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_MOVW_TPREL_G0, 547) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_MOVW_TPREL_G0_NC, 548) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_ADD_TPREL_HI12, 549) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_ADD_TPREL_LO12, 550) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_ADD_TPREL_LO12_NC, 551) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST8_TPREL_LO12, 552) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST8_TPREL_LO12_NC, 553) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST16_TPREL_LO12, 554) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST16_TPREL_LO12_NC, 555) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST32_TPREL_LO12, 556) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST32_TPREL_LO12_NC, 557) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST64_TPREL_LO12, 558) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST64_TPREL_LO12_NC, 559) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_LD_PREL19, 560) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_ADR_PREL21, 561) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_ADR_PAGE21, 562) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_LD64_LO12, 563) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_ADD_LO12, 564) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_OFF_G1, 565) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_OFF_G0_NC, 566) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_LDR, 567) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_ADD, 568) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC_CALL, 569) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST128_TPREL_LO12, 570) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLE_LDST128_TPREL_LO12_NC, 571) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST128_DTPREL_LO12, 572) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSLD_LDST128_DTPREL_LO12_NC, 573) \ _ELF_DEFINE_RELOC(R_AARCH64_COPY, 1024) \ _ELF_DEFINE_RELOC(R_AARCH64_GLOB_DAT, 1025) \ _ELF_DEFINE_RELOC(R_AARCH64_JUMP_SLOT, 1026) \ _ELF_DEFINE_RELOC(R_AARCH64_RELATIVE, 1027) \ _ELF_DEFINE_RELOC(R_AARCH64_TLS_DTPREL64, 1028) \ _ELF_DEFINE_RELOC(R_AARCH64_TLS_DTPMOD64, 1029) \ _ELF_DEFINE_RELOC(R_AARCH64_TLS_TPREL64, 1030) \ _ELF_DEFINE_RELOC(R_AARCH64_TLSDESC, 1031) \ _ELF_DEFINE_RELOC(R_AARCH64_IRELATIVE, 1032) /* * These are the symbols used in the Sun ``Linkers and Loaders * Guide'', Document No: 817-1984-17. See the X86_64 relocations list * below for the spellings used in the ELF specification. */ #define _ELF_DEFINE_AMD64_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_AMD64_NONE, 0) \ _ELF_DEFINE_RELOC(R_AMD64_64, 1) \ _ELF_DEFINE_RELOC(R_AMD64_PC32, 2) \ _ELF_DEFINE_RELOC(R_AMD64_GOT32, 3) \ _ELF_DEFINE_RELOC(R_AMD64_PLT32, 4) \ _ELF_DEFINE_RELOC(R_AMD64_COPY, 5) \ _ELF_DEFINE_RELOC(R_AMD64_GLOB_DAT, 6) \ _ELF_DEFINE_RELOC(R_AMD64_JUMP_SLOT, 7) \ _ELF_DEFINE_RELOC(R_AMD64_RELATIVE, 8) \ _ELF_DEFINE_RELOC(R_AMD64_GOTPCREL, 9) \ _ELF_DEFINE_RELOC(R_AMD64_32, 10) \ _ELF_DEFINE_RELOC(R_AMD64_32S, 11) \ _ELF_DEFINE_RELOC(R_AMD64_16, 12) \ _ELF_DEFINE_RELOC(R_AMD64_PC16, 13) \ _ELF_DEFINE_RELOC(R_AMD64_8, 14) \ _ELF_DEFINE_RELOC(R_AMD64_PC8, 15) \ _ELF_DEFINE_RELOC(R_AMD64_PC64, 24) \ _ELF_DEFINE_RELOC(R_AMD64_GOTOFF64, 25) \ _ELF_DEFINE_RELOC(R_AMD64_GOTPC32, 26) /* * Relocation definitions from the ARM ELF ABI, version "ARM IHI * 0044E" released on 30th November 2012. */ #define _ELF_DEFINE_ARM_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_ARM_NONE, 0) \ _ELF_DEFINE_RELOC(R_ARM_PC24, 1) \ _ELF_DEFINE_RELOC(R_ARM_ABS32, 2) \ _ELF_DEFINE_RELOC(R_ARM_REL32, 3) \ _ELF_DEFINE_RELOC(R_ARM_LDR_PC_G0, 4) \ _ELF_DEFINE_RELOC(R_ARM_ABS16, 5) \ _ELF_DEFINE_RELOC(R_ARM_ABS12, 6) \ _ELF_DEFINE_RELOC(R_ARM_THM_ABS5, 7) \ _ELF_DEFINE_RELOC(R_ARM_ABS8, 8) \ _ELF_DEFINE_RELOC(R_ARM_SBREL32, 9) \ _ELF_DEFINE_RELOC(R_ARM_THM_CALL, 10) \ _ELF_DEFINE_RELOC(R_ARM_THM_PC8, 11) \ _ELF_DEFINE_RELOC(R_ARM_BREL_ADJ, 12) \ _ELF_DEFINE_RELOC(R_ARM_SWI24, 13) \ _ELF_DEFINE_RELOC(R_ARM_TLS_DESC, 13) \ _ELF_DEFINE_RELOC(R_ARM_THM_SWI8, 14) \ _ELF_DEFINE_RELOC(R_ARM_XPC25, 15) \ _ELF_DEFINE_RELOC(R_ARM_THM_XPC22, 16) \ _ELF_DEFINE_RELOC(R_ARM_TLS_DTPMOD32, 17) \ _ELF_DEFINE_RELOC(R_ARM_TLS_DTPOFF32, 18) \ _ELF_DEFINE_RELOC(R_ARM_TLS_TPOFF32, 19) \ _ELF_DEFINE_RELOC(R_ARM_COPY, 20) \ _ELF_DEFINE_RELOC(R_ARM_GLOB_DAT, 21) \ _ELF_DEFINE_RELOC(R_ARM_JUMP_SLOT, 22) \ _ELF_DEFINE_RELOC(R_ARM_RELATIVE, 23) \ _ELF_DEFINE_RELOC(R_ARM_GOTOFF32, 24) \ _ELF_DEFINE_RELOC(R_ARM_BASE_PREL, 25) \ _ELF_DEFINE_RELOC(R_ARM_GOT_BREL, 26) \ _ELF_DEFINE_RELOC(R_ARM_PLT32, 27) \ _ELF_DEFINE_RELOC(R_ARM_CALL, 28) \ _ELF_DEFINE_RELOC(R_ARM_JUMP24, 29) \ _ELF_DEFINE_RELOC(R_ARM_THM_JUMP24, 30) \ _ELF_DEFINE_RELOC(R_ARM_BASE_ABS, 31) \ _ELF_DEFINE_RELOC(R_ARM_ALU_PCREL_7_0, 32) \ _ELF_DEFINE_RELOC(R_ARM_ALU_PCREL_15_8, 33) \ _ELF_DEFINE_RELOC(R_ARM_ALU_PCREL_23_15, 34) \ _ELF_DEFINE_RELOC(R_ARM_LDR_SBREL_11_0_NC, 35) \ _ELF_DEFINE_RELOC(R_ARM_ALU_SBREL_19_12_NC, 36) \ _ELF_DEFINE_RELOC(R_ARM_ALU_SBREL_27_20_CK, 37) \ _ELF_DEFINE_RELOC(R_ARM_TARGET1, 38) \ _ELF_DEFINE_RELOC(R_ARM_SBREL31, 39) \ _ELF_DEFINE_RELOC(R_ARM_V4BX, 40) \ _ELF_DEFINE_RELOC(R_ARM_TARGET2, 41) \ _ELF_DEFINE_RELOC(R_ARM_PREL31, 42) \ _ELF_DEFINE_RELOC(R_ARM_MOVW_ABS_NC, 43) \ _ELF_DEFINE_RELOC(R_ARM_MOVT_ABS, 44) \ _ELF_DEFINE_RELOC(R_ARM_MOVW_PREL_NC, 45) \ _ELF_DEFINE_RELOC(R_ARM_MOVT_PREL, 46) \ _ELF_DEFINE_RELOC(R_ARM_THM_MOVW_ABS_NC, 47) \ _ELF_DEFINE_RELOC(R_ARM_THM_MOVT_ABS, 48) \ _ELF_DEFINE_RELOC(R_ARM_THM_MOVW_PREL_NC, 49) \ _ELF_DEFINE_RELOC(R_ARM_THM_MOVT_PREL, 50) \ _ELF_DEFINE_RELOC(R_ARM_THM_JUMP19, 51) \ _ELF_DEFINE_RELOC(R_ARM_THM_JUMP6, 52) \ _ELF_DEFINE_RELOC(R_ARM_THM_ALU_PREL_11_0, 53) \ _ELF_DEFINE_RELOC(R_ARM_THM_PC12, 54) \ _ELF_DEFINE_RELOC(R_ARM_ABS32_NOI, 55) \ _ELF_DEFINE_RELOC(R_ARM_REL32_NOI, 56) \ _ELF_DEFINE_RELOC(R_ARM_ALU_PC_G0_NC, 57) \ _ELF_DEFINE_RELOC(R_ARM_ALU_PC_G0, 58) \ _ELF_DEFINE_RELOC(R_ARM_ALU_PC_G1_NC, 59) \ _ELF_DEFINE_RELOC(R_ARM_ALU_PC_G1, 60) \ _ELF_DEFINE_RELOC(R_ARM_ALU_PC_G2, 61) \ _ELF_DEFINE_RELOC(R_ARM_LDR_PC_G1, 62) \ _ELF_DEFINE_RELOC(R_ARM_LDR_PC_G2, 63) \ _ELF_DEFINE_RELOC(R_ARM_LDRS_PC_G0, 64) \ _ELF_DEFINE_RELOC(R_ARM_LDRS_PC_G1, 65) \ _ELF_DEFINE_RELOC(R_ARM_LDRS_PC_G2, 66) \ _ELF_DEFINE_RELOC(R_ARM_LDC_PC_G0, 67) \ _ELF_DEFINE_RELOC(R_ARM_LDC_PC_G1, 68) \ _ELF_DEFINE_RELOC(R_ARM_LDC_PC_G2, 69) \ _ELF_DEFINE_RELOC(R_ARM_ALU_SB_G0_NC, 70) \ _ELF_DEFINE_RELOC(R_ARM_ALU_SB_G0, 71) \ _ELF_DEFINE_RELOC(R_ARM_ALU_SB_G1_NC, 72) \ _ELF_DEFINE_RELOC(R_ARM_ALU_SB_G1, 73) \ _ELF_DEFINE_RELOC(R_ARM_ALU_SB_G2, 74) \ _ELF_DEFINE_RELOC(R_ARM_LDR_SB_G0, 75) \ _ELF_DEFINE_RELOC(R_ARM_LDR_SB_G1, 76) \ _ELF_DEFINE_RELOC(R_ARM_LDR_SB_G2, 77) \ _ELF_DEFINE_RELOC(R_ARM_LDRS_SB_G0, 78) \ _ELF_DEFINE_RELOC(R_ARM_LDRS_SB_G1, 79) \ _ELF_DEFINE_RELOC(R_ARM_LDRS_SB_G2, 80) \ _ELF_DEFINE_RELOC(R_ARM_LDC_SB_G0, 81) \ _ELF_DEFINE_RELOC(R_ARM_LDC_SB_G1, 82) \ _ELF_DEFINE_RELOC(R_ARM_LDC_SB_G2, 83) \ _ELF_DEFINE_RELOC(R_ARM_MOVW_BREL_NC, 84) \ _ELF_DEFINE_RELOC(R_ARM_MOVT_BREL, 85) \ _ELF_DEFINE_RELOC(R_ARM_MOVW_BREL, 86) \ _ELF_DEFINE_RELOC(R_ARM_THM_MOVW_BREL_NC, 87) \ _ELF_DEFINE_RELOC(R_ARM_THM_MOVT_BREL, 88) \ _ELF_DEFINE_RELOC(R_ARM_THM_MOVW_BREL, 89) \ _ELF_DEFINE_RELOC(R_ARM_TLS_GOTDESC, 90) \ _ELF_DEFINE_RELOC(R_ARM_TLS_CALL, 91) \ _ELF_DEFINE_RELOC(R_ARM_TLS_DESCSEQ, 92) \ _ELF_DEFINE_RELOC(R_ARM_THM_TLS_CALL, 93) \ _ELF_DEFINE_RELOC(R_ARM_PLT32_ABS, 94) \ _ELF_DEFINE_RELOC(R_ARM_GOT_ABS, 95) \ _ELF_DEFINE_RELOC(R_ARM_GOT_PREL, 96) \ _ELF_DEFINE_RELOC(R_ARM_GOT_BREL12, 97) \ _ELF_DEFINE_RELOC(R_ARM_GOTOFF12, 98) \ _ELF_DEFINE_RELOC(R_ARM_GOTRELAX, 99) \ _ELF_DEFINE_RELOC(R_ARM_GNU_VTENTRY, 100) \ _ELF_DEFINE_RELOC(R_ARM_GNU_VTINHERIT, 101) \ _ELF_DEFINE_RELOC(R_ARM_THM_JUMP11, 102) \ _ELF_DEFINE_RELOC(R_ARM_THM_JUMP8, 103) \ _ELF_DEFINE_RELOC(R_ARM_TLS_GD32, 104) \ _ELF_DEFINE_RELOC(R_ARM_TLS_LDM32, 105) \ _ELF_DEFINE_RELOC(R_ARM_TLS_LDO32, 106) \ _ELF_DEFINE_RELOC(R_ARM_TLS_IE32, 107) \ _ELF_DEFINE_RELOC(R_ARM_TLS_LE32, 108) \ _ELF_DEFINE_RELOC(R_ARM_TLS_LDO12, 109) \ _ELF_DEFINE_RELOC(R_ARM_TLS_LE12, 110) \ _ELF_DEFINE_RELOC(R_ARM_TLS_IE12GP, 111) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_0, 112) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_1, 113) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_2, 114) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_3, 115) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_4, 116) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_5, 117) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_6, 118) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_7, 119) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_8, 120) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_9, 121) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_10, 122) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_11, 123) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_12, 124) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_13, 125) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_14, 126) \ _ELF_DEFINE_RELOC(R_ARM_PRIVATE_15, 127) \ _ELF_DEFINE_RELOC(R_ARM_ME_TOO, 128) \ _ELF_DEFINE_RELOC(R_ARM_THM_TLS_DESCSEQ16, 129) \ _ELF_DEFINE_RELOC(R_ARM_THM_TLS_DESCSEQ32, 130) \ _ELF_DEFINE_RELOC(R_ARM_THM_GOT_BREL12, 131) \ _ELF_DEFINE_RELOC(R_ARM_IRELATIVE, 140) #define _ELF_DEFINE_IA64_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_IA_64_NONE, 0) \ _ELF_DEFINE_RELOC(R_IA_64_IMM14, 0x21) \ _ELF_DEFINE_RELOC(R_IA_64_IMM22, 0x22) \ _ELF_DEFINE_RELOC(R_IA_64_IMM64, 0x23) \ _ELF_DEFINE_RELOC(R_IA_64_DIR32MSB, 0x24) \ _ELF_DEFINE_RELOC(R_IA_64_DIR32LSB, 0x25) \ _ELF_DEFINE_RELOC(R_IA_64_DIR64MSB, 0x26) \ _ELF_DEFINE_RELOC(R_IA_64_DIR64LSB, 0x27) \ _ELF_DEFINE_RELOC(R_IA_64_GPREL22, 0x2a) \ _ELF_DEFINE_RELOC(R_IA_64_GPREL64I, 0x2b) \ _ELF_DEFINE_RELOC(R_IA_64_GPREL32MSB, 0x2c) \ _ELF_DEFINE_RELOC(R_IA_64_GPREL32LSB, 0x2d) \ _ELF_DEFINE_RELOC(R_IA_64_GPREL64MSB, 0x2e) \ _ELF_DEFINE_RELOC(R_IA_64_GPREL64LSB, 0x2f) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF22, 0x32) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF64I, 0x33) \ _ELF_DEFINE_RELOC(R_IA_64_PLTOFF22, 0x3a) \ _ELF_DEFINE_RELOC(R_IA_64_PLTOFF64I, 0x3b) \ _ELF_DEFINE_RELOC(R_IA_64_PLTOFF64MSB, 0x3e) \ _ELF_DEFINE_RELOC(R_IA_64_PLTOFF64LSB, 0x3f) \ _ELF_DEFINE_RELOC(R_IA_64_FPTR64I, 0x43) \ _ELF_DEFINE_RELOC(R_IA_64_FPTR32MSB, 0x44) \ _ELF_DEFINE_RELOC(R_IA_64_FPTR32LSB, 0x45) \ _ELF_DEFINE_RELOC(R_IA_64_FPTR64MSB, 0x46) \ _ELF_DEFINE_RELOC(R_IA_64_FPTR64LSB, 0x47) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL60B, 0x48) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL21B, 0x49) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL21M, 0x4a) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL21F, 0x4b) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL32MSB, 0x4c) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL32LSB, 0x4d) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL64MSB, 0x4e) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL64LSB, 0x4f) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_FPTR22, 0x52) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_FPTR64I, 0x53) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_FPTR32MSB, 0x54) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_FPTR32LSB, 0x55) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_FPTR64MSB, 0x56) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_FPTR64LSB, 0x57) \ _ELF_DEFINE_RELOC(R_IA_64_SEGREL32MSB, 0x5c) \ _ELF_DEFINE_RELOC(R_IA_64_SEGREL32LSB, 0x5d) \ _ELF_DEFINE_RELOC(R_IA_64_SEGREL64MSB, 0x5e) \ _ELF_DEFINE_RELOC(R_IA_64_SEGREL64LSB, 0x5f) \ _ELF_DEFINE_RELOC(R_IA_64_SECREL32MSB, 0x64) \ _ELF_DEFINE_RELOC(R_IA_64_SECREL32LSB, 0x65) \ _ELF_DEFINE_RELOC(R_IA_64_SECREL64MSB, 0x66) \ _ELF_DEFINE_RELOC(R_IA_64_SECREL64LSB, 0x67) \ _ELF_DEFINE_RELOC(R_IA_64_REL32MSB, 0x6c) \ _ELF_DEFINE_RELOC(R_IA_64_REL32LSB, 0x6d) \ _ELF_DEFINE_RELOC(R_IA_64_REL64MSB, 0x6e) \ _ELF_DEFINE_RELOC(R_IA_64_REL64LSB, 0x6f) \ _ELF_DEFINE_RELOC(R_IA_64_LTV32MSB, 0x74) \ _ELF_DEFINE_RELOC(R_IA_64_LTV32LSB, 0x75) \ _ELF_DEFINE_RELOC(R_IA_64_LTV64MSB, 0x76) \ _ELF_DEFINE_RELOC(R_IA_64_LTV64LSB, 0x77) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL21BI, 0x79) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL22, 0x7A) \ _ELF_DEFINE_RELOC(R_IA_64_PCREL64I, 0x7B) \ _ELF_DEFINE_RELOC(R_IA_64_IPLTMSB, 0x80) \ _ELF_DEFINE_RELOC(R_IA_64_IPLTLSB, 0x81) \ _ELF_DEFINE_RELOC(R_IA_64_SUB, 0x85) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF22X, 0x86) \ _ELF_DEFINE_RELOC(R_IA_64_LDXMOV, 0x87) \ _ELF_DEFINE_RELOC(R_IA_64_TPREL14, 0x91) \ _ELF_DEFINE_RELOC(R_IA_64_TPREL22, 0x92) \ _ELF_DEFINE_RELOC(R_IA_64_TPREL64I, 0x93) \ _ELF_DEFINE_RELOC(R_IA_64_TPREL64MSB, 0x96) \ _ELF_DEFINE_RELOC(R_IA_64_TPREL64LSB, 0x97) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_TPREL22, 0x9A) \ _ELF_DEFINE_RELOC(R_IA_64_DTPMOD64MSB, 0xA6) \ _ELF_DEFINE_RELOC(R_IA_64_DTPMOD64LSB, 0xA7) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_DTPMOD22, 0xAA) \ _ELF_DEFINE_RELOC(R_IA_64_DTPREL14, 0xB1) \ _ELF_DEFINE_RELOC(R_IA_64_DTPREL22, 0xB2) \ _ELF_DEFINE_RELOC(R_IA_64_DTPREL64I, 0xB3) \ _ELF_DEFINE_RELOC(R_IA_64_DTPREL32MSB, 0xB4) \ _ELF_DEFINE_RELOC(R_IA_64_DTPREL32LSB, 0xB5) \ _ELF_DEFINE_RELOC(R_IA_64_DTPREL64MSB, 0xB6) \ _ELF_DEFINE_RELOC(R_IA_64_DTPREL64LSB, 0xB7) \ _ELF_DEFINE_RELOC(R_IA_64_LTOFF_DTPREL22, 0xBA) #define _ELF_DEFINE_MIPS_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_MIPS_NONE, 0) \ _ELF_DEFINE_RELOC(R_MIPS_16, 1) \ _ELF_DEFINE_RELOC(R_MIPS_32, 2) \ _ELF_DEFINE_RELOC(R_MIPS_REL32, 3) \ _ELF_DEFINE_RELOC(R_MIPS_26, 4) \ _ELF_DEFINE_RELOC(R_MIPS_HI16, 5) \ _ELF_DEFINE_RELOC(R_MIPS_LO16, 6) \ _ELF_DEFINE_RELOC(R_MIPS_GPREL16, 7) \ _ELF_DEFINE_RELOC(R_MIPS_LITERAL, 8) \ _ELF_DEFINE_RELOC(R_MIPS_GOT16, 9) \ _ELF_DEFINE_RELOC(R_MIPS_PC16, 10) \ _ELF_DEFINE_RELOC(R_MIPS_CALL16, 11) \ _ELF_DEFINE_RELOC(R_MIPS_GPREL32, 12) \ _ELF_DEFINE_RELOC(R_MIPS_SHIFT5, 16) \ _ELF_DEFINE_RELOC(R_MIPS_SHIFT6, 17) \ _ELF_DEFINE_RELOC(R_MIPS_64, 18) \ _ELF_DEFINE_RELOC(R_MIPS_GOT_DISP, 19) \ _ELF_DEFINE_RELOC(R_MIPS_GOT_PAGE, 20) \ _ELF_DEFINE_RELOC(R_MIPS_GOT_OFST, 21) \ _ELF_DEFINE_RELOC(R_MIPS_GOT_HI16, 22) \ _ELF_DEFINE_RELOC(R_MIPS_GOT_LO16, 23) \ _ELF_DEFINE_RELOC(R_MIPS_SUB, 24) \ _ELF_DEFINE_RELOC(R_MIPS_CALLHI16, 30) \ _ELF_DEFINE_RELOC(R_MIPS_CALLLO16, 31) \ _ELF_DEFINE_RELOC(R_MIPS_JALR, 37) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_DTPMOD32, 38) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_DTPREL32, 39) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_DTPMOD64, 40) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_DTPREL64, 41) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_GD, 42) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_LDM, 43) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_DTPREL_HI16, 44) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_DTPREL_LO16, 45) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_GOTTPREL, 46) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_TPREL32, 47) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_TPREL64, 48) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_TPREL_HI16, 49) \ _ELF_DEFINE_RELOC(R_MIPS_TLS_TPREL_LO16, 50) #define _ELF_DEFINE_PPC32_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_PPC_NONE, 0) \ _ELF_DEFINE_RELOC(R_PPC_ADDR32, 1) \ _ELF_DEFINE_RELOC(R_PPC_ADDR24, 2) \ _ELF_DEFINE_RELOC(R_PPC_ADDR16, 3) \ _ELF_DEFINE_RELOC(R_PPC_ADDR16_LO, 4) \ _ELF_DEFINE_RELOC(R_PPC_ADDR16_HI, 5) \ _ELF_DEFINE_RELOC(R_PPC_ADDR16_HA, 6) \ _ELF_DEFINE_RELOC(R_PPC_ADDR14, 7) \ _ELF_DEFINE_RELOC(R_PPC_ADDR14_BRTAKEN, 8) \ _ELF_DEFINE_RELOC(R_PPC_ADDR14_BRNTAKEN, 9) \ _ELF_DEFINE_RELOC(R_PPC_REL24, 10) \ _ELF_DEFINE_RELOC(R_PPC_REL14, 11) \ _ELF_DEFINE_RELOC(R_PPC_REL14_BRTAKEN, 12) \ _ELF_DEFINE_RELOC(R_PPC_REL14_BRNTAKEN, 13) \ _ELF_DEFINE_RELOC(R_PPC_GOT16, 14) \ _ELF_DEFINE_RELOC(R_PPC_GOT16_LO, 15) \ _ELF_DEFINE_RELOC(R_PPC_GOT16_HI, 16) \ _ELF_DEFINE_RELOC(R_PPC_GOT16_HA, 17) \ _ELF_DEFINE_RELOC(R_PPC_PLTREL24, 18) \ _ELF_DEFINE_RELOC(R_PPC_COPY, 19) \ _ELF_DEFINE_RELOC(R_PPC_GLOB_DAT, 20) \ _ELF_DEFINE_RELOC(R_PPC_JMP_SLOT, 21) \ _ELF_DEFINE_RELOC(R_PPC_RELATIVE, 22) \ _ELF_DEFINE_RELOC(R_PPC_LOCAL24PC, 23) \ _ELF_DEFINE_RELOC(R_PPC_UADDR32, 24) \ _ELF_DEFINE_RELOC(R_PPC_UADDR16, 25) \ _ELF_DEFINE_RELOC(R_PPC_REL32, 26) \ _ELF_DEFINE_RELOC(R_PPC_PLT32, 27) \ _ELF_DEFINE_RELOC(R_PPC_PLTREL32, 28) \ _ELF_DEFINE_RELOC(R_PPC_PLT16_LO, 29) \ _ELF_DEFINE_RELOC(R_PPC_PLT16_HI, 30) \ _ELF_DEFINE_RELOC(R_PPC_PLT16_HA, 31) \ _ELF_DEFINE_RELOC(R_PPC_SDAREL16, 32) \ _ELF_DEFINE_RELOC(R_PPC_SECTOFF, 33) \ _ELF_DEFINE_RELOC(R_PPC_SECTOFF_LO, 34) \ _ELF_DEFINE_RELOC(R_PPC_SECTOFF_HI, 35) \ _ELF_DEFINE_RELOC(R_PPC_SECTOFF_HA, 36) \ _ELF_DEFINE_RELOC(R_PPC_ADDR30, 37) \ _ELF_DEFINE_RELOC(R_PPC_TLS, 67) \ _ELF_DEFINE_RELOC(R_PPC_DTPMOD32, 68) \ _ELF_DEFINE_RELOC(R_PPC_TPREL16, 69) \ _ELF_DEFINE_RELOC(R_PPC_TPREL16_LO, 70) \ _ELF_DEFINE_RELOC(R_PPC_TPREL16_HI, 71) \ _ELF_DEFINE_RELOC(R_PPC_TPREL16_HA, 72) \ _ELF_DEFINE_RELOC(R_PPC_TPREL32, 73) \ _ELF_DEFINE_RELOC(R_PPC_DTPREL16, 74) \ _ELF_DEFINE_RELOC(R_PPC_DTPREL16_LO, 75) \ _ELF_DEFINE_RELOC(R_PPC_DTPREL16_HI, 76) \ _ELF_DEFINE_RELOC(R_PPC_DTPREL16_HA, 77) \ _ELF_DEFINE_RELOC(R_PPC_DTPREL32, 78) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TLSGD16, 79) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TLSGD16_LO, 80) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TLSGD16_HI, 81) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TLSGD16_HA, 82) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TLSLD16, 83) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TLSLD16_LO, 84) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TLSLD16_HI, 85) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TLSLD16_HA, 86) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TPREL16, 87) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TPREL16_LO, 88) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TPREL16_HI, 89) \ _ELF_DEFINE_RELOC(R_PPC_GOT_TPREL16_HA, 90) \ _ELF_DEFINE_RELOC(R_PPC_GOT_DTPREL16, 91) \ _ELF_DEFINE_RELOC(R_PPC_GOT_DTPREL16_LO, 92) \ _ELF_DEFINE_RELOC(R_PPC_GOT_DTPREL16_HI, 93) \ _ELF_DEFINE_RELOC(R_PPC_GOT_DTPREL16_HA, 94) \ _ELF_DEFINE_RELOC(R_PPC_TLSGD, 95) \ _ELF_DEFINE_RELOC(R_PPC_TLSLD, 96) \ _ELF_DEFINE_RELOC(R_PPC_EMB_NADDR32, 101) \ _ELF_DEFINE_RELOC(R_PPC_EMB_NADDR16, 102) \ _ELF_DEFINE_RELOC(R_PPC_EMB_NADDR16_LO, 103) \ _ELF_DEFINE_RELOC(R_PPC_EMB_NADDR16_HI, 104) \ _ELF_DEFINE_RELOC(R_PPC_EMB_NADDR16_HA, 105) \ _ELF_DEFINE_RELOC(R_PPC_EMB_SDAI16, 106) \ _ELF_DEFINE_RELOC(R_PPC_EMB_SDA2I16, 107) \ _ELF_DEFINE_RELOC(R_PPC_EMB_SDA2REL, 108) \ _ELF_DEFINE_RELOC(R_PPC_EMB_SDA21, 109) \ _ELF_DEFINE_RELOC(R_PPC_EMB_MRKREF, 110) \ _ELF_DEFINE_RELOC(R_PPC_EMB_RELSEC16, 111) \ _ELF_DEFINE_RELOC(R_PPC_EMB_RELST_LO, 112) \ _ELF_DEFINE_RELOC(R_PPC_EMB_RELST_HI, 113) \ _ELF_DEFINE_RELOC(R_PPC_EMB_RELST_HA, 114) \ _ELF_DEFINE_RELOC(R_PPC_EMB_BIT_FLD, 115) \ _ELF_DEFINE_RELOC(R_PPC_EMB_RELSDA, 116) \ #define _ELF_DEFINE_PPC64_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_PPC64_NONE, 0) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR32, 1) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR24, 2) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16, 3) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_LO, 4) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_HI, 5) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_HA, 6) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR14, 7) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR14_BRTAKEN, 8) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR14_BRNTAKEN, 9) \ _ELF_DEFINE_RELOC(R_PPC64_REL24, 10) \ _ELF_DEFINE_RELOC(R_PPC64_REL14, 11) \ _ELF_DEFINE_RELOC(R_PPC64_REL14_BRTAKEN, 12) \ _ELF_DEFINE_RELOC(R_PPC64_REL14_BRNTAKEN, 13) \ _ELF_DEFINE_RELOC(R_PPC64_GOT16, 14) \ _ELF_DEFINE_RELOC(R_PPC64_GOT16_LO, 15) \ _ELF_DEFINE_RELOC(R_PPC64_GOT16_HI, 16) \ _ELF_DEFINE_RELOC(R_PPC64_GOT16_HA, 17) \ _ELF_DEFINE_RELOC(R_PPC64_COPY, 19) \ _ELF_DEFINE_RELOC(R_PPC64_GLOB_DAT, 20) \ _ELF_DEFINE_RELOC(R_PPC64_JMP_SLOT, 21) \ _ELF_DEFINE_RELOC(R_PPC64_RELATIVE, 22) \ _ELF_DEFINE_RELOC(R_PPC64_UADDR32, 24) \ _ELF_DEFINE_RELOC(R_PPC64_UADDR16, 25) \ _ELF_DEFINE_RELOC(R_PPC64_REL32, 26) \ _ELF_DEFINE_RELOC(R_PPC64_PLT32, 27) \ _ELF_DEFINE_RELOC(R_PPC64_PLTREL32, 28) \ _ELF_DEFINE_RELOC(R_PPC64_PLT16_LO, 29) \ _ELF_DEFINE_RELOC(R_PPC64_PLT16_HI, 30) \ _ELF_DEFINE_RELOC(R_PPC64_PLT16_HA, 31) \ _ELF_DEFINE_RELOC(R_PPC64_SECTOFF, 33) \ _ELF_DEFINE_RELOC(R_PPC64_SECTOFF_LO, 34) \ _ELF_DEFINE_RELOC(R_PPC64_SECTOFF_HI, 35) \ _ELF_DEFINE_RELOC(R_PPC64_SECTOFF_HA, 36) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR30, 37) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR64, 38) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_HIGHER, 39) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_HIGHERA, 40) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_HIGHEST, 41) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_HIGHESTA, 42) \ _ELF_DEFINE_RELOC(R_PPC64_UADDR64, 43) \ _ELF_DEFINE_RELOC(R_PPC64_REL64, 44) \ _ELF_DEFINE_RELOC(R_PPC64_PLT64, 45) \ _ELF_DEFINE_RELOC(R_PPC64_PLTREL64, 46) \ _ELF_DEFINE_RELOC(R_PPC64_TOC16, 47) \ _ELF_DEFINE_RELOC(R_PPC64_TOC16_LO, 48) \ _ELF_DEFINE_RELOC(R_PPC64_TOC16_HI, 49) \ _ELF_DEFINE_RELOC(R_PPC64_TOC16_HA, 50) \ _ELF_DEFINE_RELOC(R_PPC64_TOC, 51) \ _ELF_DEFINE_RELOC(R_PPC64_PLTGOT16, 52) \ _ELF_DEFINE_RELOC(R_PPC64_PLTGOT16_LO, 53) \ _ELF_DEFINE_RELOC(R_PPC64_PLTGOT16_HI, 54) \ _ELF_DEFINE_RELOC(R_PPC64_PLTGOT16_HA, 55) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_DS, 56) \ _ELF_DEFINE_RELOC(R_PPC64_ADDR16_LO_DS, 57) \ _ELF_DEFINE_RELOC(R_PPC64_GOT16_DS, 58) \ _ELF_DEFINE_RELOC(R_PPC64_GOT16_LO_DS, 59) \ _ELF_DEFINE_RELOC(R_PPC64_PLT16_LO_DS, 60) \ _ELF_DEFINE_RELOC(R_PPC64_SECTOFF_DS, 61) \ _ELF_DEFINE_RELOC(R_PPC64_SECTOFF_LO_DS, 62) \ _ELF_DEFINE_RELOC(R_PPC64_TOC16_DS, 63) \ _ELF_DEFINE_RELOC(R_PPC64_TOC16_LO_DS, 64) \ _ELF_DEFINE_RELOC(R_PPC64_PLTGOT16_DS, 65) \ _ELF_DEFINE_RELOC(R_PPC64_PLTGOT16_LO_DS, 66) \ _ELF_DEFINE_RELOC(R_PPC64_TLS, 67) \ _ELF_DEFINE_RELOC(R_PPC64_DTPMOD64, 68) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16, 69) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_LO, 60) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_HI, 71) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_HA, 72) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL64, 73) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16, 74) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_LO, 75) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_HI, 76) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_HA, 77) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL64, 78) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TLSGD16, 79) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TLSGD16_LO, 80) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TLSGD16_HI, 81) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TLSGD16_HA, 82) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TLSLD16, 83) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TLSLD16_LO, 84) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TLSLD16_HI, 85) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TLSLD16_HA, 86) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TPREL16_DS, 87) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TPREL16_LO_DS, 88) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TPREL16_HI, 89) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_TPREL16_HA, 90) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_DTPREL16_DS, 91) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_DTPREL16_LO_DS, 92) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_DTPREL16_HI, 93) \ _ELF_DEFINE_RELOC(R_PPC64_GOT_DTPREL16_HA, 94) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_DS, 95) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_LO_DS, 96) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_HIGHER, 97) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_HIGHERA, 98) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_HIGHEST, 99) \ _ELF_DEFINE_RELOC(R_PPC64_TPREL16_HIGHESTA, 100) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_DS, 101) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_LO_DS, 102) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_HIGHER, 103) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_HIGHERA, 104) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_HIGHEST, 105) \ _ELF_DEFINE_RELOC(R_PPC64_DTPREL16_HIGHESTA, 106) \ _ELF_DEFINE_RELOC(R_PPC64_TLSGD, 107) \ _ELF_DEFINE_RELOC(R_PPC64_TLSLD, 108) #define _ELF_DEFINE_RISCV_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_RISCV_NONE, 0) \ _ELF_DEFINE_RELOC(R_RISCV_32, 1) \ _ELF_DEFINE_RELOC(R_RISCV_64, 2) \ _ELF_DEFINE_RELOC(R_RISCV_RELATIVE, 3) \ _ELF_DEFINE_RELOC(R_RISCV_COPY, 4) \ _ELF_DEFINE_RELOC(R_RISCV_JUMP_SLOT, 5) \ _ELF_DEFINE_RELOC(R_RISCV_TLS_DTPMOD32, 6) \ _ELF_DEFINE_RELOC(R_RISCV_TLS_DTPMOD64, 7) \ _ELF_DEFINE_RELOC(R_RISCV_TLS_DTPREL32, 8) \ _ELF_DEFINE_RELOC(R_RISCV_TLS_DTPREL64, 9) \ _ELF_DEFINE_RELOC(R_RISCV_TLS_TPREL32, 10) \ _ELF_DEFINE_RELOC(R_RISCV_TLS_TPREL64, 11) \ _ELF_DEFINE_RELOC(R_RISCV_BRANCH, 16) \ _ELF_DEFINE_RELOC(R_RISCV_JAL, 17) \ _ELF_DEFINE_RELOC(R_RISCV_CALL, 18) \ _ELF_DEFINE_RELOC(R_RISCV_CALL_PLT, 19) \ _ELF_DEFINE_RELOC(R_RISCV_GOT_HI20, 20) \ _ELF_DEFINE_RELOC(R_RISCV_TLS_GOT_HI20, 21) \ _ELF_DEFINE_RELOC(R_RISCV_TLS_GD_HI20, 22) \ _ELF_DEFINE_RELOC(R_RISCV_PCREL_HI20, 23) \ _ELF_DEFINE_RELOC(R_RISCV_PCREL_LO12_I, 24) \ _ELF_DEFINE_RELOC(R_RISCV_PCREL_LO12_S, 25) \ _ELF_DEFINE_RELOC(R_RISCV_HI20, 26) \ _ELF_DEFINE_RELOC(R_RISCV_LO12_I, 27) \ _ELF_DEFINE_RELOC(R_RISCV_LO12_S, 28) \ _ELF_DEFINE_RELOC(R_RISCV_TPREL_HI20, 29) \ _ELF_DEFINE_RELOC(R_RISCV_TPREL_LO12_I, 30) \ _ELF_DEFINE_RELOC(R_RISCV_TPREL_LO12_S, 31) \ _ELF_DEFINE_RELOC(R_RISCV_TPREL_ADD, 32) \ _ELF_DEFINE_RELOC(R_RISCV_ADD8, 33) \ _ELF_DEFINE_RELOC(R_RISCV_ADD16, 34) \ _ELF_DEFINE_RELOC(R_RISCV_ADD32, 35) \ _ELF_DEFINE_RELOC(R_RISCV_ADD64, 36) \ _ELF_DEFINE_RELOC(R_RISCV_SUB8, 37) \ _ELF_DEFINE_RELOC(R_RISCV_SUB16, 38) \ _ELF_DEFINE_RELOC(R_RISCV_SUB32, 39) \ _ELF_DEFINE_RELOC(R_RISCV_SUB64, 40) \ _ELF_DEFINE_RELOC(R_RISCV_GNU_VTINHERIT, 41) \ _ELF_DEFINE_RELOC(R_RISCV_GNU_VTENTRY, 42) \ _ELF_DEFINE_RELOC(R_RISCV_ALIGN, 43) \ _ELF_DEFINE_RELOC(R_RISCV_RVC_BRANCH, 44) \ -_ELF_DEFINE_RELOC(R_RISCV_RVC_JUMP, 45) +_ELF_DEFINE_RELOC(R_RISCV_RVC_JUMP, 45) \ +_ELF_DEFINE_RELOC(R_RISCV_RVC_LUI, 46) \ +_ELF_DEFINE_RELOC(R_RISCV_GPREL_I, 47) \ +_ELF_DEFINE_RELOC(R_RISCV_GPREL_S, 48) #define _ELF_DEFINE_SPARC_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_SPARC_NONE, 0) \ _ELF_DEFINE_RELOC(R_SPARC_8, 1) \ _ELF_DEFINE_RELOC(R_SPARC_16, 2) \ _ELF_DEFINE_RELOC(R_SPARC_32, 3) \ _ELF_DEFINE_RELOC(R_SPARC_DISP8, 4) \ _ELF_DEFINE_RELOC(R_SPARC_DISP16, 5) \ _ELF_DEFINE_RELOC(R_SPARC_DISP32, 6) \ _ELF_DEFINE_RELOC(R_SPARC_WDISP30, 7) \ _ELF_DEFINE_RELOC(R_SPARC_WDISP22, 8) \ _ELF_DEFINE_RELOC(R_SPARC_HI22, 9) \ _ELF_DEFINE_RELOC(R_SPARC_22, 10) \ _ELF_DEFINE_RELOC(R_SPARC_13, 11) \ _ELF_DEFINE_RELOC(R_SPARC_LO10, 12) \ _ELF_DEFINE_RELOC(R_SPARC_GOT10, 13) \ _ELF_DEFINE_RELOC(R_SPARC_GOT13, 14) \ _ELF_DEFINE_RELOC(R_SPARC_GOT22, 15) \ _ELF_DEFINE_RELOC(R_SPARC_PC10, 16) \ _ELF_DEFINE_RELOC(R_SPARC_PC22, 17) \ _ELF_DEFINE_RELOC(R_SPARC_WPLT30, 18) \ _ELF_DEFINE_RELOC(R_SPARC_COPY, 19) \ _ELF_DEFINE_RELOC(R_SPARC_GLOB_DAT, 20) \ _ELF_DEFINE_RELOC(R_SPARC_JMP_SLOT, 21) \ _ELF_DEFINE_RELOC(R_SPARC_RELATIVE, 22) \ _ELF_DEFINE_RELOC(R_SPARC_UA32, 23) \ _ELF_DEFINE_RELOC(R_SPARC_PLT32, 24) \ _ELF_DEFINE_RELOC(R_SPARC_HIPLT22, 25) \ _ELF_DEFINE_RELOC(R_SPARC_LOPLT10, 26) \ _ELF_DEFINE_RELOC(R_SPARC_PCPLT32, 27) \ _ELF_DEFINE_RELOC(R_SPARC_PCPLT22, 28) \ _ELF_DEFINE_RELOC(R_SPARC_PCPLT10, 29) \ _ELF_DEFINE_RELOC(R_SPARC_10, 30) \ _ELF_DEFINE_RELOC(R_SPARC_11, 31) \ _ELF_DEFINE_RELOC(R_SPARC_64, 32) \ _ELF_DEFINE_RELOC(R_SPARC_OLO10, 33) \ _ELF_DEFINE_RELOC(R_SPARC_HH22, 34) \ _ELF_DEFINE_RELOC(R_SPARC_HM10, 35) \ _ELF_DEFINE_RELOC(R_SPARC_LM22, 36) \ _ELF_DEFINE_RELOC(R_SPARC_PC_HH22, 37) \ _ELF_DEFINE_RELOC(R_SPARC_PC_HM10, 38) \ _ELF_DEFINE_RELOC(R_SPARC_PC_LM22, 39) \ _ELF_DEFINE_RELOC(R_SPARC_WDISP16, 40) \ _ELF_DEFINE_RELOC(R_SPARC_WDISP19, 41) \ _ELF_DEFINE_RELOC(R_SPARC_GLOB_JMP, 42) \ _ELF_DEFINE_RELOC(R_SPARC_7, 43) \ _ELF_DEFINE_RELOC(R_SPARC_5, 44) \ _ELF_DEFINE_RELOC(R_SPARC_6, 45) \ _ELF_DEFINE_RELOC(R_SPARC_DISP64, 46) \ _ELF_DEFINE_RELOC(R_SPARC_PLT64, 47) \ _ELF_DEFINE_RELOC(R_SPARC_HIX22, 48) \ _ELF_DEFINE_RELOC(R_SPARC_LOX10, 49) \ _ELF_DEFINE_RELOC(R_SPARC_H44, 50) \ _ELF_DEFINE_RELOC(R_SPARC_M44, 51) \ _ELF_DEFINE_RELOC(R_SPARC_L44, 52) \ _ELF_DEFINE_RELOC(R_SPARC_REGISTER, 53) \ _ELF_DEFINE_RELOC(R_SPARC_UA64, 54) \ _ELF_DEFINE_RELOC(R_SPARC_UA16, 55) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_GD_HI22, 56) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_GD_LO10, 57) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_GD_ADD, 58) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_GD_CALL, 59) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LDM_HI22, 60) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LDM_LO10, 61) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LDM_ADD, 62) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LDM_CALL, 63) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LDO_HIX22, 64) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LDO_LOX10, 65) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LDO_ADD, 66) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_IE_HI22, 67) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_IE_LO10, 68) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_IE_LD, 69) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_IE_LDX, 70) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_IE_ADD, 71) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LE_HIX22, 72) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_LE_LOX10, 73) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_DTPMOD32, 74) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_DTPMOD64, 75) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_DTPOFF32, 76) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_DTPOFF64, 77) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_TPOFF32, 78) \ _ELF_DEFINE_RELOC(R_SPARC_TLS_TPOFF64, 79) \ _ELF_DEFINE_RELOC(R_SPARC_GOTDATA_HIX22, 80) \ _ELF_DEFINE_RELOC(R_SPARC_GOTDATA_LOX10, 81) \ _ELF_DEFINE_RELOC(R_SPARC_GOTDATA_OP_HIX22, 82) \ _ELF_DEFINE_RELOC(R_SPARC_GOTDATA_OP_LOX10, 83) \ _ELF_DEFINE_RELOC(R_SPARC_GOTDATA_OP, 84) \ _ELF_DEFINE_RELOC(R_SPARC_H34, 85) #define _ELF_DEFINE_X86_64_RELOCATIONS() \ _ELF_DEFINE_RELOC(R_X86_64_NONE, 0) \ _ELF_DEFINE_RELOC(R_X86_64_64, 1) \ _ELF_DEFINE_RELOC(R_X86_64_PC32, 2) \ _ELF_DEFINE_RELOC(R_X86_64_GOT32, 3) \ _ELF_DEFINE_RELOC(R_X86_64_PLT32, 4) \ _ELF_DEFINE_RELOC(R_X86_64_COPY, 5) \ _ELF_DEFINE_RELOC(R_X86_64_GLOB_DAT, 6) \ _ELF_DEFINE_RELOC(R_X86_64_JUMP_SLOT, 7) \ _ELF_DEFINE_RELOC(R_X86_64_RELATIVE, 8) \ _ELF_DEFINE_RELOC(R_X86_64_GOTPCREL, 9) \ _ELF_DEFINE_RELOC(R_X86_64_32, 10) \ _ELF_DEFINE_RELOC(R_X86_64_32S, 11) \ _ELF_DEFINE_RELOC(R_X86_64_16, 12) \ _ELF_DEFINE_RELOC(R_X86_64_PC16, 13) \ _ELF_DEFINE_RELOC(R_X86_64_8, 14) \ _ELF_DEFINE_RELOC(R_X86_64_PC8, 15) \ _ELF_DEFINE_RELOC(R_X86_64_DTPMOD64, 16) \ _ELF_DEFINE_RELOC(R_X86_64_DTPOFF64, 17) \ _ELF_DEFINE_RELOC(R_X86_64_TPOFF64, 18) \ _ELF_DEFINE_RELOC(R_X86_64_TLSGD, 19) \ _ELF_DEFINE_RELOC(R_X86_64_TLSLD, 20) \ _ELF_DEFINE_RELOC(R_X86_64_DTPOFF32, 21) \ _ELF_DEFINE_RELOC(R_X86_64_GOTTPOFF, 22) \ _ELF_DEFINE_RELOC(R_X86_64_TPOFF32, 23) \ _ELF_DEFINE_RELOC(R_X86_64_PC64, 24) \ _ELF_DEFINE_RELOC(R_X86_64_GOTOFF64, 25) \ _ELF_DEFINE_RELOC(R_X86_64_GOTPC32, 26) \ _ELF_DEFINE_RELOC(R_X86_64_GOT64, 27) \ _ELF_DEFINE_RELOC(R_X86_64_GOTPCREL64, 28) \ _ELF_DEFINE_RELOC(R_X86_64_GOTPC64, 29) \ _ELF_DEFINE_RELOC(R_X86_64_GOTPLT64, 30) \ _ELF_DEFINE_RELOC(R_X86_64_PLTOFF64, 31) \ _ELF_DEFINE_RELOC(R_X86_64_SIZE32, 32) \ _ELF_DEFINE_RELOC(R_X86_64_SIZE64, 33) \ _ELF_DEFINE_RELOC(R_X86_64_GOTPC32_TLSDESC, 34) \ _ELF_DEFINE_RELOC(R_X86_64_TLSDESC_CALL, 35) \ _ELF_DEFINE_RELOC(R_X86_64_TLSDESC, 36) \ _ELF_DEFINE_RELOC(R_X86_64_IRELATIVE, 37) \ _ELF_DEFINE_RELOC(R_X86_64_RELATIVE64, 38) \ _ELF_DEFINE_RELOC(R_X86_64_GOTPCRELX, 41) \ _ELF_DEFINE_RELOC(R_X86_64_REX_GOTPCRELX, 42) #define _ELF_DEFINE_RELOCATIONS() \ _ELF_DEFINE_386_RELOCATIONS() \ _ELF_DEFINE_AARCH64_RELOCATIONS() \ _ELF_DEFINE_AMD64_RELOCATIONS() \ _ELF_DEFINE_ARM_RELOCATIONS() \ _ELF_DEFINE_IA64_RELOCATIONS() \ _ELF_DEFINE_MIPS_RELOCATIONS() \ _ELF_DEFINE_PPC32_RELOCATIONS() \ _ELF_DEFINE_PPC64_RELOCATIONS() \ _ELF_DEFINE_RISCV_RELOCATIONS() \ _ELF_DEFINE_SPARC_RELOCATIONS() \ _ELF_DEFINE_X86_64_RELOCATIONS() #undef _ELF_DEFINE_RELOC #define _ELF_DEFINE_RELOC(N, V) N = V , enum { _ELF_DEFINE_RELOCATIONS() R__LAST__ }; #define PN_XNUM 0xFFFFU /* Use extended section numbering. */ /** ** ELF Types. **/ typedef uint32_t Elf32_Addr; /* Program address. */ typedef uint8_t Elf32_Byte; /* Unsigned tiny integer. */ typedef uint16_t Elf32_Half; /* Unsigned medium integer. */ typedef uint32_t Elf32_Off; /* File offset. */ typedef uint16_t Elf32_Section; /* Section index. */ typedef int32_t Elf32_Sword; /* Signed integer. */ typedef uint32_t Elf32_Word; /* Unsigned integer. */ typedef uint64_t Elf32_Lword; /* Unsigned long integer. */ typedef uint64_t Elf64_Addr; /* Program address. */ typedef uint8_t Elf64_Byte; /* Unsigned tiny integer. */ typedef uint16_t Elf64_Half; /* Unsigned medium integer. */ typedef uint64_t Elf64_Off; /* File offset. */ typedef uint16_t Elf64_Section; /* Section index. */ typedef int32_t Elf64_Sword; /* Signed integer. */ typedef uint32_t Elf64_Word; /* Unsigned integer. */ typedef uint64_t Elf64_Lword; /* Unsigned long integer. */ typedef uint64_t Elf64_Xword; /* Unsigned long integer. */ typedef int64_t Elf64_Sxword; /* Signed long integer. */ /* * Capability descriptors. */ /* 32-bit capability descriptor. */ typedef struct { Elf32_Word c_tag; /* Type of entry. */ union { Elf32_Word c_val; /* Integer value. */ Elf32_Addr c_ptr; /* Pointer value. */ } c_un; } Elf32_Cap; /* 64-bit capability descriptor. */ typedef struct { Elf64_Xword c_tag; /* Type of entry. */ union { Elf64_Xword c_val; /* Integer value. */ Elf64_Addr c_ptr; /* Pointer value. */ } c_un; } Elf64_Cap; /* * MIPS .conflict section entries. */ /* 32-bit entry. */ typedef struct { Elf32_Addr c_index; } Elf32_Conflict; /* 64-bit entry. */ typedef struct { Elf64_Addr c_index; } Elf64_Conflict; /* * Dynamic section entries. */ /* 32-bit entry. */ typedef struct { Elf32_Sword d_tag; /* Type of entry. */ union { Elf32_Word d_val; /* Integer value. */ Elf32_Addr d_ptr; /* Pointer value. */ } d_un; } Elf32_Dyn; /* 64-bit entry. */ typedef struct { Elf64_Sxword d_tag; /* Type of entry. */ union { Elf64_Xword d_val; /* Integer value. */ Elf64_Addr d_ptr; /* Pointer value; */ } d_un; } Elf64_Dyn; /* * The executable header (EHDR). */ /* 32 bit EHDR. */ typedef struct { unsigned char e_ident[EI_NIDENT]; /* ELF identification. */ Elf32_Half e_type; /* Object file type (ET_*). */ Elf32_Half e_machine; /* Machine type (EM_*). */ Elf32_Word e_version; /* File format version (EV_*). */ Elf32_Addr e_entry; /* Start address. */ Elf32_Off e_phoff; /* File offset to the PHDR table. */ Elf32_Off e_shoff; /* File offset to the SHDRheader. */ Elf32_Word e_flags; /* Flags (EF_*). */ Elf32_Half e_ehsize; /* Elf header size in bytes. */ Elf32_Half e_phentsize; /* PHDR table entry size in bytes. */ Elf32_Half e_phnum; /* Number of PHDR entries. */ Elf32_Half e_shentsize; /* SHDR table entry size in bytes. */ Elf32_Half e_shnum; /* Number of SHDR entries. */ Elf32_Half e_shstrndx; /* Index of section name string table. */ } Elf32_Ehdr; /* 64 bit EHDR. */ typedef struct { unsigned char e_ident[EI_NIDENT]; /* ELF identification. */ Elf64_Half e_type; /* Object file type (ET_*). */ Elf64_Half e_machine; /* Machine type (EM_*). */ Elf64_Word e_version; /* File format version (EV_*). */ Elf64_Addr e_entry; /* Start address. */ Elf64_Off e_phoff; /* File offset to the PHDR table. */ Elf64_Off e_shoff; /* File offset to the SHDRheader. */ Elf64_Word e_flags; /* Flags (EF_*). */ Elf64_Half e_ehsize; /* Elf header size in bytes. */ Elf64_Half e_phentsize; /* PHDR table entry size in bytes. */ Elf64_Half e_phnum; /* Number of PHDR entries. */ Elf64_Half e_shentsize; /* SHDR table entry size in bytes. */ Elf64_Half e_shnum; /* Number of SHDR entries. */ Elf64_Half e_shstrndx; /* Index of section name string table. */ } Elf64_Ehdr; /* * Shared object information. */ /* 32-bit entry. */ typedef struct { Elf32_Word l_name; /* The name of a shared object. */ Elf32_Word l_time_stamp; /* 32-bit timestamp. */ Elf32_Word l_checksum; /* Checksum of visible symbols, sizes. */ Elf32_Word l_version; /* Interface version string index. */ Elf32_Word l_flags; /* Flags (LL_*). */ } Elf32_Lib; /* 64-bit entry. */ typedef struct { Elf64_Word l_name; /* The name of a shared object. */ Elf64_Word l_time_stamp; /* 32-bit timestamp. */ Elf64_Word l_checksum; /* Checksum of visible symbols, sizes. */ Elf64_Word l_version; /* Interface version string index. */ Elf64_Word l_flags; /* Flags (LL_*). */ } Elf64_Lib; #define _ELF_DEFINE_LL_FLAGS() \ _ELF_DEFINE_LL(LL_NONE, 0, \ "no flags") \ _ELF_DEFINE_LL(LL_EXACT_MATCH, 0x1, \ "require an exact match") \ _ELF_DEFINE_LL(LL_IGNORE_INT_VER, 0x2, \ "ignore version incompatibilities") \ _ELF_DEFINE_LL(LL_REQUIRE_MINOR, 0x4, \ "") \ _ELF_DEFINE_LL(LL_EXPORTS, 0x8, \ "") \ _ELF_DEFINE_LL(LL_DELAY_LOAD, 0x10, \ "") \ _ELF_DEFINE_LL(LL_DELTA, 0x20, \ "") #undef _ELF_DEFINE_LL #define _ELF_DEFINE_LL(N, V, DESCR) N = V , enum { _ELF_DEFINE_LL_FLAGS() LL__LAST__ }; /* * Note tags */ #define _ELF_DEFINE_NOTE_ENTRY_TYPES() \ _ELF_DEFINE_NT(NT_ABI_TAG, 1, "Tag indicating the ABI") \ _ELF_DEFINE_NT(NT_GNU_HWCAP, 2, "Hardware capabilities") \ _ELF_DEFINE_NT(NT_GNU_BUILD_ID, 3, "Build id, set by ld(1)") \ _ELF_DEFINE_NT(NT_GNU_GOLD_VERSION, 4, \ "Version number of the GNU gold linker") \ _ELF_DEFINE_NT(NT_PRSTATUS, 1, "Process status") \ _ELF_DEFINE_NT(NT_FPREGSET, 2, "Floating point information") \ _ELF_DEFINE_NT(NT_PRPSINFO, 3, "Process information") \ _ELF_DEFINE_NT(NT_AUXV, 6, "Auxiliary vector") \ _ELF_DEFINE_NT(NT_PRXFPREG, 0x46E62B7FUL, \ "Linux user_xfpregs structure") \ _ELF_DEFINE_NT(NT_PSTATUS, 10, "Linux process status") \ _ELF_DEFINE_NT(NT_FPREGS, 12, "Linux floating point regset") \ _ELF_DEFINE_NT(NT_PSINFO, 13, "Linux process information") \ _ELF_DEFINE_NT(NT_LWPSTATUS, 16, "Linux lwpstatus_t type") \ _ELF_DEFINE_NT(NT_LWPSINFO, 17, "Linux lwpinfo_t type") #undef _ELF_DEFINE_NT #define _ELF_DEFINE_NT(N, V, DESCR) N = V , enum { _ELF_DEFINE_NOTE_ENTRY_TYPES() NT__LAST__ }; /* Aliases for the ABI tag. */ #define NT_FREEBSD_ABI_TAG NT_ABI_TAG #define NT_GNU_ABI_TAG NT_ABI_TAG #define NT_NETBSD_IDENT NT_ABI_TAG #define NT_OPENBSD_IDENT NT_ABI_TAG /* * Note descriptors. */ typedef struct { uint32_t n_namesz; /* Length of note's name. */ uint32_t n_descsz; /* Length of note's value. */ uint32_t n_type; /* Type of note. */ } Elf_Note; typedef Elf_Note Elf32_Nhdr; /* 32-bit note header. */ typedef Elf_Note Elf64_Nhdr; /* 64-bit note header. */ /* * MIPS ELF options descriptor header. */ typedef struct { Elf64_Byte kind; /* Type of options. */ Elf64_Byte size; /* Size of option descriptor. */ Elf64_Half section; /* Index of section affected. */ Elf64_Word info; /* Kind-specific information. */ } Elf_Options; /* * Option kinds. */ #define _ELF_DEFINE_OPTION_KINDS() \ _ELF_DEFINE_ODK(ODK_NULL, 0, "undefined") \ _ELF_DEFINE_ODK(ODK_REGINFO, 1, "register usage info") \ _ELF_DEFINE_ODK(ODK_EXCEPTIONS, 2, "exception processing info") \ _ELF_DEFINE_ODK(ODK_PAD, 3, "section padding") \ _ELF_DEFINE_ODK(ODK_HWPATCH, 4, "hardware patch applied") \ _ELF_DEFINE_ODK(ODK_FILL, 5, "fill value used by linker") \ _ELF_DEFINE_ODK(ODK_TAGS, 6, "reserved space for tools") \ _ELF_DEFINE_ODK(ODK_HWAND, 7, "hardware AND patch applied") \ _ELF_DEFINE_ODK(ODK_HWOR, 8, "hardware OR patch applied") \ _ELF_DEFINE_ODK(ODK_GP_GROUP, 9, \ "GP group to use for text/data sections") \ _ELF_DEFINE_ODK(ODK_IDENT, 10, "ID information") \ _ELF_DEFINE_ODK(ODK_PAGESIZE, 11, "page size information") #undef _ELF_DEFINE_ODK #define _ELF_DEFINE_ODK(N, V, DESCR) N = V , enum { _ELF_DEFINE_OPTION_KINDS() ODK__LAST__ }; /* * ODK_EXCEPTIONS info field masks. */ #define _ELF_DEFINE_ODK_EXCEPTIONS_MASK() \ _ELF_DEFINE_OEX(OEX_FPU_MIN, 0x0000001FUL, \ "minimum FPU exception which must be enabled") \ _ELF_DEFINE_OEX(OEX_FPU_MAX, 0x00001F00UL, \ "maximum FPU exception which can be enabled") \ _ELF_DEFINE_OEX(OEX_PAGE0, 0x00010000UL, \ "page zero must be mapped") \ _ELF_DEFINE_OEX(OEX_SMM, 0x00020000UL, \ "run in sequential memory mode") \ _ELF_DEFINE_OEX(OEX_PRECISEFP, 0x00040000UL, \ "run in precise FP exception mode") \ _ELF_DEFINE_OEX(OEX_DISMISS, 0x00080000UL, \ "dismiss invalid address traps") #undef _ELF_DEFINE_OEX #define _ELF_DEFINE_OEX(N, V, DESCR) N = V , enum { _ELF_DEFINE_ODK_EXCEPTIONS_MASK() OEX__LAST__ }; /* * ODK_PAD info field masks. */ #define _ELF_DEFINE_ODK_PAD_MASK() \ _ELF_DEFINE_OPAD(OPAD_PREFIX, 0x0001) \ _ELF_DEFINE_OPAD(OPAD_POSTFIX, 0x0002) \ _ELF_DEFINE_OPAD(OPAD_SYMBOL, 0x0004) #undef _ELF_DEFINE_OPAD #define _ELF_DEFINE_OPAD(N, V) N = V , enum { _ELF_DEFINE_ODK_PAD_MASK() OPAD__LAST__ }; /* * ODK_HWPATCH info field masks. */ #define _ELF_DEFINE_ODK_HWPATCH_MASK() \ _ELF_DEFINE_OHW(OHW_R4KEOP, 0x00000001UL, \ "patch for R4000 branch at end-of-page bug") \ _ELF_DEFINE_OHW(OHW_R8KPFETCH, 0x00000002UL, \ "R8000 prefetch bug may occur") \ _ELF_DEFINE_OHW(OHW_R5KEOP, 0x00000004UL, \ "patch for R5000 branch at end-of-page bug") \ _ELF_DEFINE_OHW(OHW_R5KCVTL, 0x00000008UL, \ "R5000 cvt.[ds].l bug: clean == 1") \ _ELF_DEFINE_OHW(OHW_R10KLDL, 0x00000010UL, \ "needd patch for R10000 misaligned load") #undef _ELF_DEFINE_OHW #define _ELF_DEFINE_OHW(N, V, DESCR) N = V , enum { _ELF_DEFINE_ODK_HWPATCH_MASK() OHW__LAST__ }; /* * ODK_HWAND/ODK_HWOR info field and hwp_flags[12] masks. */ #define _ELF_DEFINE_ODK_HWP_MASK() \ _ELF_DEFINE_HWP(OHWA0_R4KEOP_CHECKED, 0x00000001UL, \ "object checked for R4000 end-of-page bug") \ _ELF_DEFINE_HWP(OHWA0_R4KEOP_CLEAN, 0x00000002UL, \ "object verified clean for R4000 end-of-page bug") \ _ELF_DEFINE_HWP(OHWO0_FIXADE, 0x00000001UL, \ "object requires call to fixade") #undef _ELF_DEFINE_HWP #define _ELF_DEFINE_HWP(N, V, DESCR) N = V , enum { _ELF_DEFINE_ODK_HWP_MASK() OHWX0__LAST__ }; /* * ODK_IDENT/ODK_GP_GROUP info field masks. */ #define _ELF_DEFINE_ODK_GP_MASK() \ _ELF_DEFINE_OGP(OGP_GROUP, 0x0000FFFFUL, "GP group number") \ _ELF_DEFINE_OGP(OGP_SELF, 0x00010000UL, \ "GP group is self-contained") #undef _ELF_DEFINE_OGP #define _ELF_DEFINE_OGP(N, V, DESCR) N = V , enum { _ELF_DEFINE_ODK_GP_MASK() OGP__LAST__ }; /* * MIPS ELF register info descriptor. */ /* 32 bit RegInfo entry. */ typedef struct { Elf32_Word ri_gprmask; /* Mask of general register used. */ Elf32_Word ri_cprmask[4]; /* Mask of coprocessor register used. */ Elf32_Addr ri_gp_value; /* GP register value. */ } Elf32_RegInfo; /* 64 bit RegInfo entry. */ typedef struct { Elf64_Word ri_gprmask; /* Mask of general register used. */ Elf64_Word ri_pad; /* Padding. */ Elf64_Word ri_cprmask[4]; /* Mask of coprocessor register used. */ Elf64_Addr ri_gp_value; /* GP register value. */ } Elf64_RegInfo; /* * Program Header Table (PHDR) entries. */ /* 32 bit PHDR entry. */ typedef struct { Elf32_Word p_type; /* Type of segment. */ Elf32_Off p_offset; /* File offset to segment. */ Elf32_Addr p_vaddr; /* Virtual address in memory. */ Elf32_Addr p_paddr; /* Physical address (if relevant). */ Elf32_Word p_filesz; /* Size of segment in file. */ Elf32_Word p_memsz; /* Size of segment in memory. */ Elf32_Word p_flags; /* Segment flags. */ Elf32_Word p_align; /* Alignment constraints. */ } Elf32_Phdr; /* 64 bit PHDR entry. */ typedef struct { Elf64_Word p_type; /* Type of segment. */ Elf64_Word p_flags; /* Segment flags. */ Elf64_Off p_offset; /* File offset to segment. */ Elf64_Addr p_vaddr; /* Virtual address in memory. */ Elf64_Addr p_paddr; /* Physical address (if relevant). */ Elf64_Xword p_filesz; /* Size of segment in file. */ Elf64_Xword p_memsz; /* Size of segment in memory. */ Elf64_Xword p_align; /* Alignment constraints. */ } Elf64_Phdr; /* * Move entries, for describing data in COMMON blocks in a compact * manner. */ /* 32-bit move entry. */ typedef struct { Elf32_Lword m_value; /* Initialization value. */ Elf32_Word m_info; /* Encoded size and index. */ Elf32_Word m_poffset; /* Offset relative to symbol. */ Elf32_Half m_repeat; /* Repeat count. */ Elf32_Half m_stride; /* Number of units to skip. */ } Elf32_Move; /* 64-bit move entry. */ typedef struct { Elf64_Lword m_value; /* Initialization value. */ Elf64_Xword m_info; /* Encoded size and index. */ Elf64_Xword m_poffset; /* Offset relative to symbol. */ Elf64_Half m_repeat; /* Repeat count. */ Elf64_Half m_stride; /* Number of units to skip. */ } Elf64_Move; #define ELF32_M_SYM(I) ((I) >> 8) #define ELF32_M_SIZE(I) ((unsigned char) (I)) #define ELF32_M_INFO(M, S) (((M) << 8) + (unsigned char) (S)) #define ELF64_M_SYM(I) ((I) >> 8) #define ELF64_M_SIZE(I) ((unsigned char) (I)) #define ELF64_M_INFO(M, S) (((M) << 8) + (unsigned char) (S)) /* * Section Header Table (SHDR) entries. */ /* 32 bit SHDR */ typedef struct { Elf32_Word sh_name; /* index of section name */ Elf32_Word sh_type; /* section type */ Elf32_Word sh_flags; /* section flags */ Elf32_Addr sh_addr; /* in-memory address of section */ Elf32_Off sh_offset; /* file offset of section */ Elf32_Word sh_size; /* section size in bytes */ Elf32_Word sh_link; /* section header table link */ Elf32_Word sh_info; /* extra information */ Elf32_Word sh_addralign; /* alignment constraint */ Elf32_Word sh_entsize; /* size for fixed-size entries */ } Elf32_Shdr; /* 64 bit SHDR */ typedef struct { Elf64_Word sh_name; /* index of section name */ Elf64_Word sh_type; /* section type */ Elf64_Xword sh_flags; /* section flags */ Elf64_Addr sh_addr; /* in-memory address of section */ Elf64_Off sh_offset; /* file offset of section */ Elf64_Xword sh_size; /* section size in bytes */ Elf64_Word sh_link; /* section header table link */ Elf64_Word sh_info; /* extra information */ Elf64_Xword sh_addralign; /* alignment constraint */ Elf64_Xword sh_entsize; /* size for fixed-size entries */ } Elf64_Shdr; /* * Symbol table entries. */ typedef struct { Elf32_Word st_name; /* index of symbol's name */ Elf32_Addr st_value; /* value for the symbol */ Elf32_Word st_size; /* size of associated data */ unsigned char st_info; /* type and binding attributes */ unsigned char st_other; /* visibility */ Elf32_Half st_shndx; /* index of related section */ } Elf32_Sym; typedef struct { Elf64_Word st_name; /* index of symbol's name */ unsigned char st_info; /* type and binding attributes */ unsigned char st_other; /* visibility */ Elf64_Half st_shndx; /* index of related section */ Elf64_Addr st_value; /* value for the symbol */ Elf64_Xword st_size; /* size of associated data */ } Elf64_Sym; #define ELF32_ST_BIND(I) ((I) >> 4) #define ELF32_ST_TYPE(I) ((I) & 0xFU) #define ELF32_ST_INFO(B,T) (((B) << 4) + ((T) & 0xF)) #define ELF64_ST_BIND(I) ((I) >> 4) #define ELF64_ST_TYPE(I) ((I) & 0xFU) #define ELF64_ST_INFO(B,T) (((B) << 4) + ((T) & 0xF)) #define ELF32_ST_VISIBILITY(O) ((O) & 0x3) #define ELF64_ST_VISIBILITY(O) ((O) & 0x3) /* * Syminfo descriptors, containing additional symbol information. */ /* 32-bit entry. */ typedef struct { Elf32_Half si_boundto; /* Entry index with additional flags. */ Elf32_Half si_flags; /* Flags. */ } Elf32_Syminfo; /* 64-bit entry. */ typedef struct { Elf64_Half si_boundto; /* Entry index with additional flags. */ Elf64_Half si_flags; /* Flags. */ } Elf64_Syminfo; /* * Relocation descriptors. */ typedef struct { Elf32_Addr r_offset; /* location to apply relocation to */ Elf32_Word r_info; /* type+section for relocation */ } Elf32_Rel; typedef struct { Elf32_Addr r_offset; /* location to apply relocation to */ Elf32_Word r_info; /* type+section for relocation */ Elf32_Sword r_addend; /* constant addend */ } Elf32_Rela; typedef struct { Elf64_Addr r_offset; /* location to apply relocation to */ Elf64_Xword r_info; /* type+section for relocation */ } Elf64_Rel; typedef struct { Elf64_Addr r_offset; /* location to apply relocation to */ Elf64_Xword r_info; /* type+section for relocation */ Elf64_Sxword r_addend; /* constant addend */ } Elf64_Rela; #define ELF32_R_SYM(I) ((I) >> 8) #define ELF32_R_TYPE(I) ((unsigned char) (I)) #define ELF32_R_INFO(S,T) (((S) << 8) + (unsigned char) (T)) #define ELF64_R_SYM(I) ((I) >> 32) #define ELF64_R_TYPE(I) ((I) & 0xFFFFFFFFUL) #define ELF64_R_INFO(S,T) (((S) << 32) + ((T) & 0xFFFFFFFFUL)) /* * Symbol versioning structures. */ /* 32-bit structures. */ typedef struct { Elf32_Word vda_name; /* Index to name. */ Elf32_Word vda_next; /* Offset to next entry. */ } Elf32_Verdaux; typedef struct { Elf32_Word vna_hash; /* Hash value of dependency name. */ Elf32_Half vna_flags; /* Flags. */ Elf32_Half vna_other; /* Unused. */ Elf32_Word vna_name; /* Offset to dependency name. */ Elf32_Word vna_next; /* Offset to next vernaux entry. */ } Elf32_Vernaux; typedef struct { Elf32_Half vd_version; /* Version information. */ Elf32_Half vd_flags; /* Flags. */ Elf32_Half vd_ndx; /* Index into the versym section. */ Elf32_Half vd_cnt; /* Number of aux entries. */ Elf32_Word vd_hash; /* Hash value of name. */ Elf32_Word vd_aux; /* Offset to aux entries. */ Elf32_Word vd_next; /* Offset to next version definition. */ } Elf32_Verdef; typedef struct { Elf32_Half vn_version; /* Version number. */ Elf32_Half vn_cnt; /* Number of aux entries. */ Elf32_Word vn_file; /* Offset of associated file name. */ Elf32_Word vn_aux; /* Offset of vernaux array. */ Elf32_Word vn_next; /* Offset of next verneed entry. */ } Elf32_Verneed; typedef Elf32_Half Elf32_Versym; /* 64-bit structures. */ typedef struct { Elf64_Word vda_name; /* Index to name. */ Elf64_Word vda_next; /* Offset to next entry. */ } Elf64_Verdaux; typedef struct { Elf64_Word vna_hash; /* Hash value of dependency name. */ Elf64_Half vna_flags; /* Flags. */ Elf64_Half vna_other; /* Unused. */ Elf64_Word vna_name; /* Offset to dependency name. */ Elf64_Word vna_next; /* Offset to next vernaux entry. */ } Elf64_Vernaux; typedef struct { Elf64_Half vd_version; /* Version information. */ Elf64_Half vd_flags; /* Flags. */ Elf64_Half vd_ndx; /* Index into the versym section. */ Elf64_Half vd_cnt; /* Number of aux entries. */ Elf64_Word vd_hash; /* Hash value of name. */ Elf64_Word vd_aux; /* Offset to aux entries. */ Elf64_Word vd_next; /* Offset to next version definition. */ } Elf64_Verdef; typedef struct { Elf64_Half vn_version; /* Version number. */ Elf64_Half vn_cnt; /* Number of aux entries. */ Elf64_Word vn_file; /* Offset of associated file name. */ Elf64_Word vn_aux; /* Offset of vernaux array. */ Elf64_Word vn_next; /* Offset of next verneed entry. */ } Elf64_Verneed; typedef Elf64_Half Elf64_Versym; /* * The header for GNU-style hash sections. */ typedef struct { uint32_t gh_nbuckets; /* Number of hash buckets. */ uint32_t gh_symndx; /* First visible symbol in .dynsym. */ uint32_t gh_maskwords; /* #maskwords used in bloom filter. */ uint32_t gh_shift2; /* Bloom filter shift count. */ } Elf_GNU_Hash_Header; #endif /* _ELFDEFINITIONS_H_ */ Index: projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/archive.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/archive.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/archive.c (revision 305172) @@ -1,523 +1,526 @@ /*- * Copyright (c) 2007-2009 Kai Wang * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #ifndef LIBELF_AR #include #include #endif /* ! LIBELF_AR */ #include "elfcopy.h" -ELFTC_VCSID("$Id: archive.c 3287 2015-12-31 16:58:48Z emaste $"); +ELFTC_VCSID("$Id: archive.c 3490 2016-08-31 00:12:22Z emaste $"); #define _ARMAG_LEN 8 /* length of ar magic string */ #define _ARHDR_LEN 60 /* length of ar header */ #define _INIT_AS_CAP 128 /* initial archive string table size */ #define _INIT_SYMOFF_CAP (256*(sizeof(uint32_t))) /* initial so table size */ #define _INIT_SYMNAME_CAP 1024 /* initial sn table size */ #define _MAXNAMELEN_SVR4 15 /* max member name length in svr4 variant */ #ifndef LIBELF_AR static void ac_read_objs(struct elfcopy *ecp, int ifd); static void ac_write_cleanup(struct elfcopy *ecp); static void ac_write_data(struct archive *a, const void *buf, size_t s); static void ac_write_objs(struct elfcopy *ecp, int ofd); #endif /* ! LIBELF_AR */ static void add_to_ar_str_table(struct elfcopy *elfcopy, const char *name); static void add_to_ar_sym_table(struct elfcopy *ecp, const char *name); static void extract_arsym(struct elfcopy *ecp); static void process_ar_obj(struct elfcopy *ecp, struct ar_obj *obj); static void sync_ar(struct elfcopy *ecp); static void process_ar_obj(struct elfcopy *ecp, struct ar_obj *obj) { struct stat sb; char *tempfile; int fd; /* Output to a temporary file. */ create_tempfile(&tempfile, &fd); if ((ecp->eout = elf_begin(fd, ELF_C_WRITE, NULL)) == NULL) errx(EXIT_FAILURE, "elf_begin() failed: %s", elf_errmsg(-1)); elf_flagelf(ecp->eout, ELF_C_SET, ELF_F_LAYOUT); create_elf(ecp); elf_end(ecp->ein); elf_end(ecp->eout); free(obj->buf); obj->buf = NULL; /* Extract archive symbols. */ if (lseek(fd, 0, SEEK_SET) < 0) err(EXIT_FAILURE, "lseek failed for '%s'", tempfile); if ((ecp->eout = elf_begin(fd, ELF_C_READ, NULL)) == NULL) errx(EXIT_FAILURE, "elf_begin() failed: %s", elf_errmsg(-1)); extract_arsym(ecp); elf_end(ecp->eout); if (fstat(fd, &sb) == -1) err(EXIT_FAILURE, "fstat %s failed", tempfile); if (lseek(fd, 0, SEEK_SET) < 0) err(EXIT_FAILURE, "lseek %s failed", tempfile); obj->size = sb.st_size; if ((obj->maddr = malloc(obj->size)) == NULL) err(EXIT_FAILURE, "memory allocation failed for '%s'", tempfile); if ((size_t) read(fd, obj->maddr, obj->size) != obj->size) err(EXIT_FAILURE, "read failed for '%s'", tempfile); if (unlink(tempfile)) err(EXIT_FAILURE, "unlink %s failed", tempfile); free(tempfile); close(fd); if (strlen(obj->name) > _MAXNAMELEN_SVR4) add_to_ar_str_table(ecp, obj->name); ecp->rela_off += _ARHDR_LEN + obj->size + obj->size % 2; STAILQ_INSERT_TAIL(&ecp->v_arobj, obj, objs); } /* * Append to the archive string table buffer. */ static void add_to_ar_str_table(struct elfcopy *ecp, const char *name) { if (ecp->as == NULL) { ecp->as_cap = _INIT_AS_CAP; ecp->as_sz = 0; if ((ecp->as = malloc(ecp->as_cap)) == NULL) err(EXIT_FAILURE, "malloc failed"); } /* * The space required for holding one member name in as table includes: * strlen(name) + (1 for '/') + (1 for '\n') + (possibly 1 for padding). */ while (ecp->as_sz + strlen(name) + 3 > ecp->as_cap) { ecp->as_cap *= 2; ecp->as = realloc(ecp->as, ecp->as_cap); if (ecp->as == NULL) err(EXIT_FAILURE, "realloc failed"); } strncpy(&ecp->as[ecp->as_sz], name, strlen(name)); ecp->as_sz += strlen(name); ecp->as[ecp->as_sz++] = '/'; ecp->as[ecp->as_sz++] = '\n'; } /* * Append to the archive symbol table buffer. */ static void add_to_ar_sym_table(struct elfcopy *ecp, const char *name) { if (ecp->s_so == NULL) { if ((ecp->s_so = malloc(_INIT_SYMOFF_CAP)) == NULL) err(EXIT_FAILURE, "malloc failed"); ecp->s_so_cap = _INIT_SYMOFF_CAP; ecp->s_cnt = 0; } if (ecp->s_sn == NULL) { if ((ecp->s_sn = malloc(_INIT_SYMNAME_CAP)) == NULL) err(EXIT_FAILURE, "malloc failed"); ecp->s_sn_cap = _INIT_SYMNAME_CAP; ecp->s_sn_sz = 0; } if (ecp->s_cnt * sizeof(uint32_t) >= ecp->s_so_cap) { ecp->s_so_cap *= 2; ecp->s_so = realloc(ecp->s_so, ecp->s_so_cap); if (ecp->s_so == NULL) err(EXIT_FAILURE, "realloc failed"); } ecp->s_so[ecp->s_cnt] = ecp->rela_off; ecp->s_cnt++; /* * The space required for holding one symbol name in sn table includes: * strlen(name) + (1 for '\n') + (possibly 1 for padding). */ while (ecp->s_sn_sz + strlen(name) + 2 > ecp->s_sn_cap) { ecp->s_sn_cap *= 2; ecp->s_sn = realloc(ecp->s_sn, ecp->s_sn_cap); if (ecp->s_sn == NULL) err(EXIT_FAILURE, "realloc failed"); } strncpy(&ecp->s_sn[ecp->s_sn_sz], name, strlen(name)); ecp->s_sn_sz += strlen(name); ecp->s_sn[ecp->s_sn_sz++] = '\0'; } static void sync_ar(struct elfcopy *ecp) { size_t s_sz; /* size of archive symbol table. */ size_t pm_sz; /* size of pseudo members */ int i; /* * Pad the symbol name string table. It is treated specially because * symbol name table should be padded by a '\0', not the common '\n' * for other members. The size of sn table includes the pad bit. */ if (ecp->s_cnt != 0 && ecp->s_sn_sz % 2 != 0) ecp->s_sn[ecp->s_sn_sz++] = '\0'; /* * Archive string table is padded by a "\n" as the normal members. * The difference is that the size of archive string table counts * in the pad bit, while normal members' size fileds do not. */ if (ecp->as != NULL && ecp->as_sz % 2 != 0) ecp->as[ecp->as_sz++] = '\n'; /* * If there is a symbol table, calculate the size of pseudo members, * convert previously stored relative offsets to absolute ones, and * then make them Big Endian. * * absolute_offset = htobe32(relative_offset + size_of_pseudo_members) */ if (ecp->s_cnt != 0) { s_sz = (ecp->s_cnt + 1) * sizeof(uint32_t) + ecp->s_sn_sz; pm_sz = _ARMAG_LEN + (_ARHDR_LEN + s_sz); if (ecp->as != NULL) pm_sz += _ARHDR_LEN + ecp->as_sz; for (i = 0; (size_t)i < ecp->s_cnt; i++) *(ecp->s_so + i) = htobe32(*(ecp->s_so + i) + pm_sz); } } /* * Extract global symbols from archive members. */ static void extract_arsym(struct elfcopy *ecp) { Elf_Scn *scn; GElf_Shdr shdr; GElf_Sym sym; Elf_Data *data; char *name; size_t n, shstrndx; int elferr, tabndx, len, i; if (elf_kind(ecp->eout) != ELF_K_ELF) { warnx("internal: cannot extract symbols from non-elf object"); return; } if (elf_getshstrndx(ecp->eout, &shstrndx) == 0) { warnx("elf_getshstrndx failed: %s", elf_errmsg(-1)); return; } tabndx = -1; scn = NULL; while ((scn = elf_nextscn(ecp->eout, scn)) != NULL) { if (gelf_getshdr(scn, &shdr) != &shdr) { warnx("elf_getshdr failed: %s", elf_errmsg(-1)); continue; } if ((name = elf_strptr(ecp->eout, shstrndx, shdr.sh_name)) == NULL) { warnx("elf_strptr failed: %s", elf_errmsg(-1)); continue; } if (strcmp(name, ".strtab") == 0) { tabndx = elf_ndxscn(scn); break; } } elferr = elf_errno(); if (elferr != 0) warnx("elf_nextscn failed: %s", elf_errmsg(elferr)); /* Ignore members without symbol table. */ if (tabndx == -1) return; scn = NULL; while ((scn = elf_nextscn(ecp->eout, scn)) != NULL) { if (gelf_getshdr(scn, &shdr) != &shdr) { warnx("elf_getshdr failed: %s", elf_errmsg(-1)); continue; } if (shdr.sh_type != SHT_SYMTAB) continue; data = NULL; n = 0; while (n < shdr.sh_size && (data = elf_getdata(scn, data)) != NULL) { len = data->d_size / shdr.sh_entsize; for (i = 0; i < len; i++) { if (gelf_getsym(data, i, &sym) != &sym) { warnx("gelf_getsym failed: %s", elf_errmsg(-1)); continue; } /* keep only global or weak symbols */ if (GELF_ST_BIND(sym.st_info) != STB_GLOBAL && GELF_ST_BIND(sym.st_info) != STB_WEAK) continue; /* keep only defined symbols */ if (sym.st_shndx == SHN_UNDEF) continue; if ((name = elf_strptr(ecp->eout, tabndx, sym.st_name)) == NULL) { warnx("elf_strptr failed: %s", elf_errmsg(-1)); continue; } add_to_ar_sym_table(ecp, name); } } } elferr = elf_errno(); if (elferr != 0) warnx("elf_nextscn failed: %s", elf_errmsg(elferr)); } #ifndef LIBELF_AR /* * Convenient wrapper for general libarchive error handling. */ #define AC(CALL) do { \ if ((CALL)) \ errx(EXIT_FAILURE, "%s", archive_error_string(a)); \ } while (0) /* Earlier versions of libarchive had some functions that returned 'void'. */ #if ARCHIVE_VERSION_NUMBER >= 2000000 #define ACV(CALL) AC(CALL) #else #define ACV(CALL) do { \ (CALL); \ } while (0) #endif int ac_detect_ar(int ifd) { struct archive *a; struct archive_entry *entry; int r; r = -1; if ((a = archive_read_new()) == NULL) return (0); archive_read_support_format_ar(a); if (archive_read_open_fd(a, ifd, 10240) == ARCHIVE_OK) r = archive_read_next_header(a, &entry); archive_read_close(a); archive_read_free(a); return (r == ARCHIVE_OK); } void ac_create_ar(struct elfcopy *ecp, int ifd, int ofd) { ac_read_objs(ecp, ifd); sync_ar(ecp); ac_write_objs(ecp, ofd); ac_write_cleanup(ecp); } static void ac_read_objs(struct elfcopy *ecp, int ifd) { struct archive *a; struct archive_entry *entry; struct ar_obj *obj; const char *name; char *buff; size_t size; int r; ecp->rela_off = 0; if (lseek(ifd, 0, SEEK_SET) == -1) err(EXIT_FAILURE, "lseek failed"); if ((a = archive_read_new()) == NULL) errx(EXIT_FAILURE, "archive_read_new failed"); archive_read_support_format_ar(a); AC(archive_read_open_fd(a, ifd, 10240)); for(;;) { r = archive_read_next_header(a, &entry); if (r == ARCHIVE_FATAL) errx(EXIT_FAILURE, "%s", archive_error_string(a)); if (r == ARCHIVE_EOF) break; if (r == ARCHIVE_WARN || r == ARCHIVE_RETRY) warnx("%s", archive_error_string(a)); if (r == ARCHIVE_RETRY) continue; name = archive_entry_pathname(entry); /* skip pseudo members. */ if (strcmp(name, "/") == 0 || strcmp(name, "//") == 0) continue; size = archive_entry_size(entry); if (size > 0) { if ((buff = malloc(size)) == NULL) err(EXIT_FAILURE, "malloc failed"); if (archive_read_data(a, buff, size) != (ssize_t)size) { warnx("%s", archive_error_string(a)); free(buff); continue; } if ((obj = malloc(sizeof(*obj))) == NULL) err(EXIT_FAILURE, "malloc failed"); if ((obj->name = strdup(name)) == NULL) err(EXIT_FAILURE, "strdup failed"); obj->buf = buff; obj->uid = archive_entry_uid(entry); obj->gid = archive_entry_gid(entry); obj->md = archive_entry_mode(entry); obj->mtime = archive_entry_mtime(entry); if ((ecp->ein = elf_memory(buff, size)) == NULL) errx(EXIT_FAILURE, "elf_memory() failed: %s", elf_errmsg(-1)); if (elf_kind(ecp->ein) != ELF_K_ELF) errx(EXIT_FAILURE, "file format not recognized"); process_ar_obj(ecp, obj); } } AC(archive_read_close(a)); ACV(archive_read_free(a)); } static void ac_write_objs(struct elfcopy *ecp, int ofd) { struct archive *a; struct archive_entry *entry; struct ar_obj *obj; + time_t timestamp; int nr; if ((a = archive_write_new()) == NULL) errx(EXIT_FAILURE, "archive_write_new failed"); archive_write_set_format_ar_svr4(a); AC(archive_write_open_fd(a, ofd)); /* Write the archive symbol table, even if it's empty. */ entry = archive_entry_new(); archive_entry_copy_pathname(entry, "/"); - archive_entry_set_mtime(entry, time(NULL), 0); + if (elftc_timestamp(×tamp) != 0) + err(EXIT_FAILURE, "elftc_timestamp"); + archive_entry_set_mtime(entry, timestamp, 0); archive_entry_set_size(entry, (ecp->s_cnt + 1) * sizeof(uint32_t) + ecp->s_sn_sz); AC(archive_write_header(a, entry)); nr = htobe32(ecp->s_cnt); ac_write_data(a, &nr, sizeof(uint32_t)); ac_write_data(a, ecp->s_so, sizeof(uint32_t) * ecp->s_cnt); ac_write_data(a, ecp->s_sn, ecp->s_sn_sz); archive_entry_free(entry); /* Write the archive string table, if exist. */ if (ecp->as != NULL) { entry = archive_entry_new(); archive_entry_copy_pathname(entry, "//"); archive_entry_set_size(entry, ecp->as_sz); AC(archive_write_header(a, entry)); ac_write_data(a, ecp->as, ecp->as_sz); archive_entry_free(entry); } /* Write normal members. */ STAILQ_FOREACH(obj, &ecp->v_arobj, objs) { entry = archive_entry_new(); archive_entry_copy_pathname(entry, obj->name); archive_entry_set_uid(entry, obj->uid); archive_entry_set_gid(entry, obj->gid); archive_entry_set_mode(entry, obj->md); archive_entry_set_size(entry, obj->size); archive_entry_set_mtime(entry, obj->mtime, 0); archive_entry_set_filetype(entry, AE_IFREG); AC(archive_write_header(a, entry)); ac_write_data(a, obj->maddr, obj->size); archive_entry_free(entry); } AC(archive_write_close(a)); ACV(archive_write_free(a)); } static void ac_write_cleanup(struct elfcopy *ecp) { struct ar_obj *obj, *obj_temp; STAILQ_FOREACH_SAFE(obj, &ecp->v_arobj, objs, obj_temp) { STAILQ_REMOVE(&ecp->v_arobj, obj, ar_obj, objs); if (obj->maddr != NULL) free(obj->maddr); free(obj->name); free(obj); } free(ecp->as); free(ecp->s_so); free(ecp->s_sn); ecp->as = NULL; ecp->s_so = NULL; ecp->s_sn = NULL; } /* * Wrapper for archive_write_data(). */ static void ac_write_data(struct archive *a, const void *buf, size_t s) { if (archive_write_data(a, buf, s) != (ssize_t)s) errx(EXIT_FAILURE, "%s", archive_error_string(a)); } #endif /* ! LIBELF_AR */ Index: projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/ascii.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/ascii.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/ascii.c (revision 305172) @@ -1,1079 +1,1079 @@ /*- * Copyright (c) 2010,2011 Kai Wang * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include "elfcopy.h" -ELFTC_VCSID("$Id: ascii.c 3446 2016-05-03 01:31:17Z emaste $"); +ELFTC_VCSID("$Id: ascii.c 3487 2016-08-24 18:12:08Z emaste $"); static void append_data(struct section *s, const void *buf, size_t sz); static char hex_digit(uint8_t n); static int hex_value(int x); static void finalize_data_section(struct section *s); static int ishexdigit(int x); static int ihex_read(const char *line, char *type, uint64_t *addr, uint64_t *num, uint8_t *data, size_t *sz); static void ihex_write(int ofd, int type, uint64_t addr, uint64_t num, const void *buf, size_t sz); static void ihex_write_00(int ofd, uint64_t addr, const void *buf, size_t sz); static void ihex_write_01(int ofd); static void ihex_write_04(int ofd, uint16_t addr); static void ihex_write_05(int ofd, uint64_t e_entry); static struct section *new_data_section(struct elfcopy *ecp, int sec_index, uint64_t off, uint64_t addr); static int read_num(const char *line, int *len, uint64_t *num, size_t sz, int *checksum); static int srec_read(const char *line, char *type, uint64_t *addr, uint8_t *data, size_t *sz); static void srec_write(int ofd, char type, uint64_t addr, const void *buf, size_t sz); static void srec_write_symtab(int ofd, const char *ofn, Elf *e, Elf_Scn *scn, GElf_Shdr *sh); static void srec_write_S0(int ofd, const char *ofn); static void srec_write_Sd(int ofd, char dr, uint64_t addr, const void *buf, size_t sz, size_t rlen); static void srec_write_Se(int ofd, uint64_t e_entry, int forceS3); static void write_num(char *line, int *len, uint64_t num, size_t sz, int *checksum); #define _LINE_BUFSZ 1024 #define _DATA_BUFSZ 256 /* * Convert ELF object to S-Record. */ void create_srec(struct elfcopy *ecp, int ifd, int ofd, const char *ofn) { Elf *e; Elf_Scn *scn; Elf_Data *d; GElf_Ehdr eh; GElf_Shdr sh; uint64_t max_addr; size_t rlen; int elferr, addr_sz; char dr; if ((e = elf_begin(ifd, ELF_C_READ, NULL)) == NULL) errx(EXIT_FAILURE, "elf_begin() failed: %s", elf_errmsg(-1)); /* Output a symbol table for `symbolsrec' target. */ if (!strncmp(ecp->otgt, "symbolsrec", strlen("symbolsrec"))) { scn = NULL; while ((scn = elf_nextscn(e, scn)) != NULL) { if (gelf_getshdr(scn, &sh) == NULL) { warnx("gelf_getshdr failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } if (sh.sh_type != SHT_SYMTAB) continue; srec_write_symtab(ofd, ofn, e, scn, &sh); break; } } if (ecp->flags & SREC_FORCE_S3) dr = '3'; else { /* * Find maximum address size in the first iteration. */ max_addr = 0; scn = NULL; while ((scn = elf_nextscn(e, scn)) != NULL) { if (gelf_getshdr(scn, &sh) == NULL) { warnx("gelf_getshdr failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } if ((sh.sh_flags & SHF_ALLOC) == 0 || sh.sh_type == SHT_NOBITS || sh.sh_size == 0) continue; if ((uint64_t) sh.sh_addr > max_addr) max_addr = sh.sh_addr; } elferr = elf_errno(); if (elferr != 0) warnx("elf_nextscn failed: %s", elf_errmsg(elferr)); if (max_addr <= 0xFFFF) dr = '1'; else if (max_addr <= 0xFFFFFF) dr = '2'; else dr = '3'; } if (ecp->flags & SREC_FORCE_LEN) { addr_sz = dr - '0' + 1; if (ecp->srec_len < 1) rlen = 1; else if (ecp->srec_len + addr_sz + 1 > 255) rlen = 255 - (addr_sz + 1); else rlen = ecp->srec_len; } else rlen = 16; /* Generate S0 record which contains the output filename. */ srec_write_S0(ofd, ofn); /* Generate S{1,2,3} data records for section data. */ scn = NULL; while ((scn = elf_nextscn(e, scn)) != NULL) { if (gelf_getshdr(scn, &sh) == NULL) { warnx("gelf_getshdr failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } if ((sh.sh_flags & SHF_ALLOC) == 0 || sh.sh_type == SHT_NOBITS || sh.sh_size == 0) continue; if (sh.sh_addr > 0xFFFFFFFF) { warnx("address space too big for S-Record file"); continue; } (void) elf_errno(); if ((d = elf_getdata(scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); continue; } if (d->d_buf == NULL || d->d_size == 0) continue; srec_write_Sd(ofd, dr, sh.sh_addr, d->d_buf, d->d_size, rlen); } elferr = elf_errno(); if (elferr != 0) warnx("elf_nextscn failed: %s", elf_errmsg(elferr)); /* Generate S{7,8,9} end of block record. */ if (gelf_getehdr(e, &eh) == NULL) errx(EXIT_FAILURE, "gelf_getehdr() failed: %s", elf_errmsg(-1)); srec_write_Se(ofd, eh.e_entry, ecp->flags & SREC_FORCE_S3); } void create_elf_from_srec(struct elfcopy *ecp, int ifd) { char line[_LINE_BUFSZ], name[_LINE_BUFSZ]; uint8_t data[_DATA_BUFSZ]; GElf_Ehdr oeh; struct section *s, *shtab; FILE *ifp; uint64_t addr, entry, off, sec_addr; uintmax_t st_value; size_t sz; int _ifd, first, sec_index, in_symtab, symtab_created; char *rlt; char type; if ((_ifd = dup(ifd)) < 0) err(EXIT_FAILURE, "dup failed"); if ((ifp = fdopen(_ifd, "r")) == NULL) err(EXIT_FAILURE, "fdopen failed"); /* Create EHDR for output .o file. */ if (gelf_newehdr(ecp->eout, ecp->oec) == NULL) errx(EXIT_FAILURE, "gelf_newehdr failed: %s", elf_errmsg(-1)); if (gelf_getehdr(ecp->eout, &oeh) == NULL) errx(EXIT_FAILURE, "gelf_getehdr() failed: %s", elf_errmsg(-1)); /* Initialise e_ident fields. */ oeh.e_ident[EI_CLASS] = ecp->oec; oeh.e_ident[EI_DATA] = ecp->oed; /* * TODO: Set OSABI according to the OS platform where elfcopy(1) * was build. (probably) */ oeh.e_ident[EI_OSABI] = ELFOSABI_NONE; oeh.e_machine = ecp->oem; oeh.e_type = ET_REL; oeh.e_entry = 0; ecp->flags |= RELOCATABLE; /* Create .shstrtab section */ init_shstrtab(ecp); ecp->shstrtab->off = 0; /* Data sections are inserted after EHDR. */ off = gelf_fsize(ecp->eout, ELF_T_EHDR, 1, EV_CURRENT); if (off == 0) errx(EXIT_FAILURE, "gelf_fsize() failed: %s", elf_errmsg(-1)); /* Create data sections. */ s = NULL; first = 1; sec_index = 1; sec_addr = entry = 0; while (fgets(line, _LINE_BUFSZ, ifp) != NULL) { - sz = 0; /* Silence GCC 5.3 uninitialized variable warning */ + sz = 0; if (line[0] == '\r' || line[0] == '\n') continue; if (line[0] == '$' && line[1] == '$') { ecp->flags |= SYMTAB_EXIST; while ((rlt = fgets(line, _LINE_BUFSZ, ifp)) != NULL) { if (line[0] == '$' && line[1] == '$') break; } if (rlt == NULL) break; continue; } if (line[0] != 'S' || line[1] < '0' || line[1] > '9') { warnx("Invalid srec record"); continue; } if (srec_read(line, &type, &addr, data, &sz) < 0) { warnx("Invalid srec record or mismatched checksum"); continue; } switch (type) { case '1': case '2': case '3': if (sz == 0) break; if (first || sec_addr != addr) { if (s != NULL) finalize_data_section(s); s = new_data_section(ecp, sec_index, off, addr); if (s == NULL) { warnx("new_data_section failed"); break; } sec_index++; sec_addr = addr; first = 0; } append_data(s, data, sz); off += sz; sec_addr += sz; break; case '7': case '8': case '9': entry = addr; break; default: break; } } if (s != NULL) finalize_data_section(s); if (ferror(ifp)) warn("fgets failed"); /* Insert .shstrtab after data sections. */ if ((ecp->shstrtab->os = elf_newscn(ecp->eout)) == NULL) errx(EXIT_FAILURE, "elf_newscn failed: %s", elf_errmsg(-1)); insert_to_sec_list(ecp, ecp->shstrtab, 1); /* Insert section header table here. */ shtab = insert_shtab(ecp, 1); /* * Rescan and create symbol table if we found '$$' section in * the first scan. */ symtab_created = 0; in_symtab = 0; if (ecp->flags & SYMTAB_EXIST) { if (fseek(ifp, 0, SEEK_SET) < 0) { warn("fseek failed"); ecp->flags &= ~SYMTAB_EXIST; goto done; } while (fgets(line, _LINE_BUFSZ, ifp) != NULL) { if (in_symtab) { if (line[0] == '$' && line[1] == '$') { in_symtab = 0; continue; } if (sscanf(line, "%s $%jx", name, &st_value) != 2) { warnx("Invalid symbolsrec record"); continue; } if (!symtab_created) { create_external_symtab(ecp); symtab_created = 1; } add_to_symtab(ecp, name, st_value, 0, SHN_ABS, ELF32_ST_INFO(STB_GLOBAL, STT_NOTYPE), 0, 1); } if (line[0] == '$' && line[1] == '$') { in_symtab = 1; continue; } } } if (ferror(ifp)) warn("fgets failed"); if (symtab_created) { finalize_external_symtab(ecp); create_symtab_data(ecp); /* Count in .symtab and .strtab section headers. */ shtab->sz += gelf_fsize(ecp->eout, ELF_T_SHDR, 2, EV_CURRENT); } else ecp->flags &= ~SYMTAB_EXIST; done: fclose(ifp); /* Set entry point. */ oeh.e_entry = entry; /* * Write the underlying ehdr. Note that it should be called * before elf_setshstrndx() since it will overwrite e->e_shstrndx. */ if (gelf_update_ehdr(ecp->eout, &oeh) == 0) errx(EXIT_FAILURE, "gelf_update_ehdr() failed: %s", elf_errmsg(-1)); /* Generate section name string table (.shstrtab). */ set_shstrtab(ecp); /* Update sh_name pointer for each section header entry. */ update_shdr(ecp, 0); /* Renew oeh to get the updated e_shstrndx. */ if (gelf_getehdr(ecp->eout, &oeh) == NULL) errx(EXIT_FAILURE, "gelf_getehdr() failed: %s", elf_errmsg(-1)); /* Resync section offsets. */ resync_sections(ecp); /* Store SHDR offset in EHDR. */ oeh.e_shoff = shtab->off; /* Update ehdr since we modified e_shoff. */ if (gelf_update_ehdr(ecp->eout, &oeh) == 0) errx(EXIT_FAILURE, "gelf_update_ehdr() failed: %s", elf_errmsg(-1)); /* Write out the output elf object. */ if (elf_update(ecp->eout, ELF_C_WRITE) < 0) errx(EXIT_FAILURE, "elf_update() failed: %s", elf_errmsg(-1)); /* Release allocated resource. */ free_elf(ecp); } void create_ihex(int ifd, int ofd) { Elf *e; Elf_Scn *scn; Elf_Data *d; GElf_Ehdr eh; GElf_Shdr sh; int elferr; uint16_t addr_hi, old_addr_hi; if ((e = elf_begin(ifd, ELF_C_READ, NULL)) == NULL) errx(EXIT_FAILURE, "elf_begin() failed: %s", elf_errmsg(-1)); old_addr_hi = 0; scn = NULL; while ((scn = elf_nextscn(e, scn)) != NULL) { if (gelf_getshdr(scn, &sh) == NULL) { warnx("gelf_getshdr failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } if ((sh.sh_flags & SHF_ALLOC) == 0 || sh.sh_type == SHT_NOBITS || sh.sh_size == 0) continue; if (sh.sh_addr > 0xFFFFFFFF) { warnx("address space too big for Intel Hex file"); continue; } (void) elf_errno(); if ((d = elf_getdata(scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); continue; } if (d->d_buf == NULL || d->d_size == 0) continue; addr_hi = (sh.sh_addr >> 16) & 0xFFFF; if (addr_hi > 0 && addr_hi != old_addr_hi) { /* Write 04 record if addr_hi is new. */ old_addr_hi = addr_hi; ihex_write_04(ofd, addr_hi); } ihex_write_00(ofd, sh.sh_addr, d->d_buf, d->d_size); } elferr = elf_errno(); if (elferr != 0) warnx("elf_nextscn failed: %s", elf_errmsg(elferr)); if (gelf_getehdr(e, &eh) == NULL) errx(EXIT_FAILURE, "gelf_getehdr() failed: %s", elf_errmsg(-1)); ihex_write_05(ofd, eh.e_entry); ihex_write_01(ofd); } void create_elf_from_ihex(struct elfcopy *ecp, int ifd) { char line[_LINE_BUFSZ]; uint8_t data[_DATA_BUFSZ]; GElf_Ehdr oeh; struct section *s, *shtab; FILE *ifp; uint64_t addr, addr_base, entry, num, off, rec_addr, sec_addr; size_t sz; int _ifd, first, sec_index; char type; if ((_ifd = dup(ifd)) < 0) err(EXIT_FAILURE, "dup failed"); if ((ifp = fdopen(_ifd, "r")) == NULL) err(EXIT_FAILURE, "fdopen failed"); /* Create EHDR for output .o file. */ if (gelf_newehdr(ecp->eout, ecp->oec) == NULL) errx(EXIT_FAILURE, "gelf_newehdr failed: %s", elf_errmsg(-1)); if (gelf_getehdr(ecp->eout, &oeh) == NULL) errx(EXIT_FAILURE, "gelf_getehdr() failed: %s", elf_errmsg(-1)); /* Initialise e_ident fields. */ oeh.e_ident[EI_CLASS] = ecp->oec; oeh.e_ident[EI_DATA] = ecp->oed; /* * TODO: Set OSABI according to the OS platform where elfcopy(1) * was build. (probably) */ oeh.e_ident[EI_OSABI] = ELFOSABI_NONE; oeh.e_machine = ecp->oem; oeh.e_type = ET_REL; oeh.e_entry = 0; ecp->flags |= RELOCATABLE; /* Create .shstrtab section */ init_shstrtab(ecp); ecp->shstrtab->off = 0; /* Data sections are inserted after EHDR. */ off = gelf_fsize(ecp->eout, ELF_T_EHDR, 1, EV_CURRENT); if (off == 0) errx(EXIT_FAILURE, "gelf_fsize() failed: %s", elf_errmsg(-1)); /* Create data sections. */ s = NULL; first = 1; sec_index = 1; addr_base = rec_addr = sec_addr = entry = 0; while (fgets(line, _LINE_BUFSZ, ifp) != NULL) { if (line[0] == '\r' || line[0] == '\n') continue; if (line[0] != ':') { warnx("Invalid ihex record"); continue; } if (ihex_read(line, &type, &addr, &num, data, &sz) < 0) { warnx("Invalid ihex record or mismatched checksum"); continue; } switch (type) { case '0': /* Data record. */ if (sz == 0) break; rec_addr = addr_base + addr; if (first || sec_addr != rec_addr) { if (s != NULL) finalize_data_section(s); s = new_data_section(ecp, sec_index, off, rec_addr); if (s == NULL) { warnx("new_data_section failed"); break; } sec_index++; sec_addr = rec_addr; first = 0; } append_data(s, data, sz); off += sz; sec_addr += sz; break; case '1': /* End of file record. */ goto done; case '2': /* Extended segment address record. */ addr_base = addr << 4; break; case '3': /* Start segment address record (CS:IP). Ignored. */ break; case '4': /* Extended linear address record. */ addr_base = num << 16; break; case '5': /* Start linear address record. */ entry = num; break; default: break; } } done: if (s != NULL) finalize_data_section(s); if (ferror(ifp)) warn("fgets failed"); fclose(ifp); /* Insert .shstrtab after data sections. */ if ((ecp->shstrtab->os = elf_newscn(ecp->eout)) == NULL) errx(EXIT_FAILURE, "elf_newscn failed: %s", elf_errmsg(-1)); insert_to_sec_list(ecp, ecp->shstrtab, 1); /* Insert section header table here. */ shtab = insert_shtab(ecp, 1); /* Set entry point. */ oeh.e_entry = entry; /* * Write the underlying ehdr. Note that it should be called * before elf_setshstrndx() since it will overwrite e->e_shstrndx. */ if (gelf_update_ehdr(ecp->eout, &oeh) == 0) errx(EXIT_FAILURE, "gelf_update_ehdr() failed: %s", elf_errmsg(-1)); /* Generate section name string table (.shstrtab). */ set_shstrtab(ecp); /* Update sh_name pointer for each section header entry. */ update_shdr(ecp, 0); /* Renew oeh to get the updated e_shstrndx. */ if (gelf_getehdr(ecp->eout, &oeh) == NULL) errx(EXIT_FAILURE, "gelf_getehdr() failed: %s", elf_errmsg(-1)); /* Resync section offsets. */ resync_sections(ecp); /* Store SHDR offset in EHDR. */ oeh.e_shoff = shtab->off; /* Update ehdr since we modified e_shoff. */ if (gelf_update_ehdr(ecp->eout, &oeh) == 0) errx(EXIT_FAILURE, "gelf_update_ehdr() failed: %s", elf_errmsg(-1)); /* Write out the output elf object. */ if (elf_update(ecp->eout, ELF_C_WRITE) < 0) errx(EXIT_FAILURE, "elf_update() failed: %s", elf_errmsg(-1)); /* Release allocated resource. */ free_elf(ecp); } #define _SEC_NAMESZ 64 #define _SEC_INIT_CAP 1024 static struct section * new_data_section(struct elfcopy *ecp, int sec_index, uint64_t off, uint64_t addr) { char *name; if ((name = malloc(_SEC_NAMESZ)) == NULL) errx(EXIT_FAILURE, "malloc failed"); snprintf(name, _SEC_NAMESZ, ".sec%d", sec_index); return (create_external_section(ecp, name, name, NULL, 0, off, SHT_PROGBITS, ELF_T_BYTE, SHF_ALLOC | SHF_WRITE, 1, addr, 0)); } static void finalize_data_section(struct section *s) { Elf_Data *od; if ((od = elf_newdata(s->os)) == NULL) errx(EXIT_FAILURE, "elf_newdata() failed: %s", elf_errmsg(-1)); od->d_align = s->align; od->d_off = 0; od->d_buf = s->buf; od->d_size = s->sz; od->d_version = EV_CURRENT; } static void append_data(struct section *s, const void *buf, size_t sz) { uint8_t *p; if (s->buf == NULL) { s->sz = 0; s->cap = _SEC_INIT_CAP; if ((s->buf = malloc(s->cap)) == NULL) err(EXIT_FAILURE, "malloc failed"); } while (sz + s->sz > s->cap) { s->cap *= 2; if ((s->buf = realloc(s->buf, s->cap)) == NULL) err(EXIT_FAILURE, "realloc failed"); } p = s->buf; memcpy(&p[s->sz], buf, sz); s->sz += sz; } static int srec_read(const char *line, char *type, uint64_t *addr, uint8_t *data, size_t *sz) { uint64_t count, _checksum, num; size_t addr_sz; int checksum, i, len; checksum = 0; len = 2; if (read_num(line, &len, &count, 1, &checksum) < 0) return (-1); *type = line[1]; switch (*type) { case '0': case '1': case '5': case '9': addr_sz = 2; break; case '2': case '8': addr_sz = 3; break; case '3': case '7': addr_sz = 4; break; default: return (-1); } if (read_num(line, &len, addr, addr_sz, &checksum) < 0) return (-1); count -= addr_sz + 1; if (*type >= '0' && *type <= '3') { for (i = 0; (uint64_t) i < count; i++) { if (read_num(line, &len, &num, 1, &checksum) < 0) return -1; data[i] = (uint8_t) num; } *sz = count; } else *sz = 0; if (read_num(line, &len, &_checksum, 1, NULL) < 0) return (-1); if ((int) _checksum != (~checksum & 0xFF)) return (-1); return (0); } static void srec_write_symtab(int ofd, const char *ofn, Elf *e, Elf_Scn *scn, GElf_Shdr *sh) { char line[_LINE_BUFSZ]; GElf_Sym sym; Elf_Data *d; const char *name; size_t sc; int elferr, i; #define _WRITE_LINE do { \ if (write(ofd, line, strlen(line)) != (ssize_t) strlen(line)) \ errx(EXIT_FAILURE, "write failed"); \ } while (0) (void) elf_errno(); if ((d = elf_getdata(scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); return; } if (d->d_buf == NULL || d->d_size == 0) return; snprintf(line, sizeof(line), "$$ %s\r\n", ofn); _WRITE_LINE; sc = d->d_size / sh->sh_entsize; for (i = 1; (size_t) i < sc; i++) { if (gelf_getsym(d, i, &sym) != &sym) { warnx("gelf_getsym failed: %s", elf_errmsg(-1)); continue; } if (GELF_ST_TYPE(sym.st_info) == STT_SECTION || GELF_ST_TYPE(sym.st_info) == STT_FILE) continue; if ((name = elf_strptr(e, sh->sh_link, sym.st_name)) == NULL) { warnx("elf_strptr failed: %s", elf_errmsg(-1)); continue; } snprintf(line, sizeof(line), " %s $%jx\r\n", name, (uintmax_t) sym.st_value); _WRITE_LINE; } snprintf(line, sizeof(line), "$$ \r\n"); _WRITE_LINE; #undef _WRITE_LINE } static void srec_write_S0(int ofd, const char *ofn) { srec_write(ofd, '0', 0, ofn, strlen(ofn)); } static void srec_write_Sd(int ofd, char dr, uint64_t addr, const void *buf, size_t sz, size_t rlen) { const uint8_t *p, *pe; p = buf; pe = p + sz; while (pe - p >= (int) rlen) { srec_write(ofd, dr, addr, p, rlen); addr += rlen; p += rlen; } if (pe - p > 0) srec_write(ofd, dr, addr, p, pe - p); } static void srec_write_Se(int ofd, uint64_t e_entry, int forceS3) { char er; if (e_entry > 0xFFFFFFFF) { warnx("address space too big for S-Record file"); return; } if (forceS3) er = '7'; else { if (e_entry <= 0xFFFF) er = '9'; else if (e_entry <= 0xFFFFFF) er = '8'; else er = '7'; } srec_write(ofd, er, e_entry, NULL, 0); } static void srec_write(int ofd, char type, uint64_t addr, const void *buf, size_t sz) { char line[_LINE_BUFSZ]; const uint8_t *p, *pe; int len, addr_sz, checksum; if (type == '0' || type == '1' || type == '5' || type == '9') addr_sz = 2; else if (type == '2' || type == '8') addr_sz = 3; else addr_sz = 4; checksum = 0; line[0] = 'S'; line[1] = type; len = 2; write_num(line, &len, addr_sz + sz + 1, 1, &checksum); write_num(line, &len, addr, addr_sz, &checksum); for (p = buf, pe = p + sz; p < pe; p++) write_num(line, &len, *p, 1, &checksum); write_num(line, &len, ~checksum & 0xFF, 1, NULL); line[len++] = '\r'; line[len++] = '\n'; if (write(ofd, line, len) != (ssize_t) len) err(EXIT_FAILURE, "write failed"); } static void ihex_write_00(int ofd, uint64_t addr, const void *buf, size_t sz) { uint16_t addr_hi, old_addr_hi; const uint8_t *p, *pe; old_addr_hi = (addr >> 16) & 0xFFFF; p = buf; pe = p + sz; while (pe - p >= 16) { ihex_write(ofd, 0, addr, 0, p, 16); addr += 16; p += 16; addr_hi = (addr >> 16) & 0xFFFF; if (addr_hi != old_addr_hi) { old_addr_hi = addr_hi; ihex_write_04(ofd, addr_hi); } } if (pe - p > 0) ihex_write(ofd, 0, addr, 0, p, pe - p); } static int ihex_read(const char *line, char *type, uint64_t *addr, uint64_t *num, uint8_t *data, size_t *sz) { uint64_t count, _checksum; int checksum, i, len; *sz = 0; checksum = 0; len = 1; if (read_num(line, &len, &count, 1, &checksum) < 0) return (-1); if (read_num(line, &len, addr, 2, &checksum) < 0) return (-1); if (line[len++] != '0') return (-1); *type = line[len++]; checksum += *type - '0'; switch (*type) { case '0': for (i = 0; (uint64_t) i < count; i++) { if (read_num(line, &len, num, 1, &checksum) < 0) return (-1); data[i] = (uint8_t) *num; } *sz = count; break; case '1': if (count != 0) return (-1); break; case '2': case '4': if (count != 2) return (-1); if (read_num(line, &len, num, 2, &checksum) < 0) return (-1); break; case '3': case '5': if (count != 4) return (-1); if (read_num(line, &len, num, 4, &checksum) < 0) return (-1); break; default: return (-1); } if (read_num(line, &len, &_checksum, 1, &checksum) < 0) return (-1); if ((checksum & 0xFF) != 0) { return (-1); } return (0); } static void ihex_write_01(int ofd) { ihex_write(ofd, 1, 0, 0, NULL, 0); } static void ihex_write_04(int ofd, uint16_t addr) { ihex_write(ofd, 4, 0, addr, NULL, 2); } static void ihex_write_05(int ofd, uint64_t e_entry) { if (e_entry > 0xFFFFFFFF) { warnx("address space too big for Intel Hex file"); return; } ihex_write(ofd, 5, 0, e_entry, NULL, 4); } static void ihex_write(int ofd, int type, uint64_t addr, uint64_t num, const void *buf, size_t sz) { char line[_LINE_BUFSZ]; const uint8_t *p, *pe; int len, checksum; if (sz > 16) errx(EXIT_FAILURE, "Internal: ihex_write() sz too big"); checksum = 0; line[0] = ':'; len = 1; write_num(line, &len, sz, 1, &checksum); write_num(line, &len, addr, 2, &checksum); write_num(line, &len, type, 1, &checksum); if (sz > 0) { if (buf != NULL) { for (p = buf, pe = p + sz; p < pe; p++) write_num(line, &len, *p, 1, &checksum); } else write_num(line, &len, num, sz, &checksum); } write_num(line, &len, (~checksum + 1) & 0xFF, 1, NULL); line[len++] = '\r'; line[len++] = '\n'; if (write(ofd, line, len) != (ssize_t) len) err(EXIT_FAILURE, "write failed"); } static int read_num(const char *line, int *len, uint64_t *num, size_t sz, int *checksum) { uint8_t b; *num = 0; for (; sz > 0; sz--) { if (!ishexdigit(line[*len]) || !ishexdigit(line[*len + 1])) return (-1); b = (hex_value(line[*len]) << 4) | hex_value(line[*len + 1]); *num = (*num << 8) | b; *len += 2; if (checksum != NULL) *checksum = (*checksum + b) & 0xFF; } return (0); } static void write_num(char *line, int *len, uint64_t num, size_t sz, int *checksum) { uint8_t b; for (; sz > 0; sz--) { b = (num >> ((sz - 1) * 8)) & 0xFF; line[*len] = hex_digit((b >> 4) & 0xF); line[*len + 1] = hex_digit(b & 0xF); *len += 2; if (checksum != NULL) *checksum = (*checksum + b) & 0xFF; } } static char hex_digit(uint8_t n) { return ((n < 10) ? '0' + n : 'A' + (n - 10)); } static int hex_value(int x) { if (isdigit(x)) return (x - '0'); else if (x >= 'a' && x <= 'f') return (x - 'a' + 10); else return (x - 'A' + 10); } static int ishexdigit(int x) { if (isdigit(x)) return (1); if ((x >= 'a' && x <= 'f') || (x >= 'A' && x <= 'F')) return (1); return (0); } Index: projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/pe.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/pe.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/elfcopy/pe.c (revision 305172) @@ -1,231 +1,234 @@ /*- * Copyright (c) 2016 Kai Wang * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include "elfcopy.h" -ELFTC_VCSID("$Id: pe.c 3477 2016-05-25 20:00:42Z kaiwang27 $"); +ELFTC_VCSID("$Id: pe.c 3490 2016-08-31 00:12:22Z emaste $"); /* Convert ELF object to Portable Executable (PE). */ void create_pe(struct elfcopy *ecp, int ifd, int ofd) { Elf *e; Elf_Scn *scn; Elf_Data *d; GElf_Ehdr eh; GElf_Shdr sh; PE *pe; PE_Scn *ps; PE_SecHdr psh; PE_CoffHdr pch; PE_OptHdr poh; PE_Object po; PE_Buffer *pb; const char *name; size_t indx; + time_t timestamp; int elferr; if (ecp->otf == ETF_EFI || ecp->oem == EM_X86_64) po = PE_O_PE32P; else po = PE_O_PE32; if ((e = elf_begin(ifd, ELF_C_READ, NULL)) == NULL) errx(EXIT_FAILURE, "elf_begin() failed: %s", elf_errmsg(-1)); if (gelf_getehdr(e, &eh) == NULL) errx(EXIT_FAILURE, "gelf_getehdr() failed: %s", elf_errmsg(-1)); if (elf_getshstrndx(ecp->ein, &indx) == 0) errx(EXIT_FAILURE, "elf_getshstrndx() failed: %s", elf_errmsg(-1)); if ((pe = pe_init(ofd, PE_C_WRITE, po)) == NULL) err(EXIT_FAILURE, "pe_init() failed"); /* Setup PE COFF header. */ memset(&pch, 0, sizeof(pch)); switch (ecp->oem) { case EM_386: pch.ch_machine = IMAGE_FILE_MACHINE_I386; break; case EM_X86_64: pch.ch_machine = IMAGE_FILE_MACHINE_AMD64; break; default: pch.ch_machine = IMAGE_FILE_MACHINE_UNKNOWN; break; } - pch.ch_timestamp = (uint32_t) time(NULL); + if (elftc_timestamp(×tamp) != 0) + err(EXIT_FAILURE, "elftc_timestamp"); + pch.ch_timestamp = (uint32_t) timestamp; if (pe_update_coff_header(pe, &pch) < 0) err(EXIT_FAILURE, "pe_update_coff_header() failed"); /* Setup PE optional header. */ memset(&poh, 0, sizeof(poh)); if (ecp->otf == ETF_EFI) poh.oh_subsystem = IMAGE_SUBSYSTEM_EFI_APPLICATION; poh.oh_entry = (uint32_t) eh.e_entry; /* * Default section alignment and file alignment. (Here the * section alignment is set to the default page size of the * archs supported. We should use different section alignment * for some arch. (e.g. IA64) */ poh.oh_secalign = 0x1000; poh.oh_filealign = 0x200; /* Copy sections. */ scn = NULL; while ((scn = elf_nextscn(e, scn)) != NULL) { /* * Read in ELF section. */ if (gelf_getshdr(scn, &sh) == NULL) { warnx("gelf_getshdr() failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } if ((name = elf_strptr(ecp->ein, indx, sh.sh_name)) == NULL) { warnx("elf_strptr() failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } /* Skip sections unneeded. */ if (strcmp(name, ".shstrtab") == 0 || strcmp(name, ".symtab") == 0 || strcmp(name, ".strtab") == 0) continue; if ((d = elf_getdata(scn, NULL)) == NULL) { warnx("elf_getdata() failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } if (strcmp(name, ".text") == 0) { poh.oh_textbase = (uint32_t) sh.sh_addr; poh.oh_textsize = (uint32_t) roundup(sh.sh_size, poh.oh_filealign); } else { if (po == PE_O_PE32 && strcmp(name, ".data") == 0) poh.oh_database = sh.sh_addr; if (sh.sh_type == SHT_NOBITS) poh.oh_bsssize += (uint32_t) roundup(sh.sh_size, poh.oh_filealign); else if (sh.sh_flags & SHF_ALLOC) poh.oh_datasize += (uint32_t) roundup(sh.sh_size, poh.oh_filealign); } /* * Create PE/COFF section. */ if ((ps = pe_newscn(pe)) == NULL) { warn("pe_newscn() failed"); continue; } /* * Setup PE/COFF section header. The section name is not * NUL-terminated if its length happens to be 8. Long * section name should be truncated for PE image according * to the PE/COFF specification. */ memset(&psh, 0, sizeof(psh)); strncpy(psh.sh_name, name, sizeof(psh.sh_name)); psh.sh_addr = sh.sh_addr; psh.sh_virtsize = sh.sh_size; if (sh.sh_type != SHT_NOBITS) psh.sh_rawsize = roundup(sh.sh_size, poh.oh_filealign); else psh.sh_char |= IMAGE_SCN_CNT_UNINITIALIZED_DATA; /* * Translate ELF section flags to PE/COFF section flags. */ psh.sh_char |= IMAGE_SCN_MEM_READ; if (sh.sh_flags & SHF_WRITE) psh.sh_char |= IMAGE_SCN_MEM_WRITE; if (sh.sh_flags & SHF_EXECINSTR) psh.sh_char |= IMAGE_SCN_MEM_EXECUTE | IMAGE_SCN_CNT_CODE; if ((sh.sh_flags & SHF_ALLOC) && (psh.sh_char & 0xF0) == 0) psh.sh_char |= IMAGE_SCN_CNT_INITIALIZED_DATA; /* Mark relocation section "discardable". */ if (strcmp(name, ".reloc") == 0) psh.sh_char |= IMAGE_SCN_MEM_DISCARDABLE; if (pe_update_section_header(ps, &psh) < 0) { warn("pe_update_section_header() failed"); continue; } /* Copy section content. */ if ((pb = pe_newbuffer(ps)) == NULL) { warn("pe_newbuffer() failed"); continue; } pb->pb_align = 1; pb->pb_off = 0; pb->pb_size = roundup(sh.sh_size, poh.oh_filealign); if ((pb->pb_buf = calloc(1, pb->pb_size)) == NULL) { warn("calloc failed"); continue; } memcpy(pb->pb_buf, d->d_buf, sh.sh_size); } elferr = elf_errno(); if (elferr != 0) warnx("elf_nextscn() failed: %s", elf_errmsg(elferr)); /* Update PE optional header. */ if (pe_update_opt_header(pe, &poh) < 0) err(EXIT_FAILURE, "pe_update_opt_header() failed"); /* Write out PE/COFF object. */ if (pe_update(pe) < 0) err(EXIT_FAILURE, "pe_update() failed"); pe_finish(pe); elf_end(e); } Index: projects/netbsd-tests-update-12/contrib/elftoolchain/elfdump/elfdump.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/elfdump/elfdump.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/elfdump/elfdump.c (revision 305172) @@ -1,2678 +1,2680 @@ /*- * Copyright (c) 2007-2012 Kai Wang * Copyright (c) 2003 David O'Brien. All rights reserved. * Copyright (c) 2001 Jake Burkholder * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef USE_LIBARCHIVE_AR #include #include #endif #include "_elftc.h" -ELFTC_VCSID("$Id: elfdump.c 3474 2016-05-17 20:44:53Z emaste $"); +ELFTC_VCSID("$Id: elfdump.c 3482 2016-08-02 18:47:00Z emaste $"); #if defined(ELFTC_NEED_ELF_NOTE_DEFINITION) #include "native-elf-format.h" #if ELFTC_CLASS == ELFCLASS32 typedef Elf32_Nhdr Elf_Note; #else typedef Elf64_Nhdr Elf_Note; #endif #endif /* elfdump(1) options. */ #define ED_DYN (1<<0) #define ED_EHDR (1<<1) #define ED_GOT (1<<2) #define ED_HASH (1<<3) #define ED_INTERP (1<<4) #define ED_NOTE (1<<5) #define ED_PHDR (1<<6) #define ED_REL (1<<7) #define ED_SHDR (1<<8) #define ED_SYMTAB (1<<9) #define ED_SYMVER (1<<10) #define ED_CHECKSUM (1<<11) #define ED_ALL ((1<<12)-1) /* elfdump(1) run control flags. */ #define SOLARIS_FMT (1<<0) #define PRINT_FILENAME (1<<1) #define PRINT_ARSYM (1<<2) #define ONLY_ARSYM (1<<3) /* Convenient print macro. */ #define PRT(...) fprintf(ed->out, __VA_ARGS__) /* Internal data structure for sections. */ struct section { const char *name; /* section name */ Elf_Scn *scn; /* section scn */ uint64_t off; /* section offset */ uint64_t sz; /* section size */ uint64_t entsize; /* section entsize */ uint64_t align; /* section alignment */ uint64_t type; /* section type */ uint64_t flags; /* section flags */ uint64_t addr; /* section virtual addr */ uint32_t link; /* section link ndx */ uint32_t info; /* section info ndx */ }; struct spec_name { const char *name; STAILQ_ENTRY(spec_name) sn_list; }; /* Structure encapsulates the global data for readelf(1). */ struct elfdump { FILE *out; /* output redirection. */ const char *filename; /* current processing file. */ const char *archive; /* archive name */ int options; /* command line options. */ int flags; /* run control flags. */ Elf *elf; /* underlying ELF descriptor. */ #ifndef USE_LIBARCHIVE_AR Elf *ar; /* ar(1) archive descriptor. */ #endif GElf_Ehdr ehdr; /* ELF header. */ int ec; /* ELF class. */ size_t shnum; /* #sections. */ struct section *sl; /* list of sections. */ STAILQ_HEAD(, spec_name) snl; /* list of names specified by -N. */ }; /* Relocation entry. */ struct rel_entry { union { GElf_Rel rel; GElf_Rela rela; } u_r; const char *symn; uint32_t type; }; #if defined(ELFTC_NEED_BYTEORDER_EXTENSIONS) static __inline uint32_t be32dec(const void *pp) { unsigned char const *p = (unsigned char const *)pp; return ((p[0] << 24) | (p[1] << 16) | (p[2] << 8) | p[3]); } static __inline uint32_t le32dec(const void *pp) { unsigned char const *p = (unsigned char const *)pp; return ((p[3] << 24) | (p[2] << 16) | (p[1] << 8) | p[0]); } #endif /* http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#tag_encodings */ static const char * d_tags(uint64_t tag) { static char unknown_buf[64]; switch (tag) { case DT_NULL: return "DT_NULL"; case DT_NEEDED: return "DT_NEEDED"; case DT_PLTRELSZ: return "DT_PLTRELSZ"; case DT_PLTGOT: return "DT_PLTGOT"; case DT_HASH: return "DT_HASH"; case DT_STRTAB: return "DT_STRTAB"; case DT_SYMTAB: return "DT_SYMTAB"; case DT_RELA: return "DT_RELA"; case DT_RELASZ: return "DT_RELASZ"; case DT_RELAENT: return "DT_RELAENT"; case DT_STRSZ: return "DT_STRSZ"; case DT_SYMENT: return "DT_SYMENT"; case DT_INIT: return "DT_INIT"; case DT_FINI: return "DT_FINI"; case DT_SONAME: return "DT_SONAME"; case DT_RPATH: return "DT_RPATH"; case DT_SYMBOLIC: return "DT_SYMBOLIC"; case DT_REL: return "DT_REL"; case DT_RELSZ: return "DT_RELSZ"; case DT_RELENT: return "DT_RELENT"; case DT_PLTREL: return "DT_PLTREL"; case DT_DEBUG: return "DT_DEBUG"; case DT_TEXTREL: return "DT_TEXTREL"; case DT_JMPREL: return "DT_JMPREL"; case DT_BIND_NOW: return "DT_BIND_NOW"; case DT_INIT_ARRAY: return "DT_INIT_ARRAY"; case DT_FINI_ARRAY: return "DT_FINI_ARRAY"; case DT_INIT_ARRAYSZ: return "DT_INIT_ARRAYSZ"; case DT_FINI_ARRAYSZ: return "DT_FINI_ARRAYSZ"; case DT_RUNPATH: return "DT_RUNPATH"; case DT_FLAGS: return "DT_FLAGS"; case DT_PREINIT_ARRAY: return "DT_PREINIT_ARRAY"; /* XXX DT_ENCODING */ case DT_PREINIT_ARRAYSZ:return "DT_PREINIT_ARRAYSZ"; /* 0x6000000D - 0x6ffff000 operating system-specific semantics */ case 0x6ffffdf5: return "DT_GNU_PRELINKED"; case 0x6ffffdf6: return "DT_GNU_CONFLICTSZ"; case 0x6ffffdf7: return "DT_GNU_LIBLISTSZ"; case 0x6ffffdf8: return "DT_SUNW_CHECKSUM"; case DT_PLTPADSZ: return "DT_PLTPADSZ"; case DT_MOVEENT: return "DT_MOVEENT"; case DT_MOVESZ: return "DT_MOVESZ"; case 0x6ffffdfc: return "DT_FEATURE"; case DT_POSFLAG_1: return "DT_POSFLAG_1"; case DT_SYMINSZ: return "DT_SYMINSZ"; case DT_SYMINENT: return "DT_SYMINENT (DT_VALRNGHI)"; case DT_ADDRRNGLO: return "DT_ADDRRNGLO"; case DT_GNU_HASH: return "DT_GNU_HASH"; case 0x6ffffef8: return "DT_GNU_CONFLICT"; case 0x6ffffef9: return "DT_GNU_LIBLIST"; case 0x6ffffefa: return "DT_CONFIG"; case 0x6ffffefb: return "DT_DEPAUDIT"; case 0x6ffffefc: return "DT_AUDIT"; case 0x6ffffefd: return "DT_PLTPAD"; case 0x6ffffefe: return "DT_MOVETAB"; case DT_SYMINFO: return "DT_SYMINFO (DT_ADDRRNGHI)"; case DT_RELACOUNT: return "DT_RELACOUNT"; case DT_RELCOUNT: return "DT_RELCOUNT"; case DT_FLAGS_1: return "DT_FLAGS_1"; case DT_VERDEF: return "DT_VERDEF"; case DT_VERDEFNUM: return "DT_VERDEFNUM"; case DT_VERNEED: return "DT_VERNEED"; case DT_VERNEEDNUM: return "DT_VERNEEDNUM"; case 0x6ffffff0: return "DT_GNU_VERSYM"; /* 0x70000000 - 0x7fffffff processor-specific semantics */ case 0x70000000: return "DT_IA_64_PLT_RESERVE"; case 0x7ffffffd: return "DT_SUNW_AUXILIARY"; case 0x7ffffffe: return "DT_SUNW_USED"; case 0x7fffffff: return "DT_SUNW_FILTER"; } snprintf(unknown_buf, sizeof(unknown_buf), "", (unsigned long long)tag); return (unknown_buf); } static const char * e_machines(unsigned int mach) { static char machdesc[64]; switch (mach) { case EM_NONE: return "EM_NONE"; case EM_M32: return "EM_M32"; case EM_SPARC: return "EM_SPARC"; case EM_386: return "EM_386"; case EM_68K: return "EM_68K"; case EM_88K: return "EM_88K"; case EM_IAMCU: return "EM_IAMCU"; case EM_860: return "EM_860"; case EM_MIPS: return "EM_MIPS"; case EM_PPC: return "EM_PPC"; case EM_PPC64: return "EM_PPC64"; case EM_ARM: return "EM_ARM"; case EM_ALPHA: return "EM_ALPHA (legacy)"; case EM_SPARCV9:return "EM_SPARCV9"; case EM_IA_64: return "EM_IA_64"; case EM_X86_64: return "EM_X86_64"; case EM_AARCH64:return "EM_AARCH64"; case EM_RISCV: return "EM_RISCV"; } snprintf(machdesc, sizeof(machdesc), "(unknown machine) -- type 0x%x", mach); return (machdesc); } static const char * elf_type_str(unsigned int type) { static char s_type[32]; switch (type) { case ET_NONE: return "ET_NONE"; case ET_REL: return "ET_REL"; case ET_EXEC: return "ET_EXEC"; case ET_DYN: return "ET_DYN"; case ET_CORE: return "ET_CORE"; } if (type >= ET_LOPROC) snprintf(s_type, sizeof(s_type), "", type); else if (type >= ET_LOOS && type <= ET_HIOS) snprintf(s_type, sizeof(s_type), "", type); else snprintf(s_type, sizeof(s_type), "", ver); return (s_ver); } static const char * elf_class_str(unsigned int class) { static char s_class[32]; switch (class) { case ELFCLASSNONE: return "ELFCLASSNONE"; case ELFCLASS32: return "ELFCLASS32"; case ELFCLASS64: return "ELFCLASS64"; } snprintf(s_class, sizeof(s_class), "", class); return (s_class); } static const char * elf_data_str(unsigned int data) { static char s_data[32]; switch (data) { case ELFDATANONE: return "ELFDATANONE"; case ELFDATA2LSB: return "ELFDATA2LSB"; case ELFDATA2MSB: return "ELFDATA2MSB"; } snprintf(s_data, sizeof(s_data), "", data); return (s_data); } static const char *ei_abis[256] = { "ELFOSABI_NONE", "ELFOSABI_HPUX", "ELFOSABI_NETBSD", "ELFOSABI_LINUX", "ELFOSABI_HURD", "ELFOSABI_86OPEN", "ELFOSABI_SOLARIS", "ELFOSABI_AIX", "ELFOSABI_IRIX", "ELFOSABI_FREEBSD", "ELFOSABI_TRU64", "ELFOSABI_MODESTO", "ELFOSABI_OPENBSD", [17] = "ELFOSABI_CLOUDABI", + [64] = "ELFOSABI_ARM_AEABI", + [97] = "ELFOSABI_ARM", [255] = "ELFOSABI_STANDALONE" }; static const char * elf_phdr_type_str(unsigned int type) { static char s_type[32]; switch (type) { case PT_NULL: return "PT_NULL"; case PT_LOAD: return "PT_LOAD"; case PT_DYNAMIC: return "PT_DYNAMIC"; case PT_INTERP: return "PT_INTERP"; case PT_NOTE: return "PT_NOTE"; case PT_SHLIB: return "PT_SHLIB"; case PT_PHDR: return "PT_PHDR"; case PT_TLS: return "PT_TLS"; case PT_GNU_EH_FRAME: return "PT_GNU_EH_FRAME"; case PT_GNU_STACK: return "PT_GNU_STACK"; case PT_GNU_RELRO: return "PT_GNU_RELRO"; } snprintf(s_type, sizeof(s_type), "", type); return (s_type); } static const char *p_flags[] = { "", "PF_X", "PF_W", "PF_X|PF_W", "PF_R", "PF_X|PF_R", "PF_W|PF_R", "PF_X|PF_W|PF_R" }; static const char * sh_name(struct elfdump *ed, int ndx) { static char num[10]; switch (ndx) { case SHN_UNDEF: return "UNDEF"; case SHN_ABS: return "ABS"; case SHN_COMMON: return "COMMON"; default: if ((uint64_t)ndx < ed->shnum) return (ed->sl[ndx].name); else { snprintf(num, sizeof(num), "%d", ndx); return (num); } } } /* http://www.sco.com/developers/gabi/latest/ch4.sheader.html#sh_type */ static const char * sh_types(uint64_t mach, uint64_t sht) { static char unknown_buf[64]; if (sht < 0x60000000) { switch (sht) { case SHT_NULL: return "SHT_NULL"; case SHT_PROGBITS: return "SHT_PROGBITS"; case SHT_SYMTAB: return "SHT_SYMTAB"; case SHT_STRTAB: return "SHT_STRTAB"; case SHT_RELA: return "SHT_RELA"; case SHT_HASH: return "SHT_HASH"; case SHT_DYNAMIC: return "SHT_DYNAMIC"; case SHT_NOTE: return "SHT_NOTE"; case SHT_NOBITS: return "SHT_NOBITS"; case SHT_REL: return "SHT_REL"; case SHT_SHLIB: return "SHT_SHLIB"; case SHT_DYNSYM: return "SHT_DYNSYM"; case SHT_INIT_ARRAY: return "SHT_INIT_ARRAY"; case SHT_FINI_ARRAY: return "SHT_FINI_ARRAY"; case SHT_PREINIT_ARRAY: return "SHT_PREINIT_ARRAY"; case SHT_GROUP: return "SHT_GROUP"; case SHT_SYMTAB_SHNDX: return "SHT_SYMTAB_SHNDX"; } } else if (sht < 0x70000000) { /* 0x60000000-0x6fffffff operating system-specific semantics */ switch (sht) { case 0x6ffffff0: return "XXX:VERSYM"; case SHT_SUNW_dof: return "SHT_SUNW_dof"; case SHT_GNU_HASH: return "SHT_GNU_HASH"; case 0x6ffffff7: return "SHT_GNU_LIBLIST"; case 0x6ffffffc: return "XXX:VERDEF"; case SHT_SUNW_verdef: return "SHT_SUNW(GNU)_verdef"; case SHT_SUNW_verneed: return "SHT_SUNW(GNU)_verneed"; case SHT_SUNW_versym: return "SHT_SUNW(GNU)_versym"; } } else if (sht < 0x80000000) { /* 0x70000000 - 0x7fffffff processor-specific semantics */ switch (mach) { case EM_ARM: switch (sht) { case SHT_ARM_EXIDX: return "SHT_ARM_EXIDX"; case SHT_ARM_PREEMPTMAP: return "SHT_ARM_PREEMPTMAP"; case SHT_ARM_ATTRIBUTES: return "SHT_ARM_ATTRIBUTES"; case SHT_ARM_DEBUGOVERLAY: return "SHT_ARM_DEBUGOVERLAY"; case SHT_ARM_OVERLAYSECTION: return "SHT_ARM_OVERLAYSECTION"; } break; case EM_IA_64: switch (sht) { case 0x70000000: return "SHT_IA_64_EXT"; case 0x70000001: return "SHT_IA_64_UNWIND"; } break; case EM_MIPS: switch (sht) { case SHT_MIPS_REGINFO: return "SHT_MIPS_REGINFO"; case SHT_MIPS_OPTIONS: return "SHT_MIPS_OPTIONS"; case SHT_MIPS_ABIFLAGS: return "SHT_MIPS_ABIFLAGS"; } break; } switch (sht) { case 0x7ffffffd: return "XXX:AUXILIARY"; case 0x7fffffff: return "XXX:FILTER"; } } /* 0x80000000 - 0xffffffff application programs */ snprintf(unknown_buf, sizeof(unknown_buf), "", (unsigned long long)sht); return (unknown_buf); } /* * Define known section flags. These flags are defined in the order * they are to be printed out. */ #define DEFINE_SHFLAGS() \ DEFINE_SHF(WRITE) \ DEFINE_SHF(ALLOC) \ DEFINE_SHF(EXECINSTR) \ DEFINE_SHF(MERGE) \ DEFINE_SHF(STRINGS) \ DEFINE_SHF(INFO_LINK) \ DEFINE_SHF(LINK_ORDER) \ DEFINE_SHF(OS_NONCONFORMING) \ DEFINE_SHF(GROUP) \ DEFINE_SHF(TLS) \ DEFINE_SHF(COMPRESSED) #undef DEFINE_SHF #define DEFINE_SHF(F) "SHF_" #F "|" #define ALLSHFLAGS DEFINE_SHFLAGS() static const char * sh_flags(uint64_t shf) { static char flg[sizeof(ALLSHFLAGS)+1]; flg[0] = '\0'; #undef DEFINE_SHF #define DEFINE_SHF(N) \ if (shf & SHF_##N) \ strcat(flg, "SHF_" #N "|"); \ DEFINE_SHFLAGS() flg[strlen(flg) - 1] = '\0'; /* Remove the trailing "|". */ return (flg); } static const char * st_type(unsigned int mach, unsigned int type) { static char s_type[32]; switch (type) { case STT_NOTYPE: return "STT_NOTYPE"; case STT_OBJECT: return "STT_OBJECT"; case STT_FUNC: return "STT_FUNC"; case STT_SECTION: return "STT_SECTION"; case STT_FILE: return "STT_FILE"; case STT_COMMON: return "STT_COMMON"; case STT_TLS: return "STT_TLS"; case 13: if (mach == EM_SPARCV9) return "STT_SPARC_REGISTER"; break; } snprintf(s_type, sizeof(s_type), "", type); return (s_type); } static const char * st_type_S(unsigned int type) { static char s_type[32]; switch (type) { case STT_NOTYPE: return "NOTY"; case STT_OBJECT: return "OBJT"; case STT_FUNC: return "FUNC"; case STT_SECTION: return "SECT"; case STT_FILE: return "FILE"; } snprintf(s_type, sizeof(s_type), "", type); return (s_type); } static const char * st_bindings(unsigned int sbind) { static char s_sbind[32]; switch (sbind) { case STB_LOCAL: return "STB_LOCAL"; case STB_GLOBAL: return "STB_GLOBAL"; case STB_WEAK: return "STB_WEAK"; case STB_GNU_UNIQUE: return "STB_GNU_UNIQUE"; default: if (sbind >= STB_LOOS && sbind <= STB_HIOS) return "OS"; else if (sbind >= STB_LOPROC && sbind <= STB_HIPROC) return "PROC"; else snprintf(s_sbind, sizeof(s_sbind), "", sbind); return (s_sbind); } } static const char * st_bindings_S(unsigned int sbind) { static char s_sbind[32]; switch (sbind) { case STB_LOCAL: return "LOCL"; case STB_GLOBAL: return "GLOB"; case STB_WEAK: return "WEAK"; case STB_GNU_UNIQUE: return "UNIQ"; default: if (sbind >= STB_LOOS && sbind <= STB_HIOS) return "OS"; else if (sbind >= STB_LOPROC && sbind <= STB_HIPROC) return "PROC"; else snprintf(s_sbind, sizeof(s_sbind), "<%#x>", sbind); return (s_sbind); } } static unsigned char st_others[] = { 'D', 'I', 'H', 'P' }; static void add_name(struct elfdump *ed, const char *name); static void elf_print_object(struct elfdump *ed); static void elf_print_elf(struct elfdump *ed); static void elf_print_ehdr(struct elfdump *ed); static void elf_print_phdr(struct elfdump *ed); static void elf_print_shdr(struct elfdump *ed); static void elf_print_symtab(struct elfdump *ed, int i); static void elf_print_symtabs(struct elfdump *ed); static void elf_print_symver(struct elfdump *ed); static void elf_print_verdef(struct elfdump *ed, struct section *s); static void elf_print_verneed(struct elfdump *ed, struct section *s); static void elf_print_interp(struct elfdump *ed); static void elf_print_dynamic(struct elfdump *ed); static void elf_print_rel_entry(struct elfdump *ed, struct section *s, int j, struct rel_entry *r); static void elf_print_rela(struct elfdump *ed, struct section *s, Elf_Data *data); static void elf_print_rel(struct elfdump *ed, struct section *s, Elf_Data *data); static void elf_print_reloc(struct elfdump *ed); static void elf_print_got(struct elfdump *ed); static void elf_print_got_section(struct elfdump *ed, struct section *s); static void elf_print_note(struct elfdump *ed); static void elf_print_svr4_hash(struct elfdump *ed, struct section *s); static void elf_print_svr4_hash64(struct elfdump *ed, struct section *s); static void elf_print_gnu_hash(struct elfdump *ed, struct section *s); static void elf_print_hash(struct elfdump *ed); static void elf_print_checksum(struct elfdump *ed); static void find_gotrel(struct elfdump *ed, struct section *gs, struct rel_entry *got); static struct spec_name *find_name(struct elfdump *ed, const char *name); static int get_ent_count(const struct section *s, int *ent_count); static const char *get_symbol_name(struct elfdump *ed, uint32_t symtab, int i); static const char *get_string(struct elfdump *ed, int strtab, size_t off); static void get_versym(struct elfdump *ed, int i, uint16_t **vs, int *nvs); static void load_sections(struct elfdump *ed); static void unload_sections(struct elfdump *ed); static void usage(void); #ifdef USE_LIBARCHIVE_AR static int ac_detect_ar(int fd); static void ac_print_ar(struct elfdump *ed, int fd); #else static void elf_print_ar(struct elfdump *ed, int fd); #endif /* USE_LIBARCHIVE_AR */ static struct option elfdump_longopts[] = { { "help", no_argument, NULL, 'H' }, { "version", no_argument, NULL, 'V' }, { NULL, 0, NULL, 0 } }; int main(int ac, char **av) { struct elfdump *ed, ed_storage; struct spec_name *sn; int ch, i; ed = &ed_storage; memset(ed, 0, sizeof(*ed)); STAILQ_INIT(&ed->snl); ed->out = stdout; while ((ch = getopt_long(ac, av, "acdeiGHhknN:prsSvVw:", elfdump_longopts, NULL)) != -1) switch (ch) { case 'a': ed->options = ED_ALL; break; case 'c': ed->options |= ED_SHDR; break; case 'd': ed->options |= ED_DYN; break; case 'e': ed->options |= ED_EHDR; break; case 'i': ed->options |= ED_INTERP; break; case 'G': ed->options |= ED_GOT; break; case 'h': ed->options |= ED_HASH; break; case 'k': ed->options |= ED_CHECKSUM; break; case 'n': ed->options |= ED_NOTE; break; case 'N': add_name(ed, optarg); break; case 'p': ed->options |= ED_PHDR; break; case 'r': ed->options |= ED_REL; break; case 's': ed->options |= ED_SYMTAB; break; case 'S': ed->flags |= SOLARIS_FMT; break; case 'v': ed->options |= ED_SYMVER; break; case 'V': (void) printf("%s (%s)\n", ELFTC_GETPROGNAME(), elftc_version()); exit(EXIT_SUCCESS); break; case 'w': if ((ed->out = fopen(optarg, "w")) == NULL) err(EXIT_FAILURE, "%s", optarg); break; case '?': case 'H': default: usage(); } ac -= optind; av += optind; if (ed->options == 0) ed->options = ED_ALL; sn = NULL; if (ed->options & ED_SYMTAB && (STAILQ_EMPTY(&ed->snl) || (sn = find_name(ed, "ARSYM")) != NULL)) { ed->flags |= PRINT_ARSYM; if (sn != NULL) { STAILQ_REMOVE(&ed->snl, sn, spec_name, sn_list); if (STAILQ_EMPTY(&ed->snl)) ed->flags |= ONLY_ARSYM; } } if (ac == 0) usage(); if (ac > 1) ed->flags |= PRINT_FILENAME; if (elf_version(EV_CURRENT) == EV_NONE) errx(EXIT_FAILURE, "ELF library initialization failed: %s", elf_errmsg(-1)); for (i = 0; i < ac; i++) { ed->filename = av[i]; ed->archive = NULL; elf_print_object(ed); } exit(EXIT_SUCCESS); } #ifdef USE_LIBARCHIVE_AR /* Archive symbol table entry. */ struct arsym_entry { char *sym_name; size_t off; }; /* * Convenient wrapper for general libarchive error handling. */ #define AC(CALL) do { \ if ((CALL)) { \ warnx("%s", archive_error_string(a)); \ return; \ } \ } while (0) /* * Detect an ar(1) archive using libarchive(3). */ static int ac_detect_ar(int fd) { struct archive *a; struct archive_entry *entry; int r; r = -1; if ((a = archive_read_new()) == NULL) return (0); archive_read_support_format_ar(a); if (archive_read_open_fd(a, fd, 10240) == ARCHIVE_OK) r = archive_read_next_header(a, &entry); archive_read_close(a); archive_read_free(a); return (r == ARCHIVE_OK); } /* * Dump an ar(1) archive using libarchive(3). */ static void ac_print_ar(struct elfdump *ed, int fd) { struct archive *a; struct archive_entry *entry; struct arsym_entry *arsym; const char *name; char idx[10], *b; void *buff; size_t size; uint32_t cnt, i; int r; if (lseek(fd, 0, SEEK_SET) == -1) err(EXIT_FAILURE, "lseek failed"); if ((a = archive_read_new()) == NULL) errx(EXIT_FAILURE, "%s", archive_error_string(a)); archive_read_support_format_ar(a); AC(archive_read_open_fd(a, fd, 10240)); for(;;) { r = archive_read_next_header(a, &entry); if (r == ARCHIVE_FATAL) errx(EXIT_FAILURE, "%s", archive_error_string(a)); if (r == ARCHIVE_EOF) break; if (r == ARCHIVE_WARN || r == ARCHIVE_RETRY) warnx("%s", archive_error_string(a)); if (r == ARCHIVE_RETRY) continue; name = archive_entry_pathname(entry); size = archive_entry_size(entry); if (size == 0) continue; if ((buff = malloc(size)) == NULL) { warn("malloc failed"); continue; } if (archive_read_data(a, buff, size) != (ssize_t)size) { warnx("%s", archive_error_string(a)); free(buff); continue; } /* * Note that when processing arsym via libarchive, there is * no way to tell which member a certain symbol belongs to, * since we can not just "lseek" to a member offset and read * the member header. */ if (!strcmp(name, "/") && ed->flags & PRINT_ARSYM) { b = buff; cnt = be32dec(b); if (cnt == 0) { free(buff); continue; } arsym = calloc(cnt, sizeof(*arsym)); if (arsym == NULL) err(EXIT_FAILURE, "calloc failed"); b += sizeof(uint32_t); for (i = 0; i < cnt; i++) { arsym[i].off = be32dec(b); b += sizeof(uint32_t); } for (i = 0; i < cnt; i++) { arsym[i].sym_name = b; b += strlen(b) + 1; } if (ed->flags & SOLARIS_FMT) { PRT("\nSymbol Table: (archive)\n"); PRT(" index offset symbol\n"); } else PRT("\nsymbol table (archive):\n"); for (i = 0; i < cnt; i++) { if (ed->flags & SOLARIS_FMT) { snprintf(idx, sizeof(idx), "[%d]", i); PRT("%10s ", idx); PRT("0x%8.8jx ", (uintmax_t)arsym[i].off); PRT("%s\n", arsym[i].sym_name); } else { PRT("\nentry: %d\n", i); PRT("\toffset: %#jx\n", (uintmax_t)arsym[i].off); PRT("\tsymbol: %s\n", arsym[i].sym_name); } } free(arsym); free(buff); /* No need to continue if we only dump ARSYM. */ if (ed->flags & ONLY_ARSYM) { AC(archive_read_close(a)); AC(archive_read_free(a)); return; } continue; } if ((ed->elf = elf_memory(buff, size)) == NULL) { warnx("elf_memroy() failed: %s", elf_errmsg(-1)); free(buff); continue; } /* Skip non-ELF member. */ if (elf_kind(ed->elf) == ELF_K_ELF) { printf("\n%s(%s):\n", ed->archive, name); elf_print_elf(ed); } elf_end(ed->elf); free(buff); } AC(archive_read_close(a)); AC(archive_read_free(a)); } #else /* USE_LIBARCHIVE_AR */ /* * Dump an ar(1) archive. */ static void elf_print_ar(struct elfdump *ed, int fd) { Elf *e; Elf_Arhdr *arh; Elf_Arsym *arsym; Elf_Cmd cmd; char idx[10]; size_t cnt, i; ed->ar = ed->elf; if (ed->flags & PRINT_ARSYM) { cnt = 0; if ((arsym = elf_getarsym(ed->ar, &cnt)) == NULL) { warnx("elf_getarsym failed: %s", elf_errmsg(-1)); goto print_members; } if (cnt == 0) goto print_members; if (ed->flags & SOLARIS_FMT) { PRT("\nSymbol Table: (archive)\n"); PRT(" index offset member name and symbol\n"); } else PRT("\nsymbol table (archive):\n"); for (i = 0; i < cnt - 1; i++) { if (elf_rand(ed->ar, arsym[i].as_off) != arsym[i].as_off) { warnx("elf_rand failed: %s", elf_errmsg(-1)); break; } if ((e = elf_begin(fd, ELF_C_READ, ed->ar)) == NULL) { warnx("elf_begin failed: %s", elf_errmsg(-1)); break; } if ((arh = elf_getarhdr(e)) == NULL) { warnx("elf_getarhdr failed: %s", elf_errmsg(-1)); break; } if (ed->flags & SOLARIS_FMT) { snprintf(idx, sizeof(idx), "[%zu]", i); PRT("%10s ", idx); PRT("0x%8.8jx ", (uintmax_t)arsym[i].as_off); PRT("(%s):%s\n", arh->ar_name, arsym[i].as_name); } else { PRT("\nentry: %zu\n", i); PRT("\toffset: %#jx\n", (uintmax_t)arsym[i].as_off); PRT("\tmember: %s\n", arh->ar_name); PRT("\tsymbol: %s\n", arsym[i].as_name); } elf_end(e); } /* No need to continue if we only dump ARSYM. */ if (ed->flags & ONLY_ARSYM) return; } print_members: /* Rewind the archive. */ if (elf_rand(ed->ar, SARMAG) != SARMAG) { warnx("elf_rand failed: %s", elf_errmsg(-1)); return; } /* Dump each member of the archive. */ cmd = ELF_C_READ; while ((ed->elf = elf_begin(fd, cmd, ed->ar)) != NULL) { /* Skip non-ELF member. */ if (elf_kind(ed->elf) == ELF_K_ELF) { if ((arh = elf_getarhdr(ed->elf)) == NULL) { warnx("elf_getarhdr failed: %s", elf_errmsg(-1)); break; } printf("\n%s(%s):\n", ed->archive, arh->ar_name); elf_print_elf(ed); } cmd = elf_next(ed->elf); elf_end(ed->elf); } } #endif /* USE_LIBARCHIVE_AR */ /* * Dump an object. (ELF object or ar(1) archive) */ static void elf_print_object(struct elfdump *ed) { int fd; if ((fd = open(ed->filename, O_RDONLY)) == -1) { warn("open %s failed", ed->filename); return; } #ifdef USE_LIBARCHIVE_AR if (ac_detect_ar(fd)) { ed->archive = ed->filename; ac_print_ar(ed, fd); return; } #endif /* USE_LIBARCHIVE_AR */ if ((ed->elf = elf_begin(fd, ELF_C_READ, NULL)) == NULL) { warnx("elf_begin() failed: %s", elf_errmsg(-1)); return; } switch (elf_kind(ed->elf)) { case ELF_K_NONE: warnx("Not an ELF file."); return; case ELF_K_ELF: if (ed->flags & PRINT_FILENAME) printf("\n%s:\n", ed->filename); elf_print_elf(ed); break; case ELF_K_AR: #ifndef USE_LIBARCHIVE_AR ed->archive = ed->filename; elf_print_ar(ed, fd); #endif break; default: warnx("Internal: libelf returned unknown elf kind."); return; } elf_end(ed->elf); } /* * Dump an ELF object. */ static void elf_print_elf(struct elfdump *ed) { if (gelf_getehdr(ed->elf, &ed->ehdr) == NULL) { warnx("gelf_getehdr failed: %s", elf_errmsg(-1)); return; } if ((ed->ec = gelf_getclass(ed->elf)) == ELFCLASSNONE) { warnx("gelf_getclass failed: %s", elf_errmsg(-1)); return; } if (ed->options & (ED_SHDR | ED_DYN | ED_REL | ED_GOT | ED_SYMTAB | ED_SYMVER | ED_NOTE | ED_HASH)) load_sections(ed); if (ed->options & ED_EHDR) elf_print_ehdr(ed); if (ed->options & ED_PHDR) elf_print_phdr(ed); if (ed->options & ED_INTERP) elf_print_interp(ed); if (ed->options & ED_SHDR) elf_print_shdr(ed); if (ed->options & ED_DYN) elf_print_dynamic(ed); if (ed->options & ED_REL) elf_print_reloc(ed); if (ed->options & ED_GOT) elf_print_got(ed); if (ed->options & ED_SYMTAB) elf_print_symtabs(ed); if (ed->options & ED_SYMVER) elf_print_symver(ed); if (ed->options & ED_NOTE) elf_print_note(ed); if (ed->options & ED_HASH) elf_print_hash(ed); if (ed->options & ED_CHECKSUM) elf_print_checksum(ed); unload_sections(ed); } /* * Read the section headers from ELF object and store them in the * internal cache. */ static void load_sections(struct elfdump *ed) { struct section *s; const char *name; Elf_Scn *scn; GElf_Shdr sh; size_t shstrndx, ndx; int elferr; assert(ed->sl == NULL); if (!elf_getshnum(ed->elf, &ed->shnum)) { warnx("elf_getshnum failed: %s", elf_errmsg(-1)); return; } if (ed->shnum == 0) return; if ((ed->sl = calloc(ed->shnum, sizeof(*ed->sl))) == NULL) err(EXIT_FAILURE, "calloc failed"); if (!elf_getshstrndx(ed->elf, &shstrndx)) { warnx("elf_getshstrndx failed: %s", elf_errmsg(-1)); return; } if ((scn = elf_getscn(ed->elf, 0)) == NULL) { warnx("elf_getscn failed: %s", elf_errmsg(-1)); return; } (void) elf_errno(); do { if (gelf_getshdr(scn, &sh) == NULL) { warnx("gelf_getshdr failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } if ((name = elf_strptr(ed->elf, shstrndx, sh.sh_name)) == NULL) { (void) elf_errno(); name = "ERROR"; } if ((ndx = elf_ndxscn(scn)) == SHN_UNDEF) if ((elferr = elf_errno()) != 0) { warnx("elf_ndxscn failed: %s", elf_errmsg(elferr)); continue; } if (ndx >= ed->shnum) { warnx("section index of '%s' out of range", name); continue; } s = &ed->sl[ndx]; s->name = name; s->scn = scn; s->off = sh.sh_offset; s->sz = sh.sh_size; s->entsize = sh.sh_entsize; s->align = sh.sh_addralign; s->type = sh.sh_type; s->flags = sh.sh_flags; s->addr = sh.sh_addr; s->link = sh.sh_link; s->info = sh.sh_info; } while ((scn = elf_nextscn(ed->elf, scn)) != NULL); elferr = elf_errno(); if (elferr != 0) warnx("elf_nextscn failed: %s", elf_errmsg(elferr)); } /* * Release section related resources. */ static void unload_sections(struct elfdump *ed) { if (ed->sl != NULL) { free(ed->sl); ed->sl = NULL; } } /* * Add a name to the '-N' name list. */ static void add_name(struct elfdump *ed, const char *name) { struct spec_name *sn; if (find_name(ed, name)) return; if ((sn = malloc(sizeof(*sn))) == NULL) { warn("malloc failed"); return; } sn->name = name; STAILQ_INSERT_TAIL(&ed->snl, sn, sn_list); } /* * Lookup a name in the '-N' name list. */ static struct spec_name * find_name(struct elfdump *ed, const char *name) { struct spec_name *sn; STAILQ_FOREACH(sn, &ed->snl, sn_list) { if (!strcmp(sn->name, name)) return (sn); } return (NULL); } /* * Retrieve the name of a symbol using the section index of the symbol * table and the index of the symbol within that table. */ static const char * get_symbol_name(struct elfdump *ed, uint32_t symtab, int i) { static char sname[64]; struct section *s; const char *name; GElf_Sym sym; Elf_Data *data; int elferr; if (symtab >= ed->shnum) return (""); s = &ed->sl[symtab]; if (s->type != SHT_SYMTAB && s->type != SHT_DYNSYM) return (""); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return (""); } if (gelf_getsym(data, i, &sym) != &sym) return (""); if (GELF_ST_TYPE(sym.st_info) == STT_SECTION) { if (sym.st_shndx < ed->shnum) { snprintf(sname, sizeof(sname), "%s (section)", ed->sl[sym.st_shndx].name); return (sname); } else return (""); } if ((name = elf_strptr(ed->elf, s->link, sym.st_name)) == NULL) return (""); return (name); } /* * Retrieve a string using string table section index and the string offset. */ static const char* get_string(struct elfdump *ed, int strtab, size_t off) { const char *name; if ((name = elf_strptr(ed->elf, strtab, off)) == NULL) return (""); return (name); } /* * Dump the ELF Executable Header. */ static void elf_print_ehdr(struct elfdump *ed) { if (!STAILQ_EMPTY(&ed->snl)) return; if (ed->flags & SOLARIS_FMT) { PRT("\nELF Header\n"); PRT(" ei_magic: { %#x, %c, %c, %c }\n", ed->ehdr.e_ident[0], ed->ehdr.e_ident[1], ed->ehdr.e_ident[2], ed->ehdr.e_ident[3]); PRT(" ei_class: %-18s", elf_class_str(ed->ehdr.e_ident[EI_CLASS])); PRT(" ei_data: %s\n", elf_data_str(ed->ehdr.e_ident[EI_DATA])); PRT(" e_machine: %-18s", e_machines(ed->ehdr.e_machine)); PRT(" e_version: %s\n", elf_version_str(ed->ehdr.e_version)); PRT(" e_type: %s\n", elf_type_str(ed->ehdr.e_type)); PRT(" e_flags: %18d\n", ed->ehdr.e_flags); PRT(" e_entry: %#18jx", (uintmax_t)ed->ehdr.e_entry); PRT(" e_ehsize: %6d", ed->ehdr.e_ehsize); PRT(" e_shstrndx:%5d\n", ed->ehdr.e_shstrndx); PRT(" e_shoff: %#18jx", (uintmax_t)ed->ehdr.e_shoff); PRT(" e_shentsize: %3d", ed->ehdr.e_shentsize); PRT(" e_shnum: %5d\n", ed->ehdr.e_shnum); PRT(" e_phoff: %#18jx", (uintmax_t)ed->ehdr.e_phoff); PRT(" e_phentsize: %3d", ed->ehdr.e_phentsize); PRT(" e_phnum: %5d\n", ed->ehdr.e_phnum); } else { PRT("\nelf header:\n"); PRT("\n"); PRT("\te_ident: %s %s %s\n", elf_class_str(ed->ehdr.e_ident[EI_CLASS]), elf_data_str(ed->ehdr.e_ident[EI_DATA]), ei_abis[ed->ehdr.e_ident[EI_OSABI]]); PRT("\te_type: %s\n", elf_type_str(ed->ehdr.e_type)); PRT("\te_machine: %s\n", e_machines(ed->ehdr.e_machine)); PRT("\te_version: %s\n", elf_version_str(ed->ehdr.e_version)); PRT("\te_entry: %#jx\n", (uintmax_t)ed->ehdr.e_entry); PRT("\te_phoff: %ju\n", (uintmax_t)ed->ehdr.e_phoff); PRT("\te_shoff: %ju\n", (uintmax_t) ed->ehdr.e_shoff); PRT("\te_flags: %u\n", ed->ehdr.e_flags); PRT("\te_ehsize: %u\n", ed->ehdr.e_ehsize); PRT("\te_phentsize: %u\n", ed->ehdr.e_phentsize); PRT("\te_phnum: %u\n", ed->ehdr.e_phnum); PRT("\te_shentsize: %u\n", ed->ehdr.e_shentsize); PRT("\te_shnum: %u\n", ed->ehdr.e_shnum); PRT("\te_shstrndx: %u\n", ed->ehdr.e_shstrndx); } } /* * Dump the ELF Program Header Table. */ static void elf_print_phdr(struct elfdump *ed) { GElf_Phdr ph; size_t phnum, i; int header; if (elf_getphnum(ed->elf, &phnum) == 0) { warnx("elf_getphnum failed: %s", elf_errmsg(-1)); return; } header = 0; for (i = 0; i < phnum; i++) { if (gelf_getphdr(ed->elf, i, &ph) != &ph) { warnx("elf_getphdr failed: %s", elf_errmsg(-1)); continue; } if (!STAILQ_EMPTY(&ed->snl) && find_name(ed, elf_phdr_type_str(ph.p_type)) == NULL) continue; if (ed->flags & SOLARIS_FMT) { PRT("\nProgram Header[%zu]:\n", i); PRT(" p_vaddr: %#-14jx", (uintmax_t)ph.p_vaddr); PRT(" p_flags: [ %s ]\n", p_flags[ph.p_flags & 0x7]); PRT(" p_paddr: %#-14jx", (uintmax_t)ph.p_paddr); PRT(" p_type: [ %s ]\n", elf_phdr_type_str(ph.p_type)); PRT(" p_filesz: %#-14jx", (uintmax_t)ph.p_filesz); PRT(" p_memsz: %#jx\n", (uintmax_t)ph.p_memsz); PRT(" p_offset: %#-14jx", (uintmax_t)ph.p_offset); PRT(" p_align: %#jx\n", (uintmax_t)ph.p_align); } else { if (!header) { PRT("\nprogram header:\n"); header = 1; } PRT("\n"); PRT("entry: %zu\n", i); PRT("\tp_type: %s\n", elf_phdr_type_str(ph.p_type)); PRT("\tp_offset: %ju\n", (uintmax_t)ph.p_offset); PRT("\tp_vaddr: %#jx\n", (uintmax_t)ph.p_vaddr); PRT("\tp_paddr: %#jx\n", (uintmax_t)ph.p_paddr); PRT("\tp_filesz: %ju\n", (uintmax_t)ph.p_filesz); PRT("\tp_memsz: %ju\n", (uintmax_t)ph.p_memsz); PRT("\tp_flags: %s\n", p_flags[ph.p_flags & 0x7]); PRT("\tp_align: %ju\n", (uintmax_t)ph.p_align); } } } /* * Dump the ELF Section Header Table. */ static void elf_print_shdr(struct elfdump *ed) { struct section *s; size_t i; if (!STAILQ_EMPTY(&ed->snl)) return; if ((ed->flags & SOLARIS_FMT) == 0) PRT("\nsection header:\n"); for (i = 0; i < ed->shnum; i++) { s = &ed->sl[i]; if (ed->flags & SOLARIS_FMT) { if (i == 0) continue; PRT("\nSection Header[%zu]:", i); PRT(" sh_name: %s\n", s->name); PRT(" sh_addr: %#-14jx", (uintmax_t)s->addr); if (s->flags != 0) PRT(" sh_flags: [ %s ]\n", sh_flags(s->flags)); else PRT(" sh_flags: 0\n"); PRT(" sh_size: %#-14jx", (uintmax_t)s->sz); PRT(" sh_type: [ %s ]\n", sh_types(ed->ehdr.e_machine, s->type)); PRT(" sh_offset: %#-14jx", (uintmax_t)s->off); PRT(" sh_entsize: %#jx\n", (uintmax_t)s->entsize); PRT(" sh_link: %-14u", s->link); PRT(" sh_info: %u\n", s->info); PRT(" sh_addralign: %#jx\n", (uintmax_t)s->align); } else { PRT("\n"); PRT("entry: %ju\n", (uintmax_t)i); PRT("\tsh_name: %s\n", s->name); PRT("\tsh_type: %s\n", sh_types(ed->ehdr.e_machine, s->type)); PRT("\tsh_flags: %s\n", sh_flags(s->flags)); PRT("\tsh_addr: %#jx\n", (uintmax_t)s->addr); PRT("\tsh_offset: %ju\n", (uintmax_t)s->off); PRT("\tsh_size: %ju\n", (uintmax_t)s->sz); PRT("\tsh_link: %u\n", s->link); PRT("\tsh_info: %u\n", s->info); PRT("\tsh_addralign: %ju\n", (uintmax_t)s->align); PRT("\tsh_entsize: %ju\n", (uintmax_t)s->entsize); } } } /* * Return number of entries in the given section. We'd prefer ent_count be a * size_t, but libelf APIs already use int for section indices. */ static int get_ent_count(const struct section *s, int *ent_count) { if (s->entsize == 0) { warnx("section %s has entry size 0", s->name); return (0); } else if (s->sz / s->entsize > INT_MAX) { warnx("section %s has invalid section count", s->name); return (0); } *ent_count = (int)(s->sz / s->entsize); return (1); } /* * Retrieve the content of the corresponding SHT_SUNW_versym section for * a symbol table section. */ static void get_versym(struct elfdump *ed, int i, uint16_t **vs, int *nvs) { struct section *s; Elf_Data *data; size_t j; int elferr; s = NULL; for (j = 0; j < ed->shnum; j++) { s = &ed->sl[j]; if (s->type == SHT_SUNW_versym && s->link == (uint32_t)i) break; } if (j >= ed->shnum) { *vs = NULL; return; } (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); *vs = NULL; return; } *vs = data->d_buf; assert(data->d_size == s->sz); if (!get_ent_count(s, nvs)) *nvs = 0; } /* * Dump the symbol table section. */ static void elf_print_symtab(struct elfdump *ed, int i) { struct section *s; const char *name; uint16_t *vs; char idx[10]; Elf_Data *data; GElf_Sym sym; int len, j, elferr, nvs; s = &ed->sl[i]; if (ed->flags & SOLARIS_FMT) PRT("\nSymbol Table Section: %s\n", s->name); else PRT("\nsymbol table (%s):\n", s->name); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } vs = NULL; nvs = 0; assert(data->d_size == s->sz); if (!get_ent_count(s, &len)) return; if (ed->flags & SOLARIS_FMT) { if (ed->ec == ELFCLASS32) PRT(" index value "); else PRT(" index value "); PRT("size type bind oth ver shndx name\n"); get_versym(ed, i, &vs, &nvs); if (vs != NULL && nvs != len) { warnx("#symbol not equal to #versym"); vs = NULL; } } for (j = 0; j < len; j++) { if (gelf_getsym(data, j, &sym) != &sym) { warnx("gelf_getsym failed: %s", elf_errmsg(-1)); continue; } name = get_string(ed, s->link, sym.st_name); if (ed->flags & SOLARIS_FMT) { snprintf(idx, sizeof(idx), "[%d]", j); if (ed->ec == ELFCLASS32) PRT("%10s ", idx); else PRT("%10s ", idx); PRT("0x%8.8jx ", (uintmax_t)sym.st_value); if (ed->ec == ELFCLASS32) PRT("0x%8.8jx ", (uintmax_t)sym.st_size); else PRT("0x%12.12jx ", (uintmax_t)sym.st_size); PRT("%s ", st_type_S(GELF_ST_TYPE(sym.st_info))); PRT("%s ", st_bindings_S(GELF_ST_BIND(sym.st_info))); PRT("%c ", st_others[sym.st_other]); PRT("%3u ", (vs == NULL ? 0 : vs[j])); PRT("%-11.11s ", sh_name(ed, sym.st_shndx)); PRT("%s\n", name); } else { PRT("\nentry: %d\n", j); PRT("\tst_name: %s\n", name); PRT("\tst_value: %#jx\n", (uintmax_t)sym.st_value); PRT("\tst_size: %ju\n", (uintmax_t)sym.st_size); PRT("\tst_info: %s %s\n", st_type(ed->ehdr.e_machine, GELF_ST_TYPE(sym.st_info)), st_bindings(GELF_ST_BIND(sym.st_info))); PRT("\tst_shndx: %ju\n", (uintmax_t)sym.st_shndx); } } } /* * Dump the symbol tables. (.dynsym and .symtab) */ static void elf_print_symtabs(struct elfdump *ed) { size_t i; for (i = 0; i < ed->shnum; i++) if ((ed->sl[i].type == SHT_SYMTAB || ed->sl[i].type == SHT_DYNSYM) && (STAILQ_EMPTY(&ed->snl) || find_name(ed, ed->sl[i].name))) elf_print_symtab(ed, i); } /* * Dump the content of .dynamic section. */ static void elf_print_dynamic(struct elfdump *ed) { struct section *s; const char *name; char idx[10]; Elf_Data *data; GElf_Dyn dyn; int elferr, i, len; s = NULL; for (i = 0; (size_t)i < ed->shnum; i++) { s = &ed->sl[i]; if (s->type == SHT_DYNAMIC && (STAILQ_EMPTY(&ed->snl) || find_name(ed, s->name))) break; } if ((size_t)i >= ed->shnum) return; if (ed->flags & SOLARIS_FMT) { PRT("Dynamic Section: %s\n", s->name); PRT(" index tag value\n"); } else PRT("\ndynamic:\n"); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } assert(data->d_size == s->sz); if (!get_ent_count(s, &len)) return; for (i = 0; i < len; i++) { if (gelf_getdyn(data, i, &dyn) != &dyn) { warnx("gelf_getdyn failed: %s", elf_errmsg(-1)); continue; } if (ed->flags & SOLARIS_FMT) { snprintf(idx, sizeof(idx), "[%d]", i); PRT("%10s %-16s ", idx, d_tags(dyn.d_tag)); } else { PRT("\n"); PRT("entry: %d\n", i); PRT("\td_tag: %s\n", d_tags(dyn.d_tag)); } switch(dyn.d_tag) { case DT_NEEDED: case DT_SONAME: case DT_RPATH: case DT_RUNPATH: if ((name = elf_strptr(ed->elf, s->link, dyn.d_un.d_val)) == NULL) name = ""; if (ed->flags & SOLARIS_FMT) PRT("%#-16jx %s\n", (uintmax_t)dyn.d_un.d_val, name); else PRT("\td_val: %s\n", name); break; case DT_PLTRELSZ: case DT_RELA: case DT_RELASZ: case DT_RELAENT: case DT_RELACOUNT: case DT_STRSZ: case DT_SYMENT: case DT_RELSZ: case DT_RELENT: case DT_PLTREL: case DT_VERDEF: case DT_VERDEFNUM: case DT_VERNEED: case DT_VERNEEDNUM: case DT_VERSYM: if (ed->flags & SOLARIS_FMT) PRT("%#jx\n", (uintmax_t)dyn.d_un.d_val); else PRT("\td_val: %ju\n", (uintmax_t)dyn.d_un.d_val); break; case DT_PLTGOT: case DT_HASH: case DT_GNU_HASH: case DT_STRTAB: case DT_SYMTAB: case DT_INIT: case DT_FINI: case DT_REL: case DT_JMPREL: case DT_DEBUG: if (ed->flags & SOLARIS_FMT) PRT("%#jx\n", (uintmax_t)dyn.d_un.d_ptr); else PRT("\td_ptr: %#jx\n", (uintmax_t)dyn.d_un.d_ptr); break; case DT_NULL: case DT_SYMBOLIC: case DT_TEXTREL: default: if (ed->flags & SOLARIS_FMT) PRT("\n"); break; } } } /* * Dump a .rel/.rela section entry. */ static void elf_print_rel_entry(struct elfdump *ed, struct section *s, int j, struct rel_entry *r) { if (ed->flags & SOLARIS_FMT) { PRT(" %-23s ", elftc_reloc_type_str(ed->ehdr.e_machine, GELF_R_TYPE(r->u_r.rel.r_info))); PRT("%#12jx ", (uintmax_t)r->u_r.rel.r_offset); if (r->type == SHT_RELA) PRT("%10jd ", (intmax_t)r->u_r.rela.r_addend); else PRT(" "); PRT("%-14s ", s->name); PRT("%s\n", r->symn); } else { PRT("\n"); PRT("entry: %d\n", j); PRT("\tr_offset: %#jx\n", (uintmax_t)r->u_r.rel.r_offset); if (ed->ec == ELFCLASS32) PRT("\tr_info: %#jx\n", (uintmax_t) ELF32_R_INFO(ELF64_R_SYM(r->u_r.rel.r_info), ELF64_R_TYPE(r->u_r.rel.r_info))); else PRT("\tr_info: %#jx\n", (uintmax_t)r->u_r.rel.r_info); if (r->type == SHT_RELA) PRT("\tr_addend: %jd\n", (intmax_t)r->u_r.rela.r_addend); } } /* * Dump a relocation section of type SHT_RELA. */ static void elf_print_rela(struct elfdump *ed, struct section *s, Elf_Data *data) { struct rel_entry r; int j, len; if (ed->flags & SOLARIS_FMT) { PRT("\nRelocation Section: %s\n", s->name); PRT(" type offset " "addend section with respect to\n"); } else PRT("\nrelocation with addend (%s):\n", s->name); r.type = SHT_RELA; assert(data->d_size == s->sz); if (!get_ent_count(s, &len)) return; for (j = 0; j < len; j++) { if (gelf_getrela(data, j, &r.u_r.rela) != &r.u_r.rela) { warnx("gelf_getrela failed: %s", elf_errmsg(-1)); continue; } r.symn = get_symbol_name(ed, s->link, GELF_R_SYM(r.u_r.rela.r_info)); elf_print_rel_entry(ed, s, j, &r); } } /* * Dump a relocation section of type SHT_REL. */ static void elf_print_rel(struct elfdump *ed, struct section *s, Elf_Data *data) { struct rel_entry r; int j, len; if (ed->flags & SOLARIS_FMT) { PRT("\nRelocation Section: %s\n", s->name); PRT(" type offset " "section with respect to\n"); } else PRT("\nrelocation (%s):\n", s->name); r.type = SHT_REL; assert(data->d_size == s->sz); if (!get_ent_count(s, &len)) return; for (j = 0; j < len; j++) { if (gelf_getrel(data, j, &r.u_r.rel) != &r.u_r.rel) { warnx("gelf_getrel failed: %s", elf_errmsg(-1)); continue; } r.symn = get_symbol_name(ed, s->link, GELF_R_SYM(r.u_r.rel.r_info)); elf_print_rel_entry(ed, s, j, &r); } } /* * Dump relocation sections. */ static void elf_print_reloc(struct elfdump *ed) { struct section *s; Elf_Data *data; size_t i; int elferr; for (i = 0; i < ed->shnum; i++) { s = &ed->sl[i]; if ((s->type == SHT_REL || s->type == SHT_RELA) && (STAILQ_EMPTY(&ed->snl) || find_name(ed, s->name))) { (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); continue; } if (s->type == SHT_REL) elf_print_rel(ed, s, data); else elf_print_rela(ed, s, data); } } } /* * Dump the content of PT_INTERP segment. */ static void elf_print_interp(struct elfdump *ed) { const char *s; GElf_Phdr phdr; size_t filesize, i, phnum; if (!STAILQ_EMPTY(&ed->snl) && find_name(ed, "PT_INTERP") == NULL) return; if ((s = elf_rawfile(ed->elf, &filesize)) == NULL) { warnx("elf_rawfile failed: %s", elf_errmsg(-1)); return; } if (!elf_getphnum(ed->elf, &phnum)) { warnx("elf_getphnum failed: %s", elf_errmsg(-1)); return; } for (i = 0; i < phnum; i++) { if (gelf_getphdr(ed->elf, i, &phdr) != &phdr) { warnx("elf_getphdr failed: %s", elf_errmsg(-1)); continue; } if (phdr.p_type == PT_INTERP) { if (phdr.p_offset >= filesize) { warnx("invalid phdr offset"); continue; } PRT("\ninterp:\n"); PRT("\t%s\n", s + phdr.p_offset); } } } /* * Search the relocation sections for entries referring to the .got section. */ static void find_gotrel(struct elfdump *ed, struct section *gs, struct rel_entry *got) { struct section *s; struct rel_entry r; Elf_Data *data; size_t i; int elferr, j, k, len; for(i = 0; i < ed->shnum; i++) { s = &ed->sl[i]; if (s->type != SHT_REL && s->type != SHT_RELA) continue; (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } memset(&r, 0, sizeof(struct rel_entry)); r.type = s->type; assert(data->d_size == s->sz); if (!get_ent_count(s, &len)) return; for (j = 0; j < len; j++) { if (s->type == SHT_REL) { if (gelf_getrel(data, j, &r.u_r.rel) != &r.u_r.rel) { warnx("gelf_getrel failed: %s", elf_errmsg(-1)); continue; } } else { if (gelf_getrela(data, j, &r.u_r.rela) != &r.u_r.rela) { warnx("gelf_getrel failed: %s", elf_errmsg(-1)); continue; } } if (r.u_r.rel.r_offset >= gs->addr && r.u_r.rel.r_offset < gs->addr + gs->sz) { r.symn = get_symbol_name(ed, s->link, GELF_R_SYM(r.u_r.rel.r_info)); k = (r.u_r.rel.r_offset - gs->addr) / gs->entsize; memcpy(&got[k], &r, sizeof(struct rel_entry)); } } } } static void elf_print_got_section(struct elfdump *ed, struct section *s) { struct rel_entry *got; Elf_Data *data, dst; int elferr, i, len; if (s->entsize == 0) { /* XXX IA64 GOT section generated by gcc has entry size 0. */ if (s->align != 0) s->entsize = s->align; else return; } if (!get_ent_count(s, &len)) return; if (ed->flags & SOLARIS_FMT) PRT("\nGlobal Offset Table Section: %s (%d entries)\n", s->name, len); else PRT("\nglobal offset table: %s\n", s->name); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } /* * GOT section has section type SHT_PROGBITS, thus libelf treats it as * byte stream and will not perform any translation on it. As a result, * an exlicit call to gelf_xlatetom is needed here. Depends on arch, * GOT section should be translated to either WORD or XWORD. */ if (ed->ec == ELFCLASS32) data->d_type = ELF_T_WORD; else data->d_type = ELF_T_XWORD; memcpy(&dst, data, sizeof(Elf_Data)); if (gelf_xlatetom(ed->elf, &dst, data, ed->ehdr.e_ident[EI_DATA]) != &dst) { warnx("gelf_xlatetom failed: %s", elf_errmsg(-1)); return; } assert(dst.d_size == s->sz); if (ed->flags & SOLARIS_FMT) { /* * In verbose/Solaris mode, we search the relocation sections * and try to find the corresponding reloc entry for each GOT * section entry. */ if ((got = calloc(len, sizeof(struct rel_entry))) == NULL) err(EXIT_FAILURE, "calloc failed"); find_gotrel(ed, s, got); if (ed->ec == ELFCLASS32) { PRT(" ndx addr value reloc "); PRT("addend symbol\n"); } else { PRT(" ndx addr value "); PRT("reloc addend symbol\n"); } for(i = 0; i < len; i++) { PRT("[%5.5d] ", i); if (ed->ec == ELFCLASS32) { PRT("%-8.8jx ", (uintmax_t) (s->addr + i * s->entsize)); PRT("%-8.8x ", *((uint32_t *)dst.d_buf + i)); } else { PRT("%-16.16jx ", (uintmax_t) (s->addr + i * s->entsize)); PRT("%-16.16jx ", (uintmax_t) *((uint64_t *)dst.d_buf + i)); } PRT("%-18s ", elftc_reloc_type_str(ed->ehdr.e_machine, GELF_R_TYPE(got[i].u_r.rel.r_info))); if (ed->ec == ELFCLASS32) PRT("%-8.8jd ", (intmax_t)got[i].u_r.rela.r_addend); else PRT("%-12.12jd ", (intmax_t)got[i].u_r.rela.r_addend); if (got[i].symn == NULL) got[i].symn = ""; PRT("%s\n", got[i].symn); } free(got); } else { for(i = 0; i < len; i++) { PRT("\nentry: %d\n", i); if (ed->ec == ELFCLASS32) PRT("\t%#x\n", *((uint32_t *)dst.d_buf + i)); else PRT("\t%#jx\n", (uintmax_t) *((uint64_t *)dst.d_buf + i)); } } } /* * Dump the content of Global Offset Table section. */ static void elf_print_got(struct elfdump *ed) { struct section *s; size_t i; if (!STAILQ_EMPTY(&ed->snl)) return; s = NULL; for (i = 0; i < ed->shnum; i++) { s = &ed->sl[i]; if (s->name && !strncmp(s->name, ".got", 4) && (STAILQ_EMPTY(&ed->snl) || find_name(ed, s->name))) elf_print_got_section(ed, s); } } /* * Dump the content of .note.ABI-tag section. */ static void elf_print_note(struct elfdump *ed) { struct section *s; Elf_Data *data; Elf_Note *en; uint32_t namesz; uint32_t descsz; uint32_t desc; size_t count; int elferr, i; uint8_t *src; char idx[10]; s = NULL; for (i = 0; (size_t)i < ed->shnum; i++) { s = &ed->sl[i]; if (s->type == SHT_NOTE && s->name && !strcmp(s->name, ".note.ABI-tag") && (STAILQ_EMPTY(&ed->snl) || find_name(ed, s->name))) break; } if ((size_t)i >= ed->shnum) return; if (ed->flags & SOLARIS_FMT) PRT("\nNote Section: %s\n", s->name); else PRT("\nnote (%s):\n", s->name); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } src = data->d_buf; count = data->d_size; while (count > sizeof(Elf_Note)) { en = (Elf_Note *) (uintptr_t) src; namesz = en->n_namesz; descsz = en->n_descsz; src += sizeof(Elf_Note); count -= sizeof(Elf_Note); if (roundup2(namesz, 4) + roundup2(descsz, 4) > count) { warnx("truncated note section"); return; } if (ed->flags & SOLARIS_FMT) { PRT("\n type %#x\n", en->n_type); PRT(" namesz %#x:\n", en->n_namesz); PRT("%s\n", src); } else PRT("\t%s ", src); src += roundup2(namesz, 4); count -= roundup2(namesz, 4); /* * Note that we dump the whole desc part if we're in * "Solaris mode", while in the normal mode, we only look * at the first 4 bytes (a 32bit word) of the desc, i.e, * we assume that it's always a FreeBSD version number. */ if (ed->flags & SOLARIS_FMT) { PRT(" descsz %#x:", en->n_descsz); for (i = 0; (uint32_t)i < descsz; i++) { if ((i & 0xF) == 0) { snprintf(idx, sizeof(idx), "desc[%d]", i); PRT("\n %-9s", idx); } else if ((i & 0x3) == 0) PRT(" "); PRT(" %2.2x", src[i]); } PRT("\n"); } else { if (ed->ehdr.e_ident[EI_DATA] == ELFDATA2MSB) desc = be32dec(src); else desc = le32dec(src); PRT("%d\n", desc); } src += roundup2(descsz, 4); count -= roundup2(descsz, 4); } } /* * Dump a hash table. */ static void elf_print_svr4_hash(struct elfdump *ed, struct section *s) { Elf_Data *data; uint32_t *buf; uint32_t *bucket, *chain; uint32_t nbucket, nchain; uint32_t *bl, *c, maxl, total; uint32_t i, j; int first, elferr; char idx[10]; if (ed->flags & SOLARIS_FMT) PRT("\nHash Section: %s\n", s->name); else PRT("\nhash table (%s):\n", s->name); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } if (data->d_size < 2 * sizeof(uint32_t)) { warnx(".hash section too small"); return; } buf = data->d_buf; nbucket = buf[0]; nchain = buf[1]; if (nbucket <= 0 || nchain <= 0) { warnx("Malformed .hash section"); return; } if (data->d_size != ((uint64_t)nbucket + (uint64_t)nchain + 2) * sizeof(uint32_t)) { warnx("Malformed .hash section"); return; } bucket = &buf[2]; chain = &buf[2 + nbucket]; if (ed->flags & SOLARIS_FMT) { maxl = 0; if ((bl = calloc(nbucket, sizeof(*bl))) == NULL) err(EXIT_FAILURE, "calloc failed"); for (i = 0; i < nbucket; i++) for (j = bucket[i]; j > 0 && j < nchain; j = chain[j]) if (++bl[i] > maxl) maxl = bl[i]; if ((c = calloc(maxl + 1, sizeof(*c))) == NULL) err(EXIT_FAILURE, "calloc failed"); for (i = 0; i < nbucket; i++) c[bl[i]]++; PRT(" bucket symndx name\n"); for (i = 0; i < nbucket; i++) { first = 1; for (j = bucket[i]; j > 0 && j < nchain; j = chain[j]) { if (first) { PRT("%10d ", i); first = 0; } else PRT(" "); snprintf(idx, sizeof(idx), "[%d]", j); PRT("%-10s ", idx); PRT("%s\n", get_symbol_name(ed, s->link, j)); } } PRT("\n"); total = 0; for (i = 0; i <= maxl; i++) { total += c[i] * i; PRT("%10u buckets contain %8d symbols\n", c[i], i); } PRT("%10u buckets %8u symbols (globals)\n", nbucket, total); } else { PRT("\nnbucket: %u\n", nbucket); PRT("nchain: %u\n\n", nchain); for (i = 0; i < nbucket; i++) PRT("bucket[%d]:\n\t%u\n\n", i, bucket[i]); for (i = 0; i < nchain; i++) PRT("chain[%d]:\n\t%u\n\n", i, chain[i]); } } /* * Dump a 64bit hash table. */ static void elf_print_svr4_hash64(struct elfdump *ed, struct section *s) { Elf_Data *data, dst; uint64_t *buf; uint64_t *bucket, *chain; uint64_t nbucket, nchain; uint64_t *bl, *c, maxl, total; uint64_t i, j; int elferr, first; char idx[10]; if (ed->flags & SOLARIS_FMT) PRT("\nHash Section: %s\n", s->name); else PRT("\nhash table (%s):\n", s->name); /* * ALPHA uses 64-bit hash entries. Since libelf assumes that * .hash section contains only 32-bit entry, an explicit * gelf_xlatetom is needed here. */ (void) elf_errno(); if ((data = elf_rawdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_rawdata failed: %s", elf_errmsg(elferr)); return; } data->d_type = ELF_T_XWORD; memcpy(&dst, data, sizeof(Elf_Data)); if (gelf_xlatetom(ed->elf, &dst, data, ed->ehdr.e_ident[EI_DATA]) != &dst) { warnx("gelf_xlatetom failed: %s", elf_errmsg(-1)); return; } if (dst.d_size < 2 * sizeof(uint64_t)) { warnx(".hash section too small"); return; } buf = dst.d_buf; nbucket = buf[0]; nchain = buf[1]; if (nbucket <= 0 || nchain <= 0) { warnx("Malformed .hash section"); return; } if (dst.d_size != (nbucket + nchain + 2) * sizeof(uint64_t)) { warnx("Malformed .hash section"); return; } bucket = &buf[2]; chain = &buf[2 + nbucket]; if (ed->flags & SOLARIS_FMT) { maxl = 0; if ((bl = calloc(nbucket, sizeof(*bl))) == NULL) err(EXIT_FAILURE, "calloc failed"); for (i = 0; i < nbucket; i++) for (j = bucket[i]; j > 0 && j < nchain; j = chain[j]) if (++bl[i] > maxl) maxl = bl[i]; if ((c = calloc(maxl + 1, sizeof(*c))) == NULL) err(EXIT_FAILURE, "calloc failed"); for (i = 0; i < nbucket; i++) c[bl[i]]++; PRT(" bucket symndx name\n"); for (i = 0; i < nbucket; i++) { first = 1; for (j = bucket[i]; j > 0 && j < nchain; j = chain[j]) { if (first) { PRT("%10zu ", i); first = 0; } else PRT(" "); snprintf(idx, sizeof(idx), "[%zu]", (size_t)j); PRT("%-10s ", idx); PRT("%s\n", get_symbol_name(ed, s->link, j)); } } PRT("\n"); total = 0; for (i = 0; i <= maxl; i++) { total += c[i] * i; PRT("%10ju buckets contain %8zu symbols\n", (uintmax_t)c[i], i); } PRT("%10ju buckets %8ju symbols (globals)\n", (uintmax_t)nbucket, (uintmax_t)total); } else { PRT("\nnbucket: %ju\n", (uintmax_t)nbucket); PRT("nchain: %ju\n\n", (uintmax_t)nchain); for (i = 0; i < nbucket; i++) PRT("bucket[%zu]:\n\t%ju\n\n", i, (uintmax_t)bucket[i]); for (i = 0; i < nchain; i++) PRT("chain[%zu]:\n\t%ju\n\n", i, (uintmax_t)chain[i]); } } /* * Dump a GNU hash table. */ static void elf_print_gnu_hash(struct elfdump *ed, struct section *s) { struct section *ds; Elf_Data *data; uint32_t *buf; uint32_t *bucket, *chain; uint32_t nbucket, nchain, symndx, maskwords, shift2; uint32_t *bl, *c, maxl, total; uint32_t i, j; int first, elferr, dynsymcount; char idx[10]; if (ed->flags & SOLARIS_FMT) PRT("\nGNU Hash Section: %s\n", s->name); else PRT("\ngnu hash table (%s):\n", s->name); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } if (data->d_size < 4 * sizeof(uint32_t)) { warnx(".gnu.hash section too small"); return; } buf = data->d_buf; nbucket = buf[0]; symndx = buf[1]; maskwords = buf[2]; shift2 = buf[3]; buf += 4; if (s->link >= ed->shnum) { warnx("Malformed .gnu.hash section"); return; } ds = &ed->sl[s->link]; if (!get_ent_count(ds, &dynsymcount)) return; if (symndx >= (uint32_t)dynsymcount) { warnx("Malformed .gnu.hash section"); return; } nchain = dynsymcount - symndx; if (data->d_size != 4 * sizeof(uint32_t) + maskwords * (ed->ec == ELFCLASS32 ? sizeof(uint32_t) : sizeof(uint64_t)) + ((uint64_t)nbucket + (uint64_t)nchain) * sizeof(uint32_t)) { warnx("Malformed .gnu.hash section"); return; } bucket = buf + (ed->ec == ELFCLASS32 ? maskwords : maskwords * 2); chain = bucket + nbucket; if (ed->flags & SOLARIS_FMT) { maxl = 0; if ((bl = calloc(nbucket, sizeof(*bl))) == NULL) err(EXIT_FAILURE, "calloc failed"); for (i = 0; i < nbucket; i++) for (j = bucket[i]; j > 0 && j - symndx < nchain; j++) { if (++bl[i] > maxl) maxl = bl[i]; if (chain[j - symndx] & 1) break; } if ((c = calloc(maxl + 1, sizeof(*c))) == NULL) err(EXIT_FAILURE, "calloc failed"); for (i = 0; i < nbucket; i++) c[bl[i]]++; PRT(" bucket symndx name\n"); for (i = 0; i < nbucket; i++) { first = 1; for (j = bucket[i]; j > 0 && j - symndx < nchain; j++) { if (first) { PRT("%10d ", i); first = 0; } else PRT(" "); snprintf(idx, sizeof(idx), "[%d]", j ); PRT("%-10s ", idx); PRT("%s\n", get_symbol_name(ed, s->link, j)); if (chain[j - symndx] & 1) break; } } PRT("\n"); total = 0; for (i = 0; i <= maxl; i++) { total += c[i] * i; PRT("%10u buckets contain %8d symbols\n", c[i], i); } PRT("%10u buckets %8u symbols (globals)\n", nbucket, total); } else { PRT("\nnbucket: %u\n", nbucket); PRT("symndx: %u\n", symndx); PRT("maskwords: %u\n", maskwords); PRT("shift2: %u\n", shift2); PRT("nchain: %u\n\n", nchain); for (i = 0; i < nbucket; i++) PRT("bucket[%d]:\n\t%u\n\n", i, bucket[i]); for (i = 0; i < nchain; i++) PRT("chain[%d]:\n\t%u\n\n", i, chain[i]); } } /* * Dump hash tables. */ static void elf_print_hash(struct elfdump *ed) { struct section *s; size_t i; for (i = 0; i < ed->shnum; i++) { s = &ed->sl[i]; if ((s->type == SHT_HASH || s->type == SHT_GNU_HASH) && (STAILQ_EMPTY(&ed->snl) || find_name(ed, s->name))) { if (s->type == SHT_GNU_HASH) elf_print_gnu_hash(ed, s); else if (ed->ehdr.e_machine == EM_ALPHA && s->entsize == 8) elf_print_svr4_hash64(ed, s); else elf_print_svr4_hash(ed, s); } } } /* * Dump the content of a Version Definition(SHT_SUNW_Verdef) Section. */ static void elf_print_verdef(struct elfdump *ed, struct section *s) { Elf_Data *data; Elf32_Verdef *vd; Elf32_Verdaux *vda; const char *str; char idx[10]; uint8_t *buf, *end, *buf2; int i, j, elferr, count; if (ed->flags & SOLARIS_FMT) PRT("Version Definition Section: %s\n", s->name); else PRT("\nversion definition section (%s):\n", s->name); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } buf = data->d_buf; end = buf + data->d_size; i = 0; if (ed->flags & SOLARIS_FMT) PRT(" index version dependency\n"); while (buf + sizeof(Elf32_Verdef) <= end) { vd = (Elf32_Verdef *) (uintptr_t) buf; if (ed->flags & SOLARIS_FMT) { snprintf(idx, sizeof(idx), "[%d]", vd->vd_ndx); PRT("%10s ", idx); } else { PRT("\nentry: %d\n", i++); PRT("\tvd_version: %u\n", vd->vd_version); PRT("\tvd_flags: %u\n", vd->vd_flags); PRT("\tvd_ndx: %u\n", vd->vd_ndx); PRT("\tvd_cnt: %u\n", vd->vd_cnt); PRT("\tvd_hash: %u\n", vd->vd_hash); PRT("\tvd_aux: %u\n", vd->vd_aux); PRT("\tvd_next: %u\n\n", vd->vd_next); } buf2 = buf + vd->vd_aux; j = 0; count = 0; while (buf2 + sizeof(Elf32_Verdaux) <= end && j < vd->vd_cnt) { vda = (Elf32_Verdaux *) (uintptr_t) buf2; str = get_string(ed, s->link, vda->vda_name); if (ed->flags & SOLARIS_FMT) { if (count == 0) PRT("%-26.26s", str); else if (count == 1) PRT(" %-20.20s", str); else { PRT("\n%40.40s", ""); PRT("%s", str); } } else { PRT("\t\tvda: %d\n", j++); PRT("\t\t\tvda_name: %s\n", str); PRT("\t\t\tvda_next: %u\n", vda->vda_next); } if (vda->vda_next == 0) { if (ed->flags & SOLARIS_FMT) { if (vd->vd_flags & VER_FLG_BASE) { if (count == 0) PRT("%-20.20s", ""); PRT("%s", "[ BASE ]"); } PRT("\n"); } break; } if (ed->flags & SOLARIS_FMT) count++; buf2 += vda->vda_next; } if (vd->vd_next == 0) break; buf += vd->vd_next; } } /* * Dump the content of a Version Needed(SHT_SUNW_Verneed) Section. */ static void elf_print_verneed(struct elfdump *ed, struct section *s) { Elf_Data *data; Elf32_Verneed *vn; Elf32_Vernaux *vna; uint8_t *buf, *end, *buf2; int i, j, elferr, first; if (ed->flags & SOLARIS_FMT) PRT("\nVersion Needed Section: %s\n", s->name); else PRT("\nversion need section (%s):\n", s->name); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } buf = data->d_buf; end = buf + data->d_size; if (ed->flags & SOLARIS_FMT) PRT(" file version\n"); i = 0; while (buf + sizeof(Elf32_Verneed) <= end) { vn = (Elf32_Verneed *) (uintptr_t) buf; if (ed->flags & SOLARIS_FMT) PRT(" %-26.26s ", get_string(ed, s->link, vn->vn_file)); else { PRT("\nentry: %d\n", i++); PRT("\tvn_version: %u\n", vn->vn_version); PRT("\tvn_cnt: %u\n", vn->vn_cnt); PRT("\tvn_file: %s\n", get_string(ed, s->link, vn->vn_file)); PRT("\tvn_aux: %u\n", vn->vn_aux); PRT("\tvn_next: %u\n\n", vn->vn_next); } buf2 = buf + vn->vn_aux; j = 0; first = 1; while (buf2 + sizeof(Elf32_Vernaux) <= end && j < vn->vn_cnt) { vna = (Elf32_Vernaux *) (uintptr_t) buf2; if (ed->flags & SOLARIS_FMT) { if (!first) PRT("%40.40s", ""); else first = 0; PRT("%s\n", get_string(ed, s->link, vna->vna_name)); } else { PRT("\t\tvna: %d\n", j++); PRT("\t\t\tvna_hash: %u\n", vna->vna_hash); PRT("\t\t\tvna_flags: %u\n", vna->vna_flags); PRT("\t\t\tvna_other: %u\n", vna->vna_other); PRT("\t\t\tvna_name: %s\n", get_string(ed, s->link, vna->vna_name)); PRT("\t\t\tvna_next: %u\n", vna->vna_next); } if (vna->vna_next == 0) break; buf2 += vna->vna_next; } if (vn->vn_next == 0) break; buf += vn->vn_next; } } /* * Dump the symbol-versioning sections. */ static void elf_print_symver(struct elfdump *ed) { struct section *s; size_t i; for (i = 0; i < ed->shnum; i++) { s = &ed->sl[i]; if (!STAILQ_EMPTY(&ed->snl) && !find_name(ed, s->name)) continue; if (s->type == SHT_SUNW_verdef) elf_print_verdef(ed, s); if (s->type == SHT_SUNW_verneed) elf_print_verneed(ed, s); } } /* * Dump the ELF checksum. See gelf_checksum(3) for details. */ static void elf_print_checksum(struct elfdump *ed) { if (!STAILQ_EMPTY(&ed->snl)) return; PRT("\nelf checksum: %#lx\n", gelf_checksum(ed->elf)); } #define USAGE_MESSAGE "\ Usage: %s [options] file...\n\ Display information about ELF objects and ar(1) archives.\n\n\ Options:\n\ -a Show all information.\n\ -c Show shared headers.\n\ -d Show dynamic symbols.\n\ -e Show the ELF header.\n\ -G Show the GOT.\n\ -H | --help Show a usage message and exit.\n\ -h Show hash values.\n\ -i Show the dynamic interpreter.\n\ -k Show the ELF checksum.\n\ -n Show the contents of note sections.\n\ -N NAME Show the section named \"NAME\".\n\ -p Show the program header.\n\ -r Show relocations.\n\ -s Show the symbol table.\n\ -S Use the Solaris elfdump format.\n\ -v Show symbol-versioning information.\n\ -V | --version Print a version identifier and exit.\n\ -w FILE Write output to \"FILE\".\n" static void usage(void) { fprintf(stderr, USAGE_MESSAGE, ELFTC_GETPROGNAME()); exit(EXIT_FAILURE); } Index: projects/netbsd-tests-update-12/contrib/elftoolchain/elfdump =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/elfdump (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/elfdump (revision 305172) Property changes on: projects/netbsd-tests-update-12/contrib/elftoolchain/elfdump ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,2 ## Merged /vendor/elftoolchain/dist/elfdump:r305126 Merged /head/contrib/elftoolchain/elfdump:r303899-305170 Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelf/elf_flagdata.3 =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelf/elf_flagdata.3 (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelf/elf_flagdata.3 (revision 305172) @@ -1,226 +1,223 @@ .\" Copyright (c) 2006-2008,2011 Joseph Koshy. All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" This software is provided by Joseph Koshy ``as is'' and .\" any express or implied warranties, including, but not limited to, the .\" implied warranties of merchantability and fitness for a particular purpose .\" are disclaimed. in no event shall Joseph Koshy be liable .\" for any direct, indirect, incidental, special, exemplary, or consequential .\" damages (including, but not limited to, procurement of substitute goods .\" or services; loss of use, data, or profits; or business interruption) .\" however caused and on any theory of liability, whether in contract, strict .\" liability, or tort (including negligence or otherwise) arising in any way .\" out of the use of this software, even if advised of the possibility of .\" such damage. .\" -.\" $Id: elf_flagdata.3 2884 2013-01-11 02:03:46Z jkoshy $ +.\" $Id: elf_flagdata.3 3479 2016-06-25 20:44:33Z jkoshy $ .\" .Dd December 3, 2011 .Os .Dt ELF_FLAGDATA 3 .Sh NAME .Nm elf_flagarhdr , .Nm elf_flagdata , .Nm elf_flagehdr , .Nm elf_flagelf , .Nm elf_flagphdr , .Nm elf_flagscn , .Nm elf_flagshdr .Nd manipulate flags associated with ELF(3) data structures .Sh LIBRARY .Lb libelf .Sh SYNOPSIS .In libelf.h .Ft "unsigned int" .Fn elf_flagarhdr "Elf_Arhdr *arhdr" "Elf_Cmd cmd" "unsigned int flags" .Ft "unsigned int" .Fn elf_flagdata "Elf_Data *data" "Elf_Cmd cmd" "unsigned int flags" .Ft "unsigned int" .Fn elf_flagehdr "Elf *elf" "Elf_Cmd cmd" "unsigned int flags" .Ft "unsigned int" .Fn elf_flagelf "Elf *elf" "Elf_Cmd cmd" "unsigned int flags" .Ft "unsigned int" .Fn elf_flagphdr "Elf *elf" "Elf_Cmd cmd" "unsigned int flags" .Ft "unsigned int" .Fn elf_flagscn "Elf_Scn *scn" "Elf_Cmd cmd" "unsigned int flags" .Ft "unsigned int" .Fn elf_flagshdr "Elf_Scn *scn" "Elf_Cmd cmd" "unsigned int flags" .Sh DESCRIPTION These functions are used to query, set or reset flags on data structures associated with an ELF file. .Pp Arguments .Ar arhdr , .Ar data , .Ar elf and .Ar scn denote the data structures whose flags need to be changed. These values should have been returned by prior calls to functions in the ELF(3) API set: .Bl -bullet -compact .It Argument .Ar arhdr should have been returned by a prior call to .Xr elf_getarhdr 3 . .It Argument .Ar data should have been returned by a prior call to one of .Xr elf_newdata 3 , .Xr elf_getdata 3 or .Xr elf_rawdata 3 . .It Argument .Ar elf should have been allocated by a prior call to one of .Xr elf_begin 3 or .Xr elf_memory 3 . .It Argument .Ar scn should have been returned by a prior call to one of .Xr elf_getscn 3 , .Xr elf_newscn 3 or .Xr elf_nextscn 3 . .El These values are allowed to be NULL to simplify error handling in application code. .Pp Argument .Ar cmd may have the following values: .Bl -tag -width ELF_C_SET .It Dv ELF_C_CLR The argument .Ar flags specifies the flags to be cleared. .It Dv ELF_C_SET The argument .Ar flags specifies the flags to be set. .El .Pp The argument .Ar flags is allowed to have the following flags set: .Bl -tag -width ELF_F_ARCHIVE_SYSV .It Dv ELF_F_ARCHIVE This flag is only valid with the .Fn elf_flagelf API. It informs the library that the application desires to create an .Xr ar 1 archive. Argument .Ar elf should have been opened for writing using the .Dv ELF_C_WRITE command to function .Fn elf_begin . .It Dv ELF_F_ARCHIVE_SYSV This flag is used in conjunction with the .Dv ELF_F_ARCHIVE flag to indicate that library should create archives that conform to System V layout rules. The default is to create BSD style archives. .It Dv ELF_F_DIRTY Mark the associated data structure as needing to be written back to the underlying file. A subsequent call to .Xr elf_update 3 will resynchronize the library's internal data structures. .It Dv ELF_F_LAYOUT This flag is only valid with the .Fn elf_flagelf API. It informs the library that the application will take responsibility for the layout of the file and that the library is not to insert any padding in between sections. .El .Pp Marking a given data structure as .Dq dirty affects all of its contained elements. Thus marking an ELF descriptor .Ar elf with .Fn elf_flagelf "elf" "ELF_C_SET" "ELF_F_DIRTY" means that the entire contents of the descriptor are .Dq dirty . .Pp Using a value of zero for argument .Ar flags will return the current set of flags for the data structure being queried. .Sh RETURN VALUES These functions return the updated flags if successful, or zero if an error is detected. .Sh COMPATIBILITY The .Fn elf_flagarhdr function and the .Dv ELF_F_ARCHIVE and .Dv ELF_F_ARCHIVE_SYSV flags are an extension to the ELF(3) API. .Sh ERRORS These functions may fail with the following errors: .Bl -tag -width "[ELF_E_RESOURCE]" .It Bq Er ELF_E_ARGUMENT An unsupported value was used for the .Ar cmd argument. .It Bq Er ELF_E_ARGUMENT Argument .Ar flags had unsupported flags set. .It Bq Er ELF_E_ARGUMENT The argument .Ar elf was not a descriptor for an ELF object. .It Bq Er ELF_E_MODE The .Dv ELF_F_ARCHIVE flag was used with an ELF descriptor that had not been opened for writing. .It Bq Er ELF_E_SEQUENCE Function .Fn elf_flagehdr was called without an executable header being allocated. .It Bq Er ELF_E_SEQUENCE Function .Fn elf_flagphdr was called without a program header being allocated. .El .Sh SEE ALSO .Xr elf 3 , .Xr elf32_newehdr 3 , .Xr elf32_newphdr 3 , -.Xr elf32_newshdr 3 , .Xr elf64_newehdr 3 , .Xr elf64_newphdr 3 , -.Xr elf64_newshdr 3 , .Xr elf_newdata 3 , .Xr elf_update 3 , .Xr gelf 3 , .Xr gelf_newehdr 3 , .Xr gelf_newphdr 3 , -.Xr gelf_newshdr 3 , .Xr gelf_update_dyn 3 , .Xr gelf_update_move 3 , .Xr gelf_update_rel 3 , .Xr gelf_update_rela 3 , .Xr gelf_update_sym 3 , .Xr gelf_update_syminfo 3 Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/Makefile =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/Makefile (revision 305172) @@ -1,58 +1,59 @@ -# $Id: Makefile 3418 2016-02-19 20:04:42Z emaste $ +# $Id: Makefile 3489 2016-08-31 00:12:15Z emaste $ TOP= ${.CURDIR}/.. LIB= elftc SRCS= elftc_bfdtarget.c \ elftc_copyfile.c \ elftc_demangle.c \ elftc_reloc_type_str.c \ elftc_set_timestamps.c \ elftc_string_table.c \ + elftc_timestamp.c \ elftc_version.c \ libelftc_bfdtarget.c \ libelftc_dem_arm.c \ libelftc_dem_gnu2.c \ libelftc_dem_gnu3.c \ libelftc_hash.c \ libelftc_vstr.c INCS= libelftc.h INCSDIR= /usr/include RELEASE= HEAD # Change this on release branches. SHLIB_MAJOR= 1 WARNS?= 6 CLEANFILES+= elftc_version.c LDADD+= -lelf MAN= elftc.3 \ elftc_bfd_find_target.3 \ elftc_copyfile.3 \ elftc_demangle.3 \ elftc_reloc_type_str.3 \ elftc_set_timestamps.3 \ elftc_string_table_create.3 \ elftc_version.3 MLINKS= elftc_bfd_find_target.3 elftc_bfd_target_byteorder.3 \ elftc_bfd_find_target.3 elftc_bfd_target_class.3 \ elftc_bfd_find_target.3 elftc_bfd_target_flavor.3 \ elftc_string_table_create.3 elftc_string_table_from_section.3 \ elftc_string_table_create.3 elftc_string_table_destroy.3 \ elftc_string_table_create.3 elftc_string_table_image.3 \ elftc_string_table_create.3 elftc_string_table_insert.3 \ elftc_string_table_create.3 elftc_string_table_lookup.3 .if !make(clean) && !make(clobber) .BEGIN: .SILENT ${.CURDIR}/make-toolchain-version -t ${TOP} -r ${RELEASE} \ -h ${OS_HOST} .endif .include "${TOP}/mk/elftoolchain.lib.mk" Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_bfd_find_target.3 =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_bfd_find_target.3 (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_bfd_find_target.3 (revision 305172) @@ -1,194 +1,194 @@ .\" Copyright (c) 2010-2011 Joseph Koshy. All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" This software is provided by Joseph Koshy ``as is'' and .\" any express or implied warranties, including, but not limited to, the .\" implied warranties of merchantability and fitness for a particular purpose .\" are disclaimed. in no event shall Joseph Koshy be liable .\" for any direct, indirect, incidental, special, exemplary, or consequential .\" damages (including, but not limited to, procurement of substitute goods .\" or services; loss of use, data, or profits; or business interruption) .\" however caused and on any theory of liability, whether in contract, strict .\" liability, or tort (including negligence or otherwise) arising in any way .\" out of the use of this software, even if advised of the possibility of .\" such damage. .\" -.\" $Id: elftc_bfd_find_target.3 3348 2016-01-18 14:18:50Z emaste $ +.\" $Id: elftc_bfd_find_target.3 3488 2016-08-24 18:15:57Z emaste $ .\" .Dd November 30, 2011 .Os .Dt ELFTC_BFD_FIND_TARGET .Sh NAME .Nm elftc_bfd_find_target , .Nm elftc_bfd_target_byteorder , .Nm elftc_bfd_target_class , .Nm elftc_bfd_target_flavor , .Nm elftc_bfd_target_machine .Nd binary object descriptor handling .Sh LIBRARY .Lb libelftc .Sh SYNOPSIS .In libelftc.h .Vt struct Elftc_Bfd_Target; .Ft "Elftc_Bfd_Target *" .Fn elftc_bfd_find_target "const char *target_name" .Ft "unsigned int" .Fn elftc_bfd_target_class "Elftc_Bfd_Target *target" .Ft "unsigned int" .Fn elftc_bfd_target_byteorder "Elftc_Bfd_Target *target" .Ft Elftc_Bfd_Target_Flavor .Fn elftc_bfd_target_flavor "Elftc_Bfd_Target *target" .Ft "unsigned int" .Fn elftc_bfd_target_machine "Elftc_Bfd_Target *target" .Sh DESCRIPTION Function .Fn elftc_bfd_find_target locates a binary object descriptor corresponding to the descriptor name in argument .Ar "target_name" . Binary object descriptors encapsulate properties of an object format such as its file representation, ELF class, and byte endianness. .Pp Known descriptor names and their properties include: .Bl -column -offset "XXXX" ".Li elf32-x86-64-freebsd" "Object format" "Byte Order" "Bit Width" .It Em Name Ta Em "Object Format" Ta Em "Byte Order" Ta Em "Bit Width" .It Li binary Ta Binary Ta - Ta - .It Li efi-app-ia32 Ta PE Ta LSB Ta 32 .It Li efi-app-x86_64 Ta PE Ta LSB Ta 64 .It Li elf32-avr Ta ELF Ta LSB Ta 32 .It Li elf32-big Ta ELF Ta MSB Ta 32 .It Li elf32-bigarm Ta ELF Ta MSB Ta 32 .It Li elf32-bigmips Ta ELF Ta MSB Ta 32 .It Li elf32-i386 Ta ELF Ta LSB Ta 32 .It Li elf32-i386-freebsd Ta ELF Ta LSB Ta 32 .It Li elf32-ia64-big Ta ELF Ta MSB Ta 32 .It Li elf32-little Ta ELF Ta LSB Ta 32 .It Li elf32-littlearm Ta ELF Ta LSB Ta 32 .It Li elf32-littlemips Ta ELF Ta LSB Ta 32 .It Li elf32-powerpc Ta ELF Ta MSB Ta 32 .It Li elf32-powerpcle Ta ELF Ta LSB Ta 32 .It Li elf32-sh Ta ELF Ta MSB Ta 32 .It Li elf32-shl Ta ELF Ta LSB Ta 32 .It Li elf32-sh-nbsd Ta ELF Ta MSB Ta 32 .It Li elf32-shl-nbsd Ta ELF Ta LSB Ta 32 .It Li elf32-shbig-linux Ta ELF Ta MSB Ta 32 .It Li elf32-shl-linux Ta ELF Ta LSB Ta 32 .It Li elf32-sparc Ta ELF Ta MSB Ta 32 .It Li elf64-alpha Ta ELF Ta LSB Ta 64 .It Li elf64-alpha-freebsd Ta ELF Ta LSB Ta 64 .It Li elf64-big Ta ELF Ta MSB Ta 64 .It Li elf64-bigmips Ta ELF Ta MSB Ta 64 .It Li elf64-ia64-big Ta ELF Ta MSB Ta 64 .It Li elf64-ia64-little Ta ELF Ta LSB Ta 64 .It Li elf64-little Ta ELF Ta LSB Ta 64 .It Li elf64-littleaarch64 Ta ELF Ta LSB Ta 64 .It Li elf64-littlemips Ta ELF Ta LSB Ta 64 .It Li elf64-powerpc Ta ELF Ta MSB Ta 64 .It Li elf64-powerpcle Ta ELF Ta LSB Ta 64 .It Li elf64-sh64 Ta ELF Ta MSB Ta 64 .It Li elf64-sh64l Ta ELF Ta LSB Ta 64 .It Li elf64-sh64-nbsd Ta ELF Ta MSB Ta 64 .It Li elf64-sh64l-nbsd Ta ELF Ta LSB Ta 64 .It Li elf64-sh64big-linux Ta ELF Ta MSB Ta 64 .It Li elf64-sh64-linux Ta ELF Ta LSB Ta 64 .It Li elf64-sparc Ta ELF Ta MSB Ta 64 .It Li elf64-sparc-freebsd Ta ELF Ta MSB Ta 64 .It Li elf64-x86-64 Ta ELF Ta LSB Ta 64 .It Li elf64-x86-64-freebsd Ta ELF Ta LSB Ta 64 .It Li ihex Ta IHEX Ta - Ta - .It Li pei-i386 Ta PE Ta LSB Ta 32 .It Li pei-x86-64 Ta PE Ta LSB Ta 64 .It Li srec Ta SREC Ta - Ta - .It Li symbolsrec Ta SREC Ta - Ta - .El .Pp Function .Fn elftc_bfd_target_byteorder returns the ELF byte order associated with target descriptor .Ar target . .Pp Function .Fn elftc_bfd_target_class returns the ELF class associated with target descriptor .Ar target . .Pp Function .Fn elftc_bfd_target_flavor returns the object format associated with target descriptor .Ar target . The known object formats are: .Bl -tag -offset "XXXX" -width ".Dv ETF_BINARY" -compact .It Dv ETF_ELF An ELF object. .It Dv ETF_BINARY Raw binary. .It Dv ETF_IHEX An object encoded in .Tn Intel hex format. .It Dv ETF_NONE An unknown object format. .It Dv ETF_SREC An object encoded as S-records. .El .Sh RETURN VALUES Function .Fn elftc_bfd_find_target returns a valid pointer to an opaque binary target descriptor if successful, or NULL in case of an error. .Pp Function .Fn elftc_bfd_target_byteorder returns the ELF byte order associated with the target descriptor; one of .Dv ELFDATA2MSB or .Dv ELFDATA2LSB . .Pp Function .Fn elftc_bfd_target_class returns the ELF class associated with the target descriptor; one of .Dv ELFCLASS32 or .Dv ELFCLASS64 . .Pp Function .Fn elftc_bfd_target_machine returns the ELF architecture associated with the target descriptor. .Pp Function .Fn elftc_bfd_target_flavor returns one of .Dv ETF_BINARY , .Dv ETF_ELF , .Dv ETF_IHEX or .Dv ETF_SREC if successful or .Dv ETF_NONE in case of error. .Sh EXAMPLES To return descriptor information associated with target name .Dq elf64-big use: .Bd -literal -offset indent struct Elftc_Bfd_Target *t; if ((t = elftc_bfd_find_target("elf64-big")) == NULL) errx(EXIT_FAILURE, "Cannot find target descriptor"); printf("Class: %s\\n", elftc_bfd_target_class(t) == ELFCLASS32 ? "ELFCLASS32" : "ELFCLASS64"); printf("Byteorder: %s\\n", elftc_bfd_target_byteorder(t) == ELFDATA2LSB ? "LSB" : "MSB"); printf("Flavor: %d\\n", elftc_bfd_target_flavor(t)); .Ed .Sh SEE ALSO .Xr elf 3 Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_reloc_type_str.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_reloc_type_str.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_reloc_type_str.c (revision 305172) @@ -1,684 +1,687 @@ /*- * Copyright (c) 2009-2015 Kai Wang * Copyright (c) 2016 The FreeBSD Foundation * All rights reserved. * * Portions of this software were developed by Ed Maste under sponsorship * of the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include const char * elftc_reloc_type_str(unsigned int mach, unsigned int type) { static char s_type[32]; switch(mach) { case EM_386: case EM_IAMCU: switch(type) { case 0: return "R_386_NONE"; case 1: return "R_386_32"; case 2: return "R_386_PC32"; case 3: return "R_386_GOT32"; case 4: return "R_386_PLT32"; case 5: return "R_386_COPY"; case 6: return "R_386_GLOB_DAT"; case 7: return "R_386_JUMP_SLOT"; case 8: return "R_386_RELATIVE"; case 9: return "R_386_GOTOFF"; case 10: return "R_386_GOTPC"; case 11: return "R_386_32PLT"; /* Not in psabi */ case 14: return "R_386_TLS_TPOFF"; case 15: return "R_386_TLS_IE"; case 16: return "R_386_TLS_GOTIE"; case 17: return "R_386_TLS_LE"; case 18: return "R_386_TLS_GD"; case 19: return "R_386_TLS_LDM"; case 20: return "R_386_16"; case 21: return "R_386_PC16"; case 22: return "R_386_8"; case 23: return "R_386_PC8"; case 24: return "R_386_TLS_GD_32"; case 25: return "R_386_TLS_GD_PUSH"; case 26: return "R_386_TLS_GD_CALL"; case 27: return "R_386_TLS_GD_POP"; case 28: return "R_386_TLS_LDM_32"; case 29: return "R_386_TLS_LDM_PUSH"; case 30: return "R_386_TLS_LDM_CALL"; case 31: return "R_386_TLS_LDM_POP"; case 32: return "R_386_TLS_LDO_32"; case 33: return "R_386_TLS_IE_32"; case 34: return "R_386_TLS_LE_32"; case 35: return "R_386_TLS_DTPMOD32"; case 36: return "R_386_TLS_DTPOFF32"; case 37: return "R_386_TLS_TPOFF32"; case 38: return "R_386_SIZE32"; case 39: return "R_386_TLS_GOTDESC"; case 40: return "R_386_TLS_DESC_CALL"; case 41: return "R_386_TLS_DESC"; case 42: return "R_386_IRELATIVE"; case 43: return "R_386_GOT32X"; } break; case EM_AARCH64: switch(type) { case 0: return "R_AARCH64_NONE"; case 257: return "R_AARCH64_ABS64"; case 258: return "R_AARCH64_ABS32"; case 259: return "R_AARCH64_ABS16"; case 260: return "R_AARCH64_PREL64"; case 261: return "R_AARCH64_PREL32"; case 262: return "R_AARCH64_PREL16"; case 263: return "R_AARCH64_MOVW_UABS_G0"; case 264: return "R_AARCH64_MOVW_UABS_G0_NC"; case 265: return "R_AARCH64_MOVW_UABS_G1"; case 266: return "R_AARCH64_MOVW_UABS_G1_NC"; case 267: return "R_AARCH64_MOVW_UABS_G2"; case 268: return "R_AARCH64_MOVW_UABS_G2_NC"; case 269: return "R_AARCH64_MOVW_UABS_G3"; case 270: return "R_AARCH64_MOVW_SABS_G0"; case 271: return "R_AARCH64_MOVW_SABS_G1"; case 272: return "R_AARCH64_MOVW_SABS_G2"; case 273: return "R_AARCH64_LD_PREL_LO19"; case 274: return "R_AARCH64_ADR_PREL_LO21"; case 275: return "R_AARCH64_ADR_PREL_PG_HI21"; case 276: return "R_AARCH64_ADR_PREL_PG_HI21_NC"; case 277: return "R_AARCH64_ADD_ABS_LO12_NC"; case 278: return "R_AARCH64_LDST8_ABS_LO12_NC"; case 279: return "R_AARCH64_TSTBR14"; case 280: return "R_AARCH64_CONDBR19"; case 282: return "R_AARCH64_JUMP26"; case 283: return "R_AARCH64_CALL26"; case 284: return "R_AARCH64_LDST16_ABS_LO12_NC"; case 285: return "R_AARCH64_LDST32_ABS_LO12_NC"; case 286: return "R_AARCH64_LDST64_ABS_LO12_NC"; case 287: return "R_AARCH64_MOVW_PREL_G0"; case 288: return "R_AARCH64_MOVW_PREL_G0_NC"; case 289: return "R_AARCH64_MOVW_PREL_G1"; case 290: return "R_AARCH64_MOVW_PREL_G1_NC"; case 291: return "R_AARCH64_MOVW_PREL_G2"; case 292: return "R_AARCH64_MOVW_PREL_G2_NC"; case 293: return "R_AARCH64_MOVW_PREL_G3"; case 299: return "R_AARCH64_LDST128_ABS_LO12_NC"; case 300: return "R_AARCH64_MOVW_GOTOFF_G0"; case 301: return "R_AARCH64_MOVW_GOTOFF_G0_NC"; case 302: return "R_AARCH64_MOVW_GOTOFF_G1"; case 303: return "R_AARCH64_MOVW_GOTOFF_G1_NC"; case 304: return "R_AARCH64_MOVW_GOTOFF_G2"; case 305: return "R_AARCH64_MOVW_GOTOFF_G2_NC"; case 306: return "R_AARCH64_MOVW_GOTOFF_G3"; case 307: return "R_AARCH64_GOTREL64"; case 308: return "R_AARCH64_GOTREL32"; case 309: return "R_AARCH64_GOT_LD_PREL19"; case 310: return "R_AARCH64_LD64_GOTOFF_LO15"; case 311: return "R_AARCH64_ADR_GOT_PAGE"; case 312: return "R_AARCH64_LD64_GOT_LO12_NC"; case 313: return "R_AARCH64_LD64_GOTPAGE_LO15"; case 560: return "R_AARCH64_TLSDESC_LD_PREL19"; case 561: return "R_AARCH64_TLSDESC_ADR_PREL21"; case 562: return "R_AARCH64_TLSDESC_ADR_PAGE21"; case 563: return "R_AARCH64_TLSDESC_LD64_LO12"; case 564: return "R_AARCH64_TLSDESC_ADD_LO12"; case 565: return "R_AARCH64_TLSDESC_OFF_G1"; case 566: return "R_AARCH64_TLSDESC_OFF_G0_NC"; case 567: return "R_AARCH64_TLSDESC_LDR"; case 568: return "R_AARCH64_TLSDESC_ADD"; case 569: return "R_AARCH64_TLSDESC_CALL"; case 1024: return "R_AARCH64_COPY"; case 1025: return "R_AARCH64_GLOB_DAT"; case 1026: return "R_AARCH64_JUMP_SLOT"; case 1027: return "R_AARCH64_RELATIVE"; case 1028: return "R_AARCH64_TLS_DTPREL64"; case 1029: return "R_AARCH64_TLS_DTPMOD64"; case 1030: return "R_AARCH64_TLS_TPREL64"; case 1031: return "R_AARCH64_TLSDESC"; case 1032: return "R_AARCH64_IRELATIVE"; } break; case EM_ARM: switch(type) { case 0: return "R_ARM_NONE"; case 1: return "R_ARM_PC24"; /* Deprecated */ case 2: return "R_ARM_ABS32"; case 3: return "R_ARM_REL32"; case 4: return "R_ARM_LDR_PC_G0"; /* Also R_ARM_PC13 */ case 5: return "R_ARM_ABS16"; case 6: return "R_ARM_ABS12"; case 7: return "R_ARM_THM_ABS5"; case 8: return "R_ARM_ABS8"; case 9: return "R_ARM_SBREL32"; case 10: return "R_ARM_THM_CALL"; /* Also R_ARM_THM_PC22 */ case 11: return "R_ARM_THM_PC8"; case 12: return "R_ARM_BREL_ADJ"; /* Also R_ARM_AMP_VCALL9 */ case 13: return "R_ARM_TLS_DESC"; /* Also R_ARM_SWI24 */ case 14: return "R_ARM_THM_SWI8"; /* Obsolete */ case 15: return "R_ARM_XPC25"; /* Obsolete */ case 16: return "R_ARM_THM_XPC22"; /* Obsolete */ case 17: return "R_ARM_TLS_DTPMOD32"; case 18: return "R_ARM_TLS_DTPOFF32"; case 19: return "R_ARM_TLS_TPOFF32"; case 20: return "R_ARM_COPY"; case 21: return "R_ARM_GLOB_DAT"; case 22: return "R_ARM_JUMP_SLOT"; case 23: return "R_ARM_RELATIVE"; case 24: return "R_ARM_GOTOFF32"; /* Also R_ARM_GOTOFF */ case 25: return "R_ARM_BASE_PREL"; /* GNU R_ARM_GOTPC */ case 26: return "R_ARM_GOT_BREL"; /* GNU R_ARM_GOT32 */ case 27: return "R_ARM_PLT32"; /* Deprecated */ case 28: return "R_ARM_CALL"; case 29: return "R_ARM_JUMP24"; case 30: return "R_ARM_THM_JUMP24"; case 31: return "R_ARM_BASE_ABS"; case 32: return "R_ARM_ALU_PCREL_7_0"; /* Obsolete */ case 33: return "R_ARM_ALU_PCREL_15_8"; /* Obsolete */ case 34: return "R_ARM_ALU_PCREL_23_15"; /* Obsolete */ case 35: return "R_ARM_LDR_SBREL_11_0_NC"; /* Deprecated */ case 36: return "R_ARM_ALU_SBREL_19_12_NC"; /* Deprecated */ case 37: return "R_ARM_ALU_SBREL_27_20_CK"; /* Deprecated */ case 38: return "R_ARM_TARGET1"; case 39: return "R_ARM_SBREL31"; /* Deprecated. */ case 40: return "R_ARM_V4BX"; case 41: return "R_ARM_TARGET2"; case 42: return "R_ARM_PREL31"; case 43: return "R_ARM_MOVW_ABS_NC"; case 44: return "R_ARM_MOVT_ABS"; case 45: return "R_ARM_MOVW_PREL_NC"; case 46: return "R_ARM_MOVT_PREL"; case 47: return "R_ARM_THM_MOVW_ABS_NC"; case 48: return "R_ARM_THM_MOVT_ABS"; case 49: return "R_ARM_THM_MOVW_PREL_NC"; case 50: return "R_ARM_THM_MOVT_PREL"; case 51: return "R_ARM_THM_JUMP19"; case 52: return "R_ARM_THM_JUMP6"; case 53: return "R_ARM_THM_ALU_PREL_11_0"; case 54: return "R_ARM_THM_PC12"; case 55: return "R_ARM_ABS32_NOI"; case 56: return "R_ARM_REL32_NOI"; case 57: return "R_ARM_ALU_PC_G0_NC"; case 58: return "R_ARM_ALU_PC_G0"; case 59: return "R_ARM_ALU_PC_G1_NC"; case 60: return "R_ARM_ALU_PC_G1"; case 61: return "R_ARM_ALU_PC_G2"; case 62: return "R_ARM_LDR_PC_G1"; case 63: return "R_ARM_LDR_PC_G2"; case 64: return "R_ARM_LDRS_PC_G0"; case 65: return "R_ARM_LDRS_PC_G1"; case 66: return "R_ARM_LDRS_PC_G2"; case 67: return "R_ARM_LDC_PC_G0"; case 68: return "R_ARM_LDC_PC_G1"; case 69: return "R_ARM_LDC_PC_G2"; case 70: return "R_ARM_ALU_SB_G0_NC"; case 71: return "R_ARM_ALU_SB_G0"; case 72: return "R_ARM_ALU_SB_G1_NC"; case 73: return "R_ARM_ALU_SB_G1"; case 74: return "R_ARM_ALU_SB_G2"; case 75: return "R_ARM_LDR_SB_G0"; case 76: return "R_ARM_LDR_SB_G1"; case 77: return "R_ARM_LDR_SB_G2"; case 78: return "R_ARM_LDRS_SB_G0"; case 79: return "R_ARM_LDRS_SB_G1"; case 80: return "R_ARM_LDRS_SB_G2"; case 81: return "R_ARM_LDC_SB_G0"; case 82: return "R_ARM_LDC_SB_G1"; case 83: return "R_ARM_LDC_SB_G2"; case 84: return "R_ARM_MOVW_BREL_NC"; case 85: return "R_ARM_MOVT_BREL"; case 86: return "R_ARM_MOVW_BREL"; case 87: return "R_ARM_THM_MOVW_BREL_NC"; case 88: return "R_ARM_THM_MOVT_BREL"; case 89: return "R_ARM_THM_MOVW_BREL"; case 90: return "R_ARM_TLS_GOTDESC"; case 91: return "R_ARM_TLS_CALL"; case 92: return "R_ARM_TLS_DESCSEQ"; case 93: return "R_ARM_THM_TLS_CALL"; case 94: return "R_ARM_PLT32_ABS"; case 95: return "R_ARM_GOT_ABS"; case 96: return "R_ARM_GOT_PREL"; case 97: return "R_ARM_GOT_BREL12"; case 98: return "R_ARM_GOTOFF12"; case 99: return "R_ARM_GOTRELAX"; case 100: return "R_ARM_GNU_VTENTRY"; case 101: return "R_ARM_GNU_VTINHERIT"; case 102: return "R_ARM_THM_JUMP11"; /* Also R_ARM_THM_PC11 */ case 103: return "R_ARM_THM_JUMP8"; /* Also R_ARM_THM_PC9 */ case 104: return "R_ARM_TLS_GD32"; case 105: return "R_ARM_TLS_LDM32"; case 106: return "R_ARM_TLS_LDO32"; case 107: return "R_ARM_TLS_IE32"; case 108: return "R_ARM_TLS_LE32"; case 109: return "R_ARM_TLS_LDO12"; case 110: return "R_ARM_TLS_LE12"; case 111: return "R_ARM_TLS_IE12GP"; /* 112-127 R_ARM_PRIVATE_ */ case 128: return "R_ARM_ME_TOO"; /* Obsolete */ case 129: return "R_ARM_THM_TLS_DESCSEQ16"; case 130: return "R_ARM_THM_TLS_DESCSEQ32"; case 131: return "R_ARM_THM_GOT_BREL12"; case 132: return "R_ARM_THM_ALU_ABS_G0_NC"; case 133: return "R_ARM_THM_ALU_ABS_G1_NC"; case 134: return "R_ARM_THM_ALU_ABS_G2_NC"; case 135: return "R_ARM_THM_ALU_ABS_G3"; /* 136-159 Reserved for future allocation. */ case 160: return "R_ARM_IRELATIVE"; /* 161-255 Reserved for future allocation. */ case 249: return "R_ARM_RXPC25"; case 250: return "R_ARM_RSBREL32"; case 251: return "R_ARM_THM_RPC22"; case 252: return "R_ARM_RREL32"; case 253: return "R_ARM_RABS32"; case 254: return "R_ARM_RPC24"; case 255: return "R_ARM_RBASE"; } break; case EM_IA_64: switch(type) { case 0: return "R_IA_64_NONE"; case 33: return "R_IA_64_IMM14"; case 34: return "R_IA_64_IMM22"; case 35: return "R_IA_64_IMM64"; case 36: return "R_IA_64_DIR32MSB"; case 37: return "R_IA_64_DIR32LSB"; case 38: return "R_IA_64_DIR64MSB"; case 39: return "R_IA_64_DIR64LSB"; case 42: return "R_IA_64_GPREL22"; case 43: return "R_IA_64_GPREL64I"; case 44: return "R_IA_64_GPREL32MSB"; case 45: return "R_IA_64_GPREL32LSB"; case 46: return "R_IA_64_GPREL64MSB"; case 47: return "R_IA_64_GPREL64LSB"; case 50: return "R_IA_64_LTOFF22"; case 51: return "R_IA_64_LTOFF64I"; case 58: return "R_IA_64_PLTOFF22"; case 59: return "R_IA_64_PLTOFF64I"; case 62: return "R_IA_64_PLTOFF64MSB"; case 63: return "R_IA_64_PLTOFF64LSB"; case 67: return "R_IA_64_FPTR64I"; case 68: return "R_IA_64_FPTR32MSB"; case 69: return "R_IA_64_FPTR32LSB"; case 70: return "R_IA_64_FPTR64MSB"; case 71: return "R_IA_64_FPTR64LSB"; case 72: return "R_IA_64_PCREL60B"; case 73: return "R_IA_64_PCREL21B"; case 74: return "R_IA_64_PCREL21M"; case 75: return "R_IA_64_PCREL21F"; case 76: return "R_IA_64_PCREL32MSB"; case 77: return "R_IA_64_PCREL32LSB"; case 78: return "R_IA_64_PCREL64MSB"; case 79: return "R_IA_64_PCREL64LSB"; case 82: return "R_IA_64_LTOFF_FPTR22"; case 83: return "R_IA_64_LTOFF_FPTR64I"; case 84: return "R_IA_64_LTOFF_FPTR32MSB"; case 85: return "R_IA_64_LTOFF_FPTR32LSB"; case 86: return "R_IA_64_LTOFF_FPTR64MSB"; case 87: return "R_IA_64_LTOFF_FPTR64LSB"; case 92: return "R_IA_64_SEGREL32MSB"; case 93: return "R_IA_64_SEGREL32LSB"; case 94: return "R_IA_64_SEGREL64MSB"; case 95: return "R_IA_64_SEGREL64LSB"; case 100: return "R_IA_64_SECREL32MSB"; case 101: return "R_IA_64_SECREL32LSB"; case 102: return "R_IA_64_SECREL64MSB"; case 103: return "R_IA_64_SECREL64LSB"; case 108: return "R_IA_64_REL32MSB"; case 109: return "R_IA_64_REL32LSB"; case 110: return "R_IA_64_REL64MSB"; case 111: return "R_IA_64_REL64LSB"; case 116: return "R_IA_64_LTV32MSB"; case 117: return "R_IA_64_LTV32LSB"; case 118: return "R_IA_64_LTV64MSB"; case 119: return "R_IA_64_LTV64LSB"; case 121: return "R_IA_64_PCREL21BI"; case 122: return "R_IA_64_PCREL22"; case 123: return "R_IA_64_PCREL64I"; case 128: return "R_IA_64_IPLTMSB"; case 129: return "R_IA_64_IPLTLSB"; case 133: return "R_IA_64_SUB"; case 134: return "R_IA_64_LTOFF22X"; case 135: return "R_IA_64_LDXMOV"; case 145: return "R_IA_64_TPREL14"; case 146: return "R_IA_64_TPREL22"; case 147: return "R_IA_64_TPREL64I"; case 150: return "R_IA_64_TPREL64MSB"; case 151: return "R_IA_64_TPREL64LSB"; case 154: return "R_IA_64_LTOFF_TPREL22"; case 166: return "R_IA_64_DTPMOD64MSB"; case 167: return "R_IA_64_DTPMOD64LSB"; case 170: return "R_IA_64_LTOFF_DTPMOD22"; case 177: return "R_IA_64_DTPREL14"; case 178: return "R_IA_64_DTPREL22"; case 179: return "R_IA_64_DTPREL64I"; case 180: return "R_IA_64_DTPREL32MSB"; case 181: return "R_IA_64_DTPREL32LSB"; case 182: return "R_IA_64_DTPREL64MSB"; case 183: return "R_IA_64_DTPREL64LSB"; case 186: return "R_IA_64_LTOFF_DTPREL22"; } break; case EM_MIPS: switch(type) { case 0: return "R_MIPS_NONE"; case 1: return "R_MIPS_16"; case 2: return "R_MIPS_32"; case 3: return "R_MIPS_REL32"; case 4: return "R_MIPS_26"; case 5: return "R_MIPS_HI16"; case 6: return "R_MIPS_LO16"; case 7: return "R_MIPS_GPREL16"; case 8: return "R_MIPS_LITERAL"; case 9: return "R_MIPS_GOT16"; case 10: return "R_MIPS_PC16"; case 11: return "R_MIPS_CALL16"; case 12: return "R_MIPS_GPREL32"; case 16: return "R_MIPS_SHIFT5"; case 17: return "R_MIPS_SHIFT6"; case 18: return "R_MIPS_64"; case 19: return "R_MIPS_GOT_DISP"; case 20: return "R_MIPS_GOT_PAGE"; case 21: return "R_MIPS_GOT_OFST"; case 22: return "R_MIPS_GOT_HI16"; case 23: return "R_MIPS_GOT_LO16"; case 24: return "R_MIPS_SUB"; case 30: return "R_MIPS_CALLHI16"; case 31: return "R_MIPS_CALLLO16"; case 37: return "R_MIPS_JALR"; case 38: return "R_MIPS_TLS_DTPMOD32"; case 39: return "R_MIPS_TLS_DTPREL32"; case 40: return "R_MIPS_TLS_DTPMOD64"; case 41: return "R_MIPS_TLS_DTPREL64"; case 42: return "R_MIPS_TLS_GD"; case 43: return "R_MIPS_TLS_LDM"; case 44: return "R_MIPS_TLS_DTPREL_HI16"; case 45: return "R_MIPS_TLS_DTPREL_LO16"; case 46: return "R_MIPS_TLS_GOTTPREL"; case 47: return "R_MIPS_TLS_TPREL32"; case 48: return "R_MIPS_TLS_TPREL64"; case 49: return "R_MIPS_TLS_TPREL_HI16"; case 50: return "R_MIPS_TLS_TPREL_LO16"; } break; case EM_PPC: switch(type) { case 0: return "R_PPC_NONE"; case 1: return "R_PPC_ADDR32"; case 2: return "R_PPC_ADDR24"; case 3: return "R_PPC_ADDR16"; case 4: return "R_PPC_ADDR16_LO"; case 5: return "R_PPC_ADDR16_HI"; case 6: return "R_PPC_ADDR16_HA"; case 7: return "R_PPC_ADDR14"; case 8: return "R_PPC_ADDR14_BRTAKEN"; case 9: return "R_PPC_ADDR14_BRNTAKEN"; case 10: return "R_PPC_REL24"; case 11: return "R_PPC_REL14"; case 12: return "R_PPC_REL14_BRTAKEN"; case 13: return "R_PPC_REL14_BRNTAKEN"; case 14: return "R_PPC_GOT16"; case 15: return "R_PPC_GOT16_LO"; case 16: return "R_PPC_GOT16_HI"; case 17: return "R_PPC_GOT16_HA"; case 18: return "R_PPC_PLTREL24"; case 19: return "R_PPC_COPY"; case 20: return "R_PPC_GLOB_DAT"; case 21: return "R_PPC_JMP_SLOT"; case 22: return "R_PPC_RELATIVE"; case 23: return "R_PPC_LOCAL24PC"; case 24: return "R_PPC_UADDR32"; case 25: return "R_PPC_UADDR16"; case 26: return "R_PPC_REL32"; case 27: return "R_PPC_PLT32"; case 28: return "R_PPC_PLTREL32"; case 29: return "R_PPC_PLT16_LO"; case 30: return "R_PPC_PLT16_HI"; case 31: return "R_PPC_PLT16_HA"; case 32: return "R_PPC_SDAREL16"; case 33: return "R_PPC_SECTOFF"; case 34: return "R_PPC_SECTOFF_LO"; case 35: return "R_PPC_SECTOFF_HI"; case 36: return "R_PPC_SECTOFF_HA"; case 67: return "R_PPC_TLS"; case 68: return "R_PPC_DTPMOD32"; case 69: return "R_PPC_TPREL16"; case 70: return "R_PPC_TPREL16_LO"; case 71: return "R_PPC_TPREL16_HI"; case 72: return "R_PPC_TPREL16_HA"; case 73: return "R_PPC_TPREL32"; case 74: return "R_PPC_DTPREL16"; case 75: return "R_PPC_DTPREL16_LO"; case 76: return "R_PPC_DTPREL16_HI"; case 77: return "R_PPC_DTPREL16_HA"; case 78: return "R_PPC_DTPREL32"; case 79: return "R_PPC_GOT_TLSGD16"; case 80: return "R_PPC_GOT_TLSGD16_LO"; case 81: return "R_PPC_GOT_TLSGD16_HI"; case 82: return "R_PPC_GOT_TLSGD16_HA"; case 83: return "R_PPC_GOT_TLSLD16"; case 84: return "R_PPC_GOT_TLSLD16_LO"; case 85: return "R_PPC_GOT_TLSLD16_HI"; case 86: return "R_PPC_GOT_TLSLD16_HA"; case 87: return "R_PPC_GOT_TPREL16"; case 88: return "R_PPC_GOT_TPREL16_LO"; case 89: return "R_PPC_GOT_TPREL16_HI"; case 90: return "R_PPC_GOT_TPREL16_HA"; case 101: return "R_PPC_EMB_NADDR32"; case 102: return "R_PPC_EMB_NADDR16"; case 103: return "R_PPC_EMB_NADDR16_LO"; case 104: return "R_PPC_EMB_NADDR16_HI"; case 105: return "R_PPC_EMB_NADDR16_HA"; case 106: return "R_PPC_EMB_SDAI16"; case 107: return "R_PPC_EMB_SDA2I16"; case 108: return "R_PPC_EMB_SDA2REL"; case 109: return "R_PPC_EMB_SDA21"; case 110: return "R_PPC_EMB_MRKREF"; case 111: return "R_PPC_EMB_RELSEC16"; case 112: return "R_PPC_EMB_RELST_LO"; case 113: return "R_PPC_EMB_RELST_HI"; case 114: return "R_PPC_EMB_RELST_HA"; case 115: return "R_PPC_EMB_BIT_FLD"; case 116: return "R_PPC_EMB_RELSDA"; } break; case EM_RISCV: switch(type) { case 0: return "R_RISCV_NONE"; case 1: return "R_RISCV_32"; case 2: return "R_RISCV_64"; case 3: return "R_RISCV_RELATIVE"; case 4: return "R_RISCV_COPY"; case 5: return "R_RISCV_JUMP_SLOT"; case 6: return "R_RISCV_TLS_DTPMOD32"; case 7: return "R_RISCV_TLS_DTPMOD64"; case 8: return "R_RISCV_TLS_DTPREL32"; case 9: return "R_RISCV_TLS_DTPREL64"; case 10: return "R_RISCV_TLS_TPREL32"; case 11: return "R_RISCV_TLS_TPREL64"; case 16: return "R_RISCV_BRANCH"; case 17: return "R_RISCV_JAL"; case 18: return "R_RISCV_CALL"; case 19: return "R_RISCV_CALL_PLT"; case 20: return "R_RISCV_GOT_HI20"; case 21: return "R_RISCV_TLS_GOT_HI20"; case 22: return "R_RISCV_TLS_GD_HI20"; case 23: return "R_RISCV_PCREL_HI20"; case 24: return "R_RISCV_PCREL_LO12_I"; case 25: return "R_RISCV_PCREL_LO12_S"; case 26: return "R_RISCV_HI20"; case 27: return "R_RISCV_LO12_I"; case 28: return "R_RISCV_LO12_S"; case 29: return "R_RISCV_TPREL_HI20"; case 30: return "R_RISCV_TPREL_LO12_I"; case 31: return "R_RISCV_TPREL_LO12_S"; case 32: return "R_RISCV_TPREL_ADD"; case 33: return "R_RISCV_ADD8"; case 34: return "R_RISCV_ADD16"; case 35: return "R_RISCV_ADD32"; case 36: return "R_RISCV_ADD64"; case 37: return "R_RISCV_SUB8"; case 38: return "R_RISCV_SUB16"; case 39: return "R_RISCV_SUB32"; case 40: return "R_RISCV_SUB64"; case 41: return "R_RISCV_GNU_VTINHERIT"; case 42: return "R_RISCV_GNU_VTENTRY"; case 43: return "R_RISCV_ALIGN"; case 44: return "R_RISCV_RVC_BRANCH"; case 45: return "R_RISCV_RVC_JUMP"; + case 46: return "R_RISCV_RVC_LUI"; + case 47: return "R_RISCV_GPREL_I"; + case 48: return "R_RISCV_GPREL_S"; } break; case EM_SPARC: case EM_SPARCV9: switch(type) { case 0: return "R_SPARC_NONE"; case 1: return "R_SPARC_8"; case 2: return "R_SPARC_16"; case 3: return "R_SPARC_32"; case 4: return "R_SPARC_DISP8"; case 5: return "R_SPARC_DISP16"; case 6: return "R_SPARC_DISP32"; case 7: return "R_SPARC_WDISP30"; case 8: return "R_SPARC_WDISP22"; case 9: return "R_SPARC_HI22"; case 10: return "R_SPARC_22"; case 11: return "R_SPARC_13"; case 12: return "R_SPARC_LO10"; case 13: return "R_SPARC_GOT10"; case 14: return "R_SPARC_GOT13"; case 15: return "R_SPARC_GOT22"; case 16: return "R_SPARC_PC10"; case 17: return "R_SPARC_PC22"; case 18: return "R_SPARC_WPLT30"; case 19: return "R_SPARC_COPY"; case 20: return "R_SPARC_GLOB_DAT"; case 21: return "R_SPARC_JMP_SLOT"; case 22: return "R_SPARC_RELATIVE"; case 23: return "R_SPARC_UA32"; case 24: return "R_SPARC_PLT32"; case 25: return "R_SPARC_HIPLT22"; case 26: return "R_SPARC_LOPLT10"; case 27: return "R_SPARC_PCPLT32"; case 28: return "R_SPARC_PCPLT22"; case 29: return "R_SPARC_PCPLT10"; case 30: return "R_SPARC_10"; case 31: return "R_SPARC_11"; case 32: return "R_SPARC_64"; case 33: return "R_SPARC_OLO10"; case 34: return "R_SPARC_HH22"; case 35: return "R_SPARC_HM10"; case 36: return "R_SPARC_LM22"; case 37: return "R_SPARC_PC_HH22"; case 38: return "R_SPARC_PC_HM10"; case 39: return "R_SPARC_PC_LM22"; case 40: return "R_SPARC_WDISP16"; case 41: return "R_SPARC_WDISP19"; case 42: return "R_SPARC_GLOB_JMP"; case 43: return "R_SPARC_7"; case 44: return "R_SPARC_5"; case 45: return "R_SPARC_6"; case 46: return "R_SPARC_DISP64"; case 47: return "R_SPARC_PLT64"; case 48: return "R_SPARC_HIX22"; case 49: return "R_SPARC_LOX10"; case 50: return "R_SPARC_H44"; case 51: return "R_SPARC_M44"; case 52: return "R_SPARC_L44"; case 53: return "R_SPARC_REGISTER"; case 54: return "R_SPARC_UA64"; case 55: return "R_SPARC_UA16"; case 56: return "R_SPARC_TLS_GD_HI22"; case 57: return "R_SPARC_TLS_GD_LO10"; case 58: return "R_SPARC_TLS_GD_ADD"; case 59: return "R_SPARC_TLS_GD_CALL"; case 60: return "R_SPARC_TLS_LDM_HI22"; case 61: return "R_SPARC_TLS_LDM_LO10"; case 62: return "R_SPARC_TLS_LDM_ADD"; case 63: return "R_SPARC_TLS_LDM_CALL"; case 64: return "R_SPARC_TLS_LDO_HIX22"; case 65: return "R_SPARC_TLS_LDO_LOX10"; case 66: return "R_SPARC_TLS_LDO_ADD"; case 67: return "R_SPARC_TLS_IE_HI22"; case 68: return "R_SPARC_TLS_IE_LO10"; case 69: return "R_SPARC_TLS_IE_LD"; case 70: return "R_SPARC_TLS_IE_LDX"; case 71: return "R_SPARC_TLS_IE_ADD"; case 72: return "R_SPARC_TLS_LE_HIX22"; case 73: return "R_SPARC_TLS_LE_LOX10"; case 74: return "R_SPARC_TLS_DTPMOD32"; case 75: return "R_SPARC_TLS_DTPMOD64"; case 76: return "R_SPARC_TLS_DTPOFF32"; case 77: return "R_SPARC_TLS_DTPOFF64"; case 78: return "R_SPARC_TLS_TPOFF32"; case 79: return "R_SPARC_TLS_TPOFF64"; } break; case EM_X86_64: switch(type) { case 0: return "R_X86_64_NONE"; case 1: return "R_X86_64_64"; case 2: return "R_X86_64_PC32"; case 3: return "R_X86_64_GOT32"; case 4: return "R_X86_64_PLT32"; case 5: return "R_X86_64_COPY"; case 6: return "R_X86_64_GLOB_DAT"; case 7: return "R_X86_64_JUMP_SLOT"; case 8: return "R_X86_64_RELATIVE"; case 9: return "R_X86_64_GOTPCREL"; case 10: return "R_X86_64_32"; case 11: return "R_X86_64_32S"; case 12: return "R_X86_64_16"; case 13: return "R_X86_64_PC16"; case 14: return "R_X86_64_8"; case 15: return "R_X86_64_PC8"; case 16: return "R_X86_64_DTPMOD64"; case 17: return "R_X86_64_DTPOFF64"; case 18: return "R_X86_64_TPOFF64"; case 19: return "R_X86_64_TLSGD"; case 20: return "R_X86_64_TLSLD"; case 21: return "R_X86_64_DTPOFF32"; case 22: return "R_X86_64_GOTTPOFF"; case 23: return "R_X86_64_TPOFF32"; case 24: return "R_X86_64_PC64"; case 25: return "R_X86_64_GOTOFF64"; case 26: return "R_X86_64_GOTPC32"; case 27: return "R_X86_64_GOT64"; case 28: return "R_X86_64_GOTPCREL64"; case 29: return "R_X86_64_GOTPC64"; case 30: return "R_X86_64_GOTPLT64"; case 31: return "R_X86_64_PLTOFF64"; case 32: return "R_X86_64_SIZE32"; case 33: return "R_X86_64_SIZE64"; case 34: return "R_X86_64_GOTPC32_TLSDESC"; case 35: return "R_X86_64_TLSDESC_CALL"; case 36: return "R_X86_64_TLSDESC"; case 37: return "R_X86_64_IRELATIVE"; case 38: return "R_X86_64_RELATIVE64"; case 41: return "R_X86_64_GOTPCRELX"; case 42: return "R_X86_64_REX_GOTPCRELX"; } break; } snprintf(s_type, sizeof(s_type), "", type); return (s_type); } Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_timestamp.3 =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_timestamp.3 (nonexistent) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_timestamp.3 (revision 305172) @@ -0,0 +1,79 @@ +.\" Copyright (c) 2016 The FreeBSD Foundation. All rights reserved. +.\" +.\" This documentation was written by Ed Maste under sponsorship of +.\" the FreeBSD Foundation. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" This software is provided by the author and contributors ``as is'' and +.\" any express or implied warranties, including, but not limited to, the +.\" implied warranties of merchantability and fitness for a particular purpose +.\" are disclaimed. In no event shall the author or contributors be liable +.\" for any direct, indirect, incidental, special, exemplary, or consequential +.\" damages (including, but not limited to, procurement of substitute goods +.\" or services; loss of use, data, or profits; or business interruption) +.\" however caused and on any theory of liability, whether in contract, strict +.\" liability, or tort (including negligence or otherwise) arising in any way +.\" out of the use of this software, even if advised of the possibility of +.\" such damage. +.\" +.\" $Id$ +.\" +.Dd August 24, 2016 +.Os +.Dt ELFTC_TIMESTAMP 3 +.Sh NAME +.Nm elftc_timestamp +.Nd return the current or environment-provided timestamp +.Sh LIBRARY +.Lb libelftc +.Sh SYNOPSIS +.In libelftc.h +.Ft int +.Fo elftc_timestamp +.Fa "time_t *timestamp" +.Fc +.Sh DESCRIPTION +The +.Fn elftc_timestamp +function returns a timestamp supplied by the +.Ev SOURCE_DATE_EPOCH +environment variable, or the current time provided by +.Xr time 3 +if the environment variable is not set. +.Pp +The +.Ar timestamp +argument specifies a pointer to the location where the timestamp will be +stored. +.Sh RETURN VALUE +Function +.Fn elftc_timestamp +returns 0 on success, and -1 in the event of an error. +.Sh ERRORS +The +.Fn elftc_timestamp +function may fail with the following errors: +.Bl -tag -width ".Bq Er ERANGE" +.It Bq Er EINVAL +.Ev SOURCE_DATE_EPOCH +contains invalid characters. +.It Bq Er ERANGE +.Ev SOURCE_DATE_EPOCH +specifies a negative value or a value that cannot be stored in a +time_t. +.El +The +.Fn elftc_timestamp +function may also fail for any of the reasons described in +.Xr strtoll 3 . +.Sh SEE ALSO +.Xr strtoll 3 , +.Xr time 3 Property changes on: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_timestamp.3 ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_timestamp.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_timestamp.c (nonexistent) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_timestamp.c (revision 305172) @@ -0,0 +1,55 @@ +/*- + * Copyright (c) 2016 The FreeBSD Foundation + * All rights reserved. + * + * This software was developed by Ed Maste under sponsorship + * of the FreeBSD Foundation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +#include +#include +#include + +int +elftc_timestamp(time_t *timestamp) +{ + long long source_date_epoch; + char *env, *eptr; + + if ((env = getenv("SOURCE_DATE_EPOCH")) != NULL) { + errno = 0; + source_date_epoch = strtoll(env, &eptr, 10); + if (*eptr != '\0') + errno = EINVAL; + if (source_date_epoch < 0) + errno = ERANGE; + if (errno != 0) + return (-1); + *timestamp = source_date_epoch; + return (0); + } + *timestamp = time(NULL); + return (0); +} Property changes on: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/elftc_timestamp.c ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc.h =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc.h (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc.h (revision 305172) @@ -1,99 +1,100 @@ /*- * Copyright (c) 2009 Kai Wang * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer * in this position and unchanged. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD: users/kaiwang27/elftc/libelftc.h 392 2009-05-31 19:17:46Z kaiwang27 $ - * $Id: libelftc.h 3418 2016-02-19 20:04:42Z emaste $ + * $Id: libelftc.h 3489 2016-08-31 00:12:15Z emaste $ */ #ifndef _LIBELFTC_H_ #define _LIBELFTC_H_ #include #include /* * Types meant to be opaque to the consumers of these APIs. */ typedef struct _Elftc_Bfd_Target Elftc_Bfd_Target; typedef struct _Elftc_String_Table Elftc_String_Table; /* Target types. */ typedef enum { ETF_NONE, ETF_ELF, ETF_BINARY, ETF_SREC, ETF_IHEX, ETF_PE, ETF_EFI, } Elftc_Bfd_Target_Flavor; /* * Demangler flags. */ /* Name mangling style. */ #define ELFTC_DEM_UNKNOWN 0x00000000U /* Not specified. */ #define ELFTC_DEM_ARM 0x00000001U /* C++ Ann. Ref. Manual. */ #define ELFTC_DEM_GNU2 0x00000002U /* GNU version 2. */ #define ELFTC_DEM_GNU3 0x00000004U /* GNU version 3. */ /* Demangling behaviour control. */ #define ELFTC_DEM_NOPARAM 0x00010000U #ifdef __cplusplus extern "C" { #endif Elftc_Bfd_Target *elftc_bfd_find_target(const char *_tgt_name); Elftc_Bfd_Target_Flavor elftc_bfd_target_flavor(Elftc_Bfd_Target *_tgt); unsigned int elftc_bfd_target_byteorder(Elftc_Bfd_Target *_tgt); unsigned int elftc_bfd_target_class(Elftc_Bfd_Target *_tgt); unsigned int elftc_bfd_target_machine(Elftc_Bfd_Target *_tgt); int elftc_copyfile(int _srcfd, int _dstfd); int elftc_demangle(const char *_mangledname, char *_buffer, size_t _bufsize, unsigned int _flags); const char *elftc_reloc_type_str(unsigned int mach, unsigned int type); int elftc_set_timestamps(const char *_filename, struct stat *_sb); Elftc_String_Table *elftc_string_table_create(int _hint); void elftc_string_table_destroy(Elftc_String_Table *_table); Elftc_String_Table *elftc_string_table_from_section(Elf_Scn *_scn, int _hint); const char *elftc_string_table_image(Elftc_String_Table *_table, size_t *_sz); size_t elftc_string_table_insert(Elftc_String_Table *_table, const char *_string); size_t elftc_string_table_lookup(Elftc_String_Table *_table, const char *_string); int elftc_string_table_remove(Elftc_String_Table *_table, const char *_string); const char *elftc_string_table_to_string(Elftc_String_Table *_table, size_t offset); +int elftc_timestamp(time_t *_timestamp); const char *elftc_version(void); #ifdef __cplusplus } #endif #endif /* _LIBELFTC_H_ */ Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc_bfdtarget.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc_bfdtarget.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc_bfdtarget.c (revision 305172) @@ -1,413 +1,413 @@ /*- * Copyright (c) 2008,2009 Kai Wang * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer * in this position and unchanged. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include #include #include #include "_libelftc.h" -ELFTC_VCSID("$Id: libelftc_bfdtarget.c 3309 2016-01-10 09:10:51Z kaiwang27 $"); +ELFTC_VCSID("$Id: libelftc_bfdtarget.c 3488 2016-08-24 18:15:57Z emaste $"); struct _Elftc_Bfd_Target _libelftc_targets[] = { { .bt_name = "binary", .bt_type = ETF_BINARY, }, { .bt_name = "elf32-avr", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_AVR, }, { .bt_name = "elf32-big", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, }, { .bt_name = "elf32-bigarm", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_ARM, }, { .bt_name = "elf32-bigmips", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_MIPS, }, { .bt_name = "elf32-i386", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_386, }, { .bt_name = "elf32-i386-freebsd", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_386, .bt_osabi = ELFOSABI_FREEBSD, }, { .bt_name = "elf32-ia64-big", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_IA_64, }, { .bt_name = "elf32-little", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, }, { .bt_name = "elf32-littlearm", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_ARM, }, { .bt_name = "elf32-littlemips", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_MIPS, }, { .bt_name = "elf32-powerpc", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_PPC, }, { .bt_name = "elf32-powerpcle", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_PPC, }, { .bt_name = "elf32-sh", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_SH, }, { .bt_name = "elf32-shl", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_SH, }, { .bt_name = "elf32-sh-nbsd", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_SH, .bt_osabi = ELFOSABI_NETBSD, }, { .bt_name = "elf32-shl-nbsd", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_SH, .bt_osabi = ELFOSABI_NETBSD, }, { .bt_name = "elf32-shbig-linux", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_SH, .bt_osabi = ELFOSABI_LINUX, }, { .bt_name = "elf32-sh-linux", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_SH, .bt_osabi = ELFOSABI_LINUX, }, { .bt_name = "elf32-sparc", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS32, .bt_machine = EM_SPARC, }, { .bt_name = "elf64-alpha", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_ALPHA, }, { .bt_name = "elf64-alpha-freebsd", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_ALPHA, .bt_osabi = ELFOSABI_FREEBSD }, { .bt_name = "elf64-big", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, }, { .bt_name = "elf64-bigmips", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_MIPS, }, { .bt_name = "elf64-ia64-big", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_IA_64, }, { .bt_name = "elf64-ia64-little", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_IA_64, }, { .bt_name = "elf64-little", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, }, { .bt_name = "elf64-littleaarch64", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_AARCH64, }, { .bt_name = "elf64-littlemips", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_MIPS, }, { .bt_name = "elf64-powerpc", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_PPC64, }, { .bt_name = "elf64-powerpcle", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_PPC64, }, { .bt_name = "elf64-sh64", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_SH, }, { .bt_name = "elf64-sh64l", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_SH, }, { .bt_name = "elf64-sh64-nbsd", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_SH, .bt_osabi = ELFOSABI_NETBSD, }, { .bt_name = "elf64-sh64l-nbsd", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_SH, .bt_osabi = ELFOSABI_NETBSD, }, { .bt_name = "elf64-sh64big-linux", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_SH, .bt_osabi = ELFOSABI_LINUX, }, { .bt_name = "elf64-sh64-linux", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_SH, .bt_osabi = ELFOSABI_LINUX, }, { .bt_name = "elf64-sparc", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_SPARCV9, }, { .bt_name = "elf64-sparc-freebsd", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2MSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_SPARCV9, .bt_osabi = ELFOSABI_FREEBSD }, { .bt_name = "elf64-x86-64", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_X86_64, }, { .bt_name = "elf64-x86-64-freebsd", .bt_type = ETF_ELF, .bt_byteorder = ELFDATA2LSB, .bt_elfclass = ELFCLASS64, .bt_machine = EM_X86_64, .bt_osabi = ELFOSABI_FREEBSD }, { .bt_name = "ihex", .bt_type = ETF_IHEX, }, { .bt_name = "srec", .bt_type = ETF_SREC, }, { .bt_name = "symbolsrec", .bt_type = ETF_SREC, }, { .bt_name = "efi-app-ia32", .bt_type = ETF_EFI, .bt_machine = EM_386, }, { .bt_name = "efi-app-x86_64", .bt_type = ETF_EFI, .bt_machine = EM_X86_64, }, { .bt_name = "pei-i386", .bt_type = ETF_PE, .bt_machine = EM_386, }, { .bt_name = "pei-x86-64", .bt_type = ETF_PE, .bt_machine = EM_X86_64, }, { .bt_name = NULL, .bt_type = ETF_NONE, }, }; Index: projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc_dem_gnu3.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc_dem_gnu3.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/libelftc/libelftc_dem_gnu3.c (revision 305172) @@ -1,3618 +1,3618 @@ /*- * Copyright (c) 2007 Hyogeol Lee * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer * in this position and unchanged. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include "_libelftc.h" -ELFTC_VCSID("$Id: libelftc_dem_gnu3.c 3447 2016-05-03 13:32:23Z emaste $"); +ELFTC_VCSID("$Id: libelftc_dem_gnu3.c 3480 2016-07-24 23:38:41Z emaste $"); /** * @file cpp_demangle.c * @brief Decode IA-64 C++ ABI style implementation. * * IA-64 standard ABI(Itanium C++ ABI) references. * * http://www.codesourcery.com/cxx-abi/abi.html#mangling \n * http://www.codesourcery.com/cxx-abi/abi-mangling.html */ enum type_qualifier { TYPE_PTR, TYPE_REF, TYPE_CMX, TYPE_IMG, TYPE_EXT, TYPE_RST, TYPE_VAT, TYPE_CST, TYPE_VEC }; struct vector_type_qualifier { size_t size, capacity; enum type_qualifier *q_container; struct vector_str ext_name; }; enum read_cmd { READ_FAIL, READ_NEST, READ_TMPL, READ_EXPR, READ_EXPL, READ_LOCAL, READ_TYPE, READ_FUNC, READ_PTRMEM }; struct vector_read_cmd { size_t size, capacity; enum read_cmd *r_container; }; struct cpp_demangle_data { struct vector_str output; /* output string vector */ struct vector_str output_tmp; struct vector_str subst; /* substitution string vector */ struct vector_str tmpl; struct vector_str class_type; struct vector_read_cmd cmd; bool paren; /* parenthesis opened */ bool pfirst; /* first element of parameter */ bool mem_rst; /* restrict member function */ bool mem_vat; /* volatile member function */ bool mem_cst; /* const member function */ int func_type; const char *cur; /* current mangled name ptr */ const char *last_sname; /* last source name */ int push_head; }; #define CPP_DEMANGLE_TRY_LIMIT 128 #define FLOAT_SPRINTF_TRY_LIMIT 5 #define FLOAT_QUADRUPLE_BYTES 16 #define FLOAT_EXTENED_BYTES 10 #define SIMPLE_HASH(x,y) (64 * x + y) static void cpp_demangle_data_dest(struct cpp_demangle_data *); static int cpp_demangle_data_init(struct cpp_demangle_data *, const char *); static int cpp_demangle_get_subst(struct cpp_demangle_data *, size_t); static int cpp_demangle_get_tmpl_param(struct cpp_demangle_data *, size_t); static int cpp_demangle_push_fp(struct cpp_demangle_data *, char *(*)(const char *, size_t)); static int cpp_demangle_push_str(struct cpp_demangle_data *, const char *, size_t); static int cpp_demangle_push_subst(struct cpp_demangle_data *, const char *, size_t); static int cpp_demangle_push_subst_v(struct cpp_demangle_data *, struct vector_str *); static int cpp_demangle_push_type_qualifier(struct cpp_demangle_data *, struct vector_type_qualifier *, const char *); static int cpp_demangle_read_array(struct cpp_demangle_data *); static int cpp_demangle_read_encoding(struct cpp_demangle_data *); static int cpp_demangle_read_expr_primary(struct cpp_demangle_data *); static int cpp_demangle_read_expression(struct cpp_demangle_data *); static int cpp_demangle_read_expression_flat(struct cpp_demangle_data *, char **); static int cpp_demangle_read_expression_binary(struct cpp_demangle_data *, const char *, size_t); static int cpp_demangle_read_expression_unary(struct cpp_demangle_data *, const char *, size_t); static int cpp_demangle_read_expression_trinary(struct cpp_demangle_data *, const char *, size_t, const char *, size_t); static int cpp_demangle_read_function(struct cpp_demangle_data *, int *, struct vector_type_qualifier *); static int cpp_demangle_local_source_name(struct cpp_demangle_data *ddata); static int cpp_demangle_read_local_name(struct cpp_demangle_data *); static int cpp_demangle_read_name(struct cpp_demangle_data *); static int cpp_demangle_read_name_flat(struct cpp_demangle_data *, char**); static int cpp_demangle_read_nested_name(struct cpp_demangle_data *); static int cpp_demangle_read_number(struct cpp_demangle_data *, long *); static int cpp_demangle_read_number_as_string(struct cpp_demangle_data *, char **); static int cpp_demangle_read_nv_offset(struct cpp_demangle_data *); static int cpp_demangle_read_offset(struct cpp_demangle_data *); static int cpp_demangle_read_offset_number(struct cpp_demangle_data *); static int cpp_demangle_read_pointer_to_member(struct cpp_demangle_data *); static int cpp_demangle_read_sname(struct cpp_demangle_data *); static int cpp_demangle_read_subst(struct cpp_demangle_data *); static int cpp_demangle_read_subst_std(struct cpp_demangle_data *); static int cpp_demangle_read_subst_stdtmpl(struct cpp_demangle_data *, const char *, size_t); static int cpp_demangle_read_tmpl_arg(struct cpp_demangle_data *); static int cpp_demangle_read_tmpl_args(struct cpp_demangle_data *); static int cpp_demangle_read_tmpl_param(struct cpp_demangle_data *); static int cpp_demangle_read_type(struct cpp_demangle_data *, int); static int cpp_demangle_read_type_flat(struct cpp_demangle_data *, char **); static int cpp_demangle_read_uqname(struct cpp_demangle_data *); static int cpp_demangle_read_v_offset(struct cpp_demangle_data *); static char *decode_fp_to_double(const char *, size_t); static char *decode_fp_to_float(const char *, size_t); static char *decode_fp_to_float128(const char *, size_t); static char *decode_fp_to_float80(const char *, size_t); static char *decode_fp_to_long_double(const char *, size_t); static int hex_to_dec(char); static void vector_read_cmd_dest(struct vector_read_cmd *); static int vector_read_cmd_find(struct vector_read_cmd *, enum read_cmd); static int vector_read_cmd_init(struct vector_read_cmd *); static int vector_read_cmd_pop(struct vector_read_cmd *); static int vector_read_cmd_push(struct vector_read_cmd *, enum read_cmd); static void vector_type_qualifier_dest(struct vector_type_qualifier *); static int vector_type_qualifier_init(struct vector_type_qualifier *); static int vector_type_qualifier_push(struct vector_type_qualifier *, enum type_qualifier); /** * @brief Decode the input string by IA-64 C++ ABI style. * * GNU GCC v3 use IA-64 standard ABI. * @return New allocated demangled string or NULL if failed. * @todo 1. Testing and more test case. 2. Code cleaning. */ char * cpp_demangle_gnu3(const char *org) { struct cpp_demangle_data ddata; ssize_t org_len; unsigned int limit; char *rtn; if (org == NULL || (org_len = strlen(org)) < 2) return (NULL); if (org_len > 11 && !strncmp(org, "_GLOBAL__I_", 11)) { if ((rtn = malloc(org_len + 19)) == NULL) return (NULL); snprintf(rtn, org_len + 19, "global constructors keyed to %s", org + 11); return (rtn); } if (org[0] != '_' || org[1] != 'Z') return (NULL); if (!cpp_demangle_data_init(&ddata, org + 2)) return (NULL); rtn = NULL; if (!cpp_demangle_read_encoding(&ddata)) goto clean; limit = 0; while (*ddata.cur != '\0') { /* * Breaking at some gcc info at tail. e.g) @@GLIBCXX_3.4 */ if (*ddata.cur == '@' && *(ddata.cur + 1) == '@') break; if (!cpp_demangle_read_type(&ddata, 1)) goto clean; if (limit++ > CPP_DEMANGLE_TRY_LIMIT) goto clean; } if (ddata.output.size == 0) goto clean; if (ddata.paren && !vector_str_push(&ddata.output, ")", 1)) goto clean; if (ddata.mem_vat && !vector_str_push(&ddata.output, " volatile", 9)) goto clean; if (ddata.mem_cst && !vector_str_push(&ddata.output, " const", 6)) goto clean; if (ddata.mem_rst && !vector_str_push(&ddata.output, " restrict", 9)) goto clean; rtn = vector_str_get_flat(&ddata.output, (size_t *) NULL); clean: cpp_demangle_data_dest(&ddata); return (rtn); } static void cpp_demangle_data_dest(struct cpp_demangle_data *d) { if (d == NULL) return; vector_read_cmd_dest(&d->cmd); vector_str_dest(&d->class_type); vector_str_dest(&d->tmpl); vector_str_dest(&d->subst); vector_str_dest(&d->output_tmp); vector_str_dest(&d->output); } static int cpp_demangle_data_init(struct cpp_demangle_data *d, const char *cur) { if (d == NULL || cur == NULL) return (0); if (!vector_str_init(&d->output)) return (0); if (!vector_str_init(&d->output_tmp)) goto clean1; if (!vector_str_init(&d->subst)) goto clean2; if (!vector_str_init(&d->tmpl)) goto clean3; if (!vector_str_init(&d->class_type)) goto clean4; if (!vector_read_cmd_init(&d->cmd)) goto clean5; assert(d->output.container != NULL); assert(d->output_tmp.container != NULL); assert(d->subst.container != NULL); assert(d->tmpl.container != NULL); assert(d->class_type.container != NULL); d->paren = false; d->pfirst = false; d->mem_rst = false; d->mem_vat = false; d->mem_cst = false; d->func_type = 0; d->cur = cur; d->last_sname = NULL; d->push_head = 0; return (1); clean5: vector_str_dest(&d->class_type); clean4: vector_str_dest(&d->tmpl); clean3: vector_str_dest(&d->subst); clean2: vector_str_dest(&d->output_tmp); clean1: vector_str_dest(&d->output); return (0); } static int cpp_demangle_push_fp(struct cpp_demangle_data *ddata, char *(*decoder)(const char *, size_t)) { size_t len; int rtn; const char *fp; char *f; if (ddata == NULL || decoder == NULL) return (0); fp = ddata->cur; while (*ddata->cur != 'E') ++ddata->cur; if ((f = decoder(fp, ddata->cur - fp)) == NULL) return (0); rtn = 0; if ((len = strlen(f)) > 0) rtn = cpp_demangle_push_str(ddata, f, len); free(f); ++ddata->cur; return (rtn); } static int cpp_demangle_push_str(struct cpp_demangle_data *ddata, const char *str, size_t len) { if (ddata == NULL || str == NULL || len == 0) return (0); if (ddata->push_head > 0) return (vector_str_push(&ddata->output_tmp, str, len)); return (vector_str_push(&ddata->output, str, len)); } static int cpp_demangle_push_subst(struct cpp_demangle_data *ddata, const char *str, size_t len) { if (ddata == NULL || str == NULL || len == 0) return (0); if (!vector_str_find(&ddata->subst, str, len)) return (vector_str_push(&ddata->subst, str, len)); return (1); } static int cpp_demangle_push_subst_v(struct cpp_demangle_data *ddata, struct vector_str *v) { size_t str_len; int rtn; char *str; if (ddata == NULL || v == NULL) return (0); if ((str = vector_str_get_flat(v, &str_len)) == NULL) return (0); rtn = cpp_demangle_push_subst(ddata, str, str_len); free(str); return (rtn); } static int cpp_demangle_push_type_qualifier(struct cpp_demangle_data *ddata, struct vector_type_qualifier *v, const char *type_str) { struct vector_str subst_v; size_t idx, e_idx, e_len; int rtn; char *buf; if (ddata == NULL || v == NULL) return (0); if ((idx = v->size) == 0) return (1); rtn = 0; if (type_str != NULL) { if (!vector_str_init(&subst_v)) return (0); if (!vector_str_push(&subst_v, type_str, strlen(type_str))) goto clean; } e_idx = 0; while (idx > 0) { switch (v->q_container[idx - 1]) { case TYPE_PTR: if (!cpp_demangle_push_str(ddata, "*", 1)) goto clean; if (type_str != NULL) { if (!vector_str_push(&subst_v, "*", 1)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &subst_v)) goto clean; } break; case TYPE_REF: if (!cpp_demangle_push_str(ddata, "&", 1)) goto clean; if (type_str != NULL) { if (!vector_str_push(&subst_v, "&", 1)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &subst_v)) goto clean; } break; case TYPE_CMX: if (!cpp_demangle_push_str(ddata, " complex", 8)) goto clean; if (type_str != NULL) { if (!vector_str_push(&subst_v, " complex", 8)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &subst_v)) goto clean; } break; case TYPE_IMG: if (!cpp_demangle_push_str(ddata, " imaginary", 10)) goto clean; if (type_str != NULL) { if (!vector_str_push(&subst_v, " imaginary", 10)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &subst_v)) goto clean; } break; case TYPE_EXT: if (v->ext_name.size == 0 || e_idx > v->ext_name.size - 1) goto clean; if ((e_len = strlen(v->ext_name.container[e_idx])) == 0) goto clean; if ((buf = malloc(e_len + 2)) == NULL) goto clean; snprintf(buf, e_len + 2, " %s", v->ext_name.container[e_idx]); if (!cpp_demangle_push_str(ddata, buf, e_len + 1)) { free(buf); goto clean; } if (type_str != NULL) { if (!vector_str_push(&subst_v, buf, e_len + 1)) { free(buf); goto clean; } if (!cpp_demangle_push_subst_v(ddata, &subst_v)) { free(buf); goto clean; } } free(buf); ++e_idx; break; case TYPE_RST: if (!cpp_demangle_push_str(ddata, " restrict", 9)) goto clean; if (type_str != NULL) { if (!vector_str_push(&subst_v, " restrict", 9)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &subst_v)) goto clean; } break; case TYPE_VAT: if (!cpp_demangle_push_str(ddata, " volatile", 9)) goto clean; if (type_str != NULL) { if (!vector_str_push(&subst_v, " volatile", 9)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &subst_v)) goto clean; } break; case TYPE_CST: if (!cpp_demangle_push_str(ddata, " const", 6)) goto clean; if (type_str != NULL) { if (!vector_str_push(&subst_v, " const", 6)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &subst_v)) goto clean; } break; case TYPE_VEC: if (v->ext_name.size == 0 || e_idx > v->ext_name.size - 1) goto clean; if ((e_len = strlen(v->ext_name.container[e_idx])) == 0) goto clean; if ((buf = malloc(e_len + 12)) == NULL) goto clean; snprintf(buf, e_len + 12, " __vector(%s)", v->ext_name.container[e_idx]); if (!cpp_demangle_push_str(ddata, buf, e_len + 11)) { free(buf); goto clean; } if (type_str != NULL) { if (!vector_str_push(&subst_v, buf, e_len + 11)) { free(buf); goto clean; } if (!cpp_demangle_push_subst_v(ddata, &subst_v)) { free(buf); goto clean; } } free(buf); ++e_idx; break; } --idx; } rtn = 1; clean: if (type_str != NULL) vector_str_dest(&subst_v); return (rtn); } static int cpp_demangle_get_subst(struct cpp_demangle_data *ddata, size_t idx) { size_t len; if (ddata == NULL || ddata->subst.size <= idx) return (0); if ((len = strlen(ddata->subst.container[idx])) == 0) return (0); if (!cpp_demangle_push_str(ddata, ddata->subst.container[idx], len)) return (0); /* skip '_' */ ++ddata->cur; return (1); } static int cpp_demangle_get_tmpl_param(struct cpp_demangle_data *ddata, size_t idx) { size_t len; if (ddata == NULL || ddata->tmpl.size <= idx) return (0); if ((len = strlen(ddata->tmpl.container[idx])) == 0) return (0); if (!cpp_demangle_push_str(ddata, ddata->tmpl.container[idx], len)) return (0); ++ddata->cur; return (1); } static int cpp_demangle_read_array(struct cpp_demangle_data *ddata) { size_t i, num_len, exp_len, p_idx, idx; const char *num; char *exp; if (ddata == NULL || *(++ddata->cur) == '\0') return (0); if (*ddata->cur == '_') { if (*(++ddata->cur) == '\0') return (0); if (!cpp_demangle_read_type(ddata, 0)) return (0); if (!cpp_demangle_push_str(ddata, "[]", 2)) return (0); } else { if (ELFTC_ISDIGIT(*ddata->cur) != 0) { num = ddata->cur; while (ELFTC_ISDIGIT(*ddata->cur) != 0) ++ddata->cur; if (*ddata->cur != '_') return (0); num_len = ddata->cur - num; assert(num_len > 0); if (*(++ddata->cur) == '\0') return (0); if (!cpp_demangle_read_type(ddata, 0)) return (0); if (!cpp_demangle_push_str(ddata, "[", 1)) return (0); if (!cpp_demangle_push_str(ddata, num, num_len)) return (0); if (!cpp_demangle_push_str(ddata, "]", 1)) return (0); } else { p_idx = ddata->output.size; if (!cpp_demangle_read_expression(ddata)) return (0); if ((exp = vector_str_substr(&ddata->output, p_idx, ddata->output.size - 1, &exp_len)) == NULL) return (0); idx = ddata->output.size; for (i = p_idx; i < idx; ++i) if (!vector_str_pop(&ddata->output)) { free(exp); return (0); } if (*ddata->cur != '_') { free(exp); return (0); } ++ddata->cur; if (*ddata->cur == '\0') { free(exp); return (0); } if (!cpp_demangle_read_type(ddata, 0)) { free(exp); return (0); } if (!cpp_demangle_push_str(ddata, "[", 1)) { free(exp); return (0); } if (!cpp_demangle_push_str(ddata, exp, exp_len)) { free(exp); return (0); } if (!cpp_demangle_push_str(ddata, "]", 1)) { free(exp); return (0); } free(exp); } } return (1); } static int cpp_demangle_read_expr_primary(struct cpp_demangle_data *ddata) { const char *num; if (ddata == NULL || *(++ddata->cur) == '\0') return (0); if (*ddata->cur == '_' && *(ddata->cur + 1) == 'Z') { ddata->cur += 2; if (*ddata->cur == '\0') return (0); if (!cpp_demangle_read_encoding(ddata)) return (0); ++ddata->cur; return (1); } switch (*ddata->cur) { case 'b': if (*(ddata->cur + 2) != 'E') return (0); switch (*(++ddata->cur)) { case '0': ddata->cur += 2; return (cpp_demangle_push_str(ddata, "false", 5)); case '1': ddata->cur += 2; return (cpp_demangle_push_str(ddata, "true", 4)); default: return (0); } case 'd': ++ddata->cur; return (cpp_demangle_push_fp(ddata, decode_fp_to_double)); case 'e': ++ddata->cur; if (sizeof(long double) == 10) return (cpp_demangle_push_fp(ddata, decode_fp_to_double)); return (cpp_demangle_push_fp(ddata, decode_fp_to_float80)); case 'f': ++ddata->cur; return (cpp_demangle_push_fp(ddata, decode_fp_to_float)); case 'g': ++ddata->cur; if (sizeof(long double) == 16) return (cpp_demangle_push_fp(ddata, decode_fp_to_double)); return (cpp_demangle_push_fp(ddata, decode_fp_to_float128)); case 'i': case 'j': case 'l': case 'm': case 'n': case 's': case 't': case 'x': case 'y': if (*(++ddata->cur) == 'n') { if (!cpp_demangle_push_str(ddata, "-", 1)) return (0); ++ddata->cur; } num = ddata->cur; while (*ddata->cur != 'E') { if (!ELFTC_ISDIGIT(*ddata->cur)) return (0); ++ddata->cur; } ++ddata->cur; return (cpp_demangle_push_str(ddata, num, ddata->cur - num - 1)); default: return (0); } } static int cpp_demangle_read_expression(struct cpp_demangle_data *ddata) { if (ddata == NULL || *ddata->cur == '\0') return (0); switch (SIMPLE_HASH(*ddata->cur, *(ddata->cur + 1))) { case SIMPLE_HASH('s', 't'): ddata->cur += 2; return (cpp_demangle_read_type(ddata, 0)); case SIMPLE_HASH('s', 'r'): ddata->cur += 2; if (!cpp_demangle_read_type(ddata, 0)) return (0); if (!cpp_demangle_read_uqname(ddata)) return (0); if (*ddata->cur == 'I') return (cpp_demangle_read_tmpl_args(ddata)); return (1); case SIMPLE_HASH('a', 'a'): /* operator && */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "&&", 2)); case SIMPLE_HASH('a', 'd'): /* operator & (unary) */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "&", 1)); case SIMPLE_HASH('a', 'n'): /* operator & */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "&", 1)); case SIMPLE_HASH('a', 'N'): /* operator &= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "&=", 2)); case SIMPLE_HASH('a', 'S'): /* operator = */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "=", 1)); case SIMPLE_HASH('c', 'l'): /* operator () */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "()", 2)); case SIMPLE_HASH('c', 'm'): /* operator , */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, ",", 1)); case SIMPLE_HASH('c', 'o'): /* operator ~ */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "~", 1)); case SIMPLE_HASH('c', 'v'): /* operator (cast) */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "(cast)", 6)); case SIMPLE_HASH('d', 'a'): /* operator delete [] */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "delete []", 9)); case SIMPLE_HASH('d', 'e'): /* operator * (unary) */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "*", 1)); case SIMPLE_HASH('d', 'l'): /* operator delete */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "delete", 6)); case SIMPLE_HASH('d', 'v'): /* operator / */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "/", 1)); case SIMPLE_HASH('d', 'V'): /* operator /= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "/=", 2)); case SIMPLE_HASH('e', 'o'): /* operator ^ */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "^", 1)); case SIMPLE_HASH('e', 'O'): /* operator ^= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "^=", 2)); case SIMPLE_HASH('e', 'q'): /* operator == */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "==", 2)); case SIMPLE_HASH('g', 'e'): /* operator >= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, ">=", 2)); case SIMPLE_HASH('g', 't'): /* operator > */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, ">", 1)); case SIMPLE_HASH('i', 'x'): /* operator [] */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "[]", 2)); case SIMPLE_HASH('l', 'e'): /* operator <= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "<=", 2)); case SIMPLE_HASH('l', 's'): /* operator << */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "<<", 2)); case SIMPLE_HASH('l', 'S'): /* operator <<= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "<<=", 3)); case SIMPLE_HASH('l', 't'): /* operator < */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "<", 1)); case SIMPLE_HASH('m', 'i'): /* operator - */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "-", 1)); case SIMPLE_HASH('m', 'I'): /* operator -= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "-=", 2)); case SIMPLE_HASH('m', 'l'): /* operator * */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "*", 1)); case SIMPLE_HASH('m', 'L'): /* operator *= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "*=", 2)); case SIMPLE_HASH('m', 'm'): /* operator -- */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "--", 2)); case SIMPLE_HASH('n', 'a'): /* operator new[] */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "new []", 6)); case SIMPLE_HASH('n', 'e'): /* operator != */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "!=", 2)); case SIMPLE_HASH('n', 'g'): /* operator - (unary) */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "-", 1)); case SIMPLE_HASH('n', 't'): /* operator ! */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "!", 1)); case SIMPLE_HASH('n', 'w'): /* operator new */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "new", 3)); case SIMPLE_HASH('o', 'o'): /* operator || */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "||", 2)); case SIMPLE_HASH('o', 'r'): /* operator | */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "|", 1)); case SIMPLE_HASH('o', 'R'): /* operator |= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "|=", 2)); case SIMPLE_HASH('p', 'l'): /* operator + */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "+", 1)); case SIMPLE_HASH('p', 'L'): /* operator += */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "+=", 2)); case SIMPLE_HASH('p', 'm'): /* operator ->* */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "->*", 3)); case SIMPLE_HASH('p', 'p'): /* operator ++ */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "++", 2)); case SIMPLE_HASH('p', 's'): /* operator + (unary) */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "+", 1)); case SIMPLE_HASH('p', 't'): /* operator -> */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "->", 2)); case SIMPLE_HASH('q', 'u'): /* operator ? */ ddata->cur += 2; return (cpp_demangle_read_expression_trinary(ddata, "?", 1, ":", 1)); case SIMPLE_HASH('r', 'm'): /* operator % */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "%", 1)); case SIMPLE_HASH('r', 'M'): /* operator %= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, "%=", 2)); case SIMPLE_HASH('r', 's'): /* operator >> */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, ">>", 2)); case SIMPLE_HASH('r', 'S'): /* operator >>= */ ddata->cur += 2; return (cpp_demangle_read_expression_binary(ddata, ">>=", 3)); case SIMPLE_HASH('r', 'z'): /* operator sizeof */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "sizeof", 6)); case SIMPLE_HASH('s', 'v'): /* operator sizeof */ ddata->cur += 2; return (cpp_demangle_read_expression_unary(ddata, "sizeof", 6)); } switch (*ddata->cur) { case 'L': return (cpp_demangle_read_expr_primary(ddata)); case 'T': return (cpp_demangle_read_tmpl_param(ddata)); } return (0); } static int cpp_demangle_read_expression_flat(struct cpp_demangle_data *ddata, char **str) { struct vector_str *output; size_t i, p_idx, idx, exp_len; char *exp; output = ddata->push_head > 0 ? &ddata->output_tmp : &ddata->output; p_idx = output->size; if (!cpp_demangle_read_expression(ddata)) return (0); if ((exp = vector_str_substr(output, p_idx, output->size - 1, &exp_len)) == NULL) return (0); idx = output->size; for (i = p_idx; i < idx; ++i) { if (!vector_str_pop(output)) { free(exp); return (0); } } *str = exp; return (1); } static int cpp_demangle_read_expression_binary(struct cpp_demangle_data *ddata, const char *name, size_t len) { if (ddata == NULL || name == NULL || len == 0) return (0); if (!cpp_demangle_read_expression(ddata)) return (0); if (!cpp_demangle_push_str(ddata, name, len)) return (0); return (cpp_demangle_read_expression(ddata)); } static int cpp_demangle_read_expression_unary(struct cpp_demangle_data *ddata, const char *name, size_t len) { if (ddata == NULL || name == NULL || len == 0) return (0); if (!cpp_demangle_read_expression(ddata)) return (0); return (cpp_demangle_push_str(ddata, name, len)); } static int cpp_demangle_read_expression_trinary(struct cpp_demangle_data *ddata, const char *name1, size_t len1, const char *name2, size_t len2) { if (ddata == NULL || name1 == NULL || len1 == 0 || name2 == NULL || len2 == 0) return (0); if (!cpp_demangle_read_expression(ddata)) return (0); if (!cpp_demangle_push_str(ddata, name1, len1)) return (0); if (!cpp_demangle_read_expression(ddata)) return (0); if (!cpp_demangle_push_str(ddata, name2, len2)) return (0); return (cpp_demangle_read_expression(ddata)); } static int cpp_demangle_read_function(struct cpp_demangle_data *ddata, int *ext_c, struct vector_type_qualifier *v) { size_t class_type_size, class_type_len, limit; const char *class_type; if (ddata == NULL || *ddata->cur != 'F' || v == NULL) return (0); ++ddata->cur; if (*ddata->cur == 'Y') { if (ext_c != NULL) *ext_c = 1; ++ddata->cur; } if (!cpp_demangle_read_type(ddata, 0)) return (0); if (*ddata->cur != 'E') { if (!cpp_demangle_push_str(ddata, "(", 1)) return (0); if (vector_read_cmd_find(&ddata->cmd, READ_PTRMEM)) { if ((class_type_size = ddata->class_type.size) == 0) return (0); class_type = ddata->class_type.container[class_type_size - 1]; if (class_type == NULL) return (0); if ((class_type_len = strlen(class_type)) == 0) return (0); if (!cpp_demangle_push_str(ddata, class_type, class_type_len)) return (0); if (!cpp_demangle_push_str(ddata, "::*", 3)) return (0); ++ddata->func_type; } else { if (!cpp_demangle_push_type_qualifier(ddata, v, (const char *) NULL)) return (0); vector_type_qualifier_dest(v); if (!vector_type_qualifier_init(v)) return (0); } if (!cpp_demangle_push_str(ddata, ")(", 2)) return (0); limit = 0; for (;;) { if (!cpp_demangle_read_type(ddata, 0)) return (0); if (*ddata->cur == 'E') break; if (limit++ > CPP_DEMANGLE_TRY_LIMIT) return (0); } if (vector_read_cmd_find(&ddata->cmd, READ_PTRMEM) == 1) { if (!cpp_demangle_push_type_qualifier(ddata, v, (const char *) NULL)) return (0); vector_type_qualifier_dest(v); if (!vector_type_qualifier_init(v)) return (0); } if (!cpp_demangle_push_str(ddata, ")", 1)) return (0); } ++ddata->cur; return (1); } /* read encoding, encoding are function name, data name, special-name */ static int cpp_demangle_read_encoding(struct cpp_demangle_data *ddata) { char *name, *type, *num_str; long offset; int rtn; if (ddata == NULL || *ddata->cur == '\0') return (0); /* special name */ switch (SIMPLE_HASH(*ddata->cur, *(ddata->cur + 1))) { case SIMPLE_HASH('G', 'A'): if (!cpp_demangle_push_str(ddata, "hidden alias for ", 17)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); return (cpp_demangle_read_encoding(ddata)); case SIMPLE_HASH('G', 'R'): if (!cpp_demangle_push_str(ddata, "reference temporary #", 21)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); if (!cpp_demangle_read_name_flat(ddata, &name)) return (0); rtn = 0; if (!cpp_demangle_read_number_as_string(ddata, &num_str)) goto clean1; if (!cpp_demangle_push_str(ddata, num_str, strlen(num_str))) goto clean2; if (!cpp_demangle_push_str(ddata, " for ", 5)) goto clean2; if (!cpp_demangle_push_str(ddata, name, strlen(name))) goto clean2; rtn = 1; clean2: free(num_str); clean1: free(name); return (rtn); case SIMPLE_HASH('G', 'T'): ddata->cur += 2; if (*ddata->cur == '\0') return (0); switch (*ddata->cur) { case 'n': if (!cpp_demangle_push_str(ddata, "non-transaction clone for ", 26)) return (0); break; case 't': default: if (!cpp_demangle_push_str(ddata, "transaction clone for ", 22)) return (0); break; } ++ddata->cur; return (cpp_demangle_read_encoding(ddata)); case SIMPLE_HASH('G', 'V'): /* sentry object for 1 time init */ if (!cpp_demangle_push_str(ddata, "guard variable for ", 20)) return (0); ddata->cur += 2; break; case SIMPLE_HASH('T', 'c'): /* virtual function covariant override thunk */ if (!cpp_demangle_push_str(ddata, "virtual function covariant override ", 36)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); if (!cpp_demangle_read_offset(ddata)) return (0); if (!cpp_demangle_read_offset(ddata)) return (0); return (cpp_demangle_read_encoding(ddata)); case SIMPLE_HASH('T', 'C'): /* construction vtable */ if (!cpp_demangle_push_str(ddata, "construction vtable for ", 24)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); if (!cpp_demangle_read_type_flat(ddata, &type)) return (0); rtn = 0; if (!cpp_demangle_read_number(ddata, &offset)) goto clean3; if (*ddata->cur++ != '_') goto clean3; if (!cpp_demangle_read_type(ddata, 0)) goto clean3; if (!cpp_demangle_push_str(ddata, "-in-", 4)) goto clean3; if (!cpp_demangle_push_str(ddata, type, strlen(type))) goto clean3; rtn = 1; clean3: free(type); return (rtn); case SIMPLE_HASH('T', 'D'): /* typeinfo common proxy */ break; case SIMPLE_HASH('T', 'F'): /* typeinfo fn */ if (!cpp_demangle_push_str(ddata, "typeinfo fn for ", 16)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); return (cpp_demangle_read_type(ddata, 0)); case SIMPLE_HASH('T', 'h'): /* virtual function non-virtual override thunk */ if (!cpp_demangle_push_str(ddata, "virtual function non-virtual override ", 38)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); if (!cpp_demangle_read_nv_offset(ddata)) return (0); return (cpp_demangle_read_encoding(ddata)); case SIMPLE_HASH('T', 'H'): /* TLS init function */ if (!cpp_demangle_push_str(ddata, "TLS init function for ", 22)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); break; case SIMPLE_HASH('T', 'I'): /* typeinfo structure */ if (!cpp_demangle_push_str(ddata, "typeinfo for ", 13)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); return (cpp_demangle_read_type(ddata, 0)); case SIMPLE_HASH('T', 'J'): /* java class */ if (!cpp_demangle_push_str(ddata, "java Class for ", 15)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); return (cpp_demangle_read_type(ddata, 0)); case SIMPLE_HASH('T', 'S'): /* RTTI name (NTBS) */ if (!cpp_demangle_push_str(ddata, "typeinfo name for ", 18)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); return (cpp_demangle_read_type(ddata, 0)); case SIMPLE_HASH('T', 'T'): /* VTT table */ if (!cpp_demangle_push_str(ddata, "VTT for ", 8)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); return (cpp_demangle_read_type(ddata, 0)); case SIMPLE_HASH('T', 'v'): /* virtual function virtual override thunk */ if (!cpp_demangle_push_str(ddata, "virtual function virtual override ", 34)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); if (!cpp_demangle_read_v_offset(ddata)) return (0); return (cpp_demangle_read_encoding(ddata)); case SIMPLE_HASH('T', 'V'): /* virtual table */ if (!cpp_demangle_push_str(ddata, "vtable for ", 12)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); return (cpp_demangle_read_type(ddata, 0)); case SIMPLE_HASH('T', 'W'): /* TLS wrapper function */ if (!cpp_demangle_push_str(ddata, "TLS wrapper function for ", 25)) return (0); ddata->cur += 2; if (*ddata->cur == '\0') return (0); break; } return (cpp_demangle_read_name(ddata)); } static int cpp_demangle_read_local_name(struct cpp_demangle_data *ddata) { size_t limit; if (ddata == NULL) return (0); if (*(++ddata->cur) == '\0') return (0); if (!cpp_demangle_read_encoding(ddata)) return (0); limit = 0; for (;;) { if (!cpp_demangle_read_type(ddata, 1)) return (0); if (*ddata->cur == 'E') break; if (limit++ > CPP_DEMANGLE_TRY_LIMIT) return (0); } if (*(++ddata->cur) == '\0') return (0); if (ddata->paren == true) { if (!cpp_demangle_push_str(ddata, ")", 1)) return (0); ddata->paren = false; } if (*ddata->cur == 's') ++ddata->cur; else { if (!cpp_demangle_push_str(ddata, "::", 2)) return (0); if (!cpp_demangle_read_name(ddata)) return (0); } if (*ddata->cur == '_') { ++ddata->cur; while (ELFTC_ISDIGIT(*ddata->cur) != 0) ++ddata->cur; } return (1); } static int cpp_demangle_read_name(struct cpp_demangle_data *ddata) { struct vector_str *output, v; size_t p_idx, subst_str_len; int rtn; char *subst_str; if (ddata == NULL || *ddata->cur == '\0') return (0); output = ddata->push_head > 0 ? &ddata->output_tmp : &ddata->output; subst_str = NULL; switch (*ddata->cur) { case 'S': return (cpp_demangle_read_subst(ddata)); case 'N': return (cpp_demangle_read_nested_name(ddata)); case 'Z': return (cpp_demangle_read_local_name(ddata)); } if (!vector_str_init(&v)) return (0); p_idx = output->size; rtn = 0; if (!cpp_demangle_read_uqname(ddata)) goto clean; if ((subst_str = vector_str_substr(output, p_idx, output->size - 1, &subst_str_len)) == NULL) goto clean; if (subst_str_len > 8 && strstr(subst_str, "operator") != NULL) { rtn = 1; goto clean; } if (!vector_str_push(&v, subst_str, subst_str_len)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &v)) goto clean; if (*ddata->cur == 'I') { p_idx = output->size; if (!cpp_demangle_read_tmpl_args(ddata)) goto clean; free(subst_str); if ((subst_str = vector_str_substr(output, p_idx, output->size - 1, &subst_str_len)) == NULL) goto clean; if (!vector_str_push(&v, subst_str, subst_str_len)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &v)) goto clean; } rtn = 1; clean: free(subst_str); vector_str_dest(&v); return (rtn); } static int cpp_demangle_read_name_flat(struct cpp_demangle_data *ddata, char **str) { struct vector_str *output; size_t i, p_idx, idx, name_len; char *name; output = ddata->push_head > 0 ? &ddata->output_tmp : &ddata->output; p_idx = output->size; if (!cpp_demangle_read_name(ddata)) return (0); if ((name = vector_str_substr(output, p_idx, output->size - 1, &name_len)) == NULL) return (0); idx = output->size; for (i = p_idx; i < idx; ++i) { if (!vector_str_pop(output)) { free(name); return (0); } } *str = name; return (1); } static int cpp_demangle_read_nested_name(struct cpp_demangle_data *ddata) { struct vector_str *output, v; size_t limit, p_idx, subst_str_len; int rtn; char *subst_str; if (ddata == NULL || *ddata->cur != 'N') return (0); if (*(++ddata->cur) == '\0') return (0); while (*ddata->cur == 'r' || *ddata->cur == 'V' || *ddata->cur == 'K') { switch (*ddata->cur) { case 'r': ddata->mem_rst = true; break; case 'V': ddata->mem_vat = true; break; case 'K': ddata->mem_cst = true; break; } ++ddata->cur; } output = ddata->push_head > 0 ? &ddata->output_tmp : &ddata->output; if (!vector_str_init(&v)) return (0); rtn = 0; limit = 0; for (;;) { p_idx = output->size; switch (*ddata->cur) { case 'I': if (!cpp_demangle_read_tmpl_args(ddata)) goto clean; break; case 'S': if (!cpp_demangle_read_subst(ddata)) goto clean; break; case 'T': if (!cpp_demangle_read_tmpl_param(ddata)) goto clean; break; default: if (!cpp_demangle_read_uqname(ddata)) goto clean; } if ((subst_str = vector_str_substr(output, p_idx, output->size - 1, &subst_str_len)) == NULL) goto clean; if (!vector_str_push(&v, subst_str, subst_str_len)) { free(subst_str); goto clean; } free(subst_str); if (!cpp_demangle_push_subst_v(ddata, &v)) goto clean; if (*ddata->cur == 'E') break; else if (*ddata->cur != 'I' && *ddata->cur != 'C' && *ddata->cur != 'D') { if (!cpp_demangle_push_str(ddata, "::", 2)) goto clean; if (!vector_str_push(&v, "::", 2)) goto clean; } if (limit++ > CPP_DEMANGLE_TRY_LIMIT) goto clean; } ++ddata->cur; rtn = 1; clean: vector_str_dest(&v); return (rtn); } /* * read number * number ::= [n] */ static int cpp_demangle_read_number(struct cpp_demangle_data *ddata, long *rtn) { long len, negative_factor; if (ddata == NULL || rtn == NULL) return (0); negative_factor = 1; if (*ddata->cur == 'n') { negative_factor = -1; ++ddata->cur; } if (ELFTC_ISDIGIT(*ddata->cur) == 0) return (0); errno = 0; if ((len = strtol(ddata->cur, (char **) NULL, 10)) == 0 && errno != 0) return (0); while (ELFTC_ISDIGIT(*ddata->cur) != 0) ++ddata->cur; assert(len >= 0); assert(negative_factor == 1 || negative_factor == -1); *rtn = len * negative_factor; return (1); } static int cpp_demangle_read_number_as_string(struct cpp_demangle_data *ddata, char **str) { long n; if (!cpp_demangle_read_number(ddata, &n)) { *str = NULL; return (0); } if (asprintf(str, "%ld", n) < 0) { *str = NULL; return (0); } return (1); } static int cpp_demangle_read_nv_offset(struct cpp_demangle_data *ddata) { if (ddata == NULL) return (0); if (!cpp_demangle_push_str(ddata, "offset : ", 9)) return (0); return (cpp_demangle_read_offset_number(ddata)); } /* read offset, offset are nv-offset, v-offset */ static int cpp_demangle_read_offset(struct cpp_demangle_data *ddata) { if (ddata == NULL) return (0); if (*ddata->cur == 'h') { ++ddata->cur; return (cpp_demangle_read_nv_offset(ddata)); } else if (*ddata->cur == 'v') { ++ddata->cur; return (cpp_demangle_read_v_offset(ddata)); } return (0); } static int cpp_demangle_read_offset_number(struct cpp_demangle_data *ddata) { bool negative; const char *start; if (ddata == NULL || *ddata->cur == '\0') return (0); /* offset could be negative */ if (*ddata->cur == 'n') { negative = true; start = ddata->cur + 1; } else { negative = false; start = ddata->cur; } while (*ddata->cur != '_') ++ddata->cur; if (negative && !cpp_demangle_push_str(ddata, "-", 1)) return (0); assert(start != NULL); if (!cpp_demangle_push_str(ddata, start, ddata->cur - start)) return (0); if (!cpp_demangle_push_str(ddata, " ", 1)) return (0); ++ddata->cur; return (1); } static int cpp_demangle_read_pointer_to_member(struct cpp_demangle_data *ddata) { size_t class_type_len, i, idx, p_idx; int p_func_type, rtn; char *class_type; if (ddata == NULL || *ddata->cur != 'M' || *(++ddata->cur) == '\0') return (0); p_idx = ddata->output.size; if (!cpp_demangle_read_type(ddata, 0)) return (0); if ((class_type = vector_str_substr(&ddata->output, p_idx, ddata->output.size - 1, &class_type_len)) == NULL) return (0); rtn = 0; idx = ddata->output.size; for (i = p_idx; i < idx; ++i) if (!vector_str_pop(&ddata->output)) goto clean1; if (!vector_read_cmd_push(&ddata->cmd, READ_PTRMEM)) goto clean1; if (!vector_str_push(&ddata->class_type, class_type, class_type_len)) goto clean2; p_func_type = ddata->func_type; if (!cpp_demangle_read_type(ddata, 0)) goto clean3; if (p_func_type == ddata->func_type) { if (!cpp_demangle_push_str(ddata, " ", 1)) goto clean3; if (!cpp_demangle_push_str(ddata, class_type, class_type_len)) goto clean3; if (!cpp_demangle_push_str(ddata, "::*", 3)) goto clean3; } rtn = 1; clean3: if (!vector_str_pop(&ddata->class_type)) rtn = 0; clean2: if (!vector_read_cmd_pop(&ddata->cmd)) rtn = 0; clean1: free(class_type); return (rtn); } /* read source-name, source-name is */ static int cpp_demangle_read_sname(struct cpp_demangle_data *ddata) { long len; int err; if (ddata == NULL || cpp_demangle_read_number(ddata, &len) == 0 || len <= 0) return (0); if (len == 12 && (memcmp("_GLOBAL__N_1", ddata->cur, 12) == 0)) err = cpp_demangle_push_str(ddata, "(anonymous namespace)", 21); else err = cpp_demangle_push_str(ddata, ddata->cur, len); if (err == 0) return (0); assert(ddata->output.size > 0); if (vector_read_cmd_find(&ddata->cmd, READ_TMPL) == 0) ddata->last_sname = ddata->output.container[ddata->output.size - 1]; ddata->cur += len; return (1); } static int cpp_demangle_read_subst(struct cpp_demangle_data *ddata) { long nth; if (ddata == NULL || *ddata->cur == '\0') return (0); /* abbreviations of the form Sx */ switch (SIMPLE_HASH(*ddata->cur, *(ddata->cur + 1))) { case SIMPLE_HASH('S', 'a'): /* std::allocator */ if (cpp_demangle_push_str(ddata, "std::allocator", 14) == 0) return (0); ddata->cur += 2; if (*ddata->cur == 'I') return (cpp_demangle_read_subst_stdtmpl(ddata, "std::allocator", 14)); return (1); case SIMPLE_HASH('S', 'b'): /* std::basic_string */ if (!cpp_demangle_push_str(ddata, "std::basic_string", 17)) return (0); ddata->cur += 2; if (*ddata->cur == 'I') return (cpp_demangle_read_subst_stdtmpl(ddata, "std::basic_string", 17)); return (1); case SIMPLE_HASH('S', 'd'): /* std::basic_iostream > */ if (!cpp_demangle_push_str(ddata, "std::basic_iostream", 19)) return (0); ddata->last_sname = "basic_iostream"; ddata->cur += 2; if (*ddata->cur == 'I') return (cpp_demangle_read_subst_stdtmpl(ddata, "std::basic_iostream", 19)); return (1); case SIMPLE_HASH('S', 'i'): /* std::basic_istream > */ if (!cpp_demangle_push_str(ddata, "std::basic_istream", 18)) return (0); ddata->last_sname = "basic_istream"; ddata->cur += 2; if (*ddata->cur == 'I') return (cpp_demangle_read_subst_stdtmpl(ddata, "std::basic_istream", 18)); return (1); case SIMPLE_HASH('S', 'o'): /* std::basic_ostream > */ if (!cpp_demangle_push_str(ddata, "std::basic_ostream", 18)) return (0); ddata->last_sname = "basic_ostream"; ddata->cur += 2; if (*ddata->cur == 'I') return (cpp_demangle_read_subst_stdtmpl(ddata, "std::basic_ostream", 18)); return (1); case SIMPLE_HASH('S', 's'): /* * std::basic_string, * std::allocator > * * a.k.a std::string */ if (!cpp_demangle_push_str(ddata, "std::string", 11)) return (0); ddata->last_sname = "string"; ddata->cur += 2; if (*ddata->cur == 'I') return (cpp_demangle_read_subst_stdtmpl(ddata, "std::string", 11)); return (1); case SIMPLE_HASH('S', 't'): /* std:: */ return (cpp_demangle_read_subst_std(ddata)); } if (*(++ddata->cur) == '\0') return (0); /* substitution */ if (*ddata->cur == '_') return (cpp_demangle_get_subst(ddata, 0)); else { errno = 0; /* substitution number is base 36 */ if ((nth = strtol(ddata->cur, (char **) NULL, 36)) == 0 && errno != 0) return (0); /* first was '_', so increase one */ ++nth; while (*ddata->cur != '_') ++ddata->cur; assert(nth > 0); return (cpp_demangle_get_subst(ddata, nth)); } /* NOTREACHED */ return (0); } static int cpp_demangle_read_subst_std(struct cpp_demangle_data *ddata) { struct vector_str *output, v; size_t p_idx, subst_str_len; int rtn; char *subst_str; if (ddata == NULL) return (0); if (!vector_str_init(&v)) return (0); subst_str = NULL; rtn = 0; if (!cpp_demangle_push_str(ddata, "std::", 5)) goto clean; if (!vector_str_push(&v, "std::", 5)) goto clean; ddata->cur += 2; output = ddata->push_head > 0 ? &ddata->output_tmp : &ddata->output; p_idx = output->size; if (!cpp_demangle_read_uqname(ddata)) goto clean; if ((subst_str = vector_str_substr(output, p_idx, output->size - 1, &subst_str_len)) == NULL) goto clean; if (!vector_str_push(&v, subst_str, subst_str_len)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &v)) goto clean; if (*ddata->cur == 'I') { p_idx = output->size; if (!cpp_demangle_read_tmpl_args(ddata)) goto clean; free(subst_str); if ((subst_str = vector_str_substr(output, p_idx, output->size - 1, &subst_str_len)) == NULL) goto clean; if (!vector_str_push(&v, subst_str, subst_str_len)) goto clean; if (!cpp_demangle_push_subst_v(ddata, &v)) goto clean; } rtn = 1; clean: free(subst_str); vector_str_dest(&v); return (rtn); } static int cpp_demangle_read_subst_stdtmpl(struct cpp_demangle_data *ddata, const char *str, size_t len) { struct vector_str *output; size_t p_idx, substr_len; int rtn; char *subst_str, *substr; if (ddata == NULL || str == NULL || len == 0) return (0); output = ddata->push_head > 0 ? &ddata->output_tmp : &ddata->output; p_idx = output->size; substr = NULL; subst_str = NULL; if (!cpp_demangle_read_tmpl_args(ddata)) return (0); if ((substr = vector_str_substr(output, p_idx, output->size - 1, &substr_len)) == NULL) return (0); rtn = 0; if ((subst_str = malloc(sizeof(char) * (substr_len + len + 1))) == NULL) goto clean; memcpy(subst_str, str, len); memcpy(subst_str + len, substr, substr_len); subst_str[substr_len + len] = '\0'; if (!cpp_demangle_push_subst(ddata, subst_str, substr_len + len)) goto clean; rtn = 1; clean: free(subst_str); free(substr); return (rtn); } static int cpp_demangle_read_tmpl_arg(struct cpp_demangle_data *ddata) { if (ddata == NULL || *ddata->cur == '\0') return (0); switch (*ddata->cur) { case 'L': return (cpp_demangle_read_expr_primary(ddata)); case 'X': return (cpp_demangle_read_expression(ddata)); } return (cpp_demangle_read_type(ddata, 0)); } static int cpp_demangle_read_tmpl_args(struct cpp_demangle_data *ddata) { struct vector_str *v; size_t arg_len, idx, limit, size; char *arg; if (ddata == NULL || *ddata->cur == '\0') return (0); ++ddata->cur; if (!vector_read_cmd_push(&ddata->cmd, READ_TMPL)) return (0); if (!cpp_demangle_push_str(ddata, "<", 1)) return (0); limit = 0; v = ddata->push_head > 0 ? &ddata->output_tmp : &ddata->output; for (;;) { idx = v->size; if (!cpp_demangle_read_tmpl_arg(ddata)) return (0); if ((arg = vector_str_substr(v, idx, v->size - 1, &arg_len)) == NULL) return (0); if (!vector_str_find(&ddata->tmpl, arg, arg_len) && !vector_str_push(&ddata->tmpl, arg, arg_len)) { free(arg); return (0); } free(arg); if (*ddata->cur == 'E') { ++ddata->cur; size = v->size; assert(size > 0); if (!strncmp(v->container[size - 1], ">", 1)) { if (!cpp_demangle_push_str(ddata, " >", 2)) return (0); } else if (!cpp_demangle_push_str(ddata, ">", 1)) return (0); break; } else if (*ddata->cur != 'I' && !cpp_demangle_push_str(ddata, ", ", 2)) return (0); if (limit++ > CPP_DEMANGLE_TRY_LIMIT) return (0); } return (vector_read_cmd_pop(&ddata->cmd)); } /* * Read template parameter that forms in 'T[number]_'. * This function much like to read_subst but only for types. */ static int cpp_demangle_read_tmpl_param(struct cpp_demangle_data *ddata) { long nth; if (ddata == NULL || *ddata->cur != 'T') return (0); ++ddata->cur; if (*ddata->cur == '_') return (cpp_demangle_get_tmpl_param(ddata, 0)); else { errno = 0; if ((nth = strtol(ddata->cur, (char **) NULL, 36)) == 0 && errno != 0) return (0); /* T_ is first */ ++nth; while (*ddata->cur != '_') ++ddata->cur; assert(nth > 0); return (cpp_demangle_get_tmpl_param(ddata, nth)); } /* NOTREACHED */ return (0); } static int cpp_demangle_read_type(struct cpp_demangle_data *ddata, int delimit) { struct vector_type_qualifier v; struct vector_str *output; size_t p_idx, type_str_len; int extern_c, is_builtin; long len; char *type_str, *exp_str, *num_str; if (ddata == NULL) return (0); output = &ddata->output; if (!strncmp(ddata->output.container[ddata->output.size - 1], ">", 1)) { ddata->push_head++; output = &ddata->output_tmp; } else if (delimit == 1) { if (ddata->paren == false) { if (!cpp_demangle_push_str(ddata, "(", 1)) return (0); if (ddata->output.size < 2) return (0); ddata->paren = true; ddata->pfirst = true; /* Need pop function name */ if (ddata->subst.size == 1 && !vector_str_pop(&ddata->subst)) return (0); } if (ddata->pfirst) ddata->pfirst = false; else if (*ddata->cur != 'I' && !cpp_demangle_push_str(ddata, ", ", 2)) return (0); } assert(output != NULL); /* * [r, V, K] [P, R, C, G, U] builtin, function, class-enum, array * pointer-to-member, template-param, template-template-param, subst */ if (!vector_type_qualifier_init(&v)) return (0); extern_c = 0; is_builtin = 1; p_idx = output->size; type_str = exp_str = num_str = NULL; again: /* builtin type */ switch (*ddata->cur) { case 'a': /* signed char */ if (!cpp_demangle_push_str(ddata, "signed char", 11)) goto clean; ++ddata->cur; goto rtn; case 'A': /* array type */ if (!cpp_demangle_read_array(ddata)) goto clean; is_builtin = 0; goto rtn; case 'b': /* bool */ if (!cpp_demangle_push_str(ddata, "bool", 4)) goto clean; ++ddata->cur; goto rtn; case 'C': /* complex pair */ if (!vector_type_qualifier_push(&v, TYPE_CMX)) goto clean; ++ddata->cur; goto again; case 'c': /* char */ if (!cpp_demangle_push_str(ddata, "char", 4)) goto clean; ++ddata->cur; goto rtn; case 'd': /* double */ if (!cpp_demangle_push_str(ddata, "double", 6)) goto clean; ++ddata->cur; goto rtn; case 'D': ++ddata->cur; switch (*ddata->cur) { case 'd': /* IEEE 754r decimal floating point (64 bits) */ if (!cpp_demangle_push_str(ddata, "decimal64", 9)) goto clean; ++ddata->cur; break; case 'e': /* IEEE 754r decimal floating point (128 bits) */ if (!cpp_demangle_push_str(ddata, "decimal128", 10)) goto clean; ++ddata->cur; break; case 'f': /* IEEE 754r decimal floating point (32 bits) */ if (!cpp_demangle_push_str(ddata, "decimal32", 9)) goto clean; ++ddata->cur; break; case 'h': /* IEEE 754r half-precision floating point (16 bits) */ if (!cpp_demangle_push_str(ddata, "half", 4)) goto clean; ++ddata->cur; break; case 'i': /* char32_t */ if (!cpp_demangle_push_str(ddata, "char32_t", 8)) goto clean; ++ddata->cur; break; case 'n': /* std::nullptr_t (i.e., decltype(nullptr)) */ if (!cpp_demangle_push_str(ddata, "decltype(nullptr)", 17)) goto clean; ++ddata->cur; break; case 's': /* char16_t */ if (!cpp_demangle_push_str(ddata, "char16_t", 8)) goto clean; ++ddata->cur; break; case 'v': /* gcc vector_size extension. */ ++ddata->cur; if (*ddata->cur == '_') { ++ddata->cur; if (!cpp_demangle_read_expression_flat(ddata, &exp_str)) goto clean; if (!vector_str_push(&v.ext_name, exp_str, strlen(exp_str))) goto clean; } else { if (!cpp_demangle_read_number_as_string(ddata, &num_str)) goto clean; if (!vector_str_push(&v.ext_name, num_str, strlen(num_str))) goto clean; } if (*ddata->cur != '_') goto clean; ++ddata->cur; if (!vector_type_qualifier_push(&v, TYPE_VEC)) goto clean; goto again; default: goto clean; } goto rtn; case 'e': /* long double */ if (!cpp_demangle_push_str(ddata, "long double", 11)) goto clean; ++ddata->cur; goto rtn; case 'f': /* float */ if (!cpp_demangle_push_str(ddata, "float", 5)) goto clean; ++ddata->cur; goto rtn; case 'F': /* function */ if (!cpp_demangle_read_function(ddata, &extern_c, &v)) goto clean; is_builtin = 0; goto rtn; case 'g': /* __float128 */ if (!cpp_demangle_push_str(ddata, "__float128", 10)) goto clean; ++ddata->cur; goto rtn; case 'G': /* imaginary */ if (!vector_type_qualifier_push(&v, TYPE_IMG)) goto clean; ++ddata->cur; goto again; case 'h': /* unsigned char */ if (!cpp_demangle_push_str(ddata, "unsigned char", 13)) goto clean; ++ddata->cur; goto rtn; case 'i': /* int */ if (!cpp_demangle_push_str(ddata, "int", 3)) goto clean; ++ddata->cur; goto rtn; case 'j': /* unsigned int */ if (!cpp_demangle_push_str(ddata, "unsigned int", 12)) goto clean; ++ddata->cur; goto rtn; case 'K': /* const */ if (!vector_type_qualifier_push(&v, TYPE_CST)) goto clean; ++ddata->cur; goto again; case 'l': /* long */ if (!cpp_demangle_push_str(ddata, "long", 4)) goto clean; ++ddata->cur; goto rtn; case 'm': /* unsigned long */ if (!cpp_demangle_push_str(ddata, "unsigned long", 13)) goto clean; ++ddata->cur; goto rtn; case 'M': /* pointer to member */ if (!cpp_demangle_read_pointer_to_member(ddata)) goto clean; is_builtin = 0; goto rtn; case 'n': /* __int128 */ if (!cpp_demangle_push_str(ddata, "__int128", 8)) goto clean; ++ddata->cur; goto rtn; case 'o': /* unsigned __int128 */ if (!cpp_demangle_push_str(ddata, "unsigned __int128", 17)) goto clean; ++ddata->cur; goto rtn; case 'P': /* pointer */ if (!vector_type_qualifier_push(&v, TYPE_PTR)) goto clean; ++ddata->cur; goto again; case 'r': /* restrict */ if (!vector_type_qualifier_push(&v, TYPE_RST)) goto clean; ++ddata->cur; goto again; case 'R': /* reference */ if (!vector_type_qualifier_push(&v, TYPE_REF)) goto clean; ++ddata->cur; goto again; case 's': /* short, local string */ if (!cpp_demangle_push_str(ddata, "short", 5)) goto clean; ++ddata->cur; goto rtn; case 'S': /* substitution */ if (!cpp_demangle_read_subst(ddata)) goto clean; is_builtin = 0; goto rtn; case 't': /* unsigned short */ if (!cpp_demangle_push_str(ddata, "unsigned short", 14)) goto clean; ++ddata->cur; goto rtn; case 'T': /* template parameter */ if (!cpp_demangle_read_tmpl_param(ddata)) goto clean; is_builtin = 0; goto rtn; case 'u': /* vendor extended builtin */ ++ddata->cur; if (!cpp_demangle_read_sname(ddata)) goto clean; is_builtin = 0; goto rtn; case 'U': /* vendor extended type qualifier */ if (!cpp_demangle_read_number(ddata, &len)) goto clean; if (len <= 0) goto clean; if (!vector_str_push(&v.ext_name, ddata->cur, len)) return (0); ddata->cur += len; if (!vector_type_qualifier_push(&v, TYPE_EXT)) goto clean; goto again; case 'v': /* void */ if (!cpp_demangle_push_str(ddata, "void", 4)) goto clean; ++ddata->cur; goto rtn; case 'V': /* volatile */ if (!vector_type_qualifier_push(&v, TYPE_VAT)) goto clean; ++ddata->cur; goto again; case 'w': /* wchar_t */ if (!cpp_demangle_push_str(ddata, "wchar_t", 7)) goto clean; ++ddata->cur; goto rtn; case 'x': /* long long */ if (!cpp_demangle_push_str(ddata, "long long", 9)) goto clean; ++ddata->cur; goto rtn; case 'y': /* unsigned long long */ if (!cpp_demangle_push_str(ddata, "unsigned long long", 18)) goto clean; ++ddata->cur; goto rtn; case 'z': /* ellipsis */ if (!cpp_demangle_push_str(ddata, "ellipsis", 8)) goto clean; ++ddata->cur; goto rtn; } if (!cpp_demangle_read_name(ddata)) goto clean; is_builtin = 0; rtn: if ((type_str = vector_str_substr(output, p_idx, output->size - 1, &type_str_len)) == NULL) goto clean; if (is_builtin == 0) { if (!vector_str_find(&ddata->subst, type_str, type_str_len) && !vector_str_push(&ddata->subst, type_str, type_str_len)) goto clean; } if (!cpp_demangle_push_type_qualifier(ddata, &v, type_str)) goto clean; free(type_str); free(exp_str); free(num_str); vector_type_qualifier_dest(&v); if (ddata->push_head > 0) { if (*ddata->cur == 'I' && cpp_demangle_read_tmpl_args(ddata) == 0) return (0); if (--ddata->push_head > 0) return (1); if (!vector_str_push(&ddata->output_tmp, " ", 1)) return (0); if (!vector_str_push_vector_head(&ddata->output, &ddata->output_tmp)) return (0); vector_str_dest(&ddata->output_tmp); if (!vector_str_init(&ddata->output_tmp)) return (0); if (!cpp_demangle_push_str(ddata, "(", 1)) return (0); ddata->paren = true; ddata->pfirst = true; } return (1); clean: free(type_str); free(exp_str); free(num_str); vector_type_qualifier_dest(&v); return (0); } static int cpp_demangle_read_type_flat(struct cpp_demangle_data *ddata, char **str) { struct vector_str *output; size_t i, p_idx, idx, type_len; char *type; output = ddata->push_head > 0 ? &ddata->output_tmp : &ddata->output; p_idx = output->size; if (!cpp_demangle_read_type(ddata, 0)) return (0); if ((type = vector_str_substr(output, p_idx, output->size - 1, &type_len)) == NULL) return (0); idx = output->size; for (i = p_idx; i < idx; ++i) { if (!vector_str_pop(output)) { free(type); return (0); } } *str = type; return (1); } /* * read unqualified-name, unqualified name are operator-name, ctor-dtor-name, * source-name */ static int cpp_demangle_read_uqname(struct cpp_demangle_data *ddata) { size_t len; if (ddata == NULL || *ddata->cur == '\0') return (0); /* operator name */ switch (SIMPLE_HASH(*ddata->cur, *(ddata->cur + 1))) { case SIMPLE_HASH('a', 'a'): /* operator && */ if (!cpp_demangle_push_str(ddata, "operator&&", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('a', 'd'): /* operator & (unary) */ if (!cpp_demangle_push_str(ddata, "operator&", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('a', 'n'): /* operator & */ if (!cpp_demangle_push_str(ddata, "operator&", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('a', 'N'): /* operator &= */ if (!cpp_demangle_push_str(ddata, "operator&=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('a', 'S'): /* operator = */ if (!cpp_demangle_push_str(ddata, "operator=", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('c', 'l'): /* operator () */ if (!cpp_demangle_push_str(ddata, "operator()", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('c', 'm'): /* operator , */ if (!cpp_demangle_push_str(ddata, "operator,", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('c', 'o'): /* operator ~ */ if (!cpp_demangle_push_str(ddata, "operator~", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('c', 'v'): /* operator (cast) */ if (!cpp_demangle_push_str(ddata, "operator(cast)", 14)) return (0); ddata->cur += 2; return (cpp_demangle_read_type(ddata, 1)); case SIMPLE_HASH('d', 'a'): /* operator delete [] */ if (!cpp_demangle_push_str(ddata, "operator delete []", 18)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('d', 'e'): /* operator * (unary) */ if (!cpp_demangle_push_str(ddata, "operator*", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('d', 'l'): /* operator delete */ if (!cpp_demangle_push_str(ddata, "operator delete", 15)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('d', 'v'): /* operator / */ if (!cpp_demangle_push_str(ddata, "operator/", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('d', 'V'): /* operator /= */ if (!cpp_demangle_push_str(ddata, "operator/=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('e', 'o'): /* operator ^ */ if (!cpp_demangle_push_str(ddata, "operator^", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('e', 'O'): /* operator ^= */ if (!cpp_demangle_push_str(ddata, "operator^=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('e', 'q'): /* operator == */ if (!cpp_demangle_push_str(ddata, "operator==", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('g', 'e'): /* operator >= */ if (!cpp_demangle_push_str(ddata, "operator>=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('g', 't'): /* operator > */ if (!cpp_demangle_push_str(ddata, "operator>", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('i', 'x'): /* operator [] */ if (!cpp_demangle_push_str(ddata, "operator[]", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('l', 'e'): /* operator <= */ if (!cpp_demangle_push_str(ddata, "operator<=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('l', 's'): /* operator << */ if (!cpp_demangle_push_str(ddata, "operator<<", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('l', 'S'): /* operator <<= */ if (!cpp_demangle_push_str(ddata, "operator<<=", 11)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('l', 't'): /* operator < */ if (!cpp_demangle_push_str(ddata, "operator<", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('m', 'i'): /* operator - */ if (!cpp_demangle_push_str(ddata, "operator-", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('m', 'I'): /* operator -= */ if (!cpp_demangle_push_str(ddata, "operator-=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('m', 'l'): /* operator * */ if (!cpp_demangle_push_str(ddata, "operator*", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('m', 'L'): /* operator *= */ if (!cpp_demangle_push_str(ddata, "operator*=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('m', 'm'): /* operator -- */ if (!cpp_demangle_push_str(ddata, "operator--", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('n', 'a'): /* operator new[] */ if (!cpp_demangle_push_str(ddata, "operator new []", 15)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('n', 'e'): /* operator != */ if (!cpp_demangle_push_str(ddata, "operator!=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('n', 'g'): /* operator - (unary) */ if (!cpp_demangle_push_str(ddata, "operator-", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('n', 't'): /* operator ! */ if (!cpp_demangle_push_str(ddata, "operator!", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('n', 'w'): /* operator new */ if (!cpp_demangle_push_str(ddata, "operator new", 12)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('o', 'o'): /* operator || */ if (!cpp_demangle_push_str(ddata, "operator||", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('o', 'r'): /* operator | */ if (!cpp_demangle_push_str(ddata, "operator|", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('o', 'R'): /* operator |= */ if (!cpp_demangle_push_str(ddata, "operator|=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('p', 'l'): /* operator + */ if (!cpp_demangle_push_str(ddata, "operator+", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('p', 'L'): /* operator += */ if (!cpp_demangle_push_str(ddata, "operator+=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('p', 'm'): /* operator ->* */ if (!cpp_demangle_push_str(ddata, "operator->*", 11)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('p', 'p'): /* operator ++ */ if (!cpp_demangle_push_str(ddata, "operator++", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('p', 's'): /* operator + (unary) */ if (!cpp_demangle_push_str(ddata, "operator+", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('p', 't'): /* operator -> */ if (!cpp_demangle_push_str(ddata, "operator->", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('q', 'u'): /* operator ? */ if (!cpp_demangle_push_str(ddata, "operator?", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('r', 'm'): /* operator % */ if (!cpp_demangle_push_str(ddata, "operator%", 9)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('r', 'M'): /* operator %= */ if (!cpp_demangle_push_str(ddata, "operator%=", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('r', 's'): /* operator >> */ if (!cpp_demangle_push_str(ddata, "operator>>", 10)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('r', 'S'): /* operator >>= */ if (!cpp_demangle_push_str(ddata, "operator>>=", 11)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('r', 'z'): /* operator sizeof */ if (!cpp_demangle_push_str(ddata, "operator sizeof ", 16)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('s', 'r'): /* scope resolution operator */ if (!cpp_demangle_push_str(ddata, "scope resolution operator ", 26)) return (0); ddata->cur += 2; return (1); case SIMPLE_HASH('s', 'v'): /* operator sizeof */ if (!cpp_demangle_push_str(ddata, "operator sizeof ", 16)) return (0); ddata->cur += 2; return (1); } /* vendor extened operator */ if (*ddata->cur == 'v' && ELFTC_ISDIGIT(*(ddata->cur + 1))) { if (!cpp_demangle_push_str(ddata, "vendor extened operator ", 24)) return (0); if (!cpp_demangle_push_str(ddata, ddata->cur + 1, 1)) return (0); ddata->cur += 2; return (cpp_demangle_read_sname(ddata)); } /* ctor-dtor-name */ switch (SIMPLE_HASH(*ddata->cur, *(ddata->cur + 1))) { case SIMPLE_HASH('C', '1'): /* FALLTHROUGH */ case SIMPLE_HASH('C', '2'): /* FALLTHROUGH */ case SIMPLE_HASH('C', '3'): if (ddata->last_sname == NULL) return (0); if ((len = strlen(ddata->last_sname)) == 0) return (0); if (!cpp_demangle_push_str(ddata, "::", 2)) return (0); if (!cpp_demangle_push_str(ddata, ddata->last_sname, len)) return (0); ddata->cur +=2; return (1); case SIMPLE_HASH('D', '0'): /* FALLTHROUGH */ case SIMPLE_HASH('D', '1'): /* FALLTHROUGH */ case SIMPLE_HASH('D', '2'): if (ddata->last_sname == NULL) return (0); if ((len = strlen(ddata->last_sname)) == 0) return (0); if (!cpp_demangle_push_str(ddata, "::~", 3)) return (0); if (!cpp_demangle_push_str(ddata, ddata->last_sname, len)) return (0); ddata->cur +=2; return (1); } /* source name */ if (ELFTC_ISDIGIT(*ddata->cur) != 0) return (cpp_demangle_read_sname(ddata)); /* local source name */ if (*ddata->cur == 'L') return (cpp_demangle_local_source_name(ddata)); return (1); } /* * Read local source name. * * References: * http://gcc.gnu.org/bugzilla/show_bug.cgi?id=31775 * http://gcc.gnu.org/viewcvs?view=rev&revision=124467 */ static int cpp_demangle_local_source_name(struct cpp_demangle_data *ddata) { /* L */ if (ddata == NULL || *ddata->cur != 'L') return (0); ++ddata->cur; /* source name */ if (!cpp_demangle_read_sname(ddata)) return (0); /* discriminator */ if (*ddata->cur == '_') { ++ddata->cur; while (ELFTC_ISDIGIT(*ddata->cur) != 0) ++ddata->cur; } return (1); } static int cpp_demangle_read_v_offset(struct cpp_demangle_data *ddata) { if (ddata == NULL) return (0); if (!cpp_demangle_push_str(ddata, "offset : ", 9)) return (0); if (!cpp_demangle_read_offset_number(ddata)) return (0); if (!cpp_demangle_push_str(ddata, "virtual offset : ", 17)) return (0); return (!cpp_demangle_read_offset_number(ddata)); } /* * Decode floating point representation to string * Return new allocated string or NULL * * Todo * Replace these functions to macro. */ static char * decode_fp_to_double(const char *p, size_t len) { double f; size_t rtn_len, limit, i; int byte; char *rtn; if (p == NULL || len == 0 || len % 2 != 0 || len / 2 > sizeof(double)) return (NULL); memset(&f, 0, sizeof(double)); for (i = 0; i < len / 2; ++i) { byte = hex_to_dec(p[len - i * 2 - 1]) + hex_to_dec(p[len - i * 2 - 2]) * 16; if (byte < 0 || byte > 255) return (NULL); #if ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN ((unsigned char *)&f)[i] = (unsigned char)(byte); #else /* ELFTC_BYTE_ORDER != ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ ((unsigned char *)&f)[sizeof(double) - i - 1] = (unsigned char)(byte); #endif /* ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ } rtn_len = 64; limit = 0; again: if ((rtn = malloc(sizeof(char) * rtn_len)) == NULL) return (NULL); if (snprintf(rtn, rtn_len, "%fld", f) >= (int)rtn_len) { free(rtn); if (limit++ > FLOAT_SPRINTF_TRY_LIMIT) return (NULL); rtn_len *= BUFFER_GROWFACTOR; goto again; } return rtn; } static char * decode_fp_to_float(const char *p, size_t len) { size_t i, rtn_len, limit; float f; int byte; char *rtn; if (p == NULL || len == 0 || len % 2 != 0 || len / 2 > sizeof(float)) return (NULL); memset(&f, 0, sizeof(float)); for (i = 0; i < len / 2; ++i) { byte = hex_to_dec(p[len - i * 2 - 1]) + hex_to_dec(p[len - i * 2 - 2]) * 16; if (byte < 0 || byte > 255) return (NULL); #if ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN ((unsigned char *)&f)[i] = (unsigned char)(byte); #else /* ELFTC_BYTE_ORDER != ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ ((unsigned char *)&f)[sizeof(float) - i - 1] = (unsigned char)(byte); #endif /* ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ } rtn_len = 64; limit = 0; again: if ((rtn = malloc(sizeof(char) * rtn_len)) == NULL) return (NULL); if (snprintf(rtn, rtn_len, "%ff", f) >= (int)rtn_len) { free(rtn); if (limit++ > FLOAT_SPRINTF_TRY_LIMIT) return (NULL); rtn_len *= BUFFER_GROWFACTOR; goto again; } return rtn; } static char * decode_fp_to_float128(const char *p, size_t len) { long double f; size_t rtn_len, limit, i; int byte; unsigned char buf[FLOAT_QUADRUPLE_BYTES]; char *rtn; switch(sizeof(long double)) { case FLOAT_QUADRUPLE_BYTES: return (decode_fp_to_long_double(p, len)); case FLOAT_EXTENED_BYTES: if (p == NULL || len == 0 || len % 2 != 0 || len / 2 > FLOAT_QUADRUPLE_BYTES) return (NULL); memset(buf, 0, FLOAT_QUADRUPLE_BYTES); for (i = 0; i < len / 2; ++i) { byte = hex_to_dec(p[len - i * 2 - 1]) + hex_to_dec(p[len - i * 2 - 2]) * 16; if (byte < 0 || byte > 255) return (NULL); #if ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN buf[i] = (unsigned char)(byte); #else /* ELFTC_BYTE_ORDER != ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ buf[FLOAT_QUADRUPLE_BYTES - i -1] = (unsigned char)(byte); #endif /* ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ } memset(&f, 0, FLOAT_EXTENED_BYTES); #if ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN memcpy(&f, buf, FLOAT_EXTENED_BYTES); #else /* ELFTC_BYTE_ORDER != ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ memcpy(&f, buf + 6, FLOAT_EXTENED_BYTES); #endif /* ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ rtn_len = 256; limit = 0; again: if ((rtn = malloc(sizeof(char) * rtn_len)) == NULL) return (NULL); if (snprintf(rtn, rtn_len, "%Lfd", f) >= (int)rtn_len) { free(rtn); if (limit++ > FLOAT_SPRINTF_TRY_LIMIT) return (NULL); rtn_len *= BUFFER_GROWFACTOR; goto again; } return (rtn); default: return (NULL); } } static char * decode_fp_to_float80(const char *p, size_t len) { long double f; size_t rtn_len, limit, i; int byte; unsigned char buf[FLOAT_EXTENED_BYTES]; char *rtn; switch(sizeof(long double)) { case FLOAT_QUADRUPLE_BYTES: if (p == NULL || len == 0 || len % 2 != 0 || len / 2 > FLOAT_EXTENED_BYTES) return (NULL); memset(buf, 0, FLOAT_EXTENED_BYTES); for (i = 0; i < len / 2; ++i) { byte = hex_to_dec(p[len - i * 2 - 1]) + hex_to_dec(p[len - i * 2 - 2]) * 16; if (byte < 0 || byte > 255) return (NULL); #if ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN buf[i] = (unsigned char)(byte); #else /* ELFTC_BYTE_ORDER != ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ buf[FLOAT_EXTENED_BYTES - i -1] = (unsigned char)(byte); #endif /* ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ } memset(&f, 0, FLOAT_QUADRUPLE_BYTES); #if ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN memcpy(&f, buf, FLOAT_EXTENED_BYTES); #else /* ELFTC_BYTE_ORDER != ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ memcpy((unsigned char *)(&f) + 6, buf, FLOAT_EXTENED_BYTES); #endif /* ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ rtn_len = 256; limit = 0; again: if ((rtn = malloc(sizeof(char) * rtn_len)) == NULL) return (NULL); if (snprintf(rtn, rtn_len, "%Lfd", f) >= (int)rtn_len) { free(rtn); if (limit++ > FLOAT_SPRINTF_TRY_LIMIT) return (NULL); rtn_len *= BUFFER_GROWFACTOR; goto again; } return (rtn); case FLOAT_EXTENED_BYTES: return (decode_fp_to_long_double(p, len)); default: return (NULL); } } static char * decode_fp_to_long_double(const char *p, size_t len) { long double f; size_t rtn_len, limit, i; int byte; char *rtn; if (p == NULL || len == 0 || len % 2 != 0 || len / 2 > sizeof(long double)) return (NULL); memset(&f, 0, sizeof(long double)); for (i = 0; i < len / 2; ++i) { byte = hex_to_dec(p[len - i * 2 - 1]) + hex_to_dec(p[len - i * 2 - 2]) * 16; if (byte < 0 || byte > 255) return (NULL); #if ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN ((unsigned char *)&f)[i] = (unsigned char)(byte); #else /* ELFTC_BYTE_ORDER != ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ ((unsigned char *)&f)[sizeof(long double) - i - 1] = (unsigned char)(byte); #endif /* ELFTC_BYTE_ORDER == ELFTC_BYTE_ORDER_LITTLE_ENDIAN */ } rtn_len = 256; limit = 0; again: if ((rtn = malloc(sizeof(char) * rtn_len)) == NULL) return (NULL); if (snprintf(rtn, rtn_len, "%Lfd", f) >= (int)rtn_len) { free(rtn); if (limit++ > FLOAT_SPRINTF_TRY_LIMIT) return (NULL); rtn_len *= BUFFER_GROWFACTOR; goto again; } return (rtn); } /* Simple hex to integer function used by decode_to_* function. */ static int hex_to_dec(char c) { switch (c) { case '0': return (0); case '1': return (1); case '2': return (2); case '3': return (3); case '4': return (4); case '5': return (5); case '6': return (6); case '7': return (7); case '8': return (8); case '9': return (9); case 'a': return (10); case 'b': return (11); case 'c': return (12); case 'd': return (13); case 'e': return (14); case 'f': return (15); default: return (-1); } } /** * @brief Test input string is mangled by IA-64 C++ ABI style. * * Test string heads with "_Z" or "_GLOBAL__I_". * @return Return 0 at false. */ bool is_cpp_mangled_gnu3(const char *org) { size_t len; len = strlen(org); return ((len > 2 && *org == '_' && *(org + 1) == 'Z') || (len > 11 && !strncmp(org, "_GLOBAL__I_", 11))); } static void vector_read_cmd_dest(struct vector_read_cmd *v) { if (v == NULL) return; free(v->r_container); } /* return -1 at failed, 0 at not found, 1 at found. */ static int vector_read_cmd_find(struct vector_read_cmd *v, enum read_cmd dst) { size_t i; if (v == NULL || dst == READ_FAIL) return (-1); for (i = 0; i < v->size; ++i) if (v->r_container[i] == dst) return (1); return (0); } static int vector_read_cmd_init(struct vector_read_cmd *v) { if (v == NULL) return (0); v->size = 0; v->capacity = VECTOR_DEF_CAPACITY; if ((v->r_container = malloc(sizeof(enum read_cmd) * v->capacity)) == NULL) return (0); return (1); } static int vector_read_cmd_pop(struct vector_read_cmd *v) { if (v == NULL || v->size == 0) return (0); --v->size; v->r_container[v->size] = READ_FAIL; return (1); } static int vector_read_cmd_push(struct vector_read_cmd *v, enum read_cmd cmd) { enum read_cmd *tmp_r_ctn; size_t tmp_cap; size_t i; if (v == NULL) return (0); if (v->size == v->capacity) { tmp_cap = v->capacity * BUFFER_GROWFACTOR; if ((tmp_r_ctn = malloc(sizeof(enum read_cmd) * tmp_cap)) == NULL) return (0); for (i = 0; i < v->size; ++i) tmp_r_ctn[i] = v->r_container[i]; free(v->r_container); v->r_container = tmp_r_ctn; v->capacity = tmp_cap; } v->r_container[v->size] = cmd; ++v->size; return (1); } static void vector_type_qualifier_dest(struct vector_type_qualifier *v) { if (v == NULL) return; free(v->q_container); vector_str_dest(&v->ext_name); } /* size, capacity, ext_name */ static int vector_type_qualifier_init(struct vector_type_qualifier *v) { if (v == NULL) return (0); v->size = 0; v->capacity = VECTOR_DEF_CAPACITY; if ((v->q_container = malloc(sizeof(enum type_qualifier) * v->capacity)) == NULL) return (0); assert(v->q_container != NULL); if (vector_str_init(&v->ext_name) == false) { free(v->q_container); return (0); } return (1); } static int vector_type_qualifier_push(struct vector_type_qualifier *v, enum type_qualifier t) { enum type_qualifier *tmp_ctn; size_t tmp_cap; size_t i; if (v == NULL) return (0); if (v->size == v->capacity) { tmp_cap = v->capacity * BUFFER_GROWFACTOR; if ((tmp_ctn = malloc(sizeof(enum type_qualifier) * tmp_cap)) == NULL) return (0); for (i = 0; i < v->size; ++i) tmp_ctn[i] = v->q_container[i]; free(v->q_container); v->q_container = tmp_ctn; v->capacity = tmp_cap; } v->q_container[v->size] = t; ++v->size; return (1); } Index: projects/netbsd-tests-update-12/contrib/elftoolchain/readelf/readelf.1 =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/readelf/readelf.1 (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/readelf/readelf.1 (revision 305172) @@ -1,197 +1,197 @@ .\" Copyright (c) 2009,2011 Joseph Koshy .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer .\" in this position and unchanged. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS OR .\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES .\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. .\" IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, .\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT .\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF .\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .\" -.\" $Id: readelf.1 3219 2015-05-24 23:42:34Z kaiwang27 $ +.\" $Id: readelf.1 3486 2016-08-22 14:10:05Z emaste $ .\" .Dd September 13, 2012 .Os .Dt READELF 1 .Sh NAME .Nm readelf .Nd display information about ELF objects .Sh SYNOPSIS .Nm .Op Fl a | Fl -all .Op Fl c | Fl -archive-index .Op Fl d | Fl -dynamic .Op Fl e | Fl -headers .Op Fl g | Fl -section-groups .Op Fl h | Fl -file-header .Op Fl l | Fl -program-headers .Op Fl n | Fl -notes .Op Fl p Ar section | Fl -string-dump Ns = Ns Ar section .Op Fl r | Fl -relocs .Op Fl t | Fl -section-details -.Op Fl x Ar section | Fl -hex-dump Ns = Ns Ar section .Op Fl v | Fl -version .Oo .Fl w Ns Oo Ns Ar afilmoprsFLR Ns Oc | .Fl -debug-dump Ns Op Ns = Ns Ar long-option-name , Ns ... .Oc +.Op Fl x Ar section | Fl -hex-dump Ns = Ns Ar section .Op Fl A | Fl -arch-specific .Op Fl D | Fl -use-dynamic .Op Fl H | Fl -help .Op Fl I | Fl -histogram .Op Fl N | -full-section-name .Op Fl S | Fl -sections | Fl -section-headers .Op Fl V | Fl -version-info .Op Fl W | Fl -wide .Ar file... .Sh DESCRIPTION The .Nm utility displays information about ELF objects and .Xr ar 1 archives. .Pp The .Nm utility recognizes the following options: .Bl -tag -width indent .It Fl a | Fl -all Turn on the following flags: .Fl d , .Fl h , .Fl I , .Fl l , .Fl r , .Fl s , .Fl A , .Fl S and .Fl V . .It Fl c | Fl -archive-index Print the archive symbol table for archives. .It Fl d | Fl -dynamic Print the contents of the .Li SHT_DYNAMIC sections in the ELF object. .It Fl e | Fl -headers Print all program, file and section headers in the ELF object. .It Fl g | Fl -section-groups Print the contents of the section groups in the ELF object. .It Fl h | Fl -file-header Print the file header of the ELF object. .It Fl l | Fl -program-headers Print the content of the program header table for the object. .It Fl n | Fl -notes Print the contents of .Li PT_NOTE segments or .Li SHT_NOTE sections present in the ELF object. .It Fl p Ar section | Fl -string-dump Ns = Ns Ar section Print the contents of the specified section as printable strings. The argument .Ar section should be the name of a section or a numeric section index. .It Fl r | Fl -relocs Print relocation information. .It Fl s | Fl -syms | Fl -symbols Print symbol tables. .It Fl t | Fl -section-details Print additional information about sections, such as the flags fields in section headers. .It Fl v | Fl -version Prints a version identifier for .Nm and exits. .It Fl w Ns Oo afilmoprsFLR Oc | Xo .Fl -debug-dump Ns Op Ns = Ns Ar long-option-name , Ns ... .Xc Display DWARF information. The .Fl w option is used with the short options in the following table; the .Fl -debug-dump option is used with a comma-separated list of the corresponding long option names: .Bl -column ".Em Short Option" "aranges|ranges" .It Em Short Option Ta Em Long Option Ta Em Description .It a Ta abbrev Ta Show abbreviation information. .It f Ta frames Ta Show frame information, displaying frame instructions. .It i Ta info Ta Show debugging information entries. .It l Ta rawline Ta Show line information in raw form. .It m Ta macro Ta Show macro information. .It o Ta loc Ta Show location list information. .It p Ta pubnames Ta Show global names. .It r Ta aranges|ranges Ta Show address range information. .It s Ta str Ta Show the debug string table. .It F Ta frames-interp Ta Show frame information, displaying register rules. .It L Ta decodedline Ta Show line information in decoded form. .It R Ta Ranges Ta Show range lists. .El .Pp If no sub-options are specified, the default is to show information corresponding to the .Ar a , f , i, l , o , p , r , s and .Ar R short options. .It Fl x Ar section | Fl -hex-dump Ns = Ns Ar section Display the contents of the specified section in hexadecimal. The argument .Ar section should be the name of a section or a numeric section index. .It Fl A | Fl -arch-specific This option is accepted but is currently unimplemented. .It Fl D | Fl -use-dynamic Print the symbol table specified by the .Li DT_SYMTAB entry in the .Dq Li .dynamic section. .It Fl H | Fl -help Print a help message. .It Fl I | Fl -histogram Print information on bucket list lengths for sections of type .Li SHT_HASH and .Li SHT_GNU_HASH . .It Fl N | Fl -full-section-name This option is accepted but is currently unimplemented. .It Fl S | Fl -sections | Fl -section-headers Print information in the section headers for each ELF object. .It Fl V | Fl -version-info Print symbol versioning information. .It Fl W | Fl -wide Print information about ELF structures using one long line per structure. If this option is not specified, .Nm will list information in the headers of 64 bit ELF objects on two separate lines. .El .Sh EXIT STATUS .Ex -std .Sh SEE ALSO .Xr nm 1 , .Xr addr2line 1 , .Xr elfcopy 1 , .Sh AUTHORS The .Nm utility was written by .An Kai Wang Aq Mt kaiwang27@users.sourceforge.net . Index: projects/netbsd-tests-update-12/contrib/elftoolchain/readelf/readelf.c =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain/readelf/readelf.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain/readelf/readelf.c (revision 305172) @@ -1,7239 +1,7294 @@ /*- * Copyright (c) 2009-2015 Kai Wang * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "_elftc.h" -ELFTC_VCSID("$Id: readelf.c 3469 2016-05-15 23:16:09Z emaste $"); +ELFTC_VCSID("$Id: readelf.c 3484 2016-08-03 13:36:49Z emaste $"); /* Backwards compatability for older FreeBSD releases. */ #ifndef STB_GNU_UNIQUE #define STB_GNU_UNIQUE 10 #endif #ifndef STT_SPARC_REGISTER #define STT_SPARC_REGISTER 13 #endif /* * readelf(1) options. */ #define RE_AA 0x00000001 #define RE_C 0x00000002 #define RE_DD 0x00000004 #define RE_D 0x00000008 #define RE_G 0x00000010 #define RE_H 0x00000020 #define RE_II 0x00000040 #define RE_I 0x00000080 #define RE_L 0x00000100 #define RE_NN 0x00000200 #define RE_N 0x00000400 #define RE_P 0x00000800 #define RE_R 0x00001000 #define RE_SS 0x00002000 #define RE_S 0x00004000 #define RE_T 0x00008000 #define RE_U 0x00010000 #define RE_VV 0x00020000 #define RE_WW 0x00040000 #define RE_W 0x00080000 #define RE_X 0x00100000 /* * dwarf dump options. */ #define DW_A 0x00000001 #define DW_FF 0x00000002 #define DW_F 0x00000004 #define DW_I 0x00000008 #define DW_LL 0x00000010 #define DW_L 0x00000020 #define DW_M 0x00000040 #define DW_O 0x00000080 #define DW_P 0x00000100 #define DW_RR 0x00000200 #define DW_R 0x00000400 #define DW_S 0x00000800 #define DW_DEFAULT_OPTIONS (DW_A | DW_F | DW_I | DW_L | DW_O | DW_P | \ DW_R | DW_RR | DW_S) /* * readelf(1) run control flags. */ #define DISPLAY_FILENAME 0x0001 /* * Internal data structure for sections. */ struct section { const char *name; /* section name */ Elf_Scn *scn; /* section scn */ uint64_t off; /* section offset */ uint64_t sz; /* section size */ uint64_t entsize; /* section entsize */ uint64_t align; /* section alignment */ uint64_t type; /* section type */ uint64_t flags; /* section flags */ uint64_t addr; /* section virtual addr */ uint32_t link; /* section link ndx */ uint32_t info; /* section info ndx */ }; struct dumpop { union { size_t si; /* section index */ const char *sn; /* section name */ } u; enum { DUMP_BY_INDEX = 0, DUMP_BY_NAME } type; /* dump type */ #define HEX_DUMP 0x0001 #define STR_DUMP 0x0002 int op; /* dump operation */ STAILQ_ENTRY(dumpop) dumpop_list; }; struct symver { const char *name; int type; }; /* * Structure encapsulates the global data for readelf(1). */ struct readelf { const char *filename; /* current processing file. */ int options; /* command line options. */ int flags; /* run control flags. */ int dop; /* dwarf dump options. */ Elf *elf; /* underlying ELF descriptor. */ Elf *ar; /* archive ELF descriptor. */ Dwarf_Debug dbg; /* DWARF handle. */ Dwarf_Half cu_psize; /* DWARF CU pointer size. */ Dwarf_Half cu_osize; /* DWARF CU offset size. */ Dwarf_Half cu_ver; /* DWARF CU version. */ GElf_Ehdr ehdr; /* ELF header. */ int ec; /* ELF class. */ size_t shnum; /* #sections. */ struct section *vd_s; /* Verdef section. */ struct section *vn_s; /* Verneed section. */ struct section *vs_s; /* Versym section. */ uint16_t *vs; /* Versym array. */ int vs_sz; /* Versym array size. */ struct symver *ver; /* Version array. */ int ver_sz; /* Size of version array. */ struct section *sl; /* list of sections. */ STAILQ_HEAD(, dumpop) v_dumpop; /* list of dump ops. */ uint64_t (*dw_read)(Elf_Data *, uint64_t *, int); uint64_t (*dw_decode)(uint8_t **, int); }; enum options { OPTION_DEBUG_DUMP }; static struct option longopts[] = { {"all", no_argument, NULL, 'a'}, {"arch-specific", no_argument, NULL, 'A'}, {"archive-index", no_argument, NULL, 'c'}, {"debug-dump", optional_argument, NULL, OPTION_DEBUG_DUMP}, {"dynamic", no_argument, NULL, 'd'}, {"file-header", no_argument, NULL, 'h'}, {"full-section-name", no_argument, NULL, 'N'}, {"headers", no_argument, NULL, 'e'}, {"help", no_argument, 0, 'H'}, {"hex-dump", required_argument, NULL, 'x'}, {"histogram", no_argument, NULL, 'I'}, {"notes", no_argument, NULL, 'n'}, {"program-headers", no_argument, NULL, 'l'}, {"relocs", no_argument, NULL, 'r'}, {"sections", no_argument, NULL, 'S'}, {"section-headers", no_argument, NULL, 'S'}, {"section-groups", no_argument, NULL, 'g'}, {"section-details", no_argument, NULL, 't'}, {"segments", no_argument, NULL, 'l'}, {"string-dump", required_argument, NULL, 'p'}, {"symbols", no_argument, NULL, 's'}, {"syms", no_argument, NULL, 's'}, {"unwind", no_argument, NULL, 'u'}, {"use-dynamic", no_argument, NULL, 'D'}, {"version-info", no_argument, 0, 'V'}, {"version", no_argument, 0, 'v'}, {"wide", no_argument, 0, 'W'}, {NULL, 0, NULL, 0} }; struct eflags_desc { uint64_t flag; const char *desc; }; struct mips_option { uint64_t flag; const char *desc; }; static void add_dumpop(struct readelf *re, size_t si, const char *sn, int op, int t); static const char *aeabi_adv_simd_arch(uint64_t simd); static const char *aeabi_align_needed(uint64_t an); static const char *aeabi_align_preserved(uint64_t ap); static const char *aeabi_arm_isa(uint64_t ai); static const char *aeabi_cpu_arch(uint64_t arch); static const char *aeabi_cpu_arch_profile(uint64_t pf); static const char *aeabi_div(uint64_t du); static const char *aeabi_enum_size(uint64_t es); static const char *aeabi_fp_16bit_format(uint64_t fp16); static const char *aeabi_fp_arch(uint64_t fp); static const char *aeabi_fp_denormal(uint64_t fd); static const char *aeabi_fp_exceptions(uint64_t fe); static const char *aeabi_fp_hpext(uint64_t fh); static const char *aeabi_fp_number_model(uint64_t fn); static const char *aeabi_fp_optm_goal(uint64_t fog); static const char *aeabi_fp_rounding(uint64_t fr); static const char *aeabi_hardfp(uint64_t hfp); static const char *aeabi_mpext(uint64_t mp); static const char *aeabi_optm_goal(uint64_t og); static const char *aeabi_pcs_config(uint64_t pcs); static const char *aeabi_pcs_got(uint64_t got); static const char *aeabi_pcs_r9(uint64_t r9); static const char *aeabi_pcs_ro(uint64_t ro); static const char *aeabi_pcs_rw(uint64_t rw); static const char *aeabi_pcs_wchar_t(uint64_t wt); static const char *aeabi_t2ee(uint64_t t2ee); static const char *aeabi_thumb_isa(uint64_t ti); static const char *aeabi_fp_user_exceptions(uint64_t fu); static const char *aeabi_unaligned_access(uint64_t ua); static const char *aeabi_vfp_args(uint64_t va); static const char *aeabi_virtual(uint64_t vt); static const char *aeabi_wmmx_arch(uint64_t wmmx); static const char *aeabi_wmmx_args(uint64_t wa); static const char *elf_class(unsigned int class); static const char *elf_endian(unsigned int endian); static const char *elf_machine(unsigned int mach); static const char *elf_osabi(unsigned int abi); static const char *elf_type(unsigned int type); static const char *elf_ver(unsigned int ver); static const char *dt_type(unsigned int mach, unsigned int dtype); static void dump_ar(struct readelf *re, int); static void dump_arm_attributes(struct readelf *re, uint8_t *p, uint8_t *pe); static void dump_attributes(struct readelf *re); static uint8_t *dump_compatibility_tag(uint8_t *p, uint8_t *pe); static void dump_dwarf(struct readelf *re); static void dump_dwarf_abbrev(struct readelf *re); static void dump_dwarf_aranges(struct readelf *re); static void dump_dwarf_block(struct readelf *re, uint8_t *b, Dwarf_Unsigned len); static void dump_dwarf_die(struct readelf *re, Dwarf_Die die, int level); static void dump_dwarf_frame(struct readelf *re, int alt); static void dump_dwarf_frame_inst(struct readelf *re, Dwarf_Cie cie, uint8_t *insts, Dwarf_Unsigned len, Dwarf_Unsigned caf, Dwarf_Signed daf, Dwarf_Addr pc, Dwarf_Debug dbg); static int dump_dwarf_frame_regtable(struct readelf *re, Dwarf_Fde fde, Dwarf_Addr pc, Dwarf_Unsigned func_len, Dwarf_Half cie_ra); static void dump_dwarf_frame_section(struct readelf *re, struct section *s, int alt); static void dump_dwarf_info(struct readelf *re, Dwarf_Bool is_info); static void dump_dwarf_macinfo(struct readelf *re); static void dump_dwarf_line(struct readelf *re); static void dump_dwarf_line_decoded(struct readelf *re); static void dump_dwarf_loc(struct readelf *re, Dwarf_Loc *lr); static void dump_dwarf_loclist(struct readelf *re); static void dump_dwarf_pubnames(struct readelf *re); static void dump_dwarf_ranges(struct readelf *re); static void dump_dwarf_ranges_foreach(struct readelf *re, Dwarf_Die die, Dwarf_Addr base); static void dump_dwarf_str(struct readelf *re); static void dump_eflags(struct readelf *re, uint64_t e_flags); static void dump_elf(struct readelf *re); static void dump_dyn_val(struct readelf *re, GElf_Dyn *dyn, uint32_t stab); static void dump_dynamic(struct readelf *re); static void dump_liblist(struct readelf *re); static void dump_mips_attributes(struct readelf *re, uint8_t *p, uint8_t *pe); static void dump_mips_odk_reginfo(struct readelf *re, uint8_t *p, size_t sz); static void dump_mips_options(struct readelf *re, struct section *s); static void dump_mips_option_flags(const char *name, struct mips_option *opt, uint64_t info); static void dump_mips_reginfo(struct readelf *re, struct section *s); static void dump_mips_specific_info(struct readelf *re); static void dump_notes(struct readelf *re); static void dump_notes_content(struct readelf *re, const char *buf, size_t sz, off_t off); static void dump_svr4_hash(struct section *s); static void dump_svr4_hash64(struct readelf *re, struct section *s); static void dump_gnu_hash(struct readelf *re, struct section *s); static void dump_hash(struct readelf *re); static void dump_phdr(struct readelf *re); static void dump_ppc_attributes(uint8_t *p, uint8_t *pe); static void dump_section_groups(struct readelf *re); static void dump_symtab(struct readelf *re, int i); static void dump_symtabs(struct readelf *re); static uint8_t *dump_unknown_tag(uint64_t tag, uint8_t *p, uint8_t *pe); static void dump_ver(struct readelf *re); static void dump_verdef(struct readelf *re, int dump); static void dump_verneed(struct readelf *re, int dump); static void dump_versym(struct readelf *re); static const char *dwarf_reg(unsigned int mach, unsigned int reg); static const char *dwarf_regname(struct readelf *re, unsigned int num); static struct dumpop *find_dumpop(struct readelf *re, size_t si, const char *sn, int op, int t); static int get_ent_count(struct section *s, int *ent_count); static char *get_regoff_str(struct readelf *re, Dwarf_Half reg, Dwarf_Addr off); static const char *get_string(struct readelf *re, int strtab, size_t off); static const char *get_symbol_name(struct readelf *re, int symtab, int i); static uint64_t get_symbol_value(struct readelf *re, int symtab, int i); static void load_sections(struct readelf *re); static const char *mips_abi_fp(uint64_t fp); static const char *note_type(const char *note_name, unsigned int et, unsigned int nt); static const char *note_type_freebsd(unsigned int nt); static const char *note_type_freebsd_core(unsigned int nt); static const char *note_type_linux_core(unsigned int nt); static const char *note_type_gnu(unsigned int nt); static const char *note_type_netbsd(unsigned int nt); static const char *note_type_openbsd(unsigned int nt); static const char *note_type_unknown(unsigned int nt); static const char *note_type_xen(unsigned int nt); static const char *option_kind(uint8_t kind); static const char *phdr_type(unsigned int mach, unsigned int ptype); static const char *ppc_abi_fp(uint64_t fp); static const char *ppc_abi_vector(uint64_t vec); static void readelf_usage(int status); static void readelf_version(void); static void search_loclist_at(struct readelf *re, Dwarf_Die die, Dwarf_Unsigned lowpc); static void search_ver(struct readelf *re); static const char *section_type(unsigned int mach, unsigned int stype); static void set_cu_context(struct readelf *re, Dwarf_Half psize, Dwarf_Half osize, Dwarf_Half ver); static const char *st_bind(unsigned int sbind); static const char *st_shndx(unsigned int shndx); static const char *st_type(unsigned int mach, unsigned int os, unsigned int stype); static const char *st_vis(unsigned int svis); static const char *top_tag(unsigned int tag); static void unload_sections(struct readelf *re); static uint64_t _read_lsb(Elf_Data *d, uint64_t *offsetp, int bytes_to_read); static uint64_t _read_msb(Elf_Data *d, uint64_t *offsetp, int bytes_to_read); static uint64_t _decode_lsb(uint8_t **data, int bytes_to_read); static uint64_t _decode_msb(uint8_t **data, int bytes_to_read); static int64_t _decode_sleb128(uint8_t **dp, uint8_t *dpe); static uint64_t _decode_uleb128(uint8_t **dp, uint8_t *dpe); static struct eflags_desc arm_eflags_desc[] = { {EF_ARM_RELEXEC, "relocatable executable"}, {EF_ARM_HASENTRY, "has entry point"}, {EF_ARM_SYMSARESORTED, "sorted symbol tables"}, {EF_ARM_DYNSYMSUSESEGIDX, "dynamic symbols use segment index"}, {EF_ARM_MAPSYMSFIRST, "mapping symbols precede others"}, {EF_ARM_BE8, "BE8"}, {EF_ARM_LE8, "LE8"}, {EF_ARM_INTERWORK, "interworking enabled"}, {EF_ARM_APCS_26, "uses APCS/26"}, {EF_ARM_APCS_FLOAT, "uses APCS/float"}, {EF_ARM_PIC, "position independent"}, {EF_ARM_ALIGN8, "8 bit structure alignment"}, {EF_ARM_NEW_ABI, "uses new ABI"}, {EF_ARM_OLD_ABI, "uses old ABI"}, {EF_ARM_SOFT_FLOAT, "software FP"}, {EF_ARM_VFP_FLOAT, "VFP"}, {EF_ARM_MAVERICK_FLOAT, "Maverick FP"}, {0, NULL} }; static struct eflags_desc mips_eflags_desc[] = { {EF_MIPS_NOREORDER, "noreorder"}, {EF_MIPS_PIC, "pic"}, {EF_MIPS_CPIC, "cpic"}, {EF_MIPS_UCODE, "ugen_reserved"}, {EF_MIPS_ABI2, "abi2"}, {EF_MIPS_OPTIONS_FIRST, "odk first"}, {EF_MIPS_ARCH_ASE_MDMX, "mdmx"}, {EF_MIPS_ARCH_ASE_M16, "mips16"}, {0, NULL} }; static struct eflags_desc powerpc_eflags_desc[] = { {EF_PPC_EMB, "emb"}, {EF_PPC_RELOCATABLE, "relocatable"}, {EF_PPC_RELOCATABLE_LIB, "relocatable-lib"}, {0, NULL} }; static struct eflags_desc sparc_eflags_desc[] = { {EF_SPARC_32PLUS, "v8+"}, {EF_SPARC_SUN_US1, "ultrasparcI"}, {EF_SPARC_HAL_R1, "halr1"}, {EF_SPARC_SUN_US3, "ultrasparcIII"}, {0, NULL} }; static const char * elf_osabi(unsigned int abi) { static char s_abi[32]; switch(abi) { case ELFOSABI_NONE: return "NONE"; case ELFOSABI_HPUX: return "HPUX"; case ELFOSABI_NETBSD: return "NetBSD"; case ELFOSABI_GNU: return "GNU"; case ELFOSABI_HURD: return "HURD"; case ELFOSABI_86OPEN: return "86OPEN"; case ELFOSABI_SOLARIS: return "Solaris"; case ELFOSABI_AIX: return "AIX"; case ELFOSABI_IRIX: return "IRIX"; case ELFOSABI_FREEBSD: return "FreeBSD"; case ELFOSABI_TRU64: return "TRU64"; case ELFOSABI_MODESTO: return "MODESTO"; case ELFOSABI_OPENBSD: return "OpenBSD"; case ELFOSABI_OPENVMS: return "OpenVMS"; case ELFOSABI_NSK: return "NSK"; case ELFOSABI_CLOUDABI: return "CloudABI"; + case ELFOSABI_ARM_AEABI: return "ARM EABI"; case ELFOSABI_ARM: return "ARM"; case ELFOSABI_STANDALONE: return "StandAlone"; default: snprintf(s_abi, sizeof(s_abi), "", abi); return (s_abi); } }; static const char * elf_machine(unsigned int mach) { static char s_mach[32]; switch (mach) { case EM_NONE: return "Unknown machine"; case EM_M32: return "AT&T WE32100"; case EM_SPARC: return "Sun SPARC"; case EM_386: return "Intel i386"; case EM_68K: return "Motorola 68000"; case EM_IAMCU: return "Intel MCU"; case EM_88K: return "Motorola 88000"; case EM_860: return "Intel i860"; case EM_MIPS: return "MIPS R3000 Big-Endian only"; case EM_S370: return "IBM System/370"; case EM_MIPS_RS3_LE: return "MIPS R3000 Little-Endian"; case EM_PARISC: return "HP PA-RISC"; case EM_VPP500: return "Fujitsu VPP500"; case EM_SPARC32PLUS: return "SPARC v8plus"; case EM_960: return "Intel 80960"; case EM_PPC: return "PowerPC 32-bit"; case EM_PPC64: return "PowerPC 64-bit"; case EM_S390: return "IBM System/390"; case EM_V800: return "NEC V800"; case EM_FR20: return "Fujitsu FR20"; case EM_RH32: return "TRW RH-32"; case EM_RCE: return "Motorola RCE"; case EM_ARM: return "ARM"; case EM_SH: return "Hitachi SH"; case EM_SPARCV9: return "SPARC v9 64-bit"; case EM_TRICORE: return "Siemens TriCore embedded processor"; case EM_ARC: return "Argonaut RISC Core"; case EM_H8_300: return "Hitachi H8/300"; case EM_H8_300H: return "Hitachi H8/300H"; case EM_H8S: return "Hitachi H8S"; case EM_H8_500: return "Hitachi H8/500"; case EM_IA_64: return "Intel IA-64 Processor"; case EM_MIPS_X: return "Stanford MIPS-X"; case EM_COLDFIRE: return "Motorola ColdFire"; case EM_68HC12: return "Motorola M68HC12"; case EM_MMA: return "Fujitsu MMA"; case EM_PCP: return "Siemens PCP"; case EM_NCPU: return "Sony nCPU"; case EM_NDR1: return "Denso NDR1 microprocessor"; case EM_STARCORE: return "Motorola Star*Core processor"; case EM_ME16: return "Toyota ME16 processor"; case EM_ST100: return "STMicroelectronics ST100 processor"; case EM_TINYJ: return "Advanced Logic Corp. TinyJ processor"; case EM_X86_64: return "Advanced Micro Devices x86-64"; case EM_PDSP: return "Sony DSP Processor"; case EM_FX66: return "Siemens FX66 microcontroller"; case EM_ST9PLUS: return "STMicroelectronics ST9+ 8/16 microcontroller"; case EM_ST7: return "STmicroelectronics ST7 8-bit microcontroller"; case EM_68HC16: return "Motorola MC68HC16 microcontroller"; case EM_68HC11: return "Motorola MC68HC11 microcontroller"; case EM_68HC08: return "Motorola MC68HC08 microcontroller"; case EM_68HC05: return "Motorola MC68HC05 microcontroller"; case EM_SVX: return "Silicon Graphics SVx"; case EM_ST19: return "STMicroelectronics ST19 8-bit mc"; case EM_VAX: return "Digital VAX"; case EM_CRIS: return "Axis Communications 32-bit embedded processor"; case EM_JAVELIN: return "Infineon Tech. 32bit embedded processor"; case EM_FIREPATH: return "Element 14 64-bit DSP Processor"; case EM_ZSP: return "LSI Logic 16-bit DSP Processor"; case EM_MMIX: return "Donald Knuth's educational 64-bit proc"; case EM_HUANY: return "Harvard University MI object files"; case EM_PRISM: return "SiTera Prism"; case EM_AVR: return "Atmel AVR 8-bit microcontroller"; case EM_FR30: return "Fujitsu FR30"; case EM_D10V: return "Mitsubishi D10V"; case EM_D30V: return "Mitsubishi D30V"; case EM_V850: return "NEC v850"; case EM_M32R: return "Mitsubishi M32R"; case EM_MN10300: return "Matsushita MN10300"; case EM_MN10200: return "Matsushita MN10200"; case EM_PJ: return "picoJava"; case EM_OPENRISC: return "OpenRISC 32-bit embedded processor"; case EM_ARC_A5: return "ARC Cores Tangent-A5"; case EM_XTENSA: return "Tensilica Xtensa Architecture"; case EM_VIDEOCORE: return "Alphamosaic VideoCore processor"; case EM_TMM_GPP: return "Thompson Multimedia General Purpose Processor"; case EM_NS32K: return "National Semiconductor 32000 series"; case EM_TPC: return "Tenor Network TPC processor"; case EM_SNP1K: return "Trebia SNP 1000 processor"; case EM_ST200: return "STMicroelectronics ST200 microcontroller"; case EM_IP2K: return "Ubicom IP2xxx microcontroller family"; case EM_MAX: return "MAX Processor"; case EM_CR: return "National Semiconductor CompactRISC microprocessor"; case EM_F2MC16: return "Fujitsu F2MC16"; case EM_MSP430: return "TI embedded microcontroller msp430"; case EM_BLACKFIN: return "Analog Devices Blackfin (DSP) processor"; case EM_SE_C33: return "S1C33 Family of Seiko Epson processors"; case EM_SEP: return "Sharp embedded microprocessor"; case EM_ARCA: return "Arca RISC Microprocessor"; case EM_UNICORE: return "Microprocessor series from PKU-Unity Ltd"; case EM_AARCH64: return "AArch64"; case EM_RISCV: return "RISC-V"; default: snprintf(s_mach, sizeof(s_mach), "", mach); return (s_mach); } } static const char * elf_class(unsigned int class) { static char s_class[32]; switch (class) { case ELFCLASSNONE: return "none"; case ELFCLASS32: return "ELF32"; case ELFCLASS64: return "ELF64"; default: snprintf(s_class, sizeof(s_class), "", class); return (s_class); } } static const char * elf_endian(unsigned int endian) { static char s_endian[32]; switch (endian) { case ELFDATANONE: return "none"; case ELFDATA2LSB: return "2's complement, little endian"; case ELFDATA2MSB: return "2's complement, big endian"; default: snprintf(s_endian, sizeof(s_endian), "", endian); return (s_endian); } } static const char * elf_type(unsigned int type) { static char s_type[32]; switch (type) { case ET_NONE: return "NONE (None)"; case ET_REL: return "REL (Relocatable file)"; case ET_EXEC: return "EXEC (Executable file)"; case ET_DYN: return "DYN (Shared object file)"; case ET_CORE: return "CORE (Core file)"; default: if (type >= ET_LOPROC) snprintf(s_type, sizeof(s_type), "", type); else if (type >= ET_LOOS && type <= ET_HIOS) snprintf(s_type, sizeof(s_type), "", type); else snprintf(s_type, sizeof(s_type), "", type); return (s_type); } } static const char * elf_ver(unsigned int ver) { static char s_ver[32]; switch (ver) { case EV_CURRENT: return "(current)"; case EV_NONE: return "(none)"; default: snprintf(s_ver, sizeof(s_ver), "", ver); return (s_ver); } } static const char * phdr_type(unsigned int mach, unsigned int ptype) { static char s_ptype[32]; if (ptype >= PT_LOPROC && ptype <= PT_HIPROC) { switch (mach) { case EM_ARM: switch (ptype) { case PT_ARM_ARCHEXT: return "ARM_ARCHEXT"; case PT_ARM_EXIDX: return "ARM_EXIDX"; } break; } snprintf(s_ptype, sizeof(s_ptype), "LOPROC+%#x", ptype - PT_LOPROC); return (s_ptype); } switch (ptype) { case PT_NULL: return "NULL"; case PT_LOAD: return "LOAD"; case PT_DYNAMIC: return "DYNAMIC"; case PT_INTERP: return "INTERP"; case PT_NOTE: return "NOTE"; case PT_SHLIB: return "SHLIB"; case PT_PHDR: return "PHDR"; case PT_TLS: return "TLS"; case PT_GNU_EH_FRAME: return "GNU_EH_FRAME"; case PT_GNU_STACK: return "GNU_STACK"; case PT_GNU_RELRO: return "GNU_RELRO"; default: if (ptype >= PT_LOOS && ptype <= PT_HIOS) snprintf(s_ptype, sizeof(s_ptype), "LOOS+%#x", ptype - PT_LOOS); else snprintf(s_ptype, sizeof(s_ptype), "", ptype); return (s_ptype); } } static const char * section_type(unsigned int mach, unsigned int stype) { static char s_stype[32]; if (stype >= SHT_LOPROC && stype <= SHT_HIPROC) { switch (mach) { case EM_ARM: switch (stype) { case SHT_ARM_EXIDX: return "ARM_EXIDX"; case SHT_ARM_PREEMPTMAP: return "ARM_PREEMPTMAP"; case SHT_ARM_ATTRIBUTES: return "ARM_ATTRIBUTES"; case SHT_ARM_DEBUGOVERLAY: return "ARM_DEBUGOVERLAY"; case SHT_ARM_OVERLAYSECTION: return "ARM_OVERLAYSECTION"; } break; case EM_X86_64: switch (stype) { case SHT_X86_64_UNWIND: return "X86_64_UNWIND"; default: break; } break; case EM_MIPS: case EM_MIPS_RS3_LE: switch (stype) { case SHT_MIPS_LIBLIST: return "MIPS_LIBLIST"; case SHT_MIPS_MSYM: return "MIPS_MSYM"; case SHT_MIPS_CONFLICT: return "MIPS_CONFLICT"; case SHT_MIPS_GPTAB: return "MIPS_GPTAB"; case SHT_MIPS_UCODE: return "MIPS_UCODE"; case SHT_MIPS_DEBUG: return "MIPS_DEBUG"; case SHT_MIPS_REGINFO: return "MIPS_REGINFO"; case SHT_MIPS_PACKAGE: return "MIPS_PACKAGE"; case SHT_MIPS_PACKSYM: return "MIPS_PACKSYM"; case SHT_MIPS_RELD: return "MIPS_RELD"; case SHT_MIPS_IFACE: return "MIPS_IFACE"; case SHT_MIPS_CONTENT: return "MIPS_CONTENT"; case SHT_MIPS_OPTIONS: return "MIPS_OPTIONS"; case SHT_MIPS_DELTASYM: return "MIPS_DELTASYM"; case SHT_MIPS_DELTAINST: return "MIPS_DELTAINST"; case SHT_MIPS_DELTACLASS: return "MIPS_DELTACLASS"; case SHT_MIPS_DWARF: return "MIPS_DWARF"; case SHT_MIPS_DELTADECL: return "MIPS_DELTADECL"; case SHT_MIPS_SYMBOL_LIB: return "MIPS_SYMBOL_LIB"; case SHT_MIPS_EVENTS: return "MIPS_EVENTS"; case SHT_MIPS_TRANSLATE: return "MIPS_TRANSLATE"; case SHT_MIPS_PIXIE: return "MIPS_PIXIE"; case SHT_MIPS_XLATE: return "MIPS_XLATE"; case SHT_MIPS_XLATE_DEBUG: return "MIPS_XLATE_DEBUG"; case SHT_MIPS_WHIRL: return "MIPS_WHIRL"; case SHT_MIPS_EH_REGION: return "MIPS_EH_REGION"; case SHT_MIPS_XLATE_OLD: return "MIPS_XLATE_OLD"; case SHT_MIPS_PDR_EXCEPTION: return "MIPS_PDR_EXCEPTION"; default: break; } break; default: break; } snprintf(s_stype, sizeof(s_stype), "LOPROC+%#x", stype - SHT_LOPROC); return (s_stype); } switch (stype) { case SHT_NULL: return "NULL"; case SHT_PROGBITS: return "PROGBITS"; case SHT_SYMTAB: return "SYMTAB"; case SHT_STRTAB: return "STRTAB"; case SHT_RELA: return "RELA"; case SHT_HASH: return "HASH"; case SHT_DYNAMIC: return "DYNAMIC"; case SHT_NOTE: return "NOTE"; case SHT_NOBITS: return "NOBITS"; case SHT_REL: return "REL"; case SHT_SHLIB: return "SHLIB"; case SHT_DYNSYM: return "DYNSYM"; case SHT_INIT_ARRAY: return "INIT_ARRAY"; case SHT_FINI_ARRAY: return "FINI_ARRAY"; case SHT_PREINIT_ARRAY: return "PREINIT_ARRAY"; case SHT_GROUP: return "GROUP"; case SHT_SYMTAB_SHNDX: return "SYMTAB_SHNDX"; case SHT_SUNW_dof: return "SUNW_dof"; case SHT_SUNW_cap: return "SUNW_cap"; case SHT_GNU_HASH: return "GNU_HASH"; case SHT_SUNW_ANNOTATE: return "SUNW_ANNOTATE"; case SHT_SUNW_DEBUGSTR: return "SUNW_DEBUGSTR"; case SHT_SUNW_DEBUG: return "SUNW_DEBUG"; case SHT_SUNW_move: return "SUNW_move"; case SHT_SUNW_COMDAT: return "SUNW_COMDAT"; case SHT_SUNW_syminfo: return "SUNW_syminfo"; case SHT_SUNW_verdef: return "SUNW_verdef"; case SHT_SUNW_verneed: return "SUNW_verneed"; case SHT_SUNW_versym: return "SUNW_versym"; default: if (stype >= SHT_LOOS && stype <= SHT_HIOS) snprintf(s_stype, sizeof(s_stype), "LOOS+%#x", stype - SHT_LOOS); else if (stype >= SHT_LOUSER) snprintf(s_stype, sizeof(s_stype), "LOUSER+%#x", stype - SHT_LOUSER); else snprintf(s_stype, sizeof(s_stype), "", stype); return (s_stype); } } static const char * dt_type(unsigned int mach, unsigned int dtype) { static char s_dtype[32]; if (dtype >= DT_LOPROC && dtype <= DT_HIPROC) { switch (mach) { case EM_ARM: switch (dtype) { case DT_ARM_SYMTABSZ: return "ARM_SYMTABSZ"; default: break; } break; case EM_MIPS: case EM_MIPS_RS3_LE: switch (dtype) { case DT_MIPS_RLD_VERSION: return "MIPS_RLD_VERSION"; case DT_MIPS_TIME_STAMP: return "MIPS_TIME_STAMP"; case DT_MIPS_ICHECKSUM: return "MIPS_ICHECKSUM"; case DT_MIPS_IVERSION: return "MIPS_IVERSION"; case DT_MIPS_FLAGS: return "MIPS_FLAGS"; case DT_MIPS_BASE_ADDRESS: return "MIPS_BASE_ADDRESS"; case DT_MIPS_CONFLICT: return "MIPS_CONFLICT"; case DT_MIPS_LIBLIST: return "MIPS_LIBLIST"; case DT_MIPS_LOCAL_GOTNO: return "MIPS_LOCAL_GOTNO"; case DT_MIPS_CONFLICTNO: return "MIPS_CONFLICTNO"; case DT_MIPS_LIBLISTNO: return "MIPS_LIBLISTNO"; case DT_MIPS_SYMTABNO: return "MIPS_SYMTABNO"; case DT_MIPS_UNREFEXTNO: return "MIPS_UNREFEXTNO"; case DT_MIPS_GOTSYM: return "MIPS_GOTSYM"; case DT_MIPS_HIPAGENO: return "MIPS_HIPAGENO"; case DT_MIPS_RLD_MAP: return "MIPS_RLD_MAP"; case DT_MIPS_DELTA_CLASS: return "MIPS_DELTA_CLASS"; case DT_MIPS_DELTA_CLASS_NO: return "MIPS_DELTA_CLASS_NO"; case DT_MIPS_DELTA_INSTANCE: return "MIPS_DELTA_INSTANCE"; case DT_MIPS_DELTA_INSTANCE_NO: return "MIPS_DELTA_INSTANCE_NO"; case DT_MIPS_DELTA_RELOC: return "MIPS_DELTA_RELOC"; case DT_MIPS_DELTA_RELOC_NO: return "MIPS_DELTA_RELOC_NO"; case DT_MIPS_DELTA_SYM: return "MIPS_DELTA_SYM"; case DT_MIPS_DELTA_SYM_NO: return "MIPS_DELTA_SYM_NO"; case DT_MIPS_DELTA_CLASSSYM: return "MIPS_DELTA_CLASSSYM"; case DT_MIPS_DELTA_CLASSSYM_NO: return "MIPS_DELTA_CLASSSYM_NO"; case DT_MIPS_CXX_FLAGS: return "MIPS_CXX_FLAGS"; case DT_MIPS_PIXIE_INIT: return "MIPS_PIXIE_INIT"; case DT_MIPS_SYMBOL_LIB: return "MIPS_SYMBOL_LIB"; case DT_MIPS_LOCALPAGE_GOTIDX: return "MIPS_LOCALPAGE_GOTIDX"; case DT_MIPS_LOCAL_GOTIDX: return "MIPS_LOCAL_GOTIDX"; case DT_MIPS_HIDDEN_GOTIDX: return "MIPS_HIDDEN_GOTIDX"; case DT_MIPS_PROTECTED_GOTIDX: return "MIPS_PROTECTED_GOTIDX"; case DT_MIPS_OPTIONS: return "MIPS_OPTIONS"; case DT_MIPS_INTERFACE: return "MIPS_INTERFACE"; case DT_MIPS_DYNSTR_ALIGN: return "MIPS_DYNSTR_ALIGN"; case DT_MIPS_INTERFACE_SIZE: return "MIPS_INTERFACE_SIZE"; case DT_MIPS_RLD_TEXT_RESOLVE_ADDR: return "MIPS_RLD_TEXT_RESOLVE_ADDR"; case DT_MIPS_PERF_SUFFIX: return "MIPS_PERF_SUFFIX"; case DT_MIPS_COMPACT_SIZE: return "MIPS_COMPACT_SIZE"; case DT_MIPS_GP_VALUE: return "MIPS_GP_VALUE"; case DT_MIPS_AUX_DYNAMIC: return "MIPS_AUX_DYNAMIC"; case DT_MIPS_PLTGOT: return "MIPS_PLTGOT"; case DT_MIPS_RLD_OBJ_UPDATE: return "MIPS_RLD_OBJ_UPDATE"; case DT_MIPS_RWPLT: return "MIPS_RWPLT"; default: break; } break; case EM_SPARC: case EM_SPARC32PLUS: case EM_SPARCV9: switch (dtype) { case DT_SPARC_REGISTER: return "DT_SPARC_REGISTER"; default: break; } break; default: break; } snprintf(s_dtype, sizeof(s_dtype), "", dtype); return (s_dtype); } switch (dtype) { case DT_NULL: return "NULL"; case DT_NEEDED: return "NEEDED"; case DT_PLTRELSZ: return "PLTRELSZ"; case DT_PLTGOT: return "PLTGOT"; case DT_HASH: return "HASH"; case DT_STRTAB: return "STRTAB"; case DT_SYMTAB: return "SYMTAB"; case DT_RELA: return "RELA"; case DT_RELASZ: return "RELASZ"; case DT_RELAENT: return "RELAENT"; case DT_STRSZ: return "STRSZ"; case DT_SYMENT: return "SYMENT"; case DT_INIT: return "INIT"; case DT_FINI: return "FINI"; case DT_SONAME: return "SONAME"; case DT_RPATH: return "RPATH"; case DT_SYMBOLIC: return "SYMBOLIC"; case DT_REL: return "REL"; case DT_RELSZ: return "RELSZ"; case DT_RELENT: return "RELENT"; case DT_PLTREL: return "PLTREL"; case DT_DEBUG: return "DEBUG"; case DT_TEXTREL: return "TEXTREL"; case DT_JMPREL: return "JMPREL"; case DT_BIND_NOW: return "BIND_NOW"; case DT_INIT_ARRAY: return "INIT_ARRAY"; case DT_FINI_ARRAY: return "FINI_ARRAY"; case DT_INIT_ARRAYSZ: return "INIT_ARRAYSZ"; case DT_FINI_ARRAYSZ: return "FINI_ARRAYSZ"; case DT_RUNPATH: return "RUNPATH"; case DT_FLAGS: return "FLAGS"; case DT_PREINIT_ARRAY: return "PREINIT_ARRAY"; case DT_PREINIT_ARRAYSZ: return "PREINIT_ARRAYSZ"; case DT_MAXPOSTAGS: return "MAXPOSTAGS"; case DT_SUNW_AUXILIARY: return "SUNW_AUXILIARY"; case DT_SUNW_RTLDINF: return "SUNW_RTLDINF"; case DT_SUNW_FILTER: return "SUNW_FILTER"; case DT_SUNW_CAP: return "SUNW_CAP"; case DT_CHECKSUM: return "CHECKSUM"; case DT_PLTPADSZ: return "PLTPADSZ"; case DT_MOVEENT: return "MOVEENT"; case DT_MOVESZ: return "MOVESZ"; case DT_FEATURE: return "FEATURE"; case DT_POSFLAG_1: return "POSFLAG_1"; case DT_SYMINSZ: return "SYMINSZ"; case DT_SYMINENT: return "SYMINENT"; case DT_GNU_HASH: return "GNU_HASH"; case DT_TLSDESC_PLT: return "DT_TLSDESC_PLT"; case DT_TLSDESC_GOT: return "DT_TLSDESC_GOT"; case DT_GNU_CONFLICT: return "GNU_CONFLICT"; case DT_GNU_LIBLIST: return "GNU_LIBLIST"; case DT_CONFIG: return "CONFIG"; case DT_DEPAUDIT: return "DEPAUDIT"; case DT_AUDIT: return "AUDIT"; case DT_PLTPAD: return "PLTPAD"; case DT_MOVETAB: return "MOVETAB"; case DT_SYMINFO: return "SYMINFO"; case DT_VERSYM: return "VERSYM"; case DT_RELACOUNT: return "RELACOUNT"; case DT_RELCOUNT: return "RELCOUNT"; case DT_FLAGS_1: return "FLAGS_1"; case DT_VERDEF: return "VERDEF"; case DT_VERDEFNUM: return "VERDEFNUM"; case DT_VERNEED: return "VERNEED"; case DT_VERNEEDNUM: return "VERNEEDNUM"; case DT_AUXILIARY: return "AUXILIARY"; case DT_USED: return "USED"; case DT_FILTER: return "FILTER"; case DT_GNU_PRELINKED: return "GNU_PRELINKED"; case DT_GNU_CONFLICTSZ: return "GNU_CONFLICTSZ"; case DT_GNU_LIBLISTSZ: return "GNU_LIBLISTSZ"; default: snprintf(s_dtype, sizeof(s_dtype), "", dtype); return (s_dtype); } } static const char * st_bind(unsigned int sbind) { static char s_sbind[32]; switch (sbind) { case STB_LOCAL: return "LOCAL"; case STB_GLOBAL: return "GLOBAL"; case STB_WEAK: return "WEAK"; case STB_GNU_UNIQUE: return "UNIQUE"; default: if (sbind >= STB_LOOS && sbind <= STB_HIOS) return "OS"; else if (sbind >= STB_LOPROC && sbind <= STB_HIPROC) return "PROC"; else snprintf(s_sbind, sizeof(s_sbind), "", sbind); return (s_sbind); } } static const char * st_type(unsigned int mach, unsigned int os, unsigned int stype) { static char s_stype[32]; switch (stype) { case STT_NOTYPE: return "NOTYPE"; case STT_OBJECT: return "OBJECT"; case STT_FUNC: return "FUNC"; case STT_SECTION: return "SECTION"; case STT_FILE: return "FILE"; case STT_COMMON: return "COMMON"; case STT_TLS: return "TLS"; default: if (stype >= STT_LOOS && stype <= STT_HIOS) { if ((os == ELFOSABI_GNU || os == ELFOSABI_FREEBSD) && stype == STT_GNU_IFUNC) return "IFUNC"; snprintf(s_stype, sizeof(s_stype), "OS+%#x", stype - STT_LOOS); } else if (stype >= STT_LOPROC && stype <= STT_HIPROC) { if (mach == EM_SPARCV9 && stype == STT_SPARC_REGISTER) return "REGISTER"; snprintf(s_stype, sizeof(s_stype), "PROC+%#x", stype - STT_LOPROC); } else snprintf(s_stype, sizeof(s_stype), "", stype); return (s_stype); } } static const char * st_vis(unsigned int svis) { static char s_svis[32]; switch(svis) { case STV_DEFAULT: return "DEFAULT"; case STV_INTERNAL: return "INTERNAL"; case STV_HIDDEN: return "HIDDEN"; case STV_PROTECTED: return "PROTECTED"; default: snprintf(s_svis, sizeof(s_svis), "", svis); return (s_svis); } } static const char * st_shndx(unsigned int shndx) { static char s_shndx[32]; switch (shndx) { case SHN_UNDEF: return "UND"; case SHN_ABS: return "ABS"; case SHN_COMMON: return "COM"; default: if (shndx >= SHN_LOPROC && shndx <= SHN_HIPROC) return "PRC"; else if (shndx >= SHN_LOOS && shndx <= SHN_HIOS) return "OS"; else snprintf(s_shndx, sizeof(s_shndx), "%u", shndx); return (s_shndx); } } static struct { const char *ln; char sn; int value; } section_flag[] = { {"WRITE", 'W', SHF_WRITE}, {"ALLOC", 'A', SHF_ALLOC}, {"EXEC", 'X', SHF_EXECINSTR}, {"MERGE", 'M', SHF_MERGE}, {"STRINGS", 'S', SHF_STRINGS}, {"INFO LINK", 'I', SHF_INFO_LINK}, {"OS NONCONF", 'O', SHF_OS_NONCONFORMING}, {"GROUP", 'G', SHF_GROUP}, {"TLS", 'T', SHF_TLS}, {"COMPRESSED", 'C', SHF_COMPRESSED}, {NULL, 0, 0} }; static const char * note_type(const char *name, unsigned int et, unsigned int nt) { if ((strcmp(name, "CORE") == 0 || strcmp(name, "LINUX") == 0) && et == ET_CORE) return note_type_linux_core(nt); else if (strcmp(name, "FreeBSD") == 0) if (et == ET_CORE) return note_type_freebsd_core(nt); else return note_type_freebsd(nt); else if (strcmp(name, "GNU") == 0 && et != ET_CORE) return note_type_gnu(nt); else if (strcmp(name, "NetBSD") == 0 && et != ET_CORE) return note_type_netbsd(nt); else if (strcmp(name, "OpenBSD") == 0 && et != ET_CORE) return note_type_openbsd(nt); else if (strcmp(name, "Xen") == 0 && et != ET_CORE) return note_type_xen(nt); return note_type_unknown(nt); } static const char * note_type_freebsd(unsigned int nt) { switch (nt) { case 1: return "NT_FREEBSD_ABI_TAG"; case 2: return "NT_FREEBSD_NOINIT_TAG"; case 3: return "NT_FREEBSD_ARCH_TAG"; default: return (note_type_unknown(nt)); } } static const char * note_type_freebsd_core(unsigned int nt) { switch (nt) { case 1: return "NT_PRSTATUS"; case 2: return "NT_FPREGSET"; case 3: return "NT_PRPSINFO"; case 7: return "NT_THRMISC"; case 8: return "NT_PROCSTAT_PROC"; case 9: return "NT_PROCSTAT_FILES"; case 10: return "NT_PROCSTAT_VMMAP"; case 11: return "NT_PROCSTAT_GROUPS"; case 12: return "NT_PROCSTAT_UMASK"; case 13: return "NT_PROCSTAT_RLIMIT"; case 14: return "NT_PROCSTAT_OSREL"; case 15: return "NT_PROCSTAT_PSSTRINGS"; case 16: return "NT_PROCSTAT_AUXV"; case 0x202: return "NT_X86_XSTATE (x86 XSAVE extended state)"; default: return (note_type_unknown(nt)); } } static const char * note_type_linux_core(unsigned int nt) { switch (nt) { case 1: return "NT_PRSTATUS (Process status)"; case 2: return "NT_FPREGSET (Floating point information)"; case 3: return "NT_PRPSINFO (Process information)"; case 4: return "NT_TASKSTRUCT (Task structure)"; case 6: return "NT_AUXV (Auxiliary vector)"; case 10: return "NT_PSTATUS (Linux process status)"; case 12: return "NT_FPREGS (Linux floating point regset)"; case 13: return "NT_PSINFO (Linux process information)"; case 16: return "NT_LWPSTATUS (Linux lwpstatus_t type)"; case 17: return "NT_LWPSINFO (Linux lwpinfo_t type)"; case 18: return "NT_WIN32PSTATUS (win32_pstatus structure)"; case 0x100: return "NT_PPC_VMX (ppc Altivec registers)"; case 0x102: return "NT_PPC_VSX (ppc VSX registers)"; case 0x202: return "NT_X86_XSTATE (x86 XSAVE extended state)"; case 0x300: return "NT_S390_HIGH_GPRS (s390 upper register halves)"; case 0x301: return "NT_S390_TIMER (s390 timer register)"; case 0x302: return "NT_S390_TODCMP (s390 TOD comparator register)"; case 0x303: return "NT_S390_TODPREG (s390 TOD programmable register)"; case 0x304: return "NT_S390_CTRS (s390 control registers)"; case 0x305: return "NT_S390_PREFIX (s390 prefix register)"; case 0x400: return "NT_ARM_VFP (arm VFP registers)"; case 0x46494c45UL: return "NT_FILE (mapped files)"; case 0x46E62B7FUL: return "NT_PRXFPREG (Linux user_xfpregs structure)"; case 0x53494749UL: return "NT_SIGINFO (siginfo_t data)"; default: return (note_type_unknown(nt)); } } static const char * note_type_gnu(unsigned int nt) { switch (nt) { case 1: return "NT_GNU_ABI_TAG"; case 2: return "NT_GNU_HWCAP (Hardware capabilities)"; case 3: return "NT_GNU_BUILD_ID (Build id set by ld(1))"; case 4: return "NT_GNU_GOLD_VERSION (GNU gold version)"; default: return (note_type_unknown(nt)); } } static const char * note_type_netbsd(unsigned int nt) { switch (nt) { case 1: return "NT_NETBSD_IDENT"; default: return (note_type_unknown(nt)); } } static const char * note_type_openbsd(unsigned int nt) { switch (nt) { case 1: return "NT_OPENBSD_IDENT"; default: return (note_type_unknown(nt)); } } static const char * note_type_unknown(unsigned int nt) { static char s_nt[32]; snprintf(s_nt, sizeof(s_nt), nt >= 0x100 ? "" : "", nt); return (s_nt); } static const char * note_type_xen(unsigned int nt) { switch (nt) { case 0: return "XEN_ELFNOTE_INFO"; case 1: return "XEN_ELFNOTE_ENTRY"; case 2: return "XEN_ELFNOTE_HYPERCALL_PAGE"; case 3: return "XEN_ELFNOTE_VIRT_BASE"; case 4: return "XEN_ELFNOTE_PADDR_OFFSET"; case 5: return "XEN_ELFNOTE_XEN_VERSION"; case 6: return "XEN_ELFNOTE_GUEST_OS"; case 7: return "XEN_ELFNOTE_GUEST_VERSION"; case 8: return "XEN_ELFNOTE_LOADER"; case 9: return "XEN_ELFNOTE_PAE_MODE"; case 10: return "XEN_ELFNOTE_FEATURES"; case 11: return "XEN_ELFNOTE_BSD_SYMTAB"; case 12: return "XEN_ELFNOTE_HV_START_LOW"; case 13: return "XEN_ELFNOTE_L1_MFN_VALID"; case 14: return "XEN_ELFNOTE_SUSPEND_CANCEL"; case 15: return "XEN_ELFNOTE_INIT_P2M"; case 16: return "XEN_ELFNOTE_MOD_START_PFN"; case 17: return "XEN_ELFNOTE_SUPPORTED_FEATURES"; default: return (note_type_unknown(nt)); } } static struct { const char *name; int value; } l_flag[] = { {"EXACT_MATCH", LL_EXACT_MATCH}, {"IGNORE_INT_VER", LL_IGNORE_INT_VER}, {"REQUIRE_MINOR", LL_REQUIRE_MINOR}, {"EXPORTS", LL_EXPORTS}, {"DELAY_LOAD", LL_DELAY_LOAD}, {"DELTA", LL_DELTA}, {NULL, 0} }; static struct mips_option mips_exceptions_option[] = { {OEX_PAGE0, "PAGE0"}, {OEX_SMM, "SMM"}, {OEX_PRECISEFP, "PRECISEFP"}, {OEX_DISMISS, "DISMISS"}, {0, NULL} }; static struct mips_option mips_pad_option[] = { {OPAD_PREFIX, "PREFIX"}, {OPAD_POSTFIX, "POSTFIX"}, {OPAD_SYMBOL, "SYMBOL"}, {0, NULL} }; static struct mips_option mips_hwpatch_option[] = { {OHW_R4KEOP, "R4KEOP"}, {OHW_R8KPFETCH, "R8KPFETCH"}, {OHW_R5KEOP, "R5KEOP"}, {OHW_R5KCVTL, "R5KCVTL"}, {0, NULL} }; static struct mips_option mips_hwa_option[] = { {OHWA0_R4KEOP_CHECKED, "R4KEOP_CHECKED"}, {OHWA0_R4KEOP_CLEAN, "R4KEOP_CLEAN"}, {0, NULL} }; static struct mips_option mips_hwo_option[] = { {OHWO0_FIXADE, "FIXADE"}, {0, NULL} }; static const char * option_kind(uint8_t kind) { static char s_kind[32]; switch (kind) { case ODK_NULL: return "NULL"; case ODK_REGINFO: return "REGINFO"; case ODK_EXCEPTIONS: return "EXCEPTIONS"; case ODK_PAD: return "PAD"; case ODK_HWPATCH: return "HWPATCH"; case ODK_FILL: return "FILL"; case ODK_TAGS: return "TAGS"; case ODK_HWAND: return "HWAND"; case ODK_HWOR: return "HWOR"; case ODK_GP_GROUP: return "GP_GROUP"; case ODK_IDENT: return "IDENT"; default: snprintf(s_kind, sizeof(s_kind), "", kind); return (s_kind); } } static const char * top_tag(unsigned int tag) { static char s_top_tag[32]; switch (tag) { case 1: return "File Attributes"; case 2: return "Section Attributes"; case 3: return "Symbol Attributes"; default: snprintf(s_top_tag, sizeof(s_top_tag), "Unknown tag: %u", tag); return (s_top_tag); } } static const char * aeabi_cpu_arch(uint64_t arch) { static char s_cpu_arch[32]; switch (arch) { case 0: return "Pre-V4"; case 1: return "ARM v4"; case 2: return "ARM v4T"; case 3: return "ARM v5T"; case 4: return "ARM v5TE"; case 5: return "ARM v5TEJ"; case 6: return "ARM v6"; case 7: return "ARM v6KZ"; case 8: return "ARM v6T2"; case 9: return "ARM v6K"; case 10: return "ARM v7"; case 11: return "ARM v6-M"; case 12: return "ARM v6S-M"; case 13: return "ARM v7E-M"; default: snprintf(s_cpu_arch, sizeof(s_cpu_arch), "Unknown (%ju)", (uintmax_t) arch); return (s_cpu_arch); } } static const char * aeabi_cpu_arch_profile(uint64_t pf) { static char s_arch_profile[32]; switch (pf) { case 0: return "Not applicable"; case 0x41: /* 'A' */ return "Application Profile"; case 0x52: /* 'R' */ return "Real-Time Profile"; case 0x4D: /* 'M' */ return "Microcontroller Profile"; case 0x53: /* 'S' */ return "Application or Real-Time Profile"; default: snprintf(s_arch_profile, sizeof(s_arch_profile), "Unknown (%ju)\n", (uintmax_t) pf); return (s_arch_profile); } } static const char * aeabi_arm_isa(uint64_t ai) { static char s_ai[32]; switch (ai) { case 0: return "No"; case 1: return "Yes"; default: snprintf(s_ai, sizeof(s_ai), "Unknown (%ju)\n", (uintmax_t) ai); return (s_ai); } } static const char * aeabi_thumb_isa(uint64_t ti) { static char s_ti[32]; switch (ti) { case 0: return "No"; case 1: return "16-bit Thumb"; case 2: return "32-bit Thumb"; default: snprintf(s_ti, sizeof(s_ti), "Unknown (%ju)\n", (uintmax_t) ti); return (s_ti); } } static const char * aeabi_fp_arch(uint64_t fp) { static char s_fp_arch[32]; switch (fp) { case 0: return "No"; case 1: return "VFPv1"; case 2: return "VFPv2"; case 3: return "VFPv3"; case 4: return "VFPv3-D16"; case 5: return "VFPv4"; case 6: return "VFPv4-D16"; default: snprintf(s_fp_arch, sizeof(s_fp_arch), "Unknown (%ju)", (uintmax_t) fp); return (s_fp_arch); } } static const char * aeabi_wmmx_arch(uint64_t wmmx) { static char s_wmmx[32]; switch (wmmx) { case 0: return "No"; case 1: return "WMMXv1"; case 2: return "WMMXv2"; default: snprintf(s_wmmx, sizeof(s_wmmx), "Unknown (%ju)", (uintmax_t) wmmx); return (s_wmmx); } } static const char * aeabi_adv_simd_arch(uint64_t simd) { static char s_simd[32]; switch (simd) { case 0: return "No"; case 1: return "NEONv1"; case 2: return "NEONv2"; default: snprintf(s_simd, sizeof(s_simd), "Unknown (%ju)", (uintmax_t) simd); return (s_simd); } } static const char * aeabi_pcs_config(uint64_t pcs) { static char s_pcs[32]; switch (pcs) { case 0: return "None"; case 1: return "Bare platform"; case 2: return "Linux"; case 3: return "Linux DSO"; case 4: return "Palm OS 2004"; case 5: return "Palm OS (future)"; case 6: return "Symbian OS 2004"; case 7: return "Symbian OS (future)"; default: snprintf(s_pcs, sizeof(s_pcs), "Unknown (%ju)", (uintmax_t) pcs); return (s_pcs); } } static const char * aeabi_pcs_r9(uint64_t r9) { static char s_r9[32]; switch (r9) { case 0: return "V6"; case 1: return "SB"; case 2: return "TLS pointer"; case 3: return "Unused"; default: snprintf(s_r9, sizeof(s_r9), "Unknown (%ju)", (uintmax_t) r9); return (s_r9); } } static const char * aeabi_pcs_rw(uint64_t rw) { static char s_rw[32]; switch (rw) { case 0: return "Absolute"; case 1: return "PC-relative"; case 2: return "SB-relative"; case 3: return "None"; default: snprintf(s_rw, sizeof(s_rw), "Unknown (%ju)", (uintmax_t) rw); return (s_rw); } } static const char * aeabi_pcs_ro(uint64_t ro) { static char s_ro[32]; switch (ro) { case 0: return "Absolute"; case 1: return "PC-relative"; case 2: return "None"; default: snprintf(s_ro, sizeof(s_ro), "Unknown (%ju)", (uintmax_t) ro); return (s_ro); } } static const char * aeabi_pcs_got(uint64_t got) { static char s_got[32]; switch (got) { case 0: return "None"; case 1: return "direct"; case 2: return "indirect via GOT"; default: snprintf(s_got, sizeof(s_got), "Unknown (%ju)", (uintmax_t) got); return (s_got); } } static const char * aeabi_pcs_wchar_t(uint64_t wt) { static char s_wt[32]; switch (wt) { case 0: return "None"; case 2: return "wchar_t size 2"; case 4: return "wchar_t size 4"; default: snprintf(s_wt, sizeof(s_wt), "Unknown (%ju)", (uintmax_t) wt); return (s_wt); } } static const char * aeabi_enum_size(uint64_t es) { static char s_es[32]; switch (es) { case 0: return "None"; case 1: return "smallest"; case 2: return "32-bit"; case 3: return "visible 32-bit"; default: snprintf(s_es, sizeof(s_es), "Unknown (%ju)", (uintmax_t) es); return (s_es); } } static const char * aeabi_align_needed(uint64_t an) { static char s_align_n[64]; switch (an) { case 0: return "No"; case 1: return "8-byte align"; case 2: return "4-byte align"; case 3: return "Reserved"; default: if (an >= 4 && an <= 12) snprintf(s_align_n, sizeof(s_align_n), "8-byte align" " and up to 2^%ju-byte extended align", (uintmax_t) an); else snprintf(s_align_n, sizeof(s_align_n), "Unknown (%ju)", (uintmax_t) an); return (s_align_n); } } static const char * aeabi_align_preserved(uint64_t ap) { static char s_align_p[128]; switch (ap) { case 0: return "No"; case 1: return "8-byte align"; case 2: return "8-byte align and SP % 8 == 0"; case 3: return "Reserved"; default: if (ap >= 4 && ap <= 12) snprintf(s_align_p, sizeof(s_align_p), "8-byte align" " and SP %% 8 == 0 and up to 2^%ju-byte extended" " align", (uintmax_t) ap); else snprintf(s_align_p, sizeof(s_align_p), "Unknown (%ju)", (uintmax_t) ap); return (s_align_p); } } static const char * aeabi_fp_rounding(uint64_t fr) { static char s_fp_r[32]; switch (fr) { case 0: return "Unused"; case 1: return "Needed"; default: snprintf(s_fp_r, sizeof(s_fp_r), "Unknown (%ju)", (uintmax_t) fr); return (s_fp_r); } } static const char * aeabi_fp_denormal(uint64_t fd) { static char s_fp_d[32]; switch (fd) { case 0: return "Unused"; case 1: return "Needed"; case 2: return "Sign Only"; default: snprintf(s_fp_d, sizeof(s_fp_d), "Unknown (%ju)", (uintmax_t) fd); return (s_fp_d); } } static const char * aeabi_fp_exceptions(uint64_t fe) { static char s_fp_e[32]; switch (fe) { case 0: return "Unused"; case 1: return "Needed"; default: snprintf(s_fp_e, sizeof(s_fp_e), "Unknown (%ju)", (uintmax_t) fe); return (s_fp_e); } } static const char * aeabi_fp_user_exceptions(uint64_t fu) { static char s_fp_u[32]; switch (fu) { case 0: return "Unused"; case 1: return "Needed"; default: snprintf(s_fp_u, sizeof(s_fp_u), "Unknown (%ju)", (uintmax_t) fu); return (s_fp_u); } } static const char * aeabi_fp_number_model(uint64_t fn) { static char s_fp_n[32]; switch (fn) { case 0: return "Unused"; case 1: return "IEEE 754 normal"; case 2: return "RTABI"; case 3: return "IEEE 754"; default: snprintf(s_fp_n, sizeof(s_fp_n), "Unknown (%ju)", (uintmax_t) fn); return (s_fp_n); } } static const char * aeabi_fp_16bit_format(uint64_t fp16) { static char s_fp_16[64]; switch (fp16) { case 0: return "None"; case 1: return "IEEE 754"; case 2: return "VFPv3/Advanced SIMD (alternative format)"; default: snprintf(s_fp_16, sizeof(s_fp_16), "Unknown (%ju)", (uintmax_t) fp16); return (s_fp_16); } } static const char * aeabi_mpext(uint64_t mp) { static char s_mp[32]; switch (mp) { case 0: return "Not allowed"; case 1: return "Allowed"; default: snprintf(s_mp, sizeof(s_mp), "Unknown (%ju)", (uintmax_t) mp); return (s_mp); } } static const char * aeabi_div(uint64_t du) { static char s_du[32]; switch (du) { case 0: return "Yes (V7-R/V7-M)"; case 1: return "No"; case 2: return "Yes (V7-A)"; default: snprintf(s_du, sizeof(s_du), "Unknown (%ju)", (uintmax_t) du); return (s_du); } } static const char * aeabi_t2ee(uint64_t t2ee) { static char s_t2ee[32]; switch (t2ee) { case 0: return "Not allowed"; case 1: return "Allowed"; default: snprintf(s_t2ee, sizeof(s_t2ee), "Unknown(%ju)", (uintmax_t) t2ee); return (s_t2ee); } } static const char * aeabi_hardfp(uint64_t hfp) { static char s_hfp[32]; switch (hfp) { case 0: return "Tag_FP_arch"; case 1: return "only SP"; case 2: return "only DP"; case 3: return "both SP and DP"; default: snprintf(s_hfp, sizeof(s_hfp), "Unknown (%ju)", (uintmax_t) hfp); return (s_hfp); } } static const char * aeabi_vfp_args(uint64_t va) { static char s_va[32]; switch (va) { case 0: return "AAPCS (base variant)"; case 1: return "AAPCS (VFP variant)"; case 2: return "toolchain-specific"; default: snprintf(s_va, sizeof(s_va), "Unknown (%ju)", (uintmax_t) va); return (s_va); } } static const char * aeabi_wmmx_args(uint64_t wa) { static char s_wa[32]; switch (wa) { case 0: return "AAPCS (base variant)"; case 1: return "Intel WMMX"; case 2: return "toolchain-specific"; default: snprintf(s_wa, sizeof(s_wa), "Unknown(%ju)", (uintmax_t) wa); return (s_wa); } } static const char * aeabi_unaligned_access(uint64_t ua) { static char s_ua[32]; switch (ua) { case 0: return "Not allowed"; case 1: return "Allowed"; default: snprintf(s_ua, sizeof(s_ua), "Unknown(%ju)", (uintmax_t) ua); return (s_ua); } } static const char * aeabi_fp_hpext(uint64_t fh) { static char s_fh[32]; switch (fh) { case 0: return "Not allowed"; case 1: return "Allowed"; default: snprintf(s_fh, sizeof(s_fh), "Unknown(%ju)", (uintmax_t) fh); return (s_fh); } } static const char * aeabi_optm_goal(uint64_t og) { static char s_og[32]; switch (og) { case 0: return "None"; case 1: return "Speed"; case 2: return "Speed aggressive"; case 3: return "Space"; case 4: return "Space aggressive"; case 5: return "Debugging"; case 6: return "Best Debugging"; default: snprintf(s_og, sizeof(s_og), "Unknown(%ju)", (uintmax_t) og); return (s_og); } } static const char * aeabi_fp_optm_goal(uint64_t fog) { static char s_fog[32]; switch (fog) { case 0: return "None"; case 1: return "Speed"; case 2: return "Speed aggressive"; case 3: return "Space"; case 4: return "Space aggressive"; case 5: return "Accurary"; case 6: return "Best Accurary"; default: snprintf(s_fog, sizeof(s_fog), "Unknown(%ju)", (uintmax_t) fog); return (s_fog); } } static const char * aeabi_virtual(uint64_t vt) { static char s_virtual[64]; switch (vt) { case 0: return "No"; case 1: return "TrustZone"; case 2: return "Virtualization extension"; case 3: return "TrustZone and virtualization extension"; default: snprintf(s_virtual, sizeof(s_virtual), "Unknown(%ju)", (uintmax_t) vt); return (s_virtual); } } static struct { uint64_t tag; const char *s_tag; const char *(*get_desc)(uint64_t val); } aeabi_tags[] = { {4, "Tag_CPU_raw_name", NULL}, {5, "Tag_CPU_name", NULL}, {6, "Tag_CPU_arch", aeabi_cpu_arch}, {7, "Tag_CPU_arch_profile", aeabi_cpu_arch_profile}, {8, "Tag_ARM_ISA_use", aeabi_arm_isa}, {9, "Tag_THUMB_ISA_use", aeabi_thumb_isa}, {10, "Tag_FP_arch", aeabi_fp_arch}, {11, "Tag_WMMX_arch", aeabi_wmmx_arch}, {12, "Tag_Advanced_SIMD_arch", aeabi_adv_simd_arch}, {13, "Tag_PCS_config", aeabi_pcs_config}, {14, "Tag_ABI_PCS_R9_use", aeabi_pcs_r9}, {15, "Tag_ABI_PCS_RW_data", aeabi_pcs_rw}, {16, "Tag_ABI_PCS_RO_data", aeabi_pcs_ro}, {17, "Tag_ABI_PCS_GOT_use", aeabi_pcs_got}, {18, "Tag_ABI_PCS_wchar_t", aeabi_pcs_wchar_t}, {19, "Tag_ABI_FP_rounding", aeabi_fp_rounding}, {20, "Tag_ABI_FP_denormal", aeabi_fp_denormal}, {21, "Tag_ABI_FP_exceptions", aeabi_fp_exceptions}, {22, "Tag_ABI_FP_user_exceptions", aeabi_fp_user_exceptions}, {23, "Tag_ABI_FP_number_model", aeabi_fp_number_model}, {24, "Tag_ABI_align_needed", aeabi_align_needed}, {25, "Tag_ABI_align_preserved", aeabi_align_preserved}, {26, "Tag_ABI_enum_size", aeabi_enum_size}, {27, "Tag_ABI_HardFP_use", aeabi_hardfp}, {28, "Tag_ABI_VFP_args", aeabi_vfp_args}, {29, "Tag_ABI_WMMX_args", aeabi_wmmx_args}, {30, "Tag_ABI_optimization_goals", aeabi_optm_goal}, {31, "Tag_ABI_FP_optimization_goals", aeabi_fp_optm_goal}, {32, "Tag_compatibility", NULL}, {34, "Tag_CPU_unaligned_access", aeabi_unaligned_access}, {36, "Tag_FP_HP_extension", aeabi_fp_hpext}, {38, "Tag_ABI_FP_16bit_format", aeabi_fp_16bit_format}, {42, "Tag_MPextension_use", aeabi_mpext}, {44, "Tag_DIV_use", aeabi_div}, {64, "Tag_nodefaults", NULL}, {65, "Tag_also_compatible_with", NULL}, {66, "Tag_T2EE_use", aeabi_t2ee}, {67, "Tag_conformance", NULL}, {68, "Tag_Virtualization_use", aeabi_virtual}, {70, "Tag_MPextension_use", aeabi_mpext}, }; static const char * mips_abi_fp(uint64_t fp) { static char s_mips_abi_fp[64]; switch (fp) { case 0: return "N/A"; case 1: return "Hard float (double precision)"; case 2: return "Hard float (single precision)"; case 3: return "Soft float"; case 4: return "64-bit float (-mips32r2 -mfp64)"; default: snprintf(s_mips_abi_fp, sizeof(s_mips_abi_fp), "Unknown(%ju)", (uintmax_t) fp); return (s_mips_abi_fp); } } static const char * ppc_abi_fp(uint64_t fp) { static char s_ppc_abi_fp[64]; switch (fp) { case 0: return "N/A"; case 1: return "Hard float (double precision)"; case 2: return "Soft float"; case 3: return "Hard float (single precision)"; default: snprintf(s_ppc_abi_fp, sizeof(s_ppc_abi_fp), "Unknown(%ju)", (uintmax_t) fp); return (s_ppc_abi_fp); } } static const char * ppc_abi_vector(uint64_t vec) { static char s_vec[64]; switch (vec) { case 0: return "N/A"; case 1: return "Generic purpose registers"; case 2: return "AltiVec registers"; case 3: return "SPE registers"; default: snprintf(s_vec, sizeof(s_vec), "Unknown(%ju)", (uintmax_t) vec); return (s_vec); } } static const char * dwarf_reg(unsigned int mach, unsigned int reg) { switch (mach) { case EM_386: case EM_IAMCU: switch (reg) { case 0: return "eax"; case 1: return "ecx"; case 2: return "edx"; case 3: return "ebx"; case 4: return "esp"; case 5: return "ebp"; case 6: return "esi"; case 7: return "edi"; case 8: return "eip"; case 9: return "eflags"; case 11: return "st0"; case 12: return "st1"; case 13: return "st2"; case 14: return "st3"; case 15: return "st4"; case 16: return "st5"; case 17: return "st6"; case 18: return "st7"; case 21: return "xmm0"; case 22: return "xmm1"; case 23: return "xmm2"; case 24: return "xmm3"; case 25: return "xmm4"; case 26: return "xmm5"; case 27: return "xmm6"; case 28: return "xmm7"; case 29: return "mm0"; case 30: return "mm1"; case 31: return "mm2"; case 32: return "mm3"; case 33: return "mm4"; case 34: return "mm5"; case 35: return "mm6"; case 36: return "mm7"; case 37: return "fcw"; case 38: return "fsw"; case 39: return "mxcsr"; case 40: return "es"; case 41: return "cs"; case 42: return "ss"; case 43: return "ds"; case 44: return "fs"; case 45: return "gs"; case 48: return "tr"; case 49: return "ldtr"; default: return (NULL); } case EM_X86_64: switch (reg) { case 0: return "rax"; case 1: return "rdx"; case 2: return "rcx"; case 3: return "rbx"; case 4: return "rsi"; case 5: return "rdi"; case 6: return "rbp"; case 7: return "rsp"; case 16: return "rip"; case 17: return "xmm0"; case 18: return "xmm1"; case 19: return "xmm2"; case 20: return "xmm3"; case 21: return "xmm4"; case 22: return "xmm5"; case 23: return "xmm6"; case 24: return "xmm7"; case 25: return "xmm8"; case 26: return "xmm9"; case 27: return "xmm10"; case 28: return "xmm11"; case 29: return "xmm12"; case 30: return "xmm13"; case 31: return "xmm14"; case 32: return "xmm15"; case 33: return "st0"; case 34: return "st1"; case 35: return "st2"; case 36: return "st3"; case 37: return "st4"; case 38: return "st5"; case 39: return "st6"; case 40: return "st7"; case 41: return "mm0"; case 42: return "mm1"; case 43: return "mm2"; case 44: return "mm3"; case 45: return "mm4"; case 46: return "mm5"; case 47: return "mm6"; case 48: return "mm7"; case 49: return "rflags"; case 50: return "es"; case 51: return "cs"; case 52: return "ss"; case 53: return "ds"; case 54: return "fs"; case 55: return "gs"; case 58: return "fs.base"; case 59: return "gs.base"; case 62: return "tr"; case 63: return "ldtr"; case 64: return "mxcsr"; case 65: return "fcw"; case 66: return "fsw"; default: return (NULL); } default: return (NULL); } } static void dump_ehdr(struct readelf *re) { size_t shnum, shstrndx; int i; printf("ELF Header:\n"); /* e_ident[]. */ printf(" Magic: "); for (i = 0; i < EI_NIDENT; i++) printf("%.2x ", re->ehdr.e_ident[i]); putchar('\n'); /* EI_CLASS. */ printf("%-37s%s\n", " Class:", elf_class(re->ehdr.e_ident[EI_CLASS])); /* EI_DATA. */ printf("%-37s%s\n", " Data:", elf_endian(re->ehdr.e_ident[EI_DATA])); /* EI_VERSION. */ printf("%-37s%d %s\n", " Version:", re->ehdr.e_ident[EI_VERSION], elf_ver(re->ehdr.e_ident[EI_VERSION])); /* EI_OSABI. */ printf("%-37s%s\n", " OS/ABI:", elf_osabi(re->ehdr.e_ident[EI_OSABI])); /* EI_ABIVERSION. */ printf("%-37s%d\n", " ABI Version:", re->ehdr.e_ident[EI_ABIVERSION]); /* e_type. */ printf("%-37s%s\n", " Type:", elf_type(re->ehdr.e_type)); /* e_machine. */ printf("%-37s%s\n", " Machine:", elf_machine(re->ehdr.e_machine)); /* e_version. */ printf("%-37s%#x\n", " Version:", re->ehdr.e_version); /* e_entry. */ printf("%-37s%#jx\n", " Entry point address:", (uintmax_t)re->ehdr.e_entry); /* e_phoff. */ printf("%-37s%ju (bytes into file)\n", " Start of program headers:", (uintmax_t)re->ehdr.e_phoff); /* e_shoff. */ printf("%-37s%ju (bytes into file)\n", " Start of section headers:", (uintmax_t)re->ehdr.e_shoff); /* e_flags. */ printf("%-37s%#x", " Flags:", re->ehdr.e_flags); dump_eflags(re, re->ehdr.e_flags); putchar('\n'); /* e_ehsize. */ printf("%-37s%u (bytes)\n", " Size of this header:", re->ehdr.e_ehsize); /* e_phentsize. */ printf("%-37s%u (bytes)\n", " Size of program headers:", re->ehdr.e_phentsize); /* e_phnum. */ printf("%-37s%u\n", " Number of program headers:", re->ehdr.e_phnum); /* e_shentsize. */ printf("%-37s%u (bytes)\n", " Size of section headers:", re->ehdr.e_shentsize); /* e_shnum. */ printf("%-37s%u", " Number of section headers:", re->ehdr.e_shnum); if (re->ehdr.e_shnum == SHN_UNDEF) { /* Extended section numbering is in use. */ if (elf_getshnum(re->elf, &shnum)) printf(" (%ju)", (uintmax_t)shnum); } putchar('\n'); /* e_shstrndx. */ printf("%-37s%u", " Section header string table index:", re->ehdr.e_shstrndx); if (re->ehdr.e_shstrndx == SHN_XINDEX) { /* Extended section numbering is in use. */ if (elf_getshstrndx(re->elf, &shstrndx)) printf(" (%ju)", (uintmax_t)shstrndx); } putchar('\n'); } static void dump_eflags(struct readelf *re, uint64_t e_flags) { struct eflags_desc *edesc; int arm_eabi; edesc = NULL; switch (re->ehdr.e_machine) { case EM_ARM: arm_eabi = (e_flags & EF_ARM_EABIMASK) >> 24; if (arm_eabi == 0) printf(", GNU EABI"); else if (arm_eabi <= 5) printf(", Version%d EABI", arm_eabi); edesc = arm_eflags_desc; break; case EM_MIPS: case EM_MIPS_RS3_LE: switch ((e_flags & EF_MIPS_ARCH) >> 28) { case 0: printf(", mips1"); break; case 1: printf(", mips2"); break; case 2: printf(", mips3"); break; case 3: printf(", mips4"); break; case 4: printf(", mips5"); break; case 5: printf(", mips32"); break; case 6: printf(", mips64"); break; case 7: printf(", mips32r2"); break; case 8: printf(", mips64r2"); break; default: break; } switch ((e_flags & 0x00FF0000) >> 16) { case 0x81: printf(", 3900"); break; case 0x82: printf(", 4010"); break; case 0x83: printf(", 4100"); break; case 0x85: printf(", 4650"); break; case 0x87: printf(", 4120"); break; case 0x88: printf(", 4111"); break; case 0x8a: printf(", sb1"); break; case 0x8b: printf(", octeon"); break; case 0x8c: printf(", xlr"); break; case 0x91: printf(", 5400"); break; case 0x98: printf(", 5500"); break; case 0x99: printf(", 9000"); break; case 0xa0: printf(", loongson-2e"); break; case 0xa1: printf(", loongson-2f"); break; default: break; } switch ((e_flags & 0x0000F000) >> 12) { case 1: printf(", o32"); break; case 2: printf(", o64"); break; case 3: printf(", eabi32"); break; case 4: printf(", eabi64"); break; default: break; } edesc = mips_eflags_desc; break; case EM_PPC: case EM_PPC64: edesc = powerpc_eflags_desc; break; case EM_SPARC: case EM_SPARC32PLUS: case EM_SPARCV9: switch ((e_flags & EF_SPARCV9_MM)) { case EF_SPARCV9_TSO: printf(", tso"); break; case EF_SPARCV9_PSO: printf(", pso"); break; case EF_SPARCV9_MM: printf(", rmo"); break; default: break; } edesc = sparc_eflags_desc; break; default: break; } if (edesc != NULL) { while (edesc->desc != NULL) { if (e_flags & edesc->flag) printf(", %s", edesc->desc); edesc++; } } } static void dump_phdr(struct readelf *re) { const char *rawfile; GElf_Phdr phdr; size_t phnum, size; int i, j; #define PH_HDR "Type", "Offset", "VirtAddr", "PhysAddr", "FileSiz", \ "MemSiz", "Flg", "Align" #define PH_CT phdr_type(re->ehdr.e_machine, phdr.p_type), \ (uintmax_t)phdr.p_offset, (uintmax_t)phdr.p_vaddr, \ (uintmax_t)phdr.p_paddr, (uintmax_t)phdr.p_filesz, \ (uintmax_t)phdr.p_memsz, \ phdr.p_flags & PF_R ? 'R' : ' ', \ phdr.p_flags & PF_W ? 'W' : ' ', \ phdr.p_flags & PF_X ? 'E' : ' ', \ (uintmax_t)phdr.p_align if (elf_getphnum(re->elf, &phnum) == 0) { warnx("elf_getphnum failed: %s", elf_errmsg(-1)); return; } if (phnum == 0) { printf("\nThere are no program headers in this file.\n"); return; } printf("\nElf file type is %s", elf_type(re->ehdr.e_type)); printf("\nEntry point 0x%jx\n", (uintmax_t)re->ehdr.e_entry); printf("There are %ju program headers, starting at offset %ju\n", (uintmax_t)phnum, (uintmax_t)re->ehdr.e_phoff); /* Dump program headers. */ printf("\nProgram Headers:\n"); if (re->ec == ELFCLASS32) printf(" %-15s%-9s%-11s%-11s%-8s%-8s%-4s%s\n", PH_HDR); else if (re->options & RE_WW) printf(" %-15s%-9s%-19s%-19s%-9s%-9s%-4s%s\n", PH_HDR); else printf(" %-15s%-19s%-19s%s\n %-19s%-20s" "%-7s%s\n", PH_HDR); for (i = 0; (size_t) i < phnum; i++) { if (gelf_getphdr(re->elf, i, &phdr) != &phdr) { warnx("gelf_getphdr failed: %s", elf_errmsg(-1)); continue; } /* TODO: Add arch-specific segment type dump. */ if (re->ec == ELFCLASS32) printf(" %-14.14s 0x%6.6jx 0x%8.8jx 0x%8.8jx " "0x%5.5jx 0x%5.5jx %c%c%c %#jx\n", PH_CT); else if (re->options & RE_WW) printf(" %-14.14s 0x%6.6jx 0x%16.16jx 0x%16.16jx " "0x%6.6jx 0x%6.6jx %c%c%c %#jx\n", PH_CT); else printf(" %-14.14s 0x%16.16jx 0x%16.16jx 0x%16.16jx\n" " 0x%16.16jx 0x%16.16jx %c%c%c" " %#jx\n", PH_CT); if (phdr.p_type == PT_INTERP) { if ((rawfile = elf_rawfile(re->elf, &size)) == NULL) { warnx("elf_rawfile failed: %s", elf_errmsg(-1)); continue; } if (phdr.p_offset >= size) { warnx("invalid program header offset"); continue; } printf(" [Requesting program interpreter: %s]\n", rawfile + phdr.p_offset); } } /* Dump section to segment mapping. */ if (re->shnum == 0) return; printf("\n Section to Segment mapping:\n"); printf(" Segment Sections...\n"); for (i = 0; (size_t)i < phnum; i++) { if (gelf_getphdr(re->elf, i, &phdr) != &phdr) { warnx("gelf_getphdr failed: %s", elf_errmsg(-1)); continue; } printf(" %2.2d ", i); /* skip NULL section. */ for (j = 1; (size_t)j < re->shnum; j++) if (re->sl[j].addr >= phdr.p_vaddr && re->sl[j].addr + re->sl[j].sz <= phdr.p_vaddr + phdr.p_memsz) printf("%s ", re->sl[j].name); printf("\n"); } #undef PH_HDR #undef PH_CT } static char * section_flags(struct readelf *re, struct section *s) { #define BUF_SZ 256 static char buf[BUF_SZ]; int i, p, nb; p = 0; nb = re->ec == ELFCLASS32 ? 8 : 16; if (re->options & RE_T) { snprintf(buf, BUF_SZ, "[%*.*jx]: ", nb, nb, (uintmax_t)s->flags); p += nb + 4; } for (i = 0; section_flag[i].ln != NULL; i++) { if ((s->flags & section_flag[i].value) == 0) continue; if (re->options & RE_T) { snprintf(&buf[p], BUF_SZ - p, "%s, ", section_flag[i].ln); p += strlen(section_flag[i].ln) + 2; } else buf[p++] = section_flag[i].sn; } if (re->options & RE_T && p > nb + 4) p -= 2; buf[p] = '\0'; return (buf); } static void dump_shdr(struct readelf *re) { struct section *s; int i; #define S_HDR "[Nr] Name", "Type", "Addr", "Off", "Size", "ES", \ "Flg", "Lk", "Inf", "Al" #define S_HDRL "[Nr] Name", "Type", "Address", "Offset", "Size", \ "EntSize", "Flags", "Link", "Info", "Align" #define ST_HDR "[Nr] Name", "Type", "Addr", "Off", "Size", "ES", \ "Lk", "Inf", "Al", "Flags" #define ST_HDRL "[Nr] Name", "Type", "Address", "Offset", "Link", \ "Size", "EntSize", "Info", "Align", "Flags" #define S_CT i, s->name, section_type(re->ehdr.e_machine, s->type), \ (uintmax_t)s->addr, (uintmax_t)s->off, (uintmax_t)s->sz,\ (uintmax_t)s->entsize, section_flags(re, s), \ s->link, s->info, (uintmax_t)s->align #define ST_CT i, s->name, section_type(re->ehdr.e_machine, s->type), \ (uintmax_t)s->addr, (uintmax_t)s->off, (uintmax_t)s->sz,\ (uintmax_t)s->entsize, s->link, s->info, \ (uintmax_t)s->align, section_flags(re, s) #define ST_CTL i, s->name, section_type(re->ehdr.e_machine, s->type), \ (uintmax_t)s->addr, (uintmax_t)s->off, s->link, \ (uintmax_t)s->sz, (uintmax_t)s->entsize, s->info, \ (uintmax_t)s->align, section_flags(re, s) if (re->shnum == 0) { printf("\nThere are no sections in this file.\n"); return; } printf("There are %ju section headers, starting at offset 0x%jx:\n", (uintmax_t)re->shnum, (uintmax_t)re->ehdr.e_shoff); printf("\nSection Headers:\n"); if (re->ec == ELFCLASS32) { if (re->options & RE_T) printf(" %s\n %-16s%-9s%-7s%-7s%-5s%-3s%-4s%s\n" "%12s\n", ST_HDR); else printf(" %-23s%-16s%-9s%-7s%-7s%-3s%-4s%-3s%-4s%s\n", S_HDR); } else if (re->options & RE_WW) { if (re->options & RE_T) printf(" %s\n %-16s%-17s%-7s%-7s%-5s%-3s%-4s%s\n" "%12s\n", ST_HDR); else printf(" %-23s%-16s%-17s%-7s%-7s%-3s%-4s%-3s%-4s%s\n", S_HDR); } else { if (re->options & RE_T) printf(" %s\n %-18s%-17s%-18s%s\n %-18s" "%-17s%-18s%s\n%12s\n", ST_HDRL); else printf(" %-23s%-17s%-18s%s\n %-18s%-17s%-7s%" "-6s%-6s%s\n", S_HDRL); } for (i = 0; (size_t)i < re->shnum; i++) { s = &re->sl[i]; if (re->ec == ELFCLASS32) { if (re->options & RE_T) printf(" [%2d] %s\n %-15.15s %8.8jx" " %6.6jx %6.6jx %2.2jx %2u %3u %2ju\n" " %s\n", ST_CT); else printf(" [%2d] %-17.17s %-15.15s %8.8jx" " %6.6jx %6.6jx %2.2jx %3s %2u %3u %2ju\n", S_CT); } else if (re->options & RE_WW) { if (re->options & RE_T) printf(" [%2d] %s\n %-15.15s %16.16jx" " %6.6jx %6.6jx %2.2jx %2u %3u %2ju\n" " %s\n", ST_CT); else printf(" [%2d] %-17.17s %-15.15s %16.16jx" " %6.6jx %6.6jx %2.2jx %3s %2u %3u %2ju\n", S_CT); } else { if (re->options & RE_T) printf(" [%2d] %s\n %-15.15s %16.16jx" " %16.16jx %u\n %16.16jx %16.16jx" " %-16u %ju\n %s\n", ST_CTL); else printf(" [%2d] %-17.17s %-15.15s %16.16jx" " %8.8jx\n %16.16jx %16.16jx " "%3s %2u %3u %ju\n", S_CT); } } if ((re->options & RE_T) == 0) printf("Key to Flags:\n W (write), A (alloc)," " X (execute), M (merge), S (strings)\n" " I (info), L (link order), G (group), x (unknown)\n" " O (extra OS processing required)" " o (OS specific), p (processor specific)\n"); #undef S_HDR #undef S_HDRL #undef ST_HDR #undef ST_HDRL #undef S_CT #undef ST_CT #undef ST_CTL } /* * Return number of entries in the given section. We'd prefer ent_count be a * size_t *, but libelf APIs already use int for section indices. */ static int get_ent_count(struct section *s, int *ent_count) { if (s->entsize == 0) { warnx("section %s has entry size 0", s->name); return (0); } else if (s->sz / s->entsize > INT_MAX) { warnx("section %s has invalid section count", s->name); return (0); } *ent_count = (int)(s->sz / s->entsize); return (1); } static void dump_dynamic(struct readelf *re) { GElf_Dyn dyn; Elf_Data *d; struct section *s; int elferr, i, is_dynamic, j, jmax, nentries; is_dynamic = 0; for (i = 0; (size_t)i < re->shnum; i++) { s = &re->sl[i]; if (s->type != SHT_DYNAMIC) continue; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); continue; } if (d->d_size <= 0) continue; is_dynamic = 1; /* Determine the actual number of table entries. */ nentries = 0; if (!get_ent_count(s, &jmax)) continue; for (j = 0; j < jmax; j++) { if (gelf_getdyn(d, j, &dyn) != &dyn) { warnx("gelf_getdyn failed: %s", elf_errmsg(-1)); continue; } nentries ++; if (dyn.d_tag == DT_NULL) break; } printf("\nDynamic section at offset 0x%jx", (uintmax_t)s->off); printf(" contains %u entries:\n", nentries); if (re->ec == ELFCLASS32) printf("%5s%12s%28s\n", "Tag", "Type", "Name/Value"); else printf("%5s%20s%28s\n", "Tag", "Type", "Name/Value"); for (j = 0; j < nentries; j++) { if (gelf_getdyn(d, j, &dyn) != &dyn) continue; /* Dump dynamic entry type. */ if (re->ec == ELFCLASS32) printf(" 0x%8.8jx", (uintmax_t)dyn.d_tag); else printf(" 0x%16.16jx", (uintmax_t)dyn.d_tag); printf(" %-20s", dt_type(re->ehdr.e_machine, dyn.d_tag)); /* Dump dynamic entry value. */ dump_dyn_val(re, &dyn, s->link); } } if (!is_dynamic) printf("\nThere is no dynamic section in this file.\n"); } static char * timestamp(time_t ti) { static char ts[32]; struct tm *t; t = gmtime(&ti); snprintf(ts, sizeof(ts), "%04d-%02d-%02dT%02d:%02d:%02d", t->tm_year + 1900, t->tm_mon + 1, t->tm_mday, t->tm_hour, t->tm_min, t->tm_sec); return (ts); } static const char * dyn_str(struct readelf *re, uint32_t stab, uint64_t d_val) { const char *name; if (stab == SHN_UNDEF) name = "ERROR"; else if ((name = elf_strptr(re->elf, stab, d_val)) == NULL) { (void) elf_errno(); /* clear error */ name = "ERROR"; } return (name); } static void dump_arch_dyn_val(struct readelf *re, GElf_Dyn *dyn, uint32_t stab) { const char *name; switch (re->ehdr.e_machine) { case EM_MIPS: case EM_MIPS_RS3_LE: switch (dyn->d_tag) { case DT_MIPS_RLD_VERSION: case DT_MIPS_LOCAL_GOTNO: case DT_MIPS_CONFLICTNO: case DT_MIPS_LIBLISTNO: case DT_MIPS_SYMTABNO: case DT_MIPS_UNREFEXTNO: case DT_MIPS_GOTSYM: case DT_MIPS_HIPAGENO: case DT_MIPS_DELTA_CLASS_NO: case DT_MIPS_DELTA_INSTANCE_NO: case DT_MIPS_DELTA_RELOC_NO: case DT_MIPS_DELTA_SYM_NO: case DT_MIPS_DELTA_CLASSSYM_NO: case DT_MIPS_LOCALPAGE_GOTIDX: case DT_MIPS_LOCAL_GOTIDX: case DT_MIPS_HIDDEN_GOTIDX: case DT_MIPS_PROTECTED_GOTIDX: printf(" %ju\n", (uintmax_t) dyn->d_un.d_val); break; case DT_MIPS_ICHECKSUM: case DT_MIPS_FLAGS: case DT_MIPS_BASE_ADDRESS: case DT_MIPS_CONFLICT: case DT_MIPS_LIBLIST: case DT_MIPS_RLD_MAP: case DT_MIPS_DELTA_CLASS: case DT_MIPS_DELTA_INSTANCE: case DT_MIPS_DELTA_RELOC: case DT_MIPS_DELTA_SYM: case DT_MIPS_DELTA_CLASSSYM: case DT_MIPS_CXX_FLAGS: case DT_MIPS_PIXIE_INIT: case DT_MIPS_SYMBOL_LIB: case DT_MIPS_OPTIONS: case DT_MIPS_INTERFACE: case DT_MIPS_DYNSTR_ALIGN: case DT_MIPS_INTERFACE_SIZE: case DT_MIPS_RLD_TEXT_RESOLVE_ADDR: case DT_MIPS_COMPACT_SIZE: case DT_MIPS_GP_VALUE: case DT_MIPS_AUX_DYNAMIC: case DT_MIPS_PLTGOT: case DT_MIPS_RLD_OBJ_UPDATE: case DT_MIPS_RWPLT: printf(" 0x%jx\n", (uintmax_t) dyn->d_un.d_val); break; case DT_MIPS_IVERSION: case DT_MIPS_PERF_SUFFIX: case DT_AUXILIARY: case DT_FILTER: name = dyn_str(re, stab, dyn->d_un.d_val); printf(" %s\n", name); break; case DT_MIPS_TIME_STAMP: printf(" %s\n", timestamp(dyn->d_un.d_val)); break; } break; default: printf("\n"); break; } } static void dump_dyn_val(struct readelf *re, GElf_Dyn *dyn, uint32_t stab) { const char *name; if (dyn->d_tag >= DT_LOPROC && dyn->d_tag <= DT_HIPROC) { dump_arch_dyn_val(re, dyn, stab); return; } /* These entry values are index into the string table. */ name = NULL; if (dyn->d_tag == DT_NEEDED || dyn->d_tag == DT_SONAME || dyn->d_tag == DT_RPATH || dyn->d_tag == DT_RUNPATH) name = dyn_str(re, stab, dyn->d_un.d_val); switch(dyn->d_tag) { case DT_NULL: case DT_PLTGOT: case DT_HASH: case DT_STRTAB: case DT_SYMTAB: case DT_RELA: case DT_INIT: case DT_SYMBOLIC: case DT_REL: case DT_DEBUG: case DT_TEXTREL: case DT_JMPREL: case DT_FINI: case DT_VERDEF: case DT_VERNEED: case DT_VERSYM: case DT_GNU_HASH: case DT_GNU_LIBLIST: case DT_GNU_CONFLICT: printf(" 0x%jx\n", (uintmax_t) dyn->d_un.d_val); break; case DT_PLTRELSZ: case DT_RELASZ: case DT_RELAENT: case DT_STRSZ: case DT_SYMENT: case DT_RELSZ: case DT_RELENT: case DT_INIT_ARRAYSZ: case DT_FINI_ARRAYSZ: case DT_GNU_CONFLICTSZ: case DT_GNU_LIBLISTSZ: printf(" %ju (bytes)\n", (uintmax_t) dyn->d_un.d_val); break; case DT_RELACOUNT: case DT_RELCOUNT: case DT_VERDEFNUM: case DT_VERNEEDNUM: printf(" %ju\n", (uintmax_t) dyn->d_un.d_val); break; case DT_NEEDED: printf(" Shared library: [%s]\n", name); break; case DT_SONAME: printf(" Library soname: [%s]\n", name); break; case DT_RPATH: printf(" Library rpath: [%s]\n", name); break; case DT_RUNPATH: printf(" Library runpath: [%s]\n", name); break; case DT_PLTREL: printf(" %s\n", dt_type(re->ehdr.e_machine, dyn->d_un.d_val)); break; case DT_GNU_PRELINKED: printf(" %s\n", timestamp(dyn->d_un.d_val)); break; default: printf("\n"); } } static void dump_rel(struct readelf *re, struct section *s, Elf_Data *d) { GElf_Rel r; const char *symname; uint64_t symval; int i, len; + uint32_t type; + uint8_t type2, type3; if (s->link >= re->shnum) return; #define REL_HDR "r_offset", "r_info", "r_type", "st_value", "st_name" #define REL_CT32 (uintmax_t)r.r_offset, (uintmax_t)r.r_info, \ elftc_reloc_type_str(re->ehdr.e_machine, \ ELF32_R_TYPE(r.r_info)), (uintmax_t)symval, symname #define REL_CT64 (uintmax_t)r.r_offset, (uintmax_t)r.r_info, \ - elftc_reloc_type_str(re->ehdr.e_machine, \ - ELF64_R_TYPE(r.r_info)), (uintmax_t)symval, symname + elftc_reloc_type_str(re->ehdr.e_machine, type), \ + (uintmax_t)symval, symname printf("\nRelocation section (%s):\n", s->name); if (re->ec == ELFCLASS32) printf("%-8s %-8s %-19s %-8s %s\n", REL_HDR); else { if (re->options & RE_WW) printf("%-16s %-16s %-24s %-16s %s\n", REL_HDR); else printf("%-12s %-12s %-19s %-16s %s\n", REL_HDR); } assert(d->d_size == s->sz); if (!get_ent_count(s, &len)) return; for (i = 0; i < len; i++) { if (gelf_getrel(d, i, &r) != &r) { warnx("gelf_getrel failed: %s", elf_errmsg(-1)); continue; } symname = get_symbol_name(re, s->link, GELF_R_SYM(r.r_info)); symval = get_symbol_value(re, s->link, GELF_R_SYM(r.r_info)); if (re->ec == ELFCLASS32) { r.r_info = ELF32_R_INFO(ELF64_R_SYM(r.r_info), ELF64_R_TYPE(r.r_info)); printf("%8.8jx %8.8jx %-19.19s %8.8jx %s\n", REL_CT32); } else { + type = ELF64_R_TYPE(r.r_info); + if (re->ehdr.e_machine == EM_MIPS) { + type2 = (type >> 8) & 0xFF; + type3 = (type >> 16) & 0xFF; + type = type & 0xFF; + } else { + type2 = type3 = 0; + } if (re->options & RE_WW) printf("%16.16jx %16.16jx %-24.24s" " %16.16jx %s\n", REL_CT64); else printf("%12.12jx %12.12jx %-19.19s" " %16.16jx %s\n", REL_CT64); + if (re->ehdr.e_machine == EM_MIPS) { + if (re->options & RE_WW) { + printf("%32s: %s\n", "Type2", + elftc_reloc_type_str(EM_MIPS, + type2)); + printf("%32s: %s\n", "Type3", + elftc_reloc_type_str(EM_MIPS, + type3)); + } else { + printf("%24s: %s\n", "Type2", + elftc_reloc_type_str(EM_MIPS, + type2)); + printf("%24s: %s\n", "Type3", + elftc_reloc_type_str(EM_MIPS, + type3)); + } + } } } #undef REL_HDR #undef REL_CT } static void dump_rela(struct readelf *re, struct section *s, Elf_Data *d) { GElf_Rela r; const char *symname; uint64_t symval; int i, len; + uint32_t type; + uint8_t type2, type3; if (s->link >= re->shnum) return; #define RELA_HDR "r_offset", "r_info", "r_type", "st_value", \ "st_name + r_addend" #define RELA_CT32 (uintmax_t)r.r_offset, (uintmax_t)r.r_info, \ elftc_reloc_type_str(re->ehdr.e_machine, \ ELF32_R_TYPE(r.r_info)), (uintmax_t)symval, symname #define RELA_CT64 (uintmax_t)r.r_offset, (uintmax_t)r.r_info, \ - elftc_reloc_type_str(re->ehdr.e_machine, \ - ELF64_R_TYPE(r.r_info)), (uintmax_t)symval, symname + elftc_reloc_type_str(re->ehdr.e_machine, type), \ + (uintmax_t)symval, symname printf("\nRelocation section with addend (%s):\n", s->name); if (re->ec == ELFCLASS32) printf("%-8s %-8s %-19s %-8s %s\n", RELA_HDR); else { if (re->options & RE_WW) printf("%-16s %-16s %-24s %-16s %s\n", RELA_HDR); else printf("%-12s %-12s %-19s %-16s %s\n", RELA_HDR); } assert(d->d_size == s->sz); if (!get_ent_count(s, &len)) return; for (i = 0; i < len; i++) { if (gelf_getrela(d, i, &r) != &r) { warnx("gelf_getrel failed: %s", elf_errmsg(-1)); continue; } symname = get_symbol_name(re, s->link, GELF_R_SYM(r.r_info)); symval = get_symbol_value(re, s->link, GELF_R_SYM(r.r_info)); if (re->ec == ELFCLASS32) { r.r_info = ELF32_R_INFO(ELF64_R_SYM(r.r_info), ELF64_R_TYPE(r.r_info)); printf("%8.8jx %8.8jx %-19.19s %8.8jx %s", RELA_CT32); printf(" + %x\n", (uint32_t) r.r_addend); } else { + type = ELF64_R_TYPE(r.r_info); + if (re->ehdr.e_machine == EM_MIPS) { + type2 = (type >> 8) & 0xFF; + type3 = (type >> 16) & 0xFF; + type = type & 0xFF; + } else { + type2 = type3 = 0; + } if (re->options & RE_WW) printf("%16.16jx %16.16jx %-24.24s" " %16.16jx %s", RELA_CT64); else printf("%12.12jx %12.12jx %-19.19s" " %16.16jx %s", RELA_CT64); printf(" + %jx\n", (uintmax_t) r.r_addend); + if (re->ehdr.e_machine == EM_MIPS) { + if (re->options & RE_WW) { + printf("%32s: %s\n", "Type2", + elftc_reloc_type_str(EM_MIPS, + type2)); + printf("%32s: %s\n", "Type3", + elftc_reloc_type_str(EM_MIPS, + type3)); + } else { + printf("%24s: %s\n", "Type2", + elftc_reloc_type_str(EM_MIPS, + type2)); + printf("%24s: %s\n", "Type3", + elftc_reloc_type_str(EM_MIPS, + type3)); + } + } } } #undef RELA_HDR #undef RELA_CT } static void dump_reloc(struct readelf *re) { struct section *s; Elf_Data *d; int i, elferr; for (i = 0; (size_t)i < re->shnum; i++) { s = &re->sl[i]; if (s->type == SHT_REL || s->type == SHT_RELA) { (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); continue; } if (s->type == SHT_REL) dump_rel(re, s, d); else dump_rela(re, s, d); } } } static void dump_symtab(struct readelf *re, int i) { struct section *s; Elf_Data *d; GElf_Sym sym; const char *name; uint32_t stab; int elferr, j, len; uint16_t vs; s = &re->sl[i]; if (s->link >= re->shnum) return; stab = s->link; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } if (d->d_size <= 0) return; if (!get_ent_count(s, &len)) return; printf("Symbol table (%s)", s->name); printf(" contains %d entries:\n", len); printf("%7s%9s%14s%5s%8s%6s%9s%5s\n", "Num:", "Value", "Size", "Type", "Bind", "Vis", "Ndx", "Name"); for (j = 0; j < len; j++) { if (gelf_getsym(d, j, &sym) != &sym) { warnx("gelf_getsym failed: %s", elf_errmsg(-1)); continue; } printf("%6d:", j); printf(" %16.16jx", (uintmax_t) sym.st_value); printf(" %5ju", (uintmax_t) sym.st_size); printf(" %-7s", st_type(re->ehdr.e_machine, re->ehdr.e_ident[EI_OSABI], GELF_ST_TYPE(sym.st_info))); printf(" %-6s", st_bind(GELF_ST_BIND(sym.st_info))); printf(" %-8s", st_vis(GELF_ST_VISIBILITY(sym.st_other))); printf(" %3s", st_shndx(sym.st_shndx)); if ((name = elf_strptr(re->elf, stab, sym.st_name)) != NULL) printf(" %s", name); /* Append symbol version string for SHT_DYNSYM symbol table. */ if (s->type == SHT_DYNSYM && re->ver != NULL && re->vs != NULL && re->vs[j] > 1) { vs = re->vs[j] & VERSYM_VERSION; if (vs >= re->ver_sz || re->ver[vs].name == NULL) { warnx("invalid versym version index %u", vs); break; } if (re->vs[j] & VERSYM_HIDDEN || re->ver[vs].type == 0) printf("@%s (%d)", re->ver[vs].name, vs); else printf("@@%s (%d)", re->ver[vs].name, vs); } putchar('\n'); } } static void dump_symtabs(struct readelf *re) { GElf_Dyn dyn; Elf_Data *d; struct section *s; uint64_t dyn_off; int elferr, i, len; /* * If -D is specified, only dump the symbol table specified by * the DT_SYMTAB entry in the .dynamic section. */ dyn_off = 0; if (re->options & RE_DD) { s = NULL; for (i = 0; (size_t)i < re->shnum; i++) if (re->sl[i].type == SHT_DYNAMIC) { s = &re->sl[i]; break; } if (s == NULL) return; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); return; } if (d->d_size <= 0) return; if (!get_ent_count(s, &len)) return; for (i = 0; i < len; i++) { if (gelf_getdyn(d, i, &dyn) != &dyn) { warnx("gelf_getdyn failed: %s", elf_errmsg(-1)); continue; } if (dyn.d_tag == DT_SYMTAB) { dyn_off = dyn.d_un.d_val; break; } } } /* Find and dump symbol tables. */ for (i = 0; (size_t)i < re->shnum; i++) { s = &re->sl[i]; if (s->type == SHT_SYMTAB || s->type == SHT_DYNSYM) { if (re->options & RE_DD) { if (dyn_off == s->addr) { dump_symtab(re, i); break; } } else dump_symtab(re, i); } } } static void dump_svr4_hash(struct section *s) { Elf_Data *d; uint32_t *buf; uint32_t nbucket, nchain; uint32_t *bucket, *chain; uint32_t *bl, *c, maxl, total; int elferr, i, j; /* Read and parse the content of .hash section. */ (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } if (d->d_size < 2 * sizeof(uint32_t)) { warnx(".hash section too small"); return; } buf = d->d_buf; nbucket = buf[0]; nchain = buf[1]; if (nbucket <= 0 || nchain <= 0) { warnx("Malformed .hash section"); return; } if (d->d_size != (nbucket + nchain + 2) * sizeof(uint32_t)) { warnx("Malformed .hash section"); return; } bucket = &buf[2]; chain = &buf[2 + nbucket]; maxl = 0; if ((bl = calloc(nbucket, sizeof(*bl))) == NULL) errx(EXIT_FAILURE, "calloc failed"); for (i = 0; (uint32_t)i < nbucket; i++) for (j = bucket[i]; j > 0 && (uint32_t)j < nchain; j = chain[j]) if (++bl[i] > maxl) maxl = bl[i]; if ((c = calloc(maxl + 1, sizeof(*c))) == NULL) errx(EXIT_FAILURE, "calloc failed"); for (i = 0; (uint32_t)i < nbucket; i++) c[bl[i]]++; printf("\nHistogram for bucket list length (total of %u buckets):\n", nbucket); printf(" Length\tNumber\t\t%% of total\tCoverage\n"); total = 0; for (i = 0; (uint32_t)i <= maxl; i++) { total += c[i] * i; printf("%7u\t%-10u\t(%5.1f%%)\t%5.1f%%\n", i, c[i], c[i] * 100.0 / nbucket, total * 100.0 / (nchain - 1)); } free(c); free(bl); } static void dump_svr4_hash64(struct readelf *re, struct section *s) { Elf_Data *d, dst; uint64_t *buf; uint64_t nbucket, nchain; uint64_t *bucket, *chain; uint64_t *bl, *c, maxl, total; int elferr, i, j; /* * ALPHA uses 64-bit hash entries. Since libelf assumes that * .hash section contains only 32-bit entry, an explicit * gelf_xlatetom is needed here. */ (void) elf_errno(); if ((d = elf_rawdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_rawdata failed: %s", elf_errmsg(elferr)); return; } d->d_type = ELF_T_XWORD; memcpy(&dst, d, sizeof(Elf_Data)); if (gelf_xlatetom(re->elf, &dst, d, re->ehdr.e_ident[EI_DATA]) != &dst) { warnx("gelf_xlatetom failed: %s", elf_errmsg(-1)); return; } if (dst.d_size < 2 * sizeof(uint64_t)) { warnx(".hash section too small"); return; } buf = dst.d_buf; nbucket = buf[0]; nchain = buf[1]; if (nbucket <= 0 || nchain <= 0) { warnx("Malformed .hash section"); return; } if (d->d_size != (nbucket + nchain + 2) * sizeof(uint32_t)) { warnx("Malformed .hash section"); return; } bucket = &buf[2]; chain = &buf[2 + nbucket]; maxl = 0; if ((bl = calloc(nbucket, sizeof(*bl))) == NULL) errx(EXIT_FAILURE, "calloc failed"); for (i = 0; (uint32_t)i < nbucket; i++) for (j = bucket[i]; j > 0 && (uint32_t)j < nchain; j = chain[j]) if (++bl[i] > maxl) maxl = bl[i]; if ((c = calloc(maxl + 1, sizeof(*c))) == NULL) errx(EXIT_FAILURE, "calloc failed"); for (i = 0; (uint64_t)i < nbucket; i++) c[bl[i]]++; printf("Histogram for bucket list length (total of %ju buckets):\n", (uintmax_t)nbucket); printf(" Length\tNumber\t\t%% of total\tCoverage\n"); total = 0; for (i = 0; (uint64_t)i <= maxl; i++) { total += c[i] * i; printf("%7u\t%-10ju\t(%5.1f%%)\t%5.1f%%\n", i, (uintmax_t)c[i], c[i] * 100.0 / nbucket, total * 100.0 / (nchain - 1)); } free(c); free(bl); } static void dump_gnu_hash(struct readelf *re, struct section *s) { struct section *ds; Elf_Data *d; uint32_t *buf; uint32_t *bucket, *chain; uint32_t nbucket, nchain, symndx, maskwords; uint32_t *bl, *c, maxl, total; int elferr, dynsymcount, i, j; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } if (d->d_size < 4 * sizeof(uint32_t)) { warnx(".gnu.hash section too small"); return; } buf = d->d_buf; nbucket = buf[0]; symndx = buf[1]; maskwords = buf[2]; buf += 4; if (s->link >= re->shnum) return; ds = &re->sl[s->link]; if (!get_ent_count(ds, &dynsymcount)) return; if (symndx >= (uint32_t)dynsymcount) { warnx("Malformed .gnu.hash section (symndx out of range)"); return; } nchain = dynsymcount - symndx; if (d->d_size != 4 * sizeof(uint32_t) + maskwords * (re->ec == ELFCLASS32 ? sizeof(uint32_t) : sizeof(uint64_t)) + (nbucket + nchain) * sizeof(uint32_t)) { warnx("Malformed .gnu.hash section"); return; } bucket = buf + (re->ec == ELFCLASS32 ? maskwords : maskwords * 2); chain = bucket + nbucket; maxl = 0; if ((bl = calloc(nbucket, sizeof(*bl))) == NULL) errx(EXIT_FAILURE, "calloc failed"); for (i = 0; (uint32_t)i < nbucket; i++) for (j = bucket[i]; j > 0 && (uint32_t)j - symndx < nchain; j++) { if (++bl[i] > maxl) maxl = bl[i]; if (chain[j - symndx] & 1) break; } if ((c = calloc(maxl + 1, sizeof(*c))) == NULL) errx(EXIT_FAILURE, "calloc failed"); for (i = 0; (uint32_t)i < nbucket; i++) c[bl[i]]++; printf("Histogram for bucket list length (total of %u buckets):\n", nbucket); printf(" Length\tNumber\t\t%% of total\tCoverage\n"); total = 0; for (i = 0; (uint32_t)i <= maxl; i++) { total += c[i] * i; printf("%7u\t%-10u\t(%5.1f%%)\t%5.1f%%\n", i, c[i], c[i] * 100.0 / nbucket, total * 100.0 / (nchain - 1)); } free(c); free(bl); } static void dump_hash(struct readelf *re) { struct section *s; int i; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->type == SHT_HASH || s->type == SHT_GNU_HASH) { if (s->type == SHT_GNU_HASH) dump_gnu_hash(re, s); else if (re->ehdr.e_machine == EM_ALPHA && s->entsize == 8) dump_svr4_hash64(re, s); else dump_svr4_hash(s); } } } static void dump_notes(struct readelf *re) { struct section *s; const char *rawfile; GElf_Phdr phdr; Elf_Data *d; size_t filesize, phnum; int i, elferr; if (re->ehdr.e_type == ET_CORE) { /* * Search program headers in the core file for * PT_NOTE entry. */ if (elf_getphnum(re->elf, &phnum) == 0) { warnx("elf_getphnum failed: %s", elf_errmsg(-1)); return; } if (phnum == 0) return; if ((rawfile = elf_rawfile(re->elf, &filesize)) == NULL) { warnx("elf_rawfile failed: %s", elf_errmsg(-1)); return; } for (i = 0; (size_t) i < phnum; i++) { if (gelf_getphdr(re->elf, i, &phdr) != &phdr) { warnx("gelf_getphdr failed: %s", elf_errmsg(-1)); continue; } if (phdr.p_type == PT_NOTE) { if (phdr.p_offset >= filesize || phdr.p_filesz > filesize - phdr.p_offset) { warnx("invalid PHDR offset"); continue; } dump_notes_content(re, rawfile + phdr.p_offset, phdr.p_filesz, phdr.p_offset); } } } else { /* * For objects other than core files, Search for * SHT_NOTE sections. */ for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->type == SHT_NOTE) { (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); continue; } dump_notes_content(re, d->d_buf, d->d_size, s->off); } } } } static void dump_notes_content(struct readelf *re, const char *buf, size_t sz, off_t off) { Elf_Note *note; const char *end, *name; printf("\nNotes at offset %#010jx with length %#010jx:\n", (uintmax_t) off, (uintmax_t) sz); printf(" %-13s %-15s %s\n", "Owner", "Data size", "Description"); end = buf + sz; while (buf < end) { if (buf + sizeof(*note) > end) { warnx("invalid note header"); return; } note = (Elf_Note *)(uintptr_t) buf; name = (char *)(uintptr_t)(note + 1); /* * The name field is required to be nul-terminated, and * n_namesz includes the terminating nul in observed * implementations (contrary to the ELF-64 spec). A special * case is needed for cores generated by some older Linux * versions, which write a note named "CORE" without a nul * terminator and n_namesz = 4. */ if (note->n_namesz == 0) name = ""; else if (note->n_namesz == 4 && strncmp(name, "CORE", 4) == 0) name = "CORE"; else if (strnlen(name, note->n_namesz) >= note->n_namesz) name = ""; printf(" %-13s %#010jx", name, (uintmax_t) note->n_descsz); printf(" %s\n", note_type(name, re->ehdr.e_type, note->n_type)); buf += sizeof(Elf_Note) + roundup2(note->n_namesz, 4) + roundup2(note->n_descsz, 4); } } /* * Symbol versioning sections are the same for 32bit and 64bit * ELF objects. */ #define Elf_Verdef Elf32_Verdef #define Elf_Verdaux Elf32_Verdaux #define Elf_Verneed Elf32_Verneed #define Elf_Vernaux Elf32_Vernaux #define SAVE_VERSION_NAME(x, n, t) \ do { \ while (x >= re->ver_sz) { \ nv = realloc(re->ver, \ sizeof(*re->ver) * re->ver_sz * 2); \ if (nv == NULL) { \ warn("realloc failed"); \ free(re->ver); \ return; \ } \ re->ver = nv; \ for (i = re->ver_sz; i < re->ver_sz * 2; i++) { \ re->ver[i].name = NULL; \ re->ver[i].type = 0; \ } \ re->ver_sz *= 2; \ } \ if (x > 1) { \ re->ver[x].name = n; \ re->ver[x].type = t; \ } \ } while (0) static void dump_verdef(struct readelf *re, int dump) { struct section *s; struct symver *nv; Elf_Data *d; Elf_Verdef *vd; Elf_Verdaux *vda; uint8_t *buf, *end, *buf2; const char *name; int elferr, i, j; if ((s = re->vd_s) == NULL) return; if (s->link >= re->shnum) return; if (re->ver == NULL) { re->ver_sz = 16; if ((re->ver = calloc(re->ver_sz, sizeof(*re->ver))) == NULL) { warn("calloc failed"); return; } re->ver[0].name = "*local*"; re->ver[1].name = "*global*"; } if (dump) printf("\nVersion definition section (%s):\n", s->name); (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } if (d->d_size == 0) return; buf = d->d_buf; end = buf + d->d_size; while (buf + sizeof(Elf_Verdef) <= end) { vd = (Elf_Verdef *) (uintptr_t) buf; if (dump) { printf(" 0x%4.4lx", (unsigned long) (buf - (uint8_t *)d->d_buf)); printf(" vd_version: %u vd_flags: %d" " vd_ndx: %u vd_cnt: %u", vd->vd_version, vd->vd_flags, vd->vd_ndx, vd->vd_cnt); } buf2 = buf + vd->vd_aux; j = 0; while (buf2 + sizeof(Elf_Verdaux) <= end && j < vd->vd_cnt) { vda = (Elf_Verdaux *) (uintptr_t) buf2; name = get_string(re, s->link, vda->vda_name); if (j == 0) { if (dump) printf(" vda_name: %s\n", name); SAVE_VERSION_NAME((int)vd->vd_ndx, name, 1); } else if (dump) printf(" 0x%4.4lx parent: %s\n", (unsigned long) (buf2 - (uint8_t *)d->d_buf), name); if (vda->vda_next == 0) break; buf2 += vda->vda_next; j++; } if (vd->vd_next == 0) break; buf += vd->vd_next; } } static void dump_verneed(struct readelf *re, int dump) { struct section *s; struct symver *nv; Elf_Data *d; Elf_Verneed *vn; Elf_Vernaux *vna; uint8_t *buf, *end, *buf2; const char *name; int elferr, i, j; if ((s = re->vn_s) == NULL) return; if (s->link >= re->shnum) return; if (re->ver == NULL) { re->ver_sz = 16; if ((re->ver = calloc(re->ver_sz, sizeof(*re->ver))) == NULL) { warn("calloc failed"); return; } re->ver[0].name = "*local*"; re->ver[1].name = "*global*"; } if (dump) printf("\nVersion needed section (%s):\n", s->name); (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } if (d->d_size == 0) return; buf = d->d_buf; end = buf + d->d_size; while (buf + sizeof(Elf_Verneed) <= end) { vn = (Elf_Verneed *) (uintptr_t) buf; if (dump) { printf(" 0x%4.4lx", (unsigned long) (buf - (uint8_t *)d->d_buf)); printf(" vn_version: %u vn_file: %s vn_cnt: %u\n", vn->vn_version, get_string(re, s->link, vn->vn_file), vn->vn_cnt); } buf2 = buf + vn->vn_aux; j = 0; while (buf2 + sizeof(Elf_Vernaux) <= end && j < vn->vn_cnt) { vna = (Elf32_Vernaux *) (uintptr_t) buf2; if (dump) printf(" 0x%4.4lx", (unsigned long) (buf2 - (uint8_t *)d->d_buf)); name = get_string(re, s->link, vna->vna_name); if (dump) printf(" vna_name: %s vna_flags: %u" " vna_other: %u\n", name, vna->vna_flags, vna->vna_other); SAVE_VERSION_NAME((int)vna->vna_other, name, 0); if (vna->vna_next == 0) break; buf2 += vna->vna_next; j++; } if (vn->vn_next == 0) break; buf += vn->vn_next; } } static void dump_versym(struct readelf *re) { int i; uint16_t vs; if (re->vs_s == NULL || re->ver == NULL || re->vs == NULL) return; printf("\nVersion symbol section (%s):\n", re->vs_s->name); for (i = 0; i < re->vs_sz; i++) { if ((i & 3) == 0) { if (i > 0) putchar('\n'); printf(" %03x:", i); } vs = re->vs[i] & VERSYM_VERSION; if (vs >= re->ver_sz || re->ver[vs].name == NULL) { warnx("invalid versym version index %u", re->vs[i]); break; } if (re->vs[i] & VERSYM_HIDDEN) printf(" %3xh %-12s ", vs, re->ver[re->vs[i] & VERSYM_VERSION].name); else printf(" %3x %-12s ", vs, re->ver[re->vs[i]].name); } putchar('\n'); } static void dump_ver(struct readelf *re) { if (re->vs_s && re->ver && re->vs) dump_versym(re); if (re->vd_s) dump_verdef(re, 1); if (re->vn_s) dump_verneed(re, 1); } static void search_ver(struct readelf *re) { struct section *s; Elf_Data *d; int elferr, i; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->type == SHT_SUNW_versym) re->vs_s = s; if (s->type == SHT_SUNW_verneed) re->vn_s = s; if (s->type == SHT_SUNW_verdef) re->vd_s = s; } if (re->vd_s) dump_verdef(re, 0); if (re->vn_s) dump_verneed(re, 0); if (re->vs_s && re->ver != NULL) { (void) elf_errno(); if ((d = elf_getdata(re->vs_s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return; } if (d->d_size == 0) return; re->vs = d->d_buf; re->vs_sz = d->d_size / sizeof(Elf32_Half); } } #undef Elf_Verdef #undef Elf_Verdaux #undef Elf_Verneed #undef Elf_Vernaux #undef SAVE_VERSION_NAME /* * Elf32_Lib and Elf64_Lib are identical. */ #define Elf_Lib Elf32_Lib static void dump_liblist(struct readelf *re) { struct section *s; struct tm *t; time_t ti; char tbuf[20]; Elf_Data *d; Elf_Lib *lib; int i, j, k, elferr, first, len; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->type != SHT_GNU_LIBLIST) continue; if (s->link >= re->shnum) continue; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); continue; } if (d->d_size <= 0) continue; lib = d->d_buf; if (!get_ent_count(s, &len)) continue; printf("\nLibrary list section '%s' ", s->name); printf("contains %d entries:\n", len); printf("%12s%24s%18s%10s%6s\n", "Library", "Time Stamp", "Checksum", "Version", "Flags"); for (j = 0; (uint64_t) j < s->sz / s->entsize; j++) { printf("%3d: ", j); printf("%-20.20s ", get_string(re, s->link, lib->l_name)); ti = lib->l_time_stamp; t = gmtime(&ti); snprintf(tbuf, sizeof(tbuf), "%04d-%02d-%02dT%02d:%02d" ":%2d", t->tm_year + 1900, t->tm_mon + 1, t->tm_mday, t->tm_hour, t->tm_min, t->tm_sec); printf("%-19.19s ", tbuf); printf("0x%08x ", lib->l_checksum); printf("%-7d %#x", lib->l_version, lib->l_flags); if (lib->l_flags != 0) { first = 1; putchar('('); for (k = 0; l_flag[k].name != NULL; k++) { if ((l_flag[k].value & lib->l_flags) == 0) continue; if (!first) putchar(','); else first = 0; printf("%s", l_flag[k].name); } putchar(')'); } putchar('\n'); lib++; } } } #undef Elf_Lib static void dump_section_groups(struct readelf *re) { struct section *s; const char *symname; Elf_Data *d; uint32_t *w; int i, j, elferr; size_t n; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->type != SHT_GROUP) continue; if (s->link >= re->shnum) continue; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); continue; } if (d->d_size <= 0) continue; w = d->d_buf; /* We only support COMDAT section. */ #ifndef GRP_COMDAT #define GRP_COMDAT 0x1 #endif if ((*w++ & GRP_COMDAT) == 0) return; if (s->entsize == 0) s->entsize = 4; symname = get_symbol_name(re, s->link, s->info); n = s->sz / s->entsize; if (n-- < 1) return; printf("\nCOMDAT group section [%5d] `%s' [%s] contains %ju" " sections:\n", i, s->name, symname, (uintmax_t)n); printf(" %-10.10s %s\n", "[Index]", "Name"); for (j = 0; (size_t) j < n; j++, w++) { if (*w >= re->shnum) { warnx("invalid section index: %u", *w); continue; } printf(" [%5u] %s\n", *w, re->sl[*w].name); } } } static uint8_t * dump_unknown_tag(uint64_t tag, uint8_t *p, uint8_t *pe) { uint64_t val; /* * According to ARM EABI: For tags > 32, even numbered tags have * a ULEB128 param and odd numbered ones have NUL-terminated * string param. This rule probably also applies for tags <= 32 * if the object arch is not ARM. */ printf(" Tag_unknown_%ju: ", (uintmax_t) tag); if (tag & 1) { printf("%s\n", (char *) p); p += strlen((char *) p) + 1; } else { val = _decode_uleb128(&p, pe); printf("%ju\n", (uintmax_t) val); } return (p); } static uint8_t * dump_compatibility_tag(uint8_t *p, uint8_t *pe) { uint64_t val; val = _decode_uleb128(&p, pe); printf("flag = %ju, vendor = %s\n", (uintmax_t) val, p); p += strlen((char *) p) + 1; return (p); } static void dump_arm_attributes(struct readelf *re, uint8_t *p, uint8_t *pe) { uint64_t tag, val; size_t i; int found, desc; (void) re; while (p < pe) { tag = _decode_uleb128(&p, pe); found = desc = 0; for (i = 0; i < sizeof(aeabi_tags) / sizeof(aeabi_tags[0]); i++) { if (tag == aeabi_tags[i].tag) { found = 1; printf(" %s: ", aeabi_tags[i].s_tag); if (aeabi_tags[i].get_desc) { desc = 1; val = _decode_uleb128(&p, pe); printf("%s\n", aeabi_tags[i].get_desc(val)); } break; } if (tag < aeabi_tags[i].tag) break; } if (!found) { p = dump_unknown_tag(tag, p, pe); continue; } if (desc) continue; switch (tag) { case 4: /* Tag_CPU_raw_name */ case 5: /* Tag_CPU_name */ case 67: /* Tag_conformance */ printf("%s\n", (char *) p); p += strlen((char *) p) + 1; break; case 32: /* Tag_compatibility */ p = dump_compatibility_tag(p, pe); break; case 64: /* Tag_nodefaults */ /* ignored, written as 0. */ (void) _decode_uleb128(&p, pe); printf("True\n"); break; case 65: /* Tag_also_compatible_with */ val = _decode_uleb128(&p, pe); /* Must be Tag_CPU_arch */ if (val != 6) { printf("unknown\n"); break; } val = _decode_uleb128(&p, pe); printf("%s\n", aeabi_cpu_arch(val)); /* Skip NUL terminator. */ p++; break; default: putchar('\n'); break; } } } #ifndef Tag_GNU_MIPS_ABI_FP #define Tag_GNU_MIPS_ABI_FP 4 #endif static void dump_mips_attributes(struct readelf *re, uint8_t *p, uint8_t *pe) { uint64_t tag, val; (void) re; while (p < pe) { tag = _decode_uleb128(&p, pe); switch (tag) { case Tag_GNU_MIPS_ABI_FP: val = _decode_uleb128(&p, pe); printf(" Tag_GNU_MIPS_ABI_FP: %s\n", mips_abi_fp(val)); break; case 32: /* Tag_compatibility */ p = dump_compatibility_tag(p, pe); break; default: p = dump_unknown_tag(tag, p, pe); break; } } } #ifndef Tag_GNU_Power_ABI_FP #define Tag_GNU_Power_ABI_FP 4 #endif #ifndef Tag_GNU_Power_ABI_Vector #define Tag_GNU_Power_ABI_Vector 8 #endif static void dump_ppc_attributes(uint8_t *p, uint8_t *pe) { uint64_t tag, val; while (p < pe) { tag = _decode_uleb128(&p, pe); switch (tag) { case Tag_GNU_Power_ABI_FP: val = _decode_uleb128(&p, pe); printf(" Tag_GNU_Power_ABI_FP: %s\n", ppc_abi_fp(val)); break; case Tag_GNU_Power_ABI_Vector: val = _decode_uleb128(&p, pe); printf(" Tag_GNU_Power_ABI_Vector: %s\n", ppc_abi_vector(val)); break; case 32: /* Tag_compatibility */ p = dump_compatibility_tag(p, pe); break; default: p = dump_unknown_tag(tag, p, pe); break; } } } static void dump_attributes(struct readelf *re) { struct section *s; Elf_Data *d; uint8_t *p, *pe, *sp; size_t len, seclen, nlen, sublen; uint64_t val; int tag, i, elferr; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->type != SHT_GNU_ATTRIBUTES && (re->ehdr.e_machine != EM_ARM || s->type != SHT_LOPROC + 3)) continue; (void) elf_errno(); if ((d = elf_rawdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_rawdata failed: %s", elf_errmsg(elferr)); continue; } if (d->d_size <= 0) continue; p = d->d_buf; pe = p + d->d_size; if (*p != 'A') { printf("Unknown Attribute Section Format: %c\n", (char) *p); continue; } len = d->d_size - 1; p++; while (len > 0) { if (len < 4) { warnx("truncated attribute section length"); return; } seclen = re->dw_decode(&p, 4); if (seclen > len) { warnx("invalid attribute section length"); return; } len -= seclen; nlen = strlen((char *) p) + 1; if (nlen + 4 > seclen) { warnx("invalid attribute section name"); return; } printf("Attribute Section: %s\n", (char *) p); p += nlen; seclen -= nlen + 4; while (seclen > 0) { sp = p; tag = *p++; sublen = re->dw_decode(&p, 4); if (sublen > seclen) { warnx("invalid attribute sub-section" " length"); return; } seclen -= sublen; printf("%s", top_tag(tag)); if (tag == 2 || tag == 3) { putchar(':'); for (;;) { val = _decode_uleb128(&p, pe); if (val == 0) break; printf(" %ju", (uintmax_t) val); } } putchar('\n'); if (re->ehdr.e_machine == EM_ARM && s->type == SHT_LOPROC + 3) dump_arm_attributes(re, p, sp + sublen); else if (re->ehdr.e_machine == EM_MIPS || re->ehdr.e_machine == EM_MIPS_RS3_LE) dump_mips_attributes(re, p, sp + sublen); else if (re->ehdr.e_machine == EM_PPC) dump_ppc_attributes(p, sp + sublen); p = sp + sublen; } } } } static void dump_mips_specific_info(struct readelf *re) { struct section *s; int i, options_found; options_found = 0; s = NULL; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->name != NULL && (!strcmp(s->name, ".MIPS.options") || (s->type == SHT_MIPS_OPTIONS))) { dump_mips_options(re, s); options_found = 1; } } /* * According to SGI mips64 spec, .reginfo should be ignored if * .MIPS.options section is present. */ if (!options_found) { for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->name != NULL && (!strcmp(s->name, ".reginfo") || (s->type == SHT_MIPS_REGINFO))) dump_mips_reginfo(re, s); } } } static void dump_mips_reginfo(struct readelf *re, struct section *s) { Elf_Data *d; int elferr, len; (void) elf_errno(); if ((d = elf_rawdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_rawdata failed: %s", elf_errmsg(elferr)); return; } if (d->d_size <= 0) return; if (!get_ent_count(s, &len)) return; printf("\nSection '%s' contains %d entries:\n", s->name, len); dump_mips_odk_reginfo(re, d->d_buf, d->d_size); } static void dump_mips_options(struct readelf *re, struct section *s) { Elf_Data *d; uint32_t info; uint16_t sndx; uint8_t *p, *pe; uint8_t kind, size; int elferr; (void) elf_errno(); if ((d = elf_rawdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_rawdata failed: %s", elf_errmsg(elferr)); return; } if (d->d_size == 0) return; printf("\nSection %s contains:\n", s->name); p = d->d_buf; pe = p + d->d_size; while (p < pe) { if (pe - p < 8) { warnx("Truncated MIPS option header"); return; } kind = re->dw_decode(&p, 1); size = re->dw_decode(&p, 1); sndx = re->dw_decode(&p, 2); info = re->dw_decode(&p, 4); if (size < 8 || size - 8 > pe - p) { warnx("Malformed MIPS option header"); return; } size -= 8; switch (kind) { case ODK_REGINFO: dump_mips_odk_reginfo(re, p, size); break; case ODK_EXCEPTIONS: printf(" EXCEPTIONS FPU_MIN: %#x\n", info & OEX_FPU_MIN); printf("%11.11s FPU_MAX: %#x\n", "", info & OEX_FPU_MAX); dump_mips_option_flags("", mips_exceptions_option, info); break; case ODK_PAD: printf(" %-10.10s section: %ju\n", "OPAD", (uintmax_t) sndx); dump_mips_option_flags("", mips_pad_option, info); break; case ODK_HWPATCH: dump_mips_option_flags("HWPATCH", mips_hwpatch_option, info); break; case ODK_HWAND: dump_mips_option_flags("HWAND", mips_hwa_option, info); break; case ODK_HWOR: dump_mips_option_flags("HWOR", mips_hwo_option, info); break; case ODK_FILL: printf(" %-10.10s %#jx\n", "FILL", (uintmax_t) info); break; case ODK_TAGS: printf(" %-10.10s\n", "TAGS"); break; case ODK_GP_GROUP: printf(" %-10.10s GP group number: %#x\n", "GP_GROUP", info & 0xFFFF); if (info & 0x10000) printf(" %-10.10s GP group is " "self-contained\n", ""); break; case ODK_IDENT: printf(" %-10.10s default GP group number: %#x\n", "IDENT", info & 0xFFFF); if (info & 0x10000) printf(" %-10.10s default GP group is " "self-contained\n", ""); break; case ODK_PAGESIZE: printf(" %-10.10s\n", "PAGESIZE"); break; default: break; } p += size; } } static void dump_mips_option_flags(const char *name, struct mips_option *opt, uint64_t info) { int first; first = 1; for (; opt->desc != NULL; opt++) { if (info & opt->flag) { printf(" %-10.10s %s\n", first ? name : "", opt->desc); first = 0; } } } static void dump_mips_odk_reginfo(struct readelf *re, uint8_t *p, size_t sz) { uint32_t ri_gprmask; uint32_t ri_cprmask[4]; uint64_t ri_gp_value; uint8_t *pe; int i; pe = p + sz; while (p < pe) { ri_gprmask = re->dw_decode(&p, 4); /* Skip ri_pad padding field for mips64. */ if (re->ec == ELFCLASS64) re->dw_decode(&p, 4); for (i = 0; i < 4; i++) ri_cprmask[i] = re->dw_decode(&p, 4); if (re->ec == ELFCLASS32) ri_gp_value = re->dw_decode(&p, 4); else ri_gp_value = re->dw_decode(&p, 8); printf(" %s ", option_kind(ODK_REGINFO)); printf("ri_gprmask: 0x%08jx\n", (uintmax_t) ri_gprmask); for (i = 0; i < 4; i++) printf("%11.11s ri_cprmask[%d]: 0x%08jx\n", "", i, (uintmax_t) ri_cprmask[i]); printf("%12.12s", ""); printf("ri_gp_value: %#jx\n", (uintmax_t) ri_gp_value); } } static void dump_arch_specific_info(struct readelf *re) { dump_liblist(re); dump_attributes(re); switch (re->ehdr.e_machine) { case EM_MIPS: case EM_MIPS_RS3_LE: dump_mips_specific_info(re); default: break; } } static const char * dwarf_regname(struct readelf *re, unsigned int num) { static char rx[32]; const char *rn; if ((rn = dwarf_reg(re->ehdr.e_machine, num)) != NULL) return (rn); snprintf(rx, sizeof(rx), "r%u", num); return (rx); } static void dump_dwarf_line(struct readelf *re) { struct section *s; Dwarf_Die die; Dwarf_Error de; Dwarf_Half tag, version, pointer_size; Dwarf_Unsigned offset, endoff, length, hdrlen, dirndx, mtime, fsize; Dwarf_Small minlen, defstmt, lrange, opbase, oplen; Elf_Data *d; char *pn; uint64_t address, file, line, column, isa, opsize, udelta; int64_t sdelta; uint8_t *p, *pe; int8_t lbase; int i, is_stmt, dwarf_size, elferr, ret; printf("\nDump of debug contents of section .debug_line:\n"); s = NULL; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->name != NULL && !strcmp(s->name, ".debug_line")) break; } if ((size_t) i >= re->shnum) return; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); return; } if (d->d_size <= 0) return; while ((ret = dwarf_next_cu_header(re->dbg, NULL, NULL, NULL, NULL, NULL, &de)) == DW_DLV_OK) { die = NULL; while (dwarf_siblingof(re->dbg, die, &die, &de) == DW_DLV_OK) { if (dwarf_tag(die, &tag, &de) != DW_DLV_OK) { warnx("dwarf_tag failed: %s", dwarf_errmsg(de)); return; } /* XXX: What about DW_TAG_partial_unit? */ if (tag == DW_TAG_compile_unit) break; } if (die == NULL) { warnx("could not find DW_TAG_compile_unit die"); return; } if (dwarf_attrval_unsigned(die, DW_AT_stmt_list, &offset, &de) != DW_DLV_OK) continue; length = re->dw_read(d, &offset, 4); if (length == 0xffffffff) { dwarf_size = 8; length = re->dw_read(d, &offset, 8); } else dwarf_size = 4; if (length > d->d_size - offset) { warnx("invalid .dwarf_line section"); continue; } endoff = offset + length; pe = (uint8_t *) d->d_buf + endoff; version = re->dw_read(d, &offset, 2); hdrlen = re->dw_read(d, &offset, dwarf_size); minlen = re->dw_read(d, &offset, 1); defstmt = re->dw_read(d, &offset, 1); lbase = re->dw_read(d, &offset, 1); lrange = re->dw_read(d, &offset, 1); opbase = re->dw_read(d, &offset, 1); printf("\n"); printf(" Length:\t\t\t%ju\n", (uintmax_t) length); printf(" DWARF version:\t\t%u\n", version); printf(" Prologue Length:\t\t%ju\n", (uintmax_t) hdrlen); printf(" Minimum Instruction Length:\t%u\n", minlen); printf(" Initial value of 'is_stmt':\t%u\n", defstmt); printf(" Line Base:\t\t\t%d\n", lbase); printf(" Line Range:\t\t\t%u\n", lrange); printf(" Opcode Base:\t\t\t%u\n", opbase); (void) dwarf_get_address_size(re->dbg, &pointer_size, &de); printf(" (Pointer size:\t\t%u)\n", pointer_size); printf("\n"); printf(" Opcodes:\n"); for (i = 1; i < opbase; i++) { oplen = re->dw_read(d, &offset, 1); printf(" Opcode %d has %u args\n", i, oplen); } printf("\n"); printf(" The Directory Table:\n"); p = (uint8_t *) d->d_buf + offset; while (*p != '\0') { printf(" %s\n", (char *) p); p += strlen((char *) p) + 1; } p++; printf("\n"); printf(" The File Name Table:\n"); printf(" Entry\tDir\tTime\tSize\tName\n"); i = 0; while (*p != '\0') { i++; pn = (char *) p; p += strlen(pn) + 1; dirndx = _decode_uleb128(&p, pe); mtime = _decode_uleb128(&p, pe); fsize = _decode_uleb128(&p, pe); printf(" %d\t%ju\t%ju\t%ju\t%s\n", i, (uintmax_t) dirndx, (uintmax_t) mtime, (uintmax_t) fsize, pn); } #define RESET_REGISTERS \ do { \ address = 0; \ file = 1; \ line = 1; \ column = 0; \ is_stmt = defstmt; \ } while(0) #define LINE(x) (lbase + (((x) - opbase) % lrange)) #define ADDRESS(x) ((((x) - opbase) / lrange) * minlen) p++; printf("\n"); printf(" Line Number Statements:\n"); RESET_REGISTERS; while (p < pe) { if (*p == 0) { /* * Extended Opcodes. */ p++; opsize = _decode_uleb128(&p, pe); printf(" Extended opcode %u: ", *p); switch (*p) { case DW_LNE_end_sequence: p++; RESET_REGISTERS; printf("End of Sequence\n"); break; case DW_LNE_set_address: p++; address = re->dw_decode(&p, pointer_size); printf("set Address to %#jx\n", (uintmax_t) address); break; case DW_LNE_define_file: p++; pn = (char *) p; p += strlen(pn) + 1; dirndx = _decode_uleb128(&p, pe); mtime = _decode_uleb128(&p, pe); fsize = _decode_uleb128(&p, pe); printf("define new file: %s\n", pn); break; default: /* Unrecognized extened opcodes. */ p += opsize; printf("unknown opcode\n"); } } else if (*p > 0 && *p < opbase) { /* * Standard Opcodes. */ switch(*p++) { case DW_LNS_copy: printf(" Copy\n"); break; case DW_LNS_advance_pc: udelta = _decode_uleb128(&p, pe) * minlen; address += udelta; printf(" Advance PC by %ju to %#jx\n", (uintmax_t) udelta, (uintmax_t) address); break; case DW_LNS_advance_line: sdelta = _decode_sleb128(&p, pe); line += sdelta; printf(" Advance Line by %jd to %ju\n", (intmax_t) sdelta, (uintmax_t) line); break; case DW_LNS_set_file: file = _decode_uleb128(&p, pe); printf(" Set File to %ju\n", (uintmax_t) file); break; case DW_LNS_set_column: column = _decode_uleb128(&p, pe); printf(" Set Column to %ju\n", (uintmax_t) column); break; case DW_LNS_negate_stmt: is_stmt = !is_stmt; printf(" Set is_stmt to %d\n", is_stmt); break; case DW_LNS_set_basic_block: printf(" Set basic block flag\n"); break; case DW_LNS_const_add_pc: address += ADDRESS(255); printf(" Advance PC by constant %ju" " to %#jx\n", (uintmax_t) ADDRESS(255), (uintmax_t) address); break; case DW_LNS_fixed_advance_pc: udelta = re->dw_decode(&p, 2); address += udelta; printf(" Advance PC by fixed value " "%ju to %#jx\n", (uintmax_t) udelta, (uintmax_t) address); break; case DW_LNS_set_prologue_end: printf(" Set prologue end flag\n"); break; case DW_LNS_set_epilogue_begin: printf(" Set epilogue begin flag\n"); break; case DW_LNS_set_isa: isa = _decode_uleb128(&p, pe); printf(" Set isa to %ju\n", (uintmax_t) isa); break; default: /* Unrecognized extended opcodes. */ printf(" Unknown extended opcode %u\n", *(p - 1)); break; } } else { /* * Special Opcodes. */ line += LINE(*p); address += ADDRESS(*p); printf(" Special opcode %u: advance Address " "by %ju to %#jx and Line by %jd to %ju\n", *p - opbase, (uintmax_t) ADDRESS(*p), (uintmax_t) address, (intmax_t) LINE(*p), (uintmax_t) line); p++; } } } if (ret == DW_DLV_ERROR) warnx("dwarf_next_cu_header: %s", dwarf_errmsg(de)); #undef RESET_REGISTERS #undef LINE #undef ADDRESS } static void dump_dwarf_line_decoded(struct readelf *re) { Dwarf_Die die; Dwarf_Line *linebuf, ln; Dwarf_Addr lineaddr; Dwarf_Signed linecount, srccount; Dwarf_Unsigned lineno, fn; Dwarf_Error de; const char *dir, *file; char **srcfiles; int i, ret; printf("Decoded dump of debug contents of section .debug_line:\n\n"); while ((ret = dwarf_next_cu_header(re->dbg, NULL, NULL, NULL, NULL, NULL, &de)) == DW_DLV_OK) { if (dwarf_siblingof(re->dbg, NULL, &die, &de) != DW_DLV_OK) continue; if (dwarf_attrval_string(die, DW_AT_name, &file, &de) != DW_DLV_OK) file = NULL; if (dwarf_attrval_string(die, DW_AT_comp_dir, &dir, &de) != DW_DLV_OK) dir = NULL; printf("CU: "); if (dir && file) printf("%s/", dir); if (file) printf("%s", file); putchar('\n'); printf("%-37s %11s %s\n", "Filename", "Line Number", "Starting Address"); if (dwarf_srclines(die, &linebuf, &linecount, &de) != DW_DLV_OK) continue; if (dwarf_srcfiles(die, &srcfiles, &srccount, &de) != DW_DLV_OK) continue; for (i = 0; i < linecount; i++) { ln = linebuf[i]; if (dwarf_line_srcfileno(ln, &fn, &de) != DW_DLV_OK) continue; if (dwarf_lineno(ln, &lineno, &de) != DW_DLV_OK) continue; if (dwarf_lineaddr(ln, &lineaddr, &de) != DW_DLV_OK) continue; printf("%-37s %11ju %#18jx\n", basename(srcfiles[fn - 1]), (uintmax_t) lineno, (uintmax_t) lineaddr); } putchar('\n'); } } static void dump_dwarf_die(struct readelf *re, Dwarf_Die die, int level) { Dwarf_Attribute *attr_list; Dwarf_Die ret_die; Dwarf_Off dieoff, cuoff, culen, attroff; Dwarf_Unsigned ate, lang, v_udata, v_sig; Dwarf_Signed attr_count, v_sdata; Dwarf_Off v_off; Dwarf_Addr v_addr; Dwarf_Half tag, attr, form; Dwarf_Block *v_block; Dwarf_Bool v_bool, is_info; Dwarf_Sig8 v_sig8; Dwarf_Error de; Dwarf_Ptr v_expr; const char *tag_str, *attr_str, *ate_str, *lang_str; char unk_tag[32], unk_attr[32]; char *v_str; uint8_t *b, *p; int i, j, abc, ret; if (dwarf_dieoffset(die, &dieoff, &de) != DW_DLV_OK) { warnx("dwarf_dieoffset failed: %s", dwarf_errmsg(de)); goto cont_search; } printf(" <%d><%jx>: ", level, (uintmax_t) dieoff); if (dwarf_die_CU_offset_range(die, &cuoff, &culen, &de) != DW_DLV_OK) { warnx("dwarf_die_CU_offset_range failed: %s", dwarf_errmsg(de)); cuoff = 0; } abc = dwarf_die_abbrev_code(die); if (dwarf_tag(die, &tag, &de) != DW_DLV_OK) { warnx("dwarf_tag failed: %s", dwarf_errmsg(de)); goto cont_search; } if (dwarf_get_TAG_name(tag, &tag_str) != DW_DLV_OK) { snprintf(unk_tag, sizeof(unk_tag), "[Unknown Tag: %#x]", tag); tag_str = unk_tag; } printf("Abbrev Number: %d (%s)\n", abc, tag_str); if ((ret = dwarf_attrlist(die, &attr_list, &attr_count, &de)) != DW_DLV_OK) { if (ret == DW_DLV_ERROR) warnx("dwarf_attrlist failed: %s", dwarf_errmsg(de)); goto cont_search; } for (i = 0; i < attr_count; i++) { if (dwarf_whatform(attr_list[i], &form, &de) != DW_DLV_OK) { warnx("dwarf_whatform failed: %s", dwarf_errmsg(de)); continue; } if (dwarf_whatattr(attr_list[i], &attr, &de) != DW_DLV_OK) { warnx("dwarf_whatattr failed: %s", dwarf_errmsg(de)); continue; } if (dwarf_get_AT_name(attr, &attr_str) != DW_DLV_OK) { snprintf(unk_attr, sizeof(unk_attr), "[Unknown AT: %#x]", attr); attr_str = unk_attr; } if (dwarf_attroffset(attr_list[i], &attroff, &de) != DW_DLV_OK) { warnx("dwarf_attroffset failed: %s", dwarf_errmsg(de)); attroff = 0; } printf(" <%jx> %-18s: ", (uintmax_t) attroff, attr_str); switch (form) { case DW_FORM_ref_addr: case DW_FORM_sec_offset: if (dwarf_global_formref(attr_list[i], &v_off, &de) != DW_DLV_OK) { warnx("dwarf_global_formref failed: %s", dwarf_errmsg(de)); continue; } if (form == DW_FORM_ref_addr) printf("<0x%jx>", (uintmax_t) v_off); else printf("0x%jx", (uintmax_t) v_off); break; case DW_FORM_ref1: case DW_FORM_ref2: case DW_FORM_ref4: case DW_FORM_ref8: case DW_FORM_ref_udata: if (dwarf_formref(attr_list[i], &v_off, &de) != DW_DLV_OK) { warnx("dwarf_formref failed: %s", dwarf_errmsg(de)); continue; } v_off += cuoff; printf("<0x%jx>", (uintmax_t) v_off); break; case DW_FORM_addr: if (dwarf_formaddr(attr_list[i], &v_addr, &de) != DW_DLV_OK) { warnx("dwarf_formaddr failed: %s", dwarf_errmsg(de)); continue; } printf("%#jx", (uintmax_t) v_addr); break; case DW_FORM_data1: case DW_FORM_data2: case DW_FORM_data4: case DW_FORM_data8: case DW_FORM_udata: if (dwarf_formudata(attr_list[i], &v_udata, &de) != DW_DLV_OK) { warnx("dwarf_formudata failed: %s", dwarf_errmsg(de)); continue; } if (attr == DW_AT_high_pc) printf("0x%jx", (uintmax_t) v_udata); else printf("%ju", (uintmax_t) v_udata); break; case DW_FORM_sdata: if (dwarf_formsdata(attr_list[i], &v_sdata, &de) != DW_DLV_OK) { warnx("dwarf_formudata failed: %s", dwarf_errmsg(de)); continue; } printf("%jd", (intmax_t) v_sdata); break; case DW_FORM_flag: if (dwarf_formflag(attr_list[i], &v_bool, &de) != DW_DLV_OK) { warnx("dwarf_formflag failed: %s", dwarf_errmsg(de)); continue; } printf("%jd", (intmax_t) v_bool); break; case DW_FORM_flag_present: putchar('1'); break; case DW_FORM_string: case DW_FORM_strp: if (dwarf_formstring(attr_list[i], &v_str, &de) != DW_DLV_OK) { warnx("dwarf_formstring failed: %s", dwarf_errmsg(de)); continue; } if (form == DW_FORM_string) printf("%s", v_str); else printf("(indirect string) %s", v_str); break; case DW_FORM_block: case DW_FORM_block1: case DW_FORM_block2: case DW_FORM_block4: if (dwarf_formblock(attr_list[i], &v_block, &de) != DW_DLV_OK) { warnx("dwarf_formblock failed: %s", dwarf_errmsg(de)); continue; } printf("%ju byte block:", (uintmax_t) v_block->bl_len); b = v_block->bl_data; for (j = 0; (Dwarf_Unsigned) j < v_block->bl_len; j++) printf(" %x", b[j]); printf("\t("); dump_dwarf_block(re, v_block->bl_data, v_block->bl_len); putchar(')'); break; case DW_FORM_exprloc: if (dwarf_formexprloc(attr_list[i], &v_udata, &v_expr, &de) != DW_DLV_OK) { warnx("dwarf_formexprloc failed: %s", dwarf_errmsg(de)); continue; } printf("%ju byte block:", (uintmax_t) v_udata); b = v_expr; for (j = 0; (Dwarf_Unsigned) j < v_udata; j++) printf(" %x", b[j]); printf("\t("); dump_dwarf_block(re, v_expr, v_udata); putchar(')'); break; case DW_FORM_ref_sig8: if (dwarf_formsig8(attr_list[i], &v_sig8, &de) != DW_DLV_OK) { warnx("dwarf_formsig8 failed: %s", dwarf_errmsg(de)); continue; } p = (uint8_t *)(uintptr_t) &v_sig8.signature[0]; v_sig = re->dw_decode(&p, 8); printf("signature: 0x%jx", (uintmax_t) v_sig); } switch (attr) { case DW_AT_encoding: if (dwarf_attrval_unsigned(die, attr, &ate, &de) != DW_DLV_OK) break; if (dwarf_get_ATE_name(ate, &ate_str) != DW_DLV_OK) ate_str = "DW_ATE_UNKNOWN"; printf("\t(%s)", &ate_str[strlen("DW_ATE_")]); break; case DW_AT_language: if (dwarf_attrval_unsigned(die, attr, &lang, &de) != DW_DLV_OK) break; if (dwarf_get_LANG_name(lang, &lang_str) != DW_DLV_OK) break; printf("\t(%s)", &lang_str[strlen("DW_LANG_")]); break; case DW_AT_location: case DW_AT_string_length: case DW_AT_return_addr: case DW_AT_data_member_location: case DW_AT_frame_base: case DW_AT_segment: case DW_AT_static_link: case DW_AT_use_location: case DW_AT_vtable_elem_location: switch (form) { case DW_FORM_data4: case DW_FORM_data8: case DW_FORM_sec_offset: printf("\t(location list)"); break; default: break; } default: break; } putchar('\n'); } cont_search: /* Search children. */ ret = dwarf_child(die, &ret_die, &de); if (ret == DW_DLV_ERROR) warnx("dwarf_child: %s", dwarf_errmsg(de)); else if (ret == DW_DLV_OK) dump_dwarf_die(re, ret_die, level + 1); /* Search sibling. */ is_info = dwarf_get_die_infotypes_flag(die); ret = dwarf_siblingof_b(re->dbg, die, &ret_die, is_info, &de); if (ret == DW_DLV_ERROR) warnx("dwarf_siblingof: %s", dwarf_errmsg(de)); else if (ret == DW_DLV_OK) dump_dwarf_die(re, ret_die, level); dwarf_dealloc(re->dbg, die, DW_DLA_DIE); } static void set_cu_context(struct readelf *re, Dwarf_Half psize, Dwarf_Half osize, Dwarf_Half ver) { re->cu_psize = psize; re->cu_osize = osize; re->cu_ver = ver; } static void dump_dwarf_info(struct readelf *re, Dwarf_Bool is_info) { struct section *s; Dwarf_Die die; Dwarf_Error de; Dwarf_Half tag, version, pointer_size, off_size; Dwarf_Off cu_offset, cu_length; Dwarf_Off aboff; Dwarf_Unsigned typeoff; Dwarf_Sig8 sig8; Dwarf_Unsigned sig; uint8_t *p; const char *sn; int i, ret; sn = is_info ? ".debug_info" : ".debug_types"; s = NULL; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->name != NULL && !strcmp(s->name, sn)) break; } if ((size_t) i >= re->shnum) return; do { printf("\nDump of debug contents of section %s:\n", sn); while ((ret = dwarf_next_cu_header_c(re->dbg, is_info, NULL, &version, &aboff, &pointer_size, &off_size, NULL, &sig8, &typeoff, NULL, &de)) == DW_DLV_OK) { set_cu_context(re, pointer_size, off_size, version); die = NULL; while (dwarf_siblingof_b(re->dbg, die, &die, is_info, &de) == DW_DLV_OK) { if (dwarf_tag(die, &tag, &de) != DW_DLV_OK) { warnx("dwarf_tag failed: %s", dwarf_errmsg(de)); continue; } /* XXX: What about DW_TAG_partial_unit? */ if ((is_info && tag == DW_TAG_compile_unit) || (!is_info && tag == DW_TAG_type_unit)) break; } if (die == NULL && is_info) { warnx("could not find DW_TAG_compile_unit " "die"); continue; } else if (die == NULL && !is_info) { warnx("could not find DW_TAG_type_unit die"); continue; } if (dwarf_die_CU_offset_range(die, &cu_offset, &cu_length, &de) != DW_DLV_OK) { warnx("dwarf_die_CU_offset failed: %s", dwarf_errmsg(de)); continue; } cu_length -= off_size == 4 ? 4 : 12; sig = 0; if (!is_info) { p = (uint8_t *)(uintptr_t) &sig8.signature[0]; sig = re->dw_decode(&p, 8); } printf("\n Type Unit @ offset 0x%jx:\n", (uintmax_t) cu_offset); printf(" Length:\t\t%#jx (%d-bit)\n", (uintmax_t) cu_length, off_size == 4 ? 32 : 64); printf(" Version:\t\t%u\n", version); printf(" Abbrev Offset:\t0x%jx\n", (uintmax_t) aboff); printf(" Pointer Size:\t%u\n", pointer_size); if (!is_info) { printf(" Signature:\t\t0x%016jx\n", (uintmax_t) sig); printf(" Type Offset:\t0x%jx\n", (uintmax_t) typeoff); } dump_dwarf_die(re, die, 0); } if (ret == DW_DLV_ERROR) warnx("dwarf_next_cu_header: %s", dwarf_errmsg(de)); if (is_info) break; } while (dwarf_next_types_section(re->dbg, &de) == DW_DLV_OK); } static void dump_dwarf_abbrev(struct readelf *re) { Dwarf_Abbrev ab; Dwarf_Off aboff, atoff; Dwarf_Unsigned length, attr_count; Dwarf_Signed flag, form; Dwarf_Half tag, attr; Dwarf_Error de; const char *tag_str, *attr_str, *form_str; char unk_tag[32], unk_attr[32], unk_form[32]; int i, j, ret; printf("\nContents of section .debug_abbrev:\n\n"); while ((ret = dwarf_next_cu_header(re->dbg, NULL, NULL, &aboff, NULL, NULL, &de)) == DW_DLV_OK) { printf(" Number TAG\n"); i = 0; while ((ret = dwarf_get_abbrev(re->dbg, aboff, &ab, &length, &attr_count, &de)) == DW_DLV_OK) { if (length == 1) { dwarf_dealloc(re->dbg, ab, DW_DLA_ABBREV); break; } aboff += length; printf("%4d", ++i); if (dwarf_get_abbrev_tag(ab, &tag, &de) != DW_DLV_OK) { warnx("dwarf_get_abbrev_tag failed: %s", dwarf_errmsg(de)); goto next_abbrev; } if (dwarf_get_TAG_name(tag, &tag_str) != DW_DLV_OK) { snprintf(unk_tag, sizeof(unk_tag), "[Unknown Tag: %#x]", tag); tag_str = unk_tag; } if (dwarf_get_abbrev_children_flag(ab, &flag, &de) != DW_DLV_OK) { warnx("dwarf_get_abbrev_children_flag failed:" " %s", dwarf_errmsg(de)); goto next_abbrev; } printf(" %s %s\n", tag_str, flag ? "[has children]" : "[no children]"); for (j = 0; (Dwarf_Unsigned) j < attr_count; j++) { if (dwarf_get_abbrev_entry(ab, (Dwarf_Signed) j, &attr, &form, &atoff, &de) != DW_DLV_OK) { warnx("dwarf_get_abbrev_entry failed:" " %s", dwarf_errmsg(de)); continue; } if (dwarf_get_AT_name(attr, &attr_str) != DW_DLV_OK) { snprintf(unk_attr, sizeof(unk_attr), "[Unknown AT: %#x]", attr); attr_str = unk_attr; } if (dwarf_get_FORM_name(form, &form_str) != DW_DLV_OK) { snprintf(unk_form, sizeof(unk_form), "[Unknown Form: %#x]", (Dwarf_Half) form); form_str = unk_form; } printf(" %-18s %s\n", attr_str, form_str); } next_abbrev: dwarf_dealloc(re->dbg, ab, DW_DLA_ABBREV); } if (ret != DW_DLV_OK) warnx("dwarf_get_abbrev: %s", dwarf_errmsg(de)); } if (ret == DW_DLV_ERROR) warnx("dwarf_next_cu_header: %s", dwarf_errmsg(de)); } static void dump_dwarf_pubnames(struct readelf *re) { struct section *s; Dwarf_Off die_off; Dwarf_Unsigned offset, length, nt_cu_offset, nt_cu_length; Dwarf_Signed cnt; Dwarf_Global *globs; Dwarf_Half nt_version; Dwarf_Error de; Elf_Data *d; char *glob_name; int i, dwarf_size, elferr; printf("\nContents of the .debug_pubnames section:\n"); s = NULL; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->name != NULL && !strcmp(s->name, ".debug_pubnames")) break; } if ((size_t) i >= re->shnum) return; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); return; } if (d->d_size <= 0) return; /* Read in .debug_pubnames section table header. */ offset = 0; length = re->dw_read(d, &offset, 4); if (length == 0xffffffff) { dwarf_size = 8; length = re->dw_read(d, &offset, 8); } else dwarf_size = 4; if (length > d->d_size - offset) { warnx("invalid .dwarf_pubnames section"); return; } nt_version = re->dw_read(d, &offset, 2); nt_cu_offset = re->dw_read(d, &offset, dwarf_size); nt_cu_length = re->dw_read(d, &offset, dwarf_size); printf(" Length:\t\t\t\t%ju\n", (uintmax_t) length); printf(" Version:\t\t\t\t%u\n", nt_version); printf(" Offset into .debug_info section:\t%ju\n", (uintmax_t) nt_cu_offset); printf(" Size of area in .debug_info section:\t%ju\n", (uintmax_t) nt_cu_length); if (dwarf_get_globals(re->dbg, &globs, &cnt, &de) != DW_DLV_OK) { warnx("dwarf_get_globals failed: %s", dwarf_errmsg(de)); return; } printf("\n Offset Name\n"); for (i = 0; i < cnt; i++) { if (dwarf_globname(globs[i], &glob_name, &de) != DW_DLV_OK) { warnx("dwarf_globname failed: %s", dwarf_errmsg(de)); continue; } if (dwarf_global_die_offset(globs[i], &die_off, &de) != DW_DLV_OK) { warnx("dwarf_global_die_offset failed: %s", dwarf_errmsg(de)); continue; } printf(" %-11ju %s\n", (uintmax_t) die_off, glob_name); } } static void dump_dwarf_aranges(struct readelf *re) { struct section *s; Dwarf_Arange *aranges; Dwarf_Addr start; Dwarf_Unsigned offset, length, as_cu_offset; Dwarf_Off die_off; Dwarf_Signed cnt; Dwarf_Half as_version, as_addrsz, as_segsz; Dwarf_Error de; Elf_Data *d; int i, dwarf_size, elferr; printf("\nContents of section .debug_aranges:\n"); s = NULL; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->name != NULL && !strcmp(s->name, ".debug_aranges")) break; } if ((size_t) i >= re->shnum) return; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); return; } if (d->d_size <= 0) return; /* Read in the .debug_aranges section table header. */ offset = 0; length = re->dw_read(d, &offset, 4); if (length == 0xffffffff) { dwarf_size = 8; length = re->dw_read(d, &offset, 8); } else dwarf_size = 4; if (length > d->d_size - offset) { warnx("invalid .dwarf_aranges section"); return; } as_version = re->dw_read(d, &offset, 2); as_cu_offset = re->dw_read(d, &offset, dwarf_size); as_addrsz = re->dw_read(d, &offset, 1); as_segsz = re->dw_read(d, &offset, 1); printf(" Length:\t\t\t%ju\n", (uintmax_t) length); printf(" Version:\t\t\t%u\n", as_version); printf(" Offset into .debug_info:\t%ju\n", (uintmax_t) as_cu_offset); printf(" Pointer Size:\t\t\t%u\n", as_addrsz); printf(" Segment Size:\t\t\t%u\n", as_segsz); if (dwarf_get_aranges(re->dbg, &aranges, &cnt, &de) != DW_DLV_OK) { warnx("dwarf_get_aranges failed: %s", dwarf_errmsg(de)); return; } printf("\n Address Length\n"); for (i = 0; i < cnt; i++) { if (dwarf_get_arange_info(aranges[i], &start, &length, &die_off, &de) != DW_DLV_OK) { warnx("dwarf_get_arange_info failed: %s", dwarf_errmsg(de)); continue; } printf(" %08jx %ju\n", (uintmax_t) start, (uintmax_t) length); } } static void dump_dwarf_ranges_foreach(struct readelf *re, Dwarf_Die die, Dwarf_Addr base) { Dwarf_Attribute *attr_list; Dwarf_Ranges *ranges; Dwarf_Die ret_die; Dwarf_Error de; Dwarf_Addr base0; Dwarf_Half attr; Dwarf_Signed attr_count, cnt; Dwarf_Unsigned off, bytecnt; int i, j, ret; if ((ret = dwarf_attrlist(die, &attr_list, &attr_count, &de)) != DW_DLV_OK) { if (ret == DW_DLV_ERROR) warnx("dwarf_attrlist failed: %s", dwarf_errmsg(de)); goto cont_search; } for (i = 0; i < attr_count; i++) { if (dwarf_whatattr(attr_list[i], &attr, &de) != DW_DLV_OK) { warnx("dwarf_whatattr failed: %s", dwarf_errmsg(de)); continue; } if (attr != DW_AT_ranges) continue; if (dwarf_formudata(attr_list[i], &off, &de) != DW_DLV_OK) { warnx("dwarf_formudata failed: %s", dwarf_errmsg(de)); continue; } if (dwarf_get_ranges(re->dbg, (Dwarf_Off) off, &ranges, &cnt, &bytecnt, &de) != DW_DLV_OK) continue; base0 = base; for (j = 0; j < cnt; j++) { printf(" %08jx ", (uintmax_t) off); if (ranges[j].dwr_type == DW_RANGES_END) { printf("%s\n", ""); continue; } else if (ranges[j].dwr_type == DW_RANGES_ADDRESS_SELECTION) { base0 = ranges[j].dwr_addr2; continue; } if (re->ec == ELFCLASS32) printf("%08jx %08jx\n", (uintmax_t) (ranges[j].dwr_addr1 + base0), (uintmax_t) (ranges[j].dwr_addr2 + base0)); else printf("%016jx %016jx\n", (uintmax_t) (ranges[j].dwr_addr1 + base0), (uintmax_t) (ranges[j].dwr_addr2 + base0)); } } cont_search: /* Search children. */ ret = dwarf_child(die, &ret_die, &de); if (ret == DW_DLV_ERROR) warnx("dwarf_child: %s", dwarf_errmsg(de)); else if (ret == DW_DLV_OK) dump_dwarf_ranges_foreach(re, ret_die, base); /* Search sibling. */ ret = dwarf_siblingof(re->dbg, die, &ret_die, &de); if (ret == DW_DLV_ERROR) warnx("dwarf_siblingof: %s", dwarf_errmsg(de)); else if (ret == DW_DLV_OK) dump_dwarf_ranges_foreach(re, ret_die, base); } static void dump_dwarf_ranges(struct readelf *re) { Dwarf_Ranges *ranges; Dwarf_Die die; Dwarf_Signed cnt; Dwarf_Unsigned bytecnt; Dwarf_Half tag; Dwarf_Error de; Dwarf_Unsigned lowpc; int ret; if (dwarf_get_ranges(re->dbg, 0, &ranges, &cnt, &bytecnt, &de) != DW_DLV_OK) return; printf("Contents of the .debug_ranges section:\n\n"); if (re->ec == ELFCLASS32) printf(" %-8s %-8s %s\n", "Offset", "Begin", "End"); else printf(" %-8s %-16s %s\n", "Offset", "Begin", "End"); while ((ret = dwarf_next_cu_header(re->dbg, NULL, NULL, NULL, NULL, NULL, &de)) == DW_DLV_OK) { die = NULL; if (dwarf_siblingof(re->dbg, die, &die, &de) != DW_DLV_OK) continue; if (dwarf_tag(die, &tag, &de) != DW_DLV_OK) { warnx("dwarf_tag failed: %s", dwarf_errmsg(de)); continue; } /* XXX: What about DW_TAG_partial_unit? */ lowpc = 0; if (tag == DW_TAG_compile_unit) { if (dwarf_attrval_unsigned(die, DW_AT_low_pc, &lowpc, &de) != DW_DLV_OK) lowpc = 0; } dump_dwarf_ranges_foreach(re, die, (Dwarf_Addr) lowpc); } putchar('\n'); } static void dump_dwarf_macinfo(struct readelf *re) { Dwarf_Unsigned offset; Dwarf_Signed cnt; Dwarf_Macro_Details *md; Dwarf_Error de; const char *mi_str; char unk_mi[32]; int i; #define _MAX_MACINFO_ENTRY 65535 printf("\nContents of section .debug_macinfo:\n\n"); offset = 0; while (dwarf_get_macro_details(re->dbg, offset, _MAX_MACINFO_ENTRY, &cnt, &md, &de) == DW_DLV_OK) { for (i = 0; i < cnt; i++) { offset = md[i].dmd_offset + 1; if (md[i].dmd_type == 0) break; if (dwarf_get_MACINFO_name(md[i].dmd_type, &mi_str) != DW_DLV_OK) { snprintf(unk_mi, sizeof(unk_mi), "[Unknown MACINFO: %#x]", md[i].dmd_type); mi_str = unk_mi; } printf(" %s", mi_str); switch (md[i].dmd_type) { case DW_MACINFO_define: case DW_MACINFO_undef: printf(" - lineno : %jd macro : %s\n", (intmax_t) md[i].dmd_lineno, md[i].dmd_macro); break; case DW_MACINFO_start_file: printf(" - lineno : %jd filenum : %jd\n", (intmax_t) md[i].dmd_lineno, (intmax_t) md[i].dmd_fileindex); break; default: putchar('\n'); break; } } } #undef _MAX_MACINFO_ENTRY } static void dump_dwarf_frame_inst(struct readelf *re, Dwarf_Cie cie, uint8_t *insts, Dwarf_Unsigned len, Dwarf_Unsigned caf, Dwarf_Signed daf, Dwarf_Addr pc, Dwarf_Debug dbg) { Dwarf_Frame_Op *oplist; Dwarf_Signed opcnt, delta; Dwarf_Small op; Dwarf_Error de; const char *op_str; char unk_op[32]; int i; if (dwarf_expand_frame_instructions(cie, insts, len, &oplist, &opcnt, &de) != DW_DLV_OK) { warnx("dwarf_expand_frame_instructions failed: %s", dwarf_errmsg(de)); return; } for (i = 0; i < opcnt; i++) { if (oplist[i].fp_base_op != 0) op = oplist[i].fp_base_op << 6; else op = oplist[i].fp_extended_op; if (dwarf_get_CFA_name(op, &op_str) != DW_DLV_OK) { snprintf(unk_op, sizeof(unk_op), "[Unknown CFA: %#x]", op); op_str = unk_op; } printf(" %s", op_str); switch (op) { case DW_CFA_advance_loc: delta = oplist[i].fp_offset * caf; pc += delta; printf(": %ju to %08jx", (uintmax_t) delta, (uintmax_t) pc); break; case DW_CFA_offset: case DW_CFA_offset_extended: case DW_CFA_offset_extended_sf: delta = oplist[i].fp_offset * daf; printf(": r%u (%s) at cfa%+jd", oplist[i].fp_register, dwarf_regname(re, oplist[i].fp_register), (intmax_t) delta); break; case DW_CFA_restore: printf(": r%u (%s)", oplist[i].fp_register, dwarf_regname(re, oplist[i].fp_register)); break; case DW_CFA_set_loc: pc = oplist[i].fp_offset; printf(": to %08jx", (uintmax_t) pc); break; case DW_CFA_advance_loc1: case DW_CFA_advance_loc2: case DW_CFA_advance_loc4: pc += oplist[i].fp_offset; printf(": %jd to %08jx", (intmax_t) oplist[i].fp_offset, (uintmax_t) pc); break; case DW_CFA_def_cfa: printf(": r%u (%s) ofs %ju", oplist[i].fp_register, dwarf_regname(re, oplist[i].fp_register), (uintmax_t) oplist[i].fp_offset); break; case DW_CFA_def_cfa_sf: printf(": r%u (%s) ofs %jd", oplist[i].fp_register, dwarf_regname(re, oplist[i].fp_register), (intmax_t) (oplist[i].fp_offset * daf)); break; case DW_CFA_def_cfa_register: printf(": r%u (%s)", oplist[i].fp_register, dwarf_regname(re, oplist[i].fp_register)); break; case DW_CFA_def_cfa_offset: printf(": %ju", (uintmax_t) oplist[i].fp_offset); break; case DW_CFA_def_cfa_offset_sf: printf(": %jd", (intmax_t) (oplist[i].fp_offset * daf)); break; default: break; } putchar('\n'); } dwarf_dealloc(dbg, oplist, DW_DLA_FRAME_BLOCK); } static char * get_regoff_str(struct readelf *re, Dwarf_Half reg, Dwarf_Addr off) { static char rs[16]; if (reg == DW_FRAME_UNDEFINED_VAL || reg == DW_FRAME_REG_INITIAL_VALUE) snprintf(rs, sizeof(rs), "%c", 'u'); else if (reg == DW_FRAME_CFA_COL) snprintf(rs, sizeof(rs), "c%+jd", (intmax_t) off); else snprintf(rs, sizeof(rs), "%s%+jd", dwarf_regname(re, reg), (intmax_t) off); return (rs); } static int dump_dwarf_frame_regtable(struct readelf *re, Dwarf_Fde fde, Dwarf_Addr pc, Dwarf_Unsigned func_len, Dwarf_Half cie_ra) { Dwarf_Regtable rt; Dwarf_Addr row_pc, end_pc, pre_pc, cur_pc; Dwarf_Error de; char *vec; int i; #define BIT_SET(v, n) (v[(n)>>3] |= 1U << ((n) & 7)) #define BIT_CLR(v, n) (v[(n)>>3] &= ~(1U << ((n) & 7))) #define BIT_ISSET(v, n) (v[(n)>>3] & (1U << ((n) & 7))) #define RT(x) rt.rules[(x)] vec = calloc((DW_REG_TABLE_SIZE + 7) / 8, 1); if (vec == NULL) err(EXIT_FAILURE, "calloc failed"); pre_pc = ~((Dwarf_Addr) 0); cur_pc = pc; end_pc = pc + func_len; for (; cur_pc < end_pc; cur_pc++) { if (dwarf_get_fde_info_for_all_regs(fde, cur_pc, &rt, &row_pc, &de) != DW_DLV_OK) { warnx("dwarf_get_fde_info_for_all_regs failed: %s\n", dwarf_errmsg(de)); return (-1); } if (row_pc == pre_pc) continue; pre_pc = row_pc; for (i = 1; i < DW_REG_TABLE_SIZE; i++) { if (rt.rules[i].dw_regnum != DW_FRAME_REG_INITIAL_VALUE) BIT_SET(vec, i); } } printf(" LOC CFA "); for (i = 1; i < DW_REG_TABLE_SIZE; i++) { if (BIT_ISSET(vec, i)) { if ((Dwarf_Half) i == cie_ra) printf("ra "); else printf("%-5s", dwarf_regname(re, (unsigned int) i)); } } putchar('\n'); pre_pc = ~((Dwarf_Addr) 0); cur_pc = pc; end_pc = pc + func_len; for (; cur_pc < end_pc; cur_pc++) { if (dwarf_get_fde_info_for_all_regs(fde, cur_pc, &rt, &row_pc, &de) != DW_DLV_OK) { warnx("dwarf_get_fde_info_for_all_regs failed: %s\n", dwarf_errmsg(de)); return (-1); } if (row_pc == pre_pc) continue; pre_pc = row_pc; printf("%08jx ", (uintmax_t) row_pc); printf("%-8s ", get_regoff_str(re, RT(0).dw_regnum, RT(0).dw_offset)); for (i = 1; i < DW_REG_TABLE_SIZE; i++) { if (BIT_ISSET(vec, i)) { printf("%-5s", get_regoff_str(re, RT(i).dw_regnum, RT(i).dw_offset)); } } putchar('\n'); } free(vec); return (0); #undef BIT_SET #undef BIT_CLR #undef BIT_ISSET #undef RT } static void dump_dwarf_frame_section(struct readelf *re, struct section *s, int alt) { Dwarf_Cie *cie_list, cie, pre_cie; Dwarf_Fde *fde_list, fde; Dwarf_Off cie_offset, fde_offset; Dwarf_Unsigned cie_length, fde_instlen; Dwarf_Unsigned cie_caf, cie_daf, cie_instlen, func_len, fde_length; Dwarf_Signed cie_count, fde_count, cie_index; Dwarf_Addr low_pc; Dwarf_Half cie_ra; Dwarf_Small cie_version; Dwarf_Ptr fde_addr, fde_inst, cie_inst; char *cie_aug, c; int i, eh_frame; Dwarf_Error de; printf("\nThe section %s contains:\n\n", s->name); if (!strcmp(s->name, ".debug_frame")) { eh_frame = 0; if (dwarf_get_fde_list(re->dbg, &cie_list, &cie_count, &fde_list, &fde_count, &de) != DW_DLV_OK) { warnx("dwarf_get_fde_list failed: %s", dwarf_errmsg(de)); return; } } else if (!strcmp(s->name, ".eh_frame")) { eh_frame = 1; if (dwarf_get_fde_list_eh(re->dbg, &cie_list, &cie_count, &fde_list, &fde_count, &de) != DW_DLV_OK) { warnx("dwarf_get_fde_list_eh failed: %s", dwarf_errmsg(de)); return; } } else return; pre_cie = NULL; for (i = 0; i < fde_count; i++) { if (dwarf_get_fde_n(fde_list, i, &fde, &de) != DW_DLV_OK) { warnx("dwarf_get_fde_n failed: %s", dwarf_errmsg(de)); continue; } if (dwarf_get_cie_of_fde(fde, &cie, &de) != DW_DLV_OK) { warnx("dwarf_get_fde_n failed: %s", dwarf_errmsg(de)); continue; } if (dwarf_get_fde_range(fde, &low_pc, &func_len, &fde_addr, &fde_length, &cie_offset, &cie_index, &fde_offset, &de) != DW_DLV_OK) { warnx("dwarf_get_fde_range failed: %s", dwarf_errmsg(de)); continue; } if (dwarf_get_fde_instr_bytes(fde, &fde_inst, &fde_instlen, &de) != DW_DLV_OK) { warnx("dwarf_get_fde_instr_bytes failed: %s", dwarf_errmsg(de)); continue; } if (pre_cie == NULL || cie != pre_cie) { pre_cie = cie; if (dwarf_get_cie_info(cie, &cie_length, &cie_version, &cie_aug, &cie_caf, &cie_daf, &cie_ra, &cie_inst, &cie_instlen, &de) != DW_DLV_OK) { warnx("dwarf_get_cie_info failed: %s", dwarf_errmsg(de)); continue; } printf("%08jx %08jx %8.8jx CIE", (uintmax_t) cie_offset, (uintmax_t) cie_length, (uintmax_t) (eh_frame ? 0 : ~0U)); if (!alt) { putchar('\n'); printf(" Version:\t\t\t%u\n", cie_version); printf(" Augmentation:\t\t\t\""); while ((c = *cie_aug++) != '\0') putchar(c); printf("\"\n"); printf(" Code alignment factor:\t%ju\n", (uintmax_t) cie_caf); printf(" Data alignment factor:\t%jd\n", (intmax_t) cie_daf); printf(" Return address column:\t%ju\n", (uintmax_t) cie_ra); putchar('\n'); dump_dwarf_frame_inst(re, cie, cie_inst, cie_instlen, cie_caf, cie_daf, 0, re->dbg); putchar('\n'); } else { printf(" \""); while ((c = *cie_aug++) != '\0') putchar(c); putchar('"'); printf(" cf=%ju df=%jd ra=%ju\n", (uintmax_t) cie_caf, (uintmax_t) cie_daf, (uintmax_t) cie_ra); dump_dwarf_frame_regtable(re, fde, low_pc, 1, cie_ra); putchar('\n'); } } printf("%08jx %08jx %08jx FDE cie=%08jx pc=%08jx..%08jx\n", (uintmax_t) fde_offset, (uintmax_t) fde_length, (uintmax_t) cie_offset, (uintmax_t) (eh_frame ? fde_offset + 4 - cie_offset : cie_offset), (uintmax_t) low_pc, (uintmax_t) (low_pc + func_len)); if (!alt) dump_dwarf_frame_inst(re, cie, fde_inst, fde_instlen, cie_caf, cie_daf, low_pc, re->dbg); else dump_dwarf_frame_regtable(re, fde, low_pc, func_len, cie_ra); putchar('\n'); } } static void dump_dwarf_frame(struct readelf *re, int alt) { struct section *s; int i; (void) dwarf_set_frame_cfa_value(re->dbg, DW_FRAME_CFA_COL); for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->name != NULL && (!strcmp(s->name, ".debug_frame") || !strcmp(s->name, ".eh_frame"))) dump_dwarf_frame_section(re, s, alt); } } static void dump_dwarf_str(struct readelf *re) { struct section *s; Elf_Data *d; unsigned char *p; int elferr, end, i, j; printf("\nContents of section .debug_str:\n"); s = NULL; for (i = 0; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (s->name != NULL && !strcmp(s->name, ".debug_str")) break; } if ((size_t) i >= re->shnum) return; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(-1)); return; } if (d->d_size <= 0) return; for (i = 0, p = d->d_buf; (size_t) i < d->d_size; i += 16) { printf(" 0x%08x", (unsigned int) i); if ((size_t) i + 16 > d->d_size) end = d->d_size; else end = i + 16; for (j = i; j < i + 16; j++) { if ((j - i) % 4 == 0) putchar(' '); if (j >= end) { printf(" "); continue; } printf("%02x", (uint8_t) p[j]); } putchar(' '); for (j = i; j < end; j++) { if (isprint(p[j])) putchar(p[j]); else if (p[j] == 0) putchar('.'); else putchar(' '); } putchar('\n'); } } struct loc_at { Dwarf_Attribute la_at; Dwarf_Unsigned la_off; Dwarf_Unsigned la_lowpc; Dwarf_Half la_cu_psize; Dwarf_Half la_cu_osize; Dwarf_Half la_cu_ver; TAILQ_ENTRY(loc_at) la_next; }; static TAILQ_HEAD(, loc_at) lalist = TAILQ_HEAD_INITIALIZER(lalist); static void search_loclist_at(struct readelf *re, Dwarf_Die die, Dwarf_Unsigned lowpc) { Dwarf_Attribute *attr_list; Dwarf_Die ret_die; Dwarf_Unsigned off; Dwarf_Off ref; Dwarf_Signed attr_count; Dwarf_Half attr, form; Dwarf_Bool is_info; Dwarf_Error de; struct loc_at *la, *nla; int i, ret; is_info = dwarf_get_die_infotypes_flag(die); if ((ret = dwarf_attrlist(die, &attr_list, &attr_count, &de)) != DW_DLV_OK) { if (ret == DW_DLV_ERROR) warnx("dwarf_attrlist failed: %s", dwarf_errmsg(de)); goto cont_search; } for (i = 0; i < attr_count; i++) { if (dwarf_whatattr(attr_list[i], &attr, &de) != DW_DLV_OK) { warnx("dwarf_whatattr failed: %s", dwarf_errmsg(de)); continue; } if (attr != DW_AT_location && attr != DW_AT_string_length && attr != DW_AT_return_addr && attr != DW_AT_data_member_location && attr != DW_AT_frame_base && attr != DW_AT_segment && attr != DW_AT_static_link && attr != DW_AT_use_location && attr != DW_AT_vtable_elem_location) continue; if (dwarf_whatform(attr_list[i], &form, &de) != DW_DLV_OK) { warnx("dwarf_whatform failed: %s", dwarf_errmsg(de)); continue; } if (form == DW_FORM_data4 || form == DW_FORM_data8) { if (dwarf_formudata(attr_list[i], &off, &de) != DW_DLV_OK) { warnx("dwarf_formudata failed: %s", dwarf_errmsg(de)); continue; } } else if (form == DW_FORM_sec_offset) { if (dwarf_global_formref(attr_list[i], &ref, &de) != DW_DLV_OK) { warnx("dwarf_global_formref failed: %s", dwarf_errmsg(de)); continue; } off = ref; } else continue; TAILQ_FOREACH(la, &lalist, la_next) { if (off == la->la_off) break; if (off < la->la_off) { if ((nla = malloc(sizeof(*nla))) == NULL) err(EXIT_FAILURE, "malloc failed"); nla->la_at = attr_list[i]; nla->la_off = off; nla->la_lowpc = lowpc; nla->la_cu_psize = re->cu_psize; nla->la_cu_osize = re->cu_osize; nla->la_cu_ver = re->cu_ver; TAILQ_INSERT_BEFORE(la, nla, la_next); break; } } if (la == NULL) { if ((nla = malloc(sizeof(*nla))) == NULL) err(EXIT_FAILURE, "malloc failed"); nla->la_at = attr_list[i]; nla->la_off = off; nla->la_lowpc = lowpc; nla->la_cu_psize = re->cu_psize; nla->la_cu_osize = re->cu_osize; nla->la_cu_ver = re->cu_ver; TAILQ_INSERT_TAIL(&lalist, nla, la_next); } } cont_search: /* Search children. */ ret = dwarf_child(die, &ret_die, &de); if (ret == DW_DLV_ERROR) warnx("dwarf_child: %s", dwarf_errmsg(de)); else if (ret == DW_DLV_OK) search_loclist_at(re, ret_die, lowpc); /* Search sibling. */ ret = dwarf_siblingof_b(re->dbg, die, &ret_die, is_info, &de); if (ret == DW_DLV_ERROR) warnx("dwarf_siblingof: %s", dwarf_errmsg(de)); else if (ret == DW_DLV_OK) search_loclist_at(re, ret_die, lowpc); } static void dump_dwarf_loc(struct readelf *re, Dwarf_Loc *lr) { const char *op_str; char unk_op[32]; uint8_t *b, n; int i; if (dwarf_get_OP_name(lr->lr_atom, &op_str) != DW_DLV_OK) { snprintf(unk_op, sizeof(unk_op), "[Unknown OP: %#x]", lr->lr_atom); op_str = unk_op; } printf("%s", op_str); switch (lr->lr_atom) { case DW_OP_reg0: case DW_OP_reg1: case DW_OP_reg2: case DW_OP_reg3: case DW_OP_reg4: case DW_OP_reg5: case DW_OP_reg6: case DW_OP_reg7: case DW_OP_reg8: case DW_OP_reg9: case DW_OP_reg10: case DW_OP_reg11: case DW_OP_reg12: case DW_OP_reg13: case DW_OP_reg14: case DW_OP_reg15: case DW_OP_reg16: case DW_OP_reg17: case DW_OP_reg18: case DW_OP_reg19: case DW_OP_reg20: case DW_OP_reg21: case DW_OP_reg22: case DW_OP_reg23: case DW_OP_reg24: case DW_OP_reg25: case DW_OP_reg26: case DW_OP_reg27: case DW_OP_reg28: case DW_OP_reg29: case DW_OP_reg30: case DW_OP_reg31: printf(" (%s)", dwarf_regname(re, lr->lr_atom - DW_OP_reg0)); break; case DW_OP_deref: case DW_OP_lit0: case DW_OP_lit1: case DW_OP_lit2: case DW_OP_lit3: case DW_OP_lit4: case DW_OP_lit5: case DW_OP_lit6: case DW_OP_lit7: case DW_OP_lit8: case DW_OP_lit9: case DW_OP_lit10: case DW_OP_lit11: case DW_OP_lit12: case DW_OP_lit13: case DW_OP_lit14: case DW_OP_lit15: case DW_OP_lit16: case DW_OP_lit17: case DW_OP_lit18: case DW_OP_lit19: case DW_OP_lit20: case DW_OP_lit21: case DW_OP_lit22: case DW_OP_lit23: case DW_OP_lit24: case DW_OP_lit25: case DW_OP_lit26: case DW_OP_lit27: case DW_OP_lit28: case DW_OP_lit29: case DW_OP_lit30: case DW_OP_lit31: case DW_OP_dup: case DW_OP_drop: case DW_OP_over: case DW_OP_swap: case DW_OP_rot: case DW_OP_xderef: case DW_OP_abs: case DW_OP_and: case DW_OP_div: case DW_OP_minus: case DW_OP_mod: case DW_OP_mul: case DW_OP_neg: case DW_OP_not: case DW_OP_or: case DW_OP_plus: case DW_OP_shl: case DW_OP_shr: case DW_OP_shra: case DW_OP_xor: case DW_OP_eq: case DW_OP_ge: case DW_OP_gt: case DW_OP_le: case DW_OP_lt: case DW_OP_ne: case DW_OP_nop: case DW_OP_push_object_address: case DW_OP_form_tls_address: case DW_OP_call_frame_cfa: case DW_OP_stack_value: case DW_OP_GNU_push_tls_address: case DW_OP_GNU_uninit: break; case DW_OP_const1u: case DW_OP_pick: case DW_OP_deref_size: case DW_OP_xderef_size: case DW_OP_const2u: case DW_OP_bra: case DW_OP_skip: case DW_OP_const4u: case DW_OP_const8u: case DW_OP_constu: case DW_OP_plus_uconst: case DW_OP_regx: case DW_OP_piece: printf(": %ju", (uintmax_t) lr->lr_number); break; case DW_OP_const1s: case DW_OP_const2s: case DW_OP_const4s: case DW_OP_const8s: case DW_OP_consts: printf(": %jd", (intmax_t) lr->lr_number); break; case DW_OP_breg0: case DW_OP_breg1: case DW_OP_breg2: case DW_OP_breg3: case DW_OP_breg4: case DW_OP_breg5: case DW_OP_breg6: case DW_OP_breg7: case DW_OP_breg8: case DW_OP_breg9: case DW_OP_breg10: case DW_OP_breg11: case DW_OP_breg12: case DW_OP_breg13: case DW_OP_breg14: case DW_OP_breg15: case DW_OP_breg16: case DW_OP_breg17: case DW_OP_breg18: case DW_OP_breg19: case DW_OP_breg20: case DW_OP_breg21: case DW_OP_breg22: case DW_OP_breg23: case DW_OP_breg24: case DW_OP_breg25: case DW_OP_breg26: case DW_OP_breg27: case DW_OP_breg28: case DW_OP_breg29: case DW_OP_breg30: case DW_OP_breg31: printf(" (%s): %jd", dwarf_regname(re, lr->lr_atom - DW_OP_breg0), (intmax_t) lr->lr_number); break; case DW_OP_fbreg: printf(": %jd", (intmax_t) lr->lr_number); break; case DW_OP_bregx: printf(": %ju (%s) %jd", (uintmax_t) lr->lr_number, dwarf_regname(re, (unsigned int) lr->lr_number), (intmax_t) lr->lr_number2); break; case DW_OP_addr: case DW_OP_GNU_encoded_addr: printf(": %#jx", (uintmax_t) lr->lr_number); break; case DW_OP_GNU_implicit_pointer: printf(": <0x%jx> %jd", (uintmax_t) lr->lr_number, (intmax_t) lr->lr_number2); break; case DW_OP_implicit_value: printf(": %ju byte block:", (uintmax_t) lr->lr_number); b = (uint8_t *)(uintptr_t) lr->lr_number2; for (i = 0; (Dwarf_Unsigned) i < lr->lr_number; i++) printf(" %x", b[i]); break; case DW_OP_GNU_entry_value: printf(": ("); dump_dwarf_block(re, (uint8_t *)(uintptr_t) lr->lr_number2, lr->lr_number); putchar(')'); break; case DW_OP_GNU_const_type: printf(": <0x%jx> ", (uintmax_t) lr->lr_number); b = (uint8_t *)(uintptr_t) lr->lr_number2; n = *b; for (i = 1; (uint8_t) i < n; i++) printf(" %x", b[i]); break; case DW_OP_GNU_regval_type: printf(": %ju (%s) <0x%jx>", (uintmax_t) lr->lr_number, dwarf_regname(re, (unsigned int) lr->lr_number), (uintmax_t) lr->lr_number2); break; case DW_OP_GNU_convert: case DW_OP_GNU_deref_type: case DW_OP_GNU_parameter_ref: case DW_OP_GNU_reinterpret: printf(": <0x%jx>", (uintmax_t) lr->lr_number); break; default: break; } } static void dump_dwarf_block(struct readelf *re, uint8_t *b, Dwarf_Unsigned len) { Dwarf_Locdesc *llbuf; Dwarf_Signed lcnt; Dwarf_Error de; int i; if (dwarf_loclist_from_expr_b(re->dbg, b, len, re->cu_psize, re->cu_osize, re->cu_ver, &llbuf, &lcnt, &de) != DW_DLV_OK) { warnx("dwarf_loclist_form_expr_b: %s", dwarf_errmsg(de)); return; } for (i = 0; (Dwarf_Half) i < llbuf->ld_cents; i++) { dump_dwarf_loc(re, &llbuf->ld_s[i]); if (i < llbuf->ld_cents - 1) printf("; "); } dwarf_dealloc(re->dbg, llbuf->ld_s, DW_DLA_LOC_BLOCK); dwarf_dealloc(re->dbg, llbuf, DW_DLA_LOCDESC); } static void dump_dwarf_loclist(struct readelf *re) { Dwarf_Die die; Dwarf_Locdesc **llbuf; Dwarf_Unsigned lowpc; Dwarf_Signed lcnt; Dwarf_Half tag, version, pointer_size, off_size; Dwarf_Error de; struct loc_at *la; int i, j, ret; printf("\nContents of section .debug_loc:\n"); /* Search .debug_info section. */ while ((ret = dwarf_next_cu_header_b(re->dbg, NULL, &version, NULL, &pointer_size, &off_size, NULL, NULL, &de)) == DW_DLV_OK) { set_cu_context(re, pointer_size, off_size, version); die = NULL; if (dwarf_siblingof(re->dbg, die, &die, &de) != DW_DLV_OK) continue; if (dwarf_tag(die, &tag, &de) != DW_DLV_OK) { warnx("dwarf_tag failed: %s", dwarf_errmsg(de)); continue; } /* XXX: What about DW_TAG_partial_unit? */ lowpc = 0; if (tag == DW_TAG_compile_unit) { if (dwarf_attrval_unsigned(die, DW_AT_low_pc, &lowpc, &de) != DW_DLV_OK) lowpc = 0; } /* Search attributes for reference to .debug_loc section. */ search_loclist_at(re, die, lowpc); } if (ret == DW_DLV_ERROR) warnx("dwarf_next_cu_header: %s", dwarf_errmsg(de)); /* Search .debug_types section. */ do { while ((ret = dwarf_next_cu_header_c(re->dbg, 0, NULL, &version, NULL, &pointer_size, &off_size, NULL, NULL, NULL, NULL, &de)) == DW_DLV_OK) { set_cu_context(re, pointer_size, off_size, version); die = NULL; if (dwarf_siblingof(re->dbg, die, &die, &de) != DW_DLV_OK) continue; if (dwarf_tag(die, &tag, &de) != DW_DLV_OK) { warnx("dwarf_tag failed: %s", dwarf_errmsg(de)); continue; } lowpc = 0; if (tag == DW_TAG_type_unit) { if (dwarf_attrval_unsigned(die, DW_AT_low_pc, &lowpc, &de) != DW_DLV_OK) lowpc = 0; } /* * Search attributes for reference to .debug_loc * section. */ search_loclist_at(re, die, lowpc); } if (ret == DW_DLV_ERROR) warnx("dwarf_next_cu_header: %s", dwarf_errmsg(de)); } while (dwarf_next_types_section(re->dbg, &de) == DW_DLV_OK); if (TAILQ_EMPTY(&lalist)) return; printf(" Offset Begin End Expression\n"); TAILQ_FOREACH(la, &lalist, la_next) { if (dwarf_loclist_n(la->la_at, &llbuf, &lcnt, &de) != DW_DLV_OK) { warnx("dwarf_loclist_n failed: %s", dwarf_errmsg(de)); continue; } set_cu_context(re, la->la_cu_psize, la->la_cu_osize, la->la_cu_ver); for (i = 0; i < lcnt; i++) { printf(" %8.8jx ", (uintmax_t) la->la_off); if (llbuf[i]->ld_lopc == 0 && llbuf[i]->ld_hipc == 0) { printf("\n"); continue; } /* TODO: handle base selection entry. */ printf("%8.8jx %8.8jx ", (uintmax_t) (la->la_lowpc + llbuf[i]->ld_lopc), (uintmax_t) (la->la_lowpc + llbuf[i]->ld_hipc)); putchar('('); for (j = 0; (Dwarf_Half) j < llbuf[i]->ld_cents; j++) { dump_dwarf_loc(re, &llbuf[i]->ld_s[j]); if (j < llbuf[i]->ld_cents - 1) printf("; "); } putchar(')'); if (llbuf[i]->ld_lopc == llbuf[i]->ld_hipc) printf(" (start == end)"); putchar('\n'); } for (i = 0; i < lcnt; i++) { dwarf_dealloc(re->dbg, llbuf[i]->ld_s, DW_DLA_LOC_BLOCK); dwarf_dealloc(re->dbg, llbuf[i], DW_DLA_LOCDESC); } dwarf_dealloc(re->dbg, llbuf, DW_DLA_LIST); } } /* * Retrieve a string using string table section index and the string offset. */ static const char* get_string(struct readelf *re, int strtab, size_t off) { const char *name; if ((name = elf_strptr(re->elf, strtab, off)) == NULL) return (""); return (name); } /* * Retrieve the name of a symbol using the section index of the symbol * table and the index of the symbol within that table. */ static const char * get_symbol_name(struct readelf *re, int symtab, int i) { struct section *s; const char *name; GElf_Sym sym; Elf_Data *data; int elferr; s = &re->sl[symtab]; if (s->type != SHT_SYMTAB && s->type != SHT_DYNSYM) return (""); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return (""); } if (gelf_getsym(data, i, &sym) != &sym) return (""); /* Return section name for STT_SECTION symbol. */ if (GELF_ST_TYPE(sym.st_info) == STT_SECTION) { if (sym.st_shndx < re->shnum && re->sl[sym.st_shndx].name != NULL) return (re->sl[sym.st_shndx].name); return (""); } if (s->link >= re->shnum || (name = elf_strptr(re->elf, s->link, sym.st_name)) == NULL) return (""); return (name); } static uint64_t get_symbol_value(struct readelf *re, int symtab, int i) { struct section *s; GElf_Sym sym; Elf_Data *data; int elferr; s = &re->sl[symtab]; if (s->type != SHT_SYMTAB && s->type != SHT_DYNSYM) return (0); (void) elf_errno(); if ((data = elf_getdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); return (0); } if (gelf_getsym(data, i, &sym) != &sym) return (0); return (sym.st_value); } static void hex_dump(struct readelf *re) { struct section *s; Elf_Data *d; uint8_t *buf; size_t sz, nbytes; uint64_t addr; int elferr, i, j; for (i = 1; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (find_dumpop(re, (size_t) i, s->name, HEX_DUMP, -1) == NULL) continue; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL && (d = elf_rawdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); continue; } (void) elf_errno(); if (d->d_size <= 0 || d->d_buf == NULL) { printf("\nSection '%s' has no data to dump.\n", s->name); continue; } buf = d->d_buf; sz = d->d_size; addr = s->addr; printf("\nHex dump of section '%s':\n", s->name); while (sz > 0) { printf(" 0x%8.8jx ", (uintmax_t)addr); nbytes = sz > 16? 16 : sz; for (j = 0; j < 16; j++) { if ((size_t)j < nbytes) printf("%2.2x", buf[j]); else printf(" "); if ((j & 3) == 3) printf(" "); } for (j = 0; (size_t)j < nbytes; j++) { if (isprint(buf[j])) printf("%c", buf[j]); else printf("."); } printf("\n"); buf += nbytes; addr += nbytes; sz -= nbytes; } } } static void str_dump(struct readelf *re) { struct section *s; Elf_Data *d; unsigned char *start, *end, *buf_end; unsigned int len; int i, j, elferr, found; for (i = 1; (size_t) i < re->shnum; i++) { s = &re->sl[i]; if (find_dumpop(re, (size_t) i, s->name, STR_DUMP, -1) == NULL) continue; (void) elf_errno(); if ((d = elf_getdata(s->scn, NULL)) == NULL && (d = elf_rawdata(s->scn, NULL)) == NULL) { elferr = elf_errno(); if (elferr != 0) warnx("elf_getdata failed: %s", elf_errmsg(elferr)); continue; } (void) elf_errno(); if (d->d_size <= 0 || d->d_buf == NULL) { printf("\nSection '%s' has no data to dump.\n", s->name); continue; } buf_end = (unsigned char *) d->d_buf + d->d_size; start = (unsigned char *) d->d_buf; found = 0; printf("\nString dump of section '%s':\n", s->name); for (;;) { while (start < buf_end && !isprint(*start)) start++; if (start >= buf_end) break; end = start + 1; while (end < buf_end && isprint(*end)) end++; printf(" [%6lx] ", (long) (start - (unsigned char *) d->d_buf)); len = end - start; for (j = 0; (unsigned int) j < len; j++) putchar(start[j]); putchar('\n'); found = 1; if (end >= buf_end) break; start = end + 1; } if (!found) printf(" No strings found in this section."); putchar('\n'); } } static void load_sections(struct readelf *re) { struct section *s; const char *name; Elf_Scn *scn; GElf_Shdr sh; size_t shstrndx, ndx; int elferr; /* Allocate storage for internal section list. */ if (!elf_getshnum(re->elf, &re->shnum)) { warnx("elf_getshnum failed: %s", elf_errmsg(-1)); return; } if (re->sl != NULL) free(re->sl); if ((re->sl = calloc(re->shnum, sizeof(*re->sl))) == NULL) err(EXIT_FAILURE, "calloc failed"); /* Get the index of .shstrtab section. */ if (!elf_getshstrndx(re->elf, &shstrndx)) { warnx("elf_getshstrndx failed: %s", elf_errmsg(-1)); return; } if ((scn = elf_getscn(re->elf, 0)) == NULL) return; (void) elf_errno(); do { if (gelf_getshdr(scn, &sh) == NULL) { warnx("gelf_getshdr failed: %s", elf_errmsg(-1)); (void) elf_errno(); continue; } if ((name = elf_strptr(re->elf, shstrndx, sh.sh_name)) == NULL) { (void) elf_errno(); name = "ERROR"; } if ((ndx = elf_ndxscn(scn)) == SHN_UNDEF) { if ((elferr = elf_errno()) != 0) warnx("elf_ndxscn failed: %s", elf_errmsg(elferr)); continue; } if (ndx >= re->shnum) { warnx("section index of '%s' out of range", name); continue; } if (sh.sh_link >= re->shnum) warnx("section link %llu of '%s' out of range", (unsigned long long)sh.sh_link, name); s = &re->sl[ndx]; s->name = name; s->scn = scn; s->off = sh.sh_offset; s->sz = sh.sh_size; s->entsize = sh.sh_entsize; s->align = sh.sh_addralign; s->type = sh.sh_type; s->flags = sh.sh_flags; s->addr = sh.sh_addr; s->link = sh.sh_link; s->info = sh.sh_info; } while ((scn = elf_nextscn(re->elf, scn)) != NULL); elferr = elf_errno(); if (elferr != 0) warnx("elf_nextscn failed: %s", elf_errmsg(elferr)); } static void unload_sections(struct readelf *re) { if (re->sl != NULL) { free(re->sl); re->sl = NULL; } re->shnum = 0; re->vd_s = NULL; re->vn_s = NULL; re->vs_s = NULL; re->vs = NULL; re->vs_sz = 0; if (re->ver != NULL) { free(re->ver); re->ver = NULL; re->ver_sz = 0; } } static void dump_elf(struct readelf *re) { /* Fetch ELF header. No need to continue if it fails. */ if (gelf_getehdr(re->elf, &re->ehdr) == NULL) { warnx("gelf_getehdr failed: %s", elf_errmsg(-1)); return; } if ((re->ec = gelf_getclass(re->elf)) == ELFCLASSNONE) { warnx("gelf_getclass failed: %s", elf_errmsg(-1)); return; } if (re->ehdr.e_ident[EI_DATA] == ELFDATA2MSB) { re->dw_read = _read_msb; re->dw_decode = _decode_msb; } else { re->dw_read = _read_lsb; re->dw_decode = _decode_lsb; } if (re->options & ~RE_H) load_sections(re); if ((re->options & RE_VV) || (re->options & RE_S)) search_ver(re); if (re->options & RE_H) dump_ehdr(re); if (re->options & RE_L) dump_phdr(re); if (re->options & RE_SS) dump_shdr(re); if (re->options & RE_G) dump_section_groups(re); if (re->options & RE_D) dump_dynamic(re); if (re->options & RE_R) dump_reloc(re); if (re->options & RE_S) dump_symtabs(re); if (re->options & RE_N) dump_notes(re); if (re->options & RE_II) dump_hash(re); if (re->options & RE_X) hex_dump(re); if (re->options & RE_P) str_dump(re); if (re->options & RE_VV) dump_ver(re); if (re->options & RE_AA) dump_arch_specific_info(re); if (re->options & RE_W) dump_dwarf(re); if (re->options & ~RE_H) unload_sections(re); } static void dump_dwarf(struct readelf *re) { int error; Dwarf_Error de; if (dwarf_elf_init(re->elf, DW_DLC_READ, NULL, NULL, &re->dbg, &de)) { if ((error = dwarf_errno(de)) != DW_DLE_DEBUG_INFO_NULL) errx(EXIT_FAILURE, "dwarf_elf_init failed: %s", dwarf_errmsg(de)); return; } if (re->dop & DW_A) dump_dwarf_abbrev(re); if (re->dop & DW_L) dump_dwarf_line(re); if (re->dop & DW_LL) dump_dwarf_line_decoded(re); if (re->dop & DW_I) { dump_dwarf_info(re, 0); dump_dwarf_info(re, 1); } if (re->dop & DW_P) dump_dwarf_pubnames(re); if (re->dop & DW_R) dump_dwarf_aranges(re); if (re->dop & DW_RR) dump_dwarf_ranges(re); if (re->dop & DW_M) dump_dwarf_macinfo(re); if (re->dop & DW_F) dump_dwarf_frame(re, 0); else if (re->dop & DW_FF) dump_dwarf_frame(re, 1); if (re->dop & DW_S) dump_dwarf_str(re); if (re->dop & DW_O) dump_dwarf_loclist(re); dwarf_finish(re->dbg, &de); } static void dump_ar(struct readelf *re, int fd) { Elf_Arsym *arsym; Elf_Arhdr *arhdr; Elf_Cmd cmd; Elf *e; size_t sz; off_t off; int i; re->ar = re->elf; if (re->options & RE_C) { if ((arsym = elf_getarsym(re->ar, &sz)) == NULL) { warnx("elf_getarsym() failed: %s", elf_errmsg(-1)); goto process_members; } printf("Index of archive %s: (%ju entries)\n", re->filename, (uintmax_t) sz - 1); off = 0; for (i = 0; (size_t) i < sz; i++) { if (arsym[i].as_name == NULL) break; if (arsym[i].as_off != off) { off = arsym[i].as_off; if (elf_rand(re->ar, off) != off) { warnx("elf_rand() failed: %s", elf_errmsg(-1)); continue; } if ((e = elf_begin(fd, ELF_C_READ, re->ar)) == NULL) { warnx("elf_begin() failed: %s", elf_errmsg(-1)); continue; } if ((arhdr = elf_getarhdr(e)) == NULL) { warnx("elf_getarhdr() failed: %s", elf_errmsg(-1)); elf_end(e); continue; } printf("Binary %s(%s) contains:\n", re->filename, arhdr->ar_name); } printf("\t%s\n", arsym[i].as_name); } if (elf_rand(re->ar, SARMAG) != SARMAG) { warnx("elf_rand() failed: %s", elf_errmsg(-1)); return; } } process_members: if ((re->options & ~RE_C) == 0) return; cmd = ELF_C_READ; while ((re->elf = elf_begin(fd, cmd, re->ar)) != NULL) { if ((arhdr = elf_getarhdr(re->elf)) == NULL) { warnx("elf_getarhdr() failed: %s", elf_errmsg(-1)); goto next_member; } if (strcmp(arhdr->ar_name, "/") == 0 || strcmp(arhdr->ar_name, "//") == 0 || strcmp(arhdr->ar_name, "__.SYMDEF") == 0) goto next_member; printf("\nFile: %s(%s)\n", re->filename, arhdr->ar_name); dump_elf(re); next_member: cmd = elf_next(re->elf); elf_end(re->elf); } re->elf = re->ar; } static void dump_object(struct readelf *re) { int fd; if ((fd = open(re->filename, O_RDONLY)) == -1) { warn("open %s failed", re->filename); return; } if ((re->flags & DISPLAY_FILENAME) != 0) printf("\nFile: %s\n", re->filename); if ((re->elf = elf_begin(fd, ELF_C_READ, NULL)) == NULL) { warnx("elf_begin() failed: %s", elf_errmsg(-1)); return; } switch (elf_kind(re->elf)) { case ELF_K_NONE: warnx("Not an ELF file."); return; case ELF_K_ELF: dump_elf(re); break; case ELF_K_AR: dump_ar(re, fd); break; default: warnx("Internal: libelf returned unknown elf kind."); return; } elf_end(re->elf); } static void add_dumpop(struct readelf *re, size_t si, const char *sn, int op, int t) { struct dumpop *d; if ((d = find_dumpop(re, si, sn, -1, t)) == NULL) { if ((d = calloc(1, sizeof(*d))) == NULL) err(EXIT_FAILURE, "calloc failed"); if (t == DUMP_BY_INDEX) d->u.si = si; else d->u.sn = sn; d->type = t; d->op = op; STAILQ_INSERT_TAIL(&re->v_dumpop, d, dumpop_list); } else d->op |= op; } static struct dumpop * find_dumpop(struct readelf *re, size_t si, const char *sn, int op, int t) { struct dumpop *d; STAILQ_FOREACH(d, &re->v_dumpop, dumpop_list) { if ((op == -1 || op & d->op) && (t == -1 || (unsigned) t == d->type)) { if ((d->type == DUMP_BY_INDEX && d->u.si == si) || (d->type == DUMP_BY_NAME && !strcmp(d->u.sn, sn))) return (d); } } return (NULL); } static struct { const char *ln; char sn; int value; } dwarf_op[] = { {"rawline", 'l', DW_L}, {"decodedline", 'L', DW_LL}, {"info", 'i', DW_I}, {"abbrev", 'a', DW_A}, {"pubnames", 'p', DW_P}, {"aranges", 'r', DW_R}, {"ranges", 'r', DW_R}, {"Ranges", 'R', DW_RR}, {"macro", 'm', DW_M}, {"frames", 'f', DW_F}, {"frames-interp", 'F', DW_FF}, {"str", 's', DW_S}, {"loc", 'o', DW_O}, {NULL, 0, 0} }; static void parse_dwarf_op_short(struct readelf *re, const char *op) { int i; if (op == NULL) { re->dop |= DW_DEFAULT_OPTIONS; return; } for (; *op != '\0'; op++) { for (i = 0; dwarf_op[i].ln != NULL; i++) { if (dwarf_op[i].sn == *op) { re->dop |= dwarf_op[i].value; break; } } } } static void parse_dwarf_op_long(struct readelf *re, const char *op) { char *p, *token, *bp; int i; if (op == NULL) { re->dop |= DW_DEFAULT_OPTIONS; return; } if ((p = strdup(op)) == NULL) err(EXIT_FAILURE, "strdup failed"); bp = p; while ((token = strsep(&p, ",")) != NULL) { for (i = 0; dwarf_op[i].ln != NULL; i++) { if (!strcmp(token, dwarf_op[i].ln)) { re->dop |= dwarf_op[i].value; break; } } } free(bp); } static uint64_t _read_lsb(Elf_Data *d, uint64_t *offsetp, int bytes_to_read) { uint64_t ret; uint8_t *src; src = (uint8_t *) d->d_buf + *offsetp; ret = 0; switch (bytes_to_read) { case 8: ret |= ((uint64_t) src[4]) << 32 | ((uint64_t) src[5]) << 40; ret |= ((uint64_t) src[6]) << 48 | ((uint64_t) src[7]) << 56; /* FALLTHROUGH */ case 4: ret |= ((uint64_t) src[2]) << 16 | ((uint64_t) src[3]) << 24; /* FALLTHROUGH */ case 2: ret |= ((uint64_t) src[1]) << 8; /* FALLTHROUGH */ case 1: ret |= src[0]; break; default: return (0); } *offsetp += bytes_to_read; return (ret); } static uint64_t _read_msb(Elf_Data *d, uint64_t *offsetp, int bytes_to_read) { uint64_t ret; uint8_t *src; src = (uint8_t *) d->d_buf + *offsetp; switch (bytes_to_read) { case 1: ret = src[0]; break; case 2: ret = src[1] | ((uint64_t) src[0]) << 8; break; case 4: ret = src[3] | ((uint64_t) src[2]) << 8; ret |= ((uint64_t) src[1]) << 16 | ((uint64_t) src[0]) << 24; break; case 8: ret = src[7] | ((uint64_t) src[6]) << 8; ret |= ((uint64_t) src[5]) << 16 | ((uint64_t) src[4]) << 24; ret |= ((uint64_t) src[3]) << 32 | ((uint64_t) src[2]) << 40; ret |= ((uint64_t) src[1]) << 48 | ((uint64_t) src[0]) << 56; break; default: return (0); } *offsetp += bytes_to_read; return (ret); } static uint64_t _decode_lsb(uint8_t **data, int bytes_to_read) { uint64_t ret; uint8_t *src; src = *data; ret = 0; switch (bytes_to_read) { case 8: ret |= ((uint64_t) src[4]) << 32 | ((uint64_t) src[5]) << 40; ret |= ((uint64_t) src[6]) << 48 | ((uint64_t) src[7]) << 56; /* FALLTHROUGH */ case 4: ret |= ((uint64_t) src[2]) << 16 | ((uint64_t) src[3]) << 24; /* FALLTHROUGH */ case 2: ret |= ((uint64_t) src[1]) << 8; /* FALLTHROUGH */ case 1: ret |= src[0]; break; default: return (0); } *data += bytes_to_read; return (ret); } static uint64_t _decode_msb(uint8_t **data, int bytes_to_read) { uint64_t ret; uint8_t *src; src = *data; ret = 0; switch (bytes_to_read) { case 1: ret = src[0]; break; case 2: ret = src[1] | ((uint64_t) src[0]) << 8; break; case 4: ret = src[3] | ((uint64_t) src[2]) << 8; ret |= ((uint64_t) src[1]) << 16 | ((uint64_t) src[0]) << 24; break; case 8: ret = src[7] | ((uint64_t) src[6]) << 8; ret |= ((uint64_t) src[5]) << 16 | ((uint64_t) src[4]) << 24; ret |= ((uint64_t) src[3]) << 32 | ((uint64_t) src[2]) << 40; ret |= ((uint64_t) src[1]) << 48 | ((uint64_t) src[0]) << 56; break; default: return (0); break; } *data += bytes_to_read; return (ret); } static int64_t _decode_sleb128(uint8_t **dp, uint8_t *dpe) { int64_t ret = 0; uint8_t b = 0; int shift = 0; uint8_t *src = *dp; do { if (src >= dpe) break; b = *src++; ret |= ((b & 0x7f) << shift); shift += 7; } while ((b & 0x80) != 0); if (shift < 32 && (b & 0x40) != 0) ret |= (-1 << shift); *dp = src; return (ret); } static uint64_t _decode_uleb128(uint8_t **dp, uint8_t *dpe) { uint64_t ret = 0; uint8_t b; int shift = 0; uint8_t *src = *dp; do { if (src >= dpe) break; b = *src++; ret |= ((b & 0x7f) << shift); shift += 7; } while ((b & 0x80) != 0); *dp = src; return (ret); } static void readelf_version(void) { (void) printf("%s (%s)\n", ELFTC_GETPROGNAME(), elftc_version()); exit(EXIT_SUCCESS); } #define USAGE_MESSAGE "\ Usage: %s [options] file...\n\ Display information about ELF objects and ar(1) archives.\n\n\ Options:\n\ -a | --all Equivalent to specifying options '-dhIlrsASV'.\n\ -c | --archive-index Print the archive symbol table for archives.\n\ -d | --dynamic Print the contents of SHT_DYNAMIC sections.\n\ -e | --headers Print all headers in the object.\n\ -g | --section-groups Print the contents of the section groups.\n\ -h | --file-header Print the file header for the object.\n\ -l | --program-headers Print the PHDR table for the object.\n\ -n | --notes Print the contents of SHT_NOTE sections.\n\ -p INDEX | --string-dump=INDEX\n\ Print the contents of section at index INDEX.\n\ -r | --relocs Print relocation information.\n\ -s | --syms | --symbols Print symbol tables.\n\ -t | --section-details Print additional information about sections.\n\ -v | --version Print a version identifier and exit.\n\ -w[afilmoprsFLR] | --debug-dump={abbrev,aranges,decodedline,frames,\n\ frames-interp,info,loc,macro,pubnames,\n\ ranges,Ranges,rawline,str}\n\ Display DWARF information.\n\ -x INDEX | --hex-dump=INDEX\n\ Display contents of a section as hexadecimal.\n\ -A | --arch-specific (accepted, but ignored)\n\ -D | --use-dynamic Print the symbol table specified by the DT_SYMTAB\n\ entry in the \".dynamic\" section.\n\ -H | --help Print a help message.\n\ -I | --histogram Print information on bucket list lengths for \n\ hash sections.\n\ -N | --full-section-name (accepted, but ignored)\n\ -S | --sections | --section-headers\n\ Print information about section headers.\n\ -V | --version-info Print symbol versoning information.\n\ -W | --wide Print information without wrapping long lines.\n" static void readelf_usage(int status) { fprintf(stderr, USAGE_MESSAGE, ELFTC_GETPROGNAME()); exit(status); } int main(int argc, char **argv) { struct readelf *re, re_storage; unsigned long si; int opt, i; char *ep; re = &re_storage; memset(re, 0, sizeof(*re)); STAILQ_INIT(&re->v_dumpop); while ((opt = getopt_long(argc, argv, "AacDdegHhIi:lNnp:rSstuVvWw::x:", longopts, NULL)) != -1) { switch(opt) { case '?': readelf_usage(EXIT_SUCCESS); break; case 'A': re->options |= RE_AA; break; case 'a': re->options |= RE_AA | RE_D | RE_G | RE_H | RE_II | RE_L | RE_R | RE_SS | RE_S | RE_VV; break; case 'c': re->options |= RE_C; break; case 'D': re->options |= RE_DD; break; case 'd': re->options |= RE_D; break; case 'e': re->options |= RE_H | RE_L | RE_SS; break; case 'g': re->options |= RE_G; break; case 'H': readelf_usage(EXIT_SUCCESS); break; case 'h': re->options |= RE_H; break; case 'I': re->options |= RE_II; break; case 'i': /* Not implemented yet. */ break; case 'l': re->options |= RE_L; break; case 'N': re->options |= RE_NN; break; case 'n': re->options |= RE_N; break; case 'p': re->options |= RE_P; si = strtoul(optarg, &ep, 10); if (*ep == '\0') add_dumpop(re, (size_t) si, NULL, STR_DUMP, DUMP_BY_INDEX); else add_dumpop(re, 0, optarg, STR_DUMP, DUMP_BY_NAME); break; case 'r': re->options |= RE_R; break; case 'S': re->options |= RE_SS; break; case 's': re->options |= RE_S; break; case 't': re->options |= RE_T; break; case 'u': re->options |= RE_U; break; case 'V': re->options |= RE_VV; break; case 'v': readelf_version(); break; case 'W': re->options |= RE_WW; break; case 'w': re->options |= RE_W; parse_dwarf_op_short(re, optarg); break; case 'x': re->options |= RE_X; si = strtoul(optarg, &ep, 10); if (*ep == '\0') add_dumpop(re, (size_t) si, NULL, HEX_DUMP, DUMP_BY_INDEX); else add_dumpop(re, 0, optarg, HEX_DUMP, DUMP_BY_NAME); break; case OPTION_DEBUG_DUMP: re->options |= RE_W; parse_dwarf_op_long(re, optarg); } } argv += optind; argc -= optind; if (argc == 0 || re->options == 0) readelf_usage(EXIT_FAILURE); if (argc > 1) re->flags |= DISPLAY_FILENAME; if (elf_version(EV_CURRENT) == EV_NONE) errx(EXIT_FAILURE, "ELF library initialization failed: %s", elf_errmsg(-1)); for (i = 0; i < argc; i++) { re->filename = argv[i]; dump_object(re); } exit(EXIT_SUCCESS); } Index: projects/netbsd-tests-update-12/contrib/elftoolchain =================================================================== --- projects/netbsd-tests-update-12/contrib/elftoolchain (revision 305171) +++ projects/netbsd-tests-update-12/contrib/elftoolchain (revision 305172) Property changes on: projects/netbsd-tests-update-12/contrib/elftoolchain ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,2 ## Merged /head/contrib/elftoolchain:r304233-305170 Merged /vendor/elftoolchain/dist:r305126 Index: projects/netbsd-tests-update-12/contrib/gcc/config/i386/cpuid.h =================================================================== --- projects/netbsd-tests-update-12/contrib/gcc/config/i386/cpuid.h (nonexistent) +++ projects/netbsd-tests-update-12/contrib/gcc/config/i386/cpuid.h (revision 305172) @@ -0,0 +1,134 @@ +/* + * Copyright (C) 2007 Free Software Foundation, Inc. + * + * This file is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation; either version 2, or (at your option) any + * later version. + * + * In addition to the permissions in the GNU General Public License, the + * Free Software Foundation gives you unlimited permission to link the + * compiled version of this file with other programs, and to distribute + * those programs without any restriction coming from the use of this + * file. (The General Public License restrictions do apply in other + * respects; for example, they cover modification of the file, and + * distribution when not linked into another program.) + * + * This file is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; see the file COPYING. If not, write to + * the Free Software Foundation, 51 Franklin Street, Fifth Floor, + * Boston, MA 02110-1301, USA. + * + * As a special exception, if you link this library with files + * compiled with GCC to produce an executable, this does not cause + * the resulting executable to be covered by the GNU General Public License. + * This exception does not however invalidate any other reasons why + * the executable file might be covered by the GNU General Public License. + */ + +/* %ecx */ +#define bit_SSE3 (1 << 0) +#define bit_SSSE3 (1 << 9) +#define bit_CMPXCHG16B (1 << 13) +#define bit_SSE4_1 (1 << 19) +#define bit_SSE4_2 (1 << 20) +#define bit_POPCNT (1 << 23) + +/* %edx */ +#define bit_CMPXCHG8B (1 << 8) +#define bit_CMOV (1 << 15) +#define bit_MMX (1 << 23) +#define bit_FXSAVE (1 << 24) +#define bit_SSE (1 << 25) +#define bit_SSE2 (1 << 26) + +/* Extended Features */ +/* %ecx */ +#define bit_LAHF_LM (1 << 0) +#define bit_SSE4a (1 << 6) +#define bit_SSE5 (1 << 11) + +/* %edx */ +#define bit_LM (1 << 29) +#define bit_3DNOWP (1 << 30) +#define bit_3DNOW (1 << 31) + + +#if defined(__i386__) && defined(__PIC__) +/* %ebx may be the PIC register. */ +#define __cpuid(level, a, b, c, d) \ + __asm__ ("xchg{l}\t{%%}ebx, %1\n\t" \ + "cpuid\n\t" \ + "xchg{l}\t{%%}ebx, %1\n\t" \ + : "=a" (a), "=r" (b), "=c" (c), "=d" (d) \ + : "0" (level)) +#else +#define __cpuid(level, a, b, c, d) \ + __asm__ ("cpuid\n\t" \ + : "=a" (a), "=b" (b), "=c" (c), "=d" (d) \ + : "0" (level)) +#endif + +/* Return highest supported input value for cpuid instruction. ext can + be either 0x0 or 0x8000000 to return highest supported value for + basic or extended cpuid information. Function returns 0 if cpuid + is not supported or whatever cpuid returns in eax register. If sig + pointer is non-null, then first four bytes of the signature + (as found in ebx register) are returned in location pointed by sig. */ + +static __inline unsigned int +__get_cpuid_max (unsigned int __ext, unsigned int *__sig) +{ + unsigned int __eax, __ebx, __ecx, __edx; + +#ifndef __x86_64__ + /* See if we can use cpuid. On AMD64 we always can. */ + __asm__ ("pushf{l|d}\n\t" + "pushf{l|d}\n\t" + "pop{l}\t%0\n\t" + "mov{l}\t{%0, %1|%1, %0}\n\t" + "xor{l}\t{%2, %0|%0, %2}\n\t" + "push{l}\t%0\n\t" + "popf{l|d}\n\t" + "pushf{l|d}\n\t" + "pop{l}\t%0\n\t" + "popf{l|d}\n\t" + : "=&r" (__eax), "=&r" (__ebx) + : "i" (0x00200000)); + + if (!((__eax ^ __ebx) & 0x00200000)) + return 0; +#endif + + /* Host supports cpuid. Return highest supported cpuid input value. */ + __cpuid (__ext, __eax, __ebx, __ecx, __edx); + + if (__sig) + *__sig = __ebx; + + return __eax; +} + +/* Return cpuid data for requested cpuid level, as found in returned + eax, ebx, ecx and edx registers. The function checks if cpuid is + supported and returns 1 for valid cpuid information or 0 for + unsupported cpuid level. All pointers are required to be non-null. */ + +static __inline int +__get_cpuid (unsigned int __level, + unsigned int *__eax, unsigned int *__ebx, + unsigned int *__ecx, unsigned int *__edx) +{ + unsigned int __ext = __level & 0x80000000; + + if (__get_cpuid_max (__ext, 0) < __level) + return 0; + + __cpuid (__level, *__eax, *__ebx, *__ecx, *__edx); + return 1; +} Property changes on: projects/netbsd-tests-update-12/contrib/gcc/config/i386/cpuid.h ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/contrib/gcc/doc/cpp.texi =================================================================== --- projects/netbsd-tests-update-12/contrib/gcc/doc/cpp.texi (revision 305171) +++ projects/netbsd-tests-update-12/contrib/gcc/doc/cpp.texi (revision 305172) @@ -1,4255 +1,4262 @@ \input texinfo @setfilename cpp.info @settitle The C Preprocessor @setchapternewpage off @c @smallbook @c @cropmarks @c @finalout @include gcc-common.texi @copying @c man begin COPYRIGHT Copyright @copyright{} 1987, 1989, 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation. A copy of the license is included in the @c man end section entitled ``GNU Free Documentation License''. @ignore @c man begin COPYRIGHT man page gfdl(7). @c man end @end ignore @c man begin COPYRIGHT This manual contains no Invariant Sections. The Front-Cover Texts are (a) (see below), and the Back-Cover Texts are (b) (see below). (a) The FSF's Front-Cover Text is: A GNU Manual (b) The FSF's Back-Cover Text is: You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development. @c man end @end copying @c Create a separate index for command line options. @defcodeindex op @syncodeindex vr op @c Used in cppopts.texi and cppenv.texi. @set cppmanual @ifinfo @dircategory Software development @direntry * Cpp: (cpp). The GNU C preprocessor. @end direntry @end ifinfo @titlepage @title The C Preprocessor @versionsubtitle @author Richard M. Stallman, Zachary Weinberg @page @c There is a fill at the bottom of the page, so we need a filll to @c override it. @vskip 0pt plus 1filll @insertcopying @end titlepage @contents @page @ifnottex @node Top @top The C preprocessor implements the macro language used to transform C and C++ programs before they are compiled. It can also be useful on its own. @menu * Overview:: * Header Files:: * Macros:: * Conditionals:: * Diagnostics:: * Line Control:: * Pragmas:: * Other Directives:: * Preprocessor Output:: * Traditional Mode:: * Implementation Details:: * Invocation:: * Environment Variables:: * GNU Free Documentation License:: * Index of Directives:: * Option Index:: * Concept Index:: @detailmenu --- The Detailed Node Listing --- Overview * Character sets:: * Initial processing:: * Tokenization:: * The preprocessing language:: Header Files * Include Syntax:: * Include Operation:: * Search Path:: * Once-Only Headers:: * Computed Includes:: * Wrapper Headers:: * System Headers:: Macros * Object-like Macros:: * Function-like Macros:: * Macro Arguments:: * Stringification:: * Concatenation:: * Variadic Macros:: * Predefined Macros:: * Undefining and Redefining Macros:: * Directives Within Macro Arguments:: * Macro Pitfalls:: Predefined Macros * Standard Predefined Macros:: * Common Predefined Macros:: * System-specific Predefined Macros:: * C++ Named Operators:: Macro Pitfalls * Misnesting:: * Operator Precedence Problems:: * Swallowing the Semicolon:: * Duplication of Side Effects:: * Self-Referential Macros:: * Argument Prescan:: * Newlines in Arguments:: Conditionals * Conditional Uses:: * Conditional Syntax:: * Deleted Code:: Conditional Syntax * Ifdef:: * If:: * Defined:: * Else:: * Elif:: Implementation Details * Implementation-defined behavior:: * Implementation limits:: * Obsolete Features:: * Differences from previous versions:: Obsolete Features * Assertions:: * Obsolete once-only headers:: @end detailmenu @end menu @insertcopying @end ifnottex @node Overview @chapter Overview @c man begin DESCRIPTION The C preprocessor, often known as @dfn{cpp}, is a @dfn{macro processor} that is used automatically by the C compiler to transform your program before compilation. It is called a macro processor because it allows you to define @dfn{macros}, which are brief abbreviations for longer constructs. The C preprocessor is intended to be used only with C and C++ source code. In the past, it has been abused as a general text processor. It will choke on input which does not obey C's lexical rules. For example, apostrophes will be interpreted as the beginning of character constants, and cause errors. Also, you cannot rely on it preserving characteristics of the input which are not significant to C-family languages. If a Makefile is preprocessed, all the hard tabs will be removed, and the Makefile will not work. Having said that, you can often get away with using cpp on things which are not C@. Other Algol-ish programming languages are often safe (Pascal, Ada, etc.) So is assembly, with caution. @option{-traditional-cpp} mode preserves more white space, and is otherwise more permissive. Many of the problems can be avoided by writing C or C++ style comments instead of native language comments, and keeping macros simple. Wherever possible, you should use a preprocessor geared to the language you are writing in. Modern versions of the GNU assembler have macro facilities. Most high level programming languages have their own conditional compilation and inclusion mechanism. If all else fails, try a true general text processor, such as GNU M4. C preprocessors vary in some details. This manual discusses the GNU C preprocessor, which provides a small superset of the features of ISO Standard C@. In its default mode, the GNU C preprocessor does not do a few things required by the standard. These are features which are rarely, if ever, used, and may cause surprising changes to the meaning of a program which does not expect them. To get strict ISO Standard C, you should use the @option{-std=c89} or @option{-std=c99} options, depending on which version of the standard you want. To get all the mandatory diagnostics, you must also use @option{-pedantic}. @xref{Invocation}. This manual describes the behavior of the ISO preprocessor. To minimize gratuitous differences, where the ISO preprocessor's behavior does not conflict with traditional semantics, the traditional preprocessor should behave the same way. The various differences that do exist are detailed in the section @ref{Traditional Mode}. For clarity, unless noted otherwise, references to @samp{CPP} in this manual refer to GNU CPP@. @c man end @menu * Character sets:: * Initial processing:: * Tokenization:: * The preprocessing language:: @end menu @node Character sets @section Character sets Source code character set processing in C and related languages is rather complicated. The C standard discusses two character sets, but there are really at least four. The files input to CPP might be in any character set at all. CPP's very first action, before it even looks for line boundaries, is to convert the file into the character set it uses for internal processing. That set is what the C standard calls the @dfn{source} character set. It must be isomorphic with ISO 10646, also known as Unicode. CPP uses the UTF-8 encoding of Unicode. The character sets of the input files are specified using the @option{-finput-charset=} option. All preprocessing work (the subject of the rest of this manual) is carried out in the source character set. If you request textual output from the preprocessor with the @option{-E} option, it will be in UTF-8. After preprocessing is complete, string and character constants are converted again, into the @dfn{execution} character set. This character set is under control of the user; the default is UTF-8, matching the source character set. Wide string and character constants have their own character set, which is not called out specifically in the standard. Again, it is under control of the user. The default is UTF-16 or UTF-32, whichever fits in the target's @code{wchar_t} type, in the target machine's byte order.@footnote{UTF-16 does not meet the requirements of the C standard for a wide character set, but the choice of 16-bit @code{wchar_t} is enshrined in some system ABIs so we cannot fix this.} Octal and hexadecimal escape sequences do not undergo conversion; @t{'\x12'} has the value 0x12 regardless of the currently selected execution character set. All other escapes are replaced by the character in the source character set that they represent, then converted to the execution character set, just like unescaped characters. Unless the experimental @option{-fextended-identifiers} option is used, GCC does not permit the use of characters outside the ASCII range, nor @samp{\u} and @samp{\U} escapes, in identifiers. Even with that option, characters outside the ASCII range can only be specified with the @samp{\u} and @samp{\U} escapes, not used directly in identifiers. @node Initial processing @section Initial processing The preprocessor performs a series of textual transformations on its input. These happen before all other processing. Conceptually, they happen in a rigid order, and the entire file is run through each transformation before the next one begins. CPP actually does them all at once, for performance reasons. These transformations correspond roughly to the first three ``phases of translation'' described in the C standard. @enumerate @item @cindex line endings The input file is read into memory and broken into lines. Different systems use different conventions to indicate the end of a line. GCC accepts the ASCII control sequences @kbd{LF}, @kbd{@w{CR LF}} and @kbd{CR} as end-of-line markers. These are the canonical sequences used by Unix, DOS and VMS, and the classic Mac OS (before OSX) respectively. You may therefore safely copy source code written on any of those systems to a different one and use it without conversion. (GCC may lose track of the current line number if a file doesn't consistently use one convention, as sometimes happens when it is edited on computers with different conventions that share a network file system.) If the last line of any input file lacks an end-of-line marker, the end of the file is considered to implicitly supply one. The C standard says that this condition provokes undefined behavior, so GCC will emit a warning message. @item @cindex trigraphs @anchor{trigraphs}If trigraphs are enabled, they are replaced by their corresponding single characters. By default GCC ignores trigraphs, but if you request a strictly conforming mode with the @option{-std} option, or you specify the @option{-trigraphs} option, then it converts them. These are nine three-character sequences, all starting with @samp{??}, that are defined by ISO C to stand for single characters. They permit obsolete systems that lack some of C's punctuation to use C@. For example, @samp{??/} stands for @samp{\}, so @t{'??/n'} is a character constant for a newline. Trigraphs are not popular and many compilers implement them incorrectly. Portable code should not rely on trigraphs being either converted or ignored. With @option{-Wtrigraphs} GCC will warn you when a trigraph may change the meaning of your program if it were converted. @xref{Wtrigraphs}. In a string constant, you can prevent a sequence of question marks from being confused with a trigraph by inserting a backslash between the question marks, or by separating the string literal at the trigraph and making use of string literal concatenation. @t{"(??\?)"} is the string @samp{(???)}, not @samp{(?]}. Traditional C compilers do not recognize these idioms. The nine trigraphs and their replacements are @smallexample Trigraph: ??( ??) ??< ??> ??= ??/ ??' ??! ??- Replacement: [ ] @{ @} # \ ^ | ~ @end smallexample @item @cindex continued lines @cindex backslash-newline Continued lines are merged into one long line. A continued line is a line which ends with a backslash, @samp{\}. The backslash is removed and the following line is joined with the current one. No space is inserted, so you may split a line anywhere, even in the middle of a word. (It is generally more readable to split lines only at white space.) The trailing backslash on a continued line is commonly referred to as a @dfn{backslash-newline}. If there is white space between a backslash and the end of a line, that is still a continued line. However, as this is usually the result of an editing mistake, and many compilers will not accept it as a continued line, GCC will warn you about it. @item @cindex comments @cindex line comments @cindex block comments All comments are replaced with single spaces. There are two kinds of comments. @dfn{Block comments} begin with @samp{/*} and continue until the next @samp{*/}. Block comments do not nest: @smallexample /* @r{this is} /* @r{one comment} */ @r{text outside comment} @end smallexample @dfn{Line comments} begin with @samp{//} and continue to the end of the current line. Line comments do not nest either, but it does not matter, because they would end in the same place anyway. @smallexample // @r{this is} // @r{one comment} @r{text outside comment} @end smallexample @end enumerate It is safe to put line comments inside block comments, or vice versa. @smallexample @group /* @r{block comment} // @r{contains line comment} @r{yet more comment} */ @r{outside comment} // @r{line comment} /* @r{contains block comment} */ @end group @end smallexample But beware of commenting out one end of a block comment with a line comment. @smallexample @group // @r{l.c.} /* @r{block comment begins} @r{oops! this isn't a comment anymore} */ @end group @end smallexample Comments are not recognized within string literals. @t{@w{"/* blah */"}} is the string constant @samp{@w{/* blah */}}, not an empty string. Line comments are not in the 1989 edition of the C standard, but they are recognized by GCC as an extension. In C++ and in the 1999 edition of the C standard, they are an official part of the language. Since these transformations happen before all other processing, you can split a line mechanically with backslash-newline anywhere. You can comment out the end of a line. You can continue a line comment onto the next line with backslash-newline. You can even split @samp{/*}, @samp{*/}, and @samp{//} onto multiple lines with backslash-newline. For example: @smallexample @group /\ * */ # /* */ defi\ ne FO\ O 10\ 20 @end group @end smallexample @noindent is equivalent to @code{@w{#define FOO 1020}}. All these tricks are extremely confusing and should not be used in code intended to be readable. There is no way to prevent a backslash at the end of a line from being interpreted as a backslash-newline. This cannot affect any correct program, however. @node Tokenization @section Tokenization @cindex tokens @cindex preprocessing tokens After the textual transformations are finished, the input file is converted into a sequence of @dfn{preprocessing tokens}. These mostly correspond to the syntactic tokens used by the C compiler, but there are a few differences. White space separates tokens; it is not itself a token of any kind. Tokens do not have to be separated by white space, but it is often necessary to avoid ambiguities. When faced with a sequence of characters that has more than one possible tokenization, the preprocessor is greedy. It always makes each token, starting from the left, as big as possible before moving on to the next token. For instance, @code{a+++++b} is interpreted as @code{@w{a ++ ++ + b}}, not as @code{@w{a ++ + ++ b}}, even though the latter tokenization could be part of a valid C program and the former could not. Once the input file is broken into tokens, the token boundaries never change, except when the @samp{##} preprocessing operator is used to paste tokens together. @xref{Concatenation}. For example, @smallexample @group #define foo() bar foo()baz @expansion{} bar baz @emph{not} @expansion{} barbaz @end group @end smallexample The compiler does not re-tokenize the preprocessor's output. Each preprocessing token becomes one compiler token. @cindex identifiers Preprocessing tokens fall into five broad classes: identifiers, preprocessing numbers, string literals, punctuators, and other. An @dfn{identifier} is the same as an identifier in C: any sequence of letters, digits, or underscores, which begins with a letter or underscore. Keywords of C have no significance to the preprocessor; they are ordinary identifiers. You can define a macro whose name is a keyword, for instance. The only identifier which can be considered a preprocessing keyword is @code{defined}. @xref{Defined}. This is mostly true of other languages which use the C preprocessor. However, a few of the keywords of C++ are significant even in the preprocessor. @xref{C++ Named Operators}. In the 1999 C standard, identifiers may contain letters which are not part of the ``basic source character set'', at the implementation's discretion (such as accented Latin letters, Greek letters, or Chinese ideograms). This may be done with an extended character set, or the @samp{\u} and @samp{\U} escape sequences. The implementation of this feature in GCC is experimental; such characters are only accepted in the @samp{\u} and @samp{\U} forms and only if @option{-fextended-identifiers} is used. As an extension, GCC treats @samp{$} as a letter. This is for compatibility with some systems, such as VMS, where @samp{$} is commonly used in system-defined function and object names. @samp{$} is not a letter in strictly conforming mode, or if you specify the @option{-$} option. @xref{Invocation}. @cindex numbers @cindex preprocessing numbers A @dfn{preprocessing number} has a rather bizarre definition. The category includes all the normal integer and floating point constants one expects of C, but also a number of other things one might not initially recognize as a number. Formally, preprocessing numbers begin with an optional period, a required decimal digit, and then continue with any sequence of letters, digits, underscores, periods, and exponents. Exponents are the two-character sequences @samp{e+}, @samp{e-}, @samp{E+}, @samp{E-}, @samp{p+}, @samp{p-}, @samp{P+}, and @samp{P-}. (The exponents that begin with @samp{p} or @samp{P} are new to C99. They are used for hexadecimal floating-point constants.) The purpose of this unusual definition is to isolate the preprocessor from the full complexity of numeric constants. It does not have to distinguish between lexically valid and invalid floating-point numbers, which is complicated. The definition also permits you to split an identifier at any position and get exactly two tokens, which can then be pasted back together with the @samp{##} operator. It's possible for preprocessing numbers to cause programs to be misinterpreted. For example, @code{0xE+12} is a preprocessing number which does not translate to any valid numeric constant, therefore a syntax error. It does not mean @code{@w{0xE + 12}}, which is what you might have intended. @cindex string literals @cindex string constants @cindex character constants @cindex header file names @c the @: prevents makeinfo from turning '' into ". @dfn{String literals} are string constants, character constants, and header file names (the argument of @samp{#include}).@footnote{The C standard uses the term @dfn{string literal} to refer only to what we are calling @dfn{string constants}.} String constants and character constants are straightforward: @t{"@dots{}"} or @t{'@dots{}'}. In either case embedded quotes should be escaped with a backslash: @t{'\'@:'} is the character constant for @samp{'}. There is no limit on the length of a character constant, but the value of a character constant that contains more than one character is implementation-defined. @xref{Implementation Details}. Header file names either look like string constants, @t{"@dots{}"}, or are written with angle brackets instead, @t{<@dots{}>}. In either case, backslash is an ordinary character. There is no way to escape the closing quote or angle bracket. The preprocessor looks for the header file in different places depending on which form you use. @xref{Include Operation}. No string literal may extend past the end of a line. Older versions of GCC accepted multi-line string constants. You may use continued lines instead, or string constant concatenation. @xref{Differences from previous versions}. @cindex punctuators @cindex digraphs @cindex alternative tokens @dfn{Punctuators} are all the usual bits of punctuation which are meaningful to C and C++. All but three of the punctuation characters in ASCII are C punctuators. The exceptions are @samp{@@}, @samp{$}, and @samp{`}. In addition, all the two- and three-character operators are punctuators. There are also six @dfn{digraphs}, which the C++ standard calls @dfn{alternative tokens}, which are merely alternate ways to spell other punctuators. This is a second attempt to work around missing punctuation in obsolete systems. It has no negative side effects, unlike trigraphs, but does not cover as much ground. The digraphs and their corresponding normal punctuators are: @smallexample Digraph: <% %> <: :> %: %:%: Punctuator: @{ @} [ ] # ## @end smallexample @cindex other tokens Any other single character is considered ``other''. It is passed on to the preprocessor's output unmolested. The C compiler will almost certainly reject source code containing ``other'' tokens. In ASCII, the only other characters are @samp{@@}, @samp{$}, @samp{`}, and control characters other than NUL (all bits zero). (Note that @samp{$} is normally considered a letter.) All characters with the high bit set (numeric range 0x7F--0xFF) are also ``other'' in the present implementation. This will change when proper support for international character sets is added to GCC@. NUL is a special case because of the high probability that its appearance is accidental, and because it may be invisible to the user (many terminals do not display NUL at all). Within comments, NULs are silently ignored, just as any other character would be. In running text, NUL is considered white space. For example, these two directives have the same meaning. @smallexample #define X^@@1 #define X 1 @end smallexample @noindent (where @samp{^@@} is ASCII NUL)@. Within string or character constants, NULs are preserved. In the latter two cases the preprocessor emits a warning message. @node The preprocessing language @section The preprocessing language @cindex directives @cindex preprocessing directives @cindex directive line @cindex directive name After tokenization, the stream of tokens may simply be passed straight to the compiler's parser. However, if it contains any operations in the @dfn{preprocessing language}, it will be transformed first. This stage corresponds roughly to the standard's ``translation phase 4'' and is what most people think of as the preprocessor's job. The preprocessing language consists of @dfn{directives} to be executed and @dfn{macros} to be expanded. Its primary capabilities are: @itemize @bullet @item Inclusion of header files. These are files of declarations that can be substituted into your program. @item Macro expansion. You can define @dfn{macros}, which are abbreviations for arbitrary fragments of C code. The preprocessor will replace the macros with their definitions throughout the program. Some macros are automatically defined for you. @item Conditional compilation. You can include or exclude parts of the program according to various conditions. @item Line control. If you use a program to combine or rearrange source files into an intermediate file which is then compiled, you can use line control to inform the compiler where each source line originally came from. @item Diagnostics. You can detect problems at compile time and issue errors or warnings. @end itemize There are a few more, less useful, features. Except for expansion of predefined macros, all these operations are triggered with @dfn{preprocessing directives}. Preprocessing directives are lines in your program that start with @samp{#}. Whitespace is allowed before and after the @samp{#}. The @samp{#} is followed by an identifier, the @dfn{directive name}. It specifies the operation to perform. Directives are commonly referred to as @samp{#@var{name}} where @var{name} is the directive name. For example, @samp{#define} is the directive that defines a macro. The @samp{#} which begins a directive cannot come from a macro expansion. Also, the directive name is not macro expanded. Thus, if @code{foo} is defined as a macro expanding to @code{define}, that does not make @samp{#foo} a valid preprocessing directive. The set of valid directive names is fixed. Programs cannot define new preprocessing directives. Some directives require arguments; these make up the rest of the directive line and must be separated from the directive name by whitespace. For example, @samp{#define} must be followed by a macro name and the intended expansion of the macro. A preprocessing directive cannot cover more than one line. The line may, however, be continued with backslash-newline, or by a block comment which extends past the end of the line. In either case, when the directive is processed, the continuations have already been merged with the first line to make one long line. @node Header Files @chapter Header Files @cindex header file A header file is a file containing C declarations and macro definitions (@pxref{Macros}) to be shared between several source files. You request the use of a header file in your program by @dfn{including} it, with the C preprocessing directive @samp{#include}. Header files serve two purposes. @itemize @bullet @item @cindex system header files System header files declare the interfaces to parts of the operating system. You include them in your program to supply the definitions and declarations you need to invoke system calls and libraries. @item Your own header files contain declarations for interfaces between the source files of your program. Each time you have a group of related declarations and macro definitions all or most of which are needed in several different source files, it is a good idea to create a header file for them. @end itemize Including a header file produces the same results as copying the header file into each source file that needs it. Such copying would be time-consuming and error-prone. With a header file, the related declarations appear in only one place. If they need to be changed, they can be changed in one place, and programs that include the header file will automatically use the new version when next recompiled. The header file eliminates the labor of finding and changing all the copies as well as the risk that a failure to find one copy will result in inconsistencies within a program. In C, the usual convention is to give header files names that end with @file{.h}. It is most portable to use only letters, digits, dashes, and underscores in header file names, and at most one dot. @menu * Include Syntax:: * Include Operation:: * Search Path:: * Once-Only Headers:: * Computed Includes:: * Wrapper Headers:: * System Headers:: @end menu @node Include Syntax @section Include Syntax @findex #include Both user and system header files are included using the preprocessing directive @samp{#include}. It has two variants: @table @code @item #include <@var{file}> This variant is used for system header files. It searches for a file named @var{file} in a standard list of system directories. You can prepend directories to this list with the @option{-I} option (@pxref{Invocation}). @item #include "@var{file}" This variant is used for header files of your own program. It searches for a file named @var{file} first in the directory containing the current file, then in the quote directories and then the same directories used for @code{<@var{file}>}. You can prepend directories to the list of quote directories with the @option{-iquote} option. @end table The argument of @samp{#include}, whether delimited with quote marks or angle brackets, behaves like a string constant in that comments are not recognized, and macro names are not expanded. Thus, @code{@w{#include }} specifies inclusion of a system header file named @file{x/*y}. However, if backslashes occur within @var{file}, they are considered ordinary text characters, not escape characters. None of the character escape sequences appropriate to string constants in C are processed. Thus, @code{@w{#include "x\n\\y"}} specifies a filename containing three backslashes. (Some systems interpret @samp{\} as a pathname separator. All of these also interpret @samp{/} the same way. It is most portable to use only @samp{/}.) It is an error if there is anything (other than comments) on the line after the file name. @node Include Operation @section Include Operation The @samp{#include} directive works by directing the C preprocessor to scan the specified file as input before continuing with the rest of the current file. The output from the preprocessor contains the output already generated, followed by the output resulting from the included file, followed by the output that comes from the text after the @samp{#include} directive. For example, if you have a header file @file{header.h} as follows, @smallexample char *test (void); @end smallexample @noindent and a main program called @file{program.c} that uses the header file, like this, @smallexample int x; #include "header.h" int main (void) @{ puts (test ()); @} @end smallexample @noindent the compiler will see the same token stream as it would if @file{program.c} read @smallexample int x; char *test (void); int main (void) @{ puts (test ()); @} @end smallexample Included files are not limited to declarations and macro definitions; those are merely the typical uses. Any fragment of a C program can be included from another file. The include file could even contain the beginning of a statement that is concluded in the containing file, or the end of a statement that was started in the including file. However, an included file must consist of complete tokens. Comments and string literals which have not been closed by the end of an included file are invalid. For error recovery, they are considered to end at the end of the file. To avoid confusion, it is best if header files contain only complete syntactic units---function declarations or definitions, type declarations, etc. The line following the @samp{#include} directive is always treated as a separate line by the C preprocessor, even if the included file lacks a final newline. @node Search Path @section Search Path GCC looks in several different places for headers. On a normal Unix system, if you do not instruct it otherwise, it will look for headers requested with @code{@w{#include <@var{file}>}} in: @smallexample /usr/local/include @var{libdir}/gcc/@var{target}/@var{version}/include /usr/@var{target}/include /usr/include @end smallexample For C++ programs, it will also look in @file{/usr/include/g++-v3}, first. In the above, @var{target} is the canonical name of the system GCC was configured to compile code for; often but not always the same as the canonical name of the system it runs on. @var{version} is the version of GCC in use. You can add to this list with the @option{-I@var{dir}} command line option. All the directories named by @option{-I} are searched, in left-to-right order, @emph{before} the default directories. The only exception is when @file{dir} is already searched by default. In this case, the option is ignored and the search order for system directories remains unchanged. Duplicate directories are removed from the quote and bracket search chains before the two chains are merged to make the final search chain. Thus, it is possible for a directory to occur twice in the final search chain if it was specified in both the quote and bracket chains. You can prevent GCC from searching any of the default directories with the @option{-nostdinc} option. This is useful when you are compiling an operating system kernel or some other program that does not use the standard C library facilities, or the standard C library itself. @option{-I} options are not ignored as described above when @option{-nostdinc} is in effect. GCC looks for headers requested with @code{@w{#include "@var{file}"}} first in the directory containing the current file, then in the directories as specified by @option{-iquote} options, then in the same places it would have looked for a header requested with angle brackets. For example, if @file{/usr/include/sys/stat.h} contains @code{@w{#include "types.h"}}, GCC looks for @file{types.h} first in @file{/usr/include/sys}, then in its usual search path. @samp{#line} (@pxref{Line Control}) does not change GCC's idea of the directory containing the current file. You may put @option{-I-} at any point in your list of @option{-I} options. This has two effects. First, directories appearing before the @option{-I-} in the list are searched only for headers requested with quote marks. Directories after @option{-I-} are searched for all headers. Second, the directory containing the current file is not searched for anything, unless it happens to be one of the directories named by an @option{-I} switch. @option{-I-} is deprecated, @option{-iquote} should be used instead. @option{-I. -I-} is not the same as no @option{-I} options at all, and does not cause the same behavior for @samp{<>} includes that @samp{""} includes get with no special options. @option{-I.} searches the compiler's current working directory for header files. That may or may not be the same as the directory containing the current file. If you need to look for headers in a directory named @file{-}, write @option{-I./-}. There are several more ways to adjust the header search path. They are generally less useful. @xref{Invocation}. @node Once-Only Headers @section Once-Only Headers @cindex repeated inclusion @cindex including just once @cindex wrapper @code{#ifndef} If a header file happens to be included twice, the compiler will process its contents twice. This is very likely to cause an error, e.g.@: when the compiler sees the same structure definition twice. Even if it does not, it will certainly waste time. The standard way to prevent this is to enclose the entire real contents of the file in a conditional, like this: @smallexample @group /* File foo. */ #ifndef FILE_FOO_SEEN #define FILE_FOO_SEEN @var{the entire file} #endif /* !FILE_FOO_SEEN */ @end group @end smallexample This construct is commonly known as a @dfn{wrapper #ifndef}. When the header is included again, the conditional will be false, because @code{FILE_FOO_SEEN} is defined. The preprocessor will skip over the entire contents of the file, and the compiler will not see it twice. CPP optimizes even further. It remembers when a header file has a wrapper @samp{#ifndef}. If a subsequent @samp{#include} specifies that header, and the macro in the @samp{#ifndef} is still defined, it does not bother to rescan the file at all. You can put comments outside the wrapper. They will not interfere with this optimization. @cindex controlling macro @cindex guard macro The macro @code{FILE_FOO_SEEN} is called the @dfn{controlling macro} or @dfn{guard macro}. In a user header file, the macro name should not begin with @samp{_}. In a system header file, it should begin with @samp{__} to avoid conflicts with user programs. In any kind of header file, the macro name should contain the name of the file and some additional text, to avoid conflicts with other header files. @node Computed Includes @section Computed Includes @cindex computed includes @cindex macros in include Sometimes it is necessary to select one of several different header files to be included into your program. They might specify configuration parameters to be used on different sorts of operating systems, for instance. You could do this with a series of conditionals, @smallexample #if SYSTEM_1 # include "system_1.h" #elif SYSTEM_2 # include "system_2.h" #elif SYSTEM_3 @dots{} #endif @end smallexample That rapidly becomes tedious. Instead, the preprocessor offers the ability to use a macro for the header name. This is called a @dfn{computed include}. Instead of writing a header name as the direct argument of @samp{#include}, you simply put a macro name there instead: @smallexample #define SYSTEM_H "system_1.h" @dots{} #include SYSTEM_H @end smallexample @noindent @code{SYSTEM_H} will be expanded, and the preprocessor will look for @file{system_1.h} as if the @samp{#include} had been written that way originally. @code{SYSTEM_H} could be defined by your Makefile with a @option{-D} option. You must be careful when you define the macro. @samp{#define} saves tokens, not text. The preprocessor has no way of knowing that the macro will be used as the argument of @samp{#include}, so it generates ordinary tokens, not a header name. This is unlikely to cause problems if you use double-quote includes, which are close enough to string constants. If you use angle brackets, however, you may have trouble. The syntax of a computed include is actually a bit more general than the above. If the first non-whitespace character after @samp{#include} is not @samp{"} or @samp{<}, then the entire line is macro-expanded like running text would be. If the line expands to a single string constant, the contents of that string constant are the file to be included. CPP does not re-examine the string for embedded quotes, but neither does it process backslash escapes in the string. Therefore @smallexample #define HEADER "a\"b" #include HEADER @end smallexample @noindent looks for a file named @file{a\"b}. CPP searches for the file according to the rules for double-quoted includes. If the line expands to a token stream beginning with a @samp{<} token and including a @samp{>} token, then the tokens between the @samp{<} and the first @samp{>} are combined to form the filename to be included. Any whitespace between tokens is reduced to a single space; then any space after the initial @samp{<} is retained, but a trailing space before the closing @samp{>} is ignored. CPP searches for the file according to the rules for angle-bracket includes. In either case, if there are any tokens on the line after the file name, an error occurs and the directive is not processed. It is also an error if the result of expansion does not match either of the two expected forms. These rules are implementation-defined behavior according to the C standard. To minimize the risk of different compilers interpreting your computed includes differently, we recommend you use only a single object-like macro which expands to a string constant. This will also minimize confusion for people reading your program. @node Wrapper Headers @section Wrapper Headers @cindex wrapper headers @cindex overriding a header file @findex #include_next Sometimes it is necessary to adjust the contents of a system-provided header file without editing it directly. GCC's @command{fixincludes} operation does this, for example. One way to do that would be to create a new header file with the same name and insert it in the search path before the original header. That works fine as long as you're willing to replace the old header entirely. But what if you want to refer to the old header from the new one? You cannot simply include the old header with @samp{#include}. That will start from the beginning, and find your new header again. If your header is not protected from multiple inclusion (@pxref{Once-Only Headers}), it will recurse infinitely and cause a fatal error. You could include the old header with an absolute pathname: @smallexample #include "/usr/include/old-header.h" @end smallexample @noindent This works, but is not clean; should the system headers ever move, you would have to edit the new headers to match. There is no way to solve this problem within the C standard, but you can use the GNU extension @samp{#include_next}. It means, ``Include the @emph{next} file with this name''. This directive works like @samp{#include} except in searching for the specified file: it starts searching the list of header file directories @emph{after} the directory in which the current file was found. Suppose you specify @option{-I /usr/local/include}, and the list of directories to search also includes @file{/usr/include}; and suppose both directories contain @file{signal.h}. Ordinary @code{@w{#include }} finds the file under @file{/usr/local/include}. If that file contains @code{@w{#include_next }}, it starts searching after that directory, and finds the file in @file{/usr/include}. @samp{#include_next} does not distinguish between @code{<@var{file}>} and @code{"@var{file}"} inclusion, nor does it check that the file you specify has the same name as the current file. It simply looks for the file named, starting with the directory in the search path after the one where the current file was found. The use of @samp{#include_next} can lead to great confusion. We recommend it be used only when there is no other alternative. In particular, it should not be used in the headers belonging to a specific program; it should be used only to make global corrections along the lines of @command{fixincludes}. @node System Headers @section System Headers @cindex system header files The header files declaring interfaces to the operating system and runtime libraries often cannot be written in strictly conforming C@. Therefore, GCC gives code found in @dfn{system headers} special treatment. All warnings, other than those generated by @samp{#warning} (@pxref{Diagnostics}), are suppressed while GCC is processing a system header. Macros defined in a system header are immune to a few warnings wherever they are expanded. This immunity is granted on an ad-hoc basis, when we find that a warning generates lots of false positives because of code in macros defined in system headers. Normally, only the headers found in specific directories are considered system headers. These directories are determined when GCC is compiled. There are, however, two ways to make normal headers into system headers. The @option{-isystem} command line option adds its argument to the list of directories to search for headers, just like @option{-I}. Any headers found in that directory will be considered system headers. All directories named by @option{-isystem} are searched @emph{after} all directories named by @option{-I}, no matter what their order was on the command line. If the same directory is named by both @option{-I} and @option{-isystem}, the @option{-I} option is ignored. GCC provides an informative message when this occurs if @option{-v} is used. @findex #pragma GCC system_header There is also a directive, @code{@w{#pragma GCC system_header}}, which tells GCC to consider the rest of the current include file a system header, no matter where it was found. Code that comes before the @samp{#pragma} in the file will not be affected. @code{@w{#pragma GCC system_header}} has no effect in the primary source file. On very old systems, some of the pre-defined system header directories get even more special treatment. GNU C++ considers code in headers found in those directories to be surrounded by an @code{@w{extern "C"}} block. There is no way to request this behavior with a @samp{#pragma}, or from the command line. @node Macros @chapter Macros A @dfn{macro} is a fragment of code which has been given a name. Whenever the name is used, it is replaced by the contents of the macro. There are two kinds of macros. They differ mostly in what they look like when they are used. @dfn{Object-like} macros resemble data objects when used, @dfn{function-like} macros resemble function calls. You may define any valid identifier as a macro, even if it is a C keyword. The preprocessor does not know anything about keywords. This can be useful if you wish to hide a keyword such as @code{const} from an older compiler that does not understand it. However, the preprocessor operator @code{defined} (@pxref{Defined}) can never be defined as a macro, and C++'s named operators (@pxref{C++ Named Operators}) cannot be macros when you are compiling C++. @menu * Object-like Macros:: * Function-like Macros:: * Macro Arguments:: * Stringification:: * Concatenation:: * Variadic Macros:: * Predefined Macros:: * Undefining and Redefining Macros:: * Directives Within Macro Arguments:: * Macro Pitfalls:: @end menu @node Object-like Macros @section Object-like Macros @cindex object-like macro @cindex symbolic constants @cindex manifest constants An @dfn{object-like macro} is a simple identifier which will be replaced by a code fragment. It is called object-like because it looks like a data object in code that uses it. They are most commonly used to give symbolic names to numeric constants. @findex #define You create macros with the @samp{#define} directive. @samp{#define} is followed by the name of the macro and then the token sequence it should be an abbreviation for, which is variously referred to as the macro's @dfn{body}, @dfn{expansion} or @dfn{replacement list}. For example, @smallexample #define BUFFER_SIZE 1024 @end smallexample @noindent defines a macro named @code{BUFFER_SIZE} as an abbreviation for the token @code{1024}. If somewhere after this @samp{#define} directive there comes a C statement of the form @smallexample foo = (char *) malloc (BUFFER_SIZE); @end smallexample @noindent then the C preprocessor will recognize and @dfn{expand} the macro @code{BUFFER_SIZE}. The C compiler will see the same tokens as it would if you had written @smallexample foo = (char *) malloc (1024); @end smallexample By convention, macro names are written in uppercase. Programs are easier to read when it is possible to tell at a glance which names are macros. The macro's body ends at the end of the @samp{#define} line. You may continue the definition onto multiple lines, if necessary, using backslash-newline. When the macro is expanded, however, it will all come out on one line. For example, @smallexample #define NUMBERS 1, \ 2, \ 3 int x[] = @{ NUMBERS @}; @expansion{} int x[] = @{ 1, 2, 3 @}; @end smallexample @noindent The most common visible consequence of this is surprising line numbers in error messages. There is no restriction on what can go in a macro body provided it decomposes into valid preprocessing tokens. Parentheses need not balance, and the body need not resemble valid C code. (If it does not, you may get error messages from the C compiler when you use the macro.) The C preprocessor scans your program sequentially. Macro definitions take effect at the place you write them. Therefore, the following input to the C preprocessor @smallexample foo = X; #define X 4 bar = X; @end smallexample @noindent produces @smallexample foo = X; bar = 4; @end smallexample When the preprocessor expands a macro name, the macro's expansion replaces the macro invocation, then the expansion is examined for more macros to expand. For example, @smallexample @group #define TABLESIZE BUFSIZE #define BUFSIZE 1024 TABLESIZE @expansion{} BUFSIZE @expansion{} 1024 @end group @end smallexample @noindent @code{TABLESIZE} is expanded first to produce @code{BUFSIZE}, then that macro is expanded to produce the final result, @code{1024}. Notice that @code{BUFSIZE} was not defined when @code{TABLESIZE} was defined. The @samp{#define} for @code{TABLESIZE} uses exactly the expansion you specify---in this case, @code{BUFSIZE}---and does not check to see whether it too contains macro names. Only when you @emph{use} @code{TABLESIZE} is the result of its expansion scanned for more macro names. This makes a difference if you change the definition of @code{BUFSIZE} at some point in the source file. @code{TABLESIZE}, defined as shown, will always expand using the definition of @code{BUFSIZE} that is currently in effect: @smallexample #define BUFSIZE 1020 #define TABLESIZE BUFSIZE #undef BUFSIZE #define BUFSIZE 37 @end smallexample @noindent Now @code{TABLESIZE} expands (in two stages) to @code{37}. If the expansion of a macro contains its own name, either directly or via intermediate macros, it is not expanded again when the expansion is examined for more macros. This prevents infinite recursion. @xref{Self-Referential Macros}, for the precise details. @node Function-like Macros @section Function-like Macros @cindex function-like macros You can also define macros whose use looks like a function call. These are called @dfn{function-like macros}. To define a function-like macro, you use the same @samp{#define} directive, but you put a pair of parentheses immediately after the macro name. For example, @smallexample #define lang_init() c_init() lang_init() @expansion{} c_init() @end smallexample A function-like macro is only expanded if its name appears with a pair of parentheses after it. If you write just the name, it is left alone. This can be useful when you have a function and a macro of the same name, and you wish to use the function sometimes. @smallexample extern void foo(void); #define foo() /* @r{optimized inline version} */ @dots{} foo(); funcptr = foo; @end smallexample Here the call to @code{foo()} will use the macro, but the function pointer will get the address of the real function. If the macro were to be expanded, it would cause a syntax error. If you put spaces between the macro name and the parentheses in the macro definition, that does not define a function-like macro, it defines an object-like macro whose expansion happens to begin with a pair of parentheses. @smallexample #define lang_init () c_init() lang_init() @expansion{} () c_init()() @end smallexample The first two pairs of parentheses in this expansion come from the macro. The third is the pair that was originally after the macro invocation. Since @code{lang_init} is an object-like macro, it does not consume those parentheses. @node Macro Arguments @section Macro Arguments @cindex arguments @cindex macros with arguments @cindex arguments in macro definitions Function-like macros can take @dfn{arguments}, just like true functions. To define a macro that uses arguments, you insert @dfn{parameters} between the pair of parentheses in the macro definition that make the macro function-like. The parameters must be valid C identifiers, separated by commas and optionally whitespace. To invoke a macro that takes arguments, you write the name of the macro followed by a list of @dfn{actual arguments} in parentheses, separated by commas. The invocation of the macro need not be restricted to a single logical line---it can cross as many lines in the source file as you wish. The number of arguments you give must match the number of parameters in the macro definition. When the macro is expanded, each use of a parameter in its body is replaced by the tokens of the corresponding argument. (You need not use all of the parameters in the macro body.) As an example, here is a macro that computes the minimum of two numeric values, as it is defined in many C programs, and some uses. @smallexample #define min(X, Y) ((X) < (Y) ? (X) : (Y)) x = min(a, b); @expansion{} x = ((a) < (b) ? (a) : (b)); y = min(1, 2); @expansion{} y = ((1) < (2) ? (1) : (2)); z = min(a + 28, *p); @expansion{} z = ((a + 28) < (*p) ? (a + 28) : (*p)); @end smallexample @noindent (In this small example you can already see several of the dangers of macro arguments. @xref{Macro Pitfalls}, for detailed explanations.) Leading and trailing whitespace in each argument is dropped, and all whitespace between the tokens of an argument is reduced to a single space. Parentheses within each argument must balance; a comma within such parentheses does not end the argument. However, there is no requirement for square brackets or braces to balance, and they do not prevent a comma from separating arguments. Thus, @smallexample macro (array[x = y, x + 1]) @end smallexample @noindent passes two arguments to @code{macro}: @code{array[x = y} and @code{x + 1]}. If you want to supply @code{array[x = y, x + 1]} as an argument, you can write it as @code{array[(x = y, x + 1)]}, which is equivalent C code. All arguments to a macro are completely macro-expanded before they are substituted into the macro body. After substitution, the complete text is scanned again for macros to expand, including the arguments. This rule may seem strange, but it is carefully designed so you need not worry about whether any function call is actually a macro invocation. You can run into trouble if you try to be too clever, though. @xref{Argument Prescan}, for detailed discussion. For example, @code{min (min (a, b), c)} is first expanded to @smallexample min (((a) < (b) ? (a) : (b)), (c)) @end smallexample @noindent and then to @smallexample @group ((((a) < (b) ? (a) : (b))) < (c) ? (((a) < (b) ? (a) : (b))) : (c)) @end group @end smallexample @noindent (Line breaks shown here for clarity would not actually be generated.) @cindex empty macro arguments You can leave macro arguments empty; this is not an error to the preprocessor (but many macros will then expand to invalid code). You cannot leave out arguments entirely; if a macro takes two arguments, there must be exactly one comma at the top level of its argument list. Here are some silly examples using @code{min}: @smallexample min(, b) @expansion{} (( ) < (b) ? ( ) : (b)) min(a, ) @expansion{} ((a ) < ( ) ? (a ) : ( )) min(,) @expansion{} (( ) < ( ) ? ( ) : ( )) min((,),) @expansion{} (((,)) < ( ) ? ((,)) : ( )) min() @error{} macro "min" requires 2 arguments, but only 1 given min(,,) @error{} macro "min" passed 3 arguments, but takes just 2 @end smallexample Whitespace is not a preprocessing token, so if a macro @code{foo} takes one argument, @code{@w{foo ()}} and @code{@w{foo ( )}} both supply it an empty argument. Previous GNU preprocessor implementations and documentation were incorrect on this point, insisting that a function-like macro that takes a single argument be passed a space if an empty argument was required. Macro parameters appearing inside string literals are not replaced by their corresponding actual arguments. @smallexample #define foo(x) x, "x" foo(bar) @expansion{} bar, "x" @end smallexample @node Stringification @section Stringification @cindex stringification @cindex @samp{#} operator Sometimes you may want to convert a macro argument into a string constant. Parameters are not replaced inside string constants, but you can use the @samp{#} preprocessing operator instead. When a macro parameter is used with a leading @samp{#}, the preprocessor replaces it with the literal text of the actual argument, converted to a string constant. Unlike normal parameter replacement, the argument is not macro-expanded first. This is called @dfn{stringification}. There is no way to combine an argument with surrounding text and stringify it all together. Instead, you can write a series of adjacent string constants and stringified arguments. The preprocessor will replace the stringified arguments with string constants. The C compiler will then combine all the adjacent string constants into one long string. Here is an example of a macro definition that uses stringification: @smallexample @group #define WARN_IF(EXP) \ do @{ if (EXP) \ fprintf (stderr, "Warning: " #EXP "\n"); @} \ while (0) WARN_IF (x == 0); @expansion{} do @{ if (x == 0) fprintf (stderr, "Warning: " "x == 0" "\n"); @} while (0); @end group @end smallexample @noindent The argument for @code{EXP} is substituted once, as-is, into the @code{if} statement, and once, stringified, into the argument to @code{fprintf}. If @code{x} were a macro, it would be expanded in the @code{if} statement, but not in the string. The @code{do} and @code{while (0)} are a kludge to make it possible to write @code{WARN_IF (@var{arg});}, which the resemblance of @code{WARN_IF} to a function would make C programmers want to do; see @ref{Swallowing the Semicolon}. Stringification in C involves more than putting double-quote characters around the fragment. The preprocessor backslash-escapes the quotes surrounding embedded string constants, and all backslashes within string and character constants, in order to get a valid C string constant with the proper contents. Thus, stringifying @code{@w{p = "foo\n";}} results in @t{@w{"p = \"foo\\n\";"}}. However, backslashes that are not inside string or character constants are not duplicated: @samp{\n} by itself stringifies to @t{"\n"}. All leading and trailing whitespace in text being stringified is ignored. Any sequence of whitespace in the middle of the text is converted to a single space in the stringified result. Comments are replaced by whitespace long before stringification happens, so they never appear in stringified text. There is no way to convert a macro argument into a character constant. If you want to stringify the result of expansion of a macro argument, you have to use two levels of macros. @smallexample #define xstr(s) str(s) #define str(s) #s #define foo 4 str (foo) @expansion{} "foo" xstr (foo) @expansion{} xstr (4) @expansion{} str (4) @expansion{} "4" @end smallexample @code{s} is stringified when it is used in @code{str}, so it is not macro-expanded first. But @code{s} is an ordinary argument to @code{xstr}, so it is completely macro-expanded before @code{xstr} itself is expanded (@pxref{Argument Prescan}). Therefore, by the time @code{str} gets to its argument, it has already been macro-expanded. @node Concatenation @section Concatenation @cindex concatenation @cindex token pasting @cindex token concatenation @cindex @samp{##} operator It is often useful to merge two tokens into one while expanding macros. This is called @dfn{token pasting} or @dfn{token concatenation}. The @samp{##} preprocessing operator performs token pasting. When a macro is expanded, the two tokens on either side of each @samp{##} operator are combined into a single token, which then replaces the @samp{##} and the two original tokens in the macro expansion. Usually both will be identifiers, or one will be an identifier and the other a preprocessing number. When pasted, they make a longer identifier. This isn't the only valid case. It is also possible to concatenate two numbers (or a number and a name, such as @code{1.5} and @code{e3}) into a number. Also, multi-character operators such as @code{+=} can be formed by token pasting. However, two tokens that don't together form a valid token cannot be pasted together. For example, you cannot concatenate @code{x} with @code{+} in either order. If you try, the preprocessor issues a warning and emits the two tokens. Whether it puts white space between the tokens is undefined. It is common to find unnecessary uses of @samp{##} in complex macros. If you get this warning, it is likely that you can simply remove the @samp{##}. Both the tokens combined by @samp{##} could come from the macro body, but you could just as well write them as one token in the first place. Token pasting is most useful when one or both of the tokens comes from a macro argument. If either of the tokens next to an @samp{##} is a parameter name, it is replaced by its actual argument before @samp{##} executes. As with stringification, the actual argument is not macro-expanded first. If the argument is empty, that @samp{##} has no effect. Keep in mind that the C preprocessor converts comments to whitespace before macros are even considered. Therefore, you cannot create a comment by concatenating @samp{/} and @samp{*}. You can put as much whitespace between @samp{##} and its operands as you like, including comments, and you can put comments in arguments that will be concatenated. However, it is an error if @samp{##} appears at either end of a macro body. Consider a C program that interprets named commands. There probably needs to be a table of commands, perhaps an array of structures declared as follows: @smallexample @group struct command @{ char *name; void (*function) (void); @}; @end group @group struct command commands[] = @{ @{ "quit", quit_command @}, @{ "help", help_command @}, @dots{} @}; @end group @end smallexample It would be cleaner not to have to give each command name twice, once in the string constant and once in the function name. A macro which takes the name of a command as an argument can make this unnecessary. The string constant can be created with stringification, and the function name by concatenating the argument with @samp{_command}. Here is how it is done: @smallexample #define COMMAND(NAME) @{ #NAME, NAME ## _command @} struct command commands[] = @{ COMMAND (quit), COMMAND (help), @dots{} @}; @end smallexample @node Variadic Macros @section Variadic Macros @cindex variable number of arguments @cindex macros with variable arguments @cindex variadic macros A macro can be declared to accept a variable number of arguments much as a function can. The syntax for defining the macro is similar to that of a function. Here is an example: @smallexample #define eprintf(@dots{}) fprintf (stderr, __VA_ARGS__) @end smallexample This kind of macro is called @dfn{variadic}. When the macro is invoked, all the tokens in its argument list after the last named argument (this macro has none), including any commas, become the @dfn{variable argument}. This sequence of tokens replaces the identifier @code{@w{__VA_ARGS__}} in the macro body wherever it appears. Thus, we have this expansion: @smallexample eprintf ("%s:%d: ", input_file, lineno) @expansion{} fprintf (stderr, "%s:%d: ", input_file, lineno) @end smallexample The variable argument is completely macro-expanded before it is inserted into the macro expansion, just like an ordinary argument. You may use the @samp{#} and @samp{##} operators to stringify the variable argument or to paste its leading or trailing token with another token. (But see below for an important special case for @samp{##}.) If your macro is complicated, you may want a more descriptive name for the variable argument than @code{@w{__VA_ARGS__}}. CPP permits this, as an extension. You may write an argument name immediately before the @samp{@dots{}}; that name is used for the variable argument. The @code{eprintf} macro above could be written @smallexample #define eprintf(args@dots{}) fprintf (stderr, args) @end smallexample @noindent using this extension. You cannot use @code{@w{__VA_ARGS__}} and this extension in the same macro. You can have named arguments as well as variable arguments in a variadic macro. We could define @code{eprintf} like this, instead: @smallexample #define eprintf(format, @dots{}) fprintf (stderr, format, __VA_ARGS__) @end smallexample @noindent This formulation looks more descriptive, but unfortunately it is less flexible: you must now supply at least one argument after the format string. In standard C, you cannot omit the comma separating the named argument from the variable arguments. Furthermore, if you leave the variable argument empty, you will get a syntax error, because there will be an extra comma after the format string. @smallexample eprintf("success!\n", ); @expansion{} fprintf(stderr, "success!\n", ); @end smallexample GNU CPP has a pair of extensions which deal with this problem. First, you are allowed to leave the variable argument out entirely: @smallexample eprintf ("success!\n") @expansion{} fprintf(stderr, "success!\n", ); @end smallexample @noindent Second, the @samp{##} token paste operator has a special meaning when placed between a comma and a variable argument. If you write @smallexample #define eprintf(format, @dots{}) fprintf (stderr, format, ##__VA_ARGS__) @end smallexample @noindent and the variable argument is left out when the @code{eprintf} macro is used, then the comma before the @samp{##} will be deleted. This does @emph{not} happen if you pass an empty argument, nor does it happen if the token preceding @samp{##} is anything other than a comma. @smallexample eprintf ("success!\n") @expansion{} fprintf(stderr, "success!\n"); @end smallexample @noindent The above explanation is ambiguous about the case where the only macro parameter is a variable arguments parameter, as it is meaningless to try to distinguish whether no argument at all is an empty argument or a missing argument. In this case the C99 standard is clear that the comma must remain, however the existing GCC extension used to swallow the comma. So CPP retains the comma when conforming to a specific C standard, and drops it otherwise. C99 mandates that the only place the identifier @code{@w{__VA_ARGS__}} can appear is in the replacement list of a variadic macro. It may not be used as a macro name, macro argument name, or within a different type of macro. It may also be forbidden in open text; the standard is ambiguous. We recommend you avoid using it except for its defined purpose. Variadic macros are a new feature in C99. GNU CPP has supported them for a long time, but only with a named variable argument (@samp{args@dots{}}, not @samp{@dots{}} and @code{@w{__VA_ARGS__}}). If you are concerned with portability to previous versions of GCC, you should use only named variable arguments. On the other hand, if you are concerned with portability to other conforming implementations of C99, you should use only @code{@w{__VA_ARGS__}}. Previous versions of CPP implemented the comma-deletion extension much more generally. We have restricted it in this release to minimize the differences from C99. To get the same effect with both this and previous versions of GCC, the token preceding the special @samp{##} must be a comma, and there must be white space between that comma and whatever comes immediately before it: @smallexample #define eprintf(format, args@dots{}) fprintf (stderr, format , ##args) @end smallexample @noindent @xref{Differences from previous versions}, for the gory details. @node Predefined Macros @section Predefined Macros @cindex predefined macros Several object-like macros are predefined; you use them without supplying their definitions. They fall into three classes: standard, common, and system-specific. In C++, there is a fourth category, the named operators. They act like predefined macros, but you cannot undefine them. @menu * Standard Predefined Macros:: * Common Predefined Macros:: * System-specific Predefined Macros:: * C++ Named Operators:: @end menu @node Standard Predefined Macros @subsection Standard Predefined Macros @cindex standard predefined macros. The standard predefined macros are specified by the relevant language standards, so they are available with all compilers that implement those standards. Older compilers may not provide all of them. Their names all start with double underscores. @table @code @item __FILE__ This macro expands to the name of the current input file, in the form of a C string constant. This is the path by which the preprocessor opened the file, not the short name specified in @samp{#include} or as the input file name argument. For example, @code{"/usr/local/include/myheader.h"} is a possible expansion of this macro. @item __LINE__ This macro expands to the current input line number, in the form of a decimal integer constant. While we call it a predefined macro, it's a pretty strange macro, since its ``definition'' changes with each new line of source code. @end table @code{__FILE__} and @code{__LINE__} are useful in generating an error message to report an inconsistency detected by the program; the message can state the source line at which the inconsistency was detected. For example, @smallexample fprintf (stderr, "Internal error: " "negative string length " "%d at %s, line %d.", length, __FILE__, __LINE__); @end smallexample An @samp{#include} directive changes the expansions of @code{__FILE__} and @code{__LINE__} to correspond to the included file. At the end of that file, when processing resumes on the input file that contained the @samp{#include} directive, the expansions of @code{__FILE__} and @code{__LINE__} revert to the values they had before the @samp{#include} (but @code{__LINE__} is then incremented by one as processing moves to the line after the @samp{#include}). A @samp{#line} directive changes @code{__LINE__}, and may change @code{__FILE__} as well. @xref{Line Control}. C99 introduces @code{__func__}, and GCC has provided @code{__FUNCTION__} for a long time. Both of these are strings containing the name of the current function (there are slight semantic differences; see the GCC manual). Neither of them is a macro; the preprocessor does not know the name of the current function. They tend to be useful in conjunction with @code{__FILE__} and @code{__LINE__}, though. @table @code @item __DATE__ This macro expands to a string constant that describes the date on which the preprocessor is being run. The string constant contains eleven characters and looks like @code{@w{"Feb 12 1996"}}. If the day of the month is less than 10, it is padded with a space on the left. If GCC cannot determine the current date, it will emit a warning message (once per compilation) and @code{__DATE__} will expand to @code{@w{"??? ?? ????"}}. @item __TIME__ This macro expands to a string constant that describes the time at which the preprocessor is being run. The string constant contains eight characters and looks like @code{"23:59:01"}. If GCC cannot determine the current time, it will emit a warning message (once per compilation) and @code{__TIME__} will expand to @code{"??:??:??"}. @item __STDC__ In normal operation, this macro expands to the constant 1, to signify that this compiler conforms to ISO Standard C@. If GNU CPP is used with a compiler other than GCC, this is not necessarily true; however, the preprocessor always conforms to the standard unless the @option{-traditional-cpp} option is used. This macro is not defined if the @option{-traditional-cpp} option is used. On some hosts, the system compiler uses a different convention, where @code{__STDC__} is normally 0, but is 1 if the user specifies strict conformance to the C Standard. CPP follows the host convention when processing system header files, but when processing user files @code{__STDC__} is always 1. This has been reported to cause problems; for instance, some versions of Solaris provide X Windows headers that expect @code{__STDC__} to be either undefined or 1. @xref{Invocation}. @item __STDC_VERSION__ This macro expands to the C Standard's version number, a long integer constant of the form @code{@var{yyyy}@var{mm}L} where @var{yyyy} and @var{mm} are the year and month of the Standard version. This signifies which version of the C Standard the compiler conforms to. Like @code{__STDC__}, this is not necessarily accurate for the entire implementation, unless GNU CPP is being used with GCC@. The value @code{199409L} signifies the 1989 C standard as amended in 1994, which is the current default; the value @code{199901L} signifies the 1999 revision of the C standard. Support for the 1999 revision is not yet complete. This macro is not defined if the @option{-traditional-cpp} option is used, nor when compiling C++. @item __STDC_HOSTED__ This macro is defined, with value 1, if the compiler's target is a @dfn{hosted environment}. A hosted environment has the complete facilities of the standard C library available. @item __cplusplus This macro is defined when the C++ compiler is in use. You can use @code{__cplusplus} to test whether a header is compiled by a C compiler or a C++ compiler. This macro is similar to @code{__STDC_VERSION__}, in that it expands to a version number. A fully conforming implementation of the 1998 C++ standard will define this macro to @code{199711L}. The GNU C++ compiler is not yet fully conforming, so it uses @code{1} instead. It is hoped to complete the implementation of standard C++ in the near future. @item __ASSEMBLER__ This macro is defined with value 1 when preprocessing assembly language. @end table @node Common Predefined Macros @subsection Common Predefined Macros @cindex common predefined macros The common predefined macros are GNU C extensions. They are available with the same meanings regardless of the machine or operating system on which you are using GNU C@. Their names all start with double underscores. @table @code +@item __COUNTER__ +This macro expands to sequential integral values starting from 0. In +conjuction with the @code{##} operator, this provides a convenient means to +generate unique identifiers. Care must be taken to ensure that +@code{__COUNTER__} is not expanded prior to inclusion of precompiled headers +which use it. Otherwise, the precompiled headers will not be used. + @item __GNUC__ @itemx __GNUC_MINOR__ @itemx __GNUC_PATCHLEVEL__ These macros are defined by all GNU compilers that use the C preprocessor: C and C++. Their values are the major version, minor version, and patch level of the compiler, as integer constants. For example, GCC 3.2.1 will define @code{__GNUC__} to 3, @code{__GNUC_MINOR__} to 2, and @code{__GNUC_PATCHLEVEL__} to 1. These macros are also defined if you invoke the preprocessor directly. @code{__GNUC_PATCHLEVEL__} is new to GCC 3.0; it is also present in the widely-used development snapshots leading up to 3.0 (which identify themselves as GCC 2.96 or 2.97, depending on which snapshot you have). If all you need to know is whether or not your program is being compiled by GCC, or a non-GCC compiler that claims to accept the GNU C dialects, you can simply test @code{__GNUC__}. If you need to write code which depends on a specific version, you must be more careful. Each time the minor version is increased, the patch level is reset to zero; each time the major version is increased (which happens rarely), the minor version and patch level are reset. If you wish to use the predefined macros directly in the conditional, you will need to write it like this: @smallexample /* @r{Test for GCC > 3.2.0} */ #if __GNUC__ > 3 || \ (__GNUC__ == 3 && (__GNUC_MINOR__ > 2 || \ (__GNUC_MINOR__ == 2 && \ __GNUC_PATCHLEVEL__ > 0)) @end smallexample @noindent Another approach is to use the predefined macros to calculate a single number, then compare that against a threshold: @smallexample #define GCC_VERSION (__GNUC__ * 10000 \ + __GNUC_MINOR__ * 100 \ + __GNUC_PATCHLEVEL__) @dots{} /* @r{Test for GCC > 3.2.0} */ #if GCC_VERSION > 30200 @end smallexample @noindent Many people find this form easier to understand. @item __GNUG__ The GNU C++ compiler defines this. Testing it is equivalent to testing @code{@w{(__GNUC__ && __cplusplus)}}. @item __STRICT_ANSI__ GCC defines this macro if and only if the @option{-ansi} switch, or a @option{-std} switch specifying strict conformance to some version of ISO C, was specified when GCC was invoked. It is defined to @samp{1}. This macro exists primarily to direct GNU libc's header files to restrict their definitions to the minimal set found in the 1989 C standard. @item __BASE_FILE__ This macro expands to the name of the main input file, in the form of a C string constant. This is the source file that was specified on the command line of the preprocessor or C compiler. @item __INCLUDE_LEVEL__ This macro expands to a decimal integer constant that represents the depth of nesting in include files. The value of this macro is incremented on every @samp{#include} directive and decremented at the end of every included file. It starts out at 0, it's value within the base file specified on the command line. @item __ELF__ This macro is defined if the target uses the ELF object format. @item __VERSION__ This macro expands to a string constant which describes the version of the compiler in use. You should not rely on its contents having any particular form, but it can be counted on to contain at least the release number. @item __OPTIMIZE__ @itemx __OPTIMIZE_SIZE__ @itemx __NO_INLINE__ These macros describe the compilation mode. @code{__OPTIMIZE__} is defined in all optimizing compilations. @code{__OPTIMIZE_SIZE__} is defined if the compiler is optimizing for size, not speed. @code{__NO_INLINE__} is defined if no functions will be inlined into their callers (when not optimizing, or when inlining has been specifically disabled by @option{-fno-inline}). These macros cause certain GNU header files to provide optimized definitions, using macros or inline functions, of system library functions. You should not use these macros in any way unless you make sure that programs will execute with the same effect whether or not they are defined. If they are defined, their value is 1. @item __GNUC_GNU_INLINE__ GCC defines this macro if functions declared @code{inline} will be handled in GCC's traditional gnu89 mode. In this mode an @code{extern inline} function will never be compiled as a standalone function, and an @code{inline} function which is neither @code{extern} nor @code{static} will always be compiled as a standalone function. @item __GNUC_STDC_INLINE__ GCC defines this macro if functions declared @code{inline} will be handled according to the ISO C99 standard. In this mode an @code{extern inline} function will always be compiled as a standalone externally visible function, and an @code{inline} function which is neither @code{extern} nor @code{static} will never be compiled as a standalone function. If this macro is defined, GCC supports the @code{gnu_inline} function attribute as a way to always get the gnu89 behaviour. Support for this and @code{__GNUC_GNU_INLINE__} was added in GCC 4.1.3. If neither macro is defined, an older version of GCC is being used: @code{inline} functions will be compiled in gnu89 mode, and the @code{gnu_inline} function attribute will not be recognized. @item __CHAR_UNSIGNED__ GCC defines this macro if and only if the data type @code{char} is unsigned on the target machine. It exists to cause the standard header file @file{limits.h} to work correctly. You should not use this macro yourself; instead, refer to the standard macros defined in @file{limits.h}. @item __WCHAR_UNSIGNED__ Like @code{__CHAR_UNSIGNED__}, this macro is defined if and only if the data type @code{wchar_t} is unsigned and the front-end is in C++ mode. @item __REGISTER_PREFIX__ This macro expands to a single token (not a string constant) which is the prefix applied to CPU register names in assembly language for this target. You can use it to write assembly that is usable in multiple environments. For example, in the @code{m68k-aout} environment it expands to nothing, but in the @code{m68k-coff} environment it expands to a single @samp{%}. @item __USER_LABEL_PREFIX__ This macro expands to a single token which is the prefix applied to user labels (symbols visible to C code) in assembly. For example, in the @code{m68k-aout} environment it expands to an @samp{_}, but in the @code{m68k-coff} environment it expands to nothing. This macro will have the correct definition even if @option{-f(no-)underscores} is in use, but it will not be correct if target-specific options that adjust this prefix are used (e.g.@: the OSF/rose @option{-mno-underscores} option). @item __SIZE_TYPE__ @itemx __PTRDIFF_TYPE__ @itemx __WCHAR_TYPE__ @itemx __WINT_TYPE__ @itemx __INTMAX_TYPE__ @itemx __UINTMAX_TYPE__ These macros are defined to the correct underlying types for the @code{size_t}, @code{ptrdiff_t}, @code{wchar_t}, @code{wint_t}, @code{intmax_t}, and @code{uintmax_t} typedefs, respectively. They exist to make the standard header files @file{stddef.h} and @file{wchar.h} work correctly. You should not use these macros directly; instead, include the appropriate headers and use the typedefs. @item __CHAR_BIT__ Defined to the number of bits used in the representation of the @code{char} data type. It exists to make the standard header given numerical limits work correctly. You should not use this macro directly; instead, include the appropriate headers. @item __SCHAR_MAX__ @itemx __WCHAR_MAX__ @itemx __SHRT_MAX__ @itemx __INT_MAX__ @itemx __LONG_MAX__ @itemx __LONG_LONG_MAX__ @itemx __INTMAX_MAX__ Defined to the maximum value of the @code{signed char}, @code{wchar_t}, @code{signed short}, @code{signed int}, @code{signed long}, @code{signed long long}, and @code{intmax_t} types respectively. They exist to make the standard header given numerical limits work correctly. You should not use these macros directly; instead, include the appropriate headers. @item __DEPRECATED This macro is defined, with value 1, when compiling a C++ source file with warnings about deprecated constructs enabled. These warnings are enabled by default, but can be disabled with @option{-Wno-deprecated}. @item __EXCEPTIONS This macro is defined, with value 1, when compiling a C++ source file with exceptions enabled. If @option{-fno-exceptions} was used when compiling the file, then this macro will not be defined. @item __USING_SJLJ_EXCEPTIONS__ This macro is defined, with value 1, if the compiler uses the old mechanism based on @code{setjmp} and @code{longjmp} for exception handling. @item __GXX_WEAK__ This macro is defined when compiling a C++ source file. It has the value 1 if the compiler will use weak symbols, COMDAT sections, or other similar techniques to collapse symbols with ``vague linkage'' that are defined in multiple translation units. If the compiler will not collapse such symbols, this macro is defined with value 0. In general, user code should not need to make use of this macro; the purpose of this macro is to ease implementation of the C++ runtime library provided with G++. @item __LP64__ @itemx _LP64 These macros are defined, with value 1, if (and only if) the compilation is for a target where @code{long int} and pointer both use 64-bits and @code{int} uses 32-bit. @item __SSP__ This macro is defined, with value 1, when @option{-fstack-protector} is in use. @item __SSP_ALL__ This macro is defined, with value 2, when @option{-fstack-protector-all} is in use. @item __SSP_STRONG__ This macro is defined, with value 3, when @option{-fstack-protector-strong} is in use. @item __TIMESTAMP__ This macro expands to a string constant that describes the date and time of the last modification of the current source file. The string constant contains abbreviated day of the week, month, day of the month, time in hh:mm:ss form, year and looks like @code{@w{"Sun Sep 16 01:03:52 1973"}}. If the day of the month is less than 10, it is padded with a space on the left. If GCC cannot determine the current date, it will emit a warning message (once per compilation) and @code{__TIMESTAMP__} will expand to @code{@w{"??? ??? ?? ??:??:?? ????"}}. @end table @node System-specific Predefined Macros @subsection System-specific Predefined Macros @cindex system-specific predefined macros @cindex predefined macros, system-specific @cindex reserved namespace The C preprocessor normally predefines several macros that indicate what type of system and machine is in use. They are obviously different on each target supported by GCC@. This manual, being for all systems and machines, cannot tell you what their names are, but you can use @command{cpp -dM} to see them all. @xref{Invocation}. All system-specific predefined macros expand to the constant 1, so you can test them with either @samp{#ifdef} or @samp{#if}. The C standard requires that all system-specific macros be part of the @dfn{reserved namespace}. All names which begin with two underscores, or an underscore and a capital letter, are reserved for the compiler and library to use as they wish. However, historically system-specific macros have had names with no special prefix; for instance, it is common to find @code{unix} defined on Unix systems. For all such macros, GCC provides a parallel macro with two underscores added at the beginning and the end. If @code{unix} is defined, @code{__unix__} will be defined too. There will never be more than two underscores; the parallel of @code{_mips} is @code{__mips__}. When the @option{-ansi} option, or any @option{-std} option that requests strict conformance, is given to the compiler, all the system-specific predefined macros outside the reserved namespace are suppressed. The parallel macros, inside the reserved namespace, remain defined. We are slowly phasing out all predefined macros which are outside the reserved namespace. You should never use them in new programs, and we encourage you to correct older code to use the parallel macros whenever you find it. We don't recommend you use the system-specific macros that are in the reserved namespace, either. It is better in the long run to check specifically for features you need, using a tool such as @command{autoconf}. @node C++ Named Operators @subsection C++ Named Operators @cindex named operators @cindex C++ named operators @cindex iso646.h In C++, there are eleven keywords which are simply alternate spellings of operators normally written with punctuation. These keywords are treated as such even in the preprocessor. They function as operators in @samp{#if}, and they cannot be defined as macros or poisoned. In C, you can request that those keywords take their C++ meaning by including @file{iso646.h}. That header defines each one as a normal object-like macro expanding to the appropriate punctuator. These are the named operators and their corresponding punctuators: @multitable {Named Operator} {Punctuator} @item Named Operator @tab Punctuator @item @code{and} @tab @code{&&} @item @code{and_eq} @tab @code{&=} @item @code{bitand} @tab @code{&} @item @code{bitor} @tab @code{|} @item @code{compl} @tab @code{~} @item @code{not} @tab @code{!} @item @code{not_eq} @tab @code{!=} @item @code{or} @tab @code{||} @item @code{or_eq} @tab @code{|=} @item @code{xor} @tab @code{^} @item @code{xor_eq} @tab @code{^=} @end multitable @node Undefining and Redefining Macros @section Undefining and Redefining Macros @cindex undefining macros @cindex redefining macros @findex #undef If a macro ceases to be useful, it may be @dfn{undefined} with the @samp{#undef} directive. @samp{#undef} takes a single argument, the name of the macro to undefine. You use the bare macro name, even if the macro is function-like. It is an error if anything appears on the line after the macro name. @samp{#undef} has no effect if the name is not a macro. @smallexample #define FOO 4 x = FOO; @expansion{} x = 4; #undef FOO x = FOO; @expansion{} x = FOO; @end smallexample Once a macro has been undefined, that identifier may be @dfn{redefined} as a macro by a subsequent @samp{#define} directive. The new definition need not have any resemblance to the old definition. However, if an identifier which is currently a macro is redefined, then the new definition must be @dfn{effectively the same} as the old one. Two macro definitions are effectively the same if: @itemize @bullet @item Both are the same type of macro (object- or function-like). @item All the tokens of the replacement list are the same. @item If there are any parameters, they are the same. @item Whitespace appears in the same places in both. It need not be exactly the same amount of whitespace, though. Remember that comments count as whitespace. @end itemize @noindent These definitions are effectively the same: @smallexample #define FOUR (2 + 2) #define FOUR (2 + 2) #define FOUR (2 /* @r{two} */ + 2) @end smallexample @noindent but these are not: @smallexample #define FOUR (2 + 2) #define FOUR ( 2+2 ) #define FOUR (2 * 2) #define FOUR(score,and,seven,years,ago) (2 + 2) @end smallexample If a macro is redefined with a definition that is not effectively the same as the old one, the preprocessor issues a warning and changes the macro to use the new definition. If the new definition is effectively the same, the redefinition is silently ignored. This allows, for instance, two different headers to define a common macro. The preprocessor will only complain if the definitions do not match. @node Directives Within Macro Arguments @section Directives Within Macro Arguments @cindex macro arguments and directives Occasionally it is convenient to use preprocessor directives within the arguments of a macro. The C and C++ standards declare that behavior in these cases is undefined. Versions of CPP prior to 3.2 would reject such constructs with an error message. This was the only syntactic difference between normal functions and function-like macros, so it seemed attractive to remove this limitation, and people would often be surprised that they could not use macros in this way. Moreover, sometimes people would use conditional compilation in the argument list to a normal library function like @samp{printf}, only to find that after a library upgrade @samp{printf} had changed to be a function-like macro, and their code would no longer compile. So from version 3.2 we changed CPP to successfully process arbitrary directives within macro arguments in exactly the same way as it would have processed the directive were the function-like macro invocation not present. If, within a macro invocation, that macro is redefined, then the new definition takes effect in time for argument pre-expansion, but the original definition is still used for argument replacement. Here is a pathological example: @smallexample #define f(x) x x f (1 #undef f #define f 2 f) @end smallexample @noindent which expands to @smallexample 1 2 1 2 @end smallexample @noindent with the semantics described above. @node Macro Pitfalls @section Macro Pitfalls @cindex problems with macros @cindex pitfalls of macros In this section we describe some special rules that apply to macros and macro expansion, and point out certain cases in which the rules have counter-intuitive consequences that you must watch out for. @menu * Misnesting:: * Operator Precedence Problems:: * Swallowing the Semicolon:: * Duplication of Side Effects:: * Self-Referential Macros:: * Argument Prescan:: * Newlines in Arguments:: @end menu @node Misnesting @subsection Misnesting When a macro is called with arguments, the arguments are substituted into the macro body and the result is checked, together with the rest of the input file, for more macro calls. It is possible to piece together a macro call coming partially from the macro body and partially from the arguments. For example, @smallexample #define twice(x) (2*(x)) #define call_with_1(x) x(1) call_with_1 (twice) @expansion{} twice(1) @expansion{} (2*(1)) @end smallexample Macro definitions do not have to have balanced parentheses. By writing an unbalanced open parenthesis in a macro body, it is possible to create a macro call that begins inside the macro body but ends outside of it. For example, @smallexample #define strange(file) fprintf (file, "%s %d", @dots{} strange(stderr) p, 35) @expansion{} fprintf (stderr, "%s %d", p, 35) @end smallexample The ability to piece together a macro call can be useful, but the use of unbalanced open parentheses in a macro body is just confusing, and should be avoided. @node Operator Precedence Problems @subsection Operator Precedence Problems @cindex parentheses in macro bodies You may have noticed that in most of the macro definition examples shown above, each occurrence of a macro argument name had parentheses around it. In addition, another pair of parentheses usually surround the entire macro definition. Here is why it is best to write macros that way. Suppose you define a macro as follows, @smallexample #define ceil_div(x, y) (x + y - 1) / y @end smallexample @noindent whose purpose is to divide, rounding up. (One use for this operation is to compute how many @code{int} objects are needed to hold a certain number of @code{char} objects.) Then suppose it is used as follows: @smallexample a = ceil_div (b & c, sizeof (int)); @expansion{} a = (b & c + sizeof (int) - 1) / sizeof (int); @end smallexample @noindent This does not do what is intended. The operator-precedence rules of C make it equivalent to this: @smallexample a = (b & (c + sizeof (int) - 1)) / sizeof (int); @end smallexample @noindent What we want is this: @smallexample a = ((b & c) + sizeof (int) - 1)) / sizeof (int); @end smallexample @noindent Defining the macro as @smallexample #define ceil_div(x, y) ((x) + (y) - 1) / (y) @end smallexample @noindent provides the desired result. Unintended grouping can result in another way. Consider @code{sizeof ceil_div(1, 2)}. That has the appearance of a C expression that would compute the size of the type of @code{ceil_div (1, 2)}, but in fact it means something very different. Here is what it expands to: @smallexample sizeof ((1) + (2) - 1) / (2) @end smallexample @noindent This would take the size of an integer and divide it by two. The precedence rules have put the division outside the @code{sizeof} when it was intended to be inside. Parentheses around the entire macro definition prevent such problems. Here, then, is the recommended way to define @code{ceil_div}: @smallexample #define ceil_div(x, y) (((x) + (y) - 1) / (y)) @end smallexample @node Swallowing the Semicolon @subsection Swallowing the Semicolon @cindex semicolons (after macro calls) Often it is desirable to define a macro that expands into a compound statement. Consider, for example, the following macro, that advances a pointer (the argument @code{p} says where to find it) across whitespace characters: @smallexample #define SKIP_SPACES(p, limit) \ @{ char *lim = (limit); \ while (p < lim) @{ \ if (*p++ != ' ') @{ \ p--; break; @}@}@} @end smallexample @noindent Here backslash-newline is used to split the macro definition, which must be a single logical line, so that it resembles the way such code would be laid out if not part of a macro definition. A call to this macro might be @code{SKIP_SPACES (p, lim)}. Strictly speaking, the call expands to a compound statement, which is a complete statement with no need for a semicolon to end it. However, since it looks like a function call, it minimizes confusion if you can use it like a function call, writing a semicolon afterward, as in @code{SKIP_SPACES (p, lim);} This can cause trouble before @code{else} statements, because the semicolon is actually a null statement. Suppose you write @smallexample if (*p != 0) SKIP_SPACES (p, lim); else @dots{} @end smallexample @noindent The presence of two statements---the compound statement and a null statement---in between the @code{if} condition and the @code{else} makes invalid C code. The definition of the macro @code{SKIP_SPACES} can be altered to solve this problem, using a @code{do @dots{} while} statement. Here is how: @smallexample #define SKIP_SPACES(p, limit) \ do @{ char *lim = (limit); \ while (p < lim) @{ \ if (*p++ != ' ') @{ \ p--; break; @}@}@} \ while (0) @end smallexample Now @code{SKIP_SPACES (p, lim);} expands into @smallexample do @{@dots{}@} while (0); @end smallexample @noindent which is one statement. The loop executes exactly once; most compilers generate no extra code for it. @node Duplication of Side Effects @subsection Duplication of Side Effects @cindex side effects (in macro arguments) @cindex unsafe macros Many C programs define a macro @code{min}, for ``minimum'', like this: @smallexample #define min(X, Y) ((X) < (Y) ? (X) : (Y)) @end smallexample When you use this macro with an argument containing a side effect, as shown here, @smallexample next = min (x + y, foo (z)); @end smallexample @noindent it expands as follows: @smallexample next = ((x + y) < (foo (z)) ? (x + y) : (foo (z))); @end smallexample @noindent where @code{x + y} has been substituted for @code{X} and @code{foo (z)} for @code{Y}. The function @code{foo} is used only once in the statement as it appears in the program, but the expression @code{foo (z)} has been substituted twice into the macro expansion. As a result, @code{foo} might be called two times when the statement is executed. If it has side effects or if it takes a long time to compute, the results might not be what you intended. We say that @code{min} is an @dfn{unsafe} macro. The best solution to this problem is to define @code{min} in a way that computes the value of @code{foo (z)} only once. The C language offers no standard way to do this, but it can be done with GNU extensions as follows: @smallexample #define min(X, Y) \ (@{ typeof (X) x_ = (X); \ typeof (Y) y_ = (Y); \ (x_ < y_) ? x_ : y_; @}) @end smallexample The @samp{(@{ @dots{} @})} notation produces a compound statement that acts as an expression. Its value is the value of its last statement. This permits us to define local variables and assign each argument to one. The local variables have underscores after their names to reduce the risk of conflict with an identifier of wider scope (it is impossible to avoid this entirely). Now each argument is evaluated exactly once. If you do not wish to use GNU C extensions, the only solution is to be careful when @emph{using} the macro @code{min}. For example, you can calculate the value of @code{foo (z)}, save it in a variable, and use that variable in @code{min}: @smallexample @group #define min(X, Y) ((X) < (Y) ? (X) : (Y)) @dots{} @{ int tem = foo (z); next = min (x + y, tem); @} @end group @end smallexample @noindent (where we assume that @code{foo} returns type @code{int}). @node Self-Referential Macros @subsection Self-Referential Macros @cindex self-reference A @dfn{self-referential} macro is one whose name appears in its definition. Recall that all macro definitions are rescanned for more macros to replace. If the self-reference were considered a use of the macro, it would produce an infinitely large expansion. To prevent this, the self-reference is not considered a macro call. It is passed into the preprocessor output unchanged. Consider an example: @smallexample #define foo (4 + foo) @end smallexample @noindent where @code{foo} is also a variable in your program. Following the ordinary rules, each reference to @code{foo} will expand into @code{(4 + foo)}; then this will be rescanned and will expand into @code{(4 + (4 + foo))}; and so on until the computer runs out of memory. The self-reference rule cuts this process short after one step, at @code{(4 + foo)}. Therefore, this macro definition has the possibly useful effect of causing the program to add 4 to the value of @code{foo} wherever @code{foo} is referred to. In most cases, it is a bad idea to take advantage of this feature. A person reading the program who sees that @code{foo} is a variable will not expect that it is a macro as well. The reader will come across the identifier @code{foo} in the program and think its value should be that of the variable @code{foo}, whereas in fact the value is four greater. One common, useful use of self-reference is to create a macro which expands to itself. If you write @smallexample #define EPERM EPERM @end smallexample @noindent then the macro @code{EPERM} expands to @code{EPERM}. Effectively, it is left alone by the preprocessor whenever it's used in running text. You can tell that it's a macro with @samp{#ifdef}. You might do this if you want to define numeric constants with an @code{enum}, but have @samp{#ifdef} be true for each constant. If a macro @code{x} expands to use a macro @code{y}, and the expansion of @code{y} refers to the macro @code{x}, that is an @dfn{indirect self-reference} of @code{x}. @code{x} is not expanded in this case either. Thus, if we have @smallexample #define x (4 + y) #define y (2 * x) @end smallexample @noindent then @code{x} and @code{y} expand as follows: @smallexample @group x @expansion{} (4 + y) @expansion{} (4 + (2 * x)) y @expansion{} (2 * x) @expansion{} (2 * (4 + y)) @end group @end smallexample @noindent Each macro is expanded when it appears in the definition of the other macro, but not when it indirectly appears in its own definition. @node Argument Prescan @subsection Argument Prescan @cindex expansion of arguments @cindex macro argument expansion @cindex prescan of macro arguments Macro arguments are completely macro-expanded before they are substituted into a macro body, unless they are stringified or pasted with other tokens. After substitution, the entire macro body, including the substituted arguments, is scanned again for macros to be expanded. The result is that the arguments are scanned @emph{twice} to expand macro calls in them. Most of the time, this has no effect. If the argument contained any macro calls, they are expanded during the first scan. The result therefore contains no macro calls, so the second scan does not change it. If the argument were substituted as given, with no prescan, the single remaining scan would find the same macro calls and produce the same results. You might expect the double scan to change the results when a self-referential macro is used in an argument of another macro (@pxref{Self-Referential Macros}): the self-referential macro would be expanded once in the first scan, and a second time in the second scan. However, this is not what happens. The self-references that do not expand in the first scan are marked so that they will not expand in the second scan either. You might wonder, ``Why mention the prescan, if it makes no difference? And why not skip it and make the preprocessor faster?'' The answer is that the prescan does make a difference in three special cases: @itemize @bullet @item Nested calls to a macro. We say that @dfn{nested} calls to a macro occur when a macro's argument contains a call to that very macro. For example, if @code{f} is a macro that expects one argument, @code{f (f (1))} is a nested pair of calls to @code{f}. The desired expansion is made by expanding @code{f (1)} and substituting that into the definition of @code{f}. The prescan causes the expected result to happen. Without the prescan, @code{f (1)} itself would be substituted as an argument, and the inner use of @code{f} would appear during the main scan as an indirect self-reference and would not be expanded. @item Macros that call other macros that stringify or concatenate. If an argument is stringified or concatenated, the prescan does not occur. If you @emph{want} to expand a macro, then stringify or concatenate its expansion, you can do that by causing one macro to call another macro that does the stringification or concatenation. For instance, if you have @smallexample #define AFTERX(x) X_ ## x #define XAFTERX(x) AFTERX(x) #define TABLESIZE 1024 #define BUFSIZE TABLESIZE @end smallexample then @code{AFTERX(BUFSIZE)} expands to @code{X_BUFSIZE}, and @code{XAFTERX(BUFSIZE)} expands to @code{X_1024}. (Not to @code{X_TABLESIZE}. Prescan always does a complete expansion.) @item Macros used in arguments, whose expansions contain unshielded commas. This can cause a macro expanded on the second scan to be called with the wrong number of arguments. Here is an example: @smallexample #define foo a,b #define bar(x) lose(x) #define lose(x) (1 + (x)) @end smallexample We would like @code{bar(foo)} to turn into @code{(1 + (foo))}, which would then turn into @code{(1 + (a,b))}. Instead, @code{bar(foo)} expands into @code{lose(a,b)}, and you get an error because @code{lose} requires a single argument. In this case, the problem is easily solved by the same parentheses that ought to be used to prevent misnesting of arithmetic operations: @smallexample #define foo (a,b) @exdent or #define bar(x) lose((x)) @end smallexample The extra pair of parentheses prevents the comma in @code{foo}'s definition from being interpreted as an argument separator. @end itemize @node Newlines in Arguments @subsection Newlines in Arguments @cindex newlines in macro arguments The invocation of a function-like macro can extend over many logical lines. However, in the present implementation, the entire expansion comes out on one line. Thus line numbers emitted by the compiler or debugger refer to the line the invocation started on, which might be different to the line containing the argument causing the problem. Here is an example illustrating this: @smallexample #define ignore_second_arg(a,b,c) a; c ignore_second_arg (foo (), ignored (), syntax error); @end smallexample @noindent The syntax error triggered by the tokens @code{syntax error} results in an error message citing line three---the line of ignore_second_arg--- even though the problematic code comes from line five. We consider this a bug, and intend to fix it in the near future. @node Conditionals @chapter Conditionals @cindex conditionals A @dfn{conditional} is a directive that instructs the preprocessor to select whether or not to include a chunk of code in the final token stream passed to the compiler. Preprocessor conditionals can test arithmetic expressions, or whether a name is defined as a macro, or both simultaneously using the special @code{defined} operator. A conditional in the C preprocessor resembles in some ways an @code{if} statement in C, but it is important to understand the difference between them. The condition in an @code{if} statement is tested during the execution of your program. Its purpose is to allow your program to behave differently from run to run, depending on the data it is operating on. The condition in a preprocessing conditional directive is tested when your program is compiled. Its purpose is to allow different code to be included in the program depending on the situation at the time of compilation. However, the distinction is becoming less clear. Modern compilers often do test @code{if} statements when a program is compiled, if their conditions are known not to vary at run time, and eliminate code which can never be executed. If you can count on your compiler to do this, you may find that your program is more readable if you use @code{if} statements with constant conditions (perhaps determined by macros). Of course, you can only use this to exclude code, not type definitions or other preprocessing directives, and you can only do it if the code remains syntactically valid when it is not to be used. GCC version 3 eliminates this kind of never-executed code even when not optimizing. Older versions did it only when optimizing. @menu * Conditional Uses:: * Conditional Syntax:: * Deleted Code:: @end menu @node Conditional Uses @section Conditional Uses There are three general reasons to use a conditional. @itemize @bullet @item A program may need to use different code depending on the machine or operating system it is to run on. In some cases the code for one operating system may be erroneous on another operating system; for example, it might refer to data types or constants that do not exist on the other system. When this happens, it is not enough to avoid executing the invalid code. Its mere presence will cause the compiler to reject the program. With a preprocessing conditional, the offending code can be effectively excised from the program when it is not valid. @item You may want to be able to compile the same source file into two different programs. One version might make frequent time-consuming consistency checks on its intermediate data, or print the values of those data for debugging, and the other not. @item A conditional whose condition is always false is one way to exclude code from the program but keep it as a sort of comment for future reference. @end itemize Simple programs that do not need system-specific logic or complex debugging hooks generally will not need to use preprocessing conditionals. @node Conditional Syntax @section Conditional Syntax @findex #if A conditional in the C preprocessor begins with a @dfn{conditional directive}: @samp{#if}, @samp{#ifdef} or @samp{#ifndef}. @menu * Ifdef:: * If:: * Defined:: * Else:: * Elif:: @end menu @node Ifdef @subsection Ifdef @findex #ifdef @findex #endif The simplest sort of conditional is @smallexample @group #ifdef @var{MACRO} @var{controlled text} #endif /* @var{MACRO} */ @end group @end smallexample @cindex conditional group This block is called a @dfn{conditional group}. @var{controlled text} will be included in the output of the preprocessor if and only if @var{MACRO} is defined. We say that the conditional @dfn{succeeds} if @var{MACRO} is defined, @dfn{fails} if it is not. The @var{controlled text} inside of a conditional can include preprocessing directives. They are executed only if the conditional succeeds. You can nest conditional groups inside other conditional groups, but they must be completely nested. In other words, @samp{#endif} always matches the nearest @samp{#ifdef} (or @samp{#ifndef}, or @samp{#if}). Also, you cannot start a conditional group in one file and end it in another. Even if a conditional fails, the @var{controlled text} inside it is still run through initial transformations and tokenization. Therefore, it must all be lexically valid C@. Normally the only way this matters is that all comments and string literals inside a failing conditional group must still be properly ended. The comment following the @samp{#endif} is not required, but it is a good practice if there is a lot of @var{controlled text}, because it helps people match the @samp{#endif} to the corresponding @samp{#ifdef}. Older programs sometimes put @var{MACRO} directly after the @samp{#endif} without enclosing it in a comment. This is invalid code according to the C standard. CPP accepts it with a warning. It never affects which @samp{#ifndef} the @samp{#endif} matches. @findex #ifndef Sometimes you wish to use some code if a macro is @emph{not} defined. You can do this by writing @samp{#ifndef} instead of @samp{#ifdef}. One common use of @samp{#ifndef} is to include code only the first time a header file is included. @xref{Once-Only Headers}. Macro definitions can vary between compilations for several reasons. Here are some samples. @itemize @bullet @item Some macros are predefined on each kind of machine (@pxref{System-specific Predefined Macros}). This allows you to provide code specially tuned for a particular machine. @item System header files define more macros, associated with the features they implement. You can test these macros with conditionals to avoid using a system feature on a machine where it is not implemented. @item Macros can be defined or undefined with the @option{-D} and @option{-U} command line options when you compile the program. You can arrange to compile the same source file into two different programs by choosing a macro name to specify which program you want, writing conditionals to test whether or how this macro is defined, and then controlling the state of the macro with command line options, perhaps set in the Makefile. @xref{Invocation}. @item Your program might have a special header file (often called @file{config.h}) that is adjusted when the program is compiled. It can define or not define macros depending on the features of the system and the desired capabilities of the program. The adjustment can be automated by a tool such as @command{autoconf}, or done by hand. @end itemize @node If @subsection If The @samp{#if} directive allows you to test the value of an arithmetic expression, rather than the mere existence of one macro. Its syntax is @smallexample @group #if @var{expression} @var{controlled text} #endif /* @var{expression} */ @end group @end smallexample @var{expression} is a C expression of integer type, subject to stringent restrictions. It may contain @itemize @bullet @item Integer constants. @item Character constants, which are interpreted as they would be in normal code. @item Arithmetic operators for addition, subtraction, multiplication, division, bitwise operations, shifts, comparisons, and logical operations (@code{&&} and @code{||}). The latter two obey the usual short-circuiting rules of standard C@. @item Macros. All macros in the expression are expanded before actual computation of the expression's value begins. @item Uses of the @code{defined} operator, which lets you check whether macros are defined in the middle of an @samp{#if}. @item Identifiers that are not macros, which are all considered to be the number zero. This allows you to write @code{@w{#if MACRO}} instead of @code{@w{#ifdef MACRO}}, if you know that MACRO, when defined, will always have a nonzero value. Function-like macros used without their function call parentheses are also treated as zero. In some contexts this shortcut is undesirable. The @option{-Wundef} option causes GCC to warn whenever it encounters an identifier which is not a macro in an @samp{#if}. @end itemize The preprocessor does not know anything about types in the language. Therefore, @code{sizeof} operators are not recognized in @samp{#if}, and neither are @code{enum} constants. They will be taken as identifiers which are not macros, and replaced by zero. In the case of @code{sizeof}, this is likely to cause the expression to be invalid. The preprocessor calculates the value of @var{expression}. It carries out all calculations in the widest integer type known to the compiler; on most machines supported by GCC this is 64 bits. This is not the same rule as the compiler uses to calculate the value of a constant expression, and may give different results in some cases. If the value comes out to be nonzero, the @samp{#if} succeeds and the @var{controlled text} is included; otherwise it is skipped. @node Defined @subsection Defined @cindex @code{defined} The special operator @code{defined} is used in @samp{#if} and @samp{#elif} expressions to test whether a certain name is defined as a macro. @code{defined @var{name}} and @code{defined (@var{name})} are both expressions whose value is 1 if @var{name} is defined as a macro at the current point in the program, and 0 otherwise. Thus, @code{@w{#if defined MACRO}} is precisely equivalent to @code{@w{#ifdef MACRO}}. @code{defined} is useful when you wish to test more than one macro for existence at once. For example, @smallexample #if defined (__vax__) || defined (__ns16000__) @end smallexample @noindent would succeed if either of the names @code{__vax__} or @code{__ns16000__} is defined as a macro. Conditionals written like this: @smallexample #if defined BUFSIZE && BUFSIZE >= 1024 @end smallexample @noindent can generally be simplified to just @code{@w{#if BUFSIZE >= 1024}}, since if @code{BUFSIZE} is not defined, it will be interpreted as having the value zero. If the @code{defined} operator appears as a result of a macro expansion, the C standard says the behavior is undefined. GNU cpp treats it as a genuine @code{defined} operator and evaluates it normally. It will warn wherever your code uses this feature if you use the command-line option @option{-pedantic}, since other compilers may handle it differently. @node Else @subsection Else @findex #else The @samp{#else} directive can be added to a conditional to provide alternative text to be used if the condition fails. This is what it looks like: @smallexample @group #if @var{expression} @var{text-if-true} #else /* Not @var{expression} */ @var{text-if-false} #endif /* Not @var{expression} */ @end group @end smallexample @noindent If @var{expression} is nonzero, the @var{text-if-true} is included and the @var{text-if-false} is skipped. If @var{expression} is zero, the opposite happens. You can use @samp{#else} with @samp{#ifdef} and @samp{#ifndef}, too. @node Elif @subsection Elif @findex #elif One common case of nested conditionals is used to check for more than two possible alternatives. For example, you might have @smallexample #if X == 1 @dots{} #else /* X != 1 */ #if X == 2 @dots{} #else /* X != 2 */ @dots{} #endif /* X != 2 */ #endif /* X != 1 */ @end smallexample Another conditional directive, @samp{#elif}, allows this to be abbreviated as follows: @smallexample #if X == 1 @dots{} #elif X == 2 @dots{} #else /* X != 2 and X != 1*/ @dots{} #endif /* X != 2 and X != 1*/ @end smallexample @samp{#elif} stands for ``else if''. Like @samp{#else}, it goes in the middle of a conditional group and subdivides it; it does not require a matching @samp{#endif} of its own. Like @samp{#if}, the @samp{#elif} directive includes an expression to be tested. The text following the @samp{#elif} is processed only if the original @samp{#if}-condition failed and the @samp{#elif} condition succeeds. More than one @samp{#elif} can go in the same conditional group. Then the text after each @samp{#elif} is processed only if the @samp{#elif} condition succeeds after the original @samp{#if} and all previous @samp{#elif} directives within it have failed. @samp{#else} is allowed after any number of @samp{#elif} directives, but @samp{#elif} may not follow @samp{#else}. @node Deleted Code @section Deleted Code @cindex commenting out code If you replace or delete a part of the program but want to keep the old code around for future reference, you often cannot simply comment it out. Block comments do not nest, so the first comment inside the old code will end the commenting-out. The probable result is a flood of syntax errors. One way to avoid this problem is to use an always-false conditional instead. For instance, put @code{#if 0} before the deleted code and @code{#endif} after it. This works even if the code being turned off contains conditionals, but they must be entire conditionals (balanced @samp{#if} and @samp{#endif}). Some people use @code{#ifdef notdef} instead. This is risky, because @code{notdef} might be accidentally defined as a macro, and then the conditional would succeed. @code{#if 0} can be counted on to fail. Do not use @code{#if 0} for comments which are not C code. Use a real comment, instead. The interior of @code{#if 0} must consist of complete tokens; in particular, single-quote characters must balance. Comments often contain unbalanced single-quote characters (known in English as apostrophes). These confuse @code{#if 0}. They don't confuse @samp{/*}. @node Diagnostics @chapter Diagnostics @cindex diagnostic @cindex reporting errors @cindex reporting warnings @findex #error The directive @samp{#error} causes the preprocessor to report a fatal error. The tokens forming the rest of the line following @samp{#error} are used as the error message. You would use @samp{#error} inside of a conditional that detects a combination of parameters which you know the program does not properly support. For example, if you know that the program will not run properly on a VAX, you might write @smallexample @group #ifdef __vax__ #error "Won't work on VAXen. See comments at get_last_object." #endif @end group @end smallexample If you have several configuration parameters that must be set up by the installation in a consistent way, you can use conditionals to detect an inconsistency and report it with @samp{#error}. For example, @smallexample #if !defined(UNALIGNED_INT_ASM_OP) && defined(DWARF2_DEBUGGING_INFO) #error "DWARF2_DEBUGGING_INFO requires UNALIGNED_INT_ASM_OP." #endif @end smallexample @findex #warning The directive @samp{#warning} is like @samp{#error}, but causes the preprocessor to issue a warning and continue preprocessing. The tokens following @samp{#warning} are used as the warning message. You might use @samp{#warning} in obsolete header files, with a message directing the user to the header file which should be used instead. Neither @samp{#error} nor @samp{#warning} macro-expands its argument. Internal whitespace sequences are each replaced with a single space. The line must consist of complete tokens. It is wisest to make the argument of these directives be a single string constant; this avoids problems with apostrophes and the like. @node Line Control @chapter Line Control @cindex line control The C preprocessor informs the C compiler of the location in your source code where each token came from. Presently, this is just the file name and line number. All the tokens resulting from macro expansion are reported as having appeared on the line of the source file where the outermost macro was used. We intend to be more accurate in the future. If you write a program which generates source code, such as the @command{bison} parser generator, you may want to adjust the preprocessor's notion of the current file name and line number by hand. Parts of the output from @command{bison} are generated from scratch, other parts come from a standard parser file. The rest are copied verbatim from @command{bison}'s input. You would like compiler error messages and symbolic debuggers to be able to refer to @code{bison}'s input file. @findex #line @command{bison} or any such program can arrange this by writing @samp{#line} directives into the output file. @samp{#line} is a directive that specifies the original line number and source file name for subsequent input in the current preprocessor input file. @samp{#line} has three variants: @table @code @item #line @var{linenum} @var{linenum} is a non-negative decimal integer constant. It specifies the line number which should be reported for the following line of input. Subsequent lines are counted from @var{linenum}. @item #line @var{linenum} @var{filename} @var{linenum} is the same as for the first form, and has the same effect. In addition, @var{filename} is a string constant. The following line and all subsequent lines are reported to come from the file it specifies, until something else happens to change that. @var{filename} is interpreted according to the normal rules for a string constant: backslash escapes are interpreted. This is different from @samp{#include}. Previous versions of CPP did not interpret escapes in @samp{#line}; we have changed it because the standard requires they be interpreted, and most other compilers do. @item #line @var{anything else} @var{anything else} is checked for macro calls, which are expanded. The result should match one of the above two forms. @end table @samp{#line} directives alter the results of the @code{__FILE__} and @code{__LINE__} predefined macros from that point on. @xref{Standard Predefined Macros}. They do not have any effect on @samp{#include}'s idea of the directory containing the current file. This is a change from GCC 2.95. Previously, a file reading @smallexample #line 1 "../src/gram.y" #include "gram.h" @end smallexample would search for @file{gram.h} in @file{../src}, then the @option{-I} chain; the directory containing the physical source file would not be searched. In GCC 3.0 and later, the @samp{#include} is not affected by the presence of a @samp{#line} referring to a different directory. We made this change because the old behavior caused problems when generated source files were transported between machines. For instance, it is common practice to ship generated parsers with a source release, so that people building the distribution do not need to have yacc or Bison installed. These files frequently have @samp{#line} directives referring to the directory tree of the system where the distribution was created. If GCC tries to search for headers in those directories, the build is likely to fail. The new behavior can cause failures too, if the generated file is not in the same directory as its source and it attempts to include a header which would be visible searching from the directory containing the source file. However, this problem is easily solved with an additional @option{-I} switch on the command line. The failures caused by the old semantics could sometimes be corrected only by editing the generated files, which is difficult and error-prone. @node Pragmas @chapter Pragmas The @samp{#pragma} directive is the method specified by the C standard for providing additional information to the compiler, beyond what is conveyed in the language itself. Three forms of this directive (commonly known as @dfn{pragmas}) are specified by the 1999 C standard. A C compiler is free to attach any meaning it likes to other pragmas. GCC has historically preferred to use extensions to the syntax of the language, such as @code{__attribute__}, for this purpose. However, GCC does define a few pragmas of its own. These mostly have effects on the entire translation unit or source file. In GCC version 3, all GNU-defined, supported pragmas have been given a @code{GCC} prefix. This is in line with the @code{STDC} prefix on all pragmas defined by C99. For backward compatibility, pragmas which were recognized by previous versions are still recognized without the @code{GCC} prefix, but that usage is deprecated. Some older pragmas are deprecated in their entirety. They are not recognized with the @code{GCC} prefix. @xref{Obsolete Features}. @cindex @code{_Pragma} C99 introduces the @code{@w{_Pragma}} operator. This feature addresses a major problem with @samp{#pragma}: being a directive, it cannot be produced as the result of macro expansion. @code{@w{_Pragma}} is an operator, much like @code{sizeof} or @code{defined}, and can be embedded in a macro. Its syntax is @code{@w{_Pragma (@var{string-literal})}}, where @var{string-literal} can be either a normal or wide-character string literal. It is destringized, by replacing all @samp{\\} with a single @samp{\} and all @samp{\"} with a @samp{"}. The result is then processed as if it had appeared as the right hand side of a @samp{#pragma} directive. For example, @smallexample _Pragma ("GCC dependency \"parse.y\"") @end smallexample @noindent has the same effect as @code{#pragma GCC dependency "parse.y"}. The same effect could be achieved using macros, for example @smallexample #define DO_PRAGMA(x) _Pragma (#x) DO_PRAGMA (GCC dependency "parse.y") @end smallexample The standard is unclear on where a @code{_Pragma} operator can appear. The preprocessor does not accept it within a preprocessing conditional directive like @samp{#if}. To be safe, you are probably best keeping it out of directives other than @samp{#define}, and putting it on a line of its own. This manual documents the pragmas which are meaningful to the preprocessor itself. Other pragmas are meaningful to the C or C++ compilers. They are documented in the GCC manual. @ftable @code @item #pragma GCC dependency @code{#pragma GCC dependency} allows you to check the relative dates of the current file and another file. If the other file is more recent than the current file, a warning is issued. This is useful if the current file is derived from the other file, and should be regenerated. The other file is searched for using the normal include search path. Optional trailing text can be used to give more information in the warning message. @smallexample #pragma GCC dependency "parse.y" #pragma GCC dependency "/usr/include/time.h" rerun fixincludes @end smallexample @item #pragma GCC poison Sometimes, there is an identifier that you want to remove completely from your program, and make sure that it never creeps back in. To enforce this, you can @dfn{poison} the identifier with this pragma. @code{#pragma GCC poison} is followed by a list of identifiers to poison. If any of those identifiers appears anywhere in the source after the directive, it is a hard error. For example, @smallexample #pragma GCC poison printf sprintf fprintf sprintf(some_string, "hello"); @end smallexample @noindent will produce an error. If a poisoned identifier appears as part of the expansion of a macro which was defined before the identifier was poisoned, it will @emph{not} cause an error. This lets you poison an identifier without worrying about system headers defining macros that use it. For example, @smallexample #define strrchr rindex #pragma GCC poison rindex strrchr(some_string, 'h'); @end smallexample @noindent will not produce an error. @item #pragma GCC system_header This pragma takes no arguments. It causes the rest of the code in the current file to be treated as if it came from a system header. @xref{System Headers}. @end ftable @node Other Directives @chapter Other Directives @findex #ident @findex #sccs The @samp{#ident} directive takes one argument, a string constant. On some systems, that string constant is copied into a special segment of the object file. On other systems, the directive is ignored. The @samp{#sccs} directive is a synonym for @samp{#ident}. These directives are not part of the C standard, but they are not official GNU extensions either. What historical information we have been able to find, suggests they originated with System V@. @cindex null directive The @dfn{null directive} consists of a @samp{#} followed by a newline, with only whitespace (including comments) in between. A null directive is understood as a preprocessing directive but has no effect on the preprocessor output. The primary significance of the existence of the null directive is that an input line consisting of just a @samp{#} will produce no output, rather than a line of output containing just a @samp{#}. Supposedly some old C programs contain such lines. @node Preprocessor Output @chapter Preprocessor Output When the C preprocessor is used with the C or C++ compilers, it is integrated into the compiler and communicates a stream of binary tokens directly to the compiler's parser. However, it can also be used in the more conventional standalone mode, where it produces textual output. @c FIXME: Document the library interface. @cindex output format The output from the C preprocessor looks much like the input, except that all preprocessing directive lines have been replaced with blank lines and all comments with spaces. Long runs of blank lines are discarded. The ISO standard specifies that it is implementation defined whether a preprocessor preserves whitespace between tokens, or replaces it with e.g.@: a single space. In GNU CPP, whitespace between tokens is collapsed to become a single space, with the exception that the first token on a non-directive line is preceded with sufficient spaces that it appears in the same column in the preprocessed output that it appeared in the original source file. This is so the output is easy to read. @xref{Differences from previous versions}. CPP does not insert any whitespace where there was none in the original source, except where necessary to prevent an accidental token paste. @cindex linemarkers Source file name and line number information is conveyed by lines of the form @smallexample # @var{linenum} @var{filename} @var{flags} @end smallexample @noindent These are called @dfn{linemarkers}. They are inserted as needed into the output (but never within a string or character constant). They mean that the following line originated in file @var{filename} at line @var{linenum}. @var{filename} will never contain any non-printing characters; they are replaced with octal escape sequences. After the file name comes zero or more flags, which are @samp{1}, @samp{2}, @samp{3}, or @samp{4}. If there are multiple flags, spaces separate them. Here is what the flags mean: @table @samp @item 1 This indicates the start of a new file. @item 2 This indicates returning to a file (after having included another file). @item 3 This indicates that the following text comes from a system header file, so certain warnings should be suppressed. @item 4 This indicates that the following text should be treated as being wrapped in an implicit @code{extern "C"} block. @c maybe cross reference NO_IMPLICIT_EXTERN_C @end table As an extension, the preprocessor accepts linemarkers in non-assembler input files. They are treated like the corresponding @samp{#line} directive, (@pxref{Line Control}), except that trailing flags are permitted, and are interpreted with the meanings described above. If multiple flags are given, they must be in ascending order. Some directives may be duplicated in the output of the preprocessor. These are @samp{#ident} (always), @samp{#pragma} (only if the preprocessor does not handle the pragma itself), and @samp{#define} and @samp{#undef} (with certain debugging options). If this happens, the @samp{#} of the directive will always be in the first column, and there will be no space between the @samp{#} and the directive name. If macro expansion happens to generate tokens which might be mistaken for a duplicated directive, a space will be inserted between the @samp{#} and the directive name. @node Traditional Mode @chapter Traditional Mode Traditional (pre-standard) C preprocessing is rather different from the preprocessing specified by the standard. When GCC is given the @option{-traditional-cpp} option, it attempts to emulate a traditional preprocessor. GCC versions 3.2 and later only support traditional mode semantics in the preprocessor, and not in the compiler front ends. This chapter outlines the traditional preprocessor semantics we implemented. The implementation does not correspond precisely to the behavior of earlier versions of GCC, nor to any true traditional preprocessor. After all, inconsistencies among traditional implementations were a major motivation for C standardization. However, we intend that it should be compatible with true traditional preprocessors in all ways that actually matter. @menu * Traditional lexical analysis:: * Traditional macros:: * Traditional miscellany:: * Traditional warnings:: @end menu @node Traditional lexical analysis @section Traditional lexical analysis The traditional preprocessor does not decompose its input into tokens the same way a standards-conforming preprocessor does. The input is simply treated as a stream of text with minimal internal form. This implementation does not treat trigraphs (@pxref{trigraphs}) specially since they were an invention of the standards committee. It handles arbitrarily-positioned escaped newlines properly and splices the lines as you would expect; many traditional preprocessors did not do this. The form of horizontal whitespace in the input file is preserved in the output. In particular, hard tabs remain hard tabs. This can be useful if, for example, you are preprocessing a Makefile. Traditional CPP only recognizes C-style block comments, and treats the @samp{/*} sequence as introducing a comment only if it lies outside quoted text. Quoted text is introduced by the usual single and double quotes, and also by an initial @samp{<} in a @code{#include} directive. Traditionally, comments are completely removed and are not replaced with a space. Since a traditional compiler does its own tokenization of the output of the preprocessor, this means that comments can effectively be used as token paste operators. However, comments behave like separators for text handled by the preprocessor itself, since it doesn't re-lex its input. For example, in @smallexample #if foo/**/bar @end smallexample @noindent @samp{foo} and @samp{bar} are distinct identifiers and expanded separately if they happen to be macros. In other words, this directive is equivalent to @smallexample #if foo bar @end smallexample @noindent rather than @smallexample #if foobar @end smallexample Generally speaking, in traditional mode an opening quote need not have a matching closing quote. In particular, a macro may be defined with replacement text that contains an unmatched quote. Of course, if you attempt to compile preprocessed output containing an unmatched quote you will get a syntax error. However, all preprocessing directives other than @code{#define} require matching quotes. For example: @smallexample #define m This macro's fine and has an unmatched quote "/* This is not a comment. */ /* @r{This is a comment. The following #include directive is ill-formed.} */ #include (r250566) PR preprocessor/23479 * expr.c (cpp_classify_number): Implement 0b-prefixed binary integer constants. (append_digit): Likewise. * include/cpplib.h: Add CPP_N_BINARY, to be used for 0b-prefixed binary integer constants. 2007-05-31 Dave Korn (r125212) PR preprocessor/14331 * lex.c (_cpp_get_fresh_line): Don't warn if no newline at EOF. +2007-05-24 Ollie Wild (r125041) + + * macro.c (_cpp_builtin_macro_text): Handle BT_COUNTER. + * pch.c (cpp_write_pch_deps): Save __COUNTER__ state. + (cpp_write_pch_state): Save __COUNTER__ state. + (cpp_valid_state): Check valid __COUNTER__ state. + (cpp_read_state): Read new __COUNTER__ state. + * include/cpplib.h (enum builtin_type): Add BT_COUNTER enumerator. + * init.c (builtin_array): Add __COUNTER__/BT_COUNTER. + * internal.h (struct cpp_reader): Add counter member. + 2007-05-21 Ian Lance Taylor (r124929) * internal.h (struct cpp_reader): Add new fields: nonexistent_file_hash and nonexistent_file_ob. * files.c: Include "obstack.h". (find_file_in_dir): Before trying to open the file, look up the path name in the hash table of nonexistent files. After failing to open the file, add the path name to the hash table. (_cpp_find_file): Cache the results of looking up the file name starting with the quote and bracket chain heads, if we can. (nonexistent_file_hash_eq): New static function. (_cpp_init_files): Initialize pfile->nonexistent_file_hash and pfile->nonexistent_file_ob. (_cpp_cleanup_files): Free pfile->nonexistent_file_hash and pfile->nonexistent_file_ob. 2007-05-14 Janis Johnson (r124731) PR c/31924 * expr.c (interpret_float_suffix): Check for invalid suffix. 2007-05-02 Eric Christopher (r124358) * expr.c (num_div_op): Don't overflow if the result is zero. 2007-05-02 Tom Tromey (r124356) PR preprocessor/28709: * macro.c (paste_tokens): Remove PASTE_LEFT from the old lhs. 2007-01-30 Tom Tromey (r121340) PR preprocessor/29966: * macro.c (lex_expansion_token): Save and restore cpp_reader's cur_token. (_cpp_create_definition): Don't restore cur_token here. * lex.c (_cpp_lex_token): Added assertion. 2006-12-29 Ian Lance Taylor (r120263) * lex.c (_cpp_clean_line): Add uses of __builtin_expect. Don't look backward at the end of the line unless we saw a backslash. Index: projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/internal.h =================================================================== --- projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/internal.h (revision 305171) +++ projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/internal.h (revision 305172) @@ -1,692 +1,693 @@ /* Part of CPP library. Copyright (C) 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */ /* This header defines all the internal data structures and functions that need to be visible across files. It should not be used outside cpplib. */ #ifndef LIBCPP_INTERNAL_H #define LIBCPP_INTERNAL_H #include "symtab.h" #include "cpp-id-data.h" #ifndef HAVE_ICONV_H #undef HAVE_ICONV #endif #if HAVE_ICONV #include #else #define HAVE_ICONV 0 typedef int iconv_t; /* dummy */ #endif struct directive; /* Deliberately incomplete. */ struct pending_option; struct op; struct _cpp_strbuf; typedef bool (*convert_f) (iconv_t, const unsigned char *, size_t, struct _cpp_strbuf *); struct cset_converter { convert_f func; iconv_t cd; }; #define BITS_PER_CPPCHAR_T (CHAR_BIT * sizeof (cppchar_t)) /* Test if a sign is valid within a preprocessing number. */ #define VALID_SIGN(c, prevc) \ (((c) == '+' || (c) == '-') && \ ((prevc) == 'e' || (prevc) == 'E' \ || (((prevc) == 'p' || (prevc) == 'P') \ && CPP_OPTION (pfile, extended_numbers)))) #define CPP_OPTION(PFILE, OPTION) ((PFILE)->opts.OPTION) #define CPP_BUFFER(PFILE) ((PFILE)->buffer) #define CPP_BUF_COLUMN(BUF, CUR) ((CUR) - (BUF)->line_base) #define CPP_BUF_COL(BUF) CPP_BUF_COLUMN(BUF, (BUF)->cur) #define CPP_INCREMENT_LINE(PFILE, COLS_HINT) do { \ const struct line_maps *line_table = PFILE->line_table; \ const struct line_map *map = &line_table->maps[line_table->used-1]; \ unsigned int line = SOURCE_LINE (map, line_table->highest_line); \ linemap_line_start (PFILE->line_table, line + 1, COLS_HINT); \ } while (0) /* Maximum nesting of cpp_buffers. We use a static limit, partly for efficiency, and partly to limit runaway recursion. */ #define CPP_STACK_MAX 200 /* Host alignment handling. */ struct dummy { char c; union { double d; int *p; } u; }; #define DEFAULT_ALIGNMENT offsetof (struct dummy, u) #define CPP_ALIGN2(size, align) (((size) + ((align) - 1)) & ~((align) - 1)) #define CPP_ALIGN(size) CPP_ALIGN2 (size, DEFAULT_ALIGNMENT) #define _cpp_mark_macro_used(NODE) do { \ if ((NODE)->type == NT_MACRO && !((NODE)->flags & NODE_BUILTIN)) \ (NODE)->value.macro->used = 1; } while (0) /* A generic memory buffer, and operations on it. */ typedef struct _cpp_buff _cpp_buff; struct _cpp_buff { struct _cpp_buff *next; unsigned char *base, *cur, *limit; }; extern _cpp_buff *_cpp_get_buff (cpp_reader *, size_t); extern void _cpp_release_buff (cpp_reader *, _cpp_buff *); extern void _cpp_extend_buff (cpp_reader *, _cpp_buff **, size_t); extern _cpp_buff *_cpp_append_extend_buff (cpp_reader *, _cpp_buff *, size_t); extern void _cpp_free_buff (_cpp_buff *); extern unsigned char *_cpp_aligned_alloc (cpp_reader *, size_t); extern unsigned char *_cpp_unaligned_alloc (cpp_reader *, size_t); #define BUFF_ROOM(BUFF) (size_t) ((BUFF)->limit - (BUFF)->cur) #define BUFF_FRONT(BUFF) ((BUFF)->cur) #define BUFF_LIMIT(BUFF) ((BUFF)->limit) /* #include types. */ enum include_type {IT_INCLUDE, IT_INCLUDE_NEXT, IT_IMPORT, IT_CMDLINE}; union utoken { const cpp_token *token; const cpp_token **ptoken; }; /* A "run" of tokens; part of a chain of runs. */ typedef struct tokenrun tokenrun; struct tokenrun { tokenrun *next, *prev; cpp_token *base, *limit; }; /* Accessor macros for struct cpp_context. */ #define FIRST(c) ((c)->u.iso.first) #define LAST(c) ((c)->u.iso.last) #define CUR(c) ((c)->u.trad.cur) #define RLIMIT(c) ((c)->u.trad.rlimit) typedef struct cpp_context cpp_context; struct cpp_context { /* Doubly-linked list. */ cpp_context *next, *prev; union { /* For ISO macro expansion. Contexts other than the base context are contiguous tokens. e.g. macro expansions, expanded argument tokens. */ struct { union utoken first; union utoken last; } iso; /* For traditional macro expansion. */ struct { const unsigned char *cur; const unsigned char *rlimit; } trad; } u; /* If non-NULL, a buffer used for storage related to this context. When the context is popped, the buffer is released. */ _cpp_buff *buff; /* For a macro context, the macro node, otherwise NULL. */ cpp_hashnode *macro; /* True if utoken element is token, else ptoken. */ bool direct_p; }; struct lexer_state { /* Nonzero if first token on line is CPP_HASH. */ unsigned char in_directive; /* Nonzero if in a directive that will handle padding tokens itself. #include needs this to avoid problems with computed include and spacing between tokens. */ unsigned char directive_wants_padding; /* True if we are skipping a failed conditional group. */ unsigned char skipping; /* Nonzero if in a directive that takes angle-bracketed headers. */ unsigned char angled_headers; /* Nonzero if in a #if or #elif directive. */ unsigned char in_expression; /* Nonzero to save comments. Turned off if discard_comments, and in all directives apart from #define. */ unsigned char save_comments; /* Nonzero if lexing __VA_ARGS__ is valid. */ unsigned char va_args_ok; /* Nonzero if lexing poisoned identifiers is valid. */ unsigned char poisoned_ok; /* Nonzero to prevent macro expansion. */ unsigned char prevent_expansion; /* Nonzero when parsing arguments to a function-like macro. */ unsigned char parsing_args; /* Nonzero if prevent_expansion is true only because output is being discarded. */ unsigned char discarding_output; /* Nonzero to skip evaluating part of an expression. */ unsigned int skip_eval; /* Nonzero when handling a deferred pragma. */ unsigned char in_deferred_pragma; /* Nonzero if the deferred pragma being handled allows macro expansion. */ unsigned char pragma_allow_expansion; /* APPLE LOCAL begin #error with unmatched quotes 5607574 */ /* Nonzero when handling #error and #warning to allow unmatched quotes. */ unsigned char in_diagnostic; /* APPLE LOCAL end #error with unmatched quotes 5607574 */ }; /* Special nodes - identifiers with predefined significance. */ struct spec_nodes { cpp_hashnode *n_defined; /* defined operator */ cpp_hashnode *n_true; /* C++ keyword true */ cpp_hashnode *n_false; /* C++ keyword false */ cpp_hashnode *n__VA_ARGS__; /* C99 vararg macros */ }; typedef struct _cpp_line_note _cpp_line_note; struct _cpp_line_note { /* Location in the clean line the note refers to. */ const unsigned char *pos; /* Type of note. The 9 'from' trigraph characters represent those trigraphs, '\\' an escaped newline, ' ' an escaped newline with intervening space, and anything else is invalid. */ unsigned int type; }; /* Represents the contents of a file cpplib has read in. */ struct cpp_buffer { const unsigned char *cur; /* Current location. */ const unsigned char *line_base; /* Start of current physical line. */ const unsigned char *next_line; /* Start of to-be-cleaned logical line. */ const unsigned char *buf; /* Entire character buffer. */ const unsigned char *rlimit; /* Writable byte at end of file. */ _cpp_line_note *notes; /* Array of notes. */ unsigned int cur_note; /* Next note to process. */ unsigned int notes_used; /* Number of notes. */ unsigned int notes_cap; /* Size of allocated array. */ struct cpp_buffer *prev; /* Pointer into the file table; non-NULL if this is a file buffer. Used for include_next and to record control macros. */ struct _cpp_file *file; /* Saved value of __TIMESTAMP__ macro - date and time of last modification of the assotiated file. */ const unsigned char *timestamp; /* Value of if_stack at start of this file. Used to prohibit unmatched #endif (etc) in an include file. */ struct if_stack *if_stack; /* True if we need to get the next clean line. */ bool need_line; /* True if we have already warned about C++ comments in this file. The warning happens only for C89 extended mode with -pedantic on, or for -Wtraditional, and only once per file (otherwise it would be far too noisy). */ unsigned int warned_cplusplus_comments : 1; /* True if we don't process trigraphs and escaped newlines. True for preprocessed input, command line directives, and _Pragma buffers. */ unsigned int from_stage3 : 1; /* At EOF, a buffer is automatically popped. If RETURN_AT_EOF is true, a CPP_EOF token is then returned. Otherwise, the next token from the enclosing buffer is returned. */ unsigned int return_at_eof : 1; /* One for a system header, two for a C system header file that therefore needs to be extern "C" protected in C++, and zero otherwise. */ unsigned char sysp; /* The directory of the this buffer's file. Its NAME member is not allocated, so we don't need to worry about freeing it. */ struct cpp_dir dir; /* Descriptor for converting from the input character set to the source character set. */ struct cset_converter input_cset_desc; }; /* A cpp_reader encapsulates the "state" of a pre-processor run. Applying cpp_get_token repeatedly yields a stream of pre-processor tokens. Usually, there is only one cpp_reader object active. */ struct cpp_reader { /* Top of buffer stack. */ cpp_buffer *buffer; /* Overlaid buffer (can be different after processing #include). */ cpp_buffer *overlaid_buffer; /* Lexer state. */ struct lexer_state state; /* Source line tracking. */ struct line_maps *line_table; /* The line of the '#' of the current directive. */ source_location directive_line; /* Memory buffers. */ _cpp_buff *a_buff; /* Aligned permanent storage. */ _cpp_buff *u_buff; /* Unaligned permanent storage. */ _cpp_buff *free_buffs; /* Free buffer chain. */ /* Context stack. */ struct cpp_context base_context; struct cpp_context *context; /* If in_directive, the directive if known. */ const struct directive *directive; /* Token generated while handling a directive, if any. */ cpp_token directive_result; /* Search paths for include files. */ struct cpp_dir *quote_include; /* "" */ struct cpp_dir *bracket_include; /* <> */ struct cpp_dir no_search_path; /* No path. */ /* Chain of all hashed _cpp_file instances. */ struct _cpp_file *all_files; struct _cpp_file *main_file; /* File and directory hash table. */ struct htab *file_hash; struct htab *dir_hash; struct file_hash_entry *file_hash_entries; unsigned int file_hash_entries_allocated, file_hash_entries_used; /* Negative path lookup hash table. */ struct htab *nonexistent_file_hash; struct obstack nonexistent_file_ob; /* Nonzero means don't look for #include "foo" the source-file directory. */ bool quote_ignores_source_dir; /* Nonzero if any file has contained #pragma once or #import has been used. */ bool seen_once_only; /* Multiple include optimization. */ const cpp_hashnode *mi_cmacro; const cpp_hashnode *mi_ind_cmacro; bool mi_valid; /* Lexing. */ cpp_token *cur_token; tokenrun base_run, *cur_run; unsigned int lookaheads; /* Nonzero prevents the lexer from re-using the token runs. */ unsigned int keep_tokens; /* Error counter for exit code. */ unsigned int errors; /* Buffer to hold macro definition string. */ unsigned char *macro_buffer; unsigned int macro_buffer_len; /* Descriptor for converting from the source character set to the execution character set. */ struct cset_converter narrow_cset_desc; /* Descriptor for converting from the source character set to the wide execution character set. */ struct cset_converter wide_cset_desc; /* Date and time text. Calculated together if either is requested. */ const unsigned char *date; const unsigned char *time; /* EOF token, and a token forcing paste avoidance. */ cpp_token avoid_paste; cpp_token eof; /* Opaque handle to the dependencies of mkdeps.c. */ struct deps *deps; /* Obstack holding all macro hash nodes. This never shrinks. See identifiers.c */ struct obstack hash_ob; /* Obstack holding buffer and conditional structures. This is a real stack. See directives.c. */ struct obstack buffer_ob; /* Pragma table - dynamic, because a library user can add to the list of recognized pragmas. */ struct pragma_entry *pragmas; /* Call backs to cpplib client. */ struct cpp_callbacks cb; /* Identifier hash table. */ struct ht *hash_table; /* Expression parser stack. */ struct op *op_stack, *op_limit; /* User visible options. */ struct cpp_options opts; /* Special nodes - identifiers with predefined significance to the preprocessor. */ struct spec_nodes spec_nodes; /* Whether cpplib owns the hashtable. */ bool our_hashtable; /* Traditional preprocessing output buffer (a logical line). */ struct { unsigned char *base; unsigned char *limit; unsigned char *cur; source_location first_line; } out; /* Used for buffer overlays by traditional.c. */ const unsigned char *saved_cur, *saved_rlimit, *saved_line_base; /* A saved list of the defined macros, for dependency checking of precompiled headers. */ struct cpp_savedstate *savedstate; - unsigned int nextcounter; + /* Next value of __COUNTER__ macro. */ + unsigned int counter; }; /* Character classes. Based on the more primitive macros in safe-ctype.h. If the definition of `numchar' looks odd to you, please look up the definition of a pp-number in the C standard [section 6.4.8 of C99]. In the unlikely event that characters other than \r and \n enter the set is_vspace, the macro handle_newline() in lex.c must be updated. */ #define _dollar_ok(x) ((x) == '$' && CPP_OPTION (pfile, dollars_in_ident)) #define is_idchar(x) (ISIDNUM(x) || _dollar_ok(x)) #define is_numchar(x) ISIDNUM(x) #define is_idstart(x) (ISIDST(x) || _dollar_ok(x)) #define is_numstart(x) ISDIGIT(x) #define is_hspace(x) ISBLANK(x) #define is_vspace(x) IS_VSPACE(x) #define is_nvspace(x) IS_NVSPACE(x) #define is_space(x) IS_SPACE_OR_NUL(x) /* This table is constant if it can be initialized at compile time, which is the case if cpp was compiled with GCC >=2.7, or another compiler that supports C99. */ #if HAVE_DESIGNATED_INITIALIZERS extern const unsigned char _cpp_trigraph_map[UCHAR_MAX + 1]; #else extern unsigned char _cpp_trigraph_map[UCHAR_MAX + 1]; #endif /* Macros. */ static inline int cpp_in_system_header (cpp_reader *); static inline int cpp_in_system_header (cpp_reader *pfile) { return pfile->buffer ? pfile->buffer->sysp : 0; } #define CPP_PEDANTIC(PF) CPP_OPTION (PF, pedantic) #define CPP_WTRADITIONAL(PF) CPP_OPTION (PF, warn_traditional) /* In errors.c */ extern int _cpp_begin_message (cpp_reader *, int, source_location, unsigned int); /* In macro.c */ extern void _cpp_free_definition (cpp_hashnode *); extern bool _cpp_create_definition (cpp_reader *, cpp_hashnode *); extern void _cpp_pop_context (cpp_reader *); extern void _cpp_push_text_context (cpp_reader *, cpp_hashnode *, const unsigned char *, size_t); extern bool _cpp_save_parameter (cpp_reader *, cpp_macro *, cpp_hashnode *); extern bool _cpp_arguments_ok (cpp_reader *, cpp_macro *, const cpp_hashnode *, unsigned int); extern const unsigned char *_cpp_builtin_macro_text (cpp_reader *, cpp_hashnode *); extern int _cpp_warn_if_unused_macro (cpp_reader *, cpp_hashnode *, void *); extern void _cpp_push_token_context (cpp_reader *, cpp_hashnode *, const cpp_token *, unsigned int); /* In identifiers.c */ extern void _cpp_init_hashtable (cpp_reader *, hash_table *); extern void _cpp_destroy_hashtable (cpp_reader *); /* In files.c */ typedef struct _cpp_file _cpp_file; extern _cpp_file *_cpp_find_file (cpp_reader *, const char *, cpp_dir *, bool, int); extern bool _cpp_find_failed (_cpp_file *); extern void _cpp_mark_file_once_only (cpp_reader *, struct _cpp_file *); extern void _cpp_fake_include (cpp_reader *, const char *); extern bool _cpp_stack_file (cpp_reader *, _cpp_file*, bool); extern bool _cpp_stack_include (cpp_reader *, const char *, int, enum include_type); extern int _cpp_compare_file_date (cpp_reader *, const char *, int); extern void _cpp_report_missing_guards (cpp_reader *); extern void _cpp_init_files (cpp_reader *); extern void _cpp_cleanup_files (cpp_reader *); extern void _cpp_pop_file_buffer (cpp_reader *, struct _cpp_file *); extern bool _cpp_save_file_entries (cpp_reader *pfile, FILE *f); extern bool _cpp_read_file_entries (cpp_reader *, FILE *); extern struct stat *_cpp_get_file_stat (_cpp_file *); /* In expr.c */ extern bool _cpp_parse_expr (cpp_reader *); extern struct op *_cpp_expand_op_stack (cpp_reader *); /* In lex.c */ extern void _cpp_process_line_notes (cpp_reader *, int); extern void _cpp_clean_line (cpp_reader *); extern bool _cpp_get_fresh_line (cpp_reader *); extern bool _cpp_skip_block_comment (cpp_reader *); extern cpp_token *_cpp_temp_token (cpp_reader *); extern const cpp_token *_cpp_lex_token (cpp_reader *); extern cpp_token *_cpp_lex_direct (cpp_reader *); extern int _cpp_equiv_tokens (const cpp_token *, const cpp_token *); extern void _cpp_init_tokenrun (tokenrun *, unsigned int); /* In init.c. */ extern void _cpp_maybe_push_include_file (cpp_reader *); /* In directives.c */ extern int _cpp_test_assertion (cpp_reader *, unsigned int *); extern int _cpp_handle_directive (cpp_reader *, int); extern void _cpp_define_builtin (cpp_reader *, const char *); extern char ** _cpp_save_pragma_names (cpp_reader *); extern void _cpp_restore_pragma_names (cpp_reader *, char **); extern void _cpp_do__Pragma (cpp_reader *); extern void _cpp_init_directives (cpp_reader *); extern void _cpp_init_internal_pragmas (cpp_reader *); extern void _cpp_do_file_change (cpp_reader *, enum lc_reason, const char *, unsigned int, unsigned int); extern void _cpp_pop_buffer (cpp_reader *); /* In directives.c */ struct _cpp_dir_only_callbacks { /* Called to print a block of lines. */ void (*print_lines) (int, const void *, size_t); void (*maybe_print_line) (source_location); }; extern void _cpp_preprocess_dir_only (cpp_reader *, const struct _cpp_dir_only_callbacks *); /* In traditional.c. */ extern bool _cpp_scan_out_logical_line (cpp_reader *, cpp_macro *); extern bool _cpp_read_logical_line_trad (cpp_reader *); extern void _cpp_overlay_buffer (cpp_reader *pfile, const unsigned char *, size_t); extern void _cpp_remove_overlay (cpp_reader *); extern bool _cpp_create_trad_definition (cpp_reader *, cpp_macro *); extern bool _cpp_expansions_different_trad (const cpp_macro *, const cpp_macro *); extern unsigned char *_cpp_copy_replacement_text (const cpp_macro *, unsigned char *); extern size_t _cpp_replacement_text_len (const cpp_macro *); /* In charset.c. */ /* The normalization state at this point in the sequence. It starts initialized to all zeros, and at the end 'level' is the normalization level of the sequence. */ struct normalize_state { /* The previous character. */ cppchar_t previous; /* The combining class of the previous character. */ unsigned char prev_class; /* The lowest normalization level so far. */ enum cpp_normalize_level level; }; #define INITIAL_NORMALIZE_STATE { 0, 0, normalized_KC } #define NORMALIZE_STATE_RESULT(st) ((st)->level) /* We saw a character that matches ISIDNUM(), update a normalize_state appropriately. */ #define NORMALIZE_STATE_UPDATE_IDNUM(st) \ ((st)->previous = 0, (st)->prev_class = 0) extern cppchar_t _cpp_valid_ucn (cpp_reader *, const unsigned char **, const unsigned char *, int, struct normalize_state *state); extern void _cpp_destroy_iconv (cpp_reader *); extern unsigned char *_cpp_convert_input (cpp_reader *, const char *, unsigned char *, size_t, size_t, off_t *); extern const char *_cpp_default_encoding (void); extern cpp_hashnode * _cpp_interpret_identifier (cpp_reader *pfile, const unsigned char *id, size_t len); /* Utility routines and macros. */ #define DSC(str) (const unsigned char *)str, sizeof str - 1 /* These are inline functions instead of macros so we can get type checking. */ static inline int ustrcmp (const unsigned char *, const unsigned char *); static inline int ustrncmp (const unsigned char *, const unsigned char *, size_t); static inline size_t ustrlen (const unsigned char *); static inline unsigned char *uxstrdup (const unsigned char *); static inline unsigned char *ustrchr (const unsigned char *, int); static inline int ufputs (const unsigned char *, FILE *); /* Use a const char for the second parameter since it is usually a literal. */ static inline int ustrcspn (const unsigned char *, const char *); static inline int ustrcmp (const unsigned char *s1, const unsigned char *s2) { return strcmp ((const char *)s1, (const char *)s2); } static inline int ustrncmp (const unsigned char *s1, const unsigned char *s2, size_t n) { return strncmp ((const char *)s1, (const char *)s2, n); } static inline int ustrcspn (const unsigned char *s1, const char *s2) { return strcspn ((const char *)s1, s2); } static inline size_t ustrlen (const unsigned char *s1) { return strlen ((const char *)s1); } static inline unsigned char * uxstrdup (const unsigned char *s1) { return (unsigned char *) xstrdup ((const char *)s1); } static inline unsigned char * ustrchr (const unsigned char *s1, int c) { return (unsigned char *) strchr ((const char *)s1, c); } static inline int ufputs (const unsigned char *s, FILE *f) { return fputs ((const char *)s, f); } #endif /* ! LIBCPP_INTERNAL_H */ Index: projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/macro.c =================================================================== --- projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/macro.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/macro.c (revision 305172) @@ -1,1834 +1,1834 @@ /* Part of CPP library. (Macro and #define handling.) Copyright (C) 1986, 1987, 1989, 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc. Written by Per Bothner, 1994. Based on CCCP program by Paul Rubin, June 1986 Adapted to ANSI C, Richard Stallman, Jan 1987 This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. In other words, you are welcome to use, share and improve this program. You are forbidden to forbid anyone else to use, share and improve what you give them. Help stamp out software-hoarding! */ #include "config.h" #include "system.h" #include "cpplib.h" #include "internal.h" typedef struct macro_arg macro_arg; struct macro_arg { const cpp_token **first; /* First token in unexpanded argument. */ const cpp_token **expanded; /* Macro-expanded argument. */ const cpp_token *stringified; /* Stringified argument. */ unsigned int count; /* # of tokens in argument. */ unsigned int expanded_count; /* # of tokens in expanded argument. */ }; /* Macro expansion. */ static int enter_macro_context (cpp_reader *, cpp_hashnode *); static int builtin_macro (cpp_reader *, cpp_hashnode *); static void push_ptoken_context (cpp_reader *, cpp_hashnode *, _cpp_buff *, const cpp_token **, unsigned int); static _cpp_buff *collect_args (cpp_reader *, const cpp_hashnode *); static cpp_context *next_context (cpp_reader *); static const cpp_token *padding_token (cpp_reader *, const cpp_token *); static void expand_arg (cpp_reader *, macro_arg *); static const cpp_token *new_string_token (cpp_reader *, uchar *, unsigned int); static const cpp_token *stringify_arg (cpp_reader *, macro_arg *); static void paste_all_tokens (cpp_reader *, const cpp_token *); static bool paste_tokens (cpp_reader *, const cpp_token **, const cpp_token *); static void replace_args (cpp_reader *, cpp_hashnode *, cpp_macro *, macro_arg *); static _cpp_buff *funlike_invocation_p (cpp_reader *, cpp_hashnode *); static bool create_iso_definition (cpp_reader *, cpp_macro *); /* #define directive parsing and handling. */ static cpp_token *alloc_expansion_token (cpp_reader *, cpp_macro *); static cpp_token *lex_expansion_token (cpp_reader *, cpp_macro *); static bool warn_of_redefinition (cpp_reader *, const cpp_hashnode *, const cpp_macro *); static bool parse_params (cpp_reader *, cpp_macro *); static void check_trad_stringification (cpp_reader *, const cpp_macro *, const cpp_string *); /* Emits a warning if NODE is a macro defined in the main file that has not been used. */ int _cpp_warn_if_unused_macro (cpp_reader *pfile, cpp_hashnode *node, void *v ATTRIBUTE_UNUSED) { if (node->type == NT_MACRO && !(node->flags & NODE_BUILTIN)) { cpp_macro *macro = node->value.macro; if (!macro->used && MAIN_FILE_P (linemap_lookup (pfile->line_table, macro->line))) cpp_error_with_line (pfile, CPP_DL_WARNING, macro->line, 0, "macro \"%s\" is not used", NODE_NAME (node)); } return 1; } /* Allocates and returns a CPP_STRING token, containing TEXT of length LEN, after null-terminating it. TEXT must be in permanent storage. */ static const cpp_token * new_string_token (cpp_reader *pfile, unsigned char *text, unsigned int len) { cpp_token *token = _cpp_temp_token (pfile); text[len] = '\0'; token->type = CPP_STRING; token->val.str.len = len; token->val.str.text = text; token->flags = 0; return token; } static const char * const monthnames[] = { "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" }; /* Helper function for builtin_macro. Returns the text generated by a builtin macro. */ const uchar * _cpp_builtin_macro_text (cpp_reader *pfile, cpp_hashnode *node) { const struct line_map *map; const uchar *result = NULL; unsigned int number = 1; switch (node->value.builtin) { default: cpp_error (pfile, CPP_DL_ICE, "invalid built-in macro \"%s\"", NODE_NAME (node)); break; case BT_TIMESTAMP: { cpp_buffer *pbuffer = cpp_get_buffer (pfile); if (pbuffer->timestamp == NULL) { /* Initialize timestamp value of the assotiated file. */ struct _cpp_file *file = cpp_get_file (pbuffer); if (file) { /* Generate __TIMESTAMP__ string, that represents the date and time of the last modification of the current source file. The string constant looks like "Sun Sep 16 01:03:52 1973". */ struct tm *tb = NULL; struct stat *st = _cpp_get_file_stat (file); if (st) tb = localtime (&st->st_mtime); if (tb) { char *str = asctime (tb); size_t len = strlen (str); unsigned char *buf = _cpp_unaligned_alloc (pfile, len + 2); buf[0] = '"'; strcpy ((char *) buf + 1, str); buf[len] = '"'; pbuffer->timestamp = buf; } else { cpp_errno (pfile, CPP_DL_WARNING, "could not determine file timestamp"); pbuffer->timestamp = U"\"??? ??? ?? ??:??:?? ????\""; } } } result = pbuffer->timestamp; } break; case BT_FILE: case BT_BASE_FILE: { unsigned int len; const char *name; uchar *buf; map = linemap_lookup (pfile->line_table, pfile->line_table->highest_line); if (node->value.builtin == BT_BASE_FILE) while (! MAIN_FILE_P (map)) map = INCLUDED_FROM (pfile->line_table, map); name = map->to_file; len = strlen (name); buf = _cpp_unaligned_alloc (pfile, len * 2 + 3); result = buf; *buf = '"'; buf = cpp_quote_string (buf + 1, (const unsigned char *) name, len); *buf++ = '"'; *buf = '\0'; } break; case BT_INCLUDE_LEVEL: /* The line map depth counts the primary source as level 1, but historically __INCLUDE_DEPTH__ has called the primary source level 0. */ number = pfile->line_table->depth - 1; break; case BT_SPECLINE: map = &pfile->line_table->maps[pfile->line_table->used-1]; /* If __LINE__ is embedded in a macro, it must expand to the line of the macro's invocation, not its definition. Otherwise things like assert() will not work properly. */ if (CPP_OPTION (pfile, traditional)) number = pfile->line_table->highest_line; else number = pfile->cur_token[-1].src_loc; number = SOURCE_LINE (map, number); break; /* __STDC__ has the value 1 under normal circumstances. However, if (a) we are in a system header, (b) the option stdc_0_in_system_headers is true (set by target config), and (c) we are not in strictly conforming mode, then it has the value 0. (b) and (c) are already checked in cpp_init_builtins. */ case BT_STDC: if (cpp_in_system_header (pfile)) number = 0; else number = 1; break; case BT_DATE: case BT_TIME: if (pfile->date == NULL) { /* Allocate __DATE__ and __TIME__ strings from permanent storage. We only do this once, and don't generate them at init time, because time() and localtime() are very slow on some systems. */ time_t tt; struct tm *tb = NULL; /* (time_t) -1 is a legitimate value for "number of seconds since the Epoch", so we have to do a little dance to distinguish that from a genuine error. */ errno = 0; tt = time(NULL); if (tt != (time_t)-1 || errno == 0) tb = localtime (&tt); if (tb) { pfile->date = _cpp_unaligned_alloc (pfile, sizeof ("\"Oct 11 1347\"")); sprintf ((char *) pfile->date, "\"%s %2d %4d\"", monthnames[tb->tm_mon], tb->tm_mday, tb->tm_year + 1900); pfile->time = _cpp_unaligned_alloc (pfile, sizeof ("\"12:34:56\"")); sprintf ((char *) pfile->time, "\"%02d:%02d:%02d\"", tb->tm_hour, tb->tm_min, tb->tm_sec); } else { cpp_errno (pfile, CPP_DL_WARNING, "could not determine date and time"); pfile->date = U"\"??? ?? ????\""; pfile->time = U"\"??:??:??\""; } } if (node->value.builtin == BT_DATE) result = pfile->date; else result = pfile->time; break; case BT_COUNTER: if (CPP_OPTION (pfile, directives_only) && pfile->state.in_directive) cpp_error (pfile, CPP_DL_ERROR, "__COUNTER__ expanded inside directive with -fdirectives-only"); - number = pfile->nextcounter++; + number = pfile->counter++; break; } if (result == NULL) { /* 21 bytes holds all NUL-terminated unsigned 64-bit numbers. */ result = _cpp_unaligned_alloc (pfile, 21); sprintf ((char *) result, "%u", number); } return result; } /* Convert builtin macros like __FILE__ to a token and push it on the context stack. Also handles _Pragma, for which a new token may not be created. Returns 1 if it generates a new token context, 0 to return the token to the caller. */ static int builtin_macro (cpp_reader *pfile, cpp_hashnode *node) { const uchar *buf; size_t len; char *nbuf; if (node->value.builtin == BT_PRAGMA) { /* Don't interpret _Pragma within directives. The standard is not clear on this, but to me this makes most sense. */ if (pfile->state.in_directive) return 0; _cpp_do__Pragma (pfile); return 1; } buf = _cpp_builtin_macro_text (pfile, node); len = ustrlen (buf); nbuf = (char *) alloca (len + 1); memcpy (nbuf, buf, len); nbuf[len]='\n'; cpp_push_buffer (pfile, (uchar *) nbuf, len, /* from_stage3 */ true); _cpp_clean_line (pfile); /* Set pfile->cur_token as required by _cpp_lex_direct. */ pfile->cur_token = _cpp_temp_token (pfile); _cpp_push_token_context (pfile, NULL, _cpp_lex_direct (pfile), 1); if (pfile->buffer->cur != pfile->buffer->rlimit) cpp_error (pfile, CPP_DL_ICE, "invalid built-in macro \"%s\"", NODE_NAME (node)); _cpp_pop_buffer (pfile); return 1; } /* Copies SRC, of length LEN, to DEST, adding backslashes before all backslashes and double quotes. DEST must be of sufficient size. Returns a pointer to the end of the string. */ uchar * cpp_quote_string (uchar *dest, const uchar *src, unsigned int len) { while (len--) { uchar c = *src++; if (c == '\\' || c == '"') { *dest++ = '\\'; *dest++ = c; } else *dest++ = c; } return dest; } /* Convert a token sequence ARG to a single string token according to the rules of the ISO C #-operator. */ static const cpp_token * stringify_arg (cpp_reader *pfile, macro_arg *arg) { unsigned char *dest; unsigned int i, escape_it, backslash_count = 0; const cpp_token *source = NULL; size_t len; if (BUFF_ROOM (pfile->u_buff) < 3) _cpp_extend_buff (pfile, &pfile->u_buff, 3); dest = BUFF_FRONT (pfile->u_buff); *dest++ = '"'; /* Loop, reading in the argument's tokens. */ for (i = 0; i < arg->count; i++) { const cpp_token *token = arg->first[i]; if (token->type == CPP_PADDING) { if (source == NULL) source = token->val.source; continue; } escape_it = (token->type == CPP_STRING || token->type == CPP_WSTRING || token->type == CPP_CHAR || token->type == CPP_WCHAR); /* Room for each char being written in octal, initial space and final quote and NUL. */ len = cpp_token_len (token); if (escape_it) len *= 4; len += 3; if ((size_t) (BUFF_LIMIT (pfile->u_buff) - dest) < len) { size_t len_so_far = dest - BUFF_FRONT (pfile->u_buff); _cpp_extend_buff (pfile, &pfile->u_buff, len); dest = BUFF_FRONT (pfile->u_buff) + len_so_far; } /* Leading white space? */ if (dest - 1 != BUFF_FRONT (pfile->u_buff)) { if (source == NULL) source = token; if (source->flags & PREV_WHITE) *dest++ = ' '; } source = NULL; if (escape_it) { _cpp_buff *buff = _cpp_get_buff (pfile, len); unsigned char *buf = BUFF_FRONT (buff); len = cpp_spell_token (pfile, token, buf, true) - buf; dest = cpp_quote_string (dest, buf, len); _cpp_release_buff (pfile, buff); } else dest = cpp_spell_token (pfile, token, dest, true); if (token->type == CPP_OTHER && token->val.str.text[0] == '\\') backslash_count++; else backslash_count = 0; } /* Ignore the final \ of invalid string literals. */ if (backslash_count & 1) { cpp_error (pfile, CPP_DL_WARNING, "invalid string literal, ignoring final '\\'"); dest--; } /* Commit the memory, including NUL, and return the token. */ *dest++ = '"'; len = dest - BUFF_FRONT (pfile->u_buff); BUFF_FRONT (pfile->u_buff) = dest + 1; return new_string_token (pfile, dest - len, len); } /* Try to paste two tokens. On success, return nonzero. In any case, PLHS is updated to point to the pasted token, which is guaranteed to not have the PASTE_LEFT flag set. */ static bool paste_tokens (cpp_reader *pfile, const cpp_token **plhs, const cpp_token *rhs) { unsigned char *buf, *end, *lhsend; cpp_token *lhs; unsigned int len; len = cpp_token_len (*plhs) + cpp_token_len (rhs) + 1; buf = (unsigned char *) alloca (len); end = lhsend = cpp_spell_token (pfile, *plhs, buf, false); /* Avoid comment headers, since they are still processed in stage 3. It is simpler to insert a space here, rather than modifying the lexer to ignore comments in some circumstances. Simply returning false doesn't work, since we want to clear the PASTE_LEFT flag. */ if ((*plhs)->type == CPP_DIV && rhs->type != CPP_EQ) *end++ = ' '; end = cpp_spell_token (pfile, rhs, end, false); *end = '\n'; cpp_push_buffer (pfile, buf, end - buf, /* from_stage3 */ true); _cpp_clean_line (pfile); /* Set pfile->cur_token as required by _cpp_lex_direct. */ pfile->cur_token = _cpp_temp_token (pfile); lhs = _cpp_lex_direct (pfile); if (pfile->buffer->cur != pfile->buffer->rlimit) { source_location saved_loc = lhs->src_loc; _cpp_pop_buffer (pfile); _cpp_backup_tokens (pfile, 1); *lhsend = '\0'; /* We have to remove the PASTE_LEFT flag from the old lhs, but we want to keep the new location. */ *lhs = **plhs; *plhs = lhs; lhs->src_loc = saved_loc; lhs->flags &= ~PASTE_LEFT; /* Mandatory error for all apart from assembler. */ if (CPP_OPTION (pfile, lang) != CLK_ASM) cpp_error (pfile, CPP_DL_ERROR, "pasting \"%s\" and \"%s\" does not give a valid preprocessing token", buf, cpp_token_as_text (pfile, rhs)); return false; } *plhs = lhs; _cpp_pop_buffer (pfile); return true; } /* Handles an arbitrarily long sequence of ## operators, with initial operand LHS. This implementation is left-associative, non-recursive, and finishes a paste before handling succeeding ones. If a paste fails, we back up to the RHS of the failing ## operator before pushing the context containing the result of prior successful pastes, with the effect that the RHS appears in the output stream after the pasted LHS normally. */ static void paste_all_tokens (cpp_reader *pfile, const cpp_token *lhs) { const cpp_token *rhs; cpp_context *context = pfile->context; do { /* Take the token directly from the current context. We can do this, because we are in the replacement list of either an object-like macro, or a function-like macro with arguments inserted. In either case, the constraints to #define guarantee we have at least one more token. */ if (context->direct_p) rhs = FIRST (context).token++; else rhs = *FIRST (context).ptoken++; if (rhs->type == CPP_PADDING) abort (); if (!paste_tokens (pfile, &lhs, rhs)) break; } while (rhs->flags & PASTE_LEFT); /* Put the resulting token in its own context. */ _cpp_push_token_context (pfile, NULL, lhs, 1); } /* Returns TRUE if the number of arguments ARGC supplied in an invocation of the MACRO referenced by NODE is valid. An empty invocation to a macro with no parameters should pass ARGC as zero. Note that MACRO cannot necessarily be deduced from NODE, in case NODE was redefined whilst collecting arguments. */ bool _cpp_arguments_ok (cpp_reader *pfile, cpp_macro *macro, const cpp_hashnode *node, unsigned int argc) { if (argc == macro->paramc) return true; if (argc < macro->paramc) { /* As an extension, a rest argument is allowed to not appear in the invocation at all. e.g. #define debug(format, args...) something debug("string"); This is exactly the same as if there had been an empty rest argument - debug("string", ). */ if (argc + 1 == macro->paramc && macro->variadic) { if (CPP_PEDANTIC (pfile) && ! macro->syshdr) cpp_error (pfile, CPP_DL_PEDWARN, "ISO C99 requires rest arguments to be used"); return true; } cpp_error (pfile, CPP_DL_ERROR, "macro \"%s\" requires %u arguments, but only %u given", NODE_NAME (node), macro->paramc, argc); } else cpp_error (pfile, CPP_DL_ERROR, "macro \"%s\" passed %u arguments, but takes just %u", NODE_NAME (node), argc, macro->paramc); return false; } /* Reads and returns the arguments to a function-like macro invocation. Assumes the opening parenthesis has been processed. If there is an error, emits an appropriate diagnostic and returns NULL. Each argument is terminated by a CPP_EOF token, for the future benefit of expand_arg(). */ static _cpp_buff * collect_args (cpp_reader *pfile, const cpp_hashnode *node) { _cpp_buff *buff, *base_buff; cpp_macro *macro; macro_arg *args, *arg; const cpp_token *token; unsigned int argc; macro = node->value.macro; if (macro->paramc) argc = macro->paramc; else argc = 1; buff = _cpp_get_buff (pfile, argc * (50 * sizeof (cpp_token *) + sizeof (macro_arg))); base_buff = buff; args = (macro_arg *) buff->base; memset (args, 0, argc * sizeof (macro_arg)); buff->cur = (unsigned char *) &args[argc]; arg = args, argc = 0; /* Collect the tokens making up each argument. We don't yet know how many arguments have been supplied, whether too many or too few. Hence the slightly bizarre usage of "argc" and "arg". */ do { unsigned int paren_depth = 0; unsigned int ntokens = 0; argc++; arg->first = (const cpp_token **) buff->cur; for (;;) { /* Require space for 2 new tokens (including a CPP_EOF). */ if ((unsigned char *) &arg->first[ntokens + 2] > buff->limit) { buff = _cpp_append_extend_buff (pfile, buff, 1000 * sizeof (cpp_token *)); arg->first = (const cpp_token **) buff->cur; } token = cpp_get_token (pfile); if (token->type == CPP_PADDING) { /* Drop leading padding. */ if (ntokens == 0) continue; } else if (token->type == CPP_OPEN_PAREN) paren_depth++; else if (token->type == CPP_CLOSE_PAREN) { if (paren_depth-- == 0) break; } else if (token->type == CPP_COMMA) { /* A comma does not terminate an argument within parentheses or as part of a variable argument. */ if (paren_depth == 0 && ! (macro->variadic && argc == macro->paramc)) break; } else if (token->type == CPP_EOF || (token->type == CPP_HASH && token->flags & BOL)) break; arg->first[ntokens++] = token; } /* Drop trailing padding. */ while (ntokens > 0 && arg->first[ntokens - 1]->type == CPP_PADDING) ntokens--; arg->count = ntokens; arg->first[ntokens] = &pfile->eof; /* Terminate the argument. Excess arguments loop back and overwrite the final legitimate argument, before failing. */ if (argc <= macro->paramc) { buff->cur = (unsigned char *) &arg->first[ntokens + 1]; if (argc != macro->paramc) arg++; } } while (token->type != CPP_CLOSE_PAREN && token->type != CPP_EOF); if (token->type == CPP_EOF) { /* We still need the CPP_EOF to end directives, and to end pre-expansion of a macro argument. Step back is not unconditional, since we don't want to return a CPP_EOF to our callers at the end of an -include-d file. */ if (pfile->context->prev || pfile->state.in_directive) _cpp_backup_tokens (pfile, 1); cpp_error (pfile, CPP_DL_ERROR, "unterminated argument list invoking macro \"%s\"", NODE_NAME (node)); } else { /* A single empty argument is counted as no argument. */ if (argc == 1 && macro->paramc == 0 && args[0].count == 0) argc = 0; if (_cpp_arguments_ok (pfile, macro, node, argc)) { /* GCC has special semantics for , ## b where b is a varargs parameter: we remove the comma if b was omitted entirely. If b was merely an empty argument, the comma is retained. If the macro takes just one (varargs) parameter, then we retain the comma only if we are standards conforming. If FIRST is NULL replace_args () swallows the comma. */ if (macro->variadic && (argc < macro->paramc || (argc == 1 && args[0].count == 0 && !CPP_OPTION (pfile, std)))) args[macro->paramc - 1].first = NULL; return base_buff; } } /* An error occurred. */ _cpp_release_buff (pfile, base_buff); return NULL; } /* Search for an opening parenthesis to the macro of NODE, in such a way that, if none is found, we don't lose the information in any intervening padding tokens. If we find the parenthesis, collect the arguments and return the buffer containing them. */ static _cpp_buff * funlike_invocation_p (cpp_reader *pfile, cpp_hashnode *node) { const cpp_token *token, *padding = NULL; for (;;) { token = cpp_get_token (pfile); if (token->type != CPP_PADDING) break; if (padding == NULL || (!(padding->flags & PREV_WHITE) && token->val.source == NULL)) padding = token; } if (token->type == CPP_OPEN_PAREN) { pfile->state.parsing_args = 2; return collect_args (pfile, node); } /* CPP_EOF can be the end of macro arguments, or the end of the file. We mustn't back up over the latter. Ugh. */ if (token->type != CPP_EOF || token == &pfile->eof) { /* Back up. We may have skipped padding, in which case backing up more than one token when expanding macros is in general too difficult. We re-insert it in its own context. */ _cpp_backup_tokens (pfile, 1); if (padding) _cpp_push_token_context (pfile, NULL, padding, 1); } return NULL; } /* Push the context of a macro with hash entry NODE onto the context stack. If we can successfully expand the macro, we push a context containing its yet-to-be-rescanned replacement list and return one. Otherwise, we don't push a context and return zero. */ static int enter_macro_context (cpp_reader *pfile, cpp_hashnode *node) { /* The presence of a macro invalidates a file's controlling macro. */ pfile->mi_valid = false; pfile->state.angled_headers = false; /* Handle standard macros. */ if (! (node->flags & NODE_BUILTIN)) { cpp_macro *macro = node->value.macro; if (macro->fun_like) { _cpp_buff *buff; pfile->state.prevent_expansion++; pfile->keep_tokens++; pfile->state.parsing_args = 1; buff = funlike_invocation_p (pfile, node); pfile->state.parsing_args = 0; pfile->keep_tokens--; pfile->state.prevent_expansion--; if (buff == NULL) { if (CPP_WTRADITIONAL (pfile) && ! node->value.macro->syshdr) cpp_error (pfile, CPP_DL_WARNING, "function-like macro \"%s\" must be used with arguments in traditional C", NODE_NAME (node)); return 0; } if (macro->paramc > 0) replace_args (pfile, node, macro, (macro_arg *) buff->base); _cpp_release_buff (pfile, buff); } /* Disable the macro within its expansion. */ node->flags |= NODE_DISABLED; macro->used = 1; if (macro->paramc == 0) _cpp_push_token_context (pfile, node, macro->exp.tokens, macro->count); return 1; } /* Handle built-in macros and the _Pragma operator. */ return builtin_macro (pfile, node); } /* Replace the parameters in a function-like macro of NODE with the actual ARGS, and place the result in a newly pushed token context. Expand each argument before replacing, unless it is operated upon by the # or ## operators. */ static void replace_args (cpp_reader *pfile, cpp_hashnode *node, cpp_macro *macro, macro_arg *args) { unsigned int i, total; const cpp_token *src, *limit; const cpp_token **dest, **first; macro_arg *arg; _cpp_buff *buff; /* First, fully macro-expand arguments, calculating the number of tokens in the final expansion as we go. The ordering of the if statements below is subtle; we must handle stringification before pasting. */ total = macro->count; limit = macro->exp.tokens + macro->count; for (src = macro->exp.tokens; src < limit; src++) if (src->type == CPP_MACRO_ARG) { /* Leading and trailing padding tokens. */ total += 2; /* We have an argument. If it is not being stringified or pasted it is macro-replaced before insertion. */ arg = &args[src->val.arg_no - 1]; if (src->flags & STRINGIFY_ARG) { if (!arg->stringified) arg->stringified = stringify_arg (pfile, arg); } else if ((src->flags & PASTE_LEFT) || (src > macro->exp.tokens && (src[-1].flags & PASTE_LEFT))) total += arg->count - 1; else { if (!arg->expanded) expand_arg (pfile, arg); total += arg->expanded_count - 1; } } /* Now allocate space for the expansion, copy the tokens and replace the arguments. */ buff = _cpp_get_buff (pfile, total * sizeof (cpp_token *)); first = (const cpp_token **) buff->base; dest = first; for (src = macro->exp.tokens; src < limit; src++) { unsigned int count; const cpp_token **from, **paste_flag; if (src->type != CPP_MACRO_ARG) { *dest++ = src; continue; } paste_flag = 0; arg = &args[src->val.arg_no - 1]; if (src->flags & STRINGIFY_ARG) count = 1, from = &arg->stringified; else if (src->flags & PASTE_LEFT) count = arg->count, from = arg->first; else if (src != macro->exp.tokens && (src[-1].flags & PASTE_LEFT)) { count = arg->count, from = arg->first; if (dest != first) { if (dest[-1]->type == CPP_COMMA && macro->variadic && src->val.arg_no == macro->paramc) { /* Swallow a pasted comma if from == NULL, otherwise drop the paste flag. */ if (from == NULL) dest--; else paste_flag = dest - 1; } /* Remove the paste flag if the RHS is a placemarker. */ else if (count == 0) paste_flag = dest - 1; } } else count = arg->expanded_count, from = arg->expanded; /* Padding on the left of an argument (unless RHS of ##). */ if ((!pfile->state.in_directive || pfile->state.directive_wants_padding) && src != macro->exp.tokens && !(src[-1].flags & PASTE_LEFT)) *dest++ = padding_token (pfile, src); if (count) { memcpy (dest, from, count * sizeof (cpp_token *)); dest += count; /* With a non-empty argument on the LHS of ##, the last token should be flagged PASTE_LEFT. */ if (src->flags & PASTE_LEFT) paste_flag = dest - 1; } /* Avoid paste on RHS (even case count == 0). */ if (!pfile->state.in_directive && !(src->flags & PASTE_LEFT)) *dest++ = &pfile->avoid_paste; /* Add a new paste flag, or remove an unwanted one. */ if (paste_flag) { cpp_token *token = _cpp_temp_token (pfile); token->type = (*paste_flag)->type; token->val = (*paste_flag)->val; if (src->flags & PASTE_LEFT) token->flags = (*paste_flag)->flags | PASTE_LEFT; else token->flags = (*paste_flag)->flags & ~PASTE_LEFT; *paste_flag = token; } } /* Free the expanded arguments. */ for (i = 0; i < macro->paramc; i++) if (args[i].expanded) free (args[i].expanded); push_ptoken_context (pfile, node, buff, first, dest - first); } /* Return a special padding token, with padding inherited from SOURCE. */ static const cpp_token * padding_token (cpp_reader *pfile, const cpp_token *source) { cpp_token *result = _cpp_temp_token (pfile); result->type = CPP_PADDING; /* Data in GCed data structures cannot be made const so far, so we need a cast here. */ result->val.source = (cpp_token *) source; result->flags = 0; return result; } /* Get a new uninitialized context. Create a new one if we cannot re-use an old one. */ static cpp_context * next_context (cpp_reader *pfile) { cpp_context *result = pfile->context->next; if (result == 0) { result = XNEW (cpp_context); result->prev = pfile->context; result->next = 0; pfile->context->next = result; } pfile->context = result; return result; } /* Push a list of pointers to tokens. */ static void push_ptoken_context (cpp_reader *pfile, cpp_hashnode *macro, _cpp_buff *buff, const cpp_token **first, unsigned int count) { cpp_context *context = next_context (pfile); context->direct_p = false; context->macro = macro; context->buff = buff; FIRST (context).ptoken = first; LAST (context).ptoken = first + count; } /* Push a list of tokens. */ void _cpp_push_token_context (cpp_reader *pfile, cpp_hashnode *macro, const cpp_token *first, unsigned int count) { cpp_context *context = next_context (pfile); context->direct_p = true; context->macro = macro; context->buff = NULL; FIRST (context).token = first; LAST (context).token = first + count; } /* Push a traditional macro's replacement text. */ void _cpp_push_text_context (cpp_reader *pfile, cpp_hashnode *macro, const uchar *start, size_t len) { cpp_context *context = next_context (pfile); context->direct_p = true; context->macro = macro; context->buff = NULL; CUR (context) = start; RLIMIT (context) = start + len; macro->flags |= NODE_DISABLED; } /* Expand an argument ARG before replacing parameters in a function-like macro. This works by pushing a context with the argument's tokens, and then expanding that into a temporary buffer as if it were a normal part of the token stream. collect_args() has terminated the argument's tokens with a CPP_EOF so that we know when we have fully expanded the argument. */ static void expand_arg (cpp_reader *pfile, macro_arg *arg) { unsigned int capacity; bool saved_warn_trad; if (arg->count == 0) return; /* Don't warn about funlike macros when pre-expanding. */ saved_warn_trad = CPP_WTRADITIONAL (pfile); CPP_WTRADITIONAL (pfile) = 0; /* Loop, reading in the arguments. */ capacity = 256; arg->expanded = XNEWVEC (const cpp_token *, capacity); push_ptoken_context (pfile, NULL, NULL, arg->first, arg->count + 1); for (;;) { const cpp_token *token; if (arg->expanded_count + 1 >= capacity) { capacity *= 2; arg->expanded = XRESIZEVEC (const cpp_token *, arg->expanded, capacity); } token = cpp_get_token (pfile); if (token->type == CPP_EOF) break; arg->expanded[arg->expanded_count++] = token; } _cpp_pop_context (pfile); CPP_WTRADITIONAL (pfile) = saved_warn_trad; } /* Pop the current context off the stack, re-enabling the macro if the context represented a macro's replacement list. The context structure is not freed so that we can re-use it later. */ void _cpp_pop_context (cpp_reader *pfile) { cpp_context *context = pfile->context; if (context->macro) context->macro->flags &= ~NODE_DISABLED; if (context->buff) _cpp_release_buff (pfile, context->buff); pfile->context = context->prev; } /* External routine to get a token. Also used nearly everywhere internally, except for places where we know we can safely call _cpp_lex_token directly, such as lexing a directive name. Macro expansions and directives are transparently handled, including entering included files. Thus tokens are post-macro expansion, and after any intervening directives. External callers see CPP_EOF only at EOF. Internal callers also see it when meeting a directive inside a macro call, when at the end of a directive and state.in_directive is still 1, and at the end of argument pre-expansion. */ const cpp_token * cpp_get_token (cpp_reader *pfile) { const cpp_token *result; for (;;) { cpp_hashnode *node; cpp_context *context = pfile->context; /* Context->prev == 0 <=> base context. */ if (!context->prev) result = _cpp_lex_token (pfile); else if (FIRST (context).token != LAST (context).token) { if (context->direct_p) result = FIRST (context).token++; else result = *FIRST (context).ptoken++; if (result->flags & PASTE_LEFT) { paste_all_tokens (pfile, result); if (pfile->state.in_directive) continue; return padding_token (pfile, result); } } else { _cpp_pop_context (pfile); if (pfile->state.in_directive) continue; return &pfile->avoid_paste; } if (pfile->state.in_directive && result->type == CPP_COMMENT) continue; if (result->type != CPP_NAME) break; node = result->val.node; if (node->type != NT_MACRO || (result->flags & NO_EXPAND)) break; if (!(node->flags & NODE_DISABLED)) { if (!pfile->state.prevent_expansion && enter_macro_context (pfile, node)) { if (pfile->state.in_directive) continue; return padding_token (pfile, result); } } else { /* Flag this token as always unexpandable. FIXME: move this to collect_args()?. */ cpp_token *t = _cpp_temp_token (pfile); t->type = result->type; t->flags = result->flags | NO_EXPAND; t->val = result->val; result = t; } break; } return result; } /* Returns true if we're expanding an object-like macro that was defined in a system header. Just checks the macro at the top of the stack. Used for diagnostic suppression. */ int cpp_sys_macro_p (cpp_reader *pfile) { cpp_hashnode *node = pfile->context->macro; return node && node->value.macro && node->value.macro->syshdr; } /* Read each token in, until end of the current file. Directives are transparently processed. */ void cpp_scan_nooutput (cpp_reader *pfile) { /* Request a CPP_EOF token at the end of this file, rather than transparently continuing with the including file. */ pfile->buffer->return_at_eof = true; pfile->state.discarding_output++; pfile->state.prevent_expansion++; if (CPP_OPTION (pfile, traditional)) while (_cpp_read_logical_line_trad (pfile)) ; else while (cpp_get_token (pfile)->type != CPP_EOF) ; pfile->state.discarding_output--; pfile->state.prevent_expansion--; } /* Step back one (or more) tokens. Can only step back more than 1 if they are from the lexer, and not from macro expansion. */ void _cpp_backup_tokens (cpp_reader *pfile, unsigned int count) { if (pfile->context->prev == NULL) { pfile->lookaheads += count; while (count--) { pfile->cur_token--; if (pfile->cur_token == pfile->cur_run->base /* Possible with -fpreprocessed and no leading #line. */ && pfile->cur_run->prev != NULL) { pfile->cur_run = pfile->cur_run->prev; pfile->cur_token = pfile->cur_run->limit; } } } else { if (count != 1) abort (); if (pfile->context->direct_p) FIRST (pfile->context).token--; else FIRST (pfile->context).ptoken--; } } /* #define directive parsing and handling. */ /* Returns nonzero if a macro redefinition warning is required. */ static bool warn_of_redefinition (cpp_reader *pfile, const cpp_hashnode *node, const cpp_macro *macro2) { const cpp_macro *macro1; unsigned int i; /* Some redefinitions need to be warned about regardless. */ if (node->flags & NODE_WARN) return true; /* Redefinition of a macro is allowed if and only if the old and new definitions are the same. (6.10.3 paragraph 2). */ macro1 = node->value.macro; /* Don't check count here as it can be different in valid traditional redefinitions with just whitespace differences. */ if (macro1->paramc != macro2->paramc || macro1->fun_like != macro2->fun_like || macro1->variadic != macro2->variadic) return true; /* Check parameter spellings. */ for (i = 0; i < macro1->paramc; i++) if (macro1->params[i] != macro2->params[i]) return true; /* Check the replacement text or tokens. */ if (CPP_OPTION (pfile, traditional)) return _cpp_expansions_different_trad (macro1, macro2); if (macro1->count != macro2->count) return true; for (i = 0; i < macro1->count; i++) if (!_cpp_equiv_tokens (¯o1->exp.tokens[i], ¯o2->exp.tokens[i])) return true; return false; } /* Free the definition of hashnode H. */ void _cpp_free_definition (cpp_hashnode *h) { /* Macros and assertions no longer have anything to free. */ h->type = NT_VOID; /* Clear builtin flag in case of redefinition. */ h->flags &= ~(NODE_BUILTIN | NODE_DISABLED); } /* Save parameter NODE to the parameter list of macro MACRO. Returns zero on success, nonzero if the parameter is a duplicate. */ bool _cpp_save_parameter (cpp_reader *pfile, cpp_macro *macro, cpp_hashnode *node) { unsigned int len; /* Constraint 6.10.3.6 - duplicate parameter names. */ if (node->flags & NODE_MACRO_ARG) { cpp_error (pfile, CPP_DL_ERROR, "duplicate macro parameter \"%s\"", NODE_NAME (node)); return true; } if (BUFF_ROOM (pfile->a_buff) < (macro->paramc + 1) * sizeof (cpp_hashnode *)) _cpp_extend_buff (pfile, &pfile->a_buff, sizeof (cpp_hashnode *)); ((cpp_hashnode **) BUFF_FRONT (pfile->a_buff))[macro->paramc++] = node; node->flags |= NODE_MACRO_ARG; len = macro->paramc * sizeof (union _cpp_hashnode_value); if (len > pfile->macro_buffer_len) { pfile->macro_buffer = XRESIZEVEC (unsigned char, pfile->macro_buffer, len); pfile->macro_buffer_len = len; } ((union _cpp_hashnode_value *) pfile->macro_buffer)[macro->paramc - 1] = node->value; node->value.arg_index = macro->paramc; return false; } /* Check the syntax of the parameters in a MACRO definition. Returns false if an error occurs. */ static bool parse_params (cpp_reader *pfile, cpp_macro *macro) { unsigned int prev_ident = 0; for (;;) { const cpp_token *token = _cpp_lex_token (pfile); switch (token->type) { default: /* Allow/ignore comments in parameter lists if we are preserving comments in macro expansions. */ if (token->type == CPP_COMMENT && ! CPP_OPTION (pfile, discard_comments_in_macro_exp)) continue; cpp_error (pfile, CPP_DL_ERROR, "\"%s\" may not appear in macro parameter list", cpp_token_as_text (pfile, token)); return false; case CPP_NAME: if (prev_ident) { cpp_error (pfile, CPP_DL_ERROR, "macro parameters must be comma-separated"); return false; } prev_ident = 1; if (_cpp_save_parameter (pfile, macro, token->val.node)) return false; continue; case CPP_CLOSE_PAREN: if (prev_ident || macro->paramc == 0) return true; /* Fall through to pick up the error. */ case CPP_COMMA: if (!prev_ident) { cpp_error (pfile, CPP_DL_ERROR, "parameter name missing"); return false; } prev_ident = 0; continue; case CPP_ELLIPSIS: macro->variadic = 1; if (!prev_ident) { _cpp_save_parameter (pfile, macro, pfile->spec_nodes.n__VA_ARGS__); pfile->state.va_args_ok = 1; if (! CPP_OPTION (pfile, c99) && CPP_OPTION (pfile, pedantic) && CPP_OPTION (pfile, warn_variadic_macros)) cpp_error (pfile, CPP_DL_PEDWARN, "anonymous variadic macros were introduced in C99"); } else if (CPP_OPTION (pfile, pedantic) && CPP_OPTION (pfile, warn_variadic_macros)) cpp_error (pfile, CPP_DL_PEDWARN, "ISO C does not permit named variadic macros"); /* We're at the end, and just expect a closing parenthesis. */ token = _cpp_lex_token (pfile); if (token->type == CPP_CLOSE_PAREN) return true; /* Fall through. */ case CPP_EOF: cpp_error (pfile, CPP_DL_ERROR, "missing ')' in macro parameter list"); return false; } } } /* Allocate room for a token from a macro's replacement list. */ static cpp_token * alloc_expansion_token (cpp_reader *pfile, cpp_macro *macro) { if (BUFF_ROOM (pfile->a_buff) < (macro->count + 1) * sizeof (cpp_token)) _cpp_extend_buff (pfile, &pfile->a_buff, sizeof (cpp_token)); return &((cpp_token *) BUFF_FRONT (pfile->a_buff))[macro->count++]; } /* Lex a token from the expansion of MACRO, but mark parameters as we find them and warn of traditional stringification. */ static cpp_token * lex_expansion_token (cpp_reader *pfile, cpp_macro *macro) { cpp_token *token, *saved_cur_token; saved_cur_token = pfile->cur_token; pfile->cur_token = alloc_expansion_token (pfile, macro); token = _cpp_lex_direct (pfile); pfile->cur_token = saved_cur_token; /* Is this a parameter? */ if (token->type == CPP_NAME && (token->val.node->flags & NODE_MACRO_ARG) != 0) { token->type = CPP_MACRO_ARG; token->val.arg_no = token->val.node->value.arg_index; } else if (CPP_WTRADITIONAL (pfile) && macro->paramc > 0 && (token->type == CPP_STRING || token->type == CPP_CHAR)) check_trad_stringification (pfile, macro, &token->val.str); return token; } static bool create_iso_definition (cpp_reader *pfile, cpp_macro *macro) { cpp_token *token; const cpp_token *ctoken; /* Get the first token of the expansion (or the '(' of a function-like macro). */ ctoken = _cpp_lex_token (pfile); if (ctoken->type == CPP_OPEN_PAREN && !(ctoken->flags & PREV_WHITE)) { bool ok = parse_params (pfile, macro); macro->params = (cpp_hashnode **) BUFF_FRONT (pfile->a_buff); if (!ok) return false; /* Success. Commit or allocate the parameter array. */ if (pfile->hash_table->alloc_subobject) { cpp_hashnode **params = (cpp_hashnode **) pfile->hash_table->alloc_subobject (sizeof (cpp_hashnode *) * macro->paramc); memcpy (params, macro->params, sizeof (cpp_hashnode *) * macro->paramc); macro->params = params; } else BUFF_FRONT (pfile->a_buff) = (uchar *) ¯o->params[macro->paramc]; macro->fun_like = 1; } else if (ctoken->type != CPP_EOF && !(ctoken->flags & PREV_WHITE)) { /* While ISO C99 requires whitespace before replacement text in a macro definition, ISO C90 with TC1 allows there characters from the basic source character set. */ if (CPP_OPTION (pfile, c99)) cpp_error (pfile, CPP_DL_PEDWARN, "ISO C99 requires whitespace after the macro name"); else { int warntype = CPP_DL_WARNING; switch (ctoken->type) { case CPP_ATSIGN: case CPP_AT_NAME: case CPP_OBJC_STRING: /* '@' is not in basic character set. */ warntype = CPP_DL_PEDWARN; break; case CPP_OTHER: /* Basic character set sans letters, digits and _. */ if (strchr ("!\"#%&'()*+,-./:;<=>?[\\]^{|}~", ctoken->val.str.text[0]) == NULL) warntype = CPP_DL_PEDWARN; break; default: /* All other tokens start with a character from basic character set. */ break; } cpp_error (pfile, warntype, "missing whitespace after the macro name"); } } if (macro->fun_like) token = lex_expansion_token (pfile, macro); else { token = alloc_expansion_token (pfile, macro); *token = *ctoken; } for (;;) { /* Check the stringifying # constraint 6.10.3.2.1 of function-like macros when lexing the subsequent token. */ if (macro->count > 1 && token[-1].type == CPP_HASH && macro->fun_like) { if (token->type == CPP_MACRO_ARG) { token->flags &= ~PREV_WHITE; token->flags |= STRINGIFY_ARG; token->flags |= token[-1].flags & PREV_WHITE; token[-1] = token[0]; macro->count--; } /* Let assembler get away with murder. */ else if (CPP_OPTION (pfile, lang) != CLK_ASM) { cpp_error (pfile, CPP_DL_ERROR, "'#' is not followed by a macro parameter"); return false; } } if (token->type == CPP_EOF) break; /* Paste operator constraint 6.10.3.3.1. */ if (token->type == CPP_PASTE) { /* Token-paste ##, can appear in both object-like and function-like macros, but not at the ends. */ if (--macro->count > 0) token = lex_expansion_token (pfile, macro); if (macro->count == 0 || token->type == CPP_EOF) { cpp_error (pfile, CPP_DL_ERROR, "'##' cannot appear at either end of a macro expansion"); return false; } token[-1].flags |= PASTE_LEFT; } token = lex_expansion_token (pfile, macro); } macro->exp.tokens = (cpp_token *) BUFF_FRONT (pfile->a_buff); macro->traditional = 0; /* Don't count the CPP_EOF. */ macro->count--; /* Clear whitespace on first token for warn_of_redefinition(). */ if (macro->count) macro->exp.tokens[0].flags &= ~PREV_WHITE; /* Commit or allocate the memory. */ if (pfile->hash_table->alloc_subobject) { cpp_token *tokns = (cpp_token *) pfile->hash_table->alloc_subobject (sizeof (cpp_token) * macro->count); memcpy (tokns, macro->exp.tokens, sizeof (cpp_token) * macro->count); macro->exp.tokens = tokns; } else BUFF_FRONT (pfile->a_buff) = (uchar *) ¯o->exp.tokens[macro->count]; return true; } /* Parse a macro and save its expansion. Returns nonzero on success. */ bool _cpp_create_definition (cpp_reader *pfile, cpp_hashnode *node) { cpp_macro *macro; unsigned int i; bool ok; if (pfile->hash_table->alloc_subobject) macro = (cpp_macro *) pfile->hash_table->alloc_subobject (sizeof (cpp_macro)); else macro = (cpp_macro *) _cpp_aligned_alloc (pfile, sizeof (cpp_macro)); macro->line = pfile->directive_line; macro->params = 0; macro->paramc = 0; macro->variadic = 0; macro->used = !CPP_OPTION (pfile, warn_unused_macros); macro->count = 0; macro->fun_like = 0; /* To suppress some diagnostics. */ macro->syshdr = pfile->buffer && pfile->buffer->sysp != 0; if (CPP_OPTION (pfile, traditional)) ok = _cpp_create_trad_definition (pfile, macro); else { ok = create_iso_definition (pfile, macro); /* We set the type for SEEN_EOL() in directives.c. Longer term we should lex the whole line before coming here, and just copy the expansion. */ /* Stop the lexer accepting __VA_ARGS__. */ pfile->state.va_args_ok = 0; } /* Clear the fast argument lookup indices. */ for (i = macro->paramc; i-- > 0; ) { struct cpp_hashnode *node = macro->params[i]; node->flags &= ~ NODE_MACRO_ARG; node->value = ((union _cpp_hashnode_value *) pfile->macro_buffer)[i]; } if (!ok) return ok; if (node->type == NT_MACRO) { if (CPP_OPTION (pfile, warn_unused_macros)) _cpp_warn_if_unused_macro (pfile, node, NULL); if (warn_of_redefinition (pfile, node, macro)) { cpp_error_with_line (pfile, CPP_DL_PEDWARN, pfile->directive_line, 0, "\"%s\" redefined", NODE_NAME (node)); if (node->type == NT_MACRO && !(node->flags & NODE_BUILTIN)) cpp_error_with_line (pfile, CPP_DL_PEDWARN, node->value.macro->line, 0, "this is the location of the previous definition"); } } if (node->type != NT_VOID) _cpp_free_definition (node); /* Enter definition in hash table. */ node->type = NT_MACRO; node->value.macro = macro; if (! ustrncmp (NODE_NAME (node), DSC ("__STDC_"))) node->flags |= NODE_WARN; return ok; } /* Warn if a token in STRING matches one of a function-like MACRO's parameters. */ static void check_trad_stringification (cpp_reader *pfile, const cpp_macro *macro, const cpp_string *string) { unsigned int i, len; const uchar *p, *q, *limit; /* Loop over the string. */ limit = string->text + string->len - 1; for (p = string->text + 1; p < limit; p = q) { /* Find the start of an identifier. */ while (p < limit && !is_idstart (*p)) p++; /* Find the end of the identifier. */ q = p; while (q < limit && is_idchar (*q)) q++; len = q - p; /* Loop over the function macro arguments to see if the identifier inside the string matches one of them. */ for (i = 0; i < macro->paramc; i++) { const cpp_hashnode *node = macro->params[i]; if (NODE_LEN (node) == len && !memcmp (p, NODE_NAME (node), len)) { cpp_error (pfile, CPP_DL_WARNING, "macro argument \"%s\" would be stringified in traditional C", NODE_NAME (node)); break; } } } } /* Returns the name, arguments and expansion of a macro, in a format suitable to be read back in again, and therefore also for DWARF 2 debugging info. e.g. "PASTE(X, Y) X ## Y", or "MACNAME EXPANSION". Caller is expected to generate the "#define" bit if needed. The returned text is temporary, and automatically freed later. */ const unsigned char * cpp_macro_definition (cpp_reader *pfile, const cpp_hashnode *node) { unsigned int i, len; const cpp_macro *macro = node->value.macro; unsigned char *buffer; if (node->type != NT_MACRO || (node->flags & NODE_BUILTIN)) { cpp_error (pfile, CPP_DL_ICE, "invalid hash type %d in cpp_macro_definition", node->type); return 0; } /* Calculate length. */ len = NODE_LEN (node) + 2; /* ' ' and NUL. */ if (macro->fun_like) { len += 4; /* "()" plus possible final ".." of named varargs (we have + 1 below). */ for (i = 0; i < macro->paramc; i++) len += NODE_LEN (macro->params[i]) + 1; /* "," */ } /* This should match below where we fill in the buffer. */ if (CPP_OPTION (pfile, traditional)) len += _cpp_replacement_text_len (macro); else { for (i = 0; i < macro->count; i++) { cpp_token *token = ¯o->exp.tokens[i]; if (token->type == CPP_MACRO_ARG) len += NODE_LEN (macro->params[token->val.arg_no - 1]); else len += cpp_token_len (token); if (token->flags & STRINGIFY_ARG) len++; /* "#" */ if (token->flags & PASTE_LEFT) len += 3; /* " ##" */ if (token->flags & PREV_WHITE) len++; /* " " */ } } if (len > pfile->macro_buffer_len) { pfile->macro_buffer = XRESIZEVEC (unsigned char, pfile->macro_buffer, len); pfile->macro_buffer_len = len; } /* Fill in the buffer. Start with the macro name. */ buffer = pfile->macro_buffer; memcpy (buffer, NODE_NAME (node), NODE_LEN (node)); buffer += NODE_LEN (node); /* Parameter names. */ if (macro->fun_like) { *buffer++ = '('; for (i = 0; i < macro->paramc; i++) { cpp_hashnode *param = macro->params[i]; if (param != pfile->spec_nodes.n__VA_ARGS__) { memcpy (buffer, NODE_NAME (param), NODE_LEN (param)); buffer += NODE_LEN (param); } if (i + 1 < macro->paramc) /* Don't emit a space after the comma here; we're trying to emit a Dwarf-friendly definition, and the Dwarf spec forbids spaces in the argument list. */ *buffer++ = ','; else if (macro->variadic) *buffer++ = '.', *buffer++ = '.', *buffer++ = '.'; } *buffer++ = ')'; } /* The Dwarf spec requires a space after the macro name, even if the definition is the empty string. */ *buffer++ = ' '; if (CPP_OPTION (pfile, traditional)) buffer = _cpp_copy_replacement_text (macro, buffer); else if (macro->count) /* Expansion tokens. */ { for (i = 0; i < macro->count; i++) { cpp_token *token = ¯o->exp.tokens[i]; if (token->flags & PREV_WHITE) *buffer++ = ' '; if (token->flags & STRINGIFY_ARG) *buffer++ = '#'; if (token->type == CPP_MACRO_ARG) { memcpy (buffer, NODE_NAME (macro->params[token->val.arg_no - 1]), NODE_LEN (macro->params[token->val.arg_no - 1])); buffer += NODE_LEN (macro->params[token->val.arg_no - 1]); } else buffer = cpp_spell_token (pfile, token, buffer, false); if (token->flags & PASTE_LEFT) { *buffer++ = ' '; *buffer++ = '#'; *buffer++ = '#'; /* Next has PREV_WHITE; see _cpp_create_definition. */ } } } *buffer = '\0'; return pfile->macro_buffer; } Index: projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/pch.c =================================================================== --- projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/pch.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/gcclibs/libcpp/pch.c (revision 305172) @@ -1,698 +1,739 @@ /* Part of CPP library. (Precompiled header reading/writing.) Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */ #include "config.h" #include "system.h" #include "cpplib.h" #include "internal.h" #include "hashtab.h" #include "mkdeps.h" static int write_macdef (cpp_reader *, cpp_hashnode *, void *); static int save_idents (cpp_reader *, cpp_hashnode *, void *); static hashval_t hashmem (const void *, size_t); static hashval_t cpp_string_hash (const void *); static int cpp_string_eq (const void *, const void *); static int count_defs (cpp_reader *, cpp_hashnode *, void *); static int comp_hashnodes (const void *, const void *); static int collect_ht_nodes (cpp_reader *, cpp_hashnode *, void *); static int write_defs (cpp_reader *, cpp_hashnode *, void *); static int save_macros (cpp_reader *, cpp_hashnode *, void *); /* This structure represents a macro definition on disk. */ struct macrodef_struct { unsigned int definition_length; unsigned short name_length; unsigned short flags; }; /* This is how we write out a macro definition. Suitable for being called by cpp_forall_identifiers. */ static int write_macdef (cpp_reader *pfile, cpp_hashnode *hn, void *file_p) { FILE *f = (FILE *) file_p; switch (hn->type) { case NT_VOID: if (! (hn->flags & NODE_POISONED)) return 1; case NT_MACRO: if ((hn->flags & NODE_BUILTIN)) return 1; { struct macrodef_struct s; const unsigned char *defn; s.name_length = NODE_LEN (hn); s.flags = hn->flags & NODE_POISONED; if (hn->type == NT_MACRO) { defn = cpp_macro_definition (pfile, hn); s.definition_length = ustrlen (defn); } else { defn = NODE_NAME (hn); s.definition_length = s.name_length; } if (fwrite (&s, sizeof (s), 1, f) != 1 || fwrite (defn, 1, s.definition_length, f) != s.definition_length) { cpp_errno (pfile, CPP_DL_ERROR, "while writing precompiled header"); return 0; } } return 1; case NT_ASSERTION: /* Not currently implemented. */ return 1; default: abort (); } } /* This structure records the names of the defined macros. It's also used as a callback structure for size_initial_idents and save_idents. */ struct cpp_savedstate { /* A hash table of the defined identifiers. */ htab_t definedhash; /* The size of the definitions of those identifiers (the size of 'definedstrs'). */ size_t hashsize; /* Number of definitions */ size_t n_defs; /* Array of definitions. In cpp_write_pch_deps it is used for sorting. */ cpp_hashnode **defs; /* Space for the next definition. Definitions are null-terminated strings. */ unsigned char *definedstrs; }; /* Save this identifier into the state: put it in the hash table, put the definition in 'definedstrs'. */ static int save_idents (cpp_reader *pfile ATTRIBUTE_UNUSED, cpp_hashnode *hn, void *ss_p) { struct cpp_savedstate *const ss = (struct cpp_savedstate *)ss_p; if (hn->type != NT_VOID) { struct cpp_string news; void **slot; news.len = NODE_LEN (hn); news.text= NODE_NAME (hn); slot = htab_find_slot (ss->definedhash, &news, INSERT); if (*slot == NULL) { struct cpp_string *sp; unsigned char *text; sp = XNEW (struct cpp_string); *slot = sp; sp->len = NODE_LEN (hn); sp->text = text = XNEWVEC (unsigned char, NODE_LEN (hn)); memcpy (text, NODE_NAME (hn), NODE_LEN (hn)); } } return 1; } /* Hash some memory in a generic way. */ static hashval_t hashmem (const void *p_p, size_t sz) { const unsigned char *p = (const unsigned char *)p_p; size_t i; hashval_t h; h = 0; for (i = 0; i < sz; i++) h = h * 67 - (*p++ - 113); return h; } /* Hash a cpp string for the hashtable machinery. */ static hashval_t cpp_string_hash (const void *a_p) { const struct cpp_string *a = (const struct cpp_string *) a_p; return hashmem (a->text, a->len); } /* Compare two cpp strings for the hashtable machinery. */ static int cpp_string_eq (const void *a_p, const void *b_p) { const struct cpp_string *a = (const struct cpp_string *) a_p; const struct cpp_string *b = (const struct cpp_string *) b_p; return (a->len == b->len && memcmp (a->text, b->text, a->len) == 0); } /* Save the current definitions of the cpp_reader for dependency checking purposes. When writing a precompiled header, this should be called at the same point in the compilation as cpp_valid_state would be called when reading the precompiled header back in. */ int cpp_save_state (cpp_reader *r, FILE *f) { /* Save the list of non-void identifiers for the dependency checking. */ r->savedstate = XNEW (struct cpp_savedstate); r->savedstate->definedhash = htab_create (100, cpp_string_hash, cpp_string_eq, NULL); cpp_forall_identifiers (r, save_idents, r->savedstate); /* Write out the list of defined identifiers. */ cpp_forall_identifiers (r, write_macdef, f); return 0; } /* Calculate the 'hashsize' field of the saved state. */ static int count_defs (cpp_reader *pfile ATTRIBUTE_UNUSED, cpp_hashnode *hn, void *ss_p) { struct cpp_savedstate *const ss = (struct cpp_savedstate *)ss_p; switch (hn->type) { case NT_MACRO: if (hn->flags & NODE_BUILTIN) return 1; /* else fall through. */ case NT_VOID: { struct cpp_string news; void **slot; news.len = NODE_LEN (hn); news.text = NODE_NAME (hn); slot = (void **) htab_find (ss->definedhash, &news); if (slot == NULL) { ss->hashsize += NODE_LEN (hn) + 1; ss->n_defs += 1; } } return 1; case NT_ASSERTION: /* Not currently implemented. */ return 1; default: abort (); } } /* Collect the identifiers into the state's string table. */ static int write_defs (cpp_reader *pfile ATTRIBUTE_UNUSED, cpp_hashnode *hn, void *ss_p) { struct cpp_savedstate *const ss = (struct cpp_savedstate *)ss_p; switch (hn->type) { case NT_MACRO: if (hn->flags & NODE_BUILTIN) return 1; /* else fall through. */ case NT_VOID: { struct cpp_string news; void **slot; news.len = NODE_LEN (hn); news.text = NODE_NAME (hn); slot = (void **) htab_find (ss->definedhash, &news); if (slot == NULL) { ss->defs[ss->n_defs] = hn; ss->n_defs += 1; } } return 1; case NT_ASSERTION: /* Not currently implemented. */ return 1; default: abort (); } } /* Comparison function for qsort. The arguments point to pointers of type ht_hashnode *. */ static int comp_hashnodes (const void *px, const void *py) { cpp_hashnode *x = *(cpp_hashnode **) px; cpp_hashnode *y = *(cpp_hashnode **) py; return ustrcmp (NODE_NAME (x), NODE_NAME (y)); } /* Write out the remainder of the dependency information. This should be called after the PCH is ready to be saved. */ int cpp_write_pch_deps (cpp_reader *r, FILE *f) { struct macrodef_struct z; struct cpp_savedstate *const ss = r->savedstate; unsigned char *definedstrs; size_t i; /* Collect the list of identifiers which have been seen and weren't defined to anything previously. */ ss->hashsize = 0; ss->n_defs = 0; cpp_forall_identifiers (r, count_defs, ss); ss->defs = XNEWVEC (cpp_hashnode *, ss->n_defs); ss->n_defs = 0; cpp_forall_identifiers (r, write_defs, ss); /* Sort the list, copy it into a buffer, and write it out. */ qsort (ss->defs, ss->n_defs, sizeof (cpp_hashnode *), &comp_hashnodes); definedstrs = ss->definedstrs = XNEWVEC (unsigned char, ss->hashsize); for (i = 0; i < ss->n_defs; ++i) { size_t len = NODE_LEN (ss->defs[i]); memcpy (definedstrs, NODE_NAME (ss->defs[i]), len + 1); definedstrs += len + 1; } memset (&z, 0, sizeof (z)); z.definition_length = ss->hashsize; if (fwrite (&z, sizeof (z), 1, f) != 1 || fwrite (ss->definedstrs, ss->hashsize, 1, f) != 1) { cpp_errno (r, CPP_DL_ERROR, "while writing precompiled header"); return -1; } free (ss->definedstrs); /* Free the saved state. */ free (ss); r->savedstate = NULL; + + /* Save the next value of __COUNTER__. */ + if (fwrite (&r->counter, sizeof (r->counter), 1, f) != 1) + { + cpp_errno (r, CPP_DL_ERROR, "while writing precompiled header"); + return -1; + } + return 0; } /* Write out the definitions of the preprocessor, in a form suitable for cpp_read_state. */ int cpp_write_pch_state (cpp_reader *r, FILE *f) { if (!r->deps) r->deps = deps_init (); if (deps_save (r->deps, f) != 0) { cpp_errno (r, CPP_DL_ERROR, "while writing precompiled header"); return -1; } if (! _cpp_save_file_entries (r, f)) { cpp_errno (r, CPP_DL_ERROR, "while writing precompiled header"); return -1; } + /* Save the next __COUNTER__ value. When we include a precompiled header, + we need to start at the offset we would have if the header had been + included normally. */ + if (fwrite (&r->counter, sizeof (r->counter), 1, f) != 1) + { + cpp_errno (r, CPP_DL_ERROR, "while writing precompiled header"); + return -1; + } + return 0; } /* Data structure to transform hash table nodes into a sorted list */ struct ht_node_list { /* Array of nodes */ cpp_hashnode **defs; /* Number of nodes in the array */ size_t n_defs; /* Size of the allocated array */ size_t asize; }; /* Callback for collecting identifiers from hash table */ static int collect_ht_nodes (cpp_reader *pfile ATTRIBUTE_UNUSED, cpp_hashnode *hn, void *nl_p) { struct ht_node_list *const nl = (struct ht_node_list *)nl_p; if (hn->type != NT_VOID || hn->flags & NODE_POISONED) { if (nl->n_defs == nl->asize) { nl->asize *= 2; nl->defs = XRESIZEVEC (cpp_hashnode *, nl->defs, nl->asize); } nl->defs[nl->n_defs] = hn; ++nl->n_defs; } return 1; } /* Return nonzero if FD is a precompiled header which is consistent with the preprocessor's current definitions. It will be consistent when: - anything that was defined just before the PCH was generated is defined the same way now; and - anything that was not defined then, but is defined now, was not used by the PCH. NAME is used to print warnings if `warn_invalid_pch' is set in the reader's flags. */ int cpp_valid_state (cpp_reader *r, const char *name, int fd) { struct macrodef_struct m; size_t namebufsz = 256; unsigned char *namebuf = XNEWVEC (unsigned char, namebufsz); unsigned char *undeftab = NULL; struct ht_node_list nl = { 0, 0, 0 }; unsigned char *first, *last; unsigned int i; + unsigned int counter; /* Read in the list of identifiers that must be defined Check that they are defined in the same way. */ for (;;) { cpp_hashnode *h; const unsigned char *newdefn; if (read (fd, &m, sizeof (m)) != sizeof (m)) goto error; if (m.name_length == 0) break; /* If this file is already preprocessed, there won't be any macros defined, and that's OK. */ if (CPP_OPTION (r, preprocessed)) { if (lseek (fd, m.definition_length, SEEK_CUR) == -1) goto error; continue; } if (m.definition_length > namebufsz) { free (namebuf); namebufsz = m.definition_length + 256; namebuf = XNEWVEC (unsigned char, namebufsz); } if ((size_t)read (fd, namebuf, m.definition_length) != m.definition_length) goto error; h = cpp_lookup (r, namebuf, m.name_length); if (m.flags & NODE_POISONED || h->type != NT_MACRO || h->flags & NODE_POISONED) { if (CPP_OPTION (r, warn_invalid_pch)) cpp_error (r, CPP_DL_WARNING_SYSHDR, "%s: not used because `%.*s' not defined", name, m.name_length, namebuf); goto fail; } newdefn = cpp_macro_definition (r, h); if (m.definition_length != ustrlen (newdefn) || memcmp (namebuf, newdefn, m.definition_length) != 0) { if (CPP_OPTION (r, warn_invalid_pch)) cpp_error (r, CPP_DL_WARNING_SYSHDR, "%s: not used because `%.*s' defined as `%s' not `%.*s'", name, m.name_length, namebuf, newdefn + m.name_length, m.definition_length - m.name_length, namebuf + m.name_length); goto fail; } } free (namebuf); namebuf = NULL; /* Read in the list of identifiers that must not be defined. Check that they really aren't. */ undeftab = XNEWVEC (unsigned char, m.definition_length); if ((size_t) read (fd, undeftab, m.definition_length) != m.definition_length) goto error; /* Collect identifiers from the current hash table. */ nl.n_defs = 0; nl.asize = 10; nl.defs = XNEWVEC (cpp_hashnode *, nl.asize); cpp_forall_identifiers (r, &collect_ht_nodes, &nl); qsort (nl.defs, nl.n_defs, sizeof (cpp_hashnode *), &comp_hashnodes); /* Loop through nl.defs and undeftab, both of which are sorted lists. There should be no matches. */ first = undeftab; last = undeftab + m.definition_length; i = 0; while (first < last && i < nl.n_defs) { int cmp = ustrcmp (first, NODE_NAME (nl.defs[i])); if (cmp < 0) first += ustrlen (first) + 1; else if (cmp > 0) ++i; else { if (CPP_OPTION (r, warn_invalid_pch)) cpp_error (r, CPP_DL_WARNING_SYSHDR, "%s: not used because `%s' is defined", name, first); goto fail; } } free(nl.defs); + nl.defs = NULL; free (undeftab); + undeftab = NULL; + /* Read in the next value of __COUNTER__. + Check that (a) __COUNTER__ was not used in the pch or (b) __COUNTER__ + has not been used in this translation unit. */ + if (read (fd, &counter, sizeof (counter)) != sizeof (counter)) + goto error; + if (counter && r->counter) + { + if (CPP_OPTION (r, warn_invalid_pch)) + cpp_error (r, CPP_DL_WARNING_SYSHDR, + "%s: not used because `__COUNTER__' is invalid", + name); + goto fail; + } + /* We win! */ return 0; error: cpp_errno (r, CPP_DL_ERROR, "while reading precompiled header"); return -1; fail: if (namebuf != NULL) free (namebuf); if (undeftab != NULL) free (undeftab); if (nl.defs != NULL) free (nl.defs); return 1; } /* Save all the existing macros. */ struct save_macro_data { uchar **defns; size_t count; size_t array_size; char **saved_pragmas; }; /* Save the definition of a single macro, so that it will persist across a PCH restore. Because macro data is in GCed memory, which will be blown away by PCH, it must be temporarily copied to malloced memory. (The macros will refer to identifier nodes which are also GCed and so on, so the copying is done by turning them into self-contained strings.) The assumption is that most macro definitions will come from the PCH file, not from the compilation before the PCH file is loaded, so it doesn't matter that this is a little expensive. It would reduce the cost even further if macros defined in the PCH file were not saved in this way, but this is not done (yet), except for builtins, and for #assert by default. */ static int save_macros (cpp_reader *r, cpp_hashnode *h, void *data_p) { struct save_macro_data *data = (struct save_macro_data *)data_p; if (h->type != NT_VOID && (h->flags & NODE_BUILTIN) == 0) { if (data->count == data->array_size) { data->array_size *= 2; data->defns = XRESIZEVEC (uchar *, data->defns, (data->array_size)); } switch (h->type) { case NT_ASSERTION: /* Not currently implemented. */ return 1; case NT_MACRO: { const uchar * defn = cpp_macro_definition (r, h); size_t defnlen = ustrlen (defn); data->defns[data->count] = (uchar *) xmemdup (defn, defnlen, defnlen + 2); data->defns[data->count][defnlen] = '\n'; } break; default: abort (); } data->count++; } return 1; } /* Prepare to restore the state, by saving the currently-defined macros in 'data'. */ void cpp_prepare_state (cpp_reader *r, struct save_macro_data **data) { struct save_macro_data *d = XNEW (struct save_macro_data); d->array_size = 512; d->defns = XNEWVEC (uchar *, d->array_size); d->count = 0; cpp_forall_identifiers (r, save_macros, d); d->saved_pragmas = _cpp_save_pragma_names (r); *data = d; } /* Given a precompiled header that was previously determined to be valid, apply all its definitions (and undefinitions) to the current state. DEPNAME is passed to deps_restore. */ int cpp_read_state (cpp_reader *r, const char *name, FILE *f, struct save_macro_data *data) { size_t i; struct lexer_state old_state; + unsigned int counter; /* Restore spec_nodes, which will be full of references to the old hashtable entries and so will now be invalid. */ { struct spec_nodes *s = &r->spec_nodes; s->n_defined = cpp_lookup (r, DSC("defined")); s->n_true = cpp_lookup (r, DSC("true")); s->n_false = cpp_lookup (r, DSC("false")); s->n__VA_ARGS__ = cpp_lookup (r, DSC("__VA_ARGS__")); } old_state = r->state; r->state.in_directive = 1; r->state.prevent_expansion = 1; r->state.angled_headers = 0; /* Run through the carefully-saved macros, insert them. */ for (i = 0; i < data->count; i++) { cpp_hashnode *h; size_t namelen; uchar *defn; namelen = ustrcspn (data->defns[i], "( \n"); h = cpp_lookup (r, data->defns[i], namelen); defn = data->defns[i] + namelen; /* The PCH file is valid, so we know that if there is a definition from the PCH file it must be the same as the one we had originally, and so do not need to restore it. */ if (h->type == NT_VOID) { if (cpp_push_buffer (r, defn, ustrchr (defn, '\n') - defn, true) != NULL) { _cpp_clean_line (r); if (!_cpp_create_definition (r, h)) abort (); _cpp_pop_buffer (r); } else abort (); } free (data->defns[i]); } r->state = old_state; _cpp_restore_pragma_names (r, data->saved_pragmas); free (data); if (deps_restore (r->deps, f, CPP_OPTION (r, restore_pch_deps) ? name : NULL) != 0) goto error; if (! _cpp_read_file_entries (r, f)) goto error; + + if (fread (&counter, sizeof (counter), 1, f) != 1) + goto error; + + if (!r->counter) + r->counter = counter; return 0; error: cpp_errno (r, CPP_DL_ERROR, "while reading precompiled header"); return -1; } Index: projects/netbsd-tests-update-12/contrib/telnet/libtelnet/pk.c =================================================================== --- projects/netbsd-tests-update-12/contrib/telnet/libtelnet/pk.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/telnet/libtelnet/pk.c (revision 305172) @@ -1,265 +1,265 @@ /*- * Copyright (c) 1991, 1993 * Dave Safford. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #include __FBSDID("$FreeBSD$"); /* public key routines */ /* functions: genkeys(char *public, char *secret) common_key(char *secret, char *public, desData *deskey) pk_encode(char *in, *out, DesData *deskey); pk_decode(char *in, *out, DesData *deskey); where char public[HEXKEYBYTES + 1]; char secret[HEXKEYBYTES + 1]; */ #include #include #include #include #include #include #include "mp.h" #include "pk.h" static void adjust(char keyout[HEXKEYBYTES+1], char *keyin); /* * Choose top 128 bits of the common key to use as our idea key. */ static void extractideakey(MINT *ck, IdeaData *ideakey) { MINT *a; MINT *z; short r; int i; short base = (1 << 8); char *k; z = mp_itom(0); a = mp_itom(0); mp_madd(ck, z, a); for (i = 0; i < ((KEYSIZE - 128) / 8); i++) { mp_sdiv(a, base, a, &r); } k = (char *)ideakey; for (i = 0; i < 16; i++) { mp_sdiv(a, base, a, &r); *k++ = r; } mp_mfree(z); mp_mfree(a); } /* * Choose middle 64 bits of the common key to use as our des key, possibly * overwriting the lower order bits by setting parity. */ static void extractdeskey(MINT *ck, DesData *deskey) { MINT *a; MINT *z; short r; int i; short base = (1 << 8); char *k; z = mp_itom(0); a = mp_itom(0); mp_madd(ck, z, a); for (i = 0; i < ((KEYSIZE - 64) / 2) / 8; i++) { mp_sdiv(a, base, a, &r); } k = (char *)deskey; for (i = 0; i < 8; i++) { mp_sdiv(a, base, a, &r); *k++ = r; } mp_mfree(z); mp_mfree(a); } /* * get common key from my secret key and his public key */ void common_key(char *xsecret, char *xpublic, IdeaData *ideakey, DesData *deskey) { MINT *public; MINT *secret; MINT *common; MINT *modulus = mp_xtom(HEXMODULUS); public = mp_xtom(xpublic); secret = mp_xtom(xsecret); common = mp_itom(0); mp_pow(public, secret, modulus, common); extractdeskey(common, deskey); extractideakey(common, ideakey); des_set_odd_parity(deskey); mp_mfree(common); mp_mfree(secret); mp_mfree(public); mp_mfree(modulus); } /* * Generate a seed */ static void getseed(char *seed, int seedsize) { int i; srandomdev(); for (i = 0; i < seedsize; i++) { seed[i] = random() & 0xff; } } /* * Generate a random public/secret key pair */ void genkeys(char *public, char *secret) { size_t i; # define BASEBITS (8*sizeof(short) - 1) # define BASE (1 << BASEBITS) MINT *pk = mp_itom(0); MINT *sk = mp_itom(0); MINT *tmp; - MINT *base = mp_itom(BASE); + MINT *base = mp_itom((short)BASE); MINT *root = mp_itom(PROOT); MINT *modulus = mp_xtom(HEXMODULUS); short r; unsigned short seed[KEYSIZE/BASEBITS + 1]; char *xkey; getseed((char *)seed, sizeof(seed)); for (i = 0; i < KEYSIZE/BASEBITS + 1; i++) { r = seed[i] % BASE; tmp = mp_itom(r); mp_mult(sk, base, sk); mp_madd(sk, tmp, sk); mp_mfree(tmp); } tmp = mp_itom(0); mp_mdiv(sk, modulus, tmp, sk); mp_mfree(tmp); mp_pow(root, sk, modulus, pk); xkey = mp_mtox(sk); adjust(secret, xkey); xkey = mp_mtox(pk); adjust(public, xkey); mp_mfree(sk); mp_mfree(base); mp_mfree(pk); mp_mfree(root); mp_mfree(modulus); } /* * Adjust the input key so that it is 0-filled on the left */ static void adjust(char keyout[HEXKEYBYTES+1], char *keyin) { char *p; char *s; for (p = keyin; *p; p++) ; for (s = keyout + HEXKEYBYTES; p >= keyin; p--, s--) { *s = *p; } while (s >= keyout) { *s-- = '0'; } } static char hextab[17] = "0123456789ABCDEF"; /* given a DES key, cbc encrypt and translate input to terminated hex */ void pk_encode(char *in, char *out, DesData *key) { char buf[256]; DesData i; des_key_schedule k; int l,op,deslen; memset(&i,0,sizeof(i)); memset(buf,0,sizeof(buf)); deslen = ((strlen(in) + 7)/8)*8; des_key_sched(key, k); des_cbc_encrypt(in,buf,deslen, k,&i,DES_ENCRYPT); for (l=0,op=0;l> 4]; out[op++] = hextab[(buf[l] & 0x0f)]; } out[op] = '\0'; } /* given a DES key, translate input from hex and decrypt */ void pk_decode(char *in, char *out, DesData *key) { char buf[256]; DesData i; des_key_schedule k; int n1,n2,op; size_t l; memset(&i,0,sizeof(i)); memset(buf,0,sizeof(buf)); for (l=0,op=0;l '9') n1 = in[op] - 'A' + 10; else n1 = in[op] - '0'; if (in[op+1] > '9') n2 = in[op+1] - 'A' + 10; else n2 = in[op+1] - '0'; buf[l] = n1*16 +n2; } des_key_sched(key, k); des_cbc_encrypt(buf,out,strlen(in)/2, k,&i,DES_DECRYPT); out[strlen(in)/2] = '\0'; } Index: projects/netbsd-tests-update-12/contrib/telnet/telnet/commands.c =================================================================== --- projects/netbsd-tests-update-12/contrib/telnet/telnet/commands.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/telnet/telnet/commands.c (revision 305172) @@ -1,3012 +1,3012 @@ /* * Copyright (c) 1988, 1990, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by the University of * California, Berkeley and its contributors. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #if 0 #ifndef lint static const char sccsid[] = "@(#)commands.c 8.4 (Berkeley) 5/30/95"; #endif #endif #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "general.h" #include "ring.h" #include "externs.h" #include "defines.h" #include "types.h" #include "misc.h" #ifdef AUTHENTICATION #include #endif #ifdef ENCRYPTION #include #endif #include #include #include #ifndef MAXHOSTNAMELEN #define MAXHOSTNAMELEN 256 #endif typedef int (*intrtn_t)(int, char **); #ifdef AUTHENTICATION extern int auth_togdebug(int); #endif #ifdef ENCRYPTION extern int EncryptAutoEnc(int); extern int EncryptAutoDec(int); extern int EncryptDebug(int); extern int EncryptVerbose(int); #endif /* ENCRYPTION */ #if defined(IPPROTO_IP) && defined(IP_TOS) int tos = -1; #endif /* defined(IPPROTO_IP) && defined(IP_TOS) */ char *hostname; static char _hostname[MAXHOSTNAMELEN]; static int help(int, char **); static int call(intrtn_t, ...); static void cmdrc(char *, char *); #ifdef INET6 static int switch_af(struct addrinfo **); #endif static int togglehelp(void); static int send_tncmd(void (*)(int, int), const char *, char *); static int setmod(int); static int clearmode(int); static int modehelp(void); -static int sourceroute(struct addrinfo *, char *, char **, int *, int *, int *); +static int sourceroute(struct addrinfo *, char *, unsigned char **, int *, int *, int *); typedef struct { const char *name; /* command name */ const char *help; /* help string (NULL for no help) */ int (*handler)(int, char **); /* routine which executes command */ int needconnect; /* Do we need to be connected to execute? */ } Command; static char line[256]; static char saveline[256]; static int margc; static char *margv[20]; #ifdef OPIE #include #define PATH_OPIEKEY "/usr/bin/opiekey" static int opie_calc(int argc, char *argv[]) { int status; if(argc != 3) { printf("%s sequence challenge\n", argv[0]); return (0); } switch(fork()) { case 0: execv(PATH_OPIEKEY, argv); exit (1); case -1: perror("fork"); break; default: (void) wait(&status); if (WIFEXITED(status)) return (WEXITSTATUS(status)); } return (0); } #endif static void makeargv(void) { char *cp, *cp2, c; char **argp = margv; margc = 0; cp = line; if (*cp == '!') { /* Special case shell escape */ strcpy(saveline, line); /* save for shell command */ *argp++ = strdup("!"); /* No room in string to get this */ margc++; cp++; } while ((c = *cp)) { int inquote = 0; while (isspace(c)) c = *++cp; if (c == '\0') break; *argp++ = cp; margc += 1; for (cp2 = cp; c != '\0'; c = *++cp) { if (inquote) { if (c == inquote) { inquote = 0; continue; } } else { if (c == '\\') { if ((c = *++cp) == '\0') break; } else if (c == '"') { inquote = '"'; continue; } else if (c == '\'') { inquote = '\''; continue; } else if (isspace(c)) break; } *cp2++ = c; } *cp2 = '\0'; if (c == '\0') break; cp++; } *argp++ = 0; } /* * Make a character string into a number. * * Todo: 1. Could take random integers (12, 0x12, 012, 0b1). */ static int special(char *s) { char c; char b; switch (*s) { case '^': b = *++s; if (b == '?') { c = b | 0x40; /* DEL */ } else { c = b & 0x1f; } break; default: c = *s; break; } return c; } /* * Construct a control character sequence * for a special character. */ static const char * control(cc_t c) { static char buf[5]; /* * The only way I could get the Sun 3.5 compiler * to shut up about * if ((unsigned int)c >= 0x80) * was to assign "c" to an unsigned int variable... * Arggg.... */ unsigned int uic = (unsigned int)c; if (uic == 0x7f) return ("^?"); if (c == (cc_t)_POSIX_VDISABLE) { return "off"; } if (uic >= 0x80) { buf[0] = '\\'; buf[1] = ((c>>6)&07) + '0'; buf[2] = ((c>>3)&07) + '0'; buf[3] = (c&07) + '0'; buf[4] = 0; } else if (uic >= 0x20) { buf[0] = c; buf[1] = 0; } else { buf[0] = '^'; buf[1] = '@'+c; buf[2] = 0; } return (buf); } /* * The following are data structures and routines for * the "send" command. * */ struct sendlist { const char *name; /* How user refers to it (case independent) */ const char *help; /* Help information (0 ==> no help) */ int needconnect; /* Need to be connected */ int narg; /* Number of arguments */ int (*handler)(char *, ...); /* Routine to perform (for special ops) */ int nbyte; /* Number of bytes to send this command */ int what; /* Character to be sent (<0 ==> special) */ }; static int send_esc(void), send_help(void), send_docmd(char *), send_dontcmd(char *), send_willcmd(char *), send_wontcmd(char *); static struct sendlist Sendlist[] = { { "ao", "Send Telnet Abort output", 1, 0, NULL, 2, AO }, { "ayt", "Send Telnet 'Are You There'", 1, 0, NULL, 2, AYT }, { "brk", "Send Telnet Break", 1, 0, NULL, 2, BREAK }, { "break", NULL, 1, 0, NULL, 2, BREAK }, { "ec", "Send Telnet Erase Character", 1, 0, NULL, 2, EC }, { "el", "Send Telnet Erase Line", 1, 0, NULL, 2, EL }, { "escape", "Send current escape character",1, 0, (int (*)(char *, ...))send_esc, 1, 0 }, { "ga", "Send Telnet 'Go Ahead' sequence", 1, 0, NULL, 2, GA }, { "ip", "Send Telnet Interrupt Process",1, 0, NULL, 2, IP }, { "intp", NULL, 1, 0, NULL, 2, IP }, { "interrupt", NULL, 1, 0, NULL, 2, IP }, { "intr", NULL, 1, 0, NULL, 2, IP }, { "nop", "Send Telnet 'No operation'", 1, 0, NULL, 2, NOP }, { "eor", "Send Telnet 'End of Record'", 1, 0, NULL, 2, EOR }, { "abort", "Send Telnet 'Abort Process'", 1, 0, NULL, 2, ABORT }, { "susp", "Send Telnet 'Suspend Process'",1, 0, NULL, 2, SUSP }, { "eof", "Send Telnet End of File Character", 1, 0, NULL, 2, xEOF }, { "synch", "Perform Telnet 'Synch operation'", 1, 0, (int (*)(char *, ...))dosynch, 2, 0 }, { "getstatus", "Send request for STATUS", 1, 0, (int (*)(char *, ...))get_status, 6, 0 }, { "?", "Display send options", 0, 0, (int (*)(char *, ...))send_help, 0, 0 }, { "help", NULL, 0, 0, (int (*)(char *, ...))send_help, 0, 0 }, { "do", NULL, 0, 1, (int (*)(char *, ...))send_docmd, 3, 0 }, { "dont", NULL, 0, 1, (int (*)(char *, ...))send_dontcmd, 3, 0 }, { "will", NULL, 0, 1, (int (*)(char *, ...))send_willcmd, 3, 0 }, { "wont", NULL, 0, 1, (int (*)(char *, ...))send_wontcmd, 3, 0 }, { NULL, NULL, 0, 0, NULL, 0, 0 } }; #define GETSEND(name) ((struct sendlist *) genget(name, (char **) Sendlist, \ sizeof(struct sendlist))) static int sendcmd(int argc, char *argv[]) { int count; /* how many bytes we are going to need to send */ int i; struct sendlist *s; /* pointer to current command */ int success = 0; int needconnect = 0; if (argc < 2) { printf("need at least one argument for 'send' command\n"); printf("'send ?' for help\n"); return 0; } /* * First, validate all the send arguments. * In addition, we see how much space we are going to need, and * whether or not we will be doing a "SYNCH" operation (which * flushes the network queue). */ count = 0; for (i = 1; i < argc; i++) { s = GETSEND(argv[i]); if (s == 0) { printf("Unknown send argument '%s'\n'send ?' for help.\n", argv[i]); return 0; } else if (Ambiguous((void *)s)) { printf("Ambiguous send argument '%s'\n'send ?' for help.\n", argv[i]); return 0; } if (i + s->narg >= argc) { fprintf(stderr, "Need %d argument%s to 'send %s' command. 'send %s ?' for help.\n", s->narg, s->narg == 1 ? "" : "s", s->name, s->name); return 0; } count += s->nbyte; if ((void *)s->handler == (void *)send_help) { send_help(); return 0; } i += s->narg; needconnect += s->needconnect; } if (!connected && needconnect) { printf("?Need to be connected first.\n"); printf("'send ?' for help\n"); return 0; } /* Now, do we have enough room? */ if (NETROOM() < count) { printf("There is not enough room in the buffer TO the network\n"); printf("to process your request. Nothing will be done.\n"); printf("('send synch' will throw away most data in the network\n"); printf("buffer, if this might help.)\n"); return 0; } /* OK, they are all OK, now go through again and actually send */ count = 0; for (i = 1; i < argc; i++) { if ((s = GETSEND(argv[i])) == 0) { fprintf(stderr, "Telnet 'send' error - argument disappeared!\n"); quit(); /*NOTREACHED*/ } if (s->handler) { count++; success += (*s->handler)((s->narg > 0) ? argv[i+1] : 0, (s->narg > 1) ? argv[i+2] : 0); i += s->narg; } else { NET2ADD(IAC, s->what); printoption("SENT", IAC, s->what); } } return (count == success); } static int send_esc(void) { NETADD(escape); return 1; } static int send_docmd(char *name) { return(send_tncmd(send_do, "do", name)); } static int send_dontcmd(name) char *name; { return(send_tncmd(send_dont, "dont", name)); } static int send_willcmd(char *name) { return(send_tncmd(send_will, "will", name)); } static int send_wontcmd(char *name) { return(send_tncmd(send_wont, "wont", name)); } static int send_tncmd(void (*func)(int, int), const char *cmd, char *name) { char **cpp; extern char *telopts[]; int val = 0; if (isprefix(name, "help") || isprefix(name, "?")) { int col, len; printf("usage: send %s \n", cmd); printf("\"value\" must be from 0 to 255\n"); printf("Valid options are:\n\t"); col = 8; for (cpp = telopts; *cpp; cpp++) { len = strlen(*cpp) + 3; if (col + len > 65) { printf("\n\t"); col = 8; } printf(" \"%s\"", *cpp); col += len; } printf("\n"); return 0; } cpp = (char **)genget(name, telopts, sizeof(char *)); if (Ambiguous(cpp)) { fprintf(stderr,"'%s': ambiguous argument ('send %s ?' for help).\n", name, cmd); return 0; } if (cpp) { val = cpp - telopts; } else { char *cp = name; while (*cp >= '0' && *cp <= '9') { val *= 10; val += *cp - '0'; cp++; } if (*cp != 0) { fprintf(stderr, "'%s': unknown argument ('send %s ?' for help).\n", name, cmd); return 0; } else if (val < 0 || val > 255) { fprintf(stderr, "'%s': bad value ('send %s ?' for help).\n", name, cmd); return 0; } } if (!connected) { printf("?Need to be connected first.\n"); return 0; } (*func)(val, 1); return 1; } static int send_help(void) { struct sendlist *s; /* pointer to current command */ for (s = Sendlist; s->name; s++) { if (s->help) printf("%-15s %s\n", s->name, s->help); } return(0); } /* * The following are the routines and data structures referred * to by the arguments to the "toggle" command. */ static int lclchars(void) { donelclchars = 1; return 1; } static int togdebug(void) { #ifndef NOT43 if (net > 0 && (SetSockOpt(net, SOL_SOCKET, SO_DEBUG, telnet_debug)) < 0) { perror("setsockopt (SO_DEBUG)"); } #else /* NOT43 */ if (telnet_debug) { if (net > 0 && SetSockOpt(net, SOL_SOCKET, SO_DEBUG, 1) < 0) perror("setsockopt (SO_DEBUG)"); } else printf("Cannot turn off socket debugging\n"); #endif /* NOT43 */ return 1; } static int togcrlf(void) { if (crlf) { printf("Will send carriage returns as telnet .\n"); } else { printf("Will send carriage returns as telnet .\n"); } return 1; } int binmode; static int togbinary(int val) { donebinarytoggle = 1; if (val >= 0) { binmode = val; } else { if (my_want_state_is_will(TELOPT_BINARY) && my_want_state_is_do(TELOPT_BINARY)) { binmode = 1; } else if (my_want_state_is_wont(TELOPT_BINARY) && my_want_state_is_dont(TELOPT_BINARY)) { binmode = 0; } val = binmode ? 0 : 1; } if (val == 1) { if (my_want_state_is_will(TELOPT_BINARY) && my_want_state_is_do(TELOPT_BINARY)) { printf("Already operating in binary mode with remote host.\n"); } else { printf("Negotiating binary mode with remote host.\n"); tel_enter_binary(3); } } else { if (my_want_state_is_wont(TELOPT_BINARY) && my_want_state_is_dont(TELOPT_BINARY)) { printf("Already in network ascii mode with remote host.\n"); } else { printf("Negotiating network ascii mode with remote host.\n"); tel_leave_binary(3); } } return 1; } static int togrbinary(int val) { donebinarytoggle = 1; if (val == -1) val = my_want_state_is_do(TELOPT_BINARY) ? 0 : 1; if (val == 1) { if (my_want_state_is_do(TELOPT_BINARY)) { printf("Already receiving in binary mode.\n"); } else { printf("Negotiating binary mode on input.\n"); tel_enter_binary(1); } } else { if (my_want_state_is_dont(TELOPT_BINARY)) { printf("Already receiving in network ascii mode.\n"); } else { printf("Negotiating network ascii mode on input.\n"); tel_leave_binary(1); } } return 1; } static int togxbinary(int val) { donebinarytoggle = 1; if (val == -1) val = my_want_state_is_will(TELOPT_BINARY) ? 0 : 1; if (val == 1) { if (my_want_state_is_will(TELOPT_BINARY)) { printf("Already transmitting in binary mode.\n"); } else { printf("Negotiating binary mode on output.\n"); tel_enter_binary(2); } } else { if (my_want_state_is_wont(TELOPT_BINARY)) { printf("Already transmitting in network ascii mode.\n"); } else { printf("Negotiating network ascii mode on output.\n"); tel_leave_binary(2); } } return 1; } struct togglelist { const char *name; /* name of toggle */ const char *help; /* help message */ int (*handler)(int); /* routine to do actual setting */ int *variable; const char *actionexplanation; }; static struct togglelist Togglelist[] = { { "autoflush", "flushing of output when sending interrupt characters", 0, &autoflush, "flush output when sending interrupt characters" }, { "autosynch", "automatic sending of interrupt characters in urgent mode", 0, &autosynch, "send interrupt characters in urgent mode" }, #ifdef AUTHENTICATION { "autologin", "automatic sending of login and/or authentication info", 0, &autologin, "send login name and/or authentication information" }, { "authdebug", "Toggle authentication debugging", auth_togdebug, 0, "print authentication debugging information" }, #endif #ifdef ENCRYPTION { "autoencrypt", "automatic encryption of data stream", EncryptAutoEnc, 0, "automatically encrypt output" }, { "autodecrypt", "automatic decryption of data stream", EncryptAutoDec, 0, "automatically decrypt input" }, { "verbose_encrypt", "Toggle verbose encryption output", EncryptVerbose, 0, "print verbose encryption output" }, { "encdebug", "Toggle encryption debugging", EncryptDebug, 0, "print encryption debugging information" }, #endif /* ENCRYPTION */ { "skiprc", "don't read ~/.telnetrc file", 0, &skiprc, "skip reading of ~/.telnetrc file" }, { "binary", "sending and receiving of binary data", togbinary, 0, 0 }, { "inbinary", "receiving of binary data", togrbinary, 0, 0 }, { "outbinary", "sending of binary data", togxbinary, 0, 0 }, { "crlf", "sending carriage returns as telnet ", (int (*)(int))togcrlf, &crlf, 0 }, { "crmod", "mapping of received carriage returns", 0, &crmod, "map carriage return on output" }, { "localchars", "local recognition of certain control characters", (int (*)(int))lclchars, &localchars, "recognize certain control characters" }, { " ", "", NULL, NULL, NULL }, /* empty line */ { "debug", "debugging", (int (*)(int))togdebug, &telnet_debug, "turn on socket level debugging" }, { "netdata", "printing of hexadecimal network data (debugging)", 0, &netdata, "print hexadecimal representation of network traffic" }, { "prettydump", "output of \"netdata\" to user readable format (debugging)", 0, &prettydump, "print user readable output for \"netdata\"" }, { "options", "viewing of options processing (debugging)", 0, &showoptions, "show option processing" }, { "termdata", "(debugging) toggle printing of hexadecimal terminal data", 0, &termdata, "print hexadecimal representation of terminal traffic" }, { "?", NULL, (int (*)(int))togglehelp, NULL, NULL }, { NULL, NULL, NULL, NULL, NULL }, { "help", NULL, (int (*)(int))togglehelp, NULL, NULL }, { NULL, NULL, NULL, NULL, NULL } }; static int togglehelp(void) { struct togglelist *c; for (c = Togglelist; c->name; c++) { if (c->help) { if (*c->help) printf("%-15s toggle %s\n", c->name, c->help); else printf("\n"); } } printf("\n"); printf("%-15s %s\n", "?", "display help information"); return 0; } static void settogglehelp(int set) { struct togglelist *c; for (c = Togglelist; c->name; c++) { if (c->help) { if (*c->help) printf("%-15s %s %s\n", c->name, set ? "enable" : "disable", c->help); else printf("\n"); } } } #define GETTOGGLE(name) (struct togglelist *) \ genget(name, (char **) Togglelist, sizeof(struct togglelist)) static int toggle(int argc, char *argv[]) { int retval = 1; char *name; struct togglelist *c; if (argc < 2) { fprintf(stderr, "Need an argument to 'toggle' command. 'toggle ?' for help.\n"); return 0; } argc--; argv++; while (argc--) { name = *argv++; c = GETTOGGLE(name); if (Ambiguous((void *)c)) { fprintf(stderr, "'%s': ambiguous argument ('toggle ?' for help).\n", name); return 0; } else if (c == 0) { fprintf(stderr, "'%s': unknown argument ('toggle ?' for help).\n", name); return 0; } else { if (c->variable) { *c->variable = !*c->variable; /* invert it */ if (c->actionexplanation) { printf("%s %s.\n", *c->variable? "Will" : "Won't", c->actionexplanation); } } if (c->handler) { retval &= (*c->handler)(-1); } } } return retval; } /* * The following perform the "set" command. */ #ifdef USE_TERMIO struct termio new_tc = { 0, 0, 0, 0, {}, 0, 0 }; #endif struct setlist { const char *name; /* name */ const char *help; /* help information */ void (*handler)(char *); cc_t *charp; /* where it is located at */ }; static struct setlist Setlist[] = { #ifdef KLUDGELINEMODE { "echo", "character to toggle local echoing on/off", NULL, &echoc }, #endif { "escape", "character to escape back to telnet command mode", NULL, &escape }, { "rlogin", "rlogin escape character", 0, &rlogin }, { "tracefile", "file to write trace information to", SetNetTrace, (cc_t *)NetTraceFile}, { " ", "", NULL, NULL }, { " ", "The following need 'localchars' to be toggled true", NULL, NULL }, { "flushoutput", "character to cause an Abort Output", NULL, termFlushCharp }, { "interrupt", "character to cause an Interrupt Process", NULL, termIntCharp }, { "quit", "character to cause an Abort process", NULL, termQuitCharp }, { "eof", "character to cause an EOF ", NULL, termEofCharp }, { " ", "", NULL, NULL }, { " ", "The following are for local editing in linemode", NULL, NULL }, { "erase", "character to use to erase a character", NULL, termEraseCharp }, { "kill", "character to use to erase a line", NULL, termKillCharp }, { "lnext", "character to use for literal next", NULL, termLiteralNextCharp }, { "susp", "character to cause a Suspend Process", NULL, termSuspCharp }, { "reprint", "character to use for line reprint", NULL, termRprntCharp }, { "worderase", "character to use to erase a word", NULL, termWerasCharp }, { "start", "character to use for XON", NULL, termStartCharp }, { "stop", "character to use for XOFF", NULL, termStopCharp }, { "forw1", "alternate end of line character", NULL, termForw1Charp }, { "forw2", "alternate end of line character", NULL, termForw2Charp }, { "ayt", "alternate AYT character", NULL, termAytCharp }, { "baudrate", "set remote baud rate", DoBaudRate, ComPortBaudRate }, { NULL, NULL, NULL, NULL } }; static struct setlist * getset(char *name) { return (struct setlist *) genget(name, (char **) Setlist, sizeof(struct setlist)); } void set_escape_char(char *s) { if (rlogin != _POSIX_VDISABLE) { rlogin = (s && *s) ? special(s) : _POSIX_VDISABLE; printf("Telnet rlogin escape character is '%s'.\n", control(rlogin)); } else { escape = (s && *s) ? special(s) : _POSIX_VDISABLE; printf("Telnet escape character is '%s'.\n", control(escape)); } } static int setcmd(int argc, char *argv[]) { int value; struct setlist *ct; struct togglelist *c; if (argc < 2 || argc > 3) { printf("Format is 'set Name Value'\n'set ?' for help.\n"); return 0; } if ((argc == 2) && (isprefix(argv[1], "?") || isprefix(argv[1], "help"))) { for (ct = Setlist; ct->name; ct++) printf("%-15s %s\n", ct->name, ct->help); printf("\n"); settogglehelp(1); printf("%-15s %s\n", "?", "display help information"); return 0; } ct = getset(argv[1]); if (ct == 0) { c = GETTOGGLE(argv[1]); if (c == 0) { fprintf(stderr, "'%s': unknown argument ('set ?' for help).\n", argv[1]); return 0; } else if (Ambiguous((void *)c)) { fprintf(stderr, "'%s': ambiguous argument ('set ?' for help).\n", argv[1]); return 0; } if (c->variable) { if ((argc == 2) || (strcmp("on", argv[2]) == 0)) *c->variable = 1; else if (strcmp("off", argv[2]) == 0) *c->variable = 0; else { printf("Format is 'set togglename [on|off]'\n'set ?' for help.\n"); return 0; } if (c->actionexplanation) { printf("%s %s.\n", *c->variable? "Will" : "Won't", c->actionexplanation); } } if (c->handler) (*c->handler)(1); } else if (argc != 3) { printf("Format is 'set Name Value'\n'set ?' for help.\n"); return 0; } else if (Ambiguous((void *)ct)) { fprintf(stderr, "'%s': ambiguous argument ('set ?' for help).\n", argv[1]); return 0; } else if (ct->handler) { (*ct->handler)(argv[2]); printf("%s set to \"%s\".\n", ct->name, (char *)ct->charp); } else { if (strcmp("off", argv[2])) { value = special(argv[2]); } else { value = _POSIX_VDISABLE; } *(ct->charp) = (cc_t)value; printf("%s character is '%s'.\n", ct->name, control(*(ct->charp))); } slc_check(); return 1; } static int unsetcmd(int argc, char *argv[]) { struct setlist *ct; struct togglelist *c; char *name; if (argc < 2) { fprintf(stderr, "Need an argument to 'unset' command. 'unset ?' for help.\n"); return 0; } if (isprefix(argv[1], "?") || isprefix(argv[1], "help")) { for (ct = Setlist; ct->name; ct++) printf("%-15s %s\n", ct->name, ct->help); printf("\n"); settogglehelp(0); printf("%-15s %s\n", "?", "display help information"); return 0; } argc--; argv++; while (argc--) { name = *argv++; ct = getset(name); if (ct == 0) { c = GETTOGGLE(name); if (c == 0) { fprintf(stderr, "'%s': unknown argument ('unset ?' for help).\n", name); return 0; } else if (Ambiguous((void *)c)) { fprintf(stderr, "'%s': ambiguous argument ('unset ?' for help).\n", name); return 0; } if (c->variable) { *c->variable = 0; if (c->actionexplanation) { printf("%s %s.\n", *c->variable? "Will" : "Won't", c->actionexplanation); } } if (c->handler) (*c->handler)(0); } else if (Ambiguous((void *)ct)) { fprintf(stderr, "'%s': ambiguous argument ('unset ?' for help).\n", name); return 0; } else if (ct->handler) { (*ct->handler)(0); printf("%s reset to \"%s\".\n", ct->name, (char *)ct->charp); } else { *(ct->charp) = _POSIX_VDISABLE; printf("%s character is '%s'.\n", ct->name, control(*(ct->charp))); } } return 1; } /* * The following are the data structures and routines for the * 'mode' command. */ #ifdef KLUDGELINEMODE extern int kludgelinemode; static int dokludgemode(void) { kludgelinemode = 1; send_wont(TELOPT_LINEMODE, 1); send_dont(TELOPT_SGA, 1); send_dont(TELOPT_ECHO, 1); return 1; } #endif static int dolinemode(void) { #ifdef KLUDGELINEMODE if (kludgelinemode) send_dont(TELOPT_SGA, 1); #endif send_will(TELOPT_LINEMODE, 1); send_dont(TELOPT_ECHO, 1); return 1; } static int docharmode(void) { #ifdef KLUDGELINEMODE if (kludgelinemode) send_do(TELOPT_SGA, 1); else #endif send_wont(TELOPT_LINEMODE, 1); send_do(TELOPT_ECHO, 1); return 1; } static int dolmmode(int bit, int on) { unsigned char c; extern int linemode; if (my_want_state_is_wont(TELOPT_LINEMODE)) { printf("?Need to have LINEMODE option enabled first.\n"); printf("'mode ?' for help.\n"); return 0; } if (on) c = (linemode | bit); else c = (linemode & ~bit); lm_mode(&c, 1, 1); return 1; } static int setmod(int bit) { return dolmmode(bit, 1); } static int clearmode(int bit) { return dolmmode(bit, 0); } struct modelist { const char *name; /* command name */ const char *help; /* help string */ int (*handler)(int);/* routine which executes command */ int needconnect; /* Do we need to be connected to execute? */ int arg1; }; static struct modelist ModeList[] = { { "character", "Disable LINEMODE option", (int (*)(int))docharmode, 1, 0 }, #ifdef KLUDGELINEMODE { "", "(or disable obsolete line-by-line mode)", NULL, 0, 0 }, #endif { "line", "Enable LINEMODE option", (int (*)(int))dolinemode, 1, 0 }, #ifdef KLUDGELINEMODE { "", "(or enable obsolete line-by-line mode)", NULL, 0, 0 }, #endif { "", "", NULL, 0, 0 }, { "", "These require the LINEMODE option to be enabled", NULL, 0, 0 }, { "isig", "Enable signal trapping", setmod, 1, MODE_TRAPSIG }, { "+isig", 0, setmod, 1, MODE_TRAPSIG }, { "-isig", "Disable signal trapping", clearmode, 1, MODE_TRAPSIG }, { "edit", "Enable character editing", setmod, 1, MODE_EDIT }, { "+edit", 0, setmod, 1, MODE_EDIT }, { "-edit", "Disable character editing", clearmode, 1, MODE_EDIT }, { "softtabs", "Enable tab expansion", setmod, 1, MODE_SOFT_TAB }, { "+softtabs", 0, setmod, 1, MODE_SOFT_TAB }, { "-softtabs", "Disable character editing", clearmode, 1, MODE_SOFT_TAB }, { "litecho", "Enable literal character echo", setmod, 1, MODE_LIT_ECHO }, { "+litecho", 0, setmod, 1, MODE_LIT_ECHO }, { "-litecho", "Disable literal character echo", clearmode, 1, MODE_LIT_ECHO }, { "help", 0, (int (*)(int))modehelp, 0, 0 }, #ifdef KLUDGELINEMODE { "kludgeline", 0, (int (*)(int))dokludgemode, 1, 0 }, #endif { "", "", NULL, 0, 0 }, { "?", "Print help information", (int (*)(int))modehelp, 0, 0 }, { NULL, NULL, NULL, 0, 0 }, }; static int modehelp(void) { struct modelist *mt; printf("format is: 'mode Mode', where 'Mode' is one of:\n\n"); for (mt = ModeList; mt->name; mt++) { if (mt->help) { if (*mt->help) printf("%-15s %s\n", mt->name, mt->help); else printf("\n"); } } return 0; } #define GETMODECMD(name) (struct modelist *) \ genget(name, (char **) ModeList, sizeof(struct modelist)) static int modecmd(int argc, char *argv[]) { struct modelist *mt; if (argc != 2) { printf("'mode' command requires an argument\n"); printf("'mode ?' for help.\n"); } else if ((mt = GETMODECMD(argv[1])) == 0) { fprintf(stderr, "Unknown mode '%s' ('mode ?' for help).\n", argv[1]); } else if (Ambiguous((void *)mt)) { fprintf(stderr, "Ambiguous mode '%s' ('mode ?' for help).\n", argv[1]); } else if (mt->needconnect && !connected) { printf("?Need to be connected first.\n"); printf("'mode ?' for help.\n"); } else if (mt->handler) { return (*mt->handler)(mt->arg1); } return 0; } /* * The following data structures and routines implement the * "display" command. */ static int display(int argc, char *argv[]) { struct togglelist *tl; struct setlist *sl; #define dotog(tl) if (tl->variable && tl->actionexplanation) { \ if (*tl->variable) { \ printf("will"); \ } else { \ printf("won't"); \ } \ printf(" %s.\n", tl->actionexplanation); \ } #define doset(sl) if (sl->name && *sl->name != ' ') { \ if (sl->handler == 0) \ printf("%-15s [%s]\n", sl->name, control(*sl->charp)); \ else \ printf("%-15s \"%s\"\n", sl->name, (char *)sl->charp); \ } if (argc == 1) { for (tl = Togglelist; tl->name; tl++) { dotog(tl); } printf("\n"); for (sl = Setlist; sl->name; sl++) { doset(sl); } } else { int i; for (i = 1; i < argc; i++) { sl = getset(argv[i]); tl = GETTOGGLE(argv[i]); if (Ambiguous((void *)sl) || Ambiguous((void *)tl)) { printf("?Ambiguous argument '%s'.\n", argv[i]); return 0; } else if (!sl && !tl) { printf("?Unknown argument '%s'.\n", argv[i]); return 0; } else { if (tl) { dotog(tl); } if (sl) { doset(sl); } } } } /*@*/optionstatus(); #ifdef ENCRYPTION EncryptStatus(); #endif /* ENCRYPTION */ return 1; #undef doset #undef dotog } /* * The following are the data structures, and many of the routines, * relating to command processing. */ /* * Set the escape character. */ static int setescape(int argc, char *argv[]) { char *arg; char buf[50]; printf( "Deprecated usage - please use 'set escape%s%s' in the future.\n", (argc > 2)? " ":"", (argc > 2)? argv[1]: ""); if (argc > 2) arg = argv[1]; else { printf("new escape character: "); (void) fgets(buf, sizeof(buf), stdin); arg = buf; } if (arg[0] != '\0') escape = arg[0]; (void) fflush(stdout); return 1; } static int togcrmod(void) { crmod = !crmod; printf("Deprecated usage - please use 'toggle crmod' in the future.\n"); printf("%s map carriage return on output.\n", crmod ? "Will" : "Won't"); (void) fflush(stdout); return 1; } static int suspend(void) { #ifdef SIGTSTP setcommandmode(); { long oldrows, oldcols, newrows, newcols, err_; err_ = (TerminalWindowSize(&oldrows, &oldcols) == 0) ? 1 : 0; (void) kill(0, SIGTSTP); /* * If we didn't get the window size before the SUSPEND, but we * can get them now (?), then send the NAWS to make sure that * we are set up for the right window size. */ if (TerminalWindowSize(&newrows, &newcols) && connected && (err_ || ((oldrows != newrows) || (oldcols != newcols)))) { sendnaws(); } } /* reget parameters in case they were changed */ TerminalSaveState(); setconnmode(0); #else printf("Suspend is not supported. Try the '!' command instead\n"); #endif return 1; } static int shell(int argc, char *argv[] __unused) { long oldrows, oldcols, newrows, newcols, err_; setcommandmode(); err_ = (TerminalWindowSize(&oldrows, &oldcols) == 0) ? 1 : 0; switch(vfork()) { case -1: perror("Fork failed\n"); break; case 0: { /* * Fire up the shell in the child. */ const char *shellp, *shellname; shellp = getenv("SHELL"); if (shellp == NULL) shellp = "/bin/sh"; if ((shellname = strrchr(shellp, '/')) == 0) shellname = shellp; else shellname++; if (argc > 1) execl(shellp, shellname, "-c", &saveline[1], (char *)0); else execl(shellp, shellname, (char *)0); perror("Execl"); _exit(1); } default: (void)wait((int *)0); /* Wait for the shell to complete */ if (TerminalWindowSize(&newrows, &newcols) && connected && (err_ || ((oldrows != newrows) || (oldcols != newcols)))) { sendnaws(); } break; } return 1; } static int bye(int argc, char *argv[]) { extern int resettermname; if (connected) { (void) shutdown(net, 2); printf("Connection closed.\n"); (void) NetClose(net); connected = 0; resettermname = 1; #ifdef AUTHENTICATION #ifdef ENCRYPTION auth_encrypt_connect(connected); #endif #endif /* reset options */ tninit(); } if ((argc != 2) || (strcmp(argv[1], "fromquit") != 0)) { longjmp(toplevel, 1); /* NOTREACHED */ } return 1; /* Keep lint, etc., happy */ } void quit(void) { (void) call(bye, "bye", "fromquit", 0); Exit(0); } static int logout(void) { send_do(TELOPT_LOGOUT, 1); (void) netflush(); return 1; } /* * The SLC command. */ struct slclist { const char *name; const char *help; void (*handler)(int); int arg; }; static void slc_help(void); struct slclist SlcList[] = { { "export", "Use local special character definitions", (void (*)(int))slc_mode_export, 0 }, { "import", "Use remote special character definitions", slc_mode_import, 1 }, { "check", "Verify remote special character definitions", slc_mode_import, 0 }, { "help", NULL, (void (*)(int))slc_help, 0 }, { "?", "Print help information", (void (*)(int))slc_help, 0 }, { NULL, NULL, NULL, 0 }, }; static void slc_help(void) { struct slclist *c; for (c = SlcList; c->name; c++) { if (c->help) { if (*c->help) printf("%-15s %s\n", c->name, c->help); else printf("\n"); } } } static struct slclist * getslc(char *name) { return (struct slclist *) genget(name, (char **) SlcList, sizeof(struct slclist)); } static int slccmd(int argc, char *argv[]) { struct slclist *c; if (argc != 2) { fprintf(stderr, "Need an argument to 'slc' command. 'slc ?' for help.\n"); return 0; } c = getslc(argv[1]); if (c == 0) { fprintf(stderr, "'%s': unknown argument ('slc ?' for help).\n", argv[1]); return 0; } if (Ambiguous((void *)c)) { fprintf(stderr, "'%s': ambiguous argument ('slc ?' for help).\n", argv[1]); return 0; } (*c->handler)(c->arg); slcstate(); return 1; } /* * The ENVIRON command. */ struct envlist { const char *name; const char *help; void (*handler)(unsigned char *, unsigned char *); int narg; }; extern struct env_lst * env_define(const unsigned char *, unsigned char *); extern void env_undefine(unsigned char *), env_export(const unsigned char *), env_unexport(const unsigned char *), env_send(unsigned char *), #if defined(OLD_ENVIRON) && defined(ENV_HACK) env_varval(unsigned char *), #endif env_list(void); static void env_help(void); struct envlist EnvList[] = { { "define", "Define an environment variable", (void (*)(unsigned char *, unsigned char *))env_define, 2 }, { "undefine", "Undefine an environment variable", (void (*)(unsigned char *, unsigned char *))env_undefine, 1 }, { "export", "Mark an environment variable for automatic export", (void (*)(unsigned char *, unsigned char *))env_export, 1 }, { "unexport", "Don't mark an environment variable for automatic export", (void (*)(unsigned char *, unsigned char *))env_unexport, 1 }, { "send", "Send an environment variable", (void (*)(unsigned char *, unsigned char *))env_send, 1 }, { "list", "List the current environment variables", (void (*)(unsigned char *, unsigned char *))env_list, 0 }, #if defined(OLD_ENVIRON) && defined(ENV_HACK) { "varval", "Reverse VAR and VALUE (auto, right, wrong, status)", (void (*)(unsigned char *, unsigned char *))env_varval, 1 }, #endif { "help", NULL, (void (*)(unsigned char *, unsigned char *))env_help, 0 }, { "?", "Print help information", (void (*)(unsigned char *, unsigned char *))env_help, 0 }, { NULL, NULL, NULL, 0 }, }; static void env_help(void) { struct envlist *c; for (c = EnvList; c->name; c++) { if (c->help) { if (*c->help) printf("%-15s %s\n", c->name, c->help); else printf("\n"); } } } static struct envlist * getenvcmd(char *name) { return (struct envlist *) genget(name, (char **) EnvList, sizeof(struct envlist)); } static int env_cmd(int argc, char *argv[]) { struct envlist *c; if (argc < 2) { fprintf(stderr, "Need an argument to 'environ' command. 'environ ?' for help.\n"); return 0; } c = getenvcmd(argv[1]); if (c == 0) { fprintf(stderr, "'%s': unknown argument ('environ ?' for help).\n", argv[1]); return 0; } if (Ambiguous((void *)c)) { fprintf(stderr, "'%s': ambiguous argument ('environ ?' for help).\n", argv[1]); return 0; } if (c->narg + 2 != argc) { fprintf(stderr, "Need %s%d argument%s to 'environ %s' command. 'environ ?' for help.\n", c->narg < argc + 2 ? "only " : "", c->narg, c->narg == 1 ? "" : "s", c->name); return 0; } (*c->handler)(argv[2], argv[3]); return 1; } struct env_lst { struct env_lst *next; /* pointer to next structure */ struct env_lst *prev; /* pointer to previous structure */ unsigned char *var; /* pointer to variable name */ unsigned char *value; /* pointer to variable value */ int export; /* 1 -> export with default list of variables */ int welldefined; /* A well defined variable */ }; struct env_lst envlisthead; static struct env_lst * env_find(const unsigned char *var) { struct env_lst *ep; for (ep = envlisthead.next; ep; ep = ep->next) { if (strcmp(ep->var, var) == 0) return(ep); } return(NULL); } void env_init(void) { extern char **environ; char **epp, *cp; struct env_lst *ep; for (epp = environ; *epp; epp++) { if ((cp = strchr(*epp, '='))) { *cp = '\0'; ep = env_define((unsigned char *)*epp, (unsigned char *)cp+1); ep->export = 0; *cp = '='; } } /* * Special case for DISPLAY variable. If it is ":0.0" or * "unix:0.0", we have to get rid of "unix" and insert our * hostname. */ if ((ep = env_find("DISPLAY")) && ((*ep->value == ':') || (strncmp((char *)ep->value, "unix:", 5) == 0))) { char hbuf[256+1]; char *cp2 = strchr((char *)ep->value, ':'); gethostname(hbuf, 256); hbuf[256] = '\0'; cp = (char *)malloc(strlen(hbuf) + strlen(cp2) + 1); sprintf((char *)cp, "%s%s", hbuf, cp2); free(ep->value); ep->value = (unsigned char *)cp; } /* * If USER is not defined, but LOGNAME is, then add * USER with the value from LOGNAME. By default, we * don't export the USER variable. */ if ((env_find("USER") == NULL) && (ep = env_find("LOGNAME"))) { env_define("USER", ep->value); env_unexport("USER"); } env_export("DISPLAY"); env_export("PRINTER"); } struct env_lst * env_define(const unsigned char *var, unsigned char *value) { struct env_lst *ep; if ((ep = env_find(var))) { if (ep->var) free(ep->var); if (ep->value) free(ep->value); } else { ep = (struct env_lst *)malloc(sizeof(struct env_lst)); ep->next = envlisthead.next; envlisthead.next = ep; ep->prev = &envlisthead; if (ep->next) ep->next->prev = ep; } ep->welldefined = opt_welldefined(var); ep->export = 1; ep->var = strdup(var); ep->value = strdup(value); return(ep); } void env_undefine(unsigned char *var) { struct env_lst *ep; if ((ep = env_find(var))) { ep->prev->next = ep->next; if (ep->next) ep->next->prev = ep->prev; if (ep->var) free(ep->var); if (ep->value) free(ep->value); free(ep); } } void env_export(const unsigned char *var) { struct env_lst *ep; if ((ep = env_find(var))) ep->export = 1; } void env_unexport(const unsigned char *var) { struct env_lst *ep; if ((ep = env_find(var))) ep->export = 0; } void env_send(unsigned char *var) { struct env_lst *ep; if (my_state_is_wont(TELOPT_NEW_ENVIRON) #ifdef OLD_ENVIRON && my_state_is_wont(TELOPT_OLD_ENVIRON) #endif ) { fprintf(stderr, "Cannot send '%s': Telnet ENVIRON option not enabled\n", var); return; } ep = env_find(var); if (ep == 0) { fprintf(stderr, "Cannot send '%s': variable not defined\n", var); return; } env_opt_start_info(); env_opt_add(ep->var); env_opt_end(0); } void env_list(void) { struct env_lst *ep; for (ep = envlisthead.next; ep; ep = ep->next) { printf("%c %-20s %s\n", ep->export ? '*' : ' ', ep->var, ep->value); } } unsigned char * env_default(int init, int welldefined) { static struct env_lst *nep = NULL; if (init) { nep = &envlisthead; return(NULL); } if (nep) { while ((nep = nep->next)) { if (nep->export && (nep->welldefined == welldefined)) return(nep->var); } } return(NULL); } unsigned char * env_getvalue(const unsigned char *var) { struct env_lst *ep; if ((ep = env_find(var))) return(ep->value); return(NULL); } #if defined(OLD_ENVIRON) && defined(ENV_HACK) void env_varval(unsigned char *what) { extern int old_env_var, old_env_value, env_auto; int len = strlen((char *)what); if (len == 0) goto unknown; if (strncasecmp((char *)what, "status", len) == 0) { if (env_auto) printf("%s%s", "VAR and VALUE are/will be ", "determined automatically\n"); if (old_env_var == OLD_ENV_VAR) printf("VAR and VALUE set to correct definitions\n"); else printf("VAR and VALUE definitions are reversed\n"); } else if (strncasecmp((char *)what, "auto", len) == 0) { env_auto = 1; old_env_var = OLD_ENV_VALUE; old_env_value = OLD_ENV_VAR; } else if (strncasecmp((char *)what, "right", len) == 0) { env_auto = 0; old_env_var = OLD_ENV_VAR; old_env_value = OLD_ENV_VALUE; } else if (strncasecmp((char *)what, "wrong", len) == 0) { env_auto = 0; old_env_var = OLD_ENV_VALUE; old_env_value = OLD_ENV_VAR; } else { unknown: printf("Unknown \"varval\" command. (\"auto\", \"right\", \"wrong\", \"status\")\n"); } } #endif #ifdef AUTHENTICATION /* * The AUTHENTICATE command. */ struct authlist { const char *name; const char *help; int (*handler)(char *); int narg; }; extern int auth_enable(char *), auth_disable(char *), auth_status(void); static int auth_help(void); struct authlist AuthList[] = { { "status", "Display current status of authentication information", (int (*)(char *))auth_status, 0 }, { "disable", "Disable an authentication type ('auth disable ?' for more)", auth_disable, 1 }, { "enable", "Enable an authentication type ('auth enable ?' for more)", auth_enable, 1 }, { "help", NULL, (int (*)(char *))auth_help, 0 }, { "?", "Print help information", (int (*)(char *))auth_help, 0 }, { NULL, NULL, NULL, 0 }, }; static int auth_help(void) { struct authlist *c; for (c = AuthList; c->name; c++) { if (c->help) { if (*c->help) printf("%-15s %s\n", c->name, c->help); else printf("\n"); } } return 0; } int auth_cmd(int argc, char *argv[]) { struct authlist *c; if (argc < 2) { fprintf(stderr, "Need an argument to 'auth' command. 'auth ?' for help.\n"); return 0; } c = (struct authlist *) genget(argv[1], (char **) AuthList, sizeof(struct authlist)); if (c == 0) { fprintf(stderr, "'%s': unknown argument ('auth ?' for help).\n", argv[1]); return 0; } if (Ambiguous((void *)c)) { fprintf(stderr, "'%s': ambiguous argument ('auth ?' for help).\n", argv[1]); return 0; } if (c->narg + 2 != argc) { fprintf(stderr, "Need %s%d argument%s to 'auth %s' command. 'auth ?' for help.\n", c->narg < argc + 2 ? "only " : "", c->narg, c->narg == 1 ? "" : "s", c->name); return 0; } return((*c->handler)(argv[2])); } #endif #ifdef ENCRYPTION /* * The ENCRYPT command. */ struct encryptlist { const char *name; const char *help; int (*handler)(char *, char *); int needconnect; int minarg; int maxarg; }; extern int EncryptEnable(char *, char *), EncryptDisable(char *, char *), EncryptType(char *, char *), EncryptStart(char *), EncryptStartInput(void), EncryptStartOutput(void), EncryptStop(char *), EncryptStopInput(void), EncryptStopOutput(void), EncryptStatus(void); static int EncryptHelp(void); struct encryptlist EncryptList[] = { { "enable", "Enable encryption. ('encrypt enable ?' for more)", EncryptEnable, 1, 1, 2 }, { "disable", "Disable encryption. ('encrypt enable ?' for more)", EncryptDisable, 0, 1, 2 }, { "type", "Set encryption type. ('encrypt type ?' for more)", EncryptType, 0, 1, 1 }, { "start", "Start encryption. ('encrypt start ?' for more)", (int (*)(char *, char *))EncryptStart, 1, 0, 1 }, { "stop", "Stop encryption. ('encrypt stop ?' for more)", (int (*)(char *, char *))EncryptStop, 1, 0, 1 }, { "input", "Start encrypting the input stream", (int (*)(char *, char *))EncryptStartInput, 1, 0, 0 }, { "-input", "Stop encrypting the input stream", (int (*)(char *, char *))EncryptStopInput, 1, 0, 0 }, { "output", "Start encrypting the output stream", (int (*)(char *, char *))EncryptStartOutput, 1, 0, 0 }, { "-output", "Stop encrypting the output stream", (int (*)(char *, char *))EncryptStopOutput, 1, 0, 0 }, { "status", "Display current status of authentication information", (int (*)(char *, char *))EncryptStatus, 0, 0, 0 }, { "help", NULL, (int (*)(char *, char *))EncryptHelp, 0, 0, 0 }, { "?", "Print help information", (int (*)(char *, char *))EncryptHelp, 0, 0, 0 }, { NULL, NULL, NULL, 0, 0, 0 }, }; static int EncryptHelp(void) { struct encryptlist *c; for (c = EncryptList; c->name; c++) { if (c->help) { if (*c->help) printf("%-15s %s\n", c->name, c->help); else printf("\n"); } } return 0; } static int encrypt_cmd(int argc, char *argv[]) { struct encryptlist *c; if (argc < 2) { fprintf(stderr, "Need an argument to 'encrypt' command. 'encrypt ?' for help.\n"); return 0; } c = (struct encryptlist *) genget(argv[1], (char **) EncryptList, sizeof(struct encryptlist)); if (c == 0) { fprintf(stderr, "'%s': unknown argument ('encrypt ?' for help).\n", argv[1]); return 0; } if (Ambiguous((void *)c)) { fprintf(stderr, "'%s': ambiguous argument ('encrypt ?' for help).\n", argv[1]); return 0; } argc -= 2; if (argc < c->minarg || argc > c->maxarg) { if (c->minarg == c->maxarg) { fprintf(stderr, "Need %s%d argument%s ", c->minarg < argc ? "only " : "", c->minarg, c->minarg == 1 ? "" : "s"); } else { fprintf(stderr, "Need %s%d-%d arguments ", c->maxarg < argc ? "only " : "", c->minarg, c->maxarg); } fprintf(stderr, "to 'encrypt %s' command. 'encrypt ?' for help.\n", c->name); return 0; } if (c->needconnect && !connected) { if (!(argc && (isprefix(argv[2], "help") || isprefix(argv[2], "?")))) { printf("?Need to be connected first.\n"); return 0; } } return ((*c->handler)(argc > 0 ? argv[2] : 0, argc > 1 ? argv[3] : 0)); } #endif /* ENCRYPTION */ /* * Print status about the connection. */ /*ARGSUSED*/ static int status(int argc, char *argv[]) { if (connected) { printf("Connected to %s.\n", hostname); if ((argc < 2) || strcmp(argv[1], "notmuch")) { int mode = getconnmode(); if (my_want_state_is_will(TELOPT_LINEMODE)) { printf("Operating with LINEMODE option\n"); printf("%s line editing\n", (mode&MODE_EDIT) ? "Local" : "No"); printf("%s catching of signals\n", (mode&MODE_TRAPSIG) ? "Local" : "No"); slcstate(); #ifdef KLUDGELINEMODE } else if (kludgelinemode && my_want_state_is_dont(TELOPT_SGA)) { printf("Operating in obsolete linemode\n"); #endif } else { printf("Operating in single character mode\n"); if (localchars) printf("Catching signals locally\n"); } printf("%s character echo\n", (mode&MODE_ECHO) ? "Local" : "Remote"); if (my_want_state_is_will(TELOPT_LFLOW)) printf("%s flow control\n", (mode&MODE_FLOW) ? "Local" : "No"); #ifdef ENCRYPTION encrypt_display(); #endif /* ENCRYPTION */ } } else { printf("No connection.\n"); } printf("Escape character is '%s'.\n", control(escape)); (void) fflush(stdout); return 1; } #ifdef SIGINFO /* * Function that gets called when SIGINFO is received. */ void ayt_status(void) { (void) call(status, "status", "notmuch", 0); } #endif static const char * sockaddr_ntop(struct sockaddr *sa) { void *addr; static char addrbuf[INET6_ADDRSTRLEN]; switch (sa->sa_family) { case AF_INET: addr = &((struct sockaddr_in *)sa)->sin_addr; break; case AF_UNIX: addr = &((struct sockaddr_un *)sa)->sun_path; break; #ifdef INET6 case AF_INET6: addr = &((struct sockaddr_in6 *)sa)->sin6_addr; break; #endif default: return NULL; } inet_ntop(sa->sa_family, addr, addrbuf, sizeof(addrbuf)); return addrbuf; } #if defined(IPSEC) && defined(IPSEC_POLICY_IPSEC) static int setpolicy(int lnet, struct addrinfo *res, char *policy) { char *buf; int level; int optname; if (policy == NULL) return 0; buf = ipsec_set_policy(policy, strlen(policy)); if (buf == NULL) { printf("%s\n", ipsec_strerror()); return -1; } level = res->ai_family == AF_INET ? IPPROTO_IP : IPPROTO_IPV6; optname = res->ai_family == AF_INET ? IP_IPSEC_POLICY : IPV6_IPSEC_POLICY; if (setsockopt(lnet, level, optname, buf, ipsec_get_policylen(buf)) < 0){ perror("setsockopt"); return -1; } free(buf); return 0; } #endif #ifdef INET6 /* * When an Address Family related error happend, check if retry with * another AF is possible or not. * Return 1, if retry with another af is OK. Else, return 0. */ static int switch_af(struct addrinfo **aip) { int nextaf; struct addrinfo *ai; ai = *aip; nextaf = (ai->ai_family == AF_INET) ? AF_INET6 : AF_INET; do ai=ai->ai_next; while (ai != NULL && ai->ai_family != nextaf); *aip = ai; if (*aip != NULL) { return 1; } return 0; } #endif int tn(int argc, char *argv[]) { - char *srp = 0; + unsigned char *srp = 0; int proto, opt; int srlen; int srcroute = 0, result; char *cmd, *hostp = 0, *portp = 0, *user = 0; char *src_addr = NULL; struct addrinfo hints, *res, *res0 = NULL, *src_res, *src_res0 = NULL; int error = 0, af_error = 0; if (connected) { printf("?Already connected to %s\n", hostname); setuid(getuid()); return 0; } if (argc < 2) { (void) strcpy(line, "open "); printf("(to) "); (void) fgets(&line[strlen(line)], sizeof(line) - strlen(line), stdin); makeargv(); argc = margc; argv = margv; } cmd = *argv; --argc; ++argv; while (argc) { if (strcmp(*argv, "help") == 0 || isprefix(*argv, "?")) goto usage; if (strcmp(*argv, "-l") == 0) { --argc; ++argv; if (argc == 0) goto usage; user = *argv++; --argc; continue; } if (strcmp(*argv, "-a") == 0) { --argc; ++argv; autologin = 1; continue; } if (strcmp(*argv, "-s") == 0) { --argc; ++argv; if (argc == 0) goto usage; src_addr = *argv++; --argc; continue; } if (hostp == 0) { hostp = *argv++; --argc; continue; } if (portp == 0) { portp = *argv++; --argc; continue; } usage: printf("usage: %s [-l user] [-a] [-s src_addr] host-name [port]\n", cmd); setuid(getuid()); return 0; } if (hostp == 0) goto usage; if (src_addr != NULL) { memset(&hints, 0, sizeof(hints)); hints.ai_family = family; hints.ai_socktype = SOCK_STREAM; error = getaddrinfo(src_addr, 0, &hints, &src_res); if (error == EAI_NONAME) { hints.ai_flags = 0; error = getaddrinfo(src_addr, 0, &hints, &src_res); } if (error != 0) { fprintf(stderr, "%s: %s\n", src_addr, gai_strerror(error)); if (error == EAI_SYSTEM) fprintf(stderr, "%s: %s\n", src_addr, strerror(errno)); setuid(getuid()); return 0; } src_res0 = src_res; } if (hostp[0] == '/') { struct sockaddr_un su; if (strlen(hostp) >= sizeof(su.sun_path)) { fprintf(stderr, "hostname too long for unix domain socket: %s", hostp); goto fail; } hostname = hostp; memset(&su, 0, sizeof su); su.sun_family = AF_UNIX; strncpy(su.sun_path, hostp, sizeof su.sun_path); printf("Trying %s...\n", hostp); net = socket(PF_UNIX, SOCK_STREAM, 0); if ( net < 0) { perror("socket"); goto fail; } if (connect(net, (struct sockaddr *)&su, sizeof su) == -1) { perror(su.sun_path); (void) NetClose(net); goto fail; } goto af_unix; } else if (hostp[0] == '@' || hostp[0] == '!') { if ( #ifdef INET6 family == AF_INET6 || #endif (hostname = strrchr(hostp, ':')) == NULL) hostname = strrchr(hostp, '@'); if (hostname == NULL) { hostname = hostp; } else { hostname++; srcroute = 1; } } else hostname = hostp; if (!portp) { telnetport = 1; portp = strdup("telnet"); } else if (*portp == '-') { portp++; telnetport = 1; } else if (*portp == '+') { portp++; telnetport = -1; } else telnetport = 0; memset(&hints, 0, sizeof(hints)); hints.ai_flags = AI_NUMERICHOST; hints.ai_family = family; hints.ai_socktype = SOCK_STREAM; error = getaddrinfo(hostname, portp, &hints, &res); if (error) { hints.ai_flags = AI_CANONNAME; error = getaddrinfo(hostname, portp, &hints, &res); } if (error != 0) { fprintf(stderr, "%s: %s\n", hostname, gai_strerror(error)); if (error == EAI_SYSTEM) fprintf(stderr, "%s: %s\n", hostname, strerror(errno)); setuid(getuid()); goto fail; } if (hints.ai_flags == AI_NUMERICHOST) { /* hostname has numeric */ int gni_err = 1; if (doaddrlookup) gni_err = getnameinfo(res->ai_addr, res->ai_addr->sa_len, _hostname, sizeof(_hostname) - 1, NULL, 0, NI_NAMEREQD); if (gni_err != 0) (void) strncpy(_hostname, hostp, sizeof(_hostname) - 1); _hostname[sizeof(_hostname)-1] = '\0'; hostname = _hostname; } else { /* hostname has FQDN */ if (srcroute != 0) (void) strncpy(_hostname, hostname, sizeof(_hostname) - 1); else if (res->ai_canonname != NULL) strcpy(_hostname, res->ai_canonname); else (void) strncpy(_hostname, hostp, sizeof(_hostname) - 1); _hostname[sizeof(_hostname)-1] = '\0'; hostname = _hostname; } res0 = res; #ifdef INET6 af_again: #endif if (srcroute != 0) { static char hostbuf[BUFSIZ]; if (af_error == 0) { /* save intermediate hostnames for retry */ strncpy(hostbuf, hostp, BUFSIZ - 1); hostbuf[BUFSIZ - 1] = '\0'; } else hostp = hostbuf; srp = 0; result = sourceroute(res, hostp, &srp, &srlen, &proto, &opt); if (result == 0) { #ifdef INET6 if (family == AF_UNSPEC && af_error == 0 && switch_af(&res) == 1) { af_error = 1; goto af_again; } #endif setuid(getuid()); goto fail; } else if (result == -1) { printf("Bad source route option: %s\n", hostp); setuid(getuid()); goto fail; } } do { printf("Trying %s...\n", sockaddr_ntop(res->ai_addr)); net = socket(res->ai_family, res->ai_socktype, res->ai_protocol); setuid(getuid()); if (net < 0) { #ifdef INET6 if (family == AF_UNSPEC && af_error == 0 && switch_af(&res) == 1) { af_error = 1; goto af_again; } #endif perror("telnet: socket"); goto fail; } if (srp && setsockopt(net, proto, opt, (char *)srp, srlen) < 0) perror("setsockopt (source route)"); #if defined(IPPROTO_IP) && defined(IP_TOS) if (res->ai_family == PF_INET) { # if defined(HAS_GETTOS) struct tosent *tp; if (tos < 0 && (tp = gettosbyname("telnet", "tcp"))) tos = tp->t_tos; # endif if (tos < 0) tos = IPTOS_LOWDELAY; if (tos && (setsockopt(net, IPPROTO_IP, IP_TOS, (char *)&tos, sizeof(int)) < 0) && (errno != ENOPROTOOPT)) perror("telnet: setsockopt (IP_TOS) (ignored)"); } #endif /* defined(IPPROTO_IP) && defined(IP_TOS) */ if (telnet_debug && SetSockOpt(net, SOL_SOCKET, SO_DEBUG, 1) < 0) { perror("setsockopt (SO_DEBUG)"); } if (src_addr != NULL) { for (src_res = src_res0; src_res != 0; src_res = src_res->ai_next) if (src_res->ai_family == res->ai_family) break; if (src_res == NULL) src_res = src_res0; if (bind(net, src_res->ai_addr, src_res->ai_addrlen) == -1) { #ifdef INET6 if (family == AF_UNSPEC && af_error == 0 && switch_af(&res) == 1) { af_error = 1; (void) NetClose(net); goto af_again; } #endif perror("bind"); (void) NetClose(net); goto fail; } } #if defined(IPSEC) && defined(IPSEC_POLICY_IPSEC) if (setpolicy(net, res, ipsec_policy_in) < 0) { (void) NetClose(net); goto fail; } if (setpolicy(net, res, ipsec_policy_out) < 0) { (void) NetClose(net); goto fail; } #endif if (connect(net, res->ai_addr, res->ai_addrlen) < 0) { struct addrinfo *next; next = res->ai_next; /* If already an af failed, only try same af. */ if (af_error != 0) while (next != NULL && next->ai_family != res->ai_family) next = next->ai_next; warn("connect to address %s", sockaddr_ntop(res->ai_addr)); if (next != NULL) { res = next; (void) NetClose(net); continue; } warnx("Unable to connect to remote host"); (void) NetClose(net); goto fail; } connected++; #ifdef AUTHENTICATION #ifdef ENCRYPTION auth_encrypt_connect(connected); #endif #endif } while (connected == 0); freeaddrinfo(res0); if (src_res0 != NULL) freeaddrinfo(src_res0); cmdrc(hostp, hostname); af_unix: connected = 1; if (autologin && user == NULL) { struct passwd *pw; user = getenv("USER"); if (user == NULL || ((pw = getpwnam(user)) && pw->pw_uid != getuid())) { if ((pw = getpwuid(getuid()))) user = pw->pw_name; else user = NULL; } } if (user) { env_define("USER", user); env_export("USER"); } (void) call(status, "status", "notmuch", 0); telnet(user); (void) NetClose(net); ExitString("Connection closed by foreign host.\n",1); /*NOTREACHED*/ fail: if (res0 != NULL) freeaddrinfo(res0); if (src_res0 != NULL) freeaddrinfo(src_res0); return 0; } #define HELPINDENT (sizeof ("connect")) static char openhelp[] = "connect to a site", closehelp[] = "close current connection", logouthelp[] = "forcibly logout remote user and close the connection", quithelp[] = "exit telnet", statushelp[] = "print status information", helphelp[] = "print help information", sendhelp[] = "transmit special characters ('send ?' for more)", sethelp[] = "set operating parameters ('set ?' for more)", unsethelp[] = "unset operating parameters ('unset ?' for more)", togglestring[] ="toggle operating parameters ('toggle ?' for more)", slchelp[] = "change state of special charaters ('slc ?' for more)", displayhelp[] = "display operating parameters", #ifdef AUTHENTICATION authhelp[] = "turn on (off) authentication ('auth ?' for more)", #endif #ifdef ENCRYPTION encrypthelp[] = "turn on (off) encryption ('encrypt ?' for more)", #endif /* ENCRYPTION */ zhelp[] = "suspend telnet", #ifdef OPIE opiehelp[] = "compute response to OPIE challenge", #endif shellhelp[] = "invoke a subshell", envhelp[] = "change environment variables ('environ ?' for more)", modestring[] = "try to enter line or character mode ('mode ?' for more)"; static Command cmdtab[] = { { "close", closehelp, bye, 1 }, { "logout", logouthelp, (int (*)(int, char **))logout, 1 }, { "display", displayhelp, display, 0 }, { "mode", modestring, modecmd, 0 }, { "telnet", openhelp, tn, 0 }, { "open", openhelp, tn, 0 }, { "quit", quithelp, (int (*)(int, char **))quit, 0 }, { "send", sendhelp, sendcmd, 0 }, { "set", sethelp, setcmd, 0 }, { "unset", unsethelp, unsetcmd, 0 }, { "status", statushelp, status, 0 }, { "toggle", togglestring, toggle, 0 }, { "slc", slchelp, slccmd, 0 }, #ifdef AUTHENTICATION { "auth", authhelp, auth_cmd, 0 }, #endif #ifdef ENCRYPTION { "encrypt", encrypthelp, encrypt_cmd, 0 }, #endif /* ENCRYPTION */ { "z", zhelp, (int (*)(int, char **))suspend, 0 }, { "!", shellhelp, shell, 1 }, { "environ", envhelp, env_cmd, 0 }, { "?", helphelp, help, 0 }, #ifdef OPIE { "opie", opiehelp, opie_calc, 0 }, #endif { NULL, NULL, NULL, 0 } }; static char crmodhelp[] = "deprecated command -- use 'toggle crmod' instead"; static char escapehelp[] = "deprecated command -- use 'set escape' instead"; static Command cmdtab2[] = { { "help", 0, help, 0 }, { "escape", escapehelp, setescape, 0 }, { "crmod", crmodhelp, (int (*)(int, char **))togcrmod, 0 }, { NULL, NULL, NULL, 0 } }; /* * Call routine with argc, argv set from args (terminated by 0). */ static int call(intrtn_t routine, ...) { va_list ap; char *args[100]; int argno = 0; va_start(ap, routine); while ((args[argno++] = va_arg(ap, char *)) != 0); va_end(ap); return (*routine)(argno-1, args); } static Command * getcmd(char *name) { Command *cm; if ((cm = (Command *) genget(name, (char **) cmdtab, sizeof(Command)))) return cm; return (Command *) genget(name, (char **) cmdtab2, sizeof(Command)); } void command(int top, const char *tbuf, int cnt) { Command *c; setcommandmode(); if (!top) { putchar('\n'); } else { (void) signal(SIGINT, SIG_DFL); (void) signal(SIGQUIT, SIG_DFL); } for (;;) { if (rlogin == _POSIX_VDISABLE) printf("%s> ", prompt); if (tbuf) { char *cp; cp = line; while (cnt > 0 && (*cp++ = *tbuf++) != '\n') cnt--; tbuf = 0; if (cp == line || *--cp != '\n' || cp == line) goto getline; *cp = '\0'; if (rlogin == _POSIX_VDISABLE) printf("%s\n", line); } else { getline: if (rlogin != _POSIX_VDISABLE) printf("%s> ", prompt); if (fgets(line, sizeof(line), stdin) == NULL) { if (feof(stdin) || ferror(stdin)) { (void) quit(); /*NOTREACHED*/ } break; } } if (line[0] == 0) break; makeargv(); if (margv[0] == 0) { break; } c = getcmd(margv[0]); if (Ambiguous((void *)c)) { printf("?Ambiguous command\n"); continue; } if (c == 0) { printf("?Invalid command\n"); continue; } if (c->needconnect && !connected) { printf("?Need to be connected first.\n"); continue; } if ((*c->handler)(margc, margv)) { break; } } if (!top) { if (!connected) { longjmp(toplevel, 1); /*NOTREACHED*/ } setconnmode(0); } } /* * Help command. */ static int help(int argc, char *argv[]) { Command *c; if (argc == 1) { printf("Commands may be abbreviated. Commands are:\n\n"); for (c = cmdtab; c->name; c++) if (c->help) { printf("%-*s\t%s\n", (int)HELPINDENT, c->name, c->help); } return 0; } else while (--argc > 0) { char *arg; arg = *++argv; c = getcmd(arg); if (Ambiguous((void *)c)) printf("?Ambiguous help command %s\n", arg); else if (c == (Command *)0) printf("?Invalid help command %s\n", arg); else printf("%s\n", c->help); } return 0; } static char *rcname = 0; static char rcbuf[128]; void cmdrc(char *m1, char *m2) { Command *c; FILE *rcfile; int gotmachine = 0; int l1 = strlen(m1); int l2 = strlen(m2); char m1save[MAXHOSTNAMELEN]; if (skiprc) return; strlcpy(m1save, m1, sizeof(m1save)); m1 = m1save; if (rcname == 0) { rcname = getenv("HOME"); if (rcname && (strlen(rcname) + 10) < sizeof(rcbuf)) strcpy(rcbuf, rcname); else rcbuf[0] = '\0'; strcat(rcbuf, "/.telnetrc"); rcname = rcbuf; } if ((rcfile = fopen(rcname, "r")) == 0) { return; } for (;;) { if (fgets(line, sizeof(line), rcfile) == NULL) break; if (line[0] == 0) break; if (line[0] == '#') continue; if (gotmachine) { if (!isspace(line[0])) gotmachine = 0; } if (gotmachine == 0) { if (isspace(line[0])) continue; if (strncasecmp(line, m1, l1) == 0) strncpy(line, &line[l1], sizeof(line) - l1); else if (strncasecmp(line, m2, l2) == 0) strncpy(line, &line[l2], sizeof(line) - l2); else if (strncasecmp(line, "DEFAULT", 7) == 0) strncpy(line, &line[7], sizeof(line) - 7); else continue; if (line[0] != ' ' && line[0] != '\t' && line[0] != '\n') continue; gotmachine = 1; } makeargv(); if (margv[0] == 0) continue; c = getcmd(margv[0]); if (Ambiguous((void *)c)) { printf("?Ambiguous command: %s\n", margv[0]); continue; } if (c == 0) { printf("?Invalid command: %s\n", margv[0]); continue; } /* * This should never happen... */ if (c->needconnect && !connected) { printf("?Need to be connected first for %s.\n", margv[0]); continue; } (*c->handler)(margc, margv); } fclose(rcfile); } /* * Source route is handed in as * [!]@hop1@hop2...[@|:]dst * If the leading ! is present, it is a * strict source route, otherwise it is * assmed to be a loose source route. * * We fill in the source route option as * hop1,hop2,hop3...dest * and return a pointer to hop1, which will * be the address to connect() to. * * Arguments: * * res: ponter to addrinfo structure which contains sockaddr to * the host to connect to. * * arg: pointer to route list to decipher * * cpp: If *cpp is not equal to NULL, this is a * pointer to a pointer to a character array * that should be filled in with the option. * * lenp: pointer to an integer that contains the * length of *cpp if *cpp != NULL. * * protop: pointer to an integer that should be filled in with * appropriate protocol for setsockopt, as socket * protocol family. * * optp: pointer to an integer that should be filled in with * appropriate option for setsockopt, as socket protocol * family. * * Return values: * * If the return value is 1, then all operations are * successful. If the * return value is -1, there was a syntax error in the * option, either unknown characters, or too many hosts. * If the return value is 0, one of the hostnames in the * path is unknown, and *cpp is set to point to the bad * hostname. * * *cpp: If *cpp was equal to NULL, it will be filled * in with a pointer to our static area that has * the option filled in. This will be 32bit aligned. * * *lenp: This will be filled in with how long the option * pointed to by *cpp is. * * *protop: This will be filled in with appropriate protocol for * setsockopt, as socket protocol family. * * *optp: This will be filled in with appropriate option for * setsockopt, as socket protocol family. */ static int -sourceroute(struct addrinfo *ai, char *arg, char **cpp, int *lenp, int *protop, int *optp) +sourceroute(struct addrinfo *ai, char *arg, unsigned char **cpp, int *lenp, int *protop, int *optp) { static char buf[1024 + ALIGNBYTES]; /*XXX*/ - char *cp, *cp2, *lsrp, *ep; + unsigned char *cp, *cp2, *lsrp, *ep; struct sockaddr_in *_sin; #ifdef INET6 struct sockaddr_in6 *sin6; struct ip6_rthdr *rth; #endif struct addrinfo hints, *res; int error; char c; /* * Verify the arguments, and make sure we have * at least 7 bytes for the option. */ if (cpp == NULL || lenp == NULL) return -1; if (*cpp != NULL) { switch (res->ai_family) { case AF_INET: if (*lenp < 7) return -1; break; #ifdef INET6 case AF_INET6: if (*lenp < (int)CMSG_SPACE(sizeof(struct ip6_rthdr) + sizeof(struct in6_addr))) return -1; break; #endif } } /* * Decide whether we have a buffer passed to us, * or if we need to use our own static buffer. */ if (*cpp) { lsrp = *cpp; ep = lsrp + *lenp; } else { *cpp = lsrp = (char *)ALIGN(buf); ep = lsrp + 1024; } cp = arg; #ifdef INET6 if (ai->ai_family == AF_INET6) { if ((rth = inet6_rth_init((void *)*cpp, sizeof(buf), IPV6_RTHDR_TYPE_0, 0)) == NULL) return -1; if (*cp != '@') return -1; *protop = IPPROTO_IPV6; *optp = IPV6_RTHDR; } else #endif { /* * Next, decide whether we have a loose source * route or a strict source route, and fill in * the begining of the option. */ if (*cp == '!') { cp++; *lsrp++ = IPOPT_SSRR; } else *lsrp++ = IPOPT_LSRR; if (*cp != '@') return -1; lsrp++; /* skip over length, we'll fill it in later */ *lsrp++ = 4; *protop = IPPROTO_IP; *optp = IP_OPTIONS; } cp++; memset(&hints, 0, sizeof(hints)); hints.ai_family = ai->ai_family; hints.ai_socktype = SOCK_STREAM; for (c = 0;;) { if ( #ifdef INET6 ai->ai_family != AF_INET6 && #endif c == ':') cp2 = 0; else for (cp2 = cp; (c = *cp2); cp2++) { if (c == ',') { *cp2++ = '\0'; if (*cp2 == '@') cp2++; } else if (c == '@') { *cp2++ = '\0'; } else if ( #ifdef INET6 ai->ai_family != AF_INET6 && #endif c == ':') { *cp2++ = '\0'; } else continue; break; } if (!c) cp2 = 0; hints.ai_flags = AI_NUMERICHOST; error = getaddrinfo(cp, NULL, &hints, &res); if (error == EAI_NONAME) { hints.ai_flags = 0; error = getaddrinfo(cp, NULL, &hints, &res); } if (error != 0) { fprintf(stderr, "%s: %s\n", cp, gai_strerror(error)); if (error == EAI_SYSTEM) fprintf(stderr, "%s: %s\n", cp, strerror(errno)); *cpp = cp; return(0); } #ifdef INET6 if (res->ai_family == AF_INET6) { sin6 = (struct sockaddr_in6 *)res->ai_addr; if (inet6_rth_add((void *)rth, &sin6->sin6_addr) == -1) return(0); } else #endif { _sin = (struct sockaddr_in *)res->ai_addr; memcpy(lsrp, (char *)&_sin->sin_addr, 4); lsrp += 4; } if (cp2) cp = cp2; else break; /* * Check to make sure there is space for next address */ if (lsrp + 4 > ep) return -1; freeaddrinfo(res); } #ifdef INET6 if (res->ai_family == AF_INET6) { rth->ip6r_len = rth->ip6r_segleft * 2; *lenp = (rth->ip6r_len + 1) << 3; } else #endif { if ((*(*cpp+IPOPT_OLEN) = lsrp - *cpp) <= 7) { *cpp = 0; *lenp = 0; return -1; } *lsrp++ = IPOPT_NOP; /* 32 bit word align it */ *lenp = lsrp - *cpp; } freeaddrinfo(res); return 1; } Index: projects/netbsd-tests-update-12/contrib/tnftp/src/ftp.c =================================================================== --- projects/netbsd-tests-update-12/contrib/tnftp/src/ftp.c (revision 305171) +++ projects/netbsd-tests-update-12/contrib/tnftp/src/ftp.c (revision 305172) @@ -1,2149 +1,2149 @@ /* $NetBSD: ftp.c,v 1.18 2009/05/20 12:53:47 lukem Exp $ */ /* from NetBSD: ftp.c,v 1.159 2009/04/15 03:42:33 jld Exp */ /*- * Copyright (c) 1996-2009 The NetBSD Foundation, Inc. * All rights reserved. * * This code is derived from software contributed to The NetBSD Foundation * by Luke Mewburn. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ /* * Copyright (c) 1985, 1989, 1993, 1994 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * Copyright (C) 1997 and 1998 WIDE Project. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the project nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include "tnftp.h" #include #if 0 /* tnftp */ #include #ifndef lint #if 0 static char sccsid[] = "@(#)ftp.c 8.6 (Berkeley) 10/27/94"; #else __RCSID(" NetBSD: ftp.c,v 1.159 2009/04/15 03:42:33 jld Exp "); #endif #endif /* not lint */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #endif /* tnftp */ #include "ftp_var.h" volatile sig_atomic_t abrtflag; volatile sig_atomic_t timeoutflag; sigjmp_buf ptabort; int ptabflg; int ptflag = 0; char pasv[BUFSIZ]; /* passive port for proxy data connection */ static int empty(FILE *, FILE *, int); struct sockinet { union sockunion { struct sockaddr_in su_sin; #ifdef INET6 struct sockaddr_in6 su_sin6; #endif } si_su; #if !defined(HAVE_STRUCT_SOCKADDR_IN_SIN_LEN) int si_len; #endif }; #if !defined(HAVE_STRUCT_SOCKADDR_IN_SIN_LEN) # define su_len si_len #else # define su_len si_su.su_sin.sin_len #endif #define su_family si_su.su_sin.sin_family #define su_port si_su.su_sin.sin_port struct sockinet myctladdr, hisctladdr, data_addr; char * hookup(const char *host, const char *port) { int s = -1, error; struct addrinfo hints, *res, *res0; static char hostnamebuf[MAXHOSTNAMELEN]; socklen_t len; int on = 1; memset((char *)&hisctladdr, 0, sizeof (hisctladdr)); memset((char *)&myctladdr, 0, sizeof (myctladdr)); memset(&hints, 0, sizeof(hints)); hints.ai_flags = AI_CANONNAME; hints.ai_family = family; hints.ai_socktype = SOCK_STREAM; hints.ai_protocol = 0; error = getaddrinfo(host, port, &hints, &res0); if (error) { warnx("Can't lookup `%s:%s': %s", host, port, (error == EAI_SYSTEM) ? strerror(errno) : gai_strerror(error)); code = -1; return (0); } if (res0->ai_canonname) (void)strlcpy(hostnamebuf, res0->ai_canonname, sizeof(hostnamebuf)); else (void)strlcpy(hostnamebuf, host, sizeof(hostnamebuf)); hostname = hostnamebuf; for (res = res0; res; res = res->ai_next) { char hname[NI_MAXHOST], sname[NI_MAXSERV]; ai_unmapped(res); if (getnameinfo(res->ai_addr, res->ai_addrlen, hname, sizeof(hname), sname, sizeof(sname), NI_NUMERICHOST | NI_NUMERICSERV) != 0) { strlcpy(hname, "?", sizeof(hname)); strlcpy(sname, "?", sizeof(sname)); } if (verbose && res0->ai_next) { /* if we have multiple possibilities */ fprintf(ttyout, "Trying %s:%s ...\n", hname, sname); } s = socket(res->ai_family, SOCK_STREAM, res->ai_protocol); if (s < 0) { warn("Can't create socket for connection to `%s:%s'", hname, sname); continue; } if (ftp_connect(s, res->ai_addr, res->ai_addrlen) < 0) { close(s); s = -1; continue; } /* finally we got one */ break; } if (s < 0) { warnx("Can't connect to `%s:%s'", host, port); code = -1; freeaddrinfo(res0); return 0; } memcpy(&hisctladdr.si_su, res->ai_addr, res->ai_addrlen); hisctladdr.su_len = res->ai_addrlen; freeaddrinfo(res0); res0 = res = NULL; len = hisctladdr.su_len; if (getsockname(s, (struct sockaddr *)&myctladdr.si_su, &len) == -1) { warn("Can't determine my address of connection to `%s:%s'", host, port); code = -1; goto bad; } myctladdr.su_len = len; #ifdef IPTOS_LOWDELAY if (hisctladdr.su_family == AF_INET) { int tos = IPTOS_LOWDELAY; if (setsockopt(s, IPPROTO_IP, IP_TOS, (void *)&tos, sizeof(tos)) == -1) { DWARN("setsockopt %s (ignored)", "IPTOS_LOWDELAY"); } } #endif cin = fdopen(s, "r"); cout = fdopen(s, "w"); if (cin == NULL || cout == NULL) { warnx("Can't fdopen socket"); if (cin) (void)fclose(cin); if (cout) (void)fclose(cout); code = -1; goto bad; } if (verbose) fprintf(ttyout, "Connected to %s.\n", hostname); if (getreply(0) > 2) { /* read startup message from server */ if (cin) (void)fclose(cin); if (cout) (void)fclose(cout); code = -1; goto bad; } if (setsockopt(s, SOL_SOCKET, SO_OOBINLINE, (void *)&on, sizeof(on)) == -1) { DWARN("setsockopt %s (ignored)", "SO_OOBINLINE"); } return (hostname); bad: (void)close(s); return (NULL); } void cmdabort(int notused) { int oerrno = errno; sigint_raised = 1; alarmtimer(0); if (fromatty) write(fileno(ttyout), "\n", 1); abrtflag++; if (ptflag) siglongjmp(ptabort, 1); errno = oerrno; } void cmdtimeout(int notused) { int oerrno = errno; alarmtimer(0); if (fromatty) write(fileno(ttyout), "\n", 1); timeoutflag++; if (ptflag) siglongjmp(ptabort, 1); errno = oerrno; } /*VARARGS*/ int command(const char *fmt, ...) { va_list ap; int r; sigfunc oldsigint; #ifndef NO_DEBUG if (ftp_debug) { fputs("---> ", ttyout); va_start(ap, fmt); if (strncmp("PASS ", fmt, 5) == 0) fputs("PASS XXXX", ttyout); else if (strncmp("ACCT ", fmt, 5) == 0) fputs("ACCT XXXX", ttyout); else vfprintf(ttyout, fmt, ap); va_end(ap); putc('\n', ttyout); } #endif if (cout == NULL) { warnx("No control connection for command"); code = -1; return (0); } abrtflag = 0; oldsigint = xsignal(SIGINT, cmdabort); va_start(ap, fmt); vfprintf(cout, fmt, ap); va_end(ap); fputs("\r\n", cout); (void)fflush(cout); cpend = 1; r = getreply(!strcmp(fmt, "QUIT")); if (abrtflag && oldsigint != SIG_IGN) (*oldsigint)(SIGINT); (void)xsignal(SIGINT, oldsigint); return (r); } static const char *m421[] = { "remote server timed out. Connection closed", "user interrupt. Connection closed", "remote server has closed connection", }; int getreply(int expecteof) { char current_line[BUFSIZ]; /* last line of previous reply */ int c, n, lineno; int dig; int originalcode = 0, continuation = 0; sigfunc oldsigint, oldsigalrm; int pflag = 0; char *cp, *pt = pasv; abrtflag = 0; timeoutflag = 0; oldsigint = xsignal(SIGINT, cmdabort); oldsigalrm = xsignal(SIGALRM, cmdtimeout); for (lineno = 0 ;; lineno++) { dig = n = code = 0; cp = current_line; while (alarmtimer(quit_time ? quit_time : 60), ((c = getc(cin)) != '\n')) { if (c == IAC) { /* handle telnet commands */ switch (c = getc(cin)) { case WILL: case WONT: c = getc(cin); fprintf(cout, "%c%c%c", IAC, DONT, c); (void)fflush(cout); break; case DO: case DONT: c = getc(cin); fprintf(cout, "%c%c%c", IAC, WONT, c); (void)fflush(cout); break; default: break; } continue; } dig++; if (c == EOF) { /* * these will get trashed by pswitch() * in lostpeer() */ int reply_timeoutflag = timeoutflag; int reply_abrtflag = abrtflag; alarmtimer(0); if (expecteof && feof(cin)) { (void)xsignal(SIGINT, oldsigint); (void)xsignal(SIGALRM, oldsigalrm); code = 221; return (0); } cpend = 0; lostpeer(0); if (verbose) { size_t midx; if (reply_timeoutflag) midx = 0; else if (reply_abrtflag) midx = 1; else midx = 2; (void)fprintf(ttyout, "421 Service not available, %s.\n", m421[midx]); (void)fflush(ttyout); } code = 421; (void)xsignal(SIGINT, oldsigint); (void)xsignal(SIGALRM, oldsigalrm); return (4); } if (c != '\r' && (verbose > 0 || ((verbose > -1 && n == '5' && dig > 4) && (((!n && c < '5') || (n && n < '5')) || !retry_connect)))) { if (proxflag && (dig == 1 || (dig == 5 && verbose == 0))) fprintf(ttyout, "%s:", hostname); (void)putc(c, ttyout); } if (dig < 4 && isdigit(c)) code = code * 10 + (c - '0'); if (!pflag && (code == 227 || code == 228)) pflag = 1; else if (!pflag && code == 229) pflag = 100; if (dig > 4 && pflag == 1 && isdigit(c)) pflag = 2; if (pflag == 2) { if (c != '\r' && c != ')') { if (pt < &pasv[sizeof(pasv) - 1]) *pt++ = c; } else { *pt = '\0'; pflag = 3; } } if (pflag == 100 && c == '(') pflag = 2; if (dig == 4 && c == '-') { if (continuation) code = 0; continuation++; } if (n == 0) n = c; if (cp < ¤t_line[sizeof(current_line) - 1]) *cp++ = c; } if (verbose > 0 || ((verbose > -1 && n == '5') && (n < '5' || !retry_connect))) { (void)putc(c, ttyout); (void)fflush(ttyout); } if (cp[-1] == '\r') cp[-1] = '\0'; *cp = '\0'; if (lineno == 0) (void)strlcpy(reply_string, current_line, sizeof(reply_string)); if (lineno > 0 && code == 0 && reply_callback != NULL) (*reply_callback)(current_line); if (continuation && code != originalcode) { if (originalcode == 0) originalcode = code; continue; } if (n != '1') cpend = 0; alarmtimer(0); (void)xsignal(SIGINT, oldsigint); (void)xsignal(SIGALRM, oldsigalrm); if (code == 421 || originalcode == 421) lostpeer(0); if (abrtflag && oldsigint != cmdabort && oldsigint != SIG_IGN) (*oldsigint)(SIGINT); if (timeoutflag && oldsigalrm != cmdtimeout && oldsigalrm != SIG_IGN) (*oldsigalrm)(SIGINT); return (n - '0'); } } static int empty(FILE *ecin, FILE *din, int sec) { int nr, nfd; struct pollfd pfd[2]; nfd = 0; if (ecin) { pfd[nfd].fd = fileno(ecin); pfd[nfd++].events = POLLIN; } if (din) { pfd[nfd].fd = fileno(din); pfd[nfd++].events = POLLIN; } if ((nr = ftp_poll(pfd, nfd, sec * 1000)) <= 0) return nr; nr = 0; nfd = 0; if (ecin) nr |= (pfd[nfd++].revents & POLLIN) ? 1 : 0; if (din) nr |= (pfd[nfd++].revents & POLLIN) ? 2 : 0; return nr; } sigjmp_buf xferabort; void abortxfer(int notused) { char msgbuf[100]; size_t len; sigint_raised = 1; alarmtimer(0); mflag = 0; abrtflag = 0; switch (direction[0]) { case 'r': strlcpy(msgbuf, "\nreceive", sizeof(msgbuf)); break; case 's': strlcpy(msgbuf, "\nsend", sizeof(msgbuf)); break; default: errx(1, "abortxfer: unknown direction `%s'", direction); } len = strlcat(msgbuf, " aborted. Waiting for remote to finish abort.\n", sizeof(msgbuf)); write(fileno(ttyout), msgbuf, len); siglongjmp(xferabort, 1); } /* * Read data from infd & write to outfd, using buf/bufsize as the temporary * buffer, dealing with short writes. * If rate_limit != 0, rate-limit the transfer. * If hash_interval != 0, fputc('c', ttyout) every hash_interval bytes. * Updates global variables: bytes. * Returns 0 if ok, 1 if there was a read error, 2 if there was a write error. * In the case of error, errno contains the appropriate error code. */ static int copy_bytes(int infd, int outfd, char *buf, size_t bufsize, int rate_limit, int hash_interval) { volatile off_t hashc; ssize_t inc, outc; char *bufp; struct timeval tvthen, tvnow, tvdiff; off_t bufrem, bufchunk; int serr; hashc = hash_interval; if (rate_limit) bufchunk = rate_limit; else bufchunk = bufsize; while (1) { if (rate_limit) { (void)gettimeofday(&tvthen, NULL); } errno = 0; inc = outc = 0; /* copy bufchunk at a time */ bufrem = bufchunk; while (bufrem > 0) { inc = read(infd, buf, MIN((off_t)bufsize, bufrem)); if (inc <= 0) goto copy_done; bytes += inc; bufrem -= inc; bufp = buf; while (inc > 0) { outc = write(outfd, bufp, inc); if (outc < 0) goto copy_done; inc -= outc; bufp += outc; } if (hash_interval) { while (bytes >= hashc) { (void)putc('#', ttyout); hashc += hash_interval; } (void)fflush(ttyout); } } if (rate_limit) { /* rate limited; wait if necessary */ while (1) { (void)gettimeofday(&tvnow, NULL); timersub(&tvnow, &tvthen, &tvdiff); if (tvdiff.tv_sec > 0) break; usleep(1000000 - tvdiff.tv_usec); } } } copy_done: serr = errno; if (hash_interval && bytes > 0) { if (bytes < hash_interval) (void)putc('#', ttyout); (void)putc('\n', ttyout); (void)fflush(ttyout); } errno = serr; if (inc == -1) return 1; if (outc == -1) return 2; return 0; } void sendrequest(const char *cmd, const char *local, const char *remote, int printnames) { struct stat st; int c; FILE *volatile fin; FILE *volatile dout; int (*volatile closefunc)(FILE *); sigfunc volatile oldintr; sigfunc volatile oldintp; off_t volatile hashbytes; int hash_interval; const char *lmode; static size_t bufsize; static char *buf; int oprogress; hashbytes = mark; direction = "sent"; dout = NULL; bytes = 0; filesize = -1; oprogress = progress; if (verbose && printnames) { if (*local != '-') fprintf(ttyout, "local: %s ", local); if (remote) fprintf(ttyout, "remote: %s\n", remote); } if (proxy) { proxtrans(cmd, local, remote); return; } if (curtype != type) changetype(type, 0); closefunc = NULL; oldintr = NULL; oldintp = NULL; lmode = "w"; if (sigsetjmp(xferabort, 1)) { while (cpend) (void)getreply(0); code = -1; goto cleanupsend; } (void)xsignal(SIGQUIT, psummary); oldintr = xsignal(SIGINT, abortxfer); if (strcmp(local, "-") == 0) { fin = stdin; progress = 0; } else if (*local == '|') { oldintp = xsignal(SIGPIPE, SIG_IGN); fin = popen(local + 1, "r"); if (fin == NULL) { warn("Can't execute `%s'", local + 1); code = -1; goto cleanupsend; } progress = 0; closefunc = pclose; } else { fin = fopen(local, "r"); if (fin == NULL) { warn("Can't open `%s'", local); code = -1; goto cleanupsend; } closefunc = fclose; if (fstat(fileno(fin), &st) < 0 || !S_ISREG(st.st_mode)) { fprintf(ttyout, "%s: not a plain file.\n", local); code = -1; goto cleanupsend; } filesize = st.st_size; } if (initconn()) { code = -1; goto cleanupsend; } if (sigsetjmp(xferabort, 1)) goto abort; if (restart_point && (strcmp(cmd, "STOR") == 0 || strcmp(cmd, "APPE") == 0)) { int rc; rc = -1; switch (curtype) { case TYPE_A: rc = fseeko(fin, restart_point, SEEK_SET); break; case TYPE_I: case TYPE_L: rc = lseek(fileno(fin), restart_point, SEEK_SET); break; } if (rc < 0) { warn("Can't seek to restart `%s'", local); goto cleanupsend; } if (command("REST " LLF, (LLT)restart_point) != CONTINUE) goto cleanupsend; lmode = "r+"; } if (remote) { if (command("%s %s", cmd, remote) != PRELIM) goto cleanupsend; } else { if (command("%s", cmd) != PRELIM) goto cleanupsend; } dirchange = 1; dout = dataconn(lmode); if (dout == NULL) goto abort; if ((size_t)sndbuf_size > bufsize) { if (buf) (void)free(buf); bufsize = sndbuf_size; buf = ftp_malloc(bufsize); } progressmeter(-1); oldintp = xsignal(SIGPIPE, SIG_IGN); hash_interval = (hash && (!progress || filesize < 0)) ? mark : 0; switch (curtype) { case TYPE_I: case TYPE_L: c = copy_bytes(fileno(fin), fileno(dout), buf, bufsize, rate_put, hash_interval); if (c == 1) { warn("Reading `%s'", local); } else if (c == 2) { if (errno != EPIPE) warn("Writing to network"); bytes = -1; } break; case TYPE_A: while ((c = getc(fin)) != EOF) { if (c == '\n') { while (hash_interval && bytes >= hashbytes) { (void)putc('#', ttyout); (void)fflush(ttyout); hashbytes += mark; } if (ferror(dout)) break; (void)putc('\r', dout); bytes++; } (void)putc(c, dout); bytes++; #if 0 /* this violates RFC0959 */ if (c == '\r') { (void)putc('\0', dout); bytes++; } #endif } if (hash_interval) { if (bytes < hashbytes) (void)putc('#', ttyout); (void)putc('\n', ttyout); } if (ferror(fin)) warn("Reading `%s'", local); if (ferror(dout)) { if (errno != EPIPE) warn("Writing to network"); bytes = -1; } break; } progressmeter(1); if (closefunc != NULL) { (*closefunc)(fin); fin = NULL; } (void)fclose(dout); dout = NULL; (void)getreply(0); if (bytes > 0) ptransfer(0); goto cleanupsend; abort: (void)xsignal(SIGINT, oldintr); oldintr = NULL; if (!cpend) { code = -1; goto cleanupsend; } if (data >= 0) { (void)close(data); data = -1; } if (dout) { (void)fclose(dout); dout = NULL; } (void)getreply(0); code = -1; if (bytes > 0) ptransfer(0); cleanupsend: if (oldintr) (void)xsignal(SIGINT, oldintr); if (oldintp) (void)xsignal(SIGPIPE, oldintp); if (data >= 0) { (void)close(data); data = -1; } if (closefunc != NULL && fin != NULL) (*closefunc)(fin); if (dout) (void)fclose(dout); progress = oprogress; restart_point = 0; bytes = 0; } void recvrequest(const char *cmd, const char *volatile local, const char *remote, const char *lmode, int printnames, int ignorespecial) { FILE *volatile fout; FILE *volatile din; int (*volatile closefunc)(FILE *); sigfunc volatile oldintr; sigfunc volatile oldintp; int c, d; int volatile is_retr; int volatile tcrflag; int volatile bare_lfs; static size_t bufsize; static char *buf; off_t volatile hashbytes; int hash_interval; struct stat st; time_t mtime; struct timeval tval[2]; int oprogress; int opreserve; fout = NULL; din = NULL; hashbytes = mark; direction = "received"; bytes = 0; bare_lfs = 0; filesize = -1; oprogress = progress; opreserve = preserve; is_retr = (strcmp(cmd, "RETR") == 0); if (is_retr && verbose && printnames) { if (ignorespecial || *local != '-') fprintf(ttyout, "local: %s ", local); if (remote) fprintf(ttyout, "remote: %s\n", remote); } if (proxy && is_retr) { proxtrans(cmd, local, remote); return; } closefunc = NULL; oldintr = NULL; oldintp = NULL; tcrflag = !crflag && is_retr; if (sigsetjmp(xferabort, 1)) { while (cpend) (void)getreply(0); code = -1; goto cleanuprecv; } (void)xsignal(SIGQUIT, psummary); oldintr = xsignal(SIGINT, abortxfer); if (ignorespecial || (strcmp(local, "-") && *local != '|')) { if (access(local, W_OK) < 0) { char *dir = strrchr(local, '/'); if (errno != ENOENT && errno != EACCES) { warn("Can't access `%s'", local); code = -1; goto cleanuprecv; } if (dir != NULL) *dir = 0; d = access(dir == local ? "/" : dir ? local : ".", W_OK); if (dir != NULL) *dir = '/'; if (d < 0) { warn("Can't access `%s'", local); code = -1; goto cleanuprecv; } if (!runique && errno == EACCES && chmod(local, (S_IRUSR|S_IWUSR)) < 0) { warn("Can't chmod `%s'", local); code = -1; goto cleanuprecv; } if (runique && errno == EACCES && (local = gunique(local)) == NULL) { code = -1; goto cleanuprecv; } } else if (runique && (local = gunique(local)) == NULL) { code = -1; goto cleanuprecv; } } if (!is_retr) { if (curtype != TYPE_A) changetype(TYPE_A, 0); } else { if (curtype != type) changetype(type, 0); filesize = remotesize(remote, 0); if (code == 421 || code == -1) goto cleanuprecv; } if (initconn()) { code = -1; goto cleanuprecv; } if (sigsetjmp(xferabort, 1)) goto abort; if (is_retr && restart_point && command("REST " LLF, (LLT) restart_point) != CONTINUE) goto cleanuprecv; if (! EMPTYSTRING(remote)) { if (command("%s %s", cmd, remote) != PRELIM) goto cleanuprecv; } else { if (command("%s", cmd) != PRELIM) goto cleanuprecv; } din = dataconn("r"); if (din == NULL) goto abort; if (!ignorespecial && strcmp(local, "-") == 0) { fout = stdout; progress = 0; preserve = 0; } else if (!ignorespecial && *local == '|') { oldintp = xsignal(SIGPIPE, SIG_IGN); fout = popen(local + 1, "w"); if (fout == NULL) { warn("Can't execute `%s'", local+1); goto abort; } progress = 0; preserve = 0; closefunc = pclose; } else { fout = fopen(local, lmode); if (fout == NULL) { warn("Can't open `%s'", local); goto abort; } closefunc = fclose; } if (fstat(fileno(fout), &st) != -1 && !S_ISREG(st.st_mode)) { progress = 0; preserve = 0; } if ((size_t)rcvbuf_size > bufsize) { if (buf) (void)free(buf); bufsize = rcvbuf_size; buf = ftp_malloc(bufsize); } progressmeter(-1); hash_interval = (hash && (!progress || filesize < 0)) ? mark : 0; switch (curtype) { case TYPE_I: case TYPE_L: if (is_retr && restart_point && lseek(fileno(fout), restart_point, SEEK_SET) < 0) { warn("Can't seek to restart `%s'", local); goto cleanuprecv; } c = copy_bytes(fileno(din), fileno(fout), buf, bufsize, rate_get, hash_interval); if (c == 1) { if (errno != EPIPE) warn("Reading from network"); bytes = -1; } else if (c == 2) { warn("Writing `%s'", local); } break; case TYPE_A: if (is_retr && restart_point) { int ch; off_t i; if (fseeko(fout, (off_t)0, SEEK_SET) < 0) goto done; for (i = 0; i++ < restart_point;) { if ((ch = getc(fout)) == EOF) goto done; if (ch == '\n') i++; } if (fseeko(fout, (off_t)0, SEEK_CUR) < 0) { done: warn("Can't seek to restart `%s'", local); goto cleanuprecv; } } while ((c = getc(din)) != EOF) { if (c == '\n') bare_lfs++; while (c == '\r') { while (hash_interval && bytes >= hashbytes) { (void)putc('#', ttyout); (void)fflush(ttyout); hashbytes += mark; } bytes++; if ((c = getc(din)) != '\n' || tcrflag) { if (ferror(fout)) goto break2; (void)putc('\r', fout); if (c == '\0') { bytes++; goto contin2; } if (c == EOF) goto contin2; } } (void)putc(c, fout); bytes++; contin2: ; } break2: if (hash_interval) { if (bytes < hashbytes) (void)putc('#', ttyout); (void)putc('\n', ttyout); } if (ferror(din)) { if (errno != EPIPE) warn("Reading from network"); bytes = -1; } if (ferror(fout)) warn("Writing `%s'", local); break; } progressmeter(1); if (closefunc != NULL) { (*closefunc)(fout); fout = NULL; } (void)fclose(din); din = NULL; (void)getreply(0); if (bare_lfs) { fprintf(ttyout, "WARNING! %d bare linefeeds received in ASCII mode.\n", bare_lfs); fputs("File may not have transferred correctly.\n", ttyout); } if (bytes >= 0 && is_retr) { if (bytes > 0) ptransfer(0); if (preserve && (closefunc == fclose)) { mtime = remotemodtime(remote, 0); if (mtime != -1) { (void)gettimeofday(&tval[0], NULL); tval[1].tv_sec = mtime; tval[1].tv_usec = 0; if (utimes(local, tval) == -1) { fprintf(ttyout, "Can't change modification time on %s to %s", local, rfc2822time(localtime(&mtime))); } } } } goto cleanuprecv; abort: /* * abort using RFC0959 recommended IP,SYNC sequence */ if (! sigsetjmp(xferabort, 1)) { /* this is the first call */ (void)xsignal(SIGINT, abort_squared); if (!cpend) { code = -1; goto cleanuprecv; } abort_remote(din); } code = -1; if (bytes > 0) ptransfer(0); cleanuprecv: if (oldintr) (void)xsignal(SIGINT, oldintr); if (oldintp) (void)xsignal(SIGPIPE, oldintp); if (data >= 0) { (void)close(data); data = -1; } if (closefunc != NULL && fout != NULL) (*closefunc)(fout); if (din) (void)fclose(din); progress = oprogress; preserve = opreserve; bytes = 0; } /* * Need to start a listen on the data channel before we send the command, * otherwise the server's connect may fail. */ int initconn(void) { char *p, *a; int result, tmpno = 0; int on = 1; int error; unsigned int addr[16], port[2]; unsigned int af, hal, pal; socklen_t len; const char *pasvcmd = NULL; int overbose; #ifdef INET6 #ifndef NO_DEBUG if (myctladdr.su_family == AF_INET6 && ftp_debug && (IN6_IS_ADDR_LINKLOCAL(&myctladdr.si_su.su_sin6.sin6_addr) || IN6_IS_ADDR_SITELOCAL(&myctladdr.si_su.su_sin6.sin6_addr))) { warnx("Use of scoped addresses can be troublesome"); } #endif #endif reinit: if (passivemode) { data_addr = myctladdr; data = socket(data_addr.su_family, SOCK_STREAM, 0); if (data < 0) { warn("Can't create socket for data connection"); return (1); } if ((options & SO_DEBUG) && setsockopt(data, SOL_SOCKET, SO_DEBUG, (void *)&on, sizeof(on)) == -1) { DWARN("setsockopt %s (ignored)", "SO_DEBUG"); } result = COMPLETE + 1; switch (data_addr.su_family) { case AF_INET: if (epsv4 && !epsv4bad) { pasvcmd = "EPSV"; overbose = verbose; if (ftp_debug == 0) verbose = -1; result = command("EPSV"); verbose = overbose; if (verbose > 0 && (result == COMPLETE || !connected)) fprintf(ttyout, "%s\n", reply_string); if (!connected) return (1); /* * this code is to be friendly with broken * BSDI ftpd */ if (code / 10 == 22 && code != 229) { fputs( "wrong server: return code must be 229\n", ttyout); result = COMPLETE + 1; } if (result != COMPLETE) { epsv4bad = 1; DPRINTF("disabling epsv4 for this " "connection\n"); } } if (result != COMPLETE) { pasvcmd = "PASV"; result = command("PASV"); if (!connected) return (1); } break; #ifdef INET6 case AF_INET6: if (epsv6 && !epsv6bad) { pasvcmd = "EPSV"; overbose = verbose; if (ftp_debug == 0) verbose = -1; result = command("EPSV"); verbose = overbose; if (verbose > 0 && (result == COMPLETE || !connected)) fprintf(ttyout, "%s\n", reply_string); if (!connected) return (1); /* * this code is to be friendly with * broken BSDI ftpd */ if (code / 10 == 22 && code != 229) { fputs( "wrong server: return code must be 229\n", ttyout); result = COMPLETE + 1; } if (result != COMPLETE) { epsv6bad = 1; DPRINTF("disabling epsv6 for this " "connection\n"); } } if (result != COMPLETE) { pasvcmd = "LPSV"; result = command("LPSV"); } if (!connected) return (1); break; #endif default: result = COMPLETE + 1; break; } if (result != COMPLETE) { if (activefallback) { (void)close(data); data = -1; passivemode = 0; #if 0 activefallback = 0; #endif goto reinit; } fputs("Passive mode refused.\n", ttyout); goto bad; } #define pack2(var, off) \ (((var[(off) + 0] & 0xff) << 8) | ((var[(off) + 1] & 0xff) << 0)) #define pack4(var, off) \ (((var[(off) + 0] & 0xff) << 24) | ((var[(off) + 1] & 0xff) << 16) | \ ((var[(off) + 2] & 0xff) << 8) | ((var[(off) + 3] & 0xff) << 0)) #define UC(b) (((int)b)&0xff) /* * What we've got at this point is a string of comma separated * one-byte unsigned integer values, separated by commas. */ if (strcmp(pasvcmd, "PASV") == 0) { if (data_addr.su_family != AF_INET) { fputs( "Passive mode AF mismatch. Shouldn't happen!\n", ttyout); error = 1; goto bad; } if (code / 10 == 22 && code != 227) { fputs("wrong server: return code must be 227\n", ttyout); error = 1; goto bad; } error = sscanf(pasv, "%u,%u,%u,%u,%u,%u", &addr[0], &addr[1], &addr[2], &addr[3], &port[0], &port[1]); if (error != 6) { fputs( "Passive mode address scan failure. Shouldn't happen!\n", ttyout); error = 1; goto bad; } error = 0; memset(&data_addr, 0, sizeof(data_addr)); data_addr.su_family = AF_INET; data_addr.su_len = sizeof(struct sockaddr_in); data_addr.si_su.su_sin.sin_addr.s_addr = htonl(pack4(addr, 0)); data_addr.su_port = htons(pack2(port, 0)); } else if (strcmp(pasvcmd, "LPSV") == 0) { if (code / 10 == 22 && code != 228) { fputs("wrong server: return code must be 228\n", ttyout); error = 1; goto bad; } switch (data_addr.su_family) { case AF_INET: error = sscanf(pasv, "%u,%u,%u,%u,%u,%u,%u,%u,%u", &af, &hal, &addr[0], &addr[1], &addr[2], &addr[3], &pal, &port[0], &port[1]); if (error != 9) { fputs( "Passive mode address scan failure. Shouldn't happen!\n", ttyout); error = 1; goto bad; } if (af != 4 || hal != 4 || pal != 2) { fputs( "Passive mode AF mismatch. Shouldn't happen!\n", ttyout); error = 1; goto bad; } error = 0; memset(&data_addr, 0, sizeof(data_addr)); data_addr.su_family = AF_INET; data_addr.su_len = sizeof(struct sockaddr_in); data_addr.si_su.su_sin.sin_addr.s_addr = htonl(pack4(addr, 0)); data_addr.su_port = htons(pack2(port, 0)); break; #ifdef INET6 case AF_INET6: error = sscanf(pasv, "%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u", &af, &hal, &addr[0], &addr[1], &addr[2], &addr[3], &addr[4], &addr[5], &addr[6], &addr[7], &addr[8], &addr[9], &addr[10], &addr[11], &addr[12], &addr[13], &addr[14], &addr[15], &pal, &port[0], &port[1]); if (error != 21) { fputs( "Passive mode address scan failure. Shouldn't happen!\n", ttyout); error = 1; goto bad; } if (af != 6 || hal != 16 || pal != 2) { fputs( "Passive mode AF mismatch. Shouldn't happen!\n", ttyout); error = 1; goto bad; } error = 0; memset(&data_addr, 0, sizeof(data_addr)); data_addr.su_family = AF_INET6; data_addr.su_len = sizeof(struct sockaddr_in6); { size_t i; for (i = 0; i < sizeof(struct in6_addr); i++) { data_addr.si_su.su_sin6.sin6_addr.s6_addr[i] = UC(addr[i]); } } data_addr.su_port = htons(pack2(port, 0)); break; #endif default: error = 1; } } else if (strcmp(pasvcmd, "EPSV") == 0) { char delim[4]; port[0] = 0; if (code / 10 == 22 && code != 229) { fputs("wrong server: return code must be 229\n", ttyout); error = 1; goto bad; } if (sscanf(pasv, "%c%c%c%d%c", &delim[0], &delim[1], &delim[2], &port[1], &delim[3]) != 5) { fputs("parse error!\n", ttyout); error = 1; goto bad; } if (delim[0] != delim[1] || delim[0] != delim[2] || delim[0] != delim[3]) { fputs("parse error!\n", ttyout); error = 1; goto bad; } data_addr = hisctladdr; data_addr.su_port = htons(port[1]); } else goto bad; if (ftp_connect(data, (struct sockaddr *)&data_addr.si_su, data_addr.su_len) < 0) { if (activefallback) { (void)close(data); data = -1; passivemode = 0; #if 0 activefallback = 0; #endif goto reinit; } goto bad; } #ifdef IPTOS_THROUGHPUT if (data_addr.su_family == AF_INET) { on = IPTOS_THROUGHPUT; if (setsockopt(data, IPPROTO_IP, IP_TOS, (void *)&on, sizeof(on)) == -1) { DWARN("setsockopt %s (ignored)", "IPTOS_THROUGHPUT"); } } #endif return (0); } noport: data_addr = myctladdr; if (sendport) data_addr.su_port = 0; /* let system pick one */ if (data != -1) (void)close(data); data = socket(data_addr.su_family, SOCK_STREAM, 0); if (data < 0) { warn("Can't create socket for data connection"); if (tmpno) sendport = 1; return (1); } if (!sendport) if (setsockopt(data, SOL_SOCKET, SO_REUSEADDR, (void *)&on, sizeof(on)) == -1) { warn("Can't set SO_REUSEADDR on data connection"); goto bad; } if (bind(data, (struct sockaddr *)&data_addr.si_su, data_addr.su_len) < 0) { warn("Can't bind for data connection"); goto bad; } if ((options & SO_DEBUG) && setsockopt(data, SOL_SOCKET, SO_DEBUG, (void *)&on, sizeof(on)) == -1) { DWARN("setsockopt %s (ignored)", "SO_DEBUG"); } len = sizeof(data_addr.si_su); memset((char *)&data_addr, 0, sizeof (data_addr)); if (getsockname(data, (struct sockaddr *)&data_addr.si_su, &len) == -1) { warn("Can't determine my address of data connection"); goto bad; } data_addr.su_len = len; if (ftp_listen(data, 1) < 0) warn("Can't listen to data connection"); if (sendport) { char hname[NI_MAXHOST], sname[NI_MAXSERV]; struct sockinet tmp; switch (data_addr.su_family) { case AF_INET: if (!epsv4 || epsv4bad) { result = COMPLETE + 1; break; } /* FALLTHROUGH */ #ifdef INET6 case AF_INET6: if (!epsv6 || epsv6bad) { result = COMPLETE + 1; break; } #endif af = (data_addr.su_family == AF_INET) ? 1 : 2; tmp = data_addr; #ifdef INET6 if (tmp.su_family == AF_INET6) tmp.si_su.su_sin6.sin6_scope_id = 0; #endif if (getnameinfo((struct sockaddr *)&tmp.si_su, tmp.su_len, hname, sizeof(hname), sname, sizeof(sname), NI_NUMERICHOST | NI_NUMERICSERV)) { result = ERROR; } else { overbose = verbose; if (ftp_debug == 0) verbose = -1; result = command("EPRT |%u|%s|%s|", af, hname, sname); verbose = overbose; if (verbose > 0 && (result == COMPLETE || !connected)) fprintf(ttyout, "%s\n", reply_string); if (!connected) return (1); if (result != COMPLETE) { epsv4bad = 1; DPRINTF("disabling epsv4 for this " "connection\n"); } } break; default: result = COMPLETE + 1; break; } if (result == COMPLETE) goto skip_port; switch (data_addr.su_family) { case AF_INET: a = (char *)&data_addr.si_su.su_sin.sin_addr; p = (char *)&data_addr.su_port; result = command("PORT %d,%d,%d,%d,%d,%d", UC(a[0]), UC(a[1]), UC(a[2]), UC(a[3]), UC(p[0]), UC(p[1])); break; #ifdef INET6 case AF_INET6: a = (char *)&data_addr.si_su.su_sin6.sin6_addr; p = (char *)&data_addr.su_port; result = command( "LPRT %d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d", 6, 16, UC(a[0]),UC(a[1]),UC(a[2]),UC(a[3]), UC(a[4]),UC(a[5]),UC(a[6]),UC(a[7]), UC(a[8]),UC(a[9]),UC(a[10]),UC(a[11]), UC(a[12]),UC(a[13]),UC(a[14]),UC(a[15]), 2, UC(p[0]), UC(p[1])); break; #endif default: result = COMPLETE + 1; /* xxx */ } if (!connected) return (1); skip_port: if (result == ERROR && sendport == -1) { sendport = 0; tmpno = 1; goto noport; } return (result != COMPLETE); } if (tmpno) sendport = 1; #ifdef IPTOS_THROUGHPUT if (data_addr.su_family == AF_INET) { on = IPTOS_THROUGHPUT; if (setsockopt(data, IPPROTO_IP, IP_TOS, (void *)&on, sizeof(on)) == -1) { DWARN("setsockopt %s (ignored)", "IPTOS_THROUGHPUT"); } } #endif return (0); bad: (void)close(data); data = -1; if (tmpno) sendport = 1; return (1); } FILE * dataconn(const char *lmode) { struct sockinet from; int s, flags, rv, timeout; struct timeval endtime, now, td; struct pollfd pfd[1]; socklen_t fromlen; if (passivemode) /* passive data connection */ return (fdopen(data, lmode)); /* active mode data connection */ if ((flags = fcntl(data, F_GETFL, 0)) == -1) goto dataconn_failed; /* get current socket flags */ if (fcntl(data, F_SETFL, flags | O_NONBLOCK) == -1) goto dataconn_failed; /* set non-blocking connect */ /* NOTE: we now must restore socket flags on successful exit */ /* limit time waiting on listening socket */ pfd[0].fd = data; pfd[0].events = POLLIN; (void)gettimeofday(&endtime, NULL); /* determine end time */ endtime.tv_sec += (quit_time > 0) ? quit_time: 60; /* without -q, default to 60s */ do { (void)gettimeofday(&now, NULL); timersub(&endtime, &now, &td); timeout = td.tv_sec * 1000 + td.tv_usec/1000; if (timeout < 0) timeout = 0; rv = ftp_poll(pfd, 1, timeout); } while (rv == -1 && errno == EINTR); /* loop until poll ! EINTR */ if (rv == -1) { warn("Can't poll waiting before accept"); goto dataconn_failed; } if (rv == 0) { warnx("Poll timeout waiting before accept"); goto dataconn_failed; } /* (non-blocking) accept the connection */ fromlen = myctladdr.su_len; do { s = accept(data, (struct sockaddr *) &from.si_su, &fromlen); } while (s == -1 && errno == EINTR); /* loop until accept ! EINTR */ if (s == -1) { warn("Can't accept data connection"); goto dataconn_failed; } (void)close(data); data = s; if (fcntl(data, F_SETFL, flags) == -1) /* restore socket flags */ goto dataconn_failed; #ifdef IPTOS_THROUGHPUT if (from.su_family == AF_INET) { int tos = IPTOS_THROUGHPUT; if (setsockopt(s, IPPROTO_IP, IP_TOS, (void *)&tos, sizeof(tos)) == -1) { DWARN("setsockopt %s (ignored)", "IPTOS_THROUGHPUT"); } } #endif return (fdopen(data, lmode)); dataconn_failed: (void)close(data); data = -1; return (NULL); } void psabort(int notused) { int oerrno = errno; sigint_raised = 1; alarmtimer(0); abrtflag++; errno = oerrno; } void pswitch(int flag) { sigfunc oldintr; static struct comvars { int connect; char name[MAXHOSTNAMELEN]; struct sockinet mctl; struct sockinet hctl; FILE *in; FILE *out; int tpe; int curtpe; int cpnd; int sunqe; int runqe; int mcse; int ntflg; char nti[17]; char nto[17]; int mapflg; char mi[MAXPATHLEN]; char mo[MAXPATHLEN]; } proxstruct, tmpstruct; struct comvars *ip, *op; abrtflag = 0; oldintr = xsignal(SIGINT, psabort); if (flag) { if (proxy) return; ip = &tmpstruct; op = &proxstruct; proxy++; } else { if (!proxy) return; ip = &proxstruct; op = &tmpstruct; proxy = 0; } ip->connect = connected; connected = op->connect; if (hostname) (void)strlcpy(ip->name, hostname, sizeof(ip->name)); else ip->name[0] = '\0'; hostname = op->name; ip->hctl = hisctladdr; hisctladdr = op->hctl; ip->mctl = myctladdr; myctladdr = op->mctl; ip->in = cin; cin = op->in; ip->out = cout; cout = op->out; ip->tpe = type; type = op->tpe; ip->curtpe = curtype; curtype = op->curtpe; ip->cpnd = cpend; cpend = op->cpnd; ip->sunqe = sunique; sunique = op->sunqe; ip->runqe = runique; runique = op->runqe; ip->mcse = mcase; mcase = op->mcse; ip->ntflg = ntflag; ntflag = op->ntflg; (void)strlcpy(ip->nti, ntin, sizeof(ip->nti)); (void)strlcpy(ntin, op->nti, sizeof(ntin)); (void)strlcpy(ip->nto, ntout, sizeof(ip->nto)); (void)strlcpy(ntout, op->nto, sizeof(ntout)); ip->mapflg = mapflag; mapflag = op->mapflg; (void)strlcpy(ip->mi, mapin, sizeof(ip->mi)); (void)strlcpy(mapin, op->mi, sizeof(mapin)); (void)strlcpy(ip->mo, mapout, sizeof(ip->mo)); (void)strlcpy(mapout, op->mo, sizeof(mapout)); (void)xsignal(SIGINT, oldintr); if (abrtflag) { abrtflag = 0; (*oldintr)(SIGINT); } } void abortpt(int notused) { sigint_raised = 1; alarmtimer(0); if (fromatty) write(fileno(ttyout), "\n", 1); ptabflg++; mflag = 0; abrtflag = 0; siglongjmp(ptabort, 1); } void proxtrans(const char *cmd, const char *local, const char *remote) { sigfunc volatile oldintr; int prox_type, nfnd; int volatile secndflag; const char *volatile cmd2; oldintr = NULL; secndflag = 0; if (strcmp(cmd, "RETR")) cmd2 = "RETR"; else cmd2 = runique ? "STOU" : "STOR"; if ((prox_type = type) == 0) { if (unix_server && unix_proxy) prox_type = TYPE_I; else prox_type = TYPE_A; } if (curtype != prox_type) changetype(prox_type, 1); if (command("PASV") != COMPLETE) { fputs("proxy server does not support third party transfers.\n", ttyout); return; } pswitch(0); if (!connected) { fputs("No primary connection.\n", ttyout); pswitch(1); code = -1; return; } if (curtype != prox_type) changetype(prox_type, 1); if (command("PORT %s", pasv) != COMPLETE) { pswitch(1); return; } if (sigsetjmp(ptabort, 1)) goto abort; oldintr = xsignal(SIGINT, abortpt); if ((restart_point && (command("REST " LLF, (LLT) restart_point) != CONTINUE)) || (command("%s %s", cmd, remote) != PRELIM)) { (void)xsignal(SIGINT, oldintr); pswitch(1); return; } sleep(2); pswitch(1); secndflag++; if ((restart_point && (command("REST " LLF, (LLT) restart_point) != CONTINUE)) || (command("%s %s", cmd2, local) != PRELIM)) goto abort; ptflag++; (void)getreply(0); pswitch(0); (void)getreply(0); (void)xsignal(SIGINT, oldintr); pswitch(1); ptflag = 0; fprintf(ttyout, "local: %s remote: %s\n", local, remote); return; abort: if (sigsetjmp(xferabort, 1)) { (void)xsignal(SIGINT, oldintr); return; } (void)xsignal(SIGINT, abort_squared); ptflag = 0; if (strcmp(cmd, "RETR") && !proxy) pswitch(1); else if (!strcmp(cmd, "RETR") && proxy) pswitch(0); if (!cpend && !secndflag) { /* only here if cmd = "STOR" (proxy=1) */ if (command("%s %s", cmd2, local) != PRELIM) { pswitch(0); if (cpend) abort_remote(NULL); } pswitch(1); if (ptabflg) code = -1; (void)xsignal(SIGINT, oldintr); return; } if (cpend) abort_remote(NULL); pswitch(!proxy); if (!cpend && !secndflag) { /* only if cmd = "RETR" (proxy=1) */ if (command("%s %s", cmd2, local) != PRELIM) { pswitch(0); if (cpend) abort_remote(NULL); pswitch(1); if (ptabflg) code = -1; (void)xsignal(SIGINT, oldintr); return; } } if (cpend) abort_remote(NULL); pswitch(!proxy); if (cpend) { if ((nfnd = empty(cin, NULL, 10)) <= 0) { if (nfnd < 0) warn("Error aborting proxy command"); if (ptabflg) code = -1; lostpeer(0); } (void)getreply(0); (void)getreply(0); } if (proxy) pswitch(0); pswitch(1); if (ptabflg) code = -1; (void)xsignal(SIGINT, oldintr); } void reset(int argc, char *argv[]) { int nfnd = 1; if (argc == 0 && argv != NULL) { UPRINTF("usage: %s\n", argv[0]); code = -1; return; } while (nfnd > 0) { if ((nfnd = empty(cin, NULL, 0)) < 0) { warn("Error resetting connection"); code = -1; lostpeer(0); } else if (nfnd) (void)getreply(0); } } char * gunique(const char *local) { static char new[MAXPATHLEN]; char *cp = strrchr(local, '/'); int d, count=0, len; char ext = '1'; if (cp) *cp = '\0'; d = access(cp == local ? "/" : cp ? local : ".", W_OK); if (cp) *cp = '/'; if (d < 0) { warn("Can't access `%s'", local); return (NULL); } len = strlcpy(new, local, sizeof(new)); cp = &new[len]; *cp++ = '.'; while (!d) { if (++count == 100) { fputs("runique: can't find unique file name.\n", ttyout); return (NULL); } *cp++ = ext; *cp = '\0'; if (ext == '9') ext = '0'; else ext++; if ((d = access(new, F_OK)) < 0) break; if (ext != '0') cp--; else if (*(cp - 2) == '.') *(cp - 1) = '1'; else { *(cp - 2) = *(cp - 2) + 1; cp--; } } return (new); } /* * abort_squared -- * aborts abort_remote(). lostpeer() is called because if the user is * too impatient to wait or there's another problem then ftp really * needs to get back to a known state. */ void abort_squared(int dummy) { char msgbuf[100]; size_t len; sigint_raised = 1; alarmtimer(0); len = strlcpy(msgbuf, "\nremote abort aborted; closing connection.\n", sizeof(msgbuf)); write(fileno(ttyout), msgbuf, len); lostpeer(0); siglongjmp(xferabort, 1); } void abort_remote(FILE *din) { - char buf[BUFSIZ]; + unsigned char buf[BUFSIZ]; int nfnd; if (cout == NULL) { warnx("Lost control connection for abort"); if (ptabflg) code = -1; lostpeer(0); return; } /* * send IAC in urgent mode instead of DM because 4.3BSD places oob mark * after urgent byte rather than before as is protocol now */ buf[0] = IAC; buf[1] = IP; buf[2] = IAC; if (send(fileno(cout), buf, 3, MSG_OOB) != 3) warn("Can't send abort message"); fprintf(cout, "%cABOR\r\n", DM); (void)fflush(cout); if ((nfnd = empty(cin, din, 10)) <= 0) { if (nfnd < 0) warn("Can't send abort message"); if (ptabflg) code = -1; lostpeer(0); } if (din && (nfnd & 2)) { while (read(fileno(din), buf, BUFSIZ) > 0) continue; } if (getreply(0) == ERROR && code == 552) { /* 552 needed for nic style abort */ (void)getreply(0); } (void)getreply(0); } /* * Ensure that ai->ai_addr is NOT an IPv4 mapped address. * IPv4 mapped address complicates too many things in FTP * protocol handling, as FTP protocol is defined differently * between IPv4 and IPv6. * * This may not be the best way to handle this situation, * since the semantics of IPv4 mapped address is defined in * the kernel. There are configurations where we should use * IPv4 mapped address as native IPv6 address, not as * "an IPv6 address that embeds IPv4 address" (namely, SIIT). * * More complete solution would be to have an additional * getsockopt to grab "real" peername/sockname. "real" * peername/sockname will be AF_INET if IPv4 mapped address * is used to embed IPv4 address, and will be AF_INET6 if * we use it as native. What a mess! */ void ai_unmapped(struct addrinfo *ai) { #ifdef INET6 struct sockaddr_in6 *sin6; struct sockaddr_in sin; socklen_t len; if (ai->ai_family != AF_INET6) return; if (ai->ai_addrlen != sizeof(struct sockaddr_in6) || sizeof(sin) > ai->ai_addrlen) return; sin6 = (struct sockaddr_in6 *)ai->ai_addr; if (!IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)) return; memset(&sin, 0, sizeof(sin)); sin.sin_family = AF_INET; len = sizeof(struct sockaddr_in); memcpy(&sin.sin_addr, &sin6->sin6_addr.s6_addr[12], sizeof(sin.sin_addr)); sin.sin_port = sin6->sin6_port; ai->ai_family = AF_INET; #if defined(HAVE_STRUCT_SOCKADDR_IN_SIN_LEN) sin.sin_len = len; #endif memcpy(ai->ai_addr, &sin, len); ai->ai_addrlen = len; #endif } #ifdef NO_USAGE void xusage(void) { fputs("Usage error\n", ttyout); } #endif Index: projects/netbsd-tests-update-12/contrib/tnftp =================================================================== --- projects/netbsd-tests-update-12/contrib/tnftp (revision 305171) +++ projects/netbsd-tests-update-12/contrib/tnftp (revision 305172) Property changes on: projects/netbsd-tests-update-12/contrib/tnftp ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/contrib/tnftp:r303899-305170 Index: projects/netbsd-tests-update-12/crypto/openssh/auth-pam.c =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/auth-pam.c (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/auth-pam.c (revision 305172) @@ -1,1220 +1,1222 @@ /*- * Copyright (c) 2002 Networks Associates Technology, Inc. * All rights reserved. * * This software was developed for the FreeBSD Project by ThinkSec AS and * NAI Labs, the Security Research Division of Network Associates, Inc. * under DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the * DARPA CHATS research program. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * Copyright (c) 2003,2004 Damien Miller * Copyright (c) 2003,2004 Darren Tucker * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ /* Based on FreeBSD: src/crypto/openssh/auth2-pam-freebsd.c,v 1.11 2003/03/31 13:48:18 des */ #include "includes.h" #include #include #include #include #include #include #include #include #ifdef USE_PAM #if defined(HAVE_SECURITY_PAM_APPL_H) #include #elif defined (HAVE_PAM_PAM_APPL_H) #include #endif /* OpenGroup RFC86.0 and XSSO specify no "const" on arguments */ #ifdef PAM_SUN_CODEBASE # define sshpam_const /* Solaris, HP-UX, AIX */ #else # define sshpam_const const /* LinuxPAM, OpenPAM */ #endif /* Ambiguity in spec: is it an array of pointers or a pointer to an array? */ #ifdef PAM_SUN_CODEBASE # define PAM_MSG_MEMBER(msg, n, member) ((*(msg))[(n)].member) #else # define PAM_MSG_MEMBER(msg, n, member) ((msg)[(n)]->member) #endif #include "xmalloc.h" #include "buffer.h" #include "key.h" #include "hostfile.h" #include "auth.h" #include "auth-pam.h" #include "canohost.h" #include "log.h" #include "msg.h" #include "packet.h" #include "misc.h" #include "servconf.h" #include "ssh2.h" #include "auth-options.h" #ifdef GSSAPI #include "ssh-gss.h" #endif #include "monitor_wrap.h" +#include "blacklist_client.h" extern ServerOptions options; extern Buffer loginmsg; extern int compat20; extern u_int utmp_len; /* so we don't silently change behaviour */ #ifdef USE_POSIX_THREADS # error "USE_POSIX_THREADS replaced by UNSUPPORTED_POSIX_THREADS_HACK" #endif /* * Formerly known as USE_POSIX_THREADS, using this is completely unsupported * and generally a bad idea. Use at own risk and do not expect support if * this breaks. */ #ifdef UNSUPPORTED_POSIX_THREADS_HACK #include /* * Avoid namespace clash when *not* using pthreads for systems *with* * pthreads, which unconditionally define pthread_t via sys/types.h * (e.g. Linux) */ typedef pthread_t sp_pthread_t; #else typedef pid_t sp_pthread_t; #endif struct pam_ctxt { sp_pthread_t pam_thread; int pam_psock; int pam_csock; int pam_done; }; static void sshpam_free_ctx(void *); static struct pam_ctxt *cleanup_ctxt; #ifndef UNSUPPORTED_POSIX_THREADS_HACK /* * Simulate threads with processes. */ static int sshpam_thread_status = -1; static mysig_t sshpam_oldsig; static void sshpam_sigchld_handler(int sig) { signal(SIGCHLD, SIG_DFL); if (cleanup_ctxt == NULL) return; /* handler called after PAM cleanup, shouldn't happen */ if (waitpid(cleanup_ctxt->pam_thread, &sshpam_thread_status, WNOHANG) <= 0) { /* PAM thread has not exitted, privsep slave must have */ kill(cleanup_ctxt->pam_thread, SIGTERM); if (waitpid(cleanup_ctxt->pam_thread, &sshpam_thread_status, 0) <= 0) return; /* could not wait */ } if (WIFSIGNALED(sshpam_thread_status) && WTERMSIG(sshpam_thread_status) == SIGTERM) return; /* terminated by pthread_cancel */ if (!WIFEXITED(sshpam_thread_status)) sigdie("PAM: authentication thread exited unexpectedly"); if (WEXITSTATUS(sshpam_thread_status) != 0) sigdie("PAM: authentication thread exited uncleanly"); } /* ARGSUSED */ static void pthread_exit(void *value) { _exit(0); } /* ARGSUSED */ static int pthread_create(sp_pthread_t *thread, const void *attr, void *(*thread_start)(void *), void *arg) { pid_t pid; struct pam_ctxt *ctx = arg; sshpam_thread_status = -1; switch ((pid = fork())) { case -1: error("fork(): %s", strerror(errno)); return (-1); case 0: close(ctx->pam_psock); ctx->pam_psock = -1; thread_start(arg); _exit(1); default: *thread = pid; close(ctx->pam_csock); ctx->pam_csock = -1; sshpam_oldsig = signal(SIGCHLD, sshpam_sigchld_handler); return (0); } } static int pthread_cancel(sp_pthread_t thread) { signal(SIGCHLD, sshpam_oldsig); return (kill(thread, SIGTERM)); } /* ARGSUSED */ static int pthread_join(sp_pthread_t thread, void **value) { int status; if (sshpam_thread_status != -1) return (sshpam_thread_status); signal(SIGCHLD, sshpam_oldsig); waitpid(thread, &status, 0); return (status); } #endif static pam_handle_t *sshpam_handle = NULL; static int sshpam_err = 0; static int sshpam_authenticated = 0; static int sshpam_session_open = 0; static int sshpam_cred_established = 0; static int sshpam_account_status = -1; static char **sshpam_env = NULL; static Authctxt *sshpam_authctxt = NULL; static const char *sshpam_password = NULL; static char badpw[] = "\b\n\r\177INCORRECT"; /* Some PAM implementations don't implement this */ #ifndef HAVE_PAM_GETENVLIST static char ** pam_getenvlist(pam_handle_t *pamh) { /* * XXX - If necessary, we can still support envrionment passing * for platforms without pam_getenvlist by searching for known * env vars (e.g. KRB5CCNAME) from the PAM environment. */ return NULL; } #endif /* * Some platforms, notably Solaris, do not enforce password complexity * rules during pam_chauthtok() if the real uid of the calling process * is 0, on the assumption that it's being called by "passwd" run by root. * This wraps pam_chauthtok and sets/restore the real uid so PAM will do * the right thing. */ #ifdef SSHPAM_CHAUTHTOK_NEEDS_RUID static int sshpam_chauthtok_ruid(pam_handle_t *pamh, int flags) { int result; if (sshpam_authctxt == NULL) fatal("PAM: sshpam_authctxt not initialized"); if (setreuid(sshpam_authctxt->pw->pw_uid, -1) == -1) fatal("%s: setreuid failed: %s", __func__, strerror(errno)); result = pam_chauthtok(pamh, flags); if (setreuid(0, -1) == -1) fatal("%s: setreuid failed: %s", __func__, strerror(errno)); return result; } # define pam_chauthtok(a,b) (sshpam_chauthtok_ruid((a), (b))) #endif void sshpam_password_change_required(int reqd) { debug3("%s %d", __func__, reqd); if (sshpam_authctxt == NULL) fatal("%s: PAM authctxt not initialized", __func__); sshpam_authctxt->force_pwchange = reqd; if (reqd) { no_port_forwarding_flag |= 2; no_agent_forwarding_flag |= 2; no_x11_forwarding_flag |= 2; } else { no_port_forwarding_flag &= ~2; no_agent_forwarding_flag &= ~2; no_x11_forwarding_flag &= ~2; } } /* Import regular and PAM environment from subprocess */ static void import_environments(Buffer *b) { char *env; u_int i, num_env; int err; debug3("PAM: %s entering", __func__); #ifndef UNSUPPORTED_POSIX_THREADS_HACK /* Import variables set by do_pam_account */ sshpam_account_status = buffer_get_int(b); sshpam_password_change_required(buffer_get_int(b)); /* Import environment from subprocess */ num_env = buffer_get_int(b); if (num_env > 1024) fatal("%s: received %u environment variables, expected <= 1024", __func__, num_env); sshpam_env = xcalloc(num_env + 1, sizeof(*sshpam_env)); debug3("PAM: num env strings %d", num_env); for(i = 0; i < num_env; i++) sshpam_env[i] = buffer_get_string(b, NULL); sshpam_env[num_env] = NULL; /* Import PAM environment from subprocess */ num_env = buffer_get_int(b); debug("PAM: num PAM env strings %d", num_env); for(i = 0; i < num_env; i++) { env = buffer_get_string(b, NULL); #ifdef HAVE_PAM_PUTENV /* Errors are not fatal here */ if ((err = pam_putenv(sshpam_handle, env)) != PAM_SUCCESS) { error("PAM: pam_putenv: %s", pam_strerror(sshpam_handle, sshpam_err)); } #endif } #endif } /* * Conversation function for authentication thread. */ static int sshpam_thread_conv(int n, sshpam_const struct pam_message **msg, struct pam_response **resp, void *data) { Buffer buffer; struct pam_ctxt *ctxt; struct pam_response *reply; int i; debug3("PAM: %s entering, %d messages", __func__, n); *resp = NULL; if (data == NULL) { error("PAM: conversation function passed a null context"); return (PAM_CONV_ERR); } ctxt = data; if (n <= 0 || n > PAM_MAX_NUM_MSG) return (PAM_CONV_ERR); if ((reply = calloc(n, sizeof(*reply))) == NULL) return (PAM_CONV_ERR); buffer_init(&buffer); for (i = 0; i < n; ++i) { switch (PAM_MSG_MEMBER(msg, i, msg_style)) { case PAM_PROMPT_ECHO_OFF: buffer_put_cstring(&buffer, PAM_MSG_MEMBER(msg, i, msg)); if (ssh_msg_send(ctxt->pam_csock, PAM_MSG_MEMBER(msg, i, msg_style), &buffer) == -1) goto fail; if (ssh_msg_recv(ctxt->pam_csock, &buffer) == -1) goto fail; if (buffer_get_char(&buffer) != PAM_AUTHTOK) goto fail; reply[i].resp = buffer_get_string(&buffer, NULL); break; case PAM_PROMPT_ECHO_ON: buffer_put_cstring(&buffer, PAM_MSG_MEMBER(msg, i, msg)); if (ssh_msg_send(ctxt->pam_csock, PAM_MSG_MEMBER(msg, i, msg_style), &buffer) == -1) goto fail; if (ssh_msg_recv(ctxt->pam_csock, &buffer) == -1) goto fail; if (buffer_get_char(&buffer) != PAM_AUTHTOK) goto fail; reply[i].resp = buffer_get_string(&buffer, NULL); break; case PAM_ERROR_MSG: buffer_put_cstring(&buffer, PAM_MSG_MEMBER(msg, i, msg)); if (ssh_msg_send(ctxt->pam_csock, PAM_MSG_MEMBER(msg, i, msg_style), &buffer) == -1) goto fail; break; case PAM_TEXT_INFO: buffer_put_cstring(&buffer, PAM_MSG_MEMBER(msg, i, msg)); if (ssh_msg_send(ctxt->pam_csock, PAM_MSG_MEMBER(msg, i, msg_style), &buffer) == -1) goto fail; break; default: goto fail; } buffer_clear(&buffer); } buffer_free(&buffer); *resp = reply; return (PAM_SUCCESS); fail: for(i = 0; i < n; i++) { free(reply[i].resp); } free(reply); buffer_free(&buffer); return (PAM_CONV_ERR); } /* * Authentication thread. */ static void * sshpam_thread(void *ctxtp) { struct pam_ctxt *ctxt = ctxtp; Buffer buffer; struct pam_conv sshpam_conv; int flags = (options.permit_empty_passwd == 0 ? PAM_DISALLOW_NULL_AUTHTOK : 0); #ifndef UNSUPPORTED_POSIX_THREADS_HACK extern char **environ; char **env_from_pam; u_int i; const char *pam_user; const char **ptr_pam_user = &pam_user; char *tz = getenv("TZ"); sshpam_err = pam_get_item(sshpam_handle, PAM_USER, (sshpam_const void **)ptr_pam_user); if (sshpam_err != PAM_SUCCESS) goto auth_fail; environ[0] = NULL; if (tz != NULL) if (setenv("TZ", tz, 1) == -1) error("PAM: could not set TZ environment: %s", strerror(errno)); if (sshpam_authctxt != NULL) { setproctitle("%s [pam]", sshpam_authctxt->valid ? pam_user : "unknown"); } #endif sshpam_conv.conv = sshpam_thread_conv; sshpam_conv.appdata_ptr = ctxt; if (sshpam_authctxt == NULL) fatal("%s: PAM authctxt not initialized", __func__); buffer_init(&buffer); sshpam_err = pam_set_item(sshpam_handle, PAM_CONV, (const void *)&sshpam_conv); if (sshpam_err != PAM_SUCCESS) goto auth_fail; sshpam_err = pam_authenticate(sshpam_handle, flags); if (sshpam_err != PAM_SUCCESS) goto auth_fail; if (compat20) { if (!do_pam_account()) { sshpam_err = PAM_ACCT_EXPIRED; goto auth_fail; } if (sshpam_authctxt->force_pwchange) { sshpam_err = pam_chauthtok(sshpam_handle, PAM_CHANGE_EXPIRED_AUTHTOK); if (sshpam_err != PAM_SUCCESS) goto auth_fail; sshpam_password_change_required(0); } } buffer_put_cstring(&buffer, "OK"); #ifndef UNSUPPORTED_POSIX_THREADS_HACK /* Export variables set by do_pam_account */ buffer_put_int(&buffer, sshpam_account_status); buffer_put_int(&buffer, sshpam_authctxt->force_pwchange); /* Export any environment strings set in child */ for(i = 0; environ[i] != NULL; i++) ; /* Count */ buffer_put_int(&buffer, i); for(i = 0; environ[i] != NULL; i++) buffer_put_cstring(&buffer, environ[i]); /* Export any environment strings set by PAM in child */ env_from_pam = pam_getenvlist(sshpam_handle); for(i = 0; env_from_pam != NULL && env_from_pam[i] != NULL; i++) ; /* Count */ buffer_put_int(&buffer, i); for(i = 0; env_from_pam != NULL && env_from_pam[i] != NULL; i++) buffer_put_cstring(&buffer, env_from_pam[i]); #endif /* UNSUPPORTED_POSIX_THREADS_HACK */ /* XXX - can't do much about an error here */ ssh_msg_send(ctxt->pam_csock, sshpam_err, &buffer); buffer_free(&buffer); pthread_exit(NULL); auth_fail: buffer_put_cstring(&buffer, pam_strerror(sshpam_handle, sshpam_err)); /* XXX - can't do much about an error here */ if (sshpam_err == PAM_ACCT_EXPIRED) ssh_msg_send(ctxt->pam_csock, PAM_ACCT_EXPIRED, &buffer); else ssh_msg_send(ctxt->pam_csock, PAM_AUTH_ERR, &buffer); buffer_free(&buffer); pthread_exit(NULL); return (NULL); /* Avoid warning for non-pthread case */ } void sshpam_thread_cleanup(void) { struct pam_ctxt *ctxt = cleanup_ctxt; debug3("PAM: %s entering", __func__); if (ctxt != NULL && ctxt->pam_thread != 0) { pthread_cancel(ctxt->pam_thread); pthread_join(ctxt->pam_thread, NULL); close(ctxt->pam_psock); close(ctxt->pam_csock); memset(ctxt, 0, sizeof(*ctxt)); cleanup_ctxt = NULL; } } static int sshpam_null_conv(int n, sshpam_const struct pam_message **msg, struct pam_response **resp, void *data) { debug3("PAM: %s entering, %d messages", __func__, n); return (PAM_CONV_ERR); } static struct pam_conv null_conv = { sshpam_null_conv, NULL }; static int sshpam_store_conv(int n, sshpam_const struct pam_message **msg, struct pam_response **resp, void *data) { struct pam_response *reply; int i; size_t len; debug3("PAM: %s called with %d messages", __func__, n); *resp = NULL; if (n <= 0 || n > PAM_MAX_NUM_MSG) return (PAM_CONV_ERR); if ((reply = calloc(n, sizeof(*reply))) == NULL) return (PAM_CONV_ERR); for (i = 0; i < n; ++i) { switch (PAM_MSG_MEMBER(msg, i, msg_style)) { case PAM_ERROR_MSG: case PAM_TEXT_INFO: len = strlen(PAM_MSG_MEMBER(msg, i, msg)); buffer_append(&loginmsg, PAM_MSG_MEMBER(msg, i, msg), len); buffer_append(&loginmsg, "\n", 1 ); reply[i].resp_retcode = PAM_SUCCESS; break; default: goto fail; } } *resp = reply; return (PAM_SUCCESS); fail: for(i = 0; i < n; i++) { free(reply[i].resp); } free(reply); return (PAM_CONV_ERR); } static struct pam_conv store_conv = { sshpam_store_conv, NULL }; void sshpam_cleanup(void) { if (sshpam_handle == NULL || (use_privsep && !mm_is_monitor())) return; debug("PAM: cleanup"); pam_set_item(sshpam_handle, PAM_CONV, (const void *)&null_conv); if (sshpam_session_open) { debug("PAM: closing session"); pam_close_session(sshpam_handle, PAM_SILENT); sshpam_session_open = 0; } if (sshpam_cred_established) { debug("PAM: deleting credentials"); pam_setcred(sshpam_handle, PAM_DELETE_CRED); sshpam_cred_established = 0; } sshpam_authenticated = 0; pam_end(sshpam_handle, sshpam_err); sshpam_handle = NULL; } static int sshpam_init(Authctxt *authctxt) { extern char *__progname; const char *pam_rhost, *pam_user, *user = authctxt->user; const char **ptr_pam_user = &pam_user; if (sshpam_handle != NULL) { /* We already have a PAM context; check if the user matches */ sshpam_err = pam_get_item(sshpam_handle, PAM_USER, (sshpam_const void **)ptr_pam_user); if (sshpam_err == PAM_SUCCESS && strcmp(user, pam_user) == 0) return (0); pam_end(sshpam_handle, sshpam_err); sshpam_handle = NULL; } debug("PAM: initializing for \"%s\"", user); sshpam_err = pam_start(SSHD_PAM_SERVICE, user, &store_conv, &sshpam_handle); sshpam_authctxt = authctxt; if (sshpam_err != PAM_SUCCESS) { pam_end(sshpam_handle, sshpam_err); sshpam_handle = NULL; return (-1); } pam_rhost = get_remote_name_or_ip(utmp_len, options.use_dns); debug("PAM: setting PAM_RHOST to \"%s\"", pam_rhost); sshpam_err = pam_set_item(sshpam_handle, PAM_RHOST, pam_rhost); if (sshpam_err != PAM_SUCCESS) { pam_end(sshpam_handle, sshpam_err); sshpam_handle = NULL; return (-1); } #ifdef PAM_TTY_KLUDGE /* * Some silly PAM modules (e.g. pam_time) require a TTY to operate. * sshd doesn't set the tty until too late in the auth process and * may not even set one (for tty-less connections) */ debug("PAM: setting PAM_TTY to \"ssh\""); sshpam_err = pam_set_item(sshpam_handle, PAM_TTY, "ssh"); if (sshpam_err != PAM_SUCCESS) { pam_end(sshpam_handle, sshpam_err); sshpam_handle = NULL; return (-1); } #endif return (0); } static void * sshpam_init_ctx(Authctxt *authctxt) { struct pam_ctxt *ctxt; int socks[2]; debug3("PAM: %s entering", __func__); /* * Refuse to start if we don't have PAM enabled or do_pam_account * has previously failed. */ if (!options.use_pam || sshpam_account_status == 0) return NULL; /* Initialize PAM */ if (sshpam_init(authctxt) == -1) { error("PAM: initialization failed"); return (NULL); } ctxt = xcalloc(1, sizeof *ctxt); /* Start the authentication thread */ if (socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, socks) == -1) { error("PAM: failed create sockets: %s", strerror(errno)); free(ctxt); return (NULL); } ctxt->pam_psock = socks[0]; ctxt->pam_csock = socks[1]; if (pthread_create(&ctxt->pam_thread, NULL, sshpam_thread, ctxt) == -1) { error("PAM: failed to start authentication thread: %s", strerror(errno)); close(socks[0]); close(socks[1]); free(ctxt); return (NULL); } cleanup_ctxt = ctxt; return (ctxt); } static int sshpam_query(void *ctx, char **name, char **info, u_int *num, char ***prompts, u_int **echo_on) { Buffer buffer; struct pam_ctxt *ctxt = ctx; size_t plen; u_char type; char *msg; size_t len, mlen; debug3("PAM: %s entering", __func__); buffer_init(&buffer); *name = xstrdup(""); *info = xstrdup(""); *prompts = xmalloc(sizeof(char *)); **prompts = NULL; plen = 0; *echo_on = xmalloc(sizeof(u_int)); while (ssh_msg_recv(ctxt->pam_psock, &buffer) == 0) { type = buffer_get_char(&buffer); msg = buffer_get_string(&buffer, NULL); mlen = strlen(msg); switch (type) { case PAM_PROMPT_ECHO_ON: case PAM_PROMPT_ECHO_OFF: *num = 1; len = plen + mlen + 1; **prompts = xreallocarray(**prompts, 1, len); strlcpy(**prompts + plen, msg, len - plen); plen += mlen; **echo_on = (type == PAM_PROMPT_ECHO_ON); free(msg); return (0); case PAM_ERROR_MSG: case PAM_TEXT_INFO: /* accumulate messages */ len = plen + mlen + 2; **prompts = xreallocarray(**prompts, 1, len); strlcpy(**prompts + plen, msg, len - plen); plen += mlen; strlcat(**prompts + plen, "\n", len - plen); plen++; free(msg); break; case PAM_ACCT_EXPIRED: sshpam_account_status = 0; /* FALLTHROUGH */ case PAM_AUTH_ERR: debug3("PAM: %s", pam_strerror(sshpam_handle, type)); if (**prompts != NULL && strlen(**prompts) != 0) { *info = **prompts; **prompts = NULL; *num = 0; **echo_on = 0; ctxt->pam_done = -1; free(msg); return 0; } /* FALLTHROUGH */ case PAM_SUCCESS: if (**prompts != NULL) { /* drain any accumulated messages */ debug("PAM: %s", **prompts); buffer_append(&loginmsg, **prompts, strlen(**prompts)); free(**prompts); **prompts = NULL; } if (type == PAM_SUCCESS) { if (!sshpam_authctxt->valid || (sshpam_authctxt->pw->pw_uid == 0 && options.permit_root_login != PERMIT_YES)) fatal("Internal error: PAM auth " "succeeded when it should have " "failed"); import_environments(&buffer); *num = 0; **echo_on = 0; ctxt->pam_done = 1; free(msg); return (0); } + BLACKLIST_NOTIFY(BLACKLIST_AUTH_FAIL); error("PAM: %s for %s%.100s from %.100s", msg, sshpam_authctxt->valid ? "" : "illegal user ", sshpam_authctxt->user, get_remote_name_or_ip(utmp_len, options.use_dns)); /* FALLTHROUGH */ default: *num = 0; **echo_on = 0; free(msg); ctxt->pam_done = -1; return (-1); } } return (-1); } /* XXX - see also comment in auth-chall.c:verify_response */ static int sshpam_respond(void *ctx, u_int num, char **resp) { Buffer buffer; struct pam_ctxt *ctxt = ctx; debug2("PAM: %s entering, %u responses", __func__, num); switch (ctxt->pam_done) { case 1: sshpam_authenticated = 1; return (0); case 0: break; default: return (-1); } if (num != 1) { error("PAM: expected one response, got %u", num); return (-1); } buffer_init(&buffer); if (sshpam_authctxt->valid && (sshpam_authctxt->pw->pw_uid != 0 || options.permit_root_login == PERMIT_YES)) buffer_put_cstring(&buffer, *resp); else buffer_put_cstring(&buffer, badpw); if (ssh_msg_send(ctxt->pam_psock, PAM_AUTHTOK, &buffer) == -1) { buffer_free(&buffer); return (-1); } buffer_free(&buffer); return (1); } static void sshpam_free_ctx(void *ctxtp) { struct pam_ctxt *ctxt = ctxtp; debug3("PAM: %s entering", __func__); sshpam_thread_cleanup(); free(ctxt); /* * We don't call sshpam_cleanup() here because we may need the PAM * handle at a later stage, e.g. when setting up a session. It's * still on the cleanup list, so pam_end() *will* be called before * the server process terminates. */ } KbdintDevice sshpam_device = { "pam", sshpam_init_ctx, sshpam_query, sshpam_respond, sshpam_free_ctx }; KbdintDevice mm_sshpam_device = { "pam", mm_sshpam_init_ctx, mm_sshpam_query, mm_sshpam_respond, mm_sshpam_free_ctx }; /* * This replaces auth-pam.c */ void start_pam(Authctxt *authctxt) { if (!options.use_pam) fatal("PAM: initialisation requested when UsePAM=no"); if (sshpam_init(authctxt) == -1) fatal("PAM: initialisation failed"); } void finish_pam(void) { sshpam_cleanup(); } u_int do_pam_account(void) { debug("%s: called", __func__); if (sshpam_account_status != -1) return (sshpam_account_status); sshpam_err = pam_acct_mgmt(sshpam_handle, 0); debug3("PAM: %s pam_acct_mgmt = %d (%s)", __func__, sshpam_err, pam_strerror(sshpam_handle, sshpam_err)); if (sshpam_err != PAM_SUCCESS && sshpam_err != PAM_NEW_AUTHTOK_REQD) { sshpam_account_status = 0; return (sshpam_account_status); } if (sshpam_err == PAM_NEW_AUTHTOK_REQD) sshpam_password_change_required(1); sshpam_account_status = 1; return (sshpam_account_status); } void do_pam_set_tty(const char *tty) { if (tty != NULL) { debug("PAM: setting PAM_TTY to \"%s\"", tty); sshpam_err = pam_set_item(sshpam_handle, PAM_TTY, tty); if (sshpam_err != PAM_SUCCESS) fatal("PAM: failed to set PAM_TTY: %s", pam_strerror(sshpam_handle, sshpam_err)); } } void do_pam_setcred(int init) { sshpam_err = pam_set_item(sshpam_handle, PAM_CONV, (const void *)&store_conv); if (sshpam_err != PAM_SUCCESS) fatal("PAM: failed to set PAM_CONV: %s", pam_strerror(sshpam_handle, sshpam_err)); if (init) { debug("PAM: establishing credentials"); sshpam_err = pam_setcred(sshpam_handle, PAM_ESTABLISH_CRED); } else { debug("PAM: reinitializing credentials"); sshpam_err = pam_setcred(sshpam_handle, PAM_REINITIALIZE_CRED); } if (sshpam_err == PAM_SUCCESS) { sshpam_cred_established = 1; return; } if (sshpam_authenticated) fatal("PAM: pam_setcred(): %s", pam_strerror(sshpam_handle, sshpam_err)); else debug("PAM: pam_setcred(): %s", pam_strerror(sshpam_handle, sshpam_err)); } static int sshpam_tty_conv(int n, sshpam_const struct pam_message **msg, struct pam_response **resp, void *data) { char input[PAM_MAX_MSG_SIZE]; struct pam_response *reply; int i; debug3("PAM: %s called with %d messages", __func__, n); *resp = NULL; if (n <= 0 || n > PAM_MAX_NUM_MSG || !isatty(STDIN_FILENO)) return (PAM_CONV_ERR); if ((reply = calloc(n, sizeof(*reply))) == NULL) return (PAM_CONV_ERR); for (i = 0; i < n; ++i) { switch (PAM_MSG_MEMBER(msg, i, msg_style)) { case PAM_PROMPT_ECHO_OFF: reply[i].resp = read_passphrase(PAM_MSG_MEMBER(msg, i, msg), RP_ALLOW_STDIN); reply[i].resp_retcode = PAM_SUCCESS; break; case PAM_PROMPT_ECHO_ON: fprintf(stderr, "%s\n", PAM_MSG_MEMBER(msg, i, msg)); if (fgets(input, sizeof input, stdin) == NULL) input[0] = '\0'; if ((reply[i].resp = strdup(input)) == NULL) goto fail; reply[i].resp_retcode = PAM_SUCCESS; break; case PAM_ERROR_MSG: case PAM_TEXT_INFO: fprintf(stderr, "%s\n", PAM_MSG_MEMBER(msg, i, msg)); reply[i].resp_retcode = PAM_SUCCESS; break; default: goto fail; } } *resp = reply; return (PAM_SUCCESS); fail: for(i = 0; i < n; i++) { free(reply[i].resp); } free(reply); return (PAM_CONV_ERR); } static struct pam_conv tty_conv = { sshpam_tty_conv, NULL }; /* * XXX this should be done in the authentication phase, but ssh1 doesn't * support that */ void do_pam_chauthtok(void) { if (use_privsep) fatal("Password expired (unable to change with privsep)"); sshpam_err = pam_set_item(sshpam_handle, PAM_CONV, (const void *)&tty_conv); if (sshpam_err != PAM_SUCCESS) fatal("PAM: failed to set PAM_CONV: %s", pam_strerror(sshpam_handle, sshpam_err)); debug("PAM: changing password"); sshpam_err = pam_chauthtok(sshpam_handle, PAM_CHANGE_EXPIRED_AUTHTOK); if (sshpam_err != PAM_SUCCESS) fatal("PAM: pam_chauthtok(): %s", pam_strerror(sshpam_handle, sshpam_err)); } void do_pam_session(void) { debug3("PAM: opening session"); sshpam_err = pam_set_item(sshpam_handle, PAM_CONV, (const void *)&store_conv); if (sshpam_err != PAM_SUCCESS) fatal("PAM: failed to set PAM_CONV: %s", pam_strerror(sshpam_handle, sshpam_err)); sshpam_err = pam_open_session(sshpam_handle, 0); if (sshpam_err == PAM_SUCCESS) sshpam_session_open = 1; else { sshpam_session_open = 0; disable_forwarding(); error("PAM: pam_open_session(): %s", pam_strerror(sshpam_handle, sshpam_err)); } } int is_pam_session_open(void) { return sshpam_session_open; } /* * Set a PAM environment string. We need to do this so that the session * modules can handle things like Kerberos/GSI credentials that appear * during the ssh authentication process. */ int do_pam_putenv(char *name, char *value) { int ret = 1; #ifdef HAVE_PAM_PUTENV char *compound; size_t len; len = strlen(name) + strlen(value) + 2; compound = xmalloc(len); snprintf(compound, len, "%s=%s", name, value); ret = pam_putenv(sshpam_handle, compound); free(compound); #endif return (ret); } char ** fetch_pam_child_environment(void) { return sshpam_env; } char ** fetch_pam_environment(void) { return (pam_getenvlist(sshpam_handle)); } void free_pam_environment(char **env) { char **envp; if (env == NULL) return; for (envp = env; *envp; envp++) free(*envp); free(env); } /* * "Blind" conversation function for password authentication. Assumes that * echo-off prompts are for the password and stores messages for later * display. */ static int sshpam_passwd_conv(int n, sshpam_const struct pam_message **msg, struct pam_response **resp, void *data) { struct pam_response *reply; int i; size_t len; debug3("PAM: %s called with %d messages", __func__, n); *resp = NULL; if (n <= 0 || n > PAM_MAX_NUM_MSG) return (PAM_CONV_ERR); if ((reply = calloc(n, sizeof(*reply))) == NULL) return (PAM_CONV_ERR); for (i = 0; i < n; ++i) { switch (PAM_MSG_MEMBER(msg, i, msg_style)) { case PAM_PROMPT_ECHO_OFF: if (sshpam_password == NULL) goto fail; if ((reply[i].resp = strdup(sshpam_password)) == NULL) goto fail; reply[i].resp_retcode = PAM_SUCCESS; break; case PAM_ERROR_MSG: case PAM_TEXT_INFO: len = strlen(PAM_MSG_MEMBER(msg, i, msg)); if (len > 0) { buffer_append(&loginmsg, PAM_MSG_MEMBER(msg, i, msg), len); buffer_append(&loginmsg, "\n", 1); } if ((reply[i].resp = strdup("")) == NULL) goto fail; reply[i].resp_retcode = PAM_SUCCESS; break; default: goto fail; } } *resp = reply; return (PAM_SUCCESS); fail: for(i = 0; i < n; i++) { free(reply[i].resp); } free(reply); return (PAM_CONV_ERR); } static struct pam_conv passwd_conv = { sshpam_passwd_conv, NULL }; /* * Attempt password authentication via PAM */ int sshpam_auth_passwd(Authctxt *authctxt, const char *password) { int flags = (options.permit_empty_passwd == 0 ? PAM_DISALLOW_NULL_AUTHTOK : 0); if (!options.use_pam || sshpam_handle == NULL) fatal("PAM: %s called when PAM disabled or failed to " "initialise.", __func__); sshpam_password = password; sshpam_authctxt = authctxt; /* * If the user logging in is invalid, or is root but is not permitted * by PermitRootLogin, use an invalid password to prevent leaking * information via timing (eg if the PAM config has a delay on fail). */ if (!authctxt->valid || (authctxt->pw->pw_uid == 0 && options.permit_root_login != PERMIT_YES)) sshpam_password = badpw; sshpam_err = pam_set_item(sshpam_handle, PAM_CONV, (const void *)&passwd_conv); if (sshpam_err != PAM_SUCCESS) fatal("PAM: %s: failed to set PAM_CONV: %s", __func__, pam_strerror(sshpam_handle, sshpam_err)); sshpam_err = pam_authenticate(sshpam_handle, flags); sshpam_password = NULL; if (sshpam_err == PAM_SUCCESS && authctxt->valid) { debug("PAM: password authentication accepted for %.100s", authctxt->user); return 1; } else { debug("PAM: password authentication failed for %.100s: %s", authctxt->valid ? authctxt->user : "an illegal user", pam_strerror(sshpam_handle, sshpam_err)); return 0; } } #endif /* USE_PAM */ Index: projects/netbsd-tests-update-12/crypto/openssh/auth.c =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/auth.c (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/auth.c (revision 305172) @@ -1,776 +1,781 @@ /* $OpenBSD: auth.c,v 1.113 2015/08/21 03:42:19 djm Exp $ */ /* * Copyright (c) 2000 Markus Friedl. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "includes.h" __RCSID("$FreeBSD$"); #include #include #include #include #include #ifdef HAVE_PATHS_H # include #endif #include #ifdef HAVE_LOGIN_H #include #endif #ifdef USE_SHADOW #include #endif #ifdef HAVE_LIBGEN_H #include #endif #include #include #include #include #include #include "xmalloc.h" #include "match.h" #include "groupaccess.h" #include "log.h" #include "buffer.h" #include "misc.h" #include "servconf.h" #include "key.h" #include "hostfile.h" #include "auth.h" #include "auth-options.h" #include "canohost.h" #include "uidswap.h" #include "packet.h" #include "loginrec.h" #ifdef GSSAPI #include "ssh-gss.h" #endif #include "authfile.h" #include "monitor_wrap.h" #include "authfile.h" #include "ssherr.h" #include "compat.h" +#include "blacklist_client.h" /* import */ extern ServerOptions options; extern int use_privsep; extern Buffer loginmsg; extern struct passwd *privsep_pw; /* Debugging messages */ Buffer auth_debug; int auth_debug_init; /* * Check if the user is allowed to log in via ssh. If user is listed * in DenyUsers or one of user's groups is listed in DenyGroups, false * will be returned. If AllowUsers isn't empty and user isn't listed * there, or if AllowGroups isn't empty and one of user's groups isn't * listed there, false will be returned. * If the user's shell is not executable, false will be returned. * Otherwise true is returned. */ int allowed_user(struct passwd * pw) { struct stat st; const char *hostname = NULL, *ipaddr = NULL, *passwd = NULL; u_int i; #ifdef USE_SHADOW struct spwd *spw = NULL; #endif /* Shouldn't be called if pw is NULL, but better safe than sorry... */ if (!pw || !pw->pw_name) return 0; #ifdef USE_SHADOW if (!options.use_pam) spw = getspnam(pw->pw_name); #ifdef HAS_SHADOW_EXPIRE if (!options.use_pam && spw != NULL && auth_shadow_acctexpired(spw)) return 0; #endif /* HAS_SHADOW_EXPIRE */ #endif /* USE_SHADOW */ /* grab passwd field for locked account check */ passwd = pw->pw_passwd; #ifdef USE_SHADOW if (spw != NULL) #ifdef USE_LIBIAF passwd = get_iaf_password(pw); #else passwd = spw->sp_pwdp; #endif /* USE_LIBIAF */ #endif /* check for locked account */ if (!options.use_pam && passwd && *passwd) { int locked = 0; #ifdef LOCKED_PASSWD_STRING if (strcmp(passwd, LOCKED_PASSWD_STRING) == 0) locked = 1; #endif #ifdef LOCKED_PASSWD_PREFIX if (strncmp(passwd, LOCKED_PASSWD_PREFIX, strlen(LOCKED_PASSWD_PREFIX)) == 0) locked = 1; #endif #ifdef LOCKED_PASSWD_SUBSTR if (strstr(passwd, LOCKED_PASSWD_SUBSTR)) locked = 1; #endif #ifdef USE_LIBIAF free((void *) passwd); #endif /* USE_LIBIAF */ if (locked) { logit("User %.100s not allowed because account is locked", pw->pw_name); return 0; } } /* * Deny if shell does not exist or is not executable unless we * are chrooting. */ if (options.chroot_directory == NULL || strcasecmp(options.chroot_directory, "none") == 0) { char *shell = xstrdup((pw->pw_shell[0] == '\0') ? _PATH_BSHELL : pw->pw_shell); /* empty = /bin/sh */ if (stat(shell, &st) != 0) { logit("User %.100s not allowed because shell %.100s " "does not exist", pw->pw_name, shell); free(shell); return 0; } if (S_ISREG(st.st_mode) == 0 || (st.st_mode & (S_IXOTH|S_IXUSR|S_IXGRP)) == 0) { logit("User %.100s not allowed because shell %.100s " "is not executable", pw->pw_name, shell); free(shell); return 0; } free(shell); } if (options.num_deny_users > 0 || options.num_allow_users > 0 || options.num_deny_groups > 0 || options.num_allow_groups > 0) { hostname = get_canonical_hostname(options.use_dns); ipaddr = get_remote_ipaddr(); } /* Return false if user is listed in DenyUsers */ if (options.num_deny_users > 0) { for (i = 0; i < options.num_deny_users; i++) if (match_user(pw->pw_name, hostname, ipaddr, options.deny_users[i])) { logit("User %.100s from %.100s not allowed " "because listed in DenyUsers", pw->pw_name, hostname); return 0; } } /* Return false if AllowUsers isn't empty and user isn't listed there */ if (options.num_allow_users > 0) { for (i = 0; i < options.num_allow_users; i++) if (match_user(pw->pw_name, hostname, ipaddr, options.allow_users[i])) break; /* i < options.num_allow_users iff we break for loop */ if (i >= options.num_allow_users) { logit("User %.100s from %.100s not allowed because " "not listed in AllowUsers", pw->pw_name, hostname); return 0; } } if (options.num_deny_groups > 0 || options.num_allow_groups > 0) { /* Get the user's group access list (primary and supplementary) */ if (ga_init(pw->pw_name, pw->pw_gid) == 0) { logit("User %.100s from %.100s not allowed because " "not in any group", pw->pw_name, hostname); return 0; } /* Return false if one of user's groups is listed in DenyGroups */ if (options.num_deny_groups > 0) if (ga_match(options.deny_groups, options.num_deny_groups)) { ga_free(); logit("User %.100s from %.100s not allowed " "because a group is listed in DenyGroups", pw->pw_name, hostname); return 0; } /* * Return false if AllowGroups isn't empty and one of user's groups * isn't listed there */ if (options.num_allow_groups > 0) if (!ga_match(options.allow_groups, options.num_allow_groups)) { ga_free(); logit("User %.100s from %.100s not allowed " "because none of user's groups are listed " "in AllowGroups", pw->pw_name, hostname); return 0; } ga_free(); } #ifdef CUSTOM_SYS_AUTH_ALLOWED_USER if (!sys_auth_allowed_user(pw, &loginmsg)) return 0; #endif /* We found no reason not to let this user try to log on... */ return 1; } void auth_info(Authctxt *authctxt, const char *fmt, ...) { va_list ap; int i; free(authctxt->info); authctxt->info = NULL; va_start(ap, fmt); i = vasprintf(&authctxt->info, fmt, ap); va_end(ap); if (i < 0 || authctxt->info == NULL) fatal("vasprintf failed"); } void auth_log(Authctxt *authctxt, int authenticated, int partial, const char *method, const char *submethod) { void (*authlog) (const char *fmt,...) = verbose; char *authmsg; if (use_privsep && !mm_is_monitor() && !authctxt->postponed) return; /* Raise logging level */ if (authenticated == 1 || !authctxt->valid || authctxt->failures >= options.max_authtries / 2 || strcmp(method, "password") == 0) authlog = logit; if (authctxt->postponed) authmsg = "Postponed"; else if (partial) authmsg = "Partial"; - else + else { authmsg = authenticated ? "Accepted" : "Failed"; + BLACKLIST_NOTIFY(authenticated ? + BLACKLIST_AUTH_OK : BLACKLIST_AUTH_FAIL); + } authlog("%s %s%s%s for %s%.100s from %.200s port %d %s%s%s", authmsg, method, submethod != NULL ? "/" : "", submethod == NULL ? "" : submethod, authctxt->valid ? "" : "invalid user ", authctxt->user, get_remote_ipaddr(), get_remote_port(), compat20 ? "ssh2" : "ssh1", authctxt->info != NULL ? ": " : "", authctxt->info != NULL ? authctxt->info : ""); free(authctxt->info); authctxt->info = NULL; #ifdef CUSTOM_FAILED_LOGIN if (authenticated == 0 && !authctxt->postponed && (strcmp(method, "password") == 0 || strncmp(method, "keyboard-interactive", 20) == 0 || strcmp(method, "challenge-response") == 0)) record_failed_login(authctxt->user, get_canonical_hostname(options.use_dns), "ssh"); # ifdef WITH_AIXAUTHENTICATE if (authenticated) sys_auth_record_login(authctxt->user, get_canonical_hostname(options.use_dns), "ssh", &loginmsg); # endif #endif #ifdef SSH_AUDIT_EVENTS if (authenticated == 0 && !authctxt->postponed) audit_event(audit_classify_auth(method)); #endif } void auth_maxtries_exceeded(Authctxt *authctxt) { error("maximum authentication attempts exceeded for " "%s%.100s from %.200s port %d %s", authctxt->valid ? "" : "invalid user ", authctxt->user, get_remote_ipaddr(), get_remote_port(), compat20 ? "ssh2" : "ssh1"); packet_disconnect("Too many authentication failures"); /* NOTREACHED */ } /* * Check whether root logins are disallowed. */ int auth_root_allowed(const char *method) { switch (options.permit_root_login) { case PERMIT_YES: return 1; case PERMIT_NO_PASSWD: if (strcmp(method, "publickey") == 0 || strcmp(method, "hostbased") == 0 || strcmp(method, "gssapi-with-mic") == 0) return 1; break; case PERMIT_FORCED_ONLY: if (forced_command) { logit("Root login accepted for forced command."); return 1; } break; } logit("ROOT LOGIN REFUSED FROM %.200s", get_remote_ipaddr()); return 0; } /* * Given a template and a passwd structure, build a filename * by substituting % tokenised options. Currently, %% becomes '%', * %h becomes the home directory and %u the username. * * This returns a buffer allocated by xmalloc. */ char * expand_authorized_keys(const char *filename, struct passwd *pw) { char *file, ret[PATH_MAX]; int i; file = percent_expand(filename, "h", pw->pw_dir, "u", pw->pw_name, (char *)NULL); /* * Ensure that filename starts anchored. If not, be backward * compatible and prepend the '%h/' */ if (*file == '/') return (file); i = snprintf(ret, sizeof(ret), "%s/%s", pw->pw_dir, file); if (i < 0 || (size_t)i >= sizeof(ret)) fatal("expand_authorized_keys: path too long"); free(file); return (xstrdup(ret)); } char * authorized_principals_file(struct passwd *pw) { if (options.authorized_principals_file == NULL) return NULL; return expand_authorized_keys(options.authorized_principals_file, pw); } /* return ok if key exists in sysfile or userfile */ HostStatus check_key_in_hostfiles(struct passwd *pw, Key *key, const char *host, const char *sysfile, const char *userfile) { char *user_hostfile; struct stat st; HostStatus host_status; struct hostkeys *hostkeys; const struct hostkey_entry *found; hostkeys = init_hostkeys(); load_hostkeys(hostkeys, host, sysfile); if (userfile != NULL) { user_hostfile = tilde_expand_filename(userfile, pw->pw_uid); if (options.strict_modes && (stat(user_hostfile, &st) == 0) && ((st.st_uid != 0 && st.st_uid != pw->pw_uid) || (st.st_mode & 022) != 0)) { logit("Authentication refused for %.100s: " "bad owner or modes for %.200s", pw->pw_name, user_hostfile); auth_debug_add("Ignored %.200s: bad ownership or modes", user_hostfile); } else { temporarily_use_uid(pw); load_hostkeys(hostkeys, host, user_hostfile); restore_uid(); } free(user_hostfile); } host_status = check_key_in_hostkeys(hostkeys, key, &found); if (host_status == HOST_REVOKED) error("WARNING: revoked key for %s attempted authentication", found->host); else if (host_status == HOST_OK) debug("%s: key for %s found at %s:%ld", __func__, found->host, found->file, found->line); else debug("%s: key for host %s not found", __func__, host); free_hostkeys(hostkeys); return host_status; } /* * Check a given path for security. This is defined as all components * of the path to the file must be owned by either the owner of * of the file or root and no directories must be group or world writable. * * XXX Should any specific check be done for sym links ? * * Takes a file name, its stat information (preferably from fstat() to * avoid races), the uid of the expected owner, their home directory and an * error buffer plus max size as arguments. * * Returns 0 on success and -1 on failure */ int auth_secure_path(const char *name, struct stat *stp, const char *pw_dir, uid_t uid, char *err, size_t errlen) { char buf[PATH_MAX], homedir[PATH_MAX]; char *cp; int comparehome = 0; struct stat st; if (realpath(name, buf) == NULL) { snprintf(err, errlen, "realpath %s failed: %s", name, strerror(errno)); return -1; } if (pw_dir != NULL && realpath(pw_dir, homedir) != NULL) comparehome = 1; if (!S_ISREG(stp->st_mode)) { snprintf(err, errlen, "%s is not a regular file", buf); return -1; } if ((!platform_sys_dir_uid(stp->st_uid) && stp->st_uid != uid) || (stp->st_mode & 022) != 0) { snprintf(err, errlen, "bad ownership or modes for file %s", buf); return -1; } /* for each component of the canonical path, walking upwards */ for (;;) { if ((cp = dirname(buf)) == NULL) { snprintf(err, errlen, "dirname() failed"); return -1; } strlcpy(buf, cp, sizeof(buf)); if (stat(buf, &st) < 0 || (!platform_sys_dir_uid(st.st_uid) && st.st_uid != uid) || (st.st_mode & 022) != 0) { snprintf(err, errlen, "bad ownership or modes for directory %s", buf); return -1; } /* If are past the homedir then we can stop */ if (comparehome && strcmp(homedir, buf) == 0) break; /* * dirname should always complete with a "/" path, * but we can be paranoid and check for "." too */ if ((strcmp("/", buf) == 0) || (strcmp(".", buf) == 0)) break; } return 0; } /* * Version of secure_path() that accepts an open file descriptor to * avoid races. * * Returns 0 on success and -1 on failure */ static int secure_filename(FILE *f, const char *file, struct passwd *pw, char *err, size_t errlen) { struct stat st; /* check the open file to avoid races */ if (fstat(fileno(f), &st) < 0) { snprintf(err, errlen, "cannot stat file %s: %s", file, strerror(errno)); return -1; } return auth_secure_path(file, &st, pw->pw_dir, pw->pw_uid, err, errlen); } static FILE * auth_openfile(const char *file, struct passwd *pw, int strict_modes, int log_missing, char *file_type) { char line[1024]; struct stat st; int fd; FILE *f; if ((fd = open(file, O_RDONLY|O_NONBLOCK)) == -1) { if (log_missing || errno != ENOENT) debug("Could not open %s '%s': %s", file_type, file, strerror(errno)); return NULL; } if (fstat(fd, &st) < 0) { close(fd); return NULL; } if (!S_ISREG(st.st_mode)) { logit("User %s %s %s is not a regular file", pw->pw_name, file_type, file); close(fd); return NULL; } unset_nonblock(fd); if ((f = fdopen(fd, "r")) == NULL) { close(fd); return NULL; } if (strict_modes && secure_filename(f, file, pw, line, sizeof(line)) != 0) { fclose(f); logit("Authentication refused: %s", line); auth_debug_add("Ignored %s: %s", file_type, line); return NULL; } return f; } FILE * auth_openkeyfile(const char *file, struct passwd *pw, int strict_modes) { return auth_openfile(file, pw, strict_modes, 1, "authorized keys"); } FILE * auth_openprincipals(const char *file, struct passwd *pw, int strict_modes) { return auth_openfile(file, pw, strict_modes, 0, "authorized principals"); } struct passwd * getpwnamallow(const char *user) { #ifdef HAVE_LOGIN_CAP extern login_cap_t *lc; #ifdef BSD_AUTH auth_session_t *as; #endif #endif struct passwd *pw; struct connection_info *ci = get_connection_info(1, options.use_dns); ci->user = user; parse_server_match_config(&options, ci); #if defined(_AIX) && defined(HAVE_SETAUTHDB) aix_setauthdb(user); #endif pw = getpwnam(user); #if defined(_AIX) && defined(HAVE_SETAUTHDB) aix_restoreauthdb(); #endif #ifdef HAVE_CYGWIN /* * Windows usernames are case-insensitive. To avoid later problems * when trying to match the username, the user is only allowed to * login if the username is given in the same case as stored in the * user database. */ if (pw != NULL && strcmp(user, pw->pw_name) != 0) { logit("Login name %.100s does not match stored username %.100s", user, pw->pw_name); pw = NULL; } #endif if (pw == NULL) { + BLACKLIST_NOTIFY(BLACKLIST_AUTH_FAIL); logit("Invalid user %.100s from %.100s", user, get_remote_ipaddr()); #ifdef CUSTOM_FAILED_LOGIN record_failed_login(user, get_canonical_hostname(options.use_dns), "ssh"); #endif #ifdef SSH_AUDIT_EVENTS audit_event(SSH_INVALID_USER); #endif /* SSH_AUDIT_EVENTS */ return (NULL); } if (!allowed_user(pw)) return (NULL); #ifdef HAVE_LOGIN_CAP if ((lc = login_getpwclass(pw)) == NULL) { debug("unable to get login class: %s", user); return (NULL); } #ifdef BSD_AUTH if ((as = auth_open()) == NULL || auth_setpwd(as, pw) != 0 || auth_approval(as, lc, pw->pw_name, "ssh") <= 0) { debug("Approval failure for %s", user); pw = NULL; } if (as != NULL) auth_close(as); #endif #endif if (pw != NULL) return (pwcopy(pw)); return (NULL); } /* Returns 1 if key is revoked by revoked_keys_file, 0 otherwise */ int auth_key_is_revoked(Key *key) { char *fp = NULL; int r; if (options.revoked_keys_file == NULL) return 0; if ((fp = sshkey_fingerprint(key, options.fingerprint_hash, SSH_FP_DEFAULT)) == NULL) { r = SSH_ERR_ALLOC_FAIL; error("%s: fingerprint key: %s", __func__, ssh_err(r)); goto out; } r = sshkey_check_revoked(key, options.revoked_keys_file); switch (r) { case 0: break; /* not revoked */ case SSH_ERR_KEY_REVOKED: error("Authentication key %s %s revoked by file %s", sshkey_type(key), fp, options.revoked_keys_file); goto out; default: error("Error checking authentication key %s %s in " "revoked keys file %s: %s", sshkey_type(key), fp, options.revoked_keys_file, ssh_err(r)); goto out; } /* Success */ r = 0; out: free(fp); return r == 0 ? 0 : 1; } void auth_debug_add(const char *fmt,...) { char buf[1024]; va_list args; if (!auth_debug_init) return; va_start(args, fmt); vsnprintf(buf, sizeof(buf), fmt, args); va_end(args); buffer_put_cstring(&auth_debug, buf); } void auth_debug_send(void) { char *msg; if (!auth_debug_init) return; while (buffer_len(&auth_debug)) { msg = buffer_get_string(&auth_debug, NULL); packet_send_debug("%s", msg); free(msg); } } void auth_debug_reset(void) { if (auth_debug_init) buffer_clear(&auth_debug); else { buffer_init(&auth_debug); auth_debug_init = 1; } } struct passwd * fakepw(void) { static struct passwd fake; memset(&fake, 0, sizeof(fake)); fake.pw_name = "NOUSER"; fake.pw_passwd = "$2a$06$r3.juUaHZDlIbQaO2dS9FuYxL1W9M81R1Tc92PoSNmzvpEqLkLGrK"; #ifdef HAVE_STRUCT_PASSWD_PW_GECOS fake.pw_gecos = "NOUSER"; #endif fake.pw_uid = privsep_pw == NULL ? (uid_t)-1 : privsep_pw->pw_uid; fake.pw_gid = privsep_pw == NULL ? (gid_t)-1 : privsep_pw->pw_gid; #ifdef HAVE_STRUCT_PASSWD_PW_CLASS fake.pw_class = ""; #endif fake.pw_dir = "/nonexist"; fake.pw_shell = "/nonexist"; return (&fake); } Index: projects/netbsd-tests-update-12/crypto/openssh/auth1.c =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/auth1.c (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/auth1.c (revision 305172) @@ -1,444 +1,447 @@ /* $OpenBSD: auth1.c,v 1.82 2014/07/15 15:54:14 millert Exp $ */ /* * Copyright (c) 1995 Tatu Ylonen , Espoo, Finland * All rights reserved * * As far as I am concerned, the code I have written for this software * can be used freely for any purpose. Any derived versions of this * software must be clearly marked as such, and if the derived work is * incompatible with the protocol description in the RFC file, it must be * called by a name other than "ssh" or "Secure Shell". */ #include "includes.h" #ifdef WITH_SSH1 #include #include #include #include #include #include #include "openbsd-compat/sys-queue.h" #include "xmalloc.h" #include "rsa.h" #include "ssh1.h" #include "packet.h" #include "buffer.h" #include "log.h" #include "misc.h" #include "servconf.h" #include "compat.h" #include "key.h" #include "hostfile.h" #include "auth.h" #include "channels.h" #include "session.h" #include "uidswap.h" #ifdef GSSAPI #include "ssh-gss.h" #endif #include "monitor_wrap.h" #include "buffer.h" +#include "blacklist_client.h" /* import */ extern ServerOptions options; extern Buffer loginmsg; static int auth1_process_password(Authctxt *); static int auth1_process_rsa(Authctxt *); static int auth1_process_rhosts_rsa(Authctxt *); static int auth1_process_tis_challenge(Authctxt *); static int auth1_process_tis_response(Authctxt *); static char *client_user = NULL; /* Used to fill in remote user for PAM */ struct AuthMethod1 { int type; char *name; int *enabled; int (*method)(Authctxt *); }; const struct AuthMethod1 auth1_methods[] = { { SSH_CMSG_AUTH_PASSWORD, "password", &options.password_authentication, auth1_process_password }, { SSH_CMSG_AUTH_RSA, "rsa", &options.rsa_authentication, auth1_process_rsa }, { SSH_CMSG_AUTH_RHOSTS_RSA, "rhosts-rsa", &options.rhosts_rsa_authentication, auth1_process_rhosts_rsa }, { SSH_CMSG_AUTH_TIS, "challenge-response", &options.challenge_response_authentication, auth1_process_tis_challenge }, { SSH_CMSG_AUTH_TIS_RESPONSE, "challenge-response", &options.challenge_response_authentication, auth1_process_tis_response }, { -1, NULL, NULL, NULL} }; static const struct AuthMethod1 *lookup_authmethod1(int type) { int i; for (i = 0; auth1_methods[i].name != NULL; i++) if (auth1_methods[i].type == type) return (&(auth1_methods[i])); return (NULL); } static char * get_authname(int type) { const struct AuthMethod1 *a; static char buf[64]; if ((a = lookup_authmethod1(type)) != NULL) return (a->name); snprintf(buf, sizeof(buf), "bad-auth-msg-%d", type); return (buf); } /*ARGSUSED*/ static int auth1_process_password(Authctxt *authctxt) { int authenticated = 0; char *password; u_int dlen; /* * Read user password. It is in plain text, but was * transmitted over the encrypted channel so it is * not visible to an outside observer. */ password = packet_get_string(&dlen); packet_check_eom(); /* Try authentication with the password. */ authenticated = PRIVSEP(auth_password(authctxt, password)); explicit_bzero(password, dlen); free(password); return (authenticated); } /*ARGSUSED*/ static int auth1_process_rsa(Authctxt *authctxt) { int authenticated = 0; BIGNUM *n; /* RSA authentication requested. */ if ((n = BN_new()) == NULL) fatal("do_authloop: BN_new failed"); packet_get_bignum(n); packet_check_eom(); authenticated = auth_rsa(authctxt, n); BN_clear_free(n); return (authenticated); } /*ARGSUSED*/ static int auth1_process_rhosts_rsa(Authctxt *authctxt) { int keybits, authenticated = 0; u_int bits; Key *client_host_key; u_int ulen; /* * Get client user name. Note that we just have to * trust the client; root on the client machine can * claim to be any user. */ client_user = packet_get_cstring(&ulen); /* Get the client host key. */ client_host_key = key_new(KEY_RSA1); bits = packet_get_int(); packet_get_bignum(client_host_key->rsa->e); packet_get_bignum(client_host_key->rsa->n); keybits = BN_num_bits(client_host_key->rsa->n); if (keybits < 0 || bits != (u_int)keybits) { verbose("Warning: keysize mismatch for client_host_key: " "actual %d, announced %d", BN_num_bits(client_host_key->rsa->n), bits); } packet_check_eom(); authenticated = auth_rhosts_rsa(authctxt, client_user, client_host_key); key_free(client_host_key); auth_info(authctxt, "ruser %.100s", client_user); return (authenticated); } /*ARGSUSED*/ static int auth1_process_tis_challenge(Authctxt *authctxt) { char *challenge; if ((challenge = get_challenge(authctxt)) == NULL) return (0); debug("sending challenge '%s'", challenge); packet_start(SSH_SMSG_AUTH_TIS_CHALLENGE); packet_put_cstring(challenge); free(challenge); packet_send(); packet_write_wait(); return (-1); } /*ARGSUSED*/ static int auth1_process_tis_response(Authctxt *authctxt) { int authenticated = 0; char *response; u_int dlen; response = packet_get_string(&dlen); packet_check_eom(); authenticated = verify_response(authctxt, response); explicit_bzero(response, dlen); free(response); return (authenticated); } /* * read packets, try to authenticate the user and * return only if authentication is successful */ static void do_authloop(Authctxt *authctxt) { int authenticated = 0; int prev = 0, type = 0; const struct AuthMethod1 *meth; debug("Attempting authentication for %s%.100s.", authctxt->valid ? "" : "invalid user ", authctxt->user); /* If the user has no password, accept authentication immediately. */ if (options.permit_empty_passwd && options.password_authentication && #ifdef KRB5 (!options.kerberos_authentication || options.kerberos_or_local_passwd) && #endif PRIVSEP(auth_password(authctxt, ""))) { #ifdef USE_PAM if (options.use_pam && (PRIVSEP(do_pam_account()))) #endif { auth_log(authctxt, 1, 0, "without authentication", NULL); return; } } /* Indicate that authentication is needed. */ packet_start(SSH_SMSG_FAILURE); packet_send(); packet_write_wait(); for (;;) { /* default to fail */ authenticated = 0; /* Get a packet from the client. */ prev = type; type = packet_read(); /* * If we started challenge-response authentication but the * next packet is not a response to our challenge, release * the resources allocated by get_challenge() (which would * normally have been released by verify_response() had we * received such a response) */ if (prev == SSH_CMSG_AUTH_TIS && type != SSH_CMSG_AUTH_TIS_RESPONSE) abandon_challenge_response(authctxt); if (authctxt->failures >= options.max_authtries) goto skip; if ((meth = lookup_authmethod1(type)) == NULL) { logit("Unknown message during authentication: " "type %d", type); goto skip; } if (!*(meth->enabled)) { verbose("%s authentication disabled.", meth->name); goto skip; } authenticated = meth->method(authctxt); if (authenticated == -1) continue; /* "postponed" */ #ifdef BSD_AUTH if (authctxt->as) { auth_close(authctxt->as); authctxt->as = NULL; } #endif if (!authctxt->valid && authenticated) fatal("INTERNAL ERROR: authenticated invalid user %s", authctxt->user); #ifdef _UNICOS if (authenticated && cray_access_denied(authctxt->user)) { authenticated = 0; fatal("Access denied for user %s.",authctxt->user); } #endif /* _UNICOS */ #ifndef HAVE_CYGWIN /* Special handling for root */ if (authenticated && authctxt->pw->pw_uid == 0 && !auth_root_allowed(meth->name)) { authenticated = 0; # ifdef SSH_AUDIT_EVENTS PRIVSEP(audit_event(SSH_LOGIN_ROOT_DENIED)); # endif } #endif #ifdef USE_PAM if (options.use_pam && authenticated && !PRIVSEP(do_pam_account())) { char *msg; size_t len; + BLACKLIST_NOTIFY(BLACKLIST_AUTH_FAIL); error("Access denied for user %s by PAM account " "configuration", authctxt->user); len = buffer_len(&loginmsg); buffer_append(&loginmsg, "\0", 1); msg = buffer_ptr(&loginmsg); /* strip trailing newlines */ if (len > 0) while (len > 0 && msg[--len] == '\n') msg[len] = '\0'; else msg = "Access denied."; packet_disconnect("%s", msg); } #endif skip: /* Log before sending the reply */ auth_log(authctxt, authenticated, 0, get_authname(type), NULL); free(client_user); client_user = NULL; if (authenticated) return; if (++authctxt->failures >= options.max_authtries) { #ifdef SSH_AUDIT_EVENTS PRIVSEP(audit_event(SSH_LOGIN_EXCEED_MAXTRIES)); #endif auth_maxtries_exceeded(authctxt); } packet_start(SSH_SMSG_FAILURE); packet_send(); packet_write_wait(); } } /* * Performs authentication of an incoming connection. Session key has already * been exchanged and encryption is enabled. */ void do_authentication(Authctxt *authctxt) { u_int ulen; char *user, *style = NULL; /* Get the name of the user that we wish to log in as. */ packet_read_expect(SSH_CMSG_USER); /* Get the user name. */ user = packet_get_cstring(&ulen); packet_check_eom(); if ((style = strchr(user, ':')) != NULL) *style++ = '\0'; authctxt->user = user; authctxt->style = style; /* Verify that the user is a valid user. */ if ((authctxt->pw = PRIVSEP(getpwnamallow(user))) != NULL) authctxt->valid = 1; else { debug("do_authentication: invalid user %s", user); authctxt->pw = fakepw(); + BLACKLIST_NOTIFY(BLACKLIST_AUTH_FAIL); } /* Configuration may have changed as a result of Match */ if (options.num_auth_methods != 0) fatal("AuthenticationMethods is not supported with SSH " "protocol 1"); setproctitle("%s%s", authctxt->valid ? user : "unknown", use_privsep ? " [net]" : ""); #ifdef USE_PAM if (options.use_pam) PRIVSEP(start_pam(authctxt)); #endif /* * If we are not running as root, the user must have the same uid as * the server. */ #ifndef HAVE_CYGWIN if (!use_privsep && getuid() != 0 && authctxt->pw && authctxt->pw->pw_uid != getuid()) packet_disconnect("Cannot change user when server not running as root."); #endif /* * Loop until the user has been authenticated or the connection is * closed, do_authloop() returns only if authentication is successful */ do_authloop(authctxt); /* The user has been authenticated and accepted. */ packet_start(SSH_SMSG_SUCCESS); packet_send(); packet_write_wait(); } #endif /* WITH_SSH1 */ Index: projects/netbsd-tests-update-12/crypto/openssh/auth2.c =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/auth2.c (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/auth2.c (revision 305172) @@ -1,649 +1,651 @@ /* $OpenBSD: auth2.c,v 1.135 2015/01/19 20:07:45 markus Exp $ */ /* * Copyright (c) 2000 Markus Friedl. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "includes.h" __RCSID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include "atomicio.h" #include "xmalloc.h" #include "ssh2.h" #include "packet.h" #include "log.h" #include "buffer.h" #include "misc.h" #include "servconf.h" #include "compat.h" #include "key.h" #include "hostfile.h" #include "auth.h" #include "dispatch.h" #include "pathnames.h" #include "buffer.h" #include "canohost.h" +#include "blacklist_client.h" #ifdef GSSAPI #include "ssh-gss.h" #endif #include "monitor_wrap.h" /* import */ extern ServerOptions options; extern u_char *session_id2; extern u_int session_id2_len; extern Buffer loginmsg; /* methods */ extern Authmethod method_none; extern Authmethod method_pubkey; extern Authmethod method_passwd; extern Authmethod method_kbdint; extern Authmethod method_hostbased; #ifdef GSSAPI extern Authmethod method_gssapi; #endif Authmethod *authmethods[] = { &method_none, &method_pubkey, #ifdef GSSAPI &method_gssapi, #endif &method_passwd, &method_kbdint, &method_hostbased, NULL }; /* protocol */ static int input_service_request(int, u_int32_t, void *); static int input_userauth_request(int, u_int32_t, void *); /* helper */ static Authmethod *authmethod_lookup(Authctxt *, const char *); static char *authmethods_get(Authctxt *authctxt); #define MATCH_NONE 0 /* method or submethod mismatch */ #define MATCH_METHOD 1 /* method matches (no submethod specified) */ #define MATCH_BOTH 2 /* method and submethod match */ #define MATCH_PARTIAL 3 /* method matches, submethod can't be checked */ static int list_starts_with(const char *, const char *, const char *); char * auth2_read_banner(void) { struct stat st; char *banner = NULL; size_t len, n; int fd; if ((fd = open(options.banner, O_RDONLY)) == -1) return (NULL); if (fstat(fd, &st) == -1) { close(fd); return (NULL); } if (st.st_size <= 0 || st.st_size > 1*1024*1024) { close(fd); return (NULL); } len = (size_t)st.st_size; /* truncate */ banner = xmalloc(len + 1); n = atomicio(read, fd, banner, len); close(fd); if (n != len) { free(banner); return (NULL); } banner[n] = '\0'; return (banner); } void userauth_send_banner(const char *msg) { if (datafellows & SSH_BUG_BANNER) return; packet_start(SSH2_MSG_USERAUTH_BANNER); packet_put_cstring(msg); packet_put_cstring(""); /* language, unused */ packet_send(); debug("%s: sent", __func__); } static void userauth_banner(void) { char *banner = NULL; if (options.banner == NULL || (datafellows & SSH_BUG_BANNER) != 0) return; if ((banner = PRIVSEP(auth2_read_banner())) == NULL) goto done; userauth_send_banner(banner); done: free(banner); } /* * loop until authctxt->success == TRUE */ void do_authentication2(Authctxt *authctxt) { dispatch_init(&dispatch_protocol_error); dispatch_set(SSH2_MSG_SERVICE_REQUEST, &input_service_request); dispatch_run(DISPATCH_BLOCK, &authctxt->success, authctxt); } /*ARGSUSED*/ static int input_service_request(int type, u_int32_t seq, void *ctxt) { Authctxt *authctxt = ctxt; u_int len; int acceptit = 0; char *service = packet_get_cstring(&len); packet_check_eom(); if (authctxt == NULL) fatal("input_service_request: no authctxt"); if (strcmp(service, "ssh-userauth") == 0) { if (!authctxt->success) { acceptit = 1; /* now we can handle user-auth requests */ dispatch_set(SSH2_MSG_USERAUTH_REQUEST, &input_userauth_request); } } /* XXX all other service requests are denied */ if (acceptit) { packet_start(SSH2_MSG_SERVICE_ACCEPT); packet_put_cstring(service); packet_send(); packet_write_wait(); } else { debug("bad service request %s", service); packet_disconnect("bad service request %s", service); } free(service); return 0; } /*ARGSUSED*/ static int input_userauth_request(int type, u_int32_t seq, void *ctxt) { Authctxt *authctxt = ctxt; Authmethod *m = NULL; char *user, *service, *method, *style = NULL; int authenticated = 0; #ifdef HAVE_LOGIN_CAP login_cap_t *lc; const char *from_host, *from_ip; from_host = get_canonical_hostname(options.use_dns); from_ip = get_remote_ipaddr(); #endif if (authctxt == NULL) fatal("input_userauth_request: no authctxt"); user = packet_get_cstring(NULL); service = packet_get_cstring(NULL); method = packet_get_cstring(NULL); debug("userauth-request for user %s service %s method %s", user, service, method); debug("attempt %d failures %d", authctxt->attempt, authctxt->failures); if ((style = strchr(user, ':')) != NULL) *style++ = 0; if (authctxt->attempt++ == 0) { /* setup auth context */ authctxt->pw = PRIVSEP(getpwnamallow(user)); authctxt->user = xstrdup(user); if (authctxt->pw && strcmp(service, "ssh-connection")==0) { authctxt->valid = 1; debug2("input_userauth_request: setting up authctxt for %s", user); } else { logit("input_userauth_request: invalid user %s", user); authctxt->pw = fakepw(); + BLACKLIST_NOTIFY(BLACKLIST_AUTH_FAIL); #ifdef SSH_AUDIT_EVENTS PRIVSEP(audit_event(SSH_INVALID_USER)); #endif } #ifdef USE_PAM if (options.use_pam) PRIVSEP(start_pam(authctxt)); #endif setproctitle("%s%s", authctxt->valid ? user : "unknown", use_privsep ? " [net]" : ""); authctxt->service = xstrdup(service); authctxt->style = style ? xstrdup(style) : NULL; if (use_privsep) mm_inform_authserv(service, style); userauth_banner(); if (auth2_setup_methods_lists(authctxt) != 0) packet_disconnect("no authentication methods enabled"); } else if (strcmp(user, authctxt->user) != 0 || strcmp(service, authctxt->service) != 0) { packet_disconnect("Change of username or service not allowed: " "(%s,%s) -> (%s,%s)", authctxt->user, authctxt->service, user, service); } #ifdef HAVE_LOGIN_CAP if (authctxt->pw != NULL) { lc = login_getpwclass(authctxt->pw); if (lc == NULL) lc = login_getclassbyname(NULL, authctxt->pw); if (!auth_hostok(lc, from_host, from_ip)) { logit("Denied connection for %.200s from %.200s [%.200s].", authctxt->pw->pw_name, from_host, from_ip); packet_disconnect("Sorry, you are not allowed to connect."); } if (!auth_timeok(lc, time(NULL))) { logit("LOGIN %.200s REFUSED (TIME) FROM %.200s", authctxt->pw->pw_name, from_host); packet_disconnect("Logins not available right now."); } login_close(lc); lc = NULL; } #endif /* HAVE_LOGIN_CAP */ /* reset state */ auth2_challenge_stop(authctxt); #ifdef GSSAPI /* XXX move to auth2_gssapi_stop() */ dispatch_set(SSH2_MSG_USERAUTH_GSSAPI_TOKEN, NULL); dispatch_set(SSH2_MSG_USERAUTH_GSSAPI_EXCHANGE_COMPLETE, NULL); #endif authctxt->postponed = 0; authctxt->server_caused_failure = 0; /* try to authenticate user */ m = authmethod_lookup(authctxt, method); if (m != NULL && authctxt->failures < options.max_authtries) { debug2("input_userauth_request: try method %s", method); authenticated = m->userauth(authctxt); } userauth_finish(authctxt, authenticated, method, NULL); free(service); free(user); free(method); return 0; } void userauth_finish(Authctxt *authctxt, int authenticated, const char *method, const char *submethod) { char *methods; int partial = 0; if (!authctxt->valid && authenticated) fatal("INTERNAL ERROR: authenticated invalid user %s", authctxt->user); if (authenticated && authctxt->postponed) fatal("INTERNAL ERROR: authenticated and postponed"); /* Special handling for root */ if (authenticated && authctxt->pw->pw_uid == 0 && !auth_root_allowed(method)) { authenticated = 0; #ifdef SSH_AUDIT_EVENTS PRIVSEP(audit_event(SSH_LOGIN_ROOT_DENIED)); #endif } if (authenticated && options.num_auth_methods != 0) { if (!auth2_update_methods_lists(authctxt, method, submethod)) { authenticated = 0; partial = 1; } } /* Log before sending the reply */ auth_log(authctxt, authenticated, partial, method, submethod); if (authctxt->postponed) return; #ifdef USE_PAM if (options.use_pam && authenticated) { if (!PRIVSEP(do_pam_account())) { /* if PAM returned a message, send it to the user */ if (buffer_len(&loginmsg) > 0) { buffer_append(&loginmsg, "\0", 1); userauth_send_banner(buffer_ptr(&loginmsg)); packet_write_wait(); } fatal("Access denied for user %s by PAM account " "configuration", authctxt->user); } } #endif #ifdef _UNICOS if (authenticated && cray_access_denied(authctxt->user)) { authenticated = 0; fatal("Access denied for user %s.", authctxt->user); } #endif /* _UNICOS */ if (authenticated == 1) { /* turn off userauth */ dispatch_set(SSH2_MSG_USERAUTH_REQUEST, &dispatch_protocol_ignore); packet_start(SSH2_MSG_USERAUTH_SUCCESS); packet_send(); packet_write_wait(); /* now we can break out */ authctxt->success = 1; } else { /* Allow initial try of "none" auth without failure penalty */ if (!partial && !authctxt->server_caused_failure && (authctxt->attempt > 1 || strcmp(method, "none") != 0)) authctxt->failures++; if (authctxt->failures >= options.max_authtries) { #ifdef SSH_AUDIT_EVENTS PRIVSEP(audit_event(SSH_LOGIN_EXCEED_MAXTRIES)); #endif auth_maxtries_exceeded(authctxt); } methods = authmethods_get(authctxt); debug3("%s: failure partial=%d next methods=\"%s\"", __func__, partial, methods); packet_start(SSH2_MSG_USERAUTH_FAILURE); packet_put_cstring(methods); packet_put_char(partial); packet_send(); packet_write_wait(); free(methods); } } /* * Checks whether method is allowed by at least one AuthenticationMethods * methods list. Returns 1 if allowed, or no methods lists configured. * 0 otherwise. */ int auth2_method_allowed(Authctxt *authctxt, const char *method, const char *submethod) { u_int i; /* * NB. authctxt->num_auth_methods might be zero as a result of * auth2_setup_methods_lists(), so check the configuration. */ if (options.num_auth_methods == 0) return 1; for (i = 0; i < authctxt->num_auth_methods; i++) { if (list_starts_with(authctxt->auth_methods[i], method, submethod) != MATCH_NONE) return 1; } return 0; } static char * authmethods_get(Authctxt *authctxt) { Buffer b; char *list; u_int i; buffer_init(&b); for (i = 0; authmethods[i] != NULL; i++) { if (strcmp(authmethods[i]->name, "none") == 0) continue; if (authmethods[i]->enabled == NULL || *(authmethods[i]->enabled) == 0) continue; if (!auth2_method_allowed(authctxt, authmethods[i]->name, NULL)) continue; if (buffer_len(&b) > 0) buffer_append(&b, ",", 1); buffer_append(&b, authmethods[i]->name, strlen(authmethods[i]->name)); } buffer_append(&b, "\0", 1); list = xstrdup(buffer_ptr(&b)); buffer_free(&b); return list; } static Authmethod * authmethod_lookup(Authctxt *authctxt, const char *name) { int i; if (name != NULL) for (i = 0; authmethods[i] != NULL; i++) if (authmethods[i]->enabled != NULL && *(authmethods[i]->enabled) != 0 && strcmp(name, authmethods[i]->name) == 0 && auth2_method_allowed(authctxt, authmethods[i]->name, NULL)) return authmethods[i]; debug2("Unrecognized authentication method name: %s", name ? name : "NULL"); return NULL; } /* * Check a comma-separated list of methods for validity. Is need_enable is * non-zero, then also require that the methods are enabled. * Returns 0 on success or -1 if the methods list is invalid. */ int auth2_methods_valid(const char *_methods, int need_enable) { char *methods, *omethods, *method, *p; u_int i, found; int ret = -1; if (*_methods == '\0') { error("empty authentication method list"); return -1; } omethods = methods = xstrdup(_methods); while ((method = strsep(&methods, ",")) != NULL) { for (found = i = 0; !found && authmethods[i] != NULL; i++) { if ((p = strchr(method, ':')) != NULL) *p = '\0'; if (strcmp(method, authmethods[i]->name) != 0) continue; if (need_enable) { if (authmethods[i]->enabled == NULL || *(authmethods[i]->enabled) == 0) { error("Disabled method \"%s\" in " "AuthenticationMethods list \"%s\"", method, _methods); goto out; } } found = 1; break; } if (!found) { error("Unknown authentication method \"%s\" in list", method); goto out; } } ret = 0; out: free(omethods); return ret; } /* * Prune the AuthenticationMethods supplied in the configuration, removing * any methods lists that include disabled methods. Note that this might * leave authctxt->num_auth_methods == 0, even when multiple required auth * has been requested. For this reason, all tests for whether multiple is * enabled should consult options.num_auth_methods directly. */ int auth2_setup_methods_lists(Authctxt *authctxt) { u_int i; if (options.num_auth_methods == 0) return 0; debug3("%s: checking methods", __func__); authctxt->auth_methods = xcalloc(options.num_auth_methods, sizeof(*authctxt->auth_methods)); authctxt->num_auth_methods = 0; for (i = 0; i < options.num_auth_methods; i++) { if (auth2_methods_valid(options.auth_methods[i], 1) != 0) { logit("Authentication methods list \"%s\" contains " "disabled method, skipping", options.auth_methods[i]); continue; } debug("authentication methods list %d: %s", authctxt->num_auth_methods, options.auth_methods[i]); authctxt->auth_methods[authctxt->num_auth_methods++] = xstrdup(options.auth_methods[i]); } if (authctxt->num_auth_methods == 0) { error("No AuthenticationMethods left after eliminating " "disabled methods"); return -1; } return 0; } static int list_starts_with(const char *methods, const char *method, const char *submethod) { size_t l = strlen(method); int match; const char *p; if (strncmp(methods, method, l) != 0) return MATCH_NONE; p = methods + l; match = MATCH_METHOD; if (*p == ':') { if (!submethod) return MATCH_PARTIAL; l = strlen(submethod); p += 1; if (strncmp(submethod, p, l)) return MATCH_NONE; p += l; match = MATCH_BOTH; } if (*p != ',' && *p != '\0') return MATCH_NONE; return match; } /* * Remove method from the start of a comma-separated list of methods. * Returns 0 if the list of methods did not start with that method or 1 * if it did. */ static int remove_method(char **methods, const char *method, const char *submethod) { char *omethods = *methods, *p; size_t l = strlen(method); int match; match = list_starts_with(omethods, method, submethod); if (match != MATCH_METHOD && match != MATCH_BOTH) return 0; p = omethods + l; if (submethod && match == MATCH_BOTH) p += 1 + strlen(submethod); /* include colon */ if (*p == ',') p++; *methods = xstrdup(p); free(omethods); return 1; } /* * Called after successful authentication. Will remove the successful method * from the start of each list in which it occurs. If it was the last method * in any list, then authentication is deemed successful. * Returns 1 if the method completed any authentication list or 0 otherwise. */ int auth2_update_methods_lists(Authctxt *authctxt, const char *method, const char *submethod) { u_int i, found = 0; debug3("%s: updating methods list after \"%s\"", __func__, method); for (i = 0; i < authctxt->num_auth_methods; i++) { if (!remove_method(&(authctxt->auth_methods[i]), method, submethod)) continue; found = 1; if (*authctxt->auth_methods[i] == '\0') { debug2("authentication methods list %d complete", i); return 1; } debug3("authentication methods list %d remaining: \"%s\"", i, authctxt->auth_methods[i]); } /* This should not happen, but would be bad if it did */ if (!found) fatal("%s: method not in AuthenticationMethods", __func__); return 0; } Index: projects/netbsd-tests-update-12/crypto/openssh/blacklist.c =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/blacklist.c (nonexistent) +++ projects/netbsd-tests-update-12/crypto/openssh/blacklist.c (revision 305172) @@ -0,0 +1,97 @@ +/*- + * Copyright (c) 2015 The NetBSD Foundation, Inc. + * Copyright (c) 2016 The FreeBSD Foundation, Inc. + * All rights reserved. + * + * Portions of this software were developed by Kurt Lidl + * under sponsorship from the FreeBSD Foundation. + * + * This code is derived from software contributed to The NetBSD Foundation + * by Christos Zoulas. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS + * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +#include "includes.h" + +#include +#include +#include +#include +#include +#include +#include + +#include "ssh.h" +#include "packet.h" +#include "log.h" +#include "misc.h" +#include "servconf.h" +#include "blacklist_client.h" +#include + +static struct blacklist *blstate = NULL; + +/* import */ +extern ServerOptions options; + +/* internal definition from bl.h */ +struct blacklist *bl_create(bool, char *, void (*)(int, const char *, va_list)); + +/* impedence match vsyslog() to sshd's internal logging levels */ +void +im_log(int priority, const char *message, va_list args) +{ + LogLevel imlevel; + + switch (priority) { + case LOG_ERR: + imlevel = SYSLOG_LEVEL_ERROR; + break; + case LOG_DEBUG: + imlevel = SYSLOG_LEVEL_DEBUG1; + break; + case LOG_INFO: + imlevel = SYSLOG_LEVEL_INFO; + break; + default: + imlevel = SYSLOG_LEVEL_DEBUG2; + } + do_log(imlevel, message, args); +} + +void +blacklist_init(void) +{ + + if (options.use_blacklist) + blstate = bl_create(false, NULL, im_log); +} + +void +blacklist_notify(int action) +{ + + if (blstate != NULL && packet_connection_is_on_socket()) + (void)blacklist_r(blstate, action, + packet_get_connection_in(), "ssh"); +} Property changes on: projects/netbsd-tests-update-12/crypto/openssh/blacklist.c ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/crypto/openssh/blacklist_client.h =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/blacklist_client.h (nonexistent) +++ projects/netbsd-tests-update-12/crypto/openssh/blacklist_client.h (revision 305172) @@ -0,0 +1,57 @@ +/*- + * Copyright (c) 2015 The NetBSD Foundation, Inc. + * Copyright (c) 2016 The FreeBSD Foundation, Inc. + * All rights reserved. + * + * Portions of this software were developed by Kurt Lidl + * under sponsorship from the FreeBSD Foundation. + * + * This code is derived from software contributed to The NetBSD Foundation + * by Christos Zoulas. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS + * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef BLACKLIST_CLIENT_H +#define BLACKLIST_CLIENT_H + +enum { + BLACKLIST_AUTH_OK = 0, + BLACKLIST_AUTH_FAIL +}; + +#ifdef USE_BLACKLIST +void blacklist_init(void); +void blacklist_notify(int); + +#define BLACKLIST_INIT() blacklist_init() +#define BLACKLIST_NOTIFY(x) blacklist_notify(x) + +#else + +#define BLACKLIST_INIT() +#define BLACKLIST_NOTIFY(x) + +#endif + + +#endif /* BLACKLIST_CLIENT_H */ Property changes on: projects/netbsd-tests-update-12/crypto/openssh/blacklist_client.h ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/crypto/openssh/packet.c =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/packet.c (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/packet.c (revision 305172) @@ -1,2975 +1,2977 @@ /* $OpenBSD: packet.c,v 1.229 2016/02/17 22:20:14 djm Exp $ */ /* * Author: Tatu Ylonen * Copyright (c) 1995 Tatu Ylonen , Espoo, Finland * All rights reserved * This file contains code implementing the packet protocol and communication * with the other side. This same code is used both on client and server side. * * As far as I am concerned, the code I have written for this software * can be used freely for any purpose. Any derived versions of this * software must be clearly marked as such, and if the derived work is * incompatible with the protocol description in the RFC file, it must be * called by a name other than "ssh" or "Secure Shell". * * * SSH2 packet format added by Markus Friedl. * Copyright (c) 2000, 2001 Markus Friedl. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "includes.h" __RCSID("$FreeBSD$"); #include /* MIN roundup */ #include #include "openbsd-compat/sys-queue.h" #include #ifdef HAVE_SYS_TIME_H # include #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #include "buffer.h" /* typedefs XXX */ #include "key.h" /* typedefs XXX */ #include "xmalloc.h" #include "crc32.h" #include "deattack.h" #include "compat.h" #include "ssh1.h" #include "ssh2.h" #include "cipher.h" #include "sshkey.h" #include "kex.h" #include "digest.h" #include "mac.h" #include "log.h" #include "canohost.h" #include "misc.h" #include "channels.h" #include "ssh.h" #include "packet.h" #include "ssherr.h" #include "sshbuf.h" +#include "blacklist_client.h" #ifdef PACKET_DEBUG #define DBG(x) x #else #define DBG(x) #endif #define PACKET_MAX_SIZE (256 * 1024) struct packet_state { u_int32_t seqnr; u_int32_t packets; u_int64_t blocks; u_int64_t bytes; }; struct packet { TAILQ_ENTRY(packet) next; u_char type; struct sshbuf *payload; }; struct session_state { /* * This variable contains the file descriptors used for * communicating with the other side. connection_in is used for * reading; connection_out for writing. These can be the same * descriptor, in which case it is assumed to be a socket. */ int connection_in; int connection_out; /* Protocol flags for the remote side. */ u_int remote_protocol_flags; /* Encryption context for receiving data. Only used for decryption. */ struct sshcipher_ctx receive_context; /* Encryption context for sending data. Only used for encryption. */ struct sshcipher_ctx send_context; /* Buffer for raw input data from the socket. */ struct sshbuf *input; /* Buffer for raw output data going to the socket. */ struct sshbuf *output; /* Buffer for the partial outgoing packet being constructed. */ struct sshbuf *outgoing_packet; /* Buffer for the incoming packet currently being processed. */ struct sshbuf *incoming_packet; /* Scratch buffer for packet compression/decompression. */ struct sshbuf *compression_buffer; /* Incoming/outgoing compression dictionaries */ z_stream compression_in_stream; z_stream compression_out_stream; int compression_in_started; int compression_out_started; int compression_in_failures; int compression_out_failures; /* * Flag indicating whether packet compression/decompression is * enabled. */ int packet_compression; /* default maximum packet size */ u_int max_packet_size; /* Flag indicating whether this module has been initialized. */ int initialized; /* Set to true if the connection is interactive. */ int interactive_mode; /* Set to true if we are the server side. */ int server_side; /* Set to true if we are authenticated. */ int after_authentication; int keep_alive_timeouts; /* The maximum time that we will wait to send or receive a packet */ int packet_timeout_ms; /* Session key information for Encryption and MAC */ struct newkeys *newkeys[MODE_MAX]; struct packet_state p_read, p_send; /* Volume-based rekeying */ u_int64_t max_blocks_in, max_blocks_out, rekey_limit; /* Time-based rekeying */ u_int32_t rekey_interval; /* how often in seconds */ time_t rekey_time; /* time of last rekeying */ /* Session key for protocol v1 */ u_char ssh1_key[SSH_SESSION_KEY_LENGTH]; u_int ssh1_keylen; /* roundup current message to extra_pad bytes */ u_char extra_pad; /* XXX discard incoming data after MAC error */ u_int packet_discard; struct sshmac *packet_discard_mac; /* Used in packet_read_poll2() */ u_int packlen; /* Used in packet_send2 */ int rekeying; /* Used in packet_set_interactive */ int set_interactive_called; /* Used in packet_set_maxsize */ int set_maxsize_called; /* One-off warning about weak ciphers */ int cipher_warning_done; /* SSH1 CRC compensation attack detector */ struct deattack_ctx deattack; TAILQ_HEAD(, packet) outgoing; }; struct ssh * ssh_alloc_session_state(void) { struct ssh *ssh = NULL; struct session_state *state = NULL; if ((ssh = calloc(1, sizeof(*ssh))) == NULL || (state = calloc(1, sizeof(*state))) == NULL || (state->input = sshbuf_new()) == NULL || (state->output = sshbuf_new()) == NULL || (state->outgoing_packet = sshbuf_new()) == NULL || (state->incoming_packet = sshbuf_new()) == NULL) goto fail; TAILQ_INIT(&state->outgoing); TAILQ_INIT(&ssh->private_keys); TAILQ_INIT(&ssh->public_keys); state->connection_in = -1; state->connection_out = -1; state->max_packet_size = 32768; state->packet_timeout_ms = -1; state->p_send.packets = state->p_read.packets = 0; state->initialized = 1; /* * ssh_packet_send2() needs to queue packets until * we've done the initial key exchange. */ state->rekeying = 1; ssh->state = state; return ssh; fail: if (state) { sshbuf_free(state->input); sshbuf_free(state->output); sshbuf_free(state->incoming_packet); sshbuf_free(state->outgoing_packet); free(state); } free(ssh); return NULL; } /* Returns nonzero if rekeying is in progress */ int ssh_packet_is_rekeying(struct ssh *ssh) { return compat20 && (ssh->state->rekeying || (ssh->kex != NULL && ssh->kex->done == 0)); } /* * Sets the descriptors used for communication. Disables encryption until * packet_set_encryption_key is called. */ struct ssh * ssh_packet_set_connection(struct ssh *ssh, int fd_in, int fd_out) { struct session_state *state; const struct sshcipher *none = cipher_by_name("none"); int r; if (none == NULL) { error("%s: cannot load cipher 'none'", __func__); return NULL; } if (ssh == NULL) ssh = ssh_alloc_session_state(); if (ssh == NULL) { error("%s: cound not allocate state", __func__); return NULL; } state = ssh->state; state->connection_in = fd_in; state->connection_out = fd_out; if ((r = cipher_init(&state->send_context, none, (const u_char *)"", 0, NULL, 0, CIPHER_ENCRYPT)) != 0 || (r = cipher_init(&state->receive_context, none, (const u_char *)"", 0, NULL, 0, CIPHER_DECRYPT)) != 0) { error("%s: cipher_init failed: %s", __func__, ssh_err(r)); free(ssh); return NULL; } state->newkeys[MODE_IN] = state->newkeys[MODE_OUT] = NULL; deattack_init(&state->deattack); /* * Cache the IP address of the remote connection for use in error * messages that might be generated after the connection has closed. */ (void)ssh_remote_ipaddr(ssh); return ssh; } void ssh_packet_set_timeout(struct ssh *ssh, int timeout, int count) { struct session_state *state = ssh->state; if (timeout <= 0 || count <= 0) { state->packet_timeout_ms = -1; return; } if ((INT_MAX / 1000) / count < timeout) state->packet_timeout_ms = INT_MAX; else state->packet_timeout_ms = timeout * count * 1000; } int ssh_packet_stop_discard(struct ssh *ssh) { struct session_state *state = ssh->state; int r; if (state->packet_discard_mac) { char buf[1024]; memset(buf, 'a', sizeof(buf)); while (sshbuf_len(state->incoming_packet) < PACKET_MAX_SIZE) if ((r = sshbuf_put(state->incoming_packet, buf, sizeof(buf))) != 0) return r; (void) mac_compute(state->packet_discard_mac, state->p_read.seqnr, sshbuf_ptr(state->incoming_packet), PACKET_MAX_SIZE, NULL, 0); } logit("Finished discarding for %.200s port %d", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh)); return SSH_ERR_MAC_INVALID; } static int ssh_packet_start_discard(struct ssh *ssh, struct sshenc *enc, struct sshmac *mac, u_int packet_length, u_int discard) { struct session_state *state = ssh->state; int r; if (enc == NULL || !cipher_is_cbc(enc->cipher) || (mac && mac->etm)) { if ((r = sshpkt_disconnect(ssh, "Packet corrupt")) != 0) return r; return SSH_ERR_MAC_INVALID; } if (packet_length != PACKET_MAX_SIZE && mac && mac->enabled) state->packet_discard_mac = mac; if (sshbuf_len(state->input) >= discard && (r = ssh_packet_stop_discard(ssh)) != 0) return r; state->packet_discard = discard - sshbuf_len(state->input); return 0; } /* Returns 1 if remote host is connected via socket, 0 if not. */ int ssh_packet_connection_is_on_socket(struct ssh *ssh) { struct session_state *state = ssh->state; struct sockaddr_storage from, to; socklen_t fromlen, tolen; /* filedescriptors in and out are the same, so it's a socket */ if (state->connection_in == state->connection_out) return 1; fromlen = sizeof(from); memset(&from, 0, sizeof(from)); if (getpeername(state->connection_in, (struct sockaddr *)&from, &fromlen) < 0) return 0; tolen = sizeof(to); memset(&to, 0, sizeof(to)); if (getpeername(state->connection_out, (struct sockaddr *)&to, &tolen) < 0) return 0; if (fromlen != tolen || memcmp(&from, &to, fromlen) != 0) return 0; if (from.ss_family != AF_INET && from.ss_family != AF_INET6) return 0; return 1; } void ssh_packet_get_bytes(struct ssh *ssh, u_int64_t *ibytes, u_int64_t *obytes) { if (ibytes) *ibytes = ssh->state->p_read.bytes; if (obytes) *obytes = ssh->state->p_send.bytes; } int ssh_packet_connection_af(struct ssh *ssh) { struct sockaddr_storage to; socklen_t tolen = sizeof(to); memset(&to, 0, sizeof(to)); if (getsockname(ssh->state->connection_out, (struct sockaddr *)&to, &tolen) < 0) return 0; #ifdef IPV4_IN_IPV6 if (to.ss_family == AF_INET6 && IN6_IS_ADDR_V4MAPPED(&((struct sockaddr_in6 *)&to)->sin6_addr)) return AF_INET; #endif return to.ss_family; } /* Sets the connection into non-blocking mode. */ void ssh_packet_set_nonblocking(struct ssh *ssh) { /* Set the socket into non-blocking mode. */ set_nonblock(ssh->state->connection_in); if (ssh->state->connection_out != ssh->state->connection_in) set_nonblock(ssh->state->connection_out); } /* Returns the socket used for reading. */ int ssh_packet_get_connection_in(struct ssh *ssh) { return ssh->state->connection_in; } /* Returns the descriptor used for writing. */ int ssh_packet_get_connection_out(struct ssh *ssh) { return ssh->state->connection_out; } /* * Returns the IP-address of the remote host as a string. The returned * string must not be freed. */ const char * ssh_remote_ipaddr(struct ssh *ssh) { const int sock = ssh->state->connection_in; /* Check whether we have cached the ipaddr. */ if (ssh->remote_ipaddr == NULL) { if (ssh_packet_connection_is_on_socket(ssh)) { ssh->remote_ipaddr = get_peer_ipaddr(sock); ssh->remote_port = get_sock_port(sock, 0); } else { ssh->remote_ipaddr = strdup("UNKNOWN"); ssh->remote_port = 0; } } return ssh->remote_ipaddr; } /* Returns the port number of the remote host. */ int ssh_remote_port(struct ssh *ssh) { (void)ssh_remote_ipaddr(ssh); /* Will lookup and cache. */ return ssh->remote_port; } /* Closes the connection and clears and frees internal data structures. */ void ssh_packet_close(struct ssh *ssh) { struct session_state *state = ssh->state; int r; u_int mode; if (!state->initialized) return; state->initialized = 0; if (state->connection_in == state->connection_out) { shutdown(state->connection_out, SHUT_RDWR); close(state->connection_out); } else { close(state->connection_in); close(state->connection_out); } sshbuf_free(state->input); sshbuf_free(state->output); sshbuf_free(state->outgoing_packet); sshbuf_free(state->incoming_packet); for (mode = 0; mode < MODE_MAX; mode++) kex_free_newkeys(state->newkeys[mode]); if (state->compression_buffer) { sshbuf_free(state->compression_buffer); if (state->compression_out_started) { z_streamp stream = &state->compression_out_stream; debug("compress outgoing: " "raw data %llu, compressed %llu, factor %.2f", (unsigned long long)stream->total_in, (unsigned long long)stream->total_out, stream->total_in == 0 ? 0.0 : (double) stream->total_out / stream->total_in); if (state->compression_out_failures == 0) deflateEnd(stream); } if (state->compression_in_started) { z_streamp stream = &state->compression_out_stream; debug("compress incoming: " "raw data %llu, compressed %llu, factor %.2f", (unsigned long long)stream->total_out, (unsigned long long)stream->total_in, stream->total_out == 0 ? 0.0 : (double) stream->total_in / stream->total_out); if (state->compression_in_failures == 0) inflateEnd(stream); } } if ((r = cipher_cleanup(&state->send_context)) != 0) error("%s: cipher_cleanup failed: %s", __func__, ssh_err(r)); if ((r = cipher_cleanup(&state->receive_context)) != 0) error("%s: cipher_cleanup failed: %s", __func__, ssh_err(r)); free(ssh->remote_ipaddr); ssh->remote_ipaddr = NULL; free(ssh->state); ssh->state = NULL; } /* Sets remote side protocol flags. */ void ssh_packet_set_protocol_flags(struct ssh *ssh, u_int protocol_flags) { ssh->state->remote_protocol_flags = protocol_flags; } /* Returns the remote protocol flags set earlier by the above function. */ u_int ssh_packet_get_protocol_flags(struct ssh *ssh) { return ssh->state->remote_protocol_flags; } /* * Starts packet compression from the next packet on in both directions. * Level is compression level 1 (fastest) - 9 (slow, best) as in gzip. */ static int ssh_packet_init_compression(struct ssh *ssh) { if (!ssh->state->compression_buffer && ((ssh->state->compression_buffer = sshbuf_new()) == NULL)) return SSH_ERR_ALLOC_FAIL; return 0; } static int start_compression_out(struct ssh *ssh, int level) { if (level < 1 || level > 9) return SSH_ERR_INVALID_ARGUMENT; debug("Enabling compression at level %d.", level); if (ssh->state->compression_out_started == 1) deflateEnd(&ssh->state->compression_out_stream); switch (deflateInit(&ssh->state->compression_out_stream, level)) { case Z_OK: ssh->state->compression_out_started = 1; break; case Z_MEM_ERROR: return SSH_ERR_ALLOC_FAIL; default: return SSH_ERR_INTERNAL_ERROR; } return 0; } static int start_compression_in(struct ssh *ssh) { if (ssh->state->compression_in_started == 1) inflateEnd(&ssh->state->compression_in_stream); switch (inflateInit(&ssh->state->compression_in_stream)) { case Z_OK: ssh->state->compression_in_started = 1; break; case Z_MEM_ERROR: return SSH_ERR_ALLOC_FAIL; default: return SSH_ERR_INTERNAL_ERROR; } return 0; } int ssh_packet_start_compression(struct ssh *ssh, int level) { int r; if (ssh->state->packet_compression && !compat20) return SSH_ERR_INTERNAL_ERROR; ssh->state->packet_compression = 1; if ((r = ssh_packet_init_compression(ssh)) != 0 || (r = start_compression_in(ssh)) != 0 || (r = start_compression_out(ssh, level)) != 0) return r; return 0; } /* XXX remove need for separate compression buffer */ static int compress_buffer(struct ssh *ssh, struct sshbuf *in, struct sshbuf *out) { u_char buf[4096]; int r, status; if (ssh->state->compression_out_started != 1) return SSH_ERR_INTERNAL_ERROR; /* This case is not handled below. */ if (sshbuf_len(in) == 0) return 0; /* Input is the contents of the input buffer. */ if ((ssh->state->compression_out_stream.next_in = sshbuf_mutable_ptr(in)) == NULL) return SSH_ERR_INTERNAL_ERROR; ssh->state->compression_out_stream.avail_in = sshbuf_len(in); /* Loop compressing until deflate() returns with avail_out != 0. */ do { /* Set up fixed-size output buffer. */ ssh->state->compression_out_stream.next_out = buf; ssh->state->compression_out_stream.avail_out = sizeof(buf); /* Compress as much data into the buffer as possible. */ status = deflate(&ssh->state->compression_out_stream, Z_PARTIAL_FLUSH); switch (status) { case Z_MEM_ERROR: return SSH_ERR_ALLOC_FAIL; case Z_OK: /* Append compressed data to output_buffer. */ if ((r = sshbuf_put(out, buf, sizeof(buf) - ssh->state->compression_out_stream.avail_out)) != 0) return r; break; case Z_STREAM_ERROR: default: ssh->state->compression_out_failures++; return SSH_ERR_INVALID_FORMAT; } } while (ssh->state->compression_out_stream.avail_out == 0); return 0; } static int uncompress_buffer(struct ssh *ssh, struct sshbuf *in, struct sshbuf *out) { u_char buf[4096]; int r, status; if (ssh->state->compression_in_started != 1) return SSH_ERR_INTERNAL_ERROR; if ((ssh->state->compression_in_stream.next_in = sshbuf_mutable_ptr(in)) == NULL) return SSH_ERR_INTERNAL_ERROR; ssh->state->compression_in_stream.avail_in = sshbuf_len(in); for (;;) { /* Set up fixed-size output buffer. */ ssh->state->compression_in_stream.next_out = buf; ssh->state->compression_in_stream.avail_out = sizeof(buf); status = inflate(&ssh->state->compression_in_stream, Z_PARTIAL_FLUSH); switch (status) { case Z_OK: if ((r = sshbuf_put(out, buf, sizeof(buf) - ssh->state->compression_in_stream.avail_out)) != 0) return r; break; case Z_BUF_ERROR: /* * Comments in zlib.h say that we should keep calling * inflate() until we get an error. This appears to * be the error that we get. */ return 0; case Z_DATA_ERROR: return SSH_ERR_INVALID_FORMAT; case Z_MEM_ERROR: return SSH_ERR_ALLOC_FAIL; case Z_STREAM_ERROR: default: ssh->state->compression_in_failures++; return SSH_ERR_INTERNAL_ERROR; } } /* NOTREACHED */ } /* Serialise compression state into a blob for privsep */ static int ssh_packet_get_compress_state(struct sshbuf *m, struct ssh *ssh) { struct session_state *state = ssh->state; struct sshbuf *b; int r; if ((b = sshbuf_new()) == NULL) return SSH_ERR_ALLOC_FAIL; if (state->compression_in_started) { if ((r = sshbuf_put_string(b, &state->compression_in_stream, sizeof(state->compression_in_stream))) != 0) goto out; } else if ((r = sshbuf_put_string(b, NULL, 0)) != 0) goto out; if (state->compression_out_started) { if ((r = sshbuf_put_string(b, &state->compression_out_stream, sizeof(state->compression_out_stream))) != 0) goto out; } else if ((r = sshbuf_put_string(b, NULL, 0)) != 0) goto out; r = sshbuf_put_stringb(m, b); out: sshbuf_free(b); return r; } /* Deserialise compression state from a blob for privsep */ static int ssh_packet_set_compress_state(struct ssh *ssh, struct sshbuf *m) { struct session_state *state = ssh->state; struct sshbuf *b = NULL; int r; const u_char *inblob, *outblob; size_t inl, outl; if ((r = sshbuf_froms(m, &b)) != 0) goto out; if ((r = sshbuf_get_string_direct(b, &inblob, &inl)) != 0 || (r = sshbuf_get_string_direct(b, &outblob, &outl)) != 0) goto out; if (inl == 0) state->compression_in_started = 0; else if (inl != sizeof(state->compression_in_stream)) { r = SSH_ERR_INTERNAL_ERROR; goto out; } else { state->compression_in_started = 1; memcpy(&state->compression_in_stream, inblob, inl); } if (outl == 0) state->compression_out_started = 0; else if (outl != sizeof(state->compression_out_stream)) { r = SSH_ERR_INTERNAL_ERROR; goto out; } else { state->compression_out_started = 1; memcpy(&state->compression_out_stream, outblob, outl); } r = 0; out: sshbuf_free(b); return r; } void ssh_packet_set_compress_hooks(struct ssh *ssh, void *ctx, void *(*allocfunc)(void *, u_int, u_int), void (*freefunc)(void *, void *)) { ssh->state->compression_out_stream.zalloc = (alloc_func)allocfunc; ssh->state->compression_out_stream.zfree = (free_func)freefunc; ssh->state->compression_out_stream.opaque = ctx; ssh->state->compression_in_stream.zalloc = (alloc_func)allocfunc; ssh->state->compression_in_stream.zfree = (free_func)freefunc; ssh->state->compression_in_stream.opaque = ctx; } /* * Causes any further packets to be encrypted using the given key. The same * key is used for both sending and reception. However, both directions are * encrypted independently of each other. */ void ssh_packet_set_encryption_key(struct ssh *ssh, const u_char *key, u_int keylen, int number) { #ifndef WITH_SSH1 fatal("no SSH protocol 1 support"); #else /* WITH_SSH1 */ struct session_state *state = ssh->state; const struct sshcipher *cipher = cipher_by_number(number); int r; const char *wmsg; if (cipher == NULL) fatal("%s: unknown cipher number %d", __func__, number); if (keylen < 20) fatal("%s: keylen too small: %d", __func__, keylen); if (keylen > SSH_SESSION_KEY_LENGTH) fatal("%s: keylen too big: %d", __func__, keylen); memcpy(state->ssh1_key, key, keylen); state->ssh1_keylen = keylen; if ((r = cipher_init(&state->send_context, cipher, key, keylen, NULL, 0, CIPHER_ENCRYPT)) != 0 || (r = cipher_init(&state->receive_context, cipher, key, keylen, NULL, 0, CIPHER_DECRYPT) != 0)) fatal("%s: cipher_init failed: %s", __func__, ssh_err(r)); if (!state->cipher_warning_done && ((wmsg = cipher_warning_message(&state->send_context)) != NULL || (wmsg = cipher_warning_message(&state->send_context)) != NULL)) { error("Warning: %s", wmsg); state->cipher_warning_done = 1; } #endif /* WITH_SSH1 */ } /* * Finalizes and sends the packet. If the encryption key has been set, * encrypts the packet before sending. */ int ssh_packet_send1(struct ssh *ssh) { struct session_state *state = ssh->state; u_char buf[8], *cp; int r, padding, len; u_int checksum; /* * If using packet compression, compress the payload of the outgoing * packet. */ if (state->packet_compression) { sshbuf_reset(state->compression_buffer); /* Skip padding. */ if ((r = sshbuf_consume(state->outgoing_packet, 8)) != 0) goto out; /* padding */ if ((r = sshbuf_put(state->compression_buffer, "\0\0\0\0\0\0\0\0", 8)) != 0) goto out; if ((r = compress_buffer(ssh, state->outgoing_packet, state->compression_buffer)) != 0) goto out; sshbuf_reset(state->outgoing_packet); if ((r = sshbuf_putb(state->outgoing_packet, state->compression_buffer)) != 0) goto out; } /* Compute packet length without padding (add checksum, remove padding). */ len = sshbuf_len(state->outgoing_packet) + 4 - 8; /* Insert padding. Initialized to zero in packet_start1() */ padding = 8 - len % 8; if (!state->send_context.plaintext) { cp = sshbuf_mutable_ptr(state->outgoing_packet); if (cp == NULL) { r = SSH_ERR_INTERNAL_ERROR; goto out; } arc4random_buf(cp + 8 - padding, padding); } if ((r = sshbuf_consume(state->outgoing_packet, 8 - padding)) != 0) goto out; /* Add check bytes. */ checksum = ssh_crc32(sshbuf_ptr(state->outgoing_packet), sshbuf_len(state->outgoing_packet)); POKE_U32(buf, checksum); if ((r = sshbuf_put(state->outgoing_packet, buf, 4)) != 0) goto out; #ifdef PACKET_DEBUG fprintf(stderr, "packet_send plain: "); sshbuf_dump(state->outgoing_packet, stderr); #endif /* Append to output. */ POKE_U32(buf, len); if ((r = sshbuf_put(state->output, buf, 4)) != 0) goto out; if ((r = sshbuf_reserve(state->output, sshbuf_len(state->outgoing_packet), &cp)) != 0) goto out; if ((r = cipher_crypt(&state->send_context, 0, cp, sshbuf_ptr(state->outgoing_packet), sshbuf_len(state->outgoing_packet), 0, 0)) != 0) goto out; #ifdef PACKET_DEBUG fprintf(stderr, "encrypted: "); sshbuf_dump(state->output, stderr); #endif state->p_send.packets++; state->p_send.bytes += len + sshbuf_len(state->outgoing_packet); sshbuf_reset(state->outgoing_packet); /* * Note that the packet is now only buffered in output. It won't be * actually sent until ssh_packet_write_wait or ssh_packet_write_poll * is called. */ r = 0; out: return r; } int ssh_set_newkeys(struct ssh *ssh, int mode) { struct session_state *state = ssh->state; struct sshenc *enc; struct sshmac *mac; struct sshcomp *comp; struct sshcipher_ctx *cc; u_int64_t *max_blocks; const char *wmsg; int r, crypt_type; debug2("set_newkeys: mode %d", mode); if (mode == MODE_OUT) { cc = &state->send_context; crypt_type = CIPHER_ENCRYPT; state->p_send.packets = state->p_send.blocks = 0; max_blocks = &state->max_blocks_out; } else { cc = &state->receive_context; crypt_type = CIPHER_DECRYPT; state->p_read.packets = state->p_read.blocks = 0; max_blocks = &state->max_blocks_in; } if (state->newkeys[mode] != NULL) { debug("set_newkeys: rekeying, input %llu bytes %llu blocks, " "output %llu bytes %llu blocks", (unsigned long long)state->p_read.bytes, (unsigned long long)state->p_read.blocks, (unsigned long long)state->p_send.bytes, (unsigned long long)state->p_send.blocks); if ((r = cipher_cleanup(cc)) != 0) return r; enc = &state->newkeys[mode]->enc; mac = &state->newkeys[mode]->mac; comp = &state->newkeys[mode]->comp; mac_clear(mac); explicit_bzero(enc->iv, enc->iv_len); explicit_bzero(enc->key, enc->key_len); explicit_bzero(mac->key, mac->key_len); free(enc->name); free(enc->iv); free(enc->key); free(mac->name); free(mac->key); free(comp->name); free(state->newkeys[mode]); } /* move newkeys from kex to state */ if ((state->newkeys[mode] = ssh->kex->newkeys[mode]) == NULL) return SSH_ERR_INTERNAL_ERROR; ssh->kex->newkeys[mode] = NULL; enc = &state->newkeys[mode]->enc; mac = &state->newkeys[mode]->mac; comp = &state->newkeys[mode]->comp; if (cipher_authlen(enc->cipher) == 0) { if ((r = mac_init(mac)) != 0) return r; } mac->enabled = 1; DBG(debug("cipher_init_context: %d", mode)); if ((r = cipher_init(cc, enc->cipher, enc->key, enc->key_len, enc->iv, enc->iv_len, crypt_type)) != 0) return r; if (!state->cipher_warning_done && (wmsg = cipher_warning_message(cc)) != NULL) { error("Warning: %s", wmsg); state->cipher_warning_done = 1; } /* Deleting the keys does not gain extra security */ /* explicit_bzero(enc->iv, enc->block_size); explicit_bzero(enc->key, enc->key_len); explicit_bzero(mac->key, mac->key_len); */ if ((comp->type == COMP_ZLIB || (comp->type == COMP_DELAYED && state->after_authentication)) && comp->enabled == 0) { if ((r = ssh_packet_init_compression(ssh)) < 0) return r; if (mode == MODE_OUT) { if ((r = start_compression_out(ssh, 6)) != 0) return r; } else { if ((r = start_compression_in(ssh)) != 0) return r; } comp->enabled = 1; } /* * The 2^(blocksize*2) limit is too expensive for 3DES, * blowfish, etc, so enforce a 1GB limit for small blocksizes. */ if (enc->block_size >= 16) *max_blocks = (u_int64_t)1 << (enc->block_size*2); else *max_blocks = ((u_int64_t)1 << 30) / enc->block_size; if (state->rekey_limit) *max_blocks = MIN(*max_blocks, state->rekey_limit / enc->block_size); debug("rekey after %llu blocks", (unsigned long long)*max_blocks); return 0; } #define MAX_PACKETS (1U<<31) static int ssh_packet_need_rekeying(struct ssh *ssh, u_int outbound_packet_len) { struct session_state *state = ssh->state; u_int32_t out_blocks; /* XXX client can't cope with rekeying pre-auth */ if (!state->after_authentication) return 0; /* Haven't keyed yet or KEX in progress. */ if (ssh->kex == NULL || ssh_packet_is_rekeying(ssh)) return 0; /* Peer can't rekey */ if (ssh->compat & SSH_BUG_NOREKEY) return 0; /* * Permit one packet in or out per rekey - this allows us to * make progress when rekey limits are very small. */ if (state->p_send.packets == 0 && state->p_read.packets == 0) return 0; /* Time-based rekeying */ if (state->rekey_interval != 0 && state->rekey_time + state->rekey_interval <= monotime()) return 1; /* Always rekey when MAX_PACKETS sent in either direction */ if (state->p_send.packets > MAX_PACKETS || state->p_read.packets > MAX_PACKETS) return 1; /* Rekey after (cipher-specific) maxiumum blocks */ out_blocks = roundup(outbound_packet_len, state->newkeys[MODE_OUT]->enc.block_size); return (state->max_blocks_out && (state->p_send.blocks + out_blocks > state->max_blocks_out)) || (state->max_blocks_in && (state->p_read.blocks > state->max_blocks_in)); } /* * Delayed compression for SSH2 is enabled after authentication: * This happens on the server side after a SSH2_MSG_USERAUTH_SUCCESS is sent, * and on the client side after a SSH2_MSG_USERAUTH_SUCCESS is received. */ static int ssh_packet_enable_delayed_compress(struct ssh *ssh) { struct session_state *state = ssh->state; struct sshcomp *comp = NULL; int r, mode; /* * Remember that we are past the authentication step, so rekeying * with COMP_DELAYED will turn on compression immediately. */ state->after_authentication = 1; for (mode = 0; mode < MODE_MAX; mode++) { /* protocol error: USERAUTH_SUCCESS received before NEWKEYS */ if (state->newkeys[mode] == NULL) continue; comp = &state->newkeys[mode]->comp; if (comp && !comp->enabled && comp->type == COMP_DELAYED) { if ((r = ssh_packet_init_compression(ssh)) != 0) return r; if (mode == MODE_OUT) { if ((r = start_compression_out(ssh, 6)) != 0) return r; } else { if ((r = start_compression_in(ssh)) != 0) return r; } comp->enabled = 1; } } return 0; } /* Used to mute debug logging for noisy packet types */ static int ssh_packet_log_type(u_char type) { switch (type) { case SSH2_MSG_CHANNEL_DATA: case SSH2_MSG_CHANNEL_EXTENDED_DATA: case SSH2_MSG_CHANNEL_WINDOW_ADJUST: return 0; default: return 1; } } /* * Finalize packet in SSH2 format (compress, mac, encrypt, enqueue) */ int ssh_packet_send2_wrapped(struct ssh *ssh) { struct session_state *state = ssh->state; u_char type, *cp, macbuf[SSH_DIGEST_MAX_LENGTH]; u_char padlen, pad = 0; u_int authlen = 0, aadlen = 0; u_int len; struct sshenc *enc = NULL; struct sshmac *mac = NULL; struct sshcomp *comp = NULL; int r, block_size; if (state->newkeys[MODE_OUT] != NULL) { enc = &state->newkeys[MODE_OUT]->enc; mac = &state->newkeys[MODE_OUT]->mac; comp = &state->newkeys[MODE_OUT]->comp; /* disable mac for authenticated encryption */ if ((authlen = cipher_authlen(enc->cipher)) != 0) mac = NULL; } block_size = enc ? enc->block_size : 8; aadlen = (mac && mac->enabled && mac->etm) || authlen ? 4 : 0; type = (sshbuf_ptr(state->outgoing_packet))[5]; if (ssh_packet_log_type(type)) debug3("send packet: type %u", type); #ifdef PACKET_DEBUG fprintf(stderr, "plain: "); sshbuf_dump(state->outgoing_packet, stderr); #endif if (comp && comp->enabled) { len = sshbuf_len(state->outgoing_packet); /* skip header, compress only payload */ if ((r = sshbuf_consume(state->outgoing_packet, 5)) != 0) goto out; sshbuf_reset(state->compression_buffer); if ((r = compress_buffer(ssh, state->outgoing_packet, state->compression_buffer)) != 0) goto out; sshbuf_reset(state->outgoing_packet); if ((r = sshbuf_put(state->outgoing_packet, "\0\0\0\0\0", 5)) != 0 || (r = sshbuf_putb(state->outgoing_packet, state->compression_buffer)) != 0) goto out; DBG(debug("compression: raw %d compressed %zd", len, sshbuf_len(state->outgoing_packet))); } /* sizeof (packet_len + pad_len + payload) */ len = sshbuf_len(state->outgoing_packet); /* * calc size of padding, alloc space, get random data, * minimum padding is 4 bytes */ len -= aadlen; /* packet length is not encrypted for EtM modes */ padlen = block_size - (len % block_size); if (padlen < 4) padlen += block_size; if (state->extra_pad) { /* will wrap if extra_pad+padlen > 255 */ state->extra_pad = roundup(state->extra_pad, block_size); pad = state->extra_pad - ((len + padlen) % state->extra_pad); DBG(debug3("%s: adding %d (len %d padlen %d extra_pad %d)", __func__, pad, len, padlen, state->extra_pad)); padlen += pad; state->extra_pad = 0; } if ((r = sshbuf_reserve(state->outgoing_packet, padlen, &cp)) != 0) goto out; if (enc && !state->send_context.plaintext) { /* random padding */ arc4random_buf(cp, padlen); } else { /* clear padding */ explicit_bzero(cp, padlen); } /* sizeof (packet_len + pad_len + payload + padding) */ len = sshbuf_len(state->outgoing_packet); cp = sshbuf_mutable_ptr(state->outgoing_packet); if (cp == NULL) { r = SSH_ERR_INTERNAL_ERROR; goto out; } /* packet_length includes payload, padding and padding length field */ POKE_U32(cp, len - 4); cp[4] = padlen; DBG(debug("send: len %d (includes padlen %d, aadlen %d)", len, padlen, aadlen)); /* compute MAC over seqnr and packet(length fields, payload, padding) */ if (mac && mac->enabled && !mac->etm) { if ((r = mac_compute(mac, state->p_send.seqnr, sshbuf_ptr(state->outgoing_packet), len, macbuf, sizeof(macbuf))) != 0) goto out; DBG(debug("done calc MAC out #%d", state->p_send.seqnr)); } /* encrypt packet and append to output buffer. */ if ((r = sshbuf_reserve(state->output, sshbuf_len(state->outgoing_packet) + authlen, &cp)) != 0) goto out; if ((r = cipher_crypt(&state->send_context, state->p_send.seqnr, cp, sshbuf_ptr(state->outgoing_packet), len - aadlen, aadlen, authlen)) != 0) goto out; /* append unencrypted MAC */ if (mac && mac->enabled) { if (mac->etm) { /* EtM: compute mac over aadlen + cipher text */ if ((r = mac_compute(mac, state->p_send.seqnr, cp, len, macbuf, sizeof(macbuf))) != 0) goto out; DBG(debug("done calc MAC(EtM) out #%d", state->p_send.seqnr)); } if ((r = sshbuf_put(state->output, macbuf, mac->mac_len)) != 0) goto out; } #ifdef PACKET_DEBUG fprintf(stderr, "encrypted: "); sshbuf_dump(state->output, stderr); #endif /* increment sequence number for outgoing packets */ if (++state->p_send.seqnr == 0) logit("outgoing seqnr wraps around"); if (++state->p_send.packets == 0) if (!(ssh->compat & SSH_BUG_NOREKEY)) return SSH_ERR_NEED_REKEY; state->p_send.blocks += len / block_size; state->p_send.bytes += len; sshbuf_reset(state->outgoing_packet); if (type == SSH2_MSG_NEWKEYS) r = ssh_set_newkeys(ssh, MODE_OUT); else if (type == SSH2_MSG_USERAUTH_SUCCESS && state->server_side) r = ssh_packet_enable_delayed_compress(ssh); else r = 0; out: return r; } /* returns non-zero if the specified packet type is usec by KEX */ static int ssh_packet_type_is_kex(u_char type) { return type >= SSH2_MSG_TRANSPORT_MIN && type <= SSH2_MSG_TRANSPORT_MAX && type != SSH2_MSG_SERVICE_REQUEST && type != SSH2_MSG_SERVICE_ACCEPT && type != SSH2_MSG_EXT_INFO; } int ssh_packet_send2(struct ssh *ssh) { struct session_state *state = ssh->state; struct packet *p; u_char type; int r, need_rekey; if (sshbuf_len(state->outgoing_packet) < 6) return SSH_ERR_INTERNAL_ERROR; type = sshbuf_ptr(state->outgoing_packet)[5]; need_rekey = !ssh_packet_type_is_kex(type) && ssh_packet_need_rekeying(ssh, sshbuf_len(state->outgoing_packet)); /* * During rekeying we can only send key exchange messages. * Queue everything else. */ if ((need_rekey || state->rekeying) && !ssh_packet_type_is_kex(type)) { if (need_rekey) debug3("%s: rekex triggered", __func__); debug("enqueue packet: %u", type); p = calloc(1, sizeof(*p)); if (p == NULL) return SSH_ERR_ALLOC_FAIL; p->type = type; p->payload = state->outgoing_packet; TAILQ_INSERT_TAIL(&state->outgoing, p, next); state->outgoing_packet = sshbuf_new(); if (state->outgoing_packet == NULL) return SSH_ERR_ALLOC_FAIL; if (need_rekey) { /* * This packet triggered a rekey, so send the * KEXINIT now. * NB. reenters this function via kex_start_rekex(). */ return kex_start_rekex(ssh); } return 0; } /* rekeying starts with sending KEXINIT */ if (type == SSH2_MSG_KEXINIT) state->rekeying = 1; if ((r = ssh_packet_send2_wrapped(ssh)) != 0) return r; /* after a NEWKEYS message we can send the complete queue */ if (type == SSH2_MSG_NEWKEYS) { state->rekeying = 0; state->rekey_time = monotime(); while ((p = TAILQ_FIRST(&state->outgoing))) { type = p->type; /* * If this packet triggers a rekex, then skip the * remaining packets in the queue for now. * NB. re-enters this function via kex_start_rekex. */ if (ssh_packet_need_rekeying(ssh, sshbuf_len(p->payload))) { debug3("%s: queued packet triggered rekex", __func__); return kex_start_rekex(ssh); } debug("dequeue packet: %u", type); sshbuf_free(state->outgoing_packet); state->outgoing_packet = p->payload; TAILQ_REMOVE(&state->outgoing, p, next); memset(p, 0, sizeof(*p)); free(p); if ((r = ssh_packet_send2_wrapped(ssh)) != 0) return r; } } return 0; } /* * Waits until a packet has been received, and returns its type. Note that * no other data is processed until this returns, so this function should not * be used during the interactive session. */ int ssh_packet_read_seqnr(struct ssh *ssh, u_char *typep, u_int32_t *seqnr_p) { struct session_state *state = ssh->state; int len, r, ms_remain; fd_set *setp; char buf[8192]; struct timeval timeout, start, *timeoutp = NULL; DBG(debug("packet_read()")); setp = calloc(howmany(state->connection_in + 1, NFDBITS), sizeof(fd_mask)); if (setp == NULL) return SSH_ERR_ALLOC_FAIL; /* * Since we are blocking, ensure that all written packets have * been sent. */ if ((r = ssh_packet_write_wait(ssh)) != 0) goto out; /* Stay in the loop until we have received a complete packet. */ for (;;) { /* Try to read a packet from the buffer. */ r = ssh_packet_read_poll_seqnr(ssh, typep, seqnr_p); if (r != 0) break; if (!compat20 && ( *typep == SSH_SMSG_SUCCESS || *typep == SSH_SMSG_FAILURE || *typep == SSH_CMSG_EOF || *typep == SSH_CMSG_EXIT_CONFIRMATION)) if ((r = sshpkt_get_end(ssh)) != 0) break; /* If we got a packet, return it. */ if (*typep != SSH_MSG_NONE) break; /* * Otherwise, wait for some data to arrive, add it to the * buffer, and try again. */ memset(setp, 0, howmany(state->connection_in + 1, NFDBITS) * sizeof(fd_mask)); FD_SET(state->connection_in, setp); if (state->packet_timeout_ms > 0) { ms_remain = state->packet_timeout_ms; timeoutp = &timeout; } /* Wait for some data to arrive. */ for (;;) { if (state->packet_timeout_ms != -1) { ms_to_timeval(&timeout, ms_remain); gettimeofday(&start, NULL); } if ((r = select(state->connection_in + 1, setp, NULL, NULL, timeoutp)) >= 0) break; if (errno != EAGAIN && errno != EINTR && errno != EWOULDBLOCK) break; if (state->packet_timeout_ms == -1) continue; ms_subtract_diff(&start, &ms_remain); if (ms_remain <= 0) { r = 0; break; } } if (r == 0) return SSH_ERR_CONN_TIMEOUT; /* Read data from the socket. */ len = read(state->connection_in, buf, sizeof(buf)); if (len == 0) { r = SSH_ERR_CONN_CLOSED; goto out; } if (len < 0) { r = SSH_ERR_SYSTEM_ERROR; goto out; } /* Append it to the buffer. */ if ((r = ssh_packet_process_incoming(ssh, buf, len)) != 0) goto out; } out: free(setp); return r; } int ssh_packet_read(struct ssh *ssh) { u_char type; int r; if ((r = ssh_packet_read_seqnr(ssh, &type, NULL)) != 0) fatal("%s: %s", __func__, ssh_err(r)); return type; } /* * Waits until a packet has been received, verifies that its type matches * that given, and gives a fatal error and exits if there is a mismatch. */ int ssh_packet_read_expect(struct ssh *ssh, u_int expected_type) { int r; u_char type; if ((r = ssh_packet_read_seqnr(ssh, &type, NULL)) != 0) return r; if (type != expected_type) { if ((r = sshpkt_disconnect(ssh, "Protocol error: expected packet type %d, got %d", expected_type, type)) != 0) return r; return SSH_ERR_PROTOCOL_ERROR; } return 0; } /* Checks if a full packet is available in the data received so far via * packet_process_incoming. If so, reads the packet; otherwise returns * SSH_MSG_NONE. This does not wait for data from the connection. * * SSH_MSG_DISCONNECT is handled specially here. Also, * SSH_MSG_IGNORE messages are skipped by this function and are never returned * to higher levels. */ int ssh_packet_read_poll1(struct ssh *ssh, u_char *typep) { struct session_state *state = ssh->state; u_int len, padded_len; const char *emsg; const u_char *cp; u_char *p; u_int checksum, stored_checksum; int r; *typep = SSH_MSG_NONE; /* Check if input size is less than minimum packet size. */ if (sshbuf_len(state->input) < 4 + 8) return 0; /* Get length of incoming packet. */ len = PEEK_U32(sshbuf_ptr(state->input)); if (len < 1 + 2 + 2 || len > 256 * 1024) { if ((r = sshpkt_disconnect(ssh, "Bad packet length %u", len)) != 0) return r; return SSH_ERR_CONN_CORRUPT; } padded_len = (len + 8) & ~7; /* Check if the packet has been entirely received. */ if (sshbuf_len(state->input) < 4 + padded_len) return 0; /* The entire packet is in buffer. */ /* Consume packet length. */ if ((r = sshbuf_consume(state->input, 4)) != 0) goto out; /* * Cryptographic attack detector for ssh * (C)1998 CORE-SDI, Buenos Aires Argentina * Ariel Futoransky(futo@core-sdi.com) */ if (!state->receive_context.plaintext) { emsg = NULL; switch (detect_attack(&state->deattack, sshbuf_ptr(state->input), padded_len)) { case DEATTACK_OK: break; case DEATTACK_DETECTED: emsg = "crc32 compensation attack detected"; break; case DEATTACK_DOS_DETECTED: emsg = "deattack denial of service detected"; break; default: emsg = "deattack error"; break; } if (emsg != NULL) { error("%s", emsg); if ((r = sshpkt_disconnect(ssh, "%s", emsg)) != 0 || (r = ssh_packet_write_wait(ssh)) != 0) return r; return SSH_ERR_CONN_CORRUPT; } } /* Decrypt data to incoming_packet. */ sshbuf_reset(state->incoming_packet); if ((r = sshbuf_reserve(state->incoming_packet, padded_len, &p)) != 0) goto out; if ((r = cipher_crypt(&state->receive_context, 0, p, sshbuf_ptr(state->input), padded_len, 0, 0)) != 0) goto out; if ((r = sshbuf_consume(state->input, padded_len)) != 0) goto out; #ifdef PACKET_DEBUG fprintf(stderr, "read_poll plain: "); sshbuf_dump(state->incoming_packet, stderr); #endif /* Compute packet checksum. */ checksum = ssh_crc32(sshbuf_ptr(state->incoming_packet), sshbuf_len(state->incoming_packet) - 4); /* Skip padding. */ if ((r = sshbuf_consume(state->incoming_packet, 8 - len % 8)) != 0) goto out; /* Test check bytes. */ if (len != sshbuf_len(state->incoming_packet)) { error("%s: len %d != sshbuf_len %zd", __func__, len, sshbuf_len(state->incoming_packet)); if ((r = sshpkt_disconnect(ssh, "invalid packet length")) != 0 || (r = ssh_packet_write_wait(ssh)) != 0) return r; return SSH_ERR_CONN_CORRUPT; } cp = sshbuf_ptr(state->incoming_packet) + len - 4; stored_checksum = PEEK_U32(cp); if (checksum != stored_checksum) { error("Corrupted check bytes on input"); if ((r = sshpkt_disconnect(ssh, "connection corrupted")) != 0 || (r = ssh_packet_write_wait(ssh)) != 0) return r; return SSH_ERR_CONN_CORRUPT; } if ((r = sshbuf_consume_end(state->incoming_packet, 4)) < 0) goto out; if (state->packet_compression) { sshbuf_reset(state->compression_buffer); if ((r = uncompress_buffer(ssh, state->incoming_packet, state->compression_buffer)) != 0) goto out; sshbuf_reset(state->incoming_packet); if ((r = sshbuf_putb(state->incoming_packet, state->compression_buffer)) != 0) goto out; } state->p_read.packets++; state->p_read.bytes += padded_len + 4; if ((r = sshbuf_get_u8(state->incoming_packet, typep)) != 0) goto out; if (*typep < SSH_MSG_MIN || *typep > SSH_MSG_MAX) { error("Invalid ssh1 packet type: %d", *typep); if ((r = sshpkt_disconnect(ssh, "invalid packet type")) != 0 || (r = ssh_packet_write_wait(ssh)) != 0) return r; return SSH_ERR_PROTOCOL_ERROR; } r = 0; out: return r; } int ssh_packet_read_poll2(struct ssh *ssh, u_char *typep, u_int32_t *seqnr_p) { struct session_state *state = ssh->state; u_int padlen, need; u_char *cp, macbuf[SSH_DIGEST_MAX_LENGTH]; u_int maclen, aadlen = 0, authlen = 0, block_size; struct sshenc *enc = NULL; struct sshmac *mac = NULL; struct sshcomp *comp = NULL; int r; *typep = SSH_MSG_NONE; if (state->packet_discard) return 0; if (state->newkeys[MODE_IN] != NULL) { enc = &state->newkeys[MODE_IN]->enc; mac = &state->newkeys[MODE_IN]->mac; comp = &state->newkeys[MODE_IN]->comp; /* disable mac for authenticated encryption */ if ((authlen = cipher_authlen(enc->cipher)) != 0) mac = NULL; } maclen = mac && mac->enabled ? mac->mac_len : 0; block_size = enc ? enc->block_size : 8; aadlen = (mac && mac->enabled && mac->etm) || authlen ? 4 : 0; if (aadlen && state->packlen == 0) { if (cipher_get_length(&state->receive_context, &state->packlen, state->p_read.seqnr, sshbuf_ptr(state->input), sshbuf_len(state->input)) != 0) return 0; if (state->packlen < 1 + 4 || state->packlen > PACKET_MAX_SIZE) { #ifdef PACKET_DEBUG sshbuf_dump(state->input, stderr); #endif logit("Bad packet length %u.", state->packlen); if ((r = sshpkt_disconnect(ssh, "Packet corrupt")) != 0) return r; return SSH_ERR_CONN_CORRUPT; } sshbuf_reset(state->incoming_packet); } else if (state->packlen == 0) { /* * check if input size is less than the cipher block size, * decrypt first block and extract length of incoming packet */ if (sshbuf_len(state->input) < block_size) return 0; sshbuf_reset(state->incoming_packet); if ((r = sshbuf_reserve(state->incoming_packet, block_size, &cp)) != 0) goto out; if ((r = cipher_crypt(&state->receive_context, state->p_send.seqnr, cp, sshbuf_ptr(state->input), block_size, 0, 0)) != 0) goto out; state->packlen = PEEK_U32(sshbuf_ptr(state->incoming_packet)); if (state->packlen < 1 + 4 || state->packlen > PACKET_MAX_SIZE) { #ifdef PACKET_DEBUG fprintf(stderr, "input: \n"); sshbuf_dump(state->input, stderr); fprintf(stderr, "incoming_packet: \n"); sshbuf_dump(state->incoming_packet, stderr); #endif logit("Bad packet length %u.", state->packlen); return ssh_packet_start_discard(ssh, enc, mac, state->packlen, PACKET_MAX_SIZE); } if ((r = sshbuf_consume(state->input, block_size)) != 0) goto out; } DBG(debug("input: packet len %u", state->packlen+4)); if (aadlen) { /* only the payload is encrypted */ need = state->packlen; } else { /* * the payload size and the payload are encrypted, but we * have a partial packet of block_size bytes */ need = 4 + state->packlen - block_size; } DBG(debug("partial packet: block %d, need %d, maclen %d, authlen %d," " aadlen %d", block_size, need, maclen, authlen, aadlen)); if (need % block_size != 0) { logit("padding error: need %d block %d mod %d", need, block_size, need % block_size); return ssh_packet_start_discard(ssh, enc, mac, state->packlen, PACKET_MAX_SIZE - block_size); } /* * check if the entire packet has been received and * decrypt into incoming_packet: * 'aadlen' bytes are unencrypted, but authenticated. * 'need' bytes are encrypted, followed by either * 'authlen' bytes of authentication tag or * 'maclen' bytes of message authentication code. */ if (sshbuf_len(state->input) < aadlen + need + authlen + maclen) return 0; #ifdef PACKET_DEBUG fprintf(stderr, "read_poll enc/full: "); sshbuf_dump(state->input, stderr); #endif /* EtM: compute mac over encrypted input */ if (mac && mac->enabled && mac->etm) { if ((r = mac_compute(mac, state->p_read.seqnr, sshbuf_ptr(state->input), aadlen + need, macbuf, sizeof(macbuf))) != 0) goto out; } if ((r = sshbuf_reserve(state->incoming_packet, aadlen + need, &cp)) != 0) goto out; if ((r = cipher_crypt(&state->receive_context, state->p_read.seqnr, cp, sshbuf_ptr(state->input), need, aadlen, authlen)) != 0) goto out; if ((r = sshbuf_consume(state->input, aadlen + need + authlen)) != 0) goto out; /* * compute MAC over seqnr and packet, * increment sequence number for incoming packet */ if (mac && mac->enabled) { if (!mac->etm) if ((r = mac_compute(mac, state->p_read.seqnr, sshbuf_ptr(state->incoming_packet), sshbuf_len(state->incoming_packet), macbuf, sizeof(macbuf))) != 0) goto out; if (timingsafe_bcmp(macbuf, sshbuf_ptr(state->input), mac->mac_len) != 0) { logit("Corrupted MAC on input."); if (need > PACKET_MAX_SIZE) return SSH_ERR_INTERNAL_ERROR; return ssh_packet_start_discard(ssh, enc, mac, state->packlen, PACKET_MAX_SIZE - need); } DBG(debug("MAC #%d ok", state->p_read.seqnr)); if ((r = sshbuf_consume(state->input, mac->mac_len)) != 0) goto out; } if (seqnr_p != NULL) *seqnr_p = state->p_read.seqnr; if (++state->p_read.seqnr == 0) logit("incoming seqnr wraps around"); if (++state->p_read.packets == 0) if (!(ssh->compat & SSH_BUG_NOREKEY)) return SSH_ERR_NEED_REKEY; state->p_read.blocks += (state->packlen + 4) / block_size; state->p_read.bytes += state->packlen + 4; /* get padlen */ padlen = sshbuf_ptr(state->incoming_packet)[4]; DBG(debug("input: padlen %d", padlen)); if (padlen < 4) { if ((r = sshpkt_disconnect(ssh, "Corrupted padlen %d on input.", padlen)) != 0 || (r = ssh_packet_write_wait(ssh)) != 0) return r; return SSH_ERR_CONN_CORRUPT; } /* skip packet size + padlen, discard padding */ if ((r = sshbuf_consume(state->incoming_packet, 4 + 1)) != 0 || ((r = sshbuf_consume_end(state->incoming_packet, padlen)) != 0)) goto out; DBG(debug("input: len before de-compress %zd", sshbuf_len(state->incoming_packet))); if (comp && comp->enabled) { sshbuf_reset(state->compression_buffer); if ((r = uncompress_buffer(ssh, state->incoming_packet, state->compression_buffer)) != 0) goto out; sshbuf_reset(state->incoming_packet); if ((r = sshbuf_putb(state->incoming_packet, state->compression_buffer)) != 0) goto out; DBG(debug("input: len after de-compress %zd", sshbuf_len(state->incoming_packet))); } /* * get packet type, implies consume. * return length of payload (without type field) */ if ((r = sshbuf_get_u8(state->incoming_packet, typep)) != 0) goto out; if (ssh_packet_log_type(*typep)) debug3("receive packet: type %u", *typep); if (*typep < SSH2_MSG_MIN || *typep >= SSH2_MSG_LOCAL_MIN) { if ((r = sshpkt_disconnect(ssh, "Invalid ssh2 packet type: %d", *typep)) != 0 || (r = ssh_packet_write_wait(ssh)) != 0) return r; return SSH_ERR_PROTOCOL_ERROR; } if (*typep == SSH2_MSG_NEWKEYS) r = ssh_set_newkeys(ssh, MODE_IN); else if (*typep == SSH2_MSG_USERAUTH_SUCCESS && !state->server_side) r = ssh_packet_enable_delayed_compress(ssh); else r = 0; #ifdef PACKET_DEBUG fprintf(stderr, "read/plain[%d]:\r\n", *typep); sshbuf_dump(state->incoming_packet, stderr); #endif /* reset for next packet */ state->packlen = 0; /* do we need to rekey? */ if (ssh_packet_need_rekeying(ssh, 0)) { debug3("%s: rekex triggered", __func__); if ((r = kex_start_rekex(ssh)) != 0) return r; } out: return r; } int ssh_packet_read_poll_seqnr(struct ssh *ssh, u_char *typep, u_int32_t *seqnr_p) { struct session_state *state = ssh->state; u_int reason, seqnr; int r; u_char *msg; for (;;) { msg = NULL; if (compat20) { r = ssh_packet_read_poll2(ssh, typep, seqnr_p); if (r != 0) return r; if (*typep) { state->keep_alive_timeouts = 0; DBG(debug("received packet type %d", *typep)); } switch (*typep) { case SSH2_MSG_IGNORE: debug3("Received SSH2_MSG_IGNORE"); break; case SSH2_MSG_DEBUG: if ((r = sshpkt_get_u8(ssh, NULL)) != 0 || (r = sshpkt_get_string(ssh, &msg, NULL)) != 0 || (r = sshpkt_get_string(ssh, NULL, NULL)) != 0) { free(msg); return r; } debug("Remote: %.900s", msg); free(msg); break; case SSH2_MSG_DISCONNECT: if ((r = sshpkt_get_u32(ssh, &reason)) != 0 || (r = sshpkt_get_string(ssh, &msg, NULL)) != 0) return r; /* Ignore normal client exit notifications */ do_log2(ssh->state->server_side && reason == SSH2_DISCONNECT_BY_APPLICATION ? SYSLOG_LEVEL_INFO : SYSLOG_LEVEL_ERROR, "Received disconnect from %s port %d:" "%u: %.400s", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh), reason, msg); free(msg); return SSH_ERR_DISCONNECTED; case SSH2_MSG_UNIMPLEMENTED: if ((r = sshpkt_get_u32(ssh, &seqnr)) != 0) return r; debug("Received SSH2_MSG_UNIMPLEMENTED for %u", seqnr); break; default: return 0; } } else { r = ssh_packet_read_poll1(ssh, typep); switch (*typep) { case SSH_MSG_NONE: return SSH_MSG_NONE; case SSH_MSG_IGNORE: break; case SSH_MSG_DEBUG: if ((r = sshpkt_get_string(ssh, &msg, NULL)) != 0) return r; debug("Remote: %.900s", msg); free(msg); break; case SSH_MSG_DISCONNECT: if ((r = sshpkt_get_string(ssh, &msg, NULL)) != 0) return r; logit("Received disconnect from %s port %d: " "%.400s", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh), msg); free(msg); return SSH_ERR_DISCONNECTED; default: DBG(debug("received packet type %d", *typep)); return 0; } } } } /* * Buffers the given amount of input characters. This is intended to be used * together with packet_read_poll. */ int ssh_packet_process_incoming(struct ssh *ssh, const char *buf, u_int len) { struct session_state *state = ssh->state; int r; if (state->packet_discard) { state->keep_alive_timeouts = 0; /* ?? */ if (len >= state->packet_discard) { if ((r = ssh_packet_stop_discard(ssh)) != 0) return r; } state->packet_discard -= len; return 0; } if ((r = sshbuf_put(ssh->state->input, buf, len)) != 0) return r; return 0; } int ssh_packet_remaining(struct ssh *ssh) { return sshbuf_len(ssh->state->incoming_packet); } /* * Sends a diagnostic message from the server to the client. This message * can be sent at any time (but not while constructing another message). The * message is printed immediately, but only if the client is being executed * in verbose mode. These messages are primarily intended to ease debugging * authentication problems. The length of the formatted message must not * exceed 1024 bytes. This will automatically call ssh_packet_write_wait. */ void ssh_packet_send_debug(struct ssh *ssh, const char *fmt,...) { char buf[1024]; va_list args; int r; if (compat20 && (ssh->compat & SSH_BUG_DEBUG)) return; va_start(args, fmt); vsnprintf(buf, sizeof(buf), fmt, args); va_end(args); if (compat20) { if ((r = sshpkt_start(ssh, SSH2_MSG_DEBUG)) != 0 || (r = sshpkt_put_u8(ssh, 0)) != 0 || /* always display */ (r = sshpkt_put_cstring(ssh, buf)) != 0 || (r = sshpkt_put_cstring(ssh, "")) != 0 || (r = sshpkt_send(ssh)) != 0) fatal("%s: %s", __func__, ssh_err(r)); } else { if ((r = sshpkt_start(ssh, SSH_MSG_DEBUG)) != 0 || (r = sshpkt_put_cstring(ssh, buf)) != 0 || (r = sshpkt_send(ssh)) != 0) fatal("%s: %s", __func__, ssh_err(r)); } if ((r = ssh_packet_write_wait(ssh)) != 0) fatal("%s: %s", __func__, ssh_err(r)); } /* * Pretty-print connection-terminating errors and exit. */ void sshpkt_fatal(struct ssh *ssh, const char *tag, int r) { switch (r) { case SSH_ERR_CONN_CLOSED: logit("Connection closed by %.200s port %d", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh)); cleanup_exit(255); case SSH_ERR_CONN_TIMEOUT: logit("Connection %s %.200s port %d timed out", ssh->state->server_side ? "from" : "to", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh)); cleanup_exit(255); case SSH_ERR_DISCONNECTED: logit("Disconnected from %.200s port %d", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh)); cleanup_exit(255); case SSH_ERR_SYSTEM_ERROR: if (errno == ECONNRESET) { logit("Connection reset by %.200s port %d", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh)); cleanup_exit(255); } /* FALLTHROUGH */ case SSH_ERR_NO_CIPHER_ALG_MATCH: case SSH_ERR_NO_MAC_ALG_MATCH: case SSH_ERR_NO_COMPRESS_ALG_MATCH: case SSH_ERR_NO_KEX_ALG_MATCH: case SSH_ERR_NO_HOSTKEY_ALG_MATCH: if (ssh && ssh->kex && ssh->kex->failed_choice) { + BLACKLIST_NOTIFY(BLACKLIST_AUTH_FAIL); fatal("Unable to negotiate with %.200s port %d: %s. " "Their offer: %s", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh), ssh_err(r), ssh->kex->failed_choice); } /* FALLTHROUGH */ default: fatal("%s%sConnection %s %.200s port %d: %s", tag != NULL ? tag : "", tag != NULL ? ": " : "", ssh->state->server_side ? "from" : "to", ssh_remote_ipaddr(ssh), ssh_remote_port(ssh), ssh_err(r)); } } /* * Logs the error plus constructs and sends a disconnect packet, closes the * connection, and exits. This function never returns. The error message * should not contain a newline. The length of the formatted message must * not exceed 1024 bytes. */ void ssh_packet_disconnect(struct ssh *ssh, const char *fmt,...) { char buf[1024]; va_list args; static int disconnecting = 0; int r; if (disconnecting) /* Guard against recursive invocations. */ fatal("packet_disconnect called recursively."); disconnecting = 1; /* * Format the message. Note that the caller must make sure the * message is of limited size. */ va_start(args, fmt); vsnprintf(buf, sizeof(buf), fmt, args); va_end(args); /* Display the error locally */ logit("Disconnecting: %.100s", buf); /* * Send the disconnect message to the other side, and wait * for it to get sent. */ if ((r = sshpkt_disconnect(ssh, "%s", buf)) != 0) sshpkt_fatal(ssh, __func__, r); if ((r = ssh_packet_write_wait(ssh)) != 0) sshpkt_fatal(ssh, __func__, r); /* Close the connection. */ ssh_packet_close(ssh); cleanup_exit(255); } /* * Checks if there is any buffered output, and tries to write some of * the output. */ int ssh_packet_write_poll(struct ssh *ssh) { struct session_state *state = ssh->state; int len = sshbuf_len(state->output); int r; if (len > 0) { len = write(state->connection_out, sshbuf_ptr(state->output), len); if (len == -1) { if (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK) return 0; return SSH_ERR_SYSTEM_ERROR; } if (len == 0) return SSH_ERR_CONN_CLOSED; if ((r = sshbuf_consume(state->output, len)) != 0) return r; } return 0; } /* * Calls packet_write_poll repeatedly until all pending output data has been * written. */ int ssh_packet_write_wait(struct ssh *ssh) { fd_set *setp; int ret, r, ms_remain = 0; struct timeval start, timeout, *timeoutp = NULL; struct session_state *state = ssh->state; setp = calloc(howmany(state->connection_out + 1, NFDBITS), sizeof(fd_mask)); if (setp == NULL) return SSH_ERR_ALLOC_FAIL; if ((r = ssh_packet_write_poll(ssh)) != 0) { free(setp); return r; } while (ssh_packet_have_data_to_write(ssh)) { memset(setp, 0, howmany(state->connection_out + 1, NFDBITS) * sizeof(fd_mask)); FD_SET(state->connection_out, setp); if (state->packet_timeout_ms > 0) { ms_remain = state->packet_timeout_ms; timeoutp = &timeout; } for (;;) { if (state->packet_timeout_ms != -1) { ms_to_timeval(&timeout, ms_remain); gettimeofday(&start, NULL); } if ((ret = select(state->connection_out + 1, NULL, setp, NULL, timeoutp)) >= 0) break; if (errno != EAGAIN && errno != EINTR && errno != EWOULDBLOCK) break; if (state->packet_timeout_ms == -1) continue; ms_subtract_diff(&start, &ms_remain); if (ms_remain <= 0) { ret = 0; break; } } if (ret == 0) { free(setp); return SSH_ERR_CONN_TIMEOUT; } if ((r = ssh_packet_write_poll(ssh)) != 0) { free(setp); return r; } } free(setp); return 0; } /* Returns true if there is buffered data to write to the connection. */ int ssh_packet_have_data_to_write(struct ssh *ssh) { return sshbuf_len(ssh->state->output) != 0; } /* Returns true if there is not too much data to write to the connection. */ int ssh_packet_not_very_much_data_to_write(struct ssh *ssh) { if (ssh->state->interactive_mode) return sshbuf_len(ssh->state->output) < 16384; else return sshbuf_len(ssh->state->output) < 128 * 1024; } void ssh_packet_set_tos(struct ssh *ssh, int tos) { #ifndef IP_TOS_IS_BROKEN if (!ssh_packet_connection_is_on_socket(ssh)) return; switch (ssh_packet_connection_af(ssh)) { # ifdef IP_TOS case AF_INET: debug3("%s: set IP_TOS 0x%02x", __func__, tos); if (setsockopt(ssh->state->connection_in, IPPROTO_IP, IP_TOS, &tos, sizeof(tos)) < 0) error("setsockopt IP_TOS %d: %.100s:", tos, strerror(errno)); break; # endif /* IP_TOS */ # ifdef IPV6_TCLASS case AF_INET6: debug3("%s: set IPV6_TCLASS 0x%02x", __func__, tos); if (setsockopt(ssh->state->connection_in, IPPROTO_IPV6, IPV6_TCLASS, &tos, sizeof(tos)) < 0) error("setsockopt IPV6_TCLASS %d: %.100s:", tos, strerror(errno)); break; # endif /* IPV6_TCLASS */ } #endif /* IP_TOS_IS_BROKEN */ } /* Informs that the current session is interactive. Sets IP flags for that. */ void ssh_packet_set_interactive(struct ssh *ssh, int interactive, int qos_interactive, int qos_bulk) { struct session_state *state = ssh->state; if (state->set_interactive_called) return; state->set_interactive_called = 1; /* Record that we are in interactive mode. */ state->interactive_mode = interactive; /* Only set socket options if using a socket. */ if (!ssh_packet_connection_is_on_socket(ssh)) return; set_nodelay(state->connection_in); ssh_packet_set_tos(ssh, interactive ? qos_interactive : qos_bulk); } /* Returns true if the current connection is interactive. */ int ssh_packet_is_interactive(struct ssh *ssh) { return ssh->state->interactive_mode; } int ssh_packet_set_maxsize(struct ssh *ssh, u_int s) { struct session_state *state = ssh->state; if (state->set_maxsize_called) { logit("packet_set_maxsize: called twice: old %d new %d", state->max_packet_size, s); return -1; } if (s < 4 * 1024 || s > 1024 * 1024) { logit("packet_set_maxsize: bad size %d", s); return -1; } state->set_maxsize_called = 1; debug("packet_set_maxsize: setting to %d", s); state->max_packet_size = s; return s; } int ssh_packet_inc_alive_timeouts(struct ssh *ssh) { return ++ssh->state->keep_alive_timeouts; } void ssh_packet_set_alive_timeouts(struct ssh *ssh, int ka) { ssh->state->keep_alive_timeouts = ka; } u_int ssh_packet_get_maxsize(struct ssh *ssh) { return ssh->state->max_packet_size; } /* * 9.2. Ignored Data Message * * byte SSH_MSG_IGNORE * string data * * All implementations MUST understand (and ignore) this message at any * time (after receiving the protocol version). No implementation is * required to send them. This message can be used as an additional * protection measure against advanced traffic analysis techniques. */ void ssh_packet_send_ignore(struct ssh *ssh, int nbytes) { u_int32_t rnd = 0; int r, i; if ((r = sshpkt_start(ssh, compat20 ? SSH2_MSG_IGNORE : SSH_MSG_IGNORE)) != 0 || (r = sshpkt_put_u32(ssh, nbytes)) != 0) fatal("%s: %s", __func__, ssh_err(r)); for (i = 0; i < nbytes; i++) { if (i % 4 == 0) rnd = arc4random(); if ((r = sshpkt_put_u8(ssh, (u_char)rnd & 0xff)) != 0) fatal("%s: %s", __func__, ssh_err(r)); rnd >>= 8; } } void ssh_packet_set_rekey_limits(struct ssh *ssh, u_int64_t bytes, time_t seconds) { debug3("rekey after %llu bytes, %d seconds", (unsigned long long)bytes, (int)seconds); ssh->state->rekey_limit = bytes; ssh->state->rekey_interval = seconds; } time_t ssh_packet_get_rekey_timeout(struct ssh *ssh) { time_t seconds; seconds = ssh->state->rekey_time + ssh->state->rekey_interval - monotime(); return (seconds <= 0 ? 1 : seconds); } void ssh_packet_set_server(struct ssh *ssh) { ssh->state->server_side = 1; } void ssh_packet_set_authenticated(struct ssh *ssh) { ssh->state->after_authentication = 1; } void * ssh_packet_get_input(struct ssh *ssh) { return (void *)ssh->state->input; } void * ssh_packet_get_output(struct ssh *ssh) { return (void *)ssh->state->output; } /* Reset after_authentication and reset compression in post-auth privsep */ static int ssh_packet_set_postauth(struct ssh *ssh) { struct sshcomp *comp; int r, mode; debug("%s: called", __func__); /* This was set in net child, but is not visible in user child */ ssh->state->after_authentication = 1; ssh->state->rekeying = 0; for (mode = 0; mode < MODE_MAX; mode++) { if (ssh->state->newkeys[mode] == NULL) continue; comp = &ssh->state->newkeys[mode]->comp; if (comp && comp->enabled && (r = ssh_packet_init_compression(ssh)) != 0) return r; } return 0; } /* Packet state (de-)serialization for privsep */ /* turn kex into a blob for packet state serialization */ static int kex_to_blob(struct sshbuf *m, struct kex *kex) { int r; if ((r = sshbuf_put_string(m, kex->session_id, kex->session_id_len)) != 0 || (r = sshbuf_put_u32(m, kex->we_need)) != 0 || (r = sshbuf_put_u32(m, kex->hostkey_type)) != 0 || (r = sshbuf_put_u32(m, kex->kex_type)) != 0 || (r = sshbuf_put_stringb(m, kex->my)) != 0 || (r = sshbuf_put_stringb(m, kex->peer)) != 0 || (r = sshbuf_put_u32(m, kex->flags)) != 0 || (r = sshbuf_put_cstring(m, kex->client_version_string)) != 0 || (r = sshbuf_put_cstring(m, kex->server_version_string)) != 0) return r; return 0; } /* turn key exchange results into a blob for packet state serialization */ static int newkeys_to_blob(struct sshbuf *m, struct ssh *ssh, int mode) { struct sshbuf *b; struct sshcipher_ctx *cc; struct sshcomp *comp; struct sshenc *enc; struct sshmac *mac; struct newkeys *newkey; int r; if ((newkey = ssh->state->newkeys[mode]) == NULL) return SSH_ERR_INTERNAL_ERROR; enc = &newkey->enc; mac = &newkey->mac; comp = &newkey->comp; cc = (mode == MODE_OUT) ? &ssh->state->send_context : &ssh->state->receive_context; if ((r = cipher_get_keyiv(cc, enc->iv, enc->iv_len)) != 0) return r; if ((b = sshbuf_new()) == NULL) return SSH_ERR_ALLOC_FAIL; /* The cipher struct is constant and shared, you export pointer */ if ((r = sshbuf_put_cstring(b, enc->name)) != 0 || (r = sshbuf_put(b, &enc->cipher, sizeof(enc->cipher))) != 0 || (r = sshbuf_put_u32(b, enc->enabled)) != 0 || (r = sshbuf_put_u32(b, enc->block_size)) != 0 || (r = sshbuf_put_string(b, enc->key, enc->key_len)) != 0 || (r = sshbuf_put_string(b, enc->iv, enc->iv_len)) != 0) goto out; if (cipher_authlen(enc->cipher) == 0) { if ((r = sshbuf_put_cstring(b, mac->name)) != 0 || (r = sshbuf_put_u32(b, mac->enabled)) != 0 || (r = sshbuf_put_string(b, mac->key, mac->key_len)) != 0) goto out; } if ((r = sshbuf_put_u32(b, comp->type)) != 0 || (r = sshbuf_put_u32(b, comp->enabled)) != 0 || (r = sshbuf_put_cstring(b, comp->name)) != 0) goto out; r = sshbuf_put_stringb(m, b); out: sshbuf_free(b); return r; } /* serialize packet state into a blob */ int ssh_packet_get_state(struct ssh *ssh, struct sshbuf *m) { struct session_state *state = ssh->state; u_char *p; size_t slen, rlen; int r, ssh1cipher; if (!compat20) { ssh1cipher = cipher_get_number(state->receive_context.cipher); slen = cipher_get_keyiv_len(&state->send_context); rlen = cipher_get_keyiv_len(&state->receive_context); if ((r = sshbuf_put_u32(m, state->remote_protocol_flags)) != 0 || (r = sshbuf_put_u32(m, ssh1cipher)) != 0 || (r = sshbuf_put_string(m, state->ssh1_key, state->ssh1_keylen)) != 0 || (r = sshbuf_put_u32(m, slen)) != 0 || (r = sshbuf_reserve(m, slen, &p)) != 0 || (r = cipher_get_keyiv(&state->send_context, p, slen)) != 0 || (r = sshbuf_put_u32(m, rlen)) != 0 || (r = sshbuf_reserve(m, rlen, &p)) != 0 || (r = cipher_get_keyiv(&state->receive_context, p, rlen)) != 0) return r; } else { if ((r = kex_to_blob(m, ssh->kex)) != 0 || (r = newkeys_to_blob(m, ssh, MODE_OUT)) != 0 || (r = newkeys_to_blob(m, ssh, MODE_IN)) != 0 || (r = sshbuf_put_u64(m, state->rekey_limit)) != 0 || (r = sshbuf_put_u32(m, state->rekey_interval)) != 0 || (r = sshbuf_put_u32(m, state->p_send.seqnr)) != 0 || (r = sshbuf_put_u64(m, state->p_send.blocks)) != 0 || (r = sshbuf_put_u32(m, state->p_send.packets)) != 0 || (r = sshbuf_put_u64(m, state->p_send.bytes)) != 0 || (r = sshbuf_put_u32(m, state->p_read.seqnr)) != 0 || (r = sshbuf_put_u64(m, state->p_read.blocks)) != 0 || (r = sshbuf_put_u32(m, state->p_read.packets)) != 0 || (r = sshbuf_put_u64(m, state->p_read.bytes)) != 0) return r; } slen = cipher_get_keycontext(&state->send_context, NULL); rlen = cipher_get_keycontext(&state->receive_context, NULL); if ((r = sshbuf_put_u32(m, slen)) != 0 || (r = sshbuf_reserve(m, slen, &p)) != 0) return r; if (cipher_get_keycontext(&state->send_context, p) != (int)slen) return SSH_ERR_INTERNAL_ERROR; if ((r = sshbuf_put_u32(m, rlen)) != 0 || (r = sshbuf_reserve(m, rlen, &p)) != 0) return r; if (cipher_get_keycontext(&state->receive_context, p) != (int)rlen) return SSH_ERR_INTERNAL_ERROR; if ((r = ssh_packet_get_compress_state(m, ssh)) != 0 || (r = sshbuf_put_stringb(m, state->input)) != 0 || (r = sshbuf_put_stringb(m, state->output)) != 0) return r; return 0; } /* restore key exchange results from blob for packet state de-serialization */ static int newkeys_from_blob(struct sshbuf *m, struct ssh *ssh, int mode) { struct sshbuf *b = NULL; struct sshcomp *comp; struct sshenc *enc; struct sshmac *mac; struct newkeys *newkey = NULL; size_t keylen, ivlen, maclen; int r; if ((newkey = calloc(1, sizeof(*newkey))) == NULL) { r = SSH_ERR_ALLOC_FAIL; goto out; } if ((r = sshbuf_froms(m, &b)) != 0) goto out; #ifdef DEBUG_PK sshbuf_dump(b, stderr); #endif enc = &newkey->enc; mac = &newkey->mac; comp = &newkey->comp; if ((r = sshbuf_get_cstring(b, &enc->name, NULL)) != 0 || (r = sshbuf_get(b, &enc->cipher, sizeof(enc->cipher))) != 0 || (r = sshbuf_get_u32(b, (u_int *)&enc->enabled)) != 0 || (r = sshbuf_get_u32(b, &enc->block_size)) != 0 || (r = sshbuf_get_string(b, &enc->key, &keylen)) != 0 || (r = sshbuf_get_string(b, &enc->iv, &ivlen)) != 0) goto out; if (cipher_authlen(enc->cipher) == 0) { if ((r = sshbuf_get_cstring(b, &mac->name, NULL)) != 0) goto out; if ((r = mac_setup(mac, mac->name)) != 0) goto out; if ((r = sshbuf_get_u32(b, (u_int *)&mac->enabled)) != 0 || (r = sshbuf_get_string(b, &mac->key, &maclen)) != 0) goto out; if (maclen > mac->key_len) { r = SSH_ERR_INVALID_FORMAT; goto out; } mac->key_len = maclen; } if ((r = sshbuf_get_u32(b, &comp->type)) != 0 || (r = sshbuf_get_u32(b, (u_int *)&comp->enabled)) != 0 || (r = sshbuf_get_cstring(b, &comp->name, NULL)) != 0) goto out; if (enc->name == NULL || cipher_by_name(enc->name) != enc->cipher) { r = SSH_ERR_INVALID_FORMAT; goto out; } if (sshbuf_len(b) != 0) { r = SSH_ERR_INVALID_FORMAT; goto out; } enc->key_len = keylen; enc->iv_len = ivlen; ssh->kex->newkeys[mode] = newkey; newkey = NULL; r = 0; out: free(newkey); sshbuf_free(b); return r; } /* restore kex from blob for packet state de-serialization */ static int kex_from_blob(struct sshbuf *m, struct kex **kexp) { struct kex *kex; int r; if ((kex = calloc(1, sizeof(struct kex))) == NULL || (kex->my = sshbuf_new()) == NULL || (kex->peer = sshbuf_new()) == NULL) { r = SSH_ERR_ALLOC_FAIL; goto out; } if ((r = sshbuf_get_string(m, &kex->session_id, &kex->session_id_len)) != 0 || (r = sshbuf_get_u32(m, &kex->we_need)) != 0 || (r = sshbuf_get_u32(m, (u_int *)&kex->hostkey_type)) != 0 || (r = sshbuf_get_u32(m, &kex->kex_type)) != 0 || (r = sshbuf_get_stringb(m, kex->my)) != 0 || (r = sshbuf_get_stringb(m, kex->peer)) != 0 || (r = sshbuf_get_u32(m, &kex->flags)) != 0 || (r = sshbuf_get_cstring(m, &kex->client_version_string, NULL)) != 0 || (r = sshbuf_get_cstring(m, &kex->server_version_string, NULL)) != 0) goto out; kex->server = 1; kex->done = 1; r = 0; out: if (r != 0 || kexp == NULL) { if (kex != NULL) { sshbuf_free(kex->my); sshbuf_free(kex->peer); free(kex); } if (kexp != NULL) *kexp = NULL; } else { *kexp = kex; } return r; } /* * Restore packet state from content of blob 'm' (de-serialization). * Note that 'm' will be partially consumed on parsing or any other errors. */ int ssh_packet_set_state(struct ssh *ssh, struct sshbuf *m) { struct session_state *state = ssh->state; const u_char *ssh1key, *ivin, *ivout, *keyin, *keyout, *input, *output; size_t ssh1keylen, rlen, slen, ilen, olen; int r; u_int ssh1cipher = 0; if (!compat20) { if ((r = sshbuf_get_u32(m, &state->remote_protocol_flags)) != 0 || (r = sshbuf_get_u32(m, &ssh1cipher)) != 0 || (r = sshbuf_get_string_direct(m, &ssh1key, &ssh1keylen)) != 0 || (r = sshbuf_get_string_direct(m, &ivout, &slen)) != 0 || (r = sshbuf_get_string_direct(m, &ivin, &rlen)) != 0) return r; if (ssh1cipher > INT_MAX) return SSH_ERR_KEY_UNKNOWN_CIPHER; ssh_packet_set_encryption_key(ssh, ssh1key, ssh1keylen, (int)ssh1cipher); if (cipher_get_keyiv_len(&state->send_context) != (int)slen || cipher_get_keyiv_len(&state->receive_context) != (int)rlen) return SSH_ERR_INVALID_FORMAT; if ((r = cipher_set_keyiv(&state->send_context, ivout)) != 0 || (r = cipher_set_keyiv(&state->receive_context, ivin)) != 0) return r; } else { if ((r = kex_from_blob(m, &ssh->kex)) != 0 || (r = newkeys_from_blob(m, ssh, MODE_OUT)) != 0 || (r = newkeys_from_blob(m, ssh, MODE_IN)) != 0 || (r = sshbuf_get_u64(m, &state->rekey_limit)) != 0 || (r = sshbuf_get_u32(m, &state->rekey_interval)) != 0 || (r = sshbuf_get_u32(m, &state->p_send.seqnr)) != 0 || (r = sshbuf_get_u64(m, &state->p_send.blocks)) != 0 || (r = sshbuf_get_u32(m, &state->p_send.packets)) != 0 || (r = sshbuf_get_u64(m, &state->p_send.bytes)) != 0 || (r = sshbuf_get_u32(m, &state->p_read.seqnr)) != 0 || (r = sshbuf_get_u64(m, &state->p_read.blocks)) != 0 || (r = sshbuf_get_u32(m, &state->p_read.packets)) != 0 || (r = sshbuf_get_u64(m, &state->p_read.bytes)) != 0) return r; /* * We set the time here so that in post-auth privsep slave we * count from the completion of the authentication. */ state->rekey_time = monotime(); /* XXX ssh_set_newkeys overrides p_read.packets? XXX */ if ((r = ssh_set_newkeys(ssh, MODE_IN)) != 0 || (r = ssh_set_newkeys(ssh, MODE_OUT)) != 0) return r; } if ((r = sshbuf_get_string_direct(m, &keyout, &slen)) != 0 || (r = sshbuf_get_string_direct(m, &keyin, &rlen)) != 0) return r; if (cipher_get_keycontext(&state->send_context, NULL) != (int)slen || cipher_get_keycontext(&state->receive_context, NULL) != (int)rlen) return SSH_ERR_INVALID_FORMAT; cipher_set_keycontext(&state->send_context, keyout); cipher_set_keycontext(&state->receive_context, keyin); if ((r = ssh_packet_set_compress_state(ssh, m)) != 0 || (r = ssh_packet_set_postauth(ssh)) != 0) return r; sshbuf_reset(state->input); sshbuf_reset(state->output); if ((r = sshbuf_get_string_direct(m, &input, &ilen)) != 0 || (r = sshbuf_get_string_direct(m, &output, &olen)) != 0 || (r = sshbuf_put(state->input, input, ilen)) != 0 || (r = sshbuf_put(state->output, output, olen)) != 0) return r; if (sshbuf_len(m)) return SSH_ERR_INVALID_FORMAT; debug3("%s: done", __func__); return 0; } /* NEW API */ /* put data to the outgoing packet */ int sshpkt_put(struct ssh *ssh, const void *v, size_t len) { return sshbuf_put(ssh->state->outgoing_packet, v, len); } int sshpkt_putb(struct ssh *ssh, const struct sshbuf *b) { return sshbuf_putb(ssh->state->outgoing_packet, b); } int sshpkt_put_u8(struct ssh *ssh, u_char val) { return sshbuf_put_u8(ssh->state->outgoing_packet, val); } int sshpkt_put_u32(struct ssh *ssh, u_int32_t val) { return sshbuf_put_u32(ssh->state->outgoing_packet, val); } int sshpkt_put_u64(struct ssh *ssh, u_int64_t val) { return sshbuf_put_u64(ssh->state->outgoing_packet, val); } int sshpkt_put_string(struct ssh *ssh, const void *v, size_t len) { return sshbuf_put_string(ssh->state->outgoing_packet, v, len); } int sshpkt_put_cstring(struct ssh *ssh, const void *v) { return sshbuf_put_cstring(ssh->state->outgoing_packet, v); } int sshpkt_put_stringb(struct ssh *ssh, const struct sshbuf *v) { return sshbuf_put_stringb(ssh->state->outgoing_packet, v); } #ifdef WITH_OPENSSL #ifdef OPENSSL_HAS_ECC int sshpkt_put_ec(struct ssh *ssh, const EC_POINT *v, const EC_GROUP *g) { return sshbuf_put_ec(ssh->state->outgoing_packet, v, g); } #endif /* OPENSSL_HAS_ECC */ #ifdef WITH_SSH1 int sshpkt_put_bignum1(struct ssh *ssh, const BIGNUM *v) { return sshbuf_put_bignum1(ssh->state->outgoing_packet, v); } #endif /* WITH_SSH1 */ int sshpkt_put_bignum2(struct ssh *ssh, const BIGNUM *v) { return sshbuf_put_bignum2(ssh->state->outgoing_packet, v); } #endif /* WITH_OPENSSL */ /* fetch data from the incoming packet */ int sshpkt_get(struct ssh *ssh, void *valp, size_t len) { return sshbuf_get(ssh->state->incoming_packet, valp, len); } int sshpkt_get_u8(struct ssh *ssh, u_char *valp) { return sshbuf_get_u8(ssh->state->incoming_packet, valp); } int sshpkt_get_u32(struct ssh *ssh, u_int32_t *valp) { return sshbuf_get_u32(ssh->state->incoming_packet, valp); } int sshpkt_get_u64(struct ssh *ssh, u_int64_t *valp) { return sshbuf_get_u64(ssh->state->incoming_packet, valp); } int sshpkt_get_string(struct ssh *ssh, u_char **valp, size_t *lenp) { return sshbuf_get_string(ssh->state->incoming_packet, valp, lenp); } int sshpkt_get_string_direct(struct ssh *ssh, const u_char **valp, size_t *lenp) { return sshbuf_get_string_direct(ssh->state->incoming_packet, valp, lenp); } int sshpkt_get_cstring(struct ssh *ssh, char **valp, size_t *lenp) { return sshbuf_get_cstring(ssh->state->incoming_packet, valp, lenp); } #ifdef WITH_OPENSSL #ifdef OPENSSL_HAS_ECC int sshpkt_get_ec(struct ssh *ssh, EC_POINT *v, const EC_GROUP *g) { return sshbuf_get_ec(ssh->state->incoming_packet, v, g); } #endif /* OPENSSL_HAS_ECC */ #ifdef WITH_SSH1 int sshpkt_get_bignum1(struct ssh *ssh, BIGNUM *v) { return sshbuf_get_bignum1(ssh->state->incoming_packet, v); } #endif /* WITH_SSH1 */ int sshpkt_get_bignum2(struct ssh *ssh, BIGNUM *v) { return sshbuf_get_bignum2(ssh->state->incoming_packet, v); } #endif /* WITH_OPENSSL */ int sshpkt_get_end(struct ssh *ssh) { if (sshbuf_len(ssh->state->incoming_packet) > 0) return SSH_ERR_UNEXPECTED_TRAILING_DATA; return 0; } const u_char * sshpkt_ptr(struct ssh *ssh, size_t *lenp) { if (lenp != NULL) *lenp = sshbuf_len(ssh->state->incoming_packet); return sshbuf_ptr(ssh->state->incoming_packet); } /* start a new packet */ int sshpkt_start(struct ssh *ssh, u_char type) { u_char buf[9]; int len; DBG(debug("packet_start[%d]", type)); len = compat20 ? 6 : 9; memset(buf, 0, len - 1); buf[len - 1] = type; sshbuf_reset(ssh->state->outgoing_packet); return sshbuf_put(ssh->state->outgoing_packet, buf, len); } /* send it */ int sshpkt_send(struct ssh *ssh) { if (compat20) return ssh_packet_send2(ssh); else return ssh_packet_send1(ssh); } int sshpkt_disconnect(struct ssh *ssh, const char *fmt,...) { char buf[1024]; va_list args; int r; va_start(args, fmt); vsnprintf(buf, sizeof(buf), fmt, args); va_end(args); if (compat20) { if ((r = sshpkt_start(ssh, SSH2_MSG_DISCONNECT)) != 0 || (r = sshpkt_put_u32(ssh, SSH2_DISCONNECT_PROTOCOL_ERROR)) != 0 || (r = sshpkt_put_cstring(ssh, buf)) != 0 || (r = sshpkt_put_cstring(ssh, "")) != 0 || (r = sshpkt_send(ssh)) != 0) return r; } else { if ((r = sshpkt_start(ssh, SSH_MSG_DISCONNECT)) != 0 || (r = sshpkt_put_cstring(ssh, buf)) != 0 || (r = sshpkt_send(ssh)) != 0) return r; } return 0; } /* roundup current message to pad bytes */ int sshpkt_add_padding(struct ssh *ssh, u_char pad) { ssh->state->extra_pad = pad; return 0; } Index: projects/netbsd-tests-update-12/crypto/openssh/servconf.c =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/servconf.c (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/servconf.c (revision 305172) @@ -1,2377 +1,2387 @@ /* $OpenBSD: servconf.c,v 1.285 2016/02/17 05:29:04 djm Exp $ */ /* * Copyright (c) 1995 Tatu Ylonen , Espoo, Finland * All rights reserved * * As far as I am concerned, the code I have written for this software * can be used freely for any purpose. Any derived versions of this * software must be clearly marked as such, and if the derived work is * incompatible with the protocol description in the RFC file, it must be * called by a name other than "ssh" or "Secure Shell". */ #include "includes.h" __RCSID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef HAVE_UTIL_H #include #endif #include "openbsd-compat/sys-queue.h" #include "xmalloc.h" #include "ssh.h" #include "log.h" #include "buffer.h" #include "misc.h" #include "servconf.h" #include "compat.h" #include "pathnames.h" #include "cipher.h" #include "key.h" #include "kex.h" #include "mac.h" #include "match.h" #include "channels.h" #include "groupaccess.h" #include "canohost.h" #include "packet.h" #include "hostfile.h" #include "auth.h" #include "myproposal.h" #include "digest.h" #include "version.h" static void add_listen_addr(ServerOptions *, char *, int); static void add_one_listen_addr(ServerOptions *, char *, int); /* Use of privilege separation or not */ extern int use_privsep; extern Buffer cfg; /* Initializes the server options to their default values. */ void initialize_server_options(ServerOptions *options) { memset(options, 0, sizeof(*options)); /* Portable-specific options */ options->use_pam = -1; /* Standard Options */ options->num_ports = 0; options->ports_from_cmdline = 0; options->queued_listen_addrs = NULL; options->num_queued_listens = 0; options->listen_addrs = NULL; options->address_family = -1; options->num_host_key_files = 0; options->num_host_cert_files = 0; options->host_key_agent = NULL; options->pid_file = NULL; options->server_key_bits = -1; options->login_grace_time = -1; options->key_regeneration_time = -1; options->permit_root_login = PERMIT_NOT_SET; options->ignore_rhosts = -1; options->ignore_user_known_hosts = -1; options->print_motd = -1; options->print_lastlog = -1; options->x11_forwarding = -1; options->x11_display_offset = -1; options->x11_use_localhost = -1; options->permit_tty = -1; options->permit_user_rc = -1; options->xauth_location = NULL; options->strict_modes = -1; options->tcp_keep_alive = -1; options->log_facility = SYSLOG_FACILITY_NOT_SET; options->log_level = SYSLOG_LEVEL_NOT_SET; options->rhosts_rsa_authentication = -1; options->hostbased_authentication = -1; options->hostbased_uses_name_from_packet_only = -1; options->hostbased_key_types = NULL; options->hostkeyalgorithms = NULL; options->rsa_authentication = -1; options->pubkey_authentication = -1; options->pubkey_key_types = NULL; options->kerberos_authentication = -1; options->kerberos_or_local_passwd = -1; options->kerberos_ticket_cleanup = -1; options->kerberos_get_afs_token = -1; options->gss_authentication=-1; options->gss_cleanup_creds = -1; options->gss_strict_acceptor = -1; options->password_authentication = -1; options->kbd_interactive_authentication = -1; options->challenge_response_authentication = -1; options->permit_empty_passwd = -1; options->permit_user_env = -1; options->use_login = -1; options->compression = -1; options->rekey_limit = -1; options->rekey_interval = -1; options->allow_tcp_forwarding = -1; options->allow_streamlocal_forwarding = -1; options->allow_agent_forwarding = -1; options->num_allow_users = 0; options->num_deny_users = 0; options->num_allow_groups = 0; options->num_deny_groups = 0; options->ciphers = NULL; options->macs = NULL; options->kex_algorithms = NULL; options->protocol = SSH_PROTO_UNKNOWN; options->fwd_opts.gateway_ports = -1; options->fwd_opts.streamlocal_bind_mask = (mode_t)-1; options->fwd_opts.streamlocal_bind_unlink = -1; options->num_subsystems = 0; options->max_startups_begin = -1; options->max_startups_rate = -1; options->max_startups = -1; options->max_authtries = -1; options->max_sessions = -1; options->banner = NULL; options->use_dns = -1; options->client_alive_interval = -1; options->client_alive_count_max = -1; options->num_authkeys_files = 0; options->num_accept_env = 0; options->permit_tun = -1; options->num_permitted_opens = -1; options->adm_forced_command = NULL; options->chroot_directory = NULL; options->authorized_keys_command = NULL; options->authorized_keys_command_user = NULL; options->revoked_keys_file = NULL; options->trusted_user_ca_keys = NULL; options->authorized_principals_file = NULL; options->authorized_principals_command = NULL; options->authorized_principals_command_user = NULL; options->ip_qos_interactive = -1; options->ip_qos_bulk = -1; options->version_addendum = NULL; options->fingerprint_hash = -1; + options->use_blacklist = -1; } /* Returns 1 if a string option is unset or set to "none" or 0 otherwise. */ static int option_clear_or_none(const char *o) { return o == NULL || strcasecmp(o, "none") == 0; } static void assemble_algorithms(ServerOptions *o) { if (kex_assemble_names(KEX_SERVER_ENCRYPT, &o->ciphers) != 0 || kex_assemble_names(KEX_SERVER_MAC, &o->macs) != 0 || kex_assemble_names(KEX_SERVER_KEX, &o->kex_algorithms) != 0 || kex_assemble_names(KEX_DEFAULT_PK_ALG, &o->hostkeyalgorithms) != 0 || kex_assemble_names(KEX_DEFAULT_PK_ALG, &o->hostbased_key_types) != 0 || kex_assemble_names(KEX_DEFAULT_PK_ALG, &o->pubkey_key_types) != 0) fatal("kex_assemble_names failed"); } void fill_default_server_options(ServerOptions *options) { int i; /* Portable-specific options */ if (options->use_pam == -1) options->use_pam = 1; /* Standard Options */ if (options->protocol == SSH_PROTO_UNKNOWN) options->protocol = SSH_PROTO_2; #define add_host_key_file(path) \ do { \ if (access((path), O_RDONLY) == 0) \ options->host_key_files \ [options->num_host_key_files++] = (path); \ } while (0) if (options->num_host_key_files == 0) { /* fill default hostkeys for protocols */ if (options->protocol & SSH_PROTO_1) add_host_key_file(_PATH_HOST_KEY_FILE); if (options->protocol & SSH_PROTO_2) { add_host_key_file(_PATH_HOST_RSA_KEY_FILE); add_host_key_file(_PATH_HOST_DSA_KEY_FILE); #ifdef OPENSSL_HAS_ECC add_host_key_file(_PATH_HOST_ECDSA_KEY_FILE); #endif add_host_key_file(_PATH_HOST_ED25519_KEY_FILE); } } #undef add_host_key_file if (options->num_host_key_files == 0) fatal("No host key files found"); /* No certificates by default */ if (options->num_ports == 0) options->ports[options->num_ports++] = SSH_DEFAULT_PORT; if (options->address_family == -1) options->address_family = AF_UNSPEC; if (options->listen_addrs == NULL) add_listen_addr(options, NULL, 0); if (options->pid_file == NULL) options->pid_file = xstrdup(_PATH_SSH_DAEMON_PID_FILE); if (options->server_key_bits == -1) options->server_key_bits = 1024; if (options->login_grace_time == -1) options->login_grace_time = 120; if (options->key_regeneration_time == -1) options->key_regeneration_time = 3600; if (options->permit_root_login == PERMIT_NOT_SET) options->permit_root_login = PERMIT_NO; if (options->ignore_rhosts == -1) options->ignore_rhosts = 1; if (options->ignore_user_known_hosts == -1) options->ignore_user_known_hosts = 0; if (options->print_motd == -1) options->print_motd = 1; if (options->print_lastlog == -1) options->print_lastlog = 1; if (options->x11_forwarding == -1) options->x11_forwarding = 1; if (options->x11_display_offset == -1) options->x11_display_offset = 10; if (options->x11_use_localhost == -1) options->x11_use_localhost = 1; if (options->xauth_location == NULL) options->xauth_location = xstrdup(_PATH_XAUTH); if (options->permit_tty == -1) options->permit_tty = 1; if (options->permit_user_rc == -1) options->permit_user_rc = 1; if (options->strict_modes == -1) options->strict_modes = 1; if (options->tcp_keep_alive == -1) options->tcp_keep_alive = 1; if (options->log_facility == SYSLOG_FACILITY_NOT_SET) options->log_facility = SYSLOG_FACILITY_AUTH; if (options->log_level == SYSLOG_LEVEL_NOT_SET) options->log_level = SYSLOG_LEVEL_INFO; if (options->rhosts_rsa_authentication == -1) options->rhosts_rsa_authentication = 0; if (options->hostbased_authentication == -1) options->hostbased_authentication = 0; if (options->hostbased_uses_name_from_packet_only == -1) options->hostbased_uses_name_from_packet_only = 0; if (options->rsa_authentication == -1) options->rsa_authentication = 1; if (options->pubkey_authentication == -1) options->pubkey_authentication = 1; if (options->kerberos_authentication == -1) options->kerberos_authentication = 0; if (options->kerberos_or_local_passwd == -1) options->kerberos_or_local_passwd = 1; if (options->kerberos_ticket_cleanup == -1) options->kerberos_ticket_cleanup = 1; if (options->kerberos_get_afs_token == -1) options->kerberos_get_afs_token = 0; if (options->gss_authentication == -1) options->gss_authentication = 0; if (options->gss_cleanup_creds == -1) options->gss_cleanup_creds = 1; if (options->gss_strict_acceptor == -1) options->gss_strict_acceptor = 0; if (options->password_authentication == -1) options->password_authentication = 0; if (options->kbd_interactive_authentication == -1) options->kbd_interactive_authentication = 0; if (options->challenge_response_authentication == -1) options->challenge_response_authentication = 1; if (options->permit_empty_passwd == -1) options->permit_empty_passwd = 0; if (options->permit_user_env == -1) options->permit_user_env = 0; if (options->use_login == -1) options->use_login = 0; if (options->compression == -1) options->compression = COMP_DELAYED; if (options->rekey_limit == -1) options->rekey_limit = 0; if (options->rekey_interval == -1) options->rekey_interval = 0; if (options->allow_tcp_forwarding == -1) options->allow_tcp_forwarding = FORWARD_ALLOW; if (options->allow_streamlocal_forwarding == -1) options->allow_streamlocal_forwarding = FORWARD_ALLOW; if (options->allow_agent_forwarding == -1) options->allow_agent_forwarding = 1; if (options->fwd_opts.gateway_ports == -1) options->fwd_opts.gateway_ports = 0; if (options->max_startups == -1) options->max_startups = 100; if (options->max_startups_rate == -1) options->max_startups_rate = 30; /* 30% */ if (options->max_startups_begin == -1) options->max_startups_begin = 10; if (options->max_authtries == -1) options->max_authtries = DEFAULT_AUTH_FAIL_MAX; if (options->max_sessions == -1) options->max_sessions = DEFAULT_SESSIONS_MAX; if (options->use_dns == -1) options->use_dns = 1; if (options->client_alive_interval == -1) options->client_alive_interval = 0; if (options->client_alive_count_max == -1) options->client_alive_count_max = 3; if (options->num_authkeys_files == 0) { options->authorized_keys_files[options->num_authkeys_files++] = xstrdup(_PATH_SSH_USER_PERMITTED_KEYS); options->authorized_keys_files[options->num_authkeys_files++] = xstrdup(_PATH_SSH_USER_PERMITTED_KEYS2); } if (options->permit_tun == -1) options->permit_tun = SSH_TUNMODE_NO; if (options->ip_qos_interactive == -1) options->ip_qos_interactive = IPTOS_LOWDELAY; if (options->ip_qos_bulk == -1) options->ip_qos_bulk = IPTOS_THROUGHPUT; if (options->version_addendum == NULL) options->version_addendum = xstrdup(SSH_VERSION_FREEBSD); if (options->fwd_opts.streamlocal_bind_mask == (mode_t)-1) options->fwd_opts.streamlocal_bind_mask = 0177; if (options->fwd_opts.streamlocal_bind_unlink == -1) options->fwd_opts.streamlocal_bind_unlink = 0; if (options->fingerprint_hash == -1) options->fingerprint_hash = SSH_FP_HASH_DEFAULT; + if (options->use_blacklist == -1) + options->use_blacklist = 0; assemble_algorithms(options); /* Turn privilege separation and sandboxing on by default */ if (use_privsep == -1) use_privsep = PRIVSEP_ON; #define CLEAR_ON_NONE(v) \ do { \ if (option_clear_or_none(v)) { \ free(v); \ v = NULL; \ } \ } while(0) CLEAR_ON_NONE(options->pid_file); CLEAR_ON_NONE(options->xauth_location); CLEAR_ON_NONE(options->banner); CLEAR_ON_NONE(options->trusted_user_ca_keys); CLEAR_ON_NONE(options->revoked_keys_file); CLEAR_ON_NONE(options->authorized_principals_file); CLEAR_ON_NONE(options->adm_forced_command); CLEAR_ON_NONE(options->chroot_directory); for (i = 0; i < options->num_host_key_files; i++) CLEAR_ON_NONE(options->host_key_files[i]); for (i = 0; i < options->num_host_cert_files; i++) CLEAR_ON_NONE(options->host_cert_files[i]); #undef CLEAR_ON_NONE #ifndef HAVE_MMAP if (use_privsep && options->compression == 1) { error("This platform does not support both privilege " "separation and compression"); error("Compression disabled"); options->compression = 0; } #endif } /* Keyword tokens. */ typedef enum { sBadOption, /* == unknown option */ /* Portable-specific options */ sUsePAM, /* Standard Options */ sPort, sHostKeyFile, sServerKeyBits, sLoginGraceTime, sKeyRegenerationTime, sPermitRootLogin, sLogFacility, sLogLevel, sRhostsRSAAuthentication, sRSAAuthentication, sKerberosAuthentication, sKerberosOrLocalPasswd, sKerberosTicketCleanup, sKerberosGetAFSToken, sKerberosTgtPassing, sChallengeResponseAuthentication, sPasswordAuthentication, sKbdInteractiveAuthentication, sListenAddress, sAddressFamily, sPrintMotd, sPrintLastLog, sIgnoreRhosts, sX11Forwarding, sX11DisplayOffset, sX11UseLocalhost, sPermitTTY, sStrictModes, sEmptyPasswd, sTCPKeepAlive, sPermitUserEnvironment, sUseLogin, sAllowTcpForwarding, sCompression, sRekeyLimit, sAllowUsers, sDenyUsers, sAllowGroups, sDenyGroups, sIgnoreUserKnownHosts, sCiphers, sMacs, sProtocol, sPidFile, sGatewayPorts, sPubkeyAuthentication, sPubkeyAcceptedKeyTypes, sXAuthLocation, sSubsystem, sMaxStartups, sMaxAuthTries, sMaxSessions, sBanner, sUseDNS, sHostbasedAuthentication, sHostbasedUsesNameFromPacketOnly, sHostbasedAcceptedKeyTypes, sHostKeyAlgorithms, sClientAliveInterval, sClientAliveCountMax, sAuthorizedKeysFile, sGssAuthentication, sGssCleanupCreds, sGssStrictAcceptor, sAcceptEnv, sPermitTunnel, sMatch, sPermitOpen, sForceCommand, sChrootDirectory, sUsePrivilegeSeparation, sAllowAgentForwarding, sHostCertificate, sRevokedKeys, sTrustedUserCAKeys, sAuthorizedPrincipalsFile, sAuthorizedPrincipalsCommand, sAuthorizedPrincipalsCommandUser, sKexAlgorithms, sIPQoS, sVersionAddendum, sAuthorizedKeysCommand, sAuthorizedKeysCommandUser, sAuthenticationMethods, sHostKeyAgent, sPermitUserRC, sStreamLocalBindMask, sStreamLocalBindUnlink, sAllowStreamLocalForwarding, sFingerprintHash, + sUseBlacklist, sDeprecated, sUnsupported } ServerOpCodes; #define SSHCFG_GLOBAL 0x01 /* allowed in main section of sshd_config */ #define SSHCFG_MATCH 0x02 /* allowed inside a Match section */ #define SSHCFG_ALL (SSHCFG_GLOBAL|SSHCFG_MATCH) /* Textual representation of the tokens. */ static struct { const char *name; ServerOpCodes opcode; u_int flags; } keywords[] = { /* Portable-specific options */ #ifdef USE_PAM { "usepam", sUsePAM, SSHCFG_GLOBAL }, #else { "usepam", sUnsupported, SSHCFG_GLOBAL }, #endif { "pamauthenticationviakbdint", sDeprecated, SSHCFG_GLOBAL }, /* Standard Options */ { "port", sPort, SSHCFG_GLOBAL }, { "hostkey", sHostKeyFile, SSHCFG_GLOBAL }, { "hostdsakey", sHostKeyFile, SSHCFG_GLOBAL }, /* alias */ { "hostkeyagent", sHostKeyAgent, SSHCFG_GLOBAL }, { "pidfile", sPidFile, SSHCFG_GLOBAL }, { "serverkeybits", sServerKeyBits, SSHCFG_GLOBAL }, { "logingracetime", sLoginGraceTime, SSHCFG_GLOBAL }, { "keyregenerationinterval", sKeyRegenerationTime, SSHCFG_GLOBAL }, { "permitrootlogin", sPermitRootLogin, SSHCFG_ALL }, { "syslogfacility", sLogFacility, SSHCFG_GLOBAL }, { "loglevel", sLogLevel, SSHCFG_GLOBAL }, { "rhostsauthentication", sDeprecated, SSHCFG_GLOBAL }, { "rhostsrsaauthentication", sRhostsRSAAuthentication, SSHCFG_ALL }, { "hostbasedauthentication", sHostbasedAuthentication, SSHCFG_ALL }, { "hostbasedusesnamefrompacketonly", sHostbasedUsesNameFromPacketOnly, SSHCFG_ALL }, { "hostbasedacceptedkeytypes", sHostbasedAcceptedKeyTypes, SSHCFG_ALL }, { "hostkeyalgorithms", sHostKeyAlgorithms, SSHCFG_GLOBAL }, { "rsaauthentication", sRSAAuthentication, SSHCFG_ALL }, { "pubkeyauthentication", sPubkeyAuthentication, SSHCFG_ALL }, { "pubkeyacceptedkeytypes", sPubkeyAcceptedKeyTypes, SSHCFG_ALL }, { "dsaauthentication", sPubkeyAuthentication, SSHCFG_GLOBAL }, /* alias */ #ifdef KRB5 { "kerberosauthentication", sKerberosAuthentication, SSHCFG_ALL }, { "kerberosorlocalpasswd", sKerberosOrLocalPasswd, SSHCFG_GLOBAL }, { "kerberosticketcleanup", sKerberosTicketCleanup, SSHCFG_GLOBAL }, #ifdef USE_AFS { "kerberosgetafstoken", sKerberosGetAFSToken, SSHCFG_GLOBAL }, #else { "kerberosgetafstoken", sUnsupported, SSHCFG_GLOBAL }, #endif #else { "kerberosauthentication", sUnsupported, SSHCFG_ALL }, { "kerberosorlocalpasswd", sUnsupported, SSHCFG_GLOBAL }, { "kerberosticketcleanup", sUnsupported, SSHCFG_GLOBAL }, { "kerberosgetafstoken", sUnsupported, SSHCFG_GLOBAL }, #endif { "kerberostgtpassing", sUnsupported, SSHCFG_GLOBAL }, { "afstokenpassing", sUnsupported, SSHCFG_GLOBAL }, #ifdef GSSAPI { "gssapiauthentication", sGssAuthentication, SSHCFG_ALL }, { "gssapicleanupcredentials", sGssCleanupCreds, SSHCFG_GLOBAL }, { "gssapistrictacceptorcheck", sGssStrictAcceptor, SSHCFG_GLOBAL }, #else { "gssapiauthentication", sUnsupported, SSHCFG_ALL }, { "gssapicleanupcredentials", sUnsupported, SSHCFG_GLOBAL }, { "gssapistrictacceptorcheck", sUnsupported, SSHCFG_GLOBAL }, #endif { "passwordauthentication", sPasswordAuthentication, SSHCFG_ALL }, { "kbdinteractiveauthentication", sKbdInteractiveAuthentication, SSHCFG_ALL }, { "challengeresponseauthentication", sChallengeResponseAuthentication, SSHCFG_GLOBAL }, { "skeyauthentication", sChallengeResponseAuthentication, SSHCFG_GLOBAL }, /* alias */ { "checkmail", sDeprecated, SSHCFG_GLOBAL }, { "listenaddress", sListenAddress, SSHCFG_GLOBAL }, { "addressfamily", sAddressFamily, SSHCFG_GLOBAL }, { "printmotd", sPrintMotd, SSHCFG_GLOBAL }, #ifdef DISABLE_LASTLOG { "printlastlog", sUnsupported, SSHCFG_GLOBAL }, #else { "printlastlog", sPrintLastLog, SSHCFG_GLOBAL }, #endif { "ignorerhosts", sIgnoreRhosts, SSHCFG_GLOBAL }, { "ignoreuserknownhosts", sIgnoreUserKnownHosts, SSHCFG_GLOBAL }, { "x11forwarding", sX11Forwarding, SSHCFG_ALL }, { "x11displayoffset", sX11DisplayOffset, SSHCFG_ALL }, { "x11uselocalhost", sX11UseLocalhost, SSHCFG_ALL }, { "xauthlocation", sXAuthLocation, SSHCFG_GLOBAL }, { "strictmodes", sStrictModes, SSHCFG_GLOBAL }, { "permitemptypasswords", sEmptyPasswd, SSHCFG_ALL }, { "permituserenvironment", sPermitUserEnvironment, SSHCFG_GLOBAL }, { "uselogin", sUseLogin, SSHCFG_GLOBAL }, { "compression", sCompression, SSHCFG_GLOBAL }, { "rekeylimit", sRekeyLimit, SSHCFG_ALL }, { "tcpkeepalive", sTCPKeepAlive, SSHCFG_GLOBAL }, { "keepalive", sTCPKeepAlive, SSHCFG_GLOBAL }, /* obsolete alias */ { "allowtcpforwarding", sAllowTcpForwarding, SSHCFG_ALL }, { "allowagentforwarding", sAllowAgentForwarding, SSHCFG_ALL }, { "allowusers", sAllowUsers, SSHCFG_ALL }, { "denyusers", sDenyUsers, SSHCFG_ALL }, { "allowgroups", sAllowGroups, SSHCFG_ALL }, { "denygroups", sDenyGroups, SSHCFG_ALL }, { "ciphers", sCiphers, SSHCFG_GLOBAL }, { "macs", sMacs, SSHCFG_GLOBAL }, { "protocol", sProtocol, SSHCFG_GLOBAL }, { "gatewayports", sGatewayPorts, SSHCFG_ALL }, { "subsystem", sSubsystem, SSHCFG_GLOBAL }, { "maxstartups", sMaxStartups, SSHCFG_GLOBAL }, { "maxauthtries", sMaxAuthTries, SSHCFG_ALL }, { "maxsessions", sMaxSessions, SSHCFG_ALL }, { "banner", sBanner, SSHCFG_ALL }, { "usedns", sUseDNS, SSHCFG_GLOBAL }, { "verifyreversemapping", sDeprecated, SSHCFG_GLOBAL }, { "reversemappingcheck", sDeprecated, SSHCFG_GLOBAL }, { "clientaliveinterval", sClientAliveInterval, SSHCFG_GLOBAL }, { "clientalivecountmax", sClientAliveCountMax, SSHCFG_GLOBAL }, { "authorizedkeysfile", sAuthorizedKeysFile, SSHCFG_ALL }, { "authorizedkeysfile2", sDeprecated, SSHCFG_ALL }, { "useprivilegeseparation", sUsePrivilegeSeparation, SSHCFG_GLOBAL}, { "acceptenv", sAcceptEnv, SSHCFG_ALL }, { "permittunnel", sPermitTunnel, SSHCFG_ALL }, { "permittty", sPermitTTY, SSHCFG_ALL }, { "permituserrc", sPermitUserRC, SSHCFG_ALL }, { "match", sMatch, SSHCFG_ALL }, { "permitopen", sPermitOpen, SSHCFG_ALL }, { "forcecommand", sForceCommand, SSHCFG_ALL }, { "chrootdirectory", sChrootDirectory, SSHCFG_ALL }, { "hostcertificate", sHostCertificate, SSHCFG_GLOBAL }, { "revokedkeys", sRevokedKeys, SSHCFG_ALL }, { "trustedusercakeys", sTrustedUserCAKeys, SSHCFG_ALL }, { "authorizedprincipalsfile", sAuthorizedPrincipalsFile, SSHCFG_ALL }, { "kexalgorithms", sKexAlgorithms, SSHCFG_GLOBAL }, { "ipqos", sIPQoS, SSHCFG_ALL }, { "authorizedkeyscommand", sAuthorizedKeysCommand, SSHCFG_ALL }, { "authorizedkeyscommanduser", sAuthorizedKeysCommandUser, SSHCFG_ALL }, { "authorizedprincipalscommand", sAuthorizedPrincipalsCommand, SSHCFG_ALL }, { "authorizedprincipalscommanduser", sAuthorizedPrincipalsCommandUser, SSHCFG_ALL }, { "versionaddendum", sVersionAddendum, SSHCFG_GLOBAL }, { "authenticationmethods", sAuthenticationMethods, SSHCFG_ALL }, { "streamlocalbindmask", sStreamLocalBindMask, SSHCFG_ALL }, { "streamlocalbindunlink", sStreamLocalBindUnlink, SSHCFG_ALL }, { "allowstreamlocalforwarding", sAllowStreamLocalForwarding, SSHCFG_ALL }, { "fingerprinthash", sFingerprintHash, SSHCFG_GLOBAL }, + { "useblacklist", sUseBlacklist, SSHCFG_GLOBAL }, { "noneenabled", sUnsupported, SSHCFG_ALL }, { "hpndisabled", sDeprecated, SSHCFG_ALL }, { "hpnbuffersize", sDeprecated, SSHCFG_ALL }, { "tcprcvbufpoll", sDeprecated, SSHCFG_ALL }, { NULL, sBadOption, 0 } }; static struct { int val; char *text; } tunmode_desc[] = { { SSH_TUNMODE_NO, "no" }, { SSH_TUNMODE_POINTOPOINT, "point-to-point" }, { SSH_TUNMODE_ETHERNET, "ethernet" }, { SSH_TUNMODE_YES, "yes" }, { -1, NULL } }; /* * Returns the number of the token pointed to by cp or sBadOption. */ static ServerOpCodes parse_token(const char *cp, const char *filename, int linenum, u_int *flags) { u_int i; for (i = 0; keywords[i].name; i++) if (strcasecmp(cp, keywords[i].name) == 0) { *flags = keywords[i].flags; return keywords[i].opcode; } error("%s: line %d: Bad configuration option: %s", filename, linenum, cp); return sBadOption; } char * derelativise_path(const char *path) { char *expanded, *ret, cwd[PATH_MAX]; if (strcasecmp(path, "none") == 0) return xstrdup("none"); expanded = tilde_expand_filename(path, getuid()); if (*expanded == '/') return expanded; if (getcwd(cwd, sizeof(cwd)) == NULL) fatal("%s: getcwd: %s", __func__, strerror(errno)); xasprintf(&ret, "%s/%s", cwd, expanded); free(expanded); return ret; } static void add_listen_addr(ServerOptions *options, char *addr, int port) { u_int i; if (port == 0) for (i = 0; i < options->num_ports; i++) add_one_listen_addr(options, addr, options->ports[i]); else add_one_listen_addr(options, addr, port); } static void add_one_listen_addr(ServerOptions *options, char *addr, int port) { struct addrinfo hints, *ai, *aitop; char strport[NI_MAXSERV]; int gaierr; memset(&hints, 0, sizeof(hints)); hints.ai_family = options->address_family; hints.ai_socktype = SOCK_STREAM; hints.ai_flags = (addr == NULL) ? AI_PASSIVE : 0; snprintf(strport, sizeof strport, "%d", port); if ((gaierr = getaddrinfo(addr, strport, &hints, &aitop)) != 0) fatal("bad addr or host: %s (%s)", addr ? addr : "", ssh_gai_strerror(gaierr)); for (ai = aitop; ai->ai_next; ai = ai->ai_next) ; ai->ai_next = options->listen_addrs; options->listen_addrs = aitop; } /* * Queue a ListenAddress to be processed once we have all of the Ports * and AddressFamily options. */ static void queue_listen_addr(ServerOptions *options, char *addr, int port) { options->queued_listen_addrs = xreallocarray( options->queued_listen_addrs, options->num_queued_listens + 1, sizeof(addr)); options->queued_listen_ports = xreallocarray( options->queued_listen_ports, options->num_queued_listens + 1, sizeof(port)); options->queued_listen_addrs[options->num_queued_listens] = xstrdup(addr); options->queued_listen_ports[options->num_queued_listens] = port; options->num_queued_listens++; } /* * Process queued (text) ListenAddress entries. */ static void process_queued_listen_addrs(ServerOptions *options) { u_int i; if (options->num_ports == 0) options->ports[options->num_ports++] = SSH_DEFAULT_PORT; if (options->address_family == -1) options->address_family = AF_UNSPEC; for (i = 0; i < options->num_queued_listens; i++) { add_listen_addr(options, options->queued_listen_addrs[i], options->queued_listen_ports[i]); free(options->queued_listen_addrs[i]); options->queued_listen_addrs[i] = NULL; } free(options->queued_listen_addrs); options->queued_listen_addrs = NULL; free(options->queued_listen_ports); options->queued_listen_ports = NULL; options->num_queued_listens = 0; } struct connection_info * get_connection_info(int populate, int use_dns) { static struct connection_info ci; if (!populate) return &ci; ci.host = get_canonical_hostname(use_dns); ci.address = get_remote_ipaddr(); ci.laddress = get_local_ipaddr(packet_get_connection_in()); ci.lport = get_local_port(); return &ci; } /* * The strategy for the Match blocks is that the config file is parsed twice. * * The first time is at startup. activep is initialized to 1 and the * directives in the global context are processed and acted on. Hitting a * Match directive unsets activep and the directives inside the block are * checked for syntax only. * * The second time is after a connection has been established but before * authentication. activep is initialized to 2 and global config directives * are ignored since they have already been processed. If the criteria in a * Match block is met, activep is set and the subsequent directives * processed and actioned until EOF or another Match block unsets it. Any * options set are copied into the main server config. * * Potential additions/improvements: * - Add Match support for pre-kex directives, eg Protocol, Ciphers. * * - Add a Tag directive (idea from David Leonard) ala pf, eg: * Match Address 192.168.0.* * Tag trusted * Match Group wheel * Tag trusted * Match Tag trusted * AllowTcpForwarding yes * GatewayPorts clientspecified * [...] * * - Add a PermittedChannelRequests directive * Match Group shell * PermittedChannelRequests session,forwarded-tcpip */ static int match_cfg_line_group(const char *grps, int line, const char *user) { int result = 0; struct passwd *pw; if (user == NULL) goto out; if ((pw = getpwnam(user)) == NULL) { debug("Can't match group at line %d because user %.100s does " "not exist", line, user); } else if (ga_init(pw->pw_name, pw->pw_gid) == 0) { debug("Can't Match group because user %.100s not in any group " "at line %d", user, line); } else if (ga_match_pattern_list(grps) != 1) { debug("user %.100s does not match group list %.100s at line %d", user, grps, line); } else { debug("user %.100s matched group list %.100s at line %d", user, grps, line); result = 1; } out: ga_free(); return result; } /* * All of the attributes on a single Match line are ANDed together, so we need * to check every attribute and set the result to zero if any attribute does * not match. */ static int match_cfg_line(char **condition, int line, struct connection_info *ci) { int result = 1, attributes = 0, port; char *arg, *attrib, *cp = *condition; if (ci == NULL) debug3("checking syntax for 'Match %s'", cp); else debug3("checking match for '%s' user %s host %s addr %s " "laddr %s lport %d", cp, ci->user ? ci->user : "(null)", ci->host ? ci->host : "(null)", ci->address ? ci->address : "(null)", ci->laddress ? ci->laddress : "(null)", ci->lport); while ((attrib = strdelim(&cp)) && *attrib != '\0') { attributes++; if (strcasecmp(attrib, "all") == 0) { if (attributes != 1 || ((arg = strdelim(&cp)) != NULL && *arg != '\0')) { error("'all' cannot be combined with other " "Match attributes"); return -1; } *condition = cp; return 1; } if ((arg = strdelim(&cp)) == NULL || *arg == '\0') { error("Missing Match criteria for %s", attrib); return -1; } if (strcasecmp(attrib, "user") == 0) { if (ci == NULL || ci->user == NULL) { result = 0; continue; } if (match_pattern_list(ci->user, arg, 0) != 1) result = 0; else debug("user %.100s matched 'User %.100s' at " "line %d", ci->user, arg, line); } else if (strcasecmp(attrib, "group") == 0) { if (ci == NULL || ci->user == NULL) { result = 0; continue; } switch (match_cfg_line_group(arg, line, ci->user)) { case -1: return -1; case 0: result = 0; } } else if (strcasecmp(attrib, "host") == 0) { if (ci == NULL || ci->host == NULL) { result = 0; continue; } if (match_hostname(ci->host, arg) != 1) result = 0; else debug("connection from %.100s matched 'Host " "%.100s' at line %d", ci->host, arg, line); } else if (strcasecmp(attrib, "address") == 0) { if (ci == NULL || ci->address == NULL) { result = 0; continue; } switch (addr_match_list(ci->address, arg)) { case 1: debug("connection from %.100s matched 'Address " "%.100s' at line %d", ci->address, arg, line); break; case 0: case -1: result = 0; break; case -2: return -1; } } else if (strcasecmp(attrib, "localaddress") == 0){ if (ci == NULL || ci->laddress == NULL) { result = 0; continue; } switch (addr_match_list(ci->laddress, arg)) { case 1: debug("connection from %.100s matched " "'LocalAddress %.100s' at line %d", ci->laddress, arg, line); break; case 0: case -1: result = 0; break; case -2: return -1; } } else if (strcasecmp(attrib, "localport") == 0) { if ((port = a2port(arg)) == -1) { error("Invalid LocalPort '%s' on Match line", arg); return -1; } if (ci == NULL || ci->lport == 0) { result = 0; continue; } /* TODO support port lists */ if (port == ci->lport) debug("connection from %.100s matched " "'LocalPort %d' at line %d", ci->laddress, port, line); else result = 0; } else { error("Unsupported Match attribute %s", attrib); return -1; } } if (attributes == 0) { error("One or more attributes required for Match"); return -1; } if (ci != NULL) debug3("match %sfound", result ? "" : "not "); *condition = cp; return result; } #define WHITESPACE " \t\r\n" /* Multistate option parsing */ struct multistate { char *key; int value; }; static const struct multistate multistate_addressfamily[] = { { "inet", AF_INET }, { "inet6", AF_INET6 }, { "any", AF_UNSPEC }, { NULL, -1 } }; static const struct multistate multistate_permitrootlogin[] = { { "without-password", PERMIT_NO_PASSWD }, { "prohibit-password", PERMIT_NO_PASSWD }, { "forced-commands-only", PERMIT_FORCED_ONLY }, { "yes", PERMIT_YES }, { "no", PERMIT_NO }, { NULL, -1 } }; static const struct multistate multistate_compression[] = { { "delayed", COMP_DELAYED }, { "yes", COMP_ZLIB }, { "no", COMP_NONE }, { NULL, -1 } }; static const struct multistate multistate_gatewayports[] = { { "clientspecified", 2 }, { "yes", 1 }, { "no", 0 }, { NULL, -1 } }; static const struct multistate multistate_privsep[] = { { "yes", PRIVSEP_NOSANDBOX }, { "sandbox", PRIVSEP_ON }, { "nosandbox", PRIVSEP_NOSANDBOX }, { "no", PRIVSEP_OFF }, { NULL, -1 } }; static const struct multistate multistate_tcpfwd[] = { { "yes", FORWARD_ALLOW }, { "all", FORWARD_ALLOW }, { "no", FORWARD_DENY }, { "remote", FORWARD_REMOTE }, { "local", FORWARD_LOCAL }, { NULL, -1 } }; int process_server_config_line(ServerOptions *options, char *line, const char *filename, int linenum, int *activep, struct connection_info *connectinfo) { char *cp, **charptr, *arg, *p; int cmdline = 0, *intptr, value, value2, n, port; SyslogFacility *log_facility_ptr; LogLevel *log_level_ptr; ServerOpCodes opcode; u_int i, flags = 0; size_t len; long long val64; const struct multistate *multistate_ptr; cp = line; if ((arg = strdelim(&cp)) == NULL) return 0; /* Ignore leading whitespace */ if (*arg == '\0') arg = strdelim(&cp); if (!arg || !*arg || *arg == '#') return 0; intptr = NULL; charptr = NULL; opcode = parse_token(arg, filename, linenum, &flags); if (activep == NULL) { /* We are processing a command line directive */ cmdline = 1; activep = &cmdline; } if (*activep && opcode != sMatch) debug3("%s:%d setting %s %s", filename, linenum, arg, cp); if (*activep == 0 && !(flags & SSHCFG_MATCH)) { if (connectinfo == NULL) { fatal("%s line %d: Directive '%s' is not allowed " "within a Match block", filename, linenum, arg); } else { /* this is a directive we have already processed */ while (arg) arg = strdelim(&cp); return 0; } } switch (opcode) { /* Portable-specific options */ case sUsePAM: intptr = &options->use_pam; goto parse_flag; /* Standard Options */ case sBadOption: return -1; case sPort: /* ignore ports from configfile if cmdline specifies ports */ if (options->ports_from_cmdline) return 0; if (options->num_ports >= MAX_PORTS) fatal("%s line %d: too many ports.", filename, linenum); arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing port number.", filename, linenum); options->ports[options->num_ports++] = a2port(arg); if (options->ports[options->num_ports-1] <= 0) fatal("%s line %d: Badly formatted port number.", filename, linenum); break; case sServerKeyBits: intptr = &options->server_key_bits; parse_int: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing integer value.", filename, linenum); value = atoi(arg); if (*activep && *intptr == -1) *intptr = value; break; case sLoginGraceTime: intptr = &options->login_grace_time; parse_time: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing time value.", filename, linenum); if ((value = convtime(arg)) == -1) fatal("%s line %d: invalid time value.", filename, linenum); if (*activep && *intptr == -1) *intptr = value; break; case sKeyRegenerationTime: intptr = &options->key_regeneration_time; goto parse_time; case sListenAddress: arg = strdelim(&cp); if (arg == NULL || *arg == '\0') fatal("%s line %d: missing address", filename, linenum); /* check for bare IPv6 address: no "[]" and 2 or more ":" */ if (strchr(arg, '[') == NULL && (p = strchr(arg, ':')) != NULL && strchr(p+1, ':') != NULL) { queue_listen_addr(options, arg, 0); break; } p = hpdelim(&arg); if (p == NULL) fatal("%s line %d: bad address:port usage", filename, linenum); p = cleanhostname(p); if (arg == NULL) port = 0; else if ((port = a2port(arg)) <= 0) fatal("%s line %d: bad port number", filename, linenum); queue_listen_addr(options, p, port); break; case sAddressFamily: intptr = &options->address_family; multistate_ptr = multistate_addressfamily; parse_multistate: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing argument.", filename, linenum); value = -1; for (i = 0; multistate_ptr[i].key != NULL; i++) { if (strcasecmp(arg, multistate_ptr[i].key) == 0) { value = multistate_ptr[i].value; break; } } if (value == -1) fatal("%s line %d: unsupported option \"%s\".", filename, linenum, arg); if (*activep && *intptr == -1) *intptr = value; break; case sHostKeyFile: intptr = &options->num_host_key_files; if (*intptr >= MAX_HOSTKEYS) fatal("%s line %d: too many host keys specified (max %d).", filename, linenum, MAX_HOSTKEYS); charptr = &options->host_key_files[*intptr]; parse_filename: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing file name.", filename, linenum); if (*activep && *charptr == NULL) { *charptr = derelativise_path(arg); /* increase optional counter */ if (intptr != NULL) *intptr = *intptr + 1; } break; case sHostKeyAgent: charptr = &options->host_key_agent; arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing socket name.", filename, linenum); if (*activep && *charptr == NULL) *charptr = !strcmp(arg, SSH_AUTHSOCKET_ENV_NAME) ? xstrdup(arg) : derelativise_path(arg); break; case sHostCertificate: intptr = &options->num_host_cert_files; if (*intptr >= MAX_HOSTKEYS) fatal("%s line %d: too many host certificates " "specified (max %d).", filename, linenum, MAX_HOSTCERTS); charptr = &options->host_cert_files[*intptr]; goto parse_filename; break; case sPidFile: charptr = &options->pid_file; goto parse_filename; case sPermitRootLogin: intptr = &options->permit_root_login; multistate_ptr = multistate_permitrootlogin; goto parse_multistate; case sIgnoreRhosts: intptr = &options->ignore_rhosts; parse_flag: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing yes/no argument.", filename, linenum); value = 0; /* silence compiler */ if (strcmp(arg, "yes") == 0) value = 1; else if (strcmp(arg, "no") == 0) value = 0; else fatal("%s line %d: Bad yes/no argument: %s", filename, linenum, arg); if (*activep && *intptr == -1) *intptr = value; break; case sIgnoreUserKnownHosts: intptr = &options->ignore_user_known_hosts; goto parse_flag; case sRhostsRSAAuthentication: intptr = &options->rhosts_rsa_authentication; goto parse_flag; case sHostbasedAuthentication: intptr = &options->hostbased_authentication; goto parse_flag; case sHostbasedUsesNameFromPacketOnly: intptr = &options->hostbased_uses_name_from_packet_only; goto parse_flag; case sHostbasedAcceptedKeyTypes: charptr = &options->hostbased_key_types; parse_keytypes: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing argument.", filename, linenum); if (!sshkey_names_valid2(*arg == '+' ? arg + 1 : arg, 1)) fatal("%s line %d: Bad key types '%s'.", filename, linenum, arg ? arg : ""); if (*activep && *charptr == NULL) *charptr = xstrdup(arg); break; case sHostKeyAlgorithms: charptr = &options->hostkeyalgorithms; goto parse_keytypes; case sRSAAuthentication: intptr = &options->rsa_authentication; goto parse_flag; case sPubkeyAuthentication: intptr = &options->pubkey_authentication; goto parse_flag; case sPubkeyAcceptedKeyTypes: charptr = &options->pubkey_key_types; goto parse_keytypes; case sKerberosAuthentication: intptr = &options->kerberos_authentication; goto parse_flag; case sKerberosOrLocalPasswd: intptr = &options->kerberos_or_local_passwd; goto parse_flag; case sKerberosTicketCleanup: intptr = &options->kerberos_ticket_cleanup; goto parse_flag; case sKerberosGetAFSToken: intptr = &options->kerberos_get_afs_token; goto parse_flag; case sGssAuthentication: intptr = &options->gss_authentication; goto parse_flag; case sGssCleanupCreds: intptr = &options->gss_cleanup_creds; goto parse_flag; case sGssStrictAcceptor: intptr = &options->gss_strict_acceptor; goto parse_flag; case sPasswordAuthentication: intptr = &options->password_authentication; goto parse_flag; case sKbdInteractiveAuthentication: intptr = &options->kbd_interactive_authentication; goto parse_flag; case sChallengeResponseAuthentication: intptr = &options->challenge_response_authentication; goto parse_flag; case sPrintMotd: intptr = &options->print_motd; goto parse_flag; case sPrintLastLog: intptr = &options->print_lastlog; goto parse_flag; case sX11Forwarding: intptr = &options->x11_forwarding; goto parse_flag; case sX11DisplayOffset: intptr = &options->x11_display_offset; goto parse_int; case sX11UseLocalhost: intptr = &options->x11_use_localhost; goto parse_flag; case sXAuthLocation: charptr = &options->xauth_location; goto parse_filename; case sPermitTTY: intptr = &options->permit_tty; goto parse_flag; case sPermitUserRC: intptr = &options->permit_user_rc; goto parse_flag; case sStrictModes: intptr = &options->strict_modes; goto parse_flag; case sTCPKeepAlive: intptr = &options->tcp_keep_alive; goto parse_flag; case sEmptyPasswd: intptr = &options->permit_empty_passwd; goto parse_flag; case sPermitUserEnvironment: intptr = &options->permit_user_env; goto parse_flag; case sUseLogin: intptr = &options->use_login; goto parse_flag; case sCompression: intptr = &options->compression; multistate_ptr = multistate_compression; goto parse_multistate; case sRekeyLimit: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%.200s line %d: Missing argument.", filename, linenum); if (strcmp(arg, "default") == 0) { val64 = 0; } else { if (scan_scaled(arg, &val64) == -1) fatal("%.200s line %d: Bad number '%s': %s", filename, linenum, arg, strerror(errno)); if (val64 != 0 && val64 < 16) fatal("%.200s line %d: RekeyLimit too small", filename, linenum); } if (*activep && options->rekey_limit == -1) options->rekey_limit = val64; if (cp != NULL) { /* optional rekey interval present */ if (strcmp(cp, "none") == 0) { (void)strdelim(&cp); /* discard */ break; } intptr = &options->rekey_interval; goto parse_time; } break; case sGatewayPorts: intptr = &options->fwd_opts.gateway_ports; multistate_ptr = multistate_gatewayports; goto parse_multistate; case sUseDNS: intptr = &options->use_dns; goto parse_flag; case sLogFacility: log_facility_ptr = &options->log_facility; arg = strdelim(&cp); value = log_facility_number(arg); if (value == SYSLOG_FACILITY_NOT_SET) fatal("%.200s line %d: unsupported log facility '%s'", filename, linenum, arg ? arg : ""); if (*log_facility_ptr == -1) *log_facility_ptr = (SyslogFacility) value; break; case sLogLevel: log_level_ptr = &options->log_level; arg = strdelim(&cp); value = log_level_number(arg); if (value == SYSLOG_LEVEL_NOT_SET) fatal("%.200s line %d: unsupported log level '%s'", filename, linenum, arg ? arg : ""); if (*log_level_ptr == -1) *log_level_ptr = (LogLevel) value; break; case sAllowTcpForwarding: intptr = &options->allow_tcp_forwarding; multistate_ptr = multistate_tcpfwd; goto parse_multistate; case sAllowStreamLocalForwarding: intptr = &options->allow_streamlocal_forwarding; multistate_ptr = multistate_tcpfwd; goto parse_multistate; case sAllowAgentForwarding: intptr = &options->allow_agent_forwarding; goto parse_flag; case sUsePrivilegeSeparation: intptr = &use_privsep; multistate_ptr = multistate_privsep; goto parse_multistate; case sAllowUsers: while ((arg = strdelim(&cp)) && *arg != '\0') { if (options->num_allow_users >= MAX_ALLOW_USERS) fatal("%s line %d: too many allow users.", filename, linenum); if (!*activep) continue; options->allow_users[options->num_allow_users++] = xstrdup(arg); } break; case sDenyUsers: while ((arg = strdelim(&cp)) && *arg != '\0') { if (options->num_deny_users >= MAX_DENY_USERS) fatal("%s line %d: too many deny users.", filename, linenum); if (!*activep) continue; options->deny_users[options->num_deny_users++] = xstrdup(arg); } break; case sAllowGroups: while ((arg = strdelim(&cp)) && *arg != '\0') { if (options->num_allow_groups >= MAX_ALLOW_GROUPS) fatal("%s line %d: too many allow groups.", filename, linenum); if (!*activep) continue; options->allow_groups[options->num_allow_groups++] = xstrdup(arg); } break; case sDenyGroups: while ((arg = strdelim(&cp)) && *arg != '\0') { if (options->num_deny_groups >= MAX_DENY_GROUPS) fatal("%s line %d: too many deny groups.", filename, linenum); if (!*activep) continue; options->deny_groups[options->num_deny_groups++] = xstrdup(arg); } break; case sCiphers: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing argument.", filename, linenum); if (!ciphers_valid(*arg == '+' ? arg + 1 : arg)) fatal("%s line %d: Bad SSH2 cipher spec '%s'.", filename, linenum, arg ? arg : ""); if (options->ciphers == NULL) options->ciphers = xstrdup(arg); break; case sMacs: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing argument.", filename, linenum); if (!mac_valid(*arg == '+' ? arg + 1 : arg)) fatal("%s line %d: Bad SSH2 mac spec '%s'.", filename, linenum, arg ? arg : ""); if (options->macs == NULL) options->macs = xstrdup(arg); break; case sKexAlgorithms: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing argument.", filename, linenum); if (!kex_names_valid(*arg == '+' ? arg + 1 : arg)) fatal("%s line %d: Bad SSH2 KexAlgorithms '%s'.", filename, linenum, arg ? arg : ""); if (options->kex_algorithms == NULL) options->kex_algorithms = xstrdup(arg); break; case sProtocol: intptr = &options->protocol; arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing argument.", filename, linenum); value = proto_spec(arg); if (value == SSH_PROTO_UNKNOWN) fatal("%s line %d: Bad protocol spec '%s'.", filename, linenum, arg ? arg : ""); if (*intptr == SSH_PROTO_UNKNOWN) *intptr = value; break; case sSubsystem: if (options->num_subsystems >= MAX_SUBSYSTEMS) { fatal("%s line %d: too many subsystems defined.", filename, linenum); } arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing subsystem name.", filename, linenum); if (!*activep) { arg = strdelim(&cp); break; } for (i = 0; i < options->num_subsystems; i++) if (strcmp(arg, options->subsystem_name[i]) == 0) fatal("%s line %d: Subsystem '%s' already defined.", filename, linenum, arg); options->subsystem_name[options->num_subsystems] = xstrdup(arg); arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing subsystem command.", filename, linenum); options->subsystem_command[options->num_subsystems] = xstrdup(arg); /* Collect arguments (separate to executable) */ p = xstrdup(arg); len = strlen(p) + 1; while ((arg = strdelim(&cp)) != NULL && *arg != '\0') { len += 1 + strlen(arg); p = xreallocarray(p, 1, len); strlcat(p, " ", len); strlcat(p, arg, len); } options->subsystem_args[options->num_subsystems] = p; options->num_subsystems++; break; case sMaxStartups: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing MaxStartups spec.", filename, linenum); if ((n = sscanf(arg, "%d:%d:%d", &options->max_startups_begin, &options->max_startups_rate, &options->max_startups)) == 3) { if (options->max_startups_begin > options->max_startups || options->max_startups_rate > 100 || options->max_startups_rate < 1) fatal("%s line %d: Illegal MaxStartups spec.", filename, linenum); } else if (n != 1) fatal("%s line %d: Illegal MaxStartups spec.", filename, linenum); else options->max_startups = options->max_startups_begin; break; case sMaxAuthTries: intptr = &options->max_authtries; goto parse_int; case sMaxSessions: intptr = &options->max_sessions; goto parse_int; case sBanner: charptr = &options->banner; goto parse_filename; /* * These options can contain %X options expanded at * connect time, so that you can specify paths like: * * AuthorizedKeysFile /etc/ssh_keys/%u */ case sAuthorizedKeysFile: if (*activep && options->num_authkeys_files == 0) { while ((arg = strdelim(&cp)) && *arg != '\0') { if (options->num_authkeys_files >= MAX_AUTHKEYS_FILES) fatal("%s line %d: " "too many authorized keys files.", filename, linenum); options->authorized_keys_files[ options->num_authkeys_files++] = tilde_expand_filename(arg, getuid()); } } return 0; case sAuthorizedPrincipalsFile: charptr = &options->authorized_principals_file; arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing file name.", filename, linenum); if (*activep && *charptr == NULL) { *charptr = tilde_expand_filename(arg, getuid()); /* increase optional counter */ if (intptr != NULL) *intptr = *intptr + 1; } break; case sClientAliveInterval: intptr = &options->client_alive_interval; goto parse_time; case sClientAliveCountMax: intptr = &options->client_alive_count_max; goto parse_int; case sAcceptEnv: while ((arg = strdelim(&cp)) && *arg != '\0') { if (strchr(arg, '=') != NULL) fatal("%s line %d: Invalid environment name.", filename, linenum); if (options->num_accept_env >= MAX_ACCEPT_ENV) fatal("%s line %d: too many allow env.", filename, linenum); if (!*activep) continue; options->accept_env[options->num_accept_env++] = xstrdup(arg); } break; case sPermitTunnel: intptr = &options->permit_tun; arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: Missing yes/point-to-point/" "ethernet/no argument.", filename, linenum); value = -1; for (i = 0; tunmode_desc[i].val != -1; i++) if (strcmp(tunmode_desc[i].text, arg) == 0) { value = tunmode_desc[i].val; break; } if (value == -1) fatal("%s line %d: Bad yes/point-to-point/ethernet/" "no argument: %s", filename, linenum, arg); if (*activep && *intptr == -1) *intptr = value; break; case sMatch: if (cmdline) fatal("Match directive not supported as a command-line " "option"); value = match_cfg_line(&cp, linenum, connectinfo); if (value < 0) fatal("%s line %d: Bad Match condition", filename, linenum); *activep = value; break; case sPermitOpen: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing PermitOpen specification", filename, linenum); n = options->num_permitted_opens; /* modified later */ if (strcmp(arg, "any") == 0) { if (*activep && n == -1) { channel_clear_adm_permitted_opens(); options->num_permitted_opens = 0; } break; } if (strcmp(arg, "none") == 0) { if (*activep && n == -1) { options->num_permitted_opens = 1; channel_disable_adm_local_opens(); } break; } if (*activep && n == -1) channel_clear_adm_permitted_opens(); for (; arg != NULL && *arg != '\0'; arg = strdelim(&cp)) { p = hpdelim(&arg); if (p == NULL) fatal("%s line %d: missing host in PermitOpen", filename, linenum); p = cleanhostname(p); if (arg == NULL || ((port = permitopen_port(arg)) < 0)) fatal("%s line %d: bad port number in " "PermitOpen", filename, linenum); if (*activep && n == -1) options->num_permitted_opens = channel_add_adm_permitted_opens(p, port); } break; case sForceCommand: if (cp == NULL || *cp == '\0') fatal("%.200s line %d: Missing argument.", filename, linenum); len = strspn(cp, WHITESPACE); if (*activep && options->adm_forced_command == NULL) options->adm_forced_command = xstrdup(cp + len); return 0; case sChrootDirectory: charptr = &options->chroot_directory; arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing file name.", filename, linenum); if (*activep && *charptr == NULL) *charptr = xstrdup(arg); break; case sTrustedUserCAKeys: charptr = &options->trusted_user_ca_keys; goto parse_filename; case sRevokedKeys: charptr = &options->revoked_keys_file; goto parse_filename; case sIPQoS: arg = strdelim(&cp); if ((value = parse_ipqos(arg)) == -1) fatal("%s line %d: Bad IPQoS value: %s", filename, linenum, arg); arg = strdelim(&cp); if (arg == NULL) value2 = value; else if ((value2 = parse_ipqos(arg)) == -1) fatal("%s line %d: Bad IPQoS value: %s", filename, linenum, arg); if (*activep) { options->ip_qos_interactive = value; options->ip_qos_bulk = value2; } break; case sVersionAddendum: if (cp == NULL || *cp == '\0') fatal("%.200s line %d: Missing argument.", filename, linenum); len = strspn(cp, WHITESPACE); if (*activep && options->version_addendum == NULL) { if (strcasecmp(cp + len, "none") == 0) options->version_addendum = xstrdup(""); else if (strchr(cp + len, '\r') != NULL) fatal("%.200s line %d: Invalid argument", filename, linenum); else options->version_addendum = xstrdup(cp + len); } return 0; case sAuthorizedKeysCommand: if (cp == NULL) fatal("%.200s line %d: Missing argument.", filename, linenum); len = strspn(cp, WHITESPACE); if (*activep && options->authorized_keys_command == NULL) { if (cp[len] != '/' && strcasecmp(cp + len, "none") != 0) fatal("%.200s line %d: AuthorizedKeysCommand " "must be an absolute path", filename, linenum); options->authorized_keys_command = xstrdup(cp + len); } return 0; case sAuthorizedKeysCommandUser: charptr = &options->authorized_keys_command_user; arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing AuthorizedKeysCommandUser " "argument.", filename, linenum); if (*activep && *charptr == NULL) *charptr = xstrdup(arg); break; case sAuthorizedPrincipalsCommand: if (cp == NULL) fatal("%.200s line %d: Missing argument.", filename, linenum); len = strspn(cp, WHITESPACE); if (*activep && options->authorized_principals_command == NULL) { if (cp[len] != '/' && strcasecmp(cp + len, "none") != 0) fatal("%.200s line %d: " "AuthorizedPrincipalsCommand must be " "an absolute path", filename, linenum); options->authorized_principals_command = xstrdup(cp + len); } return 0; case sAuthorizedPrincipalsCommandUser: charptr = &options->authorized_principals_command_user; arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing " "AuthorizedPrincipalsCommandUser argument.", filename, linenum); if (*activep && *charptr == NULL) *charptr = xstrdup(arg); break; case sAuthenticationMethods: if (options->num_auth_methods == 0) { while ((arg = strdelim(&cp)) && *arg != '\0') { if (options->num_auth_methods >= MAX_AUTH_METHODS) fatal("%s line %d: " "too many authentication methods.", filename, linenum); if (auth2_methods_valid(arg, 0) != 0) fatal("%s line %d: invalid " "authentication method list.", filename, linenum); if (!*activep) continue; options->auth_methods[ options->num_auth_methods++] = xstrdup(arg); } } return 0; case sStreamLocalBindMask: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%s line %d: missing StreamLocalBindMask " "argument.", filename, linenum); /* Parse mode in octal format */ value = strtol(arg, &p, 8); if (arg == p || value < 0 || value > 0777) fatal("%s line %d: Bad mask.", filename, linenum); if (*activep) options->fwd_opts.streamlocal_bind_mask = (mode_t)value; break; case sStreamLocalBindUnlink: intptr = &options->fwd_opts.streamlocal_bind_unlink; goto parse_flag; case sFingerprintHash: arg = strdelim(&cp); if (!arg || *arg == '\0') fatal("%.200s line %d: Missing argument.", filename, linenum); if ((value = ssh_digest_alg_by_name(arg)) == -1) fatal("%.200s line %d: Invalid hash algorithm \"%s\".", filename, linenum, arg); if (*activep) options->fingerprint_hash = value; break; + case sUseBlacklist: + intptr = &options->use_blacklist; + goto parse_flag; + case sDeprecated: logit("%s line %d: Deprecated option %s", filename, linenum, arg); while (arg) arg = strdelim(&cp); break; case sUnsupported: logit("%s line %d: Unsupported option %s", filename, linenum, arg); while (arg) arg = strdelim(&cp); break; default: fatal("%s line %d: Missing handler for opcode %s (%d)", filename, linenum, arg, opcode); } if ((arg = strdelim(&cp)) != NULL && *arg != '\0') fatal("%s line %d: garbage at end of line; \"%.200s\".", filename, linenum, arg); return 0; } /* Reads the server configuration file. */ void load_server_config(const char *filename, Buffer *conf) { char line[4096], *cp; FILE *f; int lineno = 0; debug2("%s: filename %s", __func__, filename); if ((f = fopen(filename, "r")) == NULL) { perror(filename); exit(1); } buffer_clear(conf); while (fgets(line, sizeof(line), f)) { lineno++; if (strlen(line) == sizeof(line) - 1) fatal("%s line %d too long", filename, lineno); /* * Trim out comments and strip whitespace * NB - preserve newlines, they are needed to reproduce * line numbers later for error messages */ if ((cp = strchr(line, '#')) != NULL) memcpy(cp, "\n", 2); cp = line + strspn(line, " \t\r"); buffer_append(conf, cp, strlen(cp)); } buffer_append(conf, "\0", 1); fclose(f); debug2("%s: done config len = %d", __func__, buffer_len(conf)); } void parse_server_match_config(ServerOptions *options, struct connection_info *connectinfo) { ServerOptions mo; initialize_server_options(&mo); parse_server_config(&mo, "reprocess config", &cfg, connectinfo); copy_set_server_options(options, &mo, 0); } int parse_server_match_testspec(struct connection_info *ci, char *spec) { char *p; while ((p = strsep(&spec, ",")) && *p != '\0') { if (strncmp(p, "addr=", 5) == 0) { ci->address = xstrdup(p + 5); } else if (strncmp(p, "host=", 5) == 0) { ci->host = xstrdup(p + 5); } else if (strncmp(p, "user=", 5) == 0) { ci->user = xstrdup(p + 5); } else if (strncmp(p, "laddr=", 6) == 0) { ci->laddress = xstrdup(p + 6); } else if (strncmp(p, "lport=", 6) == 0) { ci->lport = a2port(p + 6); if (ci->lport == -1) { fprintf(stderr, "Invalid port '%s' in test mode" " specification %s\n", p+6, p); return -1; } } else { fprintf(stderr, "Invalid test mode specification %s\n", p); return -1; } } return 0; } /* * returns 1 for a complete spec, 0 for partial spec and -1 for an * empty spec. */ int server_match_spec_complete(struct connection_info *ci) { if (ci->user && ci->host && ci->address) return 1; /* complete */ if (!ci->user && !ci->host && !ci->address) return -1; /* empty */ return 0; /* partial */ } /* * Copy any supported values that are set. * * If the preauth flag is set, we do not bother copying the string or * array values that are not used pre-authentication, because any that we * do use must be explictly sent in mm_getpwnamallow(). */ void copy_set_server_options(ServerOptions *dst, ServerOptions *src, int preauth) { #define M_CP_INTOPT(n) do {\ if (src->n != -1) \ dst->n = src->n; \ } while (0) M_CP_INTOPT(password_authentication); M_CP_INTOPT(gss_authentication); M_CP_INTOPT(rsa_authentication); M_CP_INTOPT(pubkey_authentication); M_CP_INTOPT(kerberos_authentication); M_CP_INTOPT(hostbased_authentication); M_CP_INTOPT(hostbased_uses_name_from_packet_only); M_CP_INTOPT(kbd_interactive_authentication); M_CP_INTOPT(permit_root_login); M_CP_INTOPT(permit_empty_passwd); M_CP_INTOPT(allow_tcp_forwarding); M_CP_INTOPT(allow_streamlocal_forwarding); M_CP_INTOPT(allow_agent_forwarding); M_CP_INTOPT(permit_tun); M_CP_INTOPT(fwd_opts.gateway_ports); M_CP_INTOPT(x11_display_offset); M_CP_INTOPT(x11_forwarding); M_CP_INTOPT(x11_use_localhost); M_CP_INTOPT(permit_tty); M_CP_INTOPT(permit_user_rc); M_CP_INTOPT(max_sessions); M_CP_INTOPT(max_authtries); M_CP_INTOPT(ip_qos_interactive); M_CP_INTOPT(ip_qos_bulk); M_CP_INTOPT(rekey_limit); M_CP_INTOPT(rekey_interval); /* M_CP_STROPT and M_CP_STRARRAYOPT should not appear before here */ #define M_CP_STROPT(n) do {\ if (src->n != NULL && dst->n != src->n) { \ free(dst->n); \ dst->n = src->n; \ } \ } while(0) #define M_CP_STRARRAYOPT(n, num_n) do {\ if (src->num_n != 0) { \ for (dst->num_n = 0; dst->num_n < src->num_n; dst->num_n++) \ dst->n[dst->num_n] = xstrdup(src->n[dst->num_n]); \ } \ } while(0) /* See comment in servconf.h */ COPY_MATCH_STRING_OPTS(); /* Arguments that accept '+...' need to be expanded */ assemble_algorithms(dst); /* * The only things that should be below this point are string options * which are only used after authentication. */ if (preauth) return; /* These options may be "none" to clear a global setting */ M_CP_STROPT(adm_forced_command); if (option_clear_or_none(dst->adm_forced_command)) { free(dst->adm_forced_command); dst->adm_forced_command = NULL; } M_CP_STROPT(chroot_directory); if (option_clear_or_none(dst->chroot_directory)) { free(dst->chroot_directory); dst->chroot_directory = NULL; } } #undef M_CP_INTOPT #undef M_CP_STROPT #undef M_CP_STRARRAYOPT void parse_server_config(ServerOptions *options, const char *filename, Buffer *conf, struct connection_info *connectinfo) { int active, linenum, bad_options = 0; char *cp, *obuf, *cbuf; debug2("%s: config %s len %d", __func__, filename, buffer_len(conf)); obuf = cbuf = xstrdup(buffer_ptr(conf)); active = connectinfo ? 0 : 1; linenum = 1; while ((cp = strsep(&cbuf, "\n")) != NULL) { if (process_server_config_line(options, cp, filename, linenum++, &active, connectinfo) != 0) bad_options++; } free(obuf); if (bad_options > 0) fatal("%s: terminating, %d bad configuration options", filename, bad_options); process_queued_listen_addrs(options); } static const char * fmt_multistate_int(int val, const struct multistate *m) { u_int i; for (i = 0; m[i].key != NULL; i++) { if (m[i].value == val) return m[i].key; } return "UNKNOWN"; } static const char * fmt_intarg(ServerOpCodes code, int val) { if (val == -1) return "unset"; switch (code) { case sAddressFamily: return fmt_multistate_int(val, multistate_addressfamily); case sPermitRootLogin: return fmt_multistate_int(val, multistate_permitrootlogin); case sGatewayPorts: return fmt_multistate_int(val, multistate_gatewayports); case sCompression: return fmt_multistate_int(val, multistate_compression); case sUsePrivilegeSeparation: return fmt_multistate_int(val, multistate_privsep); case sAllowTcpForwarding: return fmt_multistate_int(val, multistate_tcpfwd); case sAllowStreamLocalForwarding: return fmt_multistate_int(val, multistate_tcpfwd); case sFingerprintHash: return ssh_digest_alg_name(val); case sProtocol: switch (val) { case SSH_PROTO_1: return "1"; case SSH_PROTO_2: return "2"; case (SSH_PROTO_1|SSH_PROTO_2): return "2,1"; default: return "UNKNOWN"; } default: switch (val) { case 0: return "no"; case 1: return "yes"; default: return "UNKNOWN"; } } } static const char * lookup_opcode_name(ServerOpCodes code) { u_int i; for (i = 0; keywords[i].name != NULL; i++) if (keywords[i].opcode == code) return(keywords[i].name); return "UNKNOWN"; } static void dump_cfg_int(ServerOpCodes code, int val) { printf("%s %d\n", lookup_opcode_name(code), val); } static void dump_cfg_oct(ServerOpCodes code, int val) { printf("%s 0%o\n", lookup_opcode_name(code), val); } static void dump_cfg_fmtint(ServerOpCodes code, int val) { printf("%s %s\n", lookup_opcode_name(code), fmt_intarg(code, val)); } static void dump_cfg_string(ServerOpCodes code, const char *val) { if (val == NULL) return; printf("%s %s\n", lookup_opcode_name(code), val == NULL ? "none" : val); } static void dump_cfg_strarray(ServerOpCodes code, u_int count, char **vals) { u_int i; for (i = 0; i < count; i++) printf("%s %s\n", lookup_opcode_name(code), vals[i]); } static void dump_cfg_strarray_oneline(ServerOpCodes code, u_int count, char **vals) { u_int i; if (count <= 0) return; printf("%s", lookup_opcode_name(code)); for (i = 0; i < count; i++) printf(" %s", vals[i]); printf("\n"); } void dump_config(ServerOptions *o) { u_int i; int ret; struct addrinfo *ai; char addr[NI_MAXHOST], port[NI_MAXSERV], *s = NULL; char *laddr1 = xstrdup(""), *laddr2 = NULL; /* these are usually at the top of the config */ for (i = 0; i < o->num_ports; i++) printf("port %d\n", o->ports[i]); dump_cfg_fmtint(sProtocol, o->protocol); dump_cfg_fmtint(sAddressFamily, o->address_family); /* * ListenAddress must be after Port. add_one_listen_addr pushes * addresses onto a stack, so to maintain ordering we need to * print these in reverse order. */ for (ai = o->listen_addrs; ai; ai = ai->ai_next) { if ((ret = getnameinfo(ai->ai_addr, ai->ai_addrlen, addr, sizeof(addr), port, sizeof(port), NI_NUMERICHOST|NI_NUMERICSERV)) != 0) { error("getnameinfo failed: %.100s", (ret != EAI_SYSTEM) ? gai_strerror(ret) : strerror(errno)); } else { laddr2 = laddr1; if (ai->ai_family == AF_INET6) xasprintf(&laddr1, "listenaddress [%s]:%s\n%s", addr, port, laddr2); else xasprintf(&laddr1, "listenaddress %s:%s\n%s", addr, port, laddr2); free(laddr2); } } printf("%s", laddr1); free(laddr1); /* integer arguments */ #ifdef USE_PAM dump_cfg_fmtint(sUsePAM, o->use_pam); #endif dump_cfg_int(sServerKeyBits, o->server_key_bits); dump_cfg_int(sLoginGraceTime, o->login_grace_time); dump_cfg_int(sKeyRegenerationTime, o->key_regeneration_time); dump_cfg_int(sX11DisplayOffset, o->x11_display_offset); dump_cfg_int(sMaxAuthTries, o->max_authtries); dump_cfg_int(sMaxSessions, o->max_sessions); dump_cfg_int(sClientAliveInterval, o->client_alive_interval); dump_cfg_int(sClientAliveCountMax, o->client_alive_count_max); dump_cfg_oct(sStreamLocalBindMask, o->fwd_opts.streamlocal_bind_mask); /* formatted integer arguments */ dump_cfg_fmtint(sPermitRootLogin, o->permit_root_login); dump_cfg_fmtint(sIgnoreRhosts, o->ignore_rhosts); dump_cfg_fmtint(sIgnoreUserKnownHosts, o->ignore_user_known_hosts); dump_cfg_fmtint(sRhostsRSAAuthentication, o->rhosts_rsa_authentication); dump_cfg_fmtint(sHostbasedAuthentication, o->hostbased_authentication); dump_cfg_fmtint(sHostbasedUsesNameFromPacketOnly, o->hostbased_uses_name_from_packet_only); dump_cfg_fmtint(sRSAAuthentication, o->rsa_authentication); dump_cfg_fmtint(sPubkeyAuthentication, o->pubkey_authentication); #ifdef KRB5 dump_cfg_fmtint(sKerberosAuthentication, o->kerberos_authentication); dump_cfg_fmtint(sKerberosOrLocalPasswd, o->kerberos_or_local_passwd); dump_cfg_fmtint(sKerberosTicketCleanup, o->kerberos_ticket_cleanup); # ifdef USE_AFS dump_cfg_fmtint(sKerberosGetAFSToken, o->kerberos_get_afs_token); # endif #endif #ifdef GSSAPI dump_cfg_fmtint(sGssAuthentication, o->gss_authentication); dump_cfg_fmtint(sGssCleanupCreds, o->gss_cleanup_creds); #endif dump_cfg_fmtint(sPasswordAuthentication, o->password_authentication); dump_cfg_fmtint(sKbdInteractiveAuthentication, o->kbd_interactive_authentication); dump_cfg_fmtint(sChallengeResponseAuthentication, o->challenge_response_authentication); dump_cfg_fmtint(sPrintMotd, o->print_motd); #ifndef DISABLE_LASTLOG dump_cfg_fmtint(sPrintLastLog, o->print_lastlog); #endif dump_cfg_fmtint(sX11Forwarding, o->x11_forwarding); dump_cfg_fmtint(sX11UseLocalhost, o->x11_use_localhost); dump_cfg_fmtint(sPermitTTY, o->permit_tty); dump_cfg_fmtint(sPermitUserRC, o->permit_user_rc); dump_cfg_fmtint(sStrictModes, o->strict_modes); dump_cfg_fmtint(sTCPKeepAlive, o->tcp_keep_alive); dump_cfg_fmtint(sEmptyPasswd, o->permit_empty_passwd); dump_cfg_fmtint(sPermitUserEnvironment, o->permit_user_env); dump_cfg_fmtint(sUseLogin, o->use_login); dump_cfg_fmtint(sCompression, o->compression); dump_cfg_fmtint(sGatewayPorts, o->fwd_opts.gateway_ports); dump_cfg_fmtint(sUseDNS, o->use_dns); dump_cfg_fmtint(sAllowTcpForwarding, o->allow_tcp_forwarding); dump_cfg_fmtint(sAllowAgentForwarding, o->allow_agent_forwarding); dump_cfg_fmtint(sAllowStreamLocalForwarding, o->allow_streamlocal_forwarding); dump_cfg_fmtint(sUsePrivilegeSeparation, use_privsep); dump_cfg_fmtint(sFingerprintHash, o->fingerprint_hash); + dump_cfg_fmtint(sUseBlacklist, o->use_blacklist); /* string arguments */ dump_cfg_string(sPidFile, o->pid_file); dump_cfg_string(sXAuthLocation, o->xauth_location); dump_cfg_string(sCiphers, o->ciphers ? o->ciphers : KEX_SERVER_ENCRYPT); dump_cfg_string(sMacs, o->macs ? o->macs : KEX_SERVER_MAC); dump_cfg_string(sBanner, o->banner); dump_cfg_string(sForceCommand, o->adm_forced_command); dump_cfg_string(sChrootDirectory, o->chroot_directory); dump_cfg_string(sTrustedUserCAKeys, o->trusted_user_ca_keys); dump_cfg_string(sRevokedKeys, o->revoked_keys_file); dump_cfg_string(sAuthorizedPrincipalsFile, o->authorized_principals_file); dump_cfg_string(sVersionAddendum, *o->version_addendum == '\0' ? "none" : o->version_addendum); dump_cfg_string(sAuthorizedKeysCommand, o->authorized_keys_command); dump_cfg_string(sAuthorizedKeysCommandUser, o->authorized_keys_command_user); dump_cfg_string(sAuthorizedPrincipalsCommand, o->authorized_principals_command); dump_cfg_string(sAuthorizedPrincipalsCommandUser, o->authorized_principals_command_user); dump_cfg_string(sHostKeyAgent, o->host_key_agent); dump_cfg_string(sKexAlgorithms, o->kex_algorithms ? o->kex_algorithms : KEX_SERVER_KEX); dump_cfg_string(sHostbasedAcceptedKeyTypes, o->hostbased_key_types ? o->hostbased_key_types : KEX_DEFAULT_PK_ALG); dump_cfg_string(sHostKeyAlgorithms, o->hostkeyalgorithms ? o->hostkeyalgorithms : KEX_DEFAULT_PK_ALG); dump_cfg_string(sPubkeyAcceptedKeyTypes, o->pubkey_key_types ? o->pubkey_key_types : KEX_DEFAULT_PK_ALG); /* string arguments requiring a lookup */ dump_cfg_string(sLogLevel, log_level_name(o->log_level)); dump_cfg_string(sLogFacility, log_facility_name(o->log_facility)); /* string array arguments */ dump_cfg_strarray_oneline(sAuthorizedKeysFile, o->num_authkeys_files, o->authorized_keys_files); dump_cfg_strarray(sHostKeyFile, o->num_host_key_files, o->host_key_files); dump_cfg_strarray(sHostCertificate, o->num_host_cert_files, o->host_cert_files); dump_cfg_strarray(sAllowUsers, o->num_allow_users, o->allow_users); dump_cfg_strarray(sDenyUsers, o->num_deny_users, o->deny_users); dump_cfg_strarray(sAllowGroups, o->num_allow_groups, o->allow_groups); dump_cfg_strarray(sDenyGroups, o->num_deny_groups, o->deny_groups); dump_cfg_strarray(sAcceptEnv, o->num_accept_env, o->accept_env); dump_cfg_strarray_oneline(sAuthenticationMethods, o->num_auth_methods, o->auth_methods); /* other arguments */ for (i = 0; i < o->num_subsystems; i++) printf("subsystem %s %s\n", o->subsystem_name[i], o->subsystem_args[i]); printf("maxstartups %d:%d:%d\n", o->max_startups_begin, o->max_startups_rate, o->max_startups); for (i = 0; tunmode_desc[i].val != -1; i++) if (tunmode_desc[i].val == o->permit_tun) { s = tunmode_desc[i].text; break; } dump_cfg_string(sPermitTunnel, s); printf("ipqos %s ", iptos2str(o->ip_qos_interactive)); printf("%s\n", iptos2str(o->ip_qos_bulk)); printf("rekeylimit %llu %d\n", (unsigned long long)o->rekey_limit, o->rekey_interval); channel_print_adm_permitted_opens(); } Index: projects/netbsd-tests-update-12/crypto/openssh/servconf.h =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/servconf.h (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/servconf.h (revision 305172) @@ -1,254 +1,255 @@ /* $OpenBSD: servconf.h,v 1.120 2015/07/10 06:21:53 markus Exp $ */ /* * Author: Tatu Ylonen * Copyright (c) 1995 Tatu Ylonen , Espoo, Finland * All rights reserved * Definitions for server configuration data and for the functions reading it. * * As far as I am concerned, the code I have written for this software * can be used freely for any purpose. Any derived versions of this * software must be clearly marked as such, and if the derived work is * incompatible with the protocol description in the RFC file, it must be * called by a name other than "ssh" or "Secure Shell". */ #ifndef SERVCONF_H #define SERVCONF_H #define MAX_PORTS 256 /* Max # ports. */ #define MAX_ALLOW_USERS 256 /* Max # users on allow list. */ #define MAX_DENY_USERS 256 /* Max # users on deny list. */ #define MAX_ALLOW_GROUPS 256 /* Max # groups on allow list. */ #define MAX_DENY_GROUPS 256 /* Max # groups on deny list. */ #define MAX_SUBSYSTEMS 256 /* Max # subsystems. */ #define MAX_HOSTKEYS 256 /* Max # hostkeys. */ #define MAX_HOSTCERTS 256 /* Max # host certificates. */ #define MAX_ACCEPT_ENV 256 /* Max # of env vars. */ #define MAX_MATCH_GROUPS 256 /* Max # of groups for Match. */ #define MAX_AUTHKEYS_FILES 256 /* Max # of authorized_keys files. */ #define MAX_AUTH_METHODS 256 /* Max # of AuthenticationMethods. */ /* permit_root_login */ #define PERMIT_NOT_SET -1 #define PERMIT_NO 0 #define PERMIT_FORCED_ONLY 1 #define PERMIT_NO_PASSWD 2 #define PERMIT_YES 3 /* use_privsep */ #define PRIVSEP_OFF 0 #define PRIVSEP_ON 1 #define PRIVSEP_NOSANDBOX 2 /* AllowTCPForwarding */ #define FORWARD_DENY 0 #define FORWARD_REMOTE (1) #define FORWARD_LOCAL (1<<1) #define FORWARD_ALLOW (FORWARD_REMOTE|FORWARD_LOCAL) #define DEFAULT_AUTH_FAIL_MAX 6 /* Default for MaxAuthTries */ #define DEFAULT_SESSIONS_MAX 10 /* Default for MaxSessions */ /* Magic name for internal sftp-server */ #define INTERNAL_SFTP_NAME "internal-sftp" typedef struct { u_int num_ports; u_int ports_from_cmdline; int ports[MAX_PORTS]; /* Port number to listen on. */ u_int num_queued_listens; char **queued_listen_addrs; int *queued_listen_ports; struct addrinfo *listen_addrs; /* Addresses on which the server listens. */ int address_family; /* Address family used by the server. */ char *host_key_files[MAX_HOSTKEYS]; /* Files containing host keys. */ int num_host_key_files; /* Number of files for host keys. */ char *host_cert_files[MAX_HOSTCERTS]; /* Files containing host certs. */ int num_host_cert_files; /* Number of files for host certs. */ char *host_key_agent; /* ssh-agent socket for host keys. */ char *pid_file; /* Where to put our pid */ int server_key_bits;/* Size of the server key. */ int login_grace_time; /* Disconnect if no auth in this time * (sec). */ int key_regeneration_time; /* Server key lifetime (seconds). */ int permit_root_login; /* PERMIT_*, see above */ int ignore_rhosts; /* Ignore .rhosts and .shosts. */ int ignore_user_known_hosts; /* Ignore ~/.ssh/known_hosts * for RhostsRsaAuth */ int print_motd; /* If true, print /etc/motd. */ int print_lastlog; /* If true, print lastlog */ int x11_forwarding; /* If true, permit inet (spoofing) X11 fwd. */ int x11_display_offset; /* What DISPLAY number to start * searching at */ int x11_use_localhost; /* If true, use localhost for fake X11 server. */ char *xauth_location; /* Location of xauth program */ int permit_tty; /* If false, deny pty allocation */ int permit_user_rc; /* If false, deny ~/.ssh/rc execution */ int strict_modes; /* If true, require string home dir modes. */ int tcp_keep_alive; /* If true, set SO_KEEPALIVE. */ int ip_qos_interactive; /* IP ToS/DSCP/class for interactive */ int ip_qos_bulk; /* IP ToS/DSCP/class for bulk traffic */ char *ciphers; /* Supported SSH2 ciphers. */ char *macs; /* Supported SSH2 macs. */ char *kex_algorithms; /* SSH2 kex methods in order of preference. */ int protocol; /* Supported protocol versions. */ struct ForwardOptions fwd_opts; /* forwarding options */ SyslogFacility log_facility; /* Facility for system logging. */ LogLevel log_level; /* Level for system logging. */ int rhosts_rsa_authentication; /* If true, permit rhosts RSA * authentication. */ int hostbased_authentication; /* If true, permit ssh2 hostbased auth */ int hostbased_uses_name_from_packet_only; /* experimental */ char *hostbased_key_types; /* Key types allowed for hostbased */ char *hostkeyalgorithms; /* SSH2 server key types */ int rsa_authentication; /* If true, permit RSA authentication. */ int pubkey_authentication; /* If true, permit ssh2 pubkey authentication. */ char *pubkey_key_types; /* Key types allowed for public key */ int kerberos_authentication; /* If true, permit Kerberos * authentication. */ int kerberos_or_local_passwd; /* If true, permit kerberos * and any other password * authentication mechanism, * such as SecurID or * /etc/passwd */ int kerberos_ticket_cleanup; /* If true, destroy ticket * file on logout. */ int kerberos_get_afs_token; /* If true, try to get AFS token if * authenticated with Kerberos. */ int gss_authentication; /* If true, permit GSSAPI authentication */ int gss_cleanup_creds; /* If true, destroy cred cache on logout */ int gss_strict_acceptor; /* If true, restrict the GSSAPI acceptor name */ int password_authentication; /* If true, permit password * authentication. */ int kbd_interactive_authentication; /* If true, permit */ int challenge_response_authentication; int permit_empty_passwd; /* If false, do not permit empty * passwords. */ int permit_user_env; /* If true, read ~/.ssh/environment */ int use_login; /* If true, login(1) is used */ int compression; /* If true, compression is allowed */ int allow_tcp_forwarding; /* One of FORWARD_* */ int allow_streamlocal_forwarding; /* One of FORWARD_* */ int allow_agent_forwarding; u_int num_allow_users; char *allow_users[MAX_ALLOW_USERS]; u_int num_deny_users; char *deny_users[MAX_DENY_USERS]; u_int num_allow_groups; char *allow_groups[MAX_ALLOW_GROUPS]; u_int num_deny_groups; char *deny_groups[MAX_DENY_GROUPS]; u_int num_subsystems; char *subsystem_name[MAX_SUBSYSTEMS]; char *subsystem_command[MAX_SUBSYSTEMS]; char *subsystem_args[MAX_SUBSYSTEMS]; u_int num_accept_env; char *accept_env[MAX_ACCEPT_ENV]; int max_startups_begin; int max_startups_rate; int max_startups; int max_authtries; int max_sessions; char *banner; /* SSH-2 banner message */ int use_dns; int client_alive_interval; /* * poke the client this often to * see if it's still there */ int client_alive_count_max; /* * If the client is unresponsive * for this many intervals above, * disconnect the session */ u_int num_authkeys_files; /* Files containing public keys */ char *authorized_keys_files[MAX_AUTHKEYS_FILES]; char *adm_forced_command; int use_pam; /* Enable auth via PAM */ int permit_tun; int num_permitted_opens; char *chroot_directory; char *revoked_keys_file; char *trusted_user_ca_keys; char *authorized_keys_command; char *authorized_keys_command_user; char *authorized_principals_file; char *authorized_principals_command; char *authorized_principals_command_user; int64_t rekey_limit; int rekey_interval; char *version_addendum; /* Appended to SSH banner */ u_int num_auth_methods; char *auth_methods[MAX_AUTH_METHODS]; int fingerprint_hash; + int use_blacklist; } ServerOptions; /* Information about the incoming connection as used by Match */ struct connection_info { const char *user; const char *host; /* possibly resolved hostname */ const char *address; /* remote address */ const char *laddress; /* local address */ int lport; /* local port */ }; /* * These are string config options that must be copied between the * Match sub-config and the main config, and must be sent from the * privsep slave to the privsep master. We use a macro to ensure all * the options are copied and the copies are done in the correct order. * * NB. an option must appear in servconf.c:copy_set_server_options() or * COPY_MATCH_STRING_OPTS here but never both. */ #define COPY_MATCH_STRING_OPTS() do { \ M_CP_STROPT(banner); \ M_CP_STROPT(trusted_user_ca_keys); \ M_CP_STROPT(revoked_keys_file); \ M_CP_STROPT(authorized_keys_command); \ M_CP_STROPT(authorized_keys_command_user); \ M_CP_STROPT(authorized_principals_file); \ M_CP_STROPT(authorized_principals_command); \ M_CP_STROPT(authorized_principals_command_user); \ M_CP_STROPT(hostbased_key_types); \ M_CP_STROPT(pubkey_key_types); \ M_CP_STRARRAYOPT(authorized_keys_files, num_authkeys_files); \ M_CP_STRARRAYOPT(allow_users, num_allow_users); \ M_CP_STRARRAYOPT(deny_users, num_deny_users); \ M_CP_STRARRAYOPT(allow_groups, num_allow_groups); \ M_CP_STRARRAYOPT(deny_groups, num_deny_groups); \ M_CP_STRARRAYOPT(accept_env, num_accept_env); \ M_CP_STRARRAYOPT(auth_methods, num_auth_methods); \ } while (0) struct connection_info *get_connection_info(int, int); void initialize_server_options(ServerOptions *); void fill_default_server_options(ServerOptions *); int process_server_config_line(ServerOptions *, char *, const char *, int, int *, struct connection_info *); void load_server_config(const char *, Buffer *); void parse_server_config(ServerOptions *, const char *, Buffer *, struct connection_info *); void parse_server_match_config(ServerOptions *, struct connection_info *); int parse_server_match_testspec(struct connection_info *, char *); int server_match_spec_complete(struct connection_info *); void copy_set_server_options(ServerOptions *, ServerOptions *, int); void dump_config(ServerOptions *); char *derelativise_path(const char *); #endif /* SERVCONF_H */ Index: projects/netbsd-tests-update-12/crypto/openssh/sshd.c =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/sshd.c (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/sshd.c (revision 305172) @@ -1,2700 +1,2705 @@ /* $OpenBSD: sshd.c,v 1.465 2016/02/15 09:47:49 dtucker Exp $ */ /* * Author: Tatu Ylonen * Copyright (c) 1995 Tatu Ylonen , Espoo, Finland * All rights reserved * This program is the ssh daemon. It listens for connections from clients, * and performs authentication, executes use commands or shell, and forwards * information to/from the application to the user client over an encrypted * connection. This can also handle forwarding of X11, TCP/IP, and * authentication agent connections. * * As far as I am concerned, the code I have written for this software * can be used freely for any purpose. Any derived versions of this * software must be clearly marked as such, and if the derived work is * incompatible with the protocol description in the RFC file, it must be * called by a name other than "ssh" or "Secure Shell". * * SSH2 implementation: * Privilege Separation: * * Copyright (c) 2000, 2001, 2002 Markus Friedl. All rights reserved. * Copyright (c) 2002 Niels Provos. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "includes.h" __RCSID("$FreeBSD$"); #include #include #include #include #ifdef HAVE_SYS_STAT_H # include #endif #ifdef HAVE_SYS_TIME_H # include #endif #include "openbsd-compat/sys-tree.h" #include "openbsd-compat/sys-queue.h" #include #include #include #include #ifdef HAVE_PATHS_H #include #endif #include #include #include #include #include #include #include #include #include #ifdef WITH_OPENSSL #include #include #include #include "openbsd-compat/openssl-compat.h" #endif #ifdef HAVE_SECUREWARE #include #include #endif #ifdef __FreeBSD__ #include #if defined(GSSAPI) && defined(HAVE_GSSAPI_GSSAPI_H) #include #elif defined(GSSAPI) && defined(HAVE_GSSAPI_H) #include #endif #endif #include "xmalloc.h" #include "ssh.h" #include "ssh1.h" #include "ssh2.h" #include "rsa.h" #include "sshpty.h" #include "packet.h" #include "log.h" #include "buffer.h" #include "misc.h" #include "match.h" #include "servconf.h" #include "uidswap.h" #include "compat.h" #include "cipher.h" #include "digest.h" #include "key.h" #include "kex.h" #include "myproposal.h" #include "authfile.h" #include "pathnames.h" #include "atomicio.h" #include "canohost.h" #include "hostfile.h" #include "auth.h" #include "authfd.h" #include "msg.h" #include "dispatch.h" #include "channels.h" #include "session.h" #include "monitor_mm.h" #include "monitor.h" #ifdef GSSAPI #include "ssh-gss.h" #endif #include "monitor_wrap.h" #include "ssh-sandbox.h" #include "version.h" #include "ssherr.h" +#include "blacklist_client.h" #ifdef LIBWRAP #include #include int allow_severity; int deny_severity; #endif /* LIBWRAP */ #ifndef O_NOCTTY #define O_NOCTTY 0 #endif /* Re-exec fds */ #define REEXEC_DEVCRYPTO_RESERVED_FD (STDERR_FILENO + 1) #define REEXEC_STARTUP_PIPE_FD (STDERR_FILENO + 2) #define REEXEC_CONFIG_PASS_FD (STDERR_FILENO + 3) #define REEXEC_MIN_FREE_FD (STDERR_FILENO + 4) extern char *__progname; /* Server configuration options. */ ServerOptions options; /* Name of the server configuration file. */ char *config_file_name = _PATH_SERVER_CONFIG_FILE; /* * Debug mode flag. This can be set on the command line. If debug * mode is enabled, extra debugging output will be sent to the system * log, the daemon will not go to background, and will exit after processing * the first connection. */ int debug_flag = 0; /* Flag indicating that the daemon should only test the configuration and keys. */ int test_flag = 0; /* Flag indicating that the daemon is being started from inetd. */ int inetd_flag = 0; /* Flag indicating that sshd should not detach and become a daemon. */ int no_daemon_flag = 0; /* debug goes to stderr unless inetd_flag is set */ int log_stderr = 0; /* Saved arguments to main(). */ char **saved_argv; int saved_argc; /* re-exec */ int rexeced_flag = 0; int rexec_flag = 1; int rexec_argc = 0; char **rexec_argv; /* * The sockets that the server is listening; this is used in the SIGHUP * signal handler. */ #define MAX_LISTEN_SOCKS 16 int listen_socks[MAX_LISTEN_SOCKS]; int num_listen_socks = 0; /* * the client's version string, passed by sshd2 in compat mode. if != NULL, * sshd will skip the version-number exchange */ char *client_version_string = NULL; char *server_version_string = NULL; /* Daemon's agent connection */ int auth_sock = -1; int have_agent = 0; /* * Any really sensitive data in the application is contained in this * structure. The idea is that this structure could be locked into memory so * that the pages do not get written into swap. However, there are some * problems. The private key contains BIGNUMs, and we do not (in principle) * have access to the internals of them, and locking just the structure is * not very useful. Currently, memory locking is not implemented. */ struct { Key *server_key; /* ephemeral server key */ Key *ssh1_host_key; /* ssh1 host key */ Key **host_keys; /* all private host keys */ Key **host_pubkeys; /* all public host keys */ Key **host_certificates; /* all public host certificates */ int have_ssh1_key; int have_ssh2_key; u_char ssh1_cookie[SSH_SESSION_KEY_LENGTH]; } sensitive_data; /* * Flag indicating whether the RSA server key needs to be regenerated. * Is set in the SIGALRM handler and cleared when the key is regenerated. */ static volatile sig_atomic_t key_do_regen = 0; /* This is set to true when a signal is received. */ static volatile sig_atomic_t received_sighup = 0; static volatile sig_atomic_t received_sigterm = 0; /* session identifier, used by RSA-auth */ u_char session_id[16]; /* same for ssh2 */ u_char *session_id2 = NULL; u_int session_id2_len = 0; /* record remote hostname or ip */ u_int utmp_len = HOST_NAME_MAX+1; /* options.max_startup sized array of fd ints */ int *startup_pipes = NULL; int startup_pipe; /* in child */ /* variables used for privilege separation */ int use_privsep = -1; struct monitor *pmonitor = NULL; int privsep_is_preauth = 1; /* global authentication context */ Authctxt *the_authctxt = NULL; /* sshd_config buffer */ Buffer cfg; /* message to be displayed after login */ Buffer loginmsg; /* Unprivileged user */ struct passwd *privsep_pw = NULL; /* Prototypes for various functions defined later in this file. */ void destroy_sensitive_data(void); void demote_sensitive_data(void); #ifdef WITH_SSH1 static void do_ssh1_kex(void); #endif static void do_ssh2_kex(void); /* * Close all listening sockets */ static void close_listen_socks(void) { int i; for (i = 0; i < num_listen_socks; i++) close(listen_socks[i]); num_listen_socks = -1; } static void close_startup_pipes(void) { int i; if (startup_pipes) for (i = 0; i < options.max_startups; i++) if (startup_pipes[i] != -1) close(startup_pipes[i]); } /* * Signal handler for SIGHUP. Sshd execs itself when it receives SIGHUP; * the effect is to reread the configuration file (and to regenerate * the server key). */ /*ARGSUSED*/ static void sighup_handler(int sig) { int save_errno = errno; received_sighup = 1; signal(SIGHUP, sighup_handler); errno = save_errno; } /* * Called from the main program after receiving SIGHUP. * Restarts the server. */ static void sighup_restart(void) { logit("Received SIGHUP; restarting."); platform_pre_restart(); close_listen_socks(); close_startup_pipes(); alarm(0); /* alarm timer persists across exec */ signal(SIGHUP, SIG_IGN); /* will be restored after exec */ execv(saved_argv[0], saved_argv); logit("RESTART FAILED: av[0]='%.100s', error: %.100s.", saved_argv[0], strerror(errno)); exit(1); } /* * Generic signal handler for terminating signals in the master daemon. */ /*ARGSUSED*/ static void sigterm_handler(int sig) { received_sigterm = sig; } /* * SIGCHLD handler. This is called whenever a child dies. This will then * reap any zombies left by exited children. */ /*ARGSUSED*/ static void main_sigchld_handler(int sig) { int save_errno = errno; pid_t pid; int status; while ((pid = waitpid(-1, &status, WNOHANG)) > 0 || (pid < 0 && errno == EINTR)) ; signal(SIGCHLD, main_sigchld_handler); errno = save_errno; } /* * Signal handler for the alarm after the login grace period has expired. */ /*ARGSUSED*/ static void grace_alarm_handler(int sig) { if (use_privsep && pmonitor != NULL && pmonitor->m_pid > 0) kill(pmonitor->m_pid, SIGALRM); /* * Try to kill any processes that we have spawned, E.g. authorized * keys command helpers. */ if (getpgid(0) == getpid()) { signal(SIGTERM, SIG_IGN); kill(0, SIGTERM); } + BLACKLIST_NOTIFY(BLACKLIST_AUTH_FAIL); + /* Log error and exit. */ sigdie("Timeout before authentication for %s", get_remote_ipaddr()); } /* * Signal handler for the key regeneration alarm. Note that this * alarm only occurs in the daemon waiting for connections, and it does not * do anything with the private key or random state before forking. * Thus there should be no concurrency control/asynchronous execution * problems. */ static void generate_ephemeral_server_key(void) { verbose("Generating %s%d bit RSA key.", sensitive_data.server_key ? "new " : "", options.server_key_bits); if (sensitive_data.server_key != NULL) key_free(sensitive_data.server_key); sensitive_data.server_key = key_generate(KEY_RSA1, options.server_key_bits); verbose("RSA key generation complete."); arc4random_buf(sensitive_data.ssh1_cookie, SSH_SESSION_KEY_LENGTH); } /*ARGSUSED*/ static void key_regeneration_alarm(int sig) { int save_errno = errno; signal(SIGALRM, SIG_DFL); errno = save_errno; key_do_regen = 1; } static void sshd_exchange_identification(int sock_in, int sock_out) { u_int i; int mismatch; int remote_major, remote_minor; int major, minor; char *s, *newline = "\n"; char buf[256]; /* Must not be larger than remote_version. */ char remote_version[256]; /* Must be at least as big as buf. */ if ((options.protocol & SSH_PROTO_1) && (options.protocol & SSH_PROTO_2)) { major = PROTOCOL_MAJOR_1; minor = 99; } else if (options.protocol & SSH_PROTO_2) { major = PROTOCOL_MAJOR_2; minor = PROTOCOL_MINOR_2; newline = "\r\n"; } else { major = PROTOCOL_MAJOR_1; minor = PROTOCOL_MINOR_1; } xasprintf(&server_version_string, "SSH-%d.%d-%.100s%s%s%s", major, minor, SSH_VERSION, *options.version_addendum == '\0' ? "" : " ", options.version_addendum, newline); /* Send our protocol version identification. */ if (atomicio(vwrite, sock_out, server_version_string, strlen(server_version_string)) != strlen(server_version_string)) { logit("Could not write ident string to %s", get_remote_ipaddr()); cleanup_exit(255); } /* Read other sides version identification. */ memset(buf, 0, sizeof(buf)); for (i = 0; i < sizeof(buf) - 1; i++) { if (atomicio(read, sock_in, &buf[i], 1) != 1) { logit("Did not receive identification string from %s", get_remote_ipaddr()); cleanup_exit(255); } if (buf[i] == '\r') { buf[i] = 0; /* Kludge for F-Secure Macintosh < 1.0.2 */ if (i == 12 && strncmp(buf, "SSH-1.5-W1.0", 12) == 0) break; continue; } if (buf[i] == '\n') { buf[i] = 0; break; } } buf[sizeof(buf) - 1] = 0; client_version_string = xstrdup(buf); /* * Check that the versions match. In future this might accept * several versions and set appropriate flags to handle them. */ if (sscanf(client_version_string, "SSH-%d.%d-%[^\n]\n", &remote_major, &remote_minor, remote_version) != 3) { s = "Protocol mismatch.\n"; (void) atomicio(vwrite, sock_out, s, strlen(s)); logit("Bad protocol version identification '%.100s' " "from %s port %d", client_version_string, get_remote_ipaddr(), get_remote_port()); close(sock_in); close(sock_out); cleanup_exit(255); } debug("Client protocol version %d.%d; client software version %.100s", remote_major, remote_minor, remote_version); active_state->compat = compat_datafellows(remote_version); if ((datafellows & SSH_BUG_PROBE) != 0) { logit("probed from %s with %s. Don't panic.", get_remote_ipaddr(), client_version_string); cleanup_exit(255); } if ((datafellows & SSH_BUG_SCANNER) != 0) { logit("scanned from %s with %s. Don't panic.", get_remote_ipaddr(), client_version_string); cleanup_exit(255); } if ((datafellows & SSH_BUG_RSASIGMD5) != 0) { logit("Client version \"%.100s\" uses unsafe RSA signature " "scheme; disabling use of RSA keys", remote_version); } if ((datafellows & SSH_BUG_DERIVEKEY) != 0) { fatal("Client version \"%.100s\" uses unsafe key agreement; " "refusing connection", remote_version); } mismatch = 0; switch (remote_major) { case 1: if (remote_minor == 99) { if (options.protocol & SSH_PROTO_2) enable_compat20(); else mismatch = 1; break; } if (!(options.protocol & SSH_PROTO_1)) { mismatch = 1; break; } if (remote_minor < 3) { packet_disconnect("Your ssh version is too old and " "is no longer supported. Please install a newer version."); } else if (remote_minor == 3) { /* note that this disables agent-forwarding */ enable_compat13(); } break; case 2: if (options.protocol & SSH_PROTO_2) { enable_compat20(); break; } /* FALLTHROUGH */ default: mismatch = 1; break; } chop(server_version_string); debug("Local version string %.200s", server_version_string); if (mismatch) { s = "Protocol major versions differ.\n"; (void) atomicio(vwrite, sock_out, s, strlen(s)); close(sock_in); close(sock_out); logit("Protocol major versions differ for %s: %.200s vs. %.200s", get_remote_ipaddr(), server_version_string, client_version_string); cleanup_exit(255); } } /* Destroy the host and server keys. They will no longer be needed. */ void destroy_sensitive_data(void) { int i; if (sensitive_data.server_key) { key_free(sensitive_data.server_key); sensitive_data.server_key = NULL; } for (i = 0; i < options.num_host_key_files; i++) { if (sensitive_data.host_keys[i]) { key_free(sensitive_data.host_keys[i]); sensitive_data.host_keys[i] = NULL; } if (sensitive_data.host_certificates[i]) { key_free(sensitive_data.host_certificates[i]); sensitive_data.host_certificates[i] = NULL; } } sensitive_data.ssh1_host_key = NULL; explicit_bzero(sensitive_data.ssh1_cookie, SSH_SESSION_KEY_LENGTH); } /* Demote private to public keys for network child */ void demote_sensitive_data(void) { Key *tmp; int i; if (sensitive_data.server_key) { tmp = key_demote(sensitive_data.server_key); key_free(sensitive_data.server_key); sensitive_data.server_key = tmp; } for (i = 0; i < options.num_host_key_files; i++) { if (sensitive_data.host_keys[i]) { tmp = key_demote(sensitive_data.host_keys[i]); key_free(sensitive_data.host_keys[i]); sensitive_data.host_keys[i] = tmp; if (tmp->type == KEY_RSA1) sensitive_data.ssh1_host_key = tmp; } /* Certs do not need demotion */ } /* We do not clear ssh1_host key and cookie. XXX - Okay Niels? */ } static void privsep_preauth_child(void) { u_int32_t rnd[256]; gid_t gidset[1]; /* Enable challenge-response authentication for privilege separation */ privsep_challenge_enable(); #ifdef GSSAPI /* Cache supported mechanism OIDs for later use */ if (options.gss_authentication) ssh_gssapi_prepare_supported_oids(); #endif arc4random_stir(); arc4random_buf(rnd, sizeof(rnd)); #ifdef WITH_OPENSSL RAND_seed(rnd, sizeof(rnd)); if ((RAND_bytes((u_char *)rnd, 1)) != 1) fatal("%s: RAND_bytes failed", __func__); #endif explicit_bzero(rnd, sizeof(rnd)); /* Demote the private keys to public keys. */ demote_sensitive_data(); /* Demote the child */ if (getuid() == 0 || geteuid() == 0) { /* Change our root directory */ if (chroot(_PATH_PRIVSEP_CHROOT_DIR) == -1) fatal("chroot(\"%s\"): %s", _PATH_PRIVSEP_CHROOT_DIR, strerror(errno)); if (chdir("/") == -1) fatal("chdir(\"/\"): %s", strerror(errno)); /* Drop our privileges */ debug3("privsep user:group %u:%u", (u_int)privsep_pw->pw_uid, (u_int)privsep_pw->pw_gid); gidset[0] = privsep_pw->pw_gid; if (setgroups(1, gidset) < 0) fatal("setgroups: %.100s", strerror(errno)); permanently_set_uid(privsep_pw); } } static int privsep_preauth(Authctxt *authctxt) { int status, r; pid_t pid; struct ssh_sandbox *box = NULL; /* Set up unprivileged child process to deal with network data */ pmonitor = monitor_init(); /* Store a pointer to the kex for later rekeying */ pmonitor->m_pkex = &active_state->kex; if (use_privsep == PRIVSEP_ON) box = ssh_sandbox_init(pmonitor); pid = fork(); if (pid == -1) { fatal("fork of unprivileged child failed"); } else if (pid != 0) { debug2("Network child is on pid %ld", (long)pid); pmonitor->m_pid = pid; if (have_agent) { r = ssh_get_authentication_socket(&auth_sock); if (r != 0) { error("Could not get agent socket: %s", ssh_err(r)); have_agent = 0; } } if (box != NULL) ssh_sandbox_parent_preauth(box, pid); monitor_child_preauth(authctxt, pmonitor); /* Sync memory */ monitor_sync(pmonitor); /* Wait for the child's exit status */ while (waitpid(pid, &status, 0) < 0) { if (errno == EINTR) continue; pmonitor->m_pid = -1; fatal("%s: waitpid: %s", __func__, strerror(errno)); } privsep_is_preauth = 0; pmonitor->m_pid = -1; if (WIFEXITED(status)) { if (WEXITSTATUS(status) != 0) fatal("%s: preauth child exited with status %d", __func__, WEXITSTATUS(status)); } else if (WIFSIGNALED(status)) fatal("%s: preauth child terminated by signal %d", __func__, WTERMSIG(status)); if (box != NULL) ssh_sandbox_parent_finish(box); return 1; } else { /* child */ close(pmonitor->m_sendfd); close(pmonitor->m_log_recvfd); /* Arrange for logging to be sent to the monitor */ set_log_handler(mm_log_handler, pmonitor); privsep_preauth_child(); setproctitle("%s", "[net]"); if (box != NULL) ssh_sandbox_child(box); return 0; } } static void privsep_postauth(Authctxt *authctxt) { u_int32_t rnd[256]; #ifdef DISABLE_FD_PASSING if (1) { #else if (authctxt->pw->pw_uid == 0 || options.use_login) { #endif /* File descriptor passing is broken or root login */ use_privsep = 0; goto skip; } /* New socket pair */ monitor_reinit(pmonitor); pmonitor->m_pid = fork(); if (pmonitor->m_pid == -1) fatal("fork of unprivileged child failed"); else if (pmonitor->m_pid != 0) { verbose("User child is on pid %ld", (long)pmonitor->m_pid); buffer_clear(&loginmsg); monitor_child_postauth(pmonitor); /* NEVERREACHED */ exit(0); } /* child */ close(pmonitor->m_sendfd); pmonitor->m_sendfd = -1; /* Demote the private keys to public keys. */ demote_sensitive_data(); arc4random_stir(); arc4random_buf(rnd, sizeof(rnd)); #ifdef WITH_OPENSSL RAND_seed(rnd, sizeof(rnd)); if ((RAND_bytes((u_char *)rnd, 1)) != 1) fatal("%s: RAND_bytes failed", __func__); #endif explicit_bzero(rnd, sizeof(rnd)); /* Drop privileges */ do_setusercontext(authctxt->pw); skip: /* It is safe now to apply the key state */ monitor_apply_keystate(pmonitor); /* * Tell the packet layer that authentication was successful, since * this information is not part of the key state. */ packet_set_authenticated(); } static char * list_hostkey_types(void) { Buffer b; const char *p; char *ret; int i; Key *key; buffer_init(&b); for (i = 0; i < options.num_host_key_files; i++) { key = sensitive_data.host_keys[i]; if (key == NULL) key = sensitive_data.host_pubkeys[i]; if (key == NULL || key->type == KEY_RSA1) continue; /* Check that the key is accepted in HostkeyAlgorithms */ if (match_pattern_list(sshkey_ssh_name(key), options.hostkeyalgorithms, 0) != 1) { debug3("%s: %s key not permitted by HostkeyAlgorithms", __func__, sshkey_ssh_name(key)); continue; } switch (key->type) { case KEY_RSA: case KEY_DSA: case KEY_ECDSA: case KEY_ED25519: if (buffer_len(&b) > 0) buffer_append(&b, ",", 1); p = key_ssh_name(key); buffer_append(&b, p, strlen(p)); /* for RSA we also support SHA2 signatures */ if (key->type == KEY_RSA) { p = ",rsa-sha2-512,rsa-sha2-256"; buffer_append(&b, p, strlen(p)); } break; } /* If the private key has a cert peer, then list that too */ key = sensitive_data.host_certificates[i]; if (key == NULL) continue; switch (key->type) { case KEY_RSA_CERT: case KEY_DSA_CERT: case KEY_ECDSA_CERT: case KEY_ED25519_CERT: if (buffer_len(&b) > 0) buffer_append(&b, ",", 1); p = key_ssh_name(key); buffer_append(&b, p, strlen(p)); break; } } buffer_append(&b, "\0", 1); ret = xstrdup(buffer_ptr(&b)); buffer_free(&b); debug("list_hostkey_types: %s", ret); return ret; } static Key * get_hostkey_by_type(int type, int nid, int need_private, struct ssh *ssh) { int i; Key *key; for (i = 0; i < options.num_host_key_files; i++) { switch (type) { case KEY_RSA_CERT: case KEY_DSA_CERT: case KEY_ECDSA_CERT: case KEY_ED25519_CERT: key = sensitive_data.host_certificates[i]; break; default: key = sensitive_data.host_keys[i]; if (key == NULL && !need_private) key = sensitive_data.host_pubkeys[i]; break; } if (key != NULL && key->type == type && (key->type != KEY_ECDSA || key->ecdsa_nid == nid)) return need_private ? sensitive_data.host_keys[i] : key; } return NULL; } Key * get_hostkey_public_by_type(int type, int nid, struct ssh *ssh) { return get_hostkey_by_type(type, nid, 0, ssh); } Key * get_hostkey_private_by_type(int type, int nid, struct ssh *ssh) { return get_hostkey_by_type(type, nid, 1, ssh); } Key * get_hostkey_by_index(int ind) { if (ind < 0 || ind >= options.num_host_key_files) return (NULL); return (sensitive_data.host_keys[ind]); } Key * get_hostkey_public_by_index(int ind, struct ssh *ssh) { if (ind < 0 || ind >= options.num_host_key_files) return (NULL); return (sensitive_data.host_pubkeys[ind]); } int get_hostkey_index(Key *key, int compare, struct ssh *ssh) { int i; for (i = 0; i < options.num_host_key_files; i++) { if (key_is_cert(key)) { if (key == sensitive_data.host_certificates[i] || (compare && sensitive_data.host_certificates[i] && sshkey_equal(key, sensitive_data.host_certificates[i]))) return (i); } else { if (key == sensitive_data.host_keys[i] || (compare && sensitive_data.host_keys[i] && sshkey_equal(key, sensitive_data.host_keys[i]))) return (i); if (key == sensitive_data.host_pubkeys[i] || (compare && sensitive_data.host_pubkeys[i] && sshkey_equal(key, sensitive_data.host_pubkeys[i]))) return (i); } } return (-1); } /* Inform the client of all hostkeys */ static void notify_hostkeys(struct ssh *ssh) { struct sshbuf *buf; struct sshkey *key; int i, nkeys, r; char *fp; /* Some clients cannot cope with the hostkeys message, skip those. */ if (datafellows & SSH_BUG_HOSTKEYS) return; if ((buf = sshbuf_new()) == NULL) fatal("%s: sshbuf_new", __func__); for (i = nkeys = 0; i < options.num_host_key_files; i++) { key = get_hostkey_public_by_index(i, ssh); if (key == NULL || key->type == KEY_UNSPEC || key->type == KEY_RSA1 || sshkey_is_cert(key)) continue; fp = sshkey_fingerprint(key, options.fingerprint_hash, SSH_FP_DEFAULT); debug3("%s: key %d: %s %s", __func__, i, sshkey_ssh_name(key), fp); free(fp); if (nkeys == 0) { packet_start(SSH2_MSG_GLOBAL_REQUEST); packet_put_cstring("hostkeys-00@openssh.com"); packet_put_char(0); /* want-reply */ } sshbuf_reset(buf); if ((r = sshkey_putb(key, buf)) != 0) fatal("%s: couldn't put hostkey %d: %s", __func__, i, ssh_err(r)); packet_put_string(sshbuf_ptr(buf), sshbuf_len(buf)); nkeys++; } debug3("%s: sent %d hostkeys", __func__, nkeys); if (nkeys == 0) fatal("%s: no hostkeys", __func__); packet_send(); sshbuf_free(buf); } /* * returns 1 if connection should be dropped, 0 otherwise. * dropping starts at connection #max_startups_begin with a probability * of (max_startups_rate/100). the probability increases linearly until * all connections are dropped for startups > max_startups */ static int drop_connection(int startups) { int p, r; if (startups < options.max_startups_begin) return 0; if (startups >= options.max_startups) return 1; if (options.max_startups_rate == 100) return 1; p = 100 - options.max_startups_rate; p *= startups - options.max_startups_begin; p /= options.max_startups - options.max_startups_begin; p += options.max_startups_rate; r = arc4random_uniform(100); debug("drop_connection: p %d, r %d", p, r); return (r < p) ? 1 : 0; } static void usage(void) { if (options.version_addendum && *options.version_addendum != '\0') fprintf(stderr, "%s %s, %s\n", SSH_RELEASE, options.version_addendum, OPENSSL_VERSION); else fprintf(stderr, "%s, %s\n", SSH_RELEASE, OPENSSL_VERSION); fprintf(stderr, "usage: sshd [-46DdeiqTt] [-b bits] [-C connection_spec] [-c host_cert_file]\n" " [-E log_file] [-f config_file] [-g login_grace_time]\n" " [-h host_key_file] [-k key_gen_time] [-o option] [-p port]\n" " [-u len]\n" ); exit(1); } static void send_rexec_state(int fd, Buffer *conf) { Buffer m; debug3("%s: entering fd = %d config len %d", __func__, fd, buffer_len(conf)); /* * Protocol from reexec master to child: * string configuration * u_int ephemeral_key_follows * bignum e (only if ephemeral_key_follows == 1) * bignum n " * bignum d " * bignum iqmp " * bignum p " * bignum q " * string rngseed (only if OpenSSL is not self-seeded) */ buffer_init(&m); buffer_put_cstring(&m, buffer_ptr(conf)); #ifdef WITH_SSH1 if (sensitive_data.server_key != NULL && sensitive_data.server_key->type == KEY_RSA1) { buffer_put_int(&m, 1); buffer_put_bignum(&m, sensitive_data.server_key->rsa->e); buffer_put_bignum(&m, sensitive_data.server_key->rsa->n); buffer_put_bignum(&m, sensitive_data.server_key->rsa->d); buffer_put_bignum(&m, sensitive_data.server_key->rsa->iqmp); buffer_put_bignum(&m, sensitive_data.server_key->rsa->p); buffer_put_bignum(&m, sensitive_data.server_key->rsa->q); } else #endif buffer_put_int(&m, 0); #if defined(WITH_OPENSSL) && !defined(OPENSSL_PRNG_ONLY) rexec_send_rng_seed(&m); #endif if (ssh_msg_send(fd, 0, &m) == -1) fatal("%s: ssh_msg_send failed", __func__); buffer_free(&m); debug3("%s: done", __func__); } static void recv_rexec_state(int fd, Buffer *conf) { Buffer m; char *cp; u_int len; debug3("%s: entering fd = %d", __func__, fd); buffer_init(&m); if (ssh_msg_recv(fd, &m) == -1) fatal("%s: ssh_msg_recv failed", __func__); if (buffer_get_char(&m) != 0) fatal("%s: rexec version mismatch", __func__); cp = buffer_get_string(&m, &len); if (conf != NULL) buffer_append(conf, cp, len + 1); free(cp); if (buffer_get_int(&m)) { #ifdef WITH_SSH1 if (sensitive_data.server_key != NULL) key_free(sensitive_data.server_key); sensitive_data.server_key = key_new_private(KEY_RSA1); buffer_get_bignum(&m, sensitive_data.server_key->rsa->e); buffer_get_bignum(&m, sensitive_data.server_key->rsa->n); buffer_get_bignum(&m, sensitive_data.server_key->rsa->d); buffer_get_bignum(&m, sensitive_data.server_key->rsa->iqmp); buffer_get_bignum(&m, sensitive_data.server_key->rsa->p); buffer_get_bignum(&m, sensitive_data.server_key->rsa->q); if (rsa_generate_additional_parameters( sensitive_data.server_key->rsa) != 0) fatal("%s: rsa_generate_additional_parameters " "error", __func__); #endif } #if defined(WITH_OPENSSL) && !defined(OPENSSL_PRNG_ONLY) rexec_recv_rng_seed(&m); #endif buffer_free(&m); debug3("%s: done", __func__); } /* Accept a connection from inetd */ static void server_accept_inetd(int *sock_in, int *sock_out) { int fd; startup_pipe = -1; if (rexeced_flag) { close(REEXEC_CONFIG_PASS_FD); *sock_in = *sock_out = dup(STDIN_FILENO); if (!debug_flag) { startup_pipe = dup(REEXEC_STARTUP_PIPE_FD); close(REEXEC_STARTUP_PIPE_FD); } } else { *sock_in = dup(STDIN_FILENO); *sock_out = dup(STDOUT_FILENO); } /* * We intentionally do not close the descriptors 0, 1, and 2 * as our code for setting the descriptors won't work if * ttyfd happens to be one of those. */ if ((fd = open(_PATH_DEVNULL, O_RDWR, 0)) != -1) { dup2(fd, STDIN_FILENO); dup2(fd, STDOUT_FILENO); if (!log_stderr) dup2(fd, STDERR_FILENO); if (fd > (log_stderr ? STDERR_FILENO : STDOUT_FILENO)) close(fd); } debug("inetd sockets after dupping: %d, %d", *sock_in, *sock_out); } /* * Listen for TCP connections */ static void server_listen(void) { int ret, listen_sock, on = 1; struct addrinfo *ai; char ntop[NI_MAXHOST], strport[NI_MAXSERV]; int socksize; socklen_t len; for (ai = options.listen_addrs; ai; ai = ai->ai_next) { if (ai->ai_family != AF_INET && ai->ai_family != AF_INET6) continue; if (num_listen_socks >= MAX_LISTEN_SOCKS) fatal("Too many listen sockets. " "Enlarge MAX_LISTEN_SOCKS"); if ((ret = getnameinfo(ai->ai_addr, ai->ai_addrlen, ntop, sizeof(ntop), strport, sizeof(strport), NI_NUMERICHOST|NI_NUMERICSERV)) != 0) { error("getnameinfo failed: %.100s", ssh_gai_strerror(ret)); continue; } /* Create socket for listening. */ listen_sock = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol); if (listen_sock < 0) { /* kernel may not support ipv6 */ verbose("socket: %.100s", strerror(errno)); continue; } if (set_nonblock(listen_sock) == -1) { close(listen_sock); continue; } /* * Set socket options. * Allow local port reuse in TIME_WAIT. */ if (setsockopt(listen_sock, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)) == -1) error("setsockopt SO_REUSEADDR: %s", strerror(errno)); /* Only communicate in IPv6 over AF_INET6 sockets. */ if (ai->ai_family == AF_INET6) sock_set_v6only(listen_sock); debug("Bind to port %s on %s.", strport, ntop); len = sizeof(socksize); getsockopt(listen_sock, SOL_SOCKET, SO_RCVBUF, &socksize, &len); debug("Server TCP RWIN socket size: %d", socksize); /* Bind the socket to the desired port. */ if (bind(listen_sock, ai->ai_addr, ai->ai_addrlen) < 0) { error("Bind to port %s on %s failed: %.200s.", strport, ntop, strerror(errno)); close(listen_sock); continue; } listen_socks[num_listen_socks] = listen_sock; num_listen_socks++; /* Start listening on the port. */ if (listen(listen_sock, SSH_LISTEN_BACKLOG) < 0) fatal("listen on [%s]:%s: %.100s", ntop, strport, strerror(errno)); logit("Server listening on %s port %s.", ntop, strport); } freeaddrinfo(options.listen_addrs); if (!num_listen_socks) fatal("Cannot bind any address."); } /* * The main TCP accept loop. Note that, for the non-debug case, returns * from this function are in a forked subprocess. */ static void server_accept_loop(int *sock_in, int *sock_out, int *newsock, int *config_s) { fd_set *fdset; int i, j, ret, maxfd; int key_used = 0, startups = 0; int startup_p[2] = { -1 , -1 }; struct sockaddr_storage from; socklen_t fromlen; pid_t pid; u_char rnd[256]; /* setup fd set for accept */ fdset = NULL; maxfd = 0; for (i = 0; i < num_listen_socks; i++) if (listen_socks[i] > maxfd) maxfd = listen_socks[i]; /* pipes connected to unauthenticated childs */ startup_pipes = xcalloc(options.max_startups, sizeof(int)); for (i = 0; i < options.max_startups; i++) startup_pipes[i] = -1; /* * Stay listening for connections until the system crashes or * the daemon is killed with a signal. */ for (;;) { if (received_sighup) sighup_restart(); free(fdset); fdset = xcalloc(howmany(maxfd + 1, NFDBITS), sizeof(fd_mask)); for (i = 0; i < num_listen_socks; i++) FD_SET(listen_socks[i], fdset); for (i = 0; i < options.max_startups; i++) if (startup_pipes[i] != -1) FD_SET(startup_pipes[i], fdset); /* Wait in select until there is a connection. */ ret = select(maxfd+1, fdset, NULL, NULL, NULL); if (ret < 0 && errno != EINTR) error("select: %.100s", strerror(errno)); if (received_sigterm) { logit("Received signal %d; terminating.", (int) received_sigterm); close_listen_socks(); if (options.pid_file != NULL) unlink(options.pid_file); exit(received_sigterm == SIGTERM ? 0 : 255); } if (key_used && key_do_regen) { generate_ephemeral_server_key(); key_used = 0; key_do_regen = 0; } if (ret < 0) continue; for (i = 0; i < options.max_startups; i++) if (startup_pipes[i] != -1 && FD_ISSET(startup_pipes[i], fdset)) { /* * the read end of the pipe is ready * if the child has closed the pipe * after successful authentication * or if the child has died */ close(startup_pipes[i]); startup_pipes[i] = -1; startups--; } for (i = 0; i < num_listen_socks; i++) { if (!FD_ISSET(listen_socks[i], fdset)) continue; fromlen = sizeof(from); *newsock = accept(listen_socks[i], (struct sockaddr *)&from, &fromlen); if (*newsock < 0) { if (errno != EINTR && errno != EWOULDBLOCK && errno != ECONNABORTED && errno != EAGAIN) error("accept: %.100s", strerror(errno)); if (errno == EMFILE || errno == ENFILE) usleep(100 * 1000); continue; } if (unset_nonblock(*newsock) == -1) { close(*newsock); continue; } if (drop_connection(startups) == 1) { debug("drop connection #%d", startups); close(*newsock); continue; } if (pipe(startup_p) == -1) { close(*newsock); continue; } if (rexec_flag && socketpair(AF_UNIX, SOCK_STREAM, 0, config_s) == -1) { error("reexec socketpair: %s", strerror(errno)); close(*newsock); close(startup_p[0]); close(startup_p[1]); continue; } for (j = 0; j < options.max_startups; j++) if (startup_pipes[j] == -1) { startup_pipes[j] = startup_p[0]; if (maxfd < startup_p[0]) maxfd = startup_p[0]; startups++; break; } /* * Got connection. Fork a child to handle it, unless * we are in debugging mode. */ if (debug_flag) { /* * In debugging mode. Close the listening * socket, and start processing the * connection without forking. */ debug("Server will not fork when running in debugging mode."); close_listen_socks(); *sock_in = *newsock; *sock_out = *newsock; close(startup_p[0]); close(startup_p[1]); startup_pipe = -1; pid = getpid(); if (rexec_flag) { send_rexec_state(config_s[0], &cfg); close(config_s[0]); } break; } /* * Normal production daemon. Fork, and have * the child process the connection. The * parent continues listening. */ platform_pre_fork(); if ((pid = fork()) == 0) { /* * Child. Close the listening and * max_startup sockets. Start using * the accepted socket. Reinitialize * logging (since our pid has changed). * We break out of the loop to handle * the connection. */ platform_post_fork_child(); startup_pipe = startup_p[1]; close_startup_pipes(); close_listen_socks(); *sock_in = *newsock; *sock_out = *newsock; log_init(__progname, options.log_level, options.log_facility, log_stderr); if (rexec_flag) close(config_s[0]); break; } /* Parent. Stay in the loop. */ platform_post_fork_parent(pid); if (pid < 0) error("fork: %.100s", strerror(errno)); else debug("Forked child %ld.", (long)pid); close(startup_p[1]); if (rexec_flag) { send_rexec_state(config_s[0], &cfg); close(config_s[0]); close(config_s[1]); } /* * Mark that the key has been used (it * was "given" to the child). */ if ((options.protocol & SSH_PROTO_1) && key_used == 0) { /* Schedule server key regeneration alarm. */ signal(SIGALRM, key_regeneration_alarm); alarm(options.key_regeneration_time); key_used = 1; } close(*newsock); /* * Ensure that our random state differs * from that of the child */ arc4random_stir(); arc4random_buf(rnd, sizeof(rnd)); #ifdef WITH_OPENSSL RAND_seed(rnd, sizeof(rnd)); if ((RAND_bytes((u_char *)rnd, 1)) != 1) fatal("%s: RAND_bytes failed", __func__); #endif explicit_bzero(rnd, sizeof(rnd)); } /* child process check (or debug mode) */ if (num_listen_socks < 0) break; } } /* * Main program for the daemon. */ int main(int ac, char **av) { extern char *optarg; extern int optind; int r, opt, i, j, on = 1; int sock_in = -1, sock_out = -1, newsock = -1; const char *remote_ip; int remote_port; char *fp, *line, *laddr, *logfile = NULL; int config_s[2] = { -1 , -1 }; u_int n; u_int64_t ibytes, obytes; mode_t new_umask; Key *key; Key *pubkey; int keytype; Authctxt *authctxt; struct connection_info *connection_info = get_connection_info(0, 0); ssh_malloc_init(); /* must be called before any mallocs */ #ifdef HAVE_SECUREWARE (void)set_auth_parameters(ac, av); #endif __progname = ssh_get_progname(av[0]); /* Save argv. Duplicate so setproctitle emulation doesn't clobber it */ saved_argc = ac; rexec_argc = ac; saved_argv = xcalloc(ac + 1, sizeof(*saved_argv)); for (i = 0; i < ac; i++) saved_argv[i] = xstrdup(av[i]); saved_argv[i] = NULL; #ifndef HAVE_SETPROCTITLE /* Prepare for later setproctitle emulation */ compat_init_setproctitle(ac, av); av = saved_argv; #endif if (geteuid() == 0 && setgroups(0, NULL) == -1) debug("setgroups(): %.200s", strerror(errno)); /* Ensure that fds 0, 1 and 2 are open or directed to /dev/null */ sanitise_stdfd(); /* Initialize configuration options to their default values. */ initialize_server_options(&options); /* Parse command-line arguments. */ while ((opt = getopt(ac, av, "C:E:b:c:f:g:h:k:o:p:u:46DQRTdeiqrt")) != -1) { switch (opt) { case '4': options.address_family = AF_INET; break; case '6': options.address_family = AF_INET6; break; case 'f': config_file_name = optarg; break; case 'c': if (options.num_host_cert_files >= MAX_HOSTCERTS) { fprintf(stderr, "too many host certificates.\n"); exit(1); } options.host_cert_files[options.num_host_cert_files++] = derelativise_path(optarg); break; case 'd': if (debug_flag == 0) { debug_flag = 1; options.log_level = SYSLOG_LEVEL_DEBUG1; } else if (options.log_level < SYSLOG_LEVEL_DEBUG3) options.log_level++; break; case 'D': no_daemon_flag = 1; break; case 'E': logfile = optarg; /* FALLTHROUGH */ case 'e': log_stderr = 1; break; case 'i': inetd_flag = 1; break; case 'r': rexec_flag = 0; break; case 'R': rexeced_flag = 1; inetd_flag = 1; break; case 'Q': /* ignored */ break; case 'q': options.log_level = SYSLOG_LEVEL_QUIET; break; case 'b': options.server_key_bits = (int)strtonum(optarg, 256, 32768, NULL); break; case 'p': options.ports_from_cmdline = 1; if (options.num_ports >= MAX_PORTS) { fprintf(stderr, "too many ports.\n"); exit(1); } options.ports[options.num_ports++] = a2port(optarg); if (options.ports[options.num_ports-1] <= 0) { fprintf(stderr, "Bad port number.\n"); exit(1); } break; case 'g': if ((options.login_grace_time = convtime(optarg)) == -1) { fprintf(stderr, "Invalid login grace time.\n"); exit(1); } break; case 'k': if ((options.key_regeneration_time = convtime(optarg)) == -1) { fprintf(stderr, "Invalid key regeneration interval.\n"); exit(1); } break; case 'h': if (options.num_host_key_files >= MAX_HOSTKEYS) { fprintf(stderr, "too many host keys.\n"); exit(1); } options.host_key_files[options.num_host_key_files++] = derelativise_path(optarg); break; case 't': test_flag = 1; break; case 'T': test_flag = 2; break; case 'C': if (parse_server_match_testspec(connection_info, optarg) == -1) exit(1); break; case 'u': utmp_len = (u_int)strtonum(optarg, 0, HOST_NAME_MAX+1+1, NULL); if (utmp_len > HOST_NAME_MAX+1) { fprintf(stderr, "Invalid utmp length.\n"); exit(1); } break; case 'o': line = xstrdup(optarg); if (process_server_config_line(&options, line, "command-line", 0, NULL, NULL) != 0) exit(1); free(line); break; case '?': default: usage(); break; } } if (rexeced_flag || inetd_flag) rexec_flag = 0; if (!test_flag && (rexec_flag && (av[0] == NULL || *av[0] != '/'))) fatal("sshd re-exec requires execution with an absolute path"); if (rexeced_flag) closefrom(REEXEC_MIN_FREE_FD); else closefrom(REEXEC_DEVCRYPTO_RESERVED_FD); #ifdef WITH_OPENSSL OpenSSL_add_all_algorithms(); #endif /* If requested, redirect the logs to the specified logfile. */ if (logfile != NULL) log_redirect_stderr_to(logfile); /* * Force logging to stderr until we have loaded the private host * key (unless started from inetd) */ log_init(__progname, options.log_level == SYSLOG_LEVEL_NOT_SET ? SYSLOG_LEVEL_INFO : options.log_level, options.log_facility == SYSLOG_FACILITY_NOT_SET ? SYSLOG_FACILITY_AUTH : options.log_facility, log_stderr || !inetd_flag); /* * Unset KRB5CCNAME, otherwise the user's session may inherit it from * root's environment */ if (getenv("KRB5CCNAME") != NULL) (void) unsetenv("KRB5CCNAME"); #ifdef _UNICOS /* Cray can define user privs drop all privs now! * Not needed on PRIV_SU systems! */ drop_cray_privs(); #endif sensitive_data.server_key = NULL; sensitive_data.ssh1_host_key = NULL; sensitive_data.have_ssh1_key = 0; sensitive_data.have_ssh2_key = 0; /* * If we're doing an extended config test, make sure we have all of * the parameters we need. If we're not doing an extended test, * do not silently ignore connection test params. */ if (test_flag >= 2 && server_match_spec_complete(connection_info) == 0) fatal("user, host and addr are all required when testing " "Match configs"); if (test_flag < 2 && server_match_spec_complete(connection_info) >= 0) fatal("Config test connection parameter (-C) provided without " "test mode (-T)"); /* Fetch our configuration */ buffer_init(&cfg); if (rexeced_flag) recv_rexec_state(REEXEC_CONFIG_PASS_FD, &cfg); else if (strcasecmp(config_file_name, "none") != 0) load_server_config(config_file_name, &cfg); parse_server_config(&options, rexeced_flag ? "rexec" : config_file_name, &cfg, NULL); seed_rng(); /* Fill in default values for those options not explicitly set. */ fill_default_server_options(&options); /* challenge-response is implemented via keyboard interactive */ if (options.challenge_response_authentication) options.kbd_interactive_authentication = 1; /* Check that options are sensible */ if (options.authorized_keys_command_user == NULL && (options.authorized_keys_command != NULL && strcasecmp(options.authorized_keys_command, "none") != 0)) fatal("AuthorizedKeysCommand set without " "AuthorizedKeysCommandUser"); if (options.authorized_principals_command_user == NULL && (options.authorized_principals_command != NULL && strcasecmp(options.authorized_principals_command, "none") != 0)) fatal("AuthorizedPrincipalsCommand set without " "AuthorizedPrincipalsCommandUser"); /* * Check whether there is any path through configured auth methods. * Unfortunately it is not possible to verify this generally before * daemonisation in the presence of Match block, but this catches * and warns for trivial misconfigurations that could break login. */ if (options.num_auth_methods != 0) { if ((options.protocol & SSH_PROTO_1)) fatal("AuthenticationMethods is not supported with " "SSH protocol 1"); for (n = 0; n < options.num_auth_methods; n++) { if (auth2_methods_valid(options.auth_methods[n], 1) == 0) break; } if (n >= options.num_auth_methods) fatal("AuthenticationMethods cannot be satisfied by " "enabled authentication methods"); } /* set default channel AF */ channel_set_af(options.address_family); /* Check that there are no remaining arguments. */ if (optind < ac) { fprintf(stderr, "Extra argument %s.\n", av[optind]); exit(1); } debug("sshd version %s, %s", SSH_VERSION, #ifdef WITH_OPENSSL SSLeay_version(SSLEAY_VERSION) #else "without OpenSSL" #endif ); /* Store privilege separation user for later use if required. */ if ((privsep_pw = getpwnam(SSH_PRIVSEP_USER)) == NULL) { if (use_privsep || options.kerberos_authentication) fatal("Privilege separation user %s does not exist", SSH_PRIVSEP_USER); } else { explicit_bzero(privsep_pw->pw_passwd, strlen(privsep_pw->pw_passwd)); privsep_pw = pwcopy(privsep_pw); free(privsep_pw->pw_passwd); privsep_pw->pw_passwd = xstrdup("*"); } endpwent(); /* load host keys */ sensitive_data.host_keys = xcalloc(options.num_host_key_files, sizeof(Key *)); sensitive_data.host_pubkeys = xcalloc(options.num_host_key_files, sizeof(Key *)); if (options.host_key_agent) { if (strcmp(options.host_key_agent, SSH_AUTHSOCKET_ENV_NAME)) setenv(SSH_AUTHSOCKET_ENV_NAME, options.host_key_agent, 1); if ((r = ssh_get_authentication_socket(NULL)) == 0) have_agent = 1; else error("Could not connect to agent \"%s\": %s", options.host_key_agent, ssh_err(r)); } for (i = 0; i < options.num_host_key_files; i++) { if (options.host_key_files[i] == NULL) continue; key = key_load_private(options.host_key_files[i], "", NULL); pubkey = key_load_public(options.host_key_files[i], NULL); if (pubkey == NULL && key != NULL) pubkey = key_demote(key); sensitive_data.host_keys[i] = key; sensitive_data.host_pubkeys[i] = pubkey; if (key == NULL && pubkey != NULL && pubkey->type != KEY_RSA1 && have_agent) { debug("will rely on agent for hostkey %s", options.host_key_files[i]); keytype = pubkey->type; } else if (key != NULL) { keytype = key->type; } else { error("Could not load host key: %s", options.host_key_files[i]); sensitive_data.host_keys[i] = NULL; sensitive_data.host_pubkeys[i] = NULL; continue; } switch (keytype) { case KEY_RSA1: sensitive_data.ssh1_host_key = key; sensitive_data.have_ssh1_key = 1; break; case KEY_RSA: case KEY_DSA: case KEY_ECDSA: case KEY_ED25519: if (have_agent || key != NULL) sensitive_data.have_ssh2_key = 1; break; } if ((fp = sshkey_fingerprint(pubkey, options.fingerprint_hash, SSH_FP_DEFAULT)) == NULL) fatal("sshkey_fingerprint failed"); debug("%s host key #%d: %s %s", key ? "private" : "agent", i, keytype == KEY_RSA1 ? sshkey_type(pubkey) : sshkey_ssh_name(pubkey), fp); free(fp); } if ((options.protocol & SSH_PROTO_1) && !sensitive_data.have_ssh1_key) { logit("Disabling protocol version 1. Could not load host key"); options.protocol &= ~SSH_PROTO_1; } if ((options.protocol & SSH_PROTO_2) && !sensitive_data.have_ssh2_key) { logit("Disabling protocol version 2. Could not load host key"); options.protocol &= ~SSH_PROTO_2; } if (!(options.protocol & (SSH_PROTO_1|SSH_PROTO_2))) { logit("sshd: no hostkeys available -- exiting."); exit(1); } /* * Load certificates. They are stored in an array at identical * indices to the public keys that they relate to. */ sensitive_data.host_certificates = xcalloc(options.num_host_key_files, sizeof(Key *)); for (i = 0; i < options.num_host_key_files; i++) sensitive_data.host_certificates[i] = NULL; for (i = 0; i < options.num_host_cert_files; i++) { if (options.host_cert_files[i] == NULL) continue; key = key_load_public(options.host_cert_files[i], NULL); if (key == NULL) { error("Could not load host certificate: %s", options.host_cert_files[i]); continue; } if (!key_is_cert(key)) { error("Certificate file is not a certificate: %s", options.host_cert_files[i]); key_free(key); continue; } /* Find matching private key */ for (j = 0; j < options.num_host_key_files; j++) { if (key_equal_public(key, sensitive_data.host_keys[j])) { sensitive_data.host_certificates[j] = key; break; } } if (j >= options.num_host_key_files) { error("No matching private key for certificate: %s", options.host_cert_files[i]); key_free(key); continue; } sensitive_data.host_certificates[j] = key; debug("host certificate: #%d type %d %s", j, key->type, key_type(key)); } #ifdef WITH_SSH1 /* Check certain values for sanity. */ if (options.protocol & SSH_PROTO_1) { if (options.server_key_bits < SSH_RSA_MINIMUM_MODULUS_SIZE || options.server_key_bits > OPENSSL_RSA_MAX_MODULUS_BITS) { fprintf(stderr, "Bad server key size.\n"); exit(1); } /* * Check that server and host key lengths differ sufficiently. This * is necessary to make double encryption work with rsaref. Oh, I * hate software patents. I dont know if this can go? Niels */ if (options.server_key_bits > BN_num_bits(sensitive_data.ssh1_host_key->rsa->n) - SSH_KEY_BITS_RESERVED && options.server_key_bits < BN_num_bits(sensitive_data.ssh1_host_key->rsa->n) + SSH_KEY_BITS_RESERVED) { options.server_key_bits = BN_num_bits(sensitive_data.ssh1_host_key->rsa->n) + SSH_KEY_BITS_RESERVED; debug("Forcing server key to %d bits to make it differ from host key.", options.server_key_bits); } } #endif if (use_privsep) { struct stat st; if ((stat(_PATH_PRIVSEP_CHROOT_DIR, &st) == -1) || (S_ISDIR(st.st_mode) == 0)) fatal("Missing privilege separation directory: %s", _PATH_PRIVSEP_CHROOT_DIR); #ifdef HAVE_CYGWIN if (check_ntsec(_PATH_PRIVSEP_CHROOT_DIR) && (st.st_uid != getuid () || (st.st_mode & (S_IWGRP|S_IWOTH)) != 0)) #else if (st.st_uid != 0 || (st.st_mode & (S_IWGRP|S_IWOTH)) != 0) #endif fatal("%s must be owned by root and not group or " "world-writable.", _PATH_PRIVSEP_CHROOT_DIR); } if (test_flag > 1) { if (server_match_spec_complete(connection_info) == 1) parse_server_match_config(&options, connection_info); dump_config(&options); } /* Configuration looks good, so exit if in test mode. */ if (test_flag) exit(0); /* * Clear out any supplemental groups we may have inherited. This * prevents inadvertent creation of files with bad modes (in the * portable version at least, it's certainly possible for PAM * to create a file, and we can't control the code in every * module which might be used). */ if (setgroups(0, NULL) < 0) debug("setgroups() failed: %.200s", strerror(errno)); if (rexec_flag) { rexec_argv = xcalloc(rexec_argc + 2, sizeof(char *)); for (i = 0; i < rexec_argc; i++) { debug("rexec_argv[%d]='%s'", i, saved_argv[i]); rexec_argv[i] = saved_argv[i]; } rexec_argv[rexec_argc] = "-R"; rexec_argv[rexec_argc + 1] = NULL; } /* Ensure that umask disallows at least group and world write */ new_umask = umask(0077) | 0022; (void) umask(new_umask); /* Initialize the log (it is reinitialized below in case we forked). */ if (debug_flag && (!inetd_flag || rexeced_flag)) log_stderr = 1; log_init(__progname, options.log_level, options.log_facility, log_stderr); /* * If not in debugging mode, and not started from inetd, disconnect * from the controlling terminal, and fork. The original process * exits. */ if (!(debug_flag || inetd_flag || no_daemon_flag)) { #ifdef TIOCNOTTY int fd; #endif /* TIOCNOTTY */ if (daemon(0, 0) < 0) fatal("daemon() failed: %.200s", strerror(errno)); /* Disconnect from the controlling tty. */ #ifdef TIOCNOTTY fd = open(_PATH_TTY, O_RDWR | O_NOCTTY); if (fd >= 0) { (void) ioctl(fd, TIOCNOTTY, NULL); close(fd); } #endif /* TIOCNOTTY */ } /* Reinitialize the log (because of the fork above). */ log_init(__progname, options.log_level, options.log_facility, log_stderr); /* Avoid killing the process in high-pressure swapping environments. */ if (!inetd_flag && madvise(NULL, 0, MADV_PROTECT) != 0) debug("madvise(): %.200s", strerror(errno)); /* Chdir to the root directory so that the current disk can be unmounted if desired. */ if (chdir("/") == -1) error("chdir(\"/\"): %s", strerror(errno)); /* ignore SIGPIPE */ signal(SIGPIPE, SIG_IGN); /* Get a connection, either from inetd or a listening TCP socket */ if (inetd_flag) { server_accept_inetd(&sock_in, &sock_out); } else { platform_pre_listen(); server_listen(); if (options.protocol & SSH_PROTO_1) generate_ephemeral_server_key(); signal(SIGHUP, sighup_handler); signal(SIGCHLD, main_sigchld_handler); signal(SIGTERM, sigterm_handler); signal(SIGQUIT, sigterm_handler); /* * Write out the pid file after the sigterm handler * is setup and the listen sockets are bound */ if (options.pid_file != NULL && !debug_flag) { FILE *f = fopen(options.pid_file, "w"); if (f == NULL) { error("Couldn't create pid file \"%s\": %s", options.pid_file, strerror(errno)); } else { fprintf(f, "%ld\n", (long) getpid()); fclose(f); } } /* Accept a connection and return in a forked child */ server_accept_loop(&sock_in, &sock_out, &newsock, config_s); } /* This is the child processing a new connection. */ setproctitle("%s", "[accepted]"); /* * Create a new session and process group since the 4.4BSD * setlogin() affects the entire process group. We don't * want the child to be able to affect the parent. */ #if !defined(SSHD_ACQUIRES_CTTY) /* * If setsid is called, on some platforms sshd will later acquire a * controlling terminal which will result in "could not set * controlling tty" errors. */ if (!debug_flag && !inetd_flag && setsid() < 0) error("setsid: %.100s", strerror(errno)); #endif if (rexec_flag) { int fd; debug("rexec start in %d out %d newsock %d pipe %d sock %d", sock_in, sock_out, newsock, startup_pipe, config_s[0]); dup2(newsock, STDIN_FILENO); dup2(STDIN_FILENO, STDOUT_FILENO); if (startup_pipe == -1) close(REEXEC_STARTUP_PIPE_FD); else if (startup_pipe != REEXEC_STARTUP_PIPE_FD) { dup2(startup_pipe, REEXEC_STARTUP_PIPE_FD); close(startup_pipe); startup_pipe = REEXEC_STARTUP_PIPE_FD; } dup2(config_s[1], REEXEC_CONFIG_PASS_FD); close(config_s[1]); execv(rexec_argv[0], rexec_argv); /* Reexec has failed, fall back and continue */ error("rexec of %s failed: %s", rexec_argv[0], strerror(errno)); recv_rexec_state(REEXEC_CONFIG_PASS_FD, NULL); log_init(__progname, options.log_level, options.log_facility, log_stderr); /* Clean up fds */ close(REEXEC_CONFIG_PASS_FD); newsock = sock_out = sock_in = dup(STDIN_FILENO); if ((fd = open(_PATH_DEVNULL, O_RDWR, 0)) != -1) { dup2(fd, STDIN_FILENO); dup2(fd, STDOUT_FILENO); if (fd > STDERR_FILENO) close(fd); } debug("rexec cleanup in %d out %d newsock %d pipe %d sock %d", sock_in, sock_out, newsock, startup_pipe, config_s[0]); } /* Executed child processes don't need these. */ fcntl(sock_out, F_SETFD, FD_CLOEXEC); fcntl(sock_in, F_SETFD, FD_CLOEXEC); /* * Disable the key regeneration alarm. We will not regenerate the * key since we are no longer in a position to give it to anyone. We * will not restart on SIGHUP since it no longer makes sense. */ alarm(0); signal(SIGALRM, SIG_DFL); signal(SIGHUP, SIG_DFL); signal(SIGTERM, SIG_DFL); signal(SIGQUIT, SIG_DFL); signal(SIGCHLD, SIG_DFL); signal(SIGINT, SIG_DFL); #ifdef __FreeBSD__ /* * Initialize the resolver. This may not happen automatically * before privsep chroot(). */ if ((_res.options & RES_INIT) == 0) { debug("res_init()"); res_init(); } #ifdef GSSAPI /* * Force GSS-API to parse its configuration and load any * mechanism plugins. */ { gss_OID_set mechs; OM_uint32 minor_status; gss_indicate_mechs(&minor_status, &mechs); gss_release_oid_set(&minor_status, &mechs); } #endif #endif /* * Register our connection. This turns encryption off because we do * not have a key. */ packet_set_connection(sock_in, sock_out); packet_set_server(); /* Set SO_KEEPALIVE if requested. */ if (options.tcp_keep_alive && packet_connection_is_on_socket() && setsockopt(sock_in, SOL_SOCKET, SO_KEEPALIVE, &on, sizeof(on)) < 0) error("setsockopt SO_KEEPALIVE: %.100s", strerror(errno)); if ((remote_port = get_remote_port()) < 0) { debug("get_remote_port failed"); cleanup_exit(255); } /* * We use get_canonical_hostname with usedns = 0 instead of * get_remote_ipaddr here so IP options will be checked. */ (void) get_canonical_hostname(0); /* * The rest of the code depends on the fact that * get_remote_ipaddr() caches the remote ip, even if * the socket goes away. */ remote_ip = get_remote_ipaddr(); #ifdef SSH_AUDIT_EVENTS audit_connection_from(remote_ip, remote_port); #endif #ifdef LIBWRAP allow_severity = options.log_facility|LOG_INFO; deny_severity = options.log_facility|LOG_WARNING; /* Check whether logins are denied from this host. */ if (packet_connection_is_on_socket()) { struct request_info req; request_init(&req, RQ_DAEMON, __progname, RQ_FILE, sock_in, 0); fromhost(&req); if (!hosts_access(&req)) { debug("Connection refused by tcp wrapper"); refuse(&req); /* NOTREACHED */ fatal("libwrap refuse returns"); } } #endif /* LIBWRAP */ /* Log the connection. */ laddr = get_local_ipaddr(sock_in); verbose("Connection from %s port %d on %s port %d", remote_ip, remote_port, laddr, get_local_port()); free(laddr); /* * We don't want to listen forever unless the other side * successfully authenticates itself. So we set up an alarm which is * cleared after successful authentication. A limit of zero * indicates no limit. Note that we don't set the alarm in debugging * mode; it is just annoying to have the server exit just when you * are about to discover the bug. */ signal(SIGALRM, grace_alarm_handler); if (!debug_flag) alarm(options.login_grace_time); sshd_exchange_identification(sock_in, sock_out); /* In inetd mode, generate ephemeral key only for proto 1 connections */ if (!compat20 && inetd_flag && sensitive_data.server_key == NULL) generate_ephemeral_server_key(); packet_set_nonblocking(); /* allocate authentication context */ authctxt = xcalloc(1, sizeof(*authctxt)); authctxt->loginmsg = &loginmsg; /* XXX global for cleanup, access from other modules */ the_authctxt = authctxt; /* prepare buffer to collect messages to display to user after login */ buffer_init(&loginmsg); auth_debug_reset(); + + BLACKLIST_INIT(); if (use_privsep) { if (privsep_preauth(authctxt) == 1) goto authenticated; } else if (compat20 && have_agent) { if ((r = ssh_get_authentication_socket(&auth_sock)) != 0) { error("Unable to get agent socket: %s", ssh_err(r)); have_agent = 0; } } /* perform the key exchange */ /* authenticate user and start session */ if (compat20) { do_ssh2_kex(); do_authentication2(authctxt); } else { #ifdef WITH_SSH1 do_ssh1_kex(); do_authentication(authctxt); #else fatal("ssh1 not supported"); #endif } /* * If we use privilege separation, the unprivileged child transfers * the current keystate and exits */ if (use_privsep) { mm_send_keystate(pmonitor); exit(0); } authenticated: /* * Cancel the alarm we set to limit the time taken for * authentication. */ alarm(0); signal(SIGALRM, SIG_DFL); authctxt->authenticated = 1; if (startup_pipe != -1) { close(startup_pipe); startup_pipe = -1; } #ifdef SSH_AUDIT_EVENTS audit_event(SSH_AUTH_SUCCESS); #endif #ifdef GSSAPI if (options.gss_authentication) { temporarily_use_uid(authctxt->pw); ssh_gssapi_storecreds(); restore_uid(); } #endif #ifdef USE_PAM if (options.use_pam) { do_pam_setcred(1); do_pam_session(); } #endif /* * In privilege separation, we fork another child and prepare * file descriptor passing. */ if (use_privsep) { privsep_postauth(authctxt); /* the monitor process [priv] will not return */ if (!compat20) destroy_sensitive_data(); } packet_set_timeout(options.client_alive_interval, options.client_alive_count_max); /* Try to send all our hostkeys to the client */ if (compat20) notify_hostkeys(active_state); /* Start session. */ do_authenticated(authctxt); /* The connection has been terminated. */ packet_get_bytes(&ibytes, &obytes); verbose("Transferred: sent %llu, received %llu bytes", (unsigned long long)obytes, (unsigned long long)ibytes); verbose("Closing connection to %.500s port %d", remote_ip, remote_port); #ifdef USE_PAM if (options.use_pam) finish_pam(); #endif /* USE_PAM */ #ifdef SSH_AUDIT_EVENTS PRIVSEP(audit_event(SSH_CONNECTION_CLOSE)); #endif packet_close(); if (use_privsep) mm_terminate(); exit(0); } #ifdef WITH_SSH1 /* * Decrypt session_key_int using our private server key and private host key * (key with larger modulus first). */ int ssh1_session_key(BIGNUM *session_key_int) { int rsafail = 0; if (BN_cmp(sensitive_data.server_key->rsa->n, sensitive_data.ssh1_host_key->rsa->n) > 0) { /* Server key has bigger modulus. */ if (BN_num_bits(sensitive_data.server_key->rsa->n) < BN_num_bits(sensitive_data.ssh1_host_key->rsa->n) + SSH_KEY_BITS_RESERVED) { fatal("do_connection: %s: " "server_key %d < host_key %d + SSH_KEY_BITS_RESERVED %d", get_remote_ipaddr(), BN_num_bits(sensitive_data.server_key->rsa->n), BN_num_bits(sensitive_data.ssh1_host_key->rsa->n), SSH_KEY_BITS_RESERVED); } if (rsa_private_decrypt(session_key_int, session_key_int, sensitive_data.server_key->rsa) != 0) rsafail++; if (rsa_private_decrypt(session_key_int, session_key_int, sensitive_data.ssh1_host_key->rsa) != 0) rsafail++; } else { /* Host key has bigger modulus (or they are equal). */ if (BN_num_bits(sensitive_data.ssh1_host_key->rsa->n) < BN_num_bits(sensitive_data.server_key->rsa->n) + SSH_KEY_BITS_RESERVED) { fatal("do_connection: %s: " "host_key %d < server_key %d + SSH_KEY_BITS_RESERVED %d", get_remote_ipaddr(), BN_num_bits(sensitive_data.ssh1_host_key->rsa->n), BN_num_bits(sensitive_data.server_key->rsa->n), SSH_KEY_BITS_RESERVED); } if (rsa_private_decrypt(session_key_int, session_key_int, sensitive_data.ssh1_host_key->rsa) != 0) rsafail++; if (rsa_private_decrypt(session_key_int, session_key_int, sensitive_data.server_key->rsa) != 0) rsafail++; } return (rsafail); } /* * SSH1 key exchange */ static void do_ssh1_kex(void) { int i, len; int rsafail = 0; BIGNUM *session_key_int, *fake_key_int, *real_key_int; u_char session_key[SSH_SESSION_KEY_LENGTH]; u_char fake_key_bytes[4096 / 8]; size_t fake_key_len; u_char cookie[8]; u_int cipher_type, auth_mask, protocol_flags; /* * Generate check bytes that the client must send back in the user * packet in order for it to be accepted; this is used to defy ip * spoofing attacks. Note that this only works against somebody * doing IP spoofing from a remote machine; any machine on the local * network can still see outgoing packets and catch the random * cookie. This only affects rhosts authentication, and this is one * of the reasons why it is inherently insecure. */ arc4random_buf(cookie, sizeof(cookie)); /* * Send our public key. We include in the packet 64 bits of random * data that must be matched in the reply in order to prevent IP * spoofing. */ packet_start(SSH_SMSG_PUBLIC_KEY); for (i = 0; i < 8; i++) packet_put_char(cookie[i]); /* Store our public server RSA key. */ packet_put_int(BN_num_bits(sensitive_data.server_key->rsa->n)); packet_put_bignum(sensitive_data.server_key->rsa->e); packet_put_bignum(sensitive_data.server_key->rsa->n); /* Store our public host RSA key. */ packet_put_int(BN_num_bits(sensitive_data.ssh1_host_key->rsa->n)); packet_put_bignum(sensitive_data.ssh1_host_key->rsa->e); packet_put_bignum(sensitive_data.ssh1_host_key->rsa->n); /* Put protocol flags. */ packet_put_int(SSH_PROTOFLAG_HOST_IN_FWD_OPEN); /* Declare which ciphers we support. */ packet_put_int(cipher_mask_ssh1(0)); /* Declare supported authentication types. */ auth_mask = 0; if (options.rhosts_rsa_authentication) auth_mask |= 1 << SSH_AUTH_RHOSTS_RSA; if (options.rsa_authentication) auth_mask |= 1 << SSH_AUTH_RSA; if (options.challenge_response_authentication == 1) auth_mask |= 1 << SSH_AUTH_TIS; if (options.password_authentication) auth_mask |= 1 << SSH_AUTH_PASSWORD; packet_put_int(auth_mask); /* Send the packet and wait for it to be sent. */ packet_send(); packet_write_wait(); debug("Sent %d bit server key and %d bit host key.", BN_num_bits(sensitive_data.server_key->rsa->n), BN_num_bits(sensitive_data.ssh1_host_key->rsa->n)); /* Read clients reply (cipher type and session key). */ packet_read_expect(SSH_CMSG_SESSION_KEY); /* Get cipher type and check whether we accept this. */ cipher_type = packet_get_char(); if (!(cipher_mask_ssh1(0) & (1 << cipher_type))) packet_disconnect("Warning: client selects unsupported cipher."); /* Get check bytes from the packet. These must match those we sent earlier with the public key packet. */ for (i = 0; i < 8; i++) if (cookie[i] != packet_get_char()) packet_disconnect("IP Spoofing check bytes do not match."); debug("Encryption type: %.200s", cipher_name(cipher_type)); /* Get the encrypted integer. */ if ((real_key_int = BN_new()) == NULL) fatal("do_ssh1_kex: BN_new failed"); packet_get_bignum(real_key_int); protocol_flags = packet_get_int(); packet_set_protocol_flags(protocol_flags); packet_check_eom(); /* Setup a fake key in case RSA decryption fails */ if ((fake_key_int = BN_new()) == NULL) fatal("do_ssh1_kex: BN_new failed"); fake_key_len = BN_num_bytes(real_key_int); if (fake_key_len > sizeof(fake_key_bytes)) fake_key_len = sizeof(fake_key_bytes); arc4random_buf(fake_key_bytes, fake_key_len); if (BN_bin2bn(fake_key_bytes, fake_key_len, fake_key_int) == NULL) fatal("do_ssh1_kex: BN_bin2bn failed"); /* Decrypt real_key_int using host/server keys */ rsafail = PRIVSEP(ssh1_session_key(real_key_int)); /* If decryption failed, use the fake key. Else, the real key. */ if (rsafail) session_key_int = fake_key_int; else session_key_int = real_key_int; /* * Extract session key from the decrypted integer. The key is in the * least significant 256 bits of the integer; the first byte of the * key is in the highest bits. */ (void) BN_mask_bits(session_key_int, sizeof(session_key) * 8); len = BN_num_bytes(session_key_int); if (len < 0 || (u_int)len > sizeof(session_key)) { error("do_ssh1_kex: bad session key len from %s: " "session_key_int %d > sizeof(session_key) %lu", get_remote_ipaddr(), len, (u_long)sizeof(session_key)); rsafail++; } else { explicit_bzero(session_key, sizeof(session_key)); BN_bn2bin(session_key_int, session_key + sizeof(session_key) - len); derive_ssh1_session_id( sensitive_data.ssh1_host_key->rsa->n, sensitive_data.server_key->rsa->n, cookie, session_id); /* * Xor the first 16 bytes of the session key with the * session id. */ for (i = 0; i < 16; i++) session_key[i] ^= session_id[i]; } /* Destroy the private and public keys. No longer. */ destroy_sensitive_data(); if (use_privsep) mm_ssh1_session_id(session_id); /* Destroy the decrypted integer. It is no longer needed. */ BN_clear_free(real_key_int); BN_clear_free(fake_key_int); /* Set the session key. From this on all communications will be encrypted. */ packet_set_encryption_key(session_key, SSH_SESSION_KEY_LENGTH, cipher_type); /* Destroy our copy of the session key. It is no longer needed. */ explicit_bzero(session_key, sizeof(session_key)); debug("Received session key; encryption turned on."); /* Send an acknowledgment packet. Note that this packet is sent encrypted. */ packet_start(SSH_SMSG_SUCCESS); packet_send(); packet_write_wait(); } #endif int sshd_hostkey_sign(Key *privkey, Key *pubkey, u_char **signature, size_t *slen, const u_char *data, size_t dlen, const char *alg, u_int flag) { int r; u_int xxx_slen, xxx_dlen = dlen; if (privkey) { if (PRIVSEP(key_sign(privkey, signature, &xxx_slen, data, xxx_dlen, alg) < 0)) fatal("%s: key_sign failed", __func__); if (slen) *slen = xxx_slen; } else if (use_privsep) { if (mm_key_sign(pubkey, signature, &xxx_slen, data, xxx_dlen, alg) < 0) fatal("%s: pubkey_sign failed", __func__); if (slen) *slen = xxx_slen; } else { if ((r = ssh_agent_sign(auth_sock, pubkey, signature, slen, data, dlen, alg, datafellows)) != 0) fatal("%s: ssh_agent_sign failed: %s", __func__, ssh_err(r)); } return 0; } /* SSH2 key exchange */ static void do_ssh2_kex(void) { char *myproposal[PROPOSAL_MAX] = { KEX_SERVER }; struct kex *kex; int r; myproposal[PROPOSAL_KEX_ALGS] = compat_kex_proposal( options.kex_algorithms); myproposal[PROPOSAL_ENC_ALGS_CTOS] = compat_cipher_proposal( options.ciphers); myproposal[PROPOSAL_ENC_ALGS_STOC] = compat_cipher_proposal( options.ciphers); myproposal[PROPOSAL_MAC_ALGS_CTOS] = myproposal[PROPOSAL_MAC_ALGS_STOC] = options.macs; if (options.compression == COMP_NONE) { myproposal[PROPOSAL_COMP_ALGS_CTOS] = myproposal[PROPOSAL_COMP_ALGS_STOC] = "none"; } else if (options.compression == COMP_DELAYED) { myproposal[PROPOSAL_COMP_ALGS_CTOS] = myproposal[PROPOSAL_COMP_ALGS_STOC] = "none,zlib@openssh.com"; } if (options.rekey_limit || options.rekey_interval) packet_set_rekey_limits(options.rekey_limit, (time_t)options.rekey_interval); myproposal[PROPOSAL_SERVER_HOST_KEY_ALGS] = compat_pkalg_proposal( list_hostkey_types()); /* start key exchange */ if ((r = kex_setup(active_state, myproposal)) != 0) fatal("kex_setup: %s", ssh_err(r)); kex = active_state->kex; #ifdef WITH_OPENSSL kex->kex[KEX_DH_GRP1_SHA1] = kexdh_server; kex->kex[KEX_DH_GRP14_SHA1] = kexdh_server; kex->kex[KEX_DH_GEX_SHA1] = kexgex_server; kex->kex[KEX_DH_GEX_SHA256] = kexgex_server; # ifdef OPENSSL_HAS_ECC kex->kex[KEX_ECDH_SHA2] = kexecdh_server; # endif #endif kex->kex[KEX_C25519_SHA256] = kexc25519_server; kex->server = 1; kex->client_version_string=client_version_string; kex->server_version_string=server_version_string; kex->load_host_public_key=&get_hostkey_public_by_type; kex->load_host_private_key=&get_hostkey_private_by_type; kex->host_key_index=&get_hostkey_index; kex->sign = sshd_hostkey_sign; dispatch_run(DISPATCH_BLOCK, &kex->done, active_state); session_id2 = kex->session_id; session_id2_len = kex->session_id_len; #ifdef DEBUG_KEXDH /* send 1st encrypted/maced/compressed message */ packet_start(SSH2_MSG_IGNORE); packet_put_cstring("markus"); packet_send(); packet_write_wait(); #endif debug("KEX done"); } /* server specific fatal cleanup */ void cleanup_exit(int i) { if (the_authctxt) { do_cleanup(the_authctxt); if (use_privsep && privsep_is_preauth && pmonitor != NULL && pmonitor->m_pid > 1) { debug("Killing privsep child %d", pmonitor->m_pid); if (kill(pmonitor->m_pid, SIGKILL) != 0 && errno != ESRCH) error("%s: kill(%d): %s", __func__, pmonitor->m_pid, strerror(errno)); } } #ifdef SSH_AUDIT_EVENTS /* done after do_cleanup so it can cancel the PAM auth 'thread' */ if (!use_privsep || mm_is_monitor()) audit_event(SSH_CONNECTION_ABANDON); #endif _exit(i); } Index: projects/netbsd-tests-update-12/crypto/openssh/sshd_config =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/sshd_config (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/sshd_config (revision 305172) @@ -1,136 +1,137 @@ # $OpenBSD: sshd_config,v 1.98 2016/02/17 05:29:04 djm Exp $ # $FreeBSD$ # This is the sshd server system-wide configuration file. See # sshd_config(5) for more information. # This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin # The strategy used for options in the default sshd_config shipped with # OpenSSH is to specify options with their default value where # possible, but leave them commented. Uncommented options override the # default value. # Note that some of FreeBSD's defaults differ from OpenBSD's, and # FreeBSD has a few additional options. #Port 22 #AddressFamily any #ListenAddress 0.0.0.0 #ListenAddress :: # The default requires explicit activation of protocol 1 #Protocol 2 # HostKey for protocol version 1 #HostKey /etc/ssh/ssh_host_key # HostKeys for protocol version 2 #HostKey /etc/ssh/ssh_host_rsa_key #HostKey /etc/ssh/ssh_host_dsa_key #HostKey /etc/ssh/ssh_host_ecdsa_key #HostKey /etc/ssh/ssh_host_ed25519_key # Lifetime and size of ephemeral version 1 server key #KeyRegenerationInterval 1h #ServerKeyBits 1024 # Ciphers and keying #RekeyLimit default none # Logging # obsoletes QuietMode and FascistLogging #SyslogFacility AUTH #LogLevel INFO # Authentication: #LoginGraceTime 2m #PermitRootLogin no #StrictModes yes #MaxAuthTries 6 #MaxSessions 10 #RSAAuthentication yes #PubkeyAuthentication yes # The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2 #AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2 #AuthorizedPrincipalsFile none #AuthorizedKeysCommand none #AuthorizedKeysCommandUser nobody # For this to work you will also need host keys in /etc/ssh/ssh_known_hosts #RhostsRSAAuthentication no # similar for protocol version 2 #HostbasedAuthentication no # Change to yes if you don't trust ~/.ssh/known_hosts for # RhostsRSAAuthentication and HostbasedAuthentication #IgnoreUserKnownHosts no # Don't read the user's ~/.rhosts and ~/.shosts files #IgnoreRhosts yes # Change to yes to enable built-in password authentication. #PasswordAuthentication no #PermitEmptyPasswords no # Change to no to disable PAM authentication #ChallengeResponseAuthentication yes # Kerberos options #KerberosAuthentication no #KerberosOrLocalPasswd yes #KerberosTicketCleanup yes #KerberosGetAFSToken no # GSSAPI options #GSSAPIAuthentication no #GSSAPICleanupCredentials yes # Set this to 'no' to disable PAM authentication, account processing, # and session processing. If this is enabled, PAM authentication will # be allowed through the ChallengeResponseAuthentication and # PasswordAuthentication. Depending on your PAM configuration, # PAM authentication via ChallengeResponseAuthentication may bypass # the setting of "PermitRootLogin without-password". # If you just want the PAM account and session checks to run without # PAM authentication, then enable this but set PasswordAuthentication # and ChallengeResponseAuthentication to 'no'. #UsePAM yes #AllowAgentForwarding yes #AllowTcpForwarding yes #GatewayPorts no #X11Forwarding yes #X11DisplayOffset 10 #X11UseLocalhost yes #PermitTTY yes #PrintMotd yes #PrintLastLog yes #TCPKeepAlive yes #UseLogin no #UsePrivilegeSeparation sandbox #PermitUserEnvironment no #Compression delayed #ClientAliveInterval 0 #ClientAliveCountMax 3 #UseDNS yes #PidFile /var/run/sshd.pid #MaxStartups 10:30:100 #PermitTunnel no #ChrootDirectory none +#UseBlacklist no #VersionAddendum FreeBSD-20160310 # no default banner path #Banner none # override default of no subsystems Subsystem sftp /usr/libexec/sftp-server # Example of overriding settings on a per-user basis #Match User anoncvs # X11Forwarding no # AllowTcpForwarding no # PermitTTY no # ForceCommand cvs server Index: projects/netbsd-tests-update-12/crypto/openssh/sshd_config.5 =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh/sshd_config.5 (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh/sshd_config.5 (revision 305172) @@ -1,1767 +1,1776 @@ .\" .\" Author: Tatu Ylonen .\" Copyright (c) 1995 Tatu Ylonen , Espoo, Finland .\" All rights reserved .\" .\" As far as I am concerned, the code I have written for this software .\" can be used freely for any purpose. Any derived versions of this .\" software must be clearly marked as such, and if the derived work is .\" incompatible with the protocol description in the RFC file, it must be .\" called by a name other than "ssh" or "Secure Shell". .\" .\" Copyright (c) 1999,2000 Markus Friedl. All rights reserved. .\" Copyright (c) 1999 Aaron Campbell. All rights reserved. .\" Copyright (c) 1999 Theo de Raadt. All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR .\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES .\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. .\" IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, .\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT .\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF .\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .\" .\" $OpenBSD: sshd_config.5,v 1.220 2016/02/17 08:57:34 djm Exp $ .\" $FreeBSD$ .Dd $Mdocdate: February 17 2016 $ .Dt SSHD_CONFIG 5 .Os .Sh NAME .Nm sshd_config .Nd OpenSSH SSH daemon configuration file .Sh SYNOPSIS .Nm /etc/ssh/sshd_config .Sh DESCRIPTION .Xr sshd 8 reads configuration data from .Pa /etc/ssh/sshd_config (or the file specified with .Fl f on the command line). The file contains keyword-argument pairs, one per line. Lines starting with .Ql # and empty lines are interpreted as comments. Arguments may optionally be enclosed in double quotes .Pq \&" in order to represent arguments containing spaces. .Pp The possible keywords and their meanings are as follows (note that keywords are case-insensitive and arguments are case-sensitive): .Bl -tag -width Ds .It Cm AcceptEnv Specifies what environment variables sent by the client will be copied into the session's .Xr environ 7 . See .Cm SendEnv in .Xr ssh_config 5 for how to configure the client. The .Ev TERM environment variable is always sent whenever the client requests a pseudo-terminal as it is required by the protocol. Variables are specified by name, which may contain the wildcard characters .Ql * and .Ql \&? . Multiple environment variables may be separated by whitespace or spread across multiple .Cm AcceptEnv directives. Be warned that some environment variables could be used to bypass restricted user environments. For this reason, care should be taken in the use of this directive. The default is not to accept any environment variables. .It Cm AddressFamily Specifies which address family should be used by .Xr sshd 8 . Valid arguments are .Dq any , .Dq inet (use IPv4 only), or .Dq inet6 (use IPv6 only). The default is .Dq any . .It Cm AllowAgentForwarding Specifies whether .Xr ssh-agent 1 forwarding is permitted. The default is .Dq yes . Note that disabling agent forwarding does not improve security unless users are also denied shell access, as they can always install their own forwarders. .It Cm AllowGroups This keyword can be followed by a list of group name patterns, separated by spaces. If specified, login is allowed only for users whose primary group or supplementary group list matches one of the patterns. Only group names are valid; a numerical group ID is not recognized. By default, login is allowed for all groups. The allow/deny directives are processed in the following order: .Cm DenyUsers , .Cm AllowUsers , .Cm DenyGroups , and finally .Cm AllowGroups . .Pp See PATTERNS in .Xr ssh_config 5 for more information on patterns. .It Cm AllowTcpForwarding Specifies whether TCP forwarding is permitted. The available options are .Dq yes or .Dq all to allow TCP forwarding, .Dq no to prevent all TCP forwarding, .Dq local to allow local (from the perspective of .Xr ssh 1 ) forwarding only or .Dq remote to allow remote forwarding only. The default is .Dq yes . Note that disabling TCP forwarding does not improve security unless users are also denied shell access, as they can always install their own forwarders. .It Cm AllowStreamLocalForwarding Specifies whether StreamLocal (Unix-domain socket) forwarding is permitted. The available options are .Dq yes or .Dq all to allow StreamLocal forwarding, .Dq no to prevent all StreamLocal forwarding, .Dq local to allow local (from the perspective of .Xr ssh 1 ) forwarding only or .Dq remote to allow remote forwarding only. The default is .Dq yes . Note that disabling StreamLocal forwarding does not improve security unless users are also denied shell access, as they can always install their own forwarders. .It Cm AllowUsers This keyword can be followed by a list of user name patterns, separated by spaces. If specified, login is allowed only for user names that match one of the patterns. Only user names are valid; a numerical user ID is not recognized. By default, login is allowed for all users. If the pattern takes the form USER@HOST then USER and HOST are separately checked, restricting logins to particular users from particular hosts. The allow/deny directives are processed in the following order: .Cm DenyUsers , .Cm AllowUsers , .Cm DenyGroups , and finally .Cm AllowGroups . .Pp See PATTERNS in .Xr ssh_config 5 for more information on patterns. .It Cm AuthenticationMethods Specifies the authentication methods that must be successfully completed for a user to be granted access. This option must be followed by one or more comma-separated lists of authentication method names. Successful authentication requires completion of every method in at least one of these lists. .Pp For example, an argument of .Dq publickey,password publickey,keyboard-interactive would require the user to complete public key authentication, followed by either password or keyboard interactive authentication. Only methods that are next in one or more lists are offered at each stage, so for this example, it would not be possible to attempt password or keyboard-interactive authentication before public key. .Pp For keyboard interactive authentication it is also possible to restrict authentication to a specific device by appending a colon followed by the device identifier .Dq bsdauth , .Dq pam , or .Dq skey , depending on the server configuration. For example, .Dq keyboard-interactive:bsdauth would restrict keyboard interactive authentication to the .Dq bsdauth device. .Pp If the .Dq publickey method is listed more than once, .Xr sshd 8 verifies that keys that have been used successfully are not reused for subsequent authentications. For example, an .Cm AuthenticationMethods of .Dq publickey,publickey will require successful authentication using two different public keys. .Pp This option will yield a fatal error if enabled if protocol 1 is also enabled. Note that each authentication method listed should also be explicitly enabled in the configuration. The default is not to require multiple authentication; successful completion of a single authentication method is sufficient. .It Cm AuthorizedKeysCommand Specifies a program to be used to look up the user's public keys. The program must be owned by root, not writable by group or others and specified by an absolute path. .Pp Arguments to .Cm AuthorizedKeysCommand may be provided using the following tokens, which will be expanded at runtime: %% is replaced by a literal '%', %u is replaced by the username being authenticated, %h is replaced by the home directory of the user being authenticated, %t is replaced with the key type offered for authentication, %f is replaced with the fingerprint of the key, and %k is replaced with the key being offered for authentication. If no arguments are specified then the username of the target user will be supplied. .Pp The program should produce on standard output zero or more lines of authorized_keys output (see AUTHORIZED_KEYS in .Xr sshd 8 ) . If a key supplied by AuthorizedKeysCommand does not successfully authenticate and authorize the user then public key authentication continues using the usual .Cm AuthorizedKeysFile files. By default, no AuthorizedKeysCommand is run. .It Cm AuthorizedKeysCommandUser Specifies the user under whose account the AuthorizedKeysCommand is run. It is recommended to use a dedicated user that has no other role on the host than running authorized keys commands. If .Cm AuthorizedKeysCommand is specified but .Cm AuthorizedKeysCommandUser is not, then .Xr sshd 8 will refuse to start. .It Cm AuthorizedKeysFile Specifies the file that contains the public keys that can be used for user authentication. The format is described in the AUTHORIZED_KEYS FILE FORMAT section of .Xr sshd 8 . .Cm AuthorizedKeysFile may contain tokens of the form %T which are substituted during connection setup. The following tokens are defined: %% is replaced by a literal '%', %h is replaced by the home directory of the user being authenticated, and %u is replaced by the username of that user. After expansion, .Cm AuthorizedKeysFile is taken to be an absolute path or one relative to the user's home directory. Multiple files may be listed, separated by whitespace. Alternately this option may be set to .Dq none to skip checking for user keys in files. The default is .Dq .ssh/authorized_keys .ssh/authorized_keys2 . .It Cm AuthorizedPrincipalsCommand Specifies a program to be used to generate the list of allowed certificate principals as per .Cm AuthorizedPrincipalsFile . The program must be owned by root, not writable by group or others and specified by an absolute path. .Pp Arguments to .Cm AuthorizedPrincipalsCommand may be provided using the following tokens, which will be expanded at runtime: %% is replaced by a literal '%', %u is replaced by the username being authenticated and %h is replaced by the home directory of the user being authenticated. .Pp The program should produce on standard output zero or more lines of .Cm AuthorizedPrincipalsFile output. If either .Cm AuthorizedPrincipalsCommand or .Cm AuthorizedPrincipalsFile is specified, then certificates offered by the client for authentication must contain a principal that is listed. By default, no AuthorizedPrincipalsCommand is run. .It Cm AuthorizedPrincipalsCommandUser Specifies the user under whose account the AuthorizedPrincipalsCommand is run. It is recommended to use a dedicated user that has no other role on the host than running authorized principals commands. If .Cm AuthorizedPrincipalsCommand is specified but .Cm AuthorizedPrincipalsCommandUser is not, then .Xr sshd 8 will refuse to start. .It Cm AuthorizedPrincipalsFile Specifies a file that lists principal names that are accepted for certificate authentication. When using certificates signed by a key listed in .Cm TrustedUserCAKeys , this file lists names, one of which must appear in the certificate for it to be accepted for authentication. Names are listed one per line preceded by key options (as described in AUTHORIZED_KEYS FILE FORMAT in .Xr sshd 8 ) . Empty lines and comments starting with .Ql # are ignored. .Pp .Cm AuthorizedPrincipalsFile may contain tokens of the form %T which are substituted during connection setup. The following tokens are defined: %% is replaced by a literal '%', %h is replaced by the home directory of the user being authenticated, and %u is replaced by the username of that user. After expansion, .Cm AuthorizedPrincipalsFile is taken to be an absolute path or one relative to the user's home directory. .Pp The default is .Dq none , i.e. not to use a principals file \(en in this case, the username of the user must appear in a certificate's principals list for it to be accepted. Note that .Cm AuthorizedPrincipalsFile is only used when authentication proceeds using a CA listed in .Cm TrustedUserCAKeys and is not consulted for certification authorities trusted via .Pa ~/.ssh/authorized_keys , though the .Cm principals= key option offers a similar facility (see .Xr sshd 8 for details). .It Cm Banner The contents of the specified file are sent to the remote user before authentication is allowed. If the argument is .Dq none then no banner is displayed. By default, no banner is displayed. .It Cm ChallengeResponseAuthentication Specifies whether challenge-response authentication is allowed (e.g. via PAM or through authentication styles supported in .Xr login.conf 5 ) The default is .Dq yes . .It Cm ChrootDirectory Specifies the pathname of a directory to .Xr chroot 2 to after authentication. At session startup .Xr sshd 8 checks that all components of the pathname are root-owned directories which are not writable by any other user or group. After the chroot, .Xr sshd 8 changes the working directory to the user's home directory. .Pp The pathname may contain the following tokens that are expanded at runtime once the connecting user has been authenticated: %% is replaced by a literal '%', %h is replaced by the home directory of the user being authenticated, and %u is replaced by the username of that user. .Pp The .Cm ChrootDirectory must contain the necessary files and directories to support the user's session. For an interactive session this requires at least a shell, typically .Xr sh 1 , and basic .Pa /dev nodes such as .Xr null 4 , .Xr zero 4 , .Xr stdin 4 , .Xr stdout 4 , .Xr stderr 4 , and .Xr tty 4 devices. For file transfer sessions using .Dq sftp , no additional configuration of the environment is necessary if the in-process sftp server is used, though sessions which use logging may require .Pa /dev/log inside the chroot directory on some operating systems (see .Xr sftp-server 8 for details). .Pp For safety, it is very important that the directory hierarchy be prevented from modification by other processes on the system (especially those outside the jail). Misconfiguration can lead to unsafe environments which .Xr sshd 8 cannot detect. .Pp The default is .Dq none , indicating not to .Xr chroot 2 . .It Cm Ciphers Specifies the ciphers allowed. Multiple ciphers must be comma-separated. If the specified value begins with a .Sq + character, then the specified ciphers will be appended to the default set instead of replacing them. .Pp The supported ciphers are: .Pp .Bl -item -compact -offset indent .It 3des-cbc .It aes128-cbc .It aes192-cbc .It aes256-cbc .It aes128-ctr .It aes192-ctr .It aes256-ctr .It aes128-gcm@openssh.com .It aes256-gcm@openssh.com .It arcfour .It arcfour128 .It arcfour256 .It blowfish-cbc .It cast128-cbc .It chacha20-poly1305@openssh.com .El .Pp The default is: .Bd -literal -offset indent chacha20-poly1305@openssh.com, aes128-ctr,aes192-ctr,aes256-ctr, aes128-gcm@openssh.com,aes256-gcm@openssh.com, aes128-cbc,aes192-cbc,aes256-cbc .Ed .Pp The list of available ciphers may also be obtained using the .Fl Q option of .Xr ssh 1 with an argument of .Dq cipher . .It Cm ClientAliveCountMax Sets the number of client alive messages (see below) which may be sent without .Xr sshd 8 receiving any messages back from the client. If this threshold is reached while client alive messages are being sent, sshd will disconnect the client, terminating the session. It is important to note that the use of client alive messages is very different from .Cm TCPKeepAlive (below). The client alive messages are sent through the encrypted channel and therefore will not be spoofable. The TCP keepalive option enabled by .Cm TCPKeepAlive is spoofable. The client alive mechanism is valuable when the client or server depend on knowing when a connection has become inactive. .Pp The default value is 3. If .Cm ClientAliveInterval (see below) is set to 15, and .Cm ClientAliveCountMax is left at the default, unresponsive SSH clients will be disconnected after approximately 45 seconds. .It Cm ClientAliveInterval Sets a timeout interval in seconds after which if no data has been received from the client, .Xr sshd 8 will send a message through the encrypted channel to request a response from the client. The default is 0, indicating that these messages will not be sent to the client. .It Cm Compression Specifies whether compression is allowed, or delayed until the user has authenticated successfully. The argument must be .Dq yes , .Dq delayed , or .Dq no . The default is .Dq delayed . .It Cm DenyGroups This keyword can be followed by a list of group name patterns, separated by spaces. Login is disallowed for users whose primary group or supplementary group list matches one of the patterns. Only group names are valid; a numerical group ID is not recognized. By default, login is allowed for all groups. The allow/deny directives are processed in the following order: .Cm DenyUsers , .Cm AllowUsers , .Cm DenyGroups , and finally .Cm AllowGroups . .Pp See PATTERNS in .Xr ssh_config 5 for more information on patterns. .It Cm DenyUsers This keyword can be followed by a list of user name patterns, separated by spaces. Login is disallowed for user names that match one of the patterns. Only user names are valid; a numerical user ID is not recognized. By default, login is allowed for all users. If the pattern takes the form USER@HOST then USER and HOST are separately checked, restricting logins to particular users from particular hosts. The allow/deny directives are processed in the following order: .Cm DenyUsers , .Cm AllowUsers , .Cm DenyGroups , and finally .Cm AllowGroups . .Pp See PATTERNS in .Xr ssh_config 5 for more information on patterns. .It Cm FingerprintHash Specifies the hash algorithm used when logging key fingerprints. Valid options are: .Dq md5 and .Dq sha256 . The default is .Dq sha256 . .It Cm ForceCommand Forces the execution of the command specified by .Cm ForceCommand , ignoring any command supplied by the client and .Pa ~/.ssh/rc if present. The command is invoked by using the user's login shell with the -c option. This applies to shell, command, or subsystem execution. It is most useful inside a .Cm Match block. The command originally supplied by the client is available in the .Ev SSH_ORIGINAL_COMMAND environment variable. Specifying a command of .Dq internal-sftp will force the use of an in-process sftp server that requires no support files when used with .Cm ChrootDirectory . The default is .Dq none . .It Cm GatewayPorts Specifies whether remote hosts are allowed to connect to ports forwarded for the client. By default, .Xr sshd 8 binds remote port forwardings to the loopback address. This prevents other remote hosts from connecting to forwarded ports. .Cm GatewayPorts can be used to specify that sshd should allow remote port forwardings to bind to non-loopback addresses, thus allowing other hosts to connect. The argument may be .Dq no to force remote port forwardings to be available to the local host only, .Dq yes to force remote port forwardings to bind to the wildcard address, or .Dq clientspecified to allow the client to select the address to which the forwarding is bound. The default is .Dq no . .It Cm GSSAPIAuthentication Specifies whether user authentication based on GSSAPI is allowed. The default is .Dq no . .It Cm GSSAPICleanupCredentials Specifies whether to automatically destroy the user's credentials cache on logout. The default is .Dq yes . .It Cm GSSAPIStrictAcceptorCheck Determines whether to be strict about the identity of the GSSAPI acceptor a client authenticates against. If set to .Dq yes then the client must authenticate against the .Pa host service on the current hostname. If set to .Dq no then the client may authenticate against any service key stored in the machine's default store. This facility is provided to assist with operation on multi homed machines. The default is .Dq yes . .It Cm HostbasedAcceptedKeyTypes Specifies the key types that will be accepted for hostbased authentication as a comma-separated pattern list. Alternately if the specified value begins with a .Sq + character, then the specified key types will be appended to the default set instead of replacing them. The default for this option is: .Bd -literal -offset 3n ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521-cert-v01@openssh.com, ssh-ed25519-cert-v01@openssh.com, ssh-rsa-cert-v01@openssh.com, ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521, ssh-ed25519,ssh-rsa .Ed .Pp The .Fl Q option of .Xr ssh 1 may be used to list supported key types. .It Cm HostbasedAuthentication Specifies whether rhosts or /etc/hosts.equiv authentication together with successful public key client host authentication is allowed (host-based authentication). The default is .Dq no . .It Cm HostbasedUsesNameFromPacketOnly Specifies whether or not the server will attempt to perform a reverse name lookup when matching the name in the .Pa ~/.shosts , .Pa ~/.rhosts , and .Pa /etc/hosts.equiv files during .Cm HostbasedAuthentication . A setting of .Dq yes means that .Xr sshd 8 uses the name supplied by the client rather than attempting to resolve the name from the TCP connection itself. The default is .Dq no . .It Cm HostCertificate Specifies a file containing a public host certificate. The certificate's public key must match a private host key already specified by .Cm HostKey . The default behaviour of .Xr sshd 8 is not to load any certificates. .It Cm HostKey Specifies a file containing a private host key used by SSH. The default is .Pa /etc/ssh/ssh_host_key for protocol version 1, and .Pa /etc/ssh/ssh_host_dsa_key , .Pa /etc/ssh/ssh_host_ecdsa_key , .Pa /etc/ssh/ssh_host_ed25519_key and .Pa /etc/ssh/ssh_host_rsa_key for protocol version 2. .Pp Note that .Xr sshd 8 will refuse to use a file if it is group/world-accessible and that the .Cm HostKeyAlgorithms option restricts which of the keys are actually used by .Xr sshd 8 . .Pp It is possible to have multiple host key files. .Dq rsa1 keys are used for version 1 and .Dq dsa , .Dq ecdsa , .Dq ed25519 or .Dq rsa are used for version 2 of the SSH protocol. It is also possible to specify public host key files instead. In this case operations on the private key will be delegated to an .Xr ssh-agent 1 . .It Cm HostKeyAgent Identifies the UNIX-domain socket used to communicate with an agent that has access to the private host keys. If .Dq SSH_AUTH_SOCK is specified, the location of the socket will be read from the .Ev SSH_AUTH_SOCK environment variable. .It Cm HostKeyAlgorithms Specifies the host key algorithms that the server offers. The default for this option is: .Bd -literal -offset 3n ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521-cert-v01@openssh.com, ssh-ed25519-cert-v01@openssh.com, ssh-rsa-cert-v01@openssh.com, ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521, ssh-ed25519,ssh-rsa .Ed .Pp The list of available key types may also be obtained using the .Fl Q option of .Xr ssh 1 with an argument of .Dq key . .It Cm IgnoreRhosts Specifies that .Pa .rhosts and .Pa .shosts files will not be used in .Cm RhostsRSAAuthentication or .Cm HostbasedAuthentication . .Pp .Pa /etc/hosts.equiv and .Pa /etc/ssh/shosts.equiv are still used. The default is .Dq yes . .It Cm IgnoreUserKnownHosts Specifies whether .Xr sshd 8 should ignore the user's .Pa ~/.ssh/known_hosts during .Cm RhostsRSAAuthentication or .Cm HostbasedAuthentication . The default is .Dq no . .It Cm IPQoS Specifies the IPv4 type-of-service or DSCP class for the connection. Accepted values are .Dq af11 , .Dq af12 , .Dq af13 , .Dq af21 , .Dq af22 , .Dq af23 , .Dq af31 , .Dq af32 , .Dq af33 , .Dq af41 , .Dq af42 , .Dq af43 , .Dq cs0 , .Dq cs1 , .Dq cs2 , .Dq cs3 , .Dq cs4 , .Dq cs5 , .Dq cs6 , .Dq cs7 , .Dq ef , .Dq lowdelay , .Dq throughput , .Dq reliability , or a numeric value. This option may take one or two arguments, separated by whitespace. If one argument is specified, it is used as the packet class unconditionally. If two values are specified, the first is automatically selected for interactive sessions and the second for non-interactive sessions. The default is .Dq lowdelay for interactive sessions and .Dq throughput for non-interactive sessions. .It Cm KbdInteractiveAuthentication Specifies whether to allow keyboard-interactive authentication. The argument to this keyword must be .Dq yes or .Dq no . The default is to use whatever value .Cm ChallengeResponseAuthentication is set to (by default .Dq yes ) . .It Cm KerberosAuthentication Specifies whether the password provided by the user for .Cm PasswordAuthentication will be validated through the Kerberos KDC. To use this option, the server needs a Kerberos servtab which allows the verification of the KDC's identity. The default is .Dq no . .It Cm KerberosGetAFSToken If AFS is active and the user has a Kerberos 5 TGT, attempt to acquire an AFS token before accessing the user's home directory. The default is .Dq no . .It Cm KerberosOrLocalPasswd If password authentication through Kerberos fails then the password will be validated via any additional local mechanism such as .Pa /etc/passwd . The default is .Dq yes . .It Cm KerberosTicketCleanup Specifies whether to automatically destroy the user's ticket cache file on logout. The default is .Dq yes . .It Cm KexAlgorithms Specifies the available KEX (Key Exchange) algorithms. Multiple algorithms must be comma-separated. Alternately if the specified value begins with a .Sq + character, then the specified methods will be appended to the default set instead of replacing them. The supported algorithms are: .Pp .Bl -item -compact -offset indent .It curve25519-sha256@libssh.org .It diffie-hellman-group1-sha1 .It diffie-hellman-group14-sha1 .It diffie-hellman-group-exchange-sha1 .It diffie-hellman-group-exchange-sha256 .It ecdh-sha2-nistp256 .It ecdh-sha2-nistp384 .It ecdh-sha2-nistp521 .El .Pp The default is: .Bd -literal -offset indent curve25519-sha256@libssh.org, ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521, diffie-hellman-group-exchange-sha256, diffie-hellman-group14-sha1 .Ed .Pp The list of available key exchange algorithms may also be obtained using the .Fl Q option of .Xr ssh 1 with an argument of .Dq kex . .It Cm KeyRegenerationInterval In protocol version 1, the ephemeral server key is automatically regenerated after this many seconds (if it has been used). The purpose of regeneration is to prevent decrypting captured sessions by later breaking into the machine and stealing the keys. The key is never stored anywhere. If the value is 0, the key is never regenerated. The default is 3600 (seconds). .It Cm ListenAddress Specifies the local addresses .Xr sshd 8 should listen on. The following forms may be used: .Pp .Bl -item -offset indent -compact .It .Cm ListenAddress .Sm off .Ar host | Ar IPv4_addr | Ar IPv6_addr .Sm on .It .Cm ListenAddress .Sm off .Ar host | Ar IPv4_addr : Ar port .Sm on .It .Cm ListenAddress .Sm off .Oo .Ar host | Ar IPv6_addr Oc : Ar port .Sm on .El .Pp If .Ar port is not specified, sshd will listen on the address and all .Cm Port options specified. The default is to listen on all local addresses. Multiple .Cm ListenAddress options are permitted. .It Cm LoginGraceTime The server disconnects after this time if the user has not successfully logged in. If the value is 0, there is no time limit. The default is 120 seconds. .It Cm LogLevel Gives the verbosity level that is used when logging messages from .Xr sshd 8 . The possible values are: QUIET, FATAL, ERROR, INFO, VERBOSE, DEBUG, DEBUG1, DEBUG2, and DEBUG3. The default is INFO. DEBUG and DEBUG1 are equivalent. DEBUG2 and DEBUG3 each specify higher levels of debugging output. Logging with a DEBUG level violates the privacy of users and is not recommended. .It Cm MACs Specifies the available MAC (message authentication code) algorithms. The MAC algorithm is used for data integrity protection. Multiple algorithms must be comma-separated. If the specified value begins with a .Sq + character, then the specified algorithms will be appended to the default set instead of replacing them. .Pp The algorithms that contain .Dq -etm calculate the MAC after encryption (encrypt-then-mac). These are considered safer and their use recommended. The supported MACs are: .Pp .Bl -item -compact -offset indent .It hmac-md5 .It hmac-md5-96 .It hmac-ripemd160 .It hmac-sha1 .It hmac-sha1-96 .It hmac-sha2-256 .It hmac-sha2-512 .It umac-64@openssh.com .It umac-128@openssh.com .It hmac-md5-etm@openssh.com .It hmac-md5-96-etm@openssh.com .It hmac-ripemd160-etm@openssh.com .It hmac-sha1-etm@openssh.com .It hmac-sha1-96-etm@openssh.com .It hmac-sha2-256-etm@openssh.com .It hmac-sha2-512-etm@openssh.com .It umac-64-etm@openssh.com .It umac-128-etm@openssh.com .El .Pp The default is: .Bd -literal -offset indent umac-64-etm@openssh.com,umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com,umac-128@openssh.com, hmac-sha2-256,hmac-sha2-512,hmac-sha1 .Ed .Pp The list of available MAC algorithms may also be obtained using the .Fl Q option of .Xr ssh 1 with an argument of .Dq mac . .It Cm Match Introduces a conditional block. If all of the criteria on the .Cm Match line are satisfied, the keywords on the following lines override those set in the global section of the config file, until either another .Cm Match line or the end of the file. If a keyword appears in multiple .Cm Match blocks that are satisfied, only the first instance of the keyword is applied. .Pp The arguments to .Cm Match are one or more criteria-pattern pairs or the single token .Cm All which matches all criteria. The available criteria are .Cm User , .Cm Group , .Cm Host , .Cm LocalAddress , .Cm LocalPort , and .Cm Address . The match patterns may consist of single entries or comma-separated lists and may use the wildcard and negation operators described in the PATTERNS section of .Xr ssh_config 5 . .Pp The patterns in an .Cm Address criteria may additionally contain addresses to match in CIDR address/masklen format, e.g.\& .Dq 192.0.2.0/24 or .Dq 3ffe:ffff::/32 . Note that the mask length provided must be consistent with the address - it is an error to specify a mask length that is too long for the address or one with bits set in this host portion of the address. For example, .Dq 192.0.2.0/33 and .Dq 192.0.2.0/8 respectively. .Pp Only a subset of keywords may be used on the lines following a .Cm Match keyword. Available keywords are .Cm AcceptEnv , .Cm AllowAgentForwarding , .Cm AllowGroups , .Cm AllowStreamLocalForwarding , .Cm AllowTcpForwarding , .Cm AllowUsers , .Cm AuthenticationMethods , .Cm AuthorizedKeysCommand , .Cm AuthorizedKeysCommandUser , .Cm AuthorizedKeysFile , .Cm AuthorizedPrincipalsCommand , .Cm AuthorizedPrincipalsCommandUser , .Cm AuthorizedPrincipalsFile , .Cm Banner , .Cm ChrootDirectory , .Cm DenyGroups , .Cm DenyUsers , .Cm ForceCommand , .Cm GatewayPorts , .Cm GSSAPIAuthentication , .Cm HostbasedAcceptedKeyTypes , .Cm HostbasedAuthentication , .Cm HostbasedUsesNameFromPacketOnly , .Cm IPQoS , .Cm KbdInteractiveAuthentication , .Cm KerberosAuthentication , .Cm MaxAuthTries , .Cm MaxSessions , .Cm PasswordAuthentication , .Cm PermitEmptyPasswords , .Cm PermitOpen , .Cm PermitRootLogin , .Cm PermitTTY , .Cm PermitTunnel , .Cm PermitUserRC , .Cm PubkeyAcceptedKeyTypes , .Cm PubkeyAuthentication , .Cm RekeyLimit , .Cm RevokedKeys , .Cm RhostsRSAAuthentication , .Cm RSAAuthentication , .Cm StreamLocalBindMask , .Cm StreamLocalBindUnlink , .Cm TrustedUserCAKeys , .Cm X11DisplayOffset , .Cm X11Forwarding and .Cm X11UseLocalHost . .It Cm MaxAuthTries Specifies the maximum number of authentication attempts permitted per connection. Once the number of failures reaches half this value, additional failures are logged. The default is 6. .It Cm MaxSessions Specifies the maximum number of open shell, login or subsystem (e.g. sftp) sessions permitted per network connection. Multiple sessions may be established by clients that support connection multiplexing. Setting .Cm MaxSessions to 1 will effectively disable session multiplexing, whereas setting it to 0 will prevent all shell, login and subsystem sessions while still permitting forwarding. The default is 10. .It Cm MaxStartups Specifies the maximum number of concurrent unauthenticated connections to the SSH daemon. Additional connections will be dropped until authentication succeeds or the .Cm LoginGraceTime expires for a connection. The default is 10:30:100. .Pp Alternatively, random early drop can be enabled by specifying the three colon separated values .Dq start:rate:full (e.g. "10:30:60"). .Xr sshd 8 will refuse connection attempts with a probability of .Dq rate/100 (30%) if there are currently .Dq start (10) unauthenticated connections. The probability increases linearly and all connection attempts are refused if the number of unauthenticated connections reaches .Dq full (60). .It Cm PasswordAuthentication Specifies whether password authentication is allowed. See also .Cm UsePAM . The default is .Dq no . .It Cm PermitEmptyPasswords When password authentication is allowed, it specifies whether the server allows login to accounts with empty password strings. The default is .Dq no . .It Cm PermitOpen Specifies the destinations to which TCP port forwarding is permitted. The forwarding specification must be one of the following forms: .Pp .Bl -item -offset indent -compact .It .Cm PermitOpen .Sm off .Ar host : port .Sm on .It .Cm PermitOpen .Sm off .Ar IPv4_addr : port .Sm on .It .Cm PermitOpen .Sm off .Ar \&[ IPv6_addr \&] : port .Sm on .El .Pp Multiple forwards may be specified by separating them with whitespace. An argument of .Dq any can be used to remove all restrictions and permit any forwarding requests. An argument of .Dq none can be used to prohibit all forwarding requests. By default all port forwarding requests are permitted. .It Cm PermitRootLogin Specifies whether root can log in using .Xr ssh 1 . The argument must be .Dq yes , .Dq prohibit-password , .Dq without-password , .Dq forced-commands-only , or .Dq no . The default is .Dq no . Note that if .Cm ChallengeResponseAuthentication is .Dq yes , the root user may be allowed in with its password even if .Cm PermitRootLogin is set to .Dq without-password . .Pp If this option is set to .Dq prohibit-password or .Dq without-password , password and keyboard-interactive authentication are disabled for root. .Pp If this option is set to .Dq forced-commands-only , root login with public key authentication will be allowed, but only if the .Ar command option has been specified (which may be useful for taking remote backups even if root login is normally not allowed). All other authentication methods are disabled for root. .Pp If this option is set to .Dq no , root is not allowed to log in. .It Cm PermitTunnel Specifies whether .Xr tun 4 device forwarding is allowed. The argument must be .Dq yes , .Dq point-to-point (layer 3), .Dq ethernet (layer 2), or .Dq no . Specifying .Dq yes permits both .Dq point-to-point and .Dq ethernet . The default is .Dq no . .Pp Independent of this setting, the permissions of the selected .Xr tun 4 device must allow access to the user. .It Cm PermitTTY Specifies whether .Xr pty 4 allocation is permitted. The default is .Dq yes . .It Cm PermitUserEnvironment Specifies whether .Pa ~/.ssh/environment and .Cm environment= options in .Pa ~/.ssh/authorized_keys are processed by .Xr sshd 8 . The default is .Dq no . Enabling environment processing may enable users to bypass access restrictions in some configurations using mechanisms such as .Ev LD_PRELOAD . .It Cm PermitUserRC Specifies whether any .Pa ~/.ssh/rc file is executed. The default is .Dq yes . .It Cm PidFile Specifies the file that contains the process ID of the SSH daemon, or .Dq none to not write one. The default is .Pa /var/run/sshd.pid . .It Cm Port Specifies the port number that .Xr sshd 8 listens on. The default is 22. Multiple options of this type are permitted. See also .Cm ListenAddress . .It Cm PrintLastLog Specifies whether .Xr sshd 8 should print the date and time of the last user login when a user logs in interactively. The default is .Dq yes . .It Cm PrintMotd Specifies whether .Xr sshd 8 should print .Pa /etc/motd when a user logs in interactively. (On some systems it is also printed by the shell, .Pa /etc/profile , or equivalent.) The default is .Dq yes . .It Cm Protocol Specifies the protocol versions .Xr sshd 8 supports. The possible values are .Sq 1 and .Sq 2 . Multiple versions must be comma-separated. The default is .Sq 2 . Protocol 1 suffers from a number of cryptographic weaknesses and should not be used. It is only offered to support legacy devices. .Pp Note that the order of the protocol list does not indicate preference, because the client selects among multiple protocol versions offered by the server. Specifying .Dq 2,1 is identical to .Dq 1,2 . .It Cm PubkeyAcceptedKeyTypes Specifies the key types that will be accepted for public key authentication as a comma-separated pattern list. Alternately if the specified value begins with a .Sq + character, then the specified key types will be appended to the default set instead of replacing them. The default for this option is: .Bd -literal -offset 3n ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521-cert-v01@openssh.com, ssh-ed25519-cert-v01@openssh.com, ssh-rsa-cert-v01@openssh.com, ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521, ssh-ed25519,ssh-rsa .Ed .Pp The .Fl Q option of .Xr ssh 1 may be used to list supported key types. .It Cm PubkeyAuthentication Specifies whether public key authentication is allowed. The default is .Dq yes . .It Cm RekeyLimit Specifies the maximum amount of data that may be transmitted before the session key is renegotiated, optionally followed a maximum amount of time that may pass before the session key is renegotiated. The first argument is specified in bytes and may have a suffix of .Sq K , .Sq M , or .Sq G to indicate Kilobytes, Megabytes, or Gigabytes, respectively. The default is between .Sq 1G and .Sq 4G , depending on the cipher. The optional second value is specified in seconds and may use any of the units documented in the .Sx TIME FORMATS section. The default value for .Cm RekeyLimit is .Dq default none , which means that rekeying is performed after the cipher's default amount of data has been sent or received and no time based rekeying is done. .It Cm RevokedKeys Specifies revoked public keys file, or .Dq none to not use one. Keys listed in this file will be refused for public key authentication. Note that if this file is not readable, then public key authentication will be refused for all users. Keys may be specified as a text file, listing one public key per line, or as an OpenSSH Key Revocation List (KRL) as generated by .Xr ssh-keygen 1 . For more information on KRLs, see the KEY REVOCATION LISTS section in .Xr ssh-keygen 1 . .It Cm RhostsRSAAuthentication Specifies whether rhosts or .Pa /etc/hosts.equiv authentication together with successful RSA host authentication is allowed. The default is .Dq no . This option applies to protocol version 1 only. .It Cm RSAAuthentication Specifies whether pure RSA authentication is allowed. The default is .Dq yes . This option applies to protocol version 1 only. .It Cm ServerKeyBits Defines the number of bits in the ephemeral protocol version 1 server key. The default and minimum value is 1024. .It Cm StreamLocalBindMask Sets the octal file creation mode mask .Pq umask used when creating a Unix-domain socket file for local or remote port forwarding. This option is only used for port forwarding to a Unix-domain socket file. .Pp The default value is 0177, which creates a Unix-domain socket file that is readable and writable only by the owner. Note that not all operating systems honor the file mode on Unix-domain socket files. .It Cm StreamLocalBindUnlink Specifies whether to remove an existing Unix-domain socket file for local or remote port forwarding before creating a new one. If the socket file already exists and .Cm StreamLocalBindUnlink is not enabled, .Nm sshd will be unable to forward the port to the Unix-domain socket file. This option is only used for port forwarding to a Unix-domain socket file. .Pp The argument must be .Dq yes or .Dq no . The default is .Dq no . .It Cm StrictModes Specifies whether .Xr sshd 8 should check file modes and ownership of the user's files and home directory before accepting login. This is normally desirable because novices sometimes accidentally leave their directory or files world-writable. The default is .Dq yes . Note that this does not apply to .Cm ChrootDirectory , whose permissions and ownership are checked unconditionally. .It Cm Subsystem Configures an external subsystem (e.g. file transfer daemon). Arguments should be a subsystem name and a command (with optional arguments) to execute upon subsystem request. .Pp The command .Xr sftp-server 8 implements the .Dq sftp file transfer subsystem. .Pp Alternately the name .Dq internal-sftp implements an in-process .Dq sftp server. This may simplify configurations using .Cm ChrootDirectory to force a different filesystem root on clients. .Pp By default no subsystems are defined. .It Cm SyslogFacility Gives the facility code that is used when logging messages from .Xr sshd 8 . The possible values are: DAEMON, USER, AUTH, LOCAL0, LOCAL1, LOCAL2, LOCAL3, LOCAL4, LOCAL5, LOCAL6, LOCAL7. The default is AUTH. .It Cm TCPKeepAlive Specifies whether the system should send TCP keepalive messages to the other side. If they are sent, death of the connection or crash of one of the machines will be properly noticed. However, this means that connections will die if the route is down temporarily, and some people find it annoying. On the other hand, if TCP keepalives are not sent, sessions may hang indefinitely on the server, leaving .Dq ghost users and consuming server resources. .Pp The default is .Dq yes (to send TCP keepalive messages), and the server will notice if the network goes down or the client host crashes. This avoids infinitely hanging sessions. .Pp To disable TCP keepalive messages, the value should be set to .Dq no . .It Cm TrustedUserCAKeys Specifies a file containing public keys of certificate authorities that are trusted to sign user certificates for authentication, or .Dq none to not use one. Keys are listed one per line; empty lines and comments starting with .Ql # are allowed. If a certificate is presented for authentication and has its signing CA key listed in this file, then it may be used for authentication for any user listed in the certificate's principals list. Note that certificates that lack a list of principals will not be permitted for authentication using .Cm TrustedUserCAKeys . For more details on certificates, see the CERTIFICATES section in .Xr ssh-keygen 1 . +.It Cm UseBlacklist +Specifies whether +.Xr sshd 8 +attempts to send authentication success and failure messages +to the +.Xr blacklistd 8 +daemon. +The default is +.Dq no . .It Cm UseDNS Specifies whether .Xr sshd 8 should look up the remote host name, and to check that the resolved host name for the remote IP address maps back to the very same IP address. .Pp If this option is set to .Dq no , then only addresses and not host names may be used in .Pa ~/.ssh/known_hosts .Cm from and .Nm .Cm Match .Cm Host directives. The default is .Dq yes . .It Cm UseLogin Specifies whether .Xr login 1 is used for interactive login sessions. The default is .Dq no . Note that .Xr login 1 is never used for remote command execution. Note also, that if this is enabled, .Cm X11Forwarding will be disabled because .Xr login 1 does not know how to handle .Xr xauth 1 cookies. If .Cm UsePrivilegeSeparation is specified, it will be disabled after authentication. .It Cm UsePAM Enables the Pluggable Authentication Module interface. If set to .Dq yes this will enable PAM authentication using .Cm ChallengeResponseAuthentication and .Cm PasswordAuthentication in addition to PAM account and session module processing for all authentication types. .Pp Because PAM challenge-response authentication usually serves an equivalent role to password authentication, you should disable either .Cm PasswordAuthentication or .Cm ChallengeResponseAuthentication. .Pp If .Cm UsePAM is enabled, you will not be able to run .Xr sshd 8 as a non-root user. The default is .Dq yes . .It Cm UsePrivilegeSeparation Specifies whether .Xr sshd 8 separates privileges by creating an unprivileged child process to deal with incoming network traffic. After successful authentication, another process will be created that has the privilege of the authenticated user. The goal of privilege separation is to prevent privilege escalation by containing any corruption within the unprivileged processes. The argument must be .Dq yes , .Dq no , or .Dq sandbox . If .Cm UsePrivilegeSeparation is set to .Dq sandbox then the pre-authentication unprivileged process is subject to additional restrictions. The default is .Dq sandbox . .It Cm VersionAddendum Optionally specifies additional text to append to the SSH protocol banner sent by the server upon connection. The default is .Dq FreeBSD-20160310 . The value .Dq none may be used to disable this. .It Cm X11DisplayOffset Specifies the first display number available for .Xr sshd 8 Ns 's X11 forwarding. This prevents sshd from interfering with real X11 servers. The default is 10. .It Cm X11Forwarding Specifies whether X11 forwarding is permitted. The argument must be .Dq yes or .Dq no . The default is .Dq yes . .Pp When X11 forwarding is enabled, there may be additional exposure to the server and to client displays if the .Xr sshd 8 proxy display is configured to listen on the wildcard address (see .Cm X11UseLocalhost below), though this is not the default. Additionally, the authentication spoofing and authentication data verification and substitution occur on the client side. The security risk of using X11 forwarding is that the client's X11 display server may be exposed to attack when the SSH client requests forwarding (see the warnings for .Cm ForwardX11 in .Xr ssh_config 5 ) . A system administrator may have a stance in which they want to protect clients that may expose themselves to attack by unwittingly requesting X11 forwarding, which can warrant a .Dq no setting. .Pp Note that disabling X11 forwarding does not prevent users from forwarding X11 traffic, as users can always install their own forwarders. X11 forwarding is automatically disabled if .Cm UseLogin is enabled. .It Cm X11UseLocalhost Specifies whether .Xr sshd 8 should bind the X11 forwarding server to the loopback address or to the wildcard address. By default, sshd binds the forwarding server to the loopback address and sets the hostname part of the .Ev DISPLAY environment variable to .Dq localhost . This prevents remote hosts from connecting to the proxy display. However, some older X11 clients may not function with this configuration. .Cm X11UseLocalhost may be set to .Dq no to specify that the forwarding server should be bound to the wildcard address. The argument must be .Dq yes or .Dq no . The default is .Dq yes . .It Cm XAuthLocation Specifies the full pathname of the .Xr xauth 1 program, or .Dq none to not use one. The default is .Pa /usr/local/bin/xauth . .El .Sh TIME FORMATS .Xr sshd 8 command-line arguments and configuration file options that specify time may be expressed using a sequence of the form: .Sm off .Ar time Op Ar qualifier , .Sm on where .Ar time is a positive integer value and .Ar qualifier is one of the following: .Pp .Bl -tag -width Ds -compact -offset indent .It Aq Cm none seconds .It Cm s | Cm S seconds .It Cm m | Cm M minutes .It Cm h | Cm H hours .It Cm d | Cm D days .It Cm w | Cm W weeks .El .Pp Each member of the sequence is added together to calculate the total time value. .Pp Time format examples: .Pp .Bl -tag -width Ds -compact -offset indent .It 600 600 seconds (10 minutes) .It 10m 10 minutes .It 1h30m 1 hour 30 minutes (90 minutes) .El .Sh FILES .Bl -tag -width Ds .It Pa /etc/ssh/sshd_config Contains configuration data for .Xr sshd 8 . This file should be writable by root only, but it is recommended (though not necessary) that it be world-readable. .El .Sh SEE ALSO .Xr sshd 8 .Sh AUTHORS OpenSSH is a derivative of the original and free ssh 1.2.12 release by Tatu Ylonen. Aaron Campbell, Bob Beck, Markus Friedl, Niels Provos, Theo de Raadt and Dug Song removed many bugs, re-added newer features and created OpenSSH. Markus Friedl contributed the support for SSH protocol versions 1.5 and 2.0. Niels Provos and Markus Friedl contributed support for privilege separation. Index: projects/netbsd-tests-update-12/crypto/openssh =================================================================== --- projects/netbsd-tests-update-12/crypto/openssh (revision 305171) +++ projects/netbsd-tests-update-12/crypto/openssh (revision 305172) Property changes on: projects/netbsd-tests-update-12/crypto/openssh ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/crypto/openssh:r303899-305170 Index: projects/netbsd-tests-update-12/gnu/lib/libgcc/Makefile =================================================================== --- projects/netbsd-tests-update-12/gnu/lib/libgcc/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/gnu/lib/libgcc/Makefile (revision 305172) @@ -1,408 +1,412 @@ # $FreeBSD$ PACKAGE= clibs GCCDIR= ${.CURDIR}/../../../contrib/gcc GCCLIB= ${.CURDIR}/../../../contrib/gcclibs CCDIR= ${.CURDIR}/../../usr.bin/cc COMPILERRTDIR= ${.CURDIR}/../../../contrib/compiler-rt UNWINDINCDIR= ${.CURDIR}/../../../contrib/llvm/projects/libunwind/include UNWINDSRCDIR= ${.CURDIR}/../../../contrib/llvm/projects/libunwind/src SHLIB_NAME= libgcc_s.so.1 SHLIBDIR?= /lib .include # # libgcc is linked in last and thus cannot depend on ssp symbols coming # from earlier libraries. Disable stack protection for this library. # MK_SSP= no .include "${CCDIR}/Makefile.tgt" .include "${CCDIR}/cc_tools/Makefile.hdrs" .if ${TARGET_CPUARCH} == "arm" CFLAGS+= -DTARGET_ARM_EABI .endif .PATH: ${GCCDIR}/config/${GCC_CPU} ${GCCDIR}/config ${GCCDIR} CFLAGS+= -DIN_GCC -DIN_LIBGCC2 -D__GCC_FLOAT_NOT_NEEDED \ -DHAVE_GTHR_DEFAULT \ -I${GCCLIB}/include \ -I${GCCDIR}/config -I${GCCDIR} -I. \ -I${CCDIR}/cc_tools LDFLAGS+= -nodefaultlibs LIBADD+= c SOBJS= # added to below in various ways depending on TARGET_CPUARCH #--------------------------------------------------------------------------- # # Library members defined in libgcc2.c. # When upgrading GCC, obtain the following list from mklibgcc.in # LIB2FUNCS= _muldi3 _negdi2 _lshrdi3 _ashldi3 _ashrdi3 \ _cmpdi2 _ucmpdi2 \ _enable_execute_stack _trampoline __main _absvsi2 _absvdi2 _addvsi3 \ _addvdi3 _subvsi3 _subvdi3 _mulvsi3 _mulvdi3 _negvsi2 _negvdi2 _ctors \ _ffssi2 _ffsdi2 _clz _clzsi2 _clzdi2 _ctzsi2 _ctzdi2 _popcount_tab \ _popcountsi2 _popcountdi2 _paritysi2 _paritydi2 _powisf2 _powidf2 \ _powixf2 _powitf2 _mulsc3 _muldc3 _mulxc3 _multc3 _divsc3 _divdc3 \ _divxc3 _divtc3 _bswapsi2 _bswapdi2 .if ${COMPILER_TYPE} != "clang" || ${TARGET_CPUARCH} != "arm" LIB2FUNCS+= _clear_cache .endif # The floating-point conversion routines that involve a single-word integer. .for mode in sf df xf LIB2FUNCS+= _fixuns${mode}si .endfor # Likewise double-word routines. .if ${TARGET_CPUARCH} != "aarch64" && ${TARGET_CPUARCH} != "arm" && \ ${TARGET_CPUARCH} != "riscv64" # These are implemented in an ARM specific file but will not be filtered out. # RISCVTODO: can't compile .for mode in sf df xf tf LIB2FUNCS+= _fix${mode}di _fixuns${mode}di LIB2FUNCS+= _floatdi${mode} _floatundi${mode} .endfor .endif LIB2ADD = $(LIB2FUNCS_EXTRA) LIB2ADD_ST = $(LIB2FUNCS_STATIC_EXTRA) # Additional sources to handle exceptions; overridden by targets as needed. .if ${MK_LLVM_LIBUNWIND} != "no" .PATH: ${COMPILERRTDIR}/lib/builtins .PATH: ${UNWINDSRCDIR} LIB2ADDEH = gcc_personality_v0.c \ int_util.c \ Unwind-EHABI.cpp \ Unwind-sjlj.c \ UnwindLevel1-gcc-ext.c \ UnwindLevel1.c \ UnwindRegistersRestore.S \ UnwindRegistersSave.S \ libunwind.cpp CFLAGS+= -I${UNWINDINCDIR} -I${.CURDIR} -D_LIBUNWIND_IS_NATIVE_ONLY .if empty(CXXFLAGS:M-std=*) CXXFLAGS+= -std=c++11 .endif CXXFLAGS+= -fno-rtti STATIC_CXXFLAGS+= -fvisibility=hidden -fPIC +.if ${MK_DIRDEPS_BUILD} == "yes" +# Avoid dependency on lib/libc++ +CFLAGS+= -I${SRCTOP}/contrib/libc++/include +.endif .else # MK_LLVM_LIBUNWIND .if ${TARGET_CPUARCH} == "arm" LIB2ADDEH = unwind-arm.c libunwind-arm.S pr-support.c unwind-c.c .else LIB2ADDEH = unwind-dw2.c unwind-dw2-fde-glibc.c unwind-sjlj.c gthr-gnat.c \ unwind-c.c .endif .endif # MK_LLVM_LIBUNWIND LIB2ADDEHSTATIC = $(LIB2ADDEH) LIB2ADDEHSHARED = $(LIB2ADDEH) # List of extra C and assembler files to add to static and shared libgcc2. # Assembler files should have names ending in `.asm'. LIB2FUNCS_EXTRA = # List of extra C and assembler files to add to static libgcc2. # Assembler files should have names ending in `.asm'. LIB2FUNCS_STATIC_EXTRA = # Defined in libgcc2.c, included only in the static library. # KAN: Excluded _sf_to_tf and _df_to_tf as TPBIT_FUNCS are not # built on any of our platforms. LIB2FUNCS_ST = _eprintf __gcc_bcmp FPBIT_FUNCS = _pack_sf _unpack_sf _addsub_sf _mul_sf _div_sf \ _fpcmp_parts_sf _compare_sf _eq_sf _ne_sf _gt_sf _ge_sf \ _lt_sf _le_sf _unord_sf _si_to_sf _sf_to_si _negate_sf _make_sf \ _sf_to_df _thenan_sf _sf_to_usi _usi_to_sf DPBIT_FUNCS = _pack_df _unpack_df _addsub_df _mul_df _div_df \ _fpcmp_parts_df _compare_df _eq_df _ne_df _gt_df _ge_df \ _lt_df _le_df _unord_df _si_to_df _df_to_si _negate_df _make_df \ _df_to_sf _thenan_df _df_to_usi _usi_to_df TPBIT_FUNCS = _pack_tf _unpack_tf _addsub_tf _mul_tf _div_tf \ _fpcmp_parts_tf _compare_tf _eq_tf _ne_tf _gt_tf _ge_tf \ _lt_tf _le_tf _unord_tf _si_to_tf _tf_to_si _negate_tf _make_tf \ _tf_to_df _tf_to_sf _thenan_tf _tf_to_usi _usi_to_tf # These might cause a divide overflow trap and so are compiled with # unwinder info. LIB2_DIVMOD_FUNCS = _divdi3 _moddi3 _udivdi3 _umoddi3 _udiv_w_sdiv _udivmoddi4 #----------------------------------------------------------------------- # # Platform specific bits. # When upgrading GCC, get the following definitions from config//t-* # .if ${TARGET_CPUARCH} == "arm" # from config/arm/t-strongarm-elf CFLAGS+= -Dinhibit_libc -fno-inline CFLAGS.clang+= -fheinous-gnu-extensions LIB1ASMSRC = lib1funcs.asm LIB1ASMFUNCS = _dvmd_tls _bb_init_func # Some compilers generate __aeabi_ functions libgcc_s is missing LIBADD+= compiler_rt .endif .if ${TARGET_CPUARCH} == mips LIB2FUNCS_EXTRA = floatunsidf.c floatunsisf.c # ABIs other than o32 need this .if ${TARGET_ARCH} != "mips" && ${TARGET_ARCH} != "mipsel" LIB2FUNCS_EXTRA+= floatdidf.c fixunsdfsi.c LIB2FUNCS_EXTRA+= floatdisf.c floatundidf.c LIB2FUNCS_EXTRA+= fixsfdi.c floatundisf.c LIB2FUNCS_EXTRA+= fixdfdi.c fixunssfsi.c .endif .endif .if ${TARGET_ARCH} == "powerpc" # from config/rs6000/t-ppccomm LIB2FUNCS_EXTRA = tramp.asm LIB2FUNCS_STATIC_EXTRA = eabi.asm .endif .if ${TARGET_ARCH} == "powerpc64" # from config/rs6000/t-ppccomm LIB2FUNCS_EXTRA = tramp.asm .endif .if ${TARGET_CPUARCH} == "sparc64" # from config/sparc/t-elf LIB1ASMSRC = lb1spc.asm LIB1ASMFUNCS = _mulsi3 _divsi3 _modsi3 .endif #----------------------------------------------------------------------- # Remove any objects from LIB2FUNCS and LIB2_DIVMOD_FUNCS that are # defined as optimized assembly code in LIB1ASMFUNCS. .if defined(LIB1ASMFUNCS) .for sym in ${LIB1ASMFUNCS} LIB2FUNCS:= ${LIB2FUNCS:S/${sym}//g} LIB2_DIVMOD_FUNCS:= ${LIB2_DIVMOD_FUNCS:S/${sym}//g} .endfor .endif COMMONHDRS= tm.h tconfig.h options.h gthr-default.h .if ${MK_LLVM_LIBUNWIND} != "no" # unwind.h is a generated file when MK_LLVM_LIBUNWIND == "no", and a stale # copy may be left behind in OBJDIR when switching, so remove it explicitly. beforebuild: @rm -f ${.OBJDIR}/unwind.h .else COMMONHDRS+= unwind.h .endif #----------------------------------------------------------------------- # # Helpful shortcuts for compiler invocations. # HIDE = -fvisibility=hidden -DHIDE_EXPORTS CC_T = ${CC} -c ${CFLAGS} ${HIDE} -fPIC CC_P = ${CC} -c ${CFLAGS} ${HIDE} -p -fPIC CC_S = ${CC} -c ${CFLAGS} ${SHARED_CFLAGS} ${PICFLAG} -DSHARED #----------------------------------------------------------------------- # # Functions from libgcc2.c # STD_CFLAGS = DIV_CFLAGS = -fexceptions -fnon-call-exceptions STD_FUNCS = ${LIB2FUNCS} DIV_FUNCS = ${LIB2_DIVMOD_FUNCS} STD_CFILE = libgcc2.c DIV_CFILE = libgcc2.c OBJ_GRPS = STD DIV #----------------------------------------------------------------------- # # Floating point emulation functions # .if ${TARGET_CPUARCH} == "armNOT_YET" || \ ${TARGET_CPUARCH} == "powerpc" || ${TARGET_CPUARCH} == "sparc64" FPBIT_CFLAGS = -DFINE_GRAINED_LIBRARIES -DFLOAT DPBIT_CFLAGS = -DFINE_GRAINED_LIBRARIES FPBIT_CFILE = config/fp-bit.c DPBIT_CFILE = config/fp-bit.c OBJ_GRPS += FPBIT DPBIT .endif #----------------------------------------------------------------------- # # Generic build rules for object groups defined above # .for T in ${OBJ_GRPS} ${T}_OBJS_T = ${${T}_FUNCS:S/$/.o/} ${T}_OBJS_P = ${${T}_FUNCS:S/$/.po/} ${T}_OBJS_S = ${${T}_FUNCS:S/$/.So/} SOBJS += ${${T}_FUNCS:S/$/.So/} ${${T}_OBJS_T}: ${${T}_CFILE} ${COMMONHDRS} ${CC_T} ${${T}_CFLAGS} -DL${.PREFIX} -o ${.TARGET} ${.ALLSRC:M*.c} ${${T}_OBJS_P}: ${${T}_CFILE} ${COMMONHDRS} ${CC_P} ${${T}_CFLAGS} -DL${.PREFIX} -o ${.TARGET} ${.ALLSRC:M*.c} ${${T}_OBJS_S}: ${${T}_CFILE} ${COMMONHDRS} ${CC_S} ${${T}_CFLAGS} -DL${.PREFIX} -o ${.TARGET} ${.ALLSRC:M*.c} .endfor #----------------------------------------------------------------------- # # Extra objects coming from separate files # .if !empty(LIB2ADD) SOBJS += ${LIB2ADD:R:S/$/.So/} .endif #----------------------------------------------------------------------- # # Objects that should be in static library only. # SYMS_ST = ${LIB2FUNCS_ST} ${LIB2ADD_ST} STAT_OBJS_T = ${SYMS_ST:S/$/.o/} STAT_OBJS_P = ${SYMS_ST:S/$/.po/} STATICOBJS = ${SYMS_ST:S/$/.o/} ${STAT_OBJS_T}: ${STD_CFILE} ${COMMONHDRS} ${CC_T} -DL${.PREFIX} -o ${.TARGET} ${.ALLSRC:M*.c} ${STAT_OBJS_P}: ${STD_CFILE} ${COMMONHDRS} ${CC_P} -DL${.PREFIX} -o ${.TARGET} ${.ALLSRC:M*.c} #----------------------------------------------------------------------- # # Assembler files. # .if defined(LIB1ASMSRC) ASM_T = ${LIB1ASMFUNCS:S/$/.o/} ASM_P = ${LIB1ASMFUNCS:S/$/.po/} ASM_S = ${LIB1ASMFUNCS:S/$/.So/} ASM_V = ${LIB1ASMFUNCS:S/$/.vis/} SOBJS += ${LIB1ASMFUNCS:S/$/.So/} ${ASM_T}: ${LIB1ASMSRC} ${.PREFIX}.vis ${CC} -x assembler-with-cpp -c ${CFLAGS} -DL${.PREFIX} \ -o ${.TARGET} -include ${.PREFIX}.vis ${.ALLSRC:N*.h:N*.vis} ${ASM_P}: ${LIB1ASMSRC} ${.PREFIX}.vis ${CC} -x assembler-with-cpp -p -c ${CFLAGS} -DL${.PREFIX} \ -o ${.TARGET} -include ${.PREFIX}.vis ${.ALLSRC:N*.h:N*.vis} ${ASM_S}: ${LIB1ASMSRC} ${CC} -x assembler-with-cpp -c ${PICFLAG} ${CFLAGS} -DL${.PREFIX} \ -o ${.TARGET} ${.ALLSRC:N*.h} ${ASM_V}: ${LIB1ASMSRC} ${CC} -x assembler-with-cpp -c ${CFLAGS} -DL${.PREFIX} \ -o ${.PREFIX}.vo ${.ALLSRC:N*.h} ( ${NM} -pg ${.PREFIX}.vo | \ awk 'NF == 3 && $$2 !~ /^[UN]$$/ { print "\t.hidden ", $$3 }'\ ) > ${.TARGET} CLEANFILES += ${ASM_V} ${ASM_V:R:S/$/.vo/} .endif #----------------------------------------------------------------------- # # Exception handling / unwinding support. # EH_OBJS_T = ${LIB2ADDEHSTATIC:R:S/$/.o/} EH_OBJS_P = ${LIB2ADDEHSTATIC:R:S/$/.po/} EH_OBJS_S = ${LIB2ADDEHSHARED:R:S/$/.So/} EH_CFLAGS = -fexceptions -D__GLIBC__=3 -DElfW=__ElfN .if ${TARGET_CPUARCH} != "riscv64" # RISCVTODO: unwinding support SOBJS += ${EH_OBJS_S} .endif .for _src in ${LIB2ADDEHSTATIC:M*.c} ${_src:R:S/$/.o/}: ${_src} ${COMMONHDRS} ${CC_T} ${EH_CFLAGS} -o ${.TARGET} ${.IMPSRC} ${_src:R:S/$/.po/}: ${_src} ${COMMONHDRS} ${CC_P} ${EH_CFLAGS} -o ${.TARGET} ${.IMPSRC} .endfor .for _src in ${LIB2ADDEHSHARED:M*.c} ${_src:R:S/$/.So/}: ${_src} ${COMMONHDRS} ${CC_S} ${EH_CFLAGS} -o ${.TARGET} ${.IMPSRC} .endfor CLEANFILES += ${COMMONHDRS} CLEANFILES += cs-*.h option* #----------------------------------------------------------------------- # # Build symbol version map # SHLIB_MKMAP = ${GCCDIR}/mkmap-symver.awk SHLIB_MKMAP_OPTS = SHLIB_MAPFILES = ${GCCDIR}/libgcc-std.ver .if ${TARGET_CPUARCH} == "arm" SHLIB_MAPFILES += ${GCCDIR}/config/arm/libgcc-bpabi.ver .endif VERSION_MAP = libgcc.map libgcc.map: ${SHLIB_MKMAP} ${SHLIB_MAPFILES} ${SOBJS} ( ${NM} -pg ${SOBJS};echo %% ; \ cat ${SHLIB_MAPFILES} \ | sed -e '/^[ ]*#/d' \ -e 's/^%\(if\|else\|elif\|endif\|define\)/#\1/' \ | ${CC} ${CFLAGS} -E -xassembler-with-cpp -; \ ) | awk -f ${SHLIB_MKMAP} ${SHLIB_MKMAP_OPTS} > ${.TARGET} CLEANFILES += libgcc.map #----------------------------------------------------------------------- # # Build additional static libgcc_eh[_p].a libraries. # libgcc_eh.a: ${EH_OBJS_T} @${ECHO} building static gcc_eh library @rm -f ${.TARGET} ${AR} ${ARFLAGS} ${.TARGET} `lorder ${EH_OBJS_T} | tsort -q` ${RANLIB} ${RANLIBFLAGS} ${.TARGET} _LIBS+= libgcc_eh.a .if ${MK_PROFILE} != "no" libgcc_eh_p.a: ${EH_OBJS_P} @${ECHO} building profiled gcc_eh library @rm -f ${.TARGET} ${AR} ${ARFLAGS} ${.TARGET} `lorder ${EH_OBJS_P} | tsort -q` ${RANLIB} ${RANLIBFLAGS} ${.TARGET} _LIBS+= libgcc_eh_p.a .endif _libinstall: _lib-eh-install _lib-eh-install: .if ${MK_INSTALLLIB} != "no" ${INSTALL} -C -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${_INSTALLFLAGS} libgcc_eh.a ${DESTDIR}${LIBDIR} .endif .if ${MK_PROFILE} != "no" ${INSTALL} -C -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${_INSTALLFLAGS} libgcc_eh_p.a ${DESTDIR}${LIBDIR} .endif CLEANFILES+= libgcc_eh.a libgcc_eh_p.a ${EH_OBJS_T} ${EH_OBJS_P} .include .SUFFIXES: .vis .vo Index: projects/netbsd-tests-update-12/gnu/lib =================================================================== --- projects/netbsd-tests-update-12/gnu/lib (revision 305171) +++ projects/netbsd-tests-update-12/gnu/lib (revision 305172) Property changes on: projects/netbsd-tests-update-12/gnu/lib ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/gnu/lib:r303899-305170 Index: projects/netbsd-tests-update-12/lib/atf/libatf-c/Makefile =================================================================== --- projects/netbsd-tests-update-12/lib/atf/libatf-c/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/lib/atf/libatf-c/Makefile (revision 305172) @@ -1,93 +1,105 @@ #- # Copyright (c) 2011 Google, Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # $FreeBSD$ .include .include +# Store the toolchain executable in ATF_BUILD_{CC,CPP,CXX} to ensure other +# values -- like -target, -B ..., etc -- don't get leaked into the tests. +# +# Be sure to omit ${CCACHE_BIN} (if specified) from the variable as it gets +# automatically appended to the variables in bsd.compiler.mk when +# ${MK_CCACHE_BUILD} != no. +ATF_BUILD_CC:= ${CC:N${CCACHE_BIN}:[1]} +ATF_BUILD_CPP:= ${CPP:N${CCACHE_BIN}:[1]} +ATF_BUILD_CXX:= ${CXX:N${CCACHE_BIN}:[1]} + +# Only capture defines, includes, linker flags, optimization levels, warnings +# and preprocessor flags when building ATF_BUILD_{C,CPP,CXX}FLAGS. ATF_BUILD_CFLAGS:= ${CFLAGS:M-[DILOWf]*} ATF_BUILD_CPPFLAGS:= ${CPPFLAGS:M-[DILOWf]*} ATF_BUILD_CXXFLAGS:= ${CXXFLAGS:M-[DILOWf]*} LIB= atf-c PRIVATELIB= true SHLIB_MAJOR= 1 ATF= ${SRCTOP}/contrib/atf .PATH: ${ATF} .PATH: ${ATF}/atf-c .PATH: ${ATF}/atf-c/detail -CFLAGS+= -DATF_BUILD_CC='"${CC}"' +CFLAGS+= -DATF_BUILD_CC='"${ATF_BUILD_CC}"' CFLAGS+= -DATF_BUILD_CFLAGS='"${ATF_BUILD_CFLAGS}"' -CFLAGS+= -DATF_BUILD_CPP='"${CPP}"' +CFLAGS+= -DATF_BUILD_CPP='"${ATF_BUILD_CPP}"' CFLAGS+= -DATF_BUILD_CPPFLAGS='"${ATF_BUILD_CPPFLAGS}"' -CFLAGS+= -DATF_BUILD_CXX='"${CXX}"' +CFLAGS+= -DATF_BUILD_CXX='"${ATF_BUILD_CXX}"' CFLAGS+= -DATF_BUILD_CXXFLAGS='"${ATF_BUILD_CXXFLAGS}"' CFLAGS+= -I${ATF} CFLAGS+= -I${.CURDIR} CFLAGS+= -I. SRCS= build.c \ check.c \ dynstr.c \ env.c \ error.c \ fs.c \ list.c \ map.c \ process.c \ sanity.c \ text.c \ user.c \ utils.c \ tc.c \ tp.c \ tp_main.c INCS= build.h \ check.h \ defs.h \ error.h \ error_fwd.h \ macros.h \ tc.h \ tp.h \ utils.h INCSDIR= ${INCLUDEDIR}/atf-c INCS+= atf-c.h INCSDIR_atf-c.h= ${INCLUDEDIR} MAN= atf-c.3 MLINKS+= atf-c.3 atf-c-api.3 # Backwards compatibility. .if ${MK_TESTS} != "no" SUBDIR= tests .endif .include "../common.mk" .include Index: projects/netbsd-tests-update-12/lib/clang/libclangformat/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/lib/clang/libclangformat/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/lib/clang/libclangformat/Makefile.depend (revision 305172) @@ -1,15 +1,16 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ include \ include/xlocale \ lib/libc++ \ lib/msun \ + usr.bin/clang/clang-tblgen.host \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/lib/clang/libclangtoolingcore/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/lib/clang/libclangtoolingcore/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/lib/clang/libclangtoolingcore/Makefile.depend (revision 305172) @@ -1,15 +1,16 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ include \ include/xlocale \ lib/libc++ \ lib/msun \ + usr.bin/clang/clang-tblgen.host \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/lib/libc/gen/directory.3 =================================================================== --- projects/netbsd-tests-update-12/lib/libc/gen/directory.3 (revision 305171) +++ projects/netbsd-tests-update-12/lib/libc/gen/directory.3 (revision 305172) @@ -1,286 +1,308 @@ .\" Copyright (c) 1983, 1991, 1993 .\" The Regents of the University of California. All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" 4. Neither the name of the University nor the names of its contributors .\" may be used to endorse or promote products derived from this software .\" without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" @(#)directory.3 8.1 (Berkeley) 6/4/93 .\" $FreeBSD$ .\" -.Dd February 19, 2016 +.Dd August 31, 2016 .Dt DIRECTORY 3 .Os .Sh NAME .Nm opendir , .Nm fdopendir , .Nm readdir , .Nm readdir_r , .Nm telldir , .Nm seekdir , .Nm rewinddir , .Nm closedir , .Nm fdclosedir , .Nm dirfd .Nd directory operations .Sh LIBRARY .Lb libc .Sh SYNOPSIS .In dirent.h .Ft DIR * .Fn opendir "const char *filename" .Ft DIR * .Fn fdopendir "int fd" .Ft struct dirent * .Fn readdir "DIR *dirp" .Ft int .Fn readdir_r "DIR *dirp" "struct dirent *entry" "struct dirent **result" .Ft long .Fn telldir "DIR *dirp" .Ft void .Fn seekdir "DIR *dirp" "long loc" .Ft void .Fn rewinddir "DIR *dirp" .Ft int .Fn closedir "DIR *dirp" .Ft int .Fn fdclosedir "DIR *dirp" .Ft int .Fn dirfd "DIR *dirp" .Sh DESCRIPTION +.Bf -symbolic The +.Fn readdir_r +interface is deprecated +because it cannot be used correctly unless +.Brq Va NAME_MAX +is a fixed value. +.Ef +.Pp +The .Fn opendir function opens the directory named by .Fa filename , associates a .Em directory stream with it and returns a pointer to be used to identify the .Em directory stream in subsequent operations. The pointer .Dv NULL is returned if .Fa filename cannot be accessed, or if it cannot .Xr malloc 3 enough memory to hold the whole thing. .Pp The .Fn fdopendir function is equivalent to the .Fn opendir function except that the directory is specified by a file descriptor .Fa fd rather than by a name. The file offset associated with the file descriptor at the time of the call determines which entries are returned. .Pp Upon successful return from .Fn fdopendir , the file descriptor is under the control of the system, and if any attempt is made to close the file descriptor, or to modify the state of the associated description other than by means of .Fn closedir , .Fn readdir , .Fn readdir_r , or .Fn rewinddir , the behavior is undefined. Upon calling .Fn closedir the file descriptor is closed. The .Dv FD_CLOEXEC flag is set on the file descriptor by a successful call to .Fn fdopendir . .Pp The .Fn readdir function returns a pointer to the next directory entry. -It returns +The directory entry remains valid until the next call to +.Fn readdir +or +.Fn closedir +on the same +.Em directory stream . +The function returns .Dv NULL upon reaching the end of the directory or on error. In the event of an error, .Va errno may be set to any of the values documented for the .Xr getdirentries 2 system call. .Pp The .Fn readdir_r function provides the same functionality as .Fn readdir , but the caller must provide a directory .Fa entry buffer to store the results in. +The buffer must be large enough for a +.Vt struct dirent +with a +.Va d_name +array with +.Brq Va NAME_MAX ++ 1 elements. If the read succeeds, .Fa result is pointed at the .Fa entry ; upon reaching the end of the directory .Fa result is set to .Dv NULL . The .Fn readdir_r function returns 0 on success or an error number to indicate failure. .Pp The .Fn telldir function returns a token representing the current location associated with the named .Em directory stream . Values returned by .Fn telldir are good only for the lifetime of the .Dv DIR pointer, .Fa dirp , from which they are derived. If the directory is closed and then reopened, prior values returned by .Fn telldir will no longer be valid. Values returned by .Fn telldir are also invalidated by a call to .Fn rewinddir . .Pp The .Fn seekdir function sets the position of the next .Fn readdir operation on the .Em directory stream . The new position reverts to the one associated with the .Em directory stream when the .Fn telldir operation was performed. .Pp The .Fn rewinddir function resets the position of the named .Em directory stream to the beginning of the directory. .Pp The .Fn closedir function closes the named .Em directory stream and frees the structure associated with the .Fa dirp pointer, returning 0 on success. On failure, \-1 is returned and the global variable .Va errno is set to indicate the error. .Pp The .Fn fdclosedir function is equivalent to the .Fn closedir function except that this function returns directory file descriptor instead of closing it. .Pp The .Fn dirfd function returns the integer file descriptor associated with the named .Em directory stream , see .Xr open 2 . .Pp Sample code which searches a directory for entry ``name'' is: .Bd -literal -offset indent dirp = opendir("."); if (dirp == NULL) return (ERROR); len = strlen(name); while ((dp = readdir(dirp)) != NULL) { if (dp->d_namlen == len && strcmp(dp->d_name, name) == 0) { (void)closedir(dirp); return (FOUND); } } (void)closedir(dirp); return (NOT_FOUND); .Ed .Sh SEE ALSO .Xr close 2 , .Xr lseek 2 , .Xr open 2 , .Xr read 2 , .Xr dir 5 .Sh HISTORY The .Fn opendir , .Fn readdir , .Fn telldir , .Fn seekdir , .Fn rewinddir , .Fn closedir , and .Fn dirfd functions appeared in .Bx 4.2 . The .Fn fdopendir function appeared in .Fx 8.0 . .Fn fdclosedir function appeared in .Fx 10.0 . .Sh BUGS The behaviour of .Fn telldir and .Fn seekdir is likely to be wrong if there are parallel unlinks happening and the directory is larger than one page. There is code to ensure that a .Fn seekdir to the location given by a .Fn telldir immediately before the last .Fn readdir will always set the correct location to return the same value as that last .Fn readdir performed. This is enough for some applications which want to "push back the last entry read" E.g. Samba. Seeks back to any other location, other than the beginning of the directory, may result in unexpected behaviour if deletes are present. It is hoped that this situation will be resolved with changes to .Fn getdirentries and the VFS. Index: projects/netbsd-tests-update-12/lib/libc/gen/getvfsbyname.3 =================================================================== --- projects/netbsd-tests-update-12/lib/libc/gen/getvfsbyname.3 (revision 305171) +++ projects/netbsd-tests-update-12/lib/libc/gen/getvfsbyname.3 (revision 305172) @@ -1,122 +1,117 @@ .\" Copyright (c) 1995 .\" The Regents of the University of California. All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" 4. Neither the name of the University nor the names of its contributors .\" may be used to endorse or promote products derived from this software .\" without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" @(#)kvm_getvfsbyname.3 8.3 (Berkeley) 5/4/95 .\" $FreeBSD$ .\" -.Dd April 5, 2007 +.Dd August 30, 2016 .Dt GETVFSBYNAME 3 .Os .Sh NAME .Nm getvfsbyname .Nd get information about a file system .Sh LIBRARY .Lb libc .Sh SYNOPSIS .In sys/param.h .In sys/mount.h .Ft int .Fn getvfsbyname "const char *name" "struct xvfsconf *vfc" .Sh DESCRIPTION The .Fn getvfsbyname function provides access to information about a file system module that is configured in the kernel. If successful, the requested file system .Fa xvfsconf is returned in the location pointed to by .Fa vfc . The fields in a .Dq Li struct xvfsconf are defined as follows: .Pp .Bl -tag -compact -width vfc_refcount .It vfc_name the name of the file system .It vfc_typenum the file system type number assigned by the kernel .It vfc_refcount the number of active mount points using the file system .It vfc_flags flag bits, as described below .El .Pp The flags are defined as follows: .Pp .Bl -tag -width VFCF_DELEGADMIN -compact .It Dv VFCF_STATIC statically compiled into kernel .It Dv VFCF_NETWORK may get data over the network .It Dv VFCF_READONLY writes are not implemented .It Dv VFCF_SYNTHETIC data does not represent real files .It Dv VFCF_LOOPBACK aliases some other mounted FS .It Dv VFCF_UNICODE stores file names as Unicode .It Dv VFCF_JAIL can be mounted from within a jail if .Va security.jail.mount_allowed sysctl is set to .Dv 1 .It Dv VFCF_DELEGADMIN supports delegated administration if .Va vfs.usermount sysctl is set to .Dv 1 .El .Sh RETURN VALUES .Rv -std getvfsbyname .Sh ERRORS The following errors may be reported: .Bl -tag -width Er -.It Bq Er EFAULT -The -.Fa vfc -argument -points to an invalid address. .It Bq Er ENOENT The .Fa name argument specifies a file system that is unknown or not configured in the kernel. .El .Sh SEE ALSO .Xr jail 2 , .Xr mount 2 , .Xr sysctl 3 , .Xr jail 8 , .Xr mount 8 , .Xr sysctl 8 .Sh HISTORY A variant of the .Fn getvfsbyname function first appeared in .Fx 2.0 . Index: projects/netbsd-tests-update-12/lib/libc/net/getaddrinfo.c =================================================================== --- projects/netbsd-tests-update-12/lib/libc/net/getaddrinfo.c (revision 305171) +++ projects/netbsd-tests-update-12/lib/libc/net/getaddrinfo.c (revision 305172) @@ -1,3037 +1,3037 @@ /* $KAME: getaddrinfo.c,v 1.15 2000/07/09 04:37:24 itojun Exp $ */ /* * Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the project nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * Issues to be discussed: * - Return values. There are nonstandard return values defined and used * in the source code. This is because RFC2553 is silent about which error * code must be returned for which situation. * - freeaddrinfo(NULL). RFC2553 is silent about it. XNET 5.2 says it is * invalid. current code - SEGV on freeaddrinfo(NULL) * * Note: * - The code filters out AFs that are not supported by the kernel, * when globbing NULL hostname (to loopback, or wildcard). Is it the right * thing to do? What is the relationship with post-RFC2553 AI_ADDRCONFIG * in ai_flags? * - (post-2553) semantics of AI_ADDRCONFIG itself is too vague. * (1) what should we do against numeric hostname (2) what should we do * against NULL hostname (3) what is AI_ADDRCONFIG itself. AF not ready? * non-loopback address configured? global address configured? * * OS specific notes for freebsd4: * - FreeBSD supported $GAI. The code does not. */ #include __FBSDID("$FreeBSD$"); #include "namespace.h" #include #include #include #include #include #include #include #ifdef INET6 #include #include #include #include #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "res_config.h" #ifdef DEBUG #include #endif #include #include #include "un-namespace.h" #include "netdb_private.h" #include "libc_private.h" #ifdef NS_CACHING #include "nscache.h" #endif #define ANY 0 #define YES 1 #define NO 0 static const char in_addrany[] = { 0, 0, 0, 0 }; static const char in_loopback[] = { 127, 0, 0, 1 }; #ifdef INET6 static const char in6_addrany[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }; static const char in6_loopback[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1 }; #endif struct policyqueue { TAILQ_ENTRY(policyqueue) pc_entry; #ifdef INET6 struct in6_addrpolicy pc_policy; #endif }; TAILQ_HEAD(policyhead, policyqueue); static const struct afd { int a_af; int a_addrlen; socklen_t a_socklen; int a_off; const char *a_addrany; const char *a_loopback; int a_scoped; } afdl [] = { #ifdef INET6 #define N_INET6 0 {PF_INET6, sizeof(struct in6_addr), sizeof(struct sockaddr_in6), offsetof(struct sockaddr_in6, sin6_addr), in6_addrany, in6_loopback, 1}, #define N_INET 1 #define N_LOCAL 2 #else #define N_INET 0 #define N_LOCAL 1 #endif {PF_INET, sizeof(struct in_addr), sizeof(struct sockaddr_in), offsetof(struct sockaddr_in, sin_addr), in_addrany, in_loopback, 0}, #define sizeofmember(type, member) (sizeof(((type *)0)->member)) {PF_LOCAL, sizeofmember(struct sockaddr_un, sun_path), sizeof(struct sockaddr_un), offsetof(struct sockaddr_un, sun_path), NULL, NULL, 0}, {0, 0, 0, 0, NULL, NULL, 0}, }; struct explore { int e_af; int e_socktype; int e_protocol; int e_wild; #define AF_ANY 0x01 #define SOCKTYPE_ANY 0x02 #define PROTOCOL_ANY 0x04 #define WILD_AF(ex) ((ex)->e_wild & AF_ANY) #define WILD_SOCKTYPE(ex) ((ex)->e_wild & SOCKTYPE_ANY) #define WILD_PROTOCOL(ex) ((ex)->e_wild & PROTOCOL_ANY) }; static const struct explore explore[] = { #ifdef INET6 { PF_INET6, SOCK_DGRAM, IPPROTO_UDP, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { PF_INET6, SOCK_STREAM, IPPROTO_TCP, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { PF_INET6, SOCK_STREAM, IPPROTO_SCTP, AF_ANY | SOCKTYPE_ANY }, { PF_INET6, SOCK_SEQPACKET, IPPROTO_SCTP, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { PF_INET6, SOCK_DGRAM, IPPROTO_UDPLITE, AF_ANY | SOCKTYPE_ANY }, { PF_INET6, SOCK_RAW, ANY, AF_ANY | PROTOCOL_ANY }, #endif { PF_INET, SOCK_DGRAM, IPPROTO_UDP, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { PF_INET, SOCK_STREAM, IPPROTO_TCP, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { PF_INET, SOCK_STREAM, IPPROTO_SCTP, AF_ANY | SOCKTYPE_ANY }, { PF_INET, SOCK_SEQPACKET, IPPROTO_SCTP, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { PF_INET, SOCK_DGRAM, IPPROTO_UDPLITE, AF_ANY | SOCKTYPE_ANY }, { PF_INET, SOCK_RAW, ANY, AF_ANY | PROTOCOL_ANY }, { PF_LOCAL, SOCK_DGRAM, ANY, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { PF_LOCAL, SOCK_STREAM, ANY, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { PF_LOCAL, SOCK_SEQPACKET, ANY, AF_ANY | SOCKTYPE_ANY | PROTOCOL_ANY }, { -1, 0, 0, 0 }, }; #ifdef INET6 #define PTON_MAX 16 #else #define PTON_MAX 4 #endif #define AIO_SRCFLAG_DEPRECATED 0x1 struct ai_order { union { struct sockaddr_storage aiou_ss; struct sockaddr aiou_sa; } aio_src_un; #define aio_srcsa aio_src_un.aiou_sa u_int32_t aio_srcflag; int aio_srcscope; int aio_dstscope; struct policyqueue *aio_srcpolicy; struct policyqueue *aio_dstpolicy; struct addrinfo *aio_ai; int aio_matchlen; int aio_initial_sequence; }; static const ns_src default_dns_files[] = { { NSSRC_FILES, NS_SUCCESS }, { NSSRC_DNS, NS_SUCCESS }, { 0 } }; struct res_target { struct res_target *next; const char *name; /* domain name */ int qclass, qtype; /* class and type of query */ u_char *answer; /* buffer to put answer */ int anslen; /* size of answer buffer */ int n; /* result length */ }; #define MAXPACKET (64*1024) typedef union { HEADER hdr; u_char buf[MAXPACKET]; } querybuf; static int str2number(const char *, int *); static int explore_copy(const struct addrinfo *, const struct addrinfo *, struct addrinfo **); static int explore_null(const struct addrinfo *, const char *, struct addrinfo **); static int explore_numeric(const struct addrinfo *, const char *, const char *, struct addrinfo **, const char *); static int explore_numeric_scope(const struct addrinfo *, const char *, const char *, struct addrinfo **); static int get_canonname(const struct addrinfo *, struct addrinfo *, const char *); static struct addrinfo *get_ai(const struct addrinfo *, const struct afd *, const char *); static struct addrinfo *copy_ai(const struct addrinfo *); static int get_portmatch(const struct addrinfo *, const char *); static int get_port(struct addrinfo *, const char *, int); static const struct afd *find_afd(int); static int addrconfig(struct addrinfo *); #ifdef INET6 static int is_ifdisabled(char *); #endif static void set_source(struct ai_order *, struct policyhead *); static int comp_dst(const void *, const void *); #ifdef INET6 static int ip6_str2scopeid(char *, struct sockaddr_in6 *, u_int32_t *); #endif static int gai_addr2scopetype(struct sockaddr *); static int explore_fqdn(const struct addrinfo *, const char *, const char *, struct addrinfo **); static int reorder(struct addrinfo *); static int get_addrselectpolicy(struct policyhead *); static void free_addrselectpolicy(struct policyhead *); static struct policyqueue *match_addrselectpolicy(struct sockaddr *, struct policyhead *); static int matchlen(struct sockaddr *, struct sockaddr *); static struct addrinfo *getanswer(const querybuf *, int, const char *, int, const struct addrinfo *, res_state); #if defined(RESOLVSORT) static int addr4sort(struct addrinfo *, res_state); #endif static int _dns_getaddrinfo(void *, void *, va_list); static void _sethtent(FILE **); static void _endhtent(FILE **); static struct addrinfo *_gethtent(FILE **, const char *, const struct addrinfo *); static int _files_getaddrinfo(void *, void *, va_list); #ifdef YP static struct addrinfo *_yphostent(char *, const struct addrinfo *); static int _yp_getaddrinfo(void *, void *, va_list); #endif #ifdef NS_CACHING static int addrinfo_id_func(char *, size_t *, va_list, void *); static int addrinfo_marshal_func(char *, size_t *, void *, va_list, void *); static int addrinfo_unmarshal_func(char *, size_t, void *, va_list, void *); #endif static int res_queryN(const char *, struct res_target *, res_state); static int res_searchN(const char *, struct res_target *, res_state); static int res_querydomainN(const char *, const char *, struct res_target *, res_state); /* XXX macros that make external reference is BAD. */ #define GET_AI(ai, afd, addr) \ do { \ /* external reference: pai, error, and label free */ \ (ai) = get_ai(pai, (afd), (addr)); \ if ((ai) == NULL) { \ error = EAI_MEMORY; \ goto free; \ } \ } while (/*CONSTCOND*/0) #define GET_PORT(ai, serv) \ do { \ /* external reference: error and label free */ \ error = get_port((ai), (serv), 0); \ if (error != 0) \ goto free; \ } while (/*CONSTCOND*/0) #define GET_CANONNAME(ai, str) \ do { \ /* external reference: pai, error and label free */ \ error = get_canonname(pai, (ai), (str)); \ if (error != 0) \ goto free; \ } while (/*CONSTCOND*/0) #define ERR(err) \ do { \ /* external reference: error, and label bad */ \ error = (err); \ goto bad; \ /*NOTREACHED*/ \ } while (/*CONSTCOND*/0) #define MATCH_FAMILY(x, y, w) \ ((x) == (y) || (/*CONSTCOND*/(w) && ((x) == PF_UNSPEC || (y) == PF_UNSPEC))) #define MATCH(x, y, w) \ ((x) == (y) || (/*CONSTCOND*/(w) && ((x) == ANY || (y) == ANY))) void freeaddrinfo(struct addrinfo *ai) { struct addrinfo *next; do { next = ai->ai_next; if (ai->ai_canonname) free(ai->ai_canonname); /* no need to free(ai->ai_addr) */ free(ai); ai = next; } while (ai); } static int str2number(const char *p, int *portp) { char *ep; unsigned long v; if (*p == '\0') return -1; ep = NULL; errno = 0; v = strtoul(p, &ep, 10); if (errno == 0 && ep && *ep == '\0' && v <= UINT_MAX) { *portp = v; return 0; } else return -1; } int getaddrinfo(const char *hostname, const char *servname, const struct addrinfo *hints, struct addrinfo **res) { struct addrinfo sentinel; struct addrinfo *cur; int error = 0; struct addrinfo ai, ai0, *afai; struct addrinfo *pai; const struct afd *afd; const struct explore *ex; struct addrinfo *afailist[nitems(afdl)]; struct addrinfo *afai_unspec; int found; int numeric = 0; /* ensure we return NULL on errors */ *res = NULL; memset(&ai, 0, sizeof(ai)); memset(afailist, 0, sizeof(afailist)); afai_unspec = NULL; memset(&sentinel, 0, sizeof(sentinel)); cur = &sentinel; pai = &ai; pai->ai_flags = 0; pai->ai_family = PF_UNSPEC; pai->ai_socktype = ANY; pai->ai_protocol = ANY; pai->ai_addrlen = 0; pai->ai_canonname = NULL; pai->ai_addr = NULL; pai->ai_next = NULL; if (hostname == NULL && servname == NULL) return EAI_NONAME; if (hints) { /* error check for hints */ if (hints->ai_addrlen || hints->ai_canonname || hints->ai_addr || hints->ai_next) ERR(EAI_BADHINTS); /* xxx */ if (hints->ai_flags & ~AI_MASK) ERR(EAI_BADFLAGS); switch (hints->ai_family) { case PF_UNSPEC: case PF_LOCAL: case PF_INET: #ifdef INET6 case PF_INET6: #endif break; default: ERR(EAI_FAMILY); } memcpy(pai, hints, sizeof(*pai)); /* * if both socktype/protocol are specified, check if they * are meaningful combination. */ if (pai->ai_socktype != ANY && pai->ai_protocol != ANY) { for (ex = explore; ex->e_af >= 0; ex++) { if (!MATCH_FAMILY(pai->ai_family, ex->e_af, WILD_AF(ex))) continue; if (!MATCH(pai->ai_socktype, ex->e_socktype, WILD_SOCKTYPE(ex))) continue; if (!MATCH(pai->ai_protocol, ex->e_protocol, WILD_PROTOCOL(ex))) continue; /* matched */ break; } if (ex->e_af < 0) ERR(EAI_BADHINTS); } } /* * RFC 3493: AI_ALL and AI_V4MAPPED are effective only against * AF_INET6 query. They need to be ignored if specified in other * occasions. */ switch (pai->ai_flags & (AI_ALL | AI_V4MAPPED)) { case AI_V4MAPPED: case AI_ALL | AI_V4MAPPED: #ifdef INET6 if (pai->ai_family != AF_INET6) pai->ai_flags &= ~(AI_ALL | AI_V4MAPPED); break; #endif case AI_ALL: pai->ai_flags &= ~(AI_ALL | AI_V4MAPPED); break; } /* * check for special cases. (1) numeric servname is disallowed if * socktype/protocol are left unspecified. (2) servname is disallowed * for raw and other inet{,6} sockets. */ if (MATCH_FAMILY(pai->ai_family, PF_INET, 1) #ifdef PF_INET6 || MATCH_FAMILY(pai->ai_family, PF_INET6, 1) #endif ) { ai0 = *pai; /* backup *pai */ if (pai->ai_family == PF_UNSPEC) { #ifdef PF_INET6 pai->ai_family = PF_INET6; #else pai->ai_family = PF_INET; #endif } error = get_portmatch(pai, servname); if (error) goto bad; *pai = ai0; } ai0 = *pai; /* * NULL hostname, or numeric hostname. * If numeric representation of AF1 can be interpreted as FQDN * representation of AF2, we need to think again about the code below. */ found = 0; for (afd = afdl; afd->a_af; afd++) { *pai = ai0; if (!MATCH_FAMILY(pai->ai_family, afd->a_af, 1)) continue; if (pai->ai_family == PF_UNSPEC) pai->ai_family = afd->a_af; if (hostname == NULL) { error = explore_null(pai, servname, &afailist[afd - afdl]); /* * Errors from explore_null should be unexpected and * be caught to avoid returning an incomplete result. */ if (error != 0) goto bad; } else { error = explore_numeric_scope(pai, hostname, servname, &afailist[afd - afdl]); /* * explore_numeric_scope returns an error for address * families that do not match that of hostname. * Thus we should not catch the error at this moment. */ } if (!error && afailist[afd - afdl]) found++; } if (found) { numeric = 1; goto globcopy; } if (hostname == NULL) ERR(EAI_NONAME); /* used to be EAI_NODATA */ if (pai->ai_flags & AI_NUMERICHOST) ERR(EAI_NONAME); if ((pai->ai_flags & AI_ADDRCONFIG) != 0 && !addrconfig(&ai0)) ERR(EAI_FAIL); /* * hostname as alphabetical name. */ *pai = ai0; error = explore_fqdn(pai, hostname, servname, &afai_unspec); globcopy: for (ex = explore; ex->e_af >= 0; ex++) { *pai = ai0; if (!MATCH_FAMILY(pai->ai_family, ex->e_af, WILD_AF(ex))) continue; if (!MATCH(pai->ai_socktype, ex->e_socktype, WILD_SOCKTYPE(ex))) continue; if (!MATCH(pai->ai_protocol, ex->e_protocol, WILD_PROTOCOL(ex))) continue; if (pai->ai_family == PF_UNSPEC) pai->ai_family = ex->e_af; if (pai->ai_socktype == ANY && ex->e_socktype != ANY) pai->ai_socktype = ex->e_socktype; if (pai->ai_protocol == ANY && ex->e_protocol != ANY) pai->ai_protocol = ex->e_protocol; /* * if the servname does not match socktype/protocol, ignore it. */ if (get_portmatch(pai, servname) != 0) continue; if (afai_unspec) afai = afai_unspec; else { if ((afd = find_afd(pai->ai_family)) == NULL) continue; /* XXX assumes that afd points inside afdl[] */ afai = afailist[afd - afdl]; } if (!afai) continue; error = explore_copy(pai, afai, &cur->ai_next); if (error != 0) goto bad; while (cur && cur->ai_next) cur = cur->ai_next; } /* * ensure we return either: * - error == 0, non-NULL *res * - error != 0, NULL *res */ if (error == 0) { if (sentinel.ai_next) { /* * If the returned entry is for an active connection, * and the given name is not numeric, reorder the * list, so that the application would try the list * in the most efficient order. Since the head entry * of the original list may contain ai_canonname and * that entry may be moved elsewhere in the new list, * we keep the pointer and will restore it in the new * head entry. (Note that RFC3493 requires the head * entry store it when requested by the caller). */ if (hints == NULL || !(hints->ai_flags & AI_PASSIVE)) { if (!numeric) { char *canonname; canonname = sentinel.ai_next->ai_canonname; sentinel.ai_next->ai_canonname = NULL; (void)reorder(&sentinel); if (sentinel.ai_next->ai_canonname == NULL) { sentinel.ai_next->ai_canonname = canonname; } else if (canonname != NULL) free(canonname); } } *res = sentinel.ai_next; } else error = EAI_FAIL; } bad: if (afai_unspec) freeaddrinfo(afai_unspec); for (afd = afdl; afd->a_af; afd++) { if (afailist[afd - afdl]) freeaddrinfo(afailist[afd - afdl]); } if (!*res) if (sentinel.ai_next) freeaddrinfo(sentinel.ai_next); return (error); } static int reorder(struct addrinfo *sentinel) { struct addrinfo *ai, **aip; struct ai_order *aio; int i, n; struct policyhead policyhead; /* count the number of addrinfo elements for sorting. */ for (n = 0, ai = sentinel->ai_next; ai != NULL; ai = ai->ai_next, n++) ; /* * If the number is small enough, we can skip the reordering process. */ if (n <= 1) return(n); /* allocate a temporary array for sort and initialization of it. */ if ((aio = malloc(sizeof(*aio) * n)) == NULL) return(n); /* give up reordering */ memset(aio, 0, sizeof(*aio) * n); /* retrieve address selection policy from the kernel */ TAILQ_INIT(&policyhead); if (!get_addrselectpolicy(&policyhead)) { /* no policy is installed into kernel, we don't sort. */ free(aio); return (n); } for (i = 0, ai = sentinel->ai_next; i < n; ai = ai->ai_next, i++) { aio[i].aio_ai = ai; aio[i].aio_dstscope = gai_addr2scopetype(ai->ai_addr); aio[i].aio_dstpolicy = match_addrselectpolicy(ai->ai_addr, &policyhead); set_source(&aio[i], &policyhead); aio[i].aio_initial_sequence = i; } /* perform sorting. */ qsort(aio, n, sizeof(*aio), comp_dst); /* reorder the addrinfo chain. */ for (i = 0, aip = &sentinel->ai_next; i < n; i++) { *aip = aio[i].aio_ai; aip = &aio[i].aio_ai->ai_next; } *aip = NULL; /* cleanup and return */ free(aio); free_addrselectpolicy(&policyhead); return(n); } static int get_addrselectpolicy(struct policyhead *head) { #ifdef INET6 int mib[] = { CTL_NET, PF_INET6, IPPROTO_IPV6, IPV6CTL_ADDRCTLPOLICY }; size_t l; char *buf; struct in6_addrpolicy *pol, *ep; if (sysctl(mib, nitems(mib), NULL, &l, NULL, 0) < 0) return (0); if (l == 0) return (0); if ((buf = malloc(l)) == NULL) return (0); if (sysctl(mib, nitems(mib), buf, &l, NULL, 0) < 0) { free(buf); return (0); } ep = (struct in6_addrpolicy *)(buf + l); for (pol = (struct in6_addrpolicy *)buf; pol + 1 <= ep; pol++) { struct policyqueue *new; if ((new = malloc(sizeof(*new))) == NULL) { free_addrselectpolicy(head); /* make the list empty */ break; } new->pc_policy = *pol; TAILQ_INSERT_TAIL(head, new, pc_entry); } free(buf); return (1); #else return (0); #endif } static void free_addrselectpolicy(struct policyhead *head) { struct policyqueue *ent, *nent; for (ent = TAILQ_FIRST(head); ent; ent = nent) { nent = TAILQ_NEXT(ent, pc_entry); TAILQ_REMOVE(head, ent, pc_entry); free(ent); } } static struct policyqueue * match_addrselectpolicy(struct sockaddr *addr, struct policyhead *head) { #ifdef INET6 struct policyqueue *ent, *bestent = NULL; struct in6_addrpolicy *pol; int matchlen, bestmatchlen = -1; u_char *mp, *ep, *k, *p, m; struct sockaddr_in6 key; switch(addr->sa_family) { case AF_INET6: key = *(struct sockaddr_in6 *)addr; break; case AF_INET: /* convert the address into IPv4-mapped IPv6 address. */ memset(&key, 0, sizeof(key)); key.sin6_family = AF_INET6; key.sin6_len = sizeof(key); _map_v4v6_address( (char *)&((struct sockaddr_in *)addr)->sin_addr, (char *)&key.sin6_addr); break; default: return(NULL); } for (ent = TAILQ_FIRST(head); ent; ent = TAILQ_NEXT(ent, pc_entry)) { pol = &ent->pc_policy; matchlen = 0; mp = (u_char *)&pol->addrmask.sin6_addr; ep = mp + 16; /* XXX: scope field? */ k = (u_char *)&key.sin6_addr; p = (u_char *)&pol->addr.sin6_addr; for (; mp < ep && *mp; mp++, k++, p++) { m = *mp; if ((*k & m) != *p) goto next; /* not match */ if (m == 0xff) /* short cut for a typical case */ matchlen += 8; else { while (m >= 0x80) { matchlen++; m <<= 1; } } } /* matched. check if this is better than the current best. */ if (matchlen > bestmatchlen) { bestent = ent; bestmatchlen = matchlen; } next: continue; } return(bestent); #else return(NULL); #endif } static void set_source(struct ai_order *aio, struct policyhead *ph) { struct addrinfo ai = *aio->aio_ai; struct sockaddr_storage ss; socklen_t srclen; int s; /* set unspec ("no source is available"), just in case */ aio->aio_srcsa.sa_family = AF_UNSPEC; aio->aio_srcscope = -1; switch(ai.ai_family) { case AF_INET: #ifdef INET6 case AF_INET6: #endif break; default: /* ignore unsupported AFs explicitly */ return; } /* XXX: make a dummy addrinfo to call connect() */ ai.ai_socktype = SOCK_DGRAM; ai.ai_protocol = IPPROTO_UDP; /* is UDP too specific? */ ai.ai_next = NULL; memset(&ss, 0, sizeof(ss)); memcpy(&ss, ai.ai_addr, ai.ai_addrlen); ai.ai_addr = (struct sockaddr *)&ss; get_port(&ai, "1", 0); /* open a socket to get the source address for the given dst */ if ((s = _socket(ai.ai_family, ai.ai_socktype | SOCK_CLOEXEC, ai.ai_protocol)) < 0) return; /* give up */ #ifdef INET6 if (ai.ai_family == AF_INET6) { struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)ai.ai_addr; int off = 0; if (IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)) (void)_setsockopt(s, IPPROTO_IPV6, IPV6_V6ONLY, (char *)&off, sizeof(off)); } #endif if (_connect(s, ai.ai_addr, ai.ai_addrlen) < 0) goto cleanup; srclen = ai.ai_addrlen; if (_getsockname(s, &aio->aio_srcsa, &srclen) < 0) { aio->aio_srcsa.sa_family = AF_UNSPEC; goto cleanup; } aio->aio_srcscope = gai_addr2scopetype(&aio->aio_srcsa); aio->aio_srcpolicy = match_addrselectpolicy(&aio->aio_srcsa, ph); aio->aio_matchlen = matchlen(&aio->aio_srcsa, aio->aio_ai->ai_addr); #ifdef INET6 if (ai.ai_family == AF_INET6) { struct in6_ifreq ifr6; u_int32_t flags6; memset(&ifr6, 0, sizeof(ifr6)); memcpy(&ifr6.ifr_addr, ai.ai_addr, ai.ai_addrlen); if (_ioctl(s, SIOCGIFAFLAG_IN6, &ifr6) == 0) { flags6 = ifr6.ifr_ifru.ifru_flags6; if ((flags6 & IN6_IFF_DEPRECATED)) aio->aio_srcflag |= AIO_SRCFLAG_DEPRECATED; } } #endif cleanup: _close(s); return; } static int matchlen(struct sockaddr *src, struct sockaddr *dst) { int match = 0; u_char *s, *d; u_char *lim, r; int addrlen; switch (src->sa_family) { #ifdef INET6 case AF_INET6: s = (u_char *)&((struct sockaddr_in6 *)src)->sin6_addr; d = (u_char *)&((struct sockaddr_in6 *)dst)->sin6_addr; addrlen = sizeof(struct in6_addr); lim = s + addrlen; break; #endif case AF_INET: s = (u_char *)&((struct sockaddr_in *)src)->sin_addr; d = (u_char *)&((struct sockaddr_in *)dst)->sin_addr; addrlen = sizeof(struct in_addr); lim = s + addrlen; break; default: return(0); } while (s < lim) if ((r = (*d++ ^ *s++)) != 0) { - while (r < addrlen * 8) { + while ((r & 0x80) == 0) { match++; r <<= 1; } break; } else match += 8; return(match); } static int comp_dst(const void *arg1, const void *arg2) { const struct ai_order *dst1 = arg1, *dst2 = arg2; /* * Rule 1: Avoid unusable destinations. * XXX: we currently do not consider if an appropriate route exists. */ if (dst1->aio_srcsa.sa_family != AF_UNSPEC && dst2->aio_srcsa.sa_family == AF_UNSPEC) { return(-1); } if (dst1->aio_srcsa.sa_family == AF_UNSPEC && dst2->aio_srcsa.sa_family != AF_UNSPEC) { return(1); } /* Rule 2: Prefer matching scope. */ if (dst1->aio_dstscope == dst1->aio_srcscope && dst2->aio_dstscope != dst2->aio_srcscope) { return(-1); } if (dst1->aio_dstscope != dst1->aio_srcscope && dst2->aio_dstscope == dst2->aio_srcscope) { return(1); } /* Rule 3: Avoid deprecated addresses. */ if (dst1->aio_srcsa.sa_family != AF_UNSPEC && dst2->aio_srcsa.sa_family != AF_UNSPEC) { if (!(dst1->aio_srcflag & AIO_SRCFLAG_DEPRECATED) && (dst2->aio_srcflag & AIO_SRCFLAG_DEPRECATED)) { return(-1); } if ((dst1->aio_srcflag & AIO_SRCFLAG_DEPRECATED) && !(dst2->aio_srcflag & AIO_SRCFLAG_DEPRECATED)) { return(1); } } /* Rule 4: Prefer home addresses. */ /* XXX: not implemented yet */ /* Rule 5: Prefer matching label. */ #ifdef INET6 if (dst1->aio_srcpolicy && dst1->aio_dstpolicy && dst1->aio_srcpolicy->pc_policy.label == dst1->aio_dstpolicy->pc_policy.label && (dst2->aio_srcpolicy == NULL || dst2->aio_dstpolicy == NULL || dst2->aio_srcpolicy->pc_policy.label != dst2->aio_dstpolicy->pc_policy.label)) { return(-1); } if (dst2->aio_srcpolicy && dst2->aio_dstpolicy && dst2->aio_srcpolicy->pc_policy.label == dst2->aio_dstpolicy->pc_policy.label && (dst1->aio_srcpolicy == NULL || dst1->aio_dstpolicy == NULL || dst1->aio_srcpolicy->pc_policy.label != dst1->aio_dstpolicy->pc_policy.label)) { return(1); } #endif /* Rule 6: Prefer higher precedence. */ #ifdef INET6 if (dst1->aio_dstpolicy && (dst2->aio_dstpolicy == NULL || dst1->aio_dstpolicy->pc_policy.preced > dst2->aio_dstpolicy->pc_policy.preced)) { return(-1); } if (dst2->aio_dstpolicy && (dst1->aio_dstpolicy == NULL || dst2->aio_dstpolicy->pc_policy.preced > dst1->aio_dstpolicy->pc_policy.preced)) { return(1); } #endif /* Rule 7: Prefer native transport. */ /* XXX: not implemented yet */ /* Rule 8: Prefer smaller scope. */ if (dst1->aio_dstscope >= 0 && dst1->aio_dstscope < dst2->aio_dstscope) { return(-1); } if (dst2->aio_dstscope >= 0 && dst2->aio_dstscope < dst1->aio_dstscope) { return(1); } /* * Rule 9: Use longest matching prefix. * We compare the match length in a same AF only. */ if (dst1->aio_ai->ai_addr->sa_family == dst2->aio_ai->ai_addr->sa_family && dst1->aio_ai->ai_addr->sa_family != AF_INET) { if (dst1->aio_matchlen > dst2->aio_matchlen) { return(-1); } if (dst1->aio_matchlen < dst2->aio_matchlen) { return(1); } } /* Rule 10: Otherwise, leave the order unchanged. */ /* * Note that qsort is unstable; so, we can't return zero and * expect the order to be unchanged. * That also means we can't depend on the current position of * dst2 being after dst1. We must enforce the initial order * with an explicit compare on the original position. * The qsort specification requires that "When the same objects * (consisting of width bytes, irrespective of their current * positions in the array) are passed more than once to the * comparison function, the results shall be consistent with one * another." * In other words, If A < B, then we must also return B > A. */ if (dst2->aio_initial_sequence < dst1->aio_initial_sequence) return(1); return(-1); } /* * Copy from scope.c. * XXX: we should standardize the functions and link them as standard * library. */ static int gai_addr2scopetype(struct sockaddr *sa) { #ifdef INET6 struct sockaddr_in6 *sa6; #endif struct sockaddr_in *sa4; switch(sa->sa_family) { #ifdef INET6 case AF_INET6: sa6 = (struct sockaddr_in6 *)sa; if (IN6_IS_ADDR_MULTICAST(&sa6->sin6_addr)) { /* just use the scope field of the multicast address */ return(sa6->sin6_addr.s6_addr[2] & 0x0f); } /* * Unicast addresses: map scope type to corresponding scope * value defined for multcast addresses. * XXX: hardcoded scope type values are bad... */ if (IN6_IS_ADDR_LOOPBACK(&sa6->sin6_addr)) return(1); /* node local scope */ if (IN6_IS_ADDR_LINKLOCAL(&sa6->sin6_addr)) return(2); /* link-local scope */ if (IN6_IS_ADDR_SITELOCAL(&sa6->sin6_addr)) return(5); /* site-local scope */ return(14); /* global scope */ break; #endif case AF_INET: /* * IPv4 pseudo scoping according to RFC 3484. */ sa4 = (struct sockaddr_in *)sa; /* IPv4 autoconfiguration addresses have link-local scope. */ if (((u_char *)&sa4->sin_addr)[0] == 169 && ((u_char *)&sa4->sin_addr)[1] == 254) return(2); /* Private addresses have site-local scope. */ if (((u_char *)&sa4->sin_addr)[0] == 10 || (((u_char *)&sa4->sin_addr)[0] == 172 && (((u_char *)&sa4->sin_addr)[1] & 0xf0) == 16) || (((u_char *)&sa4->sin_addr)[0] == 192 && ((u_char *)&sa4->sin_addr)[1] == 168)) return(14); /* XXX: It should be 5 unless NAT */ /* Loopback addresses have link-local scope. */ if (((u_char *)&sa4->sin_addr)[0] == 127) return(2); return(14); break; default: errno = EAFNOSUPPORT; /* is this a good error? */ return(-1); } } static int explore_copy(const struct addrinfo *pai, const struct addrinfo *src0, struct addrinfo **res) { int error; struct addrinfo sentinel, *cur; const struct addrinfo *src; error = 0; sentinel.ai_next = NULL; cur = &sentinel; for (src = src0; src != NULL; src = src->ai_next) { if (src->ai_family != pai->ai_family) continue; cur->ai_next = copy_ai(src); if (!cur->ai_next) { error = EAI_MEMORY; goto fail; } cur->ai_next->ai_socktype = pai->ai_socktype; cur->ai_next->ai_protocol = pai->ai_protocol; cur = cur->ai_next; } *res = sentinel.ai_next; return 0; fail: freeaddrinfo(sentinel.ai_next); return error; } /* * hostname == NULL. * passive socket -> anyaddr (0.0.0.0 or ::) * non-passive socket -> localhost (127.0.0.1 or ::1) */ static int explore_null(const struct addrinfo *pai, const char *servname, struct addrinfo **res) { int s; const struct afd *afd; struct addrinfo *ai; int error; *res = NULL; ai = NULL; if (pai->ai_family == PF_LOCAL) return (0); /* * filter out AFs that are not supported by the kernel * XXX errno? */ s = _socket(pai->ai_family, SOCK_DGRAM | SOCK_CLOEXEC, 0); if (s < 0) { if (errno != EMFILE) return 0; } else _close(s); afd = find_afd(pai->ai_family); if (afd == NULL) return 0; if (pai->ai_flags & AI_PASSIVE) { GET_AI(ai, afd, afd->a_addrany); GET_PORT(ai, servname); } else { GET_AI(ai, afd, afd->a_loopback); GET_PORT(ai, servname); } *res = ai; return 0; free: if (ai != NULL) freeaddrinfo(ai); return error; } /* * numeric hostname */ static int explore_numeric(const struct addrinfo *pai, const char *hostname, const char *servname, struct addrinfo **res, const char *canonname) { const struct afd *afd; struct addrinfo *ai, ai0; int error; char pton[PTON_MAX], path[PATH_MAX], *p; #ifdef CTASSERT CTASSERT(sizeofmember(struct sockaddr_un, sun_path) <= PATH_MAX); #endif *res = NULL; ai = NULL; afd = find_afd(pai->ai_family); if (afd == NULL) return 0; switch (afd->a_af) { case AF_LOCAL: if (hostname[0] != '/') ERR(EAI_NONAME); if (strlen(hostname) > afd->a_addrlen) ERR(EAI_MEMORY); /* NUL-termination does not need to be guaranteed. */ strncpy(path, hostname, afd->a_addrlen); p = &path[0]; break; case AF_INET: /* * RFC3493 requires getaddrinfo() to accept AF_INET formats * that are accepted by inet_addr() and its family. The * accepted forms includes the "classful" one, which inet_pton * does not accept. So we need to separate the case for * AF_INET. */ if (inet_aton(hostname, (struct in_addr *)pton) != 1) return 0; p = pton; break; default: if (inet_pton(afd->a_af, hostname, pton) != 1) { if (pai->ai_family != AF_INET6 || (pai->ai_flags & AI_V4MAPPED) != AI_V4MAPPED) return 0; if (inet_aton(hostname, (struct in_addr *)pton) != 1) return 0; afd = &afdl[N_INET]; ai0 = *pai; ai0.ai_family = AF_INET; pai = &ai0; } p = pton; break; } if (pai->ai_family == afd->a_af) { GET_AI(ai, afd, p); GET_PORT(ai, servname); if ((pai->ai_family == AF_INET || pai->ai_family == AF_INET6) && (pai->ai_flags & AI_CANONNAME)) { /* * Set the numeric address itself as the canonical * name, based on a clarification in RFC3493. */ GET_CANONNAME(ai, canonname); } } else { /* * XXX: This should not happen since we already matched the AF * by find_afd. */ ERR(EAI_FAMILY); } *res = ai; return 0; free: bad: if (ai != NULL) freeaddrinfo(ai); return error; } /* * numeric hostname with scope */ static int explore_numeric_scope(const struct addrinfo *pai, const char *hostname, const char *servname, struct addrinfo **res) { #if !defined(SCOPE_DELIMITER) || !defined(INET6) return explore_numeric(pai, hostname, servname, res, hostname); #else const struct afd *afd; struct addrinfo *cur; int error; char *cp, *hostname2 = NULL, *scope, *addr; struct sockaddr_in6 *sin6; afd = find_afd(pai->ai_family); if (afd == NULL) return 0; if (!afd->a_scoped) return explore_numeric(pai, hostname, servname, res, hostname); cp = strchr(hostname, SCOPE_DELIMITER); if (cp == NULL) return explore_numeric(pai, hostname, servname, res, hostname); /* * Handle special case of */ hostname2 = strdup(hostname); if (hostname2 == NULL) return EAI_MEMORY; /* terminate at the delimiter */ hostname2[cp - hostname] = '\0'; addr = hostname2; scope = cp + 1; error = explore_numeric(pai, addr, servname, res, hostname); if (error == 0) { u_int32_t scopeid; for (cur = *res; cur; cur = cur->ai_next) { if (cur->ai_family != AF_INET6) continue; sin6 = (struct sockaddr_in6 *)(void *)cur->ai_addr; if (ip6_str2scopeid(scope, sin6, &scopeid) == -1) { free(hostname2); freeaddrinfo(*res); *res = NULL; return(EAI_NONAME); /* XXX: is return OK? */ } sin6->sin6_scope_id = scopeid; } } free(hostname2); if (error && *res) { freeaddrinfo(*res); *res = NULL; } return error; #endif } static int get_canonname(const struct addrinfo *pai, struct addrinfo *ai, const char *str) { if ((pai->ai_flags & AI_CANONNAME) != 0) { ai->ai_canonname = strdup(str); if (ai->ai_canonname == NULL) return EAI_MEMORY; } return 0; } static struct addrinfo * get_ai(const struct addrinfo *pai, const struct afd *afd, const char *addr) { char *p; struct addrinfo *ai; #ifdef INET6 struct in6_addr mapaddr; if (afd->a_af == AF_INET && (pai->ai_flags & AI_V4MAPPED) != 0) { afd = &afdl[N_INET6]; _map_v4v6_address(addr, (char *)&mapaddr); addr = (char *)&mapaddr; } #endif ai = (struct addrinfo *)malloc(sizeof(struct addrinfo) + (afd->a_socklen)); if (ai == NULL) return NULL; memcpy(ai, pai, sizeof(struct addrinfo)); ai->ai_addr = (struct sockaddr *)(void *)(ai + 1); memset(ai->ai_addr, 0, (size_t)afd->a_socklen); ai->ai_addr->sa_len = afd->a_socklen; ai->ai_addrlen = afd->a_socklen; if (ai->ai_family == PF_LOCAL) { size_t n = strnlen(addr, afd->a_addrlen); ai->ai_addrlen -= afd->a_addrlen - n; ai->ai_addr->sa_len -= afd->a_addrlen - n; } ai->ai_addr->sa_family = ai->ai_family = afd->a_af; p = (char *)(void *)(ai->ai_addr); memcpy(p + afd->a_off, addr, (size_t)afd->a_addrlen); return ai; } /* XXX need to malloc() the same way we do from other functions! */ static struct addrinfo * copy_ai(const struct addrinfo *pai) { struct addrinfo *ai; size_t l; l = sizeof(*ai) + pai->ai_addrlen; if ((ai = (struct addrinfo *)malloc(l)) == NULL) return NULL; memset(ai, 0, l); memcpy(ai, pai, sizeof(*ai)); ai->ai_addr = (struct sockaddr *)(void *)(ai + 1); memcpy(ai->ai_addr, pai->ai_addr, pai->ai_addrlen); if (pai->ai_canonname) { l = strlen(pai->ai_canonname) + 1; if ((ai->ai_canonname = malloc(l)) == NULL) { free(ai); return NULL; } strlcpy(ai->ai_canonname, pai->ai_canonname, l); } else { /* just to make sure */ ai->ai_canonname = NULL; } ai->ai_next = NULL; return ai; } static int get_portmatch(const struct addrinfo *ai, const char *servname) { /* get_port does not touch first argument when matchonly == 1. */ /* LINTED const cast */ return get_port((struct addrinfo *)ai, servname, 1); } static int get_port(struct addrinfo *ai, const char *servname, int matchonly) { const char *proto; struct servent *sp; int port, error; int allownumeric; if (servname == NULL) return 0; switch (ai->ai_family) { case AF_LOCAL: /* AF_LOCAL ignores servname silently. */ return (0); case AF_INET: #ifdef AF_INET6 case AF_INET6: #endif break; default: return 0; } switch (ai->ai_socktype) { case SOCK_RAW: return EAI_SERVICE; case SOCK_DGRAM: case SOCK_STREAM: case SOCK_SEQPACKET: allownumeric = 1; break; case ANY: switch (ai->ai_family) { case AF_INET: #ifdef AF_INET6 case AF_INET6: #endif allownumeric = 1; break; default: allownumeric = 0; break; } break; default: return EAI_SOCKTYPE; } error = str2number(servname, &port); if (error == 0) { if (!allownumeric) return EAI_SERVICE; if (port < 0 || port > 65535) return EAI_SERVICE; port = htons(port); } else { if (ai->ai_flags & AI_NUMERICSERV) return EAI_NONAME; switch (ai->ai_protocol) { case IPPROTO_UDP: proto = "udp"; break; case IPPROTO_TCP: proto = "tcp"; break; case IPPROTO_SCTP: proto = "sctp"; break; case IPPROTO_UDPLITE: proto = "udplite"; break; default: proto = NULL; break; } if ((sp = getservbyname(servname, proto)) == NULL) return EAI_SERVICE; port = sp->s_port; } if (!matchonly) { switch (ai->ai_family) { case AF_INET: ((struct sockaddr_in *)(void *) ai->ai_addr)->sin_port = port; break; #ifdef INET6 case AF_INET6: ((struct sockaddr_in6 *)(void *) ai->ai_addr)->sin6_port = port; break; #endif } } return 0; } static const struct afd * find_afd(int af) { const struct afd *afd; if (af == PF_UNSPEC) return NULL; for (afd = afdl; afd->a_af; afd++) { if (afd->a_af == af) return afd; } return NULL; } /* * RFC 3493: AI_ADDRCONFIG check. Determines which address families are * configured on the local system and correlates with pai->ai_family value. * If an address family is not configured on the system, it will not be * queried for. For this purpose, loopback addresses are not considered * configured addresses. * * XXX PF_UNSPEC -> PF_INET6 + PF_INET mapping needs to be in sync with * _dns_getaddrinfo. */ static int addrconfig(struct addrinfo *pai) { struct ifaddrs *ifaddrs, *ifa; struct sockaddr_in *sin; #ifdef INET6 struct sockaddr_in6 *sin6; #endif int seen_inet = 0, seen_inet6 = 0; if (getifaddrs(&ifaddrs) != 0) return (0); for (ifa = ifaddrs; ifa != NULL; ifa = ifa->ifa_next) { if (ifa->ifa_addr == NULL || (ifa->ifa_flags & IFF_UP) == 0) continue; switch (ifa->ifa_addr->sa_family) { case AF_INET: if (seen_inet) continue; sin = (struct sockaddr_in *)(ifa->ifa_addr); if (htonl(sin->sin_addr.s_addr) == INADDR_LOOPBACK) continue; seen_inet = 1; break; #ifdef INET6 case AF_INET6: if (seen_inet6) continue; sin6 = (struct sockaddr_in6 *)(ifa->ifa_addr); if (IN6_IS_ADDR_LOOPBACK(&sin6->sin6_addr)) continue; if ((ifa->ifa_flags & IFT_LOOP) != 0 && IN6_IS_ADDR_LINKLOCAL(&sin6->sin6_addr)) continue; if (is_ifdisabled(ifa->ifa_name)) continue; seen_inet6 = 1; break; #endif } } freeifaddrs(ifaddrs); switch(pai->ai_family) { case AF_INET6: return (seen_inet6); case AF_INET: return (seen_inet); case AF_UNSPEC: if (seen_inet == seen_inet6) return (seen_inet); pai->ai_family = seen_inet ? AF_INET : AF_INET6; return (1); } return (1); } #ifdef INET6 static int is_ifdisabled(char *name) { struct in6_ndireq nd; int fd; if ((fd = _socket(AF_INET6, SOCK_DGRAM | SOCK_CLOEXEC, 0)) < 0) return (-1); memset(&nd, 0, sizeof(nd)); strlcpy(nd.ifname, name, sizeof(nd.ifname)); if (_ioctl(fd, SIOCGIFINFO_IN6, &nd) < 0) { _close(fd); return (-1); } _close(fd); return ((nd.ndi.flags & ND6_IFF_IFDISABLED) != 0); } /* convert a string to a scope identifier. XXX: IPv6 specific */ static int ip6_str2scopeid(char *scope, struct sockaddr_in6 *sin6, u_int32_t *scopeid) { u_long lscopeid; struct in6_addr *a6; char *ep; a6 = &sin6->sin6_addr; /* empty scopeid portion is invalid */ if (*scope == '\0') return -1; if (IN6_IS_ADDR_LINKLOCAL(a6) || IN6_IS_ADDR_MC_LINKLOCAL(a6) || IN6_IS_ADDR_MC_NODELOCAL(a6)) { /* * We currently assume a one-to-one mapping between links * and interfaces, so we simply use interface indices for * like-local scopes. */ *scopeid = if_nametoindex(scope); if (*scopeid == 0) goto trynumeric; return 0; } /* still unclear about literal, allow numeric only - placeholder */ if (IN6_IS_ADDR_SITELOCAL(a6) || IN6_IS_ADDR_MC_SITELOCAL(a6)) goto trynumeric; if (IN6_IS_ADDR_MC_ORGLOCAL(a6)) goto trynumeric; else goto trynumeric; /* global */ /* try to convert to a numeric id as a last resort */ trynumeric: errno = 0; lscopeid = strtoul(scope, &ep, 10); *scopeid = (u_int32_t)(lscopeid & 0xffffffffUL); if (errno == 0 && ep && *ep == '\0' && *scopeid == lscopeid) return 0; else return -1; } #endif #ifdef NS_CACHING static int addrinfo_id_func(char *buffer, size_t *buffer_size, va_list ap, void *cache_mdata) { res_state statp; u_long res_options; const int op_id = 0; /* identifies the getaddrinfo for the cache */ char *hostname; struct addrinfo *hints; char *p; int ai_flags, ai_family, ai_socktype, ai_protocol; size_t desired_size, size; statp = __res_state(); res_options = statp->options & (RES_RECURSE | RES_DEFNAMES | RES_DNSRCH | RES_NOALIASES | RES_USE_INET6); hostname = va_arg(ap, char *); hints = va_arg(ap, struct addrinfo *); desired_size = sizeof(res_options) + sizeof(int) + sizeof(int) * 4; if (hostname != NULL) { size = strlen(hostname); desired_size += size + 1; } else size = 0; if (desired_size > *buffer_size) { *buffer_size = desired_size; return (NS_RETURN); } if (hints == NULL) ai_flags = ai_family = ai_socktype = ai_protocol = 0; else { ai_flags = hints->ai_flags; ai_family = hints->ai_family; ai_socktype = hints->ai_socktype; ai_protocol = hints->ai_protocol; } p = buffer; memcpy(p, &res_options, sizeof(res_options)); p += sizeof(res_options); memcpy(p, &op_id, sizeof(int)); p += sizeof(int); memcpy(p, &ai_flags, sizeof(int)); p += sizeof(int); memcpy(p, &ai_family, sizeof(int)); p += sizeof(int); memcpy(p, &ai_socktype, sizeof(int)); p += sizeof(int); memcpy(p, &ai_protocol, sizeof(int)); p += sizeof(int); if (hostname != NULL) memcpy(p, hostname, size); *buffer_size = desired_size; return (NS_SUCCESS); } static int addrinfo_marshal_func(char *buffer, size_t *buffer_size, void *retval, va_list ap, void *cache_mdata) { struct addrinfo *ai, *cai; char *p; size_t desired_size, size, ai_size; ai = *((struct addrinfo **)retval); desired_size = sizeof(size_t); ai_size = 0; for (cai = ai; cai != NULL; cai = cai->ai_next) { desired_size += sizeof(struct addrinfo) + cai->ai_addrlen; if (cai->ai_canonname != NULL) desired_size += sizeof(size_t) + strlen(cai->ai_canonname); ++ai_size; } if (desired_size > *buffer_size) { /* this assignment is here for future use */ errno = ERANGE; *buffer_size = desired_size; return (NS_RETURN); } memset(buffer, 0, desired_size); p = buffer; memcpy(p, &ai_size, sizeof(size_t)); p += sizeof(size_t); for (cai = ai; cai != NULL; cai = cai->ai_next) { memcpy(p, cai, sizeof(struct addrinfo)); p += sizeof(struct addrinfo); memcpy(p, cai->ai_addr, cai->ai_addrlen); p += cai->ai_addrlen; if (cai->ai_canonname != NULL) { size = strlen(cai->ai_canonname); memcpy(p, &size, sizeof(size_t)); p += sizeof(size_t); memcpy(p, cai->ai_canonname, size); p += size; } } return (NS_SUCCESS); } static int addrinfo_unmarshal_func(char *buffer, size_t buffer_size, void *retval, va_list ap, void *cache_mdata) { struct addrinfo new_ai, *result, *sentinel, *lasts; char *p; size_t ai_size, ai_i, size; p = buffer; memcpy(&ai_size, p, sizeof(size_t)); p += sizeof(size_t); result = NULL; lasts = NULL; for (ai_i = 0; ai_i < ai_size; ++ai_i) { memcpy(&new_ai, p, sizeof(struct addrinfo)); p += sizeof(struct addrinfo); size = new_ai.ai_addrlen + sizeof(struct addrinfo) + _ALIGNBYTES; sentinel = (struct addrinfo *)malloc(size); memset(sentinel, 0, size); memcpy(sentinel, &new_ai, sizeof(struct addrinfo)); sentinel->ai_addr = (struct sockaddr *)_ALIGN((char *)sentinel + sizeof(struct addrinfo)); memcpy(sentinel->ai_addr, p, new_ai.ai_addrlen); p += new_ai.ai_addrlen; if (new_ai.ai_canonname != NULL) { memcpy(&size, p, sizeof(size_t)); p += sizeof(size_t); sentinel->ai_canonname = (char *)malloc(size + 1); memset(sentinel->ai_canonname, 0, size + 1); memcpy(sentinel->ai_canonname, p, size); p += size; } if (result == NULL) { result = sentinel; lasts = sentinel; } else { lasts->ai_next = sentinel; lasts = sentinel; } } *((struct addrinfo **)retval) = result; return (NS_SUCCESS); } #endif /* NS_CACHING */ /* * FQDN hostname, DNS lookup */ static int explore_fqdn(const struct addrinfo *pai, const char *hostname, const char *servname, struct addrinfo **res) { struct addrinfo *result; struct addrinfo *cur; int error = 0; #ifdef NS_CACHING static const nss_cache_info cache_info = NS_COMMON_CACHE_INFO_INITIALIZER( hosts, NULL, addrinfo_id_func, addrinfo_marshal_func, addrinfo_unmarshal_func); #endif static const ns_dtab dtab[] = { NS_FILES_CB(_files_getaddrinfo, NULL) { NSSRC_DNS, _dns_getaddrinfo, NULL }, /* force -DHESIOD */ NS_NIS_CB(_yp_getaddrinfo, NULL) #ifdef NS_CACHING NS_CACHE_CB(&cache_info) #endif { 0 } }; result = NULL; /* * if the servname does not match socktype/protocol, ignore it. */ if (get_portmatch(pai, servname) != 0) return 0; switch (_nsdispatch(&result, dtab, NSDB_HOSTS, "getaddrinfo", default_dns_files, hostname, pai)) { case NS_TRYAGAIN: error = EAI_AGAIN; goto free; case NS_UNAVAIL: error = EAI_FAIL; goto free; case NS_NOTFOUND: error = EAI_NONAME; goto free; case NS_SUCCESS: error = 0; for (cur = result; cur; cur = cur->ai_next) { GET_PORT(cur, servname); /* canonname should be filled already */ } break; } *res = result; return 0; free: if (result) freeaddrinfo(result); return error; } #ifdef DEBUG static const char AskedForGot[] = "gethostby*.getanswer: asked for \"%s\", got \"%s\""; #endif static struct addrinfo * getanswer(const querybuf *answer, int anslen, const char *qname, int qtype, const struct addrinfo *pai, res_state res) { struct addrinfo sentinel, *cur; struct addrinfo ai; const struct afd *afd; char *canonname; const HEADER *hp; const u_char *cp; int n; const u_char *eom; char *bp, *ep; int type, class, ancount, qdcount; int haveanswer, had_error; char tbuf[MAXDNAME]; int (*name_ok)(const char *); char hostbuf[8*1024]; memset(&sentinel, 0, sizeof(sentinel)); cur = &sentinel; canonname = NULL; eom = answer->buf + anslen; switch (qtype) { case T_A: case T_AAAA: case T_ANY: /*use T_ANY only for T_A/T_AAAA lookup*/ name_ok = res_hnok; break; default: return (NULL); /* XXX should be abort(); */ } /* * find first satisfactory answer */ hp = &answer->hdr; ancount = ntohs(hp->ancount); qdcount = ntohs(hp->qdcount); bp = hostbuf; ep = hostbuf + sizeof hostbuf; cp = answer->buf + HFIXEDSZ; if (qdcount != 1) { RES_SET_H_ERRNO(res, NO_RECOVERY); return (NULL); } n = dn_expand(answer->buf, eom, cp, bp, ep - bp); if ((n < 0) || !(*name_ok)(bp)) { RES_SET_H_ERRNO(res, NO_RECOVERY); return (NULL); } cp += n + QFIXEDSZ; if (qtype == T_A || qtype == T_AAAA || qtype == T_ANY) { /* res_send() has already verified that the query name is the * same as the one we sent; this just gets the expanded name * (i.e., with the succeeding search-domain tacked on). */ n = strlen(bp) + 1; /* for the \0 */ if (n >= MAXHOSTNAMELEN) { RES_SET_H_ERRNO(res, NO_RECOVERY); return (NULL); } canonname = bp; bp += n; /* The qname can be abbreviated, but h_name is now absolute. */ qname = canonname; } haveanswer = 0; had_error = 0; while (ancount-- > 0 && cp < eom && !had_error) { n = dn_expand(answer->buf, eom, cp, bp, ep - bp); if ((n < 0) || !(*name_ok)(bp)) { had_error++; continue; } cp += n; /* name */ type = _getshort(cp); cp += INT16SZ; /* type */ class = _getshort(cp); cp += INT16SZ + INT32SZ; /* class, TTL */ n = _getshort(cp); cp += INT16SZ; /* len */ if (class != C_IN) { /* XXX - debug? syslog? */ cp += n; continue; /* XXX - had_error++ ? */ } if ((qtype == T_A || qtype == T_AAAA || qtype == T_ANY) && type == T_CNAME) { n = dn_expand(answer->buf, eom, cp, tbuf, sizeof tbuf); if ((n < 0) || !(*name_ok)(tbuf)) { had_error++; continue; } cp += n; /* Get canonical name. */ n = strlen(tbuf) + 1; /* for the \0 */ if (n > ep - bp || n >= MAXHOSTNAMELEN) { had_error++; continue; } strlcpy(bp, tbuf, ep - bp); canonname = bp; bp += n; continue; } if (qtype == T_ANY) { if (!(type == T_A || type == T_AAAA)) { cp += n; continue; } } else if (type != qtype) { #ifdef DEBUG if (type != T_KEY && type != T_SIG && type != ns_t_dname) syslog(LOG_NOTICE|LOG_AUTH, "gethostby*.getanswer: asked for \"%s %s %s\", got type \"%s\"", qname, p_class(C_IN), p_type(qtype), p_type(type)); #endif cp += n; continue; /* XXX - had_error++ ? */ } switch (type) { case T_A: case T_AAAA: if (strcasecmp(canonname, bp) != 0) { #ifdef DEBUG syslog(LOG_NOTICE|LOG_AUTH, AskedForGot, canonname, bp); #endif cp += n; continue; /* XXX - had_error++ ? */ } if (type == T_A && n != INADDRSZ) { cp += n; continue; } if (type == T_AAAA && n != IN6ADDRSZ) { cp += n; continue; } #ifdef FILTER_V4MAPPED if (type == T_AAAA) { struct in6_addr in6; memcpy(&in6, cp, sizeof(in6)); if (IN6_IS_ADDR_V4MAPPED(&in6)) { cp += n; continue; } } #endif if (!haveanswer) { int nn; canonname = bp; nn = strlen(bp) + 1; /* for the \0 */ bp += nn; } /* don't overwrite pai */ ai = *pai; ai.ai_family = (type == T_A) ? AF_INET : AF_INET6; afd = find_afd(ai.ai_family); if (afd == NULL) { cp += n; continue; } cur->ai_next = get_ai(&ai, afd, (const char *)cp); if (cur->ai_next == NULL) had_error++; while (cur && cur->ai_next) cur = cur->ai_next; cp += n; break; default: abort(); } if (!had_error) haveanswer++; } if (haveanswer) { #if defined(RESOLVSORT) /* * We support only IPv4 address for backward * compatibility against gethostbyname(3). */ if (res->nsort && qtype == T_A) { if (addr4sort(&sentinel, res) < 0) { freeaddrinfo(sentinel.ai_next); RES_SET_H_ERRNO(res, NO_RECOVERY); return NULL; } } #endif /*RESOLVSORT*/ if (!canonname) (void)get_canonname(pai, sentinel.ai_next, qname); else (void)get_canonname(pai, sentinel.ai_next, canonname); RES_SET_H_ERRNO(res, NETDB_SUCCESS); return sentinel.ai_next; } /* * We could have walked a CNAME chain, but the ultimate target * may not have what we looked for. */ RES_SET_H_ERRNO(res, ntohs(hp->ancount) > 0 ? NO_DATA : NO_RECOVERY); return NULL; } #ifdef RESOLVSORT struct addr_ptr { struct addrinfo *ai; int aval; }; static int addr4sort(struct addrinfo *sentinel, res_state res) { struct addrinfo *ai; struct addr_ptr *addrs, addr; struct sockaddr_in *sin; int naddrs, i, j; int needsort = 0; if (!sentinel) return -1; naddrs = 0; for (ai = sentinel->ai_next; ai; ai = ai->ai_next) naddrs++; if (naddrs < 2) return 0; /* We don't need sorting. */ if ((addrs = malloc(sizeof(struct addr_ptr) * naddrs)) == NULL) return -1; i = 0; for (ai = sentinel->ai_next; ai; ai = ai->ai_next) { sin = (struct sockaddr_in *)ai->ai_addr; for (j = 0; (unsigned)j < res->nsort; j++) { if (res->sort_list[j].addr.s_addr == (sin->sin_addr.s_addr & res->sort_list[j].mask)) break; } addrs[i].ai = ai; addrs[i].aval = j; if (needsort == 0 && i > 0 && j < addrs[i - 1].aval) needsort = i; i++; } if (!needsort) { free(addrs); return 0; } while (needsort < naddrs) { for (j = needsort - 1; j >= 0; j--) { if (addrs[j].aval > addrs[j+1].aval) { addr = addrs[j]; addrs[j] = addrs[j + 1]; addrs[j + 1] = addr; } else break; } needsort++; } ai = sentinel; for (i = 0; i < naddrs; ++i) { ai->ai_next = addrs[i].ai; ai = ai->ai_next; } ai->ai_next = NULL; free(addrs); return 0; } #endif /*RESOLVSORT*/ /*ARGSUSED*/ static int _dns_getaddrinfo(void *rv, void *cb_data, va_list ap) { struct addrinfo *ai, ai0; querybuf *buf, *buf2; const char *hostname; const struct addrinfo *pai; struct addrinfo sentinel, *cur; struct res_target q, q2; res_state res; ai = NULL; hostname = va_arg(ap, char *); pai = va_arg(ap, const struct addrinfo *); memset(&q, 0, sizeof(q)); memset(&q2, 0, sizeof(q2)); memset(&sentinel, 0, sizeof(sentinel)); cur = &sentinel; res = __res_state(); buf = malloc(sizeof(*buf)); if (!buf) { RES_SET_H_ERRNO(res, NETDB_INTERNAL); return NS_NOTFOUND; } buf2 = malloc(sizeof(*buf2)); if (!buf2) { free(buf); RES_SET_H_ERRNO(res, NETDB_INTERNAL); return NS_NOTFOUND; } if (pai->ai_family == AF_INET6 && (pai->ai_flags & AI_V4MAPPED) == AI_V4MAPPED) { ai0 = *pai; ai0.ai_family = AF_UNSPEC; pai = &ai0; } switch (pai->ai_family) { case AF_UNSPEC: q.name = hostname; q.qclass = C_IN; q.qtype = T_A; q.answer = buf->buf; q.anslen = sizeof(buf->buf); q.next = &q2; q2.name = hostname; q2.qclass = C_IN; q2.qtype = T_AAAA; q2.answer = buf2->buf; q2.anslen = sizeof(buf2->buf); break; case AF_INET: q.name = hostname; q.qclass = C_IN; q.qtype = T_A; q.answer = buf->buf; q.anslen = sizeof(buf->buf); break; case AF_INET6: q.name = hostname; q.qclass = C_IN; q.qtype = T_AAAA; q.answer = buf->buf; q.anslen = sizeof(buf->buf); break; default: free(buf); free(buf2); return NS_UNAVAIL; } if ((res->options & RES_INIT) == 0 && res_ninit(res) == -1) { RES_SET_H_ERRNO(res, NETDB_INTERNAL); free(buf); free(buf2); return NS_NOTFOUND; } if (res_searchN(hostname, &q, res) < 0) { free(buf); free(buf2); return NS_NOTFOUND; } /* prefer IPv6 */ if (q.next) { ai = getanswer(buf2, q2.n, q2.name, q2.qtype, pai, res); if (ai != NULL) { cur->ai_next = ai; while (cur && cur->ai_next) cur = cur->ai_next; } } if (ai == NULL || pai->ai_family != AF_UNSPEC || (pai->ai_flags & (AI_ALL | AI_V4MAPPED)) != AI_V4MAPPED) { ai = getanswer(buf, q.n, q.name, q.qtype, pai, res); if (ai != NULL) cur->ai_next = ai; } free(buf); free(buf2); if (sentinel.ai_next == NULL) switch (res->res_h_errno) { case HOST_NOT_FOUND: case NO_DATA: return NS_NOTFOUND; case TRY_AGAIN: return NS_TRYAGAIN; default: return NS_UNAVAIL; } *((struct addrinfo **)rv) = sentinel.ai_next; return NS_SUCCESS; } static void _sethtent(FILE **hostf) { if (!*hostf) *hostf = fopen(_PATH_HOSTS, "re"); else rewind(*hostf); } static void _endhtent(FILE **hostf) { if (*hostf) { (void) fclose(*hostf); *hostf = NULL; } } static struct addrinfo * _gethtent(FILE **hostf, const char *name, const struct addrinfo *pai) { char *p; char *cp, *tname, *cname; struct addrinfo hints, *res0, *res; int error; const char *addr; char hostbuf[8*1024]; if (!*hostf && !(*hostf = fopen(_PATH_HOSTS, "re"))) return (NULL); again: if (!(p = fgets(hostbuf, sizeof hostbuf, *hostf))) return (NULL); if (*p == '#') goto again; cp = strpbrk(p, "#\n"); if (cp != NULL) *cp = '\0'; if (!(cp = strpbrk(p, " \t"))) goto again; *cp++ = '\0'; addr = p; cname = NULL; /* if this is not something we're looking for, skip it. */ while (cp && *cp) { if (*cp == ' ' || *cp == '\t') { cp++; continue; } tname = cp; if (cname == NULL) cname = cp; if ((cp = strpbrk(cp, " \t")) != NULL) *cp++ = '\0'; if (strcasecmp(name, tname) == 0) goto found; } goto again; found: /* we should not glob socktype/protocol here */ memset(&hints, 0, sizeof(hints)); hints.ai_family = pai->ai_family; hints.ai_socktype = SOCK_DGRAM; hints.ai_protocol = 0; hints.ai_flags = AI_NUMERICHOST; if (pai->ai_family == AF_INET6 && (pai->ai_flags & AI_V4MAPPED) == AI_V4MAPPED) hints.ai_flags |= AI_V4MAPPED; error = getaddrinfo(addr, "0", &hints, &res0); if (error) goto again; #ifdef FILTER_V4MAPPED /* XXX should check all items in the chain */ if (res0->ai_family == AF_INET6 && IN6_IS_ADDR_V4MAPPED(&((struct sockaddr_in6 *)res0->ai_addr)->sin6_addr)) { freeaddrinfo(res0); goto again; } #endif for (res = res0; res; res = res->ai_next) { /* cover it up */ res->ai_flags = pai->ai_flags; res->ai_socktype = pai->ai_socktype; res->ai_protocol = pai->ai_protocol; if (pai->ai_flags & AI_CANONNAME) { if (get_canonname(pai, res, cname) != 0) { freeaddrinfo(res0); goto again; } } } return res0; } static struct addrinfo * _getht(FILE **hostf, const char *name, const struct addrinfo *pai, struct addrinfo *cur) { struct addrinfo *p; while ((p = _gethtent(hostf, name, pai)) != NULL) { cur->ai_next = p; while (cur && cur->ai_next) cur = cur->ai_next; } return (cur); } /*ARGSUSED*/ static int _files_getaddrinfo(void *rv, void *cb_data, va_list ap) { const char *name; const struct addrinfo *pai; struct addrinfo sentinel, *cur; FILE *hostf = NULL; name = va_arg(ap, char *); pai = va_arg(ap, struct addrinfo *); memset(&sentinel, 0, sizeof(sentinel)); cur = &sentinel; _sethtent(&hostf); if (pai->ai_family == AF_INET6 && (pai->ai_flags & (AI_ALL | AI_V4MAPPED)) == AI_V4MAPPED) { struct addrinfo ai0 = *pai; ai0.ai_flags &= ~AI_V4MAPPED; cur = _getht(&hostf, name, &ai0, cur); if (sentinel.ai_next == NULL) { _sethtent(&hostf); ai0.ai_flags |= AI_V4MAPPED; cur = _getht(&hostf, name, &ai0, cur); } } else cur = _getht(&hostf, name, pai, cur); _endhtent(&hostf); *((struct addrinfo **)rv) = sentinel.ai_next; if (sentinel.ai_next == NULL) return NS_NOTFOUND; return NS_SUCCESS; } #ifdef YP /*ARGSUSED*/ static struct addrinfo * _yphostent(char *line, const struct addrinfo *pai) { struct addrinfo sentinel, *cur; struct addrinfo hints, *res, *res0; int error; char *p = line; const char *addr, *canonname; char *nextline; char *cp; addr = canonname = NULL; memset(&sentinel, 0, sizeof(sentinel)); cur = &sentinel; nextline: /* terminate line */ cp = strchr(p, '\n'); if (cp) { *cp++ = '\0'; nextline = cp; } else nextline = NULL; cp = strpbrk(p, " \t"); if (cp == NULL) { if (canonname == NULL) return (NULL); else goto done; } *cp++ = '\0'; addr = p; while (cp && *cp) { if (*cp == ' ' || *cp == '\t') { cp++; continue; } if (!canonname) canonname = cp; if ((cp = strpbrk(cp, " \t")) != NULL) *cp++ = '\0'; } hints = *pai; hints.ai_flags = AI_NUMERICHOST; if (pai->ai_family == AF_INET6 && (pai->ai_flags & AI_V4MAPPED) == AI_V4MAPPED) hints.ai_flags |= AI_V4MAPPED; error = getaddrinfo(addr, NULL, &hints, &res0); if (error == 0) { for (res = res0; res; res = res->ai_next) { /* cover it up */ res->ai_flags = pai->ai_flags; if (pai->ai_flags & AI_CANONNAME) (void)get_canonname(pai, res, canonname); } } else res0 = NULL; if (res0) { cur->ai_next = res0; while (cur && cur->ai_next) cur = cur->ai_next; } if (nextline) { p = nextline; goto nextline; } done: return sentinel.ai_next; } /*ARGSUSED*/ static int _yp_getaddrinfo(void *rv, void *cb_data, va_list ap) { struct addrinfo sentinel, *cur; struct addrinfo *ai = NULL; char *ypbuf; int ypbuflen, r; const char *name; const struct addrinfo *pai; char *ypdomain; if (_yp_check(&ypdomain) == 0) return NS_UNAVAIL; name = va_arg(ap, char *); pai = va_arg(ap, const struct addrinfo *); memset(&sentinel, 0, sizeof(sentinel)); cur = &sentinel; /* ipnodes.byname can hold both IPv4/v6 */ r = yp_match(ypdomain, "ipnodes.byname", name, (int)strlen(name), &ypbuf, &ypbuflen); if (r == 0) { ai = _yphostent(ypbuf, pai); if (ai) { cur->ai_next = ai; while (cur && cur->ai_next) cur = cur->ai_next; } free(ypbuf); } if (ai != NULL) { struct sockaddr_in6 *sin6; switch (ai->ai_family) { case AF_INET: goto done; case AF_INET6: sin6 = (struct sockaddr_in6 *)ai->ai_addr; if (IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)) goto done; break; } } /* hosts.byname is only for IPv4 (Solaris8) */ if (pai->ai_family == AF_UNSPEC || pai->ai_family == AF_INET || ((pai->ai_family == AF_INET6 && (pai->ai_flags & AI_V4MAPPED) == AI_V4MAPPED) && (ai == NULL || (pai->ai_flags & AI_ALL) == AI_ALL))) { r = yp_match(ypdomain, "hosts.byname", name, (int)strlen(name), &ypbuf, &ypbuflen); if (r == 0) { struct addrinfo ai4; ai4 = *pai; if (pai->ai_family == AF_UNSPEC) ai4.ai_family = AF_INET; ai = _yphostent(ypbuf, &ai4); if (ai) { cur->ai_next = ai; while (cur && cur->ai_next) cur = cur->ai_next; } free(ypbuf); } } done: if (sentinel.ai_next == NULL) { RES_SET_H_ERRNO(__res_state(), HOST_NOT_FOUND); return NS_NOTFOUND; } *((struct addrinfo **)rv) = sentinel.ai_next; return NS_SUCCESS; } #endif /* resolver logic */ /* * Formulate a normal query, send, and await answer. * Returned answer is placed in supplied buffer "answer". * Perform preliminary check of answer, returning success only * if no error is indicated and the answer count is nonzero. * Return the size of the response on success, -1 on error. * Error number is left in h_errno. * * Caller must parse answer and determine whether it answers the question. */ static int res_queryN(const char *name, struct res_target *target, res_state res) { u_char *buf; HEADER *hp; int n; u_int oflags; struct res_target *t; int rcode; int ancount; rcode = NOERROR; ancount = 0; buf = malloc(MAXPACKET); if (!buf) { RES_SET_H_ERRNO(res, NETDB_INTERNAL); return -1; } for (t = target; t; t = t->next) { int class, type; u_char *answer; int anslen; hp = (HEADER *)(void *)t->answer; /* make it easier... */ class = t->qclass; type = t->qtype; answer = t->answer; anslen = t->anslen; oflags = res->_flags; again: hp->rcode = NOERROR; /* default */ #ifdef DEBUG if (res->options & RES_DEBUG) printf(";; res_query(%s, %d, %d)\n", name, class, type); #endif n = res_nmkquery(res, QUERY, name, class, type, NULL, 0, NULL, buf, MAXPACKET); if (n > 0 && (res->_flags & RES_F_EDNS0ERR) == 0 && (res->options & (RES_USE_EDNS0|RES_USE_DNSSEC)) != 0U) n = res_nopt(res, n, buf, MAXPACKET, anslen); if (n <= 0) { #ifdef DEBUG if (res->options & RES_DEBUG) printf(";; res_query: mkquery failed\n"); #endif free(buf); RES_SET_H_ERRNO(res, NO_RECOVERY); return (n); } n = res_nsend(res, buf, n, answer, anslen); if (n < 0) { /* * if the query choked with EDNS0, retry * without EDNS0 */ if ((res->options & (RES_USE_EDNS0|RES_USE_DNSSEC)) != 0U && ((oflags ^ res->_flags) & RES_F_EDNS0ERR) != 0) { res->_flags |= RES_F_EDNS0ERR; if (res->options & RES_DEBUG) printf(";; res_nquery: retry without EDNS0\n"); goto again; } rcode = hp->rcode; /* record most recent error */ #ifdef DEBUG if (res->options & RES_DEBUG) printf(";; res_query: send error\n"); #endif continue; } if (n > anslen) hp->rcode = FORMERR; /* XXX not very informative */ if (hp->rcode != NOERROR || ntohs(hp->ancount) == 0) { rcode = hp->rcode; /* record most recent error */ #ifdef DEBUG if (res->options & RES_DEBUG) printf(";; rcode = %u, ancount=%u\n", hp->rcode, ntohs(hp->ancount)); #endif continue; } ancount += ntohs(hp->ancount); t->n = n; } free(buf); if (ancount == 0) { switch (rcode) { case NXDOMAIN: RES_SET_H_ERRNO(res, HOST_NOT_FOUND); break; case SERVFAIL: RES_SET_H_ERRNO(res, TRY_AGAIN); break; case NOERROR: RES_SET_H_ERRNO(res, NO_DATA); break; case FORMERR: case NOTIMP: case REFUSED: default: RES_SET_H_ERRNO(res, NO_RECOVERY); break; } return (-1); } return (ancount); } /* * Formulate a normal query, send, and retrieve answer in supplied buffer. * Return the size of the response on success, -1 on error. * If enabled, implement search rules until answer or unrecoverable failure * is detected. Error code, if any, is left in h_errno. */ static int res_searchN(const char *name, struct res_target *target, res_state res) { const char *cp, * const *domain; HEADER *hp = (HEADER *)(void *)target->answer; /*XXX*/ u_int dots; int trailing_dot, ret, saved_herrno; int got_nodata = 0, got_servfail = 0, root_on_list = 0; int tried_as_is = 0; int searched = 0; char abuf[MAXDNAME]; errno = 0; RES_SET_H_ERRNO(res, HOST_NOT_FOUND); /* default, if we never query */ dots = 0; for (cp = name; *cp; cp++) dots += (*cp == '.'); trailing_dot = 0; if (cp > name && *--cp == '.') trailing_dot++; /* * if there aren't any dots, it could be a user-level alias */ if (!dots && (cp = res_hostalias(res, name, abuf, sizeof(abuf))) != NULL) return (res_queryN(cp, target, res)); /* * If there are enough dots in the name, let's just give it a * try 'as is'. The threshold can be set with the "ndots" option. * Also, query 'as is', if there is a trailing dot in the name. */ saved_herrno = -1; if (dots >= res->ndots || trailing_dot) { ret = res_querydomainN(name, NULL, target, res); if (ret > 0 || trailing_dot) return (ret); if (errno == ECONNREFUSED) { RES_SET_H_ERRNO(res, TRY_AGAIN); return (-1); } switch (res->res_h_errno) { case NO_DATA: case HOST_NOT_FOUND: break; case TRY_AGAIN: if (hp->rcode == SERVFAIL) break; /* FALLTHROUGH */ default: return (-1); } saved_herrno = res->res_h_errno; tried_as_is++; } /* * We do at least one level of search if * - there is no dot and RES_DEFNAME is set, or * - there is at least one dot, there is no trailing dot, * and RES_DNSRCH is set. */ if ((!dots && (res->options & RES_DEFNAMES)) || (dots && !trailing_dot && (res->options & RES_DNSRCH))) { int done = 0; for (domain = (const char * const *)res->dnsrch; *domain && !done; domain++) { searched = 1; if (domain[0][0] == '\0' || (domain[0][0] == '.' && domain[0][1] == '\0')) root_on_list++; if (root_on_list && tried_as_is) continue; ret = res_querydomainN(name, *domain, target, res); if (ret > 0) return (ret); /* * If no server present, give up. * If name isn't found in this domain, * keep trying higher domains in the search list * (if that's enabled). * On a NO_DATA error, keep trying, otherwise * a wildcard entry of another type could keep us * from finding this entry higher in the domain. * If we get some other error (negative answer or * server failure), then stop searching up, * but try the input name below in case it's * fully-qualified. */ if (errno == ECONNREFUSED) { RES_SET_H_ERRNO(res, TRY_AGAIN); return (-1); } switch (res->res_h_errno) { case NO_DATA: got_nodata++; /* FALLTHROUGH */ case HOST_NOT_FOUND: /* keep trying */ break; case TRY_AGAIN: got_servfail++; if (hp->rcode == SERVFAIL) { /* try next search element, if any */ break; } /* FALLTHROUGH */ default: /* anything else implies that we're done */ done++; } /* * if we got here for some reason other than DNSRCH, * we only wanted one iteration of the loop, so stop. */ if (!(res->options & RES_DNSRCH)) done++; } } switch (res->res_h_errno) { case NO_DATA: case HOST_NOT_FOUND: break; case TRY_AGAIN: if (hp->rcode == SERVFAIL) break; /* FALLTHROUGH */ default: goto giveup; } /* * If the query has not already been tried as is then try it * unless RES_NOTLDQUERY is set and there were no dots. */ if ((dots || !searched || !(res->options & RES_NOTLDQUERY)) && !(tried_as_is || root_on_list)) { ret = res_querydomainN(name, NULL, target, res); if (ret > 0) return (ret); } /* * if we got here, we didn't satisfy the search. * if we did an initial full query, return that query's h_errno * (note that we wouldn't be here if that query had succeeded). * else if we ever got a nodata, send that back as the reason. * else send back meaningless h_errno, that being the one from * the last DNSRCH we did. */ giveup: if (saved_herrno != -1) RES_SET_H_ERRNO(res, saved_herrno); else if (got_nodata) RES_SET_H_ERRNO(res, NO_DATA); else if (got_servfail) RES_SET_H_ERRNO(res, TRY_AGAIN); return (-1); } /* * Perform a call on res_query on the concatenation of name and domain, * removing a trailing dot from name if domain is NULL. */ static int res_querydomainN(const char *name, const char *domain, struct res_target *target, res_state res) { char nbuf[MAXDNAME]; const char *longname = nbuf; size_t n, d; #ifdef DEBUG if (res->options & RES_DEBUG) printf(";; res_querydomain(%s, %s)\n", name, domain?domain:""); #endif if (domain == NULL) { /* * Check for trailing '.'; * copy without '.' if present. */ n = strlen(name); if (n >= MAXDNAME) { RES_SET_H_ERRNO(res, NO_RECOVERY); return (-1); } if (n > 0 && name[--n] == '.') { strncpy(nbuf, name, n); nbuf[n] = '\0'; } else longname = name; } else { n = strlen(name); d = strlen(domain); if (n + d + 1 >= MAXDNAME) { RES_SET_H_ERRNO(res, NO_RECOVERY); return (-1); } snprintf(nbuf, sizeof(nbuf), "%s.%s", name, domain); } return (res_queryN(longname, target, res)); } Index: projects/netbsd-tests-update-12/lib/libc/net/name6.c =================================================================== --- projects/netbsd-tests-update-12/lib/libc/net/name6.c (revision 305171) +++ projects/netbsd-tests-update-12/lib/libc/net/name6.c (revision 305172) @@ -1,1112 +1,1131 @@ /* $KAME: name6.c,v 1.25 2000/06/26 16:44:40 itojun Exp $ */ /* * Copyright (C) 1995, 1996, 1997, 1998, and 1999 WIDE Project. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the project nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * ++Copyright++ 1985, 1988, 1993 * - * Copyright (c) 1985, 1988, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * - * Portions Copyright (c) 1993 by Digital Equipment Corporation. * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies, and that * the name of Digital Equipment Corporation not be used in advertising or * publicity pertaining to distribution of the document or software without * specific, written prior permission. * * THE SOFTWARE IS PROVIDED "AS IS" AND DIGITAL EQUIPMENT CORP. DISCLAIMS ALL * WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL DIGITAL EQUIPMENT * CORPORATION BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL * DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR * PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS * ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS * SOFTWARE. * - * --Copyright-- */ /* * Atsushi Onoe */ #include __FBSDID("$FreeBSD$"); #include "namespace.h" #include #include #include #include #include #ifdef INET6 #include #include #include #include /* XXX */ #endif #include #include #include #include #include #include #include #include #include #include #include #include "un-namespace.h" #include "netdb_private.h" #include "res_private.h" #ifndef MAXALIASES #define MAXALIASES 10 #endif #ifndef MAXADDRS #define MAXADDRS 20 #endif #ifndef MAXDNAME #define MAXDNAME 1025 #endif #ifdef INET6 #define ADDRLEN(af) ((af) == AF_INET6 ? sizeof(struct in6_addr) : \ sizeof(struct in_addr)) #else #define ADDRLEN(af) sizeof(struct in_addr) #endif #define MAPADDR(ab, ina) \ do { \ memcpy(&(ab)->map_inaddr, ina, sizeof(struct in_addr)); \ memset((ab)->map_zero, 0, sizeof((ab)->map_zero)); \ memset((ab)->map_one, 0xff, sizeof((ab)->map_one)); \ } while (0) #define MAPADDRENABLED(flags) \ (((flags) & AI_V4MAPPED) || \ (((flags) & AI_V4MAPPED_CFG))) union inx_addr { struct in_addr in_addr; #ifdef INET6 struct in6_addr in6_addr; #endif struct { u_char mau_zero[10]; u_char mau_one[2]; struct in_addr mau_inaddr; } map_addr_un; #define map_zero map_addr_un.mau_zero #define map_one map_addr_un.mau_one #define map_inaddr map_addr_un.mau_inaddr }; struct policyqueue { TAILQ_ENTRY(policyqueue) pc_entry; #ifdef INET6 struct in6_addrpolicy pc_policy; #endif }; TAILQ_HEAD(policyhead, policyqueue); #define AIO_SRCFLAG_DEPRECATED 0x1 struct hp_order { union { struct sockaddr_storage aiou_ss; struct sockaddr aiou_sa; } aio_src_un; #define aio_srcsa aio_src_un.aiou_sa u_int32_t aio_srcflag; int aio_srcscope; int aio_dstscope; struct policyqueue *aio_srcpolicy; struct policyqueue *aio_dstpolicy; union { struct sockaddr_storage aiou_ss; struct sockaddr aiou_sa; } aio_un; #define aio_sa aio_un.aiou_sa int aio_matchlen; char *aio_h_addr; + int aio_initial_sequence; }; static struct hostent *_hpcopy(struct hostent *, int *); static struct hostent *_hpaddr(int, const char *, void *, int *); #ifdef INET6 static struct hostent *_hpmerge(struct hostent *, struct hostent *, int *); static struct hostent *_hpmapv6(struct hostent *, int *); #endif static struct hostent *_hpsort(struct hostent *, res_state); #ifdef INET6 static struct hostent *_hpreorder(struct hostent *); static int get_addrselectpolicy(struct policyhead *); static void free_addrselectpolicy(struct policyhead *); static struct policyqueue *match_addrselectpolicy(struct sockaddr *, struct policyhead *); static void set_source(struct hp_order *, struct policyhead *); static int matchlen(struct sockaddr *, struct sockaddr *); static int comp_dst(const void *, const void *); static int gai_addr2scopetype(struct sockaddr *); #endif /* * Functions defined in RFC2553 * getipnodebyname, getipnodebyaddr, freehostent */ struct hostent * getipnodebyname(const char *name, int af, int flags, int *errp) { struct hostent *hp; union inx_addr addrbuf; res_state statp; u_long options; switch (af) { case AF_INET: #ifdef INET6 case AF_INET6: #endif break; default: *errp = NO_RECOVERY; return NULL; } if (flags & AI_ADDRCONFIG) { int s; if ((s = _socket(af, SOCK_DGRAM | SOCK_CLOEXEC, 0)) < 0) return NULL; /* * TODO: * Note that implementation dependent test for address * configuration should be done every time called * (or appropriate interval), * because addresses will be dynamically assigned or deleted. */ _close(s); } #ifdef INET6 /* special case for literal address */ if (inet_pton(AF_INET6, name, &addrbuf) == 1) { if (af != AF_INET6) { *errp = HOST_NOT_FOUND; return NULL; } return _hpaddr(af, name, &addrbuf, errp); } #endif if (inet_aton(name, (struct in_addr *)&addrbuf) == 1) { if (af != AF_INET) { if (MAPADDRENABLED(flags)) { MAPADDR(&addrbuf, &addrbuf.in_addr); } else { *errp = HOST_NOT_FOUND; return NULL; } } return _hpaddr(af, name, &addrbuf, errp); } statp = __res_state(); if ((statp->options & RES_INIT) == 0) { if (res_ninit(statp) < 0) { *errp = NETDB_INTERNAL; return NULL; } } options = statp->options; statp->options &= ~RES_USE_INET6; hp = gethostbyname2(name, af); hp = _hpcopy(hp, errp); #ifdef INET6 if (af == AF_INET6) hp = _hpreorder(hp); if (af == AF_INET6 && ((flags & AI_ALL) || hp == NULL) && MAPADDRENABLED(flags)) { struct hostent *hp2 = gethostbyname2(name, AF_INET); if (hp == NULL) if (hp2 == NULL) *errp = statp->res_h_errno; else hp = _hpmapv6(hp2, errp); else { if (hp2 && strcmp(hp->h_name, hp2->h_name) == 0) { struct hostent *hpb = hp; hp = _hpmerge(hpb, hp2, errp); freehostent(hpb); } } } #endif if (hp == NULL) *errp = statp->res_h_errno; statp->options = options; return _hpsort(hp, statp); } struct hostent * getipnodebyaddr(const void *src, size_t len, int af, int *errp) { struct hostent *hp; res_state statp; u_long options; #ifdef INET6 struct in6_addr addrbuf; #else struct in_addr addrbuf; #endif switch (af) { case AF_INET: if (len != sizeof(struct in_addr)) { *errp = NO_RECOVERY; return NULL; } if (rounddown2((long)src, sizeof(struct in_addr))) { memcpy(&addrbuf, src, len); src = &addrbuf; } if (((struct in_addr *)src)->s_addr == 0) return NULL; break; #ifdef INET6 case AF_INET6: if (len != sizeof(struct in6_addr)) { *errp = NO_RECOVERY; return NULL; } if (rounddown2((long)src, sizeof(struct in6_addr) / 2)) { /* XXX */ memcpy(&addrbuf, src, len); src = &addrbuf; } if (IN6_IS_ADDR_UNSPECIFIED((struct in6_addr *)src)) return NULL; if (IN6_IS_ADDR_V4MAPPED((struct in6_addr *)src) || IN6_IS_ADDR_V4COMPAT((struct in6_addr *)src)) { src = (char *)src + (sizeof(struct in6_addr) - sizeof(struct in_addr)); af = AF_INET; len = sizeof(struct in_addr); } break; #endif default: *errp = NO_RECOVERY; return NULL; } statp = __res_state(); if ((statp->options & RES_INIT) == 0) { if (res_ninit(statp) < 0) { RES_SET_H_ERRNO(statp, NETDB_INTERNAL); return NULL; } } options = statp->options; statp->options &= ~RES_USE_INET6; hp = gethostbyaddr(src, len, af); if (hp == NULL) *errp = statp->res_h_errno; statp->options = options; return (_hpcopy(hp, errp)); } void freehostent(struct hostent *ptr) { free(ptr); } /* * Private utility functions */ /* * _hpcopy: allocate and copy hostent structure */ static struct hostent * _hpcopy(struct hostent *hp, int *errp) { struct hostent *nhp; char *cp, **pp; int size, addrsize; int nalias = 0, naddr = 0; int al_off; int i; if (hp == NULL) return hp; /* count size to be allocated */ size = sizeof(struct hostent); if (hp->h_name != NULL) size += strlen(hp->h_name) + 1; if ((pp = hp->h_aliases) != NULL) { for (i = 0; *pp != NULL; i++, pp++) { if (**pp != '\0') { size += strlen(*pp) + 1; nalias++; } } } /* adjust alignment */ size = ALIGN(size); al_off = size; size += sizeof(char *) * (nalias + 1); addrsize = ALIGN(hp->h_length); if ((pp = hp->h_addr_list) != NULL) { while (*pp++ != NULL) naddr++; } size += addrsize * naddr; size += sizeof(char *) * (naddr + 1); /* copy */ if ((nhp = (struct hostent *)malloc(size)) == NULL) { *errp = TRY_AGAIN; return NULL; } cp = (char *)&nhp[1]; if (hp->h_name != NULL) { nhp->h_name = cp; strcpy(cp, hp->h_name); cp += strlen(cp) + 1; } else nhp->h_name = NULL; nhp->h_aliases = (char **)((char *)nhp + al_off); if ((pp = hp->h_aliases) != NULL) { for (i = 0; *pp != NULL; pp++) { if (**pp != '\0') { nhp->h_aliases[i++] = cp; strcpy(cp, *pp); cp += strlen(cp) + 1; } } } nhp->h_aliases[nalias] = NULL; cp = (char *)&nhp->h_aliases[nalias + 1]; nhp->h_addrtype = hp->h_addrtype; nhp->h_length = hp->h_length; nhp->h_addr_list = (char **)cp; if ((pp = hp->h_addr_list) != NULL) { cp = (char *)&nhp->h_addr_list[naddr + 1]; for (i = 0; *pp != NULL; pp++) { nhp->h_addr_list[i++] = cp; memcpy(cp, *pp, hp->h_length); cp += addrsize; } } nhp->h_addr_list[naddr] = NULL; return nhp; } /* * _hpaddr: construct hostent structure with one address */ static struct hostent * _hpaddr(int af, const char *name, void *addr, int *errp) { struct hostent *hp, hpbuf; char *addrs[2]; hp = &hpbuf; hp->h_name = (char *)name; hp->h_aliases = NULL; hp->h_addrtype = af; hp->h_length = ADDRLEN(af); hp->h_addr_list = addrs; addrs[0] = (char *)addr; addrs[1] = NULL; return (_hpcopy(hp, errp)); } #ifdef INET6 /* * _hpmerge: merge 2 hostent structure, arguments will be freed */ static struct hostent * _hpmerge(struct hostent *hp1, struct hostent *hp2, int *errp) { int i, j; int naddr, nalias; char **pp; struct hostent *hp, hpbuf; char *aliases[MAXALIASES + 1], *addrs[MAXADDRS + 1]; union inx_addr addrbuf[MAXADDRS]; if (hp1 == NULL) return _hpcopy(hp2, errp); if (hp2 == NULL) return _hpcopy(hp1, errp); #define HP(i) (i == 1 ? hp1 : hp2) hp = &hpbuf; hp->h_name = (hp1->h_name != NULL ? hp1->h_name : hp2->h_name); hp->h_aliases = aliases; nalias = 0; for (i = 1; i <= 2; i++) { if ((pp = HP(i)->h_aliases) == NULL) continue; for (; nalias < MAXALIASES && *pp != NULL; pp++) { /* check duplicates */ for (j = 0; j < nalias; j++) if (strcasecmp(*pp, aliases[j]) == 0) break; if (j == nalias) aliases[nalias++] = *pp; } } aliases[nalias] = NULL; if (hp1->h_length != hp2->h_length) { hp->h_addrtype = AF_INET6; hp->h_length = sizeof(struct in6_addr); } else { hp->h_addrtype = hp1->h_addrtype; hp->h_length = hp1->h_length; } hp->h_addr_list = addrs; naddr = 0; for (i = 1; i <= 2; i++) { if ((pp = HP(i)->h_addr_list) == NULL) continue; if (HP(i)->h_length == hp->h_length) { while (naddr < MAXADDRS && *pp != NULL) addrs[naddr++] = *pp++; } else { /* copy IPv4 addr as mapped IPv6 addr */ while (naddr < MAXADDRS && *pp != NULL) { MAPADDR(&addrbuf[naddr], *pp++); addrs[naddr] = (char *)&addrbuf[naddr]; naddr++; } } } addrs[naddr] = NULL; return (_hpcopy(hp, errp)); } #endif /* * _hpmapv6: convert IPv4 hostent into IPv4-mapped IPv6 addresses */ #ifdef INET6 static struct hostent * _hpmapv6(struct hostent *hp, int *errp) { struct hostent hp6; if (hp == NULL) return NULL; if (hp->h_addrtype == AF_INET6) return _hpcopy(hp, errp); memset(&hp6, 0, sizeof(struct hostent)); hp6.h_addrtype = AF_INET6; hp6.h_length = sizeof(struct in6_addr); return _hpmerge(&hp6, hp, errp); } #endif /* * _hpsort: sort address by sortlist */ static struct hostent * _hpsort(struct hostent *hp, res_state statp) { int i, j, n; u_char *ap, *sp, *mp, **pp; char t; char order[MAXADDRS]; int nsort = statp->nsort; if (hp == NULL || hp->h_addr_list[1] == NULL || nsort == 0) return hp; for (i = 0; (ap = (u_char *)hp->h_addr_list[i]); i++) { for (j = 0; j < nsort; j++) { #ifdef INET6 if (statp->_u._ext.ext->sort_list[j].af != hp->h_addrtype) continue; sp = (u_char *)&statp->_u._ext.ext->sort_list[j].addr; mp = (u_char *)&statp->_u._ext.ext->sort_list[j].mask; #else sp = (u_char *)&statp->sort_list[j].addr; mp = (u_char *)&statp->sort_list[j].mask; #endif for (n = 0; n < hp->h_length; n++) { if ((ap[n] & mp[n]) != sp[n]) break; } if (n == hp->h_length) break; } order[i] = j; } n = i; pp = (u_char **)hp->h_addr_list; for (i = 0; i < n - 1; i++) { for (j = i + 1; j < n; j++) { if (order[i] > order[j]) { ap = pp[i]; pp[i] = pp[j]; pp[j] = ap; t = order[i]; order[i] = order[j]; order[j] = t; } } } return hp; } #ifdef INET6 /* * _hpreorder: sort address by default address selection */ static struct hostent * _hpreorder(struct hostent *hp) { struct hp_order *aio; int i, n; char *ap; struct sockaddr *sa; struct policyhead policyhead; if (hp == NULL) return hp; switch (hp->h_addrtype) { case AF_INET: #ifdef INET6 case AF_INET6: #endif break; default: return hp; } /* count the number of addrinfo elements for sorting. */ for (n = 0; hp->h_addr_list[n] != NULL; n++) ; /* * If the number is small enough, we can skip the reordering process. */ if (n <= 1) return hp; /* allocate a temporary array for sort and initialization of it. */ if ((aio = malloc(sizeof(*aio) * n)) == NULL) return hp; /* give up reordering */ memset(aio, 0, sizeof(*aio) * n); /* retrieve address selection policy from the kernel */ TAILQ_INIT(&policyhead); if (!get_addrselectpolicy(&policyhead)) { /* no policy is installed into kernel, we don't sort. */ free(aio); return hp; } for (i = 0; i < n; i++) { ap = hp->h_addr_list[i]; aio[i].aio_h_addr = ap; sa = &aio[i].aio_sa; switch (hp->h_addrtype) { case AF_INET: sa->sa_family = AF_INET; sa->sa_len = sizeof(struct sockaddr_in); memcpy(&((struct sockaddr_in *)sa)->sin_addr, ap, sizeof(struct in_addr)); break; #ifdef INET6 case AF_INET6: if (IN6_IS_ADDR_V4MAPPED((struct in6_addr *)ap)) { sa->sa_family = AF_INET; sa->sa_len = sizeof(struct sockaddr_in); memcpy(&((struct sockaddr_in *)sa)->sin_addr, &ap[12], sizeof(struct in_addr)); } else { sa->sa_family = AF_INET6; sa->sa_len = sizeof(struct sockaddr_in6); memcpy(&((struct sockaddr_in6 *)sa)->sin6_addr, ap, sizeof(struct in6_addr)); } break; #endif } aio[i].aio_dstscope = gai_addr2scopetype(sa); aio[i].aio_dstpolicy = match_addrselectpolicy(sa, &policyhead); set_source(&aio[i], &policyhead); + aio[i].aio_initial_sequence = i; } /* perform sorting. */ qsort(aio, n, sizeof(*aio), comp_dst); /* reorder the h_addr_list. */ for (i = 0; i < n; i++) hp->h_addr_list[i] = aio[i].aio_h_addr; /* cleanup and return */ free(aio); free_addrselectpolicy(&policyhead); return hp; } static int get_addrselectpolicy(struct policyhead *head) { #ifdef INET6 int mib[] = { CTL_NET, PF_INET6, IPPROTO_IPV6, IPV6CTL_ADDRCTLPOLICY }; size_t l; char *buf; struct in6_addrpolicy *pol, *ep; if (sysctl(mib, nitems(mib), NULL, &l, NULL, 0) < 0) return (0); if ((buf = malloc(l)) == NULL) return (0); if (sysctl(mib, nitems(mib), buf, &l, NULL, 0) < 0) { free(buf); return (0); } ep = (struct in6_addrpolicy *)(buf + l); for (pol = (struct in6_addrpolicy *)buf; pol + 1 <= ep; pol++) { struct policyqueue *new; if ((new = malloc(sizeof(*new))) == NULL) { free_addrselectpolicy(head); /* make the list empty */ break; } new->pc_policy = *pol; TAILQ_INSERT_TAIL(head, new, pc_entry); } free(buf); return (1); #else return (0); #endif } static void free_addrselectpolicy(struct policyhead *head) { struct policyqueue *ent, *nent; for (ent = TAILQ_FIRST(head); ent; ent = nent) { nent = TAILQ_NEXT(ent, pc_entry); TAILQ_REMOVE(head, ent, pc_entry); free(ent); } } static struct policyqueue * match_addrselectpolicy(struct sockaddr *addr, struct policyhead *head) { #ifdef INET6 struct policyqueue *ent, *bestent = NULL; struct in6_addrpolicy *pol; int matchlen, bestmatchlen = -1; u_char *mp, *ep, *k, *p, m; struct sockaddr_in6 key; switch(addr->sa_family) { case AF_INET6: key = *(struct sockaddr_in6 *)addr; break; case AF_INET: /* convert the address into IPv4-mapped IPv6 address. */ memset(&key, 0, sizeof(key)); key.sin6_family = AF_INET6; key.sin6_len = sizeof(key); _map_v4v6_address( (char *)&((struct sockaddr_in *)addr)->sin_addr, (char *)&key.sin6_addr); break; default: return(NULL); } for (ent = TAILQ_FIRST(head); ent; ent = TAILQ_NEXT(ent, pc_entry)) { pol = &ent->pc_policy; matchlen = 0; mp = (u_char *)&pol->addrmask.sin6_addr; ep = mp + 16; /* XXX: scope field? */ k = (u_char *)&key.sin6_addr; p = (u_char *)&pol->addr.sin6_addr; for (; mp < ep && *mp; mp++, k++, p++) { m = *mp; if ((*k & m) != *p) goto next; /* not match */ if (m == 0xff) /* short cut for a typical case */ matchlen += 8; else { while (m >= 0x80) { matchlen++; m <<= 1; } } } /* matched. check if this is better than the current best. */ if (matchlen > bestmatchlen) { bestent = ent; bestmatchlen = matchlen; } next: continue; } return(bestent); #else return(NULL); #endif } static void set_source(struct hp_order *aio, struct policyhead *ph) { struct sockaddr_storage ss = aio->aio_un.aiou_ss; socklen_t srclen; int s; /* set unspec ("no source is available"), just in case */ aio->aio_srcsa.sa_family = AF_UNSPEC; aio->aio_srcscope = -1; switch(ss.ss_family) { case AF_INET: ((struct sockaddr_in *)&ss)->sin_port = htons(1); break; #ifdef INET6 case AF_INET6: ((struct sockaddr_in6 *)&ss)->sin6_port = htons(1); break; #endif default: /* ignore unsupported AFs explicitly */ return; } /* open a socket to get the source address for the given dst */ if ((s = _socket(ss.ss_family, SOCK_DGRAM | SOCK_CLOEXEC, IPPROTO_UDP)) < 0) return; /* give up */ if (_connect(s, (struct sockaddr *)&ss, ss.ss_len) < 0) goto cleanup; srclen = ss.ss_len; if (_getsockname(s, &aio->aio_srcsa, &srclen) < 0) { aio->aio_srcsa.sa_family = AF_UNSPEC; goto cleanup; } aio->aio_srcscope = gai_addr2scopetype(&aio->aio_srcsa); aio->aio_srcpolicy = match_addrselectpolicy(&aio->aio_srcsa, ph); aio->aio_matchlen = matchlen(&aio->aio_srcsa, (struct sockaddr *)&ss); #ifdef INET6 if (ss.ss_family == AF_INET6) { struct in6_ifreq ifr6; u_int32_t flags6; memset(&ifr6, 0, sizeof(ifr6)); memcpy(&ifr6.ifr_addr, &ss, ss.ss_len); if (_ioctl(s, SIOCGIFAFLAG_IN6, &ifr6) == 0) { flags6 = ifr6.ifr_ifru.ifru_flags6; if ((flags6 & IN6_IFF_DEPRECATED)) aio->aio_srcflag |= AIO_SRCFLAG_DEPRECATED; } } #endif cleanup: _close(s); return; } static int matchlen(struct sockaddr *src, struct sockaddr *dst) { int match = 0; u_char *s, *d; u_char *lim, r; int addrlen; switch (src->sa_family) { #ifdef INET6 case AF_INET6: s = (u_char *)&((struct sockaddr_in6 *)src)->sin6_addr; d = (u_char *)&((struct sockaddr_in6 *)dst)->sin6_addr; addrlen = sizeof(struct in6_addr); lim = s + addrlen; break; #endif case AF_INET: s = (u_char *)&((struct sockaddr_in *)src)->sin_addr; d = (u_char *)&((struct sockaddr_in *)dst)->sin_addr; addrlen = sizeof(struct in_addr); lim = s + addrlen; break; default: return(0); } while (s < lim) if ((r = (*d++ ^ *s++)) != 0) { - while (r < addrlen * 8) { + while ((r & 0x80) == 0) { match++; r <<= 1; } break; } else match += 8; return(match); } static int comp_dst(const void *arg1, const void *arg2) { const struct hp_order *dst1 = arg1, *dst2 = arg2; /* * Rule 1: Avoid unusable destinations. * XXX: we currently do not consider if an appropriate route exists. */ if (dst1->aio_srcsa.sa_family != AF_UNSPEC && dst2->aio_srcsa.sa_family == AF_UNSPEC) { return(-1); } if (dst1->aio_srcsa.sa_family == AF_UNSPEC && dst2->aio_srcsa.sa_family != AF_UNSPEC) { return(1); } /* Rule 2: Prefer matching scope. */ if (dst1->aio_dstscope == dst1->aio_srcscope && dst2->aio_dstscope != dst2->aio_srcscope) { return(-1); } if (dst1->aio_dstscope != dst1->aio_srcscope && dst2->aio_dstscope == dst2->aio_srcscope) { return(1); } /* Rule 3: Avoid deprecated addresses. */ if (dst1->aio_srcsa.sa_family != AF_UNSPEC && dst2->aio_srcsa.sa_family != AF_UNSPEC) { if (!(dst1->aio_srcflag & AIO_SRCFLAG_DEPRECATED) && (dst2->aio_srcflag & AIO_SRCFLAG_DEPRECATED)) { return(-1); } if ((dst1->aio_srcflag & AIO_SRCFLAG_DEPRECATED) && !(dst2->aio_srcflag & AIO_SRCFLAG_DEPRECATED)) { return(1); } } /* Rule 4: Prefer home addresses. */ /* XXX: not implemented yet */ /* Rule 5: Prefer matching label. */ #ifdef INET6 if (dst1->aio_srcpolicy && dst1->aio_dstpolicy && dst1->aio_srcpolicy->pc_policy.label == dst1->aio_dstpolicy->pc_policy.label && (dst2->aio_srcpolicy == NULL || dst2->aio_dstpolicy == NULL || dst2->aio_srcpolicy->pc_policy.label != dst2->aio_dstpolicy->pc_policy.label)) { return(-1); } if (dst2->aio_srcpolicy && dst2->aio_dstpolicy && dst2->aio_srcpolicy->pc_policy.label == dst2->aio_dstpolicy->pc_policy.label && (dst1->aio_srcpolicy == NULL || dst1->aio_dstpolicy == NULL || dst1->aio_srcpolicy->pc_policy.label != dst1->aio_dstpolicy->pc_policy.label)) { return(1); } #endif /* Rule 6: Prefer higher precedence. */ #ifdef INET6 if (dst1->aio_dstpolicy && (dst2->aio_dstpolicy == NULL || dst1->aio_dstpolicy->pc_policy.preced > dst2->aio_dstpolicy->pc_policy.preced)) { return(-1); } if (dst2->aio_dstpolicy && (dst1->aio_dstpolicy == NULL || dst2->aio_dstpolicy->pc_policy.preced > dst1->aio_dstpolicy->pc_policy.preced)) { return(1); } #endif /* Rule 7: Prefer native transport. */ /* XXX: not implemented yet */ /* Rule 8: Prefer smaller scope. */ if (dst1->aio_dstscope >= 0 && dst1->aio_dstscope < dst2->aio_dstscope) { return(-1); } if (dst2->aio_dstscope >= 0 && dst2->aio_dstscope < dst1->aio_dstscope) { return(1); } /* * Rule 9: Use longest matching prefix. * We compare the match length in a same AF only. */ if (dst1->aio_sa.sa_family == dst2->aio_sa.sa_family) { if (dst1->aio_matchlen > dst2->aio_matchlen) { return(-1); } if (dst1->aio_matchlen < dst2->aio_matchlen) { return(1); } } /* Rule 10: Otherwise, leave the order unchanged. */ + + /* + * Note that qsort is unstable; so, we can't return zero and + * expect the order to be unchanged. + * That also means we can't depend on the current position of + * dst2 being after dst1. We must enforce the initial order + * with an explicit compare on the original position. + * The qsort specification requires that "When the same objects + * (consisting of width bytes, irrespective of their current + * positions in the array) are passed more than once to the + * comparison function, the results shall be consistent with one + * another." + * In other words, If A < B, then we must also return B > A. + */ + if (dst2->aio_initial_sequence < dst1->aio_initial_sequence) + return(1); + return(-1); } /* * Copy from scope.c. * XXX: we should standardize the functions and link them as standard * library. */ static int gai_addr2scopetype(struct sockaddr *sa) { #ifdef INET6 struct sockaddr_in6 *sa6; #endif struct sockaddr_in *sa4; switch(sa->sa_family) { #ifdef INET6 case AF_INET6: sa6 = (struct sockaddr_in6 *)sa; if (IN6_IS_ADDR_MULTICAST(&sa6->sin6_addr)) { /* just use the scope field of the multicast address */ return(sa6->sin6_addr.s6_addr[2] & 0x0f); } /* * Unicast addresses: map scope type to corresponding scope * value defined for multcast addresses. * XXX: hardcoded scope type values are bad... */ if (IN6_IS_ADDR_LOOPBACK(&sa6->sin6_addr)) return(1); /* node local scope */ if (IN6_IS_ADDR_LINKLOCAL(&sa6->sin6_addr)) return(2); /* link-local scope */ if (IN6_IS_ADDR_SITELOCAL(&sa6->sin6_addr)) return(5); /* site-local scope */ return(14); /* global scope */ break; #endif case AF_INET: /* * IPv4 pseudo scoping according to RFC 3484. */ sa4 = (struct sockaddr_in *)sa; /* IPv4 autoconfiguration addresses have link-local scope. */ if (((u_char *)&sa4->sin_addr)[0] == 169 && ((u_char *)&sa4->sin_addr)[1] == 254) return(2); /* Private addresses have site-local scope. */ if (((u_char *)&sa4->sin_addr)[0] == 10 || (((u_char *)&sa4->sin_addr)[0] == 172 && (((u_char *)&sa4->sin_addr)[1] & 0xf0) == 16) || (((u_char *)&sa4->sin_addr)[0] == 192 && ((u_char *)&sa4->sin_addr)[1] == 168)) return(14); /* XXX: It should be 5 unless NAT */ /* Loopback addresses have link-local scope. */ if (((u_char *)&sa4->sin_addr)[0] == 127) return(2); return(14); break; default: errno = EAFNOSUPPORT; /* is this a good error? */ return(-1); } } #endif Index: projects/netbsd-tests-update-12/lib/libc/tests/stdlib/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/lib/libc/tests/stdlib/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/lib/libc/tests/stdlib/Makefile.depend (revision 305172) @@ -1,25 +1,26 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/atf/libatf-c \ lib/atf/libatf-c++ \ lib/libc \ lib/libc++ \ - lib/libthr \ lib/libcompiler_rt \ + lib/libcxxrt \ lib/libnetbsd \ + lib/libthr \ lib/libutil \ lib/msun \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/lib/libelftc/Makefile =================================================================== --- projects/netbsd-tests-update-12/lib/libelftc/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/lib/libelftc/Makefile (revision 305172) @@ -1,45 +1,46 @@ # $FreeBSD$ .include PACKAGE=lib${LIB} INTERNALLIB= ELFTCDIR= ${SRCTOP}/contrib/elftoolchain .PATH: ${ELFTCDIR}/libelftc LIB= elftc SRCS= elftc_bfdtarget.c \ elftc_copyfile.c \ elftc_demangle.c \ elftc_reloc_type_str.c \ elftc_set_timestamps.c \ elftc_string_table.c \ + elftc_timestamp.c \ elftc_version.c \ libelftc_bfdtarget.c \ libelftc_dem_arm.c \ libelftc_dem_gnu2.c \ libelftc_dem_gnu3.c \ libelftc_hash.c \ libelftc_vstr.c INCS= libelftc.h CFLAGS+=-I${ELFTCDIR}/libelftc -I${ELFTCDIR}/common MAN= # This same hack is in lib/libelf/Makefile and usr.bin/readelf/Makefile # We need to link against the correct version of these files. One # solution is to include ../../sys in the include path. This causes # problems when a header file in sys depends on a file in another # part of the tree, e.g. a machine dependent header. # SRCS+= sys/elf_common.h CLEANDIRS= sys CFLAGS+= -I. sys/elf_common.h: ${SRCTOP}/sys/${.TARGET} .NOMETA mkdir -p ${.OBJDIR}/sys ln -sf ${.ALLSRC} ${.TARGET} .include Index: projects/netbsd-tests-update-12/lib/libelftc/elftc_version.c =================================================================== --- projects/netbsd-tests-update-12/lib/libelftc/elftc_version.c (revision 305171) +++ projects/netbsd-tests-update-12/lib/libelftc/elftc_version.c (revision 305172) @@ -1,10 +1,10 @@ /* $FreeBSD$ */ #include #include const char * elftc_version(void) { - return "elftoolchain r3477M"; + return "elftoolchain r3490M"; } Index: projects/netbsd-tests-update-12/lib/libproc/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/lib/libproc/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/lib/libproc/Makefile.depend (revision 305172) @@ -1,23 +1,24 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ cddl/lib/libctf \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/libc \ lib/libcompiler_rt \ lib/libcxxrt \ lib/libelf \ + lib/libprocstat \ lib/librtld_db \ lib/libutil \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/lib/libproc/tests/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/lib/libproc/tests/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/lib/libproc/tests/Makefile.depend (revision 305172) @@ -1,26 +1,28 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ cddl/lib/libctf \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/atf/libatf-c \ lib/libc \ lib/libcompiler_rt \ lib/libcxxrt \ lib/libelf \ + lib/libkvm \ lib/libproc \ + lib/libprocstat \ lib/librtld_db \ lib/libutil \ lib/libz \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/lib/librtld_db/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/lib/librtld_db/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/lib/librtld_db/Makefile.depend (revision 305172) @@ -1,20 +1,21 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/libc \ lib/libcompiler_rt \ lib/libelf \ + lib/libprocstat \ lib/libutil \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/lib/libstand/bootp.c =================================================================== --- projects/netbsd-tests-update-12/lib/libstand/bootp.c (revision 305171) +++ projects/netbsd-tests-update-12/lib/libstand/bootp.c (revision 305172) @@ -1,757 +1,761 @@ /* $NetBSD: bootp.c,v 1.14 1998/02/16 11:10:54 drochner Exp $ */ /* * Copyright (c) 1992 Regents of the University of California. * All rights reserved. * * This software was developed by the Computer Systems Engineering group * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and * contributed to Berkeley. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#) Header: bootp.c,v 1.4 93/09/11 03:13:51 leres Exp (LBL) */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #define BOOTP_DEBUGxx #define SUPPORT_DHCP #define DHCP_ENV_NOVENDOR 1 /* do not parse vendor options */ #define DHCP_ENV_PXE 10 /* assume pxe vendor options */ #define DHCP_ENV_FREEBSD 11 /* assume freebsd vendor options */ /* set DHCP_ENV to one of the values above to export dhcp options to kenv */ #define DHCP_ENV DHCP_ENV_NO_VENDOR #include "stand.h" #include "net.h" #include "netif.h" #include "bootp.h" struct in_addr servip; static n_long nmask, smask; static time_t bot; static char vm_rfc1048[4] = VM_RFC1048; #ifdef BOOTP_VEND_CMU static char vm_cmu[4] = VM_CMU; #endif /* Local forwards */ static ssize_t bootpsend(struct iodesc *, void *, size_t); static ssize_t bootprecv(struct iodesc *, void *, size_t, time_t); static int vend_rfc1048(u_char *, u_int); #ifdef BOOTP_VEND_CMU static void vend_cmu(u_char *); #endif #ifdef DHCP_ENV /* export the dhcp response to kenv */ struct dhcp_opt; static void setenv_(u_char *cp, u_char *ep, struct dhcp_opt *opts); #else #define setenv_(a, b, c) #endif #ifdef SUPPORT_DHCP static char expected_dhcpmsgtype = -1, dhcp_ok; struct in_addr dhcp_serverip; #endif /* Fetch required bootp infomation */ void bootp(sock, flag) int sock; int flag; { struct iodesc *d; struct bootp *bp; struct { u_char header[HEADER_SIZE]; struct bootp wbootp; } wbuf; struct { u_char header[HEADER_SIZE]; struct bootp rbootp; } rbuf; #ifdef BOOTP_DEBUG if (debug) printf("bootp: socket=%d\n", sock); #endif if (!bot) bot = getsecs(); if (!(d = socktodesc(sock))) { printf("bootp: bad socket. %d\n", sock); return; } #ifdef BOOTP_DEBUG if (debug) printf("bootp: d=%lx\n", (long)d); #endif bp = &wbuf.wbootp; bzero(bp, sizeof(*bp)); bp->bp_op = BOOTREQUEST; bp->bp_htype = 1; /* 10Mb Ethernet (48 bits) */ bp->bp_hlen = 6; bp->bp_xid = htonl(d->xid); MACPY(d->myea, bp->bp_chaddr); strncpy(bp->bp_file, bootfile, sizeof(bp->bp_file)); bcopy(vm_rfc1048, bp->bp_vend, sizeof(vm_rfc1048)); #ifdef SUPPORT_DHCP bp->bp_vend[4] = TAG_DHCP_MSGTYPE; bp->bp_vend[5] = 1; bp->bp_vend[6] = DHCPDISCOVER; /* * If we are booting from PXE, we want to send the string * 'PXEClient' to the DHCP server so you have the option of * only responding to PXE aware dhcp requests. */ if (flag & BOOTP_PXE) { bp->bp_vend[7] = TAG_CLASSID; bp->bp_vend[8] = 9; bcopy("PXEClient", &bp->bp_vend[9], 9); bp->bp_vend[18] = TAG_END; } else bp->bp_vend[7] = TAG_END; #else bp->bp_vend[4] = TAG_END; #endif d->myip.s_addr = INADDR_ANY; d->myport = htons(IPPORT_BOOTPC); d->destip.s_addr = INADDR_BROADCAST; d->destport = htons(IPPORT_BOOTPS); #ifdef SUPPORT_DHCP expected_dhcpmsgtype = DHCPOFFER; dhcp_ok = 0; #endif if(sendrecv(d, bootpsend, bp, sizeof(*bp), bootprecv, &rbuf.rbootp, sizeof(rbuf.rbootp)) == -1) { printf("bootp: no reply\n"); return; } #ifdef SUPPORT_DHCP if(dhcp_ok) { u_int32_t leasetime; bp->bp_vend[6] = DHCPREQUEST; bp->bp_vend[7] = TAG_REQ_ADDR; bp->bp_vend[8] = 4; bcopy(&rbuf.rbootp.bp_yiaddr, &bp->bp_vend[9], 4); bp->bp_vend[13] = TAG_SERVERID; bp->bp_vend[14] = 4; bcopy(&dhcp_serverip.s_addr, &bp->bp_vend[15], 4); bp->bp_vend[19] = TAG_LEASETIME; bp->bp_vend[20] = 4; leasetime = htonl(300); bcopy(&leasetime, &bp->bp_vend[21], 4); if (flag & BOOTP_PXE) { bp->bp_vend[25] = TAG_CLASSID; bp->bp_vend[26] = 9; bcopy("PXEClient", &bp->bp_vend[27], 9); bp->bp_vend[36] = TAG_END; } else bp->bp_vend[25] = TAG_END; expected_dhcpmsgtype = DHCPACK; if(sendrecv(d, bootpsend, bp, sizeof(*bp), bootprecv, &rbuf.rbootp, sizeof(rbuf.rbootp)) == -1) { printf("DHCPREQUEST failed\n"); return; } } #endif myip = d->myip = rbuf.rbootp.bp_yiaddr; servip = rbuf.rbootp.bp_siaddr; if(rootip.s_addr == INADDR_ANY) rootip = servip; bcopy(rbuf.rbootp.bp_file, bootfile, sizeof(bootfile)); bootfile[sizeof(bootfile) - 1] = '\0'; if (IN_CLASSA(ntohl(myip.s_addr))) nmask = htonl(IN_CLASSA_NET); else if (IN_CLASSB(ntohl(myip.s_addr))) nmask = htonl(IN_CLASSB_NET); else nmask = htonl(IN_CLASSC_NET); #ifdef BOOTP_DEBUG if (debug) printf("'native netmask' is %s\n", intoa(nmask)); #endif /* Check subnet mask against net mask; toss if bogus */ if ((nmask & smask) != nmask) { #ifdef BOOTP_DEBUG if (debug) printf("subnet mask (%s) bad\n", intoa(smask)); #endif smask = 0; } /* Get subnet (or natural net) mask */ netmask = nmask; if (smask) netmask = smask; #ifdef BOOTP_DEBUG if (debug) printf("mask: %s\n", intoa(netmask)); #endif /* We need a gateway if root is on a different net */ if (!SAMENET(myip, rootip, netmask)) { #ifdef BOOTP_DEBUG if (debug) printf("need gateway for root ip\n"); #endif } /* Toss gateway if on a different net */ if (!SAMENET(myip, gateip, netmask)) { #ifdef BOOTP_DEBUG if (debug) printf("gateway ip (%s) bad\n", inet_ntoa(gateip)); #endif gateip.s_addr = 0; } /* Bump xid so next request will be unique. */ ++d->xid; } /* Transmit a bootp request */ static ssize_t bootpsend(d, pkt, len) struct iodesc *d; void *pkt; size_t len; { struct bootp *bp; #ifdef BOOTP_DEBUG if (debug) printf("bootpsend: d=%lx called.\n", (long)d); #endif bp = pkt; bp->bp_secs = htons((u_short)(getsecs() - bot)); #ifdef BOOTP_DEBUG if (debug) printf("bootpsend: calling sendudp\n"); #endif return (sendudp(d, pkt, len)); } static ssize_t bootprecv(d, pkt, len, tleft) struct iodesc *d; void *pkt; size_t len; time_t tleft; { ssize_t n; struct bootp *bp; #ifdef BOOTP_DEBUGx if (debug) printf("bootp_recvoffer: called\n"); #endif n = readudp(d, pkt, len, tleft); if (n == -1 || n < sizeof(struct bootp) - BOOTP_VENDSIZE) goto bad; bp = (struct bootp *)pkt; #ifdef BOOTP_DEBUG if (debug) printf("bootprecv: checked. bp = 0x%lx, n = %d\n", (long)bp, (int)n); #endif if (bp->bp_xid != htonl(d->xid)) { #ifdef BOOTP_DEBUG if (debug) { printf("bootprecv: expected xid 0x%lx, got 0x%x\n", d->xid, ntohl(bp->bp_xid)); } #endif goto bad; } #ifdef BOOTP_DEBUG if (debug) printf("bootprecv: got one!\n"); #endif /* Suck out vendor info */ if (bcmp(vm_rfc1048, bp->bp_vend, sizeof(vm_rfc1048)) == 0) { if(vend_rfc1048(bp->bp_vend, sizeof(bp->bp_vend)) != 0) goto bad; } #ifdef BOOTP_VEND_CMU else if (bcmp(vm_cmu, bp->bp_vend, sizeof(vm_cmu)) == 0) vend_cmu(bp->bp_vend); #endif else printf("bootprecv: unknown vendor 0x%lx\n", (long)bp->bp_vend); return(n); bad: errno = 0; return (-1); } static int vend_rfc1048(cp, len) u_char *cp; u_int len; { u_char *ep; int size; u_char tag; const char *val; #ifdef BOOTP_DEBUG if (debug) printf("vend_rfc1048 bootp info. len=%d\n", len); #endif ep = cp + len; /* Step over magic cookie */ cp += sizeof(int); setenv_(cp, ep, NULL); while (cp < ep) { tag = *cp++; size = *cp++; if (tag == TAG_END) break; if (tag == TAG_SUBNET_MASK) { bcopy(cp, &smask, sizeof(smask)); } if (tag == TAG_GATEWAY) { bcopy(cp, &gateip.s_addr, sizeof(gateip.s_addr)); } if (tag == TAG_SWAPSERVER) { /* let it override bp_siaddr */ bcopy(cp, &rootip.s_addr, sizeof(rootip.s_addr)); } if (tag == TAG_ROOTPATH) { if ((val = getenv("dhcp.root-path")) == NULL) val = (const char *)cp; strlcpy(rootpath, val, sizeof(rootpath)); } if (tag == TAG_HOSTNAME) { if ((val = getenv("dhcp.host-name")) == NULL) val = (const char *)cp; strlcpy(hostname, val, sizeof(hostname)); } if (tag == TAG_INTF_MTU) { if ((val = getenv("dhcp.interface-mtu")) != NULL) { intf_mtu = (u_int)strtoul(val, NULL, 0); } else { intf_mtu = be16dec(cp); } } #ifdef SUPPORT_DHCP if (tag == TAG_DHCP_MSGTYPE) { if(*cp != expected_dhcpmsgtype) return(-1); dhcp_ok = 1; } if (tag == TAG_SERVERID) { bcopy(cp, &dhcp_serverip.s_addr, sizeof(dhcp_serverip.s_addr)); } + if (tag == TAG_TFTP_SERVER) { + bcopy(cp, &tftpip.s_addr, + sizeof(tftpip.s_addr)); + } #endif cp += size; } return(0); } #ifdef BOOTP_VEND_CMU static void vend_cmu(cp) u_char *cp; { struct cmu_vend *vp; #ifdef BOOTP_DEBUG if (debug) printf("vend_cmu bootp info.\n"); #endif vp = (struct cmu_vend *)cp; if (vp->v_smask.s_addr != 0) { smask = vp->v_smask.s_addr; } if (vp->v_dgate.s_addr != 0) { gateip = vp->v_dgate; } } #endif #ifdef DHCP_ENV /* * Parse DHCP options and store them into kenv variables. * Original code from Danny Braniss, modifications by Luigi Rizzo. * * The parser is driven by tables which specify the type and name of * each dhcp option and how it appears in kenv. * The first entry in the list contains the prefix used to set the kenv * name (including the . if needed), the last entry must have a 0 tag. * Entries do not need to be sorted though it helps for readability. * * Certain vendor-specific tables can be enabled according to DHCP_ENV. * Set it to 0 if you don't want any. */ enum opt_fmt { __NONE = 0, __8 = 1, __16 = 2, __32 = 4, /* Unsigned fields, value=size */ __IP, /* IPv4 address */ __TXT, /* C string */ __BYTES, /* byte sequence, printed %02x */ __INDIR, /* name=value */ __ILIST, /* name=value;name=value ... */ __VE, /* vendor specific, recurse */ }; struct dhcp_opt { uint8_t tag; uint8_t fmt; const char *desc; }; static struct dhcp_opt vndr_opt[] = { /* Vendor Specific Options */ #if DHCP_ENV == DHCP_ENV_FREEBSD /* FreeBSD table in the original code */ {0, 0, "FreeBSD"}, /* prefix */ {1, __TXT, "kernel"}, {2, __TXT, "kernelname"}, {3, __TXT, "kernel_options"}, {4, __IP, "usr-ip"}, {5, __TXT, "conf-path"}, {6, __TXT, "rc.conf0"}, {7, __TXT, "rc.conf1"}, {8, __TXT, "rc.conf2"}, {9, __TXT, "rc.conf3"}, {10, __TXT, "rc.conf4"}, {11, __TXT, "rc.conf5"}, {12, __TXT, "rc.conf6"}, {13, __TXT, "rc.conf7"}, {14, __TXT, "rc.conf8"}, {15, __TXT, "rc.conf9"}, {20, __TXT, "boot.nfsroot.options"}, {245, __INDIR, ""}, {246, __INDIR, ""}, {247, __INDIR, ""}, {248, __INDIR, ""}, {249, __INDIR, ""}, {250, __INDIR, ""}, {251, __INDIR, ""}, {252, __INDIR, ""}, {253, __INDIR, ""}, {254, __INDIR, ""}, #elif DHCP_ENV == DHCP_ENV_PXE /* some pxe options, RFC4578 */ {0, 0, "pxe"}, /* prefix */ {93, __16, "system-architecture"}, {94, __BYTES, "network-interface"}, {97, __BYTES, "machine-identifier"}, #else /* default (empty) table */ {0, 0, "dhcp.vendor."}, /* prefix */ #endif {0, __TXT, "%soption-%d"} }; static struct dhcp_opt dhcp_opt[] = { /* DHCP Option names, formats and codes, from RFC2132. */ {0, 0, "dhcp."}, // prefix {1, __IP, "subnet-mask"}, {2, __32, "time-offset"}, /* this is signed */ {3, __IP, "routers"}, {4, __IP, "time-servers"}, {5, __IP, "ien116-name-servers"}, {6, __IP, "domain-name-servers"}, {7, __IP, "log-servers"}, {8, __IP, "cookie-servers"}, {9, __IP, "lpr-servers"}, {10, __IP, "impress-servers"}, {11, __IP, "resource-location-servers"}, {12, __TXT, "host-name"}, {13, __16, "boot-size"}, {14, __TXT, "merit-dump"}, {15, __TXT, "domain-name"}, {16, __IP, "swap-server"}, {17, __TXT, "root-path"}, {18, __TXT, "extensions-path"}, {19, __8, "ip-forwarding"}, {20, __8, "non-local-source-routing"}, {21, __IP, "policy-filter"}, {22, __16, "max-dgram-reassembly"}, {23, __8, "default-ip-ttl"}, {24, __32, "path-mtu-aging-timeout"}, {25, __16, "path-mtu-plateau-table"}, {26, __16, "interface-mtu"}, {27, __8, "all-subnets-local"}, {28, __IP, "broadcast-address"}, {29, __8, "perform-mask-discovery"}, {30, __8, "mask-supplier"}, {31, __8, "perform-router-discovery"}, {32, __IP, "router-solicitation-address"}, {33, __IP, "static-routes"}, {34, __8, "trailer-encapsulation"}, {35, __32, "arp-cache-timeout"}, {36, __8, "ieee802-3-encapsulation"}, {37, __8, "default-tcp-ttl"}, {38, __32, "tcp-keepalive-interval"}, {39, __8, "tcp-keepalive-garbage"}, {40, __TXT, "nis-domain"}, {41, __IP, "nis-servers"}, {42, __IP, "ntp-servers"}, {43, __VE, "vendor-encapsulated-options"}, {44, __IP, "netbios-name-servers"}, {45, __IP, "netbios-dd-server"}, {46, __8, "netbios-node-type"}, {47, __TXT, "netbios-scope"}, {48, __IP, "x-font-servers"}, {49, __IP, "x-display-managers"}, {50, __IP, "dhcp-requested-address"}, {51, __32, "dhcp-lease-time"}, {52, __8, "dhcp-option-overload"}, {53, __8, "dhcp-message-type"}, {54, __IP, "dhcp-server-identifier"}, {55, __8, "dhcp-parameter-request-list"}, {56, __TXT, "dhcp-message"}, {57, __16, "dhcp-max-message-size"}, {58, __32, "dhcp-renewal-time"}, {59, __32, "dhcp-rebinding-time"}, {60, __TXT, "vendor-class-identifier"}, {61, __TXT, "dhcp-client-identifier"}, {64, __TXT, "nisplus-domain"}, {65, __IP, "nisplus-servers"}, {66, __TXT, "tftp-server-name"}, {67, __TXT, "bootfile-name"}, {68, __IP, "mobile-ip-home-agent"}, {69, __IP, "smtp-server"}, {70, __IP, "pop-server"}, {71, __IP, "nntp-server"}, {72, __IP, "www-server"}, {73, __IP, "finger-server"}, {74, __IP, "irc-server"}, {75, __IP, "streettalk-server"}, {76, __IP, "streettalk-directory-assistance-server"}, {77, __TXT, "user-class"}, {85, __IP, "nds-servers"}, {86, __TXT, "nds-tree-name"}, {87, __TXT, "nds-context"}, {210, __TXT, "authenticate"}, /* use the following entries for arbitrary variables */ {246, __ILIST, ""}, {247, __ILIST, ""}, {248, __ILIST, ""}, {249, __ILIST, ""}, {250, __INDIR, ""}, {251, __INDIR, ""}, {252, __INDIR, ""}, {253, __INDIR, ""}, {254, __INDIR, ""}, {0, __TXT, "%soption-%d"} }; /* * parse a dhcp response, set environment variables translating options * names and values according to the tables above. Also set dhcp.tags * to the list of selected tags. */ static void setenv_(u_char *cp, u_char *ep, struct dhcp_opt *opts) { u_char *ncp; u_char tag; char tags[512], *tp; /* the list of tags */ #define FLD_SEP ',' /* separator in list of elements */ ncp = cp; tp = tags; if (opts == NULL) opts = dhcp_opt; while (ncp < ep) { unsigned int size; /* option size */ char *vp, *endv, buf[256]; /* the value buffer */ struct dhcp_opt *op; tag = *ncp++; /* extract tag and size */ size = *ncp++; cp = ncp; /* current payload */ ncp += size; /* point to the next option */ if (tag == TAG_END) break; if (tag == 0) continue; for (op = opts+1; op->tag && op->tag != tag; op++) ; /* if not found we end up on the default entry */ /* * Copy data into the buffer. libstand does not have snprintf so we * need to be careful with sprintf(). With strings, the source is * always <256 char so shorter than the buffer so we are safe; with * other arguments, the longest string is inet_ntoa which is 16 bytes * so we make sure to have always enough room in the string before * trying an sprint. */ vp = buf; *vp = '\0'; endv = buf + sizeof(buf) - 1 - 16; /* last valid write position */ switch(op->fmt) { case __NONE: break; /* should not happen */ case __VE: /* recurse, vendor specific */ setenv_(cp, cp+size, vndr_opt); break; case __IP: /* ip address */ for (; size > 0 && vp < endv; size -= 4, cp += 4) { struct in_addr in_ip; /* ip addresses */ if (vp != buf) *vp++ = FLD_SEP; bcopy(cp, &in_ip.s_addr, sizeof(in_ip.s_addr)); sprintf(vp, "%s", inet_ntoa(in_ip)); vp += strlen(vp); } break; case __BYTES: /* opaque byte string */ for (; size > 0 && vp < endv; size -= 1, cp += 1) { sprintf(vp, "%02x", *cp); vp += strlen(vp); } break; case __TXT: bcopy(cp, buf, size); /* cannot overflow */ buf[size] = 0; break; case __32: case __16: case __8: /* op->fmt is also the length of each field */ for (; size > 0 && vp < endv; size -= op->fmt, cp += op->fmt) { uint32_t v; if (op->fmt == __32) v = (cp[0]<<24) + (cp[1]<<16) + (cp[2]<<8) + cp[3]; else if (op->fmt == __16) v = (cp[0]<<8) + cp[1]; else v = cp[0]; if (vp != buf) *vp++ = FLD_SEP; sprintf(vp, "%u", v); vp += strlen(vp); } break; case __INDIR: /* name=value */ case __ILIST: /* name=value;name=value... */ bcopy(cp, buf, size); /* cannot overflow */ buf[size] = '\0'; for (endv = buf; endv; endv = vp) { u_char *s = NULL; /* semicolon ? */ /* skip leading whitespace */ while (*endv && strchr(" \t\n\r", *endv)) endv++; vp = strchr(endv, '='); /* find name=value separator */ if (!vp) break; *vp++ = 0; if (op->fmt == __ILIST && (s = strchr(vp, ';'))) *s++ = '\0'; setenv(endv, vp, 1); vp = s; /* prepare for next round */ } buf[0] = '\0'; /* option already done */ } if (tp - tags < sizeof(tags) - 5) { /* add tag to the list */ if (tp != tags) *tp++ = FLD_SEP; sprintf(tp, "%d", tag); tp += strlen(tp); } if (buf[0]) { char env[128]; /* the string name */ if (op->tag == 0) sprintf(env, op->desc, opts[0].desc, tag); else sprintf(env, "%s%s", opts[0].desc, op->desc); /* * Do not replace existing values in the environment, so that * locally-obtained values can override server-provided values. */ setenv(env, buf, 0); } } if (tp != tags) { char env[128]; /* the string name */ sprintf(env, "%stags", opts[0].desc); setenv(env, tags, 1); } } #endif /* additional dhcp */ Index: projects/netbsd-tests-update-12/lib/libstand/bootp.h =================================================================== --- projects/netbsd-tests-update-12/lib/libstand/bootp.h (revision 305171) +++ projects/netbsd-tests-update-12/lib/libstand/bootp.h (revision 305172) @@ -1,146 +1,147 @@ /* $NetBSD: bootp.h,v 1.4 1997/09/06 13:55:57 drochner Exp $ */ /* * Bootstrap Protocol (BOOTP). RFC951 and RFC1048. * * This file specifies the "implementation-independent" BOOTP protocol * information which is common to both client and server. * * Copyright 1988 by Carnegie Mellon. * * Permission to use, copy, modify, and distribute this program for any * purpose and without fee is hereby granted, provided that this copyright * and permission notice appear on all copies and supporting documentation, * the name of Carnegie Mellon not be used in advertising or publicity * pertaining to distribution of the program without specific prior * permission, and notice be given in supporting documentation that copying * and distribution is by permission of Carnegie Mellon and Stanford * University. Carnegie Mellon makes no representations about the * suitability of this software for any purpose. It is provided "as is" * without express or implied warranty. * * $FreeBSD$ */ struct bootp { unsigned char bp_op; /* packet opcode type */ unsigned char bp_htype; /* hardware addr type */ unsigned char bp_hlen; /* hardware addr length */ unsigned char bp_hops; /* gateway hops */ unsigned int bp_xid; /* transaction ID */ unsigned short bp_secs; /* seconds since boot began */ unsigned short bp_flags; struct in_addr bp_ciaddr; /* client IP address */ struct in_addr bp_yiaddr; /* 'your' IP address */ struct in_addr bp_siaddr; /* server IP address */ struct in_addr bp_giaddr; /* gateway IP address */ unsigned char bp_chaddr[16]; /* client hardware address */ unsigned char bp_sname[64]; /* server host name */ unsigned char bp_file[128]; /* boot file name */ #ifdef SUPPORT_DHCP #define BOOTP_VENDSIZE 312 #else #define BOOTP_VENDSIZE 64 #endif unsigned char bp_vend[BOOTP_VENDSIZE]; /* vendor-specific area */ }; /* * UDP port numbers, server and client. */ #define IPPORT_BOOTPS 67 #define IPPORT_BOOTPC 68 #define BOOTREPLY 2 #define BOOTREQUEST 1 /* * Vendor magic cookie (v_magic) for CMU */ #define VM_CMU "CMU" /* * Vendor magic cookie (v_magic) for RFC1048 */ #define VM_RFC1048 { 99, 130, 83, 99 } /* * RFC1048 tag values used to specify what information is being supplied in * the vendor field of the packet. */ #define TAG_PAD ((unsigned char) 0) #define TAG_SUBNET_MASK ((unsigned char) 1) #define TAG_TIME_OFFSET ((unsigned char) 2) #define TAG_GATEWAY ((unsigned char) 3) #define TAG_TIME_SERVER ((unsigned char) 4) #define TAG_NAME_SERVER ((unsigned char) 5) #define TAG_DOMAIN_SERVER ((unsigned char) 6) #define TAG_LOG_SERVER ((unsigned char) 7) #define TAG_COOKIE_SERVER ((unsigned char) 8) #define TAG_LPR_SERVER ((unsigned char) 9) #define TAG_IMPRESS_SERVER ((unsigned char) 10) #define TAG_RLP_SERVER ((unsigned char) 11) #define TAG_HOSTNAME ((unsigned char) 12) #define TAG_BOOTSIZE ((unsigned char) 13) #define TAG_DUMPFILE ((unsigned char) 14) #define TAG_DOMAINNAME ((unsigned char) 15) #define TAG_SWAPSERVER ((unsigned char) 16) #define TAG_ROOTPATH ((unsigned char) 17) #define TAG_INTF_MTU ((unsigned char) 26) #ifdef SUPPORT_DHCP #define TAG_REQ_ADDR ((unsigned char) 50) #define TAG_LEASETIME ((unsigned char) 51) #define TAG_OVERLOAD ((unsigned char) 52) #define TAG_DHCP_MSGTYPE ((unsigned char) 53) #define TAG_SERVERID ((unsigned char) 54) #define TAG_PARAM_REQ ((unsigned char) 55) #define TAG_MSG ((unsigned char) 56) #define TAG_MAXSIZE ((unsigned char) 57) #define TAG_T1 ((unsigned char) 58) #define TAG_T2 ((unsigned char) 59) #define TAG_CLASSID ((unsigned char) 60) #define TAG_CLIENTID ((unsigned char) 61) +#define TAG_TFTP_SERVER ((unsigned char) 150) #endif #define TAG_END ((unsigned char) 255) #ifdef SUPPORT_DHCP #define DHCPDISCOVER 1 #define DHCPOFFER 2 #define DHCPREQUEST 3 #define DHCPDECLINE 4 #define DHCPACK 5 #define DHCPNAK 6 #define DHCPRELEASE 7 #endif /* * bootp flags */ #define BOOTP_NONE 0x0000 /* No flags */ #define BOOTP_PXE 0x0001 /* Booting from PXE. */ /* * "vendor" data permitted for CMU bootp clients. */ struct cmu_vend { unsigned char v_magic[4]; /* magic number */ unsigned int v_flags; /* flags/opcodes, etc. */ struct in_addr v_smask; /* Subnet mask */ struct in_addr v_dgate; /* Default gateway */ struct in_addr v_dns1, v_dns2; /* Domain name servers */ struct in_addr v_ins1, v_ins2; /* IEN-116 name servers */ struct in_addr v_ts1, v_ts2; /* Time servers */ unsigned char v_unused[25]; /* currently unused */ }; /* v_flags values */ #define VF_SMASK 1 /* Subnet mask field contains valid data */ Index: projects/netbsd-tests-update-12/lib/libstand/globals.c =================================================================== --- projects/netbsd-tests-update-12/lib/libstand/globals.c (revision 305171) +++ projects/netbsd-tests-update-12/lib/libstand/globals.c (revision 305172) @@ -1,37 +1,39 @@ /* $NetBSD: globals.c,v 1.3 1995/09/18 21:19:27 pk Exp $ */ /* * globals.c: * * global variables should be separate, so nothing else * must be included extraneously. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include "stand.h" #include "net.h" u_char bcea[6] = BA; /* broadcast ethernet address */ char rootpath[FNAME_SIZE] = "/"; /* root mount path */ char bootfile[FNAME_SIZE]; /* bootp says to boot this */ char hostname[FNAME_SIZE]; /* our hostname */ int hostnamelen; char domainname[FNAME_SIZE]; /* our DNS domain */ int domainnamelen; +int netproto = NET_NONE; /* Network prototol */ char ifname[IFNAME_SIZE]; /* name of interface (e.g. "le0") */ struct in_addr myip; /* my ip address */ struct in_addr nameip; /* DNS server ip address */ struct in_addr rootip; /* root ip address */ struct in_addr swapip; /* swap ip address */ struct in_addr gateip; /* gateway ip address */ +struct in_addr tftpip; /* TFTP ip address */ n_long netmask = 0xffffff00; /* subnet or net mask */ u_int intf_mtu; /* interface mtu from bootp/dhcp */ int errno; /* our old friend */ Index: projects/netbsd-tests-update-12/lib/libstand/net.h =================================================================== --- projects/netbsd-tests-update-12/lib/libstand/net.h (revision 305171) +++ projects/netbsd-tests-update-12/lib/libstand/net.h (revision 305172) @@ -1,122 +1,133 @@ /* $NetBSD: net.h,v 1.10 1995/10/20 00:46:30 cgd Exp $ */ /* * Copyright (c) 1993 Adam Glass * Copyright (c) 1992 Regents of the University of California. * All rights reserved. * * This software was developed by the Computer Systems Engineering group * at Lawrence Berkeley Laboratory under DARPA contract BG 91-66 and * contributed to Berkeley. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ +#ifndef _STAND_NET_H +#define _STAND_NET_H #ifndef _KERNEL /* XXX - see */ #undef __IPADDR #define __IPADDR(x) htonl((u_int32_t)(x)) #endif #include "iodesc.h" #define BA { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff } +enum net_proto { + NET_NONE, + NET_NFS, + NET_TFTP +}; + /* Returns true if n_long's on the same net */ #define SAMENET(a1, a2, m) ((a1.s_addr & m) == (a2.s_addr & m)) #define MACPY(s, d) bcopy((char *)s, (char *)d, 6) #define MAXTMO 120 /* seconds */ #define MINTMO 2 /* seconds */ #define FNAME_SIZE 128 #define IFNAME_SIZE 16 #define RECV_SIZE 1536 /* XXX delete this */ /* * How much room to leave for headers: * 14: struct ether_header * 20: struct ip * 8: struct udphdr * That's 42 but let's pad it out to 48 bytes. */ #define ETHER_SIZE 14 #define HEADER_SIZE 48 extern u_char bcea[6]; extern char rootpath[FNAME_SIZE]; extern char bootfile[FNAME_SIZE]; extern char hostname[FNAME_SIZE]; extern int hostnamelen; extern char domainname[FNAME_SIZE]; extern int domainnamelen; +extern int netproto; extern char ifname[IFNAME_SIZE]; /* All of these are in network order. */ extern struct in_addr myip; extern struct in_addr rootip; extern struct in_addr swapip; extern struct in_addr gateip; extern struct in_addr nameip; +extern struct in_addr tftpip; extern n_long netmask; extern u_int intf_mtu; extern int debug; /* defined in the machdep sources */ extern struct iodesc sockets[SOPEN_MAX]; /* ARP/RevARP functions: */ u_char *arpwhohas(struct iodesc *, struct in_addr); void arp_reply(struct iodesc *, void *); int rarp_getipaddress(int); /* Link functions: */ ssize_t sendether(struct iodesc *d, void *pkt, size_t len, u_char *dea, int etype); ssize_t readether(struct iodesc *d, void *pkt, size_t len, time_t tleft, u_int16_t *etype); ssize_t sendudp(struct iodesc *, void *, size_t); ssize_t readudp(struct iodesc *, void *, size_t, time_t); ssize_t sendrecv(struct iodesc *, ssize_t (*)(struct iodesc *, void *, size_t), void *, size_t, ssize_t (*)(struct iodesc *, void *, size_t, time_t), void *, size_t); /* bootp/DHCP */ void bootp(int, int); /* Utilities: */ char *ether_sprintf(u_char *); int in_cksum(void *, int); char *inet_ntoa(struct in_addr); char *intoa(n_long); /* similar to inet_ntoa */ n_long inet_addr(char *); /* Machine-dependent functions: */ time_t getsecs(void); +#endif Index: projects/netbsd-tests-update-12/lib/libstand/nfs.c =================================================================== --- projects/netbsd-tests-update-12/lib/libstand/nfs.c (revision 305171) +++ projects/netbsd-tests-update-12/lib/libstand/nfs.c (revision 305172) @@ -1,1474 +1,1480 @@ /* $NetBSD: nfs.c,v 1.2 1998/01/24 12:43:09 drochner Exp $ */ /*- * Copyright (c) 1993 John Brezak * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The name of the author may not be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR `AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE * DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include "rpcv2.h" #include "nfsv2.h" #include "stand.h" #include "net.h" #include "netif.h" #include "rpc.h" #define NFS_DEBUGxx #define NFSREAD_SIZE 1024 /* Define our own NFS attributes without NQNFS stuff. */ #ifdef OLD_NFSV2 struct nfsv2_fattrs { n_long fa_type; n_long fa_mode; n_long fa_nlink; n_long fa_uid; n_long fa_gid; n_long fa_size; n_long fa_blocksize; n_long fa_rdev; n_long fa_blocks; n_long fa_fsid; n_long fa_fileid; struct nfsv2_time fa_atime; struct nfsv2_time fa_mtime; struct nfsv2_time fa_ctime; }; struct nfs_read_args { u_char fh[NFS_FHSIZE]; n_long off; n_long len; n_long xxx; /* XXX what's this for? */ }; /* Data part of nfs rpc reply (also the largest thing we receive) */ struct nfs_read_repl { n_long errno; struct nfsv2_fattrs fa; n_long count; u_char data[NFSREAD_SIZE]; }; #ifndef NFS_NOSYMLINK struct nfs_readlnk_repl { n_long errno; n_long len; char path[NFS_MAXPATHLEN]; }; #endif struct nfs_readdir_args { u_char fh[NFS_FHSIZE]; n_long cookie; n_long count; }; struct nfs_readdir_data { n_long fileid; n_long len; char name[0]; }; struct nfs_readdir_off { n_long cookie; n_long follows; }; struct nfs_iodesc { struct iodesc *iodesc; off_t off; u_char fh[NFS_FHSIZE]; struct nfsv2_fattrs fa; /* all in network order */ }; #else /* !OLD_NFSV2 */ /* NFSv3 definitions */ #define NFS_V3MAXFHSIZE 64 #define NFS_VER3 3 #define RPCMNT_VER3 3 #define NFSPROCV3_LOOKUP 3 #define NFSPROCV3_READLINK 5 #define NFSPROCV3_READ 6 #define NFSPROCV3_READDIR 16 typedef struct { uint32_t val[2]; } n_quad; struct nfsv3_time { uint32_t nfs_sec; uint32_t nfs_nsec; }; struct nfsv3_fattrs { uint32_t fa_type; uint32_t fa_mode; uint32_t fa_nlink; uint32_t fa_uid; uint32_t fa_gid; n_quad fa_size; n_quad fa_used; n_quad fa_rdev; n_quad fa_fsid; n_quad fa_fileid; struct nfsv3_time fa_atime; struct nfsv3_time fa_mtime; struct nfsv3_time fa_ctime; }; /* * For NFSv3, the file handle is variable in size, so most fixed sized * structures for arguments won't work. For most cases, a structure * that starts with any fixed size section is followed by an array * that covers the maximum size required. */ struct nfsv3_readdir_repl { uint32_t errno; uint32_t ok; struct nfsv3_fattrs fa; uint32_t cookiev0; uint32_t cookiev1; }; struct nfsv3_readdir_entry { uint32_t follows; uint32_t fid0; uint32_t fid1; uint32_t len; uint32_t nameplus[0]; }; struct nfs_iodesc { struct iodesc *iodesc; off_t off; uint32_t fhsize; u_char fh[NFS_V3MAXFHSIZE]; struct nfsv3_fattrs fa; /* all in network order */ uint64_t cookie; }; #endif /* OLD_NFSV2 */ /* * XXX interactions with tftp? See nfswrapper.c for a confusing * issue. */ int nfs_open(const char *path, struct open_file *f); static int nfs_close(struct open_file *f); static int nfs_read(struct open_file *f, void *buf, size_t size, size_t *resid); static int nfs_write(struct open_file *f, void *buf, size_t size, size_t *resid); static off_t nfs_seek(struct open_file *f, off_t offset, int where); static int nfs_stat(struct open_file *f, struct stat *sb); static int nfs_readdir(struct open_file *f, struct dirent *d); struct nfs_iodesc nfs_root_node; struct fs_ops nfs_fsops = { "nfs", nfs_open, nfs_close, nfs_read, nfs_write, nfs_seek, nfs_stat, nfs_readdir }; #ifdef OLD_NFSV2 /* * Fetch the root file handle (call mount daemon) * Return zero or error number. */ int nfs_getrootfh(struct iodesc *d, char *path, u_char *fhp) { int len; struct args { n_long len; char path[FNAME_SIZE]; } *args; struct repl { n_long errno; u_char fh[NFS_FHSIZE]; } *repl; struct { n_long h[RPC_HEADER_WORDS]; struct args d; } sdata; struct { n_long h[RPC_HEADER_WORDS]; struct repl d; } rdata; size_t cc; #ifdef NFS_DEBUG if (debug) printf("nfs_getrootfh: %s\n", path); #endif args = &sdata.d; repl = &rdata.d; bzero(args, sizeof(*args)); len = strlen(path); if (len > sizeof(args->path)) len = sizeof(args->path); args->len = htonl(len); bcopy(path, args->path, len); len = 4 + roundup(len, 4); cc = rpc_call(d, RPCPROG_MNT, RPCMNT_VER1, RPCMNT_MOUNT, args, len, repl, sizeof(*repl)); if (cc == -1) { /* errno was set by rpc_call */ return (errno); } if (cc < 4) return (EBADRPC); if (repl->errno) return (ntohl(repl->errno)); bcopy(repl->fh, fhp, sizeof(repl->fh)); return (0); } /* * Lookup a file. Store handle and attributes. * Return zero or error number. */ int nfs_lookupfh(struct nfs_iodesc *d, const char *name, struct nfs_iodesc *newfd) { int len, rlen; struct args { u_char fh[NFS_FHSIZE]; n_long len; char name[FNAME_SIZE]; } *args; struct repl { n_long errno; u_char fh[NFS_FHSIZE]; struct nfsv2_fattrs fa; } *repl; struct { n_long h[RPC_HEADER_WORDS]; struct args d; } sdata; struct { n_long h[RPC_HEADER_WORDS]; struct repl d; } rdata; ssize_t cc; #ifdef NFS_DEBUG if (debug) printf("lookupfh: called\n"); #endif args = &sdata.d; repl = &rdata.d; bzero(args, sizeof(*args)); bcopy(d->fh, args->fh, sizeof(args->fh)); len = strlen(name); if (len > sizeof(args->name)) len = sizeof(args->name); bcopy(name, args->name, len); args->len = htonl(len); len = 4 + roundup(len, 4); len += NFS_FHSIZE; rlen = sizeof(*repl); cc = rpc_call(d->iodesc, NFS_PROG, NFS_VER2, NFSPROC_LOOKUP, args, len, repl, rlen); if (cc == -1) return (errno); /* XXX - from rpc_call */ if (cc < 4) return (EIO); if (repl->errno) { /* saerrno.h now matches NFS error numbers. */ return (ntohl(repl->errno)); } bcopy( repl->fh, &newfd->fh, sizeof(newfd->fh)); bcopy(&repl->fa, &newfd->fa, sizeof(newfd->fa)); return (0); } #ifndef NFS_NOSYMLINK /* * Get the destination of a symbolic link. */ int nfs_readlink(struct nfs_iodesc *d, char *buf) { struct { n_long h[RPC_HEADER_WORDS]; u_char fh[NFS_FHSIZE]; } sdata; struct { n_long h[RPC_HEADER_WORDS]; struct nfs_readlnk_repl d; } rdata; ssize_t cc; #ifdef NFS_DEBUG if (debug) printf("readlink: called\n"); #endif bcopy(d->fh, sdata.fh, NFS_FHSIZE); cc = rpc_call(d->iodesc, NFS_PROG, NFS_VER2, NFSPROC_READLINK, sdata.fh, NFS_FHSIZE, &rdata.d, sizeof(rdata.d)); if (cc == -1) return (errno); if (cc < 4) return (EIO); if (rdata.d.errno) return (ntohl(rdata.d.errno)); rdata.d.len = ntohl(rdata.d.len); if (rdata.d.len > NFS_MAXPATHLEN) return (ENAMETOOLONG); bcopy(rdata.d.path, buf, rdata.d.len); buf[rdata.d.len] = 0; return (0); } #endif /* * Read data from a file. * Return transfer count or -1 (and set errno) */ ssize_t nfs_readdata(struct nfs_iodesc *d, off_t off, void *addr, size_t len) { struct nfs_read_args *args; struct nfs_read_repl *repl; struct { n_long h[RPC_HEADER_WORDS]; struct nfs_read_args d; } sdata; struct { n_long h[RPC_HEADER_WORDS]; struct nfs_read_repl d; } rdata; size_t cc; long x; int hlen, rlen; args = &sdata.d; repl = &rdata.d; bcopy(d->fh, args->fh, NFS_FHSIZE); args->off = htonl((n_long)off); if (len > NFSREAD_SIZE) len = NFSREAD_SIZE; args->len = htonl((n_long)len); args->xxx = htonl((n_long)0); hlen = sizeof(*repl) - NFSREAD_SIZE; cc = rpc_call(d->iodesc, NFS_PROG, NFS_VER2, NFSPROC_READ, args, sizeof(*args), repl, sizeof(*repl)); if (cc == -1) { /* errno was already set by rpc_call */ return (-1); } if (cc < hlen) { errno = EBADRPC; return (-1); } if (repl->errno) { errno = ntohl(repl->errno); return (-1); } rlen = cc - hlen; x = ntohl(repl->count); if (rlen < x) { printf("nfsread: short packet, %d < %ld\n", rlen, x); errno = EBADRPC; return(-1); } bcopy(repl->data, addr, x); return (x); } /* * Open a file. * return zero or error number */ int nfs_open(const char *upath, struct open_file *f) { struct iodesc *desc; struct nfs_iodesc *currfd; char buf[2 * NFS_FHSIZE + 3]; u_char *fh; char *cp; int i; #ifndef NFS_NOSYMLINK struct nfs_iodesc *newfd; struct nfsv2_fattrs *fa; char *ncp; int c; char namebuf[NFS_MAXPATHLEN + 1]; char linkbuf[NFS_MAXPATHLEN + 1]; int nlinks = 0; #endif int error; char *path; + if (netproto != NET_NFS) + return (EINVAL); + #ifdef NFS_DEBUG if (debug) printf("nfs_open: %s (rootpath=%s)\n", upath, rootpath); #endif if (!rootpath[0]) { printf("no rootpath, no nfs\n"); return (ENXIO); } /* * This is silly - we should look at dv_type but that value is * arch dependant and we can't use it here. */ #ifndef __i386__ if (strcmp(f->f_dev->dv_name, "net") != 0) return(EINVAL); #else if (strcmp(f->f_dev->dv_name, "pxe") != 0) return(EINVAL); #endif if (!(desc = socktodesc(*(int *)(f->f_devdata)))) return(EINVAL); /* Bind to a reserved port. */ desc->myport = htons(--rpc_port); desc->destip = rootip; if ((error = nfs_getrootfh(desc, rootpath, nfs_root_node.fh))) return (error); nfs_root_node.fa.fa_type = htonl(NFDIR); nfs_root_node.fa.fa_mode = htonl(0755); nfs_root_node.fa.fa_nlink = htonl(2); nfs_root_node.iodesc = desc; fh = &nfs_root_node.fh[0]; buf[0] = 'X'; cp = &buf[1]; for (i = 0; i < NFS_FHSIZE; i++, cp += 2) sprintf(cp, "%02x", fh[i]); sprintf(cp, "X"); setenv("boot.nfsroot.server", inet_ntoa(rootip), 1); setenv("boot.nfsroot.path", rootpath, 1); setenv("boot.nfsroot.nfshandle", buf, 1); /* Allocate file system specific data structure */ currfd = malloc(sizeof(*newfd)); if (currfd == NULL) { error = ENOMEM; goto out; } #ifndef NFS_NOSYMLINK bcopy(&nfs_root_node, currfd, sizeof(*currfd)); newfd = NULL; cp = path = strdup(upath); if (path == NULL) { error = ENOMEM; goto out; } while (*cp) { /* * Remove extra separators */ while (*cp == '/') cp++; if (*cp == '\0') break; /* * Check that current node is a directory. */ if (currfd->fa.fa_type != htonl(NFDIR)) { error = ENOTDIR; goto out; } /* allocate file system specific data structure */ newfd = malloc(sizeof(*newfd)); newfd->iodesc = currfd->iodesc; /* * Get next component of path name. */ { int len = 0; ncp = cp; while ((c = *cp) != '\0' && c != '/') { if (++len > NFS_MAXNAMLEN) { error = ENOENT; goto out; } cp++; } *cp = '\0'; } /* lookup a file handle */ error = nfs_lookupfh(currfd, ncp, newfd); *cp = c; if (error) goto out; /* * Check for symbolic link */ if (newfd->fa.fa_type == htonl(NFLNK)) { int link_len, len; error = nfs_readlink(newfd, linkbuf); if (error) goto out; link_len = strlen(linkbuf); len = strlen(cp); if (link_len + len > MAXPATHLEN || ++nlinks > MAXSYMLINKS) { error = ENOENT; goto out; } bcopy(cp, &namebuf[link_len], len + 1); bcopy(linkbuf, namebuf, link_len); /* * If absolute pathname, restart at root. * If relative pathname, restart at parent directory. */ cp = namebuf; if (*cp == '/') bcopy(&nfs_root_node, currfd, sizeof(*currfd)); free(newfd); newfd = NULL; continue; } free(currfd); currfd = newfd; newfd = NULL; } error = 0; out: free(newfd); free(path); #else currfd->iodesc = desc; error = nfs_lookupfh(&nfs_root_node, upath, currfd); #endif if (!error) { currfd->off = 0; f->f_fsdata = (void *)currfd; return (0); } #ifdef NFS_DEBUG if (debug) printf("nfs_open: %s lookupfh failed: %s\n", path, strerror(error)); #endif free(currfd); return (error); } int nfs_close(struct open_file *f) { struct nfs_iodesc *fp = (struct nfs_iodesc *)f->f_fsdata; #ifdef NFS_DEBUG if (debug) printf("nfs_close: fp=0x%lx\n", (u_long)fp); #endif if (fp) free(fp); f->f_fsdata = (void *)0; return (0); } /* * read a portion of a file */ int nfs_read(struct open_file *f, void *buf, size_t size, size_t *resid) { struct nfs_iodesc *fp = (struct nfs_iodesc *)f->f_fsdata; ssize_t cc; char *addr = buf; #ifdef NFS_DEBUG if (debug) printf("nfs_read: size=%lu off=%d\n", (u_long)size, (int)fp->off); #endif while ((int)size > 0) { twiddle(16); cc = nfs_readdata(fp, fp->off, (void *)addr, size); /* XXX maybe should retry on certain errors */ if (cc == -1) { #ifdef NFS_DEBUG if (debug) printf("nfs_read: read: %s", strerror(errno)); #endif return (errno); /* XXX - from nfs_readdata */ } if (cc == 0) { #ifdef NFS_DEBUG if (debug) printf("nfs_read: hit EOF unexpectantly"); #endif goto ret; } fp->off += cc; addr += cc; size -= cc; } ret: if (resid) *resid = size; return (0); } /* * Not implemented. */ int nfs_write(struct open_file *f, void *buf, size_t size, size_t *resid) { return (EROFS); } off_t nfs_seek(struct open_file *f, off_t offset, int where) { struct nfs_iodesc *d = (struct nfs_iodesc *)f->f_fsdata; n_long size = ntohl(d->fa.fa_size); switch (where) { case SEEK_SET: d->off = offset; break; case SEEK_CUR: d->off += offset; break; case SEEK_END: d->off = size - offset; break; default: errno = EINVAL; return (-1); } return (d->off); } /* NFNON=0, NFREG=1, NFDIR=2, NFBLK=3, NFCHR=4, NFLNK=5 */ int nfs_stat_types[8] = { 0, S_IFREG, S_IFDIR, S_IFBLK, S_IFCHR, S_IFLNK, 0 }; int nfs_stat(struct open_file *f, struct stat *sb) { struct nfs_iodesc *fp = (struct nfs_iodesc *)f->f_fsdata; n_long ftype, mode; ftype = ntohl(fp->fa.fa_type); mode = ntohl(fp->fa.fa_mode); mode |= nfs_stat_types[ftype & 7]; sb->st_mode = mode; sb->st_nlink = ntohl(fp->fa.fa_nlink); sb->st_uid = ntohl(fp->fa.fa_uid); sb->st_gid = ntohl(fp->fa.fa_gid); sb->st_size = ntohl(fp->fa.fa_size); return (0); } static int nfs_readdir(struct open_file *f, struct dirent *d) { struct nfs_iodesc *fp = (struct nfs_iodesc *)f->f_fsdata; struct nfs_readdir_args *args; struct nfs_readdir_data *rd; struct nfs_readdir_off *roff = NULL; static char *buf; static struct nfs_iodesc *pfp = NULL; static n_long cookie = 0; size_t cc; n_long eof; struct { n_long h[RPC_HEADER_WORDS]; struct nfs_readdir_args d; } sdata; static struct { n_long h[RPC_HEADER_WORDS]; u_char d[NFS_READDIRSIZE]; } rdata; if (fp != pfp || fp->off != cookie) { pfp = NULL; refill: args = &sdata.d; bzero(args, sizeof(*args)); bcopy(fp->fh, args->fh, NFS_FHSIZE); args->cookie = htonl(fp->off); args->count = htonl(NFS_READDIRSIZE); cc = rpc_call(fp->iodesc, NFS_PROG, NFS_VER2, NFSPROC_READDIR, args, sizeof(*args), rdata.d, sizeof(rdata.d)); buf = rdata.d; roff = (struct nfs_readdir_off *)buf; if (ntohl(roff->cookie) != 0) return EIO; pfp = fp; cookie = fp->off; } roff = (struct nfs_readdir_off *)buf; if (ntohl(roff->follows) == 0) { eof = ntohl((roff+1)->cookie); if (eof) { cookie = 0; return ENOENT; } goto refill; } buf += sizeof(struct nfs_readdir_off); rd = (struct nfs_readdir_data *)buf; d->d_namlen = ntohl(rd->len); bcopy(rd->name, d->d_name, d->d_namlen); d->d_name[d->d_namlen] = '\0'; buf += (sizeof(struct nfs_readdir_data) + roundup(htonl(rd->len),4)); roff = (struct nfs_readdir_off *)buf; fp->off = cookie = ntohl(roff->cookie); return 0; } #else /* !OLD_NFSV2 */ /* * Fetch the root file handle (call mount daemon) * Return zero or error number. */ int nfs_getrootfh(struct iodesc *d, char *path, uint32_t *fhlenp, u_char *fhp) { int len; struct args { uint32_t len; char path[FNAME_SIZE]; } *args; struct repl { uint32_t errno; uint32_t fhsize; u_char fh[NFS_V3MAXFHSIZE]; uint32_t authcnt; uint32_t auth[7]; } *repl; struct { uint32_t h[RPC_HEADER_WORDS]; struct args d; } sdata; struct { uint32_t h[RPC_HEADER_WORDS]; struct repl d; } rdata; size_t cc; #ifdef NFS_DEBUG if (debug) printf("nfs_getrootfh: %s\n", path); #endif args = &sdata.d; repl = &rdata.d; bzero(args, sizeof(*args)); len = strlen(path); if (len > sizeof(args->path)) len = sizeof(args->path); args->len = htonl(len); bcopy(path, args->path, len); len = sizeof(uint32_t) + roundup(len, sizeof(uint32_t)); cc = rpc_call(d, RPCPROG_MNT, RPCMNT_VER3, RPCMNT_MOUNT, args, len, repl, sizeof(*repl)); if (cc == -1) /* errno was set by rpc_call */ return (errno); if (cc < 2 * sizeof (uint32_t)) return (EBADRPC); if (repl->errno != 0) return (ntohl(repl->errno)); *fhlenp = ntohl(repl->fhsize); bcopy(repl->fh, fhp, *fhlenp); return (0); } /* * Lookup a file. Store handle and attributes. * Return zero or error number. */ int nfs_lookupfh(struct nfs_iodesc *d, const char *name, struct nfs_iodesc *newfd) { int len, rlen, pos; struct args { uint32_t fhsize; uint32_t fhplusname[1 + (NFS_V3MAXFHSIZE + FNAME_SIZE) / sizeof(uint32_t)]; } *args; struct repl { uint32_t errno; uint32_t fhsize; uint32_t fhplusattr[(NFS_V3MAXFHSIZE + 2 * (sizeof(uint32_t) + sizeof(struct nfsv3_fattrs))) / sizeof(uint32_t)]; } *repl; struct { uint32_t h[RPC_HEADER_WORDS]; struct args d; } sdata; struct { uint32_t h[RPC_HEADER_WORDS]; struct repl d; } rdata; ssize_t cc; #ifdef NFS_DEBUG if (debug) printf("lookupfh: called\n"); #endif args = &sdata.d; repl = &rdata.d; bzero(args, sizeof(*args)); args->fhsize = htonl(d->fhsize); bcopy(d->fh, args->fhplusname, d->fhsize); len = strlen(name); if (len > FNAME_SIZE) len = FNAME_SIZE; pos = roundup(d->fhsize, sizeof(uint32_t)) / sizeof(uint32_t); args->fhplusname[pos++] = htonl(len); bcopy(name, &args->fhplusname[pos], len); len = sizeof(uint32_t) + pos * sizeof(uint32_t) + roundup(len, sizeof(uint32_t)); rlen = sizeof(*repl); cc = rpc_call(d->iodesc, NFS_PROG, NFS_VER3, NFSPROCV3_LOOKUP, args, len, repl, rlen); if (cc == -1) return (errno); /* XXX - from rpc_call */ if (cc < 2 * sizeof(uint32_t)) return (EIO); if (repl->errno != 0) /* saerrno.h now matches NFS error numbers. */ return (ntohl(repl->errno)); newfd->fhsize = ntohl(repl->fhsize); bcopy(repl->fhplusattr, &newfd->fh, newfd->fhsize); pos = roundup(newfd->fhsize, sizeof(uint32_t)) / sizeof(uint32_t); if (repl->fhplusattr[pos++] == 0) return (EIO); bcopy(&repl->fhplusattr[pos], &newfd->fa, sizeof(newfd->fa)); return (0); } #ifndef NFS_NOSYMLINK /* * Get the destination of a symbolic link. */ int nfs_readlink(struct nfs_iodesc *d, char *buf) { struct args { uint32_t fhsize; u_char fh[NFS_V3MAXFHSIZE]; } *args; struct repl { uint32_t errno; uint32_t ok; struct nfsv3_fattrs fa; uint32_t len; u_char path[NFS_MAXPATHLEN]; } *repl; struct { uint32_t h[RPC_HEADER_WORDS]; struct args d; } sdata; struct { uint32_t h[RPC_HEADER_WORDS]; struct repl d; } rdata; ssize_t cc; #ifdef NFS_DEBUG if (debug) printf("readlink: called\n"); #endif args = &sdata.d; repl = &rdata.d; bzero(args, sizeof(*args)); args->fhsize = htonl(d->fhsize); bcopy(d->fh, args->fh, d->fhsize); cc = rpc_call(d->iodesc, NFS_PROG, NFS_VER3, NFSPROCV3_READLINK, args, sizeof(uint32_t) + roundup(d->fhsize, sizeof(uint32_t)), repl, sizeof(*repl)); if (cc == -1) return (errno); if (cc < 2 * sizeof(uint32_t)) return (EIO); if (repl->errno != 0) return (ntohl(repl->errno)); if (repl->ok == 0) return (EIO); repl->len = ntohl(repl->len); if (repl->len > NFS_MAXPATHLEN) return (ENAMETOOLONG); bcopy(repl->path, buf, repl->len); buf[repl->len] = 0; return (0); } #endif /* * Read data from a file. * Return transfer count or -1 (and set errno) */ ssize_t nfs_readdata(struct nfs_iodesc *d, off_t off, void *addr, size_t len) { struct args { uint32_t fhsize; uint32_t fhoffcnt[NFS_V3MAXFHSIZE / sizeof(uint32_t) + 3]; } *args; struct repl { uint32_t errno; uint32_t ok; struct nfsv3_fattrs fa; uint32_t count; uint32_t eof; uint32_t len; u_char data[NFSREAD_SIZE]; } *repl; struct { uint32_t h[RPC_HEADER_WORDS]; struct args d; } sdata; struct { uint32_t h[RPC_HEADER_WORDS]; struct repl d; } rdata; size_t cc; long x; int hlen, rlen, pos; args = &sdata.d; repl = &rdata.d; bzero(args, sizeof(*args)); args->fhsize = htonl(d->fhsize); bcopy(d->fh, args->fhoffcnt, d->fhsize); pos = roundup(d->fhsize, sizeof(uint32_t)) / sizeof(uint32_t); args->fhoffcnt[pos++] = 0; args->fhoffcnt[pos++] = htonl((uint32_t)off); if (len > NFSREAD_SIZE) len = NFSREAD_SIZE; args->fhoffcnt[pos] = htonl((uint32_t)len); hlen = sizeof(*repl) - NFSREAD_SIZE; cc = rpc_call(d->iodesc, NFS_PROG, NFS_VER3, NFSPROCV3_READ, args, 4 * sizeof(uint32_t) + roundup(d->fhsize, sizeof(uint32_t)), repl, sizeof(*repl)); if (cc == -1) /* errno was already set by rpc_call */ return (-1); if (cc < hlen) { errno = EBADRPC; return (-1); } if (repl->errno != 0) { errno = ntohl(repl->errno); return (-1); } rlen = cc - hlen; x = ntohl(repl->count); if (rlen < x) { printf("nfsread: short packet, %d < %ld\n", rlen, x); errno = EBADRPC; return (-1); } bcopy(repl->data, addr, x); return (x); } /* * Open a file. * return zero or error number */ int nfs_open(const char *upath, struct open_file *f) { struct iodesc *desc; struct nfs_iodesc *currfd; char buf[2 * NFS_V3MAXFHSIZE + 3]; u_char *fh; char *cp; int i; #ifndef NFS_NOSYMLINK struct nfs_iodesc *newfd; struct nfsv3_fattrs *fa; char *ncp; int c; char namebuf[NFS_MAXPATHLEN + 1]; char linkbuf[NFS_MAXPATHLEN + 1]; int nlinks = 0; #endif int error; char *path; + + if (netproto != NET_NFS) + return (EINVAL); #ifdef NFS_DEBUG if (debug) printf("nfs_open: %s (rootpath=%s)\n", upath, rootpath); #endif if (!rootpath[0]) { printf("no rootpath, no nfs\n"); return (ENXIO); } /* * This is silly - we should look at dv_type but that value is * arch dependant and we can't use it here. */ #ifndef __i386__ if (strcmp(f->f_dev->dv_name, "net") != 0) return (EINVAL); #else if (strcmp(f->f_dev->dv_name, "pxe") != 0) return (EINVAL); #endif if (!(desc = socktodesc(*(int *)(f->f_devdata)))) return (EINVAL); /* Bind to a reserved port. */ desc->myport = htons(--rpc_port); desc->destip = rootip; if ((error = nfs_getrootfh(desc, rootpath, &nfs_root_node.fhsize, nfs_root_node.fh))) return (error); nfs_root_node.fa.fa_type = htonl(NFDIR); nfs_root_node.fa.fa_mode = htonl(0755); nfs_root_node.fa.fa_nlink = htonl(2); nfs_root_node.iodesc = desc; fh = &nfs_root_node.fh[0]; buf[0] = 'X'; cp = &buf[1]; for (i = 0; i < nfs_root_node.fhsize; i++, cp += 2) sprintf(cp, "%02x", fh[i]); sprintf(cp, "X"); setenv("boot.nfsroot.server", inet_ntoa(rootip), 1); setenv("boot.nfsroot.path", rootpath, 1); setenv("boot.nfsroot.nfshandle", buf, 1); sprintf(buf, "%d", nfs_root_node.fhsize); setenv("boot.nfsroot.nfshandlelen", buf, 1); /* Allocate file system specific data structure */ currfd = malloc(sizeof(*newfd)); if (currfd == NULL) { error = ENOMEM; goto out; } #ifndef NFS_NOSYMLINK bcopy(&nfs_root_node, currfd, sizeof(*currfd)); newfd = NULL; cp = path = strdup(upath); if (path == NULL) { error = ENOMEM; goto out; } while (*cp) { /* * Remove extra separators */ while (*cp == '/') cp++; if (*cp == '\0') break; /* * Check that current node is a directory. */ if (currfd->fa.fa_type != htonl(NFDIR)) { error = ENOTDIR; goto out; } /* allocate file system specific data structure */ newfd = malloc(sizeof(*newfd)); if (newfd == NULL) { error = ENOMEM; goto out; } newfd->iodesc = currfd->iodesc; /* * Get next component of path name. */ { int len = 0; ncp = cp; while ((c = *cp) != '\0' && c != '/') { if (++len > NFS_MAXNAMLEN) { error = ENOENT; goto out; } cp++; } *cp = '\0'; } /* lookup a file handle */ error = nfs_lookupfh(currfd, ncp, newfd); *cp = c; if (error) goto out; /* * Check for symbolic link */ if (newfd->fa.fa_type == htonl(NFLNK)) { int link_len, len; error = nfs_readlink(newfd, linkbuf); if (error) goto out; link_len = strlen(linkbuf); len = strlen(cp); if (link_len + len > MAXPATHLEN || ++nlinks > MAXSYMLINKS) { error = ENOENT; goto out; } bcopy(cp, &namebuf[link_len], len + 1); bcopy(linkbuf, namebuf, link_len); /* * If absolute pathname, restart at root. * If relative pathname, restart at parent directory. */ cp = namebuf; if (*cp == '/') bcopy(&nfs_root_node, currfd, sizeof(*currfd)); free(newfd); newfd = NULL; continue; } free(currfd); currfd = newfd; newfd = NULL; } error = 0; out: free(newfd); free(path); #else currfd->iodesc = desc; error = nfs_lookupfh(&nfs_root_node, upath, currfd); #endif if (!error) { currfd->off = 0; currfd->cookie = 0; f->f_fsdata = (void *)currfd; return (0); } #ifdef NFS_DEBUG if (debug) printf("nfs_open: %s lookupfh failed: %s\n", path, strerror(error)); #endif free(currfd); return (error); } int nfs_close(struct open_file *f) { struct nfs_iodesc *fp = (struct nfs_iodesc *)f->f_fsdata; #ifdef NFS_DEBUG if (debug) printf("nfs_close: fp=0x%lx\n", (u_long)fp); #endif if (fp) free(fp); f->f_fsdata = (void *)0; return (0); } /* * read a portion of a file */ int nfs_read(struct open_file *f, void *buf, size_t size, size_t *resid) { struct nfs_iodesc *fp = (struct nfs_iodesc *)f->f_fsdata; ssize_t cc; char *addr = buf; #ifdef NFS_DEBUG if (debug) printf("nfs_read: size=%lu off=%d\n", (u_long)size, (int)fp->off); #endif while ((int)size > 0) { twiddle(16); cc = nfs_readdata(fp, fp->off, (void *)addr, size); /* XXX maybe should retry on certain errors */ if (cc == -1) { #ifdef NFS_DEBUG if (debug) printf("nfs_read: read: %s", strerror(errno)); #endif return (errno); /* XXX - from nfs_readdata */ } if (cc == 0) { #ifdef NFS_DEBUG if (debug) printf("nfs_read: hit EOF unexpectantly"); #endif goto ret; } fp->off += cc; addr += cc; size -= cc; } ret: if (resid) *resid = size; return (0); } /* * Not implemented. */ int nfs_write(struct open_file *f, void *buf, size_t size, size_t *resid) { return (EROFS); } off_t nfs_seek(struct open_file *f, off_t offset, int where) { struct nfs_iodesc *d = (struct nfs_iodesc *)f->f_fsdata; uint32_t size = ntohl(d->fa.fa_size.val[1]); switch (where) { case SEEK_SET: d->off = offset; break; case SEEK_CUR: d->off += offset; break; case SEEK_END: d->off = size - offset; break; default: errno = EINVAL; return (-1); } return (d->off); } /* NFNON=0, NFREG=1, NFDIR=2, NFBLK=3, NFCHR=4, NFLNK=5, NFSOCK=6, NFFIFO=7 */ int nfs_stat_types[9] = { 0, S_IFREG, S_IFDIR, S_IFBLK, S_IFCHR, S_IFLNK, S_IFSOCK, S_IFIFO, 0 }; int nfs_stat(struct open_file *f, struct stat *sb) { struct nfs_iodesc *fp = (struct nfs_iodesc *)f->f_fsdata; uint32_t ftype, mode; ftype = ntohl(fp->fa.fa_type); mode = ntohl(fp->fa.fa_mode); mode |= nfs_stat_types[ftype & 7]; sb->st_mode = mode; sb->st_nlink = ntohl(fp->fa.fa_nlink); sb->st_uid = ntohl(fp->fa.fa_uid); sb->st_gid = ntohl(fp->fa.fa_gid); sb->st_size = ntohl(fp->fa.fa_size.val[1]); return (0); } static int nfs_readdir(struct open_file *f, struct dirent *d) { struct nfs_iodesc *fp = (struct nfs_iodesc *)f->f_fsdata; struct nfsv3_readdir_repl *repl; struct nfsv3_readdir_entry *rent; static char *buf; static struct nfs_iodesc *pfp = NULL; static uint64_t cookie = 0; size_t cc; int pos; struct args { uint32_t fhsize; uint32_t fhpluscookie[5 + NFS_V3MAXFHSIZE]; } *args; struct { uint32_t h[RPC_HEADER_WORDS]; struct args d; } sdata; static struct { uint32_t h[RPC_HEADER_WORDS]; u_char d[NFS_READDIRSIZE]; } rdata; if (fp != pfp || fp->off != cookie) { pfp = NULL; refill: args = &sdata.d; bzero(args, sizeof(*args)); args->fhsize = htonl(fp->fhsize); bcopy(fp->fh, args->fhpluscookie, fp->fhsize); pos = roundup(fp->fhsize, sizeof(uint32_t)) / sizeof(uint32_t); args->fhpluscookie[pos++] = htonl(fp->off >> 32); args->fhpluscookie[pos++] = htonl(fp->off); args->fhpluscookie[pos++] = htonl(fp->cookie >> 32); args->fhpluscookie[pos++] = htonl(fp->cookie); args->fhpluscookie[pos] = htonl(NFS_READDIRSIZE); cc = rpc_call(fp->iodesc, NFS_PROG, NFS_VER3, NFSPROCV3_READDIR, args, 6 * sizeof(uint32_t) + roundup(fp->fhsize, sizeof(uint32_t)), rdata.d, sizeof(rdata.d)); buf = rdata.d; repl = (struct nfsv3_readdir_repl *)buf; if (repl->errno != 0) return (ntohl(repl->errno)); pfp = fp; cookie = fp->off; fp->cookie = ((uint64_t)ntohl(repl->cookiev0) << 32) | ntohl(repl->cookiev1); buf += sizeof (struct nfsv3_readdir_repl); } rent = (struct nfsv3_readdir_entry *)buf; if (rent->follows == 0) { /* fid0 is actually eof */ if (rent->fid0 != 0) { cookie = 0; return (ENOENT); } goto refill; } d->d_namlen = ntohl(rent->len); bcopy(rent->nameplus, d->d_name, d->d_namlen); d->d_name[d->d_namlen] = '\0'; pos = roundup(d->d_namlen, sizeof(uint32_t)) / sizeof(uint32_t); fp->off = cookie = ((uint64_t)ntohl(rent->nameplus[pos]) << 32) | ntohl(rent->nameplus[pos + 1]); pos += 2; buf = (u_char *)&rent->nameplus[pos]; return (0); } #endif /* OLD_NFSV2 */ Index: projects/netbsd-tests-update-12/lib/libstand/tftp.c =================================================================== --- projects/netbsd-tests-update-12/lib/libstand/tftp.c (revision 305171) +++ projects/netbsd-tests-update-12/lib/libstand/tftp.c (revision 305172) @@ -1,761 +1,766 @@ /* $NetBSD: tftp.c,v 1.4 1997/09/17 16:57:07 drochner Exp $ */ /* * Copyright (c) 1996 * Matthias Drochner. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed for the NetBSD Project * by Matthias Drochner. * 4. The name of the author may not be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); /* * Simple TFTP implementation for libsa. * Assumes: * - socket descriptor (int) at open_file->f_devdata * - server host IP in global servip * Restrictions: * - read only * - lseek only with SEEK_SET or SEEK_CUR * - no big time differences between transfers ( #include #include #include #include #include #include #include "stand.h" #include "net.h" #include "netif.h" #include "tftp.h" struct tftp_handle; static int tftp_open(const char *path, struct open_file *f); static int tftp_close(struct open_file *f); static int tftp_parse_oack(struct tftp_handle *h, char *buf, size_t len); static int tftp_read(struct open_file *f, void *buf, size_t size, size_t *resid); static int tftp_write(struct open_file *f, void *buf, size_t size, size_t *resid); static off_t tftp_seek(struct open_file *f, off_t offset, int where); static int tftp_set_blksize(struct tftp_handle *h, const char *str); static int tftp_stat(struct open_file *f, struct stat *sb); static ssize_t sendrecv_tftp(struct tftp_handle *h, ssize_t (*sproc)(struct iodesc *, void *, size_t), void *sbuf, size_t ssize, ssize_t (*rproc)(struct tftp_handle *h, void *, ssize_t, time_t, unsigned short *), void *rbuf, size_t rsize, unsigned short *rtype); struct fs_ops tftp_fsops = { "tftp", tftp_open, tftp_close, tftp_read, tftp_write, tftp_seek, tftp_stat, null_readdir }; extern struct in_addr servip; static int tftpport = 2000; static int is_open = 0; /* * The legacy TFTP_BLKSIZE value was SEGSIZE(512). * TFTP_REQUESTED_BLKSIZE of 1428 is (Ethernet MTU, less the TFTP, UDP and * IP header lengths). */ #define TFTP_REQUESTED_BLKSIZE 1428 /* * Choose a blksize big enough so we can test with Ethernet * Jumbo frames in the future. */ #define TFTP_MAX_BLKSIZE 9008 struct tftp_handle { struct iodesc *iodesc; int currblock; /* contents of lastdata */ int islastblock; /* flag */ int validsize; int off; char *path; /* saved for re-requests */ unsigned int tftp_blksize; unsigned long tftp_tsize; struct { u_char header[HEADER_SIZE]; struct tftphdr t; u_char space[TFTP_MAX_BLKSIZE]; } __packed __aligned(4) lastdata; }; #define TFTP_MAX_ERRCODE EOPTNEG static const int tftperrors[TFTP_MAX_ERRCODE + 1] = { 0, /* ??? */ ENOENT, EPERM, ENOSPC, EINVAL, /* ??? */ EINVAL, /* ??? */ EEXIST, EINVAL, /* ??? */ EINVAL, /* Option negotiation failed. */ }; static int tftp_getnextblock(struct tftp_handle *h); /* send error message back. */ static void tftp_senderr(struct tftp_handle *h, u_short errcode, const char *msg) { struct { u_char header[HEADER_SIZE]; struct tftphdr t; u_char space[63]; /* +1 from t */ } __packed __aligned(4) wbuf; char *wtail; int len; len = strlen(msg); if (len > sizeof(wbuf.space)) len = sizeof(wbuf.space); wbuf.t.th_opcode = htons((u_short) ERROR); wbuf.t.th_code = htons(errcode); wtail = wbuf.t.th_msg; bcopy(msg, wtail, len); wtail[len] = '\0'; wtail += len + 1; sendudp(h->iodesc, &wbuf.t, wtail - (char *) &wbuf.t); } static void tftp_sendack(struct tftp_handle *h) { struct { u_char header[HEADER_SIZE]; struct tftphdr t; } __packed __aligned(4) wbuf; char *wtail; wbuf.t.th_opcode = htons((u_short) ACK); wtail = (char *) &wbuf.t.th_block; wbuf.t.th_block = htons((u_short) h->currblock); wtail += 2; sendudp(h->iodesc, &wbuf.t, wtail - (char *) &wbuf.t); } static ssize_t recvtftp(struct tftp_handle *h, void *pkt, ssize_t len, time_t tleft, unsigned short *rtype) { struct iodesc *d = h->iodesc; struct tftphdr *t; errno = 0; len = readudp(d, pkt, len, tleft); if (len < 4) return (-1); t = (struct tftphdr *) pkt; *rtype = ntohs(t->th_opcode); switch (ntohs(t->th_opcode)) { case DATA: { int got; - if (htons(t->th_block) != d->xid) { + if (htons(t->th_block) != (u_short) d->xid) { /* * Expected block? */ return (-1); } if (d->xid == 1) { /* * First data packet from new port. */ struct udphdr *uh; uh = (struct udphdr *) pkt - 1; d->destport = uh->uh_sport; } /* else check uh_sport has not changed??? */ got = len - (t->th_data - (char *) t); return got; } case ERROR: if ((unsigned) ntohs(t->th_code) > TFTP_MAX_ERRCODE) { printf("illegal tftp error %d\n", ntohs(t->th_code)); errno = EIO; } else { #ifdef TFTP_DEBUG printf("tftp-error %d\n", ntohs(t->th_code)); #endif errno = tftperrors[ntohs(t->th_code)]; } return (-1); case OACK: { struct udphdr *uh; int tftp_oack_len; /* * Unexpected OACK. TFTP transfer already in progress. * Drop the pkt. */ if (d->xid != 1) { return (-1); } /* * Remember which port this OACK came from, because we need * to send the ACK or errors back to it. */ uh = (struct udphdr *) pkt - 1; d->destport = uh->uh_sport; /* Parse options ACK-ed by the server. */ tftp_oack_len = len - sizeof(t->th_opcode); if (tftp_parse_oack(h, t->th_u.tu_stuff, tftp_oack_len) != 0) { tftp_senderr(h, EOPTNEG, "Malformed OACK"); errno = EIO; return (-1); } return (0); } default: #ifdef TFTP_DEBUG printf("tftp type %d not handled\n", ntohs(t->th_opcode)); #endif return (-1); } } /* send request, expect first block (or error) */ static int tftp_makereq(struct tftp_handle *h) { struct { u_char header[HEADER_SIZE]; struct tftphdr t; u_char space[FNAME_SIZE + 6]; } __packed __aligned(4) wbuf; char *wtail; int l; ssize_t res; struct tftphdr *t; char *tftp_blksize = NULL; int blksize_l; unsigned short rtype = 0; /* * Allow overriding default TFTP block size by setting * a tftp.blksize environment variable. */ if ((tftp_blksize = getenv("tftp.blksize")) != NULL) { tftp_set_blksize(h, tftp_blksize); } wbuf.t.th_opcode = htons((u_short) RRQ); wtail = wbuf.t.th_stuff; l = strlen(h->path); #ifdef TFTP_PREPEND_PATH if (l > FNAME_SIZE - (sizeof(TFTP_PREPEND_PATH) - 1)) return (ENAMETOOLONG); bcopy(TFTP_PREPEND_PATH, wtail, sizeof(TFTP_PREPEND_PATH) - 1); wtail += sizeof(TFTP_PREPEND_PATH) - 1; #else if (l > FNAME_SIZE) return (ENAMETOOLONG); #endif bcopy(h->path, wtail, l + 1); wtail += l + 1; bcopy("octet", wtail, 6); wtail += 6; bcopy("blksize", wtail, 8); wtail += 8; blksize_l = sprintf(wtail, "%d", h->tftp_blksize); wtail += blksize_l + 1; bcopy("tsize", wtail, 6); wtail += 6; bcopy("0", wtail, 2); wtail += 2; t = &h->lastdata.t; /* h->iodesc->myport = htons(--tftpport); */ h->iodesc->myport = htons(tftpport + (getsecs() & 0x3ff)); h->iodesc->destport = htons(IPPORT_TFTP); h->iodesc->xid = 1; /* expected block */ h->currblock = 0; h->islastblock = 0; h->validsize = 0; res = sendrecv_tftp(h, &sendudp, &wbuf.t, wtail - (char *) &wbuf.t, &recvtftp, t, sizeof(*t) + h->tftp_blksize, &rtype); if (rtype == OACK) return (tftp_getnextblock(h)); /* Server ignored our blksize request, revert to TFTP default. */ h->tftp_blksize = SEGSIZE; switch (rtype) { case DATA: { h->currblock = 1; h->validsize = res; h->islastblock = 0; if (res < h->tftp_blksize) { h->islastblock = 1; /* very short file */ tftp_sendack(h); } return (0); } case ERROR: default: return (errno); } } /* ack block, expect next */ static int tftp_getnextblock(struct tftp_handle *h) { struct { u_char header[HEADER_SIZE]; struct tftphdr t; } __packed __aligned(4) wbuf; char *wtail; int res; struct tftphdr *t; unsigned short rtype = 0; wbuf.t.th_opcode = htons((u_short) ACK); wtail = (char *) &wbuf.t.th_block; wbuf.t.th_block = htons((u_short) h->currblock); wtail += 2; t = &h->lastdata.t; h->iodesc->xid = h->currblock + 1; /* expected block */ res = sendrecv_tftp(h, &sendudp, &wbuf.t, wtail - (char *) &wbuf.t, &recvtftp, t, sizeof(*t) + h->tftp_blksize, &rtype); if (res == -1) /* 0 is OK! */ return (errno); h->currblock++; h->validsize = res; if (res < h->tftp_blksize) h->islastblock = 1; /* EOF */ if (h->islastblock == 1) { /* Send an ACK for the last block */ wbuf.t.th_block = htons((u_short) h->currblock); sendudp(h->iodesc, &wbuf.t, wtail - (char *)&wbuf.t); } return (0); } static int tftp_open(const char *path, struct open_file *f) { struct tftp_handle *tftpfile; struct iodesc *io; int res; size_t pathsize; const char *extraslash; + if (netproto != NET_TFTP) + return (EINVAL); + if (strcmp(f->f_dev->dv_name, "net") != 0) { #ifdef __i386__ if (strcmp(f->f_dev->dv_name, "pxe") != 0) return (EINVAL); #else return (EINVAL); #endif } if (is_open) return (EBUSY); tftpfile = (struct tftp_handle *) malloc(sizeof(*tftpfile)); if (!tftpfile) return (ENOMEM); memset(tftpfile, 0, sizeof(*tftpfile)); tftpfile->tftp_blksize = TFTP_REQUESTED_BLKSIZE; tftpfile->iodesc = io = socktodesc(*(int *) (f->f_devdata)); if (io == NULL) return (EINVAL); io->destip = servip; tftpfile->off = 0; pathsize = (strlen(rootpath) + 1 + strlen(path) + 1) * sizeof(char); tftpfile->path = malloc(pathsize); if (tftpfile->path == NULL) { free(tftpfile); return(ENOMEM); } if (rootpath[strlen(rootpath) - 1] == '/' || path[0] == '/') extraslash = ""; else extraslash = "/"; res = snprintf(tftpfile->path, pathsize, "%s%s%s", rootpath, extraslash, path); if (res < 0 || res > pathsize) { free(tftpfile->path); free(tftpfile); return(ENOMEM); } res = tftp_makereq(tftpfile); if (res) { free(tftpfile->path); free(tftpfile); return (res); } f->f_fsdata = (void *) tftpfile; is_open = 1; return (0); } static int tftp_read(struct open_file *f, void *addr, size_t size, size_t *resid /* out */) { struct tftp_handle *tftpfile; tftpfile = (struct tftp_handle *) f->f_fsdata; while (size > 0) { int needblock, count; twiddle(32); needblock = tftpfile->off / tftpfile->tftp_blksize + 1; if (tftpfile->currblock > needblock) { /* seek backwards */ tftp_senderr(tftpfile, 0, "No error: read aborted"); tftp_makereq(tftpfile); /* no error check, it worked * for open */ } while (tftpfile->currblock < needblock) { int res; res = tftp_getnextblock(tftpfile); if (res) { /* no answer */ #ifdef TFTP_DEBUG printf("tftp: read error\n"); #endif return (res); } if (tftpfile->islastblock) break; } if (tftpfile->currblock == needblock) { int offinblock, inbuffer; offinblock = tftpfile->off % tftpfile->tftp_blksize; inbuffer = tftpfile->validsize - offinblock; if (inbuffer < 0) { #ifdef TFTP_DEBUG printf("tftp: invalid offset %d\n", tftpfile->off); #endif return (EINVAL); } count = (size < inbuffer ? size : inbuffer); bcopy(tftpfile->lastdata.t.th_data + offinblock, addr, count); addr = (char *)addr + count; tftpfile->off += count; size -= count; if ((tftpfile->islastblock) && (count == inbuffer)) break; /* EOF */ } else { #ifdef TFTP_DEBUG printf("tftp: block %d not found\n", needblock); #endif return (EINVAL); } } if (resid) *resid = size; return (0); } static int tftp_close(struct open_file *f) { struct tftp_handle *tftpfile; tftpfile = (struct tftp_handle *) f->f_fsdata; /* let it time out ... */ if (tftpfile) { free(tftpfile->path); free(tftpfile); } is_open = 0; return (0); } static int tftp_write(struct open_file *f __unused, void *start __unused, size_t size __unused, size_t *resid __unused /* out */) { return (EROFS); } static int tftp_stat(struct open_file *f, struct stat *sb) { struct tftp_handle *tftpfile; tftpfile = (struct tftp_handle *) f->f_fsdata; sb->st_mode = 0444 | S_IFREG; sb->st_nlink = 1; sb->st_uid = 0; sb->st_gid = 0; - sb->st_size = -1; + sb->st_size = (off_t) tftpfile->tftp_tsize; return (0); } static off_t tftp_seek(struct open_file *f, off_t offset, int where) { struct tftp_handle *tftpfile; tftpfile = (struct tftp_handle *) f->f_fsdata; switch (where) { case SEEK_SET: tftpfile->off = offset; break; case SEEK_CUR: tftpfile->off += offset; break; default: errno = EOFFSET; return (-1); } return (tftpfile->off); } static ssize_t sendrecv_tftp(struct tftp_handle *h, ssize_t (*sproc)(struct iodesc *, void *, size_t), void *sbuf, size_t ssize, ssize_t (*rproc)(struct tftp_handle *, void *, ssize_t, time_t, unsigned short *), void *rbuf, size_t rsize, unsigned short *rtype) { struct iodesc *d = h->iodesc; ssize_t cc; time_t t, t1, tleft; #ifdef TFTP_DEBUG if (debug) printf("sendrecv: called\n"); #endif tleft = MINTMO; t = t1 = getsecs(); for (;;) { if ((getsecs() - t) > MAXTMO) { errno = ETIMEDOUT; return -1; } cc = (*sproc)(d, sbuf, ssize); if (cc != -1 && cc < ssize) panic("sendrecv: short write! (%zd < %zu)", cc, ssize); if (cc == -1) { /* Error on transmit; wait before retrying */ while ((getsecs() - t1) < tleft); continue; } recvnext: /* Try to get a packet and process it. */ cc = (*rproc)(h, rbuf, rsize, tleft, rtype); /* Return on data, EOF or real error. */ if (cc != -1 || errno != 0) return (cc); if ((getsecs() - t1) < tleft) { goto recvnext; } /* Timed out or didn't get the packet we're waiting for */ tleft += MINTMO; if (tleft > (2 * MINTMO)) { tleft = (2 * MINTMO); } t1 = getsecs(); } } static int tftp_set_blksize(struct tftp_handle *h, const char *str) { char *endptr; int new_blksize; int ret = 0; if (h == NULL || str == NULL) return (ret); new_blksize = (unsigned int)strtol(str, &endptr, 0); /* * Only accept blksize value if it is numeric. * RFC2348 specifies that acceptable values are 8-65464. * Let's choose a limit less than MAXRSPACE. */ if (*endptr == '\0' && new_blksize >= 8 && new_blksize <= TFTP_MAX_BLKSIZE) { h->tftp_blksize = new_blksize; ret = 1; } return (ret); } /* * In RFC2347, the TFTP Option Acknowledgement package (OACK) * is used to acknowledge a client's option negotiation request. * The format of an OACK packet is: * +-------+---~~---+---+---~~---+---+---~~---+---+---~~---+---+ * | opc | opt1 | 0 | value1 | 0 | optN | 0 | valueN | 0 | * +-------+---~~---+---+---~~---+---+---~~---+---+---~~---+---+ * * opc * The opcode field contains a 6, for Option Acknowledgment. * * opt1 * The first option acknowledgment, copied from the original * request. * * value1 * The acknowledged value associated with the first option. If * and how this value may differ from the original request is * detailed in the specification for the option. * * optN, valueN * The final option/value acknowledgment pair. */ static int tftp_parse_oack(struct tftp_handle *h, char *buf, size_t len) { /* * We parse the OACK strings into an array * of name-value pairs. */ char *tftp_options[128] = { 0 }; char *val = buf; int i = 0; int option_idx = 0; int blksize_is_set = 0; int tsize = 0; unsigned int orig_blksize; while (option_idx < 128 && i < len) { if (buf[i] == '\0') { if (&buf[i] > val) { tftp_options[option_idx] = val; val = &buf[i] + 1; ++option_idx; } } ++i; } /* Save the block size we requested for sanity check later. */ orig_blksize = h->tftp_blksize; /* * Parse individual TFTP options. * * "blksize" is specified in RFC2348. * * "tsize" is specified in RFC2349. */ for (i = 0; i < option_idx; i += 2) { if (strcasecmp(tftp_options[i], "blksize") == 0) { if (i + 1 < option_idx) blksize_is_set = tftp_set_blksize(h, tftp_options[i + 1]); } else if (strcasecmp(tftp_options[i], "tsize") == 0) { if (i + 1 < option_idx) tsize = strtol(tftp_options[i + 1], (char **)NULL, 10); + if (tsize != 0) + h->tftp_tsize = tsize; } else { /* Do not allow any options we did not expect to be ACKed. */ printf("unexpected tftp option '%s'\n", tftp_options[i]); return (-1); } } if (!blksize_is_set) { /* * If TFTP blksize was not set, try defaulting * to the legacy TFTP blksize of SEGSIZE(512) */ h->tftp_blksize = SEGSIZE; } else if (h->tftp_blksize > orig_blksize) { /* * Server should not be proposing block sizes that * exceed what we said we can handle. */ printf("unexpected blksize %u\n", h->tftp_blksize); return (-1); } #ifdef TFTP_DEBUG printf("tftp_blksize: %u\n", h->tftp_blksize); printf("tftp_tsize: %lu\n", h->tftp_tsize); #endif return 0; } Index: projects/netbsd-tests-update-12/libexec/rtld-elf/tests/libpythagoras/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/libexec/rtld-elf/tests/libpythagoras/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/libexec/rtld-elf/tests/libpythagoras/Makefile.depend (revision 305172) @@ -0,0 +1,18 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libcompiler_rt \ + lib/msun \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/libexec/rtld-elf/tests/libpythagoras/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/libexec/rtld-elf/tests/target/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/libexec/rtld-elf/tests/target/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/libexec/rtld-elf/tests/target/Makefile.depend (revision 305172) @@ -0,0 +1,18 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libcompiler_rt \ + libexec/rtld-elf/tests/libpythagoras \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/libexec/rtld-elf/tests/target/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sbin/newfs_msdos/mkfs_msdos.c =================================================================== --- projects/netbsd-tests-update-12/sbin/newfs_msdos/mkfs_msdos.c (revision 305171) +++ projects/netbsd-tests-update-12/sbin/newfs_msdos/mkfs_msdos.c (revision 305172) @@ -1,959 +1,952 @@ /* * Copyright (c) 1998 Robert Nordier * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE * GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER * IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN * IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef lint static const char rcsid[] = "$FreeBSD$"; #endif /* not lint */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "mkfs_msdos.h" #define MAXU16 0xffff /* maximum unsigned 16-bit quantity */ #define BPN 4 /* bits per nibble */ #define NPB 2 /* nibbles per byte */ #define DOSMAGIC 0xaa55 /* DOS magic number */ #define MINBPS 512 /* minimum bytes per sector */ #define MAXSPC 128 /* maximum sectors per cluster */ #define MAXNFT 16 /* maximum number of FATs */ #define DEFBLK 4096 /* default block size */ #define DEFBLK16 2048 /* default block size FAT16 */ #define DEFRDE 512 /* default root directory entries */ #define RESFTE 2 /* reserved FAT entries */ #define MINCLS12 1U /* minimum FAT12 clusters */ #define MINCLS16 0xff5U /* minimum FAT16 clusters */ #define MINCLS32 0xfff5U /* minimum FAT32 clusters */ #define MAXCLS12 0xff4U /* maximum FAT12 clusters */ #define MAXCLS16 0xfff4U /* maximum FAT16 clusters */ #define MAXCLS32 0xffffff4U /* maximum FAT32 clusters */ #define mincls(fat) ((fat) == 12 ? MINCLS12 : \ (fat) == 16 ? MINCLS16 : \ MINCLS32) #define maxcls(fat) ((fat) == 12 ? MAXCLS12 : \ (fat) == 16 ? MAXCLS16 : \ MAXCLS32) #define mk1(p, x) \ (p) = (u_int8_t)(x) #define mk2(p, x) \ (p)[0] = (u_int8_t)(x), \ (p)[1] = (u_int8_t)((x) >> 010) #define mk4(p, x) \ (p)[0] = (u_int8_t)(x), \ (p)[1] = (u_int8_t)((x) >> 010), \ (p)[2] = (u_int8_t)((x) >> 020), \ (p)[3] = (u_int8_t)((x) >> 030) struct bs { u_int8_t bsJump[3]; /* bootstrap entry point */ u_int8_t bsOemName[8]; /* OEM name and version */ } __packed; struct bsbpb { u_int8_t bpbBytesPerSec[2]; /* bytes per sector */ u_int8_t bpbSecPerClust; /* sectors per cluster */ u_int8_t bpbResSectors[2]; /* reserved sectors */ u_int8_t bpbFATs; /* number of FATs */ u_int8_t bpbRootDirEnts[2]; /* root directory entries */ u_int8_t bpbSectors[2]; /* total sectors */ u_int8_t bpbMedia; /* media descriptor */ u_int8_t bpbFATsecs[2]; /* sectors per FAT */ u_int8_t bpbSecPerTrack[2]; /* sectors per track */ u_int8_t bpbHeads[2]; /* drive heads */ u_int8_t bpbHiddenSecs[4]; /* hidden sectors */ u_int8_t bpbHugeSectors[4]; /* big total sectors */ } __packed; struct bsxbpb { u_int8_t bpbBigFATsecs[4]; /* big sectors per FAT */ u_int8_t bpbExtFlags[2]; /* FAT control flags */ u_int8_t bpbFSVers[2]; /* file system version */ u_int8_t bpbRootClust[4]; /* root directory start cluster */ u_int8_t bpbFSInfo[2]; /* file system info sector */ u_int8_t bpbBackup[2]; /* backup boot sector */ u_int8_t bpbReserved[12]; /* reserved */ } __packed; struct bsx { u_int8_t exDriveNumber; /* drive number */ u_int8_t exReserved1; /* reserved */ u_int8_t exBootSignature; /* extended boot signature */ u_int8_t exVolumeID[4]; /* volume ID number */ u_int8_t exVolumeLabel[11]; /* volume label */ u_int8_t exFileSysType[8]; /* file system type */ } __packed; struct de { u_int8_t deName[11]; /* name and extension */ u_int8_t deAttributes; /* attributes */ u_int8_t rsvd[10]; /* reserved */ u_int8_t deMTime[2]; /* last-modified time */ u_int8_t deMDate[2]; /* last-modified date */ u_int8_t deStartCluster[2]; /* starting cluster */ u_int8_t deFileSize[4]; /* size */ } __packed; struct bpb { u_int bpbBytesPerSec; /* bytes per sector */ u_int bpbSecPerClust; /* sectors per cluster */ u_int bpbResSectors; /* reserved sectors */ u_int bpbFATs; /* number of FATs */ u_int bpbRootDirEnts; /* root directory entries */ u_int bpbSectors; /* total sectors */ u_int bpbMedia; /* media descriptor */ u_int bpbFATsecs; /* sectors per FAT */ u_int bpbSecPerTrack; /* sectors per track */ u_int bpbHeads; /* drive heads */ u_int bpbHiddenSecs; /* hidden sectors */ u_int bpbHugeSectors; /* big total sectors */ u_int bpbBigFATsecs; /* big sectors per FAT */ u_int bpbRootClust; /* root directory start cluster */ u_int bpbFSInfo; /* file system info sector */ u_int bpbBackup; /* backup boot sector */ }; #define BPBGAP 0, 0, 0, 0, 0, 0 static struct { const char *name; struct bpb bpb; } const stdfmt[] = { {"160", {512, 1, 1, 2, 64, 320, 0xfe, 1, 8, 1, BPBGAP}}, {"180", {512, 1, 1, 2, 64, 360, 0xfc, 2, 9, 1, BPBGAP}}, {"320", {512, 2, 1, 2, 112, 640, 0xff, 1, 8, 2, BPBGAP}}, {"360", {512, 2, 1, 2, 112, 720, 0xfd, 2, 9, 2, BPBGAP}}, {"640", {512, 2, 1, 2, 112, 1280, 0xfb, 2, 8, 2, BPBGAP}}, {"720", {512, 2, 1, 2, 112, 1440, 0xf9, 3, 9, 2, BPBGAP}}, {"1200", {512, 1, 1, 2, 224, 2400, 0xf9, 7, 15, 2, BPBGAP}}, {"1232", {1024,1, 1, 2, 192, 1232, 0xfe, 2, 8, 2, BPBGAP}}, {"1440", {512, 1, 1, 2, 224, 2880, 0xf0, 9, 18, 2, BPBGAP}}, {"2880", {512, 2, 1, 2, 240, 5760, 0xf0, 9, 36, 2, BPBGAP}} }; static const u_int8_t bootcode[] = { 0xfa, /* cli */ 0x31, 0xc0, /* xor ax,ax */ 0x8e, 0xd0, /* mov ss,ax */ 0xbc, 0x00, 0x7c, /* mov sp,7c00h */ 0xfb, /* sti */ 0x8e, 0xd8, /* mov ds,ax */ 0xe8, 0x00, 0x00, /* call $ + 3 */ 0x5e, /* pop si */ 0x83, 0xc6, 0x19, /* add si,+19h */ 0xbb, 0x07, 0x00, /* mov bx,0007h */ 0xfc, /* cld */ 0xac, /* lodsb */ 0x84, 0xc0, /* test al,al */ 0x74, 0x06, /* jz $ + 8 */ 0xb4, 0x0e, /* mov ah,0eh */ 0xcd, 0x10, /* int 10h */ 0xeb, 0xf5, /* jmp $ - 9 */ 0x30, 0xe4, /* xor ah,ah */ 0xcd, 0x16, /* int 16h */ 0xcd, 0x19, /* int 19h */ 0x0d, 0x0a, 'N', 'o', 'n', '-', 's', 'y', 's', 't', 'e', 'm', ' ', 'd', 'i', 's', 'k', 0x0d, 0x0a, 'P', 'r', 'e', 's', 's', ' ', 'a', 'n', 'y', ' ', 'k', 'e', 'y', ' ', 't', 'o', ' ', 'r', 'e', 'b', 'o', 'o', 't', 0x0d, 0x0a, 0 }; static volatile sig_atomic_t got_siginfo; static void infohandler(int); static int check_mounted(const char *, mode_t); static int getstdfmt(const char *, struct bpb *); static int getdiskinfo(int, const char *, const char *, int, struct bpb *); static void print_bpb(struct bpb *); static int ckgeom(const char *, u_int, const char *); static void mklabel(u_int8_t *, const char *); static int oklabel(const char *); static void setstr(u_int8_t *, const char *, size_t); int mkfs_msdos(const char *fname, const char *dtype, const struct msdos_options *op) { char buf[MAXPATHLEN]; struct sigaction si_sa; struct stat sb; struct timeval tv; struct bpb bpb; struct tm *tm; struct bs *bs; struct bsbpb *bsbpb; struct bsxbpb *bsxbpb; struct bsx *bsx; struct de *de; u_int8_t *img; const char *bname; ssize_t n; time_t now; u_int fat, bss, rds, cls, dir, lsn, x, x1, x2; int fd, fd1, rv; struct msdos_options o = *op; img = NULL; rv = -1; if (o.block_size && o.sectors_per_cluster) { warnx("Cannot specify both block size and sectors per cluster"); goto done; } if (o.OEM_string && strlen(o.OEM_string) > 8) { warnx("%s: bad OEM string", o.OEM_string); goto done; } if (o.create_size) { if (o.no_create) { warnx("create (-C) is incompatible with -N"); goto done; } fd = open(fname, O_RDWR | O_CREAT | O_TRUNC, 0644); if (fd == -1) { warnx("failed to create %s", fname); goto done; } if (ftruncate(fd, o.create_size)) { warnx("failed to initialize %jd bytes", (intmax_t)o.create_size); goto done; } } else if ((fd = open(fname, o.no_create ? O_RDONLY : O_RDWR)) == -1) { warn("%s", fname); goto done; } if (fstat(fd, &sb)) { warn("%s", fname); goto done; } if (o.create_size) { if (!S_ISREG(sb.st_mode)) warnx("warning, %s is not a regular file", fname); } else { if (!S_ISCHR(sb.st_mode)) warnx("warning, %s is not a character device", fname); } if (!o.no_create) if (check_mounted(fname, sb.st_mode) == -1) goto done; if (o.offset && o.offset != lseek(fd, o.offset, SEEK_SET)) { warnx("cannot seek to %jd", (intmax_t)o.offset); goto done; } memset(&bpb, 0, sizeof(bpb)); if (o.floppy) { if (getstdfmt(o.floppy, &bpb) == -1) goto done; bpb.bpbHugeSectors = bpb.bpbSectors; bpb.bpbSectors = 0; bpb.bpbBigFATsecs = bpb.bpbFATsecs; bpb.bpbFATsecs = 0; } if (o.drive_heads) bpb.bpbHeads = o.drive_heads; if (o.sectors_per_track) bpb.bpbSecPerTrack = o.sectors_per_track; if (o.bytes_per_sector) bpb.bpbBytesPerSec = o.bytes_per_sector; if (o.size) bpb.bpbHugeSectors = o.size; if (o.hidden_sectors_set) bpb.bpbHiddenSecs = o.hidden_sectors; if (!(o.floppy || (o.drive_heads && o.sectors_per_track && o.bytes_per_sector && o.size && o.hidden_sectors_set))) { - off_t delta; getdiskinfo(fd, fname, dtype, o.hidden_sectors_set, &bpb); bpb.bpbHugeSectors -= (o.offset / bpb.bpbBytesPerSec); - delta = bpb.bpbHugeSectors % bpb.bpbSecPerTrack; - if (delta != 0) { - warnx("trim %d sectors to adjust to a multiple of %d", - (int)delta, bpb.bpbSecPerTrack); - bpb.bpbHugeSectors -= delta; - } if (bpb.bpbSecPerClust == 0) { /* set defaults */ if (bpb.bpbHugeSectors <= 6000) /* about 3MB -> 512 bytes */ bpb.bpbSecPerClust = 1; else if (bpb.bpbHugeSectors <= (1<<17)) /* 64M -> 4k */ bpb.bpbSecPerClust = 8; else if (bpb.bpbHugeSectors <= (1<<19)) /* 256M -> 8k */ bpb.bpbSecPerClust = 16; else if (bpb.bpbHugeSectors <= (1<<21)) /* 1G -> 16k */ bpb.bpbSecPerClust = 32; else bpb.bpbSecPerClust = 64; /* otherwise 32k */ } } if (!powerof2(bpb.bpbBytesPerSec)) { warnx("bytes/sector (%u) is not a power of 2", bpb.bpbBytesPerSec); goto done; } if (bpb.bpbBytesPerSec < MINBPS) { warnx("bytes/sector (%u) is too small; minimum is %u", bpb.bpbBytesPerSec, MINBPS); goto done; } if (o.volume_label && !oklabel(o.volume_label)) { warnx("%s: bad volume label", o.volume_label); goto done; } if (!(fat = o.fat_type)) { if (o.floppy) fat = 12; else if (!o.directory_entries && (o.info_sector || o.backup_sector)) fat = 32; } if ((fat == 32 && o.directory_entries) || (fat != 32 && (o.info_sector || o.backup_sector))) { warnx("-%c is not a legal FAT%s option", fat == 32 ? 'e' : o.info_sector ? 'i' : 'k', fat == 32 ? "32" : "12/16"); goto done; } if (o.floppy && fat == 32) bpb.bpbRootDirEnts = 0; if (fat != 0 && fat != 12 && fat != 16 && fat != 32) { warnx("%d: bad FAT type", fat); goto done; } if (o.block_size) { if (!powerof2(o.block_size)) { warnx("block size (%u) is not a power of 2", o.block_size); goto done; } if (o.block_size < bpb.bpbBytesPerSec) { warnx("block size (%u) is too small; minimum is %u", o.block_size, bpb.bpbBytesPerSec); goto done; } if (o.block_size > bpb.bpbBytesPerSec * MAXSPC) { warnx("block size (%u) is too large; maximum is %u", o.block_size, bpb.bpbBytesPerSec * MAXSPC); goto done; } bpb.bpbSecPerClust = o.block_size / bpb.bpbBytesPerSec; } if (o.sectors_per_cluster) { if (!powerof2(o.sectors_per_cluster)) { warnx("sectors/cluster (%u) is not a power of 2", o.sectors_per_cluster); goto done; } bpb.bpbSecPerClust = o.sectors_per_cluster; } if (o.reserved_sectors) bpb.bpbResSectors = o.reserved_sectors; if (o.num_FAT) { if (o.num_FAT > MAXNFT) { warnx("number of FATs (%u) is too large; maximum is %u", o.num_FAT, MAXNFT); goto done; } bpb.bpbFATs = o.num_FAT; } if (o.directory_entries) bpb.bpbRootDirEnts = o.directory_entries; if (o.media_descriptor_set) { if (o.media_descriptor < 0xf0) { warnx("illegal media descriptor (%#x)", o.media_descriptor); goto done; } bpb.bpbMedia = o.media_descriptor; } if (o.sectors_per_fat) bpb.bpbBigFATsecs = o.sectors_per_fat; if (o.info_sector) bpb.bpbFSInfo = o.info_sector; if (o.backup_sector) bpb.bpbBackup = o.backup_sector; bss = 1; bname = NULL; fd1 = -1; if (o.bootstrap) { bname = o.bootstrap; if (!strchr(bname, '/')) { snprintf(buf, sizeof(buf), "/boot/%s", bname); if (!(bname = strdup(buf))) { warn(NULL); goto done; } } if ((fd1 = open(bname, O_RDONLY)) == -1 || fstat(fd1, &sb)) { warn("%s", bname); goto done; } if (!S_ISREG(sb.st_mode) || sb.st_size % bpb.bpbBytesPerSec || sb.st_size < bpb.bpbBytesPerSec || sb.st_size > bpb.bpbBytesPerSec * MAXU16) { warnx("%s: inappropriate file type or format", bname); goto done; } bss = sb.st_size / bpb.bpbBytesPerSec; } if (!bpb.bpbFATs) bpb.bpbFATs = 2; if (!fat) { if (bpb.bpbHugeSectors < (bpb.bpbResSectors ? bpb.bpbResSectors : bss) + howmany((RESFTE + (bpb.bpbSecPerClust ? MINCLS16 : MAXCLS12 + 1)) * (bpb.bpbSecPerClust ? 16 : 12) / BPN, bpb.bpbBytesPerSec * NPB) * bpb.bpbFATs + howmany(bpb.bpbRootDirEnts ? bpb.bpbRootDirEnts : DEFRDE, bpb.bpbBytesPerSec / sizeof(struct de)) + (bpb.bpbSecPerClust ? MINCLS16 : MAXCLS12 + 1) * (bpb.bpbSecPerClust ? bpb.bpbSecPerClust : howmany(DEFBLK, bpb.bpbBytesPerSec))) fat = 12; else if (bpb.bpbRootDirEnts || bpb.bpbHugeSectors < (bpb.bpbResSectors ? bpb.bpbResSectors : bss) + howmany((RESFTE + MAXCLS16) * 2, bpb.bpbBytesPerSec) * bpb.bpbFATs + howmany(DEFRDE, bpb.bpbBytesPerSec / sizeof(struct de)) + (MAXCLS16 + 1) * (bpb.bpbSecPerClust ? bpb.bpbSecPerClust : howmany(8192, bpb.bpbBytesPerSec))) fat = 16; else fat = 32; } x = bss; if (fat == 32) { if (!bpb.bpbFSInfo) { if (x == MAXU16 || x == bpb.bpbBackup) { warnx("no room for info sector"); goto done; } bpb.bpbFSInfo = x; } if (bpb.bpbFSInfo != MAXU16 && x <= bpb.bpbFSInfo) x = bpb.bpbFSInfo + 1; if (!bpb.bpbBackup) { if (x == MAXU16) { warnx("no room for backup sector"); goto done; } bpb.bpbBackup = x; } else if (bpb.bpbBackup != MAXU16 && bpb.bpbBackup == bpb.bpbFSInfo) { warnx("backup sector would overwrite info sector"); goto done; } if (bpb.bpbBackup != MAXU16 && x <= bpb.bpbBackup) x = bpb.bpbBackup + 1; } if (!bpb.bpbResSectors) bpb.bpbResSectors = fat == 32 ? MAX(x, MAX(16384 / bpb.bpbBytesPerSec, 4)) : x; else if (bpb.bpbResSectors < x) { warnx("too few reserved sectors (need %d have %d)", x, bpb.bpbResSectors); goto done; } if (fat != 32 && !bpb.bpbRootDirEnts) bpb.bpbRootDirEnts = DEFRDE; rds = howmany(bpb.bpbRootDirEnts, bpb.bpbBytesPerSec / sizeof(struct de)); if (!bpb.bpbSecPerClust) for (bpb.bpbSecPerClust = howmany(fat == 16 ? DEFBLK16 : DEFBLK, bpb.bpbBytesPerSec); bpb.bpbSecPerClust < MAXSPC && bpb.bpbResSectors + howmany((RESFTE + maxcls(fat)) * (fat / BPN), bpb.bpbBytesPerSec * NPB) * bpb.bpbFATs + rds + (u_int64_t) (maxcls(fat) + 1) * bpb.bpbSecPerClust <= bpb.bpbHugeSectors; bpb.bpbSecPerClust <<= 1) continue; if (fat != 32 && bpb.bpbBigFATsecs > MAXU16) { warnx("too many sectors/FAT for FAT12/16"); goto done; } x1 = bpb.bpbResSectors + rds; x = bpb.bpbBigFATsecs ? bpb.bpbBigFATsecs : 1; if (x1 + (u_int64_t)x * bpb.bpbFATs > bpb.bpbHugeSectors) { warnx("meta data exceeds file system size"); goto done; } x1 += x * bpb.bpbFATs; x = (u_int64_t)(bpb.bpbHugeSectors - x1) * bpb.bpbBytesPerSec * NPB / (bpb.bpbSecPerClust * bpb.bpbBytesPerSec * NPB + fat / BPN * bpb.bpbFATs); x2 = howmany((RESFTE + MIN(x, maxcls(fat))) * (fat / BPN), bpb.bpbBytesPerSec * NPB); if (!bpb.bpbBigFATsecs) { bpb.bpbBigFATsecs = x2; x1 += (bpb.bpbBigFATsecs - 1) * bpb.bpbFATs; } cls = (bpb.bpbHugeSectors - x1) / bpb.bpbSecPerClust; x = (u_int64_t)bpb.bpbBigFATsecs * bpb.bpbBytesPerSec * NPB / (fat / BPN) - RESFTE; if (cls > x) cls = x; if (bpb.bpbBigFATsecs < x2) warnx("warning: sectors/FAT limits file system to %u clusters", cls); if (cls < mincls(fat)) { warnx("%u clusters too few clusters for FAT%u, need %u", cls, fat, mincls(fat)); goto done; } if (cls > maxcls(fat)) { cls = maxcls(fat); bpb.bpbHugeSectors = x1 + (cls + 1) * bpb.bpbSecPerClust - 1; warnx("warning: FAT type limits file system to %u sectors", bpb.bpbHugeSectors); } printf("%s: %u sector%s in %u FAT%u cluster%s " "(%u bytes/cluster)\n", fname, cls * bpb.bpbSecPerClust, cls * bpb.bpbSecPerClust == 1 ? "" : "s", cls, fat, cls == 1 ? "" : "s", bpb.bpbBytesPerSec * bpb.bpbSecPerClust); if (!bpb.bpbMedia) bpb.bpbMedia = !bpb.bpbHiddenSecs ? 0xf0 : 0xf8; if (fat == 32) bpb.bpbRootClust = RESFTE; - if (bpb.bpbHiddenSecs + bpb.bpbHugeSectors <= MAXU16) { + if (bpb.bpbHugeSectors <= MAXU16) { bpb.bpbSectors = bpb.bpbHugeSectors; bpb.bpbHugeSectors = 0; } if (fat != 32) { bpb.bpbFATsecs = bpb.bpbBigFATsecs; bpb.bpbBigFATsecs = 0; } print_bpb(&bpb); if (!o.no_create) { gettimeofday(&tv, NULL); now = tv.tv_sec; tm = localtime(&now); if (!(img = malloc(bpb.bpbBytesPerSec))) { warn(NULL); goto done; } dir = bpb.bpbResSectors + (bpb.bpbFATsecs ? bpb.bpbFATsecs : bpb.bpbBigFATsecs) * bpb.bpbFATs; memset(&si_sa, 0, sizeof(si_sa)); si_sa.sa_handler = infohandler; if (sigaction(SIGINFO, &si_sa, NULL) == -1) { warn("sigaction SIGINFO"); goto done; } for (lsn = 0; lsn < dir + (fat == 32 ? bpb.bpbSecPerClust : rds); lsn++) { if (got_siginfo) { fprintf(stderr,"%s: writing sector %u of %u (%u%%)\n", fname, lsn, (dir + (fat == 32 ? bpb.bpbSecPerClust: rds)), (lsn * 100) / (dir + (fat == 32 ? bpb.bpbSecPerClust: rds))); got_siginfo = 0; } x = lsn; if (o.bootstrap && fat == 32 && bpb.bpbBackup != MAXU16 && bss <= bpb.bpbBackup && x >= bpb.bpbBackup) { x -= bpb.bpbBackup; if (!x && lseek(fd1, o.offset, SEEK_SET)) { warn("%s", bname); goto done; } } if (o.bootstrap && x < bss) { if ((n = read(fd1, img, bpb.bpbBytesPerSec)) == -1) { warn("%s", bname); goto done; } if ((unsigned)n != bpb.bpbBytesPerSec) { warnx("%s: can't read sector %u", bname, x); goto done; } } else memset(img, 0, bpb.bpbBytesPerSec); if (!lsn || (fat == 32 && bpb.bpbBackup != MAXU16 && lsn == bpb.bpbBackup)) { x1 = sizeof(struct bs); bsbpb = (struct bsbpb *)(img + x1); mk2(bsbpb->bpbBytesPerSec, bpb.bpbBytesPerSec); mk1(bsbpb->bpbSecPerClust, bpb.bpbSecPerClust); mk2(bsbpb->bpbResSectors, bpb.bpbResSectors); mk1(bsbpb->bpbFATs, bpb.bpbFATs); mk2(bsbpb->bpbRootDirEnts, bpb.bpbRootDirEnts); mk2(bsbpb->bpbSectors, bpb.bpbSectors); mk1(bsbpb->bpbMedia, bpb.bpbMedia); mk2(bsbpb->bpbFATsecs, bpb.bpbFATsecs); mk2(bsbpb->bpbSecPerTrack, bpb.bpbSecPerTrack); mk2(bsbpb->bpbHeads, bpb.bpbHeads); mk4(bsbpb->bpbHiddenSecs, bpb.bpbHiddenSecs); mk4(bsbpb->bpbHugeSectors, bpb.bpbHugeSectors); x1 += sizeof(struct bsbpb); if (fat == 32) { bsxbpb = (struct bsxbpb *)(img + x1); mk4(bsxbpb->bpbBigFATsecs, bpb.bpbBigFATsecs); mk2(bsxbpb->bpbExtFlags, 0); mk2(bsxbpb->bpbFSVers, 0); mk4(bsxbpb->bpbRootClust, bpb.bpbRootClust); mk2(bsxbpb->bpbFSInfo, bpb.bpbFSInfo); mk2(bsxbpb->bpbBackup, bpb.bpbBackup); x1 += sizeof(struct bsxbpb); } bsx = (struct bsx *)(img + x1); mk1(bsx->exBootSignature, 0x29); if (o.volume_id_set) x = o.volume_id; else x = (((u_int)(1 + tm->tm_mon) << 8 | (u_int)tm->tm_mday) + ((u_int)tm->tm_sec << 8 | (u_int)(tv.tv_usec / 10))) << 16 | ((u_int)(1900 + tm->tm_year) + ((u_int)tm->tm_hour << 8 | (u_int)tm->tm_min)); mk4(bsx->exVolumeID, x); mklabel(bsx->exVolumeLabel, o.volume_label ? o.volume_label : "NO NAME"); snprintf(buf, sizeof(buf), "FAT%u", fat); setstr(bsx->exFileSysType, buf, sizeof(bsx->exFileSysType)); if (!o.bootstrap) { x1 += sizeof(struct bsx); bs = (struct bs *)img; mk1(bs->bsJump[0], 0xeb); mk1(bs->bsJump[1], x1 - 2); mk1(bs->bsJump[2], 0x90); setstr(bs->bsOemName, o.OEM_string ? o.OEM_string : "BSD4.4 ", sizeof(bs->bsOemName)); memcpy(img + x1, bootcode, sizeof(bootcode)); mk2(img + MINBPS - 2, DOSMAGIC); } } else if (fat == 32 && bpb.bpbFSInfo != MAXU16 && (lsn == bpb.bpbFSInfo || (bpb.bpbBackup != MAXU16 && lsn == bpb.bpbBackup + bpb.bpbFSInfo))) { mk4(img, 0x41615252); mk4(img + MINBPS - 28, 0x61417272); mk4(img + MINBPS - 24, 0xffffffff); mk4(img + MINBPS - 20, bpb.bpbRootClust); mk2(img + MINBPS - 2, DOSMAGIC); } else if (lsn >= bpb.bpbResSectors && lsn < dir && !((lsn - bpb.bpbResSectors) % (bpb.bpbFATsecs ? bpb.bpbFATsecs : bpb.bpbBigFATsecs))) { mk1(img[0], bpb.bpbMedia); for (x = 1; x < fat * (fat == 32 ? 3 : 2) / 8; x++) mk1(img[x], fat == 32 && x % 4 == 3 ? 0x0f : 0xff); } else if (lsn == dir && o.volume_label) { de = (struct de *)img; mklabel(de->deName, o.volume_label); mk1(de->deAttributes, 050); x = (u_int)tm->tm_hour << 11 | (u_int)tm->tm_min << 5 | (u_int)tm->tm_sec >> 1; mk2(de->deMTime, x); x = (u_int)(tm->tm_year - 80) << 9 | (u_int)(tm->tm_mon + 1) << 5 | (u_int)tm->tm_mday; mk2(de->deMDate, x); } if ((n = write(fd, img, bpb.bpbBytesPerSec)) == -1) { warn("%s", fname); goto done; } if ((unsigned)n != bpb.bpbBytesPerSec) { warnx("%s: can't write sector %u", fname, lsn); goto done; } } } rv = 0; done: free(img); return rv; } /* * return -1 with error if file system is mounted. */ static int check_mounted(const char *fname, mode_t mode) { struct statfs *mp; const char *s1, *s2; size_t len; int n, r; if (!(n = getmntinfo(&mp, MNT_NOWAIT))) { warn("getmntinfo"); return -1; } len = strlen(_PATH_DEV); s1 = fname; if (!strncmp(s1, _PATH_DEV, len)) s1 += len; r = S_ISCHR(mode) && s1 != fname && *s1 == 'r'; for (; n--; mp++) { s2 = mp->f_mntfromname; if (!strncmp(s2, _PATH_DEV, len)) s2 += len; if ((r && s2 != mp->f_mntfromname && !strcmp(s1 + 1, s2)) || !strcmp(s1, s2)) { warnx("%s is mounted on %s", fname, mp->f_mntonname); return -1; } } return 0; } /* * Get a standard format. */ static int getstdfmt(const char *fmt, struct bpb *bpb) { u_int x, i; x = nitems(stdfmt); for (i = 0; i < x && strcmp(fmt, stdfmt[i].name); i++); if (i == x) { warnx("%s: unknown standard format", fmt); return -1; } *bpb = stdfmt[i].bpb; return 0; } /* * Get disk slice, partition, and geometry information. */ static int getdiskinfo(int fd, const char *fname, const char *dtype, __unused int oflag, struct bpb *bpb) { struct disklabel *lp, dlp; struct fd_type type; off_t ms, hs = 0; lp = NULL; /* If the user specified a disk type, try to use that */ if (dtype != NULL) { lp = getdiskbyname(dtype); } /* Maybe it's a floppy drive */ if (lp == NULL) { if (ioctl(fd, DIOCGMEDIASIZE, &ms) == -1) { struct stat st; if (fstat(fd, &st)) err(1, "cannot get disk size"); /* create a fake geometry for a file image */ ms = st.st_size; dlp.d_secsize = 512; dlp.d_nsectors = 63; dlp.d_ntracks = 255; dlp.d_secperunit = ms / dlp.d_secsize; lp = &dlp; } else if (ioctl(fd, FD_GTYPE, &type) != -1) { dlp.d_secsize = 128 << type.secsize; dlp.d_nsectors = type.sectrac; dlp.d_ntracks = type.heads; dlp.d_secperunit = ms / dlp.d_secsize; lp = &dlp; } } /* Maybe it's a fixed drive */ if (lp == NULL) { if (bpb->bpbBytesPerSec) dlp.d_secsize = bpb->bpbBytesPerSec; if (bpb->bpbBytesPerSec == 0 && ioctl(fd, DIOCGSECTORSIZE, &dlp.d_secsize) == -1) err(1, "cannot get sector size"); dlp.d_secperunit = ms / dlp.d_secsize; if (bpb->bpbSecPerTrack == 0 && ioctl(fd, DIOCGFWSECTORS, &dlp.d_nsectors) == -1) { warn("cannot get number of sectors per track"); dlp.d_nsectors = 63; } if (bpb->bpbHeads == 0 && ioctl(fd, DIOCGFWHEADS, &dlp.d_ntracks) == -1) { warn("cannot get number of heads"); if (dlp.d_secperunit <= 63*1*1024) dlp.d_ntracks = 1; else if (dlp.d_secperunit <= 63*16*1024) dlp.d_ntracks = 16; else dlp.d_ntracks = 255; } hs = (ms / dlp.d_secsize) - dlp.d_secperunit; lp = &dlp; } if (bpb->bpbBytesPerSec == 0) { if (ckgeom(fname, lp->d_secsize, "bytes/sector") == -1) return -1; bpb->bpbBytesPerSec = lp->d_secsize; } if (bpb->bpbSecPerTrack == 0) { if (ckgeom(fname, lp->d_nsectors, "sectors/track") == -1) return -1; bpb->bpbSecPerTrack = lp->d_nsectors; } if (bpb->bpbHeads == 0) { if (ckgeom(fname, lp->d_ntracks, "drive heads") == -1) return -1; bpb->bpbHeads = lp->d_ntracks; } if (bpb->bpbHugeSectors == 0) bpb->bpbHugeSectors = lp->d_secperunit; if (bpb->bpbHiddenSecs == 0) bpb->bpbHiddenSecs = hs; return 0; } /* * Print out BPB values. */ static void print_bpb(struct bpb *bpb) { printf("BytesPerSec=%u SecPerClust=%u ResSectors=%u FATs=%u", bpb->bpbBytesPerSec, bpb->bpbSecPerClust, bpb->bpbResSectors, bpb->bpbFATs); if (bpb->bpbRootDirEnts) printf(" RootDirEnts=%u", bpb->bpbRootDirEnts); if (bpb->bpbSectors) printf(" Sectors=%u", bpb->bpbSectors); printf(" Media=%#x", bpb->bpbMedia); if (bpb->bpbFATsecs) printf(" FATsecs=%u", bpb->bpbFATsecs); printf(" SecPerTrack=%u Heads=%u HiddenSecs=%u", bpb->bpbSecPerTrack, bpb->bpbHeads, bpb->bpbHiddenSecs); if (bpb->bpbHugeSectors) printf(" HugeSectors=%u", bpb->bpbHugeSectors); if (!bpb->bpbFATsecs) { printf(" FATsecs=%u RootCluster=%u", bpb->bpbBigFATsecs, bpb->bpbRootClust); printf(" FSInfo="); printf(bpb->bpbFSInfo == MAXU16 ? "%#x" : "%u", bpb->bpbFSInfo); printf(" Backup="); printf(bpb->bpbBackup == MAXU16 ? "%#x" : "%u", bpb->bpbBackup); } printf("\n"); } /* * Check a disk geometry value. */ static int ckgeom(const char *fname, u_int val, const char *msg) { if (!val) { warnx("%s: no default %s", fname, msg); return -1; } if (val > MAXU16) { warnx("%s: illegal %s %d", fname, msg, val); return -1; } return 0; } /* * Check a volume label. */ static int oklabel(const char *src) { int c, i; for (i = 0; i <= 11; i++) { c = (u_char)*src++; if (c < ' ' + !i || strchr("\"*+,./:;<=>?[\\]|", c)) break; } return i && !c; } /* * Make a volume label. */ static void mklabel(u_int8_t *dest, const char *src) { int c, i; for (i = 0; i < 11; i++) { c = *src ? toupper(*src++) : ' '; *dest++ = !i && c == '\xe5' ? 5 : c; } } /* * Copy string, padding with spaces. */ static void setstr(u_int8_t *dest, const char *src, size_t len) { while (len--) *dest++ = *src ? *src++ : ' '; } static void infohandler(int sig __unused) { got_siginfo = 1; } Index: projects/netbsd-tests-update-12/secure/usr.bin/ssh/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/secure/usr.bin/ssh/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/secure/usr.bin/ssh/Makefile.depend (revision 305172) @@ -1,31 +1,29 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/arpa \ include/gssapi \ include/rpc \ include/xlocale \ - kerberos5/lib/libasn1 \ - kerberos5/lib/libkrb5 \ lib/${CSU_DIR} \ lib/libc \ lib/libcompiler_rt \ lib/libcrypt \ lib/libgssapi \ lib/libldns \ lib/libpam/libpam \ lib/libutil \ lib/libz \ secure/lib/libcrypto \ secure/lib/libssh \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/secure/usr.sbin/sshd/Makefile =================================================================== --- projects/netbsd-tests-update-12/secure/usr.sbin/sshd/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/secure/usr.sbin/sshd/Makefile (revision 305172) @@ -1,57 +1,64 @@ # $FreeBSD$ .include PROG= sshd SRCS= sshd.c auth-rhosts.c auth-passwd.c auth-rsa.c auth-rh-rsa.c \ audit.c audit-bsm.c audit-linux.c platform.c \ sshpty.c sshlogin.c servconf.c serverloop.c \ auth.c auth1.c auth2.c auth-options.c session.c \ auth-chall.c auth2-chall.c groupaccess.c \ auth-skey.c auth-bsdauth.c auth2-hostbased.c auth2-kbdint.c \ auth2-none.c auth2-passwd.c auth2-pubkey.c \ monitor_mm.c monitor.c monitor_wrap.c auth-krb5.c \ auth2-gss.c gss-serv.c gss-serv-krb5.c \ loginrec.c auth-pam.c auth-shadow.c auth-sia.c md5crypt.c \ sftp-server.c sftp-common.c \ sandbox-null.c sandbox-rlimit.c sandbox-systrace.c sandbox-darwin.c \ sandbox-seccomp-filter.c sandbox-capsicum.c sandbox-pledge.c \ sandbox-solaris.c PACKAGE= ssh # gss-genr.c really belongs in libssh; see src/secure/lib/libssh/Makefile SRCS+= gss-genr.c MAN= sshd.8 sshd_config.5 CFLAGS+=-I${SSHDIR} -include ssh_namespace.h SRCS+= ssh_namespace.h # pam should always happen before ssh here for static linking LIBADD= pam ssh util wrap .if ${MK_LDNS} != "no" CFLAGS+= -DHAVE_LDNS=1 #DPADD+= ${LIBLDNS} #LDADD+= -lldns .endif .if ${MK_AUDIT} != "no" CFLAGS+= -DUSE_BSM_AUDIT -DHAVE_GETAUDIT_ADDR LIBADD+= bsm .endif +.if ${MK_BLACKLIST_SUPPORT} != "no" +CFLAGS+= -DUSE_BLACKLIST -I${SRCTOP}/contrib/blacklist/include +SRCS+= blacklist.c +LIBADD+= blacklist +LDFLAGS+=-L${LIBBLACKLISTDIR} +.endif + .if ${MK_KERBEROS_SUPPORT} != "no" CFLAGS+= -include krb5_config.h SRCS+= krb5_config.h LIBADD+= gssapi_krb5 gssapi krb5 .endif LIBADD+= crypto .if defined(LOCALBASE) CFLAGS+= -DXAUTH_PATH=\"${LOCALBASE}/bin/xauth\" .endif .include .PATH: ${SSHDIR} Index: projects/netbsd-tests-update-12/secure/usr.sbin/sshd/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/secure/usr.sbin/sshd/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/secure/usr.sbin/sshd/Makefile.depend (revision 305172) @@ -1,40 +1,41 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/arpa \ include/gssapi \ include/xlocale \ kerberos5/lib/libasn1 \ kerberos5/lib/libgssapi_krb5 \ kerberos5/lib/libheimbase \ kerberos5/lib/libheimipcc \ kerberos5/lib/libhx509 \ kerberos5/lib/libkrb5 \ kerberos5/lib/libroken \ kerberos5/lib/libwind \ lib/${CSU_DIR} \ + lib/libblacklist \ lib/libbsm \ lib/libc \ lib/libcom_err \ lib/libcompiler_rt \ lib/libcrypt \ lib/libgssapi \ lib/libldns \ lib/libpam/libpam \ lib/libthr \ lib/libutil \ lib/libwrap \ lib/libz \ secure/lib/libcrypto \ secure/lib/libssh \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/share/man/man9/mbuf.9 =================================================================== --- projects/netbsd-tests-update-12/share/man/man9/mbuf.9 (revision 305171) +++ projects/netbsd-tests-update-12/share/man/man9/mbuf.9 (revision 305172) @@ -1,1225 +1,1225 @@ .\" Copyright (c) 2000 FreeBSD Inc. .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL [your name] OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" -.Dd February 29, 2016 +.Dd August 30, 2016 .Dt MBUF 9 .Os .\" .Sh NAME .Nm mbuf .Nd "memory management in the kernel IPC subsystem" .\" .Sh SYNOPSIS .In sys/param.h .In sys/systm.h .In sys/mbuf.h .\" .Ss Mbuf allocation macros .Fn MGET "struct mbuf *mbuf" "int how" "short type" .Fn MGETHDR "struct mbuf *mbuf" "int how" "short type" .Ft int .Fn MCLGET "struct mbuf *mbuf" "int how" .Fo MEXTADD .Fa "struct mbuf *mbuf" .Fa "caddr_t buf" .Fa "u_int size" .Fa "void (*free)(void *opt_arg1, void *opt_arg2)" .Fa "void *opt_arg1" .Fa "void *opt_arg2" .Fa "short flags" .Fa "int type" .Fc .\" .Ss Mbuf utility macros .Fn mtod "struct mbuf *mbuf" "type" .Fn M_ALIGN "struct mbuf *mbuf" "u_int len" .Fn MH_ALIGN "struct mbuf *mbuf" "u_int len" .Ft int .Fn M_LEADINGSPACE "struct mbuf *mbuf" .Ft int .Fn M_TRAILINGSPACE "struct mbuf *mbuf" .Fn M_MOVE_PKTHDR "struct mbuf *to" "struct mbuf *from" .Fn M_PREPEND "struct mbuf *mbuf" "int len" "int how" .Fn MCHTYPE "struct mbuf *mbuf" "short type" .Ft int .Fn M_WRITABLE "struct mbuf *mbuf" .\" .Ss Mbuf allocation functions .Ft struct mbuf * .Fn m_get "int how" "short type" .Ft struct mbuf * .Fn m_get2 "int size" "int how" "short type" "int flags" .Ft struct mbuf * .Fn m_getm "struct mbuf *orig" "int len" "int how" "short type" .Ft struct mbuf * .Fn m_getjcl "int how" "short type" "int flags" "int size" .Ft struct mbuf * .Fn m_getcl "int how" "short type" "int flags" .Ft struct mbuf * .Fn m_getclr "int how" "short type" .Ft struct mbuf * .Fn m_gethdr "int how" "short type" .Ft struct mbuf * .Fn m_free "struct mbuf *mbuf" .Ft void .Fn m_freem "struct mbuf *mbuf" .\" .Ss Mbuf utility functions .Ft void .Fn m_adj "struct mbuf *mbuf" "int len" .Ft void .Fn m_align "struct mbuf *mbuf" "int len" .Ft int .Fn m_append "struct mbuf *mbuf" "int len" "c_caddr_t cp" .Ft struct mbuf * .Fn m_prepend "struct mbuf *mbuf" "int len" "int how" .Ft struct mbuf * .Fn m_copyup "struct mbuf *mbuf" "int len" "int dstoff" .Ft struct mbuf * .Fn m_pullup "struct mbuf *mbuf" "int len" .Ft struct mbuf * .Fn m_pulldown "struct mbuf *mbuf" "int offset" "int len" "int *offsetp" .Ft struct mbuf * .Fn m_copym "struct mbuf *mbuf" "int offset" "int len" "int how" .Ft struct mbuf * .Fn m_copypacket "struct mbuf *mbuf" "int how" .Ft struct mbuf * .Fn m_dup "struct mbuf *mbuf" "int how" .Ft void .Fn m_copydata "const struct mbuf *mbuf" "int offset" "int len" "caddr_t buf" .Ft void .Fn m_copyback "struct mbuf *mbuf" "int offset" "int len" "caddr_t buf" .Ft struct mbuf * .Fo m_devget .Fa "char *buf" .Fa "int len" .Fa "int offset" .Fa "struct ifnet *ifp" .Fa "void (*copy)(char *from, caddr_t to, u_int len)" .Fc .Ft void .Fn m_cat "struct mbuf *m" "struct mbuf *n" .Ft void .Fn m_catpkt "struct mbuf *m" "struct mbuf *n" .Ft u_int .Fn m_fixhdr "struct mbuf *mbuf" .Ft void .Fn m_dup_pkthdr "struct mbuf *to" "struct mbuf *from" .Ft void .Fn m_move_pkthdr "struct mbuf *to" "struct mbuf *from" .Ft u_int .Fn m_length "struct mbuf *mbuf" "struct mbuf **last" .Ft struct mbuf * .Fn m_split "struct mbuf *mbuf" "int len" "int how" .Ft int .Fn m_apply "struct mbuf *mbuf" "int off" "int len" "int (*f)(void *arg, void *data, u_int len)" "void *arg" .Ft struct mbuf * .Fn m_getptr "struct mbuf *mbuf" "int loc" "int *off" .Ft struct mbuf * .Fn m_defrag "struct mbuf *m0" "int how" .Ft struct mbuf * .Fn m_collapse "struct mbuf *m0" "int how" "int maxfrags" .Ft struct mbuf * .Fn m_unshare "struct mbuf *m0" "int how" .\" .Sh DESCRIPTION An .Vt mbuf is a basic unit of memory management in the kernel IPC subsystem. Network packets and socket buffers are stored in .Vt mbufs . A network packet may span multiple .Vt mbufs arranged into a .Vt mbuf chain (linked list), which allows adding or trimming network headers with little overhead. .Pp While a developer should not bother with .Vt mbuf internals without serious reason in order to avoid incompatibilities with future changes, it is useful to understand the general structure of an .Vt mbuf . .Pp An .Vt mbuf consists of a variable-sized header and a small internal buffer for data. The total size of an .Vt mbuf , .Dv MSIZE , is a constant defined in .In sys/param.h . The .Vt mbuf header includes: .Bl -tag -width "m_nextpkt" -offset indent .It Va m_next .Pq Vt struct mbuf * A pointer to the next .Vt mbuf in the .Vt mbuf chain . .It Va m_nextpkt .Pq Vt struct mbuf * A pointer to the next .Vt mbuf chain in the queue. .It Va m_data .Pq Vt caddr_t A pointer to data attached to this .Vt mbuf . .It Va m_len .Pq Vt int The length of the data. .It Va m_type .Pq Vt short The type of the data. .It Va m_flags .Pq Vt int The .Vt mbuf flags. .El .Pp The .Vt mbuf flag bits are defined as follows: .Bd -literal /* mbuf flags */ #define M_EXT 0x00000001 /* has associated external storage */ #define M_PKTHDR 0x00000002 /* start of record */ #define M_EOR 0x00000004 /* end of record */ #define M_RDONLY 0x00000008 /* associated data marked read-only */ #define M_PROTO1 0x00001000 /* protocol-specific */ #define M_PROTO2 0x00002000 /* protocol-specific */ #define M_PROTO3 0x00004000 /* protocol-specific */ #define M_PROTO4 0x00008000 /* protocol-specific */ #define M_PROTO5 0x00010000 /* protocol-specific */ #define M_PROTO6 0x00020000 /* protocol-specific */ #define M_PROTO7 0x00040000 /* protocol-specific */ #define M_PROTO8 0x00080000 /* protocol-specific */ #define M_PROTO9 0x00100000 /* protocol-specific */ #define M_PROTO10 0x00200000 /* protocol-specific */ #define M_PROTO11 0x00400000 /* protocol-specific */ #define M_PROTO12 0x00800000 /* protocol-specific */ /* mbuf pkthdr flags (also stored in m_flags) */ #define M_BCAST 0x00000010 /* send/received as link-level broadcast */ #define M_MCAST 0x00000020 /* send/received as link-level multicast */ .Ed .Pp The available .Vt mbuf types are defined as follows: .Bd -literal /* mbuf types */ #define MT_DATA 1 /* dynamic (data) allocation */ #define MT_HEADER MT_DATA /* packet header */ #define MT_SONAME 8 /* socket name */ #define MT_CONTROL 14 /* extra-data protocol message */ #define MT_OOBDATA 15 /* expedited data */ .Ed .Pp The available external buffer types are defined as follows: .Bd -literal /* external buffer types */ #define EXT_CLUSTER 1 /* mbuf cluster */ #define EXT_SFBUF 2 /* sendfile(2)'s sf_bufs */ #define EXT_JUMBOP 3 /* jumbo cluster 4096 bytes */ #define EXT_JUMBO9 4 /* jumbo cluster 9216 bytes */ #define EXT_JUMBO16 5 /* jumbo cluster 16184 bytes */ #define EXT_PACKET 6 /* mbuf+cluster from packet zone */ -#define EXT_MBUF 7 /* external mbuf reference (M_IOVEC) */ +#define EXT_MBUF 7 /* external mbuf reference */ #define EXT_NET_DRV 252 /* custom ext_buf provided by net driver(s) */ #define EXT_MOD_TYPE 253 /* custom module's ext_buf type */ #define EXT_DISPOSABLE 254 /* can throw this buffer away w/page flipping */ #define EXT_EXTREF 255 /* has externally maintained ref_cnt ptr */ .Ed .Pp If the .Dv M_PKTHDR flag is set, a .Vt struct pkthdr Va m_pkthdr is added to the .Vt mbuf header. It contains a pointer to the interface the packet has been received from .Pq Vt struct ifnet Va *rcvif , and the total packet length .Pq Vt int Va len . Optionally, it may also contain an attached list of packet tags .Pq Vt "struct m_tag" . See .Xr mbuf_tags 9 for details. Fields used in offloading checksum calculation to the hardware are kept in .Va m_pkthdr as well. See .Sx HARDWARE-ASSISTED CHECKSUM CALCULATION for details. .Pp If small enough, data is stored in the internal data buffer of an .Vt mbuf . If the data is sufficiently large, another .Vt mbuf may be added to the .Vt mbuf chain , or external storage may be associated with the .Vt mbuf . .Dv MHLEN bytes of data can fit into an .Vt mbuf with the .Dv M_PKTHDR flag set, .Dv MLEN bytes can otherwise. .Pp If external storage is being associated with an .Vt mbuf , the .Va m_ext header is added at the cost of losing the internal data buffer. It includes a pointer to external storage, the size of the storage, a pointer to a function used for freeing the storage, a pointer to an optional argument that can be passed to the function, and a pointer to a reference counter. An .Vt mbuf using external storage has the .Dv M_EXT flag set. .Pp The system supplies a macro for allocating the desired external storage buffer, .Dv MEXTADD . .Pp The allocation and management of the reference counter is handled by the subsystem. .Pp The system also supplies a default type of external storage buffer called an .Vt mbuf cluster . .Vt Mbuf clusters can be allocated and configured with the use of the .Dv MCLGET macro. Each .Vt mbuf cluster is .Dv MCLBYTES in size, where MCLBYTES is a machine-dependent constant. The system defines an advisory macro .Dv MINCLSIZE , which is the smallest amount of data to put into an .Vt mbuf cluster . It is equal to .Dv MHLEN plus one. It is typically preferable to store data into the data region of an .Vt mbuf , if size permits, as opposed to allocating a separate .Vt mbuf cluster to hold the same data. .\" .Ss Macros and Functions There are numerous predefined macros and functions that provide the developer with common utilities. .\" .Bl -ohang -offset indent .It Fn mtod mbuf type Convert an .Fa mbuf pointer to a data pointer. The macro expands to the data pointer cast to the specified .Fa type . .Sy Note : It is advisable to ensure that there is enough contiguous data in .Fa mbuf . See .Fn m_pullup for details. .It Fn MGET mbuf how type Allocate an .Vt mbuf and initialize it to contain internal data. .Fa mbuf will point to the allocated .Vt mbuf on success, or be set to .Dv NULL on failure. The .Fa how argument is to be set to .Dv M_WAITOK or .Dv M_NOWAIT . It specifies whether the caller is willing to block if necessary. A number of other functions and macros related to .Vt mbufs have the same argument because they may at some point need to allocate new .Vt mbufs . .It Fn MGETHDR mbuf how type Allocate an .Vt mbuf and initialize it to contain a packet header and internal data. See .Fn MGET for details. .It Fn MEXTADD mbuf buf size free opt_arg1 opt_arg2 flags type Associate externally managed data with .Fa mbuf . Any internal data contained in the mbuf will be discarded, and the .Dv M_EXT flag will be set. The .Fa buf and .Fa size arguments are the address and length, respectively, of the data. The .Fa free argument points to a function which will be called to free the data when the mbuf is freed; it is only used if .Fa type is .Dv EXT_EXTREF . The .Fa opt_arg1 and .Fa opt_arg2 arguments will be passed unmodified to .Fa free . The .Fa flags argument specifies additional .Vt mbuf flags; it is not necessary to specify .Dv M_EXT . Finally, the .Fa type argument specifies the type of external data, which controls how it will be disposed of when the .Vt mbuf is freed. In most cases, the correct value is .Dv EXT_EXTREF . .It Fn MCLGET mbuf how Allocate and attach an .Vt mbuf cluster to .Fa mbuf . On success, a non-zero value returned; otherwise, 0. Historically, consumers would check for success by testing the .Dv M_EXT flag on the mbuf, but this is now discouraged to avoid unnecessary awareness of the implementation of external storage in protocol stacks and device drivers. .It Fn M_ALIGN mbuf len Set the pointer .Fa mbuf->m_data to place an object of the size .Fa len at the end of the internal data area of .Fa mbuf , long word aligned. Applicable only if .Fa mbuf is newly allocated with .Fn MGET or .Fn m_get . .It Fn MH_ALIGN mbuf len Serves the same purpose as .Fn M_ALIGN does, but only for .Fa mbuf newly allocated with .Fn MGETHDR or .Fn m_gethdr , or initialized by .Fn m_dup_pkthdr or .Fn m_move_pkthdr . .It Fn m_align mbuf len Services the same purpose as .Fn M_ALIGN but handles any type of mbuf. .It Fn M_LEADINGSPACE mbuf Returns the number of bytes available before the beginning of data in .Fa mbuf . .It Fn M_TRAILINGSPACE mbuf Returns the number of bytes available after the end of data in .Fa mbuf . .It Fn M_PREPEND mbuf len how This macro operates on an .Vt mbuf chain . It is an optimized wrapper for .Fn m_prepend that can make use of possible empty space before data (e.g.\& left after trimming of a link-layer header). The new .Vt mbuf chain pointer or .Dv NULL is in .Fa mbuf after the call. .It Fn M_MOVE_PKTHDR to from Using this macro is equivalent to calling .Fn m_move_pkthdr to from . .It Fn M_WRITABLE mbuf This macro will evaluate true if .Fa mbuf is not marked .Dv M_RDONLY and if either .Fa mbuf does not contain external storage or, if it does, then if the reference count of the storage is not greater than 1. The .Dv M_RDONLY flag can be set in .Fa mbuf->m_flags . This can be achieved during setup of the external storage, by passing the .Dv M_RDONLY bit as a .Fa flags argument to the .Fn MEXTADD macro, or can be directly set in individual .Vt mbufs . .It Fn MCHTYPE mbuf type Change the type of .Fa mbuf to .Fa type . This is a relatively expensive operation and should be avoided. .El .Pp The functions are: .Bl -ohang -offset indent .It Fn m_get how type A function version of .Fn MGET for non-critical paths. .It Fn m_get2 size how type flags Allocate an .Vt mbuf with enough space to hold specified amount of data. .It Fn m_getm orig len how type Allocate .Fa len bytes worth of .Vt mbufs and .Vt mbuf clusters if necessary and append the resulting allocated .Vt mbuf chain to the .Vt mbuf chain .Fa orig , if it is .No non- Ns Dv NULL . If the allocation fails at any point, free whatever was allocated and return .Dv NULL . If .Fa orig is .No non- Ns Dv NULL , it will not be freed. It is possible to use .Fn m_getm to either append .Fa len bytes to an existing .Vt mbuf or .Vt mbuf chain (for example, one which may be sitting in a pre-allocated ring) or to simply perform an all-or-nothing .Vt mbuf and .Vt mbuf cluster allocation. .It Fn m_gethdr how type A function version of .Fn MGETHDR for non-critical paths. .It Fn m_getcl how type flags Fetch an .Vt mbuf with a .Vt mbuf cluster attached to it. If one of the allocations fails, the entire allocation fails. This routine is the preferred way of fetching both the .Vt mbuf and .Vt mbuf cluster together, as it avoids having to unlock/relock between allocations. Returns .Dv NULL on failure. .It Fn m_getjcl how type flags size This is like .Fn m_getcl but it the size of the cluster allocated will be large enough for .Fa size bytes. .It Fn m_getclr how type Allocate an .Vt mbuf and zero out the data region. .It Fn m_free mbuf Frees .Vt mbuf . Returns .Va m_next of the freed .Vt mbuf . .El .Pp The functions below operate on .Vt mbuf chains . .Bl -ohang -offset indent .It Fn m_freem mbuf Free an entire .Vt mbuf chain , including any external storage. .\" .It Fn m_adj mbuf len Trim .Fa len bytes from the head of an .Vt mbuf chain if .Fa len is positive, from the tail otherwise. .\" .It Fn m_append mbuf len cp Append .Vt len bytes of data .Vt cp to the .Vt mbuf chain . Extend the mbuf chain if the new data does not fit in existing space. .\" .It Fn m_prepend mbuf len how Allocate a new .Vt mbuf and prepend it to the .Vt mbuf chain , handle .Dv M_PKTHDR properly. .Sy Note : It does not allocate any .Vt mbuf clusters , so .Fa len must be less than .Dv MLEN or .Dv MHLEN , depending on the .Dv M_PKTHDR flag setting. .\" .It Fn m_copyup mbuf len dstoff Similar to .Fn m_pullup but copies .Fa len bytes of data into a new mbuf at .Fa dstoff bytes into the mbuf. The .Fa dstoff argument aligns the data and leaves room for a link layer header. Returns the new .Vt mbuf chain on success, and frees the .Vt mbuf chain and returns .Dv NULL on failure. .Sy Note : The function does not allocate .Vt mbuf clusters , so .Fa len + dstoff must be less than .Dv MHLEN . .\" .It Fn m_pullup mbuf len Arrange that the first .Fa len bytes of an .Vt mbuf chain are contiguous and lay in the data area of .Fa mbuf , so they are accessible with .Fn mtod mbuf type . It is important to remember that this may involve reallocating some mbufs and moving data so all pointers referencing data within the old mbuf chain must be recalculated or made invalid. Return the new .Vt mbuf chain on success, .Dv NULL on failure (the .Vt mbuf chain is freed in this case). .Sy Note : It does not allocate any .Vt mbuf clusters , so .Fa len must be less than or equal to .Dv MHLEN . .\" .It Fn m_pulldown mbuf offset len offsetp Arrange that .Fa len bytes between .Fa offset and .Fa offset + len in the .Vt mbuf chain are contiguous and lay in the data area of .Fa mbuf , so they are accessible with .Fn mtod mbuf type . .Fa len must be smaller than, or equal to, the size of an .Vt mbuf cluster . Return a pointer to an intermediate .Vt mbuf in the chain containing the requested region; the offset in the data region of the .Vt mbuf chain to the data contained in the returned mbuf is stored in .Fa *offsetp . If .Fa offsetp is NULL, the region may be accessed using .Fn mtod mbuf type . If .Fa offsetp is non-NULL, the region may be accessed using .Fn mtod mbuf uint8_t + *offsetp. The region of the mbuf chain between its beginning and .Fa offset is not modified, therefore it is safe to hold pointers to data within this region before calling .Fn m_pulldown . .\" .It Fn m_copym mbuf offset len how Make a copy of an .Vt mbuf chain starting .Fa offset bytes from the beginning, continuing for .Fa len bytes. If .Fa len is .Dv M_COPYALL , copy to the end of the .Vt mbuf chain . .Sy Note : The copy is read-only, because the .Vt mbuf clusters are not copied, only their reference counts are incremented. .\" .It Fn m_copypacket mbuf how Copy an entire packet including header, which must be present. This is an optimized version of the common case .Fn m_copym mbuf 0 M_COPYALL how . .Sy Note : the copy is read-only, because the .Vt mbuf clusters are not copied, only their reference counts are incremented. .\" .It Fn m_dup mbuf how Copy a packet header .Vt mbuf chain into a completely new .Vt mbuf chain , including copying any .Vt mbuf clusters . Use this instead of .Fn m_copypacket when you need a writable copy of an .Vt mbuf chain . .\" .It Fn m_copydata mbuf offset len buf Copy data from an .Vt mbuf chain starting .Fa off bytes from the beginning, continuing for .Fa len bytes, into the indicated buffer .Fa buf . .\" .It Fn m_copyback mbuf offset len buf Copy .Fa len bytes from the buffer .Fa buf back into the indicated .Vt mbuf chain , starting at .Fa offset bytes from the beginning of the .Vt mbuf chain , extending the .Vt mbuf chain if necessary. .Sy Note : It does not allocate any .Vt mbuf clusters , just adds .Vt mbufs to the .Vt mbuf chain . It is safe to set .Fa offset beyond the current .Vt mbuf chain end: zeroed .Vt mbufs will be allocated to fill the space. .\" .It Fn m_length mbuf last Return the length of the .Vt mbuf chain , and optionally a pointer to the last .Vt mbuf . .\" .It Fn m_dup_pkthdr to from how Upon the function's completion, the .Vt mbuf .Fa to will contain an identical copy of .Fa from->m_pkthdr and the per-packet attributes found in the .Vt mbuf chain .Fa from . The .Vt mbuf .Fa from must have the flag .Dv M_PKTHDR initially set, and .Fa to must be empty on entry. .\" .It Fn m_move_pkthdr to from Move .Va m_pkthdr and the per-packet attributes from the .Vt mbuf chain .Fa from to the .Vt mbuf .Fa to . The .Vt mbuf .Fa from must have the flag .Dv M_PKTHDR initially set, and .Fa to must be empty on entry. Upon the function's completion, .Fa from will have the flag .Dv M_PKTHDR and the per-packet attributes cleared. .\" .It Fn m_fixhdr mbuf Set the packet-header length to the length of the .Vt mbuf chain . .\" .It Fn m_devget buf len offset ifp copy Copy data from a device local memory pointed to by .Fa buf to an .Vt mbuf chain . The copy is done using a specified copy routine .Fa copy , or .Fn bcopy if .Fa copy is .Dv NULL . .\" .It Fn m_cat m n Concatenate .Fa n to .Fa m . Both .Vt mbuf chains must be of the same type. .Fa n is not guaranteed to be valid after .Fn m_cat returns. .Fn m_cat does not update any packet header fields or free mbuf tags. .\" .It Fn m_catpkt m n A variant of .Fn m_cat that operates on packets. Both .Fa m and .Fa n must contain packet headers. .Fa n is not guaranteed to be valid after .Fn m_catpkt returns. .\" .It Fn m_split mbuf len how Partition an .Vt mbuf chain in two pieces, returning the tail: all but the first .Fa len bytes. In case of failure, it returns .Dv NULL and attempts to restore the .Vt mbuf chain to its original state. .\" .It Fn m_apply mbuf off len f arg Apply a function to an .Vt mbuf chain , at offset .Fa off , for length .Fa len bytes. Typically used to avoid calls to .Fn m_pullup which would otherwise be unnecessary or undesirable. .Fa arg is a convenience argument which is passed to the callback function .Fa f . .Pp Each time .Fn f is called, it will be passed .Fa arg , a pointer to the .Fa data in the current mbuf, and the length .Fa len of the data in this mbuf to which the function should be applied. .Pp The function should return zero to indicate success; otherwise, if an error is indicated, then .Fn m_apply will return the error and stop iterating through the .Vt mbuf chain . .\" .It Fn m_getptr mbuf loc off Return a pointer to the mbuf containing the data located at .Fa loc bytes from the beginning of the .Vt mbuf chain . The corresponding offset into the mbuf will be stored in .Fa *off . .It Fn m_defrag m0 how Defragment an mbuf chain, returning the shortest possible chain of mbufs and clusters. If allocation fails and this can not be completed, .Dv NULL will be returned and the original chain will be unchanged. Upon success, the original chain will be freed and the new chain will be returned. .Fa how should be either .Dv M_WAITOK or .Dv M_NOWAIT , depending on the caller's preference. .Pp This function is especially useful in network drivers, where certain long mbuf chains must be shortened before being added to TX descriptor lists. .It Fn m_collapse m0 how maxfrags Defragment an mbuf chain, returning a chain of at most .Fa maxfrags mbufs and clusters. If allocation fails or the chain cannot be collapsed as requested, .Dv NULL will be returned, with the original chain possibly modified. As with .Fn m_defrag , .Fa how should be one of .Dv M_WAITOK or .Dv M_NOWAIT . .It Fn m_unshare m0 how Create a version of the specified mbuf chain whose contents can be safely modified without affecting other users. If allocation fails and this operation can not be completed, .Dv NULL will be returned. The original mbuf chain is always reclaimed and the reference count of any shared mbuf clusters is decremented. .Fa how should be either .Dv M_WAITOK or .Dv M_NOWAIT , depending on the caller's preference. As a side-effect of this process the returned mbuf chain may be compacted. .Pp This function is especially useful in the transmit path of network code, when data must be encrypted or otherwise altered prior to transmission. .El .Sh HARDWARE-ASSISTED CHECKSUM CALCULATION This section currently applies to TCP/IP only. In order to save the host CPU resources, computing checksums is offloaded to the network interface hardware if possible. The .Va m_pkthdr member of the leading .Vt mbuf of a packet contains two fields used for that purpose, .Vt int Va csum_flags and .Vt int Va csum_data . The meaning of those fields depends on the direction a packet flows in, and on whether the packet is fragmented. Henceforth, .Va csum_flags or .Va csum_data of a packet will denote the corresponding field of the .Va m_pkthdr member of the leading .Vt mbuf in the .Vt mbuf chain containing the packet. .Pp On output, checksum offloading is attempted after the outgoing interface has been determined for a packet. The interface-specific field .Va ifnet.if_data.ifi_hwassist (see .Xr ifnet 9 ) is consulted for the capabilities of the interface to assist in computing checksums. The .Va csum_flags field of the packet header is set to indicate which actions the interface is supposed to perform on it. The actions unsupported by the network interface are done in the software prior to passing the packet down to the interface driver; such actions will never be requested through .Va csum_flags . .Pp The flags demanding a particular action from an interface are as follows: .Bl -tag -width ".Dv CSUM_TCP" -offset indent .It Dv CSUM_IP The IP header checksum is to be computed and stored in the corresponding field of the packet. The hardware is expected to know the format of an IP header to determine the offset of the IP checksum field. .It Dv CSUM_TCP The TCP checksum is to be computed. (See below.) .It Dv CSUM_UDP The UDP checksum is to be computed. (See below.) .El .Pp Should a TCP or UDP checksum be offloaded to the hardware, the field .Va csum_data will contain the byte offset of the checksum field relative to the end of the IP header. In this case, the checksum field will be initially set by the TCP/IP module to the checksum of the pseudo header defined by the TCP and UDP specifications. .Pp On input, an interface indicates the actions it has performed on a packet by setting one or more of the following flags in .Va csum_flags associated with the packet: .Bl -tag -width ".Dv CSUM_IP_CHECKED" -offset indent .It Dv CSUM_IP_CHECKED The IP header checksum has been computed. .It Dv CSUM_IP_VALID The IP header has a valid checksum. This flag can appear only in combination with .Dv CSUM_IP_CHECKED . .It Dv CSUM_DATA_VALID The checksum of the data portion of the IP packet has been computed and stored in the field .Va csum_data in network byte order. .It Dv CSUM_PSEUDO_HDR Can be set only along with .Dv CSUM_DATA_VALID to indicate that the IP data checksum found in .Va csum_data allows for the pseudo header defined by the TCP and UDP specifications. Otherwise the checksum of the pseudo header must be calculated by the host CPU and added to .Va csum_data to obtain the final checksum to be used for TCP or UDP validation purposes. .El .Pp If a particular network interface just indicates success or failure of TCP or UDP checksum validation without returning the exact value of the checksum to the host CPU, its driver can mark .Dv CSUM_DATA_VALID and .Dv CSUM_PSEUDO_HDR in .Va csum_flags , and set .Va csum_data to .Li 0xFFFF hexadecimal to indicate a valid checksum. It is a peculiarity of the algorithm used that the Internet checksum calculated over any valid packet will be .Li 0xFFFF as long as the original checksum field is included. .Sh STRESS TESTING When running a kernel compiled with the option .Dv MBUF_STRESS_TEST , the following .Xr sysctl 8 Ns -controlled options may be used to create various failure/extreme cases for testing of network drivers and other parts of the kernel that rely on .Vt mbufs . .Bl -tag -width ident .It Va net.inet.ip.mbuf_frag_size Causes .Fn ip_output to fragment outgoing .Vt mbuf chains into fragments of the specified size. Setting this variable to 1 is an excellent way to test the long .Vt mbuf chain handling ability of network drivers. .It Va kern.ipc.m_defragrandomfailures Causes the function .Fn m_defrag to randomly fail, returning .Dv NULL . Any piece of code which uses .Fn m_defrag should be tested with this feature. .El .Sh RETURN VALUES See above. .Sh SEE ALSO .Xr ifnet 9 , .Xr mbuf_tags 9 .Sh HISTORY .\" Please correct me if I'm wrong .Vt Mbufs appeared in an early version of .Bx . Besides being used for network packets, they were used to store various dynamic structures, such as routing table entries, interface addresses, protocol control blocks, etc. In more recent .Fx use of .Vt mbufs is almost entirely limited to packet storage, with .Xr uma 9 zones being used directly to store other network-related memory. .Pp Historically, the .Vt mbuf allocator has been a special-purpose memory allocator able to run in interrupt contexts and allocating from a special kernel address space map. As of .Fx 5.3 , the .Vt mbuf allocator is a wrapper around .Xr uma 9 , allowing caching of .Vt mbufs , clusters, and .Vt mbuf + cluster pairs in per-CPU caches, as well as bringing other benefits of slab allocation. .Sh AUTHORS The original .Nm manual page was written by .An Yar Tikhiy . The .Xr uma 9 .Vt mbuf allocator was written by .An Bosko Milekic . Index: projects/netbsd-tests-update-12/share/mk/bsd.dep.mk =================================================================== --- projects/netbsd-tests-update-12/share/mk/bsd.dep.mk (revision 305171) +++ projects/netbsd-tests-update-12/share/mk/bsd.dep.mk (revision 305172) @@ -1,314 +1,314 @@ # $FreeBSD$ # # The include file handles Makefile dependencies. # # # +++ variables +++ # # CLEANDEPENDDIRS Additional directories to remove for the cleandepend # target. # # CLEANDEPENDFILES Additional files to remove for the cleandepend target. # # CTAGS A tags file generation program [gtags] # # CTAGSFLAGS Options for ctags(1) [not set] # # DEPENDFILE dependencies file [.depend] # # GTAGSFLAGS Options for gtags(1) [-o] # # HTAGSFLAGS Options for htags(1) [not set] # # SRCS List of source files (c, c++, assembler) # # DPSRCS List of source files which are needed for generating # dependencies, ${SRCS} are always part of it. # # +++ targets +++ # # cleandepend: # remove ${CLEANDEPENDFILES}; remove ${CLEANDEPENDDIRS} and all # contents. # # depend: # Make the dependencies for the source files, and store # them in the file ${DEPENDFILE}. # # tags: # In "ctags" mode, create a tags file for the source files. # In "gtags" mode, create a (GLOBAL) gtags file for the # source files. If HTML is defined, htags(1) is also run # after gtags(1). .if !target(____) .error bsd.dep.mk cannot be included directly. .endif CTAGS?= gtags CTAGSFLAGS?= GTAGSFLAGS?= -o HTAGSFLAGS?= .if ${MK_DIRDEPS_BUILD} == "no" .MAKE.DEPENDFILE= ${DEPENDFILE} .endif CLEANDEPENDFILES+= ${DEPENDFILE} ${DEPENDFILE}.* .if ${MK_META_MODE} == "yes" CLEANDEPENDFILES+= *.meta .endif # Keep `tags' here, before SRCS are mangled below for `depend'. .if !target(tags) && defined(SRCS) && !defined(NO_TAGS) tags: ${SRCS} .if ${CTAGS:T} == "gtags" @cd ${.CURDIR} && ${CTAGS} ${GTAGSFLAGS} ${.OBJDIR} .if defined(HTML) @cd ${.CURDIR} && htags ${HTAGSFLAGS} -d ${.OBJDIR} ${.OBJDIR} .endif .else @${CTAGS} ${CTAGSFLAGS} -f /dev/stdout \ ${.ALLSRC:N*.h} | sed "s;${.CURDIR}/;;" > ${.TARGET} .endif .endif .if !empty(.MAKE.MODE:Mmeta) && empty(.MAKE.MODE:Mnofilemon) _meta_filemon= 1 .endif # Skip reading .depend when not needed to speed up tree-walks and simple # lookups. For install, only do this if no other targets are specified. # Also skip generating or including .depend.* files if in meta+filemon mode # since it will track dependencies itself. OBJS_DEPEND_GUESS is still used. .if !empty(.MAKEFLAGS:M-V${_V_READ_DEPEND}) || make(obj) || make(clean*) || \ ${.TARGETS:M*install*} == ${.TARGETS} || \ - make(analyze) || defined(_meta_filemon) + make(analyze) || defined(_meta_filemon) || make(print-dir) _SKIP_READ_DEPEND= 1 -.if ${MK_DIRDEPS_BUILD} == "no" +.if ${MK_DIRDEPS_BUILD} == "no" || make(analyze) || make(print-dir) .MAKE.DEPENDFILE= /dev/null .endif .endif .if defined(SRCS) CLEANFILES?= .for _S in ${SRCS:N*.[dhly]} OBJS_DEPEND_GUESS.${_S:R}.o+= ${_S} .endfor # Lexical analyzers .for _LSRC in ${SRCS:M*.l:N*/*} .for _LC in ${_LSRC:R}.c ${_LC}: ${_LSRC} ${LEX} ${LFLAGS} -o${.TARGET} ${.ALLSRC} OBJS_DEPEND_GUESS.${_LC:R}.o+= ${_LC} SRCS:= ${SRCS:S/${_LSRC}/${_LC}/} CLEANFILES+= ${_LC} .endfor .endfor # Yacc grammars .for _YSRC in ${SRCS:M*.y:N*/*} .for _YC in ${_YSRC:R}.c SRCS:= ${SRCS:S/${_YSRC}/${_YC}/} CLEANFILES+= ${_YC} .if !empty(YFLAGS:M-d) && !empty(SRCS:My.tab.h) .ORDER: ${_YC} y.tab.h y.tab.h: .NOMETA ${_YC} y.tab.h: ${_YSRC} ${YACC} ${YFLAGS} ${.ALLSRC} cp y.tab.c ${_YC} CLEANFILES+= y.tab.c y.tab.h .elif !empty(YFLAGS:M-d) .for _YH in ${_YC:R}.h .ORDER: ${_YC} ${_YH} ${_YH}: .NOMETA ${_YC} ${_YH}: ${_YSRC} ${YACC} ${YFLAGS} -o ${_YC} ${.ALLSRC} SRCS+= ${_YH} CLEANFILES+= ${_YH} .endfor .else ${_YC}: ${_YSRC} ${YACC} ${YFLAGS} -o ${_YC} ${.ALLSRC} .endif OBJS_DEPEND_GUESS.${_YC:R}.o+= ${_YC} .endfor .endfor # DTrace probe definitions .if ${SRCS:M*.d} CFLAGS+= -I${.OBJDIR} .endif .for _DSRC in ${SRCS:M*.d:N*/*} .for _D in ${_DSRC:R} SRCS+= ${_D}.h ${_D}.h: ${_DSRC} ${DTRACE} ${DTRACEFLAGS} -h -s ${.ALLSRC} SRCS:= ${SRCS:S/^${_DSRC}$//} OBJS+= ${_D}.o CLEANFILES+= ${_D}.h ${_D}.o ${_D}.o: ${_DSRC} ${OBJS:S/^${_D}.o$//} @rm -f ${.TARGET} ${DTRACE} ${DTRACEFLAGS} -G -o ${.TARGET} -s ${.ALLSRC:N*.h} .if defined(LIB) CLEANFILES+= ${_D}.So ${_D}.po ${_D}.So: ${_DSRC} ${SOBJS:S/^${_D}.So$//} @rm -f ${.TARGET} ${DTRACE} ${DTRACEFLAGS} -G -o ${.TARGET} -s ${.ALLSRC:N*.h} ${_D}.po: ${_DSRC} ${POBJS:S/^${_D}.po$//} @rm -f ${.TARGET} ${DTRACE} ${DTRACEFLAGS} -G -o ${.TARGET} -s ${.ALLSRC:N*.h} .endif .endfor .endfor .if ${MAKE_VERSION} < 20160220 DEPEND_MP?= -MP .endif # Handle OBJS=../somefile.o hacks. Just replace '/' rather than use :T to # avoid collisions. DEPEND_FILTER= C,/,_,g DEPENDSRCS= ${SRCS:M*.[cSC]} ${SRCS:M*.cxx} ${SRCS:M*.cpp} ${SRCS:M*.cc} .if !empty(DEPENDSRCS) DEPENDOBJS+= ${DEPENDSRCS:R:S,$,.o,} .endif DEPENDFILES_OBJS= ${DEPENDOBJS:O:u:${DEPEND_FILTER}:C/^/${DEPENDFILE}./} DEPEND_CFLAGS+= -MD ${DEPEND_MP} -MF${DEPENDFILE}.${.TARGET:${DEPEND_FILTER}} DEPEND_CFLAGS+= -MT${.TARGET} .if !defined(_meta_filemon) .if defined(.PARSEDIR) # Only add in DEPEND_CFLAGS for CFLAGS on files we expect from DEPENDOBJS # as those are the only ones we will include. DEPEND_CFLAGS_CONDITION= "${DEPENDOBJS:M${.TARGET:${DEPEND_FILTER}}}" != "" CFLAGS+= ${${DEPEND_CFLAGS_CONDITION}:?${DEPEND_CFLAGS}:} .else CFLAGS+= ${DEPEND_CFLAGS} .endif .if !defined(_SKIP_READ_DEPEND) .for __depend_obj in ${DEPENDFILES_OBJS} .if ${MAKE_VERSION} < 20160220 .sinclude "${.OBJDIR}/${__depend_obj}" .else .dinclude "${.OBJDIR}/${__depend_obj}" .endif .endfor .endif # !defined(_SKIP_READ_DEPEND) .endif # !defined(_meta_filemon) .endif # defined(SRCS) -.if ${MK_DIRDEPS_BUILD} == "yes" +.if ${MK_DIRDEPS_BUILD} == "yes" && !make(analyze) && !make(print-dir) # Prevent meta.autodep.mk from tracking "local dependencies". .depend: .include # If using filemon then _EXTRADEPEND is skipped since it is not needed. .if defined(_meta_filemon) # this depend: bypasses that below # the dependency helps when bootstrapping depend: beforedepend ${DPSRCS} ${SRCS} afterdepend beforedepend: afterdepend: beforedepend .endif .endif # Guess some dependencies for when no ${DEPENDFILE}.OBJ is generated yet. # For meta+filemon the .meta file is checked for since it is the dependency # file used. .for __obj in ${DEPENDOBJS:O:u} .if (defined(_meta_filemon) && !exists(${.OBJDIR}/${__obj}.meta)) || \ (!defined(_meta_filemon) && !exists(${.OBJDIR}/${DEPENDFILE}.${__obj})) ${__obj}: ${OBJS_DEPEND_GUESS} ${__obj}: ${OBJS_DEPEND_GUESS.${__obj}} .elif defined(_meta_filemon) # For meta mode we still need to know which file to depend on to avoid # ambiguous suffix transformation rules from .PATH. Meta mode does not # use .depend files. We really only need source files, not headers since # they are typically in SRCS/beforebuild already. For target-specific # guesses do include headers though since they may not be in SRCS. ${__obj}: ${OBJS_DEPEND_GUESS:N*.h} ${__obj}: ${OBJS_DEPEND_GUESS.${__obj}} .endif .endfor # Always run 'make depend' to generate dependencies early and to avoid the # need for manually running it. The dirdeps build should only do this in # sub-makes though since MAKELEVEL0 is for dirdeps calculations. .if ${MK_DIRDEPS_BUILD} == "no" || ${.MAKE.LEVEL} > 0 beforebuild: depend .endif .if !target(depend) .if defined(SRCS) depend: beforedepend ${DEPENDFILE} afterdepend # Tell bmake not to look for generated files via .PATH .NOPATH: ${DEPENDFILE} ${DEPENDFILES_OBJS} DPSRCS+= ${SRCS} # A .depend file will only be generated if there are commands in # beforedepend/_EXTRADEPEND/afterdepend The _EXTRADEPEND target is # ignored if using meta+filemon since it handles all dependencies. The other # targets are kept as they be used for generating something. The target is # kept to allow 'make depend' to generate files. ${DEPENDFILE}: ${DPSRCS} .if exists(${.OBJDIR}/${DEPENDFILE}) || \ ((commands(beforedepend) || \ (!defined(_meta_filemon) && commands(_EXTRADEPEND)) || \ commands(afterdepend)) && !empty(.MAKE.MODE:Mmeta)) rm -f ${DEPENDFILE} .endif .if !defined(_meta_filemon) && target(_EXTRADEPEND) _EXTRADEPEND: .USE ${DEPENDFILE}: _EXTRADEPEND .endif .ORDER: ${DEPENDFILE} afterdepend .else depend: beforedepend afterdepend .endif .if !target(beforedepend) beforedepend: .else .ORDER: beforedepend ${DEPENDFILE} .ORDER: beforedepend afterdepend .endif .if !target(afterdepend) afterdepend: .endif .endif .if defined(SRCS) .if ${CTAGS:T} == "gtags" CLEANDEPENDFILES+= GPATH GRTAGS GSYMS GTAGS .if defined(HTML) CLEANDEPENDDIRS+= HTML .endif .else CLEANDEPENDFILES+= tags .endif .endif .if !target(cleandepend) cleandepend: .if !empty(CLEANDEPENDFILES) rm -f ${CLEANDEPENDFILES} .endif .if !empty(CLEANDEPENDDIRS) rm -rf ${CLEANDEPENDDIRS} .endif .endif .ORDER: cleandepend all .ORDER: cleandepend depend .if !target(checkdpadd) && (defined(DPADD) || defined(LDADD)) _LDADD_FROM_DPADD= ${DPADD:R:T:C;^lib(.*)$;-l\1;g} # Ignore -Wl,--start-group/-Wl,--end-group as it might be required in the # LDADD list due to unresolved symbols _LDADD_CANONICALIZED= ${LDADD:N:R:T:C;^lib(.*)$;-l\1;g:N-Wl,--[es]*-group} checkdpadd: .if ${_LDADD_FROM_DPADD} != ${_LDADD_CANONICALIZED} @echo ${.CURDIR} @echo "DPADD -> ${_LDADD_FROM_DPADD}" @echo "LDADD -> ${_LDADD_CANONICALIZED}" .endif .endif Index: projects/netbsd-tests-update-12/share/mk/bsd.init.mk =================================================================== --- projects/netbsd-tests-update-12/share/mk/bsd.init.mk (revision 305171) +++ projects/netbsd-tests-update-12/share/mk/bsd.init.mk (revision 305172) @@ -1,41 +1,41 @@ # $FreeBSD$ # The include file includes , # ../Makefile.inc and ; this is used at the # top of all files that actually "build something". # bsd.opts.mk is included early so Makefile.inc can use the # MK_FOO variables. .if !target(____) ____: .include .-include "local.init.mk" .if exists(${.CURDIR}/../Makefile.inc) .include "${.CURDIR}/../Makefile.inc" .endif .include .MAIN: all beforebuild: .PHONY .NOTMAIN .if !defined(_SKIP_BUILD) all: beforebuild .WAIT .endif .if ${.MAKE.LEVEL:U1} == 0 && ${BUILD_AT_LEVEL0:Uyes:tl} == "no" && !make(clean*) # this tells lib.mk and prog.mk to not actually build anything _SKIP_BUILD = not building at level 0 .endif .if ${.MAKE.LEVEL} > 0 && !empty(_SKIP_BUILD) .warning ${_SKIP_BUILD} .endif .if ${MK_META_MODE} == "yes" .if !exists(/dev/filemon) && \ ${UPDATE_DEPENDFILE:Uyes:tl} != "no" && !defined(NO_FILEMON) && \ - !make(showconfig) && ${.MAKEFLAGS:M-V} == "" + !make(showconfig) && !make(print-dir) && ${.MAKEFLAGS:M-V} == "" .warning The filemon module (/dev/filemon) is not loaded. .warning META_MODE is less useful for incremental builds without filemon. .warning 'kldload filemon' or pass -DNO_FILEMON to suppress this warning. .endif .endif # ${MK_META_MODE} == "yes" .endif # !target(____) Index: projects/netbsd-tests-update-12/share/mk/bsd.obj.mk =================================================================== --- projects/netbsd-tests-update-12/share/mk/bsd.obj.mk (revision 305171) +++ projects/netbsd-tests-update-12/share/mk/bsd.obj.mk (revision 305172) @@ -1,212 +1,236 @@ # $FreeBSD$ # # The include file handles creating the 'obj' directory # and cleaning up object files, etc. # # +++ variables +++ # # CLEANDIRS Additional directories to remove for the clean target. # # CLEANFILES Additional files to remove for the clean target. # # MAKEOBJDIR A pathname for the directory where the targets # are built. Note: MAKEOBJDIR is an *environment* variable # and works properly only if set as an environment variable, # not as a global or command line variable! # # E.g. use `env MAKEOBJDIR=temp-obj make' # # MAKEOBJDIRPREFIX Specifies somewhere other than /usr/obj to root the object # tree. Note: MAKEOBJDIRPREFIX is an *environment* variable # and works properly only if set as an environment variable, # not as a global or command line variable! # # E.g. use `env MAKEOBJDIRPREFIX=/somewhere/obj make' # # NO_OBJ Do not create object directories. This should not be set # if anything is built. # # +++ targets +++ # # clean: # remove ${CLEANFILES}; remove ${CLEANDIRS} and all contents. # # cleandir: # remove the build directory (and all its contents) created by obj # # obj: # create build directory. # .if !target(____) ____: .include .if ${MK_AUTO_OBJ} == "yes" # it is done by now objwarn: obj: CANONICALOBJDIR= ${.OBJDIR} .if defined(NO_OBJ) # but this makefile does not want it! .OBJDIR: ${.CURDIR} .endif +# Handle special case where SRCS is full-pathed and requires +# nested objdirs. This duplicates some auto.obj.mk logic. +.if (!empty(SRCS:M*/*) || !empty(DPSRCS:M*/*)) && \ + (${.TARGETS} == "" || ${.TARGETS:Nclean*:N*clean:Ndestroy*} != "") +_wantdirs= ${SRCS:M*/*:H} ${DPSRCS:M*/*:H} +.if !empty(_wantdirs) +_wantdirs:= ${_wantdirs:O:u} +_needdirs= +.for _dir in ${_wantdirs} +.if !exists(${.OBJDIR}/${_dir}/) +_needdirs+= ${_dir} +.endif +.endfor +.endif +.if !empty(_needdirs) +#_mkneededdirs!= umask ${OBJDIR_UMASK:U002}; ${Mkdirs} ${_needdirs} +__objdir_made != umask ${OBJDIR_UMASK:U002}; ${Mkdirs}; \ + for dir in ${_needdirs}; do \ + dir=${.OBJDIR}/$${dir}; \ + ${ECHO_TRACE} "[Creating nested objdir $${dir}...]" >&2; \ + Mkdirs $${dir}; \ + done +.endif +.endif # !empty(SRCS:M*/*) || !empty(DPSRCS:M*/*) .elif defined(MAKEOBJDIRPREFIX) CANONICALOBJDIR:=${MAKEOBJDIRPREFIX}${.CURDIR} .elif defined(MAKEOBJDIR) && ${MAKEOBJDIR:M/*} != "" CANONICALOBJDIR:=${MAKEOBJDIR} OBJTOP?= ${MAKEOBJDIR} .else CANONICALOBJDIR:=/usr/obj${.CURDIR} .endif OBJTOP?= ${.OBJDIR:S,${.CURDIR},,}${SRCTOP} # # Warn of unorthodox object directory. # # The following directories are tried in order for ${.OBJDIR}: # # 1. ${MAKEOBJDIRPREFIX}/`pwd` # 2. ${MAKEOBJDIR} # 3. obj.${MACHINE} # 4. obj # 5. /usr/obj/`pwd` # 6. ${.CURDIR} # # If ${.OBJDIR} is constructed using canonical cases 1 or 5, or # case 2 (using MAKEOBJDIR), don't issue a warning. Otherwise, # issue a warning differentiating between cases 6 and (3 or 4). # objwarn: .if !defined(NO_OBJ) && ${.OBJDIR} != ${CANONICALOBJDIR} && \ !(defined(MAKEOBJDIRPREFIX) && exists(${CANONICALOBJDIR}/)) && \ !(defined(MAKEOBJDIR) && exists(${MAKEOBJDIR}/)) .if ${.OBJDIR} == ${.CURDIR} @${ECHO} "Warning: Object directory not changed from original ${.CURDIR}" .elif exists(${.CURDIR}/obj.${MACHINE}/) || exists(${.CURDIR}/obj/) @${ECHO} "Warning: Using ${.OBJDIR} as object directory instead of\ canonical ${CANONICALOBJDIR}" .endif .endif beforebuild: objwarn .if !defined(NO_OBJ) .if !target(obj) obj: .PHONY @if ! test -d ${CANONICALOBJDIR}/; then \ mkdir -p ${CANONICALOBJDIR}; \ if ! test -d ${CANONICALOBJDIR}/; then \ ${ECHO} "Unable to create ${CANONICALOBJDIR}."; \ exit 1; \ fi; \ ${ECHO} "${CANONICALOBJDIR} created for ${.CURDIR}"; \ fi .for dir in ${SRCS:H:O:u} ${DPSRCS:H:O:u} @if ! test -d ${CANONICALOBJDIR}/${dir}/; then \ mkdir -p ${CANONICALOBJDIR}/${dir}; \ if ! test -d ${CANONICALOBJDIR}/${dir}/; then \ ${ECHO} "Unable to create ${CANONICALOBJDIR}/${dir}."; \ exit 1; \ fi; \ ${ECHO} "${CANONICALOBJDIR}/${dir} created for ${.CURDIR}"; \ fi .endfor .endif .if !target(objlink) objlink: @if test -d ${CANONICALOBJDIR}/; then \ rm -f ${.CURDIR}/obj; \ ln -s ${CANONICALOBJDIR} ${.CURDIR}/obj; \ else \ echo "No ${CANONICALOBJDIR} to link to - do a make obj."; \ fi .endif .endif # !defined(NO_OBJ) # # where would that obj directory be? # .if !target(whereobj) whereobj: @echo ${.OBJDIR} .endif .if ${CANONICALOBJDIR} != ${.CURDIR} && exists(${CANONICALOBJDIR}/) cleanobj: @-rm -rf ${CANONICALOBJDIR} .else cleanobj: clean cleandepend .endif @if [ -L ${.CURDIR}/obj ]; then rm -f ${.CURDIR}/obj; fi # Tell bmake not to look for generated files via .PATH NOPATH_FILES+= ${CLEANFILES} .if !empty(NOPATH_FILES) .NOPATH: ${NOPATH_FILES} .endif .if !target(clean) clean: .if defined(CLEANFILES) && !empty(CLEANFILES) rm -f ${CLEANFILES} .endif .if defined(CLEANDIRS) && !empty(CLEANDIRS) -rm -rf ${CLEANDIRS} .endif .endif .ORDER: clean all cleandir: cleanobj .include .if make(destroy*) && defined(OBJROOT) # this (rm -rf objdir) is much faster and more reliable than cleaning. # just in case we are playing games with these... _OBJDIR?= ${.OBJDIR} _CURDIR?= ${.CURDIR} # destroy almost everything destroy: destroy-all destroy-all: # just remove our objdir destroy-arch: .NOMETA .if ${_OBJDIR} != ${_CURDIR} cd ${_CURDIR} && rm -rf ${_OBJDIR} .endif .if defined(HOST_OBJTOP) destroy-host: destroy.host destroy.host: .NOMETA cd ${_CURDIR} && rm -rf ${HOST_OBJTOP}/${RELDIR:N.} .endif .if make(destroy-all) && ${RELDIR} == "." destroy-all: destroy-stage .endif # remove the stage tree destroy-stage: .NOMETA .if defined(STAGE_ROOT) cd ${_CURDIR} && rm -rf ${STAGE_ROOT} .endif # allow parallel destruction _destroy_machine_list = common host ${ALL_MACHINE_LIST} .for m in ${_destroy_machine_list:O:u} destroy-all: destroy.$m .if !target(destroy.$m) destroy.$m: .NOMETA .if ${_OBJDIR} != ${_CURDIR} cd ${_CURDIR} && rm -rf ${OBJROOT}$m*/${RELDIR:N.} .endif .endif .endfor .endif .endif # !target(____) Index: projects/netbsd-tests-update-12/share/mk/bsd.subdir.mk =================================================================== --- projects/netbsd-tests-update-12/share/mk/bsd.subdir.mk (revision 305171) +++ projects/netbsd-tests-update-12/share/mk/bsd.subdir.mk (revision 305172) @@ -1,183 +1,193 @@ # from: @(#)bsd.subdir.mk 5.9 (Berkeley) 2/1/91 # $FreeBSD$ # # The include file contains the default targets # for building subdirectories. # # For all of the directories listed in the variable SUBDIRS, the # specified directory will be visited and the target made. There is # also a default target which allows the command "make subdir" where # subdir is any directory listed in the variable SUBDIRS. # # # +++ variables +++ # # DISTRIBUTION Name of distribution. [base] # # SUBDIR A list of subdirectories that should be built as well. # Each of the targets will execute the same target in the # subdirectories. SUBDIR.yes is automatically appended # to this list. # # +++ targets +++ # # distribute: # This is a variant of install, which will # put the stuff into the right "distribution". # # See SUBDIR_TARGETS for list of targets that will recurse. # # Targets defined in STANDALONE_SUBDIR_TARGETS will always be ran # with SUBDIR_PARALLEL and will not respect .WAIT or SUBDIR_DEPEND_ # values. # # SUBDIR_TARGETS and STANDALONE_SUBDIR_TARGETS can be appended to # via make.conf or src.conf. # .if !target(____) ____: SUBDIR_TARGETS+= \ all all-man analyze buildconfig buildfiles buildincludes \ checkdpadd clean cleandepend cleandir cleanilinks \ cleanobj depend distribute files includes installconfig \ - installfiles installincludes realinstall lint maninstall \ - manlint obj objlink tags \ + installfiles installincludes print-dir realinstall lint \ + maninstall manlint obj objlink tags \ # Described above. STANDALONE_SUBDIR_TARGETS+= \ all-man buildconfig buildfiles buildincludes check checkdpadd \ clean cleandepend cleandir cleanilinks cleanobj files includes \ - installconfig installincludes installfiles maninstall manlint \ - obj objlink \ + installconfig installincludes installfiles print-dir \ + maninstall manlint obj objlink # It is safe to install in parallel when staging. .if defined(NO_ROOT) STANDALONE_SUBDIR_TARGETS+= realinstall .endif .include + +.if make(print-dir) +NEED_SUBDIR= 1 +ECHODIR= : +.SILENT: +.if ${RELDIR:U.} != "." +print-dir: .PHONY + @echo ${RELDIR} +.endif +.endif .if !defined(NEED_SUBDIR) .if ${.MAKE.LEVEL} == 0 && ${MK_DIRDEPS_BUILD} == "yes" && !empty(SUBDIR) && !(make(clean*) || make(destroy*)) .include # ignore this _SUBDIR: .endif .endif DISTRIBUTION?= base .if !target(distribute) distribute: .MAKE .for dist in ${DISTRIBUTION} ${_+_}cd ${.CURDIR}; \ ${MAKE} install installconfig -DNO_SUBDIR DESTDIR=${DISTDIR}/${dist} SHARED=copies .endfor .endif # Convenience targets to run 'build${target}' and 'install${target}' when # calling 'make ${target}'. .for __target in files includes .if !target(${__target}) ${__target}: build${__target} install${__target} .ORDER: build${__target} install${__target} .endif .endfor # Make 'install' supports a before and after target. Actual install # hooks are placed in 'realinstall'. .if !target(install) .for __stage in before real after .if !target(${__stage}install) ${__stage}install: .endif .endfor install: beforeinstall realinstall afterinstall .ORDER: beforeinstall realinstall afterinstall .endif .ORDER: all install # SUBDIR recursing may be disabled for MK_DIRDEPS_BUILD .if !target(_SUBDIR) .if defined(SUBDIR) SUBDIR:=${SUBDIR} ${SUBDIR.yes} SUBDIR:=${SUBDIR:u} .endif # Subdir code shared among 'make ', 'make ' and SUBDIR_PARALLEL. _SUBDIR_SH= \ if test -d ${.CURDIR}/$${dir}.${MACHINE_ARCH}; then \ dir=$${dir}.${MACHINE_ARCH}; \ fi; \ ${ECHODIR} "===> ${DIRPRFX}$${dir} ($${target})"; \ cd ${.CURDIR}/$${dir}; \ ${MAKE} $${target} DIRPRFX=${DIRPRFX}$${dir}/ # This is kept for compatibility only. The normal handling of attaching to # SUBDIR_TARGETS will create a target for each directory. _SUBDIR: .USEBEFORE .if defined(SUBDIR) && !empty(SUBDIR) && !defined(NO_SUBDIR) @${_+_}target=${.TARGET:realinstall=install}; \ for dir in ${SUBDIR:N.WAIT}; do ( ${_SUBDIR_SH} ); done .endif # Create 'make subdir' targets to run the real 'all' target. .for __dir in ${SUBDIR:N.WAIT} ${__dir}: all_subdir_${DIRPRFX}${__dir} .PHONY .endfor .for __target in ${SUBDIR_TARGETS} # Can ordering be skipped for this and SUBDIR_PARALLEL forced? .if ${STANDALONE_SUBDIR_TARGETS:M${__target}} _is_standalone_target= 1 SUBDIR:= ${SUBDIR:N.WAIT} .else _is_standalone_target= 0 .endif __subdir_targets= .for __dir in ${SUBDIR} .if ${__dir} == .WAIT __subdir_targets+= .WAIT .else __deps= .if ${_is_standalone_target} == 0 .if defined(SUBDIR_PARALLEL) # Apply SUBDIR_DEPEND dependencies for SUBDIR_PARALLEL. .for __dep in ${SUBDIR_DEPEND_${__dir}} __deps+= ${__target}_subdir_${DIRPRFX}${__dep} .endfor .else # For non-parallel builds, directories depend on all targets before them. __deps:= ${__subdir_targets} .endif # defined(SUBDIR_PARALLEL) .endif # ${_is_standalone_target} == 0 ${__target}_subdir_${DIRPRFX}${__dir}: .PHONY .MAKE .SILENT ${__deps} @${_+_}target=${__target:realinstall=install}; \ dir=${__dir}; \ ${_SUBDIR_SH}; __subdir_targets+= ${__target}_subdir_${DIRPRFX}${__dir} .endif # ${__dir} == .WAIT .endfor # __dir in ${SUBDIR} # Attach the subdir targets to the real target. # Only recurse on directly-called targets. I.e., don't recurse on dependencies # such as 'install' becoming {before,real,after}install, just recurse # 'install'. Despite that, 'realinstall' is special due to ordering issues # with 'afterinstall'. .if !defined(NO_SUBDIR) && (make(${__target}) || \ (${__target} == realinstall && make(install))) ${__target}: ${__subdir_targets} .PHONY .endif # make(${__target}) .endfor # __target in ${SUBDIR_TARGETS} .endif # !target(_SUBDIR) # Ensure all targets exist .for __target in ${SUBDIR_TARGETS} .if !target(${__target}) ${__target}: .endif .endfor .endif Index: projects/netbsd-tests-update-12/share/mk/local.meta.sys.mk =================================================================== --- projects/netbsd-tests-update-12/share/mk/local.meta.sys.mk (revision 305171) +++ projects/netbsd-tests-update-12/share/mk/local.meta.sys.mk (revision 305172) @@ -1,285 +1,286 @@ # $FreeBSD$ # local configuration specific to meta mode # XXX some of this should be in meta.sys.mk # we assume that MK_DIRDEPS_BUILD=yes # we need this until there is an alternative MK_INSTALL_AS_USER= yes _default_makeobjdir=$${.CURDIR:S,^$${SRCTOP},$${OBJTOP},} .if empty(OBJROOT) || ${.MAKE.LEVEL} == 0 .if defined(MAKEOBJDIRPREFIX) && !empty(MAKEOBJDIRPREFIX) # put things approximately where they want OBJROOT:=${MAKEOBJDIRPREFIX}${SRCTOP}/ MAKEOBJDIRPREFIX= .export MAKEOBJDIRPREFIX .endif .if empty(MAKEOBJDIR) # OBJTOP set below MAKEOBJDIR=${_default_makeobjdir} # export but do not track .export-env MAKEOBJDIR # Expand for our own use MAKEOBJDIR:= ${MAKEOBJDIR} .endif .if !empty(SB) SB_OBJROOT ?= ${SB}/obj/ # this is what we use below OBJROOT ?= ${SB_OBJROOT} .endif OBJROOT ?= /usr/obj${SRCTOP}/ .if ${OBJROOT:M*/} != "" OBJROOT:= ${OBJROOT:H:tA}/ .else OBJROOT:= ${OBJROOT:H:tA}/${OBJROOT:T} .endif .export OBJROOT SRCTOP # we need HOST_TARGET etc below. .include .export HOST_TARGET .endif # from src/Makefile (for universe) TARGET_ARCHES_arm?= arm armeb armv6 TARGET_ARCHES_arm64?= aarch64 TARGET_ARCHES_mips?= mipsel mips mips64el mips64 mipsn32 mipsn32el TARGET_ARCHES_powerpc?= powerpc powerpc64 TARGET_ARCHES_pc98?= i386 TARGET_ARCHES_riscv?= riscv64 # some corner cases BOOT_MACHINE_DIR.amd64 = boot/i386 MACHINE_ARCH.host = ${_HOST_ARCH} # the list of machines we support ALL_MACHINE_LIST?= amd64 arm arm64 i386 mips pc98 powerpc riscv sparc64 .for m in ${ALL_MACHINE_LIST:O:u} MACHINE_ARCH_LIST.$m?= ${TARGET_ARCHES_${m}:U$m} MACHINE_ARCH.$m?= ${MACHINE_ARCH_LIST.$m:[1]} BOOT_MACHINE_DIR.$m ?= boot/$m .endfor .ifndef _TARGET_SPEC .if empty(MACHINE_ARCH) .if !empty(TARGET_ARCH) MACHINE_ARCH= ${TARGET_ARCH} .else MACHINE_ARCH= ${MACHINE_ARCH.${MACHINE}} .endif .endif MACHINE_ARCH?= ${MACHINE_ARCH.${MACHINE}} MACHINE_ARCH:= ${MACHINE_ARCH} .else # we got here via dirdeps MACHINE_ARCH:= ${MACHINE_ARCH.${MACHINE}} .endif # now because for universe we want to potentially # build for multiple MACHINE_ARCH per MACHINE # we need more than MACHINE in TARGET_SPEC TARGET_SPEC_VARS= MACHINE MACHINE_ARCH # see dirdeps.mk .if ${TARGET_SPEC:Uno:M*,*} != "" _tspec := ${TARGET_SPEC:S/,/ /g} MACHINE := ${_tspec:[1]} MACHINE_ARCH := ${_tspec:[2]} # etc. # We need to stop that TARGET_SPEC affecting any submakes # and deal with MACHINE=${TARGET_SPEC} in the environment. TARGET_SPEC= # export but do not track .export-env TARGET_SPEC .export ${TARGET_SPEC_VARS} .for v in ${TARGET_SPEC_VARS:O:u} .if empty($v) .undef $v .endif .endfor .endif # make sure we know what TARGET_SPEC is # as we may need it to find Makefile.depend* TARGET_SPEC = ${TARGET_SPEC_VARS:@v@${$v:U}@:ts,} # to be consistent with src/Makefile just concatenate with '.'s TARGET_OBJ_SPEC:= ${TARGET_SPEC:S;,;.;g} OBJTOP:= ${OBJROOT}${TARGET_OBJ_SPEC} .if defined(MAKEOBJDIR) .if ${MAKEOBJDIR:M*/*} == "" .error Cannot use MAKEOBJDIR=${MAKEOBJDIR}${.newline}Unset MAKEOBJDIR to get default: MAKEOBJDIR='${_default_makeobjdir}' .endif .endif HOST_OBJTOP ?= ${OBJROOT}${HOST_TARGET} .if ${OBJTOP} == ${HOST_OBJTOP} || ${REQUESTED_MACHINE:U${MACHINE}} == "host" MACHINE= host .if ${TARGET_MACHINE:Uno} == ${HOST_TARGET} # not what we want TARGET_MACHINE= host .endif .endif .if ${MACHINE} == "host" OBJTOP := ${HOST_OBJTOP} .endif .if ${.MAKE.LEVEL} == 0 PYTHON ?= /usr/local/bin/python .export PYTHON # this works best if share/mk is ready for it. BUILD_AT_LEVEL0= no # _SKIP_BUILD is not 100% as it requires wrapping all 'all:' targets to avoid # building in MAKELEVEL0. Just prohibit 'all' entirely in this case to avoid # problems. .if ${MK_DIRDEPS_BUILD} == "yes" && \ ${.MAKE.LEVEL} == 0 && ${BUILD_AT_LEVEL0:Uyes:tl} == "no" .MAIN: dirdeps .if make(all) .error DIRDEPS_BUILD: Please run '${MAKE}' instead of '${MAKE} all'. .endif .endif # we want to end up with a singe stage tree for all machines .if ${MK_STAGING} == "yes" .if empty(STAGE_ROOT) STAGE_ROOT?= ${OBJROOT}stage .export STAGE_ROOT .endif .endif .endif .if ${MK_STAGING} == "yes" .if ${MACHINE} == "host" STAGE_MACHINE= ${HOST_TARGET} .else STAGE_MACHINE:= ${TARGET_OBJ_SPEC} .endif STAGE_OBJTOP:= ${STAGE_ROOT}/${STAGE_MACHINE} STAGE_COMMON_OBJTOP:= ${STAGE_ROOT}/common STAGE_TARGET_OBJTOP:= ${STAGE_ROOT}/${TARGET_OBJ_SPEC} STAGE_HOST_OBJTOP:= ${STAGE_ROOT}/${HOST_TARGET} # These are exported for hooking in out-of-tree builds. They will always # be overridden in sub-makes above when building in-tree. .export STAGE_OBJTOP STAGE_TARGET_OBJTOP STAGE_HOST_OBJTOP # Use tools/install.sh which can avoid the need for xinstall for simple cases. INSTALL?= sh ${SRCTOP}/tools/install.sh # This is for stage-install to pickup from the environment. REAL_INSTALL:= ${INSTALL} .export REAL_INSTALL STAGE_INSTALL= sh ${.PARSEDIR:tA}/stage-install.sh OBJDIR=${.OBJDIR:tA} STAGE_LIBDIR= ${STAGE_OBJTOP}${_LIBDIR:U${LIBDIR:U/lib}} STAGE_INCLUDEDIR= ${STAGE_OBJTOP}${INCLUDEDIR:U/usr/include} # this is not the same as INCLUDEDIR STAGE_INCSDIR= ${STAGE_OBJTOP}${INCSDIR:U/include} # the target is usually an absolute path STAGE_SYMLINKS_DIR= ${STAGE_OBJTOP} LDFLAGS_LAST+= -Wl,-rpath-link,${STAGE_LIBDIR} .if ${MK_SYSROOT} == "yes" SYSROOT?= ${STAGE_OBJTOP} .else LDFLAGS_LAST+= -L${STAGE_LIBDIR} .endif .endif # MK_STAGING # this is sufficient for most of the tree. .MAKE.DEPENDFILE_DEFAULT = ${.MAKE.DEPENDFILE_PREFIX} # but if we have a machine qualified file it should be used in preference .MAKE.DEPENDFILE_PREFERENCE = \ ${.MAKE.DEPENDFILE_PREFIX}.${MACHINE} \ ${.MAKE.DEPENDFILE_PREFIX} .undef .MAKE.DEPENDFILE .include "sys.dependfile.mk" .if ${.MAKE.LEVEL} > 0 && ${MACHINE} == "host" && ${.MAKE.DEPENDFILE:E} != "host" # we can use this but should not update it. UPDATE_DEPENDFILE= NO .endif # define the list of places that contain files we are responsible for .MAKE.META.BAILIWICK = ${SB} ${OBJROOT} ${STAGE_ROOT} CSU_DIR.${MACHINE_ARCH} ?= csu/${MACHINE_ARCH} CSU_DIR := ${CSU_DIR.${MACHINE_ARCH}} .if !empty(TIME_STAMP) TRACER= ${TIME_STAMP} ${:U} .endif -.if !defined(_RECURSING_PROGS) && !defined(_RECURSING_CRUNCH) +.if !defined(_RECURSING_PROGS) && !defined(_RECURSING_CRUNCH) && \ + !make(print-dir) WITH_META_STATS= t .endif # toolchains can be a pain - especially bootstrappping them .if ${MACHINE} == "host" MK_SHARED_TOOLCHAIN= no .endif TOOLCHAIN_VARS= AS AR CC CLANG_TBLGEN CXX CPP LD NM OBJCOPY RANLIB \ STRINGS SIZE LLVM_TBLGEN _toolchain_bin_CLANG_TBLGEN= /usr/bin/clang-tblgen _toolchain_bin_LLVM_TBLGEN= /usr/bin/llvm-tblgen _toolchain_bin_CXX= /usr/bin/c++ .ifdef WITH_TOOLSDIR TOOLSDIR?= ${HOST_OBJTOP}/tools .elif defined(STAGE_HOST_OBJTOP) TOOLSDIR?= ${STAGE_HOST_OBJTOP} .endif # Only define if it exists in case user didn't run bootstrap-tools. Otherwise # the tool will be built during the build. Building it assumes it is # TARGET==MACHINE. .if exists(${HOST_OBJTOP}/tools${.CURDIR}) BTOOLSPATH= ${HOST_OBJTOP}/tools${.CURDIR} .endif # Don't use the bootstrap tools logic on itself. .if ${.TARGETS:Mbootstrap-tools} == "" && \ !make(showconfig) && \ !defined(BOOTSTRAPPING_TOOLS) && !empty(TOOLSDIR) && ${.MAKE.LEVEL} == 0 .for dir in /sbin /bin /usr/sbin /usr/bin PATH:= ${TOOLSDIR}${dir}:${PATH} .endfor .export PATH # Prefer the TOOLSDIR version of the toolchain if present vs the host version. .for var in ${TOOLCHAIN_VARS} _toolchain_bin.${var}= ${TOOLSDIR}${_toolchain_bin_${var}:U/usr/bin/${var:tl}} .if exists(${_toolchain_bin.${var}}) HOST_${var}?= ${_toolchain_bin.${var}} .export HOST_${var} .endif .endfor .endif .for var in ${TOOLCHAIN_VARS} HOST_${var}?= ${_toolchain_bin_${var}:U/usr/bin/${var:tl}} .endfor .if ${MACHINE} == "host" .for var in ${TOOLCHAIN_VARS} ${var}= ${HOST_${var}} .endfor .endif .if ${MACHINE:Nhost:Ncommon} != "" && ${MACHINE} != ${HOST_MACHINE} # cross-building .if !defined(FREEBSD_REVISION) FREEBSD_REVISION!= sed -n '/^REVISION=/{s,.*=,,;s,",,g;p; }' ${SRCTOP}/sys/conf/newvers.sh .export FREEBSD_REVISION .endif CROSS_TARGET_FLAGS= -target ${MACHINE_ARCH}-unknown-freebsd${FREEBSD_REVISION} CFLAGS+= ${CROSS_TARGET_FLAGS} ACFLAGS+= ${CROSS_TARGET_FLAGS} LDFLAGS+= -Wl,-m -Wl,elf_${MACHINE_ARCH}_fbsd .endif META_MODE+= missing-meta=yes .if empty(META_MODE:Mnofilemon) META_MODE+= missing-filemon=yes .endif Index: projects/netbsd-tests-update-12/share/mk/sys.mk =================================================================== --- projects/netbsd-tests-update-12/share/mk/sys.mk (revision 305171) +++ projects/netbsd-tests-update-12/share/mk/sys.mk (revision 305172) @@ -1,470 +1,470 @@ # from: @(#)sys.mk 8.2 (Berkeley) 3/21/94 # $FreeBSD$ unix ?= We run FreeBSD, not UNIX. .FreeBSD ?= true .if !defined(%POSIX) # # MACHINE_CPUARCH defines a collection of MACHINE_ARCH. Machines with # the same MACHINE_ARCH can run each other's binaries, so it necessarily # has word size and endian swizzled in. However, support files for # these machines often are shared amongst all combinations of size # and/or endian. This is called MACHINE_CPU in NetBSD, but that's used # for something different in FreeBSD. # MACHINE_CPUARCH=${MACHINE_ARCH:C/mips(n32|64)?(el)?/mips/:C/arm(v6)?(eb|hf)?/arm/:C/powerpc64/powerpc/:C/riscv64/riscv/} .endif # Some options we need now __DEFAULT_NO_OPTIONS= \ DIRDEPS_BUILD \ DIRDEPS_CACHE __DEFAULT_DEPENDENT_OPTIONS= \ AUTO_OBJ/DIRDEPS_BUILD \ META_MODE/DIRDEPS_BUILD \ STAGING/DIRDEPS_BUILD \ SYSROOT/DIRDEPS_BUILD __ENV_ONLY_OPTIONS:= \ ${__DEFAULT_NO_OPTIONS} \ ${__DEFAULT_YES_OPTIONS} \ ${__DEFAULT_DEPENDENT_OPTIONS:H} # early include for customization # see local.sys.mk below # Not included when building in fmake compatibility mode (still needed # for older system support) .if defined(.PARSEDIR) .sinclude .include # Disable MK_META_MODE with make -B .if ${MK_META_MODE} == "yes" && defined(.MAKEFLAGS) && ${.MAKEFLAGS:M-B} MK_META_MODE= no .endif .if ${MK_DIRDEPS_BUILD} == "yes" .sinclude .elif ${MK_META_MODE} == "yes" # verbose will show .MAKE.META.PREFIX for each target. META_MODE+= meta verbose .if !defined(NO_META_MISSING) META_MODE+= missing-meta=yes .endif # silent will hide command output if a .meta file is created. .if !defined(NO_SILENT) META_MODE+= silent=yes .endif .if !exists(/dev/filemon) META_MODE+= nofilemon .endif # Require filemon data with bmake .if empty(META_MODE:Mnofilemon) META_MODE+= missing-filemon=yes .endif .endif META_MODE?= normal .export META_MODE .MAKE.MODE?= ${META_MODE} .if !empty(.MAKE.MODE:Mmeta) && !defined(NO_META_IGNORE_HOST) # Ignore host file changes that will otherwise cause # buildworld -> installworld -> buildworld to rebuild everything. # Since the build is self-reliant and bootstraps everything it needs, # this should not be a real problem for incremental builds. # XXX: This relies on the existing host tools retaining ABI compatibility # through upgrades since they won't be rebuilt on header/library changes. # Note that these are prefix matching, so /lib matches /libexec. .MAKE.META.IGNORE_PATHS+= \ ${__MAKE_SHELL} \ /bin \ /lib \ /rescue \ /sbin \ /usr/bin \ /usr/include \ /usr/lib \ /usr/sbin \ /usr/share \ .endif .if ${MK_AUTO_OBJ} == "yes" # This needs to be done early - before .PATH is computed # Don't do this for 'make showconfig' as it enables all options where meta mode # is not expected. -.if !make(showconfig) +.if !make(showconfig) && !make(print-dir) .sinclude .endif .endif .else # bmake .include .endif # If the special target .POSIX appears (without prerequisites or # commands) before the first noncomment line in the makefile, make shall # process the makefile as specified by the Posix 1003.2 specification. # make(1) sets the special macro %POSIX in this case (to the actual # value "1003.2", for what it's worth). # # The rules below use this macro to distinguish between Posix-compliant # and default behaviour. # # This functionality is currently broken, since make(1) processes sys.mk # before reading any other files, and consequently has no opportunity to # set the %POSIX macro before we read this point. .if defined(%POSIX) .SUFFIXES: .o .c .y .l .a .sh .f .else .SUFFIXES: .out .a .ln .o .c .cc .cpp .cxx .C .m .F .f .e .r .y .l .S .asm .s .cl .p .h .sh .endif AR ?= ar .if defined(%POSIX) ARFLAGS ?= -rv .else ARFLAGS ?= -crD .endif RANLIB ?= ranlib .if !defined(%POSIX) RANLIBFLAGS ?= -D .endif AS ?= as AFLAGS ?= ACFLAGS ?= .if defined(%POSIX) CC ?= c89 CFLAGS ?= -O .else CC ?= cc .if ${MACHINE_CPUARCH} == "arm" || ${MACHINE_CPUARCH} == "mips" CFLAGS ?= -O -pipe .else CFLAGS ?= -O2 -pipe .endif .if defined(NO_STRICT_ALIASING) CFLAGS += -fno-strict-aliasing .endif .endif PO_CFLAGS ?= ${CFLAGS} # cp(1) is used to copy source files to ${.OBJDIR}, make sure it can handle # read-only files as non-root by passing -f. CP ?= cp -f CPP ?= cpp # C Type Format data is required for DTrace CTFFLAGS ?= -L VERSION CTFCONVERT ?= ctfconvert CTFMERGE ?= ctfmerge .if defined(CFLAGS) && (${CFLAGS:M-g} != "") CTFFLAGS += -g .endif CXX ?= c++ CXXFLAGS ?= ${CFLAGS:N-std=*:N-Wnested-externs:N-W*-prototypes:N-Wno-pointer-sign:N-Wold-style-definition} PO_CXXFLAGS ?= ${CXXFLAGS} DTRACE ?= dtrace DTRACEFLAGS ?= -C -x nolibs .if empty(.MAKEFLAGS:M-s) ECHO ?= echo ECHODIR ?= echo .else ECHO ?= true .if ${.MAKEFLAGS:M-s} == "-s" ECHODIR ?= echo .else ECHODIR ?= true .endif .endif .if ${.MAKEFLAGS:M-N} # bmake -N is supposed to skip executing anything but it does not skip # exeucting '+' commands. The '+' feature is used where .MAKE # is not safe for the entire target. -N is intended to skip building sub-makes # so it executing '+' commands is not right. Work around the bug by not # setting '+' when -N is used. _+_ ?= .else _+_ ?= + .endif .if defined(%POSIX) FC ?= fort77 FFLAGS ?= -O 1 .else FC ?= f77 FFLAGS ?= -O .endif EFLAGS ?= INSTALL ?= install LEX ?= lex LFLAGS ?= LD ?= ld LDFLAGS ?= # LDFLAGS is for CC, _LDFLAGS = ${LDFLAGS:S/-Wl,//g} # strip -Wl, for LD LINT ?= lint LINTFLAGS ?= -cghapbx LINTKERNFLAGS ?= ${LINTFLAGS} LINTOBJFLAGS ?= -cghapbxu -i LINTOBJKERNFLAGS?= ${LINTOBJFLAGS} LINTLIBFLAGS ?= -cghapbxu -C ${LIB} MAKE ?= make .if !defined(%POSIX) NM ?= nm NMFLAGS ?= OBJC ?= cc OBJCFLAGS ?= ${OBJCINCLUDES} ${CFLAGS} -Wno-import OBJCOPY ?= objcopy PC ?= pc PFLAGS ?= RC ?= f77 RFLAGS ?= .endif SHELL ?= sh .if !defined(%POSIX) SIZE ?= size .endif YACC ?= yacc .if defined(%POSIX) YFLAGS ?= .else YFLAGS ?= -d .endif .if defined(%POSIX) # Posix 1003.2 mandated rules # # Quoted directly from the Posix 1003.2 draft, only the macros # $@, $< and $* have been replaced by ${.TARGET}, ${.IMPSRC}, and # ${.PREFIX}, resp. # SINGLE SUFFIX RULES .c: ${CC} ${CFLAGS} ${LDFLAGS} -o ${.TARGET} ${.IMPSRC} .f: ${FC} ${FFLAGS} ${LDFLAGS} -o ${.TARGET} ${.IMPSRC} .sh: cp -f ${.IMPSRC} ${.TARGET} chmod a+x ${.TARGET} # DOUBLE SUFFIX RULES .c.o: ${CC} ${CFLAGS} -c ${.IMPSRC} .f.o: ${FC} ${FFLAGS} -c ${.IMPSRC} .y.o: ${YACC} ${YFLAGS} ${.IMPSRC} ${CC} ${CFLAGS} -c y.tab.c rm -f y.tab.c mv y.tab.o ${.TARGET} .l.o: ${LEX} ${LFLAGS} ${.IMPSRC} ${CC} ${CFLAGS} -c lex.yy.c rm -f lex.yy.c mv lex.yy.o ${.TARGET} .y.c: ${YACC} ${YFLAGS} ${.IMPSRC} mv y.tab.c ${.TARGET} .l.c: ${LEX} ${LFLAGS} ${.IMPSRC} mv lex.yy.c ${.TARGET} .c.a: ${CC} ${CFLAGS} -c ${.IMPSRC} ${AR} ${ARFLAGS} ${.TARGET} ${.PREFIX}.o rm -f ${.PREFIX}.o .f.a: ${FC} ${FFLAGS} -c ${.IMPSRC} ${AR} ${ARFLAGS} ${.TARGET} ${.PREFIX}.o rm -f ${.PREFIX}.o .else # non-Posix rule set .sh: cp -f ${.IMPSRC} ${.TARGET} chmod a+x ${.TARGET} .c.ln: ${LINT} ${LINTOBJFLAGS} ${CFLAGS:M-[DIU]*} ${.IMPSRC} || \ touch ${.TARGET} .cc.ln .C.ln .cpp.ln .cxx.ln: ${LINT} ${LINTOBJFLAGS} ${CXXFLAGS:M-[DIU]*} ${.IMPSRC} || \ touch ${.TARGET} .c: ${CC} ${CFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET} ${CTFCONVERT_CMD} .c.o: ${CC} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET} ${CTFCONVERT_CMD} .cc .cpp .cxx .C: ${CXX} ${CXXFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET} .cc.o .cpp.o .cxx.o .C.o: ${CXX} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET} .m.o: ${OBJC} ${OBJCFLAGS} -c ${.IMPSRC} -o ${.TARGET} ${CTFCONVERT_CMD} .p.o: ${PC} ${PFLAGS} -c ${.IMPSRC} -o ${.TARGET} ${CTFCONVERT_CMD} .e .r .F .f: ${FC} ${RFLAGS} ${EFLAGS} ${FFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} \ -o ${.TARGET} .e.o .r.o .F.o .f.o: ${FC} ${RFLAGS} ${EFLAGS} ${FFLAGS} -c ${.IMPSRC} -o ${.TARGET} .S.o: ${CC:N${CCACHE_BIN}} ${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} -o ${.TARGET} ${CTFCONVERT_CMD} .asm.o: ${CC:N${CCACHE_BIN}} -x assembler-with-cpp ${CFLAGS} ${ACFLAGS} -c ${.IMPSRC} \ -o ${.TARGET} ${CTFCONVERT_CMD} .s.o: ${AS} ${AFLAGS} -o ${.TARGET} ${.IMPSRC} ${CTFCONVERT_CMD} # XXX not -j safe .y.o: ${YACC} ${YFLAGS} ${.IMPSRC} ${CC} ${CFLAGS} -c y.tab.c -o ${.TARGET} rm -f y.tab.c ${CTFCONVERT_CMD} .l.o: ${LEX} -t ${LFLAGS} ${.IMPSRC} > ${.PREFIX}.tmp.c ${CC} ${CFLAGS} -c ${.PREFIX}.tmp.c -o ${.TARGET} rm -f ${.PREFIX}.tmp.c ${CTFCONVERT_CMD} # XXX not -j safe .y.c: ${YACC} ${YFLAGS} ${.IMPSRC} mv y.tab.c ${.TARGET} .l.c: ${LEX} -t ${LFLAGS} ${.IMPSRC} > ${.TARGET} .s.out .c.out .o.out: ${CC} ${CFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET} ${CTFCONVERT_CMD} .f.out .F.out .r.out .e.out: ${FC} ${EFLAGS} ${RFLAGS} ${FFLAGS} ${LDFLAGS} ${.IMPSRC} \ ${LDLIBS} -o ${.TARGET} rm -f ${.PREFIX}.o ${CTFCONVERT_CMD} # XXX not -j safe .y.out: ${YACC} ${YFLAGS} ${.IMPSRC} ${CC} ${CFLAGS} ${LDFLAGS} y.tab.c ${LDLIBS} -ly -o ${.TARGET} rm -f y.tab.c ${CTFCONVERT_CMD} .l.out: ${LEX} -t ${LFLAGS} ${.IMPSRC} > ${.PREFIX}.tmp.c ${CC} ${CFLAGS} ${LDFLAGS} ${.PREFIX}.tmp.c ${LDLIBS} -ll -o ${.TARGET} rm -f ${.PREFIX}.tmp.c ${CTFCONVERT_CMD} # Pull in global settings. __MAKE_CONF?=/etc/make.conf .if exists(${__MAKE_CONF}) .include "${__MAKE_CONF}" .endif # late include for customization .sinclude .if defined(META_MODE) META_MODE:= ${META_MODE:O:u} .endif .if defined(__MAKE_SHELL) && !empty(__MAKE_SHELL) SHELL= ${__MAKE_SHELL} .SHELL: path=${__MAKE_SHELL} .endif # Tell bmake to expand -V VAR by default .MAKE.EXPAND_VARIABLES= yes # Tell bmake the makefile preference .MAKE.MAKEFILE_PREFERENCE= BSDmakefile makefile Makefile # Tell bmake to always pass job tokens, regardless of target depending on # .MAKE or looking like ${MAKE}/${.MAKE}/$(MAKE)/$(.MAKE)/make. .MAKE.ALWAYS_PASS_JOB_QUEUE= yes # By default bmake does *not* use set -e # when running target scripts, this is a problem for many makefiles here. # So define a shell that will do what FreeBSD expects. .ifndef WITHOUT_SHELL_ERRCTL __MAKE_SHELL?=/bin/sh .SHELL: name=sh \ quiet="set -" echo="set -v" filter="set -" \ hasErrCtl=yes check="set -e" ignore="set +e" \ echoFlag=v errFlag=e \ path=${__MAKE_SHELL} .endif # Hack for ports compatibility. Historically, ports makefiles have # assumed they can examine MACHINE_CPU without including anything # because this was automatically included in sys.mk. For /usr/src, # this file has moved to being included from bsd.opts.mk. Until all # the ports files are modernized, and a reasonable transition # period has passed, include it while we're in a ports tree here # to preserve historic behavior. .if exists(${.CURDIR}/../../Mk/bsd.port.mk) .include .endif .endif # ! Posix Index: projects/netbsd-tests-update-12/sys/arm/allwinner/aw_usbphy.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/allwinner/aw_usbphy.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/allwinner/aw_usbphy.c (revision 305172) @@ -1,240 +1,314 @@ /*- * Copyright (c) 2016 Jared McNeill * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * Allwinner USB PHY */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include +#include #include #include #include #include #include #include #include #include "phy_if.h" #define USBPHY_NPHYS 4 +#define USBPHY_NRES USBPHY_NPHYS +enum awusbphy_type { + AWUSBPHY_TYPE_A10 = 1, + AWUSBPHY_TYPE_A13, + AWUSBPHY_TYPE_A20, + AWUSBPHY_TYPE_A31, + AWUSBPHY_TYPE_A83T, + AWUSBPHY_TYPE_H3, + AWUSBPHY_TYPE_A64 +}; + static struct ofw_compat_data compat_data[] = { - { "allwinner,sun4i-a10-usb-phy", 1 }, - { "allwinner,sun5i-a13-usb-phy", 1 }, - { "allwinner,sun6i-a31-usb-phy", 1 }, - { "allwinner,sun7i-a20-usb-phy", 1 }, - { "allwinner,sun8i-a83t-usb-phy", 1 }, - { "allwinner,sun8i-h3-usb-phy", 1 }, + { "allwinner,sun4i-a10-usb-phy", AWUSBPHY_TYPE_A10 }, + { "allwinner,sun5i-a13-usb-phy", AWUSBPHY_TYPE_A13 }, + { "allwinner,sun6i-a31-usb-phy", AWUSBPHY_TYPE_A31 }, + { "allwinner,sun7i-a20-usb-phy", AWUSBPHY_TYPE_A20 }, + { "allwinner,sun8i-a83t-usb-phy", AWUSBPHY_TYPE_A83T }, + { "allwinner,sun8i-h3-usb-phy", AWUSBPHY_TYPE_H3 }, + { "allwinner,sun50i-a64-usb-phy", AWUSBPHY_TYPE_A64 }, { NULL, 0 } }; struct awusbphy_softc { + struct resource * res[USBPHY_NRES]; regulator_t reg[USBPHY_NPHYS]; gpio_pin_t id_det_pin; int id_det_valid; gpio_pin_t vbus_det_pin; int vbus_det_valid; + enum awusbphy_type phy_type; }; +static struct resource_spec awusbphy_spec[] = { + { SYS_RES_MEMORY, 0, RF_ACTIVE }, + { SYS_RES_MEMORY, 1, RF_ACTIVE }, + { SYS_RES_MEMORY, 2, RF_ACTIVE | RF_OPTIONAL }, + { SYS_RES_MEMORY, 3, RF_ACTIVE | RF_OPTIONAL }, + { -1, 0 } +}; + +#define RD4(sc, i, o) bus_read_4((sc)->res[(i)], (o)) +#define WR4(sc, i, o, v) bus_write_4((sc)->res[(i)], (o), (v)) +#define CLR4(sc, i, o, m) WR4(sc, i, o, RD4(sc, i, o) & ~(m)) +#define SET4(sc, i, o, m) WR4(sc, i, o, RD4(sc, i, o) | (m)) + +#define OTG_PHY_CFG 0x20 +#define OTG_PHY_ROUTE_OTG (1 << 0) +#define PMU_IRQ_ENABLE 0x00 +#define PMU_AHB_INCR8 (1 << 10) +#define PMU_AHB_INCR4 (1 << 9) +#define PMU_AHB_INCRX_ALIGN (1 << 8) +#define PMU_ULPI_BYPASS (1 << 0) +#define PMU_UNK_H3 0x10 +#define PMU_UNK_H3_CLR 0x2 + +static void +awusbphy_configure(device_t dev, int phyno) +{ + struct awusbphy_softc *sc; + + sc = device_get_softc(dev); + + if (sc->res[phyno] == NULL) + return; + + if (sc->phy_type == AWUSBPHY_TYPE_A64) { + CLR4(sc, phyno, PMU_UNK_H3, PMU_UNK_H3_CLR); + + /* EHCI0 and OTG share a PHY */ + if (phyno == 0) + SET4(sc, 0, OTG_PHY_CFG, OTG_PHY_ROUTE_OTG); + else if (phyno == 1) + CLR4(sc, 0, OTG_PHY_CFG, OTG_PHY_ROUTE_OTG); + } + + if (phyno > 0) { + /* Enable passby */ + SET4(sc, phyno, PMU_IRQ_ENABLE, PMU_ULPI_BYPASS | + PMU_AHB_INCR8 | PMU_AHB_INCR4 | PMU_AHB_INCRX_ALIGN); + } +} + static int awusbphy_init(device_t dev) { struct awusbphy_softc *sc; phandle_t node; char pname[20]; int error, off; regulator_t reg; hwreset_t rst; clk_t clk; sc = device_get_softc(dev); node = ofw_bus_get_node(dev); + sc->phy_type = ofw_bus_search_compatible(dev, compat_data)->ocd_data; + /* Enable clocks */ for (off = 0; clk_get_by_ofw_index(dev, 0, off, &clk) == 0; off++) { error = clk_enable(clk); if (error != 0) { device_printf(dev, "couldn't enable clock %s\n", clk_get_name(clk)); return (error); } } /* De-assert resets */ for (off = 0; hwreset_get_by_ofw_idx(dev, 0, off, &rst) == 0; off++) { error = hwreset_deassert(rst); if (error != 0) { device_printf(dev, "couldn't de-assert reset %d\n", off); return (error); } } /* Get regulators */ for (off = 0; off < USBPHY_NPHYS; off++) { snprintf(pname, sizeof(pname), "usb%d_vbus-supply", off); if (regulator_get_by_ofw_property(dev, 0, pname, ®) == 0) sc->reg[off] = reg; } /* Get GPIOs */ error = gpio_pin_get_by_ofw_property(dev, node, "usb0_id_det-gpios", &sc->id_det_pin); if (error == 0) sc->id_det_valid = 1; error = gpio_pin_get_by_ofw_property(dev, node, "usb0_vbus_det-gpios", &sc->vbus_det_pin); if (error == 0) sc->vbus_det_valid = 1; + /* Allocate resources */ + if (bus_alloc_resources(dev, awusbphy_spec, sc->res) != 0) + device_printf(dev, "couldn't allocate resources\n"); + return (0); } static int awusbphy_vbus_detect(device_t dev, int *val) { struct awusbphy_softc *sc; bool active; int error; sc = device_get_softc(dev); if (sc->vbus_det_valid) { error = gpio_pin_is_active(sc->vbus_det_pin, &active); if (error != 0) return (error); *val = active; return (0); } *val = 1; return (0); } static int awusbphy_phy_enable(device_t dev, intptr_t phy, bool enable) { struct awusbphy_softc *sc; regulator_t reg; int error, vbus_det; if (phy < 0 || phy >= USBPHY_NPHYS) return (ERANGE); sc = device_get_softc(dev); + + /* Configure PHY */ + awusbphy_configure(dev, phy); /* Regulators are optional. If not found, return success. */ reg = sc->reg[phy]; if (reg == NULL) return (0); if (enable) { /* If an external vbus is detected, do not enable phy 0 */ if (phy == 0) { error = awusbphy_vbus_detect(dev, &vbus_det); if (error == 0 && vbus_det == 1) return (0); } else error = 0; if (error == 0) error = regulator_enable(reg); } else error = regulator_disable(reg); if (error != 0) { device_printf(dev, "couldn't %s regulator for phy %jd\n", enable ? "enable" : "disable", (intmax_t)phy); return (error); } return (0); } static int awusbphy_probe(device_t dev) { if (!ofw_bus_status_okay(dev)) return (ENXIO); if (ofw_bus_search_compatible(dev, compat_data)->ocd_data == 0) return (ENXIO); device_set_desc(dev, "Allwinner USB PHY"); return (BUS_PROBE_DEFAULT); } static int awusbphy_attach(device_t dev) { int error; error = awusbphy_init(dev); if (error) { device_printf(dev, "failed to initialize USB PHY, error %d\n", error); return (error); } phy_register_provider(dev); return (error); } static device_method_t awusbphy_methods[] = { /* Device interface */ DEVMETHOD(device_probe, awusbphy_probe), DEVMETHOD(device_attach, awusbphy_attach), /* PHY interface */ DEVMETHOD(phy_enable, awusbphy_phy_enable), DEVMETHOD_END }; static driver_t awusbphy_driver = { "awusbphy", awusbphy_methods, sizeof(struct awusbphy_softc) }; static devclass_t awusbphy_devclass; EARLY_DRIVER_MODULE(awusbphy, simplebus, awusbphy_driver, awusbphy_devclass, 0, 0, BUS_PASS_RESOURCE + BUS_PASS_ORDER_MIDDLE); MODULE_VERSION(awusbphy, 1); Index: projects/netbsd-tests-update-12/sys/arm/allwinner/aw_wdog.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/allwinner/aw_wdog.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/allwinner/aw_wdog.c (revision 305172) @@ -1,259 +1,277 @@ /*- * Copyright (c) 2013 Oleksandr Tymoshenko * Copyright (c) 2016 Emmanuel Vadot * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define READ(_sc, _r) bus_read_4((_sc)->res, (_r)) #define WRITE(_sc, _r, _v) bus_write_4((_sc)->res, (_r), (_v)) #define A10_WDOG_CTRL 0x00 #define A31_WDOG_CTRL 0x10 #define WDOG_CTRL_RESTART (1 << 0) +#define A31_WDOG_CTRL_KEY (0xa57 << 1) #define A10_WDOG_MODE 0x04 #define A31_WDOG_MODE 0x18 #define A10_WDOG_MODE_INTVL_SHIFT 3 #define A31_WDOG_MODE_INTVL_SHIFT 4 #define A10_WDOG_MODE_RST_EN (1 << 1) #define WDOG_MODE_EN (1 << 0) #define A31_WDOG_CONFIG 0x14 #define A31_WDOG_CONFIG_RST_EN_SYSTEM (1 << 0) #define A31_WDOG_CONFIG_RST_EN_INT (2 << 0) struct aw_wdog_interval { uint64_t milliseconds; unsigned int value; }; struct aw_wdog_interval wd_intervals[] = { { 500, 0 }, { 1000, 1 }, { 2000, 2 }, { 3000, 3 }, { 4000, 4 }, { 5000, 5 }, { 6000, 6 }, { 8000, 7 }, { 10000, 8 }, { 12000, 9 }, { 14000, 10 }, { 16000, 11 }, { 0, 0 } /* sentinel */ }; static struct aw_wdog_softc *aw_wdog_sc = NULL; struct aw_wdog_softc { device_t dev; struct resource * res; struct mtx mtx; uint8_t wdog_ctrl; + uint32_t wdog_ctrl_key; uint8_t wdog_mode; uint8_t wdog_mode_intvl_shift; uint8_t wdog_mode_en; uint8_t wdog_config; uint8_t wdog_config_value; }; #define A10_WATCHDOG 1 #define A31_WATCHDOG 2 static struct ofw_compat_data compat_data[] = { {"allwinner,sun4i-a10-wdt", A10_WATCHDOG}, {"allwinner,sun6i-a31-wdt", A31_WATCHDOG}, {NULL, 0} }; -static void aw_wdog_watchdog_fn(void *private, u_int cmd, int *error); +static void aw_wdog_watchdog_fn(void *, u_int, int *); +static void aw_wdog_shutdown_fn(void *, int); static int aw_wdog_probe(device_t dev) { struct aw_wdog_softc *sc; sc = device_get_softc(dev); if (!ofw_bus_status_okay(dev)) return (ENXIO); switch (ofw_bus_search_compatible(dev, compat_data)->ocd_data) { case A10_WATCHDOG: device_set_desc(dev, "Allwinner A10 Watchdog"); return (BUS_PROBE_DEFAULT); case A31_WATCHDOG: device_set_desc(dev, "Allwinner A31 Watchdog"); return (BUS_PROBE_DEFAULT); } return (ENXIO); } static int aw_wdog_attach(device_t dev) { struct aw_wdog_softc *sc; int rid; if (aw_wdog_sc != NULL) return (ENXIO); sc = device_get_softc(dev); sc->dev = dev; rid = 0; sc->res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, RF_ACTIVE); if (sc->res == NULL) { device_printf(dev, "could not allocate memory resource\n"); return (ENXIO); } aw_wdog_sc = sc; switch (ofw_bus_search_compatible(dev, compat_data)->ocd_data) { case A10_WATCHDOG: sc->wdog_ctrl = A10_WDOG_CTRL; sc->wdog_mode = A10_WDOG_MODE; sc->wdog_mode_intvl_shift = A10_WDOG_MODE_INTVL_SHIFT; sc->wdog_mode_en = A10_WDOG_MODE_RST_EN | WDOG_MODE_EN; break; case A31_WATCHDOG: sc->wdog_ctrl = A31_WDOG_CTRL; + sc->wdog_ctrl_key = A31_WDOG_CTRL_KEY; sc->wdog_mode = A31_WDOG_MODE; sc->wdog_mode_intvl_shift = A31_WDOG_MODE_INTVL_SHIFT; sc->wdog_mode_en = WDOG_MODE_EN; sc->wdog_config = A31_WDOG_CONFIG; sc->wdog_config_value = A31_WDOG_CONFIG_RST_EN_SYSTEM; break; default: bus_release_resource(dev, SYS_RES_MEMORY, rid, sc->res); return (ENXIO); } mtx_init(&sc->mtx, "AW Watchdog", "aw_wdog", MTX_DEF); EVENTHANDLER_REGISTER(watchdog_list, aw_wdog_watchdog_fn, sc, 0); + EVENTHANDLER_REGISTER(shutdown_final, aw_wdog_shutdown_fn, sc, + SHUTDOWN_PRI_LAST - 1); + return (0); } static void aw_wdog_watchdog_fn(void *private, u_int cmd, int *error) { struct aw_wdog_softc *sc; uint64_t ms; int i; sc = private; mtx_lock(&sc->mtx); cmd &= WD_INTERVAL; if (cmd > 0) { ms = ((uint64_t)1 << (cmd & WD_INTERVAL)) / 1000000; i = 0; while (wd_intervals[i].milliseconds && (ms > wd_intervals[i].milliseconds)) i++; if (wd_intervals[i].milliseconds) { WRITE(sc, sc->wdog_mode, (wd_intervals[i].value << sc->wdog_mode_intvl_shift) | sc->wdog_mode_en); - WRITE(sc, sc->wdog_ctrl, WDOG_CTRL_RESTART); + WRITE(sc, sc->wdog_ctrl, + WDOG_CTRL_RESTART | sc->wdog_ctrl_key); if (sc->wdog_config) WRITE(sc, sc->wdog_config, sc->wdog_config_value); *error = 0; } else { /* * Can't arm * disable watchdog as watchdog(9) requires */ device_printf(sc->dev, "Can't arm, timeout is more than 16 sec\n"); mtx_unlock(&sc->mtx); WRITE(sc, sc->wdog_mode, 0); return; } } else WRITE(sc, sc->wdog_mode, 0); mtx_unlock(&sc->mtx); } +static void +aw_wdog_shutdown_fn(void *private, int howto) +{ + if ((howto & (RB_POWEROFF|RB_HALT)) == 0) + aw_wdog_watchdog_reset(); +} + void aw_wdog_watchdog_reset() { if (aw_wdog_sc == NULL) { printf("Reset: watchdog device has not been initialized\n"); return; } WRITE(aw_wdog_sc, aw_wdog_sc->wdog_mode, (wd_intervals[0].value << aw_wdog_sc->wdog_mode_intvl_shift) | aw_wdog_sc->wdog_mode_en); if (aw_wdog_sc->wdog_config) WRITE(aw_wdog_sc, aw_wdog_sc->wdog_config, aw_wdog_sc->wdog_config_value); + WRITE(aw_wdog_sc, aw_wdog_sc->wdog_ctrl, + WDOG_CTRL_RESTART | aw_wdog_sc->wdog_ctrl_key); while(1) ; } static device_method_t aw_wdog_methods[] = { DEVMETHOD(device_probe, aw_wdog_probe), DEVMETHOD(device_attach, aw_wdog_attach), DEVMETHOD_END }; static driver_t aw_wdog_driver = { "aw_wdog", aw_wdog_methods, sizeof(struct aw_wdog_softc), }; static devclass_t aw_wdog_devclass; DRIVER_MODULE(aw_wdog, simplebus, aw_wdog_driver, aw_wdog_devclass, 0, 0); Index: projects/netbsd-tests-update-12/sys/arm/arm/cpufunc_asm_xscale_c3.S =================================================================== --- projects/netbsd-tests-update-12/sys/arm/arm/cpufunc_asm_xscale_c3.S (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/arm/cpufunc_asm_xscale_c3.S (revision 305172) @@ -1,400 +1,398 @@ /* $NetBSD: cpufunc_asm_xscale.S,v 1.16 2002/08/17 16:36:32 thorpej Exp $ */ /*- * Copyright (c) 2007 Olivier Houchard * Copyright (c) 2001, 2002 Wasabi Systems, Inc. * All rights reserved. * * Written by Allen Briggs and Jason R. Thorpe for Wasabi Systems, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed for the NetBSD Project by * Wasabi Systems, Inc. * 4. The name of Wasabi Systems, Inc. may not be used to endorse * or promote products derived from this software without specific prior * written permission. * * THIS SOFTWARE IS PROVIDED BY WASABI SYSTEMS, INC. ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WASABI SYSTEMS, INC * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. * */ /*- * Copyright (c) 2001 Matt Thomas. * Copyright (c) 1997,1998 Mark Brinicombe. * Copyright (c) 1997 Causality Limited * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Causality Limited. * 4. The name of Causality Limited may not be used to endorse or promote * products derived from this software without specific prior written * permission. * * THIS SOFTWARE IS PROVIDED BY CAUSALITY LIMITED ``AS IS'' AND ANY EXPRESS * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE * DISCLAIMED. IN NO EVENT SHALL CAUSALITY LIMITED BE LIABLE FOR ANY DIRECT, * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * XScale core 3 assembly functions for CPU / MMU / TLB specific operations */ #include #include __FBSDID("$FreeBSD$"); /* * Size of the XScale core D-cache. */ #define DCACHE_SIZE 0x00008000 /* * CPWAIT -- Canonical method to wait for CP15 update. * From: Intel 80200 manual, section 2.3.3. * * NOTE: Clobbers the specified temp reg. */ #define CPWAIT_BRANCH \ sub pc, pc, #4 #define CPWAIT(tmp) \ mrc p15, 0, tmp, c2, c0, 0 /* arbitrary read of CP15 */ ;\ mov tmp, tmp /* wait for it to complete */ ;\ CPWAIT_BRANCH /* branch to next insn */ #define CPWAIT_AND_RETURN_SHIFTER lsr #32 #define CPWAIT_AND_RETURN(tmp) \ mrc p15, 0, tmp, c2, c0, 0 /* arbitrary read of CP15 */ ;\ /* Wait for it to complete and branch to the return address */ \ sub pc, lr, tmp, CPWAIT_AND_RETURN_SHIFTER #define ARM_USE_L2_CACHE #define L2_CACHE_SIZE 0x80000 #define L2_CACHE_WAYS 8 #define L2_CACHE_LINE_SIZE 32 #define L2_CACHE_SETS (L2_CACHE_SIZE / \ (L2_CACHE_WAYS * L2_CACHE_LINE_SIZE)) #define L1_DCACHE_SIZE 32 * 1024 #define L1_DCACHE_WAYS 4 #define L1_DCACHE_LINE_SIZE 32 #define L1_DCACHE_SETS (L1_DCACHE_SIZE / \ (L1_DCACHE_WAYS * L1_DCACHE_LINE_SIZE)) #ifdef CACHE_CLEAN_BLOCK_INTR #define XSCALE_CACHE_CLEAN_BLOCK \ stmfd sp!, {r4} ; \ mrs r4, cpsr ; \ orr r0, r4, #(PSR_I | PSR_F) ; \ msr cpsr_fsxc, r0 #define XSCALE_CACHE_CLEAN_UNBLOCK \ msr cpsr_fsxc, r4 ; \ ldmfd sp!, {r4} #else #define XSCALE_CACHE_CLEAN_BLOCK #define XSCALE_CACHE_CLEAN_UNBLOCK #endif /* CACHE_CLEAN_BLOCK_INTR */ ENTRY_NP(xscalec3_cache_syncI) EENTRY_NP(xscalec3_cache_purgeID) mcr p15, 0, r0, c7, c5, 0 /* flush I cache (D cleaned below) */ EENTRY_NP(xscalec3_cache_cleanID) EENTRY_NP(xscalec3_cache_purgeD) EENTRY(xscalec3_cache_cleanD) XSCALE_CACHE_CLEAN_BLOCK mov r0, #0 1: mov r1, r0, asl #30 mov r2, #0 2: orr r3, r1, r2, asl #5 mcr p15, 0, r3, c7, c14, 2 /* clean and invalidate */ add r2, r2, #1 cmp r2, #L1_DCACHE_SETS bne 2b add r0, r0, #1 cmp r0, #4 bne 1b CPWAIT(r0) XSCALE_CACHE_CLEAN_UNBLOCK mcr p15, 0, r0, c7, c10, 4 /* drain write buffer */ RET EEND(xscalec3_cache_purgeID) EEND(xscalec3_cache_cleanID) EEND(xscalec3_cache_purgeD) EEND(xscalec3_cache_cleanD) END(xscalec3_cache_syncI) ENTRY(xscalec3_cache_purgeID_rng) cmp r1, #0x4000 bcs _C_LABEL(xscalec3_cache_cleanID) and r2, r0, #0x1f add r1, r1, r2 bic r0, r0, #0x1f 1: mcr p15, 0, r0, c7, c14, 1 /* clean/invalidate L1 D cache entry */ nop mcr p15, 0, r0, c7, c5, 1 /* flush I cache single entry */ add r0, r0, #32 subs r1, r1, #32 bhi 1b CPWAIT(r0) mcr p15, 0, r0, c7, c10, 4 /* drain write buffer */ CPWAIT_AND_RETURN(r0) END(xscalec3_cache_purgeID_rng) ENTRY(xscalec3_cache_syncI_rng) cmp r1, #0x4000 bcs _C_LABEL(xscalec3_cache_syncI) and r2, r0, #0x1f add r1, r1, r2 bic r0, r0, #0x1f 1: mcr p15, 0, r0, c7, c10, 1 /* clean D cache entry */ mcr p15, 0, r0, c7, c5, 1 /* flush I cache single entry */ add r0, r0, #32 subs r1, r1, #32 bhi 1b CPWAIT(r0) mcr p15, 0, r0, c7, c10, 4 /* drain write buffer */ CPWAIT_AND_RETURN(r0) END(xscalec3_cache_syncI_rng) ENTRY(xscalec3_cache_purgeD_rng) cmp r1, #0x4000 bcs _C_LABEL(xscalec3_cache_cleanID) and r2, r0, #0x1f add r1, r1, r2 bic r0, r0, #0x1f 1: mcr p15, 0, r0, c7, c14, 1 /* Clean and invalidate D cache entry */ add r0, r0, #32 subs r1, r1, #32 bhi 1b CPWAIT(r0) mcr p15, 0, r0, c7, c10, 4 /* drain write buffer */ CPWAIT_AND_RETURN(r0) END(xscalec3_cache_purgeD_rng) ENTRY(xscalec3_cache_cleanID_rng) EENTRY(xscalec3_cache_cleanD_rng) cmp r1, #0x4000 bcs _C_LABEL(xscalec3_cache_cleanID) and r2, r0, #0x1f add r1, r1, r2 bic r0, r0, #0x1f 1: mcr p15, 0, r0, c7, c10, 1 /* clean L1 D cache entry */ nop add r0, r0, #32 subs r1, r1, #32 bhi 1b CPWAIT(r0) mcr p15, 0, r0, c7, c10, 4 /* drain write buffer */ CPWAIT_AND_RETURN(r0) EEND(xscalec3_cache_cleanD_rng) END(xscalec3_cache_cleanID_rng) ENTRY(xscalec3_l2cache_purge) /* Clean-up the L2 cache */ mcr p15, 0, r0, c7, c10, 5 /* Data memory barrier */ mov r0, #0 1: mov r1, r0, asl #29 mov r2, #0 2: orr r3, r1, r2, asl #5 mcr p15, 1, r3, c7, c15, 2 add r2, r2, #1 cmp r2, #L2_CACHE_SETS bne 2b add r0, r0, #1 cmp r0, #8 bne 1b mcr p15, 0, r0, c7, c10, 4 @ data write barrier CPWAIT(r0) mcr p15, 0, r0, c7, c10, 5 /* Data memory barrier */ RET END(xscalec3_l2cache_purge) ENTRY(xscalec3_l2cache_clean_rng) mcr p15, 0, r0, c7, c10, 5 /* Data memory barrier */ and r2, r0, #0x1f add r1, r1, r2 bic r0, r0, #0x1f 1: mcr p15, 1, r0, c7, c11, 1 /* Clean L2 D cache entry */ add r0, r0, #32 subs r1, r1, #32 bhi 1b CPWAIT(r0) mcr p15, 0, r0, c7, c10, 4 @ data write barrier mcr p15, 0, r0, c7, c10, 5 CPWAIT_AND_RETURN(r0) END(xscalec3_l2cache_clean_rng) ENTRY(xscalec3_l2cache_purge_rng) mcr p15, 0, r0, c7, c10, 5 /* Data memory barrier */ and r2, r0, #0x1f add r1, r1, r2 bic r0, r0, #0x1f 1: mcr p15, 1, r0, c7, c11, 1 /* Clean L2 D cache entry */ mcr p15, 1, r0, c7, c7, 1 /* Invalidate L2 D cache entry */ add r0, r0, #32 subs r1, r1, #32 bhi 1b mcr p15, 0, r0, c7, c10, 4 @ data write barrier mcr p15, 0, r0, c7, c10, 5 CPWAIT_AND_RETURN(r0) END(xscalec3_l2cache_purge_rng) ENTRY(xscalec3_l2cache_flush_rng) mcr p15, 0, r0, c7, c10, 5 /* Data memory barrier */ and r2, r0, #0x1f add r1, r1, r2 bic r0, r0, #0x1f 1: mcr p15, 1, r0, c7, c7, 1 /* Invalidate L2 cache line */ add r0, r0, #32 subs r1, r1, #32 bhi 1b mcr p15, 0, r0, c7, c10, 4 @ data write barrier mcr p15, 0, r0, c7, c10, 5 CPWAIT_AND_RETURN(r0) END(xscalec3_l2cache_flush_rng) /* * Functions to set the MMU Translation Table Base register * * We need to clean and flush the cache as it uses virtual * addresses that are about to change. */ ENTRY(xscalec3_setttb) #ifdef CACHE_CLEAN_BLOCK_INTR mrs r3, cpsr orr r1, r3, #(PSR_I | PSR_F) msr cpsr_fsxc, r1 #endif stmfd sp!, {r0-r3, lr} bl _C_LABEL(xscalec3_cache_cleanID) mcr p15, 0, r0, c7, c5, 0 /* invalidate I$ and BTB */ mcr p15, 0, r0, c7, c10, 4 /* drain write and fill buffer */ CPWAIT(r0) ldmfd sp!, {r0-r3, lr} #ifdef ARM_USE_L2_CACHE orr r0, r0, #0x18 /* cache the page table in L2 */ #endif /* Write the TTB */ mcr p15, 0, r0, c2, c0, 0 /* If we have updated the TTB we must flush the TLB */ mcr p15, 0, r0, c8, c7, 0 /* invalidate I+D TLB */ CPWAIT(r0) #ifdef CACHE_CLEAN_BLOCK_INTR msr cpsr_fsxc, r3 -#else - str r2, [r3] #endif RET END(xscalec3_setttb) /* * Context switch. * * These is the CPU-specific parts of the context switcher cpu_switch() * These functions actually perform the TTB reload. * * NOTE: Special calling convention * r1, r4-r13 must be preserved */ ENTRY(xscalec3_context_switch) /* * CF_CACHE_PURGE_ID will *ALWAYS* be called prior to this. * Thus the data cache will contain only kernel data and the * instruction cache will contain only kernel code, and all * kernel mappings are shared by all processes. */ #ifdef ARM_USE_L2_CACHE orr r0, r0, #0x18 /* Cache the page table in L2 */ #endif /* Write the TTB */ mcr p15, 0, r0, c2, c0, 0 /* If we have updated the TTB we must flush the TLB */ mcr p15, 0, r0, c8, c7, 0 /* flush the I+D tlb */ CPWAIT_AND_RETURN(r0) END(xscalec3_context_switch) Index: projects/netbsd-tests-update-12/sys/arm/arm/locore-v4.S =================================================================== --- projects/netbsd-tests-update-12/sys/arm/arm/locore-v4.S (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/arm/locore-v4.S (revision 305172) @@ -1,483 +1,493 @@ /* $NetBSD: locore.S,v 1.14 2003/04/20 16:21:40 thorpej Exp $ */ /*- * Copyright 2011 Semihalf * Copyright (C) 1994-1997 Mark Brinicombe * Copyright (C) 1994 Brini * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Brini. * 4. The name of Brini may not be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY BRINI ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL BRINI BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * */ #include "assym.s" #include #include #include #include #include __FBSDID("$FreeBSD$"); /* 2K initial stack is plenty, it is only used by initarm() */ #define INIT_ARM_STACK_SIZE 2048 #define CPWAIT_BRANCH \ sub pc, pc, #4 #define CPWAIT(tmp) \ mrc p15, 0, tmp, c2, c0, 0 /* arbitrary read of CP15 */ ;\ mov tmp, tmp /* wait for it to complete */ ;\ CPWAIT_BRANCH /* branch to next insn */ /* * This is for libkvm, and should be the address of the beginning * of the kernel text segment (not necessarily the same as kernbase). * * These are being phased out. Newer copies of libkvm don't need these * values as the information is added to the core file by inspecting * the running kernel. */ .text .align 2 #ifdef PHYSADDR .globl kernbase .set kernbase,KERNBASE .globl physaddr .set physaddr,PHYSADDR #endif /* * On entry for FreeBSD boot ABI: * r0 - metadata pointer or 0 (boothowto on AT91's boot2) * r1 - if (r0 == 0) then metadata pointer * On entry for Linux boot ABI: * r0 - 0 * r1 - machine type (passed as arg2 to initarm) * r2 - Pointer to a tagged list or dtb image (phys addr) (passed as arg1 initarm) * * For both types of boot we gather up the args, put them in a struct arm_boot_params * structure and pass that to initarm. */ .globl btext btext: ASENTRY_NP(_start) STOP_UNWINDING /* Can't unwind into the bootloader! */ mov r9, r0 /* 0 or boot mode from boot2 */ mov r8, r1 /* Save Machine type */ mov ip, r2 /* Save meta data */ mov fp, r3 /* Future expansion */ /* Make sure interrupts are disabled. */ mrs r7, cpsr orr r7, r7, #(PSR_I | PSR_F) msr cpsr_c, r7 #if defined (FLASHADDR) && defined(LOADERRAMADDR) /* * Sanity check the configuration. * FLASHADDR and LOADERRAMADDR depend on PHYSADDR in some cases. * ARMv4 and ARMv5 make assumptions on where they are loaded. * TODO: Fix the ARMv4/v5 case. */ #ifndef PHYSADDR #error PHYSADDR must be defined for this configuration #endif /* Check if we're running from flash. */ ldr r7, =FLASHADDR /* * If we're running with MMU disabled, test against the * physical address instead. */ mrc CP15_SCTLR(r2) ands r2, r2, #CPU_CONTROL_MMU_ENABLE ldreq r6, =PHYSADDR ldrne r6, =LOADERRAMADDR cmp r7, r6 bls flash_lower cmp r7, pc bhi from_ram b do_copy flash_lower: cmp r6, pc bls from_ram do_copy: ldr r7, =KERNBASE adr r1, _start ldr r0, Lreal_start ldr r2, Lend sub r2, r2, r0 sub r0, r0, r7 add r0, r0, r6 mov r4, r0 bl memcpy ldr r0, Lram_offset add pc, r4, r0 Lram_offset: .word from_ram-_C_LABEL(_start) from_ram: nop #endif disable_mmu: /* Disable MMU for a while */ mrc CP15_SCTLR(r2) bic r2, r2, #(CPU_CONTROL_MMU_ENABLE | CPU_CONTROL_DC_ENABLE |\ CPU_CONTROL_WBUF_ENABLE) bic r2, r2, #(CPU_CONTROL_IC_ENABLE) bic r2, r2, #(CPU_CONTROL_BPRD_ENABLE) mcr CP15_SCTLR(r2) nop nop nop CPWAIT(r0) Lunmapped: /* * Build page table from scratch. */ /* * Figure out the physical address we're loaded at by assuming this * entry point code is in the first L1 section and so if we clear the * offset bits of the pc that will give us the section-aligned load * address, which remains in r5 throughout all the following code. */ ldr r2, =(L1_S_OFFSET) bic r5, pc, r2 /* Find the delta between VA and PA, result stays in r0 throughout. */ adr r0, Lpagetable bl translate_va_to_pa /* * First map the entire 4GB address space as VA=PA. It's mapped as * normal (cached) memory because it's for things like accessing the * parameters passed in from the bootloader, which might be at any * physical address, different for every platform. */ mov r1, #0 mov r2, #0 mov r3, #4096 bl build_pagetables /* * Next we do 64MiB starting at the physical load address, mapped to * the VA the kernel is linked for. */ mov r1, r5 ldr r2, =(KERNVIRTADDR) mov r3, #64 bl build_pagetables +#if defined(PHYSADDR) && (KERNVIRTADDR != KERNBASE) +/* + * If the kernel wasn't loaded at the beginning of the ram, map the memory + * before the kernel too, as some ports use that for pagetables, stack, etc... + */ + ldr r1, =PHYSADDR + ldr r2, =KERNBASE + ldr r3, =((KERNVIRTADDR - KERNBASE) / L1_S_SIZE) + bl build_pagetables +#endif /* Create a device mapping for early_printf if specified. */ #if defined(SOCDEV_PA) && defined(SOCDEV_VA) ldr r1, =SOCDEV_PA ldr r2, =SOCDEV_VA mov r3, #1 bl build_device_pagetables #endif mcr p15, 0, r0, c2, c0, 0 /* Set TTB */ mcr p15, 0, r0, c8, c7, 0 /* Flush TLB */ /* Set the Domain Access register. Very important! */ mov r0, #((DOMAIN_CLIENT << (PMAP_DOMAIN_KERNEL*2)) | DOMAIN_CLIENT) mcr p15, 0, r0, c3, c0, 0 /* * Enable MMU. */ mrc CP15_SCTLR(r0) orr r0, r0, #(CPU_CONTROL_MMU_ENABLE) mcr CP15_SCTLR(r0) nop nop nop CPWAIT(r0) /* Transition the PC from physical to virtual addressing. */ ldr pc,=mmu_done mmu_done: nop adr r1, .Lstart ldmia r1, {r1, r2, sp} /* Set initial stack and */ sub r2, r2, r1 /* get zero init data */ mov r3, #0 .L1: str r3, [r1], #0x0004 /* get zero init data */ subs r2, r2, #4 bgt .L1 virt_done: mov r1, #28 /* loader info size is 28 bytes also second arg */ subs sp, sp, r1 /* allocate arm_boot_params struct on stack */ mov r0, sp /* loader info pointer is first arg */ bic sp, sp, #7 /* align stack to 8 bytes */ str r1, [r0] /* Store length of loader info */ str r9, [r0, #4] /* Store r0 from boot loader */ str r8, [r0, #8] /* Store r1 from boot loader */ str ip, [r0, #12] /* store r2 from boot loader */ str fp, [r0, #16] /* store r3 from boot loader */ str r5, [r0, #20] /* store the physical address */ adr r4, Lpagetable /* load the pagetable address */ ldr r5, [r4, #4] str r5, [r0, #24] /* store the pagetable address */ mov fp, #0 /* trace back starts here */ bl _C_LABEL(initarm) /* Off we go */ /* init arm will return the new stack pointer. */ mov sp, r0 bl _C_LABEL(mi_startup) /* call mi_startup()! */ adr r0, .Lmainreturned b _C_LABEL(panic) /* NOTREACHED */ END(_start) #define VA_TO_PA_POINTER(name, table) \ name: ;\ .word . ;\ .word table /* * Returns the physical address of a magic va to pa pointer. * r0 - The pagetable data pointer. This must be built using the * VA_TO_PA_POINTER macro. * e.g. * VA_TO_PA_POINTER(Lpagetable, pagetable) * ... * adr r0, Lpagetable * bl translate_va_to_pa * r0 will now contain the physical address of pagetable * r1, r2 - Trashed */ translate_va_to_pa: ldr r1, [r0] sub r2, r1, r0 /* At this point: r2 = VA - PA */ /* * Find the physical address of the table. After these two * instructions: * r1 = va(pagetable) * * r0 = va(pagetable) - (VA - PA) * = va(pagetable) - VA + PA * = pa(pagetable) */ ldr r1, [r0, #4] sub r0, r1, r2 RET /* * Builds the page table * r0 - The table base address * r1 - The physical address (trashed) * r2 - The virtual address (trashed) * r3 - The number of 1MiB sections * r4 - Trashed * * Addresses must be 1MiB aligned */ build_device_pagetables: ldr r4, =(L1_TYPE_S|L1_S_AP(AP_KRW)) b 1f build_pagetables: /* Set the required page attributed */ ldr r4, =(L1_TYPE_S|L1_S_C|L1_S_AP(AP_KRW)) 1: orr r1, r4 /* Move the virtual address to the correct bit location */ lsr r2, #(L1_S_SHIFT - 2) mov r4, r3 2: str r1, [r0, r2] add r2, r2, #4 add r1, r1, #(L1_S_SIZE) adds r4, r4, #-1 bhi 2b RET VA_TO_PA_POINTER(Lpagetable, pagetable) Lreal_start: .word _start Lend: .word _edata .Lstart: .word _edata .word _ebss .word svcstk + INIT_ARM_STACK_SIZE .Lvirt_done: .word virt_done .Lmainreturned: .asciz "main() returned" .align 2 .bss svcstk: .space INIT_ARM_STACK_SIZE /* * Memory for the initial pagetable. We are unable to place this in * the bss as this will be cleared after the table is loaded. */ .section ".init_pagetable" .align 14 /* 16KiB aligned */ pagetable: .space L1_TABLE_SIZE .text .align 2 .Lcpufuncs: .word _C_LABEL(cpufuncs) ENTRY_NP(cpu_halt) mrs r2, cpsr bic r2, r2, #(PSR_MODE) orr r2, r2, #(PSR_SVC32_MODE) orr r2, r2, #(PSR_I | PSR_F) msr cpsr_fsxc, r2 ldr r4, .Lcpu_reset_address ldr r4, [r4] ldr r0, .Lcpufuncs mov lr, pc ldr pc, [r0, #CF_IDCACHE_WBINV_ALL] mov lr, pc ldr pc, [r0, #CF_L2CACHE_WBINV_ALL] /* * Load the cpu_reset_needs_v4_MMU_disable flag to determine if it's * necessary. */ ldr r1, .Lcpu_reset_needs_v4_MMU_disable ldr r1, [r1] cmp r1, #0 mov r2, #0 /* * MMU & IDC off, 32 bit program & data space * Hurl ourselves into the ROM */ mov r0, #(CPU_CONTROL_32BP_ENABLE | CPU_CONTROL_32BD_ENABLE) mcr CP15_SCTLR(r0) mcrne p15, 0, r2, c8, c7, 0 /* nail I+D TLB on ARMv4 and greater */ mov pc, r4 /* * _cpu_reset_address contains the address to branch to, to complete * the cpu reset after turning the MMU off * This variable is provided by the hardware specific code */ .Lcpu_reset_address: .word _C_LABEL(cpu_reset_address) /* * cpu_reset_needs_v4_MMU_disable contains a flag that signals if the * v4 MMU disable instruction needs executing... it is an illegal instruction * on f.e. ARM6/7 that locks up the computer in an endless illegal * instruction / data-abort / reset loop. */ .Lcpu_reset_needs_v4_MMU_disable: .word _C_LABEL(cpu_reset_needs_v4_MMU_disable) END(cpu_halt) /* * setjump + longjmp */ ENTRY(setjmp) stmia r0, {r4-r14} mov r0, #0x00000000 RET END(setjmp) ENTRY(longjmp) ldmia r0, {r4-r14} mov r0, #0x00000001 RET END(longjmp) .data .global _C_LABEL(esym) _C_LABEL(esym): .word _C_LABEL(end) ENTRY_NP(abort) b _C_LABEL(abort) END(abort) ENTRY_NP(sigcode) mov r0, sp add r0, r0, #SIGF_UC /* * Call the sigreturn system call. * * We have to load r7 manually rather than using * "ldr r7, =SYS_sigreturn" to ensure the value of szsigcode is * correct. Using the alternative places esigcode at the address * of the data rather than the address one past the data. */ ldr r7, [pc, #12] /* Load SYS_sigreturn */ swi SYS_sigreturn /* Well if that failed we better exit quick ! */ ldr r7, [pc, #8] /* Load SYS_exit */ swi SYS_exit /* Branch back to retry SYS_sigreturn */ b . - 16 END(sigcode) .word SYS_sigreturn .word SYS_exit .align 2 .global _C_LABEL(esigcode) _C_LABEL(esigcode): .data .global szsigcode szsigcode: .long esigcode-sigcode /* End of locore.S */ Index: projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_prcm.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_prcm.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_prcm.c (revision 305172) @@ -1,855 +1,856 @@ /*- * Copyright (c) 2012 Damjan Marion * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include +#include "am335x_scm.h" + #define CM_PER 0 #define CM_PER_L4LS_CLKSTCTRL (CM_PER + 0x000) #define CM_PER_L3S_CLKSTCTRL (CM_PER + 0x004) #define CM_PER_L3_CLKSTCTRL (CM_PER + 0x00C) #define CM_PER_CPGMAC0_CLKCTRL (CM_PER + 0x014) #define CM_PER_LCDC_CLKCTRL (CM_PER + 0x018) #define CM_PER_USB0_CLKCTRL (CM_PER + 0x01C) #define CM_PER_TPTC0_CLKCTRL (CM_PER + 0x024) #define CM_PER_UART5_CLKCTRL (CM_PER + 0x038) #define CM_PER_MMC0_CLKCTRL (CM_PER + 0x03C) #define CM_PER_I2C2_CLKCTRL (CM_PER + 0x044) #define CM_PER_I2C1_CLKCTRL (CM_PER + 0x048) #define CM_PER_SPI0_CLKCTRL (CM_PER + 0x04C) #define CM_PER_SPI1_CLKCTRL (CM_PER + 0x050) #define CM_PER_UART1_CLKCTRL (CM_PER + 0x06C) #define CM_PER_UART2_CLKCTRL (CM_PER + 0x070) #define CM_PER_UART3_CLKCTRL (CM_PER + 0x074) #define CM_PER_UART4_CLKCTRL (CM_PER + 0x078) #define CM_PER_TIMER7_CLKCTRL (CM_PER + 0x07C) #define CM_PER_TIMER2_CLKCTRL (CM_PER + 0x080) #define CM_PER_TIMER3_CLKCTRL (CM_PER + 0x084) #define CM_PER_TIMER4_CLKCTRL (CM_PER + 0x088) #define CM_PER_GPIO1_CLKCTRL (CM_PER + 0x0AC) #define CM_PER_GPIO2_CLKCTRL (CM_PER + 0x0B0) #define CM_PER_GPIO3_CLKCTRL (CM_PER + 0x0B4) #define CM_PER_TPCC_CLKCTRL (CM_PER + 0x0BC) #define CM_PER_EPWMSS1_CLKCTRL (CM_PER + 0x0CC) #define CM_PER_EPWMSS0_CLKCTRL (CM_PER + 0x0D4) #define CM_PER_EPWMSS2_CLKCTRL (CM_PER + 0x0D8) #define CM_PER_L3_INSTR_CLKCTRL (CM_PER + 0x0DC) #define CM_PER_L3_CLKCTRL (CM_PER + 0x0E0) #define CM_PER_PRUSS_CLKCTRL (CM_PER + 0x0E8) #define CM_PER_TIMER5_CLKCTRL (CM_PER + 0x0EC) #define CM_PER_TIMER6_CLKCTRL (CM_PER + 0x0F0) #define CM_PER_MMC1_CLKCTRL (CM_PER + 0x0F4) #define CM_PER_MMC2_CLKCTRL (CM_PER + 0x0F8) #define CM_PER_TPTC1_CLKCTRL (CM_PER + 0x0FC) #define CM_PER_TPTC2_CLKCTRL (CM_PER + 0x100) #define CM_PER_SPINLOCK0_CLKCTRL (CM_PER + 0x10C) #define CM_PER_MAILBOX0_CLKCTRL (CM_PER + 0x110) #define CM_PER_OCPWP_L3_CLKSTCTRL (CM_PER + 0x12C) #define CM_PER_OCPWP_CLKCTRL (CM_PER + 0x130) #define CM_PER_CPSW_CLKSTCTRL (CM_PER + 0x144) #define CM_PER_PRUSS_CLKSTCTRL (CM_PER + 0x140) #define CM_WKUP 0x400 #define CM_WKUP_CLKSTCTRL (CM_WKUP + 0x000) #define CM_WKUP_CONTROL_CLKCTRL (CM_WKUP + 0x004) #define CM_WKUP_GPIO0_CLKCTRL (CM_WKUP + 0x008) #define CM_WKUP_CM_L3_AON_CLKSTCTRL (CM_WKUP + 0x01C) #define CM_WKUP_CM_CLKSEL_DPLL_MPU (CM_WKUP + 0x02C) #define CM_WKUP_CM_IDLEST_DPLL_DISP (CM_WKUP + 0x048) #define CM_WKUP_CM_CLKSEL_DPLL_DISP (CM_WKUP + 0x054) #define CM_WKUP_CM_CLKDCOLDO_DPLL_PER (CM_WKUP + 0x07C) #define CM_WKUP_CM_CLKMODE_DPLL_DISP (CM_WKUP + 0x098) #define CM_WKUP_I2C0_CLKCTRL (CM_WKUP + 0x0B8) #define CM_WKUP_ADC_TSC_CLKCTRL (CM_WKUP + 0x0BC) #define CM_DPLL 0x500 #define CLKSEL_TIMER7_CLK (CM_DPLL + 0x004) #define CLKSEL_TIMER2_CLK (CM_DPLL + 0x008) #define CLKSEL_TIMER3_CLK (CM_DPLL + 0x00C) #define CLKSEL_TIMER4_CLK (CM_DPLL + 0x010) #define CLKSEL_TIMER5_CLK (CM_DPLL + 0x018) #define CLKSEL_TIMER6_CLK (CM_DPLL + 0x01C) #define CLKSEL_PRUSS_OCP_CLK (CM_DPLL + 0x030) #define CM_RTC 0x800 #define CM_RTC_RTC_CLKCTRL (CM_RTC + 0x000) #define CM_RTC_CLKSTCTRL (CM_RTC + 0x004) #define PRM_PER 0xC00 #define PRM_PER_RSTCTRL (PRM_PER + 0x00) #define PRM_DEVICE_OFFSET 0xF00 #define PRM_RSTCTRL (PRM_DEVICE_OFFSET + 0x00) struct am335x_prcm_softc { struct resource * res[2]; bus_space_tag_t bst; bus_space_handle_t bsh; }; static struct resource_spec am335x_prcm_spec[] = { { SYS_RES_MEMORY, 0, RF_ACTIVE }, { -1, 0 } }; static struct am335x_prcm_softc *am335x_prcm_sc = NULL; static int am335x_clk_noop_activate(struct ti_clock_dev *clkdev); static int am335x_clk_generic_activate(struct ti_clock_dev *clkdev); static int am335x_clk_gpio_activate(struct ti_clock_dev *clkdev); static int am335x_clk_noop_deactivate(struct ti_clock_dev *clkdev); static int am335x_clk_generic_deactivate(struct ti_clock_dev *clkdev); static int am335x_clk_noop_set_source(struct ti_clock_dev *clkdev, clk_src_t clksrc); static int am335x_clk_generic_set_source(struct ti_clock_dev *clkdev, clk_src_t clksrc); static int am335x_clk_hsmmc_get_source_freq(struct ti_clock_dev *clkdev, unsigned int *freq); static int am335x_clk_get_sysclk_freq(struct ti_clock_dev *clkdev, unsigned int *freq); static int am335x_clk_get_arm_fclk_freq(struct ti_clock_dev *clkdev, unsigned int *freq); static int am335x_clk_get_arm_disp_freq(struct ti_clock_dev *clkdev, unsigned int *freq); static int am335x_clk_set_arm_disp_freq(struct ti_clock_dev *clkdev, unsigned int freq); static void am335x_prcm_reset(void); static int am335x_clk_cpsw_activate(struct ti_clock_dev *clkdev); static int am335x_clk_musb0_activate(struct ti_clock_dev *clkdev); static int am335x_clk_lcdc_activate(struct ti_clock_dev *clkdev); static int am335x_clk_pruss_activate(struct ti_clock_dev *clkdev); #define AM335X_NOOP_CLOCK_DEV(i) \ { .id = (i), \ .clk_activate = am335x_clk_noop_activate, \ .clk_deactivate = am335x_clk_noop_deactivate, \ .clk_set_source = am335x_clk_noop_set_source, \ .clk_accessible = NULL, \ .clk_get_source_freq = NULL, \ .clk_set_source_freq = NULL \ } #define AM335X_GENERIC_CLOCK_DEV(i) \ { .id = (i), \ .clk_activate = am335x_clk_generic_activate, \ .clk_deactivate = am335x_clk_generic_deactivate, \ .clk_set_source = am335x_clk_generic_set_source, \ .clk_accessible = NULL, \ .clk_get_source_freq = NULL, \ .clk_set_source_freq = NULL \ } #define AM335X_GPIO_CLOCK_DEV(i) \ { .id = (i), \ .clk_activate = am335x_clk_gpio_activate, \ .clk_deactivate = am335x_clk_generic_deactivate, \ .clk_set_source = am335x_clk_generic_set_source, \ .clk_accessible = NULL, \ .clk_get_source_freq = NULL, \ .clk_set_source_freq = NULL \ } #define AM335X_MMCHS_CLOCK_DEV(i) \ { .id = (i), \ .clk_activate = am335x_clk_generic_activate, \ .clk_deactivate = am335x_clk_generic_deactivate, \ .clk_set_source = am335x_clk_generic_set_source, \ .clk_accessible = NULL, \ .clk_get_source_freq = am335x_clk_hsmmc_get_source_freq, \ .clk_set_source_freq = NULL \ } struct ti_clock_dev ti_am335x_clk_devmap[] = { /* System clocks */ { .id = SYS_CLK, .clk_activate = NULL, .clk_deactivate = NULL, .clk_set_source = NULL, .clk_accessible = NULL, .clk_get_source_freq = am335x_clk_get_sysclk_freq, .clk_set_source_freq = NULL, }, /* MPU (ARM) core clocks */ { .id = MPU_CLK, .clk_activate = NULL, .clk_deactivate = NULL, .clk_set_source = NULL, .clk_accessible = NULL, .clk_get_source_freq = am335x_clk_get_arm_fclk_freq, .clk_set_source_freq = NULL, }, /* CPSW Ethernet Switch core clocks */ { .id = CPSW_CLK, .clk_activate = am335x_clk_cpsw_activate, .clk_deactivate = NULL, .clk_set_source = NULL, .clk_accessible = NULL, .clk_get_source_freq = NULL, .clk_set_source_freq = NULL, }, /* Mentor USB HS controller core clocks */ { .id = MUSB0_CLK, .clk_activate = am335x_clk_musb0_activate, .clk_deactivate = NULL, .clk_set_source = NULL, .clk_accessible = NULL, .clk_get_source_freq = NULL, .clk_set_source_freq = NULL, }, /* LCD controller clocks */ { .id = LCDC_CLK, .clk_activate = am335x_clk_lcdc_activate, .clk_deactivate = NULL, .clk_set_source = NULL, .clk_accessible = NULL, .clk_get_source_freq = am335x_clk_get_arm_disp_freq, .clk_set_source_freq = am335x_clk_set_arm_disp_freq, }, /* UART */ AM335X_NOOP_CLOCK_DEV(UART1_CLK), AM335X_GENERIC_CLOCK_DEV(UART2_CLK), AM335X_GENERIC_CLOCK_DEV(UART3_CLK), AM335X_GENERIC_CLOCK_DEV(UART4_CLK), AM335X_GENERIC_CLOCK_DEV(UART5_CLK), AM335X_GENERIC_CLOCK_DEV(UART6_CLK), /* DMTimer */ AM335X_GENERIC_CLOCK_DEV(TIMER2_CLK), AM335X_GENERIC_CLOCK_DEV(TIMER3_CLK), AM335X_GENERIC_CLOCK_DEV(TIMER4_CLK), AM335X_GENERIC_CLOCK_DEV(TIMER5_CLK), AM335X_GENERIC_CLOCK_DEV(TIMER6_CLK), AM335X_GENERIC_CLOCK_DEV(TIMER7_CLK), /* GPIO, we use hwmods as reference, not units in spec */ AM335X_GPIO_CLOCK_DEV(GPIO1_CLK), AM335X_GPIO_CLOCK_DEV(GPIO2_CLK), AM335X_GPIO_CLOCK_DEV(GPIO3_CLK), AM335X_GPIO_CLOCK_DEV(GPIO4_CLK), /* I2C we use hwmods as reference, not units in spec */ AM335X_GENERIC_CLOCK_DEV(I2C1_CLK), AM335X_GENERIC_CLOCK_DEV(I2C2_CLK), AM335X_GENERIC_CLOCK_DEV(I2C3_CLK), /* McSPI we use hwmods as reference, not units in spec */ AM335X_GENERIC_CLOCK_DEV(SPI0_CLK), AM335X_GENERIC_CLOCK_DEV(SPI1_CLK), /* TSC_ADC */ AM335X_GENERIC_CLOCK_DEV(TSC_ADC_CLK), /* EDMA */ AM335X_GENERIC_CLOCK_DEV(EDMA_TPCC_CLK), AM335X_GENERIC_CLOCK_DEV(EDMA_TPTC0_CLK), AM335X_GENERIC_CLOCK_DEV(EDMA_TPTC1_CLK), AM335X_GENERIC_CLOCK_DEV(EDMA_TPTC2_CLK), /* MMCHS */ AM335X_MMCHS_CLOCK_DEV(MMC1_CLK), AM335X_MMCHS_CLOCK_DEV(MMC2_CLK), AM335X_MMCHS_CLOCK_DEV(MMC3_CLK), /* PWMSS */ AM335X_GENERIC_CLOCK_DEV(PWMSS0_CLK), AM335X_GENERIC_CLOCK_DEV(PWMSS1_CLK), AM335X_GENERIC_CLOCK_DEV(PWMSS2_CLK), /* System Mailbox clock */ AM335X_GENERIC_CLOCK_DEV(MAILBOX0_CLK), /* SPINLOCK */ AM335X_GENERIC_CLOCK_DEV(SPINLOCK0_CLK), /* PRU-ICSS */ { .id = PRUSS_CLK, .clk_activate = am335x_clk_pruss_activate, .clk_deactivate = NULL, .clk_set_source = NULL, .clk_accessible = NULL, .clk_get_source_freq = NULL, .clk_set_source_freq = NULL, }, /* RTC */ AM335X_GENERIC_CLOCK_DEV(RTC_CLK), { INVALID_CLK_IDENT, NULL, NULL, NULL, NULL } }; struct am335x_clk_details { clk_ident_t id; uint32_t clkctrl_reg; uint32_t clksel_reg; }; #define _CLK_DETAIL(i, c, s) \ { .id = (i), \ .clkctrl_reg = (c), \ .clksel_reg = (s), \ } static struct am335x_clk_details g_am335x_clk_details[] = { /* UART. UART0 clock not controllable. */ _CLK_DETAIL(UART1_CLK, 0, 0), _CLK_DETAIL(UART2_CLK, CM_PER_UART1_CLKCTRL, 0), _CLK_DETAIL(UART3_CLK, CM_PER_UART2_CLKCTRL, 0), _CLK_DETAIL(UART4_CLK, CM_PER_UART3_CLKCTRL, 0), _CLK_DETAIL(UART5_CLK, CM_PER_UART4_CLKCTRL, 0), _CLK_DETAIL(UART6_CLK, CM_PER_UART5_CLKCTRL, 0), /* DMTimer modules */ _CLK_DETAIL(TIMER2_CLK, CM_PER_TIMER2_CLKCTRL, CLKSEL_TIMER2_CLK), _CLK_DETAIL(TIMER3_CLK, CM_PER_TIMER3_CLKCTRL, CLKSEL_TIMER3_CLK), _CLK_DETAIL(TIMER4_CLK, CM_PER_TIMER4_CLKCTRL, CLKSEL_TIMER4_CLK), _CLK_DETAIL(TIMER5_CLK, CM_PER_TIMER5_CLKCTRL, CLKSEL_TIMER5_CLK), _CLK_DETAIL(TIMER6_CLK, CM_PER_TIMER6_CLKCTRL, CLKSEL_TIMER6_CLK), _CLK_DETAIL(TIMER7_CLK, CM_PER_TIMER7_CLKCTRL, CLKSEL_TIMER7_CLK), /* GPIO modules, hwmods start with gpio1 */ _CLK_DETAIL(GPIO1_CLK, CM_WKUP_GPIO0_CLKCTRL, 0), _CLK_DETAIL(GPIO2_CLK, CM_PER_GPIO1_CLKCTRL, 0), _CLK_DETAIL(GPIO3_CLK, CM_PER_GPIO2_CLKCTRL, 0), _CLK_DETAIL(GPIO4_CLK, CM_PER_GPIO3_CLKCTRL, 0), /* I2C modules, hwmods start with i2c1 */ _CLK_DETAIL(I2C1_CLK, CM_WKUP_I2C0_CLKCTRL, 0), _CLK_DETAIL(I2C2_CLK, CM_PER_I2C1_CLKCTRL, 0), _CLK_DETAIL(I2C3_CLK, CM_PER_I2C2_CLKCTRL, 0), /* McSPI modules, hwmods start with spi0 */ _CLK_DETAIL(SPI0_CLK, CM_PER_SPI0_CLKCTRL, 0), _CLK_DETAIL(SPI1_CLK, CM_PER_SPI1_CLKCTRL, 0), /* TSC_ADC module */ _CLK_DETAIL(TSC_ADC_CLK, CM_WKUP_ADC_TSC_CLKCTRL, 0), /* EDMA modules */ _CLK_DETAIL(EDMA_TPCC_CLK, CM_PER_TPCC_CLKCTRL, 0), _CLK_DETAIL(EDMA_TPTC0_CLK, CM_PER_TPTC0_CLKCTRL, 0), _CLK_DETAIL(EDMA_TPTC1_CLK, CM_PER_TPTC1_CLKCTRL, 0), _CLK_DETAIL(EDMA_TPTC2_CLK, CM_PER_TPTC2_CLKCTRL, 0), /* MMCHS modules, hwmods start with mmc1*/ _CLK_DETAIL(MMC1_CLK, CM_PER_MMC0_CLKCTRL, 0), _CLK_DETAIL(MMC2_CLK, CM_PER_MMC1_CLKCTRL, 0), _CLK_DETAIL(MMC3_CLK, CM_PER_MMC1_CLKCTRL, 0), /* PWMSS modules */ _CLK_DETAIL(PWMSS0_CLK, CM_PER_EPWMSS0_CLKCTRL, 0), _CLK_DETAIL(PWMSS1_CLK, CM_PER_EPWMSS1_CLKCTRL, 0), _CLK_DETAIL(PWMSS2_CLK, CM_PER_EPWMSS2_CLKCTRL, 0), _CLK_DETAIL(MAILBOX0_CLK, CM_PER_MAILBOX0_CLKCTRL, 0), _CLK_DETAIL(SPINLOCK0_CLK, CM_PER_SPINLOCK0_CLKCTRL, 0), /* RTC module */ _CLK_DETAIL(RTC_CLK, CM_RTC_RTC_CLKCTRL, 0), { INVALID_CLK_IDENT, 0}, }; /* Read/Write macros */ #define prcm_read_4(reg) \ bus_space_read_4(am335x_prcm_sc->bst, am335x_prcm_sc->bsh, reg) #define prcm_write_4(reg, val) \ bus_space_write_4(am335x_prcm_sc->bst, am335x_prcm_sc->bsh, reg, val) void am335x_prcm_setup_dmtimer(int); static int am335x_prcm_probe(device_t dev) { if (!ofw_bus_status_okay(dev)) return (ENXIO); if (ofw_bus_is_compatible(dev, "ti,am3-prcm")) { device_set_desc(dev, "AM335x Power and Clock Management"); return(BUS_PROBE_DEFAULT); } return (ENXIO); } static int am335x_prcm_attach(device_t dev) { struct am335x_prcm_softc *sc = device_get_softc(dev); unsigned int sysclk, fclk; if (am335x_prcm_sc) return (ENXIO); if (bus_alloc_resources(dev, am335x_prcm_spec, sc->res)) { device_printf(dev, "could not allocate resources\n"); return (ENXIO); } sc->bst = rman_get_bustag(sc->res[0]); sc->bsh = rman_get_bushandle(sc->res[0]); am335x_prcm_sc = sc; ti_cpu_reset = am335x_prcm_reset; if (am335x_clk_get_sysclk_freq(NULL, &sysclk) != 0) sysclk = 0; if (am335x_clk_get_arm_fclk_freq(NULL, &fclk) != 0) fclk = 0; if (sysclk && fclk) device_printf(dev, "Clocks: System %u.%01u MHz, CPU %u MHz\n", sysclk/1000000, (sysclk % 1000000)/100000, fclk/1000000); else device_printf(dev, "can't read frequencies yet (SCM device not ready?)\n"); return (0); } static device_method_t am335x_prcm_methods[] = { DEVMETHOD(device_probe, am335x_prcm_probe), DEVMETHOD(device_attach, am335x_prcm_attach), { 0, 0 } }; static driver_t am335x_prcm_driver = { "am335x_prcm", am335x_prcm_methods, sizeof(struct am335x_prcm_softc), }; static devclass_t am335x_prcm_devclass; DRIVER_MODULE(am335x_prcm, simplebus, am335x_prcm_driver, am335x_prcm_devclass, 0, 0); MODULE_VERSION(am335x_prcm, 1); MODULE_DEPEND(am335x_prcm, ti_scm, 1, 1, 1); static struct am335x_clk_details* am335x_clk_details(clk_ident_t id) { struct am335x_clk_details *walker; for (walker = g_am335x_clk_details; walker->id != INVALID_CLK_IDENT; walker++) { if (id == walker->id) return (walker); } return NULL; } static int am335x_clk_noop_activate(struct ti_clock_dev *clkdev) { return (0); } static int am335x_clk_generic_activate(struct ti_clock_dev *clkdev) { struct am335x_prcm_softc *sc = am335x_prcm_sc; struct am335x_clk_details* clk_details; if (sc == NULL) return ENXIO; clk_details = am335x_clk_details(clkdev->id); if (clk_details == NULL) return (ENXIO); /* set *_CLKCTRL register MODULEMODE[1:0] to enable(2) */ prcm_write_4(clk_details->clkctrl_reg, 2); while ((prcm_read_4(clk_details->clkctrl_reg) & 0x3) != 2) DELAY(10); return (0); } static int am335x_clk_gpio_activate(struct ti_clock_dev *clkdev) { struct am335x_prcm_softc *sc = am335x_prcm_sc; struct am335x_clk_details* clk_details; if (sc == NULL) return ENXIO; clk_details = am335x_clk_details(clkdev->id); if (clk_details == NULL) return (ENXIO); /* set *_CLKCTRL register MODULEMODE[1:0] to enable(2) */ /* set *_CLKCTRL register OPTFCLKEN_GPIO_1_G DBCLK[18] to FCLK_EN(1) */ prcm_write_4(clk_details->clkctrl_reg, 2 | (1 << 18)); while ((prcm_read_4(clk_details->clkctrl_reg) & (3 | (1 << 18) )) != (2 | (1 << 18))) DELAY(10); return (0); } static int am335x_clk_noop_deactivate(struct ti_clock_dev *clkdev) { return(0); } static int am335x_clk_generic_deactivate(struct ti_clock_dev *clkdev) { struct am335x_prcm_softc *sc = am335x_prcm_sc; struct am335x_clk_details* clk_details; if (sc == NULL) return ENXIO; clk_details = am335x_clk_details(clkdev->id); if (clk_details == NULL) return (ENXIO); /* set *_CLKCTRL register MODULEMODE[1:0] to disable(0) */ prcm_write_4(clk_details->clkctrl_reg, 0); while ((prcm_read_4(clk_details->clkctrl_reg) & 0x3) != 0) DELAY(10); return (0); } static int am335x_clk_noop_set_source(struct ti_clock_dev *clkdev, clk_src_t clksrc) { return (0); } static int am335x_clk_generic_set_source(struct ti_clock_dev *clkdev, clk_src_t clksrc) { struct am335x_prcm_softc *sc = am335x_prcm_sc; struct am335x_clk_details* clk_details; uint32_t reg; if (sc == NULL) return ENXIO; clk_details = am335x_clk_details(clkdev->id); if (clk_details == NULL) return (ENXIO); switch (clksrc) { case EXT_CLK: reg = 0; /* SEL2: TCLKIN clock */ break; case SYSCLK_CLK: reg = 1; /* SEL1: CLK_M_OSC clock */ break; case F32KHZ_CLK: reg = 2; /* SEL3: CLK_32KHZ clock */ break; default: return (ENXIO); } prcm_write_4(clk_details->clksel_reg, reg); while ((prcm_read_4(clk_details->clksel_reg) & 0x3) != reg) DELAY(10); return (0); } static int am335x_clk_hsmmc_get_source_freq(struct ti_clock_dev *clkdev, unsigned int *freq) { *freq = 96000000; return (0); } static int am335x_clk_get_sysclk_freq(struct ti_clock_dev *clkdev, unsigned int *freq) { uint32_t ctrl_status; - /* Read the input clock freq from the control module */ - /* control_status reg (0x40) */ - if (ti_scm_reg_read_4(0x40, &ctrl_status)) - return ENXIO; + /* Read the input clock freq from the control module. */ + if (ti_scm_reg_read_4(SCM_CTRL_STATUS, &ctrl_status)) + return (ENXIO); switch ((ctrl_status>>22) & 0x3) { case 0x0: /* 19.2Mhz */ *freq = 19200000; break; case 0x1: /* 24Mhz */ *freq = 24000000; break; case 0x2: /* 25Mhz */ *freq = 25000000; break; case 0x3: /* 26Mhz */ *freq = 26000000; break; } return (0); } #define DPLL_BYP_CLKSEL(reg) ((reg>>23) & 1) #define DPLL_DIV(reg) ((reg & 0x7f)+1) #define DPLL_MULT(reg) ((reg>>8) & 0x7FF) #define DPLL_MAX_MUL 0x800 #define DPLL_MAX_DIV 0x80 static int am335x_clk_get_arm_fclk_freq(struct ti_clock_dev *clkdev, unsigned int *freq) { uint32_t reg; uint32_t sysclk; reg = prcm_read_4(CM_WKUP_CM_CLKSEL_DPLL_MPU); /*Check if we are running in bypass */ if (DPLL_BYP_CLKSEL(reg)) return ENXIO; am335x_clk_get_sysclk_freq(NULL, &sysclk); *freq = DPLL_MULT(reg) * (sysclk / DPLL_DIV(reg)); return(0); } static int am335x_clk_get_arm_disp_freq(struct ti_clock_dev *clkdev, unsigned int *freq) { uint32_t reg; uint32_t sysclk; reg = prcm_read_4(CM_WKUP_CM_CLKSEL_DPLL_DISP); /*Check if we are running in bypass */ if (DPLL_BYP_CLKSEL(reg)) return ENXIO; am335x_clk_get_sysclk_freq(NULL, &sysclk); *freq = DPLL_MULT(reg) * (sysclk / DPLL_DIV(reg)); return(0); } static int am335x_clk_set_arm_disp_freq(struct ti_clock_dev *clkdev, unsigned int freq) { uint32_t sysclk; uint32_t mul, div; uint32_t i, j; unsigned int delta, min_delta; am335x_clk_get_sysclk_freq(NULL, &sysclk); /* Bypass mode */ prcm_write_4(CM_WKUP_CM_CLKMODE_DPLL_DISP, 0x4); /* Make sure it's in bypass mode */ while (!(prcm_read_4(CM_WKUP_CM_IDLEST_DPLL_DISP) & (1 << 8))) DELAY(10); /* Dumb and non-optimal implementation */ min_delta = freq; for (i = 1; i < DPLL_MAX_MUL; i++) { for (j = 1; j < DPLL_MAX_DIV; j++) { delta = abs(freq - i*(sysclk/j)); if (delta < min_delta) { mul = i; div = j; min_delta = delta; } if (min_delta == 0) break; } } prcm_write_4(CM_WKUP_CM_CLKSEL_DPLL_DISP, (mul << 8) | (div - 1)); /* Locked mode */ prcm_write_4(CM_WKUP_CM_CLKMODE_DPLL_DISP, 0x7); int timeout = 10000; while ((!(prcm_read_4(CM_WKUP_CM_IDLEST_DPLL_DISP) & (1 << 0))) && timeout--) DELAY(10); return(0); } static void am335x_prcm_reset(void) { prcm_write_4(PRM_RSTCTRL, (1<<1)); } static int am335x_clk_cpsw_activate(struct ti_clock_dev *clkdev) { struct am335x_prcm_softc *sc = am335x_prcm_sc; if (sc == NULL) return ENXIO; /* set MODULENAME to ENABLE */ prcm_write_4(CM_PER_CPGMAC0_CLKCTRL, 2); /* wait for IDLEST to become Func(0) */ while(prcm_read_4(CM_PER_CPGMAC0_CLKCTRL) & (3<<16)); /*set CLKTRCTRL to SW_WKUP(2) */ prcm_write_4(CM_PER_CPSW_CLKSTCTRL, 2); /* wait for 125 MHz OCP clock to become active */ while((prcm_read_4(CM_PER_CPSW_CLKSTCTRL) & (1<<4)) == 0); return(0); } static int am335x_clk_musb0_activate(struct ti_clock_dev *clkdev) { struct am335x_prcm_softc *sc = am335x_prcm_sc; if (sc == NULL) return ENXIO; /* set ST_DPLL_CLKDCOLDO(9) to CLK_GATED(1) */ /* set DPLL_CLKDCOLDO_GATE_CTRL(8) to CLK_ENABLE(1)*/ prcm_write_4(CM_WKUP_CM_CLKDCOLDO_DPLL_PER, 0x300); /*set MODULEMODE to ENABLE(2) */ prcm_write_4(CM_PER_USB0_CLKCTRL, 2); /* wait for MODULEMODE to become ENABLE(2) */ while ((prcm_read_4(CM_PER_USB0_CLKCTRL) & 0x3) != 2) DELAY(10); /* wait for IDLEST to become Func(0) */ while(prcm_read_4(CM_PER_USB0_CLKCTRL) & (3<<16)) DELAY(10); return(0); } static int am335x_clk_lcdc_activate(struct ti_clock_dev *clkdev) { struct am335x_prcm_softc *sc = am335x_prcm_sc; if (sc == NULL) return (ENXIO); /* * For now set frequency to 2*VGA_PIXEL_CLOCK */ am335x_clk_set_arm_disp_freq(clkdev, 25175000*2); /*set MODULEMODE to ENABLE(2) */ prcm_write_4(CM_PER_LCDC_CLKCTRL, 2); /* wait for MODULEMODE to become ENABLE(2) */ while ((prcm_read_4(CM_PER_LCDC_CLKCTRL) & 0x3) != 2) DELAY(10); /* wait for IDLEST to become Func(0) */ while(prcm_read_4(CM_PER_LCDC_CLKCTRL) & (3<<16)) DELAY(10); return (0); } static int am335x_clk_pruss_activate(struct ti_clock_dev *clkdev) { struct am335x_prcm_softc *sc = am335x_prcm_sc; if (sc == NULL) return (ENXIO); /* Set MODULEMODE to ENABLE(2) */ prcm_write_4(CM_PER_PRUSS_CLKCTRL, 2); /* Wait for MODULEMODE to become ENABLE(2) */ while ((prcm_read_4(CM_PER_PRUSS_CLKCTRL) & 0x3) != 2) DELAY(10); /* Set CLKTRCTRL to SW_WKUP(2) */ prcm_write_4(CM_PER_PRUSS_CLKSTCTRL, 2); /* Wait for the 200 MHz OCP clock to become active */ while ((prcm_read_4(CM_PER_PRUSS_CLKSTCTRL) & (1<<4)) == 0) DELAY(10); /* Wait for the 200 MHz IEP clock to become active */ while ((prcm_read_4(CM_PER_PRUSS_CLKSTCTRL) & (1<<5)) == 0) DELAY(10); /* Wait for the 192 MHz UART clock to become active */ while ((prcm_read_4(CM_PER_PRUSS_CLKSTCTRL) & (1<<6)) == 0) DELAY(10); /* Select L3F as OCP clock */ prcm_write_4(CLKSEL_PRUSS_OCP_CLK, 0); while ((prcm_read_4(CLKSEL_PRUSS_OCP_CLK) & 0x3) != 0) DELAY(10); /* Clear the RESET bit */ prcm_write_4(PRM_PER_RSTCTRL, prcm_read_4(PRM_PER_RSTCTRL) & ~2); return (0); } Index: projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_scm.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_scm.c (nonexistent) +++ projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_scm.c (revision 305172) @@ -0,0 +1,169 @@ +/*- + * Copyright (c) 2016 Rubicon Communications, LLC (Netgate) + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include + +#define TZ_ZEROC 2731 + +struct am335x_scm_softc { + int sc_last_temp; + struct sysctl_oid *sc_temp_oid; +}; + +static int +am335x_scm_temp_sysctl(SYSCTL_HANDLER_ARGS) +{ + device_t dev; + int i, temp; + struct am335x_scm_softc *sc; + uint32_t reg; + + dev = (device_t)arg1; + sc = device_get_softc(dev); + + /* Read the temperature and convert to Kelvin. */ + for(i = 50; i > 0; i--) { + ti_scm_reg_read_4(SCM_BGAP_CTRL, ®); + if ((reg & SCM_BGAP_EOCZ) == 0) + break; + DELAY(50); + } + if ((reg & SCM_BGAP_EOCZ) == 0) { + sc->sc_last_temp = + (reg >> SCM_BGAP_TEMP_SHIFT) & SCM_BGAP_TEMP_MASK; + sc->sc_last_temp *= 10; + } + temp = sc->sc_last_temp + TZ_ZEROC; + + return (sysctl_handle_int(oidp, &temp, 0, req)); +} + +static void +am335x_scm_identify(driver_t *driver, device_t parent) +{ + device_t child; + + /* AM335x only. */ + if (ti_chip() != CHIP_AM335X) + return; + + /* Make sure we attach only once. */ + if (device_find_child(parent, "am335x_scm", -1) != NULL) + return; + + child = device_add_child(parent, "am335x_scm", -1); + if (child == NULL) + device_printf(parent, "cannot add ti_scm child\n"); +} + +static int +am335x_scm_probe(device_t dev) +{ + + device_set_desc(dev, "AM335x Control Module Extension"); + + return (BUS_PROBE_DEFAULT); +} + +static int +am335x_scm_attach(device_t dev) +{ + struct am335x_scm_softc *sc; + struct sysctl_ctx_list *ctx; + struct sysctl_oid_list *tree; + uint32_t reg; + + /* Set ADC to continous mode, clear output reset. */ + reg = SCM_BGAP_CLRZ | SCM_BGAP_CONTCONV; + ti_scm_reg_write_4(SCM_BGAP_CTRL, reg); + /* Flush write. */ + ti_scm_reg_read_4(SCM_BGAP_CTRL, ®); + /* Start the ADC conversion. */ + reg = SCM_BGAP_CLRZ | SCM_BGAP_CONTCONV | SCM_BGAP_SOC; + ti_scm_reg_write_4(SCM_BGAP_CTRL, reg); + + /* Temperature sysctl. */ + sc = device_get_softc(dev); + ctx = device_get_sysctl_ctx(dev); + tree = SYSCTL_CHILDREN(device_get_sysctl_tree(dev)); + sc->sc_temp_oid = SYSCTL_ADD_PROC(ctx, tree, OID_AUTO, + "temperature", CTLTYPE_INT | CTLFLAG_RD | CTLFLAG_MPSAFE, + dev, 0, am335x_scm_temp_sysctl, "IK", "Current temperature"); + + return (0); +} + +static int +am335x_scm_detach(device_t dev) +{ + struct am335x_scm_softc *sc; + + sc = device_get_softc(dev); + + /* Remove temperature sysctl. */ + if (sc->sc_temp_oid != NULL) + sysctl_remove_oid(sc->sc_temp_oid, 1, 0); + + /* Stop the bandgap ADC. */ + ti_scm_reg_write_4(SCM_BGAP_CTRL, SCM_BGAP_BGOFF); + + return (0); +} + +static device_method_t am335x_scm_methods[] = { + DEVMETHOD(device_identify, am335x_scm_identify), + DEVMETHOD(device_probe, am335x_scm_probe), + DEVMETHOD(device_attach, am335x_scm_attach), + DEVMETHOD(device_detach, am335x_scm_detach), + + DEVMETHOD_END +}; + +static driver_t am335x_scm_driver = { + "am335x_scm", + am335x_scm_methods, + sizeof(struct am335x_scm_softc), +}; + +static devclass_t am335x_scm_devclass; + +DRIVER_MODULE(am335x_scm, ti_scm, am335x_scm_driver, am335x_scm_devclass, 0, 0); +MODULE_VERSION(am335x_scm, 1); +MODULE_DEPEND(am335x_scm, ti_scm, 1, 1, 1); Property changes on: projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_scm.c ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_scm.h =================================================================== --- projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_scm.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/ti/am335x/am335x_scm.h (revision 305172) @@ -1,38 +1,47 @@ /*- * Copyright (c) 2012 Oleksandr Tymoshenko * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef __AM335X_SCM_H__ #define __AM335X_SCM_H__ /* AM335x-specific registers for control module (scm) */ +#define SCM_CTRL_STATUS 0x40 +#define SCM_BGAP_CTRL 0x448 +#define SCM_BGAP_TEMP_MASK 0xff +#define SCM_BGAP_TEMP_SHIFT 8 +#define SCM_BGAP_BGOFF (1 << 6) +#define SCM_BGAP_SOC (1 << 4) +#define SCM_BGAP_CLRZ (1 << 3) +#define SCM_BGAP_CONTCONV (1 << 2) +#define SCM_BGAP_EOCZ (1 << 1) #define SCM_USB_CTRL0 0x620 #define SCM_USB_STS0 0x624 #define SCM_USB_CTRL1 0x628 #define SCM_USB_STS1 0x62C #define SCM_PWMSS_CTRL 0x664 #endif /* __AM335X_SCM_H__ */ Index: projects/netbsd-tests-update-12/sys/arm/ti/am335x/files.am335x =================================================================== --- projects/netbsd-tests-update-12/sys/arm/ti/am335x/files.am335x (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/ti/am335x/files.am335x (revision 305172) @@ -1,23 +1,24 @@ #$FreeBSD$ arm/ti/aintc.c standard arm/ti/am335x/am335x_dmtimer.c standard arm/ti/am335x/am335x_dmtpps.c optional am335x_dmtpps arm/ti/am335x/am335x_gpio.c optional gpio arm/ti/am335x/am335x_lcd.c optional sc | vt arm/ti/am335x/am335x_lcd_syscons.c optional sc arm/ti/am335x/am335x_pmic.c optional am335x_pmic arm/ti/am335x/am335x_prcm.c standard arm/ti/am335x/am335x_pwmss.c standard arm/ti/am335x/am335x_ehrpwm.c standard arm/ti/am335x/am335x_ecap.c standard arm/ti/am335x/am335x_rtc.c optional am335x_rtc +arm/ti/am335x/am335x_scm.c standard arm/ti/am335x/am335x_scm_padconf.c standard arm/ti/am335x/am335x_usbss.c optional musb fdt arm/ti/am335x/am335x_musb.c optional musb fdt arm/ti/am335x/tda19988.c optional hdmi arm/ti/ti_edma3.c standard arm/ti/cpsw/if_cpsw.c optional cpsw Index: projects/netbsd-tests-update-12/sys/arm/ti/cpsw/if_cpsw.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/ti/cpsw/if_cpsw.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/ti/cpsw/if_cpsw.c (revision 305172) @@ -1,2620 +1,2620 @@ /*- * Copyright (c) 2012 Damjan Marion * Copyright (c) 2016 Rubicon Communications, LLC (Netgate) * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * TI Common Platform Ethernet Switch (CPSW) Driver * Found in TI8148 "DaVinci" and AM335x "Sitara" SoCs. * * This controller is documented in the AM335x Technical Reference * Manual, in the TMS320DM814x DaVinci Digital Video Processors TRM * and in the TMS320C6452 3 Port Switch Ethernet Subsystem TRM. * * It is basically a single Ethernet port (port 0) wired internally to * a 3-port store-and-forward switch connected to two independent * "sliver" controllers (port 1 and port 2). You can operate the * controller in a variety of different ways by suitably configuring * the slivers and the Address Lookup Engine (ALE) that routes packets * between the ports. * * This code was developed and tested on a BeagleBone with * an AM335x SoC. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "if_cpswreg.h" #include "if_cpswvar.h" #include #include "miibus_if.h" /* Device probe/attach/detach. */ static int cpsw_probe(device_t); static int cpsw_attach(device_t); static int cpsw_detach(device_t); static int cpswp_probe(device_t); static int cpswp_attach(device_t); static int cpswp_detach(device_t); static phandle_t cpsw_get_node(device_t, device_t); /* Device Init/shutdown. */ static int cpsw_shutdown(device_t); static void cpswp_init(void *); static void cpswp_init_locked(void *); static void cpswp_stop_locked(struct cpswp_softc *); /* Device Suspend/Resume. */ static int cpsw_suspend(device_t); static int cpsw_resume(device_t); /* Ioctl. */ static int cpswp_ioctl(struct ifnet *, u_long command, caddr_t data); static int cpswp_miibus_readreg(device_t, int phy, int reg); static int cpswp_miibus_writereg(device_t, int phy, int reg, int value); static void cpswp_miibus_statchg(device_t); /* Send/Receive packets. */ static void cpsw_intr_rx(void *arg); static struct mbuf *cpsw_rx_dequeue(struct cpsw_softc *); static void cpsw_rx_enqueue(struct cpsw_softc *); static void cpswp_start(struct ifnet *); static void cpswp_tx_enqueue(struct cpswp_softc *); static int cpsw_tx_dequeue(struct cpsw_softc *); /* Misc interrupts and watchdog. */ static void cpsw_intr_rx_thresh(void *); static void cpsw_intr_misc(void *); static void cpswp_tick(void *); static void cpswp_ifmedia_sts(struct ifnet *, struct ifmediareq *); static int cpswp_ifmedia_upd(struct ifnet *); static void cpsw_tx_watchdog(void *); /* ALE support */ static void cpsw_ale_read_entry(struct cpsw_softc *, uint16_t, uint32_t *); static void cpsw_ale_write_entry(struct cpsw_softc *, uint16_t, uint32_t *); static int cpsw_ale_mc_entry_set(struct cpsw_softc *, uint8_t, int, uint8_t *); static void cpsw_ale_dump_table(struct cpsw_softc *); static int cpsw_ale_update_vlan_table(struct cpsw_softc *, int, int, int, int, int); static int cpswp_ale_update_addresses(struct cpswp_softc *, int); /* Statistics and sysctls. */ static void cpsw_add_sysctls(struct cpsw_softc *); static void cpsw_stats_collect(struct cpsw_softc *); static int cpsw_stats_sysctl(SYSCTL_HANDLER_ARGS); /* * Arbitrary limit on number of segments in an mbuf to be transmitted. * Packets with more segments than this will be defragmented before * they are queued. */ #define CPSW_TXFRAGS 16 /* Shared resources. */ static device_method_t cpsw_methods[] = { /* Device interface */ DEVMETHOD(device_probe, cpsw_probe), DEVMETHOD(device_attach, cpsw_attach), DEVMETHOD(device_detach, cpsw_detach), DEVMETHOD(device_shutdown, cpsw_shutdown), DEVMETHOD(device_suspend, cpsw_suspend), DEVMETHOD(device_resume, cpsw_resume), /* OFW methods */ DEVMETHOD(ofw_bus_get_node, cpsw_get_node), DEVMETHOD_END }; static driver_t cpsw_driver = { "cpswss", cpsw_methods, sizeof(struct cpsw_softc), }; static devclass_t cpsw_devclass; DRIVER_MODULE(cpswss, simplebus, cpsw_driver, cpsw_devclass, 0, 0); /* Port/Slave resources. */ static device_method_t cpswp_methods[] = { /* Device interface */ DEVMETHOD(device_probe, cpswp_probe), DEVMETHOD(device_attach, cpswp_attach), DEVMETHOD(device_detach, cpswp_detach), /* MII interface */ DEVMETHOD(miibus_readreg, cpswp_miibus_readreg), DEVMETHOD(miibus_writereg, cpswp_miibus_writereg), DEVMETHOD(miibus_statchg, cpswp_miibus_statchg), DEVMETHOD_END }; static driver_t cpswp_driver = { "cpsw", cpswp_methods, sizeof(struct cpswp_softc), }; static devclass_t cpswp_devclass; DRIVER_MODULE(cpsw, cpswss, cpswp_driver, cpswp_devclass, 0, 0); DRIVER_MODULE(miibus, cpsw, miibus_driver, miibus_devclass, 0, 0); MODULE_DEPEND(cpsw, ether, 1, 1, 1); MODULE_DEPEND(cpsw, miibus, 1, 1, 1); static uint32_t slave_mdio_addr[] = { 0x4a100200, 0x4a100300 }; static struct resource_spec irq_res_spec[] = { { SYS_RES_IRQ, 0, RF_ACTIVE | RF_SHAREABLE }, { SYS_RES_IRQ, 1, RF_ACTIVE | RF_SHAREABLE }, { SYS_RES_IRQ, 2, RF_ACTIVE | RF_SHAREABLE }, { SYS_RES_IRQ, 3, RF_ACTIVE | RF_SHAREABLE }, { -1, 0 } }; /* Number of entries here must match size of stats * array in struct cpswp_softc. */ static struct cpsw_stat { int reg; char *oid; } cpsw_stat_sysctls[CPSW_SYSCTL_COUNT] = { {0x00, "GoodRxFrames"}, {0x04, "BroadcastRxFrames"}, {0x08, "MulticastRxFrames"}, {0x0C, "PauseRxFrames"}, {0x10, "RxCrcErrors"}, {0x14, "RxAlignErrors"}, {0x18, "OversizeRxFrames"}, {0x1c, "RxJabbers"}, {0x20, "ShortRxFrames"}, {0x24, "RxFragments"}, {0x30, "RxOctets"}, {0x34, "GoodTxFrames"}, {0x38, "BroadcastTxFrames"}, {0x3c, "MulticastTxFrames"}, {0x40, "PauseTxFrames"}, {0x44, "DeferredTxFrames"}, {0x48, "CollisionsTxFrames"}, {0x4c, "SingleCollisionTxFrames"}, {0x50, "MultipleCollisionTxFrames"}, {0x54, "ExcessiveCollisions"}, {0x58, "LateCollisions"}, {0x5c, "TxUnderrun"}, {0x60, "CarrierSenseErrors"}, {0x64, "TxOctets"}, {0x68, "RxTx64OctetFrames"}, {0x6c, "RxTx65to127OctetFrames"}, {0x70, "RxTx128to255OctetFrames"}, {0x74, "RxTx256to511OctetFrames"}, {0x78, "RxTx512to1024OctetFrames"}, {0x7c, "RxTx1024upOctetFrames"}, {0x80, "NetOctets"}, {0x84, "RxStartOfFrameOverruns"}, {0x88, "RxMiddleOfFrameOverruns"}, {0x8c, "RxDmaOverruns"} }; /* * Basic debug support. */ #define IF_DEBUG(_sc) if ((_sc)->if_flags & IFF_DEBUG) static void cpsw_debugf_head(const char *funcname) { int t = (int)(time_second % (24 * 60 * 60)); printf("%02d:%02d:%02d %s ", t / (60 * 60), (t / 60) % 60, t % 60, funcname); } #include static void cpsw_debugf(const char *fmt, ...) { va_list ap; va_start(ap, fmt); vprintf(fmt, ap); va_end(ap); printf("\n"); } #define CPSW_DEBUGF(_sc, a) do { \ if (sc->debug) { \ cpsw_debugf_head(__func__); \ cpsw_debugf a; \ } \ } while (0) #define CPSWP_DEBUGF(_sc, a) do { \ IF_DEBUG((_sc)) { \ cpsw_debugf_head(__func__); \ cpsw_debugf a; \ } \ } while (0) /* * Locking macros */ #define CPSW_TX_LOCK(sc) do { \ mtx_assert(&(sc)->rx.lock, MA_NOTOWNED); \ mtx_lock(&(sc)->tx.lock); \ } while (0) #define CPSW_TX_UNLOCK(sc) mtx_unlock(&(sc)->tx.lock) #define CPSW_TX_LOCK_ASSERT(sc) mtx_assert(&(sc)->tx.lock, MA_OWNED) #define CPSW_RX_LOCK(sc) do { \ mtx_assert(&(sc)->tx.lock, MA_NOTOWNED); \ mtx_lock(&(sc)->rx.lock); \ } while (0) #define CPSW_RX_UNLOCK(sc) mtx_unlock(&(sc)->rx.lock) #define CPSW_RX_LOCK_ASSERT(sc) mtx_assert(&(sc)->rx.lock, MA_OWNED) #define CPSW_GLOBAL_LOCK(sc) do { \ if ((mtx_owned(&(sc)->tx.lock) ? 1 : 0) != \ (mtx_owned(&(sc)->rx.lock) ? 1 : 0)) { \ panic("cpsw deadlock possibility detection!"); \ } \ mtx_lock(&(sc)->tx.lock); \ mtx_lock(&(sc)->rx.lock); \ } while (0) #define CPSW_GLOBAL_UNLOCK(sc) do { \ CPSW_RX_UNLOCK(sc); \ CPSW_TX_UNLOCK(sc); \ } while (0) #define CPSW_GLOBAL_LOCK_ASSERT(sc) do { \ CPSW_TX_LOCK_ASSERT(sc); \ CPSW_RX_LOCK_ASSERT(sc); \ } while (0) #define CPSW_PORT_LOCK(_sc) do { \ mtx_assert(&(_sc)->lock, MA_NOTOWNED); \ mtx_lock(&(_sc)->lock); \ } while (0) #define CPSW_PORT_UNLOCK(_sc) mtx_unlock(&(_sc)->lock) #define CPSW_PORT_LOCK_ASSERT(_sc) mtx_assert(&(_sc)->lock, MA_OWNED) /* * Read/Write macros */ #define cpsw_read_4(_sc, _reg) bus_read_4((_sc)->mem_res, (_reg)) #define cpsw_write_4(_sc, _reg, _val) \ bus_write_4((_sc)->mem_res, (_reg), (_val)) #define cpsw_cpdma_bd_offset(i) (CPSW_CPPI_RAM_OFFSET + ((i)*16)) #define cpsw_cpdma_bd_paddr(sc, slot) \ BUS_SPACE_PHYSADDR(sc->mem_res, slot->bd_offset) #define cpsw_cpdma_read_bd(sc, slot, val) \ bus_read_region_4(sc->mem_res, slot->bd_offset, (uint32_t *) val, 4) #define cpsw_cpdma_write_bd(sc, slot, val) \ bus_write_region_4(sc->mem_res, slot->bd_offset, (uint32_t *) val, 4) #define cpsw_cpdma_write_bd_next(sc, slot, next_slot) \ cpsw_write_4(sc, slot->bd_offset, cpsw_cpdma_bd_paddr(sc, next_slot)) #define cpsw_cpdma_read_bd_flags(sc, slot) \ bus_read_2(sc->mem_res, slot->bd_offset + 14) #define cpsw_write_hdp_slot(sc, queue, slot) \ cpsw_write_4(sc, (queue)->hdp_offset, cpsw_cpdma_bd_paddr(sc, slot)) #define CP_OFFSET (CPSW_CPDMA_TX_CP(0) - CPSW_CPDMA_TX_HDP(0)) #define cpsw_read_cp(sc, queue) \ cpsw_read_4(sc, (queue)->hdp_offset + CP_OFFSET) #define cpsw_write_cp(sc, queue, val) \ cpsw_write_4(sc, (queue)->hdp_offset + CP_OFFSET, (val)) #define cpsw_write_cp_slot(sc, queue, slot) \ cpsw_write_cp(sc, queue, cpsw_cpdma_bd_paddr(sc, slot)) #if 0 /* XXX temporary function versions for debugging. */ static void cpsw_write_hdp_slotX(struct cpsw_softc *sc, struct cpsw_queue *queue, struct cpsw_slot *slot) { uint32_t reg = queue->hdp_offset; uint32_t v = cpsw_cpdma_bd_paddr(sc, slot); CPSW_DEBUGF(("HDP <=== 0x%08x (was 0x%08x)", v, cpsw_read_4(sc, reg))); cpsw_write_4(sc, reg, v); } static void cpsw_write_cp_slotX(struct cpsw_softc *sc, struct cpsw_queue *queue, struct cpsw_slot *slot) { uint32_t v = cpsw_cpdma_bd_paddr(sc, slot); CPSW_DEBUGF(("CP <=== 0x%08x (expecting 0x%08x)", v, cpsw_read_cp(sc, queue))); cpsw_write_cp(sc, queue, v); } #endif /* * Expanded dump routines for verbose debugging. */ static void cpsw_dump_slot(struct cpsw_softc *sc, struct cpsw_slot *slot) { static const char *flags[] = {"SOP", "EOP", "Owner", "EOQ", "TDownCmplt", "PassCRC", "Long", "Short", "MacCtl", "Overrun", "PktErr1", "PortEn/PktErr0", "RxVlanEncap", "Port2", "Port1", "Port0"}; struct cpsw_cpdma_bd bd; const char *sep; int i; cpsw_cpdma_read_bd(sc, slot, &bd); printf("BD Addr: 0x%08x Next: 0x%08x\n", cpsw_cpdma_bd_paddr(sc, slot), bd.next); printf(" BufPtr: 0x%08x BufLen: 0x%08x\n", bd.bufptr, bd.buflen); printf(" BufOff: 0x%08x PktLen: 0x%08x\n", bd.bufoff, bd.pktlen); printf(" Flags: "); sep = ""; for (i = 0; i < 16; ++i) { if (bd.flags & (1 << (15 - i))) { printf("%s%s", sep, flags[i]); sep = ","; } } printf("\n"); if (slot->mbuf) { printf(" Ether: %14D\n", (char *)(slot->mbuf->m_data), " "); printf(" Packet: %16D\n", (char *)(slot->mbuf->m_data) + 14, " "); } } #define CPSW_DUMP_SLOT(cs, slot) do { \ IF_DEBUG(sc) { \ cpsw_dump_slot(sc, slot); \ } \ } while (0) static void cpsw_dump_queue(struct cpsw_softc *sc, struct cpsw_slots *q) { struct cpsw_slot *slot; int i = 0; int others = 0; STAILQ_FOREACH(slot, q, next) { if (i > 4) ++others; else cpsw_dump_slot(sc, slot); ++i; } if (others) printf(" ... and %d more.\n", others); printf("\n"); } #define CPSW_DUMP_QUEUE(sc, q) do { \ IF_DEBUG(sc) { \ cpsw_dump_queue(sc, q); \ } \ } while (0) static void cpsw_init_slots(struct cpsw_softc *sc) { struct cpsw_slot *slot; int i; STAILQ_INIT(&sc->avail); /* Put the slot descriptors onto the global avail list. */ for (i = 0; i < nitems(sc->_slots); i++) { slot = &sc->_slots[i]; slot->bd_offset = cpsw_cpdma_bd_offset(i); STAILQ_INSERT_TAIL(&sc->avail, slot, next); } } static int cpsw_add_slots(struct cpsw_softc *sc, struct cpsw_queue *queue, int requested) { const int max_slots = nitems(sc->_slots); struct cpsw_slot *slot; int i; if (requested < 0) requested = max_slots; for (i = 0; i < requested; ++i) { slot = STAILQ_FIRST(&sc->avail); if (slot == NULL) return (0); if (bus_dmamap_create(sc->mbuf_dtag, 0, &slot->dmamap)) { device_printf(sc->dev, "failed to create dmamap\n"); return (ENOMEM); } STAILQ_REMOVE_HEAD(&sc->avail, next); STAILQ_INSERT_TAIL(&queue->avail, slot, next); ++queue->avail_queue_len; ++queue->queue_slots; } return (0); } static void cpsw_free_slot(struct cpsw_softc *sc, struct cpsw_slot *slot) { int error; if (slot->dmamap) { if (slot->mbuf) bus_dmamap_unload(sc->mbuf_dtag, slot->dmamap); error = bus_dmamap_destroy(sc->mbuf_dtag, slot->dmamap); KASSERT(error == 0, ("Mapping still active")); slot->dmamap = NULL; } if (slot->mbuf) { m_freem(slot->mbuf); slot->mbuf = NULL; } } static void cpsw_reset(struct cpsw_softc *sc) { int i; callout_stop(&sc->watchdog.callout); /* Reset RMII/RGMII wrapper. */ cpsw_write_4(sc, CPSW_WR_SOFT_RESET, 1); while (cpsw_read_4(sc, CPSW_WR_SOFT_RESET) & 1) ; /* Disable TX and RX interrupts for all cores. */ for (i = 0; i < 3; ++i) { cpsw_write_4(sc, CPSW_WR_C_RX_THRESH_EN(i), 0x00); cpsw_write_4(sc, CPSW_WR_C_TX_EN(i), 0x00); cpsw_write_4(sc, CPSW_WR_C_RX_EN(i), 0x00); cpsw_write_4(sc, CPSW_WR_C_MISC_EN(i), 0x00); } /* Reset CPSW subsystem. */ cpsw_write_4(sc, CPSW_SS_SOFT_RESET, 1); while (cpsw_read_4(sc, CPSW_SS_SOFT_RESET) & 1) ; /* Reset Sliver port 1 and 2 */ for (i = 0; i < 2; i++) { /* Reset */ cpsw_write_4(sc, CPSW_SL_SOFT_RESET(i), 1); while (cpsw_read_4(sc, CPSW_SL_SOFT_RESET(i)) & 1) ; } /* Reset DMA controller. */ cpsw_write_4(sc, CPSW_CPDMA_SOFT_RESET, 1); while (cpsw_read_4(sc, CPSW_CPDMA_SOFT_RESET) & 1) ; /* Disable TX & RX DMA */ cpsw_write_4(sc, CPSW_CPDMA_TX_CONTROL, 0); cpsw_write_4(sc, CPSW_CPDMA_RX_CONTROL, 0); /* Clear all queues. */ for (i = 0; i < 8; i++) { cpsw_write_4(sc, CPSW_CPDMA_TX_HDP(i), 0); cpsw_write_4(sc, CPSW_CPDMA_RX_HDP(i), 0); cpsw_write_4(sc, CPSW_CPDMA_TX_CP(i), 0); cpsw_write_4(sc, CPSW_CPDMA_RX_CP(i), 0); } /* Clear all interrupt Masks */ cpsw_write_4(sc, CPSW_CPDMA_RX_INTMASK_CLEAR, 0xFFFFFFFF); cpsw_write_4(sc, CPSW_CPDMA_TX_INTMASK_CLEAR, 0xFFFFFFFF); } static void cpsw_init(struct cpsw_softc *sc) { struct cpsw_slot *slot; uint32_t reg; /* Clear ALE */ cpsw_write_4(sc, CPSW_ALE_CONTROL, CPSW_ALE_CTL_CLEAR_TBL); /* Enable ALE */ reg = CPSW_ALE_CTL_ENABLE; if (sc->dualemac) reg |= CPSW_ALE_CTL_VLAN_AWARE; cpsw_write_4(sc, CPSW_ALE_CONTROL, reg); /* Set Host Port Mapping. */ cpsw_write_4(sc, CPSW_PORT_P0_CPDMA_TX_PRI_MAP, 0x76543210); cpsw_write_4(sc, CPSW_PORT_P0_CPDMA_RX_CH_MAP, 0); /* Initialize ALE: set host port to forwarding(3). */ cpsw_write_4(sc, CPSW_ALE_PORTCTL(0), 3); cpsw_write_4(sc, CPSW_SS_PTYPE, 0); /* Enable statistics for ports 0, 1 and 2 */ cpsw_write_4(sc, CPSW_SS_STAT_PORT_EN, 7); /* Experiment: Turn off flow control */ /* This seems to fix the watchdog resets that have plagued earlier versions of this driver; I'm not yet sure if there are negative effects yet. */ cpsw_write_4(sc, CPSW_SS_FLOW_CONTROL, 0); /* Make IP hdr aligned with 4 */ cpsw_write_4(sc, CPSW_CPDMA_RX_BUFFER_OFFSET, 2); /* Initialize RX Buffer Descriptors */ cpsw_write_4(sc, CPSW_CPDMA_RX_FREEBUFFER(0), 0); /* Enable TX & RX DMA */ cpsw_write_4(sc, CPSW_CPDMA_TX_CONTROL, 1); cpsw_write_4(sc, CPSW_CPDMA_RX_CONTROL, 1); /* Enable Interrupts for core 0 */ cpsw_write_4(sc, CPSW_WR_C_RX_THRESH_EN(0), 0xFF); cpsw_write_4(sc, CPSW_WR_C_RX_EN(0), 0xFF); cpsw_write_4(sc, CPSW_WR_C_MISC_EN(0), 0x1F); /* Enable host Error Interrupt */ cpsw_write_4(sc, CPSW_CPDMA_DMA_INTMASK_SET, 3); /* Enable interrupts for RX Channel 0 */ cpsw_write_4(sc, CPSW_CPDMA_RX_INTMASK_SET, 1); /* Initialze MDIO - ENABLE, PREAMBLE=0, FAULTENB, CLKDIV=0xFF */ /* TODO Calculate MDCLK=CLK/(CLKDIV+1) */ cpsw_write_4(sc, MDIOCONTROL, MDIOCTL_ENABLE | MDIOCTL_FAULTENB | 0xff); /* Select MII in GMII_SEL, Internal Delay mode */ //ti_scm_reg_write_4(0x650, 0); /* Initialize active queues. */ slot = STAILQ_FIRST(&sc->tx.active); if (slot != NULL) cpsw_write_hdp_slot(sc, &sc->tx, slot); slot = STAILQ_FIRST(&sc->rx.active); if (slot != NULL) cpsw_write_hdp_slot(sc, &sc->rx, slot); cpsw_rx_enqueue(sc); /* Activate network interface. */ sc->rx.running = 1; sc->tx.running = 1; sc->watchdog.timer = 0; callout_init(&sc->watchdog.callout, 0); callout_reset(&sc->watchdog.callout, hz, cpsw_tx_watchdog, sc); } /* * * Device Probe, Attach, Detach. * */ static int cpsw_probe(device_t dev) { if (!ofw_bus_status_okay(dev)) return (ENXIO); if (!ofw_bus_is_compatible(dev, "ti,cpsw")) return (ENXIO); device_set_desc(dev, "3-port Switch Ethernet Subsystem"); return (BUS_PROBE_DEFAULT); } static int cpsw_intr_attach(struct cpsw_softc *sc) { /* Note: We don't use sc->irq_res[2] (TX interrupt) */ if (bus_setup_intr(sc->dev, sc->irq_res[0], INTR_TYPE_NET | INTR_MPSAFE, NULL, cpsw_intr_rx_thresh, sc, &sc->ih_cookie[0]) != 0) { return (-1); } if (bus_setup_intr(sc->dev, sc->irq_res[1], INTR_TYPE_NET | INTR_MPSAFE, NULL, cpsw_intr_rx, sc, &sc->ih_cookie[1]) != 0) { return (-1); } if (bus_setup_intr(sc->dev, sc->irq_res[3], INTR_TYPE_NET | INTR_MPSAFE, NULL, cpsw_intr_misc, sc, &sc->ih_cookie[3]) != 0) { return (-1); } return (0); } static void cpsw_intr_detach(struct cpsw_softc *sc) { int i; for (i = 0; i < CPSW_INTR_COUNT; i++) { if (sc->ih_cookie[i]) { bus_teardown_intr(sc->dev, sc->irq_res[i], sc->ih_cookie[i]); } } } static int cpsw_get_fdt_data(struct cpsw_softc *sc, int port) { char *name; int len, phy, vlan; pcell_t phy_id[3], vlan_id; phandle_t child; unsigned long mdio_child_addr; /* Find any slave with phy_id */ phy = -1; vlan = -1; for (child = OF_child(sc->node); child != 0; child = OF_peer(child)) { if (OF_getprop_alloc(child, "name", 1, (void **)&name) < 0) continue; if (sscanf(name, "slave@%x", &mdio_child_addr) != 1) { OF_prop_free(name); continue; } OF_prop_free(name); if (mdio_child_addr != slave_mdio_addr[port]) continue; len = OF_getproplen(child, "phy_id"); if (len / sizeof(pcell_t) == 2) { /* Get phy address from fdt */ if (OF_getencprop(child, "phy_id", phy_id, len) > 0) phy = phy_id[1]; } len = OF_getproplen(child, "dual_emac_res_vlan"); if (len / sizeof(pcell_t) == 1) { /* Get phy address from fdt */ if (OF_getencprop(child, "dual_emac_res_vlan", &vlan_id, len) > 0) { vlan = vlan_id; } } break; } if (phy == -1) return (ENXIO); sc->port[port].phy = phy; sc->port[port].vlan = vlan; return (0); } static int cpsw_attach(device_t dev) { bus_dma_segment_t segs[1]; int error, i, nsegs; struct cpsw_softc *sc; uint32_t reg; sc = device_get_softc(dev); sc->dev = dev; sc->node = ofw_bus_get_node(dev); getbinuptime(&sc->attach_uptime); if (OF_getencprop(sc->node, "active_slave", &sc->active_slave, sizeof(sc->active_slave)) <= 0) { sc->active_slave = 0; } if (sc->active_slave > 1) sc->active_slave = 1; if (OF_hasprop(sc->node, "dual_emac")) sc->dualemac = 1; for (i = 0; i < CPSW_PORTS; i++) { if (!sc->dualemac && i != sc->active_slave) continue; if (cpsw_get_fdt_data(sc, i) != 0) { device_printf(dev, "failed to get PHY address from FDT\n"); return (ENXIO); } } /* Initialize mutexes */ mtx_init(&sc->tx.lock, device_get_nameunit(dev), "cpsw TX lock", MTX_DEF); mtx_init(&sc->rx.lock, device_get_nameunit(dev), "cpsw RX lock", MTX_DEF); /* Allocate IRQ resources */ error = bus_alloc_resources(dev, irq_res_spec, sc->irq_res); if (error) { device_printf(dev, "could not allocate IRQ resources\n"); cpsw_detach(dev); return (ENXIO); } sc->mem_rid = 0; sc->mem_res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &sc->mem_rid, RF_ACTIVE); if (sc->mem_res == NULL) { device_printf(sc->dev, "failed to allocate memory resource\n"); cpsw_detach(dev); return (ENXIO); } reg = cpsw_read_4(sc, CPSW_SS_IDVER); device_printf(dev, "CPSW SS Version %d.%d (%d)\n", (reg >> 8 & 0x7), reg & 0xFF, (reg >> 11) & 0x1F); cpsw_add_sysctls(sc); /* Allocate a busdma tag and DMA safe memory for mbufs. */ error = bus_dma_tag_create( bus_get_dma_tag(sc->dev), /* parent */ 1, 0, /* alignment, boundary */ BUS_SPACE_MAXADDR_32BIT, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filtfunc, filtfuncarg */ MCLBYTES, CPSW_TXFRAGS, /* maxsize, nsegments */ MCLBYTES, 0, /* maxsegsz, flags */ NULL, NULL, /* lockfunc, lockfuncarg */ &sc->mbuf_dtag); /* dmatag */ if (error) { device_printf(dev, "bus_dma_tag_create failed\n"); cpsw_detach(dev); return (error); } /* Allocate the null mbuf and pre-sync it. */ sc->null_mbuf = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); memset(sc->null_mbuf->m_data, 0, sc->null_mbuf->m_ext.ext_size); bus_dmamap_create(sc->mbuf_dtag, 0, &sc->null_mbuf_dmamap); bus_dmamap_load_mbuf_sg(sc->mbuf_dtag, sc->null_mbuf_dmamap, sc->null_mbuf, segs, &nsegs, BUS_DMA_NOWAIT); bus_dmamap_sync(sc->mbuf_dtag, sc->null_mbuf_dmamap, BUS_DMASYNC_PREWRITE); sc->null_mbuf_paddr = segs[0].ds_addr; cpsw_init_slots(sc); /* Allocate slots to TX and RX queues. */ STAILQ_INIT(&sc->rx.avail); STAILQ_INIT(&sc->rx.active); STAILQ_INIT(&sc->tx.avail); STAILQ_INIT(&sc->tx.active); // For now: 128 slots to TX, rest to RX. // XXX TODO: start with 32/64 and grow dynamically based on demand. if (cpsw_add_slots(sc, &sc->tx, 128) || cpsw_add_slots(sc, &sc->rx, -1)) { device_printf(dev, "failed to allocate dmamaps\n"); cpsw_detach(dev); return (ENOMEM); } device_printf(dev, "Initial queue size TX=%d RX=%d\n", sc->tx.queue_slots, sc->rx.queue_slots); sc->tx.hdp_offset = CPSW_CPDMA_TX_HDP(0); sc->rx.hdp_offset = CPSW_CPDMA_RX_HDP(0); if (cpsw_intr_attach(sc) == -1) { device_printf(dev, "failed to setup interrupts\n"); cpsw_detach(dev); return (ENXIO); } /* Reset the controller. */ cpsw_reset(sc); cpsw_init(sc); for (i = 0; i < CPSW_PORTS; i++) { if (!sc->dualemac && i != sc->active_slave) continue; sc->port[i].dev = device_add_child(dev, "cpsw", i); if (sc->port[i].dev == NULL) { cpsw_detach(dev); return (ENXIO); } } bus_generic_attach(dev); return (0); } static int cpsw_detach(device_t dev) { struct cpsw_softc *sc; int error, i; bus_generic_detach(dev); sc = device_get_softc(dev); for (i = 0; i < CPSW_PORTS; i++) { if (sc->port[i].dev) device_delete_child(dev, sc->port[i].dev); } if (device_is_attached(dev)) { callout_stop(&sc->watchdog.callout); callout_drain(&sc->watchdog.callout); } /* Stop and release all interrupts */ cpsw_intr_detach(sc); /* Free dmamaps and mbufs */ for (i = 0; i < nitems(sc->_slots); ++i) cpsw_free_slot(sc, &sc->_slots[i]); /* Free null mbuf. */ if (sc->null_mbuf_dmamap) { bus_dmamap_unload(sc->mbuf_dtag, sc->null_mbuf_dmamap); error = bus_dmamap_destroy(sc->mbuf_dtag, sc->null_mbuf_dmamap); KASSERT(error == 0, ("Mapping still active")); m_freem(sc->null_mbuf); } /* Free DMA tag */ if (sc->mbuf_dtag) { error = bus_dma_tag_destroy(sc->mbuf_dtag); KASSERT(error == 0, ("Unable to destroy DMA tag")); } /* Free IO memory handler */ if (sc->mem_res != NULL) bus_release_resource(dev, SYS_RES_MEMORY, sc->mem_rid, sc->mem_res); bus_release_resources(dev, irq_res_spec, sc->irq_res); /* Destroy mutexes */ mtx_destroy(&sc->rx.lock); mtx_destroy(&sc->tx.lock); return (0); } static phandle_t cpsw_get_node(device_t bus, device_t dev) { /* Share controller node with port device. */ return (ofw_bus_get_node(bus)); } static int cpswp_probe(device_t dev) { if (device_get_unit(dev) > 1) { device_printf(dev, "Only two ports are supported.\n"); return (ENXIO); } device_set_desc(dev, "Ethernet Switch Port"); return (BUS_PROBE_DEFAULT); } static int cpswp_attach(device_t dev) { int error; struct ifnet *ifp; struct cpswp_softc *sc; uint32_t reg; uint8_t mac_addr[ETHER_ADDR_LEN]; sc = device_get_softc(dev); sc->dev = dev; sc->pdev = device_get_parent(dev); sc->swsc = device_get_softc(sc->pdev); sc->unit = device_get_unit(dev); sc->phy = sc->swsc->port[sc->unit].phy; sc->vlan = sc->swsc->port[sc->unit].vlan; if (sc->swsc->dualemac && sc->vlan == -1) sc->vlan = sc->unit + 1; if (sc->unit == 0) { sc->physel = MDIOUSERPHYSEL0; sc->phyaccess = MDIOUSERACCESS0; } else { sc->physel = MDIOUSERPHYSEL1; sc->phyaccess = MDIOUSERACCESS1; } mtx_init(&sc->lock, device_get_nameunit(dev), "cpsw port lock", MTX_DEF); /* Allocate network interface */ ifp = sc->ifp = if_alloc(IFT_ETHER); if (ifp == NULL) { cpswp_detach(dev); return (ENXIO); } if_initname(ifp, device_get_name(sc->dev), sc->unit); ifp->if_softc = sc; ifp->if_flags = IFF_SIMPLEX | IFF_MULTICAST | IFF_BROADCAST; ifp->if_capabilities = IFCAP_VLAN_MTU | IFCAP_HWCSUM; //FIXME VLAN? ifp->if_capenable = ifp->if_capabilities; ifp->if_init = cpswp_init; ifp->if_start = cpswp_start; ifp->if_ioctl = cpswp_ioctl; ifp->if_snd.ifq_drv_maxlen = sc->swsc->tx.queue_slots; IFQ_SET_MAXLEN(&ifp->if_snd, ifp->if_snd.ifq_drv_maxlen); IFQ_SET_READY(&ifp->if_snd); /* Get high part of MAC address from control module (mac_id[0|1]_hi) */ - ti_scm_reg_read_4(0x634 + sc->unit * 8, ®); + ti_scm_reg_read_4(CPSW_MAC_ID0_HI + sc->unit * 8, ®); mac_addr[0] = reg & 0xFF; mac_addr[1] = (reg >> 8) & 0xFF; mac_addr[2] = (reg >> 16) & 0xFF; mac_addr[3] = (reg >> 24) & 0xFF; /* Get low part of MAC address from control module (mac_id[0|1]_lo) */ - ti_scm_reg_read_4(0x630 + sc->unit * 8, ®); + ti_scm_reg_read_4(CPSW_MAC_ID0_LO + sc->unit * 8, ®); mac_addr[4] = reg & 0xFF; mac_addr[5] = (reg >> 8) & 0xFF; error = mii_attach(dev, &sc->miibus, ifp, cpswp_ifmedia_upd, cpswp_ifmedia_sts, BMSR_DEFCAPMASK, sc->phy, MII_OFFSET_ANY, 0); if (error) { device_printf(dev, "attaching PHYs failed\n"); cpswp_detach(dev); return (error); } sc->mii = device_get_softc(sc->miibus); /* Select PHY and enable interrupts */ cpsw_write_4(sc->swsc, sc->physel, MDIO_PHYSEL_LINKINTENB | (sc->phy & 0x1F)); ether_ifattach(sc->ifp, mac_addr); callout_init(&sc->mii_callout, 0); return (0); } static int cpswp_detach(device_t dev) { struct cpswp_softc *sc; sc = device_get_softc(dev); CPSWP_DEBUGF(sc, ("")); if (device_is_attached(dev)) { ether_ifdetach(sc->ifp); CPSW_PORT_LOCK(sc); cpswp_stop_locked(sc); CPSW_PORT_UNLOCK(sc); callout_drain(&sc->mii_callout); } bus_generic_detach(dev); if_free(sc->ifp); mtx_destroy(&sc->lock); return (0); } /* * * Init/Shutdown. * */ static int cpsw_ports_down(struct cpsw_softc *sc) { struct cpswp_softc *psc; struct ifnet *ifp1, *ifp2; if (!sc->dualemac) return (1); psc = device_get_softc(sc->port[0].dev); ifp1 = psc->ifp; psc = device_get_softc(sc->port[1].dev); ifp2 = psc->ifp; if ((ifp1->if_flags & IFF_UP) == 0 && (ifp2->if_flags & IFF_UP) == 0) return (1); return (0); } static void cpswp_init(void *arg) { struct cpswp_softc *sc = arg; CPSWP_DEBUGF(sc, ("")); CPSW_PORT_LOCK(sc); cpswp_init_locked(arg); CPSW_PORT_UNLOCK(sc); } static void cpswp_init_locked(void *arg) { struct cpswp_softc *sc = arg; struct ifnet *ifp; uint32_t reg; CPSWP_DEBUGF(sc, ("")); CPSW_PORT_LOCK_ASSERT(sc); ifp = sc->ifp; if ((ifp->if_drv_flags & IFF_DRV_RUNNING) != 0) return; getbinuptime(&sc->init_uptime); if (!sc->swsc->rx.running && !sc->swsc->tx.running) { /* Reset the controller. */ cpsw_reset(sc->swsc); cpsw_init(sc->swsc); } /* Set Slave Mapping. */ cpsw_write_4(sc->swsc, CPSW_SL_RX_PRI_MAP(sc->unit), 0x76543210); cpsw_write_4(sc->swsc, CPSW_PORT_P_TX_PRI_MAP(sc->unit + 1), 0x33221100); cpsw_write_4(sc->swsc, CPSW_SL_RX_MAXLEN(sc->unit), 0x5f2); /* Enable MAC RX/TX modules. */ /* TODO: Docs claim that IFCTL_B and IFCTL_A do the same thing? */ /* Huh? Docs call bit 0 "Loopback" some places, "FullDuplex" others. */ reg = cpsw_read_4(sc->swsc, CPSW_SL_MACCONTROL(sc->unit)); reg |= CPSW_SL_MACTL_GMII_ENABLE; cpsw_write_4(sc->swsc, CPSW_SL_MACCONTROL(sc->unit), reg); /* Initialize ALE: set port to forwarding(3), initialize addrs */ cpsw_write_4(sc->swsc, CPSW_ALE_PORTCTL(sc->unit + 1), 3); cpswp_ale_update_addresses(sc, 1); if (sc->swsc->dualemac) { /* Set Port VID. */ cpsw_write_4(sc->swsc, CPSW_PORT_P_VLAN(sc->unit + 1), sc->vlan & 0xfff); cpsw_ale_update_vlan_table(sc->swsc, sc->vlan, (1 << (sc->unit + 1)) | (1 << 0), /* Member list */ (1 << (sc->unit + 1)) | (1 << 0), /* Untagged egress */ (1 << (sc->unit + 1)) | (1 << 0), 0); /* mcast reg flood */ } mii_mediachg(sc->mii); callout_reset(&sc->mii_callout, hz, cpswp_tick, sc); ifp->if_drv_flags |= IFF_DRV_RUNNING; ifp->if_drv_flags &= ~IFF_DRV_OACTIVE; } static int cpsw_shutdown(device_t dev) { struct cpsw_softc *sc; struct cpswp_softc *psc; int i; sc = device_get_softc(dev); CPSW_DEBUGF(sc, ("")); for (i = 0; i < CPSW_PORTS; i++) { if (!sc->dualemac && i != sc->active_slave) continue; psc = device_get_softc(sc->port[i].dev); CPSW_PORT_LOCK(psc); cpswp_stop_locked(psc); CPSW_PORT_UNLOCK(psc); } return (0); } static void cpsw_rx_teardown_locked(struct cpsw_softc *sc) { struct ifnet *ifp; struct mbuf *received, *next; int i = 0; CPSW_DEBUGF(sc, ("starting RX teardown")); cpsw_write_4(sc, CPSW_CPDMA_RX_TEARDOWN, 0); for (;;) { received = cpsw_rx_dequeue(sc); CPSW_GLOBAL_UNLOCK(sc); while (received != NULL) { next = received->m_nextpkt; received->m_nextpkt = NULL; ifp = received->m_pkthdr.rcvif; (*ifp->if_input)(ifp, received); if_inc_counter(ifp, IFCOUNTER_IPACKETS, 1); received = next; } CPSW_GLOBAL_LOCK(sc); if (!sc->rx.running) { CPSW_DEBUGF(sc, ("finished RX teardown (%d retries)", i)); return; } if (++i > 10) { device_printf(sc->dev, "Unable to cleanly shutdown receiver\n"); return; } DELAY(10); } } static void cpsw_tx_teardown_locked(struct cpsw_softc *sc) { int i = 0; CPSW_DEBUGF(sc, ("starting TX teardown")); cpsw_write_4(sc, CPSW_CPDMA_TX_TEARDOWN, 0); cpsw_tx_dequeue(sc); while (sc->tx.running && ++i < 10) { DELAY(10); cpsw_tx_dequeue(sc); } if (sc->tx.running) { device_printf(sc->dev, "Unable to cleanly shutdown transmitter\n"); } CPSW_DEBUGF(sc, ("finished TX teardown (%d retries, %d idle buffers)", i, sc->tx.active_queue_len)); } static void cpswp_stop_locked(struct cpswp_softc *sc) { struct ifnet *ifp; uint32_t reg; ifp = sc->ifp; CPSWP_DEBUGF(sc, ("")); CPSW_PORT_LOCK_ASSERT(sc); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) return; /* Disable interface */ ifp->if_drv_flags &= ~IFF_DRV_RUNNING; ifp->if_drv_flags |= IFF_DRV_OACTIVE; /* Stop ticker */ callout_stop(&sc->mii_callout); /* Tear down the RX/TX queues. */ if (cpsw_ports_down(sc->swsc)) { CPSW_GLOBAL_LOCK(sc->swsc); cpsw_rx_teardown_locked(sc->swsc); cpsw_tx_teardown_locked(sc->swsc); CPSW_GLOBAL_UNLOCK(sc->swsc); } /* Stop MAC RX/TX modules. */ reg = cpsw_read_4(sc->swsc, CPSW_SL_MACCONTROL(sc->unit)); reg &= ~CPSW_SL_MACTL_GMII_ENABLE; cpsw_write_4(sc->swsc, CPSW_SL_MACCONTROL(sc->unit), reg); if (cpsw_ports_down(sc->swsc)) { /* Capture stats before we reset controller. */ cpsw_stats_collect(sc->swsc); cpsw_reset(sc->swsc); cpsw_init(sc->swsc); } } /* * Suspend/Resume. */ static int cpsw_suspend(device_t dev) { struct cpsw_softc *sc; struct cpswp_softc *psc; int i; sc = device_get_softc(dev); CPSW_DEBUGF(sc, ("")); for (i = 0; i < CPSW_PORTS; i++) { if (!sc->dualemac && i != sc->active_slave) continue; psc = device_get_softc(sc->port[i].dev); CPSW_PORT_LOCK(psc); cpswp_stop_locked(psc); CPSW_PORT_UNLOCK(psc); } return (0); } static int cpsw_resume(device_t dev) { struct cpsw_softc *sc; sc = device_get_softc(dev); CPSW_DEBUGF(sc, ("UNIMPLEMENTED")); return (0); } /* * * IOCTL * */ static void cpsw_set_promisc(struct cpswp_softc *sc, int set) { uint32_t reg; /* * Enabling promiscuous mode requires ALE_BYPASS to be enabled. * That disables the ALE forwarding logic and causes every * packet to be sent only to the host port. In bypass mode, * the ALE processes host port transmit packets the same as in * normal mode. */ reg = cpsw_read_4(sc->swsc, CPSW_ALE_CONTROL); reg &= ~CPSW_ALE_CTL_BYPASS; if (set) reg |= CPSW_ALE_CTL_BYPASS; cpsw_write_4(sc->swsc, CPSW_ALE_CONTROL, reg); } static void cpsw_set_allmulti(struct cpswp_softc *sc, int set) { if (set) { printf("All-multicast mode unimplemented\n"); } } static int cpswp_ioctl(struct ifnet *ifp, u_long command, caddr_t data) { struct cpswp_softc *sc; struct ifreq *ifr; int error; uint32_t changed; error = 0; sc = ifp->if_softc; ifr = (struct ifreq *)data; switch (command) { case SIOCSIFFLAGS: CPSW_PORT_LOCK(sc); if (ifp->if_flags & IFF_UP) { if (ifp->if_drv_flags & IFF_DRV_RUNNING) { changed = ifp->if_flags ^ sc->if_flags; CPSWP_DEBUGF(sc, ("SIOCSIFFLAGS: UP & RUNNING (changed=0x%x)", changed)); if (changed & IFF_PROMISC) cpsw_set_promisc(sc, ifp->if_flags & IFF_PROMISC); if (changed & IFF_ALLMULTI) cpsw_set_allmulti(sc, ifp->if_flags & IFF_ALLMULTI); } else { CPSWP_DEBUGF(sc, ("SIOCSIFFLAGS: UP but not RUNNING; starting up")); cpswp_init_locked(sc); } } else if (ifp->if_drv_flags & IFF_DRV_RUNNING) { CPSWP_DEBUGF(sc, ("SIOCSIFFLAGS: not UP but RUNNING; shutting down")); cpswp_stop_locked(sc); } sc->if_flags = ifp->if_flags; CPSW_PORT_UNLOCK(sc); break; case SIOCADDMULTI: cpswp_ale_update_addresses(sc, 0); break; case SIOCDELMULTI: /* Ugh. DELMULTI doesn't provide the specific address being removed, so the best we can do is remove everything and rebuild it all. */ cpswp_ale_update_addresses(sc, 1); break; case SIOCGIFMEDIA: case SIOCSIFMEDIA: error = ifmedia_ioctl(ifp, ifr, &sc->mii->mii_media, command); break; default: error = ether_ioctl(ifp, command, data); } return (error); } /* * * MIIBUS * */ static int cpswp_miibus_ready(struct cpsw_softc *sc, uint32_t reg) { uint32_t r, retries = CPSW_MIIBUS_RETRIES; while (--retries) { r = cpsw_read_4(sc, reg); if ((r & MDIO_PHYACCESS_GO) == 0) return (1); DELAY(CPSW_MIIBUS_DELAY); } return (0); } static int cpswp_miibus_readreg(device_t dev, int phy, int reg) { struct cpswp_softc *sc; uint32_t cmd, r; sc = device_get_softc(dev); if (!cpswp_miibus_ready(sc->swsc, sc->phyaccess)) { device_printf(dev, "MDIO not ready to read\n"); return (0); } /* Set GO, reg, phy */ cmd = MDIO_PHYACCESS_GO | (reg & 0x1F) << 21 | (phy & 0x1F) << 16; cpsw_write_4(sc->swsc, sc->phyaccess, cmd); if (!cpswp_miibus_ready(sc->swsc, sc->phyaccess)) { device_printf(dev, "MDIO timed out during read\n"); return (0); } r = cpsw_read_4(sc->swsc, sc->phyaccess); if ((r & MDIO_PHYACCESS_ACK) == 0) { device_printf(dev, "Failed to read from PHY.\n"); r = 0; } return (r & 0xFFFF); } static int cpswp_miibus_writereg(device_t dev, int phy, int reg, int value) { struct cpswp_softc *sc; uint32_t cmd; sc = device_get_softc(dev); if (!cpswp_miibus_ready(sc->swsc, sc->phyaccess)) { device_printf(dev, "MDIO not ready to write\n"); return (0); } /* Set GO, WRITE, reg, phy, and value */ cmd = MDIO_PHYACCESS_GO | MDIO_PHYACCESS_WRITE | (reg & 0x1F) << 21 | (phy & 0x1F) << 16 | (value & 0xFFFF); cpsw_write_4(sc->swsc, sc->phyaccess, cmd); if (!cpswp_miibus_ready(sc->swsc, sc->phyaccess)) { device_printf(dev, "MDIO timed out during write\n"); return (0); } if ((cpsw_read_4(sc->swsc, sc->phyaccess) & MDIO_PHYACCESS_ACK) == 0) device_printf(dev, "Failed to write to PHY.\n"); return (0); } static void cpswp_miibus_statchg(device_t dev) { struct cpswp_softc *sc; uint32_t mac_control, reg; sc = device_get_softc(dev); CPSWP_DEBUGF(sc, ("")); reg = CPSW_SL_MACCONTROL(sc->unit); mac_control = cpsw_read_4(sc->swsc, reg); mac_control &= ~(CPSW_SL_MACTL_GIG | CPSW_SL_MACTL_IFCTL_A | CPSW_SL_MACTL_IFCTL_B | CPSW_SL_MACTL_FULLDUPLEX); switch(IFM_SUBTYPE(sc->mii->mii_media_active)) { case IFM_1000_SX: case IFM_1000_LX: case IFM_1000_CX: case IFM_1000_T: mac_control |= CPSW_SL_MACTL_GIG; break; case IFM_100_TX: mac_control |= CPSW_SL_MACTL_IFCTL_A; break; } if (sc->mii->mii_media_active & IFM_FDX) mac_control |= CPSW_SL_MACTL_FULLDUPLEX; cpsw_write_4(sc->swsc, reg, mac_control); } /* * * Transmit/Receive Packets. * */ static void cpsw_intr_rx(void *arg) { struct cpsw_softc *sc = arg; struct ifnet *ifp; struct mbuf *received, *next; CPSW_RX_LOCK(sc); received = cpsw_rx_dequeue(sc); cpsw_rx_enqueue(sc); cpsw_write_4(sc, CPSW_CPDMA_CPDMA_EOI_VECTOR, 1); CPSW_RX_UNLOCK(sc); while (received != NULL) { next = received->m_nextpkt; received->m_nextpkt = NULL; ifp = received->m_pkthdr.rcvif; (*ifp->if_input)(ifp, received); if_inc_counter(ifp, IFCOUNTER_IPACKETS, 1); received = next; } } static struct mbuf * cpsw_rx_dequeue(struct cpsw_softc *sc) { struct cpsw_cpdma_bd bd; struct cpsw_slot *slot; struct cpswp_softc *psc; struct mbuf *mb_head, *mb_tail; int port, removed = 0; mb_head = mb_tail = NULL; /* Pull completed packets off hardware RX queue. */ while ((slot = STAILQ_FIRST(&sc->rx.active)) != NULL) { cpsw_cpdma_read_bd(sc, slot, &bd); if (bd.flags & CPDMA_BD_OWNER) break; /* Still in use by hardware */ CPSW_DEBUGF(sc, ("Removing received packet from RX queue")); ++removed; STAILQ_REMOVE_HEAD(&sc->rx.active, next); STAILQ_INSERT_TAIL(&sc->rx.avail, slot, next); bus_dmamap_sync(sc->mbuf_dtag, slot->dmamap, BUS_DMASYNC_POSTREAD); bus_dmamap_unload(sc->mbuf_dtag, slot->dmamap); if (bd.flags & CPDMA_BD_TDOWNCMPLT) { CPSW_DEBUGF(sc, ("RX teardown in progress")); m_freem(slot->mbuf); slot->mbuf = NULL; cpsw_write_cp(sc, &sc->rx, 0xfffffffc); sc->rx.running = 0; break; } cpsw_write_cp_slot(sc, &sc->rx, slot); port = (bd.flags & CPDMA_BD_PORT_MASK) - 1; KASSERT(port >= 0 && port <= 1, ("patcket received with invalid port: %d", port)); psc = device_get_softc(sc->port[port].dev); /* Set up mbuf */ /* TODO: track SOP/EOP bits to assemble a full mbuf out of received fragments. */ slot->mbuf->m_data += bd.bufoff; slot->mbuf->m_len = bd.pktlen - 4; slot->mbuf->m_pkthdr.len = bd.pktlen - 4; slot->mbuf->m_flags |= M_PKTHDR; slot->mbuf->m_pkthdr.rcvif = psc->ifp; slot->mbuf->m_nextpkt = NULL; if ((psc->ifp->if_capenable & IFCAP_RXCSUM) != 0) { /* check for valid CRC by looking into pkt_err[5:4] */ if ((bd.flags & CPDMA_BD_PKT_ERR_MASK) == 0) { slot->mbuf->m_pkthdr.csum_flags |= CSUM_IP_CHECKED; slot->mbuf->m_pkthdr.csum_flags |= CSUM_IP_VALID; slot->mbuf->m_pkthdr.csum_data = 0xffff; } } /* Add mbuf to packet list to be returned. */ if (mb_tail) { mb_tail->m_nextpkt = slot->mbuf; } else { mb_head = slot->mbuf; } mb_tail = slot->mbuf; slot->mbuf = NULL; } if (removed != 0) { sc->rx.queue_removes += removed; sc->rx.active_queue_len -= removed; sc->rx.avail_queue_len += removed; if (sc->rx.avail_queue_len > sc->rx.max_avail_queue_len) sc->rx.max_avail_queue_len = sc->rx.avail_queue_len; } return (mb_head); } static void cpsw_rx_enqueue(struct cpsw_softc *sc) { bus_dma_segment_t seg[1]; struct cpsw_cpdma_bd bd; struct cpsw_slots tmpqueue = STAILQ_HEAD_INITIALIZER(tmpqueue); struct cpsw_slot *slot, *prev_slot = NULL; struct cpsw_slot *last_old_slot, *first_new_slot; int error, nsegs, added = 0; /* Register new mbufs with hardware. */ while ((slot = STAILQ_FIRST(&sc->rx.avail)) != NULL) { if (slot->mbuf == NULL) { slot->mbuf = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); if (slot->mbuf == NULL) { device_printf(sc->dev, "Unable to fill RX queue\n"); break; } slot->mbuf->m_len = slot->mbuf->m_pkthdr.len = slot->mbuf->m_ext.ext_size; } error = bus_dmamap_load_mbuf_sg(sc->mbuf_dtag, slot->dmamap, slot->mbuf, seg, &nsegs, BUS_DMA_NOWAIT); KASSERT(nsegs == 1, ("More than one segment (nsegs=%d)", nsegs)); KASSERT(error == 0, ("DMA error (error=%d)", error)); if (error != 0 || nsegs != 1) { device_printf(sc->dev, "%s: Can't prep RX buf for DMA (nsegs=%d, error=%d)\n", __func__, nsegs, error); bus_dmamap_unload(sc->mbuf_dtag, slot->dmamap); m_freem(slot->mbuf); slot->mbuf = NULL; break; } bus_dmamap_sync(sc->mbuf_dtag, slot->dmamap, BUS_DMASYNC_PREREAD); /* Create and submit new rx descriptor*/ bd.next = 0; bd.bufptr = seg->ds_addr; bd.bufoff = 0; bd.buflen = MCLBYTES - 1; bd.pktlen = bd.buflen; bd.flags = CPDMA_BD_OWNER; cpsw_cpdma_write_bd(sc, slot, &bd); ++added; if (prev_slot != NULL) cpsw_cpdma_write_bd_next(sc, prev_slot, slot); prev_slot = slot; STAILQ_REMOVE_HEAD(&sc->rx.avail, next); sc->rx.avail_queue_len--; STAILQ_INSERT_TAIL(&tmpqueue, slot, next); } if (added == 0) return; CPSW_DEBUGF(sc, ("Adding %d buffers to RX queue", added)); /* Link new entries to hardware RX queue. */ last_old_slot = STAILQ_LAST(&sc->rx.active, cpsw_slot, next); first_new_slot = STAILQ_FIRST(&tmpqueue); STAILQ_CONCAT(&sc->rx.active, &tmpqueue); if (first_new_slot == NULL) { return; } else if (last_old_slot == NULL) { /* Start a fresh queue. */ cpsw_write_hdp_slot(sc, &sc->rx, first_new_slot); } else { /* Add buffers to end of current queue. */ cpsw_cpdma_write_bd_next(sc, last_old_slot, first_new_slot); /* If underrun, restart queue. */ if (cpsw_cpdma_read_bd_flags(sc, last_old_slot) & CPDMA_BD_EOQ) { cpsw_write_hdp_slot(sc, &sc->rx, first_new_slot); } } sc->rx.queue_adds += added; sc->rx.active_queue_len += added; if (sc->rx.active_queue_len > sc->rx.max_active_queue_len) { sc->rx.max_active_queue_len = sc->rx.active_queue_len; } } static void cpswp_start(struct ifnet *ifp) { struct cpswp_softc *sc = ifp->if_softc; CPSW_TX_LOCK(sc->swsc); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) && sc->swsc->tx.running) { cpswp_tx_enqueue(sc); cpsw_tx_dequeue(sc->swsc); } CPSW_TX_UNLOCK(sc->swsc); } static void cpswp_tx_enqueue(struct cpswp_softc *sc) { bus_dma_segment_t segs[CPSW_TXFRAGS]; struct cpsw_cpdma_bd bd; struct cpsw_slots tmpqueue = STAILQ_HEAD_INITIALIZER(tmpqueue); struct cpsw_slot *slot, *prev_slot = NULL; struct cpsw_slot *last_old_slot, *first_new_slot; struct mbuf *m0; int error, flags, nsegs, seg, added = 0, padlen; flags = 0; if (sc->swsc->dualemac) { flags = CPDMA_BD_TO_PORT | ((sc->unit + 1) & CPDMA_BD_PORT_MASK); } /* Pull pending packets from IF queue and prep them for DMA. */ while ((slot = STAILQ_FIRST(&sc->swsc->tx.avail)) != NULL) { IF_DEQUEUE(&sc->ifp->if_snd, m0); if (m0 == NULL) break; slot->mbuf = m0; padlen = ETHER_MIN_LEN - slot->mbuf->m_pkthdr.len; if (padlen < 0) padlen = 0; /* Create mapping in DMA memory */ error = bus_dmamap_load_mbuf_sg(sc->swsc->mbuf_dtag, slot->dmamap, slot->mbuf, segs, &nsegs, BUS_DMA_NOWAIT); /* If the packet is too fragmented, try to simplify. */ if (error == EFBIG || (error == 0 && nsegs + (padlen > 0 ? 1 : 0) > sc->swsc->tx.avail_queue_len)) { bus_dmamap_unload(sc->swsc->mbuf_dtag, slot->dmamap); if (padlen > 0) /* May as well add padding. */ m_append(slot->mbuf, padlen, sc->swsc->null_mbuf->m_data); m0 = m_defrag(slot->mbuf, M_NOWAIT); if (m0 == NULL) { device_printf(sc->dev, "Can't defragment packet; dropping\n"); m_freem(slot->mbuf); } else { CPSWP_DEBUGF(sc, ("Requeueing defragmented packet")); IF_PREPEND(&sc->ifp->if_snd, m0); } slot->mbuf = NULL; continue; } if (error != 0) { device_printf(sc->dev, "%s: Can't setup DMA (error=%d), dropping packet\n", __func__, error); bus_dmamap_unload(sc->swsc->mbuf_dtag, slot->dmamap); m_freem(slot->mbuf); slot->mbuf = NULL; break; } bus_dmamap_sync(sc->swsc->mbuf_dtag, slot->dmamap, BUS_DMASYNC_PREWRITE); CPSWP_DEBUGF(sc, ("Queueing TX packet: %d segments + %d pad bytes", nsegs, padlen)); slot->ifp = sc->ifp; /* If there is only one segment, the for() loop * gets skipped and the single buffer gets set up * as both SOP and EOP. */ /* Start by setting up the first buffer */ bd.next = 0; bd.bufptr = segs[0].ds_addr; bd.bufoff = 0; bd.buflen = segs[0].ds_len; bd.pktlen = m_length(slot->mbuf, NULL) + padlen; bd.flags = CPDMA_BD_SOP | CPDMA_BD_OWNER | flags; for (seg = 1; seg < nsegs; ++seg) { /* Save the previous buffer (which isn't EOP) */ cpsw_cpdma_write_bd(sc->swsc, slot, &bd); if (prev_slot != NULL) { cpsw_cpdma_write_bd_next(sc->swsc, prev_slot, slot); } prev_slot = slot; STAILQ_REMOVE_HEAD(&sc->swsc->tx.avail, next); sc->swsc->tx.avail_queue_len--; STAILQ_INSERT_TAIL(&tmpqueue, slot, next); ++added; slot = STAILQ_FIRST(&sc->swsc->tx.avail); /* Setup next buffer (which isn't SOP) */ bd.next = 0; bd.bufptr = segs[seg].ds_addr; bd.bufoff = 0; bd.buflen = segs[seg].ds_len; bd.pktlen = 0; bd.flags = CPDMA_BD_OWNER | flags; } /* Save the final buffer. */ if (padlen <= 0) bd.flags |= CPDMA_BD_EOP; cpsw_cpdma_write_bd(sc->swsc, slot, &bd); if (prev_slot != NULL) cpsw_cpdma_write_bd_next(sc->swsc, prev_slot, slot); prev_slot = slot; STAILQ_REMOVE_HEAD(&sc->swsc->tx.avail, next); sc->swsc->tx.avail_queue_len--; STAILQ_INSERT_TAIL(&tmpqueue, slot, next); ++added; if (padlen > 0) { slot = STAILQ_FIRST(&sc->swsc->tx.avail); STAILQ_REMOVE_HEAD(&sc->swsc->tx.avail, next); sc->swsc->tx.avail_queue_len--; STAILQ_INSERT_TAIL(&tmpqueue, slot, next); ++added; /* Setup buffer of null pad bytes (definitely EOP) */ cpsw_cpdma_write_bd_next(sc->swsc, prev_slot, slot); prev_slot = slot; bd.next = 0; bd.bufptr = sc->swsc->null_mbuf_paddr; bd.bufoff = 0; bd.buflen = padlen; bd.pktlen = 0; bd.flags = CPDMA_BD_EOP | CPDMA_BD_OWNER | flags; cpsw_cpdma_write_bd(sc->swsc, slot, &bd); ++nsegs; } if (nsegs > sc->swsc->tx.longest_chain) sc->swsc->tx.longest_chain = nsegs; // TODO: Should we defer the BPF tap until // after all packets are queued? BPF_MTAP(sc->ifp, m0); } /* Attach the list of new buffers to the hardware TX queue. */ last_old_slot = STAILQ_LAST(&sc->swsc->tx.active, cpsw_slot, next); first_new_slot = STAILQ_FIRST(&tmpqueue); STAILQ_CONCAT(&sc->swsc->tx.active, &tmpqueue); if (first_new_slot == NULL) { return; } else if (last_old_slot == NULL) { /* Start a fresh queue. */ sc->swsc->last_hdp = cpsw_cpdma_bd_paddr(sc->swsc, first_new_slot); cpsw_write_hdp_slot(sc->swsc, &sc->swsc->tx, first_new_slot); } else { /* Add buffers to end of current queue. */ cpsw_cpdma_write_bd_next(sc->swsc, last_old_slot, first_new_slot); /* If underrun, restart queue. */ if (cpsw_cpdma_read_bd_flags(sc->swsc, last_old_slot) & CPDMA_BD_EOQ) { sc->swsc->last_hdp = cpsw_cpdma_bd_paddr(sc->swsc, first_new_slot); cpsw_write_hdp_slot(sc->swsc, &sc->swsc->tx, first_new_slot); } } sc->swsc->tx.queue_adds += added; sc->swsc->tx.active_queue_len += added; if (sc->swsc->tx.active_queue_len > sc->swsc->tx.max_active_queue_len) { sc->swsc->tx.max_active_queue_len = sc->swsc->tx.active_queue_len; } } static int cpsw_tx_dequeue(struct cpsw_softc *sc) { struct cpsw_slot *slot, *last_removed_slot = NULL; struct cpsw_cpdma_bd bd; uint32_t flags, removed = 0; slot = STAILQ_FIRST(&sc->tx.active); if (slot == NULL && cpsw_read_cp(sc, &sc->tx) == 0xfffffffc) { CPSW_DEBUGF(sc, ("TX teardown of an empty queue")); cpsw_write_cp(sc, &sc->tx, 0xfffffffc); sc->tx.running = 0; return (0); } /* Pull completed buffers off the hardware TX queue. */ while (slot != NULL) { flags = cpsw_cpdma_read_bd_flags(sc, slot); if (flags & CPDMA_BD_OWNER) break; /* Hardware is still using this packet. */ CPSW_DEBUGF(sc, ("TX removing completed packet")); bus_dmamap_sync(sc->mbuf_dtag, slot->dmamap, BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(sc->mbuf_dtag, slot->dmamap); m_freem(slot->mbuf); slot->mbuf = NULL; if (slot->ifp) if_inc_counter(slot->ifp, IFCOUNTER_OPACKETS, 1); /* Dequeue any additional buffers used by this packet. */ while (slot != NULL && slot->mbuf == NULL) { STAILQ_REMOVE_HEAD(&sc->tx.active, next); STAILQ_INSERT_TAIL(&sc->tx.avail, slot, next); ++removed; last_removed_slot = slot; slot = STAILQ_FIRST(&sc->tx.active); } /* TearDown complete is only marked on the SOP for the packet. */ if ((flags & (CPDMA_BD_SOP | CPDMA_BD_TDOWNCMPLT)) == (CPDMA_BD_EOP | CPDMA_BD_TDOWNCMPLT)) { CPSW_DEBUGF(sc, ("TX teardown in progress")); cpsw_write_cp(sc, &sc->tx, 0xfffffffc); // TODO: Increment a count of dropped TX packets sc->tx.running = 0; break; } if ((flags & CPDMA_BD_EOP) == 0) flags = cpsw_cpdma_read_bd_flags(sc, last_removed_slot); if ((flags & (CPDMA_BD_EOP | CPDMA_BD_EOQ)) == (CPDMA_BD_EOP | CPDMA_BD_EOQ)) { cpsw_cpdma_read_bd(sc, last_removed_slot, &bd); if (bd.next != 0 && bd.next != sc->last_hdp) { /* Restart the queue. */ sc->last_hdp = bd.next; cpsw_write_4(sc, sc->tx.hdp_offset, bd.next); } } } if (removed != 0) { cpsw_write_cp_slot(sc, &sc->tx, last_removed_slot); sc->tx.queue_removes += removed; sc->tx.active_queue_len -= removed; sc->tx.avail_queue_len += removed; if (sc->tx.avail_queue_len > sc->tx.max_avail_queue_len) sc->tx.max_avail_queue_len = sc->tx.avail_queue_len; } return (removed); } /* * * Miscellaneous interrupts. * */ static void cpsw_intr_rx_thresh(void *arg) { struct cpsw_softc *sc = arg; uint32_t stat = cpsw_read_4(sc, CPSW_WR_C_RX_THRESH_STAT(0)); CPSW_DEBUGF(sc, ("stat=%x", stat)); cpsw_write_4(sc, CPSW_CPDMA_CPDMA_EOI_VECTOR, 0); } static void cpsw_intr_misc_host_error(struct cpsw_softc *sc) { uint32_t intstat; uint32_t dmastat; int txerr, rxerr, txchan, rxchan; printf("\n\n"); device_printf(sc->dev, "HOST ERROR: PROGRAMMING ERROR DETECTED BY HARDWARE\n"); printf("\n\n"); intstat = cpsw_read_4(sc, CPSW_CPDMA_DMA_INTSTAT_MASKED); device_printf(sc->dev, "CPSW_CPDMA_DMA_INTSTAT_MASKED=0x%x\n", intstat); dmastat = cpsw_read_4(sc, CPSW_CPDMA_DMASTATUS); device_printf(sc->dev, "CPSW_CPDMA_DMASTATUS=0x%x\n", dmastat); txerr = (dmastat >> 20) & 15; txchan = (dmastat >> 16) & 7; rxerr = (dmastat >> 12) & 15; rxchan = (dmastat >> 8) & 7; switch (txerr) { case 0: break; case 1: printf("SOP error on TX channel %d\n", txchan); break; case 2: printf("Ownership bit not set on SOP buffer on TX channel %d\n", txchan); break; case 3: printf("Zero Next Buffer but not EOP on TX channel %d\n", txchan); break; case 4: printf("Zero Buffer Pointer on TX channel %d\n", txchan); break; case 5: printf("Zero Buffer Length on TX channel %d\n", txchan); break; case 6: printf("Packet length error on TX channel %d\n", txchan); break; default: printf("Unknown error on TX channel %d\n", txchan); break; } if (txerr != 0) { printf("CPSW_CPDMA_TX%d_HDP=0x%x\n", txchan, cpsw_read_4(sc, CPSW_CPDMA_TX_HDP(txchan))); printf("CPSW_CPDMA_TX%d_CP=0x%x\n", txchan, cpsw_read_4(sc, CPSW_CPDMA_TX_CP(txchan))); cpsw_dump_queue(sc, &sc->tx.active); } switch (rxerr) { case 0: break; case 2: printf("Ownership bit not set on RX channel %d\n", rxchan); break; case 4: printf("Zero Buffer Pointer on RX channel %d\n", rxchan); break; case 5: printf("Zero Buffer Length on RX channel %d\n", rxchan); break; case 6: printf("Buffer offset too big on RX channel %d\n", rxchan); break; default: printf("Unknown RX error on RX channel %d\n", rxchan); break; } if (rxerr != 0) { printf("CPSW_CPDMA_RX%d_HDP=0x%x\n", rxchan, cpsw_read_4(sc,CPSW_CPDMA_RX_HDP(rxchan))); printf("CPSW_CPDMA_RX%d_CP=0x%x\n", rxchan, cpsw_read_4(sc, CPSW_CPDMA_RX_CP(rxchan))); cpsw_dump_queue(sc, &sc->rx.active); } printf("\nALE Table\n"); cpsw_ale_dump_table(sc); // XXX do something useful here?? panic("CPSW HOST ERROR INTERRUPT"); // Suppress this interrupt in the future. cpsw_write_4(sc, CPSW_CPDMA_DMA_INTMASK_CLEAR, intstat); printf("XXX HOST ERROR INTERRUPT SUPPRESSED\n"); // The watchdog will probably reset the controller // in a little while. It will probably fail again. } static void cpsw_intr_misc(void *arg) { struct cpsw_softc *sc = arg; uint32_t stat = cpsw_read_4(sc, CPSW_WR_C_MISC_STAT(0)); if (stat & CPSW_WR_C_MISC_EVNT_PEND) CPSW_DEBUGF(sc, ("Time sync event interrupt unimplemented")); if (stat & CPSW_WR_C_MISC_STAT_PEND) cpsw_stats_collect(sc); if (stat & CPSW_WR_C_MISC_HOST_PEND) cpsw_intr_misc_host_error(sc); if (stat & CPSW_WR_C_MISC_MDIOLINK) { cpsw_write_4(sc, MDIOLINKINTMASKED, cpsw_read_4(sc, MDIOLINKINTMASKED)); } if (stat & CPSW_WR_C_MISC_MDIOUSER) { CPSW_DEBUGF(sc, ("MDIO operation completed interrupt unimplemented")); } cpsw_write_4(sc, CPSW_CPDMA_CPDMA_EOI_VECTOR, 3); } /* * * Periodic Checks and Watchdog. * */ static void cpswp_tick(void *msc) { struct cpswp_softc *sc = msc; /* Check for media type change */ mii_tick(sc->mii); if (sc->media_status != sc->mii->mii_media.ifm_media) { printf("%s: media type changed (ifm_media=%x)\n", __func__, sc->mii->mii_media.ifm_media); cpswp_ifmedia_upd(sc->ifp); } /* Schedule another timeout one second from now */ callout_reset(&sc->mii_callout, hz, cpswp_tick, sc); } static void cpswp_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr) { struct cpswp_softc *sc; struct mii_data *mii; sc = ifp->if_softc; CPSWP_DEBUGF(sc, ("")); CPSW_PORT_LOCK(sc); mii = sc->mii; mii_pollstat(mii); ifmr->ifm_active = mii->mii_media_active; ifmr->ifm_status = mii->mii_media_status; CPSW_PORT_UNLOCK(sc); } static int cpswp_ifmedia_upd(struct ifnet *ifp) { struct cpswp_softc *sc; sc = ifp->if_softc; CPSWP_DEBUGF(sc, ("")); CPSW_PORT_LOCK(sc); mii_mediachg(sc->mii); sc->media_status = sc->mii->mii_media.ifm_media; CPSW_PORT_UNLOCK(sc); return (0); } static void cpsw_tx_watchdog_full_reset(struct cpsw_softc *sc) { struct cpswp_softc *psc; int i; cpsw_debugf_head("CPSW watchdog"); device_printf(sc->dev, "watchdog timeout\n"); for (i = 0; i < CPSW_PORTS; i++) { if (!sc->dualemac && i != sc->active_slave) continue; psc = device_get_softc(sc->port[i].dev); CPSW_PORT_LOCK(psc); cpswp_stop_locked(psc); CPSW_PORT_UNLOCK(psc); } } static void cpsw_tx_watchdog(void *msc) { struct cpsw_softc *sc; sc = msc; CPSW_GLOBAL_LOCK(sc); if (sc->tx.active_queue_len == 0 || !sc->tx.running) { sc->watchdog.timer = 0; /* Nothing to do. */ } else if (sc->tx.queue_removes > sc->tx.queue_removes_at_last_tick) { sc->watchdog.timer = 0; /* Stuff done while we weren't looking. */ } else if (cpsw_tx_dequeue(sc) > 0) { sc->watchdog.timer = 0; /* We just did something. */ } else { /* There was something to do but it didn't get done. */ ++sc->watchdog.timer; if (sc->watchdog.timer > 5) { sc->watchdog.timer = 0; ++sc->watchdog.resets; cpsw_tx_watchdog_full_reset(sc); } } sc->tx.queue_removes_at_last_tick = sc->tx.queue_removes; CPSW_GLOBAL_UNLOCK(sc); /* Schedule another timeout one second from now */ callout_reset(&sc->watchdog.callout, hz, cpsw_tx_watchdog, sc); } /* * * ALE support routines. * */ static void cpsw_ale_read_entry(struct cpsw_softc *sc, uint16_t idx, uint32_t *ale_entry) { cpsw_write_4(sc, CPSW_ALE_TBLCTL, idx & 1023); ale_entry[0] = cpsw_read_4(sc, CPSW_ALE_TBLW0); ale_entry[1] = cpsw_read_4(sc, CPSW_ALE_TBLW1); ale_entry[2] = cpsw_read_4(sc, CPSW_ALE_TBLW2); } static void cpsw_ale_write_entry(struct cpsw_softc *sc, uint16_t idx, uint32_t *ale_entry) { cpsw_write_4(sc, CPSW_ALE_TBLW0, ale_entry[0]); cpsw_write_4(sc, CPSW_ALE_TBLW1, ale_entry[1]); cpsw_write_4(sc, CPSW_ALE_TBLW2, ale_entry[2]); cpsw_write_4(sc, CPSW_ALE_TBLCTL, 1 << 31 | (idx & 1023)); } static void cpsw_ale_remove_all_mc_entries(struct cpsw_softc *sc) { int i; uint32_t ale_entry[3]; /* First four entries are link address and broadcast. */ for (i = 10; i < CPSW_MAX_ALE_ENTRIES; i++) { cpsw_ale_read_entry(sc, i, ale_entry); if ((ALE_TYPE(ale_entry) == ALE_TYPE_ADDR || ALE_TYPE(ale_entry) == ALE_TYPE_VLAN_ADDR) && ALE_MCAST(ale_entry) == 1) { /* MCast link addr */ ale_entry[0] = ale_entry[1] = ale_entry[2] = 0; cpsw_ale_write_entry(sc, i, ale_entry); } } } static int cpsw_ale_mc_entry_set(struct cpsw_softc *sc, uint8_t portmap, int vlan, uint8_t *mac) { int free_index = -1, matching_index = -1, i; uint32_t ale_entry[3], ale_type; /* Find a matching entry or a free entry. */ for (i = 10; i < CPSW_MAX_ALE_ENTRIES; i++) { cpsw_ale_read_entry(sc, i, ale_entry); /* Entry Type[61:60] is 0 for free entry */ if (free_index < 0 && ALE_TYPE(ale_entry) == 0) free_index = i; if ((((ale_entry[1] >> 8) & 0xFF) == mac[0]) && (((ale_entry[1] >> 0) & 0xFF) == mac[1]) && (((ale_entry[0] >>24) & 0xFF) == mac[2]) && (((ale_entry[0] >>16) & 0xFF) == mac[3]) && (((ale_entry[0] >> 8) & 0xFF) == mac[4]) && (((ale_entry[0] >> 0) & 0xFF) == mac[5])) { matching_index = i; break; } } if (matching_index < 0) { if (free_index < 0) return (ENOMEM); i = free_index; } if (vlan != -1) ale_type = ALE_TYPE_VLAN_ADDR << 28 | vlan << 16; else ale_type = ALE_TYPE_ADDR << 28; /* Set MAC address */ ale_entry[0] = mac[2] << 24 | mac[3] << 16 | mac[4] << 8 | mac[5]; ale_entry[1] = mac[0] << 8 | mac[1]; /* Entry type[61:60] and Mcast fwd state[63:62] is fw(3). */ ale_entry[1] |= ALE_MCAST_FWD | ale_type; /* Set portmask [68:66] */ ale_entry[2] = (portmap & 7) << 2; cpsw_ale_write_entry(sc, i, ale_entry); return 0; } static void cpsw_ale_dump_table(struct cpsw_softc *sc) { int i; uint32_t ale_entry[3]; for (i = 0; i < CPSW_MAX_ALE_ENTRIES; i++) { cpsw_ale_read_entry(sc, i, ale_entry); switch (ALE_TYPE(ale_entry)) { case ALE_TYPE_VLAN: printf("ALE[%4u] %08x %08x %08x ", i, ale_entry[2], ale_entry[1], ale_entry[0]); printf("type: %u ", ALE_TYPE(ale_entry)); printf("vlan: %u ", ALE_VLAN(ale_entry)); printf("untag: %u ", ALE_VLAN_UNTAG(ale_entry)); printf("reg flood: %u ", ALE_VLAN_REGFLOOD(ale_entry)); printf("unreg flood: %u ", ALE_VLAN_UNREGFLOOD(ale_entry)); printf("members: %u ", ALE_VLAN_MEMBERS(ale_entry)); printf("\n"); break; case ALE_TYPE_ADDR: case ALE_TYPE_VLAN_ADDR: printf("ALE[%4u] %08x %08x %08x ", i, ale_entry[2], ale_entry[1], ale_entry[0]); printf("type: %u ", ALE_TYPE(ale_entry)); printf("mac: %02x:%02x:%02x:%02x:%02x:%02x ", (ale_entry[1] >> 8) & 0xFF, (ale_entry[1] >> 0) & 0xFF, (ale_entry[0] >>24) & 0xFF, (ale_entry[0] >>16) & 0xFF, (ale_entry[0] >> 8) & 0xFF, (ale_entry[0] >> 0) & 0xFF); printf(ALE_MCAST(ale_entry) ? "mcast " : "ucast "); if (ALE_TYPE(ale_entry) == ALE_TYPE_VLAN_ADDR) printf("vlan: %u ", ALE_VLAN(ale_entry)); printf("port: %u ", ALE_PORTS(ale_entry)); printf("\n"); break; } } printf("\n"); } static int cpswp_ale_update_addresses(struct cpswp_softc *sc, int purge) { uint8_t *mac; uint32_t ale_entry[3], ale_type, portmask; struct ifmultiaddr *ifma; if (sc->swsc->dualemac) { ale_type = ALE_TYPE_VLAN_ADDR << 28 | sc->vlan << 16; portmask = 1 << (sc->unit + 1) | 1 << 0; } else { ale_type = ALE_TYPE_ADDR << 28; portmask = 7; } /* * Route incoming packets for our MAC address to Port 0 (host). * For simplicity, keep this entry at table index 0 for port 1 and * at index 2 for port 2 in the ALE. */ if_addr_rlock(sc->ifp); mac = LLADDR((struct sockaddr_dl *)sc->ifp->if_addr->ifa_addr); ale_entry[0] = mac[2] << 24 | mac[3] << 16 | mac[4] << 8 | mac[5]; ale_entry[1] = ale_type | mac[0] << 8 | mac[1]; /* addr entry + mac */ ale_entry[2] = 0; /* port = 0 */ cpsw_ale_write_entry(sc->swsc, 0 + 2 * sc->unit, ale_entry); /* Set outgoing MAC Address for slave port. */ cpsw_write_4(sc->swsc, CPSW_PORT_P_SA_HI(sc->unit + 1), mac[3] << 24 | mac[2] << 16 | mac[1] << 8 | mac[0]); cpsw_write_4(sc->swsc, CPSW_PORT_P_SA_LO(sc->unit + 1), mac[5] << 8 | mac[4]); if_addr_runlock(sc->ifp); /* Keep the broadcast address at table entry 1 (or 3). */ ale_entry[0] = 0xffffffff; /* Lower 32 bits of MAC */ /* ALE_MCAST_FWD, Addr type, upper 16 bits of Mac */ ale_entry[1] = ALE_MCAST_FWD | ale_type | 0xffff; ale_entry[2] = portmask << 2; cpsw_ale_write_entry(sc->swsc, 1 + 2 * sc->unit, ale_entry); /* SIOCDELMULTI doesn't specify the particular address being removed, so we have to remove all and rebuild. */ if (purge) cpsw_ale_remove_all_mc_entries(sc->swsc); /* Set other multicast addrs desired. */ if_maddr_rlock(sc->ifp); TAILQ_FOREACH(ifma, &sc->ifp->if_multiaddrs, ifma_link) { if (ifma->ifma_addr->sa_family != AF_LINK) continue; cpsw_ale_mc_entry_set(sc->swsc, portmask, sc->vlan, LLADDR((struct sockaddr_dl *)ifma->ifma_addr)); } if_maddr_runlock(sc->ifp); return (0); } static int cpsw_ale_update_vlan_table(struct cpsw_softc *sc, int vlan, int ports, int untag, int mcregflood, int mcunregflood) { int free_index, i, matching_index; uint32_t ale_entry[3]; free_index = matching_index = -1; /* Find a matching entry or a free entry. */ for (i = 5; i < CPSW_MAX_ALE_ENTRIES; i++) { cpsw_ale_read_entry(sc, i, ale_entry); /* Entry Type[61:60] is 0 for free entry */ if (free_index < 0 && ALE_TYPE(ale_entry) == 0) free_index = i; if (ALE_VLAN(ale_entry) == vlan) { matching_index = i; break; } } if (matching_index < 0) { if (free_index < 0) return (-1); i = free_index; } ale_entry[0] = (untag & 7) << 24 | (mcregflood & 7) << 16 | (mcunregflood & 7) << 8 | (ports & 7); ale_entry[1] = ALE_TYPE_VLAN << 28 | vlan << 16; ale_entry[2] = 0; cpsw_ale_write_entry(sc, i, ale_entry); return (0); } /* * * Statistics and Sysctls. * */ #if 0 static void cpsw_stats_dump(struct cpsw_softc *sc) { int i; uint32_t r; for (i = 0; i < CPSW_SYSCTL_COUNT; ++i) { r = cpsw_read_4(sc, CPSW_STATS_OFFSET + cpsw_stat_sysctls[i].reg); CPSW_DEBUGF(sc, ("%s: %ju + %u = %ju", cpsw_stat_sysctls[i].oid, (intmax_t)sc->shadow_stats[i], r, (intmax_t)sc->shadow_stats[i] + r)); } } #endif static void cpsw_stats_collect(struct cpsw_softc *sc) { int i; uint32_t r; CPSW_DEBUGF(sc, ("Controller shadow statistics updated.")); for (i = 0; i < CPSW_SYSCTL_COUNT; ++i) { r = cpsw_read_4(sc, CPSW_STATS_OFFSET + cpsw_stat_sysctls[i].reg); sc->shadow_stats[i] += r; cpsw_write_4(sc, CPSW_STATS_OFFSET + cpsw_stat_sysctls[i].reg, r); } } static int cpsw_stats_sysctl(SYSCTL_HANDLER_ARGS) { struct cpsw_softc *sc; struct cpsw_stat *stat; uint64_t result; sc = (struct cpsw_softc *)arg1; stat = &cpsw_stat_sysctls[oidp->oid_number]; result = sc->shadow_stats[oidp->oid_number]; result += cpsw_read_4(sc, CPSW_STATS_OFFSET + stat->reg); return (sysctl_handle_64(oidp, &result, 0, req)); } static int cpsw_stat_attached(SYSCTL_HANDLER_ARGS) { struct cpsw_softc *sc; struct bintime t; unsigned result; sc = (struct cpsw_softc *)arg1; getbinuptime(&t); bintime_sub(&t, &sc->attach_uptime); result = t.sec; return (sysctl_handle_int(oidp, &result, 0, req)); } static int cpsw_stat_uptime(SYSCTL_HANDLER_ARGS) { struct cpsw_softc *swsc; struct cpswp_softc *sc; struct bintime t; unsigned result; swsc = arg1; sc = device_get_softc(swsc->port[arg2].dev); if (sc->ifp->if_drv_flags & IFF_DRV_RUNNING) { getbinuptime(&t); bintime_sub(&t, &sc->init_uptime); result = t.sec; } else result = 0; return (sysctl_handle_int(oidp, &result, 0, req)); } static void cpsw_add_queue_sysctls(struct sysctl_ctx_list *ctx, struct sysctl_oid *node, struct cpsw_queue *queue) { struct sysctl_oid_list *parent; parent = SYSCTL_CHILDREN(node); SYSCTL_ADD_INT(ctx, parent, OID_AUTO, "totalBuffers", CTLFLAG_RD, &queue->queue_slots, 0, "Total buffers currently assigned to this queue"); SYSCTL_ADD_INT(ctx, parent, OID_AUTO, "activeBuffers", CTLFLAG_RD, &queue->active_queue_len, 0, "Buffers currently registered with hardware controller"); SYSCTL_ADD_INT(ctx, parent, OID_AUTO, "maxActiveBuffers", CTLFLAG_RD, &queue->max_active_queue_len, 0, "Max value of activeBuffers since last driver reset"); SYSCTL_ADD_INT(ctx, parent, OID_AUTO, "availBuffers", CTLFLAG_RD, &queue->avail_queue_len, 0, "Buffers allocated to this queue but not currently " "registered with hardware controller"); SYSCTL_ADD_INT(ctx, parent, OID_AUTO, "maxAvailBuffers", CTLFLAG_RD, &queue->max_avail_queue_len, 0, "Max value of availBuffers since last driver reset"); SYSCTL_ADD_UINT(ctx, parent, OID_AUTO, "totalEnqueued", CTLFLAG_RD, &queue->queue_adds, 0, "Total buffers added to queue"); SYSCTL_ADD_UINT(ctx, parent, OID_AUTO, "totalDequeued", CTLFLAG_RD, &queue->queue_removes, 0, "Total buffers removed from queue"); SYSCTL_ADD_UINT(ctx, parent, OID_AUTO, "longestChain", CTLFLAG_RD, &queue->longest_chain, 0, "Max buffers used for a single packet"); } static void cpsw_add_watchdog_sysctls(struct sysctl_ctx_list *ctx, struct sysctl_oid *node, struct cpsw_softc *sc) { struct sysctl_oid_list *parent; parent = SYSCTL_CHILDREN(node); SYSCTL_ADD_INT(ctx, parent, OID_AUTO, "resets", CTLFLAG_RD, &sc->watchdog.resets, 0, "Total number of watchdog resets"); } static void cpsw_add_sysctls(struct cpsw_softc *sc) { struct sysctl_ctx_list *ctx; struct sysctl_oid *stats_node, *queue_node, *node; struct sysctl_oid_list *parent, *stats_parent, *queue_parent; struct sysctl_oid_list *ports_parent, *port_parent; char port[16]; int i; ctx = device_get_sysctl_ctx(sc->dev); parent = SYSCTL_CHILDREN(device_get_sysctl_tree(sc->dev)); SYSCTL_ADD_INT(ctx, parent, OID_AUTO, "debug", CTLFLAG_RW, &sc->debug, 0, "Enable switch debug messages"); SYSCTL_ADD_PROC(ctx, parent, OID_AUTO, "attachedSecs", CTLTYPE_UINT | CTLFLAG_RD, sc, 0, cpsw_stat_attached, "IU", "Time since driver attach"); node = SYSCTL_ADD_NODE(ctx, parent, OID_AUTO, "ports", CTLFLAG_RD, NULL, "CPSW Ports Statistics"); ports_parent = SYSCTL_CHILDREN(node); for (i = 0; i < CPSW_PORTS; i++) { if (!sc->dualemac && i != sc->active_slave) continue; port[0] = '0' + i; port[1] = '\0'; node = SYSCTL_ADD_NODE(ctx, ports_parent, OID_AUTO, port, CTLFLAG_RD, NULL, "CPSW Port Statistics"); port_parent = SYSCTL_CHILDREN(node); SYSCTL_ADD_PROC(ctx, port_parent, OID_AUTO, "uptime", CTLTYPE_UINT | CTLFLAG_RD, sc, i, cpsw_stat_uptime, "IU", "Seconds since driver init"); } stats_node = SYSCTL_ADD_NODE(ctx, parent, OID_AUTO, "stats", CTLFLAG_RD, NULL, "CPSW Statistics"); stats_parent = SYSCTL_CHILDREN(stats_node); for (i = 0; i < CPSW_SYSCTL_COUNT; ++i) { SYSCTL_ADD_PROC(ctx, stats_parent, i, cpsw_stat_sysctls[i].oid, CTLTYPE_U64 | CTLFLAG_RD, sc, 0, cpsw_stats_sysctl, "IU", cpsw_stat_sysctls[i].oid); } queue_node = SYSCTL_ADD_NODE(ctx, parent, OID_AUTO, "queue", CTLFLAG_RD, NULL, "CPSW Queue Statistics"); queue_parent = SYSCTL_CHILDREN(queue_node); node = SYSCTL_ADD_NODE(ctx, queue_parent, OID_AUTO, "tx", CTLFLAG_RD, NULL, "TX Queue Statistics"); cpsw_add_queue_sysctls(ctx, node, &sc->tx); node = SYSCTL_ADD_NODE(ctx, queue_parent, OID_AUTO, "rx", CTLFLAG_RD, NULL, "RX Queue Statistics"); cpsw_add_queue_sysctls(ctx, node, &sc->rx); node = SYSCTL_ADD_NODE(ctx, parent, OID_AUTO, "watchdog", CTLFLAG_RD, NULL, "Watchdog Statistics"); cpsw_add_watchdog_sysctls(ctx, node, sc); } Index: projects/netbsd-tests-update-12/sys/arm/ti/cpsw/if_cpswreg.h =================================================================== --- projects/netbsd-tests-update-12/sys/arm/ti/cpsw/if_cpswreg.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/ti/cpsw/if_cpswreg.h (revision 305172) @@ -1,178 +1,181 @@ /*- * Copyright (c) 2012 Damjan Marion * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _IF_CPSWREG_H #define _IF_CPSWREG_H #define CPSW_SS_OFFSET 0x0000 #define CPSW_SS_IDVER (CPSW_SS_OFFSET + 0x00) #define CPSW_SS_SOFT_RESET (CPSW_SS_OFFSET + 0x08) #define CPSW_SS_STAT_PORT_EN (CPSW_SS_OFFSET + 0x0C) #define CPSW_SS_PTYPE (CPSW_SS_OFFSET + 0x10) #define CPSW_SS_FLOW_CONTROL (CPSW_SS_OFFSET + 0x24) #define CPSW_PORT_OFFSET 0x0100 #define CPSW_PORT_P_MAX_BLKS(p) (CPSW_PORT_OFFSET + 0x08 + ((p) * 0x100)) #define CPSW_PORT_P_BLK_CNT(p) (CPSW_PORT_OFFSET + 0x0C + ((p) * 0x100)) #define CPSW_PORT_P_VLAN(p) (CPSW_PORT_OFFSET + 0x14 + ((p) * 0x100)) #define CPSW_PORT_P_TX_PRI_MAP(p) (CPSW_PORT_OFFSET + 0x118 + ((p-1) * 0x100)) #define CPSW_PORT_P0_CPDMA_TX_PRI_MAP (CPSW_PORT_OFFSET + 0x01C) #define CPSW_PORT_P0_CPDMA_RX_CH_MAP (CPSW_PORT_OFFSET + 0x020) #define CPSW_PORT_P_SA_LO(p) (CPSW_PORT_OFFSET + 0x120 + ((p-1) * 0x100)) #define CPSW_PORT_P_SA_HI(p) (CPSW_PORT_OFFSET + 0x124 + ((p-1) * 0x100)) +#define CPSW_MAC_ID0_LO 0x0630 +#define CPSW_MAC_ID0_HI 0x0634 + #define CPSW_CPDMA_OFFSET 0x0800 #define CPSW_CPDMA_TX_CONTROL (CPSW_CPDMA_OFFSET + 0x04) #define CPSW_CPDMA_TX_TEARDOWN (CPSW_CPDMA_OFFSET + 0x08) #define CPSW_CPDMA_RX_CONTROL (CPSW_CPDMA_OFFSET + 0x14) #define CPSW_CPDMA_RX_TEARDOWN (CPSW_CPDMA_OFFSET + 0x18) #define CPSW_CPDMA_SOFT_RESET (CPSW_CPDMA_OFFSET + 0x1c) #define CPSW_CPDMA_DMACONTROL (CPSW_CPDMA_OFFSET + 0x20) #define CPSW_CPDMA_DMASTATUS (CPSW_CPDMA_OFFSET + 0x24) #define CPSW_CPDMA_RX_BUFFER_OFFSET (CPSW_CPDMA_OFFSET + 0x28) #define CPSW_CPDMA_TX_INTSTAT_RAW (CPSW_CPDMA_OFFSET + 0x80) #define CPSW_CPDMA_TX_INTSTAT_MASKED (CPSW_CPDMA_OFFSET + 0x84) #define CPSW_CPDMA_TX_INTMASK_SET (CPSW_CPDMA_OFFSET + 0x88) #define CPSW_CPDMA_TX_INTMASK_CLEAR (CPSW_CPDMA_OFFSET + 0x8C) #define CPSW_CPDMA_CPDMA_EOI_VECTOR (CPSW_CPDMA_OFFSET + 0x94) #define CPSW_CPDMA_RX_INTSTAT_RAW (CPSW_CPDMA_OFFSET + 0xA0) #define CPSW_CPDMA_RX_INTSTAT_MASKED (CPSW_CPDMA_OFFSET + 0xA4) #define CPSW_CPDMA_RX_INTMASK_SET (CPSW_CPDMA_OFFSET + 0xA8) #define CPSW_CPDMA_RX_INTMASK_CLEAR (CPSW_CPDMA_OFFSET + 0xAc) #define CPSW_CPDMA_DMA_INTSTAT_RAW (CPSW_CPDMA_OFFSET + 0xB0) #define CPSW_CPDMA_DMA_INTSTAT_MASKED (CPSW_CPDMA_OFFSET + 0xB4) #define CPSW_CPDMA_DMA_INTMASK_SET (CPSW_CPDMA_OFFSET + 0xB8) #define CPSW_CPDMA_DMA_INTMASK_CLEAR (CPSW_CPDMA_OFFSET + 0xBC) #define CPSW_CPDMA_RX_FREEBUFFER(p) (CPSW_CPDMA_OFFSET + 0x0e0 + ((p) * 0x04)) #define CPSW_STATS_OFFSET 0x0900 #define CPSW_STATERAM_OFFSET 0x0A00 #define CPSW_CPDMA_TX_HDP(p) (CPSW_STATERAM_OFFSET + 0x00 + ((p) * 0x04)) #define CPSW_CPDMA_RX_HDP(p) (CPSW_STATERAM_OFFSET + 0x20 + ((p) * 0x04)) #define CPSW_CPDMA_TX_CP(p) (CPSW_STATERAM_OFFSET + 0x40 + ((p) * 0x04)) #define CPSW_CPDMA_RX_CP(p) (CPSW_STATERAM_OFFSET + 0x60 + ((p) * 0x04)) #define CPSW_CPTS_OFFSET 0x0C00 #define CPSW_ALE_OFFSET 0x0D00 #define CPSW_ALE_CONTROL (CPSW_ALE_OFFSET + 0x08) #define CPSW_ALE_CTL_ENABLE (1U << 31) #define CPSW_ALE_CTL_CLEAR_TBL (1 << 30) #define CPSW_ALE_CTL_BYPASS (1 << 4) #define CPSW_ALE_CTL_VLAN_AWARE (1 << 2) #define CPSW_ALE_TBLCTL (CPSW_ALE_OFFSET + 0x20) #define CPSW_ALE_TBLW2 (CPSW_ALE_OFFSET + 0x34) #define CPSW_ALE_TBLW1 (CPSW_ALE_OFFSET + 0x38) #define CPSW_ALE_TBLW0 (CPSW_ALE_OFFSET + 0x3C) #define ALE_MCAST(_a) ((_a[1] >> 8) & 1) #define ALE_MCAST_FWD (3 << 30) #define ALE_PORTS(_a) ((_a[2] >> 2) & 7) #define ALE_TYPE(_a) ((_a[1] >> 28) & 3) #define ALE_TYPE_ADDR 1 #define ALE_TYPE_VLAN 2 #define ALE_TYPE_VLAN_ADDR 3 #define ALE_VLAN(_a) ((_a[1] >> 16) & 0xfff) #define ALE_VLAN_UNREGFLOOD(_a) ((_a[0] >> 8) & 7) #define ALE_VLAN_REGFLOOD(_a) ((_a[0] >> 16) & 7) #define ALE_VLAN_UNTAG(_a) ((_a[0] >> 24) & 7) #define ALE_VLAN_MEMBERS(_a) (_a[0] & 7) #define CPSW_ALE_PORTCTL(p) (CPSW_ALE_OFFSET + 0x40 + ((p) * 0x04)) /* SL1 is at 0x0D80, SL2 is at 0x0DC0 */ #define CPSW_SL_OFFSET 0x0D80 #define CPSW_SL_MACCONTROL(p) (CPSW_SL_OFFSET + (0x40 * (p)) + 0x04) #define CPSW_SL_MACTL_IFCTL_B (1 << 16) #define CPSW_SL_MACTL_IFCTL_A (1 << 15) #define CPSW_SL_MACTL_GIG (1 << 7) #define CPSW_SL_MACTL_GMII_ENABLE (1 << 5) #define CPSW_SL_MACTL_FULLDUPLEX (1 << 0) #define CPSW_SL_MACSTATUS(p) (CPSW_SL_OFFSET + (0x40 * (p)) + 0x08) #define CPSW_SL_SOFT_RESET(p) (CPSW_SL_OFFSET + (0x40 * (p)) + 0x0C) #define CPSW_SL_RX_MAXLEN(p) (CPSW_SL_OFFSET + (0x40 * (p)) + 0x10) #define CPSW_SL_RX_PAUSE(p) (CPSW_SL_OFFSET + (0x40 * (p)) + 0x18) #define CPSW_SL_TX_PAUSE(p) (CPSW_SL_OFFSET + (0x40 * (p)) + 0x1C) #define CPSW_SL_RX_PRI_MAP(p) (CPSW_SL_OFFSET + (0x40 * (p)) + 0x24) #define MDIO_OFFSET 0x1000 #define MDIOCONTROL (MDIO_OFFSET + 0x04) #define MDIOCTL_ENABLE (1 << 30) #define MDIOCTL_FAULTENB (1 << 18) #define MDIOLINKINTRAW (MDIO_OFFSET + 0x10) #define MDIOLINKINTMASKED (MDIO_OFFSET + 0x14) #define MDIOUSERACCESS0 (MDIO_OFFSET + 0x80) #define MDIOUSERPHYSEL0 (MDIO_OFFSET + 0x84) #define MDIOUSERACCESS1 (MDIO_OFFSET + 0x88) #define MDIOUSERPHYSEL1 (MDIO_OFFSET + 0x8C) #define MDIO_PHYSEL_LINKINTENB (1 << 6) #define MDIO_PHYACCESS_GO (1U << 31) #define MDIO_PHYACCESS_WRITE (1 << 30) #define MDIO_PHYACCESS_ACK (1 << 29) #define CPSW_WR_OFFSET 0x1200 #define CPSW_WR_SOFT_RESET (CPSW_WR_OFFSET + 0x04) #define CPSW_WR_CONTROL (CPSW_WR_OFFSET + 0x08) #define CPSW_WR_INT_CONTROL (CPSW_WR_OFFSET + 0x0c) #define CPSW_WR_C_RX_THRESH_EN(p) (CPSW_WR_OFFSET + (0x10 * (p)) + 0x10) #define CPSW_WR_C_RX_EN(p) (CPSW_WR_OFFSET + (0x10 * (p)) + 0x14) #define CPSW_WR_C_TX_EN(p) (CPSW_WR_OFFSET + (0x10 * (p)) + 0x18) #define CPSW_WR_C_MISC_EN(p) (CPSW_WR_OFFSET + (0x10 * (p)) + 0x1C) #define CPSW_WR_C_RX_THRESH_STAT(p) (CPSW_WR_OFFSET + (0x10 * (p)) + 0x40) #define CPSW_WR_C_RX_STAT(p) (CPSW_WR_OFFSET + (0x10 * (p)) + 0x44) #define CPSW_WR_C_TX_STAT(p) (CPSW_WR_OFFSET + (0x10 * (p)) + 0x48) #define CPSW_WR_C_MISC_STAT(p) (CPSW_WR_OFFSET + (0x10 * (p)) + 0x4C) #define CPSW_WR_C_MISC_EVNT_PEND (1 << 4) #define CPSW_WR_C_MISC_STAT_PEND (1 << 3) #define CPSW_WR_C_MISC_HOST_PEND (1 << 2) #define CPSW_WR_C_MISC_MDIOLINK (1 << 1) #define CPSW_WR_C_MISC_MDIOUSER (1 << 0) #define CPSW_CPPI_RAM_OFFSET 0x2000 #define CPSW_CPPI_RAM_SIZE 0x2000 #define CPSW_MEMWINDOW_SIZE 0x4000 #define CPDMA_BD_SOP (1 << 15) #define CPDMA_BD_EOP (1 << 14) #define CPDMA_BD_OWNER (1 << 13) #define CPDMA_BD_EOQ (1 << 12) #define CPDMA_BD_TDOWNCMPLT (1 << 11) #define CPDMA_BD_PKT_ERR_MASK (3 << 4) #define CPDMA_BD_TO_PORT (1 << 4) #define CPDMA_BD_PORT_MASK 3 struct cpsw_cpdma_bd { volatile uint32_t next; volatile uint32_t bufptr; volatile uint16_t buflen; volatile uint16_t bufoff; volatile uint16_t pktlen; volatile uint16_t flags; }; #endif /*_IF_CPSWREG_H */ Index: projects/netbsd-tests-update-12/sys/arm/ti/ti_scm.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/ti/ti_scm.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/ti/ti_scm.c (revision 305172) @@ -1,174 +1,177 @@ /* * Copyright (c) 2010 * Ben Gray . * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Ben Gray. * 4. The name of the company nor the name of the author may be used to * endorse or promote products derived from this software without specific * prior written permission. * * THIS SOFTWARE IS PROVIDED BY BEN GRAY ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL BEN GRAY BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /** * SCM - System Control Module * * Hopefully in the end this module will contain a bunch of utility functions * for configuring and querying the general system control registers, but for * now it only does pin(pad) multiplexing. * * This is different from the GPIO module in that it is used to configure the * pins between modules not just GPIO input/output. * * This file contains the generic top level driver, however it relies on chip * specific settings and therefore expects an array of ti_scm_padconf structs * call ti_padconf_devmap to be located somewhere in the kernel. * */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "ti_scm.h" static struct resource_spec ti_scm_res_spec[] = { { SYS_RES_MEMORY, 0, RF_ACTIVE }, /* Control memory window */ { -1, 0 } }; static struct ti_scm_softc *ti_scm_sc; #define ti_scm_read_4(sc, reg) \ bus_space_read_4((sc)->sc_bst, (sc)->sc_bsh, (reg)) #define ti_scm_write_4(sc, reg, val) \ bus_space_write_4((sc)->sc_bst, (sc)->sc_bsh, (reg), (val)) /* * Device part of OMAP SCM driver */ static int ti_scm_probe(device_t dev) { if (!ofw_bus_status_okay(dev)) return (ENXIO); if (!ofw_bus_is_compatible(dev, "syscon")) return (ENXIO); if (ti_scm_sc) { return (EEXIST); } device_set_desc(dev, "TI Control Module"); return (BUS_PROBE_DEFAULT); } /** * ti_scm_attach - attaches the timer to the simplebus * @dev: new device * * Reserves memory and interrupt resources, stores the softc structure * globally and registers both the timecount and eventtimer objects. * * RETURNS * Zero on success or ENXIO if an error occuried. */ static int ti_scm_attach(device_t dev) { struct ti_scm_softc *sc = device_get_softc(dev); sc->sc_dev = dev; if (bus_alloc_resources(dev, ti_scm_res_spec, sc->sc_res)) { device_printf(dev, "could not allocate resources\n"); return (ENXIO); } /* Global timer interface */ sc->sc_bst = rman_get_bustag(sc->sc_res[0]); sc->sc_bsh = rman_get_bushandle(sc->sc_res[0]); ti_scm_sc = sc; - return (0); + /* Attach platform extensions, if any. */ + bus_generic_probe(dev); + + return (bus_generic_attach(dev)); } int ti_scm_reg_read_4(uint32_t reg, uint32_t *val) { if (!ti_scm_sc) return (ENXIO); *val = ti_scm_read_4(ti_scm_sc, reg); return (0); } int ti_scm_reg_write_4(uint32_t reg, uint32_t val) { if (!ti_scm_sc) return (ENXIO); ti_scm_write_4(ti_scm_sc, reg, val); return (0); } static device_method_t ti_scm_methods[] = { DEVMETHOD(device_probe, ti_scm_probe), DEVMETHOD(device_attach, ti_scm_attach), { 0, 0 } }; static driver_t ti_scm_driver = { "ti_scm", ti_scm_methods, sizeof(struct ti_scm_softc), }; static devclass_t ti_scm_devclass; EARLY_DRIVER_MODULE(ti_scm, simplebus, ti_scm_driver, ti_scm_devclass, 0, 0, BUS_PASS_BUS + BUS_PASS_ORDER_MIDDLE); Index: projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/crb_machdep.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/crb_machdep.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/crb_machdep.c (revision 305172) @@ -1,331 +1,331 @@ /* $NetBSD: hpc_machdep.c,v 1.70 2003/09/16 08:18:22 agc Exp $ */ /*- * Copyright (c) 1994-1998 Mark Brinicombe. * Copyright (c) 1994 Brini. * All rights reserved. * * This code is derived from software written for Brini by Mark Brinicombe * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Brini. * 4. The name of the company nor the name of the author may be used to * endorse or promote products derived from this software without specific * prior written permission. * * THIS SOFTWARE IS PROVIDED BY BRINI ``AS IS'' AND ANY EXPRESS OR IMPLIED * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL BRINI OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * RiscBSD kernel project * * machdep.c * * Machine dependent functions for kernel setup * * This file needs a lot of work. * * Created : 17/09/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_kstack_pages.h" #define _ARM32_BUS_DMA_PRIVATE #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* For i80321_calibrate_delay() */ #include #include #include #define KERNEL_PT_SYS 0 /* Page table for mapping proc0 zero page */ #define KERNEL_PT_IOPXS 1 #define KERNEL_PT_BEFOREKERN 2 #define KERNEL_PT_AFKERNEL 3 /* L2 table for mapping after kernel */ #define KERNEL_PT_AFKERNEL_NUM 9 /* this should be evenly divisable by PAGE_SIZE / L2_TABLE_SIZE_REAL (or 4) */ #define NUM_KERNEL_PTS (KERNEL_PT_AFKERNEL + KERNEL_PT_AFKERNEL_NUM) struct pv_addr kernel_pt_table[NUM_KERNEL_PTS]; /* Physical and virtual addresses for some global pages */ struct pv_addr systempage; struct pv_addr msgbufpv; struct pv_addr irqstack; struct pv_addr undstack; struct pv_addr abtstack; struct pv_addr kernelstack; /* Static device mappings. */ static const struct devmap_entry iq81342_devmap[] = { { IOP34X_VADDR, IOP34X_HWADDR, IOP34X_SIZE, }, { /* * Cheat and map a whole section, this will bring * both PCI-X and PCI-E outbound I/O */ rounddown2(IOP34X_PCIX_OIOBAR_VADDR, 0x100000), rounddown2(IOP34X_PCIX_OIOBAR, 0x100000), 0x100000, }, { IOP34X_PCE1_VADDR, IOP34X_PCE1, IOP34X_PCE1_SIZE, }, { 0, 0, 0, } }; #define SDRAM_START 0x00000000 extern vm_offset_t xscale_cache_clean_addr; void * initarm(struct arm_boot_params *abp) { struct pv_addr kernel_l1pt; struct pv_addr dpcpu; int loop, i; u_int l1pagetable; vm_offset_t freemempos; vm_offset_t freemem_pt; vm_offset_t afterkern; vm_offset_t freemem_after; vm_offset_t lastaddr; uint32_t memsize, memstart; lastaddr = parse_boot_param(abp); arm_physmem_kernaddr = abp->abp_physaddr; set_cpufuncs(); pcpu_init(pcpup, 0, sizeof(struct pcpu)); PCPU_SET(curthread, &thread0); /* Do basic tuning, hz etc */ init_param1(); freemempos = 0x00200000; /* Define a macro to simplify memory allocation */ #define valloc_pages(var, np) \ alloc_pages((var).pv_pa, (np)); \ (var).pv_va = (var).pv_pa + 0xc0000000; #define alloc_pages(var, np) \ freemempos -= (np * PAGE_SIZE); \ (var) = freemempos; \ memset((char *)(var), 0, ((np) * PAGE_SIZE)); while (((freemempos - L1_TABLE_SIZE) & (L1_TABLE_SIZE - 1)) != 0) freemempos -= PAGE_SIZE; valloc_pages(kernel_l1pt, L1_TABLE_SIZE / PAGE_SIZE); for (loop = 0; loop < NUM_KERNEL_PTS; ++loop) { if (!(loop % (PAGE_SIZE / L2_TABLE_SIZE_REAL))) { valloc_pages(kernel_pt_table[loop], L2_TABLE_SIZE / PAGE_SIZE); } else { kernel_pt_table[loop].pv_pa = freemempos + (loop % (PAGE_SIZE / L2_TABLE_SIZE_REAL)) * L2_TABLE_SIZE_REAL; kernel_pt_table[loop].pv_va = kernel_pt_table[loop].pv_pa + 0xc0000000; } } freemem_pt = freemempos; freemempos = 0x00100000; /* * Allocate a page for the system page mapped to V0x00000000 * This page will just contain the system vectors and can be * shared by all processes. */ valloc_pages(systempage, 1); /* Allocate dynamic per-cpu area. */ valloc_pages(dpcpu, DPCPU_SIZE / PAGE_SIZE); dpcpu_init((void *)dpcpu.pv_va, 0); /* Allocate stacks for all modes */ valloc_pages(irqstack, IRQ_STACK_SIZE); valloc_pages(abtstack, ABT_STACK_SIZE); valloc_pages(undstack, UND_STACK_SIZE); valloc_pages(kernelstack, kstack_pages); valloc_pages(msgbufpv, round_page(msgbufsize) / PAGE_SIZE); /* * Now we start construction of the L1 page table * We start by mapping the L2 page tables into the L1. * This means that we can replace L1 mappings later on if necessary */ l1pagetable = kernel_l1pt.pv_va; /* Map the L2 pages tables in the L1 page table */ pmap_link_l2pt(l1pagetable, rounddown2(ARM_VECTORS_HIGH, 0x00100000), &kernel_pt_table[KERNEL_PT_SYS]); pmap_map_chunk(l1pagetable, KERNBASE, SDRAM_START, 0x100000, VM_PROT_READ|VM_PROT_WRITE, PTE_CACHE); pmap_map_chunk(l1pagetable, KERNBASE + 0x100000, SDRAM_START + 0x100000, 0x100000, VM_PROT_READ|VM_PROT_WRITE, PTE_PAGETABLE); pmap_map_chunk(l1pagetable, KERNBASE + 0x200000, SDRAM_START + 0x200000, rounddown2(((uint32_t)(lastaddr) - KERNBASE - 0x200000) + L1_S_SIZE, L1_S_SIZE), VM_PROT_READ|VM_PROT_WRITE, PTE_CACHE); freemem_after = rounddown2((int)lastaddr + PAGE_SIZE, PAGE_SIZE); afterkern = round_page(rounddown2((vm_offset_t)lastaddr + L1_S_SIZE, L1_S_SIZE)); for (i = 0; i < KERNEL_PT_AFKERNEL_NUM; i++) { pmap_link_l2pt(l1pagetable, afterkern + i * 0x00100000, &kernel_pt_table[KERNEL_PT_AFKERNEL + i]); } /* Map the vector page. */ pmap_map_entry(l1pagetable, ARM_VECTORS_HIGH, systempage.pv_pa, VM_PROT_READ|VM_PROT_WRITE, PTE_CACHE); devmap_bootstrap(l1pagetable, iq81342_devmap); /* * Give the XScale global cache clean code an appropriately * sized chunk of unmapped VA space starting at 0xff000000 * (our device mappings end before this address). */ xscale_cache_clean_addr = 0xff000000U; cpu_domains((DOMAIN_CLIENT << (PMAP_DOMAIN_KERNEL*2)) | DOMAIN_CLIENT); cpu_setttb(kernel_l1pt.pv_pa); cpu_tlb_flushID(); cpu_domains(DOMAIN_CLIENT << (PMAP_DOMAIN_KERNEL*2)); /* * Pages were allocated during the secondary bootstrap for the * stacks for different CPU modes. * We must now set the r13 registers in the different CPU modes to * point to these stacks. * Since the ARM stacks use STMFD etc. we must set r13 to the top end * of the stack memory. */ set_stackptrs(0); /* * We must now clean the cache again.... * Cleaning may be done by reading new data to displace any * dirty data in the cache. This will have happened in cpu_setttb() * but since we are boot strapping the addresses used for the read * may have just been remapped and thus the cache could be out * of sync. A re-clean after the switch will cure this. * After booting there are no gross relocations of the kernel thus * this problem will not occur after initarm(). */ cpu_idcache_wbinv_all(); cpu_setup(); i80321_calibrate_delay(); - i81342_sdram_bounds(obio_bs_tag, IOP34X_VADDR, &memstart, &memsize); + i81342_sdram_bounds(arm_base_bs_tag, IOP34X_VADDR, &memstart, &memsize); physmem = memsize / PAGE_SIZE; cninit(); /* Set stack for exception handlers */ undefined_init(); init_proc0(kernelstack.pv_va); arm_vector_init(ARM_VECTORS_HIGH, ARM_VEC_ALL); pmap_curmaxkvaddr = afterkern + PAGE_SIZE; vm_max_kernel_address = 0xe0000000; pmap_bootstrap(pmap_curmaxkvaddr, &kernel_l1pt); msgbufp = (void*)msgbufpv.pv_va; msgbufinit(msgbufp, msgbufsize); mutex_init(); /* * Add the physical ram we have available. * * Exclude the kernel (and all the things we allocated which immediately * follow the kernel) from the VM allocation pool but not from crash * dumps. virtual_avail is a global variable which tracks the kva we've * "allocated" while setting up pmaps. * * Prepare the list of physical memory available to the vm subsystem. */ arm_physmem_hardware_region(SDRAM_START, memsize); arm_physmem_exclude_region(freemem_pt, abp->abp_physaddr - freemem_pt, EXFLAG_NOALLOC); arm_physmem_exclude_region(freemempos, abp->abp_physaddr - 0x100000 - freemempos, EXFLAG_NOALLOC); arm_physmem_exclude_region(abp->abp_physaddr, virtual_avail - KERNVIRTADDR, EXFLAG_NOALLOC); arm_physmem_init_kernel_globals(); init_param2(physmem); kdb_init(); return ((void *)(kernelstack.pv_va + USPACE_SVC_STACK_TOP - sizeof(struct pcb))); } Index: projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/obio.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/obio.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/obio.c (revision 305172) @@ -1,169 +1,166 @@ /* $NetBSD: obio.c,v 1.11 2003/07/15 00:25:05 lukem Exp $ */ /*- * Copyright (c) 2001, 2002, 2003 Wasabi Systems, Inc. * All rights reserved. * * Written by Jason R. Thorpe for Wasabi Systems, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed for the NetBSD Project by * Wasabi Systems, Inc. * 4. The name of Wasabi Systems, Inc. may not be used to endorse * or promote products derived from this software without specific prior * written permission. * * THIS SOFTWARE IS PROVIDED BY WASABI SYSTEMS, INC. ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WASABI SYSTEMS, INC * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ /* * On-board device autoconfiguration support for Intel IQ80321 * evaluation boards. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include -bus_space_tag_t obio_bs_tag; - static int obio_probe(device_t dev) { return (0); } static int obio_attach(device_t dev) { struct obio_softc *sc = device_get_softc(dev); - obio_bs_tag = arm_base_bs_tag; - sc->oba_st = obio_bs_tag; + sc->oba_st = arm_base_bs_tag; sc->oba_rman.rm_type = RMAN_ARRAY; sc->oba_rman.rm_descr = "OBIO I/O"; if (rman_init(&sc->oba_rman) != 0 || rman_manage_region(&sc->oba_rman, IOP34X_UART0_VADDR, IOP34X_UART1_VADDR + 0x40) != 0) panic("obio_attach: failed to set up I/O rman"); sc->oba_irq_rman.rm_type = RMAN_ARRAY; sc->oba_irq_rman.rm_descr = "OBIO IRQ"; if (rman_init(&sc->oba_irq_rman) != 0 || rman_manage_region(&sc->oba_irq_rman, ICU_INT_UART0, ICU_INT_UART1) != 0) panic("obio_attach: failed to set up IRQ rman"); device_add_child(dev, "uart", 0); device_add_child(dev, "uart", 1); bus_generic_probe(dev); bus_generic_attach(dev); return (0); } static struct resource * obio_alloc_resource(device_t bus, device_t child, int type, int *rid, rman_res_t start, rman_res_t end, rman_res_t count, u_int flags) { struct resource *rv; struct rman *rm; bus_space_tag_t bt = NULL; bus_space_handle_t bh = 0; struct obio_softc *sc = device_get_softc(bus); int unit = device_get_unit(child); switch (type) { case SYS_RES_IRQ: rm = &sc->oba_irq_rman; if (unit == 0) start = end = ICU_INT_UART0; else start = end = ICU_INT_UART1; break; case SYS_RES_MEMORY: return (NULL); case SYS_RES_IOPORT: rm = &sc->oba_rman; bt = sc->oba_st; if (unit == 0) { bh = IOP34X_UART0_VADDR; start = bh; end = IOP34X_UART1_VADDR; } else { bh = IOP34X_UART1_VADDR; start = bh; end = start + 0x40; } break; default: return (NULL); } rv = rman_reserve_resource(rm, start, end, count, flags, child); if (rv == NULL) return (NULL); if (type == SYS_RES_IRQ) return (rv); rman_set_rid(rv, *rid); rman_set_bustag(rv, bt); rman_set_bushandle(rv, bh); return (rv); } static int obio_activate_resource(device_t bus, device_t child, int type, int rid, struct resource *r) { return (0); } static device_method_t obio_methods[] = { DEVMETHOD(device_probe, obio_probe), DEVMETHOD(device_attach, obio_attach), DEVMETHOD(bus_alloc_resource, obio_alloc_resource), DEVMETHOD(bus_activate_resource, obio_activate_resource), DEVMETHOD(bus_setup_intr, bus_generic_setup_intr), DEVMETHOD(bus_teardown_intr, bus_generic_teardown_intr), {0, 0}, }; static driver_t obio_driver = { "obio", obio_methods, sizeof(struct obio_softc), }; static devclass_t obio_devclass; DRIVER_MODULE(obio, iq, obio_driver, obio_devclass, 0, 0); Index: projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/obiovar.h =================================================================== --- projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/obiovar.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/obiovar.h (revision 305172) @@ -1,55 +1,54 @@ /* $NetBSD: obiovar.h,v 1.4 2003/06/16 17:40:53 thorpej Exp $ */ /*- * Copyright (c) 2002, 2003 Wasabi Systems, Inc. * All rights reserved. * * Written by Jason R. Thorpe for Wasabi Systems, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed for the NetBSD Project by * Wasabi Systems, Inc. * 4. The name of Wasabi Systems, Inc. may not be used to endorse * or promote products derived from this software without specific prior * written permission. * * THIS SOFTWARE IS PROVIDED BY WASABI SYSTEMS, INC. ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WASABI SYSTEMS, INC * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ * */ #ifndef _IQ81342_OBIOVAR_H_ #define _IQ81342_OBIOVAR_H_ #include struct obio_softc { bus_space_tag_t oba_st; /* bus space tag */ int oba_irq; /* XINT interrupt bit # */ struct rman oba_rman; struct rman oba_irq_rman; }; -extern bus_space_tag_t obio_bs_tag; #endif /* _IQ80321_OBIOVAR_H_ */ Index: projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/uart_cpu_i81342.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/uart_cpu_i81342.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm/xscale/i8134x/uart_cpu_i81342.c (revision 305172) @@ -1,68 +1,68 @@ /*- * Copyright (c) 2003 Marcel Moolenaar * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include bus_space_tag_t uart_bus_space_io; bus_space_tag_t uart_bus_space_mem; int uart_cpu_eqres(struct uart_bas *b1, struct uart_bas *b2) { return ((b1->bsh == b2->bsh && b1->bst == b2->bst) ? 1 : 0); } int uart_cpu_getdev(int devtype, struct uart_devinfo *di) { di->ops = uart_getops(&uart_ns8250_class); di->bas.chan = 0; - di->bas.bst = obio_bs_tag; + di->bas.bst = arm_base_bs_tag; di->bas.regshft = 2; di->bas.rclk = 33334000; di->baudrate = 115200; di->databits = 8; di->stopbits = 1; di->parity = UART_PARITY_NONE; - uart_bus_space_io = obio_bs_tag; + uart_bus_space_io = arm_base_bs_tag; uart_bus_space_mem = NULL; di->bas.bsh = IOP34X_UART0_VADDR; return (0); } Index: projects/netbsd-tests-update-12/sys/arm64/arm64/pmap.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm64/arm64/pmap.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm64/arm64/pmap.c (revision 305172) @@ -1,4741 +1,4744 @@ /*- * Copyright (c) 1991 Regents of the University of California. * All rights reserved. * Copyright (c) 1994 John S. Dyson * All rights reserved. * Copyright (c) 1994 David Greenman * All rights reserved. * Copyright (c) 2003 Peter Wemm * All rights reserved. * Copyright (c) 2005-2010 Alan L. Cox * All rights reserved. * Copyright (c) 2014 Andrew Turner * All rights reserved. * Copyright (c) 2014-2016 The FreeBSD Foundation * All rights reserved. * * This code is derived from software contributed to Berkeley by * the Systems Programming Group of the University of Utah Computer * Science Department and William Jolitz of UUNET Technologies Inc. * * This software was developed by Andrew Turner under sponsorship from * the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by the University of * California, Berkeley and its contributors. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: @(#)pmap.c 7.7 (Berkeley) 5/12/91 */ /*- * Copyright (c) 2003 Networks Associates Technology, Inc. * All rights reserved. * * This software was developed for the FreeBSD Project by Jake Burkholder, * Safeport Network Services, and Network Associates Laboratories, the * Security Research Division of Network Associates, Inc. under * DARPA/SPAWAR contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA * CHATS research program. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); /* * Manages physical address maps. * * Since the information managed by this module is * also stored by the logical address mapping module, * this module may throw away valid virtual-to-physical * mappings at almost any time. However, invalidations * of virtual-to-physical mappings must be done as * requested. * * In order to cope with hardware architectures which * make virtual-to-physical map invalidates expensive, * this module may delay invalidate or reduced protection * operations until such time as they are actually * necessary. This module is given full information as * to which processors are currently using which maps, * and to when physical maps must be made correct. */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define NL0PG (PAGE_SIZE/(sizeof (pd_entry_t))) #define NL1PG (PAGE_SIZE/(sizeof (pd_entry_t))) #define NL2PG (PAGE_SIZE/(sizeof (pd_entry_t))) #define NL3PG (PAGE_SIZE/(sizeof (pt_entry_t))) #define NUL0E L0_ENTRIES #define NUL1E (NUL0E * NL1PG) #define NUL2E (NUL1E * NL2PG) #if !defined(DIAGNOSTIC) #ifdef __GNUC_GNU_INLINE__ #define PMAP_INLINE __attribute__((__gnu_inline__)) inline #else #define PMAP_INLINE extern inline #endif #else #define PMAP_INLINE #endif /* * These are configured by the mair_el1 register. This is set up in locore.S */ #define DEVICE_MEMORY 0 #define UNCACHED_MEMORY 1 #define CACHED_MEMORY 2 #ifdef PV_STATS #define PV_STAT(x) do { x ; } while (0) #else #define PV_STAT(x) do { } while (0) #endif #define pmap_l2_pindex(v) ((v) >> L2_SHIFT) #define pa_to_pvh(pa) (&pv_table[pmap_l2_pindex(pa)]) #define NPV_LIST_LOCKS MAXCPU #define PHYS_TO_PV_LIST_LOCK(pa) \ (&pv_list_locks[pa_index(pa) % NPV_LIST_LOCKS]) #define CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, pa) do { \ struct rwlock **_lockp = (lockp); \ struct rwlock *_new_lock; \ \ _new_lock = PHYS_TO_PV_LIST_LOCK(pa); \ if (_new_lock != *_lockp) { \ if (*_lockp != NULL) \ rw_wunlock(*_lockp); \ *_lockp = _new_lock; \ rw_wlock(*_lockp); \ } \ } while (0) #define CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m) \ CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, VM_PAGE_TO_PHYS(m)) #define RELEASE_PV_LIST_LOCK(lockp) do { \ struct rwlock **_lockp = (lockp); \ \ if (*_lockp != NULL) { \ rw_wunlock(*_lockp); \ *_lockp = NULL; \ } \ } while (0) #define VM_PAGE_TO_PV_LIST_LOCK(m) \ PHYS_TO_PV_LIST_LOCK(VM_PAGE_TO_PHYS(m)) struct pmap kernel_pmap_store; vm_offset_t virtual_avail; /* VA of first avail page (after kernel bss) */ vm_offset_t virtual_end; /* VA of last avail page (end of kernel AS) */ vm_offset_t kernel_vm_end = 0; struct msgbuf *msgbufp = NULL; /* * Data for the pv entry allocation mechanism. * Updates to pv_invl_gen are protected by the pv_list_locks[] * elements, but reads are not. */ static struct md_page *pv_table; static struct md_page pv_dummy; vm_paddr_t dmap_phys_base; /* The start of the dmap region */ vm_paddr_t dmap_phys_max; /* The limit of the dmap region */ vm_offset_t dmap_max_addr; /* The virtual address limit of the dmap */ /* This code assumes all L1 DMAP entries will be used */ CTASSERT((DMAP_MIN_ADDRESS & ~L0_OFFSET) == DMAP_MIN_ADDRESS); CTASSERT((DMAP_MAX_ADDRESS & ~L0_OFFSET) == DMAP_MAX_ADDRESS); #define DMAP_TABLES ((DMAP_MAX_ADDRESS - DMAP_MIN_ADDRESS) >> L0_SHIFT) extern pt_entry_t pagetable_dmap[]; static SYSCTL_NODE(_vm, OID_AUTO, pmap, CTLFLAG_RD, 0, "VM/pmap parameters"); static int superpages_enabled = 0; SYSCTL_INT(_vm_pmap, OID_AUTO, superpages_enabled, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &superpages_enabled, 0, "Are large page mappings enabled?"); /* * Data for the pv entry allocation mechanism */ static TAILQ_HEAD(pch, pv_chunk) pv_chunks = TAILQ_HEAD_INITIALIZER(pv_chunks); static struct mtx pv_chunks_mutex; static struct rwlock pv_list_locks[NPV_LIST_LOCKS]; static void free_pv_chunk(struct pv_chunk *pc); static void free_pv_entry(pmap_t pmap, pv_entry_t pv); static pv_entry_t get_pv_entry(pmap_t pmap, struct rwlock **lockp); static vm_page_t reclaim_pv_chunk(pmap_t locked_pmap, struct rwlock **lockp); static void pmap_pvh_free(struct md_page *pvh, pmap_t pmap, vm_offset_t va); static pv_entry_t pmap_pvh_remove(struct md_page *pvh, pmap_t pmap, vm_offset_t va); static int pmap_change_attr(vm_offset_t va, vm_size_t size, int mode); static int pmap_change_attr_locked(vm_offset_t va, vm_size_t size, int mode); static pt_entry_t *pmap_demote_l1(pmap_t pmap, pt_entry_t *l1, vm_offset_t va); static pt_entry_t *pmap_demote_l2_locked(pmap_t pmap, pt_entry_t *l2, vm_offset_t va, struct rwlock **lockp); static pt_entry_t *pmap_demote_l2(pmap_t pmap, pt_entry_t *l2, vm_offset_t va); static vm_page_t pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, vm_page_t mpte, struct rwlock **lockp); static int pmap_remove_l3(pmap_t pmap, pt_entry_t *l3, vm_offset_t sva, pd_entry_t ptepde, struct spglist *free, struct rwlock **lockp); static boolean_t pmap_try_insert_pv_entry(pmap_t pmap, vm_offset_t va, vm_page_t m, struct rwlock **lockp); static vm_page_t _pmap_alloc_l3(pmap_t pmap, vm_pindex_t ptepindex, struct rwlock **lockp); static void _pmap_unwire_l3(pmap_t pmap, vm_offset_t va, vm_page_t m, struct spglist *free); static int pmap_unuse_l3(pmap_t, vm_offset_t, pd_entry_t, struct spglist *); /* * These load the old table data and store the new value. * They need to be atomic as the System MMU may write to the table at * the same time as the CPU. */ #define pmap_load_store(table, entry) atomic_swap_64(table, entry) #define pmap_set(table, mask) atomic_set_64(table, mask) #define pmap_load_clear(table) atomic_swap_64(table, 0) #define pmap_load(table) (*table) /********************/ /* Inline functions */ /********************/ static __inline void pagecopy(void *s, void *d) { memcpy(d, s, PAGE_SIZE); } #define pmap_l0_index(va) (((va) >> L0_SHIFT) & L0_ADDR_MASK) #define pmap_l1_index(va) (((va) >> L1_SHIFT) & Ln_ADDR_MASK) #define pmap_l2_index(va) (((va) >> L2_SHIFT) & Ln_ADDR_MASK) #define pmap_l3_index(va) (((va) >> L3_SHIFT) & Ln_ADDR_MASK) static __inline pd_entry_t * pmap_l0(pmap_t pmap, vm_offset_t va) { return (&pmap->pm_l0[pmap_l0_index(va)]); } static __inline pd_entry_t * pmap_l0_to_l1(pd_entry_t *l0, vm_offset_t va) { pd_entry_t *l1; l1 = (pd_entry_t *)PHYS_TO_DMAP(pmap_load(l0) & ~ATTR_MASK); return (&l1[pmap_l1_index(va)]); } static __inline pd_entry_t * pmap_l1(pmap_t pmap, vm_offset_t va) { pd_entry_t *l0; l0 = pmap_l0(pmap, va); if ((pmap_load(l0) & ATTR_DESCR_MASK) != L0_TABLE) return (NULL); return (pmap_l0_to_l1(l0, va)); } static __inline pd_entry_t * pmap_l1_to_l2(pd_entry_t *l1, vm_offset_t va) { pd_entry_t *l2; l2 = (pd_entry_t *)PHYS_TO_DMAP(pmap_load(l1) & ~ATTR_MASK); return (&l2[pmap_l2_index(va)]); } static __inline pd_entry_t * pmap_l2(pmap_t pmap, vm_offset_t va) { pd_entry_t *l1; l1 = pmap_l1(pmap, va); if ((pmap_load(l1) & ATTR_DESCR_MASK) != L1_TABLE) return (NULL); return (pmap_l1_to_l2(l1, va)); } static __inline pt_entry_t * pmap_l2_to_l3(pd_entry_t *l2, vm_offset_t va) { pt_entry_t *l3; l3 = (pd_entry_t *)PHYS_TO_DMAP(pmap_load(l2) & ~ATTR_MASK); return (&l3[pmap_l3_index(va)]); } /* * Returns the lowest valid pde for a given virtual address. * The next level may or may not point to a valid page or block. */ static __inline pd_entry_t * pmap_pde(pmap_t pmap, vm_offset_t va, int *level) { pd_entry_t *l0, *l1, *l2, desc; l0 = pmap_l0(pmap, va); desc = pmap_load(l0) & ATTR_DESCR_MASK; if (desc != L0_TABLE) { *level = -1; return (NULL); } l1 = pmap_l0_to_l1(l0, va); desc = pmap_load(l1) & ATTR_DESCR_MASK; if (desc != L1_TABLE) { *level = 0; return (l0); } l2 = pmap_l1_to_l2(l1, va); desc = pmap_load(l2) & ATTR_DESCR_MASK; if (desc != L2_TABLE) { *level = 1; return (l1); } *level = 2; return (l2); } /* * Returns the lowest valid pte block or table entry for a given virtual * address. If there are no valid entries return NULL and set the level to * the first invalid level. */ static __inline pt_entry_t * pmap_pte(pmap_t pmap, vm_offset_t va, int *level) { pd_entry_t *l1, *l2, desc; pt_entry_t *l3; l1 = pmap_l1(pmap, va); if (l1 == NULL) { *level = 0; return (NULL); } desc = pmap_load(l1) & ATTR_DESCR_MASK; if (desc == L1_BLOCK) { *level = 1; return (l1); } if (desc != L1_TABLE) { *level = 1; return (NULL); } l2 = pmap_l1_to_l2(l1, va); desc = pmap_load(l2) & ATTR_DESCR_MASK; if (desc == L2_BLOCK) { *level = 2; return (l2); } if (desc != L2_TABLE) { *level = 2; return (NULL); } *level = 3; l3 = pmap_l2_to_l3(l2, va); if ((pmap_load(l3) & ATTR_DESCR_MASK) != L3_PAGE) return (NULL); return (l3); } static inline bool pmap_superpages_enabled(void) { return (superpages_enabled != 0); } bool pmap_get_tables(pmap_t pmap, vm_offset_t va, pd_entry_t **l0, pd_entry_t **l1, pd_entry_t **l2, pt_entry_t **l3) { pd_entry_t *l0p, *l1p, *l2p; if (pmap->pm_l0 == NULL) return (false); l0p = pmap_l0(pmap, va); *l0 = l0p; if ((pmap_load(l0p) & ATTR_DESCR_MASK) != L0_TABLE) return (false); l1p = pmap_l0_to_l1(l0p, va); *l1 = l1p; if ((pmap_load(l1p) & ATTR_DESCR_MASK) == L1_BLOCK) { *l2 = NULL; *l3 = NULL; return (true); } if ((pmap_load(l1p) & ATTR_DESCR_MASK) != L1_TABLE) return (false); l2p = pmap_l1_to_l2(l1p, va); *l2 = l2p; if ((pmap_load(l2p) & ATTR_DESCR_MASK) == L2_BLOCK) { *l3 = NULL; return (true); } *l3 = pmap_l2_to_l3(l2p, va); return (true); } static __inline int pmap_is_current(pmap_t pmap) { return ((pmap == pmap_kernel()) || (pmap == curthread->td_proc->p_vmspace->vm_map.pmap)); } static __inline int pmap_l3_valid(pt_entry_t l3) { return ((l3 & ATTR_DESCR_MASK) == L3_PAGE); } /* Is a level 1 or 2entry a valid block and cacheable */ CTASSERT(L1_BLOCK == L2_BLOCK); static __inline int pmap_pte_valid_cacheable(pt_entry_t pte) { return (((pte & ATTR_DESCR_MASK) == L1_BLOCK) && ((pte & ATTR_IDX_MASK) == ATTR_IDX(CACHED_MEMORY))); } static __inline int pmap_l3_valid_cacheable(pt_entry_t l3) { return (((l3 & ATTR_DESCR_MASK) == L3_PAGE) && ((l3 & ATTR_IDX_MASK) == ATTR_IDX(CACHED_MEMORY))); } #define PTE_SYNC(pte) cpu_dcache_wb_range((vm_offset_t)pte, sizeof(*pte)) /* * Checks if the page is dirty. We currently lack proper tracking of this on * arm64 so for now assume is a page mapped as rw was accessed it is. */ static inline int pmap_page_dirty(pt_entry_t pte) { return ((pte & (ATTR_AF | ATTR_AP_RW_BIT)) == (ATTR_AF | ATTR_AP(ATTR_AP_RW))); } static __inline void pmap_resident_count_inc(pmap_t pmap, int count) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); pmap->pm_stats.resident_count += count; } static __inline void pmap_resident_count_dec(pmap_t pmap, int count) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); KASSERT(pmap->pm_stats.resident_count >= count, ("pmap %p resident count underflow %ld %d", pmap, pmap->pm_stats.resident_count, count)); pmap->pm_stats.resident_count -= count; } static pt_entry_t * pmap_early_page_idx(vm_offset_t l1pt, vm_offset_t va, u_int *l1_slot, u_int *l2_slot) { pt_entry_t *l2; pd_entry_t *l1; l1 = (pd_entry_t *)l1pt; *l1_slot = (va >> L1_SHIFT) & Ln_ADDR_MASK; /* Check locore has used a table L1 map */ KASSERT((l1[*l1_slot] & ATTR_DESCR_MASK) == L1_TABLE, ("Invalid bootstrap L1 table")); /* Find the address of the L2 table */ l2 = (pt_entry_t *)init_pt_va; *l2_slot = pmap_l2_index(va); return (l2); } static vm_paddr_t pmap_early_vtophys(vm_offset_t l1pt, vm_offset_t va) { u_int l1_slot, l2_slot; pt_entry_t *l2; l2 = pmap_early_page_idx(l1pt, va, &l1_slot, &l2_slot); return ((l2[l2_slot] & ~ATTR_MASK) + (va & L2_OFFSET)); } static void pmap_bootstrap_dmap(vm_offset_t kern_l1, vm_paddr_t min_pa, vm_paddr_t max_pa) { vm_offset_t va; vm_paddr_t pa; u_int l1_slot; pa = dmap_phys_base = min_pa & ~L1_OFFSET; va = DMAP_MIN_ADDRESS; for (; va < DMAP_MAX_ADDRESS && pa < max_pa; pa += L1_SIZE, va += L1_SIZE, l1_slot++) { l1_slot = ((va - DMAP_MIN_ADDRESS) >> L1_SHIFT); pmap_load_store(&pagetable_dmap[l1_slot], (pa & ~L1_OFFSET) | ATTR_DEFAULT | ATTR_IDX(CACHED_MEMORY) | L1_BLOCK); } /* Set the upper limit of the DMAP region */ dmap_phys_max = pa; dmap_max_addr = va; cpu_dcache_wb_range((vm_offset_t)pagetable_dmap, PAGE_SIZE * DMAP_TABLES); cpu_tlb_flushID(); } static vm_offset_t pmap_bootstrap_l2(vm_offset_t l1pt, vm_offset_t va, vm_offset_t l2_start) { vm_offset_t l2pt; vm_paddr_t pa; pd_entry_t *l1; u_int l1_slot; KASSERT((va & L1_OFFSET) == 0, ("Invalid virtual address")); l1 = (pd_entry_t *)l1pt; l1_slot = pmap_l1_index(va); l2pt = l2_start; for (; va < VM_MAX_KERNEL_ADDRESS; l1_slot++, va += L1_SIZE) { KASSERT(l1_slot < Ln_ENTRIES, ("Invalid L1 index")); pa = pmap_early_vtophys(l1pt, l2pt); pmap_load_store(&l1[l1_slot], (pa & ~Ln_TABLE_MASK) | L1_TABLE); l2pt += PAGE_SIZE; } /* Clean the L2 page table */ memset((void *)l2_start, 0, l2pt - l2_start); cpu_dcache_wb_range(l2_start, l2pt - l2_start); /* Flush the l1 table to ram */ cpu_dcache_wb_range((vm_offset_t)l1, PAGE_SIZE); return l2pt; } static vm_offset_t pmap_bootstrap_l3(vm_offset_t l1pt, vm_offset_t va, vm_offset_t l3_start) { vm_offset_t l2pt, l3pt; vm_paddr_t pa; pd_entry_t *l2; u_int l2_slot; KASSERT((va & L2_OFFSET) == 0, ("Invalid virtual address")); l2 = pmap_l2(kernel_pmap, va); l2 = (pd_entry_t *)rounddown2((uintptr_t)l2, PAGE_SIZE); l2pt = (vm_offset_t)l2; l2_slot = pmap_l2_index(va); l3pt = l3_start; for (; va < VM_MAX_KERNEL_ADDRESS; l2_slot++, va += L2_SIZE) { KASSERT(l2_slot < Ln_ENTRIES, ("Invalid L2 index")); pa = pmap_early_vtophys(l1pt, l3pt); pmap_load_store(&l2[l2_slot], (pa & ~Ln_TABLE_MASK) | L2_TABLE); l3pt += PAGE_SIZE; } /* Clean the L2 page table */ memset((void *)l3_start, 0, l3pt - l3_start); cpu_dcache_wb_range(l3_start, l3pt - l3_start); cpu_dcache_wb_range((vm_offset_t)l2, PAGE_SIZE); return l3pt; } /* * Bootstrap the system enough to run with virtual memory. */ void pmap_bootstrap(vm_offset_t l0pt, vm_offset_t l1pt, vm_paddr_t kernstart, vm_size_t kernlen) { u_int l1_slot, l2_slot, avail_slot, map_slot, used_map_slot; uint64_t kern_delta; pt_entry_t *l2; vm_offset_t va, freemempos; vm_offset_t dpcpu, msgbufpv; vm_paddr_t pa, max_pa, min_pa; int i; kern_delta = KERNBASE - kernstart; physmem = 0; printf("pmap_bootstrap %lx %lx %lx\n", l1pt, kernstart, kernlen); printf("%lx\n", l1pt); printf("%lx\n", (KERNBASE >> L1_SHIFT) & Ln_ADDR_MASK); /* Set this early so we can use the pagetable walking functions */ kernel_pmap_store.pm_l0 = (pd_entry_t *)l0pt; PMAP_LOCK_INIT(kernel_pmap); /* Assume the address we were loaded to is a valid physical address */ min_pa = max_pa = KERNBASE - kern_delta; /* * Find the minimum physical address. physmap is sorted, * but may contain empty ranges. */ for (i = 0; i < (physmap_idx * 2); i += 2) { if (physmap[i] == physmap[i + 1]) continue; if (physmap[i] <= min_pa) min_pa = physmap[i]; if (physmap[i + 1] > max_pa) max_pa = physmap[i + 1]; } /* Create a direct map region early so we can use it for pa -> va */ pmap_bootstrap_dmap(l1pt, min_pa, max_pa); va = KERNBASE; pa = KERNBASE - kern_delta; /* * Start to initialise phys_avail by copying from physmap * up to the physical address KERNBASE points at. */ map_slot = avail_slot = 0; for (; map_slot < (physmap_idx * 2) && avail_slot < (PHYS_AVAIL_SIZE - 2); map_slot += 2) { if (physmap[map_slot] == physmap[map_slot + 1]) continue; if (physmap[map_slot] <= pa && physmap[map_slot + 1] > pa) break; phys_avail[avail_slot] = physmap[map_slot]; phys_avail[avail_slot + 1] = physmap[map_slot + 1]; physmem += (phys_avail[avail_slot + 1] - phys_avail[avail_slot]) >> PAGE_SHIFT; avail_slot += 2; } /* Add the memory before the kernel */ if (physmap[avail_slot] < pa && avail_slot < (PHYS_AVAIL_SIZE - 2)) { phys_avail[avail_slot] = physmap[map_slot]; phys_avail[avail_slot + 1] = pa; physmem += (phys_avail[avail_slot + 1] - phys_avail[avail_slot]) >> PAGE_SHIFT; avail_slot += 2; } used_map_slot = map_slot; /* * Read the page table to find out what is already mapped. * This assumes we have mapped a block of memory from KERNBASE * using a single L1 entry. */ l2 = pmap_early_page_idx(l1pt, KERNBASE, &l1_slot, &l2_slot); /* Sanity check the index, KERNBASE should be the first VA */ KASSERT(l2_slot == 0, ("The L2 index is non-zero")); /* Find how many pages we have mapped */ for (; l2_slot < Ln_ENTRIES; l2_slot++) { if ((l2[l2_slot] & ATTR_DESCR_MASK) == 0) break; /* Check locore used L2 blocks */ KASSERT((l2[l2_slot] & ATTR_DESCR_MASK) == L2_BLOCK, ("Invalid bootstrap L2 table")); KASSERT((l2[l2_slot] & ~ATTR_MASK) == pa, ("Incorrect PA in L2 table")); va += L2_SIZE; pa += L2_SIZE; } va = roundup2(va, L1_SIZE); freemempos = KERNBASE + kernlen; freemempos = roundup2(freemempos, PAGE_SIZE); /* Create the l2 tables up to VM_MAX_KERNEL_ADDRESS */ freemempos = pmap_bootstrap_l2(l1pt, va, freemempos); /* And the l3 tables for the early devmap */ freemempos = pmap_bootstrap_l3(l1pt, VM_MAX_KERNEL_ADDRESS - L2_SIZE, freemempos); cpu_tlb_flushID(); #define alloc_pages(var, np) \ (var) = freemempos; \ freemempos += (np * PAGE_SIZE); \ memset((char *)(var), 0, ((np) * PAGE_SIZE)); /* Allocate dynamic per-cpu area. */ alloc_pages(dpcpu, DPCPU_SIZE / PAGE_SIZE); dpcpu_init((void *)dpcpu, 0); /* Allocate memory for the msgbuf, e.g. for /sbin/dmesg */ alloc_pages(msgbufpv, round_page(msgbufsize) / PAGE_SIZE); msgbufp = (void *)msgbufpv; virtual_avail = roundup2(freemempos, L1_SIZE); virtual_end = VM_MAX_KERNEL_ADDRESS - L2_SIZE; kernel_vm_end = virtual_avail; pa = pmap_early_vtophys(l1pt, freemempos); /* Finish initialising physmap */ map_slot = used_map_slot; for (; avail_slot < (PHYS_AVAIL_SIZE - 2) && map_slot < (physmap_idx * 2); map_slot += 2) { if (physmap[map_slot] == physmap[map_slot + 1]) continue; /* Have we used the current range? */ if (physmap[map_slot + 1] <= pa) continue; /* Do we need to split the entry? */ if (physmap[map_slot] < pa) { phys_avail[avail_slot] = pa; phys_avail[avail_slot + 1] = physmap[map_slot + 1]; } else { phys_avail[avail_slot] = physmap[map_slot]; phys_avail[avail_slot + 1] = physmap[map_slot + 1]; } physmem += (phys_avail[avail_slot + 1] - phys_avail[avail_slot]) >> PAGE_SHIFT; avail_slot += 2; } phys_avail[avail_slot] = 0; phys_avail[avail_slot + 1] = 0; /* * Maxmem isn't the "maximum memory", it's one larger than the * highest page of the physical address space. It should be * called something like "Maxphyspage". */ Maxmem = atop(phys_avail[avail_slot - 1]); cpu_tlb_flushID(); } /* * Initialize a vm_page's machine-dependent fields. */ void pmap_page_init(vm_page_t m) { TAILQ_INIT(&m->md.pv_list); m->md.pv_memattr = VM_MEMATTR_WRITE_BACK; } /* * Initialize the pmap module. * Called by vm_init, to initialize any structures that the pmap * system needs to map virtual memory. */ void pmap_init(void) { vm_size_t s; int i, pv_npg; /* * Are large page mappings enabled? */ TUNABLE_INT_FETCH("vm.pmap.superpages_enabled", &superpages_enabled); /* * Initialize the pv chunk list mutex. */ mtx_init(&pv_chunks_mutex, "pmap pv chunk list", NULL, MTX_DEF); /* * Initialize the pool of pv list locks. */ for (i = 0; i < NPV_LIST_LOCKS; i++) rw_init(&pv_list_locks[i], "pmap pv list"); /* * Calculate the size of the pv head table for superpages. */ pv_npg = howmany(vm_phys_segs[vm_phys_nsegs - 1].end, L2_SIZE); /* * Allocate memory for the pv head table for superpages. */ s = (vm_size_t)(pv_npg * sizeof(struct md_page)); s = round_page(s); pv_table = (struct md_page *)kmem_malloc(kernel_arena, s, M_WAITOK | M_ZERO); for (i = 0; i < pv_npg; i++) TAILQ_INIT(&pv_table[i].pv_list); TAILQ_INIT(&pv_dummy.pv_list); } static SYSCTL_NODE(_vm_pmap, OID_AUTO, l2, CTLFLAG_RD, 0, "2MB page mapping counters"); static u_long pmap_l2_demotions; SYSCTL_ULONG(_vm_pmap_l2, OID_AUTO, demotions, CTLFLAG_RD, &pmap_l2_demotions, 0, "2MB page demotions"); static u_long pmap_l2_p_failures; SYSCTL_ULONG(_vm_pmap_l2, OID_AUTO, p_failures, CTLFLAG_RD, &pmap_l2_p_failures, 0, "2MB page promotion failures"); static u_long pmap_l2_promotions; SYSCTL_ULONG(_vm_pmap_l2, OID_AUTO, promotions, CTLFLAG_RD, &pmap_l2_promotions, 0, "2MB page promotions"); /* * Invalidate a single TLB entry. */ PMAP_INLINE void pmap_invalidate_page(pmap_t pmap, vm_offset_t va) { sched_pin(); __asm __volatile( "dsb ishst \n" "tlbi vaae1is, %0 \n" "dsb ish \n" "isb \n" : : "r"(va >> PAGE_SHIFT)); sched_unpin(); } PMAP_INLINE void pmap_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { vm_offset_t addr; sched_pin(); dsb(ishst); for (addr = sva; addr < eva; addr += PAGE_SIZE) { __asm __volatile( "tlbi vaae1is, %0" : : "r"(addr >> PAGE_SHIFT)); } __asm __volatile( "dsb ish \n" "isb \n"); sched_unpin(); } PMAP_INLINE void pmap_invalidate_all(pmap_t pmap) { sched_pin(); __asm __volatile( "dsb ishst \n" "tlbi vmalle1is \n" "dsb ish \n" "isb \n"); sched_unpin(); } /* * Routine: pmap_extract * Function: * Extract the physical page address associated * with the given map/virtual_address pair. */ vm_paddr_t pmap_extract(pmap_t pmap, vm_offset_t va) { pt_entry_t *pte, tpte; vm_paddr_t pa; int lvl; pa = 0; PMAP_LOCK(pmap); /* * Find the block or page map for this virtual address. pmap_pte * will return either a valid block/page entry, or NULL. */ pte = pmap_pte(pmap, va, &lvl); if (pte != NULL) { tpte = pmap_load(pte); pa = tpte & ~ATTR_MASK; switch(lvl) { case 1: KASSERT((tpte & ATTR_DESCR_MASK) == L1_BLOCK, ("pmap_extract: Invalid L1 pte found: %lx", tpte & ATTR_DESCR_MASK)); pa |= (va & L1_OFFSET); break; case 2: KASSERT((tpte & ATTR_DESCR_MASK) == L2_BLOCK, ("pmap_extract: Invalid L2 pte found: %lx", tpte & ATTR_DESCR_MASK)); pa |= (va & L2_OFFSET); break; case 3: KASSERT((tpte & ATTR_DESCR_MASK) == L3_PAGE, ("pmap_extract: Invalid L3 pte found: %lx", tpte & ATTR_DESCR_MASK)); pa |= (va & L3_OFFSET); break; } } PMAP_UNLOCK(pmap); return (pa); } /* * Routine: pmap_extract_and_hold * Function: * Atomically extract and hold the physical page * with the given pmap and virtual address pair * if that mapping permits the given protection. */ vm_page_t pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, vm_prot_t prot) { pt_entry_t *pte, tpte; vm_offset_t off; vm_paddr_t pa; vm_page_t m; int lvl; pa = 0; m = NULL; PMAP_LOCK(pmap); retry: pte = pmap_pte(pmap, va, &lvl); if (pte != NULL) { tpte = pmap_load(pte); KASSERT(lvl > 0 && lvl <= 3, ("pmap_extract_and_hold: Invalid level %d", lvl)); CTASSERT(L1_BLOCK == L2_BLOCK); KASSERT((lvl == 3 && (tpte & ATTR_DESCR_MASK) == L3_PAGE) || (lvl < 3 && (tpte & ATTR_DESCR_MASK) == L1_BLOCK), ("pmap_extract_and_hold: Invalid pte at L%d: %lx", lvl, tpte & ATTR_DESCR_MASK)); if (((tpte & ATTR_AP_RW_BIT) == ATTR_AP(ATTR_AP_RW)) || ((prot & VM_PROT_WRITE) == 0)) { switch(lvl) { case 1: off = va & L1_OFFSET; break; case 2: off = va & L2_OFFSET; break; case 3: default: off = 0; } if (vm_page_pa_tryrelock(pmap, (tpte & ~ATTR_MASK) | off, &pa)) goto retry; m = PHYS_TO_VM_PAGE((tpte & ~ATTR_MASK) | off); vm_page_hold(m); } } PA_UNLOCK_COND(pa); PMAP_UNLOCK(pmap); return (m); } vm_paddr_t pmap_kextract(vm_offset_t va) { pt_entry_t *pte, tpte; vm_paddr_t pa; int lvl; if (va >= DMAP_MIN_ADDRESS && va < DMAP_MAX_ADDRESS) { pa = DMAP_TO_PHYS(va); } else { pa = 0; pte = pmap_pte(kernel_pmap, va, &lvl); if (pte != NULL) { tpte = pmap_load(pte); pa = tpte & ~ATTR_MASK; switch(lvl) { case 1: KASSERT((tpte & ATTR_DESCR_MASK) == L1_BLOCK, ("pmap_kextract: Invalid L1 pte found: %lx", tpte & ATTR_DESCR_MASK)); pa |= (va & L1_OFFSET); break; case 2: KASSERT((tpte & ATTR_DESCR_MASK) == L2_BLOCK, ("pmap_kextract: Invalid L2 pte found: %lx", tpte & ATTR_DESCR_MASK)); pa |= (va & L2_OFFSET); break; case 3: KASSERT((tpte & ATTR_DESCR_MASK) == L3_PAGE, ("pmap_kextract: Invalid L3 pte found: %lx", tpte & ATTR_DESCR_MASK)); pa |= (va & L3_OFFSET); break; } } } return (pa); } /*************************************************** * Low level mapping routines..... ***************************************************/ static void pmap_kenter(vm_offset_t sva, vm_size_t size, vm_paddr_t pa, int mode) { pd_entry_t *pde; pt_entry_t *pte; vm_offset_t va; int lvl; KASSERT((pa & L3_OFFSET) == 0, ("pmap_kenter: Invalid physical address")); KASSERT((sva & L3_OFFSET) == 0, ("pmap_kenter: Invalid virtual address")); KASSERT((size & PAGE_MASK) == 0, ("pmap_kenter: Mapping is not page-sized")); va = sva; while (size != 0) { pde = pmap_pde(kernel_pmap, va, &lvl); KASSERT(pde != NULL, ("pmap_kenter: Invalid page entry, va: 0x%lx", va)); KASSERT(lvl == 2, ("pmap_kenter: Invalid level %d", lvl)); pte = pmap_l2_to_l3(pde, va); pmap_load_store(pte, (pa & ~L3_OFFSET) | ATTR_DEFAULT | ATTR_IDX(mode) | L3_PAGE); PTE_SYNC(pte); va += PAGE_SIZE; pa += PAGE_SIZE; size -= PAGE_SIZE; } pmap_invalidate_range(kernel_pmap, sva, va); } void pmap_kenter_device(vm_offset_t sva, vm_size_t size, vm_paddr_t pa) { pmap_kenter(sva, size, pa, DEVICE_MEMORY); } /* * Remove a page from the kernel pagetables. */ PMAP_INLINE void pmap_kremove(vm_offset_t va) { pt_entry_t *pte; int lvl; pte = pmap_pte(kernel_pmap, va, &lvl); KASSERT(pte != NULL, ("pmap_kremove: Invalid address")); KASSERT(lvl == 3, ("pmap_kremove: Invalid pte level %d", lvl)); if (pmap_l3_valid_cacheable(pmap_load(pte))) cpu_dcache_wb_range(va, L3_SIZE); pmap_load_clear(pte); PTE_SYNC(pte); pmap_invalidate_page(kernel_pmap, va); } void pmap_kremove_device(vm_offset_t sva, vm_size_t size) { pt_entry_t *pte; vm_offset_t va; int lvl; KASSERT((sva & L3_OFFSET) == 0, ("pmap_kremove_device: Invalid virtual address")); KASSERT((size & PAGE_MASK) == 0, ("pmap_kremove_device: Mapping is not page-sized")); va = sva; while (size != 0) { pte = pmap_pte(kernel_pmap, va, &lvl); KASSERT(pte != NULL, ("Invalid page table, va: 0x%lx", va)); KASSERT(lvl == 3, ("Invalid device pagetable level: %d != 3", lvl)); pmap_load_clear(pte); PTE_SYNC(pte); va += PAGE_SIZE; size -= PAGE_SIZE; } pmap_invalidate_range(kernel_pmap, sva, va); } /* * Used to map a range of physical addresses into kernel * virtual address space. * * The value passed in '*virt' is a suggested virtual address for * the mapping. Architectures which can support a direct-mapped * physical to virtual region can return the appropriate address * within that region, leaving '*virt' unchanged. Other * architectures should map the pages starting at '*virt' and * update '*virt' with the first usable address after the mapped * region. */ vm_offset_t pmap_map(vm_offset_t *virt, vm_paddr_t start, vm_paddr_t end, int prot) { return PHYS_TO_DMAP(start); } /* * Add a list of wired pages to the kva * this routine is only used for temporary * kernel mappings that do not need to have * page modification or references recorded. * Note that old mappings are simply written * over. The page *must* be wired. * Note: SMP coherent. Uses a ranged shootdown IPI. */ void pmap_qenter(vm_offset_t sva, vm_page_t *ma, int count) { pd_entry_t *pde; pt_entry_t *pte, pa; vm_offset_t va; vm_page_t m; int i, lvl; va = sva; for (i = 0; i < count; i++) { pde = pmap_pde(kernel_pmap, va, &lvl); KASSERT(pde != NULL, ("pmap_qenter: Invalid page entry, va: 0x%lx", va)); KASSERT(lvl == 2, ("pmap_qenter: Invalid level %d", lvl)); m = ma[i]; pa = VM_PAGE_TO_PHYS(m) | ATTR_DEFAULT | ATTR_AP(ATTR_AP_RW) | ATTR_IDX(m->md.pv_memattr) | L3_PAGE; pte = pmap_l2_to_l3(pde, va); pmap_load_store(pte, pa); PTE_SYNC(pte); va += L3_SIZE; } pmap_invalidate_range(kernel_pmap, sva, va); } /* * This routine tears out page mappings from the * kernel -- it is meant only for temporary mappings. */ void pmap_qremove(vm_offset_t sva, int count) { pt_entry_t *pte; vm_offset_t va; int lvl; KASSERT(sva >= VM_MIN_KERNEL_ADDRESS, ("usermode va %lx", sva)); va = sva; while (count-- > 0) { pte = pmap_pte(kernel_pmap, va, &lvl); KASSERT(lvl == 3, ("Invalid device pagetable level: %d != 3", lvl)); if (pte != NULL) { if (pmap_l3_valid_cacheable(pmap_load(pte))) cpu_dcache_wb_range(va, L3_SIZE); pmap_load_clear(pte); PTE_SYNC(pte); } va += PAGE_SIZE; } pmap_invalidate_range(kernel_pmap, sva, va); } /*************************************************** * Page table page management routines..... ***************************************************/ static __inline void pmap_free_zero_pages(struct spglist *free) { vm_page_t m; while ((m = SLIST_FIRST(free)) != NULL) { SLIST_REMOVE_HEAD(free, plinks.s.ss); /* Preserve the page's PG_ZERO setting. */ vm_page_free_toq(m); } } /* * Schedule the specified unused page table page to be freed. Specifically, * add the page to the specified list of pages that will be released to the * physical memory manager after the TLB has been updated. */ static __inline void pmap_add_delayed_free_list(vm_page_t m, struct spglist *free, boolean_t set_PG_ZERO) { if (set_PG_ZERO) m->flags |= PG_ZERO; else m->flags &= ~PG_ZERO; SLIST_INSERT_HEAD(free, m, plinks.s.ss); } /* * Decrements a page table page's wire count, which is used to record the * number of valid page table entries within the page. If the wire count * drops to zero, then the page table page is unmapped. Returns TRUE if the * page table page was unmapped and FALSE otherwise. */ static inline boolean_t pmap_unwire_l3(pmap_t pmap, vm_offset_t va, vm_page_t m, struct spglist *free) { --m->wire_count; if (m->wire_count == 0) { _pmap_unwire_l3(pmap, va, m, free); return (TRUE); } else return (FALSE); } static void _pmap_unwire_l3(pmap_t pmap, vm_offset_t va, vm_page_t m, struct spglist *free) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* * unmap the page table page */ if (m->pindex >= (NUL2E + NUL1E)) { /* l1 page */ pd_entry_t *l0; l0 = pmap_l0(pmap, va); pmap_load_clear(l0); PTE_SYNC(l0); } else if (m->pindex >= NUL2E) { /* l2 page */ pd_entry_t *l1; l1 = pmap_l1(pmap, va); pmap_load_clear(l1); PTE_SYNC(l1); } else { /* l3 page */ pd_entry_t *l2; l2 = pmap_l2(pmap, va); pmap_load_clear(l2); PTE_SYNC(l2); } pmap_resident_count_dec(pmap, 1); if (m->pindex < NUL2E) { /* We just released an l3, unhold the matching l2 */ pd_entry_t *l1, tl1; vm_page_t l2pg; l1 = pmap_l1(pmap, va); tl1 = pmap_load(l1); l2pg = PHYS_TO_VM_PAGE(tl1 & ~ATTR_MASK); pmap_unwire_l3(pmap, va, l2pg, free); } else if (m->pindex < (NUL2E + NUL1E)) { /* We just released an l2, unhold the matching l1 */ pd_entry_t *l0, tl0; vm_page_t l1pg; l0 = pmap_l0(pmap, va); tl0 = pmap_load(l0); l1pg = PHYS_TO_VM_PAGE(tl0 & ~ATTR_MASK); pmap_unwire_l3(pmap, va, l1pg, free); } pmap_invalidate_page(pmap, va); /* * This is a release store so that the ordinary store unmapping * the page table page is globally performed before TLB shoot- * down is begun. */ atomic_subtract_rel_int(&vm_cnt.v_wire_count, 1); /* * Put page on a list so that it is released after * *ALL* TLB shootdown is done */ pmap_add_delayed_free_list(m, free, TRUE); } /* * After removing an l3 entry, this routine is used to * conditionally free the page, and manage the hold/wire counts. */ static int pmap_unuse_l3(pmap_t pmap, vm_offset_t va, pd_entry_t ptepde, struct spglist *free) { vm_page_t mpte; if (va >= VM_MAXUSER_ADDRESS) return (0); KASSERT(ptepde != 0, ("pmap_unuse_pt: ptepde != 0")); mpte = PHYS_TO_VM_PAGE(ptepde & ~ATTR_MASK); return (pmap_unwire_l3(pmap, va, mpte, free)); } void pmap_pinit0(pmap_t pmap) { PMAP_LOCK_INIT(pmap); bzero(&pmap->pm_stats, sizeof(pmap->pm_stats)); pmap->pm_l0 = kernel_pmap->pm_l0; pmap->pm_root.rt_root = 0; } int pmap_pinit(pmap_t pmap) { vm_paddr_t l0phys; vm_page_t l0pt; /* * allocate the l0 page */ while ((l0pt = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO)) == NULL) VM_WAIT; l0phys = VM_PAGE_TO_PHYS(l0pt); pmap->pm_l0 = (pd_entry_t *)PHYS_TO_DMAP(l0phys); if ((l0pt->flags & PG_ZERO) == 0) pagezero(pmap->pm_l0); pmap->pm_root.rt_root = 0; bzero(&pmap->pm_stats, sizeof(pmap->pm_stats)); return (1); } /* * This routine is called if the desired page table page does not exist. * * If page table page allocation fails, this routine may sleep before * returning NULL. It sleeps only if a lock pointer was given. * * Note: If a page allocation fails at page table level two or three, * one or two pages may be held during the wait, only to be released * afterwards. This conservative approach is easily argued to avoid * race conditions. */ static vm_page_t _pmap_alloc_l3(pmap_t pmap, vm_pindex_t ptepindex, struct rwlock **lockp) { vm_page_t m, l1pg, l2pg; PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* * Allocate a page table page. */ if ((m = vm_page_alloc(NULL, ptepindex, VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO)) == NULL) { if (lockp != NULL) { RELEASE_PV_LIST_LOCK(lockp); PMAP_UNLOCK(pmap); VM_WAIT; PMAP_LOCK(pmap); } /* * Indicate the need to retry. While waiting, the page table * page may have been allocated. */ return (NULL); } if ((m->flags & PG_ZERO) == 0) pmap_zero_page(m); /* * Map the pagetable page into the process address space, if * it isn't already there. */ if (ptepindex >= (NUL2E + NUL1E)) { pd_entry_t *l0; vm_pindex_t l0index; l0index = ptepindex - (NUL2E + NUL1E); l0 = &pmap->pm_l0[l0index]; pmap_load_store(l0, VM_PAGE_TO_PHYS(m) | L0_TABLE); PTE_SYNC(l0); } else if (ptepindex >= NUL2E) { vm_pindex_t l0index, l1index; pd_entry_t *l0, *l1; pd_entry_t tl0; l1index = ptepindex - NUL2E; l0index = l1index >> L0_ENTRIES_SHIFT; l0 = &pmap->pm_l0[l0index]; tl0 = pmap_load(l0); if (tl0 == 0) { /* recurse for allocating page dir */ if (_pmap_alloc_l3(pmap, NUL2E + NUL1E + l0index, lockp) == NULL) { --m->wire_count; /* XXX: release mem barrier? */ atomic_subtract_int(&vm_cnt.v_wire_count, 1); vm_page_free_zero(m); return (NULL); } } else { l1pg = PHYS_TO_VM_PAGE(tl0 & ~ATTR_MASK); l1pg->wire_count++; } l1 = (pd_entry_t *)PHYS_TO_DMAP(pmap_load(l0) & ~ATTR_MASK); l1 = &l1[ptepindex & Ln_ADDR_MASK]; pmap_load_store(l1, VM_PAGE_TO_PHYS(m) | L1_TABLE); PTE_SYNC(l1); } else { vm_pindex_t l0index, l1index; pd_entry_t *l0, *l1, *l2; pd_entry_t tl0, tl1; l1index = ptepindex >> Ln_ENTRIES_SHIFT; l0index = l1index >> L0_ENTRIES_SHIFT; l0 = &pmap->pm_l0[l0index]; tl0 = pmap_load(l0); if (tl0 == 0) { /* recurse for allocating page dir */ if (_pmap_alloc_l3(pmap, NUL2E + l1index, lockp) == NULL) { --m->wire_count; atomic_subtract_int(&vm_cnt.v_wire_count, 1); vm_page_free_zero(m); return (NULL); } tl0 = pmap_load(l0); l1 = (pd_entry_t *)PHYS_TO_DMAP(tl0 & ~ATTR_MASK); l1 = &l1[l1index & Ln_ADDR_MASK]; } else { l1 = (pd_entry_t *)PHYS_TO_DMAP(tl0 & ~ATTR_MASK); l1 = &l1[l1index & Ln_ADDR_MASK]; tl1 = pmap_load(l1); if (tl1 == 0) { /* recurse for allocating page dir */ if (_pmap_alloc_l3(pmap, NUL2E + l1index, lockp) == NULL) { --m->wire_count; /* XXX: release mem barrier? */ atomic_subtract_int( &vm_cnt.v_wire_count, 1); vm_page_free_zero(m); return (NULL); } } else { l2pg = PHYS_TO_VM_PAGE(tl1 & ~ATTR_MASK); l2pg->wire_count++; } } l2 = (pd_entry_t *)PHYS_TO_DMAP(pmap_load(l1) & ~ATTR_MASK); l2 = &l2[ptepindex & Ln_ADDR_MASK]; pmap_load_store(l2, VM_PAGE_TO_PHYS(m) | L2_TABLE); PTE_SYNC(l2); } pmap_resident_count_inc(pmap, 1); return (m); } static vm_page_t pmap_alloc_l3(pmap_t pmap, vm_offset_t va, struct rwlock **lockp) { vm_pindex_t ptepindex; pd_entry_t *pde, tpde; #ifdef INVARIANTS pt_entry_t *pte; #endif vm_page_t m; int lvl; /* * Calculate pagetable page index */ ptepindex = pmap_l2_pindex(va); retry: /* * Get the page directory entry */ pde = pmap_pde(pmap, va, &lvl); /* * If the page table page is mapped, we just increment the hold count, * and activate it. If we get a level 2 pde it will point to a level 3 * table. */ switch (lvl) { case -1: break; case 0: #ifdef INVARIANTS pte = pmap_l0_to_l1(pde, va); KASSERT(pmap_load(pte) == 0, ("pmap_alloc_l3: TODO: l0 superpages")); #endif break; case 1: #ifdef INVARIANTS pte = pmap_l1_to_l2(pde, va); KASSERT(pmap_load(pte) == 0, ("pmap_alloc_l3: TODO: l1 superpages")); #endif break; case 2: tpde = pmap_load(pde); if (tpde != 0) { m = PHYS_TO_VM_PAGE(tpde & ~ATTR_MASK); m->wire_count++; return (m); } break; default: panic("pmap_alloc_l3: Invalid level %d", lvl); } /* * Here if the pte page isn't mapped, or if it has been deallocated. */ m = _pmap_alloc_l3(pmap, ptepindex, lockp); if (m == NULL && lockp != NULL) goto retry; return (m); } /*************************************************** * Pmap allocation/deallocation routines. ***************************************************/ /* * Release any resources held by the given physical map. * Called when a pmap initialized by pmap_pinit is being released. * Should only be called if the map contains no valid mappings. */ void pmap_release(pmap_t pmap) { vm_page_t m; KASSERT(pmap->pm_stats.resident_count == 0, ("pmap_release: pmap resident count %ld != 0", pmap->pm_stats.resident_count)); KASSERT(vm_radix_is_empty(&pmap->pm_root), ("pmap_release: pmap has reserved page table page(s)")); m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pmap->pm_l0)); m->wire_count--; atomic_subtract_int(&vm_cnt.v_wire_count, 1); vm_page_free_zero(m); } static int kvm_size(SYSCTL_HANDLER_ARGS) { unsigned long ksize = VM_MAX_KERNEL_ADDRESS - VM_MIN_KERNEL_ADDRESS; return sysctl_handle_long(oidp, &ksize, 0, req); } SYSCTL_PROC(_vm, OID_AUTO, kvm_size, CTLTYPE_LONG|CTLFLAG_RD, 0, 0, kvm_size, "LU", "Size of KVM"); static int kvm_free(SYSCTL_HANDLER_ARGS) { unsigned long kfree = VM_MAX_KERNEL_ADDRESS - kernel_vm_end; return sysctl_handle_long(oidp, &kfree, 0, req); } SYSCTL_PROC(_vm, OID_AUTO, kvm_free, CTLTYPE_LONG|CTLFLAG_RD, 0, 0, kvm_free, "LU", "Amount of KVM free"); /* * grow the number of kernel page table entries, if needed */ void pmap_growkernel(vm_offset_t addr) { vm_paddr_t paddr; vm_page_t nkpg; pd_entry_t *l0, *l1, *l2; mtx_assert(&kernel_map->system_mtx, MA_OWNED); addr = roundup2(addr, L2_SIZE); if (addr - 1 >= kernel_map->max_offset) addr = kernel_map->max_offset; while (kernel_vm_end < addr) { l0 = pmap_l0(kernel_pmap, kernel_vm_end); KASSERT(pmap_load(l0) != 0, ("pmap_growkernel: No level 0 kernel entry")); l1 = pmap_l0_to_l1(l0, kernel_vm_end); if (pmap_load(l1) == 0) { /* We need a new PDP entry */ nkpg = vm_page_alloc(NULL, kernel_vm_end >> L1_SHIFT, VM_ALLOC_INTERRUPT | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO); if (nkpg == NULL) panic("pmap_growkernel: no memory to grow kernel"); if ((nkpg->flags & PG_ZERO) == 0) pmap_zero_page(nkpg); paddr = VM_PAGE_TO_PHYS(nkpg); pmap_load_store(l1, paddr | L1_TABLE); PTE_SYNC(l1); continue; /* try again */ } l2 = pmap_l1_to_l2(l1, kernel_vm_end); if ((pmap_load(l2) & ATTR_AF) != 0) { kernel_vm_end = (kernel_vm_end + L2_SIZE) & ~L2_OFFSET; if (kernel_vm_end - 1 >= kernel_map->max_offset) { kernel_vm_end = kernel_map->max_offset; break; } continue; } nkpg = vm_page_alloc(NULL, kernel_vm_end >> L2_SHIFT, VM_ALLOC_INTERRUPT | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO); if (nkpg == NULL) panic("pmap_growkernel: no memory to grow kernel"); if ((nkpg->flags & PG_ZERO) == 0) pmap_zero_page(nkpg); paddr = VM_PAGE_TO_PHYS(nkpg); pmap_load_store(l2, paddr | L2_TABLE); PTE_SYNC(l2); pmap_invalidate_page(kernel_pmap, kernel_vm_end); kernel_vm_end = (kernel_vm_end + L2_SIZE) & ~L2_OFFSET; if (kernel_vm_end - 1 >= kernel_map->max_offset) { kernel_vm_end = kernel_map->max_offset; break; } } } /*************************************************** * page management routines. ***************************************************/ CTASSERT(sizeof(struct pv_chunk) == PAGE_SIZE); CTASSERT(_NPCM == 3); CTASSERT(_NPCPV == 168); static __inline struct pv_chunk * pv_to_chunk(pv_entry_t pv) { return ((struct pv_chunk *)((uintptr_t)pv & ~(uintptr_t)PAGE_MASK)); } #define PV_PMAP(pv) (pv_to_chunk(pv)->pc_pmap) #define PC_FREE0 0xfffffffffffffffful #define PC_FREE1 0xfffffffffffffffful #define PC_FREE2 0x000000fffffffffful static const uint64_t pc_freemask[_NPCM] = { PC_FREE0, PC_FREE1, PC_FREE2 }; #if 0 #ifdef PV_STATS static int pc_chunk_count, pc_chunk_allocs, pc_chunk_frees, pc_chunk_tryfail; SYSCTL_INT(_vm_pmap, OID_AUTO, pc_chunk_count, CTLFLAG_RD, &pc_chunk_count, 0, "Current number of pv entry chunks"); SYSCTL_INT(_vm_pmap, OID_AUTO, pc_chunk_allocs, CTLFLAG_RD, &pc_chunk_allocs, 0, "Current number of pv entry chunks allocated"); SYSCTL_INT(_vm_pmap, OID_AUTO, pc_chunk_frees, CTLFLAG_RD, &pc_chunk_frees, 0, "Current number of pv entry chunks frees"); SYSCTL_INT(_vm_pmap, OID_AUTO, pc_chunk_tryfail, CTLFLAG_RD, &pc_chunk_tryfail, 0, "Number of times tried to get a chunk page but failed."); static long pv_entry_frees, pv_entry_allocs, pv_entry_count; static int pv_entry_spare; SYSCTL_LONG(_vm_pmap, OID_AUTO, pv_entry_frees, CTLFLAG_RD, &pv_entry_frees, 0, "Current number of pv entry frees"); SYSCTL_LONG(_vm_pmap, OID_AUTO, pv_entry_allocs, CTLFLAG_RD, &pv_entry_allocs, 0, "Current number of pv entry allocs"); SYSCTL_LONG(_vm_pmap, OID_AUTO, pv_entry_count, CTLFLAG_RD, &pv_entry_count, 0, "Current number of pv entries"); SYSCTL_INT(_vm_pmap, OID_AUTO, pv_entry_spare, CTLFLAG_RD, &pv_entry_spare, 0, "Current number of spare pv entries"); #endif #endif /* 0 */ /* * We are in a serious low memory condition. Resort to * drastic measures to free some pages so we can allocate * another pv entry chunk. * * Returns NULL if PV entries were reclaimed from the specified pmap. * * We do not, however, unmap 2mpages because subsequent accesses will * allocate per-page pv entries until repromotion occurs, thereby * exacerbating the shortage of free pv entries. */ static vm_page_t reclaim_pv_chunk(pmap_t locked_pmap, struct rwlock **lockp) { panic("ARM64TODO: reclaim_pv_chunk"); } /* * free the pv_entry back to the free list */ static void free_pv_entry(pmap_t pmap, pv_entry_t pv) { struct pv_chunk *pc; int idx, field, bit; PMAP_LOCK_ASSERT(pmap, MA_OWNED); PV_STAT(atomic_add_long(&pv_entry_frees, 1)); PV_STAT(atomic_add_int(&pv_entry_spare, 1)); PV_STAT(atomic_subtract_long(&pv_entry_count, 1)); pc = pv_to_chunk(pv); idx = pv - &pc->pc_pventry[0]; field = idx / 64; bit = idx % 64; pc->pc_map[field] |= 1ul << bit; if (pc->pc_map[0] != PC_FREE0 || pc->pc_map[1] != PC_FREE1 || pc->pc_map[2] != PC_FREE2) { /* 98% of the time, pc is already at the head of the list. */ if (__predict_false(pc != TAILQ_FIRST(&pmap->pm_pvchunk))) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list); } return; } TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); free_pv_chunk(pc); } static void free_pv_chunk(struct pv_chunk *pc) { vm_page_t m; mtx_lock(&pv_chunks_mutex); TAILQ_REMOVE(&pv_chunks, pc, pc_lru); mtx_unlock(&pv_chunks_mutex); PV_STAT(atomic_subtract_int(&pv_entry_spare, _NPCPV)); PV_STAT(atomic_subtract_int(&pc_chunk_count, 1)); PV_STAT(atomic_add_int(&pc_chunk_frees, 1)); /* entire chunk is free, return it */ m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pc)); dump_drop_page(m->phys_addr); vm_page_unwire(m, PQ_NONE); vm_page_free(m); } /* * Returns a new PV entry, allocating a new PV chunk from the system when * needed. If this PV chunk allocation fails and a PV list lock pointer was * given, a PV chunk is reclaimed from an arbitrary pmap. Otherwise, NULL is * returned. * * The given PV list lock may be released. */ static pv_entry_t get_pv_entry(pmap_t pmap, struct rwlock **lockp) { int bit, field; pv_entry_t pv; struct pv_chunk *pc; vm_page_t m; PMAP_LOCK_ASSERT(pmap, MA_OWNED); PV_STAT(atomic_add_long(&pv_entry_allocs, 1)); retry: pc = TAILQ_FIRST(&pmap->pm_pvchunk); if (pc != NULL) { for (field = 0; field < _NPCM; field++) { if (pc->pc_map[field]) { bit = ffsl(pc->pc_map[field]) - 1; break; } } if (field < _NPCM) { pv = &pc->pc_pventry[field * 64 + bit]; pc->pc_map[field] &= ~(1ul << bit); /* If this was the last item, move it to tail */ if (pc->pc_map[0] == 0 && pc->pc_map[1] == 0 && pc->pc_map[2] == 0) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&pmap->pm_pvchunk, pc, pc_list); } PV_STAT(atomic_add_long(&pv_entry_count, 1)); PV_STAT(atomic_subtract_int(&pv_entry_spare, 1)); return (pv); } } /* No free items, allocate another chunk */ m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED); if (m == NULL) { if (lockp == NULL) { PV_STAT(pc_chunk_tryfail++); return (NULL); } m = reclaim_pv_chunk(pmap, lockp); if (m == NULL) goto retry; } PV_STAT(atomic_add_int(&pc_chunk_count, 1)); PV_STAT(atomic_add_int(&pc_chunk_allocs, 1)); dump_add_page(m->phys_addr); pc = (void *)PHYS_TO_DMAP(m->phys_addr); pc->pc_pmap = pmap; pc->pc_map[0] = PC_FREE0 & ~1ul; /* preallocated bit 0 */ pc->pc_map[1] = PC_FREE1; pc->pc_map[2] = PC_FREE2; mtx_lock(&pv_chunks_mutex); TAILQ_INSERT_TAIL(&pv_chunks, pc, pc_lru); mtx_unlock(&pv_chunks_mutex); pv = &pc->pc_pventry[0]; TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list); PV_STAT(atomic_add_long(&pv_entry_count, 1)); PV_STAT(atomic_add_int(&pv_entry_spare, _NPCPV - 1)); return (pv); } /* * Ensure that the number of spare PV entries in the specified pmap meets or * exceeds the given count, "needed". * * The given PV list lock may be released. */ static void reserve_pv_entries(pmap_t pmap, int needed, struct rwlock **lockp) { struct pch new_tail; struct pv_chunk *pc; int avail, free; vm_page_t m; PMAP_LOCK_ASSERT(pmap, MA_OWNED); KASSERT(lockp != NULL, ("reserve_pv_entries: lockp is NULL")); /* * Newly allocated PV chunks must be stored in a private list until * the required number of PV chunks have been allocated. Otherwise, * reclaim_pv_chunk() could recycle one of these chunks. In * contrast, these chunks must be added to the pmap upon allocation. */ TAILQ_INIT(&new_tail); retry: avail = 0; TAILQ_FOREACH(pc, &pmap->pm_pvchunk, pc_list) { bit_count((bitstr_t *)pc->pc_map, 0, sizeof(pc->pc_map) * NBBY, &free); if (free == 0) break; avail += free; if (avail >= needed) break; } for (; avail < needed; avail += _NPCPV) { m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED); if (m == NULL) { m = reclaim_pv_chunk(pmap, lockp); if (m == NULL) goto retry; } PV_STAT(atomic_add_int(&pc_chunk_count, 1)); PV_STAT(atomic_add_int(&pc_chunk_allocs, 1)); dump_add_page(m->phys_addr); pc = (void *)PHYS_TO_DMAP(m->phys_addr); pc->pc_pmap = pmap; pc->pc_map[0] = PC_FREE0; pc->pc_map[1] = PC_FREE1; pc->pc_map[2] = PC_FREE2; TAILQ_INSERT_HEAD(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&new_tail, pc, pc_lru); PV_STAT(atomic_add_int(&pv_entry_spare, _NPCPV)); } if (!TAILQ_EMPTY(&new_tail)) { mtx_lock(&pv_chunks_mutex); TAILQ_CONCAT(&pv_chunks, &new_tail, pc_lru); mtx_unlock(&pv_chunks_mutex); } } /* * First find and then remove the pv entry for the specified pmap and virtual * address from the specified pv list. Returns the pv entry if found and NULL * otherwise. This operation can be performed on pv lists for either 4KB or * 2MB page mappings. */ static __inline pv_entry_t pmap_pvh_remove(struct md_page *pvh, pmap_t pmap, vm_offset_t va) { pv_entry_t pv; TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) { if (pmap == PV_PMAP(pv) && va == pv->pv_va) { TAILQ_REMOVE(&pvh->pv_list, pv, pv_next); pvh->pv_gen++; break; } } return (pv); } /* * After demotion from a 2MB page mapping to 512 4KB page mappings, * destroy the pv entry for the 2MB page mapping and reinstantiate the pv * entries for each of the 4KB page mappings. */ static void pmap_pv_demote_l2(pmap_t pmap, vm_offset_t va, vm_paddr_t pa, struct rwlock **lockp) { struct md_page *pvh; struct pv_chunk *pc; pv_entry_t pv; vm_offset_t va_last; vm_page_t m; int bit, field; PMAP_LOCK_ASSERT(pmap, MA_OWNED); KASSERT((pa & L2_OFFSET) == 0, ("pmap_pv_demote_l2: pa is not 2mpage aligned")); CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, pa); /* * Transfer the 2mpage's pv entry for this mapping to the first * page's pv list. Once this transfer begins, the pv list lock * must not be released until the last pv entry is reinstantiated. */ pvh = pa_to_pvh(pa); va = va & ~L2_OFFSET; pv = pmap_pvh_remove(pvh, pmap, va); KASSERT(pv != NULL, ("pmap_pv_demote_l2: pv not found")); m = PHYS_TO_VM_PAGE(pa); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; /* Instantiate the remaining Ln_ENTRIES - 1 pv entries. */ PV_STAT(atomic_add_long(&pv_entry_allocs, Ln_ENTRIES - 1)); va_last = va + L2_SIZE - PAGE_SIZE; for (;;) { pc = TAILQ_FIRST(&pmap->pm_pvchunk); KASSERT(pc->pc_map[0] != 0 || pc->pc_map[1] != 0 || pc->pc_map[2] != 0, ("pmap_pv_demote_l2: missing spare")); for (field = 0; field < _NPCM; field++) { while (pc->pc_map[field]) { bit = ffsl(pc->pc_map[field]) - 1; pc->pc_map[field] &= ~(1ul << bit); pv = &pc->pc_pventry[field * 64 + bit]; va += PAGE_SIZE; pv->pv_va = va; m++; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_pv_demote_l2: page %p is not managed", m)); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; if (va == va_last) goto out; } } TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&pmap->pm_pvchunk, pc, pc_list); } out: if (pc->pc_map[0] == 0 && pc->pc_map[1] == 0 && pc->pc_map[2] == 0) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); TAILQ_INSERT_TAIL(&pmap->pm_pvchunk, pc, pc_list); } PV_STAT(atomic_add_long(&pv_entry_count, Ln_ENTRIES - 1)); PV_STAT(atomic_subtract_int(&pv_entry_spare, Ln_ENTRIES - 1)); } /* * First find and then destroy the pv entry for the specified pmap and virtual * address. This operation can be performed on pv lists for either 4KB or 2MB * page mappings. */ static void pmap_pvh_free(struct md_page *pvh, pmap_t pmap, vm_offset_t va) { pv_entry_t pv; pv = pmap_pvh_remove(pvh, pmap, va); KASSERT(pv != NULL, ("pmap_pvh_free: pv not found")); free_pv_entry(pmap, pv); } /* * Conditionally create the PV entry for a 4KB page mapping if the required * memory can be allocated without resorting to reclamation. */ static boolean_t pmap_try_insert_pv_entry(pmap_t pmap, vm_offset_t va, vm_page_t m, struct rwlock **lockp) { pv_entry_t pv; PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* Pass NULL instead of the lock pointer to disable reclamation. */ if ((pv = get_pv_entry(pmap, NULL)) != NULL) { pv->pv_va = va; CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; return (TRUE); } else return (FALSE); } /* * pmap_remove_l3: do the things to unmap a page in a process */ static int pmap_remove_l3(pmap_t pmap, pt_entry_t *l3, vm_offset_t va, pd_entry_t l2e, struct spglist *free, struct rwlock **lockp) { struct md_page *pvh; pt_entry_t old_l3; vm_page_t m; PMAP_LOCK_ASSERT(pmap, MA_OWNED); if (pmap_is_current(pmap) && pmap_l3_valid_cacheable(pmap_load(l3))) cpu_dcache_wb_range(va, L3_SIZE); old_l3 = pmap_load_clear(l3); PTE_SYNC(l3); pmap_invalidate_page(pmap, va); if (old_l3 & ATTR_SW_WIRED) pmap->pm_stats.wired_count -= 1; pmap_resident_count_dec(pmap, 1); if (old_l3 & ATTR_SW_MANAGED) { m = PHYS_TO_VM_PAGE(old_l3 & ~ATTR_MASK); if (pmap_page_dirty(old_l3)) vm_page_dirty(m); if (old_l3 & ATTR_AF) vm_page_aflag_set(m, PGA_REFERENCED); CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m); pmap_pvh_free(&m->md, pmap, va); if (TAILQ_EMPTY(&m->md.pv_list) && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); if (TAILQ_EMPTY(&pvh->pv_list)) vm_page_aflag_clear(m, PGA_WRITEABLE); } } return (pmap_unuse_l3(pmap, va, l2e, free)); } /* * Remove the given range of addresses from the specified map. * * It is assumed that the start and end are properly * rounded to the page size. */ void pmap_remove(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { struct rwlock *lock; vm_offset_t va, va_next; pd_entry_t *l0, *l1, *l2; pt_entry_t l3_paddr, *l3; struct spglist free; int anyvalid; /* * Perform an unsynchronized read. This is, however, safe. */ if (pmap->pm_stats.resident_count == 0) return; anyvalid = 0; SLIST_INIT(&free); PMAP_LOCK(pmap); lock = NULL; for (; sva < eva; sva = va_next) { if (pmap->pm_stats.resident_count == 0) break; l0 = pmap_l0(pmap, sva); if (pmap_load(l0) == 0) { va_next = (sva + L0_SIZE) & ~L0_OFFSET; if (va_next < sva) va_next = eva; continue; } l1 = pmap_l0_to_l1(l0, sva); if (pmap_load(l1) == 0) { va_next = (sva + L1_SIZE) & ~L1_OFFSET; if (va_next < sva) va_next = eva; continue; } /* * Calculate index for next page table. */ va_next = (sva + L2_SIZE) & ~L2_OFFSET; if (va_next < sva) va_next = eva; l2 = pmap_l1_to_l2(l1, sva); if (l2 == NULL) continue; l3_paddr = pmap_load(l2); if ((l3_paddr & ATTR_DESCR_MASK) == L2_BLOCK) { /* TODO: Add pmap_remove_l2 */ if (pmap_demote_l2_locked(pmap, l2, sva & ~L2_OFFSET, &lock) == NULL) continue; l3_paddr = pmap_load(l2); } /* * Weed out invalid mappings. */ if ((l3_paddr & ATTR_DESCR_MASK) != L2_TABLE) continue; /* * Limit our scan to either the end of the va represented * by the current page table page, or to the end of the * range being removed. */ if (va_next > eva) va_next = eva; va = va_next; for (l3 = pmap_l2_to_l3(l2, sva); sva != va_next; l3++, sva += L3_SIZE) { if (l3 == NULL) panic("l3 == NULL"); if (pmap_load(l3) == 0) { if (va != va_next) { pmap_invalidate_range(pmap, va, sva); va = va_next; } continue; } if (va == va_next) va = sva; if (pmap_remove_l3(pmap, l3, sva, l3_paddr, &free, &lock)) { sva += L3_SIZE; break; } } if (va != va_next) pmap_invalidate_range(pmap, va, sva); } if (lock != NULL) rw_wunlock(lock); if (anyvalid) pmap_invalidate_all(pmap); PMAP_UNLOCK(pmap); pmap_free_zero_pages(&free); } /* * Routine: pmap_remove_all * Function: * Removes this physical page from * all physical maps in which it resides. * Reflects back modify bits to the pager. * * Notes: * Original versions of this routine were very * inefficient because they iteratively called * pmap_remove (slow...) */ void pmap_remove_all(vm_page_t m) { struct md_page *pvh; pv_entry_t pv; pmap_t pmap; struct rwlock *lock; pd_entry_t *pde, tpde; pt_entry_t *pte, tpte; vm_offset_t va; struct spglist free; int lvl, pvh_gen, md_gen; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_remove_all: page %p is not managed", m)); SLIST_INIT(&free); lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); retry: rw_wlock(lock); while ((pv = TAILQ_FIRST(&pvh->pv_list)) != NULL) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { rw_wunlock(lock); PMAP_UNLOCK(pmap); goto retry; } } va = pv->pv_va; pte = pmap_pte(pmap, va, &lvl); KASSERT(pte != NULL, ("pmap_remove_all: no page table entry found")); KASSERT(lvl == 2, ("pmap_remove_all: invalid pte level %d", lvl)); pmap_demote_l2_locked(pmap, pte, va, &lock); PMAP_UNLOCK(pmap); } while ((pv = TAILQ_FIRST(&m->md.pv_list)) != NULL) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; md_gen = m->md.pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { rw_wunlock(lock); PMAP_UNLOCK(pmap); goto retry; } } pmap_resident_count_dec(pmap, 1); pde = pmap_pde(pmap, pv->pv_va, &lvl); KASSERT(pde != NULL, ("pmap_remove_all: no page directory entry found")); KASSERT(lvl == 2, ("pmap_remove_all: invalid pde level %d", lvl)); tpde = pmap_load(pde); pte = pmap_l2_to_l3(pde, pv->pv_va); tpte = pmap_load(pte); if (pmap_is_current(pmap) && pmap_l3_valid_cacheable(tpte)) cpu_dcache_wb_range(pv->pv_va, L3_SIZE); pmap_load_clear(pte); PTE_SYNC(pte); pmap_invalidate_page(pmap, pv->pv_va); if (tpte & ATTR_SW_WIRED) pmap->pm_stats.wired_count--; if ((tpte & ATTR_AF) != 0) vm_page_aflag_set(m, PGA_REFERENCED); /* * Update the vm_page_t clean and reference bits. */ if (pmap_page_dirty(tpte)) vm_page_dirty(m); pmap_unuse_l3(pmap, pv->pv_va, tpde, &free); TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; free_pv_entry(pmap, pv); PMAP_UNLOCK(pmap); } vm_page_aflag_clear(m, PGA_WRITEABLE); rw_wunlock(lock); pmap_free_zero_pages(&free); } /* * Set the physical protection on the * specified range of this map as requested. */ void pmap_protect(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, vm_prot_t prot) { vm_offset_t va, va_next; pd_entry_t *l0, *l1, *l2; pt_entry_t *l3p, l3; if ((prot & VM_PROT_READ) == VM_PROT_NONE) { pmap_remove(pmap, sva, eva); return; } if ((prot & VM_PROT_WRITE) == VM_PROT_WRITE) return; PMAP_LOCK(pmap); for (; sva < eva; sva = va_next) { l0 = pmap_l0(pmap, sva); if (pmap_load(l0) == 0) { va_next = (sva + L0_SIZE) & ~L0_OFFSET; if (va_next < sva) va_next = eva; continue; } l1 = pmap_l0_to_l1(l0, sva); if (pmap_load(l1) == 0) { va_next = (sva + L1_SIZE) & ~L1_OFFSET; if (va_next < sva) va_next = eva; continue; } va_next = (sva + L2_SIZE) & ~L2_OFFSET; if (va_next < sva) va_next = eva; l2 = pmap_l1_to_l2(l1, sva); if (pmap_load(l2) == 0) continue; if ((pmap_load(l2) & ATTR_DESCR_MASK) == L2_BLOCK) { l3p = pmap_demote_l2(pmap, l2, sva); if (l3p == NULL) continue; } KASSERT((pmap_load(l2) & ATTR_DESCR_MASK) == L2_TABLE, ("pmap_protect: Invalid L2 entry after demotion")); if (va_next > eva) va_next = eva; va = va_next; for (l3p = pmap_l2_to_l3(l2, sva); sva != va_next; l3p++, sva += L3_SIZE) { l3 = pmap_load(l3p); if (pmap_l3_valid(l3)) { pmap_set(l3p, ATTR_AP(ATTR_AP_RO)); PTE_SYNC(l3p); /* XXX: Use pmap_invalidate_range */ pmap_invalidate_page(pmap, va); } } } PMAP_UNLOCK(pmap); /* TODO: Only invalidate entries we are touching */ pmap_invalidate_all(pmap); } /* * Inserts the specified page table page into the specified pmap's collection * of idle page table pages. Each of a pmap's page table pages is responsible * for mapping a distinct range of virtual addresses. The pmap's collection is * ordered by this virtual address range. */ static __inline int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); return (vm_radix_insert(&pmap->pm_root, mpte)); } /* * Looks for a page table page mapping the specified virtual address in the * specified pmap's collection of idle page table pages. Returns NULL if there * is no page table page corresponding to the specified virtual address. */ static __inline vm_page_t pmap_lookup_pt_page(pmap_t pmap, vm_offset_t va) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); return (vm_radix_lookup(&pmap->pm_root, pmap_l2_pindex(va))); } /* * Removes the specified page table page from the specified pmap's collection * of idle page table pages. The specified page table page must be a member of * the pmap's collection. */ static __inline void pmap_remove_pt_page(pmap_t pmap, vm_page_t mpte) { PMAP_LOCK_ASSERT(pmap, MA_OWNED); vm_radix_remove(&pmap->pm_root, mpte->pindex); } /* * Performs a break-before-make update of a pmap entry. This is needed when * either promoting or demoting pages to ensure the TLB doesn't get into an * inconsistent state. */ static void pmap_update_entry(pmap_t pmap, pd_entry_t *pte, pd_entry_t newpte, vm_offset_t va, vm_size_t size) { register_t intr; PMAP_LOCK_ASSERT(pmap, MA_OWNED); /* * Ensure we don't get switched out with the page table in an * inconsistent state. We also need to ensure no interrupts fire * as they may make use of an address we are about to invalidate. */ intr = intr_disable(); critical_enter(); /* Clear the old mapping */ pmap_load_clear(pte); PTE_SYNC(pte); pmap_invalidate_range(pmap, va, va + size); /* Create the new mapping */ pmap_load_store(pte, newpte); PTE_SYNC(pte); critical_exit(); intr_restore(intr); } /* * After promotion from 512 4KB page mappings to a single 2MB page mapping, * replace the many pv entries for the 4KB page mappings by a single pv entry * for the 2MB page mapping. */ static void pmap_pv_promote_l2(pmap_t pmap, vm_offset_t va, vm_paddr_t pa, struct rwlock **lockp) { struct md_page *pvh; pv_entry_t pv; vm_offset_t va_last; vm_page_t m; KASSERT((pa & L2_OFFSET) == 0, ("pmap_pv_promote_l2: pa is not 2mpage aligned")); CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, pa); /* * Transfer the first page's pv entry for this mapping to the 2mpage's * pv list. Aside from avoiding the cost of a call to get_pv_entry(), * a transfer avoids the possibility that get_pv_entry() calls * reclaim_pv_chunk() and that reclaim_pv_chunk() removes one of the * mappings that is being promoted. */ m = PHYS_TO_VM_PAGE(pa); va = va & ~L2_OFFSET; pv = pmap_pvh_remove(&m->md, pmap, va); KASSERT(pv != NULL, ("pmap_pv_promote_l2: pv not found")); pvh = pa_to_pvh(pa); TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_next); pvh->pv_gen++; /* Free the remaining NPTEPG - 1 pv entries. */ va_last = va + L2_SIZE - PAGE_SIZE; do { m++; va += PAGE_SIZE; pmap_pvh_free(&m->md, pmap, va); } while (va < va_last); } /* * Tries to promote the 512, contiguous 4KB page mappings that are within a * single level 2 table entry to a single 2MB page mapping. For promotion * to occur, two conditions must be met: (1) the 4KB page mappings must map * aligned, contiguous physical memory and (2) the 4KB page mappings must have * identical characteristics. */ static void pmap_promote_l2(pmap_t pmap, pd_entry_t *l2, vm_offset_t va, struct rwlock **lockp) { pt_entry_t *firstl3, *l3, newl2, oldl3, pa; vm_page_t mpte; vm_offset_t sva; PMAP_LOCK_ASSERT(pmap, MA_OWNED); sva = va & ~L2_OFFSET; firstl3 = pmap_l2_to_l3(l2, sva); newl2 = pmap_load(firstl3); /* Check the alingment is valid */ if (((newl2 & ~ATTR_MASK) & L2_OFFSET) != 0) { atomic_add_long(&pmap_l2_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_l2: failure for va %#lx" " in pmap %p", va, pmap); return; } pa = newl2 + L2_SIZE - PAGE_SIZE; for (l3 = firstl3 + NL3PG - 1; l3 > firstl3; l3--) { oldl3 = pmap_load(l3); if (oldl3 != pa) { atomic_add_long(&pmap_l2_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_l2: failure for va %#lx" " in pmap %p", va, pmap); return; } pa -= PAGE_SIZE; } /* * Save the page table page in its current state until the L2 * mapping the superpage is demoted by pmap_demote_l2() or * destroyed by pmap_remove_l3(). */ mpte = PHYS_TO_VM_PAGE(pmap_load(l2) & ~ATTR_MASK); KASSERT(mpte >= vm_page_array && mpte < &vm_page_array[vm_page_array_size], ("pmap_promote_l2: page table page is out of range")); KASSERT(mpte->pindex == pmap_l2_pindex(va), ("pmap_promote_l2: page table page's pindex is wrong")); if (pmap_insert_pt_page(pmap, mpte)) { atomic_add_long(&pmap_l2_p_failures, 1); CTR2(KTR_PMAP, "pmap_promote_l2: failure for va %#lx in pmap %p", va, pmap); return; } if ((newl2 & ATTR_SW_MANAGED) != 0) pmap_pv_promote_l2(pmap, va, newl2 & ~ATTR_MASK, lockp); newl2 &= ~ATTR_DESCR_MASK; newl2 |= L2_BLOCK; pmap_update_entry(pmap, l2, newl2, sva, L2_SIZE); atomic_add_long(&pmap_l2_promotions, 1); CTR2(KTR_PMAP, "pmap_promote_l2: success for va %#lx in pmap %p", va, pmap); } /* * Insert the given physical page (p) at * the specified virtual address (v) in the * target physical map with the protection requested. * * If specified, the page will be wired down, meaning * that the related pte can not be reclaimed. * * NB: This is the only routine which MAY NOT lazy-evaluate * or lose information. That is, this routine must actually * insert this page into the given map NOW. */ int pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, u_int flags, int8_t psind __unused) { struct rwlock *lock; pd_entry_t *pde; pt_entry_t new_l3, orig_l3; pt_entry_t *l2, *l3; pv_entry_t pv; vm_paddr_t opa, pa, l1_pa, l2_pa, l3_pa; vm_page_t mpte, om, l1_m, l2_m, l3_m; boolean_t nosleep; int lvl; va = trunc_page(va); if ((m->oflags & VPO_UNMANAGED) == 0 && !vm_page_xbusied(m)) VM_OBJECT_ASSERT_LOCKED(m->object); pa = VM_PAGE_TO_PHYS(m); new_l3 = (pt_entry_t)(pa | ATTR_DEFAULT | ATTR_IDX(m->md.pv_memattr) | L3_PAGE); if ((prot & VM_PROT_WRITE) == 0) new_l3 |= ATTR_AP(ATTR_AP_RO); if ((flags & PMAP_ENTER_WIRED) != 0) new_l3 |= ATTR_SW_WIRED; if ((va >> 63) == 0) new_l3 |= ATTR_AP(ATTR_AP_USER); CTR2(KTR_PMAP, "pmap_enter: %.16lx -> %.16lx", va, pa); mpte = NULL; lock = NULL; PMAP_LOCK(pmap); pde = pmap_pde(pmap, va, &lvl); if (pde != NULL && lvl == 1) { l2 = pmap_l1_to_l2(pde, va); if ((pmap_load(l2) & ATTR_DESCR_MASK) == L2_BLOCK && (l3 = pmap_demote_l2_locked(pmap, l2, va, &lock)) != NULL) { if (va < VM_MAXUSER_ADDRESS) { mpte = PHYS_TO_VM_PAGE( pmap_load(l2) & ~ATTR_MASK); mpte->wire_count++; } goto havel3; } } if (va < VM_MAXUSER_ADDRESS) { nosleep = (flags & PMAP_ENTER_NOSLEEP) != 0; mpte = pmap_alloc_l3(pmap, va, nosleep ? NULL : &lock); if (mpte == NULL && nosleep) { CTR0(KTR_PMAP, "pmap_enter: mpte == NULL"); if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); return (KERN_RESOURCE_SHORTAGE); } pde = pmap_pde(pmap, va, &lvl); KASSERT(pde != NULL, ("pmap_enter: Invalid page entry, va: 0x%lx", va)); KASSERT(lvl == 2, ("pmap_enter: Invalid level %d", lvl)); l3 = pmap_l2_to_l3(pde, va); } else { /* * If we get a level 2 pde it must point to a level 3 entry * otherwise we will need to create the intermediate tables */ if (lvl < 2) { switch(lvl) { default: case -1: /* Get the l0 pde to update */ pde = pmap_l0(pmap, va); KASSERT(pde != NULL, ("...")); l1_m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO); if (l1_m == NULL) panic("pmap_enter: l1 pte_m == NULL"); if ((l1_m->flags & PG_ZERO) == 0) pmap_zero_page(l1_m); l1_pa = VM_PAGE_TO_PHYS(l1_m); pmap_load_store(pde, l1_pa | L0_TABLE); PTE_SYNC(pde); /* FALLTHROUGH */ case 0: /* Get the l1 pde to update */ pde = pmap_l1_to_l2(pde, va); KASSERT(pde != NULL, ("...")); l2_m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO); if (l2_m == NULL) panic("pmap_enter: l2 pte_m == NULL"); if ((l2_m->flags & PG_ZERO) == 0) pmap_zero_page(l2_m); l2_pa = VM_PAGE_TO_PHYS(l2_m); pmap_load_store(pde, l2_pa | L1_TABLE); PTE_SYNC(pde); /* FALLTHROUGH */ case 1: /* Get the l2 pde to update */ pde = pmap_l1_to_l2(pde, va); l3_m = vm_page_alloc(NULL, 0, VM_ALLOC_NORMAL | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED | VM_ALLOC_ZERO); if (l3_m == NULL) panic("pmap_enter: l3 pte_m == NULL"); if ((l3_m->flags & PG_ZERO) == 0) pmap_zero_page(l3_m); l3_pa = VM_PAGE_TO_PHYS(l3_m); pmap_load_store(pde, l3_pa | L2_TABLE); PTE_SYNC(pde); break; } } l3 = pmap_l2_to_l3(pde, va); pmap_invalidate_page(pmap, va); } havel3: om = NULL; orig_l3 = pmap_load(l3); opa = orig_l3 & ~ATTR_MASK; /* * Is the specified virtual address already mapped? */ if (pmap_l3_valid(orig_l3)) { /* * Wiring change, just update stats. We don't worry about * wiring PT pages as they remain resident as long as there * are valid mappings in them. Hence, if a user page is wired, * the PT page will be also. */ if ((flags & PMAP_ENTER_WIRED) != 0 && (orig_l3 & ATTR_SW_WIRED) == 0) pmap->pm_stats.wired_count++; else if ((flags & PMAP_ENTER_WIRED) == 0 && (orig_l3 & ATTR_SW_WIRED) != 0) pmap->pm_stats.wired_count--; /* * Remove the extra PT page reference. */ if (mpte != NULL) { mpte->wire_count--; KASSERT(mpte->wire_count > 0, ("pmap_enter: missing reference to page table page," " va: 0x%lx", va)); } /* * Has the physical page changed? */ if (opa == pa) { /* * No, might be a protection or wiring change. */ if ((orig_l3 & ATTR_SW_MANAGED) != 0) { new_l3 |= ATTR_SW_MANAGED; if ((new_l3 & ATTR_AP(ATTR_AP_RW)) == ATTR_AP(ATTR_AP_RW)) { vm_page_aflag_set(m, PGA_WRITEABLE); } } goto validate; } /* Flush the cache, there might be uncommitted data in it */ if (pmap_is_current(pmap) && pmap_l3_valid_cacheable(orig_l3)) cpu_dcache_wb_range(va, L3_SIZE); } else { /* * Increment the counters. */ if ((new_l3 & ATTR_SW_WIRED) != 0) pmap->pm_stats.wired_count++; pmap_resident_count_inc(pmap, 1); } /* * Enter on the PV list if part of our managed memory. */ if ((m->oflags & VPO_UNMANAGED) == 0) { new_l3 |= ATTR_SW_MANAGED; pv = get_pv_entry(pmap, &lock); pv->pv_va = va; CHANGE_PV_LIST_LOCK_TO_PHYS(&lock, pa); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; if ((new_l3 & ATTR_AP_RW_BIT) == ATTR_AP(ATTR_AP_RW)) vm_page_aflag_set(m, PGA_WRITEABLE); } /* * Update the L3 entry. */ if (orig_l3 != 0) { validate: orig_l3 = pmap_load(l3); opa = orig_l3 & ~ATTR_MASK; if (opa != pa) { pmap_update_entry(pmap, l3, new_l3, va, PAGE_SIZE); if ((orig_l3 & ATTR_SW_MANAGED) != 0) { om = PHYS_TO_VM_PAGE(opa); if (pmap_page_dirty(orig_l3)) vm_page_dirty(om); if ((orig_l3 & ATTR_AF) != 0) vm_page_aflag_set(om, PGA_REFERENCED); CHANGE_PV_LIST_LOCK_TO_PHYS(&lock, opa); pmap_pvh_free(&om->md, pmap, va); if ((om->aflags & PGA_WRITEABLE) != 0 && TAILQ_EMPTY(&om->md.pv_list) && ((om->flags & PG_FICTITIOUS) != 0 || TAILQ_EMPTY(&pa_to_pvh(opa)->pv_list))) vm_page_aflag_clear(om, PGA_WRITEABLE); } } else { pmap_load_store(l3, new_l3); PTE_SYNC(l3); pmap_invalidate_page(pmap, va); if (pmap_page_dirty(orig_l3) && (orig_l3 & ATTR_SW_MANAGED) != 0) vm_page_dirty(m); } } else { pmap_load_store(l3, new_l3); } PTE_SYNC(l3); pmap_invalidate_page(pmap, va); - if ((pmap != pmap_kernel()) && (pmap == &curproc->p_vmspace->vm_pmap)) - cpu_icache_sync_range(va, PAGE_SIZE); + if (pmap != pmap_kernel()) { + if (pmap == &curproc->p_vmspace->vm_pmap) + cpu_icache_sync_range(va, PAGE_SIZE); - if ((mpte == NULL || mpte->wire_count == NL3PG) && - pmap_superpages_enabled() && (m->flags & PG_FICTITIOUS) == 0 && - vm_reserv_level_iffullpop(m) == 0) { - KASSERT(lvl == 2, ("Invalid pde level %d", lvl)); - pmap_promote_l2(pmap, pde, va, &lock); + if ((mpte == NULL || mpte->wire_count == NL3PG) && + pmap_superpages_enabled() && + (m->flags & PG_FICTITIOUS) == 0 && + vm_reserv_level_iffullpop(m) == 0) { + KASSERT(lvl == 2, ("Invalid pde level %d", lvl)); + pmap_promote_l2(pmap, pde, va, &lock); + } } if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); return (KERN_SUCCESS); } /* * Maps a sequence of resident pages belonging to the same object. * The sequence begins with the given page m_start. This page is * mapped at the given virtual address start. Each subsequent page is * mapped at a virtual address that is offset from start by the same * amount as the page is offset from m_start within the object. The * last page in the sequence is the page with the largest offset from * m_start that can be mapped at a virtual address less than the given * virtual address end. Not every virtual page between start and end * is mapped; only those for which a resident page exists with the * corresponding offset from m_start are mapped. */ void pmap_enter_object(pmap_t pmap, vm_offset_t start, vm_offset_t end, vm_page_t m_start, vm_prot_t prot) { struct rwlock *lock; vm_offset_t va; vm_page_t m, mpte; vm_pindex_t diff, psize; VM_OBJECT_ASSERT_LOCKED(m_start->object); psize = atop(end - start); mpte = NULL; m = m_start; lock = NULL; PMAP_LOCK(pmap); while (m != NULL && (diff = m->pindex - m_start->pindex) < psize) { va = start + ptoa(diff); mpte = pmap_enter_quick_locked(pmap, va, m, prot, mpte, &lock); m = TAILQ_NEXT(m, listq); } if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); } /* * this code makes some *MAJOR* assumptions: * 1. Current pmap & pmap exists. * 2. Not wired. * 3. Read access. * 4. No page table pages. * but is *MUCH* faster than pmap_enter... */ void pmap_enter_quick(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot) { struct rwlock *lock; lock = NULL; PMAP_LOCK(pmap); (void)pmap_enter_quick_locked(pmap, va, m, prot, NULL, &lock); if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); } static vm_page_t pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_prot_t prot, vm_page_t mpte, struct rwlock **lockp) { struct spglist free; pd_entry_t *pde; pt_entry_t *l2, *l3; vm_paddr_t pa; int lvl; KASSERT(va < kmi.clean_sva || va >= kmi.clean_eva || (m->oflags & VPO_UNMANAGED) != 0, ("pmap_enter_quick_locked: managed mapping within the clean submap")); PMAP_LOCK_ASSERT(pmap, MA_OWNED); CTR2(KTR_PMAP, "pmap_enter_quick_locked: %p %lx", pmap, va); /* * In the case that a page table page is not * resident, we are creating it here. */ if (va < VM_MAXUSER_ADDRESS) { vm_pindex_t l2pindex; /* * Calculate pagetable page index */ l2pindex = pmap_l2_pindex(va); if (mpte && (mpte->pindex == l2pindex)) { mpte->wire_count++; } else { /* * Get the l2 entry */ pde = pmap_pde(pmap, va, &lvl); /* * If the page table page is mapped, we just increment * the hold count, and activate it. Otherwise, we * attempt to allocate a page table page. If this * attempt fails, we don't retry. Instead, we give up. */ if (lvl == 1) { l2 = pmap_l1_to_l2(pde, va); if ((pmap_load(l2) & ATTR_DESCR_MASK) == L2_BLOCK) return (NULL); } if (lvl == 2 && pmap_load(pde) != 0) { mpte = PHYS_TO_VM_PAGE(pmap_load(pde) & ~ATTR_MASK); mpte->wire_count++; } else { /* * Pass NULL instead of the PV list lock * pointer, because we don't intend to sleep. */ mpte = _pmap_alloc_l3(pmap, l2pindex, NULL); if (mpte == NULL) return (mpte); } } l3 = (pt_entry_t *)PHYS_TO_DMAP(VM_PAGE_TO_PHYS(mpte)); l3 = &l3[pmap_l3_index(va)]; } else { mpte = NULL; pde = pmap_pde(kernel_pmap, va, &lvl); KASSERT(pde != NULL, ("pmap_enter_quick_locked: Invalid page entry, va: 0x%lx", va)); KASSERT(lvl == 2, ("pmap_enter_quick_locked: Invalid level %d", lvl)); l3 = pmap_l2_to_l3(pde, va); } if (pmap_load(l3) != 0) { if (mpte != NULL) { mpte->wire_count--; mpte = NULL; } return (mpte); } /* * Enter on the PV list if part of our managed memory. */ if ((m->oflags & VPO_UNMANAGED) == 0 && !pmap_try_insert_pv_entry(pmap, va, m, lockp)) { if (mpte != NULL) { SLIST_INIT(&free); if (pmap_unwire_l3(pmap, va, mpte, &free)) { pmap_invalidate_page(pmap, va); pmap_free_zero_pages(&free); } mpte = NULL; } return (mpte); } /* * Increment counters */ pmap_resident_count_inc(pmap, 1); pa = VM_PAGE_TO_PHYS(m) | ATTR_DEFAULT | ATTR_IDX(m->md.pv_memattr) | ATTR_AP(ATTR_AP_RO) | L3_PAGE; /* * Now validate mapping with RO protection */ if ((m->oflags & VPO_UNMANAGED) == 0) pa |= ATTR_SW_MANAGED; pmap_load_store(l3, pa); PTE_SYNC(l3); pmap_invalidate_page(pmap, va); return (mpte); } /* * This code maps large physical mmap regions into the * processor address space. Note that some shortcuts * are taken, but the code works. */ void pmap_object_init_pt(pmap_t pmap, vm_offset_t addr, vm_object_t object, vm_pindex_t pindex, vm_size_t size) { VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(object->type == OBJT_DEVICE || object->type == OBJT_SG, ("pmap_object_init_pt: non-device object")); } /* * Clear the wired attribute from the mappings for the specified range of * addresses in the given pmap. Every valid mapping within that range * must have the wired attribute set. In contrast, invalid mappings * cannot have the wired attribute set, so they are ignored. * * The wired attribute of the page table entry is not a hardware feature, * so there is no need to invalidate any TLB entries. */ void pmap_unwire(pmap_t pmap, vm_offset_t sva, vm_offset_t eva) { vm_offset_t va_next; pd_entry_t *l0, *l1, *l2; pt_entry_t *l3; PMAP_LOCK(pmap); for (; sva < eva; sva = va_next) { l0 = pmap_l0(pmap, sva); if (pmap_load(l0) == 0) { va_next = (sva + L0_SIZE) & ~L0_OFFSET; if (va_next < sva) va_next = eva; continue; } l1 = pmap_l0_to_l1(l0, sva); if (pmap_load(l1) == 0) { va_next = (sva + L1_SIZE) & ~L1_OFFSET; if (va_next < sva) va_next = eva; continue; } va_next = (sva + L2_SIZE) & ~L2_OFFSET; if (va_next < sva) va_next = eva; l2 = pmap_l1_to_l2(l1, sva); if (pmap_load(l2) == 0) continue; if ((pmap_load(l2) & ATTR_DESCR_MASK) == L2_BLOCK) { l3 = pmap_demote_l2(pmap, l2, sva); if (l3 == NULL) continue; } KASSERT((pmap_load(l2) & ATTR_DESCR_MASK) == L2_TABLE, ("pmap_unwire: Invalid l2 entry after demotion")); if (va_next > eva) va_next = eva; for (l3 = pmap_l2_to_l3(l2, sva); sva != va_next; l3++, sva += L3_SIZE) { if (pmap_load(l3) == 0) continue; if ((pmap_load(l3) & ATTR_SW_WIRED) == 0) panic("pmap_unwire: l3 %#jx is missing " "ATTR_SW_WIRED", (uintmax_t)pmap_load(l3)); /* * PG_W must be cleared atomically. Although the pmap * lock synchronizes access to PG_W, another processor * could be setting PG_M and/or PG_A concurrently. */ atomic_clear_long(l3, ATTR_SW_WIRED); pmap->pm_stats.wired_count--; } } PMAP_UNLOCK(pmap); } /* * Copy the range specified by src_addr/len * from the source map to the range dst_addr/len * in the destination map. * * This routine is only advisory and need not do anything. */ void pmap_copy(pmap_t dst_pmap, pmap_t src_pmap, vm_offset_t dst_addr, vm_size_t len, vm_offset_t src_addr) { } /* * pmap_zero_page zeros the specified hardware page by mapping * the page into KVM and using bzero to clear its contents. */ void pmap_zero_page(vm_page_t m) { vm_offset_t va = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)); pagezero((void *)va); } /* * pmap_zero_page_area zeros the specified hardware page by mapping * the page into KVM and using bzero to clear its contents. * * off and size may not cover an area beyond a single hardware page. */ void pmap_zero_page_area(vm_page_t m, int off, int size) { vm_offset_t va = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)); if (off == 0 && size == PAGE_SIZE) pagezero((void *)va); else bzero((char *)va + off, size); } /* * pmap_zero_page_idle zeros the specified hardware page by mapping * the page into KVM and using bzero to clear its contents. This * is intended to be called from the vm_pagezero process only and * outside of Giant. */ void pmap_zero_page_idle(vm_page_t m) { vm_offset_t va = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)); pagezero((void *)va); } /* * pmap_copy_page copies the specified (machine independent) * page by mapping the page into virtual memory and using * bcopy to copy the page, one machine dependent page at a * time. */ void pmap_copy_page(vm_page_t msrc, vm_page_t mdst) { vm_offset_t src = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(msrc)); vm_offset_t dst = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(mdst)); pagecopy((void *)src, (void *)dst); } int unmapped_buf_allowed = 1; void pmap_copy_pages(vm_page_t ma[], vm_offset_t a_offset, vm_page_t mb[], vm_offset_t b_offset, int xfersize) { void *a_cp, *b_cp; vm_page_t m_a, m_b; vm_paddr_t p_a, p_b; vm_offset_t a_pg_offset, b_pg_offset; int cnt; while (xfersize > 0) { a_pg_offset = a_offset & PAGE_MASK; m_a = ma[a_offset >> PAGE_SHIFT]; p_a = m_a->phys_addr; b_pg_offset = b_offset & PAGE_MASK; m_b = mb[b_offset >> PAGE_SHIFT]; p_b = m_b->phys_addr; cnt = min(xfersize, PAGE_SIZE - a_pg_offset); cnt = min(cnt, PAGE_SIZE - b_pg_offset); if (__predict_false(!PHYS_IN_DMAP(p_a))) { panic("!DMAP a %lx", p_a); } else { a_cp = (char *)PHYS_TO_DMAP(p_a) + a_pg_offset; } if (__predict_false(!PHYS_IN_DMAP(p_b))) { panic("!DMAP b %lx", p_b); } else { b_cp = (char *)PHYS_TO_DMAP(p_b) + b_pg_offset; } bcopy(a_cp, b_cp, cnt); a_offset += cnt; b_offset += cnt; xfersize -= cnt; } } vm_offset_t pmap_quick_enter_page(vm_page_t m) { return (PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m))); } void pmap_quick_remove_page(vm_offset_t addr) { } /* * Returns true if the pmap's pv is one of the first * 16 pvs linked to from this page. This count may * be changed upwards or downwards in the future; it * is only necessary that true be returned for a small * subset of pmaps for proper page aging. */ boolean_t pmap_page_exists_quick(pmap_t pmap, vm_page_t m) { struct md_page *pvh; struct rwlock *lock; pv_entry_t pv; int loops = 0; boolean_t rv; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_page_exists_quick: page %p is not managed", m)); rv = FALSE; lock = VM_PAGE_TO_PV_LIST_LOCK(m); rw_rlock(lock); TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { if (PV_PMAP(pv) == pmap) { rv = TRUE; break; } loops++; if (loops >= 16) break; } if (!rv && loops < 16 && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) { if (PV_PMAP(pv) == pmap) { rv = TRUE; break; } loops++; if (loops >= 16) break; } } rw_runlock(lock); return (rv); } /* * pmap_page_wired_mappings: * * Return the number of managed mappings to the given physical page * that are wired. */ int pmap_page_wired_mappings(vm_page_t m) { struct rwlock *lock; struct md_page *pvh; pmap_t pmap; pt_entry_t *pte; pv_entry_t pv; int count, lvl, md_gen, pvh_gen; if ((m->oflags & VPO_UNMANAGED) != 0) return (0); lock = VM_PAGE_TO_PV_LIST_LOCK(m); rw_rlock(lock); restart: count = 0; TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; rw_runlock(lock); PMAP_LOCK(pmap); rw_rlock(lock); if (md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } pte = pmap_pte(pmap, pv->pv_va, &lvl); if (pte != NULL && (pmap_load(pte) & ATTR_SW_WIRED) != 0) count++; PMAP_UNLOCK(pmap); } if ((m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; pvh_gen = pvh->pv_gen; rw_runlock(lock); PMAP_LOCK(pmap); rw_rlock(lock); if (md_gen != m->md.pv_gen || pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } pte = pmap_pte(pmap, pv->pv_va, &lvl); if (pte != NULL && (pmap_load(pte) & ATTR_SW_WIRED) != 0) count++; PMAP_UNLOCK(pmap); } } rw_runlock(lock); return (count); } /* * Destroy all managed, non-wired mappings in the given user-space * pmap. This pmap cannot be active on any processor besides the * caller. * * This function cannot be applied to the kernel pmap. Moreover, it * is not intended for general use. It is only to be used during * process termination. Consequently, it can be implemented in ways * that make it faster than pmap_remove(). First, it can more quickly * destroy mappings by iterating over the pmap's collection of PV * entries, rather than searching the page table. Second, it doesn't * have to test and clear the page table entries atomically, because * no processor is currently accessing the user address space. In * particular, a page table entry's dirty bit won't change state once * this function starts. */ void pmap_remove_pages(pmap_t pmap) { pd_entry_t *pde; pt_entry_t *pte, tpte; struct spglist free; vm_page_t m, ml3, mt; pv_entry_t pv; struct md_page *pvh; struct pv_chunk *pc, *npc; struct rwlock *lock; int64_t bit; uint64_t inuse, bitmask; int allfree, field, freed, idx, lvl; vm_paddr_t pa; lock = NULL; SLIST_INIT(&free); PMAP_LOCK(pmap); TAILQ_FOREACH_SAFE(pc, &pmap->pm_pvchunk, pc_list, npc) { allfree = 1; freed = 0; for (field = 0; field < _NPCM; field++) { inuse = ~pc->pc_map[field] & pc_freemask[field]; while (inuse != 0) { bit = ffsl(inuse) - 1; bitmask = 1UL << bit; idx = field * 64 + bit; pv = &pc->pc_pventry[idx]; inuse &= ~bitmask; pde = pmap_pde(pmap, pv->pv_va, &lvl); KASSERT(pde != NULL, ("Attempting to remove an unmapped page")); switch(lvl) { case 1: pte = pmap_l1_to_l2(pde, pv->pv_va); tpte = pmap_load(pte); KASSERT((tpte & ATTR_DESCR_MASK) == L2_BLOCK, ("Attempting to remove an invalid " "block: %lx", tpte)); tpte = pmap_load(pte); break; case 2: pte = pmap_l2_to_l3(pde, pv->pv_va); tpte = pmap_load(pte); KASSERT((tpte & ATTR_DESCR_MASK) == L3_PAGE, ("Attempting to remove an invalid " "page: %lx", tpte)); break; default: panic( "Invalid page directory level: %d", lvl); } /* * We cannot remove wired pages from a process' mapping at this time */ if (tpte & ATTR_SW_WIRED) { allfree = 0; continue; } pa = tpte & ~ATTR_MASK; m = PHYS_TO_VM_PAGE(pa); KASSERT(m->phys_addr == pa, ("vm_page_t %p phys_addr mismatch %016jx %016jx", m, (uintmax_t)m->phys_addr, (uintmax_t)tpte)); KASSERT((m->flags & PG_FICTITIOUS) != 0 || m < &vm_page_array[vm_page_array_size], ("pmap_remove_pages: bad pte %#jx", (uintmax_t)tpte)); if (pmap_is_current(pmap)) { if (lvl == 2 && pmap_l3_valid_cacheable(tpte)) { cpu_dcache_wb_range(pv->pv_va, L3_SIZE); } else if (lvl == 1 && pmap_pte_valid_cacheable(tpte)) { cpu_dcache_wb_range(pv->pv_va, L2_SIZE); } } pmap_load_clear(pte); PTE_SYNC(pte); pmap_invalidate_page(pmap, pv->pv_va); /* * Update the vm_page_t clean/reference bits. */ if ((tpte & ATTR_AP_RW_BIT) == ATTR_AP(ATTR_AP_RW)) { switch (lvl) { case 1: for (mt = m; mt < &m[L2_SIZE / PAGE_SIZE]; mt++) vm_page_dirty(m); break; case 2: vm_page_dirty(m); break; } } CHANGE_PV_LIST_LOCK_TO_VM_PAGE(&lock, m); /* Mark free */ pc->pc_map[field] |= bitmask; switch (lvl) { case 1: pmap_resident_count_dec(pmap, L2_SIZE / PAGE_SIZE); pvh = pa_to_pvh(tpte & ~ATTR_MASK); TAILQ_REMOVE(&pvh->pv_list, pv,pv_next); pvh->pv_gen++; if (TAILQ_EMPTY(&pvh->pv_list)) { for (mt = m; mt < &m[L2_SIZE / PAGE_SIZE]; mt++) if ((mt->aflags & PGA_WRITEABLE) != 0 && TAILQ_EMPTY(&mt->md.pv_list)) vm_page_aflag_clear(mt, PGA_WRITEABLE); } ml3 = pmap_lookup_pt_page(pmap, pv->pv_va); if (ml3 != NULL) { pmap_remove_pt_page(pmap, ml3); pmap_resident_count_dec(pmap,1); KASSERT(ml3->wire_count == NL3PG, ("pmap_remove_pages: l3 page wire count error")); ml3->wire_count = 0; pmap_add_delayed_free_list(ml3, &free, FALSE); atomic_subtract_int( &vm_cnt.v_wire_count, 1); } break; case 2: pmap_resident_count_dec(pmap, 1); TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; if ((m->aflags & PGA_WRITEABLE) != 0 && TAILQ_EMPTY(&m->md.pv_list) && (m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh( VM_PAGE_TO_PHYS(m)); if (TAILQ_EMPTY(&pvh->pv_list)) vm_page_aflag_clear(m, PGA_WRITEABLE); } break; } pmap_unuse_l3(pmap, pv->pv_va, pmap_load(pde), &free); freed++; } } PV_STAT(atomic_add_long(&pv_entry_frees, freed)); PV_STAT(atomic_add_int(&pv_entry_spare, freed)); PV_STAT(atomic_subtract_long(&pv_entry_count, freed)); if (allfree) { TAILQ_REMOVE(&pmap->pm_pvchunk, pc, pc_list); free_pv_chunk(pc); } } pmap_invalidate_all(pmap); if (lock != NULL) rw_wunlock(lock); PMAP_UNLOCK(pmap); pmap_free_zero_pages(&free); } /* * This is used to check if a page has been accessed or modified. As we * don't have a bit to see if it has been modified we have to assume it * has been if the page is read/write. */ static boolean_t pmap_page_test_mappings(vm_page_t m, boolean_t accessed, boolean_t modified) { struct rwlock *lock; pv_entry_t pv; struct md_page *pvh; pt_entry_t *pte, mask, value; pmap_t pmap; int lvl, md_gen, pvh_gen; boolean_t rv; rv = FALSE; lock = VM_PAGE_TO_PV_LIST_LOCK(m); rw_rlock(lock); restart: TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; rw_runlock(lock); PMAP_LOCK(pmap); rw_rlock(lock); if (md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } pte = pmap_pte(pmap, pv->pv_va, &lvl); KASSERT(lvl == 3, ("pmap_page_test_mappings: Invalid level %d", lvl)); mask = 0; value = 0; if (modified) { mask |= ATTR_AP_RW_BIT; value |= ATTR_AP(ATTR_AP_RW); } if (accessed) { mask |= ATTR_AF | ATTR_DESCR_MASK; value |= ATTR_AF | L3_PAGE; } rv = (pmap_load(pte) & mask) == value; PMAP_UNLOCK(pmap); if (rv) goto out; } if ((m->flags & PG_FICTITIOUS) == 0) { pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m)); TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { md_gen = m->md.pv_gen; pvh_gen = pvh->pv_gen; rw_runlock(lock); PMAP_LOCK(pmap); rw_rlock(lock); if (md_gen != m->md.pv_gen || pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); goto restart; } } pte = pmap_pte(pmap, pv->pv_va, &lvl); KASSERT(lvl == 2, ("pmap_page_test_mappings: Invalid level %d", lvl)); mask = 0; value = 0; if (modified) { mask |= ATTR_AP_RW_BIT; value |= ATTR_AP(ATTR_AP_RW); } if (accessed) { mask |= ATTR_AF | ATTR_DESCR_MASK; value |= ATTR_AF | L2_BLOCK; } rv = (pmap_load(pte) & mask) == value; PMAP_UNLOCK(pmap); if (rv) goto out; } } out: rw_runlock(lock); return (rv); } /* * pmap_is_modified: * * Return whether or not the specified physical page was modified * in any physical maps. */ boolean_t pmap_is_modified(vm_page_t m) { KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_is_modified: page %p is not managed", m)); /* * If the page is not exclusive busied, then PGA_WRITEABLE cannot be * concurrently set while the object is locked. Thus, if PGA_WRITEABLE * is clear, no PTEs can have PG_M set. */ VM_OBJECT_ASSERT_WLOCKED(m->object); if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) return (FALSE); return (pmap_page_test_mappings(m, FALSE, TRUE)); } /* * pmap_is_prefaultable: * * Return whether or not the specified virtual address is eligible * for prefault. */ boolean_t pmap_is_prefaultable(pmap_t pmap, vm_offset_t addr) { pt_entry_t *pte; boolean_t rv; int lvl; rv = FALSE; PMAP_LOCK(pmap); pte = pmap_pte(pmap, addr, &lvl); if (pte != NULL && pmap_load(pte) != 0) { rv = TRUE; } PMAP_UNLOCK(pmap); return (rv); } /* * pmap_is_referenced: * * Return whether or not the specified physical page was referenced * in any physical maps. */ boolean_t pmap_is_referenced(vm_page_t m) { KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_is_referenced: page %p is not managed", m)); return (pmap_page_test_mappings(m, TRUE, FALSE)); } /* * Clear the write and modified bits in each of the given page's mappings. */ void pmap_remove_write(vm_page_t m) { struct md_page *pvh; pmap_t pmap; struct rwlock *lock; pv_entry_t next_pv, pv; pt_entry_t oldpte, *pte; vm_offset_t va; int lvl, md_gen, pvh_gen; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_remove_write: page %p is not managed", m)); /* * If the page is not exclusive busied, then PGA_WRITEABLE cannot be * set by another thread while the object is locked. Thus, * if PGA_WRITEABLE is clear, no page table entries need updating. */ VM_OBJECT_ASSERT_WLOCKED(m->object); if (!vm_page_xbusied(m) && (m->aflags & PGA_WRITEABLE) == 0) return; lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); retry_pv_loop: rw_wlock(lock); TAILQ_FOREACH_SAFE(pv, &pvh->pv_list, pv_next, next_pv) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); rw_wunlock(lock); goto retry_pv_loop; } } va = pv->pv_va; pte = pmap_pte(pmap, pv->pv_va, &lvl); if ((pmap_load(pte) & ATTR_AP_RW_BIT) == ATTR_AP(ATTR_AP_RW)) pmap_demote_l2_locked(pmap, pte, va & ~L2_OFFSET, &lock); KASSERT(lock == VM_PAGE_TO_PV_LIST_LOCK(m), ("inconsistent pv lock %p %p for page %p", lock, VM_PAGE_TO_PV_LIST_LOCK(m), m)); PMAP_UNLOCK(pmap); } TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; md_gen = m->md.pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); rw_wunlock(lock); goto retry_pv_loop; } } pte = pmap_pte(pmap, pv->pv_va, &lvl); retry: oldpte = pmap_load(pte); if ((oldpte & ATTR_AP_RW_BIT) == ATTR_AP(ATTR_AP_RW)) { if (!atomic_cmpset_long(pte, oldpte, oldpte | ATTR_AP(ATTR_AP_RO))) goto retry; if ((oldpte & ATTR_AF) != 0) vm_page_dirty(m); pmap_invalidate_page(pmap, pv->pv_va); } PMAP_UNLOCK(pmap); } rw_wunlock(lock); vm_page_aflag_clear(m, PGA_WRITEABLE); } static __inline boolean_t safe_to_clear_referenced(pmap_t pmap, pt_entry_t pte) { return (FALSE); } #define PMAP_TS_REFERENCED_MAX 5 /* * pmap_ts_referenced: * * Return a count of reference bits for a page, clearing those bits. * It is not necessary for every reference bit to be cleared, but it * is necessary that 0 only be returned when there are truly no * reference bits set. * * XXX: The exact number of bits to check and clear is a matter that * should be tested and standardized at some point in the future for * optimal aging of shared pages. */ int pmap_ts_referenced(vm_page_t m) { struct md_page *pvh; pv_entry_t pv, pvf; pmap_t pmap; struct rwlock *lock; pd_entry_t *pde, tpde; pt_entry_t *pte, tpte; pt_entry_t *l3; vm_offset_t va; vm_paddr_t pa; int cleared, md_gen, not_cleared, lvl, pvh_gen; struct spglist free; bool demoted; KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_ts_referenced: page %p is not managed", m)); SLIST_INIT(&free); cleared = 0; pa = VM_PAGE_TO_PHYS(m); lock = PHYS_TO_PV_LIST_LOCK(pa); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(pa); rw_wlock(lock); retry: not_cleared = 0; if ((pvf = TAILQ_FIRST(&pvh->pv_list)) == NULL) goto small_mappings; pv = pvf; do { if (pvf == NULL) pvf = pv; pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); goto retry; } } va = pv->pv_va; pde = pmap_pde(pmap, pv->pv_va, &lvl); KASSERT(pde != NULL, ("pmap_ts_referenced: no l1 table found")); KASSERT(lvl == 1, ("pmap_ts_referenced: invalid pde level %d", lvl)); tpde = pmap_load(pde); KASSERT((tpde & ATTR_DESCR_MASK) == L1_TABLE, ("pmap_ts_referenced: found an invalid l1 table")); pte = pmap_l1_to_l2(pde, pv->pv_va); tpte = pmap_load(pte); if ((tpte & ATTR_AF) != 0) { /* * Since this reference bit is shared by 512 4KB * pages, it should not be cleared every time it is * tested. Apply a simple "hash" function on the * physical page number, the virtual superpage number, * and the pmap address to select one 4KB page out of * the 512 on which testing the reference bit will * result in clearing that reference bit. This * function is designed to avoid the selection of the * same 4KB page for every 2MB page mapping. * * On demotion, a mapping that hasn't been referenced * is simply destroyed. To avoid the possibility of a * subsequent page fault on a demoted wired mapping, * always leave its reference bit set. Moreover, * since the superpage is wired, the current state of * its reference bit won't affect page replacement. */ if ((((pa >> PAGE_SHIFT) ^ (pv->pv_va >> L2_SHIFT) ^ (uintptr_t)pmap) & (Ln_ENTRIES - 1)) == 0 && (tpte & ATTR_SW_WIRED) == 0) { if (safe_to_clear_referenced(pmap, tpte)) { /* * TODO: We don't handle the access * flag at all. We need to be able * to set it in the exception handler. */ panic("ARM64TODO: " "safe_to_clear_referenced\n"); } else if (pmap_demote_l2_locked(pmap, pte, pv->pv_va, &lock) != NULL) { demoted = true; va += VM_PAGE_TO_PHYS(m) - (tpte & ~ATTR_MASK); l3 = pmap_l2_to_l3(pte, va); pmap_remove_l3(pmap, l3, va, pmap_load(pte), NULL, &lock); } else demoted = true; if (demoted) { /* * The superpage mapping was removed * entirely and therefore 'pv' is no * longer valid. */ if (pvf == pv) pvf = NULL; pv = NULL; } cleared++; KASSERT(lock == VM_PAGE_TO_PV_LIST_LOCK(m), ("inconsistent pv lock %p %p for page %p", lock, VM_PAGE_TO_PV_LIST_LOCK(m), m)); } else not_cleared++; } PMAP_UNLOCK(pmap); /* Rotate the PV list if it has more than one entry. */ if (pv != NULL && TAILQ_NEXT(pv, pv_next) != NULL) { TAILQ_REMOVE(&pvh->pv_list, pv, pv_next); TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_next); pvh->pv_gen++; } if (cleared + not_cleared >= PMAP_TS_REFERENCED_MAX) goto out; } while ((pv = TAILQ_FIRST(&pvh->pv_list)) != pvf); small_mappings: if ((pvf = TAILQ_FIRST(&m->md.pv_list)) == NULL) goto out; pv = pvf; do { if (pvf == NULL) pvf = pv; pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { pvh_gen = pvh->pv_gen; md_gen = m->md.pv_gen; rw_wunlock(lock); PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); goto retry; } } pde = pmap_pde(pmap, pv->pv_va, &lvl); KASSERT(pde != NULL, ("pmap_ts_referenced: no l2 table found")); KASSERT(lvl == 2, ("pmap_ts_referenced: invalid pde level %d", lvl)); tpde = pmap_load(pde); KASSERT((tpde & ATTR_DESCR_MASK) == L2_TABLE, ("pmap_ts_referenced: found an invalid l2 table")); pte = pmap_l2_to_l3(pde, pv->pv_va); tpte = pmap_load(pte); if ((tpte & ATTR_AF) != 0) { if (safe_to_clear_referenced(pmap, tpte)) { /* * TODO: We don't handle the access flag * at all. We need to be able to set it in * the exception handler. */ panic("ARM64TODO: safe_to_clear_referenced\n"); } else if ((tpte & ATTR_SW_WIRED) == 0) { /* * Wired pages cannot be paged out so * doing accessed bit emulation for * them is wasted effort. We do the * hard work for unwired pages only. */ pmap_remove_l3(pmap, pte, pv->pv_va, tpde, &free, &lock); pmap_invalidate_page(pmap, pv->pv_va); cleared++; if (pvf == pv) pvf = NULL; pv = NULL; KASSERT(lock == VM_PAGE_TO_PV_LIST_LOCK(m), ("inconsistent pv lock %p %p for page %p", lock, VM_PAGE_TO_PV_LIST_LOCK(m), m)); } else not_cleared++; } PMAP_UNLOCK(pmap); /* Rotate the PV list if it has more than one entry. */ if (pv != NULL && TAILQ_NEXT(pv, pv_next) != NULL) { TAILQ_REMOVE(&m->md.pv_list, pv, pv_next); TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); m->md.pv_gen++; } } while ((pv = TAILQ_FIRST(&m->md.pv_list)) != pvf && cleared + not_cleared < PMAP_TS_REFERENCED_MAX); out: rw_wunlock(lock); pmap_free_zero_pages(&free); return (cleared + not_cleared); } /* * Apply the given advice to the specified range of addresses within the * given pmap. Depending on the advice, clear the referenced and/or * modified flags in each mapping and set the mapped page's dirty field. */ void pmap_advise(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, int advice) { } /* * Clear the modify bits on the specified physical page. */ void pmap_clear_modify(vm_page_t m) { KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("pmap_clear_modify: page %p is not managed", m)); VM_OBJECT_ASSERT_WLOCKED(m->object); KASSERT(!vm_page_xbusied(m), ("pmap_clear_modify: page %p is exclusive busied", m)); /* * If the page is not PGA_WRITEABLE, then no PTEs can have PG_M set. * If the object containing the page is locked and the page is not * exclusive busied, then PGA_WRITEABLE cannot be concurrently set. */ if ((m->aflags & PGA_WRITEABLE) == 0) return; /* ARM64TODO: We lack support for tracking if a page is modified */ } void * pmap_mapbios(vm_paddr_t pa, vm_size_t size) { return ((void *)PHYS_TO_DMAP(pa)); } void pmap_unmapbios(vm_paddr_t pa, vm_size_t size) { } /* * Sets the memory attribute for the specified page. */ void pmap_page_set_memattr(vm_page_t m, vm_memattr_t ma) { m->md.pv_memattr = ma; /* * If "m" is a normal page, update its direct mapping. This update * can be relied upon to perform any cache operations that are * required for data coherence. */ if ((m->flags & PG_FICTITIOUS) == 0 && pmap_change_attr(PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m)), PAGE_SIZE, m->md.pv_memattr) != 0) panic("memory attribute change on the direct map failed"); } /* * Changes the specified virtual address range's memory type to that given by * the parameter "mode". The specified virtual address range must be * completely contained within either the direct map or the kernel map. If * the virtual address range is contained within the kernel map, then the * memory type for each of the corresponding ranges of the direct map is also * changed. (The corresponding ranges of the direct map are those ranges that * map the same physical pages as the specified virtual address range.) These * changes to the direct map are necessary because Intel describes the * behavior of their processors as "undefined" if two or more mappings to the * same physical page have different memory types. * * Returns zero if the change completed successfully, and either EINVAL or * ENOMEM if the change failed. Specifically, EINVAL is returned if some part * of the virtual address range was not mapped, and ENOMEM is returned if * there was insufficient memory available to complete the change. In the * latter case, the memory type may have been changed on some part of the * virtual address range or the direct map. */ static int pmap_change_attr(vm_offset_t va, vm_size_t size, int mode) { int error; PMAP_LOCK(kernel_pmap); error = pmap_change_attr_locked(va, size, mode); PMAP_UNLOCK(kernel_pmap); return (error); } static int pmap_change_attr_locked(vm_offset_t va, vm_size_t size, int mode) { vm_offset_t base, offset, tmpva; pt_entry_t l3, *pte, *newpte; int lvl; PMAP_LOCK_ASSERT(kernel_pmap, MA_OWNED); base = trunc_page(va); offset = va & PAGE_MASK; size = round_page(offset + size); if (!VIRT_IN_DMAP(base)) return (EINVAL); for (tmpva = base; tmpva < base + size; ) { pte = pmap_pte(kernel_pmap, va, &lvl); if (pte == NULL) return (EINVAL); if ((pmap_load(pte) & ATTR_IDX_MASK) == ATTR_IDX(mode)) { /* * We already have the correct attribute, * ignore this entry. */ switch (lvl) { default: panic("Invalid DMAP table level: %d\n", lvl); case 1: tmpva = (tmpva & ~L1_OFFSET) + L1_SIZE; break; case 2: tmpva = (tmpva & ~L2_OFFSET) + L2_SIZE; break; case 3: tmpva += PAGE_SIZE; break; } } else { /* * Split the entry to an level 3 table, then * set the new attribute. */ switch (lvl) { default: panic("Invalid DMAP table level: %d\n", lvl); case 1: newpte = pmap_demote_l1(kernel_pmap, pte, tmpva & ~L1_OFFSET); if (newpte == NULL) return (EINVAL); pte = pmap_l1_to_l2(pte, tmpva); case 2: newpte = pmap_demote_l2(kernel_pmap, pte, tmpva & ~L2_OFFSET); if (newpte == NULL) return (EINVAL); pte = pmap_l2_to_l3(pte, tmpva); case 3: /* Update the entry */ l3 = pmap_load(pte); l3 &= ~ATTR_IDX_MASK; l3 |= ATTR_IDX(mode); pmap_update_entry(kernel_pmap, pte, l3, tmpva, PAGE_SIZE); /* * If moving to a non-cacheable entry flush * the cache. */ if (mode == VM_MEMATTR_UNCACHEABLE) cpu_dcache_wbinv_range(tmpva, L3_SIZE); break; } tmpva += PAGE_SIZE; } } return (0); } /* * Create an L2 table to map all addresses within an L1 mapping. */ static pt_entry_t * pmap_demote_l1(pmap_t pmap, pt_entry_t *l1, vm_offset_t va) { pt_entry_t *l2, newl2, oldl1; vm_offset_t tmpl1; vm_paddr_t l2phys, phys; vm_page_t ml2; int i; PMAP_LOCK_ASSERT(pmap, MA_OWNED); oldl1 = pmap_load(l1); KASSERT((oldl1 & ATTR_DESCR_MASK) == L1_BLOCK, ("pmap_demote_l1: Demoting a non-block entry")); KASSERT((va & L1_OFFSET) == 0, ("pmap_demote_l1: Invalid virtual address %#lx", va)); KASSERT((oldl1 & ATTR_SW_MANAGED) == 0, ("pmap_demote_l1: Level 1 table shouldn't be managed")); tmpl1 = 0; if (va <= (vm_offset_t)l1 && va + L1_SIZE > (vm_offset_t)l1) { tmpl1 = kva_alloc(PAGE_SIZE); if (tmpl1 == 0) return (NULL); } if ((ml2 = vm_page_alloc(NULL, 0, VM_ALLOC_INTERRUPT | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED)) == NULL) { CTR2(KTR_PMAP, "pmap_demote_l1: failure for va %#lx" " in pmap %p", va, pmap); return (NULL); } l2phys = VM_PAGE_TO_PHYS(ml2); l2 = (pt_entry_t *)PHYS_TO_DMAP(l2phys); /* Address the range points at */ phys = oldl1 & ~ATTR_MASK; /* The attributed from the old l1 table to be copied */ newl2 = oldl1 & ATTR_MASK; /* Create the new entries */ for (i = 0; i < Ln_ENTRIES; i++) { l2[i] = newl2 | phys; phys += L2_SIZE; } cpu_dcache_wb_range((vm_offset_t)l2, PAGE_SIZE); KASSERT(l2[0] == ((oldl1 & ~ATTR_DESCR_MASK) | L2_BLOCK), ("Invalid l2 page (%lx != %lx)", l2[0], (oldl1 & ~ATTR_DESCR_MASK) | L2_BLOCK)); if (tmpl1 != 0) { pmap_kenter(tmpl1, PAGE_SIZE, DMAP_TO_PHYS((vm_offset_t)l1) & ~L3_OFFSET, CACHED_MEMORY); l1 = (pt_entry_t *)(tmpl1 + ((vm_offset_t)l1 & PAGE_MASK)); } pmap_update_entry(pmap, l1, l2phys | L1_TABLE, va, PAGE_SIZE); if (tmpl1 != 0) { pmap_kremove(tmpl1); kva_free(tmpl1, PAGE_SIZE); } return (l2); } /* * Create an L3 table to map all addresses within an L2 mapping. */ static pt_entry_t * pmap_demote_l2_locked(pmap_t pmap, pt_entry_t *l2, vm_offset_t va, struct rwlock **lockp) { pt_entry_t *l3, newl3, oldl2; vm_offset_t tmpl2; vm_paddr_t l3phys, phys; vm_page_t ml3; int i; PMAP_LOCK_ASSERT(pmap, MA_OWNED); l3 = NULL; oldl2 = pmap_load(l2); KASSERT((oldl2 & ATTR_DESCR_MASK) == L2_BLOCK, ("pmap_demote_l2: Demoting a non-block entry")); KASSERT((va & L2_OFFSET) == 0, ("pmap_demote_l2: Invalid virtual address %#lx", va)); tmpl2 = 0; if (va <= (vm_offset_t)l2 && va + L2_SIZE > (vm_offset_t)l2) { tmpl2 = kva_alloc(PAGE_SIZE); if (tmpl2 == 0) return (NULL); } if ((ml3 = pmap_lookup_pt_page(pmap, va)) != NULL) { pmap_remove_pt_page(pmap, ml3); } else { ml3 = vm_page_alloc(NULL, pmap_l2_pindex(va), (VIRT_IN_DMAP(va) ? VM_ALLOC_INTERRUPT : VM_ALLOC_NORMAL) | VM_ALLOC_NOOBJ | VM_ALLOC_WIRED); if (ml3 == NULL) { CTR2(KTR_PMAP, "pmap_demote_l2: failure for va %#lx" " in pmap %p", va, pmap); goto fail; } if (va < VM_MAXUSER_ADDRESS) pmap_resident_count_inc(pmap, 1); } l3phys = VM_PAGE_TO_PHYS(ml3); l3 = (pt_entry_t *)PHYS_TO_DMAP(l3phys); /* Address the range points at */ phys = oldl2 & ~ATTR_MASK; /* The attributed from the old l2 table to be copied */ newl3 = (oldl2 & (ATTR_MASK & ~ATTR_DESCR_MASK)) | L3_PAGE; /* * If the page table page is new, initialize it. */ if (ml3->wire_count == 1) { for (i = 0; i < Ln_ENTRIES; i++) { l3[i] = newl3 | phys; phys += L3_SIZE; } cpu_dcache_wb_range((vm_offset_t)l3, PAGE_SIZE); } KASSERT(l3[0] == ((oldl2 & ~ATTR_DESCR_MASK) | L3_PAGE), ("Invalid l3 page (%lx != %lx)", l3[0], (oldl2 & ~ATTR_DESCR_MASK) | L3_PAGE)); /* * Map the temporary page so we don't lose access to the l2 table. */ if (tmpl2 != 0) { pmap_kenter(tmpl2, PAGE_SIZE, DMAP_TO_PHYS((vm_offset_t)l2) & ~L3_OFFSET, CACHED_MEMORY); l2 = (pt_entry_t *)(tmpl2 + ((vm_offset_t)l2 & PAGE_MASK)); } /* * The spare PV entries must be reserved prior to demoting the * mapping, that is, prior to changing the PDE. Otherwise, the state * of the L2 and the PV lists will be inconsistent, which can result * in reclaim_pv_chunk() attempting to remove a PV entry from the * wrong PV list and pmap_pv_demote_l2() failing to find the expected * PV entry for the 2MB page mapping that is being demoted. */ if ((oldl2 & ATTR_SW_MANAGED) != 0) reserve_pv_entries(pmap, Ln_ENTRIES - 1, lockp); pmap_update_entry(pmap, l2, l3phys | L2_TABLE, va, PAGE_SIZE); /* * Demote the PV entry. */ if ((oldl2 & ATTR_SW_MANAGED) != 0) pmap_pv_demote_l2(pmap, va, oldl2 & ~ATTR_MASK, lockp); atomic_add_long(&pmap_l2_demotions, 1); CTR3(KTR_PMAP, "pmap_demote_l2: success for va %#lx" " in pmap %p %lx", va, pmap, l3[0]); fail: if (tmpl2 != 0) { pmap_kremove(tmpl2); kva_free(tmpl2, PAGE_SIZE); } return (l3); } static pt_entry_t * pmap_demote_l2(pmap_t pmap, pt_entry_t *l2, vm_offset_t va) { struct rwlock *lock; pt_entry_t *l3; lock = NULL; l3 = pmap_demote_l2_locked(pmap, l2, va, &lock); if (lock != NULL) rw_wunlock(lock); return (l3); } /* * perform the pmap work for mincore */ int pmap_mincore(pmap_t pmap, vm_offset_t addr, vm_paddr_t *locked_pa) { pd_entry_t *l1p, l1; pd_entry_t *l2p, l2; pt_entry_t *l3p, l3; vm_paddr_t pa; bool managed; int val; PMAP_LOCK(pmap); retry: pa = 0; val = 0; managed = false; l1p = pmap_l1(pmap, addr); if (l1p == NULL) /* No l1 */ goto done; l1 = pmap_load(l1p); if ((l1 & ATTR_DESCR_MASK) == L1_INVAL) goto done; if ((l1 & ATTR_DESCR_MASK) == L1_BLOCK) { pa = (l1 & ~ATTR_MASK) | (addr & L1_OFFSET); managed = (l1 & ATTR_SW_MANAGED) == ATTR_SW_MANAGED; val = MINCORE_SUPER | MINCORE_INCORE; if (pmap_page_dirty(l1)) val |= MINCORE_MODIFIED | MINCORE_MODIFIED_OTHER; if ((l1 & ATTR_AF) == ATTR_AF) val |= MINCORE_REFERENCED | MINCORE_REFERENCED_OTHER; goto done; } l2p = pmap_l1_to_l2(l1p, addr); if (l2p == NULL) /* No l2 */ goto done; l2 = pmap_load(l2p); if ((l2 & ATTR_DESCR_MASK) == L2_INVAL) goto done; if ((l2 & ATTR_DESCR_MASK) == L2_BLOCK) { pa = (l2 & ~ATTR_MASK) | (addr & L2_OFFSET); managed = (l2 & ATTR_SW_MANAGED) == ATTR_SW_MANAGED; val = MINCORE_SUPER | MINCORE_INCORE; if (pmap_page_dirty(l2)) val |= MINCORE_MODIFIED | MINCORE_MODIFIED_OTHER; if ((l2 & ATTR_AF) == ATTR_AF) val |= MINCORE_REFERENCED | MINCORE_REFERENCED_OTHER; goto done; } l3p = pmap_l2_to_l3(l2p, addr); if (l3p == NULL) /* No l3 */ goto done; l3 = pmap_load(l2p); if ((l3 & ATTR_DESCR_MASK) == L3_INVAL) goto done; if ((l3 & ATTR_DESCR_MASK) == L3_PAGE) { pa = (l3 & ~ATTR_MASK) | (addr & L3_OFFSET); managed = (l3 & ATTR_SW_MANAGED) == ATTR_SW_MANAGED; val = MINCORE_INCORE; if (pmap_page_dirty(l3)) val |= MINCORE_MODIFIED | MINCORE_MODIFIED_OTHER; if ((l3 & ATTR_AF) == ATTR_AF) val |= MINCORE_REFERENCED | MINCORE_REFERENCED_OTHER; } done: if ((val & (MINCORE_MODIFIED_OTHER | MINCORE_REFERENCED_OTHER)) != (MINCORE_MODIFIED_OTHER | MINCORE_REFERENCED_OTHER) && managed) { /* Ensure that "PHYS_TO_VM_PAGE(pa)->object" doesn't change. */ if (vm_page_pa_tryrelock(pmap, pa, locked_pa)) goto retry; } else PA_UNLOCK_COND(*locked_pa); PMAP_UNLOCK(pmap); return (val); } void pmap_activate(struct thread *td) { pmap_t pmap; critical_enter(); pmap = vmspace_pmap(td->td_proc->p_vmspace); td->td_pcb->pcb_l0addr = vtophys(pmap->pm_l0); __asm __volatile("msr ttbr0_el1, %0" : : "r"(td->td_pcb->pcb_l0addr)); pmap_invalidate_all(pmap); critical_exit(); } void pmap_sync_icache(pmap_t pmap, vm_offset_t va, vm_size_t sz) { if (va >= VM_MIN_KERNEL_ADDRESS) { cpu_icache_sync_range(va, sz); } else { u_int len, offset; vm_paddr_t pa; /* Find the length of data in this page to flush */ offset = va & PAGE_MASK; len = imin(PAGE_SIZE - offset, sz); while (sz != 0) { /* Extract the physical address & find it in the DMAP */ pa = pmap_extract(pmap, va); if (pa != 0) cpu_icache_sync_range(PHYS_TO_DMAP(pa), len); /* Move to the next page */ sz -= len; va += len; /* Set the length for the next iteration */ len = imin(PAGE_SIZE, sz); } } } int pmap_fault(pmap_t pmap, uint64_t esr, uint64_t far) { #ifdef SMP uint64_t par; #endif switch (ESR_ELx_EXCEPTION(esr)) { case EXCP_DATA_ABORT_L: case EXCP_DATA_ABORT: break; default: return (KERN_FAILURE); } #ifdef SMP PMAP_LOCK(pmap); switch (esr & ISS_DATA_DFSC_MASK) { case ISS_DATA_DFSC_TF_L0: case ISS_DATA_DFSC_TF_L1: case ISS_DATA_DFSC_TF_L2: case ISS_DATA_DFSC_TF_L3: /* Ask the MMU to check the address */ if (pmap == kernel_pmap) par = arm64_address_translate_s1e1r(far); else par = arm64_address_translate_s1e0r(far); /* * If the translation was successful the address was invalid * due to a break-before-make sequence. We can unlock and * return success to the trap handler. */ if (PAR_SUCCESS(par)) { PMAP_UNLOCK(pmap); return (KERN_SUCCESS); } break; default: break; } PMAP_UNLOCK(pmap); #endif return (KERN_FAILURE); } /* * Increase the starting virtual address of the given mapping if a * different alignment might result in more superpage mappings. */ void pmap_align_superpage(vm_object_t object, vm_ooffset_t offset, vm_offset_t *addr, vm_size_t size) { vm_offset_t superpage_offset; if (size < L2_SIZE) return; if (object != NULL && (object->flags & OBJ_COLORED) != 0) offset += ptoa(object->pg_color); superpage_offset = offset & L2_OFFSET; if (size - ((L2_SIZE - superpage_offset) & L2_OFFSET) < L2_SIZE || (*addr & L2_OFFSET) == superpage_offset) return; if ((*addr & L2_OFFSET) < superpage_offset) *addr = (*addr & ~L2_OFFSET) + superpage_offset; else *addr = ((*addr + L2_OFFSET) & ~L2_OFFSET) + superpage_offset; } /** * Get the kernel virtual address of a set of physical pages. If there are * physical addresses not covered by the DMAP perform a transient mapping * that will be removed when calling pmap_unmap_io_transient. * * \param page The pages the caller wishes to obtain the virtual * address on the kernel memory map. * \param vaddr On return contains the kernel virtual memory address * of the pages passed in the page parameter. * \param count Number of pages passed in. * \param can_fault TRUE if the thread using the mapped pages can take * page faults, FALSE otherwise. * * \returns TRUE if the caller must call pmap_unmap_io_transient when * finished or FALSE otherwise. * */ boolean_t pmap_map_io_transient(vm_page_t page[], vm_offset_t vaddr[], int count, boolean_t can_fault) { vm_paddr_t paddr; boolean_t needs_mapping; int error, i; /* * Allocate any KVA space that we need, this is done in a separate * loop to prevent calling vmem_alloc while pinned. */ needs_mapping = FALSE; for (i = 0; i < count; i++) { paddr = VM_PAGE_TO_PHYS(page[i]); if (__predict_false(!PHYS_IN_DMAP(paddr))) { error = vmem_alloc(kernel_arena, PAGE_SIZE, M_BESTFIT | M_WAITOK, &vaddr[i]); KASSERT(error == 0, ("vmem_alloc failed: %d", error)); needs_mapping = TRUE; } else { vaddr[i] = PHYS_TO_DMAP(paddr); } } /* Exit early if everything is covered by the DMAP */ if (!needs_mapping) return (FALSE); if (!can_fault) sched_pin(); for (i = 0; i < count; i++) { paddr = VM_PAGE_TO_PHYS(page[i]); if (!PHYS_IN_DMAP(paddr)) { panic( "pmap_map_io_transient: TODO: Map out of DMAP data"); } } return (needs_mapping); } void pmap_unmap_io_transient(vm_page_t page[], vm_offset_t vaddr[], int count, boolean_t can_fault) { vm_paddr_t paddr; int i; if (!can_fault) sched_unpin(); for (i = 0; i < count; i++) { paddr = VM_PAGE_TO_PHYS(page[i]); if (!PHYS_IN_DMAP(paddr)) { panic("ARM64TODO: pmap_unmap_io_transient: Unmap data"); } } } Index: projects/netbsd-tests-update-12/sys/arm64/arm64/trap.c =================================================================== --- projects/netbsd-tests-update-12/sys/arm64/arm64/trap.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/arm64/arm64/trap.c (revision 305172) @@ -1,415 +1,416 @@ /*- * Copyright (c) 2014 Andrew Turner * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #ifdef KDB #include #endif #include #include #include #include #include #include #include #include #include #ifdef KDTRACE_HOOKS #include #endif #ifdef VFP #include #endif #ifdef KDB #include #endif #ifdef DDB #include #endif extern register_t fsu_intr_fault; /* Called from exception.S */ void do_el1h_sync(struct trapframe *); void do_el0_sync(struct trapframe *); void do_el0_error(struct trapframe *); static void print_registers(struct trapframe *frame); int (*dtrace_invop_jump_addr)(struct trapframe *); static __inline void call_trapsignal(struct thread *td, int sig, int code, void *addr) { ksiginfo_t ksi; ksiginfo_init_trap(&ksi); ksi.ksi_signo = sig; ksi.ksi_code = code; ksi.ksi_addr = addr; trapsignal(td, &ksi); } int cpu_fetch_syscall_args(struct thread *td, struct syscall_args *sa) { struct proc *p; register_t *ap; int nap; nap = 8; p = td->td_proc; ap = td->td_frame->tf_x; sa->code = td->td_frame->tf_x[8]; if (sa->code == SYS_syscall || sa->code == SYS___syscall) { sa->code = *ap++; nap--; } if (p->p_sysent->sv_mask) sa->code &= p->p_sysent->sv_mask; if (sa->code >= p->p_sysent->sv_size) sa->callp = &p->p_sysent->sv_table[0]; else sa->callp = &p->p_sysent->sv_table[sa->code]; sa->narg = sa->callp->sy_narg; memcpy(sa->args, ap, nap * sizeof(register_t)); if (sa->narg > nap) panic("ARM64TODO: Could we have more than 8 args?"); td->td_retval[0] = 0; td->td_retval[1] = 0; return (0); } #include "../../kern/subr_syscall.c" static void svc_handler(struct trapframe *frame) { struct syscall_args sa; struct thread *td; int error; td = curthread; error = syscallenter(td, &sa); syscallret(td, error, &sa); } static void data_abort(struct trapframe *frame, uint64_t esr, uint64_t far, int lower) { struct vm_map *map; struct thread *td; struct proc *p; struct pcb *pcb; vm_prot_t ftype; vm_offset_t va; int error, sig, ucode; /* * According to the ARMv8-A rev. A.g, B2.10.5 "Load-Exclusive * and Store-Exclusive instruction usage restrictions", state * of the exclusive monitors after data abort exception is unknown. */ clrex(); #ifdef KDB if (kdb_active) { kdb_reenter(); return; } #endif td = curthread; pcb = td->td_pcb; /* * Special case for fuswintr and suswintr. These can't sleep so * handle them early on in the trap handler. */ if (__predict_false(pcb->pcb_onfault == (vm_offset_t)&fsu_intr_fault)) { frame->tf_elr = pcb->pcb_onfault; return; } p = td->td_proc; if (lower) map = &p->p_vmspace->vm_map; else { /* The top bit tells us which range to use */ if ((far >> 63) == 1) { map = kernel_map; } else { map = &p->p_vmspace->vm_map; if (map == NULL) map = kernel_map; } } if (pmap_fault(map->pmap, esr, far) == KERN_SUCCESS) return; KASSERT(td->td_md.md_spinlock_count == 0, ("data abort with spinlock held")); if (td->td_critnest != 0 || WITNESS_CHECK(WARN_SLEEPOK | WARN_GIANTOK, NULL, "Kernel page fault") != 0) { print_registers(frame); printf(" far: %16lx\n", far); printf(" esr: %.8lx\n", esr); panic("data abort in critical section or under mutex"); } va = trunc_page(far); ftype = ((esr >> 6) & 1) ? VM_PROT_READ | VM_PROT_WRITE : VM_PROT_READ; /* Fault in the page. */ error = vm_fault(map, va, ftype, VM_FAULT_NORMAL); if (error != KERN_SUCCESS) { if (lower) { sig = SIGSEGV; if (error == KERN_PROTECTION_FAILURE) ucode = SEGV_ACCERR; else ucode = SEGV_MAPERR; call_trapsignal(td, sig, ucode, (void *)far); } else { if (td->td_intr_nesting_level == 0 && pcb->pcb_onfault != 0) { frame->tf_x[0] = error; frame->tf_elr = pcb->pcb_onfault; return; } printf("Fatal data abort:\n"); print_registers(frame); printf(" far: %16lx\n", far); printf(" esr: %.8lx\n", esr); #ifdef KDB if (debugger_on_panic || kdb_active) if (kdb_trap(ESR_ELx_EXCEPTION(esr), 0, frame)) return; #endif panic("vm_fault failed: %lx", frame->tf_elr); } } if (lower) userret(td, frame); } static void print_registers(struct trapframe *frame) { u_int reg; for (reg = 0; reg < 31; reg++) { printf(" %sx%d: %16lx\n", (reg < 10) ? " " : "", reg, frame->tf_x[reg]); } printf(" sp: %16lx\n", frame->tf_sp); printf(" lr: %16lx\n", frame->tf_lr); printf(" elr: %16lx\n", frame->tf_elr); printf("spsr: %16lx\n", frame->tf_spsr); } void do_el1h_sync(struct trapframe *frame) { uint32_t exception; uint64_t esr, far; /* Read the esr register to get the exception details */ esr = READ_SPECIALREG(esr_el1); exception = ESR_ELx_EXCEPTION(esr); #ifdef KDTRACE_HOOKS if (dtrace_trap_func != NULL && (*dtrace_trap_func)(frame, exception)) return; #endif CTR4(KTR_TRAP, "do_el1_sync: curthread: %p, esr %lx, elr: %lx, frame: %p", curthread, esr, frame->tf_elr, frame); switch(exception) { case EXCP_FP_SIMD: case EXCP_TRAP_FP: print_registers(frame); printf(" esr: %.8lx\n", esr); panic("VFP exception in the kernel"); + case EXCP_INSN_ABORT: case EXCP_DATA_ABORT: far = READ_SPECIALREG(far_el1); intr_enable(); data_abort(frame, esr, far, 0); break; case EXCP_BRK: #ifdef KDTRACE_HOOKS if ((esr & ESR_ELx_ISS_MASK) == 0x40d && \ dtrace_invop_jump_addr != 0) { dtrace_invop_jump_addr(frame); break; } #endif /* FALLTHROUGH */ case EXCP_WATCHPT_EL1: case EXCP_SOFTSTP_EL1: #ifdef KDB kdb_trap(exception, 0, frame); #else panic("No debugger in kernel.\n"); #endif break; default: print_registers(frame); panic("Unknown kernel exception %x esr_el1 %lx\n", exception, esr); } } /* * The attempted execution of an instruction bit pattern that has no allocated * instruction results in an exception with an unknown reason. */ static void el0_excp_unknown(struct trapframe *frame, uint64_t far) { struct thread *td; td = curthread; call_trapsignal(td, SIGILL, ILL_ILLTRP, (void *)far); userret(td, frame); } void do_el0_sync(struct trapframe *frame) { struct thread *td; uint32_t exception; uint64_t esr, far; /* Check we have a sane environment when entering from userland */ KASSERT((uintptr_t)get_pcpu() >= VM_MIN_KERNEL_ADDRESS, ("Invalid pcpu address from userland: %p (tpidr %lx)", get_pcpu(), READ_SPECIALREG(tpidr_el1))); td = curthread; td->td_frame = frame; esr = READ_SPECIALREG(esr_el1); exception = ESR_ELx_EXCEPTION(esr); switch (exception) { case EXCP_UNKNOWN: case EXCP_INSN_ABORT_L: case EXCP_DATA_ABORT_L: case EXCP_DATA_ABORT: far = READ_SPECIALREG(far_el1); } intr_enable(); CTR4(KTR_TRAP, "do_el0_sync: curthread: %p, esr %lx, elr: %lx, frame: %p", curthread, esr, frame->tf_elr, frame); switch(exception) { case EXCP_FP_SIMD: case EXCP_TRAP_FP: #ifdef VFP vfp_restore_state(); #else panic("VFP exception in userland"); #endif break; case EXCP_SVC: svc_handler(frame); break; case EXCP_INSN_ABORT_L: case EXCP_DATA_ABORT_L: case EXCP_DATA_ABORT: data_abort(frame, esr, far, 1); break; case EXCP_UNKNOWN: el0_excp_unknown(frame, far); break; case EXCP_SP_ALIGN: call_trapsignal(td, SIGBUS, BUS_ADRALN, (void *)frame->tf_sp); userret(td, frame); break; case EXCP_PC_ALIGN: call_trapsignal(td, SIGBUS, BUS_ADRALN, (void *)frame->tf_elr); userret(td, frame); break; case EXCP_BRK: call_trapsignal(td, SIGTRAP, TRAP_BRKPT, (void *)frame->tf_elr); userret(td, frame); break; case EXCP_SOFTSTP_EL0: td->td_frame->tf_spsr &= ~PSR_SS; td->td_pcb->pcb_flags &= ~PCB_SINGLE_STEP; WRITE_SPECIALREG(MDSCR_EL1, READ_SPECIALREG(MDSCR_EL1) & ~DBG_MDSCR_SS); call_trapsignal(td, SIGTRAP, TRAP_TRACE, (void *)frame->tf_elr); userret(td, frame); break; default: print_registers(frame); panic("Unknown userland exception %x esr_el1 %lx\n", exception, esr); } } void do_el0_error(struct trapframe *frame) { panic("ARM64TODO: do_el0_error"); } Index: projects/netbsd-tests-update-12/sys/boot/common/dev_net.c =================================================================== --- projects/netbsd-tests-update-12/sys/boot/common/dev_net.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/common/dev_net.c (revision 305172) @@ -1,382 +1,391 @@ /* $NetBSD: dev_net.c,v 1.23 2008/04/28 20:24:06 martin Exp $ */ /*- * Copyright (c) 1997 The NetBSD Foundation, Inc. * All rights reserved. * * This code is derived from software contributed to The NetBSD Foundation * by Gordon W. Ross. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); /*- * This module implements a "raw device" interface suitable for * use by the stand-alone I/O library NFS code. This interface * does not support any "block" access, and exists only for the * purpose of initializing the network interface, getting boot * parameters, and performing the NFS mount. * * At open time, this does: * * find interface - netif_open() * RARP for IP address - rarp_getipaddress() * RPC/bootparams - callrpc(d, RPC_BOOTPARAMS, ...) * RPC/mountd - nfs_mount(sock, ip, path) * * the root file handle from mountd is saved in a global * for use by the NFS open code (NFS/lookup). */ #include #include #include #include #include #include #include #include #include #include #include #include #include "dev_net.h" #include "bootstrap.h" #ifdef NETIF_DEBUG int debug = 0; #endif static char *netdev_name; static int netdev_sock = -1; static int netdev_opens; static int net_init(void); static int net_open(struct open_file *, ...); static int net_close(struct open_file *); static void net_cleanup(void); static int net_strategy(); static void net_print(int); static int net_getparams(int sock); struct devsw netdev = { "net", DEVT_NET, net_init, net_strategy, net_open, net_close, noioctl, net_print, net_cleanup }; static int net_init(void) { return (0); } /* * Called by devopen after it sets f->f_dev to our devsw entry. * This opens the low-level device and sets f->f_devdata. * This is declared with variable arguments... */ static int net_open(struct open_file *f, ...) { struct iodesc *d; va_list args; char *devname; /* Device part of file name (or NULL). */ int error = 0; va_start(args, f); devname = va_arg(args, char*); va_end(args); #ifdef NETIF_OPEN_CLOSE_ONCE /* Before opening another interface, close the previous one first. */ if (netdev_sock >= 0 && strcmp(devname, netdev_name) != 0) net_cleanup(); #endif /* On first open, do netif open, mount, etc. */ if (netdev_opens == 0) { /* Find network interface. */ if (netdev_sock < 0) { netdev_sock = netif_open(devname); if (netdev_sock < 0) { printf("net_open: netif_open() failed\n"); return (ENXIO); } netdev_name = strdup(devname); #ifdef NETIF_DEBUG if (debug) printf("net_open: netif_open() succeeded\n"); #endif } /* * If network params were not set by netif_open(), try to get * them via bootp, rarp, etc. */ if (rootip.s_addr == 0) { /* Get root IP address, and path, etc. */ error = net_getparams(netdev_sock); if (error) { /* getparams makes its own noise */ free(netdev_name); netif_close(netdev_sock); netdev_sock = -1; return (error); } } /* * Set the variables required by the kernel's nfs_diskless * mechanism. This is the minimum set of variables required to * mount a root filesystem without needing to obtain additional * info from bootp or other sources. */ d = socktodesc(netdev_sock); setenv("boot.netif.hwaddr", ether_sprintf(d->myea), 1); setenv("boot.netif.ip", inet_ntoa(myip), 1); setenv("boot.netif.netmask", intoa(netmask), 1); setenv("boot.netif.gateway", inet_ntoa(gateip), 1); -#ifdef LOADER_TFTP_SUPPORT - setenv("boot.tftproot.server", inet_ntoa(rootip), 1); - setenv("boot.tftproot.path", rootpath, 1); -#else - setenv("boot.nfsroot.server", inet_ntoa(rootip), 1); - setenv("boot.nfsroot.path", rootpath, 1); -#endif + setenv("boot.netif.server", inet_ntoa(rootip), 1); + if (netproto == NET_TFTP) { + setenv("boot.tftproot.server", inet_ntoa(rootip), 1); + setenv("boot.tftproot.path", rootpath, 1); + } else if (netproto == NET_NFS) { + setenv("boot.nfsroot.server", inet_ntoa(rootip), 1); + setenv("boot.nfsroot.path", rootpath, 1); + } if (intf_mtu != 0) { char mtu[16]; sprintf(mtu, "%u", intf_mtu); setenv("boot.netif.mtu", mtu, 1); } } netdev_opens++; f->f_devdata = &netdev_sock; return (error); } static int net_close(struct open_file *f) { #ifdef NETIF_DEBUG if (debug) printf("net_close: opens=%d\n", netdev_opens); #endif f->f_devdata = NULL; #ifndef NETIF_OPEN_CLOSE_ONCE /* Extra close call? */ if (netdev_opens <= 0) return (0); netdev_opens--; /* Not last close? */ if (netdev_opens > 0) return (0); /* On last close, do netif close, etc. */ #ifdef NETIF_DEBUG if (debug) printf("net_close: calling net_cleanup()\n"); #endif net_cleanup(); #endif return (0); } static void net_cleanup(void) { if (netdev_sock >= 0) { #ifdef NETIF_DEBUG if (debug) printf("net_cleanup: calling netif_close()\n"); #endif rootip.s_addr = 0; free(netdev_name); netif_close(netdev_sock); netdev_sock = -1; } } static int net_strategy() { return (EIO); } #define SUPPORT_BOOTP /* * Get info for NFS boot: our IP address, our hostname, * server IP address, and our root path on the server. * There are two ways to do this: The old, Sun way, * and the more modern, BOOTP way. (RFC951, RFC1048) * * The default is to use the Sun bootparams RPC * (because that is what the kernel will do). * MD code can make try_bootp initialied data, * which will override this common definition. */ #ifdef SUPPORT_BOOTP int try_bootp = 1; #endif extern n_long ip_convertaddr(char *p); static int net_getparams(int sock) { char buf[MAXHOSTNAMELEN]; n_long rootaddr, smask; #ifdef SUPPORT_BOOTP /* * Try to get boot info using BOOTP. If we succeed, then * the server IP address, gateway, and root path will all * be initialized. If any remain uninitialized, we will * use RARP and RPC/bootparam (the Sun way) to get them. */ if (try_bootp) bootp(sock, BOOTP_NONE); if (myip.s_addr != 0) goto exit; #ifdef NETIF_DEBUG if (debug) printf("net_open: BOOTP failed, trying RARP/RPC...\n"); #endif #endif /* * Use RARP to get our IP address. This also sets our * netmask to the "natural" default for our address. */ if (rarp_getipaddress(sock)) { printf("net_open: RARP failed\n"); return (EIO); } printf("net_open: client addr: %s\n", inet_ntoa(myip)); /* Get our hostname, server IP address, gateway. */ if (bp_whoami(sock)) { printf("net_open: bootparam/whoami RPC failed\n"); return (EIO); } #ifdef NETIF_DEBUG if (debug) printf("net_open: client name: %s\n", hostname); #endif /* * Ignore the gateway from whoami (unreliable). * Use the "gateway" parameter instead. */ smask = 0; gateip.s_addr = 0; if (bp_getfile(sock, "gateway", &gateip, buf) == 0) { /* Got it! Parse the netmask. */ smask = ip_convertaddr(buf); } if (smask) { netmask = smask; #ifdef NETIF_DEBUG if (debug) printf("net_open: subnet mask: %s\n", intoa(netmask)); #endif } #ifdef NETIF_DEBUG if (gateip.s_addr && debug) printf("net_open: net gateway: %s\n", inet_ntoa(gateip)); #endif /* Get the root server and pathname. */ if (bp_getfile(sock, "root", &rootip, rootpath)) { printf("net_open: bootparam/getfile RPC failed\n"); return (EIO); } exit: if ((rootaddr = net_parse_rootpath()) != INADDR_NONE) rootip.s_addr = rootaddr; #ifdef NETIF_DEBUG if (debug) { printf("net_open: server addr: %s\n", inet_ntoa(rootip)); printf("net_open: server path: %s\n", rootpath); } #endif return (0); } static void net_print(int verbose) { struct netif_driver *drv; int i, d, cnt; cnt = 0; for (d = 0; netif_drivers[d]; d++) { drv = netif_drivers[d]; for (i = 0; i < drv->netif_nifs; i++) { printf("\t%s%d:", "net", cnt++); if (verbose) printf(" (%s%d)", drv->netif_bname, drv->netif_ifs[i].dif_unit); } } printf("\n"); } /* * Strip the server's address off of the rootpath if present and return it in * network byte order, leaving just the pathname part in the global rootpath. */ uint32_t net_parse_rootpath() { - int i; + int i, ipstart; n_long addr = INADDR_NONE; + netproto = NET_NFS; + + if (tftpip.s_addr != 0) { + netproto = NET_TFTP; + addr = tftpip.s_addr; + } + for (i = 0; rootpath[i] != '\0' && i < FNAME_SIZE; i++) if (rootpath[i] == ':') break; if (i && i != FNAME_SIZE && rootpath[i] == ':') { rootpath[i++] = '\0'; - addr = inet_addr(&rootpath[0]); + addr = inet_addr(&rootpath[ipstart]); bcopy(&rootpath[i], rootpath, strlen(&rootpath[i])+1); } + return (addr); } Index: projects/netbsd-tests-update-12/sys/boot/efi/boot1/Makefile =================================================================== --- projects/netbsd-tests-update-12/sys/boot/efi/boot1/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/efi/boot1/Makefile (revision 305172) @@ -1,141 +1,145 @@ # $FreeBSD$ MAN= .include MK_SSP= no PROG= boot1.sym INTERNALPROG= WARNS?= 6 .if ${MK_ZFS} != "no" # Disable warnings that are currently incompatible with the zfs boot code CWARNFLAGS.zfs_module.c += -Wno-array-bounds CWARNFLAGS.zfs_module.c += -Wno-cast-align CWARNFLAGS.zfs_module.c += -Wno-cast-qual CWARNFLAGS.zfs_module.c += -Wno-missing-prototypes CWARNFLAGS.zfs_module.c += -Wno-sign-compare CWARNFLAGS.zfs_module.c += -Wno-unused-parameter CWARNFLAGS.zfs_module.c += -Wno-unused-function CWARNFLAGS.skein.c += -Wno-cast-align CWARNFLAGS.skein.c += -Wno-missing-variable-declarations .endif # architecture-specific loader code SRCS= boot1.c self_reloc.c start.S ufs_module.c .if ${MK_ZFS} != "no" SRCS+= zfs_module.c SRCS+= skein.c skein_block.c .PATH: ${.CURDIR}/../../../crypto/skein .endif CFLAGS+= -I. CFLAGS+= -I${.CURDIR}/../include CFLAGS+= -I${.CURDIR}/../include/${MACHINE} CFLAGS+= -I${.CURDIR}/../../../contrib/dev/acpica/include CFLAGS+= -I${.CURDIR}/../../.. CFLAGS+= -DEFI_UFS_BOOT .ifdef(EFI_DEBUG) CFLAGS+= -DEFI_DEBUG .endif .if ${MK_ZFS} != "no" CFLAGS+= -I${.CURDIR}/../../zfs/ CFLAGS+= -I${.CURDIR}/../../../cddl/boot/zfs/ CFLAGS+= -I${.CURDIR}/../../../crypto/skein CFLAGS+= -DEFI_ZFS_BOOT .endif # Always add MI sources and REGULAR efi loader bits .PATH: ${.CURDIR}/../loader/arch/${MACHINE} .PATH: ${.CURDIR}/../loader .PATH: ${.CURDIR}/../../common CFLAGS+= -I${.CURDIR}/../../common FILES= boot1.efi boot1.efifat FILESMODE_boot1.efi= ${BINMODE} LDSCRIPT= ${.CURDIR}/../loader/arch/${MACHINE}/ldscript.${MACHINE} LDFLAGS+= -Wl,-T${LDSCRIPT} -Wl,-Bsymbolic -shared .if ${MACHINE_CPUARCH} == "aarch64" CFLAGS+= -msoft-float -mgeneral-regs-only .endif .if ${MACHINE_CPUARCH} == "amd64" || ${MACHINE_CPUARCH} == "i386" CFLAGS+= -fPIC LDFLAGS+= -Wl,-znocombreloc .endif # # Add libstand for the runtime functions used by the compiler - for example # __aeabi_* (arm) or __divdi3 (i386). # as well as required string and memory functions for all platforms. # DPADD+= ${LIBSTAND} LDADD+= -lstand DPADD+= ${LDSCRIPT} NM?= nm OBJCOPY?= objcopy .if ${MACHINE_CPUARCH} == "amd64" EFI_TARGET= efi-app-x86_64 .elif ${MACHINE_CPUARCH} == "i386" EFI_TARGET= efi-app-ia32 .else EFI_TARGET= binary .endif +# Arbitrarily set the PE/COFF header timestamps to 1 Jan 2016 00:00:00 +# for build reproducibility. +SOURCE_DATE_EPOCH?=1451606400 boot1.efi: ${PROG} if ${NM} ${.ALLSRC} | grep ' U '; then \ echo "Undefined symbols in ${.ALLSRC}"; \ exit 1; \ fi + SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH} \ ${OBJCOPY} -j .peheader -j .text -j .sdata -j .data \ -j .dynamic -j .dynsym -j .rel.dyn \ -j .rela.dyn -j .reloc -j .eh_frame \ --output-target=${EFI_TARGET} ${.ALLSRC} ${.TARGET} boot1.o: ${.CURDIR}/../../common/ufsread.c # The following inserts our objects into a template FAT file system # created by generate-fat.sh .include "${.CURDIR}/Makefile.fat" BOOT1_MAXSIZE?= 131072 boot1.efifat: boot1.efi @set -- `ls -l boot1.efi`; \ x=$$(($$5-${BOOT1_MAXSIZE})); \ if [ $$x -ge 0 ]; then \ echo "boot1 $$x bytes too large; regenerate FAT templates?" >&2 ;\ exit 1; \ fi echo ${.OBJDIR} uudecode ${.CURDIR}/fat-${MACHINE}.tmpl.bz2.uu mv fat-${MACHINE}.tmpl.bz2 ${.TARGET}.bz2 bzip2 -f -d ${.TARGET}.bz2 dd if=boot1.efi of=${.TARGET} seek=${BOOT1_OFFSET} conv=notrunc \ status=none CLEANFILES= boot1.efi boot1.efifat .include beforedepend ${OBJS}: machine CLEANFILES+= machine machine: .NOMETA ln -sf ${.CURDIR}/../../../${MACHINE}/include machine .if ${MACHINE_CPUARCH} == "amd64" || ${MACHINE_CPUARCH} == "i386" beforedepend ${OBJS}: x86 CLEANFILES+= x86 x86: .NOMETA ln -sf ${.CURDIR}/../../../x86/include x86 .endif Index: projects/netbsd-tests-update-12/sys/boot/efi/libefi/Makefile =================================================================== --- projects/netbsd-tests-update-12/sys/boot/efi/libefi/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/efi/libefi/Makefile (revision 305172) @@ -1,48 +1,44 @@ # $FreeBSD$ LIB= efi INTERNALLIB= WARNS?= 2 SRCS= delay.c devpath.c efi_console.c efinet.c efipart.c env.c errno.c \ handles.c libefi.c .if ${MACHINE_CPUARCH} == "amd64" || ${MACHINE_CPUARCH} == "i386" SRCS+= time.c .elif ${MACHINE_CPUARCH} == "aarch64" || ${MACHINE_CPUARCH} == "arm" SRCS+= time_event.c .endif -.if defined(LOADER_TFTP_SUPPORT) -CFLAGS+= -DLOADER_TFTP_SUPPORT -DNETIF_OPEN_CLOSE_ONCE -.endif - # We implement a slightly non-standard %S in that it always takes a # CHAR16 that's common in UEFI-land instead of a wchar_t. This only # seems to matter on arm64 where wchar_t defaults to an int instead # of a short. There's no good cast to use here so just ignore the # warnings for now. CWARNFLAGS.efinet.c+= -Wno-format .if ${MACHINE_CPUARCH} == "aarch64" CFLAGS+= -msoft-float -mgeneral-regs-only .endif .if ${MACHINE_ARCH} == "amd64" CFLAGS+= -fPIC -mno-red-zone .endif CFLAGS+= -I${.CURDIR}/../include CFLAGS+= -I${.CURDIR}/../include/${MACHINE} CFLAGS+= -I${.CURDIR}/../../../../lib/libstand # Pick up the bootstrap header for some interface items CFLAGS+= -I${.CURDIR}/../../common # Handle FreeBSD specific %b and %D printf format specifiers CFLAGS+= ${FORMAT_EXTENSIONS} # Do not use TERM_EMU on arm and arm64 as it doesn't behave well with serial console .if ${MACHINE_CPUARCH} != "arm" && ${MACHINE_CPUARCH} != "aarch64" CFLAGS+= -DTERM_EMU .endif .include Index: projects/netbsd-tests-update-12/sys/boot/efi/loader/Makefile =================================================================== --- projects/netbsd-tests-update-12/sys/boot/efi/loader/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/efi/loader/Makefile (revision 305172) @@ -1,169 +1,169 @@ # $FreeBSD$ MAN= .include MK_SSP= no PROG= loader.sym INTERNALPROG= WARNS?= 3 # architecture-specific loader code SRCS= autoload.c \ bootinfo.c \ conf.c \ copy.c \ devicename.c \ main.c \ self_reloc.c \ smbios.c \ vers.c -.if defined(LOADER_TFTP_SUPPORT) -CFLAGS+= -DLOADER_TFTP_SUPPORT -DNETIF_OPEN_CLOSE_ONCE -.endif - .if ${MK_ZFS} != "no" SRCS+= zfs.c .PATH: ${.CURDIR}/../../zfs SRCS+= skein.c skein_block.c .PATH: ${.CURDIR}/../../../crypto/skein # Disable warnings that are currently incompatible with the zfs boot code CWARNFLAGS.zfs.c+= -Wno-sign-compare CWARNFLAGS.zfs.c+= -Wno-array-bounds CWARNFLAGS.zfs.c+= -Wno-missing-prototypes .endif # We implement a slightly non-standard %S in that it always takes a # CHAR16 that's common in UEFI-land instead of a wchar_t. This only # seems to matter on arm64 where wchar_t defaults to an int instead # of a short. There's no good cast to use here so just ignore the # warnings for now. CWARNFLAGS.main.c+= -Wno-format .PATH: ${.CURDIR}/arch/${MACHINE} # For smbios.c .PATH: ${.CURDIR}/../../i386/libi386 .include "${.CURDIR}/arch/${MACHINE}/Makefile.inc" CFLAGS+= -I${.CURDIR} CFLAGS+= -I${.CURDIR}/arch/${MACHINE} CFLAGS+= -I${.CURDIR}/../include CFLAGS+= -I${.CURDIR}/../include/${MACHINE} CFLAGS+= -I${.CURDIR}/../../../contrib/dev/acpica/include CFLAGS+= -I${.CURDIR}/../../.. CFLAGS+= -I${.CURDIR}/../../i386/libi386 .if ${MK_ZFS} != "no" CFLAGS+= -I${.CURDIR}/../../zfs CFLAGS+= -I${.CURDIR}/../../../cddl/boot/zfs CFLAGS+= -I${.CURDIR}/../../../crypto/skein CFLAGS+= -DEFI_ZFS_BOOT .endif CFLAGS+= -DNO_PCI -DEFI # make buildenv doesn't set DESTDIR, this means LIBSTAND # will be wrong when crossbuilding. .if exists(${.OBJDIR}/../../../../lib/libstand/libstand.a) LIBSTAND= ${.OBJDIR}/../../../../lib/libstand/libstand.a .endif .if !defined(BOOT_HIDE_SERIAL_NUMBERS) # Export serial numbers, UUID, and asset tag from loader. CFLAGS+= -DSMBIOS_SERIAL_NUMBERS .if defined(BOOT_LITTLE_ENDIAN_UUID) # Use little-endian UUID format as defined in SMBIOS 2.6. CFLAGS+= -DSMBIOS_LITTLE_ENDIAN_UUID .elif defined(BOOT_NETWORK_ENDIAN_UUID) # Use network-endian UUID format for backward compatibility. CFLAGS+= -DSMBIOS_NETWORK_ENDIAN_UUID .endif .endif .if ${MK_FORTH} != "no" BOOT_FORTH= yes CFLAGS+= -DBOOT_FORTH CFLAGS+= -I${.CURDIR}/../../ficl CFLAGS+= -I${.CURDIR}/../../ficl/${MACHINE_CPUARCH} LIBFICL= ${.OBJDIR}/../../ficl/libficl.a .endif LOADER_FDT_SUPPORT?= no .if ${MK_FDT} != "no" && ${LOADER_FDT_SUPPORT} != "no" CFLAGS+= -I${.CURDIR}/../../fdt CFLAGS+= -I${.OBJDIR}/../../fdt CFLAGS+= -DLOADER_FDT_SUPPORT LIBEFI_FDT= ${.OBJDIR}/../../efi/fdt/libefi_fdt.a LIBFDT= ${.OBJDIR}/../../fdt/libfdt.a .endif # Include bcache code. HAVE_BCACHE= yes .if defined(EFI_STAGING_SIZE) CFLAGS+= -DEFI_STAGING_SIZE=${EFI_STAGING_SIZE} .endif # Always add MI sources .PATH: ${.CURDIR}/../../common .include "${.CURDIR}/../../common/Makefile.inc" CFLAGS+= -I${.CURDIR}/../../common FILES+= loader.efi FILESMODE_loader.efi= ${BINMODE} LDSCRIPT= ${.CURDIR}/arch/${MACHINE}/ldscript.${MACHINE} LDFLAGS+= -Wl,-T${LDSCRIPT} -Wl,-Bsymbolic -shared CLEANFILES+= vers.c loader.efi NEWVERSWHAT= "EFI loader" ${MACHINE} vers.c: ${.CURDIR}/../../common/newvers.sh ${.CURDIR}/../../efi/loader/version sh ${.CURDIR}/../../common/newvers.sh ${.CURDIR}/version ${NEWVERSWHAT} NM?= nm OBJCOPY?= objcopy .if ${MACHINE_CPUARCH} == "amd64" EFI_TARGET= efi-app-x86_64 .elif ${MACHINE_CPUARCH} == "i386" EFI_TARGET= efi-app-ia32 .else EFI_TARGET= binary .endif +# Arbitrarily set the PE/COFF header timestamps to 1 Jan 2016 00:00:00 +# for build reproducibility. +SOURCE_DATE_EPOCH?=1451606400 loader.efi: ${PROG} if ${NM} ${.ALLSRC} | grep ' U '; then \ echo "Undefined symbols in ${.ALLSRC}"; \ exit 1; \ fi + SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH} \ ${OBJCOPY} -j .peheader -j .text -j .sdata -j .data \ -j .dynamic -j .dynsym -j .rel.dyn \ -j .rela.dyn -j .reloc -j .eh_frame -j set_Xcommand_set \ --output-target=${EFI_TARGET} ${.ALLSRC} ${.TARGET} LIBEFI= ${.OBJDIR}/../libefi/libefi.a DPADD= ${LIBFICL} ${LIBEFI} ${LIBFDT} ${LIBEFI_FDT} ${LIBSTAND} \ ${LDSCRIPT} LDADD= ${LIBFICL} ${LIBEFI} ${LIBFDT} ${LIBEFI_FDT} ${LIBSTAND} .include beforedepend ${OBJS}: machine CLEANFILES+= machine machine: .NOMETA ln -sf ${.CURDIR}/../../../${MACHINE}/include machine .if ${MACHINE_CPUARCH} == "amd64" || ${MACHINE_CPUARCH} == "i386" beforedepend ${OBJS}: x86 CLEANFILES+= x86 x86: .NOMETA ln -sf ${.CURDIR}/../../../x86/include x86 .endif Index: projects/netbsd-tests-update-12/sys/boot/efi/loader/conf.c =================================================================== --- projects/netbsd-tests-update-12/sys/boot/efi/loader/conf.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/efi/loader/conf.c (revision 305172) @@ -1,82 +1,79 @@ /*- * Copyright (c) 2006 Marcel Moolenaar * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #ifdef EFI_ZFS_BOOT #include #endif struct devsw *devsw[] = { &efipart_dev, &efinet_dev, #ifdef EFI_ZFS_BOOT &zfs_dev, #endif NULL }; struct fs_ops *file_system[] = { #ifdef EFI_ZFS_BOOT &zfs_fsops, #endif &dosfs_fsops, &ufs_fsops, &cd9660_fsops, -#ifdef LOADER_TFTP_SUPPORT &tftp_fsops, -#else &nfs_fsops, -#endif &gzipfs_fsops, &bzipfs_fsops, NULL }; struct netif_driver *netif_drivers[] = { &efinetif, NULL }; extern struct console efi_console; #if defined(__amd64__) || defined(__i386__) extern struct console comconsole; extern struct console nullconsole; #endif struct console *consoles[] = { &efi_console, #if defined(__amd64__) || defined(__i386__) &comconsole, &nullconsole, #endif NULL }; Index: projects/netbsd-tests-update-12/sys/boot/fdt/dts/Makefile =================================================================== --- projects/netbsd-tests-update-12/sys/boot/fdt/dts/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/fdt/dts/Makefile (revision 305172) @@ -1,5 +1,5 @@ # $FreeBSD$ -SUBDIR=arm mips powerpc +SUBDIR=arm arm64 mips powerpc .include Index: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/Makefile =================================================================== --- projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/Makefile (nonexistent) +++ projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/Makefile (revision 305172) @@ -0,0 +1,7 @@ +# $FreeBSD$ + +DTS!=ls *.dts + +all: test-dts + +.include Property changes on: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/Makefile ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/a64.dtsi =================================================================== --- projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/a64.dtsi (nonexistent) +++ projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/a64.dtsi (revision 305172) @@ -0,0 +1,151 @@ +/*- + * Copyright (c) 2016 Jared McNeill + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +/ { + clocks { + pll_hsic: clk@01c20044 { + #clock-cells = <0>; + compatible = "allwinner,sun50i-a64-pllhsic-clk"; + reg = <0x01c20044 0x4>; + clocks = <&osc24M>; + clock-output-names = "pll_hsic"; + }; + + usb_clk: clk@01c200cc { + #clock-cells = <1>; + #reset-cells = <1>; + compatible = "allwinner,sun8i-a83t-usb-clk"; + reg = <0x01c200cc 0x4>; + clocks = <&osc24M>, <&pll_hsic>; + clock-indices = <8>, <9>, + <10>, <11>, + <16>, <17>; + clock-output-names = "usb_phy0", "usb_phy1", + "usb_hsic_pll", "usb_hsic_12m", + "usb_otg_ohci", "usb_ohci0"; + }; + }; + + soc { + watchdog: watchdog@01c20ca0 { + compatible = "allwinner,sun6i-a31-wdt"; + reg = <0x01c20ca0 0x20>; + interrupts = ; + clocks = <&osc24M>; + }; + + nmi_intc: interrupt-controller@01f00c0c { + compatible = "allwinner,sun6i-a31-sc-nmi"; + interrupt-controller; + #interrupt-cells = <2>; + reg = <0x01f00c0c 0x38>; + interrupts = ; + }; + + r_rsb: i2c@01f03400 { + compatible = "allwinner,sun8i-a23-rsb"; + reg = <0x01f03400 0x400>; + interrupts = ; + clock-frequency = <3000000>; + pinctrl-names = "default"; + pinctrl-0 = <&r_rsb_pins>; + status = "disabled"; + #address-cells = <1>; + #size-cells = <0>; + }; + + usbphy: phy@01c19400 { + compatible = "allwinner,sun50i-a64-usb-phy"; + reg = <0x01c19400 0x24 0x01c1a800 0x4 0x01c1b800 0x4>; + reg-names = "phy_ctrl", "pmu1", "pmu2"; + clocks = <&usb_clk 8>, + <&usb_clk 9>; + clock-names = "usb0_phy", + "usb1_phy"; + resets = <&usb_clk 0>, + <&usb_clk 1>; + reset-names = "usb0_reset", + "usb1_reset"; + status = "disabled"; + #phy-cells = <1>; + }; + + ohci0: usb@01c1a400 { + compatible = "generic-ohci"; + reg = <0x01c1a400 0x100>; + interrupts = ; + clocks = <&bus_gates 28>, <&usb_clk 16>, <&usb_clk 17>; + resets = <&ahb_rst 28>; + phys = <&usbphy 1>; + phy-names = "usb"; + status = "disabled"; + }; + + ehci0: usb@01c1a000 { + compatible = "allwinner,sun8i-a83t-ehci", "generic-ehci"; + reg = <0x01c1a000 0x100>; + interrupts = ; + clocks = <&bus_gates 24>; + resets = <&ahb_rst 24>; + phys = <&usbphy 1>; + phy-names = "usb"; + status = "disabled"; + }; + + ohci1: usb@01c1b400 { + compatible = "generic-ohci"; + reg = <0x01c1b400 0x100>; + interrupts = ; + clocks = <&bus_gates 29>, <&usb_clk 16>, <&usb_clk 17>; + resets = <&ahb_rst 29>; + phys = <&usbphy 2>; + phy-names = "usb"; + status = "disabled"; + }; + + ehci1: usb@01c1b000 { + compatible = "allwinner,sun8i-a83t-ehci", "generic-ehci"; + reg = <0x01c1b000 0x100>; + interrupts = ; + clocks = <&bus_gates 25>; + resets = <&ahb_rst 25>; + phys = <&usbphy 2>; + phy-names = "usb"; + status = "disabled"; + }; + }; +}; + +&pio { + r_rsb_pins: r_rsb { + allwinner,pins = "PL0", "PL1"; + allwinner,function = "s_rsb"; + allwinner,drive = ; + allwinner,pull = ; + }; +}; Property changes on: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/a64.dtsi ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/pine64_plus.dts =================================================================== --- projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/pine64_plus.dts (nonexistent) +++ projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/pine64_plus.dts (revision 305172) @@ -0,0 +1,89 @@ +/*- + * Copyright (c) 2016 Jared McNeill + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include "sun50i-a64-pine64-plus.dts" +#include "a64.dtsi" + +#include + +&pio { + emac_phy_reset_pin_pine64_plus: emac_phy_reset_pin@0 { + allwinner,pins = "PD14"; + allwinner,function = "gpio_out"; + allwinner,drive = ; + allwinner,pull = ; + }; +}; + +&emac { + pinctrl-names = "default"; + pinctrl-0 = <&rgmii_pins>, <&emac_phy_reset_pin_pine64_plus>; + phy-supply = <®_dc1dc>; + allwinner,reset-gpio = <&pio 3 14 GPIO_ACTIVE_HIGH>; + allwinner,reset-active-low; + allwinner,reset-delays-us = <0 10000 30000>; +}; + +&r_rsb { + status = "okay"; + + axp81x: pmic@3a3 { + compatible = "x-powers,axp813"; + reg = <0x3a3>; + interrupt-parent = <&nmi_intc>; + interrupts = <0 IRQ_TYPE_LEVEL_LOW>; + gpio-controller; + #gpio-cells = <1>; + + regulators { + reg_dc1dc: dc1sw { + regulator-name = "dc1sw"; + }; + }; + }; +}; + +&usbphy { + status = "okay"; +}; + +&ehci0 { + status = "okay"; +}; + +&ohci0 { + status = "okay"; +}; + +&ehci1 { + status = "okay"; +}; + +&ohci1 { + status = "okay"; +}; Property changes on: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/pine64_plus.dts ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64-common.dtsi =================================================================== --- projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64-common.dtsi (nonexistent) +++ projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64-common.dtsi (revision 305172) @@ -0,0 +1,82 @@ +/* + * Copyright (c) 2016 ARM Ltd. + * + * This file is dual-licensed: you can use it either under the terms + * of the GPL or the X11 license, at your option. Note that this dual + * licensing only applies to this file, and not this project as a + * whole. + * + * a) This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of the + * License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * Or, alternatively, + * + * b) Permission is hereby granted, free of charge, to any person + * obtaining a copy of this software and associated documentation + * files (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following + * conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * $FreeBSD$ + */ + +#include "sun50i-a64.dtsi" + +/ { + + aliases { + serial0 = &uart0; + }; + + soc { + reg_vcc3v3: vcc3v3 { + compatible = "regulator-fixed"; + regulator-name = "vcc3v3"; + regulator-min-microvolt = <3300000>; + regulator-max-microvolt = <3300000>; + }; + }; +}; + +&mmc0 { + pinctrl-names = "default"; + pinctrl-0 = <&mmc0_pins>, <&mmc0_default_cd_pin>; + vmmc-supply = <®_vcc3v3>; + cd-gpios = <&pio 5 6 0>; + cd-inverted; + status = "okay"; +}; + +&uart0 { + pinctrl-names = "default"; + pinctrl-0 = <&uart0_pins_a>; + status = "okay"; +}; + +&i2c1 { + pinctrl-names = "default"; + pinctrl-0 = <&i2c1_pins>; + status = "okay"; +}; Property changes on: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64-common.dtsi ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64-plus.dts =================================================================== --- projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64-plus.dts (nonexistent) +++ projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64-plus.dts (revision 305172) @@ -0,0 +1,73 @@ +/* + * Copyright (c) 2016 ARM Ltd. + * + * This file is dual-licensed: you can use it either under the terms + * of the GPL or the X11 license, at your option. Note that this dual + * licensing only applies to this file, and not this project as a + * whole. + * + * a) This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of the + * License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * Or, alternatively, + * + * b) Permission is hereby granted, free of charge, to any person + * obtaining a copy of this software and associated documentation + * files (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following + * conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * $FreeBSD$ + */ + +/dts-v1/; + +#include "sun50i-a64-pine64-common.dtsi" + +/ { + model = "Pine64+"; + compatible = "pine64,pine64-plus", "allwinner,sun50i-a64"; + + chosen { + stdout-path = "serial0:115200n8"; + }; + + /* There is a model with 2GB of DRAM, but U-Boot fixes this for us. */ + memory { + reg = <0x40000000 0x40000000>; + }; +}; + +&emac { + pinctrl-names = "default"; + pinctrl-0 = <&rgmii_pins>; + phy-mode = "rgmii"; + phy = <&phy1>; + status = "okay"; + + phy1: ethernet-phy@1 { + reg = <1>; + }; +}; Property changes on: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64-plus.dts ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64.dts =================================================================== --- projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64.dts (nonexistent) +++ projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64.dts (revision 305172) @@ -0,0 +1,72 @@ +/* + * Copyright (c) 2016 ARM Ltd. + * + * This file is dual-licensed: you can use it either under the terms + * of the GPL or the X11 license, at your option. Note that this dual + * licensing only applies to this file, and not this project as a + * whole. + * + * a) This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of the + * License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * Or, alternatively, + * + * b) Permission is hereby granted, free of charge, to any person + * obtaining a copy of this software and associated documentation + * files (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following + * conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * $FreeBSD$ + */ + +/dts-v1/; + +#include "sun50i-a64-pine64-common.dtsi" + +/ { + model = "Pine64"; + compatible = "pine64,pine64", "allwinner,sun50i-a64"; + + chosen { + stdout-path = "serial0:115200n8"; + }; + + memory { + reg = <0x40000000 0x20000000>; + }; +}; + +&emac { + pinctrl-names = "default"; + pinctrl-0 = <&rmii_pins>; + phy-mode = "rmii"; + phy = <&phy1>; + status = "okay"; + + phy1: ethernet-phy@1 { + reg = <1>; + }; +}; Property changes on: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64-pine64.dts ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64.dtsi =================================================================== --- projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64.dtsi (nonexistent) +++ projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64.dtsi (revision 305172) @@ -0,0 +1,661 @@ +/* + * Copyright (C) 2016 ARM Ltd. + * based on the Allwinner H3 dtsi: + * Copyright (C) 2015 Jens Kuske + * + * This file is dual-licensed: you can use it either under the terms + * of the GPL or the X11 license, at your option. Note that this dual + * licensing only applies to this file, and not this project as a + * whole. + * + * a) This file is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of the + * License, or (at your option) any later version. + * + * This file is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * Or, alternatively, + * + * b) Permission is hereby granted, free of charge, to any person + * obtaining a copy of this software and associated documentation + * files (the "Software"), to deal in the Software without + * restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or + * sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following + * conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES + * OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT + * HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * $FreeBSD$ + */ + +#include +#include + +/ { + interrupt-parent = <&gic>; + #address-cells = <1>; + #size-cells = <1>; + + cpus { + #address-cells = <1>; + #size-cells = <0>; + + cpu@0 { + compatible = "arm,cortex-a53", "arm,armv8"; + device_type = "cpu"; + reg = <0>; + enable-method = "psci"; + }; + + cpu@1 { + compatible = "arm,cortex-a53", "arm,armv8"; + device_type = "cpu"; + reg = <1>; + enable-method = "psci"; + }; + + cpu@2 { + compatible = "arm,cortex-a53", "arm,armv8"; + device_type = "cpu"; + reg = <2>; + enable-method = "psci"; + }; + + cpu@3 { + compatible = "arm,cortex-a53", "arm,armv8"; + device_type = "cpu"; + reg = <3>; + enable-method = "psci"; + }; + }; + + psci { + compatible = "arm,psci-0.2"; + method = "smc"; + }; + + memory { + device_type = "memory"; + reg = <0x40000000 0>; + }; + + gic: interrupt-controller@1c81000 { + compatible = "arm,gic-400"; + interrupt-controller; + #interrupt-cells = <3>; + #address-cells = <0>; + + reg = <0x01c81000 0x1000>, + <0x01c82000 0x2000>, + <0x01c84000 0x2000>, + <0x01c86000 0x2000>; + interrupts = ; + }; + + timer { + compatible = "arm,armv8-timer"; + interrupts = , + , + , + ; + }; + + clocks { + #address-cells = <1>; + #size-cells = <1>; + ranges; + + osc24M: osc24M_clk { + #clock-cells = <0>; + compatible = "fixed-clock"; + clock-frequency = <24000000>; + clock-output-names = "osc24M"; + }; + + osc32k: osc32k_clk { + #clock-cells = <0>; + compatible = "fixed-clock"; + clock-frequency = <32768>; + clock-output-names = "osc32k"; + }; + + cpux: clk@1c20000 { + #clock-cells = <0>; + compatible = "allwinner,sun8i-a23-pll1-clk"; + reg = <0x01c20000 0x4>; + clocks = <&osc24M>; + clock-output-names = "cpux"; + }; + + periph0: clk@1c20028 { + #clock-cells = <1>; + compatible = "allwinner,sun6i-a31-pll6-clk"; + reg = <0x01c20028 0x4>; + clocks = <&osc24M>; + clock-output-names = "periph0", "periph0x2"; + }; + + periph0d2: periph0d2_clk { + #clock-cells = <0>; + compatible = "fixed-factor-clock"; + clock-div = <2>; + clock-mult = <1>; + clocks = <&periph0 0>; + clock-output-names = "periph0d2"; + }; + + periph1: clk@1c2002c { + #clock-cells = <1>; + compatible = "allwinner,sun6i-a31-pll6-clk"; + reg = <0x01c2002c 0x4>; + clocks = <&osc24M>; + clock-output-names = "periph1", "periph1x2"; + }; + + cpu: cpu_clk@1c20050 { + #clock-cells = <0>; + compatible = "allwinner,sun4i-a10-cpu-clk"; + reg = <0x01c20050 0x4>; + clocks = <&osc32k>, <&osc24M>, <&cpux>, <&cpux>; + clock-output-names = "cpu"; + critical-clocks = <0>; + }; + + axi: axi_clk@1c20050 { + #clock-cells = <0>; + compatible = "allwinner,sun4i-a10-axi-clk"; + reg = <0x01c20050 0x4>; + clocks = <&cpu>; + clock-output-names = "axi"; + }; + + ahb1: ahb1_clk@1c20054 { + #clock-cells = <0>; + compatible = "allwinner,sun6i-a31-ahb1-clk"; + reg = <0x01c20054 0x4>; + clocks = <&osc32k>, <&osc24M>, <&axi>, <&periph0 0>; + clock-output-names = "ahb1"; + }; + + ahb2: ahb2_clk@1c2005c { + #clock-cells = <0>; + compatible = "allwinner,sun8i-h3-ahb2-clk"; + reg = <0x01c2005c 0x4>; + clocks = <&ahb1>, <&periph0d2>; + clock-output-names = "ahb2"; + }; + + apb1: apb1_clk@1c20054 { + #clock-cells = <0>; + compatible = "allwinner,sun4i-a10-apb0-clk"; + reg = <0x01c20054 0x4>; + clocks = <&ahb1>; + clock-output-names = "apb1"; + }; + + apb2: apb2_clk@1c20058 { + #clock-cells = <0>; + compatible = "allwinner,sun4i-a10-apb1-clk"; + reg = <0x01c20058 0x4>; + clocks = <&osc32k>, <&osc24M>, + <&periph0 1>, <&periph0 1>; + clock-output-names = "apb2"; + }; + + bus_gates: bus_gates_clk@1c20060 { + #clock-cells = <1>; + compatible = "allwinner,sun50i-a64-bus-gates-clk", + "allwinner,sunxi-multi-bus-gates-clk"; + reg = <0x01c20060 0x14>; + ahb1_parent { + clocks = <&ahb1>; + clock-indices = <1>, <5>, + <6>, <8>, + <9>, <10>, + <13>, <14>, + <18>, <19>, + <20>, <21>, + <23>, <24>, + <25>, <28>, + <32>, <35>, + <36>, <37>, + <40>, <43>, + <44>, <52>, + <53>, <54>, + <135>; + clock-output-names = "bus_mipidsi", "bus_ce", + "bus_dma", "bus_mmc0", + "bus_mmc1", "bus_mmc2", + "bus_nand", "bus_sdram", + "bus_ts", "bus_hstimer", + "bus_spi0", "bus_spi1", + "bus_otg", "bus_otg_ehci0", + "bus_ehci0", "bus_otg_ohci0", + "bus_ve", "bus_lcd0", + "bus_lcd1", "bus_deint", + "bus_csi", "bus_hdmi", + "bus_de", "bus_gpu", + "bus_msgbox", "bus_spinlock", + "bus_dbg"; + }; + ahb2_parent { + clocks = <&ahb2>; + clock-indices = <17>, <29>; + clock-output-names = "bus_gmac", "bus_ohci0"; + }; + apb1_parent { + clocks = <&apb1>; + clock-indices = <64>, <65>, + <69>, <72>, + <76>, <77>, + <78>; + clock-output-names = "bus_codec", "bus_spdif", + "bus_pio", "bus_ths", + "bus_i2s0", "bus_i2s1", + "bus_i2s2"; + }; + abp2_parent { + clocks = <&apb2>; + clock-indices = <96>, <97>, + <98>, <101>, + <112>, <113>, + <114>, <115>, + <116>; + clock-output-names = "bus_i2c0", "bus_i2c1", + "bus_i2c2", "bus_scr", + "bus_uart0", "bus_uart1", + "bus_uart2", "bus_uart3", + "bus_uart4"; + }; + }; + + mmc0_clk: mmc0_clk@1c20088 { + #clock-cells = <0>; + compatible = "allwinner,sun4i-a10-mod0-clk"; + reg = <0x01c20088 0x4>; + clocks = <&osc24M>, <&periph0 1>, <&periph1 1>; + clock-output-names = "mmc0"; + }; + + mmc1_clk: mmc1_clk@1c2008c { + #clock-cells = <0>; + compatible = "allwinner,sun4i-a10-mod0-clk"; + reg = <0x01c2008c 0x4>; + clocks = <&osc24M>, <&periph0 1>, <&periph1 1>; + clock-output-names = "mmc1"; + }; + + mmc2_clk: mmc2_clk@1c20090 { + #clock-cells = <0>; + compatible = "allwinner,sun4i-a10-mod0-clk"; + reg = <0x01c20090 0x4>; + clocks = <&osc24M>, <&periph0 1>, <&periph1 1>; + clock-output-names = "mmc2"; + }; + }; + + soc { + compatible = "simple-bus"; + #address-cells = <1>; + #size-cells = <1>; + ranges; + + mmc0: mmc@1c0f000 { + compatible = "allwinner,sun50i-a64-mmc", + "allwinner,sun5i-a13-mmc"; + reg = <0x01c0f000 0x1000>; + clocks = <&bus_gates 8>, <&mmc0_clk>, + <&mmc0_clk>, <&mmc0_clk>; + clock-names = "ahb", "mmc", + "output", "sample"; + resets = <&ahb_rst 8>; + reset-names = "ahb"; + interrupts = ; + status = "disabled"; + #address-cells = <1>; + #size-cells = <0>; + }; + + mmc1: mmc@1c10000 { + compatible = "allwinner,sun50i-a64-mmc", + "allwinner,sun5i-a13-mmc"; + reg = <0x01c10000 0x1000>; + clocks = <&bus_gates 9>, <&mmc1_clk>, + <&mmc1_clk>, <&mmc1_clk>; + clock-names = "ahb", "mmc", + "output", "sample"; + resets = <&ahb_rst 9>; + reset-names = "ahb"; + interrupts = ; + status = "disabled"; + #address-cells = <1>; + #size-cells = <0>; + }; + + mmc2: mmc@1c11000 { + compatible = "allwinner,sun50i-a64-mmc", + "allwinner,sun5i-a13-mmc"; + reg = <0x01c11000 0x1000>; + clocks = <&bus_gates 10>, <&mmc2_clk>, + <&mmc2_clk>, <&mmc2_clk>; + clock-names = "ahb", "mmc", + "output", "sample"; + resets = <&ahb_rst 10>; + reset-names = "ahb"; + interrupts = ; + status = "disabled"; + #address-cells = <1>; + #size-cells = <0>; + }; + + pio: pinctrl@1c20800 { + compatible = "allwinner,sun50i-a64-pinctrl"; + reg = <0x01c20800 0x400>; + interrupts = , + , + ; + clocks = <&bus_gates 69>; + gpio-controller; + #gpio-cells = <3>; + interrupt-controller; + #interrupt-cells = <2>; + + uart0_pins_a: uart0@0 { + allwinner,pins = "PB8", "PB9"; + allwinner,function = "uart0"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart0_pins_b: uart0@1 { + allwinner,pins = "PF2", "PF3"; + allwinner,function = "uart0"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart1_2pins: uart1_2@0 { + allwinner,pins = "PG6", "PG7"; + allwinner,function = "uart1"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart1_4pins: uart1_4@0 { + allwinner,pins = "PG6", "PG7", "PG8", "PG9"; + allwinner,function = "uart1"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart2_2pins: uart2_2@0 { + allwinner,pins = "PB0", "PB1"; + allwinner,function = "uart2"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart2_4pins: uart2_4@0 { + allwinner,pins = "PB0", "PB1", "PB2", "PB3"; + allwinner,function = "uart2"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart3_pins_a: uart3@0 { + allwinner,pins = "PD0", "PD1"; + allwinner,function = "uart3"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart3_2pins_b: uart3_2@1 { + allwinner,pins = "PH4", "PH5"; + allwinner,function = "uart3"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart3_4pins_b: uart3_4@1 { + allwinner,pins = "PH4", "PH5", "PH6", "PH7"; + allwinner,function = "uart3"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart4_2pins: uart4_2@0 { + allwinner,pins = "PD2", "PD3"; + allwinner,function = "uart4"; + allwinner,drive = ; + allwinner,pull = ; + }; + + uart4_4pins: uart4_4@0 { + allwinner,pins = "PD2", "PD3", "PD4", "PD5"; + allwinner,function = "uart4"; + allwinner,drive = ; + allwinner,pull = ; + }; + + mmc0_pins: mmc0@0 { + allwinner,pins = "PF0", "PF1", "PF2", "PF3", + "PF4", "PF5"; + allwinner,function = "mmc0"; + allwinner,drive = ; + allwinner,pull = ; + }; + + mmc0_default_cd_pin: mmc0_cd_pin@0 { + allwinner,pins = "PF6"; + allwinner,function = "gpio_in"; + allwinner,drive = ; + allwinner,pull = ; + }; + + mmc1_pins: mmc1@0 { + allwinner,pins = "PG0", "PG1", "PG2", "PG3", + "PG4", "PG5"; + allwinner,function = "mmc1"; + allwinner,drive = ; + allwinner,pull = ; + }; + + mmc2_pins: mmc2@0 { + allwinner,pins = "PC1", "PC5", "PC6", "PC8", + "PC9", "PC10"; + allwinner,function = "mmc2"; + allwinner,drive = ; + allwinner,pull = ; + }; + + i2c0_pins: i2c0_pins { + allwinner,pins = "PH0", "PH1"; + allwinner,function = "i2c0"; + allwinner,drive = ; + allwinner,pull = ; + }; + + i2c1_pins: i2c1_pins { + allwinner,pins = "PH2", "PH3"; + allwinner,function = "i2c1"; + allwinner,drive = ; + allwinner,pull = ; + }; + + i2c2_pins: i2c2_pins { + allwinner,pins = "PE14", "PE15"; + allwinner,function = "i2c2"; + allwinner,drive = ; + allwinner,pull = ; + }; + + rmii_pins: rmii_pins { + allwinner,pins = "PD10", "PD11", "PD13", "PD14", + "PD17", "PD18", "PD19", "PD20", + "PD22", "PD23"; + allwinner,function = "emac"; + allwinner,drive = ; + allwinner,pull = ; + }; + + rgmii_pins: rgmii_pins { + allwinner,pins = "PD8", "PD9", "PD10", "PD11", + "PD12", "PD13", "PD15", + "PD16", "PD17", "PD18", "PD19", + "PD20", "PD21", "PD22", "PD23"; + allwinner,function = "emac"; + allwinner,drive = ; + allwinner,pull = ; + }; + }; + + ahb_rst: reset@1c202c0 { + #reset-cells = <1>; + compatible = "allwinner,sun6i-a31-clock-reset"; + reg = <0x01c202c0 0xc>; + }; + + apb1_rst: reset@1c202d0 { + #reset-cells = <1>; + compatible = "allwinner,sun6i-a31-clock-reset"; + reg = <0x01c202d0 0x4>; + }; + + apb2_rst: reset@1c202d8 { + #reset-cells = <1>; + compatible = "allwinner,sun6i-a31-clock-reset"; + reg = <0x01c202d8 0x4>; + }; + + uart0: serial@1c28000 { + compatible = "snps,dw-apb-uart"; + reg = <0x01c28000 0x400>; + interrupts = ; + reg-shift = <2>; + reg-io-width = <4>; + clocks = <&bus_gates 112>; + resets = <&apb2_rst 16>; + status = "disabled"; + }; + + uart1: serial@1c28400 { + compatible = "snps,dw-apb-uart"; + reg = <0x01c28400 0x400>; + interrupts = ; + reg-shift = <2>; + reg-io-width = <4>; + clocks = <&bus_gates 113>; + resets = <&apb2_rst 17>; + status = "disabled"; + }; + + uart2: serial@1c28800 { + compatible = "snps,dw-apb-uart"; + reg = <0x01c28800 0x400>; + interrupts = ; + reg-shift = <2>; + reg-io-width = <4>; + clocks = <&bus_gates 114>; + resets = <&apb2_rst 18>; + status = "disabled"; + }; + + uart3: serial@1c28c00 { + compatible = "snps,dw-apb-uart"; + reg = <0x01c28c00 0x400>; + interrupts = ; + reg-shift = <2>; + reg-io-width = <4>; + clocks = <&bus_gates 115>; + resets = <&apb2_rst 19>; + status = "disabled"; + }; + + uart4: serial@1c29000 { + compatible = "snps,dw-apb-uart"; + reg = <0x01c29000 0x400>; + interrupts = ; + reg-shift = <2>; + reg-io-width = <4>; + clocks = <&bus_gates 116>; + resets = <&apb2_rst 20>; + status = "disabled"; + }; + + rtc: rtc@1f00000 { + compatible = "allwinner,sun6i-a31-rtc"; + reg = <0x01f00000 0x54>; + interrupts = , + ; + }; + + i2c0: i2c@1c2ac00 { + compatible = "allwinner,sun6i-a31-i2c"; + reg = <0x01c2ac00 0x400>; + interrupts = ; + clocks = <&bus_gates 96>; + resets = <&apb2_rst 0>; + status = "disabled"; + #address-cells = <1>; + #size-cells = <0>; + }; + + i2c1: i2c@1c2b000 { + compatible = "allwinner,sun6i-a31-i2c"; + reg = <0x01c2b000 0x400>; + interrupts = ; + clocks = <&bus_gates 97>; + resets = <&apb2_rst 1>; + status = "disabled"; + #address-cells = <1>; + #size-cells = <0>; + }; + + i2c2: i2c@1c2b400 { + compatible = "allwinner,sun6i-a31-i2c"; + reg = <0x01c2b400 0x400>; + interrupts = ; + clocks = <&bus_gates 98>; + resets = <&apb2_rst 2>; + status = "disabled"; + #address-cells = <1>; + #size-cells = <0>; + }; + + emac: ethernet@1c30000 { + compatible = "allwinner,sun50i-a64-emac", + "allwinner,sun8i-h3-emac"; + reg = <0x01c30000 0x100>, <0x01c00030 0x4>; + reg-names = "emac", "syscon"; + interrupts = ; + resets = <&ahb_rst 17>; + reset-names = "ahb"; + clocks = <&bus_gates 17>; + clock-names = "ahb"; + status = "disabled"; + #address-cells = <1>; + #size-cells = <0>; + }; + }; +}; Property changes on: projects/netbsd-tests-update-12/sys/boot/fdt/dts/arm64/sun50i-a64.dtsi ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/sys/boot/forth/loader.4th =================================================================== --- projects/netbsd-tests-update-12/sys/boot/forth/loader.4th (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/forth/loader.4th (revision 305172) @@ -1,259 +1,263 @@ \ Copyright (c) 1999 Daniel C. Sobral \ Copyright (c) 2011-2015 Devin Teske \ All rights reserved. \ \ Redistribution and use in source and binary forms, with or without \ modification, are permitted provided that the following conditions \ are met: \ 1. Redistributions of source code must retain the above copyright \ notice, this list of conditions and the following disclaimer. \ 2. Redistributions in binary form must reproduce the above copyright \ notice, this list of conditions and the following disclaimer in the \ documentation and/or other materials provided with the distribution. \ \ THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND \ ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE \ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE \ ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE \ FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL \ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS \ OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) \ HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT \ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY \ OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF \ SUCH DAMAGE. \ \ $FreeBSD$ only forth definitions s" arch-i386" environment? [if] [if] s" loader_version" environment? [if] 11 < [if] .( Loader version 1.1+ required) cr abort [then] [else] .( Could not get loader version!) cr abort [then] [then] [then] 256 dictthreshold ! \ 256 cells minimum free space 2048 dictincrease ! \ 2048 additional cells each time include /boot/support.4th include /boot/color.4th include /boot/delay.4th include /boot/check-password.4th only forth definitions : bootmsg ( -- ) loader_color? dup ( -- bool bool ) if 7 fg 4 bg then ." Booting..." if me then cr ; : try-menu-unset \ menu-unset may not be present s" beastie_disable" getenv dup -1 <> if s" YES" compare-insensitive 0= if exit then else drop then s" menu-unset" sfind if execute else drop then s" menusets-unset" sfind if execute else drop then ; only forth also support-functions also builtins definitions : boot 0= if ( interpreted ) get_arguments then \ Unload only if a path was passed dup if >r over r> swap c@ [char] - <> if 0 1 unload drop else s" kernelname" getenv? if ( a kernel has been loaded ) try-menu-unset bootmsg 1 boot exit then load_kernel_and_modules ?dup if exit then try-menu-unset bootmsg 0 1 boot exit then else s" kernelname" getenv? if ( a kernel has been loaded ) try-menu-unset bootmsg 1 boot exit then load_kernel_and_modules ?dup if exit then try-menu-unset bootmsg 0 1 boot exit then load_kernel_and_modules ?dup 0= if bootmsg 0 1 boot then ; \ ***** boot-conf \ \ Prepares to boot as specified by loaded configuration files. : boot-conf 0= if ( interpreted ) get_arguments then 0 1 unload drop load_kernel_and_modules ?dup 0= if 0 1 autoboot then ; also forth definitions previous builtin: boot builtin: boot-conf only forth definitions also support-functions \ ***** start \ \ Initializes support.4th global variables, sets loader_conf_files, \ processes conf files, and, if any one such file was successfully \ read to the end, loads kernel and modules. : start ( -- ) ( throws: abort & user-defined ) s" /boot/defaults/loader.conf" initialize include_conf_files include_nextboot_file + \ If the user defined a post-initialize hook, call it now + s" post-initialize" sfind if execute else drop then \ Will *NOT* try to load kernel and modules if no configuration file \ was successfully loaded! any_conf_read? if s" loader_delay" getenv -1 = if load_xen_throw load_kernel load_modules else drop ." Loading Kernel and Modules (Ctrl-C to Abort)" cr s" also support-functions" evaluate s" set delay_command='load_xen_throw load_kernel load_modules'" evaluate s" set delay_showdots" evaluate delay_execute then then ; \ ***** initialize \ \ Overrides support.4th initialization word with one that does \ everything start one does, short of loading the kernel and -\ modules. Returns a flag +\ modules. Returns a flag. : initialize ( -- flag ) s" /boot/defaults/loader.conf" initialize include_conf_files include_nextboot_file + \ If the user defined a post-initialize hook, call it now + s" post-initialize" sfind if execute else drop then any_conf_read? ; \ ***** read-conf \ \ Read a configuration file, whose name was specified on the command \ line, if interpreted, or given on the stack, if compiled in. : (read-conf) ( addr len -- ) conf_files string= include_conf_files \ Will recurse on new loader_conf_files definitions ; : read-conf ( | addr len -- ) ( throws: abort & user-defined ) state @ if \ Compiling postpone (read-conf) else \ Interpreting bl parse (read-conf) then ; immediate \ show, enable, disable, toggle module loading. They all take module from \ the next word : set-module-flag ( module_addr val -- ) \ set and print flag over module.flag ! dup module.name strtype module.flag @ if ." will be loaded" else ." will not be loaded" then cr ; : enable-module find-module ?dup if true set-module-flag then ; : disable-module find-module ?dup if false set-module-flag then ; : toggle-module find-module ?dup if dup module.flag @ 0= set-module-flag then ; \ ***** show-module \ \ Show loading information about a module. : show-module ( -- ) find-module ?dup if show-one-module then ; \ Words to be used inside configuration files : retry false ; \ For use in load error commands : ignore true ; \ For use in load error commands \ Return to strict forth vocabulary : #type over - >r type r> spaces ; : .? 2 spaces 2swap 15 #type 2 spaces type cr ; \ Execute the ? command to print all the commands defined in \ C, then list the ones we support here. Please note that this \ doesn't use pager_* routines that the C implementation of ? \ does, so these will always appear, even if you stop early \ there. And they may cause the commands to scroll off the \ screen if the number of commands modulus LINES is close \ to LINEs.... : ? ['] ? execute s" boot-conf" s" load kernel and modules, then autoboot" .? s" read-conf" s" read a configuration file" .? s" enable-module" s" enable loading of a module" .? s" disable-module" s" disable loading of a module" .? s" toggle-module" s" toggle loading of a module" .? s" show-module" s" show module load data" .? s" try-include" s" try to load/interpret files" .? ; : try-include ( -- ) \ see loader.4th(8) ['] include ( -- xt ) \ get the execution token of `include' catch ( xt -- exception# | 0 ) if \ failed LF parse ( c -- s-addr/u ) 2drop \ advance >in to EOL (drop data) \ ... prevents words unused by `include' from being interpreted then ; immediate \ interpret immediately for access to `source' (aka tib) only forth definitions Index: projects/netbsd-tests-update-12/sys/boot/i386/libi386/pxe.c =================================================================== --- projects/netbsd-tests-update-12/sys/boot/i386/libi386/pxe.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/i386/libi386/pxe.c (revision 305172) @@ -1,714 +1,717 @@ /*- * Copyright (c) 2000 Alfred Perlstein * Copyright (c) 2000 Paul Saab * Copyright (c) 2000 John Baldwin * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include "btxv86.h" #include "pxe.h" /* * Allocate the PXE buffers statically instead of sticking grimy fingers into * BTX's private data area. The scratch buffer is used to send information to * the PXE BIOS, and the data buffer is used to receive data from the PXE BIOS. */ #define PXE_BUFFER_SIZE 0x2000 #define PXE_TFTP_BUFFER_SIZE 512 static char scratch_buffer[PXE_BUFFER_SIZE]; static char data_buffer[PXE_BUFFER_SIZE]; static pxenv_t *pxenv_p = NULL; /* PXENV+ */ static pxe_t *pxe_p = NULL; /* !PXE */ static BOOTPLAYER bootplayer; /* PXE Cached information. */ static int pxe_debug = 0; static int pxe_sock = -1; static int pxe_opens = 0; void pxe_enable(void *pxeinfo); static void (*pxe_call)(int func); static void pxenv_call(int func); static void bangpxe_call(int func); static int pxe_init(void); static int pxe_strategy(void *devdata, int flag, daddr_t dblk, size_t offset, size_t size, char *buf, size_t *rsize); static int pxe_open(struct open_file *f, ...); static int pxe_close(struct open_file *f); static void pxe_print(int verbose); static void pxe_cleanup(void); static void pxe_setnfshandle(char *rootpath); static void pxe_perror(int error); static int pxe_netif_match(struct netif *nif, void *machdep_hint); static int pxe_netif_probe(struct netif *nif, void *machdep_hint); static void pxe_netif_init(struct iodesc *desc, void *machdep_hint); static int pxe_netif_get(struct iodesc *desc, void *pkt, size_t len, time_t timeout); static int pxe_netif_put(struct iodesc *desc, void *pkt, size_t len); static void pxe_netif_end(struct netif *nif); #ifdef OLD_NFSV2 int nfs_getrootfh(struct iodesc*, char*, u_char*); #else int nfs_getrootfh(struct iodesc*, char*, uint32_t*, u_char*); #endif extern struct netif_stats pxe_st[]; extern u_int16_t __bangpxeseg; extern u_int16_t __bangpxeoff; extern void __bangpxeentry(void); extern u_int16_t __pxenvseg; extern u_int16_t __pxenvoff; extern void __pxenventry(void); struct netif_dif pxe_ifs[] = { /* dif_unit dif_nsel dif_stats dif_private */ {0, 1, &pxe_st[0], 0} }; struct netif_stats pxe_st[NENTS(pxe_ifs)]; struct netif_driver pxenetif = { "pxenet", pxe_netif_match, pxe_netif_probe, pxe_netif_init, pxe_netif_get, pxe_netif_put, pxe_netif_end, pxe_ifs, NENTS(pxe_ifs) }; struct netif_driver *netif_drivers[] = { &pxenetif, NULL }; struct devsw pxedisk = { "pxe", DEVT_NET, pxe_init, pxe_strategy, pxe_open, pxe_close, noioctl, pxe_print, pxe_cleanup }; /* * This function is called by the loader to enable PXE support if we * are booted by PXE. The passed in pointer is a pointer to the * PXENV+ structure. */ void pxe_enable(void *pxeinfo) { pxenv_p = (pxenv_t *)pxeinfo; pxe_p = (pxe_t *)PTOV(pxenv_p->PXEPtr.segment * 16 + pxenv_p->PXEPtr.offset); pxe_call = NULL; } /* * return true if pxe structures are found/initialized, * also figures out our IP information via the pxe cached info struct */ static int pxe_init(void) { t_PXENV_GET_CACHED_INFO *gci_p; int counter; uint8_t checksum; uint8_t *checkptr; if(pxenv_p == NULL) return (0); /* look for "PXENV+" */ if (bcmp((void *)pxenv_p->Signature, S_SIZE("PXENV+"))) { pxenv_p = NULL; return (0); } /* make sure the size is something we can handle */ if (pxenv_p->Length > sizeof(*pxenv_p)) { printf("PXENV+ structure too large, ignoring\n"); pxenv_p = NULL; return (0); } /* * do byte checksum: * add up each byte in the structure, the total should be 0 */ checksum = 0; checkptr = (uint8_t *) pxenv_p; for (counter = 0; counter < pxenv_p->Length; counter++) checksum += *checkptr++; if (checksum != 0) { printf("PXENV+ structure failed checksum, ignoring\n"); pxenv_p = NULL; return (0); } /* * PXENV+ passed, so use that if !PXE is not available or * the checksum fails. */ pxe_call = pxenv_call; if (pxenv_p->Version >= 0x0200) { for (;;) { if (bcmp((void *)pxe_p->Signature, S_SIZE("!PXE"))) { pxe_p = NULL; break; } checksum = 0; checkptr = (uint8_t *)pxe_p; for (counter = 0; counter < pxe_p->StructLength; counter++) checksum += *checkptr++; if (checksum != 0) { pxe_p = NULL; break; } pxe_call = bangpxe_call; break; } } printf("\nPXE version %d.%d, real mode entry point ", (uint8_t) (pxenv_p->Version >> 8), (uint8_t) (pxenv_p->Version & 0xFF)); if (pxe_call == bangpxe_call) printf("@%04x:%04x\n", pxe_p->EntryPointSP.segment, pxe_p->EntryPointSP.offset); else printf("@%04x:%04x\n", pxenv_p->RMEntry.segment, pxenv_p->RMEntry.offset); gci_p = (t_PXENV_GET_CACHED_INFO *) scratch_buffer; bzero(gci_p, sizeof(*gci_p)); gci_p->PacketType = PXENV_PACKET_TYPE_BINL_REPLY; pxe_call(PXENV_GET_CACHED_INFO); if (gci_p->Status != 0) { pxe_perror(gci_p->Status); pxe_p = NULL; return (0); } bcopy(PTOV((gci_p->Buffer.segment << 4) + gci_p->Buffer.offset), &bootplayer, gci_p->BufferSize); return (1); } static int pxe_strategy(void *devdata, int flag, daddr_t dblk, size_t offset, size_t size, char *buf, size_t *rsize) { return (EIO); } static int pxe_open(struct open_file *f, ...) { va_list args; char *devname; /* Device part of file name (or NULL). */ char temp[FNAME_SIZE]; int error = 0; int i; va_start(args, f); devname = va_arg(args, char*); va_end(args); /* On first open, do netif open, mount, etc. */ if (pxe_opens == 0) { /* Find network interface. */ if (pxe_sock < 0) { pxe_sock = netif_open(devname); if (pxe_sock < 0) { printf("pxe_open: netif_open() failed\n"); return (ENXIO); } if (pxe_debug) printf("pxe_open: netif_open() succeeded\n"); } if (rootip.s_addr == 0) { /* * Do a bootp/dhcp request to find out where our * NFS/TFTP server is. Even if we dont get back * the proper information, fall back to the server * which brought us to life and a default rootpath. */ bootp(pxe_sock, BOOTP_PXE); if (rootip.s_addr == 0) rootip.s_addr = bootplayer.sip; -#ifdef LOADER_NFS_SUPPORT - if (!rootpath[0]) + + netproto = NET_NFS; + if (tftpip.s_addr != 0) { + netproto = NET_TFTP; + rootip.s_addr = tftpip.s_addr; + } + + if (netproto == NET_NFS && !rootpath[0]) strcpy(rootpath, PXENFSROOTPATH); -#endif for (i = 0; rootpath[i] != '\0' && i < FNAME_SIZE; i++) if (rootpath[i] == ':') break; if (i && i != FNAME_SIZE && rootpath[i] == ':') { rootpath[i++] = '\0'; if (inet_addr(&rootpath[0]) != INADDR_NONE) rootip.s_addr = inet_addr(&rootpath[0]); bcopy(&rootpath[i], &temp[0], strlen(&rootpath[i])+1); bcopy(&temp[0], &rootpath[0], strlen(&rootpath[i])+1); } setenv("boot.netif.ip", inet_ntoa(myip), 1); setenv("boot.netif.netmask", intoa(netmask), 1); setenv("boot.netif.gateway", inet_ntoa(gateip), 1); if (bootplayer.Hardware == ETHER_TYPE) { sprintf(temp, "%6D", bootplayer.CAddr, ":"); setenv("boot.netif.hwaddr", temp, 1); } if (intf_mtu != 0) { char mtu[16]; sprintf(mtu, "%u", intf_mtu); setenv("boot.netif.mtu", mtu, 1); } -#ifdef LOADER_NFS_SUPPORT printf("pxe_open: server addr: %s\n", inet_ntoa(rootip)); printf("pxe_open: server path: %s\n", rootpath); printf("pxe_open: gateway ip: %s\n", inet_ntoa(gateip)); - setenv("boot.nfsroot.server", inet_ntoa(rootip), 1); - setenv("boot.nfsroot.path", rootpath, 1); -#else - setenv("boot.netif.server", inet_ntoa(rootip), 1); - setenv("boot.tftproot.path", rootpath, 1); -#endif + if (netproto == NET_NFS) { + setenv("boot.nfsroot.server", inet_ntoa(rootip), 1); + setenv("boot.nfsroot.path", rootpath, 1); + } else if (netproto == NET_TFTP) { + setenv("boot.netif.server", inet_ntoa(rootip), 1); + setenv("boot.tftproot.path", rootpath, 1); + } setenv("dhcp.host-name", hostname, 1); setenv("pxeboot.ip", inet_ntoa(myip), 1); if (bootplayer.Hardware == ETHER_TYPE) { sprintf(temp, "%6D", bootplayer.CAddr, ":"); setenv("pxeboot.hwaddr", temp, 1); } } } pxe_opens++; f->f_devdata = &pxe_sock; return (error); } static int pxe_close(struct open_file *f) { #ifdef PXE_DEBUG if (pxe_debug) printf("pxe_close: opens=%d\n", pxe_opens); #endif /* On last close, do netif close, etc. */ f->f_devdata = NULL; /* Extra close call? */ if (pxe_opens <= 0) return (0); pxe_opens--; /* Not last close? */ if (pxe_opens > 0) return(0); -#ifdef LOADER_NFS_SUPPORT - /* get an NFS filehandle for our root filesystem */ - pxe_setnfshandle(rootpath); -#endif + if (netproto == NET_NFS) { + /* get an NFS filehandle for our root filesystem */ + pxe_setnfshandle(rootpath); + } if (pxe_sock >= 0) { #ifdef PXE_DEBUG if (pxe_debug) printf("pxe_close: calling netif_close()\n"); #endif netif_close(pxe_sock); pxe_sock = -1; } return (0); } static void pxe_print(int verbose) { if (pxe_call == NULL) return; printf(" pxe0: %s:%s\n", inet_ntoa(rootip), rootpath); } static void pxe_cleanup(void) { #ifdef PXE_DEBUG t_PXENV_UNLOAD_STACK *unload_stack_p = (t_PXENV_UNLOAD_STACK *)scratch_buffer; t_PXENV_UNDI_SHUTDOWN *undi_shutdown_p = (t_PXENV_UNDI_SHUTDOWN *)scratch_buffer; #endif if (pxe_call == NULL) return; pxe_call(PXENV_UNDI_SHUTDOWN); #ifdef PXE_DEBUG if (pxe_debug && undi_shutdown_p->Status != 0) printf("pxe_cleanup: UNDI_SHUTDOWN failed %x\n", undi_shutdown_p->Status); #endif pxe_call(PXENV_UNLOAD_STACK); #ifdef PXE_DEBUG if (pxe_debug && unload_stack_p->Status != 0) printf("pxe_cleanup: UNLOAD_STACK failed %x\n", unload_stack_p->Status); #endif } void pxe_perror(int err) { return; } -#ifdef LOADER_NFS_SUPPORT /* * Reach inside the libstand NFS code and dig out an NFS handle * for the root filesystem. */ #ifdef OLD_NFSV2 struct nfs_iodesc { struct iodesc *iodesc; off_t off; u_char fh[NFS_FHSIZE]; /* structure truncated here */ }; extern struct nfs_iodesc nfs_root_node; extern int rpc_port; static void pxe_rpcmountcall() { struct iodesc *d; int error; if (!(d = socktodesc(pxe_sock))) return; d->myport = htons(--rpc_port); d->destip = rootip; if ((error = nfs_getrootfh(d, rootpath, nfs_root_node.fh)) != 0) printf("NFS MOUNT RPC error: %d\n", error); nfs_root_node.iodesc = d; } static void pxe_setnfshandle(char *rootpath) { int i; u_char *fh; char buf[2 * NFS_FHSIZE + 3], *cp; /* * If NFS files were never opened, we need to do mount call * ourselves. Use nfs_root_node.iodesc as flag indicating * previous NFS usage. */ if (nfs_root_node.iodesc == NULL) pxe_rpcmountcall(); fh = &nfs_root_node.fh[0]; buf[0] = 'X'; cp = &buf[1]; for (i = 0; i < NFS_FHSIZE; i++, cp += 2) sprintf(cp, "%02x", fh[i]); sprintf(cp, "X"); setenv("boot.nfsroot.nfshandle", buf, 1); } #else /* !OLD_NFSV2 */ #define NFS_V3MAXFHSIZE 64 struct nfs_iodesc { struct iodesc *iodesc; off_t off; uint32_t fhsize; u_char fh[NFS_V3MAXFHSIZE]; /* structure truncated */ }; extern struct nfs_iodesc nfs_root_node; extern int rpc_port; static void pxe_rpcmountcall() { struct iodesc *d; int error; if (!(d = socktodesc(pxe_sock))) return; d->myport = htons(--rpc_port); d->destip = rootip; if ((error = nfs_getrootfh(d, rootpath, &nfs_root_node.fhsize, nfs_root_node.fh)) != 0) { printf("NFS MOUNT RPC error: %d\n", error); nfs_root_node.fhsize = 0; } nfs_root_node.iodesc = d; } static void pxe_setnfshandle(char *rootpath) { int i; u_char *fh; char buf[2 * NFS_V3MAXFHSIZE + 3], *cp; /* * If NFS files were never opened, we need to do mount call * ourselves. Use nfs_root_node.iodesc as flag indicating * previous NFS usage. */ if (nfs_root_node.iodesc == NULL) pxe_rpcmountcall(); fh = &nfs_root_node.fh[0]; buf[0] = 'X'; cp = &buf[1]; for (i = 0; i < nfs_root_node.fhsize; i++, cp += 2) sprintf(cp, "%02x", fh[i]); sprintf(cp, "X"); setenv("boot.nfsroot.nfshandle", buf, 1); sprintf(buf, "%d", nfs_root_node.fhsize); setenv("boot.nfsroot.nfshandlelen", buf, 1); } #endif /* OLD_NFSV2 */ -#endif /* LOADER_NFS_SUPPORT */ void pxenv_call(int func) { #ifdef PXE_DEBUG if (pxe_debug) printf("pxenv_call %x\n", func); #endif bzero(&v86, sizeof(v86)); bzero(data_buffer, sizeof(data_buffer)); __pxenvseg = pxenv_p->RMEntry.segment; __pxenvoff = pxenv_p->RMEntry.offset; v86.ctl = V86_ADDR | V86_CALLF | V86_FLAGS; v86.es = VTOPSEG(scratch_buffer); v86.edi = VTOPOFF(scratch_buffer); v86.addr = (VTOPSEG(__pxenventry) << 16) | VTOPOFF(__pxenventry); v86.ebx = func; v86int(); v86.ctl = V86_FLAGS; } void bangpxe_call(int func) { #ifdef PXE_DEBUG if (pxe_debug) printf("bangpxe_call %x\n", func); #endif bzero(&v86, sizeof(v86)); bzero(data_buffer, sizeof(data_buffer)); __bangpxeseg = pxe_p->EntryPointSP.segment; __bangpxeoff = pxe_p->EntryPointSP.offset; v86.ctl = V86_ADDR | V86_CALLF | V86_FLAGS; v86.edx = VTOPSEG(scratch_buffer); v86.eax = VTOPOFF(scratch_buffer); v86.addr = (VTOPSEG(__bangpxeentry) << 16) | VTOPOFF(__bangpxeentry); v86.ebx = func; v86int(); v86.ctl = V86_FLAGS; } time_t getsecs() { time_t n = 0; time(&n); return n; } static int pxe_netif_match(struct netif *nif, void *machdep_hint) { return 1; } static int pxe_netif_probe(struct netif *nif, void *machdep_hint) { t_PXENV_UDP_OPEN *udpopen_p = (t_PXENV_UDP_OPEN *)scratch_buffer; if (pxe_call == NULL) return -1; bzero(udpopen_p, sizeof(*udpopen_p)); udpopen_p->src_ip = bootplayer.yip; pxe_call(PXENV_UDP_OPEN); if (udpopen_p->status != 0) { printf("pxe_netif_probe: failed %x\n", udpopen_p->status); return -1; } return 0; } static void pxe_netif_end(struct netif *nif) { t_PXENV_UDP_CLOSE *udpclose_p = (t_PXENV_UDP_CLOSE *)scratch_buffer; bzero(udpclose_p, sizeof(*udpclose_p)); pxe_call(PXENV_UDP_CLOSE); if (udpclose_p->status != 0) printf("pxe_end failed %x\n", udpclose_p->status); } static void pxe_netif_init(struct iodesc *desc, void *machdep_hint) { int i; for (i = 0; i < 6; ++i) desc->myea[i] = bootplayer.CAddr[i]; desc->xid = bootplayer.ident; } static int pxe_netif_get(struct iodesc *desc, void *pkt, size_t len, time_t timeout) { return len; } static int pxe_netif_put(struct iodesc *desc, void *pkt, size_t len) { return len; } ssize_t sendudp(struct iodesc *h, void *pkt, size_t len) { t_PXENV_UDP_WRITE *udpwrite_p = (t_PXENV_UDP_WRITE *)scratch_buffer; bzero(udpwrite_p, sizeof(*udpwrite_p)); udpwrite_p->ip = h->destip.s_addr; udpwrite_p->dst_port = h->destport; udpwrite_p->src_port = h->myport; udpwrite_p->buffer_size = len; udpwrite_p->buffer.segment = VTOPSEG(pkt); udpwrite_p->buffer.offset = VTOPOFF(pkt); if (netmask == 0 || SAMENET(myip, h->destip, netmask)) udpwrite_p->gw = 0; else udpwrite_p->gw = gateip.s_addr; pxe_call(PXENV_UDP_WRITE); #if 0 /* XXX - I dont know why we need this. */ delay(1000); #endif if (udpwrite_p->status != 0) { /* XXX: This happens a lot. It shouldn't. */ if (udpwrite_p->status != 1) printf("sendudp failed %x\n", udpwrite_p->status); return -1; } return len; } ssize_t readudp(struct iodesc *h, void *pkt, size_t len, time_t timeout) { t_PXENV_UDP_READ *udpread_p = (t_PXENV_UDP_READ *)scratch_buffer; struct udphdr *uh = NULL; uh = (struct udphdr *) pkt - 1; bzero(udpread_p, sizeof(*udpread_p)); udpread_p->dest_ip = h->myip.s_addr; udpread_p->d_port = h->myport; udpread_p->buffer_size = len; udpread_p->buffer.segment = VTOPSEG(data_buffer); udpread_p->buffer.offset = VTOPOFF(data_buffer); pxe_call(PXENV_UDP_READ); #if 0 /* XXX - I dont know why we need this. */ delay(1000); #endif if (udpread_p->status != 0) { /* XXX: This happens a lot. It shouldn't. */ if (udpread_p->status != 1) printf("readudp failed %x\n", udpread_p->status); return -1; } bcopy(data_buffer, pkt, udpread_p->buffer_size); uh->uh_sport = udpread_p->s_port; return udpread_p->buffer_size; } Index: projects/netbsd-tests-update-12/sys/boot/i386/loader/conf.c =================================================================== --- projects/netbsd-tests-update-12/sys/boot/i386/loader/conf.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/i386/loader/conf.c (revision 305172) @@ -1,163 +1,159 @@ /*- * Copyright (c) 1998 Michael Smith * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include "libi386/libi386.h" #if defined(LOADER_ZFS_SUPPORT) #include "../zfs/libzfs.h" #endif /* * We could use linker sets for some or all of these, but * then we would have to control what ended up linked into * the bootstrap. So it's easier to conditionalise things * here. * * XXX rename these arrays to be consistent and less namespace-hostile * * XXX as libi386 and biosboot merge, some of these can become linker sets. */ -#if defined(LOADER_NFS_SUPPORT) && defined(LOADER_TFTP_SUPPORT) -#error "Cannot have both tftp and nfs support yet." -#endif - #if defined(LOADER_FIREWIRE_SUPPORT) extern struct devsw fwohci; #endif /* Exported for libstand */ struct devsw *devsw[] = { &bioscd, &biosdisk, #if defined(LOADER_NFS_SUPPORT) || defined(LOADER_TFTP_SUPPORT) &pxedisk, #endif #if defined(LOADER_FIREWIRE_SUPPORT) &fwohci, #endif #if defined(LOADER_ZFS_SUPPORT) &zfs_dev, #endif NULL }; struct fs_ops *file_system[] = { #if defined(LOADER_ZFS_SUPPORT) &zfs_fsops, #endif &ufs_fsops, &ext2fs_fsops, &dosfs_fsops, &cd9660_fsops, #if defined(LOADER_NANDFS_SUPPORT) &nandfs_fsops, #endif #ifdef LOADER_NFS_SUPPORT &nfs_fsops, #endif #ifdef LOADER_TFTP_SUPPORT &tftp_fsops, #endif #ifdef LOADER_GZIP_SUPPORT &gzipfs_fsops, #endif #ifdef LOADER_BZIP2_SUPPORT &bzipfs_fsops, #endif #ifdef LOADER_SPLIT_SUPPORT &splitfs_fsops, #endif NULL }; /* Exported for i386 only */ /* * Sort formats so that those that can detect based on arguments * rather than reading the file go first. */ extern struct file_format i386_elf; extern struct file_format i386_elf_obj; extern struct file_format amd64_elf; extern struct file_format amd64_elf_obj; extern struct file_format multiboot; extern struct file_format multiboot_obj; struct file_format *file_formats[] = { &multiboot, &multiboot_obj, #ifdef LOADER_PREFER_AMD64 &amd64_elf, &amd64_elf_obj, #endif &i386_elf, &i386_elf_obj, #ifndef LOADER_PREFER_AMD64 &amd64_elf, &amd64_elf_obj, #endif NULL }; /* * Consoles * * We don't prototype these in libi386.h because they require * data structures from bootstrap.h as well. */ extern struct console vidconsole; extern struct console comconsole; #if defined(LOADER_FIREWIRE_SUPPORT) extern struct console dconsole; #endif extern struct console nullconsole; extern struct console spinconsole; struct console *consoles[] = { &vidconsole, &comconsole, #if defined(LOADER_FIREWIRE_SUPPORT) &dconsole, #endif &nullconsole, &spinconsole, NULL }; extern struct pnphandler isapnphandler; extern struct pnphandler biospnphandler; extern struct pnphandler biospcihandler; struct pnphandler *pnphandlers[] = { &biospnphandler, /* should go first, as it may set isapnp_readport */ &isapnphandler, &biospcihandler, NULL }; Index: projects/netbsd-tests-update-12/sys/boot/pc98/loader/conf.c =================================================================== --- projects/netbsd-tests-update-12/sys/boot/pc98/loader/conf.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/boot/pc98/loader/conf.c (revision 305172) @@ -1,120 +1,116 @@ /*- * Copyright (c) 1998 Michael Smith * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include "libi386/libi386.h" /* * We could use linker sets for some or all of these, but * then we would have to control what ended up linked into * the bootstrap. So it's easier to conditionalise things * here. * * XXX rename these arrays to be consistent and less namespace-hostile * * XXX as libi386 and biosboot merge, some of these can become linker sets. */ -#if defined(LOADER_NFS_SUPPORT) && defined(LOADER_TFTP_SUPPORT) -#error "Cannot have both tftp and nfs support yet." -#endif - /* Exported for libstand */ struct devsw *devsw[] = { &bioscd, &biosdisk, #if defined(LOADER_NFS_SUPPORT) || defined(LOADER_TFTP_SUPPORT) &pxedisk, #endif NULL }; struct fs_ops *file_system[] = { &ufs_fsops, &ext2fs_fsops, &dosfs_fsops, &cd9660_fsops, #ifdef LOADER_NFS_SUPPORT &nfs_fsops, #endif #ifdef LOADER_TFTP_SUPPORT &tftp_fsops, #endif #ifdef LOADER_GZIP_SUPPORT &gzipfs_fsops, #endif #ifdef LOADER_BZIP2_SUPPORT &bzipfs_fsops, #endif &splitfs_fsops, NULL }; /* Exported for i386 only */ /* * Sort formats so that those that can detect based on arguments * rather than reading the file go first. */ extern struct file_format i386_elf; extern struct file_format i386_elf_obj; struct file_format *file_formats[] = { &i386_elf, &i386_elf_obj, NULL }; /* * Consoles * * We don't prototype these in libi386.h because they require * data structures from bootstrap.h as well. */ extern struct console vidconsole; extern struct console comconsole; extern struct console nullconsole; struct console *consoles[] = { &vidconsole, &comconsole, &nullconsole, NULL }; extern struct pnphandler isapnphandler; extern struct pnphandler biospnphandler; extern struct pnphandler biospcihandler; struct pnphandler *pnphandlers[] = { &biospnphandler, /* should go first, as it may set isapnp_readport */ &isapnphandler, &biospcihandler, NULL }; Index: projects/netbsd-tests-update-12/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_prop.c =================================================================== --- projects/netbsd-tests-update-12/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_prop.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_prop.c (revision 305172) @@ -1,1205 +1,1207 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2012, 2014 by Delphix. All rights reserved. * Copyright 2015, Joyent, Inc. */ #include #include #include #include #include #include #include #include #include #include #include #include "zfs_prop.h" #define ZPROP_INHERIT_SUFFIX "$inherit" #define ZPROP_RECVD_SUFFIX "$recvd" static int dodefault(zfs_prop_t prop, int intsz, int numints, void *buf) { /* * The setonce properties are read-only, BUT they still * have a default value that can be used as the initial * value. */ if (prop == ZPROP_INVAL || (zfs_prop_readonly(prop) && !zfs_prop_setonce(prop))) return (SET_ERROR(ENOENT)); if (zfs_prop_get_type(prop) == PROP_TYPE_STRING) { + if (zfs_prop_default_string(prop) == NULL) + return (SET_ERROR(ENOENT)); if (intsz != 1) return (SET_ERROR(EOVERFLOW)); (void) strncpy(buf, zfs_prop_default_string(prop), numints); } else { if (intsz != 8 || numints < 1) return (SET_ERROR(EOVERFLOW)); *(uint64_t *)buf = zfs_prop_default_numeric(prop); } return (0); } int dsl_prop_get_dd(dsl_dir_t *dd, const char *propname, int intsz, int numints, void *buf, char *setpoint, boolean_t snapshot) { int err = ENOENT; dsl_dir_t *target = dd; objset_t *mos = dd->dd_pool->dp_meta_objset; zfs_prop_t prop; boolean_t inheritable; boolean_t inheriting = B_FALSE; char *inheritstr; char *recvdstr; ASSERT(dsl_pool_config_held(dd->dd_pool)); if (setpoint) setpoint[0] = '\0'; prop = zfs_name_to_prop(propname); inheritable = (prop == ZPROP_INVAL || zfs_prop_inheritable(prop)); inheritstr = kmem_asprintf("%s%s", propname, ZPROP_INHERIT_SUFFIX); recvdstr = kmem_asprintf("%s%s", propname, ZPROP_RECVD_SUFFIX); /* * Note: dd may become NULL, therefore we shouldn't dereference it * after this loop. */ for (; dd != NULL; dd = dd->dd_parent) { if (dd != target || snapshot) { if (!inheritable) break; inheriting = B_TRUE; } /* Check for a local value. */ err = zap_lookup(mos, dsl_dir_phys(dd)->dd_props_zapobj, propname, intsz, numints, buf); if (err != ENOENT) { if (setpoint != NULL && err == 0) dsl_dir_name(dd, setpoint); break; } /* * Skip the check for a received value if there is an explicit * inheritance entry. */ err = zap_contains(mos, dsl_dir_phys(dd)->dd_props_zapobj, inheritstr); if (err != 0 && err != ENOENT) break; if (err == ENOENT) { /* Check for a received value. */ err = zap_lookup(mos, dsl_dir_phys(dd)->dd_props_zapobj, recvdstr, intsz, numints, buf); if (err != ENOENT) { if (setpoint != NULL && err == 0) { if (inheriting) { dsl_dir_name(dd, setpoint); } else { (void) strcpy(setpoint, ZPROP_SOURCE_VAL_RECVD); } } break; } } /* * If we found an explicit inheritance entry, err is zero even * though we haven't yet found the value, so reinitializing err * at the end of the loop (instead of at the beginning) ensures * that err has a valid post-loop value. */ err = SET_ERROR(ENOENT); } if (err == ENOENT) err = dodefault(prop, intsz, numints, buf); strfree(inheritstr); strfree(recvdstr); return (err); } int dsl_prop_get_ds(dsl_dataset_t *ds, const char *propname, int intsz, int numints, void *buf, char *setpoint) { zfs_prop_t prop = zfs_name_to_prop(propname); boolean_t inheritable; uint64_t zapobj; ASSERT(dsl_pool_config_held(ds->ds_dir->dd_pool)); inheritable = (prop == ZPROP_INVAL || zfs_prop_inheritable(prop)); zapobj = dsl_dataset_phys(ds)->ds_props_obj; if (zapobj != 0) { objset_t *mos = ds->ds_dir->dd_pool->dp_meta_objset; int err; ASSERT(ds->ds_is_snapshot); /* Check for a local value. */ err = zap_lookup(mos, zapobj, propname, intsz, numints, buf); if (err != ENOENT) { if (setpoint != NULL && err == 0) dsl_dataset_name(ds, setpoint); return (err); } /* * Skip the check for a received value if there is an explicit * inheritance entry. */ if (inheritable) { char *inheritstr = kmem_asprintf("%s%s", propname, ZPROP_INHERIT_SUFFIX); err = zap_contains(mos, zapobj, inheritstr); strfree(inheritstr); if (err != 0 && err != ENOENT) return (err); } if (err == ENOENT) { /* Check for a received value. */ char *recvdstr = kmem_asprintf("%s%s", propname, ZPROP_RECVD_SUFFIX); err = zap_lookup(mos, zapobj, recvdstr, intsz, numints, buf); strfree(recvdstr); if (err != ENOENT) { if (setpoint != NULL && err == 0) (void) strcpy(setpoint, ZPROP_SOURCE_VAL_RECVD); return (err); } } } return (dsl_prop_get_dd(ds->ds_dir, propname, intsz, numints, buf, setpoint, ds->ds_is_snapshot)); } static dsl_prop_record_t * dsl_prop_record_find(dsl_dir_t *dd, const char *propname) { dsl_prop_record_t *pr = NULL; ASSERT(MUTEX_HELD(&dd->dd_lock)); for (pr = list_head(&dd->dd_props); pr != NULL; pr = list_next(&dd->dd_props, pr)) { if (strcmp(pr->pr_propname, propname) == 0) break; } return (pr); } static dsl_prop_record_t * dsl_prop_record_create(dsl_dir_t *dd, const char *propname) { dsl_prop_record_t *pr; ASSERT(MUTEX_HELD(&dd->dd_lock)); pr = kmem_alloc(sizeof (dsl_prop_record_t), KM_SLEEP); pr->pr_propname = spa_strdup(propname); list_create(&pr->pr_cbs, sizeof (dsl_prop_cb_record_t), offsetof(dsl_prop_cb_record_t, cbr_pr_node)); list_insert_head(&dd->dd_props, pr); return (pr); } void dsl_prop_init(dsl_dir_t *dd) { list_create(&dd->dd_props, sizeof (dsl_prop_record_t), offsetof(dsl_prop_record_t, pr_node)); } void dsl_prop_fini(dsl_dir_t *dd) { dsl_prop_record_t *pr; while ((pr = list_remove_head(&dd->dd_props)) != NULL) { list_destroy(&pr->pr_cbs); strfree((char *)pr->pr_propname); kmem_free(pr, sizeof (dsl_prop_record_t)); } list_destroy(&dd->dd_props); } /* * Register interest in the named property. We'll call the callback * once to notify it of the current property value, and again each time * the property changes, until this callback is unregistered. * * Return 0 on success, errno if the prop is not an integer value. */ int dsl_prop_register(dsl_dataset_t *ds, const char *propname, dsl_prop_changed_cb_t *callback, void *cbarg) { dsl_dir_t *dd = ds->ds_dir; dsl_pool_t *dp = dd->dd_pool; uint64_t value; dsl_prop_record_t *pr; dsl_prop_cb_record_t *cbr; int err; ASSERT(dsl_pool_config_held(dp)); err = dsl_prop_get_int_ds(ds, propname, &value); if (err != 0) return (err); cbr = kmem_alloc(sizeof (dsl_prop_cb_record_t), KM_SLEEP); cbr->cbr_ds = ds; cbr->cbr_func = callback; cbr->cbr_arg = cbarg; mutex_enter(&dd->dd_lock); pr = dsl_prop_record_find(dd, propname); if (pr == NULL) pr = dsl_prop_record_create(dd, propname); cbr->cbr_pr = pr; list_insert_head(&pr->pr_cbs, cbr); list_insert_head(&ds->ds_prop_cbs, cbr); mutex_exit(&dd->dd_lock); cbr->cbr_func(cbr->cbr_arg, value); return (0); } int dsl_prop_get(const char *dsname, const char *propname, int intsz, int numints, void *buf, char *setpoint) { objset_t *os; int error; error = dmu_objset_hold(dsname, FTAG, &os); if (error != 0) return (error); error = dsl_prop_get_ds(dmu_objset_ds(os), propname, intsz, numints, buf, setpoint); dmu_objset_rele(os, FTAG); return (error); } /* * Get the current property value. It may have changed by the time this * function returns, so it is NOT safe to follow up with * dsl_prop_register() and assume that the value has not changed in * between. * * Return 0 on success, ENOENT if ddname is invalid. */ int dsl_prop_get_integer(const char *ddname, const char *propname, uint64_t *valuep, char *setpoint) { return (dsl_prop_get(ddname, propname, 8, 1, valuep, setpoint)); } int dsl_prop_get_int_ds(dsl_dataset_t *ds, const char *propname, uint64_t *valuep) { return (dsl_prop_get_ds(ds, propname, 8, 1, valuep, NULL)); } /* * Predict the effective value of the given special property if it were set with * the given value and source. This is not a general purpose function. It exists * only to handle the special requirements of the quota and reservation * properties. The fact that these properties are non-inheritable greatly * simplifies the prediction logic. * * Returns 0 on success, a positive error code on failure, or -1 if called with * a property not handled by this function. */ int dsl_prop_predict(dsl_dir_t *dd, const char *propname, zprop_source_t source, uint64_t value, uint64_t *newvalp) { zfs_prop_t prop = zfs_name_to_prop(propname); objset_t *mos; uint64_t zapobj; uint64_t version; char *recvdstr; int err = 0; switch (prop) { case ZFS_PROP_QUOTA: case ZFS_PROP_RESERVATION: case ZFS_PROP_REFQUOTA: case ZFS_PROP_REFRESERVATION: break; default: return (-1); } mos = dd->dd_pool->dp_meta_objset; zapobj = dsl_dir_phys(dd)->dd_props_zapobj; recvdstr = kmem_asprintf("%s%s", propname, ZPROP_RECVD_SUFFIX); version = spa_version(dd->dd_pool->dp_spa); if (version < SPA_VERSION_RECVD_PROPS) { if (source & ZPROP_SRC_NONE) source = ZPROP_SRC_NONE; else if (source & ZPROP_SRC_RECEIVED) source = ZPROP_SRC_LOCAL; } switch (source) { case ZPROP_SRC_NONE: /* Revert to the received value, if any. */ err = zap_lookup(mos, zapobj, recvdstr, 8, 1, newvalp); if (err == ENOENT) *newvalp = 0; break; case ZPROP_SRC_LOCAL: *newvalp = value; break; case ZPROP_SRC_RECEIVED: /* * If there's no local setting, then the new received value will * be the effective value. */ err = zap_lookup(mos, zapobj, propname, 8, 1, newvalp); if (err == ENOENT) *newvalp = value; break; case (ZPROP_SRC_NONE | ZPROP_SRC_RECEIVED): /* * We're clearing the received value, so the local setting (if * it exists) remains the effective value. */ err = zap_lookup(mos, zapobj, propname, 8, 1, newvalp); if (err == ENOENT) *newvalp = 0; break; default: panic("unexpected property source: %d", source); } strfree(recvdstr); if (err == ENOENT) return (0); return (err); } /* * Unregister all callbacks that are registered with the * given callback argument. */ void dsl_prop_unregister_all(dsl_dataset_t *ds, void *cbarg) { dsl_prop_cb_record_t *cbr, *next_cbr; dsl_dir_t *dd = ds->ds_dir; mutex_enter(&dd->dd_lock); next_cbr = list_head(&ds->ds_prop_cbs); while (next_cbr != NULL) { cbr = next_cbr; next_cbr = list_next(&ds->ds_prop_cbs, cbr); if (cbr->cbr_arg == cbarg) { list_remove(&ds->ds_prop_cbs, cbr); list_remove(&cbr->cbr_pr->pr_cbs, cbr); kmem_free(cbr, sizeof (dsl_prop_cb_record_t)); } } mutex_exit(&dd->dd_lock); } boolean_t dsl_prop_hascb(dsl_dataset_t *ds) { return (!list_is_empty(&ds->ds_prop_cbs)); } /* ARGSUSED */ static int dsl_prop_notify_all_cb(dsl_pool_t *dp, dsl_dataset_t *ds, void *arg) { dsl_dir_t *dd = ds->ds_dir; dsl_prop_record_t *pr; dsl_prop_cb_record_t *cbr; mutex_enter(&dd->dd_lock); for (pr = list_head(&dd->dd_props); pr; pr = list_next(&dd->dd_props, pr)) { for (cbr = list_head(&pr->pr_cbs); cbr; cbr = list_next(&pr->pr_cbs, cbr)) { uint64_t value; /* * Callback entries do not have holds on their * datasets so that datasets with registered * callbacks are still eligible for eviction. * Unlike operations to update properties on a * single dataset, we are performing a recursive * descent of related head datasets. The caller * of this function only has a dataset hold on * the passed in head dataset, not the snapshots * associated with this dataset. Without a hold, * the dataset pointer within callback records * for snapshots can be invalidated by eviction * at any time. * * Use dsl_dataset_try_add_ref() to verify * that the dataset for a snapshot has not * begun eviction processing and to prevent * eviction from occurring for the duration of * the callback. If the hold attempt fails, * this object is already being evicted and the * callback can be safely ignored. */ if (ds != cbr->cbr_ds && !dsl_dataset_try_add_ref(dp, cbr->cbr_ds, FTAG)) continue; if (dsl_prop_get_ds(cbr->cbr_ds, cbr->cbr_pr->pr_propname, sizeof (value), 1, &value, NULL) == 0) cbr->cbr_func(cbr->cbr_arg, value); if (ds != cbr->cbr_ds) dsl_dataset_rele(cbr->cbr_ds, FTAG); } } mutex_exit(&dd->dd_lock); return (0); } /* * Update all property values for ddobj & its descendants. This is used * when renaming the dir. */ void dsl_prop_notify_all(dsl_dir_t *dd) { dsl_pool_t *dp = dd->dd_pool; ASSERT(RRW_WRITE_HELD(&dp->dp_config_rwlock)); (void) dmu_objset_find_dp(dp, dd->dd_object, dsl_prop_notify_all_cb, NULL, DS_FIND_CHILDREN); } static void dsl_prop_changed_notify(dsl_pool_t *dp, uint64_t ddobj, const char *propname, uint64_t value, int first) { dsl_dir_t *dd; dsl_prop_record_t *pr; dsl_prop_cb_record_t *cbr; objset_t *mos = dp->dp_meta_objset; zap_cursor_t zc; zap_attribute_t *za; int err; ASSERT(RRW_WRITE_HELD(&dp->dp_config_rwlock)); err = dsl_dir_hold_obj(dp, ddobj, NULL, FTAG, &dd); if (err) return; if (!first) { /* * If the prop is set here, then this change is not * being inherited here or below; stop the recursion. */ err = zap_contains(mos, dsl_dir_phys(dd)->dd_props_zapobj, propname); if (err == 0) { dsl_dir_rele(dd, FTAG); return; } ASSERT3U(err, ==, ENOENT); } mutex_enter(&dd->dd_lock); pr = dsl_prop_record_find(dd, propname); if (pr != NULL) { for (cbr = list_head(&pr->pr_cbs); cbr; cbr = list_next(&pr->pr_cbs, cbr)) { uint64_t propobj; /* * cbr->cbr_ds may be invalidated due to eviction, * requiring the use of dsl_dataset_try_add_ref(). * See comment block in dsl_prop_notify_all_cb() * for details. */ if (!dsl_dataset_try_add_ref(dp, cbr->cbr_ds, FTAG)) continue; propobj = dsl_dataset_phys(cbr->cbr_ds)->ds_props_obj; /* * If the property is not set on this ds, then it is * inherited here; call the callback. */ if (propobj == 0 || zap_contains(mos, propobj, propname) != 0) cbr->cbr_func(cbr->cbr_arg, value); dsl_dataset_rele(cbr->cbr_ds, FTAG); } } mutex_exit(&dd->dd_lock); za = kmem_alloc(sizeof (zap_attribute_t), KM_SLEEP); for (zap_cursor_init(&zc, mos, dsl_dir_phys(dd)->dd_child_dir_zapobj); zap_cursor_retrieve(&zc, za) == 0; zap_cursor_advance(&zc)) { dsl_prop_changed_notify(dp, za->za_first_integer, propname, value, FALSE); } kmem_free(za, sizeof (zap_attribute_t)); zap_cursor_fini(&zc); dsl_dir_rele(dd, FTAG); } void dsl_prop_set_sync_impl(dsl_dataset_t *ds, const char *propname, zprop_source_t source, int intsz, int numints, const void *value, dmu_tx_t *tx) { objset_t *mos = ds->ds_dir->dd_pool->dp_meta_objset; uint64_t zapobj, intval, dummy; int isint; char valbuf[32]; const char *valstr = NULL; char *inheritstr; char *recvdstr; char *tbuf = NULL; int err; uint64_t version = spa_version(ds->ds_dir->dd_pool->dp_spa); isint = (dodefault(zfs_name_to_prop(propname), 8, 1, &intval) == 0); if (ds->ds_is_snapshot) { ASSERT(version >= SPA_VERSION_SNAP_PROPS); if (dsl_dataset_phys(ds)->ds_props_obj == 0) { dmu_buf_will_dirty(ds->ds_dbuf, tx); dsl_dataset_phys(ds)->ds_props_obj = zap_create(mos, DMU_OT_DSL_PROPS, DMU_OT_NONE, 0, tx); } zapobj = dsl_dataset_phys(ds)->ds_props_obj; } else { zapobj = dsl_dir_phys(ds->ds_dir)->dd_props_zapobj; } if (version < SPA_VERSION_RECVD_PROPS) { if (source & ZPROP_SRC_NONE) source = ZPROP_SRC_NONE; else if (source & ZPROP_SRC_RECEIVED) source = ZPROP_SRC_LOCAL; } inheritstr = kmem_asprintf("%s%s", propname, ZPROP_INHERIT_SUFFIX); recvdstr = kmem_asprintf("%s%s", propname, ZPROP_RECVD_SUFFIX); switch (source) { case ZPROP_SRC_NONE: /* * revert to received value, if any (inherit -S) * - remove propname * - remove propname$inherit */ err = zap_remove(mos, zapobj, propname, tx); ASSERT(err == 0 || err == ENOENT); err = zap_remove(mos, zapobj, inheritstr, tx); ASSERT(err == 0 || err == ENOENT); break; case ZPROP_SRC_LOCAL: /* * remove propname$inherit * set propname -> value */ err = zap_remove(mos, zapobj, inheritstr, tx); ASSERT(err == 0 || err == ENOENT); VERIFY0(zap_update(mos, zapobj, propname, intsz, numints, value, tx)); break; case ZPROP_SRC_INHERITED: /* * explicitly inherit * - remove propname * - set propname$inherit */ err = zap_remove(mos, zapobj, propname, tx); ASSERT(err == 0 || err == ENOENT); if (version >= SPA_VERSION_RECVD_PROPS && dsl_prop_get_int_ds(ds, ZPROP_HAS_RECVD, &dummy) == 0) { dummy = 0; VERIFY0(zap_update(mos, zapobj, inheritstr, 8, 1, &dummy, tx)); } break; case ZPROP_SRC_RECEIVED: /* * set propname$recvd -> value */ err = zap_update(mos, zapobj, recvdstr, intsz, numints, value, tx); ASSERT(err == 0); break; case (ZPROP_SRC_NONE | ZPROP_SRC_LOCAL | ZPROP_SRC_RECEIVED): /* * clear local and received settings * - remove propname * - remove propname$inherit * - remove propname$recvd */ err = zap_remove(mos, zapobj, propname, tx); ASSERT(err == 0 || err == ENOENT); err = zap_remove(mos, zapobj, inheritstr, tx); ASSERT(err == 0 || err == ENOENT); /* FALLTHRU */ case (ZPROP_SRC_NONE | ZPROP_SRC_RECEIVED): /* * remove propname$recvd */ err = zap_remove(mos, zapobj, recvdstr, tx); ASSERT(err == 0 || err == ENOENT); break; default: cmn_err(CE_PANIC, "unexpected property source: %d", source); } strfree(inheritstr); strfree(recvdstr); if (isint) { VERIFY0(dsl_prop_get_int_ds(ds, propname, &intval)); if (ds->ds_is_snapshot) { dsl_prop_cb_record_t *cbr; /* * It's a snapshot; nothing can inherit this * property, so just look for callbacks on this * ds here. */ mutex_enter(&ds->ds_dir->dd_lock); for (cbr = list_head(&ds->ds_prop_cbs); cbr; cbr = list_next(&ds->ds_prop_cbs, cbr)) { if (strcmp(cbr->cbr_pr->pr_propname, propname) == 0) cbr->cbr_func(cbr->cbr_arg, intval); } mutex_exit(&ds->ds_dir->dd_lock); } else { dsl_prop_changed_notify(ds->ds_dir->dd_pool, ds->ds_dir->dd_object, propname, intval, TRUE); } (void) snprintf(valbuf, sizeof (valbuf), "%lld", (longlong_t)intval); valstr = valbuf; } else { if (source == ZPROP_SRC_LOCAL) { valstr = value; } else { tbuf = kmem_alloc(ZAP_MAXVALUELEN, KM_SLEEP); if (dsl_prop_get_ds(ds, propname, 1, ZAP_MAXVALUELEN, tbuf, NULL) == 0) valstr = tbuf; } } spa_history_log_internal_ds(ds, (source == ZPROP_SRC_NONE || source == ZPROP_SRC_INHERITED) ? "inherit" : "set", tx, "%s=%s", propname, (valstr == NULL ? "" : valstr)); if (tbuf != NULL) kmem_free(tbuf, ZAP_MAXVALUELEN); } int dsl_prop_set_int(const char *dsname, const char *propname, zprop_source_t source, uint64_t value) { nvlist_t *nvl = fnvlist_alloc(); int error; fnvlist_add_uint64(nvl, propname, value); error = dsl_props_set(dsname, source, nvl); fnvlist_free(nvl); return (error); } int dsl_prop_set_string(const char *dsname, const char *propname, zprop_source_t source, const char *value) { nvlist_t *nvl = fnvlist_alloc(); int error; fnvlist_add_string(nvl, propname, value); error = dsl_props_set(dsname, source, nvl); fnvlist_free(nvl); return (error); } int dsl_prop_inherit(const char *dsname, const char *propname, zprop_source_t source) { nvlist_t *nvl = fnvlist_alloc(); int error; fnvlist_add_boolean(nvl, propname); error = dsl_props_set(dsname, source, nvl); fnvlist_free(nvl); return (error); } typedef struct dsl_props_set_arg { const char *dpsa_dsname; zprop_source_t dpsa_source; nvlist_t *dpsa_props; } dsl_props_set_arg_t; static int dsl_props_set_check(void *arg, dmu_tx_t *tx) { dsl_props_set_arg_t *dpsa = arg; dsl_pool_t *dp = dmu_tx_pool(tx); dsl_dataset_t *ds; uint64_t version; nvpair_t *elem = NULL; int err; err = dsl_dataset_hold(dp, dpsa->dpsa_dsname, FTAG, &ds); if (err != 0) return (err); version = spa_version(ds->ds_dir->dd_pool->dp_spa); while ((elem = nvlist_next_nvpair(dpsa->dpsa_props, elem)) != NULL) { if (strlen(nvpair_name(elem)) >= ZAP_MAXNAMELEN) { dsl_dataset_rele(ds, FTAG); return (SET_ERROR(ENAMETOOLONG)); } if (nvpair_type(elem) == DATA_TYPE_STRING) { char *valstr = fnvpair_value_string(elem); if (strlen(valstr) >= (version < SPA_VERSION_STMF_PROP ? ZAP_OLDMAXVALUELEN : ZAP_MAXVALUELEN)) { dsl_dataset_rele(ds, FTAG); return (E2BIG); } } } if (ds->ds_is_snapshot && version < SPA_VERSION_SNAP_PROPS) { dsl_dataset_rele(ds, FTAG); return (SET_ERROR(ENOTSUP)); } dsl_dataset_rele(ds, FTAG); return (0); } void dsl_props_set_sync_impl(dsl_dataset_t *ds, zprop_source_t source, nvlist_t *props, dmu_tx_t *tx) { nvpair_t *elem = NULL; while ((elem = nvlist_next_nvpair(props, elem)) != NULL) { nvpair_t *pair = elem; if (nvpair_type(pair) == DATA_TYPE_NVLIST) { /* * dsl_prop_get_all_impl() returns properties in this * format. */ nvlist_t *attrs = fnvpair_value_nvlist(pair); pair = fnvlist_lookup_nvpair(attrs, ZPROP_VALUE); } if (nvpair_type(pair) == DATA_TYPE_STRING) { const char *value = fnvpair_value_string(pair); dsl_prop_set_sync_impl(ds, nvpair_name(pair), source, 1, strlen(value) + 1, value, tx); } else if (nvpair_type(pair) == DATA_TYPE_UINT64) { uint64_t intval = fnvpair_value_uint64(pair); dsl_prop_set_sync_impl(ds, nvpair_name(pair), source, sizeof (intval), 1, &intval, tx); } else if (nvpair_type(pair) == DATA_TYPE_BOOLEAN) { dsl_prop_set_sync_impl(ds, nvpair_name(pair), source, 0, 0, NULL, tx); } else { panic("invalid nvpair type"); } } } static void dsl_props_set_sync(void *arg, dmu_tx_t *tx) { dsl_props_set_arg_t *dpsa = arg; dsl_pool_t *dp = dmu_tx_pool(tx); dsl_dataset_t *ds; VERIFY0(dsl_dataset_hold(dp, dpsa->dpsa_dsname, FTAG, &ds)); dsl_props_set_sync_impl(ds, dpsa->dpsa_source, dpsa->dpsa_props, tx); dsl_dataset_rele(ds, FTAG); } /* * All-or-nothing; if any prop can't be set, nothing will be modified. */ int dsl_props_set(const char *dsname, zprop_source_t source, nvlist_t *props) { dsl_props_set_arg_t dpsa; int nblks = 0; dpsa.dpsa_dsname = dsname; dpsa.dpsa_source = source; dpsa.dpsa_props = props; /* * If the source includes NONE, then we will only be removing entries * from the ZAP object. In that case don't check for ENOSPC. */ if ((source & ZPROP_SRC_NONE) == 0) nblks = 2 * fnvlist_num_pairs(props); return (dsl_sync_task(dsname, dsl_props_set_check, dsl_props_set_sync, &dpsa, nblks, ZFS_SPACE_CHECK_RESERVED)); } typedef enum dsl_prop_getflags { DSL_PROP_GET_INHERITING = 0x1, /* searching parent of target ds */ DSL_PROP_GET_SNAPSHOT = 0x2, /* snapshot dataset */ DSL_PROP_GET_LOCAL = 0x4, /* local properties */ DSL_PROP_GET_RECEIVED = 0x8 /* received properties */ } dsl_prop_getflags_t; static int dsl_prop_get_all_impl(objset_t *mos, uint64_t propobj, const char *setpoint, dsl_prop_getflags_t flags, nvlist_t *nv) { zap_cursor_t zc; zap_attribute_t za; int err = 0; for (zap_cursor_init(&zc, mos, propobj); (err = zap_cursor_retrieve(&zc, &za)) == 0; zap_cursor_advance(&zc)) { nvlist_t *propval; zfs_prop_t prop; char buf[ZAP_MAXNAMELEN]; char *valstr; const char *suffix; const char *propname; const char *source; suffix = strchr(za.za_name, '$'); if (suffix == NULL) { /* * Skip local properties if we only want received * properties. */ if (flags & DSL_PROP_GET_RECEIVED) continue; propname = za.za_name; source = setpoint; } else if (strcmp(suffix, ZPROP_INHERIT_SUFFIX) == 0) { /* Skip explicitly inherited entries. */ continue; } else if (strcmp(suffix, ZPROP_RECVD_SUFFIX) == 0) { if (flags & DSL_PROP_GET_LOCAL) continue; (void) strncpy(buf, za.za_name, (suffix - za.za_name)); buf[suffix - za.za_name] = '\0'; propname = buf; if (!(flags & DSL_PROP_GET_RECEIVED)) { /* Skip if locally overridden. */ err = zap_contains(mos, propobj, propname); if (err == 0) continue; if (err != ENOENT) break; /* Skip if explicitly inherited. */ valstr = kmem_asprintf("%s%s", propname, ZPROP_INHERIT_SUFFIX); err = zap_contains(mos, propobj, valstr); strfree(valstr); if (err == 0) continue; if (err != ENOENT) break; } source = ((flags & DSL_PROP_GET_INHERITING) ? setpoint : ZPROP_SOURCE_VAL_RECVD); } else { /* * For backward compatibility, skip suffixes we don't * recognize. */ continue; } prop = zfs_name_to_prop(propname); /* Skip non-inheritable properties. */ if ((flags & DSL_PROP_GET_INHERITING) && prop != ZPROP_INVAL && !zfs_prop_inheritable(prop)) continue; /* Skip properties not valid for this type. */ if ((flags & DSL_PROP_GET_SNAPSHOT) && prop != ZPROP_INVAL && !zfs_prop_valid_for_type(prop, ZFS_TYPE_SNAPSHOT)) continue; /* Skip properties already defined. */ if (nvlist_exists(nv, propname)) continue; VERIFY(nvlist_alloc(&propval, NV_UNIQUE_NAME, KM_SLEEP) == 0); if (za.za_integer_length == 1) { /* * String property */ char *tmp = kmem_alloc(za.za_num_integers, KM_SLEEP); err = zap_lookup(mos, propobj, za.za_name, 1, za.za_num_integers, tmp); if (err != 0) { kmem_free(tmp, za.za_num_integers); break; } VERIFY(nvlist_add_string(propval, ZPROP_VALUE, tmp) == 0); kmem_free(tmp, za.za_num_integers); } else { /* * Integer property */ ASSERT(za.za_integer_length == 8); (void) nvlist_add_uint64(propval, ZPROP_VALUE, za.za_first_integer); } VERIFY(nvlist_add_string(propval, ZPROP_SOURCE, source) == 0); VERIFY(nvlist_add_nvlist(nv, propname, propval) == 0); nvlist_free(propval); } zap_cursor_fini(&zc); if (err == ENOENT) err = 0; return (err); } /* * Iterate over all properties for this dataset and return them in an nvlist. */ static int dsl_prop_get_all_ds(dsl_dataset_t *ds, nvlist_t **nvp, dsl_prop_getflags_t flags) { dsl_dir_t *dd = ds->ds_dir; dsl_pool_t *dp = dd->dd_pool; objset_t *mos = dp->dp_meta_objset; int err = 0; char setpoint[MAXNAMELEN]; VERIFY(nvlist_alloc(nvp, NV_UNIQUE_NAME, KM_SLEEP) == 0); if (ds->ds_is_snapshot) flags |= DSL_PROP_GET_SNAPSHOT; ASSERT(dsl_pool_config_held(dp)); if (dsl_dataset_phys(ds)->ds_props_obj != 0) { ASSERT(flags & DSL_PROP_GET_SNAPSHOT); dsl_dataset_name(ds, setpoint); err = dsl_prop_get_all_impl(mos, dsl_dataset_phys(ds)->ds_props_obj, setpoint, flags, *nvp); if (err) goto out; } for (; dd != NULL; dd = dd->dd_parent) { if (dd != ds->ds_dir || (flags & DSL_PROP_GET_SNAPSHOT)) { if (flags & (DSL_PROP_GET_LOCAL | DSL_PROP_GET_RECEIVED)) break; flags |= DSL_PROP_GET_INHERITING; } dsl_dir_name(dd, setpoint); err = dsl_prop_get_all_impl(mos, dsl_dir_phys(dd)->dd_props_zapobj, setpoint, flags, *nvp); if (err) break; } out: return (err); } boolean_t dsl_prop_get_hasrecvd(const char *dsname) { uint64_t dummy; return (0 == dsl_prop_get_integer(dsname, ZPROP_HAS_RECVD, &dummy, NULL)); } static int dsl_prop_set_hasrecvd_impl(const char *dsname, zprop_source_t source) { uint64_t version; spa_t *spa; int error = 0; VERIFY0(spa_open(dsname, &spa, FTAG)); version = spa_version(spa); spa_close(spa, FTAG); if (version >= SPA_VERSION_RECVD_PROPS) error = dsl_prop_set_int(dsname, ZPROP_HAS_RECVD, source, 0); return (error); } /* * Call after successfully receiving properties to ensure that only the first * receive on or after SPA_VERSION_RECVD_PROPS blows away local properties. */ int dsl_prop_set_hasrecvd(const char *dsname) { int error = 0; if (!dsl_prop_get_hasrecvd(dsname)) error = dsl_prop_set_hasrecvd_impl(dsname, ZPROP_SRC_LOCAL); return (error); } void dsl_prop_unset_hasrecvd(const char *dsname) { VERIFY0(dsl_prop_set_hasrecvd_impl(dsname, ZPROP_SRC_NONE)); } int dsl_prop_get_all(objset_t *os, nvlist_t **nvp) { return (dsl_prop_get_all_ds(os->os_dsl_dataset, nvp, 0)); } int dsl_prop_get_received(const char *dsname, nvlist_t **nvp) { objset_t *os; int error; /* * Received properties are not distinguishable from local properties * until the dataset has received properties on or after * SPA_VERSION_RECVD_PROPS. */ dsl_prop_getflags_t flags = (dsl_prop_get_hasrecvd(dsname) ? DSL_PROP_GET_RECEIVED : DSL_PROP_GET_LOCAL); error = dmu_objset_hold(dsname, FTAG, &os); if (error != 0) return (error); error = dsl_prop_get_all_ds(os->os_dsl_dataset, nvp, flags); dmu_objset_rele(os, FTAG); return (error); } void dsl_prop_nvlist_add_uint64(nvlist_t *nv, zfs_prop_t prop, uint64_t value) { nvlist_t *propval; const char *propname = zfs_prop_to_name(prop); uint64_t default_value; if (nvlist_lookup_nvlist(nv, propname, &propval) == 0) { VERIFY(nvlist_add_uint64(propval, ZPROP_VALUE, value) == 0); return; } VERIFY(nvlist_alloc(&propval, NV_UNIQUE_NAME, KM_SLEEP) == 0); VERIFY(nvlist_add_uint64(propval, ZPROP_VALUE, value) == 0); /* Indicate the default source if we can. */ if (dodefault(prop, 8, 1, &default_value) == 0 && value == default_value) { VERIFY(nvlist_add_string(propval, ZPROP_SOURCE, "") == 0); } VERIFY(nvlist_add_nvlist(nv, propname, propval) == 0); nvlist_free(propval); } void dsl_prop_nvlist_add_string(nvlist_t *nv, zfs_prop_t prop, const char *value) { nvlist_t *propval; const char *propname = zfs_prop_to_name(prop); if (nvlist_lookup_nvlist(nv, propname, &propval) == 0) { VERIFY(nvlist_add_string(propval, ZPROP_VALUE, value) == 0); return; } VERIFY(nvlist_alloc(&propval, NV_UNIQUE_NAME, KM_SLEEP) == 0); VERIFY(nvlist_add_string(propval, ZPROP_VALUE, value) == 0); VERIFY(nvlist_add_nvlist(nv, propname, propval) == 0); nvlist_free(propval); } Index: projects/netbsd-tests-update-12/sys/cddl/contrib/opensolaris =================================================================== --- projects/netbsd-tests-update-12/sys/cddl/contrib/opensolaris (revision 305171) +++ projects/netbsd-tests-update-12/sys/cddl/contrib/opensolaris (revision 305172) Property changes on: projects/netbsd-tests-update-12/sys/cddl/contrib/opensolaris ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head/sys/cddl/contrib/opensolaris:r304233-305170 Index: projects/netbsd-tests-update-12/sys/contrib/vchiq/interface/vchiq_arm/vchiq_arm.c =================================================================== --- projects/netbsd-tests-update-12/sys/contrib/vchiq/interface/vchiq_arm/vchiq_arm.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/contrib/vchiq/interface/vchiq_arm/vchiq_arm.c (revision 305172) @@ -1,2911 +1,2930 @@ /** * Copyright (c) 2014 Raspberry Pi (Trading) Ltd. All rights reserved. * Copyright (c) 2010-2012 Broadcom. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The names of the above-listed copyright holders may not be used * to endorse or promote products derived from this software without * specific prior written permission. * * ALTERNATIVELY, this software may be distributed under the terms of the * GNU General Public License ("GPL") version 2, as published by the Free * Software Foundation. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "vchiq_core.h" #include "vchiq_ioctl.h" #include "vchiq_arm.h" #define DEVICE_NAME "vchiq" /* Override the default prefix, which would be vchiq_arm (from the filename) */ #undef MODULE_PARAM_PREFIX #define MODULE_PARAM_PREFIX DEVICE_NAME "." #define VCHIQ_MINOR 0 /* Some per-instance constants */ -#define MAX_COMPLETIONS 16 +#define MAX_COMPLETIONS 128 #define MAX_SERVICES 64 #define MAX_ELEMENTS 8 -#define MSG_QUEUE_SIZE 64 +#define MSG_QUEUE_SIZE 128 #define KEEPALIVE_VER 1 #define KEEPALIVE_VER_MIN KEEPALIVE_VER /* Run time control of log level, based on KERN_XXX level. */ int vchiq_arm_log_level = VCHIQ_LOG_DEFAULT; int vchiq_susp_log_level = VCHIQ_LOG_ERROR; #define SUSPEND_TIMER_TIMEOUT_MS 100 #define SUSPEND_RETRY_TIMER_TIMEOUT_MS 1000 #define VC_SUSPEND_NUM_OFFSET 3 /* number of values before idle which are -ve */ static const char *const suspend_state_names[] = { "VC_SUSPEND_FORCE_CANCELED", "VC_SUSPEND_REJECTED", "VC_SUSPEND_FAILED", "VC_SUSPEND_IDLE", "VC_SUSPEND_REQUESTED", "VC_SUSPEND_IN_PROGRESS", "VC_SUSPEND_SUSPENDED" }; #define VC_RESUME_NUM_OFFSET 1 /* number of values before idle which are -ve */ static const char *const resume_state_names[] = { "VC_RESUME_FAILED", "VC_RESUME_IDLE", "VC_RESUME_REQUESTED", "VC_RESUME_IN_PROGRESS", "VC_RESUME_RESUMED" }; /* The number of times we allow force suspend to timeout before actually ** _forcing_ suspend. This is to cater for SW which fails to release vchiq ** correctly - we don't want to prevent ARM suspend indefinitely in this case. */ #define FORCE_SUSPEND_FAIL_MAX 8 /* The time in ms allowed for videocore to go idle when force suspend has been * requested */ #define FORCE_SUSPEND_TIMEOUT_MS 200 static void suspend_timer_callback(unsigned long context); #ifdef notyet static int vchiq_proc_add_instance(VCHIQ_INSTANCE_T instance); static void vchiq_proc_remove_instance(VCHIQ_INSTANCE_T instance); #endif typedef struct user_service_struct { VCHIQ_SERVICE_T *service; void *userdata; VCHIQ_INSTANCE_T instance; char is_vchi; char dequeue_pending; char close_pending; int message_available_pos; int msg_insert; int msg_remove; struct semaphore insert_event; struct semaphore remove_event; struct semaphore close_event; VCHIQ_HEADER_T * msg_queue[MSG_QUEUE_SIZE]; } USER_SERVICE_T; struct bulk_waiter_node { struct bulk_waiter bulk_waiter; int pid; struct list_head list; }; struct vchiq_instance_struct { VCHIQ_STATE_T *state; VCHIQ_COMPLETION_DATA_T completions[MAX_COMPLETIONS]; int completion_insert; int completion_remove; struct semaphore insert_event; struct semaphore remove_event; struct mutex completion_mutex; int connected; int closing; int pid; int mark; int use_close_delivered; int trace; struct list_head bulk_waiter_list; struct mutex bulk_waiter_list_mutex; #ifdef notyet VCHIQ_DEBUGFS_NODE_T proc_entry; #endif }; typedef struct dump_context_struct { char __user *buf; size_t actual; size_t space; loff_t offset; } DUMP_CONTEXT_T; static struct cdev * vchiq_cdev; VCHIQ_STATE_T g_state; static DEFINE_SPINLOCK(msg_queue_spinlock); static const char *const ioctl_names[] = { "CONNECT", "SHUTDOWN", "CREATE_SERVICE", "REMOVE_SERVICE", "QUEUE_MESSAGE", "QUEUE_BULK_TRANSMIT", "QUEUE_BULK_RECEIVE", "AWAIT_COMPLETION", "DEQUEUE_MESSAGE", "GET_CLIENT_ID", "GET_CONFIG", "CLOSE_SERVICE", "USE_SERVICE", "RELEASE_SERVICE", "SET_SERVICE_OPTION", "DUMP_PHYS_MEM", "LIB_VERSION", "CLOSE_DELIVERED" }; vchiq_static_assert((sizeof(ioctl_names)/sizeof(ioctl_names[0])) == (VCHIQ_IOC_MAX + 1)); static eventhandler_tag vchiq_ehtag = NULL; static d_open_t vchiq_open; static d_close_t vchiq_close; static d_ioctl_t vchiq_ioctl; static struct cdevsw vchiq_cdevsw = { .d_version = D_VERSION, .d_ioctl = vchiq_ioctl, .d_open = vchiq_open, .d_close = vchiq_close, .d_name = DEVICE_NAME, }; #if 0 static void dump_phys_mem(void *virt_addr, uint32_t num_bytes); #endif /**************************************************************************** * * add_completion * ***************************************************************************/ static VCHIQ_STATUS_T add_completion(VCHIQ_INSTANCE_T instance, VCHIQ_REASON_T reason, VCHIQ_HEADER_T *header, USER_SERVICE_T *user_service, void *bulk_userdata) { VCHIQ_COMPLETION_DATA_T *completion; + int insert; DEBUG_INITIALISE(g_state.local) - while (instance->completion_insert == - (instance->completion_remove + MAX_COMPLETIONS)) { + insert = instance->completion_insert; + while ((insert - instance->completion_remove) >= MAX_COMPLETIONS) { /* Out of space - wait for the client */ DEBUG_TRACE(SERVICE_CALLBACK_LINE); vchiq_log_trace(vchiq_arm_log_level, "add_completion - completion queue full"); DEBUG_COUNT(COMPLETION_QUEUE_FULL_COUNT); + if (down_interruptible(&instance->remove_event) != 0) { vchiq_log_info(vchiq_arm_log_level, "service_callback interrupted"); return VCHIQ_RETRY; - } else if (instance->closing) { + } + + if (instance->closing) { vchiq_log_info(vchiq_arm_log_level, "service_callback closing"); - return VCHIQ_ERROR; + return VCHIQ_SUCCESS; } DEBUG_TRACE(SERVICE_CALLBACK_LINE); } - completion = - &instance->completions[instance->completion_insert & - (MAX_COMPLETIONS - 1)]; + completion = &instance->completions[insert & (MAX_COMPLETIONS - 1)]; completion->header = header; completion->reason = reason; /* N.B. service_userdata is updated while processing AWAIT_COMPLETION */ completion->service_userdata = user_service->service; completion->bulk_userdata = bulk_userdata; if (reason == VCHIQ_SERVICE_CLOSED) { /* Take an extra reference, to be held until this CLOSED notification is delivered. */ lock_service(user_service->service); if (instance->use_close_delivered) user_service->close_pending = 1; } /* A write barrier is needed here to ensure that the entire completion record is written out before the insert point. */ wmb(); if (reason == VCHIQ_MESSAGE_AVAILABLE) - user_service->message_available_pos = - instance->completion_insert; - instance->completion_insert++; + user_service->message_available_pos = insert; + instance->completion_insert = ++insert; + up(&instance->insert_event); return VCHIQ_SUCCESS; } /**************************************************************************** * * service_callback * ***************************************************************************/ static VCHIQ_STATUS_T service_callback(VCHIQ_REASON_T reason, VCHIQ_HEADER_T *header, VCHIQ_SERVICE_HANDLE_T handle, void *bulk_userdata) { /* How do we ensure the callback goes to the right client? ** The service_user data points to a USER_SERVICE_T record containing ** the original callback and the user state structure, which contains a ** circular buffer for completion records. */ USER_SERVICE_T *user_service; VCHIQ_SERVICE_T *service; VCHIQ_INSTANCE_T instance; + int skip_completion = 0; DEBUG_INITIALISE(g_state.local) DEBUG_TRACE(SERVICE_CALLBACK_LINE); service = handle_to_service(handle); BUG_ON(!service); user_service = (USER_SERVICE_T *)service->base.userdata; instance = user_service->instance; if (!instance || instance->closing) return VCHIQ_SUCCESS; vchiq_log_trace(vchiq_arm_log_level, "service_callback - service %lx(%d,%p), reason %d, header %lx, " "instance %lx, bulk_userdata %lx", (unsigned long)user_service, service->localport, user_service->userdata, reason, (unsigned long)header, (unsigned long)instance, (unsigned long)bulk_userdata); if (header && user_service->is_vchi) { spin_lock(&msg_queue_spinlock); while (user_service->msg_insert == (user_service->msg_remove + MSG_QUEUE_SIZE)) { spin_unlock(&msg_queue_spinlock); DEBUG_TRACE(SERVICE_CALLBACK_LINE); DEBUG_COUNT(MSG_QUEUE_FULL_COUNT); vchiq_log_trace(vchiq_arm_log_level, "service_callback - msg queue full"); /* If there is no MESSAGE_AVAILABLE in the completion ** queue, add one */ if ((user_service->message_available_pos - instance->completion_remove) < 0) { VCHIQ_STATUS_T status; vchiq_log_info(vchiq_arm_log_level, "Inserting extra MESSAGE_AVAILABLE"); DEBUG_TRACE(SERVICE_CALLBACK_LINE); status = add_completion(instance, reason, NULL, user_service, bulk_userdata); if (status != VCHIQ_SUCCESS) { DEBUG_TRACE(SERVICE_CALLBACK_LINE); return status; } } DEBUG_TRACE(SERVICE_CALLBACK_LINE); if (down_interruptible(&user_service->remove_event) != 0) { vchiq_log_info(vchiq_arm_log_level, "service_callback interrupted"); DEBUG_TRACE(SERVICE_CALLBACK_LINE); return VCHIQ_RETRY; } else if (instance->closing) { vchiq_log_info(vchiq_arm_log_level, "service_callback closing"); DEBUG_TRACE(SERVICE_CALLBACK_LINE); return VCHIQ_ERROR; } DEBUG_TRACE(SERVICE_CALLBACK_LINE); spin_lock(&msg_queue_spinlock); } user_service->msg_queue[user_service->msg_insert & (MSG_QUEUE_SIZE - 1)] = header; user_service->msg_insert++; - spin_unlock(&msg_queue_spinlock); - up(&user_service->insert_event); - /* If there is a thread waiting in DEQUEUE_MESSAGE, or if ** there is a MESSAGE_AVAILABLE in the completion queue then ** bypass the completion queue. */ if (((user_service->message_available_pos - instance->completion_remove) >= 0) || user_service->dequeue_pending) { - DEBUG_TRACE(SERVICE_CALLBACK_LINE); user_service->dequeue_pending = 0; - return VCHIQ_SUCCESS; + skip_completion = 1; } + spin_unlock(&msg_queue_spinlock); + + up(&user_service->insert_event); + header = NULL; } + + if (skip_completion) { + DEBUG_TRACE(SERVICE_CALLBACK_LINE); + return VCHIQ_SUCCESS; + } + DEBUG_TRACE(SERVICE_CALLBACK_LINE); return add_completion(instance, reason, header, user_service, bulk_userdata); } /**************************************************************************** * * user_service_free * ***************************************************************************/ static void user_service_free(void *userdata) { USER_SERVICE_T *user_service = userdata; _sema_destroy(&user_service->insert_event); _sema_destroy(&user_service->remove_event); kfree(user_service); } /**************************************************************************** * * close_delivered * ***************************************************************************/ static void close_delivered(USER_SERVICE_T *user_service) { vchiq_log_info(vchiq_arm_log_level, "close_delivered(handle=%x)", user_service->service->handle); if (user_service->close_pending) { /* Allow the underlying service to be culled */ unlock_service(user_service->service); /* Wake the user-thread blocked in close_ or remove_service */ up(&user_service->close_event); user_service->close_pending = 0; } } /**************************************************************************** * * vchiq_ioctl * ***************************************************************************/ static int vchiq_ioctl(struct cdev *cdev, u_long cmd, caddr_t arg, int fflag, struct thread *td) { VCHIQ_INSTANCE_T instance; VCHIQ_STATUS_T status = VCHIQ_SUCCESS; VCHIQ_SERVICE_T *service = NULL; int ret = 0; int i, rc; DEBUG_INITIALISE(g_state.local) if ((ret = devfs_get_cdevpriv((void**)&instance))) { printf("vchiq_ioctl: devfs_get_cdevpriv failed: error %d\n", ret); return (ret); } /* XXXBSD: HACK! */ #define _IOC_NR(x) ((x) & 0xff) #define _IOC_TYPE(x) IOCGROUP(x) vchiq_log_trace(vchiq_arm_log_level, "vchiq_ioctl - instance %x, cmd %s, arg %p", (unsigned int)instance, ((_IOC_TYPE(cmd) == VCHIQ_IOC_MAGIC) && (_IOC_NR(cmd) <= VCHIQ_IOC_MAX)) ? ioctl_names[_IOC_NR(cmd)] : "", arg); switch (cmd) { case VCHIQ_IOC_SHUTDOWN: if (!instance->connected) break; /* Remove all services */ i = 0; while ((service = next_service_by_instance(instance->state, instance, &i)) != NULL) { status = vchiq_remove_service(service->handle); unlock_service(service); if (status != VCHIQ_SUCCESS) break; } service = NULL; if (status == VCHIQ_SUCCESS) { /* Wake the completion thread and ask it to exit */ instance->closing = 1; up(&instance->insert_event); } break; case VCHIQ_IOC_CONNECT: if (instance->connected) { ret = -EINVAL; break; } rc = lmutex_lock_interruptible(&instance->state->mutex); if (rc != 0) { vchiq_log_error(vchiq_arm_log_level, "vchiq: connect: could not lock mutex for " "state %d: %d", instance->state->id, rc); ret = -EINTR; break; } status = vchiq_connect_internal(instance->state, instance); lmutex_unlock(&instance->state->mutex); if (status == VCHIQ_SUCCESS) instance->connected = 1; else vchiq_log_error(vchiq_arm_log_level, "vchiq: could not connect: %d", status); break; case VCHIQ_IOC_CREATE_SERVICE: { VCHIQ_CREATE_SERVICE_T args; USER_SERVICE_T *user_service = NULL; void *userdata; int srvstate; memcpy(&args, (const void*)arg, sizeof(args)); user_service = kmalloc(sizeof(USER_SERVICE_T), GFP_KERNEL); if (!user_service) { ret = -ENOMEM; break; } if (args.is_open) { if (!instance->connected) { ret = -ENOTCONN; kfree(user_service); break; } srvstate = VCHIQ_SRVSTATE_OPENING; } else { srvstate = instance->connected ? VCHIQ_SRVSTATE_LISTENING : VCHIQ_SRVSTATE_HIDDEN; } userdata = args.params.userdata; args.params.callback = service_callback; args.params.userdata = user_service; service = vchiq_add_service_internal( instance->state, &args.params, srvstate, instance, user_service_free); if (service != NULL) { user_service->service = service; user_service->userdata = userdata; user_service->instance = instance; user_service->is_vchi = (args.is_vchi != 0); user_service->dequeue_pending = 0; user_service->close_pending = 0; user_service->message_available_pos = instance->completion_remove - 1; user_service->msg_insert = 0; user_service->msg_remove = 0; _sema_init(&user_service->insert_event, 0); _sema_init(&user_service->remove_event, 0); _sema_init(&user_service->close_event, 0); if (args.is_open) { status = vchiq_open_service_internal (service, instance->pid); if (status != VCHIQ_SUCCESS) { vchiq_remove_service(service->handle); service = NULL; ret = (status == VCHIQ_RETRY) ? -EINTR : -EIO; break; } } #ifdef VCHIQ_IOCTL_DEBUG printf("%s: [CREATE SERVICE] handle = %08x\n", __func__, service->handle); #endif memcpy((void *) &(((VCHIQ_CREATE_SERVICE_T*) arg)->handle), (const void *)&service->handle, sizeof(service->handle)); service = NULL; } else { ret = -EEXIST; kfree(user_service); } } break; case VCHIQ_IOC_CLOSE_SERVICE: { VCHIQ_SERVICE_HANDLE_T handle; memcpy(&handle, (const void*)arg, sizeof(handle)); #ifdef VCHIQ_IOCTL_DEBUG printf("%s: [CLOSE SERVICE] handle = %08x\n", __func__, handle); #endif service = find_service_for_instance(instance, handle); if (service != NULL) { USER_SERVICE_T *user_service = (USER_SERVICE_T *)service->base.userdata; /* close_pending is false on first entry, and when the wait in vchiq_close_service has been interrupted. */ if (!user_service->close_pending) { status = vchiq_close_service(service->handle); if (status != VCHIQ_SUCCESS) break; } /* close_pending is true once the underlying service has been closed until the client library calls the CLOSE_DELIVERED ioctl, signalling close_event. */ if (user_service->close_pending && down_interruptible(&user_service->close_event)) status = VCHIQ_RETRY; } else ret = -EINVAL; } break; case VCHIQ_IOC_REMOVE_SERVICE: { VCHIQ_SERVICE_HANDLE_T handle; memcpy(&handle, (const void*)arg, sizeof(handle)); #ifdef VCHIQ_IOCTL_DEBUG printf("%s: [REMOVE SERVICE] handle = %08x\n", __func__, handle); #endif service = find_service_for_instance(instance, handle); if (service != NULL) { USER_SERVICE_T *user_service = (USER_SERVICE_T *)service->base.userdata; /* close_pending is false on first entry, and when the wait in vchiq_close_service has been interrupted. */ if (!user_service->close_pending) { status = vchiq_remove_service(service->handle); if (status != VCHIQ_SUCCESS) break; } /* close_pending is true once the underlying service has been closed until the client library calls the CLOSE_DELIVERED ioctl, signalling close_event. */ if (user_service->close_pending && down_interruptible(&user_service->close_event)) status = VCHIQ_RETRY; } else ret = -EINVAL; } break; case VCHIQ_IOC_USE_SERVICE: case VCHIQ_IOC_RELEASE_SERVICE: { VCHIQ_SERVICE_HANDLE_T handle; memcpy(&handle, (const void*)arg, sizeof(handle)); #ifdef VCHIQ_IOCTL_DEBUG printf("%s: [%s SERVICE] handle = %08x\n", __func__, cmd == VCHIQ_IOC_USE_SERVICE ? "USE" : "RELEASE", handle); #endif service = find_service_for_instance(instance, handle); if (service != NULL) { status = (cmd == VCHIQ_IOC_USE_SERVICE) ? vchiq_use_service_internal(service) : vchiq_release_service_internal(service); if (status != VCHIQ_SUCCESS) { vchiq_log_error(vchiq_susp_log_level, "%s: cmd %s returned error %d for " "service %c%c%c%c:%8x", __func__, (cmd == VCHIQ_IOC_USE_SERVICE) ? "VCHIQ_IOC_USE_SERVICE" : "VCHIQ_IOC_RELEASE_SERVICE", status, VCHIQ_FOURCC_AS_4CHARS( service->base.fourcc), service->client_id); ret = -EINVAL; } } else ret = -EINVAL; } break; case VCHIQ_IOC_QUEUE_MESSAGE: { VCHIQ_QUEUE_MESSAGE_T args; memcpy(&args, (const void*)arg, sizeof(args)); #ifdef VCHIQ_IOCTL_DEBUG printf("%s: [QUEUE MESSAGE] handle = %08x\n", __func__, args.handle); #endif service = find_service_for_instance(instance, args.handle); if ((service != NULL) && (args.count <= MAX_ELEMENTS)) { /* Copy elements into kernel space */ VCHIQ_ELEMENT_T elements[MAX_ELEMENTS]; if (copy_from_user(elements, args.elements, args.count * sizeof(VCHIQ_ELEMENT_T)) == 0) status = vchiq_queue_message (args.handle, elements, args.count); else ret = -EFAULT; } else { ret = -EINVAL; } } break; case VCHIQ_IOC_QUEUE_BULK_TRANSMIT: case VCHIQ_IOC_QUEUE_BULK_RECEIVE: { VCHIQ_QUEUE_BULK_TRANSFER_T args; struct bulk_waiter_node *waiter = NULL; VCHIQ_BULK_DIR_T dir = (cmd == VCHIQ_IOC_QUEUE_BULK_TRANSMIT) ? VCHIQ_BULK_TRANSMIT : VCHIQ_BULK_RECEIVE; memcpy(&args, (const void*)arg, sizeof(args)); service = find_service_for_instance(instance, args.handle); if (!service) { ret = -EINVAL; break; } if (args.mode == VCHIQ_BULK_MODE_BLOCKING) { waiter = kzalloc(sizeof(struct bulk_waiter_node), GFP_KERNEL); if (!waiter) { ret = -ENOMEM; break; } args.userdata = &waiter->bulk_waiter; } else if (args.mode == VCHIQ_BULK_MODE_WAITING) { struct list_head *pos; lmutex_lock(&instance->bulk_waiter_list_mutex); list_for_each(pos, &instance->bulk_waiter_list) { if (list_entry(pos, struct bulk_waiter_node, list)->pid == current->p_pid) { waiter = list_entry(pos, struct bulk_waiter_node, list); list_del(pos); break; } } lmutex_unlock(&instance->bulk_waiter_list_mutex); if (!waiter) { vchiq_log_error(vchiq_arm_log_level, "no bulk_waiter found for pid %d", current->p_pid); ret = -ESRCH; break; } vchiq_log_info(vchiq_arm_log_level, "found bulk_waiter %x for pid %d", (unsigned int)waiter, current->p_pid); args.userdata = &waiter->bulk_waiter; } status = vchiq_bulk_transfer (args.handle, VCHI_MEM_HANDLE_INVALID, args.data, args.size, args.userdata, args.mode, dir); if (!waiter) break; if ((status != VCHIQ_RETRY) || fatal_signal_pending(current) || !waiter->bulk_waiter.bulk) { if (waiter->bulk_waiter.bulk) { /* Cancel the signal when the transfer ** completes. */ spin_lock(&bulk_waiter_spinlock); waiter->bulk_waiter.bulk->userdata = NULL; spin_unlock(&bulk_waiter_spinlock); } _sema_destroy(&waiter->bulk_waiter.event); kfree(waiter); } else { const VCHIQ_BULK_MODE_T mode_waiting = VCHIQ_BULK_MODE_WAITING; waiter->pid = current->p_pid; lmutex_lock(&instance->bulk_waiter_list_mutex); list_add(&waiter->list, &instance->bulk_waiter_list); lmutex_unlock(&instance->bulk_waiter_list_mutex); vchiq_log_info(vchiq_arm_log_level, "saved bulk_waiter %x for pid %d", (unsigned int)waiter, current->p_pid); memcpy((void *) &(((VCHIQ_QUEUE_BULK_TRANSFER_T *) arg)->mode), (const void *)&mode_waiting, sizeof(mode_waiting)); } } break; case VCHIQ_IOC_AWAIT_COMPLETION: { VCHIQ_AWAIT_COMPLETION_T args; int count = 0; DEBUG_TRACE(AWAIT_COMPLETION_LINE); if (!instance->connected) { ret = -ENOTCONN; break; } - memcpy(&args, (const void*)arg, sizeof(args)); + memcpy(&args, (const void*)arg, sizeof(args)); lmutex_lock(&instance->completion_mutex); DEBUG_TRACE(AWAIT_COMPLETION_LINE); while ((instance->completion_remove == instance->completion_insert) && !instance->closing) { + DEBUG_TRACE(AWAIT_COMPLETION_LINE); lmutex_unlock(&instance->completion_mutex); rc = down_interruptible(&instance->insert_event); lmutex_lock(&instance->completion_mutex); if (rc != 0) { DEBUG_TRACE(AWAIT_COMPLETION_LINE); vchiq_log_info(vchiq_arm_log_level, "AWAIT_COMPLETION interrupted"); ret = -EINTR; break; } } DEBUG_TRACE(AWAIT_COMPLETION_LINE); - /* A read memory barrier is needed to stop prefetch of a stale - ** completion record - */ - rmb(); - if (ret == 0) { int msgbufcount = args.msgbufcount; + int remove; + + remove = instance->completion_remove; + for (count = 0; count < args.count; count++) { VCHIQ_COMPLETION_DATA_T *completion; VCHIQ_SERVICE_T *service1; USER_SERVICE_T *user_service; VCHIQ_HEADER_T *header; - if (instance->completion_remove == - instance->completion_insert) + + if (remove == instance->completion_insert) break; + completion = &instance->completions[ - instance->completion_remove & - (MAX_COMPLETIONS - 1)]; + remove & (MAX_COMPLETIONS - 1)]; + + /* A read memory barrier is needed to prevent + ** the prefetch of a stale completion record + */ + rmb(); + service1 = completion->service_userdata; user_service = service1->base.userdata; completion->service_userdata = user_service->userdata; header = completion->header; if (header) { void __user *msgbuf; int msglen; msglen = header->size + sizeof(VCHIQ_HEADER_T); /* This must be a VCHIQ-style service */ if (args.msgbufsize < msglen) { vchiq_log_error( vchiq_arm_log_level, "header %x: msgbufsize" " %x < msglen %x", (unsigned int)header, args.msgbufsize, msglen); WARN(1, "invalid message " "size\n"); if (count == 0) ret = -EMSGSIZE; break; } if (msgbufcount <= 0) /* Stall here for lack of a ** buffer for the message. */ break; /* Get the pointer from user space */ msgbufcount--; if (copy_from_user(&msgbuf, (const void __user *) &args.msgbufs[msgbufcount], sizeof(msgbuf)) != 0) { if (count == 0) ret = -EFAULT; break; } /* Copy the message to user space */ if (copy_to_user(msgbuf, header, msglen) != 0) { if (count == 0) ret = -EFAULT; break; } /* Now it has been copied, the message ** can be released. */ vchiq_release_message(service1->handle, header); /* The completion must point to the ** msgbuf. */ completion->header = msgbuf; } if ((completion->reason == VCHIQ_SERVICE_CLOSED) && !instance->use_close_delivered) unlock_service(service1); if (copy_to_user((void __user *)( (size_t)args.buf + count * sizeof(VCHIQ_COMPLETION_DATA_T)), completion, sizeof(VCHIQ_COMPLETION_DATA_T)) != 0) { if (ret == 0) ret = -EFAULT; break; } - instance->completion_remove++; + /* Ensure that the above copy has completed + ** before advancing the remove pointer. */ + mb(); + + instance->completion_remove = ++remove; } if (msgbufcount != args.msgbufcount) { memcpy((void __user *) &((VCHIQ_AWAIT_COMPLETION_T *)arg)-> msgbufcount, &msgbufcount, sizeof(msgbufcount)); } if (count != args.count) { memcpy((void __user *) &((VCHIQ_AWAIT_COMPLETION_T *)arg)->count, &count, sizeof(count)); } } if (count != 0) up(&instance->remove_event); if ((ret == 0) && instance->closing) ret = -ENOTCONN; /* * XXXBSD: ioctl return codes are not negative as in linux, so * we can not indicate success with positive number of passed * messages */ if (ret > 0) ret = 0; lmutex_unlock(&instance->completion_mutex); DEBUG_TRACE(AWAIT_COMPLETION_LINE); } break; case VCHIQ_IOC_DEQUEUE_MESSAGE: { VCHIQ_DEQUEUE_MESSAGE_T args; USER_SERVICE_T *user_service; VCHIQ_HEADER_T *header; DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); memcpy(&args, (const void*)arg, sizeof(args)); service = find_service_for_instance(instance, args.handle); if (!service) { ret = -EINVAL; break; } user_service = (USER_SERVICE_T *)service->base.userdata; if (user_service->is_vchi == 0) { ret = -EINVAL; break; } spin_lock(&msg_queue_spinlock); if (user_service->msg_remove == user_service->msg_insert) { if (!args.blocking) { spin_unlock(&msg_queue_spinlock); DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); ret = -EWOULDBLOCK; break; } user_service->dequeue_pending = 1; do { spin_unlock(&msg_queue_spinlock); DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); if (down_interruptible( &user_service->insert_event) != 0) { vchiq_log_info(vchiq_arm_log_level, "DEQUEUE_MESSAGE interrupted"); ret = -EINTR; break; } spin_lock(&msg_queue_spinlock); } while (user_service->msg_remove == user_service->msg_insert); if (ret) break; } BUG_ON((int)(user_service->msg_insert - user_service->msg_remove) < 0); header = user_service->msg_queue[user_service->msg_remove & (MSG_QUEUE_SIZE - 1)]; user_service->msg_remove++; spin_unlock(&msg_queue_spinlock); up(&user_service->remove_event); if (header == NULL) ret = -ENOTCONN; else if (header->size <= args.bufsize) { /* Copy to user space if msgbuf is not NULL */ if ((args.buf == NULL) || (copy_to_user((void __user *)args.buf, header->data, header->size) == 0)) { args.bufsize = header->size; memcpy((void *)arg, &args, sizeof(args)); vchiq_release_message( service->handle, header); } else ret = -EFAULT; } else { vchiq_log_error(vchiq_arm_log_level, "header %x: bufsize %x < size %x", (unsigned int)header, args.bufsize, header->size); WARN(1, "invalid size\n"); ret = -EMSGSIZE; } DEBUG_TRACE(DEQUEUE_MESSAGE_LINE); } break; case VCHIQ_IOC_GET_CLIENT_ID: { VCHIQ_SERVICE_HANDLE_T handle; memcpy(&handle, (const void*)arg, sizeof(handle)); ret = vchiq_get_client_id(handle); } break; case VCHIQ_IOC_GET_CONFIG: { VCHIQ_GET_CONFIG_T args; VCHIQ_CONFIG_T config; memcpy(&args, (const void*)arg, sizeof(args)); if (args.config_size > sizeof(config)) { ret = -EINVAL; break; } status = vchiq_get_config(instance, args.config_size, &config); if (status == VCHIQ_SUCCESS) { if (copy_to_user((void __user *)args.pconfig, &config, args.config_size) != 0) { ret = -EFAULT; break; } } } break; case VCHIQ_IOC_SET_SERVICE_OPTION: { VCHIQ_SET_SERVICE_OPTION_T args; memcpy(&args, (const void*)arg, sizeof(args)); service = find_service_for_instance(instance, args.handle); if (!service) { ret = -EINVAL; break; } status = vchiq_set_service_option( args.handle, args.option, args.value); } break; case VCHIQ_IOC_DUMP_PHYS_MEM: { VCHIQ_DUMP_MEM_T args; memcpy(&args, (const void*)arg, sizeof(args)); printf("IMPLEMENT ME: %s:%d\n", __FILE__, __LINE__); #if 0 dump_phys_mem(args.virt_addr, args.num_bytes); #endif } break; case VCHIQ_IOC_LIB_VERSION: { unsigned int lib_version = (unsigned int)arg; if (lib_version < VCHIQ_VERSION_MIN) ret = -EINVAL; else if (lib_version >= VCHIQ_VERSION_CLOSE_DELIVERED) instance->use_close_delivered = 1; } break; case VCHIQ_IOC_CLOSE_DELIVERED: { VCHIQ_SERVICE_HANDLE_T handle; memcpy(&handle, (const void*)arg, sizeof(handle)); service = find_closed_service_for_instance(instance, handle); if (service != NULL) { USER_SERVICE_T *user_service = (USER_SERVICE_T *)service->base.userdata; close_delivered(user_service); } else ret = -EINVAL; } break; default: ret = -ENOTTY; break; } if (service) unlock_service(service); if (ret == 0) { if (status == VCHIQ_ERROR) ret = -EIO; else if (status == VCHIQ_RETRY) ret = -EINTR; } if ((status == VCHIQ_SUCCESS) && (ret < 0) && (ret != -EINTR) && (ret != -EWOULDBLOCK)) vchiq_log_info(vchiq_arm_log_level, " ioctl instance %lx, cmd %s -> status %d, %d", (unsigned long)instance, (_IOC_NR(cmd) <= VCHIQ_IOC_MAX) ? ioctl_names[_IOC_NR(cmd)] : "", status, ret); else vchiq_log_trace(vchiq_arm_log_level, " ioctl instance %lx, cmd %s -> status %d, %d", (unsigned long)instance, (_IOC_NR(cmd) <= VCHIQ_IOC_MAX) ? ioctl_names[_IOC_NR(cmd)] : "", status, ret); /* XXXBSD: report BSD-style error to userland */ if (ret < 0) ret = -ret; return ret; } static void instance_dtr(void *data) { kfree(data); } /**************************************************************************** * * vchiq_open * ***************************************************************************/ static int vchiq_open(struct cdev *dev, int flags, int fmt __unused, struct thread *td) { vchiq_log_info(vchiq_arm_log_level, "vchiq_open"); /* XXXBSD: do we really need this check? */ if (1) { VCHIQ_STATE_T *state = vchiq_get_state(); VCHIQ_INSTANCE_T instance; if (!state) { vchiq_log_error(vchiq_arm_log_level, "vchiq has no connection to VideoCore"); return -ENOTCONN; } instance = kmalloc(sizeof(*instance), GFP_KERNEL); if (!instance) return -ENOMEM; instance->state = state; /* XXXBSD: PID or thread ID? */ instance->pid = td->td_proc->p_pid; #ifdef notyet ret = vchiq_proc_add_instance(instance); if (ret != 0) { kfree(instance); return ret; } #endif _sema_init(&instance->insert_event, 0); _sema_init(&instance->remove_event, 0); lmutex_init(&instance->completion_mutex); lmutex_init(&instance->bulk_waiter_list_mutex); INIT_LIST_HEAD(&instance->bulk_waiter_list); devfs_set_cdevpriv(instance, instance_dtr); } else { vchiq_log_error(vchiq_arm_log_level, "Unknown minor device"); return -ENXIO; } return 0; } /**************************************************************************** * * vchiq_release * ***************************************************************************/ static int vchiq_close(struct cdev *dev, int flags __unused, int fmt __unused, struct thread *td) { int ret = 0; if (1) { VCHIQ_INSTANCE_T instance; VCHIQ_STATE_T *state = vchiq_get_state(); VCHIQ_SERVICE_T *service; int i; if ((ret = devfs_get_cdevpriv((void**)&instance))) { printf("devfs_get_cdevpriv failed: error %d\n", ret); return (ret); } vchiq_log_info(vchiq_arm_log_level, "vchiq_release: instance=%lx", (unsigned long)instance); if (!state) { ret = -EPERM; goto out; } /* Ensure videocore is awake to allow termination. */ vchiq_use_internal(instance->state, NULL, USE_TYPE_VCHIQ); lmutex_lock(&instance->completion_mutex); /* Wake the completion thread and ask it to exit */ instance->closing = 1; up(&instance->insert_event); lmutex_unlock(&instance->completion_mutex); /* Wake the slot handler if the completion queue is full. */ up(&instance->remove_event); /* Mark all services for termination... */ i = 0; while ((service = next_service_by_instance(state, instance, &i)) != NULL) { USER_SERVICE_T *user_service = service->base.userdata; /* Wake the slot handler if the msg queue is full. */ up(&user_service->remove_event); vchiq_terminate_service_internal(service); unlock_service(service); } /* ...and wait for them to die */ i = 0; while ((service = next_service_by_instance(state, instance, &i)) != NULL) { USER_SERVICE_T *user_service = service->base.userdata; down(&service->remove_event); BUG_ON(service->srvstate != VCHIQ_SRVSTATE_FREE); spin_lock(&msg_queue_spinlock); while (user_service->msg_remove != user_service->msg_insert) { VCHIQ_HEADER_T *header = user_service-> msg_queue[user_service->msg_remove & (MSG_QUEUE_SIZE - 1)]; user_service->msg_remove++; spin_unlock(&msg_queue_spinlock); if (header) vchiq_release_message( service->handle, header); spin_lock(&msg_queue_spinlock); } spin_unlock(&msg_queue_spinlock); unlock_service(service); } /* Release any closed services */ while (instance->completion_remove != instance->completion_insert) { VCHIQ_COMPLETION_DATA_T *completion; VCHIQ_SERVICE_T *service1; completion = &instance->completions[ instance->completion_remove & (MAX_COMPLETIONS - 1)]; service1 = completion->service_userdata; if (completion->reason == VCHIQ_SERVICE_CLOSED) { USER_SERVICE_T *user_service = service->base.userdata; /* Wake any blocked user-thread */ if (instance->use_close_delivered) up(&user_service->close_event); unlock_service(service1); } instance->completion_remove++; } /* Release the PEER service count. */ vchiq_release_internal(instance->state, NULL); { struct list_head *pos, *next; list_for_each_safe(pos, next, &instance->bulk_waiter_list) { struct bulk_waiter_node *waiter; waiter = list_entry(pos, struct bulk_waiter_node, list); list_del(pos); vchiq_log_info(vchiq_arm_log_level, "bulk_waiter - cleaned up %x " "for pid %d", (unsigned int)waiter, waiter->pid); _sema_destroy(&waiter->bulk_waiter.event); kfree(waiter); } } } else { vchiq_log_error(vchiq_arm_log_level, "Unknown minor device"); ret = -ENXIO; } out: return ret; } /**************************************************************************** * * vchiq_dump * ***************************************************************************/ void vchiq_dump(void *dump_context, const char *str, int len) { DUMP_CONTEXT_T *context = (DUMP_CONTEXT_T *)dump_context; if (context->actual < context->space) { int copy_bytes; if (context->offset > 0) { int skip_bytes = min(len, (int)context->offset); str += skip_bytes; len -= skip_bytes; context->offset -= skip_bytes; if (context->offset > 0) return; } copy_bytes = min(len, (int)(context->space - context->actual)); if (copy_bytes == 0) return; memcpy(context->buf + context->actual, str, copy_bytes); context->actual += copy_bytes; len -= copy_bytes; /* If tne terminating NUL is included in the length, then it ** marks the end of a line and should be replaced with a ** carriage return. */ if ((len == 0) && (str[copy_bytes - 1] == '\0')) { char cr = '\n'; memcpy(context->buf + context->actual - 1, &cr, 1); } } } /**************************************************************************** * * vchiq_dump_platform_instance_state * ***************************************************************************/ void vchiq_dump_platform_instances(void *dump_context) { VCHIQ_STATE_T *state = vchiq_get_state(); char buf[80]; int len; int i; /* There is no list of instances, so instead scan all services, marking those that have been dumped. */ for (i = 0; i < state->unused_service; i++) { VCHIQ_SERVICE_T *service = state->services[i]; VCHIQ_INSTANCE_T instance; if (service && (service->base.callback == service_callback)) { instance = service->instance; if (instance) instance->mark = 0; } } for (i = 0; i < state->unused_service; i++) { VCHIQ_SERVICE_T *service = state->services[i]; VCHIQ_INSTANCE_T instance; if (service && (service->base.callback == service_callback)) { instance = service->instance; if (instance && !instance->mark) { len = snprintf(buf, sizeof(buf), "Instance %x: pid %d,%s completions " "%d/%d", (unsigned int)instance, instance->pid, instance->connected ? " connected, " : "", instance->completion_insert - instance->completion_remove, MAX_COMPLETIONS); vchiq_dump(dump_context, buf, len + 1); instance->mark = 1; } } } } /**************************************************************************** * * vchiq_dump_platform_service_state * ***************************************************************************/ void vchiq_dump_platform_service_state(void *dump_context, VCHIQ_SERVICE_T *service) { USER_SERVICE_T *user_service = (USER_SERVICE_T *)service->base.userdata; char buf[80]; int len; len = snprintf(buf, sizeof(buf), " instance %x", (unsigned int)service->instance); if ((service->base.callback == service_callback) && user_service->is_vchi) { len += snprintf(buf + len, sizeof(buf) - len, ", %d/%d messages", user_service->msg_insert - user_service->msg_remove, MSG_QUEUE_SIZE); if (user_service->dequeue_pending) len += snprintf(buf + len, sizeof(buf) - len, " (dequeue pending)"); } vchiq_dump(dump_context, buf, len + 1); } #ifdef notyet /**************************************************************************** * * dump_user_mem * ***************************************************************************/ static void dump_phys_mem(void *virt_addr, uint32_t num_bytes) { int rc; uint8_t *end_virt_addr = virt_addr + num_bytes; int num_pages; int offset; int end_offset; int page_idx; int prev_idx; struct page *page; struct page **pages; uint8_t *kmapped_virt_ptr; /* Align virtAddr and endVirtAddr to 16 byte boundaries. */ virt_addr = (void *)((unsigned long)virt_addr & ~0x0fuL); end_virt_addr = (void *)(((unsigned long)end_virt_addr + 15uL) & ~0x0fuL); offset = (int)(long)virt_addr & (PAGE_SIZE - 1); end_offset = (int)(long)end_virt_addr & (PAGE_SIZE - 1); num_pages = (offset + num_bytes + PAGE_SIZE - 1) / PAGE_SIZE; pages = kmalloc(sizeof(struct page *) * num_pages, GFP_KERNEL); if (pages == NULL) { vchiq_log_error(vchiq_arm_log_level, "Unable to allocation memory for %d pages\n", num_pages); return; } down_read(¤t->mm->mmap_sem); rc = get_user_pages(current, /* task */ current->mm, /* mm */ (unsigned long)virt_addr, /* start */ num_pages, /* len */ 0, /* write */ 0, /* force */ pages, /* pages (array of page pointers) */ NULL); /* vmas */ up_read(¤t->mm->mmap_sem); prev_idx = -1; page = NULL; while (offset < end_offset) { int page_offset = offset % PAGE_SIZE; page_idx = offset / PAGE_SIZE; if (page_idx != prev_idx) { if (page != NULL) kunmap(page); page = pages[page_idx]; kmapped_virt_ptr = kmap(page); prev_idx = page_idx; } if (vchiq_arm_log_level >= VCHIQ_LOG_TRACE) vchiq_log_dump_mem("ph", (uint32_t)(unsigned long)&kmapped_virt_ptr[ page_offset], &kmapped_virt_ptr[page_offset], 16); offset += 16; } if (page != NULL) kunmap(page); for (page_idx = 0; page_idx < num_pages; page_idx++) page_cache_release(pages[page_idx]); kfree(pages); } /**************************************************************************** * * vchiq_read * ***************************************************************************/ static ssize_t vchiq_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { DUMP_CONTEXT_T context; context.buf = buf; context.actual = 0; context.space = count; context.offset = *ppos; vchiq_dump_state(&context, &g_state); *ppos += context.actual; return context.actual; } #endif VCHIQ_STATE_T * vchiq_get_state(void) { if (g_state.remote == NULL) printk(KERN_ERR "%s: g_state.remote == NULL\n", __func__); else if (g_state.remote->initialised != 1) printk(KERN_NOTICE "%s: g_state.remote->initialised != 1 (%d)\n", __func__, g_state.remote->initialised); return ((g_state.remote != NULL) && (g_state.remote->initialised == 1)) ? &g_state : NULL; } /* * Autosuspend related functionality */ int vchiq_videocore_wanted(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); if (!arm_state) /* autosuspend not supported - always return wanted */ return 1; else if (arm_state->blocked_count) return 1; else if (!arm_state->videocore_use_count) /* usage count zero - check for override unless we're forcing */ if (arm_state->resume_blocked) return 0; else return vchiq_platform_videocore_wanted(state); else /* non-zero usage count - videocore still required */ return 1; } static VCHIQ_STATUS_T vchiq_keepalive_vchiq_callback(VCHIQ_REASON_T reason, VCHIQ_HEADER_T *header, VCHIQ_SERVICE_HANDLE_T service_user, void *bulk_user) { vchiq_log_error(vchiq_susp_log_level, "%s callback reason %d", __func__, reason); return 0; } static int vchiq_keepalive_thread_func(void *v) { VCHIQ_STATE_T *state = (VCHIQ_STATE_T *) v; VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); VCHIQ_STATUS_T status; VCHIQ_INSTANCE_T instance; VCHIQ_SERVICE_HANDLE_T ka_handle; VCHIQ_SERVICE_PARAMS_T params = { .fourcc = VCHIQ_MAKE_FOURCC('K', 'E', 'E', 'P'), .callback = vchiq_keepalive_vchiq_callback, .version = KEEPALIVE_VER, .version_min = KEEPALIVE_VER_MIN }; status = vchiq_initialise(&instance); if (status != VCHIQ_SUCCESS) { vchiq_log_error(vchiq_susp_log_level, "%s vchiq_initialise failed %d", __func__, status); goto exit; } status = vchiq_connect(instance); if (status != VCHIQ_SUCCESS) { vchiq_log_error(vchiq_susp_log_level, "%s vchiq_connect failed %d", __func__, status); goto shutdown; } status = vchiq_add_service(instance, ¶ms, &ka_handle); if (status != VCHIQ_SUCCESS) { vchiq_log_error(vchiq_susp_log_level, "%s vchiq_open_service failed %d", __func__, status); goto shutdown; } while (1) { long rc = 0, uc = 0; if (wait_for_completion_interruptible(&arm_state->ka_evt) != 0) { vchiq_log_error(vchiq_susp_log_level, "%s interrupted", __func__); flush_signals(current); continue; } /* read and clear counters. Do release_count then use_count to * prevent getting more releases than uses */ rc = atomic_xchg(&arm_state->ka_release_count, 0); uc = atomic_xchg(&arm_state->ka_use_count, 0); /* Call use/release service the requisite number of times. * Process use before release so use counts don't go negative */ while (uc--) { atomic_inc(&arm_state->ka_use_ack_count); status = vchiq_use_service(ka_handle); if (status != VCHIQ_SUCCESS) { vchiq_log_error(vchiq_susp_log_level, "%s vchiq_use_service error %d", __func__, status); } } while (rc--) { status = vchiq_release_service(ka_handle); if (status != VCHIQ_SUCCESS) { vchiq_log_error(vchiq_susp_log_level, "%s vchiq_release_service error %d", __func__, status); } } } shutdown: vchiq_shutdown(instance); exit: return 0; } VCHIQ_STATUS_T vchiq_arm_init_state(VCHIQ_STATE_T *state, VCHIQ_ARM_STATE_T *arm_state) { VCHIQ_STATUS_T status = VCHIQ_SUCCESS; if (arm_state) { rwlock_init(&arm_state->susp_res_lock); init_completion(&arm_state->ka_evt); atomic_set(&arm_state->ka_use_count, 0); atomic_set(&arm_state->ka_use_ack_count, 0); atomic_set(&arm_state->ka_release_count, 0); init_completion(&arm_state->vc_suspend_complete); init_completion(&arm_state->vc_resume_complete); /* Initialise to 'done' state. We only want to block on resume * completion while videocore is suspended. */ set_resume_state(arm_state, VC_RESUME_RESUMED); init_completion(&arm_state->resume_blocker); /* Initialise to 'done' state. We only want to block on this * completion while resume is blocked */ complete_all(&arm_state->resume_blocker); init_completion(&arm_state->blocked_blocker); /* Initialise to 'done' state. We only want to block on this * completion while things are waiting on the resume blocker */ complete_all(&arm_state->blocked_blocker); arm_state->suspend_timer_timeout = SUSPEND_TIMER_TIMEOUT_MS; arm_state->suspend_timer_running = 0; init_timer(&arm_state->suspend_timer); arm_state->suspend_timer.data = (unsigned long)(state); arm_state->suspend_timer.function = suspend_timer_callback; arm_state->first_connect = 0; } return status; } /* ** Functions to modify the state variables; ** set_suspend_state ** set_resume_state ** ** There are more state variables than we might like, so ensure they remain in ** step. Suspend and resume state are maintained separately, since most of ** these state machines can operate independently. However, there are a few ** states where state transitions in one state machine cause a reset to the ** other state machine. In addition, there are some completion events which ** need to occur on state machine reset and end-state(s), so these are also ** dealt with in these functions. ** ** In all states we set the state variable according to the input, but in some ** cases we perform additional steps outlined below; ** ** VC_SUSPEND_IDLE - Initialise the suspend completion at the same time. ** The suspend completion is completed after any suspend ** attempt. When we reset the state machine we also reset ** the completion. This reset occurs when videocore is ** resumed, and also if we initiate suspend after a suspend ** failure. ** ** VC_SUSPEND_IN_PROGRESS - This state is considered the point of no return for ** suspend - ie from this point on we must try to suspend ** before resuming can occur. We therefore also reset the ** resume state machine to VC_RESUME_IDLE in this state. ** ** VC_SUSPEND_SUSPENDED - Suspend has completed successfully. Also call ** complete_all on the suspend completion to notify ** anything waiting for suspend to happen. ** ** VC_SUSPEND_REJECTED - Videocore rejected suspend. Videocore will also ** initiate resume, so no need to alter resume state. ** We call complete_all on the suspend completion to notify ** of suspend rejection. ** ** VC_SUSPEND_FAILED - We failed to initiate videocore suspend. We notify the ** suspend completion and reset the resume state machine. ** ** VC_RESUME_IDLE - Initialise the resume completion at the same time. The ** resume completion is in it's 'done' state whenever ** videcore is running. Therfore, the VC_RESUME_IDLE state ** implies that videocore is suspended. ** Hence, any thread which needs to wait until videocore is ** running can wait on this completion - it will only block ** if videocore is suspended. ** ** VC_RESUME_RESUMED - Resume has completed successfully. Videocore is running. ** Call complete_all on the resume completion to unblock ** any threads waiting for resume. Also reset the suspend ** state machine to it's idle state. ** ** VC_RESUME_FAILED - Currently unused - no mechanism to fail resume exists. */ void set_suspend_state(VCHIQ_ARM_STATE_T *arm_state, enum vc_suspend_status new_state) { /* set the state in all cases */ arm_state->vc_suspend_state = new_state; /* state specific additional actions */ switch (new_state) { case VC_SUSPEND_FORCE_CANCELED: complete_all(&arm_state->vc_suspend_complete); break; case VC_SUSPEND_REJECTED: complete_all(&arm_state->vc_suspend_complete); break; case VC_SUSPEND_FAILED: complete_all(&arm_state->vc_suspend_complete); arm_state->vc_resume_state = VC_RESUME_RESUMED; complete_all(&arm_state->vc_resume_complete); break; case VC_SUSPEND_IDLE: /* TODO: reinit_completion */ INIT_COMPLETION(arm_state->vc_suspend_complete); break; case VC_SUSPEND_REQUESTED: break; case VC_SUSPEND_IN_PROGRESS: set_resume_state(arm_state, VC_RESUME_IDLE); break; case VC_SUSPEND_SUSPENDED: complete_all(&arm_state->vc_suspend_complete); break; default: BUG(); break; } } void set_resume_state(VCHIQ_ARM_STATE_T *arm_state, enum vc_resume_status new_state) { /* set the state in all cases */ arm_state->vc_resume_state = new_state; /* state specific additional actions */ switch (new_state) { case VC_RESUME_FAILED: break; case VC_RESUME_IDLE: /* TODO: reinit_completion */ INIT_COMPLETION(arm_state->vc_resume_complete); break; case VC_RESUME_REQUESTED: break; case VC_RESUME_IN_PROGRESS: break; case VC_RESUME_RESUMED: complete_all(&arm_state->vc_resume_complete); set_suspend_state(arm_state, VC_SUSPEND_IDLE); break; default: BUG(); break; } } /* should be called with the write lock held */ inline void start_suspend_timer(VCHIQ_ARM_STATE_T *arm_state) { del_timer(&arm_state->suspend_timer); arm_state->suspend_timer.expires = jiffies + msecs_to_jiffies(arm_state-> suspend_timer_timeout); add_timer(&arm_state->suspend_timer); arm_state->suspend_timer_running = 1; } /* should be called with the write lock held */ static inline void stop_suspend_timer(VCHIQ_ARM_STATE_T *arm_state) { if (arm_state->suspend_timer_running) { del_timer(&arm_state->suspend_timer); arm_state->suspend_timer_running = 0; } } static inline int need_resume(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); return (arm_state->vc_suspend_state > VC_SUSPEND_IDLE) && (arm_state->vc_resume_state < VC_RESUME_REQUESTED) && vchiq_videocore_wanted(state); } static int block_resume(VCHIQ_ARM_STATE_T *arm_state) { int status = VCHIQ_SUCCESS; const unsigned long timeout_val = msecs_to_jiffies(FORCE_SUSPEND_TIMEOUT_MS); int resume_count = 0; /* Allow any threads which were blocked by the last force suspend to * complete if they haven't already. Only give this one shot; if * blocked_count is incremented after blocked_blocker is completed * (which only happens when blocked_count hits 0) then those threads * will have to wait until next time around */ if (arm_state->blocked_count) { /* TODO: reinit_completion */ INIT_COMPLETION(arm_state->blocked_blocker); write_unlock_bh(&arm_state->susp_res_lock); vchiq_log_info(vchiq_susp_log_level, "%s wait for previously " "blocked clients", __func__); if (wait_for_completion_interruptible_timeout( &arm_state->blocked_blocker, timeout_val) <= 0) { vchiq_log_error(vchiq_susp_log_level, "%s wait for " "previously blocked clients failed" , __func__); status = VCHIQ_ERROR; write_lock_bh(&arm_state->susp_res_lock); goto out; } vchiq_log_info(vchiq_susp_log_level, "%s previously blocked " "clients resumed", __func__); write_lock_bh(&arm_state->susp_res_lock); } /* We need to wait for resume to complete if it's in process */ while (arm_state->vc_resume_state != VC_RESUME_RESUMED && arm_state->vc_resume_state > VC_RESUME_IDLE) { if (resume_count > 1) { status = VCHIQ_ERROR; vchiq_log_error(vchiq_susp_log_level, "%s waited too " "many times for resume" , __func__); goto out; } write_unlock_bh(&arm_state->susp_res_lock); vchiq_log_info(vchiq_susp_log_level, "%s wait for resume", __func__); if (wait_for_completion_interruptible_timeout( &arm_state->vc_resume_complete, timeout_val) <= 0) { vchiq_log_error(vchiq_susp_log_level, "%s wait for " "resume failed (%s)", __func__, resume_state_names[arm_state->vc_resume_state + VC_RESUME_NUM_OFFSET]); status = VCHIQ_ERROR; write_lock_bh(&arm_state->susp_res_lock); goto out; } vchiq_log_info(vchiq_susp_log_level, "%s resumed", __func__); write_lock_bh(&arm_state->susp_res_lock); resume_count++; } /* TODO: reinit_completion */ INIT_COMPLETION(arm_state->resume_blocker); arm_state->resume_blocked = 1; out: return status; } static inline void unblock_resume(VCHIQ_ARM_STATE_T *arm_state) { complete_all(&arm_state->resume_blocker); arm_state->resume_blocked = 0; } /* Initiate suspend via slot handler. Should be called with the write lock * held */ VCHIQ_STATUS_T vchiq_arm_vcsuspend(VCHIQ_STATE_T *state) { VCHIQ_STATUS_T status = VCHIQ_ERROR; VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); status = VCHIQ_SUCCESS; switch (arm_state->vc_suspend_state) { case VC_SUSPEND_REQUESTED: vchiq_log_info(vchiq_susp_log_level, "%s: suspend already " "requested", __func__); break; case VC_SUSPEND_IN_PROGRESS: vchiq_log_info(vchiq_susp_log_level, "%s: suspend already in " "progress", __func__); break; default: /* We don't expect to be in other states, so log but continue * anyway */ vchiq_log_error(vchiq_susp_log_level, "%s unexpected suspend state %s", __func__, suspend_state_names[arm_state->vc_suspend_state + VC_SUSPEND_NUM_OFFSET]); /* fall through */ case VC_SUSPEND_REJECTED: case VC_SUSPEND_FAILED: /* Ensure any idle state actions have been run */ set_suspend_state(arm_state, VC_SUSPEND_IDLE); /* fall through */ case VC_SUSPEND_IDLE: vchiq_log_info(vchiq_susp_log_level, "%s: suspending", __func__); set_suspend_state(arm_state, VC_SUSPEND_REQUESTED); /* kick the slot handler thread to initiate suspend */ request_poll(state, NULL, 0); break; } out: vchiq_log_trace(vchiq_susp_log_level, "%s exit %d", __func__, status); return status; } void vchiq_platform_check_suspend(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); int susp = 0; if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); write_lock_bh(&arm_state->susp_res_lock); if (arm_state->vc_suspend_state == VC_SUSPEND_REQUESTED && arm_state->vc_resume_state == VC_RESUME_RESUMED) { set_suspend_state(arm_state, VC_SUSPEND_IN_PROGRESS); susp = 1; } write_unlock_bh(&arm_state->susp_res_lock); if (susp) vchiq_platform_suspend(state); out: vchiq_log_trace(vchiq_susp_log_level, "%s exit", __func__); return; } static void output_timeout_error(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); char service_err[50] = ""; int vc_use_count = arm_state->videocore_use_count; int active_services = state->unused_service; int i; if (!arm_state->videocore_use_count) { snprintf(service_err, 50, " Videocore usecount is 0"); goto output_msg; } for (i = 0; i < active_services; i++) { VCHIQ_SERVICE_T *service_ptr = state->services[i]; if (service_ptr && service_ptr->service_use_count && (service_ptr->srvstate != VCHIQ_SRVSTATE_FREE)) { snprintf(service_err, 50, " %c%c%c%c(%8x) service has " "use count %d%s", VCHIQ_FOURCC_AS_4CHARS( service_ptr->base.fourcc), service_ptr->client_id, service_ptr->service_use_count, service_ptr->service_use_count == vc_use_count ? "" : " (+ more)"); break; } } output_msg: vchiq_log_error(vchiq_susp_log_level, "timed out waiting for vc suspend (%d).%s", arm_state->autosuspend_override, service_err); } /* Try to get videocore into suspended state, regardless of autosuspend state. ** We don't actually force suspend, since videocore may get into a bad state ** if we force suspend at a bad time. Instead, we wait for autosuspend to ** determine a good point to suspend. If this doesn't happen within 100ms we ** report failure. ** ** Returns VCHIQ_SUCCESS if videocore suspended successfully, VCHIQ_RETRY if ** videocore failed to suspend in time or VCHIQ_ERROR if interrupted. */ VCHIQ_STATUS_T vchiq_arm_force_suspend(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); VCHIQ_STATUS_T status = VCHIQ_ERROR; long rc = 0; int repeat = -1; if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); write_lock_bh(&arm_state->susp_res_lock); status = block_resume(arm_state); if (status != VCHIQ_SUCCESS) goto unlock; if (arm_state->vc_suspend_state == VC_SUSPEND_SUSPENDED) { /* Already suspended - just block resume and exit */ vchiq_log_info(vchiq_susp_log_level, "%s already suspended", __func__); status = VCHIQ_SUCCESS; goto unlock; } else if (arm_state->vc_suspend_state <= VC_SUSPEND_IDLE) { /* initiate suspend immediately in the case that we're waiting * for the timeout */ stop_suspend_timer(arm_state); if (!vchiq_videocore_wanted(state)) { vchiq_log_info(vchiq_susp_log_level, "%s videocore " "idle, initiating suspend", __func__); status = vchiq_arm_vcsuspend(state); } else if (arm_state->autosuspend_override < FORCE_SUSPEND_FAIL_MAX) { vchiq_log_info(vchiq_susp_log_level, "%s letting " "videocore go idle", __func__); status = VCHIQ_SUCCESS; } else { vchiq_log_warning(vchiq_susp_log_level, "%s failed too " "many times - attempting suspend", __func__); status = vchiq_arm_vcsuspend(state); } } else { vchiq_log_info(vchiq_susp_log_level, "%s videocore suspend " "in progress - wait for completion", __func__); status = VCHIQ_SUCCESS; } /* Wait for suspend to happen due to system idle (not forced..) */ if (status != VCHIQ_SUCCESS) goto unblock_resume; do { write_unlock_bh(&arm_state->susp_res_lock); rc = wait_for_completion_interruptible_timeout( &arm_state->vc_suspend_complete, msecs_to_jiffies(FORCE_SUSPEND_TIMEOUT_MS)); write_lock_bh(&arm_state->susp_res_lock); if (rc < 0) { vchiq_log_warning(vchiq_susp_log_level, "%s " "interrupted waiting for suspend", __func__); status = VCHIQ_ERROR; goto unblock_resume; } else if (rc == 0) { if (arm_state->vc_suspend_state > VC_SUSPEND_IDLE) { /* Repeat timeout once if in progress */ if (repeat < 0) { repeat = 1; continue; } } arm_state->autosuspend_override++; output_timeout_error(state); status = VCHIQ_RETRY; goto unblock_resume; } } while (0 < (repeat--)); /* Check and report state in case we need to abort ARM suspend */ if (arm_state->vc_suspend_state != VC_SUSPEND_SUSPENDED) { status = VCHIQ_RETRY; vchiq_log_error(vchiq_susp_log_level, "%s videocore suspend failed (state %s)", __func__, suspend_state_names[arm_state->vc_suspend_state + VC_SUSPEND_NUM_OFFSET]); /* Reset the state only if it's still in an error state. * Something could have already initiated another suspend. */ if (arm_state->vc_suspend_state < VC_SUSPEND_IDLE) set_suspend_state(arm_state, VC_SUSPEND_IDLE); goto unblock_resume; } /* successfully suspended - unlock and exit */ goto unlock; unblock_resume: /* all error states need to unblock resume before exit */ unblock_resume(arm_state); unlock: write_unlock_bh(&arm_state->susp_res_lock); out: vchiq_log_trace(vchiq_susp_log_level, "%s exit %d", __func__, status); return status; } void vchiq_check_suspend(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); write_lock_bh(&arm_state->susp_res_lock); if (arm_state->vc_suspend_state != VC_SUSPEND_SUSPENDED && arm_state->first_connect && !vchiq_videocore_wanted(state)) { vchiq_arm_vcsuspend(state); } write_unlock_bh(&arm_state->susp_res_lock); out: vchiq_log_trace(vchiq_susp_log_level, "%s exit", __func__); return; } int vchiq_arm_allow_resume(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); int resume = 0; int ret = -1; if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); write_lock_bh(&arm_state->susp_res_lock); unblock_resume(arm_state); resume = vchiq_check_resume(state); write_unlock_bh(&arm_state->susp_res_lock); if (resume) { if (wait_for_completion_interruptible( &arm_state->vc_resume_complete) < 0) { vchiq_log_error(vchiq_susp_log_level, "%s interrupted", __func__); /* failed, cannot accurately derive suspend * state, so exit early. */ goto out; } } read_lock_bh(&arm_state->susp_res_lock); if (arm_state->vc_suspend_state == VC_SUSPEND_SUSPENDED) { vchiq_log_info(vchiq_susp_log_level, "%s: Videocore remains suspended", __func__); } else { vchiq_log_info(vchiq_susp_log_level, "%s: Videocore resumed", __func__); ret = 0; } read_unlock_bh(&arm_state->susp_res_lock); out: vchiq_log_trace(vchiq_susp_log_level, "%s exit %d", __func__, ret); return ret; } /* This function should be called with the write lock held */ int vchiq_check_resume(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); int resume = 0; if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); if (need_resume(state)) { set_resume_state(arm_state, VC_RESUME_REQUESTED); request_poll(state, NULL, 0); resume = 1; } out: vchiq_log_trace(vchiq_susp_log_level, "%s exit", __func__); return resume; } #ifdef notyet void vchiq_platform_check_resume(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); int res = 0; if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); write_lock_bh(&arm_state->susp_res_lock); if (arm_state->wake_address == 0) { vchiq_log_info(vchiq_susp_log_level, "%s: already awake", __func__); goto unlock; } if (arm_state->vc_resume_state == VC_RESUME_IN_PROGRESS) { vchiq_log_info(vchiq_susp_log_level, "%s: already resuming", __func__); goto unlock; } if (arm_state->vc_resume_state == VC_RESUME_REQUESTED) { set_resume_state(arm_state, VC_RESUME_IN_PROGRESS); res = 1; } else vchiq_log_trace(vchiq_susp_log_level, "%s: not resuming (resume state %s)", __func__, resume_state_names[arm_state->vc_resume_state + VC_RESUME_NUM_OFFSET]); unlock: write_unlock_bh(&arm_state->susp_res_lock); if (res) vchiq_platform_resume(state); out: vchiq_log_trace(vchiq_susp_log_level, "%s exit", __func__); return; } #endif VCHIQ_STATUS_T vchiq_use_internal(VCHIQ_STATE_T *state, VCHIQ_SERVICE_T *service, enum USE_TYPE_E use_type) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); VCHIQ_STATUS_T ret = VCHIQ_SUCCESS; char entity[16]; int *entity_uc; int local_uc, local_entity_uc; if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); if (use_type == USE_TYPE_VCHIQ) { snprintf(entity, sizeof(entity), "VCHIQ: "); entity_uc = &arm_state->peer_use_count; } else if (service) { snprintf(entity, sizeof(entity), "%c%c%c%c:%8x", VCHIQ_FOURCC_AS_4CHARS(service->base.fourcc), service->client_id); entity_uc = &service->service_use_count; } else { vchiq_log_error(vchiq_susp_log_level, "%s null service " "ptr", __func__); ret = VCHIQ_ERROR; goto out; } write_lock_bh(&arm_state->susp_res_lock); while (arm_state->resume_blocked) { /* If we call 'use' while force suspend is waiting for suspend, * then we're about to block the thread which the force is * waiting to complete, so we're bound to just time out. In this * case, set the suspend state such that the wait will be * canceled, so we can complete as quickly as possible. */ if (arm_state->resume_blocked && arm_state->vc_suspend_state == VC_SUSPEND_IDLE) { set_suspend_state(arm_state, VC_SUSPEND_FORCE_CANCELED); break; } /* If suspend is already in progress then we need to block */ if (!try_wait_for_completion(&arm_state->resume_blocker)) { /* Indicate that there are threads waiting on the resume * blocker. These need to be allowed to complete before * a _second_ call to force suspend can complete, * otherwise low priority threads might never actually * continue */ arm_state->blocked_count++; write_unlock_bh(&arm_state->susp_res_lock); vchiq_log_info(vchiq_susp_log_level, "%s %s resume " "blocked - waiting...", __func__, entity); if (wait_for_completion_killable( &arm_state->resume_blocker) != 0) { vchiq_log_error(vchiq_susp_log_level, "%s %s " "wait for resume blocker interrupted", __func__, entity); ret = VCHIQ_ERROR; write_lock_bh(&arm_state->susp_res_lock); arm_state->blocked_count--; write_unlock_bh(&arm_state->susp_res_lock); goto out; } vchiq_log_info(vchiq_susp_log_level, "%s %s resume " "unblocked", __func__, entity); write_lock_bh(&arm_state->susp_res_lock); if (--arm_state->blocked_count == 0) complete_all(&arm_state->blocked_blocker); } } stop_suspend_timer(arm_state); local_uc = ++arm_state->videocore_use_count; local_entity_uc = ++(*entity_uc); /* If there's a pending request which hasn't yet been serviced then * just clear it. If we're past VC_SUSPEND_REQUESTED state then * vc_resume_complete will block until we either resume or fail to * suspend */ if (arm_state->vc_suspend_state <= VC_SUSPEND_REQUESTED) set_suspend_state(arm_state, VC_SUSPEND_IDLE); if ((use_type != USE_TYPE_SERVICE_NO_RESUME) && need_resume(state)) { set_resume_state(arm_state, VC_RESUME_REQUESTED); vchiq_log_info(vchiq_susp_log_level, "%s %s count %d, state count %d", __func__, entity, local_entity_uc, local_uc); request_poll(state, NULL, 0); } else vchiq_log_trace(vchiq_susp_log_level, "%s %s count %d, state count %d", __func__, entity, *entity_uc, local_uc); write_unlock_bh(&arm_state->susp_res_lock); /* Completion is in a done state when we're not suspended, so this won't * block for the non-suspended case. */ if (!try_wait_for_completion(&arm_state->vc_resume_complete)) { vchiq_log_info(vchiq_susp_log_level, "%s %s wait for resume", __func__, entity); if (wait_for_completion_killable( &arm_state->vc_resume_complete) != 0) { vchiq_log_error(vchiq_susp_log_level, "%s %s wait for " "resume interrupted", __func__, entity); ret = VCHIQ_ERROR; goto out; } vchiq_log_info(vchiq_susp_log_level, "%s %s resumed", __func__, entity); } if (ret == VCHIQ_SUCCESS) { VCHIQ_STATUS_T status = VCHIQ_SUCCESS; long ack_cnt = atomic_xchg(&arm_state->ka_use_ack_count, 0); while (ack_cnt && (status == VCHIQ_SUCCESS)) { /* Send the use notify to videocore */ status = vchiq_send_remote_use_active(state); if (status == VCHIQ_SUCCESS) ack_cnt--; else atomic_add(ack_cnt, &arm_state->ka_use_ack_count); } } out: vchiq_log_trace(vchiq_susp_log_level, "%s exit %d", __func__, ret); return ret; } VCHIQ_STATUS_T vchiq_release_internal(VCHIQ_STATE_T *state, VCHIQ_SERVICE_T *service) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); VCHIQ_STATUS_T ret = VCHIQ_SUCCESS; char entity[16]; int *entity_uc; if (!arm_state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); if (service) { snprintf(entity, sizeof(entity), "%c%c%c%c:%8x", VCHIQ_FOURCC_AS_4CHARS(service->base.fourcc), service->client_id); entity_uc = &service->service_use_count; } else { snprintf(entity, sizeof(entity), "PEER: "); entity_uc = &arm_state->peer_use_count; } write_lock_bh(&arm_state->susp_res_lock); if (!arm_state->videocore_use_count || !(*entity_uc)) { /* Don't use BUG_ON - don't allow user thread to crash kernel */ WARN_ON(!arm_state->videocore_use_count); WARN_ON(!(*entity_uc)); ret = VCHIQ_ERROR; goto unlock; } --arm_state->videocore_use_count; --(*entity_uc); if (!vchiq_videocore_wanted(state)) { if (vchiq_platform_use_suspend_timer() && !arm_state->resume_blocked) { /* Only use the timer if we're not trying to force * suspend (=> resume_blocked) */ start_suspend_timer(arm_state); } else { vchiq_log_info(vchiq_susp_log_level, "%s %s count %d, state count %d - suspending", __func__, entity, *entity_uc, arm_state->videocore_use_count); vchiq_arm_vcsuspend(state); } } else vchiq_log_trace(vchiq_susp_log_level, "%s %s count %d, state count %d", __func__, entity, *entity_uc, arm_state->videocore_use_count); unlock: write_unlock_bh(&arm_state->susp_res_lock); out: vchiq_log_trace(vchiq_susp_log_level, "%s exit %d", __func__, ret); return ret; } void vchiq_on_remote_use(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); atomic_inc(&arm_state->ka_use_count); complete(&arm_state->ka_evt); } void vchiq_on_remote_release(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); atomic_inc(&arm_state->ka_release_count); complete(&arm_state->ka_evt); } VCHIQ_STATUS_T vchiq_use_service_internal(VCHIQ_SERVICE_T *service) { return vchiq_use_internal(service->state, service, USE_TYPE_SERVICE); } VCHIQ_STATUS_T vchiq_release_service_internal(VCHIQ_SERVICE_T *service) { return vchiq_release_internal(service->state, service); } static void suspend_timer_callback(unsigned long context) { VCHIQ_STATE_T *state = (VCHIQ_STATE_T *)context; VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); if (!arm_state) goto out; vchiq_log_info(vchiq_susp_log_level, "%s - suspend timer expired - check suspend", __func__); vchiq_check_suspend(state); out: return; } VCHIQ_STATUS_T vchiq_use_service_no_resume(VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_STATUS_T ret = VCHIQ_ERROR; VCHIQ_SERVICE_T *service = find_service_by_handle(handle); if (service) { ret = vchiq_use_internal(service->state, service, USE_TYPE_SERVICE_NO_RESUME); unlock_service(service); } return ret; } VCHIQ_STATUS_T vchiq_use_service(VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_STATUS_T ret = VCHIQ_ERROR; VCHIQ_SERVICE_T *service = find_service_by_handle(handle); if (service) { ret = vchiq_use_internal(service->state, service, USE_TYPE_SERVICE); unlock_service(service); } return ret; } VCHIQ_STATUS_T vchiq_release_service(VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_STATUS_T ret = VCHIQ_ERROR; VCHIQ_SERVICE_T *service = find_service_by_handle(handle); if (service) { ret = vchiq_release_internal(service->state, service); unlock_service(service); } return ret; } void vchiq_dump_service_use_state(VCHIQ_STATE_T *state) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); int i, j = 0; /* Only dump 64 services */ static const int local_max_services = 64; /* If there's more than 64 services, only dump ones with * non-zero counts */ int only_nonzero = 0; static const char *nz = "<-- preventing suspend"; enum vc_suspend_status vc_suspend_state; enum vc_resume_status vc_resume_state; int peer_count; int vc_use_count; int active_services; struct service_data_struct { int fourcc; int clientid; int use_count; } service_data[local_max_services]; if (!arm_state) return; read_lock_bh(&arm_state->susp_res_lock); vc_suspend_state = arm_state->vc_suspend_state; vc_resume_state = arm_state->vc_resume_state; peer_count = arm_state->peer_use_count; vc_use_count = arm_state->videocore_use_count; active_services = state->unused_service; if (active_services > local_max_services) only_nonzero = 1; for (i = 0; (i < active_services) && (j < local_max_services); i++) { VCHIQ_SERVICE_T *service_ptr = state->services[i]; if (!service_ptr) continue; if (only_nonzero && !service_ptr->service_use_count) continue; if (service_ptr->srvstate != VCHIQ_SRVSTATE_FREE) { service_data[j].fourcc = service_ptr->base.fourcc; service_data[j].clientid = service_ptr->client_id; service_data[j++].use_count = service_ptr-> service_use_count; } } read_unlock_bh(&arm_state->susp_res_lock); vchiq_log_warning(vchiq_susp_log_level, "-- Videcore suspend state: %s --", suspend_state_names[vc_suspend_state + VC_SUSPEND_NUM_OFFSET]); vchiq_log_warning(vchiq_susp_log_level, "-- Videcore resume state: %s --", resume_state_names[vc_resume_state + VC_RESUME_NUM_OFFSET]); if (only_nonzero) vchiq_log_warning(vchiq_susp_log_level, "Too many active " "services (%d). Only dumping up to first %d services " "with non-zero use-count", active_services, local_max_services); for (i = 0; i < j; i++) { vchiq_log_warning(vchiq_susp_log_level, "----- %c%c%c%c:%d service count %d %s", VCHIQ_FOURCC_AS_4CHARS(service_data[i].fourcc), service_data[i].clientid, service_data[i].use_count, service_data[i].use_count ? nz : ""); } vchiq_log_warning(vchiq_susp_log_level, "----- VCHIQ use count count %d", peer_count); vchiq_log_warning(vchiq_susp_log_level, "--- Overall vchiq instance use count %d", vc_use_count); vchiq_dump_platform_use_state(state); } VCHIQ_STATUS_T vchiq_check_service(VCHIQ_SERVICE_T *service) { VCHIQ_ARM_STATE_T *arm_state; VCHIQ_STATUS_T ret = VCHIQ_ERROR; if (!service || !service->state) goto out; vchiq_log_trace(vchiq_susp_log_level, "%s", __func__); arm_state = vchiq_platform_get_arm_state(service->state); read_lock_bh(&arm_state->susp_res_lock); if (service->service_use_count) ret = VCHIQ_SUCCESS; read_unlock_bh(&arm_state->susp_res_lock); if (ret == VCHIQ_ERROR) { vchiq_log_error(vchiq_susp_log_level, "%s ERROR - %c%c%c%c:%8x service count %d, " "state count %d, videocore suspend state %s", __func__, VCHIQ_FOURCC_AS_4CHARS(service->base.fourcc), service->client_id, service->service_use_count, arm_state->videocore_use_count, suspend_state_names[arm_state->vc_suspend_state + VC_SUSPEND_NUM_OFFSET]); vchiq_dump_service_use_state(service->state); } out: return ret; } /* stub functions */ void vchiq_on_remote_use_active(VCHIQ_STATE_T *state) { (void)state; } void vchiq_platform_conn_state_changed(VCHIQ_STATE_T *state, VCHIQ_CONNSTATE_T oldstate, VCHIQ_CONNSTATE_T newstate) { VCHIQ_ARM_STATE_T *arm_state = vchiq_platform_get_arm_state(state); vchiq_log_info(vchiq_susp_log_level, "%d: %s->%s", state->id, get_conn_state_name(oldstate), get_conn_state_name(newstate)); if (state->conn_state == VCHIQ_CONNSTATE_CONNECTED) { write_lock_bh(&arm_state->susp_res_lock); if (!arm_state->first_connect) { char threadname[10]; arm_state->first_connect = 1; write_unlock_bh(&arm_state->susp_res_lock); snprintf(threadname, sizeof(threadname), "VCHIQka-%d", state->id); arm_state->ka_thread = vchiq_thread_create( &vchiq_keepalive_thread_func, (void *)state, threadname); if (arm_state->ka_thread == NULL) { vchiq_log_error(vchiq_susp_log_level, "vchiq: FATAL: couldn't create thread %s", threadname); } else { wake_up_process(arm_state->ka_thread); } } else write_unlock_bh(&arm_state->susp_res_lock); } } /**************************************************************************** * * vchiq_init - called when the module is loaded. * ***************************************************************************/ int __init vchiq_init(void); int __init vchiq_init(void) { int err; #ifdef notyet /* create proc entries */ err = vchiq_proc_init(); if (err != 0) goto failed_proc_init; #endif vchiq_cdev = make_dev(&vchiq_cdevsw, 0, UID_ROOT, GID_WHEEL, 0600, "vchiq"); if (!vchiq_cdev) { printf("Failed to create /dev/vchiq"); return (-ENXIO); } spin_lock_init(&msg_queue_spinlock); err = vchiq_platform_init(&g_state); if (err != 0) goto failed_platform_init; vchiq_log_info(vchiq_arm_log_level, "vchiq: initialised - version %d (min %d)", VCHIQ_VERSION, VCHIQ_VERSION_MIN); return 0; failed_platform_init: if (vchiq_cdev) { destroy_dev(vchiq_cdev); vchiq_cdev = NULL; } vchiq_log_warning(vchiq_arm_log_level, "could not load vchiq"); return err; } #ifdef notyet static int vchiq_instance_get_use_count(VCHIQ_INSTANCE_T instance) { VCHIQ_SERVICE_T *service; int use_count = 0, i; i = 0; while ((service = next_service_by_instance(instance->state, instance, &i)) != NULL) { use_count += service->service_use_count; unlock_service(service); } return use_count; } /* read the per-process use-count */ static int proc_read_use_count(char *page, char **start, off_t off, int count, int *eof, void *data) { VCHIQ_INSTANCE_T instance = data; int len, use_count; use_count = vchiq_instance_get_use_count(instance); len = snprintf(page+off, count, "%d\n", use_count); return len; } /* add an instance (process) to the proc entries */ static int vchiq_proc_add_instance(VCHIQ_INSTANCE_T instance) { char pidstr[32]; struct proc_dir_entry *top, *use_count; struct proc_dir_entry *clients = vchiq_clients_top(); int pid = instance->pid; snprintf(pidstr, sizeof(pidstr), "%d", pid); top = proc_mkdir(pidstr, clients); if (!top) goto fail_top; use_count = create_proc_read_entry("use_count", 0444, top, proc_read_use_count, instance); if (!use_count) goto fail_use_count; instance->proc_entry = top; return 0; fail_use_count: remove_proc_entry(top->name, clients); fail_top: return -ENOMEM; } static void vchiq_proc_remove_instance(VCHIQ_INSTANCE_T instance) { struct proc_dir_entry *clients = vchiq_clients_top(); remove_proc_entry("use_count", instance->proc_entry); remove_proc_entry(instance->proc_entry->name, clients); } #endif /**************************************************************************** * * vchiq_exit - called when the module is unloaded. * ***************************************************************************/ void vchiq_exit(void); void vchiq_exit(void) { if (vchiq_ehtag == NULL) EVENTHANDLER_DEREGISTER(dev_clone, vchiq_ehtag); vchiq_ehtag = NULL; vchiq_platform_exit(&g_state); if (vchiq_cdev) { destroy_dev(vchiq_cdev); vchiq_cdev = NULL; } } Index: projects/netbsd-tests-update-12/sys/contrib/vchiq/interface/vchiq_arm/vchiq_core.c =================================================================== --- projects/netbsd-tests-update-12/sys/contrib/vchiq/interface/vchiq_arm/vchiq_core.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/contrib/vchiq/interface/vchiq_arm/vchiq_core.c (revision 305172) @@ -1,3954 +1,3953 @@ /** * Copyright (c) 2010-2012 Broadcom. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions, and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The names of the above-listed copyright holders may not be used * to endorse or promote products derived from this software without * specific prior written permission. * * ALTERNATIVELY, this software may be distributed under the terms of the * GNU General Public License ("GPL") version 2, as published by the Free * Software Foundation. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "vchiq_core.h" #include "vchiq_killable.h" #define VCHIQ_SLOT_HANDLER_STACK 8192 #define HANDLE_STATE_SHIFT 12 #define SLOT_INFO_FROM_INDEX(state, index) (state->slot_info + (index)) #define SLOT_DATA_FROM_INDEX(state, index) (state->slot_data + (index)) #define SLOT_INDEX_FROM_DATA(state, data) \ (((unsigned int)((char *)data - (char *)state->slot_data)) / \ VCHIQ_SLOT_SIZE) #define SLOT_INDEX_FROM_INFO(state, info) \ ((unsigned int)(info - state->slot_info)) #define SLOT_QUEUE_INDEX_FROM_POS(pos) \ ((int)((unsigned int)(pos) / VCHIQ_SLOT_SIZE)) #define BULK_INDEX(x) (x & (VCHIQ_NUM_SERVICE_BULKS - 1)) #define SRVTRACE_LEVEL(srv) \ (((srv) && (srv)->trace) ? VCHIQ_LOG_TRACE : vchiq_core_msg_log_level) #define SRVTRACE_ENABLED(srv, lev) \ (((srv) && (srv)->trace) || (vchiq_core_msg_log_level >= (lev))) struct vchiq_open_payload { int fourcc; int client_id; short version; short version_min; }; struct vchiq_openack_payload { short version; }; enum { QMFLAGS_IS_BLOCKING = (1 << 0), QMFLAGS_NO_MUTEX_LOCK = (1 << 1), QMFLAGS_NO_MUTEX_UNLOCK = (1 << 2) }; /* we require this for consistency between endpoints */ vchiq_static_assert(sizeof(VCHIQ_HEADER_T) == 8); vchiq_static_assert(IS_POW2(sizeof(VCHIQ_HEADER_T))); vchiq_static_assert(IS_POW2(VCHIQ_NUM_CURRENT_BULKS)); vchiq_static_assert(IS_POW2(VCHIQ_NUM_SERVICE_BULKS)); vchiq_static_assert(IS_POW2(VCHIQ_MAX_SERVICES)); vchiq_static_assert(VCHIQ_VERSION >= VCHIQ_VERSION_MIN); /* Run time control of log level, based on KERN_XXX level. */ int vchiq_core_log_level = VCHIQ_LOG_DEFAULT; int vchiq_core_msg_log_level = VCHIQ_LOG_DEFAULT; int vchiq_sync_log_level = VCHIQ_LOG_DEFAULT; static atomic_t pause_bulks_count = ATOMIC_INIT(0); static DEFINE_SPINLOCK(service_spinlock); DEFINE_SPINLOCK(bulk_waiter_spinlock); DEFINE_SPINLOCK(quota_spinlock); void vchiq_core_initialize(void) { spin_lock_init(&service_spinlock); spin_lock_init(&bulk_waiter_spinlock); spin_lock_init("a_spinlock); } VCHIQ_STATE_T *vchiq_states[VCHIQ_MAX_STATES]; static unsigned int handle_seq; static const char *const srvstate_names[] = { "FREE", "HIDDEN", "LISTENING", "OPENING", "OPEN", "OPENSYNC", "CLOSESENT", "CLOSERECVD", "CLOSEWAIT", "CLOSED" }; static const char *const reason_names[] = { "SERVICE_OPENED", "SERVICE_CLOSED", "MESSAGE_AVAILABLE", "BULK_TRANSMIT_DONE", "BULK_RECEIVE_DONE", "BULK_TRANSMIT_ABORTED", "BULK_RECEIVE_ABORTED" }; static const char *const conn_state_names[] = { "DISCONNECTED", "CONNECTING", "CONNECTED", "PAUSING", "PAUSE_SENT", "PAUSED", "RESUMING", "PAUSE_TIMEOUT", "RESUME_TIMEOUT" }; static void release_message_sync(VCHIQ_STATE_T *state, VCHIQ_HEADER_T *header); static const char *msg_type_str(unsigned int msg_type) { switch (msg_type) { case VCHIQ_MSG_PADDING: return "PADDING"; case VCHIQ_MSG_CONNECT: return "CONNECT"; case VCHIQ_MSG_OPEN: return "OPEN"; case VCHIQ_MSG_OPENACK: return "OPENACK"; case VCHIQ_MSG_CLOSE: return "CLOSE"; case VCHIQ_MSG_DATA: return "DATA"; case VCHIQ_MSG_BULK_RX: return "BULK_RX"; case VCHIQ_MSG_BULK_TX: return "BULK_TX"; case VCHIQ_MSG_BULK_RX_DONE: return "BULK_RX_DONE"; case VCHIQ_MSG_BULK_TX_DONE: return "BULK_TX_DONE"; case VCHIQ_MSG_PAUSE: return "PAUSE"; case VCHIQ_MSG_RESUME: return "RESUME"; case VCHIQ_MSG_REMOTE_USE: return "REMOTE_USE"; case VCHIQ_MSG_REMOTE_RELEASE: return "REMOTE_RELEASE"; case VCHIQ_MSG_REMOTE_USE_ACTIVE: return "REMOTE_USE_ACTIVE"; } return "???"; } static inline void vchiq_set_service_state(VCHIQ_SERVICE_T *service, int newstate) { vchiq_log_info(vchiq_core_log_level, "%d: srv:%d %s->%s", service->state->id, service->localport, srvstate_names[service->srvstate], srvstate_names[newstate]); service->srvstate = newstate; } VCHIQ_SERVICE_T * find_service_by_handle(VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_SERVICE_T *service; spin_lock(&service_spinlock); service = handle_to_service(handle); if (service && (service->srvstate != VCHIQ_SRVSTATE_FREE) && (service->handle == handle)) { BUG_ON(service->ref_count == 0); service->ref_count++; } else service = NULL; spin_unlock(&service_spinlock); if (!service) vchiq_log_info(vchiq_core_log_level, "Invalid service handle 0x%x", handle); return service; } VCHIQ_SERVICE_T * find_service_by_port(VCHIQ_STATE_T *state, int localport) { VCHIQ_SERVICE_T *service = NULL; if ((unsigned int)localport <= VCHIQ_PORT_MAX) { spin_lock(&service_spinlock); service = state->services[localport]; if (service && (service->srvstate != VCHIQ_SRVSTATE_FREE)) { BUG_ON(service->ref_count == 0); service->ref_count++; } else service = NULL; spin_unlock(&service_spinlock); } if (!service) vchiq_log_info(vchiq_core_log_level, "Invalid port %d", localport); return service; } VCHIQ_SERVICE_T * find_service_for_instance(VCHIQ_INSTANCE_T instance, VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_SERVICE_T *service; spin_lock(&service_spinlock); service = handle_to_service(handle); if (service && (service->srvstate != VCHIQ_SRVSTATE_FREE) && (service->handle == handle) && (service->instance == instance)) { BUG_ON(service->ref_count == 0); service->ref_count++; } else service = NULL; spin_unlock(&service_spinlock); if (!service) vchiq_log_info(vchiq_core_log_level, "Invalid service handle 0x%x", handle); return service; } VCHIQ_SERVICE_T * find_closed_service_for_instance(VCHIQ_INSTANCE_T instance, VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_SERVICE_T *service; spin_lock(&service_spinlock); service = handle_to_service(handle); if (service && ((service->srvstate == VCHIQ_SRVSTATE_FREE) || (service->srvstate == VCHIQ_SRVSTATE_CLOSED)) && (service->handle == handle) && (service->instance == instance)) { BUG_ON(service->ref_count == 0); service->ref_count++; } else service = NULL; spin_unlock(&service_spinlock); if (!service) vchiq_log_info(vchiq_core_log_level, "Invalid service handle 0x%x", handle); return service; } VCHIQ_SERVICE_T * next_service_by_instance(VCHIQ_STATE_T *state, VCHIQ_INSTANCE_T instance, int *pidx) { VCHIQ_SERVICE_T *service = NULL; int idx = *pidx; spin_lock(&service_spinlock); while (idx < state->unused_service) { VCHIQ_SERVICE_T *srv = state->services[idx++]; if (srv && (srv->srvstate != VCHIQ_SRVSTATE_FREE) && (srv->instance == instance)) { service = srv; BUG_ON(service->ref_count == 0); service->ref_count++; break; } } spin_unlock(&service_spinlock); *pidx = idx; return service; } void lock_service(VCHIQ_SERVICE_T *service) { spin_lock(&service_spinlock); BUG_ON(!service || (service->ref_count == 0)); if (service) service->ref_count++; spin_unlock(&service_spinlock); } void unlock_service(VCHIQ_SERVICE_T *service) { VCHIQ_STATE_T *state = service->state; spin_lock(&service_spinlock); BUG_ON(!service || (service->ref_count == 0)); if (service && service->ref_count) { service->ref_count--; if (!service->ref_count) { BUG_ON(service->srvstate != VCHIQ_SRVSTATE_FREE); state->services[service->localport] = NULL; _sema_destroy(&service->remove_event); _sema_destroy(&service->bulk_remove_event); lmutex_destroy(&service->bulk_mutex); } else service = NULL; } spin_unlock(&service_spinlock); if (service && service->userdata_term) service->userdata_term(service->base.userdata); kfree(service); } int vchiq_get_client_id(VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_SERVICE_T *service = find_service_by_handle(handle); int id; id = service ? service->client_id : 0; if (service) unlock_service(service); return id; } void * vchiq_get_service_userdata(VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_SERVICE_T *service = handle_to_service(handle); return service ? service->base.userdata : NULL; } int vchiq_get_service_fourcc(VCHIQ_SERVICE_HANDLE_T handle) { VCHIQ_SERVICE_T *service = handle_to_service(handle); return service ? service->base.fourcc : 0; } static void mark_service_closing_internal(VCHIQ_SERVICE_T *service, int sh_thread) { VCHIQ_STATE_T *state = service->state; VCHIQ_SERVICE_QUOTA_T *service_quota; service->closing = 1; /* Synchronise with other threads. */ lmutex_lock(&state->recycle_mutex); lmutex_unlock(&state->recycle_mutex); if (!sh_thread || (state->conn_state != VCHIQ_CONNSTATE_PAUSE_SENT)) { /* If we're pausing then the slot_mutex is held until resume * by the slot handler. Therefore don't try to acquire this * mutex if we're the slot handler and in the pause sent state. * We don't need to in this case anyway. */ lmutex_lock(&state->slot_mutex); lmutex_unlock(&state->slot_mutex); } /* Unblock any sending thread. */ service_quota = &state->service_quotas[service->localport]; up(&service_quota->quota_event); } static void mark_service_closing(VCHIQ_SERVICE_T *service) { mark_service_closing_internal(service, 0); } static inline VCHIQ_STATUS_T make_service_callback(VCHIQ_SERVICE_T *service, VCHIQ_REASON_T reason, VCHIQ_HEADER_T *header, void *bulk_userdata) { VCHIQ_STATUS_T status; vchiq_log_trace(vchiq_core_log_level, "%d: callback:%d (%s, %x, %x)", service->state->id, service->localport, reason_names[reason], (unsigned int)header, (unsigned int)bulk_userdata); status = service->base.callback(reason, header, service->handle, bulk_userdata); if (status == VCHIQ_ERROR) { vchiq_log_warning(vchiq_core_log_level, "%d: ignoring ERROR from callback to service %x", service->state->id, service->handle); status = VCHIQ_SUCCESS; } return status; } inline void vchiq_set_conn_state(VCHIQ_STATE_T *state, VCHIQ_CONNSTATE_T newstate) { VCHIQ_CONNSTATE_T oldstate = state->conn_state; vchiq_log_info(vchiq_core_log_level, "%d: %s->%s", state->id, conn_state_names[oldstate], conn_state_names[newstate]); state->conn_state = newstate; vchiq_platform_conn_state_changed(state, oldstate, newstate); } static inline void remote_event_create(REMOTE_EVENT_T *event) { event->armed = 0; /* Don't clear the 'fired' flag because it may already have been set ** by the other side. */ _sema_init(event->event, 0); } __unused static inline void remote_event_destroy(REMOTE_EVENT_T *event) { (void)event; } static inline int remote_event_wait(REMOTE_EVENT_T *event) { if (!event->fired) { event->armed = 1; dsb(); if (!event->fired) { if (down_interruptible(event->event) != 0) { event->armed = 0; return 0; } } event->armed = 0; wmb(); } event->fired = 0; return 1; } static inline void remote_event_signal_local(REMOTE_EVENT_T *event) { event->armed = 0; up(event->event); } static inline void remote_event_poll(REMOTE_EVENT_T *event) { if (event->fired && event->armed) remote_event_signal_local(event); } void remote_event_pollall(VCHIQ_STATE_T *state) { remote_event_poll(&state->local->sync_trigger); remote_event_poll(&state->local->sync_release); remote_event_poll(&state->local->trigger); remote_event_poll(&state->local->recycle); } /* Round up message sizes so that any space at the end of a slot is always big ** enough for a header. This relies on header size being a power of two, which ** has been verified earlier by a static assertion. */ static inline unsigned int calc_stride(unsigned int size) { /* Allow room for the header */ size += sizeof(VCHIQ_HEADER_T); /* Round up */ return (size + sizeof(VCHIQ_HEADER_T) - 1) & ~(sizeof(VCHIQ_HEADER_T) - 1); } /* Called by the slot handler thread */ static VCHIQ_SERVICE_T * get_listening_service(VCHIQ_STATE_T *state, int fourcc) { int i; WARN_ON(fourcc == VCHIQ_FOURCC_INVALID); for (i = 0; i < state->unused_service; i++) { VCHIQ_SERVICE_T *service = state->services[i]; if (service && (service->public_fourcc == fourcc) && ((service->srvstate == VCHIQ_SRVSTATE_LISTENING) || ((service->srvstate == VCHIQ_SRVSTATE_OPEN) && (service->remoteport == VCHIQ_PORT_FREE)))) { lock_service(service); return service; } } return NULL; } /* Called by the slot handler thread */ static VCHIQ_SERVICE_T * get_connected_service(VCHIQ_STATE_T *state, unsigned int port) { int i; for (i = 0; i < state->unused_service; i++) { VCHIQ_SERVICE_T *service = state->services[i]; if (service && (service->srvstate == VCHIQ_SRVSTATE_OPEN) && (service->remoteport == port)) { lock_service(service); return service; } } return NULL; } inline void request_poll(VCHIQ_STATE_T *state, VCHIQ_SERVICE_T *service, int poll_type) { uint32_t value; if (service) { do { value = atomic_read(&service->poll_flags); } while (atomic_cmpxchg(&service->poll_flags, value, value | (1 << poll_type)) != value); do { value = atomic_read(&state->poll_services[ service->localport>>5]); } while (atomic_cmpxchg( &state->poll_services[service->localport>>5], value, value | (1 << (service->localport & 0x1f))) != value); } state->poll_needed = 1; wmb(); /* ... and ensure the slot handler runs. */ remote_event_signal_local(&state->local->trigger); } /* Called from queue_message, by the slot handler and application threads, ** with slot_mutex held */ static VCHIQ_HEADER_T * reserve_space(VCHIQ_STATE_T *state, int space, int is_blocking) { VCHIQ_SHARED_STATE_T *local = state->local; int tx_pos = state->local_tx_pos; int slot_space = VCHIQ_SLOT_SIZE - (tx_pos & VCHIQ_SLOT_MASK); if (space > slot_space) { VCHIQ_HEADER_T *header; /* Fill the remaining space with padding */ WARN_ON(state->tx_data == NULL); header = (VCHIQ_HEADER_T *) (state->tx_data + (tx_pos & VCHIQ_SLOT_MASK)); header->msgid = VCHIQ_MSGID_PADDING; header->size = slot_space - sizeof(VCHIQ_HEADER_T); tx_pos += slot_space; } /* If necessary, get the next slot. */ if ((tx_pos & VCHIQ_SLOT_MASK) == 0) { int slot_index; /* If there is no free slot... */ if (down_trylock(&state->slot_available_event) != 0) { /* ...wait for one. */ VCHIQ_STATS_INC(state, slot_stalls); /* But first, flush through the last slot. */ state->local_tx_pos = tx_pos; local->tx_pos = tx_pos; remote_event_signal(&state->remote->trigger); if (!is_blocking || (down_interruptible( &state->slot_available_event) != 0)) return NULL; /* No space available */ } BUG_ON(tx_pos == (state->slot_queue_available * VCHIQ_SLOT_SIZE)); slot_index = local->slot_queue[ SLOT_QUEUE_INDEX_FROM_POS(tx_pos) & VCHIQ_SLOT_QUEUE_MASK]; state->tx_data = (char *)SLOT_DATA_FROM_INDEX(state, slot_index); } state->local_tx_pos = tx_pos + space; return (VCHIQ_HEADER_T *)(state->tx_data + (tx_pos & VCHIQ_SLOT_MASK)); } /* Called by the recycle thread. */ static void process_free_queue(VCHIQ_STATE_T *state) { VCHIQ_SHARED_STATE_T *local = state->local; BITSET_T service_found[BITSET_SIZE(VCHIQ_MAX_SERVICES)]; int slot_queue_available; - /* Use a read memory barrier to ensure that any state that may have - ** been modified by another thread is not masked by stale prefetched - ** values. */ - rmb(); - /* Find slots which have been freed by the other side, and return them ** to the available queue. */ slot_queue_available = state->slot_queue_available; + /* Use a memory barrier to ensure that any state that may have been + ** modified by another thread is not masked by stale prefetched + ** values. */ + mb(); + while (slot_queue_available != local->slot_queue_recycle) { unsigned int pos; int slot_index = local->slot_queue[slot_queue_available++ & VCHIQ_SLOT_QUEUE_MASK]; char *data = (char *)SLOT_DATA_FROM_INDEX(state, slot_index); int data_found = 0; + rmb(); + vchiq_log_trace(vchiq_core_log_level, "%d: pfq %d=%x %x %x", state->id, slot_index, (unsigned int)data, local->slot_queue_recycle, slot_queue_available); /* Initialise the bitmask for services which have used this ** slot */ BITSET_ZERO(service_found); pos = 0; while (pos < VCHIQ_SLOT_SIZE) { VCHIQ_HEADER_T *header = (VCHIQ_HEADER_T *)(data + pos); int msgid = header->msgid; if (VCHIQ_MSG_TYPE(msgid) == VCHIQ_MSG_DATA) { int port = VCHIQ_MSG_SRCPORT(msgid); VCHIQ_SERVICE_QUOTA_T *service_quota = &state->service_quotas[port]; int count; spin_lock("a_spinlock); count = service_quota->message_use_count; if (count > 0) service_quota->message_use_count = count - 1; spin_unlock("a_spinlock); if (count == service_quota->message_quota) /* Signal the service that it ** has dropped below its quota */ up(&service_quota->quota_event); else if (count == 0) { vchiq_log_error(vchiq_core_log_level, "service %d " "message_use_count=%d " "(header %x, msgid %x, " "header->msgid %x, " "header->size %x)", port, service_quota-> message_use_count, (unsigned int)header, msgid, header->msgid, header->size); WARN(1, "invalid message use count\n"); } if (!BITSET_IS_SET(service_found, port)) { /* Set the found bit for this service */ BITSET_SET(service_found, port); spin_lock("a_spinlock); count = service_quota->slot_use_count; if (count > 0) service_quota->slot_use_count = count - 1; spin_unlock("a_spinlock); if (count > 0) { /* Signal the service in case ** it has dropped below its ** quota */ up(&service_quota->quota_event); vchiq_log_trace( vchiq_core_log_level, "%d: pfq:%d %x@%x - " "slot_use->%d", state->id, port, header->size, (unsigned int)header, count - 1); } else { vchiq_log_error( vchiq_core_log_level, "service %d " "slot_use_count" "=%d (header %x" ", msgid %x, " "header->msgid" " %x, header->" "size %x)", port, count, (unsigned int)header, msgid, header->msgid, header->size); WARN(1, "bad slot use count\n"); } } data_found = 1; } pos += calc_stride(header->size); if (pos > VCHIQ_SLOT_SIZE) { vchiq_log_error(vchiq_core_log_level, "pfq - pos %x: header %x, msgid %x, " "header->msgid %x, header->size %x", pos, (unsigned int)header, msgid, header->msgid, header->size); WARN(1, "invalid slot position\n"); } } if (data_found) { int count; spin_lock("a_spinlock); count = state->data_use_count; if (count > 0) state->data_use_count = count - 1; spin_unlock("a_spinlock); if (count == state->data_quota) up(&state->data_quota_event); } + mb(); + state->slot_queue_available = slot_queue_available; up(&state->slot_available_event); } } /* Called by the slot handler and application threads */ static VCHIQ_STATUS_T queue_message(VCHIQ_STATE_T *state, VCHIQ_SERVICE_T *service, int msgid, const VCHIQ_ELEMENT_T *elements, int count, int size, int flags) { VCHIQ_SHARED_STATE_T *local; VCHIQ_SERVICE_QUOTA_T *service_quota = NULL; VCHIQ_HEADER_T *header; int type = VCHIQ_MSG_TYPE(msgid); unsigned int stride; local = state->local; stride = calc_stride(size); WARN_ON(!(stride <= VCHIQ_SLOT_SIZE)); if (!(flags & QMFLAGS_NO_MUTEX_LOCK) && (lmutex_lock_interruptible(&state->slot_mutex) != 0)) return VCHIQ_RETRY; if (type == VCHIQ_MSG_DATA) { int tx_end_index; BUG_ON(!service); BUG_ON((flags & (QMFLAGS_NO_MUTEX_LOCK | QMFLAGS_NO_MUTEX_UNLOCK)) != 0); if (service->closing) { /* The service has been closed */ lmutex_unlock(&state->slot_mutex); return VCHIQ_ERROR; } service_quota = &state->service_quotas[service->localport]; spin_lock("a_spinlock); /* Ensure this service doesn't use more than its quota of ** messages or slots */ tx_end_index = SLOT_QUEUE_INDEX_FROM_POS( state->local_tx_pos + stride - 1); /* Ensure data messages don't use more than their quota of ** slots */ while ((tx_end_index != state->previous_data_index) && (state->data_use_count == state->data_quota)) { VCHIQ_STATS_INC(state, data_stalls); spin_unlock("a_spinlock); lmutex_unlock(&state->slot_mutex); if (down_interruptible(&state->data_quota_event) != 0) return VCHIQ_RETRY; lmutex_lock(&state->slot_mutex); spin_lock("a_spinlock); tx_end_index = SLOT_QUEUE_INDEX_FROM_POS( state->local_tx_pos + stride - 1); if ((tx_end_index == state->previous_data_index) || (state->data_use_count < state->data_quota)) { /* Pass the signal on to other waiters */ up(&state->data_quota_event); break; } } while ((service_quota->message_use_count == service_quota->message_quota) || ((tx_end_index != service_quota->previous_tx_index) && (service_quota->slot_use_count == service_quota->slot_quota))) { spin_unlock("a_spinlock); vchiq_log_trace(vchiq_core_log_level, "%d: qm:%d %s,%x - quota stall " "(msg %d, slot %d)", state->id, service->localport, msg_type_str(type), size, service_quota->message_use_count, service_quota->slot_use_count); VCHIQ_SERVICE_STATS_INC(service, quota_stalls); lmutex_unlock(&state->slot_mutex); if (down_interruptible(&service_quota->quota_event) != 0) return VCHIQ_RETRY; if (service->closing) return VCHIQ_ERROR; if (lmutex_lock_interruptible(&state->slot_mutex) != 0) return VCHIQ_RETRY; if (service->srvstate != VCHIQ_SRVSTATE_OPEN) { /* The service has been closed */ lmutex_unlock(&state->slot_mutex); return VCHIQ_ERROR; } spin_lock("a_spinlock); tx_end_index = SLOT_QUEUE_INDEX_FROM_POS( state->local_tx_pos + stride - 1); } spin_unlock("a_spinlock); } header = reserve_space(state, stride, flags & QMFLAGS_IS_BLOCKING); if (!header) { if (service) VCHIQ_SERVICE_STATS_INC(service, slot_stalls); /* In the event of a failure, return the mutex to the state it was in */ if (!(flags & QMFLAGS_NO_MUTEX_LOCK)) lmutex_unlock(&state->slot_mutex); return VCHIQ_RETRY; } if (type == VCHIQ_MSG_DATA) { int i, pos; int tx_end_index; int slot_use_count; vchiq_log_info(vchiq_core_log_level, "%d: qm %s@%x,%x (%d->%d)", state->id, msg_type_str(VCHIQ_MSG_TYPE(msgid)), (unsigned int)header, size, VCHIQ_MSG_SRCPORT(msgid), VCHIQ_MSG_DSTPORT(msgid)); BUG_ON(!service); BUG_ON((flags & (QMFLAGS_NO_MUTEX_LOCK | QMFLAGS_NO_MUTEX_UNLOCK)) != 0); for (i = 0, pos = 0; i < (unsigned int)count; pos += elements[i++].size) if (elements[i].size) { if (vchiq_copy_from_user (header->data + pos, elements[i].data, (size_t) elements[i].size) != VCHIQ_SUCCESS) { lmutex_unlock(&state->slot_mutex); VCHIQ_SERVICE_STATS_INC(service, error_count); return VCHIQ_ERROR; } - if (i == 0) { - if (SRVTRACE_ENABLED(service, - VCHIQ_LOG_INFO)) - vchiq_log_dump_mem("Sent", 0, - header->data + pos, - min(64u, - elements[0].size)); - } } + if (SRVTRACE_ENABLED(service, + VCHIQ_LOG_INFO)) + vchiq_log_dump_mem("Sent", 0, + header->data, + min(16, pos)); + spin_lock("a_spinlock); service_quota->message_use_count++; tx_end_index = SLOT_QUEUE_INDEX_FROM_POS(state->local_tx_pos - 1); /* If this transmission can't fit in the last slot used by any ** service, the data_use_count must be increased. */ if (tx_end_index != state->previous_data_index) { state->previous_data_index = tx_end_index; state->data_use_count++; } /* If this isn't the same slot last used by this service, ** the service's slot_use_count must be increased. */ if (tx_end_index != service_quota->previous_tx_index) { service_quota->previous_tx_index = tx_end_index; slot_use_count = ++service_quota->slot_use_count; } else { slot_use_count = 0; } spin_unlock("a_spinlock); if (slot_use_count) vchiq_log_trace(vchiq_core_log_level, "%d: qm:%d %s,%x - slot_use->%d (hdr %p)", state->id, service->localport, msg_type_str(VCHIQ_MSG_TYPE(msgid)), size, slot_use_count, header); VCHIQ_SERVICE_STATS_INC(service, ctrl_tx_count); VCHIQ_SERVICE_STATS_ADD(service, ctrl_tx_bytes, size); } else { vchiq_log_info(vchiq_core_log_level, "%d: qm %s@%x,%x (%d->%d)", state->id, msg_type_str(VCHIQ_MSG_TYPE(msgid)), (unsigned int)header, size, VCHIQ_MSG_SRCPORT(msgid), VCHIQ_MSG_DSTPORT(msgid)); if (size != 0) { WARN_ON(!((count == 1) && (size == elements[0].size))); memcpy(header->data, elements[0].data, elements[0].size); } VCHIQ_STATS_INC(state, ctrl_tx_count); } header->msgid = msgid; header->size = size; { int svc_fourcc; svc_fourcc = service ? service->base.fourcc : VCHIQ_MAKE_FOURCC('?', '?', '?', '?'); vchiq_log_info(SRVTRACE_LEVEL(service), "Sent Msg %s(%u) to %c%c%c%c s:%u d:%d len:%d", msg_type_str(VCHIQ_MSG_TYPE(msgid)), VCHIQ_MSG_TYPE(msgid), VCHIQ_FOURCC_AS_4CHARS(svc_fourcc), VCHIQ_MSG_SRCPORT(msgid), VCHIQ_MSG_DSTPORT(msgid), size); } /* Make sure the new header is visible to the peer. */ wmb(); /* Make the new tx_pos visible to the peer. */ local->tx_pos = state->local_tx_pos; wmb(); if (service && (type == VCHIQ_MSG_CLOSE)) vchiq_set_service_state(service, VCHIQ_SRVSTATE_CLOSESENT); if (!(flags & QMFLAGS_NO_MUTEX_UNLOCK)) lmutex_unlock(&state->slot_mutex); remote_event_signal(&state->remote->trigger); return VCHIQ_SUCCESS; } /* Called by the slot handler and application threads */ static VCHIQ_STATUS_T queue_message_sync(VCHIQ_STATE_T *state, VCHIQ_SERVICE_T *service, int msgid, const VCHIQ_ELEMENT_T *elements, int count, int size, int is_blocking) { VCHIQ_SHARED_STATE_T *local; VCHIQ_HEADER_T *header; local = state->local; if ((VCHIQ_MSG_TYPE(msgid) != VCHIQ_MSG_RESUME) && (lmutex_lock_interruptible(&state->sync_mutex) != 0)) return VCHIQ_RETRY; remote_event_wait(&local->sync_release); rmb(); header = (VCHIQ_HEADER_T *)SLOT_DATA_FROM_INDEX(state, local->slot_sync); { int oldmsgid = header->msgid; if (oldmsgid != VCHIQ_MSGID_PADDING) vchiq_log_error(vchiq_core_log_level, "%d: qms - msgid %x, not PADDING", state->id, oldmsgid); } if (service) { int i, pos; vchiq_log_info(vchiq_sync_log_level, "%d: qms %s@%x,%x (%d->%d)", state->id, msg_type_str(VCHIQ_MSG_TYPE(msgid)), (unsigned int)header, size, VCHIQ_MSG_SRCPORT(msgid), VCHIQ_MSG_DSTPORT(msgid)); for (i = 0, pos = 0; i < (unsigned int)count; pos += elements[i++].size) if (elements[i].size) { if (vchiq_copy_from_user (header->data + pos, elements[i].data, (size_t) elements[i].size) != VCHIQ_SUCCESS) { lmutex_unlock(&state->sync_mutex); VCHIQ_SERVICE_STATS_INC(service, error_count); return VCHIQ_ERROR; } - if (i == 0) { - if (vchiq_sync_log_level >= - VCHIQ_LOG_TRACE) - vchiq_log_dump_mem("Sent Sync", - 0, header->data + pos, - min(64u, - elements[0].size)); - } } + if (vchiq_sync_log_level >= VCHIQ_LOG_TRACE) + vchiq_log_dump_mem("Sent Sync", + 0, header->data, + min(16, pos)); + VCHIQ_SERVICE_STATS_INC(service, ctrl_tx_count); VCHIQ_SERVICE_STATS_ADD(service, ctrl_tx_bytes, size); } else { vchiq_log_info(vchiq_sync_log_level, "%d: qms %s@%x,%x (%d->%d)", state->id, msg_type_str(VCHIQ_MSG_TYPE(msgid)), (unsigned int)header, size, VCHIQ_MSG_SRCPORT(msgid), VCHIQ_MSG_DSTPORT(msgid)); if (size != 0) { WARN_ON(!((count == 1) && (size == elements[0].size))); memcpy(header->data, elements[0].data, elements[0].size); } VCHIQ_STATS_INC(state, ctrl_tx_count); } header->size = size; header->msgid = msgid; if (vchiq_sync_log_level >= VCHIQ_LOG_TRACE) { int svc_fourcc; svc_fourcc = service ? service->base.fourcc : VCHIQ_MAKE_FOURCC('?', '?', '?', '?'); vchiq_log_trace(vchiq_sync_log_level, "Sent Sync Msg %s(%u) to %c%c%c%c s:%u d:%d len:%d", msg_type_str(VCHIQ_MSG_TYPE(msgid)), VCHIQ_MSG_TYPE(msgid), VCHIQ_FOURCC_AS_4CHARS(svc_fourcc), VCHIQ_MSG_SRCPORT(msgid), VCHIQ_MSG_DSTPORT(msgid), size); } /* Make sure the new header is visible to the peer. */ wmb(); remote_event_signal(&state->remote->sync_trigger); if (VCHIQ_MSG_TYPE(msgid) != VCHIQ_MSG_PAUSE) lmutex_unlock(&state->sync_mutex); return VCHIQ_SUCCESS; } static inline void claim_slot(VCHIQ_SLOT_INFO_T *slot) { slot->use_count++; } static void release_slot(VCHIQ_STATE_T *state, VCHIQ_SLOT_INFO_T *slot_info, VCHIQ_HEADER_T *header, VCHIQ_SERVICE_T *service) { int release_count; lmutex_lock(&state->recycle_mutex); if (header) { int msgid = header->msgid; if (((msgid & VCHIQ_MSGID_CLAIMED) == 0) || (service && service->closing)) { lmutex_unlock(&state->recycle_mutex); return; } /* Rewrite the message header to prevent a double ** release */ header->msgid = msgid & ~VCHIQ_MSGID_CLAIMED; } release_count = slot_info->release_count; slot_info->release_count = ++release_count; if (release_count == slot_info->use_count) { int slot_queue_recycle; /* Add to the freed queue */ /* A read barrier is necessary here to prevent speculative ** fetches of remote->slot_queue_recycle from overtaking the ** mutex. */ rmb(); slot_queue_recycle = state->remote->slot_queue_recycle; state->remote->slot_queue[slot_queue_recycle & VCHIQ_SLOT_QUEUE_MASK] = SLOT_INDEX_FROM_INFO(state, slot_info); state->remote->slot_queue_recycle = slot_queue_recycle + 1; vchiq_log_info(vchiq_core_log_level, "%d: release_slot %d - recycle->%x", state->id, SLOT_INDEX_FROM_INFO(state, slot_info), state->remote->slot_queue_recycle); /* A write barrier is necessary, but remote_event_signal ** contains one. */ remote_event_signal(&state->remote->recycle); } lmutex_unlock(&state->recycle_mutex); } /* Called by the slot handler - don't hold the bulk mutex */ static VCHIQ_STATUS_T notify_bulks(VCHIQ_SERVICE_T *service, VCHIQ_BULK_QUEUE_T *queue, int retry_poll) { VCHIQ_STATUS_T status = VCHIQ_SUCCESS; vchiq_log_trace(vchiq_core_log_level, "%d: nb:%d %cx - p=%x rn=%x r=%x", service->state->id, service->localport, (queue == &service->bulk_tx) ? 't' : 'r', queue->process, queue->remote_notify, queue->remove); if (service->state->is_master) { while (queue->remote_notify != queue->process) { VCHIQ_BULK_T *bulk = &queue->bulks[BULK_INDEX(queue->remote_notify)]; int msgtype = (bulk->dir == VCHIQ_BULK_TRANSMIT) ? VCHIQ_MSG_BULK_RX_DONE : VCHIQ_MSG_BULK_TX_DONE; int msgid = VCHIQ_MAKE_MSG(msgtype, service->localport, service->remoteport); VCHIQ_ELEMENT_T element = { &bulk->actual, 4 }; /* Only reply to non-dummy bulk requests */ if (bulk->remote_data) { status = queue_message(service->state, NULL, msgid, &element, 1, 4, 0); if (status != VCHIQ_SUCCESS) break; } queue->remote_notify++; } } else { queue->remote_notify = queue->process; } if (status == VCHIQ_SUCCESS) { while (queue->remove != queue->remote_notify) { VCHIQ_BULK_T *bulk = &queue->bulks[BULK_INDEX(queue->remove)]; /* Only generate callbacks for non-dummy bulk ** requests, and non-terminated services */ if (bulk->data && service->instance) { if (bulk->actual != VCHIQ_BULK_ACTUAL_ABORTED) { if (bulk->dir == VCHIQ_BULK_TRANSMIT) { VCHIQ_SERVICE_STATS_INC(service, bulk_tx_count); VCHIQ_SERVICE_STATS_ADD(service, bulk_tx_bytes, bulk->actual); } else { VCHIQ_SERVICE_STATS_INC(service, bulk_rx_count); VCHIQ_SERVICE_STATS_ADD(service, bulk_rx_bytes, bulk->actual); } } else { VCHIQ_SERVICE_STATS_INC(service, bulk_aborted_count); } if (bulk->mode == VCHIQ_BULK_MODE_BLOCKING) { struct bulk_waiter *waiter; spin_lock(&bulk_waiter_spinlock); waiter = bulk->userdata; if (waiter) { waiter->actual = bulk->actual; up(&waiter->event); } spin_unlock(&bulk_waiter_spinlock); } else if (bulk->mode == VCHIQ_BULK_MODE_CALLBACK) { VCHIQ_REASON_T reason = (bulk->dir == VCHIQ_BULK_TRANSMIT) ? ((bulk->actual == VCHIQ_BULK_ACTUAL_ABORTED) ? VCHIQ_BULK_TRANSMIT_ABORTED : VCHIQ_BULK_TRANSMIT_DONE) : ((bulk->actual == VCHIQ_BULK_ACTUAL_ABORTED) ? VCHIQ_BULK_RECEIVE_ABORTED : VCHIQ_BULK_RECEIVE_DONE); status = make_service_callback(service, reason, NULL, bulk->userdata); if (status == VCHIQ_RETRY) break; } } queue->remove++; up(&service->bulk_remove_event); } if (!retry_poll) status = VCHIQ_SUCCESS; } if (status == VCHIQ_RETRY) request_poll(service->state, service, (queue == &service->bulk_tx) ? VCHIQ_POLL_TXNOTIFY : VCHIQ_POLL_RXNOTIFY); return status; } /* Called by the slot handler thread */ static void poll_services(VCHIQ_STATE_T *state) { int group, i; for (group = 0; group < BITSET_SIZE(state->unused_service); group++) { uint32_t flags; flags = atomic_xchg(&state->poll_services[group], 0); for (i = 0; flags; i++) { if (flags & (1 << i)) { VCHIQ_SERVICE_T *service = find_service_by_port(state, (group<<5) + i); uint32_t service_flags; flags &= ~(1 << i); if (!service) continue; service_flags = atomic_xchg(&service->poll_flags, 0); if (service_flags & (1 << VCHIQ_POLL_REMOVE)) { vchiq_log_info(vchiq_core_log_level, "%d: ps - remove %d<->%d", state->id, service->localport, service->remoteport); /* Make it look like a client, because it must be removed and not left in the LISTENING state. */ service->public_fourcc = VCHIQ_FOURCC_INVALID; if (vchiq_close_service_internal( service, 0/*!close_recvd*/) != VCHIQ_SUCCESS) request_poll(state, service, VCHIQ_POLL_REMOVE); } else if (service_flags & (1 << VCHIQ_POLL_TERMINATE)) { vchiq_log_info(vchiq_core_log_level, "%d: ps - terminate %d<->%d", state->id, service->localport, service->remoteport); if (vchiq_close_service_internal( service, 0/*!close_recvd*/) != VCHIQ_SUCCESS) request_poll(state, service, VCHIQ_POLL_TERMINATE); } if (service_flags & (1 << VCHIQ_POLL_TXNOTIFY)) notify_bulks(service, &service->bulk_tx, 1/*retry_poll*/); if (service_flags & (1 << VCHIQ_POLL_RXNOTIFY)) notify_bulks(service, &service->bulk_rx, 1/*retry_poll*/); unlock_service(service); } } } } /* Called by the slot handler or application threads, holding the bulk mutex. */ static int resolve_bulks(VCHIQ_SERVICE_T *service, VCHIQ_BULK_QUEUE_T *queue) { VCHIQ_STATE_T *state = service->state; int resolved = 0; int rc; while ((queue->process != queue->local_insert) && (queue->process != queue->remote_insert)) { VCHIQ_BULK_T *bulk = &queue->bulks[BULK_INDEX(queue->process)]; vchiq_log_trace(vchiq_core_log_level, "%d: rb:%d %cx - li=%x ri=%x p=%x", state->id, service->localport, (queue == &service->bulk_tx) ? 't' : 'r', queue->local_insert, queue->remote_insert, queue->process); WARN_ON(!((int)(queue->local_insert - queue->process) > 0)); WARN_ON(!((int)(queue->remote_insert - queue->process) > 0)); rc = lmutex_lock_interruptible(&state->bulk_transfer_mutex); if (rc != 0) break; vchiq_transfer_bulk(bulk); lmutex_unlock(&state->bulk_transfer_mutex); if (SRVTRACE_ENABLED(service, VCHIQ_LOG_INFO)) { const char *header = (queue == &service->bulk_tx) ? "Send Bulk to" : "Recv Bulk from"; if (bulk->actual != VCHIQ_BULK_ACTUAL_ABORTED) vchiq_log_info(SRVTRACE_LEVEL(service), "%s %c%c%c%c d:%d len:%d %x<->%x", header, VCHIQ_FOURCC_AS_4CHARS( service->base.fourcc), service->remoteport, bulk->size, (unsigned int)bulk->data, (unsigned int)bulk->remote_data); else vchiq_log_info(SRVTRACE_LEVEL(service), "%s %c%c%c%c d:%d ABORTED - tx len:%d," " rx len:%d %x<->%x", header, VCHIQ_FOURCC_AS_4CHARS( service->base.fourcc), service->remoteport, bulk->size, bulk->remote_size, (unsigned int)bulk->data, (unsigned int)bulk->remote_data); } vchiq_complete_bulk(bulk); queue->process++; resolved++; } return resolved; } /* Called with the bulk_mutex held */ static void abort_outstanding_bulks(VCHIQ_SERVICE_T *service, VCHIQ_BULK_QUEUE_T *queue) { int is_tx = (queue == &service->bulk_tx); vchiq_log_trace(vchiq_core_log_level, "%d: aob:%d %cx - li=%x ri=%x p=%x", service->state->id, service->localport, is_tx ? 't' : 'r', queue->local_insert, queue->remote_insert, queue->process); WARN_ON(!((int)(queue->local_insert - queue->process) >= 0)); WARN_ON(!((int)(queue->remote_insert - queue->process) >= 0)); while ((queue->process != queue->local_insert) || (queue->process != queue->remote_insert)) { VCHIQ_BULK_T *bulk = &queue->bulks[BULK_INDEX(queue->process)]; if (queue->process == queue->remote_insert) { /* fabricate a matching dummy bulk */ bulk->remote_data = NULL; bulk->remote_size = 0; queue->remote_insert++; } if (queue->process != queue->local_insert) { vchiq_complete_bulk(bulk); vchiq_log_info(SRVTRACE_LEVEL(service), "%s %c%c%c%c d:%d ABORTED - tx len:%d, " "rx len:%d", is_tx ? "Send Bulk to" : "Recv Bulk from", VCHIQ_FOURCC_AS_4CHARS(service->base.fourcc), service->remoteport, bulk->size, bulk->remote_size); } else { /* fabricate a matching dummy bulk */ bulk->data = NULL; bulk->size = 0; bulk->actual = VCHIQ_BULK_ACTUAL_ABORTED; bulk->dir = is_tx ? VCHIQ_BULK_TRANSMIT : VCHIQ_BULK_RECEIVE; queue->local_insert++; } queue->process++; } } /* Called from the slot handler thread */ static void pause_bulks(VCHIQ_STATE_T *state) { if (unlikely(atomic_inc_return(&pause_bulks_count) != 1)) { WARN_ON_ONCE(1); atomic_set(&pause_bulks_count, 1); return; } /* Block bulk transfers from all services */ lmutex_lock(&state->bulk_transfer_mutex); } /* Called from the slot handler thread */ static void resume_bulks(VCHIQ_STATE_T *state) { int i; if (unlikely(atomic_dec_return(&pause_bulks_count) != 0)) { WARN_ON_ONCE(1); atomic_set(&pause_bulks_count, 0); return; } /* Allow bulk transfers from all services */ lmutex_unlock(&state->bulk_transfer_mutex); if (state->deferred_bulks == 0) return; /* Deal with any bulks which had to be deferred due to being in * paused state. Don't try to match up to number of deferred bulks * in case we've had something come and close the service in the * interim - just process all bulk queues for all services */ vchiq_log_info(vchiq_core_log_level, "%s: processing %d deferred bulks", __func__, state->deferred_bulks); for (i = 0; i < state->unused_service; i++) { VCHIQ_SERVICE_T *service = state->services[i]; int resolved_rx = 0; int resolved_tx = 0; if (!service || (service->srvstate != VCHIQ_SRVSTATE_OPEN)) continue; lmutex_lock(&service->bulk_mutex); resolved_rx = resolve_bulks(service, &service->bulk_rx); resolved_tx = resolve_bulks(service, &service->bulk_tx); lmutex_unlock(&service->bulk_mutex); if (resolved_rx) notify_bulks(service, &service->bulk_rx, 1); if (resolved_tx) notify_bulks(service, &service->bulk_tx, 1); } state->deferred_bulks = 0; } static int parse_open(VCHIQ_STATE_T *state, VCHIQ_HEADER_T *header) { VCHIQ_SERVICE_T *service = NULL; int msgid, size; unsigned int localport, remoteport; msgid = header->msgid; size = header->size; //int type = VCHIQ_MSG_TYPE(msgid); localport = VCHIQ_MSG_DSTPORT(msgid); remoteport = VCHIQ_MSG_SRCPORT(msgid); if (size >= sizeof(struct vchiq_open_payload)) { const struct vchiq_open_payload *payload = (struct vchiq_open_payload *)header->data; unsigned int fourcc; fourcc = payload->fourcc; vchiq_log_info(vchiq_core_log_level, "%d: prs OPEN@%x (%d->'%c%c%c%c')", state->id, (unsigned int)header, localport, VCHIQ_FOURCC_AS_4CHARS(fourcc)); service = get_listening_service(state, fourcc); if (service) { /* A matching service exists */ short version = payload->version; short version_min = payload->version_min; if ((service->version < version_min) || (version < service->version_min)) { /* Version mismatch */ vchiq_loud_error_header(); vchiq_loud_error("%d: service %d (%c%c%c%c) " "version mismatch - local (%d, min %d)" " vs. remote (%d, min %d)", state->id, service->localport, VCHIQ_FOURCC_AS_4CHARS(fourcc), service->version, service->version_min, version, version_min); vchiq_loud_error_footer(); unlock_service(service); service = NULL; goto fail_open; } service->peer_version = version; if (service->srvstate == VCHIQ_SRVSTATE_LISTENING) { struct vchiq_openack_payload ack_payload = { service->version }; VCHIQ_ELEMENT_T body = { &ack_payload, sizeof(ack_payload) }; if (state->version_common < VCHIQ_VERSION_SYNCHRONOUS_MODE) service->sync = 0; /* Acknowledge the OPEN */ if (service->sync && (state->version_common >= VCHIQ_VERSION_SYNCHRONOUS_MODE)) { if (queue_message_sync(state, NULL, VCHIQ_MAKE_MSG( VCHIQ_MSG_OPENACK, service->localport, remoteport), &body, 1, sizeof(ack_payload), 0) == VCHIQ_RETRY) goto bail_not_ready; } else { if (queue_message(state, NULL, VCHIQ_MAKE_MSG( VCHIQ_MSG_OPENACK, service->localport, remoteport), &body, 1, sizeof(ack_payload), 0) == VCHIQ_RETRY) goto bail_not_ready; } /* The service is now open */ vchiq_set_service_state(service, service->sync ? VCHIQ_SRVSTATE_OPENSYNC : VCHIQ_SRVSTATE_OPEN); } service->remoteport = remoteport; service->client_id = ((int *)header->data)[1]; if (make_service_callback(service, VCHIQ_SERVICE_OPENED, NULL, NULL) == VCHIQ_RETRY) { /* Bail out if not ready */ service->remoteport = VCHIQ_PORT_FREE; goto bail_not_ready; } /* Success - the message has been dealt with */ unlock_service(service); return 1; } } fail_open: /* No available service, or an invalid request - send a CLOSE */ if (queue_message(state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_CLOSE, 0, VCHIQ_MSG_SRCPORT(msgid)), NULL, 0, 0, 0) == VCHIQ_RETRY) goto bail_not_ready; return 1; bail_not_ready: if (service) unlock_service(service); return 0; } /* Called by the slot handler thread */ static void parse_rx_slots(VCHIQ_STATE_T *state) { VCHIQ_SHARED_STATE_T *remote = state->remote; VCHIQ_SERVICE_T *service = NULL; int tx_pos; DEBUG_INITIALISE(state->local) tx_pos = remote->tx_pos; while (state->rx_pos != tx_pos) { VCHIQ_HEADER_T *header; int msgid, size; int type; unsigned int localport, remoteport; DEBUG_TRACE(PARSE_LINE); if (!state->rx_data) { int rx_index; WARN_ON(!((state->rx_pos & VCHIQ_SLOT_MASK) == 0)); rx_index = remote->slot_queue[ SLOT_QUEUE_INDEX_FROM_POS(state->rx_pos) & VCHIQ_SLOT_QUEUE_MASK]; state->rx_data = (char *)SLOT_DATA_FROM_INDEX(state, rx_index); state->rx_info = SLOT_INFO_FROM_INDEX(state, rx_index); /* Initialise use_count to one, and increment ** release_count at the end of the slot to avoid ** releasing the slot prematurely. */ state->rx_info->use_count = 1; state->rx_info->release_count = 0; } header = (VCHIQ_HEADER_T *)(state->rx_data + (state->rx_pos & VCHIQ_SLOT_MASK)); DEBUG_VALUE(PARSE_HEADER, (int)header); msgid = header->msgid; DEBUG_VALUE(PARSE_MSGID, msgid); size = header->size; type = VCHIQ_MSG_TYPE(msgid); localport = VCHIQ_MSG_DSTPORT(msgid); remoteport = VCHIQ_MSG_SRCPORT(msgid); if (type != VCHIQ_MSG_DATA) VCHIQ_STATS_INC(state, ctrl_rx_count); switch (type) { case VCHIQ_MSG_OPENACK: case VCHIQ_MSG_CLOSE: case VCHIQ_MSG_DATA: case VCHIQ_MSG_BULK_RX: case VCHIQ_MSG_BULK_TX: case VCHIQ_MSG_BULK_RX_DONE: case VCHIQ_MSG_BULK_TX_DONE: service = find_service_by_port(state, localport); if ((!service || ((service->remoteport != remoteport) && (service->remoteport != VCHIQ_PORT_FREE))) && (localport == 0) && (type == VCHIQ_MSG_CLOSE)) { /* This could be a CLOSE from a client which hadn't yet received the OPENACK - look for the connected service */ if (service) unlock_service(service); service = get_connected_service(state, remoteport); if (service) vchiq_log_warning(vchiq_core_log_level, "%d: prs %s@%x (%d->%d) - " "found connected service %d", state->id, msg_type_str(type), (unsigned int)header, remoteport, localport, service->localport); } if (!service) { vchiq_log_error(vchiq_core_log_level, "%d: prs %s@%x (%d->%d) - " "invalid/closed service %d", state->id, msg_type_str(type), (unsigned int)header, remoteport, localport, localport); goto skip_message; } break; default: break; } if (SRVTRACE_ENABLED(service, VCHIQ_LOG_INFO)) { int svc_fourcc; svc_fourcc = service ? service->base.fourcc : VCHIQ_MAKE_FOURCC('?', '?', '?', '?'); vchiq_log_info(SRVTRACE_LEVEL(service), "Rcvd Msg %s(%u) from %c%c%c%c s:%d d:%d " "len:%d", msg_type_str(type), type, VCHIQ_FOURCC_AS_4CHARS(svc_fourcc), remoteport, localport, size); if (size > 0) vchiq_log_dump_mem("Rcvd", 0, header->data, - min(64, size)); + min(16, size)); } if (((unsigned int)header & VCHIQ_SLOT_MASK) + calc_stride(size) > VCHIQ_SLOT_SIZE) { vchiq_log_error(vchiq_core_log_level, "header %x (msgid %x) - size %x too big for " "slot", (unsigned int)header, (unsigned int)msgid, (unsigned int)size); WARN(1, "oversized for slot\n"); } switch (type) { case VCHIQ_MSG_OPEN: WARN_ON(!(VCHIQ_MSG_DSTPORT(msgid) == 0)); if (!parse_open(state, header)) goto bail_not_ready; break; case VCHIQ_MSG_OPENACK: if (size >= sizeof(struct vchiq_openack_payload)) { const struct vchiq_openack_payload *payload = (struct vchiq_openack_payload *) header->data; service->peer_version = payload->version; } vchiq_log_info(vchiq_core_log_level, "%d: prs OPENACK@%x,%x (%d->%d) v:%d", state->id, (unsigned int)header, size, remoteport, localport, service->peer_version); if (service->srvstate == VCHIQ_SRVSTATE_OPENING) { service->remoteport = remoteport; vchiq_set_service_state(service, VCHIQ_SRVSTATE_OPEN); up(&service->remove_event); } else vchiq_log_error(vchiq_core_log_level, "OPENACK received in state %s", srvstate_names[service->srvstate]); break; case VCHIQ_MSG_CLOSE: WARN_ON(size != 0); /* There should be no data */ vchiq_log_info(vchiq_core_log_level, "%d: prs CLOSE@%x (%d->%d)", state->id, (unsigned int)header, remoteport, localport); mark_service_closing_internal(service, 1); if (vchiq_close_service_internal(service, 1/*close_recvd*/) == VCHIQ_RETRY) goto bail_not_ready; vchiq_log_info(vchiq_core_log_level, "Close Service %c%c%c%c s:%u d:%d", VCHIQ_FOURCC_AS_4CHARS(service->base.fourcc), service->localport, service->remoteport); break; case VCHIQ_MSG_DATA: vchiq_log_info(vchiq_core_log_level, "%d: prs DATA@%x,%x (%d->%d)", state->id, (unsigned int)header, size, remoteport, localport); if ((service->remoteport == remoteport) && (service->srvstate == VCHIQ_SRVSTATE_OPEN)) { header->msgid = msgid | VCHIQ_MSGID_CLAIMED; claim_slot(state->rx_info); DEBUG_TRACE(PARSE_LINE); if (make_service_callback(service, VCHIQ_MESSAGE_AVAILABLE, header, NULL) == VCHIQ_RETRY) { DEBUG_TRACE(PARSE_LINE); goto bail_not_ready; } VCHIQ_SERVICE_STATS_INC(service, ctrl_rx_count); VCHIQ_SERVICE_STATS_ADD(service, ctrl_rx_bytes, size); } else { VCHIQ_STATS_INC(state, error_count); } break; case VCHIQ_MSG_CONNECT: vchiq_log_info(vchiq_core_log_level, "%d: prs CONNECT@%x", state->id, (unsigned int)header); state->version_common = ((VCHIQ_SLOT_ZERO_T *) state->slot_data)->version; up(&state->connect); break; case VCHIQ_MSG_BULK_RX: case VCHIQ_MSG_BULK_TX: { VCHIQ_BULK_QUEUE_T *queue; WARN_ON(!state->is_master); queue = (type == VCHIQ_MSG_BULK_RX) ? &service->bulk_tx : &service->bulk_rx; if ((service->remoteport == remoteport) && (service->srvstate == VCHIQ_SRVSTATE_OPEN)) { VCHIQ_BULK_T *bulk; int resolved = 0; DEBUG_TRACE(PARSE_LINE); if (lmutex_lock_interruptible( &service->bulk_mutex) != 0) { DEBUG_TRACE(PARSE_LINE); goto bail_not_ready; } WARN_ON(!(queue->remote_insert < queue->remove + VCHIQ_NUM_SERVICE_BULKS)); bulk = &queue->bulks[ BULK_INDEX(queue->remote_insert)]; bulk->remote_data = (void *)((int *)header->data)[0]; bulk->remote_size = ((int *)header->data)[1]; wmb(); vchiq_log_info(vchiq_core_log_level, "%d: prs %s@%x (%d->%d) %x@%x", state->id, msg_type_str(type), (unsigned int)header, remoteport, localport, bulk->remote_size, (unsigned int)bulk->remote_data); queue->remote_insert++; if (atomic_read(&pause_bulks_count)) { state->deferred_bulks++; vchiq_log_info(vchiq_core_log_level, "%s: deferring bulk (%d)", __func__, state->deferred_bulks); if (state->conn_state != VCHIQ_CONNSTATE_PAUSE_SENT) vchiq_log_error( vchiq_core_log_level, "%s: bulks paused in " "unexpected state %s", __func__, conn_state_names[ state->conn_state]); } else if (state->conn_state == VCHIQ_CONNSTATE_CONNECTED) { DEBUG_TRACE(PARSE_LINE); resolved = resolve_bulks(service, queue); } lmutex_unlock(&service->bulk_mutex); if (resolved) notify_bulks(service, queue, 1/*retry_poll*/); } } break; case VCHIQ_MSG_BULK_RX_DONE: case VCHIQ_MSG_BULK_TX_DONE: WARN_ON(state->is_master); if ((service->remoteport == remoteport) && (service->srvstate != VCHIQ_SRVSTATE_FREE)) { VCHIQ_BULK_QUEUE_T *queue; VCHIQ_BULK_T *bulk; queue = (type == VCHIQ_MSG_BULK_RX_DONE) ? &service->bulk_rx : &service->bulk_tx; DEBUG_TRACE(PARSE_LINE); if (lmutex_lock_interruptible( &service->bulk_mutex) != 0) { DEBUG_TRACE(PARSE_LINE); goto bail_not_ready; } if ((int)(queue->remote_insert - queue->local_insert) >= 0) { vchiq_log_error(vchiq_core_log_level, "%d: prs %s@%x (%d->%d) " "unexpected (ri=%d,li=%d)", state->id, msg_type_str(type), (unsigned int)header, remoteport, localport, queue->remote_insert, queue->local_insert); lmutex_unlock(&service->bulk_mutex); break; } BUG_ON(queue->process == queue->local_insert); BUG_ON(queue->process != queue->remote_insert); bulk = &queue->bulks[ BULK_INDEX(queue->remote_insert)]; bulk->actual = *(int *)header->data; queue->remote_insert++; vchiq_log_info(vchiq_core_log_level, "%d: prs %s@%x (%d->%d) %x@%x", state->id, msg_type_str(type), (unsigned int)header, remoteport, localport, bulk->actual, (unsigned int)bulk->data); vchiq_log_trace(vchiq_core_log_level, "%d: prs:%d %cx li=%x ri=%x p=%x", state->id, localport, (type == VCHIQ_MSG_BULK_RX_DONE) ? 'r' : 't', queue->local_insert, queue->remote_insert, queue->process); DEBUG_TRACE(PARSE_LINE); WARN_ON(queue->process == queue->local_insert); vchiq_complete_bulk(bulk); queue->process++; lmutex_unlock(&service->bulk_mutex); DEBUG_TRACE(PARSE_LINE); notify_bulks(service, queue, 1/*retry_poll*/); DEBUG_TRACE(PARSE_LINE); } break; case VCHIQ_MSG_PADDING: vchiq_log_trace(vchiq_core_log_level, "%d: prs PADDING@%x,%x", state->id, (unsigned int)header, size); break; case VCHIQ_MSG_PAUSE: /* If initiated, signal the application thread */ vchiq_log_trace(vchiq_core_log_level, "%d: prs PAUSE@%x,%x", state->id, (unsigned int)header, size); if (state->conn_state == VCHIQ_CONNSTATE_PAUSED) { vchiq_log_error(vchiq_core_log_level, "%d: PAUSE received in state PAUSED", state->id); break; } if (state->conn_state != VCHIQ_CONNSTATE_PAUSE_SENT) { /* Send a PAUSE in response */ if (queue_message(state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_PAUSE, 0, 0), NULL, 0, 0, QMFLAGS_NO_MUTEX_UNLOCK) == VCHIQ_RETRY) goto bail_not_ready; if (state->is_master) pause_bulks(state); } /* At this point slot_mutex is held */ vchiq_set_conn_state(state, VCHIQ_CONNSTATE_PAUSED); vchiq_platform_paused(state); break; case VCHIQ_MSG_RESUME: vchiq_log_trace(vchiq_core_log_level, "%d: prs RESUME@%x,%x", state->id, (unsigned int)header, size); /* Release the slot mutex */ lmutex_unlock(&state->slot_mutex); if (state->is_master) resume_bulks(state); vchiq_set_conn_state(state, VCHIQ_CONNSTATE_CONNECTED); vchiq_platform_resumed(state); break; case VCHIQ_MSG_REMOTE_USE: vchiq_on_remote_use(state); break; case VCHIQ_MSG_REMOTE_RELEASE: vchiq_on_remote_release(state); break; case VCHIQ_MSG_REMOTE_USE_ACTIVE: vchiq_on_remote_use_active(state); break; default: vchiq_log_error(vchiq_core_log_level, "%d: prs invalid msgid %x@%x,%x", state->id, msgid, (unsigned int)header, size); WARN(1, "invalid message\n"); break; } skip_message: if (service) { unlock_service(service); service = NULL; } state->rx_pos += calc_stride(size); DEBUG_TRACE(PARSE_LINE); /* Perform some housekeeping when the end of the slot is ** reached. */ if ((state->rx_pos & VCHIQ_SLOT_MASK) == 0) { /* Remove the extra reference count. */ release_slot(state, state->rx_info, NULL, NULL); state->rx_data = NULL; } } bail_not_ready: if (service) unlock_service(service); } /* Called by the slot handler thread */ int slot_handler_func(void *v); int slot_handler_func(void *v) { VCHIQ_STATE_T *state = (VCHIQ_STATE_T *) v; VCHIQ_SHARED_STATE_T *local = state->local; DEBUG_INITIALISE(local) while (1) { DEBUG_COUNT(SLOT_HANDLER_COUNT); DEBUG_TRACE(SLOT_HANDLER_LINE); remote_event_wait(&local->trigger); rmb(); DEBUG_TRACE(SLOT_HANDLER_LINE); if (state->poll_needed) { /* Check if we need to suspend - may change our * conn_state */ vchiq_platform_check_suspend(state); state->poll_needed = 0; /* Handle service polling and other rare conditions here ** out of the mainline code */ switch (state->conn_state) { case VCHIQ_CONNSTATE_CONNECTED: /* Poll the services as requested */ poll_services(state); break; case VCHIQ_CONNSTATE_PAUSING: if (state->is_master) pause_bulks(state); if (queue_message(state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_PAUSE, 0, 0), NULL, 0, 0, QMFLAGS_NO_MUTEX_UNLOCK) != VCHIQ_RETRY) { vchiq_set_conn_state(state, VCHIQ_CONNSTATE_PAUSE_SENT); } else { if (state->is_master) resume_bulks(state); /* Retry later */ state->poll_needed = 1; } break; case VCHIQ_CONNSTATE_PAUSED: vchiq_platform_resume(state); break; case VCHIQ_CONNSTATE_RESUMING: if (queue_message(state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_RESUME, 0, 0), NULL, 0, 0, QMFLAGS_NO_MUTEX_LOCK) != VCHIQ_RETRY) { if (state->is_master) resume_bulks(state); vchiq_set_conn_state(state, VCHIQ_CONNSTATE_CONNECTED); vchiq_platform_resumed(state); } else { /* This should really be impossible, ** since the PAUSE should have flushed ** through outstanding messages. */ vchiq_log_error(vchiq_core_log_level, "Failed to send RESUME " "message"); BUG(); } break; case VCHIQ_CONNSTATE_PAUSE_TIMEOUT: case VCHIQ_CONNSTATE_RESUME_TIMEOUT: vchiq_platform_handle_timeout(state); break; default: break; } } DEBUG_TRACE(SLOT_HANDLER_LINE); parse_rx_slots(state); } return 0; } /* Called by the recycle thread */ int recycle_func(void *v); int recycle_func(void *v) { VCHIQ_STATE_T *state = (VCHIQ_STATE_T *) v; VCHIQ_SHARED_STATE_T *local = state->local; while (1) { remote_event_wait(&local->recycle); process_free_queue(state); } return 0; } /* Called by the sync thread */ int sync_func(void *v); int sync_func(void *v) { VCHIQ_STATE_T *state = (VCHIQ_STATE_T *) v; VCHIQ_SHARED_STATE_T *local = state->local; VCHIQ_HEADER_T *header = (VCHIQ_HEADER_T *)SLOT_DATA_FROM_INDEX(state, state->remote->slot_sync); while (1) { VCHIQ_SERVICE_T *service; int msgid, size; int type; unsigned int localport, remoteport; remote_event_wait(&local->sync_trigger); rmb(); msgid = header->msgid; size = header->size; type = VCHIQ_MSG_TYPE(msgid); localport = VCHIQ_MSG_DSTPORT(msgid); remoteport = VCHIQ_MSG_SRCPORT(msgid); service = find_service_by_port(state, localport); if (!service) { vchiq_log_error(vchiq_sync_log_level, "%d: sf %s@%x (%d->%d) - " "invalid/closed service %d", state->id, msg_type_str(type), (unsigned int)header, remoteport, localport, localport); release_message_sync(state, header); continue; } if (vchiq_sync_log_level >= VCHIQ_LOG_TRACE) { int svc_fourcc; svc_fourcc = service ? service->base.fourcc : VCHIQ_MAKE_FOURCC('?', '?', '?', '?'); vchiq_log_trace(vchiq_sync_log_level, "Rcvd Msg %s from %c%c%c%c s:%d d:%d len:%d", msg_type_str(type), VCHIQ_FOURCC_AS_4CHARS(svc_fourcc), remoteport, localport, size); if (size > 0) vchiq_log_dump_mem("Rcvd", 0, header->data, - min(64, size)); + min(16, size)); } switch (type) { case VCHIQ_MSG_OPENACK: if (size >= sizeof(struct vchiq_openack_payload)) { const struct vchiq_openack_payload *payload = (struct vchiq_openack_payload *) header->data; service->peer_version = payload->version; } vchiq_log_info(vchiq_sync_log_level, "%d: sf OPENACK@%x,%x (%d->%d) v:%d", state->id, (unsigned int)header, size, remoteport, localport, service->peer_version); if (service->srvstate == VCHIQ_SRVSTATE_OPENING) { service->remoteport = remoteport; vchiq_set_service_state(service, VCHIQ_SRVSTATE_OPENSYNC); service->sync = 1; up(&service->remove_event); } release_message_sync(state, header); break; case VCHIQ_MSG_DATA: vchiq_log_trace(vchiq_sync_log_level, "%d: sf DATA@%x,%x (%d->%d)", state->id, (unsigned int)header, size, remoteport, localport); if ((service->remoteport == remoteport) && (service->srvstate == VCHIQ_SRVSTATE_OPENSYNC)) { if (make_service_callback(service, VCHIQ_MESSAGE_AVAILABLE, header, NULL) == VCHIQ_RETRY) vchiq_log_error(vchiq_sync_log_level, "synchronous callback to " "service %d returns " "VCHIQ_RETRY", localport); } break; default: vchiq_log_error(vchiq_sync_log_level, "%d: sf unexpected msgid %x@%x,%x", state->id, msgid, (unsigned int)header, size); release_message_sync(state, header); break; } unlock_service(service); } return 0; } static void init_bulk_queue(VCHIQ_BULK_QUEUE_T *queue) { queue->local_insert = 0; queue->remote_insert = 0; queue->process = 0; queue->remote_notify = 0; queue->remove = 0; } inline const char * get_conn_state_name(VCHIQ_CONNSTATE_T conn_state) { return conn_state_names[conn_state]; } VCHIQ_SLOT_ZERO_T * vchiq_init_slots(void *mem_base, int mem_size) { int mem_align = (VCHIQ_SLOT_SIZE - (int)mem_base) & VCHIQ_SLOT_MASK; VCHIQ_SLOT_ZERO_T *slot_zero = (VCHIQ_SLOT_ZERO_T *)((char *)mem_base + mem_align); int num_slots = (mem_size - mem_align)/VCHIQ_SLOT_SIZE; int first_data_slot = VCHIQ_SLOT_ZERO_SLOTS; /* Ensure there is enough memory to run an absolutely minimum system */ num_slots -= first_data_slot; if (num_slots < 4) { vchiq_log_error(vchiq_core_log_level, "vchiq_init_slots - insufficient memory %x bytes", mem_size); return NULL; } memset(slot_zero, 0, sizeof(VCHIQ_SLOT_ZERO_T)); slot_zero->magic = VCHIQ_MAGIC; slot_zero->version = VCHIQ_VERSION; slot_zero->version_min = VCHIQ_VERSION_MIN; slot_zero->slot_zero_size = sizeof(VCHIQ_SLOT_ZERO_T); slot_zero->slot_size = VCHIQ_SLOT_SIZE; slot_zero->max_slots = VCHIQ_MAX_SLOTS; slot_zero->max_slots_per_side = VCHIQ_MAX_SLOTS_PER_SIDE; slot_zero->master.slot_sync = first_data_slot; slot_zero->master.slot_first = first_data_slot + 1; slot_zero->master.slot_last = first_data_slot + (num_slots/2) - 1; slot_zero->slave.slot_sync = first_data_slot + (num_slots/2); slot_zero->slave.slot_first = first_data_slot + (num_slots/2) + 1; slot_zero->slave.slot_last = first_data_slot + num_slots - 1; return slot_zero; } VCHIQ_STATUS_T vchiq_init_state(VCHIQ_STATE_T *state, VCHIQ_SLOT_ZERO_T *slot_zero, int is_master) { VCHIQ_SHARED_STATE_T *local; VCHIQ_SHARED_STATE_T *remote; VCHIQ_STATUS_T status; char threadname[10]; static int id; int i; /* Check the input configuration */ if (slot_zero->magic != VCHIQ_MAGIC) { vchiq_loud_error_header(); vchiq_loud_error("Invalid VCHIQ magic value found."); vchiq_loud_error("slot_zero=%x: magic=%x (expected %x)", (unsigned int)slot_zero, slot_zero->magic, VCHIQ_MAGIC); vchiq_loud_error_footer(); return VCHIQ_ERROR; } vchiq_log_warning(vchiq_core_log_level, "local ver %d (min %d), remote ver %d.", VCHIQ_VERSION, VCHIQ_VERSION_MIN, slot_zero->version); if (slot_zero->version < VCHIQ_VERSION_MIN) { vchiq_loud_error_header(); vchiq_loud_error("Incompatible VCHIQ versions found."); vchiq_loud_error("slot_zero=%x: VideoCore version=%d " "(minimum %d)", (unsigned int)slot_zero, slot_zero->version, VCHIQ_VERSION_MIN); vchiq_loud_error("Restart with a newer VideoCore image."); vchiq_loud_error_footer(); return VCHIQ_ERROR; } if (VCHIQ_VERSION < slot_zero->version_min) { vchiq_loud_error_header(); vchiq_loud_error("Incompatible VCHIQ versions found."); vchiq_loud_error("slot_zero=%x: version=%d (VideoCore " "minimum %d)", (unsigned int)slot_zero, VCHIQ_VERSION, slot_zero->version_min); vchiq_loud_error("Restart with a newer kernel."); vchiq_loud_error_footer(); return VCHIQ_ERROR; } if ((slot_zero->slot_zero_size != sizeof(VCHIQ_SLOT_ZERO_T)) || (slot_zero->slot_size != VCHIQ_SLOT_SIZE) || (slot_zero->max_slots != VCHIQ_MAX_SLOTS) || (slot_zero->max_slots_per_side != VCHIQ_MAX_SLOTS_PER_SIDE)) { vchiq_loud_error_header(); if (slot_zero->slot_zero_size != sizeof(VCHIQ_SLOT_ZERO_T)) vchiq_loud_error("slot_zero=%x: slot_zero_size=%x " "(expected %zx)", (unsigned int)slot_zero, slot_zero->slot_zero_size, sizeof(VCHIQ_SLOT_ZERO_T)); if (slot_zero->slot_size != VCHIQ_SLOT_SIZE) vchiq_loud_error("slot_zero=%x: slot_size=%d " "(expected %d", (unsigned int)slot_zero, slot_zero->slot_size, VCHIQ_SLOT_SIZE); if (slot_zero->max_slots != VCHIQ_MAX_SLOTS) vchiq_loud_error("slot_zero=%x: max_slots=%d " "(expected %d)", (unsigned int)slot_zero, slot_zero->max_slots, VCHIQ_MAX_SLOTS); if (slot_zero->max_slots_per_side != VCHIQ_MAX_SLOTS_PER_SIDE) vchiq_loud_error("slot_zero=%x: max_slots_per_side=%d " "(expected %d)", (unsigned int)slot_zero, slot_zero->max_slots_per_side, VCHIQ_MAX_SLOTS_PER_SIDE); vchiq_loud_error_footer(); return VCHIQ_ERROR; } if (VCHIQ_VERSION < slot_zero->version) slot_zero->version = VCHIQ_VERSION; if (is_master) { local = &slot_zero->master; remote = &slot_zero->slave; } else { local = &slot_zero->slave; remote = &slot_zero->master; } if (local->initialised) { vchiq_loud_error_header(); if (remote->initialised) vchiq_loud_error("local state has already been " "initialised"); else vchiq_loud_error("master/slave mismatch - two %ss", is_master ? "master" : "slave"); vchiq_loud_error_footer(); return VCHIQ_ERROR; } memset(state, 0, sizeof(VCHIQ_STATE_T)); state->id = id++; state->is_master = is_master; /* initialize shared state pointers */ state->local = local; state->remote = remote; state->slot_data = (VCHIQ_SLOT_T *)slot_zero; /* initialize events and mutexes */ _sema_init(&state->connect, 0); lmutex_init(&state->mutex); _sema_init(&state->trigger_event, 0); _sema_init(&state->recycle_event, 0); _sema_init(&state->sync_trigger_event, 0); _sema_init(&state->sync_release_event, 0); lmutex_init(&state->slot_mutex); lmutex_init(&state->recycle_mutex); lmutex_init(&state->sync_mutex); lmutex_init(&state->bulk_transfer_mutex); _sema_init(&state->slot_available_event, 0); _sema_init(&state->slot_remove_event, 0); _sema_init(&state->data_quota_event, 0); state->slot_queue_available = 0; for (i = 0; i < VCHIQ_MAX_SERVICES; i++) { VCHIQ_SERVICE_QUOTA_T *service_quota = &state->service_quotas[i]; _sema_init(&service_quota->quota_event, 0); } for (i = local->slot_first; i <= local->slot_last; i++) { local->slot_queue[state->slot_queue_available++] = i; up(&state->slot_available_event); } state->default_slot_quota = state->slot_queue_available/2; state->default_message_quota = min((unsigned short)(state->default_slot_quota * 256), (unsigned short)~0); state->previous_data_index = -1; state->data_use_count = 0; state->data_quota = state->slot_queue_available - 1; local->trigger.event = &state->trigger_event; remote_event_create(&local->trigger); local->tx_pos = 0; local->recycle.event = &state->recycle_event; remote_event_create(&local->recycle); local->slot_queue_recycle = state->slot_queue_available; local->sync_trigger.event = &state->sync_trigger_event; remote_event_create(&local->sync_trigger); local->sync_release.event = &state->sync_release_event; remote_event_create(&local->sync_release); /* At start-of-day, the slot is empty and available */ ((VCHIQ_HEADER_T *)SLOT_DATA_FROM_INDEX(state, local->slot_sync))->msgid = VCHIQ_MSGID_PADDING; remote_event_signal_local(&local->sync_release); local->debug[DEBUG_ENTRIES] = DEBUG_MAX; status = vchiq_platform_init_state(state); /* bring up slot handler thread */ snprintf(threadname, sizeof(threadname), "VCHIQ-%d", state->id); state->slot_handler_thread = vchiq_thread_create(&slot_handler_func, (void *)state, threadname); if (state->slot_handler_thread == NULL) { vchiq_loud_error_header(); vchiq_loud_error("couldn't create thread %s", threadname); vchiq_loud_error_footer(); return VCHIQ_ERROR; } set_user_nice(state->slot_handler_thread, -19); wake_up_process(state->slot_handler_thread); snprintf(threadname, sizeof(threadname), "VCHIQr-%d", state->id); state->recycle_thread = vchiq_thread_create(&recycle_func, (void *)state, threadname); if (state->recycle_thread == NULL) { vchiq_loud_error_header(); vchiq_loud_error("couldn't create thread %s", threadname); vchiq_loud_error_footer(); return VCHIQ_ERROR; } set_user_nice(state->recycle_thread, -19); wake_up_process(state->recycle_thread); snprintf(threadname, sizeof(threadname), "VCHIQs-%d", state->id); state->sync_thread = vchiq_thread_create(&sync_func, (void *)state, threadname); if (state->sync_thread == NULL) { vchiq_loud_error_header(); vchiq_loud_error("couldn't create thread %s", threadname); vchiq_loud_error_footer(); return VCHIQ_ERROR; } set_user_nice(state->sync_thread, -20); wake_up_process(state->sync_thread); BUG_ON(state->id >= VCHIQ_MAX_STATES); vchiq_states[state->id] = state; /* Indicate readiness to the other side */ local->initialised = 1; return status; } /* Called from application thread when a client or server service is created. */ VCHIQ_SERVICE_T * vchiq_add_service_internal(VCHIQ_STATE_T *state, const VCHIQ_SERVICE_PARAMS_T *params, int srvstate, VCHIQ_INSTANCE_T instance, VCHIQ_USERDATA_TERM_T userdata_term) { VCHIQ_SERVICE_T *service; service = kmalloc(sizeof(VCHIQ_SERVICE_T), GFP_KERNEL); if (service) { service->base.fourcc = params->fourcc; service->base.callback = params->callback; service->base.userdata = params->userdata; service->handle = VCHIQ_SERVICE_HANDLE_INVALID; service->ref_count = 1; service->srvstate = VCHIQ_SRVSTATE_FREE; service->userdata_term = userdata_term; service->localport = VCHIQ_PORT_FREE; service->remoteport = VCHIQ_PORT_FREE; service->public_fourcc = (srvstate == VCHIQ_SRVSTATE_OPENING) ? VCHIQ_FOURCC_INVALID : params->fourcc; service->client_id = 0; service->auto_close = 1; service->sync = 0; service->closing = 0; service->trace = 0; atomic_set(&service->poll_flags, 0); service->version = params->version; service->version_min = params->version_min; service->state = state; service->instance = instance; service->service_use_count = 0; init_bulk_queue(&service->bulk_tx); init_bulk_queue(&service->bulk_rx); _sema_init(&service->remove_event, 0); _sema_init(&service->bulk_remove_event, 0); lmutex_init(&service->bulk_mutex); memset(&service->stats, 0, sizeof(service->stats)); } else { vchiq_log_error(vchiq_core_log_level, "Out of memory"); } if (service) { VCHIQ_SERVICE_T **pservice = NULL; int i; /* Although it is perfectly possible to use service_spinlock ** to protect the creation of services, it is overkill as it ** disables interrupts while the array is searched. ** The only danger is of another thread trying to create a ** service - service deletion is safe. ** Therefore it is preferable to use state->mutex which, ** although slower to claim, doesn't block interrupts while ** it is held. */ lmutex_lock(&state->mutex); /* Prepare to use a previously unused service */ if (state->unused_service < VCHIQ_MAX_SERVICES) pservice = &state->services[state->unused_service]; if (srvstate == VCHIQ_SRVSTATE_OPENING) { for (i = 0; i < state->unused_service; i++) { VCHIQ_SERVICE_T *srv = state->services[i]; if (!srv) { pservice = &state->services[i]; break; } } } else { for (i = (state->unused_service - 1); i >= 0; i--) { VCHIQ_SERVICE_T *srv = state->services[i]; if (!srv) pservice = &state->services[i]; else if ((srv->public_fourcc == params->fourcc) && ((srv->instance != instance) || (srv->base.callback != params->callback))) { /* There is another server using this ** fourcc which doesn't match. */ pservice = NULL; break; } } } if (pservice) { service->localport = (pservice - state->services); if (!handle_seq) handle_seq = VCHIQ_MAX_STATES * VCHIQ_MAX_SERVICES; service->handle = handle_seq | (state->id * VCHIQ_MAX_SERVICES) | service->localport; handle_seq += VCHIQ_MAX_STATES * VCHIQ_MAX_SERVICES; *pservice = service; if (pservice == &state->services[state->unused_service]) state->unused_service++; } lmutex_unlock(&state->mutex); if (!pservice) { _sema_destroy(&service->remove_event); _sema_destroy(&service->bulk_remove_event); lmutex_destroy(&service->bulk_mutex); kfree(service); service = NULL; } } if (service) { VCHIQ_SERVICE_QUOTA_T *service_quota = &state->service_quotas[service->localport]; service_quota->slot_quota = state->default_slot_quota; service_quota->message_quota = state->default_message_quota; if (service_quota->slot_use_count == 0) service_quota->previous_tx_index = SLOT_QUEUE_INDEX_FROM_POS(state->local_tx_pos) - 1; /* Bring this service online */ vchiq_set_service_state(service, srvstate); vchiq_log_info(vchiq_core_msg_log_level, "%s Service %c%c%c%c SrcPort:%d", (srvstate == VCHIQ_SRVSTATE_OPENING) ? "Open" : "Add", VCHIQ_FOURCC_AS_4CHARS(params->fourcc), service->localport); } /* Don't unlock the service - leave it with a ref_count of 1. */ return service; } VCHIQ_STATUS_T vchiq_open_service_internal(VCHIQ_SERVICE_T *service, int client_id) { struct vchiq_open_payload payload = { service->base.fourcc, client_id, service->version, service->version_min }; VCHIQ_ELEMENT_T body = { &payload, sizeof(payload) }; VCHIQ_STATUS_T status = VCHIQ_SUCCESS; service->client_id = client_id; vchiq_use_service_internal(service); status = queue_message(service->state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_OPEN, service->localport, 0), &body, 1, sizeof(payload), QMFLAGS_IS_BLOCKING); if (status == VCHIQ_SUCCESS) { /* Wait for the ACK/NAK */ if (down_interruptible(&service->remove_event) != 0) { status = VCHIQ_RETRY; vchiq_release_service_internal(service); } else if ((service->srvstate != VCHIQ_SRVSTATE_OPEN) && (service->srvstate != VCHIQ_SRVSTATE_OPENSYNC)) { if (service->srvstate != VCHIQ_SRVSTATE_CLOSEWAIT) vchiq_log_error(vchiq_core_log_level, "%d: osi - srvstate = %s (ref %d)", service->state->id, srvstate_names[service->srvstate], service->ref_count); status = VCHIQ_ERROR; VCHIQ_SERVICE_STATS_INC(service, error_count); vchiq_release_service_internal(service); } } return status; } static void release_service_messages(VCHIQ_SERVICE_T *service) { VCHIQ_STATE_T *state = service->state; int slot_last = state->remote->slot_last; int i; /* Release any claimed messages aimed at this service */ if (service->sync) { VCHIQ_HEADER_T *header = (VCHIQ_HEADER_T *)SLOT_DATA_FROM_INDEX(state, state->remote->slot_sync); if (VCHIQ_MSG_DSTPORT(header->msgid) == service->localport) release_message_sync(state, header); return; } for (i = state->remote->slot_first; i <= slot_last; i++) { VCHIQ_SLOT_INFO_T *slot_info = SLOT_INFO_FROM_INDEX(state, i); if (slot_info->release_count != slot_info->use_count) { char *data = (char *)SLOT_DATA_FROM_INDEX(state, i); unsigned int pos, end; end = VCHIQ_SLOT_SIZE; if (data == state->rx_data) /* This buffer is still being read from - stop ** at the current read position */ end = state->rx_pos & VCHIQ_SLOT_MASK; pos = 0; while (pos < end) { VCHIQ_HEADER_T *header = (VCHIQ_HEADER_T *)(data + pos); int msgid = header->msgid; int port = VCHIQ_MSG_DSTPORT(msgid); if ((port == service->localport) && (msgid & VCHIQ_MSGID_CLAIMED)) { vchiq_log_info(vchiq_core_log_level, " fsi - hdr %x", (unsigned int)header); release_slot(state, slot_info, header, NULL); } pos += calc_stride(header->size); if (pos > VCHIQ_SLOT_SIZE) { vchiq_log_error(vchiq_core_log_level, "fsi - pos %x: header %x, " "msgid %x, header->msgid %x, " "header->size %x", pos, (unsigned int)header, msgid, header->msgid, header->size); WARN(1, "invalid slot position\n"); } } } } } static int do_abort_bulks(VCHIQ_SERVICE_T *service) { VCHIQ_STATUS_T status; /* Abort any outstanding bulk transfers */ if (lmutex_lock_interruptible(&service->bulk_mutex) != 0) return 0; abort_outstanding_bulks(service, &service->bulk_tx); abort_outstanding_bulks(service, &service->bulk_rx); lmutex_unlock(&service->bulk_mutex); status = notify_bulks(service, &service->bulk_tx, 0/*!retry_poll*/); if (status == VCHIQ_SUCCESS) status = notify_bulks(service, &service->bulk_rx, 0/*!retry_poll*/); return (status == VCHIQ_SUCCESS); } static VCHIQ_STATUS_T close_service_complete(VCHIQ_SERVICE_T *service, int failstate) { VCHIQ_STATUS_T status; int is_server = (service->public_fourcc != VCHIQ_FOURCC_INVALID); int newstate; switch (service->srvstate) { case VCHIQ_SRVSTATE_OPEN: case VCHIQ_SRVSTATE_CLOSESENT: case VCHIQ_SRVSTATE_CLOSERECVD: if (is_server) { if (service->auto_close) { service->client_id = 0; service->remoteport = VCHIQ_PORT_FREE; newstate = VCHIQ_SRVSTATE_LISTENING; } else newstate = VCHIQ_SRVSTATE_CLOSEWAIT; } else newstate = VCHIQ_SRVSTATE_CLOSED; vchiq_set_service_state(service, newstate); break; case VCHIQ_SRVSTATE_LISTENING: break; default: vchiq_log_error(vchiq_core_log_level, "close_service_complete(%x) called in state %s", service->handle, srvstate_names[service->srvstate]); WARN(1, "close_service_complete in unexpected state\n"); return VCHIQ_ERROR; } status = make_service_callback(service, VCHIQ_SERVICE_CLOSED, NULL, NULL); if (status != VCHIQ_RETRY) { int uc = service->service_use_count; int i; /* Complete the close process */ for (i = 0; i < uc; i++) /* cater for cases where close is forced and the ** client may not close all it's handles */ vchiq_release_service_internal(service); service->client_id = 0; service->remoteport = VCHIQ_PORT_FREE; if (service->srvstate == VCHIQ_SRVSTATE_CLOSED) vchiq_free_service_internal(service); else if (service->srvstate != VCHIQ_SRVSTATE_CLOSEWAIT) { if (is_server) service->closing = 0; up(&service->remove_event); } } else vchiq_set_service_state(service, failstate); return status; } /* Called by the slot handler */ VCHIQ_STATUS_T vchiq_close_service_internal(VCHIQ_SERVICE_T *service, int close_recvd) { VCHIQ_STATE_T *state = service->state; VCHIQ_STATUS_T status = VCHIQ_SUCCESS; int is_server = (service->public_fourcc != VCHIQ_FOURCC_INVALID); vchiq_log_info(vchiq_core_log_level, "%d: csi:%d,%d (%s)", service->state->id, service->localport, close_recvd, srvstate_names[service->srvstate]); switch (service->srvstate) { case VCHIQ_SRVSTATE_CLOSED: case VCHIQ_SRVSTATE_HIDDEN: case VCHIQ_SRVSTATE_LISTENING: case VCHIQ_SRVSTATE_CLOSEWAIT: if (close_recvd) vchiq_log_error(vchiq_core_log_level, "vchiq_close_service_internal(1) called " "in state %s", srvstate_names[service->srvstate]); else if (is_server) { if (service->srvstate == VCHIQ_SRVSTATE_LISTENING) { status = VCHIQ_ERROR; } else { service->client_id = 0; service->remoteport = VCHIQ_PORT_FREE; if (service->srvstate == VCHIQ_SRVSTATE_CLOSEWAIT) vchiq_set_service_state(service, VCHIQ_SRVSTATE_LISTENING); } up(&service->remove_event); } else vchiq_free_service_internal(service); break; case VCHIQ_SRVSTATE_OPENING: if (close_recvd) { /* The open was rejected - tell the user */ vchiq_set_service_state(service, VCHIQ_SRVSTATE_CLOSEWAIT); up(&service->remove_event); } else { /* Shutdown mid-open - let the other side know */ status = queue_message(state, service, VCHIQ_MAKE_MSG (VCHIQ_MSG_CLOSE, service->localport, VCHIQ_MSG_DSTPORT(service->remoteport)), NULL, 0, 0, 0); } break; case VCHIQ_SRVSTATE_OPENSYNC: lmutex_lock(&state->sync_mutex); /* Drop through */ case VCHIQ_SRVSTATE_OPEN: if (state->is_master || close_recvd) { if (!do_abort_bulks(service)) status = VCHIQ_RETRY; } release_service_messages(service); if (status == VCHIQ_SUCCESS) status = queue_message(state, service, VCHIQ_MAKE_MSG (VCHIQ_MSG_CLOSE, service->localport, VCHIQ_MSG_DSTPORT(service->remoteport)), NULL, 0, 0, QMFLAGS_NO_MUTEX_UNLOCK); if (status == VCHIQ_SUCCESS) { if (!close_recvd) { /* Change the state while the mutex is still held */ vchiq_set_service_state(service, VCHIQ_SRVSTATE_CLOSESENT); lmutex_unlock(&state->slot_mutex); if (service->sync) lmutex_unlock(&state->sync_mutex); break; } } else if (service->srvstate == VCHIQ_SRVSTATE_OPENSYNC) { lmutex_unlock(&state->sync_mutex); break; } else break; /* Change the state while the mutex is still held */ vchiq_set_service_state(service, VCHIQ_SRVSTATE_CLOSERECVD); lmutex_unlock(&state->slot_mutex); if (service->sync) lmutex_unlock(&state->sync_mutex); status = close_service_complete(service, VCHIQ_SRVSTATE_CLOSERECVD); break; case VCHIQ_SRVSTATE_CLOSESENT: if (!close_recvd) /* This happens when a process is killed mid-close */ break; if (!state->is_master) { if (!do_abort_bulks(service)) { status = VCHIQ_RETRY; break; } } if (status == VCHIQ_SUCCESS) status = close_service_complete(service, VCHIQ_SRVSTATE_CLOSERECVD); break; case VCHIQ_SRVSTATE_CLOSERECVD: if (!close_recvd && is_server) /* Force into LISTENING mode */ vchiq_set_service_state(service, VCHIQ_SRVSTATE_LISTENING); status = close_service_complete(service, VCHIQ_SRVSTATE_CLOSERECVD); break; default: vchiq_log_error(vchiq_core_log_level, "vchiq_close_service_internal(%d) called in state %s", close_recvd, srvstate_names[service->srvstate]); break; } return status; } /* Called from the application process upon process death */ void vchiq_terminate_service_internal(VCHIQ_SERVICE_T *service) { VCHIQ_STATE_T *state = service->state; vchiq_log_info(vchiq_core_log_level, "%d: tsi - (%d<->%d)", state->id, service->localport, service->remoteport); mark_service_closing(service); /* Mark the service for removal by the slot handler */ request_poll(state, service, VCHIQ_POLL_REMOVE); } /* Called from the slot handler */ void vchiq_free_service_internal(VCHIQ_SERVICE_T *service) { VCHIQ_STATE_T *state = service->state; vchiq_log_info(vchiq_core_log_level, "%d: fsi - (%d)", state->id, service->localport); switch (service->srvstate) { case VCHIQ_SRVSTATE_OPENING: case VCHIQ_SRVSTATE_CLOSED: case VCHIQ_SRVSTATE_HIDDEN: case VCHIQ_SRVSTATE_LISTENING: case VCHIQ_SRVSTATE_CLOSEWAIT: break; default: vchiq_log_error(vchiq_core_log_level, "%d: fsi - (%d) in state %s", state->id, service->localport, srvstate_names[service->srvstate]); return; } vchiq_set_service_state(service, VCHIQ_SRVSTATE_FREE); up(&service->remove_event); /* Release the initial lock */ unlock_service(service); } VCHIQ_STATUS_T vchiq_connect_internal(VCHIQ_STATE_T *state, VCHIQ_INSTANCE_T instance) { VCHIQ_SERVICE_T *service; int i; /* Find all services registered to this client and enable them. */ i = 0; while ((service = next_service_by_instance(state, instance, &i)) != NULL) { if (service->srvstate == VCHIQ_SRVSTATE_HIDDEN) vchiq_set_service_state(service, VCHIQ_SRVSTATE_LISTENING); unlock_service(service); } if (state->conn_state == VCHIQ_CONNSTATE_DISCONNECTED) { if (queue_message(state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_CONNECT, 0, 0), NULL, 0, 0, QMFLAGS_IS_BLOCKING) == VCHIQ_RETRY) return VCHIQ_RETRY; vchiq_set_conn_state(state, VCHIQ_CONNSTATE_CONNECTING); } if (state->conn_state == VCHIQ_CONNSTATE_CONNECTING) { if (down_interruptible(&state->connect) != 0) return VCHIQ_RETRY; vchiq_set_conn_state(state, VCHIQ_CONNSTATE_CONNECTED); up(&state->connect); } return VCHIQ_SUCCESS; } VCHIQ_STATUS_T vchiq_shutdown_internal(VCHIQ_STATE_T *state, VCHIQ_INSTANCE_T instance) { VCHIQ_SERVICE_T *service; int i; /* Find all services registered to this client and enable them. */ i = 0; while ((service = next_service_by_instance(state, instance, &i)) != NULL) { (void)vchiq_remove_service(service->handle); unlock_service(service); } return VCHIQ_SUCCESS; } VCHIQ_STATUS_T vchiq_pause_internal(VCHIQ_STATE_T *state) { VCHIQ_STATUS_T status = VCHIQ_SUCCESS; switch (state->conn_state) { case VCHIQ_CONNSTATE_CONNECTED: /* Request a pause */ vchiq_set_conn_state(state, VCHIQ_CONNSTATE_PAUSING); request_poll(state, NULL, 0); break; default: vchiq_log_error(vchiq_core_log_level, "vchiq_pause_internal in state %s\n", conn_state_names[state->conn_state]); status = VCHIQ_ERROR; VCHIQ_STATS_INC(state, error_count); break; } return status; } VCHIQ_STATUS_T vchiq_resume_internal(VCHIQ_STATE_T *state) { VCHIQ_STATUS_T status = VCHIQ_SUCCESS; if (state->conn_state == VCHIQ_CONNSTATE_PAUSED) { vchiq_set_conn_state(state, VCHIQ_CONNSTATE_RESUMING); request_poll(state, NULL, 0); } else { status = VCHIQ_ERROR; VCHIQ_STATS_INC(state, error_count); } return status; } VCHIQ_STATUS_T vchiq_close_service(VCHIQ_SERVICE_HANDLE_T handle) { /* Unregister the service */ VCHIQ_SERVICE_T *service = find_service_by_handle(handle); VCHIQ_STATUS_T status = VCHIQ_SUCCESS; if (!service) return VCHIQ_ERROR; vchiq_log_info(vchiq_core_log_level, "%d: close_service:%d", service->state->id, service->localport); if ((service->srvstate == VCHIQ_SRVSTATE_FREE) || (service->srvstate == VCHIQ_SRVSTATE_LISTENING) || (service->srvstate == VCHIQ_SRVSTATE_HIDDEN)) { unlock_service(service); return VCHIQ_ERROR; } mark_service_closing(service); if (current == service->state->slot_handler_thread) { status = vchiq_close_service_internal(service, 0/*!close_recvd*/); BUG_ON(status == VCHIQ_RETRY); } else { /* Mark the service for termination by the slot handler */ request_poll(service->state, service, VCHIQ_POLL_TERMINATE); } while (1) { if (down_interruptible(&service->remove_event) != 0) { status = VCHIQ_RETRY; break; } if ((service->srvstate == VCHIQ_SRVSTATE_FREE) || (service->srvstate == VCHIQ_SRVSTATE_LISTENING) || (service->srvstate == VCHIQ_SRVSTATE_OPEN)) break; vchiq_log_warning(vchiq_core_log_level, "%d: close_service:%d - waiting in state %s", service->state->id, service->localport, srvstate_names[service->srvstate]); } if ((status == VCHIQ_SUCCESS) && (service->srvstate != VCHIQ_SRVSTATE_FREE) && (service->srvstate != VCHIQ_SRVSTATE_LISTENING)) status = VCHIQ_ERROR; unlock_service(service); return status; } VCHIQ_STATUS_T vchiq_remove_service(VCHIQ_SERVICE_HANDLE_T handle) { /* Unregister the service */ VCHIQ_SERVICE_T *service = find_service_by_handle(handle); VCHIQ_STATUS_T status = VCHIQ_SUCCESS; if (!service) return VCHIQ_ERROR; vchiq_log_info(vchiq_core_log_level, "%d: remove_service:%d", service->state->id, service->localport); if (service->srvstate == VCHIQ_SRVSTATE_FREE) { unlock_service(service); return VCHIQ_ERROR; } mark_service_closing(service); if ((service->srvstate == VCHIQ_SRVSTATE_HIDDEN) || (current == service->state->slot_handler_thread)) { /* Make it look like a client, because it must be removed and not left in the LISTENING state. */ service->public_fourcc = VCHIQ_FOURCC_INVALID; status = vchiq_close_service_internal(service, 0/*!close_recvd*/); BUG_ON(status == VCHIQ_RETRY); } else { /* Mark the service for removal by the slot handler */ request_poll(service->state, service, VCHIQ_POLL_REMOVE); } while (1) { if (down_interruptible(&service->remove_event) != 0) { status = VCHIQ_RETRY; break; } if ((service->srvstate == VCHIQ_SRVSTATE_FREE) || (service->srvstate == VCHIQ_SRVSTATE_OPEN)) break; vchiq_log_warning(vchiq_core_log_level, "%d: remove_service:%d - waiting in state %s", service->state->id, service->localport, srvstate_names[service->srvstate]); } if ((status == VCHIQ_SUCCESS) && (service->srvstate != VCHIQ_SRVSTATE_FREE)) status = VCHIQ_ERROR; unlock_service(service); return status; } /* This function may be called by kernel threads or user threads. * User threads may receive VCHIQ_RETRY to indicate that a signal has been * received and the call should be retried after being returned to user * context. * When called in blocking mode, the userdata field points to a bulk_waiter * structure. */ VCHIQ_STATUS_T vchiq_bulk_transfer(VCHIQ_SERVICE_HANDLE_T handle, VCHI_MEM_HANDLE_T memhandle, void *offset, int size, void *userdata, VCHIQ_BULK_MODE_T mode, VCHIQ_BULK_DIR_T dir) { VCHIQ_SERVICE_T *service = find_service_by_handle(handle); VCHIQ_BULK_QUEUE_T *queue; VCHIQ_BULK_T *bulk; VCHIQ_STATE_T *state; struct bulk_waiter *bulk_waiter = NULL; const char dir_char = (dir == VCHIQ_BULK_TRANSMIT) ? 't' : 'r'; const int dir_msgtype = (dir == VCHIQ_BULK_TRANSMIT) ? VCHIQ_MSG_BULK_TX : VCHIQ_MSG_BULK_RX; VCHIQ_STATUS_T status = VCHIQ_ERROR; if (!service || (service->srvstate != VCHIQ_SRVSTATE_OPEN) || ((memhandle == VCHI_MEM_HANDLE_INVALID) && (offset == NULL)) || (vchiq_check_service(service) != VCHIQ_SUCCESS)) goto error_exit; switch (mode) { case VCHIQ_BULK_MODE_NOCALLBACK: case VCHIQ_BULK_MODE_CALLBACK: break; case VCHIQ_BULK_MODE_BLOCKING: bulk_waiter = (struct bulk_waiter *)userdata; _sema_init(&bulk_waiter->event, 0); bulk_waiter->actual = 0; bulk_waiter->bulk = NULL; break; case VCHIQ_BULK_MODE_WAITING: bulk_waiter = (struct bulk_waiter *)userdata; bulk = bulk_waiter->bulk; goto waiting; default: goto error_exit; } state = service->state; queue = (dir == VCHIQ_BULK_TRANSMIT) ? &service->bulk_tx : &service->bulk_rx; if (lmutex_lock_interruptible(&service->bulk_mutex) != 0) { status = VCHIQ_RETRY; goto error_exit; } if (queue->local_insert == queue->remove + VCHIQ_NUM_SERVICE_BULKS) { VCHIQ_SERVICE_STATS_INC(service, bulk_stalls); do { lmutex_unlock(&service->bulk_mutex); if (down_interruptible(&service->bulk_remove_event) != 0) { status = VCHIQ_RETRY; goto error_exit; } if (lmutex_lock_interruptible(&service->bulk_mutex) != 0) { status = VCHIQ_RETRY; goto error_exit; } } while (queue->local_insert == queue->remove + VCHIQ_NUM_SERVICE_BULKS); } bulk = &queue->bulks[BULK_INDEX(queue->local_insert)]; bulk->mode = mode; bulk->dir = dir; bulk->userdata = userdata; bulk->size = size; bulk->actual = VCHIQ_BULK_ACTUAL_ABORTED; if (vchiq_prepare_bulk_data(bulk, memhandle, offset, size, dir) != VCHIQ_SUCCESS) goto unlock_error_exit; wmb(); vchiq_log_info(vchiq_core_log_level, "%d: bt (%d->%d) %cx %x@%x %x", state->id, service->localport, service->remoteport, dir_char, size, (unsigned int)bulk->data, (unsigned int)userdata); /* The slot mutex must be held when the service is being closed, so claim it here to ensure that isn't happening */ if (lmutex_lock_interruptible(&state->slot_mutex) != 0) { status = VCHIQ_RETRY; goto cancel_bulk_error_exit; } if (service->srvstate != VCHIQ_SRVSTATE_OPEN) goto unlock_both_error_exit; if (state->is_master) { queue->local_insert++; if (resolve_bulks(service, queue)) request_poll(state, service, (dir == VCHIQ_BULK_TRANSMIT) ? VCHIQ_POLL_TXNOTIFY : VCHIQ_POLL_RXNOTIFY); } else { int payload[2] = { (int)bulk->data, bulk->size }; VCHIQ_ELEMENT_T element = { payload, sizeof(payload) }; status = queue_message(state, NULL, VCHIQ_MAKE_MSG(dir_msgtype, service->localport, service->remoteport), &element, 1, sizeof(payload), QMFLAGS_IS_BLOCKING | QMFLAGS_NO_MUTEX_LOCK | QMFLAGS_NO_MUTEX_UNLOCK); if (status != VCHIQ_SUCCESS) { goto unlock_both_error_exit; } queue->local_insert++; } lmutex_unlock(&state->slot_mutex); lmutex_unlock(&service->bulk_mutex); vchiq_log_trace(vchiq_core_log_level, "%d: bt:%d %cx li=%x ri=%x p=%x", state->id, service->localport, dir_char, queue->local_insert, queue->remote_insert, queue->process); waiting: unlock_service(service); status = VCHIQ_SUCCESS; if (bulk_waiter) { bulk_waiter->bulk = bulk; if (down_interruptible(&bulk_waiter->event) != 0) status = VCHIQ_RETRY; else if (bulk_waiter->actual == VCHIQ_BULK_ACTUAL_ABORTED) status = VCHIQ_ERROR; } return status; unlock_both_error_exit: lmutex_unlock(&state->slot_mutex); cancel_bulk_error_exit: vchiq_complete_bulk(bulk); unlock_error_exit: lmutex_unlock(&service->bulk_mutex); error_exit: if (service) unlock_service(service); return status; } VCHIQ_STATUS_T vchiq_queue_message(VCHIQ_SERVICE_HANDLE_T handle, const VCHIQ_ELEMENT_T *elements, unsigned int count) { VCHIQ_SERVICE_T *service = find_service_by_handle(handle); VCHIQ_STATUS_T status = VCHIQ_ERROR; unsigned int size = 0; unsigned int i; if (!service || (vchiq_check_service(service) != VCHIQ_SUCCESS)) goto error_exit; for (i = 0; i < (unsigned int)count; i++) { if (elements[i].size) { if (elements[i].data == NULL) { VCHIQ_SERVICE_STATS_INC(service, error_count); goto error_exit; } size += elements[i].size; } } if (size > VCHIQ_MAX_MSG_SIZE) { VCHIQ_SERVICE_STATS_INC(service, error_count); goto error_exit; } switch (service->srvstate) { case VCHIQ_SRVSTATE_OPEN: status = queue_message(service->state, service, VCHIQ_MAKE_MSG(VCHIQ_MSG_DATA, service->localport, service->remoteport), elements, count, size, 1); break; case VCHIQ_SRVSTATE_OPENSYNC: status = queue_message_sync(service->state, service, VCHIQ_MAKE_MSG(VCHIQ_MSG_DATA, service->localport, service->remoteport), elements, count, size, 1); break; default: status = VCHIQ_ERROR; break; } error_exit: if (service) unlock_service(service); return status; } void vchiq_release_message(VCHIQ_SERVICE_HANDLE_T handle, VCHIQ_HEADER_T *header) { VCHIQ_SERVICE_T *service = find_service_by_handle(handle); VCHIQ_SHARED_STATE_T *remote; VCHIQ_STATE_T *state; int slot_index; if (!service) return; state = service->state; remote = state->remote; slot_index = SLOT_INDEX_FROM_DATA(state, (void *)header); if ((slot_index >= remote->slot_first) && (slot_index <= remote->slot_last)) { int msgid = header->msgid; if (msgid & VCHIQ_MSGID_CLAIMED) { VCHIQ_SLOT_INFO_T *slot_info = SLOT_INFO_FROM_INDEX(state, slot_index); release_slot(state, slot_info, header, service); } } else if (slot_index == remote->slot_sync) release_message_sync(state, header); unlock_service(service); } static void release_message_sync(VCHIQ_STATE_T *state, VCHIQ_HEADER_T *header) { header->msgid = VCHIQ_MSGID_PADDING; wmb(); remote_event_signal(&state->remote->sync_release); } VCHIQ_STATUS_T vchiq_get_peer_version(VCHIQ_SERVICE_HANDLE_T handle, short *peer_version) { VCHIQ_STATUS_T status = VCHIQ_ERROR; VCHIQ_SERVICE_T *service = find_service_by_handle(handle); if (!service || (vchiq_check_service(service) != VCHIQ_SUCCESS) || !peer_version) goto exit; *peer_version = service->peer_version; status = VCHIQ_SUCCESS; exit: if (service) unlock_service(service); return status; } VCHIQ_STATUS_T vchiq_get_config(VCHIQ_INSTANCE_T instance, int config_size, VCHIQ_CONFIG_T *pconfig) { VCHIQ_CONFIG_T config; (void)instance; config.max_msg_size = VCHIQ_MAX_MSG_SIZE; config.bulk_threshold = VCHIQ_MAX_MSG_SIZE; config.max_outstanding_bulks = VCHIQ_NUM_SERVICE_BULKS; config.max_services = VCHIQ_MAX_SERVICES; config.version = VCHIQ_VERSION; config.version_min = VCHIQ_VERSION_MIN; if (config_size > sizeof(VCHIQ_CONFIG_T)) return VCHIQ_ERROR; memcpy(pconfig, &config, min(config_size, (int)(sizeof(VCHIQ_CONFIG_T)))); return VCHIQ_SUCCESS; } VCHIQ_STATUS_T vchiq_set_service_option(VCHIQ_SERVICE_HANDLE_T handle, VCHIQ_SERVICE_OPTION_T option, int value) { VCHIQ_SERVICE_T *service = find_service_by_handle(handle); VCHIQ_STATUS_T status = VCHIQ_ERROR; if (service) { switch (option) { case VCHIQ_SERVICE_OPTION_AUTOCLOSE: service->auto_close = value; status = VCHIQ_SUCCESS; break; case VCHIQ_SERVICE_OPTION_SLOT_QUOTA: { VCHIQ_SERVICE_QUOTA_T *service_quota = &service->state->service_quotas[ service->localport]; if (value == 0) value = service->state->default_slot_quota; if ((value >= service_quota->slot_use_count) && (value < (unsigned short)~0)) { service_quota->slot_quota = value; if ((value >= service_quota->slot_use_count) && (service_quota->message_quota >= service_quota->message_use_count)) { /* Signal the service that it may have ** dropped below its quota */ up(&service_quota->quota_event); } status = VCHIQ_SUCCESS; } } break; case VCHIQ_SERVICE_OPTION_MESSAGE_QUOTA: { VCHIQ_SERVICE_QUOTA_T *service_quota = &service->state->service_quotas[ service->localport]; if (value == 0) value = service->state->default_message_quota; if ((value >= service_quota->message_use_count) && (value < (unsigned short)~0)) { service_quota->message_quota = value; if ((value >= service_quota->message_use_count) && (service_quota->slot_quota >= service_quota->slot_use_count)) /* Signal the service that it may have ** dropped below its quota */ up(&service_quota->quota_event); status = VCHIQ_SUCCESS; } } break; case VCHIQ_SERVICE_OPTION_SYNCHRONOUS: if ((service->srvstate == VCHIQ_SRVSTATE_HIDDEN) || (service->srvstate == VCHIQ_SRVSTATE_LISTENING)) { service->sync = value; status = VCHIQ_SUCCESS; } break; case VCHIQ_SERVICE_OPTION_TRACE: service->trace = value; status = VCHIQ_SUCCESS; break; default: break; } unlock_service(service); } return status; } static void vchiq_dump_shared_state(void *dump_context, VCHIQ_STATE_T *state, VCHIQ_SHARED_STATE_T *shared, const char *label) { static const char *const debug_names[] = { "", "SLOT_HANDLER_COUNT", "SLOT_HANDLER_LINE", "PARSE_LINE", "PARSE_HEADER", "PARSE_MSGID", "AWAIT_COMPLETION_LINE", "DEQUEUE_MESSAGE_LINE", "SERVICE_CALLBACK_LINE", "MSG_QUEUE_FULL_COUNT", "COMPLETION_QUEUE_FULL_COUNT" }; int i; char buf[80]; int len; len = snprintf(buf, sizeof(buf), " %s: slots %d-%d tx_pos=%x recycle=%x", label, shared->slot_first, shared->slot_last, shared->tx_pos, shared->slot_queue_recycle); vchiq_dump(dump_context, buf, len + 1); len = snprintf(buf, sizeof(buf), " Slots claimed:"); vchiq_dump(dump_context, buf, len + 1); for (i = shared->slot_first; i <= shared->slot_last; i++) { VCHIQ_SLOT_INFO_T slot_info = *SLOT_INFO_FROM_INDEX(state, i); if (slot_info.use_count != slot_info.release_count) { len = snprintf(buf, sizeof(buf), " %d: %d/%d", i, slot_info.use_count, slot_info.release_count); vchiq_dump(dump_context, buf, len + 1); } } for (i = 1; i < shared->debug[DEBUG_ENTRIES]; i++) { len = snprintf(buf, sizeof(buf), " DEBUG: %s = %d(%x)", debug_names[i], shared->debug[i], shared->debug[i]); vchiq_dump(dump_context, buf, len + 1); } } void vchiq_dump_state(void *dump_context, VCHIQ_STATE_T *state) { char buf[80]; int len; int i; len = snprintf(buf, sizeof(buf), "State %d: %s", state->id, conn_state_names[state->conn_state]); vchiq_dump(dump_context, buf, len + 1); len = snprintf(buf, sizeof(buf), " tx_pos=%x(@%x), rx_pos=%x(@%x)", state->local->tx_pos, (uint32_t)state->tx_data + (state->local_tx_pos & VCHIQ_SLOT_MASK), state->rx_pos, (uint32_t)state->rx_data + (state->rx_pos & VCHIQ_SLOT_MASK)); vchiq_dump(dump_context, buf, len + 1); len = snprintf(buf, sizeof(buf), " Version: %d (min %d)", VCHIQ_VERSION, VCHIQ_VERSION_MIN); vchiq_dump(dump_context, buf, len + 1); if (VCHIQ_ENABLE_STATS) { len = snprintf(buf, sizeof(buf), " Stats: ctrl_tx_count=%d, ctrl_rx_count=%d, " "error_count=%d", state->stats.ctrl_tx_count, state->stats.ctrl_rx_count, state->stats.error_count); vchiq_dump(dump_context, buf, len + 1); } len = snprintf(buf, sizeof(buf), " Slots: %d available (%d data), %d recyclable, %d stalls " "(%d data)", ((state->slot_queue_available * VCHIQ_SLOT_SIZE) - state->local_tx_pos) / VCHIQ_SLOT_SIZE, state->data_quota - state->data_use_count, state->local->slot_queue_recycle - state->slot_queue_available, state->stats.slot_stalls, state->stats.data_stalls); vchiq_dump(dump_context, buf, len + 1); vchiq_dump_platform_state(dump_context); vchiq_dump_shared_state(dump_context, state, state->local, "Local"); vchiq_dump_shared_state(dump_context, state, state->remote, "Remote"); vchiq_dump_platform_instances(dump_context); for (i = 0; i < state->unused_service; i++) { VCHIQ_SERVICE_T *service = find_service_by_port(state, i); if (service) { vchiq_dump_service_state(dump_context, service); unlock_service(service); } } } void vchiq_dump_service_state(void *dump_context, VCHIQ_SERVICE_T *service) { char buf[120]; int len; len = snprintf(buf, sizeof(buf), "Service %d: %s (ref %u)", service->localport, srvstate_names[service->srvstate], service->ref_count - 1); /*Don't include the lock just taken*/ if (service->srvstate != VCHIQ_SRVSTATE_FREE) { char remoteport[30]; VCHIQ_SERVICE_QUOTA_T *service_quota = &service->state->service_quotas[service->localport]; int fourcc = service->base.fourcc; int tx_pending, rx_pending; if (service->remoteport != VCHIQ_PORT_FREE) { int len2 = snprintf(remoteport, sizeof(remoteport), "%d", service->remoteport); if (service->public_fourcc != VCHIQ_FOURCC_INVALID) snprintf(remoteport + len2, sizeof(remoteport) - len2, " (client %8x)", service->client_id); } else strcpy(remoteport, "n/a"); len += snprintf(buf + len, sizeof(buf) - len, " '%c%c%c%c' remote %s (msg use %d/%d, slot use %d/%d)", VCHIQ_FOURCC_AS_4CHARS(fourcc), remoteport, service_quota->message_use_count, service_quota->message_quota, service_quota->slot_use_count, service_quota->slot_quota); vchiq_dump(dump_context, buf, len + 1); tx_pending = service->bulk_tx.local_insert - service->bulk_tx.remote_insert; rx_pending = service->bulk_rx.local_insert - service->bulk_rx.remote_insert; len = snprintf(buf, sizeof(buf), " Bulk: tx_pending=%d (size %d)," " rx_pending=%d (size %d)", tx_pending, tx_pending ? service->bulk_tx.bulks[ BULK_INDEX(service->bulk_tx.remove)].size : 0, rx_pending, rx_pending ? service->bulk_rx.bulks[ BULK_INDEX(service->bulk_rx.remove)].size : 0); if (VCHIQ_ENABLE_STATS) { vchiq_dump(dump_context, buf, len + 1); len = snprintf(buf, sizeof(buf), " Ctrl: tx_count=%d, tx_bytes=%llu, " "rx_count=%d, rx_bytes=%llu", service->stats.ctrl_tx_count, service->stats.ctrl_tx_bytes, service->stats.ctrl_rx_count, service->stats.ctrl_rx_bytes); vchiq_dump(dump_context, buf, len + 1); len = snprintf(buf, sizeof(buf), " Bulk: tx_count=%d, tx_bytes=%llu, " "rx_count=%d, rx_bytes=%llu", service->stats.bulk_tx_count, service->stats.bulk_tx_bytes, service->stats.bulk_rx_count, service->stats.bulk_rx_bytes); vchiq_dump(dump_context, buf, len + 1); len = snprintf(buf, sizeof(buf), " %d quota stalls, %d slot stalls, " "%d bulk stalls, %d aborted, %d errors", service->stats.quota_stalls, service->stats.slot_stalls, service->stats.bulk_stalls, service->stats.bulk_aborted_count, service->stats.error_count); } } vchiq_dump(dump_context, buf, len + 1); if (service->srvstate != VCHIQ_SRVSTATE_FREE) vchiq_dump_platform_service_state(dump_context, service); } void vchiq_loud_error_header(void) { vchiq_log_error(vchiq_core_log_level, "============================================================" "================"); vchiq_log_error(vchiq_core_log_level, "============================================================" "================"); vchiq_log_error(vchiq_core_log_level, "====="); } void vchiq_loud_error_footer(void) { vchiq_log_error(vchiq_core_log_level, "====="); vchiq_log_error(vchiq_core_log_level, "============================================================" "================"); vchiq_log_error(vchiq_core_log_level, "============================================================" "================"); } VCHIQ_STATUS_T vchiq_send_remote_use(VCHIQ_STATE_T *state) { VCHIQ_STATUS_T status = VCHIQ_RETRY; if (state->conn_state != VCHIQ_CONNSTATE_DISCONNECTED) status = queue_message(state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_REMOTE_USE, 0, 0), NULL, 0, 0, 0); return status; } VCHIQ_STATUS_T vchiq_send_remote_release(VCHIQ_STATE_T *state) { VCHIQ_STATUS_T status = VCHIQ_RETRY; if (state->conn_state != VCHIQ_CONNSTATE_DISCONNECTED) status = queue_message(state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_REMOTE_RELEASE, 0, 0), NULL, 0, 0, 0); return status; } VCHIQ_STATUS_T vchiq_send_remote_use_active(VCHIQ_STATE_T *state) { VCHIQ_STATUS_T status = VCHIQ_RETRY; if (state->conn_state != VCHIQ_CONNSTATE_DISCONNECTED) status = queue_message(state, NULL, VCHIQ_MAKE_MSG(VCHIQ_MSG_REMOTE_USE_ACTIVE, 0, 0), NULL, 0, 0, 0); return status; } void vchiq_log_dump_mem(const char *label, uint32_t addr, const void *voidMem, size_t numBytes) { const uint8_t *mem = (const uint8_t *)voidMem; size_t offset; char lineBuf[100]; char *s; while (numBytes > 0) { s = lineBuf; for (offset = 0; offset < 16; offset++) { if (offset < numBytes) s += snprintf(s, 4, "%02x ", mem[offset]); else s += snprintf(s, 4, " "); } for (offset = 0; offset < 16; offset++) { if (offset < numBytes) { uint8_t ch = mem[offset]; if ((ch < ' ') || (ch > '~')) ch = '.'; *s++ = (char)ch; } } *s++ = '\0'; if ((label != NULL) && (*label != '\0')) vchiq_log_trace(VCHIQ_LOG_TRACE, "%s: %08x: %s", label, addr, lineBuf); else vchiq_log_trace(VCHIQ_LOG_TRACE, "%08x: %s", addr, lineBuf); addr += 16; mem += 16; if (numBytes > 16) numBytes -= 16; else numBytes = 0; } } Index: projects/netbsd-tests-update-12/sys/dev/cxgbe/cxgbei/cxgbei.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/cxgbe/cxgbei/cxgbei.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/cxgbe/cxgbei/cxgbei.c (revision 305172) @@ -1,1151 +1,1150 @@ /*- * Copyright (c) 2012 Chelsio Communications, Inc. * All rights reserved. * * Chelsio T5xx iSCSI driver * * Written by: Sreenivasa Honnur * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include "opt_inet6.h" #include #include #include #include #include #ifdef TCP_OFFLOAD #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "common/common.h" #include "common/t4_msg.h" #include "common/t4_regs.h" /* for PCIE_MEM_ACCESS */ #include "tom/t4_tom.h" #include "cxgbei.h" #include "cxgbei_ulp2_ddp.h" static int worker_thread_count; static struct cxgbei_worker_thread_softc *cwt_softc; static struct proc *cxgbei_proc; /* XXXNP some header instead. */ struct icl_pdu *icl_cxgbei_new_pdu(int); void icl_cxgbei_new_pdu_set_conn(struct icl_pdu *, struct icl_conn *); void icl_cxgbei_conn_pdu_free(struct icl_conn *, struct icl_pdu *); /* * Direct Data Placement - * Directly place the iSCSI Data-In or Data-Out PDU's payload into pre-posted * final destination host-memory buffers based on the Initiator Task Tag (ITT) * in Data-In or Target Task Tag (TTT) in Data-Out PDUs. * The host memory address is programmed into h/w in the format of pagepod * entries. * The location of the pagepod entry is encoded into ddp tag which is used as * the base for ITT/TTT. */ /* * functions to program the pagepod in h/w */ static void inline ppod_set(struct pagepod *ppod, struct cxgbei_ulp2_pagepod_hdr *hdr, struct cxgbei_ulp2_gather_list *gl, unsigned int pidx) { int i; memcpy(ppod, hdr, sizeof(*hdr)); for (i = 0; i < (PPOD_PAGES + 1); i++, pidx++) { ppod->addr[i] = pidx < gl->nelem ? cpu_to_be64(gl->dma_sg[pidx].phys_addr) : 0ULL; } } static void inline ppod_clear(struct pagepod *ppod) { memset(ppod, 0, sizeof(*ppod)); } static inline void ulp_mem_io_set_hdr(struct adapter *sc, int tid, struct ulp_mem_io *req, unsigned int wr_len, unsigned int dlen, unsigned int pm_addr) { struct ulptx_idata *idata = (struct ulptx_idata *)(req + 1); INIT_ULPTX_WR(req, wr_len, 0, 0); req->cmd = cpu_to_be32(V_ULPTX_CMD(ULP_TX_MEM_WRITE) | V_ULP_MEMIO_ORDER(is_t4(sc)) | V_T5_ULP_MEMIO_IMM(is_t5(sc))); req->dlen = htonl(V_ULP_MEMIO_DATA_LEN(dlen >> 5)); req->len16 = htonl(DIV_ROUND_UP(wr_len - sizeof(req->wr), 16) | V_FW_WR_FLOWID(tid)); req->lock_addr = htonl(V_ULP_MEMIO_ADDR(pm_addr >> 5)); idata->cmd_more = htonl(V_ULPTX_CMD(ULP_TX_SC_IMM)); idata->len = htonl(dlen); } -#define PPOD_SIZE sizeof(struct pagepod) #define ULPMEM_IDATA_MAX_NPPODS 1 /* 256/PPOD_SIZE */ #define PCIE_MEMWIN_MAX_NPPODS 16 /* 1024/PPOD_SIZE */ static int ppod_write_idata(struct cxgbei_data *ci, struct cxgbei_ulp2_pagepod_hdr *hdr, unsigned int idx, unsigned int npods, struct cxgbei_ulp2_gather_list *gl, unsigned int gl_pidx, struct toepcb *toep) { u_int dlen = PPOD_SIZE * npods; u_int pm_addr = idx * PPOD_SIZE + ci->llimit; u_int wr_len = roundup(sizeof(struct ulp_mem_io) + sizeof(struct ulptx_idata) + dlen, 16); struct ulp_mem_io *req; struct ulptx_idata *idata; struct pagepod *ppod; u_int i; struct wrqe *wr; struct adapter *sc = toep->vi->pi->adapter; wr = alloc_wrqe(wr_len, toep->ctrlq); if (wr == NULL) { CXGBE_UNIMPLEMENTED("ppod_write_idata: alloc_wrqe failure"); return (ENOMEM); } req = wrtod(wr); memset(req, 0, wr_len); ulp_mem_io_set_hdr(sc, toep->tid, req, wr_len, dlen, pm_addr); idata = (struct ulptx_idata *)(req + 1); ppod = (struct pagepod *)(idata + 1); for (i = 0; i < npods; i++, ppod++, gl_pidx += PPOD_PAGES) { if (!hdr) /* clear the pagepod */ ppod_clear(ppod); else /* set the pagepod */ ppod_set(ppod, hdr, gl, gl_pidx); } t4_wrq_tx(sc, wr); return 0; } int t4_ddp_set_map(struct cxgbei_data *ci, void *iccp, struct cxgbei_ulp2_pagepod_hdr *hdr, u_int idx, u_int npods, struct cxgbei_ulp2_gather_list *gl, int reply) { struct icl_cxgbei_conn *icc = (struct icl_cxgbei_conn *)iccp; struct toepcb *toep = icc->toep; int err; unsigned int pidx = 0, w_npods = 0, cnt; /* * on T4, if we use a mix of IMMD and DSGL with ULP_MEM_WRITE, * the order would not be guaranteed, so we will stick with IMMD */ gl->tid = toep->tid; gl->port_id = toep->vi->pi->port_id; gl->egress_dev = (void *)toep->vi->ifp; /* send via immediate data */ for (; w_npods < npods; idx += cnt, w_npods += cnt, pidx += PPOD_PAGES) { cnt = npods - w_npods; if (cnt > ULPMEM_IDATA_MAX_NPPODS) cnt = ULPMEM_IDATA_MAX_NPPODS; err = ppod_write_idata(ci, hdr, idx, cnt, gl, pidx, toep); if (err) { printf("%s: ppod_write_idata failed\n", __func__); break; } } return err; } void t4_ddp_clear_map(struct cxgbei_data *ci, struct cxgbei_ulp2_gather_list *gl, u_int tag, u_int idx, u_int npods, struct icl_cxgbei_conn *icc) { struct toepcb *toep = icc->toep; int err = -1; u_int pidx = 0; u_int w_npods = 0; u_int cnt; for (; w_npods < npods; idx += cnt, w_npods += cnt, pidx += PPOD_PAGES) { cnt = npods - w_npods; if (cnt > ULPMEM_IDATA_MAX_NPPODS) cnt = ULPMEM_IDATA_MAX_NPPODS; err = ppod_write_idata(ci, NULL, idx, cnt, gl, 0, toep); if (err) break; } } static int cxgbei_map_sg(struct cxgbei_sgl *sgl, struct ccb_scsiio *csio) { unsigned int data_len = csio->dxfer_len; unsigned int sgoffset = (uint64_t)csio->data_ptr & PAGE_MASK; unsigned int nsge; unsigned char *sgaddr = csio->data_ptr; unsigned int len = 0; nsge = (csio->dxfer_len + sgoffset + PAGE_SIZE - 1) >> PAGE_SHIFT; sgl->sg_addr = sgaddr; sgl->sg_offset = sgoffset; if (data_len < (PAGE_SIZE - sgoffset)) len = data_len; else len = PAGE_SIZE - sgoffset; sgl->sg_length = len; data_len -= len; sgaddr += len; sgl = sgl+1; while (data_len > 0) { sgl->sg_addr = sgaddr; len = (data_len < PAGE_SIZE)? data_len: PAGE_SIZE; sgl->sg_length = len; sgaddr += len; data_len -= len; sgl = sgl + 1; } return nsge; } static int cxgbei_map_sg_tgt(struct cxgbei_sgl *sgl, union ctl_io *io) { unsigned int data_len, sgoffset, nsge; unsigned char *sgaddr; unsigned int len = 0, index = 0, ctl_sg_count, i; struct ctl_sg_entry ctl_sg_entry, *ctl_sglist; if (io->scsiio.kern_sg_entries > 0) { ctl_sglist = (struct ctl_sg_entry *)io->scsiio.kern_data_ptr; ctl_sg_count = io->scsiio.kern_sg_entries; } else { ctl_sglist = &ctl_sg_entry; ctl_sglist->addr = io->scsiio.kern_data_ptr; ctl_sglist->len = io->scsiio.kern_data_len; ctl_sg_count = 1; } sgaddr = sgl->sg_addr = ctl_sglist[index].addr; sgoffset = sgl->sg_offset = (uint64_t)sgl->sg_addr & PAGE_MASK; data_len = ctl_sglist[index].len; if (data_len < (PAGE_SIZE - sgoffset)) len = data_len; else len = PAGE_SIZE - sgoffset; sgl->sg_length = len; data_len -= len; sgaddr += len; sgl = sgl+1; len = 0; for (i = 0; i< ctl_sg_count; i++) len += ctl_sglist[i].len; nsge = (len + sgoffset + PAGE_SIZE -1) >> PAGE_SHIFT; while (data_len > 0) { sgl->sg_addr = sgaddr; len = (data_len < PAGE_SIZE)? data_len: PAGE_SIZE; sgl->sg_length = len; sgaddr += len; data_len -= len; sgl = sgl + 1; if (data_len == 0) { if (index == ctl_sg_count - 1) break; index++; sgaddr = ctl_sglist[index].addr; data_len = ctl_sglist[index].len; } } return nsge; } static int t4_sk_ddp_tag_reserve(struct cxgbei_data *ci, struct icl_cxgbei_conn *icc, u_int xferlen, struct cxgbei_sgl *sgl, u_int sgcnt, u_int *ddp_tag) { struct cxgbei_ulp2_gather_list *gl; int err = -EINVAL; struct toepcb *toep = icc->toep; gl = cxgbei_ulp2_ddp_make_gl_from_iscsi_sgvec(xferlen, sgl, sgcnt, ci, 0); if (gl) { err = cxgbei_ulp2_ddp_tag_reserve(ci, icc, toep->tid, &ci->tag_format, ddp_tag, gl, 0, 0); if (err) { cxgbei_ulp2_ddp_release_gl(ci, gl); } } return err; } static unsigned int cxgbei_task_reserve_itt(struct icl_conn *ic, void **prv, struct ccb_scsiio *scmd, unsigned int *itt) { struct icl_cxgbei_conn *icc = ic_to_icc(ic); int xferlen = scmd->dxfer_len; struct cxgbei_task_data *tdata = NULL; struct cxgbei_sgl *sge = NULL; struct toepcb *toep = icc->toep; struct adapter *sc = td_adapter(toep->td); struct cxgbei_data *ci = sc->iscsi_ulp_softc; int err = -1; MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); tdata = (struct cxgbei_task_data *)*prv; if (xferlen == 0 || tdata == NULL) goto out; if (xferlen < DDP_THRESHOLD) goto out; if ((scmd->ccb_h.flags & CAM_DIR_MASK) == CAM_DIR_IN) { tdata->nsge = cxgbei_map_sg(tdata->sgl, scmd); if (tdata->nsge == 0) { CTR1(KTR_CXGBE, "%s: map_sg failed", __func__); return 0; } sge = tdata->sgl; tdata->sc_ddp_tag = *itt; CTR3(KTR_CXGBE, "%s: *itt:0x%x sc_ddp_tag:0x%x", __func__, *itt, tdata->sc_ddp_tag); if (cxgbei_ulp2_sw_tag_usable(&ci->tag_format, tdata->sc_ddp_tag)) { err = t4_sk_ddp_tag_reserve(ci, icc, scmd->dxfer_len, sge, tdata->nsge, &tdata->sc_ddp_tag); } else { CTR3(KTR_CXGBE, "%s: itt:0x%x sc_ddp_tag:0x%x not usable", __func__, *itt, tdata->sc_ddp_tag); } } out: if (err < 0) tdata->sc_ddp_tag = cxgbei_ulp2_set_non_ddp_tag(&ci->tag_format, *itt); return tdata->sc_ddp_tag; } static unsigned int cxgbei_task_reserve_ttt(struct icl_conn *ic, void **prv, union ctl_io *io, unsigned int *ttt) { struct icl_cxgbei_conn *icc = ic_to_icc(ic); struct toepcb *toep = icc->toep; struct adapter *sc = td_adapter(toep->td); struct cxgbei_data *ci = sc->iscsi_ulp_softc; struct cxgbei_task_data *tdata = NULL; int xferlen, err = -1; struct cxgbei_sgl *sge = NULL; MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); xferlen = (io->scsiio.kern_data_len - io->scsiio.ext_data_filled); tdata = (struct cxgbei_task_data *)*prv; if ((xferlen == 0) || (tdata == NULL)) goto out; if (xferlen < DDP_THRESHOLD) goto out; tdata->nsge = cxgbei_map_sg_tgt(tdata->sgl, io); if (tdata->nsge == 0) { CTR1(KTR_CXGBE, "%s: map_sg failed", __func__); return 0; } sge = tdata->sgl; tdata->sc_ddp_tag = *ttt; if (cxgbei_ulp2_sw_tag_usable(&ci->tag_format, tdata->sc_ddp_tag)) { err = t4_sk_ddp_tag_reserve(ci, icc, xferlen, sge, tdata->nsge, &tdata->sc_ddp_tag); } else { CTR2(KTR_CXGBE, "%s: sc_ddp_tag:0x%x not usable", __func__, tdata->sc_ddp_tag); } out: if (err < 0) tdata->sc_ddp_tag = cxgbei_ulp2_set_non_ddp_tag(&ci->tag_format, *ttt); return tdata->sc_ddp_tag; } static int t4_sk_ddp_tag_release(struct icl_cxgbei_conn *icc, unsigned int ddp_tag) { struct toepcb *toep = icc->toep; struct adapter *sc = td_adapter(toep->td); struct cxgbei_data *ci = sc->iscsi_ulp_softc; cxgbei_ulp2_ddp_tag_release(ci, ddp_tag, icc); return (0); } static void read_pdu_limits(struct adapter *sc, uint32_t *max_tx_pdu_len, uint32_t *max_rx_pdu_len) { uint32_t tx_len, rx_len, r, v; rx_len = t4_read_reg(sc, A_TP_PMM_RX_PAGE_SIZE); tx_len = t4_read_reg(sc, A_TP_PMM_TX_PAGE_SIZE); r = t4_read_reg(sc, A_TP_PARA_REG2); rx_len = min(rx_len, G_MAXRXDATA(r)); tx_len = min(tx_len, G_MAXRXDATA(r)); r = t4_read_reg(sc, A_TP_PARA_REG7); v = min(G_PMMAXXFERLEN0(r), G_PMMAXXFERLEN1(r)); rx_len = min(rx_len, v); tx_len = min(tx_len, v); /* Remove after FW_FLOWC_MNEM_TXDATAPLEN_MAX fix in firmware. */ tx_len = min(tx_len, 3 * 4096); *max_tx_pdu_len = rounddown2(tx_len, 512); *max_rx_pdu_len = rounddown2(rx_len, 512); } /* * Initialize the software state of the iSCSI ULP driver. * * ENXIO means firmware didn't set up something that it was supposed to. */ static int cxgbei_init(struct adapter *sc, struct cxgbei_data *ci) { int nppods, bits, rc; static const u_int pgsz_order[] = {0, 1, 2, 3}; MPASS(sc->vres.iscsi.size > 0); ci->llimit = sc->vres.iscsi.start; ci->ulimit = sc->vres.iscsi.start + sc->vres.iscsi.size - 1; read_pdu_limits(sc, &ci->max_tx_pdu_len, &ci->max_rx_pdu_len); nppods = sc->vres.iscsi.size >> IPPOD_SIZE_SHIFT; if (nppods <= 1024) return (ENXIO); bits = fls(nppods); if (bits > IPPOD_IDX_MAX_SIZE) bits = IPPOD_IDX_MAX_SIZE; nppods = (1 << (bits - 1)) - 1; rc = bus_dma_tag_create(NULL, 1, 0, BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, UINT32_MAX , 8, BUS_SPACE_MAXSIZE, BUS_DMA_ALLOCNOW, NULL, NULL, &ci->ulp_ddp_tag); if (rc != 0) { device_printf(sc->dev, "%s: failed to create DMA tag: %u.\n", __func__, rc); return (rc); } ci->colors = malloc(nppods * sizeof(char), M_CXGBE, M_NOWAIT | M_ZERO); ci->gl_map = malloc(nppods * sizeof(struct cxgbei_ulp2_gather_list *), M_CXGBE, M_NOWAIT | M_ZERO); if (ci->colors == NULL || ci->gl_map == NULL) { bus_dma_tag_destroy(ci->ulp_ddp_tag); free(ci->colors, M_CXGBE); free(ci->gl_map, M_CXGBE); return (ENOMEM); } mtx_init(&ci->map_lock, "ddp lock", NULL, MTX_DEF | MTX_DUPOK); ci->nppods = nppods; ci->idx_last = nppods; ci->idx_bits = bits; ci->idx_mask = (1 << bits) - 1; ci->rsvd_tag_mask = (1 << (bits + IPPOD_IDX_SHIFT)) - 1; ci->tag_format.sw_bits = bits; ci->tag_format.rsvd_bits = bits; ci->tag_format.rsvd_shift = IPPOD_IDX_SHIFT; ci->tag_format.rsvd_mask = ci->idx_mask; t4_iscsi_init(sc, ci->idx_mask << IPPOD_IDX_SHIFT, pgsz_order); return (rc); } static int do_rx_iscsi_hdr(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) { struct adapter *sc = iq->adapter; struct cpl_iscsi_hdr *cpl = mtod(m, struct cpl_iscsi_hdr *); u_int tid = GET_TID(cpl); struct toepcb *toep = lookup_tid(sc, tid); struct icl_pdu *ip; struct icl_cxgbei_pdu *icp; M_ASSERTPKTHDR(m); ip = icl_cxgbei_new_pdu(M_NOWAIT); if (ip == NULL) CXGBE_UNIMPLEMENTED("PDU allocation failure"); icp = ip_to_icp(ip); bcopy(mtod(m, caddr_t) + sizeof(*cpl), icp->ip.ip_bhs, sizeof(struct iscsi_bhs)); icp->icp_seq = ntohl(cpl->seq); icp->icp_flags = ICPF_RX_HDR; /* This is the start of a new PDU. There should be no old state. */ MPASS(toep->ulpcb2 == NULL); toep->ulpcb2 = icp; #if 0 CTR4(KTR_CXGBE, "%s: tid %u, cpl->len hlen %u, m->m_len hlen %u", __func__, tid, ntohs(cpl->len), m->m_len); #endif m_freem(m); return (0); } static int do_rx_iscsi_data(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) { struct adapter *sc = iq->adapter; struct cpl_iscsi_data *cpl = mtod(m, struct cpl_iscsi_data *); u_int tid = GET_TID(cpl); struct toepcb *toep = lookup_tid(sc, tid); struct icl_cxgbei_pdu *icp = toep->ulpcb2; M_ASSERTPKTHDR(m); /* Must already have received the header (but not the data). */ MPASS(icp != NULL); MPASS(icp->icp_flags == ICPF_RX_HDR); MPASS(icp->ip.ip_data_mbuf == NULL); MPASS(icp->ip.ip_data_len == 0); m_adj(m, sizeof(*cpl)); icp->icp_flags |= ICPF_RX_FLBUF; icp->ip.ip_data_mbuf = m; icp->ip.ip_data_len = m->m_pkthdr.len; #if 0 CTR4(KTR_CXGBE, "%s: tid %u, cpl->len dlen %u, m->m_len dlen %u", __func__, tid, ntohs(cpl->len), m->m_len); #endif return (0); } static int do_rx_iscsi_ddp(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) { struct adapter *sc = iq->adapter; const struct cpl_rx_data_ddp *cpl = (const void *)(rss + 1); u_int tid = GET_TID(cpl); struct toepcb *toep = lookup_tid(sc, tid); struct inpcb *inp = toep->inp; struct socket *so; struct sockbuf *sb; struct tcpcb *tp; struct icl_cxgbei_conn *icc; struct icl_conn *ic; struct icl_cxgbei_pdu *icp = toep->ulpcb2; struct icl_pdu *ip; u_int pdu_len, val; MPASS(m == NULL); /* Must already be assembling a PDU. */ MPASS(icp != NULL); MPASS(icp->icp_flags & ICPF_RX_HDR); /* Data is optional. */ ip = &icp->ip; icp->icp_flags |= ICPF_RX_STATUS; val = ntohl(cpl->ddpvld); if (val & F_DDP_PADDING_ERR) icp->icp_flags |= ICPF_PAD_ERR; if (val & F_DDP_HDRCRC_ERR) icp->icp_flags |= ICPF_HCRC_ERR; if (val & F_DDP_DATACRC_ERR) icp->icp_flags |= ICPF_DCRC_ERR; if (ip->ip_data_mbuf == NULL) { /* XXXNP: what should ip->ip_data_len be, and why? */ icp->icp_flags |= ICPF_RX_DDP; } pdu_len = ntohs(cpl->len); /* includes everything. */ INP_WLOCK(inp); if (__predict_false(inp->inp_flags & (INP_DROPPED | INP_TIMEWAIT))) { CTR4(KTR_CXGBE, "%s: tid %u, rx (%d bytes), inp_flags 0x%x", __func__, tid, pdu_len, inp->inp_flags); INP_WUNLOCK(inp); icl_cxgbei_conn_pdu_free(NULL, ip); #ifdef INVARIANTS toep->ulpcb2 = NULL; #endif return (0); } tp = intotcpcb(inp); MPASS(icp->icp_seq == tp->rcv_nxt); MPASS(tp->rcv_wnd >= pdu_len); tp->rcv_nxt += pdu_len; tp->rcv_wnd -= pdu_len; tp->t_rcvtime = ticks; /* update rx credits */ toep->rx_credits += pdu_len; t4_rcvd(&toep->td->tod, tp); /* XXX: sc->tom_softc.tod */ so = inp->inp_socket; sb = &so->so_rcv; SOCKBUF_LOCK(sb); icc = toep->ulpcb; if (__predict_false(icc == NULL || sb->sb_state & SBS_CANTRCVMORE)) { CTR5(KTR_CXGBE, "%s: tid %u, excess rx (%d bytes), icc %p, sb_state 0x%x", __func__, tid, pdu_len, icc, sb->sb_state); SOCKBUF_UNLOCK(sb); INP_WUNLOCK(inp); INP_INFO_RLOCK(&V_tcbinfo); INP_WLOCK(inp); tp = tcp_drop(tp, ECONNRESET); if (tp) INP_WUNLOCK(inp); INP_INFO_RUNLOCK(&V_tcbinfo); icl_cxgbei_conn_pdu_free(NULL, ip); #ifdef INVARIANTS toep->ulpcb2 = NULL; #endif return (0); } MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); ic = &icc->ic; icl_cxgbei_new_pdu_set_conn(ip, ic); MPASS(m == NULL); /* was unused, we'll use it now. */ m = sbcut_locked(sb, sbused(sb)); /* XXXNP: toep->sb_cc accounting? */ if (__predict_false(m != NULL)) { int len = m_length(m, NULL); /* * PDUs were received before the tid transitioned to ULP mode. * Convert them to icl_cxgbei_pdus and send them to ICL before * the PDU in icp/ip. */ CTR3(KTR_CXGBE, "%s: tid %u, %u bytes in so_rcv", __func__, tid, len); /* XXXNP: needs to be rewritten. */ if (len == sizeof(struct iscsi_bhs) || len == 4 + sizeof(struct iscsi_bhs)) { struct icl_cxgbei_pdu *icp0; struct icl_pdu *ip0; ip0 = icl_cxgbei_new_pdu(M_NOWAIT); icl_cxgbei_new_pdu_set_conn(ip0, ic); if (ip0 == NULL) CXGBE_UNIMPLEMENTED("PDU allocation failure"); icp0 = ip_to_icp(ip0); icp0->icp_seq = 0; /* XXX */ icp0->icp_flags = ICPF_RX_HDR | ICPF_RX_STATUS; m_copydata(m, 0, sizeof(struct iscsi_bhs), (void *)ip0->ip_bhs); STAILQ_INSERT_TAIL(&icc->rcvd_pdus, ip0, ip_next); } m_freem(m); } #if 0 CTR4(KTR_CXGBE, "%s: tid %u, pdu_len %u, pdu_flags 0x%x", __func__, tid, pdu_len, icp->icp_flags); #endif STAILQ_INSERT_TAIL(&icc->rcvd_pdus, ip, ip_next); if ((icc->rx_flags & RXF_ACTIVE) == 0) { struct cxgbei_worker_thread_softc *cwt = &cwt_softc[icc->cwt]; mtx_lock(&cwt->cwt_lock); icc->rx_flags |= RXF_ACTIVE; TAILQ_INSERT_TAIL(&cwt->rx_head, icc, rx_link); if (cwt->cwt_state == CWT_SLEEPING) { cwt->cwt_state = CWT_RUNNING; cv_signal(&cwt->cwt_cv); } mtx_unlock(&cwt->cwt_lock); } SOCKBUF_UNLOCK(sb); INP_WUNLOCK(inp); #ifdef INVARIANTS toep->ulpcb2 = NULL; #endif return (0); } /* initiator */ void cxgbei_conn_task_reserve_itt(void *conn, void **prv, void *scmd, unsigned int *itt) { unsigned int tag; tag = cxgbei_task_reserve_itt(conn, prv, scmd, itt); if (tag) *itt = htonl(tag); return; } /* target */ void cxgbei_conn_transfer_reserve_ttt(void *conn, void **prv, void *scmd, unsigned int *ttt) { unsigned int tag; tag = cxgbei_task_reserve_ttt(conn, prv, scmd, ttt); if (tag) *ttt = htonl(tag); return; } void cxgbei_cleanup_task(void *conn, void *ofld_priv) { struct icl_conn *ic = (struct icl_conn *)conn; struct icl_cxgbei_conn *icc = ic_to_icc(ic); struct cxgbei_task_data *tdata = ofld_priv; struct adapter *sc = icc->sc; struct cxgbei_data *ci = sc->iscsi_ulp_softc; MPASS(icc->icc_signature == CXGBEI_CONN_SIGNATURE); MPASS(tdata != NULL); if (cxgbei_ulp2_is_ddp_tag(&ci->tag_format, tdata->sc_ddp_tag)) t4_sk_ddp_tag_release(icc, tdata->sc_ddp_tag); memset(tdata, 0, sizeof(*tdata)); } static int cxgbei_activate(struct adapter *sc) { struct cxgbei_data *ci; int rc; ASSERT_SYNCHRONIZED_OP(sc); if (uld_active(sc, ULD_ISCSI)) { KASSERT(0, ("%s: iSCSI offload already enabled on adapter %p", __func__, sc)); return (0); } if (sc->iscsicaps == 0 || sc->vres.iscsi.size == 0) { device_printf(sc->dev, "not iSCSI offload capable, or capability disabled.\n"); return (ENOSYS); } /* per-adapter softc for iSCSI */ ci = malloc(sizeof(*ci), M_CXGBE, M_ZERO | M_NOWAIT); if (ci == NULL) return (ENOMEM); rc = cxgbei_init(sc, ci); if (rc != 0) { free(ci, M_CXGBE); return (rc); } sc->iscsi_ulp_softc = ci; return (0); } static int cxgbei_deactivate(struct adapter *sc) { ASSERT_SYNCHRONIZED_OP(sc); if (sc->iscsi_ulp_softc != NULL) { cxgbei_ddp_cleanup(sc->iscsi_ulp_softc); free(sc->iscsi_ulp_softc, M_CXGBE); sc->iscsi_ulp_softc = NULL; } return (0); } static void cxgbei_activate_all(struct adapter *sc, void *arg __unused) { if (begin_synchronized_op(sc, NULL, SLEEP_OK | INTR_OK, "t4isact") != 0) return; /* Activate iSCSI if any port on this adapter has IFCAP_TOE enabled. */ if (sc->offload_map && !uld_active(sc, ULD_ISCSI)) (void) t4_activate_uld(sc, ULD_ISCSI); end_synchronized_op(sc, 0); } static void cxgbei_deactivate_all(struct adapter *sc, void *arg __unused) { if (begin_synchronized_op(sc, NULL, SLEEP_OK | INTR_OK, "t4isdea") != 0) return; if (uld_active(sc, ULD_ISCSI)) (void) t4_deactivate_uld(sc, ULD_ISCSI); end_synchronized_op(sc, 0); } static struct uld_info cxgbei_uld_info = { .uld_id = ULD_ISCSI, .activate = cxgbei_activate, .deactivate = cxgbei_deactivate, }; static void cwt_main(void *arg) { struct cxgbei_worker_thread_softc *cwt = arg; struct icl_cxgbei_conn *icc = NULL; struct icl_conn *ic; struct icl_pdu *ip; struct sockbuf *sb; STAILQ_HEAD(, icl_pdu) rx_pdus = STAILQ_HEAD_INITIALIZER(rx_pdus); MPASS(cwt != NULL); mtx_lock(&cwt->cwt_lock); MPASS(cwt->cwt_state == 0); cwt->cwt_state = CWT_RUNNING; cv_signal(&cwt->cwt_cv); while (__predict_true(cwt->cwt_state != CWT_STOP)) { cwt->cwt_state = CWT_RUNNING; while ((icc = TAILQ_FIRST(&cwt->rx_head)) != NULL) { TAILQ_REMOVE(&cwt->rx_head, icc, rx_link); mtx_unlock(&cwt->cwt_lock); ic = &icc->ic; sb = &ic->ic_socket->so_rcv; SOCKBUF_LOCK(sb); MPASS(icc->rx_flags & RXF_ACTIVE); if (__predict_true(!(sb->sb_state & SBS_CANTRCVMORE))) { MPASS(STAILQ_EMPTY(&rx_pdus)); STAILQ_SWAP(&icc->rcvd_pdus, &rx_pdus, icl_pdu); SOCKBUF_UNLOCK(sb); /* Hand over PDUs to ICL. */ while ((ip = STAILQ_FIRST(&rx_pdus)) != NULL) { STAILQ_REMOVE_HEAD(&rx_pdus, ip_next); ic->ic_receive(ip); } SOCKBUF_LOCK(sb); MPASS(STAILQ_EMPTY(&rx_pdus)); } MPASS(icc->rx_flags & RXF_ACTIVE); if (STAILQ_EMPTY(&icc->rcvd_pdus) || __predict_false(sb->sb_state & SBS_CANTRCVMORE)) { icc->rx_flags &= ~RXF_ACTIVE; } else { /* * More PDUs were received while we were busy * handing over the previous batch to ICL. * Re-add this connection to the end of the * queue. */ mtx_lock(&cwt->cwt_lock); TAILQ_INSERT_TAIL(&cwt->rx_head, icc, rx_link); mtx_unlock(&cwt->cwt_lock); } SOCKBUF_UNLOCK(sb); mtx_lock(&cwt->cwt_lock); } /* Inner loop doesn't check for CWT_STOP, do that first. */ if (__predict_false(cwt->cwt_state == CWT_STOP)) break; cwt->cwt_state = CWT_SLEEPING; cv_wait(&cwt->cwt_cv, &cwt->cwt_lock); } MPASS(TAILQ_FIRST(&cwt->rx_head) == NULL); mtx_assert(&cwt->cwt_lock, MA_OWNED); cwt->cwt_state = CWT_STOPPED; cv_signal(&cwt->cwt_cv); mtx_unlock(&cwt->cwt_lock); kthread_exit(); } static int start_worker_threads(void) { int i, rc; struct cxgbei_worker_thread_softc *cwt; worker_thread_count = min(mp_ncpus, 32); cwt_softc = malloc(worker_thread_count * sizeof(*cwt), M_CXGBE, M_WAITOK | M_ZERO); MPASS(cxgbei_proc == NULL); for (i = 0, cwt = &cwt_softc[0]; i < worker_thread_count; i++, cwt++) { mtx_init(&cwt->cwt_lock, "cwt lock", NULL, MTX_DEF); cv_init(&cwt->cwt_cv, "cwt cv"); TAILQ_INIT(&cwt->rx_head); rc = kproc_kthread_add(cwt_main, cwt, &cxgbei_proc, NULL, 0, 0, "cxgbei", "%d", i); if (rc != 0) { printf("cxgbei: failed to start thread #%d/%d (%d)\n", i + 1, worker_thread_count, rc); mtx_destroy(&cwt->cwt_lock); cv_destroy(&cwt->cwt_cv); bzero(&cwt, sizeof(*cwt)); if (i == 0) { free(cwt_softc, M_CXGBE); worker_thread_count = 0; return (rc); } /* Not fatal, carry on with fewer threads. */ worker_thread_count = i; rc = 0; break; } /* Wait for thread to start before moving on to the next one. */ mtx_lock(&cwt->cwt_lock); while (cwt->cwt_state == 0) cv_wait(&cwt->cwt_cv, &cwt->cwt_lock); mtx_unlock(&cwt->cwt_lock); } MPASS(cwt_softc != NULL); MPASS(worker_thread_count > 0); return (0); } static void stop_worker_threads(void) { int i; struct cxgbei_worker_thread_softc *cwt = &cwt_softc[0]; MPASS(worker_thread_count >= 0); for (i = 0, cwt = &cwt_softc[0]; i < worker_thread_count; i++, cwt++) { mtx_lock(&cwt->cwt_lock); MPASS(cwt->cwt_state == CWT_RUNNING || cwt->cwt_state == CWT_SLEEPING); cwt->cwt_state = CWT_STOP; cv_signal(&cwt->cwt_cv); do { cv_wait(&cwt->cwt_cv, &cwt->cwt_lock); } while (cwt->cwt_state != CWT_STOPPED); mtx_unlock(&cwt->cwt_lock); } free(cwt_softc, M_CXGBE); } /* Select a worker thread for a connection. */ u_int cxgbei_select_worker_thread(struct icl_cxgbei_conn *icc) { struct adapter *sc = icc->sc; struct toepcb *toep = icc->toep; u_int i, n; n = worker_thread_count / sc->sge.nofldrxq; if (n > 0) i = toep->vi->pi->port_id * n + arc4random() % n; else i = arc4random() % worker_thread_count; CTR3(KTR_CXGBE, "%s: tid %u, cwt %u", __func__, toep->tid, i); return (i); } static int cxgbei_mod_load(void) { int rc; t4_register_cpl_handler(CPL_ISCSI_HDR, do_rx_iscsi_hdr); t4_register_cpl_handler(CPL_ISCSI_DATA, do_rx_iscsi_data); t4_register_cpl_handler(CPL_RX_ISCSI_DDP, do_rx_iscsi_ddp); rc = start_worker_threads(); if (rc != 0) return (rc); rc = t4_register_uld(&cxgbei_uld_info); if (rc != 0) { stop_worker_threads(); return (rc); } t4_iterate(cxgbei_activate_all, NULL); return (rc); } static int cxgbei_mod_unload(void) { t4_iterate(cxgbei_deactivate_all, NULL); if (t4_unregister_uld(&cxgbei_uld_info) == EBUSY) return (EBUSY); stop_worker_threads(); t4_register_cpl_handler(CPL_ISCSI_HDR, NULL); t4_register_cpl_handler(CPL_ISCSI_DATA, NULL); t4_register_cpl_handler(CPL_RX_ISCSI_DDP, NULL); return (0); } #endif static int cxgbei_modevent(module_t mod, int cmd, void *arg) { int rc = 0; #ifdef TCP_OFFLOAD switch (cmd) { case MOD_LOAD: rc = cxgbei_mod_load(); if (rc == 0) rc = icl_cxgbei_mod_load(); break; case MOD_UNLOAD: rc = icl_cxgbei_mod_unload(); if (rc == 0) rc = cxgbei_mod_unload(); break; default: rc = EINVAL; } #else printf("cxgbei: compiled without TCP_OFFLOAD support.\n"); rc = EOPNOTSUPP; #endif return (rc); } static moduledata_t cxgbei_mod = { "cxgbei", cxgbei_modevent, NULL, }; MODULE_VERSION(cxgbei, 1); DECLARE_MODULE(cxgbei, cxgbei_mod, SI_SUB_EXEC, SI_ORDER_ANY); MODULE_DEPEND(cxgbei, t4_tom, 1, 1, 1); MODULE_DEPEND(cxgbei, cxgbe, 1, 1, 1); MODULE_DEPEND(cxgbei, icl, 1, 1, 1); Index: projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_ddp.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_ddp.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_ddp.c (revision 305172) @@ -1,1794 +1,1965 @@ /*- * Copyright (c) 2012 Chelsio Communications, Inc. * All rights reserved. * Written by: Navdeep Parhar * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define TCPSTATES #include #include #include #include #include #include #include #include #include #ifdef TCP_OFFLOAD #include "common/common.h" #include "common/t4_msg.h" #include "common/t4_regs.h" #include "common/t4_tcb.h" #include "tom/t4_tom.h" VNET_DECLARE(int, tcp_do_autorcvbuf); #define V_tcp_do_autorcvbuf VNET(tcp_do_autorcvbuf) VNET_DECLARE(int, tcp_autorcvbuf_inc); #define V_tcp_autorcvbuf_inc VNET(tcp_autorcvbuf_inc) VNET_DECLARE(int, tcp_autorcvbuf_max); #define V_tcp_autorcvbuf_max VNET(tcp_autorcvbuf_max) /* * Use the 'backend3' field in AIO jobs to store the amount of data * received by the AIO job so far. */ #define aio_received backend3 static void aio_ddp_requeue_task(void *context, int pending); static void ddp_complete_all(struct toepcb *toep, int error); static void t4_aio_cancel_active(struct kaiocb *job); static void t4_aio_cancel_queued(struct kaiocb *job); -#define PPOD_SZ(n) ((n) * sizeof(struct pagepod)) -#define PPOD_SIZE (PPOD_SZ(1)) - static TAILQ_HEAD(, pageset) ddp_orphan_pagesets; static struct mtx ddp_orphan_pagesets_lock; static struct task ddp_orphan_task; #define MAX_DDP_BUFFER_SIZE (M_TCB_RX_DDP_BUF0_LEN) -static int -alloc_ppods(struct tom_data *td, int n, u_int *ppod_addr) -{ - vmem_addr_t v; - int rc; - MPASS(n > 0); - - rc = vmem_alloc(td->ppod_arena, PPOD_SZ(n), M_NOWAIT | M_FIRSTFIT, &v); - *ppod_addr = (u_int)v; - - return (rc); -} - -static void -free_ppods(struct tom_data *td, u_int ppod_addr, int n) -{ - - MPASS(n > 0); - - vmem_free(td->ppod_arena, (vmem_addr_t)ppod_addr, PPOD_SZ(n)); -} - -static inline int -pages_to_nppods(int npages, int ddp_pgsz) -{ - int nsegs = npages * PAGE_SIZE / ddp_pgsz; - - return (howmany(nsegs, PPOD_PAGES)); -} - /* * A page set holds information about a buffer used for DDP. The page * set holds resources such as the VM pages backing the buffer (either * held or wired) and the page pods associated with the buffer. * Recently used page sets are cached to allow for efficient reuse of * buffers (avoiding the need to re-fault in pages, hold them, etc.). * Note that cached page sets keep the backing pages wired. The * number of wired pages is capped by only allowing for two wired * pagesets per connection. This is not a perfect cap, but is a * trade-off for performance. * * If an application ping-pongs two buffers for a connection via * aio_read(2) then those buffers should remain wired and expensive VM * fault lookups should be avoided after each buffer has been used * once. If an application uses more than two buffers then this will * fall back to doing expensive VM fault lookups for each operation. */ static void free_pageset(struct tom_data *td, struct pageset *ps) { vm_page_t p; int i; - if (ps->nppods > 0) - free_ppods(td, ps->ppod_addr, ps->nppods); + if (ps->prsv.prsv_nppods > 0) + t4_free_page_pods(&ps->prsv); if (ps->flags & PS_WIRED) { for (i = 0; i < ps->npages; i++) { p = ps->pages[i]; vm_page_lock(p); vm_page_unwire(p, PQ_INACTIVE); vm_page_unlock(p); } } else vm_page_unhold_pages(ps->pages, ps->npages); mtx_lock(&ddp_orphan_pagesets_lock); TAILQ_INSERT_TAIL(&ddp_orphan_pagesets, ps, link); taskqueue_enqueue(taskqueue_thread, &ddp_orphan_task); mtx_unlock(&ddp_orphan_pagesets_lock); } static void ddp_free_orphan_pagesets(void *context, int pending) { struct pageset *ps; mtx_lock(&ddp_orphan_pagesets_lock); while (!TAILQ_EMPTY(&ddp_orphan_pagesets)) { ps = TAILQ_FIRST(&ddp_orphan_pagesets); TAILQ_REMOVE(&ddp_orphan_pagesets, ps, link); mtx_unlock(&ddp_orphan_pagesets_lock); if (ps->vm) vmspace_free(ps->vm); free(ps, M_CXGBE); mtx_lock(&ddp_orphan_pagesets_lock); } mtx_unlock(&ddp_orphan_pagesets_lock); } static void recycle_pageset(struct toepcb *toep, struct pageset *ps) { DDP_ASSERT_LOCKED(toep); if (!(toep->ddp_flags & DDP_DEAD) && ps->flags & PS_WIRED) { KASSERT(toep->ddp_cached_count + toep->ddp_active_count < nitems(toep->db), ("too many wired pagesets")); TAILQ_INSERT_HEAD(&toep->ddp_cached_pagesets, ps, link); toep->ddp_cached_count++; } else free_pageset(toep->td, ps); } static void ddp_complete_one(struct kaiocb *job, int error) { long copied; /* * If this job had copied data out of the socket buffer before * it was cancelled, report it as a short read rather than an * error. */ copied = job->aio_received; if (copied != 0 || error == 0) aio_complete(job, copied, 0); else aio_complete(job, -1, error); } static void free_ddp_buffer(struct tom_data *td, struct ddp_buffer *db) { if (db->job) { /* * XXX: If we are un-offloading the socket then we * should requeue these on the socket somehow. If we * got a FIN from the remote end, then this completes * any remaining requests with an EOF read. */ if (!aio_clear_cancel_function(db->job)) ddp_complete_one(db->job, 0); } if (db->ps) free_pageset(td, db->ps); } void ddp_init_toep(struct toepcb *toep) { TAILQ_INIT(&toep->ddp_aiojobq); TASK_INIT(&toep->ddp_requeue_task, 0, aio_ddp_requeue_task, toep); toep->ddp_active_id = -1; mtx_init(&toep->ddp_lock, "t4 ddp", NULL, MTX_DEF); } void ddp_uninit_toep(struct toepcb *toep) { mtx_destroy(&toep->ddp_lock); } void release_ddp_resources(struct toepcb *toep) { struct pageset *ps; int i; DDP_LOCK(toep); toep->flags |= DDP_DEAD; for (i = 0; i < nitems(toep->db); i++) { free_ddp_buffer(toep->td, &toep->db[i]); } while ((ps = TAILQ_FIRST(&toep->ddp_cached_pagesets)) != NULL) { TAILQ_REMOVE(&toep->ddp_cached_pagesets, ps, link); free_pageset(toep->td, ps); } ddp_complete_all(toep, 0); DDP_UNLOCK(toep); } #ifdef INVARIANTS void ddp_assert_empty(struct toepcb *toep) { int i; MPASS(!(toep->ddp_flags & DDP_TASK_ACTIVE)); for (i = 0; i < nitems(toep->db); i++) { MPASS(toep->db[i].job == NULL); MPASS(toep->db[i].ps == NULL); } MPASS(TAILQ_EMPTY(&toep->ddp_cached_pagesets)); MPASS(TAILQ_EMPTY(&toep->ddp_aiojobq)); } #endif static void complete_ddp_buffer(struct toepcb *toep, struct ddp_buffer *db, unsigned int db_idx) { unsigned int db_flag; toep->ddp_active_count--; if (toep->ddp_active_id == db_idx) { if (toep->ddp_active_count == 0) { KASSERT(toep->db[db_idx ^ 1].job == NULL, ("%s: active_count mismatch", __func__)); toep->ddp_active_id = -1; } else toep->ddp_active_id ^= 1; #ifdef VERBOSE_TRACES CTR2(KTR_CXGBE, "%s: ddp_active_id = %d", __func__, toep->ddp_active_id); #endif } else { KASSERT(toep->ddp_active_count != 0 && toep->ddp_active_id != -1, ("%s: active count mismatch", __func__)); } db->cancel_pending = 0; db->job = NULL; recycle_pageset(toep, db->ps); db->ps = NULL; db_flag = db_idx == 1 ? DDP_BUF1_ACTIVE : DDP_BUF0_ACTIVE; KASSERT(toep->ddp_flags & db_flag, ("%s: DDP buffer not active. toep %p, ddp_flags 0x%x", __func__, toep, toep->ddp_flags)); toep->ddp_flags &= ~db_flag; } /* XXX: handle_ddp_data code duplication */ void insert_ddp_data(struct toepcb *toep, uint32_t n) { struct inpcb *inp = toep->inp; struct tcpcb *tp = intotcpcb(inp); struct ddp_buffer *db; struct kaiocb *job; size_t placed; long copied; unsigned int db_flag, db_idx; INP_WLOCK_ASSERT(inp); DDP_ASSERT_LOCKED(toep); tp->rcv_nxt += n; #ifndef USE_DDP_RX_FLOW_CONTROL KASSERT(tp->rcv_wnd >= n, ("%s: negative window size", __func__)); tp->rcv_wnd -= n; #endif #ifndef USE_DDP_RX_FLOW_CONTROL toep->rx_credits += n; #endif CTR2(KTR_CXGBE, "%s: placed %u bytes before falling out of DDP", __func__, n); while (toep->ddp_active_count > 0) { MPASS(toep->ddp_active_id != -1); db_idx = toep->ddp_active_id; db_flag = db_idx == 1 ? DDP_BUF1_ACTIVE : DDP_BUF0_ACTIVE; MPASS((toep->ddp_flags & db_flag) != 0); db = &toep->db[db_idx]; job = db->job; copied = job->aio_received; placed = n; if (placed > job->uaiocb.aio_nbytes - copied) placed = job->uaiocb.aio_nbytes - copied; if (placed > 0) job->msgrcv = 1; if (!aio_clear_cancel_function(job)) { /* * Update the copied length for when * t4_aio_cancel_active() completes this * request. */ job->aio_received += placed; } else if (copied + placed != 0) { CTR4(KTR_CXGBE, "%s: completing %p (copied %ld, placed %lu)", __func__, job, copied, placed); /* XXX: This always completes if there is some data. */ aio_complete(job, copied + placed, 0); } else if (aio_set_cancel_function(job, t4_aio_cancel_queued)) { TAILQ_INSERT_HEAD(&toep->ddp_aiojobq, job, list); toep->ddp_waiting_count++; } else aio_cancel(job); n -= placed; complete_ddp_buffer(toep, db, db_idx); } MPASS(n == 0); } /* SET_TCB_FIELD sent as a ULP command looks like this */ #define LEN__SET_TCB_FIELD_ULP (sizeof(struct ulp_txpkt) + \ sizeof(struct ulptx_idata) + sizeof(struct cpl_set_tcb_field_core)) /* RX_DATA_ACK sent as a ULP command looks like this */ #define LEN__RX_DATA_ACK_ULP (sizeof(struct ulp_txpkt) + \ sizeof(struct ulptx_idata) + sizeof(struct cpl_rx_data_ack_core)) static inline void * mk_set_tcb_field_ulp(struct ulp_txpkt *ulpmc, struct toepcb *toep, uint64_t word, uint64_t mask, uint64_t val) { struct ulptx_idata *ulpsc; struct cpl_set_tcb_field_core *req; ulpmc->cmd_dest = htonl(V_ULPTX_CMD(ULP_TX_PKT) | V_ULP_TXPKT_DEST(0)); ulpmc->len = htobe32(howmany(LEN__SET_TCB_FIELD_ULP, 16)); ulpsc = (struct ulptx_idata *)(ulpmc + 1); ulpsc->cmd_more = htobe32(V_ULPTX_CMD(ULP_TX_SC_IMM)); ulpsc->len = htobe32(sizeof(*req)); req = (struct cpl_set_tcb_field_core *)(ulpsc + 1); OPCODE_TID(req) = htobe32(MK_OPCODE_TID(CPL_SET_TCB_FIELD, toep->tid)); req->reply_ctrl = htobe16(V_NO_REPLY(1) | V_QUEUENO(toep->ofld_rxq->iq.abs_id)); req->word_cookie = htobe16(V_WORD(word) | V_COOKIE(0)); req->mask = htobe64(mask); req->val = htobe64(val); ulpsc = (struct ulptx_idata *)(req + 1); if (LEN__SET_TCB_FIELD_ULP % 16) { ulpsc->cmd_more = htobe32(V_ULPTX_CMD(ULP_TX_SC_NOOP)); ulpsc->len = htobe32(0); return (ulpsc + 1); } return (ulpsc); } static inline void * mk_rx_data_ack_ulp(struct ulp_txpkt *ulpmc, struct toepcb *toep) { struct ulptx_idata *ulpsc; struct cpl_rx_data_ack_core *req; ulpmc->cmd_dest = htonl(V_ULPTX_CMD(ULP_TX_PKT) | V_ULP_TXPKT_DEST(0)); ulpmc->len = htobe32(howmany(LEN__RX_DATA_ACK_ULP, 16)); ulpsc = (struct ulptx_idata *)(ulpmc + 1); ulpsc->cmd_more = htobe32(V_ULPTX_CMD(ULP_TX_SC_IMM)); ulpsc->len = htobe32(sizeof(*req)); req = (struct cpl_rx_data_ack_core *)(ulpsc + 1); OPCODE_TID(req) = htobe32(MK_OPCODE_TID(CPL_RX_DATA_ACK, toep->tid)); req->credit_dack = htobe32(F_RX_MODULATE_RX); ulpsc = (struct ulptx_idata *)(req + 1); if (LEN__RX_DATA_ACK_ULP % 16) { ulpsc->cmd_more = htobe32(V_ULPTX_CMD(ULP_TX_SC_NOOP)); ulpsc->len = htobe32(0); return (ulpsc + 1); } return (ulpsc); } static struct wrqe * mk_update_tcb_for_ddp(struct adapter *sc, struct toepcb *toep, int db_idx, struct pageset *ps, int offset, uint64_t ddp_flags, uint64_t ddp_flags_mask) { struct wrqe *wr; struct work_request_hdr *wrh; struct ulp_txpkt *ulpmc; int len; KASSERT(db_idx == 0 || db_idx == 1, ("%s: bad DDP buffer index %d", __func__, db_idx)); /* * We'll send a compound work request that has 3 SET_TCB_FIELDs and an * RX_DATA_ACK (with RX_MODULATE to speed up delivery). * * The work request header is 16B and always ends at a 16B boundary. * The ULPTX master commands that follow must all end at 16B boundaries * too so we round up the size to 16. */ len = sizeof(*wrh) + 3 * roundup2(LEN__SET_TCB_FIELD_ULP, 16) + roundup2(LEN__RX_DATA_ACK_ULP, 16); wr = alloc_wrqe(len, toep->ctrlq); if (wr == NULL) return (NULL); wrh = wrtod(wr); INIT_ULPTX_WRH(wrh, len, 1, 0); /* atomic */ ulpmc = (struct ulp_txpkt *)(wrh + 1); /* Write the buffer's tag */ ulpmc = mk_set_tcb_field_ulp(ulpmc, toep, W_TCB_RX_DDP_BUF0_TAG + db_idx, V_TCB_RX_DDP_BUF0_TAG(M_TCB_RX_DDP_BUF0_TAG), - V_TCB_RX_DDP_BUF0_TAG(ps->tag)); + V_TCB_RX_DDP_BUF0_TAG(ps->prsv.prsv_tag)); /* Update the current offset in the DDP buffer and its total length */ if (db_idx == 0) ulpmc = mk_set_tcb_field_ulp(ulpmc, toep, W_TCB_RX_DDP_BUF0_OFFSET, V_TCB_RX_DDP_BUF0_OFFSET(M_TCB_RX_DDP_BUF0_OFFSET) | V_TCB_RX_DDP_BUF0_LEN(M_TCB_RX_DDP_BUF0_LEN), V_TCB_RX_DDP_BUF0_OFFSET(offset) | V_TCB_RX_DDP_BUF0_LEN(ps->len)); else ulpmc = mk_set_tcb_field_ulp(ulpmc, toep, W_TCB_RX_DDP_BUF1_OFFSET, V_TCB_RX_DDP_BUF1_OFFSET(M_TCB_RX_DDP_BUF1_OFFSET) | V_TCB_RX_DDP_BUF1_LEN((u64)M_TCB_RX_DDP_BUF1_LEN << 32), V_TCB_RX_DDP_BUF1_OFFSET(offset) | V_TCB_RX_DDP_BUF1_LEN((u64)ps->len << 32)); /* Update DDP flags */ ulpmc = mk_set_tcb_field_ulp(ulpmc, toep, W_TCB_RX_DDP_FLAGS, ddp_flags_mask, ddp_flags); /* Gratuitous RX_DATA_ACK with RX_MODULATE set to speed up delivery. */ ulpmc = mk_rx_data_ack_ulp(ulpmc, toep); return (wr); } static int handle_ddp_data(struct toepcb *toep, __be32 ddp_report, __be32 rcv_nxt, int len) { uint32_t report = be32toh(ddp_report); unsigned int db_idx; struct inpcb *inp = toep->inp; struct ddp_buffer *db; struct tcpcb *tp; struct socket *so; struct sockbuf *sb; struct kaiocb *job; long copied; db_idx = report & F_DDP_BUF_IDX ? 1 : 0; if (__predict_false(!(report & F_DDP_INV))) CXGBE_UNIMPLEMENTED("DDP buffer still valid"); INP_WLOCK(inp); so = inp_inpcbtosocket(inp); sb = &so->so_rcv; DDP_LOCK(toep); KASSERT(toep->ddp_active_id == db_idx, ("completed DDP buffer (%d) != active_id (%d) for tid %d", db_idx, toep->ddp_active_id, toep->tid)); db = &toep->db[db_idx]; job = db->job; if (__predict_false(inp->inp_flags & (INP_DROPPED | INP_TIMEWAIT))) { /* * This can happen due to an administrative tcpdrop(8). * Just fail the request with ECONNRESET. */ CTR5(KTR_CXGBE, "%s: tid %u, seq 0x%x, len %d, inp_flags 0x%x", __func__, toep->tid, be32toh(rcv_nxt), len, inp->inp_flags); if (aio_clear_cancel_function(job)) ddp_complete_one(job, ECONNRESET); goto completed; } tp = intotcpcb(inp); /* * For RX_DDP_COMPLETE, len will be zero and rcv_nxt is the * sequence number of the next byte to receive. The length of * the data received for this message must be computed by * comparing the new and old values of rcv_nxt. * * For RX_DATA_DDP, len might be non-zero, but it is only the * length of the most recent DMA. It does not include the * total length of the data received since the previous update * for this DDP buffer. rcv_nxt is the sequence number of the * first received byte from the most recent DMA. */ len += be32toh(rcv_nxt) - tp->rcv_nxt; tp->rcv_nxt += len; tp->t_rcvtime = ticks; #ifndef USE_DDP_RX_FLOW_CONTROL KASSERT(tp->rcv_wnd >= len, ("%s: negative window size", __func__)); tp->rcv_wnd -= len; #endif #ifdef VERBOSE_TRACES CTR4(KTR_CXGBE, "%s: DDP[%d] placed %d bytes (%#x)", __func__, db_idx, len, report); #endif /* receive buffer autosize */ CURVNET_SET(so->so_vnet); SOCKBUF_LOCK(sb); if (sb->sb_flags & SB_AUTOSIZE && V_tcp_do_autorcvbuf && sb->sb_hiwat < V_tcp_autorcvbuf_max && len > (sbspace(sb) / 8 * 7)) { unsigned int hiwat = sb->sb_hiwat; unsigned int newsize = min(hiwat + V_tcp_autorcvbuf_inc, V_tcp_autorcvbuf_max); if (!sbreserve_locked(sb, newsize, so, NULL)) sb->sb_flags &= ~SB_AUTOSIZE; else toep->rx_credits += newsize - hiwat; } SOCKBUF_UNLOCK(sb); CURVNET_RESTORE(); #ifndef USE_DDP_RX_FLOW_CONTROL toep->rx_credits += len; #endif job->msgrcv = 1; if (db->cancel_pending) { /* * Update the job's length but defer completion to the * TCB_RPL callback. */ job->aio_received += len; goto out; } else if (!aio_clear_cancel_function(job)) { /* * Update the copied length for when * t4_aio_cancel_active() completes this request. */ job->aio_received += len; } else { copied = job->aio_received; #ifdef VERBOSE_TRACES CTR4(KTR_CXGBE, "%s: completing %p (copied %ld, placed %d)", __func__, job, copied, len); #endif aio_complete(job, copied + len, 0); t4_rcvd(&toep->td->tod, tp); } completed: complete_ddp_buffer(toep, db, db_idx); if (toep->ddp_waiting_count > 0) ddp_queue_toep(toep); out: DDP_UNLOCK(toep); INP_WUNLOCK(inp); return (0); } void handle_ddp_indicate(struct toepcb *toep) { DDP_ASSERT_LOCKED(toep); MPASS(toep->ddp_active_count == 0); MPASS((toep->ddp_flags & (DDP_BUF0_ACTIVE | DDP_BUF1_ACTIVE)) == 0); if (toep->ddp_waiting_count == 0) { /* * The pending requests that triggered the request for an * an indicate were cancelled. Those cancels should have * already disabled DDP. Just ignore this as the data is * going into the socket buffer anyway. */ return; } CTR3(KTR_CXGBE, "%s: tid %d indicated (%d waiting)", __func__, toep->tid, toep->ddp_waiting_count); ddp_queue_toep(toep); } enum { DDP_BUF0_INVALIDATED = 0x2, DDP_BUF1_INVALIDATED }; void handle_ddp_tcb_rpl(struct toepcb *toep, const struct cpl_set_tcb_rpl *cpl) { unsigned int db_idx; struct inpcb *inp = toep->inp; struct ddp_buffer *db; struct kaiocb *job; long copied; if (cpl->status != CPL_ERR_NONE) panic("XXX: tcp_rpl failed: %d", cpl->status); switch (cpl->cookie) { case V_WORD(W_TCB_RX_DDP_FLAGS) | V_COOKIE(DDP_BUF0_INVALIDATED): case V_WORD(W_TCB_RX_DDP_FLAGS) | V_COOKIE(DDP_BUF1_INVALIDATED): /* * XXX: This duplicates a lot of code with handle_ddp_data(). */ db_idx = G_COOKIE(cpl->cookie) - DDP_BUF0_INVALIDATED; INP_WLOCK(inp); DDP_LOCK(toep); db = &toep->db[db_idx]; /* * handle_ddp_data() should leave the job around until * this callback runs once a cancel is pending. */ MPASS(db != NULL); MPASS(db->job != NULL); MPASS(db->cancel_pending); /* * XXX: It's not clear what happens if there is data * placed when the buffer is invalidated. I suspect we * need to read the TCB to see how much data was placed. * * For now this just pretends like nothing was placed. * * XXX: Note that if we did check the PCB we would need to * also take care of updating the tp, etc. */ job = db->job; copied = job->aio_received; if (copied == 0) { CTR2(KTR_CXGBE, "%s: cancelling %p", __func__, job); aio_cancel(job); } else { CTR3(KTR_CXGBE, "%s: completing %p (copied %ld)", __func__, job, copied); aio_complete(job, copied, 0); t4_rcvd(&toep->td->tod, intotcpcb(inp)); } complete_ddp_buffer(toep, db, db_idx); if (toep->ddp_waiting_count > 0) ddp_queue_toep(toep); DDP_UNLOCK(toep); INP_WUNLOCK(inp); break; default: panic("XXX: unknown tcb_rpl offset %#x, cookie %#x", G_WORD(cpl->cookie), G_COOKIE(cpl->cookie)); } } void handle_ddp_close(struct toepcb *toep, struct tcpcb *tp, __be32 rcv_nxt) { struct ddp_buffer *db; struct kaiocb *job; long copied; unsigned int db_flag, db_idx; int len, placed; INP_WLOCK_ASSERT(toep->inp); DDP_ASSERT_LOCKED(toep); len = be32toh(rcv_nxt) - tp->rcv_nxt; tp->rcv_nxt += len; #ifndef USE_DDP_RX_FLOW_CONTROL toep->rx_credits += len; #endif while (toep->ddp_active_count > 0) { MPASS(toep->ddp_active_id != -1); db_idx = toep->ddp_active_id; db_flag = db_idx == 1 ? DDP_BUF1_ACTIVE : DDP_BUF0_ACTIVE; MPASS((toep->ddp_flags & db_flag) != 0); db = &toep->db[db_idx]; job = db->job; copied = job->aio_received; placed = len; if (placed > job->uaiocb.aio_nbytes - copied) placed = job->uaiocb.aio_nbytes - copied; if (placed > 0) job->msgrcv = 1; if (!aio_clear_cancel_function(job)) { /* * Update the copied length for when * t4_aio_cancel_active() completes this * request. */ job->aio_received += placed; } else { CTR4(KTR_CXGBE, "%s: tid %d completed buf %d len %d", __func__, toep->tid, db_idx, placed); aio_complete(job, copied + placed, 0); } len -= placed; complete_ddp_buffer(toep, db, db_idx); } MPASS(len == 0); ddp_complete_all(toep, 0); } #define DDP_ERR (F_DDP_PPOD_MISMATCH | F_DDP_LLIMIT_ERR | F_DDP_ULIMIT_ERR |\ F_DDP_PPOD_PARITY_ERR | F_DDP_PADDING_ERR | F_DDP_OFFSET_ERR |\ F_DDP_INVALID_TAG | F_DDP_COLOR_ERR | F_DDP_TID_MISMATCH |\ F_DDP_INVALID_PPOD | F_DDP_HDRCRC_ERR | F_DDP_DATACRC_ERR) extern cpl_handler_t t4_cpl_handler[]; static int do_rx_data_ddp(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) { struct adapter *sc = iq->adapter; const struct cpl_rx_data_ddp *cpl = (const void *)(rss + 1); unsigned int tid = GET_TID(cpl); uint32_t vld; struct toepcb *toep = lookup_tid(sc, tid); KASSERT(m == NULL, ("%s: wasn't expecting payload", __func__)); KASSERT(toep->tid == tid, ("%s: toep tid/atid mismatch", __func__)); KASSERT(!(toep->flags & TPF_SYNQE), ("%s: toep %p claims to be a synq entry", __func__, toep)); vld = be32toh(cpl->ddpvld); if (__predict_false(vld & DDP_ERR)) { panic("%s: DDP error 0x%x (tid %d, toep %p)", __func__, vld, tid, toep); } if (toep->ulp_mode == ULP_MODE_ISCSI) { t4_cpl_handler[CPL_RX_ISCSI_DDP](iq, rss, m); return (0); } handle_ddp_data(toep, cpl->u.ddp_report, cpl->seq, be16toh(cpl->len)); return (0); } static int do_rx_ddp_complete(struct sge_iq *iq, const struct rss_header *rss, struct mbuf *m) { struct adapter *sc = iq->adapter; const struct cpl_rx_ddp_complete *cpl = (const void *)(rss + 1); unsigned int tid = GET_TID(cpl); struct toepcb *toep = lookup_tid(sc, tid); KASSERT(m == NULL, ("%s: wasn't expecting payload", __func__)); KASSERT(toep->tid == tid, ("%s: toep tid/atid mismatch", __func__)); KASSERT(!(toep->flags & TPF_SYNQE), ("%s: toep %p claims to be a synq entry", __func__, toep)); handle_ddp_data(toep, cpl->ddp_report, cpl->rcv_nxt, 0); return (0); } static void enable_ddp(struct adapter *sc, struct toepcb *toep) { KASSERT((toep->ddp_flags & (DDP_ON | DDP_OK | DDP_SC_REQ)) == DDP_OK, ("%s: toep %p has bad ddp_flags 0x%x", __func__, toep, toep->ddp_flags)); CTR3(KTR_CXGBE, "%s: tid %u (time %u)", __func__, toep->tid, time_uptime); DDP_ASSERT_LOCKED(toep); toep->ddp_flags |= DDP_SC_REQ; t4_set_tcb_field(sc, toep->ctrlq, toep->tid, W_TCB_RX_DDP_FLAGS, V_TF_DDP_OFF(1) | V_TF_DDP_INDICATE_OUT(1) | V_TF_DDP_BUF0_INDICATE(1) | V_TF_DDP_BUF1_INDICATE(1) | V_TF_DDP_BUF0_VALID(1) | V_TF_DDP_BUF1_VALID(1), V_TF_DDP_BUF0_INDICATE(1) | V_TF_DDP_BUF1_INDICATE(1), 0, 0, toep->ofld_rxq->iq.abs_id); t4_set_tcb_field(sc, toep->ctrlq, toep->tid, W_TCB_T_FLAGS, V_TF_RCV_COALESCE_ENABLE(1), 0, 0, 0, toep->ofld_rxq->iq.abs_id); } static int calculate_hcf(int n1, int n2) { int a, b, t; if (n1 <= n2) { a = n1; b = n2; } else { a = n2; b = n1; } while (a != 0) { t = a; a = b % a; b = t; } return (b); } +static inline int +pages_to_nppods(int npages, int ddp_page_shift) +{ + + MPASS(ddp_page_shift >= PAGE_SHIFT); + + return (howmany(npages >> (ddp_page_shift - PAGE_SHIFT), PPOD_PAGES)); +} + static int -alloc_page_pods(struct tom_data *td, struct pageset *ps) +alloc_page_pods(struct ppod_region *pr, u_int nppods, u_int pgsz_idx, + struct ppod_reservation *prsv) { - int i, hcf, seglen, idx, ppod, nppods; - u_int ppod_addr; + vmem_addr_t addr; /* relative to start of region */ - KASSERT(ps->nppods == 0, ("%s: page pods already allocated", __func__)); + if (vmem_alloc(pr->pr_arena, PPOD_SZ(nppods), M_NOWAIT | M_FIRSTFIT, + &addr) != 0) + return (ENOMEM); + CTR5(KTR_CXGBE, "%-17s arena %p, addr 0x%08x, nppods %d, pgsz %d", + __func__, pr->pr_arena, (uint32_t)addr & pr->pr_tag_mask, + nppods, 1 << pr->pr_page_shift[pgsz_idx]); + /* + * The hardware tagmask includes an extra invalid bit but the arena was + * seeded with valid values only. An allocation out of this arena will + * fit inside the tagmask but won't have the invalid bit set. + */ + MPASS((addr & pr->pr_tag_mask) == addr); + MPASS((addr & pr->pr_invalid_bit) == 0); + + prsv->prsv_pr = pr; + prsv->prsv_tag = V_PPOD_PGSZ(pgsz_idx) | addr; + prsv->prsv_nppods = nppods; + + return (0); +} + +int +t4_alloc_page_pods_for_ps(struct ppod_region *pr, struct pageset *ps) +{ + int i, hcf, seglen, idx, nppods; + struct ppod_reservation *prsv = &ps->prsv; + + KASSERT(prsv->prsv_nppods == 0, + ("%s: page pods already allocated", __func__)); + + /* * The DDP page size is unrelated to the VM page size. We combine * contiguous physical pages into larger segments to get the best DDP * page size possible. This is the largest of the four sizes in * A_ULP_RX_TDDP_PSZ that evenly divides the HCF of the segment sizes in * the page list. */ hcf = 0; for (i = 0; i < ps->npages; i++) { seglen = PAGE_SIZE; while (i < ps->npages - 1 && ps->pages[i]->phys_addr + PAGE_SIZE == ps->pages[i + 1]->phys_addr) { seglen += PAGE_SIZE; i++; } hcf = calculate_hcf(hcf, seglen); - if (hcf < td->ddp_pgsz[1]) { + if (hcf < (1 << pr->pr_page_shift[1])) { idx = 0; goto have_pgsz; /* give up, short circuit */ } } - if (hcf % td->ddp_pgsz[0] != 0) { - /* hmmm. This could only happen when PAGE_SIZE < 4K */ - KASSERT(PAGE_SIZE < 4096, - ("%s: PAGE_SIZE %d, hcf %d", __func__, PAGE_SIZE, hcf)); - CTR3(KTR_CXGBE, "%s: PAGE_SIZE %d, hcf %d", - __func__, PAGE_SIZE, hcf); +#define PR_PAGE_MASK(x) ((1 << pr->pr_page_shift[(x)]) - 1) + MPASS((hcf & PR_PAGE_MASK(0)) == 0); /* PAGE_SIZE is >= 4K everywhere */ + for (idx = nitems(pr->pr_page_shift) - 1; idx > 0; idx--) { + if ((hcf & PR_PAGE_MASK(idx)) == 0) + break; + } +#undef PR_PAGE_MASK + +have_pgsz: + MPASS(idx <= M_PPOD_PGSZ); + + nppods = pages_to_nppods(ps->npages, pr->pr_page_shift[idx]); + if (alloc_page_pods(pr, nppods, idx, prsv) != 0) return (0); + MPASS(prsv->prsv_nppods > 0); + + return (1); +} + +int +t4_alloc_page_pods_for_buf(struct ppod_region *pr, vm_offset_t buf, int len, + struct ppod_reservation *prsv) +{ + int hcf, seglen, idx, npages, nppods; + uintptr_t start_pva, end_pva, pva, p1; + + MPASS(buf > 0); + MPASS(len > 0); + + /* + * The DDP page size is unrelated to the VM page size. We combine + * contiguous physical pages into larger segments to get the best DDP + * page size possible. This is the largest of the four sizes in + * A_ULP_RX_ISCSI_PSZ that evenly divides the HCF of the segment sizes + * in the page list. + */ + hcf = 0; + start_pva = trunc_page(buf); + end_pva = trunc_page(buf + len - 1); + pva = start_pva; + while (pva <= end_pva) { + seglen = PAGE_SIZE; + p1 = pmap_kextract(pva); + pva += PAGE_SIZE; + while (pva <= end_pva && p1 + seglen == pmap_kextract(pva)) { + seglen += PAGE_SIZE; + pva += PAGE_SIZE; + } + + hcf = calculate_hcf(hcf, seglen); + if (hcf < (1 << pr->pr_page_shift[1])) { + idx = 0; + goto have_pgsz; /* give up, short circuit */ + } } - for (idx = nitems(td->ddp_pgsz) - 1; idx > 0; idx--) { - if (hcf % td->ddp_pgsz[idx] == 0) +#define PR_PAGE_MASK(x) ((1 << pr->pr_page_shift[(x)]) - 1) + MPASS((hcf & PR_PAGE_MASK(0)) == 0); /* PAGE_SIZE is >= 4K everywhere */ + for (idx = nitems(pr->pr_page_shift) - 1; idx > 0; idx--) { + if ((hcf & PR_PAGE_MASK(idx)) == 0) break; } +#undef PR_PAGE_MASK + have_pgsz: MPASS(idx <= M_PPOD_PGSZ); - nppods = pages_to_nppods(ps->npages, td->ddp_pgsz[idx]); - if (alloc_ppods(td, nppods, &ppod_addr) != 0) { - CTR4(KTR_CXGBE, "%s: no pods, nppods %d, npages %d, pgsz %d", - __func__, nppods, ps->npages, td->ddp_pgsz[idx]); - return (0); - } + npages = 1; + npages += (end_pva - start_pva) >> pr->pr_page_shift[idx]; + nppods = howmany(npages, PPOD_PAGES); + if (alloc_page_pods(pr, nppods, idx, prsv) != 0) + return (ENOMEM); + MPASS(prsv->prsv_nppods > 0); - ppod = (ppod_addr - td->ppod_start) / PPOD_SIZE; - ps->tag = V_PPOD_PGSZ(idx) | V_PPOD_TAG(ppod); - ps->ppod_addr = ppod_addr; - ps->nppods = nppods; + return (0); +} - CTR5(KTR_CXGBE, "New page pods. " - "ps %p, ddp_pgsz %d, ppod 0x%x, npages %d, nppods %d", - ps, td->ddp_pgsz[idx], ppod, ps->npages, ps->nppods); +void +t4_free_page_pods(struct ppod_reservation *prsv) +{ + struct ppod_region *pr = prsv->prsv_pr; + vmem_addr_t addr; - return (1); + MPASS(prsv != NULL); + MPASS(prsv->prsv_nppods != 0); + + addr = prsv->prsv_tag & pr->pr_tag_mask; + MPASS((addr & pr->pr_invalid_bit) == 0); + + CTR4(KTR_CXGBE, "%-17s arena %p, addr 0x%08x, nppods %d", __func__, + pr->pr_arena, addr, prsv->prsv_nppods); + + vmem_free(pr->pr_arena, addr, PPOD_SZ(prsv->prsv_nppods)); + prsv->prsv_nppods = 0; } #define NUM_ULP_TX_SC_IMM_PPODS (256 / PPOD_SIZE) -static int -write_page_pods(struct adapter *sc, struct toepcb *toep, struct pageset *ps) +int +t4_write_page_pods_for_ps(struct adapter *sc, struct sge_wrq *wrq, int tid, + struct pageset *ps) { struct wrqe *wr; struct ulp_mem_io *ulpmc; struct ulptx_idata *ulpsc; struct pagepod *ppod; - struct tom_data *td = sc->tom_softc; int i, j, k, n, chunk, len, ddp_pgsz, idx; u_int ppod_addr; uint32_t cmd; + struct ppod_reservation *prsv = &ps->prsv; + struct ppod_region *pr = prsv->prsv_pr; KASSERT(!(ps->flags & PS_PPODS_WRITTEN), ("%s: page pods already written", __func__)); + MPASS(prsv->prsv_nppods > 0); cmd = htobe32(V_ULPTX_CMD(ULP_TX_MEM_WRITE)); if (is_t4(sc)) cmd |= htobe32(F_ULP_MEMIO_ORDER); else cmd |= htobe32(F_T5_ULP_MEMIO_IMM); - ddp_pgsz = td->ddp_pgsz[G_PPOD_PGSZ(ps->tag)]; - ppod_addr = ps->ppod_addr; - for (i = 0; i < ps->nppods; ppod_addr += chunk) { + ddp_pgsz = 1 << pr->pr_page_shift[G_PPOD_PGSZ(prsv->prsv_tag)]; + ppod_addr = pr->pr_start + (prsv->prsv_tag & pr->pr_tag_mask); + for (i = 0; i < prsv->prsv_nppods; ppod_addr += chunk) { /* How many page pods are we writing in this cycle */ - n = min(ps->nppods - i, NUM_ULP_TX_SC_IMM_PPODS); + n = min(prsv->prsv_nppods - i, NUM_ULP_TX_SC_IMM_PPODS); chunk = PPOD_SZ(n); len = roundup2(sizeof(*ulpmc) + sizeof(*ulpsc) + chunk, 16); - wr = alloc_wrqe(len, toep->ctrlq); + wr = alloc_wrqe(len, wrq); if (wr == NULL) return (ENOMEM); /* ok to just bail out */ ulpmc = wrtod(wr); INIT_ULPTX_WR(ulpmc, len, 0, 0); ulpmc->cmd = cmd; ulpmc->dlen = htobe32(V_ULP_MEMIO_DATA_LEN(chunk / 32)); ulpmc->len16 = htobe32(howmany(len - sizeof(ulpmc->wr), 16)); ulpmc->lock_addr = htobe32(V_ULP_MEMIO_ADDR(ppod_addr >> 5)); ulpsc = (struct ulptx_idata *)(ulpmc + 1); ulpsc->cmd_more = htobe32(V_ULPTX_CMD(ULP_TX_SC_IMM)); ulpsc->len = htobe32(chunk); ppod = (struct pagepod *)(ulpsc + 1); for (j = 0; j < n; i++, j++, ppod++) { ppod->vld_tid_pgsz_tag_color = htobe64(F_PPOD_VALID | - V_PPOD_TID(toep->tid) | ps->tag); + V_PPOD_TID(tid) | prsv->prsv_tag); ppod->len_offset = htobe64(V_PPOD_LEN(ps->len) | V_PPOD_OFST(ps->offset)); ppod->rsvd = 0; idx = i * PPOD_PAGES * (ddp_pgsz / PAGE_SIZE); for (k = 0; k < nitems(ppod->addr); k++) { if (idx < ps->npages) { ppod->addr[k] = htobe64(ps->pages[idx]->phys_addr); idx += ddp_pgsz / PAGE_SIZE; } else ppod->addr[k] = 0; #if 0 CTR5(KTR_CXGBE, "%s: tid %d ppod[%d]->addr[%d] = %p", __func__, toep->tid, i, k, htobe64(ppod->addr[k])); #endif } } t4_wrq_tx(sc, wr); } ps->flags |= PS_PPODS_WRITTEN; return (0); } +int +t4_write_page_pods_for_buf(struct adapter *sc, struct sge_wrq *wrq, int tid, + struct ppod_reservation *prsv, vm_offset_t buf, int buflen) +{ + struct wrqe *wr; + struct ulp_mem_io *ulpmc; + struct ulptx_idata *ulpsc; + struct pagepod *ppod; + int i, j, k, n, chunk, len, ddp_pgsz; + u_int ppod_addr, offset; + uint32_t cmd; + struct ppod_region *pr = prsv->prsv_pr; + uintptr_t end_pva, pva, pa; + + cmd = htobe32(V_ULPTX_CMD(ULP_TX_MEM_WRITE)); + if (is_t4(sc)) + cmd |= htobe32(F_ULP_MEMIO_ORDER); + else + cmd |= htobe32(F_T5_ULP_MEMIO_IMM); + ddp_pgsz = 1 << pr->pr_page_shift[G_PPOD_PGSZ(prsv->prsv_tag)]; + offset = buf & PAGE_MASK; + ppod_addr = pr->pr_start + (prsv->prsv_tag & pr->pr_tag_mask); + pva = trunc_page(buf); + end_pva = trunc_page(buf + buflen - 1); + for (i = 0; i < prsv->prsv_nppods; ppod_addr += chunk) { + + /* How many page pods are we writing in this cycle */ + n = min(prsv->prsv_nppods - i, NUM_ULP_TX_SC_IMM_PPODS); + MPASS(n > 0); + chunk = PPOD_SZ(n); + len = roundup2(sizeof(*ulpmc) + sizeof(*ulpsc) + chunk, 16); + + wr = alloc_wrqe(len, wrq); + if (wr == NULL) + return (ENOMEM); /* ok to just bail out */ + ulpmc = wrtod(wr); + + INIT_ULPTX_WR(ulpmc, len, 0, 0); + ulpmc->cmd = cmd; + ulpmc->dlen = htobe32(V_ULP_MEMIO_DATA_LEN(chunk / 32)); + ulpmc->len16 = htobe32(howmany(len - sizeof(ulpmc->wr), 16)); + ulpmc->lock_addr = htobe32(V_ULP_MEMIO_ADDR(ppod_addr >> 5)); + + ulpsc = (struct ulptx_idata *)(ulpmc + 1); + ulpsc->cmd_more = htobe32(V_ULPTX_CMD(ULP_TX_SC_IMM)); + ulpsc->len = htobe32(chunk); + + ppod = (struct pagepod *)(ulpsc + 1); + for (j = 0; j < n; i++, j++, ppod++) { + ppod->vld_tid_pgsz_tag_color = htobe64(F_PPOD_VALID | + V_PPOD_TID(tid) | + (prsv->prsv_tag & ~V_PPOD_PGSZ(M_PPOD_PGSZ))); + ppod->len_offset = htobe64(V_PPOD_LEN(buflen) | + V_PPOD_OFST(offset)); + ppod->rsvd = 0; + + for (k = 0; k < nitems(ppod->addr); k++) { + if (pva > end_pva) + ppod->addr[k] = 0; + else { + pa = pmap_kextract(pva); + ppod->addr[k] = htobe64(pa); + pva += ddp_pgsz; + } +#if 0 + CTR5(KTR_CXGBE, + "%s: tid %d ppod[%d]->addr[%d] = %p", + __func__, tid, i, k, + htobe64(ppod->addr[k])); +#endif + } + + /* + * Walk back 1 segment so that the first address in the + * next pod is the same as the last one in the current + * pod. + */ + pva -= ddp_pgsz; + } + + t4_wrq_tx(sc, wr); + } + + MPASS(pva <= end_pva); + + return (0); +} + static void wire_pageset(struct pageset *ps) { vm_page_t p; int i; KASSERT(!(ps->flags & PS_WIRED), ("pageset already wired")); for (i = 0; i < ps->npages; i++) { p = ps->pages[i]; vm_page_lock(p); vm_page_wire(p); vm_page_unhold(p); vm_page_unlock(p); } ps->flags |= PS_WIRED; } /* * Prepare a pageset for DDP. This wires the pageset and sets up page * pods. */ static int prep_pageset(struct adapter *sc, struct toepcb *toep, struct pageset *ps) { struct tom_data *td = sc->tom_softc; if (!(ps->flags & PS_WIRED)) wire_pageset(ps); - if (ps->nppods == 0 && !alloc_page_pods(td, ps)) { + if (ps->prsv.prsv_nppods == 0 && + !t4_alloc_page_pods_for_ps(&td->pr, ps)) { return (0); } if (!(ps->flags & PS_PPODS_WRITTEN) && - write_page_pods(sc, toep, ps) != 0) { + t4_write_page_pods_for_ps(sc, toep->ctrlq, toep->tid, ps) != 0) { return (0); } return (1); } -void -t4_init_ddp(struct adapter *sc, struct tom_data *td) +int +t4_init_ppod_region(struct ppod_region *pr, struct t4_range *r, u_int psz, + const char *name) { int i; - uint32_t r; - r = t4_read_reg(sc, A_ULP_RX_TDDP_PSZ); - td->ddp_pgsz[0] = 4096 << G_HPZ0(r); - td->ddp_pgsz[1] = 4096 << G_HPZ1(r); - td->ddp_pgsz[2] = 4096 << G_HPZ2(r); - td->ddp_pgsz[3] = 4096 << G_HPZ3(r); + MPASS(pr != NULL); + MPASS(r->size > 0); - /* - * The SGL -> page pod algorithm requires the sizes to be in increasing - * order. - */ - for (i = 1; i < nitems(td->ddp_pgsz); i++) { - if (td->ddp_pgsz[i] <= td->ddp_pgsz[i - 1]) - return; + pr->pr_start = r->start; + pr->pr_len = r->size; + pr->pr_page_shift[0] = 12 + G_HPZ0(psz); + pr->pr_page_shift[1] = 12 + G_HPZ1(psz); + pr->pr_page_shift[2] = 12 + G_HPZ2(psz); + pr->pr_page_shift[3] = 12 + G_HPZ3(psz); + + /* The SGL -> page pod algorithm requires the sizes to be in order. */ + for (i = 1; i < nitems(pr->pr_page_shift); i++) { + if (pr->pr_page_shift[i] <= pr->pr_page_shift[i - 1]) + return (ENXIO); } - td->ppod_start = sc->vres.ddp.start; - td->ppod_arena = vmem_create("DDP page pods", sc->vres.ddp.start, - sc->vres.ddp.size, PPOD_SIZE, 512, M_FIRSTFIT | M_NOWAIT); + pr->pr_tag_mask = ((1 << fls(r->size)) - 1) & V_PPOD_TAG(M_PPOD_TAG); + pr->pr_alias_mask = V_PPOD_TAG(M_PPOD_TAG) & ~pr->pr_tag_mask; + if (pr->pr_tag_mask == 0 || pr->pr_alias_mask == 0) + return (ENXIO); + pr->pr_alias_shift = fls(pr->pr_tag_mask); + pr->pr_invalid_bit = 1 << (pr->pr_alias_shift - 1); + + pr->pr_arena = vmem_create(name, 0, pr->pr_len, PPOD_SIZE, 0, + M_FIRSTFIT | M_NOWAIT); + if (pr->pr_arena == NULL) + return (ENOMEM); + + return (0); } void -t4_uninit_ddp(struct adapter *sc __unused, struct tom_data *td) +t4_free_ppod_region(struct ppod_region *pr) { - if (td->ppod_arena != NULL) { - vmem_destroy(td->ppod_arena); - td->ppod_arena = NULL; - } + MPASS(pr != NULL); + + if (pr->pr_arena) + vmem_destroy(pr->pr_arena); + bzero(pr, sizeof(*pr)); } static int pscmp(struct pageset *ps, struct vmspace *vm, vm_offset_t start, int npages, int pgoff, int len) { if (ps->npages != npages || ps->offset != pgoff || ps->len != len) return (1); return (ps->vm != vm || ps->vm_timestamp != vm->vm_map.timestamp); } static int hold_aio(struct toepcb *toep, struct kaiocb *job, struct pageset **pps) { struct vmspace *vm; vm_map_t map; vm_offset_t start, end, pgoff; struct pageset *ps; int n; DDP_ASSERT_LOCKED(toep); /* * The AIO subsystem will cancel and drain all requests before * permitting a process to exit or exec, so p_vmspace should * be stable here. */ vm = job->userproc->p_vmspace; map = &vm->vm_map; start = (uintptr_t)job->uaiocb.aio_buf; pgoff = start & PAGE_MASK; end = round_page(start + job->uaiocb.aio_nbytes); start = trunc_page(start); if (end - start > MAX_DDP_BUFFER_SIZE) { /* * Truncate the request to a short read. * Alternatively, we could DDP in chunks to the larger * buffer, but that would be quite a bit more work. * * When truncating, round the request down to avoid * crossing a cache line on the final transaction. */ end = rounddown2(start + MAX_DDP_BUFFER_SIZE, CACHE_LINE_SIZE); #ifdef VERBOSE_TRACES CTR4(KTR_CXGBE, "%s: tid %d, truncating size from %lu to %lu", __func__, toep->tid, (unsigned long)job->uaiocb.aio_nbytes, (unsigned long)(end - (start + pgoff))); job->uaiocb.aio_nbytes = end - (start + pgoff); #endif end = round_page(end); } n = atop(end - start); /* * Try to reuse a cached pageset. */ TAILQ_FOREACH(ps, &toep->ddp_cached_pagesets, link) { if (pscmp(ps, vm, start, n, pgoff, job->uaiocb.aio_nbytes) == 0) { TAILQ_REMOVE(&toep->ddp_cached_pagesets, ps, link); toep->ddp_cached_count--; *pps = ps; return (0); } } /* * If there are too many cached pagesets to create a new one, * free a pageset before creating a new one. */ KASSERT(toep->ddp_active_count + toep->ddp_cached_count <= nitems(toep->db), ("%s: too many wired pagesets", __func__)); if (toep->ddp_active_count + toep->ddp_cached_count == nitems(toep->db)) { KASSERT(toep->ddp_cached_count > 0, ("no cached pageset to free")); ps = TAILQ_LAST(&toep->ddp_cached_pagesets, pagesetq); TAILQ_REMOVE(&toep->ddp_cached_pagesets, ps, link); toep->ddp_cached_count--; free_pageset(toep->td, ps); } DDP_UNLOCK(toep); /* Create a new pageset. */ ps = malloc(sizeof(*ps) + n * sizeof(vm_page_t), M_CXGBE, M_WAITOK | M_ZERO); ps->pages = (vm_page_t *)(ps + 1); ps->vm_timestamp = map->timestamp; ps->npages = vm_fault_quick_hold_pages(map, start, end - start, VM_PROT_WRITE, ps->pages, n); DDP_LOCK(toep); if (ps->npages < 0) { free(ps, M_CXGBE); return (EFAULT); } KASSERT(ps->npages == n, ("hold_aio: page count mismatch: %d vs %d", ps->npages, n)); ps->offset = pgoff; ps->len = job->uaiocb.aio_nbytes; atomic_add_int(&vm->vm_refcnt, 1); ps->vm = vm; CTR5(KTR_CXGBE, "%s: tid %d, new pageset %p for job %p, npages %d", __func__, toep->tid, ps, job, ps->npages); *pps = ps; return (0); } static void ddp_complete_all(struct toepcb *toep, int error) { struct kaiocb *job; DDP_ASSERT_LOCKED(toep); while (!TAILQ_EMPTY(&toep->ddp_aiojobq)) { job = TAILQ_FIRST(&toep->ddp_aiojobq); TAILQ_REMOVE(&toep->ddp_aiojobq, job, list); toep->ddp_waiting_count--; if (aio_clear_cancel_function(job)) ddp_complete_one(job, error); } } static void aio_ddp_cancel_one(struct kaiocb *job) { long copied; /* * If this job had copied data out of the socket buffer before * it was cancelled, report it as a short read rather than an * error. */ copied = job->aio_received; if (copied != 0) aio_complete(job, copied, 0); else aio_cancel(job); } /* * Called when the main loop wants to requeue a job to retry it later. * Deals with the race of the job being cancelled while it was being * examined. */ static void aio_ddp_requeue_one(struct toepcb *toep, struct kaiocb *job) { DDP_ASSERT_LOCKED(toep); if (!(toep->ddp_flags & DDP_DEAD) && aio_set_cancel_function(job, t4_aio_cancel_queued)) { TAILQ_INSERT_HEAD(&toep->ddp_aiojobq, job, list); toep->ddp_waiting_count++; } else aio_ddp_cancel_one(job); } static void aio_ddp_requeue(struct toepcb *toep) { struct adapter *sc = td_adapter(toep->td); struct socket *so; struct sockbuf *sb; struct inpcb *inp; struct kaiocb *job; struct ddp_buffer *db; size_t copied, offset, resid; struct pageset *ps; struct mbuf *m; uint64_t ddp_flags, ddp_flags_mask; struct wrqe *wr; int buf_flag, db_idx, error; DDP_ASSERT_LOCKED(toep); restart: if (toep->ddp_flags & DDP_DEAD) { MPASS(toep->ddp_waiting_count == 0); MPASS(toep->ddp_active_count == 0); return; } if (toep->ddp_waiting_count == 0 || toep->ddp_active_count == nitems(toep->db)) { return; } job = TAILQ_FIRST(&toep->ddp_aiojobq); so = job->fd_file->f_data; sb = &so->so_rcv; SOCKBUF_LOCK(sb); /* We will never get anything unless we are or were connected. */ if (!(so->so_state & (SS_ISCONNECTED|SS_ISDISCONNECTED))) { SOCKBUF_UNLOCK(sb); ddp_complete_all(toep, ENOTCONN); return; } KASSERT(toep->ddp_active_count == 0 || sbavail(sb) == 0, ("%s: pending sockbuf data and DDP is active", __func__)); /* Abort if socket has reported problems. */ /* XXX: Wait for any queued DDP's to finish and/or flush them? */ if (so->so_error && sbavail(sb) == 0) { toep->ddp_waiting_count--; TAILQ_REMOVE(&toep->ddp_aiojobq, job, list); if (!aio_clear_cancel_function(job)) { SOCKBUF_UNLOCK(sb); goto restart; } /* * If this job has previously copied some data, report * a short read and leave the error to be reported by * a future request. */ copied = job->aio_received; if (copied != 0) { SOCKBUF_UNLOCK(sb); aio_complete(job, copied, 0); goto restart; } error = so->so_error; so->so_error = 0; SOCKBUF_UNLOCK(sb); aio_complete(job, -1, error); goto restart; } /* * Door is closed. If there is pending data in the socket buffer, * deliver it. If there are pending DDP requests, wait for those * to complete. Once they have completed, return EOF reads. */ if (sb->sb_state & SBS_CANTRCVMORE && sbavail(sb) == 0) { SOCKBUF_UNLOCK(sb); if (toep->ddp_active_count != 0) return; ddp_complete_all(toep, 0); return; } /* * If DDP is not enabled and there is no pending socket buffer * data, try to enable DDP. */ if (sbavail(sb) == 0 && (toep->ddp_flags & DDP_ON) == 0) { SOCKBUF_UNLOCK(sb); /* * Wait for the card to ACK that DDP is enabled before * queueing any buffers. Currently this waits for an * indicate to arrive. This could use a TCB_SET_FIELD_RPL * message to know that DDP was enabled instead of waiting * for the indicate which would avoid copying the indicate * if no data is pending. * * XXX: Might want to limit the indicate size to the size * of the first queued request. */ if ((toep->ddp_flags & DDP_SC_REQ) == 0) enable_ddp(sc, toep); return; } SOCKBUF_UNLOCK(sb); /* * If another thread is queueing a buffer for DDP, let it * drain any work and return. */ if (toep->ddp_queueing != NULL) return; /* Take the next job to prep it for DDP. */ toep->ddp_waiting_count--; TAILQ_REMOVE(&toep->ddp_aiojobq, job, list); if (!aio_clear_cancel_function(job)) goto restart; toep->ddp_queueing = job; /* NB: This drops DDP_LOCK while it holds the backing VM pages. */ error = hold_aio(toep, job, &ps); if (error != 0) { ddp_complete_one(job, error); toep->ddp_queueing = NULL; goto restart; } SOCKBUF_LOCK(sb); if (so->so_error && sbavail(sb) == 0) { copied = job->aio_received; if (copied != 0) { SOCKBUF_UNLOCK(sb); recycle_pageset(toep, ps); aio_complete(job, copied, 0); toep->ddp_queueing = NULL; goto restart; } error = so->so_error; so->so_error = 0; SOCKBUF_UNLOCK(sb); recycle_pageset(toep, ps); aio_complete(job, -1, error); toep->ddp_queueing = NULL; goto restart; } if (sb->sb_state & SBS_CANTRCVMORE && sbavail(sb) == 0) { SOCKBUF_UNLOCK(sb); recycle_pageset(toep, ps); if (toep->ddp_active_count != 0) { /* * The door is closed, but there are still pending * DDP buffers. Requeue. These jobs will all be * completed once those buffers drain. */ aio_ddp_requeue_one(toep, job); toep->ddp_queueing = NULL; return; } ddp_complete_one(job, 0); ddp_complete_all(toep, 0); toep->ddp_queueing = NULL; return; } sbcopy: /* * If the toep is dead, there shouldn't be any data in the socket * buffer, so the above case should have handled this. */ MPASS(!(toep->ddp_flags & DDP_DEAD)); /* * If there is pending data in the socket buffer (either * from before the requests were queued or a DDP indicate), * copy those mbufs out directly. */ copied = 0; offset = ps->offset + job->aio_received; MPASS(job->aio_received <= job->uaiocb.aio_nbytes); resid = job->uaiocb.aio_nbytes - job->aio_received; m = sb->sb_mb; KASSERT(m == NULL || toep->ddp_active_count == 0, ("%s: sockbuf data with active DDP", __func__)); while (m != NULL && resid > 0) { struct iovec iov[1]; struct uio uio; int error; iov[0].iov_base = mtod(m, void *); iov[0].iov_len = m->m_len; if (iov[0].iov_len > resid) iov[0].iov_len = resid; uio.uio_iov = iov; uio.uio_iovcnt = 1; uio.uio_offset = 0; uio.uio_resid = iov[0].iov_len; uio.uio_segflg = UIO_SYSSPACE; uio.uio_rw = UIO_WRITE; error = uiomove_fromphys(ps->pages, offset + copied, uio.uio_resid, &uio); MPASS(error == 0 && uio.uio_resid == 0); copied += uio.uio_offset; resid -= uio.uio_offset; m = m->m_next; } if (copied != 0) { sbdrop_locked(sb, copied); job->aio_received += copied; job->msgrcv = 1; copied = job->aio_received; inp = sotoinpcb(so); if (!INP_TRY_WLOCK(inp)) { /* * The reference on the socket file descriptor in * the AIO job should keep 'sb' and 'inp' stable. * Our caller has a reference on the 'toep' that * keeps it stable. */ SOCKBUF_UNLOCK(sb); DDP_UNLOCK(toep); INP_WLOCK(inp); DDP_LOCK(toep); SOCKBUF_LOCK(sb); /* * If the socket has been closed, we should detect * that and complete this request if needed on * the next trip around the loop. */ } t4_rcvd_locked(&toep->td->tod, intotcpcb(inp)); INP_WUNLOCK(inp); if (resid == 0 || toep->ddp_flags & DDP_DEAD) { /* * We filled the entire buffer with socket * data, DDP is not being used, or the socket * is being shut down, so complete the * request. */ SOCKBUF_UNLOCK(sb); recycle_pageset(toep, ps); aio_complete(job, copied, 0); toep->ddp_queueing = NULL; goto restart; } /* * If DDP is not enabled, requeue this request and restart. * This will either enable DDP or wait for more data to * arrive on the socket buffer. */ if ((toep->ddp_flags & (DDP_ON | DDP_SC_REQ)) != DDP_ON) { SOCKBUF_UNLOCK(sb); recycle_pageset(toep, ps); aio_ddp_requeue_one(toep, job); toep->ddp_queueing = NULL; goto restart; } /* * An indicate might have arrived and been added to * the socket buffer while it was unlocked after the * copy to lock the INP. If so, restart the copy. */ if (sbavail(sb) != 0) goto sbcopy; } SOCKBUF_UNLOCK(sb); if (prep_pageset(sc, toep, ps) == 0) { recycle_pageset(toep, ps); aio_ddp_requeue_one(toep, job); toep->ddp_queueing = NULL; /* * XXX: Need to retry this later. Mostly need a trigger * when page pods are freed up. */ printf("%s: prep_pageset failed\n", __func__); return; } /* Determine which DDP buffer to use. */ if (toep->db[0].job == NULL) { db_idx = 0; } else { MPASS(toep->db[1].job == NULL); db_idx = 1; } ddp_flags = 0; ddp_flags_mask = 0; if (db_idx == 0) { ddp_flags |= V_TF_DDP_BUF0_VALID(1); if (so->so_state & SS_NBIO) ddp_flags |= V_TF_DDP_BUF0_FLUSH(1); ddp_flags_mask |= V_TF_DDP_PSH_NO_INVALIDATE0(1) | V_TF_DDP_PUSH_DISABLE_0(1) | V_TF_DDP_PSHF_ENABLE_0(1) | V_TF_DDP_BUF0_FLUSH(1) | V_TF_DDP_BUF0_VALID(1); buf_flag = DDP_BUF0_ACTIVE; } else { ddp_flags |= V_TF_DDP_BUF1_VALID(1); if (so->so_state & SS_NBIO) ddp_flags |= V_TF_DDP_BUF1_FLUSH(1); ddp_flags_mask |= V_TF_DDP_PSH_NO_INVALIDATE1(1) | V_TF_DDP_PUSH_DISABLE_1(1) | V_TF_DDP_PSHF_ENABLE_1(1) | V_TF_DDP_BUF1_FLUSH(1) | V_TF_DDP_BUF1_VALID(1); buf_flag = DDP_BUF1_ACTIVE; } MPASS((toep->ddp_flags & buf_flag) == 0); if ((toep->ddp_flags & (DDP_BUF0_ACTIVE | DDP_BUF1_ACTIVE)) == 0) { MPASS(db_idx == 0); MPASS(toep->ddp_active_id == -1); MPASS(toep->ddp_active_count == 0); ddp_flags_mask |= V_TF_DDP_ACTIVE_BUF(1); } /* * The TID for this connection should still be valid. If DDP_DEAD * is set, SBS_CANTRCVMORE should be set, so we shouldn't be * this far anyway. Even if the socket is closing on the other * end, the AIO job holds a reference on this end of the socket * which will keep it open and keep the TCP PCB attached until * after the job is completed. */ wr = mk_update_tcb_for_ddp(sc, toep, db_idx, ps, job->aio_received, ddp_flags, ddp_flags_mask); if (wr == NULL) { recycle_pageset(toep, ps); aio_ddp_requeue_one(toep, job); toep->ddp_queueing = NULL; /* * XXX: Need a way to kick a retry here. * * XXX: We know the fixed size needed and could * preallocate this using a blocking request at the * start of the task to avoid having to handle this * edge case. */ printf("%s: mk_update_tcb_for_ddp failed\n", __func__); return; } if (!aio_set_cancel_function(job, t4_aio_cancel_active)) { free_wrqe(wr); recycle_pageset(toep, ps); aio_ddp_cancel_one(job); toep->ddp_queueing = NULL; goto restart; } #ifdef VERBOSE_TRACES CTR5(KTR_CXGBE, "%s: scheduling %p for DDP[%d] (flags %#lx/%#lx)", __func__, job, db_idx, ddp_flags, ddp_flags_mask); #endif /* Give the chip the go-ahead. */ t4_wrq_tx(sc, wr); db = &toep->db[db_idx]; db->cancel_pending = 0; db->job = job; db->ps = ps; toep->ddp_queueing = NULL; toep->ddp_flags |= buf_flag; toep->ddp_active_count++; if (toep->ddp_active_count == 1) { MPASS(toep->ddp_active_id == -1); toep->ddp_active_id = db_idx; CTR2(KTR_CXGBE, "%s: ddp_active_id = %d", __func__, toep->ddp_active_id); } goto restart; } void ddp_queue_toep(struct toepcb *toep) { DDP_ASSERT_LOCKED(toep); if (toep->ddp_flags & DDP_TASK_ACTIVE) return; toep->ddp_flags |= DDP_TASK_ACTIVE; hold_toepcb(toep); soaio_enqueue(&toep->ddp_requeue_task); } static void aio_ddp_requeue_task(void *context, int pending) { struct toepcb *toep = context; DDP_LOCK(toep); aio_ddp_requeue(toep); toep->ddp_flags &= ~DDP_TASK_ACTIVE; DDP_UNLOCK(toep); free_toepcb(toep); } static void t4_aio_cancel_active(struct kaiocb *job) { struct socket *so = job->fd_file->f_data; struct tcpcb *tp = so_sototcpcb(so); struct toepcb *toep = tp->t_toe; struct adapter *sc = td_adapter(toep->td); uint64_t valid_flag; int i; DDP_LOCK(toep); if (aio_cancel_cleared(job)) { DDP_UNLOCK(toep); aio_ddp_cancel_one(job); return; } for (i = 0; i < nitems(toep->db); i++) { if (toep->db[i].job == job) { /* Should only ever get one cancel request for a job. */ MPASS(toep->db[i].cancel_pending == 0); /* * Invalidate this buffer. It will be * cancelled or partially completed once the * card ACKs the invalidate. */ valid_flag = i == 0 ? V_TF_DDP_BUF0_VALID(1) : V_TF_DDP_BUF1_VALID(1); t4_set_tcb_field(sc, toep->ctrlq, toep->tid, W_TCB_RX_DDP_FLAGS, valid_flag, 0, 1, i + DDP_BUF0_INVALIDATED, toep->ofld_rxq->iq.abs_id); toep->db[i].cancel_pending = 1; CTR2(KTR_CXGBE, "%s: request %p marked pending", __func__, job); break; } } DDP_UNLOCK(toep); } static void t4_aio_cancel_queued(struct kaiocb *job) { struct socket *so = job->fd_file->f_data; struct tcpcb *tp = so_sototcpcb(so); struct toepcb *toep = tp->t_toe; DDP_LOCK(toep); if (!aio_cancel_cleared(job)) { TAILQ_REMOVE(&toep->ddp_aiojobq, job, list); toep->ddp_waiting_count--; if (toep->ddp_waiting_count == 0) ddp_queue_toep(toep); } CTR2(KTR_CXGBE, "%s: request %p cancelled", __func__, job); DDP_UNLOCK(toep); aio_ddp_cancel_one(job); } int t4_aio_queue_ddp(struct socket *so, struct kaiocb *job) { struct tcpcb *tp = so_sototcpcb(so); struct toepcb *toep = tp->t_toe; /* Ignore writes. */ if (job->uaiocb.aio_lio_opcode != LIO_READ) return (EOPNOTSUPP); DDP_LOCK(toep); /* * XXX: Think about possibly returning errors for ENOTCONN, * etc. Perhaps the caller would only queue the request * if it failed with EOPNOTSUPP? */ #ifdef VERBOSE_TRACES CTR2(KTR_CXGBE, "%s: queueing %p", __func__, job); #endif if (!aio_set_cancel_function(job, t4_aio_cancel_queued)) panic("new job was cancelled"); TAILQ_INSERT_TAIL(&toep->ddp_aiojobq, job, list); toep->ddp_waiting_count++; toep->ddp_flags |= DDP_OK; /* * Try to handle this request synchronously. If this has * to block because the task is running, it will just bail * and let the task handle it instead. */ aio_ddp_requeue(toep); DDP_UNLOCK(toep); return (0); } int t4_ddp_mod_load(void) { t4_register_cpl_handler(CPL_RX_DATA_DDP, do_rx_data_ddp); t4_register_cpl_handler(CPL_RX_DDP_COMPLETE, do_rx_ddp_complete); TAILQ_INIT(&ddp_orphan_pagesets); mtx_init(&ddp_orphan_pagesets_lock, "ddp orphans", NULL, MTX_DEF); TASK_INIT(&ddp_orphan_task, 0, ddp_free_orphan_pagesets, NULL); return (0); } void t4_ddp_mod_unload(void) { taskqueue_drain(taskqueue_thread, &ddp_orphan_task); MPASS(TAILQ_EMPTY(&ddp_orphan_pagesets)); mtx_destroy(&ddp_orphan_pagesets_lock); t4_register_cpl_handler(CPL_RX_DATA_DDP, NULL); t4_register_cpl_handler(CPL_RX_DDP_COMPLETE, NULL); } #endif Index: projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_tom.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_tom.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_tom.c (revision 305172) @@ -1,1245 +1,1249 @@ /*- * Copyright (c) 2012 Chelsio Communications, Inc. * All rights reserved. * Written by: Navdeep Parhar * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_inet.h" #include "opt_inet6.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define TCPSTATES #include #include #include #ifdef TCP_OFFLOAD #include "common/common.h" #include "common/t4_msg.h" #include "common/t4_regs.h" #include "common/t4_regs_values.h" #include "common/t4_tcb.h" #include "tom/t4_tom_l2t.h" #include "tom/t4_tom.h" static struct protosw toe_protosw; static struct pr_usrreqs toe_usrreqs; static struct protosw toe6_protosw; static struct pr_usrreqs toe6_usrreqs; /* Module ops */ static int t4_tom_mod_load(void); static int t4_tom_mod_unload(void); static int t4_tom_modevent(module_t, int, void *); /* ULD ops and helpers */ static int t4_tom_activate(struct adapter *); static int t4_tom_deactivate(struct adapter *); static struct uld_info tom_uld_info = { .uld_id = ULD_TOM, .activate = t4_tom_activate, .deactivate = t4_tom_deactivate, }; static void queue_tid_release(struct adapter *, int); static void release_offload_resources(struct toepcb *); static int alloc_tid_tabs(struct tid_info *); static void free_tid_tabs(struct tid_info *); static int add_lip(struct adapter *, struct in6_addr *); static int delete_lip(struct adapter *, struct in6_addr *); static struct clip_entry *search_lip(struct tom_data *, struct in6_addr *); static void init_clip_table(struct adapter *, struct tom_data *); static void update_clip(struct adapter *, void *); static void t4_clip_task(void *, int); static void update_clip_table(struct adapter *, struct tom_data *); static void destroy_clip_table(struct adapter *, struct tom_data *); static void free_tom_data(struct adapter *, struct tom_data *); static void reclaim_wr_resources(void *, int); static int in6_ifaddr_gen; static eventhandler_tag ifaddr_evhandler; static struct timeout_task clip_task; struct toepcb * alloc_toepcb(struct vi_info *vi, int txqid, int rxqid, int flags) { struct port_info *pi = vi->pi; struct adapter *sc = pi->adapter; struct toepcb *toep; int tx_credits, txsd_total, len; /* * The firmware counts tx work request credits in units of 16 bytes * each. Reserve room for an ABORT_REQ so the driver never has to worry * about tx credits if it wants to abort a connection. */ tx_credits = sc->params.ofldq_wr_cred; tx_credits -= howmany(sizeof(struct cpl_abort_req), 16); /* * Shortest possible tx work request is a fw_ofld_tx_data_wr + 1 byte * immediate payload, and firmware counts tx work request credits in * units of 16 byte. Calculate the maximum work requests possible. */ txsd_total = tx_credits / howmany(sizeof(struct fw_ofld_tx_data_wr) + 1, 16); if (txqid < 0) txqid = (arc4random() % vi->nofldtxq) + vi->first_ofld_txq; KASSERT(txqid >= vi->first_ofld_txq && txqid < vi->first_ofld_txq + vi->nofldtxq, ("%s: txqid %d for vi %p (first %d, n %d)", __func__, txqid, vi, vi->first_ofld_txq, vi->nofldtxq)); if (rxqid < 0) rxqid = (arc4random() % vi->nofldrxq) + vi->first_ofld_rxq; KASSERT(rxqid >= vi->first_ofld_rxq && rxqid < vi->first_ofld_rxq + vi->nofldrxq, ("%s: rxqid %d for vi %p (first %d, n %d)", __func__, rxqid, vi, vi->first_ofld_rxq, vi->nofldrxq)); len = offsetof(struct toepcb, txsd) + txsd_total * sizeof(struct ofld_tx_sdesc); toep = malloc(len, M_CXGBE, M_ZERO | flags); if (toep == NULL) return (NULL); refcount_init(&toep->refcount, 1); toep->td = sc->tom_softc; toep->vi = vi; toep->tx_total = tx_credits; toep->tx_credits = tx_credits; toep->ofld_txq = &sc->sge.ofld_txq[txqid]; toep->ofld_rxq = &sc->sge.ofld_rxq[rxqid]; toep->ctrlq = &sc->sge.ctrlq[pi->port_id]; mbufq_init(&toep->ulp_pduq, INT_MAX); mbufq_init(&toep->ulp_pdu_reclaimq, INT_MAX); toep->txsd_total = txsd_total; toep->txsd_avail = txsd_total; toep->txsd_pidx = 0; toep->txsd_cidx = 0; aiotx_init_toep(toep); ddp_init_toep(toep); return (toep); } struct toepcb * hold_toepcb(struct toepcb *toep) { refcount_acquire(&toep->refcount); return (toep); } void free_toepcb(struct toepcb *toep) { if (refcount_release(&toep->refcount) == 0) return; KASSERT(!(toep->flags & TPF_ATTACHED), ("%s: attached to an inpcb", __func__)); KASSERT(!(toep->flags & TPF_CPL_PENDING), ("%s: CPL pending", __func__)); ddp_uninit_toep(toep); free(toep, M_CXGBE); } /* * Set up the socket for TCP offload. */ void offload_socket(struct socket *so, struct toepcb *toep) { struct tom_data *td = toep->td; struct inpcb *inp = sotoinpcb(so); struct tcpcb *tp = intotcpcb(inp); struct sockbuf *sb; INP_WLOCK_ASSERT(inp); /* Update socket */ sb = &so->so_snd; SOCKBUF_LOCK(sb); sb->sb_flags |= SB_NOCOALESCE; SOCKBUF_UNLOCK(sb); sb = &so->so_rcv; SOCKBUF_LOCK(sb); sb->sb_flags |= SB_NOCOALESCE; if (inp->inp_vflag & INP_IPV6) so->so_proto = &toe6_protosw; else so->so_proto = &toe_protosw; SOCKBUF_UNLOCK(sb); /* Update TCP PCB */ tp->tod = &td->tod; tp->t_toe = toep; tp->t_flags |= TF_TOE; /* Install an extra hold on inp */ toep->inp = inp; toep->flags |= TPF_ATTACHED; in_pcbref(inp); /* Add the TOE PCB to the active list */ mtx_lock(&td->toep_list_lock); TAILQ_INSERT_HEAD(&td->toep_list, toep, link); mtx_unlock(&td->toep_list_lock); } /* This is _not_ the normal way to "unoffload" a socket. */ void undo_offload_socket(struct socket *so) { struct inpcb *inp = sotoinpcb(so); struct tcpcb *tp = intotcpcb(inp); struct toepcb *toep = tp->t_toe; struct tom_data *td = toep->td; struct sockbuf *sb; INP_WLOCK_ASSERT(inp); sb = &so->so_snd; SOCKBUF_LOCK(sb); sb->sb_flags &= ~SB_NOCOALESCE; SOCKBUF_UNLOCK(sb); sb = &so->so_rcv; SOCKBUF_LOCK(sb); sb->sb_flags &= ~SB_NOCOALESCE; SOCKBUF_UNLOCK(sb); tp->tod = NULL; tp->t_toe = NULL; tp->t_flags &= ~TF_TOE; toep->inp = NULL; toep->flags &= ~TPF_ATTACHED; if (in_pcbrele_wlocked(inp)) panic("%s: inp freed.", __func__); mtx_lock(&td->toep_list_lock); TAILQ_REMOVE(&td->toep_list, toep, link); mtx_unlock(&td->toep_list_lock); free_toepcb(toep); } static void release_offload_resources(struct toepcb *toep) { struct tom_data *td = toep->td; struct adapter *sc = td_adapter(td); int tid = toep->tid; KASSERT(!(toep->flags & TPF_CPL_PENDING), ("%s: %p has CPL pending.", __func__, toep)); KASSERT(!(toep->flags & TPF_ATTACHED), ("%s: %p is still attached.", __func__, toep)); CTR5(KTR_CXGBE, "%s: toep %p (tid %d, l2te %p, ce %p)", __func__, toep, tid, toep->l2te, toep->ce); /* * These queues should have been emptied at approximately the same time * that a normal connection's socket's so_snd would have been purged or * drained. Do _not_ clean up here. */ MPASS(mbufq_len(&toep->ulp_pduq) == 0); MPASS(mbufq_len(&toep->ulp_pdu_reclaimq) == 0); #ifdef INVARIANTS ddp_assert_empty(toep); #endif if (toep->l2te) t4_l2t_release(toep->l2te); if (tid >= 0) { remove_tid(sc, tid); release_tid(sc, tid, toep->ctrlq); } if (toep->ce) release_lip(td, toep->ce); mtx_lock(&td->toep_list_lock); TAILQ_REMOVE(&td->toep_list, toep, link); mtx_unlock(&td->toep_list_lock); free_toepcb(toep); } /* * The kernel is done with the TCP PCB and this is our opportunity to unhook the * toepcb hanging off of it. If the TOE driver is also done with the toepcb (no * pending CPL) then it is time to release all resources tied to the toepcb. * * Also gets called when an offloaded active open fails and the TOM wants the * kernel to take the TCP PCB back. */ static void t4_pcb_detach(struct toedev *tod __unused, struct tcpcb *tp) { #if defined(KTR) || defined(INVARIANTS) struct inpcb *inp = tp->t_inpcb; #endif struct toepcb *toep = tp->t_toe; INP_WLOCK_ASSERT(inp); KASSERT(toep != NULL, ("%s: toep is NULL", __func__)); KASSERT(toep->flags & TPF_ATTACHED, ("%s: not attached", __func__)); #ifdef KTR if (tp->t_state == TCPS_SYN_SENT) { CTR6(KTR_CXGBE, "%s: atid %d, toep %p (0x%x), inp %p (0x%x)", __func__, toep->tid, toep, toep->flags, inp, inp->inp_flags); } else { CTR6(KTR_CXGBE, "t4_pcb_detach: tid %d (%s), toep %p (0x%x), inp %p (0x%x)", toep->tid, tcpstates[tp->t_state], toep, toep->flags, inp, inp->inp_flags); } #endif tp->t_toe = NULL; tp->t_flags &= ~TF_TOE; toep->flags &= ~TPF_ATTACHED; if (!(toep->flags & TPF_CPL_PENDING)) release_offload_resources(toep); } /* * setsockopt handler. */ static void t4_ctloutput(struct toedev *tod, struct tcpcb *tp, int dir, int name) { struct adapter *sc = tod->tod_softc; struct toepcb *toep = tp->t_toe; if (dir == SOPT_GET) return; CTR4(KTR_CXGBE, "%s: tp %p, dir %u, name %u", __func__, tp, dir, name); switch (name) { case TCP_NODELAY: t4_set_tcb_field(sc, toep->ctrlq, toep->tid, W_TCB_T_FLAGS, V_TF_NAGLE(1), V_TF_NAGLE(tp->t_flags & TF_NODELAY ? 0 : 1), 0, 0, toep->ofld_rxq->iq.abs_id); break; default: break; } } /* * The TOE driver will not receive any more CPLs for the tid associated with the * toepcb; release the hold on the inpcb. */ void final_cpl_received(struct toepcb *toep) { struct inpcb *inp = toep->inp; KASSERT(inp != NULL, ("%s: inp is NULL", __func__)); INP_WLOCK_ASSERT(inp); KASSERT(toep->flags & TPF_CPL_PENDING, ("%s: CPL not pending already?", __func__)); CTR6(KTR_CXGBE, "%s: tid %d, toep %p (0x%x), inp %p (0x%x)", __func__, toep->tid, toep, toep->flags, inp, inp->inp_flags); if (toep->ulp_mode == ULP_MODE_TCPDDP) release_ddp_resources(toep); toep->inp = NULL; toep->flags &= ~TPF_CPL_PENDING; mbufq_drain(&toep->ulp_pdu_reclaimq); if (!(toep->flags & TPF_ATTACHED)) release_offload_resources(toep); if (!in_pcbrele_wlocked(inp)) INP_WUNLOCK(inp); } void insert_tid(struct adapter *sc, int tid, void *ctx) { struct tid_info *t = &sc->tids; t->tid_tab[tid] = ctx; atomic_add_int(&t->tids_in_use, 1); } void * lookup_tid(struct adapter *sc, int tid) { struct tid_info *t = &sc->tids; return (t->tid_tab[tid]); } void update_tid(struct adapter *sc, int tid, void *ctx) { struct tid_info *t = &sc->tids; t->tid_tab[tid] = ctx; } void remove_tid(struct adapter *sc, int tid) { struct tid_info *t = &sc->tids; t->tid_tab[tid] = NULL; atomic_subtract_int(&t->tids_in_use, 1); } void release_tid(struct adapter *sc, int tid, struct sge_wrq *ctrlq) { struct wrqe *wr; struct cpl_tid_release *req; wr = alloc_wrqe(sizeof(*req), ctrlq); if (wr == NULL) { queue_tid_release(sc, tid); /* defer */ return; } req = wrtod(wr); INIT_TP_WR_MIT_CPL(req, CPL_TID_RELEASE, tid); t4_wrq_tx(sc, wr); } static void queue_tid_release(struct adapter *sc, int tid) { CXGBE_UNIMPLEMENTED("deferred tid release"); } /* * What mtu_idx to use, given a 4-tuple and/or an MSS cap */ int find_best_mtu_idx(struct adapter *sc, struct in_conninfo *inc, int pmss) { unsigned short *mtus = &sc->params.mtus[0]; int i, mss, n; KASSERT(inc != NULL || pmss > 0, ("%s: at least one of inc/pmss must be specified", __func__)); mss = inc ? tcp_mssopt(inc) : pmss; if (pmss > 0 && mss > pmss) mss = pmss; if (inc->inc_flags & INC_ISIPV6) n = sizeof(struct ip6_hdr) + sizeof(struct tcphdr); else n = sizeof(struct ip) + sizeof(struct tcphdr); for (i = 0; i < NMTUS - 1 && mtus[i + 1] <= mss + n; i++) continue; return (i); } /* * Determine the receive window size for a socket. */ u_long select_rcv_wnd(struct socket *so) { unsigned long wnd; SOCKBUF_LOCK_ASSERT(&so->so_rcv); wnd = sbspace(&so->so_rcv); if (wnd < MIN_RCV_WND) wnd = MIN_RCV_WND; return min(wnd, MAX_RCV_WND); } int select_rcv_wscale(void) { int wscale = 0; unsigned long space = sb_max; if (space > MAX_RCV_WND) space = MAX_RCV_WND; while (wscale < TCP_MAX_WINSHIFT && (TCP_MAXWIN << wscale) < space) wscale++; return (wscale); } extern int always_keepalive; #define VIID_SMACIDX(v) (((unsigned int)(v) & 0x7f) << 1) /* * socket so could be a listening socket too. */ uint64_t calc_opt0(struct socket *so, struct vi_info *vi, struct l2t_entry *e, int mtu_idx, int rscale, int rx_credits, int ulp_mode) { uint64_t opt0; KASSERT(rx_credits <= M_RCV_BUFSIZ, ("%s: rcv_bufsiz too high", __func__)); opt0 = F_TCAM_BYPASS | V_WND_SCALE(rscale) | V_MSS_IDX(mtu_idx) | V_ULP_MODE(ulp_mode) | V_RCV_BUFSIZ(rx_credits); if (so != NULL) { struct inpcb *inp = sotoinpcb(so); struct tcpcb *tp = intotcpcb(inp); int keepalive = always_keepalive || so_options_get(so) & SO_KEEPALIVE; opt0 |= V_NAGLE((tp->t_flags & TF_NODELAY) == 0); opt0 |= V_KEEP_ALIVE(keepalive != 0); } if (e != NULL) opt0 |= V_L2T_IDX(e->idx); if (vi != NULL) { opt0 |= V_SMAC_SEL(VIID_SMACIDX(vi->viid)); opt0 |= V_TX_CHAN(vi->pi->tx_chan); } return htobe64(opt0); } uint64_t select_ntuple(struct vi_info *vi, struct l2t_entry *e) { struct adapter *sc = vi->pi->adapter; struct tp_params *tp = &sc->params.tp; uint16_t viid = vi->viid; uint64_t ntuple = 0; /* * Initialize each of the fields which we care about which are present * in the Compressed Filter Tuple. */ if (tp->vlan_shift >= 0 && e->vlan != CPL_L2T_VLAN_NONE) ntuple |= (uint64_t)(F_FT_VLAN_VLD | e->vlan) << tp->vlan_shift; if (tp->port_shift >= 0) ntuple |= (uint64_t)e->lport << tp->port_shift; if (tp->protocol_shift >= 0) ntuple |= (uint64_t)IPPROTO_TCP << tp->protocol_shift; if (tp->vnic_shift >= 0) { uint32_t vf = G_FW_VIID_VIN(viid); uint32_t pf = G_FW_VIID_PFN(viid); uint32_t vld = G_FW_VIID_VIVLD(viid); ntuple |= (uint64_t)(V_FT_VNID_ID_VF(vf) | V_FT_VNID_ID_PF(pf) | V_FT_VNID_ID_VLD(vld)) << tp->vnic_shift; } if (is_t4(sc)) return (htobe32((uint32_t)ntuple)); else return (htobe64(V_FILTER_TUPLE(ntuple))); } void set_tcpddp_ulp_mode(struct toepcb *toep) { toep->ulp_mode = ULP_MODE_TCPDDP; toep->ddp_flags = DDP_OK; } int negative_advice(int status) { return (status == CPL_ERR_RTX_NEG_ADVICE || status == CPL_ERR_PERSIST_NEG_ADVICE || status == CPL_ERR_KEEPALV_NEG_ADVICE); } static int alloc_tid_tabs(struct tid_info *t) { size_t size; unsigned int i; size = t->ntids * sizeof(*t->tid_tab) + t->natids * sizeof(*t->atid_tab) + t->nstids * sizeof(*t->stid_tab); t->tid_tab = malloc(size, M_CXGBE, M_ZERO | M_NOWAIT); if (t->tid_tab == NULL) return (ENOMEM); mtx_init(&t->atid_lock, "atid lock", NULL, MTX_DEF); t->atid_tab = (union aopen_entry *)&t->tid_tab[t->ntids]; t->afree = t->atid_tab; t->atids_in_use = 0; for (i = 1; i < t->natids; i++) t->atid_tab[i - 1].next = &t->atid_tab[i]; t->atid_tab[t->natids - 1].next = NULL; mtx_init(&t->stid_lock, "stid lock", NULL, MTX_DEF); t->stid_tab = (struct listen_ctx **)&t->atid_tab[t->natids]; t->stids_in_use = 0; TAILQ_INIT(&t->stids); t->nstids_free_head = t->nstids; atomic_store_rel_int(&t->tids_in_use, 0); return (0); } static void free_tid_tabs(struct tid_info *t) { KASSERT(t->tids_in_use == 0, ("%s: %d tids still in use.", __func__, t->tids_in_use)); KASSERT(t->atids_in_use == 0, ("%s: %d atids still in use.", __func__, t->atids_in_use)); KASSERT(t->stids_in_use == 0, ("%s: %d tids still in use.", __func__, t->stids_in_use)); free(t->tid_tab, M_CXGBE); t->tid_tab = NULL; if (mtx_initialized(&t->atid_lock)) mtx_destroy(&t->atid_lock); if (mtx_initialized(&t->stid_lock)) mtx_destroy(&t->stid_lock); } static int add_lip(struct adapter *sc, struct in6_addr *lip) { struct fw_clip_cmd c; ASSERT_SYNCHRONIZED_OP(sc); /* mtx_assert(&td->clip_table_lock, MA_OWNED); */ memset(&c, 0, sizeof(c)); c.op_to_write = htonl(V_FW_CMD_OP(FW_CLIP_CMD) | F_FW_CMD_REQUEST | F_FW_CMD_WRITE); c.alloc_to_len16 = htonl(F_FW_CLIP_CMD_ALLOC | FW_LEN16(c)); c.ip_hi = *(uint64_t *)&lip->s6_addr[0]; c.ip_lo = *(uint64_t *)&lip->s6_addr[8]; return (-t4_wr_mbox_ns(sc, sc->mbox, &c, sizeof(c), &c)); } static int delete_lip(struct adapter *sc, struct in6_addr *lip) { struct fw_clip_cmd c; ASSERT_SYNCHRONIZED_OP(sc); /* mtx_assert(&td->clip_table_lock, MA_OWNED); */ memset(&c, 0, sizeof(c)); c.op_to_write = htonl(V_FW_CMD_OP(FW_CLIP_CMD) | F_FW_CMD_REQUEST | F_FW_CMD_READ); c.alloc_to_len16 = htonl(F_FW_CLIP_CMD_FREE | FW_LEN16(c)); c.ip_hi = *(uint64_t *)&lip->s6_addr[0]; c.ip_lo = *(uint64_t *)&lip->s6_addr[8]; return (-t4_wr_mbox_ns(sc, sc->mbox, &c, sizeof(c), &c)); } static struct clip_entry * search_lip(struct tom_data *td, struct in6_addr *lip) { struct clip_entry *ce; mtx_assert(&td->clip_table_lock, MA_OWNED); TAILQ_FOREACH(ce, &td->clip_table, link) { if (IN6_ARE_ADDR_EQUAL(&ce->lip, lip)) return (ce); } return (NULL); } struct clip_entry * hold_lip(struct tom_data *td, struct in6_addr *lip) { struct clip_entry *ce; mtx_lock(&td->clip_table_lock); ce = search_lip(td, lip); if (ce != NULL) ce->refcount++; mtx_unlock(&td->clip_table_lock); return (ce); } void release_lip(struct tom_data *td, struct clip_entry *ce) { mtx_lock(&td->clip_table_lock); KASSERT(search_lip(td, &ce->lip) == ce, ("%s: CLIP entry %p p not in CLIP table.", __func__, ce)); KASSERT(ce->refcount > 0, ("%s: CLIP entry %p has refcount 0", __func__, ce)); --ce->refcount; mtx_unlock(&td->clip_table_lock); } static void init_clip_table(struct adapter *sc, struct tom_data *td) { ASSERT_SYNCHRONIZED_OP(sc); mtx_init(&td->clip_table_lock, "CLIP table lock", NULL, MTX_DEF); TAILQ_INIT(&td->clip_table); td->clip_gen = -1; update_clip_table(sc, td); } static void update_clip(struct adapter *sc, void *arg __unused) { if (begin_synchronized_op(sc, NULL, HOLD_LOCK, "t4tomuc")) return; if (uld_active(sc, ULD_TOM)) update_clip_table(sc, sc->tom_softc); end_synchronized_op(sc, LOCK_HELD); } static void t4_clip_task(void *arg, int count) { t4_iterate(update_clip, NULL); } static void update_clip_table(struct adapter *sc, struct tom_data *td) { struct rm_priotracker in6_ifa_tracker; struct in6_ifaddr *ia; struct in6_addr *lip, tlip; struct clip_head stale; struct clip_entry *ce, *ce_temp; int rc, gen = atomic_load_acq_int(&in6_ifaddr_gen); ASSERT_SYNCHRONIZED_OP(sc); IN6_IFADDR_RLOCK(&in6_ifa_tracker); mtx_lock(&td->clip_table_lock); if (gen == td->clip_gen) goto done; TAILQ_INIT(&stale); TAILQ_CONCAT(&stale, &td->clip_table, link); TAILQ_FOREACH(ia, &V_in6_ifaddrhead, ia_link) { lip = &ia->ia_addr.sin6_addr; KASSERT(!IN6_IS_ADDR_MULTICAST(lip), ("%s: mcast address in in6_ifaddr list", __func__)); if (IN6_IS_ADDR_LOOPBACK(lip)) continue; if (IN6_IS_SCOPE_EMBED(lip)) { /* Remove the embedded scope */ tlip = *lip; lip = &tlip; in6_clearscope(lip); } /* * XXX: how to weed out the link local address for the loopback * interface? It's fe80::1 usually (always?). */ /* * If it's in the main list then we already know it's not stale. */ TAILQ_FOREACH(ce, &td->clip_table, link) { if (IN6_ARE_ADDR_EQUAL(&ce->lip, lip)) goto next; } /* * If it's in the stale list we should move it to the main list. */ TAILQ_FOREACH(ce, &stale, link) { if (IN6_ARE_ADDR_EQUAL(&ce->lip, lip)) { TAILQ_REMOVE(&stale, ce, link); TAILQ_INSERT_TAIL(&td->clip_table, ce, link); goto next; } } /* A new IP6 address; add it to the CLIP table */ ce = malloc(sizeof(*ce), M_CXGBE, M_NOWAIT); memcpy(&ce->lip, lip, sizeof(ce->lip)); ce->refcount = 0; rc = add_lip(sc, lip); if (rc == 0) TAILQ_INSERT_TAIL(&td->clip_table, ce, link); else { char ip[INET6_ADDRSTRLEN]; inet_ntop(AF_INET6, &ce->lip, &ip[0], sizeof(ip)); log(LOG_ERR, "%s: could not add %s (%d)\n", __func__, ip, rc); free(ce, M_CXGBE); } next: continue; } /* * Remove stale addresses (those no longer in V_in6_ifaddrhead) that are * no longer referenced by the driver. */ TAILQ_FOREACH_SAFE(ce, &stale, link, ce_temp) { if (ce->refcount == 0) { rc = delete_lip(sc, &ce->lip); if (rc == 0) { TAILQ_REMOVE(&stale, ce, link); free(ce, M_CXGBE); } else { char ip[INET6_ADDRSTRLEN]; inet_ntop(AF_INET6, &ce->lip, &ip[0], sizeof(ip)); log(LOG_ERR, "%s: could not delete %s (%d)\n", __func__, ip, rc); } } } /* The ones that are still referenced need to stay in the CLIP table */ TAILQ_CONCAT(&td->clip_table, &stale, link); td->clip_gen = gen; done: mtx_unlock(&td->clip_table_lock); IN6_IFADDR_RUNLOCK(&in6_ifa_tracker); } static void destroy_clip_table(struct adapter *sc, struct tom_data *td) { struct clip_entry *ce, *ce_temp; if (mtx_initialized(&td->clip_table_lock)) { mtx_lock(&td->clip_table_lock); TAILQ_FOREACH_SAFE(ce, &td->clip_table, link, ce_temp) { KASSERT(ce->refcount == 0, ("%s: CLIP entry %p still in use (%d)", __func__, ce, ce->refcount)); TAILQ_REMOVE(&td->clip_table, ce, link); delete_lip(sc, &ce->lip); free(ce, M_CXGBE); } mtx_unlock(&td->clip_table_lock); mtx_destroy(&td->clip_table_lock); } } static void free_tom_data(struct adapter *sc, struct tom_data *td) { ASSERT_SYNCHRONIZED_OP(sc); KASSERT(TAILQ_EMPTY(&td->toep_list), ("%s: TOE PCB list is not empty.", __func__)); KASSERT(td->lctx_count == 0, ("%s: lctx hash table is not empty.", __func__)); - t4_uninit_ddp(sc, td); + t4_free_ppod_region(&td->pr); destroy_clip_table(sc, td); if (td->listen_mask != 0) hashdestroy(td->listen_hash, M_CXGBE, td->listen_mask); if (mtx_initialized(&td->unsent_wr_lock)) mtx_destroy(&td->unsent_wr_lock); if (mtx_initialized(&td->lctx_hash_lock)) mtx_destroy(&td->lctx_hash_lock); if (mtx_initialized(&td->toep_list_lock)) mtx_destroy(&td->toep_list_lock); free_tid_tabs(&sc->tids); free(td, M_CXGBE); } static void reclaim_wr_resources(void *arg, int count) { struct tom_data *td = arg; STAILQ_HEAD(, wrqe) twr_list = STAILQ_HEAD_INITIALIZER(twr_list); struct cpl_act_open_req *cpl; u_int opcode, atid; struct wrqe *wr; struct adapter *sc; mtx_lock(&td->unsent_wr_lock); STAILQ_SWAP(&td->unsent_wr_list, &twr_list, wrqe); mtx_unlock(&td->unsent_wr_lock); while ((wr = STAILQ_FIRST(&twr_list)) != NULL) { STAILQ_REMOVE_HEAD(&twr_list, link); cpl = wrtod(wr); opcode = GET_OPCODE(cpl); switch (opcode) { case CPL_ACT_OPEN_REQ: case CPL_ACT_OPEN_REQ6: atid = G_TID_TID(be32toh(OPCODE_TID(cpl))); sc = td_adapter(td); CTR2(KTR_CXGBE, "%s: atid %u ", __func__, atid); act_open_failure_cleanup(sc, atid, EHOSTUNREACH); free(wr, M_CXGBE); break; default: log(LOG_ERR, "%s: leaked work request %p, wr_len %d, " "opcode %x\n", __func__, wr, wr->wr_len, opcode); /* WR not freed here; go look at it with a debugger. */ } } } /* * Ground control to Major TOM * Commencing countdown, engines on */ static int t4_tom_activate(struct adapter *sc) { struct tom_data *td; struct toedev *tod; struct vi_info *vi; struct sge_ofld_rxq *ofld_rxq; int i, j, rc, v; ASSERT_SYNCHRONIZED_OP(sc); /* per-adapter softc for TOM */ td = malloc(sizeof(*td), M_CXGBE, M_ZERO | M_NOWAIT); if (td == NULL) return (ENOMEM); /* List of TOE PCBs and associated lock */ mtx_init(&td->toep_list_lock, "PCB list lock", NULL, MTX_DEF); TAILQ_INIT(&td->toep_list); /* Listen context */ mtx_init(&td->lctx_hash_lock, "lctx hash lock", NULL, MTX_DEF); td->listen_hash = hashinit_flags(LISTEN_HASH_SIZE, M_CXGBE, &td->listen_mask, HASH_NOWAIT); /* List of WRs for which L2 resolution failed */ mtx_init(&td->unsent_wr_lock, "Unsent WR list lock", NULL, MTX_DEF); STAILQ_INIT(&td->unsent_wr_list); TASK_INIT(&td->reclaim_wr_resources, 0, reclaim_wr_resources, td); /* TID tables */ rc = alloc_tid_tabs(&sc->tids); if (rc != 0) goto done; - /* DDP page pods and CPL handlers */ - t4_init_ddp(sc, td); + rc = t4_init_ppod_region(&td->pr, &sc->vres.ddp, + t4_read_reg(sc, A_ULP_RX_TDDP_PSZ), "TDDP page pods"); + if (rc != 0) + goto done; + t4_set_reg_field(sc, A_ULP_RX_TDDP_TAGMASK, + V_TDDPTAGMASK(M_TDDPTAGMASK), td->pr.pr_tag_mask); /* CLIP table for IPv6 offload */ init_clip_table(sc, td); /* toedev ops */ tod = &td->tod; init_toedev(tod); tod->tod_softc = sc; tod->tod_connect = t4_connect; tod->tod_listen_start = t4_listen_start; tod->tod_listen_stop = t4_listen_stop; tod->tod_rcvd = t4_rcvd; tod->tod_output = t4_tod_output; tod->tod_send_rst = t4_send_rst; tod->tod_send_fin = t4_send_fin; tod->tod_pcb_detach = t4_pcb_detach; tod->tod_l2_update = t4_l2_update; tod->tod_syncache_added = t4_syncache_added; tod->tod_syncache_removed = t4_syncache_removed; tod->tod_syncache_respond = t4_syncache_respond; tod->tod_offload_socket = t4_offload_socket; tod->tod_ctloutput = t4_ctloutput; for_each_port(sc, i) { for_each_vi(sc->port[i], v, vi) { TOEDEV(vi->ifp) = &td->tod; for_each_ofld_rxq(vi, j, ofld_rxq) { ofld_rxq->iq.set_tcb_rpl = do_set_tcb_rpl; ofld_rxq->iq.l2t_write_rpl = do_l2t_write_rpl2; } } } sc->tom_softc = td; register_toedev(sc->tom_softc); done: if (rc != 0) free_tom_data(sc, td); return (rc); } static int t4_tom_deactivate(struct adapter *sc) { int rc = 0; struct tom_data *td = sc->tom_softc; ASSERT_SYNCHRONIZED_OP(sc); if (td == NULL) return (0); /* XXX. KASSERT? */ if (sc->offload_map != 0) return (EBUSY); /* at least one port has IFCAP_TOE enabled */ if (uld_active(sc, ULD_IWARP) || uld_active(sc, ULD_ISCSI)) return (EBUSY); /* both iWARP and iSCSI rely on the TOE. */ mtx_lock(&td->toep_list_lock); if (!TAILQ_EMPTY(&td->toep_list)) rc = EBUSY; mtx_unlock(&td->toep_list_lock); mtx_lock(&td->lctx_hash_lock); if (td->lctx_count > 0) rc = EBUSY; mtx_unlock(&td->lctx_hash_lock); taskqueue_drain(taskqueue_thread, &td->reclaim_wr_resources); mtx_lock(&td->unsent_wr_lock); if (!STAILQ_EMPTY(&td->unsent_wr_list)) rc = EBUSY; mtx_unlock(&td->unsent_wr_lock); if (rc == 0) { unregister_toedev(sc->tom_softc); free_tom_data(sc, td); sc->tom_softc = NULL; } return (rc); } static void t4_tom_ifaddr_event(void *arg __unused, struct ifnet *ifp) { atomic_add_rel_int(&in6_ifaddr_gen, 1); taskqueue_enqueue_timeout(taskqueue_thread, &clip_task, -hz / 4); } static int t4_aio_queue_tom(struct socket *so, struct kaiocb *job) { struct tcpcb *tp = so_sototcpcb(so); struct toepcb *toep = tp->t_toe; int error; if (toep->ulp_mode == ULP_MODE_TCPDDP) { error = t4_aio_queue_ddp(so, job); if (error != EOPNOTSUPP) return (error); } return (t4_aio_queue_aiotx(so, job)); } static int t4_tom_mod_load(void) { int rc; struct protosw *tcp_protosw, *tcp6_protosw; /* CPL handlers */ t4_init_connect_cpl_handlers(); t4_init_listen_cpl_handlers(); t4_init_cpl_io_handlers(); rc = t4_ddp_mod_load(); if (rc != 0) return (rc); tcp_protosw = pffindproto(PF_INET, IPPROTO_TCP, SOCK_STREAM); if (tcp_protosw == NULL) return (ENOPROTOOPT); bcopy(tcp_protosw, &toe_protosw, sizeof(toe_protosw)); bcopy(tcp_protosw->pr_usrreqs, &toe_usrreqs, sizeof(toe_usrreqs)); toe_usrreqs.pru_aio_queue = t4_aio_queue_tom; toe_protosw.pr_usrreqs = &toe_usrreqs; tcp6_protosw = pffindproto(PF_INET6, IPPROTO_TCP, SOCK_STREAM); if (tcp6_protosw == NULL) return (ENOPROTOOPT); bcopy(tcp6_protosw, &toe6_protosw, sizeof(toe6_protosw)); bcopy(tcp6_protosw->pr_usrreqs, &toe6_usrreqs, sizeof(toe6_usrreqs)); toe6_usrreqs.pru_aio_queue = t4_aio_queue_tom; toe6_protosw.pr_usrreqs = &toe6_usrreqs; TIMEOUT_TASK_INIT(taskqueue_thread, &clip_task, 0, t4_clip_task, NULL); ifaddr_evhandler = EVENTHANDLER_REGISTER(ifaddr_event, t4_tom_ifaddr_event, NULL, EVENTHANDLER_PRI_ANY); rc = t4_register_uld(&tom_uld_info); if (rc != 0) t4_tom_mod_unload(); return (rc); } static void tom_uninit(struct adapter *sc, void *arg __unused) { if (begin_synchronized_op(sc, NULL, SLEEP_OK | INTR_OK, "t4tomun")) return; /* Try to free resources (works only if no port has IFCAP_TOE) */ if (uld_active(sc, ULD_TOM)) t4_deactivate_uld(sc, ULD_TOM); end_synchronized_op(sc, 0); } static int t4_tom_mod_unload(void) { t4_iterate(tom_uninit, NULL); if (t4_unregister_uld(&tom_uld_info) == EBUSY) return (EBUSY); if (ifaddr_evhandler) { EVENTHANDLER_DEREGISTER(ifaddr_event, ifaddr_evhandler); taskqueue_cancel_timeout(taskqueue_thread, &clip_task, NULL); } t4_ddp_mod_unload(); return (0); } #endif /* TCP_OFFLOAD */ static int t4_tom_modevent(module_t mod, int cmd, void *arg) { int rc = 0; #ifdef TCP_OFFLOAD switch (cmd) { case MOD_LOAD: rc = t4_tom_mod_load(); break; case MOD_UNLOAD: rc = t4_tom_mod_unload(); break; default: rc = EINVAL; } #else printf("t4_tom: compiled without TCP_OFFLOAD support.\n"); rc = EOPNOTSUPP; #endif return (rc); } static moduledata_t t4_tom_moddata= { "t4_tom", t4_tom_modevent, 0 }; MODULE_VERSION(t4_tom, 1); MODULE_DEPEND(t4_tom, toecore, 1, 1, 1); MODULE_DEPEND(t4_tom, t4nex, 1, 1, 1); DECLARE_MODULE(t4_tom, t4_tom_moddata, SI_SUB_EXEC, SI_ORDER_ANY); Index: projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_tom.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_tom.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/cxgbe/tom/t4_tom.h (revision 305172) @@ -1,366 +1,391 @@ /*- * Copyright (c) 2012, 2015 Chelsio Communications, Inc. * All rights reserved. * Written by: Navdeep Parhar * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ * */ #ifndef __T4_TOM_H__ #define __T4_TOM_H__ #include #define LISTEN_HASH_SIZE 32 /* * Min receive window. We want it to be large enough to accommodate receive * coalescing, handle jumbo frames, and not trigger sender SWS avoidance. */ #define MIN_RCV_WND (24 * 1024U) /* * Max receive window supported by HW in bytes. Only a small part of it can * be set through option0, the rest needs to be set through RX_DATA_ACK. */ #define MAX_RCV_WND ((1U << 27) - 1) #define DDP_RSVD_WIN (16 * 1024U) #define SB_DDP_INDICATE SB_IN_TOE /* soreceive must respond to indicate */ #define USE_DDP_RX_FLOW_CONTROL +#define PPOD_SZ(n) ((n) * sizeof(struct pagepod)) +#define PPOD_SIZE (PPOD_SZ(1)) + /* TOE PCB flags */ enum { TPF_ATTACHED = (1 << 0), /* a tcpcb refers to this toepcb */ TPF_FLOWC_WR_SENT = (1 << 1), /* firmware flow context WR sent */ TPF_TX_DATA_SENT = (1 << 2), /* some data sent */ TPF_TX_SUSPENDED = (1 << 3), /* tx suspended for lack of resources */ TPF_SEND_FIN = (1 << 4), /* send FIN after all pending data */ TPF_FIN_SENT = (1 << 5), /* FIN has been sent */ TPF_ABORT_SHUTDOWN = (1 << 6), /* connection abort is in progress */ TPF_CPL_PENDING = (1 << 7), /* haven't received the last CPL */ TPF_SYNQE = (1 << 8), /* synq_entry, not really a toepcb */ TPF_SYNQE_NEEDFREE = (1 << 9), /* synq_entry was malloc'd separately */ TPF_SYNQE_TCPDDP = (1 << 10), /* ulp_mode TCPDDP in toepcb */ TPF_SYNQE_EXPANDED = (1 << 11), /* toepcb ready, tid context updated */ TPF_SYNQE_HAS_L2TE = (1 << 12), /* we've replied to PASS_ACCEPT_REQ */ }; enum { DDP_OK = (1 << 0), /* OK to turn on DDP */ DDP_SC_REQ = (1 << 1), /* state change (on/off) requested */ DDP_ON = (1 << 2), /* DDP is turned on */ DDP_BUF0_ACTIVE = (1 << 3), /* buffer 0 in use (not invalidated) */ DDP_BUF1_ACTIVE = (1 << 4), /* buffer 1 in use (not invalidated) */ DDP_TASK_ACTIVE = (1 << 5), /* requeue task is queued / running */ DDP_DEAD = (1 << 6), /* toepcb is shutting down */ }; struct ofld_tx_sdesc { uint32_t plen; /* payload length */ uint8_t tx_credits; /* firmware tx credits (unit is 16B) */ }; +struct ppod_region { + u_int pr_start; + u_int pr_len; + u_int pr_page_shift[4]; + uint32_t pr_tag_mask; /* hardware tagmask for this region. */ + uint32_t pr_invalid_bit; /* OR with this to invalidate tag. */ + uint32_t pr_alias_mask; /* AND with tag to get alias bits. */ + u_int pr_alias_shift; /* shift this much for first alias bit. */ + vmem_t *pr_arena; +}; + +struct ppod_reservation { + struct ppod_region *prsv_pr; + uint32_t prsv_tag; /* Full tag: pgsz, alias, tag, color */ + u_int prsv_nppods; +}; + struct pageset { TAILQ_ENTRY(pageset) link; vm_page_t *pages; int npages; int flags; - u_int ppod_addr; - int nppods; - uint32_t tag; /* includes color, page pod addr, and DDP page size */ int offset; /* offset in first page */ int len; + struct ppod_reservation prsv; struct vmspace *vm; u_int vm_timestamp; }; TAILQ_HEAD(pagesetq, pageset); #define PS_WIRED 0x0001 /* Pages wired rather than held. */ #define PS_PPODS_WRITTEN 0x0002 /* Page pods written to the card. */ #define EXT_FLAG_AIOTX EXT_FLAG_VENDOR1 struct ddp_buffer { struct pageset *ps; struct kaiocb *job; int cancel_pending; }; struct aiotx_buffer { struct pageset ps; struct kaiocb *job; int refcount; }; struct toepcb { TAILQ_ENTRY(toepcb) link; /* toep_list */ u_int flags; /* miscellaneous flags */ int refcount; struct tom_data *td; struct inpcb *inp; /* backpointer to host stack's PCB */ struct vi_info *vi; /* virtual interface */ struct sge_wrq *ofld_txq; struct sge_ofld_rxq *ofld_rxq; struct sge_wrq *ctrlq; struct l2t_entry *l2te; /* L2 table entry used by this connection */ struct clip_entry *ce; /* CLIP table entry used by this tid */ int tid; /* Connection identifier */ /* tx credit handling */ u_int tx_total; /* total tx WR credits (in 16B units) */ u_int tx_credits; /* tx WR credits (in 16B units) available */ u_int tx_nocompl; /* tx WR credits since last compl request */ u_int plen_nocompl; /* payload since last compl request */ /* rx credit handling */ u_int sb_cc; /* last noted value of so_rcv->sb_cc */ int rx_credits; /* rx credits (in bytes) to be returned to hw */ u_int ulp_mode; /* ULP mode */ void *ulpcb; void *ulpcb2; struct mbufq ulp_pduq; /* PDUs waiting to be sent out. */ struct mbufq ulp_pdu_reclaimq; u_int ddp_flags; struct ddp_buffer db[2]; TAILQ_HEAD(, pageset) ddp_cached_pagesets; TAILQ_HEAD(, kaiocb) ddp_aiojobq; u_int ddp_waiting_count; u_int ddp_active_count; u_int ddp_cached_count; int ddp_active_id; /* the currently active DDP buffer */ struct task ddp_requeue_task; struct kaiocb *ddp_queueing; struct mtx ddp_lock; TAILQ_HEAD(, kaiocb) aiotx_jobq; struct task aiotx_task; bool aiotx_task_active; /* Tx software descriptor */ uint8_t txsd_total; uint8_t txsd_pidx; uint8_t txsd_cidx; uint8_t txsd_avail; struct ofld_tx_sdesc txsd[]; }; #define DDP_LOCK(toep) mtx_lock(&(toep)->ddp_lock) #define DDP_UNLOCK(toep) mtx_unlock(&(toep)->ddp_lock) #define DDP_ASSERT_LOCKED(toep) mtx_assert(&(toep)->ddp_lock, MA_OWNED) struct flowc_tx_params { uint32_t snd_nxt; uint32_t rcv_nxt; unsigned int snd_space; unsigned int mss; }; #define DDP_RETRY_WAIT 5 /* seconds to wait before re-enabling DDP */ #define DDP_LOW_SCORE 1 #define DDP_HIGH_SCORE 3 /* * Compressed state for embryonic connections for a listener. Barely fits in * 64B, try not to grow it further. */ struct synq_entry { TAILQ_ENTRY(synq_entry) link; /* listen_ctx's synq link */ int flags; /* same as toepcb's tp_flags */ int tid; struct listen_ctx *lctx; /* backpointer to listen ctx */ struct mbuf *syn; uint32_t iss; uint32_t ts; volatile uintptr_t wr; volatile u_int refcnt; uint16_t l2e_idx; uint16_t rcv_bufsize; }; /* listen_ctx flags */ #define LCTX_RPL_PENDING 1 /* waiting for a CPL_PASS_OPEN_RPL */ struct listen_ctx { LIST_ENTRY(listen_ctx) link; /* listen hash linkage */ volatile int refcount; int stid; struct stid_region stid_region; int flags; struct inpcb *inp; /* listening socket's inp */ struct sge_wrq *ctrlq; struct sge_ofld_rxq *ofld_rxq; struct clip_entry *ce; TAILQ_HEAD(, synq_entry) synq; }; struct clip_entry { TAILQ_ENTRY(clip_entry) link; struct in6_addr lip; /* local IPv6 address */ u_int refcount; }; TAILQ_HEAD(clip_head, clip_entry); struct tom_data { struct toedev tod; /* toepcb's associated with this TOE device */ struct mtx toep_list_lock; TAILQ_HEAD(, toepcb) toep_list; struct mtx lctx_hash_lock; LIST_HEAD(, listen_ctx) *listen_hash; u_long listen_mask; int lctx_count; /* # of lctx in the hash table */ - u_int ppod_start; - u_int ddp_pgsz[4]; - vmem_t *ppod_arena; + struct ppod_region pr; struct mtx clip_table_lock; struct clip_head clip_table; int clip_gen; /* WRs that will not be sent to the chip because L2 resolution failed */ struct mtx unsent_wr_lock; STAILQ_HEAD(, wrqe) unsent_wr_list; struct task reclaim_wr_resources; }; static inline struct tom_data * tod_td(struct toedev *tod) { return (__containerof(tod, struct tom_data, tod)); } static inline struct adapter * td_adapter(struct tom_data *td) { return (td->tod.tod_softc); } static inline void set_mbuf_ulp_submode(struct mbuf *m, uint8_t ulp_submode) { M_ASSERTPKTHDR(m); m->m_pkthdr.PH_per.eight[0] = ulp_submode; } static inline uint8_t mbuf_ulp_submode(struct mbuf *m) { M_ASSERTPKTHDR(m); return (m->m_pkthdr.PH_per.eight[0]); } /* t4_tom.c */ struct toepcb *alloc_toepcb(struct vi_info *, int, int, int); struct toepcb *hold_toepcb(struct toepcb *); void free_toepcb(struct toepcb *); void offload_socket(struct socket *, struct toepcb *); void undo_offload_socket(struct socket *); void final_cpl_received(struct toepcb *); void insert_tid(struct adapter *, int, void *); void *lookup_tid(struct adapter *, int); void update_tid(struct adapter *, int, void *); void remove_tid(struct adapter *, int); void release_tid(struct adapter *, int, struct sge_wrq *); int find_best_mtu_idx(struct adapter *, struct in_conninfo *, int); u_long select_rcv_wnd(struct socket *); int select_rcv_wscale(void); uint64_t calc_opt0(struct socket *, struct vi_info *, struct l2t_entry *, int, int, int, int); uint64_t select_ntuple(struct vi_info *, struct l2t_entry *); void set_tcpddp_ulp_mode(struct toepcb *); int negative_advice(int); struct clip_entry *hold_lip(struct tom_data *, struct in6_addr *); void release_lip(struct tom_data *, struct clip_entry *); /* t4_connect.c */ void t4_init_connect_cpl_handlers(void); int t4_connect(struct toedev *, struct socket *, struct rtentry *, struct sockaddr *); void act_open_failure_cleanup(struct adapter *, u_int, u_int); /* t4_listen.c */ void t4_init_listen_cpl_handlers(void); int t4_listen_start(struct toedev *, struct tcpcb *); int t4_listen_stop(struct toedev *, struct tcpcb *); void t4_syncache_added(struct toedev *, void *); void t4_syncache_removed(struct toedev *, void *); int t4_syncache_respond(struct toedev *, void *, struct mbuf *); int do_abort_req_synqe(struct sge_iq *, const struct rss_header *, struct mbuf *); int do_abort_rpl_synqe(struct sge_iq *, const struct rss_header *, struct mbuf *); void t4_offload_socket(struct toedev *, void *, struct socket *); /* t4_cpl_io.c */ void aiotx_init_toep(struct toepcb *); int t4_aio_queue_aiotx(struct socket *, struct kaiocb *); void t4_init_cpl_io_handlers(void); void t4_uninit_cpl_io_handlers(void); void send_abort_rpl(struct adapter *, struct sge_wrq *, int , int); void send_flowc_wr(struct toepcb *, struct flowc_tx_params *); void send_reset(struct adapter *, struct toepcb *, uint32_t); void make_established(struct toepcb *, uint32_t, uint32_t, uint16_t); void t4_rcvd(struct toedev *, struct tcpcb *); void t4_rcvd_locked(struct toedev *, struct tcpcb *); int t4_tod_output(struct toedev *, struct tcpcb *); int t4_send_fin(struct toedev *, struct tcpcb *); int t4_send_rst(struct toedev *, struct tcpcb *); void t4_set_tcb_field(struct adapter *, struct sge_wrq *, int, uint16_t, uint64_t, uint64_t, int, int, int); void t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop); void t4_push_pdus(struct adapter *sc, struct toepcb *toep, int drop); int do_set_tcb_rpl(struct sge_iq *, const struct rss_header *, struct mbuf *); /* t4_ddp.c */ -void t4_init_ddp(struct adapter *, struct tom_data *); -void t4_uninit_ddp(struct adapter *, struct tom_data *); +int t4_init_ppod_region(struct ppod_region *, struct t4_range *, u_int, + const char *); +void t4_free_ppod_region(struct ppod_region *); +int t4_alloc_page_pods_for_ps(struct ppod_region *, struct pageset *); +int t4_alloc_page_pods_for_buf(struct ppod_region *, vm_offset_t, int, + struct ppod_reservation *); +int t4_write_page_pods_for_ps(struct adapter *, struct sge_wrq *, int, + struct pageset *); +int t4_write_page_pods_for_buf(struct adapter *, struct sge_wrq *, int tid, + struct ppod_reservation *, vm_offset_t, int); +void t4_free_page_pods(struct ppod_reservation *); int t4_soreceive_ddp(struct socket *, struct sockaddr **, struct uio *, struct mbuf **, struct mbuf **, int *); int t4_aio_queue_ddp(struct socket *, struct kaiocb *); int t4_ddp_mod_load(void); void t4_ddp_mod_unload(void); void ddp_assert_empty(struct toepcb *); void ddp_init_toep(struct toepcb *); void ddp_uninit_toep(struct toepcb *); void ddp_queue_toep(struct toepcb *); void release_ddp_resources(struct toepcb *toep); void handle_ddp_close(struct toepcb *, struct tcpcb *, uint32_t); void handle_ddp_indicate(struct toepcb *); void handle_ddp_tcb_rpl(struct toepcb *, const struct cpl_set_tcb_rpl *); void insert_ddp_data(struct toepcb *, uint32_t); #endif Index: projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_net_vsc.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_net_vsc.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_net_vsc.c (revision 305172) @@ -1,900 +1,892 @@ /*- * Copyright (c) 2009-2012,2016 Microsoft Corp. * Copyright (c) 2010-2012 Citrix Inc. * Copyright (c) 2012 NetApp Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ /** * HyperV vmbus network VSC (virtual services client) module * */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include MALLOC_DEFINE(M_NETVSC, "netvsc", "Hyper-V netvsc driver"); /* * Forward declarations */ static void hv_nv_on_channel_callback(struct vmbus_channel *chan, void *xrxr); static int hv_nv_init_send_buffer_with_net_vsp(struct hn_softc *sc); static int hv_nv_init_rx_buffer_with_net_vsp(struct hn_softc *, int); static int hv_nv_destroy_send_buffer(struct hn_softc *sc); static int hv_nv_destroy_rx_buffer(struct hn_softc *sc); static int hv_nv_connect_to_vsp(struct hn_softc *sc); static void hv_nv_on_send_completion(struct hn_softc *sc, struct vmbus_channel *, const struct vmbus_chanpkt_hdr *pkt); static void hv_nv_on_receive_completion(struct vmbus_channel *chan, uint64_t tid); static void hv_nv_on_receive(struct hn_softc *sc, struct hn_rx_ring *rxr, struct vmbus_channel *chan, const struct vmbus_chanpkt_hdr *pkt); static void hn_nvs_sent_none(struct hn_send_ctx *sndc, struct hn_softc *, struct vmbus_channel *chan, const void *, int); -static void hn_nvs_sent_xact(struct hn_send_ctx *, struct hn_softc *sc, - struct vmbus_channel *, const void *, int); struct hn_send_ctx hn_send_ctx_none = HN_SEND_CTX_INITIALIZER(hn_nvs_sent_none, NULL); uint32_t hn_chim_alloc(struct hn_softc *sc) { int i, bmap_cnt = sc->hn_chim_bmap_cnt; u_long *bmap = sc->hn_chim_bmap; uint32_t ret = HN_NVS_CHIM_IDX_INVALID; for (i = 0; i < bmap_cnt; ++i) { int idx; idx = ffsl(~bmap[i]); if (idx == 0) continue; --idx; /* ffsl is 1-based */ KASSERT(i * LONG_BIT + idx < sc->hn_chim_cnt, ("invalid i %d and idx %d", i, idx)); if (atomic_testandset_long(&bmap[i], idx)) continue; ret = i * LONG_BIT + idx; break; } return (ret); } const void * hn_nvs_xact_execute(struct hn_softc *sc, struct vmbus_xact *xact, - void *req, int reqlen, size_t *resp_len) + void *req, int reqlen, size_t *resplen0, uint32_t type) { struct hn_send_ctx sndc; + size_t resplen, min_resplen = *resplen0; + const struct hn_nvs_hdr *hdr; int error; + KASSERT(min_resplen >= sizeof(*hdr), + ("invalid minimum response len %zu", min_resplen)); + + /* + * Execute the xact setup by the caller. + */ hn_send_ctx_init_simple(&sndc, hn_nvs_sent_xact, xact); - vmbus_xact_activate(xact); + vmbus_xact_activate(xact); error = hn_nvs_send(sc->hn_prichan, VMBUS_CHANPKT_FLAG_RC, req, reqlen, &sndc); if (error) { vmbus_xact_deactivate(xact); - return NULL; + return (NULL); } - return (vmbus_xact_wait(xact, resp_len)); + hdr = vmbus_xact_wait(xact, &resplen); + + /* + * Check this NVS response message. + */ + if (resplen < min_resplen) { + if_printf(sc->hn_ifp, "invalid NVS resp len %zu\n", resplen); + return (NULL); + } + if (hdr->nvs_type != type) { + if_printf(sc->hn_ifp, "unexpected NVS resp 0x%08x, " + "expect 0x%08x\n", hdr->nvs_type, type); + return (NULL); + } + /* All pass! */ + *resplen0 = resplen; + return (hdr); } static __inline int hn_nvs_req_send(struct hn_softc *sc, void *req, int reqlen) { return (hn_nvs_send(sc->hn_prichan, VMBUS_CHANPKT_FLAG_NONE, req, reqlen, &hn_send_ctx_none)); } /* * Net VSC initialize receive buffer with net VSP * * Net VSP: Network virtual services client, also known as the * Hyper-V extensible switch and the synthetic data path. */ static int hv_nv_init_rx_buffer_with_net_vsp(struct hn_softc *sc, int rxbuf_size) { struct vmbus_xact *xact = NULL; struct hn_nvs_rxbuf_conn *conn; const struct hn_nvs_rxbuf_connresp *resp; size_t resp_len; uint32_t status; int error; KASSERT(rxbuf_size <= NETVSC_RECEIVE_BUFFER_SIZE, ("invalid rxbuf size %d", rxbuf_size)); /* * Connect the RXBUF GPADL to the primary channel. * * NOTE: * Only primary channel has RXBUF connected to it. Sub-channels * just share this RXBUF. */ error = vmbus_chan_gpadl_connect(sc->hn_prichan, sc->hn_rxbuf_dma.hv_paddr, rxbuf_size, &sc->hn_rxbuf_gpadl); if (error) { if_printf(sc->hn_ifp, "rxbuf gpadl connect failed: %d\n", error); goto cleanup; } /* * Connect RXBUF to NVS. */ xact = vmbus_xact_get(sc->hn_xact, sizeof(*conn)); if (xact == NULL) { if_printf(sc->hn_ifp, "no xact for nvs rxbuf conn\n"); error = ENXIO; goto cleanup; } conn = vmbus_xact_req_data(xact); conn->nvs_type = HN_NVS_TYPE_RXBUF_CONN; conn->nvs_gpadl = sc->hn_rxbuf_gpadl; conn->nvs_sig = HN_NVS_RXBUF_SIG; - resp = hn_nvs_xact_execute(sc, xact, conn, sizeof(*conn), &resp_len); + resp_len = sizeof(*resp); + resp = hn_nvs_xact_execute(sc, xact, conn, sizeof(*conn), &resp_len, + HN_NVS_TYPE_RXBUF_CONNRESP); if (resp == NULL) { if_printf(sc->hn_ifp, "exec rxbuf conn failed\n"); error = EIO; goto cleanup; } - if (resp_len < sizeof(*resp)) { - if_printf(sc->hn_ifp, "invalid rxbuf conn resp length %zu\n", - resp_len); - error = EINVAL; - goto cleanup; - } - if (resp->nvs_type != HN_NVS_TYPE_RXBUF_CONNRESP) { - if_printf(sc->hn_ifp, "not rxbuf conn resp, type %u\n", - resp->nvs_type); - error = EINVAL; - goto cleanup; - } status = resp->nvs_status; vmbus_xact_put(xact); xact = NULL; if (status != HN_NVS_STATUS_OK) { if_printf(sc->hn_ifp, "rxbuf conn failed: %x\n", status); error = EIO; goto cleanup; } sc->hn_flags |= HN_FLAG_RXBUF_CONNECTED; return (0); cleanup: if (xact != NULL) vmbus_xact_put(xact); hv_nv_destroy_rx_buffer(sc); return (error); } /* * Net VSC initialize send buffer with net VSP */ static int hv_nv_init_send_buffer_with_net_vsp(struct hn_softc *sc) { struct vmbus_xact *xact = NULL; struct hn_nvs_chim_conn *chim; const struct hn_nvs_chim_connresp *resp; size_t resp_len; uint32_t status, sectsz; int error; /* * Connect chimney sending buffer GPADL to the primary channel. * * NOTE: * Only primary channel has chimney sending buffer connected to it. * Sub-channels just share this chimney sending buffer. */ error = vmbus_chan_gpadl_connect(sc->hn_prichan, sc->hn_chim_dma.hv_paddr, NETVSC_SEND_BUFFER_SIZE, &sc->hn_chim_gpadl); if (error) { if_printf(sc->hn_ifp, "chimney sending buffer gpadl " "connect failed: %d\n", error); goto cleanup; } /* * Connect chimney sending buffer to NVS */ xact = vmbus_xact_get(sc->hn_xact, sizeof(*chim)); if (xact == NULL) { if_printf(sc->hn_ifp, "no xact for nvs chim conn\n"); error = ENXIO; goto cleanup; } chim = vmbus_xact_req_data(xact); chim->nvs_type = HN_NVS_TYPE_CHIM_CONN; chim->nvs_gpadl = sc->hn_chim_gpadl; chim->nvs_sig = HN_NVS_CHIM_SIG; - resp = hn_nvs_xact_execute(sc, xact, chim, sizeof(*chim), &resp_len); + resp_len = sizeof(*resp); + resp = hn_nvs_xact_execute(sc, xact, chim, sizeof(*chim), &resp_len, + HN_NVS_TYPE_CHIM_CONNRESP); if (resp == NULL) { if_printf(sc->hn_ifp, "exec chim conn failed\n"); error = EIO; goto cleanup; } - if (resp_len < sizeof(*resp)) { - if_printf(sc->hn_ifp, "invalid chim conn resp length %zu\n", - resp_len); - error = EINVAL; - goto cleanup; - } - if (resp->nvs_type != HN_NVS_TYPE_CHIM_CONNRESP) { - if_printf(sc->hn_ifp, "not chim conn resp, type %u\n", - resp->nvs_type); - error = EINVAL; - goto cleanup; - } status = resp->nvs_status; sectsz = resp->nvs_sectsz; vmbus_xact_put(xact); xact = NULL; if (status != HN_NVS_STATUS_OK) { if_printf(sc->hn_ifp, "chim conn failed: %x\n", status); error = EIO; goto cleanup; } if (sectsz == 0) { if_printf(sc->hn_ifp, "zero chimney sending buffer " "section size\n"); return 0; } sc->hn_chim_szmax = sectsz; sc->hn_chim_cnt = NETVSC_SEND_BUFFER_SIZE / sc->hn_chim_szmax; if (NETVSC_SEND_BUFFER_SIZE % sc->hn_chim_szmax != 0) { if_printf(sc->hn_ifp, "chimney sending sections are " "not properly aligned\n"); } if (sc->hn_chim_cnt % LONG_BIT != 0) { if_printf(sc->hn_ifp, "discard %d chimney sending sections\n", sc->hn_chim_cnt % LONG_BIT); } sc->hn_chim_bmap_cnt = sc->hn_chim_cnt / LONG_BIT; sc->hn_chim_bmap = malloc(sc->hn_chim_bmap_cnt * sizeof(u_long), M_NETVSC, M_WAITOK | M_ZERO); /* Done! */ sc->hn_flags |= HN_FLAG_CHIM_CONNECTED; if (bootverbose) { if_printf(sc->hn_ifp, "chimney sending buffer %d/%d\n", sc->hn_chim_szmax, sc->hn_chim_cnt); } return 0; cleanup: if (xact != NULL) vmbus_xact_put(xact); hv_nv_destroy_send_buffer(sc); return (error); } /* * Net VSC destroy receive buffer */ static int hv_nv_destroy_rx_buffer(struct hn_softc *sc) { int ret = 0; if (sc->hn_flags & HN_FLAG_RXBUF_CONNECTED) { struct hn_nvs_rxbuf_disconn disconn; /* * Disconnect RXBUF from NVS. */ memset(&disconn, 0, sizeof(disconn)); disconn.nvs_type = HN_NVS_TYPE_RXBUF_DISCONN; disconn.nvs_sig = HN_NVS_RXBUF_SIG; /* NOTE: No response. */ ret = hn_nvs_req_send(sc, &disconn, sizeof(disconn)); if (ret != 0) { if_printf(sc->hn_ifp, "send rxbuf disconn failed: %d\n", ret); return (ret); } sc->hn_flags &= ~HN_FLAG_RXBUF_CONNECTED; } if (sc->hn_rxbuf_gpadl != 0) { /* * Disconnect RXBUF from primary channel. */ ret = vmbus_chan_gpadl_disconnect(sc->hn_prichan, sc->hn_rxbuf_gpadl); if (ret != 0) { if_printf(sc->hn_ifp, "rxbuf disconn failed: %d\n", ret); return (ret); } sc->hn_rxbuf_gpadl = 0; } return (ret); } /* * Net VSC destroy send buffer */ static int hv_nv_destroy_send_buffer(struct hn_softc *sc) { int ret = 0; if (sc->hn_flags & HN_FLAG_CHIM_CONNECTED) { struct hn_nvs_chim_disconn disconn; /* * Disconnect chimney sending buffer from NVS. */ memset(&disconn, 0, sizeof(disconn)); disconn.nvs_type = HN_NVS_TYPE_CHIM_DISCONN; disconn.nvs_sig = HN_NVS_CHIM_SIG; /* NOTE: No response. */ ret = hn_nvs_req_send(sc, &disconn, sizeof(disconn)); if (ret != 0) { if_printf(sc->hn_ifp, "send chim disconn failed: %d\n", ret); return (ret); } sc->hn_flags &= ~HN_FLAG_CHIM_CONNECTED; } if (sc->hn_chim_gpadl != 0) { /* * Disconnect chimney sending buffer from primary channel. */ ret = vmbus_chan_gpadl_disconnect(sc->hn_prichan, sc->hn_chim_gpadl); if (ret != 0) { if_printf(sc->hn_ifp, "chim disconn failed: %d\n", ret); return (ret); } sc->hn_chim_gpadl = 0; } if (sc->hn_chim_bmap != NULL) { free(sc->hn_chim_bmap, M_NETVSC); sc->hn_chim_bmap = NULL; } return (ret); } static int hv_nv_negotiate_nvsp_protocol(struct hn_softc *sc, uint32_t nvs_ver) { struct vmbus_xact *xact; struct hn_nvs_init *init; const struct hn_nvs_init_resp *resp; size_t resp_len; uint32_t status; xact = vmbus_xact_get(sc->hn_xact, sizeof(*init)); if (xact == NULL) { if_printf(sc->hn_ifp, "no xact for nvs init\n"); return (ENXIO); } init = vmbus_xact_req_data(xact); init->nvs_type = HN_NVS_TYPE_INIT; init->nvs_ver_min = nvs_ver; init->nvs_ver_max = nvs_ver; - resp = hn_nvs_xact_execute(sc, xact, init, sizeof(*init), &resp_len); + resp_len = sizeof(*resp); + resp = hn_nvs_xact_execute(sc, xact, init, sizeof(*init), &resp_len, + HN_NVS_TYPE_INIT_RESP); if (resp == NULL) { if_printf(sc->hn_ifp, "exec init failed\n"); vmbus_xact_put(xact); return (EIO); } - if (resp_len < sizeof(*resp)) { - if_printf(sc->hn_ifp, "invalid init resp length %zu\n", - resp_len); - vmbus_xact_put(xact); - return (EINVAL); - } - if (resp->nvs_type != HN_NVS_TYPE_INIT_RESP) { - if_printf(sc->hn_ifp, "not init resp, type %u\n", - resp->nvs_type); - vmbus_xact_put(xact); - return (EINVAL); - } status = resp->nvs_status; vmbus_xact_put(xact); if (status != HN_NVS_STATUS_OK) { if_printf(sc->hn_ifp, "nvs init failed for ver 0x%x\n", nvs_ver); return (EINVAL); } return (0); } /* * Send NDIS version 2 config packet containing MTU. * * Not valid for NDIS version 1. */ static int hv_nv_send_ndis_config(struct hn_softc *sc, uint32_t mtu) { struct hn_nvs_ndis_conf conf; int error; memset(&conf, 0, sizeof(conf)); conf.nvs_type = HN_NVS_TYPE_NDIS_CONF; conf.nvs_mtu = mtu; conf.nvs_caps = HN_NVS_NDIS_CONF_VLAN; /* NOTE: No response. */ error = hn_nvs_req_send(sc, &conf, sizeof(conf)); if (error) if_printf(sc->hn_ifp, "send nvs ndis conf failed: %d\n", error); return (error); } /* * Net VSC connect to VSP */ static int hv_nv_connect_to_vsp(struct hn_softc *sc) { uint32_t protocol_list[] = { NVSP_PROTOCOL_VERSION_1, NVSP_PROTOCOL_VERSION_2, NVSP_PROTOCOL_VERSION_4, NVSP_PROTOCOL_VERSION_5 }; int i; int protocol_number = nitems(protocol_list); int ret = 0; device_t dev = sc->hn_dev; struct ifnet *ifp = sc->hn_ifp; struct hn_nvs_ndis_init ndis; int rxbuf_size; /* * Negotiate the NVSP version. Try the latest NVSP first. */ for (i = protocol_number - 1; i >= 0; i--) { if (hv_nv_negotiate_nvsp_protocol(sc, protocol_list[i]) == 0) { sc->hn_nvs_ver = protocol_list[i]; sc->hn_ndis_ver = NDIS_VERSION_6_30; if (sc->hn_nvs_ver <= NVSP_PROTOCOL_VERSION_4) sc->hn_ndis_ver = NDIS_VERSION_6_1; if (bootverbose) { if_printf(sc->hn_ifp, "NVS version 0x%x, " "NDIS version %u.%u\n", sc->hn_nvs_ver, NDIS_VERSION_MAJOR(sc->hn_ndis_ver), NDIS_VERSION_MINOR(sc->hn_ndis_ver)); } break; } } if (i < 0) { if (bootverbose) device_printf(dev, "failed to negotiate a valid " "protocol.\n"); return (EPROTO); } /* * Set the MTU if supported by this NVSP protocol version * This needs to be right after the NVSP init message per Haiyang */ if (sc->hn_nvs_ver >= NVSP_PROTOCOL_VERSION_2) ret = hv_nv_send_ndis_config(sc, ifp->if_mtu); /* * Initialize NDIS. */ memset(&ndis, 0, sizeof(ndis)); ndis.nvs_type = HN_NVS_TYPE_NDIS_INIT; ndis.nvs_ndis_major = NDIS_VERSION_MAJOR(sc->hn_ndis_ver); ndis.nvs_ndis_minor = NDIS_VERSION_MINOR(sc->hn_ndis_ver); /* NOTE: No response. */ ret = hn_nvs_req_send(sc, &ndis, sizeof(ndis)); if (ret != 0) { if_printf(sc->hn_ifp, "send nvs ndis init failed: %d\n", ret); goto cleanup; } /* Post the big receive buffer to NetVSP */ if (sc->hn_nvs_ver <= NVSP_PROTOCOL_VERSION_2) rxbuf_size = NETVSC_RECEIVE_BUFFER_SIZE_LEGACY; else rxbuf_size = NETVSC_RECEIVE_BUFFER_SIZE; ret = hv_nv_init_rx_buffer_with_net_vsp(sc, rxbuf_size); if (ret == 0) ret = hv_nv_init_send_buffer_with_net_vsp(sc); cleanup: return (ret); } /* * Net VSC disconnect from VSP */ static void hv_nv_disconnect_from_vsp(struct hn_softc *sc) { hv_nv_destroy_rx_buffer(sc); hv_nv_destroy_send_buffer(sc); } void hv_nv_subchan_attach(struct vmbus_channel *chan, struct hn_rx_ring *rxr) { KASSERT(rxr->hn_rx_idx == vmbus_chan_subidx(chan), ("chan%u subidx %u, rxr%d mismatch", vmbus_chan_id(chan), vmbus_chan_subidx(chan), rxr->hn_rx_idx)); vmbus_chan_open(chan, NETVSC_DEVICE_RING_BUFFER_SIZE, NETVSC_DEVICE_RING_BUFFER_SIZE, NULL, 0, hv_nv_on_channel_callback, rxr); } /* * Net VSC on device add * * Callback when the device belonging to this driver is added */ int hv_nv_on_device_add(struct hn_softc *sc, struct hn_rx_ring *rxr) { struct vmbus_channel *chan = sc->hn_prichan; int ret = 0; /* * Open the channel */ KASSERT(rxr->hn_rx_idx == vmbus_chan_subidx(chan), ("chan%u subidx %u, rxr%d mismatch", vmbus_chan_id(chan), vmbus_chan_subidx(chan), rxr->hn_rx_idx)); ret = vmbus_chan_open(chan, NETVSC_DEVICE_RING_BUFFER_SIZE, NETVSC_DEVICE_RING_BUFFER_SIZE, NULL, 0, hv_nv_on_channel_callback, rxr); if (ret != 0) goto cleanup; /* * Connect with the NetVsp */ ret = hv_nv_connect_to_vsp(sc); if (ret != 0) goto close; return (0); close: /* Now, we can close the channel safely */ vmbus_chan_close(chan); cleanup: return (ret); } /* * Net VSC on device remove */ int -hv_nv_on_device_remove(struct hn_softc *sc, boolean_t destroy_channel) +hv_nv_on_device_remove(struct hn_softc *sc) { hv_nv_disconnect_from_vsp(sc); /* Now, we can close the channel safely */ vmbus_chan_close(sc->hn_prichan); return (0); } -static void +void hn_nvs_sent_xact(struct hn_send_ctx *sndc, struct hn_softc *sc __unused, struct vmbus_channel *chan __unused, const void *data, int dlen) { vmbus_xact_wakeup(sndc->hn_cbarg, data, dlen); } static void hn_nvs_sent_none(struct hn_send_ctx *sndc __unused, struct hn_softc *sc __unused, struct vmbus_channel *chan __unused, const void *data __unused, int dlen __unused) { /* EMPTY */ } void hn_chim_free(struct hn_softc *sc, uint32_t chim_idx) { u_long mask; uint32_t idx; idx = chim_idx / LONG_BIT; KASSERT(idx < sc->hn_chim_bmap_cnt, ("invalid chimney index 0x%x", chim_idx)); mask = 1UL << (chim_idx % LONG_BIT); KASSERT(sc->hn_chim_bmap[idx] & mask, ("index bitmap 0x%lx, chimney index %u, " "bitmap idx %d, bitmask 0x%lx", sc->hn_chim_bmap[idx], chim_idx, idx, mask)); atomic_clear_long(&sc->hn_chim_bmap[idx], mask); } /* * Net VSC on send completion */ static void hv_nv_on_send_completion(struct hn_softc *sc, struct vmbus_channel *chan, const struct vmbus_chanpkt_hdr *pkt) { struct hn_send_ctx *sndc; sndc = (struct hn_send_ctx *)(uintptr_t)pkt->cph_xactid; sndc->hn_cb(sndc, sc, chan, VMBUS_CHANPKT_CONST_DATA(pkt), VMBUS_CHANPKT_DATALEN(pkt)); /* * NOTE: * 'sndc' CAN NOT be accessed anymore, since it can be freed by * its callback. */ } /* * Net VSC on send * Sends a packet on the specified Hyper-V device. * Returns 0 on success, non-zero on failure. */ int hv_nv_on_send(struct vmbus_channel *chan, uint32_t rndis_mtype, struct hn_send_ctx *sndc, struct vmbus_gpa *gpa, int gpa_cnt) { struct hn_nvs_rndis rndis; int ret; rndis.nvs_type = HN_NVS_TYPE_RNDIS; rndis.nvs_rndis_mtype = rndis_mtype; rndis.nvs_chim_idx = sndc->hn_chim_idx; rndis.nvs_chim_sz = sndc->hn_chim_sz; if (gpa_cnt) { ret = hn_nvs_send_sglist(chan, gpa, gpa_cnt, &rndis, sizeof(rndis), sndc); } else { ret = hn_nvs_send(chan, VMBUS_CHANPKT_FLAG_RC, &rndis, sizeof(rndis), sndc); } return (ret); } /* * Net VSC on receive * * In the FreeBSD Hyper-V virtual world, this function deals exclusively * with virtual addresses. */ static void hv_nv_on_receive(struct hn_softc *sc, struct hn_rx_ring *rxr, struct vmbus_channel *chan, const struct vmbus_chanpkt_hdr *pkthdr) { const struct vmbus_chanpkt_rxbuf *pkt; const struct hn_nvs_hdr *nvs_hdr; int count = 0; int i = 0; /* Make sure that this is a RNDIS message. */ nvs_hdr = VMBUS_CHANPKT_CONST_DATA(pkthdr); if (__predict_false(nvs_hdr->nvs_type != HN_NVS_TYPE_RNDIS)) { if_printf(rxr->hn_ifp, "nvs type %u, not RNDIS\n", nvs_hdr->nvs_type); return; } pkt = (const struct vmbus_chanpkt_rxbuf *)pkthdr; if (pkt->cp_rxbuf_id != NETVSC_RECEIVE_BUFFER_ID) { if_printf(rxr->hn_ifp, "rxbuf_id %d is invalid!\n", pkt->cp_rxbuf_id); return; } count = pkt->cp_rxbuf_cnt; /* Each range represents 1 RNDIS pkt that contains 1 Ethernet frame */ for (i = 0; i < count; i++) { hv_rf_on_receive(sc, rxr, rxr->hn_rxbuf + pkt->cp_rxbuf[i].rb_ofs, pkt->cp_rxbuf[i].rb_len); } /* * Moved completion call back here so that all received * messages (not just data messages) will trigger a response * message back to the host. */ hv_nv_on_receive_completion(chan, pkt->cp_hdr.cph_xactid); } /* * Net VSC on receive completion * * Send a receive completion packet to RNDIS device (ie NetVsp) */ static void hv_nv_on_receive_completion(struct vmbus_channel *chan, uint64_t tid) { struct hn_nvs_rndis_ack ack; int retries = 0; int ret = 0; ack.nvs_type = HN_NVS_TYPE_RNDIS_ACK; ack.nvs_status = HN_NVS_STATUS_OK; retry_send_cmplt: /* Send the completion */ ret = vmbus_chan_send(chan, VMBUS_CHANPKT_TYPE_COMP, VMBUS_CHANPKT_FLAG_NONE, &ack, sizeof(ack), tid); if (ret == 0) { /* success */ /* no-op */ } else if (ret == EAGAIN) { /* no more room... wait a bit and attempt to retry 3 times */ retries++; if (retries < 4) { DELAY(100); goto retry_send_cmplt; } } } static void hn_proc_notify(struct hn_softc *sc, const struct vmbus_chanpkt_hdr *pkt) { const struct hn_nvs_hdr *hdr; hdr = VMBUS_CHANPKT_CONST_DATA(pkt); if (hdr->nvs_type == HN_NVS_TYPE_TXTBL_NOTE) { /* Useless; ignore */ return; } if_printf(sc->hn_ifp, "got notify, nvs type %u\n", hdr->nvs_type); } /* * Net VSC on channel callback */ static void hv_nv_on_channel_callback(struct vmbus_channel *chan, void *xrxr) { struct hn_rx_ring *rxr = xrxr; struct hn_softc *sc = rxr->hn_ifp->if_softc; void *buffer; int bufferlen = NETVSC_PACKET_SIZE; buffer = rxr->hn_rdbuf; do { struct vmbus_chanpkt_hdr *pkt = buffer; uint32_t bytes_rxed; int ret; bytes_rxed = bufferlen; ret = vmbus_chan_recv_pkt(chan, pkt, &bytes_rxed); if (ret == 0) { if (bytes_rxed > 0) { switch (pkt->cph_type) { case VMBUS_CHANPKT_TYPE_COMP: hv_nv_on_send_completion(sc, chan, pkt); break; case VMBUS_CHANPKT_TYPE_RXBUF: hv_nv_on_receive(sc, rxr, chan, pkt); break; case VMBUS_CHANPKT_TYPE_INBAND: hn_proc_notify(sc, pkt); break; default: if_printf(rxr->hn_ifp, "unknown chan pkt %u\n", pkt->cph_type); break; } } } else if (ret == ENOBUFS) { /* Handle large packet */ if (bufferlen > NETVSC_PACKET_SIZE) { free(buffer, M_NETVSC); buffer = NULL; } /* alloc new buffer */ buffer = malloc(bytes_rxed, M_NETVSC, M_NOWAIT); if (buffer == NULL) { if_printf(rxr->hn_ifp, "hv_cb malloc buffer failed, len=%u\n", bytes_rxed); bufferlen = 0; break; } bufferlen = bytes_rxed; } else { /* No more packets */ break; } } while (1); if (bufferlen > NETVSC_PACKET_SIZE) free(buffer, M_NETVSC); hv_rf_channel_rollup(rxr, rxr->hn_txr); } Index: projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_net_vsc.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_net_vsc.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_net_vsc.h (revision 305172) @@ -1,411 +1,409 @@ /*- * Copyright (c) 2009-2012,2016 Microsoft Corp. * Copyright (c) 2010-2012 Citrix Inc. * Copyright (c) 2012 NetApp Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ /* * HyperV vmbus (virtual machine bus) network VSC (virtual services client) * header file * * (Updated from unencumbered NvspProtocol.h) */ #ifndef __HV_NET_VSC_H__ #define __HV_NET_VSC_H__ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define HN_USE_TXDESC_BUFRING MALLOC_DECLARE(M_NETVSC); #define NVSP_INVALID_PROTOCOL_VERSION (0xFFFFFFFF) #define NVSP_PROTOCOL_VERSION_1 2 #define NVSP_PROTOCOL_VERSION_2 0x30002 #define NVSP_PROTOCOL_VERSION_4 0x40000 #define NVSP_PROTOCOL_VERSION_5 0x50000 #define NVSP_MIN_PROTOCOL_VERSION (NVSP_PROTOCOL_VERSION_1) #define NVSP_MAX_PROTOCOL_VERSION (NVSP_PROTOCOL_VERSION_2) #define NVSP_PROTOCOL_VERSION_CURRENT NVSP_PROTOCOL_VERSION_2 #define VERSION_4_OFFLOAD_SIZE 22 #define NVSP_OPERATIONAL_STATUS_OK (0x00000000) #define NVSP_OPERATIONAL_STATUS_DEGRADED (0x00000001) #define NVSP_OPERATIONAL_STATUS_NONRECOVERABLE (0x00000002) #define NVSP_OPERATIONAL_STATUS_NO_CONTACT (0x00000003) #define NVSP_OPERATIONAL_STATUS_LOST_COMMUNICATION (0x00000004) /* * Maximun number of transfer pages (packets) the VSP will use on a receive */ #define NVSP_MAX_PACKETS_PER_RECEIVE 375 /* vRSS stuff */ #define RNDIS_OBJECT_TYPE_RSS_CAPABILITIES 0x88 #define RNDIS_OBJECT_TYPE_RSS_PARAMETERS 0x89 #define RNDIS_RECEIVE_SCALE_CAPABILITIES_REVISION_2 2 #define RNDIS_RECEIVE_SCALE_PARAMETERS_REVISION_2 2 struct rndis_obj_header { uint8_t type; uint8_t rev; uint16_t size; } __packed; /* rndis_recv_scale_cap/cap_flag */ #define RNDIS_RSS_CAPS_MESSAGE_SIGNALED_INTERRUPTS 0x01000000 #define RNDIS_RSS_CAPS_CLASSIFICATION_AT_ISR 0x02000000 #define RNDIS_RSS_CAPS_CLASSIFICATION_AT_DPC 0x04000000 #define RNDIS_RSS_CAPS_USING_MSI_X 0x08000000 #define RNDIS_RSS_CAPS_RSS_AVAILABLE_ON_PORTS 0x10000000 #define RNDIS_RSS_CAPS_SUPPORTS_MSI_X 0x20000000 #define RNDIS_RSS_CAPS_HASH_TYPE_TCP_IPV4 0x00000100 #define RNDIS_RSS_CAPS_HASH_TYPE_TCP_IPV6 0x00000200 #define RNDIS_RSS_CAPS_HASH_TYPE_TCP_IPV6_EX 0x00000400 /* RNDIS_RECEIVE_SCALE_CAPABILITIES */ struct rndis_recv_scale_cap { struct rndis_obj_header hdr; uint32_t cap_flag; uint32_t num_int_msg; uint32_t num_recv_que; uint16_t num_indirect_tabent; } __packed; /* rndis_recv_scale_param flags */ #define RNDIS_RSS_PARAM_FLAG_BASE_CPU_UNCHANGED 0x0001 #define RNDIS_RSS_PARAM_FLAG_HASH_INFO_UNCHANGED 0x0002 #define RNDIS_RSS_PARAM_FLAG_ITABLE_UNCHANGED 0x0004 #define RNDIS_RSS_PARAM_FLAG_HASH_KEY_UNCHANGED 0x0008 #define RNDIS_RSS_PARAM_FLAG_DISABLE_RSS 0x0010 /* Hash info bits */ #define RNDIS_HASH_FUNC_TOEPLITZ 0x00000001 #define RNDIS_HASH_IPV4 0x00000100 #define RNDIS_HASH_TCP_IPV4 0x00000200 #define RNDIS_HASH_IPV6 0x00000400 #define RNDIS_HASH_IPV6_EX 0x00000800 #define RNDIS_HASH_TCP_IPV6 0x00001000 #define RNDIS_HASH_TCP_IPV6_EX 0x00002000 #define RNDIS_RSS_INDIRECTION_TABLE_MAX_SIZE_REVISION_2 (128 * 4) #define RNDIS_RSS_HASH_SECRET_KEY_MAX_SIZE_REVISION_2 40 #define ITAB_NUM 128 #define HASH_KEYLEN RNDIS_RSS_HASH_SECRET_KEY_MAX_SIZE_REVISION_2 /* RNDIS_RECEIVE_SCALE_PARAMETERS */ typedef struct rndis_recv_scale_param_ { struct rndis_obj_header hdr; /* Qualifies the rest of the information */ uint16_t flag; /* The base CPU number to do receive processing. not used */ uint16_t base_cpu_number; /* This describes the hash function and type being enabled */ uint32_t hashinfo; /* The size of indirection table array */ uint16_t indirect_tabsize; /* The offset of the indirection table from the beginning of this * structure */ uint32_t indirect_taboffset; /* The size of the hash secret key */ uint16_t hashkey_size; /* The offset of the secret key from the beginning of this structure */ uint32_t hashkey_offset; uint32_t processor_masks_offset; uint32_t num_processor_masks; uint32_t processor_masks_entry_size; } rndis_recv_scale_param; /* * The following arguably belongs in a separate header file */ /* * Defines */ #define NETVSC_SEND_BUFFER_SIZE (1024*1024*15) /* 15M */ #define NETVSC_SEND_BUFFER_ID 0xface #define NETVSC_RECEIVE_BUFFER_SIZE_LEGACY (1024*1024*15) /* 15MB */ #define NETVSC_RECEIVE_BUFFER_SIZE (1024*1024*16) /* 16MB */ #define NETVSC_RECEIVE_BUFFER_ID 0xcafe #define NETVSC_RECEIVE_SG_COUNT 1 /* Preallocated receive packets */ #define NETVSC_RECEIVE_PACKETLIST_COUNT 256 /* * Maximum MTU we permit to be configured for a netvsc interface. * When the code was developed, a max MTU of 12232 was tested and * proven to work. 9K is a reasonable maximum for an Ethernet. */ #define NETVSC_MAX_CONFIGURABLE_MTU (9 * 1024) #define NETVSC_PACKET_SIZE PAGE_SIZE #define VRSS_SEND_TABLE_SIZE 16 /* * Data types */ struct vmbus_channel; typedef void (*pfn_on_send_rx_completion)(struct vmbus_channel *, void *); #define NETVSC_DEVICE_RING_BUFFER_SIZE (128 * PAGE_SIZE) #define NETVSC_PACKET_MAXPAGE 32 #define NETVSC_VLAN_PRIO_MASK 0xe000 #define NETVSC_VLAN_PRIO_SHIFT 13 #define NETVSC_VLAN_VID_MASK 0x0fff #define TYPE_IPV4 2 #define TYPE_IPV6 4 #define TYPE_TCP 2 #define TYPE_UDP 4 #define TRANSPORT_TYPE_NOT_IP 0 #define TRANSPORT_TYPE_IPV4_TCP ((TYPE_IPV4 << 16) | TYPE_TCP) #define TRANSPORT_TYPE_IPV4_UDP ((TYPE_IPV4 << 16) | TYPE_UDP) #define TRANSPORT_TYPE_IPV6_TCP ((TYPE_IPV6 << 16) | TYPE_TCP) #define TRANSPORT_TYPE_IPV6_UDP ((TYPE_IPV6 << 16) | TYPE_UDP) typedef struct { - uint8_t mac_addr[6]; /* Assumption unsigned long */ - uint8_t link_state; + uint8_t mac_addr[ETHER_ADDR_LEN]; + uint32_t link_state; } netvsc_device_info; #define HN_XACT_REQ_PGCNT 2 #define HN_XACT_RESP_PGCNT 2 #define HN_XACT_REQ_SIZE (HN_XACT_REQ_PGCNT * PAGE_SIZE) #define HN_XACT_RESP_SIZE (HN_XACT_RESP_PGCNT * PAGE_SIZE) #ifndef HN_USE_TXDESC_BUFRING struct hn_txdesc; SLIST_HEAD(hn_txdesc_list, hn_txdesc); #else struct buf_ring; #endif struct hn_tx_ring; struct hn_rx_ring { struct ifnet *hn_ifp; struct hn_tx_ring *hn_txr; void *hn_rdbuf; uint8_t *hn_rxbuf; /* shadow sc->hn_rxbuf */ int hn_rx_idx; /* Trust csum verification on host side */ int hn_trust_hcsum; /* HN_TRUST_HCSUM_ */ struct lro_ctrl hn_lro; u_long hn_csum_ip; u_long hn_csum_tcp; u_long hn_csum_udp; u_long hn_csum_trusted; u_long hn_lro_tried; u_long hn_small_pkts; u_long hn_pkts; u_long hn_rss_pkts; /* Rarely used stuffs */ struct sysctl_oid *hn_rx_sysctl_tree; int hn_rx_flags; } __aligned(CACHE_LINE_SIZE); #define HN_TRUST_HCSUM_IP 0x0001 #define HN_TRUST_HCSUM_TCP 0x0002 #define HN_TRUST_HCSUM_UDP 0x0004 #define HN_RX_FLAG_ATTACHED 0x1 struct hn_tx_ring { #ifndef HN_USE_TXDESC_BUFRING struct mtx hn_txlist_spin; struct hn_txdesc_list hn_txlist; #else struct buf_ring *hn_txdesc_br; #endif int hn_txdesc_cnt; int hn_txdesc_avail; u_short hn_has_txeof; u_short hn_txdone_cnt; int hn_sched_tx; void (*hn_txeof)(struct hn_tx_ring *); struct taskqueue *hn_tx_taskq; struct task hn_tx_task; struct task hn_txeof_task; struct buf_ring *hn_mbuf_br; int hn_oactive; int hn_tx_idx; struct mtx hn_tx_lock; struct hn_softc *hn_sc; struct vmbus_channel *hn_chan; int hn_direct_tx_size; int hn_chim_size; bus_dma_tag_t hn_tx_data_dtag; uint64_t hn_csum_assist; int hn_gpa_cnt; struct vmbus_gpa hn_gpa[NETVSC_PACKET_MAXPAGE]; u_long hn_no_txdescs; u_long hn_send_failed; u_long hn_txdma_failed; u_long hn_tx_collapsed; u_long hn_tx_chimney_tried; u_long hn_tx_chimney; u_long hn_pkts; /* Rarely used stuffs */ struct hn_txdesc *hn_txdesc; bus_dma_tag_t hn_tx_rndis_dtag; struct sysctl_oid *hn_tx_sysctl_tree; int hn_tx_flags; } __aligned(CACHE_LINE_SIZE); #define HN_TX_FLAG_ATTACHED 0x1 /* * Device-specific softc structure */ typedef struct hn_softc { struct ifnet *hn_ifp; struct ifmedia hn_media; device_t hn_dev; uint8_t hn_unit; int hn_carrier; int hn_if_flags; struct mtx hn_lock; int hn_initdone; /* See hv_netvsc_drv_freebsd.c for rules on how to use */ int temp_unusable; - struct rndis_device_ *rndis_dev; struct vmbus_channel *hn_prichan; int hn_rx_ring_cnt; int hn_rx_ring_inuse; struct hn_rx_ring *hn_rx_ring; int hn_tx_ring_cnt; int hn_tx_ring_inuse; struct hn_tx_ring *hn_tx_ring; uint8_t *hn_chim; u_long *hn_chim_bmap; int hn_chim_bmap_cnt; int hn_chim_cnt; int hn_chim_szmax; int hn_cpu; struct taskqueue *hn_tx_taskq; struct sysctl_oid *hn_tx_sysctl_tree; struct sysctl_oid *hn_rx_sysctl_tree; struct vmbus_xact_ctx *hn_xact; uint32_t hn_nvs_ver; uint32_t hn_flags; void *hn_rxbuf; uint32_t hn_rxbuf_gpadl; struct hyperv_dma hn_rxbuf_dma; uint32_t hn_chim_gpadl; struct hyperv_dma hn_chim_dma; uint32_t hn_rndis_rid; uint32_t hn_ndis_ver; struct ndis_rssprm_toeplitz hn_rss; } hn_softc_t; #define HN_FLAG_RXBUF_CONNECTED 0x0001 #define HN_FLAG_CHIM_CONNECTED 0x0002 /* * Externs */ extern int hv_promisc_mode; struct hn_send_ctx; void netvsc_linkstatus_callback(struct hn_softc *sc, uint32_t status); int hv_nv_on_device_add(struct hn_softc *sc, struct hn_rx_ring *rxr); -int hv_nv_on_device_remove(struct hn_softc *sc, - boolean_t destroy_channel); +int hv_nv_on_device_remove(struct hn_softc *sc); int hv_nv_on_send(struct vmbus_channel *chan, uint32_t rndis_mtype, struct hn_send_ctx *sndc, struct vmbus_gpa *gpa, int gpa_cnt); void hv_nv_subchan_attach(struct vmbus_channel *chan, struct hn_rx_ring *rxr); #endif /* __HV_NET_VSC_H__ */ Index: projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c (revision 305172) @@ -1,3098 +1,3098 @@ /*- * Copyright (c) 2010-2012 Citrix Inc. * Copyright (c) 2009-2012,2016 Microsoft Corp. * Copyright (c) 2012 NetApp Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /*- * Copyright (c) 2004-2006 Kip Macy * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_inet6.h" #include "opt_inet.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "vmbus_if.h" /* Short for Hyper-V network interface */ #define NETVSC_DEVNAME "hn" /* * It looks like offset 0 of buf is reserved to hold the softc pointer. * The sc pointer evidently not needed, and is not presently populated. * The packet offset is where the netvsc_packet starts in the buffer. */ #define HV_NV_SC_PTR_OFFSET_IN_BUF 0 #define HV_NV_PACKET_OFFSET_IN_BUF 16 /* YYY should get it from the underlying channel */ #define HN_TX_DESC_CNT 512 #define HN_LROENT_CNT_DEF 128 #define HN_RING_CNT_DEF_MAX 8 #define HN_RNDIS_MSG_LEN \ (sizeof(rndis_msg) + \ RNDIS_HASHVAL_PPI_SIZE + \ RNDIS_VLAN_PPI_SIZE + \ RNDIS_TSO_PPI_SIZE + \ RNDIS_CSUM_PPI_SIZE) #define HN_RNDIS_MSG_BOUNDARY PAGE_SIZE #define HN_RNDIS_MSG_ALIGN CACHE_LINE_SIZE #define HN_TX_DATA_BOUNDARY PAGE_SIZE #define HN_TX_DATA_MAXSIZE IP_MAXPACKET #define HN_TX_DATA_SEGSIZE PAGE_SIZE -#define HN_TX_DATA_SEGCNT_MAX \ - (NETVSC_PACKET_MAXPAGE - HV_RF_NUM_TX_RESERVED_PAGE_BUFS) +/* -1 for RNDIS packet message */ +#define HN_TX_DATA_SEGCNT_MAX (NETVSC_PACKET_MAXPAGE - 1) #define HN_DIRECT_TX_SIZE_DEF 128 #define HN_EARLY_TXEOF_THRESH 8 struct hn_txdesc { #ifndef HN_USE_TXDESC_BUFRING SLIST_ENTRY(hn_txdesc) link; #endif struct mbuf *m; struct hn_tx_ring *txr; int refs; uint32_t flags; /* HN_TXD_FLAG_ */ struct hn_send_ctx send_ctx; bus_dmamap_t data_dmap; bus_addr_t rndis_msg_paddr; rndis_msg *rndis_msg; bus_dmamap_t rndis_msg_dmap; }; #define HN_TXD_FLAG_ONLIST 0x1 #define HN_TXD_FLAG_DMAMAP 0x2 /* * Only enable UDP checksum offloading when it is on 2012R2 or * later. UDP checksum offloading doesn't work on earlier * Windows releases. */ #define HN_CSUM_ASSIST_WIN8 (CSUM_IP | CSUM_TCP) #define HN_CSUM_ASSIST (CSUM_IP | CSUM_UDP | CSUM_TCP) #define HN_LRO_LENLIM_MULTIRX_DEF (12 * ETHERMTU) #define HN_LRO_LENLIM_DEF (25 * ETHERMTU) /* YYY 2*MTU is a bit rough, but should be good enough. */ #define HN_LRO_LENLIM_MIN(ifp) (2 * (ifp)->if_mtu) #define HN_LRO_ACKCNT_DEF 1 /* * Be aware that this sleepable mutex will exhibit WITNESS errors when * certain TCP and ARP code paths are taken. This appears to be a * well-known condition, as all other drivers checked use a sleeping * mutex to protect their transmit paths. * Also Be aware that mutexes do not play well with semaphores, and there * is a conflicting semaphore in a certain channel code path. */ #define NV_LOCK_INIT(_sc, _name) \ mtx_init(&(_sc)->hn_lock, _name, MTX_NETWORK_LOCK, MTX_DEF) #define NV_LOCK(_sc) mtx_lock(&(_sc)->hn_lock) #define NV_LOCK_ASSERT(_sc) mtx_assert(&(_sc)->hn_lock, MA_OWNED) #define NV_UNLOCK(_sc) mtx_unlock(&(_sc)->hn_lock) #define NV_LOCK_DESTROY(_sc) mtx_destroy(&(_sc)->hn_lock) /* * Globals */ int hv_promisc_mode = 0; /* normal mode by default */ SYSCTL_NODE(_hw, OID_AUTO, hn, CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, "Hyper-V network interface"); /* Trust tcp segements verification on host side. */ static int hn_trust_hosttcp = 1; SYSCTL_INT(_hw_hn, OID_AUTO, trust_hosttcp, CTLFLAG_RDTUN, &hn_trust_hosttcp, 0, "Trust tcp segement verification on host side, " "when csum info is missing (global setting)"); /* Trust udp datagrams verification on host side. */ static int hn_trust_hostudp = 1; SYSCTL_INT(_hw_hn, OID_AUTO, trust_hostudp, CTLFLAG_RDTUN, &hn_trust_hostudp, 0, "Trust udp datagram verification on host side, " "when csum info is missing (global setting)"); /* Trust ip packets verification on host side. */ static int hn_trust_hostip = 1; SYSCTL_INT(_hw_hn, OID_AUTO, trust_hostip, CTLFLAG_RDTUN, &hn_trust_hostip, 0, "Trust ip packet verification on host side, " "when csum info is missing (global setting)"); #if __FreeBSD_version >= 1100045 /* Limit TSO burst size */ static int hn_tso_maxlen = 0; SYSCTL_INT(_hw_hn, OID_AUTO, tso_maxlen, CTLFLAG_RDTUN, &hn_tso_maxlen, 0, "TSO burst limit"); #endif /* Limit chimney send size */ static int hn_tx_chimney_size = 0; SYSCTL_INT(_hw_hn, OID_AUTO, tx_chimney_size, CTLFLAG_RDTUN, &hn_tx_chimney_size, 0, "Chimney send packet size limit"); /* Limit the size of packet for direct transmission */ static int hn_direct_tx_size = HN_DIRECT_TX_SIZE_DEF; SYSCTL_INT(_hw_hn, OID_AUTO, direct_tx_size, CTLFLAG_RDTUN, &hn_direct_tx_size, 0, "Size of the packet for direct transmission"); #if defined(INET) || defined(INET6) #if __FreeBSD_version >= 1100095 static int hn_lro_entry_count = HN_LROENT_CNT_DEF; SYSCTL_INT(_hw_hn, OID_AUTO, lro_entry_count, CTLFLAG_RDTUN, &hn_lro_entry_count, 0, "LRO entry count"); #endif #endif static int hn_share_tx_taskq = 0; SYSCTL_INT(_hw_hn, OID_AUTO, share_tx_taskq, CTLFLAG_RDTUN, &hn_share_tx_taskq, 0, "Enable shared TX taskqueue"); static struct taskqueue *hn_tx_taskq; #ifndef HN_USE_TXDESC_BUFRING static int hn_use_txdesc_bufring = 0; #else static int hn_use_txdesc_bufring = 1; #endif SYSCTL_INT(_hw_hn, OID_AUTO, use_txdesc_bufring, CTLFLAG_RD, &hn_use_txdesc_bufring, 0, "Use buf_ring for TX descriptors"); static int hn_bind_tx_taskq = -1; SYSCTL_INT(_hw_hn, OID_AUTO, bind_tx_taskq, CTLFLAG_RDTUN, &hn_bind_tx_taskq, 0, "Bind TX taskqueue to the specified cpu"); static int hn_use_if_start = 0; SYSCTL_INT(_hw_hn, OID_AUTO, use_if_start, CTLFLAG_RDTUN, &hn_use_if_start, 0, "Use if_start TX method"); static int hn_chan_cnt = 0; SYSCTL_INT(_hw_hn, OID_AUTO, chan_cnt, CTLFLAG_RDTUN, &hn_chan_cnt, 0, "# of channels to use; each channel has one RX ring and one TX ring"); static int hn_tx_ring_cnt = 0; SYSCTL_INT(_hw_hn, OID_AUTO, tx_ring_cnt, CTLFLAG_RDTUN, &hn_tx_ring_cnt, 0, "# of TX rings to use"); static int hn_tx_swq_depth = 0; SYSCTL_INT(_hw_hn, OID_AUTO, tx_swq_depth, CTLFLAG_RDTUN, &hn_tx_swq_depth, 0, "Depth of IFQ or BUFRING"); #if __FreeBSD_version >= 1100095 static u_int hn_lro_mbufq_depth = 0; SYSCTL_UINT(_hw_hn, OID_AUTO, lro_mbufq_depth, CTLFLAG_RDTUN, &hn_lro_mbufq_depth, 0, "Depth of LRO mbuf queue"); #endif static u_int hn_cpu_index; /* * Forward declarations */ static void hn_stop(hn_softc_t *sc); static void hn_ifinit_locked(hn_softc_t *sc); static void hn_ifinit(void *xsc); static int hn_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data); static int hn_start_locked(struct hn_tx_ring *txr, int len); static void hn_start(struct ifnet *ifp); static void hn_start_txeof(struct hn_tx_ring *); static int hn_ifmedia_upd(struct ifnet *ifp); static void hn_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr); #if __FreeBSD_version >= 1100099 static int hn_lro_lenlim_sysctl(SYSCTL_HANDLER_ARGS); static int hn_lro_ackcnt_sysctl(SYSCTL_HANDLER_ARGS); #endif static int hn_trust_hcsum_sysctl(SYSCTL_HANDLER_ARGS); static int hn_chim_size_sysctl(SYSCTL_HANDLER_ARGS); static int hn_rx_stat_ulong_sysctl(SYSCTL_HANDLER_ARGS); static int hn_rx_stat_u64_sysctl(SYSCTL_HANDLER_ARGS); static int hn_tx_stat_ulong_sysctl(SYSCTL_HANDLER_ARGS); static int hn_tx_conf_int_sysctl(SYSCTL_HANDLER_ARGS); static int hn_ndis_version_sysctl(SYSCTL_HANDLER_ARGS); static int hn_check_iplen(const struct mbuf *, int); static int hn_create_tx_ring(struct hn_softc *, int); static void hn_destroy_tx_ring(struct hn_tx_ring *); static int hn_create_tx_data(struct hn_softc *, int); static void hn_destroy_tx_data(struct hn_softc *); static void hn_start_taskfunc(void *, int); static void hn_start_txeof_taskfunc(void *, int); static void hn_stop_tx_tasks(struct hn_softc *); static int hn_encap(struct hn_tx_ring *, struct hn_txdesc *, struct mbuf **); static int hn_create_rx_data(struct hn_softc *sc, int); static void hn_destroy_rx_data(struct hn_softc *sc); static void hn_set_chim_size(struct hn_softc *, int); static void hn_channel_attach(struct hn_softc *, struct vmbus_channel *); static void hn_subchan_attach(struct hn_softc *, struct vmbus_channel *); static void hn_subchan_setup(struct hn_softc *); static int hn_transmit(struct ifnet *, struct mbuf *); static void hn_xmit_qflush(struct ifnet *); static int hn_xmit(struct hn_tx_ring *, int); static void hn_xmit_txeof(struct hn_tx_ring *); static void hn_xmit_taskfunc(void *, int); static void hn_xmit_txeof_taskfunc(void *, int); #if __FreeBSD_version >= 1100099 static void hn_set_lro_lenlim(struct hn_softc *sc, int lenlim) { int i; for (i = 0; i < sc->hn_rx_ring_inuse; ++i) sc->hn_rx_ring[i].hn_lro.lro_length_lim = lenlim; } #endif static int hn_get_txswq_depth(const struct hn_tx_ring *txr) { KASSERT(txr->hn_txdesc_cnt > 0, ("tx ring is not setup yet")); if (hn_tx_swq_depth < txr->hn_txdesc_cnt) return txr->hn_txdesc_cnt; return hn_tx_swq_depth; } static int hn_ifmedia_upd(struct ifnet *ifp __unused) { return EOPNOTSUPP; } static void hn_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr) { struct hn_softc *sc = ifp->if_softc; ifmr->ifm_status = IFM_AVALID; ifmr->ifm_active = IFM_ETHER; if (!sc->hn_carrier) { ifmr->ifm_active |= IFM_NONE; return; } ifmr->ifm_status |= IFM_ACTIVE; ifmr->ifm_active |= IFM_10G_T | IFM_FDX; } /* {F8615163-DF3E-46c5-913F-F2D2F965ED0E} */ static const struct hyperv_guid g_net_vsc_device_type = { .hv_guid = {0x63, 0x51, 0x61, 0xF8, 0x3E, 0xDF, 0xc5, 0x46, 0x91, 0x3F, 0xF2, 0xD2, 0xF9, 0x65, 0xED, 0x0E} }; /* * Standard probe entry point. * */ static int netvsc_probe(device_t dev) { if (VMBUS_PROBE_GUID(device_get_parent(dev), dev, &g_net_vsc_device_type) == 0) { device_set_desc(dev, "Hyper-V Network Interface"); return BUS_PROBE_DEFAULT; } return ENXIO; } /* * Standard attach entry point. * * Called when the driver is loaded. It allocates needed resources, * and initializes the "hardware" and software. */ static int netvsc_attach(device_t dev) { struct sysctl_oid_list *child; struct sysctl_ctx_list *ctx; netvsc_device_info device_info; hn_softc_t *sc; int unit = device_get_unit(dev); struct ifnet *ifp = NULL; int error, ring_cnt, tx_ring_cnt; #if __FreeBSD_version >= 1100045 int tso_maxlen; #endif sc = device_get_softc(dev); sc->hn_unit = unit; sc->hn_dev = dev; sc->hn_prichan = vmbus_get_channel(dev); if (hn_tx_taskq == NULL) { sc->hn_tx_taskq = taskqueue_create("hn_tx", M_WAITOK, taskqueue_thread_enqueue, &sc->hn_tx_taskq); if (hn_bind_tx_taskq >= 0) { int cpu = hn_bind_tx_taskq; cpuset_t cpu_set; if (cpu > mp_ncpus - 1) cpu = mp_ncpus - 1; CPU_SETOF(cpu, &cpu_set); taskqueue_start_threads_cpuset(&sc->hn_tx_taskq, 1, PI_NET, &cpu_set, "%s tx", device_get_nameunit(dev)); } else { taskqueue_start_threads(&sc->hn_tx_taskq, 1, PI_NET, "%s tx", device_get_nameunit(dev)); } } else { sc->hn_tx_taskq = hn_tx_taskq; } NV_LOCK_INIT(sc, "NetVSCLock"); ifp = sc->hn_ifp = if_alloc(IFT_ETHER); ifp->if_softc = sc; if_initname(ifp, device_get_name(dev), device_get_unit(dev)); /* * Figure out the # of RX rings (ring_cnt) and the # of TX rings * to use (tx_ring_cnt). * * NOTE: * The # of RX rings to use is same as the # of channels to use. */ ring_cnt = hn_chan_cnt; if (ring_cnt <= 0) { /* Default */ ring_cnt = mp_ncpus; if (ring_cnt > HN_RING_CNT_DEF_MAX) ring_cnt = HN_RING_CNT_DEF_MAX; } else if (ring_cnt > mp_ncpus) { ring_cnt = mp_ncpus; } tx_ring_cnt = hn_tx_ring_cnt; if (tx_ring_cnt <= 0 || tx_ring_cnt > ring_cnt) tx_ring_cnt = ring_cnt; if (hn_use_if_start) { /* ifnet.if_start only needs one TX ring. */ tx_ring_cnt = 1; } /* * Set the leader CPU for channels. */ sc->hn_cpu = atomic_fetchadd_int(&hn_cpu_index, ring_cnt) % mp_ncpus; error = hn_create_tx_data(sc, tx_ring_cnt); if (error) goto failed; error = hn_create_rx_data(sc, ring_cnt); if (error) goto failed; /* * Associate the first TX/RX ring w/ the primary channel. */ hn_channel_attach(sc, sc->hn_prichan); ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST; ifp->if_ioctl = hn_ioctl; ifp->if_init = hn_ifinit; /* needed by hv_rf_on_device_add() code */ ifp->if_mtu = ETHERMTU; if (hn_use_if_start) { int qdepth = hn_get_txswq_depth(&sc->hn_tx_ring[0]); ifp->if_start = hn_start; IFQ_SET_MAXLEN(&ifp->if_snd, qdepth); ifp->if_snd.ifq_drv_maxlen = qdepth - 1; IFQ_SET_READY(&ifp->if_snd); } else { ifp->if_transmit = hn_transmit; ifp->if_qflush = hn_xmit_qflush; } ifmedia_init(&sc->hn_media, 0, hn_ifmedia_upd, hn_ifmedia_sts); ifmedia_add(&sc->hn_media, IFM_ETHER | IFM_AUTO, 0, NULL); ifmedia_set(&sc->hn_media, IFM_ETHER | IFM_AUTO); /* XXX ifmedia_set really should do this for us */ sc->hn_media.ifm_media = sc->hn_media.ifm_cur->ifm_media; /* * Tell upper layers that we support full VLAN capability. */ ifp->if_hdrlen = sizeof(struct ether_vlan_header); ifp->if_capabilities |= IFCAP_VLAN_HWTAGGING | IFCAP_VLAN_MTU | IFCAP_HWCSUM | IFCAP_TSO | IFCAP_LRO; ifp->if_capenable |= IFCAP_VLAN_HWTAGGING | IFCAP_VLAN_MTU | IFCAP_HWCSUM | IFCAP_TSO | IFCAP_LRO; ifp->if_hwassist = sc->hn_tx_ring[0].hn_csum_assist | CSUM_TSO; sc->hn_xact = vmbus_xact_ctx_create(bus_get_dma_tag(dev), HN_XACT_REQ_SIZE, HN_XACT_RESP_SIZE, 0); if (sc->hn_xact == NULL) goto failed; error = hv_rf_on_device_add(sc, &device_info, &ring_cnt, &sc->hn_rx_ring[0]); if (error) goto failed; KASSERT(ring_cnt > 0 && ring_cnt <= sc->hn_rx_ring_inuse, ("invalid channel count %d, should be less than %d", ring_cnt, sc->hn_rx_ring_inuse)); /* * Set the # of TX/RX rings that could be used according to * the # of channels that host offered. */ if (sc->hn_tx_ring_inuse > ring_cnt) sc->hn_tx_ring_inuse = ring_cnt; sc->hn_rx_ring_inuse = ring_cnt; device_printf(dev, "%d TX ring, %d RX ring\n", sc->hn_tx_ring_inuse, sc->hn_rx_ring_inuse); if (sc->hn_rx_ring_inuse > 1) hn_subchan_setup(sc); #if __FreeBSD_version >= 1100099 if (sc->hn_rx_ring_inuse > 1) { /* * Reduce TCP segment aggregation limit for multiple * RX rings to increase ACK timeliness. */ hn_set_lro_lenlim(sc, HN_LRO_LENLIM_MULTIRX_DEF); } #endif if (device_info.link_state == NDIS_MEDIA_STATE_CONNECTED) { sc->hn_carrier = 1; } #if __FreeBSD_version >= 1100045 tso_maxlen = hn_tso_maxlen; if (tso_maxlen <= 0 || tso_maxlen > IP_MAXPACKET) tso_maxlen = IP_MAXPACKET; ifp->if_hw_tsomaxsegcount = HN_TX_DATA_SEGCNT_MAX; ifp->if_hw_tsomaxsegsize = PAGE_SIZE; ifp->if_hw_tsomax = tso_maxlen - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN); #endif ether_ifattach(ifp, device_info.mac_addr); #if __FreeBSD_version >= 1100045 if_printf(ifp, "TSO: %u/%u/%u\n", ifp->if_hw_tsomax, ifp->if_hw_tsomaxsegcount, ifp->if_hw_tsomaxsegsize); #endif hn_set_chim_size(sc, sc->hn_chim_szmax); if (hn_tx_chimney_size > 0 && hn_tx_chimney_size < sc->hn_chim_szmax) hn_set_chim_size(sc, hn_tx_chimney_size); ctx = device_get_sysctl_ctx(dev); child = SYSCTL_CHILDREN(device_get_sysctl_tree(dev)); SYSCTL_ADD_UINT(ctx, child, OID_AUTO, "nvs_version", CTLFLAG_RD, &sc->hn_nvs_ver, 0, "NVS version"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "ndis_version", CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, sc, 0, hn_ndis_version_sysctl, "A", "NDIS version"); return (0); failed: hn_destroy_tx_data(sc); if (ifp != NULL) if_free(ifp); return (error); } /* * Standard detach entry point */ static int netvsc_detach(device_t dev) { struct hn_softc *sc = device_get_softc(dev); if (bootverbose) printf("netvsc_detach\n"); /* * XXXKYS: Need to clean up all our * driver state; this is the driver * unloading. */ /* * XXXKYS: Need to stop outgoing traffic and unregister * the netdevice. */ - hv_rf_on_device_remove(sc, HV_RF_NV_DESTROY_CHANNEL); + hv_rf_on_device_remove(sc); hn_stop_tx_tasks(sc); ifmedia_removeall(&sc->hn_media); hn_destroy_rx_data(sc); hn_destroy_tx_data(sc); if (sc->hn_tx_taskq != hn_tx_taskq) taskqueue_free(sc->hn_tx_taskq); vmbus_xact_ctx_destroy(sc->hn_xact); return (0); } /* * Standard shutdown entry point */ static int netvsc_shutdown(device_t dev) { return (0); } static __inline int hn_txdesc_dmamap_load(struct hn_tx_ring *txr, struct hn_txdesc *txd, struct mbuf **m_head, bus_dma_segment_t *segs, int *nsegs) { struct mbuf *m = *m_head; int error; error = bus_dmamap_load_mbuf_sg(txr->hn_tx_data_dtag, txd->data_dmap, m, segs, nsegs, BUS_DMA_NOWAIT); if (error == EFBIG) { struct mbuf *m_new; m_new = m_collapse(m, M_NOWAIT, HN_TX_DATA_SEGCNT_MAX); if (m_new == NULL) return ENOBUFS; else *m_head = m = m_new; txr->hn_tx_collapsed++; error = bus_dmamap_load_mbuf_sg(txr->hn_tx_data_dtag, txd->data_dmap, m, segs, nsegs, BUS_DMA_NOWAIT); } if (!error) { bus_dmamap_sync(txr->hn_tx_data_dtag, txd->data_dmap, BUS_DMASYNC_PREWRITE); txd->flags |= HN_TXD_FLAG_DMAMAP; } return error; } static __inline void hn_txdesc_dmamap_unload(struct hn_tx_ring *txr, struct hn_txdesc *txd) { if (txd->flags & HN_TXD_FLAG_DMAMAP) { bus_dmamap_sync(txr->hn_tx_data_dtag, txd->data_dmap, BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(txr->hn_tx_data_dtag, txd->data_dmap); txd->flags &= ~HN_TXD_FLAG_DMAMAP; } } static __inline int hn_txdesc_put(struct hn_tx_ring *txr, struct hn_txdesc *txd) { KASSERT((txd->flags & HN_TXD_FLAG_ONLIST) == 0, ("put an onlist txd %#x", txd->flags)); KASSERT(txd->refs > 0, ("invalid txd refs %d", txd->refs)); if (atomic_fetchadd_int(&txd->refs, -1) != 1) return 0; hn_txdesc_dmamap_unload(txr, txd); if (txd->m != NULL) { m_freem(txd->m); txd->m = NULL; } txd->flags |= HN_TXD_FLAG_ONLIST; #ifndef HN_USE_TXDESC_BUFRING mtx_lock_spin(&txr->hn_txlist_spin); KASSERT(txr->hn_txdesc_avail >= 0 && txr->hn_txdesc_avail < txr->hn_txdesc_cnt, ("txdesc_put: invalid txd avail %d", txr->hn_txdesc_avail)); txr->hn_txdesc_avail++; SLIST_INSERT_HEAD(&txr->hn_txlist, txd, link); mtx_unlock_spin(&txr->hn_txlist_spin); #else atomic_add_int(&txr->hn_txdesc_avail, 1); buf_ring_enqueue(txr->hn_txdesc_br, txd); #endif return 1; } static __inline struct hn_txdesc * hn_txdesc_get(struct hn_tx_ring *txr) { struct hn_txdesc *txd; #ifndef HN_USE_TXDESC_BUFRING mtx_lock_spin(&txr->hn_txlist_spin); txd = SLIST_FIRST(&txr->hn_txlist); if (txd != NULL) { KASSERT(txr->hn_txdesc_avail > 0, ("txdesc_get: invalid txd avail %d", txr->hn_txdesc_avail)); txr->hn_txdesc_avail--; SLIST_REMOVE_HEAD(&txr->hn_txlist, link); } mtx_unlock_spin(&txr->hn_txlist_spin); #else txd = buf_ring_dequeue_sc(txr->hn_txdesc_br); #endif if (txd != NULL) { #ifdef HN_USE_TXDESC_BUFRING atomic_subtract_int(&txr->hn_txdesc_avail, 1); #endif KASSERT(txd->m == NULL && txd->refs == 0 && (txd->flags & HN_TXD_FLAG_ONLIST), ("invalid txd")); txd->flags &= ~HN_TXD_FLAG_ONLIST; txd->refs = 1; } return txd; } static __inline void hn_txdesc_hold(struct hn_txdesc *txd) { /* 0->1 transition will never work */ KASSERT(txd->refs > 0, ("invalid refs %d", txd->refs)); atomic_add_int(&txd->refs, 1); } static __inline void hn_txeof(struct hn_tx_ring *txr) { txr->hn_has_txeof = 0; txr->hn_txeof(txr); } static void hn_tx_done(struct hn_send_ctx *sndc, struct hn_softc *sc, struct vmbus_channel *chan, const void *data __unused, int dlen __unused) { struct hn_txdesc *txd = sndc->hn_cbarg; struct hn_tx_ring *txr; if (sndc->hn_chim_idx != HN_NVS_CHIM_IDX_INVALID) hn_chim_free(sc, sndc->hn_chim_idx); txr = txd->txr; KASSERT(txr->hn_chan == chan, ("channel mismatch, on chan%u, should be chan%u", vmbus_chan_subidx(chan), vmbus_chan_subidx(txr->hn_chan))); txr->hn_has_txeof = 1; hn_txdesc_put(txr, txd); ++txr->hn_txdone_cnt; if (txr->hn_txdone_cnt >= HN_EARLY_TXEOF_THRESH) { txr->hn_txdone_cnt = 0; if (txr->hn_oactive) hn_txeof(txr); } } void netvsc_channel_rollup(struct hn_rx_ring *rxr, struct hn_tx_ring *txr) { #if defined(INET) || defined(INET6) tcp_lro_flush_all(&rxr->hn_lro); #endif /* * NOTE: * 'txr' could be NULL, if multiple channels and * ifnet.if_start method are enabled. */ if (txr == NULL || !txr->hn_has_txeof) return; txr->hn_txdone_cnt = 0; hn_txeof(txr); } /* * NOTE: * If this function fails, then both txd and m_head0 will be freed. */ static int hn_encap(struct hn_tx_ring *txr, struct hn_txdesc *txd, struct mbuf **m_head0) { bus_dma_segment_t segs[HN_TX_DATA_SEGCNT_MAX]; int error, nsegs, i; struct mbuf *m_head = *m_head0; rndis_msg *rndis_mesg; rndis_packet *rndis_pkt; rndis_per_packet_info *rppi; struct rndis_hash_value *hash_value; uint32_t rndis_msg_size, tot_data_buf_len, send_buf_section_idx; int send_buf_section_size; tot_data_buf_len = m_head->m_pkthdr.len; /* * extension points to the area reserved for the * rndis_filter_packet, which is placed just after * the netvsc_packet (and rppi struct, if present; * length is updated later). */ rndis_mesg = txd->rndis_msg; /* XXX not necessary */ memset(rndis_mesg, 0, HN_RNDIS_MSG_LEN); rndis_mesg->ndis_msg_type = REMOTE_NDIS_PACKET_MSG; rndis_pkt = &rndis_mesg->msg.packet; rndis_pkt->data_offset = sizeof(rndis_packet); rndis_pkt->data_length = tot_data_buf_len; rndis_pkt->per_pkt_info_offset = sizeof(rndis_packet); rndis_msg_size = RNDIS_MESSAGE_SIZE(rndis_packet); /* * Set the hash value for this packet, so that the host could * dispatch the TX done event for this packet back to this TX * ring's channel. */ rndis_msg_size += RNDIS_HASHVAL_PPI_SIZE; rppi = hv_set_rppi_data(rndis_mesg, RNDIS_HASHVAL_PPI_SIZE, nbl_hash_value); hash_value = (struct rndis_hash_value *)((uint8_t *)rppi + rppi->per_packet_info_offset); hash_value->hash_value = txr->hn_tx_idx; if (m_head->m_flags & M_VLANTAG) { ndis_8021q_info *rppi_vlan_info; rndis_msg_size += RNDIS_VLAN_PPI_SIZE; rppi = hv_set_rppi_data(rndis_mesg, RNDIS_VLAN_PPI_SIZE, ieee_8021q_info); rppi_vlan_info = (ndis_8021q_info *)((uint8_t *)rppi + rppi->per_packet_info_offset); rppi_vlan_info->u1.s1.vlan_id = m_head->m_pkthdr.ether_vtag & 0xfff; } if (m_head->m_pkthdr.csum_flags & CSUM_TSO) { rndis_tcp_tso_info *tso_info; struct ether_vlan_header *eh; int ether_len; /* * XXX need m_pullup and use mtodo */ eh = mtod(m_head, struct ether_vlan_header*); if (eh->evl_encap_proto == htons(ETHERTYPE_VLAN)) ether_len = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN; else ether_len = ETHER_HDR_LEN; rndis_msg_size += RNDIS_TSO_PPI_SIZE; rppi = hv_set_rppi_data(rndis_mesg, RNDIS_TSO_PPI_SIZE, tcp_large_send_info); tso_info = (rndis_tcp_tso_info *)((uint8_t *)rppi + rppi->per_packet_info_offset); tso_info->lso_v2_xmit.type = RNDIS_TCP_LARGE_SEND_OFFLOAD_V2_TYPE; #ifdef INET if (m_head->m_pkthdr.csum_flags & CSUM_IP_TSO) { struct ip *ip = (struct ip *)(m_head->m_data + ether_len); unsigned long iph_len = ip->ip_hl << 2; struct tcphdr *th = (struct tcphdr *)((caddr_t)ip + iph_len); tso_info->lso_v2_xmit.ip_version = RNDIS_TCP_LARGE_SEND_OFFLOAD_IPV4; ip->ip_len = 0; ip->ip_sum = 0; th->th_sum = in_pseudo(ip->ip_src.s_addr, ip->ip_dst.s_addr, htons(IPPROTO_TCP)); } #endif #if defined(INET6) && defined(INET) else #endif #ifdef INET6 { struct ip6_hdr *ip6 = (struct ip6_hdr *) (m_head->m_data + ether_len); struct tcphdr *th = (struct tcphdr *)(ip6 + 1); tso_info->lso_v2_xmit.ip_version = RNDIS_TCP_LARGE_SEND_OFFLOAD_IPV6; ip6->ip6_plen = 0; th->th_sum = in6_cksum_pseudo(ip6, 0, IPPROTO_TCP, 0); } #endif tso_info->lso_v2_xmit.tcp_header_offset = 0; tso_info->lso_v2_xmit.mss = m_head->m_pkthdr.tso_segsz; } else if (m_head->m_pkthdr.csum_flags & txr->hn_csum_assist) { rndis_tcp_ip_csum_info *csum_info; rndis_msg_size += RNDIS_CSUM_PPI_SIZE; rppi = hv_set_rppi_data(rndis_mesg, RNDIS_CSUM_PPI_SIZE, tcpip_chksum_info); csum_info = (rndis_tcp_ip_csum_info *)((uint8_t *)rppi + rppi->per_packet_info_offset); csum_info->xmit.is_ipv4 = 1; if (m_head->m_pkthdr.csum_flags & CSUM_IP) csum_info->xmit.ip_header_csum = 1; if (m_head->m_pkthdr.csum_flags & CSUM_TCP) { csum_info->xmit.tcp_csum = 1; csum_info->xmit.tcp_header_offset = 0; } else if (m_head->m_pkthdr.csum_flags & CSUM_UDP) { csum_info->xmit.udp_csum = 1; } } rndis_mesg->msg_len = tot_data_buf_len + rndis_msg_size; tot_data_buf_len = rndis_mesg->msg_len; /* * Chimney send, if the packet could fit into one chimney buffer. */ if (tot_data_buf_len < txr->hn_chim_size) { txr->hn_tx_chimney_tried++; send_buf_section_idx = hn_chim_alloc(txr->hn_sc); if (send_buf_section_idx != HN_NVS_CHIM_IDX_INVALID) { uint8_t *dest = txr->hn_sc->hn_chim + (send_buf_section_idx * txr->hn_sc->hn_chim_szmax); memcpy(dest, rndis_mesg, rndis_msg_size); dest += rndis_msg_size; m_copydata(m_head, 0, m_head->m_pkthdr.len, dest); send_buf_section_size = tot_data_buf_len; txr->hn_gpa_cnt = 0; txr->hn_tx_chimney++; goto done; } } error = hn_txdesc_dmamap_load(txr, txd, &m_head, segs, &nsegs); if (error) { int freed; /* * This mbuf is not linked w/ the txd yet, so free it now. */ m_freem(m_head); *m_head0 = NULL; freed = hn_txdesc_put(txr, txd); KASSERT(freed != 0, ("fail to free txd upon txdma error")); txr->hn_txdma_failed++; if_inc_counter(txr->hn_sc->hn_ifp, IFCOUNTER_OERRORS, 1); return error; } *m_head0 = m_head; - txr->hn_gpa_cnt = nsegs + HV_RF_NUM_TX_RESERVED_PAGE_BUFS; + /* +1 RNDIS packet message */ + txr->hn_gpa_cnt = nsegs + 1; /* send packet with page buffer */ txr->hn_gpa[0].gpa_page = atop(txd->rndis_msg_paddr); txr->hn_gpa[0].gpa_ofs = txd->rndis_msg_paddr & PAGE_MASK; txr->hn_gpa[0].gpa_len = rndis_msg_size; /* - * Fill the page buffers with mbuf info starting at index - * HV_RF_NUM_TX_RESERVED_PAGE_BUFS. + * Fill the page buffers with mbuf info after the page + * buffer for RNDIS packet message. */ for (i = 0; i < nsegs; ++i) { - struct vmbus_gpa *gpa = &txr->hn_gpa[ - i + HV_RF_NUM_TX_RESERVED_PAGE_BUFS]; + struct vmbus_gpa *gpa = &txr->hn_gpa[i + 1]; gpa->gpa_page = atop(segs[i].ds_addr); gpa->gpa_ofs = segs[i].ds_addr & PAGE_MASK; gpa->gpa_len = segs[i].ds_len; } send_buf_section_idx = HN_NVS_CHIM_IDX_INVALID; send_buf_section_size = 0; done: txd->m = m_head; /* Set the completion routine */ hn_send_ctx_init(&txd->send_ctx, hn_tx_done, txd, send_buf_section_idx, send_buf_section_size); return 0; } /* * NOTE: * If this function fails, then txd will be freed, but the mbuf * associated w/ the txd will _not_ be freed. */ static int hn_send_pkt(struct ifnet *ifp, struct hn_tx_ring *txr, struct hn_txdesc *txd) { int error, send_failed = 0; again: /* * Make sure that txd is not freed before ETHER_BPF_MTAP. */ hn_txdesc_hold(txd); error = hv_nv_on_send(txr->hn_chan, HN_NVS_RNDIS_MTYPE_DATA, &txd->send_ctx, txr->hn_gpa, txr->hn_gpa_cnt); if (!error) { ETHER_BPF_MTAP(ifp, txd->m); if_inc_counter(ifp, IFCOUNTER_OPACKETS, 1); if (!hn_use_if_start) { if_inc_counter(ifp, IFCOUNTER_OBYTES, txd->m->m_pkthdr.len); if (txd->m->m_flags & M_MCAST) if_inc_counter(ifp, IFCOUNTER_OMCASTS, 1); } txr->hn_pkts++; } hn_txdesc_put(txr, txd); if (__predict_false(error)) { int freed; /* * This should "really rarely" happen. * * XXX Too many RX to be acked or too many sideband * commands to run? Ask netvsc_channel_rollup() * to kick start later. */ txr->hn_has_txeof = 1; if (!send_failed) { txr->hn_send_failed++; send_failed = 1; /* * Try sending again after set hn_has_txeof; * in case that we missed the last * netvsc_channel_rollup(). */ goto again; } if_printf(ifp, "send failed\n"); /* * Caller will perform further processing on the * associated mbuf, so don't free it in hn_txdesc_put(); * only unload it from the DMA map in hn_txdesc_put(), * if it was loaded. */ txd->m = NULL; freed = hn_txdesc_put(txr, txd); KASSERT(freed != 0, ("fail to free txd upon send error")); txr->hn_send_failed++; } return error; } /* * Start a transmit of one or more packets */ static int hn_start_locked(struct hn_tx_ring *txr, int len) { struct hn_softc *sc = txr->hn_sc; struct ifnet *ifp = sc->hn_ifp; KASSERT(hn_use_if_start, ("hn_start_locked is called, when if_start is disabled")); KASSERT(txr == &sc->hn_tx_ring[0], ("not the first TX ring")); mtx_assert(&txr->hn_tx_lock, MA_OWNED); if ((ifp->if_drv_flags & (IFF_DRV_RUNNING | IFF_DRV_OACTIVE)) != IFF_DRV_RUNNING) return 0; while (!IFQ_DRV_IS_EMPTY(&ifp->if_snd)) { struct hn_txdesc *txd; struct mbuf *m_head; int error; IFQ_DRV_DEQUEUE(&ifp->if_snd, m_head); if (m_head == NULL) break; if (len > 0 && m_head->m_pkthdr.len > len) { /* * This sending could be time consuming; let callers * dispatch this packet sending (and sending of any * following up packets) to tx taskqueue. */ IFQ_DRV_PREPEND(&ifp->if_snd, m_head); return 1; } txd = hn_txdesc_get(txr); if (txd == NULL) { txr->hn_no_txdescs++; IFQ_DRV_PREPEND(&ifp->if_snd, m_head); atomic_set_int(&ifp->if_drv_flags, IFF_DRV_OACTIVE); break; } error = hn_encap(txr, txd, &m_head); if (error) { /* Both txd and m_head are freed */ continue; } error = hn_send_pkt(ifp, txr, txd); if (__predict_false(error)) { /* txd is freed, but m_head is not */ IFQ_DRV_PREPEND(&ifp->if_snd, m_head); atomic_set_int(&ifp->if_drv_flags, IFF_DRV_OACTIVE); break; } } return 0; } /* * Link up/down notification */ void netvsc_linkstatus_callback(struct hn_softc *sc, uint32_t status) { if (status == 1) { sc->hn_carrier = 1; } else { sc->hn_carrier = 0; } } /* * Append the specified data to the indicated mbuf chain, * Extend the mbuf chain if the new data does not fit in * existing space. * * This is a minor rewrite of m_append() from sys/kern/uipc_mbuf.c. * There should be an equivalent in the kernel mbuf code, * but there does not appear to be one yet. * * Differs from m_append() in that additional mbufs are * allocated with cluster size MJUMPAGESIZE, and filled * accordingly. * * Return 1 if able to complete the job; otherwise 0. */ static int hv_m_append(struct mbuf *m0, int len, c_caddr_t cp) { struct mbuf *m, *n; int remainder, space; for (m = m0; m->m_next != NULL; m = m->m_next) ; remainder = len; space = M_TRAILINGSPACE(m); if (space > 0) { /* * Copy into available space. */ if (space > remainder) space = remainder; bcopy(cp, mtod(m, caddr_t) + m->m_len, space); m->m_len += space; cp += space; remainder -= space; } while (remainder > 0) { /* * Allocate a new mbuf; could check space * and allocate a cluster instead. */ n = m_getjcl(M_NOWAIT, m->m_type, 0, MJUMPAGESIZE); if (n == NULL) break; n->m_len = min(MJUMPAGESIZE, remainder); bcopy(cp, mtod(n, caddr_t), n->m_len); cp += n->m_len; remainder -= n->m_len; m->m_next = n; m = n; } if (m0->m_flags & M_PKTHDR) m0->m_pkthdr.len += len - remainder; return (remainder == 0); } #if defined(INET) || defined(INET6) static __inline int hn_lro_rx(struct lro_ctrl *lc, struct mbuf *m) { #if __FreeBSD_version >= 1100095 if (hn_lro_mbufq_depth) { tcp_lro_queue_mbuf(lc, m); return 0; } #endif return tcp_lro_rx(lc, m, 0); } #endif /* * Called when we receive a data packet from the "wire" on the * specified device * * Note: This is no longer used as a callback */ int netvsc_recv(struct hn_rx_ring *rxr, const void *data, int dlen, const struct hn_recvinfo *info) { struct ifnet *ifp = rxr->hn_ifp; struct mbuf *m_new; int size, do_lro = 0, do_csum = 1; int hash_type = M_HASHTYPE_OPAQUE_HASH; if (!(ifp->if_drv_flags & IFF_DRV_RUNNING)) return (0); /* * Bail out if packet contains more data than configured MTU. */ if (dlen > (ifp->if_mtu + ETHER_HDR_LEN)) { return (0); } else if (dlen <= MHLEN) { m_new = m_gethdr(M_NOWAIT, MT_DATA); if (m_new == NULL) { if_inc_counter(ifp, IFCOUNTER_IQDROPS, 1); return (0); } memcpy(mtod(m_new, void *), data, dlen); m_new->m_pkthdr.len = m_new->m_len = dlen; rxr->hn_small_pkts++; } else { /* * Get an mbuf with a cluster. For packets 2K or less, * get a standard 2K cluster. For anything larger, get a * 4K cluster. Any buffers larger than 4K can cause problems * if looped around to the Hyper-V TX channel, so avoid them. */ size = MCLBYTES; if (dlen > MCLBYTES) { /* 4096 */ size = MJUMPAGESIZE; } m_new = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, size); if (m_new == NULL) { if_inc_counter(ifp, IFCOUNTER_IQDROPS, 1); return (0); } hv_m_append(m_new, dlen, data); } m_new->m_pkthdr.rcvif = ifp; if (__predict_false((ifp->if_capenable & IFCAP_RXCSUM) == 0)) do_csum = 0; /* receive side checksum offload */ if (info->csum_info != NULL) { /* IP csum offload */ if (info->csum_info->receive.ip_csum_succeeded && do_csum) { m_new->m_pkthdr.csum_flags |= (CSUM_IP_CHECKED | CSUM_IP_VALID); rxr->hn_csum_ip++; } /* TCP/UDP csum offload */ if ((info->csum_info->receive.tcp_csum_succeeded || info->csum_info->receive.udp_csum_succeeded) && do_csum) { m_new->m_pkthdr.csum_flags |= (CSUM_DATA_VALID | CSUM_PSEUDO_HDR); m_new->m_pkthdr.csum_data = 0xffff; if (info->csum_info->receive.tcp_csum_succeeded) rxr->hn_csum_tcp++; else rxr->hn_csum_udp++; } if (info->csum_info->receive.ip_csum_succeeded && info->csum_info->receive.tcp_csum_succeeded) do_lro = 1; } else { const struct ether_header *eh; uint16_t etype; int hoff; hoff = sizeof(*eh); if (m_new->m_len < hoff) goto skip; eh = mtod(m_new, struct ether_header *); etype = ntohs(eh->ether_type); if (etype == ETHERTYPE_VLAN) { const struct ether_vlan_header *evl; hoff = sizeof(*evl); if (m_new->m_len < hoff) goto skip; evl = mtod(m_new, struct ether_vlan_header *); etype = ntohs(evl->evl_proto); } if (etype == ETHERTYPE_IP) { int pr; pr = hn_check_iplen(m_new, hoff); if (pr == IPPROTO_TCP) { if (do_csum && (rxr->hn_trust_hcsum & HN_TRUST_HCSUM_TCP)) { rxr->hn_csum_trusted++; m_new->m_pkthdr.csum_flags |= (CSUM_IP_CHECKED | CSUM_IP_VALID | CSUM_DATA_VALID | CSUM_PSEUDO_HDR); m_new->m_pkthdr.csum_data = 0xffff; } do_lro = 1; } else if (pr == IPPROTO_UDP) { if (do_csum && (rxr->hn_trust_hcsum & HN_TRUST_HCSUM_UDP)) { rxr->hn_csum_trusted++; m_new->m_pkthdr.csum_flags |= (CSUM_IP_CHECKED | CSUM_IP_VALID | CSUM_DATA_VALID | CSUM_PSEUDO_HDR); m_new->m_pkthdr.csum_data = 0xffff; } } else if (pr != IPPROTO_DONE && do_csum && (rxr->hn_trust_hcsum & HN_TRUST_HCSUM_IP)) { rxr->hn_csum_trusted++; m_new->m_pkthdr.csum_flags |= (CSUM_IP_CHECKED | CSUM_IP_VALID); } } } skip: if (info->vlan_info != NULL) { m_new->m_pkthdr.ether_vtag = info->vlan_info->u1.s1.vlan_id; m_new->m_flags |= M_VLANTAG; } if (info->hash_info != NULL && info->hash_value != NULL) { rxr->hn_rss_pkts++; m_new->m_pkthdr.flowid = info->hash_value->hash_value; if ((info->hash_info->hash_info & NDIS_HASH_FUNCTION_MASK) == NDIS_HASH_FUNCTION_TOEPLITZ) { uint32_t type = (info->hash_info->hash_info & NDIS_HASH_TYPE_MASK); switch (type) { case NDIS_HASH_IPV4: hash_type = M_HASHTYPE_RSS_IPV4; break; case NDIS_HASH_TCP_IPV4: hash_type = M_HASHTYPE_RSS_TCP_IPV4; break; case NDIS_HASH_IPV6: hash_type = M_HASHTYPE_RSS_IPV6; break; case NDIS_HASH_IPV6_EX: hash_type = M_HASHTYPE_RSS_IPV6_EX; break; case NDIS_HASH_TCP_IPV6: hash_type = M_HASHTYPE_RSS_TCP_IPV6; break; case NDIS_HASH_TCP_IPV6_EX: hash_type = M_HASHTYPE_RSS_TCP_IPV6_EX; break; } } } else { if (info->hash_value != NULL) { m_new->m_pkthdr.flowid = info->hash_value->hash_value; } else { m_new->m_pkthdr.flowid = rxr->hn_rx_idx; hash_type = M_HASHTYPE_OPAQUE; } } M_HASHTYPE_SET(m_new, hash_type); /* * Note: Moved RX completion back to hv_nv_on_receive() so all * messages (not just data messages) will trigger a response. */ if_inc_counter(ifp, IFCOUNTER_IPACKETS, 1); rxr->hn_pkts++; if ((ifp->if_capenable & IFCAP_LRO) && do_lro) { #if defined(INET) || defined(INET6) struct lro_ctrl *lro = &rxr->hn_lro; if (lro->lro_cnt) { rxr->hn_lro_tried++; if (hn_lro_rx(lro, m_new) == 0) { /* DONE! */ return 0; } } #endif } /* We're not holding the lock here, so don't release it */ (*ifp->if_input)(ifp, m_new); return (0); } /* * Rules for using sc->temp_unusable: * 1. sc->temp_unusable can only be read or written while holding NV_LOCK() * 2. code reading sc->temp_unusable under NV_LOCK(), and finding * sc->temp_unusable set, must release NV_LOCK() and exit * 3. to retain exclusive control of the interface, * sc->temp_unusable must be set by code before releasing NV_LOCK() * 4. only code setting sc->temp_unusable can clear sc->temp_unusable * 5. code setting sc->temp_unusable must eventually clear sc->temp_unusable */ /* * Standard ioctl entry point. Called when the user wants to configure * the interface. */ static int hn_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data) { hn_softc_t *sc = ifp->if_softc; struct ifreq *ifr = (struct ifreq *)data; #ifdef INET struct ifaddr *ifa = (struct ifaddr *)data; #endif netvsc_device_info device_info; int mask, error = 0, ring_cnt; int retry_cnt = 500; switch(cmd) { case SIOCSIFADDR: #ifdef INET if (ifa->ifa_addr->sa_family == AF_INET) { ifp->if_flags |= IFF_UP; if (!(ifp->if_drv_flags & IFF_DRV_RUNNING)) hn_ifinit(sc); arp_ifinit(ifp, ifa); } else #endif error = ether_ioctl(ifp, cmd, data); break; case SIOCSIFMTU: /* Check MTU value change */ if (ifp->if_mtu == ifr->ifr_mtu) break; if (ifr->ifr_mtu > NETVSC_MAX_CONFIGURABLE_MTU) { error = EINVAL; break; } /* Obtain and record requested MTU */ ifp->if_mtu = ifr->ifr_mtu; #if __FreeBSD_version >= 1100099 /* * Make sure that LRO aggregation length limit is still * valid, after the MTU change. */ NV_LOCK(sc); if (sc->hn_rx_ring[0].hn_lro.lro_length_lim < HN_LRO_LENLIM_MIN(ifp)) hn_set_lro_lenlim(sc, HN_LRO_LENLIM_MIN(ifp)); NV_UNLOCK(sc); #endif do { NV_LOCK(sc); if (!sc->temp_unusable) { sc->temp_unusable = TRUE; retry_cnt = -1; } NV_UNLOCK(sc); if (retry_cnt > 0) { retry_cnt--; DELAY(5 * 1000); } } while (retry_cnt > 0); if (retry_cnt == 0) { error = EINVAL; break; } /* We must remove and add back the device to cause the new * MTU to take effect. This includes tearing down, but not * deleting the channel, then bringing it back up. */ - error = hv_rf_on_device_remove(sc, HV_RF_NV_RETAIN_CHANNEL); + error = hv_rf_on_device_remove(sc); if (error) { NV_LOCK(sc); sc->temp_unusable = FALSE; NV_UNLOCK(sc); break; } /* Wait for subchannels to be destroyed */ vmbus_subchan_drain(sc->hn_prichan); ring_cnt = sc->hn_rx_ring_inuse; error = hv_rf_on_device_add(sc, &device_info, &ring_cnt, &sc->hn_rx_ring[0]); if (error) { NV_LOCK(sc); sc->temp_unusable = FALSE; NV_UNLOCK(sc); break; } /* # of channels can _not_ be changed */ KASSERT(sc->hn_rx_ring_inuse == ring_cnt, ("RX ring count %d and channel count %u mismatch", sc->hn_rx_ring_cnt, ring_cnt)); if (sc->hn_rx_ring_inuse > 1) { int r; /* * Skip the rings on primary channel; they are * handled by the hv_rf_on_device_add() above. */ for (r = 1; r < sc->hn_rx_ring_cnt; ++r) { sc->hn_rx_ring[r].hn_rx_flags &= ~HN_RX_FLAG_ATTACHED; } for (r = 1; r < sc->hn_tx_ring_cnt; ++r) { sc->hn_tx_ring[r].hn_tx_flags &= ~HN_TX_FLAG_ATTACHED; } hn_subchan_setup(sc); } if (sc->hn_tx_ring[0].hn_chim_size > sc->hn_chim_szmax) hn_set_chim_size(sc, sc->hn_chim_szmax); hn_ifinit_locked(sc); NV_LOCK(sc); sc->temp_unusable = FALSE; NV_UNLOCK(sc); break; case SIOCSIFFLAGS: do { NV_LOCK(sc); if (!sc->temp_unusable) { sc->temp_unusable = TRUE; retry_cnt = -1; } NV_UNLOCK(sc); if (retry_cnt > 0) { retry_cnt--; DELAY(5 * 1000); } } while (retry_cnt > 0); if (retry_cnt == 0) { error = EINVAL; break; } if (ifp->if_flags & IFF_UP) { /* * If only the state of the PROMISC flag changed, * then just use the 'set promisc mode' command * instead of reinitializing the entire NIC. Doing * a full re-init means reloading the firmware and * waiting for it to start up, which may take a * second or two. */ #ifdef notyet /* Fixme: Promiscuous mode? */ if (ifp->if_drv_flags & IFF_DRV_RUNNING && ifp->if_flags & IFF_PROMISC && !(sc->hn_if_flags & IFF_PROMISC)) { /* do something here for Hyper-V */ } else if (ifp->if_drv_flags & IFF_DRV_RUNNING && !(ifp->if_flags & IFF_PROMISC) && sc->hn_if_flags & IFF_PROMISC) { /* do something here for Hyper-V */ } else #endif hn_ifinit_locked(sc); } else { if (ifp->if_drv_flags & IFF_DRV_RUNNING) { hn_stop(sc); } } NV_LOCK(sc); sc->temp_unusable = FALSE; NV_UNLOCK(sc); sc->hn_if_flags = ifp->if_flags; error = 0; break; case SIOCSIFCAP: NV_LOCK(sc); mask = ifr->ifr_reqcap ^ ifp->if_capenable; if (mask & IFCAP_TXCSUM) { ifp->if_capenable ^= IFCAP_TXCSUM; if (ifp->if_capenable & IFCAP_TXCSUM) { ifp->if_hwassist |= sc->hn_tx_ring[0].hn_csum_assist; } else { ifp->if_hwassist &= ~sc->hn_tx_ring[0].hn_csum_assist; } } if (mask & IFCAP_RXCSUM) ifp->if_capenable ^= IFCAP_RXCSUM; if (mask & IFCAP_LRO) ifp->if_capenable ^= IFCAP_LRO; if (mask & IFCAP_TSO4) { ifp->if_capenable ^= IFCAP_TSO4; if (ifp->if_capenable & IFCAP_TSO4) ifp->if_hwassist |= CSUM_IP_TSO; else ifp->if_hwassist &= ~CSUM_IP_TSO; } if (mask & IFCAP_TSO6) { ifp->if_capenable ^= IFCAP_TSO6; if (ifp->if_capenable & IFCAP_TSO6) ifp->if_hwassist |= CSUM_IP6_TSO; else ifp->if_hwassist &= ~CSUM_IP6_TSO; } NV_UNLOCK(sc); error = 0; break; case SIOCADDMULTI: case SIOCDELMULTI: #ifdef notyet /* Fixme: Multicast mode? */ if (ifp->if_drv_flags & IFF_DRV_RUNNING) { NV_LOCK(sc); netvsc_setmulti(sc); NV_UNLOCK(sc); error = 0; } #endif error = EINVAL; break; case SIOCSIFMEDIA: case SIOCGIFMEDIA: error = ifmedia_ioctl(ifp, ifr, &sc->hn_media, cmd); break; default: error = ether_ioctl(ifp, cmd, data); break; } return (error); } /* * */ static void hn_stop(hn_softc_t *sc) { struct ifnet *ifp; int ret, i; ifp = sc->hn_ifp; if (bootverbose) printf(" Closing Device ...\n"); atomic_clear_int(&ifp->if_drv_flags, (IFF_DRV_RUNNING | IFF_DRV_OACTIVE)); for (i = 0; i < sc->hn_tx_ring_inuse; ++i) sc->hn_tx_ring[i].hn_oactive = 0; if_link_state_change(ifp, LINK_STATE_DOWN); sc->hn_initdone = 0; ret = hv_rf_on_close(sc); } /* * FreeBSD transmit entry point */ static void hn_start(struct ifnet *ifp) { struct hn_softc *sc = ifp->if_softc; struct hn_tx_ring *txr = &sc->hn_tx_ring[0]; if (txr->hn_sched_tx) goto do_sched; if (mtx_trylock(&txr->hn_tx_lock)) { int sched; sched = hn_start_locked(txr, txr->hn_direct_tx_size); mtx_unlock(&txr->hn_tx_lock); if (!sched) return; } do_sched: taskqueue_enqueue(txr->hn_tx_taskq, &txr->hn_tx_task); } static void hn_start_txeof(struct hn_tx_ring *txr) { struct hn_softc *sc = txr->hn_sc; struct ifnet *ifp = sc->hn_ifp; KASSERT(txr == &sc->hn_tx_ring[0], ("not the first TX ring")); if (txr->hn_sched_tx) goto do_sched; if (mtx_trylock(&txr->hn_tx_lock)) { int sched; atomic_clear_int(&ifp->if_drv_flags, IFF_DRV_OACTIVE); sched = hn_start_locked(txr, txr->hn_direct_tx_size); mtx_unlock(&txr->hn_tx_lock); if (sched) { taskqueue_enqueue(txr->hn_tx_taskq, &txr->hn_tx_task); } } else { do_sched: /* * Release the OACTIVE earlier, with the hope, that * others could catch up. The task will clear the * flag again with the hn_tx_lock to avoid possible * races. */ atomic_clear_int(&ifp->if_drv_flags, IFF_DRV_OACTIVE); taskqueue_enqueue(txr->hn_tx_taskq, &txr->hn_txeof_task); } } /* * */ static void hn_ifinit_locked(hn_softc_t *sc) { struct ifnet *ifp; int ret, i; ifp = sc->hn_ifp; if (ifp->if_drv_flags & IFF_DRV_RUNNING) { return; } hv_promisc_mode = 1; ret = hv_rf_on_open(sc); if (ret != 0) { return; } else { sc->hn_initdone = 1; } atomic_clear_int(&ifp->if_drv_flags, IFF_DRV_OACTIVE); for (i = 0; i < sc->hn_tx_ring_inuse; ++i) sc->hn_tx_ring[i].hn_oactive = 0; atomic_set_int(&ifp->if_drv_flags, IFF_DRV_RUNNING); if_link_state_change(ifp, LINK_STATE_UP); } /* * */ static void hn_ifinit(void *xsc) { hn_softc_t *sc = xsc; NV_LOCK(sc); if (sc->temp_unusable) { NV_UNLOCK(sc); return; } sc->temp_unusable = TRUE; NV_UNLOCK(sc); hn_ifinit_locked(sc); NV_LOCK(sc); sc->temp_unusable = FALSE; NV_UNLOCK(sc); } #ifdef LATER /* * */ static void hn_watchdog(struct ifnet *ifp) { hn_softc_t *sc; sc = ifp->if_softc; printf("hn%d: watchdog timeout -- resetting\n", sc->hn_unit); hn_ifinit(sc); /*???*/ if_inc_counter(ifp, IFCOUNTER_OERRORS, 1); } #endif #if __FreeBSD_version >= 1100099 static int hn_lro_lenlim_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; unsigned int lenlim; int error; lenlim = sc->hn_rx_ring[0].hn_lro.lro_length_lim; error = sysctl_handle_int(oidp, &lenlim, 0, req); if (error || req->newptr == NULL) return error; if (lenlim < HN_LRO_LENLIM_MIN(sc->hn_ifp) || lenlim > TCP_LRO_LENGTH_MAX) return EINVAL; NV_LOCK(sc); hn_set_lro_lenlim(sc, lenlim); NV_UNLOCK(sc); return 0; } static int hn_lro_ackcnt_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; int ackcnt, error, i; /* * lro_ackcnt_lim is append count limit, * +1 to turn it into aggregation limit. */ ackcnt = sc->hn_rx_ring[0].hn_lro.lro_ackcnt_lim + 1; error = sysctl_handle_int(oidp, &ackcnt, 0, req); if (error || req->newptr == NULL) return error; if (ackcnt < 2 || ackcnt > (TCP_LRO_ACKCNT_MAX + 1)) return EINVAL; /* * Convert aggregation limit back to append * count limit. */ --ackcnt; NV_LOCK(sc); for (i = 0; i < sc->hn_rx_ring_inuse; ++i) sc->hn_rx_ring[i].hn_lro.lro_ackcnt_lim = ackcnt; NV_UNLOCK(sc); return 0; } #endif static int hn_trust_hcsum_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; int hcsum = arg2; int on, error, i; on = 0; if (sc->hn_rx_ring[0].hn_trust_hcsum & hcsum) on = 1; error = sysctl_handle_int(oidp, &on, 0, req); if (error || req->newptr == NULL) return error; NV_LOCK(sc); for (i = 0; i < sc->hn_rx_ring_inuse; ++i) { struct hn_rx_ring *rxr = &sc->hn_rx_ring[i]; if (on) rxr->hn_trust_hcsum |= hcsum; else rxr->hn_trust_hcsum &= ~hcsum; } NV_UNLOCK(sc); return 0; } static int hn_chim_size_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; int chim_size, error; chim_size = sc->hn_tx_ring[0].hn_chim_size; error = sysctl_handle_int(oidp, &chim_size, 0, req); if (error || req->newptr == NULL) return error; if (chim_size > sc->hn_chim_szmax || chim_size <= 0) return EINVAL; hn_set_chim_size(sc, chim_size); return 0; } static int hn_rx_stat_ulong_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; int ofs = arg2, i, error; struct hn_rx_ring *rxr; u_long stat; stat = 0; for (i = 0; i < sc->hn_rx_ring_inuse; ++i) { rxr = &sc->hn_rx_ring[i]; stat += *((u_long *)((uint8_t *)rxr + ofs)); } error = sysctl_handle_long(oidp, &stat, 0, req); if (error || req->newptr == NULL) return error; /* Zero out this stat. */ for (i = 0; i < sc->hn_rx_ring_inuse; ++i) { rxr = &sc->hn_rx_ring[i]; *((u_long *)((uint8_t *)rxr + ofs)) = 0; } return 0; } static int hn_rx_stat_u64_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; int ofs = arg2, i, error; struct hn_rx_ring *rxr; uint64_t stat; stat = 0; for (i = 0; i < sc->hn_rx_ring_inuse; ++i) { rxr = &sc->hn_rx_ring[i]; stat += *((uint64_t *)((uint8_t *)rxr + ofs)); } error = sysctl_handle_64(oidp, &stat, 0, req); if (error || req->newptr == NULL) return error; /* Zero out this stat. */ for (i = 0; i < sc->hn_rx_ring_inuse; ++i) { rxr = &sc->hn_rx_ring[i]; *((uint64_t *)((uint8_t *)rxr + ofs)) = 0; } return 0; } static int hn_tx_stat_ulong_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; int ofs = arg2, i, error; struct hn_tx_ring *txr; u_long stat; stat = 0; for (i = 0; i < sc->hn_tx_ring_inuse; ++i) { txr = &sc->hn_tx_ring[i]; stat += *((u_long *)((uint8_t *)txr + ofs)); } error = sysctl_handle_long(oidp, &stat, 0, req); if (error || req->newptr == NULL) return error; /* Zero out this stat. */ for (i = 0; i < sc->hn_tx_ring_inuse; ++i) { txr = &sc->hn_tx_ring[i]; *((u_long *)((uint8_t *)txr + ofs)) = 0; } return 0; } static int hn_tx_conf_int_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; int ofs = arg2, i, error, conf; struct hn_tx_ring *txr; txr = &sc->hn_tx_ring[0]; conf = *((int *)((uint8_t *)txr + ofs)); error = sysctl_handle_int(oidp, &conf, 0, req); if (error || req->newptr == NULL) return error; NV_LOCK(sc); for (i = 0; i < sc->hn_tx_ring_inuse; ++i) { txr = &sc->hn_tx_ring[i]; *((int *)((uint8_t *)txr + ofs)) = conf; } NV_UNLOCK(sc); return 0; } static int hn_ndis_version_sysctl(SYSCTL_HANDLER_ARGS) { struct hn_softc *sc = arg1; char verstr[16]; snprintf(verstr, sizeof(verstr), "%u.%u", NDIS_VERSION_MAJOR(sc->hn_ndis_ver), NDIS_VERSION_MINOR(sc->hn_ndis_ver)); return sysctl_handle_string(oidp, verstr, sizeof(verstr), req); } static int hn_check_iplen(const struct mbuf *m, int hoff) { const struct ip *ip; int len, iphlen, iplen; const struct tcphdr *th; int thoff; /* TCP data offset */ len = hoff + sizeof(struct ip); /* The packet must be at least the size of an IP header. */ if (m->m_pkthdr.len < len) return IPPROTO_DONE; /* The fixed IP header must reside completely in the first mbuf. */ if (m->m_len < len) return IPPROTO_DONE; ip = mtodo(m, hoff); /* Bound check the packet's stated IP header length. */ iphlen = ip->ip_hl << 2; if (iphlen < sizeof(struct ip)) /* minimum header length */ return IPPROTO_DONE; /* The full IP header must reside completely in the one mbuf. */ if (m->m_len < hoff + iphlen) return IPPROTO_DONE; iplen = ntohs(ip->ip_len); /* * Check that the amount of data in the buffers is as * at least much as the IP header would have us expect. */ if (m->m_pkthdr.len < hoff + iplen) return IPPROTO_DONE; /* * Ignore IP fragments. */ if (ntohs(ip->ip_off) & (IP_OFFMASK | IP_MF)) return IPPROTO_DONE; /* * The TCP/IP or UDP/IP header must be entirely contained within * the first fragment of a packet. */ switch (ip->ip_p) { case IPPROTO_TCP: if (iplen < iphlen + sizeof(struct tcphdr)) return IPPROTO_DONE; if (m->m_len < hoff + iphlen + sizeof(struct tcphdr)) return IPPROTO_DONE; th = (const struct tcphdr *)((const uint8_t *)ip + iphlen); thoff = th->th_off << 2; if (thoff < sizeof(struct tcphdr) || thoff + iphlen > iplen) return IPPROTO_DONE; if (m->m_len < hoff + iphlen + thoff) return IPPROTO_DONE; break; case IPPROTO_UDP: if (iplen < iphlen + sizeof(struct udphdr)) return IPPROTO_DONE; if (m->m_len < hoff + iphlen + sizeof(struct udphdr)) return IPPROTO_DONE; break; default: if (iplen < iphlen) return IPPROTO_DONE; break; } return ip->ip_p; } static int hn_create_rx_data(struct hn_softc *sc, int ring_cnt) { struct sysctl_oid_list *child; struct sysctl_ctx_list *ctx; device_t dev = sc->hn_dev; #if defined(INET) || defined(INET6) #if __FreeBSD_version >= 1100095 int lroent_cnt; #endif #endif int i; /* * Create RXBUF for reception. * * NOTE: * - It is shared by all channels. * - A large enough buffer is allocated, certain version of NVSes * may further limit the usable space. */ sc->hn_rxbuf = hyperv_dmamem_alloc(bus_get_dma_tag(dev), PAGE_SIZE, 0, NETVSC_RECEIVE_BUFFER_SIZE, &sc->hn_rxbuf_dma, BUS_DMA_WAITOK | BUS_DMA_ZERO); if (sc->hn_rxbuf == NULL) { device_printf(sc->hn_dev, "allocate rxbuf failed\n"); return (ENOMEM); } sc->hn_rx_ring_cnt = ring_cnt; sc->hn_rx_ring_inuse = sc->hn_rx_ring_cnt; sc->hn_rx_ring = malloc(sizeof(struct hn_rx_ring) * sc->hn_rx_ring_cnt, M_NETVSC, M_WAITOK | M_ZERO); #if defined(INET) || defined(INET6) #if __FreeBSD_version >= 1100095 lroent_cnt = hn_lro_entry_count; if (lroent_cnt < TCP_LRO_ENTRIES) lroent_cnt = TCP_LRO_ENTRIES; device_printf(dev, "LRO: entry count %d\n", lroent_cnt); #endif #endif /* INET || INET6 */ ctx = device_get_sysctl_ctx(dev); child = SYSCTL_CHILDREN(device_get_sysctl_tree(dev)); /* Create dev.hn.UNIT.rx sysctl tree */ sc->hn_rx_sysctl_tree = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, "rx", CTLFLAG_RD | CTLFLAG_MPSAFE, 0, ""); for (i = 0; i < sc->hn_rx_ring_cnt; ++i) { struct hn_rx_ring *rxr = &sc->hn_rx_ring[i]; if (hn_trust_hosttcp) rxr->hn_trust_hcsum |= HN_TRUST_HCSUM_TCP; if (hn_trust_hostudp) rxr->hn_trust_hcsum |= HN_TRUST_HCSUM_UDP; if (hn_trust_hostip) rxr->hn_trust_hcsum |= HN_TRUST_HCSUM_IP; rxr->hn_ifp = sc->hn_ifp; if (i < sc->hn_tx_ring_cnt) rxr->hn_txr = &sc->hn_tx_ring[i]; rxr->hn_rdbuf = malloc(NETVSC_PACKET_SIZE, M_NETVSC, M_WAITOK); rxr->hn_rx_idx = i; rxr->hn_rxbuf = sc->hn_rxbuf; /* * Initialize LRO. */ #if defined(INET) || defined(INET6) #if __FreeBSD_version >= 1100095 tcp_lro_init_args(&rxr->hn_lro, sc->hn_ifp, lroent_cnt, hn_lro_mbufq_depth); #else tcp_lro_init(&rxr->hn_lro); rxr->hn_lro.ifp = sc->hn_ifp; #endif #if __FreeBSD_version >= 1100099 rxr->hn_lro.lro_length_lim = HN_LRO_LENLIM_DEF; rxr->hn_lro.lro_ackcnt_lim = HN_LRO_ACKCNT_DEF; #endif #endif /* INET || INET6 */ if (sc->hn_rx_sysctl_tree != NULL) { char name[16]; /* * Create per RX ring sysctl tree: * dev.hn.UNIT.rx.RINGID */ snprintf(name, sizeof(name), "%d", i); rxr->hn_rx_sysctl_tree = SYSCTL_ADD_NODE(ctx, SYSCTL_CHILDREN(sc->hn_rx_sysctl_tree), OID_AUTO, name, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, ""); if (rxr->hn_rx_sysctl_tree != NULL) { SYSCTL_ADD_ULONG(ctx, SYSCTL_CHILDREN(rxr->hn_rx_sysctl_tree), OID_AUTO, "packets", CTLFLAG_RW, &rxr->hn_pkts, "# of packets received"); SYSCTL_ADD_ULONG(ctx, SYSCTL_CHILDREN(rxr->hn_rx_sysctl_tree), OID_AUTO, "rss_pkts", CTLFLAG_RW, &rxr->hn_rss_pkts, "# of packets w/ RSS info received"); } } } SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "lro_queued", CTLTYPE_U64 | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_rx_ring, hn_lro.lro_queued), hn_rx_stat_u64_sysctl, "LU", "LRO queued"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "lro_flushed", CTLTYPE_U64 | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_rx_ring, hn_lro.lro_flushed), hn_rx_stat_u64_sysctl, "LU", "LRO flushed"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "lro_tried", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_rx_ring, hn_lro_tried), hn_rx_stat_ulong_sysctl, "LU", "# of LRO tries"); #if __FreeBSD_version >= 1100099 SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "lro_length_lim", CTLTYPE_UINT | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, 0, hn_lro_lenlim_sysctl, "IU", "Max # of data bytes to be aggregated by LRO"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "lro_ackcnt_lim", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, 0, hn_lro_ackcnt_sysctl, "I", "Max # of ACKs to be aggregated by LRO"); #endif SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "trust_hosttcp", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, HN_TRUST_HCSUM_TCP, hn_trust_hcsum_sysctl, "I", "Trust tcp segement verification on host side, " "when csum info is missing"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "trust_hostudp", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, HN_TRUST_HCSUM_UDP, hn_trust_hcsum_sysctl, "I", "Trust udp datagram verification on host side, " "when csum info is missing"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "trust_hostip", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, HN_TRUST_HCSUM_IP, hn_trust_hcsum_sysctl, "I", "Trust ip packet verification on host side, " "when csum info is missing"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "csum_ip", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_rx_ring, hn_csum_ip), hn_rx_stat_ulong_sysctl, "LU", "RXCSUM IP"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "csum_tcp", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_rx_ring, hn_csum_tcp), hn_rx_stat_ulong_sysctl, "LU", "RXCSUM TCP"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "csum_udp", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_rx_ring, hn_csum_udp), hn_rx_stat_ulong_sysctl, "LU", "RXCSUM UDP"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "csum_trusted", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_rx_ring, hn_csum_trusted), hn_rx_stat_ulong_sysctl, "LU", "# of packets that we trust host's csum verification"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "small_pkts", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_rx_ring, hn_small_pkts), hn_rx_stat_ulong_sysctl, "LU", "# of small packets received"); SYSCTL_ADD_INT(ctx, child, OID_AUTO, "rx_ring_cnt", CTLFLAG_RD, &sc->hn_rx_ring_cnt, 0, "# created RX rings"); SYSCTL_ADD_INT(ctx, child, OID_AUTO, "rx_ring_inuse", CTLFLAG_RD, &sc->hn_rx_ring_inuse, 0, "# used RX rings"); return (0); } static void hn_destroy_rx_data(struct hn_softc *sc) { int i; if (sc->hn_rxbuf != NULL) { hyperv_dmamem_free(&sc->hn_rxbuf_dma, sc->hn_rxbuf); sc->hn_rxbuf = NULL; } if (sc->hn_rx_ring_cnt == 0) return; for (i = 0; i < sc->hn_rx_ring_cnt; ++i) { struct hn_rx_ring *rxr = &sc->hn_rx_ring[i]; #if defined(INET) || defined(INET6) tcp_lro_free(&rxr->hn_lro); #endif free(rxr->hn_rdbuf, M_NETVSC); } free(sc->hn_rx_ring, M_NETVSC); sc->hn_rx_ring = NULL; sc->hn_rx_ring_cnt = 0; sc->hn_rx_ring_inuse = 0; } static int hn_create_tx_ring(struct hn_softc *sc, int id) { struct hn_tx_ring *txr = &sc->hn_tx_ring[id]; device_t dev = sc->hn_dev; bus_dma_tag_t parent_dtag; int error, i; uint32_t version; txr->hn_sc = sc; txr->hn_tx_idx = id; #ifndef HN_USE_TXDESC_BUFRING mtx_init(&txr->hn_txlist_spin, "hn txlist", NULL, MTX_SPIN); #endif mtx_init(&txr->hn_tx_lock, "hn tx", NULL, MTX_DEF); txr->hn_txdesc_cnt = HN_TX_DESC_CNT; txr->hn_txdesc = malloc(sizeof(struct hn_txdesc) * txr->hn_txdesc_cnt, M_NETVSC, M_WAITOK | M_ZERO); #ifndef HN_USE_TXDESC_BUFRING SLIST_INIT(&txr->hn_txlist); #else txr->hn_txdesc_br = buf_ring_alloc(txr->hn_txdesc_cnt, M_NETVSC, M_WAITOK, &txr->hn_tx_lock); #endif txr->hn_tx_taskq = sc->hn_tx_taskq; if (hn_use_if_start) { txr->hn_txeof = hn_start_txeof; TASK_INIT(&txr->hn_tx_task, 0, hn_start_taskfunc, txr); TASK_INIT(&txr->hn_txeof_task, 0, hn_start_txeof_taskfunc, txr); } else { int br_depth; txr->hn_txeof = hn_xmit_txeof; TASK_INIT(&txr->hn_tx_task, 0, hn_xmit_taskfunc, txr); TASK_INIT(&txr->hn_txeof_task, 0, hn_xmit_txeof_taskfunc, txr); br_depth = hn_get_txswq_depth(txr); txr->hn_mbuf_br = buf_ring_alloc(br_depth, M_NETVSC, M_WAITOK, &txr->hn_tx_lock); } txr->hn_direct_tx_size = hn_direct_tx_size; version = VMBUS_GET_VERSION(device_get_parent(dev), dev); if (version >= VMBUS_VERSION_WIN8_1) { txr->hn_csum_assist = HN_CSUM_ASSIST; } else { txr->hn_csum_assist = HN_CSUM_ASSIST_WIN8; if (id == 0) { device_printf(dev, "bus version %u.%u, " "no UDP checksum offloading\n", VMBUS_VERSION_MAJOR(version), VMBUS_VERSION_MINOR(version)); } } /* * Always schedule transmission instead of trying to do direct * transmission. This one gives the best performance so far. */ txr->hn_sched_tx = 1; parent_dtag = bus_get_dma_tag(dev); /* DMA tag for RNDIS messages. */ error = bus_dma_tag_create(parent_dtag, /* parent */ HN_RNDIS_MSG_ALIGN, /* alignment */ HN_RNDIS_MSG_BOUNDARY, /* boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ HN_RNDIS_MSG_LEN, /* maxsize */ 1, /* nsegments */ HN_RNDIS_MSG_LEN, /* maxsegsize */ 0, /* flags */ NULL, /* lockfunc */ NULL, /* lockfuncarg */ &txr->hn_tx_rndis_dtag); if (error) { device_printf(dev, "failed to create rndis dmatag\n"); return error; } /* DMA tag for data. */ error = bus_dma_tag_create(parent_dtag, /* parent */ 1, /* alignment */ HN_TX_DATA_BOUNDARY, /* boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ HN_TX_DATA_MAXSIZE, /* maxsize */ HN_TX_DATA_SEGCNT_MAX, /* nsegments */ HN_TX_DATA_SEGSIZE, /* maxsegsize */ 0, /* flags */ NULL, /* lockfunc */ NULL, /* lockfuncarg */ &txr->hn_tx_data_dtag); if (error) { device_printf(dev, "failed to create data dmatag\n"); return error; } for (i = 0; i < txr->hn_txdesc_cnt; ++i) { struct hn_txdesc *txd = &txr->hn_txdesc[i]; txd->txr = txr; /* * Allocate and load RNDIS messages. */ error = bus_dmamem_alloc(txr->hn_tx_rndis_dtag, (void **)&txd->rndis_msg, BUS_DMA_WAITOK | BUS_DMA_COHERENT, &txd->rndis_msg_dmap); if (error) { device_printf(dev, "failed to allocate rndis_msg, %d\n", i); return error; } error = bus_dmamap_load(txr->hn_tx_rndis_dtag, txd->rndis_msg_dmap, txd->rndis_msg, HN_RNDIS_MSG_LEN, hyperv_dma_map_paddr, &txd->rndis_msg_paddr, BUS_DMA_NOWAIT); if (error) { device_printf(dev, "failed to load rndis_msg, %d\n", i); bus_dmamem_free(txr->hn_tx_rndis_dtag, txd->rndis_msg, txd->rndis_msg_dmap); return error; } /* DMA map for TX data. */ error = bus_dmamap_create(txr->hn_tx_data_dtag, 0, &txd->data_dmap); if (error) { device_printf(dev, "failed to allocate tx data dmamap\n"); bus_dmamap_unload(txr->hn_tx_rndis_dtag, txd->rndis_msg_dmap); bus_dmamem_free(txr->hn_tx_rndis_dtag, txd->rndis_msg, txd->rndis_msg_dmap); return error; } /* All set, put it to list */ txd->flags |= HN_TXD_FLAG_ONLIST; #ifndef HN_USE_TXDESC_BUFRING SLIST_INSERT_HEAD(&txr->hn_txlist, txd, link); #else buf_ring_enqueue(txr->hn_txdesc_br, txd); #endif } txr->hn_txdesc_avail = txr->hn_txdesc_cnt; if (sc->hn_tx_sysctl_tree != NULL) { struct sysctl_oid_list *child; struct sysctl_ctx_list *ctx; char name[16]; /* * Create per TX ring sysctl tree: * dev.hn.UNIT.tx.RINGID */ ctx = device_get_sysctl_ctx(dev); child = SYSCTL_CHILDREN(sc->hn_tx_sysctl_tree); snprintf(name, sizeof(name), "%d", id); txr->hn_tx_sysctl_tree = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, name, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, ""); if (txr->hn_tx_sysctl_tree != NULL) { child = SYSCTL_CHILDREN(txr->hn_tx_sysctl_tree); SYSCTL_ADD_INT(ctx, child, OID_AUTO, "txdesc_avail", CTLFLAG_RD, &txr->hn_txdesc_avail, 0, "# of available TX descs"); if (!hn_use_if_start) { SYSCTL_ADD_INT(ctx, child, OID_AUTO, "oactive", CTLFLAG_RD, &txr->hn_oactive, 0, "over active"); } SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "packets", CTLFLAG_RW, &txr->hn_pkts, "# of packets transmitted"); } } return 0; } static void hn_txdesc_dmamap_destroy(struct hn_txdesc *txd) { struct hn_tx_ring *txr = txd->txr; KASSERT(txd->m == NULL, ("still has mbuf installed")); KASSERT((txd->flags & HN_TXD_FLAG_DMAMAP) == 0, ("still dma mapped")); bus_dmamap_unload(txr->hn_tx_rndis_dtag, txd->rndis_msg_dmap); bus_dmamem_free(txr->hn_tx_rndis_dtag, txd->rndis_msg, txd->rndis_msg_dmap); bus_dmamap_destroy(txr->hn_tx_data_dtag, txd->data_dmap); } static void hn_destroy_tx_ring(struct hn_tx_ring *txr) { struct hn_txdesc *txd; if (txr->hn_txdesc == NULL) return; #ifndef HN_USE_TXDESC_BUFRING while ((txd = SLIST_FIRST(&txr->hn_txlist)) != NULL) { SLIST_REMOVE_HEAD(&txr->hn_txlist, link); hn_txdesc_dmamap_destroy(txd); } #else mtx_lock(&txr->hn_tx_lock); while ((txd = buf_ring_dequeue_sc(txr->hn_txdesc_br)) != NULL) hn_txdesc_dmamap_destroy(txd); mtx_unlock(&txr->hn_tx_lock); #endif if (txr->hn_tx_data_dtag != NULL) bus_dma_tag_destroy(txr->hn_tx_data_dtag); if (txr->hn_tx_rndis_dtag != NULL) bus_dma_tag_destroy(txr->hn_tx_rndis_dtag); #ifdef HN_USE_TXDESC_BUFRING buf_ring_free(txr->hn_txdesc_br, M_NETVSC); #endif free(txr->hn_txdesc, M_NETVSC); txr->hn_txdesc = NULL; if (txr->hn_mbuf_br != NULL) buf_ring_free(txr->hn_mbuf_br, M_NETVSC); #ifndef HN_USE_TXDESC_BUFRING mtx_destroy(&txr->hn_txlist_spin); #endif mtx_destroy(&txr->hn_tx_lock); } static int hn_create_tx_data(struct hn_softc *sc, int ring_cnt) { struct sysctl_oid_list *child; struct sysctl_ctx_list *ctx; int i; /* * Create TXBUF for chimney sending. * * NOTE: It is shared by all channels. */ sc->hn_chim = hyperv_dmamem_alloc(bus_get_dma_tag(sc->hn_dev), PAGE_SIZE, 0, NETVSC_SEND_BUFFER_SIZE, &sc->hn_chim_dma, BUS_DMA_WAITOK | BUS_DMA_ZERO); if (sc->hn_chim == NULL) { device_printf(sc->hn_dev, "allocate txbuf failed\n"); return (ENOMEM); } sc->hn_tx_ring_cnt = ring_cnt; sc->hn_tx_ring_inuse = sc->hn_tx_ring_cnt; sc->hn_tx_ring = malloc(sizeof(struct hn_tx_ring) * sc->hn_tx_ring_cnt, M_NETVSC, M_WAITOK | M_ZERO); ctx = device_get_sysctl_ctx(sc->hn_dev); child = SYSCTL_CHILDREN(device_get_sysctl_tree(sc->hn_dev)); /* Create dev.hn.UNIT.tx sysctl tree */ sc->hn_tx_sysctl_tree = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, "tx", CTLFLAG_RD | CTLFLAG_MPSAFE, 0, ""); for (i = 0; i < sc->hn_tx_ring_cnt; ++i) { int error; error = hn_create_tx_ring(sc, i); if (error) return error; } SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "no_txdescs", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_tx_ring, hn_no_txdescs), hn_tx_stat_ulong_sysctl, "LU", "# of times short of TX descs"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "send_failed", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_tx_ring, hn_send_failed), hn_tx_stat_ulong_sysctl, "LU", "# of hyper-v sending failure"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "txdma_failed", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_tx_ring, hn_txdma_failed), hn_tx_stat_ulong_sysctl, "LU", "# of TX DMA failure"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "tx_collapsed", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_tx_ring, hn_tx_collapsed), hn_tx_stat_ulong_sysctl, "LU", "# of TX mbuf collapsed"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "tx_chimney", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_tx_ring, hn_tx_chimney), hn_tx_stat_ulong_sysctl, "LU", "# of chimney send"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "tx_chimney_tried", CTLTYPE_ULONG | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_tx_ring, hn_tx_chimney_tried), hn_tx_stat_ulong_sysctl, "LU", "# of chimney send tries"); SYSCTL_ADD_INT(ctx, child, OID_AUTO, "txdesc_cnt", CTLFLAG_RD, &sc->hn_tx_ring[0].hn_txdesc_cnt, 0, "# of total TX descs"); SYSCTL_ADD_INT(ctx, child, OID_AUTO, "tx_chimney_max", CTLFLAG_RD, &sc->hn_chim_szmax, 0, "Chimney send packet size upper boundary"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "tx_chimney_size", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, 0, hn_chim_size_sysctl, "I", "Chimney send packet size limit"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "direct_tx_size", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_tx_ring, hn_direct_tx_size), hn_tx_conf_int_sysctl, "I", "Size of the packet for direct transmission"); SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "sched_tx", CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, sc, __offsetof(struct hn_tx_ring, hn_sched_tx), hn_tx_conf_int_sysctl, "I", "Always schedule transmission " "instead of doing direct transmission"); SYSCTL_ADD_INT(ctx, child, OID_AUTO, "tx_ring_cnt", CTLFLAG_RD, &sc->hn_tx_ring_cnt, 0, "# created TX rings"); SYSCTL_ADD_INT(ctx, child, OID_AUTO, "tx_ring_inuse", CTLFLAG_RD, &sc->hn_tx_ring_inuse, 0, "# used TX rings"); return 0; } static void hn_set_chim_size(struct hn_softc *sc, int chim_size) { int i; NV_LOCK(sc); for (i = 0; i < sc->hn_tx_ring_inuse; ++i) sc->hn_tx_ring[i].hn_chim_size = chim_size; NV_UNLOCK(sc); } static void hn_destroy_tx_data(struct hn_softc *sc) { int i; if (sc->hn_chim != NULL) { hyperv_dmamem_free(&sc->hn_chim_dma, sc->hn_chim); sc->hn_chim = NULL; } if (sc->hn_tx_ring_cnt == 0) return; for (i = 0; i < sc->hn_tx_ring_cnt; ++i) hn_destroy_tx_ring(&sc->hn_tx_ring[i]); free(sc->hn_tx_ring, M_NETVSC); sc->hn_tx_ring = NULL; sc->hn_tx_ring_cnt = 0; sc->hn_tx_ring_inuse = 0; } static void hn_start_taskfunc(void *xtxr, int pending __unused) { struct hn_tx_ring *txr = xtxr; mtx_lock(&txr->hn_tx_lock); hn_start_locked(txr, 0); mtx_unlock(&txr->hn_tx_lock); } static void hn_start_txeof_taskfunc(void *xtxr, int pending __unused) { struct hn_tx_ring *txr = xtxr; mtx_lock(&txr->hn_tx_lock); atomic_clear_int(&txr->hn_sc->hn_ifp->if_drv_flags, IFF_DRV_OACTIVE); hn_start_locked(txr, 0); mtx_unlock(&txr->hn_tx_lock); } static void hn_stop_tx_tasks(struct hn_softc *sc) { int i; for (i = 0; i < sc->hn_tx_ring_inuse; ++i) { struct hn_tx_ring *txr = &sc->hn_tx_ring[i]; taskqueue_drain(txr->hn_tx_taskq, &txr->hn_tx_task); taskqueue_drain(txr->hn_tx_taskq, &txr->hn_txeof_task); } } static int hn_xmit(struct hn_tx_ring *txr, int len) { struct hn_softc *sc = txr->hn_sc; struct ifnet *ifp = sc->hn_ifp; struct mbuf *m_head; mtx_assert(&txr->hn_tx_lock, MA_OWNED); KASSERT(hn_use_if_start == 0, ("hn_xmit is called, when if_start is enabled")); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0 || txr->hn_oactive) return 0; while ((m_head = drbr_peek(ifp, txr->hn_mbuf_br)) != NULL) { struct hn_txdesc *txd; int error; if (len > 0 && m_head->m_pkthdr.len > len) { /* * This sending could be time consuming; let callers * dispatch this packet sending (and sending of any * following up packets) to tx taskqueue. */ drbr_putback(ifp, txr->hn_mbuf_br, m_head); return 1; } txd = hn_txdesc_get(txr); if (txd == NULL) { txr->hn_no_txdescs++; drbr_putback(ifp, txr->hn_mbuf_br, m_head); txr->hn_oactive = 1; break; } error = hn_encap(txr, txd, &m_head); if (error) { /* Both txd and m_head are freed; discard */ drbr_advance(ifp, txr->hn_mbuf_br); continue; } error = hn_send_pkt(ifp, txr, txd); if (__predict_false(error)) { /* txd is freed, but m_head is not */ drbr_putback(ifp, txr->hn_mbuf_br, m_head); txr->hn_oactive = 1; break; } /* Sent */ drbr_advance(ifp, txr->hn_mbuf_br); } return 0; } static int hn_transmit(struct ifnet *ifp, struct mbuf *m) { struct hn_softc *sc = ifp->if_softc; struct hn_tx_ring *txr; int error, idx = 0; /* * Select the TX ring based on flowid */ if (M_HASHTYPE_GET(m) != M_HASHTYPE_NONE) idx = m->m_pkthdr.flowid % sc->hn_tx_ring_inuse; txr = &sc->hn_tx_ring[idx]; error = drbr_enqueue(ifp, txr->hn_mbuf_br, m); if (error) { if_inc_counter(ifp, IFCOUNTER_OQDROPS, 1); return error; } if (txr->hn_oactive) return 0; if (txr->hn_sched_tx) goto do_sched; if (mtx_trylock(&txr->hn_tx_lock)) { int sched; sched = hn_xmit(txr, txr->hn_direct_tx_size); mtx_unlock(&txr->hn_tx_lock); if (!sched) return 0; } do_sched: taskqueue_enqueue(txr->hn_tx_taskq, &txr->hn_tx_task); return 0; } static void hn_xmit_qflush(struct ifnet *ifp) { struct hn_softc *sc = ifp->if_softc; int i; for (i = 0; i < sc->hn_tx_ring_inuse; ++i) { struct hn_tx_ring *txr = &sc->hn_tx_ring[i]; struct mbuf *m; mtx_lock(&txr->hn_tx_lock); while ((m = buf_ring_dequeue_sc(txr->hn_mbuf_br)) != NULL) m_freem(m); mtx_unlock(&txr->hn_tx_lock); } if_qflush(ifp); } static void hn_xmit_txeof(struct hn_tx_ring *txr) { if (txr->hn_sched_tx) goto do_sched; if (mtx_trylock(&txr->hn_tx_lock)) { int sched; txr->hn_oactive = 0; sched = hn_xmit(txr, txr->hn_direct_tx_size); mtx_unlock(&txr->hn_tx_lock); if (sched) { taskqueue_enqueue(txr->hn_tx_taskq, &txr->hn_tx_task); } } else { do_sched: /* * Release the oactive earlier, with the hope, that * others could catch up. The task will clear the * oactive again with the hn_tx_lock to avoid possible * races. */ txr->hn_oactive = 0; taskqueue_enqueue(txr->hn_tx_taskq, &txr->hn_txeof_task); } } static void hn_xmit_taskfunc(void *xtxr, int pending __unused) { struct hn_tx_ring *txr = xtxr; mtx_lock(&txr->hn_tx_lock); hn_xmit(txr, 0); mtx_unlock(&txr->hn_tx_lock); } static void hn_xmit_txeof_taskfunc(void *xtxr, int pending __unused) { struct hn_tx_ring *txr = xtxr; mtx_lock(&txr->hn_tx_lock); txr->hn_oactive = 0; hn_xmit(txr, 0); mtx_unlock(&txr->hn_tx_lock); } static void hn_channel_attach(struct hn_softc *sc, struct vmbus_channel *chan) { struct hn_rx_ring *rxr; int idx; idx = vmbus_chan_subidx(chan); KASSERT(idx >= 0 && idx < sc->hn_rx_ring_inuse, ("invalid channel index %d, should > 0 && < %d", idx, sc->hn_rx_ring_inuse)); rxr = &sc->hn_rx_ring[idx]; KASSERT((rxr->hn_rx_flags & HN_RX_FLAG_ATTACHED) == 0, ("RX ring %d already attached", idx)); rxr->hn_rx_flags |= HN_RX_FLAG_ATTACHED; if (bootverbose) { if_printf(sc->hn_ifp, "link RX ring %d to channel%u\n", idx, vmbus_chan_id(chan)); } if (idx < sc->hn_tx_ring_inuse) { struct hn_tx_ring *txr = &sc->hn_tx_ring[idx]; KASSERT((txr->hn_tx_flags & HN_TX_FLAG_ATTACHED) == 0, ("TX ring %d already attached", idx)); txr->hn_tx_flags |= HN_TX_FLAG_ATTACHED; txr->hn_chan = chan; if (bootverbose) { if_printf(sc->hn_ifp, "link TX ring %d to channel%u\n", idx, vmbus_chan_id(chan)); } } /* Bind channel to a proper CPU */ vmbus_chan_cpu_set(chan, (sc->hn_cpu + idx) % mp_ncpus); } static void hn_subchan_attach(struct hn_softc *sc, struct vmbus_channel *chan) { KASSERT(!vmbus_chan_is_primary(chan), ("subchannel callback on primary channel")); hn_channel_attach(sc, chan); } static void hn_subchan_setup(struct hn_softc *sc) { struct vmbus_channel **subchans; int subchan_cnt = sc->hn_rx_ring_inuse - 1; int i; /* Wait for sub-channels setup to complete. */ subchans = vmbus_subchan_get(sc->hn_prichan, subchan_cnt); /* Attach the sub-channels. */ for (i = 0; i < subchan_cnt; ++i) { struct vmbus_channel *subchan = subchans[i]; /* NOTE: Calling order is critical. */ hn_subchan_attach(sc, subchan); hv_nv_subchan_attach(subchan, &sc->hn_rx_ring[vmbus_chan_subidx(subchan)]); } /* Release the sub-channels */ vmbus_subchan_rel(subchans, subchan_cnt); if_printf(sc->hn_ifp, "%d sub-channels setup done\n", subchan_cnt); } static void hn_tx_taskq_create(void *arg __unused) { if (!hn_share_tx_taskq) return; hn_tx_taskq = taskqueue_create("hn_tx", M_WAITOK, taskqueue_thread_enqueue, &hn_tx_taskq); if (hn_bind_tx_taskq >= 0) { int cpu = hn_bind_tx_taskq; cpuset_t cpu_set; if (cpu > mp_ncpus - 1) cpu = mp_ncpus - 1; CPU_SETOF(cpu, &cpu_set); taskqueue_start_threads_cpuset(&hn_tx_taskq, 1, PI_NET, &cpu_set, "hn tx"); } else { taskqueue_start_threads(&hn_tx_taskq, 1, PI_NET, "hn tx"); } } SYSINIT(hn_txtq_create, SI_SUB_DRIVERS, SI_ORDER_FIRST, hn_tx_taskq_create, NULL); static void hn_tx_taskq_destroy(void *arg __unused) { if (hn_tx_taskq != NULL) taskqueue_free(hn_tx_taskq); } SYSUNINIT(hn_txtq_destroy, SI_SUB_DRIVERS, SI_ORDER_FIRST, hn_tx_taskq_destroy, NULL); static device_method_t netvsc_methods[] = { /* Device interface */ DEVMETHOD(device_probe, netvsc_probe), DEVMETHOD(device_attach, netvsc_attach), DEVMETHOD(device_detach, netvsc_detach), DEVMETHOD(device_shutdown, netvsc_shutdown), { 0, 0 } }; static driver_t netvsc_driver = { NETVSC_DEVNAME, netvsc_methods, sizeof(hn_softc_t) }; static devclass_t netvsc_devclass; DRIVER_MODULE(hn, vmbus, netvsc_driver, netvsc_devclass, 0, 0); MODULE_VERSION(hn, 1); MODULE_DEPEND(hn, vmbus, 1, 1, 1); Index: projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis.h (revision 305172) @@ -1,923 +1,910 @@ /*- * Copyright (c) 2009-2012,2016 Microsoft Corp. * Copyright (c) 2010-2012 Citrix Inc. * Copyright (c) 2012 NetApp Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ #ifndef __HV_RNDIS_H__ #define __HV_RNDIS_H__ #include /* * NDIS protocol version numbers */ #define NDIS_VERSION_5_0 0x00050000 #define NDIS_VERSION_5_1 0x00050001 #define NDIS_VERSION_6_0 0x00060000 #define NDIS_VERSION_6_1 0x00060001 #define NDIS_VERSION_6_30 0x0006001e #define NDIS_VERSION_MAJOR(ver) (((ver) & 0xffff0000) >> 16) #define NDIS_VERSION_MINOR(ver) ((ver) & 0xffff) /* * Object Identifiers used by NdisRequest Query/Set Information */ /* * General Objects */ #define RNDIS_OID_GEN_SUPPORTED_LIST 0x00010101 #define RNDIS_OID_GEN_HARDWARE_STATUS 0x00010102 #define RNDIS_OID_GEN_MEDIA_SUPPORTED 0x00010103 #define RNDIS_OID_GEN_MEDIA_IN_USE 0x00010104 #define RNDIS_OID_GEN_MAXIMUM_LOOKAHEAD 0x00010105 #define RNDIS_OID_GEN_MAXIMUM_FRAME_SIZE 0x00010106 #define RNDIS_OID_GEN_LINK_SPEED 0x00010107 #define RNDIS_OID_GEN_TRANSMIT_BUFFER_SPACE 0x00010108 #define RNDIS_OID_GEN_RECEIVE_BUFFER_SPACE 0x00010109 #define RNDIS_OID_GEN_TRANSMIT_BLOCK_SIZE 0x0001010A #define RNDIS_OID_GEN_RECEIVE_BLOCK_SIZE 0x0001010B #define RNDIS_OID_GEN_VENDOR_ID 0x0001010C #define RNDIS_OID_GEN_VENDOR_DESCRIPTION 0x0001010D #define RNDIS_OID_GEN_CURRENT_PACKET_FILTER 0x0001010E #define RNDIS_OID_GEN_CURRENT_LOOKAHEAD 0x0001010F #define RNDIS_OID_GEN_DRIVER_VERSION 0x00010110 #define RNDIS_OID_GEN_MAXIMUM_TOTAL_SIZE 0x00010111 #define RNDIS_OID_GEN_PROTOCOL_OPTIONS 0x00010112 #define RNDIS_OID_GEN_MAC_OPTIONS 0x00010113 #define RNDIS_OID_GEN_MEDIA_CONNECT_STATUS 0x00010114 #define RNDIS_OID_GEN_MAXIMUM_SEND_PACKETS 0x00010115 #define RNDIS_OID_GEN_VENDOR_DRIVER_VERSION 0x00010116 #define RNDIS_OID_GEN_NETWORK_LAYER_ADDRESSES 0x00010118 #define RNDIS_OID_GEN_TRANSPORT_HEADER_OFFSET 0x00010119 #define RNDIS_OID_GEN_MACHINE_NAME 0x0001021A #define RNDIS_OID_GEN_RNDIS_CONFIG_PARAMETER 0x0001021B /* * For receive side scale */ /* Query only */ #define RNDIS_OID_GEN_RSS_CAPABILITIES 0x00010203 /* Query and set */ #define RNDIS_OID_GEN_RSS_PARAMETERS 0x00010204 #define RNDIS_OID_GEN_XMIT_OK 0x00020101 #define RNDIS_OID_GEN_RCV_OK 0x00020102 #define RNDIS_OID_GEN_XMIT_ERROR 0x00020103 #define RNDIS_OID_GEN_RCV_ERROR 0x00020104 #define RNDIS_OID_GEN_RCV_NO_BUFFER 0x00020105 #define RNDIS_OID_GEN_DIRECTED_BYTES_XMIT 0x00020201 #define RNDIS_OID_GEN_DIRECTED_FRAMES_XMIT 0x00020202 #define RNDIS_OID_GEN_MULTICAST_BYTES_XMIT 0x00020203 #define RNDIS_OID_GEN_MULTICAST_FRAMES_XMIT 0x00020204 #define RNDIS_OID_GEN_BROADCAST_BYTES_XMIT 0x00020205 #define RNDIS_OID_GEN_BROADCAST_FRAMES_XMIT 0x00020206 #define RNDIS_OID_GEN_DIRECTED_BYTES_RCV 0x00020207 #define RNDIS_OID_GEN_DIRECTED_FRAMES_RCV 0x00020208 #define RNDIS_OID_GEN_MULTICAST_BYTES_RCV 0x00020209 #define RNDIS_OID_GEN_MULTICAST_FRAMES_RCV 0x0002020A #define RNDIS_OID_GEN_BROADCAST_BYTES_RCV 0x0002020B #define RNDIS_OID_GEN_BROADCAST_FRAMES_RCV 0x0002020C #define RNDIS_OID_GEN_RCV_CRC_ERROR 0x0002020D #define RNDIS_OID_GEN_TRANSMIT_QUEUE_LENGTH 0x0002020E #define RNDIS_OID_GEN_GET_TIME_CAPS 0x0002020F #define RNDIS_OID_GEN_GET_NETCARD_TIME 0x00020210 /* * These are connection-oriented general OIDs. * These replace the above OIDs for connection-oriented media. */ #define RNDIS_OID_GEN_CO_SUPPORTED_LIST 0x00010101 #define RNDIS_OID_GEN_CO_HARDWARE_STATUS 0x00010102 #define RNDIS_OID_GEN_CO_MEDIA_SUPPORTED 0x00010103 #define RNDIS_OID_GEN_CO_MEDIA_IN_USE 0x00010104 #define RNDIS_OID_GEN_CO_LINK_SPEED 0x00010105 #define RNDIS_OID_GEN_CO_VENDOR_ID 0x00010106 #define RNDIS_OID_GEN_CO_VENDOR_DESCRIPTION 0x00010107 #define RNDIS_OID_GEN_CO_DRIVER_VERSION 0x00010108 #define RNDIS_OID_GEN_CO_PROTOCOL_OPTIONS 0x00010109 #define RNDIS_OID_GEN_CO_MAC_OPTIONS 0x0001010A #define RNDIS_OID_GEN_CO_MEDIA_CONNECT_STATUS 0x0001010B #define RNDIS_OID_GEN_CO_VENDOR_DRIVER_VERSION 0x0001010C #define RNDIS_OID_GEN_CO_MINIMUM_LINK_SPEED 0x0001010D #define RNDIS_OID_GEN_CO_GET_TIME_CAPS 0x00010201 #define RNDIS_OID_GEN_CO_GET_NETCARD_TIME 0x00010202 /* * These are connection-oriented statistics OIDs. */ #define RNDIS_OID_GEN_CO_XMIT_PDUS_OK 0x00020101 #define RNDIS_OID_GEN_CO_RCV_PDUS_OK 0x00020102 #define RNDIS_OID_GEN_CO_XMIT_PDUS_ERROR 0x00020103 #define RNDIS_OID_GEN_CO_RCV_PDUS_ERROR 0x00020104 #define RNDIS_OID_GEN_CO_RCV_PDUS_NO_BUFFER 0x00020105 #define RNDIS_OID_GEN_CO_RCV_CRC_ERROR 0x00020201 #define RNDIS_OID_GEN_CO_TRANSMIT_QUEUE_LENGTH 0x00020202 #define RNDIS_OID_GEN_CO_BYTES_XMIT 0x00020203 #define RNDIS_OID_GEN_CO_BYTES_RCV 0x00020204 #define RNDIS_OID_GEN_CO_BYTES_XMIT_OUTSTANDING 0x00020205 #define RNDIS_OID_GEN_CO_NETCARD_LOAD 0x00020206 /* * These are objects for Connection-oriented media call-managers. */ #define RNDIS_OID_CO_ADD_PVC 0xFF000001 #define RNDIS_OID_CO_DELETE_PVC 0xFF000002 #define RNDIS_OID_CO_GET_CALL_INFORMATION 0xFF000003 #define RNDIS_OID_CO_ADD_ADDRESS 0xFF000004 #define RNDIS_OID_CO_DELETE_ADDRESS 0xFF000005 #define RNDIS_OID_CO_GET_ADDRESSES 0xFF000006 #define RNDIS_OID_CO_ADDRESS_CHANGE 0xFF000007 #define RNDIS_OID_CO_SIGNALING_ENABLED 0xFF000008 #define RNDIS_OID_CO_SIGNALING_DISABLED 0xFF000009 /* * 802.3 Objects (Ethernet) */ #define RNDIS_OID_802_3_PERMANENT_ADDRESS 0x01010101 #define RNDIS_OID_802_3_CURRENT_ADDRESS 0x01010102 #define RNDIS_OID_802_3_MULTICAST_LIST 0x01010103 #define RNDIS_OID_802_3_MAXIMUM_LIST_SIZE 0x01010104 #define RNDIS_OID_802_3_MAC_OPTIONS 0x01010105 /* * */ #define NDIS_802_3_MAC_OPTION_PRIORITY 0x00000001 #define RNDIS_OID_802_3_RCV_ERROR_ALIGNMENT 0x01020101 #define RNDIS_OID_802_3_XMIT_ONE_COLLISION 0x01020102 #define RNDIS_OID_802_3_XMIT_MORE_COLLISIONS 0x01020103 #define RNDIS_OID_802_3_XMIT_DEFERRED 0x01020201 #define RNDIS_OID_802_3_XMIT_MAX_COLLISIONS 0x01020202 #define RNDIS_OID_802_3_RCV_OVERRUN 0x01020203 #define RNDIS_OID_802_3_XMIT_UNDERRUN 0x01020204 #define RNDIS_OID_802_3_XMIT_HEARTBEAT_FAILURE 0x01020205 #define RNDIS_OID_802_3_XMIT_TIMES_CRS_LOST 0x01020206 #define RNDIS_OID_802_3_XMIT_LATE_COLLISIONS 0x01020207 /* * RNDIS MP custom OID for test */ #define OID_RNDISMP_GET_RECEIVE_BUFFERS 0xFFA0C90D // Query only /* * Remote NDIS offload parameters */ #define RNDIS_OBJECT_TYPE_DEFAULT 0x80 #define RNDIS_OFFLOAD_PARAMETERS_REVISION_3 3 #define RNDIS_OFFLOAD_PARAMETERS_NO_CHANGE 0 #define RNDIS_OFFLOAD_PARAMETERS_LSOV2_DISABLED 1 #define RNDIS_OFFLOAD_PARAMETERS_LSOV2_ENABLED 2 #define RNDIS_OFFLOAD_PARAMETERS_LSOV1_ENABLED 2 #define RNDIS_OFFLOAD_PARAMETERS_RSC_DISABLED 1 #define RNDIS_OFFLOAD_PARAMETERS_RSC_ENABLED 2 #define RNDIS_OFFLOAD_PARAMETERS_TX_RX_DISABLED 1 #define RNDIS_OFFLOAD_PARAMETERS_TX_ENABLED_RX_DISABLED 2 #define RNDIS_OFFLOAD_PARAMETERS_RX_ENABLED_TX_DISABLED 3 #define RNDIS_OFFLOAD_PARAMETERS_TX_RX_ENABLED 4 #define RNDIS_TCP_LARGE_SEND_OFFLOAD_V2_TYPE 1 #define RNDIS_TCP_LARGE_SEND_OFFLOAD_IPV4 0 #define RNDIS_TCP_LARGE_SEND_OFFLOAD_IPV6 1 #define RNDIS_OID_TCP_OFFLOAD_CURRENT_CONFIG 0xFC01020B /* query only */ #define RNDIS_OID_TCP_OFFLOAD_PARAMETERS 0xFC01020C /* set only */ #define RNDIS_OID_TCP_OFFLOAD_HARDWARE_CAPABILITIES 0xFC01020D/* query only */ #define RNDIS_OID_TCP_CONNECTION_OFFLOAD_CURRENT_CONFIG 0xFC01020E /* query only */ #define RNDIS_OID_TCP_CONNECTION_OFFLOAD_HARDWARE_CAPABILITIES 0xFC01020F /* query */ #define RNDIS_OID_OFFLOAD_ENCAPSULATION 0x0101010A /* set/query */ /* * NdisInitialize message */ typedef struct rndis_initialize_request_ { /* RNDIS request ID */ uint32_t request_id; uint32_t major_version; uint32_t minor_version; uint32_t max_xfer_size; } rndis_initialize_request; /* * Response to NdisInitialize */ typedef struct rndis_initialize_complete_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS status */ uint32_t status; uint32_t major_version; uint32_t minor_version; uint32_t device_flags; /* RNDIS medium */ uint32_t medium; uint32_t max_pkts_per_msg; uint32_t max_xfer_size; uint32_t pkt_align_factor; uint32_t af_list_offset; uint32_t af_list_size; } rndis_initialize_complete; /* * Call manager devices only: Information about an address family * supported by the device is appended to the response to NdisInitialize. */ typedef struct rndis_co_address_family_ { /* RNDIS AF */ uint32_t address_family; uint32_t major_version; uint32_t minor_version; } rndis_co_address_family; /* * NdisHalt message */ typedef struct rndis_halt_request_ { /* RNDIS request ID */ uint32_t request_id; } rndis_halt_request; /* * NdisQueryRequest message */ typedef struct rndis_query_request_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS OID */ uint32_t oid; uint32_t info_buffer_length; uint32_t info_buffer_offset; /* RNDIS handle */ uint32_t device_vc_handle; } rndis_query_request; /* * Response to NdisQueryRequest */ typedef struct rndis_query_complete_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS status */ uint32_t status; uint32_t info_buffer_length; uint32_t info_buffer_offset; } rndis_query_complete; /* * NdisSetRequest message */ typedef struct rndis_set_request_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS OID */ uint32_t oid; uint32_t info_buffer_length; uint32_t info_buffer_offset; /* RNDIS handle */ uint32_t device_vc_handle; } rndis_set_request; /* * Response to NdisSetRequest */ typedef struct rndis_set_complete_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS status */ uint32_t status; } rndis_set_complete; /* * NdisReset message */ typedef struct rndis_reset_request_ { uint32_t reserved; } rndis_reset_request; /* * Response to NdisReset */ typedef struct rndis_reset_complete_ { /* RNDIS status */ uint32_t status; uint32_t addressing_reset; } rndis_reset_complete; /* * NdisMIndicateStatus message */ typedef struct rndis_indicate_status_ { /* RNDIS status */ uint32_t status; uint32_t status_buf_length; uint32_t status_buf_offset; } rndis_indicate_status; /* * Diagnostic information passed as the status buffer in * rndis_indicate_status messages signifying error conditions. */ typedef struct rndis_diagnostic_info_ { /* RNDIS status */ uint32_t diag_status; uint32_t error_offset; } rndis_diagnostic_info; /* * NdisKeepAlive message */ typedef struct rndis_keepalive_request_ { /* RNDIS request ID */ uint32_t request_id; } rndis_keepalive_request; /* * Response to NdisKeepAlive */ typedef struct rndis_keepalive_complete_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS status */ uint32_t status; } rndis_keepalive_complete; /* * Data message. All offset fields contain byte offsets from the beginning * of the rndis_packet structure. All length fields are in bytes. * VcHandle is set to 0 for connectionless data, otherwise it * contains the VC handle. */ typedef struct rndis_packet_ { uint32_t data_offset; uint32_t data_length; uint32_t oob_data_offset; uint32_t oob_data_length; uint32_t num_oob_data_elements; uint32_t per_pkt_info_offset; uint32_t per_pkt_info_length; /* RNDIS handle */ uint32_t vc_handle; uint32_t reserved; } rndis_packet; typedef struct rndis_packet_ex_ { uint32_t data_offset; uint32_t data_length; uint32_t oob_data_offset; uint32_t oob_data_length; uint32_t num_oob_data_elements; uint32_t per_pkt_info_offset; uint32_t per_pkt_info_length; /* RNDIS handle */ uint32_t vc_handle; uint32_t reserved; uint64_t data_buf_id; uint32_t data_buf_offset; uint64_t next_header_buf_id; uint32_t next_header_byte_offset; uint32_t next_header_byte_count; } rndis_packet_ex; /* * Optional Out of Band data associated with a Data message. */ typedef struct rndis_oobd_ { uint32_t size; /* RNDIS class ID */ uint32_t type; uint32_t class_info_offset; } rndis_oobd; /* * Packet extension field contents associated with a Data message. */ typedef struct rndis_per_packet_info_ { uint32_t size; uint32_t type; uint32_t per_packet_info_offset; } rndis_per_packet_info; typedef enum ndis_per_pkt_infotype_ { tcpip_chksum_info, ipsec_info, tcp_large_send_info, classification_handle_info, ndis_reserved, sgl_info, ieee_8021q_info, original_pkt_info, pkt_cancel_id, original_netbuf_list, cached_netbuf_list, short_pkt_padding_info, max_perpkt_info } ndis_per_pkt_infotype; #define nbl_hash_value pkt_cancel_id #define nbl_hash_info original_netbuf_list typedef struct ndis_8021q_info_ { union { struct { uint32_t user_pri : 3; /* User Priority */ uint32_t cfi : 1; /* Canonical Format ID */ uint32_t vlan_id : 12; uint32_t reserved : 16; } s1; uint32_t value; } u1; } ndis_8021q_info; struct rndis_object_header { uint8_t type; uint8_t revision; uint16_t size; }; typedef struct rndis_offload_params_ { struct rndis_object_header header; uint8_t ipv4_csum; uint8_t tcp_ipv4_csum; uint8_t udp_ipv4_csum; uint8_t tcp_ipv6_csum; uint8_t udp_ipv6_csum; uint8_t lso_v1; uint8_t ip_sec_v1; uint8_t lso_v2_ipv4; uint8_t lso_v2_ipv6; uint8_t tcp_connection_ipv4; uint8_t tcp_connection_ipv6; uint32_t flags; uint8_t ip_sec_v2; uint8_t ip_sec_v2_ipv4; struct { uint8_t rsc_ipv4; uint8_t rsc_ipv6; }; struct { uint8_t encapsulated_packet_task_offload; uint8_t encapsulation_types; }; } rndis_offload_params; typedef struct rndis_tcp_ip_csum_info_ { union { struct { uint32_t is_ipv4:1; uint32_t is_ipv6:1; uint32_t tcp_csum:1; uint32_t udp_csum:1; uint32_t ip_header_csum:1; uint32_t reserved:11; uint32_t tcp_header_offset:10; } xmit; struct { uint32_t tcp_csum_failed:1; uint32_t udp_csum_failed:1; uint32_t ip_csum_failed:1; uint32_t tcp_csum_succeeded:1; uint32_t udp_csum_succeeded:1; uint32_t ip_csum_succeeded:1; uint32_t loopback:1; uint32_t tcp_csum_value_invalid:1; uint32_t ip_csum_value_invalid:1; } receive; uint32_t value; }; } rndis_tcp_ip_csum_info; struct rndis_hash_value { uint32_t hash_value; } __packed; struct rndis_hash_info { uint32_t hash_info; } __packed; typedef struct rndis_tcp_tso_info_ { union { struct { uint32_t unused:30; uint32_t type:1; uint32_t reserved2:1; } xmit; struct { uint32_t mss:20; uint32_t tcp_header_offset:10; uint32_t type:1; uint32_t reserved2:1; } lso_v1_xmit; struct { uint32_t tcp_payload:30; uint32_t type:1; uint32_t reserved2:1; } lso_v1_xmit_complete; struct { uint32_t mss:20; uint32_t tcp_header_offset:10; uint32_t type:1; uint32_t ip_version:1; } lso_v2_xmit; struct { uint32_t reserved:30; uint32_t type:1; uint32_t reserved2:1; } lso_v2_xmit_complete; uint32_t value; }; } rndis_tcp_tso_info; #define RNDIS_HASHVAL_PPI_SIZE (sizeof(rndis_per_packet_info) + \ sizeof(struct rndis_hash_value)) #define RNDIS_VLAN_PPI_SIZE (sizeof(rndis_per_packet_info) + \ sizeof(ndis_8021q_info)) #define RNDIS_CSUM_PPI_SIZE (sizeof(rndis_per_packet_info) + \ sizeof(rndis_tcp_ip_csum_info)) #define RNDIS_TSO_PPI_SIZE (sizeof(rndis_per_packet_info) + \ sizeof(rndis_tcp_tso_info)) /* * Format of Information buffer passed in a SetRequest for the OID * OID_GEN_RNDIS_CONFIG_PARAMETER. */ typedef struct rndis_config_parameter_info_ { uint32_t parameter_name_offset; uint32_t parameter_name_length; uint32_t parameter_type; uint32_t parameter_value_offset; uint32_t parameter_value_length; } rndis_config_parameter_info; /* * Values for ParameterType in rndis_config_parameter_info */ #define RNDIS_CONFIG_PARAM_TYPE_INTEGER 0 #define RNDIS_CONFIG_PARAM_TYPE_STRING 2 /* * CONDIS Miniport messages for connection oriented devices * that do not implement a call manager. */ /* * CoNdisMiniportCreateVc message */ typedef struct rcondis_mp_create_vc_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS handle */ uint32_t ndis_vc_handle; } rcondis_mp_create_vc; /* * Response to CoNdisMiniportCreateVc */ typedef struct rcondis_mp_create_vc_complete_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS handle */ uint32_t device_vc_handle; /* RNDIS status */ uint32_t status; } rcondis_mp_create_vc_complete; /* * CoNdisMiniportDeleteVc message */ typedef struct rcondis_mp_delete_vc_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS handle */ uint32_t device_vc_handle; } rcondis_mp_delete_vc; /* * Response to CoNdisMiniportDeleteVc */ typedef struct rcondis_mp_delete_vc_complete_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS status */ uint32_t status; } rcondis_mp_delete_vc_complete; /* * CoNdisMiniportQueryRequest message */ typedef struct rcondis_mp_query_request_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS request type */ uint32_t request_type; /* RNDIS OID */ uint32_t oid; /* RNDIS handle */ uint32_t device_vc_handle; uint32_t info_buf_length; uint32_t info_buf_offset; } rcondis_mp_query_request; /* * CoNdisMiniportSetRequest message */ typedef struct rcondis_mp_set_request_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS request type */ uint32_t request_type; /* RNDIS OID */ uint32_t oid; /* RNDIS handle */ uint32_t device_vc_handle; uint32_t info_buf_length; uint32_t info_buf_offset; } rcondis_mp_set_request; /* * CoNdisIndicateStatus message */ typedef struct rcondis_indicate_status_ { /* RNDIS handle */ uint32_t ndis_vc_handle; /* RNDIS status */ uint32_t status; uint32_t status_buf_length; uint32_t status_buf_offset; } rcondis_indicate_status; /* * CONDIS Call/VC parameters */ typedef struct rcondis_specific_parameters_ { uint32_t parameter_type; uint32_t parameter_length; uint32_t parameter_offset; } rcondis_specific_parameters; typedef struct rcondis_media_parameters_ { uint32_t flags; uint32_t reserved1; uint32_t reserved2; rcondis_specific_parameters media_specific; } rcondis_media_parameters; typedef struct rndis_flowspec_ { uint32_t token_rate; uint32_t token_bucket_size; uint32_t peak_bandwidth; uint32_t latency; uint32_t delay_variation; uint32_t service_type; uint32_t max_sdu_size; uint32_t minimum_policed_size; } rndis_flowspec; typedef struct rcondis_call_manager_parameters_ { rndis_flowspec transmit; rndis_flowspec receive; rcondis_specific_parameters call_mgr_specific; } rcondis_call_manager_parameters; /* * CoNdisMiniportActivateVc message */ typedef struct rcondis_mp_activate_vc_request_ { /* RNDIS request ID */ uint32_t request_id; uint32_t flags; /* RNDIS handle */ uint32_t device_vc_handle; uint32_t media_params_offset; uint32_t media_params_length; uint32_t call_mgr_params_offset; uint32_t call_mgr_params_length; } rcondis_mp_activate_vc_request; /* * Response to CoNdisMiniportActivateVc */ typedef struct rcondis_mp_activate_vc_complete_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS status */ uint32_t status; } rcondis_mp_activate_vc_complete; /* * CoNdisMiniportDeactivateVc message */ typedef struct rcondis_mp_deactivate_vc_request_ { /* RNDIS request ID */ uint32_t request_id; uint32_t flags; /* RNDIS handle */ uint32_t device_vc_handle; } rcondis_mp_deactivate_vc_request; /* * Response to CoNdisMiniportDeactivateVc */ typedef struct rcondis_mp_deactivate_vc_complete_ { /* RNDIS request ID */ uint32_t request_id; /* RNDIS status */ uint32_t status; } rcondis_mp_deactivate_vc_complete; /* * union with all of the RNDIS messages */ typedef union rndis_msg_container_ { rndis_packet packet; rndis_initialize_request init_request; rndis_halt_request halt_request; rndis_query_request query_request; rndis_set_request set_request; rndis_reset_request reset_request; rndis_keepalive_request keepalive_request; rndis_indicate_status indicate_status; rndis_initialize_complete init_complete; rndis_query_complete query_complete; rndis_set_complete set_complete; rndis_reset_complete reset_complete; rndis_keepalive_complete keepalive_complete; rcondis_mp_create_vc co_miniport_create_vc; rcondis_mp_delete_vc co_miniport_delete_vc; rcondis_indicate_status co_miniport_status; rcondis_mp_activate_vc_request co_miniport_activate_vc; rcondis_mp_deactivate_vc_request co_miniport_deactivate_vc; rcondis_mp_create_vc_complete co_miniport_create_vc_complete; rcondis_mp_delete_vc_complete co_miniport_delete_vc_complete; rcondis_mp_activate_vc_complete co_miniport_activate_vc_complete; rcondis_mp_deactivate_vc_complete co_miniport_deactivate_vc_complete; rndis_packet_ex packet_ex; } rndis_msg_container; /* * Remote NDIS message format */ typedef struct rndis_msg_ { uint32_t ndis_msg_type; /* * Total length of this message, from the beginning * of the rndis_msg struct, in bytes. */ uint32_t msg_len; /* Actual message */ rndis_msg_container msg; } rndis_msg; /* * Handy macros */ /* * get the size of an RNDIS message. Pass in the message type, * rndis_set_request, rndis_packet for example */ #define RNDIS_MESSAGE_SIZE(message) \ (sizeof(message) + (sizeof(rndis_msg) - sizeof(rndis_msg_container))) /* * get pointer to info buffer with message pointer */ #define MESSAGE_TO_INFO_BUFFER(message) \ (((PUCHAR)(message)) + message->InformationBufferOffset) /* * get pointer to status buffer with message pointer */ #define MESSAGE_TO_STATUS_BUFFER(message) \ (((PUCHAR)(message)) + message->StatusBufferOffset) /* * get pointer to OOBD buffer with message pointer */ #define MESSAGE_TO_OOBD_BUFFER(message) \ (((PUCHAR)(message)) + message->OOBDataOffset) /* * get pointer to data buffer with message pointer */ #define MESSAGE_TO_DATA_BUFFER(message) \ (((PUCHAR)(message)) + message->PerPacketInfoOffset) /* * get pointer to contained message from NDIS_MESSAGE pointer */ #define RNDIS_MESSAGE_PTR_TO_MESSAGE_PTR(rndis_message) \ ((void *) &rndis_message->Message) /* * get pointer to contained message from NDIS_MESSAGE pointer */ #define RNDIS_MESSAGE_RAW_PTR_TO_MESSAGE_PTR(rndis_message) \ ((void *) rndis_message) /* * Structures used in OID_RNDISMP_GET_RECEIVE_BUFFERS */ #define RNDISMP_RECEIVE_BUFFER_ELEM_FLAG_VMQ_RECEIVE_BUFFER 0x00000001 typedef struct rndismp_rx_buf_elem_ { uint32_t flags; uint32_t length; uint64_t rx_buf_id; uint32_t gpadl_handle; void *rx_buf; } rndismp_rx_buf_elem; typedef struct rndismp_rx_bufs_info_ { uint32_t num_rx_bufs; rndismp_rx_buf_elem rx_buf_elems[1]; } rndismp_rx_bufs_info; #define RNDIS_HEADER_SIZE (sizeof(rndis_msg) - sizeof(rndis_msg_container)) -#define NDIS_PACKET_TYPE_DIRECTED 0x00000001 -#define NDIS_PACKET_TYPE_MULTICAST 0x00000002 -#define NDIS_PACKET_TYPE_ALL_MULTICAST 0x00000004 -#define NDIS_PACKET_TYPE_BROADCAST 0x00000008 -#define NDIS_PACKET_TYPE_SOURCE_ROUTING 0x00000010 -#define NDIS_PACKET_TYPE_PROMISCUOUS 0x00000020 -#define NDIS_PACKET_TYPE_SMT 0x00000040 -#define NDIS_PACKET_TYPE_ALL_LOCAL 0x00000080 -#define NDIS_PACKET_TYPE_GROUP 0x00000100 -#define NDIS_PACKET_TYPE_ALL_FUNCTIONAL 0x00000200 -#define NDIS_PACKET_TYPE_FUNCTIONAL 0x00000400 -#define NDIS_PACKET_TYPE_MAC_FRAME 0x00000800 - /* * Externs */ struct hn_rx_ring; struct hn_tx_ring; struct hn_recvinfo; int netvsc_recv(struct hn_rx_ring *rxr, const void *data, int dlen, const struct hn_recvinfo *info); void netvsc_channel_rollup(struct hn_rx_ring *rxr, struct hn_tx_ring *txr); void* hv_set_rppi_data(rndis_msg *rndis_mesg, uint32_t rppi_size, int pkt_type); void* hv_get_ppi_data(rndis_packet *rpkt, uint32_t type); #endif /* __HV_RNDIS_H__ */ Index: projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis_filter.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis_filter.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis_filter.c (revision 305172) @@ -1,1490 +1,1061 @@ /*- * Copyright (c) 2009-2012,2016 Microsoft Corp. * Copyright (c) 2010-2012 Citrix Inc. * Copyright (c) 2012 NetApp Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define HV_RF_RECVINFO_VLAN 0x1 #define HV_RF_RECVINFO_CSUM 0x2 #define HV_RF_RECVINFO_HASHINF 0x4 #define HV_RF_RECVINFO_HASHVAL 0x8 #define HV_RF_RECVINFO_ALL \ (HV_RF_RECVINFO_VLAN | \ HV_RF_RECVINFO_CSUM | \ HV_RF_RECVINFO_HASHINF | \ HV_RF_RECVINFO_HASHVAL) #define HN_RNDIS_RID_COMPAT_MASK 0xffff #define HN_RNDIS_RID_COMPAT_MAX HN_RNDIS_RID_COMPAT_MASK #define HN_RNDIS_XFER_SIZE 2048 /* * Forward declarations */ -static int hv_rf_send_request(rndis_device *device, rndis_request *request, - uint32_t message_type); -static void hv_rf_receive_response(rndis_device *device, +static void hv_rf_receive_indicate_status(struct hn_softc *sc, const rndis_msg *response); -static void hv_rf_receive_indicate_status(rndis_device *device, - const rndis_msg *response); static void hv_rf_receive_data(struct hn_rx_ring *rxr, const void *data, int dlen); -static inline int hv_rf_query_device_mac(rndis_device *device); -static inline int hv_rf_query_device_link_status(rndis_device *device); -static int hv_rf_set_packet_filter(rndis_device *device, uint32_t new_filter); -static int hv_rf_init_device(rndis_device *device); -static int hv_rf_open_device(rndis_device *device); -static int hv_rf_close_device(rndis_device *device); -int -hv_rf_send_offload_request(struct hn_softc *sc, - rndis_offload_params *offloads); +static int hv_rf_query_device_mac(struct hn_softc *sc, uint8_t *eaddr); +static int hv_rf_query_device_link_status(struct hn_softc *sc, + uint32_t *link_status); +static int hv_rf_init_device(struct hn_softc *sc); -static void hn_rndis_sent_halt(struct hn_send_ctx *sndc, - struct hn_softc *sc, struct vmbus_channel *chan, - const void *data, int dlen); -static void hn_rndis_sent_cb(struct hn_send_ctx *sndc, - struct hn_softc *sc, struct vmbus_channel *chan, - const void *data, int dlen); static int hn_rndis_query(struct hn_softc *sc, uint32_t oid, const void *idata, size_t idlen, void *odata, size_t *odlen0); static int hn_rndis_set(struct hn_softc *sc, uint32_t oid, const void *data, size_t dlen); static int hn_rndis_conf_offload(struct hn_softc *sc); static int hn_rndis_get_rsscaps(struct hn_softc *sc, int *rxr_cnt); static int hn_rndis_conf_rss(struct hn_softc *sc, int nchan); static __inline uint32_t hn_rndis_rid(struct hn_softc *sc) { uint32_t rid; again: rid = atomic_fetchadd_int(&sc->hn_rndis_rid, 1); if (rid == 0) goto again; /* Use upper 16 bits for non-compat RNDIS messages. */ return ((rid & 0xffff) << 16); } /* * Set the Per-Packet-Info with the specified type */ void * hv_set_rppi_data(rndis_msg *rndis_mesg, uint32_t rppi_size, int pkt_type) { rndis_packet *rndis_pkt; rndis_per_packet_info *rppi; rndis_pkt = &rndis_mesg->msg.packet; rndis_pkt->data_offset += rppi_size; rppi = (rndis_per_packet_info *)((char *)rndis_pkt + rndis_pkt->per_pkt_info_offset + rndis_pkt->per_pkt_info_length); rppi->size = rppi_size; rppi->type = pkt_type; rppi->per_packet_info_offset = sizeof(rndis_per_packet_info); rndis_pkt->per_pkt_info_length += rppi_size; return (rppi); } /* * Get the Per-Packet-Info with the specified type * return NULL if not found. */ void * hv_get_ppi_data(rndis_packet *rpkt, uint32_t type) { rndis_per_packet_info *ppi; int len; if (rpkt->per_pkt_info_offset == 0) return (NULL); ppi = (rndis_per_packet_info *)((unsigned long)rpkt + rpkt->per_pkt_info_offset); len = rpkt->per_pkt_info_length; while (len > 0) { if (ppi->type == type) return (void *)((unsigned long)ppi + ppi->per_packet_info_offset); len -= ppi->size; ppi = (rndis_per_packet_info *)((unsigned long)ppi + ppi->size); } return (NULL); } - /* - * Allow module_param to work and override to switch to promiscuous mode. - */ -static inline rndis_device * -hv_get_rndis_device(void) -{ - rndis_device *device; - - device = malloc(sizeof(rndis_device), M_NETVSC, M_WAITOK | M_ZERO); - - mtx_init(&device->req_lock, "HV-FRL", NULL, MTX_DEF); - - /* Same effect as STAILQ_HEAD_INITIALIZER() static initializer */ - STAILQ_INIT(&device->myrequest_list); - - device->state = RNDIS_DEV_UNINITIALIZED; - - return (device); -} - -/* - * - */ -static inline void -hv_put_rndis_device(rndis_device *device) -{ - mtx_destroy(&device->req_lock); - free(device, M_NETVSC); -} - -/* - * - */ -static inline rndis_request * -hv_rndis_request(rndis_device *device, uint32_t message_type, - uint32_t message_length) -{ - rndis_request *request; - rndis_msg *rndis_mesg; - rndis_set_request *set; - - request = malloc(sizeof(rndis_request), M_NETVSC, M_WAITOK | M_ZERO); - - sema_init(&request->wait_sema, 0, "rndis sema"); - - rndis_mesg = &request->request_msg; - rndis_mesg->ndis_msg_type = message_type; - rndis_mesg->msg_len = message_length; - - /* - * Set the request id. This field is always after the rndis header - * for request/response packet types so we just use the set_request - * as a template. - */ - set = &rndis_mesg->msg.set_request; - set->request_id = atomic_fetchadd_int(&device->new_request_id, 1) & - HN_RNDIS_RID_COMPAT_MASK; - - /* Add to the request list */ - mtx_lock(&device->req_lock); - STAILQ_INSERT_TAIL(&device->myrequest_list, request, mylist_entry); - mtx_unlock(&device->req_lock); - - return (request); -} - -/* - * - */ -static inline void -hv_put_rndis_request(rndis_device *device, rndis_request *request) -{ - mtx_lock(&device->req_lock); - /* Fixme: Has O(n) performance */ - /* - * XXXKYS: Use Doubly linked lists. - */ - STAILQ_REMOVE(&device->myrequest_list, request, rndis_request_, - mylist_entry); - mtx_unlock(&device->req_lock); - - sema_destroy(&request->wait_sema); - free(request, M_NETVSC); -} - -/* - * - */ -static int -hv_rf_send_request(rndis_device *device, rndis_request *request, - uint32_t message_type) -{ - struct hn_softc *sc = device->sc; - uint32_t send_buf_section_idx, tot_data_buf_len; - struct vmbus_gpa gpa[2]; - int gpa_cnt, send_buf_section_size; - hn_sent_callback_t cb; - - /* Set up the packet to send it */ - tot_data_buf_len = request->request_msg.msg_len; - - gpa_cnt = 1; - gpa[0].gpa_page = hv_get_phys_addr(&request->request_msg) >> PAGE_SHIFT; - gpa[0].gpa_len = request->request_msg.msg_len; - gpa[0].gpa_ofs = (unsigned long)&request->request_msg & (PAGE_SIZE - 1); - - if (gpa[0].gpa_ofs + gpa[0].gpa_len > PAGE_SIZE) { - gpa_cnt = 2; - gpa[0].gpa_len = PAGE_SIZE - gpa[0].gpa_ofs; - gpa[1].gpa_page = - hv_get_phys_addr((char*)&request->request_msg + - gpa[0].gpa_len) >> PAGE_SHIFT; - gpa[1].gpa_ofs = 0; - gpa[1].gpa_len = request->request_msg.msg_len - gpa[0].gpa_len; - } - - if (message_type != REMOTE_NDIS_HALT_MSG) - cb = hn_rndis_sent_cb; - else - cb = hn_rndis_sent_halt; - - if (tot_data_buf_len < sc->hn_chim_szmax) { - send_buf_section_idx = hn_chim_alloc(sc); - if (send_buf_section_idx != HN_NVS_CHIM_IDX_INVALID) { - uint8_t *dest = sc->hn_chim + - (send_buf_section_idx * sc->hn_chim_szmax); - - memcpy(dest, &request->request_msg, request->request_msg.msg_len); - send_buf_section_size = tot_data_buf_len; - gpa_cnt = 0; - goto sendit; - } - /* Failed to allocate chimney send buffer; move on */ - } - send_buf_section_idx = HN_NVS_CHIM_IDX_INVALID; - send_buf_section_size = 0; - -sendit: - hn_send_ctx_init(&request->send_ctx, cb, request, - send_buf_section_idx, send_buf_section_size); - return hv_nv_on_send(sc->hn_prichan, HN_NVS_RNDIS_MTYPE_CTRL, - &request->send_ctx, gpa, gpa_cnt); -} - -/* - * RNDIS filter receive response - */ -static void -hv_rf_receive_response(rndis_device *device, const rndis_msg *response) -{ - rndis_request *request = NULL; - rndis_request *next_request; - boolean_t found = FALSE; - - mtx_lock(&device->req_lock); - request = STAILQ_FIRST(&device->myrequest_list); - while (request != NULL) { - /* - * All request/response message contains request_id as the - * first field - */ - if (request->request_msg.msg.init_request.request_id == - response->msg.init_complete.request_id) { - found = TRUE; - break; - } - next_request = STAILQ_NEXT(request, mylist_entry); - request = next_request; - } - mtx_unlock(&device->req_lock); - - if (found) { - if (response->msg_len <= sizeof(rndis_msg)) { - memcpy(&request->response_msg, response, - response->msg_len); - } else { - request->response_msg.msg.init_complete.status = - RNDIS_STATUS_BUFFER_OVERFLOW; - } - sema_post(&request->wait_sema); - } -} - -int -hv_rf_send_offload_request(struct hn_softc *sc, - rndis_offload_params *offloads) -{ - rndis_request *request; - rndis_set_request *set; - rndis_offload_params *offload_req; - rndis_set_complete *set_complete; - rndis_device *rndis_dev = sc->rndis_dev; - device_t dev = sc->hn_dev; - uint32_t extlen = sizeof(rndis_offload_params); - int ret; - - if (sc->hn_nvs_ver <= NVSP_PROTOCOL_VERSION_4) { - extlen = VERSION_4_OFFLOAD_SIZE; - /* On NVSP_PROTOCOL_VERSION_4 and below, we do not support - * UDP checksum offload. - */ - offloads->udp_ipv4_csum = 0; - offloads->udp_ipv6_csum = 0; - } - - request = hv_rndis_request(rndis_dev, REMOTE_NDIS_SET_MSG, - RNDIS_MESSAGE_SIZE(rndis_set_request) + extlen); - if (!request) - return (ENOMEM); - - set = &request->request_msg.msg.set_request; - set->oid = RNDIS_OID_TCP_OFFLOAD_PARAMETERS; - set->info_buffer_length = extlen; - set->info_buffer_offset = sizeof(rndis_set_request); - set->device_vc_handle = 0; - - offload_req = (rndis_offload_params *)((unsigned long)set + - set->info_buffer_offset); - *offload_req = *offloads; - offload_req->header.type = RNDIS_OBJECT_TYPE_DEFAULT; - offload_req->header.revision = RNDIS_OFFLOAD_PARAMETERS_REVISION_3; - offload_req->header.size = extlen; - - ret = hv_rf_send_request(rndis_dev, request, REMOTE_NDIS_SET_MSG); - if (ret != 0) { - device_printf(dev, "hv send offload request failed, ret=%d!\n", - ret); - goto cleanup; - } - - ret = sema_timedwait(&request->wait_sema, 5 * hz); - if (ret != 0) { - device_printf(dev, "hv send offload request timeout\n"); - goto cleanup; - } - - set_complete = &request->response_msg.msg.set_complete; - if (set_complete->status == RNDIS_STATUS_SUCCESS) { - device_printf(dev, "hv send offload request succeeded\n"); - ret = 0; - } else { - if (set_complete->status == RNDIS_STATUS_NOT_SUPPORTED) { - device_printf(dev, "HV Not support offload\n"); - ret = 0; - } else { - ret = set_complete->status; - } - } - -cleanup: - hv_put_rndis_request(rndis_dev, request); - - return (ret); -} - -/* * RNDIS filter receive indicate status */ static void -hv_rf_receive_indicate_status(rndis_device *device, const rndis_msg *response) +hv_rf_receive_indicate_status(struct hn_softc *sc, const rndis_msg *response) { const rndis_indicate_status *indicate = &response->msg.indicate_status; switch(indicate->status) { case RNDIS_STATUS_MEDIA_CONNECT: - netvsc_linkstatus_callback(device->sc, 1); + netvsc_linkstatus_callback(sc, 1); break; case RNDIS_STATUS_MEDIA_DISCONNECT: - netvsc_linkstatus_callback(device->sc, 0); + netvsc_linkstatus_callback(sc, 0); break; default: /* TODO: */ - device_printf(device->sc->hn_dev, + if_printf(sc->hn_ifp, "unknown status %d received\n", indicate->status); break; } } static int hv_rf_find_recvinfo(const rndis_packet *rpkt, struct hn_recvinfo *info) { const rndis_per_packet_info *ppi; uint32_t mask, len; info->vlan_info = NULL; info->csum_info = NULL; info->hash_info = NULL; info->hash_value = NULL; if (rpkt->per_pkt_info_offset == 0) return 0; ppi = (const rndis_per_packet_info *) ((const uint8_t *)rpkt + rpkt->per_pkt_info_offset); len = rpkt->per_pkt_info_length; mask = 0; while (len != 0) { const void *ppi_dptr; uint32_t ppi_dlen; if (__predict_false(ppi->size < ppi->per_packet_info_offset)) return EINVAL; ppi_dlen = ppi->size - ppi->per_packet_info_offset; ppi_dptr = (const uint8_t *)ppi + ppi->per_packet_info_offset; switch (ppi->type) { case ieee_8021q_info: if (__predict_false(ppi_dlen < sizeof(ndis_8021q_info))) return EINVAL; info->vlan_info = ppi_dptr; mask |= HV_RF_RECVINFO_VLAN; break; case tcpip_chksum_info: if (__predict_false(ppi_dlen < sizeof(rndis_tcp_ip_csum_info))) return EINVAL; info->csum_info = ppi_dptr; mask |= HV_RF_RECVINFO_CSUM; break; case nbl_hash_value: if (__predict_false(ppi_dlen < sizeof(struct rndis_hash_value))) return EINVAL; info->hash_value = ppi_dptr; mask |= HV_RF_RECVINFO_HASHVAL; break; case nbl_hash_info: if (__predict_false(ppi_dlen < sizeof(struct rndis_hash_info))) return EINVAL; info->hash_info = ppi_dptr; mask |= HV_RF_RECVINFO_HASHINF; break; default: goto skip; } if (mask == HV_RF_RECVINFO_ALL) { /* All found; done */ break; } skip: if (__predict_false(len < ppi->size)) return EINVAL; len -= ppi->size; ppi = (const rndis_per_packet_info *) ((const uint8_t *)ppi + ppi->size); } return 0; } /* * RNDIS filter receive data */ static void hv_rf_receive_data(struct hn_rx_ring *rxr, const void *data, int dlen) { const rndis_msg *message = data; const rndis_packet *rndis_pkt; uint32_t data_offset; struct hn_recvinfo info; rndis_pkt = &message->msg.packet; /* * Fixme: Handle multiple rndis pkt msgs that may be enclosed in this * netvsc packet (ie tot_data_buf_len != message_length) */ /* Remove rndis header, then pass data packet up the stack */ data_offset = RNDIS_HEADER_SIZE + rndis_pkt->data_offset; dlen -= data_offset; if (dlen < rndis_pkt->data_length) { if_printf(rxr->hn_ifp, "total length %u is less than data length %u\n", dlen, rndis_pkt->data_length); return; } dlen = rndis_pkt->data_length; data = (const uint8_t *)data + data_offset; if (hv_rf_find_recvinfo(rndis_pkt, &info)) { if_printf(rxr->hn_ifp, "recvinfo parsing failed\n"); return; } netvsc_recv(rxr, data, dlen, &info); } /* * RNDIS filter on receive */ int hv_rf_on_receive(struct hn_softc *sc, struct hn_rx_ring *rxr, const void *data, int dlen) { - rndis_device *rndis_dev; const rndis_msg *rndis_hdr; const struct rndis_comp_hdr *comp; - rndis_dev = sc->rndis_dev; - if (rndis_dev->state == RNDIS_DEV_UNINITIALIZED) - return (EINVAL); - rndis_hdr = data; switch (rndis_hdr->ndis_msg_type) { /* data message */ case REMOTE_NDIS_PACKET_MSG: hv_rf_receive_data(rxr, data, dlen); break; /* completion messages */ case REMOTE_NDIS_INITIALIZE_CMPLT: case REMOTE_NDIS_QUERY_CMPLT: case REMOTE_NDIS_SET_CMPLT: case REMOTE_NDIS_KEEPALIVE_CMPLT: comp = data; - if (comp->rm_rid <= HN_RNDIS_RID_COMPAT_MAX) { - /* Transition time compat code */ - hv_rf_receive_response(rndis_dev, rndis_hdr); - } else { - vmbus_xact_ctx_wakeup(sc->hn_xact, data, dlen); - } + KASSERT(comp->rm_rid > HN_RNDIS_RID_COMPAT_MAX, + ("invalid rid 0x%08x\n", comp->rm_rid)); + vmbus_xact_ctx_wakeup(sc->hn_xact, comp, dlen); break; /* notification message */ case REMOTE_NDIS_INDICATE_STATUS_MSG: - hv_rf_receive_indicate_status(rndis_dev, rndis_hdr); + hv_rf_receive_indicate_status(sc, rndis_hdr); break; case REMOTE_NDIS_RESET_CMPLT: /* * Reset completed, no rid. * * NOTE: * RESET is not issued by hn(4), so this message should * _not_ be observed. */ if_printf(sc->hn_ifp, "RESET CMPLT received\n"); break; default: if_printf(sc->hn_ifp, "unknown RNDIS message 0x%x\n", rndis_hdr->ndis_msg_type); break; } return (0); } /* * RNDIS filter query device MAC address */ static int -hv_rf_query_device_mac(rndis_device *device) +hv_rf_query_device_mac(struct hn_softc *sc, uint8_t *eaddr) { - struct hn_softc *sc = device->sc; - size_t hwaddr_len; + size_t eaddr_len; int error; - hwaddr_len = ETHER_ADDR_LEN; + eaddr_len = ETHER_ADDR_LEN; error = hn_rndis_query(sc, OID_802_3_PERMANENT_ADDRESS, NULL, 0, - device->hw_mac_addr, &hwaddr_len); + eaddr, &eaddr_len); if (error) return (error); - if (hwaddr_len != ETHER_ADDR_LEN) { - if_printf(sc->hn_ifp, "invalid hwaddr len %zu\n", hwaddr_len); + if (eaddr_len != ETHER_ADDR_LEN) { + if_printf(sc->hn_ifp, "invalid eaddr len %zu\n", eaddr_len); return (EINVAL); } return (0); } /* * RNDIS filter query device link status */ -static inline int -hv_rf_query_device_link_status(rndis_device *device) +static int +hv_rf_query_device_link_status(struct hn_softc *sc, uint32_t *link_status) { - struct hn_softc *sc = device->sc; size_t size; int error; - size = sizeof(uint32_t); + size = sizeof(*link_status); error = hn_rndis_query(sc, OID_GEN_MEDIA_CONNECT_STATUS, NULL, 0, - &device->link_status, &size); + link_status, &size); if (error) return (error); if (size != sizeof(uint32_t)) { if_printf(sc->hn_ifp, "invalid link status len %zu\n", size); return (EINVAL); } return (0); } static uint8_t netvsc_hash_key[NDIS_HASH_KEYSIZE_TOEPLITZ] = { 0x6d, 0x5a, 0x56, 0xda, 0x25, 0x5b, 0x0e, 0xc2, 0x41, 0x67, 0x25, 0x3d, 0x43, 0xa3, 0x8f, 0xb0, 0xd0, 0xca, 0x2b, 0xcb, 0xae, 0x7b, 0x30, 0xb4, 0x77, 0xcb, 0x2d, 0xa3, 0x80, 0x30, 0xf2, 0x0c, 0x6a, 0x42, 0xb7, 0x3b, 0xbe, 0xac, 0x01, 0xfa }; -/* - * RNDIS filter set packet filter - * Sends an rndis request with the new filter, then waits for a response - * from the host. - * Returns zero on success, non-zero on failure. - */ -static int -hv_rf_set_packet_filter(rndis_device *device, uint32_t new_filter) -{ - rndis_request *request; - rndis_set_request *set; - rndis_set_complete *set_complete; - uint32_t status; - int ret; - - request = hv_rndis_request(device, REMOTE_NDIS_SET_MSG, - RNDIS_MESSAGE_SIZE(rndis_set_request) + sizeof(uint32_t)); - if (request == NULL) { - ret = -1; - goto cleanup; - } - - /* Set up the rndis set */ - set = &request->request_msg.msg.set_request; - set->oid = RNDIS_OID_GEN_CURRENT_PACKET_FILTER; - set->info_buffer_length = sizeof(uint32_t); - set->info_buffer_offset = sizeof(rndis_set_request); - - memcpy((void *)((unsigned long)set + sizeof(rndis_set_request)), - &new_filter, sizeof(uint32_t)); - - ret = hv_rf_send_request(device, request, REMOTE_NDIS_SET_MSG); - if (ret != 0) { - goto cleanup; - } - - /* - * Wait for the response from the host. Another thread will signal - * us when the response has arrived. In the failure case, - * sema_timedwait() returns a non-zero status after waiting 5 seconds. - */ - ret = sema_timedwait(&request->wait_sema, 5 * hz); - if (ret == 0) { - /* Response received, check status */ - set_complete = &request->response_msg.msg.set_complete; - status = set_complete->status; - if (status != RNDIS_STATUS_SUCCESS) { - /* Bad response status, return error */ - ret = -2; - } - } else { - /* - * We cannot deallocate the request since we may still - * receive a send completion for it. - */ - goto exit; - } - -cleanup: - if (request != NULL) { - hv_put_rndis_request(device, request); - } -exit: - return (ret); -} - static const void * -hn_rndis_xact_execute(struct hn_softc *sc, struct vmbus_xact *xact, uint32_t rid, - size_t reqlen, size_t *comp_len0, uint32_t comp_type) +hn_rndis_xact_exec1(struct hn_softc *sc, struct vmbus_xact *xact, size_t reqlen, + struct hn_send_ctx *sndc, size_t *comp_len) { struct vmbus_gpa gpa[HN_XACT_REQ_PGCNT]; - const struct rndis_comp_hdr *comp; - bus_addr_t paddr; - size_t comp_len, min_complen = *comp_len0; int gpa_cnt, error; + bus_addr_t paddr; - KASSERT(rid > HN_RNDIS_RID_COMPAT_MAX, ("invalid rid %u\n", rid)); KASSERT(reqlen <= HN_XACT_REQ_SIZE && reqlen > 0, ("invalid request length %zu", reqlen)); - KASSERT(min_complen >= sizeof(*comp), - ("invalid minimum complete len %zu", min_complen)); /* * Setup the SG list. */ paddr = vmbus_xact_req_paddr(xact); KASSERT((paddr & PAGE_MASK) == 0, ("vmbus xact request is not page aligned 0x%jx", (uintmax_t)paddr)); for (gpa_cnt = 0; gpa_cnt < HN_XACT_REQ_PGCNT; ++gpa_cnt) { int len = PAGE_SIZE; if (reqlen == 0) break; if (reqlen < len) len = reqlen; gpa[gpa_cnt].gpa_page = atop(paddr) + gpa_cnt; gpa[gpa_cnt].gpa_len = len; gpa[gpa_cnt].gpa_ofs = 0; reqlen -= len; } KASSERT(reqlen == 0, ("still have %zu request data left", reqlen)); /* * Send this RNDIS control message and wait for its completion * message. */ vmbus_xact_activate(xact); - error = hv_nv_on_send(sc->hn_prichan, HN_NVS_RNDIS_MTYPE_CTRL, - &hn_send_ctx_none, gpa, gpa_cnt); + error = hv_nv_on_send(sc->hn_prichan, HN_NVS_RNDIS_MTYPE_CTRL, sndc, + gpa, gpa_cnt); if (error) { vmbus_xact_deactivate(xact); if_printf(sc->hn_ifp, "RNDIS ctrl send failed: %d\n", error); return (NULL); } - comp = vmbus_xact_wait(xact, &comp_len); + return (vmbus_xact_wait(xact, comp_len)); +} +static const void * +hn_rndis_xact_execute(struct hn_softc *sc, struct vmbus_xact *xact, uint32_t rid, + size_t reqlen, size_t *comp_len0, uint32_t comp_type) +{ + const struct rndis_comp_hdr *comp; + size_t comp_len, min_complen = *comp_len0; + + KASSERT(rid > HN_RNDIS_RID_COMPAT_MAX, ("invalid rid %u\n", rid)); + KASSERT(min_complen >= sizeof(*comp), + ("invalid minimum complete len %zu", min_complen)); + /* + * Execute the xact setup by the caller. + */ + comp = hn_rndis_xact_exec1(sc, xact, reqlen, &hn_send_ctx_none, + &comp_len); + if (comp == NULL) + return (NULL); + + /* * Check this RNDIS complete message. */ if (comp_len < min_complen) { if (comp_len >= sizeof(*comp)) { /* rm_status field is valid */ if_printf(sc->hn_ifp, "invalid RNDIS comp len %zu, " "status 0x%08x\n", comp_len, comp->rm_status); } else { if_printf(sc->hn_ifp, "invalid RNDIS comp len %zu\n", comp_len); } return (NULL); } if (comp->rm_len < min_complen) { if_printf(sc->hn_ifp, "invalid RNDIS comp msglen %u\n", comp->rm_len); return (NULL); } if (comp->rm_type != comp_type) { if_printf(sc->hn_ifp, "unexpected RNDIS comp 0x%08x, " "expect 0x%08x\n", comp->rm_type, comp_type); return (NULL); } if (comp->rm_rid != rid) { if_printf(sc->hn_ifp, "RNDIS comp rid mismatch %u, " "expect %u\n", comp->rm_rid, rid); return (NULL); } /* All pass! */ *comp_len0 = comp_len; return (comp); } static int hn_rndis_query(struct hn_softc *sc, uint32_t oid, const void *idata, size_t idlen, void *odata, size_t *odlen0) { struct rndis_query_req *req; const struct rndis_query_comp *comp; struct vmbus_xact *xact; size_t reqlen, odlen = *odlen0, comp_len; int error, ofs; uint32_t rid; reqlen = sizeof(*req) + idlen; xact = vmbus_xact_get(sc->hn_xact, reqlen); if (xact == NULL) { if_printf(sc->hn_ifp, "no xact for RNDIS query 0x%08x\n", oid); return (ENXIO); } rid = hn_rndis_rid(sc); req = vmbus_xact_req_data(xact); req->rm_type = REMOTE_NDIS_QUERY_MSG; req->rm_len = reqlen; req->rm_rid = rid; req->rm_oid = oid; /* * XXX * This is _not_ RNDIS Spec conforming: * "This MUST be set to 0 when there is no input data * associated with the OID." * * If this field was set to 0 according to the RNDIS Spec, * Hyper-V would set non-SUCCESS status in the query * completion. */ req->rm_infobufoffset = RNDIS_QUERY_REQ_INFOBUFOFFSET; if (idlen > 0) { req->rm_infobuflen = idlen; /* Input data immediately follows RNDIS query. */ memcpy(req + 1, idata, idlen); } comp_len = sizeof(*comp) + odlen; comp = hn_rndis_xact_execute(sc, xact, rid, reqlen, &comp_len, REMOTE_NDIS_QUERY_CMPLT); if (comp == NULL) { if_printf(sc->hn_ifp, "exec RNDIS query 0x%08x failed\n", oid); error = EIO; goto done; } if (comp->rm_status != RNDIS_STATUS_SUCCESS) { if_printf(sc->hn_ifp, "RNDIS query 0x%08x failed: " "status 0x%08x\n", oid, comp->rm_status); error = EIO; goto done; } if (comp->rm_infobuflen == 0 || comp->rm_infobufoffset == 0) { /* No output data! */ if_printf(sc->hn_ifp, "RNDIS query 0x%08x, no data\n", oid); *odlen0 = 0; error = 0; goto done; } /* * Check output data length and offset. */ /* ofs is the offset from the beginning of comp. */ ofs = RNDIS_QUERY_COMP_INFOBUFABS(comp->rm_infobufoffset); if (ofs < sizeof(*comp) || ofs + comp->rm_infobuflen > comp_len) { if_printf(sc->hn_ifp, "RNDIS query invalid comp ib off/len, " "%u/%u\n", comp->rm_infobufoffset, comp->rm_infobuflen); error = EINVAL; goto done; } /* * Save output data. */ if (comp->rm_infobuflen < odlen) odlen = comp->rm_infobuflen; memcpy(odata, ((const uint8_t *)comp) + ofs, odlen); *odlen0 = odlen; error = 0; done: vmbus_xact_put(xact); return (error); } static int hn_rndis_get_rsscaps(struct hn_softc *sc, int *rxr_cnt) { struct ndis_rss_caps in, caps; size_t caps_len; int error; /* * Only NDIS 6.30+ is supported. */ KASSERT(sc->hn_ndis_ver >= NDIS_VERSION_6_30, ("NDIS 6.30+ is required, NDIS version 0x%08x", sc->hn_ndis_ver)); *rxr_cnt = 0; memset(&in, 0, sizeof(in)); in.ndis_hdr.ndis_type = NDIS_OBJTYPE_RSS_CAPS; in.ndis_hdr.ndis_rev = NDIS_RSS_CAPS_REV_2; in.ndis_hdr.ndis_size = NDIS_RSS_CAPS_SIZE; caps_len = NDIS_RSS_CAPS_SIZE; - error = hn_rndis_query(sc, OID_GEN_RSS_CAPABILITIES, + error = hn_rndis_query(sc, OID_GEN_RECEIVE_SCALE_CAPABILITIES, &in, NDIS_RSS_CAPS_SIZE, &caps, &caps_len); if (error) return (error); if (caps_len < NDIS_RSS_CAPS_SIZE_6_0) { if_printf(sc->hn_ifp, "invalid NDIS RSS caps len %zu", caps_len); return (EINVAL); } if (caps.ndis_nrxr == 0) { if_printf(sc->hn_ifp, "0 RX rings!?\n"); return (EINVAL); } *rxr_cnt = caps.ndis_nrxr; if (caps_len == NDIS_RSS_CAPS_SIZE) { if (bootverbose) { if_printf(sc->hn_ifp, "RSS indirect table size %u\n", caps.ndis_nind); } } return (0); } static int hn_rndis_set(struct hn_softc *sc, uint32_t oid, const void *data, size_t dlen) { struct rndis_set_req *req; const struct rndis_set_comp *comp; struct vmbus_xact *xact; size_t reqlen, comp_len; uint32_t rid; int error; KASSERT(dlen > 0, ("invalid dlen %zu", dlen)); reqlen = sizeof(*req) + dlen; xact = vmbus_xact_get(sc->hn_xact, reqlen); if (xact == NULL) { if_printf(sc->hn_ifp, "no xact for RNDIS set 0x%08x\n", oid); return (ENXIO); } rid = hn_rndis_rid(sc); req = vmbus_xact_req_data(xact); req->rm_type = REMOTE_NDIS_SET_MSG; req->rm_len = reqlen; req->rm_rid = rid; req->rm_oid = oid; req->rm_infobuflen = dlen; req->rm_infobufoffset = RNDIS_SET_REQ_INFOBUFOFFSET; /* Data immediately follows RNDIS set. */ memcpy(req + 1, data, dlen); comp_len = sizeof(*comp); comp = hn_rndis_xact_execute(sc, xact, rid, reqlen, &comp_len, REMOTE_NDIS_SET_CMPLT); if (comp == NULL) { if_printf(sc->hn_ifp, "exec RNDIS set 0x%08x failed\n", oid); error = EIO; goto done; } if (comp->rm_status != RNDIS_STATUS_SUCCESS) { if_printf(sc->hn_ifp, "RNDIS set 0x%08x failed: " "status 0x%08x\n", oid, comp->rm_status); error = EIO; goto done; } error = 0; done: vmbus_xact_put(xact); return (error); } static int hn_rndis_conf_offload(struct hn_softc *sc) { struct ndis_offload_params params; size_t paramsz; int error; /* NOTE: 0 means "no change" */ memset(¶ms, 0, sizeof(params)); params.ndis_hdr.ndis_type = NDIS_OBJTYPE_DEFAULT; if (sc->hn_ndis_ver < NDIS_VERSION_6_30) { params.ndis_hdr.ndis_rev = NDIS_OFFLOAD_PARAMS_REV_2; paramsz = NDIS_OFFLOAD_PARAMS_SIZE_6_1; } else { params.ndis_hdr.ndis_rev = NDIS_OFFLOAD_PARAMS_REV_3; paramsz = NDIS_OFFLOAD_PARAMS_SIZE; } params.ndis_hdr.ndis_size = paramsz; params.ndis_ip4csum = NDIS_OFFLOAD_PARAM_TXRX; params.ndis_tcp4csum = NDIS_OFFLOAD_PARAM_TXRX; params.ndis_tcp6csum = NDIS_OFFLOAD_PARAM_TXRX; if (sc->hn_ndis_ver >= NDIS_VERSION_6_30) { params.ndis_udp4csum = NDIS_OFFLOAD_PARAM_TXRX; params.ndis_udp6csum = NDIS_OFFLOAD_PARAM_TXRX; } params.ndis_lsov2_ip4 = NDIS_OFFLOAD_LSOV2_ON; /* XXX ndis_lsov2_ip6 = NDIS_OFFLOAD_LSOV2_ON */ error = hn_rndis_set(sc, OID_TCP_OFFLOAD_PARAMETERS, ¶ms, paramsz); if (error) { if_printf(sc->hn_ifp, "offload config failed: %d\n", error); } else { if (bootverbose) if_printf(sc->hn_ifp, "offload config done\n"); } return (error); } static int hn_rndis_conf_rss(struct hn_softc *sc, int nchan) { struct ndis_rssprm_toeplitz *rss = &sc->hn_rss; struct ndis_rss_params *prm = &rss->rss_params; int i, error; /* * Only NDIS 6.30+ is supported. */ KASSERT(sc->hn_ndis_ver >= NDIS_VERSION_6_30, ("NDIS 6.30+ is required, NDIS version 0x%08x", sc->hn_ndis_ver)); memset(rss, 0, sizeof(*rss)); prm->ndis_hdr.ndis_type = NDIS_OBJTYPE_RSS_PARAMS; prm->ndis_hdr.ndis_rev = NDIS_RSS_PARAMS_REV_2; prm->ndis_hdr.ndis_size = sizeof(*rss); prm->ndis_hash = NDIS_HASH_FUNCTION_TOEPLITZ | NDIS_HASH_IPV4 | NDIS_HASH_TCP_IPV4 | NDIS_HASH_IPV6 | NDIS_HASH_TCP_IPV6; /* TODO: Take ndis_rss_caps.ndis_nind into account */ prm->ndis_indsize = sizeof(rss->rss_ind); prm->ndis_indoffset = __offsetof(struct ndis_rssprm_toeplitz, rss_ind[0]); prm->ndis_keysize = sizeof(rss->rss_key); prm->ndis_keyoffset = __offsetof(struct ndis_rssprm_toeplitz, rss_key[0]); /* Setup RSS key */ memcpy(rss->rss_key, netvsc_hash_key, sizeof(rss->rss_key)); /* Setup RSS indirect table */ /* TODO: Take ndis_rss_caps.ndis_nind into account */ for (i = 0; i < NDIS_HASH_INDCNT; ++i) rss->rss_ind[i] = i % nchan; - error = hn_rndis_set(sc, OID_GEN_RSS_PARAMETERS, rss, sizeof(*rss)); + error = hn_rndis_set(sc, OID_GEN_RECEIVE_SCALE_PARAMETERS, + rss, sizeof(*rss)); if (error) { if_printf(sc->hn_ifp, "RSS config failed: %d\n", error); } else { if (bootverbose) if_printf(sc->hn_ifp, "RSS config done\n"); } return (error); } +static int +hn_rndis_set_rxfilter(struct hn_softc *sc, uint32_t filter) +{ + int error; + + error = hn_rndis_set(sc, OID_GEN_CURRENT_PACKET_FILTER, + &filter, sizeof(filter)); + if (error) { + if_printf(sc->hn_ifp, "set RX filter 0x%08x failed: %d\n", + filter, error); + } else { + if (bootverbose) { + if_printf(sc->hn_ifp, "set RX filter 0x%08x done\n", + filter); + } + } + return (error); +} + /* * RNDIS filter init device */ static int -hv_rf_init_device(rndis_device *device) +hv_rf_init_device(struct hn_softc *sc) { - struct hn_softc *sc = device->sc; struct rndis_init_req *req; const struct rndis_init_comp *comp; struct vmbus_xact *xact; size_t comp_len; uint32_t rid; int error; - /* XXX */ - device->state = RNDIS_DEV_INITIALIZED; - xact = vmbus_xact_get(sc->hn_xact, sizeof(*req)); if (xact == NULL) { if_printf(sc->hn_ifp, "no xact for RNDIS init\n"); return (ENXIO); } rid = hn_rndis_rid(sc); req = vmbus_xact_req_data(xact); req->rm_type = REMOTE_NDIS_INITIALIZE_MSG; req->rm_len = sizeof(*req); req->rm_rid = rid; req->rm_ver_major = RNDIS_VERSION_MAJOR; req->rm_ver_minor = RNDIS_VERSION_MINOR; req->rm_max_xfersz = HN_RNDIS_XFER_SIZE; comp_len = RNDIS_INIT_COMP_SIZE_MIN; comp = hn_rndis_xact_execute(sc, xact, rid, sizeof(*req), &comp_len, REMOTE_NDIS_INITIALIZE_CMPLT); if (comp == NULL) { if_printf(sc->hn_ifp, "exec RNDIS init failed\n"); error = EIO; goto done; } if (comp->rm_status != RNDIS_STATUS_SUCCESS) { if_printf(sc->hn_ifp, "RNDIS init failed: status 0x%08x\n", comp->rm_status); error = EIO; goto done; } if (bootverbose) { - if_printf(sc->hn_ifp, "RNDIS ver %u.%u, pktsz %u, pktcnt %u\n", - comp->rm_ver_major, comp->rm_ver_minor, - comp->rm_pktmaxsz, comp->rm_pktmaxcnt); + if_printf(sc->hn_ifp, "RNDIS ver %u.%u, pktsz %u, pktcnt %u, " + "align %u\n", comp->rm_ver_major, comp->rm_ver_minor, + comp->rm_pktmaxsz, comp->rm_pktmaxcnt, + 1U << comp->rm_align); } error = 0; - done: - if (xact != NULL) - vmbus_xact_put(xact); + vmbus_xact_put(xact); return (error); } -#define HALT_COMPLETION_WAIT_COUNT 25 - /* * RNDIS filter halt device */ static int -hv_rf_halt_device(rndis_device *device) +hv_rf_halt_device(struct hn_softc *sc) { - rndis_request *request; - int i, ret; + struct vmbus_xact *xact; + struct rndis_halt_req *halt; + struct hn_send_ctx sndc; + size_t comp_len; - /* Attempt to do a rndis device halt */ - request = hv_rndis_request(device, REMOTE_NDIS_HALT_MSG, - RNDIS_MESSAGE_SIZE(rndis_halt_request)); - if (request == NULL) { - return (-1); + xact = vmbus_xact_get(sc->hn_xact, sizeof(*halt)); + if (xact == NULL) { + if_printf(sc->hn_ifp, "no xact for RNDIS halt\n"); + return (ENXIO); } + halt = vmbus_xact_req_data(xact); + halt->rm_type = REMOTE_NDIS_HALT_MSG; + halt->rm_len = sizeof(*halt); + halt->rm_rid = hn_rndis_rid(sc); - /* initialize "poor man's semaphore" */ - request->halt_complete_flag = 0; + /* No RNDIS completion; rely on NVS message send completion */ + hn_send_ctx_init_simple(&sndc, hn_nvs_sent_xact, xact); + hn_rndis_xact_exec1(sc, xact, sizeof(*halt), &sndc, &comp_len); - ret = hv_rf_send_request(device, request, REMOTE_NDIS_HALT_MSG); - if (ret != 0) { - return (-1); - } - - /* - * Wait for halt response from halt callback. We must wait for - * the transaction response before freeing the request and other - * resources. - */ - for (i=HALT_COMPLETION_WAIT_COUNT; i > 0; i--) { - if (request->halt_complete_flag != 0) { - break; - } - DELAY(400); - } - if (i == 0) { - return (-1); - } - - device->state = RNDIS_DEV_UNINITIALIZED; - - hv_put_rndis_request(device, request); - + vmbus_xact_put(xact); + if (bootverbose) + if_printf(sc->hn_ifp, "RNDIS halt done\n"); return (0); } /* - * RNDIS filter open device - */ -static int -hv_rf_open_device(rndis_device *device) -{ - int ret; - - if (device->state != RNDIS_DEV_INITIALIZED) { - return (0); - } - - if (hv_promisc_mode != 1) { - ret = hv_rf_set_packet_filter(device, - NDIS_PACKET_TYPE_BROADCAST | - NDIS_PACKET_TYPE_ALL_MULTICAST | - NDIS_PACKET_TYPE_DIRECTED); - } else { - ret = hv_rf_set_packet_filter(device, - NDIS_PACKET_TYPE_PROMISCUOUS); - } - - if (ret == 0) { - device->state = RNDIS_DEV_DATAINITIALIZED; - } - - return (ret); -} - -/* - * RNDIS filter close device - */ -static int -hv_rf_close_device(rndis_device *device) -{ - int ret; - - if (device->state != RNDIS_DEV_DATAINITIALIZED) { - return (0); - } - - ret = hv_rf_set_packet_filter(device, 0); - if (ret == 0) { - device->state = RNDIS_DEV_INITIALIZED; - } - - return (ret); -} - -/* * RNDIS filter on device add */ int hv_rf_on_device_add(struct hn_softc *sc, void *additl_info, int *nchan0, struct hn_rx_ring *rxr) { int ret; - rndis_device *rndis_dev; netvsc_device_info *dev_info = (netvsc_device_info *)additl_info; device_t dev = sc->hn_dev; struct hn_nvs_subch_req *req; const struct hn_nvs_subch_resp *resp; size_t resp_len; struct vmbus_xact *xact = NULL; uint32_t status, nsubch; int nchan = *nchan0; int rxr_cnt; - rndis_dev = hv_get_rndis_device(); - if (rndis_dev == NULL) { - return (ENOMEM); - } - sc->rndis_dev = rndis_dev; - rndis_dev->sc = sc; - /* * Let the inner driver handle this first to create the netvsc channel * NOTE! Once the channel is created, we may get a receive callback * (hv_rf_on_receive()) before this call is completed. * Note: Earlier code used a function pointer here. */ ret = hv_nv_on_device_add(sc, rxr); - if (ret != 0) { - hv_put_rndis_device(rndis_dev); + if (ret != 0) return (ret); - } /* * Initialize the rndis device */ /* Send the rndis initialization message */ - ret = hv_rf_init_device(rndis_dev); + ret = hv_rf_init_device(sc); if (ret != 0) { /* * TODO: If rndis init failed, we will need to shut down * the channel */ } /* Get the mac address */ - ret = hv_rf_query_device_mac(rndis_dev); + ret = hv_rf_query_device_mac(sc, dev_info->mac_addr); if (ret != 0) { /* TODO: shut down rndis device and the channel */ } /* Configure NDIS offload settings */ hn_rndis_conf_offload(sc); - - memcpy(dev_info->mac_addr, rndis_dev->hw_mac_addr, ETHER_ADDR_LEN); - hv_rf_query_device_link_status(rndis_dev); - - dev_info->link_state = rndis_dev->link_status; + hv_rf_query_device_link_status(sc, &dev_info->link_state); if (sc->hn_ndis_ver < NDIS_VERSION_6_30 || nchan == 1) { /* * Either RSS is not supported, or multiple RX/TX rings * are not requested. */ *nchan0 = 1; return (0); } /* * Get RSS capabilities, e.g. # of RX rings, and # of indirect * table entries. */ ret = hn_rndis_get_rsscaps(sc, &rxr_cnt); if (ret) { /* No RSS; this is benign. */ *nchan0 = 1; return (0); } if (nchan > rxr_cnt) nchan = rxr_cnt; if_printf(sc->hn_ifp, "RX rings offered %u, requested %d\n", rxr_cnt, nchan); if (nchan == 1) { device_printf(dev, "only 1 channel is supported, no vRSS\n"); goto out; } /* * Ask NVS to allocate sub-channels. */ xact = vmbus_xact_get(sc->hn_xact, sizeof(*req)); if (xact == NULL) { if_printf(sc->hn_ifp, "no xact for nvs subch req\n"); ret = ENXIO; goto out; } req = vmbus_xact_req_data(xact); req->nvs_type = HN_NVS_TYPE_SUBCH_REQ; req->nvs_op = HN_NVS_SUBCH_OP_ALLOC; req->nvs_nsubch = nchan - 1; - resp = hn_nvs_xact_execute(sc, xact, req, sizeof(*req), &resp_len); + resp_len = sizeof(*resp); + resp = hn_nvs_xact_execute(sc, xact, req, sizeof(*req), &resp_len, + HN_NVS_TYPE_SUBCH_RESP); if (resp == NULL) { if_printf(sc->hn_ifp, "exec subch failed\n"); ret = EIO; goto out; } - if (resp_len < sizeof(*resp)) { - if_printf(sc->hn_ifp, "invalid subch resp length %zu\n", - resp_len); - ret = EINVAL; - goto out; - } - if (resp->nvs_type != HN_NVS_TYPE_SUBCH_RESP) { - if_printf(sc->hn_ifp, "not subch resp, type %u\n", - resp->nvs_type); - ret = EINVAL; - goto out; - } status = resp->nvs_status; nsubch = resp->nvs_nsubch; vmbus_xact_put(xact); xact = NULL; if (status != HN_NVS_STATUS_OK) { if_printf(sc->hn_ifp, "subch req failed: %x\n", status); ret = EIO; goto out; } if (nsubch > nchan - 1) { if_printf(sc->hn_ifp, "%u subchans are allocated, requested %u\n", nsubch, nchan - 1); nsubch = nchan - 1; } nchan = nsubch + 1; ret = hn_rndis_conf_rss(sc, nchan); if (ret != 0) *nchan0 = 1; else *nchan0 = nchan; out: if (xact != NULL) vmbus_xact_put(xact); return (ret); } /* * RNDIS filter on device remove */ int -hv_rf_on_device_remove(struct hn_softc *sc, boolean_t destroy_channel) +hv_rf_on_device_remove(struct hn_softc *sc) { - rndis_device *rndis_dev = sc->rndis_dev; int ret; /* Halt and release the rndis device */ - ret = hv_rf_halt_device(rndis_dev); + ret = hv_rf_halt_device(sc); - sc->rndis_dev = NULL; - hv_put_rndis_device(rndis_dev); - /* Pass control to inner driver to remove the device */ - ret |= hv_nv_on_device_remove(sc, destroy_channel); + ret |= hv_nv_on_device_remove(sc); return (ret); } /* * RNDIS filter on open */ int hv_rf_on_open(struct hn_softc *sc) { + uint32_t filter; - return (hv_rf_open_device(sc->rndis_dev)); + /* XXX */ + if (hv_promisc_mode != 1) { + filter = NDIS_PACKET_TYPE_BROADCAST | + NDIS_PACKET_TYPE_ALL_MULTICAST | + NDIS_PACKET_TYPE_DIRECTED; + } else { + filter = NDIS_PACKET_TYPE_PROMISCUOUS; + } + return (hn_rndis_set_rxfilter(sc, filter)); } /* * RNDIS filter on close */ int hv_rf_on_close(struct hn_softc *sc) { - return (hv_rf_close_device(sc->rndis_dev)); -} - -static void -hn_rndis_sent_cb(struct hn_send_ctx *sndc, struct hn_softc *sc, - struct vmbus_channel *chan __unused, const void *data __unused, - int dlen __unused) -{ - if (sndc->hn_chim_idx != HN_NVS_CHIM_IDX_INVALID) - hn_chim_free(sc, sndc->hn_chim_idx); -} - -static void -hn_rndis_sent_halt(struct hn_send_ctx *sndc, struct hn_softc *sc, - struct vmbus_channel *chan __unused, const void *data __unused, - int dlen __unused) -{ - rndis_request *request = sndc->hn_cbarg; - - if (sndc->hn_chim_idx != HN_NVS_CHIM_IDX_INVALID) - hn_chim_free(sc, sndc->hn_chim_idx); - - /* - * Notify hv_rf_halt_device() about halt completion. - * The halt code must wait for completion before freeing - * the transaction resources. - */ - request->halt_complete_flag = 1; + return (hn_rndis_set_rxfilter(sc, 0)); } void hv_rf_channel_rollup(struct hn_rx_ring *rxr, struct hn_tx_ring *txr) { netvsc_channel_rollup(rxr, txr); } Index: projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis_filter.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis_filter.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/hv_rndis_filter.h (revision 305172) @@ -1,127 +1,53 @@ /*- * Copyright (c) 2009-2012,2016 Microsoft Corp. * Copyright (c) 2010-2012 Citrix Inc. * Copyright (c) 2012 NetApp Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ #ifndef __HV_RNDIS_FILTER_H__ #define __HV_RNDIS_FILTER_H__ #include #include #include /* - * Defines - */ - -/* Destroy or preserve channel on filter/netvsc teardown */ -#define HV_RF_NV_DESTROY_CHANNEL TRUE -#define HV_RF_NV_RETAIN_CHANNEL FALSE - -/* - * Number of page buffers to reserve for the RNDIS filter packet in the - * transmitted message. - */ -#define HV_RF_NUM_TX_RESERVED_PAGE_BUFS 1 - - -/* - * Data types - */ - -typedef enum { - RNDIS_DEV_UNINITIALIZED = 0, - RNDIS_DEV_INITIALIZING, - RNDIS_DEV_INITIALIZED, - RNDIS_DEV_DATAINITIALIZED, -} rndis_device_state; - -typedef struct rndis_request_ { - STAILQ_ENTRY(rndis_request_) mylist_entry; - struct sema wait_sema; - - /* - * The max response size is sizeof(rndis_msg) + PAGE_SIZE. - * - * XXX - * This is ugly and should be cleaned up once we busdma-fy - * RNDIS request bits. - */ - rndis_msg response_msg; - uint8_t buf_resp[PAGE_SIZE]; - - /* Simplify allocation by having a netvsc packet inline */ - struct hn_send_ctx send_ctx; - - /* - * The max request size is sizeof(rndis_msg) + PAGE_SIZE. - * - * NOTE: - * This is required for the large request like RSS settings. - * - * XXX - * This is ugly and should be cleaned up once we busdma-fy - * RNDIS request bits. - */ - rndis_msg request_msg; - uint8_t buf_req[PAGE_SIZE]; - - /* Fixme: Poor man's semaphore. */ - uint32_t halt_complete_flag; -} rndis_request; - -typedef struct rndis_device_ { - struct hn_softc *sc; - - rndis_device_state state; - uint32_t link_status; - uint32_t new_request_id; - - struct mtx req_lock; - - STAILQ_HEAD(RQ, rndis_request_) myrequest_list; - - uint8_t hw_mac_addr[ETHER_ADDR_LEN]; -} rndis_device; - -/* * Externs */ struct hn_rx_ring; int hv_rf_on_receive(struct hn_softc *sc, struct hn_rx_ring *rxr, const void *data, int dlen); void hv_rf_channel_rollup(struct hn_rx_ring *rxr, struct hn_tx_ring *txr); int hv_rf_on_device_add(struct hn_softc *sc, void *additl_info, int *nchan, struct hn_rx_ring *rxr); -int hv_rf_on_device_remove(struct hn_softc *sc, boolean_t destroy_channel); +int hv_rf_on_device_remove(struct hn_softc *sc); int hv_rf_on_open(struct hn_softc *sc); int hv_rf_on_close(struct hn_softc *sc); #endif /* __HV_RNDIS_FILTER_H__ */ Index: projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/if_hnvar.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/if_hnvar.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/if_hnvar.h (revision 305172) @@ -1,120 +1,122 @@ /*- * Copyright (c) 2016 Microsoft Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _IF_HNVAR_H_ #define _IF_HNVAR_H_ #include #include #include struct hn_softc; struct vmbus_channel; struct hn_send_ctx; typedef void (*hn_sent_callback_t) (struct hn_send_ctx *, struct hn_softc *, struct vmbus_channel *, const void *, int); struct hn_send_ctx { hn_sent_callback_t hn_cb; void *hn_cbarg; uint32_t hn_chim_idx; int hn_chim_sz; }; struct rndis_hash_info; struct rndix_hash_value; struct ndis_8021q_info_; struct rndis_tcp_ip_csum_info_; struct hn_recvinfo { const struct ndis_8021q_info_ *vlan_info; const struct rndis_tcp_ip_csum_info_ *csum_info; const struct rndis_hash_info *hash_info; const struct rndis_hash_value *hash_value; }; #define HN_SEND_CTX_INITIALIZER(cb, cbarg) \ { \ .hn_cb = cb, \ .hn_cbarg = cbarg, \ .hn_chim_idx = HN_NVS_CHIM_IDX_INVALID, \ .hn_chim_sz = 0 \ } static __inline void hn_send_ctx_init(struct hn_send_ctx *sndc, hn_sent_callback_t cb, void *cbarg, uint32_t chim_idx, int chim_sz) { sndc->hn_cb = cb; sndc->hn_cbarg = cbarg; sndc->hn_chim_idx = chim_idx; sndc->hn_chim_sz = chim_sz; } static __inline void hn_send_ctx_init_simple(struct hn_send_ctx *sndc, hn_sent_callback_t cb, void *cbarg) { hn_send_ctx_init(sndc, cb, cbarg, HN_NVS_CHIM_IDX_INVALID, 0); } static __inline int hn_nvs_send(struct vmbus_channel *chan, uint16_t flags, void *nvs_msg, int nvs_msglen, struct hn_send_ctx *sndc) { return (vmbus_chan_send(chan, VMBUS_CHANPKT_TYPE_INBAND, flags, nvs_msg, nvs_msglen, (uint64_t)(uintptr_t)sndc)); } static __inline int hn_nvs_send_sglist(struct vmbus_channel *chan, struct vmbus_gpa sg[], int sglen, void *nvs_msg, int nvs_msglen, struct hn_send_ctx *sndc) { return (vmbus_chan_send_sglist(chan, sg, sglen, nvs_msg, nvs_msglen, (uint64_t)(uintptr_t)sndc)); } struct vmbus_xact; const void *hn_nvs_xact_execute(struct hn_softc *sc, struct vmbus_xact *xact, void *req, int reqlen, - size_t *resp_len); + size_t *resp_len, uint32_t type); +void hn_nvs_sent_xact(struct hn_send_ctx *sndc, struct hn_softc *sc, + struct vmbus_channel *chan, const void *data, int dlen); uint32_t hn_chim_alloc(struct hn_softc *sc); void hn_chim_free(struct hn_softc *sc, uint32_t chim_idx); extern struct hn_send_ctx hn_send_ctx_none; #endif /* !_IF_HNVAR_H_ */ Index: projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/ndis.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/ndis.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/netvsc/ndis.h (revision 305172) @@ -1,217 +1,206 @@ /*- * Copyright (c) 2016 Microsoft Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _NET_NDIS_H_ #define _NET_NDIS_H_ -#define NDIS_MEDIA_STATE_CONNECTED 0 -#define NDIS_MEDIA_STATE_DISCONNECTED 1 +#define NDIS_MEDIA_STATE_CONNECTED 0 +#define NDIS_MEDIA_STATE_DISCONNECTED 1 -#define OID_GEN_RSS_CAPABILITIES 0x00010203 -#define OID_GEN_RSS_PARAMETERS 0x00010204 -#define OID_TCP_OFFLOAD_PARAMETERS 0xFC01020C +#define NDIS_OFFLOAD_SET_NOCHG 0 +#define NDIS_OFFLOAD_SET_ON 1 +#define NDIS_OFFLOAD_SET_OFF 2 -#define NDIS_OBJTYPE_DEFAULT 0x80 -#define NDIS_OBJTYPE_RSS_CAPS 0x88 -#define NDIS_OBJTYPE_RSS_PARAMS 0x89 - -/* common_set */ -#define NDIS_OFFLOAD_SET_NOCHG 0 -#define NDIS_OFFLOAD_SET_ON 1 -#define NDIS_OFFLOAD_SET_OFF 2 - /* a.k.a GRE MAC */ -#define NDIS_ENCAP_TYPE_NVGRE 0x00000001 +#define NDIS_ENCAP_TYPE_NVGRE 0x00000001 -#define NDIS_HASH_FUNCTION_MASK 0x000000FF /* see hash function */ -#define NDIS_HASH_TYPE_MASK 0x00FFFF00 /* see hash type */ +#define NDIS_HASH_FUNCTION_MASK 0x000000FF /* see hash function */ +#define NDIS_HASH_TYPE_MASK 0x00FFFF00 /* see hash type */ /* hash function */ -#define NDIS_HASH_FUNCTION_TOEPLITZ 0x00000001 +#define NDIS_HASH_FUNCTION_TOEPLITZ 0x00000001 /* hash type */ -#define NDIS_HASH_IPV4 0x00000100 -#define NDIS_HASH_TCP_IPV4 0x00000200 -#define NDIS_HASH_IPV6 0x00000400 -#define NDIS_HASH_IPV6_EX 0x00000800 -#define NDIS_HASH_TCP_IPV6 0x00001000 -#define NDIS_HASH_TCP_IPV6_EX 0x00002000 +#define NDIS_HASH_IPV4 0x00000100 +#define NDIS_HASH_TCP_IPV4 0x00000200 +#define NDIS_HASH_IPV6 0x00000400 +#define NDIS_HASH_IPV6_EX 0x00000800 +#define NDIS_HASH_TCP_IPV6 0x00001000 +#define NDIS_HASH_TCP_IPV6_EX 0x00002000 -#define NDIS_HASH_KEYSIZE_TOEPLITZ 40 -#define NDIS_HASH_INDCNT 128 +#define NDIS_HASH_KEYSIZE_TOEPLITZ 40 +#define NDIS_HASH_INDCNT 128 +#define NDIS_OBJTYPE_DEFAULT 0x80 +#define NDIS_OBJTYPE_RSS_CAPS 0x88 +#define NDIS_OBJTYPE_RSS_PARAMS 0x89 + struct ndis_object_hdr { - uint8_t ndis_type; /* NDIS_OBJTYPE_ */ - uint8_t ndis_rev; /* type specific */ - uint16_t ndis_size; /* incl. this hdr */ + uint8_t ndis_type; /* NDIS_OBJTYPE_ */ + uint8_t ndis_rev; /* type specific */ + uint16_t ndis_size; /* incl. this hdr */ }; /* * OID_TCP_OFFLOAD_PARAMETERS * ndis_type: NDIS_OBJTYPE_DEFAULT */ struct ndis_offload_params { struct ndis_object_hdr ndis_hdr; - uint8_t ndis_ip4csum; /* param_set */ - uint8_t ndis_tcp4csum; /* param_set */ - uint8_t ndis_udp4csum; /* param_set */ - uint8_t ndis_tcp6csum; /* param_set */ - uint8_t ndis_udp6csum; /* param_set */ - uint8_t ndis_lsov1; /* lsov1_set */ - uint8_t ndis_ipsecv1; /* ipsecv1_set */ - uint8_t ndis_lsov2_ip4; /* lsov2_set */ - uint8_t ndis_lsov2_ip6; /* lsov2_set */ - uint8_t ndis_tcp4conn; /* PARAM_NOCHG */ - uint8_t ndis_tcp6conn; /* PARAM_NOCHG */ - uint32_t ndis_flags; /* 0 */ + uint8_t ndis_ip4csum; /* NDIS_OFFLOAD_PARAM_ */ + uint8_t ndis_tcp4csum; /* NDIS_OFFLOAD_PARAM_ */ + uint8_t ndis_udp4csum; /* NDIS_OFFLOAD_PARAM_ */ + uint8_t ndis_tcp6csum; /* NDIS_OFFLOAD_PARAM_ */ + uint8_t ndis_udp6csum; /* NDIS_OFFLOAD_PARAM_ */ + uint8_t ndis_lsov1; /* NDIS_OFFLOAD_PARAM_ */ + uint8_t ndis_ipsecv1; /* NDIS_OFFLOAD_IPSECV1_ */ + uint8_t ndis_lsov2_ip4; /* NDIS_OFFLOAD_LSOV2_ */ + uint8_t ndis_lsov2_ip6; /* NDIS_OFFLOAD_LSOV2_ */ + uint8_t ndis_tcp4conn; /* 0 */ + uint8_t ndis_tcp6conn; /* 0 */ + uint32_t ndis_flags; /* 0 */ /* NDIS >= 6.1 */ - uint8_t ndis_ipsecv2; /* ipsecv2_set */ - uint8_t ndis_ipsecv2_ip4; /* ipsecv2_set */ + uint8_t ndis_ipsecv2; /* NDIS_OFFLOAD_IPSECV2_ */ + uint8_t ndis_ipsecv2_ip4;/* NDIS_OFFLOAD_IPSECV2_ */ /* NDIS >= 6.30 */ - uint8_t ndis_rsc_ip4; /* rsc_set */ - uint8_t ndis_rsc_ip6; /* rsc_set */ - uint8_t ndis_encap; /* common_set */ - uint8_t ndis_encap_types; /* NDIS_ENCAP_TYPE_ */ + uint8_t ndis_rsc_ip4; /* NDIS_OFFLOAD_RSC_ */ + uint8_t ndis_rsc_ip6; /* NDIS_OFFLOAD_RSC_ */ + uint8_t ndis_encap; /* NDIS_OFFLOAD_SET_ */ + uint8_t ndis_encap_types;/* NDIS_ENCAP_TYPE_ */ }; -#define NDIS_OFFLOAD_PARAMS_SIZE sizeof(struct ndis_offload_params) -#define NDIS_OFFLOAD_PARAMS_SIZE_6_1 \ +#define NDIS_OFFLOAD_PARAMS_SIZE sizeof(struct ndis_offload_params) +#define NDIS_OFFLOAD_PARAMS_SIZE_6_1 \ __offsetof(struct ndis_offload_params, ndis_rsc_ip4) -#define NDIS_OFFLOAD_PARAMS_REV_2 2 /* NDIS 6.1 */ -#define NDIS_OFFLOAD_PARAMS_REV_3 3 /* NDIS 6.30 */ +#define NDIS_OFFLOAD_PARAMS_REV_2 2 /* NDIS 6.1 */ +#define NDIS_OFFLOAD_PARAMS_REV_3 3 /* NDIS 6.30 */ -/* param_set */ -#define NDIS_OFFLOAD_PARAM_NOCHG 0 /* common to all sets */ -#define NDIS_OFFLOAD_PARAM_OFF 1 -#define NDIS_OFFLOAD_PARAM_TX 2 -#define NDIS_OFFLOAD_PARAM_RX 3 -#define NDIS_OFFLOAD_PARAM_TXRX 4 +#define NDIS_OFFLOAD_PARAM_NOCHG 0 /* common */ +#define NDIS_OFFLOAD_PARAM_OFF 1 +#define NDIS_OFFLOAD_PARAM_TX 2 +#define NDIS_OFFLOAD_PARAM_RX 3 +#define NDIS_OFFLOAD_PARAM_TXRX 4 -/* lsov1_set */ /* NDIS_OFFLOAD_PARAM_NOCHG */ -#define NDIS_OFFLOAD_LSOV1_OFF 1 -#define NDIS_OFFLOAD_LSOV1_ON 2 +#define NDIS_OFFLOAD_LSOV1_OFF 1 +#define NDIS_OFFLOAD_LSOV1_ON 2 -/* ipsecv1_set */ /* NDIS_OFFLOAD_PARAM_NOCHG */ -#define NDIS_OFFLOAD_IPSECV1_OFF 1 -#define NDIS_OFFLOAD_IPSECV1_AH 2 -#define NDIS_OFFLOAD_IPSECV1_ESP 3 -#define NDIS_OFFLOAD_IPSECV1_AH_ESP 4 +#define NDIS_OFFLOAD_IPSECV1_OFF 1 +#define NDIS_OFFLOAD_IPSECV1_AH 2 +#define NDIS_OFFLOAD_IPSECV1_ESP 3 +#define NDIS_OFFLOAD_IPSECV1_AH_ESP 4 -/* lsov2_set */ /* NDIS_OFFLOAD_PARAM_NOCHG */ -#define NDIS_OFFLOAD_LSOV2_OFF 1 -#define NDIS_OFFLOAD_LSOV2_ON 2 +#define NDIS_OFFLOAD_LSOV2_OFF 1 +#define NDIS_OFFLOAD_LSOV2_ON 2 -/* ipsecv2_set */ /* NDIS_OFFLOAD_PARAM_NOCHG */ -#define NDIS_OFFLOAD_IPSECV2_OFF 1 -#define NDIS_OFFLOAD_IPSECV2_AH 2 -#define NDIS_OFFLOAD_IPSECV2_ESP 3 -#define NDIS_OFFLOAD_IPSECV2_AH_ESP 4 +#define NDIS_OFFLOAD_IPSECV2_OFF 1 +#define NDIS_OFFLOAD_IPSECV2_AH 2 +#define NDIS_OFFLOAD_IPSECV2_ESP 3 +#define NDIS_OFFLOAD_IPSECV2_AH_ESP 4 -/* rsc_set */ /* NDIS_OFFLOAD_PARAM_NOCHG */ -#define NDIS_OFFLOAD_RSC_OFF 1 -#define NDIS_OFFLOAD_RSC_ON 2 +#define NDIS_OFFLOAD_RSC_OFF 1 +#define NDIS_OFFLOAD_RSC_ON 2 /* - * OID_GEN_RSS_CAPABILITIES + * OID_GEN_RECEIVE_SCALE_CAPABILITIES * ndis_type: NDIS_OBJTYPE_RSS_CAPS */ struct ndis_rss_caps { struct ndis_object_hdr ndis_hdr; uint32_t ndis_flags; /* NDIS_RSS_CAP_ */ uint32_t ndis_nmsi; /* # of MSIs */ uint32_t ndis_nrxr; /* # of RX rings */ /* NDIS >= 6.30 */ uint16_t ndis_nind; /* # of indtbl ent. */ uint16_t ndis_pad; }; -#define NDIS_RSS_CAPS_SIZE \ +#define NDIS_RSS_CAPS_SIZE \ __offsetof(struct ndis_rss_caps, ndis_pad) -#define NDIS_RSS_CAPS_SIZE_6_0 \ +#define NDIS_RSS_CAPS_SIZE_6_0 \ __offsetof(struct ndis_rss_caps, ndis_nind) -#define NDIS_RSS_CAPS_REV_1 1 /* NDIS 6.{0,1,20} */ -#define NDIS_RSS_CAPS_REV_2 2 /* NDIS 6.30 */ +#define NDIS_RSS_CAPS_REV_1 1 /* NDIS 6.{0,1,20} */ +#define NDIS_RSS_CAPS_REV_2 2 /* NDIS 6.30 */ -#define NDIS_RSS_CAP_MSI 0x01000000 -#define NDIS_RSS_CAP_CLASSIFY_ISR 0x02000000 -#define NDIS_RSS_CAP_CLASSIFY_DPC 0x04000000 -#define NDIS_RSS_CAP_MSIX 0x08000000 -#define NDIS_RSS_CAP_IPV4 0x00000100 -#define NDIS_RSS_CAP_IPV6 0x00000200 -#define NDIS_RSS_CAP_IPV6_EX 0x00000400 -#define NDIS_RSS_CAP_HASH_TOEPLITZ 0x00000001 +#define NDIS_RSS_CAP_MSI 0x01000000 +#define NDIS_RSS_CAP_CLASSIFY_ISR 0x02000000 +#define NDIS_RSS_CAP_CLASSIFY_DPC 0x04000000 +#define NDIS_RSS_CAP_MSIX 0x08000000 +#define NDIS_RSS_CAP_IPV4 0x00000100 +#define NDIS_RSS_CAP_IPV6 0x00000200 +#define NDIS_RSS_CAP_IPV6_EX 0x00000400 +#define NDIS_RSS_CAP_HASH_TOEPLITZ 0x00000001 /* - * OID_GEN_RSS_PARAMETERS + * OID_GEN_RECEIVE_SCALE_PARAMETERS * ndis_type: NDIS_OBJTYPE_RSS_PARAMS */ struct ndis_rss_params { struct ndis_object_hdr ndis_hdr; uint16_t ndis_flags; /* NDIS_RSS_FLAG_ */ uint16_t ndis_bcpu; /* base cpu 0 */ uint32_t ndis_hash; /* NDIS_HASH_ */ uint16_t ndis_indsize; /* indirect table */ uint32_t ndis_indoffset; uint16_t ndis_keysize; /* hash key */ uint32_t ndis_keyoffset; /* NDIS >= 6.20 */ uint32_t ndis_cpumaskoffset; uint32_t ndis_cpumaskcnt; uint32_t ndis_cpumaskentsz; }; -#define NDIS_RSS_PARAMS_SIZE sizeof(struct ndis_rss_params) -#define NDIS_RSS_PARAMS_SIZE_6_0 \ +#define NDIS_RSS_PARAMS_SIZE sizeof(struct ndis_rss_params) +#define NDIS_RSS_PARAMS_SIZE_6_0 \ __offsetof(struct ndis_rss_params, ndis_cpumaskoffset) -#define NDIS_RSS_PARAMS_REV_1 1 /* NDIS 6.0 */ -#define NDIS_RSS_PARAMS_REV_2 2 /* NDIS 6.20 */ +#define NDIS_RSS_PARAMS_REV_1 1 /* NDIS 6.0 */ +#define NDIS_RSS_PARAMS_REV_2 2 /* NDIS 6.20 */ -#define NDIS_RSS_FLAG_BCPU_UNCHG 0x0001 -#define NDIS_RSS_FLAG_HASH_UNCHG 0x0002 -#define NDIS_RSS_FLAG_IND_UNCHG 0x0004 -#define NDIS_RSS_FLAG_KEY_UNCHG 0x0008 -#define NDIS_RSS_FLAG_DISABLE 0x0010 +#define NDIS_RSS_FLAG_BCPU_UNCHG 0x0001 +#define NDIS_RSS_FLAG_HASH_UNCHG 0x0002 +#define NDIS_RSS_FLAG_IND_UNCHG 0x0004 +#define NDIS_RSS_FLAG_KEY_UNCHG 0x0008 +#define NDIS_RSS_FLAG_DISABLE 0x0010 /* non-standard convenient struct */ struct ndis_rssprm_toeplitz { struct ndis_rss_params rss_params; /* Toeplitz hash key */ uint8_t rss_key[NDIS_HASH_KEYSIZE_TOEPLITZ]; /* Indirect table */ uint32_t rss_ind[NDIS_HASH_INDCNT]; }; #endif /* !_NET_NDIS_H_ */ Index: projects/netbsd-tests-update-12/sys/dev/hyperv/utilities/hv_timesync.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/hyperv/utilities/hv_timesync.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/hyperv/utilities/hv_timesync.c (revision 305172) @@ -1,222 +1,231 @@ /*- * Copyright (c) 2014,2016 Microsoft Corp. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice unmodified, this list of conditions, and the following * disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ /* * A common driver for all hyper-V util services. */ #include #include #include #include #include #include #include #include #include -#include +#include #include #include #include #include "hv_util.h" #include "vmbus_if.h" #define HV_WLTIMEDELTA 116444736000000000L /* in 100ns unit */ #define HV_ICTIMESYNCFLAG_PROBE 0 #define HV_ICTIMESYNCFLAG_SYNC 1 #define HV_ICTIMESYNCFLAG_SAMPLE 2 #define HV_NANO_SEC_PER_SEC 1000000000 +#define HV_NANO_SEC_PER_MILLI_SEC 1000000 -/* Time Sync data */ -typedef struct { - uint64_t data; -} time_sync_data; - static const struct vmbus_ic_desc vmbus_timesync_descs[] = { { .ic_guid = { .hv_guid = { 0x30, 0xe6, 0x27, 0x95, 0xae, 0xd0, 0x7b, 0x49, 0xad, 0xce, 0xe8, 0x0a, 0xb0, 0x17, 0x5c, 0xaf } }, .ic_desc = "Hyper-V Timesync" }, VMBUS_IC_DESC_END }; struct hv_ictimesync_data { uint64_t parenttime; uint64_t childtime; uint64_t roundtriptime; uint8_t flags; } __packed; -typedef struct hv_timesync_sc { - hv_util_sc util_sc; - struct task task; - time_sync_data time_msg; -} hv_timesync_sc; +/* + * Globals + */ +SYSCTL_NODE(_hw, OID_AUTO, hvtimesync, CTLFLAG_RW | CTLFLAG_MPSAFE, NULL, + "Hyper-V timesync interface"); +/* Ignore the sync request when set to 1. */ +static int ignore_sync_req = 0; +SYSCTL_INT(_hw_hvtimesync, OID_AUTO, ignore_sync_req, CTLFLAG_RWTUN, + &ignore_sync_req, 0, + "Ignore the sync request when set to 1."); + +/* + * Trigger sample sync when drift exceeds threshold (ms). + * Ignore the sample request when set to 0. + */ +static int sample_drift = 100; +SYSCTL_INT(_hw_hvtimesync, OID_AUTO, sample_drift, CTLFLAG_RWTUN, + &sample_drift, 0, + "Threshold that makes sample request trigger the sync."); + /** - * Set host time based on time sync message from host + * @brief Synchronize time with host after reboot, restore, etc. + * + * ICTIMESYNCFLAG_SYNC flag bit indicates reboot, restore events of the VM. + * After reboot the flag ICTIMESYNCFLAG_SYNC is included in the first time + * message after the timesync channel is opened. Since the hv_utils module is + * loaded after hv_vmbus, the first message is usually missed. The other + * thing is, systime is automatically set to emulated hardware clock which may + * not be UTC time or in the same time zone. So, to override these effects, we + * use the first 50 time samples for initial system time setting. */ -static void -hv_set_host_time(void *context, int pending) +static inline +void hv_adj_guesttime(hv_util_sc *sc, uint64_t hosttime, uint8_t flags) { - hv_timesync_sc *softc = (hv_timesync_sc*)context; - uint64_t hosttime = softc->time_msg.data; struct timespec guest_ts, host_ts; - uint64_t host_tns; + uint64_t host_tns, guest_tns; int64_t diff; int error; host_tns = (hosttime - HV_WLTIMEDELTA) * 100; host_ts.tv_sec = (time_t)(host_tns/HV_NANO_SEC_PER_SEC); host_ts.tv_nsec = (long)(host_tns%HV_NANO_SEC_PER_SEC); nanotime(&guest_ts); + guest_tns = guest_ts.tv_sec * HV_NANO_SEC_PER_SEC + guest_ts.tv_nsec; - diff = (int64_t)host_ts.tv_sec - (int64_t)guest_ts.tv_sec; + if ((flags & HV_ICTIMESYNCFLAG_SYNC) != 0 && ignore_sync_req == 0) { + if (bootverbose) { + device_printf(sc->ic_dev, "handle sync request " + "{host: %ju, guest: %ju}\n", + (uintmax_t)host_tns, (uintmax_t)guest_tns); + } - /* - * If host differs by 5 seconds then make the guest catch up - */ - if (diff > 5 || diff < -5) { error = kern_clock_settime(curthread, CLOCK_REALTIME, &host_ts); + return; } -} -/** - * @brief Synchronize time with host after reboot, restore, etc. - * - * ICTIMESYNCFLAG_SYNC flag bit indicates reboot, restore events of the VM. - * After reboot the flag ICTIMESYNCFLAG_SYNC is included in the first time - * message after the timesync channel is opened. Since the hv_utils module is - * loaded after hv_vmbus, the first message is usually missed. The other - * thing is, systime is automatically set to emulated hardware clock which may - * not be UTC time or in the same time zone. So, to override these effects, we - * use the first 50 time samples for initial system time setting. - */ -static inline -void hv_adj_guesttime(hv_timesync_sc *sc, uint64_t hosttime, uint8_t flags) -{ - sc->time_msg.data = hosttime; + if ((flags & HV_ICTIMESYNCFLAG_SAMPLE) != 0 && sample_drift != 0) { + if (bootverbose) { + device_printf(sc->ic_dev, "handle sample request " + "{host: %ju, guest: %ju}\n", + (uintmax_t)host_tns, (uintmax_t)guest_tns); + } - if (((flags & HV_ICTIMESYNCFLAG_SYNC) != 0) || - ((flags & HV_ICTIMESYNCFLAG_SAMPLE) != 0)) { - taskqueue_enqueue(taskqueue_thread, &sc->task); + diff = (int64_t)(host_tns - guest_tns) / HV_NANO_SEC_PER_MILLI_SEC; + if (diff > sample_drift || diff < -sample_drift) { + error = kern_clock_settime(curthread, CLOCK_REALTIME, + &host_ts); + if (bootverbose) + device_printf(sc->ic_dev, "trigger sample sync"); + } + return; } } /** * Time Sync Channel message handler */ static void hv_timesync_cb(struct vmbus_channel *channel, void *context) { hv_vmbus_icmsg_hdr* icmsghdrp; uint32_t recvlen; uint64_t requestId; int ret; uint8_t* time_buf; struct hv_ictimesync_data* timedatap; - hv_timesync_sc *softc; + hv_util_sc *softc; - softc = (hv_timesync_sc*)context; - time_buf = softc->util_sc.receive_buffer; + softc = (hv_util_sc*)context; + time_buf = softc->receive_buffer; - recvlen = softc->util_sc.ic_buflen; + recvlen = softc->ic_buflen; ret = vmbus_chan_recv(channel, time_buf, &recvlen, &requestId); KASSERT(ret != ENOBUFS, ("hvtimesync recvbuf is not large enough")); /* XXX check recvlen to make sure that it contains enough data */ if ((ret == 0) && recvlen > 0) { icmsghdrp = (struct hv_vmbus_icmsg_hdr *) &time_buf[ sizeof(struct hv_vmbus_pipe_hdr)]; if (icmsghdrp->icmsgtype == HV_ICMSGTYPE_NEGOTIATE) { int error; - error = vmbus_ic_negomsg(&softc->util_sc, time_buf, &recvlen); + error = vmbus_ic_negomsg(softc, time_buf, &recvlen); if (error) return; } else { timedatap = (struct hv_ictimesync_data *) &time_buf[ sizeof(struct hv_vmbus_pipe_hdr) + sizeof(struct hv_vmbus_icmsg_hdr)]; hv_adj_guesttime(softc, timedatap->parenttime, timedatap->flags); } icmsghdrp->icflags = HV_ICMSGHDRFLAG_TRANSACTION | HV_ICMSGHDRFLAG_RESPONSE; vmbus_chan_send(channel, VMBUS_CHANPKT_TYPE_INBAND, 0, time_buf, recvlen, requestId); } } static int hv_timesync_probe(device_t dev) { return (vmbus_ic_probe(dev, vmbus_timesync_descs)); } static int hv_timesync_attach(device_t dev) { - hv_timesync_sc *softc = device_get_softc(dev); - - TASK_INIT(&softc->task, 1, hv_set_host_time, softc); return hv_util_attach(dev, hv_timesync_cb); } static int hv_timesync_detach(device_t dev) { - hv_timesync_sc *softc = device_get_softc(dev); - - taskqueue_drain(taskqueue_thread, &softc->task); return hv_util_detach(dev); } static device_method_t timesync_methods[] = { /* Device interface */ DEVMETHOD(device_probe, hv_timesync_probe), DEVMETHOD(device_attach, hv_timesync_attach), DEVMETHOD(device_detach, hv_timesync_detach), { 0, 0 } }; -static driver_t timesync_driver = { "hvtimesync", timesync_methods, sizeof(hv_timesync_sc)}; +static driver_t timesync_driver = { "hvtimesync", timesync_methods, sizeof(hv_util_sc)}; static devclass_t timesync_devclass; DRIVER_MODULE(hv_timesync, vmbus, timesync_driver, timesync_devclass, NULL, NULL); MODULE_VERSION(hv_timesync, 1); MODULE_DEPEND(hv_timesync, vmbus, 1, 1, 1); Index: projects/netbsd-tests-update-12/sys/dev/iscsi_initiator/iscsi.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/iscsi_initiator/iscsi.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/iscsi_initiator/iscsi.h (revision 305172) @@ -1,500 +1,500 @@ /*- * Copyright (c) 2005-2010 Daniel Braniss * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* | $Id: iscsi.h 743 2009-08-08 10:54:53Z danny $ */ #define TRUE 1 #define FALSE 0 #ifndef _KERNEL typedef int boolean_t; #endif #include #define ISCSIDEV "iscsi" #define ISCSI_MAX_TARGETS 64 /* | iSCSI commands */ /* | Initiator Opcodes: */ #define ISCSI_NOP_OUT 0x00 #define ISCSI_SCSI_CMD 0x01 #define ISCSI_TASK_CMD 0x02 #define ISCSI_LOGIN_CMD 0x03 #define ISCSI_TEXT_CMD 0x04 #define ISCSI_WRITE_DATA 0x05 #define ISCSI_LOGOUT_CMD 0x06 #define ISCSI_SNACK 0x10 /* | Target Opcodes: */ #define ISCSI_NOP_IN 0x20 #define ISCSI_SCSI_RSP 0x21 #define ISCSI_TASK_RSP 0x22 #define ISCSI_LOGIN_RSP 0x23 #define ISCSI_TEXT_RSP 0x24 #define ISCSI_READ_DATA 0x25 #define ISCSI_LOGOUT_RSP 0x26 #define ISCSI_R2T 0x31 #define ISCSI_ASYNC 0x32 #define ISCSI_REJECT 0x3f /* | PDU stuff */ /* | BHS Basic Header Segment */ typedef struct bhs { // the order is network byte order! u_char opcode:6; u_char I:1; u_char _:1; u_char __:7; u_char F:1; // Final bit u_char ___[2]; u_int AHSLength:8; // in 4byte words u_int DSLength:24; // in bytes u_int LUN[2]; // or Opcode-specific fields u_int itt; u_int OpcodeSpecificFields[7]; #define CmdSN OpcodeSpecificFields[1] #define ExpStSN OpcodeSpecificFields[2] #define MaxCmdSN OpcodeSpecificFields[3] } bhs_t; typedef struct ahs { u_int len:16; u_int type:8; u_int spec:8; char data[0]; } ahs_t; typedef struct { // Sequence Numbers // (computers were invented to count, right?) int cmd; int expcmd; int maxcmd; } req_sn_t; typedef struct { // Sequence Numbers // (computers were invented to count, right?) int stat; int expcmd; int maxcmd; } rsp_sn_t; typedef struct scsi_req { u_char opcode:6; // 0x01 u_char I:1; u_char _:1; u_char attr:3; u_char _0:2; u_char W:1; u_char R:1; u_char F:1; #define iSCSI_TASK_UNTAGGED 0 #define iSCSI_TASK_SIMPLE 1 #define iSCSI_TASK_ORDER 2 #define iSCSI_TASK_HOFQ 3 #define iSCSI_TASK_ACA 4 char _1[2]; int len; int lun[2]; int itt; int edtlen; // expectect data transfere length int cmdSN; int extStatSN; int cdb[4]; } scsi_req_t; typedef struct scsi_rsp { char opcode; // 0x21 u_char flag; u_char response; u_char status; int len; int _[2]; int itt; int stag; rsp_sn_t sn; int expdatasn; int bdrcnt; // bidirectional residual count int rcnt; // residual count } scsi_rsp_t; typedef struct nop_out { // the order is network byte order! u_char opcode:6; u_char I:1; u_char _:1; u_char __:7; u_char F:1; // Final bit u_char ___[2]; u_int len; u_int lun[2]; u_int itt; u_int ttt; req_sn_t sn; u_int mbz[3]; } nop_out_t; typedef struct nop_in { // the order is network byte order! u_char opcode:6; u_char I:1; u_char _:1; u_char __:7; u_char F:1; // Final bit u_char ___[2]; u_int len; u_int lun[2]; u_int itt; u_int ttt; rsp_sn_t sn; u_int ____[2]; } nop_in_t; typedef struct r2t { u_char opcode:6; u_char I:1; u_char _:1; u_char __:7; u_char F:1; // Final bit u_char ___[2]; u_int len; u_int lun[2]; u_int itt; u_int ttt; rsp_sn_t sn; u_int r2tSN; u_int bo; u_int ddtl; } r2t_t; typedef struct data_out { u_char opcode:6; u_char I:1; u_char _:1; u_char __:7; u_char F:1; // Final bit u_char ___[2]; u_int len; u_int lun[2]; u_int itt; u_int ttt; rsp_sn_t sn; u_int dsn; // data seq. number u_int bo; u_int ____; } data_out_t; typedef struct data_in { u_char opcode:6; u_char I:1; u_char _:1; u_char S:1; u_char U:1; u_char O:1; u_char __:3; u_char A:1; u_char F:1; // Final bit u_char ___[1]; u_char status; u_int len; u_int lun[2]; u_int itt; u_int ttt; rsp_sn_t sn; u_int dataSN; u_int bo; u_int ____; } data_in_t; typedef struct reject { u_char opcode:6; u_char _:2; u_char F:1; u_char __:7; u_char reason; u_char ___; u_int len; u_int ____[2]; u_int tt[2]; // must be -1 rsp_sn_t sn; u_int dataSN; // or R2TSN or reserved u_int _____[2]; } reject_t; typedef struct async { u_char opcode:6; u_char _:2; u_char F:1; u_char __:7; u_char ___[2]; u_int len; u_int lun[2]; u_int itt; // must be -1 u_int ____; rsp_sn_t sn; u_char asyncEvent; u_char asyncVCode; u_char param1[2]; u_char param2[2]; u_char param3[2]; u_int _____; } async_t; typedef struct login_req { char cmd; // 0x03 u_char NSG:2; u_char CSG:2; u_char _:2; u_char C:1; u_char T:1; char v_max; char v_min; int len; // remapped via standard bhs char isid[6]; short tsih; int itt; // Initiator Task Tag; int CID:16; int rsv:16; int cmdSN; int expStatSN; int unused[4]; } login_req_t; typedef struct login_rsp { char cmd; // 0x23 u_char NSG:2; u_char CSG:2; u_char _1:2; u_char C:1; u_char T:1; char v_max; char v_act; int len; // remapped via standard bhs char isid[6]; short tsih; int itt; // Initiator Task Tag; int _2; rsp_sn_t sn; int status:16; int _3:16; int _4[2]; } login_rsp_t; typedef struct text_req { char cmd; // 0x04 u_char _1:6; u_char C:1; // Continuation u_char F:1; // Final char _2[2]; int len; int itt; // Initiator Task Tag int LUN[2]; int ttt; // Target Transfer Tag int cmdSN; int expStatSN; int unused[4]; } text_req_t; typedef struct logout_req { char cmd; // 0x06 - char reason; // 0 - close session + u_char reason; // 0 - close session // 1 - close connection // 2 - remove the connection for recovery char _2[2]; int len; int _r[2]; int itt; // Initiator Task Tag; u_int CID:16; u_int rsv:16; int cmdSN; int expStatSN; int unused[4]; } logout_req_t; typedef struct logout_rsp { char cmd; // 0x26 char cbits; char _1[2]; int len; int _2[2]; int itt; int _3; rsp_sn_t sn; short time2wait; short time2retain; int _4; } logout_rsp_t; union ipdu_u { bhs_t bhs; scsi_req_t scsi_req; scsi_rsp_t scsi_rsp; nop_out_t nop_out; nop_in_t nop_in; r2t_t r2t; data_out_t data_out; data_in_t data_in; reject_t reject; async_t async; }; /* | Sequence Numbers */ typedef struct { u_int itt; u_int cmd; u_int expCmd; u_int maxCmd; u_int stat; u_int expStat; u_int data; } sn_t; /* | in-core version of a Protocol Data Unit */ typedef struct { union ipdu_u ipdu; u_int hdr_dig; // header digest ahs_t *ahs_addr; u_int ahs_len; u_int ahs_size; // the allocated size u_char *ds_addr; u_int ds_len; u_int ds_size; // the allocated size u_int ds_dig; // data digest } pdu_t; typedef struct opvals { int port; int tags; int maxluns; int sockbufsize; int maxConnections; int maxRecvDataSegmentLength; int maxXmitDataSegmentLength; // pseudo ... int maxBurstLength; int firstBurstLength; int defaultTime2Wait; int defaultTime2Retain; int maxOutstandingR2T; int errorRecoveryLevel; int targetPortalGroupTag; boolean_t initialR2T; boolean_t immediateData; boolean_t dataPDUInOrder; boolean_t dataSequenceInOrder; char *headerDigest; char *dataDigest; char *sessionType; char *sendTargets; char *targetAddress; char *targetAlias; char *targetName; char *initiatorName; char *initiatorAlias; char *authMethod; char *chapSecret; char *chapIName; char *chapDigest; char *tgtChapName; char *tgtChapSecret; int tgtChallengeLen; u_char tgtChapID; char *tgtChapDigest; char *iqn; char *pidfile; } isc_opt_t; /* | ioctl */ #define ISCSISETSES _IOR('i', 1, int) #define ISCSISETSOC _IOW('i', 2, int) #define ISCSISETOPT _IOW('i', 5, isc_opt_t) #define ISCSIGETOPT _IOR('i', 6, isc_opt_t) #define ISCSISEND _IOW('i', 10, pdu_t) #define ISCSIRECV _IOWR('i', 11, pdu_t) #define ISCSIPING _IO('i', 20) #define ISCSISIGNAL _IOW('i', 21, int *) #define ISCSISTART _IO('i', 30) #define ISCSIRESTART _IO('i', 31) #define ISCSISTOP _IO('i', 32) typedef struct iscsi_cam { path_id_t path_id; target_id_t target_id; int target_nluns; } iscsi_cam_t; #define ISCSIGETCAM _IOR('i', 33, iscsi_cam_t) Index: projects/netbsd-tests-update-12/sys/dev/ixl/if_ixlv.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/ixl/if_ixlv.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/ixl/if_ixlv.c (revision 305172) @@ -1,3065 +1,3060 @@ /****************************************************************************** Copyright (c) 2013-2015, Intel Corporation All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the Intel Corporation nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ******************************************************************************/ /*$FreeBSD$*/ #include "ixl.h" #include "ixlv.h" /********************************************************************* * Driver version *********************************************************************/ char ixlv_driver_version[] = "1.4.6-k"; /********************************************************************* * PCI Device ID Table * * Used by probe to select devices to load on * Last field stores an index into ixlv_strings * Last entry must be all 0s * * { Vendor ID, Device ID, SubVendor ID, SubDevice ID, String Index } *********************************************************************/ static ixl_vendor_info_t ixlv_vendor_info_array[] = { {I40E_INTEL_VENDOR_ID, I40E_DEV_ID_VF, 0, 0, 0}, {I40E_INTEL_VENDOR_ID, I40E_DEV_ID_VF_HV, 0, 0, 0}, {I40E_INTEL_VENDOR_ID, I40E_DEV_ID_X722_VF, 0, 0, 0}, {I40E_INTEL_VENDOR_ID, I40E_DEV_ID_X722_A0_VF, 0, 0, 0}, {I40E_INTEL_VENDOR_ID, I40E_DEV_ID_X722_VF_HV, 0, 0, 0}, /* required last entry */ {0, 0, 0, 0, 0} }; /********************************************************************* * Table of branding strings *********************************************************************/ static char *ixlv_strings[] = { "Intel(R) Ethernet Connection XL710/X722 VF Driver" }; /********************************************************************* * Function prototypes *********************************************************************/ static int ixlv_probe(device_t); static int ixlv_attach(device_t); static int ixlv_detach(device_t); static int ixlv_shutdown(device_t); static void ixlv_init_locked(struct ixlv_sc *); static int ixlv_allocate_pci_resources(struct ixlv_sc *); static void ixlv_free_pci_resources(struct ixlv_sc *); static int ixlv_assign_msix(struct ixlv_sc *); static int ixlv_init_msix(struct ixlv_sc *); static int ixlv_init_taskqueue(struct ixlv_sc *); static int ixlv_setup_queues(struct ixlv_sc *); static void ixlv_config_rss(struct ixlv_sc *); static void ixlv_stop(struct ixlv_sc *); static void ixlv_add_multi(struct ixl_vsi *); static void ixlv_del_multi(struct ixl_vsi *); static void ixlv_free_queues(struct ixl_vsi *); static int ixlv_setup_interface(device_t, struct ixlv_sc *); static int ixlv_media_change(struct ifnet *); static void ixlv_media_status(struct ifnet *, struct ifmediareq *); static void ixlv_local_timer(void *); static int ixlv_add_mac_filter(struct ixlv_sc *, u8 *, u16); static int ixlv_del_mac_filter(struct ixlv_sc *sc, u8 *macaddr); static void ixlv_init_filters(struct ixlv_sc *); static void ixlv_free_filters(struct ixlv_sc *); static void ixlv_msix_que(void *); static void ixlv_msix_adminq(void *); static void ixlv_do_adminq(void *, int); static void ixlv_do_adminq_locked(struct ixlv_sc *sc); static void ixlv_handle_que(void *, int); static int ixlv_reset(struct ixlv_sc *); static int ixlv_reset_complete(struct i40e_hw *); static void ixlv_set_queue_rx_itr(struct ixl_queue *); static void ixlv_set_queue_tx_itr(struct ixl_queue *); static void ixl_init_cmd_complete(struct ixl_vc_cmd *, void *, enum i40e_status_code); static void ixlv_configure_itr(struct ixlv_sc *); static void ixlv_enable_adminq_irq(struct i40e_hw *); static void ixlv_disable_adminq_irq(struct i40e_hw *); static void ixlv_enable_queue_irq(struct i40e_hw *, int); static void ixlv_disable_queue_irq(struct i40e_hw *, int); static void ixlv_setup_vlan_filters(struct ixlv_sc *); static void ixlv_register_vlan(void *, struct ifnet *, u16); static void ixlv_unregister_vlan(void *, struct ifnet *, u16); static void ixlv_init_hw(struct ixlv_sc *); static int ixlv_setup_vc(struct ixlv_sc *); static int ixlv_vf_config(struct ixlv_sc *); static void ixlv_cap_txcsum_tso(struct ixl_vsi *, struct ifnet *, int); static void ixlv_add_sysctls(struct ixlv_sc *); #ifdef IXL_DEBUG static int ixlv_sysctl_qtx_tail_handler(SYSCTL_HANDLER_ARGS); static int ixlv_sysctl_qrx_tail_handler(SYSCTL_HANDLER_ARGS); #endif /********************************************************************* * FreeBSD Device Interface Entry Points *********************************************************************/ static device_method_t ixlv_methods[] = { /* Device interface */ DEVMETHOD(device_probe, ixlv_probe), DEVMETHOD(device_attach, ixlv_attach), DEVMETHOD(device_detach, ixlv_detach), DEVMETHOD(device_shutdown, ixlv_shutdown), {0, 0} }; static driver_t ixlv_driver = { "ixlv", ixlv_methods, sizeof(struct ixlv_sc), }; devclass_t ixlv_devclass; DRIVER_MODULE(ixlv, pci, ixlv_driver, ixlv_devclass, 0, 0); MODULE_DEPEND(ixlv, pci, 1, 1, 1); MODULE_DEPEND(ixlv, ether, 1, 1, 1); /* ** TUNEABLE PARAMETERS: */ static SYSCTL_NODE(_hw, OID_AUTO, ixlv, CTLFLAG_RD, 0, "IXLV driver parameters"); /* ** Number of descriptors per ring: ** - TX and RX are the same size */ static int ixlv_ringsz = DEFAULT_RING; TUNABLE_INT("hw.ixlv.ringsz", &ixlv_ringsz); SYSCTL_INT(_hw_ixlv, OID_AUTO, ring_size, CTLFLAG_RDTUN, &ixlv_ringsz, 0, "Descriptor Ring Size"); /* Set to zero to auto calculate */ int ixlv_max_queues = 0; TUNABLE_INT("hw.ixlv.max_queues", &ixlv_max_queues); SYSCTL_INT(_hw_ixlv, OID_AUTO, max_queues, CTLFLAG_RDTUN, &ixlv_max_queues, 0, "Number of Queues"); /* ** Number of entries in Tx queue buf_ring. ** Increasing this will reduce the number of ** errors when transmitting fragmented UDP ** packets. */ static int ixlv_txbrsz = DEFAULT_TXBRSZ; TUNABLE_INT("hw.ixlv.txbrsz", &ixlv_txbrsz); SYSCTL_INT(_hw_ixlv, OID_AUTO, txbr_size, CTLFLAG_RDTUN, &ixlv_txbrsz, 0, "TX Buf Ring Size"); /* ** Controls for Interrupt Throttling ** - true/false for dynamic adjustment ** - default values for static ITR */ int ixlv_dynamic_rx_itr = 0; TUNABLE_INT("hw.ixlv.dynamic_rx_itr", &ixlv_dynamic_rx_itr); SYSCTL_INT(_hw_ixlv, OID_AUTO, dynamic_rx_itr, CTLFLAG_RDTUN, &ixlv_dynamic_rx_itr, 0, "Dynamic RX Interrupt Rate"); int ixlv_dynamic_tx_itr = 0; TUNABLE_INT("hw.ixlv.dynamic_tx_itr", &ixlv_dynamic_tx_itr); SYSCTL_INT(_hw_ixlv, OID_AUTO, dynamic_tx_itr, CTLFLAG_RDTUN, &ixlv_dynamic_tx_itr, 0, "Dynamic TX Interrupt Rate"); int ixlv_rx_itr = IXL_ITR_8K; TUNABLE_INT("hw.ixlv.rx_itr", &ixlv_rx_itr); SYSCTL_INT(_hw_ixlv, OID_AUTO, rx_itr, CTLFLAG_RDTUN, &ixlv_rx_itr, 0, "RX Interrupt Rate"); int ixlv_tx_itr = IXL_ITR_4K; TUNABLE_INT("hw.ixlv.tx_itr", &ixlv_tx_itr); SYSCTL_INT(_hw_ixlv, OID_AUTO, tx_itr, CTLFLAG_RDTUN, &ixlv_tx_itr, 0, "TX Interrupt Rate"); -/* Fix when building as a standalone module when netmap is enabled */ -#if defined(DEV_NETMAP) && !defined(NETMAP_IXL_MAIN) -int ixl_rx_miss, ixl_rx_miss_bufs, ixl_crcstrip; -#endif - /********************************************************************* * Device identification routine * * ixlv_probe determines if the driver should be loaded on * the hardware based on PCI vendor/device id of the device. * * return BUS_PROBE_DEFAULT on success, positive on failure *********************************************************************/ static int ixlv_probe(device_t dev) { ixl_vendor_info_t *ent; u16 pci_vendor_id, pci_device_id; u16 pci_subvendor_id, pci_subdevice_id; char device_name[256]; #if 0 INIT_DEBUGOUT("ixlv_probe: begin"); #endif pci_vendor_id = pci_get_vendor(dev); if (pci_vendor_id != I40E_INTEL_VENDOR_ID) return (ENXIO); pci_device_id = pci_get_device(dev); pci_subvendor_id = pci_get_subvendor(dev); pci_subdevice_id = pci_get_subdevice(dev); ent = ixlv_vendor_info_array; while (ent->vendor_id != 0) { if ((pci_vendor_id == ent->vendor_id) && (pci_device_id == ent->device_id) && ((pci_subvendor_id == ent->subvendor_id) || (ent->subvendor_id == 0)) && ((pci_subdevice_id == ent->subdevice_id) || (ent->subdevice_id == 0))) { sprintf(device_name, "%s, Version - %s", ixlv_strings[ent->index], ixlv_driver_version); device_set_desc_copy(dev, device_name); return (BUS_PROBE_DEFAULT); } ent++; } return (ENXIO); } /********************************************************************* * Device initialization routine * * The attach entry point is called when the driver is being loaded. * This routine identifies the type of hardware, allocates all resources * and initializes the hardware. * * return 0 on success, positive on failure *********************************************************************/ static int ixlv_attach(device_t dev) { struct ixlv_sc *sc; struct i40e_hw *hw; struct ixl_vsi *vsi; int error = 0; INIT_DBG_DEV(dev, "begin"); /* Allocate, clear, and link in our primary soft structure */ sc = device_get_softc(dev); sc->dev = sc->osdep.dev = dev; hw = &sc->hw; vsi = &sc->vsi; vsi->dev = dev; /* Initialize hw struct */ ixlv_init_hw(sc); /* Allocate filter lists */ ixlv_init_filters(sc); /* Core Lock Init */ mtx_init(&sc->mtx, device_get_nameunit(dev), "IXL SC Lock", MTX_DEF); /* Set up the timer callout */ callout_init_mtx(&sc->timer, &sc->mtx, 0); /* Do PCI setup - map BAR0, etc */ if (ixlv_allocate_pci_resources(sc)) { device_printf(dev, "%s: Allocation of PCI resources failed\n", __func__); error = ENXIO; goto err_early; } INIT_DBG_DEV(dev, "Allocated PCI resources and MSIX vectors"); error = i40e_set_mac_type(hw); if (error) { device_printf(dev, "%s: set_mac_type failed: %d\n", __func__, error); goto err_pci_res; } error = ixlv_reset_complete(hw); if (error) { device_printf(dev, "%s: Device is still being reset\n", __func__); goto err_pci_res; } INIT_DBG_DEV(dev, "VF Device is ready for configuration"); error = ixlv_setup_vc(sc); if (error) { device_printf(dev, "%s: Error setting up PF comms, %d\n", __func__, error); goto err_pci_res; } INIT_DBG_DEV(dev, "PF API version verified"); /* Need API version before sending reset message */ error = ixlv_reset(sc); if (error) { device_printf(dev, "VF reset failed; reload the driver\n"); goto err_aq; } INIT_DBG_DEV(dev, "VF reset complete"); /* Ask for VF config from PF */ error = ixlv_vf_config(sc); if (error) { device_printf(dev, "Error getting configuration from PF: %d\n", error); goto err_aq; } device_printf(dev, "VSIs %d, QPs %d, MSIX %d, RSS sizes: key %d lut %d\n", sc->vf_res->num_vsis, sc->vf_res->num_queue_pairs, sc->vf_res->max_vectors, sc->vf_res->rss_key_size, sc->vf_res->rss_lut_size); #ifdef IXL_DEBUG device_printf(dev, "Offload flags: 0x%b\n", sc->vf_res->vf_offload_flags, IXLV_PRINTF_VF_OFFLOAD_FLAGS); #endif /* got VF config message back from PF, now we can parse it */ for (int i = 0; i < sc->vf_res->num_vsis; i++) { if (sc->vf_res->vsi_res[i].vsi_type == I40E_VSI_SRIOV) sc->vsi_res = &sc->vf_res->vsi_res[i]; } if (!sc->vsi_res) { device_printf(dev, "%s: no LAN VSI found\n", __func__); error = EIO; goto err_res_buf; } INIT_DBG_DEV(dev, "Resource Acquisition complete"); /* If no mac address was assigned just make a random one */ if (!ixlv_check_ether_addr(hw->mac.addr)) { u8 addr[ETHER_ADDR_LEN]; arc4rand(&addr, sizeof(addr), 0); addr[0] &= 0xFE; addr[0] |= 0x02; bcopy(addr, hw->mac.addr, sizeof(addr)); } /* Now that the number of queues for this VF is known, set up interrupts */ sc->msix = ixlv_init_msix(sc); /* We fail without MSIX support */ if (sc->msix == 0) { error = ENXIO; goto err_res_buf; } vsi->id = sc->vsi_res->vsi_id; vsi->back = (void *)sc; sc->link_up = TRUE; /* This allocates the memory and early settings */ if (ixlv_setup_queues(sc) != 0) { device_printf(dev, "%s: setup queues failed!\n", __func__); error = EIO; goto out; } /* Setup the stack interface */ if (ixlv_setup_interface(dev, sc) != 0) { device_printf(dev, "%s: setup interface failed!\n", __func__); error = EIO; goto out; } INIT_DBG_DEV(dev, "Queue memory and interface setup"); /* Do queue interrupt setup */ if (ixlv_assign_msix(sc) != 0) { device_printf(dev, "%s: allocating queue interrupts failed!\n", __func__); error = ENXIO; goto out; } /* Start AdminQ taskqueue */ ixlv_init_taskqueue(sc); /* Initialize stats */ bzero(&sc->vsi.eth_stats, sizeof(struct i40e_eth_stats)); ixlv_add_sysctls(sc); /* Register for VLAN events */ vsi->vlan_attach = EVENTHANDLER_REGISTER(vlan_config, ixlv_register_vlan, vsi, EVENTHANDLER_PRI_FIRST); vsi->vlan_detach = EVENTHANDLER_REGISTER(vlan_unconfig, ixlv_unregister_vlan, vsi, EVENTHANDLER_PRI_FIRST); /* We want AQ enabled early */ ixlv_enable_adminq_irq(hw); /* Set things up to run init */ sc->init_state = IXLV_INIT_READY; ixl_vc_init_mgr(sc, &sc->vc_mgr); INIT_DBG_DEV(dev, "end"); return (error); out: ixlv_free_queues(vsi); err_res_buf: free(sc->vf_res, M_DEVBUF); err_aq: i40e_shutdown_adminq(hw); err_pci_res: ixlv_free_pci_resources(sc); err_early: mtx_destroy(&sc->mtx); ixlv_free_filters(sc); INIT_DBG_DEV(dev, "end: error %d", error); return (error); } /********************************************************************* * Device removal routine * * The detach entry point is called when the driver is being removed. * This routine stops the adapter and deallocates all the resources * that were allocated for driver operation. * * return 0 on success, positive on failure *********************************************************************/ static int ixlv_detach(device_t dev) { struct ixlv_sc *sc = device_get_softc(dev); struct ixl_vsi *vsi = &sc->vsi; INIT_DBG_DEV(dev, "begin"); /* Make sure VLANS are not using driver */ if (vsi->ifp->if_vlantrunk != NULL) { if_printf(vsi->ifp, "Vlan in use, detach first\n"); INIT_DBG_DEV(dev, "end"); return (EBUSY); } /* Stop driver */ ether_ifdetach(vsi->ifp); if (vsi->ifp->if_drv_flags & IFF_DRV_RUNNING) { mtx_lock(&sc->mtx); ixlv_stop(sc); mtx_unlock(&sc->mtx); } /* Unregister VLAN events */ if (vsi->vlan_attach != NULL) EVENTHANDLER_DEREGISTER(vlan_config, vsi->vlan_attach); if (vsi->vlan_detach != NULL) EVENTHANDLER_DEREGISTER(vlan_unconfig, vsi->vlan_detach); /* Drain VC mgr */ callout_drain(&sc->vc_mgr.callout); i40e_shutdown_adminq(&sc->hw); taskqueue_free(sc->tq); if_free(vsi->ifp); free(sc->vf_res, M_DEVBUF); ixlv_free_pci_resources(sc); ixlv_free_queues(vsi); mtx_destroy(&sc->mtx); ixlv_free_filters(sc); bus_generic_detach(dev); INIT_DBG_DEV(dev, "end"); return (0); } /********************************************************************* * * Shutdown entry point * **********************************************************************/ static int ixlv_shutdown(device_t dev) { struct ixlv_sc *sc = device_get_softc(dev); INIT_DBG_DEV(dev, "begin"); mtx_lock(&sc->mtx); ixlv_stop(sc); mtx_unlock(&sc->mtx); INIT_DBG_DEV(dev, "end"); return (0); } /* * Configure TXCSUM(IPV6) and TSO(4/6) * - the hardware handles these together so we * need to tweak them */ static void ixlv_cap_txcsum_tso(struct ixl_vsi *vsi, struct ifnet *ifp, int mask) { /* Enable/disable TXCSUM/TSO4 */ if (!(ifp->if_capenable & IFCAP_TXCSUM) && !(ifp->if_capenable & IFCAP_TSO4)) { if (mask & IFCAP_TXCSUM) { ifp->if_capenable |= IFCAP_TXCSUM; /* enable TXCSUM, restore TSO if previously enabled */ if (vsi->flags & IXL_FLAGS_KEEP_TSO4) { vsi->flags &= ~IXL_FLAGS_KEEP_TSO4; ifp->if_capenable |= IFCAP_TSO4; } } else if (mask & IFCAP_TSO4) { ifp->if_capenable |= (IFCAP_TXCSUM | IFCAP_TSO4); vsi->flags &= ~IXL_FLAGS_KEEP_TSO4; if_printf(ifp, "TSO4 requires txcsum, enabling both...\n"); } } else if((ifp->if_capenable & IFCAP_TXCSUM) && !(ifp->if_capenable & IFCAP_TSO4)) { if (mask & IFCAP_TXCSUM) ifp->if_capenable &= ~IFCAP_TXCSUM; else if (mask & IFCAP_TSO4) ifp->if_capenable |= IFCAP_TSO4; } else if((ifp->if_capenable & IFCAP_TXCSUM) && (ifp->if_capenable & IFCAP_TSO4)) { if (mask & IFCAP_TXCSUM) { vsi->flags |= IXL_FLAGS_KEEP_TSO4; ifp->if_capenable &= ~(IFCAP_TXCSUM | IFCAP_TSO4); if_printf(ifp, "TSO4 requires txcsum, disabling both...\n"); } else if (mask & IFCAP_TSO4) ifp->if_capenable &= ~IFCAP_TSO4; } /* Enable/disable TXCSUM_IPV6/TSO6 */ if (!(ifp->if_capenable & IFCAP_TXCSUM_IPV6) && !(ifp->if_capenable & IFCAP_TSO6)) { if (mask & IFCAP_TXCSUM_IPV6) { ifp->if_capenable |= IFCAP_TXCSUM_IPV6; if (vsi->flags & IXL_FLAGS_KEEP_TSO6) { vsi->flags &= ~IXL_FLAGS_KEEP_TSO6; ifp->if_capenable |= IFCAP_TSO6; } } else if (mask & IFCAP_TSO6) { ifp->if_capenable |= (IFCAP_TXCSUM_IPV6 | IFCAP_TSO6); vsi->flags &= ~IXL_FLAGS_KEEP_TSO6; if_printf(ifp, "TSO6 requires txcsum6, enabling both...\n"); } } else if((ifp->if_capenable & IFCAP_TXCSUM_IPV6) && !(ifp->if_capenable & IFCAP_TSO6)) { if (mask & IFCAP_TXCSUM_IPV6) ifp->if_capenable &= ~IFCAP_TXCSUM_IPV6; else if (mask & IFCAP_TSO6) ifp->if_capenable |= IFCAP_TSO6; } else if ((ifp->if_capenable & IFCAP_TXCSUM_IPV6) && (ifp->if_capenable & IFCAP_TSO6)) { if (mask & IFCAP_TXCSUM_IPV6) { vsi->flags |= IXL_FLAGS_KEEP_TSO6; ifp->if_capenable &= ~(IFCAP_TXCSUM_IPV6 | IFCAP_TSO6); if_printf(ifp, "TSO6 requires txcsum6, disabling both...\n"); } else if (mask & IFCAP_TSO6) ifp->if_capenable &= ~IFCAP_TSO6; } } /********************************************************************* * Ioctl entry point * * ixlv_ioctl is called when the user wants to configure the * interface. * * return 0 on success, positive on failure **********************************************************************/ static int ixlv_ioctl(struct ifnet *ifp, u_long command, caddr_t data) { struct ixl_vsi *vsi = ifp->if_softc; struct ixlv_sc *sc = vsi->back; struct ifreq *ifr = (struct ifreq *)data; #if defined(INET) || defined(INET6) struct ifaddr *ifa = (struct ifaddr *)data; bool avoid_reset = FALSE; #endif int error = 0; switch (command) { case SIOCSIFADDR: #ifdef INET if (ifa->ifa_addr->sa_family == AF_INET) avoid_reset = TRUE; #endif #ifdef INET6 if (ifa->ifa_addr->sa_family == AF_INET6) avoid_reset = TRUE; #endif #if defined(INET) || defined(INET6) /* ** Calling init results in link renegotiation, ** so we avoid doing it when possible. */ if (avoid_reset) { ifp->if_flags |= IFF_UP; if (!(ifp->if_drv_flags & IFF_DRV_RUNNING)) ixlv_init(vsi); #ifdef INET if (!(ifp->if_flags & IFF_NOARP)) arp_ifinit(ifp, ifa); #endif } else error = ether_ioctl(ifp, command, data); break; #endif case SIOCSIFMTU: IOCTL_DBG_IF2(ifp, "SIOCSIFMTU (Set Interface MTU)"); mtx_lock(&sc->mtx); if (ifr->ifr_mtu > IXL_MAX_FRAME - ETHER_HDR_LEN - ETHER_CRC_LEN - ETHER_VLAN_ENCAP_LEN) { error = EINVAL; IOCTL_DBG_IF(ifp, "mtu too large"); } else { IOCTL_DBG_IF2(ifp, "mtu: %lu -> %d", (u_long)ifp->if_mtu, ifr->ifr_mtu); // ERJ: Interestingly enough, these types don't match ifp->if_mtu = (u_long)ifr->ifr_mtu; vsi->max_frame_size = ifp->if_mtu + ETHER_HDR_LEN + ETHER_CRC_LEN + ETHER_VLAN_ENCAP_LEN; if (ifp->if_drv_flags & IFF_DRV_RUNNING) ixlv_init_locked(sc); } mtx_unlock(&sc->mtx); break; case SIOCSIFFLAGS: IOCTL_DBG_IF2(ifp, "SIOCSIFFLAGS (Set Interface Flags)"); mtx_lock(&sc->mtx); if (ifp->if_flags & IFF_UP) { if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) ixlv_init_locked(sc); } else if (ifp->if_drv_flags & IFF_DRV_RUNNING) ixlv_stop(sc); sc->if_flags = ifp->if_flags; mtx_unlock(&sc->mtx); break; case SIOCADDMULTI: IOCTL_DBG_IF2(ifp, "SIOCADDMULTI"); if (ifp->if_drv_flags & IFF_DRV_RUNNING) { mtx_lock(&sc->mtx); ixlv_disable_intr(vsi); ixlv_add_multi(vsi); ixlv_enable_intr(vsi); mtx_unlock(&sc->mtx); } break; case SIOCDELMULTI: IOCTL_DBG_IF2(ifp, "SIOCDELMULTI"); if (sc->init_state == IXLV_RUNNING) { mtx_lock(&sc->mtx); ixlv_disable_intr(vsi); ixlv_del_multi(vsi); ixlv_enable_intr(vsi); mtx_unlock(&sc->mtx); } break; case SIOCSIFMEDIA: case SIOCGIFMEDIA: IOCTL_DBG_IF2(ifp, "SIOCxIFMEDIA (Get/Set Interface Media)"); error = ifmedia_ioctl(ifp, ifr, &sc->media, command); break; case SIOCSIFCAP: { int mask = ifr->ifr_reqcap ^ ifp->if_capenable; IOCTL_DBG_IF2(ifp, "SIOCSIFCAP (Set Capabilities)"); ixlv_cap_txcsum_tso(vsi, ifp, mask); if (mask & IFCAP_RXCSUM) ifp->if_capenable ^= IFCAP_RXCSUM; if (mask & IFCAP_RXCSUM_IPV6) ifp->if_capenable ^= IFCAP_RXCSUM_IPV6; if (mask & IFCAP_LRO) ifp->if_capenable ^= IFCAP_LRO; if (mask & IFCAP_VLAN_HWTAGGING) ifp->if_capenable ^= IFCAP_VLAN_HWTAGGING; if (mask & IFCAP_VLAN_HWFILTER) ifp->if_capenable ^= IFCAP_VLAN_HWFILTER; if (mask & IFCAP_VLAN_HWTSO) ifp->if_capenable ^= IFCAP_VLAN_HWTSO; if (ifp->if_drv_flags & IFF_DRV_RUNNING) { ixlv_init(vsi); } VLAN_CAPABILITIES(ifp); break; } default: IOCTL_DBG_IF2(ifp, "UNKNOWN (0x%X)", (int)command); error = ether_ioctl(ifp, command, data); break; } return (error); } /* ** To do a reinit on the VF is unfortunately more complicated ** than a physical device, we must have the PF more or less ** completely recreate our memory, so many things that were ** done only once at attach in traditional drivers now must be ** redone at each reinitialization. This function does that ** 'prelude' so we can then call the normal locked init code. */ int ixlv_reinit_locked(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; struct ixl_vsi *vsi = &sc->vsi; struct ifnet *ifp = vsi->ifp; struct ixlv_mac_filter *mf, *mf_temp; struct ixlv_vlan_filter *vf; int error = 0; INIT_DBG_IF(ifp, "begin"); if (ifp->if_drv_flags & IFF_DRV_RUNNING) ixlv_stop(sc); error = ixlv_reset(sc); INIT_DBG_IF(ifp, "VF was reset"); /* set the state in case we went thru RESET */ sc->init_state = IXLV_RUNNING; /* ** Resetting the VF drops all filters from hardware; ** we need to mark them to be re-added in init. */ SLIST_FOREACH_SAFE(mf, sc->mac_filters, next, mf_temp) { if (mf->flags & IXL_FILTER_DEL) { SLIST_REMOVE(sc->mac_filters, mf, ixlv_mac_filter, next); free(mf, M_DEVBUF); } else mf->flags |= IXL_FILTER_ADD; } if (vsi->num_vlans != 0) SLIST_FOREACH(vf, sc->vlan_filters, next) vf->flags = IXL_FILTER_ADD; else { /* clean any stale filters */ while (!SLIST_EMPTY(sc->vlan_filters)) { vf = SLIST_FIRST(sc->vlan_filters); SLIST_REMOVE_HEAD(sc->vlan_filters, next); free(vf, M_DEVBUF); } } ixlv_enable_adminq_irq(hw); ixl_vc_flush(&sc->vc_mgr); INIT_DBG_IF(ifp, "end"); return (error); } static void ixl_init_cmd_complete(struct ixl_vc_cmd *cmd, void *arg, enum i40e_status_code code) { struct ixlv_sc *sc; sc = arg; /* * Ignore "Adapter Stopped" message as that happens if an ifconfig down * happens while a command is in progress, so we don't print an error * in that case. */ if (code != I40E_SUCCESS && code != I40E_ERR_ADAPTER_STOPPED) { if_printf(sc->vsi.ifp, "Error %s waiting for PF to complete operation %d\n", i40e_stat_str(&sc->hw, code), cmd->request); } } static void ixlv_init_locked(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; struct ixl_vsi *vsi = &sc->vsi; struct ixl_queue *que = vsi->queues; struct ifnet *ifp = vsi->ifp; int error = 0; INIT_DBG_IF(ifp, "begin"); IXLV_CORE_LOCK_ASSERT(sc); /* Do a reinit first if an init has already been done */ if ((sc->init_state == IXLV_RUNNING) || (sc->init_state == IXLV_RESET_REQUIRED) || (sc->init_state == IXLV_RESET_PENDING)) error = ixlv_reinit_locked(sc); /* Don't bother with init if we failed reinit */ if (error) goto init_done; /* Remove existing MAC filter if new MAC addr is set */ if (bcmp(IF_LLADDR(ifp), hw->mac.addr, ETHER_ADDR_LEN) != 0) { error = ixlv_del_mac_filter(sc, hw->mac.addr); if (error == 0) ixl_vc_enqueue(&sc->vc_mgr, &sc->del_mac_cmd, IXLV_FLAG_AQ_DEL_MAC_FILTER, ixl_init_cmd_complete, sc); } /* Check for an LAA mac address... */ bcopy(IF_LLADDR(ifp), hw->mac.addr, ETHER_ADDR_LEN); ifp->if_hwassist = 0; if (ifp->if_capenable & IFCAP_TSO) ifp->if_hwassist |= CSUM_TSO; if (ifp->if_capenable & IFCAP_TXCSUM) ifp->if_hwassist |= (CSUM_OFFLOAD_IPV4 & ~CSUM_IP); if (ifp->if_capenable & IFCAP_TXCSUM_IPV6) ifp->if_hwassist |= CSUM_OFFLOAD_IPV6; /* Add mac filter for this VF to PF */ if (i40e_validate_mac_addr(hw->mac.addr) == I40E_SUCCESS) { error = ixlv_add_mac_filter(sc, hw->mac.addr, 0); if (!error || error == EEXIST) ixl_vc_enqueue(&sc->vc_mgr, &sc->add_mac_cmd, IXLV_FLAG_AQ_ADD_MAC_FILTER, ixl_init_cmd_complete, sc); } /* Setup vlan's if needed */ ixlv_setup_vlan_filters(sc); /* Prepare the queues for operation */ for (int i = 0; i < vsi->num_queues; i++, que++) { struct rx_ring *rxr = &que->rxr; ixl_init_tx_ring(que); if (vsi->max_frame_size <= MCLBYTES) rxr->mbuf_sz = MCLBYTES; else rxr->mbuf_sz = MJUMPAGESIZE; ixl_init_rx_ring(que); } /* Set initial ITR values */ ixlv_configure_itr(sc); /* Configure queues */ ixl_vc_enqueue(&sc->vc_mgr, &sc->config_queues_cmd, IXLV_FLAG_AQ_CONFIGURE_QUEUES, ixl_init_cmd_complete, sc); /* Set up RSS */ ixlv_config_rss(sc); /* Map vectors */ ixl_vc_enqueue(&sc->vc_mgr, &sc->map_vectors_cmd, IXLV_FLAG_AQ_MAP_VECTORS, ixl_init_cmd_complete, sc); /* Enable queues */ ixl_vc_enqueue(&sc->vc_mgr, &sc->enable_queues_cmd, IXLV_FLAG_AQ_ENABLE_QUEUES, ixl_init_cmd_complete, sc); /* Start the local timer */ callout_reset(&sc->timer, hz, ixlv_local_timer, sc); sc->init_state = IXLV_RUNNING; init_done: INIT_DBG_IF(ifp, "end"); return; } /* ** Init entry point for the stack */ void ixlv_init(void *arg) { struct ixl_vsi *vsi = (struct ixl_vsi *)arg; struct ixlv_sc *sc = vsi->back; int retries = 0; /* Prevent init from running again while waiting for AQ calls * made in init_locked() to complete. */ mtx_lock(&sc->mtx); if (sc->init_in_progress) { mtx_unlock(&sc->mtx); return; } else sc->init_in_progress = true; ixlv_init_locked(sc); mtx_unlock(&sc->mtx); /* Wait for init_locked to finish */ while (!(vsi->ifp->if_drv_flags & IFF_DRV_RUNNING) && ++retries < IXLV_AQ_MAX_ERR) { i40e_msec_pause(25); } if (retries >= IXLV_AQ_MAX_ERR) { if_printf(vsi->ifp, "Init failed to complete in allotted time!\n"); } mtx_lock(&sc->mtx); sc->init_in_progress = false; mtx_unlock(&sc->mtx); } /* * ixlv_attach() helper function; gathers information about * the (virtual) hardware for use elsewhere in the driver. */ static void ixlv_init_hw(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; device_t dev = sc->dev; /* Save off the information about this board */ hw->vendor_id = pci_get_vendor(dev); hw->device_id = pci_get_device(dev); hw->revision_id = pci_read_config(dev, PCIR_REVID, 1); hw->subsystem_vendor_id = pci_read_config(dev, PCIR_SUBVEND_0, 2); hw->subsystem_device_id = pci_read_config(dev, PCIR_SUBDEV_0, 2); hw->bus.device = pci_get_slot(dev); hw->bus.func = pci_get_function(dev); } /* * ixlv_attach() helper function; initalizes the admin queue * and attempts to establish contact with the PF by * retrying the initial "API version" message several times * or until the PF responds. */ static int ixlv_setup_vc(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; device_t dev = sc->dev; int error = 0, ret_error = 0, asq_retries = 0; bool send_api_ver_retried = 0; /* Need to set these AQ paramters before initializing AQ */ hw->aq.num_arq_entries = IXL_AQ_LEN; hw->aq.num_asq_entries = IXL_AQ_LEN; hw->aq.arq_buf_size = IXL_AQ_BUF_SZ; hw->aq.asq_buf_size = IXL_AQ_BUF_SZ; for (int i = 0; i < IXLV_AQ_MAX_ERR; i++) { /* Initialize admin queue */ error = i40e_init_adminq(hw); if (error) { device_printf(dev, "%s: init_adminq failed: %d\n", __func__, error); ret_error = 1; continue; } INIT_DBG_DEV(dev, "Initialized Admin Queue; starting" " send_api_ver attempt %d", i+1); retry_send: /* Send VF's API version */ error = ixlv_send_api_ver(sc); if (error) { i40e_shutdown_adminq(hw); ret_error = 2; device_printf(dev, "%s: unable to send api" " version to PF on attempt %d, error %d\n", __func__, i+1, error); } asq_retries = 0; while (!i40e_asq_done(hw)) { if (++asq_retries > IXLV_AQ_MAX_ERR) { i40e_shutdown_adminq(hw); device_printf(dev, "Admin Queue timeout " "(waiting for send_api_ver), %d more tries...\n", IXLV_AQ_MAX_ERR - (i + 1)); ret_error = 3; break; } i40e_msec_pause(10); } if (asq_retries > IXLV_AQ_MAX_ERR) continue; INIT_DBG_DEV(dev, "Sent API version message to PF"); /* Verify that the VF accepts the PF's API version */ error = ixlv_verify_api_ver(sc); if (error == ETIMEDOUT) { if (!send_api_ver_retried) { /* Resend message, one more time */ send_api_ver_retried++; device_printf(dev, "%s: Timeout while verifying API version on first" " try!\n", __func__); goto retry_send; } else { device_printf(dev, "%s: Timeout while verifying API version on second" " try!\n", __func__); ret_error = 4; break; } } if (error) { device_printf(dev, "%s: Unable to verify API version," " error %s\n", __func__, i40e_stat_str(hw, error)); ret_error = 5; } break; } if (ret_error >= 4) i40e_shutdown_adminq(hw); return (ret_error); } /* * ixlv_attach() helper function; asks the PF for this VF's * configuration, and saves the information if it receives it. */ static int ixlv_vf_config(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; device_t dev = sc->dev; int bufsz, error = 0, ret_error = 0; int asq_retries, retried = 0; retry_config: error = ixlv_send_vf_config_msg(sc); if (error) { device_printf(dev, "%s: Unable to send VF config request, attempt %d," " error %d\n", __func__, retried + 1, error); ret_error = 2; } asq_retries = 0; while (!i40e_asq_done(hw)) { if (++asq_retries > IXLV_AQ_MAX_ERR) { device_printf(dev, "%s: Admin Queue timeout " "(waiting for send_vf_config_msg), attempt %d\n", __func__, retried + 1); ret_error = 3; goto fail; } i40e_msec_pause(10); } INIT_DBG_DEV(dev, "Sent VF config message to PF, attempt %d", retried + 1); if (!sc->vf_res) { bufsz = sizeof(struct i40e_virtchnl_vf_resource) + (I40E_MAX_VF_VSI * sizeof(struct i40e_virtchnl_vsi_resource)); sc->vf_res = malloc(bufsz, M_DEVBUF, M_NOWAIT); if (!sc->vf_res) { device_printf(dev, "%s: Unable to allocate memory for VF configuration" " message from PF on attempt %d\n", __func__, retried + 1); ret_error = 1; goto fail; } } /* Check for VF config response */ error = ixlv_get_vf_config(sc); if (error == ETIMEDOUT) { /* The 1st time we timeout, send the configuration message again */ if (!retried) { retried++; goto retry_config; } device_printf(dev, "%s: ixlv_get_vf_config() timed out waiting for a response\n", __func__); } if (error) { device_printf(dev, "%s: Unable to get VF configuration from PF after %d tries!\n", __func__, retried + 1); ret_error = 4; } goto done; fail: free(sc->vf_res, M_DEVBUF); done: return (ret_error); } /* * Allocate MSI/X vectors, setup the AQ vector early */ static int ixlv_init_msix(struct ixlv_sc *sc) { device_t dev = sc->dev; int rid, want, vectors, queues, available; int auto_max_queues; rid = PCIR_BAR(IXL_BAR); sc->msix_mem = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, RF_ACTIVE); if (!sc->msix_mem) { /* May not be enabled */ device_printf(sc->dev, "Unable to map MSIX table\n"); goto fail; } available = pci_msix_count(dev); if (available == 0) { /* system has msix disabled */ bus_release_resource(dev, SYS_RES_MEMORY, rid, sc->msix_mem); sc->msix_mem = NULL; goto fail; } /* Clamp queues to number of CPUs and # of MSI-X vectors available */ auto_max_queues = min(mp_ncpus, available - 1); /* Clamp queues to # assigned to VF by PF */ auto_max_queues = min(auto_max_queues, sc->vf_res->num_queue_pairs); /* Override with tunable value if tunable is less than autoconfig count */ if ((ixlv_max_queues != 0) && (ixlv_max_queues <= auto_max_queues)) queues = ixlv_max_queues; /* Use autoconfig amount if that's lower */ else if ((ixlv_max_queues != 0) && (ixlv_max_queues > auto_max_queues)) { device_printf(dev, "ixlv_max_queues (%d) is too large, using " "autoconfig amount (%d)...\n", ixlv_max_queues, auto_max_queues); queues = auto_max_queues; } /* Limit maximum auto-configured queues to 8 if no user value is set */ else queues = min(auto_max_queues, 8); #ifdef RSS /* If we're doing RSS, clamp at the number of RSS buckets */ if (queues > rss_getnumbuckets()) queues = rss_getnumbuckets(); #endif /* ** Want one vector (RX/TX pair) per queue ** plus an additional for the admin queue. */ want = queues + 1; if (want <= available) /* Have enough */ vectors = want; else { device_printf(sc->dev, "MSIX Configuration Problem, " "%d vectors available but %d wanted!\n", available, want); goto fail; } #ifdef RSS /* * If we're doing RSS, the number of queues needs to * match the number of RSS buckets that are configured. * * + If there's more queues than RSS buckets, we'll end * up with queues that get no traffic. * * + If there's more RSS buckets than queues, we'll end * up having multiple RSS buckets map to the same queue, * so there'll be some contention. */ if (queues != rss_getnumbuckets()) { device_printf(dev, "%s: queues (%d) != RSS buckets (%d)" "; performance will be impacted.\n", __func__, queues, rss_getnumbuckets()); } #endif if (pci_alloc_msix(dev, &vectors) == 0) { device_printf(sc->dev, "Using MSIX interrupts with %d vectors\n", vectors); sc->msix = vectors; sc->vsi.num_queues = queues; } /* Next we need to setup the vector for the Admin Queue */ rid = 1; // zero vector + 1 sc->res = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); if (sc->res == NULL) { device_printf(dev,"Unable to allocate" " bus resource: AQ interrupt \n"); goto fail; } if (bus_setup_intr(dev, sc->res, INTR_TYPE_NET | INTR_MPSAFE, NULL, ixlv_msix_adminq, sc, &sc->tag)) { sc->res = NULL; device_printf(dev, "Failed to register AQ handler"); goto fail; } bus_describe_intr(dev, sc->res, sc->tag, "adminq"); return (vectors); fail: /* The VF driver MUST use MSIX */ return (0); } static int ixlv_allocate_pci_resources(struct ixlv_sc *sc) { int rid; device_t dev = sc->dev; rid = PCIR_BAR(0); sc->pci_mem = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, RF_ACTIVE); if (!(sc->pci_mem)) { device_printf(dev, "Unable to allocate bus resource: memory\n"); return (ENXIO); } sc->osdep.mem_bus_space_tag = rman_get_bustag(sc->pci_mem); sc->osdep.mem_bus_space_handle = rman_get_bushandle(sc->pci_mem); sc->osdep.mem_bus_space_size = rman_get_size(sc->pci_mem); sc->osdep.flush_reg = I40E_VFGEN_RSTAT; sc->hw.hw_addr = (u8 *) &sc->osdep.mem_bus_space_handle; sc->hw.back = &sc->osdep; /* ** Explicitly set the guest PCI BUSMASTER capability ** and we must rewrite the ENABLE in the MSIX control ** register again at this point to cause the host to ** successfully initialize us. ** ** This must be set before accessing any registers. */ { u16 pci_cmd_word; int msix_ctrl; pci_cmd_word = pci_read_config(dev, PCIR_COMMAND, 2); pci_cmd_word |= PCIM_CMD_BUSMASTEREN; pci_write_config(dev, PCIR_COMMAND, pci_cmd_word, 2); pci_find_cap(dev, PCIY_MSIX, &rid); rid += PCIR_MSIX_CTRL; msix_ctrl = pci_read_config(dev, rid, 2); msix_ctrl |= PCIM_MSIXCTRL_MSIX_ENABLE; pci_write_config(dev, rid, msix_ctrl, 2); } /* Disable adminq interrupts (just in case) */ ixlv_disable_adminq_irq(&sc->hw); return (0); } static void ixlv_free_pci_resources(struct ixlv_sc *sc) { struct ixl_vsi *vsi = &sc->vsi; struct ixl_queue *que = vsi->queues; device_t dev = sc->dev; /* We may get here before stations are setup */ if (que == NULL) goto early; /* ** Release all msix queue resources: */ for (int i = 0; i < vsi->num_queues; i++, que++) { int rid = que->msix + 1; if (que->tag != NULL) { bus_teardown_intr(dev, que->res, que->tag); que->tag = NULL; } if (que->res != NULL) { bus_release_resource(dev, SYS_RES_IRQ, rid, que->res); que->res = NULL; } } early: /* Clean the AdminQ interrupt */ if (sc->tag != NULL) { bus_teardown_intr(dev, sc->res, sc->tag); sc->tag = NULL; } if (sc->res != NULL) { bus_release_resource(dev, SYS_RES_IRQ, 1, sc->res); sc->res = NULL; } pci_release_msi(dev); if (sc->msix_mem != NULL) bus_release_resource(dev, SYS_RES_MEMORY, PCIR_BAR(IXL_BAR), sc->msix_mem); if (sc->pci_mem != NULL) bus_release_resource(dev, SYS_RES_MEMORY, PCIR_BAR(0), sc->pci_mem); } /* * Create taskqueue and tasklet for Admin Queue interrupts. */ static int ixlv_init_taskqueue(struct ixlv_sc *sc) { int error = 0; TASK_INIT(&sc->aq_irq, 0, ixlv_do_adminq, sc); sc->tq = taskqueue_create_fast("ixl_adm", M_NOWAIT, taskqueue_thread_enqueue, &sc->tq); taskqueue_start_threads(&sc->tq, 1, PI_NET, "%s sc->tq", device_get_nameunit(sc->dev)); return (error); } /********************************************************************* * * Setup MSIX Interrupt resources and handlers for the VSI queues * **********************************************************************/ static int ixlv_assign_msix(struct ixlv_sc *sc) { device_t dev = sc->dev; struct ixl_vsi *vsi = &sc->vsi; struct ixl_queue *que = vsi->queues; struct tx_ring *txr; int error, rid, vector = 1; #ifdef RSS cpuset_t cpu_mask; #endif for (int i = 0; i < vsi->num_queues; i++, vector++, que++) { int cpu_id = i; rid = vector + 1; txr = &que->txr; que->res = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_SHAREABLE | RF_ACTIVE); if (que->res == NULL) { device_printf(dev,"Unable to allocate" " bus resource: que interrupt [%d]\n", vector); return (ENXIO); } /* Set the handler function */ error = bus_setup_intr(dev, que->res, INTR_TYPE_NET | INTR_MPSAFE, NULL, ixlv_msix_que, que, &que->tag); if (error) { que->res = NULL; device_printf(dev, "Failed to register que handler"); return (error); } bus_describe_intr(dev, que->res, que->tag, "que %d", i); /* Bind the vector to a CPU */ #ifdef RSS cpu_id = rss_getcpu(i % rss_getnumbuckets()); #endif bus_bind_intr(dev, que->res, cpu_id); que->msix = vector; TASK_INIT(&que->tx_task, 0, ixl_deferred_mq_start, que); TASK_INIT(&que->task, 0, ixlv_handle_que, que); que->tq = taskqueue_create_fast("ixlv_que", M_NOWAIT, taskqueue_thread_enqueue, &que->tq); #ifdef RSS CPU_SETOF(cpu_id, &cpu_mask); taskqueue_start_threads_cpuset(&que->tq, 1, PI_NET, &cpu_mask, "%s (bucket %d)", device_get_nameunit(dev), cpu_id); #else taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que", device_get_nameunit(dev)); #endif } return (0); } /* ** Requests a VF reset from the PF. ** ** Requires the VF's Admin Queue to be initialized. */ static int ixlv_reset(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; device_t dev = sc->dev; int error = 0; /* Ask the PF to reset us if we are initiating */ if (sc->init_state != IXLV_RESET_PENDING) ixlv_request_reset(sc); i40e_msec_pause(100); error = ixlv_reset_complete(hw); if (error) { device_printf(dev, "%s: VF reset failed\n", __func__); return (error); } error = i40e_shutdown_adminq(hw); if (error) { device_printf(dev, "%s: shutdown_adminq failed: %d\n", __func__, error); return (error); } error = i40e_init_adminq(hw); if (error) { device_printf(dev, "%s: init_adminq failed: %d\n", __func__, error); return(error); } return (0); } static int ixlv_reset_complete(struct i40e_hw *hw) { u32 reg; /* Wait up to ~10 seconds */ for (int i = 0; i < 100; i++) { reg = rd32(hw, I40E_VFGEN_RSTAT) & I40E_VFGEN_RSTAT_VFR_STATE_MASK; if ((reg == I40E_VFR_VFACTIVE) || (reg == I40E_VFR_COMPLETED)) return (0); i40e_msec_pause(100); } return (EBUSY); } /********************************************************************* * * Setup networking device structure and register an interface. * **********************************************************************/ static int ixlv_setup_interface(device_t dev, struct ixlv_sc *sc) { struct ifnet *ifp; struct ixl_vsi *vsi = &sc->vsi; struct ixl_queue *que = vsi->queues; INIT_DBG_DEV(dev, "begin"); ifp = vsi->ifp = if_alloc(IFT_ETHER); if (ifp == NULL) { device_printf(dev, "%s: could not allocate ifnet" " structure!\n", __func__); return (-1); } if_initname(ifp, device_get_name(dev), device_get_unit(dev)); ifp->if_mtu = ETHERMTU; ifp->if_baudrate = IF_Gbps(40); ifp->if_init = ixlv_init; ifp->if_softc = vsi; ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST; ifp->if_ioctl = ixlv_ioctl; #if __FreeBSD_version >= 1100000 if_setgetcounterfn(ifp, ixl_get_counter); #endif ifp->if_transmit = ixl_mq_start; ifp->if_qflush = ixl_qflush; ifp->if_snd.ifq_maxlen = que->num_desc - 2; ether_ifattach(ifp, sc->hw.mac.addr); vsi->max_frame_size = ifp->if_mtu + ETHER_HDR_LEN + ETHER_CRC_LEN + ETHER_VLAN_ENCAP_LEN; /* * Tell the upper layer(s) we support long frames. */ ifp->if_hdrlen = sizeof(struct ether_vlan_header); ifp->if_capabilities |= IFCAP_HWCSUM; ifp->if_capabilities |= IFCAP_HWCSUM_IPV6; ifp->if_capabilities |= IFCAP_TSO; ifp->if_capabilities |= IFCAP_JUMBO_MTU; ifp->if_capabilities |= IFCAP_VLAN_HWTAGGING | IFCAP_VLAN_HWTSO | IFCAP_VLAN_MTU | IFCAP_VLAN_HWCSUM | IFCAP_LRO; ifp->if_capenable = ifp->if_capabilities; /* ** Don't turn this on by default, if vlans are ** created on another pseudo device (eg. lagg) ** then vlan events are not passed thru, breaking ** operation, but with HW FILTER off it works. If ** using vlans directly on the ixl driver you can ** enable this and get full hardware tag filtering. */ ifp->if_capabilities |= IFCAP_VLAN_HWFILTER; /* * Specify the media types supported by this adapter and register * callbacks to update media and link information */ ifmedia_init(&sc->media, IFM_IMASK, ixlv_media_change, ixlv_media_status); // JFV Add media types later? ifmedia_add(&sc->media, IFM_ETHER | IFM_AUTO, 0, NULL); ifmedia_set(&sc->media, IFM_ETHER | IFM_AUTO); INIT_DBG_DEV(dev, "end"); return (0); } /* ** Allocate and setup the interface queues */ static int ixlv_setup_queues(struct ixlv_sc *sc) { device_t dev = sc->dev; struct ixl_vsi *vsi; struct ixl_queue *que; struct tx_ring *txr; struct rx_ring *rxr; int rsize, tsize; int error = I40E_SUCCESS; vsi = &sc->vsi; vsi->back = (void *)sc; vsi->hw = &sc->hw; vsi->num_vlans = 0; /* Get memory for the station queues */ if (!(vsi->queues = (struct ixl_queue *) malloc(sizeof(struct ixl_queue) * vsi->num_queues, M_DEVBUF, M_NOWAIT | M_ZERO))) { device_printf(dev, "Unable to allocate queue memory\n"); error = ENOMEM; goto early; } for (int i = 0; i < vsi->num_queues; i++) { que = &vsi->queues[i]; que->num_desc = ixlv_ringsz; que->me = i; que->vsi = vsi; /* mark the queue as active */ vsi->active_queues |= (u64)1 << que->me; txr = &que->txr; txr->que = que; txr->tail = I40E_QTX_TAIL1(que->me); /* Initialize the TX lock */ snprintf(txr->mtx_name, sizeof(txr->mtx_name), "%s:tx(%d)", device_get_nameunit(dev), que->me); mtx_init(&txr->mtx, txr->mtx_name, NULL, MTX_DEF); /* ** Create the TX descriptor ring, the extra int is ** added as the location for HEAD WB. */ tsize = roundup2((que->num_desc * sizeof(struct i40e_tx_desc)) + sizeof(u32), DBA_ALIGN); if (i40e_allocate_dma_mem(&sc->hw, &txr->dma, i40e_mem_reserved, tsize, DBA_ALIGN)) { device_printf(dev, "Unable to allocate TX Descriptor memory\n"); error = ENOMEM; goto fail; } txr->base = (struct i40e_tx_desc *)txr->dma.va; bzero((void *)txr->base, tsize); /* Now allocate transmit soft structs for the ring */ if (ixl_allocate_tx_data(que)) { device_printf(dev, "Critical Failure setting up TX structures\n"); error = ENOMEM; goto fail; } /* Allocate a buf ring */ txr->br = buf_ring_alloc(ixlv_txbrsz, M_DEVBUF, M_WAITOK, &txr->mtx); if (txr->br == NULL) { device_printf(dev, "Critical Failure setting up TX buf ring\n"); error = ENOMEM; goto fail; } /* * Next the RX queues... */ rsize = roundup2(que->num_desc * sizeof(union i40e_rx_desc), DBA_ALIGN); rxr = &que->rxr; rxr->que = que; rxr->tail = I40E_QRX_TAIL1(que->me); /* Initialize the RX side lock */ snprintf(rxr->mtx_name, sizeof(rxr->mtx_name), "%s:rx(%d)", device_get_nameunit(dev), que->me); mtx_init(&rxr->mtx, rxr->mtx_name, NULL, MTX_DEF); if (i40e_allocate_dma_mem(&sc->hw, &rxr->dma, i40e_mem_reserved, rsize, 4096)) { //JFV - should this be DBA? device_printf(dev, "Unable to allocate RX Descriptor memory\n"); error = ENOMEM; goto fail; } rxr->base = (union i40e_rx_desc *)rxr->dma.va; bzero((void *)rxr->base, rsize); /* Allocate receive soft structs for the ring */ if (ixl_allocate_rx_data(que)) { device_printf(dev, "Critical Failure setting up receive structs\n"); error = ENOMEM; goto fail; } } return (0); fail: for (int i = 0; i < vsi->num_queues; i++) { que = &vsi->queues[i]; rxr = &que->rxr; txr = &que->txr; if (rxr->base) i40e_free_dma_mem(&sc->hw, &rxr->dma); if (txr->base) i40e_free_dma_mem(&sc->hw, &txr->dma); } free(vsi->queues, M_DEVBUF); early: return (error); } /* ** This routine is run via an vlan config EVENT, ** it enables us to use the HW Filter table since ** we can get the vlan id. This just creates the ** entry in the soft version of the VFTA, init will ** repopulate the real table. */ static void ixlv_register_vlan(void *arg, struct ifnet *ifp, u16 vtag) { struct ixl_vsi *vsi = arg; struct ixlv_sc *sc = vsi->back; struct ixlv_vlan_filter *v; if (ifp->if_softc != arg) /* Not our event */ return; if ((vtag == 0) || (vtag > 4095)) /* Invalid */ return; /* Sanity check - make sure it doesn't already exist */ SLIST_FOREACH(v, sc->vlan_filters, next) { if (v->vlan == vtag) return; } mtx_lock(&sc->mtx); ++vsi->num_vlans; v = malloc(sizeof(struct ixlv_vlan_filter), M_DEVBUF, M_NOWAIT | M_ZERO); SLIST_INSERT_HEAD(sc->vlan_filters, v, next); v->vlan = vtag; v->flags = IXL_FILTER_ADD; ixl_vc_enqueue(&sc->vc_mgr, &sc->add_vlan_cmd, IXLV_FLAG_AQ_ADD_VLAN_FILTER, ixl_init_cmd_complete, sc); mtx_unlock(&sc->mtx); return; } /* ** This routine is run via an vlan ** unconfig EVENT, remove our entry ** in the soft vfta. */ static void ixlv_unregister_vlan(void *arg, struct ifnet *ifp, u16 vtag) { struct ixl_vsi *vsi = arg; struct ixlv_sc *sc = vsi->back; struct ixlv_vlan_filter *v; int i = 0; if (ifp->if_softc != arg) return; if ((vtag == 0) || (vtag > 4095)) /* Invalid */ return; mtx_lock(&sc->mtx); SLIST_FOREACH(v, sc->vlan_filters, next) { if (v->vlan == vtag) { v->flags = IXL_FILTER_DEL; ++i; --vsi->num_vlans; } } if (i) ixl_vc_enqueue(&sc->vc_mgr, &sc->del_vlan_cmd, IXLV_FLAG_AQ_DEL_VLAN_FILTER, ixl_init_cmd_complete, sc); mtx_unlock(&sc->mtx); return; } /* ** Get a new filter and add it to the mac filter list. */ static struct ixlv_mac_filter * ixlv_get_mac_filter(struct ixlv_sc *sc) { struct ixlv_mac_filter *f; f = malloc(sizeof(struct ixlv_mac_filter), M_DEVBUF, M_NOWAIT | M_ZERO); if (f) SLIST_INSERT_HEAD(sc->mac_filters, f, next); return (f); } /* ** Find the filter with matching MAC address */ static struct ixlv_mac_filter * ixlv_find_mac_filter(struct ixlv_sc *sc, u8 *macaddr) { struct ixlv_mac_filter *f; bool match = FALSE; SLIST_FOREACH(f, sc->mac_filters, next) { if (cmp_etheraddr(f->macaddr, macaddr)) { match = TRUE; break; } } if (!match) f = NULL; return (f); } /* ** Admin Queue interrupt handler */ static void ixlv_msix_adminq(void *arg) { struct ixlv_sc *sc = arg; struct i40e_hw *hw = &sc->hw; u32 reg, mask; reg = rd32(hw, I40E_VFINT_ICR01); mask = rd32(hw, I40E_VFINT_ICR0_ENA1); reg = rd32(hw, I40E_VFINT_DYN_CTL01); reg |= I40E_VFINT_DYN_CTL01_CLEARPBA_MASK; wr32(hw, I40E_VFINT_DYN_CTL01, reg); /* schedule task */ taskqueue_enqueue(sc->tq, &sc->aq_irq); return; } void ixlv_enable_intr(struct ixl_vsi *vsi) { struct i40e_hw *hw = vsi->hw; struct ixl_queue *que = vsi->queues; ixlv_enable_adminq_irq(hw); for (int i = 0; i < vsi->num_queues; i++, que++) ixlv_enable_queue_irq(hw, que->me); } void ixlv_disable_intr(struct ixl_vsi *vsi) { struct i40e_hw *hw = vsi->hw; struct ixl_queue *que = vsi->queues; ixlv_disable_adminq_irq(hw); for (int i = 0; i < vsi->num_queues; i++, que++) ixlv_disable_queue_irq(hw, que->me); } static void ixlv_disable_adminq_irq(struct i40e_hw *hw) { wr32(hw, I40E_VFINT_DYN_CTL01, 0); wr32(hw, I40E_VFINT_ICR0_ENA1, 0); /* flush */ rd32(hw, I40E_VFGEN_RSTAT); return; } static void ixlv_enable_adminq_irq(struct i40e_hw *hw) { wr32(hw, I40E_VFINT_DYN_CTL01, I40E_VFINT_DYN_CTL01_INTENA_MASK | I40E_VFINT_DYN_CTL01_ITR_INDX_MASK); wr32(hw, I40E_VFINT_ICR0_ENA1, I40E_VFINT_ICR0_ENA1_ADMINQ_MASK); /* flush */ rd32(hw, I40E_VFGEN_RSTAT); return; } static void ixlv_enable_queue_irq(struct i40e_hw *hw, int id) { u32 reg; reg = I40E_VFINT_DYN_CTLN1_INTENA_MASK | I40E_VFINT_DYN_CTLN1_CLEARPBA_MASK | I40E_VFINT_DYN_CTLN1_ITR_INDX_MASK; wr32(hw, I40E_VFINT_DYN_CTLN1(id), reg); } static void ixlv_disable_queue_irq(struct i40e_hw *hw, int id) { wr32(hw, I40E_VFINT_DYN_CTLN1(id), I40E_VFINT_DYN_CTLN1_ITR_INDX_MASK); rd32(hw, I40E_VFGEN_RSTAT); return; } /* * Get initial ITR values from tunable values. */ static void ixlv_configure_itr(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; struct ixl_vsi *vsi = &sc->vsi; struct ixl_queue *que = vsi->queues; vsi->rx_itr_setting = ixlv_rx_itr; vsi->tx_itr_setting = ixlv_tx_itr; for (int i = 0; i < vsi->num_queues; i++, que++) { struct tx_ring *txr = &que->txr; struct rx_ring *rxr = &que->rxr; wr32(hw, I40E_VFINT_ITRN1(IXL_RX_ITR, i), vsi->rx_itr_setting); rxr->itr = vsi->rx_itr_setting; rxr->latency = IXL_AVE_LATENCY; wr32(hw, I40E_VFINT_ITRN1(IXL_TX_ITR, i), vsi->tx_itr_setting); txr->itr = vsi->tx_itr_setting; txr->latency = IXL_AVE_LATENCY; } } /* ** Provide a update to the queue RX ** interrupt moderation value. */ static void ixlv_set_queue_rx_itr(struct ixl_queue *que) { struct ixl_vsi *vsi = que->vsi; struct i40e_hw *hw = vsi->hw; struct rx_ring *rxr = &que->rxr; u16 rx_itr; u16 rx_latency = 0; int rx_bytes; /* Idle, do nothing */ if (rxr->bytes == 0) return; if (ixlv_dynamic_rx_itr) { rx_bytes = rxr->bytes/rxr->itr; rx_itr = rxr->itr; /* Adjust latency range */ switch (rxr->latency) { case IXL_LOW_LATENCY: if (rx_bytes > 10) { rx_latency = IXL_AVE_LATENCY; rx_itr = IXL_ITR_20K; } break; case IXL_AVE_LATENCY: if (rx_bytes > 20) { rx_latency = IXL_BULK_LATENCY; rx_itr = IXL_ITR_8K; } else if (rx_bytes <= 10) { rx_latency = IXL_LOW_LATENCY; rx_itr = IXL_ITR_100K; } break; case IXL_BULK_LATENCY: if (rx_bytes <= 20) { rx_latency = IXL_AVE_LATENCY; rx_itr = IXL_ITR_20K; } break; } rxr->latency = rx_latency; if (rx_itr != rxr->itr) { /* do an exponential smoothing */ rx_itr = (10 * rx_itr * rxr->itr) / ((9 * rx_itr) + rxr->itr); rxr->itr = rx_itr & IXL_MAX_ITR; wr32(hw, I40E_VFINT_ITRN1(IXL_RX_ITR, que->me), rxr->itr); } } else { /* We may have have toggled to non-dynamic */ if (vsi->rx_itr_setting & IXL_ITR_DYNAMIC) vsi->rx_itr_setting = ixlv_rx_itr; /* Update the hardware if needed */ if (rxr->itr != vsi->rx_itr_setting) { rxr->itr = vsi->rx_itr_setting; wr32(hw, I40E_VFINT_ITRN1(IXL_RX_ITR, que->me), rxr->itr); } } rxr->bytes = 0; rxr->packets = 0; return; } /* ** Provide a update to the queue TX ** interrupt moderation value. */ static void ixlv_set_queue_tx_itr(struct ixl_queue *que) { struct ixl_vsi *vsi = que->vsi; struct i40e_hw *hw = vsi->hw; struct tx_ring *txr = &que->txr; u16 tx_itr; u16 tx_latency = 0; int tx_bytes; /* Idle, do nothing */ if (txr->bytes == 0) return; if (ixlv_dynamic_tx_itr) { tx_bytes = txr->bytes/txr->itr; tx_itr = txr->itr; switch (txr->latency) { case IXL_LOW_LATENCY: if (tx_bytes > 10) { tx_latency = IXL_AVE_LATENCY; tx_itr = IXL_ITR_20K; } break; case IXL_AVE_LATENCY: if (tx_bytes > 20) { tx_latency = IXL_BULK_LATENCY; tx_itr = IXL_ITR_8K; } else if (tx_bytes <= 10) { tx_latency = IXL_LOW_LATENCY; tx_itr = IXL_ITR_100K; } break; case IXL_BULK_LATENCY: if (tx_bytes <= 20) { tx_latency = IXL_AVE_LATENCY; tx_itr = IXL_ITR_20K; } break; } txr->latency = tx_latency; if (tx_itr != txr->itr) { /* do an exponential smoothing */ tx_itr = (10 * tx_itr * txr->itr) / ((9 * tx_itr) + txr->itr); txr->itr = tx_itr & IXL_MAX_ITR; wr32(hw, I40E_VFINT_ITRN1(IXL_TX_ITR, que->me), txr->itr); } } else { /* We may have have toggled to non-dynamic */ if (vsi->tx_itr_setting & IXL_ITR_DYNAMIC) vsi->tx_itr_setting = ixlv_tx_itr; /* Update the hardware if needed */ if (txr->itr != vsi->tx_itr_setting) { txr->itr = vsi->tx_itr_setting; wr32(hw, I40E_VFINT_ITRN1(IXL_TX_ITR, que->me), txr->itr); } } txr->bytes = 0; txr->packets = 0; return; } /* ** ** MSIX Interrupt Handlers and Tasklets ** */ static void ixlv_handle_que(void *context, int pending) { struct ixl_queue *que = context; struct ixl_vsi *vsi = que->vsi; struct i40e_hw *hw = vsi->hw; struct tx_ring *txr = &que->txr; struct ifnet *ifp = vsi->ifp; bool more; if (ifp->if_drv_flags & IFF_DRV_RUNNING) { more = ixl_rxeof(que, IXL_RX_LIMIT); mtx_lock(&txr->mtx); ixl_txeof(que); if (!drbr_empty(ifp, txr->br)) ixl_mq_start_locked(ifp, txr); mtx_unlock(&txr->mtx); if (more) { taskqueue_enqueue(que->tq, &que->task); return; } } /* Reenable this interrupt - hmmm */ ixlv_enable_queue_irq(hw, que->me); return; } /********************************************************************* * * MSIX Queue Interrupt Service routine * **********************************************************************/ static void ixlv_msix_que(void *arg) { struct ixl_queue *que = arg; struct ixl_vsi *vsi = que->vsi; struct i40e_hw *hw = vsi->hw; struct tx_ring *txr = &que->txr; bool more_tx, more_rx; /* Spurious interrupts are ignored */ if (!(vsi->ifp->if_drv_flags & IFF_DRV_RUNNING)) return; ++que->irqs; more_rx = ixl_rxeof(que, IXL_RX_LIMIT); mtx_lock(&txr->mtx); more_tx = ixl_txeof(que); /* ** Make certain that if the stack ** has anything queued the task gets ** scheduled to handle it. */ if (!drbr_empty(vsi->ifp, txr->br)) more_tx = 1; mtx_unlock(&txr->mtx); ixlv_set_queue_rx_itr(que); ixlv_set_queue_tx_itr(que); if (more_tx || more_rx) taskqueue_enqueue(que->tq, &que->task); else ixlv_enable_queue_irq(hw, que->me); return; } /********************************************************************* * * Media Ioctl callback * * This routine is called whenever the user queries the status of * the interface using ifconfig. * **********************************************************************/ static void ixlv_media_status(struct ifnet * ifp, struct ifmediareq * ifmr) { struct ixl_vsi *vsi = ifp->if_softc; struct ixlv_sc *sc = vsi->back; INIT_DBG_IF(ifp, "begin"); mtx_lock(&sc->mtx); ixlv_update_link_status(sc); ifmr->ifm_status = IFM_AVALID; ifmr->ifm_active = IFM_ETHER; if (!sc->link_up) { mtx_unlock(&sc->mtx); INIT_DBG_IF(ifp, "end: link not up"); return; } ifmr->ifm_status |= IFM_ACTIVE; /* Hardware is always full-duplex */ ifmr->ifm_active |= IFM_FDX; mtx_unlock(&sc->mtx); INIT_DBG_IF(ifp, "end"); return; } /********************************************************************* * * Media Ioctl callback * * This routine is called when the user changes speed/duplex using * media/mediopt option with ifconfig. * **********************************************************************/ static int ixlv_media_change(struct ifnet * ifp) { struct ixl_vsi *vsi = ifp->if_softc; struct ifmedia *ifm = &vsi->media; INIT_DBG_IF(ifp, "begin"); if (IFM_TYPE(ifm->ifm_media) != IFM_ETHER) return (EINVAL); INIT_DBG_IF(ifp, "end"); return (0); } /********************************************************************* * Multicast Initialization * * This routine is called by init to reset a fresh state. * **********************************************************************/ static void ixlv_init_multi(struct ixl_vsi *vsi) { struct ixlv_mac_filter *f; struct ixlv_sc *sc = vsi->back; int mcnt = 0; IOCTL_DBG_IF(vsi->ifp, "begin"); /* First clear any multicast filters */ SLIST_FOREACH(f, sc->mac_filters, next) { if ((f->flags & IXL_FILTER_USED) && (f->flags & IXL_FILTER_MC)) { f->flags |= IXL_FILTER_DEL; mcnt++; } } if (mcnt > 0) ixl_vc_enqueue(&sc->vc_mgr, &sc->del_multi_cmd, IXLV_FLAG_AQ_DEL_MAC_FILTER, ixl_init_cmd_complete, sc); IOCTL_DBG_IF(vsi->ifp, "end"); } static void ixlv_add_multi(struct ixl_vsi *vsi) { struct ifmultiaddr *ifma; struct ifnet *ifp = vsi->ifp; struct ixlv_sc *sc = vsi->back; int mcnt = 0; IOCTL_DBG_IF(ifp, "begin"); if_maddr_rlock(ifp); /* ** Get a count, to decide if we ** simply use multicast promiscuous. */ TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) { if (ifma->ifma_addr->sa_family != AF_LINK) continue; mcnt++; } if_maddr_runlock(ifp); /* TODO: Remove -- cannot set promiscuous mode in a VF */ if (__predict_false(mcnt >= MAX_MULTICAST_ADDR)) { /* delete all multicast filters */ ixlv_init_multi(vsi); sc->promiscuous_flags |= I40E_FLAG_VF_MULTICAST_PROMISC; ixl_vc_enqueue(&sc->vc_mgr, &sc->add_multi_cmd, IXLV_FLAG_AQ_CONFIGURE_PROMISC, ixl_init_cmd_complete, sc); IOCTL_DEBUGOUT("%s: end: too many filters", __func__); return; } mcnt = 0; if_maddr_rlock(ifp); TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) { if (ifma->ifma_addr->sa_family != AF_LINK) continue; if (!ixlv_add_mac_filter(sc, (u8*)LLADDR((struct sockaddr_dl *) ifma->ifma_addr), IXL_FILTER_MC)) mcnt++; } if_maddr_runlock(ifp); /* ** Notify AQ task that sw filters need to be ** added to hw list */ if (mcnt > 0) ixl_vc_enqueue(&sc->vc_mgr, &sc->add_multi_cmd, IXLV_FLAG_AQ_ADD_MAC_FILTER, ixl_init_cmd_complete, sc); IOCTL_DBG_IF(ifp, "end"); } static void ixlv_del_multi(struct ixl_vsi *vsi) { struct ixlv_mac_filter *f; struct ifmultiaddr *ifma; struct ifnet *ifp = vsi->ifp; struct ixlv_sc *sc = vsi->back; int mcnt = 0; bool match = FALSE; IOCTL_DBG_IF(ifp, "begin"); /* Search for removed multicast addresses */ if_maddr_rlock(ifp); SLIST_FOREACH(f, sc->mac_filters, next) { if ((f->flags & IXL_FILTER_USED) && (f->flags & IXL_FILTER_MC)) { /* check if mac address in filter is in sc's list */ match = FALSE; TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) { if (ifma->ifma_addr->sa_family != AF_LINK) continue; u8 *mc_addr = (u8 *)LLADDR((struct sockaddr_dl *)ifma->ifma_addr); if (cmp_etheraddr(f->macaddr, mc_addr)) { match = TRUE; break; } } /* if this filter is not in the sc's list, remove it */ if (match == FALSE && !(f->flags & IXL_FILTER_DEL)) { f->flags |= IXL_FILTER_DEL; mcnt++; IOCTL_DBG_IF(ifp, "marked: " MAC_FORMAT, MAC_FORMAT_ARGS(f->macaddr)); } else if (match == FALSE) IOCTL_DBG_IF(ifp, "exists: " MAC_FORMAT, MAC_FORMAT_ARGS(f->macaddr)); } } if_maddr_runlock(ifp); if (mcnt > 0) ixl_vc_enqueue(&sc->vc_mgr, &sc->del_multi_cmd, IXLV_FLAG_AQ_DEL_MAC_FILTER, ixl_init_cmd_complete, sc); IOCTL_DBG_IF(ifp, "end"); } /********************************************************************* * Timer routine * * This routine checks for link status,updates statistics, * and runs the watchdog check. * **********************************************************************/ static void ixlv_local_timer(void *arg) { struct ixlv_sc *sc = arg; struct i40e_hw *hw = &sc->hw; struct ixl_vsi *vsi = &sc->vsi; struct ixl_queue *que = vsi->queues; device_t dev = sc->dev; int hung = 0; u32 mask, val; IXLV_CORE_LOCK_ASSERT(sc); /* If Reset is in progress just bail */ if (sc->init_state == IXLV_RESET_PENDING) return; /* Check for when PF triggers a VF reset */ val = rd32(hw, I40E_VFGEN_RSTAT) & I40E_VFGEN_RSTAT_VFR_STATE_MASK; if (val != I40E_VFR_VFACTIVE && val != I40E_VFR_COMPLETED) { DDPRINTF(dev, "reset in progress! (%d)", val); return; } ixlv_request_stats(sc); /* clean and process any events */ taskqueue_enqueue(sc->tq, &sc->aq_irq); /* ** Check status on the queues for a hang */ mask = (I40E_VFINT_DYN_CTLN1_INTENA_MASK | I40E_VFINT_DYN_CTLN1_SWINT_TRIG_MASK | I40E_VFINT_DYN_CTLN1_ITR_INDX_MASK); for (int i = 0; i < vsi->num_queues; i++,que++) { /* Any queues with outstanding work get a sw irq */ if (que->busy) wr32(hw, I40E_VFINT_DYN_CTLN1(que->me), mask); /* ** Each time txeof runs without cleaning, but there ** are uncleaned descriptors it increments busy. If ** we get to 5 we declare it hung. */ if (que->busy == IXL_QUEUE_HUNG) { ++hung; /* Mark the queue as inactive */ vsi->active_queues &= ~((u64)1 << que->me); continue; } else { /* Check if we've come back from hung */ if ((vsi->active_queues & ((u64)1 << que->me)) == 0) vsi->active_queues |= ((u64)1 << que->me); } if (que->busy >= IXL_MAX_TX_BUSY) { device_printf(dev,"Warning queue %d " "appears to be hung!\n", i); que->busy = IXL_QUEUE_HUNG; ++hung; } } /* Only reset when all queues show hung */ if (hung == vsi->num_queues) goto hung; callout_reset(&sc->timer, hz, ixlv_local_timer, sc); return; hung: device_printf(dev, "Local Timer: TX HANG DETECTED - Resetting!!\n"); sc->init_state = IXLV_RESET_REQUIRED; ixlv_init_locked(sc); } /* ** Note: this routine updates the OS on the link state ** the real check of the hardware only happens with ** a link interrupt. */ void ixlv_update_link_status(struct ixlv_sc *sc) { struct ixl_vsi *vsi = &sc->vsi; struct ifnet *ifp = vsi->ifp; if (sc->link_up){ if (vsi->link_active == FALSE) { if (bootverbose) if_printf(ifp,"Link is Up, %d Gbps\n", (sc->link_speed == I40E_LINK_SPEED_40GB) ? 40:10); vsi->link_active = TRUE; if_link_state_change(ifp, LINK_STATE_UP); } } else { /* Link down */ if (vsi->link_active == TRUE) { if (bootverbose) if_printf(ifp,"Link is Down\n"); if_link_state_change(ifp, LINK_STATE_DOWN); vsi->link_active = FALSE; } } return; } /********************************************************************* * * This routine disables all traffic on the adapter by issuing a * global reset on the MAC and deallocates TX/RX buffers. * **********************************************************************/ static void ixlv_stop(struct ixlv_sc *sc) { struct ifnet *ifp; int start; ifp = sc->vsi.ifp; INIT_DBG_IF(ifp, "begin"); IXLV_CORE_LOCK_ASSERT(sc); ixl_vc_flush(&sc->vc_mgr); ixlv_disable_queues(sc); start = ticks; while ((ifp->if_drv_flags & IFF_DRV_RUNNING) && ((ticks - start) < hz/10)) ixlv_do_adminq_locked(sc); /* Stop the local timer */ callout_stop(&sc->timer); INIT_DBG_IF(ifp, "end"); } /********************************************************************* * * Free all station queue structs. * **********************************************************************/ static void ixlv_free_queues(struct ixl_vsi *vsi) { struct ixlv_sc *sc = (struct ixlv_sc *)vsi->back; struct ixl_queue *que = vsi->queues; for (int i = 0; i < vsi->num_queues; i++, que++) { struct tx_ring *txr = &que->txr; struct rx_ring *rxr = &que->rxr; if (!mtx_initialized(&txr->mtx)) /* uninitialized */ continue; IXL_TX_LOCK(txr); ixl_free_que_tx(que); if (txr->base) i40e_free_dma_mem(&sc->hw, &txr->dma); IXL_TX_UNLOCK(txr); IXL_TX_LOCK_DESTROY(txr); if (!mtx_initialized(&rxr->mtx)) /* uninitialized */ continue; IXL_RX_LOCK(rxr); ixl_free_que_rx(que); if (rxr->base) i40e_free_dma_mem(&sc->hw, &rxr->dma); IXL_RX_UNLOCK(rxr); IXL_RX_LOCK_DESTROY(rxr); } free(vsi->queues, M_DEVBUF); } static void ixlv_config_rss_reg(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; struct ixl_vsi *vsi = &sc->vsi; u32 lut = 0; u64 set_hena = 0, hena; int i, j, que_id; u32 rss_seed[IXL_RSS_KEY_SIZE_REG]; #ifdef RSS u32 rss_hash_config; #endif /* Don't set up RSS if using a single queue */ if (vsi->num_queues == 1) { wr32(hw, I40E_VFQF_HENA(0), 0); wr32(hw, I40E_VFQF_HENA(1), 0); ixl_flush(hw); return; } #ifdef RSS /* Fetch the configured RSS key */ rss_getkey((uint8_t *) &rss_seed); #else ixl_get_default_rss_key(rss_seed); #endif /* Fill out hash function seed */ for (i = 0; i < IXL_RSS_KEY_SIZE_REG; i++) wr32(hw, I40E_VFQF_HKEY(i), rss_seed[i]); /* Enable PCTYPES for RSS: */ #ifdef RSS rss_hash_config = rss_gethashconfig(); if (rss_hash_config & RSS_HASHTYPE_RSS_IPV4) set_hena |= ((u64)1 << I40E_FILTER_PCTYPE_NONF_IPV4_OTHER); if (rss_hash_config & RSS_HASHTYPE_RSS_TCP_IPV4) set_hena |= ((u64)1 << I40E_FILTER_PCTYPE_NONF_IPV4_TCP); if (rss_hash_config & RSS_HASHTYPE_RSS_UDP_IPV4) set_hena |= ((u64)1 << I40E_FILTER_PCTYPE_NONF_IPV4_UDP); if (rss_hash_config & RSS_HASHTYPE_RSS_IPV6) set_hena |= ((u64)1 << I40E_FILTER_PCTYPE_NONF_IPV6_OTHER); if (rss_hash_config & RSS_HASHTYPE_RSS_IPV6_EX) set_hena |= ((u64)1 << I40E_FILTER_PCTYPE_FRAG_IPV6); if (rss_hash_config & RSS_HASHTYPE_RSS_TCP_IPV6) set_hena |= ((u64)1 << I40E_FILTER_PCTYPE_NONF_IPV6_TCP); if (rss_hash_config & RSS_HASHTYPE_RSS_UDP_IPV6) set_hena |= ((u64)1 << I40E_FILTER_PCTYPE_NONF_IPV6_UDP); #else set_hena = IXL_DEFAULT_RSS_HENA; #endif hena = (u64)rd32(hw, I40E_VFQF_HENA(0)) | ((u64)rd32(hw, I40E_VFQF_HENA(1)) << 32); hena |= set_hena; wr32(hw, I40E_VFQF_HENA(0), (u32)hena); wr32(hw, I40E_VFQF_HENA(1), (u32)(hena >> 32)); /* Populate the LUT with max no. of queues in round robin fashion */ for (i = 0, j = 0; i < IXL_RSS_VSI_LUT_SIZE; i++, j++) { if (j == vsi->num_queues) j = 0; #ifdef RSS /* * Fetch the RSS bucket id for the given indirection entry. * Cap it at the number of configured buckets (which is * num_queues.) */ que_id = rss_get_indirection_to_bucket(i); que_id = que_id % vsi->num_queues; #else que_id = j; #endif /* lut = 4-byte sliding window of 4 lut entries */ lut = (lut << 8) | (que_id & IXL_RSS_VF_LUT_ENTRY_MASK); /* On i = 3, we have 4 entries in lut; write to the register */ if ((i & 3) == 3) { wr32(hw, I40E_VFQF_HLUT(i >> 2), lut); DDPRINTF(sc->dev, "HLUT(%2d): %#010x", i, lut); } } ixl_flush(hw); } static void ixlv_config_rss_pf(struct ixlv_sc *sc) { ixl_vc_enqueue(&sc->vc_mgr, &sc->config_rss_key_cmd, IXLV_FLAG_AQ_CONFIG_RSS_KEY, ixl_init_cmd_complete, sc); ixl_vc_enqueue(&sc->vc_mgr, &sc->set_rss_hena_cmd, IXLV_FLAG_AQ_SET_RSS_HENA, ixl_init_cmd_complete, sc); ixl_vc_enqueue(&sc->vc_mgr, &sc->config_rss_lut_cmd, IXLV_FLAG_AQ_CONFIG_RSS_LUT, ixl_init_cmd_complete, sc); } /* ** ixlv_config_rss - setup RSS ** ** RSS keys and table are cleared on VF reset. */ static void ixlv_config_rss(struct ixlv_sc *sc) { if (sc->vf_res->vf_offload_flags & I40E_VIRTCHNL_VF_OFFLOAD_RSS_REG) { DDPRINTF(sc->dev, "Setting up RSS using VF registers..."); ixlv_config_rss_reg(sc); } else if (sc->vf_res->vf_offload_flags & I40E_VIRTCHNL_VF_OFFLOAD_RSS_PF) { DDPRINTF(sc->dev, "Setting up RSS using messages to PF..."); ixlv_config_rss_pf(sc); } else device_printf(sc->dev, "VF does not support RSS capability sent by PF.\n"); } /* ** This routine refreshes vlan filters, called by init ** it scans the filter table and then updates the AQ */ static void ixlv_setup_vlan_filters(struct ixlv_sc *sc) { struct ixl_vsi *vsi = &sc->vsi; struct ixlv_vlan_filter *f; int cnt = 0; if (vsi->num_vlans == 0) return; /* ** Scan the filter table for vlan entries, ** and if found call for the AQ update. */ SLIST_FOREACH(f, sc->vlan_filters, next) if (f->flags & IXL_FILTER_ADD) cnt++; if (cnt > 0) ixl_vc_enqueue(&sc->vc_mgr, &sc->add_vlan_cmd, IXLV_FLAG_AQ_ADD_VLAN_FILTER, ixl_init_cmd_complete, sc); } /* ** This routine adds new MAC filters to the sc's list; ** these are later added in hardware by sending a virtual ** channel message. */ static int ixlv_add_mac_filter(struct ixlv_sc *sc, u8 *macaddr, u16 flags) { struct ixlv_mac_filter *f; /* Does one already exist? */ f = ixlv_find_mac_filter(sc, macaddr); if (f != NULL) { IDPRINTF(sc->vsi.ifp, "exists: " MAC_FORMAT, MAC_FORMAT_ARGS(macaddr)); return (EEXIST); } /* If not, get a new empty filter */ f = ixlv_get_mac_filter(sc); if (f == NULL) { if_printf(sc->vsi.ifp, "%s: no filters available!!\n", __func__); return (ENOMEM); } IDPRINTF(sc->vsi.ifp, "marked: " MAC_FORMAT, MAC_FORMAT_ARGS(macaddr)); bcopy(macaddr, f->macaddr, ETHER_ADDR_LEN); f->flags |= (IXL_FILTER_ADD | IXL_FILTER_USED); f->flags |= flags; return (0); } /* ** Marks a MAC filter for deletion. */ static int ixlv_del_mac_filter(struct ixlv_sc *sc, u8 *macaddr) { struct ixlv_mac_filter *f; f = ixlv_find_mac_filter(sc, macaddr); if (f == NULL) return (ENOENT); f->flags |= IXL_FILTER_DEL; return (0); } /* ** Tasklet handler for MSIX Adminq interrupts ** - done outside interrupt context since it might sleep */ static void ixlv_do_adminq(void *context, int pending) { struct ixlv_sc *sc = context; mtx_lock(&sc->mtx); ixlv_do_adminq_locked(sc); mtx_unlock(&sc->mtx); return; } static void ixlv_do_adminq_locked(struct ixlv_sc *sc) { struct i40e_hw *hw = &sc->hw; struct i40e_arq_event_info event; struct i40e_virtchnl_msg *v_msg; device_t dev = sc->dev; u16 result = 0; u32 reg, oldreg; i40e_status ret; IXLV_CORE_LOCK_ASSERT(sc); event.buf_len = IXL_AQ_BUF_SZ; event.msg_buf = sc->aq_buffer; v_msg = (struct i40e_virtchnl_msg *)&event.desc; do { ret = i40e_clean_arq_element(hw, &event, &result); if (ret) break; ixlv_vc_completion(sc, v_msg->v_opcode, v_msg->v_retval, event.msg_buf, event.msg_len); if (result != 0) bzero(event.msg_buf, IXL_AQ_BUF_SZ); } while (result); /* check for Admin queue errors */ oldreg = reg = rd32(hw, hw->aq.arq.len); if (reg & I40E_VF_ARQLEN1_ARQVFE_MASK) { device_printf(dev, "ARQ VF Error detected\n"); reg &= ~I40E_VF_ARQLEN1_ARQVFE_MASK; } if (reg & I40E_VF_ARQLEN1_ARQOVFL_MASK) { device_printf(dev, "ARQ Overflow Error detected\n"); reg &= ~I40E_VF_ARQLEN1_ARQOVFL_MASK; } if (reg & I40E_VF_ARQLEN1_ARQCRIT_MASK) { device_printf(dev, "ARQ Critical Error detected\n"); reg &= ~I40E_VF_ARQLEN1_ARQCRIT_MASK; } if (oldreg != reg) wr32(hw, hw->aq.arq.len, reg); oldreg = reg = rd32(hw, hw->aq.asq.len); if (reg & I40E_VF_ATQLEN1_ATQVFE_MASK) { device_printf(dev, "ASQ VF Error detected\n"); reg &= ~I40E_VF_ATQLEN1_ATQVFE_MASK; } if (reg & I40E_VF_ATQLEN1_ATQOVFL_MASK) { device_printf(dev, "ASQ Overflow Error detected\n"); reg &= ~I40E_VF_ATQLEN1_ATQOVFL_MASK; } if (reg & I40E_VF_ATQLEN1_ATQCRIT_MASK) { device_printf(dev, "ASQ Critical Error detected\n"); reg &= ~I40E_VF_ATQLEN1_ATQCRIT_MASK; } if (oldreg != reg) wr32(hw, hw->aq.asq.len, reg); ixlv_enable_adminq_irq(hw); } static void ixlv_add_sysctls(struct ixlv_sc *sc) { device_t dev = sc->dev; struct ixl_vsi *vsi = &sc->vsi; struct i40e_eth_stats *es = &vsi->eth_stats; struct sysctl_ctx_list *ctx = device_get_sysctl_ctx(dev); struct sysctl_oid *tree = device_get_sysctl_tree(dev); struct sysctl_oid_list *child = SYSCTL_CHILDREN(tree); struct sysctl_oid *vsi_node, *queue_node; struct sysctl_oid_list *vsi_list, *queue_list; #define QUEUE_NAME_LEN 32 char queue_namebuf[QUEUE_NAME_LEN]; struct ixl_queue *queues = vsi->queues; struct tx_ring *txr; struct rx_ring *rxr; /* Driver statistics sysctls */ SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "watchdog_events", CTLFLAG_RD, &sc->watchdog_events, "Watchdog timeouts"); SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "admin_irq", CTLFLAG_RD, &sc->admin_irq, "Admin Queue IRQ Handled"); /* VSI statistics sysctls */ vsi_node = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, "vsi", CTLFLAG_RD, NULL, "VSI-specific statistics"); vsi_list = SYSCTL_CHILDREN(vsi_node); struct ixl_sysctl_info ctls[] = { {&es->rx_bytes, "good_octets_rcvd", "Good Octets Received"}, {&es->rx_unicast, "ucast_pkts_rcvd", "Unicast Packets Received"}, {&es->rx_multicast, "mcast_pkts_rcvd", "Multicast Packets Received"}, {&es->rx_broadcast, "bcast_pkts_rcvd", "Broadcast Packets Received"}, {&es->rx_discards, "rx_discards", "Discarded RX packets"}, {&es->rx_unknown_protocol, "rx_unknown_proto", "RX unknown protocol packets"}, {&es->tx_bytes, "good_octets_txd", "Good Octets Transmitted"}, {&es->tx_unicast, "ucast_pkts_txd", "Unicast Packets Transmitted"}, {&es->tx_multicast, "mcast_pkts_txd", "Multicast Packets Transmitted"}, {&es->tx_broadcast, "bcast_pkts_txd", "Broadcast Packets Transmitted"}, {&es->tx_errors, "tx_errors", "TX packet errors"}, // end {0,0,0} }; struct ixl_sysctl_info *entry = ctls; while (entry->stat != NULL) { SYSCTL_ADD_QUAD(ctx, child, OID_AUTO, entry->name, CTLFLAG_RD, entry->stat, entry->description); entry++; } /* Queue sysctls */ for (int q = 0; q < vsi->num_queues; q++) { snprintf(queue_namebuf, QUEUE_NAME_LEN, "que%d", q); queue_node = SYSCTL_ADD_NODE(ctx, vsi_list, OID_AUTO, queue_namebuf, CTLFLAG_RD, NULL, "Queue Name"); queue_list = SYSCTL_CHILDREN(queue_node); txr = &(queues[q].txr); rxr = &(queues[q].rxr); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "mbuf_defrag_failed", CTLFLAG_RD, &(queues[q].mbuf_defrag_failed), "m_defrag() failed"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "dropped", CTLFLAG_RD, &(queues[q].dropped_pkts), "Driver dropped packets"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "irqs", CTLFLAG_RD, &(queues[q].irqs), "irqs on this queue"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "tso_tx", CTLFLAG_RD, &(queues[q].tso), "TSO"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "tx_dmamap_failed", CTLFLAG_RD, &(queues[q].tx_dmamap_failed), "Driver tx dma failure in xmit"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "no_desc_avail", CTLFLAG_RD, &(txr->no_desc), "Queue No Descriptor Available"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "tx_packets", CTLFLAG_RD, &(txr->total_packets), "Queue Packets Transmitted"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "tx_bytes", CTLFLAG_RD, &(txr->tx_bytes), "Queue Bytes Transmitted"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "rx_packets", CTLFLAG_RD, &(rxr->rx_packets), "Queue Packets Received"); SYSCTL_ADD_QUAD(ctx, queue_list, OID_AUTO, "rx_bytes", CTLFLAG_RD, &(rxr->rx_bytes), "Queue Bytes Received"); SYSCTL_ADD_UINT(ctx, queue_list, OID_AUTO, "rx_itr", CTLFLAG_RD, &(rxr->itr), 0, "Queue Rx ITR Interval"); SYSCTL_ADD_UINT(ctx, queue_list, OID_AUTO, "tx_itr", CTLFLAG_RD, &(txr->itr), 0, "Queue Tx ITR Interval"); #ifdef IXL_DEBUG /* Examine queue state */ SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "qtx_head", CTLTYPE_UINT | CTLFLAG_RD, &queues[q], sizeof(struct ixl_queue), ixlv_sysctl_qtx_tail_handler, "IU", "Queue Transmit Descriptor Tail"); SYSCTL_ADD_PROC(ctx, queue_list, OID_AUTO, "qrx_head", CTLTYPE_UINT | CTLFLAG_RD, &queues[q], sizeof(struct ixl_queue), ixlv_sysctl_qrx_tail_handler, "IU", "Queue Receive Descriptor Tail"); #endif } } static void ixlv_init_filters(struct ixlv_sc *sc) { sc->mac_filters = malloc(sizeof(struct ixlv_mac_filter), M_DEVBUF, M_NOWAIT | M_ZERO); SLIST_INIT(sc->mac_filters); sc->vlan_filters = malloc(sizeof(struct ixlv_vlan_filter), M_DEVBUF, M_NOWAIT | M_ZERO); SLIST_INIT(sc->vlan_filters); return; } static void ixlv_free_filters(struct ixlv_sc *sc) { struct ixlv_mac_filter *f; struct ixlv_vlan_filter *v; while (!SLIST_EMPTY(sc->mac_filters)) { f = SLIST_FIRST(sc->mac_filters); SLIST_REMOVE_HEAD(sc->mac_filters, next); free(f, M_DEVBUF); } while (!SLIST_EMPTY(sc->vlan_filters)) { v = SLIST_FIRST(sc->vlan_filters); SLIST_REMOVE_HEAD(sc->vlan_filters, next); free(v, M_DEVBUF); } return; } #ifdef IXL_DEBUG /** * ixlv_sysctl_qtx_tail_handler * Retrieves I40E_QTX_TAIL1 value from hardware * for a sysctl. */ static int ixlv_sysctl_qtx_tail_handler(SYSCTL_HANDLER_ARGS) { struct ixl_queue *que; int error; u32 val; que = ((struct ixl_queue *)oidp->oid_arg1); if (!que) return 0; val = rd32(que->vsi->hw, que->txr.tail); error = sysctl_handle_int(oidp, &val, 0, req); if (error || !req->newptr) return error; return (0); } /** * ixlv_sysctl_qrx_tail_handler * Retrieves I40E_QRX_TAIL1 value from hardware * for a sysctl. */ static int ixlv_sysctl_qrx_tail_handler(SYSCTL_HANDLER_ARGS) { struct ixl_queue *que; int error; u32 val; que = ((struct ixl_queue *)oidp->oid_arg1); if (!que) return 0; val = rd32(que->vsi->hw, que->rxr.tail); error = sysctl_handle_int(oidp, &val, 0, req); if (error || !req->newptr) return error; return (0); } #endif Index: projects/netbsd-tests-update-12/sys/dev/ixl/ixl_txrx.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/ixl/ixl_txrx.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/ixl/ixl_txrx.c (revision 305172) @@ -1,1845 +1,1846 @@ /****************************************************************************** Copyright (c) 2013-2015, Intel Corporation All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the Intel Corporation nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ******************************************************************************/ /*$FreeBSD$*/ /* ** IXL driver TX/RX Routines: ** This was seperated to allow usage by ** both the PF and VF drivers. */ #ifndef IXL_STANDALONE_BUILD #include "opt_inet.h" #include "opt_inet6.h" #include "opt_rss.h" #endif #include "ixl.h" #ifdef RSS #include #endif /* Local Prototypes */ static void ixl_rx_checksum(struct mbuf *, u32, u32, u8); static void ixl_refresh_mbufs(struct ixl_queue *, int); static int ixl_xmit(struct ixl_queue *, struct mbuf **); static int ixl_tx_setup_offload(struct ixl_queue *, struct mbuf *, u32 *, u32 *); static bool ixl_tso_setup(struct ixl_queue *, struct mbuf *); static inline void ixl_rx_discard(struct rx_ring *, int); static inline void ixl_rx_input(struct rx_ring *, struct ifnet *, struct mbuf *, u8); static inline bool ixl_tso_detect_sparse(struct mbuf *mp); static int ixl_tx_setup_offload(struct ixl_queue *que, struct mbuf *mp, u32 *cmd, u32 *off); static inline u32 ixl_get_tx_head(struct ixl_queue *que); #ifdef DEV_NETMAP #include +int ixl_rx_miss, ixl_rx_miss_bufs, ixl_crcstrip = 1; #endif /* DEV_NETMAP */ /* * @key key is saved into this parameter */ void ixl_get_default_rss_key(u32 *key) { MPASS(key != NULL); u32 rss_seed[IXL_RSS_KEY_SIZE_REG] = {0x41b01687, 0x183cfd8c, 0xce880440, 0x580cbc3c, 0x35897377, 0x328b25e1, 0x4fa98922, 0xb7d90c14, 0xd5bad70d, 0xcd15a2c1, 0x0, 0x0, 0x0}; bcopy(rss_seed, key, IXL_RSS_KEY_SIZE); } /* ** Multiqueue Transmit driver */ int ixl_mq_start(struct ifnet *ifp, struct mbuf *m) { struct ixl_vsi *vsi = ifp->if_softc; struct ixl_queue *que; struct tx_ring *txr; int err, i; #ifdef RSS u32 bucket_id; #endif /* ** Which queue to use: ** ** When doing RSS, map it to the same outbound ** queue as the incoming flow would be mapped to. ** If everything is setup correctly, it should be ** the same bucket that the current CPU we're on is. */ if (M_HASHTYPE_GET(m) != M_HASHTYPE_NONE) { #ifdef RSS if (rss_hash2bucket(m->m_pkthdr.flowid, M_HASHTYPE_GET(m), &bucket_id) == 0) { i = bucket_id % vsi->num_queues; } else #endif i = m->m_pkthdr.flowid % vsi->num_queues; } else i = curcpu % vsi->num_queues; que = &vsi->queues[i]; txr = &que->txr; err = drbr_enqueue(ifp, txr->br, m); if (err) return (err); if (IXL_TX_TRYLOCK(txr)) { ixl_mq_start_locked(ifp, txr); IXL_TX_UNLOCK(txr); } else taskqueue_enqueue(que->tq, &que->tx_task); return (0); } int ixl_mq_start_locked(struct ifnet *ifp, struct tx_ring *txr) { struct ixl_queue *que = txr->que; struct ixl_vsi *vsi = que->vsi; struct mbuf *next; int err = 0; if (((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) || vsi->link_active == 0) return (ENETDOWN); /* Process the transmit queue */ while ((next = drbr_peek(ifp, txr->br)) != NULL) { if ((err = ixl_xmit(que, &next)) != 0) { if (next == NULL) drbr_advance(ifp, txr->br); else drbr_putback(ifp, txr->br, next); break; } drbr_advance(ifp, txr->br); /* Send a copy of the frame to the BPF listener */ ETHER_BPF_MTAP(ifp, next); if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) break; } if (txr->avail < IXL_TX_CLEANUP_THRESHOLD) ixl_txeof(que); return (err); } /* * Called from a taskqueue to drain queued transmit packets. */ void ixl_deferred_mq_start(void *arg, int pending) { struct ixl_queue *que = arg; struct tx_ring *txr = &que->txr; struct ixl_vsi *vsi = que->vsi; struct ifnet *ifp = vsi->ifp; IXL_TX_LOCK(txr); if (!drbr_empty(ifp, txr->br)) ixl_mq_start_locked(ifp, txr); IXL_TX_UNLOCK(txr); } /* ** Flush all queue ring buffers */ void ixl_qflush(struct ifnet *ifp) { struct ixl_vsi *vsi = ifp->if_softc; for (int i = 0; i < vsi->num_queues; i++) { struct ixl_queue *que = &vsi->queues[i]; struct tx_ring *txr = &que->txr; struct mbuf *m; IXL_TX_LOCK(txr); while ((m = buf_ring_dequeue_sc(txr->br)) != NULL) m_freem(m); IXL_TX_UNLOCK(txr); } if_qflush(ifp); } /* ** Find mbuf chains passed to the driver ** that are 'sparse', using more than 8 ** mbufs to deliver an mss-size chunk of data */ static inline bool ixl_tso_detect_sparse(struct mbuf *mp) { struct mbuf *m; int num = 0, mss; bool ret = FALSE; mss = mp->m_pkthdr.tso_segsz; for (m = mp->m_next; m != NULL; m = m->m_next) { num++; mss -= m->m_len; if (mss < 1) break; if (m->m_next == NULL) break; } if (num > IXL_SPARSE_CHAIN) ret = TRUE; return (ret); } /********************************************************************* * * This routine maps the mbufs to tx descriptors, allowing the * TX engine to transmit the packets. * - return 0 on success, positive on failure * **********************************************************************/ #define IXL_TXD_CMD (I40E_TX_DESC_CMD_EOP | I40E_TX_DESC_CMD_RS) static int ixl_xmit(struct ixl_queue *que, struct mbuf **m_headp) { struct ixl_vsi *vsi = que->vsi; struct i40e_hw *hw = vsi->hw; struct tx_ring *txr = &que->txr; struct ixl_tx_buf *buf; struct i40e_tx_desc *txd = NULL; struct mbuf *m_head, *m; int i, j, error, nsegs; int first, last = 0; u16 vtag = 0; u32 cmd, off; bus_dmamap_t map; bus_dma_tag_t tag; bus_dma_segment_t segs[IXL_MAX_TSO_SEGS]; cmd = off = 0; m_head = *m_headp; /* * Important to capture the first descriptor * used because it will contain the index of * the one we tell the hardware to report back */ first = txr->next_avail; buf = &txr->buffers[first]; map = buf->map; tag = txr->tx_tag; if (m_head->m_pkthdr.csum_flags & CSUM_TSO) { /* Use larger mapping for TSO */ tag = txr->tso_tag; if (ixl_tso_detect_sparse(m_head)) { m = m_defrag(m_head, M_NOWAIT); if (m == NULL) { m_freem(*m_headp); *m_headp = NULL; return (ENOBUFS); } *m_headp = m; } } /* * Map the packet for DMA. */ error = bus_dmamap_load_mbuf_sg(tag, map, *m_headp, segs, &nsegs, BUS_DMA_NOWAIT); if (error == EFBIG) { struct mbuf *m; m = m_defrag(*m_headp, M_NOWAIT); if (m == NULL) { que->mbuf_defrag_failed++; m_freem(*m_headp); *m_headp = NULL; return (ENOBUFS); } *m_headp = m; /* Try it again */ error = bus_dmamap_load_mbuf_sg(tag, map, *m_headp, segs, &nsegs, BUS_DMA_NOWAIT); if (error == ENOMEM) { que->tx_dmamap_failed++; return (error); } else if (error != 0) { que->tx_dmamap_failed++; m_freem(*m_headp); *m_headp = NULL; return (error); } } else if (error == ENOMEM) { que->tx_dmamap_failed++; return (error); } else if (error != 0) { que->tx_dmamap_failed++; m_freem(*m_headp); *m_headp = NULL; return (error); } /* Make certain there are enough descriptors */ if (nsegs > txr->avail - 2) { txr->no_desc++; error = ENOBUFS; goto xmit_fail; } m_head = *m_headp; /* Set up the TSO/CSUM offload */ if (m_head->m_pkthdr.csum_flags & CSUM_OFFLOAD) { error = ixl_tx_setup_offload(que, m_head, &cmd, &off); if (error) goto xmit_fail; } cmd |= I40E_TX_DESC_CMD_ICRC; /* Grab the VLAN tag */ if (m_head->m_flags & M_VLANTAG) { cmd |= I40E_TX_DESC_CMD_IL2TAG1; vtag = htole16(m_head->m_pkthdr.ether_vtag); } i = txr->next_avail; for (j = 0; j < nsegs; j++) { bus_size_t seglen; buf = &txr->buffers[i]; buf->tag = tag; /* Keep track of the type tag */ txd = &txr->base[i]; seglen = segs[j].ds_len; txd->buffer_addr = htole64(segs[j].ds_addr); txd->cmd_type_offset_bsz = htole64(I40E_TX_DESC_DTYPE_DATA | ((u64)cmd << I40E_TXD_QW1_CMD_SHIFT) | ((u64)off << I40E_TXD_QW1_OFFSET_SHIFT) | ((u64)seglen << I40E_TXD_QW1_TX_BUF_SZ_SHIFT) | ((u64)vtag << I40E_TXD_QW1_L2TAG1_SHIFT)); last = i; /* descriptor that will get completion IRQ */ if (++i == que->num_desc) i = 0; buf->m_head = NULL; buf->eop_index = -1; } /* Set the last descriptor for report */ txd->cmd_type_offset_bsz |= htole64(((u64)IXL_TXD_CMD << I40E_TXD_QW1_CMD_SHIFT)); txr->avail -= nsegs; txr->next_avail = i; buf->m_head = m_head; /* Swap the dma map between the first and last descriptor */ txr->buffers[first].map = buf->map; buf->map = map; bus_dmamap_sync(tag, map, BUS_DMASYNC_PREWRITE); /* Set the index of the descriptor that will be marked done */ buf = &txr->buffers[first]; buf->eop_index = last; bus_dmamap_sync(txr->dma.tag, txr->dma.map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* * Advance the Transmit Descriptor Tail (Tdt), this tells the * hardware that this frame is available to transmit. */ ++txr->total_packets; wr32(hw, txr->tail, i); /* Mark outstanding work */ if (que->busy == 0) que->busy = 1; return (0); xmit_fail: bus_dmamap_unload(tag, buf->map); return (error); } /********************************************************************* * * Allocate memory for tx_buffer structures. The tx_buffer stores all * the information needed to transmit a packet on the wire. This is * called only once at attach, setup is done every reset. * **********************************************************************/ int ixl_allocate_tx_data(struct ixl_queue *que) { struct tx_ring *txr = &que->txr; struct ixl_vsi *vsi = que->vsi; device_t dev = vsi->dev; struct ixl_tx_buf *buf; int error = 0; /* * Setup DMA descriptor areas. */ if ((error = bus_dma_tag_create(NULL, /* parent */ 1, 0, /* alignment, bounds */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ IXL_TSO_SIZE, /* maxsize */ IXL_MAX_TX_SEGS, /* nsegments */ PAGE_SIZE, /* maxsegsize */ 0, /* flags */ NULL, /* lockfunc */ NULL, /* lockfuncarg */ &txr->tx_tag))) { device_printf(dev,"Unable to allocate TX DMA tag\n"); goto fail; } /* Make a special tag for TSO */ if ((error = bus_dma_tag_create(NULL, /* parent */ 1, 0, /* alignment, bounds */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ IXL_TSO_SIZE, /* maxsize */ IXL_MAX_TSO_SEGS, /* nsegments */ PAGE_SIZE, /* maxsegsize */ 0, /* flags */ NULL, /* lockfunc */ NULL, /* lockfuncarg */ &txr->tso_tag))) { device_printf(dev,"Unable to allocate TX TSO DMA tag\n"); goto fail; } if (!(txr->buffers = (struct ixl_tx_buf *) malloc(sizeof(struct ixl_tx_buf) * que->num_desc, M_DEVBUF, M_NOWAIT | M_ZERO))) { device_printf(dev, "Unable to allocate tx_buffer memory\n"); error = ENOMEM; goto fail; } /* Create the descriptor buffer default dma maps */ buf = txr->buffers; for (int i = 0; i < que->num_desc; i++, buf++) { buf->tag = txr->tx_tag; error = bus_dmamap_create(buf->tag, 0, &buf->map); if (error != 0) { device_printf(dev, "Unable to create TX DMA map\n"); goto fail; } } fail: return (error); } /********************************************************************* * * (Re)Initialize a queue transmit ring. * - called by init, it clears the descriptor ring, * and frees any stale mbufs * **********************************************************************/ void ixl_init_tx_ring(struct ixl_queue *que) { #ifdef DEV_NETMAP struct netmap_adapter *na = NA(que->vsi->ifp); struct netmap_slot *slot; #endif /* DEV_NETMAP */ struct tx_ring *txr = &que->txr; struct ixl_tx_buf *buf; /* Clear the old ring contents */ IXL_TX_LOCK(txr); #ifdef DEV_NETMAP /* * (under lock): if in netmap mode, do some consistency * checks and set slot to entry 0 of the netmap ring. */ slot = netmap_reset(na, NR_TX, que->me, 0); #endif /* DEV_NETMAP */ bzero((void *)txr->base, (sizeof(struct i40e_tx_desc)) * que->num_desc); /* Reset indices */ txr->next_avail = 0; txr->next_to_clean = 0; #ifdef IXL_FDIR /* Initialize flow director */ txr->atr_rate = ixl_atr_rate; txr->atr_count = 0; #endif /* Free any existing tx mbufs. */ buf = txr->buffers; for (int i = 0; i < que->num_desc; i++, buf++) { if (buf->m_head != NULL) { bus_dmamap_sync(buf->tag, buf->map, BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(buf->tag, buf->map); m_freem(buf->m_head); buf->m_head = NULL; } #ifdef DEV_NETMAP /* * In netmap mode, set the map for the packet buffer. * NOTE: Some drivers (not this one) also need to set * the physical buffer address in the NIC ring. * netmap_idx_n2k() maps a nic index, i, into the corresponding * netmap slot index, si */ if (slot) { int si = netmap_idx_n2k(&na->tx_rings[que->me], i); netmap_load_map(na, buf->tag, buf->map, NMB(na, slot + si)); } #endif /* DEV_NETMAP */ /* Clear the EOP index */ buf->eop_index = -1; } /* Set number of descriptors available */ txr->avail = que->num_desc; bus_dmamap_sync(txr->dma.tag, txr->dma.map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); IXL_TX_UNLOCK(txr); } /********************************************************************* * * Free transmit ring related data structures. * **********************************************************************/ void ixl_free_que_tx(struct ixl_queue *que) { struct tx_ring *txr = &que->txr; struct ixl_tx_buf *buf; INIT_DBG_IF(que->vsi->ifp, "queue %d: begin", que->me); for (int i = 0; i < que->num_desc; i++) { buf = &txr->buffers[i]; if (buf->m_head != NULL) { bus_dmamap_sync(buf->tag, buf->map, BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(buf->tag, buf->map); m_freem(buf->m_head); buf->m_head = NULL; if (buf->map != NULL) { bus_dmamap_destroy(buf->tag, buf->map); buf->map = NULL; } } else if (buf->map != NULL) { bus_dmamap_unload(buf->tag, buf->map); bus_dmamap_destroy(buf->tag, buf->map); buf->map = NULL; } } if (txr->br != NULL) buf_ring_free(txr->br, M_DEVBUF); if (txr->buffers != NULL) { free(txr->buffers, M_DEVBUF); txr->buffers = NULL; } if (txr->tx_tag != NULL) { bus_dma_tag_destroy(txr->tx_tag); txr->tx_tag = NULL; } if (txr->tso_tag != NULL) { bus_dma_tag_destroy(txr->tso_tag); txr->tso_tag = NULL; } INIT_DBG_IF(que->vsi->ifp, "queue %d: end", que->me); return; } /********************************************************************* * * Setup descriptor for hw offloads * **********************************************************************/ static int ixl_tx_setup_offload(struct ixl_queue *que, struct mbuf *mp, u32 *cmd, u32 *off) { struct ether_vlan_header *eh; #ifdef INET struct ip *ip = NULL; #endif struct tcphdr *th = NULL; #ifdef INET6 struct ip6_hdr *ip6; #endif int elen, ip_hlen = 0, tcp_hlen; u16 etype; u8 ipproto = 0; bool tso = FALSE; /* Set up the TSO context descriptor if required */ if (mp->m_pkthdr.csum_flags & CSUM_TSO) { tso = ixl_tso_setup(que, mp); if (tso) ++que->tso; else return (ENXIO); } /* * Determine where frame payload starts. * Jump over vlan headers if already present, * helpful for QinQ too. */ eh = mtod(mp, struct ether_vlan_header *); if (eh->evl_encap_proto == htons(ETHERTYPE_VLAN)) { etype = ntohs(eh->evl_proto); elen = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN; } else { etype = ntohs(eh->evl_encap_proto); elen = ETHER_HDR_LEN; } switch (etype) { #ifdef INET case ETHERTYPE_IP: ip = (struct ip *)(mp->m_data + elen); ip_hlen = ip->ip_hl << 2; ipproto = ip->ip_p; th = (struct tcphdr *)((caddr_t)ip + ip_hlen); /* The IP checksum must be recalculated with TSO */ if (tso) *cmd |= I40E_TX_DESC_CMD_IIPT_IPV4_CSUM; else *cmd |= I40E_TX_DESC_CMD_IIPT_IPV4; break; #endif #ifdef INET6 case ETHERTYPE_IPV6: ip6 = (struct ip6_hdr *)(mp->m_data + elen); ip_hlen = sizeof(struct ip6_hdr); ipproto = ip6->ip6_nxt; th = (struct tcphdr *)((caddr_t)ip6 + ip_hlen); *cmd |= I40E_TX_DESC_CMD_IIPT_IPV6; break; #endif default: break; } *off |= (elen >> 1) << I40E_TX_DESC_LENGTH_MACLEN_SHIFT; *off |= (ip_hlen >> 2) << I40E_TX_DESC_LENGTH_IPLEN_SHIFT; switch (ipproto) { case IPPROTO_TCP: tcp_hlen = th->th_off << 2; if (mp->m_pkthdr.csum_flags & (CSUM_TCP|CSUM_TCP_IPV6)) { *cmd |= I40E_TX_DESC_CMD_L4T_EOFT_TCP; *off |= (tcp_hlen >> 2) << I40E_TX_DESC_LENGTH_L4_FC_LEN_SHIFT; } #ifdef IXL_FDIR ixl_atr(que, th, etype); #endif break; case IPPROTO_UDP: if (mp->m_pkthdr.csum_flags & (CSUM_UDP|CSUM_UDP_IPV6)) { *cmd |= I40E_TX_DESC_CMD_L4T_EOFT_UDP; *off |= (sizeof(struct udphdr) >> 2) << I40E_TX_DESC_LENGTH_L4_FC_LEN_SHIFT; } break; case IPPROTO_SCTP: if (mp->m_pkthdr.csum_flags & (CSUM_SCTP|CSUM_SCTP_IPV6)) { *cmd |= I40E_TX_DESC_CMD_L4T_EOFT_SCTP; *off |= (sizeof(struct sctphdr) >> 2) << I40E_TX_DESC_LENGTH_L4_FC_LEN_SHIFT; } /* Fall Thru */ default: break; } return (0); } /********************************************************************** * * Setup context for hardware segmentation offload (TSO) * **********************************************************************/ static bool ixl_tso_setup(struct ixl_queue *que, struct mbuf *mp) { struct tx_ring *txr = &que->txr; struct i40e_tx_context_desc *TXD; struct ixl_tx_buf *buf; u32 cmd, mss, type, tsolen; u16 etype; int idx, elen, ip_hlen, tcp_hlen; struct ether_vlan_header *eh; #ifdef INET struct ip *ip; #endif #ifdef INET6 struct ip6_hdr *ip6; #endif #if defined(INET6) || defined(INET) struct tcphdr *th; #endif u64 type_cmd_tso_mss; /* * Determine where frame payload starts. * Jump over vlan headers if already present */ eh = mtod(mp, struct ether_vlan_header *); if (eh->evl_encap_proto == htons(ETHERTYPE_VLAN)) { elen = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN; etype = eh->evl_proto; } else { elen = ETHER_HDR_LEN; etype = eh->evl_encap_proto; } switch (ntohs(etype)) { #ifdef INET6 case ETHERTYPE_IPV6: ip6 = (struct ip6_hdr *)(mp->m_data + elen); if (ip6->ip6_nxt != IPPROTO_TCP) return (ENXIO); ip_hlen = sizeof(struct ip6_hdr); th = (struct tcphdr *)((caddr_t)ip6 + ip_hlen); th->th_sum = in6_cksum_pseudo(ip6, 0, IPPROTO_TCP, 0); tcp_hlen = th->th_off << 2; /* * The corresponding flag is set by the stack in the IPv4 * TSO case, but not in IPv6 (at least in FreeBSD 10.2). * So, set it here because the rest of the flow requires it. */ mp->m_pkthdr.csum_flags |= CSUM_TCP_IPV6; break; #endif #ifdef INET case ETHERTYPE_IP: ip = (struct ip *)(mp->m_data + elen); if (ip->ip_p != IPPROTO_TCP) return (ENXIO); ip->ip_sum = 0; ip_hlen = ip->ip_hl << 2; th = (struct tcphdr *)((caddr_t)ip + ip_hlen); th->th_sum = in_pseudo(ip->ip_src.s_addr, ip->ip_dst.s_addr, htons(IPPROTO_TCP)); tcp_hlen = th->th_off << 2; break; #endif default: printf("%s: CSUM_TSO but no supported IP version (0x%04x)", __func__, ntohs(etype)); return FALSE; } /* Ensure we have at least the IP+TCP header in the first mbuf. */ if (mp->m_len < elen + ip_hlen + sizeof(struct tcphdr)) return FALSE; idx = txr->next_avail; buf = &txr->buffers[idx]; TXD = (struct i40e_tx_context_desc *) &txr->base[idx]; tsolen = mp->m_pkthdr.len - (elen + ip_hlen + tcp_hlen); type = I40E_TX_DESC_DTYPE_CONTEXT; cmd = I40E_TX_CTX_DESC_TSO; /* ERJ: this must not be less than 64 */ mss = mp->m_pkthdr.tso_segsz; type_cmd_tso_mss = ((u64)type << I40E_TXD_CTX_QW1_DTYPE_SHIFT) | ((u64)cmd << I40E_TXD_CTX_QW1_CMD_SHIFT) | ((u64)tsolen << I40E_TXD_CTX_QW1_TSO_LEN_SHIFT) | ((u64)mss << I40E_TXD_CTX_QW1_MSS_SHIFT); TXD->type_cmd_tso_mss = htole64(type_cmd_tso_mss); TXD->tunneling_params = htole32(0); buf->m_head = NULL; buf->eop_index = -1; if (++idx == que->num_desc) idx = 0; txr->avail--; txr->next_avail = idx; return TRUE; } /* ** ixl_get_tx_head - Retrieve the value from the ** location the HW records its HEAD index */ static inline u32 ixl_get_tx_head(struct ixl_queue *que) { struct tx_ring *txr = &que->txr; void *head = &txr->base[que->num_desc]; return LE32_TO_CPU(*(volatile __le32 *)head); } /********************************************************************** * * Examine each tx_buffer in the used queue. If the hardware is done * processing the packet then free associated resources. The * tx_buffer is put back on the free queue. * **********************************************************************/ bool ixl_txeof(struct ixl_queue *que) { struct tx_ring *txr = &que->txr; u32 first, last, head, done, processed; struct ixl_tx_buf *buf; struct i40e_tx_desc *tx_desc, *eop_desc; mtx_assert(&txr->mtx, MA_OWNED); #ifdef DEV_NETMAP // XXX todo: implement moderation if (netmap_tx_irq(que->vsi->ifp, que->me)) return FALSE; #endif /* DEF_NETMAP */ /* These are not the descriptors you seek, move along :) */ if (txr->avail == que->num_desc) { que->busy = 0; return FALSE; } processed = 0; first = txr->next_to_clean; buf = &txr->buffers[first]; tx_desc = (struct i40e_tx_desc *)&txr->base[first]; last = buf->eop_index; if (last == -1) return FALSE; eop_desc = (struct i40e_tx_desc *)&txr->base[last]; /* Get the Head WB value */ head = ixl_get_tx_head(que); /* ** Get the index of the first descriptor ** BEYOND the EOP and call that 'done'. ** I do this so the comparison in the ** inner while loop below can be simple */ if (++last == que->num_desc) last = 0; done = last; bus_dmamap_sync(txr->dma.tag, txr->dma.map, BUS_DMASYNC_POSTREAD); /* ** The HEAD index of the ring is written in a ** defined location, this rather than a done bit ** is what is used to keep track of what must be ** 'cleaned'. */ while (first != head) { /* We clean the range of the packet */ while (first != done) { ++txr->avail; ++processed; if (buf->m_head) { txr->bytes += /* for ITR adjustment */ buf->m_head->m_pkthdr.len; txr->tx_bytes += /* for TX stats */ buf->m_head->m_pkthdr.len; bus_dmamap_sync(buf->tag, buf->map, BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(buf->tag, buf->map); m_freem(buf->m_head); buf->m_head = NULL; buf->map = NULL; } buf->eop_index = -1; if (++first == que->num_desc) first = 0; buf = &txr->buffers[first]; tx_desc = &txr->base[first]; } ++txr->packets; /* See if there is more work now */ last = buf->eop_index; if (last != -1) { eop_desc = &txr->base[last]; /* Get next done point */ if (++last == que->num_desc) last = 0; done = last; } else break; } bus_dmamap_sync(txr->dma.tag, txr->dma.map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); txr->next_to_clean = first; /* ** Hang detection, we know there's ** work outstanding or the first return ** would have been taken, so indicate an ** unsuccessful pass, in local_timer if ** the value is too great the queue will ** be considered hung. If anything has been ** cleaned then reset the state. */ if ((processed == 0) && (que->busy != IXL_QUEUE_HUNG)) ++que->busy; if (processed) que->busy = 1; /* Note this turns off HUNG */ /* * If there are no pending descriptors, clear the timeout. */ if (txr->avail == que->num_desc) { que->busy = 0; return FALSE; } return TRUE; } /********************************************************************* * * Refresh mbuf buffers for RX descriptor rings * - now keeps its own state so discards due to resource * exhaustion are unnecessary, if an mbuf cannot be obtained * it just returns, keeping its placeholder, thus it can simply * be recalled to try again. * **********************************************************************/ static void ixl_refresh_mbufs(struct ixl_queue *que, int limit) { struct ixl_vsi *vsi = que->vsi; struct rx_ring *rxr = &que->rxr; bus_dma_segment_t hseg[1]; bus_dma_segment_t pseg[1]; struct ixl_rx_buf *buf; struct mbuf *mh, *mp; int i, j, nsegs, error; bool refreshed = FALSE; i = j = rxr->next_refresh; /* Control the loop with one beyond */ if (++j == que->num_desc) j = 0; while (j != limit) { buf = &rxr->buffers[i]; if (rxr->hdr_split == FALSE) goto no_split; if (buf->m_head == NULL) { mh = m_gethdr(M_NOWAIT, MT_DATA); if (mh == NULL) goto update; } else mh = buf->m_head; mh->m_pkthdr.len = mh->m_len = MHLEN; mh->m_len = MHLEN; mh->m_flags |= M_PKTHDR; /* Get the memory mapping */ error = bus_dmamap_load_mbuf_sg(rxr->htag, buf->hmap, mh, hseg, &nsegs, BUS_DMA_NOWAIT); if (error != 0) { printf("Refresh mbufs: hdr dmamap load" " failure - %d\n", error); m_free(mh); buf->m_head = NULL; goto update; } buf->m_head = mh; bus_dmamap_sync(rxr->htag, buf->hmap, BUS_DMASYNC_PREREAD); rxr->base[i].read.hdr_addr = htole64(hseg[0].ds_addr); no_split: if (buf->m_pack == NULL) { mp = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, rxr->mbuf_sz); if (mp == NULL) goto update; } else mp = buf->m_pack; mp->m_pkthdr.len = mp->m_len = rxr->mbuf_sz; /* Get the memory mapping */ error = bus_dmamap_load_mbuf_sg(rxr->ptag, buf->pmap, mp, pseg, &nsegs, BUS_DMA_NOWAIT); if (error != 0) { printf("Refresh mbufs: payload dmamap load" " failure - %d\n", error); m_free(mp); buf->m_pack = NULL; goto update; } buf->m_pack = mp; bus_dmamap_sync(rxr->ptag, buf->pmap, BUS_DMASYNC_PREREAD); rxr->base[i].read.pkt_addr = htole64(pseg[0].ds_addr); /* Used only when doing header split */ rxr->base[i].read.hdr_addr = 0; refreshed = TRUE; /* Next is precalculated */ i = j; rxr->next_refresh = i; if (++j == que->num_desc) j = 0; } update: if (refreshed) /* Update hardware tail index */ wr32(vsi->hw, rxr->tail, rxr->next_refresh); return; } /********************************************************************* * * Allocate memory for rx_buffer structures. Since we use one * rx_buffer per descriptor, the maximum number of rx_buffer's * that we'll need is equal to the number of receive descriptors * that we've defined. * **********************************************************************/ int ixl_allocate_rx_data(struct ixl_queue *que) { struct rx_ring *rxr = &que->rxr; struct ixl_vsi *vsi = que->vsi; device_t dev = vsi->dev; struct ixl_rx_buf *buf; int i, bsize, error; bsize = sizeof(struct ixl_rx_buf) * que->num_desc; if (!(rxr->buffers = (struct ixl_rx_buf *) malloc(bsize, M_DEVBUF, M_NOWAIT | M_ZERO))) { device_printf(dev, "Unable to allocate rx_buffer memory\n"); error = ENOMEM; return (error); } if ((error = bus_dma_tag_create(NULL, /* parent */ 1, 0, /* alignment, bounds */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ MSIZE, /* maxsize */ 1, /* nsegments */ MSIZE, /* maxsegsize */ 0, /* flags */ NULL, /* lockfunc */ NULL, /* lockfuncarg */ &rxr->htag))) { device_printf(dev, "Unable to create RX DMA htag\n"); return (error); } if ((error = bus_dma_tag_create(NULL, /* parent */ 1, 0, /* alignment, bounds */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ MJUM16BYTES, /* maxsize */ 1, /* nsegments */ MJUM16BYTES, /* maxsegsize */ 0, /* flags */ NULL, /* lockfunc */ NULL, /* lockfuncarg */ &rxr->ptag))) { device_printf(dev, "Unable to create RX DMA ptag\n"); return (error); } for (i = 0; i < que->num_desc; i++) { buf = &rxr->buffers[i]; error = bus_dmamap_create(rxr->htag, BUS_DMA_NOWAIT, &buf->hmap); if (error) { device_printf(dev, "Unable to create RX head map\n"); break; } error = bus_dmamap_create(rxr->ptag, BUS_DMA_NOWAIT, &buf->pmap); if (error) { device_printf(dev, "Unable to create RX pkt map\n"); break; } } return (error); } /********************************************************************* * * (Re)Initialize the queue receive ring and its buffers. * **********************************************************************/ int ixl_init_rx_ring(struct ixl_queue *que) { struct rx_ring *rxr = &que->rxr; struct ixl_vsi *vsi = que->vsi; #if defined(INET6) || defined(INET) struct ifnet *ifp = vsi->ifp; struct lro_ctrl *lro = &rxr->lro; #endif struct ixl_rx_buf *buf; bus_dma_segment_t pseg[1], hseg[1]; int rsize, nsegs, error = 0; #ifdef DEV_NETMAP struct netmap_adapter *na = NA(que->vsi->ifp); struct netmap_slot *slot; #endif /* DEV_NETMAP */ IXL_RX_LOCK(rxr); #ifdef DEV_NETMAP /* same as in ixl_init_tx_ring() */ slot = netmap_reset(na, NR_RX, que->me, 0); #endif /* DEV_NETMAP */ /* Clear the ring contents */ rsize = roundup2(que->num_desc * sizeof(union i40e_rx_desc), DBA_ALIGN); bzero((void *)rxr->base, rsize); /* Cleanup any existing buffers */ for (int i = 0; i < que->num_desc; i++) { buf = &rxr->buffers[i]; if (buf->m_head != NULL) { bus_dmamap_sync(rxr->htag, buf->hmap, BUS_DMASYNC_POSTREAD); bus_dmamap_unload(rxr->htag, buf->hmap); buf->m_head->m_flags |= M_PKTHDR; m_freem(buf->m_head); } if (buf->m_pack != NULL) { bus_dmamap_sync(rxr->ptag, buf->pmap, BUS_DMASYNC_POSTREAD); bus_dmamap_unload(rxr->ptag, buf->pmap); buf->m_pack->m_flags |= M_PKTHDR; m_freem(buf->m_pack); } buf->m_head = NULL; buf->m_pack = NULL; } /* header split is off */ rxr->hdr_split = FALSE; /* Now replenish the mbufs */ for (int j = 0; j != que->num_desc; ++j) { struct mbuf *mh, *mp; buf = &rxr->buffers[j]; #ifdef DEV_NETMAP /* * In netmap mode, fill the map and set the buffer * address in the NIC ring, considering the offset * between the netmap and NIC rings (see comment in * ixgbe_setup_transmit_ring() ). No need to allocate * an mbuf, so end the block with a continue; */ if (slot) { int sj = netmap_idx_n2k(&na->rx_rings[que->me], j); uint64_t paddr; void *addr; addr = PNMB(na, slot + sj, &paddr); netmap_load_map(na, rxr->dma.tag, buf->pmap, addr); /* Update descriptor and the cached value */ rxr->base[j].read.pkt_addr = htole64(paddr); rxr->base[j].read.hdr_addr = 0; continue; } #endif /* DEV_NETMAP */ /* ** Don't allocate mbufs if not ** doing header split, its wasteful */ if (rxr->hdr_split == FALSE) goto skip_head; /* First the header */ buf->m_head = m_gethdr(M_NOWAIT, MT_DATA); if (buf->m_head == NULL) { error = ENOBUFS; goto fail; } m_adj(buf->m_head, ETHER_ALIGN); mh = buf->m_head; mh->m_len = mh->m_pkthdr.len = MHLEN; mh->m_flags |= M_PKTHDR; /* Get the memory mapping */ error = bus_dmamap_load_mbuf_sg(rxr->htag, buf->hmap, buf->m_head, hseg, &nsegs, BUS_DMA_NOWAIT); if (error != 0) /* Nothing elegant to do here */ goto fail; bus_dmamap_sync(rxr->htag, buf->hmap, BUS_DMASYNC_PREREAD); /* Update descriptor */ rxr->base[j].read.hdr_addr = htole64(hseg[0].ds_addr); skip_head: /* Now the payload cluster */ buf->m_pack = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, rxr->mbuf_sz); if (buf->m_pack == NULL) { error = ENOBUFS; goto fail; } mp = buf->m_pack; mp->m_pkthdr.len = mp->m_len = rxr->mbuf_sz; /* Get the memory mapping */ error = bus_dmamap_load_mbuf_sg(rxr->ptag, buf->pmap, mp, pseg, &nsegs, BUS_DMA_NOWAIT); if (error != 0) goto fail; bus_dmamap_sync(rxr->ptag, buf->pmap, BUS_DMASYNC_PREREAD); /* Update descriptor */ rxr->base[j].read.pkt_addr = htole64(pseg[0].ds_addr); rxr->base[j].read.hdr_addr = 0; } /* Setup our descriptor indices */ rxr->next_check = 0; rxr->next_refresh = 0; rxr->lro_enabled = FALSE; rxr->split = 0; rxr->bytes = 0; rxr->discard = FALSE; wr32(vsi->hw, rxr->tail, que->num_desc - 1); ixl_flush(vsi->hw); #if defined(INET6) || defined(INET) /* ** Now set up the LRO interface: */ if (ifp->if_capenable & IFCAP_LRO) { int err = tcp_lro_init(lro); if (err) { if_printf(ifp, "queue %d: LRO Initialization failed!\n", que->me); goto fail; } INIT_DBG_IF(ifp, "queue %d: RX Soft LRO Initialized", que->me); rxr->lro_enabled = TRUE; lro->ifp = vsi->ifp; } #endif bus_dmamap_sync(rxr->dma.tag, rxr->dma.map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); fail: IXL_RX_UNLOCK(rxr); return (error); } /********************************************************************* * * Free station receive ring data structures * **********************************************************************/ void ixl_free_que_rx(struct ixl_queue *que) { struct rx_ring *rxr = &que->rxr; struct ixl_rx_buf *buf; INIT_DBG_IF(que->vsi->ifp, "queue %d: begin", que->me); /* Cleanup any existing buffers */ if (rxr->buffers != NULL) { for (int i = 0; i < que->num_desc; i++) { buf = &rxr->buffers[i]; if (buf->m_head != NULL) { bus_dmamap_sync(rxr->htag, buf->hmap, BUS_DMASYNC_POSTREAD); bus_dmamap_unload(rxr->htag, buf->hmap); buf->m_head->m_flags |= M_PKTHDR; m_freem(buf->m_head); } if (buf->m_pack != NULL) { bus_dmamap_sync(rxr->ptag, buf->pmap, BUS_DMASYNC_POSTREAD); bus_dmamap_unload(rxr->ptag, buf->pmap); buf->m_pack->m_flags |= M_PKTHDR; m_freem(buf->m_pack); } buf->m_head = NULL; buf->m_pack = NULL; if (buf->hmap != NULL) { bus_dmamap_destroy(rxr->htag, buf->hmap); buf->hmap = NULL; } if (buf->pmap != NULL) { bus_dmamap_destroy(rxr->ptag, buf->pmap); buf->pmap = NULL; } } if (rxr->buffers != NULL) { free(rxr->buffers, M_DEVBUF); rxr->buffers = NULL; } } if (rxr->htag != NULL) { bus_dma_tag_destroy(rxr->htag); rxr->htag = NULL; } if (rxr->ptag != NULL) { bus_dma_tag_destroy(rxr->ptag); rxr->ptag = NULL; } INIT_DBG_IF(que->vsi->ifp, "queue %d: end", que->me); return; } static inline void ixl_rx_input(struct rx_ring *rxr, struct ifnet *ifp, struct mbuf *m, u8 ptype) { #if defined(INET6) || defined(INET) /* * ATM LRO is only for IPv4/TCP packets and TCP checksum of the packet * should be computed by hardware. Also it should not have VLAN tag in * ethernet header. */ if (rxr->lro_enabled && (ifp->if_capenable & IFCAP_VLAN_HWTAGGING) != 0 && (m->m_pkthdr.csum_flags & (CSUM_DATA_VALID | CSUM_PSEUDO_HDR)) == (CSUM_DATA_VALID | CSUM_PSEUDO_HDR)) { /* * Send to the stack if: ** - LRO not enabled, or ** - no LRO resources, or ** - lro enqueue fails */ if (rxr->lro.lro_cnt != 0) if (tcp_lro_rx(&rxr->lro, m, 0) == 0) return; } #endif IXL_RX_UNLOCK(rxr); (*ifp->if_input)(ifp, m); IXL_RX_LOCK(rxr); } static inline void ixl_rx_discard(struct rx_ring *rxr, int i) { struct ixl_rx_buf *rbuf; rbuf = &rxr->buffers[i]; if (rbuf->fmp != NULL) {/* Partial chain ? */ rbuf->fmp->m_flags |= M_PKTHDR; m_freem(rbuf->fmp); rbuf->fmp = NULL; } /* ** With advanced descriptors the writeback ** clobbers the buffer addrs, so its easier ** to just free the existing mbufs and take ** the normal refresh path to get new buffers ** and mapping. */ if (rbuf->m_head) { m_free(rbuf->m_head); rbuf->m_head = NULL; } if (rbuf->m_pack) { m_free(rbuf->m_pack); rbuf->m_pack = NULL; } return; } #ifdef RSS /* ** i40e_ptype_to_hash: parse the packet type ** to determine the appropriate hash. */ static inline int ixl_ptype_to_hash(u8 ptype) { struct i40e_rx_ptype_decoded decoded; u8 ex = 0; decoded = decode_rx_desc_ptype(ptype); ex = decoded.outer_frag; if (!decoded.known) return M_HASHTYPE_OPAQUE_HASH; if (decoded.outer_ip == I40E_RX_PTYPE_OUTER_L2) return M_HASHTYPE_OPAQUE_HASH; /* Note: anything that gets to this point is IP */ if (decoded.outer_ip_ver == I40E_RX_PTYPE_OUTER_IPV6) { switch (decoded.inner_prot) { case I40E_RX_PTYPE_INNER_PROT_TCP: if (ex) return M_HASHTYPE_RSS_TCP_IPV6_EX; else return M_HASHTYPE_RSS_TCP_IPV6; case I40E_RX_PTYPE_INNER_PROT_UDP: if (ex) return M_HASHTYPE_RSS_UDP_IPV6_EX; else return M_HASHTYPE_RSS_UDP_IPV6; default: if (ex) return M_HASHTYPE_RSS_IPV6_EX; else return M_HASHTYPE_RSS_IPV6; } } if (decoded.outer_ip_ver == I40E_RX_PTYPE_OUTER_IPV4) { switch (decoded.inner_prot) { case I40E_RX_PTYPE_INNER_PROT_TCP: return M_HASHTYPE_RSS_TCP_IPV4; case I40E_RX_PTYPE_INNER_PROT_UDP: if (ex) return M_HASHTYPE_RSS_UDP_IPV4_EX; else return M_HASHTYPE_RSS_UDP_IPV4; default: return M_HASHTYPE_RSS_IPV4; } } /* We should never get here!! */ return M_HASHTYPE_OPAQUE_HASH; } #endif /* RSS */ /********************************************************************* * * This routine executes in interrupt context. It replenishes * the mbufs in the descriptor and sends data which has been * dma'ed into host memory to upper layer. * * We loop at most count times if count is > 0, or until done if * count < 0. * * Return TRUE for more work, FALSE for all clean. *********************************************************************/ bool ixl_rxeof(struct ixl_queue *que, int count) { struct ixl_vsi *vsi = que->vsi; struct rx_ring *rxr = &que->rxr; struct ifnet *ifp = vsi->ifp; #if defined(INET6) || defined(INET) struct lro_ctrl *lro = &rxr->lro; #endif int i, nextp, processed = 0; union i40e_rx_desc *cur; struct ixl_rx_buf *rbuf, *nbuf; IXL_RX_LOCK(rxr); #ifdef DEV_NETMAP if (netmap_rx_irq(ifp, que->me, &count)) { IXL_RX_UNLOCK(rxr); return (FALSE); } #endif /* DEV_NETMAP */ for (i = rxr->next_check; count != 0;) { struct mbuf *sendmp, *mh, *mp; u32 status, error; u16 hlen, plen, vtag; u64 qword; u8 ptype; bool eop; /* Sync the ring. */ bus_dmamap_sync(rxr->dma.tag, rxr->dma.map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); cur = &rxr->base[i]; qword = le64toh(cur->wb.qword1.status_error_len); status = (qword & I40E_RXD_QW1_STATUS_MASK) >> I40E_RXD_QW1_STATUS_SHIFT; error = (qword & I40E_RXD_QW1_ERROR_MASK) >> I40E_RXD_QW1_ERROR_SHIFT; plen = (qword & I40E_RXD_QW1_LENGTH_PBUF_MASK) >> I40E_RXD_QW1_LENGTH_PBUF_SHIFT; hlen = (qword & I40E_RXD_QW1_LENGTH_HBUF_MASK) >> I40E_RXD_QW1_LENGTH_HBUF_SHIFT; ptype = (qword & I40E_RXD_QW1_PTYPE_MASK) >> I40E_RXD_QW1_PTYPE_SHIFT; if ((status & (1 << I40E_RX_DESC_STATUS_DD_SHIFT)) == 0) { ++rxr->not_done; break; } if ((ifp->if_drv_flags & IFF_DRV_RUNNING) == 0) break; count--; sendmp = NULL; nbuf = NULL; cur->wb.qword1.status_error_len = 0; rbuf = &rxr->buffers[i]; mh = rbuf->m_head; mp = rbuf->m_pack; eop = (status & (1 << I40E_RX_DESC_STATUS_EOF_SHIFT)); if (status & (1 << I40E_RX_DESC_STATUS_L2TAG1P_SHIFT)) vtag = le16toh(cur->wb.qword0.lo_dword.l2tag1); else vtag = 0; /* ** Make sure bad packets are discarded, ** note that only EOP descriptor has valid ** error results. */ if (eop && (error & (1 << I40E_RX_DESC_ERROR_RXE_SHIFT))) { rxr->desc_errs++; ixl_rx_discard(rxr, i); goto next_desc; } /* Prefetch the next buffer */ if (!eop) { nextp = i + 1; if (nextp == que->num_desc) nextp = 0; nbuf = &rxr->buffers[nextp]; prefetch(nbuf); } /* ** The header mbuf is ONLY used when header ** split is enabled, otherwise we get normal ** behavior, ie, both header and payload ** are DMA'd into the payload buffer. ** ** Rather than using the fmp/lmp global pointers ** we now keep the head of a packet chain in the ** buffer struct and pass this along from one ** descriptor to the next, until we get EOP. */ if (rxr->hdr_split && (rbuf->fmp == NULL)) { if (hlen > IXL_RX_HDR) hlen = IXL_RX_HDR; mh->m_len = hlen; mh->m_flags |= M_PKTHDR; mh->m_next = NULL; mh->m_pkthdr.len = mh->m_len; /* Null buf pointer so it is refreshed */ rbuf->m_head = NULL; /* ** Check the payload length, this ** could be zero if its a small ** packet. */ if (plen > 0) { mp->m_len = plen; mp->m_next = NULL; mp->m_flags &= ~M_PKTHDR; mh->m_next = mp; mh->m_pkthdr.len += mp->m_len; /* Null buf pointer so it is refreshed */ rbuf->m_pack = NULL; rxr->split++; } /* ** Now create the forward ** chain so when complete ** we wont have to. */ if (eop == 0) { /* stash the chain head */ nbuf->fmp = mh; /* Make forward chain */ if (plen) mp->m_next = nbuf->m_pack; else mh->m_next = nbuf->m_pack; } else { /* Singlet, prepare to send */ sendmp = mh; if (vtag) { sendmp->m_pkthdr.ether_vtag = vtag; sendmp->m_flags |= M_VLANTAG; } } } else { /* ** Either no header split, or a ** secondary piece of a fragmented ** split packet. */ mp->m_len = plen; /* ** See if there is a stored head ** that determines what we are */ sendmp = rbuf->fmp; rbuf->m_pack = rbuf->fmp = NULL; if (sendmp != NULL) /* secondary frag */ sendmp->m_pkthdr.len += mp->m_len; else { /* first desc of a non-ps chain */ sendmp = mp; sendmp->m_flags |= M_PKTHDR; sendmp->m_pkthdr.len = mp->m_len; } /* Pass the head pointer on */ if (eop == 0) { nbuf->fmp = sendmp; sendmp = NULL; mp->m_next = nbuf->m_pack; } } ++processed; /* Sending this frame? */ if (eop) { sendmp->m_pkthdr.rcvif = ifp; /* gather stats */ rxr->rx_packets++; rxr->rx_bytes += sendmp->m_pkthdr.len; /* capture data for dynamic ITR adjustment */ rxr->packets++; rxr->bytes += sendmp->m_pkthdr.len; /* Set VLAN tag (field only valid in eop desc) */ if (vtag) { sendmp->m_pkthdr.ether_vtag = vtag; sendmp->m_flags |= M_VLANTAG; } if ((ifp->if_capenable & IFCAP_RXCSUM) != 0) ixl_rx_checksum(sendmp, status, error, ptype); #ifdef RSS sendmp->m_pkthdr.flowid = le32toh(cur->wb.qword0.hi_dword.rss); M_HASHTYPE_SET(sendmp, ixl_ptype_to_hash(ptype)); #else sendmp->m_pkthdr.flowid = que->msix; M_HASHTYPE_SET(sendmp, M_HASHTYPE_OPAQUE); #endif } next_desc: bus_dmamap_sync(rxr->dma.tag, rxr->dma.map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* Advance our pointers to the next descriptor. */ if (++i == que->num_desc) i = 0; /* Now send to the stack or do LRO */ if (sendmp != NULL) { rxr->next_check = i; ixl_rx_input(rxr, ifp, sendmp, ptype); i = rxr->next_check; } /* Every 8 descriptors we go to refresh mbufs */ if (processed == 8) { ixl_refresh_mbufs(que, i); processed = 0; } } /* Refresh any remaining buf structs */ if (ixl_rx_unrefreshed(que)) ixl_refresh_mbufs(que, i); rxr->next_check = i; #if defined(INET6) || defined(INET) /* * Flush any outstanding LRO work */ tcp_lro_flush_all(lro); #endif IXL_RX_UNLOCK(rxr); return (FALSE); } /********************************************************************* * * Verify that the hardware indicated that the checksum is valid. * Inform the stack about the status of checksum so that stack * doesn't spend time verifying the checksum. * *********************************************************************/ static void ixl_rx_checksum(struct mbuf * mp, u32 status, u32 error, u8 ptype) { struct i40e_rx_ptype_decoded decoded; decoded = decode_rx_desc_ptype(ptype); /* Errors? */ if (error & ((1 << I40E_RX_DESC_ERROR_IPE_SHIFT) | (1 << I40E_RX_DESC_ERROR_L4E_SHIFT))) { mp->m_pkthdr.csum_flags = 0; return; } /* IPv6 with extension headers likely have bad csum */ if (decoded.outer_ip == I40E_RX_PTYPE_OUTER_IP && decoded.outer_ip_ver == I40E_RX_PTYPE_OUTER_IPV6) if (status & (1 << I40E_RX_DESC_STATUS_IPV6EXADD_SHIFT)) { mp->m_pkthdr.csum_flags = 0; return; } /* IP Checksum Good */ mp->m_pkthdr.csum_flags = CSUM_IP_CHECKED; mp->m_pkthdr.csum_flags |= CSUM_IP_VALID; if (status & (1 << I40E_RX_DESC_STATUS_L3L4P_SHIFT)) { mp->m_pkthdr.csum_flags |= (CSUM_DATA_VALID | CSUM_PSEUDO_HDR); mp->m_pkthdr.csum_data |= htons(0xffff); } return; } #if __FreeBSD_version >= 1100000 uint64_t ixl_get_counter(if_t ifp, ift_counter cnt) { struct ixl_vsi *vsi; vsi = if_getsoftc(ifp); switch (cnt) { case IFCOUNTER_IPACKETS: return (vsi->ipackets); case IFCOUNTER_IERRORS: return (vsi->ierrors); case IFCOUNTER_OPACKETS: return (vsi->opackets); case IFCOUNTER_OERRORS: return (vsi->oerrors); case IFCOUNTER_COLLISIONS: /* Collisions are by standard impossible in 40G/10G Ethernet */ return (0); case IFCOUNTER_IBYTES: return (vsi->ibytes); case IFCOUNTER_OBYTES: return (vsi->obytes); case IFCOUNTER_IMCASTS: return (vsi->imcasts); case IFCOUNTER_OMCASTS: return (vsi->omcasts); case IFCOUNTER_IQDROPS: return (vsi->iqdrops); case IFCOUNTER_OQDROPS: return (vsi->oqdrops); case IFCOUNTER_NOPROTO: return (vsi->noproto); default: return (if_get_counter_default(ifp, cnt)); } } #endif Index: projects/netbsd-tests-update-12/sys/dev/kbdmux/kbdmux.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/kbdmux/kbdmux.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/kbdmux/kbdmux.c (revision 305172) @@ -1,1398 +1,1399 @@ /* * kbdmux.c */ /*- * Copyright (c) 2005 Maksim Yevmenkin * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $Id: kbdmux.c,v 1.4 2005/07/14 17:38:35 max Exp $ * $FreeBSD$ */ #include "opt_compat.h" #include "opt_kbd.h" #include "opt_kbdmux.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* the initial key map, accent map and fkey strings */ #ifdef KBDMUX_DFLT_KEYMAP #define KBD_DFLT_KEYMAP #include "kbdmuxmap.h" #endif #include #define KEYBOARD_NAME "kbdmux" MALLOC_DECLARE(M_KBDMUX); MALLOC_DEFINE(M_KBDMUX, KEYBOARD_NAME, "Keyboard multiplexor"); /***************************************************************************** ***************************************************************************** ** Keyboard state ***************************************************************************** *****************************************************************************/ #define KBDMUX_Q_SIZE 512 /* input queue size */ /* * XXX * For now rely on Giant mutex to protect our data structures. * Just like the rest of keyboard drivers and syscons(4) do. * Note that callout is initialized as not MP-safe to make sure * Giant is held. */ #if 0 /* not yet */ #define KBDMUX_LOCK_DECL_GLOBAL \ struct mtx ks_lock #define KBDMUX_LOCK_INIT(s) \ mtx_init(&(s)->ks_lock, "kbdmux", NULL, MTX_DEF|MTX_RECURSE) #define KBDMUX_LOCK_DESTROY(s) \ mtx_destroy(&(s)->ks_lock) #define KBDMUX_LOCK(s) \ mtx_lock(&(s)->ks_lock) #define KBDMUX_UNLOCK(s) \ mtx_unlock(&(s)->ks_lock) #define KBDMUX_LOCK_ASSERT(s, w) \ mtx_assert(&(s)->ks_lock, (w)) #define KBDMUX_SLEEP(s, f, d, t) \ msleep(&(s)->f, &(s)->ks_lock, PCATCH | (PZERO + 1), (d), (t)) #define KBDMUX_CALLOUT_INIT(s) \ callout_init_mtx(&(s)->ks_timo, &(s)->ks_lock, 0) #define KBDMUX_QUEUE_INTR(s) \ taskqueue_enqueue(taskqueue_swi_giant, &(s)->ks_task) #else #define KBDMUX_LOCK_DECL_GLOBAL #define KBDMUX_LOCK_INIT(s) #define KBDMUX_LOCK_DESTROY(s) #define KBDMUX_LOCK(s) #define KBDMUX_UNLOCK(s) #define KBDMUX_LOCK_ASSERT(s, w) #define KBDMUX_SLEEP(s, f, d, t) \ tsleep(&(s)->f, PCATCH | (PZERO + 1), (d), (t)) #define KBDMUX_CALLOUT_INIT(s) \ callout_init(&(s)->ks_timo, 0) #define KBDMUX_QUEUE_INTR(s) \ taskqueue_enqueue(taskqueue_swi_giant, &(s)->ks_task) #endif /* not yet */ /* * kbdmux keyboard */ struct kbdmux_kbd { keyboard_t *kbd; /* keyboard */ SLIST_ENTRY(kbdmux_kbd) next; /* link to next */ }; typedef struct kbdmux_kbd kbdmux_kbd_t; /* * kbdmux state */ struct kbdmux_state { char ks_inq[KBDMUX_Q_SIZE]; /* input chars queue */ unsigned int ks_inq_start; unsigned int ks_inq_length; struct task ks_task; /* interrupt task */ struct callout ks_timo; /* timeout handler */ #define TICKS (hz) /* rate */ int ks_flags; /* flags */ #define COMPOSE (1 << 0) /* compose char flag */ -#define POLLING (1 << 1) /* polling */ #define TASK (1 << 2) /* interrupt task queued */ + int ks_polling; /* poll nesting count */ int ks_mode; /* K_XLATE, K_RAW, K_CODE */ int ks_state; /* state */ int ks_accents; /* accent key index (> 0) */ u_int ks_composed_char; /* composed char code */ u_char ks_prefix; /* AT scan code prefix */ SLIST_HEAD(, kbdmux_kbd) ks_kbds; /* keyboards */ KBDMUX_LOCK_DECL_GLOBAL; }; typedef struct kbdmux_state kbdmux_state_t; /***************************************************************************** ***************************************************************************** ** Helper functions ***************************************************************************** *****************************************************************************/ static task_fn_t kbdmux_kbd_intr; static timeout_t kbdmux_kbd_intr_timo; static kbd_callback_func_t kbdmux_kbd_event; static void kbdmux_kbd_putc(kbdmux_state_t *state, char c) { unsigned int p; if (state->ks_inq_length == KBDMUX_Q_SIZE) return; p = (state->ks_inq_start + state->ks_inq_length) % KBDMUX_Q_SIZE; state->ks_inq[p] = c; state->ks_inq_length++; } static int kbdmux_kbd_getc(kbdmux_state_t *state) { unsigned char c; if (state->ks_inq_length == 0) return (-1); c = state->ks_inq[state->ks_inq_start]; state->ks_inq_start = (state->ks_inq_start + 1) % KBDMUX_Q_SIZE; state->ks_inq_length--; return (c); } /* * Interrupt handler task */ void kbdmux_kbd_intr(void *xkbd, int pending) { keyboard_t *kbd = (keyboard_t *) xkbd; kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; kbdd_intr(kbd, NULL); KBDMUX_LOCK(state); state->ks_flags &= ~TASK; wakeup(&state->ks_task); KBDMUX_UNLOCK(state); } /* * Schedule interrupt handler on timeout. Called with locked state. */ void kbdmux_kbd_intr_timo(void *xstate) { kbdmux_state_t *state = (kbdmux_state_t *) xstate; KBDMUX_LOCK_ASSERT(state, MA_OWNED); if (callout_pending(&state->ks_timo)) return; /* callout was reset */ if (!callout_active(&state->ks_timo)) return; /* callout was stopped */ callout_deactivate(&state->ks_timo); /* queue interrupt task if needed */ if (state->ks_inq_length > 0 && !(state->ks_flags & TASK) && KBDMUX_QUEUE_INTR(state) == 0) state->ks_flags |= TASK; /* re-schedule timeout */ callout_reset(&state->ks_timo, TICKS, kbdmux_kbd_intr_timo, state); } /* * Process event from one of our keyboards */ static int kbdmux_kbd_event(keyboard_t *kbd, int event, void *arg) { kbdmux_state_t *state = (kbdmux_state_t *) arg; switch (event) { case KBDIO_KEYINPUT: { int c; KBDMUX_LOCK(state); /* * Read all chars from the keyboard * * Turns out that atkbd(4) check_char() method may return * "true" while read_char() method returns NOKEY. If this * happens we could stuck in the loop below. Avoid this * by breaking out of the loop if read_char() method returns * NOKEY. */ while (kbdd_check_char(kbd)) { c = kbdd_read_char(kbd, 0); if (c == NOKEY) break; if (c == ERRKEY) continue; /* XXX ring bell */ if (!KBD_IS_BUSY(kbd)) continue; /* not open - discard the input */ kbdmux_kbd_putc(state, c); } /* queue interrupt task if needed */ if (state->ks_inq_length > 0 && !(state->ks_flags & TASK) && KBDMUX_QUEUE_INTR(state) == 0) state->ks_flags |= TASK; KBDMUX_UNLOCK(state); } break; case KBDIO_UNLOADING: { kbdmux_kbd_t *k; KBDMUX_LOCK(state); SLIST_FOREACH(k, &state->ks_kbds, next) if (k->kbd == kbd) break; if (k != NULL) { kbd_release(k->kbd, &k->kbd); SLIST_REMOVE(&state->ks_kbds, k, kbdmux_kbd, next); k->kbd = NULL; free(k, M_KBDMUX); } KBDMUX_UNLOCK(state); } break; default: return (EINVAL); /* NOT REACHED */ } return (0); } /**************************************************************************** **************************************************************************** ** Keyboard driver **************************************************************************** ****************************************************************************/ static int kbdmux_configure(int flags); static kbd_probe_t kbdmux_probe; static kbd_init_t kbdmux_init; static kbd_term_t kbdmux_term; static kbd_intr_t kbdmux_intr; static kbd_test_if_t kbdmux_test_if; static kbd_enable_t kbdmux_enable; static kbd_disable_t kbdmux_disable; static kbd_read_t kbdmux_read; static kbd_check_t kbdmux_check; static kbd_read_char_t kbdmux_read_char; static kbd_check_char_t kbdmux_check_char; static kbd_ioctl_t kbdmux_ioctl; static kbd_lock_t kbdmux_lock; static void kbdmux_clear_state_locked(kbdmux_state_t *state); static kbd_clear_state_t kbdmux_clear_state; static kbd_get_state_t kbdmux_get_state; static kbd_set_state_t kbdmux_set_state; static kbd_poll_mode_t kbdmux_poll; static keyboard_switch_t kbdmuxsw = { .probe = kbdmux_probe, .init = kbdmux_init, .term = kbdmux_term, .intr = kbdmux_intr, .test_if = kbdmux_test_if, .enable = kbdmux_enable, .disable = kbdmux_disable, .read = kbdmux_read, .check = kbdmux_check, .read_char = kbdmux_read_char, .check_char = kbdmux_check_char, .ioctl = kbdmux_ioctl, .lock = kbdmux_lock, .clear_state = kbdmux_clear_state, .get_state = kbdmux_get_state, .set_state = kbdmux_set_state, .get_fkeystr = genkbd_get_fkeystr, .poll = kbdmux_poll, .diag = genkbd_diag, }; /* * Return the number of found keyboards */ static int kbdmux_configure(int flags) { return (1); } /* * Detect a keyboard */ static int kbdmux_probe(int unit, void *arg, int flags) { if (resource_disabled(KEYBOARD_NAME, unit)) return (ENXIO); return (0); } /* * Reset and initialize the keyboard (stolen from atkbd.c) */ static int kbdmux_init(int unit, keyboard_t **kbdp, void *arg, int flags) { keyboard_t *kbd = NULL; kbdmux_state_t *state = NULL; keymap_t *keymap = NULL; accentmap_t *accmap = NULL; fkeytab_t *fkeymap = NULL; int error, needfree, fkeymap_size, delay[2]; if (*kbdp == NULL) { *kbdp = kbd = malloc(sizeof(*kbd), M_KBDMUX, M_NOWAIT | M_ZERO); state = malloc(sizeof(*state), M_KBDMUX, M_NOWAIT | M_ZERO); keymap = malloc(sizeof(key_map), M_KBDMUX, M_NOWAIT); accmap = malloc(sizeof(accent_map), M_KBDMUX, M_NOWAIT); fkeymap = malloc(sizeof(fkey_tab), M_KBDMUX, M_NOWAIT); fkeymap_size = sizeof(fkey_tab)/sizeof(fkey_tab[0]); needfree = 1; if ((kbd == NULL) || (state == NULL) || (keymap == NULL) || (accmap == NULL) || (fkeymap == NULL)) { error = ENOMEM; goto bad; } KBDMUX_LOCK_INIT(state); TASK_INIT(&state->ks_task, 0, kbdmux_kbd_intr, (void *) kbd); KBDMUX_CALLOUT_INIT(state); SLIST_INIT(&state->ks_kbds); } else if (KBD_IS_INITIALIZED(*kbdp) && KBD_IS_CONFIGURED(*kbdp)) { return (0); } else { kbd = *kbdp; state = (kbdmux_state_t *) kbd->kb_data; keymap = kbd->kb_keymap; accmap = kbd->kb_accentmap; fkeymap = kbd->kb_fkeytab; fkeymap_size = kbd->kb_fkeytab_size; needfree = 0; } if (!KBD_IS_PROBED(kbd)) { /* XXX assume 101/102 keys keyboard */ kbd_init_struct(kbd, KEYBOARD_NAME, KB_101, unit, flags, 0, 0); bcopy(&key_map, keymap, sizeof(key_map)); bcopy(&accent_map, accmap, sizeof(accent_map)); bcopy(fkey_tab, fkeymap, imin(fkeymap_size*sizeof(fkeymap[0]), sizeof(fkey_tab))); kbd_set_maps(kbd, keymap, accmap, fkeymap, fkeymap_size); kbd->kb_data = (void *)state; KBD_FOUND_DEVICE(kbd); KBD_PROBE_DONE(kbd); KBDMUX_LOCK(state); kbdmux_clear_state_locked(state); state->ks_mode = K_XLATE; KBDMUX_UNLOCK(state); } if (!KBD_IS_INITIALIZED(kbd) && !(flags & KB_CONF_PROBE_ONLY)) { kbd->kb_config = flags & ~KB_CONF_PROBE_ONLY; kbdmux_ioctl(kbd, KDSETLED, (caddr_t)&state->ks_state); delay[0] = kbd->kb_delay1; delay[1] = kbd->kb_delay2; kbdmux_ioctl(kbd, KDSETREPEAT, (caddr_t)delay); KBD_INIT_DONE(kbd); } if (!KBD_IS_CONFIGURED(kbd)) { if (kbd_register(kbd) < 0) { error = ENXIO; goto bad; } KBD_CONFIG_DONE(kbd); KBDMUX_LOCK(state); callout_reset(&state->ks_timo, TICKS, kbdmux_kbd_intr_timo, state); KBDMUX_UNLOCK(state); } return (0); bad: if (needfree) { if (state != NULL) free(state, M_KBDMUX); if (keymap != NULL) free(keymap, M_KBDMUX); if (accmap != NULL) free(accmap, M_KBDMUX); if (fkeymap != NULL) free(fkeymap, M_KBDMUX); if (kbd != NULL) { free(kbd, M_KBDMUX); *kbdp = NULL; /* insure ref doesn't leak to caller */ } } return (error); } /* * Finish using this keyboard */ static int kbdmux_term(keyboard_t *kbd) { kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; kbdmux_kbd_t *k; KBDMUX_LOCK(state); /* kill callout */ callout_stop(&state->ks_timo); /* wait for interrupt task */ while (state->ks_flags & TASK) KBDMUX_SLEEP(state, ks_task, "kbdmuxc", 0); /* release all keyboards from the mux */ while ((k = SLIST_FIRST(&state->ks_kbds)) != NULL) { kbd_release(k->kbd, &k->kbd); SLIST_REMOVE_HEAD(&state->ks_kbds, next); k->kbd = NULL; free(k, M_KBDMUX); } KBDMUX_UNLOCK(state); kbd_unregister(kbd); KBDMUX_LOCK_DESTROY(state); bzero(state, sizeof(*state)); free(state, M_KBDMUX); free(kbd->kb_keymap, M_KBDMUX); free(kbd->kb_accentmap, M_KBDMUX); free(kbd->kb_fkeytab, M_KBDMUX); free(kbd, M_KBDMUX); return (0); } /* * Keyboard interrupt routine */ static int kbdmux_intr(keyboard_t *kbd, void *arg) { int c; if (KBD_IS_ACTIVE(kbd) && KBD_IS_BUSY(kbd)) { /* let the callback function to process the input */ (*kbd->kb_callback.kc_func)(kbd, KBDIO_KEYINPUT, kbd->kb_callback.kc_arg); } else { /* read and discard the input; no one is waiting for input */ do { c = kbdmux_read_char(kbd, FALSE); } while (c != NOKEY); } return (0); } /* * Test the interface to the device */ static int kbdmux_test_if(keyboard_t *kbd) { return (0); } /* * Enable the access to the device; until this function is called, * the client cannot read from the keyboard. */ static int kbdmux_enable(keyboard_t *kbd) { KBD_ACTIVATE(kbd); return (0); } /* * Disallow the access to the device */ static int kbdmux_disable(keyboard_t *kbd) { KBD_DEACTIVATE(kbd); return (0); } /* * Read one byte from the keyboard if it's allowed */ static int kbdmux_read(keyboard_t *kbd, int wait) { kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; int c; KBDMUX_LOCK(state); c = kbdmux_kbd_getc(state); KBDMUX_UNLOCK(state); if (c != -1) kbd->kb_count ++; return (KBD_IS_ACTIVE(kbd)? c : -1); } /* * Check if data is waiting */ static int kbdmux_check(keyboard_t *kbd) { kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; int ready; if (!KBD_IS_ACTIVE(kbd)) return (FALSE); KBDMUX_LOCK(state); ready = (state->ks_inq_length > 0) ? TRUE : FALSE; KBDMUX_UNLOCK(state); return (ready); } /* * Read char from the keyboard (stolen from atkbd.c) */ static u_int kbdmux_read_char(keyboard_t *kbd, int wait) { kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; u_int action; int scancode, keycode; KBDMUX_LOCK(state); next_code: /* do we have a composed char to return? */ if (!(state->ks_flags & COMPOSE) && (state->ks_composed_char > 0)) { action = state->ks_composed_char; state->ks_composed_char = 0; if (action > UCHAR_MAX) { KBDMUX_UNLOCK(state); return (ERRKEY); } KBDMUX_UNLOCK(state); return (action); } /* see if there is something in the keyboard queue */ scancode = kbdmux_kbd_getc(state); if (scancode == -1) { - if (state->ks_flags & POLLING) { + if (state->ks_polling != 0) { kbdmux_kbd_t *k; SLIST_FOREACH(k, &state->ks_kbds, next) { while (kbdd_check_char(k->kbd)) { scancode = kbdd_read_char(k->kbd, 0); if (scancode == NOKEY) break; if (scancode == ERRKEY) continue; if (!KBD_IS_BUSY(k->kbd)) continue; kbdmux_kbd_putc(state, scancode); } } if (state->ks_inq_length > 0) goto next_code; } KBDMUX_UNLOCK(state); return (NOKEY); } /* XXX FIXME: check for -1 if wait == 1! */ kbd->kb_count ++; /* return the byte as is for the K_RAW mode */ if (state->ks_mode == K_RAW) { KBDMUX_UNLOCK(state); return (scancode); } /* translate the scan code into a keycode */ keycode = scancode & 0x7F; switch (state->ks_prefix) { case 0x00: /* normal scancode */ switch(scancode) { case 0xB8: /* left alt (compose key) released */ if (state->ks_flags & COMPOSE) { state->ks_flags &= ~COMPOSE; if (state->ks_composed_char > UCHAR_MAX) state->ks_composed_char = 0; } break; case 0x38: /* left alt (compose key) pressed */ if (!(state->ks_flags & COMPOSE)) { state->ks_flags |= COMPOSE; state->ks_composed_char = 0; } break; case 0xE0: case 0xE1: state->ks_prefix = scancode; goto next_code; } break; case 0xE0: /* 0xE0 prefix */ state->ks_prefix = 0; switch (keycode) { case 0x1C: /* right enter key */ keycode = 0x59; break; case 0x1D: /* right ctrl key */ keycode = 0x5A; break; case 0x35: /* keypad divide key */ keycode = 0x5B; break; case 0x37: /* print scrn key */ keycode = 0x5C; break; case 0x38: /* right alt key (alt gr) */ keycode = 0x5D; break; case 0x46: /* ctrl-pause/break on AT 101 (see below) */ keycode = 0x68; break; case 0x47: /* grey home key */ keycode = 0x5E; break; case 0x48: /* grey up arrow key */ keycode = 0x5F; break; case 0x49: /* grey page up key */ keycode = 0x60; break; case 0x4B: /* grey left arrow key */ keycode = 0x61; break; case 0x4D: /* grey right arrow key */ keycode = 0x62; break; case 0x4F: /* grey end key */ keycode = 0x63; break; case 0x50: /* grey down arrow key */ keycode = 0x64; break; case 0x51: /* grey page down key */ keycode = 0x65; break; case 0x52: /* grey insert key */ keycode = 0x66; break; case 0x53: /* grey delete key */ keycode = 0x67; break; /* the following 3 are only used on the MS "Natural" keyboard */ case 0x5b: /* left Window key */ keycode = 0x69; break; case 0x5c: /* right Window key */ keycode = 0x6a; break; case 0x5d: /* menu key */ keycode = 0x6b; break; case 0x5e: /* power key */ keycode = 0x6d; break; case 0x5f: /* sleep key */ keycode = 0x6e; break; case 0x63: /* wake key */ keycode = 0x6f; break; case 0x64: /* [JP106USB] backslash, underscore */ keycode = 0x73; break; default: /* ignore everything else */ goto next_code; } break; case 0xE1: /* 0xE1 prefix */ /* * The pause/break key on the 101 keyboard produces: * E1-1D-45 E1-9D-C5 * Ctrl-pause/break produces: * E0-46 E0-C6 (See above.) */ state->ks_prefix = 0; if (keycode == 0x1D) state->ks_prefix = 0x1D; goto next_code; /* NOT REACHED */ case 0x1D: /* pause / break */ state->ks_prefix = 0; if (keycode != 0x45) goto next_code; keycode = 0x68; break; } /* XXX assume 101/102 keys AT keyboard */ switch (keycode) { case 0x5c: /* print screen */ if (state->ks_flags & ALTS) keycode = 0x54; /* sysrq */ break; case 0x68: /* pause/break */ if (state->ks_flags & CTLS) keycode = 0x6c; /* break */ break; } /* return the key code in the K_CODE mode */ if (state->ks_mode == K_CODE) { KBDMUX_UNLOCK(state); return (keycode | (scancode & 0x80)); } /* compose a character code */ if (state->ks_flags & COMPOSE) { switch (keycode | (scancode & 0x80)) { /* key pressed, process it */ case 0x47: case 0x48: case 0x49: /* keypad 7,8,9 */ state->ks_composed_char *= 10; state->ks_composed_char += keycode - 0x40; if (state->ks_composed_char > UCHAR_MAX) { KBDMUX_UNLOCK(state); return (ERRKEY); } goto next_code; case 0x4B: case 0x4C: case 0x4D: /* keypad 4,5,6 */ state->ks_composed_char *= 10; state->ks_composed_char += keycode - 0x47; if (state->ks_composed_char > UCHAR_MAX) { KBDMUX_UNLOCK(state); return (ERRKEY); } goto next_code; case 0x4F: case 0x50: case 0x51: /* keypad 1,2,3 */ state->ks_composed_char *= 10; state->ks_composed_char += keycode - 0x4E; if (state->ks_composed_char > UCHAR_MAX) { KBDMUX_UNLOCK(state); return (ERRKEY); } goto next_code; case 0x52: /* keypad 0 */ state->ks_composed_char *= 10; if (state->ks_composed_char > UCHAR_MAX) { KBDMUX_UNLOCK(state); return (ERRKEY); } goto next_code; /* key released, no interest here */ case 0xC7: case 0xC8: case 0xC9: /* keypad 7,8,9 */ case 0xCB: case 0xCC: case 0xCD: /* keypad 4,5,6 */ case 0xCF: case 0xD0: case 0xD1: /* keypad 1,2,3 */ case 0xD2: /* keypad 0 */ goto next_code; case 0x38: /* left alt key */ break; default: if (state->ks_composed_char > 0) { state->ks_flags &= ~COMPOSE; state->ks_composed_char = 0; KBDMUX_UNLOCK(state); return (ERRKEY); } break; } } /* keycode to key action */ action = genkbd_keyaction(kbd, keycode, scancode & 0x80, &state->ks_state, &state->ks_accents); if (action == NOKEY) goto next_code; KBDMUX_UNLOCK(state); return (action); } /* * Check if char is waiting */ static int kbdmux_check_char(keyboard_t *kbd) { kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; int ready; if (!KBD_IS_ACTIVE(kbd)) return (FALSE); KBDMUX_LOCK(state); if (!(state->ks_flags & COMPOSE) && (state->ks_composed_char != 0)) ready = TRUE; else ready = (state->ks_inq_length > 0) ? TRUE : FALSE; KBDMUX_UNLOCK(state); return (ready); } /* * Keyboard ioctl's */ static int kbdmux_ioctl(keyboard_t *kbd, u_long cmd, caddr_t arg) { static int delays[] = { 250, 500, 750, 1000 }; static int rates[] = { 34, 38, 42, 46, 50, 55, 59, 63, 68, 76, 84, 92, 100, 110, 118, 126, 136, 152, 168, 184, 200, 220, 236, 252, 272, 304, 336, 368, 400, 440, 472, 504 }; kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; kbdmux_kbd_t *k; keyboard_info_t *ki; int error = 0, mode; #ifdef COMPAT_FREEBSD6 int ival; #endif if (state == NULL) return (ENXIO); switch (cmd) { case KBADDKBD: /* add keyboard to the mux */ ki = (keyboard_info_t *) arg; if (ki == NULL || ki->kb_unit < 0 || ki->kb_name[0] == '\0' || strcmp(ki->kb_name, "*") == 0) return (EINVAL); /* bad input */ KBDMUX_LOCK(state); SLIST_FOREACH(k, &state->ks_kbds, next) if (k->kbd->kb_unit == ki->kb_unit && strcmp(k->kbd->kb_name, ki->kb_name) == 0) break; if (k != NULL) { KBDMUX_UNLOCK(state); return (0); /* keyboard already in the mux */ } k = malloc(sizeof(*k), M_KBDMUX, M_NOWAIT | M_ZERO); if (k == NULL) { KBDMUX_UNLOCK(state); return (ENOMEM); /* out of memory */ } k->kbd = kbd_get_keyboard( kbd_allocate( ki->kb_name, ki->kb_unit, (void *) &k->kbd, kbdmux_kbd_event, (void *) state)); if (k->kbd == NULL) { KBDMUX_UNLOCK(state); free(k, M_KBDMUX); return (EINVAL); /* bad keyboard */ } kbdd_enable(k->kbd); kbdd_clear_state(k->kbd); /* set K_RAW mode on slave keyboard */ mode = K_RAW; error = kbdd_ioctl(k->kbd, KDSKBMODE, (caddr_t)&mode); if (error == 0) { /* set lock keys state on slave keyboard */ mode = state->ks_state & LOCK_MASK; error = kbdd_ioctl(k->kbd, KDSKBSTATE, (caddr_t)&mode); } if (error != 0) { KBDMUX_UNLOCK(state); kbd_release(k->kbd, &k->kbd); k->kbd = NULL; free(k, M_KBDMUX); return (error); /* could not set mode */ } SLIST_INSERT_HEAD(&state->ks_kbds, k, next); KBDMUX_UNLOCK(state); break; case KBRELKBD: /* release keyboard from the mux */ ki = (keyboard_info_t *) arg; if (ki == NULL || ki->kb_unit < 0 || ki->kb_name[0] == '\0' || strcmp(ki->kb_name, "*") == 0) return (EINVAL); /* bad input */ KBDMUX_LOCK(state); SLIST_FOREACH(k, &state->ks_kbds, next) if (k->kbd->kb_unit == ki->kb_unit && strcmp(k->kbd->kb_name, ki->kb_name) == 0) break; if (k != NULL) { error = kbd_release(k->kbd, &k->kbd); if (error == 0) { SLIST_REMOVE(&state->ks_kbds, k, kbdmux_kbd, next); k->kbd = NULL; free(k, M_KBDMUX); } } else error = ENXIO; /* keyboard is not in the mux */ KBDMUX_UNLOCK(state); break; case KDGKBMODE: /* get kyboard mode */ KBDMUX_LOCK(state); *(int *)arg = state->ks_mode; KBDMUX_UNLOCK(state); break; #ifdef COMPAT_FREEBSD6 case _IO('K', 7): ival = IOCPARM_IVAL(arg); arg = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDSKBMODE: /* set keyboard mode */ KBDMUX_LOCK(state); switch (*(int *)arg) { case K_XLATE: if (state->ks_mode != K_XLATE) { /* make lock key state and LED state match */ state->ks_state &= ~LOCK_MASK; state->ks_state |= KBD_LED_VAL(kbd); } /* FALLTHROUGH */ case K_RAW: case K_CODE: if (state->ks_mode != *(int *)arg) { kbdmux_clear_state_locked(state); state->ks_mode = *(int *)arg; } break; default: error = EINVAL; break; } KBDMUX_UNLOCK(state); break; case KDGETLED: /* get keyboard LED */ KBDMUX_LOCK(state); *(int *)arg = KBD_LED_VAL(kbd); KBDMUX_UNLOCK(state); break; #ifdef COMPAT_FREEBSD6 case _IO('K', 66): ival = IOCPARM_IVAL(arg); arg = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDSETLED: /* set keyboard LED */ KBDMUX_LOCK(state); /* NOTE: lock key state in ks_state won't be changed */ if (*(int *)arg & ~LOCK_MASK) { KBDMUX_UNLOCK(state); return (EINVAL); } KBD_LED_VAL(kbd) = *(int *)arg; /* KDSETLED on all slave keyboards */ SLIST_FOREACH(k, &state->ks_kbds, next) (void)kbdd_ioctl(k->kbd, KDSETLED, arg); KBDMUX_UNLOCK(state); break; case KDGKBSTATE: /* get lock key state */ KBDMUX_LOCK(state); *(int *)arg = state->ks_state & LOCK_MASK; KBDMUX_UNLOCK(state); break; #ifdef COMPAT_FREEBSD6 case _IO('K', 20): ival = IOCPARM_IVAL(arg); arg = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDSKBSTATE: /* set lock key state */ KBDMUX_LOCK(state); if (*(int *)arg & ~LOCK_MASK) { KBDMUX_UNLOCK(state); return (EINVAL); } state->ks_state &= ~LOCK_MASK; state->ks_state |= *(int *)arg; /* KDSKBSTATE on all slave keyboards */ SLIST_FOREACH(k, &state->ks_kbds, next) (void)kbdd_ioctl(k->kbd, KDSKBSTATE, arg); KBDMUX_UNLOCK(state); return (kbdmux_ioctl(kbd, KDSETLED, arg)); /* NOT REACHED */ #ifdef COMPAT_FREEBSD6 case _IO('K', 67): cmd = KDSETRAD; ival = IOCPARM_IVAL(arg); arg = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDSETREPEAT: /* set keyboard repeat rate (new interface) */ case KDSETRAD: /* set keyboard repeat rate (old interface) */ KBDMUX_LOCK(state); if (cmd == KDSETREPEAT) { int i; /* lookup delay */ for (i = sizeof(delays)/sizeof(delays[0]) - 1; i > 0; i --) if (((int *)arg)[0] >= delays[i]) break; mode = i << 5; /* lookup rate */ for (i = sizeof(rates)/sizeof(rates[0]) - 1; i > 0; i --) if (((int *)arg)[1] >= rates[i]) break; mode |= i; } else mode = *(int *)arg; if (mode & ~0x7f) { KBDMUX_UNLOCK(state); return (EINVAL); } kbd->kb_delay1 = delays[(mode >> 5) & 3]; kbd->kb_delay2 = rates[mode & 0x1f]; /* perform command on all slave keyboards */ SLIST_FOREACH(k, &state->ks_kbds, next) (void)kbdd_ioctl(k->kbd, cmd, arg); KBDMUX_UNLOCK(state); break; case PIO_KEYMAP: /* set keyboard translation table */ case OPIO_KEYMAP: /* set keyboard translation table (compat) */ case PIO_KEYMAPENT: /* set keyboard translation table entry */ case PIO_DEADKEYMAP: /* set accent key translation table */ KBDMUX_LOCK(state); state->ks_accents = 0; /* perform command on all slave keyboards */ SLIST_FOREACH(k, &state->ks_kbds, next) (void)kbdd_ioctl(k->kbd, cmd, arg); KBDMUX_UNLOCK(state); /* FALLTHROUGH */ default: error = genkbd_commonioctl(kbd, cmd, arg); break; } return (error); } /* * Lock the access to the keyboard */ static int kbdmux_lock(keyboard_t *kbd, int lock) { return (1); /* XXX */ } /* * Clear the internal state of the keyboard */ static void kbdmux_clear_state_locked(kbdmux_state_t *state) { KBDMUX_LOCK_ASSERT(state, MA_OWNED); - state->ks_flags &= ~(COMPOSE|POLLING); + state->ks_flags &= ~COMPOSE; + state->ks_polling = 0; state->ks_state &= LOCK_MASK; /* preserve locking key state */ state->ks_accents = 0; state->ks_composed_char = 0; /* state->ks_prefix = 0; XXX */ state->ks_inq_length = 0; } static void kbdmux_clear_state(keyboard_t *kbd) { kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; KBDMUX_LOCK(state); kbdmux_clear_state_locked(state); KBDMUX_UNLOCK(state); } /* * Save the internal state */ static int kbdmux_get_state(keyboard_t *kbd, void *buf, size_t len) { if (len == 0) return (sizeof(kbdmux_state_t)); if (len < sizeof(kbdmux_state_t)) return (-1); bcopy(kbd->kb_data, buf, sizeof(kbdmux_state_t)); /* XXX locking? */ return (0); } /* * Set the internal state */ static int kbdmux_set_state(keyboard_t *kbd, void *buf, size_t len) { if (len < sizeof(kbdmux_state_t)) return (ENOMEM); bcopy(buf, kbd->kb_data, sizeof(kbdmux_state_t)); /* XXX locking? */ return (0); } /* * Set polling */ static int kbdmux_poll(keyboard_t *kbd, int on) { kbdmux_state_t *state = (kbdmux_state_t *) kbd->kb_data; kbdmux_kbd_t *k; KBDMUX_LOCK(state); if (on) - state->ks_flags |= POLLING; + state->ks_polling++; else - state->ks_flags &= ~POLLING; + state->ks_polling--; /* set poll on slave keyboards */ SLIST_FOREACH(k, &state->ks_kbds, next) kbdd_poll(k->kbd, on); KBDMUX_UNLOCK(state); return (0); } /***************************************************************************** ***************************************************************************** ** Module ***************************************************************************** *****************************************************************************/ KEYBOARD_DRIVER(kbdmux, kbdmuxsw, kbdmux_configure); static int kbdmux_modevent(module_t mod, int type, void *data) { keyboard_switch_t *sw; keyboard_t *kbd; int error; switch (type) { case MOD_LOAD: if ((error = kbd_add_driver(&kbdmux_kbd_driver)) != 0) break; if ((sw = kbd_get_switch(KEYBOARD_NAME)) == NULL) { kbd_delete_driver(&kbdmux_kbd_driver); error = ENXIO; break; } kbd = NULL; if ((error = (*sw->probe)(0, NULL, 0)) != 0 || (error = (*sw->init)(0, &kbd, NULL, 0)) != 0) { kbd_delete_driver(&kbdmux_kbd_driver); break; } #ifdef KBD_INSTALL_CDEV if ((error = kbd_attach(kbd)) != 0) { (*sw->term)(kbd); kbd_delete_driver(&kbdmux_kbd_driver); break; } #endif if ((error = (*sw->enable)(kbd)) != 0) { (*sw->disable)(kbd); #ifdef KBD_INSTALL_CDEV kbd_detach(kbd); #endif (*sw->term)(kbd); kbd_delete_driver(&kbdmux_kbd_driver); break; } break; case MOD_UNLOAD: if ((sw = kbd_get_switch(KEYBOARD_NAME)) == NULL) panic("kbd_get_switch(" KEYBOARD_NAME ") == NULL"); kbd = kbd_get_keyboard(kbd_find_keyboard(KEYBOARD_NAME, 0)); if (kbd != NULL) { (*sw->disable)(kbd); #ifdef KBD_INSTALL_CDEV kbd_detach(kbd); #endif (*sw->term)(kbd); kbd_delete_driver(&kbdmux_kbd_driver); } error = 0; break; default: error = EOPNOTSUPP; break; } return (error); } DEV_MODULE(kbdmux, kbdmux_modevent, NULL); Index: projects/netbsd-tests-update-12/sys/dev/netmap/if_ixl_netmap.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/netmap/if_ixl_netmap.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/netmap/if_ixl_netmap.h (revision 305172) @@ -1,422 +1,421 @@ /* * Copyright (C) 2015, Luigi Rizzo. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * $FreeBSD$ * * netmap support for: ixl * * derived from ixgbe * netmap support for a network driver. * This file contains code but only static or inline functions used * by a single driver. To avoid replication of code we just #include * it near the beginning of the standard driver. * For ixl the file is imported in two places, hence the conditional at the * beginning. */ #include #include /* * Some drivers may need the following headers. Others * already include them by default #include #include */ #include int ixl_netmap_txsync(struct netmap_kring *kring, int flags); int ixl_netmap_rxsync(struct netmap_kring *kring, int flags); extern int ixl_rx_miss, ixl_rx_miss_bufs, ixl_crcstrip; #ifdef NETMAP_IXL_MAIN /* * device-specific sysctl variables: * * ixl_crcstrip: 0: keep CRC in rx frames (default), 1: strip it. * During regular operations the CRC is stripped, but on some * hardware reception of frames not multiple of 64 is slower, * so using crcstrip=0 helps in benchmarks. * * ixl_rx_miss, ixl_rx_miss_bufs: * count packets that might be missed due to lost interrupts. */ SYSCTL_DECL(_dev_netmap); /* * The xl driver by default strips CRCs and we do not override it. */ -int ixl_rx_miss, ixl_rx_miss_bufs, ixl_crcstrip = 1; #if 0 SYSCTL_INT(_dev_netmap, OID_AUTO, ixl_crcstrip, CTLFLAG_RW, &ixl_crcstrip, 1, "strip CRC on rx frames"); #endif SYSCTL_INT(_dev_netmap, OID_AUTO, ixl_rx_miss, CTLFLAG_RW, &ixl_rx_miss, 0, "potentially missed rx intr"); SYSCTL_INT(_dev_netmap, OID_AUTO, ixl_rx_miss_bufs, CTLFLAG_RW, &ixl_rx_miss_bufs, 0, "potentially missed rx intr bufs"); /* * Register/unregister. We are already under netmap lock. * Only called on the first register or the last unregister. */ static int ixl_netmap_reg(struct netmap_adapter *na, int onoff) { struct ifnet *ifp = na->ifp; struct ixl_vsi *vsi = ifp->if_softc; struct ixl_pf *pf = (struct ixl_pf *)vsi->back; IXL_PF_LOCK(pf); ixl_disable_intr(vsi); /* Tell the stack that the interface is no longer active */ ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE); //set_crcstrip(&adapter->hw, onoff); /* enable or disable flags and callbacks in na and ifp */ if (onoff) { nm_set_native_flags(na); } else { nm_clear_native_flags(na); } ixl_init_locked(pf); /* also enables intr */ //set_crcstrip(&adapter->hw, onoff); // XXX why twice ? IXL_PF_UNLOCK(pf); return (ifp->if_drv_flags & IFF_DRV_RUNNING ? 0 : 1); } /* * The attach routine, called near the end of ixl_attach(), * fills the parameters for netmap_attach() and calls it. * It cannot fail, in the worst case (such as no memory) * netmap mode will be disabled and the driver will only * operate in standard mode. */ static void ixl_netmap_attach(struct ixl_vsi *vsi) { struct netmap_adapter na; bzero(&na, sizeof(na)); na.ifp = vsi->ifp; na.na_flags = NAF_BDG_MAYSLEEP; // XXX check that queues is set. printf("queues is %p\n", vsi->queues); if (vsi->queues) { na.num_tx_desc = vsi->queues[0].num_desc; na.num_rx_desc = vsi->queues[0].num_desc; } na.nm_txsync = ixl_netmap_txsync; na.nm_rxsync = ixl_netmap_rxsync; na.nm_register = ixl_netmap_reg; na.num_tx_rings = na.num_rx_rings = vsi->num_queues; netmap_attach(&na); } #else /* !NETMAP_IXL_MAIN, code for ixl_txrx.c */ /* * Reconcile kernel and user view of the transmit ring. * * All information is in the kring. * Userspace wants to send packets up to the one before kring->rhead, * kernel knows kring->nr_hwcur is the first unsent packet. * * Here we push packets out (as many as possible), and possibly * reclaim buffers from previously completed transmission. * * The caller (netmap) guarantees that there is only one instance * running at any time. Any interference with other driver * methods should be handled by the individual drivers. */ int ixl_netmap_txsync(struct netmap_kring *kring, int flags) { struct netmap_adapter *na = kring->na; struct ifnet *ifp = na->ifp; struct netmap_ring *ring = kring->ring; u_int nm_i; /* index into the netmap ring */ u_int nic_i; /* index into the NIC ring */ u_int n; u_int const lim = kring->nkr_num_slots - 1; u_int const head = kring->rhead; /* * interrupts on every tx packet are expensive so request * them every half ring, or where NS_REPORT is set */ u_int report_frequency = kring->nkr_num_slots >> 1; /* device-specific */ struct ixl_vsi *vsi = ifp->if_softc; struct ixl_queue *que = &vsi->queues[kring->ring_id]; struct tx_ring *txr = &que->txr; bus_dmamap_sync(txr->dma.tag, txr->dma.map, BUS_DMASYNC_POSTREAD); /* * First part: process new packets to send. * nm_i is the current index in the netmap ring, * nic_i is the corresponding index in the NIC ring. * * If we have packets to send (nm_i != head) * iterate over the netmap ring, fetch length and update * the corresponding slot in the NIC ring. Some drivers also * need to update the buffer's physical address in the NIC slot * even NS_BUF_CHANGED is not set (PNMB computes the addresses). * * The netmap_reload_map() calls is especially expensive, * even when (as in this case) the tag is 0, so do only * when the buffer has actually changed. * * If possible do not set the report/intr bit on all slots, * but only a few times per ring or when NS_REPORT is set. * * Finally, on 10G and faster drivers, it might be useful * to prefetch the next slot and txr entry. */ nm_i = kring->nr_hwcur; if (nm_i != head) { /* we have new packets to send */ nic_i = netmap_idx_k2n(kring, nm_i); __builtin_prefetch(&ring->slot[nm_i]); __builtin_prefetch(&txr->buffers[nic_i]); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; u_int len = slot->len; uint64_t paddr; void *addr = PNMB(na, slot, &paddr); /* device-specific */ struct i40e_tx_desc *curr = &txr->base[nic_i]; struct ixl_tx_buf *txbuf = &txr->buffers[nic_i]; u64 flags = (slot->flags & NS_REPORT || nic_i == 0 || nic_i == report_frequency) ? ((u64)I40E_TX_DESC_CMD_RS << I40E_TXD_QW1_CMD_SHIFT) : 0; /* prefetch for next round */ __builtin_prefetch(&ring->slot[nm_i + 1]); __builtin_prefetch(&txr->buffers[nic_i + 1]); NM_CHECK_ADDR_LEN(na, addr, len); if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ netmap_reload_map(na, txr->dma.tag, txbuf->map, addr); } slot->flags &= ~(NS_REPORT | NS_BUF_CHANGED); /* Fill the slot in the NIC ring. */ curr->buffer_addr = htole64(paddr); curr->cmd_type_offset_bsz = htole64( ((u64)len << I40E_TXD_QW1_TX_BUF_SZ_SHIFT) | flags | ((u64)I40E_TX_DESC_CMD_EOP << I40E_TXD_QW1_CMD_SHIFT) ); // XXX more ? /* make sure changes to the buffer are synced */ bus_dmamap_sync(txr->dma.tag, txbuf->map, BUS_DMASYNC_PREWRITE); nm_i = nm_next(nm_i, lim); nic_i = nm_next(nic_i, lim); } kring->nr_hwcur = head; /* synchronize the NIC ring */ bus_dmamap_sync(txr->dma.tag, txr->dma.map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* (re)start the tx unit up to slot nic_i (excluded) */ wr32(vsi->hw, txr->tail, nic_i); } /* * Second part: reclaim buffers for completed transmissions. */ nic_i = LE32_TO_CPU(*(volatile __le32 *)&txr->base[que->num_desc]); if (nic_i != txr->next_to_clean) { /* some tx completed, increment avail */ txr->next_to_clean = nic_i; kring->nr_hwtail = nm_prev(netmap_idx_n2k(kring, nic_i), lim); } return 0; } /* * Reconcile kernel and user view of the receive ring. * Same as for the txsync, this routine must be efficient. * The caller guarantees a single invocations, but races against * the rest of the driver should be handled here. * * On call, kring->rhead is the first packet that userspace wants * to keep, and kring->rcur is the wakeup point. * The kernel has previously reported packets up to kring->rtail. * * If (flags & NAF_FORCE_READ) also check for incoming packets irrespective * of whether or not we received an interrupt. */ int ixl_netmap_rxsync(struct netmap_kring *kring, int flags) { struct netmap_adapter *na = kring->na; struct ifnet *ifp = na->ifp; struct netmap_ring *ring = kring->ring; u_int nm_i; /* index into the netmap ring */ u_int nic_i; /* index into the NIC ring */ u_int n; u_int const lim = kring->nkr_num_slots - 1; u_int const head = kring->rhead; int force_update = (flags & NAF_FORCE_READ) || kring->nr_kflags & NKR_PENDINTR; /* device-specific */ struct ixl_vsi *vsi = ifp->if_softc; struct ixl_queue *que = &vsi->queues[kring->ring_id]; struct rx_ring *rxr = &que->rxr; if (head > lim) return netmap_ring_reinit(kring); /* XXX check sync modes */ bus_dmamap_sync(rxr->dma.tag, rxr->dma.map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); /* * First part: import newly received packets. * * nm_i is the index of the next free slot in the netmap ring, * nic_i is the index of the next received packet in the NIC ring, * and they may differ in case if_init() has been called while * in netmap mode. For the receive ring we have * * nic_i = rxr->next_check; * nm_i = kring->nr_hwtail (previous) * and * nm_i == (nic_i + kring->nkr_hwofs) % ring_size * * rxr->next_check is set to 0 on a ring reinit */ if (netmap_no_pendintr || force_update) { int crclen = ixl_crcstrip ? 0 : 4; uint16_t slot_flags = kring->nkr_slot_flags; nic_i = rxr->next_check; // or also k2n(kring->nr_hwtail) nm_i = netmap_idx_n2k(kring, nic_i); for (n = 0; ; n++) { union i40e_32byte_rx_desc *curr = &rxr->base[nic_i]; uint64_t qword = le64toh(curr->wb.qword1.status_error_len); uint32_t staterr = (qword & I40E_RXD_QW1_STATUS_MASK) >> I40E_RXD_QW1_STATUS_SHIFT; if ((staterr & (1<slot[nm_i].len = ((qword & I40E_RXD_QW1_LENGTH_PBUF_MASK) >> I40E_RXD_QW1_LENGTH_PBUF_SHIFT) - crclen; ring->slot[nm_i].flags = slot_flags; bus_dmamap_sync(rxr->ptag, rxr->buffers[nic_i].pmap, BUS_DMASYNC_POSTREAD); nm_i = nm_next(nm_i, lim); nic_i = nm_next(nic_i, lim); } if (n) { /* update the state variables */ if (netmap_no_pendintr && !force_update) { /* diagnostics */ ixl_rx_miss ++; ixl_rx_miss_bufs += n; } rxr->next_check = nic_i; kring->nr_hwtail = nm_i; } kring->nr_kflags &= ~NKR_PENDINTR; } /* * Second part: skip past packets that userspace has released. * (kring->nr_hwcur to head excluded), * and make the buffers available for reception. * As usual nm_i is the index in the netmap ring, * nic_i is the index in the NIC ring, and * nm_i == (nic_i + kring->nkr_hwofs) % ring_size */ nm_i = kring->nr_hwcur; if (nm_i != head) { nic_i = netmap_idx_k2n(kring, nm_i); for (n = 0; nm_i != head; n++) { struct netmap_slot *slot = &ring->slot[nm_i]; uint64_t paddr; void *addr = PNMB(na, slot, &paddr); union i40e_32byte_rx_desc *curr = &rxr->base[nic_i]; struct ixl_rx_buf *rxbuf = &rxr->buffers[nic_i]; if (addr == NETMAP_BUF_BASE(na)) /* bad buf */ goto ring_reset; if (slot->flags & NS_BUF_CHANGED) { /* buffer has changed, reload map */ netmap_reload_map(na, rxr->ptag, rxbuf->pmap, addr); slot->flags &= ~NS_BUF_CHANGED; } curr->read.pkt_addr = htole64(paddr); curr->read.hdr_addr = 0; // XXX needed bus_dmamap_sync(rxr->ptag, rxbuf->pmap, BUS_DMASYNC_PREREAD); nm_i = nm_next(nm_i, lim); nic_i = nm_next(nic_i, lim); } kring->nr_hwcur = head; bus_dmamap_sync(rxr->dma.tag, rxr->dma.map, BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); /* * IMPORTANT: we must leave one free slot in the ring, * so move nic_i back by one unit */ nic_i = nm_prev(nic_i, lim); wr32(vsi->hw, rxr->tail, nic_i); } return 0; ring_reset: return netmap_ring_reinit(kring); } #endif /* !NETMAP_IXL_MAIN */ /* end of file */ Index: projects/netbsd-tests-update-12/sys/dev/syscons/syscons.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/syscons/syscons.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/syscons/syscons.c (revision 305172) @@ -1,4004 +1,4047 @@ /*- * Copyright (c) 1992-1998 Søren Schmidt * All rights reserved. * * This code is derived from software contributed to The DragonFly Project * by Sascha Wildner * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification, immediately at the beginning of the file. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The name of the author may not be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include "opt_compat.h" #include "opt_syscons.h" #include "opt_splash.h" #include "opt_ddb.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #if defined(__arm__) || defined(__mips__) || \ defined(__powerpc__) || defined(__sparc64__) #include #else #include #endif #if defined( __i386__) || defined(__amd64__) #include #include #endif #include #include #include #include #include #define COLD 0 #define WARM 1 #define DEFAULT_BLANKTIME (5*60) /* 5 minutes */ #define MAX_BLANKTIME (7*24*60*60) /* 7 days!? */ #define KEYCODE_BS 0x0e /* "<-- Backspace" key, XXX */ /* NULL-safe version of "tty_opened()" */ #define tty_opened_ns(tp) ((tp) != NULL && tty_opened(tp)) typedef struct default_attr { int std_color; /* normal hardware color */ int rev_color; /* reverse hardware color */ } default_attr; static default_attr user_default = { SC_NORM_ATTR, SC_NORM_REV_ATTR, }; static int sc_console_unit = -1; static int sc_saver_keyb_only = 1; static scr_stat *sc_console; static struct consdev *sc_consptr; static scr_stat main_console; static struct tty *main_devs[MAXCONS]; static char init_done = COLD; static int shutdown_in_progress = FALSE; static int suspend_in_progress = FALSE; static char sc_malloc = FALSE; static int saver_mode = CONS_NO_SAVER; /* LKM/user saver */ static int run_scrn_saver = FALSE; /* should run the saver? */ static int enable_bell = TRUE; /* enable beeper */ #ifndef SC_DISABLE_REBOOT static int enable_reboot = TRUE; /* enable keyboard reboot */ #endif #ifndef SC_DISABLE_KDBKEY static int enable_kdbkey = TRUE; /* enable keyboard debug */ #endif static long scrn_blank_time = 0; /* screen saver timeout value */ #ifdef DEV_SPLASH static int scrn_blanked; /* # of blanked screen */ static int sticky_splash = FALSE; static void none_saver(sc_softc_t *sc, int blank) { } static void (*current_saver)(sc_softc_t *, int) = none_saver; #endif #ifdef SC_NO_SUSPEND_VTYSWITCH static int sc_no_suspend_vtswitch = 1; #else static int sc_no_suspend_vtswitch = 0; #endif static int sc_susp_scr; static SYSCTL_NODE(_hw, OID_AUTO, syscons, CTLFLAG_RD, 0, "syscons"); static SYSCTL_NODE(_hw_syscons, OID_AUTO, saver, CTLFLAG_RD, 0, "saver"); SYSCTL_INT(_hw_syscons_saver, OID_AUTO, keybonly, CTLFLAG_RW, &sc_saver_keyb_only, 0, "screen saver interrupted by input only"); SYSCTL_INT(_hw_syscons, OID_AUTO, bell, CTLFLAG_RW, &enable_bell, 0, "enable bell"); #ifndef SC_DISABLE_REBOOT SYSCTL_INT(_hw_syscons, OID_AUTO, kbd_reboot, CTLFLAG_RW|CTLFLAG_SECURE, &enable_reboot, 0, "enable keyboard reboot"); #endif #ifndef SC_DISABLE_KDBKEY SYSCTL_INT(_hw_syscons, OID_AUTO, kbd_debug, CTLFLAG_RW|CTLFLAG_SECURE, &enable_kdbkey, 0, "enable keyboard debug"); #endif SYSCTL_INT(_hw_syscons, OID_AUTO, sc_no_suspend_vtswitch, CTLFLAG_RWTUN, &sc_no_suspend_vtswitch, 0, "Disable VT switch before suspend."); #if !defined(SC_NO_FONT_LOADING) && defined(SC_DFLT_FONT) #include "font.h" #endif tsw_ioctl_t *sc_user_ioctl; static bios_values_t bios_value; static int enable_panic_key; SYSCTL_INT(_machdep, OID_AUTO, enable_panic_key, CTLFLAG_RW, &enable_panic_key, 0, "Enable panic via keypress specified in kbdmap(5)"); #define SC_CONSOLECTL 255 #define VTY_WCHAN(sc, vty) (&SC_DEV(sc, vty)) /* prototypes */ static int sc_allocate_keyboard(sc_softc_t *sc, int unit); static int scvidprobe(int unit, int flags, int cons); static int sckbdprobe(int unit, int flags, int cons); static void scmeminit(void *arg); static int scdevtounit(struct tty *tp); static kbd_callback_func_t sckbdevent; static void scinit(int unit, int flags); static scr_stat *sc_get_stat(struct tty *tp); static void scterm(int unit, int flags); static void scshutdown(void *, int); static void scsuspend(void *); static void scresume(void *); static u_int scgetc(sc_softc_t *sc, u_int flags, struct sc_cnstate *sp); static void sc_puts(scr_stat *scp, u_char *buf, int len, int kernel); #define SCGETC_CN 1 #define SCGETC_NONBLOCK 2 static void sccnupdate(scr_stat *scp); static scr_stat *alloc_scp(sc_softc_t *sc, int vty); static void init_scp(sc_softc_t *sc, int vty, scr_stat *scp); static timeout_t scrn_timer; static int and_region(int *s1, int *e1, int s2, int e2); static void scrn_update(scr_stat *scp, int show_cursor); #ifdef DEV_SPLASH static int scsplash_callback(int event, void *arg); static void scsplash_saver(sc_softc_t *sc, int show); static int add_scrn_saver(void (*this_saver)(sc_softc_t *, int)); static int remove_scrn_saver(void (*this_saver)(sc_softc_t *, int)); static int set_scrn_saver_mode(scr_stat *scp, int mode, u_char *pal, int border); static int restore_scrn_saver_mode(scr_stat *scp, int changemode); static void stop_scrn_saver(sc_softc_t *sc, void (*saver)(sc_softc_t *, int)); static int wait_scrn_saver_stop(sc_softc_t *sc); #define scsplash_stick(stick) (sticky_splash = (stick)) #else /* !DEV_SPLASH */ #define scsplash_stick(stick) #endif /* DEV_SPLASH */ static int do_switch_scr(sc_softc_t *sc, int s); static int vt_proc_alive(scr_stat *scp); static int signal_vt_rel(scr_stat *scp); static int signal_vt_acq(scr_stat *scp); static int finish_vt_rel(scr_stat *scp, int release, int *s); static int finish_vt_acq(scr_stat *scp); static void exchange_scr(sc_softc_t *sc); static void update_cursor_image(scr_stat *scp); static void change_cursor_shape(scr_stat *scp, int flags, int base, int height); static void update_font(scr_stat *); static int save_kbd_state(scr_stat *scp); static int update_kbd_state(scr_stat *scp, int state, int mask); static int update_kbd_leds(scr_stat *scp, int which); static timeout_t blink_screen; static struct tty *sc_alloc_tty(int, int); static cn_probe_t sc_cnprobe; static cn_init_t sc_cninit; static cn_term_t sc_cnterm; static cn_getc_t sc_cngetc; static cn_putc_t sc_cnputc; static cn_grab_t sc_cngrab; static cn_ungrab_t sc_cnungrab; CONSOLE_DRIVER(sc); static tsw_open_t sctty_open; static tsw_close_t sctty_close; static tsw_outwakeup_t sctty_outwakeup; static tsw_ioctl_t sctty_ioctl; static tsw_mmap_t sctty_mmap; static struct ttydevsw sc_ttydevsw = { .tsw_open = sctty_open, .tsw_close = sctty_close, .tsw_outwakeup = sctty_outwakeup, .tsw_ioctl = sctty_ioctl, .tsw_mmap = sctty_mmap, }; static d_ioctl_t consolectl_ioctl; static d_close_t consolectl_close; static struct cdevsw consolectl_devsw = { .d_version = D_VERSION, .d_flags = D_NEEDGIANT | D_TRACKCLOSE, .d_ioctl = consolectl_ioctl, .d_close = consolectl_close, .d_name = "consolectl", }; int sc_probe_unit(int unit, int flags) { if (!vty_enabled(VTY_SC)) return ENXIO; if (!scvidprobe(unit, flags, FALSE)) { if (bootverbose) printf("%s%d: no video adapter found.\n", SC_DRIVER_NAME, unit); return ENXIO; } /* syscons will be attached even when there is no keyboard */ sckbdprobe(unit, flags, FALSE); return 0; } /* probe video adapters, return TRUE if found */ static int scvidprobe(int unit, int flags, int cons) { /* * Access the video adapter driver through the back door! * Video adapter drivers need to be configured before syscons. * However, when syscons is being probed as the low-level console, * they have not been initialized yet. We force them to initialize * themselves here. XXX */ vid_configure(cons ? VIO_PROBE_ONLY : 0); return (vid_find_adapter("*", unit) >= 0); } /* probe the keyboard, return TRUE if found */ static int sckbdprobe(int unit, int flags, int cons) { /* access the keyboard driver through the backdoor! */ kbd_configure(cons ? KB_CONF_PROBE_ONLY : 0); return (kbd_find_keyboard("*", unit) >= 0); } static char *adapter_name(video_adapter_t *adp) { static struct { int type; char *name[2]; } names[] = { { KD_MONO, { "MDA", "MDA" } }, { KD_HERCULES, { "Hercules", "Hercules" } }, { KD_CGA, { "CGA", "CGA" } }, { KD_EGA, { "EGA", "EGA (mono)" } }, { KD_VGA, { "VGA", "VGA (mono)" } }, { KD_PC98, { "PC-98x1", "PC-98x1" } }, { KD_TGA, { "TGA", "TGA" } }, { -1, { "Unknown", "Unknown" } }, }; int i; for (i = 0; names[i].type != -1; ++i) if (names[i].type == adp->va_type) break; return names[i].name[(adp->va_flags & V_ADP_COLOR) ? 0 : 1]; } static void sctty_outwakeup(struct tty *tp) { size_t len; u_char buf[PCBURST]; scr_stat *scp = sc_get_stat(tp); if (scp->status & SLKED || (scp == scp->sc->cur_scp && scp->sc->blink_in_progress)) return; for (;;) { len = ttydisc_getc(tp, buf, sizeof buf); if (len == 0) break; SC_VIDEO_LOCK(scp->sc); sc_puts(scp, buf, len, 0); SC_VIDEO_UNLOCK(scp->sc); } } static struct tty * sc_alloc_tty(int index, int devnum) { struct sc_ttysoftc *stc; struct tty *tp; /* Allocate TTY object and softc to store unit number. */ stc = malloc(sizeof(struct sc_ttysoftc), M_DEVBUF, M_WAITOK); stc->st_index = index; stc->st_stat = NULL; tp = tty_alloc_mutex(&sc_ttydevsw, stc, &Giant); /* Create device node. */ tty_makedev(tp, NULL, "v%r", devnum); return (tp); } #ifdef SC_PIXEL_MODE static void sc_set_vesa_mode(scr_stat *scp, sc_softc_t *sc, int unit) { video_info_t info; u_char *font; int depth; int fontsize; int i; int vmode; vmode = 0; (void)resource_int_value("sc", unit, "vesa_mode", &vmode); if (vmode < M_VESA_BASE || vmode > M_VESA_MODE_MAX || vidd_get_info(sc->adp, vmode, &info) != 0 || !sc_support_pixel_mode(&info)) vmode = 0; /* * If the mode is unset or unsupported, search for an available * 800x600 graphics mode with the highest color depth. */ if (vmode == 0) { for (depth = 0, i = M_VESA_BASE; i <= M_VESA_MODE_MAX; i++) if (vidd_get_info(sc->adp, i, &info) == 0 && info.vi_width == 800 && info.vi_height == 600 && sc_support_pixel_mode(&info) && info.vi_depth > depth) { vmode = i; depth = info.vi_depth; } if (vmode == 0) return; vidd_get_info(sc->adp, vmode, &info); } #if !defined(SC_NO_FONT_LOADING) && defined(SC_DFLT_FONT) fontsize = info.vi_cheight; #else fontsize = scp->font_size; #endif if (fontsize < 14) fontsize = 8; else if (fontsize >= 16) fontsize = 16; else fontsize = 14; #ifndef SC_NO_FONT_LOADING switch (fontsize) { case 8: if ((sc->fonts_loaded & FONT_8) == 0) return; font = sc->font_8; break; case 14: if ((sc->fonts_loaded & FONT_14) == 0) return; font = sc->font_14; break; case 16: if ((sc->fonts_loaded & FONT_16) == 0) return; font = sc->font_16; break; } #else font = NULL; #endif #ifdef DEV_SPLASH if ((sc->flags & SC_SPLASH_SCRN) != 0) splash_term(sc->adp); #endif #ifndef SC_NO_HISTORY if (scp->history != NULL) { sc_vtb_append(&scp->vtb, 0, scp->history, scp->ypos * scp->xsize + scp->xpos); scp->history_pos = sc_vtb_tail(scp->history); } #endif vidd_set_mode(sc->adp, vmode); scp->status |= (UNKNOWN_MODE | PIXEL_MODE | MOUSE_HIDDEN); scp->status &= ~(GRAPHICS_MODE | MOUSE_VISIBLE); scp->xpixel = info.vi_width; scp->ypixel = info.vi_height; scp->xsize = scp->xpixel / 8; scp->ysize = scp->ypixel / fontsize; scp->xpos = 0; scp->ypos = scp->ysize - 1; scp->xoff = scp->yoff = 0; scp->font = font; scp->font_size = fontsize; scp->font_width = 8; scp->start = scp->xsize * scp->ysize - 1; scp->end = 0; scp->cursor_pos = scp->cursor_oldpos = scp->xsize * scp->xsize; scp->mode = sc->initial_mode = vmode; #ifndef __sparc64__ sc_vtb_init(&scp->scr, VTB_FRAMEBUFFER, scp->xsize, scp->ysize, (void *)sc->adp->va_window, FALSE); #endif sc_alloc_scr_buffer(scp, FALSE, FALSE); sc_init_emulator(scp, NULL); #ifndef SC_NO_CUTPASTE sc_alloc_cut_buffer(scp, FALSE); #endif #ifndef SC_NO_HISTORY sc_alloc_history_buffer(scp, 0, 0, FALSE); #endif sc_set_border(scp, scp->border); sc_set_cursor_image(scp); scp->status &= ~UNKNOWN_MODE; #ifdef DEV_SPLASH if ((sc->flags & SC_SPLASH_SCRN) != 0) splash_init(sc->adp, scsplash_callback, sc); #endif } #endif int sc_attach_unit(int unit, int flags) { sc_softc_t *sc; scr_stat *scp; struct cdev *dev; int vc; if (!vty_enabled(VTY_SC)) return ENXIO; flags &= ~SC_KERNEL_CONSOLE; if (sc_console_unit == unit) { /* * If this unit is being used as the system console, we need to * adjust some variables and buffers before and after scinit(). */ /* assert(sc_console != NULL) */ flags |= SC_KERNEL_CONSOLE; scmeminit(NULL); } scinit(unit, flags); sc = sc_get_softc(unit, flags & SC_KERNEL_CONSOLE); sc->config = flags; callout_init(&sc->ctimeout, 0); callout_init(&sc->cblink, 0); scp = sc_get_stat(sc->dev[0]); if (sc_console == NULL) /* sc_console_unit < 0 */ sc_console = scp; #ifdef SC_PIXEL_MODE if ((sc->config & SC_VESAMODE) != 0) sc_set_vesa_mode(scp, sc, unit); #endif /* SC_PIXEL_MODE */ /* initialize cursor */ if (!ISGRAPHSC(scp)) update_cursor_image(scp); /* get screen update going */ scrn_timer(sc); /* set up the keyboard */ (void)kbdd_ioctl(sc->kbd, KDSKBMODE, (caddr_t)&scp->kbd_mode); update_kbd_state(scp, scp->status, LOCK_MASK); printf("%s%d: %s <%d virtual consoles, flags=0x%x>\n", SC_DRIVER_NAME, unit, adapter_name(sc->adp), sc->vtys, sc->config); if (bootverbose) { printf("%s%d:", SC_DRIVER_NAME, unit); if (sc->adapter >= 0) printf(" fb%d", sc->adapter); if (sc->keyboard >= 0) printf(", kbd%d", sc->keyboard); if (scp->tsw) printf(", terminal emulator: %s (%s)", scp->tsw->te_name, scp->tsw->te_desc); printf("\n"); } /* Register suspend/resume/shutdown callbacks for the kernel console. */ if (sc_console_unit == unit) { EVENTHANDLER_REGISTER(power_suspend_early, scsuspend, NULL, EVENTHANDLER_PRI_ANY); EVENTHANDLER_REGISTER(power_resume, scresume, NULL, EVENTHANDLER_PRI_ANY); EVENTHANDLER_REGISTER(shutdown_pre_sync, scshutdown, NULL, SHUTDOWN_PRI_DEFAULT); } for (vc = 0; vc < sc->vtys; vc++) { if (sc->dev[vc] == NULL) { sc->dev[vc] = sc_alloc_tty(vc, vc + unit * MAXCONS); if (vc == 0 && sc->dev == main_devs) SC_STAT(sc->dev[0]) = &main_console; } /* * The first vty already has struct tty and scr_stat initialized * in scinit(). The other vtys will have these structs when * first opened. */ } dev = make_dev(&consolectl_devsw, 0, UID_ROOT, GID_WHEEL, 0600, "consolectl"); dev->si_drv1 = sc->dev[0]; return 0; } static void scmeminit(void *arg) { if (!vty_enabled(VTY_SC)) return; if (sc_malloc) return; sc_malloc = TRUE; /* * As soon as malloc() becomes functional, we had better allocate * various buffers for the kernel console. */ if (sc_console_unit < 0) /* sc_console == NULL */ return; /* copy the temporary buffer to the final buffer */ sc_alloc_scr_buffer(sc_console, FALSE, FALSE); #ifndef SC_NO_CUTPASTE sc_alloc_cut_buffer(sc_console, FALSE); #endif #ifndef SC_NO_HISTORY /* initialize history buffer & pointers */ sc_alloc_history_buffer(sc_console, 0, 0, FALSE); #endif } /* XXX */ SYSINIT(sc_mem, SI_SUB_KMEM, SI_ORDER_ANY, scmeminit, NULL); static int scdevtounit(struct tty *tp) { int vty = SC_VTY(tp); if (vty == SC_CONSOLECTL) return ((sc_console != NULL) ? sc_console->sc->unit : -1); else if ((vty < 0) || (vty >= MAXCONS*sc_max_unit())) return -1; else return vty/MAXCONS; } static int sctty_open(struct tty *tp) { int unit = scdevtounit(tp); sc_softc_t *sc; scr_stat *scp; #ifndef __sparc64__ keyarg_t key; #endif DPRINTF(5, ("scopen: dev:%s, unit:%d, vty:%d\n", devtoname(tp->t_dev), unit, SC_VTY(tp))); sc = sc_get_softc(unit, (sc_console_unit == unit) ? SC_KERNEL_CONSOLE : 0); if (sc == NULL) return ENXIO; if (!tty_opened(tp)) { /* Use the current setting of the <-- key as default VERASE. */ /* If the Delete key is preferable, an stty is necessary */ #ifndef __sparc64__ if (sc->kbd != NULL) { key.keynum = KEYCODE_BS; (void)kbdd_ioctl(sc->kbd, GIO_KEYMAPENT, (caddr_t)&key); tp->t_termios.c_cc[VERASE] = key.key.map[0]; } #endif } scp = sc_get_stat(tp); if (scp == NULL) { scp = SC_STAT(tp) = alloc_scp(sc, SC_VTY(tp)); if (ISGRAPHSC(scp)) sc_set_pixel_mode(scp, NULL, 0, 0, 16, 8); } if (!tp->t_winsize.ws_col && !tp->t_winsize.ws_row) { tp->t_winsize.ws_col = scp->xsize; tp->t_winsize.ws_row = scp->ysize; } return (0); } static void sctty_close(struct tty *tp) { scr_stat *scp; int s; if (SC_VTY(tp) != SC_CONSOLECTL) { scp = sc_get_stat(tp); /* were we in the middle of the VT switching process? */ DPRINTF(5, ("sc%d: scclose(), ", scp->sc->unit)); s = spltty(); if ((scp == scp->sc->cur_scp) && (scp->sc->unit == sc_console_unit)) cnavailable(sc_consptr, TRUE); if (finish_vt_rel(scp, TRUE, &s) == 0) /* force release */ DPRINTF(5, ("reset WAIT_REL, ")); if (finish_vt_acq(scp) == 0) /* force acknowledge */ DPRINTF(5, ("reset WAIT_ACQ, ")); #ifdef not_yet_done if (scp == &main_console) { scp->pid = 0; scp->proc = NULL; scp->smode.mode = VT_AUTO; } else { sc_vtb_destroy(&scp->vtb); #ifndef __sparc64__ sc_vtb_destroy(&scp->scr); #endif sc_free_history_buffer(scp, scp->ysize); SC_STAT(tp) = NULL; free(scp, M_DEVBUF); } #else scp->pid = 0; scp->proc = NULL; scp->smode.mode = VT_AUTO; #endif scp->kbd_mode = K_XLATE; if (scp == scp->sc->cur_scp) (void)kbdd_ioctl(scp->sc->kbd, KDSKBMODE, (caddr_t)&scp->kbd_mode); DPRINTF(5, ("done.\n")); } } #if 0 /* XXX mpsafetty: fix screensaver. What about outwakeup? */ static int scread(struct cdev *dev, struct uio *uio, int flag) { if (!sc_saver_keyb_only) sc_touch_scrn_saver(); return ttyread(dev, uio, flag); } #endif static int sckbdevent(keyboard_t *thiskbd, int event, void *arg) { sc_softc_t *sc; struct tty *cur_tty; int c, error = 0; size_t len; const u_char *cp; sc = (sc_softc_t *)arg; /* assert(thiskbd == sc->kbd) */ mtx_lock(&Giant); switch (event) { case KBDIO_KEYINPUT: break; case KBDIO_UNLOADING: sc->kbd = NULL; sc->keyboard = -1; kbd_release(thiskbd, (void *)&sc->keyboard); goto done; default: error = EINVAL; goto done; } /* * Loop while there is still input to get from the keyboard. * I don't think this is nessesary, and it doesn't fix * the Xaccel-2.1 keyboard hang, but it can't hurt. XXX */ while ((c = scgetc(sc, SCGETC_NONBLOCK, NULL)) != NOKEY) { cur_tty = SC_DEV(sc, sc->cur_scp->index); if (!tty_opened_ns(cur_tty)) continue; if ((*sc->cur_scp->tsw->te_input)(sc->cur_scp, c, cur_tty)) continue; switch (KEYFLAGS(c)) { case 0x0000: /* normal key */ ttydisc_rint(cur_tty, KEYCHAR(c), 0); break; case FKEY: /* function key, return string */ cp = (*sc->cur_scp->tsw->te_fkeystr)(sc->cur_scp, c); if (cp != NULL) { ttydisc_rint_simple(cur_tty, cp, strlen(cp)); break; } cp = kbdd_get_fkeystr(thiskbd, KEYCHAR(c), &len); if (cp != NULL) ttydisc_rint_simple(cur_tty, cp, len); break; case MKEY: /* meta is active, prepend ESC */ ttydisc_rint(cur_tty, 0x1b, 0); ttydisc_rint(cur_tty, KEYCHAR(c), 0); break; case BKEY: /* backtab fixed sequence (esc [ Z) */ ttydisc_rint_simple(cur_tty, "\x1B[Z", 3); break; } ttydisc_rint_done(cur_tty); } sc->cur_scp->status |= MOUSE_HIDDEN; done: mtx_unlock(&Giant); return (error); } static int sctty_ioctl(struct tty *tp, u_long cmd, caddr_t data, struct thread *td) { int error; int i; sc_softc_t *sc; scr_stat *scp; int s; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) int ival; #endif /* If there is a user_ioctl function call that first */ if (sc_user_ioctl) { error = (*sc_user_ioctl)(tp, cmd, data, td); if (error != ENOIOCTL) return error; } error = sc_vid_ioctl(tp, cmd, data, td); if (error != ENOIOCTL) return error; #ifndef SC_NO_HISTORY error = sc_hist_ioctl(tp, cmd, data, td); if (error != ENOIOCTL) return error; #endif #ifndef SC_NO_SYSMOUSE error = sc_mouse_ioctl(tp, cmd, data, td); if (error != ENOIOCTL) return error; #endif scp = sc_get_stat(tp); /* assert(scp != NULL) */ /* scp is sc_console, if SC_VTY(dev) == SC_CONSOLECTL. */ sc = scp->sc; if (scp->tsw) { error = (*scp->tsw->te_ioctl)(scp, tp, cmd, data, td); if (error != ENOIOCTL) return error; } switch (cmd) { /* process console hardware related ioctl's */ case GIO_ATTR: /* get current attributes */ /* this ioctl is not processed here, but in the terminal emulator */ return ENOTTY; case GIO_COLOR: /* is this a color console ? */ *(int *)data = (sc->adp->va_flags & V_ADP_COLOR) ? 1 : 0; return 0; case CONS_BLANKTIME: /* set screen saver timeout (0 = no saver) */ if (*(int *)data < 0 || *(int *)data > MAX_BLANKTIME) return EINVAL; s = spltty(); scrn_blank_time = *(int *)data; run_scrn_saver = (scrn_blank_time != 0); splx(s); return 0; case CONS_CURSORTYPE: /* set cursor type (obsolete) */ s = spltty(); *(int *)data &= CONS_CURSOR_ATTRS; sc_change_cursor_shape(scp, *(int *)data, -1, -1); splx(s); return 0; case CONS_GETCURSORSHAPE: /* get cursor shape (new interface) */ if (((int *)data)[0] & CONS_LOCAL_CURSOR) { ((int *)data)[0] = scp->curr_curs_attr.flags; ((int *)data)[1] = scp->curr_curs_attr.base; ((int *)data)[2] = scp->curr_curs_attr.height; } else { ((int *)data)[0] = sc->curs_attr.flags; ((int *)data)[1] = sc->curs_attr.base; ((int *)data)[2] = sc->curs_attr.height; } return 0; case CONS_SETCURSORSHAPE: /* set cursor shape (new interface) */ s = spltty(); sc_change_cursor_shape(scp, ((int *)data)[0], ((int *)data)[1], ((int *)data)[2]); splx(s); return 0; case CONS_BELLTYPE: /* set bell type sound/visual */ if ((*(int *)data) & CONS_VISUAL_BELL) sc->flags |= SC_VISUAL_BELL; else sc->flags &= ~SC_VISUAL_BELL; if ((*(int *)data) & CONS_QUIET_BELL) sc->flags |= SC_QUIET_BELL; else sc->flags &= ~SC_QUIET_BELL; return 0; case CONS_GETINFO: /* get current (virtual) console info */ { vid_info_t *ptr = (vid_info_t*)data; if (ptr->size == sizeof(struct vid_info)) { ptr->m_num = sc->cur_scp->index; ptr->font_size = scp->font_size; ptr->mv_col = scp->xpos; ptr->mv_row = scp->ypos; ptr->mv_csz = scp->xsize; ptr->mv_rsz = scp->ysize; ptr->mv_hsz = (scp->history != NULL) ? scp->history->vtb_rows : 0; /* * The following fields are filled by the terminal emulator. XXX * * ptr->mv_norm.fore * ptr->mv_norm.back * ptr->mv_rev.fore * ptr->mv_rev.back */ ptr->mv_grfc.fore = 0; /* not supported */ ptr->mv_grfc.back = 0; /* not supported */ ptr->mv_ovscan = scp->border; if (scp == sc->cur_scp) save_kbd_state(scp); ptr->mk_keylock = scp->status & LOCK_MASK; return 0; } return EINVAL; } case CONS_GETVERS: /* get version number */ *(int*)data = 0x200; /* version 2.0 */ return 0; case CONS_IDLE: /* see if the screen has been idle */ /* * When the screen is in the GRAPHICS_MODE or UNKNOWN_MODE, * the user process may have been writing something on the * screen and syscons is not aware of it. Declare the screen * is NOT idle if it is in one of these modes. But there is * an exception to it; if a screen saver is running in the * graphics mode in the current screen, we should say that the * screen has been idle. */ *(int *)data = (sc->flags & SC_SCRN_IDLE) && (!ISGRAPHSC(sc->cur_scp) || (sc->cur_scp->status & SAVER_RUNNING)); return 0; case CONS_SAVERMODE: /* set saver mode */ switch(*(int *)data) { case CONS_NO_SAVER: case CONS_USR_SAVER: /* if a LKM screen saver is running, stop it first. */ scsplash_stick(FALSE); saver_mode = *(int *)data; s = spltty(); #ifdef DEV_SPLASH if ((error = wait_scrn_saver_stop(NULL))) { splx(s); return error; } #endif run_scrn_saver = TRUE; if (saver_mode == CONS_USR_SAVER) scp->status |= SAVER_RUNNING; else scp->status &= ~SAVER_RUNNING; scsplash_stick(TRUE); splx(s); break; case CONS_LKM_SAVER: s = spltty(); if ((saver_mode == CONS_USR_SAVER) && (scp->status & SAVER_RUNNING)) scp->status &= ~SAVER_RUNNING; saver_mode = *(int *)data; splx(s); break; default: return EINVAL; } return 0; case CONS_SAVERSTART: /* immediately start/stop the screen saver */ /* * Note that this ioctl does not guarantee the screen saver * actually starts or stops. It merely attempts to do so... */ s = spltty(); run_scrn_saver = (*(int *)data != 0); if (run_scrn_saver) sc->scrn_time_stamp -= scrn_blank_time; splx(s); return 0; case CONS_SCRSHOT: /* get a screen shot */ { int retval, hist_rsz; size_t lsize, csize; vm_offset_t frbp, hstp; unsigned lnum; scrshot_t *ptr = (scrshot_t *)data; void *outp = ptr->buf; if (ptr->x < 0 || ptr->y < 0 || ptr->xsize < 0 || ptr->ysize < 0) return EINVAL; s = spltty(); if (ISGRAPHSC(scp)) { splx(s); return EOPNOTSUPP; } hist_rsz = (scp->history != NULL) ? scp->history->vtb_rows : 0; if (((u_int)ptr->x + ptr->xsize) > scp->xsize || ((u_int)ptr->y + ptr->ysize) > (scp->ysize + hist_rsz)) { splx(s); return EINVAL; } lsize = scp->xsize * sizeof(u_int16_t); csize = ptr->xsize * sizeof(u_int16_t); /* Pointer to the last line of framebuffer */ frbp = scp->vtb.vtb_buffer + scp->ysize * lsize + ptr->x * sizeof(u_int16_t); /* Pointer to the last line of target buffer */ outp = (char *)outp + ptr->ysize * csize; /* Pointer to the last line of history buffer */ if (scp->history != NULL) hstp = scp->history->vtb_buffer + sc_vtb_tail(scp->history) * sizeof(u_int16_t) + ptr->x * sizeof(u_int16_t); else hstp = 0; retval = 0; for (lnum = 0; lnum < (ptr->y + ptr->ysize); lnum++) { if (lnum < scp->ysize) { frbp -= lsize; } else { hstp -= lsize; if (hstp < scp->history->vtb_buffer) hstp += scp->history->vtb_rows * lsize; frbp = hstp; } if (lnum < ptr->y) continue; outp = (char *)outp - csize; retval = copyout((void *)frbp, outp, csize); if (retval != 0) break; } splx(s); return retval; } case VT_SETMODE: /* set screen switcher mode */ { struct vt_mode *mode; struct proc *p1; mode = (struct vt_mode *)data; DPRINTF(5, ("%s%d: VT_SETMODE ", SC_DRIVER_NAME, sc->unit)); if (scp->smode.mode == VT_PROCESS) { p1 = pfind(scp->pid); if (scp->proc == p1 && scp->proc != td->td_proc) { if (p1) PROC_UNLOCK(p1); DPRINTF(5, ("error EPERM\n")); return EPERM; } if (p1) PROC_UNLOCK(p1); } s = spltty(); if (mode->mode == VT_AUTO) { scp->smode.mode = VT_AUTO; scp->proc = NULL; scp->pid = 0; DPRINTF(5, ("VT_AUTO, ")); if ((scp == sc->cur_scp) && (sc->unit == sc_console_unit)) cnavailable(sc_consptr, TRUE); /* were we in the middle of the vty switching process? */ if (finish_vt_rel(scp, TRUE, &s) == 0) DPRINTF(5, ("reset WAIT_REL, ")); if (finish_vt_acq(scp) == 0) DPRINTF(5, ("reset WAIT_ACQ, ")); } else { if (!ISSIGVALID(mode->relsig) || !ISSIGVALID(mode->acqsig) || !ISSIGVALID(mode->frsig)) { splx(s); DPRINTF(5, ("error EINVAL\n")); return EINVAL; } DPRINTF(5, ("VT_PROCESS %d, ", td->td_proc->p_pid)); bcopy(data, &scp->smode, sizeof(struct vt_mode)); scp->proc = td->td_proc; scp->pid = scp->proc->p_pid; if ((scp == sc->cur_scp) && (sc->unit == sc_console_unit)) cnavailable(sc_consptr, FALSE); } splx(s); DPRINTF(5, ("\n")); return 0; } case VT_GETMODE: /* get screen switcher mode */ bcopy(&scp->smode, data, sizeof(struct vt_mode)); return 0; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('v', 4): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case VT_RELDISP: /* screen switcher ioctl */ s = spltty(); /* * This must be the current vty which is in the VT_PROCESS * switching mode... */ if ((scp != sc->cur_scp) || (scp->smode.mode != VT_PROCESS)) { splx(s); return EINVAL; } /* ...and this process is controlling it. */ if (scp->proc != td->td_proc) { splx(s); return EPERM; } error = EINVAL; switch(*(int *)data) { case VT_FALSE: /* user refuses to release screen, abort */ if ((error = finish_vt_rel(scp, FALSE, &s)) == 0) DPRINTF(5, ("%s%d: VT_FALSE\n", SC_DRIVER_NAME, sc->unit)); break; case VT_TRUE: /* user has released screen, go on */ if ((error = finish_vt_rel(scp, TRUE, &s)) == 0) DPRINTF(5, ("%s%d: VT_TRUE\n", SC_DRIVER_NAME, sc->unit)); break; case VT_ACKACQ: /* acquire acknowledged, switch completed */ if ((error = finish_vt_acq(scp)) == 0) DPRINTF(5, ("%s%d: VT_ACKACQ\n", SC_DRIVER_NAME, sc->unit)); break; default: break; } splx(s); return error; case VT_OPENQRY: /* return free virtual console */ for (i = sc->first_vty; i < sc->first_vty + sc->vtys; i++) { tp = SC_DEV(sc, i); if (!tty_opened_ns(tp)) { *(int *)data = i + 1; return 0; } } return EINVAL; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('v', 5): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case VT_ACTIVATE: /* switch to screen *data */ i = (*(int *)data == 0) ? scp->index : (*(int *)data - 1); s = spltty(); error = sc_clean_up(sc->cur_scp); splx(s); if (error) return error; error = sc_switch_scr(sc, i); return (error); #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('v', 6): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case VT_WAITACTIVE: /* wait for switch to occur */ i = (*(int *)data == 0) ? scp->index : (*(int *)data - 1); if ((i < sc->first_vty) || (i >= sc->first_vty + sc->vtys)) return EINVAL; if (i == sc->cur_scp->index) return 0; error = tsleep(VTY_WCHAN(sc, i), (PZERO + 1) | PCATCH, "waitvt", 0); return error; case VT_GETACTIVE: /* get active vty # */ *(int *)data = sc->cur_scp->index + 1; return 0; case VT_GETINDEX: /* get this vty # */ *(int *)data = scp->index + 1; return 0; case VT_LOCKSWITCH: /* prevent vty switching */ if ((*(int *)data) & 0x01) sc->flags |= SC_SCRN_VTYLOCK; else sc->flags &= ~SC_SCRN_VTYLOCK; return 0; case KDENABIO: /* allow io operations */ error = priv_check(td, PRIV_IO); if (error != 0) return error; error = securelevel_gt(td->td_ucred, 0); if (error != 0) return error; #ifdef __i386__ td->td_frame->tf_eflags |= PSL_IOPL; #elif defined(__amd64__) td->td_frame->tf_rflags |= PSL_IOPL; #endif return 0; case KDDISABIO: /* disallow io operations (default) */ #ifdef __i386__ td->td_frame->tf_eflags &= ~PSL_IOPL; #elif defined(__amd64__) td->td_frame->tf_rflags &= ~PSL_IOPL; #endif return 0; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('K', 20): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDSKBSTATE: /* set keyboard state (locks) */ if (*(int *)data & ~LOCK_MASK) return EINVAL; scp->status &= ~LOCK_MASK; scp->status |= *(int *)data; if (scp == sc->cur_scp) update_kbd_state(scp, scp->status, LOCK_MASK); return 0; case KDGKBSTATE: /* get keyboard state (locks) */ if (scp == sc->cur_scp) save_kbd_state(scp); *(int *)data = scp->status & LOCK_MASK; return 0; case KDGETREPEAT: /* get keyboard repeat & delay rates */ case KDSETREPEAT: /* set keyboard repeat & delay rates (new) */ error = kbdd_ioctl(sc->kbd, cmd, data); if (error == ENOIOCTL) error = ENODEV; return error; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('K', 67): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDSETRAD: /* set keyboard repeat & delay rates (old) */ if (*(int *)data & ~0x7f) return EINVAL; error = kbdd_ioctl(sc->kbd, KDSETRAD, data); if (error == ENOIOCTL) error = ENODEV; return error; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('K', 7): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDSKBMODE: /* set keyboard mode */ switch (*(int *)data) { case K_XLATE: /* switch to XLT ascii mode */ case K_RAW: /* switch to RAW scancode mode */ case K_CODE: /* switch to CODE mode */ scp->kbd_mode = *(int *)data; if (scp == sc->cur_scp) (void)kbdd_ioctl(sc->kbd, KDSKBMODE, data); return 0; default: return EINVAL; } /* NOT REACHED */ case KDGKBMODE: /* get keyboard mode */ *(int *)data = scp->kbd_mode; return 0; case KDGKBINFO: error = kbdd_ioctl(sc->kbd, cmd, data); if (error == ENOIOCTL) error = ENODEV; return error; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('K', 8): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDMKTONE: /* sound the bell */ if (*(int*)data) sc_bell(scp, (*(int*)data)&0xffff, (((*(int*)data)>>16)&0xffff)*hz/1000); else sc_bell(scp, scp->bell_pitch, scp->bell_duration); return 0; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('K', 63): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KIOCSOUND: /* make tone (*data) hz */ if (scp == sc->cur_scp) { if (*(int *)data) return sc_tone(*(int *)data); else return sc_tone(0); } return 0; case KDGKBTYPE: /* get keyboard type */ error = kbdd_ioctl(sc->kbd, cmd, data); if (error == ENOIOCTL) { /* always return something? XXX */ *(int *)data = 0; } return 0; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('K', 66): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case KDSETLED: /* set keyboard LED status */ if (*(int *)data & ~LED_MASK) /* FIXME: LOCK_MASK? */ return EINVAL; scp->status &= ~LED_MASK; scp->status |= *(int *)data; if (scp == sc->cur_scp) update_kbd_leds(scp, scp->status); return 0; case KDGETLED: /* get keyboard LED status */ if (scp == sc->cur_scp) save_kbd_state(scp); *(int *)data = scp->status & LED_MASK; return 0; case KBADDKBD: /* add/remove keyboard to/from mux */ case KBRELKBD: error = kbdd_ioctl(sc->kbd, cmd, data); if (error == ENOIOCTL) error = ENODEV; return error; #if defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD4) || defined(COMPAT_43) case _IO('c', 110): ival = IOCPARM_IVAL(data); data = (caddr_t)&ival; /* FALLTHROUGH */ #endif case CONS_SETKBD: /* set the new keyboard */ { keyboard_t *newkbd; s = spltty(); newkbd = kbd_get_keyboard(*(int *)data); if (newkbd == NULL) { splx(s); return EINVAL; } error = 0; if (sc->kbd != newkbd) { i = kbd_allocate(newkbd->kb_name, newkbd->kb_unit, (void *)&sc->keyboard, sckbdevent, sc); /* i == newkbd->kb_index */ if (i >= 0) { if (sc->kbd != NULL) { save_kbd_state(sc->cur_scp); kbd_release(sc->kbd, (void *)&sc->keyboard); } sc->kbd = kbd_get_keyboard(i); /* sc->kbd == newkbd */ sc->keyboard = i; (void)kbdd_ioctl(sc->kbd, KDSKBMODE, (caddr_t)&sc->cur_scp->kbd_mode); update_kbd_state(sc->cur_scp, sc->cur_scp->status, LOCK_MASK); } else { error = EPERM; /* XXX */ } } splx(s); return error; } case CONS_RELKBD: /* release the current keyboard */ s = spltty(); error = 0; if (sc->kbd != NULL) { save_kbd_state(sc->cur_scp); error = kbd_release(sc->kbd, (void *)&sc->keyboard); if (error == 0) { sc->kbd = NULL; sc->keyboard = -1; } } splx(s); return error; case CONS_GETTERM: /* get the current terminal emulator info */ { sc_term_sw_t *sw; if (((term_info_t *)data)->ti_index == 0) { sw = scp->tsw; } else { sw = sc_term_match_by_number(((term_info_t *)data)->ti_index); } if (sw != NULL) { strncpy(((term_info_t *)data)->ti_name, sw->te_name, sizeof(((term_info_t *)data)->ti_name)); strncpy(((term_info_t *)data)->ti_desc, sw->te_desc, sizeof(((term_info_t *)data)->ti_desc)); ((term_info_t *)data)->ti_flags = 0; return 0; } else { ((term_info_t *)data)->ti_name[0] = '\0'; ((term_info_t *)data)->ti_desc[0] = '\0'; ((term_info_t *)data)->ti_flags = 0; return EINVAL; } } case CONS_SETTERM: /* set the current terminal emulator */ s = spltty(); error = sc_init_emulator(scp, ((term_info_t *)data)->ti_name); /* FIXME: what if scp == sc_console! XXX */ splx(s); return error; case GIO_SCRNMAP: /* get output translation table */ bcopy(&sc->scr_map, data, sizeof(sc->scr_map)); return 0; case PIO_SCRNMAP: /* set output translation table */ bcopy(data, &sc->scr_map, sizeof(sc->scr_map)); for (i=0; iscr_map); i++) { sc->scr_rmap[sc->scr_map[i]] = i; } return 0; case GIO_KEYMAP: /* get keyboard translation table */ case PIO_KEYMAP: /* set keyboard translation table */ case OGIO_KEYMAP: /* get keyboard translation table (compat) */ case OPIO_KEYMAP: /* set keyboard translation table (compat) */ case GIO_DEADKEYMAP: /* get accent key translation table */ case PIO_DEADKEYMAP: /* set accent key translation table */ case GETFKEY: /* get function key string */ case SETFKEY: /* set function key string */ error = kbdd_ioctl(sc->kbd, cmd, data); if (error == ENOIOCTL) error = ENODEV; return error; #ifndef SC_NO_FONT_LOADING case PIO_FONT8x8: /* set 8x8 dot font */ if (!ISFONTAVAIL(sc->adp->va_flags)) return ENXIO; bcopy(data, sc->font_8, 8*256); sc->fonts_loaded |= FONT_8; /* * FONT KLUDGE * Always use the font page #0. XXX * Don't load if the current font size is not 8x8. */ if (ISTEXTSC(sc->cur_scp) && (sc->cur_scp->font_size < 14)) sc_load_font(sc->cur_scp, 0, 8, 8, sc->font_8, 0, 256); return 0; case GIO_FONT8x8: /* get 8x8 dot font */ if (!ISFONTAVAIL(sc->adp->va_flags)) return ENXIO; if (sc->fonts_loaded & FONT_8) { bcopy(sc->font_8, data, 8*256); return 0; } else return ENXIO; case PIO_FONT8x14: /* set 8x14 dot font */ if (!ISFONTAVAIL(sc->adp->va_flags)) return ENXIO; bcopy(data, sc->font_14, 14*256); sc->fonts_loaded |= FONT_14; /* * FONT KLUDGE * Always use the font page #0. XXX * Don't load if the current font size is not 8x14. */ if (ISTEXTSC(sc->cur_scp) && (sc->cur_scp->font_size >= 14) && (sc->cur_scp->font_size < 16)) sc_load_font(sc->cur_scp, 0, 14, 8, sc->font_14, 0, 256); return 0; case GIO_FONT8x14: /* get 8x14 dot font */ if (!ISFONTAVAIL(sc->adp->va_flags)) return ENXIO; if (sc->fonts_loaded & FONT_14) { bcopy(sc->font_14, data, 14*256); return 0; } else return ENXIO; case PIO_FONT8x16: /* set 8x16 dot font */ if (!ISFONTAVAIL(sc->adp->va_flags)) return ENXIO; bcopy(data, sc->font_16, 16*256); sc->fonts_loaded |= FONT_16; /* * FONT KLUDGE * Always use the font page #0. XXX * Don't load if the current font size is not 8x16. */ if (ISTEXTSC(sc->cur_scp) && (sc->cur_scp->font_size >= 16)) sc_load_font(sc->cur_scp, 0, 16, 8, sc->font_16, 0, 256); return 0; case GIO_FONT8x16: /* get 8x16 dot font */ if (!ISFONTAVAIL(sc->adp->va_flags)) return ENXIO; if (sc->fonts_loaded & FONT_16) { bcopy(sc->font_16, data, 16*256); return 0; } else return ENXIO; #endif /* SC_NO_FONT_LOADING */ default: break; } return (ENOIOCTL); } static int consolectl_ioctl(struct cdev *dev, u_long cmd, caddr_t data, int fflag, struct thread *td) { return sctty_ioctl(dev->si_drv1, cmd, data, td); } static int consolectl_close(struct cdev *dev, int flags, int mode, struct thread *td) { #ifndef SC_NO_SYSMOUSE mouse_info_t info; memset(&info, 0, sizeof(info)); info.operation = MOUSE_ACTION; /* * Make sure all buttons are released when moused and other * console daemons exit, so that no buttons are left pressed. */ (void) sctty_ioctl(dev->si_drv1, CONS_MOUSECTL, (caddr_t)&info, td); #endif return (0); } static void sc_cnprobe(struct consdev *cp) { int unit; int flags; if (!vty_enabled(VTY_SC)) { cp->cn_pri = CN_DEAD; return; } cp->cn_pri = sc_get_cons_priority(&unit, &flags); /* a video card is always required */ if (!scvidprobe(unit, flags, TRUE)) cp->cn_pri = CN_DEAD; /* syscons will become console even when there is no keyboard */ sckbdprobe(unit, flags, TRUE); if (cp->cn_pri == CN_DEAD) return; /* initialize required fields */ strcpy(cp->cn_name, "ttyv0"); } static void sc_cninit(struct consdev *cp) { int unit; int flags; sc_get_cons_priority(&unit, &flags); scinit(unit, flags | SC_KERNEL_CONSOLE); sc_console_unit = unit; sc_console = sc_get_stat(sc_get_softc(unit, SC_KERNEL_CONSOLE)->dev[0]); sc_consptr = cp; } static void sc_cnterm(struct consdev *cp) { /* we are not the kernel console any more, release everything */ if (sc_console_unit < 0) return; /* shouldn't happen */ #if 0 /* XXX */ sc_clear_screen(sc_console); sccnupdate(sc_console); #endif scterm(sc_console_unit, SC_KERNEL_CONSOLE); sc_console_unit = -1; sc_console = NULL; } static void sccnclose(sc_softc_t *sc, struct sc_cnstate *sp); +static int sc_cngetc_locked(struct sc_cnstate *sp); +static void sccnkbdlock(sc_softc_t *sc, struct sc_cnstate *sp); +static void sccnkbdunlock(sc_softc_t *sc, struct sc_cnstate *sp); static void sccnopen(sc_softc_t *sc, struct sc_cnstate *sp, int flags); static void sccnscrlock(sc_softc_t *sc, struct sc_cnstate *sp); static void sccnscrunlock(sc_softc_t *sc, struct sc_cnstate *sp); static void +sccnkbdlock(sc_softc_t *sc, struct sc_cnstate *sp) +{ + /* + * Locking method: hope for the best. + * The keyboard is supposed to be Giant locked. We can't handle that + * in general. The kdb_active case here is not safe, and we will + * proceed without the lock in all cases. + */ + sp->kbd_locked = !kdb_active && mtx_trylock(&Giant); +} + +static void +sccnkbdunlock(sc_softc_t *sc, struct sc_cnstate *sp) +{ + if (sp->kbd_locked) + mtx_unlock(&Giant); + sp->kbd_locked = FALSE; +} + +static void sccnscrlock(sc_softc_t *sc, struct sc_cnstate *sp) { SC_VIDEO_LOCK(sc); } static void sccnscrunlock(sc_softc_t *sc, struct sc_cnstate *sp) { SC_VIDEO_UNLOCK(sc); } static void sccnopen(sc_softc_t *sc, struct sc_cnstate *sp, int flags) { int kbd_mode; /* assert(sc_console_unit >= 0) */ sp->kbd_opened = FALSE; sp->scr_opened = FALSE; + sp->kbd_locked = FALSE; /* Opening the keyboard is optional. */ if (!(flags & 1) || sc->kbd == NULL) goto over_keyboard; + sccnkbdlock(sc, sp); + /* * Make sure the keyboard is accessible even when the kbd device * driver is disabled. */ kbdd_enable(sc->kbd); /* Switch the keyboard to console mode (K_XLATE, polled) on all scp's. */ kbd_mode = K_XLATE; (void)kbdd_ioctl(sc->kbd, KDSKBMODE, (caddr_t)&kbd_mode); sc->kbd_open_level++; kbdd_poll(sc->kbd, TRUE); sp->kbd_opened = TRUE; over_keyboard: ; /* The screen is opened iff locking it succeeds. */ sccnscrlock(sc, sp); sp->scr_opened = TRUE; /* The screen switch is optional. */ if (!(flags & 2)) return; /* try to switch to the kernel console screen */ if (!cold && sc->cur_scp->index != sc_console->index && sc->cur_scp->smode.mode == VT_AUTO && sc_console->smode.mode == VT_AUTO) sc_switch_scr(sc, sc_console->index); } static void sccnclose(sc_softc_t *sc, struct sc_cnstate *sp) { sp->scr_opened = FALSE; sccnscrunlock(sc, sp); if (!sp->kbd_opened) return; /* Restore keyboard mode (for the current, possibly-changed scp). */ kbdd_poll(sc->kbd, FALSE); if (--sc->kbd_open_level == 0) (void)kbdd_ioctl(sc->kbd, KDSKBMODE, (caddr_t)&sc->cur_scp->kbd_mode); kbdd_disable(sc->kbd); sp->kbd_opened = FALSE; + sccnkbdunlock(sc, sp); } /* * Grabbing switches the screen and keyboard focus to sc_console and the * keyboard mode to (K_XLATE, polled). Only switching to polled mode is * essential (for preventing the interrupt handler from eating input * between polls). Focus is part of the UI, and the other switches are * work just was well when they are done on every entry and exit. * * Screen switches while grabbed are supported, and to maintain focus for * this ungrabbing and closing only restore the polling state and then * the keyboard mode if on the original screen. */ static void sc_cngrab(struct consdev *cp) { sc_softc_t *sc; int lev; sc = sc_console->sc; lev = atomic_fetchadd_int(&sc->grab_level, 1); if (lev >= 0 && lev < 2) { sccnopen(sc, &sc->grab_state[lev], 1 | 2); sccnscrunlock(sc, &sc->grab_state[lev]); + sccnkbdunlock(sc, &sc->grab_state[lev]); } } static void sc_cnungrab(struct consdev *cp) { sc_softc_t *sc; int lev; sc = sc_console->sc; lev = atomic_load_acq_int(&sc->grab_level) - 1; if (lev >= 0 && lev < 2) { + sccnkbdlock(sc, &sc->grab_state[lev]); sccnscrlock(sc, &sc->grab_state[lev]); sccnclose(sc, &sc->grab_state[lev]); } atomic_add_int(&sc->grab_level, -1); } static void sc_cnputc(struct consdev *cd, int c) { struct sc_cnstate st; u_char buf[1]; scr_stat *scp = sc_console; #ifndef SC_NO_HISTORY #if 0 struct tty *tp; #endif #endif /* !SC_NO_HISTORY */ int s; /* assert(sc_console != NULL) */ sccnopen(scp->sc, &st, 0); #ifndef SC_NO_HISTORY if (scp == scp->sc->cur_scp && scp->status & SLKED) { scp->status &= ~SLKED; update_kbd_state(scp, scp->status, SLKED); if (scp->status & BUFFER_SAVED) { if (!sc_hist_restore(scp)) sc_remove_cutmarking(scp); scp->status &= ~BUFFER_SAVED; scp->status |= CURSOR_ENABLED; sc_draw_cursor_image(scp); } #if 0 /* * XXX: Now that TTY's have their own locks, we cannot process * any data after disabling scroll lock. cnputs already holds a * spinlock. */ tp = SC_DEV(scp->sc, scp->index); /* XXX "tp" can be NULL */ tty_lock(tp); if (tty_opened(tp)) sctty_outwakeup(tp); tty_unlock(tp); #endif } #endif /* !SC_NO_HISTORY */ buf[0] = c; sc_puts(scp, buf, 1, 1); s = spltty(); /* block sckbdevent and scrn_timer */ sccnupdate(scp); splx(s); sccnclose(scp->sc, &st); } static int sc_cngetc(struct consdev *cd) { + struct sc_cnstate st; + int c, s; + + /* assert(sc_console != NULL) */ + sccnopen(sc_console->sc, &st, 1); + s = spltty(); /* block sckbdevent and scrn_timer while we poll */ + if (!st.kbd_opened) { + splx(s); + sccnclose(sc_console->sc, &st); + return -1; /* means no keyboard since we fudged the locking */ + } + c = sc_cngetc_locked(&st); + splx(s); + sccnclose(sc_console->sc, &st); + return c; +} + +static int +sc_cngetc_locked(struct sc_cnstate *sp) +{ static struct fkeytab fkey; static int fkeycp; scr_stat *scp; const u_char *p; - int s = spltty(); /* block sckbdevent and scrn_timer while we poll */ int c; - /* assert(sc_console != NULL) */ - /* * Stop the screen saver and update the screen if necessary. * What if we have been running in the screen saver code... XXX */ sc_touch_scrn_saver(); scp = sc_console->sc->cur_scp; /* XXX */ sccnupdate(scp); - if (fkeycp < fkey.len) { - splx(s); + if (fkeycp < fkey.len) return fkey.str[fkeycp++]; - } - if (scp->sc->kbd == NULL) { - splx(s); - return -1; - } + c = scgetc(scp->sc, SCGETC_CN | SCGETC_NONBLOCK, sp); - c = scgetc(scp->sc, SCGETC_CN | SCGETC_NONBLOCK, NULL); - switch (KEYFLAGS(c)) { case 0: /* normal char */ return KEYCHAR(c); case FKEY: /* function key */ p = (*scp->tsw->te_fkeystr)(scp, c); if (p != NULL) { fkey.len = strlen(p); bcopy(p, fkey.str, fkey.len); fkeycp = 1; return fkey.str[0]; } p = kbdd_get_fkeystr(scp->sc->kbd, KEYCHAR(c), (size_t *)&fkeycp); fkey.len = fkeycp; if ((p != NULL) && (fkey.len > 0)) { bcopy(p, fkey.str, fkey.len); fkeycp = 1; return fkey.str[0]; } return c; /* XXX */ case NOKEY: case ERRKEY: default: return -1; } /* NOT REACHED */ } static void sccnupdate(scr_stat *scp) { /* this is a cut-down version of scrn_timer()... */ if (suspend_in_progress || scp->sc->font_loading_in_progress) return; if (kdb_active || panicstr || shutdown_in_progress) { sc_touch_scrn_saver(); } else if (scp != scp->sc->cur_scp) { return; } if (!run_scrn_saver) scp->sc->flags &= ~SC_SCRN_IDLE; #ifdef DEV_SPLASH if ((saver_mode != CONS_LKM_SAVER) || !(scp->sc->flags & SC_SCRN_IDLE)) if (scp->sc->flags & SC_SCRN_BLANKED) stop_scrn_saver(scp->sc, current_saver); #endif if (scp != scp->sc->cur_scp || scp->sc->blink_in_progress || scp->sc->switch_in_progress) return; /* * FIXME: unlike scrn_timer(), we call scrn_update() from here even * when write_in_progress is non-zero. XXX */ if (!ISGRAPHSC(scp) && !(scp->sc->flags & SC_SCRN_BLANKED)) scrn_update(scp, TRUE); } static void scrn_timer(void *arg) { #ifndef PC98 static time_t kbd_time_stamp = 0; #endif sc_softc_t *sc; scr_stat *scp; int again, rate; again = (arg != NULL); if (arg != NULL) sc = (sc_softc_t *)arg; else if (sc_console != NULL) sc = sc_console->sc; else return; /* find the vty to update */ scp = sc->cur_scp; /* don't do anything when we are performing some I/O operations */ if (suspend_in_progress || sc->font_loading_in_progress) goto done; #ifndef PC98 if ((sc->kbd == NULL) && (sc->config & SC_AUTODETECT_KBD)) { /* try to allocate a keyboard automatically */ if (kbd_time_stamp != time_uptime) { kbd_time_stamp = time_uptime; sc->keyboard = sc_allocate_keyboard(sc, -1); if (sc->keyboard >= 0) { sc->kbd = kbd_get_keyboard(sc->keyboard); (void)kbdd_ioctl(sc->kbd, KDSKBMODE, (caddr_t)&sc->cur_scp->kbd_mode); update_kbd_state(sc->cur_scp, sc->cur_scp->status, LOCK_MASK); } } } #endif /* PC98 */ /* should we stop the screen saver? */ if (kdb_active || panicstr || shutdown_in_progress) sc_touch_scrn_saver(); if (run_scrn_saver) { if (time_uptime > sc->scrn_time_stamp + scrn_blank_time) sc->flags |= SC_SCRN_IDLE; else sc->flags &= ~SC_SCRN_IDLE; } else { sc->scrn_time_stamp = time_uptime; sc->flags &= ~SC_SCRN_IDLE; if (scrn_blank_time > 0) run_scrn_saver = TRUE; } #ifdef DEV_SPLASH if ((saver_mode != CONS_LKM_SAVER) || !(sc->flags & SC_SCRN_IDLE)) if (sc->flags & SC_SCRN_BLANKED) stop_scrn_saver(sc, current_saver); #endif /* should we just return ? */ if (sc->blink_in_progress || sc->switch_in_progress || sc->write_in_progress) goto done; /* Update the screen */ scp = sc->cur_scp; /* cur_scp may have changed... */ if (!ISGRAPHSC(scp) && !(sc->flags & SC_SCRN_BLANKED)) scrn_update(scp, TRUE); #ifdef DEV_SPLASH /* should we activate the screen saver? */ if ((saver_mode == CONS_LKM_SAVER) && (sc->flags & SC_SCRN_IDLE)) if (!ISGRAPHSC(scp) || (sc->flags & SC_SCRN_BLANKED)) (*current_saver)(sc, TRUE); #endif done: if (again) { /* * Use reduced "refresh" rate if we are in graphics and that is not a * graphical screen saver. In such case we just have nothing to do. */ if (ISGRAPHSC(scp) && !(sc->flags & SC_SCRN_BLANKED)) rate = 2; else rate = 30; callout_reset_sbt(&sc->ctimeout, SBT_1S / rate, 0, scrn_timer, sc, C_PREL(1)); } } static int and_region(int *s1, int *e1, int s2, int e2) { if (*e1 < s2 || e2 < *s1) return FALSE; *s1 = imax(*s1, s2); *e1 = imin(*e1, e2); return TRUE; } static void scrn_update(scr_stat *scp, int show_cursor) { int start; int end; int s; int e; /* assert(scp == scp->sc->cur_scp) */ SC_VIDEO_LOCK(scp->sc); #ifndef SC_NO_CUTPASTE /* remove the previous mouse pointer image if necessary */ if (scp->status & MOUSE_VISIBLE) { s = scp->mouse_pos; e = scp->mouse_pos + scp->xsize + 1; if ((scp->status & (MOUSE_MOVED | MOUSE_HIDDEN)) || and_region(&s, &e, scp->start, scp->end) || ((scp->status & CURSOR_ENABLED) && (scp->cursor_pos != scp->cursor_oldpos) && (and_region(&s, &e, scp->cursor_pos, scp->cursor_pos) || and_region(&s, &e, scp->cursor_oldpos, scp->cursor_oldpos)))) { sc_remove_mouse_image(scp); if (scp->end >= scp->xsize*scp->ysize) scp->end = scp->xsize*scp->ysize - 1; } } #endif /* !SC_NO_CUTPASTE */ #if 1 /* debug: XXX */ if (scp->end >= scp->xsize*scp->ysize) { printf("scrn_update(): scp->end %d > size_of_screen!!\n", scp->end); scp->end = scp->xsize*scp->ysize - 1; } if (scp->start < 0) { printf("scrn_update(): scp->start %d < 0\n", scp->start); scp->start = 0; } #endif /* update screen image */ if (scp->start <= scp->end) { if (scp->mouse_cut_end >= 0) { /* there is a marked region for cut & paste */ if (scp->mouse_cut_start <= scp->mouse_cut_end) { start = scp->mouse_cut_start; end = scp->mouse_cut_end; } else { start = scp->mouse_cut_end; end = scp->mouse_cut_start - 1; } s = start; e = end; /* does the cut-mark region overlap with the update region? */ if (and_region(&s, &e, scp->start, scp->end)) { (*scp->rndr->draw)(scp, s, e - s + 1, TRUE); s = 0; e = start - 1; if (and_region(&s, &e, scp->start, scp->end)) (*scp->rndr->draw)(scp, s, e - s + 1, FALSE); s = end + 1; e = scp->xsize*scp->ysize - 1; if (and_region(&s, &e, scp->start, scp->end)) (*scp->rndr->draw)(scp, s, e - s + 1, FALSE); } else { (*scp->rndr->draw)(scp, scp->start, scp->end - scp->start + 1, FALSE); } } else { (*scp->rndr->draw)(scp, scp->start, scp->end - scp->start + 1, FALSE); } } /* we are not to show the cursor and the mouse pointer... */ if (!show_cursor) { scp->end = 0; scp->start = scp->xsize*scp->ysize - 1; SC_VIDEO_UNLOCK(scp->sc); return; } /* update cursor image */ if (scp->status & CURSOR_ENABLED) { s = scp->start; e = scp->end; /* did cursor move since last time ? */ if (scp->cursor_pos != scp->cursor_oldpos) { /* do we need to remove old cursor image ? */ if (!and_region(&s, &e, scp->cursor_oldpos, scp->cursor_oldpos)) sc_remove_cursor_image(scp); sc_draw_cursor_image(scp); } else { if (and_region(&s, &e, scp->cursor_pos, scp->cursor_pos)) /* cursor didn't move, but has been overwritten */ sc_draw_cursor_image(scp); else if (scp->curs_attr.flags & CONS_BLINK_CURSOR) /* if it's a blinking cursor, update it */ (*scp->rndr->blink_cursor)(scp, scp->cursor_pos, sc_inside_cutmark(scp, scp->cursor_pos)); } } #ifndef SC_NO_CUTPASTE /* update "pseudo" mouse pointer image */ if (scp->sc->flags & SC_MOUSE_ENABLED) { if (!(scp->status & (MOUSE_VISIBLE | MOUSE_HIDDEN))) { scp->status &= ~MOUSE_MOVED; sc_draw_mouse_image(scp); } } #endif /* SC_NO_CUTPASTE */ scp->end = 0; scp->start = scp->xsize*scp->ysize - 1; SC_VIDEO_UNLOCK(scp->sc); } #ifdef DEV_SPLASH static int scsplash_callback(int event, void *arg) { sc_softc_t *sc; int error; sc = (sc_softc_t *)arg; switch (event) { case SPLASH_INIT: if (add_scrn_saver(scsplash_saver) == 0) { sc->flags &= ~SC_SAVER_FAILED; run_scrn_saver = TRUE; if (cold && !(boothowto & RB_VERBOSE)) { scsplash_stick(TRUE); (*current_saver)(sc, TRUE); } } return 0; case SPLASH_TERM: if (current_saver == scsplash_saver) { scsplash_stick(FALSE); error = remove_scrn_saver(scsplash_saver); if (error) return error; } return 0; default: return EINVAL; } } static void scsplash_saver(sc_softc_t *sc, int show) { static int busy = FALSE; scr_stat *scp; if (busy) return; busy = TRUE; scp = sc->cur_scp; if (show) { if (!(sc->flags & SC_SAVER_FAILED)) { if (!(sc->flags & SC_SCRN_BLANKED)) set_scrn_saver_mode(scp, -1, NULL, 0); switch (splash(sc->adp, TRUE)) { case 0: /* succeeded */ break; case EAGAIN: /* try later */ restore_scrn_saver_mode(scp, FALSE); sc_touch_scrn_saver(); /* XXX */ break; default: sc->flags |= SC_SAVER_FAILED; scsplash_stick(FALSE); restore_scrn_saver_mode(scp, TRUE); printf("scsplash_saver(): failed to put up the image\n"); break; } } } else if (!sticky_splash) { if ((sc->flags & SC_SCRN_BLANKED) && (splash(sc->adp, FALSE) == 0)) restore_scrn_saver_mode(scp, TRUE); } busy = FALSE; } static int add_scrn_saver(void (*this_saver)(sc_softc_t *, int)) { #if 0 int error; if (current_saver != none_saver) { error = remove_scrn_saver(current_saver); if (error) return error; } #endif if (current_saver != none_saver) return EBUSY; run_scrn_saver = FALSE; saver_mode = CONS_LKM_SAVER; current_saver = this_saver; return 0; } static int remove_scrn_saver(void (*this_saver)(sc_softc_t *, int)) { if (current_saver != this_saver) return EINVAL; #if 0 /* * In order to prevent `current_saver' from being called by * the timeout routine `scrn_timer()' while we manipulate * the saver list, we shall set `current_saver' to `none_saver' * before stopping the current saver, rather than blocking by `splXX()'. */ current_saver = none_saver; if (scrn_blanked) stop_scrn_saver(this_saver); #endif /* unblank all blanked screens */ wait_scrn_saver_stop(NULL); if (scrn_blanked) return EBUSY; current_saver = none_saver; return 0; } static int set_scrn_saver_mode(scr_stat *scp, int mode, u_char *pal, int border) { int s; /* assert(scp == scp->sc->cur_scp) */ s = spltty(); if (!ISGRAPHSC(scp)) sc_remove_cursor_image(scp); scp->splash_save_mode = scp->mode; scp->splash_save_status = scp->status & (GRAPHICS_MODE | PIXEL_MODE); scp->status &= ~(GRAPHICS_MODE | PIXEL_MODE); scp->status |= (UNKNOWN_MODE | SAVER_RUNNING); scp->sc->flags |= SC_SCRN_BLANKED; ++scrn_blanked; splx(s); if (mode < 0) return 0; scp->mode = mode; if (set_mode(scp) == 0) { if (scp->sc->adp->va_info.vi_flags & V_INFO_GRAPHICS) scp->status |= GRAPHICS_MODE; #ifndef SC_NO_PALETTE_LOADING if (pal != NULL) vidd_load_palette(scp->sc->adp, pal); #endif sc_set_border(scp, border); return 0; } else { s = spltty(); scp->mode = scp->splash_save_mode; scp->status &= ~(UNKNOWN_MODE | SAVER_RUNNING); scp->status |= scp->splash_save_status; splx(s); return 1; } } static int restore_scrn_saver_mode(scr_stat *scp, int changemode) { int mode; int status; int s; /* assert(scp == scp->sc->cur_scp) */ s = spltty(); mode = scp->mode; status = scp->status; scp->mode = scp->splash_save_mode; scp->status &= ~(UNKNOWN_MODE | SAVER_RUNNING); scp->status |= scp->splash_save_status; scp->sc->flags &= ~SC_SCRN_BLANKED; if (!changemode) { if (!ISGRAPHSC(scp)) sc_draw_cursor_image(scp); --scrn_blanked; splx(s); return 0; } if (set_mode(scp) == 0) { #ifndef SC_NO_PALETTE_LOADING #ifdef SC_PIXEL_MODE if (scp->sc->adp->va_info.vi_mem_model == V_INFO_MM_DIRECT) vidd_load_palette(scp->sc->adp, scp->sc->palette2); else #endif vidd_load_palette(scp->sc->adp, scp->sc->palette); #endif --scrn_blanked; splx(s); return 0; } else { scp->mode = mode; scp->status = status; splx(s); return 1; } } static void stop_scrn_saver(sc_softc_t *sc, void (*saver)(sc_softc_t *, int)) { (*saver)(sc, FALSE); run_scrn_saver = FALSE; /* the screen saver may have chosen not to stop after all... */ if (sc->flags & SC_SCRN_BLANKED) return; mark_all(sc->cur_scp); if (sc->delayed_next_scr) sc_switch_scr(sc, sc->delayed_next_scr - 1); if (!kdb_active) wakeup(&scrn_blanked); } static int wait_scrn_saver_stop(sc_softc_t *sc) { int error = 0; while (scrn_blanked > 0) { run_scrn_saver = FALSE; if (sc && !(sc->flags & SC_SCRN_BLANKED)) { error = 0; break; } error = tsleep(&scrn_blanked, PZERO | PCATCH, "scrsav", 0); if ((error != 0) && (error != ERESTART)) break; } run_scrn_saver = FALSE; return error; } #endif /* DEV_SPLASH */ void sc_touch_scrn_saver(void) { scsplash_stick(FALSE); run_scrn_saver = FALSE; } int sc_switch_scr(sc_softc_t *sc, u_int next_scr) { scr_stat *cur_scp; struct tty *tp; struct proc *p; int s; DPRINTF(5, ("sc0: sc_switch_scr() %d ", next_scr + 1)); if (sc->cur_scp == NULL) return (0); /* prevent switch if previously requested */ if (sc->flags & SC_SCRN_VTYLOCK) { sc_bell(sc->cur_scp, sc->cur_scp->bell_pitch, sc->cur_scp->bell_duration); return EPERM; } /* delay switch if the screen is blanked or being updated */ if ((sc->flags & SC_SCRN_BLANKED) || sc->write_in_progress || sc->blink_in_progress) { sc->delayed_next_scr = next_scr + 1; sc_touch_scrn_saver(); DPRINTF(5, ("switch delayed\n")); return 0; } sc->delayed_next_scr = 0; s = spltty(); cur_scp = sc->cur_scp; /* we are in the middle of the vty switching process... */ if (sc->switch_in_progress && (cur_scp->smode.mode == VT_PROCESS) && cur_scp->proc) { p = pfind(cur_scp->pid); if (cur_scp->proc != p) { if (p) PROC_UNLOCK(p); /* * The controlling process has died!!. Do some clean up. * NOTE:`cur_scp->proc' and `cur_scp->smode.mode' * are not reset here yet; they will be cleared later. */ DPRINTF(5, ("cur_scp controlling process %d died, ", cur_scp->pid)); if (cur_scp->status & SWITCH_WAIT_REL) { /* * Force the previous switch to finish, but return now * with error. */ DPRINTF(5, ("reset WAIT_REL, ")); finish_vt_rel(cur_scp, TRUE, &s); splx(s); DPRINTF(5, ("finishing previous switch\n")); return EINVAL; } else if (cur_scp->status & SWITCH_WAIT_ACQ) { /* let's assume screen switch has been completed. */ DPRINTF(5, ("reset WAIT_ACQ, ")); finish_vt_acq(cur_scp); } else { /* * We are in between screen release and acquisition, and * reached here via scgetc() or scrn_timer() which has * interrupted exchange_scr(). Don't do anything stupid. */ DPRINTF(5, ("waiting nothing, ")); } } else { if (p) PROC_UNLOCK(p); /* * The controlling process is alive, but not responding... * It is either buggy or it may be just taking time. * The following code is a gross kludge to cope with this * problem for which there is no clean solution. XXX */ if (cur_scp->status & SWITCH_WAIT_REL) { switch (sc->switch_in_progress++) { case 1: break; case 2: DPRINTF(5, ("sending relsig again, ")); signal_vt_rel(cur_scp); break; case 3: break; case 4: default: /* * Act as if the controlling program returned * VT_FALSE. */ DPRINTF(5, ("force reset WAIT_REL, ")); finish_vt_rel(cur_scp, FALSE, &s); splx(s); DPRINTF(5, ("act as if VT_FALSE was seen\n")); return EINVAL; } } else if (cur_scp->status & SWITCH_WAIT_ACQ) { switch (sc->switch_in_progress++) { case 1: break; case 2: DPRINTF(5, ("sending acqsig again, ")); signal_vt_acq(cur_scp); break; case 3: break; case 4: default: /* clear the flag and finish the previous switch */ DPRINTF(5, ("force reset WAIT_ACQ, ")); finish_vt_acq(cur_scp); break; } } } } /* * Return error if an invalid argument is given, or vty switch * is still in progress. */ if ((next_scr < sc->first_vty) || (next_scr >= sc->first_vty + sc->vtys) || sc->switch_in_progress) { splx(s); sc_bell(cur_scp, bios_value.bell_pitch, BELL_DURATION); DPRINTF(5, ("error 1\n")); return EINVAL; } /* * Don't allow switching away from the graphics mode vty * if the switch mode is VT_AUTO, unless the next vty is the same * as the current or the current vty has been closed (but showing). */ tp = SC_DEV(sc, cur_scp->index); if ((cur_scp->index != next_scr) && tty_opened_ns(tp) && (cur_scp->smode.mode == VT_AUTO) && ISGRAPHSC(cur_scp)) { splx(s); sc_bell(cur_scp, bios_value.bell_pitch, BELL_DURATION); DPRINTF(5, ("error, graphics mode\n")); return EINVAL; } /* * Is the wanted vty open? Don't allow switching to a closed vty. * If we are in DDB, don't switch to a vty in the VT_PROCESS mode. * Note that we always allow the user to switch to the kernel * console even if it is closed. */ if ((sc_console == NULL) || (next_scr != sc_console->index)) { tp = SC_DEV(sc, next_scr); if (!tty_opened_ns(tp)) { splx(s); sc_bell(cur_scp, bios_value.bell_pitch, BELL_DURATION); DPRINTF(5, ("error 2, requested vty isn't open!\n")); return EINVAL; } if (kdb_active && SC_STAT(tp)->smode.mode == VT_PROCESS) { splx(s); DPRINTF(5, ("error 3, requested vty is in the VT_PROCESS mode\n")); return EINVAL; } } /* this is the start of vty switching process... */ ++sc->switch_in_progress; sc->old_scp = cur_scp; sc->new_scp = sc_get_stat(SC_DEV(sc, next_scr)); if (sc->new_scp == sc->old_scp) { sc->switch_in_progress = 0; /* * XXX wakeup() locks the scheduler lock which will hang if * the lock is in an in-between state, e.g., when we stop at * a breakpoint at fork_exit. It has always been wrong to call * wakeup() when the debugger is active. In RELENG_4, wakeup() * is supposed to be locked by splhigh(), but the debugger may * be invoked at splhigh(). */ if (!kdb_active) wakeup(VTY_WCHAN(sc,next_scr)); splx(s); DPRINTF(5, ("switch done (new == old)\n")); return 0; } /* has controlling process died? */ vt_proc_alive(sc->old_scp); vt_proc_alive(sc->new_scp); /* wait for the controlling process to release the screen, if necessary */ if (signal_vt_rel(sc->old_scp)) { splx(s); return 0; } /* go set up the new vty screen */ splx(s); exchange_scr(sc); s = spltty(); /* wake up processes waiting for this vty */ if (!kdb_active) wakeup(VTY_WCHAN(sc,next_scr)); /* wait for the controlling process to acknowledge, if necessary */ if (signal_vt_acq(sc->cur_scp)) { splx(s); return 0; } sc->switch_in_progress = 0; if (sc->unit == sc_console_unit) cnavailable(sc_consptr, TRUE); splx(s); DPRINTF(5, ("switch done\n")); return 0; } static int do_switch_scr(sc_softc_t *sc, int s) { vt_proc_alive(sc->new_scp); splx(s); exchange_scr(sc); s = spltty(); /* sc->cur_scp == sc->new_scp */ wakeup(VTY_WCHAN(sc,sc->cur_scp->index)); /* wait for the controlling process to acknowledge, if necessary */ if (!signal_vt_acq(sc->cur_scp)) { sc->switch_in_progress = 0; if (sc->unit == sc_console_unit) cnavailable(sc_consptr, TRUE); } return s; } static int vt_proc_alive(scr_stat *scp) { struct proc *p; if (scp->proc) { if ((p = pfind(scp->pid)) != NULL) PROC_UNLOCK(p); if (scp->proc == p) return TRUE; scp->proc = NULL; scp->smode.mode = VT_AUTO; DPRINTF(5, ("vt controlling process %d died\n", scp->pid)); } return FALSE; } static int signal_vt_rel(scr_stat *scp) { if (scp->smode.mode != VT_PROCESS) return FALSE; scp->status |= SWITCH_WAIT_REL; PROC_LOCK(scp->proc); kern_psignal(scp->proc, scp->smode.relsig); PROC_UNLOCK(scp->proc); DPRINTF(5, ("sending relsig to %d\n", scp->pid)); return TRUE; } static int signal_vt_acq(scr_stat *scp) { if (scp->smode.mode != VT_PROCESS) return FALSE; if (scp->sc->unit == sc_console_unit) cnavailable(sc_consptr, FALSE); scp->status |= SWITCH_WAIT_ACQ; PROC_LOCK(scp->proc); kern_psignal(scp->proc, scp->smode.acqsig); PROC_UNLOCK(scp->proc); DPRINTF(5, ("sending acqsig to %d\n", scp->pid)); return TRUE; } static int finish_vt_rel(scr_stat *scp, int release, int *s) { if (scp == scp->sc->old_scp && scp->status & SWITCH_WAIT_REL) { scp->status &= ~SWITCH_WAIT_REL; if (release) *s = do_switch_scr(scp->sc, *s); else scp->sc->switch_in_progress = 0; return 0; } return EINVAL; } static int finish_vt_acq(scr_stat *scp) { if (scp == scp->sc->new_scp && scp->status & SWITCH_WAIT_ACQ) { scp->status &= ~SWITCH_WAIT_ACQ; scp->sc->switch_in_progress = 0; return 0; } return EINVAL; } static void exchange_scr(sc_softc_t *sc) { scr_stat *scp; /* save the current state of video and keyboard */ sc_move_cursor(sc->old_scp, sc->old_scp->xpos, sc->old_scp->ypos); if (!ISGRAPHSC(sc->old_scp)) sc_remove_cursor_image(sc->old_scp); if (sc->old_scp->kbd_mode == K_XLATE) save_kbd_state(sc->old_scp); /* set up the video for the new screen */ scp = sc->cur_scp = sc->new_scp; #ifdef PC98 if (sc->old_scp->mode != scp->mode || ISUNKNOWNSC(sc->old_scp) || ISUNKNOWNSC(sc->new_scp)) #else if (sc->old_scp->mode != scp->mode || ISUNKNOWNSC(sc->old_scp)) #endif set_mode(scp); #ifndef __sparc64__ else sc_vtb_init(&scp->scr, VTB_FRAMEBUFFER, scp->xsize, scp->ysize, (void *)sc->adp->va_window, FALSE); #endif scp->status |= MOUSE_HIDDEN; sc_move_cursor(scp, scp->xpos, scp->ypos); if (!ISGRAPHSC(scp)) sc_set_cursor_image(scp); #ifndef SC_NO_PALETTE_LOADING if (ISGRAPHSC(sc->old_scp)) { #ifdef SC_PIXEL_MODE if (sc->adp->va_info.vi_mem_model == V_INFO_MM_DIRECT) vidd_load_palette(sc->adp, sc->palette2); else #endif vidd_load_palette(sc->adp, sc->palette); } #endif sc_set_border(scp, scp->border); /* set up the keyboard for the new screen */ if (sc->kbd_open_level == 0 && sc->old_scp->kbd_mode != scp->kbd_mode) (void)kbdd_ioctl(sc->kbd, KDSKBMODE, (caddr_t)&scp->kbd_mode); update_kbd_state(scp, scp->status, LOCK_MASK); mark_all(scp); } static void sc_puts(scr_stat *scp, u_char *buf, int len, int kernel) { #ifdef DEV_SPLASH /* make screensaver happy */ if (!sticky_splash && scp == scp->sc->cur_scp && !sc_saver_keyb_only) run_scrn_saver = FALSE; #endif if (scp->tsw) (*scp->tsw->te_puts)(scp, buf, len, kernel); if (scp->sc->delayed_next_scr) sc_switch_scr(scp->sc, scp->sc->delayed_next_scr - 1); } void sc_draw_cursor_image(scr_stat *scp) { /* assert(scp == scp->sc->cur_scp); */ SC_VIDEO_LOCK(scp->sc); (*scp->rndr->draw_cursor)(scp, scp->cursor_pos, scp->curs_attr.flags & CONS_BLINK_CURSOR, TRUE, sc_inside_cutmark(scp, scp->cursor_pos)); scp->cursor_oldpos = scp->cursor_pos; SC_VIDEO_UNLOCK(scp->sc); } void sc_remove_cursor_image(scr_stat *scp) { /* assert(scp == scp->sc->cur_scp); */ SC_VIDEO_LOCK(scp->sc); (*scp->rndr->draw_cursor)(scp, scp->cursor_oldpos, scp->curs_attr.flags & CONS_BLINK_CURSOR, FALSE, sc_inside_cutmark(scp, scp->cursor_oldpos)); SC_VIDEO_UNLOCK(scp->sc); } static void update_cursor_image(scr_stat *scp) { /* assert(scp == scp->sc->cur_scp); */ sc_remove_cursor_image(scp); sc_set_cursor_image(scp); sc_draw_cursor_image(scp); } void sc_set_cursor_image(scr_stat *scp) { scp->curs_attr.flags = scp->curr_curs_attr.flags; if (scp->curs_attr.flags & CONS_HIDDEN_CURSOR) { /* hidden cursor is internally represented as zero-height underline */ scp->curs_attr.flags = CONS_CHAR_CURSOR; scp->curs_attr.base = scp->curs_attr.height = 0; } else if (scp->curs_attr.flags & CONS_CHAR_CURSOR) { scp->curs_attr.base = imin(scp->curr_curs_attr.base, scp->font_size - 1); scp->curs_attr.height = imin(scp->curr_curs_attr.height, scp->font_size - scp->curs_attr.base); } else { /* block cursor */ scp->curs_attr.base = 0; scp->curs_attr.height = scp->font_size; } /* assert(scp == scp->sc->cur_scp); */ SC_VIDEO_LOCK(scp->sc); (*scp->rndr->set_cursor)(scp, scp->curs_attr.base, scp->curs_attr.height, scp->curs_attr.flags & CONS_BLINK_CURSOR); SC_VIDEO_UNLOCK(scp->sc); } static void change_cursor_shape(scr_stat *scp, int flags, int base, int height) { if ((scp == scp->sc->cur_scp) && !ISGRAPHSC(scp)) sc_remove_cursor_image(scp); if (base >= 0) scp->curr_curs_attr.base = base; if (height >= 0) scp->curr_curs_attr.height = height; if (flags & CONS_RESET_CURSOR) scp->curr_curs_attr = scp->dflt_curs_attr; else scp->curr_curs_attr.flags = flags & CONS_CURSOR_ATTRS; if ((scp == scp->sc->cur_scp) && !ISGRAPHSC(scp)) { sc_set_cursor_image(scp); sc_draw_cursor_image(scp); } } void sc_change_cursor_shape(scr_stat *scp, int flags, int base, int height) { sc_softc_t *sc; struct tty *tp; int s; int i; s = spltty(); if ((flags != -1) && (flags & CONS_LOCAL_CURSOR)) { /* local (per vty) change */ change_cursor_shape(scp, flags, base, height); splx(s); return; } /* global change */ sc = scp->sc; if (base >= 0) sc->curs_attr.base = base; if (height >= 0) sc->curs_attr.height = height; if (flags != -1) { if (flags & CONS_RESET_CURSOR) sc->curs_attr = sc->dflt_curs_attr; else sc->curs_attr.flags = flags & CONS_CURSOR_ATTRS; } for (i = sc->first_vty; i < sc->first_vty + sc->vtys; ++i) { if ((tp = SC_DEV(sc, i)) == NULL) continue; if ((scp = sc_get_stat(tp)) == NULL) continue; scp->dflt_curs_attr = sc->curs_attr; change_cursor_shape(scp, CONS_RESET_CURSOR, -1, -1); } splx(s); } static void scinit(int unit, int flags) { /* * When syscons is being initialized as the kernel console, malloc() * is not yet functional, because various kernel structures has not been * fully initialized yet. Therefore, we need to declare the following * static buffers for the console. This is less than ideal, * but is necessry evil for the time being. XXX */ #ifdef PC98 static u_short sc_buffer[ROW*COL*2];/* XXX */ #else static u_short sc_buffer[ROW*COL]; /* XXX */ #endif #ifndef SC_NO_FONT_LOADING static u_char font_8[256*8]; static u_char font_14[256*14]; static u_char font_16[256*16]; #endif sc_softc_t *sc; scr_stat *scp; video_adapter_t *adp; int col; int row; int i; /* one time initialization */ if (init_done == COLD) sc_get_bios_values(&bios_value); init_done = WARM; /* * Allocate resources. Even if we are being called for the second * time, we must allocate them again, because they might have * disappeared... */ sc = sc_get_softc(unit, flags & SC_KERNEL_CONSOLE); if ((sc->flags & SC_INIT_DONE) == 0) SC_VIDEO_LOCKINIT(sc); adp = NULL; if (sc->adapter >= 0) { vid_release(sc->adp, (void *)&sc->adapter); adp = sc->adp; sc->adp = NULL; } if (sc->keyboard >= 0) { DPRINTF(5, ("sc%d: releasing kbd%d\n", unit, sc->keyboard)); i = kbd_release(sc->kbd, (void *)&sc->keyboard); DPRINTF(5, ("sc%d: kbd_release returned %d\n", unit, i)); if (sc->kbd != NULL) { DPRINTF(5, ("sc%d: kbd != NULL!, index:%d, unit:%d, flags:0x%x\n", unit, sc->kbd->kb_index, sc->kbd->kb_unit, sc->kbd->kb_flags)); } sc->kbd = NULL; } sc->adapter = vid_allocate("*", unit, (void *)&sc->adapter); sc->adp = vid_get_adapter(sc->adapter); /* assert((sc->adapter >= 0) && (sc->adp != NULL)) */ sc->keyboard = sc_allocate_keyboard(sc, unit); DPRINTF(1, ("sc%d: keyboard %d\n", unit, sc->keyboard)); sc->kbd = kbd_get_keyboard(sc->keyboard); if (sc->kbd != NULL) { DPRINTF(1, ("sc%d: kbd index:%d, unit:%d, flags:0x%x\n", unit, sc->kbd->kb_index, sc->kbd->kb_unit, sc->kbd->kb_flags)); } if (!(sc->flags & SC_INIT_DONE) || (adp != sc->adp)) { sc->initial_mode = sc->adp->va_initial_mode; #ifndef SC_NO_FONT_LOADING if (flags & SC_KERNEL_CONSOLE) { sc->font_8 = font_8; sc->font_14 = font_14; sc->font_16 = font_16; } else if (sc->font_8 == NULL) { /* assert(sc_malloc) */ sc->font_8 = malloc(sizeof(font_8), M_DEVBUF, M_WAITOK); sc->font_14 = malloc(sizeof(font_14), M_DEVBUF, M_WAITOK); sc->font_16 = malloc(sizeof(font_16), M_DEVBUF, M_WAITOK); } #endif /* extract the hardware cursor location and hide the cursor for now */ vidd_read_hw_cursor(sc->adp, &col, &row); vidd_set_hw_cursor(sc->adp, -1, -1); /* set up the first console */ sc->first_vty = unit*MAXCONS; sc->vtys = MAXCONS; /* XXX: should be configurable */ if (flags & SC_KERNEL_CONSOLE) { /* * Set up devs structure but don't use it yet, calling make_dev() * might panic kernel. Wait for sc_attach_unit() to actually * create the devices. */ sc->dev = main_devs; scp = &main_console; init_scp(sc, sc->first_vty, scp); sc_vtb_init(&scp->vtb, VTB_MEMORY, scp->xsize, scp->ysize, (void *)sc_buffer, FALSE); /* move cursors to the initial positions */ if (col >= scp->xsize) col = 0; if (row >= scp->ysize) row = scp->ysize - 1; scp->xpos = col; scp->ypos = row; scp->cursor_pos = scp->cursor_oldpos = row*scp->xsize + col; if (sc_init_emulator(scp, SC_DFLT_TERM)) sc_init_emulator(scp, "*"); (*scp->tsw->te_default_attr)(scp, user_default.std_color, user_default.rev_color); } else { /* assert(sc_malloc) */ sc->dev = malloc(sizeof(struct tty *)*sc->vtys, M_DEVBUF, M_WAITOK|M_ZERO); sc->dev[0] = sc_alloc_tty(0, unit * MAXCONS); scp = alloc_scp(sc, sc->first_vty); SC_STAT(sc->dev[0]) = scp; } sc->cur_scp = scp; #ifndef __sparc64__ /* copy screen to temporary buffer */ sc_vtb_init(&scp->scr, VTB_FRAMEBUFFER, scp->xsize, scp->ysize, (void *)scp->sc->adp->va_window, FALSE); if (ISTEXTSC(scp)) sc_vtb_copy(&scp->scr, 0, &scp->vtb, 0, scp->xsize*scp->ysize); #endif if (bios_value.cursor_end < scp->font_size) sc->dflt_curs_attr.base = scp->font_size - bios_value.cursor_end - 1; else sc->dflt_curs_attr.base = 0; i = bios_value.cursor_end - bios_value.cursor_start + 1; sc->dflt_curs_attr.height = imin(i, scp->font_size); sc->dflt_curs_attr.flags = 0; sc->curs_attr = sc->dflt_curs_attr; scp->curr_curs_attr = scp->dflt_curs_attr = sc->curs_attr; #ifndef SC_NO_SYSMOUSE sc_mouse_move(scp, scp->xpixel/2, scp->ypixel/2); #endif if (!ISGRAPHSC(scp)) { sc_set_cursor_image(scp); sc_draw_cursor_image(scp); } /* save font and palette */ #ifndef SC_NO_FONT_LOADING sc->fonts_loaded = 0; if (ISFONTAVAIL(sc->adp->va_flags)) { #ifdef SC_DFLT_FONT bcopy(dflt_font_8, sc->font_8, sizeof(dflt_font_8)); bcopy(dflt_font_14, sc->font_14, sizeof(dflt_font_14)); bcopy(dflt_font_16, sc->font_16, sizeof(dflt_font_16)); sc->fonts_loaded = FONT_16 | FONT_14 | FONT_8; if (scp->font_size < 14) { sc_load_font(scp, 0, 8, 8, sc->font_8, 0, 256); } else if (scp->font_size >= 16) { sc_load_font(scp, 0, 16, 8, sc->font_16, 0, 256); } else { sc_load_font(scp, 0, 14, 8, sc->font_14, 0, 256); } #else /* !SC_DFLT_FONT */ if (scp->font_size < 14) { sc_save_font(scp, 0, 8, 8, sc->font_8, 0, 256); sc->fonts_loaded = FONT_8; } else if (scp->font_size >= 16) { sc_save_font(scp, 0, 16, 8, sc->font_16, 0, 256); sc->fonts_loaded = FONT_16; } else { sc_save_font(scp, 0, 14, 8, sc->font_14, 0, 256); sc->fonts_loaded = FONT_14; } #endif /* SC_DFLT_FONT */ /* FONT KLUDGE: always use the font page #0. XXX */ sc_show_font(scp, 0); } #endif /* !SC_NO_FONT_LOADING */ #ifndef SC_NO_PALETTE_LOADING vidd_save_palette(sc->adp, sc->palette); #ifdef SC_PIXEL_MODE for (i = 0; i < sizeof(sc->palette2); i++) sc->palette2[i] = i / 3; #endif #endif #ifdef DEV_SPLASH if (!(sc->flags & SC_SPLASH_SCRN)) { /* we are ready to put up the splash image! */ splash_init(sc->adp, scsplash_callback, sc); sc->flags |= SC_SPLASH_SCRN; } #endif } /* the rest is not necessary, if we have done it once */ if (sc->flags & SC_INIT_DONE) return; /* initialize mapscrn arrays to a one to one map */ for (i = 0; i < sizeof(sc->scr_map); i++) sc->scr_map[i] = sc->scr_rmap[i] = i; #ifdef PC98 sc->scr_map[0x5c] = (u_char)0xfc; /* for backslash */ #endif sc->flags |= SC_INIT_DONE; } static void scterm(int unit, int flags) { sc_softc_t *sc; scr_stat *scp; sc = sc_get_softc(unit, flags & SC_KERNEL_CONSOLE); if (sc == NULL) return; /* shouldn't happen */ #ifdef DEV_SPLASH /* this console is no longer available for the splash screen */ if (sc->flags & SC_SPLASH_SCRN) { splash_term(sc->adp); sc->flags &= ~SC_SPLASH_SCRN; } #endif #if 0 /* XXX */ /* move the hardware cursor to the upper-left corner */ vidd_set_hw_cursor(sc->adp, 0, 0); #endif /* release the keyboard and the video card */ if (sc->keyboard >= 0) kbd_release(sc->kbd, &sc->keyboard); if (sc->adapter >= 0) vid_release(sc->adp, &sc->adapter); /* stop the terminal emulator, if any */ scp = sc_get_stat(sc->dev[0]); if (scp->tsw) (*scp->tsw->te_term)(scp, &scp->ts); if (scp->ts != NULL) free(scp->ts, M_DEVBUF); mtx_destroy(&sc->video_mtx); /* clear the structure */ if (!(flags & SC_KERNEL_CONSOLE)) { /* XXX: We need delete_dev() for this */ free(sc->dev, M_DEVBUF); #if 0 /* XXX: We need a ttyunregister for this */ free(sc->tty, M_DEVBUF); #endif #ifndef SC_NO_FONT_LOADING free(sc->font_8, M_DEVBUF); free(sc->font_14, M_DEVBUF); free(sc->font_16, M_DEVBUF); #endif /* XXX vtb, history */ } bzero(sc, sizeof(*sc)); sc->keyboard = -1; sc->adapter = -1; } static void scshutdown(__unused void *arg, __unused int howto) { KASSERT(sc_console != NULL, ("sc_console != NULL")); KASSERT(sc_console->sc != NULL, ("sc_console->sc != NULL")); KASSERT(sc_console->sc->cur_scp != NULL, ("sc_console->sc->cur_scp != NULL")); sc_touch_scrn_saver(); if (!cold && sc_console->sc->cur_scp->index != sc_console->index && sc_console->sc->cur_scp->smode.mode == VT_AUTO && sc_console->smode.mode == VT_AUTO) sc_switch_scr(sc_console->sc, sc_console->index); shutdown_in_progress = TRUE; } static void scsuspend(__unused void *arg) { int retry; KASSERT(sc_console != NULL, ("sc_console != NULL")); KASSERT(sc_console->sc != NULL, ("sc_console->sc != NULL")); KASSERT(sc_console->sc->cur_scp != NULL, ("sc_console->sc->cur_scp != NULL")); sc_susp_scr = sc_console->sc->cur_scp->index; if (sc_no_suspend_vtswitch || sc_susp_scr == sc_console->index) { sc_touch_scrn_saver(); sc_susp_scr = -1; return; } for (retry = 0; retry < 10; retry++) { sc_switch_scr(sc_console->sc, sc_console->index); if (!sc_console->sc->switch_in_progress) break; pause("scsuspend", hz); } suspend_in_progress = TRUE; } static void scresume(__unused void *arg) { KASSERT(sc_console != NULL, ("sc_console != NULL")); KASSERT(sc_console->sc != NULL, ("sc_console->sc != NULL")); KASSERT(sc_console->sc->cur_scp != NULL, ("sc_console->sc->cur_scp != NULL")); suspend_in_progress = FALSE; if (sc_susp_scr < 0) { update_font(sc_console->sc->cur_scp); return; } sc_switch_scr(sc_console->sc, sc_susp_scr); } int sc_clean_up(scr_stat *scp) { #ifdef DEV_SPLASH int error; #endif if (scp->sc->flags & SC_SCRN_BLANKED) { sc_touch_scrn_saver(); #ifdef DEV_SPLASH if ((error = wait_scrn_saver_stop(scp->sc))) return error; #endif } scp->status |= MOUSE_HIDDEN; sc_remove_mouse_image(scp); sc_remove_cutmarking(scp); return 0; } void sc_alloc_scr_buffer(scr_stat *scp, int wait, int discard) { sc_vtb_t new; sc_vtb_t old; old = scp->vtb; sc_vtb_init(&new, VTB_MEMORY, scp->xsize, scp->ysize, NULL, wait); if (!discard && (old.vtb_flags & VTB_VALID)) { /* retain the current cursor position and buffer contants */ scp->cursor_oldpos = scp->cursor_pos; /* * This works only if the old buffer has the same size as or larger * than the new one. XXX */ sc_vtb_copy(&old, 0, &new, 0, scp->xsize*scp->ysize); scp->vtb = new; } else { scp->vtb = new; sc_vtb_destroy(&old); } #ifndef SC_NO_SYSMOUSE /* move the mouse cursor at the center of the screen */ sc_mouse_move(scp, scp->xpixel / 2, scp->ypixel / 2); #endif } static scr_stat *alloc_scp(sc_softc_t *sc, int vty) { scr_stat *scp; /* assert(sc_malloc) */ scp = (scr_stat *)malloc(sizeof(scr_stat), M_DEVBUF, M_WAITOK); init_scp(sc, vty, scp); sc_alloc_scr_buffer(scp, TRUE, TRUE); if (sc_init_emulator(scp, SC_DFLT_TERM)) sc_init_emulator(scp, "*"); #ifndef SC_NO_CUTPASTE sc_alloc_cut_buffer(scp, TRUE); #endif #ifndef SC_NO_HISTORY sc_alloc_history_buffer(scp, 0, 0, TRUE); #endif return scp; } static void init_scp(sc_softc_t *sc, int vty, scr_stat *scp) { video_info_t info; bzero(scp, sizeof(*scp)); scp->index = vty; scp->sc = sc; scp->status = 0; scp->mode = sc->initial_mode; vidd_get_info(sc->adp, scp->mode, &info); if (info.vi_flags & V_INFO_GRAPHICS) { scp->status |= GRAPHICS_MODE; scp->xpixel = info.vi_width; scp->ypixel = info.vi_height; scp->xsize = info.vi_width/info.vi_cwidth; scp->ysize = info.vi_height/info.vi_cheight; scp->font_size = 0; scp->font = NULL; } else { scp->xsize = info.vi_width; scp->ysize = info.vi_height; scp->xpixel = scp->xsize*info.vi_cwidth; scp->ypixel = scp->ysize*info.vi_cheight; } scp->font_size = info.vi_cheight; scp->font_width = info.vi_cwidth; #ifndef SC_NO_FONT_LOADING if (info.vi_cheight < 14) scp->font = sc->font_8; else if (info.vi_cheight >= 16) scp->font = sc->font_16; else scp->font = sc->font_14; #else scp->font = NULL; #endif sc_vtb_init(&scp->vtb, VTB_MEMORY, 0, 0, NULL, FALSE); #ifndef __sparc64__ sc_vtb_init(&scp->scr, VTB_FRAMEBUFFER, 0, 0, NULL, FALSE); #endif scp->xoff = scp->yoff = 0; scp->xpos = scp->ypos = 0; scp->start = scp->xsize * scp->ysize - 1; scp->end = 0; scp->tsw = NULL; scp->ts = NULL; scp->rndr = NULL; scp->border = (SC_NORM_ATTR >> 4) & 0x0f; scp->curr_curs_attr = scp->dflt_curs_attr = sc->curs_attr; scp->mouse_cut_start = scp->xsize*scp->ysize; scp->mouse_cut_end = -1; scp->mouse_signal = 0; scp->mouse_pid = 0; scp->mouse_proc = NULL; scp->kbd_mode = K_XLATE; scp->bell_pitch = bios_value.bell_pitch; scp->bell_duration = BELL_DURATION; scp->status |= (bios_value.shift_state & NLKED); scp->status |= CURSOR_ENABLED | MOUSE_HIDDEN; scp->pid = 0; scp->proc = NULL; scp->smode.mode = VT_AUTO; scp->history = NULL; scp->history_pos = 0; scp->history_size = 0; } int sc_init_emulator(scr_stat *scp, char *name) { sc_term_sw_t *sw; sc_rndr_sw_t *rndr; void *p; int error; if (name == NULL) /* if no name is given, use the current emulator */ sw = scp->tsw; else /* ...otherwise find the named emulator */ sw = sc_term_match(name); if (sw == NULL) return EINVAL; rndr = NULL; if (strcmp(sw->te_renderer, "*") != 0) { rndr = sc_render_match(scp, sw->te_renderer, scp->status & (GRAPHICS_MODE | PIXEL_MODE)); } if (rndr == NULL) { rndr = sc_render_match(scp, scp->sc->adp->va_name, scp->status & (GRAPHICS_MODE | PIXEL_MODE)); if (rndr == NULL) return ENODEV; } if (sw == scp->tsw) { error = (*sw->te_init)(scp, &scp->ts, SC_TE_WARM_INIT); scp->rndr = rndr; scp->rndr->init(scp); sc_clear_screen(scp); /* assert(error == 0); */ return error; } if (sc_malloc && (sw->te_size > 0)) p = malloc(sw->te_size, M_DEVBUF, M_NOWAIT); else p = NULL; error = (*sw->te_init)(scp, &p, SC_TE_COLD_INIT); if (error) return error; if (scp->tsw) (*scp->tsw->te_term)(scp, &scp->ts); if (scp->ts != NULL) free(scp->ts, M_DEVBUF); scp->tsw = sw; scp->ts = p; scp->rndr = rndr; scp->rndr->init(scp); /* XXX */ (*sw->te_default_attr)(scp, user_default.std_color, user_default.rev_color); sc_clear_screen(scp); return 0; } /* * scgetc(flags) - get character from keyboard. * If flags & SCGETC_CN, then avoid harmful side effects. * If flags & SCGETC_NONBLOCK, then wait until a key is pressed, else * return NOKEY if there is nothing there. */ static u_int scgetc(sc_softc_t *sc, u_int flags, struct sc_cnstate *sp) { scr_stat *scp; #ifndef SC_NO_HISTORY struct tty *tp; #endif u_int c; int this_scr; int f; int i; if (sc->kbd == NULL) return NOKEY; next_code: #if 1 /* I don't like this, but... XXX */ if (flags & SCGETC_CN) sccnupdate(sc->cur_scp); #endif scp = sc->cur_scp; /* first see if there is something in the keyboard port */ for (;;) { + if (flags & SCGETC_CN) + sccnscrunlock(sc, sp); c = kbdd_read_char(sc->kbd, !(flags & SCGETC_NONBLOCK)); + if (flags & SCGETC_CN) + sccnscrlock(sc, sp); if (c == ERRKEY) { if (!(flags & SCGETC_CN)) sc_bell(scp, bios_value.bell_pitch, BELL_DURATION); } else if (c == NOKEY) return c; else break; } /* make screensaver happy */ if (!(c & RELKEY)) sc_touch_scrn_saver(); if (!(flags & SCGETC_CN)) random_harvest_queue(&c, sizeof(c), 1, RANDOM_KEYBOARD); if (sc->kbd_open_level == 0 && scp->kbd_mode != K_XLATE) return KEYCHAR(c); /* if scroll-lock pressed allow history browsing */ if (!ISGRAPHSC(scp) && scp->history && scp->status & SLKED) { scp->status &= ~CURSOR_ENABLED; sc_remove_cursor_image(scp); #ifndef SC_NO_HISTORY if (!(scp->status & BUFFER_SAVED)) { scp->status |= BUFFER_SAVED; sc_hist_save(scp); } switch (c) { /* FIXME: key codes */ case SPCLKEY | FKEY | F(49): /* home key */ sc_remove_cutmarking(scp); sc_hist_home(scp); goto next_code; case SPCLKEY | FKEY | F(57): /* end key */ sc_remove_cutmarking(scp); sc_hist_end(scp); goto next_code; case SPCLKEY | FKEY | F(50): /* up arrow key */ sc_remove_cutmarking(scp); if (sc_hist_up_line(scp)) if (!(flags & SCGETC_CN)) sc_bell(scp, bios_value.bell_pitch, BELL_DURATION); goto next_code; case SPCLKEY | FKEY | F(58): /* down arrow key */ sc_remove_cutmarking(scp); if (sc_hist_down_line(scp)) if (!(flags & SCGETC_CN)) sc_bell(scp, bios_value.bell_pitch, BELL_DURATION); goto next_code; case SPCLKEY | FKEY | F(51): /* page up key */ sc_remove_cutmarking(scp); for (i=0; iysize; i++) if (sc_hist_up_line(scp)) { if (!(flags & SCGETC_CN)) sc_bell(scp, bios_value.bell_pitch, BELL_DURATION); break; } goto next_code; case SPCLKEY | FKEY | F(59): /* page down key */ sc_remove_cutmarking(scp); for (i=0; iysize; i++) if (sc_hist_down_line(scp)) { if (!(flags & SCGETC_CN)) sc_bell(scp, bios_value.bell_pitch, BELL_DURATION); break; } goto next_code; } #endif /* SC_NO_HISTORY */ } /* * Process and consume special keys here. Return a plain char code * or a char code with the META flag or a function key code. */ if (c & RELKEY) { /* key released */ /* goto next_code */ } else { /* key pressed */ if (c & SPCLKEY) { c &= ~SPCLKEY; switch (KEYCHAR(c)) { /* LOCKING KEYS */ case NLK: case CLK: case ALK: break; case SLK: (void)kbdd_ioctl(sc->kbd, KDGKBSTATE, (caddr_t)&f); if (f & SLKED) { scp->status |= SLKED; } else { if (scp->status & SLKED) { scp->status &= ~SLKED; #ifndef SC_NO_HISTORY if (scp->status & BUFFER_SAVED) { if (!sc_hist_restore(scp)) sc_remove_cutmarking(scp); scp->status &= ~BUFFER_SAVED; scp->status |= CURSOR_ENABLED; sc_draw_cursor_image(scp); } /* Only safe in Giant-locked context. */ tp = SC_DEV(sc, scp->index); if (!(flags & SCGETC_CN) && tty_opened_ns(tp)) sctty_outwakeup(tp); #endif } } break; case PASTE: #ifndef SC_NO_CUTPASTE sc_mouse_paste(scp); #endif break; /* NON-LOCKING KEYS */ case NOP: case LSH: case RSH: case LCTR: case RCTR: case LALT: case RALT: case ASH: case META: break; case BTAB: if (!(sc->flags & SC_SCRN_BLANKED)) return c; break; case SPSC: #ifdef DEV_SPLASH /* force activatation/deactivation of the screen saver */ if (!(sc->flags & SC_SCRN_BLANKED)) { run_scrn_saver = TRUE; sc->scrn_time_stamp -= scrn_blank_time; } if (cold) { /* * While devices are being probed, the screen saver need * to be invoked explicitly. XXX */ if (sc->flags & SC_SCRN_BLANKED) { scsplash_stick(FALSE); stop_scrn_saver(sc, current_saver); } else { if (!ISGRAPHSC(scp)) { scsplash_stick(TRUE); (*current_saver)(sc, TRUE); } } } #endif /* DEV_SPLASH */ break; case RBT: #ifndef SC_DISABLE_REBOOT if (enable_reboot && !(flags & SCGETC_CN)) shutdown_nice(0); #endif break; case HALT: #ifndef SC_DISABLE_REBOOT if (enable_reboot && !(flags & SCGETC_CN)) shutdown_nice(RB_HALT); #endif break; case PDWN: #ifndef SC_DISABLE_REBOOT if (enable_reboot && !(flags & SCGETC_CN)) shutdown_nice(RB_HALT|RB_POWEROFF); #endif break; case SUSP: power_pm_suspend(POWER_SLEEP_STATE_SUSPEND); break; case STBY: power_pm_suspend(POWER_SLEEP_STATE_STANDBY); break; case DBG: #ifndef SC_DISABLE_KDBKEY if (enable_kdbkey) kdb_break(); #endif break; case PNC: if (enable_panic_key) panic("Forced by the panic key"); break; case NEXT: this_scr = scp->index; for (i = (this_scr - sc->first_vty + 1)%sc->vtys; sc->first_vty + i != this_scr; i = (i + 1)%sc->vtys) { struct tty *tp = SC_DEV(sc, sc->first_vty + i); if (tty_opened_ns(tp)) { sc_switch_scr(scp->sc, sc->first_vty + i); break; } } break; case PREV: this_scr = scp->index; for (i = (this_scr - sc->first_vty + sc->vtys - 1)%sc->vtys; sc->first_vty + i != this_scr; i = (i + sc->vtys - 1)%sc->vtys) { struct tty *tp = SC_DEV(sc, sc->first_vty + i); if (tty_opened_ns(tp)) { sc_switch_scr(scp->sc, sc->first_vty + i); break; } } break; default: if (KEYCHAR(c) >= F_SCR && KEYCHAR(c) <= L_SCR) { sc_switch_scr(scp->sc, sc->first_vty + KEYCHAR(c) - F_SCR); break; } /* assert(c & FKEY) */ if (!(sc->flags & SC_SCRN_BLANKED)) return c; break; } /* goto next_code */ } else { /* regular keys (maybe MKEY is set) */ #if !defined(SC_DISABLE_KDBKEY) && defined(KDB) if (enable_kdbkey) kdb_alt_break(c, &sc->sc_altbrk); #endif if (!(sc->flags & SC_SCRN_BLANKED)) return c; } } goto next_code; } static int sctty_mmap(struct tty *tp, vm_ooffset_t offset, vm_paddr_t *paddr, int nprot, vm_memattr_t *memattr) { scr_stat *scp; scp = sc_get_stat(tp); if (scp != scp->sc->cur_scp) return -1; return vidd_mmap(scp->sc->adp, offset, paddr, nprot, memattr); } static void update_font(scr_stat *scp) { #ifndef SC_NO_FONT_LOADING /* load appropriate font */ if (!(scp->status & GRAPHICS_MODE)) { if (!(scp->status & PIXEL_MODE) && ISFONTAVAIL(scp->sc->adp->va_flags)) { if (scp->font_size < 14) { if (scp->sc->fonts_loaded & FONT_8) sc_load_font(scp, 0, 8, 8, scp->sc->font_8, 0, 256); } else if (scp->font_size >= 16) { if (scp->sc->fonts_loaded & FONT_16) sc_load_font(scp, 0, 16, 8, scp->sc->font_16, 0, 256); } else { if (scp->sc->fonts_loaded & FONT_14) sc_load_font(scp, 0, 14, 8, scp->sc->font_14, 0, 256); } /* * FONT KLUDGE: * This is an interim kludge to display correct font. * Always use the font page #0 on the video plane 2. * Somehow we cannot show the font in other font pages on * some video cards... XXX */ sc_show_font(scp, 0); } mark_all(scp); } #endif /* !SC_NO_FONT_LOADING */ } static int save_kbd_state(scr_stat *scp) { int state; int error; error = kbdd_ioctl(scp->sc->kbd, KDGKBSTATE, (caddr_t)&state); if (error == ENOIOCTL) error = ENODEV; if (error == 0) { scp->status &= ~LOCK_MASK; scp->status |= state; } return error; } static int update_kbd_state(scr_stat *scp, int new_bits, int mask) { int state; int error; if (mask != LOCK_MASK) { error = kbdd_ioctl(scp->sc->kbd, KDGKBSTATE, (caddr_t)&state); if (error == ENOIOCTL) error = ENODEV; if (error) return error; state &= ~mask; state |= new_bits & mask; } else { state = new_bits & LOCK_MASK; } error = kbdd_ioctl(scp->sc->kbd, KDSKBSTATE, (caddr_t)&state); if (error == ENOIOCTL) error = ENODEV; return error; } static int update_kbd_leds(scr_stat *scp, int which) { int error; which &= LOCK_MASK; error = kbdd_ioctl(scp->sc->kbd, KDSETLED, (caddr_t)&which); if (error == ENOIOCTL) error = ENODEV; return error; } int set_mode(scr_stat *scp) { video_info_t info; /* reject unsupported mode */ if (vidd_get_info(scp->sc->adp, scp->mode, &info)) return 1; /* if this vty is not currently showing, do nothing */ if (scp != scp->sc->cur_scp) return 0; /* setup video hardware for the given mode */ vidd_set_mode(scp->sc->adp, scp->mode); scp->rndr->init(scp); #ifndef __sparc64__ sc_vtb_init(&scp->scr, VTB_FRAMEBUFFER, scp->xsize, scp->ysize, (void *)scp->sc->adp->va_window, FALSE); #endif update_font(scp); sc_set_border(scp, scp->border); sc_set_cursor_image(scp); return 0; } void sc_set_border(scr_stat *scp, int color) { SC_VIDEO_LOCK(scp->sc); (*scp->rndr->draw_border)(scp, color); SC_VIDEO_UNLOCK(scp->sc); } #ifndef SC_NO_FONT_LOADING void sc_load_font(scr_stat *scp, int page, int size, int width, u_char *buf, int base, int count) { sc_softc_t *sc; sc = scp->sc; sc->font_loading_in_progress = TRUE; vidd_load_font(sc->adp, page, size, width, buf, base, count); sc->font_loading_in_progress = FALSE; } void sc_save_font(scr_stat *scp, int page, int size, int width, u_char *buf, int base, int count) { sc_softc_t *sc; sc = scp->sc; sc->font_loading_in_progress = TRUE; vidd_save_font(sc->adp, page, size, width, buf, base, count); sc->font_loading_in_progress = FALSE; } void sc_show_font(scr_stat *scp, int page) { vidd_show_font(scp->sc->adp, page); } #endif /* !SC_NO_FONT_LOADING */ void sc_paste(scr_stat *scp, const u_char *p, int count) { struct tty *tp; u_char *rmap; tp = SC_DEV(scp->sc, scp->sc->cur_scp->index); if (!tty_opened_ns(tp)) return; rmap = scp->sc->scr_rmap; for (; count > 0; --count) ttydisc_rint(tp, rmap[*p++], 0); ttydisc_rint_done(tp); } void sc_respond(scr_stat *scp, const u_char *p, int count, int wakeup) { struct tty *tp; tp = SC_DEV(scp->sc, scp->sc->cur_scp->index); if (!tty_opened_ns(tp)) return; ttydisc_rint_simple(tp, p, count); if (wakeup) { /* XXX: we can't always call ttydisc_rint_done() here! */ ttydisc_rint_done(tp); } } void sc_bell(scr_stat *scp, int pitch, int duration) { if (cold || kdb_active || shutdown_in_progress || !enable_bell) return; if (scp != scp->sc->cur_scp && (scp->sc->flags & SC_QUIET_BELL)) return; if (scp->sc->flags & SC_VISUAL_BELL) { if (scp->sc->blink_in_progress) return; scp->sc->blink_in_progress = 3; if (scp != scp->sc->cur_scp) scp->sc->blink_in_progress += 2; blink_screen(scp->sc->cur_scp); } else if (duration != 0 && pitch != 0) { if (scp != scp->sc->cur_scp) pitch *= 2; sysbeep(1193182 / pitch, duration); } } static void blink_screen(void *arg) { scr_stat *scp = arg; struct tty *tp; if (ISGRAPHSC(scp) || (scp->sc->blink_in_progress <= 1)) { scp->sc->blink_in_progress = 0; mark_all(scp); tp = SC_DEV(scp->sc, scp->index); if (tty_opened_ns(tp)) sctty_outwakeup(tp); if (scp->sc->delayed_next_scr) sc_switch_scr(scp->sc, scp->sc->delayed_next_scr - 1); } else { (*scp->rndr->draw)(scp, 0, scp->xsize*scp->ysize, scp->sc->blink_in_progress & 1); scp->sc->blink_in_progress--; callout_reset_sbt(&scp->sc->cblink, SBT_1S / 15, 0, blink_screen, scp, C_PREL(0)); } } /* * Until sc_attach_unit() gets called no dev structures will be available * to store the per-screen current status. This is the case when the * kernel is initially booting and needs access to its console. During * this early phase of booting the console's current status is kept in * one statically defined scr_stat structure, and any pointers to the * dev structures will be NULL. */ static scr_stat * sc_get_stat(struct tty *tp) { if (tp == NULL) return (&main_console); return (SC_STAT(tp)); } /* * Allocate active keyboard. Try to allocate "kbdmux" keyboard first, and, * if found, add all non-busy keyboards to "kbdmux". Otherwise look for * any keyboard. */ static int sc_allocate_keyboard(sc_softc_t *sc, int unit) { int idx0, idx; keyboard_t *k0, *k; keyboard_info_t ki; idx0 = kbd_allocate("kbdmux", -1, (void *)&sc->keyboard, sckbdevent, sc); if (idx0 != -1) { k0 = kbd_get_keyboard(idx0); for (idx = kbd_find_keyboard2("*", -1, 0); idx != -1; idx = kbd_find_keyboard2("*", -1, idx + 1)) { k = kbd_get_keyboard(idx); if (idx == idx0 || KBD_IS_BUSY(k)) continue; bzero(&ki, sizeof(ki)); strcpy(ki.kb_name, k->kb_name); ki.kb_unit = k->kb_unit; (void)kbdd_ioctl(k0, KBADDKBD, (caddr_t) &ki); } } else idx0 = kbd_allocate("*", unit, (void *)&sc->keyboard, sckbdevent, sc); return (idx0); } Index: projects/netbsd-tests-update-12/sys/dev/syscons/syscons.h =================================================================== --- projects/netbsd-tests-update-12/sys/dev/syscons/syscons.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/syscons/syscons.h (revision 305172) @@ -1,694 +1,695 @@ /*- * Copyright (c) 1995-1998 Søren Schmidt * All rights reserved. * * This code is derived from software contributed to The DragonFly Project * by Sascha Wildner * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification, immediately at the beginning of the file. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. The name of the author may not be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _DEV_SYSCONS_SYSCONS_H_ #define _DEV_SYSCONS_SYSCONS_H_ #include /* XXX */ #include #include /* machine-dependent part of the header */ #ifdef PC98 #include #elif defined(__i386__) /* nothing for the moment */ #endif /* default values for configuration options */ #ifndef MAXCONS #define MAXCONS 16 #endif #ifdef SC_NO_SYSMOUSE #undef SC_NO_CUTPASTE #define SC_NO_CUTPASTE 1 #endif #ifdef SC_NO_MODE_CHANGE #undef SC_PIXEL_MODE #endif /* Always load font data if the pixel (raster text) mode is to be used. */ #ifdef SC_PIXEL_MODE #undef SC_NO_FONT_LOADING #endif /* * If font data is not available, the `arrow'-shaped mouse cursor cannot * be drawn. Use the alternative drawing method. */ #ifdef SC_NO_FONT_LOADING #undef SC_ALT_MOUSE_IMAGE #define SC_ALT_MOUSE_IMAGE 1 #endif #ifndef SC_CURSOR_CHAR #define SC_CURSOR_CHAR (0x07) #endif #ifndef SC_MOUSE_CHAR #define SC_MOUSE_CHAR (0xd0) #endif #if SC_MOUSE_CHAR <= SC_CURSOR_CHAR && SC_CURSOR_CHAR < (SC_MOUSE_CHAR + 4) #undef SC_CURSOR_CHAR #define SC_CURSOR_CHAR (SC_MOUSE_CHAR + 4) #endif #ifndef SC_DEBUG_LEVEL #define SC_DEBUG_LEVEL 0 #endif #define DPRINTF(l, p) if (SC_DEBUG_LEVEL >= (l)) printf p #ifndef __sparc64__ #define SC_DRIVER_NAME "sc" #else /* * Use a different driver name on sparc64 so it does not get confused * with the system controller devices which are also termed 'sc' in OFW. */ #define SC_DRIVER_NAME "syscons" #endif #define SC_VTY(dev) (((sc_ttysoftc *)tty_softc(tp))->st_index) #define SC_DEV(sc, vty) ((sc)->dev[(vty) - (sc)->first_vty]) #define SC_STAT(tp) (*((scr_stat **)&((sc_ttysoftc *)tty_softc(tp))->st_stat)) /* printable chars */ #ifndef PRINTABLE #define PRINTABLE(ch) ((ch) > 0x1b || ((ch) > 0x0d && (ch) < 0x1b) \ || (ch) < 0x07) #endif /* macros for "intelligent" screen update */ #define mark_for_update(scp, x) {\ if ((x) < scp->start) scp->start = (x);\ else if ((x) > scp->end) scp->end = (x);\ } #define mark_all(scp) {\ scp->start = 0;\ scp->end = scp->xsize * scp->ysize - 1;\ } /* vty status flags (scp->status) */ #define UNKNOWN_MODE 0x00010 /* unknown video mode */ #define SWITCH_WAIT_REL 0x00080 /* waiting for vty release */ #define SWITCH_WAIT_ACQ 0x00100 /* waiting for vty ack */ #define BUFFER_SAVED 0x00200 /* vty buffer is saved */ #define CURSOR_ENABLED 0x00400 /* text cursor is enabled */ #define MOUSE_MOVED 0x01000 /* mouse cursor has moved */ #define MOUSE_CUTTING 0x02000 /* mouse cursor is cutting text */ #define MOUSE_VISIBLE 0x04000 /* mouse cursor is showing */ #define GRAPHICS_MODE 0x08000 /* vty is in a graphics mode */ #define PIXEL_MODE 0x10000 /* vty is in a raster text mode */ #define SAVER_RUNNING 0x20000 /* screen saver is running */ #define VR_CURSOR_BLINK 0x40000 /* blinking text cursor */ #define VR_CURSOR_ON 0x80000 /* text cursor is on */ #define MOUSE_HIDDEN 0x100000 /* mouse cursor is temporarily hidden */ /* misc defines */ #define FALSE 0 #define TRUE 1 /* The following #defines are hard-coded for a maximum text resolution corresponding to a maximum framebuffer resolution of 1920x1200 with an 8x8 font... */ #define COL 240 #define ROW 150 #define PCBURST 128 #ifndef BELL_DURATION #define BELL_DURATION ((5 * hz + 99) / 100) #define BELL_PITCH 800 #endif /* virtual terminal buffer */ typedef struct sc_vtb { int vtb_flags; #define VTB_VALID (1 << 0) #define VTB_ALLOCED (1 << 1) int vtb_type; #define VTB_INVALID 0 #define VTB_MEMORY 1 #define VTB_FRAMEBUFFER 2 #define VTB_RINGBUFFER 3 int vtb_cols; int vtb_rows; int vtb_size; vm_offset_t vtb_buffer; int vtb_tail; /* valid for VTB_RINGBUFFER only */ } sc_vtb_t; /* text cursor attributes */ struct cursor_attr { int flags; int base; int height; }; /* softc */ struct keyboard; struct video_adapter; struct scr_stat; struct tty; struct sc_cnstate { + u_char kbd_locked; u_char kbd_opened; u_char scr_opened; }; typedef struct sc_softc { int unit; /* unit # */ int config; /* configuration flags */ #define SC_VESAMODE (1 << 7) #define SC_AUTODETECT_KBD (1 << 8) #define SC_KERNEL_CONSOLE (1 << 9) int flags; /* status flags */ #define SC_VISUAL_BELL (1 << 0) #define SC_QUIET_BELL (1 << 1) #if 0 /* not used anymore */ #define SC_BLINK_CURSOR (1 << 2) #define SC_CHAR_CURSOR (1 << 3) #endif #define SC_MOUSE_ENABLED (1 << 4) #define SC_SCRN_IDLE (1 << 5) #define SC_SCRN_BLANKED (1 << 6) #define SC_SAVER_FAILED (1 << 7) #define SC_SCRN_VTYLOCK (1 << 8) #define SC_INIT_DONE (1 << 16) #define SC_SPLASH_SCRN (1 << 17) int keyboard; /* -1 if unavailable */ struct keyboard *kbd; int adapter; struct video_adapter *adp; int initial_mode; /* initial video mode */ int first_vty; int vtys; struct tty **dev; struct scr_stat *cur_scp; struct scr_stat *new_scp; struct scr_stat *old_scp; int delayed_next_scr; char font_loading_in_progress; char switch_in_progress; char write_in_progress; char blink_in_progress; int grab_level; /* 2 is just enough for kdb to grab for stepping normal grabbing: */ struct sc_cnstate grab_state[2]; int kbd_open_level; struct mtx video_mtx; long scrn_time_stamp; struct cursor_attr dflt_curs_attr; struct cursor_attr curs_attr; u_char scr_map[256]; u_char scr_rmap[256]; #ifdef _SC_MD_SOFTC_DECLARED_ sc_md_softc_t md; /* machine dependent vars */ #endif #ifndef SC_NO_PALETTE_LOADING u_char palette[256 * 3]; #ifdef SC_PIXEL_MODE u_char palette2[256 * 3]; #endif #endif #ifndef SC_NO_FONT_LOADING int fonts_loaded; #define FONT_8 2 #define FONT_14 4 #define FONT_16 8 #define FONT_22 8 u_char *font_8; u_char *font_14; u_char *font_16; u_char *font_22; #endif u_char cursor_char; u_char mouse_char; #ifdef KDB int sc_altbrk; #endif struct callout ctimeout; struct callout cblink; } sc_softc_t; /* virtual screen */ typedef struct scr_stat { int index; /* index of this vty */ struct sc_softc *sc; /* pointer to softc */ struct sc_rndr_sw *rndr; /* renderer */ #ifndef __sparc64__ sc_vtb_t scr; #endif sc_vtb_t vtb; int xpos; /* current X position */ int ypos; /* current Y position */ int xsize; /* X text size */ int ysize; /* Y text size */ int xpixel; /* X graphics size */ int ypixel; /* Y graphics size */ int xoff; /* X offset in pixel mode */ int yoff; /* Y offset in pixel mode */ u_char *font; /* current font */ int font_size; /* fontsize in Y direction */ int font_width; /* fontsize in X direction */ int start; /* modified area start */ int end; /* modified area end */ struct sc_term_sw *tsw; void *ts; int status; /* status (bitfield) */ int kbd_mode; /* keyboard I/O mode */ int cursor_pos; /* cursor buffer position */ int cursor_oldpos; /* cursor old buffer position */ u_short cursor_saveunder_char; /* saved char under cursor */ u_short cursor_saveunder_attr; /* saved attr under cursor */ struct cursor_attr dflt_curs_attr; struct cursor_attr curr_curs_attr; struct cursor_attr curs_attr; int mouse_pos; /* mouse buffer position */ int mouse_oldpos; /* mouse old buffer position */ short mouse_xpos; /* mouse x coordinate */ short mouse_ypos; /* mouse y coordinate */ short mouse_oldxpos; /* mouse previous x coordinate */ short mouse_oldypos; /* mouse previous y coordinate */ short mouse_buttons; /* mouse buttons */ int mouse_cut_start; /* mouse cut start pos */ int mouse_cut_end; /* mouse cut end pos */ int mouse_level; /* xterm mouse protocol */ struct proc *mouse_proc; /* proc* of controlling proc */ pid_t mouse_pid; /* pid of controlling proc */ int mouse_signal; /* signal # to report with */ u_short bell_duration; u_short bell_pitch; u_char border; /* border color */ int mode; /* mode */ pid_t pid; /* pid of controlling proc */ struct proc *proc; /* proc* of controlling proc */ struct vt_mode smode; /* switch mode */ sc_vtb_t *history; /* circular history buffer */ int history_pos; /* position shown on screen */ int history_size; /* size of history buffer */ int splash_save_mode; /* saved mode for splash screen */ int splash_save_status; /* saved status for splash screen */ #ifdef _SCR_MD_STAT_DECLARED_ scr_md_stat_t md; /* machine dependent vars */ #endif } scr_stat; /* TTY softc. */ typedef struct sc_ttysoftc { int st_index; scr_stat *st_stat; } sc_ttysoftc; #ifndef SC_NORM_ATTR #define SC_NORM_ATTR (FG_LIGHTGREY | BG_BLACK) #endif #ifndef SC_NORM_REV_ATTR #define SC_NORM_REV_ATTR (FG_BLACK | BG_LIGHTGREY) #endif #ifndef SC_KERNEL_CONS_ATTR #define SC_KERNEL_CONS_ATTR (FG_WHITE | BG_BLACK) #endif #ifndef SC_KERNEL_CONS_REV_ATTR #define SC_KERNEL_CONS_REV_ATTR (FG_BLACK | BG_LIGHTGREY) #endif /* terminal emulator */ #ifndef SC_DFLT_TERM #define SC_DFLT_TERM "*" /* any */ #endif typedef int sc_term_init_t(scr_stat *scp, void **tcp, int code); #define SC_TE_COLD_INIT 0 #define SC_TE_WARM_INIT 1 typedef int sc_term_term_t(scr_stat *scp, void **tcp); typedef void sc_term_puts_t(scr_stat *scp, u_char *buf, int len, int kernel); typedef int sc_term_ioctl_t(scr_stat *scp, struct tty *tp, u_long cmd, caddr_t data, struct thread *td); typedef int sc_term_reset_t(scr_stat *scp, int code); #define SC_TE_HARD_RESET 0 #define SC_TE_SOFT_RESET 1 typedef void sc_term_default_attr_t(scr_stat *scp, int norm, int rev); typedef void sc_term_clear_t(scr_stat *scp); typedef void sc_term_notify_t(scr_stat *scp, int event); #define SC_TE_NOTIFY_VTSWITCH_IN 0 #define SC_TE_NOTIFY_VTSWITCH_OUT 1 typedef int sc_term_input_t(scr_stat *scp, int c, struct tty *tp); typedef const char *sc_term_fkeystr_t(scr_stat *scp, int c); typedef struct sc_term_sw { LIST_ENTRY(sc_term_sw) link; char *te_name; /* name of the emulator */ char *te_desc; /* description */ char *te_renderer; /* matching renderer */ size_t te_size; /* size of internal buffer */ int te_refcount; /* reference counter */ sc_term_init_t *te_init; sc_term_term_t *te_term; sc_term_puts_t *te_puts; sc_term_ioctl_t *te_ioctl; sc_term_reset_t *te_reset; sc_term_default_attr_t *te_default_attr; sc_term_clear_t *te_clear; sc_term_notify_t *te_notify; sc_term_input_t *te_input; sc_term_fkeystr_t *te_fkeystr; } sc_term_sw_t; #define SCTERM_MODULE(name, sw) \ DATA_SET(scterm_set, sw); \ static int \ scterm_##name##_event(module_t mod, int type, void *data) \ { \ switch (type) { \ case MOD_LOAD: \ return sc_term_add(&sw); \ case MOD_UNLOAD: \ if (sw.te_refcount > 0) \ return EBUSY; \ return sc_term_remove(&sw); \ default: \ return EOPNOTSUPP; \ break; \ } \ return 0; \ } \ static moduledata_t scterm_##name##_mod = { \ "scterm-" #name, \ scterm_##name##_event, \ NULL, \ }; \ DECLARE_MODULE(scterm_##name, scterm_##name##_mod, \ SI_SUB_DRIVERS, SI_ORDER_MIDDLE) /* renderer function table */ typedef void vr_init_t(scr_stat *scp); typedef void vr_clear_t(scr_stat *scp, int c, int attr); typedef void vr_draw_border_t(scr_stat *scp, int color); typedef void vr_draw_t(scr_stat *scp, int from, int count, int flip); typedef void vr_set_cursor_t(scr_stat *scp, int base, int height, int blink); typedef void vr_draw_cursor_t(scr_stat *scp, int at, int blink, int on, int flip); typedef void vr_blink_cursor_t(scr_stat *scp, int at, int flip); typedef void vr_set_mouse_t(scr_stat *scp); typedef void vr_draw_mouse_t(scr_stat *scp, int x, int y, int on); typedef struct sc_rndr_sw { vr_init_t *init; vr_clear_t *clear; vr_draw_border_t *draw_border; vr_draw_t *draw; vr_set_cursor_t *set_cursor; vr_draw_cursor_t *draw_cursor; vr_blink_cursor_t *blink_cursor; vr_set_mouse_t *set_mouse; vr_draw_mouse_t *draw_mouse; } sc_rndr_sw_t; typedef struct sc_renderer { char *name; int mode; sc_rndr_sw_t *rndrsw; LIST_ENTRY(sc_renderer) link; } sc_renderer_t; #define RENDERER(name, mode, sw, set) \ static struct sc_renderer scrndr_##name##_##mode = { \ #name, mode, &sw \ }; \ DATA_SET(scrndr_set, scrndr_##name##_##mode); \ DATA_SET(set, scrndr_##name##_##mode) #define RENDERER_MODULE(name, set) \ SET_DECLARE(set, sc_renderer_t); \ static int \ scrndr_##name##_event(module_t mod, int type, void *data) \ { \ sc_renderer_t **list; \ int error = 0; \ switch (type) { \ case MOD_LOAD: \ SET_FOREACH(list, set) { \ error = sc_render_add(*list); \ if (error) \ break; \ } \ break; \ case MOD_UNLOAD: \ SET_FOREACH(list, set) { \ error = sc_render_remove(*list);\ if (error) \ break; \ } \ break; \ default: \ return EOPNOTSUPP; \ break; \ } \ return error; \ } \ static moduledata_t scrndr_##name##_mod = { \ "scrndr-" #name, \ scrndr_##name##_event, \ NULL, \ }; \ DECLARE_MODULE(scrndr_##name, scrndr_##name##_mod, \ SI_SUB_DRIVERS, SI_ORDER_MIDDLE) typedef struct { int cursor_start; int cursor_end; int shift_state; int bell_pitch; } bios_values_t; /* other macros */ #define ISTEXTSC(scp) (!((scp)->status \ & (UNKNOWN_MODE | GRAPHICS_MODE | PIXEL_MODE))) #define ISGRAPHSC(scp) (((scp)->status \ & (UNKNOWN_MODE | GRAPHICS_MODE))) #define ISPIXELSC(scp) (((scp)->status \ & (UNKNOWN_MODE | GRAPHICS_MODE | PIXEL_MODE))\ == PIXEL_MODE) #define ISUNKNOWNSC(scp) ((scp)->status & UNKNOWN_MODE) #define ISMOUSEAVAIL(af) ((af) & V_ADP_FONT) #define ISFONTAVAIL(af) ((af) & V_ADP_FONT) #define ISPALAVAIL(af) ((af) & V_ADP_PALETTE) #define ISSIGVALID(sig) ((sig) > 0 && (sig) < NSIG) #define SC_VIDEO_LOCKINIT(sc) \ mtx_init(&(sc)->video_mtx, "syscons video lock", NULL, \ MTX_SPIN | MTX_RECURSE); #define SC_VIDEO_LOCK(sc) \ do { \ if (!kdb_active) \ mtx_lock_spin(&(sc)->video_mtx); \ } while(0) #define SC_VIDEO_UNLOCK(sc) \ do { \ if (!kdb_active) \ mtx_unlock_spin(&(sc)->video_mtx); \ } while(0) /* syscons.c */ extern int (*sc_user_ioctl)(struct tty *tp, u_long cmd, caddr_t data, struct thread *td); int sc_probe_unit(int unit, int flags); int sc_attach_unit(int unit, int flags); int set_mode(scr_stat *scp); void sc_set_border(scr_stat *scp, int color); void sc_load_font(scr_stat *scp, int page, int size, int width, u_char *font, int base, int count); void sc_save_font(scr_stat *scp, int page, int size, int width, u_char *font, int base, int count); void sc_show_font(scr_stat *scp, int page); void sc_touch_scrn_saver(void); void sc_draw_cursor_image(scr_stat *scp); void sc_remove_cursor_image(scr_stat *scp); void sc_set_cursor_image(scr_stat *scp); void sc_change_cursor_shape(scr_stat *scp, int flags, int base, int height); int sc_clean_up(scr_stat *scp); int sc_switch_scr(sc_softc_t *sc, u_int next_scr); void sc_alloc_scr_buffer(scr_stat *scp, int wait, int discard); int sc_init_emulator(scr_stat *scp, char *name); void sc_paste(scr_stat *scp, const u_char *p, int count); void sc_respond(scr_stat *scp, const u_char *p, int count, int wakeup); void sc_bell(scr_stat *scp, int pitch, int duration); /* schistory.c */ #ifndef SC_NO_HISTORY int sc_alloc_history_buffer(scr_stat *scp, int lines, int prev_ysize, int wait); void sc_free_history_buffer(scr_stat *scp, int prev_ysize); void sc_hist_save(scr_stat *scp); #define sc_hist_save_one_line(scp, from) \ sc_vtb_append(&(scp)->vtb, (from), (scp)->history, (scp)->xsize) int sc_hist_restore(scr_stat *scp); void sc_hist_home(scr_stat *scp); void sc_hist_end(scr_stat *scp); int sc_hist_up_line(scr_stat *scp); int sc_hist_down_line(scr_stat *scp); int sc_hist_ioctl(struct tty *tp, u_long cmd, caddr_t data, struct thread *td); #endif /* SC_NO_HISTORY */ /* scmouse.c */ #ifndef SC_NO_CUTPASTE void sc_alloc_cut_buffer(scr_stat *scp, int wait); void sc_draw_mouse_image(scr_stat *scp); void sc_remove_mouse_image(scr_stat *scp); int sc_inside_cutmark(scr_stat *scp, int pos); void sc_remove_cutmarking(scr_stat *scp); void sc_remove_all_cutmarkings(sc_softc_t *scp); void sc_remove_all_mouse(sc_softc_t *scp); void sc_mouse_paste(scr_stat *scp); #else #define sc_draw_mouse_image(scp) #define sc_remove_mouse_image(scp) #define sc_inside_cutmark(scp, pos) FALSE #define sc_remove_cutmarking(scp) #define sc_remove_all_cutmarkings(scp) #define sc_remove_all_mouse(scp) #define sc_mouse_paste(scp) #endif /* SC_NO_CUTPASTE */ #ifndef SC_NO_SYSMOUSE void sc_mouse_move(scr_stat *scp, int x, int y); int sc_mouse_ioctl(struct tty *tp, u_long cmd, caddr_t data, struct thread *td); #endif /* SC_NO_SYSMOUSE */ /* scvidctl.c */ int sc_set_text_mode(scr_stat *scp, struct tty *tp, int mode, int xsize, int ysize, int fontsize, int font_width); int sc_set_graphics_mode(scr_stat *scp, struct tty *tp, int mode); int sc_set_pixel_mode(scr_stat *scp, struct tty *tp, int xsize, int ysize, int fontsize, int font_width); int sc_support_pixel_mode(void *arg); int sc_vid_ioctl(struct tty *tp, u_long cmd, caddr_t data, struct thread *td); int sc_render_add(sc_renderer_t *rndr); int sc_render_remove(sc_renderer_t *rndr); sc_rndr_sw_t *sc_render_match(scr_stat *scp, char *name, int mode); /* scvtb.c */ void sc_vtb_init(sc_vtb_t *vtb, int type, int cols, int rows, void *buffer, int wait); void sc_vtb_destroy(sc_vtb_t *vtb); size_t sc_vtb_size(int cols, int rows); void sc_vtb_clear(sc_vtb_t *vtb, int c, int attr); int sc_vtb_getc(sc_vtb_t *vtb, int at); int sc_vtb_geta(sc_vtb_t *vtb, int at); void sc_vtb_putc(sc_vtb_t *vtb, int at, int c, int a); vm_offset_t sc_vtb_putchar(sc_vtb_t *vtb, vm_offset_t p, int c, int a); vm_offset_t sc_vtb_pointer(sc_vtb_t *vtb, int at); int sc_vtb_pos(sc_vtb_t *vtb, int pos, int offset); #define sc_vtb_tail(vtb) ((vtb)->vtb_tail) #define sc_vtb_rows(vtb) ((vtb)->vtb_rows) #define sc_vtb_cols(vtb) ((vtb)->vtb_cols) void sc_vtb_copy(sc_vtb_t *vtb1, int from, sc_vtb_t *vtb2, int to, int count); void sc_vtb_append(sc_vtb_t *vtb1, int from, sc_vtb_t *vtb2, int count); void sc_vtb_seek(sc_vtb_t *vtb, int pos); void sc_vtb_erase(sc_vtb_t *vtb, int at, int count, int c, int attr); void sc_vtb_move(sc_vtb_t *vtb, int from, int to, int count); void sc_vtb_delete(sc_vtb_t *vtb, int at, int count, int c, int attr); void sc_vtb_ins(sc_vtb_t *vtb, int at, int count, int c, int attr); /* sysmouse.c */ int sysmouse_event(mouse_info_t *info); /* scterm.c */ void sc_move_cursor(scr_stat *scp, int x, int y); void sc_clear_screen(scr_stat *scp); int sc_term_add(sc_term_sw_t *sw); int sc_term_remove(sc_term_sw_t *sw); sc_term_sw_t *sc_term_match(char *name); sc_term_sw_t *sc_term_match_by_number(int index); /* machine dependent functions */ int sc_max_unit(void); sc_softc_t *sc_get_softc(int unit, int flags); sc_softc_t *sc_find_softc(struct video_adapter *adp, struct keyboard *kbd); int sc_get_cons_priority(int *unit, int *flags); void sc_get_bios_values(bios_values_t *values); int sc_tone(int herz); #endif /* !_DEV_SYSCONS_SYSCONS_H_ */ Index: projects/netbsd-tests-update-12/sys/dev/usb/net/if_urndis.c =================================================================== --- projects/netbsd-tests-update-12/sys/dev/usb/net/if_urndis.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/dev/usb/net/if_urndis.c (revision 305172) @@ -1,1055 +1,1055 @@ /* $OpenBSD: if_urndis.c,v 1.46 2013/12/09 15:45:29 pirofti Exp $ */ /* * Copyright (c) 2010 Jonathan Armani * Copyright (c) 2010 Fabien Romano * Copyright (c) 2010 Michael Knudsen * Copyright (c) 2014 Hans Petter Selasky * All rights reserved. * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "usbdevs.h" #define USB_DEBUG_VAR urndis_debug #include #include #include "usb_if.h" #include #include #include static device_probe_t urndis_probe; static device_attach_t urndis_attach; static device_detach_t urndis_detach; static device_suspend_t urndis_suspend; static device_resume_t urndis_resume; static usb_callback_t urndis_bulk_write_callback; static usb_callback_t urndis_bulk_read_callback; static usb_callback_t urndis_intr_read_callback; static uether_fn_t urndis_attach_post; static uether_fn_t urndis_init; static uether_fn_t urndis_stop; static uether_fn_t urndis_start; static uether_fn_t urndis_setmulti; static uether_fn_t urndis_setpromisc; static uint32_t urndis_ctrl_query(struct urndis_softc *sc, uint32_t oid, struct rndis_query_req *msg, uint16_t len, const void **rbuf, uint16_t *rbufsz); static uint32_t urndis_ctrl_set(struct urndis_softc *sc, uint32_t oid, struct rndis_set_req *msg, uint16_t len); static uint32_t urndis_ctrl_handle_init(struct urndis_softc *sc, const struct rndis_comp_hdr *hdr); static uint32_t urndis_ctrl_handle_query(struct urndis_softc *sc, const struct rndis_comp_hdr *hdr, const void **buf, uint16_t *bufsz); static uint32_t urndis_ctrl_handle_reset(struct urndis_softc *sc, const struct rndis_comp_hdr *hdr); static uint32_t urndis_ctrl_init(struct urndis_softc *sc); static uint32_t urndis_ctrl_halt(struct urndis_softc *sc); #ifdef USB_DEBUG static int urndis_debug = 0; static SYSCTL_NODE(_hw_usb, OID_AUTO, urndis, CTLFLAG_RW, 0, "USB RNDIS-Ethernet"); SYSCTL_INT(_hw_usb_urndis, OID_AUTO, debug, CTLFLAG_RWTUN, &urndis_debug, 0, "Debug level"); #endif static const struct usb_config urndis_config[URNDIS_N_TRANSFER] = { [URNDIS_BULK_RX] = { .type = UE_BULK, .endpoint = UE_ADDR_ANY, .direction = UE_DIR_RX, .if_index = 0, .frames = 1, .bufsize = RNDIS_RX_MAXLEN, .flags = {.short_xfer_ok = 1,}, .callback = urndis_bulk_read_callback, .timeout = 0, /* no timeout */ .usb_mode = USB_MODE_HOST, }, [URNDIS_BULK_TX] = { .type = UE_BULK, .endpoint = UE_ADDR_ANY, .direction = UE_DIR_TX, .if_index = 0, .frames = RNDIS_TX_FRAMES_MAX, .bufsize = (RNDIS_TX_FRAMES_MAX * RNDIS_TX_MAXLEN), .flags = { .force_short_xfer = 1, }, .callback = urndis_bulk_write_callback, .timeout = 10000, /* 10 seconds */ .usb_mode = USB_MODE_HOST, }, [URNDIS_INTR_RX] = { .type = UE_INTERRUPT, .endpoint = UE_ADDR_ANY, .direction = UE_DIR_RX, .if_index = 1, .bufsize = 0, /* use wMaxPacketSize */ .flags = {.short_xfer_ok = 1,.no_pipe_ok = 1,}, .callback = urndis_intr_read_callback, .timeout = 0, .usb_mode = USB_MODE_HOST, }, }; static device_method_t urndis_methods[] = { /* Device interface */ DEVMETHOD(device_probe, urndis_probe), DEVMETHOD(device_attach, urndis_attach), DEVMETHOD(device_detach, urndis_detach), DEVMETHOD(device_suspend, urndis_suspend), DEVMETHOD(device_resume, urndis_resume), DEVMETHOD_END }; static driver_t urndis_driver = { .name = "urndis", .methods = urndis_methods, .size = sizeof(struct urndis_softc), }; static devclass_t urndis_devclass; static const STRUCT_USB_HOST_ID urndis_host_devs[] = { /* Generic RNDIS class match */ {USB_IFACE_CLASS(UICLASS_CDC), USB_IFACE_SUBCLASS(UISUBCLASS_ABSTRACT_CONTROL_MODEL), USB_IFACE_PROTOCOL(0xff)}, {USB_IFACE_CLASS(UICLASS_WIRELESS), USB_IFACE_SUBCLASS(UISUBCLASS_RF), USB_IFACE_PROTOCOL(UIPROTO_RNDIS)}, {USB_IFACE_CLASS(UICLASS_IAD), USB_IFACE_SUBCLASS(UISUBCLASS_SYNC), USB_IFACE_PROTOCOL(UIPROTO_ACTIVESYNC)}, /* HP-WebOS */ {USB_VENDOR(USB_VENDOR_PALM), USB_IFACE_CLASS(UICLASS_CDC), USB_IFACE_SUBCLASS(UISUBCLASS_ABSTRACT_CONTROL_MODEL), USB_IFACE_PROTOCOL(0xff)}, }; DRIVER_MODULE(urndis, uhub, urndis_driver, urndis_devclass, NULL, NULL); MODULE_VERSION(urndis, 1); MODULE_DEPEND(urndis, uether, 1, 1, 1); MODULE_DEPEND(urndis, usb, 1, 1, 1); MODULE_DEPEND(urndis, ether, 1, 1, 1); USB_PNP_HOST_INFO(urndis_host_devs); static const struct usb_ether_methods urndis_ue_methods = { .ue_attach_post = urndis_attach_post, .ue_start = urndis_start, .ue_init = urndis_init, .ue_stop = urndis_stop, .ue_setmulti = urndis_setmulti, .ue_setpromisc = urndis_setpromisc, }; static int urndis_probe(device_t dev) { struct usb_attach_arg *uaa = device_get_ivars(dev); return (usbd_lookup_id_by_uaa(urndis_host_devs, sizeof(urndis_host_devs), uaa)); } static void urndis_attach_post(struct usb_ether *ue) { /* no-op */ } static int urndis_attach(device_t dev) { static struct { union { struct rndis_query_req query; struct rndis_set_req set; } hdr; union { uint8_t eaddr[ETHER_ADDR_LEN]; uint32_t filter; } ibuf; } msg; struct urndis_softc *sc = device_get_softc(dev); struct usb_ether *ue = &sc->sc_ue; struct usb_attach_arg *uaa = device_get_ivars(dev); struct usb_cdc_cm_descriptor *cmd; const void *buf; uint16_t bufsz; uint8_t iface_index[2] = { uaa->info.bIfaceIndex + 1, uaa->info.bIfaceIndex }; int error; uint8_t i; sc->sc_ue.ue_udev = uaa->device; sc->sc_ifaceno_ctl = uaa->info.bIfaceNum; cmd = usbd_find_descriptor(uaa->device, NULL, uaa->info.bIfaceIndex, UDESC_CS_INTERFACE, 0xFF, UDESCSUB_CDC_CM, 0xFF); if (cmd != NULL) { DPRINTF("Call Mode Descriptor found, dataif=%d\n", cmd->bDataInterface); iface_index[0] = cmd->bDataInterface; } device_set_usb_desc(dev); mtx_init(&sc->sc_mtx, device_get_nameunit(dev), NULL, MTX_DEF); /* scan the alternate settings looking for a valid one */ for (i = 0; i != 32; i++) { error = usbd_set_alt_interface_index(uaa->device, iface_index[0], i); if (error != 0) break; error = usbd_transfer_setup(uaa->device, iface_index, sc->sc_xfer, urndis_config, URNDIS_N_TRANSFER, sc, &sc->sc_mtx); if (error == 0) break; } if ((error != 0) || (i == 32)) { device_printf(dev, "No valid alternate setting found\n"); goto detach; } /* Initialize device - must be done before even querying it */ URNDIS_LOCK(sc); error = urndis_ctrl_init(sc); URNDIS_UNLOCK(sc); if (error != (int)RNDIS_STATUS_SUCCESS) { device_printf(dev, "Unable to initialize hardware\n"); goto detach; } /* Determine MAC address */ memset(msg.ibuf.eaddr, 0, sizeof(msg.ibuf.eaddr)); URNDIS_LOCK(sc); error = urndis_ctrl_query(sc, OID_802_3_PERMANENT_ADDRESS, &msg.hdr.query, sizeof(msg.hdr.query) + sizeof(msg.ibuf.eaddr), &buf, &bufsz); URNDIS_UNLOCK(sc); if (error != (int)RNDIS_STATUS_SUCCESS) { device_printf(dev, "Unable to get hardware address\n"); goto detach; } if (bufsz != ETHER_ADDR_LEN) { device_printf(dev, "Invalid address length: %d bytes\n", bufsz); goto detach; } memcpy(&sc->sc_ue.ue_eaddr, buf, ETHER_ADDR_LEN); /* Initialize packet filter */ - sc->sc_filter = RNDIS_PACKET_TYPE_BROADCAST | - RNDIS_PACKET_TYPE_ALL_MULTICAST; + sc->sc_filter = NDIS_PACKET_TYPE_BROADCAST | + NDIS_PACKET_TYPE_ALL_MULTICAST; msg.ibuf.filter = htole32(sc->sc_filter); URNDIS_LOCK(sc); error = urndis_ctrl_set(sc, OID_GEN_CURRENT_PACKET_FILTER, &msg.hdr.set, sizeof(msg.hdr.set) + sizeof(msg.ibuf.filter)); URNDIS_UNLOCK(sc); if (error != (int)RNDIS_STATUS_SUCCESS) { device_printf(dev, "Unable to set data filters\n"); goto detach; } ue->ue_sc = sc; ue->ue_dev = dev; ue->ue_udev = uaa->device; ue->ue_mtx = &sc->sc_mtx; ue->ue_methods = &urndis_ue_methods; error = uether_ifattach(ue); if (error) { device_printf(dev, "Could not attach interface\n"); goto detach; } URNDIS_LOCK(sc); /* start interrupt endpoint, if any */ usbd_transfer_start(sc->sc_xfer[URNDIS_INTR_RX]); URNDIS_UNLOCK(sc); return (0); /* success */ detach: (void)urndis_detach(dev); return (ENXIO); /* failure */ } static int urndis_detach(device_t dev) { struct urndis_softc *sc = device_get_softc(dev); struct usb_ether *ue = &sc->sc_ue; /* stop all USB transfers first */ usbd_transfer_unsetup(sc->sc_xfer, URNDIS_N_TRANSFER); uether_ifdetach(ue); URNDIS_LOCK(sc); (void)urndis_ctrl_halt(sc); URNDIS_UNLOCK(sc); mtx_destroy(&sc->sc_mtx); return (0); } static void urndis_start(struct usb_ether *ue) { struct urndis_softc *sc = uether_getsc(ue); /* * Start the USB transfers, if not already started: */ usbd_transfer_start(sc->sc_xfer[URNDIS_BULK_TX]); usbd_transfer_start(sc->sc_xfer[URNDIS_BULK_RX]); } static void urndis_init(struct usb_ether *ue) { struct urndis_softc *sc = uether_getsc(ue); struct ifnet *ifp = uether_getifp(ue); URNDIS_LOCK_ASSERT(sc, MA_OWNED); ifp->if_drv_flags |= IFF_DRV_RUNNING; /* stall data write direction, which depends on USB mode */ usbd_xfer_set_stall(sc->sc_xfer[URNDIS_BULK_TX]); /* start data transfers */ urndis_start(ue); } static void urndis_stop(struct usb_ether *ue) { struct urndis_softc *sc = uether_getsc(ue); struct ifnet *ifp = uether_getifp(ue); URNDIS_LOCK_ASSERT(sc, MA_OWNED); ifp->if_drv_flags &= ~IFF_DRV_RUNNING; /* * stop all the transfers, if not already stopped: */ usbd_transfer_stop(sc->sc_xfer[URNDIS_BULK_RX]); usbd_transfer_stop(sc->sc_xfer[URNDIS_BULK_TX]); } static void urndis_setmulti(struct usb_ether *ue) { /* no-op */ } static void urndis_setpromisc(struct usb_ether *ue) { /* no-op */ } static int urndis_suspend(device_t dev) { device_printf(dev, "Suspending\n"); return (0); } static int urndis_resume(device_t dev) { device_printf(dev, "Resuming\n"); return (0); } static usb_error_t urndis_ctrl_msg(struct urndis_softc *sc, uint8_t rt, uint8_t r, uint16_t index, uint16_t value, void *buf, uint16_t buflen) { usb_device_request_t req; req.bmRequestType = rt; req.bRequest = r; USETW(req.wValue, value); USETW(req.wIndex, index); USETW(req.wLength, buflen); return (usbd_do_request_flags(sc->sc_ue.ue_udev, &sc->sc_mtx, &req, buf, (rt & UT_READ) ? USB_SHORT_XFER_OK : 0, NULL, 2000 /* ms */ )); } static usb_error_t urndis_ctrl_send(struct urndis_softc *sc, void *buf, uint16_t len) { usb_error_t err; err = urndis_ctrl_msg(sc, UT_WRITE_CLASS_INTERFACE, UCDC_SEND_ENCAPSULATED_COMMAND, sc->sc_ifaceno_ctl, 0, buf, len); DPRINTF("%s\n", usbd_errstr(err)); return (err); } static struct rndis_comp_hdr * urndis_ctrl_recv(struct urndis_softc *sc) { struct rndis_comp_hdr *hdr; usb_error_t err; err = urndis_ctrl_msg(sc, UT_READ_CLASS_INTERFACE, UCDC_GET_ENCAPSULATED_RESPONSE, sc->sc_ifaceno_ctl, 0, sc->sc_response_buf, RNDIS_RESPONSE_LEN); if (err != USB_ERR_NORMAL_COMPLETION) return (NULL); hdr = (struct rndis_comp_hdr *)sc->sc_response_buf; DPRINTF("type 0x%x len %u\n", le32toh(hdr->rm_type), le32toh(hdr->rm_len)); if (le32toh(hdr->rm_len) > RNDIS_RESPONSE_LEN) { DPRINTF("ctrl message error: wrong size %u > %u\n", le32toh(hdr->rm_len), RNDIS_RESPONSE_LEN); return (NULL); } return (hdr); } static uint32_t urndis_ctrl_handle(struct urndis_softc *sc, struct rndis_comp_hdr *hdr, const void **buf, uint16_t *bufsz) { uint32_t rval; DPRINTF("\n"); if (buf != NULL && bufsz != NULL) { *buf = NULL; *bufsz = 0; } switch (le32toh(hdr->rm_type)) { case REMOTE_NDIS_INITIALIZE_CMPLT: rval = urndis_ctrl_handle_init(sc, hdr); break; case REMOTE_NDIS_QUERY_CMPLT: rval = urndis_ctrl_handle_query(sc, hdr, buf, bufsz); break; case REMOTE_NDIS_RESET_CMPLT: rval = urndis_ctrl_handle_reset(sc, hdr); break; case REMOTE_NDIS_KEEPALIVE_CMPLT: case REMOTE_NDIS_SET_CMPLT: rval = le32toh(hdr->rm_status); break; default: device_printf(sc->sc_ue.ue_dev, "ctrl message error: unknown event 0x%x\n", le32toh(hdr->rm_type)); rval = RNDIS_STATUS_FAILURE; break; } return (rval); } static uint32_t urndis_ctrl_handle_init(struct urndis_softc *sc, const struct rndis_comp_hdr *hdr) { const struct rndis_init_comp *msg; msg = (const struct rndis_init_comp *)hdr; DPRINTF("len %u rid %u status 0x%x " "ver_major %u ver_minor %u devflags 0x%x medium 0x%x pktmaxcnt %u " "pktmaxsz %u align %u aflistoffset %u aflistsz %u\n", le32toh(msg->rm_len), le32toh(msg->rm_rid), le32toh(msg->rm_status), le32toh(msg->rm_ver_major), le32toh(msg->rm_ver_minor), le32toh(msg->rm_devflags), le32toh(msg->rm_medium), le32toh(msg->rm_pktmaxcnt), le32toh(msg->rm_pktmaxsz), le32toh(msg->rm_align), le32toh(msg->rm_aflistoffset), le32toh(msg->rm_aflistsz)); if (le32toh(msg->rm_status) != RNDIS_STATUS_SUCCESS) { DPRINTF("init failed 0x%x\n", le32toh(msg->rm_status)); return (le32toh(msg->rm_status)); } if (le32toh(msg->rm_devflags) != RNDIS_DF_CONNECTIONLESS) { DPRINTF("wrong device type (current type: 0x%x)\n", le32toh(msg->rm_devflags)); return (RNDIS_STATUS_FAILURE); } if (le32toh(msg->rm_medium) != RNDIS_MEDIUM_802_3) { DPRINTF("medium not 802.3 (current medium: 0x%x)\n", le32toh(msg->rm_medium)); return (RNDIS_STATUS_FAILURE); } sc->sc_lim_pktsz = le32toh(msg->rm_pktmaxsz); return (le32toh(msg->rm_status)); } static uint32_t urndis_ctrl_handle_query(struct urndis_softc *sc, const struct rndis_comp_hdr *hdr, const void **buf, uint16_t *bufsz) { const struct rndis_query_comp *msg; uint64_t limit; msg = (const struct rndis_query_comp *)hdr; DPRINTF("len %u rid %u status 0x%x " "buflen %u bufoff %u\n", le32toh(msg->rm_len), le32toh(msg->rm_rid), le32toh(msg->rm_status), le32toh(msg->rm_infobuflen), le32toh(msg->rm_infobufoffset)); *buf = NULL; *bufsz = 0; if (le32toh(msg->rm_status) != RNDIS_STATUS_SUCCESS) { DPRINTF("query failed 0x%x\n", le32toh(msg->rm_status)); return (le32toh(msg->rm_status)); } limit = le32toh(msg->rm_infobuflen); limit += le32toh(msg->rm_infobufoffset); limit += RNDIS_HEADER_OFFSET; if (limit > (uint64_t)le32toh(msg->rm_len)) { DPRINTF("ctrl message error: invalid query info " "len/offset/end_position(%u/%u/%u) -> " "go out of buffer limit %u\n", le32toh(msg->rm_infobuflen), le32toh(msg->rm_infobufoffset), le32toh(msg->rm_infobuflen) + le32toh(msg->rm_infobufoffset) + RNDIS_HEADER_OFFSET, le32toh(msg->rm_len)); return (RNDIS_STATUS_FAILURE); } *buf = ((const uint8_t *)msg) + RNDIS_HEADER_OFFSET + le32toh(msg->rm_infobufoffset); *bufsz = le32toh(msg->rm_infobuflen); return (le32toh(msg->rm_status)); } static uint32_t urndis_ctrl_handle_reset(struct urndis_softc *sc, const struct rndis_comp_hdr *hdr) { const struct rndis_reset_comp *msg; uint32_t rval; msg = (const struct rndis_reset_comp *)hdr; rval = le32toh(msg->rm_status); DPRINTF("len %u status 0x%x " "adrreset %u\n", le32toh(msg->rm_len), rval, le32toh(msg->rm_adrreset)); if (rval != RNDIS_STATUS_SUCCESS) { DPRINTF("reset failed 0x%x\n", rval); return (rval); } if (msg->rm_adrreset != 0) { struct { struct rndis_set_req hdr; uint32_t filter; } msg_filter; msg_filter.filter = htole32(sc->sc_filter); rval = urndis_ctrl_set(sc, OID_GEN_CURRENT_PACKET_FILTER, &msg_filter.hdr, sizeof(msg_filter)); if (rval != RNDIS_STATUS_SUCCESS) { DPRINTF("unable to reset data filters\n"); return (rval); } } return (rval); } static uint32_t urndis_ctrl_init(struct urndis_softc *sc) { struct rndis_init_req msg; struct rndis_comp_hdr *hdr; uint32_t rval; msg.rm_type = htole32(REMOTE_NDIS_INITIALIZE_MSG); msg.rm_len = htole32(sizeof(msg)); msg.rm_rid = 0; msg.rm_ver_major = htole32(RNDIS_VERSION_MAJOR); msg.rm_ver_minor = htole32(1); msg.rm_max_xfersz = htole32(RNDIS_RX_MAXLEN); DPRINTF("type %u len %u rid %u ver_major %u " "ver_minor %u max_xfersz %u\n", le32toh(msg.rm_type), le32toh(msg.rm_len), le32toh(msg.rm_rid), le32toh(msg.rm_ver_major), le32toh(msg.rm_ver_minor), le32toh(msg.rm_max_xfersz)); rval = urndis_ctrl_send(sc, &msg, sizeof(msg)); if (rval != RNDIS_STATUS_SUCCESS) { DPRINTF("init failed\n"); return (rval); } if ((hdr = urndis_ctrl_recv(sc)) == NULL) { DPRINTF("unable to get init response\n"); return (RNDIS_STATUS_FAILURE); } rval = urndis_ctrl_handle(sc, hdr, NULL, NULL); return (rval); } static uint32_t urndis_ctrl_halt(struct urndis_softc *sc) { struct rndis_halt_req msg; uint32_t rval; msg.rm_type = htole32(REMOTE_NDIS_HALT_MSG); msg.rm_len = htole32(sizeof(msg)); msg.rm_rid = 0; DPRINTF("type %u len %u rid %u\n", le32toh(msg.rm_type), le32toh(msg.rm_len), le32toh(msg.rm_rid)); rval = urndis_ctrl_send(sc, &msg, sizeof(msg)); if (rval != RNDIS_STATUS_SUCCESS) DPRINTF("halt failed\n"); return (rval); } /* * NB: Querying a device has the requirement of using an input buffer the size * of the expected reply or larger, except for variably sized replies. */ static uint32_t urndis_ctrl_query(struct urndis_softc *sc, uint32_t oid, struct rndis_query_req *msg, uint16_t len, const void **rbuf, uint16_t *rbufsz) { struct rndis_comp_hdr *hdr; uint32_t datalen, rval; msg->rm_type = htole32(REMOTE_NDIS_QUERY_MSG); msg->rm_len = htole32(len); msg->rm_rid = 0; /* XXX */ msg->rm_oid = htole32(oid); datalen = len - sizeof(*msg); msg->rm_infobuflen = htole32(datalen); if (datalen != 0) { msg->rm_infobufoffset = htole32(sizeof(*msg) - RNDIS_HEADER_OFFSET); } else { msg->rm_infobufoffset = 0; } msg->rm_devicevchdl = 0; DPRINTF("type %u len %u rid %u oid 0x%x " "infobuflen %u infobufoffset %u devicevchdl %u\n", le32toh(msg->rm_type), le32toh(msg->rm_len), le32toh(msg->rm_rid), le32toh(msg->rm_oid), le32toh(msg->rm_infobuflen), le32toh(msg->rm_infobufoffset), le32toh(msg->rm_devicevchdl)); rval = urndis_ctrl_send(sc, msg, len); if (rval != RNDIS_STATUS_SUCCESS) { DPRINTF("query failed\n"); return (rval); } if ((hdr = urndis_ctrl_recv(sc)) == NULL) { DPRINTF("unable to get query response\n"); return (RNDIS_STATUS_FAILURE); } rval = urndis_ctrl_handle(sc, hdr, rbuf, rbufsz); return (rval); } static uint32_t urndis_ctrl_set(struct urndis_softc *sc, uint32_t oid, struct rndis_set_req *msg, uint16_t len) { struct rndis_comp_hdr *hdr; uint32_t datalen, rval; msg->rm_type = htole32(REMOTE_NDIS_SET_MSG); msg->rm_len = htole32(len); msg->rm_rid = 0; /* XXX */ msg->rm_oid = htole32(oid); datalen = len - sizeof(*msg); msg->rm_infobuflen = htole32(datalen); if (datalen != 0) { msg->rm_infobufoffset = htole32(sizeof(*msg) - RNDIS_HEADER_OFFSET); } else { msg->rm_infobufoffset = 0; } msg->rm_devicevchdl = 0; DPRINTF("type %u len %u rid %u oid 0x%x " "infobuflen %u infobufoffset %u devicevchdl %u\n", le32toh(msg->rm_type), le32toh(msg->rm_len), le32toh(msg->rm_rid), le32toh(msg->rm_oid), le32toh(msg->rm_infobuflen), le32toh(msg->rm_infobufoffset), le32toh(msg->rm_devicevchdl)); rval = urndis_ctrl_send(sc, msg, len); if (rval != RNDIS_STATUS_SUCCESS) { DPRINTF("set failed\n"); return (rval); } if ((hdr = urndis_ctrl_recv(sc)) == NULL) { DPRINTF("unable to get set response\n"); return (RNDIS_STATUS_FAILURE); } rval = urndis_ctrl_handle(sc, hdr, NULL, NULL); if (rval != RNDIS_STATUS_SUCCESS) DPRINTF("set failed 0x%x\n", rval); return (rval); } static void urndis_bulk_read_callback(struct usb_xfer *xfer, usb_error_t error) { struct urndis_softc *sc = usbd_xfer_softc(xfer); struct usb_page_cache *pc = usbd_xfer_get_frame(xfer, 0); struct ifnet *ifp = uether_getifp(&sc->sc_ue); struct rndis_packet_msg msg; struct mbuf *m; int actlen; int aframes; int offset; switch (USB_GET_STATE(xfer)) { case USB_ST_TRANSFERRED: usbd_xfer_status(xfer, &actlen, NULL, &aframes, NULL); DPRINTFN(1, "received %u bytes in %u frames\n", actlen, aframes); for (offset = 0; actlen >= (uint32_t)sizeof(msg);) { /* copy out header */ usbd_copy_out(pc, offset, &msg, sizeof(msg)); if (le32toh(0x1234567U) != 0x1234567U) { /* swap endianness */ msg.rm_type = le32toh(msg.rm_type); msg.rm_len = le32toh(msg.rm_len); msg.rm_dataoffset = le32toh(msg.rm_dataoffset); msg.rm_datalen = le32toh(msg.rm_datalen); msg.rm_oobdataoffset = le32toh(msg.rm_oobdataoffset); msg.rm_oobdatalen = le32toh(msg.rm_oobdatalen); msg.rm_oobdataelements = le32toh(msg.rm_oobdataelements); msg.rm_pktinfooffset = le32toh(msg.rm_pktinfooffset); msg.rm_pktinfolen = le32toh(msg.rm_pktinfolen); msg.rm_vchandle = le32toh(msg.rm_vchandle); msg.rm_reserved = le32toh(msg.rm_reserved); } DPRINTF("len %u data(off:%u len:%u) " "oobdata(off:%u len:%u nb:%u) perpacket(off:%u len:%u)\n", msg.rm_len, msg.rm_dataoffset, msg.rm_datalen, msg.rm_oobdataoffset, msg.rm_oobdatalen, msg.rm_oobdataelements, msg.rm_pktinfooffset, msg.rm_pktinfooffset); /* sanity check the RNDIS header */ if (msg.rm_type != REMOTE_NDIS_PACKET_MSG) { DPRINTF("invalid type 0x%x != 0x%x\n", msg.rm_type, REMOTE_NDIS_PACKET_MSG); goto tr_setup; } else if (msg.rm_len < (uint32_t)sizeof(msg)) { DPRINTF("invalid msg len %u < %u\n", msg.rm_len, (unsigned)sizeof(msg)); goto tr_setup; } else if (msg.rm_len > (uint32_t)actlen) { DPRINTF("invalid msg len %u > buffer " "len %u\n", msg.rm_len, actlen); goto tr_setup; } else if (msg.rm_dataoffset >= (uint32_t)actlen) { DPRINTF("invalid msg dataoffset %u > buffer " "dataoffset %u\n", msg.rm_dataoffset, actlen); goto tr_setup; } else if (msg.rm_datalen > (uint32_t)actlen) { DPRINTF("invalid msg datalen %u > buffer " "datalen %u\n", msg.rm_datalen, actlen); goto tr_setup; } else if ((msg.rm_dataoffset + msg.rm_datalen + (uint32_t)__offsetof(struct rndis_packet_msg, rm_dataoffset)) > (uint32_t)actlen) { DPRINTF("invalid dataoffset %u larger than %u\n", msg.rm_dataoffset + msg.rm_datalen + (uint32_t)__offsetof(struct rndis_packet_msg, rm_dataoffset), actlen); goto tr_setup; } else if (msg.rm_datalen < (uint32_t)sizeof(struct ether_header)) { if_inc_counter(ifp, IFCOUNTER_IERRORS, 1); DPRINTF("invalid ethernet size " "%u < %u\n", msg.rm_datalen, (unsigned)sizeof(struct ether_header)); goto tr_setup; } else if (msg.rm_datalen > (uint32_t)(MCLBYTES - ETHER_ALIGN)) { if_inc_counter(ifp, IFCOUNTER_IERRORS, 1); DPRINTF("invalid ethernet size " "%u > %u\n", msg.rm_datalen, (unsigned)MCLBYTES); goto tr_setup; } else if (msg.rm_datalen > (uint32_t)(MHLEN - ETHER_ALIGN)) { m = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); } else { m = m_gethdr(M_NOWAIT, MT_DATA); } /* check if we have a buffer */ if (m != NULL) { m->m_len = m->m_pkthdr.len = msg.rm_datalen + ETHER_ALIGN; m_adj(m, ETHER_ALIGN); usbd_copy_out(pc, offset + msg.rm_dataoffset + __offsetof(struct rndis_packet_msg, rm_dataoffset), m->m_data, msg.rm_datalen); /* enqueue */ uether_rxmbuf(&sc->sc_ue, m, msg.rm_datalen); } else { if_inc_counter(ifp, IFCOUNTER_IERRORS, 1); } offset += msg.rm_len; actlen -= msg.rm_len; } case USB_ST_SETUP: tr_setup: usbd_xfer_set_frame_len(xfer, 0, RNDIS_RX_MAXLEN); usbd_xfer_set_frames(xfer, 1); usbd_transfer_submit(xfer); uether_rxflush(&sc->sc_ue); /* must be last */ break; default: /* Error */ DPRINTFN(1, "error = %s\n", usbd_errstr(error)); if (error != USB_ERR_CANCELLED) { /* try to clear stall first */ usbd_xfer_set_stall(xfer); usbd_xfer_set_frames(xfer, 0); usbd_transfer_submit(xfer); } break; } } static void urndis_bulk_write_callback(struct usb_xfer *xfer, usb_error_t error) { struct rndis_packet_msg msg; struct urndis_softc *sc = usbd_xfer_softc(xfer); struct ifnet *ifp = uether_getifp(&sc->sc_ue); struct mbuf *m; unsigned x; int actlen; int aframes; usbd_xfer_status(xfer, &actlen, NULL, &aframes, NULL); DPRINTFN(1, "\n"); switch (USB_GET_STATE(xfer)) { case USB_ST_TRANSFERRED: DPRINTFN(11, "%u bytes in %u frames\n", actlen, aframes); if_inc_counter(ifp, IFCOUNTER_OPACKETS, 1); /* FALLTHROUGH */ case USB_ST_SETUP: tr_setup: memset(&msg, 0, sizeof(msg)); for (x = 0; x != RNDIS_TX_FRAMES_MAX; x++) { struct usb_page_cache *pc = usbd_xfer_get_frame(xfer, x); usbd_xfer_set_frame_offset(xfer, x * RNDIS_TX_MAXLEN, x); next_pkt: IFQ_DRV_DEQUEUE(&ifp->if_snd, m); if (m == NULL) break; if ((m->m_pkthdr.len + sizeof(msg)) > RNDIS_TX_MAXLEN) { DPRINTF("Too big packet\n"); if_inc_counter(ifp, IFCOUNTER_OERRORS, 1); /* Free buffer */ m_freem(m); goto next_pkt; } msg.rm_type = htole32(REMOTE_NDIS_PACKET_MSG); msg.rm_len = htole32(sizeof(msg) + m->m_pkthdr.len); msg.rm_dataoffset = htole32(RNDIS_DATA_OFFSET); msg.rm_datalen = htole32(m->m_pkthdr.len); /* copy in all data */ usbd_copy_in(pc, 0, &msg, sizeof(msg)); usbd_m_copy_in(pc, sizeof(msg), m, 0, m->m_pkthdr.len); usbd_xfer_set_frame_len(xfer, x, sizeof(msg) + m->m_pkthdr.len); /* * If there's a BPF listener, bounce a copy of * this frame to him: */ BPF_MTAP(ifp, m); /* Free buffer */ m_freem(m); } if (x != 0) { usbd_xfer_set_frames(xfer, x); usbd_transfer_submit(xfer); } break; default: /* Error */ DPRINTFN(11, "transfer error, %s\n", usbd_errstr(error)); /* count output errors */ if_inc_counter(ifp, IFCOUNTER_OERRORS, 1); if (error != USB_ERR_CANCELLED) { /* try to clear stall first */ usbd_xfer_set_stall(xfer); goto tr_setup; } break; } } static void urndis_intr_read_callback(struct usb_xfer *xfer, usb_error_t error) { int actlen; usbd_xfer_status(xfer, &actlen, NULL, NULL, NULL); switch (USB_GET_STATE(xfer)) { case USB_ST_TRANSFERRED: DPRINTF("Received %d bytes\n", actlen); /* TODO: decode some indications */ /* FALLTHROUGH */ case USB_ST_SETUP: tr_setup: usbd_xfer_set_frame_len(xfer, 0, usbd_xfer_max_len(xfer)); usbd_transfer_submit(xfer); break; default: /* Error */ if (error != USB_ERR_CANCELLED) { /* start clear stall */ usbd_xfer_set_stall(xfer); goto tr_setup; } break; } } Index: projects/netbsd-tests-update-12/sys/kern/capabilities.conf =================================================================== --- projects/netbsd-tests-update-12/sys/kern/capabilities.conf (revision 305171) +++ projects/netbsd-tests-update-12/sys/kern/capabilities.conf (revision 305172) @@ -1,741 +1,742 @@ ## ## Copyright (c) 2008-2010 Robert N. M. Watson ## All rights reserved. ## ## This software was developed at the University of Cambridge Computer ## Laboratory with support from a grant from Google, Inc. ## ## Redistribution and use in source and binary forms, with or without ## modification, are permitted provided that the following conditions ## are met: ## 1. Redistributions of source code must retain the above copyright ## notice, this list of conditions and the following disclaimer. ## 2. Redistributions in binary form must reproduce the above copyright ## notice, this list of conditions and the following disclaimer in the ## documentation and/or other materials provided with the distribution. ## ## THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND ## ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE ## IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ## ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE ## FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ## DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS ## OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) ## HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT ## LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY ## OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF ## SUCH DAMAGE. ## ## List of system calls enabled in capability mode, one name per line. ## ## Notes: ## - sys_exit(2), abort2(2) and close(2) are very important. ## - Sorted alphabetically, please keep it that way. ## ## $FreeBSD$ ## ## ## Allow ACL and MAC label operations by file descriptor, subject to ## capability rights. Allow MAC label operations on the current process but ## we will need to scope __mac_get_pid(2). ## __acl_aclcheck_fd __acl_delete_fd __acl_get_fd __acl_set_fd __mac_get_fd #__mac_get_pid __mac_get_proc __mac_set_fd __mac_set_proc ## ## Allow sysctl(2) as we scope internal to the call; this is a global ## namespace, but there are several critical sysctls required for almost ## anything to run, such as hw.pagesize. For now that policy lives in the ## kernel for performance and simplicity, but perhaps it could move to a ## proxying daemon in userspace. ## __sysctl ## ## Allow umtx operations as these are scoped by address space. ## ## XXRW: Need to check this very carefully. ## _umtx_op ## ## Allow process termination using abort2(2). ## abort2 ## ## Allow accept(2) since it doesn't manipulate namespaces directly, rather ## relies on existing bindings on a socket, subject to capability rights. ## accept accept4 ## ## Allow AIO operations by file descriptor, subject to capability rights. ## aio_cancel aio_error aio_fsync aio_read aio_return aio_suspend aio_waitcomplete aio_write ## ## audit(2) is a global operation, submitting to the global trail, but it is ## controlled by privilege, and it might be useful to be able to submit ## records from sandboxes. For now, disallow, but we may want to think about ## providing some sort of proxy service for this. ## #audit ## ## Allow bindat(2). ## bindat ## ## Allow capability mode and capability system calls. ## cap_enter cap_fcntls_get cap_fcntls_limit cap_getmode cap_ioctls_get cap_ioctls_limit __cap_rights_get cap_rights_limit ## ## Allow read-only clock operations. ## clock_getres clock_gettime ## ## Always allow file descriptor close(2). ## close closefrom ## ## Allow connectat(2). ## connectat ## ## cpuset(2) and related calls require scoping by process, but should ## eventually be allowed, at least in the current process case. ## #cpuset #cpuset_getaffinity #cpuset_getid #cpuset_setaffinity #cpuset_setid ## ## Always allow dup(2) and dup2(2) manipulation of the file descriptor table. ## dup dup2 ## ## Allow extended attribute operations by file descriptor, subject to ## capability rights. ## extattr_delete_fd extattr_get_fd extattr_list_fd extattr_set_fd ## ## Allow changing file flags, mode, and owner by file descriptor, subject to ## capability rights. ## fchflags fchmod fchown ## ## For now, allow fcntl(2), subject to capability rights, but this probably ## needs additional scoping. ## fcntl ## ## Allow fexecve(2), subject to capability rights. We perform some scoping, ## such as disallowing privilege escalation. ## fexecve ## ## Allow flock(2), subject to capability rights. ## flock ## ## Allow fork(2), even though it returns pids -- some applications seem to ## prefer this interface. ## fork ## ## Allow fpathconf(2), subject to capability rights. ## fpathconf ## ## Allow various file descriptor-based I/O operations, subject to capability ## rights. ## freebsd6_ftruncate freebsd6_lseek freebsd6_mmap freebsd6_pread freebsd6_pwrite ## ## Allow querying file and file system state with fstat(2) and fstatfs(2), ## subject to capability rights. ## fstat fstatfs ## ## Allow further file descriptor-based I/O operations, subject to capability ## rights. ## fsync ftruncate ## ## Allow futimens(2) and futimes(2), subject to capability rights. ## futimens futimes ## ## Allow querying process audit state, subject to normal access control. ## getaudit getaudit_addr getauid ## ## Allow thread context management with getcontext(2). ## getcontext ## ## Allow directory I/O on a file descriptor, subject to capability rights. ## Originally we had separate capabilities for directory-specific read ## operations, but on BSD we allow reading the raw directory data, so we just ## rely on CAP_READ now. ## getdents getdirentries ## ## Allow querying certain trivial global state. ## getdomainname +getdtablesize ## ## Allow querying current process credential state. ## getegid geteuid ## ## Allow querying certain trivial global state. ## gethostid gethostname ## ## Allow querying per-process timer. ## getitimer ## ## Allow querying current process credential state. ## getgid getgroups getlogin ## ## Allow querying certain trivial global state. ## getpagesize getpeername ## ## Allow querying certain per-process scheduling, resource limit, and ## credential state. ## ## XXXRW: getpgid(2) needs scoping. It's not clear if it's worth scoping ## getppid(2). getpriority(2) needs scoping. getrusage(2) needs scoping. ## getsid(2) needs scoping. ## getpgid getpgrp getpid getppid getpriority getresgid getresuid getrlimit getrusage getsid ## ## Allow querying socket state, subject to capability rights. ## ## XXXRW: getsockopt(2) may need more attention. ## getsockname getsockopt ## ## Allow querying the global clock. ## gettimeofday ## ## Allow querying current process credential state. ## getuid ## ## Allow ioctl(2), which hopefully will be limited by applications only to ## required commands with cap_ioctls_limit(2) syscall. ## ioctl ## ## Allow querying current process credential state. ## issetugid ## ## Allow kevent(2), as we will authorize based on capability rights on the ## target descriptor. ## kevent ## ## Allow kill(2), as we allow the process to send signals only to himself. ## kill ## ## Allow message queue operations on file descriptors, subject to capability ## rights. ## kmq_notify kmq_setattr kmq_timedreceive kmq_timedsend ## ## Allow kqueue(2), we will control use. ## kqueue ## ## Allow managing per-process timers. ## ktimer_create ktimer_delete ktimer_getoverrun ktimer_gettime ktimer_settime ## ## We can't allow ktrace(2) because it relies on a global namespace, but we ## might want to introduce an fktrace(2) of some sort. ## #ktrace ## ## Allow AIO operations by file descriptor, subject to capability rights. ## lio_listio ## ## Allow listen(2), subject to capability rights. ## ## XXXRW: One might argue this manipulates a global namespace. ## listen ## ## Allow I/O-related file descriptors, subject to capability rights. ## lseek ## ## Allow simple VM operations on the current process. ## madvise mincore minherit mlock mlockall ## ## Allow memory mapping a file descriptor, and updating protections, subject ## to capability rights. ## mmap mprotect ## ## Allow simple VM operations on the current process. ## msync munlock munlockall munmap ## ## Allow the current process to sleep. ## nanosleep ## ## Allow querying the global clock. ## ntp_gettime ## ## Allow AIO operations by file descriptor, subject to capability rights. ## oaio_read oaio_write ## ## Allow simple VM operations on the current process. ## obreak ## ## Allow AIO operations by file descriptor, subject to capability rights. ## olio_listio ## ## Operations relative to directory capabilities. ## chflagsat faccessat fchmodat fchownat fstatat futimesat linkat mkdirat mkfifoat mknodat openat readlinkat renameat symlinkat unlinkat utimensat ## ## Allow entry into open(2). This system call will fail, since access to the ## global file namespace has been disallowed, but allowing entry into the ## syscall means that an audit trail will be generated (which is also very ## useful for debugging). ## open ## ## Process descriptor-related system calls are allowed. ## pdfork pdgetpid pdkill #pdwait4 # not yet implemented ## ## Allow pipe(2). ## pipe pipe2 ## ## Allow poll(2), which will be scoped by capability rights. ## XXXRW: We don't yet do that scoping. ## poll ## ## Allow I/O-related file descriptors, subject to capability rights. ## pread preadv ## ## Allow access to profiling state on the current process. ## profil ## ## Disallow ptrace(2) for now, but we do need debugging facilities in ## capability mode, so we will want to revisit this, possibly by scoping its ## operation. ## #ptrace ## ## Allow I/O-related file descriptors, subject to capability rights. ## pwrite pwritev read readv recv recvfrom recvmsg ## ## Allow real-time scheduling primitives to be used. ## ## XXXRW: These require scoping. ## rtprio rtprio_thread ## ## Allow simple VM operations on the current process. ## sbrk ## ## Allow querying trivial global scheduler state. ## sched_get_priority_max sched_get_priority_min ## ## Allow various thread/process scheduler operations. ## ## XXXRW: Some of these require further scoping. ## sched_getparam sched_getscheduler sched_rr_get_interval sched_setparam sched_setscheduler sched_yield ## ## Allow I/O-related file descriptors, subject to capability rights. ## sctp_generic_recvmsg sctp_generic_sendmsg sctp_generic_sendmsg_iov sctp_peeloff ## ## Allow pselect(2) and select(2), which will be scoped by capability rights. ## ## XXXRW: But is it? ## pselect select ## ## Allow I/O-related file descriptors, subject to capability rights. Use of ## explicit addresses here is restricted by the system calls themselves. ## send sendfile sendmsg sendto ## ## Allow setting per-process audit state, which is controlled separately by ## privileges. ## setaudit setaudit_addr setauid ## ## Allow setting thread context. ## setcontext ## ## Allow setting current process credential state, which is controlled ## separately by privilege. ## setegid seteuid setgid ## ## Allow use of the process interval timer. ## setitimer ## ## Allow setpriority(2). ## ## XXXRW: Requires scoping. ## setpriority ## ## Allow setting current process credential state, which is controlled ## separately by privilege. ## setregid setresgid setresuid setreuid ## ## Allow setting process resource limits with setrlimit(2). ## setrlimit ## ## Allow creating a new session with setsid(2). ## setsid ## ## Allow setting socket options with setsockopt(2), subject to capability ## rights. ## ## XXXRW: Might require scoping. ## setsockopt ## ## Allow setting current process credential state, which is controlled ## separately by privilege. ## setuid ## ## shm_open(2) is scoped so as to allow only access to new anonymous objects. ## shm_open ## ## Allow I/O-related file descriptors, subject to capability rights. ## shutdown ## ## Allow signal control on current process. ## sigaction sigaltstack sigblock sigpending sigprocmask sigqueue sigreturn sigsetmask sigstack sigsuspend sigtimedwait sigvec sigwaitinfo sigwait ## ## Allow creating new socket pairs with socket(2) and socketpair(2). ## socket socketpair ## ## Allow simple VM operations on the current process. ## ## XXXRW: Kernel doesn't implement this, so drop? ## sstk ## ## Do allow sync(2) for now, but possibly shouldn't. ## sync ## ## Always allow process termination with sys_exit(2). ## sys_exit ## ## sysarch(2) does rather diverse things, but is required on at least i386 ## in order to configure per-thread data. As such, it's scoped on each ## architecture. ## sysarch ## ## Allow thread operations operating only on current process. ## thr_create thr_exit thr_kill ## ## Disallow thr_kill2(2), as it may operate beyond the current process. ## ## XXXRW: Requires scoping. ## #thr_kill2 ## ## Allow thread operations operating only on current process. ## thr_new thr_self thr_set_name thr_suspend thr_wake ## ## Allow manipulation of the current process umask with umask(2). ## umask ## ## Allow submitting of process trace entries with utrace(2). ## utrace ## ## Allow generating UUIDs with uuidgen(2). ## uuidgen ## ## Allow I/O-related file descriptors, subject to capability rights. ## write writev ## ## Allow processes to yield(2). ## yield Index: projects/netbsd-tests-update-12/sys/kern/init_sysent.c =================================================================== --- projects/netbsd-tests-update-12/sys/kern/init_sysent.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/kern/init_sysent.c (revision 305172) @@ -1,600 +1,600 @@ /* * System call switch table. * * DO NOT EDIT-- this file is automatically generated. * $FreeBSD$ * created from FreeBSD: head/sys/kern/syscalls.master 304395 2016-08-18 10:50:40Z gnn */ #include "opt_compat.h" #include #include #include #define AS(name) (sizeof(struct name) / sizeof(register_t)) #ifdef COMPAT_43 #define compat(n, name) n, (sy_call_t *)__CONCAT(o,name) #else #define compat(n, name) 0, (sy_call_t *)nosys #endif #ifdef COMPAT_FREEBSD4 #define compat4(n, name) n, (sy_call_t *)__CONCAT(freebsd4_,name) #else #define compat4(n, name) 0, (sy_call_t *)nosys #endif #ifdef COMPAT_FREEBSD6 #define compat6(n, name) n, (sy_call_t *)__CONCAT(freebsd6_,name) #else #define compat6(n, name) 0, (sy_call_t *)nosys #endif #ifdef COMPAT_FREEBSD7 #define compat7(n, name) n, (sy_call_t *)__CONCAT(freebsd7_,name) #else #define compat7(n, name) 0, (sy_call_t *)nosys #endif #ifdef COMPAT_FREEBSD10 #define compat10(n, name) n, (sy_call_t *)__CONCAT(freebsd10_,name) #else #define compat10(n, name) 0, (sy_call_t *)nosys #endif /* The casts are bogus but will do for now. */ struct sysent sysent[] = { { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 0 = syscall */ { AS(sys_exit_args), (sy_call_t *)sys_sys_exit, AUE_EXIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 1 = exit */ { 0, (sy_call_t *)sys_fork, AUE_FORK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 2 = fork */ { AS(read_args), (sy_call_t *)sys_read, AUE_READ, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 3 = read */ { AS(write_args), (sy_call_t *)sys_write, AUE_WRITE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 4 = write */ { AS(open_args), (sy_call_t *)sys_open, AUE_OPEN_RWTC, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 5 = open */ { AS(close_args), (sy_call_t *)sys_close, AUE_CLOSE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 6 = close */ { AS(wait4_args), (sy_call_t *)sys_wait4, AUE_WAIT4, NULL, 0, 0, 0, SY_THR_STATIC }, /* 7 = wait4 */ { compat(AS(ocreat_args),creat), AUE_CREAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 8 = old creat */ { AS(link_args), (sy_call_t *)sys_link, AUE_LINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 9 = link */ { AS(unlink_args), (sy_call_t *)sys_unlink, AUE_UNLINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 10 = unlink */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 11 = obsolete execv */ { AS(chdir_args), (sy_call_t *)sys_chdir, AUE_CHDIR, NULL, 0, 0, 0, SY_THR_STATIC }, /* 12 = chdir */ { AS(fchdir_args), (sy_call_t *)sys_fchdir, AUE_FCHDIR, NULL, 0, 0, 0, SY_THR_STATIC }, /* 13 = fchdir */ { AS(mknod_args), (sy_call_t *)sys_mknod, AUE_MKNOD, NULL, 0, 0, 0, SY_THR_STATIC }, /* 14 = mknod */ { AS(chmod_args), (sy_call_t *)sys_chmod, AUE_CHMOD, NULL, 0, 0, 0, SY_THR_STATIC }, /* 15 = chmod */ { AS(chown_args), (sy_call_t *)sys_chown, AUE_CHOWN, NULL, 0, 0, 0, SY_THR_STATIC }, /* 16 = chown */ { AS(obreak_args), (sy_call_t *)sys_obreak, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 17 = break */ { compat4(AS(freebsd4_getfsstat_args),getfsstat), AUE_GETFSSTAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 18 = freebsd4 getfsstat */ { compat(AS(olseek_args),lseek), AUE_LSEEK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 19 = old lseek */ { 0, (sy_call_t *)sys_getpid, AUE_GETPID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 20 = getpid */ { AS(mount_args), (sy_call_t *)sys_mount, AUE_MOUNT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 21 = mount */ { AS(unmount_args), (sy_call_t *)sys_unmount, AUE_UMOUNT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 22 = unmount */ { AS(setuid_args), (sy_call_t *)sys_setuid, AUE_SETUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 23 = setuid */ { 0, (sy_call_t *)sys_getuid, AUE_GETUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 24 = getuid */ { 0, (sy_call_t *)sys_geteuid, AUE_GETEUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 25 = geteuid */ { AS(ptrace_args), (sy_call_t *)sys_ptrace, AUE_PTRACE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 26 = ptrace */ { AS(recvmsg_args), (sy_call_t *)sys_recvmsg, AUE_RECVMSG, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 27 = recvmsg */ { AS(sendmsg_args), (sy_call_t *)sys_sendmsg, AUE_SENDMSG, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 28 = sendmsg */ { AS(recvfrom_args), (sy_call_t *)sys_recvfrom, AUE_RECVFROM, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 29 = recvfrom */ { AS(accept_args), (sy_call_t *)sys_accept, AUE_ACCEPT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 30 = accept */ { AS(getpeername_args), (sy_call_t *)sys_getpeername, AUE_GETPEERNAME, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 31 = getpeername */ { AS(getsockname_args), (sy_call_t *)sys_getsockname, AUE_GETSOCKNAME, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 32 = getsockname */ { AS(access_args), (sy_call_t *)sys_access, AUE_ACCESS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 33 = access */ { AS(chflags_args), (sy_call_t *)sys_chflags, AUE_CHFLAGS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 34 = chflags */ { AS(fchflags_args), (sy_call_t *)sys_fchflags, AUE_FCHFLAGS, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 35 = fchflags */ { 0, (sy_call_t *)sys_sync, AUE_SYNC, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 36 = sync */ { AS(kill_args), (sy_call_t *)sys_kill, AUE_KILL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 37 = kill */ { compat(AS(ostat_args),stat), AUE_STAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 38 = old stat */ { 0, (sy_call_t *)sys_getppid, AUE_GETPPID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 39 = getppid */ { compat(AS(olstat_args),lstat), AUE_LSTAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 40 = old lstat */ { AS(dup_args), (sy_call_t *)sys_dup, AUE_DUP, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 41 = dup */ { compat10(0,pipe), AUE_PIPE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 42 = freebsd10 pipe */ { 0, (sy_call_t *)sys_getegid, AUE_GETEGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 43 = getegid */ { AS(profil_args), (sy_call_t *)sys_profil, AUE_PROFILE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 44 = profil */ { AS(ktrace_args), (sy_call_t *)sys_ktrace, AUE_KTRACE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 45 = ktrace */ { compat(AS(osigaction_args),sigaction), AUE_SIGACTION, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 46 = old sigaction */ { 0, (sy_call_t *)sys_getgid, AUE_GETGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 47 = getgid */ { compat(AS(osigprocmask_args),sigprocmask), AUE_SIGPROCMASK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 48 = old sigprocmask */ { AS(getlogin_args), (sy_call_t *)sys_getlogin, AUE_GETLOGIN, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 49 = getlogin */ { AS(setlogin_args), (sy_call_t *)sys_setlogin, AUE_SETLOGIN, NULL, 0, 0, 0, SY_THR_STATIC }, /* 50 = setlogin */ { AS(acct_args), (sy_call_t *)sys_acct, AUE_ACCT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 51 = acct */ { compat(0,sigpending), AUE_SIGPENDING, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 52 = old sigpending */ { AS(sigaltstack_args), (sy_call_t *)sys_sigaltstack, AUE_SIGALTSTACK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 53 = sigaltstack */ { AS(ioctl_args), (sy_call_t *)sys_ioctl, AUE_IOCTL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 54 = ioctl */ { AS(reboot_args), (sy_call_t *)sys_reboot, AUE_REBOOT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 55 = reboot */ { AS(revoke_args), (sy_call_t *)sys_revoke, AUE_REVOKE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 56 = revoke */ { AS(symlink_args), (sy_call_t *)sys_symlink, AUE_SYMLINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 57 = symlink */ { AS(readlink_args), (sy_call_t *)sys_readlink, AUE_READLINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 58 = readlink */ { AS(execve_args), (sy_call_t *)sys_execve, AUE_EXECVE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 59 = execve */ { AS(umask_args), (sy_call_t *)sys_umask, AUE_UMASK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 60 = umask */ { AS(chroot_args), (sy_call_t *)sys_chroot, AUE_CHROOT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 61 = chroot */ { compat(AS(ofstat_args),fstat), AUE_FSTAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 62 = old fstat */ { compat(AS(getkerninfo_args),getkerninfo), AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 63 = old getkerninfo */ { compat(0,getpagesize), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 64 = old getpagesize */ { AS(msync_args), (sy_call_t *)sys_msync, AUE_MSYNC, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 65 = msync */ { 0, (sy_call_t *)sys_vfork, AUE_VFORK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 66 = vfork */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 67 = obsolete vread */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 68 = obsolete vwrite */ { AS(sbrk_args), (sy_call_t *)sys_sbrk, AUE_SBRK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 69 = sbrk */ { AS(sstk_args), (sy_call_t *)sys_sstk, AUE_SSTK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 70 = sstk */ { compat(AS(ommap_args),mmap), AUE_MMAP, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 71 = old mmap */ { AS(ovadvise_args), (sy_call_t *)sys_ovadvise, AUE_O_VADVISE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 72 = vadvise */ { AS(munmap_args), (sy_call_t *)sys_munmap, AUE_MUNMAP, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 73 = munmap */ { AS(mprotect_args), (sy_call_t *)sys_mprotect, AUE_MPROTECT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 74 = mprotect */ { AS(madvise_args), (sy_call_t *)sys_madvise, AUE_MADVISE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 75 = madvise */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 76 = obsolete vhangup */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 77 = obsolete vlimit */ { AS(mincore_args), (sy_call_t *)sys_mincore, AUE_MINCORE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 78 = mincore */ { AS(getgroups_args), (sy_call_t *)sys_getgroups, AUE_GETGROUPS, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 79 = getgroups */ { AS(setgroups_args), (sy_call_t *)sys_setgroups, AUE_SETGROUPS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 80 = setgroups */ { 0, (sy_call_t *)sys_getpgrp, AUE_GETPGRP, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 81 = getpgrp */ { AS(setpgid_args), (sy_call_t *)sys_setpgid, AUE_SETPGRP, NULL, 0, 0, 0, SY_THR_STATIC }, /* 82 = setpgid */ { AS(setitimer_args), (sy_call_t *)sys_setitimer, AUE_SETITIMER, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 83 = setitimer */ { compat(0,wait), AUE_WAIT4, NULL, 0, 0, 0, SY_THR_STATIC }, /* 84 = old wait */ { AS(swapon_args), (sy_call_t *)sys_swapon, AUE_SWAPON, NULL, 0, 0, 0, SY_THR_STATIC }, /* 85 = swapon */ { AS(getitimer_args), (sy_call_t *)sys_getitimer, AUE_GETITIMER, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 86 = getitimer */ { compat(AS(gethostname_args),gethostname), AUE_SYSCTL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 87 = old gethostname */ { compat(AS(sethostname_args),sethostname), AUE_SYSCTL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 88 = old sethostname */ - { 0, (sy_call_t *)sys_getdtablesize, AUE_GETDTABLESIZE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 89 = getdtablesize */ + { 0, (sy_call_t *)sys_getdtablesize, AUE_GETDTABLESIZE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 89 = getdtablesize */ { AS(dup2_args), (sy_call_t *)sys_dup2, AUE_DUP2, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 90 = dup2 */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 91 = getdopt */ { AS(fcntl_args), (sy_call_t *)sys_fcntl, AUE_FCNTL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 92 = fcntl */ { AS(select_args), (sy_call_t *)sys_select, AUE_SELECT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 93 = select */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 94 = setdopt */ { AS(fsync_args), (sy_call_t *)sys_fsync, AUE_FSYNC, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 95 = fsync */ { AS(setpriority_args), (sy_call_t *)sys_setpriority, AUE_SETPRIORITY, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 96 = setpriority */ { AS(socket_args), (sy_call_t *)sys_socket, AUE_SOCKET, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 97 = socket */ { AS(connect_args), (sy_call_t *)sys_connect, AUE_CONNECT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 98 = connect */ { compat(AS(accept_args),accept), AUE_ACCEPT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 99 = old accept */ { AS(getpriority_args), (sy_call_t *)sys_getpriority, AUE_GETPRIORITY, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 100 = getpriority */ { compat(AS(osend_args),send), AUE_SEND, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 101 = old send */ { compat(AS(orecv_args),recv), AUE_RECV, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 102 = old recv */ { compat(AS(osigreturn_args),sigreturn), AUE_SIGRETURN, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 103 = old sigreturn */ { AS(bind_args), (sy_call_t *)sys_bind, AUE_BIND, NULL, 0, 0, 0, SY_THR_STATIC }, /* 104 = bind */ { AS(setsockopt_args), (sy_call_t *)sys_setsockopt, AUE_SETSOCKOPT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 105 = setsockopt */ { AS(listen_args), (sy_call_t *)sys_listen, AUE_LISTEN, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 106 = listen */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 107 = obsolete vtimes */ { compat(AS(osigvec_args),sigvec), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 108 = old sigvec */ { compat(AS(osigblock_args),sigblock), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 109 = old sigblock */ { compat(AS(osigsetmask_args),sigsetmask), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 110 = old sigsetmask */ { compat(AS(osigsuspend_args),sigsuspend), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 111 = old sigsuspend */ { compat(AS(osigstack_args),sigstack), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 112 = old sigstack */ { compat(AS(orecvmsg_args),recvmsg), AUE_RECVMSG, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 113 = old recvmsg */ { compat(AS(osendmsg_args),sendmsg), AUE_SENDMSG, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 114 = old sendmsg */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 115 = obsolete vtrace */ { AS(gettimeofday_args), (sy_call_t *)sys_gettimeofday, AUE_GETTIMEOFDAY, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 116 = gettimeofday */ { AS(getrusage_args), (sy_call_t *)sys_getrusage, AUE_GETRUSAGE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 117 = getrusage */ { AS(getsockopt_args), (sy_call_t *)sys_getsockopt, AUE_GETSOCKOPT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 118 = getsockopt */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 119 = resuba */ { AS(readv_args), (sy_call_t *)sys_readv, AUE_READV, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 120 = readv */ { AS(writev_args), (sy_call_t *)sys_writev, AUE_WRITEV, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 121 = writev */ { AS(settimeofday_args), (sy_call_t *)sys_settimeofday, AUE_SETTIMEOFDAY, NULL, 0, 0, 0, SY_THR_STATIC }, /* 122 = settimeofday */ { AS(fchown_args), (sy_call_t *)sys_fchown, AUE_FCHOWN, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 123 = fchown */ { AS(fchmod_args), (sy_call_t *)sys_fchmod, AUE_FCHMOD, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 124 = fchmod */ { compat(AS(recvfrom_args),recvfrom), AUE_RECVFROM, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 125 = old recvfrom */ { AS(setreuid_args), (sy_call_t *)sys_setreuid, AUE_SETREUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 126 = setreuid */ { AS(setregid_args), (sy_call_t *)sys_setregid, AUE_SETREGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 127 = setregid */ { AS(rename_args), (sy_call_t *)sys_rename, AUE_RENAME, NULL, 0, 0, 0, SY_THR_STATIC }, /* 128 = rename */ { compat(AS(otruncate_args),truncate), AUE_TRUNCATE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 129 = old truncate */ { compat(AS(oftruncate_args),ftruncate), AUE_FTRUNCATE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 130 = old ftruncate */ { AS(flock_args), (sy_call_t *)sys_flock, AUE_FLOCK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 131 = flock */ { AS(mkfifo_args), (sy_call_t *)sys_mkfifo, AUE_MKFIFO, NULL, 0, 0, 0, SY_THR_STATIC }, /* 132 = mkfifo */ { AS(sendto_args), (sy_call_t *)sys_sendto, AUE_SENDTO, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 133 = sendto */ { AS(shutdown_args), (sy_call_t *)sys_shutdown, AUE_SHUTDOWN, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 134 = shutdown */ { AS(socketpair_args), (sy_call_t *)sys_socketpair, AUE_SOCKETPAIR, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 135 = socketpair */ { AS(mkdir_args), (sy_call_t *)sys_mkdir, AUE_MKDIR, NULL, 0, 0, 0, SY_THR_STATIC }, /* 136 = mkdir */ { AS(rmdir_args), (sy_call_t *)sys_rmdir, AUE_RMDIR, NULL, 0, 0, 0, SY_THR_STATIC }, /* 137 = rmdir */ { AS(utimes_args), (sy_call_t *)sys_utimes, AUE_UTIMES, NULL, 0, 0, 0, SY_THR_STATIC }, /* 138 = utimes */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 139 = obsolete 4.2 sigreturn */ { AS(adjtime_args), (sy_call_t *)sys_adjtime, AUE_ADJTIME, NULL, 0, 0, 0, SY_THR_STATIC }, /* 140 = adjtime */ { compat(AS(ogetpeername_args),getpeername), AUE_GETPEERNAME, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 141 = old getpeername */ { compat(0,gethostid), AUE_SYSCTL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 142 = old gethostid */ { compat(AS(osethostid_args),sethostid), AUE_SYSCTL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 143 = old sethostid */ { compat(AS(ogetrlimit_args),getrlimit), AUE_GETRLIMIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 144 = old getrlimit */ { compat(AS(osetrlimit_args),setrlimit), AUE_SETRLIMIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 145 = old setrlimit */ { compat(AS(okillpg_args),killpg), AUE_KILLPG, NULL, 0, 0, 0, SY_THR_STATIC }, /* 146 = old killpg */ { 0, (sy_call_t *)sys_setsid, AUE_SETSID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 147 = setsid */ { AS(quotactl_args), (sy_call_t *)sys_quotactl, AUE_QUOTACTL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 148 = quotactl */ { compat(0,quota), AUE_O_QUOTA, NULL, 0, 0, 0, SY_THR_STATIC }, /* 149 = old quota */ { compat(AS(getsockname_args),getsockname), AUE_GETSOCKNAME, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 150 = old getsockname */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 151 = sem_lock */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 152 = sem_wakeup */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 153 = asyncdaemon */ { AS(nlm_syscall_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 154 = nlm_syscall */ { AS(nfssvc_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 155 = nfssvc */ { compat(AS(ogetdirentries_args),getdirentries), AUE_GETDIRENTRIES, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 156 = old getdirentries */ { compat4(AS(freebsd4_statfs_args),statfs), AUE_STATFS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 157 = freebsd4 statfs */ { compat4(AS(freebsd4_fstatfs_args),fstatfs), AUE_FSTATFS, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 158 = freebsd4 fstatfs */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 159 = nosys */ { AS(lgetfh_args), (sy_call_t *)sys_lgetfh, AUE_LGETFH, NULL, 0, 0, 0, SY_THR_STATIC }, /* 160 = lgetfh */ { AS(getfh_args), (sy_call_t *)sys_getfh, AUE_NFS_GETFH, NULL, 0, 0, 0, SY_THR_STATIC }, /* 161 = getfh */ { compat4(AS(freebsd4_getdomainname_args),getdomainname), AUE_SYSCTL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 162 = freebsd4 getdomainname */ { compat4(AS(freebsd4_setdomainname_args),setdomainname), AUE_SYSCTL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 163 = freebsd4 setdomainname */ { compat4(AS(freebsd4_uname_args),uname), AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 164 = freebsd4 uname */ { AS(sysarch_args), (sy_call_t *)sysarch, AUE_SYSARCH, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 165 = sysarch */ { AS(rtprio_args), (sy_call_t *)sys_rtprio, AUE_RTPRIO, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 166 = rtprio */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 167 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 168 = nosys */ { AS(semsys_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 169 = semsys */ { AS(msgsys_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 170 = msgsys */ { AS(shmsys_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 171 = shmsys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 172 = nosys */ { compat6(AS(freebsd6_pread_args),pread), AUE_PREAD, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 173 = freebsd6 pread */ { compat6(AS(freebsd6_pwrite_args),pwrite), AUE_PWRITE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 174 = freebsd6 pwrite */ { AS(setfib_args), (sy_call_t *)sys_setfib, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 175 = setfib */ { AS(ntp_adjtime_args), (sy_call_t *)sys_ntp_adjtime, AUE_NTP_ADJTIME, NULL, 0, 0, 0, SY_THR_STATIC }, /* 176 = ntp_adjtime */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 177 = sfork */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 178 = getdescriptor */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 179 = setdescriptor */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 180 = nosys */ { AS(setgid_args), (sy_call_t *)sys_setgid, AUE_SETGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 181 = setgid */ { AS(setegid_args), (sy_call_t *)sys_setegid, AUE_SETEGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 182 = setegid */ { AS(seteuid_args), (sy_call_t *)sys_seteuid, AUE_SETEUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 183 = seteuid */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 184 = lfs_bmapv */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 185 = lfs_markv */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 186 = lfs_segclean */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 187 = lfs_segwait */ { AS(stat_args), (sy_call_t *)sys_stat, AUE_STAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 188 = stat */ { AS(fstat_args), (sy_call_t *)sys_fstat, AUE_FSTAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 189 = fstat */ { AS(lstat_args), (sy_call_t *)sys_lstat, AUE_LSTAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 190 = lstat */ { AS(pathconf_args), (sy_call_t *)sys_pathconf, AUE_PATHCONF, NULL, 0, 0, 0, SY_THR_STATIC }, /* 191 = pathconf */ { AS(fpathconf_args), (sy_call_t *)sys_fpathconf, AUE_FPATHCONF, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 192 = fpathconf */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 193 = nosys */ { AS(__getrlimit_args), (sy_call_t *)sys_getrlimit, AUE_GETRLIMIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 194 = getrlimit */ { AS(__setrlimit_args), (sy_call_t *)sys_setrlimit, AUE_SETRLIMIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 195 = setrlimit */ { AS(getdirentries_args), (sy_call_t *)sys_getdirentries, AUE_GETDIRENTRIES, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 196 = getdirentries */ { compat6(AS(freebsd6_mmap_args),mmap), AUE_MMAP, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 197 = freebsd6 mmap */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 198 = __syscall */ { compat6(AS(freebsd6_lseek_args),lseek), AUE_LSEEK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 199 = freebsd6 lseek */ { compat6(AS(freebsd6_truncate_args),truncate), AUE_TRUNCATE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 200 = freebsd6 truncate */ { compat6(AS(freebsd6_ftruncate_args),ftruncate), AUE_FTRUNCATE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 201 = freebsd6 ftruncate */ { AS(sysctl_args), (sy_call_t *)sys___sysctl, AUE_SYSCTL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 202 = __sysctl */ { AS(mlock_args), (sy_call_t *)sys_mlock, AUE_MLOCK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 203 = mlock */ { AS(munlock_args), (sy_call_t *)sys_munlock, AUE_MUNLOCK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 204 = munlock */ { AS(undelete_args), (sy_call_t *)sys_undelete, AUE_UNDELETE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 205 = undelete */ { AS(futimes_args), (sy_call_t *)sys_futimes, AUE_FUTIMES, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 206 = futimes */ { AS(getpgid_args), (sy_call_t *)sys_getpgid, AUE_GETPGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 207 = getpgid */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 208 = newreboot */ { AS(poll_args), (sy_call_t *)sys_poll, AUE_POLL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 209 = poll */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 210 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 211 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 212 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 213 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 214 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 215 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 216 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 217 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 218 = lkmnosys */ { AS(nosys_args), (sy_call_t *)lkmnosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 219 = lkmnosys */ { 0, (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 220 = freebsd7 __semctl */ { AS(semget_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 221 = semget */ { AS(semop_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 222 = semop */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 223 = semconfig */ { 0, (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 224 = freebsd7 msgctl */ { AS(msgget_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 225 = msgget */ { AS(msgsnd_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 226 = msgsnd */ { AS(msgrcv_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 227 = msgrcv */ { AS(shmat_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 228 = shmat */ { 0, (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 229 = freebsd7 shmctl */ { AS(shmdt_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 230 = shmdt */ { AS(shmget_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 231 = shmget */ { AS(clock_gettime_args), (sy_call_t *)sys_clock_gettime, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 232 = clock_gettime */ { AS(clock_settime_args), (sy_call_t *)sys_clock_settime, AUE_CLOCK_SETTIME, NULL, 0, 0, 0, SY_THR_STATIC }, /* 233 = clock_settime */ { AS(clock_getres_args), (sy_call_t *)sys_clock_getres, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 234 = clock_getres */ { AS(ktimer_create_args), (sy_call_t *)sys_ktimer_create, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 235 = ktimer_create */ { AS(ktimer_delete_args), (sy_call_t *)sys_ktimer_delete, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 236 = ktimer_delete */ { AS(ktimer_settime_args), (sy_call_t *)sys_ktimer_settime, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 237 = ktimer_settime */ { AS(ktimer_gettime_args), (sy_call_t *)sys_ktimer_gettime, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 238 = ktimer_gettime */ { AS(ktimer_getoverrun_args), (sy_call_t *)sys_ktimer_getoverrun, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 239 = ktimer_getoverrun */ { AS(nanosleep_args), (sy_call_t *)sys_nanosleep, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 240 = nanosleep */ { AS(ffclock_getcounter_args), (sy_call_t *)sys_ffclock_getcounter, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 241 = ffclock_getcounter */ { AS(ffclock_setestimate_args), (sy_call_t *)sys_ffclock_setestimate, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 242 = ffclock_setestimate */ { AS(ffclock_getestimate_args), (sy_call_t *)sys_ffclock_getestimate, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 243 = ffclock_getestimate */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 244 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 245 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 246 = nosys */ { AS(clock_getcpuclockid2_args), (sy_call_t *)sys_clock_getcpuclockid2, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 247 = clock_getcpuclockid2 */ { AS(ntp_gettime_args), (sy_call_t *)sys_ntp_gettime, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 248 = ntp_gettime */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 249 = nosys */ { AS(minherit_args), (sy_call_t *)sys_minherit, AUE_MINHERIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 250 = minherit */ { AS(rfork_args), (sy_call_t *)sys_rfork, AUE_RFORK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 251 = rfork */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 252 = obsolete openbsd_poll */ { 0, (sy_call_t *)sys_issetugid, AUE_ISSETUGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 253 = issetugid */ { AS(lchown_args), (sy_call_t *)sys_lchown, AUE_LCHOWN, NULL, 0, 0, 0, SY_THR_STATIC }, /* 254 = lchown */ { AS(aio_read_args), (sy_call_t *)sys_aio_read, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 255 = aio_read */ { AS(aio_write_args), (sy_call_t *)sys_aio_write, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 256 = aio_write */ { AS(lio_listio_args), (sy_call_t *)sys_lio_listio, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 257 = lio_listio */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 258 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 259 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 260 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 261 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 262 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 263 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 264 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 265 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 266 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 267 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 268 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 269 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 270 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 271 = nosys */ { AS(getdents_args), (sy_call_t *)sys_getdents, AUE_O_GETDENTS, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 272 = getdents */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 273 = nosys */ { AS(lchmod_args), (sy_call_t *)sys_lchmod, AUE_LCHMOD, NULL, 0, 0, 0, SY_THR_STATIC }, /* 274 = lchmod */ { AS(lchown_args), (sy_call_t *)sys_lchown, AUE_LCHOWN, NULL, 0, 0, 0, SY_THR_STATIC }, /* 275 = netbsd_lchown */ { AS(lutimes_args), (sy_call_t *)sys_lutimes, AUE_LUTIMES, NULL, 0, 0, 0, SY_THR_STATIC }, /* 276 = lutimes */ { AS(msync_args), (sy_call_t *)sys_msync, AUE_MSYNC, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 277 = netbsd_msync */ { AS(nstat_args), (sy_call_t *)sys_nstat, AUE_STAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 278 = nstat */ { AS(nfstat_args), (sy_call_t *)sys_nfstat, AUE_FSTAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 279 = nfstat */ { AS(nlstat_args), (sy_call_t *)sys_nlstat, AUE_LSTAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 280 = nlstat */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 281 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 282 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 283 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 284 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 285 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 286 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 287 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 288 = nosys */ { AS(preadv_args), (sy_call_t *)sys_preadv, AUE_PREADV, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 289 = preadv */ { AS(pwritev_args), (sy_call_t *)sys_pwritev, AUE_PWRITEV, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 290 = pwritev */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 291 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 292 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 293 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 294 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 295 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 296 = nosys */ { compat4(AS(freebsd4_fhstatfs_args),fhstatfs), AUE_FHSTATFS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 297 = freebsd4 fhstatfs */ { AS(fhopen_args), (sy_call_t *)sys_fhopen, AUE_FHOPEN, NULL, 0, 0, 0, SY_THR_STATIC }, /* 298 = fhopen */ { AS(fhstat_args), (sy_call_t *)sys_fhstat, AUE_FHSTAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 299 = fhstat */ { AS(modnext_args), (sy_call_t *)sys_modnext, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 300 = modnext */ { AS(modstat_args), (sy_call_t *)sys_modstat, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 301 = modstat */ { AS(modfnext_args), (sy_call_t *)sys_modfnext, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 302 = modfnext */ { AS(modfind_args), (sy_call_t *)sys_modfind, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 303 = modfind */ { AS(kldload_args), (sy_call_t *)sys_kldload, AUE_MODLOAD, NULL, 0, 0, 0, SY_THR_STATIC }, /* 304 = kldload */ { AS(kldunload_args), (sy_call_t *)sys_kldunload, AUE_MODUNLOAD, NULL, 0, 0, 0, SY_THR_STATIC }, /* 305 = kldunload */ { AS(kldfind_args), (sy_call_t *)sys_kldfind, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 306 = kldfind */ { AS(kldnext_args), (sy_call_t *)sys_kldnext, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 307 = kldnext */ { AS(kldstat_args), (sy_call_t *)sys_kldstat, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 308 = kldstat */ { AS(kldfirstmod_args), (sy_call_t *)sys_kldfirstmod, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 309 = kldfirstmod */ { AS(getsid_args), (sy_call_t *)sys_getsid, AUE_GETSID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 310 = getsid */ { AS(setresuid_args), (sy_call_t *)sys_setresuid, AUE_SETRESUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 311 = setresuid */ { AS(setresgid_args), (sy_call_t *)sys_setresgid, AUE_SETRESGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 312 = setresgid */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 313 = obsolete signanosleep */ { AS(aio_return_args), (sy_call_t *)sys_aio_return, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 314 = aio_return */ { AS(aio_suspend_args), (sy_call_t *)sys_aio_suspend, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 315 = aio_suspend */ { AS(aio_cancel_args), (sy_call_t *)sys_aio_cancel, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 316 = aio_cancel */ { AS(aio_error_args), (sy_call_t *)sys_aio_error, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 317 = aio_error */ { compat6(AS(freebsd6_aio_read_args),aio_read), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 318 = freebsd6 aio_read */ { compat6(AS(freebsd6_aio_write_args),aio_write), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 319 = freebsd6 aio_write */ { compat6(AS(freebsd6_lio_listio_args),lio_listio), AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 320 = freebsd6 lio_listio */ { 0, (sy_call_t *)sys_yield, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 321 = yield */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 322 = obsolete thr_sleep */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 323 = obsolete thr_wakeup */ { AS(mlockall_args), (sy_call_t *)sys_mlockall, AUE_MLOCKALL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 324 = mlockall */ { 0, (sy_call_t *)sys_munlockall, AUE_MUNLOCKALL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 325 = munlockall */ { AS(__getcwd_args), (sy_call_t *)sys___getcwd, AUE_GETCWD, NULL, 0, 0, 0, SY_THR_STATIC }, /* 326 = __getcwd */ { AS(sched_setparam_args), (sy_call_t *)sys_sched_setparam, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 327 = sched_setparam */ { AS(sched_getparam_args), (sy_call_t *)sys_sched_getparam, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 328 = sched_getparam */ { AS(sched_setscheduler_args), (sy_call_t *)sys_sched_setscheduler, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 329 = sched_setscheduler */ { AS(sched_getscheduler_args), (sy_call_t *)sys_sched_getscheduler, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 330 = sched_getscheduler */ { 0, (sy_call_t *)sys_sched_yield, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 331 = sched_yield */ { AS(sched_get_priority_max_args), (sy_call_t *)sys_sched_get_priority_max, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 332 = sched_get_priority_max */ { AS(sched_get_priority_min_args), (sy_call_t *)sys_sched_get_priority_min, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 333 = sched_get_priority_min */ { AS(sched_rr_get_interval_args), (sy_call_t *)sys_sched_rr_get_interval, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 334 = sched_rr_get_interval */ { AS(utrace_args), (sy_call_t *)sys_utrace, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 335 = utrace */ { compat4(AS(freebsd4_sendfile_args),sendfile), AUE_SENDFILE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 336 = freebsd4 sendfile */ { AS(kldsym_args), (sy_call_t *)sys_kldsym, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 337 = kldsym */ { AS(jail_args), (sy_call_t *)sys_jail, AUE_JAIL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 338 = jail */ { AS(nnpfs_syscall_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 339 = nnpfs_syscall */ { AS(sigprocmask_args), (sy_call_t *)sys_sigprocmask, AUE_SIGPROCMASK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 340 = sigprocmask */ { AS(sigsuspend_args), (sy_call_t *)sys_sigsuspend, AUE_SIGSUSPEND, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 341 = sigsuspend */ { compat4(AS(freebsd4_sigaction_args),sigaction), AUE_SIGACTION, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 342 = freebsd4 sigaction */ { AS(sigpending_args), (sy_call_t *)sys_sigpending, AUE_SIGPENDING, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 343 = sigpending */ { compat4(AS(freebsd4_sigreturn_args),sigreturn), AUE_SIGRETURN, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 344 = freebsd4 sigreturn */ { AS(sigtimedwait_args), (sy_call_t *)sys_sigtimedwait, AUE_SIGWAIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 345 = sigtimedwait */ { AS(sigwaitinfo_args), (sy_call_t *)sys_sigwaitinfo, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 346 = sigwaitinfo */ { AS(__acl_get_file_args), (sy_call_t *)sys___acl_get_file, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 347 = __acl_get_file */ { AS(__acl_set_file_args), (sy_call_t *)sys___acl_set_file, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 348 = __acl_set_file */ { AS(__acl_get_fd_args), (sy_call_t *)sys___acl_get_fd, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 349 = __acl_get_fd */ { AS(__acl_set_fd_args), (sy_call_t *)sys___acl_set_fd, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 350 = __acl_set_fd */ { AS(__acl_delete_file_args), (sy_call_t *)sys___acl_delete_file, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 351 = __acl_delete_file */ { AS(__acl_delete_fd_args), (sy_call_t *)sys___acl_delete_fd, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 352 = __acl_delete_fd */ { AS(__acl_aclcheck_file_args), (sy_call_t *)sys___acl_aclcheck_file, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 353 = __acl_aclcheck_file */ { AS(__acl_aclcheck_fd_args), (sy_call_t *)sys___acl_aclcheck_fd, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 354 = __acl_aclcheck_fd */ { AS(extattrctl_args), (sy_call_t *)sys_extattrctl, AUE_EXTATTRCTL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 355 = extattrctl */ { AS(extattr_set_file_args), (sy_call_t *)sys_extattr_set_file, AUE_EXTATTR_SET_FILE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 356 = extattr_set_file */ { AS(extattr_get_file_args), (sy_call_t *)sys_extattr_get_file, AUE_EXTATTR_GET_FILE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 357 = extattr_get_file */ { AS(extattr_delete_file_args), (sy_call_t *)sys_extattr_delete_file, AUE_EXTATTR_DELETE_FILE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 358 = extattr_delete_file */ { AS(aio_waitcomplete_args), (sy_call_t *)sys_aio_waitcomplete, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 359 = aio_waitcomplete */ { AS(getresuid_args), (sy_call_t *)sys_getresuid, AUE_GETRESUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 360 = getresuid */ { AS(getresgid_args), (sy_call_t *)sys_getresgid, AUE_GETRESGID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 361 = getresgid */ { 0, (sy_call_t *)sys_kqueue, AUE_KQUEUE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 362 = kqueue */ { AS(kevent_args), (sy_call_t *)sys_kevent, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 363 = kevent */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 364 = __cap_get_proc */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 365 = __cap_set_proc */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 366 = __cap_get_fd */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 367 = __cap_get_file */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 368 = __cap_set_fd */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 369 = __cap_set_file */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 370 = nosys */ { AS(extattr_set_fd_args), (sy_call_t *)sys_extattr_set_fd, AUE_EXTATTR_SET_FD, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 371 = extattr_set_fd */ { AS(extattr_get_fd_args), (sy_call_t *)sys_extattr_get_fd, AUE_EXTATTR_GET_FD, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 372 = extattr_get_fd */ { AS(extattr_delete_fd_args), (sy_call_t *)sys_extattr_delete_fd, AUE_EXTATTR_DELETE_FD, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 373 = extattr_delete_fd */ { AS(__setugid_args), (sy_call_t *)sys___setugid, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 374 = __setugid */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 375 = nfsclnt */ { AS(eaccess_args), (sy_call_t *)sys_eaccess, AUE_EACCESS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 376 = eaccess */ { AS(afs3_syscall_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 377 = afs3_syscall */ { AS(nmount_args), (sy_call_t *)sys_nmount, AUE_NMOUNT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 378 = nmount */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 379 = kse_exit */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 380 = kse_wakeup */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 381 = kse_create */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 382 = kse_thr_interrupt */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 383 = kse_release */ { AS(__mac_get_proc_args), (sy_call_t *)sys___mac_get_proc, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 384 = __mac_get_proc */ { AS(__mac_set_proc_args), (sy_call_t *)sys___mac_set_proc, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 385 = __mac_set_proc */ { AS(__mac_get_fd_args), (sy_call_t *)sys___mac_get_fd, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 386 = __mac_get_fd */ { AS(__mac_get_file_args), (sy_call_t *)sys___mac_get_file, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 387 = __mac_get_file */ { AS(__mac_set_fd_args), (sy_call_t *)sys___mac_set_fd, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 388 = __mac_set_fd */ { AS(__mac_set_file_args), (sy_call_t *)sys___mac_set_file, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 389 = __mac_set_file */ { AS(kenv_args), (sy_call_t *)sys_kenv, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 390 = kenv */ { AS(lchflags_args), (sy_call_t *)sys_lchflags, AUE_LCHFLAGS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 391 = lchflags */ { AS(uuidgen_args), (sy_call_t *)sys_uuidgen, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 392 = uuidgen */ { AS(sendfile_args), (sy_call_t *)sys_sendfile, AUE_SENDFILE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 393 = sendfile */ { AS(mac_syscall_args), (sy_call_t *)sys_mac_syscall, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 394 = mac_syscall */ { AS(getfsstat_args), (sy_call_t *)sys_getfsstat, AUE_GETFSSTAT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 395 = getfsstat */ { AS(statfs_args), (sy_call_t *)sys_statfs, AUE_STATFS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 396 = statfs */ { AS(fstatfs_args), (sy_call_t *)sys_fstatfs, AUE_FSTATFS, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 397 = fstatfs */ { AS(fhstatfs_args), (sy_call_t *)sys_fhstatfs, AUE_FHSTATFS, NULL, 0, 0, 0, SY_THR_STATIC }, /* 398 = fhstatfs */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 399 = nosys */ { AS(ksem_close_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 400 = ksem_close */ { AS(ksem_post_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 401 = ksem_post */ { AS(ksem_wait_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 402 = ksem_wait */ { AS(ksem_trywait_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 403 = ksem_trywait */ { AS(ksem_init_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 404 = ksem_init */ { AS(ksem_open_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 405 = ksem_open */ { AS(ksem_unlink_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 406 = ksem_unlink */ { AS(ksem_getvalue_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 407 = ksem_getvalue */ { AS(ksem_destroy_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 408 = ksem_destroy */ { AS(__mac_get_pid_args), (sy_call_t *)sys___mac_get_pid, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 409 = __mac_get_pid */ { AS(__mac_get_link_args), (sy_call_t *)sys___mac_get_link, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 410 = __mac_get_link */ { AS(__mac_set_link_args), (sy_call_t *)sys___mac_set_link, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 411 = __mac_set_link */ { AS(extattr_set_link_args), (sy_call_t *)sys_extattr_set_link, AUE_EXTATTR_SET_LINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 412 = extattr_set_link */ { AS(extattr_get_link_args), (sy_call_t *)sys_extattr_get_link, AUE_EXTATTR_GET_LINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 413 = extattr_get_link */ { AS(extattr_delete_link_args), (sy_call_t *)sys_extattr_delete_link, AUE_EXTATTR_DELETE_LINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 414 = extattr_delete_link */ { AS(__mac_execve_args), (sy_call_t *)sys___mac_execve, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 415 = __mac_execve */ { AS(sigaction_args), (sy_call_t *)sys_sigaction, AUE_SIGACTION, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 416 = sigaction */ { AS(sigreturn_args), (sy_call_t *)sys_sigreturn, AUE_SIGRETURN, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 417 = sigreturn */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 418 = __xstat */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 419 = __xfstat */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 420 = __xlstat */ { AS(getcontext_args), (sy_call_t *)sys_getcontext, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 421 = getcontext */ { AS(setcontext_args), (sy_call_t *)sys_setcontext, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 422 = setcontext */ { AS(swapcontext_args), (sy_call_t *)sys_swapcontext, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 423 = swapcontext */ { AS(swapoff_args), (sy_call_t *)sys_swapoff, AUE_SWAPOFF, NULL, 0, 0, 0, SY_THR_STATIC }, /* 424 = swapoff */ { AS(__acl_get_link_args), (sy_call_t *)sys___acl_get_link, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 425 = __acl_get_link */ { AS(__acl_set_link_args), (sy_call_t *)sys___acl_set_link, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 426 = __acl_set_link */ { AS(__acl_delete_link_args), (sy_call_t *)sys___acl_delete_link, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 427 = __acl_delete_link */ { AS(__acl_aclcheck_link_args), (sy_call_t *)sys___acl_aclcheck_link, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 428 = __acl_aclcheck_link */ { AS(sigwait_args), (sy_call_t *)sys_sigwait, AUE_SIGWAIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 429 = sigwait */ { AS(thr_create_args), (sy_call_t *)sys_thr_create, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 430 = thr_create */ { AS(thr_exit_args), (sy_call_t *)sys_thr_exit, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 431 = thr_exit */ { AS(thr_self_args), (sy_call_t *)sys_thr_self, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 432 = thr_self */ { AS(thr_kill_args), (sy_call_t *)sys_thr_kill, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 433 = thr_kill */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 434 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 435 = nosys */ { AS(jail_attach_args), (sy_call_t *)sys_jail_attach, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 436 = jail_attach */ { AS(extattr_list_fd_args), (sy_call_t *)sys_extattr_list_fd, AUE_EXTATTR_LIST_FD, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 437 = extattr_list_fd */ { AS(extattr_list_file_args), (sy_call_t *)sys_extattr_list_file, AUE_EXTATTR_LIST_FILE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 438 = extattr_list_file */ { AS(extattr_list_link_args), (sy_call_t *)sys_extattr_list_link, AUE_EXTATTR_LIST_LINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 439 = extattr_list_link */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 440 = kse_switchin */ { AS(ksem_timedwait_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 441 = ksem_timedwait */ { AS(thr_suspend_args), (sy_call_t *)sys_thr_suspend, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 442 = thr_suspend */ { AS(thr_wake_args), (sy_call_t *)sys_thr_wake, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 443 = thr_wake */ { AS(kldunloadf_args), (sy_call_t *)sys_kldunloadf, AUE_MODUNLOAD, NULL, 0, 0, 0, SY_THR_STATIC }, /* 444 = kldunloadf */ { AS(audit_args), (sy_call_t *)sys_audit, AUE_AUDIT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 445 = audit */ { AS(auditon_args), (sy_call_t *)sys_auditon, AUE_AUDITON, NULL, 0, 0, 0, SY_THR_STATIC }, /* 446 = auditon */ { AS(getauid_args), (sy_call_t *)sys_getauid, AUE_GETAUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 447 = getauid */ { AS(setauid_args), (sy_call_t *)sys_setauid, AUE_SETAUID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 448 = setauid */ { AS(getaudit_args), (sy_call_t *)sys_getaudit, AUE_GETAUDIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 449 = getaudit */ { AS(setaudit_args), (sy_call_t *)sys_setaudit, AUE_SETAUDIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 450 = setaudit */ { AS(getaudit_addr_args), (sy_call_t *)sys_getaudit_addr, AUE_GETAUDIT_ADDR, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 451 = getaudit_addr */ { AS(setaudit_addr_args), (sy_call_t *)sys_setaudit_addr, AUE_SETAUDIT_ADDR, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 452 = setaudit_addr */ { AS(auditctl_args), (sy_call_t *)sys_auditctl, AUE_AUDITCTL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 453 = auditctl */ { AS(_umtx_op_args), (sy_call_t *)sys__umtx_op, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 454 = _umtx_op */ { AS(thr_new_args), (sy_call_t *)sys_thr_new, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 455 = thr_new */ { AS(sigqueue_args), (sy_call_t *)sys_sigqueue, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 456 = sigqueue */ { AS(kmq_open_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 457 = kmq_open */ { AS(kmq_setattr_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_ABSENT }, /* 458 = kmq_setattr */ { AS(kmq_timedreceive_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_ABSENT }, /* 459 = kmq_timedreceive */ { AS(kmq_timedsend_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_ABSENT }, /* 460 = kmq_timedsend */ { AS(kmq_notify_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_ABSENT }, /* 461 = kmq_notify */ { AS(kmq_unlink_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 462 = kmq_unlink */ { AS(abort2_args), (sy_call_t *)sys_abort2, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 463 = abort2 */ { AS(thr_set_name_args), (sy_call_t *)sys_thr_set_name, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 464 = thr_set_name */ { AS(aio_fsync_args), (sy_call_t *)sys_aio_fsync, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 465 = aio_fsync */ { AS(rtprio_thread_args), (sy_call_t *)sys_rtprio_thread, AUE_RTPRIO, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 466 = rtprio_thread */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 467 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 468 = nosys */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 469 = __getpath_fromfd */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 470 = __getpath_fromaddr */ { AS(sctp_peeloff_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_ABSENT }, /* 471 = sctp_peeloff */ { AS(sctp_generic_sendmsg_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_ABSENT }, /* 472 = sctp_generic_sendmsg */ { AS(sctp_generic_sendmsg_iov_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_ABSENT }, /* 473 = sctp_generic_sendmsg_iov */ { AS(sctp_generic_recvmsg_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_ABSENT }, /* 474 = sctp_generic_recvmsg */ { AS(pread_args), (sy_call_t *)sys_pread, AUE_PREAD, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 475 = pread */ { AS(pwrite_args), (sy_call_t *)sys_pwrite, AUE_PWRITE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 476 = pwrite */ { AS(mmap_args), (sy_call_t *)sys_mmap, AUE_MMAP, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 477 = mmap */ { AS(lseek_args), (sy_call_t *)sys_lseek, AUE_LSEEK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 478 = lseek */ { AS(truncate_args), (sy_call_t *)sys_truncate, AUE_TRUNCATE, NULL, 0, 0, 0, SY_THR_STATIC }, /* 479 = truncate */ { AS(ftruncate_args), (sy_call_t *)sys_ftruncate, AUE_FTRUNCATE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 480 = ftruncate */ { AS(thr_kill2_args), (sy_call_t *)sys_thr_kill2, AUE_KILL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 481 = thr_kill2 */ { AS(shm_open_args), (sy_call_t *)sys_shm_open, AUE_SHMOPEN, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 482 = shm_open */ { AS(shm_unlink_args), (sy_call_t *)sys_shm_unlink, AUE_SHMUNLINK, NULL, 0, 0, 0, SY_THR_STATIC }, /* 483 = shm_unlink */ { AS(cpuset_args), (sy_call_t *)sys_cpuset, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 484 = cpuset */ { AS(cpuset_setid_args), (sy_call_t *)sys_cpuset_setid, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 485 = cpuset_setid */ { AS(cpuset_getid_args), (sy_call_t *)sys_cpuset_getid, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 486 = cpuset_getid */ { AS(cpuset_getaffinity_args), (sy_call_t *)sys_cpuset_getaffinity, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 487 = cpuset_getaffinity */ { AS(cpuset_setaffinity_args), (sy_call_t *)sys_cpuset_setaffinity, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 488 = cpuset_setaffinity */ { AS(faccessat_args), (sy_call_t *)sys_faccessat, AUE_FACCESSAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 489 = faccessat */ { AS(fchmodat_args), (sy_call_t *)sys_fchmodat, AUE_FCHMODAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 490 = fchmodat */ { AS(fchownat_args), (sy_call_t *)sys_fchownat, AUE_FCHOWNAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 491 = fchownat */ { AS(fexecve_args), (sy_call_t *)sys_fexecve, AUE_FEXECVE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 492 = fexecve */ { AS(fstatat_args), (sy_call_t *)sys_fstatat, AUE_FSTATAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 493 = fstatat */ { AS(futimesat_args), (sy_call_t *)sys_futimesat, AUE_FUTIMESAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 494 = futimesat */ { AS(linkat_args), (sy_call_t *)sys_linkat, AUE_LINKAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 495 = linkat */ { AS(mkdirat_args), (sy_call_t *)sys_mkdirat, AUE_MKDIRAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 496 = mkdirat */ { AS(mkfifoat_args), (sy_call_t *)sys_mkfifoat, AUE_MKFIFOAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 497 = mkfifoat */ { AS(mknodat_args), (sy_call_t *)sys_mknodat, AUE_MKNODAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 498 = mknodat */ { AS(openat_args), (sy_call_t *)sys_openat, AUE_OPENAT_RWTC, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 499 = openat */ { AS(readlinkat_args), (sy_call_t *)sys_readlinkat, AUE_READLINKAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 500 = readlinkat */ { AS(renameat_args), (sy_call_t *)sys_renameat, AUE_RENAMEAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 501 = renameat */ { AS(symlinkat_args), (sy_call_t *)sys_symlinkat, AUE_SYMLINKAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 502 = symlinkat */ { AS(unlinkat_args), (sy_call_t *)sys_unlinkat, AUE_UNLINKAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 503 = unlinkat */ { AS(posix_openpt_args), (sy_call_t *)sys_posix_openpt, AUE_POSIX_OPENPT, NULL, 0, 0, 0, SY_THR_STATIC }, /* 504 = posix_openpt */ { AS(gssd_syscall_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 505 = gssd_syscall */ { AS(jail_get_args), (sy_call_t *)sys_jail_get, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 506 = jail_get */ { AS(jail_set_args), (sy_call_t *)sys_jail_set, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 507 = jail_set */ { AS(jail_remove_args), (sy_call_t *)sys_jail_remove, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 508 = jail_remove */ { AS(closefrom_args), (sy_call_t *)sys_closefrom, AUE_CLOSEFROM, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 509 = closefrom */ { AS(__semctl_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 510 = __semctl */ { AS(msgctl_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 511 = msgctl */ { AS(shmctl_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 512 = shmctl */ { AS(lpathconf_args), (sy_call_t *)sys_lpathconf, AUE_LPATHCONF, NULL, 0, 0, 0, SY_THR_STATIC }, /* 513 = lpathconf */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 514 = obsolete cap_new */ { AS(__cap_rights_get_args), (sy_call_t *)sys___cap_rights_get, AUE_CAP_RIGHTS_GET, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 515 = __cap_rights_get */ { 0, (sy_call_t *)sys_cap_enter, AUE_CAP_ENTER, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 516 = cap_enter */ { AS(cap_getmode_args), (sy_call_t *)sys_cap_getmode, AUE_CAP_GETMODE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 517 = cap_getmode */ { AS(pdfork_args), (sy_call_t *)sys_pdfork, AUE_PDFORK, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 518 = pdfork */ { AS(pdkill_args), (sy_call_t *)sys_pdkill, AUE_PDKILL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 519 = pdkill */ { AS(pdgetpid_args), (sy_call_t *)sys_pdgetpid, AUE_PDGETPID, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 520 = pdgetpid */ { 0, (sy_call_t *)nosys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 521 = pdwait4 */ { AS(pselect_args), (sy_call_t *)sys_pselect, AUE_SELECT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 522 = pselect */ { AS(getloginclass_args), (sy_call_t *)sys_getloginclass, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 523 = getloginclass */ { AS(setloginclass_args), (sy_call_t *)sys_setloginclass, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 524 = setloginclass */ { AS(rctl_get_racct_args), (sy_call_t *)sys_rctl_get_racct, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 525 = rctl_get_racct */ { AS(rctl_get_rules_args), (sy_call_t *)sys_rctl_get_rules, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 526 = rctl_get_rules */ { AS(rctl_get_limits_args), (sy_call_t *)sys_rctl_get_limits, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 527 = rctl_get_limits */ { AS(rctl_add_rule_args), (sy_call_t *)sys_rctl_add_rule, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 528 = rctl_add_rule */ { AS(rctl_remove_rule_args), (sy_call_t *)sys_rctl_remove_rule, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 529 = rctl_remove_rule */ { AS(posix_fallocate_args), (sy_call_t *)sys_posix_fallocate, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 530 = posix_fallocate */ { AS(posix_fadvise_args), (sy_call_t *)sys_posix_fadvise, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 531 = posix_fadvise */ { AS(wait6_args), (sy_call_t *)sys_wait6, AUE_WAIT6, NULL, 0, 0, 0, SY_THR_STATIC }, /* 532 = wait6 */ { AS(cap_rights_limit_args), (sy_call_t *)sys_cap_rights_limit, AUE_CAP_RIGHTS_LIMIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 533 = cap_rights_limit */ { AS(cap_ioctls_limit_args), (sy_call_t *)sys_cap_ioctls_limit, AUE_CAP_IOCTLS_LIMIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 534 = cap_ioctls_limit */ { AS(cap_ioctls_get_args), (sy_call_t *)sys_cap_ioctls_get, AUE_CAP_IOCTLS_GET, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 535 = cap_ioctls_get */ { AS(cap_fcntls_limit_args), (sy_call_t *)sys_cap_fcntls_limit, AUE_CAP_FCNTLS_LIMIT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 536 = cap_fcntls_limit */ { AS(cap_fcntls_get_args), (sy_call_t *)sys_cap_fcntls_get, AUE_CAP_FCNTLS_GET, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 537 = cap_fcntls_get */ { AS(bindat_args), (sy_call_t *)sys_bindat, AUE_BINDAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 538 = bindat */ { AS(connectat_args), (sy_call_t *)sys_connectat, AUE_CONNECTAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 539 = connectat */ { AS(chflagsat_args), (sy_call_t *)sys_chflagsat, AUE_CHFLAGSAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 540 = chflagsat */ { AS(accept4_args), (sy_call_t *)sys_accept4, AUE_ACCEPT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 541 = accept4 */ { AS(pipe2_args), (sy_call_t *)sys_pipe2, AUE_PIPE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 542 = pipe2 */ { AS(aio_mlock_args), (sy_call_t *)sys_aio_mlock, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 543 = aio_mlock */ { AS(procctl_args), (sy_call_t *)sys_procctl, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 544 = procctl */ { AS(ppoll_args), (sy_call_t *)sys_ppoll, AUE_POLL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 545 = ppoll */ { AS(futimens_args), (sy_call_t *)sys_futimens, AUE_FUTIMES, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 546 = futimens */ { AS(utimensat_args), (sy_call_t *)sys_utimensat, AUE_FUTIMESAT, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 547 = utimensat */ { AS(numa_getaffinity_args), (sy_call_t *)sys_numa_getaffinity, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 548 = numa_getaffinity */ { AS(numa_setaffinity_args), (sy_call_t *)sys_numa_setaffinity, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 549 = numa_setaffinity */ { AS(fdatasync_args), (sy_call_t *)sys_fdatasync, AUE_FSYNC, NULL, 0, 0, 0, SY_THR_STATIC }, /* 550 = fdatasync */ }; Index: projects/netbsd-tests-update-12/sys/kern/kern_descrip.c =================================================================== --- projects/netbsd-tests-update-12/sys/kern/kern_descrip.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/kern/kern_descrip.c (revision 305172) @@ -1,4108 +1,4106 @@ /*- * Copyright (c) 1982, 1986, 1989, 1991, 1993 * The Regents of the University of California. All rights reserved. * (c) UNIX System Laboratories, Inc. * All or some portions of this file are derived from material licensed * to the University of California by American Telephone and Telegraph * Co. or Unix System Laboratories, Inc. and are reproduced herein with * the permission of UNIX System Laboratories, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)kern_descrip.c 8.6 (Berkeley) 4/19/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_compat.h" #include "opt_ddb.h" #include "opt_ktrace.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef KTRACE #include #endif #include #include #include #include #include static MALLOC_DEFINE(M_FILEDESC, "filedesc", "Open file descriptor table"); static MALLOC_DEFINE(M_FILEDESC_TO_LEADER, "filedesc_to_leader", "file desc to leader structures"); static MALLOC_DEFINE(M_SIGIO, "sigio", "sigio structures"); MALLOC_DEFINE(M_FILECAPS, "filecaps", "descriptor capabilities"); MALLOC_DECLARE(M_FADVISE); static uma_zone_t file_zone; static uma_zone_t filedesc0_zone; static int closefp(struct filedesc *fdp, int fd, struct file *fp, struct thread *td, int holdleaders); static int fd_first_free(struct filedesc *fdp, int low, int size); static int fd_last_used(struct filedesc *fdp, int size); static void fdgrowtable(struct filedesc *fdp, int nfd); static void fdgrowtable_exp(struct filedesc *fdp, int nfd); static void fdunused(struct filedesc *fdp, int fd); static void fdused(struct filedesc *fdp, int fd); static int getmaxfd(struct thread *td); /* * Each process has: * * - An array of open file descriptors (fd_ofiles) * - An array of file flags (fd_ofileflags) * - A bitmap recording which descriptors are in use (fd_map) * * A process starts out with NDFILE descriptors. The value of NDFILE has * been selected based the historical limit of 20 open files, and an * assumption that the majority of processes, especially short-lived * processes like shells, will never need more. * * If this initial allocation is exhausted, a larger descriptor table and * map are allocated dynamically, and the pointers in the process's struct * filedesc are updated to point to those. This is repeated every time * the process runs out of file descriptors (provided it hasn't hit its * resource limit). * * Since threads may hold references to individual descriptor table * entries, the tables are never freed. Instead, they are placed on a * linked list and freed only when the struct filedesc is released. */ #define NDFILE 20 #define NDSLOTSIZE sizeof(NDSLOTTYPE) #define NDENTRIES (NDSLOTSIZE * __CHAR_BIT) #define NDSLOT(x) ((x) / NDENTRIES) #define NDBIT(x) ((NDSLOTTYPE)1 << ((x) % NDENTRIES)) #define NDSLOTS(x) (((x) + NDENTRIES - 1) / NDENTRIES) /* * SLIST entry used to keep track of ofiles which must be reclaimed when * the process exits. */ struct freetable { struct fdescenttbl *ft_table; SLIST_ENTRY(freetable) ft_next; }; /* * Initial allocation: a filedesc structure + the head of SLIST used to * keep track of old ofiles + enough space for NDFILE descriptors. */ struct fdescenttbl0 { int fdt_nfiles; struct filedescent fdt_ofiles[NDFILE]; }; struct filedesc0 { struct filedesc fd_fd; SLIST_HEAD(, freetable) fd_free; struct fdescenttbl0 fd_dfiles; NDSLOTTYPE fd_dmap[NDSLOTS(NDFILE)]; }; /* * Descriptor management. */ volatile int openfiles; /* actual number of open files */ struct mtx sigio_lock; /* mtx to protect pointers to sigio */ void (*mq_fdclose)(struct thread *td, int fd, struct file *fp); /* * If low >= size, just return low. Otherwise find the first zero bit in the * given bitmap, starting at low and not exceeding size - 1. Return size if * not found. */ static int fd_first_free(struct filedesc *fdp, int low, int size) { NDSLOTTYPE *map = fdp->fd_map; NDSLOTTYPE mask; int off, maxoff; if (low >= size) return (low); off = NDSLOT(low); if (low % NDENTRIES) { mask = ~(~(NDSLOTTYPE)0 >> (NDENTRIES - (low % NDENTRIES))); if ((mask &= ~map[off]) != 0UL) return (off * NDENTRIES + ffsl(mask) - 1); ++off; } for (maxoff = NDSLOTS(size); off < maxoff; ++off) if (map[off] != ~0UL) return (off * NDENTRIES + ffsl(~map[off]) - 1); return (size); } /* * Find the highest non-zero bit in the given bitmap, starting at 0 and * not exceeding size - 1. Return -1 if not found. */ static int fd_last_used(struct filedesc *fdp, int size) { NDSLOTTYPE *map = fdp->fd_map; NDSLOTTYPE mask; int off, minoff; off = NDSLOT(size); if (size % NDENTRIES) { mask = ~(~(NDSLOTTYPE)0 << (size % NDENTRIES)); if ((mask &= map[off]) != 0) return (off * NDENTRIES + flsl(mask) - 1); --off; } for (minoff = NDSLOT(0); off >= minoff; --off) if (map[off] != 0) return (off * NDENTRIES + flsl(map[off]) - 1); return (-1); } static int fdisused(struct filedesc *fdp, int fd) { KASSERT(fd >= 0 && fd < fdp->fd_nfiles, ("file descriptor %d out of range (0, %d)", fd, fdp->fd_nfiles)); return ((fdp->fd_map[NDSLOT(fd)] & NDBIT(fd)) != 0); } /* * Mark a file descriptor as used. */ static void fdused_init(struct filedesc *fdp, int fd) { KASSERT(!fdisused(fdp, fd), ("fd=%d is already used", fd)); fdp->fd_map[NDSLOT(fd)] |= NDBIT(fd); } static void fdused(struct filedesc *fdp, int fd) { FILEDESC_XLOCK_ASSERT(fdp); fdused_init(fdp, fd); if (fd > fdp->fd_lastfile) fdp->fd_lastfile = fd; if (fd == fdp->fd_freefile) fdp->fd_freefile = fd_first_free(fdp, fd, fdp->fd_nfiles); } /* * Mark a file descriptor as unused. */ static void fdunused(struct filedesc *fdp, int fd) { FILEDESC_XLOCK_ASSERT(fdp); KASSERT(fdisused(fdp, fd), ("fd=%d is already unused", fd)); KASSERT(fdp->fd_ofiles[fd].fde_file == NULL, ("fd=%d is still in use", fd)); fdp->fd_map[NDSLOT(fd)] &= ~NDBIT(fd); if (fd < fdp->fd_freefile) fdp->fd_freefile = fd; if (fd == fdp->fd_lastfile) fdp->fd_lastfile = fd_last_used(fdp, fd); } /* * Free a file descriptor. * * Avoid some work if fdp is about to be destroyed. */ static inline void fdefree_last(struct filedescent *fde) { filecaps_free(&fde->fde_caps); } static inline void fdfree(struct filedesc *fdp, int fd) { struct filedescent *fde; fde = &fdp->fd_ofiles[fd]; #ifdef CAPABILITIES seq_write_begin(&fde->fde_seq); #endif fdefree_last(fde); fde->fde_file = NULL; fdunused(fdp, fd); #ifdef CAPABILITIES seq_write_end(&fde->fde_seq); #endif } void pwd_ensure_dirs(void) { struct filedesc *fdp; fdp = curproc->p_fd; FILEDESC_XLOCK(fdp); if (fdp->fd_cdir == NULL) { fdp->fd_cdir = rootvnode; VREF(rootvnode); } if (fdp->fd_rdir == NULL) { fdp->fd_rdir = rootvnode; VREF(rootvnode); } FILEDESC_XUNLOCK(fdp); } /* * System calls on descriptors. */ #ifndef _SYS_SYSPROTO_H_ struct getdtablesize_args { int dummy; }; #endif /* ARGSUSED */ int sys_getdtablesize(struct thread *td, struct getdtablesize_args *uap) { #ifdef RACCT uint64_t lim; #endif td->td_retval[0] = min((int)lim_cur(td, RLIMIT_NOFILE), maxfilesperproc); #ifdef RACCT PROC_LOCK(td->td_proc); lim = racct_get_limit(td->td_proc, RACCT_NOFILE); PROC_UNLOCK(td->td_proc); if (lim < td->td_retval[0]) td->td_retval[0] = lim; #endif return (0); } /* * Duplicate a file descriptor to a particular value. * * Note: keep in mind that a potential race condition exists when closing * descriptors from a shared descriptor table (via rfork). */ #ifndef _SYS_SYSPROTO_H_ struct dup2_args { u_int from; u_int to; }; #endif /* ARGSUSED */ int sys_dup2(struct thread *td, struct dup2_args *uap) { return (kern_dup(td, FDDUP_FIXED, 0, (int)uap->from, (int)uap->to)); } /* * Duplicate a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct dup_args { u_int fd; }; #endif /* ARGSUSED */ int sys_dup(struct thread *td, struct dup_args *uap) { return (kern_dup(td, FDDUP_NORMAL, 0, (int)uap->fd, 0)); } /* * The file control system call. */ #ifndef _SYS_SYSPROTO_H_ struct fcntl_args { int fd; int cmd; long arg; }; #endif /* ARGSUSED */ int sys_fcntl(struct thread *td, struct fcntl_args *uap) { return (kern_fcntl_freebsd(td, uap->fd, uap->cmd, uap->arg)); } int kern_fcntl_freebsd(struct thread *td, int fd, int cmd, long arg) { struct flock fl; struct __oflock ofl; intptr_t arg1; int error, newcmd; error = 0; newcmd = cmd; switch (cmd) { case F_OGETLK: case F_OSETLK: case F_OSETLKW: /* * Convert old flock structure to new. */ error = copyin((void *)(intptr_t)arg, &ofl, sizeof(ofl)); fl.l_start = ofl.l_start; fl.l_len = ofl.l_len; fl.l_pid = ofl.l_pid; fl.l_type = ofl.l_type; fl.l_whence = ofl.l_whence; fl.l_sysid = 0; switch (cmd) { case F_OGETLK: newcmd = F_GETLK; break; case F_OSETLK: newcmd = F_SETLK; break; case F_OSETLKW: newcmd = F_SETLKW; break; } arg1 = (intptr_t)&fl; break; case F_GETLK: case F_SETLK: case F_SETLKW: case F_SETLK_REMOTE: error = copyin((void *)(intptr_t)arg, &fl, sizeof(fl)); arg1 = (intptr_t)&fl; break; default: arg1 = arg; break; } if (error) return (error); error = kern_fcntl(td, fd, newcmd, arg1); if (error) return (error); if (cmd == F_OGETLK) { ofl.l_start = fl.l_start; ofl.l_len = fl.l_len; ofl.l_pid = fl.l_pid; ofl.l_type = fl.l_type; ofl.l_whence = fl.l_whence; error = copyout(&ofl, (void *)(intptr_t)arg, sizeof(ofl)); } else if (cmd == F_GETLK) { error = copyout(&fl, (void *)(intptr_t)arg, sizeof(fl)); } return (error); } int kern_fcntl(struct thread *td, int fd, int cmd, intptr_t arg) { struct filedesc *fdp; struct flock *flp; struct file *fp, *fp2; struct filedescent *fde; struct proc *p; struct vnode *vp; cap_rights_t rights; int error, flg, tmp; uint64_t bsize; off_t foffset; error = 0; flg = F_POSIX; p = td->td_proc; fdp = p->p_fd; switch (cmd) { case F_DUPFD: tmp = arg; error = kern_dup(td, FDDUP_FCNTL, 0, fd, tmp); break; case F_DUPFD_CLOEXEC: tmp = arg; error = kern_dup(td, FDDUP_FCNTL, FDDUP_FLAG_CLOEXEC, fd, tmp); break; case F_DUP2FD: tmp = arg; error = kern_dup(td, FDDUP_FIXED, 0, fd, tmp); break; case F_DUP2FD_CLOEXEC: tmp = arg; error = kern_dup(td, FDDUP_FIXED, FDDUP_FLAG_CLOEXEC, fd, tmp); break; case F_GETFD: + error = EBADF; FILEDESC_SLOCK(fdp); - if (fget_locked(fdp, fd) == NULL) { - FILEDESC_SUNLOCK(fdp); - error = EBADF; - break; + fde = fdeget_locked(fdp, fd); + if (fde != NULL) { + td->td_retval[0] = + (fde->fde_flags & UF_EXCLOSE) ? FD_CLOEXEC : 0; + error = 0; } - fde = &fdp->fd_ofiles[fd]; - td->td_retval[0] = - (fde->fde_flags & UF_EXCLOSE) ? FD_CLOEXEC : 0; FILEDESC_SUNLOCK(fdp); break; case F_SETFD: + error = EBADF; FILEDESC_XLOCK(fdp); - if (fget_locked(fdp, fd) == NULL) { - FILEDESC_XUNLOCK(fdp); - error = EBADF; - break; + fde = fdeget_locked(fdp, fd); + if (fde != NULL) { + fde->fde_flags = (fde->fde_flags & ~UF_EXCLOSE) | + (arg & FD_CLOEXEC ? UF_EXCLOSE : 0); + error = 0; } - fde = &fdp->fd_ofiles[fd]; - fde->fde_flags = (fde->fde_flags & ~UF_EXCLOSE) | - (arg & FD_CLOEXEC ? UF_EXCLOSE : 0); FILEDESC_XUNLOCK(fdp); break; case F_GETFL: error = fget_fcntl(td, fd, cap_rights_init(&rights, CAP_FCNTL), F_GETFL, &fp); if (error != 0) break; td->td_retval[0] = OFLAGS(fp->f_flag); fdrop(fp, td); break; case F_SETFL: error = fget_fcntl(td, fd, cap_rights_init(&rights, CAP_FCNTL), F_SETFL, &fp); if (error != 0) break; do { tmp = flg = fp->f_flag; tmp &= ~FCNTLFLAGS; tmp |= FFLAGS(arg & ~O_ACCMODE) & FCNTLFLAGS; } while(atomic_cmpset_int(&fp->f_flag, flg, tmp) == 0); tmp = fp->f_flag & FNONBLOCK; error = fo_ioctl(fp, FIONBIO, &tmp, td->td_ucred, td); if (error != 0) { fdrop(fp, td); break; } tmp = fp->f_flag & FASYNC; error = fo_ioctl(fp, FIOASYNC, &tmp, td->td_ucred, td); if (error == 0) { fdrop(fp, td); break; } atomic_clear_int(&fp->f_flag, FNONBLOCK); tmp = 0; (void)fo_ioctl(fp, FIONBIO, &tmp, td->td_ucred, td); fdrop(fp, td); break; case F_GETOWN: error = fget_fcntl(td, fd, cap_rights_init(&rights, CAP_FCNTL), F_GETOWN, &fp); if (error != 0) break; error = fo_ioctl(fp, FIOGETOWN, &tmp, td->td_ucred, td); if (error == 0) td->td_retval[0] = tmp; fdrop(fp, td); break; case F_SETOWN: error = fget_fcntl(td, fd, cap_rights_init(&rights, CAP_FCNTL), F_SETOWN, &fp); if (error != 0) break; tmp = arg; error = fo_ioctl(fp, FIOSETOWN, &tmp, td->td_ucred, td); fdrop(fp, td); break; case F_SETLK_REMOTE: error = priv_check(td, PRIV_NFS_LOCKD); if (error) return (error); flg = F_REMOTE; goto do_setlk; case F_SETLKW: flg |= F_WAIT; /* FALLTHROUGH F_SETLK */ case F_SETLK: do_setlk: cap_rights_init(&rights, CAP_FLOCK); error = fget_unlocked(fdp, fd, &rights, &fp, NULL); if (error != 0) break; if (fp->f_type != DTYPE_VNODE) { error = EBADF; fdrop(fp, td); break; } flp = (struct flock *)arg; if (flp->l_whence == SEEK_CUR) { foffset = foffset_get(fp); if (foffset < 0 || (flp->l_start > 0 && foffset > OFF_MAX - flp->l_start)) { error = EOVERFLOW; fdrop(fp, td); break; } flp->l_start += foffset; } vp = fp->f_vnode; switch (flp->l_type) { case F_RDLCK: if ((fp->f_flag & FREAD) == 0) { error = EBADF; break; } PROC_LOCK(p->p_leader); p->p_leader->p_flag |= P_ADVLOCK; PROC_UNLOCK(p->p_leader); error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_SETLK, flp, flg); break; case F_WRLCK: if ((fp->f_flag & FWRITE) == 0) { error = EBADF; break; } PROC_LOCK(p->p_leader); p->p_leader->p_flag |= P_ADVLOCK; PROC_UNLOCK(p->p_leader); error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_SETLK, flp, flg); break; case F_UNLCK: error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_UNLCK, flp, flg); break; case F_UNLCKSYS: /* * Temporary api for testing remote lock * infrastructure. */ if (flg != F_REMOTE) { error = EINVAL; break; } error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_UNLCKSYS, flp, flg); break; default: error = EINVAL; break; } if (error != 0 || flp->l_type == F_UNLCK || flp->l_type == F_UNLCKSYS) { fdrop(fp, td); break; } /* * Check for a race with close. * * The vnode is now advisory locked (or unlocked, but this case * is not really important) as the caller requested. * We had to drop the filedesc lock, so we need to recheck if * the descriptor is still valid, because if it was closed * in the meantime we need to remove advisory lock from the * vnode - close on any descriptor leading to an advisory * locked vnode, removes that lock. * We will return 0 on purpose in that case, as the result of * successful advisory lock might have been externally visible * already. This is fine - effectively we pretend to the caller * that the closing thread was a bit slower and that the * advisory lock succeeded before the close. */ error = fget_unlocked(fdp, fd, &rights, &fp2, NULL); if (error != 0) { fdrop(fp, td); break; } if (fp != fp2) { flp->l_whence = SEEK_SET; flp->l_start = 0; flp->l_len = 0; flp->l_type = F_UNLCK; (void) VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_UNLCK, flp, F_POSIX); } fdrop(fp, td); fdrop(fp2, td); break; case F_GETLK: error = fget_unlocked(fdp, fd, cap_rights_init(&rights, CAP_FLOCK), &fp, NULL); if (error != 0) break; if (fp->f_type != DTYPE_VNODE) { error = EBADF; fdrop(fp, td); break; } flp = (struct flock *)arg; if (flp->l_type != F_RDLCK && flp->l_type != F_WRLCK && flp->l_type != F_UNLCK) { error = EINVAL; fdrop(fp, td); break; } if (flp->l_whence == SEEK_CUR) { foffset = foffset_get(fp); if ((flp->l_start > 0 && foffset > OFF_MAX - flp->l_start) || (flp->l_start < 0 && foffset < OFF_MIN - flp->l_start)) { error = EOVERFLOW; fdrop(fp, td); break; } flp->l_start += foffset; } vp = fp->f_vnode; error = VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_GETLK, flp, F_POSIX); fdrop(fp, td); break; case F_RDAHEAD: arg = arg ? 128 * 1024: 0; /* FALLTHROUGH */ case F_READAHEAD: error = fget_unlocked(fdp, fd, cap_rights_init(&rights), &fp, NULL); if (error != 0) break; if (fp->f_type != DTYPE_VNODE) { fdrop(fp, td); error = EBADF; break; } vp = fp->f_vnode; /* * Exclusive lock synchronizes against f_seqcount reads and * writes in sequential_heuristic(). */ error = vn_lock(vp, LK_EXCLUSIVE); if (error != 0) { fdrop(fp, td); break; } if (arg >= 0) { bsize = fp->f_vnode->v_mount->mnt_stat.f_iosize; fp->f_seqcount = (arg + bsize - 1) / bsize; atomic_set_int(&fp->f_flag, FRDAHEAD); } else { atomic_clear_int(&fp->f_flag, FRDAHEAD); } VOP_UNLOCK(vp, 0); fdrop(fp, td); break; default: error = EINVAL; break; } return (error); } static int getmaxfd(struct thread *td) { return (min((int)lim_cur(td, RLIMIT_NOFILE), maxfilesperproc)); } /* * Common code for dup, dup2, fcntl(F_DUPFD) and fcntl(F_DUP2FD). */ int kern_dup(struct thread *td, u_int mode, int flags, int old, int new) { struct filedesc *fdp; struct filedescent *oldfde, *newfde; struct proc *p; struct file *delfp; int error, maxfd; p = td->td_proc; fdp = p->p_fd; MPASS((flags & ~(FDDUP_FLAG_CLOEXEC)) == 0); MPASS(mode < FDDUP_LASTMODE); AUDIT_ARG_FD(old); /* XXXRW: if (flags & FDDUP_FIXED) AUDIT_ARG_FD2(new); */ /* * Verify we have a valid descriptor to dup from and possibly to * dup to. Unlike dup() and dup2(), fcntl()'s F_DUPFD should * return EINVAL when the new descriptor is out of bounds. */ if (old < 0) return (EBADF); if (new < 0) return (mode == FDDUP_FCNTL ? EINVAL : EBADF); maxfd = getmaxfd(td); if (new >= maxfd) return (mode == FDDUP_FCNTL ? EINVAL : EBADF); error = EBADF; FILEDESC_XLOCK(fdp); if (fget_locked(fdp, old) == NULL) goto unlock; if ((mode == FDDUP_FIXED || mode == FDDUP_MUSTREPLACE) && old == new) { td->td_retval[0] = new; if (flags & FDDUP_FLAG_CLOEXEC) fdp->fd_ofiles[new].fde_flags |= UF_EXCLOSE; error = 0; goto unlock; } /* * If the caller specified a file descriptor, make sure the file * table is large enough to hold it, and grab it. Otherwise, just * allocate a new descriptor the usual way. */ switch (mode) { case FDDUP_NORMAL: case FDDUP_FCNTL: if ((error = fdalloc(td, new, &new)) != 0) goto unlock; break; case FDDUP_MUSTREPLACE: /* Target file descriptor must exist. */ if (fget_locked(fdp, new) == NULL) goto unlock; break; case FDDUP_FIXED: if (new >= fdp->fd_nfiles) { /* * The resource limits are here instead of e.g. * fdalloc(), because the file descriptor table may be * shared between processes, so we can't really use * racct_add()/racct_sub(). Instead of counting the * number of actually allocated descriptors, just put * the limit on the size of the file descriptor table. */ #ifdef RACCT if (racct_enable) { PROC_LOCK(p); error = racct_set(p, RACCT_NOFILE, new + 1); PROC_UNLOCK(p); if (error != 0) { error = EMFILE; goto unlock; } } #endif fdgrowtable_exp(fdp, new + 1); } if (!fdisused(fdp, new)) fdused(fdp, new); break; default: KASSERT(0, ("%s unsupported mode %d", __func__, mode)); } KASSERT(old != new, ("new fd is same as old")); oldfde = &fdp->fd_ofiles[old]; fhold(oldfde->fde_file); newfde = &fdp->fd_ofiles[new]; delfp = newfde->fde_file; /* * Duplicate the source descriptor. */ #ifdef CAPABILITIES seq_write_begin(&newfde->fde_seq); #endif filecaps_free(&newfde->fde_caps); memcpy(newfde, oldfde, fde_change_size); filecaps_copy(&oldfde->fde_caps, &newfde->fde_caps, true); if ((flags & FDDUP_FLAG_CLOEXEC) != 0) newfde->fde_flags = oldfde->fde_flags | UF_EXCLOSE; else newfde->fde_flags = oldfde->fde_flags & ~UF_EXCLOSE; #ifdef CAPABILITIES seq_write_end(&newfde->fde_seq); #endif td->td_retval[0] = new; error = 0; if (delfp != NULL) { (void) closefp(fdp, new, delfp, td, 1); FILEDESC_UNLOCK_ASSERT(fdp); } else { unlock: FILEDESC_XUNLOCK(fdp); } return (error); } /* * If sigio is on the list associated with a process or process group, * disable signalling from the device, remove sigio from the list and * free sigio. */ void funsetown(struct sigio **sigiop) { struct sigio *sigio; if (*sigiop == NULL) return; SIGIO_LOCK(); sigio = *sigiop; if (sigio == NULL) { SIGIO_UNLOCK(); return; } *(sigio->sio_myref) = NULL; if ((sigio)->sio_pgid < 0) { struct pgrp *pg = (sigio)->sio_pgrp; PGRP_LOCK(pg); SLIST_REMOVE(&sigio->sio_pgrp->pg_sigiolst, sigio, sigio, sio_pgsigio); PGRP_UNLOCK(pg); } else { struct proc *p = (sigio)->sio_proc; PROC_LOCK(p); SLIST_REMOVE(&sigio->sio_proc->p_sigiolst, sigio, sigio, sio_pgsigio); PROC_UNLOCK(p); } SIGIO_UNLOCK(); crfree(sigio->sio_ucred); free(sigio, M_SIGIO); } /* * Free a list of sigio structures. * We only need to lock the SIGIO_LOCK because we have made ourselves * inaccessible to callers of fsetown and therefore do not need to lock * the proc or pgrp struct for the list manipulation. */ void funsetownlst(struct sigiolst *sigiolst) { struct proc *p; struct pgrp *pg; struct sigio *sigio; sigio = SLIST_FIRST(sigiolst); if (sigio == NULL) return; p = NULL; pg = NULL; /* * Every entry of the list should belong * to a single proc or pgrp. */ if (sigio->sio_pgid < 0) { pg = sigio->sio_pgrp; PGRP_LOCK_ASSERT(pg, MA_NOTOWNED); } else /* if (sigio->sio_pgid > 0) */ { p = sigio->sio_proc; PROC_LOCK_ASSERT(p, MA_NOTOWNED); } SIGIO_LOCK(); while ((sigio = SLIST_FIRST(sigiolst)) != NULL) { *(sigio->sio_myref) = NULL; if (pg != NULL) { KASSERT(sigio->sio_pgid < 0, ("Proc sigio in pgrp sigio list")); KASSERT(sigio->sio_pgrp == pg, ("Bogus pgrp in sigio list")); PGRP_LOCK(pg); SLIST_REMOVE(&pg->pg_sigiolst, sigio, sigio, sio_pgsigio); PGRP_UNLOCK(pg); } else /* if (p != NULL) */ { KASSERT(sigio->sio_pgid > 0, ("Pgrp sigio in proc sigio list")); KASSERT(sigio->sio_proc == p, ("Bogus proc in sigio list")); PROC_LOCK(p); SLIST_REMOVE(&p->p_sigiolst, sigio, sigio, sio_pgsigio); PROC_UNLOCK(p); } SIGIO_UNLOCK(); crfree(sigio->sio_ucred); free(sigio, M_SIGIO); SIGIO_LOCK(); } SIGIO_UNLOCK(); } /* * This is common code for FIOSETOWN ioctl called by fcntl(fd, F_SETOWN, arg). * * After permission checking, add a sigio structure to the sigio list for * the process or process group. */ int fsetown(pid_t pgid, struct sigio **sigiop) { struct proc *proc; struct pgrp *pgrp; struct sigio *sigio; int ret; if (pgid == 0) { funsetown(sigiop); return (0); } ret = 0; /* Allocate and fill in the new sigio out of locks. */ sigio = malloc(sizeof(struct sigio), M_SIGIO, M_WAITOK); sigio->sio_pgid = pgid; sigio->sio_ucred = crhold(curthread->td_ucred); sigio->sio_myref = sigiop; sx_slock(&proctree_lock); if (pgid > 0) { proc = pfind(pgid); if (proc == NULL) { ret = ESRCH; goto fail; } /* * Policy - Don't allow a process to FSETOWN a process * in another session. * * Remove this test to allow maximum flexibility or * restrict FSETOWN to the current process or process * group for maximum safety. */ PROC_UNLOCK(proc); if (proc->p_session != curthread->td_proc->p_session) { ret = EPERM; goto fail; } pgrp = NULL; } else /* if (pgid < 0) */ { pgrp = pgfind(-pgid); if (pgrp == NULL) { ret = ESRCH; goto fail; } PGRP_UNLOCK(pgrp); /* * Policy - Don't allow a process to FSETOWN a process * in another session. * * Remove this test to allow maximum flexibility or * restrict FSETOWN to the current process or process * group for maximum safety. */ if (pgrp->pg_session != curthread->td_proc->p_session) { ret = EPERM; goto fail; } proc = NULL; } funsetown(sigiop); if (pgid > 0) { PROC_LOCK(proc); /* * Since funsetownlst() is called without the proctree * locked, we need to check for P_WEXIT. * XXX: is ESRCH correct? */ if ((proc->p_flag & P_WEXIT) != 0) { PROC_UNLOCK(proc); ret = ESRCH; goto fail; } SLIST_INSERT_HEAD(&proc->p_sigiolst, sigio, sio_pgsigio); sigio->sio_proc = proc; PROC_UNLOCK(proc); } else { PGRP_LOCK(pgrp); SLIST_INSERT_HEAD(&pgrp->pg_sigiolst, sigio, sio_pgsigio); sigio->sio_pgrp = pgrp; PGRP_UNLOCK(pgrp); } sx_sunlock(&proctree_lock); SIGIO_LOCK(); *sigiop = sigio; SIGIO_UNLOCK(); return (0); fail: sx_sunlock(&proctree_lock); crfree(sigio->sio_ucred); free(sigio, M_SIGIO); return (ret); } /* * This is common code for FIOGETOWN ioctl called by fcntl(fd, F_GETOWN, arg). */ pid_t fgetown(sigiop) struct sigio **sigiop; { pid_t pgid; SIGIO_LOCK(); pgid = (*sigiop != NULL) ? (*sigiop)->sio_pgid : 0; SIGIO_UNLOCK(); return (pgid); } /* * Function drops the filedesc lock on return. */ static int closefp(struct filedesc *fdp, int fd, struct file *fp, struct thread *td, int holdleaders) { int error; FILEDESC_XLOCK_ASSERT(fdp); if (holdleaders) { if (td->td_proc->p_fdtol != NULL) { /* * Ask fdfree() to sleep to ensure that all relevant * process leaders can be traversed in closef(). */ fdp->fd_holdleaderscount++; } else { holdleaders = 0; } } /* * We now hold the fp reference that used to be owned by the * descriptor array. We have to unlock the FILEDESC *AFTER* * knote_fdclose to prevent a race of the fd getting opened, a knote * added, and deleteing a knote for the new fd. */ knote_fdclose(td, fd); /* * We need to notify mqueue if the object is of type mqueue. */ if (fp->f_type == DTYPE_MQUEUE) mq_fdclose(td, fd, fp); FILEDESC_XUNLOCK(fdp); error = closef(fp, td); if (holdleaders) { FILEDESC_XLOCK(fdp); fdp->fd_holdleaderscount--; if (fdp->fd_holdleaderscount == 0 && fdp->fd_holdleaderswakeup != 0) { fdp->fd_holdleaderswakeup = 0; wakeup(&fdp->fd_holdleaderscount); } FILEDESC_XUNLOCK(fdp); } return (error); } /* * Close a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct close_args { int fd; }; #endif /* ARGSUSED */ int sys_close(struct thread *td, struct close_args *uap) { return (kern_close(td, uap->fd)); } int kern_close(struct thread *td, int fd) { struct filedesc *fdp; struct file *fp; fdp = td->td_proc->p_fd; AUDIT_SYSCLOSE(td, fd); FILEDESC_XLOCK(fdp); if ((fp = fget_locked(fdp, fd)) == NULL) { FILEDESC_XUNLOCK(fdp); return (EBADF); } fdfree(fdp, fd); /* closefp() drops the FILEDESC lock for us. */ return (closefp(fdp, fd, fp, td, 1)); } /* * Close open file descriptors. */ #ifndef _SYS_SYSPROTO_H_ struct closefrom_args { int lowfd; }; #endif /* ARGSUSED */ int sys_closefrom(struct thread *td, struct closefrom_args *uap) { struct filedesc *fdp; int fd; fdp = td->td_proc->p_fd; AUDIT_ARG_FD(uap->lowfd); /* * Treat negative starting file descriptor values identical to * closefrom(0) which closes all files. */ if (uap->lowfd < 0) uap->lowfd = 0; FILEDESC_SLOCK(fdp); for (fd = uap->lowfd; fd <= fdp->fd_lastfile; fd++) { if (fdp->fd_ofiles[fd].fde_file != NULL) { FILEDESC_SUNLOCK(fdp); (void)kern_close(td, fd); FILEDESC_SLOCK(fdp); } } FILEDESC_SUNLOCK(fdp); return (0); } #if defined(COMPAT_43) /* * Return status information about a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct ofstat_args { int fd; struct ostat *sb; }; #endif /* ARGSUSED */ int ofstat(struct thread *td, struct ofstat_args *uap) { struct ostat oub; struct stat ub; int error; error = kern_fstat(td, uap->fd, &ub); if (error == 0) { cvtstat(&ub, &oub); error = copyout(&oub, uap->sb, sizeof(oub)); } return (error); } #endif /* COMPAT_43 */ /* * Return status information about a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct fstat_args { int fd; struct stat *sb; }; #endif /* ARGSUSED */ int sys_fstat(struct thread *td, struct fstat_args *uap) { struct stat ub; int error; error = kern_fstat(td, uap->fd, &ub); if (error == 0) error = copyout(&ub, uap->sb, sizeof(ub)); return (error); } int kern_fstat(struct thread *td, int fd, struct stat *sbp) { struct file *fp; cap_rights_t rights; int error; AUDIT_ARG_FD(fd); error = fget(td, fd, cap_rights_init(&rights, CAP_FSTAT), &fp); if (error != 0) return (error); AUDIT_ARG_FILE(td->td_proc, fp); error = fo_stat(fp, sbp, td->td_ucred, td); fdrop(fp, td); #ifdef KTRACE if (error == 0 && KTRPOINT(td, KTR_STRUCT)) ktrstat(sbp); #endif return (error); } /* * Return status information about a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct nfstat_args { int fd; struct nstat *sb; }; #endif /* ARGSUSED */ int sys_nfstat(struct thread *td, struct nfstat_args *uap) { struct nstat nub; struct stat ub; int error; error = kern_fstat(td, uap->fd, &ub); if (error == 0) { cvtnstat(&ub, &nub); error = copyout(&nub, uap->sb, sizeof(nub)); } return (error); } /* * Return pathconf information about a file descriptor. */ #ifndef _SYS_SYSPROTO_H_ struct fpathconf_args { int fd; int name; }; #endif /* ARGSUSED */ int sys_fpathconf(struct thread *td, struct fpathconf_args *uap) { struct file *fp; struct vnode *vp; cap_rights_t rights; int error; error = fget(td, uap->fd, cap_rights_init(&rights, CAP_FPATHCONF), &fp); if (error != 0) return (error); if (uap->name == _PC_ASYNC_IO) { td->td_retval[0] = _POSIX_ASYNCHRONOUS_IO; goto out; } vp = fp->f_vnode; if (vp != NULL) { vn_lock(vp, LK_SHARED | LK_RETRY); error = VOP_PATHCONF(vp, uap->name, td->td_retval); VOP_UNLOCK(vp, 0); } else if (fp->f_type == DTYPE_PIPE || fp->f_type == DTYPE_SOCKET) { if (uap->name != _PC_PIPE_BUF) { error = EINVAL; } else { td->td_retval[0] = PIPE_BUF; error = 0; } } else { error = EOPNOTSUPP; } out: fdrop(fp, td); return (error); } /* * Initialize filecaps structure. */ void filecaps_init(struct filecaps *fcaps) { bzero(fcaps, sizeof(*fcaps)); fcaps->fc_nioctls = -1; } /* * Copy filecaps structure allocating memory for ioctls array if needed. * * The last parameter indicates whether the fdtable is locked. If it is not and * ioctls are encountered, copying fails and the caller must lock the table. * * Note that if the table was not locked, the caller has to check the relevant * sequence counter to determine whether the operation was successful. */ int filecaps_copy(const struct filecaps *src, struct filecaps *dst, bool locked) { size_t size; *dst = *src; if (src->fc_ioctls == NULL) return (0); if (!locked) return (1); KASSERT(src->fc_nioctls > 0, ("fc_ioctls != NULL, but fc_nioctls=%hd", src->fc_nioctls)); size = sizeof(src->fc_ioctls[0]) * src->fc_nioctls; dst->fc_ioctls = malloc(size, M_FILECAPS, M_WAITOK); bcopy(src->fc_ioctls, dst->fc_ioctls, size); return (0); } /* * Move filecaps structure to the new place and clear the old place. */ void filecaps_move(struct filecaps *src, struct filecaps *dst) { *dst = *src; bzero(src, sizeof(*src)); } /* * Fill the given filecaps structure with full rights. */ static void filecaps_fill(struct filecaps *fcaps) { CAP_ALL(&fcaps->fc_rights); fcaps->fc_ioctls = NULL; fcaps->fc_nioctls = -1; fcaps->fc_fcntls = CAP_FCNTL_ALL; } /* * Free memory allocated within filecaps structure. */ void filecaps_free(struct filecaps *fcaps) { free(fcaps->fc_ioctls, M_FILECAPS); bzero(fcaps, sizeof(*fcaps)); } /* * Validate the given filecaps structure. */ static void filecaps_validate(const struct filecaps *fcaps, const char *func) { KASSERT(cap_rights_is_valid(&fcaps->fc_rights), ("%s: invalid rights", func)); KASSERT((fcaps->fc_fcntls & ~CAP_FCNTL_ALL) == 0, ("%s: invalid fcntls", func)); KASSERT(fcaps->fc_fcntls == 0 || cap_rights_is_set(&fcaps->fc_rights, CAP_FCNTL), ("%s: fcntls without CAP_FCNTL", func)); KASSERT(fcaps->fc_ioctls != NULL ? fcaps->fc_nioctls > 0 : (fcaps->fc_nioctls == -1 || fcaps->fc_nioctls == 0), ("%s: invalid ioctls", func)); KASSERT(fcaps->fc_nioctls == 0 || cap_rights_is_set(&fcaps->fc_rights, CAP_IOCTL), ("%s: ioctls without CAP_IOCTL", func)); } static void fdgrowtable_exp(struct filedesc *fdp, int nfd) { int nfd1; FILEDESC_XLOCK_ASSERT(fdp); nfd1 = fdp->fd_nfiles * 2; if (nfd1 < nfd) nfd1 = nfd; fdgrowtable(fdp, nfd1); } /* * Grow the file table to accommodate (at least) nfd descriptors. */ static void fdgrowtable(struct filedesc *fdp, int nfd) { struct filedesc0 *fdp0; struct freetable *ft; struct fdescenttbl *ntable; struct fdescenttbl *otable; int nnfiles, onfiles; NDSLOTTYPE *nmap, *omap; /* * If lastfile is -1 this struct filedesc was just allocated and we are * growing it to accommodate for the one we are going to copy from. There * is no need to have a lock on this one as it's not visible to anyone. */ if (fdp->fd_lastfile != -1) FILEDESC_XLOCK_ASSERT(fdp); KASSERT(fdp->fd_nfiles > 0, ("zero-length file table")); /* save old values */ onfiles = fdp->fd_nfiles; otable = fdp->fd_files; omap = fdp->fd_map; /* compute the size of the new table */ nnfiles = NDSLOTS(nfd) * NDENTRIES; /* round up */ if (nnfiles <= onfiles) /* the table is already large enough */ return; /* * Allocate a new table. We need enough space for the number of * entries, file entries themselves and the struct freetable we will use * when we decommission the table and place it on the freelist. * We place the struct freetable in the middle so we don't have * to worry about padding. */ ntable = malloc(offsetof(struct fdescenttbl, fdt_ofiles) + nnfiles * sizeof(ntable->fdt_ofiles[0]) + sizeof(struct freetable), M_FILEDESC, M_ZERO | M_WAITOK); /* copy the old data */ ntable->fdt_nfiles = nnfiles; memcpy(ntable->fdt_ofiles, otable->fdt_ofiles, onfiles * sizeof(ntable->fdt_ofiles[0])); /* * Allocate a new map only if the old is not large enough. It will * grow at a slower rate than the table as it can map more * entries than the table can hold. */ if (NDSLOTS(nnfiles) > NDSLOTS(onfiles)) { nmap = malloc(NDSLOTS(nnfiles) * NDSLOTSIZE, M_FILEDESC, M_ZERO | M_WAITOK); /* copy over the old data and update the pointer */ memcpy(nmap, omap, NDSLOTS(onfiles) * sizeof(*omap)); fdp->fd_map = nmap; } /* * Make sure that ntable is correctly initialized before we replace * fd_files poiner. Otherwise fget_unlocked() may see inconsistent * data. */ atomic_store_rel_ptr((volatile void *)&fdp->fd_files, (uintptr_t)ntable); /* * Do not free the old file table, as some threads may still * reference entries within it. Instead, place it on a freelist * which will be processed when the struct filedesc is released. * * Note that if onfiles == NDFILE, we're dealing with the original * static allocation contained within (struct filedesc0 *)fdp, * which must not be freed. */ if (onfiles > NDFILE) { ft = (struct freetable *)&otable->fdt_ofiles[onfiles]; fdp0 = (struct filedesc0 *)fdp; ft->ft_table = otable; SLIST_INSERT_HEAD(&fdp0->fd_free, ft, ft_next); } /* * The map does not have the same possibility of threads still * holding references to it. So always free it as long as it * does not reference the original static allocation. */ if (NDSLOTS(onfiles) > NDSLOTS(NDFILE)) free(omap, M_FILEDESC); } /* * Allocate a file descriptor for the process. */ int fdalloc(struct thread *td, int minfd, int *result) { struct proc *p = td->td_proc; struct filedesc *fdp = p->p_fd; int fd, maxfd, allocfd; #ifdef RACCT int error; #endif FILEDESC_XLOCK_ASSERT(fdp); if (fdp->fd_freefile > minfd) minfd = fdp->fd_freefile; maxfd = getmaxfd(td); /* * Search the bitmap for a free descriptor starting at minfd. * If none is found, grow the file table. */ fd = fd_first_free(fdp, minfd, fdp->fd_nfiles); if (fd >= maxfd) return (EMFILE); if (fd >= fdp->fd_nfiles) { allocfd = min(fd * 2, maxfd); #ifdef RACCT if (racct_enable) { PROC_LOCK(p); error = racct_set(p, RACCT_NOFILE, allocfd); PROC_UNLOCK(p); if (error != 0) return (EMFILE); } #endif /* * fd is already equal to first free descriptor >= minfd, so * we only need to grow the table and we are done. */ fdgrowtable_exp(fdp, allocfd); } /* * Perform some sanity checks, then mark the file descriptor as * used and return it to the caller. */ KASSERT(fd >= 0 && fd < min(maxfd, fdp->fd_nfiles), ("invalid descriptor %d", fd)); KASSERT(!fdisused(fdp, fd), ("fd_first_free() returned non-free descriptor")); KASSERT(fdp->fd_ofiles[fd].fde_file == NULL, ("file descriptor isn't free")); fdused(fdp, fd); *result = fd; return (0); } /* * Allocate n file descriptors for the process. */ int fdallocn(struct thread *td, int minfd, int *fds, int n) { struct proc *p = td->td_proc; struct filedesc *fdp = p->p_fd; int i; FILEDESC_XLOCK_ASSERT(fdp); for (i = 0; i < n; i++) if (fdalloc(td, 0, &fds[i]) != 0) break; if (i < n) { for (i--; i >= 0; i--) fdunused(fdp, fds[i]); return (EMFILE); } return (0); } /* * Create a new open file structure and allocate a file descriptor for the * process that refers to it. We add one reference to the file for the * descriptor table and one reference for resultfp. This is to prevent us * being preempted and the entry in the descriptor table closed after we * release the FILEDESC lock. */ int falloc_caps(struct thread *td, struct file **resultfp, int *resultfd, int flags, struct filecaps *fcaps) { struct file *fp; int error, fd; error = falloc_noinstall(td, &fp); if (error) return (error); /* no reference held on error */ error = finstall(td, fp, &fd, flags, fcaps); if (error) { fdrop(fp, td); /* one reference (fp only) */ return (error); } if (resultfp != NULL) *resultfp = fp; /* copy out result */ else fdrop(fp, td); /* release local reference */ if (resultfd != NULL) *resultfd = fd; return (0); } /* * Create a new open file structure without allocating a file descriptor. */ int falloc_noinstall(struct thread *td, struct file **resultfp) { struct file *fp; int maxuserfiles = maxfiles - (maxfiles / 20); static struct timeval lastfail; static int curfail; KASSERT(resultfp != NULL, ("%s: resultfp == NULL", __func__)); if ((openfiles >= maxuserfiles && priv_check(td, PRIV_MAXFILES) != 0) || openfiles >= maxfiles) { if (ppsratecheck(&lastfail, &curfail, 1)) { printf("kern.maxfiles limit exceeded by uid %i, " "please see tuning(7).\n", td->td_ucred->cr_ruid); } return (ENFILE); } atomic_add_int(&openfiles, 1); fp = uma_zalloc(file_zone, M_WAITOK | M_ZERO); refcount_init(&fp->f_count, 1); fp->f_cred = crhold(td->td_ucred); fp->f_ops = &badfileops; *resultfp = fp; return (0); } /* * Install a file in a file descriptor table. */ void _finstall(struct filedesc *fdp, struct file *fp, int fd, int flags, struct filecaps *fcaps) { struct filedescent *fde; MPASS(fp != NULL); if (fcaps != NULL) filecaps_validate(fcaps, __func__); FILEDESC_XLOCK_ASSERT(fdp); fde = &fdp->fd_ofiles[fd]; #ifdef CAPABILITIES seq_write_begin(&fde->fde_seq); #endif fde->fde_file = fp; fde->fde_flags = (flags & O_CLOEXEC) != 0 ? UF_EXCLOSE : 0; if (fcaps != NULL) filecaps_move(fcaps, &fde->fde_caps); else filecaps_fill(&fde->fde_caps); #ifdef CAPABILITIES seq_write_end(&fde->fde_seq); #endif } int finstall(struct thread *td, struct file *fp, int *fd, int flags, struct filecaps *fcaps) { struct filedesc *fdp = td->td_proc->p_fd; int error; MPASS(fd != NULL); FILEDESC_XLOCK(fdp); if ((error = fdalloc(td, 0, fd))) { FILEDESC_XUNLOCK(fdp); return (error); } fhold(fp); _finstall(fdp, fp, *fd, flags, fcaps); FILEDESC_XUNLOCK(fdp); return (0); } /* * Build a new filedesc structure from another. * Copy the current, root, and jail root vnode references. * * If fdp is not NULL, return with it shared locked. */ struct filedesc * fdinit(struct filedesc *fdp, bool prepfiles) { struct filedesc0 *newfdp0; struct filedesc *newfdp; newfdp0 = uma_zalloc(filedesc0_zone, M_WAITOK | M_ZERO); newfdp = &newfdp0->fd_fd; /* Create the file descriptor table. */ FILEDESC_LOCK_INIT(newfdp); refcount_init(&newfdp->fd_refcnt, 1); refcount_init(&newfdp->fd_holdcnt, 1); newfdp->fd_cmask = CMASK; newfdp->fd_map = newfdp0->fd_dmap; newfdp->fd_lastfile = -1; newfdp->fd_files = (struct fdescenttbl *)&newfdp0->fd_dfiles; newfdp->fd_files->fdt_nfiles = NDFILE; if (fdp == NULL) return (newfdp); if (prepfiles && fdp->fd_lastfile >= newfdp->fd_nfiles) fdgrowtable(newfdp, fdp->fd_lastfile + 1); FILEDESC_SLOCK(fdp); newfdp->fd_cdir = fdp->fd_cdir; if (newfdp->fd_cdir) VREF(newfdp->fd_cdir); newfdp->fd_rdir = fdp->fd_rdir; if (newfdp->fd_rdir) VREF(newfdp->fd_rdir); newfdp->fd_jdir = fdp->fd_jdir; if (newfdp->fd_jdir) VREF(newfdp->fd_jdir); if (!prepfiles) { FILEDESC_SUNLOCK(fdp); } else { while (fdp->fd_lastfile >= newfdp->fd_nfiles) { FILEDESC_SUNLOCK(fdp); fdgrowtable(newfdp, fdp->fd_lastfile + 1); FILEDESC_SLOCK(fdp); } } return (newfdp); } static struct filedesc * fdhold(struct proc *p) { struct filedesc *fdp; PROC_LOCK_ASSERT(p, MA_OWNED); fdp = p->p_fd; if (fdp != NULL) refcount_acquire(&fdp->fd_holdcnt); return (fdp); } static void fddrop(struct filedesc *fdp) { if (fdp->fd_holdcnt > 1) { if (refcount_release(&fdp->fd_holdcnt) == 0) return; } FILEDESC_LOCK_DESTROY(fdp); uma_zfree(filedesc0_zone, fdp); } /* * Share a filedesc structure. */ struct filedesc * fdshare(struct filedesc *fdp) { refcount_acquire(&fdp->fd_refcnt); return (fdp); } /* * Unshare a filedesc structure, if necessary by making a copy */ void fdunshare(struct thread *td) { struct filedesc *tmp; struct proc *p = td->td_proc; if (p->p_fd->fd_refcnt == 1) return; tmp = fdcopy(p->p_fd); fdescfree(td); p->p_fd = tmp; } void fdinstall_remapped(struct thread *td, struct filedesc *fdp) { fdescfree(td); td->td_proc->p_fd = fdp; } /* * Copy a filedesc structure. A NULL pointer in returns a NULL reference, * this is to ease callers, not catch errors. */ struct filedesc * fdcopy(struct filedesc *fdp) { struct filedesc *newfdp; struct filedescent *nfde, *ofde; int i; MPASS(fdp != NULL); newfdp = fdinit(fdp, true); /* copy all passable descriptors (i.e. not kqueue) */ newfdp->fd_freefile = -1; for (i = 0; i <= fdp->fd_lastfile; ++i) { ofde = &fdp->fd_ofiles[i]; if (ofde->fde_file == NULL || (ofde->fde_file->f_ops->fo_flags & DFLAG_PASSABLE) == 0) { if (newfdp->fd_freefile == -1) newfdp->fd_freefile = i; continue; } nfde = &newfdp->fd_ofiles[i]; *nfde = *ofde; filecaps_copy(&ofde->fde_caps, &nfde->fde_caps, true); fhold(nfde->fde_file); fdused_init(newfdp, i); newfdp->fd_lastfile = i; } if (newfdp->fd_freefile == -1) newfdp->fd_freefile = i; newfdp->fd_cmask = fdp->fd_cmask; FILEDESC_SUNLOCK(fdp); return (newfdp); } /* * Copies a filedesc structure, while remapping all file descriptors * stored inside using a translation table. * * File descriptors are copied over to the new file descriptor table, * regardless of whether the close-on-exec flag is set. */ int fdcopy_remapped(struct filedesc *fdp, const int *fds, size_t nfds, struct filedesc **ret) { struct filedesc *newfdp; struct filedescent *nfde, *ofde; int error, i; MPASS(fdp != NULL); newfdp = fdinit(fdp, true); if (nfds > fdp->fd_lastfile + 1) { /* New table cannot be larger than the old one. */ error = E2BIG; goto bad; } /* Copy all passable descriptors (i.e. not kqueue). */ newfdp->fd_freefile = nfds; for (i = 0; i < nfds; ++i) { if (fds[i] < 0 || fds[i] > fdp->fd_lastfile) { /* File descriptor out of bounds. */ error = EBADF; goto bad; } ofde = &fdp->fd_ofiles[fds[i]]; if (ofde->fde_file == NULL) { /* Unused file descriptor. */ error = EBADF; goto bad; } if ((ofde->fde_file->f_ops->fo_flags & DFLAG_PASSABLE) == 0) { /* File descriptor cannot be passed. */ error = EINVAL; goto bad; } nfde = &newfdp->fd_ofiles[i]; *nfde = *ofde; filecaps_copy(&ofde->fde_caps, &nfde->fde_caps, true); fhold(nfde->fde_file); fdused_init(newfdp, i); newfdp->fd_lastfile = i; } newfdp->fd_cmask = fdp->fd_cmask; FILEDESC_SUNLOCK(fdp); *ret = newfdp; return (0); bad: FILEDESC_SUNLOCK(fdp); fdescfree_remapped(newfdp); return (error); } /* * Clear POSIX style locks. This is only used when fdp looses a reference (i.e. * one of processes using it exits) and the table used to be shared. */ static void fdclearlocks(struct thread *td) { struct filedesc *fdp; struct filedesc_to_leader *fdtol; struct flock lf; struct file *fp; struct proc *p; struct vnode *vp; int i; p = td->td_proc; fdp = p->p_fd; fdtol = p->p_fdtol; MPASS(fdtol != NULL); FILEDESC_XLOCK(fdp); KASSERT(fdtol->fdl_refcount > 0, ("filedesc_to_refcount botch: fdl_refcount=%d", fdtol->fdl_refcount)); if (fdtol->fdl_refcount == 1 && (p->p_leader->p_flag & P_ADVLOCK) != 0) { for (i = 0; i <= fdp->fd_lastfile; i++) { fp = fdp->fd_ofiles[i].fde_file; if (fp == NULL || fp->f_type != DTYPE_VNODE) continue; fhold(fp); FILEDESC_XUNLOCK(fdp); lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; lf.l_type = F_UNLCK; vp = fp->f_vnode; (void) VOP_ADVLOCK(vp, (caddr_t)p->p_leader, F_UNLCK, &lf, F_POSIX); FILEDESC_XLOCK(fdp); fdrop(fp, td); } } retry: if (fdtol->fdl_refcount == 1) { if (fdp->fd_holdleaderscount > 0 && (p->p_leader->p_flag & P_ADVLOCK) != 0) { /* * close() or kern_dup() has cleared a reference * in a shared file descriptor table. */ fdp->fd_holdleaderswakeup = 1; sx_sleep(&fdp->fd_holdleaderscount, FILEDESC_LOCK(fdp), PLOCK, "fdlhold", 0); goto retry; } if (fdtol->fdl_holdcount > 0) { /* * Ensure that fdtol->fdl_leader remains * valid in closef(). */ fdtol->fdl_wakeup = 1; sx_sleep(fdtol, FILEDESC_LOCK(fdp), PLOCK, "fdlhold", 0); goto retry; } } fdtol->fdl_refcount--; if (fdtol->fdl_refcount == 0 && fdtol->fdl_holdcount == 0) { fdtol->fdl_next->fdl_prev = fdtol->fdl_prev; fdtol->fdl_prev->fdl_next = fdtol->fdl_next; } else fdtol = NULL; p->p_fdtol = NULL; FILEDESC_XUNLOCK(fdp); if (fdtol != NULL) free(fdtol, M_FILEDESC_TO_LEADER); } /* * Release a filedesc structure. */ static void fdescfree_fds(struct thread *td, struct filedesc *fdp, bool needclose) { struct filedesc0 *fdp0; struct freetable *ft, *tft; struct filedescent *fde; struct file *fp; int i; for (i = 0; i <= fdp->fd_lastfile; i++) { fde = &fdp->fd_ofiles[i]; fp = fde->fde_file; if (fp != NULL) { fdefree_last(fde); if (needclose) (void) closef(fp, td); else fdrop(fp, td); } } if (NDSLOTS(fdp->fd_nfiles) > NDSLOTS(NDFILE)) free(fdp->fd_map, M_FILEDESC); if (fdp->fd_nfiles > NDFILE) free(fdp->fd_files, M_FILEDESC); fdp0 = (struct filedesc0 *)fdp; SLIST_FOREACH_SAFE(ft, &fdp0->fd_free, ft_next, tft) free(ft->ft_table, M_FILEDESC); fddrop(fdp); } void fdescfree(struct thread *td) { struct proc *p; struct filedesc *fdp; struct vnode *cdir, *jdir, *rdir; p = td->td_proc; fdp = p->p_fd; MPASS(fdp != NULL); #ifdef RACCT if (racct_enable) { PROC_LOCK(p); racct_set(p, RACCT_NOFILE, 0); PROC_UNLOCK(p); } #endif if (p->p_fdtol != NULL) fdclearlocks(td); PROC_LOCK(p); p->p_fd = NULL; PROC_UNLOCK(p); if (refcount_release(&fdp->fd_refcnt) == 0) return; FILEDESC_XLOCK(fdp); cdir = fdp->fd_cdir; fdp->fd_cdir = NULL; rdir = fdp->fd_rdir; fdp->fd_rdir = NULL; jdir = fdp->fd_jdir; fdp->fd_jdir = NULL; FILEDESC_XUNLOCK(fdp); if (cdir != NULL) vrele(cdir); if (rdir != NULL) vrele(rdir); if (jdir != NULL) vrele(jdir); fdescfree_fds(td, fdp, 1); } void fdescfree_remapped(struct filedesc *fdp) { if (fdp->fd_cdir != NULL) vrele(fdp->fd_cdir); if (fdp->fd_rdir != NULL) vrele(fdp->fd_rdir); if (fdp->fd_jdir != NULL) vrele(fdp->fd_jdir); fdescfree_fds(curthread, fdp, 0); } /* * For setugid programs, we don't want to people to use that setugidness * to generate error messages which write to a file which otherwise would * otherwise be off-limits to the process. We check for filesystems where * the vnode can change out from under us after execve (like [lin]procfs). * * Since fdsetugidsafety calls this only for fd 0, 1 and 2, this check is * sufficient. We also don't check for setugidness since we know we are. */ static bool is_unsafe(struct file *fp) { struct vnode *vp; if (fp->f_type != DTYPE_VNODE) return (false); vp = fp->f_vnode; return ((vp->v_vflag & VV_PROCDEP) != 0); } /* * Make this setguid thing safe, if at all possible. */ void fdsetugidsafety(struct thread *td) { struct filedesc *fdp; struct file *fp; int i; fdp = td->td_proc->p_fd; KASSERT(fdp->fd_refcnt == 1, ("the fdtable should not be shared")); MPASS(fdp->fd_nfiles >= 3); for (i = 0; i <= 2; i++) { fp = fdp->fd_ofiles[i].fde_file; if (fp != NULL && is_unsafe(fp)) { FILEDESC_XLOCK(fdp); knote_fdclose(td, i); /* * NULL-out descriptor prior to close to avoid * a race while close blocks. */ fdfree(fdp, i); FILEDESC_XUNLOCK(fdp); (void) closef(fp, td); } } } /* * If a specific file object occupies a specific file descriptor, close the * file descriptor entry and drop a reference on the file object. This is a * convenience function to handle a subsequent error in a function that calls * falloc() that handles the race that another thread might have closed the * file descriptor out from under the thread creating the file object. */ void fdclose(struct thread *td, struct file *fp, int idx) { struct filedesc *fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); if (fdp->fd_ofiles[idx].fde_file == fp) { fdfree(fdp, idx); FILEDESC_XUNLOCK(fdp); fdrop(fp, td); } else FILEDESC_XUNLOCK(fdp); } /* * Close any files on exec? */ void fdcloseexec(struct thread *td) { struct filedesc *fdp; struct filedescent *fde; struct file *fp; int i; fdp = td->td_proc->p_fd; KASSERT(fdp->fd_refcnt == 1, ("the fdtable should not be shared")); for (i = 0; i <= fdp->fd_lastfile; i++) { fde = &fdp->fd_ofiles[i]; fp = fde->fde_file; if (fp != NULL && (fp->f_type == DTYPE_MQUEUE || (fde->fde_flags & UF_EXCLOSE))) { FILEDESC_XLOCK(fdp); fdfree(fdp, i); (void) closefp(fdp, i, fp, td, 0); FILEDESC_UNLOCK_ASSERT(fdp); } } } /* * It is unsafe for set[ug]id processes to be started with file * descriptors 0..2 closed, as these descriptors are given implicit * significance in the Standard C library. fdcheckstd() will create a * descriptor referencing /dev/null for each of stdin, stdout, and * stderr that is not already open. */ int fdcheckstd(struct thread *td) { struct filedesc *fdp; register_t save; int i, error, devnull; fdp = td->td_proc->p_fd; KASSERT(fdp->fd_refcnt == 1, ("the fdtable should not be shared")); MPASS(fdp->fd_nfiles >= 3); devnull = -1; for (i = 0; i <= 2; i++) { if (fdp->fd_ofiles[i].fde_file != NULL) continue; save = td->td_retval[0]; if (devnull != -1) { error = kern_dup(td, FDDUP_FIXED, 0, devnull, i); } else { error = kern_openat(td, AT_FDCWD, "/dev/null", UIO_SYSSPACE, O_RDWR, 0); if (error == 0) { devnull = td->td_retval[0]; KASSERT(devnull == i, ("we didn't get our fd")); } } td->td_retval[0] = save; if (error != 0) return (error); } return (0); } /* * Internal form of close. Decrement reference count on file structure. * Note: td may be NULL when closing a file that was being passed in a * message. * * XXXRW: Giant is not required for the caller, but often will be held; this * makes it moderately likely the Giant will be recursed in the VFS case. */ int closef(struct file *fp, struct thread *td) { struct vnode *vp; struct flock lf; struct filedesc_to_leader *fdtol; struct filedesc *fdp; /* * POSIX record locking dictates that any close releases ALL * locks owned by this process. This is handled by setting * a flag in the unlock to free ONLY locks obeying POSIX * semantics, and not to free BSD-style file locks. * If the descriptor was in a message, POSIX-style locks * aren't passed with the descriptor, and the thread pointer * will be NULL. Callers should be careful only to pass a * NULL thread pointer when there really is no owning * context that might have locks, or the locks will be * leaked. */ if (fp->f_type == DTYPE_VNODE && td != NULL) { vp = fp->f_vnode; if ((td->td_proc->p_leader->p_flag & P_ADVLOCK) != 0) { lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; lf.l_type = F_UNLCK; (void) VOP_ADVLOCK(vp, (caddr_t)td->td_proc->p_leader, F_UNLCK, &lf, F_POSIX); } fdtol = td->td_proc->p_fdtol; if (fdtol != NULL) { /* * Handle special case where file descriptor table is * shared between multiple process leaders. */ fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); for (fdtol = fdtol->fdl_next; fdtol != td->td_proc->p_fdtol; fdtol = fdtol->fdl_next) { if ((fdtol->fdl_leader->p_flag & P_ADVLOCK) == 0) continue; fdtol->fdl_holdcount++; FILEDESC_XUNLOCK(fdp); lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; lf.l_type = F_UNLCK; vp = fp->f_vnode; (void) VOP_ADVLOCK(vp, (caddr_t)fdtol->fdl_leader, F_UNLCK, &lf, F_POSIX); FILEDESC_XLOCK(fdp); fdtol->fdl_holdcount--; if (fdtol->fdl_holdcount == 0 && fdtol->fdl_wakeup != 0) { fdtol->fdl_wakeup = 0; wakeup(fdtol); } } FILEDESC_XUNLOCK(fdp); } } return (fdrop(fp, td)); } /* * Initialize the file pointer with the specified properties. * * The ops are set with release semantics to be certain that the flags, type, * and data are visible when ops is. This is to prevent ops methods from being * called with bad data. */ void finit(struct file *fp, u_int flag, short type, void *data, struct fileops *ops) { fp->f_data = data; fp->f_flag = flag; fp->f_type = type; atomic_store_rel_ptr((volatile uintptr_t *)&fp->f_ops, (uintptr_t)ops); } int fget_unlocked(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, struct file **fpp, seq_t *seqp) { #ifdef CAPABILITIES struct filedescent *fde; #endif struct fdescenttbl *fdt; struct file *fp; u_int count; #ifdef CAPABILITIES seq_t seq; cap_rights_t haverights; int error; #endif fdt = fdp->fd_files; if ((u_int)fd >= fdt->fdt_nfiles) return (EBADF); /* * Fetch the descriptor locklessly. We avoid fdrop() races by * never raising a refcount above 0. To accomplish this we have * to use a cmpset loop rather than an atomic_add. The descriptor * must be re-verified once we acquire a reference to be certain * that the identity is still correct and we did not lose a race * due to preemption. */ for (;;) { #ifdef CAPABILITIES seq = seq_read(fd_seq(fdt, fd)); fde = &fdt->fdt_ofiles[fd]; haverights = *cap_rights_fde(fde); fp = fde->fde_file; if (!seq_consistent(fd_seq(fdt, fd), seq)) { cpu_spinwait(); continue; } #else fp = fdt->fdt_ofiles[fd].fde_file; #endif if (fp == NULL) return (EBADF); #ifdef CAPABILITIES error = cap_check(&haverights, needrightsp); if (error != 0) return (error); #endif retry: count = fp->f_count; if (count == 0) { /* * Force a reload. Other thread could reallocate the * table before this fd was closed, so it possible that * there is a stale fp pointer in cached version. */ fdt = *(struct fdescenttbl * volatile *)&(fdp->fd_files); continue; } /* * Use an acquire barrier to force re-reading of fdt so it is * refreshed for verification. */ if (atomic_cmpset_acq_int(&fp->f_count, count, count + 1) == 0) goto retry; fdt = fdp->fd_files; #ifdef CAPABILITIES if (seq_consistent_nomb(fd_seq(fdt, fd), seq)) #else if (fp == fdt->fdt_ofiles[fd].fde_file) #endif break; fdrop(fp, curthread); } *fpp = fp; if (seqp != NULL) { #ifdef CAPABILITIES *seqp = seq; #endif } return (0); } /* * Extract the file pointer associated with the specified descriptor for the * current user process. * * If the descriptor doesn't exist or doesn't match 'flags', EBADF is * returned. * * File's rights will be checked against the capability rights mask. * * If an error occurred the non-zero error is returned and *fpp is set to * NULL. Otherwise *fpp is held and set and zero is returned. Caller is * responsible for fdrop(). */ static __inline int _fget(struct thread *td, int fd, struct file **fpp, int flags, cap_rights_t *needrightsp, seq_t *seqp) { struct filedesc *fdp; struct file *fp; int error; *fpp = NULL; fdp = td->td_proc->p_fd; error = fget_unlocked(fdp, fd, needrightsp, &fp, seqp); if (error != 0) return (error); if (fp->f_ops == &badfileops) { fdrop(fp, td); return (EBADF); } /* * FREAD and FWRITE failure return EBADF as per POSIX. */ error = 0; switch (flags) { case FREAD: case FWRITE: if ((fp->f_flag & flags) == 0) error = EBADF; break; case FEXEC: if ((fp->f_flag & (FREAD | FEXEC)) == 0 || ((fp->f_flag & FWRITE) != 0)) error = EBADF; break; case 0: break; default: KASSERT(0, ("wrong flags")); } if (error != 0) { fdrop(fp, td); return (error); } *fpp = fp; return (0); } int fget(struct thread *td, int fd, cap_rights_t *rightsp, struct file **fpp) { return (_fget(td, fd, fpp, 0, rightsp, NULL)); } int fget_mmap(struct thread *td, int fd, cap_rights_t *rightsp, u_char *maxprotp, struct file **fpp) { int error; #ifndef CAPABILITIES error = _fget(td, fd, fpp, 0, rightsp, NULL); if (maxprotp != NULL) *maxprotp = VM_PROT_ALL; #else struct filedesc *fdp = td->td_proc->p_fd; seq_t seq; MPASS(cap_rights_is_set(rightsp, CAP_MMAP)); for (;;) { error = _fget(td, fd, fpp, 0, rightsp, &seq); if (error != 0) return (error); /* * If requested, convert capability rights to access flags. */ if (maxprotp != NULL) *maxprotp = cap_rights_to_vmprot(cap_rights(fdp, fd)); if (!fd_modified(fdp, fd, seq)) break; fdrop(*fpp, td); } #endif return (error); } int fget_read(struct thread *td, int fd, cap_rights_t *rightsp, struct file **fpp) { return (_fget(td, fd, fpp, FREAD, rightsp, NULL)); } int fget_write(struct thread *td, int fd, cap_rights_t *rightsp, struct file **fpp) { return (_fget(td, fd, fpp, FWRITE, rightsp, NULL)); } int fget_fcntl(struct thread *td, int fd, cap_rights_t *rightsp, int needfcntl, struct file **fpp) { struct filedesc *fdp = td->td_proc->p_fd; #ifndef CAPABILITIES return (fget_unlocked(fdp, fd, rightsp, fpp, NULL)); #else int error; seq_t seq; MPASS(cap_rights_is_set(rightsp, CAP_FCNTL)); for (;;) { error = fget_unlocked(fdp, fd, rightsp, fpp, &seq); if (error != 0) return (error); error = cap_fcntl_check(fdp, fd, needfcntl); if (!fd_modified(fdp, fd, seq)) break; fdrop(*fpp, td); } if (error != 0) { fdrop(*fpp, td); *fpp = NULL; } return (error); #endif } /* * Like fget() but loads the underlying vnode, or returns an error if the * descriptor does not represent a vnode. Note that pipes use vnodes but * never have VM objects. The returned vnode will be vref()'d. * * XXX: what about the unused flags ? */ static __inline int _fgetvp(struct thread *td, int fd, int flags, cap_rights_t *needrightsp, struct vnode **vpp) { struct file *fp; int error; *vpp = NULL; error = _fget(td, fd, &fp, flags, needrightsp, NULL); if (error != 0) return (error); if (fp->f_vnode == NULL) { error = EINVAL; } else { *vpp = fp->f_vnode; vref(*vpp); } fdrop(fp, td); return (error); } int fgetvp(struct thread *td, int fd, cap_rights_t *rightsp, struct vnode **vpp) { return (_fgetvp(td, fd, 0, rightsp, vpp)); } int fgetvp_rights(struct thread *td, int fd, cap_rights_t *needrightsp, struct filecaps *havecaps, struct vnode **vpp) { struct filedesc *fdp; struct file *fp; #ifdef CAPABILITIES int error; #endif fdp = td->td_proc->p_fd; fp = fget_locked(fdp, fd); if (fp == NULL || fp->f_ops == &badfileops) return (EBADF); #ifdef CAPABILITIES error = cap_check(cap_rights(fdp, fd), needrightsp); if (error != 0) return (error); #endif if (fp->f_vnode == NULL) return (EINVAL); *vpp = fp->f_vnode; vref(*vpp); filecaps_copy(&fdp->fd_ofiles[fd].fde_caps, havecaps, true); return (0); } int fgetvp_read(struct thread *td, int fd, cap_rights_t *rightsp, struct vnode **vpp) { return (_fgetvp(td, fd, FREAD, rightsp, vpp)); } int fgetvp_exec(struct thread *td, int fd, cap_rights_t *rightsp, struct vnode **vpp) { return (_fgetvp(td, fd, FEXEC, rightsp, vpp)); } #ifdef notyet int fgetvp_write(struct thread *td, int fd, cap_rights_t *rightsp, struct vnode **vpp) { return (_fgetvp(td, fd, FWRITE, rightsp, vpp)); } #endif /* * Like fget() but loads the underlying socket, or returns an error if the * descriptor does not represent a socket. * * We bump the ref count on the returned socket. XXX Also obtain the SX lock * in the future. * * Note: fgetsock() and fputsock() are deprecated, as consumers should rely * on their file descriptor reference to prevent the socket from being free'd * during use. */ int fgetsock(struct thread *td, int fd, cap_rights_t *rightsp, struct socket **spp, u_int *fflagp) { struct file *fp; int error; *spp = NULL; if (fflagp != NULL) *fflagp = 0; if ((error = _fget(td, fd, &fp, 0, rightsp, NULL)) != 0) return (error); if (fp->f_type != DTYPE_SOCKET) { error = ENOTSOCK; } else { *spp = fp->f_data; if (fflagp) *fflagp = fp->f_flag; SOCK_LOCK(*spp); soref(*spp); SOCK_UNLOCK(*spp); } fdrop(fp, td); return (error); } /* * Drop the reference count on the socket and XXX release the SX lock in the * future. The last reference closes the socket. * * Note: fputsock() is deprecated, see comment for fgetsock(). */ void fputsock(struct socket *so) { ACCEPT_LOCK(); SOCK_LOCK(so); CURVNET_SET(so->so_vnet); sorele(so); CURVNET_RESTORE(); } /* * Handle the last reference to a file being closed. */ int _fdrop(struct file *fp, struct thread *td) { int error; if (fp->f_count != 0) panic("fdrop: count %d", fp->f_count); error = fo_close(fp, td); atomic_subtract_int(&openfiles, 1); crfree(fp->f_cred); free(fp->f_advice, M_FADVISE); uma_zfree(file_zone, fp); return (error); } /* * Apply an advisory lock on a file descriptor. * * Just attempt to get a record lock of the requested type on the entire file * (l_whence = SEEK_SET, l_start = 0, l_len = 0). */ #ifndef _SYS_SYSPROTO_H_ struct flock_args { int fd; int how; }; #endif /* ARGSUSED */ int sys_flock(struct thread *td, struct flock_args *uap) { struct file *fp; struct vnode *vp; struct flock lf; cap_rights_t rights; int error; error = fget(td, uap->fd, cap_rights_init(&rights, CAP_FLOCK), &fp); if (error != 0) return (error); if (fp->f_type != DTYPE_VNODE) { fdrop(fp, td); return (EOPNOTSUPP); } vp = fp->f_vnode; lf.l_whence = SEEK_SET; lf.l_start = 0; lf.l_len = 0; if (uap->how & LOCK_UN) { lf.l_type = F_UNLCK; atomic_clear_int(&fp->f_flag, FHASLOCK); error = VOP_ADVLOCK(vp, (caddr_t)fp, F_UNLCK, &lf, F_FLOCK); goto done2; } if (uap->how & LOCK_EX) lf.l_type = F_WRLCK; else if (uap->how & LOCK_SH) lf.l_type = F_RDLCK; else { error = EBADF; goto done2; } atomic_set_int(&fp->f_flag, FHASLOCK); error = VOP_ADVLOCK(vp, (caddr_t)fp, F_SETLK, &lf, (uap->how & LOCK_NB) ? F_FLOCK : F_FLOCK | F_WAIT); done2: fdrop(fp, td); return (error); } /* * Duplicate the specified descriptor to a free descriptor. */ int dupfdopen(struct thread *td, struct filedesc *fdp, int dfd, int mode, int openerror, int *indxp) { struct filedescent *newfde, *oldfde; struct file *fp; int error, indx; KASSERT(openerror == ENODEV || openerror == ENXIO, ("unexpected error %d in %s", openerror, __func__)); /* * If the to-be-dup'd fd number is greater than the allowed number * of file descriptors, or the fd to be dup'd has already been * closed, then reject. */ FILEDESC_XLOCK(fdp); if ((fp = fget_locked(fdp, dfd)) == NULL) { FILEDESC_XUNLOCK(fdp); return (EBADF); } error = fdalloc(td, 0, &indx); if (error != 0) { FILEDESC_XUNLOCK(fdp); return (error); } /* * There are two cases of interest here. * * For ENODEV simply dup (dfd) to file descriptor (indx) and return. * * For ENXIO steal away the file structure from (dfd) and store it in * (indx). (dfd) is effectively closed by this operation. */ switch (openerror) { case ENODEV: /* * Check that the mode the file is being opened for is a * subset of the mode of the existing descriptor. */ if (((mode & (FREAD|FWRITE)) | fp->f_flag) != fp->f_flag) { fdunused(fdp, indx); FILEDESC_XUNLOCK(fdp); return (EACCES); } fhold(fp); newfde = &fdp->fd_ofiles[indx]; oldfde = &fdp->fd_ofiles[dfd]; #ifdef CAPABILITIES seq_write_begin(&newfde->fde_seq); #endif memcpy(newfde, oldfde, fde_change_size); filecaps_copy(&oldfde->fde_caps, &newfde->fde_caps, true); #ifdef CAPABILITIES seq_write_end(&newfde->fde_seq); #endif break; case ENXIO: /* * Steal away the file pointer from dfd and stuff it into indx. */ newfde = &fdp->fd_ofiles[indx]; oldfde = &fdp->fd_ofiles[dfd]; #ifdef CAPABILITIES seq_write_begin(&newfde->fde_seq); #endif memcpy(newfde, oldfde, fde_change_size); oldfde->fde_file = NULL; fdunused(fdp, dfd); #ifdef CAPABILITIES seq_write_end(&newfde->fde_seq); #endif break; } FILEDESC_XUNLOCK(fdp); *indxp = indx; return (0); } /* * This sysctl determines if we will allow a process to chroot(2) if it * has a directory open: * 0: disallowed for all processes. * 1: allowed for processes that were not already chroot(2)'ed. * 2: allowed for all processes. */ static int chroot_allow_open_directories = 1; SYSCTL_INT(_kern, OID_AUTO, chroot_allow_open_directories, CTLFLAG_RW, &chroot_allow_open_directories, 0, "Allow a process to chroot(2) if it has a directory open"); /* * Helper function for raised chroot(2) security function: Refuse if * any filedescriptors are open directories. */ static int chroot_refuse_vdir_fds(struct filedesc *fdp) { struct vnode *vp; struct file *fp; int fd; FILEDESC_LOCK_ASSERT(fdp); for (fd = 0; fd <= fdp->fd_lastfile; fd++) { fp = fget_locked(fdp, fd); if (fp == NULL) continue; if (fp->f_type == DTYPE_VNODE) { vp = fp->f_vnode; if (vp->v_type == VDIR) return (EPERM); } } return (0); } /* * Common routine for kern_chroot() and jail_attach(). The caller is * responsible for invoking priv_check() and mac_vnode_check_chroot() to * authorize this operation. */ int pwd_chroot(struct thread *td, struct vnode *vp) { struct filedesc *fdp; struct vnode *oldvp; int error; fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); if (chroot_allow_open_directories == 0 || (chroot_allow_open_directories == 1 && fdp->fd_rdir != rootvnode)) { error = chroot_refuse_vdir_fds(fdp); if (error != 0) { FILEDESC_XUNLOCK(fdp); return (error); } } oldvp = fdp->fd_rdir; VREF(vp); fdp->fd_rdir = vp; if (fdp->fd_jdir == NULL) { VREF(vp); fdp->fd_jdir = vp; } FILEDESC_XUNLOCK(fdp); vrele(oldvp); return (0); } void pwd_chdir(struct thread *td, struct vnode *vp) { struct filedesc *fdp; struct vnode *oldvp; fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); VNASSERT(vp->v_usecount > 0, vp, ("chdir to a vnode with zero usecount")); oldvp = fdp->fd_cdir; fdp->fd_cdir = vp; FILEDESC_XUNLOCK(fdp); vrele(oldvp); } /* * Scan all active processes and prisons to see if any of them have a current * or root directory of `olddp'. If so, replace them with the new mount point. */ void mountcheckdirs(struct vnode *olddp, struct vnode *newdp) { struct filedesc *fdp; struct prison *pr; struct proc *p; int nrele; if (vrefcnt(olddp) == 1) return; nrele = 0; sx_slock(&allproc_lock); FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) continue; FILEDESC_XLOCK(fdp); if (fdp->fd_cdir == olddp) { vref(newdp); fdp->fd_cdir = newdp; nrele++; } if (fdp->fd_rdir == olddp) { vref(newdp); fdp->fd_rdir = newdp; nrele++; } if (fdp->fd_jdir == olddp) { vref(newdp); fdp->fd_jdir = newdp; nrele++; } FILEDESC_XUNLOCK(fdp); fddrop(fdp); } sx_sunlock(&allproc_lock); if (rootvnode == olddp) { vref(newdp); rootvnode = newdp; nrele++; } mtx_lock(&prison0.pr_mtx); if (prison0.pr_root == olddp) { vref(newdp); prison0.pr_root = newdp; nrele++; } mtx_unlock(&prison0.pr_mtx); sx_slock(&allprison_lock); TAILQ_FOREACH(pr, &allprison, pr_list) { mtx_lock(&pr->pr_mtx); if (pr->pr_root == olddp) { vref(newdp); pr->pr_root = newdp; nrele++; } mtx_unlock(&pr->pr_mtx); } sx_sunlock(&allprison_lock); while (nrele--) vrele(olddp); } struct filedesc_to_leader * filedesc_to_leader_alloc(struct filedesc_to_leader *old, struct filedesc *fdp, struct proc *leader) { struct filedesc_to_leader *fdtol; fdtol = malloc(sizeof(struct filedesc_to_leader), M_FILEDESC_TO_LEADER, M_WAITOK); fdtol->fdl_refcount = 1; fdtol->fdl_holdcount = 0; fdtol->fdl_wakeup = 0; fdtol->fdl_leader = leader; if (old != NULL) { FILEDESC_XLOCK(fdp); fdtol->fdl_next = old->fdl_next; fdtol->fdl_prev = old; old->fdl_next = fdtol; fdtol->fdl_next->fdl_prev = fdtol; FILEDESC_XUNLOCK(fdp); } else { fdtol->fdl_next = fdtol; fdtol->fdl_prev = fdtol; } return (fdtol); } static int sysctl_kern_proc_nfds(SYSCTL_HANDLER_ARGS) { struct filedesc *fdp; int i, count, slots; if (*(int *)arg1 != 0) return (EINVAL); fdp = curproc->p_fd; count = 0; FILEDESC_SLOCK(fdp); slots = NDSLOTS(fdp->fd_lastfile + 1); for (i = 0; i < slots; i++) count += bitcountl(fdp->fd_map[i]); FILEDESC_SUNLOCK(fdp); return (SYSCTL_OUT(req, &count, sizeof(count))); } static SYSCTL_NODE(_kern_proc, KERN_PROC_NFDS, nfds, CTLFLAG_RD|CTLFLAG_MPSAFE, sysctl_kern_proc_nfds, "Number of open file descriptors"); /* * Get file structures globally. */ static int sysctl_kern_file(SYSCTL_HANDLER_ARGS) { struct xfile xf; struct filedesc *fdp; struct file *fp; struct proc *p; int error, n; error = sysctl_wire_old_buffer(req, 0); if (error != 0) return (error); if (req->oldptr == NULL) { n = 0; sx_slock(&allproc_lock); FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); if (p->p_state == PRS_NEW) { PROC_UNLOCK(p); continue; } fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) continue; /* overestimates sparse tables. */ if (fdp->fd_lastfile > 0) n += fdp->fd_lastfile; fddrop(fdp); } sx_sunlock(&allproc_lock); return (SYSCTL_OUT(req, 0, n * sizeof(xf))); } error = 0; bzero(&xf, sizeof(xf)); xf.xf_size = sizeof(xf); sx_slock(&allproc_lock); FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); if (p->p_state == PRS_NEW) { PROC_UNLOCK(p); continue; } if (p_cansee(req->td, p) != 0) { PROC_UNLOCK(p); continue; } xf.xf_pid = p->p_pid; xf.xf_uid = p->p_ucred->cr_uid; fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) continue; FILEDESC_SLOCK(fdp); for (n = 0; fdp->fd_refcnt > 0 && n <= fdp->fd_lastfile; ++n) { if ((fp = fdp->fd_ofiles[n].fde_file) == NULL) continue; xf.xf_fd = n; xf.xf_file = fp; xf.xf_data = fp->f_data; xf.xf_vnode = fp->f_vnode; xf.xf_type = fp->f_type; xf.xf_count = fp->f_count; xf.xf_msgcount = 0; xf.xf_offset = foffset_get(fp); xf.xf_flag = fp->f_flag; error = SYSCTL_OUT(req, &xf, sizeof(xf)); if (error) break; } FILEDESC_SUNLOCK(fdp); fddrop(fdp); if (error) break; } sx_sunlock(&allproc_lock); return (error); } SYSCTL_PROC(_kern, KERN_FILE, file, CTLTYPE_OPAQUE|CTLFLAG_RD|CTLFLAG_MPSAFE, 0, 0, sysctl_kern_file, "S,xfile", "Entire file table"); #ifdef KINFO_FILE_SIZE CTASSERT(sizeof(struct kinfo_file) == KINFO_FILE_SIZE); #endif static int xlate_fflags(int fflags) { static const struct { int fflag; int kf_fflag; } fflags_table[] = { { FAPPEND, KF_FLAG_APPEND }, { FASYNC, KF_FLAG_ASYNC }, { FFSYNC, KF_FLAG_FSYNC }, { FHASLOCK, KF_FLAG_HASLOCK }, { FNONBLOCK, KF_FLAG_NONBLOCK }, { FREAD, KF_FLAG_READ }, { FWRITE, KF_FLAG_WRITE }, { O_CREAT, KF_FLAG_CREAT }, { O_DIRECT, KF_FLAG_DIRECT }, { O_EXCL, KF_FLAG_EXCL }, { O_EXEC, KF_FLAG_EXEC }, { O_EXLOCK, KF_FLAG_EXLOCK }, { O_NOFOLLOW, KF_FLAG_NOFOLLOW }, { O_SHLOCK, KF_FLAG_SHLOCK }, { O_TRUNC, KF_FLAG_TRUNC } }; unsigned int i; int kflags; kflags = 0; for (i = 0; i < nitems(fflags_table); i++) if (fflags & fflags_table[i].fflag) kflags |= fflags_table[i].kf_fflag; return (kflags); } /* Trim unused data from kf_path by truncating the structure size. */ static void pack_kinfo(struct kinfo_file *kif) { kif->kf_structsize = offsetof(struct kinfo_file, kf_path) + strlen(kif->kf_path) + 1; kif->kf_structsize = roundup(kif->kf_structsize, sizeof(uint64_t)); } static void export_file_to_kinfo(struct file *fp, int fd, cap_rights_t *rightsp, struct kinfo_file *kif, struct filedesc *fdp, int flags) { int error; bzero(kif, sizeof(*kif)); /* Set a default type to allow for empty fill_kinfo() methods. */ kif->kf_type = KF_TYPE_UNKNOWN; kif->kf_flags = xlate_fflags(fp->f_flag); if (rightsp != NULL) kif->kf_cap_rights = *rightsp; else cap_rights_init(&kif->kf_cap_rights); kif->kf_fd = fd; kif->kf_ref_count = fp->f_count; kif->kf_offset = foffset_get(fp); /* * This may drop the filedesc lock, so the 'fp' cannot be * accessed after this call. */ error = fo_fill_kinfo(fp, kif, fdp); if (error == 0) kif->kf_status |= KF_ATTR_VALID; if ((flags & KERN_FILEDESC_PACK_KINFO) != 0) pack_kinfo(kif); else kif->kf_structsize = roundup2(sizeof(*kif), sizeof(uint64_t)); } static void export_vnode_to_kinfo(struct vnode *vp, int fd, int fflags, struct kinfo_file *kif, int flags) { int error; bzero(kif, sizeof(*kif)); kif->kf_type = KF_TYPE_VNODE; error = vn_fill_kinfo_vnode(vp, kif); if (error == 0) kif->kf_status |= KF_ATTR_VALID; kif->kf_flags = xlate_fflags(fflags); cap_rights_init(&kif->kf_cap_rights); kif->kf_fd = fd; kif->kf_ref_count = -1; kif->kf_offset = -1; if ((flags & KERN_FILEDESC_PACK_KINFO) != 0) pack_kinfo(kif); else kif->kf_structsize = roundup2(sizeof(*kif), sizeof(uint64_t)); vrele(vp); } struct export_fd_buf { struct filedesc *fdp; struct sbuf *sb; ssize_t remainder; struct kinfo_file kif; int flags; }; static int export_kinfo_to_sb(struct export_fd_buf *efbuf) { struct kinfo_file *kif; kif = &efbuf->kif; if (efbuf->remainder != -1) { if (efbuf->remainder < kif->kf_structsize) { /* Terminate export. */ efbuf->remainder = 0; return (0); } efbuf->remainder -= kif->kf_structsize; } return (sbuf_bcat(efbuf->sb, kif, kif->kf_structsize) == 0 ? 0 : ENOMEM); } static int export_file_to_sb(struct file *fp, int fd, cap_rights_t *rightsp, struct export_fd_buf *efbuf) { int error; if (efbuf->remainder == 0) return (0); export_file_to_kinfo(fp, fd, rightsp, &efbuf->kif, efbuf->fdp, efbuf->flags); FILEDESC_SUNLOCK(efbuf->fdp); error = export_kinfo_to_sb(efbuf); FILEDESC_SLOCK(efbuf->fdp); return (error); } static int export_vnode_to_sb(struct vnode *vp, int fd, int fflags, struct export_fd_buf *efbuf) { int error; if (efbuf->remainder == 0) return (0); if (efbuf->fdp != NULL) FILEDESC_SUNLOCK(efbuf->fdp); export_vnode_to_kinfo(vp, fd, fflags, &efbuf->kif, efbuf->flags); error = export_kinfo_to_sb(efbuf); if (efbuf->fdp != NULL) FILEDESC_SLOCK(efbuf->fdp); return (error); } /* * Store a process file descriptor information to sbuf. * * Takes a locked proc as argument, and returns with the proc unlocked. */ int kern_proc_filedesc_out(struct proc *p, struct sbuf *sb, ssize_t maxlen, int flags) { struct file *fp; struct filedesc *fdp; struct export_fd_buf *efbuf; struct vnode *cttyvp, *textvp, *tracevp; int error, i; cap_rights_t rights; PROC_LOCK_ASSERT(p, MA_OWNED); /* ktrace vnode */ tracevp = p->p_tracevp; if (tracevp != NULL) vref(tracevp); /* text vnode */ textvp = p->p_textvp; if (textvp != NULL) vref(textvp); /* Controlling tty. */ cttyvp = NULL; if (p->p_pgrp != NULL && p->p_pgrp->pg_session != NULL) { cttyvp = p->p_pgrp->pg_session->s_ttyvp; if (cttyvp != NULL) vref(cttyvp); } fdp = fdhold(p); PROC_UNLOCK(p); efbuf = malloc(sizeof(*efbuf), M_TEMP, M_WAITOK); efbuf->fdp = NULL; efbuf->sb = sb; efbuf->remainder = maxlen; efbuf->flags = flags; if (tracevp != NULL) export_vnode_to_sb(tracevp, KF_FD_TYPE_TRACE, FREAD | FWRITE, efbuf); if (textvp != NULL) export_vnode_to_sb(textvp, KF_FD_TYPE_TEXT, FREAD, efbuf); if (cttyvp != NULL) export_vnode_to_sb(cttyvp, KF_FD_TYPE_CTTY, FREAD | FWRITE, efbuf); error = 0; if (fdp == NULL) goto fail; efbuf->fdp = fdp; FILEDESC_SLOCK(fdp); /* working directory */ if (fdp->fd_cdir != NULL) { vref(fdp->fd_cdir); export_vnode_to_sb(fdp->fd_cdir, KF_FD_TYPE_CWD, FREAD, efbuf); } /* root directory */ if (fdp->fd_rdir != NULL) { vref(fdp->fd_rdir); export_vnode_to_sb(fdp->fd_rdir, KF_FD_TYPE_ROOT, FREAD, efbuf); } /* jail directory */ if (fdp->fd_jdir != NULL) { vref(fdp->fd_jdir); export_vnode_to_sb(fdp->fd_jdir, KF_FD_TYPE_JAIL, FREAD, efbuf); } for (i = 0; fdp->fd_refcnt > 0 && i <= fdp->fd_lastfile; i++) { if ((fp = fdp->fd_ofiles[i].fde_file) == NULL) continue; #ifdef CAPABILITIES rights = *cap_rights(fdp, i); #else /* !CAPABILITIES */ cap_rights_init(&rights); #endif /* * Create sysctl entry. It is OK to drop the filedesc * lock inside of export_file_to_sb() as we will * re-validate and re-evaluate its properties when the * loop continues. */ error = export_file_to_sb(fp, i, &rights, efbuf); if (error != 0 || efbuf->remainder == 0) break; } FILEDESC_SUNLOCK(fdp); fddrop(fdp); fail: free(efbuf, M_TEMP); return (error); } #define FILEDESC_SBUF_SIZE (sizeof(struct kinfo_file) * 5) /* * Get per-process file descriptors for use by procstat(1), et al. */ static int sysctl_kern_proc_filedesc(SYSCTL_HANDLER_ARGS) { struct sbuf sb; struct proc *p; ssize_t maxlen; int error, error2, *name; name = (int *)arg1; sbuf_new_for_sysctl(&sb, NULL, FILEDESC_SBUF_SIZE, req); sbuf_clear_flags(&sb, SBUF_INCLUDENUL); error = pget((pid_t)name[0], PGET_CANDEBUG | PGET_NOTWEXIT, &p); if (error != 0) { sbuf_delete(&sb); return (error); } maxlen = req->oldptr != NULL ? req->oldlen : -1; error = kern_proc_filedesc_out(p, &sb, maxlen, KERN_FILEDESC_PACK_KINFO); error2 = sbuf_finish(&sb); sbuf_delete(&sb); return (error != 0 ? error : error2); } #ifdef KINFO_OFILE_SIZE CTASSERT(sizeof(struct kinfo_ofile) == KINFO_OFILE_SIZE); #endif #ifdef COMPAT_FREEBSD7 static void kinfo_to_okinfo(struct kinfo_file *kif, struct kinfo_ofile *okif) { okif->kf_structsize = sizeof(*okif); okif->kf_type = kif->kf_type; okif->kf_fd = kif->kf_fd; okif->kf_ref_count = kif->kf_ref_count; okif->kf_flags = kif->kf_flags & (KF_FLAG_READ | KF_FLAG_WRITE | KF_FLAG_APPEND | KF_FLAG_ASYNC | KF_FLAG_FSYNC | KF_FLAG_NONBLOCK | KF_FLAG_DIRECT | KF_FLAG_HASLOCK); okif->kf_offset = kif->kf_offset; okif->kf_vnode_type = kif->kf_vnode_type; okif->kf_sock_domain = kif->kf_sock_domain; okif->kf_sock_type = kif->kf_sock_type; okif->kf_sock_protocol = kif->kf_sock_protocol; strlcpy(okif->kf_path, kif->kf_path, sizeof(okif->kf_path)); okif->kf_sa_local = kif->kf_sa_local; okif->kf_sa_peer = kif->kf_sa_peer; } static int export_vnode_for_osysctl(struct vnode *vp, int type, struct kinfo_file *kif, struct kinfo_ofile *okif, struct filedesc *fdp, struct sysctl_req *req) { int error; vref(vp); FILEDESC_SUNLOCK(fdp); export_vnode_to_kinfo(vp, type, 0, kif, KERN_FILEDESC_PACK_KINFO); kinfo_to_okinfo(kif, okif); error = SYSCTL_OUT(req, okif, sizeof(*okif)); FILEDESC_SLOCK(fdp); return (error); } /* * Get per-process file descriptors for use by procstat(1), et al. */ static int sysctl_kern_proc_ofiledesc(SYSCTL_HANDLER_ARGS) { struct kinfo_ofile *okif; struct kinfo_file *kif; struct filedesc *fdp; int error, i, *name; struct file *fp; struct proc *p; name = (int *)arg1; error = pget((pid_t)name[0], PGET_CANDEBUG | PGET_NOTWEXIT, &p); if (error != 0) return (error); fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) return (ENOENT); kif = malloc(sizeof(*kif), M_TEMP, M_WAITOK); okif = malloc(sizeof(*okif), M_TEMP, M_WAITOK); FILEDESC_SLOCK(fdp); if (fdp->fd_cdir != NULL) export_vnode_for_osysctl(fdp->fd_cdir, KF_FD_TYPE_CWD, kif, okif, fdp, req); if (fdp->fd_rdir != NULL) export_vnode_for_osysctl(fdp->fd_rdir, KF_FD_TYPE_ROOT, kif, okif, fdp, req); if (fdp->fd_jdir != NULL) export_vnode_for_osysctl(fdp->fd_jdir, KF_FD_TYPE_JAIL, kif, okif, fdp, req); for (i = 0; fdp->fd_refcnt > 0 && i <= fdp->fd_lastfile; i++) { if ((fp = fdp->fd_ofiles[i].fde_file) == NULL) continue; export_file_to_kinfo(fp, i, NULL, kif, fdp, KERN_FILEDESC_PACK_KINFO); FILEDESC_SUNLOCK(fdp); kinfo_to_okinfo(kif, okif); error = SYSCTL_OUT(req, okif, sizeof(*okif)); FILEDESC_SLOCK(fdp); if (error) break; } FILEDESC_SUNLOCK(fdp); fddrop(fdp); free(kif, M_TEMP); free(okif, M_TEMP); return (0); } static SYSCTL_NODE(_kern_proc, KERN_PROC_OFILEDESC, ofiledesc, CTLFLAG_RD|CTLFLAG_MPSAFE, sysctl_kern_proc_ofiledesc, "Process ofiledesc entries"); #endif /* COMPAT_FREEBSD7 */ int vntype_to_kinfo(int vtype) { struct { int vtype; int kf_vtype; } vtypes_table[] = { { VBAD, KF_VTYPE_VBAD }, { VBLK, KF_VTYPE_VBLK }, { VCHR, KF_VTYPE_VCHR }, { VDIR, KF_VTYPE_VDIR }, { VFIFO, KF_VTYPE_VFIFO }, { VLNK, KF_VTYPE_VLNK }, { VNON, KF_VTYPE_VNON }, { VREG, KF_VTYPE_VREG }, { VSOCK, KF_VTYPE_VSOCK } }; unsigned int i; /* * Perform vtype translation. */ for (i = 0; i < nitems(vtypes_table); i++) if (vtypes_table[i].vtype == vtype) return (vtypes_table[i].kf_vtype); return (KF_VTYPE_UNKNOWN); } static SYSCTL_NODE(_kern_proc, KERN_PROC_FILEDESC, filedesc, CTLFLAG_RD|CTLFLAG_MPSAFE, sysctl_kern_proc_filedesc, "Process filedesc entries"); /* * Store a process current working directory information to sbuf. * * Takes a locked proc as argument, and returns with the proc unlocked. */ int kern_proc_cwd_out(struct proc *p, struct sbuf *sb, ssize_t maxlen) { struct filedesc *fdp; struct export_fd_buf *efbuf; int error; PROC_LOCK_ASSERT(p, MA_OWNED); fdp = fdhold(p); PROC_UNLOCK(p); if (fdp == NULL) return (EINVAL); efbuf = malloc(sizeof(*efbuf), M_TEMP, M_WAITOK); efbuf->fdp = fdp; efbuf->sb = sb; efbuf->remainder = maxlen; FILEDESC_SLOCK(fdp); if (fdp->fd_cdir == NULL) error = EINVAL; else { vref(fdp->fd_cdir); error = export_vnode_to_sb(fdp->fd_cdir, KF_FD_TYPE_CWD, FREAD, efbuf); } FILEDESC_SUNLOCK(fdp); fddrop(fdp); free(efbuf, M_TEMP); return (error); } /* * Get per-process current working directory. */ static int sysctl_kern_proc_cwd(SYSCTL_HANDLER_ARGS) { struct sbuf sb; struct proc *p; ssize_t maxlen; int error, error2, *name; name = (int *)arg1; sbuf_new_for_sysctl(&sb, NULL, sizeof(struct kinfo_file), req); sbuf_clear_flags(&sb, SBUF_INCLUDENUL); error = pget((pid_t)name[0], PGET_CANDEBUG | PGET_NOTWEXIT, &p); if (error != 0) { sbuf_delete(&sb); return (error); } maxlen = req->oldptr != NULL ? req->oldlen : -1; error = kern_proc_cwd_out(p, &sb, maxlen); error2 = sbuf_finish(&sb); sbuf_delete(&sb); return (error != 0 ? error : error2); } static SYSCTL_NODE(_kern_proc, KERN_PROC_CWD, cwd, CTLFLAG_RD|CTLFLAG_MPSAFE, sysctl_kern_proc_cwd, "Process current working directory"); #ifdef DDB /* * For the purposes of debugging, generate a human-readable string for the * file type. */ static const char * file_type_to_name(short type) { switch (type) { case 0: return ("zero"); case DTYPE_VNODE: return ("vnod"); case DTYPE_SOCKET: return ("sock"); case DTYPE_PIPE: return ("pipe"); case DTYPE_FIFO: return ("fifo"); case DTYPE_KQUEUE: return ("kque"); case DTYPE_CRYPTO: return ("crpt"); case DTYPE_MQUEUE: return ("mque"); case DTYPE_SHM: return ("shm"); case DTYPE_SEM: return ("ksem"); default: return ("unkn"); } } /* * For the purposes of debugging, identify a process (if any, perhaps one of * many) that references the passed file in its file descriptor array. Return * NULL if none. */ static struct proc * file_to_first_proc(struct file *fp) { struct filedesc *fdp; struct proc *p; int n; FOREACH_PROC_IN_SYSTEM(p) { if (p->p_state == PRS_NEW) continue; fdp = p->p_fd; if (fdp == NULL) continue; for (n = 0; n <= fdp->fd_lastfile; n++) { if (fp == fdp->fd_ofiles[n].fde_file) return (p); } } return (NULL); } static void db_print_file(struct file *fp, int header) { struct proc *p; if (header) db_printf("%8s %4s %8s %8s %4s %5s %6s %8s %5s %12s\n", "File", "Type", "Data", "Flag", "GCFl", "Count", "MCount", "Vnode", "FPID", "FCmd"); p = file_to_first_proc(fp); db_printf("%8p %4s %8p %08x %04x %5d %6d %8p %5d %12s\n", fp, file_type_to_name(fp->f_type), fp->f_data, fp->f_flag, 0, fp->f_count, 0, fp->f_vnode, p != NULL ? p->p_pid : -1, p != NULL ? p->p_comm : "-"); } DB_SHOW_COMMAND(file, db_show_file) { struct file *fp; if (!have_addr) { db_printf("usage: show file \n"); return; } fp = (struct file *)addr; db_print_file(fp, 1); } DB_SHOW_COMMAND(files, db_show_files) { struct filedesc *fdp; struct file *fp; struct proc *p; int header; int n; header = 1; FOREACH_PROC_IN_SYSTEM(p) { if (p->p_state == PRS_NEW) continue; if ((fdp = p->p_fd) == NULL) continue; for (n = 0; n <= fdp->fd_lastfile; ++n) { if ((fp = fdp->fd_ofiles[n].fde_file) == NULL) continue; db_print_file(fp, header); header = 0; } } } #endif SYSCTL_INT(_kern, KERN_MAXFILESPERPROC, maxfilesperproc, CTLFLAG_RW, &maxfilesperproc, 0, "Maximum files allowed open per process"); SYSCTL_INT(_kern, KERN_MAXFILES, maxfiles, CTLFLAG_RW, &maxfiles, 0, "Maximum number of files"); SYSCTL_INT(_kern, OID_AUTO, openfiles, CTLFLAG_RD, __DEVOLATILE(int *, &openfiles), 0, "System-wide number of open files"); /* ARGSUSED*/ static void filelistinit(void *dummy) { file_zone = uma_zcreate("Files", sizeof(struct file), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE); filedesc0_zone = uma_zcreate("filedesc0", sizeof(struct filedesc0), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); mtx_init(&sigio_lock, "sigio lock", NULL, MTX_DEF); } SYSINIT(select, SI_SUB_LOCK, SI_ORDER_FIRST, filelistinit, NULL); /*-------------------------------------------------------------------*/ static int badfo_readwrite(struct file *fp, struct uio *uio, struct ucred *active_cred, int flags, struct thread *td) { return (EBADF); } static int badfo_truncate(struct file *fp, off_t length, struct ucred *active_cred, struct thread *td) { return (EINVAL); } static int badfo_ioctl(struct file *fp, u_long com, void *data, struct ucred *active_cred, struct thread *td) { return (EBADF); } static int badfo_poll(struct file *fp, int events, struct ucred *active_cred, struct thread *td) { return (0); } static int badfo_kqfilter(struct file *fp, struct knote *kn) { return (EBADF); } static int badfo_stat(struct file *fp, struct stat *sb, struct ucred *active_cred, struct thread *td) { return (EBADF); } static int badfo_close(struct file *fp, struct thread *td) { return (0); } static int badfo_chmod(struct file *fp, mode_t mode, struct ucred *active_cred, struct thread *td) { return (EBADF); } static int badfo_chown(struct file *fp, uid_t uid, gid_t gid, struct ucred *active_cred, struct thread *td) { return (EBADF); } static int badfo_sendfile(struct file *fp, int sockfd, struct uio *hdr_uio, struct uio *trl_uio, off_t offset, size_t nbytes, off_t *sent, int flags, struct thread *td) { return (EBADF); } static int badfo_fill_kinfo(struct file *fp, struct kinfo_file *kif, struct filedesc *fdp) { return (0); } struct fileops badfileops = { .fo_read = badfo_readwrite, .fo_write = badfo_readwrite, .fo_truncate = badfo_truncate, .fo_ioctl = badfo_ioctl, .fo_poll = badfo_poll, .fo_kqfilter = badfo_kqfilter, .fo_stat = badfo_stat, .fo_close = badfo_close, .fo_chmod = badfo_chmod, .fo_chown = badfo_chown, .fo_sendfile = badfo_sendfile, .fo_fill_kinfo = badfo_fill_kinfo, }; int invfo_rdwr(struct file *fp, struct uio *uio, struct ucred *active_cred, int flags, struct thread *td) { return (EOPNOTSUPP); } int invfo_truncate(struct file *fp, off_t length, struct ucred *active_cred, struct thread *td) { return (EINVAL); } int invfo_ioctl(struct file *fp, u_long com, void *data, struct ucred *active_cred, struct thread *td) { return (ENOTTY); } int invfo_poll(struct file *fp, int events, struct ucred *active_cred, struct thread *td) { return (poll_no_poll(events)); } int invfo_kqfilter(struct file *fp, struct knote *kn) { return (EINVAL); } int invfo_chmod(struct file *fp, mode_t mode, struct ucred *active_cred, struct thread *td) { return (EINVAL); } int invfo_chown(struct file *fp, uid_t uid, gid_t gid, struct ucred *active_cred, struct thread *td) { return (EINVAL); } int invfo_sendfile(struct file *fp, int sockfd, struct uio *hdr_uio, struct uio *trl_uio, off_t offset, size_t nbytes, off_t *sent, int flags, struct thread *td) { return (EINVAL); } /*-------------------------------------------------------------------*/ /* * File Descriptor pseudo-device driver (/dev/fd/). * * Opening minor device N dup()s the file (if any) connected to file * descriptor N belonging to the calling process. Note that this driver * consists of only the ``open()'' routine, because all subsequent * references to this file will be direct to the other driver. * * XXX: we could give this one a cloning event handler if necessary. */ /* ARGSUSED */ static int fdopen(struct cdev *dev, int mode, int type, struct thread *td) { /* * XXX Kludge: set curthread->td_dupfd to contain the value of the * the file descriptor being sought for duplication. The error * return ensures that the vnode for this device will be released * by vn_open. Open will detect this special error and take the * actions in dupfdopen below. Other callers of vn_open or VOP_OPEN * will simply report the error. */ td->td_dupfd = dev2unit(dev); return (ENODEV); } static struct cdevsw fildesc_cdevsw = { .d_version = D_VERSION, .d_open = fdopen, .d_name = "FD", }; static void fildesc_drvinit(void *unused) { struct cdev *dev; dev = make_dev_credf(MAKEDEV_ETERNAL, &fildesc_cdevsw, 0, NULL, UID_ROOT, GID_WHEEL, 0666, "fd/0"); make_dev_alias(dev, "stdin"); dev = make_dev_credf(MAKEDEV_ETERNAL, &fildesc_cdevsw, 1, NULL, UID_ROOT, GID_WHEEL, 0666, "fd/1"); make_dev_alias(dev, "stdout"); dev = make_dev_credf(MAKEDEV_ETERNAL, &fildesc_cdevsw, 2, NULL, UID_ROOT, GID_WHEEL, 0666, "fd/2"); make_dev_alias(dev, "stderr"); } SYSINIT(fildescdev, SI_SUB_DRIVERS, SI_ORDER_MIDDLE, fildesc_drvinit, NULL); Index: projects/netbsd-tests-update-12/sys/kern/subr_gtaskqueue.c =================================================================== --- projects/netbsd-tests-update-12/sys/kern/subr_gtaskqueue.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/kern/subr_gtaskqueue.c (revision 305172) @@ -1,864 +1,867 @@ /*- * Copyright (c) 2000 Doug Rabson * Copyright (c) 2014 Jeff Roberson * Copyright (c) 2016 Matthew Macy * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static MALLOC_DEFINE(M_GTASKQUEUE, "taskqueue", "Task Queues"); static void gtaskqueue_thread_enqueue(void *); static void gtaskqueue_thread_loop(void *arg); struct gtaskqueue_busy { struct gtask *tb_running; TAILQ_ENTRY(gtaskqueue_busy) tb_link; }; static struct gtask * const TB_DRAIN_WAITER = (struct gtask *)0x1; struct gtaskqueue { STAILQ_HEAD(, gtask) tq_queue; gtaskqueue_enqueue_fn tq_enqueue; void *tq_context; char *tq_name; TAILQ_HEAD(, gtaskqueue_busy) tq_active; struct mtx tq_mutex; struct thread **tq_threads; int tq_tcount; int tq_spin; int tq_flags; int tq_callouts; taskqueue_callback_fn tq_callbacks[TASKQUEUE_NUM_CALLBACKS]; void *tq_cb_contexts[TASKQUEUE_NUM_CALLBACKS]; }; #define TQ_FLAGS_ACTIVE (1 << 0) #define TQ_FLAGS_BLOCKED (1 << 1) #define TQ_FLAGS_UNLOCKED_ENQUEUE (1 << 2) #define DT_CALLOUT_ARMED (1 << 0) #define TQ_LOCK(tq) \ do { \ if ((tq)->tq_spin) \ mtx_lock_spin(&(tq)->tq_mutex); \ else \ mtx_lock(&(tq)->tq_mutex); \ } while (0) #define TQ_ASSERT_LOCKED(tq) mtx_assert(&(tq)->tq_mutex, MA_OWNED) #define TQ_UNLOCK(tq) \ do { \ if ((tq)->tq_spin) \ mtx_unlock_spin(&(tq)->tq_mutex); \ else \ mtx_unlock(&(tq)->tq_mutex); \ } while (0) #define TQ_ASSERT_UNLOCKED(tq) mtx_assert(&(tq)->tq_mutex, MA_NOTOWNED) static __inline int TQ_SLEEP(struct gtaskqueue *tq, void *p, struct mtx *m, int pri, const char *wm, int t) { if (tq->tq_spin) return (msleep_spin(p, m, wm, t)); return (msleep(p, m, pri, wm, t)); } static struct gtaskqueue * _gtaskqueue_create(const char *name, int mflags, taskqueue_enqueue_fn enqueue, void *context, int mtxflags, const char *mtxname __unused) { struct gtaskqueue *queue; char *tq_name; tq_name = malloc(TASKQUEUE_NAMELEN, M_GTASKQUEUE, mflags | M_ZERO); if (!tq_name) return (NULL); snprintf(tq_name, TASKQUEUE_NAMELEN, "%s", (name) ? name : "taskqueue"); queue = malloc(sizeof(struct gtaskqueue), M_GTASKQUEUE, mflags | M_ZERO); if (!queue) return (NULL); STAILQ_INIT(&queue->tq_queue); TAILQ_INIT(&queue->tq_active); queue->tq_enqueue = enqueue; queue->tq_context = context; queue->tq_name = tq_name; queue->tq_spin = (mtxflags & MTX_SPIN) != 0; queue->tq_flags |= TQ_FLAGS_ACTIVE; if (enqueue == gtaskqueue_thread_enqueue) queue->tq_flags |= TQ_FLAGS_UNLOCKED_ENQUEUE; mtx_init(&queue->tq_mutex, tq_name, NULL, mtxflags); return (queue); } /* * Signal a taskqueue thread to terminate. */ static void gtaskqueue_terminate(struct thread **pp, struct gtaskqueue *tq) { while (tq->tq_tcount > 0 || tq->tq_callouts > 0) { wakeup(tq); TQ_SLEEP(tq, pp, &tq->tq_mutex, PWAIT, "taskqueue_destroy", 0); } } static void gtaskqueue_free(struct gtaskqueue *queue) { TQ_LOCK(queue); queue->tq_flags &= ~TQ_FLAGS_ACTIVE; gtaskqueue_terminate(queue->tq_threads, queue); KASSERT(TAILQ_EMPTY(&queue->tq_active), ("Tasks still running?")); KASSERT(queue->tq_callouts == 0, ("Armed timeout tasks")); mtx_destroy(&queue->tq_mutex); free(queue->tq_threads, M_GTASKQUEUE); free(queue->tq_name, M_GTASKQUEUE); free(queue, M_GTASKQUEUE); } int grouptaskqueue_enqueue(struct gtaskqueue *queue, struct gtask *gtask) { TQ_LOCK(queue); if (gtask->ta_flags & TASK_ENQUEUED) { TQ_UNLOCK(queue); return (0); } STAILQ_INSERT_TAIL(&queue->tq_queue, gtask, ta_link); gtask->ta_flags |= TASK_ENQUEUED; TQ_UNLOCK(queue); if ((queue->tq_flags & TQ_FLAGS_BLOCKED) == 0) queue->tq_enqueue(queue->tq_context); return (0); } static void gtaskqueue_task_nop_fn(void *context) { } /* * Block until all currently queued tasks in this taskqueue * have begun execution. Tasks queued during execution of * this function are ignored. */ static void gtaskqueue_drain_tq_queue(struct gtaskqueue *queue) { struct gtask t_barrier; if (STAILQ_EMPTY(&queue->tq_queue)) return; /* * Enqueue our barrier after all current tasks, but with * the highest priority so that newly queued tasks cannot * pass it. Because of the high priority, we can not use * taskqueue_enqueue_locked directly (which drops the lock * anyway) so just insert it at tail while we have the * queue lock. */ GTASK_INIT(&t_barrier, 0, USHRT_MAX, gtaskqueue_task_nop_fn, &t_barrier); STAILQ_INSERT_TAIL(&queue->tq_queue, &t_barrier, ta_link); t_barrier.ta_flags |= TASK_ENQUEUED; /* * Once the barrier has executed, all previously queued tasks * have completed or are currently executing. */ while (t_barrier.ta_flags & TASK_ENQUEUED) TQ_SLEEP(queue, &t_barrier, &queue->tq_mutex, PWAIT, "-", 0); } /* * Block until all currently executing tasks for this taskqueue * complete. Tasks that begin execution during the execution * of this function are ignored. */ static void gtaskqueue_drain_tq_active(struct gtaskqueue *queue) { struct gtaskqueue_busy tb_marker, *tb_first; if (TAILQ_EMPTY(&queue->tq_active)) return; /* Block taskq_terminate().*/ queue->tq_callouts++; /* * Wait for all currently executing taskqueue threads * to go idle. */ tb_marker.tb_running = TB_DRAIN_WAITER; TAILQ_INSERT_TAIL(&queue->tq_active, &tb_marker, tb_link); while (TAILQ_FIRST(&queue->tq_active) != &tb_marker) TQ_SLEEP(queue, &tb_marker, &queue->tq_mutex, PWAIT, "-", 0); TAILQ_REMOVE(&queue->tq_active, &tb_marker, tb_link); /* * Wakeup any other drain waiter that happened to queue up * without any intervening active thread. */ tb_first = TAILQ_FIRST(&queue->tq_active); if (tb_first != NULL && tb_first->tb_running == TB_DRAIN_WAITER) wakeup(tb_first); /* Release taskqueue_terminate(). */ queue->tq_callouts--; if ((queue->tq_flags & TQ_FLAGS_ACTIVE) == 0) wakeup_one(queue->tq_threads); } void gtaskqueue_block(struct gtaskqueue *queue) { TQ_LOCK(queue); queue->tq_flags |= TQ_FLAGS_BLOCKED; TQ_UNLOCK(queue); } void gtaskqueue_unblock(struct gtaskqueue *queue) { TQ_LOCK(queue); queue->tq_flags &= ~TQ_FLAGS_BLOCKED; if (!STAILQ_EMPTY(&queue->tq_queue)) queue->tq_enqueue(queue->tq_context); TQ_UNLOCK(queue); } static void gtaskqueue_run_locked(struct gtaskqueue *queue) { struct gtaskqueue_busy tb; struct gtaskqueue_busy *tb_first; struct gtask *gtask; KASSERT(queue != NULL, ("tq is NULL")); TQ_ASSERT_LOCKED(queue); tb.tb_running = NULL; while (STAILQ_FIRST(&queue->tq_queue)) { TAILQ_INSERT_TAIL(&queue->tq_active, &tb, tb_link); /* * Carefully remove the first task from the queue and * clear its TASK_ENQUEUED flag */ gtask = STAILQ_FIRST(&queue->tq_queue); KASSERT(gtask != NULL, ("task is NULL")); STAILQ_REMOVE_HEAD(&queue->tq_queue, ta_link); gtask->ta_flags &= ~TASK_ENQUEUED; tb.tb_running = gtask; TQ_UNLOCK(queue); KASSERT(gtask->ta_func != NULL, ("task->ta_func is NULL")); gtask->ta_func(gtask->ta_context); TQ_LOCK(queue); tb.tb_running = NULL; wakeup(gtask); TAILQ_REMOVE(&queue->tq_active, &tb, tb_link); tb_first = TAILQ_FIRST(&queue->tq_active); if (tb_first != NULL && tb_first->tb_running == TB_DRAIN_WAITER) wakeup(tb_first); } } static int task_is_running(struct gtaskqueue *queue, struct gtask *gtask) { struct gtaskqueue_busy *tb; TQ_ASSERT_LOCKED(queue); TAILQ_FOREACH(tb, &queue->tq_active, tb_link) { if (tb->tb_running == gtask) return (1); } return (0); } static int gtaskqueue_cancel_locked(struct gtaskqueue *queue, struct gtask *gtask) { if (gtask->ta_flags & TASK_ENQUEUED) STAILQ_REMOVE(&queue->tq_queue, gtask, gtask, ta_link); gtask->ta_flags &= ~TASK_ENQUEUED; return (task_is_running(queue, gtask) ? EBUSY : 0); } int gtaskqueue_cancel(struct gtaskqueue *queue, struct gtask *gtask) { int error; TQ_LOCK(queue); error = gtaskqueue_cancel_locked(queue, gtask); TQ_UNLOCK(queue); return (error); } void gtaskqueue_drain(struct gtaskqueue *queue, struct gtask *gtask) { if (!queue->tq_spin) WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, NULL, __func__); TQ_LOCK(queue); while ((gtask->ta_flags & TASK_ENQUEUED) || task_is_running(queue, gtask)) TQ_SLEEP(queue, gtask, &queue->tq_mutex, PWAIT, "-", 0); TQ_UNLOCK(queue); } void gtaskqueue_drain_all(struct gtaskqueue *queue) { if (!queue->tq_spin) WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, NULL, __func__); TQ_LOCK(queue); gtaskqueue_drain_tq_queue(queue); gtaskqueue_drain_tq_active(queue); TQ_UNLOCK(queue); } static int _gtaskqueue_start_threads(struct gtaskqueue **tqp, int count, int pri, cpuset_t *mask, const char *name, va_list ap) { char ktname[MAXCOMLEN + 1]; struct thread *td; struct gtaskqueue *tq; int i, error; if (count <= 0) return (EINVAL); vsnprintf(ktname, sizeof(ktname), name, ap); tq = *tqp; tq->tq_threads = malloc(sizeof(struct thread *) * count, M_GTASKQUEUE, M_NOWAIT | M_ZERO); if (tq->tq_threads == NULL) { printf("%s: no memory for %s threads\n", __func__, ktname); return (ENOMEM); } for (i = 0; i < count; i++) { if (count == 1) error = kthread_add(gtaskqueue_thread_loop, tqp, NULL, &tq->tq_threads[i], RFSTOPPED, 0, "%s", ktname); else error = kthread_add(gtaskqueue_thread_loop, tqp, NULL, &tq->tq_threads[i], RFSTOPPED, 0, "%s_%d", ktname, i); if (error) { /* should be ok to continue, taskqueue_free will dtrt */ printf("%s: kthread_add(%s): error %d", __func__, ktname, error); tq->tq_threads[i] = NULL; /* paranoid */ } else tq->tq_tcount++; } for (i = 0; i < count; i++) { if (tq->tq_threads[i] == NULL) continue; td = tq->tq_threads[i]; if (mask) { error = cpuset_setthread(td->td_tid, mask); /* * Failing to pin is rarely an actual fatal error; * it'll just affect performance. */ if (error) printf("%s: curthread=%llu: can't pin; " "error=%d\n", __func__, (unsigned long long) td->td_tid, error); } thread_lock(td); sched_prio(td, pri); sched_add(td, SRQ_BORING); thread_unlock(td); } return (0); } static int gtaskqueue_start_threads(struct gtaskqueue **tqp, int count, int pri, const char *name, ...) { va_list ap; int error; va_start(ap, name); error = _gtaskqueue_start_threads(tqp, count, pri, NULL, name, ap); va_end(ap); return (error); } static inline void gtaskqueue_run_callback(struct gtaskqueue *tq, enum taskqueue_callback_type cb_type) { taskqueue_callback_fn tq_callback; TQ_ASSERT_UNLOCKED(tq); tq_callback = tq->tq_callbacks[cb_type]; if (tq_callback != NULL) tq_callback(tq->tq_cb_contexts[cb_type]); } static void gtaskqueue_thread_loop(void *arg) { struct gtaskqueue **tqp, *tq; tqp = arg; tq = *tqp; gtaskqueue_run_callback(tq, TASKQUEUE_CALLBACK_TYPE_INIT); TQ_LOCK(tq); while ((tq->tq_flags & TQ_FLAGS_ACTIVE) != 0) { /* XXX ? */ gtaskqueue_run_locked(tq); /* * Because taskqueue_run() can drop tq_mutex, we need to * check if the TQ_FLAGS_ACTIVE flag wasn't removed in the * meantime, which means we missed a wakeup. */ if ((tq->tq_flags & TQ_FLAGS_ACTIVE) == 0) break; TQ_SLEEP(tq, tq, &tq->tq_mutex, 0, "-", 0); } gtaskqueue_run_locked(tq); /* * This thread is on its way out, so just drop the lock temporarily * in order to call the shutdown callback. This allows the callback * to look at the taskqueue, even just before it dies. */ TQ_UNLOCK(tq); gtaskqueue_run_callback(tq, TASKQUEUE_CALLBACK_TYPE_SHUTDOWN); TQ_LOCK(tq); /* rendezvous with thread that asked us to terminate */ tq->tq_tcount--; wakeup_one(tq->tq_threads); TQ_UNLOCK(tq); kthread_exit(); } static void gtaskqueue_thread_enqueue(void *context) { struct gtaskqueue **tqp, *tq; tqp = context; tq = *tqp; wakeup_one(tq); } static struct gtaskqueue * gtaskqueue_create_fast(const char *name, int mflags, taskqueue_enqueue_fn enqueue, void *context) { return _gtaskqueue_create(name, mflags, enqueue, context, MTX_SPIN, "fast_taskqueue"); } struct taskqgroup_cpu { LIST_HEAD(, grouptask) tgc_tasks; struct gtaskqueue *tgc_taskq; int tgc_cnt; int tgc_cpu; }; struct taskqgroup { struct taskqgroup_cpu tqg_queue[MAXCPU]; struct mtx tqg_lock; char * tqg_name; int tqg_adjusting; int tqg_stride; int tqg_cnt; }; struct taskq_bind_task { struct gtask bt_task; int bt_cpuid; }; static void taskqgroup_cpu_create(struct taskqgroup *qgroup, int idx) { struct taskqgroup_cpu *qcpu; qcpu = &qgroup->tqg_queue[idx]; LIST_INIT(&qcpu->tgc_tasks); qcpu->tgc_taskq = gtaskqueue_create_fast(NULL, M_WAITOK, taskqueue_thread_enqueue, &qcpu->tgc_taskq); gtaskqueue_start_threads(&qcpu->tgc_taskq, 1, PI_SOFT, "%s_%d", qgroup->tqg_name, idx); qcpu->tgc_cpu = idx * qgroup->tqg_stride; } static void taskqgroup_cpu_remove(struct taskqgroup *qgroup, int idx) { gtaskqueue_free(qgroup->tqg_queue[idx].tgc_taskq); } /* * Find the taskq with least # of tasks that doesn't currently have any * other queues from the uniq identifier. */ static int taskqgroup_find(struct taskqgroup *qgroup, void *uniq) { struct grouptask *n; int i, idx, mincnt; int strict; mtx_assert(&qgroup->tqg_lock, MA_OWNED); if (qgroup->tqg_cnt == 0) return (0); idx = -1; mincnt = INT_MAX; /* * Two passes; First scan for a queue with the least tasks that * does not already service this uniq id. If that fails simply find * the queue with the least total tasks; */ for (strict = 1; mincnt == INT_MAX; strict = 0) { for (i = 0; i < qgroup->tqg_cnt; i++) { if (qgroup->tqg_queue[i].tgc_cnt > mincnt) continue; if (strict) { LIST_FOREACH(n, &qgroup->tqg_queue[i].tgc_tasks, gt_list) if (n->gt_uniq == uniq) break; if (n != NULL) continue; } mincnt = qgroup->tqg_queue[i].tgc_cnt; idx = i; } } if (idx == -1) panic("taskqgroup_find: Failed to pick a qid."); return (idx); } void taskqgroup_attach(struct taskqgroup *qgroup, struct grouptask *gtask, void *uniq, int irq, char *name) { cpuset_t mask; int qid; gtask->gt_uniq = uniq; gtask->gt_name = name; gtask->gt_irq = irq; gtask->gt_cpu = -1; mtx_lock(&qgroup->tqg_lock); qid = taskqgroup_find(qgroup, uniq); qgroup->tqg_queue[qid].tgc_cnt++; LIST_INSERT_HEAD(&qgroup->tqg_queue[qid].tgc_tasks, gtask, gt_list); gtask->gt_taskqueue = qgroup->tqg_queue[qid].tgc_taskq; if (irq != -1 && smp_started) { CPU_ZERO(&mask); CPU_SET(qgroup->tqg_queue[qid].tgc_cpu, &mask); mtx_unlock(&qgroup->tqg_lock); intr_setaffinity(irq, &mask); } else mtx_unlock(&qgroup->tqg_lock); } int taskqgroup_attach_cpu(struct taskqgroup *qgroup, struct grouptask *gtask, void *uniq, int cpu, int irq, char *name) { cpuset_t mask; int i, qid; qid = -1; gtask->gt_uniq = uniq; gtask->gt_name = name; gtask->gt_irq = irq; gtask->gt_cpu = cpu; mtx_lock(&qgroup->tqg_lock); if (smp_started) { for (i = 0; i < qgroup->tqg_cnt; i++) if (qgroup->tqg_queue[i].tgc_cpu == cpu) { qid = i; break; } if (qid == -1) { mtx_unlock(&qgroup->tqg_lock); return (EINVAL); } } else qid = 0; qgroup->tqg_queue[qid].tgc_cnt++; LIST_INSERT_HEAD(&qgroup->tqg_queue[qid].tgc_tasks, gtask, gt_list); gtask->gt_taskqueue = qgroup->tqg_queue[qid].tgc_taskq; if (irq != -1 && smp_started) { CPU_ZERO(&mask); CPU_SET(qgroup->tqg_queue[qid].tgc_cpu, &mask); mtx_unlock(&qgroup->tqg_lock); intr_setaffinity(irq, &mask); } else mtx_unlock(&qgroup->tqg_lock); return (0); } void taskqgroup_detach(struct taskqgroup *qgroup, struct grouptask *gtask) { int i; mtx_lock(&qgroup->tqg_lock); for (i = 0; i < qgroup->tqg_cnt; i++) if (qgroup->tqg_queue[i].tgc_taskq == gtask->gt_taskqueue) break; if (i == qgroup->tqg_cnt) panic("taskqgroup_detach: task not in group\n"); qgroup->tqg_queue[i].tgc_cnt--; LIST_REMOVE(gtask, gt_list); mtx_unlock(&qgroup->tqg_lock); gtask->gt_taskqueue = NULL; } static void taskqgroup_binder(void *ctx) { struct taskq_bind_task *gtask = (struct taskq_bind_task *)ctx; cpuset_t mask; int error; CPU_ZERO(&mask); CPU_SET(gtask->bt_cpuid, &mask); error = cpuset_setthread(curthread->td_tid, &mask); thread_lock(curthread); sched_bind(curthread, gtask->bt_cpuid); thread_unlock(curthread); if (error) printf("taskqgroup_binder: setaffinity failed: %d\n", error); free(gtask, M_DEVBUF); } static void taskqgroup_bind(struct taskqgroup *qgroup) { struct taskq_bind_task *gtask; int i; /* * Bind taskqueue threads to specific CPUs, if they have been assigned * one. */ for (i = 0; i < qgroup->tqg_cnt; i++) { gtask = malloc(sizeof (*gtask), M_DEVBUF, M_NOWAIT); GTASK_INIT(>ask->bt_task, 0, 0, taskqgroup_binder, gtask); gtask->bt_cpuid = qgroup->tqg_queue[i].tgc_cpu; grouptaskqueue_enqueue(qgroup->tqg_queue[i].tgc_taskq, >ask->bt_task); } } static int _taskqgroup_adjust(struct taskqgroup *qgroup, int cnt, int stride) { LIST_HEAD(, grouptask) gtask_head = LIST_HEAD_INITIALIZER(NULL); cpuset_t mask; struct grouptask *gtask; - int i, old_cnt, qid; + int i, k, old_cnt, qid, cpu; mtx_assert(&qgroup->tqg_lock, MA_OWNED); if (cnt < 1 || cnt * stride > mp_ncpus || !smp_started) { printf("taskqgroup_adjust failed cnt: %d stride: %d mp_ncpus: %d smp_started: %d\n", cnt, stride, mp_ncpus, smp_started); return (EINVAL); } if (qgroup->tqg_adjusting) { printf("taskqgroup_adjust failed: adjusting\n"); return (EBUSY); } qgroup->tqg_adjusting = 1; old_cnt = qgroup->tqg_cnt; mtx_unlock(&qgroup->tqg_lock); /* * Set up queue for tasks added before boot. */ if (old_cnt == 0) { LIST_SWAP(>ask_head, &qgroup->tqg_queue[0].tgc_tasks, grouptask, gt_list); qgroup->tqg_queue[0].tgc_cnt = 0; } /* * If new taskq threads have been added. */ for (i = old_cnt; i < cnt; i++) taskqgroup_cpu_create(qgroup, i); mtx_lock(&qgroup->tqg_lock); qgroup->tqg_cnt = cnt; qgroup->tqg_stride = stride; /* * Adjust drivers to use new taskqs. */ for (i = 0; i < old_cnt; i++) { while ((gtask = LIST_FIRST(&qgroup->tqg_queue[i].tgc_tasks))) { LIST_REMOVE(gtask, gt_list); qgroup->tqg_queue[i].tgc_cnt--; LIST_INSERT_HEAD(>ask_head, gtask, gt_list); } } while ((gtask = LIST_FIRST(>ask_head))) { LIST_REMOVE(gtask, gt_list); if (gtask->gt_cpu == -1) qid = taskqgroup_find(qgroup, gtask->gt_uniq); else { for (i = 0; i < qgroup->tqg_cnt; i++) if (qgroup->tqg_queue[i].tgc_cpu == gtask->gt_cpu) { qid = i; break; } } qgroup->tqg_queue[qid].tgc_cnt++; LIST_INSERT_HEAD(&qgroup->tqg_queue[qid].tgc_tasks, gtask, gt_list); gtask->gt_taskqueue = qgroup->tqg_queue[qid].tgc_taskq; } /* * Set new CPU and IRQ affinity */ + cpu = CPU_FIRST(); for (i = 0; i < cnt; i++) { - qgroup->tqg_queue[i].tgc_cpu = i * qgroup->tqg_stride; + qgroup->tqg_queue[i].tgc_cpu = cpu; + for (k = 0; k < qgroup->tqg_stride; k++) + cpu = CPU_NEXT(cpu); CPU_ZERO(&mask); CPU_SET(qgroup->tqg_queue[i].tgc_cpu, &mask); LIST_FOREACH(gtask, &qgroup->tqg_queue[i].tgc_tasks, gt_list) { if (gtask->gt_irq == -1) continue; intr_setaffinity(gtask->gt_irq, &mask); } } mtx_unlock(&qgroup->tqg_lock); /* * If taskq thread count has been reduced. */ for (i = cnt; i < old_cnt; i++) taskqgroup_cpu_remove(qgroup, i); mtx_lock(&qgroup->tqg_lock); qgroup->tqg_adjusting = 0; taskqgroup_bind(qgroup); return (0); } int taskqgroup_adjust(struct taskqgroup *qgroup, int cpu, int stride) { int error; mtx_lock(&qgroup->tqg_lock); error = _taskqgroup_adjust(qgroup, cpu, stride); mtx_unlock(&qgroup->tqg_lock); return (error); } struct taskqgroup * taskqgroup_create(char *name) { struct taskqgroup *qgroup; qgroup = malloc(sizeof(*qgroup), M_GTASKQUEUE, M_WAITOK | M_ZERO); mtx_init(&qgroup->tqg_lock, "taskqgroup", NULL, MTX_DEF); qgroup->tqg_name = name; LIST_INIT(&qgroup->tqg_queue[0].tgc_tasks); return (qgroup); } void taskqgroup_destroy(struct taskqgroup *qgroup) { } Index: projects/netbsd-tests-update-12/sys/kern/subr_smp.c =================================================================== --- projects/netbsd-tests-update-12/sys/kern/subr_smp.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/kern/subr_smp.c (revision 305172) @@ -1,1140 +1,1133 @@ /*- * Copyright (c) 2001, John Baldwin . * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * This module holds the global variables and machine independent functions * used for the kernel SMP support. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "opt_sched.h" #ifdef SMP MALLOC_DEFINE(M_TOPO, "toponodes", "SMP topology data"); volatile cpuset_t stopped_cpus; volatile cpuset_t started_cpus; volatile cpuset_t suspended_cpus; cpuset_t hlt_cpus_mask; cpuset_t logical_cpus_mask; void (*cpustop_restartfunc)(void); #endif static int sysctl_kern_smp_active(SYSCTL_HANDLER_ARGS); /* This is used in modules that need to work in both SMP and UP. */ cpuset_t all_cpus; int mp_ncpus; /* export this for libkvm consumers. */ int mp_maxcpus = MAXCPU; volatile int smp_started; u_int mp_maxid; static SYSCTL_NODE(_kern, OID_AUTO, smp, CTLFLAG_RD|CTLFLAG_CAPRD, NULL, "Kernel SMP"); SYSCTL_INT(_kern_smp, OID_AUTO, maxid, CTLFLAG_RD|CTLFLAG_CAPRD, &mp_maxid, 0, "Max CPU ID."); SYSCTL_INT(_kern_smp, OID_AUTO, maxcpus, CTLFLAG_RD|CTLFLAG_CAPRD, &mp_maxcpus, 0, "Max number of CPUs that the system was compiled for."); SYSCTL_PROC(_kern_smp, OID_AUTO, active, CTLFLAG_RD | CTLTYPE_INT, NULL, 0, sysctl_kern_smp_active, "I", "Indicates system is running in SMP mode"); int smp_disabled = 0; /* has smp been disabled? */ SYSCTL_INT(_kern_smp, OID_AUTO, disabled, CTLFLAG_RDTUN|CTLFLAG_CAPRD, &smp_disabled, 0, "SMP has been disabled from the loader"); int smp_cpus = 1; /* how many cpu's running */ SYSCTL_INT(_kern_smp, OID_AUTO, cpus, CTLFLAG_RD|CTLFLAG_CAPRD, &smp_cpus, 0, "Number of CPUs online"); int smp_topology = 0; /* Which topology we're using. */ SYSCTL_INT(_kern_smp, OID_AUTO, topology, CTLFLAG_RDTUN, &smp_topology, 0, "Topology override setting; 0 is default provided by hardware."); #ifdef SMP /* Enable forwarding of a signal to a process running on a different CPU */ static int forward_signal_enabled = 1; SYSCTL_INT(_kern_smp, OID_AUTO, forward_signal_enabled, CTLFLAG_RW, &forward_signal_enabled, 0, "Forwarding of a signal to a process on a different CPU"); /* Variables needed for SMP rendezvous. */ static volatile int smp_rv_ncpus; static void (*volatile smp_rv_setup_func)(void *arg); static void (*volatile smp_rv_action_func)(void *arg); static void (*volatile smp_rv_teardown_func)(void *arg); static void *volatile smp_rv_func_arg; static volatile int smp_rv_waiters[4]; /* * Shared mutex to restrict busywaits between smp_rendezvous() and * smp(_targeted)_tlb_shootdown(). A deadlock occurs if both of these * functions trigger at once and cause multiple CPUs to busywait with * interrupts disabled. */ struct mtx smp_ipi_mtx; /* * Let the MD SMP code initialize mp_maxid very early if it can. */ static void mp_setmaxid(void *dummy) { cpu_mp_setmaxid(); KASSERT(mp_ncpus >= 1, ("%s: CPU count < 1", __func__)); KASSERT(mp_ncpus > 1 || mp_maxid == 0, ("%s: one CPU but mp_maxid is not zero", __func__)); KASSERT(mp_maxid >= mp_ncpus - 1, ("%s: counters out of sync: max %d, count %d", __func__, mp_maxid, mp_ncpus)); } SYSINIT(cpu_mp_setmaxid, SI_SUB_TUNABLES, SI_ORDER_FIRST, mp_setmaxid, NULL); /* * Call the MD SMP initialization code. */ static void mp_start(void *dummy) { mtx_init(&smp_ipi_mtx, "smp rendezvous", NULL, MTX_SPIN); /* Probe for MP hardware. */ if (smp_disabled != 0 || cpu_mp_probe() == 0) { mp_ncpus = 1; CPU_SETOF(PCPU_GET(cpuid), &all_cpus); return; } cpu_mp_start(); printf("FreeBSD/SMP: Multiprocessor System Detected: %d CPUs\n", mp_ncpus); cpu_mp_announce(); } SYSINIT(cpu_mp, SI_SUB_CPU, SI_ORDER_THIRD, mp_start, NULL); void forward_signal(struct thread *td) { int id; /* * signotify() has already set TDF_ASTPENDING and TDF_NEEDSIGCHECK on * this thread, so all we need to do is poke it if it is currently * executing so that it executes ast(). */ THREAD_LOCK_ASSERT(td, MA_OWNED); KASSERT(TD_IS_RUNNING(td), ("forward_signal: thread is not TDS_RUNNING")); CTR1(KTR_SMP, "forward_signal(%p)", td->td_proc); if (!smp_started || cold || panicstr) return; if (!forward_signal_enabled) return; /* No need to IPI ourself. */ if (td == curthread) return; id = td->td_oncpu; if (id == NOCPU) return; ipi_cpu(id, IPI_AST); } /* * When called the executing CPU will send an IPI to all other CPUs * requesting that they halt execution. * * Usually (but not necessarily) called with 'other_cpus' as its arg. * * - Signals all CPUs in map to stop. * - Waits for each to stop. * * Returns: * -1: error * 0: NA * 1: ok * */ static int generic_stop_cpus(cpuset_t map, u_int type) { #ifdef KTR char cpusetbuf[CPUSETBUFSIZ]; #endif static volatile u_int stopping_cpu = NOCPU; int i; volatile cpuset_t *cpus; KASSERT( #if defined(__amd64__) || defined(__i386__) type == IPI_STOP || type == IPI_STOP_HARD || type == IPI_SUSPEND, #else type == IPI_STOP || type == IPI_STOP_HARD, #endif ("%s: invalid stop type", __func__)); if (!smp_started) return (0); CTR2(KTR_SMP, "stop_cpus(%s) with %u type", cpusetobj_strprint(cpusetbuf, &map), type); #if defined(__amd64__) || defined(__i386__) /* * When suspending, ensure there are are no IPIs in progress. * IPIs that have been issued, but not yet delivered (e.g. * not pending on a vCPU when running under virtualization) * will be lost, violating FreeBSD's assumption of reliable * IPI delivery. */ if (type == IPI_SUSPEND) mtx_lock_spin(&smp_ipi_mtx); #endif if (stopping_cpu != PCPU_GET(cpuid)) while (atomic_cmpset_int(&stopping_cpu, NOCPU, PCPU_GET(cpuid)) == 0) while (stopping_cpu != NOCPU) cpu_spinwait(); /* spin */ /* send the stop IPI to all CPUs in map */ ipi_selected(map, type); #if defined(__amd64__) || defined(__i386__) if (type == IPI_SUSPEND) cpus = &suspended_cpus; else #endif cpus = &stopped_cpus; i = 0; while (!CPU_SUBSET(cpus, &map)) { /* spin */ cpu_spinwait(); i++; if (i == 100000000) { printf("timeout stopping cpus\n"); break; } } #if defined(__amd64__) || defined(__i386__) if (type == IPI_SUSPEND) mtx_unlock_spin(&smp_ipi_mtx); #endif stopping_cpu = NOCPU; return (1); } int stop_cpus(cpuset_t map) { return (generic_stop_cpus(map, IPI_STOP)); } int stop_cpus_hard(cpuset_t map) { return (generic_stop_cpus(map, IPI_STOP_HARD)); } #if defined(__amd64__) || defined(__i386__) int suspend_cpus(cpuset_t map) { return (generic_stop_cpus(map, IPI_SUSPEND)); } #endif /* * Called by a CPU to restart stopped CPUs. * * Usually (but not necessarily) called with 'stopped_cpus' as its arg. * * - Signals all CPUs in map to restart. * - Waits for each to restart. * * Returns: * -1: error * 0: NA * 1: ok */ static int generic_restart_cpus(cpuset_t map, u_int type) { #ifdef KTR char cpusetbuf[CPUSETBUFSIZ]; #endif volatile cpuset_t *cpus; KASSERT( #if defined(__amd64__) || defined(__i386__) type == IPI_STOP || type == IPI_STOP_HARD || type == IPI_SUSPEND, #else type == IPI_STOP || type == IPI_STOP_HARD, #endif ("%s: invalid stop type", __func__)); if (!smp_started) return 0; CTR1(KTR_SMP, "restart_cpus(%s)", cpusetobj_strprint(cpusetbuf, &map)); #if defined(__amd64__) || defined(__i386__) if (type == IPI_SUSPEND) cpus = &suspended_cpus; else #endif cpus = &stopped_cpus; /* signal other cpus to restart */ CPU_COPY_STORE_REL(&map, &started_cpus); /* wait for each to clear its bit */ while (CPU_OVERLAP(cpus, &map)) cpu_spinwait(); return 1; } int restart_cpus(cpuset_t map) { return (generic_restart_cpus(map, IPI_STOP)); } #if defined(__amd64__) || defined(__i386__) int resume_cpus(cpuset_t map) { return (generic_restart_cpus(map, IPI_SUSPEND)); } #endif /* * All-CPU rendezvous. CPUs are signalled, all execute the setup function * (if specified), rendezvous, execute the action function (if specified), * rendezvous again, execute the teardown function (if specified), and then * resume. * * Note that the supplied external functions _must_ be reentrant and aware * that they are running in parallel and in an unknown lock context. */ void smp_rendezvous_action(void) { struct thread *td; void *local_func_arg; void (*local_setup_func)(void*); void (*local_action_func)(void*); void (*local_teardown_func)(void*); #ifdef INVARIANTS int owepreempt; #endif /* Ensure we have up-to-date values. */ atomic_add_acq_int(&smp_rv_waiters[0], 1); while (smp_rv_waiters[0] < smp_rv_ncpus) cpu_spinwait(); /* Fetch rendezvous parameters after acquire barrier. */ local_func_arg = smp_rv_func_arg; local_setup_func = smp_rv_setup_func; local_action_func = smp_rv_action_func; local_teardown_func = smp_rv_teardown_func; /* * Use a nested critical section to prevent any preemptions * from occurring during a rendezvous action routine. * Specifically, if a rendezvous handler is invoked via an IPI * and the interrupted thread was in the critical_exit() * function after setting td_critnest to 0 but before * performing a deferred preemption, this routine can be * invoked with td_critnest set to 0 and td_owepreempt true. * In that case, a critical_exit() during the rendezvous * action would trigger a preemption which is not permitted in * a rendezvous action. To fix this, wrap all of the * rendezvous action handlers in a critical section. We * cannot use a regular critical section however as having * critical_exit() preempt from this routine would also be * problematic (the preemption must not occur before the IPI * has been acknowledged via an EOI). Instead, we * intentionally ignore td_owepreempt when leaving the * critical section. This should be harmless because we do * not permit rendezvous action routines to schedule threads, * and thus td_owepreempt should never transition from 0 to 1 * during this routine. */ td = curthread; td->td_critnest++; #ifdef INVARIANTS owepreempt = td->td_owepreempt; #endif /* * If requested, run a setup function before the main action * function. Ensure all CPUs have completed the setup * function before moving on to the action function. */ if (local_setup_func != smp_no_rendevous_barrier) { if (smp_rv_setup_func != NULL) smp_rv_setup_func(smp_rv_func_arg); atomic_add_int(&smp_rv_waiters[1], 1); while (smp_rv_waiters[1] < smp_rv_ncpus) cpu_spinwait(); } if (local_action_func != NULL) local_action_func(local_func_arg); if (local_teardown_func != smp_no_rendevous_barrier) { /* * Signal that the main action has been completed. If a * full exit rendezvous is requested, then all CPUs will * wait here until all CPUs have finished the main action. */ atomic_add_int(&smp_rv_waiters[2], 1); while (smp_rv_waiters[2] < smp_rv_ncpus) cpu_spinwait(); if (local_teardown_func != NULL) local_teardown_func(local_func_arg); } /* * Signal that the rendezvous is fully completed by this CPU. * This means that no member of smp_rv_* pseudo-structure will be * accessed by this target CPU after this point; in particular, * memory pointed by smp_rv_func_arg. * * The release semantic ensures that all accesses performed by * the current CPU are visible when smp_rendezvous_cpus() * returns, by synchronizing with the * atomic_load_acq_int(&smp_rv_waiters[3]). */ atomic_add_rel_int(&smp_rv_waiters[3], 1); td->td_critnest--; KASSERT(owepreempt == td->td_owepreempt, ("rendezvous action changed td_owepreempt")); } void smp_rendezvous_cpus(cpuset_t map, void (* setup_func)(void *), void (* action_func)(void *), void (* teardown_func)(void *), void *arg) { int curcpumap, i, ncpus = 0; /* Look comments in the !SMP case. */ if (!smp_started) { spinlock_enter(); if (setup_func != NULL) setup_func(arg); if (action_func != NULL) action_func(arg); if (teardown_func != NULL) teardown_func(arg); spinlock_exit(); return; } CPU_FOREACH(i) { if (CPU_ISSET(i, &map)) ncpus++; } if (ncpus == 0) panic("ncpus is 0 with non-zero map"); mtx_lock_spin(&smp_ipi_mtx); /* Pass rendezvous parameters via global variables. */ smp_rv_ncpus = ncpus; smp_rv_setup_func = setup_func; smp_rv_action_func = action_func; smp_rv_teardown_func = teardown_func; smp_rv_func_arg = arg; smp_rv_waiters[1] = 0; smp_rv_waiters[2] = 0; smp_rv_waiters[3] = 0; atomic_store_rel_int(&smp_rv_waiters[0], 0); /* * Signal other processors, which will enter the IPI with * interrupts off. */ curcpumap = CPU_ISSET(curcpu, &map); CPU_CLR(curcpu, &map); ipi_selected(map, IPI_RENDEZVOUS); /* Check if the current CPU is in the map */ if (curcpumap != 0) smp_rendezvous_action(); /* * Ensure that the master CPU waits for all the other * CPUs to finish the rendezvous, so that smp_rv_* * pseudo-structure and the arg are guaranteed to not * be in use. * * Load acquire synchronizes with the release add in * smp_rendezvous_action(), which ensures that our caller sees * all memory actions done by the called functions on other * CPUs. */ while (atomic_load_acq_int(&smp_rv_waiters[3]) < ncpus) cpu_spinwait(); mtx_unlock_spin(&smp_ipi_mtx); } void smp_rendezvous(void (* setup_func)(void *), void (* action_func)(void *), void (* teardown_func)(void *), void *arg) { smp_rendezvous_cpus(all_cpus, setup_func, action_func, teardown_func, arg); } static struct cpu_group group[MAXCPU * MAX_CACHE_LEVELS + 1]; struct cpu_group * smp_topo(void) { char cpusetbuf[CPUSETBUFSIZ], cpusetbuf2[CPUSETBUFSIZ]; struct cpu_group *top; /* * Check for a fake topology request for debugging purposes. */ switch (smp_topology) { case 1: /* Dual core with no sharing. */ top = smp_topo_1level(CG_SHARE_NONE, 2, 0); break; case 2: /* No topology, all cpus are equal. */ top = smp_topo_none(); break; case 3: /* Dual core with shared L2. */ top = smp_topo_1level(CG_SHARE_L2, 2, 0); break; case 4: /* quad core, shared l3 among each package, private l2. */ top = smp_topo_1level(CG_SHARE_L3, 4, 0); break; case 5: /* quad core, 2 dualcore parts on each package share l2. */ top = smp_topo_2level(CG_SHARE_NONE, 2, CG_SHARE_L2, 2, 0); break; case 6: /* Single-core 2xHTT */ top = smp_topo_1level(CG_SHARE_L1, 2, CG_FLAG_HTT); break; case 7: /* quad core with a shared l3, 8 threads sharing L2. */ top = smp_topo_2level(CG_SHARE_L3, 4, CG_SHARE_L2, 8, CG_FLAG_SMT); break; default: /* Default, ask the system what it wants. */ top = cpu_topo(); break; } /* * Verify the returned topology. */ if (top->cg_count != mp_ncpus) panic("Built bad topology at %p. CPU count %d != %d", top, top->cg_count, mp_ncpus); if (CPU_CMP(&top->cg_mask, &all_cpus)) panic("Built bad topology at %p. CPU mask (%s) != (%s)", top, cpusetobj_strprint(cpusetbuf, &top->cg_mask), cpusetobj_strprint(cpusetbuf2, &all_cpus)); return (top); } struct cpu_group * smp_topo_alloc(u_int count) { static u_int index; u_int curr; curr = index; index += count; return (&group[curr]); } struct cpu_group * smp_topo_none(void) { struct cpu_group *top; top = &group[0]; top->cg_parent = NULL; top->cg_child = NULL; top->cg_mask = all_cpus; top->cg_count = mp_ncpus; top->cg_children = 0; top->cg_level = CG_SHARE_NONE; top->cg_flags = 0; return (top); } static int smp_topo_addleaf(struct cpu_group *parent, struct cpu_group *child, int share, int count, int flags, int start) { char cpusetbuf[CPUSETBUFSIZ], cpusetbuf2[CPUSETBUFSIZ]; cpuset_t mask; int i; CPU_ZERO(&mask); for (i = 0; i < count; i++, start++) CPU_SET(start, &mask); child->cg_parent = parent; child->cg_child = NULL; child->cg_children = 0; child->cg_level = share; child->cg_count = count; child->cg_flags = flags; child->cg_mask = mask; parent->cg_children++; for (; parent != NULL; parent = parent->cg_parent) { if (CPU_OVERLAP(&parent->cg_mask, &child->cg_mask)) panic("Duplicate children in %p. mask (%s) child (%s)", parent, cpusetobj_strprint(cpusetbuf, &parent->cg_mask), cpusetobj_strprint(cpusetbuf2, &child->cg_mask)); CPU_OR(&parent->cg_mask, &child->cg_mask); parent->cg_count += child->cg_count; } return (start); } struct cpu_group * smp_topo_1level(int share, int count, int flags) { struct cpu_group *child; struct cpu_group *top; int packages; int cpu; int i; cpu = 0; top = &group[0]; packages = mp_ncpus / count; top->cg_child = child = &group[1]; top->cg_level = CG_SHARE_NONE; for (i = 0; i < packages; i++, child++) cpu = smp_topo_addleaf(top, child, share, count, flags, cpu); return (top); } struct cpu_group * smp_topo_2level(int l2share, int l2count, int l1share, int l1count, int l1flags) { struct cpu_group *top; struct cpu_group *l1g; struct cpu_group *l2g; int cpu; int i; int j; cpu = 0; top = &group[0]; l2g = &group[1]; top->cg_child = l2g; top->cg_level = CG_SHARE_NONE; top->cg_children = mp_ncpus / (l2count * l1count); l1g = l2g + top->cg_children; for (i = 0; i < top->cg_children; i++, l2g++) { l2g->cg_parent = top; l2g->cg_child = l1g; l2g->cg_level = l2share; for (j = 0; j < l2count; j++, l1g++) cpu = smp_topo_addleaf(l2g, l1g, l1share, l1count, l1flags, cpu); } return (top); } struct cpu_group * smp_topo_find(struct cpu_group *top, int cpu) { struct cpu_group *cg; cpuset_t mask; int children; int i; CPU_SETOF(cpu, &mask); cg = top; for (;;) { if (!CPU_OVERLAP(&cg->cg_mask, &mask)) return (NULL); if (cg->cg_children == 0) return (cg); children = cg->cg_children; for (i = 0, cg = cg->cg_child; i < children; cg++, i++) if (CPU_OVERLAP(&cg->cg_mask, &mask)) break; } return (NULL); } #else /* !SMP */ void smp_rendezvous_cpus(cpuset_t map, void (*setup_func)(void *), void (*action_func)(void *), void (*teardown_func)(void *), void *arg) { /* * In the !SMP case we just need to ensure the same initial conditions * as the SMP case. */ spinlock_enter(); if (setup_func != NULL) setup_func(arg); if (action_func != NULL) action_func(arg); if (teardown_func != NULL) teardown_func(arg); spinlock_exit(); } void smp_rendezvous(void (*setup_func)(void *), void (*action_func)(void *), void (*teardown_func)(void *), void *arg) { - /* Look comments in the smp_rendezvous_cpus() case. */ - spinlock_enter(); - if (setup_func != NULL) - setup_func(arg); - if (action_func != NULL) - action_func(arg); - if (teardown_func != NULL) - teardown_func(arg); - spinlock_exit(); + smp_rendezvous_cpus(all_cpus, setup_func, action_func, teardown_func, + arg); } /* * Provide dummy SMP support for UP kernels. Modules that need to use SMP * APIs will still work using this dummy support. */ static void mp_setvariables_for_up(void *dummy) { mp_ncpus = 1; mp_maxid = PCPU_GET(cpuid); CPU_SETOF(mp_maxid, &all_cpus); KASSERT(PCPU_GET(cpuid) == 0, ("UP must have a CPU ID of zero")); } SYSINIT(cpu_mp_setvariables, SI_SUB_TUNABLES, SI_ORDER_FIRST, mp_setvariables_for_up, NULL); #endif /* SMP */ void smp_no_rendevous_barrier(void *dummy) { #ifdef SMP KASSERT((!smp_started),("smp_no_rendevous called and smp is started")); #endif } /* * Wait specified idle threads to switch once. This ensures that even * preempted threads have cycled through the switch function once, * exiting their codepaths. This allows us to change global pointers * with no other synchronization. */ int quiesce_cpus(cpuset_t map, const char *wmesg, int prio) { struct pcpu *pcpu; u_int gen[MAXCPU]; int error; int cpu; error = 0; for (cpu = 0; cpu <= mp_maxid; cpu++) { if (!CPU_ISSET(cpu, &map) || CPU_ABSENT(cpu)) continue; pcpu = pcpu_find(cpu); gen[cpu] = pcpu->pc_idlethread->td_generation; } for (cpu = 0; cpu <= mp_maxid; cpu++) { if (!CPU_ISSET(cpu, &map) || CPU_ABSENT(cpu)) continue; pcpu = pcpu_find(cpu); thread_lock(curthread); sched_bind(curthread, cpu); thread_unlock(curthread); while (gen[cpu] == pcpu->pc_idlethread->td_generation) { error = tsleep(quiesce_cpus, prio, wmesg, 1); if (error != EWOULDBLOCK) goto out; error = 0; } } out: thread_lock(curthread); sched_unbind(curthread); thread_unlock(curthread); return (error); } int quiesce_all_cpus(const char *wmesg, int prio) { return quiesce_cpus(all_cpus, wmesg, prio); } /* Extra care is taken with this sysctl because the data type is volatile */ static int sysctl_kern_smp_active(SYSCTL_HANDLER_ARGS) { int error, active; active = smp_started; error = SYSCTL_OUT(req, &active, sizeof(active)); return (error); } #ifdef SMP void topo_init_node(struct topo_node *node) { bzero(node, sizeof(*node)); TAILQ_INIT(&node->children); } void topo_init_root(struct topo_node *root) { topo_init_node(root); root->type = TOPO_TYPE_SYSTEM; } /* * Add a child node with the given ID under the given parent. * Do nothing if there is already a child with that ID. */ struct topo_node * topo_add_node_by_hwid(struct topo_node *parent, int hwid, topo_node_type type, uintptr_t subtype) { struct topo_node *node; TAILQ_FOREACH_REVERSE(node, &parent->children, topo_children, siblings) { if (node->hwid == hwid && node->type == type && node->subtype == subtype) { return (node); } } node = malloc(sizeof(*node), M_TOPO, M_WAITOK); topo_init_node(node); node->parent = parent; node->hwid = hwid; node->type = type; node->subtype = subtype; TAILQ_INSERT_TAIL(&parent->children, node, siblings); parent->nchildren++; return (node); } /* * Find a child node with the given ID under the given parent. */ struct topo_node * topo_find_node_by_hwid(struct topo_node *parent, int hwid, topo_node_type type, uintptr_t subtype) { struct topo_node *node; TAILQ_FOREACH(node, &parent->children, siblings) { if (node->hwid == hwid && node->type == type && node->subtype == subtype) { return (node); } } return (NULL); } /* * Given a node change the order of its parent's child nodes such * that the node becomes the firt child while preserving the cyclic * order of the children. In other words, the given node is promoted * by rotation. */ void topo_promote_child(struct topo_node *child) { struct topo_node *next; struct topo_node *node; struct topo_node *parent; parent = child->parent; next = TAILQ_NEXT(child, siblings); TAILQ_REMOVE(&parent->children, child, siblings); TAILQ_INSERT_HEAD(&parent->children, child, siblings); while (next != NULL) { node = next; next = TAILQ_NEXT(node, siblings); TAILQ_REMOVE(&parent->children, node, siblings); TAILQ_INSERT_AFTER(&parent->children, child, node, siblings); child = node; } } /* * Iterate to the next node in the depth-first search (traversal) of * the topology tree. */ struct topo_node * topo_next_node(struct topo_node *top, struct topo_node *node) { struct topo_node *next; if ((next = TAILQ_FIRST(&node->children)) != NULL) return (next); if ((next = TAILQ_NEXT(node, siblings)) != NULL) return (next); while ((node = node->parent) != top) if ((next = TAILQ_NEXT(node, siblings)) != NULL) return (next); return (NULL); } /* * Iterate to the next node in the depth-first search of the topology tree, * but without descending below the current node. */ struct topo_node * topo_next_nonchild_node(struct topo_node *top, struct topo_node *node) { struct topo_node *next; if ((next = TAILQ_NEXT(node, siblings)) != NULL) return (next); while ((node = node->parent) != top) if ((next = TAILQ_NEXT(node, siblings)) != NULL) return (next); return (NULL); } /* * Assign the given ID to the given topology node that represents a logical * processor. */ void topo_set_pu_id(struct topo_node *node, cpuid_t id) { KASSERT(node->type == TOPO_TYPE_PU, ("topo_set_pu_id: wrong node type: %u", node->type)); KASSERT(CPU_EMPTY(&node->cpuset) && node->cpu_count == 0, ("topo_set_pu_id: cpuset already not empty")); node->id = id; CPU_SET(id, &node->cpuset); node->cpu_count = 1; node->subtype = 1; while ((node = node->parent) != NULL) { KASSERT(!CPU_ISSET(id, &node->cpuset), ("logical ID %u is already set in node %p", id, node)); CPU_SET(id, &node->cpuset); node->cpu_count++; } } /* * Check if the topology is uniform, that is, each package has the same number * of cores in it and each core has the same number of threads (logical * processors) in it. If so, calculate the number of package, the number of * cores per package and the number of logical processors per core. * 'all' parameter tells whether to include administratively disabled logical * processors into the analysis. */ int topo_analyze(struct topo_node *topo_root, int all, int *pkg_count, int *cores_per_pkg, int *thrs_per_core) { struct topo_node *pkg_node; struct topo_node *core_node; struct topo_node *pu_node; int thrs_per_pkg; int cpp_counter; int tpc_counter; int tpp_counter; *pkg_count = 0; *cores_per_pkg = -1; *thrs_per_core = -1; thrs_per_pkg = -1; pkg_node = topo_root; while (pkg_node != NULL) { if (pkg_node->type != TOPO_TYPE_PKG) { pkg_node = topo_next_node(topo_root, pkg_node); continue; } if (!all && CPU_EMPTY(&pkg_node->cpuset)) { pkg_node = topo_next_nonchild_node(topo_root, pkg_node); continue; } (*pkg_count)++; cpp_counter = 0; tpp_counter = 0; core_node = pkg_node; while (core_node != NULL) { if (core_node->type == TOPO_TYPE_CORE) { if (!all && CPU_EMPTY(&core_node->cpuset)) { core_node = topo_next_nonchild_node(pkg_node, core_node); continue; } cpp_counter++; tpc_counter = 0; pu_node = core_node; while (pu_node != NULL) { if (pu_node->type == TOPO_TYPE_PU && (all || !CPU_EMPTY(&pu_node->cpuset))) tpc_counter++; pu_node = topo_next_node(core_node, pu_node); } if (*thrs_per_core == -1) *thrs_per_core = tpc_counter; else if (*thrs_per_core != tpc_counter) return (0); core_node = topo_next_nonchild_node(pkg_node, core_node); } else { /* PU node directly under PKG. */ if (core_node->type == TOPO_TYPE_PU && (all || !CPU_EMPTY(&core_node->cpuset))) tpp_counter++; core_node = topo_next_node(pkg_node, core_node); } } if (*cores_per_pkg == -1) *cores_per_pkg = cpp_counter; else if (*cores_per_pkg != cpp_counter) return (0); if (thrs_per_pkg == -1) thrs_per_pkg = tpp_counter; else if (thrs_per_pkg != tpp_counter) return (0); pkg_node = topo_next_nonchild_node(topo_root, pkg_node); } KASSERT(*pkg_count > 0, ("bug in topology or analysis")); if (*cores_per_pkg == 0) { KASSERT(*thrs_per_core == -1 && thrs_per_pkg > 0, ("bug in topology or analysis")); *thrs_per_core = thrs_per_pkg; } return (1); } #endif /* SMP */ Index: projects/netbsd-tests-update-12/sys/kern/subr_taskqueue.c =================================================================== --- projects/netbsd-tests-update-12/sys/kern/subr_taskqueue.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/kern/subr_taskqueue.c (revision 305172) @@ -1,792 +1,797 @@ /*- * Copyright (c) 2000 Doug Rabson * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static MALLOC_DEFINE(M_TASKQUEUE, "taskqueue", "Task Queues"); static void *taskqueue_giant_ih; static void *taskqueue_ih; static void taskqueue_fast_enqueue(void *); static void taskqueue_swi_enqueue(void *); static void taskqueue_swi_giant_enqueue(void *); struct taskqueue_busy { struct task *tb_running; TAILQ_ENTRY(taskqueue_busy) tb_link; }; struct task * const TB_DRAIN_WAITER = (struct task *)0x1; struct taskqueue { STAILQ_HEAD(, task) tq_queue; taskqueue_enqueue_fn tq_enqueue; void *tq_context; char *tq_name; TAILQ_HEAD(, taskqueue_busy) tq_active; struct mtx tq_mutex; struct thread **tq_threads; int tq_tcount; int tq_spin; int tq_flags; int tq_callouts; taskqueue_callback_fn tq_callbacks[TASKQUEUE_NUM_CALLBACKS]; void *tq_cb_contexts[TASKQUEUE_NUM_CALLBACKS]; }; #define TQ_FLAGS_ACTIVE (1 << 0) #define TQ_FLAGS_BLOCKED (1 << 1) #define TQ_FLAGS_UNLOCKED_ENQUEUE (1 << 2) #define DT_CALLOUT_ARMED (1 << 0) #define TQ_LOCK(tq) \ do { \ if ((tq)->tq_spin) \ mtx_lock_spin(&(tq)->tq_mutex); \ else \ mtx_lock(&(tq)->tq_mutex); \ } while (0) #define TQ_ASSERT_LOCKED(tq) mtx_assert(&(tq)->tq_mutex, MA_OWNED) #define TQ_UNLOCK(tq) \ do { \ if ((tq)->tq_spin) \ mtx_unlock_spin(&(tq)->tq_mutex); \ else \ mtx_unlock(&(tq)->tq_mutex); \ } while (0) #define TQ_ASSERT_UNLOCKED(tq) mtx_assert(&(tq)->tq_mutex, MA_NOTOWNED) void _timeout_task_init(struct taskqueue *queue, struct timeout_task *timeout_task, int priority, task_fn_t func, void *context) { TASK_INIT(&timeout_task->t, priority, func, context); callout_init_mtx(&timeout_task->c, &queue->tq_mutex, CALLOUT_RETURNUNLOCKED); timeout_task->q = queue; timeout_task->f = 0; } static __inline int TQ_SLEEP(struct taskqueue *tq, void *p, struct mtx *m, int pri, const char *wm, int t) { if (tq->tq_spin) return (msleep_spin(p, m, wm, t)); return (msleep(p, m, pri, wm, t)); } static struct taskqueue * _taskqueue_create(const char *name, int mflags, taskqueue_enqueue_fn enqueue, void *context, int mtxflags, const char *mtxname __unused) { struct taskqueue *queue; char *tq_name; tq_name = malloc(TASKQUEUE_NAMELEN, M_TASKQUEUE, mflags | M_ZERO); if (tq_name == NULL) return (NULL); queue = malloc(sizeof(struct taskqueue), M_TASKQUEUE, mflags | M_ZERO); if (queue == NULL) { free(tq_name, M_TASKQUEUE); return (NULL); } snprintf(tq_name, TASKQUEUE_NAMELEN, "%s", (name) ? name : "taskqueue"); STAILQ_INIT(&queue->tq_queue); TAILQ_INIT(&queue->tq_active); queue->tq_enqueue = enqueue; queue->tq_context = context; queue->tq_name = tq_name; queue->tq_spin = (mtxflags & MTX_SPIN) != 0; queue->tq_flags |= TQ_FLAGS_ACTIVE; if (enqueue == taskqueue_fast_enqueue || enqueue == taskqueue_swi_enqueue || enqueue == taskqueue_swi_giant_enqueue || enqueue == taskqueue_thread_enqueue) queue->tq_flags |= TQ_FLAGS_UNLOCKED_ENQUEUE; mtx_init(&queue->tq_mutex, tq_name, NULL, mtxflags); return (queue); } struct taskqueue * taskqueue_create(const char *name, int mflags, taskqueue_enqueue_fn enqueue, void *context) { return _taskqueue_create(name, mflags, enqueue, context, MTX_DEF, name); } void taskqueue_set_callback(struct taskqueue *queue, enum taskqueue_callback_type cb_type, taskqueue_callback_fn callback, void *context) { KASSERT(((cb_type >= TASKQUEUE_CALLBACK_TYPE_MIN) && (cb_type <= TASKQUEUE_CALLBACK_TYPE_MAX)), ("Callback type %d not valid, must be %d-%d", cb_type, TASKQUEUE_CALLBACK_TYPE_MIN, TASKQUEUE_CALLBACK_TYPE_MAX)); KASSERT((queue->tq_callbacks[cb_type] == NULL), ("Re-initialization of taskqueue callback?")); queue->tq_callbacks[cb_type] = callback; queue->tq_cb_contexts[cb_type] = context; } /* * Signal a taskqueue thread to terminate. */ static void taskqueue_terminate(struct thread **pp, struct taskqueue *tq) { while (tq->tq_tcount > 0 || tq->tq_callouts > 0) { wakeup(tq); TQ_SLEEP(tq, pp, &tq->tq_mutex, PWAIT, "taskqueue_destroy", 0); } } void taskqueue_free(struct taskqueue *queue) { TQ_LOCK(queue); queue->tq_flags &= ~TQ_FLAGS_ACTIVE; taskqueue_terminate(queue->tq_threads, queue); KASSERT(TAILQ_EMPTY(&queue->tq_active), ("Tasks still running?")); KASSERT(queue->tq_callouts == 0, ("Armed timeout tasks")); mtx_destroy(&queue->tq_mutex); free(queue->tq_threads, M_TASKQUEUE); free(queue->tq_name, M_TASKQUEUE); free(queue, M_TASKQUEUE); } static int taskqueue_enqueue_locked(struct taskqueue *queue, struct task *task) { struct task *ins; struct task *prev; KASSERT(task->ta_func != NULL, ("enqueueing task with NULL func")); /* * Count multiple enqueues. */ if (task->ta_pending) { if (task->ta_pending < USHRT_MAX) task->ta_pending++; TQ_UNLOCK(queue); return (0); } /* * Optimise the case when all tasks have the same priority. */ prev = STAILQ_LAST(&queue->tq_queue, task, ta_link); if (!prev || prev->ta_priority >= task->ta_priority) { STAILQ_INSERT_TAIL(&queue->tq_queue, task, ta_link); } else { prev = NULL; for (ins = STAILQ_FIRST(&queue->tq_queue); ins; prev = ins, ins = STAILQ_NEXT(ins, ta_link)) if (ins->ta_priority < task->ta_priority) break; if (prev) STAILQ_INSERT_AFTER(&queue->tq_queue, prev, task, ta_link); else STAILQ_INSERT_HEAD(&queue->tq_queue, task, ta_link); } task->ta_pending = 1; if ((queue->tq_flags & TQ_FLAGS_UNLOCKED_ENQUEUE) != 0) TQ_UNLOCK(queue); if ((queue->tq_flags & TQ_FLAGS_BLOCKED) == 0) queue->tq_enqueue(queue->tq_context); if ((queue->tq_flags & TQ_FLAGS_UNLOCKED_ENQUEUE) == 0) TQ_UNLOCK(queue); /* Return with lock released. */ return (0); } int taskqueue_enqueue(struct taskqueue *queue, struct task *task) { int res; TQ_LOCK(queue); res = taskqueue_enqueue_locked(queue, task); /* The lock is released inside. */ return (res); } static void taskqueue_timeout_func(void *arg) { struct taskqueue *queue; struct timeout_task *timeout_task; timeout_task = arg; queue = timeout_task->q; KASSERT((timeout_task->f & DT_CALLOUT_ARMED) != 0, ("Stray timeout")); timeout_task->f &= ~DT_CALLOUT_ARMED; queue->tq_callouts--; taskqueue_enqueue_locked(timeout_task->q, &timeout_task->t); /* The lock is released inside. */ } int taskqueue_enqueue_timeout(struct taskqueue *queue, struct timeout_task *timeout_task, int ticks) { int res; TQ_LOCK(queue); KASSERT(timeout_task->q == NULL || timeout_task->q == queue, ("Migrated queue")); KASSERT(!queue->tq_spin, ("Timeout for spin-queue")); timeout_task->q = queue; res = timeout_task->t.ta_pending; if (ticks == 0) { taskqueue_enqueue_locked(queue, &timeout_task->t); /* The lock is released inside. */ } else { if ((timeout_task->f & DT_CALLOUT_ARMED) != 0) { res++; } else { queue->tq_callouts++; timeout_task->f |= DT_CALLOUT_ARMED; if (ticks < 0) ticks = -ticks; /* Ignore overflow. */ } if (ticks > 0) { callout_reset(&timeout_task->c, ticks, taskqueue_timeout_func, timeout_task); } TQ_UNLOCK(queue); } return (res); } static void taskqueue_task_nop_fn(void *context, int pending) { } /* * Block until all currently queued tasks in this taskqueue * have begun execution. Tasks queued during execution of * this function are ignored. */ static void taskqueue_drain_tq_queue(struct taskqueue *queue) { struct task t_barrier; if (STAILQ_EMPTY(&queue->tq_queue)) return; /* * Enqueue our barrier after all current tasks, but with * the highest priority so that newly queued tasks cannot * pass it. Because of the high priority, we can not use * taskqueue_enqueue_locked directly (which drops the lock * anyway) so just insert it at tail while we have the * queue lock. */ TASK_INIT(&t_barrier, USHRT_MAX, taskqueue_task_nop_fn, &t_barrier); STAILQ_INSERT_TAIL(&queue->tq_queue, &t_barrier, ta_link); t_barrier.ta_pending = 1; /* * Once the barrier has executed, all previously queued tasks * have completed or are currently executing. */ while (t_barrier.ta_pending != 0) TQ_SLEEP(queue, &t_barrier, &queue->tq_mutex, PWAIT, "-", 0); } /* * Block until all currently executing tasks for this taskqueue * complete. Tasks that begin execution during the execution * of this function are ignored. */ static void taskqueue_drain_tq_active(struct taskqueue *queue) { struct taskqueue_busy tb_marker, *tb_first; if (TAILQ_EMPTY(&queue->tq_active)) return; /* Block taskq_terminate().*/ queue->tq_callouts++; /* * Wait for all currently executing taskqueue threads * to go idle. */ tb_marker.tb_running = TB_DRAIN_WAITER; TAILQ_INSERT_TAIL(&queue->tq_active, &tb_marker, tb_link); while (TAILQ_FIRST(&queue->tq_active) != &tb_marker) TQ_SLEEP(queue, &tb_marker, &queue->tq_mutex, PWAIT, "-", 0); TAILQ_REMOVE(&queue->tq_active, &tb_marker, tb_link); /* * Wakeup any other drain waiter that happened to queue up * without any intervening active thread. */ tb_first = TAILQ_FIRST(&queue->tq_active); if (tb_first != NULL && tb_first->tb_running == TB_DRAIN_WAITER) wakeup(tb_first); /* Release taskqueue_terminate(). */ queue->tq_callouts--; if ((queue->tq_flags & TQ_FLAGS_ACTIVE) == 0) wakeup_one(queue->tq_threads); } void taskqueue_block(struct taskqueue *queue) { TQ_LOCK(queue); queue->tq_flags |= TQ_FLAGS_BLOCKED; TQ_UNLOCK(queue); } void taskqueue_unblock(struct taskqueue *queue) { TQ_LOCK(queue); queue->tq_flags &= ~TQ_FLAGS_BLOCKED; if (!STAILQ_EMPTY(&queue->tq_queue)) queue->tq_enqueue(queue->tq_context); TQ_UNLOCK(queue); } static void taskqueue_run_locked(struct taskqueue *queue) { struct taskqueue_busy tb; struct taskqueue_busy *tb_first; struct task *task; int pending; KASSERT(queue != NULL, ("tq is NULL")); TQ_ASSERT_LOCKED(queue); tb.tb_running = NULL; while (STAILQ_FIRST(&queue->tq_queue)) { TAILQ_INSERT_TAIL(&queue->tq_active, &tb, tb_link); /* * Carefully remove the first task from the queue and * zero its pending count. */ task = STAILQ_FIRST(&queue->tq_queue); KASSERT(task != NULL, ("task is NULL")); STAILQ_REMOVE_HEAD(&queue->tq_queue, ta_link); pending = task->ta_pending; task->ta_pending = 0; tb.tb_running = task; TQ_UNLOCK(queue); KASSERT(task->ta_func != NULL, ("task->ta_func is NULL")); task->ta_func(task->ta_context, pending); TQ_LOCK(queue); tb.tb_running = NULL; wakeup(task); TAILQ_REMOVE(&queue->tq_active, &tb, tb_link); tb_first = TAILQ_FIRST(&queue->tq_active); if (tb_first != NULL && tb_first->tb_running == TB_DRAIN_WAITER) wakeup(tb_first); } } void taskqueue_run(struct taskqueue *queue) { TQ_LOCK(queue); taskqueue_run_locked(queue); TQ_UNLOCK(queue); } static int task_is_running(struct taskqueue *queue, struct task *task) { struct taskqueue_busy *tb; TQ_ASSERT_LOCKED(queue); TAILQ_FOREACH(tb, &queue->tq_active, tb_link) { if (tb->tb_running == task) return (1); } return (0); } static int taskqueue_cancel_locked(struct taskqueue *queue, struct task *task, u_int *pendp) { if (task->ta_pending > 0) STAILQ_REMOVE(&queue->tq_queue, task, task, ta_link); if (pendp != NULL) *pendp = task->ta_pending; task->ta_pending = 0; return (task_is_running(queue, task) ? EBUSY : 0); } int taskqueue_cancel(struct taskqueue *queue, struct task *task, u_int *pendp) { int error; TQ_LOCK(queue); error = taskqueue_cancel_locked(queue, task, pendp); TQ_UNLOCK(queue); return (error); } int taskqueue_cancel_timeout(struct taskqueue *queue, struct timeout_task *timeout_task, u_int *pendp) { u_int pending, pending1; int error; TQ_LOCK(queue); pending = !!(callout_stop(&timeout_task->c) > 0); error = taskqueue_cancel_locked(queue, &timeout_task->t, &pending1); if ((timeout_task->f & DT_CALLOUT_ARMED) != 0) { timeout_task->f &= ~DT_CALLOUT_ARMED; queue->tq_callouts--; } TQ_UNLOCK(queue); if (pendp != NULL) *pendp = pending + pending1; return (error); } void taskqueue_drain(struct taskqueue *queue, struct task *task) { if (!queue->tq_spin) WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, NULL, __func__); TQ_LOCK(queue); while (task->ta_pending != 0 || task_is_running(queue, task)) TQ_SLEEP(queue, task, &queue->tq_mutex, PWAIT, "-", 0); TQ_UNLOCK(queue); } void taskqueue_drain_all(struct taskqueue *queue) { if (!queue->tq_spin) WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, NULL, __func__); TQ_LOCK(queue); taskqueue_drain_tq_queue(queue); taskqueue_drain_tq_active(queue); TQ_UNLOCK(queue); } void taskqueue_drain_timeout(struct taskqueue *queue, struct timeout_task *timeout_task) { callout_drain(&timeout_task->c); taskqueue_drain(queue, &timeout_task->t); } static void taskqueue_swi_enqueue(void *context) { swi_sched(taskqueue_ih, 0); } static void taskqueue_swi_run(void *dummy) { taskqueue_run(taskqueue_swi); } static void taskqueue_swi_giant_enqueue(void *context) { swi_sched(taskqueue_giant_ih, 0); } static void taskqueue_swi_giant_run(void *dummy) { taskqueue_run(taskqueue_swi_giant); } static int _taskqueue_start_threads(struct taskqueue **tqp, int count, int pri, cpuset_t *mask, const char *name, va_list ap) { char ktname[MAXCOMLEN + 1]; struct thread *td; struct taskqueue *tq; int i, error; if (count <= 0) return (EINVAL); vsnprintf(ktname, sizeof(ktname), name, ap); tq = *tqp; tq->tq_threads = malloc(sizeof(struct thread *) * count, M_TASKQUEUE, M_NOWAIT | M_ZERO); if (tq->tq_threads == NULL) { printf("%s: no memory for %s threads\n", __func__, ktname); return (ENOMEM); } for (i = 0; i < count; i++) { if (count == 1) error = kthread_add(taskqueue_thread_loop, tqp, NULL, &tq->tq_threads[i], RFSTOPPED, 0, "%s", ktname); else error = kthread_add(taskqueue_thread_loop, tqp, NULL, &tq->tq_threads[i], RFSTOPPED, 0, "%s_%d", ktname, i); if (error) { /* should be ok to continue, taskqueue_free will dtrt */ printf("%s: kthread_add(%s): error %d", __func__, ktname, error); tq->tq_threads[i] = NULL; /* paranoid */ } else tq->tq_tcount++; } + if (tq->tq_tcount == 0) { + free(tq->tq_threads, M_TASKQUEUE); + tq->tq_threads = NULL; + return (ENOMEM); + } for (i = 0; i < count; i++) { if (tq->tq_threads[i] == NULL) continue; td = tq->tq_threads[i]; if (mask) { error = cpuset_setthread(td->td_tid, mask); /* * Failing to pin is rarely an actual fatal error; * it'll just affect performance. */ if (error) printf("%s: curthread=%llu: can't pin; " "error=%d\n", __func__, (unsigned long long) td->td_tid, error); } thread_lock(td); sched_prio(td, pri); sched_add(td, SRQ_BORING); thread_unlock(td); } return (0); } int taskqueue_start_threads(struct taskqueue **tqp, int count, int pri, const char *name, ...) { va_list ap; int error; va_start(ap, name); error = _taskqueue_start_threads(tqp, count, pri, NULL, name, ap); va_end(ap); return (error); } int taskqueue_start_threads_cpuset(struct taskqueue **tqp, int count, int pri, cpuset_t *mask, const char *name, ...) { va_list ap; int error; va_start(ap, name); error = _taskqueue_start_threads(tqp, count, pri, mask, name, ap); va_end(ap); return (error); } static inline void taskqueue_run_callback(struct taskqueue *tq, enum taskqueue_callback_type cb_type) { taskqueue_callback_fn tq_callback; TQ_ASSERT_UNLOCKED(tq); tq_callback = tq->tq_callbacks[cb_type]; if (tq_callback != NULL) tq_callback(tq->tq_cb_contexts[cb_type]); } void taskqueue_thread_loop(void *arg) { struct taskqueue **tqp, *tq; tqp = arg; tq = *tqp; taskqueue_run_callback(tq, TASKQUEUE_CALLBACK_TYPE_INIT); TQ_LOCK(tq); while ((tq->tq_flags & TQ_FLAGS_ACTIVE) != 0) { /* XXX ? */ taskqueue_run_locked(tq); /* * Because taskqueue_run() can drop tq_mutex, we need to * check if the TQ_FLAGS_ACTIVE flag wasn't removed in the * meantime, which means we missed a wakeup. */ if ((tq->tq_flags & TQ_FLAGS_ACTIVE) == 0) break; TQ_SLEEP(tq, tq, &tq->tq_mutex, 0, "-", 0); } taskqueue_run_locked(tq); /* * This thread is on its way out, so just drop the lock temporarily * in order to call the shutdown callback. This allows the callback * to look at the taskqueue, even just before it dies. */ TQ_UNLOCK(tq); taskqueue_run_callback(tq, TASKQUEUE_CALLBACK_TYPE_SHUTDOWN); TQ_LOCK(tq); /* rendezvous with thread that asked us to terminate */ tq->tq_tcount--; wakeup_one(tq->tq_threads); TQ_UNLOCK(tq); kthread_exit(); } void taskqueue_thread_enqueue(void *context) { struct taskqueue **tqp, *tq; tqp = context; tq = *tqp; wakeup_one(tq); } TASKQUEUE_DEFINE(swi, taskqueue_swi_enqueue, NULL, swi_add(NULL, "task queue", taskqueue_swi_run, NULL, SWI_TQ, INTR_MPSAFE, &taskqueue_ih)); TASKQUEUE_DEFINE(swi_giant, taskqueue_swi_giant_enqueue, NULL, swi_add(NULL, "Giant taskq", taskqueue_swi_giant_run, NULL, SWI_TQ_GIANT, 0, &taskqueue_giant_ih)); TASKQUEUE_DEFINE_THREAD(thread); struct taskqueue * taskqueue_create_fast(const char *name, int mflags, taskqueue_enqueue_fn enqueue, void *context) { return _taskqueue_create(name, mflags, enqueue, context, MTX_SPIN, "fast_taskqueue"); } static void *taskqueue_fast_ih; static void taskqueue_fast_enqueue(void *context) { swi_sched(taskqueue_fast_ih, 0); } static void taskqueue_fast_run(void *dummy) { taskqueue_run(taskqueue_fast); } TASKQUEUE_FAST_DEFINE(fast, taskqueue_fast_enqueue, NULL, swi_add(NULL, "fast taskq", taskqueue_fast_run, NULL, SWI_TQ_FAST, INTR_MPSAFE, &taskqueue_fast_ih)); int taskqueue_member(struct taskqueue *queue, struct thread *td) { int i, j, ret = 0; for (i = 0, j = 0; ; i++) { if (queue->tq_threads[i] == NULL) continue; if (queue->tq_threads[i] == td) { ret = 1; break; } if (++j >= queue->tq_tcount) break; } return (ret); } Index: projects/netbsd-tests-update-12/sys/kern/sys_capability.c =================================================================== --- projects/netbsd-tests-update-12/sys/kern/sys_capability.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/kern/sys_capability.c (revision 305172) @@ -1,632 +1,629 @@ /*- * Copyright (c) 2008-2011 Robert N. M. Watson * Copyright (c) 2010-2011 Jonathan Anderson * Copyright (c) 2012 FreeBSD Foundation * All rights reserved. * * This software was developed at the University of Cambridge Computer * Laboratory with support from a grant from Google, Inc. * * Portions of this software were developed by Pawel Jakub Dawidek under * sponsorship from the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * FreeBSD kernel capability facility. * * Two kernel features are implemented here: capability mode, a sandboxed mode * of execution for processes, and capabilities, a refinement on file * descriptors that allows fine-grained control over operations on the file * descriptor. Collectively, these allow processes to run in the style of a * historic "capability system" in which they can use only resources * explicitly delegated to them. This model is enforced by restricting access * to global namespaces in capability mode. * * Capabilities wrap other file descriptor types, binding them to a constant * rights mask set when the capability is created. New capabilities may be * derived from existing capabilities, but only if they have the same or a * strict subset of the rights on the original capability. * * System calls permitted in capability mode are defined in capabilities.conf; * calls must be carefully audited for safety to ensure that they don't allow * escape from a sandbox. Some calls permit only a subset of operations in * capability mode -- for example, shm_open(2) is limited to creating * anonymous, rather than named, POSIX shared memory objects. */ #include __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_ktrace.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef CAPABILITY_MODE FEATURE(security_capability_mode, "Capsicum Capability Mode"); /* * System call to enter capability mode for the process. */ int sys_cap_enter(struct thread *td, struct cap_enter_args *uap) { struct ucred *newcred, *oldcred; struct proc *p; if (IN_CAPABILITY_MODE(td)) return (0); newcred = crget(); p = td->td_proc; PROC_LOCK(p); oldcred = crcopysafe(p, newcred); newcred->cr_flags |= CRED_FLAG_CAPMODE; proc_set_cred(p, newcred); PROC_UNLOCK(p); crfree(oldcred); return (0); } /* * System call to query whether the process is in capability mode. */ int sys_cap_getmode(struct thread *td, struct cap_getmode_args *uap) { u_int i; i = IN_CAPABILITY_MODE(td) ? 1 : 0; return (copyout(&i, uap->modep, sizeof(i))); } #else /* !CAPABILITY_MODE */ int sys_cap_enter(struct thread *td, struct cap_enter_args *uap) { return (ENOSYS); } int sys_cap_getmode(struct thread *td, struct cap_getmode_args *uap) { return (ENOSYS); } #endif /* CAPABILITY_MODE */ #ifdef CAPABILITIES FEATURE(security_capabilities, "Capsicum Capabilities"); MALLOC_DECLARE(M_FILECAPS); static inline int _cap_check(const cap_rights_t *havep, const cap_rights_t *needp, enum ktr_cap_fail_type type) { - int i; - for (i = 0; i < nitems(havep->cr_rights); i++) { - if (!cap_rights_contains(havep, needp)) { + if (!cap_rights_contains(havep, needp)) { #ifdef KTRACE - if (KTRPOINT(curthread, KTR_CAPFAIL)) - ktrcapfail(type, needp, havep); + if (KTRPOINT(curthread, KTR_CAPFAIL)) + ktrcapfail(type, needp, havep); #endif - return (ENOTCAPABLE); - } + return (ENOTCAPABLE); } return (0); } /* * Test whether a capability grants the requested rights. */ int cap_check(const cap_rights_t *havep, const cap_rights_t *needp) { return (_cap_check(havep, needp, CAPFAIL_NOTCAPABLE)); } /* * Convert capability rights into VM access flags. */ u_char cap_rights_to_vmprot(cap_rights_t *havep) { u_char maxprot; maxprot = VM_PROT_NONE; if (cap_rights_is_set(havep, CAP_MMAP_R)) maxprot |= VM_PROT_READ; if (cap_rights_is_set(havep, CAP_MMAP_W)) maxprot |= VM_PROT_WRITE; if (cap_rights_is_set(havep, CAP_MMAP_X)) maxprot |= VM_PROT_EXECUTE; return (maxprot); } /* * Extract rights from a capability for monitoring purposes -- not for use in * any other way, as we want to keep all capability permission evaluation in * this one file. */ cap_rights_t * cap_rights_fde(struct filedescent *fde) { return (&fde->fde_rights); } cap_rights_t * cap_rights(struct filedesc *fdp, int fd) { return (cap_rights_fde(&fdp->fd_ofiles[fd])); } int kern_cap_rights_limit(struct thread *td, int fd, cap_rights_t *rights) { struct filedesc *fdp; int error; fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); if (fget_locked(fdp, fd) == NULL) { FILEDESC_XUNLOCK(fdp); return (EBADF); } error = _cap_check(cap_rights(fdp, fd), rights, CAPFAIL_INCREASE); if (error == 0) { fdp->fd_ofiles[fd].fde_rights = *rights; if (!cap_rights_is_set(rights, CAP_IOCTL)) { free(fdp->fd_ofiles[fd].fde_ioctls, M_FILECAPS); fdp->fd_ofiles[fd].fde_ioctls = NULL; fdp->fd_ofiles[fd].fde_nioctls = 0; } if (!cap_rights_is_set(rights, CAP_FCNTL)) fdp->fd_ofiles[fd].fde_fcntls = 0; } FILEDESC_XUNLOCK(fdp); return (error); } /* * System call to limit rights of the given capability. */ int sys_cap_rights_limit(struct thread *td, struct cap_rights_limit_args *uap) { cap_rights_t rights; int error, version; cap_rights_init(&rights); error = copyin(uap->rightsp, &rights, sizeof(rights.cr_rights[0])); if (error != 0) return (error); version = CAPVER(&rights); if (version != CAP_RIGHTS_VERSION_00) return (EINVAL); error = copyin(uap->rightsp, &rights, sizeof(rights.cr_rights[0]) * CAPARSIZE(&rights)); if (error != 0) return (error); /* Check for race. */ if (CAPVER(&rights) != version) return (EINVAL); if (!cap_rights_is_valid(&rights)) return (EINVAL); if (version != CAP_RIGHTS_VERSION) { rights.cr_rights[0] &= ~(0x3ULL << 62); rights.cr_rights[0] |= ((uint64_t)CAP_RIGHTS_VERSION << 62); } #ifdef KTRACE if (KTRPOINT(td, KTR_STRUCT)) ktrcaprights(&rights); #endif AUDIT_ARG_FD(uap->fd); AUDIT_ARG_RIGHTS(&rights); return (kern_cap_rights_limit(td, uap->fd, &rights)); } /* * System call to query the rights mask associated with a capability. */ int sys___cap_rights_get(struct thread *td, struct __cap_rights_get_args *uap) { struct filedesc *fdp; cap_rights_t rights; int error, fd, i, n; if (uap->version != CAP_RIGHTS_VERSION_00) return (EINVAL); fd = uap->fd; AUDIT_ARG_FD(fd); fdp = td->td_proc->p_fd; FILEDESC_SLOCK(fdp); if (fget_locked(fdp, fd) == NULL) { FILEDESC_SUNLOCK(fdp); return (EBADF); } rights = *cap_rights(fdp, fd); FILEDESC_SUNLOCK(fdp); n = uap->version + 2; if (uap->version != CAPVER(&rights)) { /* * For older versions we need to check if the descriptor * doesn't contain rights not understood by the caller. * If it does, we have to return an error. */ for (i = n; i < CAPARSIZE(&rights); i++) { if ((rights.cr_rights[i] & ~(0x7FULL << 57)) != 0) return (EINVAL); } } error = copyout(&rights, uap->rightsp, sizeof(rights.cr_rights[0]) * n); #ifdef KTRACE if (error == 0 && KTRPOINT(td, KTR_STRUCT)) ktrcaprights(&rights); #endif return (error); } /* * Test whether a capability grants the given ioctl command. * If descriptor doesn't have CAP_IOCTL, then ioctls list is empty and * ENOTCAPABLE will be returned. */ int cap_ioctl_check(struct filedesc *fdp, int fd, u_long cmd) { u_long *cmds; ssize_t ncmds; long i; FILEDESC_LOCK_ASSERT(fdp); KASSERT(fd >= 0 && fd < fdp->fd_nfiles, ("%s: invalid fd=%d", __func__, fd)); ncmds = fdp->fd_ofiles[fd].fde_nioctls; if (ncmds == -1) return (0); cmds = fdp->fd_ofiles[fd].fde_ioctls; for (i = 0; i < ncmds; i++) { if (cmds[i] == cmd) return (0); } return (ENOTCAPABLE); } /* * Check if the current ioctls list can be replaced by the new one. */ static int cap_ioctl_limit_check(struct filedesc *fdp, int fd, const u_long *cmds, size_t ncmds) { u_long *ocmds; ssize_t oncmds; u_long i; long j; oncmds = fdp->fd_ofiles[fd].fde_nioctls; if (oncmds == -1) return (0); if (oncmds < (ssize_t)ncmds) return (ENOTCAPABLE); ocmds = fdp->fd_ofiles[fd].fde_ioctls; for (i = 0; i < ncmds; i++) { for (j = 0; j < oncmds; j++) { if (cmds[i] == ocmds[j]) break; } if (j == oncmds) return (ENOTCAPABLE); } return (0); } int kern_cap_ioctls_limit(struct thread *td, int fd, u_long *cmds, size_t ncmds) { struct filedesc *fdp; u_long *ocmds; int error; AUDIT_ARG_FD(fd); fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); if (fget_locked(fdp, fd) == NULL) { error = EBADF; goto out; } error = cap_ioctl_limit_check(fdp, fd, cmds, ncmds); if (error != 0) goto out; ocmds = fdp->fd_ofiles[fd].fde_ioctls; fdp->fd_ofiles[fd].fde_ioctls = cmds; fdp->fd_ofiles[fd].fde_nioctls = ncmds; cmds = ocmds; error = 0; out: FILEDESC_XUNLOCK(fdp); free(cmds, M_FILECAPS); return (error); } int sys_cap_ioctls_limit(struct thread *td, struct cap_ioctls_limit_args *uap) { u_long *cmds; size_t ncmds; int error; ncmds = uap->ncmds; if (ncmds > 256) /* XXX: Is 256 sane? */ return (EINVAL); if (ncmds == 0) { cmds = NULL; } else { cmds = malloc(sizeof(cmds[0]) * ncmds, M_FILECAPS, M_WAITOK); error = copyin(uap->cmds, cmds, sizeof(cmds[0]) * ncmds); if (error != 0) { free(cmds, M_FILECAPS); return (error); } } return (kern_cap_ioctls_limit(td, uap->fd, cmds, ncmds)); } int sys_cap_ioctls_get(struct thread *td, struct cap_ioctls_get_args *uap) { struct filedesc *fdp; struct filedescent *fdep; u_long *cmds; size_t maxcmds; int error, fd; fd = uap->fd; cmds = uap->cmds; maxcmds = uap->maxcmds; AUDIT_ARG_FD(fd); fdp = td->td_proc->p_fd; FILEDESC_SLOCK(fdp); if (fget_locked(fdp, fd) == NULL) { error = EBADF; goto out; } /* * If all ioctls are allowed (fde_nioctls == -1 && fde_ioctls == NULL) * the only sane thing we can do is to not populate the given array and * return CAP_IOCTLS_ALL. */ fdep = &fdp->fd_ofiles[fd]; if (cmds != NULL && fdep->fde_ioctls != NULL) { error = copyout(fdep->fde_ioctls, cmds, sizeof(cmds[0]) * MIN(fdep->fde_nioctls, maxcmds)); if (error != 0) goto out; } if (fdep->fde_nioctls == -1) td->td_retval[0] = CAP_IOCTLS_ALL; else td->td_retval[0] = fdep->fde_nioctls; error = 0; out: FILEDESC_SUNLOCK(fdp); return (error); } /* * Test whether a capability grants the given fcntl command. */ int cap_fcntl_check_fde(struct filedescent *fde, int cmd) { uint32_t fcntlcap; fcntlcap = (1 << cmd); KASSERT((CAP_FCNTL_ALL & fcntlcap) != 0, ("Unsupported fcntl=%d.", cmd)); if ((fde->fde_fcntls & fcntlcap) != 0) return (0); return (ENOTCAPABLE); } int cap_fcntl_check(struct filedesc *fdp, int fd, int cmd) { KASSERT(fd >= 0 && fd < fdp->fd_nfiles, ("%s: invalid fd=%d", __func__, fd)); return (cap_fcntl_check_fde(&fdp->fd_ofiles[fd], cmd)); } int sys_cap_fcntls_limit(struct thread *td, struct cap_fcntls_limit_args *uap) { struct filedesc *fdp; uint32_t fcntlrights; int fd; fd = uap->fd; fcntlrights = uap->fcntlrights; AUDIT_ARG_FD(fd); AUDIT_ARG_FCNTL_RIGHTS(fcntlrights); if ((fcntlrights & ~CAP_FCNTL_ALL) != 0) return (EINVAL); fdp = td->td_proc->p_fd; FILEDESC_XLOCK(fdp); if (fget_locked(fdp, fd) == NULL) { FILEDESC_XUNLOCK(fdp); return (EBADF); } if ((fcntlrights & ~fdp->fd_ofiles[fd].fde_fcntls) != 0) { FILEDESC_XUNLOCK(fdp); return (ENOTCAPABLE); } fdp->fd_ofiles[fd].fde_fcntls = fcntlrights; FILEDESC_XUNLOCK(fdp); return (0); } int sys_cap_fcntls_get(struct thread *td, struct cap_fcntls_get_args *uap) { struct filedesc *fdp; uint32_t rights; int fd; fd = uap->fd; AUDIT_ARG_FD(fd); fdp = td->td_proc->p_fd; FILEDESC_SLOCK(fdp); if (fget_locked(fdp, fd) == NULL) { FILEDESC_SUNLOCK(fdp); return (EBADF); } rights = fdp->fd_ofiles[fd].fde_fcntls; FILEDESC_SUNLOCK(fdp); return (copyout(&rights, uap->fcntlrightsp, sizeof(rights))); } #else /* !CAPABILITIES */ /* * Stub Capability functions for when options CAPABILITIES isn't compiled * into the kernel. */ int sys_cap_rights_limit(struct thread *td, struct cap_rights_limit_args *uap) { return (ENOSYS); } int sys___cap_rights_get(struct thread *td, struct __cap_rights_get_args *uap) { return (ENOSYS); } int sys_cap_ioctls_limit(struct thread *td, struct cap_ioctls_limit_args *uap) { return (ENOSYS); } int sys_cap_ioctls_get(struct thread *td, struct cap_ioctls_get_args *uap) { return (ENOSYS); } int sys_cap_fcntls_limit(struct thread *td, struct cap_fcntls_limit_args *uap) { return (ENOSYS); } int sys_cap_fcntls_get(struct thread *td, struct cap_fcntls_get_args *uap) { return (ENOSYS); } #endif /* CAPABILITIES */ Index: projects/netbsd-tests-update-12/sys/kern/uipc_usrreq.c =================================================================== --- projects/netbsd-tests-update-12/sys/kern/uipc_usrreq.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/kern/uipc_usrreq.c (revision 305172) @@ -1,2571 +1,2571 @@ /*- * Copyright (c) 1982, 1986, 1989, 1991, 1993 * The Regents of the University of California. * Copyright (c) 2004-2009 Robert N. M. Watson * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * From: @(#)uipc_usrreq.c 8.3 (Berkeley) 1/4/94 */ /* * UNIX Domain (Local) Sockets * * This is an implementation of UNIX (local) domain sockets. Each socket has * an associated struct unpcb (UNIX protocol control block). Stream sockets * may be connected to 0 or 1 other socket. Datagram sockets may be * connected to 0, 1, or many other sockets. Sockets may be created and * connected in pairs (socketpair(2)), or bound/connected to using the file * system name space. For most purposes, only the receive socket buffer is * used, as sending on one socket delivers directly to the receive socket * buffer of a second socket. * * The implementation is substantially complicated by the fact that * "ancillary data", such as file descriptors or credentials, may be passed * across UNIX domain sockets. The potential for passing UNIX domain sockets * over other UNIX domain sockets requires the implementation of a simple * garbage collector to find and tear down cycles of disconnected sockets. * * TODO: * RDM * rethink name space problems * need a proper out-of-band */ #include __FBSDID("$FreeBSD$"); #include "opt_ddb.h" #include #include #include #include #include /* XXX must be before */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef DDB #include #endif #include #include MALLOC_DECLARE(M_FILECAPS); /* * Locking key: * (l) Locked using list lock * (g) Locked using linkage lock */ static uma_zone_t unp_zone; static unp_gen_t unp_gencnt; /* (l) */ static u_int unp_count; /* (l) Count of local sockets. */ static ino_t unp_ino; /* Prototype for fake inode numbers. */ static int unp_rights; /* (g) File descriptors in flight. */ static struct unp_head unp_shead; /* (l) List of stream sockets. */ static struct unp_head unp_dhead; /* (l) List of datagram sockets. */ static struct unp_head unp_sphead; /* (l) List of seqpacket sockets. */ struct unp_defer { SLIST_ENTRY(unp_defer) ud_link; struct file *ud_fp; }; static SLIST_HEAD(, unp_defer) unp_defers; static int unp_defers_count; static const struct sockaddr sun_noname = { sizeof(sun_noname), AF_LOCAL }; /* * Garbage collection of cyclic file descriptor/socket references occurs * asynchronously in a taskqueue context in order to avoid recursion and * reentrance in the UNIX domain socket, file descriptor, and socket layer * code. See unp_gc() for a full description. */ static struct timeout_task unp_gc_task; /* * The close of unix domain sockets attached as SCM_RIGHTS is * postponed to the taskqueue, to avoid arbitrary recursion depth. * The attached sockets might have another sockets attached. */ static struct task unp_defer_task; /* * Both send and receive buffers are allocated PIPSIZ bytes of buffering for * stream sockets, although the total for sender and receiver is actually * only PIPSIZ. * * Datagram sockets really use the sendspace as the maximum datagram size, * and don't really want to reserve the sendspace. Their recvspace should be * large enough for at least one max-size datagram plus address. */ #ifndef PIPSIZ #define PIPSIZ 8192 #endif static u_long unpst_sendspace = PIPSIZ; static u_long unpst_recvspace = PIPSIZ; static u_long unpdg_sendspace = 2*1024; /* really max datagram size */ static u_long unpdg_recvspace = 4*1024; static u_long unpsp_sendspace = PIPSIZ; /* really max datagram size */ static u_long unpsp_recvspace = PIPSIZ; static SYSCTL_NODE(_net, PF_LOCAL, local, CTLFLAG_RW, 0, "Local domain"); static SYSCTL_NODE(_net_local, SOCK_STREAM, stream, CTLFLAG_RW, 0, "SOCK_STREAM"); static SYSCTL_NODE(_net_local, SOCK_DGRAM, dgram, CTLFLAG_RW, 0, "SOCK_DGRAM"); static SYSCTL_NODE(_net_local, SOCK_SEQPACKET, seqpacket, CTLFLAG_RW, 0, "SOCK_SEQPACKET"); SYSCTL_ULONG(_net_local_stream, OID_AUTO, sendspace, CTLFLAG_RW, &unpst_sendspace, 0, "Default stream send space."); SYSCTL_ULONG(_net_local_stream, OID_AUTO, recvspace, CTLFLAG_RW, &unpst_recvspace, 0, "Default stream receive space."); SYSCTL_ULONG(_net_local_dgram, OID_AUTO, maxdgram, CTLFLAG_RW, &unpdg_sendspace, 0, "Default datagram send space."); SYSCTL_ULONG(_net_local_dgram, OID_AUTO, recvspace, CTLFLAG_RW, &unpdg_recvspace, 0, "Default datagram receive space."); SYSCTL_ULONG(_net_local_seqpacket, OID_AUTO, maxseqpacket, CTLFLAG_RW, &unpsp_sendspace, 0, "Default seqpacket send space."); SYSCTL_ULONG(_net_local_seqpacket, OID_AUTO, recvspace, CTLFLAG_RW, &unpsp_recvspace, 0, "Default seqpacket receive space."); SYSCTL_INT(_net_local, OID_AUTO, inflight, CTLFLAG_RD, &unp_rights, 0, "File descriptors in flight."); SYSCTL_INT(_net_local, OID_AUTO, deferred, CTLFLAG_RD, &unp_defers_count, 0, "File descriptors deferred to taskqueue for close."); /* * Locking and synchronization: * * Three types of locks exit in the local domain socket implementation: a * global list mutex, a global linkage rwlock, and per-unpcb mutexes. Of the * global locks, the list lock protects the socket count, global generation * number, and stream/datagram global lists. The linkage lock protects the * interconnection of unpcbs, the v_socket and unp_vnode pointers, and can be * held exclusively over the acquisition of multiple unpcb locks to prevent * deadlock. * * UNIX domain sockets each have an unpcb hung off of their so_pcb pointer, * allocated in pru_attach() and freed in pru_detach(). The validity of that * pointer is an invariant, so no lock is required to dereference the so_pcb * pointer if a valid socket reference is held by the caller. In practice, * this is always true during operations performed on a socket. Each unpcb * has a back-pointer to its socket, unp_socket, which will be stable under * the same circumstances. * * This pointer may only be safely dereferenced as long as a valid reference * to the unpcb is held. Typically, this reference will be from the socket, * or from another unpcb when the referring unpcb's lock is held (in order * that the reference not be invalidated during use). For example, to follow * unp->unp_conn->unp_socket, you need unlock the lock on unp, not unp_conn, * as unp_socket remains valid as long as the reference to unp_conn is valid. * * Fields of unpcbss are locked using a per-unpcb lock, unp_mtx. Individual * atomic reads without the lock may be performed "lockless", but more * complex reads and read-modify-writes require the mutex to be held. No * lock order is defined between unpcb locks -- multiple unpcb locks may be * acquired at the same time only when holding the linkage rwlock * exclusively, which prevents deadlocks. * * Blocking with UNIX domain sockets is a tricky issue: unlike most network * protocols, bind() is a non-atomic operation, and connect() requires * potential sleeping in the protocol, due to potentially waiting on local or * distributed file systems. We try to separate "lookup" operations, which * may sleep, and the IPC operations themselves, which typically can occur * with relative atomicity as locks can be held over the entire operation. * * Another tricky issue is simultaneous multi-threaded or multi-process * access to a single UNIX domain socket. These are handled by the flags * UNP_CONNECTING and UNP_BINDING, which prevent concurrent connecting or * binding, both of which involve dropping UNIX domain socket locks in order * to perform namei() and other file system operations. */ static struct rwlock unp_link_rwlock; static struct mtx unp_list_lock; static struct mtx unp_defers_lock; #define UNP_LINK_LOCK_INIT() rw_init(&unp_link_rwlock, \ "unp_link_rwlock") #define UNP_LINK_LOCK_ASSERT() rw_assert(&unp_link_rwlock, \ RA_LOCKED) #define UNP_LINK_UNLOCK_ASSERT() rw_assert(&unp_link_rwlock, \ RA_UNLOCKED) #define UNP_LINK_RLOCK() rw_rlock(&unp_link_rwlock) #define UNP_LINK_RUNLOCK() rw_runlock(&unp_link_rwlock) #define UNP_LINK_WLOCK() rw_wlock(&unp_link_rwlock) #define UNP_LINK_WUNLOCK() rw_wunlock(&unp_link_rwlock) #define UNP_LINK_WLOCK_ASSERT() rw_assert(&unp_link_rwlock, \ RA_WLOCKED) #define UNP_LIST_LOCK_INIT() mtx_init(&unp_list_lock, \ "unp_list_lock", NULL, MTX_DEF) #define UNP_LIST_LOCK() mtx_lock(&unp_list_lock) #define UNP_LIST_UNLOCK() mtx_unlock(&unp_list_lock) #define UNP_DEFERRED_LOCK_INIT() mtx_init(&unp_defers_lock, \ "unp_defer", NULL, MTX_DEF) #define UNP_DEFERRED_LOCK() mtx_lock(&unp_defers_lock) #define UNP_DEFERRED_UNLOCK() mtx_unlock(&unp_defers_lock) #define UNP_PCB_LOCK_INIT(unp) mtx_init(&(unp)->unp_mtx, \ "unp_mtx", "unp_mtx", \ MTX_DUPOK|MTX_DEF|MTX_RECURSE) #define UNP_PCB_LOCK_DESTROY(unp) mtx_destroy(&(unp)->unp_mtx) #define UNP_PCB_LOCK(unp) mtx_lock(&(unp)->unp_mtx) #define UNP_PCB_UNLOCK(unp) mtx_unlock(&(unp)->unp_mtx) #define UNP_PCB_LOCK_ASSERT(unp) mtx_assert(&(unp)->unp_mtx, MA_OWNED) static int uipc_connect2(struct socket *, struct socket *); static int uipc_ctloutput(struct socket *, struct sockopt *); static int unp_connect(struct socket *, struct sockaddr *, struct thread *); static int unp_connectat(int, struct socket *, struct sockaddr *, struct thread *); static int unp_connect2(struct socket *so, struct socket *so2, int); static void unp_disconnect(struct unpcb *unp, struct unpcb *unp2); -static void unp_dispose(struct mbuf *); -static void unp_dispose_so(struct socket *so); +static void unp_dispose(struct socket *so); +static void unp_dispose_mbuf(struct mbuf *); static void unp_shutdown(struct unpcb *); static void unp_drop(struct unpcb *); static void unp_gc(__unused void *, int); static void unp_scan(struct mbuf *, void (*)(struct filedescent **, int)); static void unp_discard(struct file *); static void unp_freerights(struct filedescent **, int); static void unp_init(void); static int unp_internalize(struct mbuf **, struct thread *); static void unp_internalize_fp(struct file *); static int unp_externalize(struct mbuf *, struct mbuf **, int); static int unp_externalize_fp(struct file *); static struct mbuf *unp_addsockcred(struct thread *, struct mbuf *); static void unp_process_defers(void * __unused, int); /* * Definitions of protocols supported in the LOCAL domain. */ static struct domain localdomain; static struct pr_usrreqs uipc_usrreqs_dgram, uipc_usrreqs_stream; static struct pr_usrreqs uipc_usrreqs_seqpacket; static struct protosw localsw[] = { { .pr_type = SOCK_STREAM, .pr_domain = &localdomain, .pr_flags = PR_CONNREQUIRED|PR_WANTRCVD|PR_RIGHTS, .pr_ctloutput = &uipc_ctloutput, .pr_usrreqs = &uipc_usrreqs_stream }, { .pr_type = SOCK_DGRAM, .pr_domain = &localdomain, .pr_flags = PR_ATOMIC|PR_ADDR|PR_RIGHTS, .pr_ctloutput = &uipc_ctloutput, .pr_usrreqs = &uipc_usrreqs_dgram }, { .pr_type = SOCK_SEQPACKET, .pr_domain = &localdomain, /* * XXXRW: For now, PR_ADDR because soreceive will bump into them * due to our use of sbappendaddr. A new sbappend variants is needed * that supports both atomic record writes and control data. */ .pr_flags = PR_ADDR|PR_ATOMIC|PR_CONNREQUIRED|PR_WANTRCVD| PR_RIGHTS, .pr_ctloutput = &uipc_ctloutput, .pr_usrreqs = &uipc_usrreqs_seqpacket, }, }; static struct domain localdomain = { .dom_family = AF_LOCAL, .dom_name = "local", .dom_init = unp_init, .dom_externalize = unp_externalize, - .dom_dispose = unp_dispose_so, + .dom_dispose = unp_dispose, .dom_protosw = localsw, .dom_protoswNPROTOSW = &localsw[nitems(localsw)] }; DOMAIN_SET(local); static void uipc_abort(struct socket *so) { struct unpcb *unp, *unp2; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_abort: unp == NULL")); UNP_LINK_WLOCK(); UNP_PCB_LOCK(unp); unp2 = unp->unp_conn; if (unp2 != NULL) { UNP_PCB_LOCK(unp2); unp_drop(unp2); UNP_PCB_UNLOCK(unp2); } UNP_PCB_UNLOCK(unp); UNP_LINK_WUNLOCK(); } static int uipc_accept(struct socket *so, struct sockaddr **nam) { struct unpcb *unp, *unp2; const struct sockaddr *sa; /* * Pass back name of connected socket, if it was bound and we are * still connected (our peer may have closed already!). */ unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_accept: unp == NULL")); *nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK); UNP_LINK_RLOCK(); unp2 = unp->unp_conn; if (unp2 != NULL && unp2->unp_addr != NULL) { UNP_PCB_LOCK(unp2); sa = (struct sockaddr *) unp2->unp_addr; bcopy(sa, *nam, sa->sa_len); UNP_PCB_UNLOCK(unp2); } else { sa = &sun_noname; bcopy(sa, *nam, sa->sa_len); } UNP_LINK_RUNLOCK(); return (0); } static int uipc_attach(struct socket *so, int proto, struct thread *td) { u_long sendspace, recvspace; struct unpcb *unp; int error; KASSERT(so->so_pcb == NULL, ("uipc_attach: so_pcb != NULL")); if (so->so_snd.sb_hiwat == 0 || so->so_rcv.sb_hiwat == 0) { switch (so->so_type) { case SOCK_STREAM: sendspace = unpst_sendspace; recvspace = unpst_recvspace; break; case SOCK_DGRAM: sendspace = unpdg_sendspace; recvspace = unpdg_recvspace; break; case SOCK_SEQPACKET: sendspace = unpsp_sendspace; recvspace = unpsp_recvspace; break; default: panic("uipc_attach"); } error = soreserve(so, sendspace, recvspace); if (error) return (error); } unp = uma_zalloc(unp_zone, M_NOWAIT | M_ZERO); if (unp == NULL) return (ENOBUFS); LIST_INIT(&unp->unp_refs); UNP_PCB_LOCK_INIT(unp); unp->unp_socket = so; so->so_pcb = unp; unp->unp_refcount = 1; if (so->so_head != NULL) unp->unp_flags |= UNP_NASCENT; UNP_LIST_LOCK(); unp->unp_gencnt = ++unp_gencnt; unp_count++; switch (so->so_type) { case SOCK_STREAM: LIST_INSERT_HEAD(&unp_shead, unp, unp_link); break; case SOCK_DGRAM: LIST_INSERT_HEAD(&unp_dhead, unp, unp_link); break; case SOCK_SEQPACKET: LIST_INSERT_HEAD(&unp_sphead, unp, unp_link); break; default: panic("uipc_attach"); } UNP_LIST_UNLOCK(); return (0); } static int uipc_bindat(int fd, struct socket *so, struct sockaddr *nam, struct thread *td) { struct sockaddr_un *soun = (struct sockaddr_un *)nam; struct vattr vattr; int error, namelen; struct nameidata nd; struct unpcb *unp; struct vnode *vp; struct mount *mp; cap_rights_t rights; char *buf; if (nam->sa_family != AF_UNIX) return (EAFNOSUPPORT); unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_bind: unp == NULL")); if (soun->sun_len > sizeof(struct sockaddr_un)) return (EINVAL); namelen = soun->sun_len - offsetof(struct sockaddr_un, sun_path); if (namelen <= 0) return (EINVAL); /* * We don't allow simultaneous bind() calls on a single UNIX domain * socket, so flag in-progress operations, and return an error if an * operation is already in progress. * * Historically, we have not allowed a socket to be rebound, so this * also returns an error. Not allowing re-binding simplifies the * implementation and avoids a great many possible failure modes. */ UNP_PCB_LOCK(unp); if (unp->unp_vnode != NULL) { UNP_PCB_UNLOCK(unp); return (EINVAL); } if (unp->unp_flags & UNP_BINDING) { UNP_PCB_UNLOCK(unp); return (EALREADY); } unp->unp_flags |= UNP_BINDING; UNP_PCB_UNLOCK(unp); buf = malloc(namelen + 1, M_TEMP, M_WAITOK); bcopy(soun->sun_path, buf, namelen); buf[namelen] = 0; restart: NDINIT_ATRIGHTS(&nd, CREATE, NOFOLLOW | LOCKPARENT | SAVENAME | NOCACHE, UIO_SYSSPACE, buf, fd, cap_rights_init(&rights, CAP_BINDAT), td); /* SHOULD BE ABLE TO ADOPT EXISTING AND wakeup() ALA FIFO's */ error = namei(&nd); if (error) goto error; vp = nd.ni_vp; if (vp != NULL || vn_start_write(nd.ni_dvp, &mp, V_NOWAIT) != 0) { NDFREE(&nd, NDF_ONLY_PNBUF); if (nd.ni_dvp == vp) vrele(nd.ni_dvp); else vput(nd.ni_dvp); if (vp != NULL) { vrele(vp); error = EADDRINUSE; goto error; } error = vn_start_write(NULL, &mp, V_XSLEEP | PCATCH); if (error) goto error; goto restart; } VATTR_NULL(&vattr); vattr.va_type = VSOCK; vattr.va_mode = (ACCESSPERMS & ~td->td_proc->p_fd->fd_cmask); #ifdef MAC error = mac_vnode_check_create(td->td_ucred, nd.ni_dvp, &nd.ni_cnd, &vattr); #endif if (error == 0) error = VOP_CREATE(nd.ni_dvp, &nd.ni_vp, &nd.ni_cnd, &vattr); NDFREE(&nd, NDF_ONLY_PNBUF); vput(nd.ni_dvp); if (error) { vn_finished_write(mp); goto error; } vp = nd.ni_vp; ASSERT_VOP_ELOCKED(vp, "uipc_bind"); soun = (struct sockaddr_un *)sodupsockaddr(nam, M_WAITOK); UNP_LINK_WLOCK(); UNP_PCB_LOCK(unp); VOP_UNP_BIND(vp, unp->unp_socket); unp->unp_vnode = vp; unp->unp_addr = soun; unp->unp_flags &= ~UNP_BINDING; UNP_PCB_UNLOCK(unp); UNP_LINK_WUNLOCK(); VOP_UNLOCK(vp, 0); vn_finished_write(mp); free(buf, M_TEMP); return (0); error: UNP_PCB_LOCK(unp); unp->unp_flags &= ~UNP_BINDING; UNP_PCB_UNLOCK(unp); free(buf, M_TEMP); return (error); } static int uipc_bind(struct socket *so, struct sockaddr *nam, struct thread *td) { return (uipc_bindat(AT_FDCWD, so, nam, td)); } static int uipc_connect(struct socket *so, struct sockaddr *nam, struct thread *td) { int error; KASSERT(td == curthread, ("uipc_connect: td != curthread")); UNP_LINK_WLOCK(); error = unp_connect(so, nam, td); UNP_LINK_WUNLOCK(); return (error); } static int uipc_connectat(int fd, struct socket *so, struct sockaddr *nam, struct thread *td) { int error; KASSERT(td == curthread, ("uipc_connectat: td != curthread")); UNP_LINK_WLOCK(); error = unp_connectat(fd, so, nam, td); UNP_LINK_WUNLOCK(); return (error); } static void uipc_close(struct socket *so) { struct unpcb *unp, *unp2; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_close: unp == NULL")); UNP_LINK_WLOCK(); UNP_PCB_LOCK(unp); unp2 = unp->unp_conn; if (unp2 != NULL) { UNP_PCB_LOCK(unp2); unp_disconnect(unp, unp2); UNP_PCB_UNLOCK(unp2); } UNP_PCB_UNLOCK(unp); UNP_LINK_WUNLOCK(); } static int uipc_connect2(struct socket *so1, struct socket *so2) { struct unpcb *unp, *unp2; int error; UNP_LINK_WLOCK(); unp = so1->so_pcb; KASSERT(unp != NULL, ("uipc_connect2: unp == NULL")); UNP_PCB_LOCK(unp); unp2 = so2->so_pcb; KASSERT(unp2 != NULL, ("uipc_connect2: unp2 == NULL")); UNP_PCB_LOCK(unp2); error = unp_connect2(so1, so2, PRU_CONNECT2); UNP_PCB_UNLOCK(unp2); UNP_PCB_UNLOCK(unp); UNP_LINK_WUNLOCK(); return (error); } static void uipc_detach(struct socket *so) { struct unpcb *unp, *unp2; struct sockaddr_un *saved_unp_addr; struct vnode *vp; int freeunp, local_unp_rights; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_detach: unp == NULL")); vp = NULL; local_unp_rights = 0; UNP_LIST_LOCK(); LIST_REMOVE(unp, unp_link); unp->unp_gencnt = ++unp_gencnt; --unp_count; UNP_LIST_UNLOCK(); if ((unp->unp_flags & UNP_NASCENT) != 0) { UNP_PCB_LOCK(unp); goto teardown; } UNP_LINK_WLOCK(); UNP_PCB_LOCK(unp); /* * XXXRW: Should assert vp->v_socket == so. */ if ((vp = unp->unp_vnode) != NULL) { VOP_UNP_DETACH(vp); unp->unp_vnode = NULL; } unp2 = unp->unp_conn; if (unp2 != NULL) { UNP_PCB_LOCK(unp2); unp_disconnect(unp, unp2); UNP_PCB_UNLOCK(unp2); } /* * We hold the linkage lock exclusively, so it's OK to acquire * multiple pcb locks at a time. */ while (!LIST_EMPTY(&unp->unp_refs)) { struct unpcb *ref = LIST_FIRST(&unp->unp_refs); UNP_PCB_LOCK(ref); unp_drop(ref); UNP_PCB_UNLOCK(ref); } local_unp_rights = unp_rights; UNP_LINK_WUNLOCK(); teardown: unp->unp_socket->so_pcb = NULL; saved_unp_addr = unp->unp_addr; unp->unp_addr = NULL; unp->unp_refcount--; freeunp = (unp->unp_refcount == 0); if (saved_unp_addr != NULL) free(saved_unp_addr, M_SONAME); if (freeunp) { UNP_PCB_LOCK_DESTROY(unp); uma_zfree(unp_zone, unp); } else UNP_PCB_UNLOCK(unp); if (vp) vrele(vp); if (local_unp_rights) taskqueue_enqueue_timeout(taskqueue_thread, &unp_gc_task, -1); } static int uipc_disconnect(struct socket *so) { struct unpcb *unp, *unp2; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_disconnect: unp == NULL")); UNP_LINK_WLOCK(); UNP_PCB_LOCK(unp); unp2 = unp->unp_conn; if (unp2 != NULL) { UNP_PCB_LOCK(unp2); unp_disconnect(unp, unp2); UNP_PCB_UNLOCK(unp2); } UNP_PCB_UNLOCK(unp); UNP_LINK_WUNLOCK(); return (0); } static int uipc_listen(struct socket *so, int backlog, struct thread *td) { struct unpcb *unp; int error; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_listen: unp == NULL")); UNP_PCB_LOCK(unp); if (unp->unp_vnode == NULL) { /* Already connected or not bound to an address. */ error = unp->unp_conn != NULL ? EINVAL : EDESTADDRREQ; UNP_PCB_UNLOCK(unp); return (error); } SOCK_LOCK(so); error = solisten_proto_check(so); if (error == 0) { cru2x(td->td_ucred, &unp->unp_peercred); unp->unp_flags |= UNP_HAVEPCCACHED; solisten_proto(so, backlog); } SOCK_UNLOCK(so); UNP_PCB_UNLOCK(unp); return (error); } static int uipc_peeraddr(struct socket *so, struct sockaddr **nam) { struct unpcb *unp, *unp2; const struct sockaddr *sa; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_peeraddr: unp == NULL")); *nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK); UNP_LINK_RLOCK(); /* * XXX: It seems that this test always fails even when connection is * established. So, this else clause is added as workaround to * return PF_LOCAL sockaddr. */ unp2 = unp->unp_conn; if (unp2 != NULL) { UNP_PCB_LOCK(unp2); if (unp2->unp_addr != NULL) sa = (struct sockaddr *) unp2->unp_addr; else sa = &sun_noname; bcopy(sa, *nam, sa->sa_len); UNP_PCB_UNLOCK(unp2); } else { sa = &sun_noname; bcopy(sa, *nam, sa->sa_len); } UNP_LINK_RUNLOCK(); return (0); } static int uipc_rcvd(struct socket *so, int flags) { struct unpcb *unp, *unp2; struct socket *so2; u_int mbcnt, sbcc; unp = sotounpcb(so); KASSERT(unp != NULL, ("%s: unp == NULL", __func__)); KASSERT(so->so_type == SOCK_STREAM || so->so_type == SOCK_SEQPACKET, ("%s: socktype %d", __func__, so->so_type)); /* * Adjust backpressure on sender and wakeup any waiting to write. * * The unp lock is acquired to maintain the validity of the unp_conn * pointer; no lock on unp2 is required as unp2->unp_socket will be * static as long as we don't permit unp2 to disconnect from unp, * which is prevented by the lock on unp. We cache values from * so_rcv to avoid holding the so_rcv lock over the entire * transaction on the remote so_snd. */ SOCKBUF_LOCK(&so->so_rcv); mbcnt = so->so_rcv.sb_mbcnt; sbcc = sbavail(&so->so_rcv); SOCKBUF_UNLOCK(&so->so_rcv); /* * There is a benign race condition at this point. If we're planning to * clear SB_STOP, but uipc_send is called on the connected socket at * this instant, it might add data to the sockbuf and set SB_STOP. Then * we would erroneously clear SB_STOP below, even though the sockbuf is * full. The race is benign because the only ill effect is to allow the * sockbuf to exceed its size limit, and the size limits are not * strictly guaranteed anyway. */ UNP_PCB_LOCK(unp); unp2 = unp->unp_conn; if (unp2 == NULL) { UNP_PCB_UNLOCK(unp); return (0); } so2 = unp2->unp_socket; SOCKBUF_LOCK(&so2->so_snd); if (sbcc < so2->so_snd.sb_hiwat && mbcnt < so2->so_snd.sb_mbmax) so2->so_snd.sb_flags &= ~SB_STOP; sowwakeup_locked(so2); UNP_PCB_UNLOCK(unp); return (0); } static int uipc_send(struct socket *so, int flags, struct mbuf *m, struct sockaddr *nam, struct mbuf *control, struct thread *td) { struct unpcb *unp, *unp2; struct socket *so2; u_int mbcnt, sbcc; int error = 0; unp = sotounpcb(so); KASSERT(unp != NULL, ("%s: unp == NULL", __func__)); KASSERT(so->so_type == SOCK_STREAM || so->so_type == SOCK_DGRAM || so->so_type == SOCK_SEQPACKET, ("%s: socktype %d", __func__, so->so_type)); if (flags & PRUS_OOB) { error = EOPNOTSUPP; goto release; } if (control != NULL && (error = unp_internalize(&control, td))) goto release; if ((nam != NULL) || (flags & PRUS_EOF)) UNP_LINK_WLOCK(); else UNP_LINK_RLOCK(); switch (so->so_type) { case SOCK_DGRAM: { const struct sockaddr *from; unp2 = unp->unp_conn; if (nam != NULL) { UNP_LINK_WLOCK_ASSERT(); if (unp2 != NULL) { error = EISCONN; break; } error = unp_connect(so, nam, td); if (error) break; unp2 = unp->unp_conn; } /* * Because connect() and send() are non-atomic in a sendto() * with a target address, it's possible that the socket will * have disconnected before the send() can run. In that case * return the slightly counter-intuitive but otherwise * correct error that the socket is not connected. */ if (unp2 == NULL) { error = ENOTCONN; break; } /* Lockless read. */ if (unp2->unp_flags & UNP_WANTCRED) control = unp_addsockcred(td, control); UNP_PCB_LOCK(unp); if (unp->unp_addr != NULL) from = (struct sockaddr *)unp->unp_addr; else from = &sun_noname; so2 = unp2->unp_socket; SOCKBUF_LOCK(&so2->so_rcv); if (sbappendaddr_locked(&so2->so_rcv, from, m, control)) { sorwakeup_locked(so2); m = NULL; control = NULL; } else { SOCKBUF_UNLOCK(&so2->so_rcv); error = ENOBUFS; } if (nam != NULL) { UNP_LINK_WLOCK_ASSERT(); UNP_PCB_LOCK(unp2); unp_disconnect(unp, unp2); UNP_PCB_UNLOCK(unp2); } UNP_PCB_UNLOCK(unp); break; } case SOCK_SEQPACKET: case SOCK_STREAM: if ((so->so_state & SS_ISCONNECTED) == 0) { if (nam != NULL) { UNP_LINK_WLOCK_ASSERT(); error = unp_connect(so, nam, td); if (error) break; /* XXX */ } else { error = ENOTCONN; break; } } /* Lockless read. */ if (so->so_snd.sb_state & SBS_CANTSENDMORE) { error = EPIPE; break; } /* * Because connect() and send() are non-atomic in a sendto() * with a target address, it's possible that the socket will * have disconnected before the send() can run. In that case * return the slightly counter-intuitive but otherwise * correct error that the socket is not connected. * * Locking here must be done carefully: the linkage lock * prevents interconnections between unpcbs from changing, so * we can traverse from unp to unp2 without acquiring unp's * lock. Socket buffer locks follow unpcb locks, so we can * acquire both remote and lock socket buffer locks. */ unp2 = unp->unp_conn; if (unp2 == NULL) { error = ENOTCONN; break; } so2 = unp2->unp_socket; UNP_PCB_LOCK(unp2); SOCKBUF_LOCK(&so2->so_rcv); if (unp2->unp_flags & UNP_WANTCRED) { /* * Credentials are passed only once on SOCK_STREAM * and SOCK_SEQPACKET. */ unp2->unp_flags &= ~UNP_WANTCRED; control = unp_addsockcred(td, control); } /* * Send to paired receive port, and then reduce send buffer * hiwater marks to maintain backpressure. Wake up readers. */ switch (so->so_type) { case SOCK_STREAM: if (control != NULL) { if (sbappendcontrol_locked(&so2->so_rcv, m, control)) control = NULL; } else sbappend_locked(&so2->so_rcv, m, flags); break; case SOCK_SEQPACKET: { const struct sockaddr *from; from = &sun_noname; /* * Don't check for space available in so2->so_rcv. * Unix domain sockets only check for space in the * sending sockbuf, and that check is performed one * level up the stack. */ if (sbappendaddr_nospacecheck_locked(&so2->so_rcv, from, m, control)) control = NULL; break; } } mbcnt = so2->so_rcv.sb_mbcnt; sbcc = sbavail(&so2->so_rcv); if (sbcc) sorwakeup_locked(so2); else SOCKBUF_UNLOCK(&so2->so_rcv); /* * The PCB lock on unp2 protects the SB_STOP flag. Without it, * it would be possible for uipc_rcvd to be called at this * point, drain the receiving sockbuf, clear SB_STOP, and then * we would set SB_STOP below. That could lead to an empty * sockbuf having SB_STOP set */ SOCKBUF_LOCK(&so->so_snd); if (sbcc >= so->so_snd.sb_hiwat || mbcnt >= so->so_snd.sb_mbmax) so->so_snd.sb_flags |= SB_STOP; SOCKBUF_UNLOCK(&so->so_snd); UNP_PCB_UNLOCK(unp2); m = NULL; break; } /* * PRUS_EOF is equivalent to pru_send followed by pru_shutdown. */ if (flags & PRUS_EOF) { UNP_PCB_LOCK(unp); socantsendmore(so); unp_shutdown(unp); UNP_PCB_UNLOCK(unp); } if ((nam != NULL) || (flags & PRUS_EOF)) UNP_LINK_WUNLOCK(); else UNP_LINK_RUNLOCK(); if (control != NULL && error != 0) - unp_dispose(control); + unp_dispose_mbuf(control); release: if (control != NULL) m_freem(control); if (m != NULL) m_freem(m); return (error); } static int uipc_ready(struct socket *so, struct mbuf *m, int count) { struct unpcb *unp, *unp2; struct socket *so2; int error; unp = sotounpcb(so); UNP_LINK_RLOCK(); unp2 = unp->unp_conn; UNP_PCB_LOCK(unp2); so2 = unp2->unp_socket; SOCKBUF_LOCK(&so2->so_rcv); if ((error = sbready(&so2->so_rcv, m, count)) == 0) sorwakeup_locked(so2); else SOCKBUF_UNLOCK(&so2->so_rcv); UNP_PCB_UNLOCK(unp2); UNP_LINK_RUNLOCK(); return (error); } static int uipc_sense(struct socket *so, struct stat *sb) { struct unpcb *unp; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_sense: unp == NULL")); sb->st_blksize = so->so_snd.sb_hiwat; UNP_PCB_LOCK(unp); sb->st_dev = NODEV; if (unp->unp_ino == 0) unp->unp_ino = (++unp_ino == 0) ? ++unp_ino : unp_ino; sb->st_ino = unp->unp_ino; UNP_PCB_UNLOCK(unp); return (0); } static int uipc_shutdown(struct socket *so) { struct unpcb *unp; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_shutdown: unp == NULL")); UNP_LINK_WLOCK(); UNP_PCB_LOCK(unp); socantsendmore(so); unp_shutdown(unp); UNP_PCB_UNLOCK(unp); UNP_LINK_WUNLOCK(); return (0); } static int uipc_sockaddr(struct socket *so, struct sockaddr **nam) { struct unpcb *unp; const struct sockaddr *sa; unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_sockaddr: unp == NULL")); *nam = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK); UNP_PCB_LOCK(unp); if (unp->unp_addr != NULL) sa = (struct sockaddr *) unp->unp_addr; else sa = &sun_noname; bcopy(sa, *nam, sa->sa_len); UNP_PCB_UNLOCK(unp); return (0); } static struct pr_usrreqs uipc_usrreqs_dgram = { .pru_abort = uipc_abort, .pru_accept = uipc_accept, .pru_attach = uipc_attach, .pru_bind = uipc_bind, .pru_bindat = uipc_bindat, .pru_connect = uipc_connect, .pru_connectat = uipc_connectat, .pru_connect2 = uipc_connect2, .pru_detach = uipc_detach, .pru_disconnect = uipc_disconnect, .pru_listen = uipc_listen, .pru_peeraddr = uipc_peeraddr, .pru_rcvd = uipc_rcvd, .pru_send = uipc_send, .pru_sense = uipc_sense, .pru_shutdown = uipc_shutdown, .pru_sockaddr = uipc_sockaddr, .pru_soreceive = soreceive_dgram, .pru_close = uipc_close, }; static struct pr_usrreqs uipc_usrreqs_seqpacket = { .pru_abort = uipc_abort, .pru_accept = uipc_accept, .pru_attach = uipc_attach, .pru_bind = uipc_bind, .pru_bindat = uipc_bindat, .pru_connect = uipc_connect, .pru_connectat = uipc_connectat, .pru_connect2 = uipc_connect2, .pru_detach = uipc_detach, .pru_disconnect = uipc_disconnect, .pru_listen = uipc_listen, .pru_peeraddr = uipc_peeraddr, .pru_rcvd = uipc_rcvd, .pru_send = uipc_send, .pru_sense = uipc_sense, .pru_shutdown = uipc_shutdown, .pru_sockaddr = uipc_sockaddr, .pru_soreceive = soreceive_generic, /* XXX: or...? */ .pru_close = uipc_close, }; static struct pr_usrreqs uipc_usrreqs_stream = { .pru_abort = uipc_abort, .pru_accept = uipc_accept, .pru_attach = uipc_attach, .pru_bind = uipc_bind, .pru_bindat = uipc_bindat, .pru_connect = uipc_connect, .pru_connectat = uipc_connectat, .pru_connect2 = uipc_connect2, .pru_detach = uipc_detach, .pru_disconnect = uipc_disconnect, .pru_listen = uipc_listen, .pru_peeraddr = uipc_peeraddr, .pru_rcvd = uipc_rcvd, .pru_send = uipc_send, .pru_ready = uipc_ready, .pru_sense = uipc_sense, .pru_shutdown = uipc_shutdown, .pru_sockaddr = uipc_sockaddr, .pru_soreceive = soreceive_generic, .pru_close = uipc_close, }; static int uipc_ctloutput(struct socket *so, struct sockopt *sopt) { struct unpcb *unp; struct xucred xu; int error, optval; if (sopt->sopt_level != 0) return (EINVAL); unp = sotounpcb(so); KASSERT(unp != NULL, ("uipc_ctloutput: unp == NULL")); error = 0; switch (sopt->sopt_dir) { case SOPT_GET: switch (sopt->sopt_name) { case LOCAL_PEERCRED: UNP_PCB_LOCK(unp); if (unp->unp_flags & UNP_HAVEPC) xu = unp->unp_peercred; else { if (so->so_type == SOCK_STREAM) error = ENOTCONN; else error = EINVAL; } UNP_PCB_UNLOCK(unp); if (error == 0) error = sooptcopyout(sopt, &xu, sizeof(xu)); break; case LOCAL_CREDS: /* Unlocked read. */ optval = unp->unp_flags & UNP_WANTCRED ? 1 : 0; error = sooptcopyout(sopt, &optval, sizeof(optval)); break; case LOCAL_CONNWAIT: /* Unlocked read. */ optval = unp->unp_flags & UNP_CONNWAIT ? 1 : 0; error = sooptcopyout(sopt, &optval, sizeof(optval)); break; default: error = EOPNOTSUPP; break; } break; case SOPT_SET: switch (sopt->sopt_name) { case LOCAL_CREDS: case LOCAL_CONNWAIT: error = sooptcopyin(sopt, &optval, sizeof(optval), sizeof(optval)); if (error) break; #define OPTSET(bit) do { \ UNP_PCB_LOCK(unp); \ if (optval) \ unp->unp_flags |= bit; \ else \ unp->unp_flags &= ~bit; \ UNP_PCB_UNLOCK(unp); \ } while (0) switch (sopt->sopt_name) { case LOCAL_CREDS: OPTSET(UNP_WANTCRED); break; case LOCAL_CONNWAIT: OPTSET(UNP_CONNWAIT); break; default: break; } break; #undef OPTSET default: error = ENOPROTOOPT; break; } break; default: error = EOPNOTSUPP; break; } return (error); } static int unp_connect(struct socket *so, struct sockaddr *nam, struct thread *td) { return (unp_connectat(AT_FDCWD, so, nam, td)); } static int unp_connectat(int fd, struct socket *so, struct sockaddr *nam, struct thread *td) { struct sockaddr_un *soun = (struct sockaddr_un *)nam; struct vnode *vp; struct socket *so2, *so3; struct unpcb *unp, *unp2, *unp3; struct nameidata nd; char buf[SOCK_MAXADDRLEN]; struct sockaddr *sa; cap_rights_t rights; int error, len; if (nam->sa_family != AF_UNIX) return (EAFNOSUPPORT); UNP_LINK_WLOCK_ASSERT(); unp = sotounpcb(so); KASSERT(unp != NULL, ("unp_connect: unp == NULL")); if (nam->sa_len > sizeof(struct sockaddr_un)) return (EINVAL); len = nam->sa_len - offsetof(struct sockaddr_un, sun_path); if (len <= 0) return (EINVAL); bcopy(soun->sun_path, buf, len); buf[len] = 0; UNP_PCB_LOCK(unp); if (unp->unp_flags & UNP_CONNECTING) { UNP_PCB_UNLOCK(unp); return (EALREADY); } UNP_LINK_WUNLOCK(); unp->unp_flags |= UNP_CONNECTING; UNP_PCB_UNLOCK(unp); sa = malloc(sizeof(struct sockaddr_un), M_SONAME, M_WAITOK); NDINIT_ATRIGHTS(&nd, LOOKUP, FOLLOW | LOCKSHARED | LOCKLEAF, UIO_SYSSPACE, buf, fd, cap_rights_init(&rights, CAP_CONNECTAT), td); error = namei(&nd); if (error) vp = NULL; else vp = nd.ni_vp; ASSERT_VOP_LOCKED(vp, "unp_connect"); NDFREE(&nd, NDF_ONLY_PNBUF); if (error) goto bad; if (vp->v_type != VSOCK) { error = ENOTSOCK; goto bad; } #ifdef MAC error = mac_vnode_check_open(td->td_ucred, vp, VWRITE | VREAD); if (error) goto bad; #endif error = VOP_ACCESS(vp, VWRITE, td->td_ucred, td); if (error) goto bad; unp = sotounpcb(so); KASSERT(unp != NULL, ("unp_connect: unp == NULL")); /* * Lock linkage lock for two reasons: make sure v_socket is stable, * and to protect simultaneous locking of multiple pcbs. */ UNP_LINK_WLOCK(); VOP_UNP_CONNECT(vp, &so2); if (so2 == NULL) { error = ECONNREFUSED; goto bad2; } if (so->so_type != so2->so_type) { error = EPROTOTYPE; goto bad2; } if (so->so_proto->pr_flags & PR_CONNREQUIRED) { if (so2->so_options & SO_ACCEPTCONN) { CURVNET_SET(so2->so_vnet); so3 = sonewconn(so2, 0); CURVNET_RESTORE(); } else so3 = NULL; if (so3 == NULL) { error = ECONNREFUSED; goto bad2; } unp = sotounpcb(so); unp2 = sotounpcb(so2); unp3 = sotounpcb(so3); UNP_PCB_LOCK(unp); UNP_PCB_LOCK(unp2); UNP_PCB_LOCK(unp3); if (unp2->unp_addr != NULL) { bcopy(unp2->unp_addr, sa, unp2->unp_addr->sun_len); unp3->unp_addr = (struct sockaddr_un *) sa; sa = NULL; } /* * The connector's (client's) credentials are copied from its * process structure at the time of connect() (which is now). */ cru2x(td->td_ucred, &unp3->unp_peercred); unp3->unp_flags |= UNP_HAVEPC; /* * The receiver's (server's) credentials are copied from the * unp_peercred member of socket on which the former called * listen(); uipc_listen() cached that process's credentials * at that time so we can use them now. */ KASSERT(unp2->unp_flags & UNP_HAVEPCCACHED, ("unp_connect: listener without cached peercred")); memcpy(&unp->unp_peercred, &unp2->unp_peercred, sizeof(unp->unp_peercred)); unp->unp_flags |= UNP_HAVEPC; if (unp2->unp_flags & UNP_WANTCRED) unp3->unp_flags |= UNP_WANTCRED; UNP_PCB_UNLOCK(unp3); UNP_PCB_UNLOCK(unp2); UNP_PCB_UNLOCK(unp); #ifdef MAC mac_socketpeer_set_from_socket(so, so3); mac_socketpeer_set_from_socket(so3, so); #endif so2 = so3; } unp = sotounpcb(so); KASSERT(unp != NULL, ("unp_connect: unp == NULL")); unp2 = sotounpcb(so2); KASSERT(unp2 != NULL, ("unp_connect: unp2 == NULL")); UNP_PCB_LOCK(unp); UNP_PCB_LOCK(unp2); error = unp_connect2(so, so2, PRU_CONNECT); UNP_PCB_UNLOCK(unp2); UNP_PCB_UNLOCK(unp); bad2: UNP_LINK_WUNLOCK(); bad: if (vp != NULL) vput(vp); free(sa, M_SONAME); UNP_LINK_WLOCK(); UNP_PCB_LOCK(unp); unp->unp_flags &= ~UNP_CONNECTING; UNP_PCB_UNLOCK(unp); return (error); } static int unp_connect2(struct socket *so, struct socket *so2, int req) { struct unpcb *unp; struct unpcb *unp2; unp = sotounpcb(so); KASSERT(unp != NULL, ("unp_connect2: unp == NULL")); unp2 = sotounpcb(so2); KASSERT(unp2 != NULL, ("unp_connect2: unp2 == NULL")); UNP_LINK_WLOCK_ASSERT(); UNP_PCB_LOCK_ASSERT(unp); UNP_PCB_LOCK_ASSERT(unp2); if (so2->so_type != so->so_type) return (EPROTOTYPE); unp2->unp_flags &= ~UNP_NASCENT; unp->unp_conn = unp2; switch (so->so_type) { case SOCK_DGRAM: LIST_INSERT_HEAD(&unp2->unp_refs, unp, unp_reflink); soisconnected(so); break; case SOCK_STREAM: case SOCK_SEQPACKET: unp2->unp_conn = unp; if (req == PRU_CONNECT && ((unp->unp_flags | unp2->unp_flags) & UNP_CONNWAIT)) soisconnecting(so); else soisconnected(so); soisconnected(so2); break; default: panic("unp_connect2"); } return (0); } static void unp_disconnect(struct unpcb *unp, struct unpcb *unp2) { struct socket *so; KASSERT(unp2 != NULL, ("unp_disconnect: unp2 == NULL")); UNP_LINK_WLOCK_ASSERT(); UNP_PCB_LOCK_ASSERT(unp); UNP_PCB_LOCK_ASSERT(unp2); unp->unp_conn = NULL; switch (unp->unp_socket->so_type) { case SOCK_DGRAM: LIST_REMOVE(unp, unp_reflink); so = unp->unp_socket; SOCK_LOCK(so); so->so_state &= ~SS_ISCONNECTED; SOCK_UNLOCK(so); break; case SOCK_STREAM: case SOCK_SEQPACKET: soisdisconnected(unp->unp_socket); unp2->unp_conn = NULL; soisdisconnected(unp2->unp_socket); break; } } /* * unp_pcblist() walks the global list of struct unpcb's to generate a * pointer list, bumping the refcount on each unpcb. It then copies them out * sequentially, validating the generation number on each to see if it has * been detached. All of this is necessary because copyout() may sleep on * disk I/O. */ static int unp_pcblist(SYSCTL_HANDLER_ARGS) { int error, i, n; int freeunp; struct unpcb *unp, **unp_list; unp_gen_t gencnt; struct xunpgen *xug; struct unp_head *head; struct xunpcb *xu; switch ((intptr_t)arg1) { case SOCK_STREAM: head = &unp_shead; break; case SOCK_DGRAM: head = &unp_dhead; break; case SOCK_SEQPACKET: head = &unp_sphead; break; default: panic("unp_pcblist: arg1 %d", (int)(intptr_t)arg1); } /* * The process of preparing the PCB list is too time-consuming and * resource-intensive to repeat twice on every request. */ if (req->oldptr == NULL) { n = unp_count; req->oldidx = 2 * (sizeof *xug) + (n + n/8) * sizeof(struct xunpcb); return (0); } if (req->newptr != NULL) return (EPERM); /* * OK, now we're committed to doing something. */ xug = malloc(sizeof(*xug), M_TEMP, M_WAITOK); UNP_LIST_LOCK(); gencnt = unp_gencnt; n = unp_count; UNP_LIST_UNLOCK(); xug->xug_len = sizeof *xug; xug->xug_count = n; xug->xug_gen = gencnt; xug->xug_sogen = so_gencnt; error = SYSCTL_OUT(req, xug, sizeof *xug); if (error) { free(xug, M_TEMP); return (error); } unp_list = malloc(n * sizeof *unp_list, M_TEMP, M_WAITOK); UNP_LIST_LOCK(); for (unp = LIST_FIRST(head), i = 0; unp && i < n; unp = LIST_NEXT(unp, unp_link)) { UNP_PCB_LOCK(unp); if (unp->unp_gencnt <= gencnt) { if (cr_cansee(req->td->td_ucred, unp->unp_socket->so_cred)) { UNP_PCB_UNLOCK(unp); continue; } unp_list[i++] = unp; unp->unp_refcount++; } UNP_PCB_UNLOCK(unp); } UNP_LIST_UNLOCK(); n = i; /* In case we lost some during malloc. */ error = 0; xu = malloc(sizeof(*xu), M_TEMP, M_WAITOK | M_ZERO); for (i = 0; i < n; i++) { unp = unp_list[i]; UNP_PCB_LOCK(unp); unp->unp_refcount--; if (unp->unp_refcount != 0 && unp->unp_gencnt <= gencnt) { xu->xu_len = sizeof *xu; xu->xu_unpp = unp; /* * XXX - need more locking here to protect against * connect/disconnect races for SMP. */ if (unp->unp_addr != NULL) bcopy(unp->unp_addr, &xu->xu_addr, unp->unp_addr->sun_len); if (unp->unp_conn != NULL && unp->unp_conn->unp_addr != NULL) bcopy(unp->unp_conn->unp_addr, &xu->xu_caddr, unp->unp_conn->unp_addr->sun_len); bcopy(unp, &xu->xu_unp, sizeof *unp); sotoxsocket(unp->unp_socket, &xu->xu_socket); UNP_PCB_UNLOCK(unp); error = SYSCTL_OUT(req, xu, sizeof *xu); } else { freeunp = (unp->unp_refcount == 0); UNP_PCB_UNLOCK(unp); if (freeunp) { UNP_PCB_LOCK_DESTROY(unp); uma_zfree(unp_zone, unp); } } } free(xu, M_TEMP); if (!error) { /* * Give the user an updated idea of our state. If the * generation differs from what we told her before, she knows * that something happened while we were processing this * request, and it might be necessary to retry. */ xug->xug_gen = unp_gencnt; xug->xug_sogen = so_gencnt; xug->xug_count = unp_count; error = SYSCTL_OUT(req, xug, sizeof *xug); } free(unp_list, M_TEMP); free(xug, M_TEMP); return (error); } SYSCTL_PROC(_net_local_dgram, OID_AUTO, pcblist, CTLTYPE_OPAQUE | CTLFLAG_RD, (void *)(intptr_t)SOCK_DGRAM, 0, unp_pcblist, "S,xunpcb", "List of active local datagram sockets"); SYSCTL_PROC(_net_local_stream, OID_AUTO, pcblist, CTLTYPE_OPAQUE | CTLFLAG_RD, (void *)(intptr_t)SOCK_STREAM, 0, unp_pcblist, "S,xunpcb", "List of active local stream sockets"); SYSCTL_PROC(_net_local_seqpacket, OID_AUTO, pcblist, CTLTYPE_OPAQUE | CTLFLAG_RD, (void *)(intptr_t)SOCK_SEQPACKET, 0, unp_pcblist, "S,xunpcb", "List of active local seqpacket sockets"); static void unp_shutdown(struct unpcb *unp) { struct unpcb *unp2; struct socket *so; UNP_LINK_WLOCK_ASSERT(); UNP_PCB_LOCK_ASSERT(unp); unp2 = unp->unp_conn; if ((unp->unp_socket->so_type == SOCK_STREAM || (unp->unp_socket->so_type == SOCK_SEQPACKET)) && unp2 != NULL) { so = unp2->unp_socket; if (so != NULL) socantrcvmore(so); } } static void unp_drop(struct unpcb *unp) { struct socket *so = unp->unp_socket; struct unpcb *unp2; UNP_LINK_WLOCK_ASSERT(); UNP_PCB_LOCK_ASSERT(unp); /* * Regardless of whether the socket's peer dropped the connection * with this socket by aborting or disconnecting, POSIX requires * that ECONNRESET is returned. */ so->so_error = ECONNRESET; unp2 = unp->unp_conn; if (unp2 == NULL) return; UNP_PCB_LOCK(unp2); unp_disconnect(unp, unp2); UNP_PCB_UNLOCK(unp2); } static void unp_freerights(struct filedescent **fdep, int fdcount) { struct file *fp; int i; KASSERT(fdcount > 0, ("%s: fdcount %d", __func__, fdcount)); for (i = 0; i < fdcount; i++) { fp = fdep[i]->fde_file; filecaps_free(&fdep[i]->fde_caps); unp_discard(fp); } free(fdep[0], M_FILECAPS); } static int unp_externalize(struct mbuf *control, struct mbuf **controlp, int flags) { struct thread *td = curthread; /* XXX */ struct cmsghdr *cm = mtod(control, struct cmsghdr *); int i; int *fdp; struct filedesc *fdesc = td->td_proc->p_fd; struct filedescent **fdep; void *data; socklen_t clen = control->m_len, datalen; int error, newfds; u_int newlen; UNP_LINK_UNLOCK_ASSERT(); error = 0; if (controlp != NULL) /* controlp == NULL => free control messages */ *controlp = NULL; while (cm != NULL) { if (sizeof(*cm) > clen || cm->cmsg_len > clen) { error = EINVAL; break; } data = CMSG_DATA(cm); datalen = (caddr_t)cm + cm->cmsg_len - (caddr_t)data; if (cm->cmsg_level == SOL_SOCKET && cm->cmsg_type == SCM_RIGHTS) { newfds = datalen / sizeof(*fdep); if (newfds == 0) goto next; fdep = data; /* If we're not outputting the descriptors free them. */ if (error || controlp == NULL) { unp_freerights(fdep, newfds); goto next; } FILEDESC_XLOCK(fdesc); /* * Now change each pointer to an fd in the global * table to an integer that is the index to the local * fd table entry that we set up to point to the * global one we are transferring. */ newlen = newfds * sizeof(int); *controlp = sbcreatecontrol(NULL, newlen, SCM_RIGHTS, SOL_SOCKET); if (*controlp == NULL) { FILEDESC_XUNLOCK(fdesc); error = E2BIG; unp_freerights(fdep, newfds); goto next; } fdp = (int *) CMSG_DATA(mtod(*controlp, struct cmsghdr *)); if (fdallocn(td, 0, fdp, newfds) != 0) { FILEDESC_XUNLOCK(fdesc); error = EMSGSIZE; unp_freerights(fdep, newfds); m_freem(*controlp); *controlp = NULL; goto next; } for (i = 0; i < newfds; i++, fdp++) { _finstall(fdesc, fdep[i]->fde_file, *fdp, (flags & MSG_CMSG_CLOEXEC) != 0 ? UF_EXCLOSE : 0, &fdep[i]->fde_caps); unp_externalize_fp(fdep[i]->fde_file); } FILEDESC_XUNLOCK(fdesc); free(fdep[0], M_FILECAPS); } else { /* We can just copy anything else across. */ if (error || controlp == NULL) goto next; *controlp = sbcreatecontrol(NULL, datalen, cm->cmsg_type, cm->cmsg_level); if (*controlp == NULL) { error = ENOBUFS; goto next; } bcopy(data, CMSG_DATA(mtod(*controlp, struct cmsghdr *)), datalen); } controlp = &(*controlp)->m_next; next: if (CMSG_SPACE(datalen) < clen) { clen -= CMSG_SPACE(datalen); cm = (struct cmsghdr *) ((caddr_t)cm + CMSG_SPACE(datalen)); } else { clen = 0; cm = NULL; } } m_freem(control); return (error); } static void unp_zone_change(void *tag) { uma_zone_set_max(unp_zone, maxsockets); } static void unp_init(void) { #ifdef VIMAGE if (!IS_DEFAULT_VNET(curvnet)) return; #endif unp_zone = uma_zcreate("unpcb", sizeof(struct unpcb), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, 0); if (unp_zone == NULL) panic("unp_init"); uma_zone_set_max(unp_zone, maxsockets); uma_zone_set_warning(unp_zone, "kern.ipc.maxsockets limit reached"); EVENTHANDLER_REGISTER(maxsockets_change, unp_zone_change, NULL, EVENTHANDLER_PRI_ANY); LIST_INIT(&unp_dhead); LIST_INIT(&unp_shead); LIST_INIT(&unp_sphead); SLIST_INIT(&unp_defers); TIMEOUT_TASK_INIT(taskqueue_thread, &unp_gc_task, 0, unp_gc, NULL); TASK_INIT(&unp_defer_task, 0, unp_process_defers, NULL); UNP_LINK_LOCK_INIT(); UNP_LIST_LOCK_INIT(); UNP_DEFERRED_LOCK_INIT(); } static int unp_internalize(struct mbuf **controlp, struct thread *td) { struct mbuf *control = *controlp; struct proc *p = td->td_proc; struct filedesc *fdesc = p->p_fd; struct bintime *bt; struct cmsghdr *cm = mtod(control, struct cmsghdr *); struct cmsgcred *cmcred; struct filedescent *fde, **fdep, *fdev; struct file *fp; struct timeval *tv; int i, *fdp; void *data; socklen_t clen = control->m_len, datalen; int error, oldfds; u_int newlen; UNP_LINK_UNLOCK_ASSERT(); error = 0; *controlp = NULL; while (cm != NULL) { if (sizeof(*cm) > clen || cm->cmsg_level != SOL_SOCKET || cm->cmsg_len > clen || cm->cmsg_len < sizeof(*cm)) { error = EINVAL; goto out; } data = CMSG_DATA(cm); datalen = (caddr_t)cm + cm->cmsg_len - (caddr_t)data; switch (cm->cmsg_type) { /* * Fill in credential information. */ case SCM_CREDS: *controlp = sbcreatecontrol(NULL, sizeof(*cmcred), SCM_CREDS, SOL_SOCKET); if (*controlp == NULL) { error = ENOBUFS; goto out; } cmcred = (struct cmsgcred *) CMSG_DATA(mtod(*controlp, struct cmsghdr *)); cmcred->cmcred_pid = p->p_pid; cmcred->cmcred_uid = td->td_ucred->cr_ruid; cmcred->cmcred_gid = td->td_ucred->cr_rgid; cmcred->cmcred_euid = td->td_ucred->cr_uid; cmcred->cmcred_ngroups = MIN(td->td_ucred->cr_ngroups, CMGROUP_MAX); for (i = 0; i < cmcred->cmcred_ngroups; i++) cmcred->cmcred_groups[i] = td->td_ucred->cr_groups[i]; break; case SCM_RIGHTS: oldfds = datalen / sizeof (int); if (oldfds == 0) break; /* * Check that all the FDs passed in refer to legal * files. If not, reject the entire operation. */ fdp = data; FILEDESC_SLOCK(fdesc); for (i = 0; i < oldfds; i++, fdp++) { fp = fget_locked(fdesc, *fdp); if (fp == NULL) { FILEDESC_SUNLOCK(fdesc); error = EBADF; goto out; } if (!(fp->f_ops->fo_flags & DFLAG_PASSABLE)) { FILEDESC_SUNLOCK(fdesc); error = EOPNOTSUPP; goto out; } } /* * Now replace the integer FDs with pointers to the * file structure and capability rights. */ newlen = oldfds * sizeof(fdep[0]); *controlp = sbcreatecontrol(NULL, newlen, SCM_RIGHTS, SOL_SOCKET); if (*controlp == NULL) { FILEDESC_SUNLOCK(fdesc); error = E2BIG; goto out; } fdp = data; fdep = (struct filedescent **) CMSG_DATA(mtod(*controlp, struct cmsghdr *)); fdev = malloc(sizeof(*fdev) * oldfds, M_FILECAPS, M_WAITOK); for (i = 0; i < oldfds; i++, fdev++, fdp++) { fde = &fdesc->fd_ofiles[*fdp]; fdep[i] = fdev; fdep[i]->fde_file = fde->fde_file; filecaps_copy(&fde->fde_caps, &fdep[i]->fde_caps, true); unp_internalize_fp(fdep[i]->fde_file); } FILEDESC_SUNLOCK(fdesc); break; case SCM_TIMESTAMP: *controlp = sbcreatecontrol(NULL, sizeof(*tv), SCM_TIMESTAMP, SOL_SOCKET); if (*controlp == NULL) { error = ENOBUFS; goto out; } tv = (struct timeval *) CMSG_DATA(mtod(*controlp, struct cmsghdr *)); microtime(tv); break; case SCM_BINTIME: *controlp = sbcreatecontrol(NULL, sizeof(*bt), SCM_BINTIME, SOL_SOCKET); if (*controlp == NULL) { error = ENOBUFS; goto out; } bt = (struct bintime *) CMSG_DATA(mtod(*controlp, struct cmsghdr *)); bintime(bt); break; default: error = EINVAL; goto out; } controlp = &(*controlp)->m_next; if (CMSG_SPACE(datalen) < clen) { clen -= CMSG_SPACE(datalen); cm = (struct cmsghdr *) ((caddr_t)cm + CMSG_SPACE(datalen)); } else { clen = 0; cm = NULL; } } out: m_freem(control); return (error); } static struct mbuf * unp_addsockcred(struct thread *td, struct mbuf *control) { struct mbuf *m, *n, *n_prev; struct sockcred *sc; const struct cmsghdr *cm; int ngroups; int i; ngroups = MIN(td->td_ucred->cr_ngroups, CMGROUP_MAX); m = sbcreatecontrol(NULL, SOCKCREDSIZE(ngroups), SCM_CREDS, SOL_SOCKET); if (m == NULL) return (control); sc = (struct sockcred *) CMSG_DATA(mtod(m, struct cmsghdr *)); sc->sc_uid = td->td_ucred->cr_ruid; sc->sc_euid = td->td_ucred->cr_uid; sc->sc_gid = td->td_ucred->cr_rgid; sc->sc_egid = td->td_ucred->cr_gid; sc->sc_ngroups = ngroups; for (i = 0; i < sc->sc_ngroups; i++) sc->sc_groups[i] = td->td_ucred->cr_groups[i]; /* * Unlink SCM_CREDS control messages (struct cmsgcred), since just * created SCM_CREDS control message (struct sockcred) has another * format. */ if (control != NULL) for (n = control, n_prev = NULL; n != NULL;) { cm = mtod(n, struct cmsghdr *); if (cm->cmsg_level == SOL_SOCKET && cm->cmsg_type == SCM_CREDS) { if (n_prev == NULL) control = n->m_next; else n_prev->m_next = n->m_next; n = m_free(n); } else { n_prev = n; n = n->m_next; } } /* Prepend it to the head. */ m->m_next = control; return (m); } static struct unpcb * fptounp(struct file *fp) { struct socket *so; if (fp->f_type != DTYPE_SOCKET) return (NULL); if ((so = fp->f_data) == NULL) return (NULL); if (so->so_proto->pr_domain != &localdomain) return (NULL); return sotounpcb(so); } static void unp_discard(struct file *fp) { struct unp_defer *dr; if (unp_externalize_fp(fp)) { dr = malloc(sizeof(*dr), M_TEMP, M_WAITOK); dr->ud_fp = fp; UNP_DEFERRED_LOCK(); SLIST_INSERT_HEAD(&unp_defers, dr, ud_link); UNP_DEFERRED_UNLOCK(); atomic_add_int(&unp_defers_count, 1); taskqueue_enqueue(taskqueue_thread, &unp_defer_task); } else (void) closef(fp, (struct thread *)NULL); } static void unp_process_defers(void *arg __unused, int pending) { struct unp_defer *dr; SLIST_HEAD(, unp_defer) drl; int count; SLIST_INIT(&drl); for (;;) { UNP_DEFERRED_LOCK(); if (SLIST_FIRST(&unp_defers) == NULL) { UNP_DEFERRED_UNLOCK(); break; } SLIST_SWAP(&unp_defers, &drl, unp_defer); UNP_DEFERRED_UNLOCK(); count = 0; while ((dr = SLIST_FIRST(&drl)) != NULL) { SLIST_REMOVE_HEAD(&drl, ud_link); closef(dr->ud_fp, NULL); free(dr, M_TEMP); count++; } atomic_add_int(&unp_defers_count, -count); } } static void unp_internalize_fp(struct file *fp) { struct unpcb *unp; UNP_LINK_WLOCK(); if ((unp = fptounp(fp)) != NULL) { unp->unp_file = fp; unp->unp_msgcount++; } fhold(fp); unp_rights++; UNP_LINK_WUNLOCK(); } static int unp_externalize_fp(struct file *fp) { struct unpcb *unp; int ret; UNP_LINK_WLOCK(); if ((unp = fptounp(fp)) != NULL) { unp->unp_msgcount--; ret = 1; } else ret = 0; unp_rights--; UNP_LINK_WUNLOCK(); return (ret); } /* * unp_defer indicates whether additional work has been defered for a future * pass through unp_gc(). It is thread local and does not require explicit * synchronization. */ static int unp_marked; static int unp_unreachable; static void unp_accessable(struct filedescent **fdep, int fdcount) { struct unpcb *unp; struct file *fp; int i; for (i = 0; i < fdcount; i++) { fp = fdep[i]->fde_file; if ((unp = fptounp(fp)) == NULL) continue; if (unp->unp_gcflag & UNPGC_REF) continue; unp->unp_gcflag &= ~UNPGC_DEAD; unp->unp_gcflag |= UNPGC_REF; unp_marked++; } } static void unp_gc_process(struct unpcb *unp) { struct socket *soa; struct socket *so; struct file *fp; /* Already processed. */ if (unp->unp_gcflag & UNPGC_SCANNED) return; fp = unp->unp_file; /* * Check for a socket potentially in a cycle. It must be in a * queue as indicated by msgcount, and this must equal the file * reference count. Note that when msgcount is 0 the file is NULL. */ if ((unp->unp_gcflag & UNPGC_REF) == 0 && fp && unp->unp_msgcount != 0 && fp->f_count == unp->unp_msgcount) { unp->unp_gcflag |= UNPGC_DEAD; unp_unreachable++; return; } /* * Mark all sockets we reference with RIGHTS. */ so = unp->unp_socket; if ((unp->unp_gcflag & UNPGC_IGNORE_RIGHTS) == 0) { SOCKBUF_LOCK(&so->so_rcv); unp_scan(so->so_rcv.sb_mb, unp_accessable); SOCKBUF_UNLOCK(&so->so_rcv); } /* * Mark all sockets in our accept queue. */ ACCEPT_LOCK(); TAILQ_FOREACH(soa, &so->so_comp, so_list) { if ((sotounpcb(soa)->unp_gcflag & UNPGC_IGNORE_RIGHTS) != 0) continue; SOCKBUF_LOCK(&soa->so_rcv); unp_scan(soa->so_rcv.sb_mb, unp_accessable); SOCKBUF_UNLOCK(&soa->so_rcv); } ACCEPT_UNLOCK(); unp->unp_gcflag |= UNPGC_SCANNED; } static int unp_recycled; SYSCTL_INT(_net_local, OID_AUTO, recycled, CTLFLAG_RD, &unp_recycled, 0, "Number of unreachable sockets claimed by the garbage collector."); static int unp_taskcount; SYSCTL_INT(_net_local, OID_AUTO, taskcount, CTLFLAG_RD, &unp_taskcount, 0, "Number of times the garbage collector has run."); static void unp_gc(__unused void *arg, int pending) { struct unp_head *heads[] = { &unp_dhead, &unp_shead, &unp_sphead, NULL }; struct unp_head **head; struct file *f, **unref; struct unpcb *unp; int i, total; unp_taskcount++; UNP_LIST_LOCK(); /* * First clear all gc flags from previous runs, apart from * UNPGC_IGNORE_RIGHTS. */ for (head = heads; *head != NULL; head++) LIST_FOREACH(unp, *head, unp_link) unp->unp_gcflag = (unp->unp_gcflag & UNPGC_IGNORE_RIGHTS); /* * Scan marking all reachable sockets with UNPGC_REF. Once a socket * is reachable all of the sockets it references are reachable. * Stop the scan once we do a complete loop without discovering * a new reachable socket. */ do { unp_unreachable = 0; unp_marked = 0; for (head = heads; *head != NULL; head++) LIST_FOREACH(unp, *head, unp_link) unp_gc_process(unp); } while (unp_marked); UNP_LIST_UNLOCK(); if (unp_unreachable == 0) return; /* * Allocate space for a local list of dead unpcbs. */ unref = malloc(unp_unreachable * sizeof(struct file *), M_TEMP, M_WAITOK); /* * Iterate looking for sockets which have been specifically marked * as as unreachable and store them locally. */ UNP_LINK_RLOCK(); UNP_LIST_LOCK(); for (total = 0, head = heads; *head != NULL; head++) LIST_FOREACH(unp, *head, unp_link) if ((unp->unp_gcflag & UNPGC_DEAD) != 0) { f = unp->unp_file; if (unp->unp_msgcount == 0 || f == NULL || f->f_count != unp->unp_msgcount) continue; unref[total++] = f; fhold(f); KASSERT(total <= unp_unreachable, ("unp_gc: incorrect unreachable count.")); } UNP_LIST_UNLOCK(); UNP_LINK_RUNLOCK(); /* * Now flush all sockets, free'ing rights. This will free the * struct files associated with these sockets but leave each socket * with one remaining ref. */ for (i = 0; i < total; i++) { struct socket *so; so = unref[i]->f_data; CURVNET_SET(so->so_vnet); sorflush(so); CURVNET_RESTORE(); } /* * And finally release the sockets so they can be reclaimed. */ for (i = 0; i < total; i++) fdrop(unref[i], NULL); unp_recycled += total; free(unref, M_TEMP); } static void -unp_dispose(struct mbuf *m) +unp_dispose_mbuf(struct mbuf *m) { if (m) unp_scan(m, unp_freerights); } /* * Synchronize against unp_gc, which can trip over data as we are freeing it. */ static void -unp_dispose_so(struct socket *so) +unp_dispose(struct socket *so) { struct unpcb *unp; unp = sotounpcb(so); UNP_LIST_LOCK(); unp->unp_gcflag |= UNPGC_IGNORE_RIGHTS; UNP_LIST_UNLOCK(); - unp_dispose(so->so_rcv.sb_mb); + unp_dispose_mbuf(so->so_rcv.sb_mb); } static void unp_scan(struct mbuf *m0, void (*op)(struct filedescent **, int)) { struct mbuf *m; struct cmsghdr *cm; void *data; socklen_t clen, datalen; while (m0 != NULL) { for (m = m0; m; m = m->m_next) { if (m->m_type != MT_CONTROL) continue; cm = mtod(m, struct cmsghdr *); clen = m->m_len; while (cm != NULL) { if (sizeof(*cm) > clen || cm->cmsg_len > clen) break; data = CMSG_DATA(cm); datalen = (caddr_t)cm + cm->cmsg_len - (caddr_t)data; if (cm->cmsg_level == SOL_SOCKET && cm->cmsg_type == SCM_RIGHTS) { (*op)(data, datalen / sizeof(struct filedescent *)); } if (CMSG_SPACE(datalen) < clen) { clen -= CMSG_SPACE(datalen); cm = (struct cmsghdr *) ((caddr_t)cm + CMSG_SPACE(datalen)); } else { clen = 0; cm = NULL; } } } m0 = m0->m_nextpkt; } } /* * A helper function called by VFS before socket-type vnode reclamation. * For an active vnode it clears unp_vnode pointer and decrements unp_vnode * use count. */ void vfs_unp_reclaim(struct vnode *vp) { struct socket *so; struct unpcb *unp; int active; ASSERT_VOP_ELOCKED(vp, "vfs_unp_reclaim"); KASSERT(vp->v_type == VSOCK, ("vfs_unp_reclaim: vp->v_type != VSOCK")); active = 0; UNP_LINK_WLOCK(); VOP_UNP_CONNECT(vp, &so); if (so == NULL) goto done; unp = sotounpcb(so); if (unp == NULL) goto done; UNP_PCB_LOCK(unp); if (unp->unp_vnode == vp) { VOP_UNP_DETACH(vp); unp->unp_vnode = NULL; active = 1; } UNP_PCB_UNLOCK(unp); done: UNP_LINK_WUNLOCK(); if (active) vunref(vp); } #ifdef DDB static void db_print_indent(int indent) { int i; for (i = 0; i < indent; i++) db_printf(" "); } static void db_print_unpflags(int unp_flags) { int comma; comma = 0; if (unp_flags & UNP_HAVEPC) { db_printf("%sUNP_HAVEPC", comma ? ", " : ""); comma = 1; } if (unp_flags & UNP_HAVEPCCACHED) { db_printf("%sUNP_HAVEPCCACHED", comma ? ", " : ""); comma = 1; } if (unp_flags & UNP_WANTCRED) { db_printf("%sUNP_WANTCRED", comma ? ", " : ""); comma = 1; } if (unp_flags & UNP_CONNWAIT) { db_printf("%sUNP_CONNWAIT", comma ? ", " : ""); comma = 1; } if (unp_flags & UNP_CONNECTING) { db_printf("%sUNP_CONNECTING", comma ? ", " : ""); comma = 1; } if (unp_flags & UNP_BINDING) { db_printf("%sUNP_BINDING", comma ? ", " : ""); comma = 1; } } static void db_print_xucred(int indent, struct xucred *xu) { int comma, i; db_print_indent(indent); db_printf("cr_version: %u cr_uid: %u cr_ngroups: %d\n", xu->cr_version, xu->cr_uid, xu->cr_ngroups); db_print_indent(indent); db_printf("cr_groups: "); comma = 0; for (i = 0; i < xu->cr_ngroups; i++) { db_printf("%s%u", comma ? ", " : "", xu->cr_groups[i]); comma = 1; } db_printf("\n"); } static void db_print_unprefs(int indent, struct unp_head *uh) { struct unpcb *unp; int counter; counter = 0; LIST_FOREACH(unp, uh, unp_reflink) { if (counter % 4 == 0) db_print_indent(indent); db_printf("%p ", unp); if (counter % 4 == 3) db_printf("\n"); counter++; } if (counter != 0 && counter % 4 != 0) db_printf("\n"); } DB_SHOW_COMMAND(unpcb, db_show_unpcb) { struct unpcb *unp; if (!have_addr) { db_printf("usage: show unpcb \n"); return; } unp = (struct unpcb *)addr; db_printf("unp_socket: %p unp_vnode: %p\n", unp->unp_socket, unp->unp_vnode); db_printf("unp_ino: %ju unp_conn: %p\n", (uintmax_t)unp->unp_ino, unp->unp_conn); db_printf("unp_refs:\n"); db_print_unprefs(2, &unp->unp_refs); /* XXXRW: Would be nice to print the full address, if any. */ db_printf("unp_addr: %p\n", unp->unp_addr); db_printf("unp_gencnt: %llu\n", (unsigned long long)unp->unp_gencnt); db_printf("unp_flags: %x (", unp->unp_flags); db_print_unpflags(unp->unp_flags); db_printf(")\n"); db_printf("unp_peercred:\n"); db_print_xucred(2, &unp->unp_peercred); db_printf("unp_refcount: %u\n", unp->unp_refcount); } #endif Index: projects/netbsd-tests-update-12/sys/net/rndis.h =================================================================== --- projects/netbsd-tests-update-12/sys/net/rndis.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/net/rndis.h (revision 305172) @@ -1,283 +1,287 @@ /* $FreeBSD$ */ /* $OpenBSD: if_urndisreg.h,v 1.19 2013/11/21 14:08:05 mpi Exp $ */ /* * Copyright (c) 2010 Jonathan Armani * Copyright (c) 2010 Fabien Romano * Copyright (c) 2010 Michael Knudsen * All rights reserved. * * Permission to use, copy, modify, and distribute this software for any * purpose with or without fee is hereby granted, provided that the above * copyright notice and this permission notice appear in all copies. * * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ #ifndef _NET_RNDIS_H_ #define _NET_RNDIS_H_ /* Canonical major/minor version as of 22th Aug. 2016. */ #define RNDIS_VERSION_MAJOR 0x00000001 #define RNDIS_VERSION_MINOR 0x00000000 #define RNDIS_STATUS_SUCCESS 0x00000000L #define RNDIS_STATUS_PENDING 0x00000103L #define RNDIS_STATUS_MEDIA_CONNECT 0x4001000BL #define RNDIS_STATUS_MEDIA_DISCONNECT 0x4001000CL #define RNDIS_STATUS_BUFFER_OVERFLOW 0x80000005L #define RNDIS_STATUS_FAILURE 0xC0000001L #define RNDIS_STATUS_NOT_SUPPORTED 0xC00000BBL #define RNDIS_STATUS_RESOURCES 0xC000009AL #define RNDIS_STATUS_INVALID_DATA 0xC0010015L #define OID_GEN_SUPPORTED_LIST 0x00010101 #define OID_GEN_HARDWARE_STATUS 0x00010102 #define OID_GEN_MEDIA_SUPPORTED 0x00010103 #define OID_GEN_MEDIA_IN_USE 0x00010104 #define OID_GEN_MAXIMUM_LOOKAHEAD 0x00010105 #define OID_GEN_MAXIMUM_FRAME_SIZE 0x00010106 #define OID_GEN_LINK_SPEED 0x00010107 #define OID_GEN_TRANSMIT_BUFFER_SPACE 0x00010108 #define OID_GEN_RECEIVE_BUFFER_SPACE 0x00010109 #define OID_GEN_TRANSMIT_BLOCK_SIZE 0x0001010A #define OID_GEN_RECEIVE_BLOCK_SIZE 0x0001010B #define OID_GEN_VENDOR_ID 0x0001010C #define OID_GEN_VENDOR_DESCRIPTION 0x0001010D #define OID_GEN_CURRENT_PACKET_FILTER 0x0001010E #define OID_GEN_CURRENT_LOOKAHEAD 0x0001010F #define OID_GEN_DRIVER_VERSION 0x00010110 #define OID_GEN_MAXIMUM_TOTAL_SIZE 0x00010111 #define OID_GEN_PROTOCOL_OPTIONS 0x00010112 #define OID_GEN_MAC_OPTIONS 0x00010113 #define OID_GEN_MEDIA_CONNECT_STATUS 0x00010114 #define OID_GEN_MAXIMUM_SEND_PACKETS 0x00010115 #define OID_GEN_VENDOR_DRIVER_VERSION 0x00010116 #define OID_GEN_SUPPORTED_GUIDS 0x00010117 #define OID_GEN_NETWORK_LAYER_ADDRESSES 0x00010118 #define OID_GEN_TRANSPORT_HEADER_OFFSET 0x00010119 +#define OID_GEN_RECEIVE_SCALE_CAPABILITIES 0x00010203 +#define OID_GEN_RECEIVE_SCALE_PARAMETERS 0x00010204 #define OID_GEN_MACHINE_NAME 0x0001021A #define OID_GEN_RNDIS_CONFIG_PARAMETER 0x0001021B #define OID_GEN_VLAN_ID 0x0001021C #define OID_802_3_PERMANENT_ADDRESS 0x01010101 #define OID_802_3_CURRENT_ADDRESS 0x01010102 #define OID_802_3_MULTICAST_LIST 0x01010103 #define OID_802_3_MAXIMUM_LIST_SIZE 0x01010104 #define OID_802_3_MAC_OPTIONS 0x01010105 #define OID_802_3_RCV_ERROR_ALIGNMENT 0x01020101 #define OID_802_3_XMIT_ONE_COLLISION 0x01020102 #define OID_802_3_XMIT_MORE_COLLISIONS 0x01020103 #define OID_802_3_XMIT_DEFERRED 0x01020201 #define OID_802_3_XMIT_MAX_COLLISIONS 0x01020202 #define OID_802_3_RCV_OVERRUN 0x01020203 #define OID_802_3_XMIT_UNDERRUN 0x01020204 #define OID_802_3_XMIT_HEARTBEAT_FAILURE 0x01020205 #define OID_802_3_XMIT_TIMES_CRS_LOST 0x01020206 #define OID_802_3_XMIT_LATE_COLLISIONS 0x01020207 +#define OID_TCP_OFFLOAD_PARAMETERS 0xFC01020C + #define RNDIS_MEDIUM_802_3 0x00000000 /* Device flags */ #define RNDIS_DF_CONNECTIONLESS 0x00000001 #define RNDIS_DF_CONNECTION_ORIENTED 0x00000002 /* * RNDIS data message */ #define REMOTE_NDIS_PACKET_MSG 0x00000001 struct rndis_packet_msg { uint32_t rm_type; uint32_t rm_len; uint32_t rm_dataoffset; uint32_t rm_datalen; uint32_t rm_oobdataoffset; uint32_t rm_oobdatalen; uint32_t rm_oobdataelements; uint32_t rm_pktinfooffset; uint32_t rm_pktinfolen; uint32_t rm_vchandle; uint32_t rm_reserved; }; /* * RNDIS control messages */ struct rndis_comp_hdr { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; uint32_t rm_status; }; /* Initialize the device. */ #define REMOTE_NDIS_INITIALIZE_MSG 0x00000002 #define REMOTE_NDIS_INITIALIZE_CMPLT 0x80000002 struct rndis_init_req { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; uint32_t rm_ver_major; uint32_t rm_ver_minor; uint32_t rm_max_xfersz; }; struct rndis_init_comp { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; uint32_t rm_status; uint32_t rm_ver_major; uint32_t rm_ver_minor; uint32_t rm_devflags; uint32_t rm_medium; uint32_t rm_pktmaxcnt; uint32_t rm_pktmaxsz; uint32_t rm_align; uint32_t rm_aflistoffset; uint32_t rm_aflistsz; }; #define RNDIS_INIT_COMP_SIZE_MIN \ __offsetof(struct rndis_init_comp, rm_aflistsz) /* Halt the device. No response sent. */ #define REMOTE_NDIS_HALT_MSG 0x00000003 struct rndis_halt_req { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; }; /* Send a query object. */ #define REMOTE_NDIS_QUERY_MSG 0x00000004 #define REMOTE_NDIS_QUERY_CMPLT 0x80000004 struct rndis_query_req { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; uint32_t rm_oid; uint32_t rm_infobuflen; uint32_t rm_infobufoffset; uint32_t rm_devicevchdl; }; #define RNDIS_QUERY_REQ_INFOBUFOFFSET \ (sizeof(struct rndis_query_req) - \ __offsetof(struct rndis_query_req, rm_rid)) struct rndis_query_comp { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; uint32_t rm_status; uint32_t rm_infobuflen; uint32_t rm_infobufoffset; }; #define RNDIS_QUERY_COMP_INFOBUFABS(ofs) \ ((ofs) + __offsetof(struct rndis_query_req, rm_rid)) /* Send a set object request. */ #define REMOTE_NDIS_SET_MSG 0x00000005 #define REMOTE_NDIS_SET_CMPLT 0x80000005 struct rndis_set_req { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; uint32_t rm_oid; uint32_t rm_infobuflen; uint32_t rm_infobufoffset; uint32_t rm_devicevchdl; }; #define RNDIS_SET_REQ_INFOBUFOFFSET \ (sizeof(struct rndis_set_req) - \ __offsetof(struct rndis_set_req, rm_rid)) struct rndis_set_comp { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; uint32_t rm_status; }; #define REMOTE_NDIS_SET_PARAM_NUMERIC 0x00000000 #define REMOTE_NDIS_SET_PARAM_STRING 0x00000002 struct rndis_set_parameter { uint32_t rm_nameoffset; uint32_t rm_namelen; uint32_t rm_type; uint32_t rm_valueoffset; uint32_t rm_valuelen; }; /* Perform a soft reset on the device. */ #define REMOTE_NDIS_RESET_MSG 0x00000006 #define REMOTE_NDIS_RESET_CMPLT 0x80000006 struct rndis_reset_req { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; }; struct rndis_reset_comp { uint32_t rm_type; uint32_t rm_len; uint32_t rm_status; uint32_t rm_adrreset; }; /* 802.3 link-state or undefined message error. */ #define REMOTE_NDIS_INDICATE_STATUS_MSG 0x00000007 /* Keepalive messsage. May be sent by device. */ #define REMOTE_NDIS_KEEPALIVE_MSG 0x00000008 #define REMOTE_NDIS_KEEPALIVE_CMPLT 0x80000008 struct rndis_keepalive_req { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; }; struct rndis_keepalive_comp { uint32_t rm_type; uint32_t rm_len; uint32_t rm_rid; uint32_t rm_status; }; /* packet filter bits used by OID_GEN_CURRENT_PACKET_FILTER */ -#define RNDIS_PACKET_TYPE_DIRECTED 0x00000001 -#define RNDIS_PACKET_TYPE_MULTICAST 0x00000002 -#define RNDIS_PACKET_TYPE_ALL_MULTICAST 0x00000004 -#define RNDIS_PACKET_TYPE_BROADCAST 0x00000008 -#define RNDIS_PACKET_TYPE_SOURCE_ROUTING 0x00000010 -#define RNDIS_PACKET_TYPE_PROMISCUOUS 0x00000020 -#define RNDIS_PACKET_TYPE_SMT 0x00000040 -#define RNDIS_PACKET_TYPE_ALL_LOCAL 0x00000080 -#define RNDIS_PACKET_TYPE_GROUP 0x00001000 -#define RNDIS_PACKET_TYPE_ALL_FUNCTIONAL 0x00002000 -#define RNDIS_PACKET_TYPE_FUNCTIONAL 0x00004000 -#define RNDIS_PACKET_TYPE_MAC_FRAME 0x00008000 +#define NDIS_PACKET_TYPE_DIRECTED 0x00000001 +#define NDIS_PACKET_TYPE_MULTICAST 0x00000002 +#define NDIS_PACKET_TYPE_ALL_MULTICAST 0x00000004 +#define NDIS_PACKET_TYPE_BROADCAST 0x00000008 +#define NDIS_PACKET_TYPE_SOURCE_ROUTING 0x00000010 +#define NDIS_PACKET_TYPE_PROMISCUOUS 0x00000020 +#define NDIS_PACKET_TYPE_SMT 0x00000040 +#define NDIS_PACKET_TYPE_ALL_LOCAL 0x00000080 +#define NDIS_PACKET_TYPE_GROUP 0x00001000 +#define NDIS_PACKET_TYPE_ALL_FUNCTIONAL 0x00002000 +#define NDIS_PACKET_TYPE_FUNCTIONAL 0x00004000 +#define NDIS_PACKET_TYPE_MAC_FRAME 0x00008000 /* RNDIS offsets */ #define RNDIS_HEADER_OFFSET 8 /* bytes */ #define RNDIS_DATA_OFFSET \ ((uint32_t)(sizeof(struct rndis_packet_msg) - RNDIS_HEADER_OFFSET)) #endif /* !_NET_RNDIS_H_ */ Index: projects/netbsd-tests-update-12/sys/netipsec/ipsec_input.c =================================================================== --- projects/netbsd-tests-update-12/sys/netipsec/ipsec_input.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/netipsec/ipsec_input.c (revision 305172) @@ -1,924 +1,908 @@ /* $FreeBSD$ */ /* $OpenBSD: ipsec_input.c,v 1.63 2003/02/20 18:35:43 deraadt Exp $ */ /*- * The authors of this code are John Ioannidis (ji@tla.org), * Angelos D. Keromytis (kermit@csd.uch.gr) and * Niels Provos (provos@physnet.uni-hamburg.de). * * This code was written by John Ioannidis for BSD/OS in Athens, Greece, * in November 1995. * * Ported to OpenBSD and NetBSD, with additional transforms, in December 1996, * by Angelos D. Keromytis. * * Additional transforms and features in 1997 and 1998 by Angelos D. Keromytis * and Niels Provos. * * Additional features in 1999 by Angelos D. Keromytis. * * Copyright (C) 1995, 1996, 1997, 1998, 1999 by John Ioannidis, * Angelos D. Keromytis and Niels Provos. * Copyright (c) 2001, Angelos D. Keromytis. * * Permission to use, copy, and modify this software with or without fee * is hereby granted, provided that this entire notice is included in * all copies of any software which is or includes a copy or * modification of this software. * You may use this code under the GNU public license if you so wish. Please * contribute changes back to the authors under this freer than GPL license * so that we may further the use of strong encryption without limitations to * all. * * THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR * IMPLIED WARRANTY. IN PARTICULAR, NONE OF THE AUTHORS MAKES ANY * REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE * MERCHANTABILITY OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR * PURPOSE. */ /* * IPsec input processing. */ #include "opt_inet.h" #include "opt_inet6.h" #include "opt_ipsec.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifdef INET6 #include #endif #include #ifdef INET6 #include #endif #include #ifdef INET6 #include #endif #include #include #include #include #include #include #include #include #include #include #define IPSEC_ISTAT(proto, name) do { \ if ((proto) == IPPROTO_ESP) \ ESPSTAT_INC(esps_##name); \ else if ((proto) == IPPROTO_AH) \ AHSTAT_INC(ahs_##name); \ else \ IPCOMPSTAT_INC(ipcomps_##name); \ } while (0) #ifdef INET static void ipsec4_common_ctlinput(int, struct sockaddr *, void *, int); #endif /* * ipsec_common_input gets called when an IPsec-protected packet * is received by IPv4 or IPv6. Its job is to find the right SA * and call the appropriate transform. The transform callback * takes care of further processing (like ingress filtering). */ int ipsec_common_input(struct mbuf *m, int skip, int protoff, int af, int sproto) { char buf[INET6_ADDRSTRLEN]; union sockaddr_union dst_address; struct secasvar *sav; u_int32_t spi; int error; #ifdef INET #ifdef IPSEC_NAT_T struct m_tag *tag; #endif #endif IPSEC_ISTAT(sproto, input); IPSEC_ASSERT(m != NULL, ("null packet")); IPSEC_ASSERT(sproto == IPPROTO_ESP || sproto == IPPROTO_AH || sproto == IPPROTO_IPCOMP, ("unexpected security protocol %u", sproto)); if ((sproto == IPPROTO_ESP && !V_esp_enable) || (sproto == IPPROTO_AH && !V_ah_enable) || (sproto == IPPROTO_IPCOMP && !V_ipcomp_enable)) { m_freem(m); IPSEC_ISTAT(sproto, pdrops); return EOPNOTSUPP; } if (m->m_pkthdr.len - skip < 2 * sizeof (u_int32_t)) { m_freem(m); IPSEC_ISTAT(sproto, hdrops); DPRINTF(("%s: packet too small\n", __func__)); return EINVAL; } /* Retrieve the SPI from the relevant IPsec header */ if (sproto == IPPROTO_ESP) m_copydata(m, skip, sizeof(u_int32_t), (caddr_t) &spi); else if (sproto == IPPROTO_AH) m_copydata(m, skip + sizeof(u_int32_t), sizeof(u_int32_t), (caddr_t) &spi); else if (sproto == IPPROTO_IPCOMP) { u_int16_t cpi; m_copydata(m, skip + sizeof(u_int16_t), sizeof(u_int16_t), (caddr_t) &cpi); spi = ntohl(htons(cpi)); } /* * Find the SA and (indirectly) call the appropriate * kernel crypto routine. The resulting mbuf chain is a valid * IP packet ready to go through input processing. */ bzero(&dst_address, sizeof (dst_address)); dst_address.sa.sa_family = af; switch (af) { #ifdef INET case AF_INET: dst_address.sin.sin_len = sizeof(struct sockaddr_in); m_copydata(m, offsetof(struct ip, ip_dst), sizeof(struct in_addr), (caddr_t) &dst_address.sin.sin_addr); #ifdef IPSEC_NAT_T /* Find the source port for NAT-T; see udp*_espdecap. */ tag = m_tag_find(m, PACKET_TAG_IPSEC_NAT_T_PORTS, NULL); if (tag != NULL) dst_address.sin.sin_port = ((u_int16_t *)(tag + 1))[1]; #endif /* IPSEC_NAT_T */ break; #endif /* INET */ #ifdef INET6 case AF_INET6: dst_address.sin6.sin6_len = sizeof(struct sockaddr_in6); m_copydata(m, offsetof(struct ip6_hdr, ip6_dst), sizeof(struct in6_addr), (caddr_t) &dst_address.sin6.sin6_addr); /* We keep addresses in SADB without embedded scope id */ if (IN6_IS_SCOPE_LINKLOCAL(&dst_address.sin6.sin6_addr)) { /* XXX: sa6_recoverscope() */ dst_address.sin6.sin6_scope_id = ntohs(dst_address.sin6.sin6_addr.s6_addr16[1]); dst_address.sin6.sin6_addr.s6_addr16[1] = 0; } break; #endif /* INET6 */ default: DPRINTF(("%s: unsupported protocol family %u\n", __func__, af)); m_freem(m); IPSEC_ISTAT(sproto, nopf); return EPFNOSUPPORT; } /* NB: only pass dst since key_allocsa follows RFC2401 */ sav = KEY_ALLOCSA(&dst_address, sproto, spi); if (sav == NULL) { DPRINTF(("%s: no key association found for SA %s/%08lx/%u\n", __func__, ipsec_address(&dst_address, buf, sizeof(buf)), (u_long) ntohl(spi), sproto)); IPSEC_ISTAT(sproto, notdb); m_freem(m); return ENOENT; } if (sav->tdb_xform == NULL) { DPRINTF(("%s: attempted to use uninitialized SA %s/%08lx/%u\n", __func__, ipsec_address(&dst_address, buf, sizeof(buf)), (u_long) ntohl(spi), sproto)); IPSEC_ISTAT(sproto, noxform); KEY_FREESAV(&sav); m_freem(m); return ENXIO; } /* * Call appropriate transform and return -- callback takes care of * everything else. */ error = (*sav->tdb_xform->xf_input)(m, sav, skip, protoff); KEY_FREESAV(&sav); return error; } #ifdef INET int ah4_input(struct mbuf **mp, int *offp, int proto) { struct mbuf *m; int off; m = *mp; off = *offp; *mp = NULL; ipsec_common_input(m, off, offsetof(struct ip, ip_p), AF_INET, IPPROTO_AH); return (IPPROTO_DONE); } void ah4_ctlinput(int cmd, struct sockaddr *sa, void *v) { if (sa->sa_family == AF_INET && sa->sa_len == sizeof(struct sockaddr_in)) ipsec4_common_ctlinput(cmd, sa, v, IPPROTO_AH); } int esp4_input(struct mbuf **mp, int *offp, int proto) { struct mbuf *m; int off; m = *mp; off = *offp; mp = NULL; ipsec_common_input(m, off, offsetof(struct ip, ip_p), AF_INET, IPPROTO_ESP); return (IPPROTO_DONE); } void esp4_ctlinput(int cmd, struct sockaddr *sa, void *v) { if (sa->sa_family == AF_INET && sa->sa_len == sizeof(struct sockaddr_in)) ipsec4_common_ctlinput(cmd, sa, v, IPPROTO_ESP); } int ipcomp4_input(struct mbuf **mp, int *offp, int proto) { struct mbuf *m; int off; m = *mp; off = *offp; mp = NULL; ipsec_common_input(m, off, offsetof(struct ip, ip_p), AF_INET, IPPROTO_IPCOMP); return (IPPROTO_DONE); } /* * IPsec input callback for INET protocols. * This routine is called as the transform callback. * Takes care of filtering and other sanity checks on * the processed packet. */ int ipsec4_common_input_cb(struct mbuf *m, struct secasvar *sav, int skip, int protoff) { char buf[INET6_ADDRSTRLEN]; struct ipsec_ctx_data ctx; int prot, af, sproto, isr_prot; struct ip *ip; struct m_tag *mtag; struct tdb_ident *tdbi; struct secasindex *saidx; int error; #ifdef INET6 #ifdef notyet char ip6buf[INET6_ADDRSTRLEN]; #endif #endif IPSEC_ASSERT(m != NULL, ("null mbuf")); IPSEC_ASSERT(sav != NULL, ("null SA")); IPSEC_ASSERT(sav->sah != NULL, ("null SAH")); saidx = &sav->sah->saidx; af = saidx->dst.sa.sa_family; IPSEC_ASSERT(af == AF_INET, ("unexpected af %u", af)); sproto = saidx->proto; IPSEC_ASSERT(sproto == IPPROTO_ESP || sproto == IPPROTO_AH || sproto == IPPROTO_IPCOMP, ("unexpected security protocol %u", sproto)); - /* Sanity check */ - if (m == NULL) { - DPRINTF(("%s: null mbuf", __func__)); - IPSEC_ISTAT(sproto, badkcr); - KEY_FREESAV(&sav); - return EINVAL; - } - if (skip != 0) { /* * Fix IPv4 header * XXXGL: do we need this entire block? */ if (m->m_len < skip && (m = m_pullup(m, skip)) == NULL) { DPRINTF(("%s: processing failed for SA %s/%08lx\n", __func__, ipsec_address(&sav->sah->saidx.dst, buf, sizeof(buf)), (u_long) ntohl(sav->spi))); IPSEC_ISTAT(sproto, hdrops); error = ENOBUFS; goto bad; } ip = mtod(m, struct ip *); ip->ip_len = htons(m->m_pkthdr.len); ip->ip_sum = 0; ip->ip_sum = in_cksum(m, ip->ip_hl << 2); } else { ip = mtod(m, struct ip *); } prot = ip->ip_p; IPSEC_INIT_CTX(&ctx, &m, sav, AF_INET, IPSEC_ENC_BEFORE); if ((error = ipsec_run_hhooks(&ctx, HHOOK_TYPE_IPSEC_IN)) != 0) goto bad; ip = mtod(m, struct ip *); /* IP-in-IP encapsulation */ if (prot == IPPROTO_IPIP && saidx->mode != IPSEC_MODE_TRANSPORT) { if (m->m_pkthdr.len - skip < sizeof(struct ip)) { IPSEC_ISTAT(sproto, hdrops); error = EINVAL; goto bad; } /* enc0: strip outer IPv4 header */ m_striphdr(m, 0, ip->ip_hl << 2); #ifdef notyet /* XXX PROXY address isn't recorded in SAH */ /* * Check that the inner source address is the same as * the proxy address, if available. */ if ((saidx->proxy.sa.sa_family == AF_INET && saidx->proxy.sin.sin_addr.s_addr != INADDR_ANY && ipn.ip_src.s_addr != saidx->proxy.sin.sin_addr.s_addr) || (saidx->proxy.sa.sa_family != AF_INET && saidx->proxy.sa.sa_family != 0)) { DPRINTF(("%s: inner source address %s doesn't " "correspond to expected proxy source %s, " "SA %s/%08lx\n", __func__, inet_ntoa4(ipn.ip_src), ipsp_address(saidx->proxy), ipsp_address(saidx->dst), (u_long) ntohl(sav->spi))); IPSEC_ISTAT(sproto, pdrops); error = EACCES; goto bad; } #endif /* notyet */ } #ifdef INET6 /* IPv6-in-IP encapsulation. */ else if (prot == IPPROTO_IPV6 && saidx->mode != IPSEC_MODE_TRANSPORT) { if (m->m_pkthdr.len - skip < sizeof(struct ip6_hdr)) { IPSEC_ISTAT(sproto, hdrops); error = EINVAL; goto bad; } /* enc0: strip IPv4 header, keep IPv6 header only */ m_striphdr(m, 0, ip->ip_hl << 2); #ifdef notyet /* * Check that the inner source address is the same as * the proxy address, if available. */ if ((saidx->proxy.sa.sa_family == AF_INET6 && !IN6_IS_ADDR_UNSPECIFIED(&saidx->proxy.sin6.sin6_addr) && !IN6_ARE_ADDR_EQUAL(&ip6n.ip6_src, &saidx->proxy.sin6.sin6_addr)) || (saidx->proxy.sa.sa_family != AF_INET6 && saidx->proxy.sa.sa_family != 0)) { DPRINTF(("%s: inner source address %s doesn't " "correspond to expected proxy source %s, " "SA %s/%08lx\n", __func__, ip6_sprintf(ip6buf, &ip6n.ip6_src), ipsec_address(&saidx->proxy), ipsec_address(&saidx->dst), (u_long) ntohl(sav->spi))); IPSEC_ISTAT(sproto, pdrops); error = EACCES; goto bad; } #endif /* notyet */ } #endif /* INET6 */ else if (prot != IPPROTO_IPV6 && saidx->mode == IPSEC_MODE_ANY) { /* * When mode is wildcard, inner protocol is IPv6 and * we have no INET6 support - drop this packet a bit later. * In other cases we assume transport mode and outer * header was already stripped in xform_xxx_cb. */ prot = IPPROTO_IPIP; } /* * Record what we've done to the packet (under what SA it was * processed). */ if (sproto != IPPROTO_IPCOMP) { mtag = m_tag_get(PACKET_TAG_IPSEC_IN_DONE, sizeof(struct tdb_ident), M_NOWAIT); if (mtag == NULL) { DPRINTF(("%s: failed to get tag\n", __func__)); IPSEC_ISTAT(sproto, hdrops); error = ENOMEM; goto bad; } tdbi = (struct tdb_ident *)(mtag + 1); bcopy(&saidx->dst, &tdbi->dst, saidx->dst.sa.sa_len); tdbi->proto = sproto; tdbi->spi = sav->spi; /* Cache those two for enc(4) in xform_ipip. */ tdbi->alg_auth = sav->alg_auth; tdbi->alg_enc = sav->alg_enc; m_tag_prepend(m, mtag); } key_sa_recordxfer(sav, m); /* record data transfer */ /* * In transport mode requeue decrypted mbuf back to IPv4 protocol * handler. This is necessary to correctly expose rcvif. */ if (saidx->mode == IPSEC_MODE_TRANSPORT) prot = IPPROTO_IPIP; /* * Re-dispatch via software interrupt. */ switch (prot) { case IPPROTO_IPIP: isr_prot = NETISR_IP; af = AF_INET; break; #ifdef INET6 case IPPROTO_IPV6: isr_prot = NETISR_IPV6; af = AF_INET6; break; #endif default: DPRINTF(("%s: cannot handle inner ip proto %d\n", __func__, prot)); IPSEC_ISTAT(sproto, nopf); error = EPFNOSUPPORT; goto bad; } IPSEC_INIT_CTX(&ctx, &m, sav, af, IPSEC_ENC_AFTER); if ((error = ipsec_run_hhooks(&ctx, HHOOK_TYPE_IPSEC_IN)) != 0) goto bad; error = netisr_queue_src(isr_prot, (uintptr_t)sav->spi, m); if (error) { IPSEC_ISTAT(sproto, qfull); DPRINTF(("%s: queue full; proto %u packet dropped\n", __func__, sproto)); return error; } return 0; bad: m_freem(m); return error; } void ipsec4_common_ctlinput(int cmd, struct sockaddr *sa, void *v, int proto) { /* XXX nothing just yet */ } #endif /* INET */ #ifdef INET6 /* IPv6 AH wrapper. */ int ipsec6_common_input(struct mbuf **mp, int *offp, int proto) { int l = 0; int protoff; struct ip6_ext ip6e; if (*offp < sizeof(struct ip6_hdr)) { DPRINTF(("%s: bad offset %u\n", __func__, *offp)); return IPPROTO_DONE; } else if (*offp == sizeof(struct ip6_hdr)) { protoff = offsetof(struct ip6_hdr, ip6_nxt); } else { /* Chase down the header chain... */ protoff = sizeof(struct ip6_hdr); do { protoff += l; m_copydata(*mp, protoff, sizeof(ip6e), (caddr_t) &ip6e); if (ip6e.ip6e_nxt == IPPROTO_AH) l = (ip6e.ip6e_len + 2) << 2; else l = (ip6e.ip6e_len + 1) << 3; IPSEC_ASSERT(l > 0, ("l went zero or negative")); } while (protoff + l < *offp); /* Malformed packet check */ if (protoff + l != *offp) { DPRINTF(("%s: bad packet header chain, protoff %u, " "l %u, off %u\n", __func__, protoff, l, *offp)); IPSEC_ISTAT(proto, hdrops); m_freem(*mp); *mp = NULL; return IPPROTO_DONE; } protoff += offsetof(struct ip6_ext, ip6e_nxt); } (void) ipsec_common_input(*mp, *offp, protoff, AF_INET6, proto); return IPPROTO_DONE; } /* * IPsec input callback, called by the transform callback. Takes care of * filtering and other sanity checks on the processed packet. */ int ipsec6_common_input_cb(struct mbuf *m, struct secasvar *sav, int skip, int protoff) { char buf[INET6_ADDRSTRLEN]; struct ipsec_ctx_data ctx; int prot, af, sproto; struct ip6_hdr *ip6; struct m_tag *mtag; struct tdb_ident *tdbi; struct secasindex *saidx; int nxt, isr_prot; u_int8_t nxt8; int error, nest; #ifdef notyet char ip6buf[INET6_ADDRSTRLEN]; #endif IPSEC_ASSERT(m != NULL, ("null mbuf")); IPSEC_ASSERT(sav != NULL, ("null SA")); IPSEC_ASSERT(sav->sah != NULL, ("null SAH")); saidx = &sav->sah->saidx; af = saidx->dst.sa.sa_family; IPSEC_ASSERT(af == AF_INET6, ("unexpected af %u", af)); sproto = saidx->proto; IPSEC_ASSERT(sproto == IPPROTO_ESP || sproto == IPPROTO_AH || sproto == IPPROTO_IPCOMP, ("unexpected security protocol %u", sproto)); - - /* Sanity check */ - if (m == NULL) { - DPRINTF(("%s: null mbuf", __func__)); - IPSEC_ISTAT(sproto, badkcr); - error = EINVAL; - goto bad; - } /* Fix IPv6 header */ if (m->m_len < sizeof(struct ip6_hdr) && (m = m_pullup(m, sizeof(struct ip6_hdr))) == NULL) { DPRINTF(("%s: processing failed for SA %s/%08lx\n", __func__, ipsec_address(&sav->sah->saidx.dst, buf, sizeof(buf)), (u_long) ntohl(sav->spi))); IPSEC_ISTAT(sproto, hdrops); error = EACCES; goto bad; } ip6 = mtod(m, struct ip6_hdr *); ip6->ip6_plen = htons(m->m_pkthdr.len - sizeof(struct ip6_hdr)); IPSEC_INIT_CTX(&ctx, &m, sav, af, IPSEC_ENC_BEFORE); if ((error = ipsec_run_hhooks(&ctx, HHOOK_TYPE_IPSEC_IN)) != 0) goto bad; /* Save protocol */ m_copydata(m, protoff, 1, &nxt8); prot = nxt8; /* IPv6-in-IP encapsulation */ if (prot == IPPROTO_IPV6 && saidx->mode != IPSEC_MODE_TRANSPORT) { if (m->m_pkthdr.len - skip < sizeof(struct ip6_hdr)) { IPSEC_ISTAT(sproto, hdrops); error = EINVAL; goto bad; } /* ip6n will now contain the inner IPv6 header. */ m_striphdr(m, 0, skip); skip = 0; #ifdef notyet /* * Check that the inner source address is the same as * the proxy address, if available. */ if ((saidx->proxy.sa.sa_family == AF_INET6 && !IN6_IS_ADDR_UNSPECIFIED(&saidx->proxy.sin6.sin6_addr) && !IN6_ARE_ADDR_EQUAL(&ip6n.ip6_src, &saidx->proxy.sin6.sin6_addr)) || (saidx->proxy.sa.sa_family != AF_INET6 && saidx->proxy.sa.sa_family != 0)) { DPRINTF(("%s: inner source address %s doesn't " "correspond to expected proxy source %s, " "SA %s/%08lx\n", __func__, ip6_sprintf(ip6buf, &ip6n.ip6_src), ipsec_address(&saidx->proxy), ipsec_address(&saidx->dst), (u_long) ntohl(sav->spi))); IPSEC_ISTAT(sproto, pdrops); error = EACCES; goto bad; } #endif /* notyet */ } #ifdef INET /* IP-in-IP encapsulation */ else if (prot == IPPROTO_IPIP && saidx->mode != IPSEC_MODE_TRANSPORT) { if (m->m_pkthdr.len - skip < sizeof(struct ip)) { IPSEC_ISTAT(sproto, hdrops); error = EINVAL; goto bad; } /* ipn will now contain the inner IPv4 header */ m_striphdr(m, 0, skip); skip = 0; #ifdef notyet /* * Check that the inner source address is the same as * the proxy address, if available. */ if ((saidx->proxy.sa.sa_family == AF_INET && saidx->proxy.sin.sin_addr.s_addr != INADDR_ANY && ipn.ip_src.s_addr != saidx->proxy.sin.sin_addr.s_addr) || (saidx->proxy.sa.sa_family != AF_INET && saidx->proxy.sa.sa_family != 0)) { DPRINTF(("%s: inner source address %s doesn't " "correspond to expected proxy source %s, " "SA %s/%08lx\n", __func__, inet_ntoa4(ipn.ip_src), ipsec_address(&saidx->proxy), ipsec_address(&saidx->dst), (u_long) ntohl(sav->spi))); IPSEC_ISTAT(sproto, pdrops); error = EACCES; goto bad; } #endif /* notyet */ } #endif /* INET */ else { prot = IPPROTO_IPV6; /* for correct BPF processing */ } /* * Record what we've done to the packet (under what SA it was * processed). */ if (sproto != IPPROTO_IPCOMP) { mtag = m_tag_get(PACKET_TAG_IPSEC_IN_DONE, sizeof(struct tdb_ident), M_NOWAIT); if (mtag == NULL) { DPRINTF(("%s: failed to get tag\n", __func__)); IPSEC_ISTAT(sproto, hdrops); error = ENOMEM; goto bad; } tdbi = (struct tdb_ident *)(mtag + 1); bcopy(&saidx->dst, &tdbi->dst, sizeof(union sockaddr_union)); tdbi->proto = sproto; tdbi->spi = sav->spi; /* Cache those two for enc(4) in xform_ipip. */ tdbi->alg_auth = sav->alg_auth; tdbi->alg_enc = sav->alg_enc; m_tag_prepend(m, mtag); } key_sa_recordxfer(sav, m); #ifdef INET if (prot == IPPROTO_IPIP) af = AF_INET; else #endif af = AF_INET6; IPSEC_INIT_CTX(&ctx, &m, sav, af, IPSEC_ENC_AFTER); if ((error = ipsec_run_hhooks(&ctx, HHOOK_TYPE_IPSEC_IN)) != 0) goto bad; if (skip == 0) { /* * We stripped outer IPv6 header. * Now we should requeue decrypted packet via netisr. */ switch (prot) { #ifdef INET case IPPROTO_IPIP: isr_prot = NETISR_IP; break; #endif case IPPROTO_IPV6: isr_prot = NETISR_IPV6; break; default: DPRINTF(("%s: cannot handle inner ip proto %d\n", __func__, prot)); IPSEC_ISTAT(sproto, nopf); error = EPFNOSUPPORT; goto bad; } error = netisr_queue_src(isr_prot, (uintptr_t)sav->spi, m); if (error) { IPSEC_ISTAT(sproto, qfull); DPRINTF(("%s: queue full; proto %u packet dropped\n", __func__, sproto)); } return (error); } /* * See the end of ip6_input for this logic. * IPPROTO_IPV[46] case will be processed just like other ones */ nest = 0; nxt = nxt8; while (nxt != IPPROTO_DONE) { if (V_ip6_hdrnestlimit && (++nest > V_ip6_hdrnestlimit)) { IP6STAT_INC(ip6s_toomanyhdr); error = EINVAL; goto bad; } /* * Protection against faulty packet - there should be * more sanity checks in header chain processing. */ if (m->m_pkthdr.len < skip) { IP6STAT_INC(ip6s_tooshort); in6_ifstat_inc(m->m_pkthdr.rcvif, ifs6_in_truncated); error = EINVAL; goto bad; } /* * Enforce IPsec policy checking if we are seeing last header. * note that we do not visit this with protocols with pcb layer * code - like udp/tcp/raw ip. */ if ((inet6sw[ip6_protox[nxt]].pr_flags & PR_LASTHDR) != 0 && ipsec6_in_reject(m, NULL)) { error = EINVAL; goto bad; } nxt = (*inet6sw[ip6_protox[nxt]].pr_input)(&m, &skip, nxt); } return 0; bad: if (m) m_freem(m); return error; } void esp6_ctlinput(int cmd, struct sockaddr *sa, void *d) { struct ip6ctlparam *ip6cp = NULL; struct mbuf *m = NULL; struct ip6_hdr *ip6; int off; if (sa->sa_family != AF_INET6 || sa->sa_len != sizeof(struct sockaddr_in6)) return; if ((unsigned)cmd >= PRC_NCMDS) return; /* if the parameter is from icmp6, decode it. */ if (d != NULL) { ip6cp = (struct ip6ctlparam *)d; m = ip6cp->ip6c_m; ip6 = ip6cp->ip6c_ip6; off = ip6cp->ip6c_off; } else { m = NULL; ip6 = NULL; off = 0; /* calm gcc */ } if (ip6 != NULL) { struct ip6ctlparam ip6cp1; /* * Notify the error to all possible sockets via pfctlinput2. * Since the upper layer information (such as protocol type, * source and destination ports) is embedded in the encrypted * data and might have been cut, we can't directly call * an upper layer ctlinput function. However, the pcbnotify * function will consider source and destination addresses * as well as the flow info value, and may be able to find * some PCB that should be notified. * Although pfctlinput2 will call esp6_ctlinput(), there is * no possibility of an infinite loop of function calls, * because we don't pass the inner IPv6 header. */ bzero(&ip6cp1, sizeof(ip6cp1)); ip6cp1.ip6c_src = ip6cp->ip6c_src; pfctlinput2(cmd, sa, (void *)&ip6cp1); /* * Then go to special cases that need ESP header information. * XXX: We assume that when ip6 is non NULL, * M and OFF are valid. */ if (cmd == PRC_MSGSIZE) { struct secasvar *sav; u_int32_t spi; int valid; /* check header length before using m_copydata */ if (m->m_pkthdr.len < off + sizeof (struct esp)) return; m_copydata(m, off + offsetof(struct esp, esp_spi), sizeof(u_int32_t), (caddr_t) &spi); /* * Check to see if we have a valid SA corresponding to * the address in the ICMP message payload. */ sav = KEY_ALLOCSA((union sockaddr_union *)sa, IPPROTO_ESP, spi); valid = (sav != NULL); if (sav) KEY_FREESAV(&sav); /* XXX Further validation? */ /* * Depending on whether the SA is "valid" and * routing table size (mtudisc_{hi,lo}wat), we will: * - recalcurate the new MTU and create the * corresponding routing entry, or * - ignore the MTU change notification. */ icmp6_mtudisc_update(ip6cp, valid); } } else { /* we normally notify any pcb here */ } } #endif /* INET6 */ Index: projects/netbsd-tests-update-12/sys/powerpc/mpc85xx/platform_mpc85xx.c =================================================================== --- projects/netbsd-tests-update-12/sys/powerpc/mpc85xx/platform_mpc85xx.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/powerpc/mpc85xx/platform_mpc85xx.c (revision 305172) @@ -1,569 +1,556 @@ /*- * Copyright (c) 2008-2012 Semihalf. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "opt_platform.h" #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "platform_if.h" #ifdef SMP extern void *ap_pcpu; extern vm_paddr_t kernload; /* Kernel physical load address */ extern uint8_t __boot_page[]; /* Boot page body */ extern uint32_t bp_kernload; struct cpu_release { uint32_t entry_h; uint32_t entry_l; uint32_t r3_h; uint32_t r3_l; uint32_t reserved; uint32_t pir; }; #endif extern uint32_t *bootinfo; vm_offset_t ccsrbar_va; static int cpu, maxcpu; static int mpc85xx_probe(platform_t); static void mpc85xx_mem_regions(platform_t, struct mem_region *phys, int *physsz, struct mem_region *avail, int *availsz); static u_long mpc85xx_timebase_freq(platform_t, struct cpuref *cpuref); static int mpc85xx_smp_first_cpu(platform_t, struct cpuref *cpuref); static int mpc85xx_smp_next_cpu(platform_t, struct cpuref *cpuref); static int mpc85xx_smp_get_bsp(platform_t, struct cpuref *cpuref); static int mpc85xx_smp_start_cpu(platform_t, struct pcpu *cpu); static void mpc85xx_idle(platform_t, int cpu); static int mpc85xx_idle_wakeup(platform_t plat, int cpu); static void mpc85xx_reset(platform_t); static platform_method_t mpc85xx_methods[] = { PLATFORMMETHOD(platform_probe, mpc85xx_probe), PLATFORMMETHOD(platform_attach, mpc85xx_attach), PLATFORMMETHOD(platform_mem_regions, mpc85xx_mem_regions), PLATFORMMETHOD(platform_timebase_freq, mpc85xx_timebase_freq), PLATFORMMETHOD(platform_smp_first_cpu, mpc85xx_smp_first_cpu), PLATFORMMETHOD(platform_smp_next_cpu, mpc85xx_smp_next_cpu), PLATFORMMETHOD(platform_smp_get_bsp, mpc85xx_smp_get_bsp), PLATFORMMETHOD(platform_smp_start_cpu, mpc85xx_smp_start_cpu), PLATFORMMETHOD(platform_reset, mpc85xx_reset), PLATFORMMETHOD(platform_idle, mpc85xx_idle), PLATFORMMETHOD(platform_idle_wakeup, mpc85xx_idle_wakeup), PLATFORMMETHOD_END }; DEFINE_CLASS_0(mpc85xx, mpc85xx_platform, mpc85xx_methods, 0); PLATFORM_DEF(mpc85xx_platform); static int mpc85xx_probe(platform_t plat) { - u_int pvr = mfpvr() >> 16; + u_int pvr = (mfpvr() >> 16) & 0xFFFF; - if ((pvr & 0xfff0) == FSL_E500v1) - return (BUS_PROBE_DEFAULT); - + switch (pvr) { + case FSL_E500v1: + case FSL_E500v2: + case FSL_E500mc: + case FSL_E5500: + case FSL_E6500: + return (BUS_PROBE_DEFAULT); + } return (ENXIO); } int mpc85xx_attach(platform_t plat) { phandle_t cpus, child, ccsr; const char *soc_name_guesses[] = {"/soc", "soc", NULL}; const char **name; pcell_t ranges[6], acells, pacells, scells; - uint32_t sr; uint64_t ccsrbar, ccsrsize; - int i, law_max, tgt; + int i; if ((cpus = OF_finddevice("/cpus")) != -1) { for (maxcpu = 0, child = OF_child(cpus); child != 0; child = OF_peer(child), maxcpu++) ; } else maxcpu = 1; /* * Locate CCSR region. Irritatingly, there is no way to find it * unless you already know where it is. Try to infer its location * from the device tree. */ ccsr = -1; for (name = soc_name_guesses; *name != NULL && ccsr == -1; name++) ccsr = OF_finddevice(*name); if (ccsr == -1) { char type[64]; /* That didn't work. Search for devices of type "soc" */ child = OF_child(OF_peer(0)); for (OF_child(child); child != 0; child = OF_peer(child)) { if (OF_getprop(child, "device_type", type, sizeof(type)) <= 0) continue; if (strcmp(type, "soc") == 0) { ccsr = child; break; } } } if (ccsr == -1) panic("Could not locate CCSR window!"); OF_getprop(ccsr, "#size-cells", &scells, sizeof(scells)); OF_getprop(ccsr, "#address-cells", &acells, sizeof(acells)); OF_searchprop(OF_parent(ccsr), "#address-cells", &pacells, sizeof(pacells)); OF_getprop(ccsr, "ranges", ranges, sizeof(ranges)); ccsrbar = ccsrsize = 0; for (i = acells; i < acells + pacells; i++) { ccsrbar <<= 32; ccsrbar |= ranges[i]; } for (i = acells + pacells; i < acells + pacells + scells; i++) { ccsrsize <<= 32; ccsrsize |= ranges[i]; } ccsrbar_va = pmap_early_io_map(ccsrbar, ccsrsize); mpc85xx_fix_errata(ccsrbar_va); mpc85xx_enable_l3_cache(); - - /* - * Clear local access windows. Skip DRAM entries, so we don't shoot - * ourselves in the foot. - */ - law_max = law_getmax(); - for (i = 0; i < law_max; i++) { - sr = ccsr_read4(OCP85XX_LAWSR(i)); - if ((sr & OCP85XX_ENA_MASK) == 0) - continue; - tgt = (sr & 0x01f00000) >> 20; - if (tgt == OCP85XX_TGTIF_RAM1 || tgt == OCP85XX_TGTIF_RAM2 || - tgt == OCP85XX_TGTIF_RAM_INTL) - continue; - - ccsr_write4(OCP85XX_LAWSR(i), sr & OCP85XX_DIS_MASK); - } return (0); } void mpc85xx_mem_regions(platform_t plat, struct mem_region *phys, int *physsz, struct mem_region *avail, int *availsz) { ofw_mem_regions(phys, physsz, avail, availsz); } static u_long mpc85xx_timebase_freq(platform_t plat, struct cpuref *cpuref) { u_long ticks; phandle_t cpus, child; pcell_t freq; if (bootinfo != NULL) { if (bootinfo[0] == 1) { /* Backward compatibility. See 8-STABLE. */ ticks = bootinfo[3] >> 3; } else { /* Compatibility with Juniper's loader. */ ticks = bootinfo[5] >> 3; } } else ticks = 0; if ((cpus = OF_finddevice("/cpus")) == -1) goto out; if ((child = OF_child(cpus)) == 0) goto out; switch (OF_getproplen(child, "timebase-frequency")) { case 4: { uint32_t tbase; OF_getprop(child, "timebase-frequency", &tbase, sizeof(tbase)); ticks = tbase; return (ticks); } case 8: { uint64_t tbase; OF_getprop(child, "timebase-frequency", &tbase, sizeof(tbase)); ticks = tbase; return (ticks); } default: break; } freq = 0; if (OF_getprop(child, "bus-frequency", (void *)&freq, sizeof(freq)) <= 0) goto out; if (freq == 0) goto out; /* * Time Base and Decrementer are updated every 8 CCB bus clocks. * HID0[SEL_TBCLK] = 0 */ if (mpc85xx_is_qoriq()) ticks = freq / 32; else ticks = freq / 8; out: if (ticks <= 0) panic("Unable to determine timebase frequency!"); return (ticks); } static int mpc85xx_smp_first_cpu(platform_t plat, struct cpuref *cpuref) { cpu = 0; cpuref->cr_cpuid = cpu; cpuref->cr_hwref = cpuref->cr_cpuid; if (bootverbose) printf("powerpc_smp_first_cpu: cpuid %d\n", cpuref->cr_cpuid); cpu++; return (0); } static int mpc85xx_smp_next_cpu(platform_t plat, struct cpuref *cpuref) { if (cpu >= maxcpu) return (ENOENT); cpuref->cr_cpuid = cpu++; cpuref->cr_hwref = cpuref->cr_cpuid; if (bootverbose) printf("powerpc_smp_next_cpu: cpuid %d\n", cpuref->cr_cpuid); return (0); } static int mpc85xx_smp_get_bsp(platform_t plat, struct cpuref *cpuref) { cpuref->cr_cpuid = mfspr(SPR_PIR); cpuref->cr_hwref = cpuref->cr_cpuid; return (0); } #ifdef SMP static int mpc85xx_smp_start_cpu_epapr(platform_t plat, struct pcpu *pc) { vm_paddr_t rel_pa, bptr; volatile struct cpu_release *rel; vm_offset_t rel_va, rel_page; phandle_t node; int i; /* If we're calling this, the node already exists. */ node = OF_finddevice("/cpus"); for (i = 0, node = OF_child(node); i < pc->pc_cpuid; i++, node = OF_peer(node)) ; if (OF_getencprop(node, "cpu-release-addr", (pcell_t *)&rel_pa, sizeof(rel_pa)) == -1) { return (ENOENT); } rel_page = kva_alloc(PAGE_SIZE); if (rel_page == 0) return (ENOMEM); critical_enter(); rel_va = rel_page + (rel_pa & PAGE_MASK); pmap_kenter(rel_page, rel_pa & ~PAGE_MASK); rel = (struct cpu_release *)rel_va; bptr = ((vm_paddr_t)(uintptr_t)__boot_page - KERNBASE) + kernload; cpu_flush_dcache(__DEVOLATILE(struct cpu_release *,rel), sizeof(*rel)); rel->pir = pc->pc_cpuid; __asm __volatile("sync"); rel->entry_h = (bptr >> 32); rel->entry_l = bptr; __asm __volatile("sync"); cpu_flush_dcache(__DEVOLATILE(struct cpu_release *,rel), sizeof(*rel)); if (bootverbose) printf("Waking up CPU %d via CPU release page %p\n", pc->pc_cpuid, rel); critical_exit(); pmap_kremove(rel_page); kva_free(rel_page, PAGE_SIZE); return (0); } #endif static int mpc85xx_smp_start_cpu(platform_t plat, struct pcpu *pc) { #ifdef SMP vm_paddr_t bptr; uint32_t reg; int timeout; uintptr_t brr; int cpuid; int epapr_boot = 0; uint32_t tgt; if (mpc85xx_is_qoriq()) { reg = ccsr_read4(OCP85XX_COREDISR); cpuid = pc->pc_cpuid; if ((reg & (1 << cpuid)) != 0) { printf("%s: CPU %d is disabled!\n", __func__, pc->pc_cpuid); return (-1); } brr = OCP85XX_BRR; } else { brr = OCP85XX_EEBPCR; cpuid = pc->pc_cpuid + 24; } bp_kernload = kernload; /* * bp_kernload is in the boot page. Sync the cache because ePAPR * booting has the other core(s) already running. */ cpu_flush_dcache(&bp_kernload, sizeof(bp_kernload)); ap_pcpu = pc; __asm __volatile("msync; isync"); /* First try the ePAPR way. */ if (mpc85xx_smp_start_cpu_epapr(plat, pc) == 0) { epapr_boot = 1; goto spin_wait; } reg = ccsr_read4(brr); if ((reg & (1 << cpuid)) != 0) { printf("SMP: CPU %d already out of hold-off state!\n", pc->pc_cpuid); return (ENXIO); } /* Flush caches to have our changes hit DRAM. */ cpu_flush_dcache(__boot_page, 4096); bptr = ((vm_paddr_t)(uintptr_t)__boot_page - KERNBASE) + kernload; KASSERT((bptr & 0xfff) == 0, ("%s: boot page is not aligned (%#jx)", __func__, (uintmax_t)bptr)); if (mpc85xx_is_qoriq()) { /* * Read DDR controller configuration to select proper BPTR target ID. * * On P5020 bit 29 of DDR1_CS0_CONFIG enables DDR controllers * interleaving. If this bit is set, we have to use * OCP85XX_TGTIF_RAM_INTL as BPTR target ID. On other QorIQ DPAA SoCs, * this bit is reserved and always 0. */ reg = ccsr_read4(OCP85XX_DDR1_CS0_CONFIG); if (reg & (1 << 29)) tgt = OCP85XX_TGTIF_RAM_INTL; else tgt = OCP85XX_TGTIF_RAM1; /* * Set BSTR to the physical address of the boot page */ ccsr_write4(OCP85XX_BSTRH, bptr >> 32); ccsr_write4(OCP85XX_BSTRL, bptr); ccsr_write4(OCP85XX_BSTAR, OCP85XX_ENA_MASK | (tgt << OCP85XX_TRGT_SHIFT_QORIQ) | (ffsl(PAGE_SIZE) - 2)); /* Read back OCP85XX_BSTAR to synchronize write */ ccsr_read4(OCP85XX_BSTAR); /* * Enable and configure time base on new CPU. */ /* Set TB clock source to platform clock / 32 */ reg = ccsr_read4(CCSR_CTBCKSELR); ccsr_write4(CCSR_CTBCKSELR, reg & ~(1 << pc->pc_cpuid)); /* Enable TB */ reg = ccsr_read4(CCSR_CTBENR); ccsr_write4(CCSR_CTBENR, reg | (1 << pc->pc_cpuid)); } else { /* * Set BPTR to the physical address of the boot page */ bptr = (bptr >> 12) | 0x80000000u; ccsr_write4(OCP85XX_BPTR, bptr); __asm __volatile("isync; msync"); } /* * Release AP from hold-off state */ reg = ccsr_read4(brr); ccsr_write4(brr, reg | (1 << cpuid)); __asm __volatile("isync; msync"); spin_wait: timeout = 500; while (!pc->pc_awake && timeout--) DELAY(1000); /* wait 1ms */ /* * Disable boot page translation so that the 4K page at the default * address (= 0xfffff000) isn't permanently remapped and thus not * usable otherwise. */ if (!epapr_boot) { if (mpc85xx_is_qoriq()) ccsr_write4(OCP85XX_BSTAR, 0); else ccsr_write4(OCP85XX_BPTR, 0); __asm __volatile("isync; msync"); } if (!pc->pc_awake) panic("SMP: CPU %d didn't wake up.\n", pc->pc_cpuid); return ((pc->pc_awake) ? 0 : EBUSY); #else /* No SMP support */ return (ENXIO); #endif } static void mpc85xx_reset(platform_t plat) { /* * Try the dedicated reset register first. * If the SoC doesn't have one, we'll fall * back to using the debug control register. */ ccsr_write4(OCP85XX_RSTCR, 2); /* Clear DBCR0, disables debug interrupts and events. */ mtspr(SPR_DBCR0, 0); __asm __volatile("isync"); /* Enable Debug Interrupts in MSR. */ mtmsr(mfmsr() | PSL_DE); /* Enable debug interrupts and issue reset. */ mtspr(SPR_DBCR0, mfspr(SPR_DBCR0) | DBCR0_IDM | DBCR0_RST_SYSTEM); printf("Reset failed...\n"); while (1) ; } static void mpc85xx_idle(platform_t plat, int cpu) { uint32_t reg; if (mpc85xx_is_qoriq()) { reg = ccsr_read4(OCP85XX_RCPM_CDOZCR); ccsr_write4(OCP85XX_RCPM_CDOZCR, reg | (1 << cpu)); ccsr_read4(OCP85XX_RCPM_CDOZCR); } else { reg = mfmsr(); /* Freescale E500 core RM section 6.4.1. */ __asm __volatile("msync; mtmsr %0; isync" :: "r" (reg | PSL_WE)); } } static int mpc85xx_idle_wakeup(platform_t plat, int cpu) { uint32_t reg; if (mpc85xx_is_qoriq()) { reg = ccsr_read4(OCP85XX_RCPM_CDOZCR); ccsr_write4(OCP85XX_RCPM_CDOZCR, reg & ~(1 << cpu)); ccsr_read4(OCP85XX_RCPM_CDOZCR); return (1); } return (0); } Index: projects/netbsd-tests-update-12/sys/sys/filedesc.h =================================================================== --- projects/netbsd-tests-update-12/sys/sys/filedesc.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/sys/filedesc.h (revision 305172) @@ -1,224 +1,236 @@ /*- * Copyright (c) 1990, 1993 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)filedesc.h 8.1 (Berkeley) 6/2/93 * $FreeBSD$ */ #ifndef _SYS_FILEDESC_H_ #define _SYS_FILEDESC_H_ #include #include #include #include #include #include #include #include struct filecaps { cap_rights_t fc_rights; /* per-descriptor capability rights */ u_long *fc_ioctls; /* per-descriptor allowed ioctls */ int16_t fc_nioctls; /* fc_ioctls array size */ uint32_t fc_fcntls; /* per-descriptor allowed fcntls */ }; struct filedescent { struct file *fde_file; /* file structure for open file */ struct filecaps fde_caps; /* per-descriptor rights */ uint8_t fde_flags; /* per-process open file flags */ seq_t fde_seq; /* keep file and caps in sync */ }; #define fde_rights fde_caps.fc_rights #define fde_fcntls fde_caps.fc_fcntls #define fde_ioctls fde_caps.fc_ioctls #define fde_nioctls fde_caps.fc_nioctls #define fde_change_size (offsetof(struct filedescent, fde_seq)) struct fdescenttbl { int fdt_nfiles; /* number of open files allocated */ struct filedescent fdt_ofiles[0]; /* open files */ }; #define fd_seq(fdt, fd) (&(fdt)->fdt_ofiles[(fd)].fde_seq) /* * This structure is used for the management of descriptors. It may be * shared by multiple processes. */ #define NDSLOTTYPE u_long struct filedesc { struct fdescenttbl *fd_files; /* open files table */ struct vnode *fd_cdir; /* current directory */ struct vnode *fd_rdir; /* root directory */ struct vnode *fd_jdir; /* jail root directory */ NDSLOTTYPE *fd_map; /* bitmap of free fds */ int fd_lastfile; /* high-water mark of fd_ofiles */ int fd_freefile; /* approx. next free file */ u_short fd_cmask; /* mask for file creation */ int fd_refcnt; /* thread reference count */ int fd_holdcnt; /* hold count on structure + mutex */ struct sx fd_sx; /* protects members of this struct */ struct kqlist fd_kqlist; /* list of kqueues on this filedesc */ int fd_holdleaderscount; /* block fdfree() for shared close() */ int fd_holdleaderswakeup; /* fdfree() needs wakeup */ }; /* * Structure to keep track of (process leader, struct fildedesc) tuples. * Each process has a pointer to such a structure when detailed tracking * is needed, e.g., when rfork(RFPROC | RFMEM) causes a file descriptor * table to be shared by processes having different "p_leader" pointers * and thus distinct POSIX style locks. * * fdl_refcount and fdl_holdcount are protected by struct filedesc mtx. */ struct filedesc_to_leader { int fdl_refcount; /* references from struct proc */ int fdl_holdcount; /* temporary hold during closef */ int fdl_wakeup; /* fdfree() waits on closef() */ struct proc *fdl_leader; /* owner of POSIX locks */ /* Circular list: */ struct filedesc_to_leader *fdl_prev; struct filedesc_to_leader *fdl_next; }; #define fd_nfiles fd_files->fdt_nfiles #define fd_ofiles fd_files->fdt_ofiles /* * Per-process open flags. */ #define UF_EXCLOSE 0x01 /* auto-close on exec */ #ifdef _KERNEL /* Lock a file descriptor table. */ #define FILEDESC_LOCK_INIT(fdp) sx_init(&(fdp)->fd_sx, "filedesc structure") #define FILEDESC_LOCK_DESTROY(fdp) sx_destroy(&(fdp)->fd_sx) #define FILEDESC_LOCK(fdp) (&(fdp)->fd_sx) #define FILEDESC_XLOCK(fdp) sx_xlock(&(fdp)->fd_sx) #define FILEDESC_XUNLOCK(fdp) sx_xunlock(&(fdp)->fd_sx) #define FILEDESC_SLOCK(fdp) sx_slock(&(fdp)->fd_sx) #define FILEDESC_SUNLOCK(fdp) sx_sunlock(&(fdp)->fd_sx) #define FILEDESC_LOCK_ASSERT(fdp) sx_assert(&(fdp)->fd_sx, SX_LOCKED | \ SX_NOTRECURSED) #define FILEDESC_XLOCK_ASSERT(fdp) sx_assert(&(fdp)->fd_sx, SX_XLOCKED | \ SX_NOTRECURSED) #define FILEDESC_UNLOCK_ASSERT(fdp) sx_assert(&(fdp)->fd_sx, SX_UNLOCKED) /* Operation types for kern_dup(). */ enum { FDDUP_NORMAL, /* dup() behavior. */ FDDUP_FCNTL, /* fcntl()-style errors. */ FDDUP_FIXED, /* Force fixed allocation. */ FDDUP_MUSTREPLACE, /* Target must exist. */ FDDUP_LASTMODE, }; /* Flags for kern_dup(). */ #define FDDUP_FLAG_CLOEXEC 0x1 /* Atomically set UF_EXCLOSE. */ /* For backward compatibility. */ #define falloc(td, resultfp, resultfd, flags) \ falloc_caps(td, resultfp, resultfd, flags, NULL) struct thread; void filecaps_init(struct filecaps *fcaps); int filecaps_copy(const struct filecaps *src, struct filecaps *dst, bool locked); void filecaps_move(struct filecaps *src, struct filecaps *dst); void filecaps_free(struct filecaps *fcaps); int closef(struct file *fp, struct thread *td); int dupfdopen(struct thread *td, struct filedesc *fdp, int dfd, int mode, int openerror, int *indxp); int falloc_caps(struct thread *td, struct file **resultfp, int *resultfd, int flags, struct filecaps *fcaps); int falloc_noinstall(struct thread *td, struct file **resultfp); void _finstall(struct filedesc *fdp, struct file *fp, int fd, int flags, struct filecaps *fcaps); int finstall(struct thread *td, struct file *fp, int *resultfd, int flags, struct filecaps *fcaps); int fdalloc(struct thread *td, int minfd, int *result); int fdallocn(struct thread *td, int minfd, int *fds, int n); int fdcheckstd(struct thread *td); void fdclose(struct thread *td, struct file *fp, int idx); void fdcloseexec(struct thread *td); void fdsetugidsafety(struct thread *td); struct filedesc *fdcopy(struct filedesc *fdp); int fdcopy_remapped(struct filedesc *fdp, const int *fds, size_t nfds, struct filedesc **newfdp); void fdinstall_remapped(struct thread *td, struct filedesc *fdp); void fdunshare(struct thread *td); void fdescfree(struct thread *td); void fdescfree_remapped(struct filedesc *fdp); struct filedesc *fdinit(struct filedesc *fdp, bool prepfiles); struct filedesc *fdshare(struct filedesc *fdp); struct filedesc_to_leader * filedesc_to_leader_alloc(struct filedesc_to_leader *old, struct filedesc *fdp, struct proc *leader); int getvnode(struct thread *td, int fd, cap_rights_t *rightsp, struct file **fpp); void mountcheckdirs(struct vnode *olddp, struct vnode *newdp); /* Return a referenced file from an unlocked descriptor. */ int fget_unlocked(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, struct file **fpp, seq_t *seqp); /* Requires a FILEDESC_{S,X}LOCK held and returns without a ref. */ static __inline struct file * fget_locked(struct filedesc *fdp, int fd) { FILEDESC_LOCK_ASSERT(fdp); if (fd < 0 || fd > fdp->fd_lastfile) return (NULL); return (fdp->fd_ofiles[fd].fde_file); } +static __inline struct filedescent * +fdeget_locked(struct filedesc *fdp, int fd) +{ + + FILEDESC_LOCK_ASSERT(fdp); + + if (fd < 0 || fd > fdp->fd_lastfile) + return (NULL); + + return (&fdp->fd_ofiles[fd]); +} + static __inline bool fd_modified(struct filedesc *fdp, int fd, seq_t seq) { return (!seq_consistent(fd_seq(fdp->fd_files, fd), seq)); } /* cdir/rdir/jdir manipulation functions. */ void pwd_chdir(struct thread *td, struct vnode *vp); int pwd_chroot(struct thread *td, struct vnode *vp); void pwd_ensure_dirs(void); #endif /* _KERNEL */ #endif /* !_SYS_FILEDESC_H_ */ Index: projects/netbsd-tests-update-12/sys/sys/mbuf.h =================================================================== --- projects/netbsd-tests-update-12/sys/sys/mbuf.h (revision 305171) +++ projects/netbsd-tests-update-12/sys/sys/mbuf.h (revision 305172) @@ -1,1316 +1,1316 @@ /*- * Copyright (c) 1982, 1986, 1988, 1993 * The Regents of the University of California. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)mbuf.h 8.5 (Berkeley) 2/19/95 * $FreeBSD$ */ #ifndef _SYS_MBUF_H_ #define _SYS_MBUF_H_ /* XXX: These includes suck. Sorry! */ #include #ifdef _KERNEL #include #include #ifdef WITNESS #include #endif #endif #ifdef _KERNEL #include #define MBUF_PROBE1(probe, arg0) \ SDT_PROBE1(sdt, , , probe, arg0) #define MBUF_PROBE2(probe, arg0, arg1) \ SDT_PROBE2(sdt, , , probe, arg0, arg1) #define MBUF_PROBE3(probe, arg0, arg1, arg2) \ SDT_PROBE3(sdt, , , probe, arg0, arg1, arg2) #define MBUF_PROBE4(probe, arg0, arg1, arg2, arg3) \ SDT_PROBE4(sdt, , , probe, arg0, arg1, arg2, arg3) #define MBUF_PROBE5(probe, arg0, arg1, arg2, arg3, arg4) \ SDT_PROBE5(sdt, , , probe, arg0, arg1, arg2, arg3, arg4) SDT_PROBE_DECLARE(sdt, , , m__init); SDT_PROBE_DECLARE(sdt, , , m__gethdr); SDT_PROBE_DECLARE(sdt, , , m__get); SDT_PROBE_DECLARE(sdt, , , m__getcl); SDT_PROBE_DECLARE(sdt, , , m__clget); SDT_PROBE_DECLARE(sdt, , , m__cljget); SDT_PROBE_DECLARE(sdt, , , m__cljset); SDT_PROBE_DECLARE(sdt, , , m__free); SDT_PROBE_DECLARE(sdt, , , m__freem); #endif /* _KERNEL */ /* * Mbufs are of a single size, MSIZE (sys/param.h), which includes overhead. * An mbuf may add a single "mbuf cluster" of size MCLBYTES (also in * sys/param.h), which has no additional overhead and is used instead of the * internal data area; this is done when at least MINCLSIZE of data must be * stored. Additionally, it is possible to allocate a separate buffer * externally and attach it to the mbuf in a way similar to that of mbuf * clusters. * * NB: These calculation do not take actual compiler-induced alignment and * padding inside the complete struct mbuf into account. Appropriate * attention is required when changing members of struct mbuf. * * MLEN is data length in a normal mbuf. * MHLEN is data length in an mbuf with pktheader. * MINCLSIZE is a smallest amount of data that should be put into cluster. * * Compile-time assertions in uipc_mbuf.c test these values to ensure that * they are sensible. */ struct mbuf; #define MHSIZE offsetof(struct mbuf, m_dat) #define MPKTHSIZE offsetof(struct mbuf, m_pktdat) #define MLEN ((int)(MSIZE - MHSIZE)) #define MHLEN ((int)(MSIZE - MPKTHSIZE)) #define MINCLSIZE (MHLEN + 1) #ifdef _KERNEL /*- * Macro for type conversion: convert mbuf pointer to data pointer of correct * type: * * mtod(m, t) -- Convert mbuf pointer to data pointer of correct type. * mtodo(m, o) -- Same as above but with offset 'o' into data. */ #define mtod(m, t) ((t)((m)->m_data)) #define mtodo(m, o) ((void *)(((m)->m_data) + (o))) /* * Argument structure passed to UMA routines during mbuf and packet * allocations. */ struct mb_args { int flags; /* Flags for mbuf being allocated */ short type; /* Type of mbuf being allocated */ }; #endif /* _KERNEL */ /* * Packet tag structure (see below for details). */ struct m_tag { SLIST_ENTRY(m_tag) m_tag_link; /* List of packet tags */ u_int16_t m_tag_id; /* Tag ID */ u_int16_t m_tag_len; /* Length of data */ u_int32_t m_tag_cookie; /* ABI/Module ID */ void (*m_tag_free)(struct m_tag *); }; /* * Record/packet header in first mbuf of chain; valid only if M_PKTHDR is set. * Size ILP32: 48 * LP64: 56 * Compile-time assertions in uipc_mbuf.c test these values to ensure that * they are correct. */ struct pkthdr { struct ifnet *rcvif; /* rcv interface */ SLIST_HEAD(packet_tags, m_tag) tags; /* list of packet tags */ int32_t len; /* total packet length */ /* Layer crossing persistent information. */ uint32_t flowid; /* packet's 4-tuple system */ uint64_t csum_flags; /* checksum and offload features */ uint16_t fibnum; /* this packet should use this fib */ uint8_t cosqos; /* class/quality of service */ uint8_t rsstype; /* hash type */ uint8_t l2hlen; /* layer 2 header length */ uint8_t l3hlen; /* layer 3 header length */ uint8_t l4hlen; /* layer 4 header length */ uint8_t l5hlen; /* layer 5 header length */ union { uint8_t eight[8]; uint16_t sixteen[4]; uint32_t thirtytwo[2]; uint64_t sixtyfour[1]; uintptr_t unintptr[1]; void *ptr; } PH_per; /* Layer specific non-persistent local storage for reassembly, etc. */ union { uint8_t eight[8]; uint16_t sixteen[4]; uint32_t thirtytwo[2]; uint64_t sixtyfour[1]; uintptr_t unintptr[1]; void *ptr; } PH_loc; }; #define ether_vtag PH_per.sixteen[0] #define PH_vt PH_per #define vt_nrecs sixteen[0] #define tso_segsz PH_per.sixteen[1] #define lro_nsegs tso_segsz #define csum_phsum PH_per.sixteen[2] #define csum_data PH_per.thirtytwo[1] /* * Description of external storage mapped into mbuf; valid only if M_EXT is * set. * Size ILP32: 28 * LP64: 48 * Compile-time assertions in uipc_mbuf.c test these values to ensure that * they are correct. */ struct m_ext { union { volatile u_int ext_count; /* value of ref count info */ volatile u_int *ext_cnt; /* pointer to ref count info */ }; caddr_t ext_buf; /* start of buffer */ uint32_t ext_size; /* size of buffer, for ext_free */ uint32_t ext_type:8, /* type of external storage */ ext_flags:24; /* external storage mbuf flags */ void (*ext_free) /* free routine if not the usual */ (struct mbuf *, void *, void *); void *ext_arg1; /* optional argument pointer */ void *ext_arg2; /* optional argument pointer */ }; /* * The core of the mbuf object along with some shortcut defines for practical * purposes. */ struct mbuf { /* * Header present at the beginning of every mbuf. * Size ILP32: 24 * LP64: 32 * Compile-time assertions in uipc_mbuf.c test these values to ensure * that they are correct. */ union { /* next buffer in chain */ struct mbuf *m_next; SLIST_ENTRY(mbuf) m_slist; STAILQ_ENTRY(mbuf) m_stailq; }; union { /* next chain in queue/record */ struct mbuf *m_nextpkt; SLIST_ENTRY(mbuf) m_slistpkt; STAILQ_ENTRY(mbuf) m_stailqpkt; }; caddr_t m_data; /* location of data */ int32_t m_len; /* amount of data in this mbuf */ uint32_t m_type:8, /* type of data in this mbuf */ m_flags:24; /* flags; see below */ #if !defined(__LP64__) uint32_t m_pad; /* pad for 64bit alignment */ #endif /* * A set of optional headers (packet header, external storage header) * and internal data storage. Historically, these arrays were sized * to MHLEN (space left after a packet header) and MLEN (space left * after only a regular mbuf header); they are now variable size in * order to support future work on variable-size mbufs. */ union { struct { struct pkthdr m_pkthdr; /* M_PKTHDR set */ union { struct m_ext m_ext; /* M_EXT set */ char m_pktdat[0]; }; }; char m_dat[0]; /* !M_PKTHDR, !M_EXT */ }; }; /* * mbuf flags of global significance and layer crossing. * Those of only protocol/layer specific significance are to be mapped * to M_PROTO[1-12] and cleared at layer handoff boundaries. * NB: Limited to the lower 24 bits. */ #define M_EXT 0x00000001 /* has associated external storage */ #define M_PKTHDR 0x00000002 /* start of record */ #define M_EOR 0x00000004 /* end of record */ #define M_RDONLY 0x00000008 /* associated data is marked read-only */ #define M_BCAST 0x00000010 /* send/received as link-level broadcast */ #define M_MCAST 0x00000020 /* send/received as link-level multicast */ #define M_PROMISC 0x00000040 /* packet was not for us */ #define M_VLANTAG 0x00000080 /* ether_vtag is valid */ #define M_UNUSED_8 0x00000100 /* --available-- */ #define M_NOFREE 0x00000200 /* do not free mbuf, embedded in cluster */ #define M_PROTO1 0x00001000 /* protocol-specific */ #define M_PROTO2 0x00002000 /* protocol-specific */ #define M_PROTO3 0x00004000 /* protocol-specific */ #define M_PROTO4 0x00008000 /* protocol-specific */ #define M_PROTO5 0x00010000 /* protocol-specific */ #define M_PROTO6 0x00020000 /* protocol-specific */ #define M_PROTO7 0x00040000 /* protocol-specific */ #define M_PROTO8 0x00080000 /* protocol-specific */ #define M_PROTO9 0x00100000 /* protocol-specific */ #define M_PROTO10 0x00200000 /* protocol-specific */ #define M_PROTO11 0x00400000 /* protocol-specific */ #define M_PROTO12 0x00800000 /* protocol-specific */ #define MB_DTOR_SKIP 0x1 /* don't pollute the cache by touching a freed mbuf */ /* * Flags to purge when crossing layers. */ #define M_PROTOFLAGS \ (M_PROTO1|M_PROTO2|M_PROTO3|M_PROTO4|M_PROTO5|M_PROTO6|M_PROTO7|M_PROTO8|\ M_PROTO9|M_PROTO10|M_PROTO11|M_PROTO12) /* * Flags preserved when copying m_pkthdr. */ #define M_COPYFLAGS \ (M_PKTHDR|M_EOR|M_RDONLY|M_BCAST|M_MCAST|M_PROMISC|M_VLANTAG| \ M_PROTOFLAGS) /* * Mbuf flag description for use with printf(9) %b identifier. */ #define M_FLAG_BITS \ "\20\1M_EXT\2M_PKTHDR\3M_EOR\4M_RDONLY\5M_BCAST\6M_MCAST" \ "\7M_PROMISC\10M_VLANTAG" #define M_FLAG_PROTOBITS \ "\15M_PROTO1\16M_PROTO2\17M_PROTO3\20M_PROTO4\21M_PROTO5" \ "\22M_PROTO6\23M_PROTO7\24M_PROTO8\25M_PROTO9\26M_PROTO10" \ "\27M_PROTO11\30M_PROTO12" #define M_FLAG_PRINTF (M_FLAG_BITS M_FLAG_PROTOBITS) /* * Network interface cards are able to hash protocol fields (such as IPv4 * addresses and TCP port numbers) classify packets into flows. These flows * can then be used to maintain ordering while delivering packets to the OS * via parallel input queues, as well as to provide a stateless affinity * model. NIC drivers can pass up the hash via m->m_pkthdr.flowid, and set * m_flag fields to indicate how the hash should be interpreted by the * network stack. * * Most NICs support RSS, which provides ordering and explicit affinity, and * use the hash m_flag bits to indicate what header fields were covered by * the hash. M_HASHTYPE_OPAQUE and M_HASHTYPE_OPAQUE_HASH can be set by non- * RSS cards or configurations that provide an opaque flow identifier, allowing * for ordering and distribution without explicit affinity. Additionally, * M_HASHTYPE_OPAQUE_HASH indicates that the flow identifier has hash * properties. */ #define M_HASHTYPE_HASHPROP 0x80 /* has hash properties */ #define M_HASHTYPE_HASH(t) (M_HASHTYPE_HASHPROP | (t)) /* Microsoft RSS standard hash types */ #define M_HASHTYPE_NONE 0 #define M_HASHTYPE_RSS_IPV4 M_HASHTYPE_HASH(1) /* IPv4 2-tuple */ #define M_HASHTYPE_RSS_TCP_IPV4 M_HASHTYPE_HASH(2) /* TCPv4 4-tuple */ #define M_HASHTYPE_RSS_IPV6 M_HASHTYPE_HASH(3) /* IPv6 2-tuple */ #define M_HASHTYPE_RSS_TCP_IPV6 M_HASHTYPE_HASH(4) /* TCPv6 4-tuple */ #define M_HASHTYPE_RSS_IPV6_EX M_HASHTYPE_HASH(5) /* IPv6 2-tuple + * ext hdrs */ #define M_HASHTYPE_RSS_TCP_IPV6_EX M_HASHTYPE_HASH(6) /* TCPv6 4-tiple + * ext hdrs */ /* Non-standard RSS hash types */ #define M_HASHTYPE_RSS_UDP_IPV4 M_HASHTYPE_HASH(7) /* IPv4 UDP 4-tuple*/ #define M_HASHTYPE_RSS_UDP_IPV4_EX M_HASHTYPE_HASH(8) /* IPv4 UDP 4-tuple + * ext hdrs */ #define M_HASHTYPE_RSS_UDP_IPV6 M_HASHTYPE_HASH(9) /* IPv6 UDP 4-tuple*/ #define M_HASHTYPE_RSS_UDP_IPV6_EX M_HASHTYPE_HASH(10)/* IPv6 UDP 4-tuple + * ext hdrs */ #define M_HASHTYPE_OPAQUE 63 /* ordering, not affinity */ #define M_HASHTYPE_OPAQUE_HASH M_HASHTYPE_HASH(M_HASHTYPE_OPAQUE) /* ordering+hash, not affinity*/ #define M_HASHTYPE_CLEAR(m) ((m)->m_pkthdr.rsstype = 0) #define M_HASHTYPE_GET(m) ((m)->m_pkthdr.rsstype) #define M_HASHTYPE_SET(m, v) ((m)->m_pkthdr.rsstype = (v)) #define M_HASHTYPE_TEST(m, v) (M_HASHTYPE_GET(m) == (v)) #define M_HASHTYPE_ISHASH(m) (M_HASHTYPE_GET(m) & M_HASHTYPE_HASHPROP) /* * COS/QOS class and quality of service tags. * It uses DSCP code points as base. */ #define QOS_DSCP_CS0 0x00 #define QOS_DSCP_DEF QOS_DSCP_CS0 #define QOS_DSCP_CS1 0x20 #define QOS_DSCP_AF11 0x28 #define QOS_DSCP_AF12 0x30 #define QOS_DSCP_AF13 0x38 #define QOS_DSCP_CS2 0x40 #define QOS_DSCP_AF21 0x48 #define QOS_DSCP_AF22 0x50 #define QOS_DSCP_AF23 0x58 #define QOS_DSCP_CS3 0x60 #define QOS_DSCP_AF31 0x68 #define QOS_DSCP_AF32 0x70 #define QOS_DSCP_AF33 0x78 #define QOS_DSCP_CS4 0x80 #define QOS_DSCP_AF41 0x88 #define QOS_DSCP_AF42 0x90 #define QOS_DSCP_AF43 0x98 #define QOS_DSCP_CS5 0xa0 #define QOS_DSCP_EF 0xb8 #define QOS_DSCP_CS6 0xc0 #define QOS_DSCP_CS7 0xe0 /* * External mbuf storage buffer types. */ #define EXT_CLUSTER 1 /* mbuf cluster */ #define EXT_SFBUF 2 /* sendfile(2)'s sf_buf */ #define EXT_JUMBOP 3 /* jumbo cluster page sized */ #define EXT_JUMBO9 4 /* jumbo cluster 9216 bytes */ #define EXT_JUMBO16 5 /* jumbo cluster 16184 bytes */ #define EXT_PACKET 6 /* mbuf+cluster from packet zone */ -#define EXT_MBUF 7 /* external mbuf reference (M_IOVEC) */ +#define EXT_MBUF 7 /* external mbuf reference */ #define EXT_SFBUF_NOCACHE 8 /* sendfile(2)'s sf_buf not to be cached */ #define EXT_VENDOR1 224 /* for vendor-internal use */ #define EXT_VENDOR2 225 /* for vendor-internal use */ #define EXT_VENDOR3 226 /* for vendor-internal use */ #define EXT_VENDOR4 227 /* for vendor-internal use */ #define EXT_EXP1 244 /* for experimental use */ #define EXT_EXP2 245 /* for experimental use */ #define EXT_EXP3 246 /* for experimental use */ #define EXT_EXP4 247 /* for experimental use */ #define EXT_NET_DRV 252 /* custom ext_buf provided by net driver(s) */ #define EXT_MOD_TYPE 253 /* custom module's ext_buf type */ #define EXT_DISPOSABLE 254 /* can throw this buffer away w/page flipping */ #define EXT_EXTREF 255 /* has externally maintained ext_cnt ptr */ /* * Flags for external mbuf buffer types. * NB: limited to the lower 24 bits. */ #define EXT_FLAG_EMBREF 0x000001 /* embedded ext_count */ #define EXT_FLAG_EXTREF 0x000002 /* external ext_cnt, notyet */ #define EXT_FLAG_NOFREE 0x000010 /* don't free mbuf to pool, notyet */ #define EXT_FLAG_VENDOR1 0x010000 /* for vendor-internal use */ #define EXT_FLAG_VENDOR2 0x020000 /* for vendor-internal use */ #define EXT_FLAG_VENDOR3 0x040000 /* for vendor-internal use */ #define EXT_FLAG_VENDOR4 0x080000 /* for vendor-internal use */ #define EXT_FLAG_EXP1 0x100000 /* for experimental use */ #define EXT_FLAG_EXP2 0x200000 /* for experimental use */ #define EXT_FLAG_EXP3 0x400000 /* for experimental use */ #define EXT_FLAG_EXP4 0x800000 /* for experimental use */ /* * EXT flag description for use with printf(9) %b identifier. */ #define EXT_FLAG_BITS \ "\20\1EXT_FLAG_EMBREF\2EXT_FLAG_EXTREF\5EXT_FLAG_NOFREE" \ "\21EXT_FLAG_VENDOR1\22EXT_FLAG_VENDOR2\23EXT_FLAG_VENDOR3" \ "\24EXT_FLAG_VENDOR4\25EXT_FLAG_EXP1\26EXT_FLAG_EXP2\27EXT_FLAG_EXP3" \ "\30EXT_FLAG_EXP4" /* * External reference/free functions. */ void sf_ext_free(void *, void *); void sf_ext_free_nocache(void *, void *); /* * Flags indicating checksum, segmentation and other offload work to be * done, or already done, by hardware or lower layers. It is split into * separate inbound and outbound flags. * * Outbound flags that are set by upper protocol layers requesting lower * layers, or ideally the hardware, to perform these offloading tasks. * For outbound packets this field and its flags can be directly tested * against ifnet if_hwassist. */ #define CSUM_IP 0x00000001 /* IP header checksum offload */ #define CSUM_IP_UDP 0x00000002 /* UDP checksum offload */ #define CSUM_IP_TCP 0x00000004 /* TCP checksum offload */ #define CSUM_IP_SCTP 0x00000008 /* SCTP checksum offload */ #define CSUM_IP_TSO 0x00000010 /* TCP segmentation offload */ #define CSUM_IP_ISCSI 0x00000020 /* iSCSI checksum offload */ #define CSUM_IP6_UDP 0x00000200 /* UDP checksum offload */ #define CSUM_IP6_TCP 0x00000400 /* TCP checksum offload */ #define CSUM_IP6_SCTP 0x00000800 /* SCTP checksum offload */ #define CSUM_IP6_TSO 0x00001000 /* TCP segmentation offload */ #define CSUM_IP6_ISCSI 0x00002000 /* iSCSI checksum offload */ /* Inbound checksum support where the checksum was verified by hardware. */ #define CSUM_L3_CALC 0x01000000 /* calculated layer 3 csum */ #define CSUM_L3_VALID 0x02000000 /* checksum is correct */ #define CSUM_L4_CALC 0x04000000 /* calculated layer 4 csum */ #define CSUM_L4_VALID 0x08000000 /* checksum is correct */ #define CSUM_L5_CALC 0x10000000 /* calculated layer 5 csum */ #define CSUM_L5_VALID 0x20000000 /* checksum is correct */ #define CSUM_COALESED 0x40000000 /* contains merged segments */ /* * CSUM flag description for use with printf(9) %b identifier. */ #define CSUM_BITS \ "\20\1CSUM_IP\2CSUM_IP_UDP\3CSUM_IP_TCP\4CSUM_IP_SCTP\5CSUM_IP_TSO" \ "\6CSUM_IP_ISCSI" \ "\12CSUM_IP6_UDP\13CSUM_IP6_TCP\14CSUM_IP6_SCTP\15CSUM_IP6_TSO" \ "\16CSUM_IP6_ISCSI" \ "\31CSUM_L3_CALC\32CSUM_L3_VALID\33CSUM_L4_CALC\34CSUM_L4_VALID" \ "\35CSUM_L5_CALC\36CSUM_L5_VALID\37CSUM_COALESED" /* CSUM flags compatibility mappings. */ #define CSUM_IP_CHECKED CSUM_L3_CALC #define CSUM_IP_VALID CSUM_L3_VALID #define CSUM_DATA_VALID CSUM_L4_VALID #define CSUM_PSEUDO_HDR CSUM_L4_CALC #define CSUM_SCTP_VALID CSUM_L4_VALID #define CSUM_DELAY_DATA (CSUM_TCP|CSUM_UDP) #define CSUM_DELAY_IP CSUM_IP /* Only v4, no v6 IP hdr csum */ #define CSUM_DELAY_DATA_IPV6 (CSUM_TCP_IPV6|CSUM_UDP_IPV6) #define CSUM_DATA_VALID_IPV6 CSUM_DATA_VALID #define CSUM_TCP CSUM_IP_TCP #define CSUM_UDP CSUM_IP_UDP #define CSUM_SCTP CSUM_IP_SCTP #define CSUM_TSO (CSUM_IP_TSO|CSUM_IP6_TSO) #define CSUM_UDP_IPV6 CSUM_IP6_UDP #define CSUM_TCP_IPV6 CSUM_IP6_TCP #define CSUM_SCTP_IPV6 CSUM_IP6_SCTP /* * mbuf types describing the content of the mbuf (including external storage). */ #define MT_NOTMBUF 0 /* USED INTERNALLY ONLY! Object is not mbuf */ #define MT_DATA 1 /* dynamic (data) allocation */ #define MT_HEADER MT_DATA /* packet header, use M_PKTHDR instead */ #define MT_VENDOR1 4 /* for vendor-internal use */ #define MT_VENDOR2 5 /* for vendor-internal use */ #define MT_VENDOR3 6 /* for vendor-internal use */ #define MT_VENDOR4 7 /* for vendor-internal use */ #define MT_SONAME 8 /* socket name */ #define MT_EXP1 9 /* for experimental use */ #define MT_EXP2 10 /* for experimental use */ #define MT_EXP3 11 /* for experimental use */ #define MT_EXP4 12 /* for experimental use */ #define MT_CONTROL 14 /* extra-data protocol message */ #define MT_OOBDATA 15 /* expedited data */ #define MT_NTYPES 16 /* number of mbuf types for mbtypes[] */ #define MT_NOINIT 255 /* Not a type but a flag to allocate a non-initialized mbuf */ /* * String names of mbuf-related UMA(9) and malloc(9) types. Exposed to * !_KERNEL so that monitoring tools can look up the zones with * libmemstat(3). */ #define MBUF_MEM_NAME "mbuf" #define MBUF_CLUSTER_MEM_NAME "mbuf_cluster" #define MBUF_PACKET_MEM_NAME "mbuf_packet" #define MBUF_JUMBOP_MEM_NAME "mbuf_jumbo_page" #define MBUF_JUMBO9_MEM_NAME "mbuf_jumbo_9k" #define MBUF_JUMBO16_MEM_NAME "mbuf_jumbo_16k" #define MBUF_TAG_MEM_NAME "mbuf_tag" #define MBUF_EXTREFCNT_MEM_NAME "mbuf_ext_refcnt" #ifdef _KERNEL #ifdef WITNESS #define MBUF_CHECKSLEEP(how) do { \ if (how == M_WAITOK) \ WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK, NULL, \ "Sleeping in \"%s\"", __func__); \ } while (0) #else #define MBUF_CHECKSLEEP(how) #endif /* * Network buffer allocation API * * The rest of it is defined in kern/kern_mbuf.c */ extern uma_zone_t zone_mbuf; extern uma_zone_t zone_clust; extern uma_zone_t zone_pack; extern uma_zone_t zone_jumbop; extern uma_zone_t zone_jumbo9; extern uma_zone_t zone_jumbo16; void mb_dupcl(struct mbuf *, struct mbuf *); void mb_free_ext(struct mbuf *); void m_adj(struct mbuf *, int); int m_apply(struct mbuf *, int, int, int (*)(void *, void *, u_int), void *); int m_append(struct mbuf *, int, c_caddr_t); void m_cat(struct mbuf *, struct mbuf *); void m_catpkt(struct mbuf *, struct mbuf *); int m_clget(struct mbuf *m, int how); void *m_cljget(struct mbuf *m, int how, int size); struct mbuf *m_collapse(struct mbuf *, int, int); void m_copyback(struct mbuf *, int, int, c_caddr_t); void m_copydata(const struct mbuf *, int, int, caddr_t); struct mbuf *m_copym(struct mbuf *, int, int, int); struct mbuf *m_copypacket(struct mbuf *, int); void m_copy_pkthdr(struct mbuf *, struct mbuf *); struct mbuf *m_copyup(struct mbuf *, int, int); struct mbuf *m_defrag(struct mbuf *, int); void m_demote_pkthdr(struct mbuf *); void m_demote(struct mbuf *, int, int); struct mbuf *m_devget(char *, int, int, struct ifnet *, void (*)(char *, caddr_t, u_int)); struct mbuf *m_dup(const struct mbuf *, int); int m_dup_pkthdr(struct mbuf *, const struct mbuf *, int); void m_extadd(struct mbuf *, caddr_t, u_int, void (*)(struct mbuf *, void *, void *), void *, void *, int, int); u_int m_fixhdr(struct mbuf *); struct mbuf *m_fragment(struct mbuf *, int, int); void m_freem(struct mbuf *); struct mbuf *m_get2(int, int, short, int); struct mbuf *m_getjcl(int, short, int, int); struct mbuf *m_getm2(struct mbuf *, int, int, short, int); struct mbuf *m_getptr(struct mbuf *, int, int *); u_int m_length(struct mbuf *, struct mbuf **); int m_mbuftouio(struct uio *, struct mbuf *, int); void m_move_pkthdr(struct mbuf *, struct mbuf *); int m_pkthdr_init(struct mbuf *, int); struct mbuf *m_prepend(struct mbuf *, int, int); void m_print(const struct mbuf *, int); struct mbuf *m_pulldown(struct mbuf *, int, int, int *); struct mbuf *m_pullup(struct mbuf *, int); int m_sanity(struct mbuf *, int); struct mbuf *m_split(struct mbuf *, int, int); struct mbuf *m_uiotombuf(struct uio *, int, int, int, int); struct mbuf *m_unshare(struct mbuf *, int); static __inline int m_gettype(int size) { int type; switch (size) { case MSIZE: type = EXT_MBUF; break; case MCLBYTES: type = EXT_CLUSTER; break; #if MJUMPAGESIZE != MCLBYTES case MJUMPAGESIZE: type = EXT_JUMBOP; break; #endif case MJUM9BYTES: type = EXT_JUMBO9; break; case MJUM16BYTES: type = EXT_JUMBO16; break; default: panic("%s: invalid cluster size %d", __func__, size); } return (type); } /* * Associated an external reference counted buffer with an mbuf. */ static __inline void m_extaddref(struct mbuf *m, caddr_t buf, u_int size, u_int *ref_cnt, void (*freef)(struct mbuf *, void *, void *), void *arg1, void *arg2) { KASSERT(ref_cnt != NULL, ("%s: ref_cnt not provided", __func__)); atomic_add_int(ref_cnt, 1); m->m_flags |= M_EXT; m->m_ext.ext_buf = buf; m->m_ext.ext_cnt = ref_cnt; m->m_data = m->m_ext.ext_buf; m->m_ext.ext_size = size; m->m_ext.ext_free = freef; m->m_ext.ext_arg1 = arg1; m->m_ext.ext_arg2 = arg2; m->m_ext.ext_type = EXT_EXTREF; m->m_ext.ext_flags = 0; } static __inline uma_zone_t m_getzone(int size) { uma_zone_t zone; switch (size) { case MCLBYTES: zone = zone_clust; break; #if MJUMPAGESIZE != MCLBYTES case MJUMPAGESIZE: zone = zone_jumbop; break; #endif case MJUM9BYTES: zone = zone_jumbo9; break; case MJUM16BYTES: zone = zone_jumbo16; break; default: panic("%s: invalid cluster size %d", __func__, size); } return (zone); } /* * Initialize an mbuf with linear storage. * * Inline because the consumer text overhead will be roughly the same to * initialize or call a function with this many parameters and M_PKTHDR * should go away with constant propagation for !MGETHDR. */ static __inline int m_init(struct mbuf *m, int how, short type, int flags) { int error; m->m_next = NULL; m->m_nextpkt = NULL; m->m_data = m->m_dat; m->m_len = 0; m->m_flags = flags; m->m_type = type; if (flags & M_PKTHDR) error = m_pkthdr_init(m, how); else error = 0; MBUF_PROBE5(m__init, m, how, type, flags, error); return (error); } static __inline struct mbuf * m_get(int how, short type) { struct mbuf *m; struct mb_args args; args.flags = 0; args.type = type; m = uma_zalloc_arg(zone_mbuf, &args, how); MBUF_PROBE3(m__get, how, type, m); return (m); } static __inline struct mbuf * m_gethdr(int how, short type) { struct mbuf *m; struct mb_args args; args.flags = M_PKTHDR; args.type = type; m = uma_zalloc_arg(zone_mbuf, &args, how); MBUF_PROBE3(m__gethdr, how, type, m); return (m); } static __inline struct mbuf * m_getcl(int how, short type, int flags) { struct mbuf *m; struct mb_args args; args.flags = flags; args.type = type; m = uma_zalloc_arg(zone_pack, &args, how); MBUF_PROBE4(m__getcl, how, type, flags, m); return (m); } /* * XXX: m_cljset() is a dangerous API. One must attach only a new, * unreferenced cluster to an mbuf(9). It is not possible to assert * that, so care can be taken only by users of the API. */ static __inline void m_cljset(struct mbuf *m, void *cl, int type) { int size; switch (type) { case EXT_CLUSTER: size = MCLBYTES; break; #if MJUMPAGESIZE != MCLBYTES case EXT_JUMBOP: size = MJUMPAGESIZE; break; #endif case EXT_JUMBO9: size = MJUM9BYTES; break; case EXT_JUMBO16: size = MJUM16BYTES; break; default: panic("%s: unknown cluster type %d", __func__, type); break; } m->m_data = m->m_ext.ext_buf = cl; m->m_ext.ext_free = m->m_ext.ext_arg1 = m->m_ext.ext_arg2 = NULL; m->m_ext.ext_size = size; m->m_ext.ext_type = type; m->m_ext.ext_flags = EXT_FLAG_EMBREF; m->m_ext.ext_count = 1; m->m_flags |= M_EXT; MBUF_PROBE3(m__cljset, m, cl, type); } static __inline void m_chtype(struct mbuf *m, short new_type) { m->m_type = new_type; } static __inline void m_clrprotoflags(struct mbuf *m) { while (m) { m->m_flags &= ~M_PROTOFLAGS; m = m->m_next; } } static __inline struct mbuf * m_last(struct mbuf *m) { while (m->m_next) m = m->m_next; return (m); } static inline u_int m_extrefcnt(struct mbuf *m) { KASSERT(m->m_flags & M_EXT, ("%s: M_EXT missing", __func__)); return ((m->m_ext.ext_flags & EXT_FLAG_EMBREF) ? m->m_ext.ext_count : *m->m_ext.ext_cnt); } /* * mbuf, cluster, and external object allocation macros (for compatibility * purposes). */ #define M_MOVE_PKTHDR(to, from) m_move_pkthdr((to), (from)) #define MGET(m, how, type) ((m) = m_get((how), (type))) #define MGETHDR(m, how, type) ((m) = m_gethdr((how), (type))) #define MCLGET(m, how) m_clget((m), (how)) #define MEXTADD(m, buf, size, free, arg1, arg2, flags, type) \ m_extadd((m), (caddr_t)(buf), (size), (free), (arg1), (arg2), \ (flags), (type)) #define m_getm(m, len, how, type) \ m_getm2((m), (len), (how), (type), M_PKTHDR) /* * Evaluate TRUE if it's safe to write to the mbuf m's data region (this can * be both the local data payload, or an external buffer area, depending on * whether M_EXT is set). */ #define M_WRITABLE(m) (!((m)->m_flags & M_RDONLY) && \ (!(((m)->m_flags & M_EXT)) || \ (m_extrefcnt(m) == 1))) /* Check if the supplied mbuf has a packet header, or else panic. */ #define M_ASSERTPKTHDR(m) \ KASSERT((m) != NULL && (m)->m_flags & M_PKTHDR, \ ("%s: no mbuf packet header!", __func__)) /* * Ensure that the supplied mbuf is a valid, non-free mbuf. * * XXX: Broken at the moment. Need some UMA magic to make it work again. */ #define M_ASSERTVALID(m) \ KASSERT((((struct mbuf *)m)->m_flags & 0) == 0, \ ("%s: attempted use of a free mbuf!", __func__)) /* * Return the address of the start of the buffer associated with an mbuf, * handling external storage, packet-header mbufs, and regular data mbufs. */ #define M_START(m) \ (((m)->m_flags & M_EXT) ? (m)->m_ext.ext_buf : \ ((m)->m_flags & M_PKTHDR) ? &(m)->m_pktdat[0] : \ &(m)->m_dat[0]) /* * Return the size of the buffer associated with an mbuf, handling external * storage, packet-header mbufs, and regular data mbufs. */ #define M_SIZE(m) \ (((m)->m_flags & M_EXT) ? (m)->m_ext.ext_size : \ ((m)->m_flags & M_PKTHDR) ? MHLEN : \ MLEN) /* * Set the m_data pointer of a newly allocated mbuf to place an object of the * specified size at the end of the mbuf, longword aligned. * * NB: Historically, we had M_ALIGN(), MH_ALIGN(), and MEXT_ALIGN() as * separate macros, each asserting that it was called at the proper moment. * This required callers to themselves test the storage type and call the * right one. Rather than require callers to be aware of those layout * decisions, we centralize here. */ static __inline void m_align(struct mbuf *m, int len) { #ifdef INVARIANTS const char *msg = "%s: not a virgin mbuf"; #endif int adjust; KASSERT(m->m_data == M_START(m), (msg, __func__)); adjust = M_SIZE(m) - len; m->m_data += adjust &~ (sizeof(long)-1); } #define M_ALIGN(m, len) m_align(m, len) #define MH_ALIGN(m, len) m_align(m, len) #define MEXT_ALIGN(m, len) m_align(m, len) /* * Compute the amount of space available before the current start of data in * an mbuf. * * The M_WRITABLE() is a temporary, conservative safety measure: the burden * of checking writability of the mbuf data area rests solely with the caller. * * NB: In previous versions, M_LEADINGSPACE() would only check M_WRITABLE() * for mbufs with external storage. We now allow mbuf-embedded data to be * read-only as well. */ #define M_LEADINGSPACE(m) \ (M_WRITABLE(m) ? ((m)->m_data - M_START(m)) : 0) /* * Compute the amount of space available after the end of data in an mbuf. * * The M_WRITABLE() is a temporary, conservative safety measure: the burden * of checking writability of the mbuf data area rests solely with the caller. * * NB: In previous versions, M_TRAILINGSPACE() would only check M_WRITABLE() * for mbufs with external storage. We now allow mbuf-embedded data to be * read-only as well. */ #define M_TRAILINGSPACE(m) \ (M_WRITABLE(m) ? \ ((M_START(m) + M_SIZE(m)) - ((m)->m_data + (m)->m_len)) : 0) /* * Arrange to prepend space of size plen to mbuf m. If a new mbuf must be * allocated, how specifies whether to wait. If the allocation fails, the * original mbuf chain is freed and m is set to NULL. */ #define M_PREPEND(m, plen, how) do { \ struct mbuf **_mmp = &(m); \ struct mbuf *_mm = *_mmp; \ int _mplen = (plen); \ int __mhow = (how); \ \ MBUF_CHECKSLEEP(how); \ if (M_LEADINGSPACE(_mm) >= _mplen) { \ _mm->m_data -= _mplen; \ _mm->m_len += _mplen; \ } else \ _mm = m_prepend(_mm, _mplen, __mhow); \ if (_mm != NULL && _mm->m_flags & M_PKTHDR) \ _mm->m_pkthdr.len += _mplen; \ *_mmp = _mm; \ } while (0) /* * Change mbuf to new type. This is a relatively expensive operation and * should be avoided. */ #define MCHTYPE(m, t) m_chtype((m), (t)) /* Length to m_copy to copy all. */ #define M_COPYALL 1000000000 /* Compatibility with 4.3. */ #define m_copy(m, o, l) m_copym((m), (o), (l), M_NOWAIT) extern int max_datalen; /* MHLEN - max_hdr */ extern int max_hdr; /* Largest link + protocol header */ extern int max_linkhdr; /* Largest link-level header */ extern int max_protohdr; /* Largest protocol header */ extern int nmbclusters; /* Maximum number of clusters */ /*- * Network packets may have annotations attached by affixing a list of * "packet tags" to the pkthdr structure. Packet tags are dynamically * allocated semi-opaque data structures that have a fixed header * (struct m_tag) that specifies the size of the memory block and a * pair that identifies it. The cookie is a 32-bit unique * unsigned value used to identify a module or ABI. By convention this value * is chosen as the date+time that the module is created, expressed as the * number of seconds since the epoch (e.g., using date -u +'%s'). The type * value is an ABI/module-specific value that identifies a particular * annotation and is private to the module. For compatibility with systems * like OpenBSD that define packet tags w/o an ABI/module cookie, the value * PACKET_ABI_COMPAT is used to implement m_tag_get and m_tag_find * compatibility shim functions and several tag types are defined below. * Users that do not require compatibility should use a private cookie value * so that packet tag-related definitions can be maintained privately. * * Note that the packet tag returned by m_tag_alloc has the default memory * alignment implemented by malloc. To reference private data one can use a * construct like: * * struct m_tag *mtag = m_tag_alloc(...); * struct foo *p = (struct foo *)(mtag+1); * * if the alignment of struct m_tag is sufficient for referencing members of * struct foo. Otherwise it is necessary to embed struct m_tag within the * private data structure to insure proper alignment; e.g., * * struct foo { * struct m_tag tag; * ... * }; * struct foo *p = (struct foo *) m_tag_alloc(...); * struct m_tag *mtag = &p->tag; */ /* * Persistent tags stay with an mbuf until the mbuf is reclaimed. Otherwise * tags are expected to ``vanish'' when they pass through a network * interface. For most interfaces this happens normally as the tags are * reclaimed when the mbuf is free'd. However in some special cases * reclaiming must be done manually. An example is packets that pass through * the loopback interface. Also, one must be careful to do this when * ``turning around'' packets (e.g., icmp_reflect). * * To mark a tag persistent bit-or this flag in when defining the tag id. * The tag will then be treated as described above. */ #define MTAG_PERSISTENT 0x800 #define PACKET_TAG_NONE 0 /* Nadda */ /* Packet tags for use with PACKET_ABI_COMPAT. */ #define PACKET_TAG_IPSEC_IN_DONE 1 /* IPsec applied, in */ #define PACKET_TAG_IPSEC_OUT_DONE 2 /* IPsec applied, out */ #define PACKET_TAG_IPSEC_IN_CRYPTO_DONE 3 /* NIC IPsec crypto done */ #define PACKET_TAG_IPSEC_OUT_CRYPTO_NEEDED 4 /* NIC IPsec crypto req'ed */ #define PACKET_TAG_IPSEC_IN_COULD_DO_CRYPTO 5 /* NIC notifies IPsec */ #define PACKET_TAG_IPSEC_PENDING_TDB 6 /* Reminder to do IPsec */ #define PACKET_TAG_BRIDGE 7 /* Bridge processing done */ #define PACKET_TAG_GIF 8 /* GIF processing done */ #define PACKET_TAG_GRE 9 /* GRE processing done */ #define PACKET_TAG_IN_PACKET_CHECKSUM 10 /* NIC checksumming done */ #define PACKET_TAG_ENCAP 11 /* Encap. processing */ #define PACKET_TAG_IPSEC_SOCKET 12 /* IPSEC socket ref */ #define PACKET_TAG_IPSEC_HISTORY 13 /* IPSEC history */ #define PACKET_TAG_IPV6_INPUT 14 /* IPV6 input processing */ #define PACKET_TAG_DUMMYNET 15 /* dummynet info */ #define PACKET_TAG_DIVERT 17 /* divert info */ #define PACKET_TAG_IPFORWARD 18 /* ipforward info */ #define PACKET_TAG_MACLABEL (19 | MTAG_PERSISTENT) /* MAC label */ #define PACKET_TAG_PF (21 | MTAG_PERSISTENT) /* PF/ALTQ information */ #define PACKET_TAG_RTSOCKFAM 25 /* rtsock sa family */ #define PACKET_TAG_IPOPTIONS 27 /* Saved IP options */ #define PACKET_TAG_CARP 28 /* CARP info */ #define PACKET_TAG_IPSEC_NAT_T_PORTS 29 /* two uint16_t */ #define PACKET_TAG_ND_OUTGOING 30 /* ND outgoing */ /* Specific cookies and tags. */ /* Packet tag routines. */ struct m_tag *m_tag_alloc(u_int32_t, int, int, int); void m_tag_delete(struct mbuf *, struct m_tag *); void m_tag_delete_chain(struct mbuf *, struct m_tag *); void m_tag_free_default(struct m_tag *); struct m_tag *m_tag_locate(struct mbuf *, u_int32_t, int, struct m_tag *); struct m_tag *m_tag_copy(struct m_tag *, int); int m_tag_copy_chain(struct mbuf *, const struct mbuf *, int); void m_tag_delete_nonpersistent(struct mbuf *); /* * Initialize the list of tags associated with an mbuf. */ static __inline void m_tag_init(struct mbuf *m) { SLIST_INIT(&m->m_pkthdr.tags); } /* * Set up the contents of a tag. Note that this does not fill in the free * method; the caller is expected to do that. * * XXX probably should be called m_tag_init, but that was already taken. */ static __inline void m_tag_setup(struct m_tag *t, u_int32_t cookie, int type, int len) { t->m_tag_id = type; t->m_tag_len = len; t->m_tag_cookie = cookie; } /* * Reclaim resources associated with a tag. */ static __inline void m_tag_free(struct m_tag *t) { (*t->m_tag_free)(t); } /* * Return the first tag associated with an mbuf. */ static __inline struct m_tag * m_tag_first(struct mbuf *m) { return (SLIST_FIRST(&m->m_pkthdr.tags)); } /* * Return the next tag in the list of tags associated with an mbuf. */ static __inline struct m_tag * m_tag_next(struct mbuf *m __unused, struct m_tag *t) { return (SLIST_NEXT(t, m_tag_link)); } /* * Prepend a tag to the list of tags associated with an mbuf. */ static __inline void m_tag_prepend(struct mbuf *m, struct m_tag *t) { SLIST_INSERT_HEAD(&m->m_pkthdr.tags, t, m_tag_link); } /* * Unlink a tag from the list of tags associated with an mbuf. */ static __inline void m_tag_unlink(struct mbuf *m, struct m_tag *t) { SLIST_REMOVE(&m->m_pkthdr.tags, t, m_tag, m_tag_link); } /* These are for OpenBSD compatibility. */ #define MTAG_ABI_COMPAT 0 /* compatibility ABI */ static __inline struct m_tag * m_tag_get(int type, int length, int wait) { return (m_tag_alloc(MTAG_ABI_COMPAT, type, length, wait)); } static __inline struct m_tag * m_tag_find(struct mbuf *m, int type, struct m_tag *start) { return (SLIST_EMPTY(&m->m_pkthdr.tags) ? (struct m_tag *)NULL : m_tag_locate(m, MTAG_ABI_COMPAT, type, start)); } static __inline struct mbuf * m_free(struct mbuf *m) { struct mbuf *n = m->m_next; MBUF_PROBE1(m__free, m); if ((m->m_flags & (M_PKTHDR|M_NOFREE)) == (M_PKTHDR|M_NOFREE)) m_tag_delete_chain(m, NULL); if (m->m_flags & M_EXT) mb_free_ext(m); else if ((m->m_flags & M_NOFREE) == 0) uma_zfree(zone_mbuf, m); return (n); } static __inline int rt_m_getfib(struct mbuf *m) { KASSERT(m->m_flags & M_PKTHDR , ("Attempt to get FIB from non header mbuf.")); return (m->m_pkthdr.fibnum); } #define M_GETFIB(_m) rt_m_getfib(_m) #define M_SETFIB(_m, _fib) do { \ KASSERT((_m)->m_flags & M_PKTHDR, ("Attempt to set FIB on non header mbuf.")); \ ((_m)->m_pkthdr.fibnum) = (_fib); \ } while (0) /* flags passed as first argument for "m_ether_tcpip_hash()" */ #define MBUF_HASHFLAG_L2 (1 << 2) #define MBUF_HASHFLAG_L3 (1 << 3) #define MBUF_HASHFLAG_L4 (1 << 4) /* mbuf hashing helper routines */ uint32_t m_ether_tcpip_hash_init(void); uint32_t m_ether_tcpip_hash(const uint32_t, const struct mbuf *, const uint32_t); #ifdef MBUF_PROFILING void m_profile(struct mbuf *m); #define M_PROFILE(m) m_profile(m) #else #define M_PROFILE(m) #endif struct mbufq { STAILQ_HEAD(, mbuf) mq_head; int mq_len; int mq_maxlen; }; static inline void mbufq_init(struct mbufq *mq, int maxlen) { STAILQ_INIT(&mq->mq_head); mq->mq_maxlen = maxlen; mq->mq_len = 0; } static inline struct mbuf * mbufq_flush(struct mbufq *mq) { struct mbuf *m; m = STAILQ_FIRST(&mq->mq_head); STAILQ_INIT(&mq->mq_head); mq->mq_len = 0; return (m); } static inline void mbufq_drain(struct mbufq *mq) { struct mbuf *m, *n; n = mbufq_flush(mq); while ((m = n) != NULL) { n = STAILQ_NEXT(m, m_stailqpkt); m_freem(m); } } static inline struct mbuf * mbufq_first(const struct mbufq *mq) { return (STAILQ_FIRST(&mq->mq_head)); } static inline struct mbuf * mbufq_last(const struct mbufq *mq) { return (STAILQ_LAST(&mq->mq_head, mbuf, m_stailqpkt)); } static inline int mbufq_full(const struct mbufq *mq) { return (mq->mq_len >= mq->mq_maxlen); } static inline int mbufq_len(const struct mbufq *mq) { return (mq->mq_len); } static inline int mbufq_enqueue(struct mbufq *mq, struct mbuf *m) { if (mbufq_full(mq)) return (ENOBUFS); STAILQ_INSERT_TAIL(&mq->mq_head, m, m_stailqpkt); mq->mq_len++; return (0); } static inline struct mbuf * mbufq_dequeue(struct mbufq *mq) { struct mbuf *m; m = STAILQ_FIRST(&mq->mq_head); if (m) { STAILQ_REMOVE_HEAD(&mq->mq_head, m_stailqpkt); m->m_nextpkt = NULL; mq->mq_len--; } return (m); } static inline void mbufq_prepend(struct mbufq *mq, struct mbuf *m) { STAILQ_INSERT_HEAD(&mq->mq_head, m, m_stailqpkt); mq->mq_len++; } #endif /* _KERNEL */ #endif /* !_SYS_MBUF_H_ */ Index: projects/netbsd-tests-update-12/sys/vm/swap_pager.c =================================================================== --- projects/netbsd-tests-update-12/sys/vm/swap_pager.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/vm/swap_pager.c (revision 305172) @@ -1,2739 +1,2776 @@ /*- * Copyright (c) 1998 Matthew Dillon, * Copyright (c) 1994 John S. Dyson * Copyright (c) 1990 University of Utah. * Copyright (c) 1982, 1986, 1989, 1993 * The Regents of the University of California. All rights reserved. * * This code is derived from software contributed to Berkeley by * the Systems Programming Group of the University of Utah Computer * Science Department. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by the University of * California, Berkeley and its contributors. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * New Swap System * Matthew Dillon * * Radix Bitmap 'blists'. * * - The new swapper uses the new radix bitmap code. This should scale * to arbitrarily small or arbitrarily large swap spaces and an almost * arbitrary degree of fragmentation. * * Features: * * - on the fly reallocation of swap during putpages. The new system * does not try to keep previously allocated swap blocks for dirty * pages. * * - on the fly deallocation of swap * * - No more garbage collection required. Unnecessarily allocated swap * blocks only exist for dirty vm_page_t's now and these are already * cycled (in a high-load system) by the pager. We also do on-the-fly * removal of invalidated swap blocks when a page is destroyed * or renamed. * * from: Utah $Hdr: swap_pager.c 1.4 91/04/30$ * * @(#)swap_pager.c 8.9 (Berkeley) 3/21/94 * @(#)vm_swap.c 8.5 (Berkeley) 2/17/94 */ #include __FBSDID("$FreeBSD$"); #include "opt_swap.h" #include "opt_vm.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* * SWB_NPAGES must be a power of 2. It may be set to 1, 2, 4, 8, 16 * or 32 pages per allocation. * The 32-page limit is due to the radix code (kern/subr_blist.c). */ #ifndef MAX_PAGEOUT_CLUSTER #define MAX_PAGEOUT_CLUSTER 16 #endif #if !defined(SWB_NPAGES) #define SWB_NPAGES MAX_PAGEOUT_CLUSTER #endif /* * The swblock structure maps an object and a small, fixed-size range * of page indices to disk addresses within a swap area. * The collection of these mappings is implemented as a hash table. * Unused disk addresses within a swap area are allocated and managed * using a blist. */ #define SWCORRECT(n) (sizeof(void *) * (n) / sizeof(daddr_t)) #define SWAP_META_PAGES (SWB_NPAGES * 2) #define SWAP_META_MASK (SWAP_META_PAGES - 1) struct swblock { struct swblock *swb_hnext; vm_object_t swb_object; vm_pindex_t swb_index; int swb_count; daddr_t swb_pages[SWAP_META_PAGES]; }; static MALLOC_DEFINE(M_VMPGDATA, "vm_pgdata", "swap pager private data"); static struct mtx sw_dev_mtx; static TAILQ_HEAD(, swdevt) swtailq = TAILQ_HEAD_INITIALIZER(swtailq); static struct swdevt *swdevhd; /* Allocate from here next */ static int nswapdev; /* Number of swap devices */ int swap_pager_avail; static struct sx swdev_syscall_lock; /* serialize swap(on|off) */ static vm_ooffset_t swap_total; SYSCTL_QUAD(_vm, OID_AUTO, swap_total, CTLFLAG_RD, &swap_total, 0, "Total amount of available swap storage."); static vm_ooffset_t swap_reserved; SYSCTL_QUAD(_vm, OID_AUTO, swap_reserved, CTLFLAG_RD, &swap_reserved, 0, "Amount of swap storage needed to back all allocated anonymous memory."); static int overcommit = 0; SYSCTL_INT(_vm, OID_AUTO, overcommit, CTLFLAG_RW, &overcommit, 0, "Configure virtual memory overcommit behavior. See tuning(7) " "for details."); static unsigned long swzone; SYSCTL_ULONG(_vm, OID_AUTO, swzone, CTLFLAG_RD, &swzone, 0, "Actual size of swap metadata zone"); static unsigned long swap_maxpages; SYSCTL_ULONG(_vm, OID_AUTO, swap_maxpages, CTLFLAG_RD, &swap_maxpages, 0, "Maximum amount of swap supported"); /* bits from overcommit */ #define SWAP_RESERVE_FORCE_ON (1 << 0) #define SWAP_RESERVE_RLIMIT_ON (1 << 1) #define SWAP_RESERVE_ALLOW_NONWIRED (1 << 2) int swap_reserve(vm_ooffset_t incr) { return (swap_reserve_by_cred(incr, curthread->td_ucred)); } int swap_reserve_by_cred(vm_ooffset_t incr, struct ucred *cred) { vm_ooffset_t r, s; int res, error; static int curfail; static struct timeval lastfail; struct uidinfo *uip; uip = cred->cr_ruidinfo; if (incr & PAGE_MASK) panic("swap_reserve: & PAGE_MASK"); #ifdef RACCT if (racct_enable) { PROC_LOCK(curproc); error = racct_add(curproc, RACCT_SWAP, incr); PROC_UNLOCK(curproc); if (error != 0) return (0); } #endif res = 0; mtx_lock(&sw_dev_mtx); r = swap_reserved + incr; if (overcommit & SWAP_RESERVE_ALLOW_NONWIRED) { s = vm_cnt.v_page_count - vm_cnt.v_free_reserved - vm_cnt.v_wire_count; s *= PAGE_SIZE; } else s = 0; s += swap_total; if ((overcommit & SWAP_RESERVE_FORCE_ON) == 0 || r <= s || (error = priv_check(curthread, PRIV_VM_SWAP_NOQUOTA)) == 0) { res = 1; swap_reserved = r; } mtx_unlock(&sw_dev_mtx); if (res) { UIDINFO_VMSIZE_LOCK(uip); if ((overcommit & SWAP_RESERVE_RLIMIT_ON) != 0 && uip->ui_vmsize + incr > lim_cur(curthread, RLIMIT_SWAP) && priv_check(curthread, PRIV_VM_SWAP_NORLIMIT)) res = 0; else uip->ui_vmsize += incr; UIDINFO_VMSIZE_UNLOCK(uip); if (!res) { mtx_lock(&sw_dev_mtx); swap_reserved -= incr; mtx_unlock(&sw_dev_mtx); } } if (!res && ppsratecheck(&lastfail, &curfail, 1)) { printf("uid %d, pid %d: swap reservation for %jd bytes failed\n", uip->ui_uid, curproc->p_pid, incr); } #ifdef RACCT if (!res) { PROC_LOCK(curproc); racct_sub(curproc, RACCT_SWAP, incr); PROC_UNLOCK(curproc); } #endif return (res); } void swap_reserve_force(vm_ooffset_t incr) { struct uidinfo *uip; mtx_lock(&sw_dev_mtx); swap_reserved += incr; mtx_unlock(&sw_dev_mtx); #ifdef RACCT PROC_LOCK(curproc); racct_add_force(curproc, RACCT_SWAP, incr); PROC_UNLOCK(curproc); #endif uip = curthread->td_ucred->cr_ruidinfo; PROC_LOCK(curproc); UIDINFO_VMSIZE_LOCK(uip); uip->ui_vmsize += incr; UIDINFO_VMSIZE_UNLOCK(uip); PROC_UNLOCK(curproc); } void swap_release(vm_ooffset_t decr) { struct ucred *cred; PROC_LOCK(curproc); cred = curthread->td_ucred; swap_release_by_cred(decr, cred); PROC_UNLOCK(curproc); } void swap_release_by_cred(vm_ooffset_t decr, struct ucred *cred) { struct uidinfo *uip; uip = cred->cr_ruidinfo; if (decr & PAGE_MASK) panic("swap_release: & PAGE_MASK"); mtx_lock(&sw_dev_mtx); if (swap_reserved < decr) panic("swap_reserved < decr"); swap_reserved -= decr; mtx_unlock(&sw_dev_mtx); UIDINFO_VMSIZE_LOCK(uip); if (uip->ui_vmsize < decr) printf("negative vmsize for uid = %d\n", uip->ui_uid); uip->ui_vmsize -= decr; UIDINFO_VMSIZE_UNLOCK(uip); racct_sub_cred(cred, RACCT_SWAP, decr); } #define SWM_FREE 0x02 /* free, period */ #define SWM_POP 0x04 /* pop out */ int swap_pager_full = 2; /* swap space exhaustion (task killing) */ static int swap_pager_almost_full = 1; /* swap space exhaustion (w/hysteresis)*/ static int nsw_rcount; /* free read buffers */ static int nsw_wcount_sync; /* limit write buffers / synchronous */ static int nsw_wcount_async; /* limit write buffers / asynchronous */ static int nsw_wcount_async_max;/* assigned maximum */ static int nsw_cluster_max; /* maximum VOP I/O allowed */ static int sysctl_swap_async_max(SYSCTL_HANDLER_ARGS); SYSCTL_PROC(_vm, OID_AUTO, swap_async_max, CTLTYPE_INT | CTLFLAG_RW | CTLFLAG_MPSAFE, NULL, 0, sysctl_swap_async_max, "I", "Maximum running async swap ops"); static struct swblock **swhash; static int swhash_mask; static struct mtx swhash_mtx; static struct sx sw_alloc_sx; /* * "named" and "unnamed" anon region objects. Try to reduce the overhead * of searching a named list by hashing it just a little. */ #define NOBJLISTS 8 #define NOBJLIST(handle) \ (&swap_pager_object_list[((int)(intptr_t)handle >> 4) & (NOBJLISTS-1)]) static struct pagerlst swap_pager_object_list[NOBJLISTS]; static uma_zone_t swap_zone; /* * pagerops for OBJT_SWAP - "swap pager". Some ops are also global procedure * calls hooked from other parts of the VM system and do not appear here. * (see vm/swap_pager.h). */ static vm_object_t swap_pager_alloc(void *handle, vm_ooffset_t size, vm_prot_t prot, vm_ooffset_t offset, struct ucred *); static void swap_pager_dealloc(vm_object_t object); static int swap_pager_getpages(vm_object_t, vm_page_t *, int, int *, int *); static int swap_pager_getpages_async(vm_object_t, vm_page_t *, int, int *, int *, pgo_getpages_iodone_t, void *); static void swap_pager_putpages(vm_object_t, vm_page_t *, int, boolean_t, int *); static boolean_t swap_pager_haspage(vm_object_t object, vm_pindex_t pindex, int *before, int *after); static void swap_pager_init(void); static void swap_pager_unswapped(vm_page_t); static void swap_pager_swapoff(struct swdevt *sp); struct pagerops swappagerops = { .pgo_init = swap_pager_init, /* early system initialization of pager */ .pgo_alloc = swap_pager_alloc, /* allocate an OBJT_SWAP object */ .pgo_dealloc = swap_pager_dealloc, /* deallocate an OBJT_SWAP object */ .pgo_getpages = swap_pager_getpages, /* pagein */ .pgo_getpages_async = swap_pager_getpages_async, /* pagein (async) */ .pgo_putpages = swap_pager_putpages, /* pageout */ .pgo_haspage = swap_pager_haspage, /* get backing store status for page */ .pgo_pageunswapped = swap_pager_unswapped, /* remove swap related to page */ }; /* * dmmax is in page-sized chunks with the new swap system. It was * dev-bsized chunks in the old. dmmax is always a power of 2. * * swap_*() routines are externally accessible. swp_*() routines are * internal. */ static int dmmax; static int nswap_lowat = 128; /* in pages, swap_pager_almost_full warn */ static int nswap_hiwat = 512; /* in pages, swap_pager_almost_full warn */ SYSCTL_INT(_vm, OID_AUTO, dmmax, CTLFLAG_RD, &dmmax, 0, "Maximum size of a swap block"); static void swp_sizecheck(void); static void swp_pager_async_iodone(struct buf *bp); static int swapongeom(struct vnode *); static int swaponvp(struct thread *, struct vnode *, u_long); static int swapoff_one(struct swdevt *sp, struct ucred *cred); /* * Swap bitmap functions */ static void swp_pager_freeswapspace(daddr_t blk, int npages); static daddr_t swp_pager_getswapspace(int npages); /* * Metadata functions */ static struct swblock **swp_pager_hash(vm_object_t object, vm_pindex_t index); static void swp_pager_meta_build(vm_object_t, vm_pindex_t, daddr_t); static void swp_pager_meta_free(vm_object_t, vm_pindex_t, daddr_t); static void swp_pager_meta_free_all(vm_object_t); static daddr_t swp_pager_meta_ctl(vm_object_t, vm_pindex_t, int); /* * SWP_SIZECHECK() - update swap_pager_full indication * * update the swap_pager_almost_full indication and warn when we are * about to run out of swap space, using lowat/hiwat hysteresis. * * Clear swap_pager_full ( task killing ) indication when lowat is met. * * No restrictions on call * This routine may not block. */ static void swp_sizecheck(void) { if (swap_pager_avail < nswap_lowat) { if (swap_pager_almost_full == 0) { printf("swap_pager: out of swap space\n"); swap_pager_almost_full = 1; } } else { swap_pager_full = 0; if (swap_pager_avail > nswap_hiwat) swap_pager_almost_full = 0; } } /* * SWP_PAGER_HASH() - hash swap meta data * * This is an helper function which hashes the swapblk given * the object and page index. It returns a pointer to a pointer * to the object, or a pointer to a NULL pointer if it could not * find a swapblk. */ static struct swblock ** swp_pager_hash(vm_object_t object, vm_pindex_t index) { struct swblock **pswap; struct swblock *swap; index &= ~(vm_pindex_t)SWAP_META_MASK; pswap = &swhash[(index ^ (int)(intptr_t)object) & swhash_mask]; while ((swap = *pswap) != NULL) { if (swap->swb_object == object && swap->swb_index == index ) { break; } pswap = &swap->swb_hnext; } return (pswap); } /* * SWAP_PAGER_INIT() - initialize the swap pager! * * Expected to be started from system init. NOTE: This code is run * before much else so be careful what you depend on. Most of the VM * system has yet to be initialized at this point. */ static void swap_pager_init(void) { /* * Initialize object lists */ int i; for (i = 0; i < NOBJLISTS; ++i) TAILQ_INIT(&swap_pager_object_list[i]); mtx_init(&sw_dev_mtx, "swapdev", NULL, MTX_DEF); sx_init(&sw_alloc_sx, "swspsx"); sx_init(&swdev_syscall_lock, "swsysc"); /* * Device Stripe, in PAGE_SIZE'd blocks */ dmmax = SWB_NPAGES * 2; } /* * SWAP_PAGER_SWAP_INIT() - swap pager initialization from pageout process * * Expected to be started from pageout process once, prior to entering * its main loop. */ void swap_pager_swap_init(void) { unsigned long n, n2; /* * Number of in-transit swap bp operations. Don't * exhaust the pbufs completely. Make sure we * initialize workable values (0 will work for hysteresis * but it isn't very efficient). * * The nsw_cluster_max is constrained by the bp->b_pages[] * array (MAXPHYS/PAGE_SIZE) and our locally defined * MAX_PAGEOUT_CLUSTER. Also be aware that swap ops are * constrained by the swap device interleave stripe size. * * Currently we hardwire nsw_wcount_async to 4. This limit is * designed to prevent other I/O from having high latencies due to * our pageout I/O. The value 4 works well for one or two active swap * devices but is probably a little low if you have more. Even so, * a higher value would probably generate only a limited improvement * with three or four active swap devices since the system does not * typically have to pageout at extreme bandwidths. We will want * at least 2 per swap devices, and 4 is a pretty good value if you * have one NFS swap device due to the command/ack latency over NFS. * So it all works out pretty well. */ nsw_cluster_max = min((MAXPHYS/PAGE_SIZE), MAX_PAGEOUT_CLUSTER); mtx_lock(&pbuf_mtx); nsw_rcount = (nswbuf + 1) / 2; nsw_wcount_sync = (nswbuf + 3) / 4; nsw_wcount_async = 4; nsw_wcount_async_max = nsw_wcount_async; mtx_unlock(&pbuf_mtx); /* * Initialize our zone. Right now I'm just guessing on the number * we need based on the number of pages in the system. Each swblock * can hold 32 pages, so this is probably overkill. This reservation * is typically limited to around 32MB by default. */ n = vm_cnt.v_page_count / 2; if (maxswzone && n > maxswzone / sizeof(struct swblock)) n = maxswzone / sizeof(struct swblock); n2 = n; swap_zone = uma_zcreate("SWAPMETA", sizeof(struct swblock), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE | UMA_ZONE_VM); if (swap_zone == NULL) panic("failed to create swap_zone."); do { if (uma_zone_reserve_kva(swap_zone, n)) break; /* * if the allocation failed, try a zone two thirds the * size of the previous attempt. */ n -= ((n + 2) / 3); } while (n > 0); if (n2 != n) printf("Swap zone entries reduced from %lu to %lu.\n", n2, n); swap_maxpages = n * SWAP_META_PAGES; swzone = n * sizeof(struct swblock); n2 = n; /* * Initialize our meta-data hash table. The swapper does not need to * be quite as efficient as the VM system, so we do not use an * oversized hash table. * * n: size of hash table, must be power of 2 * swhash_mask: hash table index mask */ for (n = 1; n < n2 / 8; n *= 2) ; swhash = malloc(sizeof(struct swblock *) * n, M_VMPGDATA, M_WAITOK | M_ZERO); swhash_mask = n - 1; mtx_init(&swhash_mtx, "swap_pager swhash", NULL, MTX_DEF); } static vm_object_t swap_pager_alloc_init(void *handle, struct ucred *cred, vm_ooffset_t size, vm_ooffset_t offset) { vm_object_t object; if (cred != NULL) { if (!swap_reserve_by_cred(size, cred)) return (NULL); crhold(cred); } object = vm_object_allocate(OBJT_SWAP, OFF_TO_IDX(offset + PAGE_MASK + size)); object->handle = handle; if (cred != NULL) { object->cred = cred; object->charge = size; } object->un_pager.swp.swp_bcount = 0; return (object); } /* * SWAP_PAGER_ALLOC() - allocate a new OBJT_SWAP VM object and instantiate * its metadata structures. * * This routine is called from the mmap and fork code to create a new * OBJT_SWAP object. * * This routine must ensure that no live duplicate is created for * the named object request, which is protected against by * holding the sw_alloc_sx lock in case handle != NULL. */ static vm_object_t swap_pager_alloc(void *handle, vm_ooffset_t size, vm_prot_t prot, vm_ooffset_t offset, struct ucred *cred) { vm_object_t object; if (handle != NULL) { /* * Reference existing named region or allocate new one. There * should not be a race here against swp_pager_meta_build() * as called from vm_page_remove() in regards to the lookup * of the handle. */ sx_xlock(&sw_alloc_sx); object = vm_pager_object_lookup(NOBJLIST(handle), handle); if (object == NULL) { object = swap_pager_alloc_init(handle, cred, size, offset); if (object != NULL) { TAILQ_INSERT_TAIL(NOBJLIST(object->handle), object, pager_object_list); } } sx_xunlock(&sw_alloc_sx); } else { object = swap_pager_alloc_init(handle, cred, size, offset); } return (object); } /* * SWAP_PAGER_DEALLOC() - remove swap metadata from object * * The swap backing for the object is destroyed. The code is * designed such that we can reinstantiate it later, but this * routine is typically called only when the entire object is * about to be destroyed. * * The object must be locked. */ static void swap_pager_dealloc(vm_object_t object) { VM_OBJECT_ASSERT_WLOCKED(object); KASSERT((object->flags & OBJ_DEAD) != 0, ("dealloc of reachable obj")); /* * Remove from list right away so lookups will fail if we block for * pageout completion. */ if (object->handle != NULL) { VM_OBJECT_WUNLOCK(object); sx_xlock(&sw_alloc_sx); TAILQ_REMOVE(NOBJLIST(object->handle), object, pager_object_list); sx_xunlock(&sw_alloc_sx); VM_OBJECT_WLOCK(object); } vm_object_pip_wait(object, "swpdea"); /* * Free all remaining metadata. We only bother to free it from * the swap meta data. We do not attempt to free swapblk's still * associated with vm_page_t's for this object. We do not care * if paging is still in progress on some objects. */ swp_pager_meta_free_all(object); object->handle = NULL; object->type = OBJT_DEAD; } /************************************************************************ * SWAP PAGER BITMAP ROUTINES * ************************************************************************/ /* * SWP_PAGER_GETSWAPSPACE() - allocate raw swap space * * Allocate swap for the requested number of pages. The starting * swap block number (a page index) is returned or SWAPBLK_NONE * if the allocation failed. * * Also has the side effect of advising that somebody made a mistake * when they configured swap and didn't configure enough. * * This routine may not sleep. * * We allocate in round-robin fashion from the configured devices. */ static daddr_t swp_pager_getswapspace(int npages) { daddr_t blk; struct swdevt *sp; int i; blk = SWAPBLK_NONE; mtx_lock(&sw_dev_mtx); sp = swdevhd; for (i = 0; i < nswapdev; i++) { if (sp == NULL) sp = TAILQ_FIRST(&swtailq); if (!(sp->sw_flags & SW_CLOSING)) { blk = blist_alloc(sp->sw_blist, npages); if (blk != SWAPBLK_NONE) { blk += sp->sw_first; sp->sw_used += npages; swap_pager_avail -= npages; swp_sizecheck(); swdevhd = TAILQ_NEXT(sp, sw_list); goto done; } } sp = TAILQ_NEXT(sp, sw_list); } if (swap_pager_full != 2) { printf("swap_pager_getswapspace(%d): failed\n", npages); swap_pager_full = 2; swap_pager_almost_full = 1; } swdevhd = NULL; done: mtx_unlock(&sw_dev_mtx); return (blk); } static int swp_pager_isondev(daddr_t blk, struct swdevt *sp) { return (blk >= sp->sw_first && blk < sp->sw_end); } static void swp_pager_strategy(struct buf *bp) { struct swdevt *sp; mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(sp, &swtailq, sw_list) { if (bp->b_blkno >= sp->sw_first && bp->b_blkno < sp->sw_end) { mtx_unlock(&sw_dev_mtx); if ((sp->sw_flags & SW_UNMAPPED) != 0 && unmapped_buf_allowed) { bp->b_data = unmapped_buf; bp->b_offset = 0; } else { pmap_qenter((vm_offset_t)bp->b_data, &bp->b_pages[0], bp->b_bcount / PAGE_SIZE); } sp->sw_strategy(bp, sp); return; } } panic("Swapdev not found"); } /* * SWP_PAGER_FREESWAPSPACE() - free raw swap space * * This routine returns the specified swap blocks back to the bitmap. * * This routine may not sleep. */ static void swp_pager_freeswapspace(daddr_t blk, int npages) { struct swdevt *sp; mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(sp, &swtailq, sw_list) { if (blk >= sp->sw_first && blk < sp->sw_end) { sp->sw_used -= npages; /* * If we are attempting to stop swapping on * this device, we don't want to mark any * blocks free lest they be reused. */ if ((sp->sw_flags & SW_CLOSING) == 0) { blist_free(sp->sw_blist, blk - sp->sw_first, npages); swap_pager_avail += npages; swp_sizecheck(); } mtx_unlock(&sw_dev_mtx); return; } } panic("Swapdev not found"); } /* * SWAP_PAGER_FREESPACE() - frees swap blocks associated with a page * range within an object. * * This is a globally accessible routine. * * This routine removes swapblk assignments from swap metadata. * * The external callers of this routine typically have already destroyed * or renamed vm_page_t's associated with this range in the object so * we should be ok. * * The object must be locked. */ void swap_pager_freespace(vm_object_t object, vm_pindex_t start, vm_size_t size) { swp_pager_meta_free(object, start, size); } /* * SWAP_PAGER_RESERVE() - reserve swap blocks in object * * Assigns swap blocks to the specified range within the object. The * swap blocks are not zeroed. Any previous swap assignment is destroyed. * * Returns 0 on success, -1 on failure. */ int swap_pager_reserve(vm_object_t object, vm_pindex_t start, vm_size_t size) { int n = 0; daddr_t blk = SWAPBLK_NONE; vm_pindex_t beg = start; /* save start index */ VM_OBJECT_WLOCK(object); while (size) { if (n == 0) { n = BLIST_MAX_ALLOC; while ((blk = swp_pager_getswapspace(n)) == SWAPBLK_NONE) { n >>= 1; if (n == 0) { swp_pager_meta_free(object, beg, start - beg); VM_OBJECT_WUNLOCK(object); return (-1); } } } swp_pager_meta_build(object, start, blk); --size; ++start; ++blk; --n; } swp_pager_meta_free(object, start, n); VM_OBJECT_WUNLOCK(object); return (0); } /* * SWAP_PAGER_COPY() - copy blocks from source pager to destination pager * and destroy the source. * * Copy any valid swapblks from the source to the destination. In * cases where both the source and destination have a valid swapblk, * we keep the destination's. * * This routine is allowed to sleep. It may sleep allocating metadata * indirectly through swp_pager_meta_build() or if paging is still in * progress on the source. * * The source object contains no vm_page_t's (which is just as well) * * The source object is of type OBJT_SWAP. * * The source and destination objects must be locked. * Both object locks may temporarily be released. */ void swap_pager_copy(vm_object_t srcobject, vm_object_t dstobject, vm_pindex_t offset, int destroysource) { vm_pindex_t i; VM_OBJECT_ASSERT_WLOCKED(srcobject); VM_OBJECT_ASSERT_WLOCKED(dstobject); /* * If destroysource is set, we remove the source object from the * swap_pager internal queue now. */ if (destroysource && srcobject->handle != NULL) { vm_object_pip_add(srcobject, 1); VM_OBJECT_WUNLOCK(srcobject); vm_object_pip_add(dstobject, 1); VM_OBJECT_WUNLOCK(dstobject); sx_xlock(&sw_alloc_sx); TAILQ_REMOVE(NOBJLIST(srcobject->handle), srcobject, pager_object_list); sx_xunlock(&sw_alloc_sx); VM_OBJECT_WLOCK(dstobject); vm_object_pip_wakeup(dstobject); VM_OBJECT_WLOCK(srcobject); vm_object_pip_wakeup(srcobject); } /* * transfer source to destination. */ for (i = 0; i < dstobject->size; ++i) { daddr_t dstaddr; /* * Locate (without changing) the swapblk on the destination, * unless it is invalid in which case free it silently, or * if the destination is a resident page, in which case the * source is thrown away. */ dstaddr = swp_pager_meta_ctl(dstobject, i, 0); if (dstaddr == SWAPBLK_NONE) { /* * Destination has no swapblk and is not resident, * copy source. */ daddr_t srcaddr; srcaddr = swp_pager_meta_ctl( srcobject, i + offset, SWM_POP ); if (srcaddr != SWAPBLK_NONE) { /* * swp_pager_meta_build() can sleep. */ vm_object_pip_add(srcobject, 1); VM_OBJECT_WUNLOCK(srcobject); vm_object_pip_add(dstobject, 1); swp_pager_meta_build(dstobject, i, srcaddr); vm_object_pip_wakeup(dstobject); VM_OBJECT_WLOCK(srcobject); vm_object_pip_wakeup(srcobject); } } else { /* * Destination has valid swapblk or it is represented * by a resident page. We destroy the sourceblock. */ swp_pager_meta_ctl(srcobject, i + offset, SWM_FREE); } } /* * Free left over swap blocks in source. * * We have to revert the type to OBJT_DEFAULT so we do not accidentally * double-remove the object from the swap queues. */ if (destroysource) { swp_pager_meta_free_all(srcobject); /* * Reverting the type is not necessary, the caller is going * to destroy srcobject directly, but I'm doing it here * for consistency since we've removed the object from its * queues. */ srcobject->type = OBJT_DEFAULT; } } /* * SWAP_PAGER_HASPAGE() - determine if we have good backing store for * the requested page. * * We determine whether good backing store exists for the requested * page and return TRUE if it does, FALSE if it doesn't. * * If TRUE, we also try to determine how much valid, contiguous backing - * store exists before and after the requested page within a reasonable - * distance. We do not try to restrict it to the swap device stripe - * (that is handled in getpages/putpages). It probably isn't worth - * doing here. + * store exists before and after the requested page. */ static boolean_t -swap_pager_haspage(vm_object_t object, vm_pindex_t pindex, int *before, int *after) +swap_pager_haspage(vm_object_t object, vm_pindex_t pindex, int *before, + int *after) { - daddr_t blk0; + daddr_t blk, blk0; + int i; VM_OBJECT_ASSERT_LOCKED(object); + /* * do we have good backing store at the requested index ? */ blk0 = swp_pager_meta_ctl(object, pindex, 0); - if (blk0 == SWAPBLK_NONE) { if (before) *before = 0; if (after) *after = 0; return (FALSE); } /* * find backwards-looking contiguous good backing store */ if (before != NULL) { - int i; - - for (i = 1; i < (SWB_NPAGES/2); ++i) { - daddr_t blk; - + for (i = 1; i < SWB_NPAGES; i++) { if (i > pindex) break; blk = swp_pager_meta_ctl(object, pindex - i, 0); if (blk != blk0 - i) break; } - *before = (i - 1); + *before = i - 1; } /* * find forward-looking contiguous good backing store */ if (after != NULL) { - int i; - - for (i = 1; i < (SWB_NPAGES/2); ++i) { - daddr_t blk; - + for (i = 1; i < SWB_NPAGES; i++) { blk = swp_pager_meta_ctl(object, pindex + i, 0); if (blk != blk0 + i) break; } - *after = (i - 1); + *after = i - 1; } return (TRUE); } /* * SWAP_PAGER_PAGE_UNSWAPPED() - remove swap backing store related to page * * This removes any associated swap backing store, whether valid or * not, from the page. * * This routine is typically called when a page is made dirty, at * which point any associated swap can be freed. MADV_FREE also * calls us in a special-case situation * * NOTE!!! If the page is clean and the swap was valid, the caller * should make the page dirty before calling this routine. This routine * does NOT change the m->dirty status of the page. Also: MADV_FREE * depends on it. * * This routine may not sleep. * * The object containing the page must be locked. */ static void swap_pager_unswapped(vm_page_t m) { swp_pager_meta_ctl(m->object, m->pindex, SWM_FREE); } /* - * SWAP_PAGER_GETPAGES() - bring pages in from swap + * swap_pager_getpages() - bring pages in from swap * - * Attempt to retrieve (m, count) pages from backing store, but make - * sure we retrieve at least m[reqpage]. We try to load in as large - * a chunk surrounding m[reqpage] as is contiguous in swap and which - * belongs to the same object. + * Attempt to page in the pages in array "m" of length "count". The caller + * may optionally specify that additional pages preceding and succeeding + * the specified range be paged in. The number of such pages is returned + * in the "rbehind" and "rahead" parameters, and they will be in the + * inactive queue upon return. * - * The code is designed for asynchronous operation and - * immediate-notification of 'reqpage' but tends not to be - * used that way. Please do not optimize-out this algorithmic - * feature, I intend to improve on it in the future. - * - * The parent has a single vm_object_pip_add() reference prior to - * calling us and we should return with the same. - * - * The parent has BUSY'd the pages. We should return with 'm' - * left busy, but the others adjusted. + * The pages in "m" must be busied and will remain busied upon return. */ static int swap_pager_getpages(vm_object_t object, vm_page_t *m, int count, int *rbehind, int *rahead) { struct buf *bp; + vm_page_t mpred, msucc, p; + vm_pindex_t pindex; daddr_t blk; + int i, j, reqcount, shift; - /* - * Calculate range to retrieve. The pages have already been assigned - * their swapblks. We require a *contiguous* range but we know it to - * not span devices. If we do not supply it, bad things - * happen. Note that blk, iblk & jblk can be SWAPBLK_NONE, but the - * loops are set up such that the case(s) are handled implicitly. - * - * The swp_*() calls must be made with the object locked. - */ - blk = swp_pager_meta_ctl(m[0]->object, m[0]->pindex, 0); + reqcount = count; - if (blk == SWAPBLK_NONE) + VM_OBJECT_WUNLOCK(object); + bp = getpbuf(&nsw_rcount); + VM_OBJECT_WLOCK(object); + + if (!swap_pager_haspage(object, m[0]->pindex, rbehind, rahead)) { + relpbuf(bp, &nsw_rcount); return (VM_PAGER_FAIL); + } -#ifdef INVARIANTS - for (int i = 0; i < count; i++) - KASSERT(blk + i == - swp_pager_meta_ctl(m[i]->object, m[i]->pindex, 0), - ("%s: range is not contiguous", __func__)); -#endif - /* - * Getpbuf() can sleep. + * Clip the readahead and readbehind ranges to exclude resident pages. */ - VM_OBJECT_WUNLOCK(object); + if (rahead != NULL) { + KASSERT(reqcount - 1 <= *rahead, + ("page count %d extends beyond swap block", reqcount)); + *rahead -= reqcount - 1; + pindex = m[reqcount - 1]->pindex; + msucc = TAILQ_NEXT(m[reqcount - 1], listq); + if (msucc != NULL && msucc->pindex - pindex - 1 < *rahead) + *rahead = msucc->pindex - pindex - 1; + } + if (rbehind != NULL) { + pindex = m[0]->pindex; + mpred = TAILQ_PREV(m[0], pglist, listq); + if (mpred != NULL && pindex - mpred->pindex - 1 < *rbehind) + *rbehind = pindex - mpred->pindex - 1; + } + /* - * Get a swap buffer header to perform the IO + * Allocate readahead and readbehind pages. */ - bp = getpbuf(&nsw_rcount); - bp->b_flags |= B_PAGING; + shift = rbehind != NULL ? *rbehind : 0; + if (shift != 0) { + for (i = 1; i <= shift; i++) { + p = vm_page_alloc(object, m[0]->pindex - i, + VM_ALLOC_NORMAL | VM_ALLOC_IFNOTCACHED); + if (p == NULL) { + /* Shift allocated pages to the left. */ + for (j = 0; j < i - 1; j++) + bp->b_pages[j] = + bp->b_pages[j + shift - i + 1]; + break; + } + bp->b_pages[shift - i] = p; + } + shift = i - 1; + *rbehind = shift; + } + for (i = 0; i < reqcount; i++) + bp->b_pages[i + shift] = m[i]; + if (rahead != NULL) { + for (i = 0; i < *rahead; i++) { + p = vm_page_alloc(object, + m[reqcount - 1]->pindex + i + 1, + VM_ALLOC_NORMAL | VM_ALLOC_IFNOTCACHED); + if (p == NULL) + break; + bp->b_pages[shift + reqcount + i] = p; + } + *rahead = i; + } + if (rbehind != NULL) + count += *rbehind; + if (rahead != NULL) + count += *rahead; + vm_object_pip_add(object, count); + + for (i = 0; i < count; i++) + bp->b_pages[i]->oflags |= VPO_SWAPINPROG; + + pindex = bp->b_pages[0]->pindex; + blk = swp_pager_meta_ctl(object, pindex, 0); + KASSERT(blk != SWAPBLK_NONE, + ("no swap blocking containing %p(%jx)", object, (uintmax_t)pindex)); + + VM_OBJECT_WUNLOCK(object); + + bp->b_flags |= B_PAGING; bp->b_iocmd = BIO_READ; bp->b_iodone = swp_pager_async_iodone; bp->b_rcred = crhold(thread0.td_ucred); bp->b_wcred = crhold(thread0.td_ucred); bp->b_blkno = blk; bp->b_bcount = PAGE_SIZE * count; bp->b_bufsize = PAGE_SIZE * count; bp->b_npages = count; + bp->b_pgbefore = rbehind != NULL ? *rbehind : 0; + bp->b_pgafter = rahead != NULL ? *rahead : 0; - VM_OBJECT_WLOCK(object); - for (int i = 0; i < count; i++) { - bp->b_pages[i] = m[i]; - m[i]->oflags |= VPO_SWAPINPROG; - } - PCPU_INC(cnt.v_swapin); - PCPU_ADD(cnt.v_swappgsin, bp->b_npages); + PCPU_ADD(cnt.v_swappgsin, count); /* - * We still hold the lock on mreq, and our automatic completion routine - * does not remove it. - */ - vm_object_pip_add(object, bp->b_npages); - VM_OBJECT_WUNLOCK(object); - - /* * perform the I/O. NOTE!!! bp cannot be considered valid after * this point because we automatically release it on completion. * Instead, we look at the one page we are interested in which we * still hold a lock on even through the I/O completion. * * The other pages in our m[] array are also released on completion, * so we cannot assume they are valid anymore either. * * NOTE: b_blkno is destroyed by the call to swapdev_strategy */ BUF_KERNPROC(bp); swp_pager_strategy(bp); /* - * wait for the page we want to complete. VPO_SWAPINPROG is always + * Wait for the pages we want to complete. VPO_SWAPINPROG is always * cleared on completion. If an I/O error occurs, SWAPBLK_NONE - * is set in the meta-data. + * is set in the metadata for each page in the request. */ VM_OBJECT_WLOCK(object); while ((m[0]->oflags & VPO_SWAPINPROG) != 0) { m[0]->oflags |= VPO_SWAPSLEEP; PCPU_INC(cnt.v_intrans); if (VM_OBJECT_SLEEP(object, &object->paging_in_progress, PSWP, "swread", hz * 20)) { printf( "swap_pager: indefinite wait buffer: bufobj: %p, blkno: %jd, size: %ld\n", bp->b_bufobj, (intmax_t)bp->b_blkno, bp->b_bcount); } } /* * If we had an unrecoverable read error pages will not be valid. */ - for (int i = 0; i < count; i++) + for (i = 0; i < reqcount; i++) if (m[i]->valid != VM_PAGE_BITS_ALL) return (VM_PAGER_ERROR); - if (rbehind) - *rbehind = 0; - if (rahead) - *rahead = 0; - return (VM_PAGER_OK); /* * A final note: in a low swap situation, we cannot deallocate swap * and mark a page dirty here because the caller is likely to mark * the page clean when we return, causing the page to possibly revert * to all-zero's later. */ } /* * swap_pager_getpages_async(): * * Right now this is emulation of asynchronous operation on top of * swap_pager_getpages(). */ static int swap_pager_getpages_async(vm_object_t object, vm_page_t *m, int count, int *rbehind, int *rahead, pgo_getpages_iodone_t iodone, void *arg) { int r, error; r = swap_pager_getpages(object, m, count, rbehind, rahead); VM_OBJECT_WUNLOCK(object); switch (r) { case VM_PAGER_OK: error = 0; break; case VM_PAGER_ERROR: error = EIO; break; case VM_PAGER_FAIL: error = EINVAL; break; default: panic("unhandled swap_pager_getpages() error %d", r); } (iodone)(arg, m, count, error); VM_OBJECT_WLOCK(object); return (r); } /* * swap_pager_putpages: * * Assign swap (if necessary) and initiate I/O on the specified pages. * * We support both OBJT_DEFAULT and OBJT_SWAP objects. DEFAULT objects * are automatically converted to SWAP objects. * * In a low memory situation we may block in VOP_STRATEGY(), but the new * vm_page reservation system coupled with properly written VFS devices * should ensure that no low-memory deadlock occurs. This is an area * which needs work. * * The parent has N vm_object_pip_add() references prior to * calling us and will remove references for rtvals[] that are * not set to VM_PAGER_PEND. We need to remove the rest on I/O * completion. * * The parent has soft-busy'd the pages it passes us and will unbusy * those whos rtvals[] entry is not set to VM_PAGER_PEND on return. * We need to unbusy the rest on I/O completion. */ static void swap_pager_putpages(vm_object_t object, vm_page_t *m, int count, int flags, int *rtvals) { int i, n; boolean_t sync; if (count && m[0]->object != object) { panic("swap_pager_putpages: object mismatch %p/%p", object, m[0]->object ); } /* * Step 1 * * Turn object into OBJT_SWAP * check for bogus sysops * force sync if not pageout process */ if (object->type != OBJT_SWAP) swp_pager_meta_build(object, 0, SWAPBLK_NONE); VM_OBJECT_WUNLOCK(object); n = 0; if (curproc != pageproc) sync = TRUE; else sync = (flags & VM_PAGER_PUT_SYNC) != 0; /* * Step 2 * * Assign swap blocks and issue I/O. We reallocate swap on the fly. * The page is left dirty until the pageout operation completes * successfully. */ for (i = 0; i < count; i += n) { int j; struct buf *bp; daddr_t blk; /* * Maximum I/O size is limited by a number of factors. */ n = min(BLIST_MAX_ALLOC, count - i); n = min(n, nsw_cluster_max); /* * Get biggest block of swap we can. If we fail, fall * back and try to allocate a smaller block. Don't go * overboard trying to allocate space if it would overly * fragment swap. */ while ( (blk = swp_pager_getswapspace(n)) == SWAPBLK_NONE && n > 4 ) { n >>= 1; } if (blk == SWAPBLK_NONE) { for (j = 0; j < n; ++j) rtvals[i+j] = VM_PAGER_FAIL; continue; } /* * All I/O parameters have been satisfied, build the I/O * request and assign the swap space. */ if (sync == TRUE) { bp = getpbuf(&nsw_wcount_sync); } else { bp = getpbuf(&nsw_wcount_async); bp->b_flags = B_ASYNC; } bp->b_flags |= B_PAGING; bp->b_iocmd = BIO_WRITE; bp->b_rcred = crhold(thread0.td_ucred); bp->b_wcred = crhold(thread0.td_ucred); bp->b_bcount = PAGE_SIZE * n; bp->b_bufsize = PAGE_SIZE * n; bp->b_blkno = blk; VM_OBJECT_WLOCK(object); for (j = 0; j < n; ++j) { vm_page_t mreq = m[i+j]; swp_pager_meta_build( mreq->object, mreq->pindex, blk + j ); vm_page_dirty(mreq); mreq->oflags |= VPO_SWAPINPROG; bp->b_pages[j] = mreq; } VM_OBJECT_WUNLOCK(object); bp->b_npages = n; /* * Must set dirty range for NFS to work. */ bp->b_dirtyoff = 0; bp->b_dirtyend = bp->b_bcount; PCPU_INC(cnt.v_swapout); PCPU_ADD(cnt.v_swappgsout, bp->b_npages); /* * We unconditionally set rtvals[] to VM_PAGER_PEND so that we * can call the async completion routine at the end of a * synchronous I/O operation. Otherwise, our caller would * perform duplicate unbusy and wakeup operations on the page * and object, respectively. */ for (j = 0; j < n; j++) rtvals[i + j] = VM_PAGER_PEND; /* * asynchronous * * NOTE: b_blkno is destroyed by the call to swapdev_strategy */ if (sync == FALSE) { bp->b_iodone = swp_pager_async_iodone; BUF_KERNPROC(bp); swp_pager_strategy(bp); continue; } /* * synchronous * * NOTE: b_blkno is destroyed by the call to swapdev_strategy */ bp->b_iodone = bdone; swp_pager_strategy(bp); /* * Wait for the sync I/O to complete. */ bwait(bp, PVM, "swwrt"); /* * Now that we are through with the bp, we can call the * normal async completion, which frees everything up. */ swp_pager_async_iodone(bp); } VM_OBJECT_WLOCK(object); } /* * swp_pager_async_iodone: * * Completion routine for asynchronous reads and writes from/to swap. * Also called manually by synchronous code to finish up a bp. * * This routine may not sleep. */ static void swp_pager_async_iodone(struct buf *bp) { int i; vm_object_t object = NULL; /* * report error */ if (bp->b_ioflags & BIO_ERROR) { printf( "swap_pager: I/O error - %s failed; blkno %ld," "size %ld, error %d\n", ((bp->b_iocmd == BIO_READ) ? "pagein" : "pageout"), (long)bp->b_blkno, (long)bp->b_bcount, bp->b_error ); } /* * remove the mapping for kernel virtual */ if (buf_mapped(bp)) pmap_qremove((vm_offset_t)bp->b_data, bp->b_npages); else bp->b_data = bp->b_kvabase; if (bp->b_npages) { object = bp->b_pages[0]->object; VM_OBJECT_WLOCK(object); } /* * cleanup pages. If an error occurs writing to swap, we are in * very serious trouble. If it happens to be a disk error, though, * we may be able to recover by reassigning the swap later on. So * in this case we remove the m->swapblk assignment for the page * but do not free it in the rlist. The errornous block(s) are thus * never reallocated as swap. Redirty the page and continue. */ for (i = 0; i < bp->b_npages; ++i) { vm_page_t m = bp->b_pages[i]; m->oflags &= ~VPO_SWAPINPROG; if (m->oflags & VPO_SWAPSLEEP) { m->oflags &= ~VPO_SWAPSLEEP; wakeup(&object->paging_in_progress); } if (bp->b_ioflags & BIO_ERROR) { /* * If an error occurs I'd love to throw the swapblk * away without freeing it back to swapspace, so it * can never be used again. But I can't from an * interrupt. */ if (bp->b_iocmd == BIO_READ) { /* * NOTE: for reads, m->dirty will probably * be overridden by the original caller of * getpages so don't play cute tricks here. */ m->valid = 0; } else { /* * If a write error occurs, reactivate page * so it doesn't clog the inactive list, * then finish the I/O. */ vm_page_dirty(m); vm_page_lock(m); vm_page_activate(m); vm_page_unlock(m); vm_page_sunbusy(m); } } else if (bp->b_iocmd == BIO_READ) { /* * NOTE: for reads, m->dirty will probably be * overridden by the original caller of getpages so * we cannot set them in order to free the underlying * swap in a low-swap situation. I don't think we'd * want to do that anyway, but it was an optimization * that existed in the old swapper for a time before * it got ripped out due to precisely this problem. */ KASSERT(!pmap_page_is_mapped(m), ("swp_pager_async_iodone: page %p is mapped", m)); KASSERT(m->dirty == 0, ("swp_pager_async_iodone: page %p is dirty", m)); + m->valid = VM_PAGE_BITS_ALL; + if (i < bp->b_pgbefore || + i >= bp->b_npages - bp->b_pgafter) + vm_page_readahead_finish(m); } else { /* * For write success, clear the dirty * status, then finish the I/O ( which decrements the * busy count and possibly wakes waiter's up ). */ KASSERT(!pmap_page_is_write_mapped(m), ("swp_pager_async_iodone: page %p is not write" " protected", m)); vm_page_undirty(m); vm_page_sunbusy(m); if (vm_page_count_severe()) { vm_page_lock(m); vm_page_try_to_cache(m); vm_page_unlock(m); } } } /* * adjust pip. NOTE: the original parent may still have its own * pip refs on the object. */ if (object != NULL) { vm_object_pip_wakeupn(object, bp->b_npages); VM_OBJECT_WUNLOCK(object); } /* * swapdev_strategy() manually sets b_vp and b_bufobj before calling * bstrategy(). Set them back to NULL now we're done with it, or we'll * trigger a KASSERT in relpbuf(). */ if (bp->b_vp) { bp->b_vp = NULL; bp->b_bufobj = NULL; } /* * release the physical I/O buffer */ relpbuf( bp, ((bp->b_iocmd == BIO_READ) ? &nsw_rcount : ((bp->b_flags & B_ASYNC) ? &nsw_wcount_async : &nsw_wcount_sync ) ) ); } /* * swap_pager_isswapped: * * Return 1 if at least one page in the given object is paged * out to the given swap device. * * This routine may not sleep. */ int swap_pager_isswapped(vm_object_t object, struct swdevt *sp) { daddr_t index = 0; int bcount; int i; VM_OBJECT_ASSERT_WLOCKED(object); if (object->type != OBJT_SWAP) return (0); mtx_lock(&swhash_mtx); for (bcount = 0; bcount < object->un_pager.swp.swp_bcount; bcount++) { struct swblock *swap; if ((swap = *swp_pager_hash(object, index)) != NULL) { for (i = 0; i < SWAP_META_PAGES; ++i) { if (swp_pager_isondev(swap->swb_pages[i], sp)) { mtx_unlock(&swhash_mtx); return (1); } } } index += SWAP_META_PAGES; } mtx_unlock(&swhash_mtx); return (0); } /* * SWP_PAGER_FORCE_PAGEIN() - force a swap block to be paged in * * This routine dissociates the page at the given index within a * swap block from its backing store, paging it in if necessary. * If the page is paged in, it is placed in the inactive queue, * since it had its backing store ripped out from under it. * We also attempt to swap in all other pages in the swap block, * we only guarantee that the one at the specified index is * paged in. * * XXX - The code to page the whole block in doesn't work, so we * revert to the one-by-one behavior for now. Sigh. */ static inline void swp_pager_force_pagein(vm_object_t object, vm_pindex_t pindex) { vm_page_t m; vm_object_pip_add(object, 1); m = vm_page_grab(object, pindex, VM_ALLOC_NORMAL); if (m->valid == VM_PAGE_BITS_ALL) { vm_object_pip_wakeup(object); vm_page_dirty(m); vm_page_lock(m); vm_page_activate(m); vm_page_unlock(m); vm_page_xunbusy(m); vm_pager_page_unswapped(m); return; } if (swap_pager_getpages(object, &m, 1, NULL, NULL) != VM_PAGER_OK) panic("swap_pager_force_pagein: read from swap failed");/*XXX*/ vm_object_pip_wakeup(object); vm_page_dirty(m); vm_page_lock(m); vm_page_deactivate(m); vm_page_unlock(m); vm_page_xunbusy(m); vm_pager_page_unswapped(m); } /* * swap_pager_swapoff: * * Page in all of the pages that have been paged out to the * given device. The corresponding blocks in the bitmap must be * marked as allocated and the device must be flagged SW_CLOSING. * There may be no processes swapped out to the device. * * This routine may block. */ static void swap_pager_swapoff(struct swdevt *sp) { struct swblock *swap; + vm_object_t locked_obj, object; + vm_pindex_t pindex; int i, j, retries; sx_assert(&swdev_syscall_lock, SA_XLOCKED); retries = 0; + locked_obj = NULL; full_rescan: mtx_lock(&swhash_mtx); for (i = 0; i <= swhash_mask; i++) { /* '<=' is correct here */ restart: for (swap = swhash[i]; swap != NULL; swap = swap->swb_hnext) { - vm_object_t object = swap->swb_object; - vm_pindex_t pindex = swap->swb_index; + object = swap->swb_object; + pindex = swap->swb_index; for (j = 0; j < SWAP_META_PAGES; ++j) { - if (swp_pager_isondev(swap->swb_pages[j], sp)) { - /* avoid deadlock */ + if (!swp_pager_isondev(swap->swb_pages[j], sp)) + continue; + if (locked_obj != object) { + if (locked_obj != NULL) + VM_OBJECT_WUNLOCK(locked_obj); + locked_obj = object; if (!VM_OBJECT_TRYWLOCK(object)) { - break; - } else { mtx_unlock(&swhash_mtx); - swp_pager_force_pagein(object, - pindex + j); - VM_OBJECT_WUNLOCK(object); + /* Depends on type-stability. */ + VM_OBJECT_WLOCK(object); mtx_lock(&swhash_mtx); goto restart; } } + MPASS(locked_obj == object); + mtx_unlock(&swhash_mtx); + swp_pager_force_pagein(object, pindex + j); + mtx_lock(&swhash_mtx); + goto restart; } } } mtx_unlock(&swhash_mtx); + if (locked_obj != NULL) { + VM_OBJECT_WUNLOCK(locked_obj); + locked_obj = NULL; + } if (sp->sw_used) { /* * Objects may be locked or paging to the device being * removed, so we will miss their pages and need to * make another pass. We have marked this device as * SW_CLOSING, so the activity should finish soon. */ retries++; if (retries > 100) { panic("swapoff: failed to locate %d swap blocks", sp->sw_used); } pause("swpoff", hz / 20); goto full_rescan; } } /************************************************************************ * SWAP META DATA * ************************************************************************ * * These routines manipulate the swap metadata stored in the * OBJT_SWAP object. * * Swap metadata is implemented with a global hash and not directly * linked into the object. Instead the object simply contains * appropriate tracking counters. */ /* * SWP_PAGER_META_BUILD() - add swap block to swap meta data for object * * We first convert the object to a swap object if it is a default * object. * * The specified swapblk is added to the object's swap metadata. If * the swapblk is not valid, it is freed instead. Any previously * assigned swapblk is freed. */ static void swp_pager_meta_build(vm_object_t object, vm_pindex_t pindex, daddr_t swapblk) { static volatile int exhausted; struct swblock *swap; struct swblock **pswap; int idx; VM_OBJECT_ASSERT_WLOCKED(object); /* * Convert default object to swap object if necessary */ if (object->type != OBJT_SWAP) { object->type = OBJT_SWAP; object->un_pager.swp.swp_bcount = 0; KASSERT(object->handle == NULL, ("default pager with handle")); } /* * Locate hash entry. If not found create, but if we aren't adding * anything just return. If we run out of space in the map we wait * and, since the hash table may have changed, retry. */ retry: mtx_lock(&swhash_mtx); pswap = swp_pager_hash(object, pindex); if ((swap = *pswap) == NULL) { int i; if (swapblk == SWAPBLK_NONE) goto done; swap = *pswap = uma_zalloc(swap_zone, M_NOWAIT | (curproc == pageproc ? M_USE_RESERVE : 0)); if (swap == NULL) { mtx_unlock(&swhash_mtx); VM_OBJECT_WUNLOCK(object); if (uma_zone_exhausted(swap_zone)) { if (atomic_cmpset_int(&exhausted, 0, 1)) printf("swap zone exhausted, " "increase kern.maxswzone\n"); vm_pageout_oom(VM_OOM_SWAPZ); pause("swzonex", 10); } else VM_WAIT; VM_OBJECT_WLOCK(object); goto retry; } if (atomic_cmpset_int(&exhausted, 1, 0)) printf("swap zone ok\n"); swap->swb_hnext = NULL; swap->swb_object = object; swap->swb_index = pindex & ~(vm_pindex_t)SWAP_META_MASK; swap->swb_count = 0; ++object->un_pager.swp.swp_bcount; for (i = 0; i < SWAP_META_PAGES; ++i) swap->swb_pages[i] = SWAPBLK_NONE; } /* * Delete prior contents of metadata */ idx = pindex & SWAP_META_MASK; if (swap->swb_pages[idx] != SWAPBLK_NONE) { swp_pager_freeswapspace(swap->swb_pages[idx], 1); --swap->swb_count; } /* * Enter block into metadata */ swap->swb_pages[idx] = swapblk; if (swapblk != SWAPBLK_NONE) ++swap->swb_count; done: mtx_unlock(&swhash_mtx); } /* * SWP_PAGER_META_FREE() - free a range of blocks in the object's swap metadata * * The requested range of blocks is freed, with any associated swap * returned to the swap bitmap. * * This routine will free swap metadata structures as they are cleaned * out. This routine does *NOT* operate on swap metadata associated * with resident pages. */ static void swp_pager_meta_free(vm_object_t object, vm_pindex_t index, daddr_t count) { VM_OBJECT_ASSERT_LOCKED(object); if (object->type != OBJT_SWAP) return; while (count > 0) { struct swblock **pswap; struct swblock *swap; mtx_lock(&swhash_mtx); pswap = swp_pager_hash(object, index); if ((swap = *pswap) != NULL) { daddr_t v = swap->swb_pages[index & SWAP_META_MASK]; if (v != SWAPBLK_NONE) { swp_pager_freeswapspace(v, 1); swap->swb_pages[index & SWAP_META_MASK] = SWAPBLK_NONE; if (--swap->swb_count == 0) { *pswap = swap->swb_hnext; uma_zfree(swap_zone, swap); --object->un_pager.swp.swp_bcount; } } --count; ++index; } else { int n = SWAP_META_PAGES - (index & SWAP_META_MASK); count -= n; index += n; } mtx_unlock(&swhash_mtx); } } /* * SWP_PAGER_META_FREE_ALL() - destroy all swap metadata associated with object * * This routine locates and destroys all swap metadata associated with * an object. */ static void swp_pager_meta_free_all(vm_object_t object) { daddr_t index = 0; VM_OBJECT_ASSERT_WLOCKED(object); if (object->type != OBJT_SWAP) return; while (object->un_pager.swp.swp_bcount) { struct swblock **pswap; struct swblock *swap; mtx_lock(&swhash_mtx); pswap = swp_pager_hash(object, index); if ((swap = *pswap) != NULL) { int i; for (i = 0; i < SWAP_META_PAGES; ++i) { daddr_t v = swap->swb_pages[i]; if (v != SWAPBLK_NONE) { --swap->swb_count; swp_pager_freeswapspace(v, 1); } } if (swap->swb_count != 0) panic("swap_pager_meta_free_all: swb_count != 0"); *pswap = swap->swb_hnext; uma_zfree(swap_zone, swap); --object->un_pager.swp.swp_bcount; } mtx_unlock(&swhash_mtx); index += SWAP_META_PAGES; } } /* * SWP_PAGER_METACTL() - misc control of swap and vm_page_t meta data. * * This routine is capable of looking up, popping, or freeing * swapblk assignments in the swap meta data or in the vm_page_t. * The routine typically returns the swapblk being looked-up, or popped, * or SWAPBLK_NONE if the block was freed, or SWAPBLK_NONE if the block * was invalid. This routine will automatically free any invalid * meta-data swapblks. * * It is not possible to store invalid swapblks in the swap meta data * (other then a literal 'SWAPBLK_NONE'), so we don't bother checking. * * When acting on a busy resident page and paging is in progress, we * have to wait until paging is complete but otherwise can act on the * busy page. * * SWM_FREE remove and free swap block from metadata * SWM_POP remove from meta data but do not free.. pop it out */ static daddr_t swp_pager_meta_ctl(vm_object_t object, vm_pindex_t pindex, int flags) { struct swblock **pswap; struct swblock *swap; daddr_t r1; int idx; VM_OBJECT_ASSERT_LOCKED(object); /* * The meta data only exists of the object is OBJT_SWAP * and even then might not be allocated yet. */ if (object->type != OBJT_SWAP) return (SWAPBLK_NONE); r1 = SWAPBLK_NONE; mtx_lock(&swhash_mtx); pswap = swp_pager_hash(object, pindex); if ((swap = *pswap) != NULL) { idx = pindex & SWAP_META_MASK; r1 = swap->swb_pages[idx]; if (r1 != SWAPBLK_NONE) { if (flags & SWM_FREE) { swp_pager_freeswapspace(r1, 1); r1 = SWAPBLK_NONE; } if (flags & (SWM_FREE|SWM_POP)) { swap->swb_pages[idx] = SWAPBLK_NONE; if (--swap->swb_count == 0) { *pswap = swap->swb_hnext; uma_zfree(swap_zone, swap); --object->un_pager.swp.swp_bcount; } } } } mtx_unlock(&swhash_mtx); return (r1); } /* * System call swapon(name) enables swapping on device name, * which must be in the swdevsw. Return EBUSY * if already swapping on this device. */ #ifndef _SYS_SYSPROTO_H_ struct swapon_args { char *name; }; #endif /* * MPSAFE */ /* ARGSUSED */ int sys_swapon(struct thread *td, struct swapon_args *uap) { struct vattr attr; struct vnode *vp; struct nameidata nd; int error; error = priv_check(td, PRIV_SWAPON); if (error) return (error); sx_xlock(&swdev_syscall_lock); /* * Swap metadata may not fit in the KVM if we have physical * memory of >1GB. */ if (swap_zone == NULL) { error = ENOMEM; goto done; } NDINIT(&nd, LOOKUP, ISOPEN | FOLLOW | AUDITVNODE1, UIO_USERSPACE, uap->name, td); error = namei(&nd); if (error) goto done; NDFREE(&nd, NDF_ONLY_PNBUF); vp = nd.ni_vp; if (vn_isdisk(vp, &error)) { error = swapongeom(vp); } else if (vp->v_type == VREG && (vp->v_mount->mnt_vfc->vfc_flags & VFCF_NETWORK) != 0 && (error = VOP_GETATTR(vp, &attr, td->td_ucred)) == 0) { /* * Allow direct swapping to NFS regular files in the same * way that nfs_mountroot() sets up diskless swapping. */ error = swaponvp(td, vp, attr.va_size / DEV_BSIZE); } if (error) vrele(vp); done: sx_xunlock(&swdev_syscall_lock); return (error); } /* * Check that the total amount of swap currently configured does not * exceed half the theoretical maximum. If it does, print a warning * message and return -1; otherwise, return 0. */ static int swapon_check_swzone(unsigned long npages) { unsigned long maxpages; /* absolute maximum we can handle assuming 100% efficiency */ maxpages = uma_zone_get_max(swap_zone) * SWAP_META_PAGES; /* recommend using no more than half that amount */ if (npages > maxpages / 2) { printf("warning: total configured swap (%lu pages) " "exceeds maximum recommended amount (%lu pages).\n", npages, maxpages / 2); printf("warning: increase kern.maxswzone " "or reduce amount of swap.\n"); return (-1); } return (0); } static void swaponsomething(struct vnode *vp, void *id, u_long nblks, sw_strategy_t *strategy, sw_close_t *close, dev_t dev, int flags) { struct swdevt *sp, *tsp; swblk_t dvbase; u_long mblocks; /* * nblks is in DEV_BSIZE'd chunks, convert to PAGE_SIZE'd chunks. * First chop nblks off to page-align it, then convert. * * sw->sw_nblks is in page-sized chunks now too. */ nblks &= ~(ctodb(1) - 1); nblks = dbtoc(nblks); /* * If we go beyond this, we get overflows in the radix * tree bitmap code. */ mblocks = 0x40000000 / BLIST_META_RADIX; if (nblks > mblocks) { printf( "WARNING: reducing swap size to maximum of %luMB per unit\n", mblocks / 1024 / 1024 * PAGE_SIZE); nblks = mblocks; } sp = malloc(sizeof *sp, M_VMPGDATA, M_WAITOK | M_ZERO); sp->sw_vp = vp; sp->sw_id = id; sp->sw_dev = dev; sp->sw_flags = 0; sp->sw_nblks = nblks; sp->sw_used = 0; sp->sw_strategy = strategy; sp->sw_close = close; sp->sw_flags = flags; sp->sw_blist = blist_create(nblks, M_WAITOK); /* * Do not free the first two block in order to avoid overwriting * any bsd label at the front of the partition */ blist_free(sp->sw_blist, 2, nblks - 2); dvbase = 0; mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(tsp, &swtailq, sw_list) { if (tsp->sw_end >= dvbase) { /* * We put one uncovered page between the devices * in order to definitively prevent any cross-device * I/O requests */ dvbase = tsp->sw_end + 1; } } sp->sw_first = dvbase; sp->sw_end = dvbase + nblks; TAILQ_INSERT_TAIL(&swtailq, sp, sw_list); nswapdev++; swap_pager_avail += nblks; swap_total += (vm_ooffset_t)nblks * PAGE_SIZE; swapon_check_swzone(swap_total / PAGE_SIZE); swp_sizecheck(); mtx_unlock(&sw_dev_mtx); } /* * SYSCALL: swapoff(devname) * * Disable swapping on the given device. * * XXX: Badly designed system call: it should use a device index * rather than filename as specification. We keep sw_vp around * only to make this work. */ #ifndef _SYS_SYSPROTO_H_ struct swapoff_args { char *name; }; #endif /* * MPSAFE */ /* ARGSUSED */ int sys_swapoff(struct thread *td, struct swapoff_args *uap) { struct vnode *vp; struct nameidata nd; struct swdevt *sp; int error; error = priv_check(td, PRIV_SWAPOFF); if (error) return (error); sx_xlock(&swdev_syscall_lock); NDINIT(&nd, LOOKUP, FOLLOW | AUDITVNODE1, UIO_USERSPACE, uap->name, td); error = namei(&nd); if (error) goto done; NDFREE(&nd, NDF_ONLY_PNBUF); vp = nd.ni_vp; mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(sp, &swtailq, sw_list) { if (sp->sw_vp == vp) break; } mtx_unlock(&sw_dev_mtx); if (sp == NULL) { error = EINVAL; goto done; } error = swapoff_one(sp, td->td_ucred); done: sx_xunlock(&swdev_syscall_lock); return (error); } static int swapoff_one(struct swdevt *sp, struct ucred *cred) { u_long nblks, dvbase; #ifdef MAC int error; #endif sx_assert(&swdev_syscall_lock, SA_XLOCKED); #ifdef MAC (void) vn_lock(sp->sw_vp, LK_EXCLUSIVE | LK_RETRY); error = mac_system_check_swapoff(cred, sp->sw_vp); (void) VOP_UNLOCK(sp->sw_vp, 0); if (error != 0) return (error); #endif nblks = sp->sw_nblks; /* * We can turn off this swap device safely only if the * available virtual memory in the system will fit the amount * of data we will have to page back in, plus an epsilon so * the system doesn't become critically low on swap space. */ if (vm_cnt.v_free_count + vm_cnt.v_cache_count + swap_pager_avail < nblks + nswap_lowat) { return (ENOMEM); } /* * Prevent further allocations on this device. */ mtx_lock(&sw_dev_mtx); sp->sw_flags |= SW_CLOSING; for (dvbase = 0; dvbase < sp->sw_end; dvbase += dmmax) { swap_pager_avail -= blist_fill(sp->sw_blist, dvbase, dmmax); } swap_total -= (vm_ooffset_t)nblks * PAGE_SIZE; mtx_unlock(&sw_dev_mtx); /* * Page in the contents of the device and close it. */ swap_pager_swapoff(sp); sp->sw_close(curthread, sp); mtx_lock(&sw_dev_mtx); sp->sw_id = NULL; TAILQ_REMOVE(&swtailq, sp, sw_list); nswapdev--; if (nswapdev == 0) { swap_pager_full = 2; swap_pager_almost_full = 1; } if (swdevhd == sp) swdevhd = NULL; mtx_unlock(&sw_dev_mtx); blist_destroy(sp->sw_blist); free(sp, M_VMPGDATA); return (0); } void swapoff_all(void) { struct swdevt *sp, *spt; const char *devname; int error; sx_xlock(&swdev_syscall_lock); mtx_lock(&sw_dev_mtx); TAILQ_FOREACH_SAFE(sp, &swtailq, sw_list, spt) { mtx_unlock(&sw_dev_mtx); if (vn_isdisk(sp->sw_vp, NULL)) devname = devtoname(sp->sw_vp->v_rdev); else devname = "[file]"; error = swapoff_one(sp, thread0.td_ucred); if (error != 0) { printf("Cannot remove swap device %s (error=%d), " "skipping.\n", devname, error); } else if (bootverbose) { printf("Swap device %s removed.\n", devname); } mtx_lock(&sw_dev_mtx); } mtx_unlock(&sw_dev_mtx); sx_xunlock(&swdev_syscall_lock); } void swap_pager_status(int *total, int *used) { struct swdevt *sp; *total = 0; *used = 0; mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(sp, &swtailq, sw_list) { *total += sp->sw_nblks; *used += sp->sw_used; } mtx_unlock(&sw_dev_mtx); } int swap_dev_info(int name, struct xswdev *xs, char *devname, size_t len) { struct swdevt *sp; const char *tmp_devname; int error, n; n = 0; error = ENOENT; mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(sp, &swtailq, sw_list) { if (n != name) { n++; continue; } xs->xsw_version = XSWDEV_VERSION; xs->xsw_dev = sp->sw_dev; xs->xsw_flags = sp->sw_flags; xs->xsw_nblks = sp->sw_nblks; xs->xsw_used = sp->sw_used; if (devname != NULL) { if (vn_isdisk(sp->sw_vp, NULL)) tmp_devname = devtoname(sp->sw_vp->v_rdev); else tmp_devname = "[file]"; strncpy(devname, tmp_devname, len); } error = 0; break; } mtx_unlock(&sw_dev_mtx); return (error); } static int sysctl_vm_swap_info(SYSCTL_HANDLER_ARGS) { struct xswdev xs; int error; if (arg2 != 1) /* name length */ return (EINVAL); error = swap_dev_info(*(int *)arg1, &xs, NULL, 0); if (error != 0) return (error); error = SYSCTL_OUT(req, &xs, sizeof(xs)); return (error); } SYSCTL_INT(_vm, OID_AUTO, nswapdev, CTLFLAG_RD, &nswapdev, 0, "Number of swap devices"); SYSCTL_NODE(_vm, OID_AUTO, swap_info, CTLFLAG_RD | CTLFLAG_MPSAFE, sysctl_vm_swap_info, "Swap statistics by device"); /* * vmspace_swap_count() - count the approximate swap usage in pages for a * vmspace. * * The map must be locked. * * Swap usage is determined by taking the proportional swap used by * VM objects backing the VM map. To make up for fractional losses, * if the VM object has any swap use at all the associated map entries * count for at least 1 swap page. */ long vmspace_swap_count(struct vmspace *vmspace) { vm_map_t map; vm_map_entry_t cur; vm_object_t object; long count, n; map = &vmspace->vm_map; count = 0; for (cur = map->header.next; cur != &map->header; cur = cur->next) { if ((cur->eflags & MAP_ENTRY_IS_SUB_MAP) == 0 && (object = cur->object.vm_object) != NULL) { VM_OBJECT_WLOCK(object); if (object->type == OBJT_SWAP && object->un_pager.swp.swp_bcount != 0) { n = (cur->end - cur->start) / PAGE_SIZE; count += object->un_pager.swp.swp_bcount * SWAP_META_PAGES * n / object->size + 1; } VM_OBJECT_WUNLOCK(object); } } return (count); } /* * GEOM backend * * Swapping onto disk devices. * */ static g_orphan_t swapgeom_orphan; static struct g_class g_swap_class = { .name = "SWAP", .version = G_VERSION, .orphan = swapgeom_orphan, }; DECLARE_GEOM_CLASS(g_swap_class, g_class); static void swapgeom_close_ev(void *arg, int flags) { struct g_consumer *cp; cp = arg; g_access(cp, -1, -1, 0); g_detach(cp); g_destroy_consumer(cp); } /* * Add a reference to the g_consumer for an inflight transaction. */ static void swapgeom_acquire(struct g_consumer *cp) { mtx_assert(&sw_dev_mtx, MA_OWNED); cp->index++; } /* * Remove a reference from the g_consumer. Post a close event if all * references go away, since the function might be called from the * biodone context. */ static void swapgeom_release(struct g_consumer *cp, struct swdevt *sp) { mtx_assert(&sw_dev_mtx, MA_OWNED); cp->index--; if (cp->index == 0) { if (g_post_event(swapgeom_close_ev, cp, M_NOWAIT, NULL) == 0) sp->sw_id = NULL; } } static void swapgeom_done(struct bio *bp2) { struct swdevt *sp; struct buf *bp; struct g_consumer *cp; bp = bp2->bio_caller2; cp = bp2->bio_from; bp->b_ioflags = bp2->bio_flags; if (bp2->bio_error) bp->b_ioflags |= BIO_ERROR; bp->b_resid = bp->b_bcount - bp2->bio_completed; bp->b_error = bp2->bio_error; bufdone(bp); sp = bp2->bio_caller1; mtx_lock(&sw_dev_mtx); swapgeom_release(cp, sp); mtx_unlock(&sw_dev_mtx); g_destroy_bio(bp2); } static void swapgeom_strategy(struct buf *bp, struct swdevt *sp) { struct bio *bio; struct g_consumer *cp; mtx_lock(&sw_dev_mtx); cp = sp->sw_id; if (cp == NULL) { mtx_unlock(&sw_dev_mtx); bp->b_error = ENXIO; bp->b_ioflags |= BIO_ERROR; bufdone(bp); return; } swapgeom_acquire(cp); mtx_unlock(&sw_dev_mtx); if (bp->b_iocmd == BIO_WRITE) bio = g_new_bio(); else bio = g_alloc_bio(); if (bio == NULL) { mtx_lock(&sw_dev_mtx); swapgeom_release(cp, sp); mtx_unlock(&sw_dev_mtx); bp->b_error = ENOMEM; bp->b_ioflags |= BIO_ERROR; bufdone(bp); return; } bio->bio_caller1 = sp; bio->bio_caller2 = bp; bio->bio_cmd = bp->b_iocmd; bio->bio_offset = (bp->b_blkno - sp->sw_first) * PAGE_SIZE; bio->bio_length = bp->b_bcount; bio->bio_done = swapgeom_done; if (!buf_mapped(bp)) { bio->bio_ma = bp->b_pages; bio->bio_data = unmapped_buf; bio->bio_ma_offset = (vm_offset_t)bp->b_offset & PAGE_MASK; bio->bio_ma_n = bp->b_npages; bio->bio_flags |= BIO_UNMAPPED; } else { bio->bio_data = bp->b_data; bio->bio_ma = NULL; } g_io_request(bio, cp); return; } static void swapgeom_orphan(struct g_consumer *cp) { struct swdevt *sp; int destroy; mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(sp, &swtailq, sw_list) { if (sp->sw_id == cp) { sp->sw_flags |= SW_CLOSING; break; } } /* * Drop reference we were created with. Do directly since we're in a * special context where we don't have to queue the call to * swapgeom_close_ev(). */ cp->index--; destroy = ((sp != NULL) && (cp->index == 0)); if (destroy) sp->sw_id = NULL; mtx_unlock(&sw_dev_mtx); if (destroy) swapgeom_close_ev(cp, 0); } static void swapgeom_close(struct thread *td, struct swdevt *sw) { struct g_consumer *cp; mtx_lock(&sw_dev_mtx); cp = sw->sw_id; sw->sw_id = NULL; mtx_unlock(&sw_dev_mtx); /* * swapgeom_close() may be called from the biodone context, * where we cannot perform topology changes. Delegate the * work to the events thread. */ if (cp != NULL) g_waitfor_event(swapgeom_close_ev, cp, M_WAITOK, NULL); } static int swapongeom_locked(struct cdev *dev, struct vnode *vp) { struct g_provider *pp; struct g_consumer *cp; static struct g_geom *gp; struct swdevt *sp; u_long nblks; int error; pp = g_dev_getprovider(dev); if (pp == NULL) return (ENODEV); mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(sp, &swtailq, sw_list) { cp = sp->sw_id; if (cp != NULL && cp->provider == pp) { mtx_unlock(&sw_dev_mtx); return (EBUSY); } } mtx_unlock(&sw_dev_mtx); if (gp == NULL) gp = g_new_geomf(&g_swap_class, "swap"); cp = g_new_consumer(gp); cp->index = 1; /* Number of active I/Os, plus one for being active. */ cp->flags |= G_CF_DIRECT_SEND | G_CF_DIRECT_RECEIVE; g_attach(cp, pp); /* * XXX: Every time you think you can improve the margin for * footshooting, somebody depends on the ability to do so: * savecore(8) wants to write to our swapdev so we cannot * set an exclusive count :-( */ error = g_access(cp, 1, 1, 0); if (error != 0) { g_detach(cp); g_destroy_consumer(cp); return (error); } nblks = pp->mediasize / DEV_BSIZE; swaponsomething(vp, cp, nblks, swapgeom_strategy, swapgeom_close, dev2udev(dev), (pp->flags & G_PF_ACCEPT_UNMAPPED) != 0 ? SW_UNMAPPED : 0); return (0); } static int swapongeom(struct vnode *vp) { int error; vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); if (vp->v_type != VCHR || (vp->v_iflag & VI_DOOMED) != 0) { error = ENOENT; } else { g_topology_lock(); error = swapongeom_locked(vp->v_rdev, vp); g_topology_unlock(); } VOP_UNLOCK(vp, 0); return (error); } /* * VNODE backend * * This is used mainly for network filesystem (read: probably only tested * with NFS) swapfiles. * */ static void swapdev_strategy(struct buf *bp, struct swdevt *sp) { struct vnode *vp2; bp->b_blkno = ctodb(bp->b_blkno - sp->sw_first); vp2 = sp->sw_id; vhold(vp2); if (bp->b_iocmd == BIO_WRITE) { if (bp->b_bufobj) bufobj_wdrop(bp->b_bufobj); bufobj_wref(&vp2->v_bufobj); } if (bp->b_bufobj != &vp2->v_bufobj) bp->b_bufobj = &vp2->v_bufobj; bp->b_vp = vp2; bp->b_iooffset = dbtob(bp->b_blkno); bstrategy(bp); return; } static void swapdev_close(struct thread *td, struct swdevt *sp) { VOP_CLOSE(sp->sw_vp, FREAD | FWRITE, td->td_ucred, td); vrele(sp->sw_vp); } static int swaponvp(struct thread *td, struct vnode *vp, u_long nblks) { struct swdevt *sp; int error; if (nblks == 0) return (ENXIO); mtx_lock(&sw_dev_mtx); TAILQ_FOREACH(sp, &swtailq, sw_list) { if (sp->sw_id == vp) { mtx_unlock(&sw_dev_mtx); return (EBUSY); } } mtx_unlock(&sw_dev_mtx); (void) vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); #ifdef MAC error = mac_system_check_swapon(td->td_ucred, vp); if (error == 0) #endif error = VOP_OPEN(vp, FREAD | FWRITE, td->td_ucred, td, NULL); (void) VOP_UNLOCK(vp, 0); if (error) return (error); swaponsomething(vp, vp, nblks, swapdev_strategy, swapdev_close, NODEV, 0); return (0); } static int sysctl_swap_async_max(SYSCTL_HANDLER_ARGS) { int error, new, n; new = nsw_wcount_async_max; error = sysctl_handle_int(oidp, &new, 0, req); if (error != 0 || req->newptr == NULL) return (error); if (new > nswbuf / 2 || new < 1) return (EINVAL); mtx_lock(&pbuf_mtx); while (nsw_wcount_async_max != new) { /* * Adjust difference. If the current async count is too low, * we will need to sqeeze our update slowly in. Sleep with a * higher priority than getpbuf() to finish faster. */ n = new - nsw_wcount_async_max; if (nsw_wcount_async + n >= 0) { nsw_wcount_async += n; nsw_wcount_async_max += n; wakeup(&nsw_wcount_async); } else { nsw_wcount_async_max -= nsw_wcount_async; nsw_wcount_async = 0; msleep(&nsw_wcount_async, &pbuf_mtx, PSWP, "swpsysctl", 0); } } mtx_unlock(&pbuf_mtx); return (0); } Index: projects/netbsd-tests-update-12/sys/vm/vm_page.c =================================================================== --- projects/netbsd-tests-update-12/sys/vm/vm_page.c (revision 305171) +++ projects/netbsd-tests-update-12/sys/vm/vm_page.c (revision 305172) @@ -1,3974 +1,3974 @@ /*- * Copyright (c) 1991 Regents of the University of California. * All rights reserved. * Copyright (c) 1998 Matthew Dillon. All Rights Reserved. * * This code is derived from software contributed to Berkeley by * The Mach Operating System project at Carnegie-Mellon University. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * from: @(#)vm_page.c 7.4 (Berkeley) 5/7/91 */ /*- * Copyright (c) 1987, 1990 Carnegie-Mellon University. * All rights reserved. * * Authors: Avadis Tevanian, Jr., Michael Wayne Young * * Permission to use, copy, modify and distribute this software and * its documentation is hereby granted, provided that both the copyright * notice and this permission notice appear in all copies of the * software, derivative works or modified versions, and any portions * thereof, and that both notices appear in supporting documentation. * * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. * * Carnegie Mellon requests users of this software to return to * * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU * School of Computer Science * Carnegie Mellon University * Pittsburgh PA 15213-3890 * * any improvements or extensions that they make and grant Carnegie the * rights to redistribute these changes. */ /* * GENERAL RULES ON VM_PAGE MANIPULATION * * - A page queue lock is required when adding or removing a page from a * page queue regardless of other locks or the busy state of a page. * * * In general, no thread besides the page daemon can acquire or * hold more than one page queue lock at a time. * * * The page daemon can acquire and hold any pair of page queue * locks in any order. * * - The object lock is required when inserting or removing * pages from an object (vm_page_insert() or vm_page_remove()). * */ /* * Resident memory management module. */ #include __FBSDID("$FreeBSD$"); #include "opt_vm.h" #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include /* * Associated with page of user-allocatable memory is a * page structure. */ struct vm_domain vm_dom[MAXMEMDOM]; struct mtx_padalign vm_page_queue_free_mtx; struct mtx_padalign pa_lock[PA_LOCK_COUNT]; vm_page_t vm_page_array; long vm_page_array_size; long first_page; int vm_page_zero_count; static int boot_pages = UMA_BOOT_PAGES; SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, &boot_pages, 0, "number of pages allocated for bootstrapping the VM system"); static int pa_tryrelock_restart; SYSCTL_INT(_vm, OID_AUTO, tryrelock_restart, CTLFLAG_RD, &pa_tryrelock_restart, 0, "Number of tryrelock restarts"); static TAILQ_HEAD(, vm_page) blacklist_head; static int sysctl_vm_page_blacklist(SYSCTL_HANDLER_ARGS); SYSCTL_PROC(_vm, OID_AUTO, page_blacklist, CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, 0, sysctl_vm_page_blacklist, "A", "Blacklist pages"); /* Is the page daemon waiting for free pages? */ static int vm_pageout_pages_needed; static uma_zone_t fakepg_zone; static struct vnode *vm_page_alloc_init(vm_page_t m); static void vm_page_cache_turn_free(vm_page_t m); static void vm_page_clear_dirty_mask(vm_page_t m, vm_page_bits_t pagebits); static void vm_page_enqueue(uint8_t queue, vm_page_t m); static void vm_page_free_wakeup(void); static void vm_page_init_fakepg(void *dummy); static int vm_page_insert_after(vm_page_t m, vm_object_t object, vm_pindex_t pindex, vm_page_t mpred); static void vm_page_insert_radixdone(vm_page_t m, vm_object_t object, vm_page_t mpred); static int vm_page_reclaim_run(int req_class, u_long npages, vm_page_t m_run, vm_paddr_t high); SYSINIT(vm_page, SI_SUB_VM, SI_ORDER_SECOND, vm_page_init_fakepg, NULL); static void vm_page_init_fakepg(void *dummy) { fakepg_zone = uma_zcreate("fakepg", sizeof(struct vm_page), NULL, NULL, NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_NOFREE | UMA_ZONE_VM); } /* Make sure that u_long is at least 64 bits when PAGE_SIZE is 32K. */ #if PAGE_SIZE == 32768 #ifdef CTASSERT CTASSERT(sizeof(u_long) >= 8); #endif #endif /* * Try to acquire a physical address lock while a pmap is locked. If we * fail to trylock we unlock and lock the pmap directly and cache the * locked pa in *locked. The caller should then restart their loop in case * the virtual to physical mapping has changed. */ int vm_page_pa_tryrelock(pmap_t pmap, vm_paddr_t pa, vm_paddr_t *locked) { vm_paddr_t lockpa; lockpa = *locked; *locked = pa; if (lockpa) { PA_LOCK_ASSERT(lockpa, MA_OWNED); if (PA_LOCKPTR(pa) == PA_LOCKPTR(lockpa)) return (0); PA_UNLOCK(lockpa); } if (PA_TRYLOCK(pa)) return (0); PMAP_UNLOCK(pmap); atomic_add_int(&pa_tryrelock_restart, 1); PA_LOCK(pa); PMAP_LOCK(pmap); return (EAGAIN); } /* * vm_set_page_size: * * Sets the page size, perhaps based upon the memory * size. Must be called before any use of page-size * dependent functions. */ void vm_set_page_size(void) { if (vm_cnt.v_page_size == 0) vm_cnt.v_page_size = PAGE_SIZE; if (((vm_cnt.v_page_size - 1) & vm_cnt.v_page_size) != 0) panic("vm_set_page_size: page size not a power of two"); } /* * vm_page_blacklist_next: * * Find the next entry in the provided string of blacklist * addresses. Entries are separated by space, comma, or newline. * If an invalid integer is encountered then the rest of the * string is skipped. Updates the list pointer to the next * character, or NULL if the string is exhausted or invalid. */ static vm_paddr_t vm_page_blacklist_next(char **list, char *end) { vm_paddr_t bad; char *cp, *pos; if (list == NULL || *list == NULL) return (0); if (**list =='\0') { *list = NULL; return (0); } /* * If there's no end pointer then the buffer is coming from * the kenv and we know it's null-terminated. */ if (end == NULL) end = *list + strlen(*list); /* Ensure that strtoq() won't walk off the end */ if (*end != '\0') { if (*end == '\n' || *end == ' ' || *end == ',') *end = '\0'; else { printf("Blacklist not terminated, skipping\n"); *list = NULL; return (0); } } for (pos = *list; *pos != '\0'; pos = cp) { bad = strtoq(pos, &cp, 0); if (*cp == '\0' || *cp == ' ' || *cp == ',' || *cp == '\n') { if (bad == 0) { if (++cp < end) continue; else break; } } else break; if (*cp == '\0' || ++cp >= end) *list = NULL; else *list = cp; return (trunc_page(bad)); } printf("Garbage in RAM blacklist, skipping\n"); *list = NULL; return (0); } /* * vm_page_blacklist_check: * * Iterate through the provided string of blacklist addresses, pulling * each entry out of the physical allocator free list and putting it * onto a list for reporting via the vm.page_blacklist sysctl. */ static void vm_page_blacklist_check(char *list, char *end) { vm_paddr_t pa; vm_page_t m; char *next; int ret; next = list; while (next != NULL) { if ((pa = vm_page_blacklist_next(&next, end)) == 0) continue; m = vm_phys_paddr_to_vm_page(pa); if (m == NULL) continue; mtx_lock(&vm_page_queue_free_mtx); ret = vm_phys_unfree_page(m); mtx_unlock(&vm_page_queue_free_mtx); if (ret == TRUE) { TAILQ_INSERT_TAIL(&blacklist_head, m, listq); if (bootverbose) printf("Skipping page with pa 0x%jx\n", (uintmax_t)pa); } } } /* * vm_page_blacklist_load: * * Search for a special module named "ram_blacklist". It'll be a * plain text file provided by the user via the loader directive * of the same name. */ static void vm_page_blacklist_load(char **list, char **end) { void *mod; u_char *ptr; u_int len; mod = NULL; ptr = NULL; mod = preload_search_by_type("ram_blacklist"); if (mod != NULL) { ptr = preload_fetch_addr(mod); len = preload_fetch_size(mod); } *list = ptr; if (ptr != NULL) *end = ptr + len; else *end = NULL; return; } static int sysctl_vm_page_blacklist(SYSCTL_HANDLER_ARGS) { vm_page_t m; struct sbuf sbuf; int error, first; first = 1; error = sysctl_wire_old_buffer(req, 0); if (error != 0) return (error); sbuf_new_for_sysctl(&sbuf, NULL, 128, req); TAILQ_FOREACH(m, &blacklist_head, listq) { sbuf_printf(&sbuf, "%s%#jx", first ? "" : ",", (uintmax_t)m->phys_addr); first = 0; } error = sbuf_finish(&sbuf); sbuf_delete(&sbuf); return (error); } static void vm_page_domain_init(struct vm_domain *vmd) { struct vm_pagequeue *pq; int i; *__DECONST(char **, &vmd->vmd_pagequeues[PQ_INACTIVE].pq_name) = "vm inactive pagequeue"; *__DECONST(u_int **, &vmd->vmd_pagequeues[PQ_INACTIVE].pq_vcnt) = &vm_cnt.v_inactive_count; *__DECONST(char **, &vmd->vmd_pagequeues[PQ_ACTIVE].pq_name) = "vm active pagequeue"; *__DECONST(u_int **, &vmd->vmd_pagequeues[PQ_ACTIVE].pq_vcnt) = &vm_cnt.v_active_count; vmd->vmd_page_count = 0; vmd->vmd_free_count = 0; vmd->vmd_segs = 0; vmd->vmd_oom = FALSE; vmd->vmd_pass = 0; for (i = 0; i < PQ_COUNT; i++) { pq = &vmd->vmd_pagequeues[i]; TAILQ_INIT(&pq->pq_pl); mtx_init(&pq->pq_mutex, pq->pq_name, "vm pagequeue", MTX_DEF | MTX_DUPOK); } } /* * vm_page_startup: * * Initializes the resident memory module. * * Allocates memory for the page cells, and * for the object/offset-to-page hash table headers. * Each page cell is initialized and placed on the free list. */ vm_offset_t vm_page_startup(vm_offset_t vaddr) { vm_offset_t mapped; vm_paddr_t page_range; vm_paddr_t new_end; int i; vm_paddr_t pa; vm_paddr_t last_pa; char *list, *listend; vm_paddr_t end; vm_paddr_t biggestsize; vm_paddr_t low_water, high_water; int biggestone; int pages_per_zone; biggestsize = 0; biggestone = 0; vaddr = round_page(vaddr); for (i = 0; phys_avail[i + 1]; i += 2) { phys_avail[i] = round_page(phys_avail[i]); phys_avail[i + 1] = trunc_page(phys_avail[i + 1]); } low_water = phys_avail[0]; high_water = phys_avail[1]; for (i = 0; i < vm_phys_nsegs; i++) { if (vm_phys_segs[i].start < low_water) low_water = vm_phys_segs[i].start; if (vm_phys_segs[i].end > high_water) high_water = vm_phys_segs[i].end; } for (i = 0; phys_avail[i + 1]; i += 2) { vm_paddr_t size = phys_avail[i + 1] - phys_avail[i]; if (size > biggestsize) { biggestone = i; biggestsize = size; } if (phys_avail[i] < low_water) low_water = phys_avail[i]; if (phys_avail[i + 1] > high_water) high_water = phys_avail[i + 1]; } end = phys_avail[biggestone+1]; /* * Initialize the page and queue locks. */ mtx_init(&vm_page_queue_free_mtx, "vm page free queue", NULL, MTX_DEF); for (i = 0; i < PA_LOCK_COUNT; i++) mtx_init(&pa_lock[i], "vm page", NULL, MTX_DEF); for (i = 0; i < vm_ndomains; i++) vm_page_domain_init(&vm_dom[i]); /* * Almost all of the pages needed for boot strapping UMA are used * for zone structures, so if the number of CPUs results in those * structures taking more than one page each, we set aside more pages * in proportion to the zone structure size. */ pages_per_zone = howmany(sizeof(struct uma_zone) + sizeof(struct uma_cache) * (mp_maxid + 1), UMA_SLAB_SIZE); if (pages_per_zone > 1) { /* Reserve more pages so that we don't run out. */ boot_pages = UMA_BOOT_PAGES_ZONES * pages_per_zone; } /* * Allocate memory for use when boot strapping the kernel memory * allocator. * * CTFLAG_RDTUN doesn't work during the early boot process, so we must * manually fetch the value. */ TUNABLE_INT_FETCH("vm.boot_pages", &boot_pages); new_end = end - (boot_pages * UMA_SLAB_SIZE); new_end = trunc_page(new_end); mapped = pmap_map(&vaddr, new_end, end, VM_PROT_READ | VM_PROT_WRITE); bzero((void *)mapped, end - new_end); uma_startup((void *)mapped, boot_pages); #if defined(__aarch64__) || defined(__amd64__) || defined(__arm__) || \ defined(__i386__) || defined(__mips__) /* * Allocate a bitmap to indicate that a random physical page * needs to be included in a minidump. * * The amd64 port needs this to indicate which direct map pages * need to be dumped, via calls to dump_add_page()/dump_drop_page(). * * However, i386 still needs this workspace internally within the * minidump code. In theory, they are not needed on i386, but are * included should the sf_buf code decide to use them. */ last_pa = 0; for (i = 0; dump_avail[i + 1] != 0; i += 2) if (dump_avail[i + 1] > last_pa) last_pa = dump_avail[i + 1]; page_range = last_pa / PAGE_SIZE; vm_page_dump_size = round_page(roundup2(page_range, NBBY) / NBBY); new_end -= vm_page_dump_size; vm_page_dump = (void *)(uintptr_t)pmap_map(&vaddr, new_end, new_end + vm_page_dump_size, VM_PROT_READ | VM_PROT_WRITE); bzero((void *)vm_page_dump, vm_page_dump_size); #endif #ifdef __amd64__ /* * Request that the physical pages underlying the message buffer be * included in a crash dump. Since the message buffer is accessed * through the direct map, they are not automatically included. */ pa = DMAP_TO_PHYS((vm_offset_t)msgbufp->msg_ptr); last_pa = pa + round_page(msgbufsize); while (pa < last_pa) { dump_add_page(pa); pa += PAGE_SIZE; } #endif /* * Compute the number of pages of memory that will be available for * use (taking into account the overhead of a page structure per * page). */ first_page = low_water / PAGE_SIZE; #ifdef VM_PHYSSEG_SPARSE page_range = 0; for (i = 0; i < vm_phys_nsegs; i++) { page_range += atop(vm_phys_segs[i].end - vm_phys_segs[i].start); } for (i = 0; phys_avail[i + 1] != 0; i += 2) page_range += atop(phys_avail[i + 1] - phys_avail[i]); #elif defined(VM_PHYSSEG_DENSE) page_range = high_water / PAGE_SIZE - first_page; #else #error "Either VM_PHYSSEG_DENSE or VM_PHYSSEG_SPARSE must be defined." #endif end = new_end; /* * Reserve an unmapped guard page to trap access to vm_page_array[-1]. */ vaddr += PAGE_SIZE; /* * Initialize the mem entry structures now, and put them in the free * queue. */ new_end = trunc_page(end - page_range * sizeof(struct vm_page)); mapped = pmap_map(&vaddr, new_end, end, VM_PROT_READ | VM_PROT_WRITE); vm_page_array = (vm_page_t) mapped; #if VM_NRESERVLEVEL > 0 /* * Allocate memory for the reservation management system's data * structures. */ new_end = vm_reserv_startup(&vaddr, new_end, high_water); #endif #if defined(__aarch64__) || defined(__amd64__) || defined(__mips__) /* * pmap_map on arm64, amd64, and mips can come out of the direct-map, * not kvm like i386, so the pages must be tracked for a crashdump to * include this data. This includes the vm_page_array and the early * UMA bootstrap pages. */ for (pa = new_end; pa < phys_avail[biggestone + 1]; pa += PAGE_SIZE) dump_add_page(pa); #endif phys_avail[biggestone + 1] = new_end; /* * Add physical memory segments corresponding to the available * physical pages. */ for (i = 0; phys_avail[i + 1] != 0; i += 2) vm_phys_add_seg(phys_avail[i], phys_avail[i + 1]); /* * Clear all of the page structures */ bzero((caddr_t) vm_page_array, page_range * sizeof(struct vm_page)); for (i = 0; i < page_range; i++) vm_page_array[i].order = VM_NFREEORDER; vm_page_array_size = page_range; /* * Initialize the physical memory allocator. */ vm_phys_init(); /* * Add every available physical page that is not blacklisted to * the free lists. */ vm_cnt.v_page_count = 0; vm_cnt.v_free_count = 0; for (i = 0; phys_avail[i + 1] != 0; i += 2) { pa = phys_avail[i]; last_pa = phys_avail[i + 1]; while (pa < last_pa) { vm_phys_add_page(pa); pa += PAGE_SIZE; } } TAILQ_INIT(&blacklist_head); vm_page_blacklist_load(&list, &listend); vm_page_blacklist_check(list, listend); list = kern_getenv("vm.blacklist"); vm_page_blacklist_check(list, NULL); freeenv(list); #if VM_NRESERVLEVEL > 0 /* * Initialize the reservation management system. */ vm_reserv_init(); #endif return (vaddr); } void vm_page_reference(vm_page_t m) { vm_page_aflag_set(m, PGA_REFERENCED); } /* * vm_page_busy_downgrade: * * Downgrade an exclusive busy page into a single shared busy page. */ void vm_page_busy_downgrade(vm_page_t m) { u_int x; vm_page_assert_xbusied(m); for (;;) { x = m->busy_lock; x &= VPB_BIT_WAITERS; if (atomic_cmpset_rel_int(&m->busy_lock, VPB_SINGLE_EXCLUSIVER | x, VPB_SHARERS_WORD(1) | x)) break; } } /* * vm_page_sbusied: * * Return a positive value if the page is shared busied, 0 otherwise. */ int vm_page_sbusied(vm_page_t m) { u_int x; x = m->busy_lock; return ((x & VPB_BIT_SHARED) != 0 && x != VPB_UNBUSIED); } /* * vm_page_sunbusy: * * Shared unbusy a page. */ void vm_page_sunbusy(vm_page_t m) { u_int x; vm_page_assert_sbusied(m); for (;;) { x = m->busy_lock; if (VPB_SHARERS(x) > 1) { if (atomic_cmpset_int(&m->busy_lock, x, x - VPB_ONE_SHARER)) break; continue; } if ((x & VPB_BIT_WAITERS) == 0) { KASSERT(x == VPB_SHARERS_WORD(1), ("vm_page_sunbusy: invalid lock state")); if (atomic_cmpset_int(&m->busy_lock, VPB_SHARERS_WORD(1), VPB_UNBUSIED)) break; continue; } KASSERT(x == (VPB_SHARERS_WORD(1) | VPB_BIT_WAITERS), ("vm_page_sunbusy: invalid lock state for waiters")); vm_page_lock(m); if (!atomic_cmpset_int(&m->busy_lock, x, VPB_UNBUSIED)) { vm_page_unlock(m); continue; } wakeup(m); vm_page_unlock(m); break; } } /* * vm_page_busy_sleep: * * Sleep and release the page lock, using the page pointer as wchan. * This is used to implement the hard-path of busying mechanism. * * The given page must be locked. */ void vm_page_busy_sleep(vm_page_t m, const char *wmesg) { u_int x; vm_page_lock_assert(m, MA_OWNED); x = m->busy_lock; if (x == VPB_UNBUSIED) { vm_page_unlock(m); return; } if ((x & VPB_BIT_WAITERS) == 0 && !atomic_cmpset_int(&m->busy_lock, x, x | VPB_BIT_WAITERS)) { vm_page_unlock(m); return; } msleep(m, vm_page_lockptr(m), PVM | PDROP, wmesg, 0); } /* * vm_page_trysbusy: * * Try to shared busy a page. * If the operation succeeds 1 is returned otherwise 0. * The operation never sleeps. */ int vm_page_trysbusy(vm_page_t m) { u_int x; for (;;) { x = m->busy_lock; if ((x & VPB_BIT_SHARED) == 0) return (0); if (atomic_cmpset_acq_int(&m->busy_lock, x, x + VPB_ONE_SHARER)) return (1); } } static void vm_page_xunbusy_locked(vm_page_t m) { vm_page_assert_xbusied(m); vm_page_assert_locked(m); atomic_store_rel_int(&m->busy_lock, VPB_UNBUSIED); /* There is a waiter, do wakeup() instead of vm_page_flash(). */ wakeup(m); } static void vm_page_xunbusy_maybelocked(vm_page_t m) { bool lockacq; vm_page_assert_xbusied(m); /* * Fast path for unbusy. If it succeeds, we know that there * are no waiters, so we do not need a wakeup. */ if (atomic_cmpset_rel_int(&m->busy_lock, VPB_SINGLE_EXCLUSIVER, VPB_UNBUSIED)) return; lockacq = !mtx_owned(vm_page_lockptr(m)); if (lockacq) vm_page_lock(m); vm_page_xunbusy_locked(m); if (lockacq) vm_page_unlock(m); } /* * vm_page_xunbusy_hard: * * Called after the first try the exclusive unbusy of a page failed. * It is assumed that the waiters bit is on. */ void vm_page_xunbusy_hard(vm_page_t m) { vm_page_assert_xbusied(m); vm_page_lock(m); vm_page_xunbusy_locked(m); vm_page_unlock(m); } /* * vm_page_flash: * * Wakeup anyone waiting for the page. * The ownership bits do not change. * * The given page must be locked. */ void vm_page_flash(vm_page_t m) { u_int x; vm_page_lock_assert(m, MA_OWNED); for (;;) { x = m->busy_lock; if ((x & VPB_BIT_WAITERS) == 0) return; if (atomic_cmpset_int(&m->busy_lock, x, x & (~VPB_BIT_WAITERS))) break; } wakeup(m); } /* * Keep page from being freed by the page daemon * much of the same effect as wiring, except much lower * overhead and should be used only for *very* temporary * holding ("wiring"). */ void vm_page_hold(vm_page_t mem) { vm_page_lock_assert(mem, MA_OWNED); mem->hold_count++; } void vm_page_unhold(vm_page_t mem) { vm_page_lock_assert(mem, MA_OWNED); KASSERT(mem->hold_count >= 1, ("vm_page_unhold: hold count < 0!!!")); --mem->hold_count; if (mem->hold_count == 0 && (mem->flags & PG_UNHOLDFREE) != 0) vm_page_free_toq(mem); } /* * vm_page_unhold_pages: * * Unhold each of the pages that is referenced by the given array. */ void vm_page_unhold_pages(vm_page_t *ma, int count) { struct mtx *mtx, *new_mtx; mtx = NULL; for (; count != 0; count--) { /* * Avoid releasing and reacquiring the same page lock. */ new_mtx = vm_page_lockptr(*ma); if (mtx != new_mtx) { if (mtx != NULL) mtx_unlock(mtx); mtx = new_mtx; mtx_lock(mtx); } vm_page_unhold(*ma); ma++; } if (mtx != NULL) mtx_unlock(mtx); } vm_page_t PHYS_TO_VM_PAGE(vm_paddr_t pa) { vm_page_t m; #ifdef VM_PHYSSEG_SPARSE m = vm_phys_paddr_to_vm_page(pa); if (m == NULL) m = vm_phys_fictitious_to_vm_page(pa); return (m); #elif defined(VM_PHYSSEG_DENSE) long pi; pi = atop(pa); if (pi >= first_page && (pi - first_page) < vm_page_array_size) { m = &vm_page_array[pi - first_page]; return (m); } return (vm_phys_fictitious_to_vm_page(pa)); #else #error "Either VM_PHYSSEG_DENSE or VM_PHYSSEG_SPARSE must be defined." #endif } /* * vm_page_getfake: * * Create a fictitious page with the specified physical address and * memory attribute. The memory attribute is the only the machine- * dependent aspect of a fictitious page that must be initialized. */ vm_page_t vm_page_getfake(vm_paddr_t paddr, vm_memattr_t memattr) { vm_page_t m; m = uma_zalloc(fakepg_zone, M_WAITOK | M_ZERO); vm_page_initfake(m, paddr, memattr); return (m); } void vm_page_initfake(vm_page_t m, vm_paddr_t paddr, vm_memattr_t memattr) { if ((m->flags & PG_FICTITIOUS) != 0) { /* * The page's memattr might have changed since the * previous initialization. Update the pmap to the * new memattr. */ goto memattr; } m->phys_addr = paddr; m->queue = PQ_NONE; /* Fictitious pages don't use "segind". */ m->flags = PG_FICTITIOUS; /* Fictitious pages don't use "order" or "pool". */ m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_SINGLE_EXCLUSIVER; m->wire_count = 1; pmap_page_init(m); memattr: pmap_page_set_memattr(m, memattr); } /* * vm_page_putfake: * * Release a fictitious page. */ void vm_page_putfake(vm_page_t m) { KASSERT((m->oflags & VPO_UNMANAGED) != 0, ("managed %p", m)); KASSERT((m->flags & PG_FICTITIOUS) != 0, ("vm_page_putfake: bad page %p", m)); uma_zfree(fakepg_zone, m); } /* * vm_page_updatefake: * * Update the given fictitious page to the specified physical address and * memory attribute. */ void vm_page_updatefake(vm_page_t m, vm_paddr_t paddr, vm_memattr_t memattr) { KASSERT((m->flags & PG_FICTITIOUS) != 0, ("vm_page_updatefake: bad page %p", m)); m->phys_addr = paddr; pmap_page_set_memattr(m, memattr); } /* * vm_page_free: * * Free a page. */ void vm_page_free(vm_page_t m) { m->flags &= ~PG_ZERO; vm_page_free_toq(m); } /* * vm_page_free_zero: * * Free a page to the zerod-pages queue */ void vm_page_free_zero(vm_page_t m) { m->flags |= PG_ZERO; vm_page_free_toq(m); } /* - * Unbusy and handle the page queueing for a page from the VOP_GETPAGES() - * array which was optionally read ahead or behind. + * Unbusy and handle the page queueing for a page from a getpages request that + * was optionally read ahead or behind. */ void vm_page_readahead_finish(vm_page_t m) { /* We shouldn't put invalid pages on queues. */ KASSERT(m->valid != 0, ("%s: %p is invalid", __func__, m)); /* * Since the page is not the actually needed one, whether it should * be activated or deactivated is not obvious. Empirical results * have shown that deactivating the page is usually the best choice, * unless the page is wanted by another thread. */ vm_page_lock(m); if ((m->busy_lock & VPB_BIT_WAITERS) != 0) vm_page_activate(m); else vm_page_deactivate(m); vm_page_unlock(m); vm_page_xunbusy(m); } /* * vm_page_sleep_if_busy: * * Sleep and release the page queues lock if the page is busied. * Returns TRUE if the thread slept. * * The given page must be unlocked and object containing it must * be locked. */ int vm_page_sleep_if_busy(vm_page_t m, const char *msg) { vm_object_t obj; vm_page_lock_assert(m, MA_NOTOWNED); VM_OBJECT_ASSERT_WLOCKED(m->object); if (vm_page_busied(m)) { /* * The page-specific object must be cached because page * identity can change during the sleep, causing the * re-lock of a different object. * It is assumed that a reference to the object is already * held by the callers. */ obj = m->object; vm_page_lock(m); VM_OBJECT_WUNLOCK(obj); vm_page_busy_sleep(m, msg); VM_OBJECT_WLOCK(obj); return (TRUE); } return (FALSE); } /* * vm_page_dirty_KBI: [ internal use only ] * * Set all bits in the page's dirty field. * * The object containing the specified page must be locked if the * call is made from the machine-independent layer. * * See vm_page_clear_dirty_mask(). * * This function should only be called by vm_page_dirty(). */ void vm_page_dirty_KBI(vm_page_t m) { /* These assertions refer to this operation by its public name. */ KASSERT((m->flags & PG_CACHED) == 0, ("vm_page_dirty: page in cache!")); KASSERT(m->valid == VM_PAGE_BITS_ALL, ("vm_page_dirty: page is invalid!")); m->dirty = VM_PAGE_BITS_ALL; } /* * vm_page_insert: [ internal use only ] * * Inserts the given mem entry into the object and object list. * * The object must be locked. */ int vm_page_insert(vm_page_t m, vm_object_t object, vm_pindex_t pindex) { vm_page_t mpred; VM_OBJECT_ASSERT_WLOCKED(object); mpred = vm_radix_lookup_le(&object->rtree, pindex); return (vm_page_insert_after(m, object, pindex, mpred)); } /* * vm_page_insert_after: * * Inserts the page "m" into the specified object at offset "pindex". * * The page "mpred" must immediately precede the offset "pindex" within * the specified object. * * The object must be locked. */ static int vm_page_insert_after(vm_page_t m, vm_object_t object, vm_pindex_t pindex, vm_page_t mpred) { vm_page_t msucc; VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(m->object == NULL, ("vm_page_insert_after: page already inserted")); if (mpred != NULL) { KASSERT(mpred->object == object, ("vm_page_insert_after: object doesn't contain mpred")); KASSERT(mpred->pindex < pindex, ("vm_page_insert_after: mpred doesn't precede pindex")); msucc = TAILQ_NEXT(mpred, listq); } else msucc = TAILQ_FIRST(&object->memq); if (msucc != NULL) KASSERT(msucc->pindex > pindex, ("vm_page_insert_after: msucc doesn't succeed pindex")); /* * Record the object/offset pair in this page */ m->object = object; m->pindex = pindex; /* * Now link into the object's ordered list of backed pages. */ if (vm_radix_insert(&object->rtree, m)) { m->object = NULL; m->pindex = 0; return (1); } vm_page_insert_radixdone(m, object, mpred); return (0); } /* * vm_page_insert_radixdone: * * Complete page "m" insertion into the specified object after the * radix trie hooking. * * The page "mpred" must precede the offset "m->pindex" within the * specified object. * * The object must be locked. */ static void vm_page_insert_radixdone(vm_page_t m, vm_object_t object, vm_page_t mpred) { VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(object != NULL && m->object == object, ("vm_page_insert_radixdone: page %p has inconsistent object", m)); if (mpred != NULL) { KASSERT(mpred->object == object, ("vm_page_insert_after: object doesn't contain mpred")); KASSERT(mpred->pindex < m->pindex, ("vm_page_insert_after: mpred doesn't precede pindex")); } if (mpred != NULL) TAILQ_INSERT_AFTER(&object->memq, mpred, m, listq); else TAILQ_INSERT_HEAD(&object->memq, m, listq); /* * Show that the object has one more resident page. */ object->resident_page_count++; /* * Hold the vnode until the last page is released. */ if (object->resident_page_count == 1 && object->type == OBJT_VNODE) vhold(object->handle); /* * Since we are inserting a new and possibly dirty page, * update the object's OBJ_MIGHTBEDIRTY flag. */ if (pmap_page_is_write_mapped(m)) vm_object_set_writeable_dirty(object); } /* * vm_page_remove: * * Removes the given mem entry from the object/offset-page * table and the object page list, but do not invalidate/terminate * the backing store. * * The object must be locked. The page must be locked if it is managed. */ void vm_page_remove(vm_page_t m) { vm_object_t object; if ((m->oflags & VPO_UNMANAGED) == 0) vm_page_assert_locked(m); if ((object = m->object) == NULL) return; VM_OBJECT_ASSERT_WLOCKED(object); if (vm_page_xbusied(m)) vm_page_xunbusy_maybelocked(m); /* * Now remove from the object's list of backed pages. */ vm_radix_remove(&object->rtree, m->pindex); TAILQ_REMOVE(&object->memq, m, listq); /* * And show that the object has one fewer resident page. */ object->resident_page_count--; /* * The vnode may now be recycled. */ if (object->resident_page_count == 0 && object->type == OBJT_VNODE) vdrop(object->handle); m->object = NULL; } /* * vm_page_lookup: * * Returns the page associated with the object/offset * pair specified; if none is found, NULL is returned. * * The object must be locked. */ vm_page_t vm_page_lookup(vm_object_t object, vm_pindex_t pindex) { VM_OBJECT_ASSERT_LOCKED(object); return (vm_radix_lookup(&object->rtree, pindex)); } /* * vm_page_find_least: * * Returns the page associated with the object with least pindex * greater than or equal to the parameter pindex, or NULL. * * The object must be locked. */ vm_page_t vm_page_find_least(vm_object_t object, vm_pindex_t pindex) { vm_page_t m; VM_OBJECT_ASSERT_LOCKED(object); if ((m = TAILQ_FIRST(&object->memq)) != NULL && m->pindex < pindex) m = vm_radix_lookup_ge(&object->rtree, pindex); return (m); } /* * Returns the given page's successor (by pindex) within the object if it is * resident; if none is found, NULL is returned. * * The object must be locked. */ vm_page_t vm_page_next(vm_page_t m) { vm_page_t next; VM_OBJECT_ASSERT_LOCKED(m->object); if ((next = TAILQ_NEXT(m, listq)) != NULL && next->pindex != m->pindex + 1) next = NULL; return (next); } /* * Returns the given page's predecessor (by pindex) within the object if it is * resident; if none is found, NULL is returned. * * The object must be locked. */ vm_page_t vm_page_prev(vm_page_t m) { vm_page_t prev; VM_OBJECT_ASSERT_LOCKED(m->object); if ((prev = TAILQ_PREV(m, pglist, listq)) != NULL && prev->pindex != m->pindex - 1) prev = NULL; return (prev); } /* * Uses the page mnew as a replacement for an existing page at index * pindex which must be already present in the object. * * The existing page must not be on a paging queue. */ vm_page_t vm_page_replace(vm_page_t mnew, vm_object_t object, vm_pindex_t pindex) { vm_page_t mold; VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(mnew->object == NULL, ("vm_page_replace: page already in object")); /* * This function mostly follows vm_page_insert() and * vm_page_remove() without the radix, object count and vnode * dance. Double check such functions for more comments. */ mnew->object = object; mnew->pindex = pindex; mold = vm_radix_replace(&object->rtree, mnew); KASSERT(mold->queue == PQ_NONE, ("vm_page_replace: mold is on a paging queue")); /* Keep the resident page list in sorted order. */ TAILQ_INSERT_AFTER(&object->memq, mold, mnew, listq); TAILQ_REMOVE(&object->memq, mold, listq); mold->object = NULL; vm_page_xunbusy_maybelocked(mold); /* * The object's resident_page_count does not change because we have * swapped one page for another, but OBJ_MIGHTBEDIRTY. */ if (pmap_page_is_write_mapped(mnew)) vm_object_set_writeable_dirty(object); return (mold); } /* * vm_page_rename: * * Move the given memory entry from its * current object to the specified target object/offset. * * Note: swap associated with the page must be invalidated by the move. We * have to do this for several reasons: (1) we aren't freeing the * page, (2) we are dirtying the page, (3) the VM system is probably * moving the page from object A to B, and will then later move * the backing store from A to B and we can't have a conflict. * * Note: we *always* dirty the page. It is necessary both for the * fact that we moved it, and because we may be invalidating * swap. If the page is on the cache, we have to deactivate it * or vm_page_dirty() will panic. Dirty pages are not allowed * on the cache. * * The objects must be locked. */ int vm_page_rename(vm_page_t m, vm_object_t new_object, vm_pindex_t new_pindex) { vm_page_t mpred; vm_pindex_t opidx; VM_OBJECT_ASSERT_WLOCKED(new_object); mpred = vm_radix_lookup_le(&new_object->rtree, new_pindex); KASSERT(mpred == NULL || mpred->pindex != new_pindex, ("vm_page_rename: pindex already renamed")); /* * Create a custom version of vm_page_insert() which does not depend * by m_prev and can cheat on the implementation aspects of the * function. */ opidx = m->pindex; m->pindex = new_pindex; if (vm_radix_insert(&new_object->rtree, m)) { m->pindex = opidx; return (1); } /* * The operation cannot fail anymore. The removal must happen before * the listq iterator is tainted. */ m->pindex = opidx; vm_page_lock(m); vm_page_remove(m); /* Return back to the new pindex to complete vm_page_insert(). */ m->pindex = new_pindex; m->object = new_object; vm_page_unlock(m); vm_page_insert_radixdone(m, new_object, mpred); vm_page_dirty(m); return (0); } /* * Convert all of the given object's cached pages that have a * pindex within the given range into free pages. If the value * zero is given for "end", then the range's upper bound is * infinity. If the given object is backed by a vnode and it * transitions from having one or more cached pages to none, the * vnode's hold count is reduced. */ void vm_page_cache_free(vm_object_t object, vm_pindex_t start, vm_pindex_t end) { vm_page_t m; boolean_t empty; mtx_lock(&vm_page_queue_free_mtx); if (__predict_false(vm_radix_is_empty(&object->cache))) { mtx_unlock(&vm_page_queue_free_mtx); return; } while ((m = vm_radix_lookup_ge(&object->cache, start)) != NULL) { if (end != 0 && m->pindex >= end) break; vm_radix_remove(&object->cache, m->pindex); vm_page_cache_turn_free(m); } empty = vm_radix_is_empty(&object->cache); mtx_unlock(&vm_page_queue_free_mtx); if (object->type == OBJT_VNODE && empty) vdrop(object->handle); } /* * Returns the cached page that is associated with the given * object and offset. If, however, none exists, returns NULL. * * The free page queue must be locked. */ static inline vm_page_t vm_page_cache_lookup(vm_object_t object, vm_pindex_t pindex) { mtx_assert(&vm_page_queue_free_mtx, MA_OWNED); return (vm_radix_lookup(&object->cache, pindex)); } /* * Remove the given cached page from its containing object's * collection of cached pages. * * The free page queue must be locked. */ static void vm_page_cache_remove(vm_page_t m) { mtx_assert(&vm_page_queue_free_mtx, MA_OWNED); KASSERT((m->flags & PG_CACHED) != 0, ("vm_page_cache_remove: page %p is not cached", m)); vm_radix_remove(&m->object->cache, m->pindex); m->object = NULL; vm_cnt.v_cache_count--; } /* * Transfer all of the cached pages with offset greater than or * equal to 'offidxstart' from the original object's cache to the * new object's cache. However, any cached pages with offset * greater than or equal to the new object's size are kept in the * original object. Initially, the new object's cache must be * empty. Offset 'offidxstart' in the original object must * correspond to offset zero in the new object. * * The new object must be locked. */ void vm_page_cache_transfer(vm_object_t orig_object, vm_pindex_t offidxstart, vm_object_t new_object) { vm_page_t m; /* * Insertion into an object's collection of cached pages * requires the object to be locked. In contrast, removal does * not. */ VM_OBJECT_ASSERT_WLOCKED(new_object); KASSERT(vm_radix_is_empty(&new_object->cache), ("vm_page_cache_transfer: object %p has cached pages", new_object)); mtx_lock(&vm_page_queue_free_mtx); while ((m = vm_radix_lookup_ge(&orig_object->cache, offidxstart)) != NULL) { /* * Transfer all of the pages with offset greater than or * equal to 'offidxstart' from the original object's * cache to the new object's cache. */ if ((m->pindex - offidxstart) >= new_object->size) break; vm_radix_remove(&orig_object->cache, m->pindex); /* Update the page's object and offset. */ m->object = new_object; m->pindex -= offidxstart; if (vm_radix_insert(&new_object->cache, m)) vm_page_cache_turn_free(m); } mtx_unlock(&vm_page_queue_free_mtx); } /* * Returns TRUE if a cached page is associated with the given object and * offset, and FALSE otherwise. * * The object must be locked. */ boolean_t vm_page_is_cached(vm_object_t object, vm_pindex_t pindex) { vm_page_t m; /* * Insertion into an object's collection of cached pages requires the * object to be locked. Therefore, if the object is locked and the * object's collection is empty, there is no need to acquire the free * page queues lock in order to prove that the specified page doesn't * exist. */ VM_OBJECT_ASSERT_WLOCKED(object); if (__predict_true(vm_object_cache_is_empty(object))) return (FALSE); mtx_lock(&vm_page_queue_free_mtx); m = vm_page_cache_lookup(object, pindex); mtx_unlock(&vm_page_queue_free_mtx); return (m != NULL); } /* * vm_page_alloc: * * Allocate and return a page that is associated with the specified * object and offset pair. By default, this page is exclusive busied. * * The caller must always specify an allocation class. * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs a page * VM_ALLOC_INTERRUPT interrupt time request * * optional allocation flags: * VM_ALLOC_COUNT(number) the number of additional pages that the caller * intends to allocate * VM_ALLOC_IFCACHED return page only if it is cached * VM_ALLOC_IFNOTCACHED return NULL, do not reactivate if the page * is cached * VM_ALLOC_NOBUSY do not exclusive busy the page * VM_ALLOC_NODUMP do not include the page in a kernel core dump * VM_ALLOC_NOOBJ page is not associated with an object and * should not be exclusive busy * VM_ALLOC_SBUSY shared busy the allocated page * VM_ALLOC_WIRED wire the allocated page * VM_ALLOC_ZERO prefer a zeroed page * * This routine may not sleep. */ vm_page_t vm_page_alloc(vm_object_t object, vm_pindex_t pindex, int req) { struct vnode *vp = NULL; vm_object_t m_object; vm_page_t m, mpred; int flags, req_class; mpred = 0; /* XXX: pacify gcc */ KASSERT((object != NULL) == ((req & VM_ALLOC_NOOBJ) == 0) && (object != NULL || (req & VM_ALLOC_SBUSY) == 0) && ((req & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) != (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)), ("vm_page_alloc: inconsistent object(%p)/req(%x)", (void *)object, req)); if (object != NULL) VM_OBJECT_ASSERT_WLOCKED(object); req_class = req & VM_ALLOC_CLASS_MASK; /* * The page daemon is allowed to dig deeper into the free page list. */ if (curproc == pageproc && req_class != VM_ALLOC_INTERRUPT) req_class = VM_ALLOC_SYSTEM; if (object != NULL) { mpred = vm_radix_lookup_le(&object->rtree, pindex); KASSERT(mpred == NULL || mpred->pindex != pindex, ("vm_page_alloc: pindex already allocated")); } /* * The page allocation request can came from consumers which already * hold the free page queue mutex, like vm_page_insert() in * vm_page_cache(). */ mtx_lock_flags(&vm_page_queue_free_mtx, MTX_RECURSE); if (vm_cnt.v_free_count + vm_cnt.v_cache_count > vm_cnt.v_free_reserved || (req_class == VM_ALLOC_SYSTEM && vm_cnt.v_free_count + vm_cnt.v_cache_count > vm_cnt.v_interrupt_free_min) || (req_class == VM_ALLOC_INTERRUPT && vm_cnt.v_free_count + vm_cnt.v_cache_count > 0)) { /* * Allocate from the free queue if the number of free pages * exceeds the minimum for the request class. */ if (object != NULL && (m = vm_page_cache_lookup(object, pindex)) != NULL) { if ((req & VM_ALLOC_IFNOTCACHED) != 0) { mtx_unlock(&vm_page_queue_free_mtx); return (NULL); } if (vm_phys_unfree_page(m)) vm_phys_set_pool(VM_FREEPOOL_DEFAULT, m, 0); #if VM_NRESERVLEVEL > 0 else if (!vm_reserv_reactivate_page(m)) #else else #endif panic("vm_page_alloc: cache page %p is missing" " from the free queue", m); } else if ((req & VM_ALLOC_IFCACHED) != 0) { mtx_unlock(&vm_page_queue_free_mtx); return (NULL); #if VM_NRESERVLEVEL > 0 } else if (object == NULL || (object->flags & (OBJ_COLORED | OBJ_FICTITIOUS)) != OBJ_COLORED || (m = vm_reserv_alloc_page(object, pindex, mpred)) == NULL) { #else } else { #endif m = vm_phys_alloc_pages(object != NULL ? VM_FREEPOOL_DEFAULT : VM_FREEPOOL_DIRECT, 0); #if VM_NRESERVLEVEL > 0 if (m == NULL && vm_reserv_reclaim_inactive()) { m = vm_phys_alloc_pages(object != NULL ? VM_FREEPOOL_DEFAULT : VM_FREEPOOL_DIRECT, 0); } #endif } } else { /* * Not allocatable, give up. */ mtx_unlock(&vm_page_queue_free_mtx); atomic_add_int(&vm_pageout_deficit, max((u_int)req >> VM_ALLOC_COUNT_SHIFT, 1)); pagedaemon_wakeup(); return (NULL); } /* * At this point we had better have found a good page. */ KASSERT(m != NULL, ("vm_page_alloc: missing page")); KASSERT(m->queue == PQ_NONE, ("vm_page_alloc: page %p has unexpected queue %d", m, m->queue)); KASSERT(m->wire_count == 0, ("vm_page_alloc: page %p is wired", m)); KASSERT(m->hold_count == 0, ("vm_page_alloc: page %p is held", m)); KASSERT(!vm_page_busied(m), ("vm_page_alloc: page %p is busy", m)); KASSERT(m->dirty == 0, ("vm_page_alloc: page %p is dirty", m)); KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("vm_page_alloc: page %p has unexpected memattr %d", m, pmap_page_get_memattr(m))); if ((m->flags & PG_CACHED) != 0) { KASSERT((m->flags & PG_ZERO) == 0, ("vm_page_alloc: cached page %p is PG_ZERO", m)); KASSERT(m->valid != 0, ("vm_page_alloc: cached page %p is invalid", m)); if (m->object == object && m->pindex == pindex) vm_cnt.v_reactivated++; else m->valid = 0; m_object = m->object; vm_page_cache_remove(m); if (m_object->type == OBJT_VNODE && vm_object_cache_is_empty(m_object)) vp = m_object->handle; } else { KASSERT(m->valid == 0, ("vm_page_alloc: free page %p is valid", m)); vm_phys_freecnt_adj(m, -1); if ((m->flags & PG_ZERO) != 0) vm_page_zero_count--; } mtx_unlock(&vm_page_queue_free_mtx); /* * Initialize the page. Only the PG_ZERO flag is inherited. */ flags = 0; if ((req & VM_ALLOC_ZERO) != 0) flags = PG_ZERO; flags &= m->flags; if ((req & VM_ALLOC_NODUMP) != 0) flags |= PG_NODUMP; m->flags = flags; m->aflags = 0; m->oflags = object == NULL || (object->flags & OBJ_UNMANAGED) != 0 ? VPO_UNMANAGED : 0; m->busy_lock = VPB_UNBUSIED; if ((req & (VM_ALLOC_NOBUSY | VM_ALLOC_NOOBJ | VM_ALLOC_SBUSY)) == 0) m->busy_lock = VPB_SINGLE_EXCLUSIVER; if ((req & VM_ALLOC_SBUSY) != 0) m->busy_lock = VPB_SHARERS_WORD(1); if (req & VM_ALLOC_WIRED) { /* * The page lock is not required for wiring a page until that * page is inserted into the object. */ atomic_add_int(&vm_cnt.v_wire_count, 1); m->wire_count = 1; } m->act_count = 0; if (object != NULL) { if (vm_page_insert_after(m, object, pindex, mpred)) { /* See the comment below about hold count. */ if (vp != NULL) vdrop(vp); pagedaemon_wakeup(); if (req & VM_ALLOC_WIRED) { atomic_subtract_int(&vm_cnt.v_wire_count, 1); m->wire_count = 0; } m->object = NULL; m->oflags = VPO_UNMANAGED; m->busy_lock = VPB_UNBUSIED; vm_page_free(m); return (NULL); } /* Ignore device objects; the pager sets "memattr" for them. */ if (object->memattr != VM_MEMATTR_DEFAULT && (object->flags & OBJ_FICTITIOUS) == 0) pmap_page_set_memattr(m, object->memattr); } else m->pindex = pindex; /* * The following call to vdrop() must come after the above call * to vm_page_insert() in case both affect the same object and * vnode. Otherwise, the affected vnode's hold count could * temporarily become zero. */ if (vp != NULL) vdrop(vp); /* * Don't wakeup too often - wakeup the pageout daemon when * we would be nearly out of memory. */ if (vm_paging_needed()) pagedaemon_wakeup(); return (m); } static void vm_page_alloc_contig_vdrop(struct spglist *lst) { while (!SLIST_EMPTY(lst)) { vdrop((struct vnode *)SLIST_FIRST(lst)-> plinks.s.pv); SLIST_REMOVE_HEAD(lst, plinks.s.ss); } } /* * vm_page_alloc_contig: * * Allocate a contiguous set of physical pages of the given size "npages" * from the free lists. All of the physical pages must be at or above * the given physical address "low" and below the given physical address * "high". The given value "alignment" determines the alignment of the * first physical page in the set. If the given value "boundary" is * non-zero, then the set of physical pages cannot cross any physical * address boundary that is a multiple of that value. Both "alignment" * and "boundary" must be a power of two. * * If the specified memory attribute, "memattr", is VM_MEMATTR_DEFAULT, * then the memory attribute setting for the physical pages is configured * to the object's memory attribute setting. Otherwise, the memory * attribute setting for the physical pages is configured to "memattr", * overriding the object's memory attribute setting. However, if the * object's memory attribute setting is not VM_MEMATTR_DEFAULT, then the * memory attribute setting for the physical pages cannot be configured * to VM_MEMATTR_DEFAULT. * * The caller must always specify an allocation class. * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs a page * VM_ALLOC_INTERRUPT interrupt time request * * optional allocation flags: * VM_ALLOC_NOBUSY do not exclusive busy the page * VM_ALLOC_NODUMP do not include the page in a kernel core dump * VM_ALLOC_NOOBJ page is not associated with an object and * should not be exclusive busy * VM_ALLOC_SBUSY shared busy the allocated page * VM_ALLOC_WIRED wire the allocated page * VM_ALLOC_ZERO prefer a zeroed page * * This routine may not sleep. */ vm_page_t vm_page_alloc_contig(vm_object_t object, vm_pindex_t pindex, int req, u_long npages, vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary, vm_memattr_t memattr) { struct vnode *drop; struct spglist deferred_vdrop_list; vm_page_t m, m_tmp, m_ret; u_int flags; int req_class; KASSERT((object != NULL) == ((req & VM_ALLOC_NOOBJ) == 0) && (object != NULL || (req & VM_ALLOC_SBUSY) == 0) && ((req & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) != (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)), ("vm_page_alloc: inconsistent object(%p)/req(%x)", (void *)object, req)); if (object != NULL) { VM_OBJECT_ASSERT_WLOCKED(object); KASSERT(object->type == OBJT_PHYS, ("vm_page_alloc_contig: object %p isn't OBJT_PHYS", object)); } KASSERT(npages > 0, ("vm_page_alloc_contig: npages is zero")); req_class = req & VM_ALLOC_CLASS_MASK; /* * The page daemon is allowed to dig deeper into the free page list. */ if (curproc == pageproc && req_class != VM_ALLOC_INTERRUPT) req_class = VM_ALLOC_SYSTEM; SLIST_INIT(&deferred_vdrop_list); mtx_lock(&vm_page_queue_free_mtx); if (vm_cnt.v_free_count + vm_cnt.v_cache_count >= npages + vm_cnt.v_free_reserved || (req_class == VM_ALLOC_SYSTEM && vm_cnt.v_free_count + vm_cnt.v_cache_count >= npages + vm_cnt.v_interrupt_free_min) || (req_class == VM_ALLOC_INTERRUPT && vm_cnt.v_free_count + vm_cnt.v_cache_count >= npages)) { #if VM_NRESERVLEVEL > 0 retry: if (object == NULL || (object->flags & OBJ_COLORED) == 0 || (m_ret = vm_reserv_alloc_contig(object, pindex, npages, low, high, alignment, boundary)) == NULL) #endif m_ret = vm_phys_alloc_contig(npages, low, high, alignment, boundary); } else { mtx_unlock(&vm_page_queue_free_mtx); atomic_add_int(&vm_pageout_deficit, npages); pagedaemon_wakeup(); return (NULL); } if (m_ret != NULL) for (m = m_ret; m < &m_ret[npages]; m++) { drop = vm_page_alloc_init(m); if (drop != NULL) { /* * Enqueue the vnode for deferred vdrop(). */ m->plinks.s.pv = drop; SLIST_INSERT_HEAD(&deferred_vdrop_list, m, plinks.s.ss); } } else { #if VM_NRESERVLEVEL > 0 if (vm_reserv_reclaim_contig(npages, low, high, alignment, boundary)) goto retry; #endif } mtx_unlock(&vm_page_queue_free_mtx); if (m_ret == NULL) return (NULL); /* * Initialize the pages. Only the PG_ZERO flag is inherited. */ flags = 0; if ((req & VM_ALLOC_ZERO) != 0) flags = PG_ZERO; if ((req & VM_ALLOC_NODUMP) != 0) flags |= PG_NODUMP; if ((req & VM_ALLOC_WIRED) != 0) atomic_add_int(&vm_cnt.v_wire_count, npages); if (object != NULL) { if (object->memattr != VM_MEMATTR_DEFAULT && memattr == VM_MEMATTR_DEFAULT) memattr = object->memattr; } for (m = m_ret; m < &m_ret[npages]; m++) { m->aflags = 0; m->flags = (m->flags | PG_NODUMP) & flags; m->busy_lock = VPB_UNBUSIED; if (object != NULL) { if ((req & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) == 0) m->busy_lock = VPB_SINGLE_EXCLUSIVER; if ((req & VM_ALLOC_SBUSY) != 0) m->busy_lock = VPB_SHARERS_WORD(1); } if ((req & VM_ALLOC_WIRED) != 0) m->wire_count = 1; /* Unmanaged pages don't use "act_count". */ m->oflags = VPO_UNMANAGED; if (object != NULL) { if (vm_page_insert(m, object, pindex)) { vm_page_alloc_contig_vdrop( &deferred_vdrop_list); if (vm_paging_needed()) pagedaemon_wakeup(); if ((req & VM_ALLOC_WIRED) != 0) atomic_subtract_int(&vm_cnt.v_wire_count, npages); for (m_tmp = m, m = m_ret; m < &m_ret[npages]; m++) { if ((req & VM_ALLOC_WIRED) != 0) m->wire_count = 0; if (m >= m_tmp) { m->object = NULL; m->oflags |= VPO_UNMANAGED; } m->busy_lock = VPB_UNBUSIED; vm_page_free(m); } return (NULL); } } else m->pindex = pindex; if (memattr != VM_MEMATTR_DEFAULT) pmap_page_set_memattr(m, memattr); pindex++; } vm_page_alloc_contig_vdrop(&deferred_vdrop_list); if (vm_paging_needed()) pagedaemon_wakeup(); return (m_ret); } /* * Initialize a page that has been freshly dequeued from a freelist. * The caller has to drop the vnode returned, if it is not NULL. * * This function may only be used to initialize unmanaged pages. * * To be called with vm_page_queue_free_mtx held. */ static struct vnode * vm_page_alloc_init(vm_page_t m) { struct vnode *drop; vm_object_t m_object; KASSERT(m->queue == PQ_NONE, ("vm_page_alloc_init: page %p has unexpected queue %d", m, m->queue)); KASSERT(m->wire_count == 0, ("vm_page_alloc_init: page %p is wired", m)); KASSERT(m->hold_count == 0, ("vm_page_alloc_init: page %p is held", m)); KASSERT(!vm_page_busied(m), ("vm_page_alloc_init: page %p is busy", m)); KASSERT(m->dirty == 0, ("vm_page_alloc_init: page %p is dirty", m)); KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("vm_page_alloc_init: page %p has unexpected memattr %d", m, pmap_page_get_memattr(m))); mtx_assert(&vm_page_queue_free_mtx, MA_OWNED); drop = NULL; if ((m->flags & PG_CACHED) != 0) { KASSERT((m->flags & PG_ZERO) == 0, ("vm_page_alloc_init: cached page %p is PG_ZERO", m)); m->valid = 0; m_object = m->object; vm_page_cache_remove(m); if (m_object->type == OBJT_VNODE && vm_object_cache_is_empty(m_object)) drop = m_object->handle; } else { KASSERT(m->valid == 0, ("vm_page_alloc_init: free page %p is valid", m)); vm_phys_freecnt_adj(m, -1); if ((m->flags & PG_ZERO) != 0) vm_page_zero_count--; } return (drop); } /* * vm_page_alloc_freelist: * * Allocate a physical page from the specified free page list. * * The caller must always specify an allocation class. * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs a page * VM_ALLOC_INTERRUPT interrupt time request * * optional allocation flags: * VM_ALLOC_COUNT(number) the number of additional pages that the caller * intends to allocate * VM_ALLOC_WIRED wire the allocated page * VM_ALLOC_ZERO prefer a zeroed page * * This routine may not sleep. */ vm_page_t vm_page_alloc_freelist(int flind, int req) { struct vnode *drop; vm_page_t m; u_int flags; int req_class; req_class = req & VM_ALLOC_CLASS_MASK; /* * The page daemon is allowed to dig deeper into the free page list. */ if (curproc == pageproc && req_class != VM_ALLOC_INTERRUPT) req_class = VM_ALLOC_SYSTEM; /* * Do not allocate reserved pages unless the req has asked for it. */ mtx_lock_flags(&vm_page_queue_free_mtx, MTX_RECURSE); if (vm_cnt.v_free_count + vm_cnt.v_cache_count > vm_cnt.v_free_reserved || (req_class == VM_ALLOC_SYSTEM && vm_cnt.v_free_count + vm_cnt.v_cache_count > vm_cnt.v_interrupt_free_min) || (req_class == VM_ALLOC_INTERRUPT && vm_cnt.v_free_count + vm_cnt.v_cache_count > 0)) m = vm_phys_alloc_freelist_pages(flind, VM_FREEPOOL_DIRECT, 0); else { mtx_unlock(&vm_page_queue_free_mtx); atomic_add_int(&vm_pageout_deficit, max((u_int)req >> VM_ALLOC_COUNT_SHIFT, 1)); pagedaemon_wakeup(); return (NULL); } if (m == NULL) { mtx_unlock(&vm_page_queue_free_mtx); return (NULL); } drop = vm_page_alloc_init(m); mtx_unlock(&vm_page_queue_free_mtx); /* * Initialize the page. Only the PG_ZERO flag is inherited. */ m->aflags = 0; flags = 0; if ((req & VM_ALLOC_ZERO) != 0) flags = PG_ZERO; m->flags &= flags; if ((req & VM_ALLOC_WIRED) != 0) { /* * The page lock is not required for wiring a page that does * not belong to an object. */ atomic_add_int(&vm_cnt.v_wire_count, 1); m->wire_count = 1; } /* Unmanaged pages don't use "act_count". */ m->oflags = VPO_UNMANAGED; if (drop != NULL) vdrop(drop); if (vm_paging_needed()) pagedaemon_wakeup(); return (m); } #define VPSC_ANY 0 /* No restrictions. */ #define VPSC_NORESERV 1 /* Skip reservations; implies VPSC_NOSUPER. */ #define VPSC_NOSUPER 2 /* Skip superpages. */ /* * vm_page_scan_contig: * * Scan vm_page_array[] between the specified entries "m_start" and * "m_end" for a run of contiguous physical pages that satisfy the * specified conditions, and return the lowest page in the run. The * specified "alignment" determines the alignment of the lowest physical * page in the run. If the specified "boundary" is non-zero, then the * run of physical pages cannot span a physical address that is a * multiple of "boundary". * * "m_end" is never dereferenced, so it need not point to a vm_page * structure within vm_page_array[]. * * "npages" must be greater than zero. "m_start" and "m_end" must not * span a hole (or discontiguity) in the physical address space. Both * "alignment" and "boundary" must be a power of two. */ vm_page_t vm_page_scan_contig(u_long npages, vm_page_t m_start, vm_page_t m_end, u_long alignment, vm_paddr_t boundary, int options) { struct mtx *m_mtx, *new_mtx; vm_object_t object; vm_paddr_t pa; vm_page_t m, m_run; #if VM_NRESERVLEVEL > 0 int level; #endif int m_inc, order, run_ext, run_len; KASSERT(npages > 0, ("npages is 0")); KASSERT(powerof2(alignment), ("alignment is not a power of 2")); KASSERT(powerof2(boundary), ("boundary is not a power of 2")); m_run = NULL; run_len = 0; m_mtx = NULL; for (m = m_start; m < m_end && run_len < npages; m += m_inc) { KASSERT((m->flags & (PG_FICTITIOUS | PG_MARKER)) == 0, ("page %p is PG_FICTITIOUS or PG_MARKER", m)); /* * If the current page would be the start of a run, check its * physical address against the end, alignment, and boundary * conditions. If it doesn't satisfy these conditions, either * terminate the scan or advance to the next page that * satisfies the failed condition. */ if (run_len == 0) { KASSERT(m_run == NULL, ("m_run != NULL")); if (m + npages > m_end) break; pa = VM_PAGE_TO_PHYS(m); if ((pa & (alignment - 1)) != 0) { m_inc = atop(roundup2(pa, alignment) - pa); continue; } if (rounddown2(pa ^ (pa + ptoa(npages) - 1), boundary) != 0) { m_inc = atop(roundup2(pa, boundary) - pa); continue; } } else KASSERT(m_run != NULL, ("m_run == NULL")); /* * Avoid releasing and reacquiring the same page lock. */ new_mtx = vm_page_lockptr(m); if (m_mtx != new_mtx) { if (m_mtx != NULL) mtx_unlock(m_mtx); m_mtx = new_mtx; mtx_lock(m_mtx); } m_inc = 1; retry: if (m->wire_count != 0 || m->hold_count != 0) run_ext = 0; #if VM_NRESERVLEVEL > 0 else if ((level = vm_reserv_level(m)) >= 0 && (options & VPSC_NORESERV) != 0) { run_ext = 0; /* Advance to the end of the reservation. */ pa = VM_PAGE_TO_PHYS(m); m_inc = atop(roundup2(pa + 1, vm_reserv_size(level)) - pa); } #endif else if ((object = m->object) != NULL) { /* * The page is considered eligible for relocation if * and only if it could be laundered or reclaimed by * the page daemon. */ if (!VM_OBJECT_TRYRLOCK(object)) { mtx_unlock(m_mtx); VM_OBJECT_RLOCK(object); mtx_lock(m_mtx); if (m->object != object) { /* * The page may have been freed. */ VM_OBJECT_RUNLOCK(object); goto retry; } else if (m->wire_count != 0 || m->hold_count != 0) { run_ext = 0; goto unlock; } } KASSERT((m->flags & PG_UNHOLDFREE) == 0, ("page %p is PG_UNHOLDFREE", m)); /* Don't care: PG_NODUMP, PG_WINATCFLS, PG_ZERO. */ if (object->type != OBJT_DEFAULT && object->type != OBJT_SWAP && object->type != OBJT_VNODE) run_ext = 0; else if ((m->flags & PG_CACHED) != 0 || m != vm_page_lookup(object, m->pindex)) { /* * The page is cached or recently converted * from cached to free. */ #if VM_NRESERVLEVEL > 0 if (level >= 0) { /* * The page is reserved. Extend the * current run by one page. */ run_ext = 1; } else #endif if ((order = m->order) < VM_NFREEORDER) { /* * The page is enqueued in the * physical memory allocator's cache/ * free page queues. Moreover, it is * the first page in a power-of-two- * sized run of contiguous cache/free * pages. Add these pages to the end * of the current run, and jump * ahead. */ run_ext = 1 << order; m_inc = 1 << order; } else run_ext = 0; #if VM_NRESERVLEVEL > 0 } else if ((options & VPSC_NOSUPER) != 0 && (level = vm_reserv_level_iffullpop(m)) >= 0) { run_ext = 0; /* Advance to the end of the superpage. */ pa = VM_PAGE_TO_PHYS(m); m_inc = atop(roundup2(pa + 1, vm_reserv_size(level)) - pa); #endif } else if (object->memattr == VM_MEMATTR_DEFAULT && m->queue != PQ_NONE && !vm_page_busied(m)) { /* * The page is allocated but eligible for * relocation. Extend the current run by one * page. */ KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("page %p has an unexpected memattr", m)); KASSERT((m->oflags & (VPO_SWAPINPROG | VPO_SWAPSLEEP | VPO_UNMANAGED)) == 0, ("page %p has unexpected oflags", m)); /* Don't care: VPO_NOSYNC. */ run_ext = 1; } else run_ext = 0; unlock: VM_OBJECT_RUNLOCK(object); #if VM_NRESERVLEVEL > 0 } else if (level >= 0) { /* * The page is reserved but not yet allocated. In * other words, it is still cached or free. Extend * the current run by one page. */ run_ext = 1; #endif } else if ((order = m->order) < VM_NFREEORDER) { /* * The page is enqueued in the physical memory * allocator's cache/free page queues. Moreover, it * is the first page in a power-of-two-sized run of * contiguous cache/free pages. Add these pages to * the end of the current run, and jump ahead. */ run_ext = 1 << order; m_inc = 1 << order; } else { /* * Skip the page for one of the following reasons: (1) * It is enqueued in the physical memory allocator's * cache/free page queues. However, it is not the * first page in a run of contiguous cache/free pages. * (This case rarely occurs because the scan is * performed in ascending order.) (2) It is not * reserved, and it is transitioning from free to * allocated. (Conversely, the transition from * allocated to free for managed pages is blocked by * the page lock.) (3) It is allocated but not * contained by an object and not wired, e.g., * allocated by Xen's balloon driver. */ run_ext = 0; } /* * Extend or reset the current run of pages. */ if (run_ext > 0) { if (run_len == 0) m_run = m; run_len += run_ext; } else { if (run_len > 0) { m_run = NULL; run_len = 0; } } } if (m_mtx != NULL) mtx_unlock(m_mtx); if (run_len >= npages) return (m_run); return (NULL); } /* * vm_page_reclaim_run: * * Try to relocate each of the allocated virtual pages within the * specified run of physical pages to a new physical address. Free the * physical pages underlying the relocated virtual pages. A virtual page * is relocatable if and only if it could be laundered or reclaimed by * the page daemon. Whenever possible, a virtual page is relocated to a * physical address above "high". * * Returns 0 if every physical page within the run was already free or * just freed by a successful relocation. Otherwise, returns a non-zero * value indicating why the last attempt to relocate a virtual page was * unsuccessful. * * "req_class" must be an allocation class. */ static int vm_page_reclaim_run(int req_class, u_long npages, vm_page_t m_run, vm_paddr_t high) { struct mtx *m_mtx, *new_mtx; struct spglist free; vm_object_t object; vm_paddr_t pa; vm_page_t m, m_end, m_new; int error, order, req; KASSERT((req_class & VM_ALLOC_CLASS_MASK) == req_class, ("req_class is not an allocation class")); SLIST_INIT(&free); error = 0; m = m_run; m_end = m_run + npages; m_mtx = NULL; for (; error == 0 && m < m_end; m++) { KASSERT((m->flags & (PG_FICTITIOUS | PG_MARKER)) == 0, ("page %p is PG_FICTITIOUS or PG_MARKER", m)); /* * Avoid releasing and reacquiring the same page lock. */ new_mtx = vm_page_lockptr(m); if (m_mtx != new_mtx) { if (m_mtx != NULL) mtx_unlock(m_mtx); m_mtx = new_mtx; mtx_lock(m_mtx); } retry: if (m->wire_count != 0 || m->hold_count != 0) error = EBUSY; else if ((object = m->object) != NULL) { /* * The page is relocated if and only if it could be * laundered or reclaimed by the page daemon. */ if (!VM_OBJECT_TRYWLOCK(object)) { mtx_unlock(m_mtx); VM_OBJECT_WLOCK(object); mtx_lock(m_mtx); if (m->object != object) { /* * The page may have been freed. */ VM_OBJECT_WUNLOCK(object); goto retry; } else if (m->wire_count != 0 || m->hold_count != 0) { error = EBUSY; goto unlock; } } KASSERT((m->flags & PG_UNHOLDFREE) == 0, ("page %p is PG_UNHOLDFREE", m)); /* Don't care: PG_NODUMP, PG_WINATCFLS, PG_ZERO. */ if (object->type != OBJT_DEFAULT && object->type != OBJT_SWAP && object->type != OBJT_VNODE) error = EINVAL; else if ((m->flags & PG_CACHED) != 0 || m != vm_page_lookup(object, m->pindex)) { /* * The page is cached or recently converted * from cached to free. */ VM_OBJECT_WUNLOCK(object); goto cached; } else if (object->memattr != VM_MEMATTR_DEFAULT) error = EINVAL; else if (m->queue != PQ_NONE && !vm_page_busied(m)) { KASSERT(pmap_page_get_memattr(m) == VM_MEMATTR_DEFAULT, ("page %p has an unexpected memattr", m)); KASSERT((m->oflags & (VPO_SWAPINPROG | VPO_SWAPSLEEP | VPO_UNMANAGED)) == 0, ("page %p has unexpected oflags", m)); /* Don't care: VPO_NOSYNC. */ if (m->valid != 0) { /* * First, try to allocate a new page * that is above "high". Failing * that, try to allocate a new page * that is below "m_run". Allocate * the new page between the end of * "m_run" and "high" only as a last * resort. */ req = req_class | VM_ALLOC_NOOBJ; if ((m->flags & PG_NODUMP) != 0) req |= VM_ALLOC_NODUMP; if (trunc_page(high) != ~(vm_paddr_t)PAGE_MASK) { m_new = vm_page_alloc_contig( NULL, 0, req, 1, round_page(high), ~(vm_paddr_t)0, PAGE_SIZE, 0, VM_MEMATTR_DEFAULT); } else m_new = NULL; if (m_new == NULL) { pa = VM_PAGE_TO_PHYS(m_run); m_new = vm_page_alloc_contig( NULL, 0, req, 1, 0, pa - 1, PAGE_SIZE, 0, VM_MEMATTR_DEFAULT); } if (m_new == NULL) { pa += ptoa(npages); m_new = vm_page_alloc_contig( NULL, 0, req, 1, pa, high, PAGE_SIZE, 0, VM_MEMATTR_DEFAULT); } if (m_new == NULL) { error = ENOMEM; goto unlock; } KASSERT(m_new->wire_count == 0, ("page %p is wired", m)); /* * Replace "m" with the new page. For * vm_page_replace(), "m" must be busy * and dequeued. Finally, change "m" * as if vm_page_free() was called. */ if (object->ref_count != 0) pmap_remove_all(m); m_new->aflags = m->aflags; KASSERT(m_new->oflags == VPO_UNMANAGED, ("page %p is managed", m)); m_new->oflags = m->oflags & VPO_NOSYNC; pmap_copy_page(m, m_new); m_new->valid = m->valid; m_new->dirty = m->dirty; m->flags &= ~PG_ZERO; vm_page_xbusy(m); vm_page_remque(m); vm_page_replace_checked(m_new, object, m->pindex, m); m->valid = 0; vm_page_undirty(m); /* * The new page must be deactivated * before the object is unlocked. */ new_mtx = vm_page_lockptr(m_new); if (m_mtx != new_mtx) { mtx_unlock(m_mtx); m_mtx = new_mtx; mtx_lock(m_mtx); } vm_page_deactivate(m_new); } else { m->flags &= ~PG_ZERO; vm_page_remque(m); vm_page_remove(m); KASSERT(m->dirty == 0, ("page %p is dirty", m)); } SLIST_INSERT_HEAD(&free, m, plinks.s.ss); } else error = EBUSY; unlock: VM_OBJECT_WUNLOCK(object); } else { cached: mtx_lock(&vm_page_queue_free_mtx); order = m->order; if (order < VM_NFREEORDER) { /* * The page is enqueued in the physical memory * allocator's cache/free page queues. * Moreover, it is the first page in a power- * of-two-sized run of contiguous cache/free * pages. Jump ahead to the last page within * that run, and continue from there. */ m += (1 << order) - 1; } #if VM_NRESERVLEVEL > 0 else if (vm_reserv_is_page_free(m)) order = 0; #endif mtx_unlock(&vm_page_queue_free_mtx); if (order == VM_NFREEORDER) error = EINVAL; } } if (m_mtx != NULL) mtx_unlock(m_mtx); if ((m = SLIST_FIRST(&free)) != NULL) { mtx_lock(&vm_page_queue_free_mtx); do { SLIST_REMOVE_HEAD(&free, plinks.s.ss); vm_phys_freecnt_adj(m, 1); #if VM_NRESERVLEVEL > 0 if (!vm_reserv_free_page(m)) #else if (true) #endif vm_phys_free_pages(m, 0); } while ((m = SLIST_FIRST(&free)) != NULL); vm_page_zero_idle_wakeup(); vm_page_free_wakeup(); mtx_unlock(&vm_page_queue_free_mtx); } return (error); } #define NRUNS 16 CTASSERT(powerof2(NRUNS)); #define RUN_INDEX(count) ((count) & (NRUNS - 1)) #define MIN_RECLAIM 8 /* * vm_page_reclaim_contig: * * Reclaim allocated, contiguous physical memory satisfying the specified * conditions by relocating the virtual pages using that physical memory. * Returns true if reclamation is successful and false otherwise. Since * relocation requires the allocation of physical pages, reclamation may * fail due to a shortage of cache/free pages. When reclamation fails, * callers are expected to perform VM_WAIT before retrying a failed * allocation operation, e.g., vm_page_alloc_contig(). * * The caller must always specify an allocation class through "req". * * allocation classes: * VM_ALLOC_NORMAL normal process request * VM_ALLOC_SYSTEM system *really* needs a page * VM_ALLOC_INTERRUPT interrupt time request * * The optional allocation flags are ignored. * * "npages" must be greater than zero. Both "alignment" and "boundary" * must be a power of two. */ bool vm_page_reclaim_contig(int req, u_long npages, vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary) { vm_paddr_t curr_low; vm_page_t m_run, m_runs[NRUNS]; u_long count, reclaimed; int error, i, options, req_class; KASSERT(npages > 0, ("npages is 0")); KASSERT(powerof2(alignment), ("alignment is not a power of 2")); KASSERT(powerof2(boundary), ("boundary is not a power of 2")); req_class = req & VM_ALLOC_CLASS_MASK; /* * The page daemon is allowed to dig deeper into the free page list. */ if (curproc == pageproc && req_class != VM_ALLOC_INTERRUPT) req_class = VM_ALLOC_SYSTEM; /* * Return if the number of cached and free pages cannot satisfy the * requested allocation. */ count = vm_cnt.v_free_count + vm_cnt.v_cache_count; if (count < npages + vm_cnt.v_free_reserved || (count < npages + vm_cnt.v_interrupt_free_min && req_class == VM_ALLOC_SYSTEM) || (count < npages && req_class == VM_ALLOC_INTERRUPT)) return (false); /* * Scan up to three times, relaxing the restrictions ("options") on * the reclamation of reservations and superpages each time. */ for (options = VPSC_NORESERV;;) { /* * Find the highest runs that satisfy the given constraints * and restrictions, and record them in "m_runs". */ curr_low = low; count = 0; for (;;) { m_run = vm_phys_scan_contig(npages, curr_low, high, alignment, boundary, options); if (m_run == NULL) break; curr_low = VM_PAGE_TO_PHYS(m_run) + ptoa(npages); m_runs[RUN_INDEX(count)] = m_run; count++; } /* * Reclaim the highest runs in LIFO (descending) order until * the number of reclaimed pages, "reclaimed", is at least * MIN_RECLAIM. Reset "reclaimed" each time because each * reclamation is idempotent, and runs will (likely) recur * from one scan to the next as restrictions are relaxed. */ reclaimed = 0; for (i = 0; count > 0 && i < NRUNS; i++) { count--; m_run = m_runs[RUN_INDEX(count)]; error = vm_page_reclaim_run(req_class, npages, m_run, high); if (error == 0) { reclaimed += npages; if (reclaimed >= MIN_RECLAIM) return (true); } } /* * Either relax the restrictions on the next scan or return if * the last scan had no restrictions. */ if (options == VPSC_NORESERV) options = VPSC_NOSUPER; else if (options == VPSC_NOSUPER) options = VPSC_ANY; else if (options == VPSC_ANY) return (reclaimed != 0); } } /* * vm_wait: (also see VM_WAIT macro) * * Sleep until free pages are available for allocation. * - Called in various places before memory allocations. */ void vm_wait(void) { mtx_lock(&vm_page_queue_free_mtx); if (curproc == pageproc) { vm_pageout_pages_needed = 1; msleep(&vm_pageout_pages_needed, &vm_page_queue_free_mtx, PDROP | PSWP, "VMWait", 0); } else { if (!vm_pageout_wanted) { vm_pageout_wanted = true; wakeup(&vm_pageout_wanted); } vm_pages_needed = true; msleep(&vm_cnt.v_free_count, &vm_page_queue_free_mtx, PDROP | PVM, "vmwait", 0); } } /* * vm_waitpfault: (also see VM_WAITPFAULT macro) * * Sleep until free pages are available for allocation. * - Called only in vm_fault so that processes page faulting * can be easily tracked. * - Sleeps at a lower priority than vm_wait() so that vm_wait()ing * processes will be able to grab memory first. Do not change * this balance without careful testing first. */ void vm_waitpfault(void) { mtx_lock(&vm_page_queue_free_mtx); if (!vm_pageout_wanted) { vm_pageout_wanted = true; wakeup(&vm_pageout_wanted); } vm_pages_needed = true; msleep(&vm_cnt.v_free_count, &vm_page_queue_free_mtx, PDROP | PUSER, "pfault", 0); } struct vm_pagequeue * vm_page_pagequeue(vm_page_t m) { return (&vm_phys_domain(m)->vmd_pagequeues[m->queue]); } /* * vm_page_dequeue: * * Remove the given page from its current page queue. * * The page must be locked. */ void vm_page_dequeue(vm_page_t m) { struct vm_pagequeue *pq; vm_page_assert_locked(m); KASSERT(m->queue < PQ_COUNT, ("vm_page_dequeue: page %p is not queued", m)); pq = vm_page_pagequeue(m); vm_pagequeue_lock(pq); m->queue = PQ_NONE; TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); vm_pagequeue_cnt_dec(pq); vm_pagequeue_unlock(pq); } /* * vm_page_dequeue_locked: * * Remove the given page from its current page queue. * * The page and page queue must be locked. */ void vm_page_dequeue_locked(vm_page_t m) { struct vm_pagequeue *pq; vm_page_lock_assert(m, MA_OWNED); pq = vm_page_pagequeue(m); vm_pagequeue_assert_locked(pq); m->queue = PQ_NONE; TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); vm_pagequeue_cnt_dec(pq); } /* * vm_page_enqueue: * * Add the given page to the specified page queue. * * The page must be locked. */ static void vm_page_enqueue(uint8_t queue, vm_page_t m) { struct vm_pagequeue *pq; vm_page_lock_assert(m, MA_OWNED); KASSERT(queue < PQ_COUNT, ("vm_page_enqueue: invalid queue %u request for page %p", queue, m)); pq = &vm_phys_domain(m)->vmd_pagequeues[queue]; vm_pagequeue_lock(pq); m->queue = queue; TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); vm_pagequeue_cnt_inc(pq); vm_pagequeue_unlock(pq); } /* * vm_page_requeue: * * Move the given page to the tail of its current page queue. * * The page must be locked. */ void vm_page_requeue(vm_page_t m) { struct vm_pagequeue *pq; vm_page_lock_assert(m, MA_OWNED); KASSERT(m->queue != PQ_NONE, ("vm_page_requeue: page %p is not queued", m)); pq = vm_page_pagequeue(m); vm_pagequeue_lock(pq); TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); vm_pagequeue_unlock(pq); } /* * vm_page_requeue_locked: * * Move the given page to the tail of its current page queue. * * The page queue must be locked. */ void vm_page_requeue_locked(vm_page_t m) { struct vm_pagequeue *pq; KASSERT(m->queue != PQ_NONE, ("vm_page_requeue_locked: page %p is not queued", m)); pq = vm_page_pagequeue(m); vm_pagequeue_assert_locked(pq); TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); } /* * vm_page_activate: * * Put the specified page on the active list (if appropriate). * Ensure that act_count is at least ACT_INIT but do not otherwise * mess with it. * * The page must be locked. */ void vm_page_activate(vm_page_t m) { int queue; vm_page_lock_assert(m, MA_OWNED); if ((queue = m->queue) != PQ_ACTIVE) { if (m->wire_count == 0 && (m->oflags & VPO_UNMANAGED) == 0) { if (m->act_count < ACT_INIT) m->act_count = ACT_INIT; if (queue != PQ_NONE) vm_page_dequeue(m); vm_page_enqueue(PQ_ACTIVE, m); } else KASSERT(queue == PQ_NONE, ("vm_page_activate: wired page %p is queued", m)); } else { if (m->act_count < ACT_INIT) m->act_count = ACT_INIT; } } /* * vm_page_free_wakeup: * * Helper routine for vm_page_free_toq() and vm_page_cache(). This * routine is called when a page has been added to the cache or free * queues. * * The page queues must be locked. */ static inline void vm_page_free_wakeup(void) { mtx_assert(&vm_page_queue_free_mtx, MA_OWNED); /* * if pageout daemon needs pages, then tell it that there are * some free. */ if (vm_pageout_pages_needed && vm_cnt.v_cache_count + vm_cnt.v_free_count >= vm_cnt.v_pageout_free_min) { wakeup(&vm_pageout_pages_needed); vm_pageout_pages_needed = 0; } /* * wakeup processes that are waiting on memory if we hit a * high water mark. And wakeup scheduler process if we have * lots of memory. this process will swapin processes. */ if (vm_pages_needed && !vm_page_count_min()) { vm_pages_needed = false; wakeup(&vm_cnt.v_free_count); } } /* * Turn a cached page into a free page, by changing its attributes. * Keep the statistics up-to-date. * * The free page queue must be locked. */ static void vm_page_cache_turn_free(vm_page_t m) { mtx_assert(&vm_page_queue_free_mtx, MA_OWNED); m->object = NULL; m->valid = 0; KASSERT((m->flags & PG_CACHED) != 0, ("vm_page_cache_turn_free: page %p is not cached", m)); m->flags &= ~PG_CACHED; vm_cnt.v_cache_count--; vm_phys_freecnt_adj(m, 1); } /* * vm_page_free_toq: * * Returns the given page to the free list, * disassociating it with any VM object. * * The object must be locked. The page must be locked if it is managed. */ void vm_page_free_toq(vm_page_t m) { if ((m->oflags & VPO_UNMANAGED) == 0) { vm_page_lock_assert(m, MA_OWNED); KASSERT(!pmap_page_is_mapped(m), ("vm_page_free_toq: freeing mapped page %p", m)); } else KASSERT(m->queue == PQ_NONE, ("vm_page_free_toq: unmanaged page %p is queued", m)); PCPU_INC(cnt.v_tfree); if (vm_page_sbusied(m)) panic("vm_page_free: freeing busy page %p", m); /* * Unqueue, then remove page. Note that we cannot destroy * the page here because we do not want to call the pager's * callback routine until after we've put the page on the * appropriate free queue. */ vm_page_remque(m); vm_page_remove(m); /* * If fictitious remove object association and * return, otherwise delay object association removal. */ if ((m->flags & PG_FICTITIOUS) != 0) { return; } m->valid = 0; vm_page_undirty(m); if (m->wire_count != 0) panic("vm_page_free: freeing wired page %p", m); if (m->hold_count != 0) { m->flags &= ~PG_ZERO; KASSERT((m->flags & PG_UNHOLDFREE) == 0, ("vm_page_free: freeing PG_UNHOLDFREE page %p", m)); m->flags |= PG_UNHOLDFREE; } else { /* * Restore the default memory attribute to the page. */ if (pmap_page_get_memattr(m) != VM_MEMATTR_DEFAULT) pmap_page_set_memattr(m, VM_MEMATTR_DEFAULT); /* * Insert the page into the physical memory allocator's * cache/free page queues. */ mtx_lock(&vm_page_queue_free_mtx); vm_phys_freecnt_adj(m, 1); #if VM_NRESERVLEVEL > 0 if (!vm_reserv_free_page(m)) #else if (TRUE) #endif vm_phys_free_pages(m, 0); if ((m->flags & PG_ZERO) != 0) ++vm_page_zero_count; else vm_page_zero_idle_wakeup(); vm_page_free_wakeup(); mtx_unlock(&vm_page_queue_free_mtx); } } /* * vm_page_wire: * * Mark this page as wired down by yet * another map, removing it from paging queues * as necessary. * * If the page is fictitious, then its wire count must remain one. * * The page must be locked. */ void vm_page_wire(vm_page_t m) { /* * Only bump the wire statistics if the page is not already wired, * and only unqueue the page if it is on some queue (if it is unmanaged * it is already off the queues). */ vm_page_lock_assert(m, MA_OWNED); if ((m->flags & PG_FICTITIOUS) != 0) { KASSERT(m->wire_count == 1, ("vm_page_wire: fictitious page %p's wire count isn't one", m)); return; } if (m->wire_count == 0) { KASSERT((m->oflags & VPO_UNMANAGED) == 0 || m->queue == PQ_NONE, ("vm_page_wire: unmanaged page %p is queued", m)); vm_page_remque(m); atomic_add_int(&vm_cnt.v_wire_count, 1); } m->wire_count++; KASSERT(m->wire_count != 0, ("vm_page_wire: wire_count overflow m=%p", m)); } /* * vm_page_unwire: * * Release one wiring of the specified page, potentially allowing it to be * paged out. Returns TRUE if the number of wirings transitions to zero and * FALSE otherwise. * * Only managed pages belonging to an object can be paged out. If the number * of wirings transitions to zero and the page is eligible for page out, then * the page is added to the specified paging queue (unless PQ_NONE is * specified). * * If a page is fictitious, then its wire count must always be one. * * A managed page must be locked. */ boolean_t vm_page_unwire(vm_page_t m, uint8_t queue) { KASSERT(queue < PQ_COUNT || queue == PQ_NONE, ("vm_page_unwire: invalid queue %u request for page %p", queue, m)); if ((m->oflags & VPO_UNMANAGED) == 0) vm_page_assert_locked(m); if ((m->flags & PG_FICTITIOUS) != 0) { KASSERT(m->wire_count == 1, ("vm_page_unwire: fictitious page %p's wire count isn't one", m)); return (FALSE); } if (m->wire_count > 0) { m->wire_count--; if (m->wire_count == 0) { atomic_subtract_int(&vm_cnt.v_wire_count, 1); if ((m->oflags & VPO_UNMANAGED) == 0 && m->object != NULL && queue != PQ_NONE) { if (queue == PQ_INACTIVE) m->flags &= ~PG_WINATCFLS; vm_page_enqueue(queue, m); } return (TRUE); } else return (FALSE); } else panic("vm_page_unwire: page %p's wire count is zero", m); } /* * Move the specified page to the inactive queue. * * Many pages placed on the inactive queue should actually go * into the cache, but it is difficult to figure out which. What * we do instead, if the inactive target is well met, is to put * clean pages at the head of the inactive queue instead of the tail. * This will cause them to be moved to the cache more quickly and * if not actively re-referenced, reclaimed more quickly. If we just * stick these pages at the end of the inactive queue, heavy filesystem * meta-data accesses can cause an unnecessary paging load on memory bound * processes. This optimization causes one-time-use metadata to be * reused more quickly. * * Normally noreuse is FALSE, resulting in LRU operation. noreuse is set * to TRUE if we want this page to be 'as if it were placed in the cache', * except without unmapping it from the process address space. In * practice this is implemented by inserting the page at the head of the * queue, using a marker page to guide FIFO insertion ordering. * * The page must be locked. */ static inline void _vm_page_deactivate(vm_page_t m, boolean_t noreuse) { struct vm_pagequeue *pq; int queue; vm_page_assert_locked(m); /* * Ignore if the page is already inactive, unless it is unlikely to be * reactivated. */ if ((queue = m->queue) == PQ_INACTIVE && !noreuse) return; if (m->wire_count == 0 && (m->oflags & VPO_UNMANAGED) == 0) { pq = &vm_phys_domain(m)->vmd_pagequeues[PQ_INACTIVE]; /* Avoid multiple acquisitions of the inactive queue lock. */ if (queue == PQ_INACTIVE) { vm_pagequeue_lock(pq); vm_page_dequeue_locked(m); } else { if (queue != PQ_NONE) vm_page_dequeue(m); m->flags &= ~PG_WINATCFLS; vm_pagequeue_lock(pq); } m->queue = PQ_INACTIVE; if (noreuse) TAILQ_INSERT_BEFORE(&vm_phys_domain(m)->vmd_inacthead, m, plinks.q); else TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); vm_pagequeue_cnt_inc(pq); vm_pagequeue_unlock(pq); } } /* * Move the specified page to the inactive queue. * * The page must be locked. */ void vm_page_deactivate(vm_page_t m) { _vm_page_deactivate(m, FALSE); } /* * Move the specified page to the inactive queue with the expectation * that it is unlikely to be reused. * * The page must be locked. */ void vm_page_deactivate_noreuse(vm_page_t m) { _vm_page_deactivate(m, TRUE); } /* * vm_page_try_to_cache: * * Returns 0 on failure, 1 on success */ int vm_page_try_to_cache(vm_page_t m) { vm_page_lock_assert(m, MA_OWNED); VM_OBJECT_ASSERT_WLOCKED(m->object); if (m->dirty || m->hold_count || m->wire_count || (m->oflags & VPO_UNMANAGED) != 0 || vm_page_busied(m)) return (0); pmap_remove_all(m); if (m->dirty) return (0); vm_page_cache(m); return (1); } /* * vm_page_try_to_free() * * Attempt to free the page. If we cannot free it, we do nothing. * 1 is returned on success, 0 on failure. */ int vm_page_try_to_free(vm_page_t m) { vm_page_lock_assert(m, MA_OWNED); if (m->object != NULL) VM_OBJECT_ASSERT_WLOCKED(m->object); if (m->dirty || m->hold_count || m->wire_count || (m->oflags & VPO_UNMANAGED) != 0 || vm_page_busied(m)) return (0); pmap_remove_all(m); if (m->dirty) return (0); vm_page_free(m); return (1); } /* * vm_page_cache * * Put the specified page onto the page cache queue (if appropriate). * * The object and page must be locked. */ void vm_page_cache(vm_page_t m) { vm_object_t object; boolean_t cache_was_empty; vm_page_lock_assert(m, MA_OWNED); object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); if (vm_page_busied(m) || (m->oflags & VPO_UNMANAGED) || m->hold_count || m->wire_count) panic("vm_page_cache: attempting to cache busy page"); KASSERT(!pmap_page_is_mapped(m), ("vm_page_cache: page %p is mapped", m)); KASSERT(m->dirty == 0, ("vm_page_cache: page %p is dirty", m)); if (m->valid == 0 || object->type == OBJT_DEFAULT || (object->type == OBJT_SWAP && !vm_pager_has_page(object, m->pindex, NULL, NULL))) { /* * Hypothesis: A cache-eligible page belonging to a * default object or swap object but without a backing * store must be zero filled. */ vm_page_free(m); return; } KASSERT((m->flags & PG_CACHED) == 0, ("vm_page_cache: page %p is already cached", m)); /* * Remove the page from the paging queues. */ vm_page_remque(m); /* * Remove the page from the object's collection of resident * pages. */ vm_radix_remove(&object->rtree, m->pindex); TAILQ_REMOVE(&object->memq, m, listq); object->resident_page_count--; /* * Restore the default memory attribute to the page. */ if (pmap_page_get_memattr(m) != VM_MEMATTR_DEFAULT) pmap_page_set_memattr(m, VM_MEMATTR_DEFAULT); /* * Insert the page into the object's collection of cached pages * and the physical memory allocator's cache/free page queues. */ m->flags &= ~PG_ZERO; mtx_lock(&vm_page_queue_free_mtx); cache_was_empty = vm_radix_is_empty(&object->cache); if (vm_radix_insert(&object->cache, m)) { mtx_unlock(&vm_page_queue_free_mtx); if (object->type == OBJT_VNODE && object->resident_page_count == 0) vdrop(object->handle); m->object = NULL; vm_page_free(m); return; } /* * The above call to vm_radix_insert() could reclaim the one pre- * existing cached page from this object, resulting in a call to * vdrop(). */ if (!cache_was_empty) cache_was_empty = vm_radix_is_singleton(&object->cache); m->flags |= PG_CACHED; vm_cnt.v_cache_count++; PCPU_INC(cnt.v_tcached); #if VM_NRESERVLEVEL > 0 if (!vm_reserv_free_page(m)) { #else if (TRUE) { #endif vm_phys_free_pages(m, 0); } vm_page_free_wakeup(); mtx_unlock(&vm_page_queue_free_mtx); /* * Increment the vnode's hold count if this is the object's only * cached page. Decrement the vnode's hold count if this was * the object's only resident page. */ if (object->type == OBJT_VNODE) { if (cache_was_empty && object->resident_page_count != 0) vhold(object->handle); else if (!cache_was_empty && object->resident_page_count == 0) vdrop(object->handle); } } /* * vm_page_advise * * Deactivate or do nothing, as appropriate. * * The object and page must be locked. */ void vm_page_advise(vm_page_t m, int advice) { vm_page_assert_locked(m); VM_OBJECT_ASSERT_WLOCKED(m->object); if (advice == MADV_FREE) /* * Mark the page clean. This will allow the page to be freed * up by the system. However, such pages are often reused * quickly by malloc() so we do not do anything that would * cause a page fault if we can help it. * * Specifically, we do not try to actually free the page now * nor do we try to put it in the cache (which would cause a * page fault on reuse). * * But we do make the page as freeable as we can without * actually taking the step of unmapping it. */ vm_page_undirty(m); else if (advice != MADV_DONTNEED) return; /* * Clear any references to the page. Otherwise, the page daemon will * immediately reactivate the page. */ vm_page_aflag_clear(m, PGA_REFERENCED); if (advice != MADV_FREE && m->dirty == 0 && pmap_is_modified(m)) vm_page_dirty(m); /* * Place clean pages near the head of the inactive queue rather than * the tail, thus defeating the queue's LRU operation and ensuring that * the page will be reused quickly. Dirty pages are given a chance to * cycle once through the inactive queue before becoming eligible for * laundering. */ _vm_page_deactivate(m, m->dirty == 0); } /* * Grab a page, waiting until we are waken up due to the page * changing state. We keep on waiting, if the page continues * to be in the object. If the page doesn't exist, first allocate it * and then conditionally zero it. * * This routine may sleep. * * The object must be locked on entry. The lock will, however, be released * and reacquired if the routine sleeps. */ vm_page_t vm_page_grab(vm_object_t object, vm_pindex_t pindex, int allocflags) { vm_page_t m; int sleep; VM_OBJECT_ASSERT_WLOCKED(object); KASSERT((allocflags & VM_ALLOC_SBUSY) == 0 || (allocflags & VM_ALLOC_IGN_SBUSY) != 0, ("vm_page_grab: VM_ALLOC_SBUSY/VM_ALLOC_IGN_SBUSY mismatch")); retrylookup: if ((m = vm_page_lookup(object, pindex)) != NULL) { sleep = (allocflags & VM_ALLOC_IGN_SBUSY) != 0 ? vm_page_xbusied(m) : vm_page_busied(m); if (sleep) { if ((allocflags & VM_ALLOC_NOWAIT) != 0) return (NULL); /* * Reference the page before unlocking and * sleeping so that the page daemon is less * likely to reclaim it. */ vm_page_aflag_set(m, PGA_REFERENCED); vm_page_lock(m); VM_OBJECT_WUNLOCK(object); vm_page_busy_sleep(m, "pgrbwt"); VM_OBJECT_WLOCK(object); goto retrylookup; } else { if ((allocflags & VM_ALLOC_WIRED) != 0) { vm_page_lock(m); vm_page_wire(m); vm_page_unlock(m); } if ((allocflags & (VM_ALLOC_NOBUSY | VM_ALLOC_SBUSY)) == 0) vm_page_xbusy(m); if ((allocflags & VM_ALLOC_SBUSY) != 0) vm_page_sbusy(m); return (m); } } m = vm_page_alloc(object, pindex, allocflags); if (m == NULL) { if ((allocflags & VM_ALLOC_NOWAIT) != 0) return (NULL); VM_OBJECT_WUNLOCK(object); VM_WAIT; VM_OBJECT_WLOCK(object); goto retrylookup; } else if (m->valid != 0) return (m); if (allocflags & VM_ALLOC_ZERO && (m->flags & PG_ZERO) == 0) pmap_zero_page(m); return (m); } /* * Mapping function for valid or dirty bits in a page. * * Inputs are required to range within a page. */ vm_page_bits_t vm_page_bits(int base, int size) { int first_bit; int last_bit; KASSERT( base + size <= PAGE_SIZE, ("vm_page_bits: illegal base/size %d/%d", base, size) ); if (size == 0) /* handle degenerate case */ return (0); first_bit = base >> DEV_BSHIFT; last_bit = (base + size - 1) >> DEV_BSHIFT; return (((vm_page_bits_t)2 << last_bit) - ((vm_page_bits_t)1 << first_bit)); } /* * vm_page_set_valid_range: * * Sets portions of a page valid. The arguments are expected * to be DEV_BSIZE aligned but if they aren't the bitmap is inclusive * of any partial chunks touched by the range. The invalid portion of * such chunks will be zeroed. * * (base + size) must be less then or equal to PAGE_SIZE. */ void vm_page_set_valid_range(vm_page_t m, int base, int size) { int endoff, frag; VM_OBJECT_ASSERT_WLOCKED(m->object); if (size == 0) /* handle degenerate case */ return; /* * If the base is not DEV_BSIZE aligned and the valid * bit is clear, we have to zero out a portion of the * first block. */ if ((frag = rounddown2(base, DEV_BSIZE)) != base && (m->valid & (1 << (base >> DEV_BSHIFT))) == 0) pmap_zero_page_area(m, frag, base - frag); /* * If the ending offset is not DEV_BSIZE aligned and the * valid bit is clear, we have to zero out a portion of * the last block. */ endoff = base + size; if ((frag = rounddown2(endoff, DEV_BSIZE)) != endoff && (m->valid & (1 << (endoff >> DEV_BSHIFT))) == 0) pmap_zero_page_area(m, endoff, DEV_BSIZE - (endoff & (DEV_BSIZE - 1))); /* * Assert that no previously invalid block that is now being validated * is already dirty. */ KASSERT((~m->valid & vm_page_bits(base, size) & m->dirty) == 0, ("vm_page_set_valid_range: page %p is dirty", m)); /* * Set valid bits inclusive of any overlap. */ m->valid |= vm_page_bits(base, size); } /* * Clear the given bits from the specified page's dirty field. */ static __inline void vm_page_clear_dirty_mask(vm_page_t m, vm_page_bits_t pagebits) { uintptr_t addr; #if PAGE_SIZE < 16384 int shift; #endif /* * If the object is locked and the page is neither exclusive busy nor * write mapped, then the page's dirty field cannot possibly be * set by a concurrent pmap operation. */ VM_OBJECT_ASSERT_WLOCKED(m->object); if (!vm_page_xbusied(m) && !pmap_page_is_write_mapped(m)) m->dirty &= ~pagebits; else { /* * The pmap layer can call vm_page_dirty() without * holding a distinguished lock. The combination of * the object's lock and an atomic operation suffice * to guarantee consistency of the page dirty field. * * For PAGE_SIZE == 32768 case, compiler already * properly aligns the dirty field, so no forcible * alignment is needed. Only require existence of * atomic_clear_64 when page size is 32768. */ addr = (uintptr_t)&m->dirty; #if PAGE_SIZE == 32768 atomic_clear_64((uint64_t *)addr, pagebits); #elif PAGE_SIZE == 16384 atomic_clear_32((uint32_t *)addr, pagebits); #else /* PAGE_SIZE <= 8192 */ /* * Use a trick to perform a 32-bit atomic on the * containing aligned word, to not depend on the existence * of atomic_clear_{8, 16}. */ shift = addr & (sizeof(uint32_t) - 1); #if BYTE_ORDER == BIG_ENDIAN shift = (sizeof(uint32_t) - sizeof(m->dirty) - shift) * NBBY; #else shift *= NBBY; #endif addr &= ~(sizeof(uint32_t) - 1); atomic_clear_32((uint32_t *)addr, pagebits << shift); #endif /* PAGE_SIZE */ } } /* * vm_page_set_validclean: * * Sets portions of a page valid and clean. The arguments are expected * to be DEV_BSIZE aligned but if they aren't the bitmap is inclusive * of any partial chunks touched by the range. The invalid portion of * such chunks will be zero'd. * * (base + size) must be less then or equal to PAGE_SIZE. */ void vm_page_set_validclean(vm_page_t m, int base, int size) { vm_page_bits_t oldvalid, pagebits; int endoff, frag; VM_OBJECT_ASSERT_WLOCKED(m->object); if (size == 0) /* handle degenerate case */ return; /* * If the base is not DEV_BSIZE aligned and the valid * bit is clear, we have to zero out a portion of the * first block. */ if ((frag = rounddown2(base, DEV_BSIZE)) != base && (m->valid & ((vm_page_bits_t)1 << (base >> DEV_BSHIFT))) == 0) pmap_zero_page_area(m, frag, base - frag); /* * If the ending offset is not DEV_BSIZE aligned and the * valid bit is clear, we have to zero out a portion of * the last block. */ endoff = base + size; if ((frag = rounddown2(endoff, DEV_BSIZE)) != endoff && (m->valid & ((vm_page_bits_t)1 << (endoff >> DEV_BSHIFT))) == 0) pmap_zero_page_area(m, endoff, DEV_BSIZE - (endoff & (DEV_BSIZE - 1))); /* * Set valid, clear dirty bits. If validating the entire * page we can safely clear the pmap modify bit. We also * use this opportunity to clear the VPO_NOSYNC flag. If a process * takes a write fault on a MAP_NOSYNC memory area the flag will * be set again. * * We set valid bits inclusive of any overlap, but we can only * clear dirty bits for DEV_BSIZE chunks that are fully within * the range. */ oldvalid = m->valid; pagebits = vm_page_bits(base, size); m->valid |= pagebits; #if 0 /* NOT YET */ if ((frag = base & (DEV_BSIZE - 1)) != 0) { frag = DEV_BSIZE - frag; base += frag; size -= frag; if (size < 0) size = 0; } pagebits = vm_page_bits(base, size & (DEV_BSIZE - 1)); #endif if (base == 0 && size == PAGE_SIZE) { /* * The page can only be modified within the pmap if it is * mapped, and it can only be mapped if it was previously * fully valid. */ if (oldvalid == VM_PAGE_BITS_ALL) /* * Perform the pmap_clear_modify() first. Otherwise, * a concurrent pmap operation, such as * pmap_protect(), could clear a modification in the * pmap and set the dirty field on the page before * pmap_clear_modify() had begun and after the dirty * field was cleared here. */ pmap_clear_modify(m); m->dirty = 0; m->oflags &= ~VPO_NOSYNC; } else if (oldvalid != VM_PAGE_BITS_ALL) m->dirty &= ~pagebits; else vm_page_clear_dirty_mask(m, pagebits); } void vm_page_clear_dirty(vm_page_t m, int base, int size) { vm_page_clear_dirty_mask(m, vm_page_bits(base, size)); } /* * vm_page_set_invalid: * * Invalidates DEV_BSIZE'd chunks within a page. Both the * valid and dirty bits for the effected areas are cleared. */ void vm_page_set_invalid(vm_page_t m, int base, int size) { vm_page_bits_t bits; vm_object_t object; object = m->object; VM_OBJECT_ASSERT_WLOCKED(object); if (object->type == OBJT_VNODE && base == 0 && IDX_TO_OFF(m->pindex) + size >= object->un_pager.vnp.vnp_size) bits = VM_PAGE_BITS_ALL; else bits = vm_page_bits(base, size); if (object->ref_count != 0 && m->valid == VM_PAGE_BITS_ALL && bits != 0) pmap_remove_all(m); KASSERT((bits == 0 && m->valid == VM_PAGE_BITS_ALL) || !pmap_page_is_mapped(m), ("vm_page_set_invalid: page %p is mapped", m)); m->valid &= ~bits; m->dirty &= ~bits; } /* * vm_page_zero_invalid() * * The kernel assumes that the invalid portions of a page contain * garbage, but such pages can be mapped into memory by user code. * When this occurs, we must zero out the non-valid portions of the * page so user code sees what it expects. * * Pages are most often semi-valid when the end of a file is mapped * into memory and the file's size is not page aligned. */ void vm_page_zero_invalid(vm_page_t m, boolean_t setvalid) { int b; int i; VM_OBJECT_ASSERT_WLOCKED(m->object); /* * Scan the valid bits looking for invalid sections that * must be zeroed. Invalid sub-DEV_BSIZE'd areas ( where the * valid bit may be set ) have already been zeroed by * vm_page_set_validclean(). */ for (b = i = 0; i <= PAGE_SIZE / DEV_BSIZE; ++i) { if (i == (PAGE_SIZE / DEV_BSIZE) || (m->valid & ((vm_page_bits_t)1 << i))) { if (i > b) { pmap_zero_page_area(m, b << DEV_BSHIFT, (i - b) << DEV_BSHIFT); } b = i + 1; } } /* * setvalid is TRUE when we can safely set the zero'd areas * as being valid. We can do this if there are no cache consistancy * issues. e.g. it is ok to do with UFS, but not ok to do with NFS. */ if (setvalid) m->valid = VM_PAGE_BITS_ALL; } /* * vm_page_is_valid: * * Is (partial) page valid? Note that the case where size == 0 * will return FALSE in the degenerate case where the page is * entirely invalid, and TRUE otherwise. */ int vm_page_is_valid(vm_page_t m, int base, int size) { vm_page_bits_t bits; VM_OBJECT_ASSERT_LOCKED(m->object); bits = vm_page_bits(base, size); return (m->valid != 0 && (m->valid & bits) == bits); } /* * vm_page_ps_is_valid: * * Returns TRUE if the entire (super)page is valid and FALSE otherwise. */ boolean_t vm_page_ps_is_valid(vm_page_t m) { int i, npages; VM_OBJECT_ASSERT_LOCKED(m->object); npages = atop(pagesizes[m->psind]); /* * The physically contiguous pages that make up a superpage, i.e., a * page with a page size index ("psind") greater than zero, will * occupy adjacent entries in vm_page_array[]. */ for (i = 0; i < npages; i++) { if (m[i].valid != VM_PAGE_BITS_ALL) return (FALSE); } return (TRUE); } /* * Set the page's dirty bits if the page is modified. */ void vm_page_test_dirty(vm_page_t m) { VM_OBJECT_ASSERT_WLOCKED(m->object); if (m->dirty != VM_PAGE_BITS_ALL && pmap_is_modified(m)) vm_page_dirty(m); } void vm_page_lock_KBI(vm_page_t m, const char *file, int line) { mtx_lock_flags_(vm_page_lockptr(m), 0, file, line); } void vm_page_unlock_KBI(vm_page_t m, const char *file, int line) { mtx_unlock_flags_(vm_page_lockptr(m), 0, file, line); } int vm_page_trylock_KBI(vm_page_t m, const char *file, int line) { return (mtx_trylock_flags_(vm_page_lockptr(m), 0, file, line)); } #if defined(INVARIANTS) || defined(INVARIANT_SUPPORT) void vm_page_assert_locked_KBI(vm_page_t m, const char *file, int line) { vm_page_lock_assert_KBI(m, MA_OWNED, file, line); } void vm_page_lock_assert_KBI(vm_page_t m, int a, const char *file, int line) { mtx_assert_(vm_page_lockptr(m), a, file, line); } #endif #ifdef INVARIANTS void vm_page_object_lock_assert(vm_page_t m) { /* * Certain of the page's fields may only be modified by the * holder of the containing object's lock or the exclusive busy. * holder. Unfortunately, the holder of the write busy is * not recorded, and thus cannot be checked here. */ if (m->object != NULL && !vm_page_xbusied(m)) VM_OBJECT_ASSERT_WLOCKED(m->object); } void vm_page_assert_pga_writeable(vm_page_t m, uint8_t bits) { if ((bits & PGA_WRITEABLE) == 0) return; /* * The PGA_WRITEABLE flag can only be set if the page is * managed, is exclusively busied or the object is locked. * Currently, this flag is only set by pmap_enter(). */ KASSERT((m->oflags & VPO_UNMANAGED) == 0, ("PGA_WRITEABLE on unmanaged page")); if (!vm_page_xbusied(m)) VM_OBJECT_ASSERT_LOCKED(m->object); } #endif #include "opt_ddb.h" #ifdef DDB #include #include DB_SHOW_COMMAND(page, vm_page_print_page_info) { db_printf("vm_cnt.v_free_count: %d\n", vm_cnt.v_free_count); db_printf("vm_cnt.v_cache_count: %d\n", vm_cnt.v_cache_count); db_printf("vm_cnt.v_inactive_count: %d\n", vm_cnt.v_inactive_count); db_printf("vm_cnt.v_active_count: %d\n", vm_cnt.v_active_count); db_printf("vm_cnt.v_wire_count: %d\n", vm_cnt.v_wire_count); db_printf("vm_cnt.v_free_reserved: %d\n", vm_cnt.v_free_reserved); db_printf("vm_cnt.v_free_min: %d\n", vm_cnt.v_free_min); db_printf("vm_cnt.v_free_target: %d\n", vm_cnt.v_free_target); db_printf("vm_cnt.v_inactive_target: %d\n", vm_cnt.v_inactive_target); } DB_SHOW_COMMAND(pageq, vm_page_print_pageq_info) { int dom; db_printf("pq_free %d pq_cache %d\n", vm_cnt.v_free_count, vm_cnt.v_cache_count); for (dom = 0; dom < vm_ndomains; dom++) { db_printf( "dom %d page_cnt %d free %d pq_act %d pq_inact %d pass %d\n", dom, vm_dom[dom].vmd_page_count, vm_dom[dom].vmd_free_count, vm_dom[dom].vmd_pagequeues[PQ_ACTIVE].pq_cnt, vm_dom[dom].vmd_pagequeues[PQ_INACTIVE].pq_cnt, vm_dom[dom].vmd_pass); } } DB_SHOW_COMMAND(pginfo, vm_page_print_pginfo) { vm_page_t m; boolean_t phys; if (!have_addr) { db_printf("show pginfo addr\n"); return; } phys = strchr(modif, 'p') != NULL; if (phys) m = PHYS_TO_VM_PAGE(addr); else m = (vm_page_t)addr; db_printf( "page %p obj %p pidx 0x%jx phys 0x%jx q %d hold %d wire %d\n" " af 0x%x of 0x%x f 0x%x act %d busy %x valid 0x%x dirty 0x%x\n", m, m->object, (uintmax_t)m->pindex, (uintmax_t)m->phys_addr, m->queue, m->hold_count, m->wire_count, m->aflags, m->oflags, m->flags, m->act_count, m->busy_lock, m->valid, m->dirty); } #endif /* DDB */ Index: projects/netbsd-tests-update-12/targets/pseudo/tests/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/targets/pseudo/tests/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/targets/pseudo/tests/Makefile.depend (revision 305172) @@ -1,338 +1,345 @@ # $FreeBSD$ # This file is not autogenerated - take care! .include # find . -name Makefile -exec grep -l '^\.include.*\.test.mk' {} + | grep -v '^\./contrib' | sed -e 's,/Makefile,,' -e 's,^\./,,' -e 's,^, ,' -e 's,$, \\,' | sort DIRDEPS= \ bin/cat/tests \ bin/date/tests \ bin/dd/tests \ bin/expr/tests \ bin/ls/tests \ bin/mv/tests \ bin/pax/tests \ bin/pkill/tests \ bin/sh/tests \ bin/sh/tests/builtins \ bin/sh/tests/errors \ bin/sh/tests/execution \ bin/sh/tests/expansion \ bin/sh/tests/parameters \ bin/sh/tests/parser \ bin/sh/tests/set-e \ bin/sleep/tests \ bin/test/tests \ bin/tests \ cddl/lib/tests \ cddl/sbin/tests \ cddl/tests \ cddl/usr.bin/tests \ cddl/usr.sbin/dtrace/tests \ cddl/usr.sbin/dtrace/tests/common \ cddl/usr.sbin/dtrace/tests/common/aggs \ cddl/usr.sbin/dtrace/tests/common/arithmetic \ cddl/usr.sbin/dtrace/tests/common/arrays \ cddl/usr.sbin/dtrace/tests/common/assocs \ cddl/usr.sbin/dtrace/tests/common/begin \ cddl/usr.sbin/dtrace/tests/common/bitfields \ cddl/usr.sbin/dtrace/tests/common/buffering \ cddl/usr.sbin/dtrace/tests/common/builtinvar \ cddl/usr.sbin/dtrace/tests/common/cg \ cddl/usr.sbin/dtrace/tests/common/clauses \ cddl/usr.sbin/dtrace/tests/common/cpc \ cddl/usr.sbin/dtrace/tests/common/decls \ cddl/usr.sbin/dtrace/tests/common/docsExamples \ cddl/usr.sbin/dtrace/tests/common/drops \ cddl/usr.sbin/dtrace/tests/common/dtraceUtil \ cddl/usr.sbin/dtrace/tests/common/end \ cddl/usr.sbin/dtrace/tests/common/enum \ cddl/usr.sbin/dtrace/tests/common/error \ cddl/usr.sbin/dtrace/tests/common/exit \ cddl/usr.sbin/dtrace/tests/common/fbtprovider \ cddl/usr.sbin/dtrace/tests/common/funcs \ cddl/usr.sbin/dtrace/tests/common/grammar \ cddl/usr.sbin/dtrace/tests/common/include \ cddl/usr.sbin/dtrace/tests/common/inline \ cddl/usr.sbin/dtrace/tests/common/io \ cddl/usr.sbin/dtrace/tests/common/ip \ cddl/usr.sbin/dtrace/tests/common/java_api \ cddl/usr.sbin/dtrace/tests/common/json \ cddl/usr.sbin/dtrace/tests/common/lexer \ cddl/usr.sbin/dtrace/tests/common/llquantize \ cddl/usr.sbin/dtrace/tests/common/mdb \ cddl/usr.sbin/dtrace/tests/common/mib \ cddl/usr.sbin/dtrace/tests/common/misc \ cddl/usr.sbin/dtrace/tests/common/multiaggs \ cddl/usr.sbin/dtrace/tests/common/nfs \ cddl/usr.sbin/dtrace/tests/common/offsetof \ cddl/usr.sbin/dtrace/tests/common/operators \ cddl/usr.sbin/dtrace/tests/common/pid \ cddl/usr.sbin/dtrace/tests/common/plockstat \ cddl/usr.sbin/dtrace/tests/common/pointers \ cddl/usr.sbin/dtrace/tests/common/pragma \ cddl/usr.sbin/dtrace/tests/common/predicates \ cddl/usr.sbin/dtrace/tests/common/preprocessor \ cddl/usr.sbin/dtrace/tests/common/print \ cddl/usr.sbin/dtrace/tests/common/printa \ cddl/usr.sbin/dtrace/tests/common/printf \ cddl/usr.sbin/dtrace/tests/common/privs \ cddl/usr.sbin/dtrace/tests/common/probes \ cddl/usr.sbin/dtrace/tests/common/proc \ cddl/usr.sbin/dtrace/tests/common/profile-n \ cddl/usr.sbin/dtrace/tests/common/providers \ cddl/usr.sbin/dtrace/tests/common/raise \ cddl/usr.sbin/dtrace/tests/common/rates \ cddl/usr.sbin/dtrace/tests/common/safety \ cddl/usr.sbin/dtrace/tests/common/scalars \ cddl/usr.sbin/dtrace/tests/common/sched \ cddl/usr.sbin/dtrace/tests/common/scripting \ cddl/usr.sbin/dtrace/tests/common/sdt \ cddl/usr.sbin/dtrace/tests/common/sizeof \ cddl/usr.sbin/dtrace/tests/common/speculation \ cddl/usr.sbin/dtrace/tests/common/stability \ cddl/usr.sbin/dtrace/tests/common/stack \ cddl/usr.sbin/dtrace/tests/common/stackdepth \ cddl/usr.sbin/dtrace/tests/common/stop \ cddl/usr.sbin/dtrace/tests/common/strlen \ cddl/usr.sbin/dtrace/tests/common/strtoll \ cddl/usr.sbin/dtrace/tests/common/struct \ cddl/usr.sbin/dtrace/tests/common/syscall \ cddl/usr.sbin/dtrace/tests/common/sysevent \ cddl/usr.sbin/dtrace/tests/common/tick-n \ cddl/usr.sbin/dtrace/tests/common/trace \ cddl/usr.sbin/dtrace/tests/common/tracemem \ cddl/usr.sbin/dtrace/tests/common/translators \ cddl/usr.sbin/dtrace/tests/common/typedef \ cddl/usr.sbin/dtrace/tests/common/types \ cddl/usr.sbin/dtrace/tests/common/uctf \ cddl/usr.sbin/dtrace/tests/common/union \ cddl/usr.sbin/dtrace/tests/common/usdt \ cddl/usr.sbin/dtrace/tests/common/ustack \ cddl/usr.sbin/dtrace/tests/common/vars \ cddl/usr.sbin/dtrace/tests/common/version \ cddl/usr.sbin/tests \ gnu/lib/tests \ gnu/tests \ gnu/usr.bin/diff/tests \ gnu/usr.bin/tests \ lib/atf/libatf-c++/tests \ lib/atf/libatf-c++/tests/detail \ lib/atf/libatf-c/tests \ lib/atf/libatf-c/tests/detail \ lib/atf/tests \ lib/atf/tests/test-programs \ lib/libarchive/tests \ lib/libc/tests \ lib/libc/tests/c063 \ lib/libc/tests/db \ lib/libc/tests/gen \ lib/libc/tests/gen/execve \ lib/libc/tests/gen/posix_spawn \ lib/libc/tests/hash \ lib/libc/tests/inet \ lib/libc/tests/locale \ lib/libc/tests/net \ lib/libc/tests/net/getaddrinfo \ lib/libc/tests/nss \ lib/libc/tests/regex \ lib/libc/tests/resolv \ lib/libc/tests/rpc \ lib/libc/tests/setjmp \ lib/libc/tests/ssp \ lib/libc/tests/stdio \ lib/libc/tests/stdlib \ lib/libc/tests/string \ lib/libc/tests/sys \ lib/libc/tests/termios \ lib/libc/tests/time \ lib/libc/tests/tls \ lib/libc/tests/ttyio \ lib/libcrypt/tests \ lib/libmp/tests \ lib/libnv/tests \ lib/libpam/libpam/tests \ lib/libproc/tests \ lib/librt/tests \ lib/libthr/tests \ lib/libthr/tests/dlopen \ lib/libthr/tests/dlopen/dso \ lib/libutil/tests \ lib/libxo/tests \ lib/msun/tests \ lib/tests \ libexec/atf/atf-check/tests \ libexec/atf/atf-sh/tests \ libexec/atf/tests \ libexec/rtld-elf/tests \ + libexec/rtld-elf/tests/libpythagoras \ + libexec/rtld-elf/tests/target \ libexec/tests \ sbin/devd/tests \ sbin/dhclient/tests \ sbin/growfs/tests \ sbin/ifconfig/tests \ sbin/mdconfig/tests \ sbin/tests \ secure/lib/tests \ secure/libexec/tests \ secure/tests \ secure/usr.bin/tests \ secure/usr.sbin/tests \ share/examples/tests \ share/examples/tests/tests \ share/examples/tests/tests/atf \ share/examples/tests/tests/plain \ share/tests \ tests \ tests/etc \ tests/etc/rc.d \ tests/sys \ tests/sys/acl \ tests/sys/aio \ tests/sys/fifo \ tests/sys/file \ tests/sys/geom \ tests/sys/geom/class \ tests/sys/geom/class/concat \ tests/sys/geom/class/eli \ tests/sys/geom/class/gate \ tests/sys/geom/class/mirror \ tests/sys/geom/class/nop \ tests/sys/geom/class/raid3 \ tests/sys/geom/class/shsec \ tests/sys/geom/class/stripe \ tests/sys/geom/class/uzip \ tests/sys/kern \ tests/sys/kern/acct \ tests/sys/kern/execve \ tests/sys/kern/pipe \ tests/sys/kqueue \ tests/sys/mac \ tests/sys/mac/bsdextended \ tests/sys/mac/portacl \ tests/sys/mqueue \ tests/sys/netinet \ tests/sys/opencrypto \ + tests/sys/pjdfstest/pjdfstest \ tests/sys/pjdfstest/tests \ tests/sys/pjdfstest/tests/chflags \ tests/sys/pjdfstest/tests/chmod \ tests/sys/pjdfstest/tests/chown \ tests/sys/pjdfstest/tests/ftruncate \ tests/sys/pjdfstest/tests/granular \ tests/sys/pjdfstest/tests/link \ tests/sys/pjdfstest/tests/mkdir \ tests/sys/pjdfstest/tests/mkfifo \ tests/sys/pjdfstest/tests/mknod \ tests/sys/pjdfstest/tests/open \ tests/sys/pjdfstest/tests/rename \ tests/sys/pjdfstest/tests/rmdir \ tests/sys/pjdfstest/tests/symlink \ tests/sys/pjdfstest/tests/truncate \ tests/sys/pjdfstest/tests/unlink \ tests/sys/posixshm \ + tests/sys/sys \ tests/sys/vfs \ tests/sys/vm \ usr.bin/apply/tests \ usr.bin/basename/tests \ usr.bin/bmake/tests \ usr.bin/bmake/tests/archives \ usr.bin/bmake/tests/archives/fmt_44bsd \ usr.bin/bmake/tests/archives/fmt_44bsd_mod \ usr.bin/bmake/tests/archives/fmt_oldbsd \ usr.bin/bmake/tests/basic \ usr.bin/bmake/tests/basic/t0 \ usr.bin/bmake/tests/basic/t1 \ usr.bin/bmake/tests/basic/t2 \ usr.bin/bmake/tests/basic/t3 \ usr.bin/bmake/tests/execution \ usr.bin/bmake/tests/execution/ellipsis \ usr.bin/bmake/tests/execution/empty \ usr.bin/bmake/tests/execution/joberr \ usr.bin/bmake/tests/execution/plus \ usr.bin/bmake/tests/shell \ usr.bin/bmake/tests/shell/builtin \ usr.bin/bmake/tests/shell/meta \ usr.bin/bmake/tests/shell/path \ usr.bin/bmake/tests/shell/path_select \ usr.bin/bmake/tests/shell/replace \ usr.bin/bmake/tests/shell/select \ usr.bin/bmake/tests/suffixes \ usr.bin/bmake/tests/suffixes/basic \ usr.bin/bmake/tests/suffixes/src_wild1 \ usr.bin/bmake/tests/suffixes/src_wild2 \ usr.bin/bmake/tests/syntax \ usr.bin/bmake/tests/syntax/directive-t0 \ usr.bin/bmake/tests/syntax/enl \ usr.bin/bmake/tests/syntax/funny-targets \ usr.bin/bmake/tests/syntax/semi \ usr.bin/bmake/tests/sysmk \ usr.bin/bmake/tests/sysmk/t0 \ usr.bin/bmake/tests/sysmk/t0/2 \ usr.bin/bmake/tests/sysmk/t0/2/1 \ usr.bin/bmake/tests/sysmk/t0/mk \ usr.bin/bmake/tests/sysmk/t1 \ usr.bin/bmake/tests/sysmk/t1/2 \ usr.bin/bmake/tests/sysmk/t1/2/1 \ usr.bin/bmake/tests/sysmk/t1/mk \ usr.bin/bmake/tests/sysmk/t2 \ usr.bin/bmake/tests/sysmk/t2/2 \ usr.bin/bmake/tests/sysmk/t2/2/1 \ usr.bin/bmake/tests/sysmk/t2/mk \ usr.bin/bmake/tests/variables \ usr.bin/bmake/tests/variables/modifier_M \ usr.bin/bmake/tests/variables/modifier_t \ usr.bin/bmake/tests/variables/opt_V \ usr.bin/bmake/tests/variables/t0 \ usr.bin/bsdcat/tests \ usr.bin/calendar/tests \ usr.bin/cmp/tests \ usr.bin/col/tests \ usr.bin/comm/tests \ usr.bin/cpio/tests \ usr.bin/cut/tests \ usr.bin/dirname/tests \ usr.bin/file2c/tests \ usr.bin/grep/tests \ usr.bin/gzip/tests \ usr.bin/ident/tests \ usr.bin/join/tests \ usr.bin/jot/tests \ usr.bin/lastcomm/tests \ usr.bin/limits/tests \ usr.bin/m4/tests \ usr.bin/mkimg/tests \ usr.bin/ncal/tests \ usr.bin/printf/tests \ + usr.bin/sdiff/tests \ usr.bin/sed/tests \ usr.bin/sed/tests/regress.multitest.out \ usr.bin/soelim/tests \ usr.bin/tar/tests \ usr.bin/tests \ usr.bin/timeout/tests \ usr.bin/tr/tests \ usr.bin/truncate/tests \ usr.bin/units/tests \ usr.bin/uudecode/tests \ usr.bin/uuencode/tests \ usr.bin/xargs/tests \ + usr.bin/xinstall/tests \ usr.bin/xo/tests \ usr.bin/yacc/tests \ usr.sbin/chown/tests \ usr.sbin/etcupdate/tests \ + usr.sbin/extattr/tests \ usr.sbin/fstyp/tests \ usr.sbin/makefs/tests \ usr.sbin/newsyslog/tests \ usr.sbin/nmtree/tests \ usr.sbin/pw/tests \ usr.sbin/rpcbind/tests \ usr.sbin/sa/tests \ usr.sbin/tests \ # Remove some known to be broken DIRDEPS:= ${DIRDEPS:Ncddl/usr.sbin/dtrace/tests/common/nfs} DIRDEPS:= ${DIRDEPS:Ncddl/usr.sbin/dtrace/tests/common/sysevent} DIRDEPS:= ${DIRDEPS:Ncddl/usr.sbin/dtrace/tests/common/docsExamples} DIRDEPS:= ${DIRDEPS:Nlib/libc/tests/net/getaddrinfo} .include Index: projects/netbsd-tests-update-12/targets/pseudo/userland/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/targets/pseudo/userland/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/targets/pseudo/userland/Makefile.depend (revision 305172) @@ -1,928 +1,930 @@ # $FreeBSD$ # This file is not autogenerated - take care! .if !defined(MK_MANDOCDB) .include .endif DIRDEPS= .if ${MK_MANDOCDB} == "no" DIRDEPS+= usr.bin/makewhatis .endif DIRDEPS+= \ bin/cat \ bin/chflags \ bin/chio \ bin/chmod \ bin/cp \ bin/csh \ bin/date \ bin/dd \ bin/df \ bin/domainname \ bin/echo \ bin/ed \ bin/expr \ bin/freebsd-version \ bin/getfacl \ bin/hostname \ bin/kenv \ bin/kill \ bin/ln \ bin/ls \ bin/mkdir \ bin/mv \ bin/pax \ bin/pkill \ bin/ps \ bin/pwait \ bin/pwd \ bin/rcp \ bin/realpath \ bin/rm \ bin/rmail \ bin/rmdir \ bin/setfacl \ bin/sh \ bin/sleep \ bin/stty \ bin/sync \ bin/test \ bin/uuidgen \ sbin/adjkerntz \ sbin/atacontrol \ sbin/atm/atmconfig \ sbin/badsect \ sbin/camcontrol \ sbin/ccdconfig \ sbin/clri \ sbin/comcontrol \ sbin/conscontrol \ sbin/ddb \ sbin/devd \ sbin/devfs \ sbin/dhclient \ sbin/dmesg \ sbin/dump \ sbin/dumpfs \ sbin/dumpon \ sbin/etherswitchcfg \ sbin/ffsinfo \ sbin/fsck \ sbin/fsck_ffs \ sbin/fsck_msdosfs \ sbin/fsdb \ sbin/fsirand \ sbin/gbde \ sbin/geom/class/cache \ sbin/geom/class/concat \ sbin/geom/class/eli \ sbin/geom/class/journal \ sbin/geom/class/label \ sbin/geom/class/mirror \ sbin/geom/class/mountver \ sbin/geom/class/multipath \ sbin/geom/class/nop \ sbin/geom/class/part \ sbin/geom/class/raid \ sbin/geom/class/raid3 \ sbin/geom/class/sched \ sbin/geom/class/shsec \ sbin/geom/class/stripe \ sbin/geom/class/virstor \ sbin/geom/core \ sbin/ggate/ggatec \ sbin/ggate/ggated \ sbin/ggate/ggatel \ sbin/growfs \ sbin/gvinum \ sbin/hastctl \ sbin/hastd \ sbin/ifconfig \ sbin/init \ sbin/ipf/ipf \ sbin/ipf/ipfs \ sbin/ipf/ipfstat \ sbin/ipf/ipftest \ sbin/ipf/ipmon \ sbin/ipf/ipnat \ sbin/ipf/ippool \ sbin/ipf/ipresend \ sbin/ipf/libipf \ sbin/ipfw \ sbin/iscontrol \ sbin/kldconfig \ sbin/kldload \ sbin/kldstat \ sbin/kldunload \ sbin/ldconfig \ sbin/md5 \ sbin/mdconfig \ sbin/mdmfs \ sbin/mknod \ sbin/mksnap_ffs \ sbin/mount \ sbin/mount_cd9660 \ sbin/mount_fusefs \ sbin/mount_msdosfs \ sbin/mount_nfs \ sbin/mount_nullfs \ sbin/mount_udf \ sbin/mount_unionfs \ sbin/natd \ sbin/newfs \ sbin/newfs_msdos \ sbin/nfsiod \ sbin/nos-tun \ sbin/pfctl \ sbin/pflogd \ sbin/ping \ sbin/ping6 \ sbin/quotacheck \ sbin/rcorder \ sbin/reboot \ sbin/recoverdisk \ sbin/resolvconf \ sbin/restore \ sbin/route \ sbin/routed \ sbin/routed/rtquery \ sbin/rtsol \ sbin/savecore \ sbin/setkey \ sbin/shutdown \ sbin/spppcontrol \ sbin/swapon \ sbin/sysctl \ sbin/tunefs \ sbin/umount \ usr.bin/alias \ usr.bin/apply \ usr.bin/ar \ usr.bin/asa \ usr.bin/at \ usr.bin/atf/atf-config \ usr.bin/atf/atf-report \ usr.bin/atf/atf-run \ usr.bin/atf/atf-version \ usr.bin/atm/sscop \ usr.bin/awk \ usr.bin/banner \ usr.bin/basename \ usr.bin/bc \ usr.bin/biff \ usr.bin/bluetooth/bthost \ usr.bin/bluetooth/btsockstat \ usr.bin/bluetooth/rfcomm_sppd \ usr.bin/bmake \ usr.bin/brandelf \ usr.bin/bsdiff/bsdiff \ usr.bin/bsdiff/bspatch \ usr.bin/bsdcat \ usr.bin/bzip2 \ usr.bin/bzip2recover \ usr.bin/c89 \ usr.bin/c99 \ usr.bin/calendar \ usr.bin/cap_mkdb \ usr.bin/catman \ usr.bin/chat \ usr.bin/checknr \ usr.bin/chkey \ usr.bin/chpass \ usr.bin/cksum \ usr.bin/cmp \ usr.bin/col \ usr.bin/colcrt \ usr.bin/colldef \ usr.bin/colrm \ usr.bin/column \ usr.bin/comm \ usr.bin/compile_et \ usr.bin/compress \ usr.bin/cpio \ usr.bin/cpuset \ usr.bin/csplit \ usr.bin/csup \ usr.bin/ctags \ usr.bin/ctlstat \ usr.bin/cut \ usr.bin/dc \ usr.bin/dig \ usr.bin/dirname \ usr.bin/dpv \ usr.bin/drill \ usr.bin/du \ usr.bin/ee \ usr.bin/elf2aout \ usr.bin/elfdump \ usr.bin/enigma \ usr.bin/env \ usr.bin/expand \ usr.bin/false \ usr.bin/fetch \ usr.bin/file \ usr.bin/file2c \ usr.bin/find \ usr.bin/finger \ usr.bin/fmt \ usr.bin/fold \ usr.bin/from \ usr.bin/fstat \ usr.bin/fsync \ usr.bin/ftp \ usr.bin/gcore \ usr.bin/gencat \ usr.bin/getconf \ usr.bin/getent \ usr.bin/getopt \ usr.bin/gprof \ usr.bin/grep \ usr.bin/gzip \ usr.bin/head \ usr.bin/hexdump \ usr.bin/host \ usr.bin/iconv \ usr.bin/id \ usr.bin/ident \ usr.bin/indent \ usr.bin/ipcrm \ usr.bin/ipcs \ usr.bin/iscsictl \ usr.bin/join \ usr.bin/jot \ usr.bin/kdump \ usr.bin/keylogin \ usr.bin/keylogout \ usr.bin/killall \ usr.bin/ktrace \ usr.bin/ktrdump \ usr.bin/lam \ usr.bin/last \ usr.bin/lastcomm \ usr.bin/lex \ usr.bin/ldd \ usr.bin/leave \ usr.bin/less \ usr.bin/lessecho \ usr.bin/lesskey \ usr.bin/lex/lib \ usr.bin/limits \ usr.bin/locale \ usr.bin/locate/bigram \ usr.bin/locate/code \ usr.bin/locate/locate \ usr.bin/lock \ usr.bin/lockf \ usr.bin/logger \ usr.bin/login \ usr.bin/logins \ usr.bin/logname \ usr.bin/look \ usr.bin/lorder \ usr.bin/lsvfs \ usr.bin/lzmainfo \ usr.bin/m4 \ usr.bin/mail \ usr.bin/man \ usr.bin/mandoc \ usr.bin/mesg \ usr.bin/minigzip \ usr.bin/ministat \ usr.bin/mkcsmapper \ usr.bin/mkdep \ usr.bin/mkesdb \ usr.bin/mkfifo \ usr.bin/mkimg \ usr.bin/mklocale \ usr.bin/mkstr \ usr.bin/mktemp \ usr.bin/mkuzip \ usr.bin/msgs \ usr.bin/mt \ usr.bin/nc \ usr.bin/ncal \ usr.bin/netstat \ usr.bin/newgrp \ usr.bin/newkey \ usr.bin/nfsstat \ usr.bin/nice \ usr.bin/nl \ usr.bin/nohup \ usr.bin/nslookup \ usr.bin/nsupdate \ usr.bin/numactl \ usr.bin/opieinfo \ usr.bin/opiekey \ usr.bin/opiepasswd \ usr.bin/pagesize \ usr.bin/passwd \ usr.bin/paste \ usr.bin/patch \ usr.bin/pathchk \ usr.bin/perror \ usr.bin/pr \ usr.bin/printenv \ usr.bin/printf \ usr.bin/procstat \ usr.bin/protect \ usr.bin/quota \ usr.bin/rctl \ usr.bin/renice \ usr.bin/resizewin \ usr.bin/rev \ usr.bin/revoke \ usr.bin/rlogin \ usr.bin/rpcgen \ usr.bin/rpcinfo \ usr.bin/rs \ usr.bin/rsh \ usr.bin/rup \ usr.bin/ruptime \ usr.bin/rusers \ usr.bin/rwall \ usr.bin/rwho \ usr.bin/script \ usr.bin/sdiff \ usr.bin/sed \ usr.bin/send-pr \ usr.bin/seq \ usr.bin/shar \ usr.bin/showmount \ usr.bin/smbutil \ usr.bin/sockstat \ usr.bin/soelim \ usr.bin/sort \ usr.bin/split \ usr.bin/ssh-copy-id \ usr.bin/stat \ usr.bin/stdbuf \ usr.bin/su \ usr.bin/svn/svn \ usr.bin/svn/svnadmin \ usr.bin/svn/svnbench \ usr.bin/svn/svndumpfilter \ usr.bin/svn/svnfsfs \ usr.bin/svn/svnlook \ usr.bin/svn/svnmucc \ usr.bin/svn/svnrdump \ usr.bin/svn/svnserve \ usr.bin/svn/svnsync \ usr.bin/svn/svnversion \ usr.bin/systat \ usr.bin/tabs \ usr.bin/tail \ usr.bin/talk \ usr.bin/tar \ usr.bin/tcopy \ usr.bin/tee \ usr.bin/telnet \ usr.bin/tftp \ usr.bin/time \ usr.bin/timeout \ usr.bin/tip/tip \ usr.bin/top \ usr.bin/touch \ usr.bin/tput \ usr.bin/tr \ usr.bin/true \ usr.bin/truncate \ usr.bin/truss \ usr.bin/tset \ usr.bin/tsort \ usr.bin/tty \ usr.bin/ul \ usr.bin/uname \ usr.bin/unexpand \ usr.bin/unifdef \ usr.bin/uniq \ usr.bin/units \ usr.bin/unvis \ usr.bin/unzip \ usr.bin/usbhidaction \ usr.bin/usbhidctl \ usr.bin/users \ usr.bin/uudecode \ usr.bin/uuencode \ usr.bin/vacation \ usr.bin/vgrind \ usr.bin/vi \ usr.bin/vi/catalog \ usr.bin/vis \ usr.bin/vmstat \ usr.bin/vtfontcvt \ usr.bin/w \ usr.bin/wall \ usr.bin/wc \ usr.bin/what \ usr.bin/whereis \ usr.bin/which \ usr.bin/who \ usr.bin/whois \ usr.bin/write \ usr.bin/xargs \ usr.bin/xinstall \ usr.bin/xlint/lint1 \ usr.bin/xlint/lint2 \ usr.bin/xlint/llib \ usr.bin/xlint/xlint \ usr.bin/xo \ usr.bin/xstr \ usr.bin/xz \ usr.bin/xzdec \ usr.bin/yacc \ usr.bin/yes \ usr.bin/ypcat \ usr.bin/ypmatch \ usr.bin/ypwhich \ usr.sbin/IPXrouted \ usr.sbin/ac \ usr.sbin/accton \ usr.sbin/adduser \ usr.sbin/amd/amd \ usr.sbin/amd/amq \ usr.sbin/amd/doc \ usr.sbin/amd/fixmount \ usr.sbin/amd/fsinfo \ usr.sbin/amd/hlfsd \ usr.sbin/amd/include \ usr.sbin/amd/libamu \ usr.sbin/amd/mk-amd-map \ usr.sbin/amd/pawd \ usr.sbin/amd/scripts \ usr.sbin/amd/wire-test \ usr.sbin/ancontrol \ usr.sbin/apm \ usr.sbin/arp \ usr.sbin/arpaname \ usr.sbin/audit \ usr.sbin/auditd \ usr.sbin/auditdistd \ usr.sbin/auditreduce \ usr.sbin/authpf \ usr.sbin/autofs \ usr.sbin/binmiscctl \ usr.sbin/bluetooth/ath3kfw \ usr.sbin/bluetooth/bcmfw \ usr.sbin/bluetooth/bt3cfw \ usr.sbin/bluetooth/bthidcontrol \ usr.sbin/bluetooth/bthidd \ usr.sbin/bluetooth/btpand \ usr.sbin/bluetooth/hccontrol \ usr.sbin/bluetooth/hcsecd \ usr.sbin/bluetooth/hcseriald \ usr.sbin/bluetooth/l2control \ usr.sbin/bluetooth/l2ping \ usr.sbin/bluetooth/rfcomm_pppd \ usr.sbin/bluetooth/sdpcontrol \ usr.sbin/bluetooth/sdpd \ usr.sbin/bootparamd/bootparamd \ usr.sbin/bootparamd/callbootd \ usr.sbin/bsdconfig \ usr.sbin/bsdconfig/console \ usr.sbin/bsdconfig/console/include \ usr.sbin/bsdconfig/diskmgmt \ usr.sbin/bsdconfig/diskmgmt/include \ usr.sbin/bsdconfig/docsinstall \ usr.sbin/bsdconfig/docsinstall/include \ usr.sbin/bsdconfig/dot \ usr.sbin/bsdconfig/dot/include \ usr.sbin/bsdconfig/examples \ usr.sbin/bsdconfig/include \ usr.sbin/bsdconfig/includes \ usr.sbin/bsdconfig/includes/include \ usr.sbin/bsdconfig/mouse \ usr.sbin/bsdconfig/mouse/include \ usr.sbin/bsdconfig/networking \ usr.sbin/bsdconfig/networking/include \ usr.sbin/bsdconfig/networking/share \ usr.sbin/bsdconfig/packages \ usr.sbin/bsdconfig/packages/include \ usr.sbin/bsdconfig/password \ usr.sbin/bsdconfig/password/include \ usr.sbin/bsdconfig/password/share \ usr.sbin/bsdconfig/security \ usr.sbin/bsdconfig/security/include \ usr.sbin/bsdconfig/share \ usr.sbin/bsdconfig/share/media \ usr.sbin/bsdconfig/share/packages \ usr.sbin/bsdconfig/startup \ usr.sbin/bsdconfig/startup/include \ usr.sbin/bsdconfig/startup/share \ usr.sbin/bsdconfig/timezone \ usr.sbin/bsdconfig/timezone/include \ usr.sbin/bsdconfig/timezone/share \ usr.sbin/bsdconfig/ttys \ usr.sbin/bsdconfig/ttys/include \ usr.sbin/bsdconfig/usermgmt \ usr.sbin/bsdconfig/usermgmt/include \ usr.sbin/bsdconfig/usermgmt/share \ usr.sbin/bsdinstall/distextract \ usr.sbin/bsdinstall/distfetch \ usr.sbin/bsdinstall/partedit \ usr.sbin/bsdinstall/scripts \ usr.sbin/bsnmpd/bsnmpd \ usr.sbin/bsnmpd/gensnmptree \ usr.sbin/bsnmpd/modules/snmp_atm \ usr.sbin/bsnmpd/modules/snmp_bridge \ usr.sbin/bsnmpd/modules/snmp_hast \ usr.sbin/bsnmpd/modules/snmp_hostres \ usr.sbin/bsnmpd/modules/snmp_lm75 \ usr.sbin/bsnmpd/modules/snmp_mibII \ usr.sbin/bsnmpd/modules/snmp_netgraph \ usr.sbin/bsnmpd/modules/snmp_pf \ usr.sbin/bsnmpd/modules/snmp_target \ usr.sbin/bsnmpd/modules/snmp_usm \ usr.sbin/bsnmpd/modules/snmp_vacm \ usr.sbin/bsnmpd/modules/snmp_wlan \ usr.sbin/bsnmpd/tools/bsnmptools \ usr.sbin/bsnmpd/tools/libbsnmptools \ usr.sbin/burncd \ usr.sbin/cdcontrol \ usr.sbin/chkgrp \ usr.sbin/chown \ usr.sbin/chroot \ usr.sbin/ckdist \ usr.sbin/clear_locks \ usr.sbin/config \ usr.sbin/crashinfo \ usr.sbin/cron/cron \ usr.sbin/cron/crontab \ usr.sbin/cron/lib \ usr.sbin/crunch/crunchgen \ usr.sbin/crunch/crunchide \ usr.sbin/ctladm \ usr.sbin/ctld \ usr.sbin/ctm/ctm \ usr.sbin/ctm/ctm_dequeue \ usr.sbin/ctm/ctm_rmail \ usr.sbin/ctm/ctm_smail \ usr.sbin/daemon \ usr.sbin/dconschat \ usr.sbin/ddns-confgen \ usr.sbin/devctl \ usr.sbin/devinfo \ usr.sbin/digictl \ usr.sbin/diskinfo \ usr.sbin/dnssec-dsfromkey \ usr.sbin/dnssec-keyfromlabel \ usr.sbin/dnssec-keygen \ usr.sbin/dnssec-revoke \ usr.sbin/dnssec-settime \ usr.sbin/dnssec-signzone \ usr.sbin/dumpcis \ usr.sbin/editmap \ usr.sbin/edquota \ usr.sbin/etcupdate \ usr.sbin/extattr \ usr.sbin/extattrctl \ usr.sbin/fdcontrol \ usr.sbin/fdformat \ usr.sbin/fdread \ usr.sbin/fdwrite \ usr.sbin/fifolog/fifolog_create \ usr.sbin/fifolog/fifolog_reader \ usr.sbin/fifolog/fifolog_writer \ usr.sbin/fifolog/lib \ usr.sbin/flowctl \ usr.sbin/fmtree \ usr.sbin/freebsd-update \ usr.sbin/fstyp \ usr.sbin/ftp-proxy \ usr.sbin/fwcontrol \ usr.sbin/genrandom \ usr.sbin/getfmac \ usr.sbin/getpmac \ usr.sbin/gpioctl \ usr.sbin/gssd \ usr.sbin/gstat \ usr.sbin/i2c \ usr.sbin/ifmcstat \ usr.sbin/inetd \ usr.sbin/iostat \ usr.sbin/iovctl \ usr.sbin/ip6addrctl \ usr.sbin/ipfwpcap \ usr.sbin/isc-hmac-fixup \ usr.sbin/iscsid \ usr.sbin/isfctl \ usr.sbin/jail \ usr.sbin/jexec \ usr.sbin/jls \ usr.sbin/kbdcontrol \ usr.sbin/kbdmap \ usr.sbin/keyserv \ usr.sbin/kldxref \ usr.sbin/lastlogin \ usr.sbin/lmcconfig \ usr.sbin/lpr/chkprintcap \ usr.sbin/lpr/common_source \ usr.sbin/lpr/filters \ usr.sbin/lpr/filters.ru/koi2855 \ usr.sbin/lpr/filters.ru/koi2alt \ usr.sbin/lpr/lp \ usr.sbin/lpr/lpc \ usr.sbin/lpr/lpd \ usr.sbin/lpr/lpq \ usr.sbin/lpr/lpr \ usr.sbin/lpr/lprm \ usr.sbin/lpr/lptest \ usr.sbin/lpr/pac \ usr.sbin/mailstats \ usr.sbin/mailwrapper \ usr.sbin/makefs \ usr.sbin/makemap \ usr.sbin/manctl \ usr.sbin/memcontrol \ usr.sbin/mergemaster \ usr.sbin/mfiutil \ usr.sbin/mixer \ usr.sbin/mld6query \ usr.sbin/mlxcontrol \ usr.sbin/mount_smbfs \ usr.sbin/mountd \ usr.sbin/moused \ usr.sbin/mpsutil \ usr.sbin/mptutil \ usr.sbin/mtest \ usr.sbin/named \ usr.sbin/named-checkconf \ usr.sbin/named-checkzone \ usr.sbin/named-journalprint \ usr.sbin/ndp \ usr.sbin/newsyslog \ usr.sbin/nfscbd \ usr.sbin/nfsd \ usr.sbin/nfsdumpstate \ usr.sbin/nfsrevoke \ usr.sbin/nfsuserd \ usr.sbin/ngctl \ usr.sbin/nghook \ usr.sbin/nmtree \ usr.sbin/nologin \ usr.sbin/nscd \ usr.sbin/nsec3hash \ usr.sbin/ntp/doc \ usr.sbin/ntp/doc/drivers/icons \ usr.sbin/ntp/doc/drivers/scripts \ usr.sbin/ntp/doc/drivers \ usr.sbin/ntp/doc/hints \ usr.sbin/ntp/doc/icons \ usr.sbin/ntp/doc/pic \ usr.sbin/ntp/doc/scripts \ usr.sbin/ntp/libntp \ usr.sbin/ntp/libopts \ usr.sbin/ntp/libparse \ usr.sbin/ntp/ntp-keygen \ usr.sbin/ntp/ntpd \ usr.sbin/ntp/ntpdate \ usr.sbin/ntp/ntpdc \ usr.sbin/ntp/ntpq \ usr.sbin/ntp/ntptime \ usr.sbin/ntp/sntp \ usr.sbin/pc-sysinstall/backend \ usr.sbin/pc-sysinstall/backend-partmanager \ usr.sbin/pc-sysinstall/backend-query \ usr.sbin/pc-sysinstall/conf \ usr.sbin/pc-sysinstall/doc \ usr.sbin/pc-sysinstall/examples \ usr.sbin/pc-sysinstall/pc-sysinstall \ usr.sbin/pciconf \ usr.sbin/periodic \ usr.sbin/pkg \ usr.sbin/pkg_install/add \ usr.sbin/pkg_install/create \ usr.sbin/pkg_install/delete \ usr.sbin/pkg_install/info \ usr.sbin/pkg_install/lib \ usr.sbin/pkg_install/updating \ usr.sbin/pkg_install/version \ usr.sbin/pmcannotate \ usr.sbin/pmccontrol \ usr.sbin/pmcstat \ + usr.sbin/pmcstudy \ usr.sbin/portsnap/make_index \ usr.sbin/portsnap/phttpget \ usr.sbin/portsnap/portsnap \ usr.sbin/powerd \ usr.sbin/ppp \ usr.sbin/pppctl \ usr.sbin/praliases \ usr.sbin/praudit \ usr.sbin/procctl \ usr.sbin/pstat \ usr.sbin/pw \ usr.sbin/pwd_mkdb \ usr.sbin/quot \ usr.sbin/quotaon \ usr.sbin/rarpd \ usr.sbin/repquota \ usr.sbin/rip6query \ usr.sbin/rmt \ usr.sbin/rndc \ usr.sbin/rndc-confgen \ usr.sbin/route6d \ usr.sbin/rpc.lockd \ usr.sbin/rpc.statd \ usr.sbin/rpc.umntall \ usr.sbin/rpc.yppasswdd \ usr.sbin/rpc.ypupdated \ usr.sbin/rpc.ypxfrd \ usr.sbin/rpcbind \ usr.sbin/rrenumd \ usr.sbin/rtadvctl \ usr.sbin/rtadvd \ usr.sbin/rtprio \ usr.sbin/rtsold \ usr.sbin/rwhod \ usr.sbin/sa \ usr.sbin/sendmail \ usr.sbin/service \ usr.sbin/services_mkdb \ usr.sbin/sesutil \ usr.sbin/setfib \ usr.sbin/setfmac \ usr.sbin/setpmac \ usr.sbin/smbmsg \ usr.sbin/snapinfo \ usr.sbin/spray \ usr.sbin/syslogd \ usr.sbin/sysrc \ usr.sbin/tcpdchk \ usr.sbin/tcpdmatch \ usr.sbin/tcpdrop \ usr.sbin/tcpdump/tcpdump \ usr.sbin/timed/timed \ usr.sbin/timed/timedc \ usr.sbin/traceroute \ usr.sbin/traceroute6 \ usr.sbin/trpt \ usr.sbin/tzsetup \ usr.sbin/uathload \ usr.sbin/uefisign \ usr.sbin/ugidfw \ usr.sbin/uhsoctl \ usr.sbin/unbound/anchor \ usr.sbin/unbound/checkconf \ usr.sbin/unbound/control \ usr.sbin/unbound/daemon \ usr.sbin/unbound/local-setup \ usr.sbin/usbconfig \ usr.sbin/usbdump \ usr.sbin/utx \ usr.sbin/vidcontrol \ usr.sbin/vigr \ usr.sbin/vipw \ usr.sbin/wake \ usr.sbin/watch \ usr.sbin/watchdogd \ usr.sbin/wlandebug \ usr.sbin/wpa/hostapd \ usr.sbin/wpa/hostapd_cli \ usr.sbin/wpa/ndis_events \ usr.sbin/wpa/wpa_cli \ usr.sbin/wpa/wpa_passphrase \ usr.sbin/wpa/wpa_supplicant \ usr.sbin/yp_mkdb \ usr.sbin/ypbind \ + usr.sbin/ypldap \ usr.sbin/yppoll \ usr.sbin/yppush \ usr.sbin/ypserv \ usr.sbin/ypset \ usr.sbin/zic/zdump \ usr.sbin/zic/zic \ usr.sbin/zonectl \ ${DEP_RELDIR}/cddl \ ${DEP_RELDIR}/games \ ${DEP_RELDIR}/gnu \ ${DEP_RELDIR}/include \ ${DEP_RELDIR}/kerberos5 \ ${DEP_RELDIR}/lib \ ${DEP_RELDIR}/libexec \ ${DEP_RELDIR}/misc \ ${DEP_RELDIR}/secure \ ${DEP_RELDIR}/share \ .if ${MK_NAND} != "no" DIRDEPS+= \ sbin/nandfs \ sbin/newfs_nandfs \ usr.sbin/nandsim \ usr.sbin/nandtool \ .endif DIRDEPS.amd64= \ sbin/bsdlabel \ sbin/fdisk \ sbin/nvmecontrol \ usr.sbin/acpi/acpiconf \ usr.sbin/acpi/acpidb \ usr.sbin/acpi/acpidump \ usr.sbin/acpi/iasl \ usr.sbin/apm \ usr.sbin/asf \ usr.sbin/bhyve \ usr.sbin/bhyvectl \ usr.sbin/bhyveload \ usr.sbin/boot0cfg \ usr.sbin/btxld \ usr.sbin/camdd \ usr.sbin/cpucontrol \ usr.sbin/hyperv/tools \ usr.sbin/kgmon \ usr.sbin/lptcontrol \ usr.sbin/mptable \ usr.sbin/ndiscvt \ usr.sbin/spkrtest \ usr.sbin/sade \ usr.sbin/zzz DIRDEPS.arm= \ sbin/bsdlabel \ sbin/fdisk \ usr.sbin/ofwdump \ usr.sbin/kgmon DIRDEPS.i386= \ sbin/bsdlabel \ sbin/fdisk \ sbin/nvmecontrol \ sbin/sconfig \ usr.sbin/apm \ usr.sbin/apmd \ usr.sbin/asf \ usr.sbin/btxld \ usr.sbin/cpucontrol \ usr.sbin/hyperv/tools \ usr.sbin/kgmon \ usr.sbin/kgzip \ usr.sbin/lptcontrol \ usr.sbin/mptable \ usr.sbin/ndiscvt \ usr.sbin/pnpinfo \ usr.sbin/sade \ usr.sbin/spkrtest \ usr.sbin/zzz \ usr.sbin/acpi \ usr.sbin/boot0cfg DIRDEPS.arm64= \ usr.sbin/acpi \ usr.sbin/ofwdump DIRDEPS.mips= \ sbin/bsdlabel \ sbin/fdisk DIRDEPS.pc98= \ sbin/bsdlabel \ sbin/fdisk_pc98 \ sbin/sconfig DIRDEPS.sparc64= \ sbin/bsdlabel \ sbin/sunlabel \ usr.sbin/eeprom \ usr.sbin/ofwdump \ usr.sbin/sade DIRDEPS.powerpc= \ usr.sbin/nvram \ usr.sbin/ofwdump .if ${MK_BLACKLIST_SUPPORT} != "no" DIRDEPS+= \ usr.sbin/blacklistctl \ usr.sbin/blacklistd .endif .if ${MK_GPL_DTC} != "yes" DIRDEPS+= usr.bin/dtc .endif .if ${MK_OFED} != "no" DIRDEPS+= \ contrib/ofed/usr.bin/ibaddr \ contrib/ofed/usr.bin/ibnetdiscover \ contrib/ofed/usr.bin/ibping \ contrib/ofed/usr.bin/ibportstate \ contrib/ofed/usr.bin/ibroute \ contrib/ofed/usr.bin/ibsendtrap \ contrib/ofed/usr.bin/ibstat \ contrib/ofed/usr.bin/ibsysstat \ contrib/ofed/usr.bin/ibtracert \ contrib/ofed/usr.bin/opensm \ contrib/ofed/usr.bin/osmtest \ contrib/ofed/usr.bin/perfquery \ contrib/ofed/usr.bin/saquery \ contrib/ofed/usr.bin/sminfo \ contrib/ofed/usr.bin/smpdump \ contrib/ofed/usr.bin/smpquery \ contrib/ofed/usr.bin/vendstat .endif DIRDEPS+= ${DIRDEPS.${MACHINE}:U} .include Index: projects/netbsd-tests-update-12/targets/pseudo/userland/cddl/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/targets/pseudo/userland/cddl/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/targets/pseudo/userland/cddl/Makefile.depend (revision 305172) @@ -1,45 +1,46 @@ # $FreeBSD$ # This file is not autogenerated - take care! .if !defined(MK_CTF) .include "${SRCTOP}/share/mk/src.opts.mk" .endif DIRDEPS = \ cddl/lib/drti \ cddl/lib/libavl \ cddl/lib/libctf \ cddl/lib/libdtrace \ cddl/lib/libnvpair \ cddl/lib/libumem \ cddl/lib/libuutil \ cddl/usr.bin/ctfconvert \ cddl/usr.bin/ctfdump \ cddl/usr.bin/ctfmerge \ cddl/usr.sbin/dtrace \ cddl/usr.sbin/dtruss \ cddl/usr.sbin/lockstat \ cddl/usr.sbin/plockstat \ + cddl/usr.sbin/zfsd \ DIRDEPS.ZFS = \ cddl/lib/libzfs \ cddl/lib/libzfs_core \ cddl/lib/libzpool \ cddl/sbin/zfs \ cddl/sbin/zpool \ cddl/usr.bin/zinject \ cddl/usr.bin/zstreamdump \ cddl/usr.bin/ztest \ cddl/usr.sbin/zdb \ cddl/usr.sbin/zhack \ .for O in ZFS .if ${MK_$O} == "yes" DIRDEPS+= ${DIRDEPS.$O} .endif .endfor .include Index: projects/netbsd-tests-update-12/targets/pseudo/userland/libexec/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/targets/pseudo/userland/libexec/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/targets/pseudo/userland/libexec/Makefile.depend (revision 305172) @@ -1,48 +1,57 @@ # $FreeBSD$ # This file is not autogenerated - take care! .include DIRDEPS = \ libexec/atf/atf-check \ libexec/atf/atf-sh \ libexec/atrun \ libexec/bootpd \ libexec/bootpd/bootpgw \ libexec/bootpd/tools/bootpef \ libexec/bootpd/tools/bootptest \ libexec/comsat \ libexec/fingerd \ libexec/ftpd \ libexec/getty \ libexec/hyperv \ libexec/mail.local \ libexec/mknetid \ libexec/pppoed \ libexec/rbootd \ libexec/revnetgroup \ libexec/rlogind \ libexec/rpc.rquotad \ libexec/rpc.rstatd \ libexec/rpc.rusersd \ libexec/rpc.rwalld \ libexec/rpc.sprayd \ libexec/rshd \ libexec/rtld-elf \ libexec/save-entropy \ libexec/smrsh \ libexec/talkd \ libexec/tcpd \ libexec/telnetd \ libexec/tftp-proxy \ libexec/tftpd \ libexec/ulog-helper \ libexec/ypxfr \ .if ${MK_BLACKLIST_SUPPORT} != "no" DIRDEPS+= libexec/blacklistd-helper .endif +.if ${MK_DMAGENT} != "no" +DIRDEPS+= libexec/dma/dma-mbox-create +DIRDEPS+= libexec/dma/dmagent +.endif + +.if ${MK_MAN_UTILS} != "no" +DIRDEPS+= libexec/makewhatis.local +.endif + .include Index: projects/netbsd-tests-update-12/tests/sys/pjdfstest/pjdfstest/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/tests/sys/pjdfstest/pjdfstest/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/tests/sys/pjdfstest/pjdfstest/Makefile.depend (revision 305172) @@ -0,0 +1,18 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + include/xlocale \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libcompiler_rt \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/tests/sys/pjdfstest/pjdfstest/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/tests/sys/sys/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/tests/sys/sys/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/tests/sys/sys/Makefile.depend (revision 305172) @@ -0,0 +1,19 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + include/xlocale \ + lib/${CSU_DIR} \ + lib/atf/libatf-c \ + lib/libc \ + lib/libcompiler_rt \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/tests/sys/sys/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/tools/regression/capsicum/syscalls/cap_fcntls_limit.c =================================================================== --- projects/netbsd-tests-update-12/tools/regression/capsicum/syscalls/cap_fcntls_limit.c (revision 305171) +++ projects/netbsd-tests-update-12/tools/regression/capsicum/syscalls/cap_fcntls_limit.c (revision 305172) @@ -1,540 +1,546 @@ /*- * Copyright (c) 2012 The FreeBSD Foundation * All rights reserved. * * This software was developed by Pawel Jakub Dawidek under sponsorship from * the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include "misc.h" static void fcntl_tests_0(int fd) { uint32_t fcntlrights; fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == CAP_FCNTL_ALL); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == 0); CHECK(fcntl(fd, F_GETFL) == (O_RDWR | O_NONBLOCK)); CHECK(fcntl(fd, F_SETFL, 0) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); errno = 0; CHECK(cap_fcntls_limit(fd, ~CAP_FCNTL_ALL) == -1); CHECK(errno == EINVAL); CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == (CAP_FCNTL_GETFL | CAP_FCNTL_SETFL)); CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == (CAP_FCNTL_GETFL | CAP_FCNTL_SETFL)); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == 0); CHECK(fcntl(fd, F_GETFL) == (O_RDWR | O_NONBLOCK)); CHECK(fcntl(fd, F_SETFL, 0) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == CAP_FCNTL_GETFL); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == CAP_FCNTL_GETFL); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); errno = 0; CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFL) == O_RDWR); errno = 0; CHECK(fcntl(fd, F_SETFL, 0) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(cap_fcntls_limit(fd, 0) == 0); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); errno = 0; CHECK(fcntl(fd, F_GETFL) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_SETFL, 0) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_GETFL) == -1); CHECK(errno == ENOTCAPABLE); } static void fcntl_tests_1(int fd) { uint32_t fcntlrights; + cap_rights_t rights; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == CAP_FCNTL_GETFL); - CHECK(cap_rights_limit(fd, CAP_ALL & ~CAP_FCNTL) == 0); + CAP_ALL(&rights); + cap_rights_clear(&rights, CAP_FCNTL); + CHECK(cap_rights_limit(fd, &rights) == 0); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); errno = 0; CHECK(fcntl(fd, F_GETFL) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_SETFL, 0) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_GETFL) == -1); CHECK(errno == ENOTCAPABLE); } static void fcntl_tests_2(int fd) { uint32_t fcntlrights; + cap_rights_t rights; - CHECK(cap_rights_limit(fd, CAP_ALL & ~CAP_FCNTL) == 0); + CAP_ALL(&rights); + cap_rights_clear(&rights, CAP_FCNTL); + CHECK(cap_rights_limit(fd, &rights) == 0); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = CAP_FCNTL_ALL; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); errno = 0; CHECK(fcntl(fd, F_GETFL) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_SETFL, 0) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_GETFL) == -1); CHECK(errno == ENOTCAPABLE); } static void fcntl_tests_send_0(int sock) { int fd; CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK(descriptor_send(sock, fd) == 0); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == 0); CHECK(descriptor_send(sock, fd) == 0); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL) == 0); CHECK(descriptor_send(sock, fd) == 0); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK(cap_fcntls_limit(fd, 0) == 0); CHECK(descriptor_send(sock, fd) == 0); CHECK(close(fd) == 0); } static void fcntl_tests_recv_0(int sock) { uint32_t fcntlrights; int fd; CHECK(descriptor_recv(sock, &fd) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == CAP_FCNTL_ALL); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == 0); CHECK(fcntl(fd, F_GETFL) == (O_RDWR | O_NONBLOCK)); CHECK(fcntl(fd, F_SETFL, 0) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(close(fd) == 0); CHECK(descriptor_recv(sock, &fd) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == (CAP_FCNTL_GETFL | CAP_FCNTL_SETFL)); CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == (CAP_FCNTL_GETFL | CAP_FCNTL_SETFL)); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == 0); CHECK(fcntl(fd, F_GETFL) == (O_RDWR | O_NONBLOCK)); CHECK(fcntl(fd, F_SETFL, 0) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(close(fd) == 0); CHECK(descriptor_recv(sock, &fd) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == CAP_FCNTL_GETFL); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == CAP_FCNTL_GETFL); CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == CAP_FCNTL_GETFL); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_GETFL) == O_RDWR); errno = 0; CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFL) == O_RDWR); errno = 0; CHECK(fcntl(fd, F_SETFL, 0) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFL) == O_RDWR); CHECK(close(fd) == 0); CHECK(descriptor_recv(sock, &fd) == 0); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL | CAP_FCNTL_SETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_GETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); errno = 0; CHECK(cap_fcntls_limit(fd, CAP_FCNTL_SETFL) == -1); CHECK(errno == ENOTCAPABLE); fcntlrights = 0; CHECK(cap_fcntls_get(fd, &fcntlrights) == 0); CHECK(fcntlrights == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); errno = 0; CHECK(fcntl(fd, F_GETFL) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_SETFL, O_NONBLOCK) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_SETFL, 0) == -1); CHECK(errno == ENOTCAPABLE); errno = 0; CHECK(fcntl(fd, F_GETFL) == -1); CHECK(errno == ENOTCAPABLE); CHECK(close(fd) == 0); } int main(void) { int fd, pfd, sp[2]; pid_t pid; printf("1..870\n"); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); fcntl_tests_0(fd); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); fcntl_tests_1(fd); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); fcntl_tests_2(fd); CHECK(close(fd) == 0); /* Child inherits descriptor and operates on it first. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK((pid = fork()) >= 0); if (pid == 0) { fcntl_tests_0(fd); CHECK(close(fd) == 0); exit(0); } else { CHECK(waitpid(pid, NULL, 0) == pid); fcntl_tests_0(fd); } CHECK(close(fd) == 0); /* Child inherits descriptor, but operates on it after parent. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK((pid = fork()) >= 0); if (pid == 0) { sleep(1); fcntl_tests_0(fd); CHECK(close(fd) == 0); exit(0); } else { fcntl_tests_0(fd); CHECK(waitpid(pid, NULL, 0) == pid); } CHECK(close(fd) == 0); /* Child inherits descriptor and operates on it first. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK((pid = pdfork(&pfd, 0)) >= 0); if (pid == 0) { fcntl_tests_1(fd); exit(0); } else { CHECK(pdwait(pfd) == 0); /* It fails with EBADF, which I believe is a bug. CHECK(close(pfd) == 0); */ fcntl_tests_1(fd); } CHECK(close(fd) == 0); /* Child inherits descriptor, but operates on it after parent. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK((pid = pdfork(&pfd, 0)) >= 0); if (pid == 0) { sleep(1); fcntl_tests_1(fd); exit(0); } else { fcntl_tests_1(fd); CHECK(pdwait(pfd) == 0); /* It fails with EBADF, which I believe is a bug. CHECK(close(pfd) == 0); */ } CHECK(close(fd) == 0); /* Child inherits descriptor and operates on it first. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK((pid = fork()) >= 0); if (pid == 0) { fcntl_tests_2(fd); exit(0); } else { CHECK(waitpid(pid, NULL, 0) == pid); fcntl_tests_2(fd); } CHECK(close(fd) == 0); /* Child inherits descriptor, but operates on it after parent. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK((pid = fork()) >= 0); if (pid == 0) { sleep(1); fcntl_tests_2(fd); exit(0); } else { fcntl_tests_2(fd); CHECK(waitpid(pid, NULL, 0) == pid); } CHECK(close(fd) == 0); /* Send descriptors from parent to child. */ CHECK(socketpair(AF_UNIX, SOCK_STREAM, 0, sp) == 0); CHECK((pid = fork()) >= 0); if (pid == 0) { CHECK(close(sp[0]) == 0); fcntl_tests_recv_0(sp[1]); CHECK(close(sp[1]) == 0); exit(0); } else { CHECK(close(sp[1]) == 0); fcntl_tests_send_0(sp[0]); CHECK(waitpid(pid, NULL, 0) == pid); CHECK(close(sp[0]) == 0); } /* Send descriptors from child to parent. */ CHECK(socketpair(AF_UNIX, SOCK_STREAM, 0, sp) == 0); CHECK((pid = fork()) >= 0); if (pid == 0) { CHECK(close(sp[0]) == 0); fcntl_tests_send_0(sp[1]); CHECK(close(sp[1]) == 0); exit(0); } else { CHECK(close(sp[1]) == 0); fcntl_tests_recv_0(sp[0]); CHECK(waitpid(pid, NULL, 0) == pid); CHECK(close(sp[0]) == 0); } exit(0); } Index: projects/netbsd-tests-update-12/tools/regression/capsicum/syscalls/cap_ioctls_limit.c =================================================================== --- projects/netbsd-tests-update-12/tools/regression/capsicum/syscalls/cap_ioctls_limit.c (revision 305171) +++ projects/netbsd-tests-update-12/tools/regression/capsicum/syscalls/cap_ioctls_limit.c (revision 305172) @@ -1,462 +1,470 @@ /*- * Copyright (c) 2012 The FreeBSD Foundation * All rights reserved. * * This software was developed by Pawel Jakub Dawidek under sponsorship from * the FreeBSD Foundation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include "misc.h" static void ioctl_tests_0(int fd) { unsigned long cmds[2]; CHECK(cap_ioctls_get(fd, NULL, 0) == CAP_IOCTLS_ALL); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(ioctl(fd, FIOCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(ioctl(fd, FIONCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == 0); cmds[0] = FIOCLEX; cmds[1] = FIONCLEX; CHECK(cap_ioctls_limit(fd, cmds, nitems(cmds)) == 0); cmds[0] = cmds[1] = 0; CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == nitems(cmds)); CHECK((cmds[0] == FIOCLEX && cmds[1] == FIONCLEX) || (cmds[0] == FIONCLEX && cmds[1] == FIOCLEX)); cmds[0] = FIOCLEX; cmds[1] = FIONCLEX; CHECK(cap_ioctls_limit(fd, cmds, nitems(cmds)) == 0); cmds[0] = cmds[1] = 0; CHECK(cap_ioctls_get(fd, cmds, 1) == nitems(cmds)); CHECK(cmds[0] == FIOCLEX || cmds[0] == FIONCLEX); CHECK(cmds[1] == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(ioctl(fd, FIOCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(ioctl(fd, FIONCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == 0); cmds[0] = FIOCLEX; CHECK(cap_ioctls_limit(fd, cmds, 1) == 0); cmds[0] = cmds[1] = 0; CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 1); CHECK(cmds[0] == FIOCLEX); cmds[0] = FIOCLEX; cmds[1] = FIONCLEX; errno = 0; CHECK(cap_ioctls_limit(fd, cmds, nitems(cmds)) == -1); CHECK(errno == ENOTCAPABLE); cmds[0] = cmds[1] = 0; CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 1); CHECK(cmds[0] == FIOCLEX); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(ioctl(fd, FIOCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); errno = 0; CHECK(ioctl(fd, FIONCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(cap_ioctls_limit(fd, NULL, 0) == 0); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); cmds[0] = FIOCLEX; errno = 0; CHECK(cap_ioctls_limit(fd, cmds, 1) == -1); CHECK(errno == ENOTCAPABLE); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); CHECK(fcntl(fd, F_GETFD) == 0); errno = 0; CHECK(ioctl(fd, FIOCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); errno = 0; CHECK(ioctl(fd, FIONCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); } static void ioctl_tests_1(int fd) { unsigned long cmds[2]; + cap_rights_t rights; cmds[0] = FIOCLEX; CHECK(cap_ioctls_limit(fd, cmds, 1) == 0); cmds[0] = cmds[1] = 0; CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 1); CHECK(cmds[0] == FIOCLEX); CHECK(cmds[1] == 0); - CHECK(cap_rights_limit(fd, CAP_ALL & ~CAP_IOCTL) == 0); + CAP_ALL(&rights); + cap_rights_clear(&rights, CAP_IOCTL); + + CHECK(cap_rights_limit(fd, &rights) == 0); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); cmds[0] = FIOCLEX; cmds[1] = FIONCLEX; errno = 0; CHECK(cap_ioctls_limit(fd, cmds, nitems(cmds)) == -1); CHECK(errno == ENOTCAPABLE); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); cmds[0] = FIOCLEX; errno = 0; CHECK(cap_ioctls_limit(fd, cmds, 1) == -1); CHECK(errno == ENOTCAPABLE); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); CHECK(fcntl(fd, F_GETFD) == 0); errno = 0; CHECK(ioctl(fd, FIOCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); errno = 0; CHECK(ioctl(fd, FIONCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); } static void ioctl_tests_2(int fd) { unsigned long cmds[2]; + cap_rights_t rights; - CHECK(cap_rights_limit(fd, CAP_ALL & ~CAP_IOCTL) == 0); + CAP_ALL(&rights); + cap_rights_clear(&rights, CAP_IOCTL); + + CHECK(cap_rights_limit(fd, &rights) == 0); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); cmds[0] = FIOCLEX; cmds[1] = FIONCLEX; errno = 0; CHECK(cap_ioctls_limit(fd, cmds, nitems(cmds)) == -1); CHECK(errno == ENOTCAPABLE); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); cmds[0] = FIOCLEX; errno = 0; CHECK(cap_ioctls_limit(fd, cmds, 1) == -1); CHECK(errno == ENOTCAPABLE); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); CHECK(fcntl(fd, F_GETFD) == 0); errno = 0; CHECK(ioctl(fd, FIOCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); errno = 0; CHECK(ioctl(fd, FIONCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); } static void ioctl_tests_send_0(int sock) { unsigned long cmds[2]; int fd; CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK(descriptor_send(sock, fd) == 0); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); cmds[0] = FIOCLEX; cmds[1] = FIONCLEX; CHECK(cap_ioctls_limit(fd, cmds, nitems(cmds)) == 0); CHECK(descriptor_send(sock, fd) == 0); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); cmds[0] = FIOCLEX; CHECK(cap_ioctls_limit(fd, cmds, 1) == 0); CHECK(descriptor_send(sock, fd) == 0); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); CHECK(cap_ioctls_limit(fd, NULL, 0) == 0); CHECK(descriptor_send(sock, fd) == 0); CHECK(close(fd) == 0); } static void ioctl_tests_recv_0(int sock) { unsigned long cmds[2]; int fd; CHECK(descriptor_recv(sock, &fd) == 0); CHECK(cap_ioctls_get(fd, NULL, 0) == CAP_IOCTLS_ALL); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(ioctl(fd, FIOCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(ioctl(fd, FIONCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(close(fd) == 0); CHECK(descriptor_recv(sock, &fd) == 0); cmds[0] = cmds[1] = 0; CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == nitems(cmds)); CHECK((cmds[0] == FIOCLEX && cmds[1] == FIONCLEX) || (cmds[0] == FIONCLEX && cmds[1] == FIOCLEX)); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(ioctl(fd, FIOCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(ioctl(fd, FIONCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(close(fd) == 0); CHECK(descriptor_recv(sock, &fd) == 0); cmds[0] = cmds[1] = 0; CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 1); CHECK(cmds[0] == FIOCLEX); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(ioctl(fd, FIOCLEX) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); errno = 0; CHECK(ioctl(fd, FIONCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(close(fd) == 0); CHECK(descriptor_recv(sock, &fd) == 0); CHECK(cap_ioctls_get(fd, cmds, nitems(cmds)) == 0); CHECK(fcntl(fd, F_GETFD) == 0); errno = 0; CHECK(ioctl(fd, FIOCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(fcntl(fd, F_SETFD, FD_CLOEXEC) == 0); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); errno = 0; CHECK(ioctl(fd, FIONCLEX) == -1); CHECK(errno == ENOTCAPABLE); CHECK(fcntl(fd, F_GETFD) == FD_CLOEXEC); CHECK(fcntl(fd, F_SETFD, 0) == 0); CHECK(fcntl(fd, F_GETFD) == 0); CHECK(close(fd) == 0); } int main(void) { int fd, pfd, sp[2]; pid_t pid; printf("1..607\n"); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); ioctl_tests_0(fd); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); ioctl_tests_1(fd); CHECK(close(fd) == 0); CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); ioctl_tests_2(fd); CHECK(close(fd) == 0); /* Child inherits descriptor and operates on it first. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); pid = fork(); switch (pid) { case -1: err(1, "fork() failed"); case 0: ioctl_tests_0(fd); CHECK(close(fd) == 0); exit(0); default: if (waitpid(pid, NULL, 0) == -1) err(1, "waitpid() failed"); ioctl_tests_0(fd); } CHECK(close(fd) == 0); /* Child inherits descriptor, but operates on it after parent. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); pid = fork(); switch (pid) { case -1: err(1, "fork() failed"); case 0: sleep(1); ioctl_tests_0(fd); CHECK(close(fd) == 0); exit(0); default: ioctl_tests_0(fd); if (waitpid(pid, NULL, 0) == -1) err(1, "waitpid() failed"); } CHECK(close(fd) == 0); /* Child inherits descriptor and operates on it first. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); pid = pdfork(&pfd, 0); switch (pid) { case -1: err(1, "pdfork() failed"); case 0: ioctl_tests_1(fd); exit(0); default: if (pdwait(pfd) == -1) err(1, "pdwait() failed"); close(pfd); ioctl_tests_1(fd); } CHECK(close(fd) == 0); /* Child inherits descriptor, but operates on it after parent. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); pid = pdfork(&pfd, 0); switch (pid) { case -1: err(1, "pdfork() failed"); case 0: sleep(1); ioctl_tests_1(fd); exit(0); default: ioctl_tests_1(fd); if (pdwait(pfd) == -1) err(1, "pdwait() failed"); close(pfd); } CHECK(close(fd) == 0); /* Child inherits descriptor and operates on it first. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); pid = fork(); switch (pid) { case -1: err(1, "fork() failed"); case 0: ioctl_tests_2(fd); exit(0); default: if (waitpid(pid, NULL, 0) == -1) err(1, "waitpid() failed"); ioctl_tests_2(fd); } CHECK(close(fd) == 0); /* Child inherits descriptor, but operates on it after parent. */ CHECK((fd = socket(AF_UNIX, SOCK_STREAM, 0)) >= 0); pid = fork(); switch (pid) { case -1: err(1, "fork() failed"); case 0: sleep(1); ioctl_tests_2(fd); exit(0); default: ioctl_tests_2(fd); if (waitpid(pid, NULL, 0) == -1) err(1, "waitpid() failed"); } CHECK(close(fd) == 0); /* Send descriptors from parent to child. */ CHECK(socketpair(AF_UNIX, SOCK_STREAM, 0, sp) == 0); CHECK((pid = fork()) >= 0); if (pid == 0) { CHECK(close(sp[0]) == 0); ioctl_tests_recv_0(sp[1]); CHECK(close(sp[1]) == 0); exit(0); } else { CHECK(close(sp[1]) == 0); ioctl_tests_send_0(sp[0]); CHECK(waitpid(pid, NULL, 0) == pid); CHECK(close(sp[0]) == 0); } /* Send descriptors from child to parent. */ CHECK(socketpair(AF_UNIX, SOCK_STREAM, 0, sp) == 0); CHECK((pid = fork()) >= 0); if (pid == 0) { CHECK(close(sp[0]) == 0); ioctl_tests_send_0(sp[1]); CHECK(close(sp[1]) == 0); exit(0); } else { CHECK(close(sp[1]) == 0); ioctl_tests_recv_0(sp[0]); CHECK(waitpid(pid, NULL, 0) == pid); CHECK(close(sp[0]) == 0); } exit(0); } Index: projects/netbsd-tests-update-12/tools/tools/crypto/cryptotest.c =================================================================== --- projects/netbsd-tests-update-12/tools/tools/crypto/cryptotest.c (revision 305171) +++ projects/netbsd-tests-update-12/tools/tools/crypto/cryptotest.c (revision 305172) @@ -1,622 +1,627 @@ /*- * Copyright (c) 2004 Sam Leffler, Errno Consulting * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer, * without modification. * 2. Redistributions in binary form must reproduce at minimum a disclaimer * similar to the "NO WARRANTY" disclaimer below ("Disclaimer") and any * redistribution must be conditioned upon including a substantially * similar Disclaimer requirement for further binary redistribution. * 3. Neither the names of the above-listed copyright holders nor the names * of any contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * NO WARRANTY * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTIBILITY * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL * THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, * OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER * IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF * THE POSSIBILITY OF SUCH DAMAGES. * * $FreeBSD$ */ /* * Simple tool for testing hardware/system crypto support. * * cryptotest [-czsbv] [-a algorithm] [count] [size ...] * * Run count iterations of a crypt+decrypt or mac operation on a buffer of * size bytes. A random key and iv are used. Options: * -c check the results * -d dev pin work on device dev * -z run all available algorithms on a variety of buffer sizes * -v be verbose * -b mark operations for batching * -p profile kernel crypto operations (must be root) * -t n fork n threads and run tests concurrently * Known algorithms are: * null null cbc * des des cbc * 3des 3des cbc * blf blowfish cbc * cast cast cbc * skj skipjack cbc * aes rijndael/aes 128-bit cbc * aes192 rijndael/aes 192-bit cbc * aes256 rijndael/aes 256-bit cbc * md5 md5 hmac * sha1 sha1 hmac * sha256 256-bit sha2 hmac * sha384 384-bit sha2 hmac * sha512 512--bit sha2 hmac * * For a test of how fast a crypto card is, use something like: * cryptotest -z 1024 * This will run a series of tests using the available crypto/cipher * algorithms over a variety of buffer sizes. The 1024 says to do 1024 * iterations. Extra arguments can be used to specify one or more buffer * sizes to use in doing tests. * * To fork multiple processes all doing the same work, specify -t X on the * command line to get X "threads" running simultaneously. No effort is made * to synchronize the threads or otherwise maximize load. * * If the kernel crypto code is built with CRYPTO_TIMING and you run as root, * then you can specify the -p option to get a "profile" of the time spent * processing crypto operations. At present this data is only meaningful for * symmetric operations. To get meaningful numbers you must run on an idle * machine. * * Expect ~400 Mb/s for a Broadcom 582x for 8K buffers on a reasonable CPU * (64-bit PCI helps). Hifn 7811 parts top out at ~110 Mb/s. */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #define CHUNK 64 /* how much to display */ #define streq(a,b) (strcasecmp(a,b) == 0) void hexdump(char *, int); int verbose = 0; int opflags = 0; int verify = 0; int crid = CRYPTO_FLAG_HARDWARE; struct alg { const char* name; int ishash; int blocksize; int minkeylen; int maxkeylen; int code; } algorithms[] = { #ifdef CRYPTO_NULL_CBC { "null", 0, 8, 1, 256, CRYPTO_NULL_CBC }, #endif { "des", 0, 8, 8, 8, CRYPTO_DES_CBC }, { "3des", 0, 8, 24, 24, CRYPTO_3DES_CBC }, { "blf", 0, 8, 5, 56, CRYPTO_BLF_CBC }, { "cast", 0, 8, 5, 16, CRYPTO_CAST_CBC }, { "skj", 0, 8, 10, 10, CRYPTO_SKIPJACK_CBC }, - { "aes", 0, 16, 16, 16, CRYPTO_RIJNDAEL128_CBC}, - { "aes192", 0, 16, 24, 24, CRYPTO_RIJNDAEL128_CBC}, - { "aes256", 0, 16, 32, 32, CRYPTO_RIJNDAEL128_CBC}, + { "rij", 0, 16, 16, 16, CRYPTO_RIJNDAEL128_CBC}, + { "aes", 0, 16, 16, 16, CRYPTO_AES_CBC}, + { "aes192", 0, 16, 24, 24, CRYPTO_AES_CBC}, + { "aes256", 0, 16, 32, 32, CRYPTO_AES_CBC}, #ifdef notdef { "arc4", 0, 8, 1, 32, CRYPTO_ARC4 }, #endif { "md5", 1, 8, 16, 16, CRYPTO_MD5_HMAC }, { "sha1", 1, 8, 20, 20, CRYPTO_SHA1_HMAC }, { "sha256", 1, 8, 32, 32, CRYPTO_SHA2_256_HMAC }, { "sha384", 1, 8, 48, 48, CRYPTO_SHA2_384_HMAC }, { "sha512", 1, 8, 64, 64, CRYPTO_SHA2_512_HMAC }, }; -static void +void usage(const char* cmd) { printf("usage: %s [-czsbv] [-d dev] [-a algorithm] [count] [size ...]\n", cmd); printf("where algorithm is one of:\n"); - printf(" des 3des (default) blowfish cast skipjack\n"); - printf(" aes (aka rijndael) aes192 aes256 arc4\n"); + printf(" des 3des (default) blowfish cast skipjack rij\n"); + printf(" aes aes192 aes256 arc4\n"); printf("count is the number of encrypt/decrypt ops to do\n"); printf("size is the number of bytes of text to encrypt+decrypt\n"); printf("\n"); printf("-c check the results (slows timing)\n"); - printf("-d use specific device\n"); + printf("-d use specific device, specify 'soft' for testing software implementations\n"); + printf("\tNOTE: to use software you must set:\n\t sysctl kern.cryptodevallowsoft=1\n"); printf("-z run all available algorithms on a variety of sizes\n"); printf("-v be verbose\n"); printf("-b mark operations for batching\n"); printf("-p profile kernel crypto operation (must be root)\n"); exit(-1); } -static struct alg* +struct alg* getalgbycode(int cipher) { int i; for (i = 0; i < nitems(algorithms); i++) if (cipher == algorithms[i].code) return &algorithms[i]; return NULL; } -static struct alg* +struct alg* getalgbyname(const char* name) { int i; for (i = 0; i < nitems(algorithms); i++) if (streq(name, algorithms[i].name)) return &algorithms[i]; return NULL; } -static int +int devcrypto(void) { - static int fd = -1; + int fd = -1; if (fd < 0) { fd = open(_PATH_DEV "crypto", O_RDWR, 0); if (fd < 0) err(1, _PATH_DEV "crypto"); if (fcntl(fd, F_SETFD, 1) == -1) err(1, "fcntl(F_SETFD) (devcrypto)"); } return fd; } -static int +int crlookup(const char *devname) { struct crypt_find_op find; + if (strncmp(devname, "soft", 4) == 0) + return CRYPTO_FLAG_SOFTWARE; + find.crid = -1; strlcpy(find.name, devname, sizeof(find.name)); if (ioctl(devcrypto(), CIOCFINDDEV, &find) == -1) err(1, "ioctl(CIOCFINDDEV)"); return find.crid; } -static const char * +const char * crfind(int crid) { - static struct crypt_find_op find; + struct crypt_find_op find; bzero(&find, sizeof(find)); find.crid = crid; if (ioctl(devcrypto(), CRIOFINDDEV, &find) == -1) err(1, "ioctl(CIOCFINDDEV): crid %d", crid); return find.name; } -static int +int crget(void) { int fd; if (ioctl(devcrypto(), CRIOGET, &fd) == -1) err(1, "ioctl(CRIOGET)"); if (fcntl(fd, F_SETFD, 1) == -1) err(1, "fcntl(F_SETFD) (crget)"); return fd; } -static char +char rdigit(void) { const char a[] = { 0x10,0x54,0x11,0x48,0x45,0x12,0x4f,0x13,0x49,0x53,0x14,0x41, 0x15,0x16,0x4e,0x55,0x54,0x17,0x18,0x4a,0x4f,0x42,0x19,0x01 }; return 0x20+a[random()%nitems(a)]; } -static void +void runtest(struct alg *alg, int count, int size, u_long cmd, struct timeval *tv) { int i, fd = crget(); struct timeval start, stop, dt; char *cleartext, *ciphertext, *originaltext; struct session2_op sop; struct crypt_op cop; char iv[EALG_MAX_BLOCK_LEN]; bzero(&sop, sizeof(sop)); if (!alg->ishash) { sop.keylen = (alg->minkeylen + alg->maxkeylen)/2; sop.key = (char *) malloc(sop.keylen); if (sop.key == NULL) err(1, "malloc (key)"); for (i = 0; i < sop.keylen; i++) sop.key[i] = rdigit(); sop.cipher = alg->code; } else { sop.mackeylen = (alg->minkeylen + alg->maxkeylen)/2; sop.mackey = (char *) malloc(sop.mackeylen); if (sop.mackey == NULL) err(1, "malloc (mac)"); for (i = 0; i < sop.mackeylen; i++) sop.mackey[i] = rdigit(); sop.mac = alg->code; } sop.crid = crid; if (ioctl(fd, cmd, &sop) < 0) { if (cmd == CIOCGSESSION || cmd == CIOCGSESSION2) { close(fd); if (verbose) { printf("cipher %s", alg->name); if (alg->ishash) printf(" mackeylen %u\n", sop.mackeylen); else printf(" keylen %u\n", sop.keylen); perror("CIOCGSESSION"); } /* hardware doesn't support algorithm; skip it */ return; } printf("cipher %s keylen %u mackeylen %u\n", alg->name, sop.keylen, sop.mackeylen); err(1, "CIOCGSESSION"); } originaltext = malloc(3*size); if (originaltext == NULL) err(1, "malloc (text)"); cleartext = originaltext+size; ciphertext = cleartext+size; for (i = 0; i < size; i++) cleartext[i] = rdigit(); memcpy(originaltext, cleartext, size); for (i = 0; i < nitems(iv); i++) iv[i] = rdigit(); if (verbose) { printf("session = 0x%x\n", sop.ses); printf("device = %s\n", crfind(sop.crid)); printf("count = %d, size = %d\n", count, size); if (!alg->ishash) { printf("iv:"); hexdump(iv, sizeof iv); } printf("cleartext:"); hexdump(cleartext, MIN(size, CHUNK)); } gettimeofday(&start, NULL); if (!alg->ishash) { for (i = 0; i < count; i++) { cop.ses = sop.ses; cop.op = COP_ENCRYPT; cop.flags = opflags; cop.len = size; cop.src = cleartext; cop.dst = ciphertext; cop.mac = 0; cop.iv = iv; if (ioctl(fd, CIOCCRYPT, &cop) < 0) err(1, "ioctl(CIOCCRYPT)"); if (verify && bcmp(ciphertext, cleartext, size) == 0) { printf("cipher text unchanged:"); hexdump(ciphertext, size); } memset(cleartext, 'x', MIN(size, CHUNK)); cop.ses = sop.ses; cop.op = COP_DECRYPT; cop.flags = opflags; cop.len = size; cop.src = ciphertext; cop.dst = cleartext; cop.mac = 0; cop.iv = iv; if (ioctl(fd, CIOCCRYPT, &cop) < 0) err(1, "ioctl(CIOCCRYPT)"); if (verify && bcmp(cleartext, originaltext, size) != 0) { printf("decrypt mismatch:\n"); printf("original:"); hexdump(originaltext, size); printf("cleartext:"); hexdump(cleartext, size); } } } else { for (i = 0; i < count; i++) { cop.ses = sop.ses; cop.op = 0; cop.flags = opflags; cop.len = size; cop.src = cleartext; cop.dst = 0; cop.mac = ciphertext; cop.iv = 0; if (ioctl(fd, CIOCCRYPT, &cop) < 0) err(1, "ioctl(CIOCCRYPT)"); } } gettimeofday(&stop, NULL); if (ioctl(fd, CIOCFSESSION, &sop.ses) < 0) perror("ioctl(CIOCFSESSION)"); if (verbose) { printf("cleartext:"); hexdump(cleartext, MIN(size, CHUNK)); } timersub(&stop, &start, tv); free(originaltext); close(fd); } #ifdef __FreeBSD__ -static void +void resetstats() { struct cryptostats stats; size_t slen; slen = sizeof (stats); if (sysctlbyname("kern.crypto_stats", &stats, &slen, NULL, 0) < 0) { perror("kern.crypto_stats"); return; } bzero(&stats.cs_invoke, sizeof (stats.cs_invoke)); bzero(&stats.cs_done, sizeof (stats.cs_done)); bzero(&stats.cs_cb, sizeof (stats.cs_cb)); bzero(&stats.cs_finis, sizeof (stats.cs_finis)); stats.cs_invoke.min.tv_sec = 10000; stats.cs_done.min.tv_sec = 10000; stats.cs_cb.min.tv_sec = 10000; stats.cs_finis.min.tv_sec = 10000; if (sysctlbyname("kern.crypto_stats", NULL, NULL, &stats, sizeof (stats)) < 0) perror("kern.cryptostats"); } -static void +void printt(const char* tag, struct cryptotstat *ts) { uint64_t avg, min, max; if (ts->count == 0) return; avg = (1000000000LL*ts->acc.tv_sec + ts->acc.tv_nsec) / ts->count; min = 1000000000LL*ts->min.tv_sec + ts->min.tv_nsec; max = 1000000000LL*ts->max.tv_sec + ts->max.tv_nsec; printf("%16.16s: avg %6llu ns : min %6llu ns : max %7llu ns [%u samps]\n", tag, avg, min, max, ts->count); } #endif -static void +void runtests(struct alg *alg, int count, int size, u_long cmd, int threads, int profile) { int i, status; double t; void *region; struct timeval *tvp; struct timeval total; int otiming; if (size % alg->blocksize) { if (verbose) printf("skipping blocksize %u 'cuz not a multiple of " "%s blocksize %u\n", size, alg->name, alg->blocksize); return; } region = mmap(NULL, threads * sizeof (struct timeval), PROT_READ|PROT_WRITE, MAP_ANON|MAP_SHARED, -1, 0); if (region == MAP_FAILED) { perror("mmap"); return; } tvp = (struct timeval *) region; #ifdef __FreeBSD__ if (profile) { size_t tlen = sizeof (otiming); int timing = 1; resetstats(); if (sysctlbyname("debug.crypto_timing", &otiming, &tlen, &timing, sizeof (timing)) < 0) perror("debug.crypto_timing"); } #endif if (threads > 1) { for (i = 0; i < threads; i++) if (fork() == 0) { runtest(alg, count, size, cmd, &tvp[i]); exit(0); } while (waitpid(WAIT_MYPGRP, &status, 0) != -1) ; } else runtest(alg, count, size, cmd, tvp); t = 0; for (i = 0; i < threads; i++) t += (((double)tvp[i].tv_sec * 1000000 + tvp[i].tv_usec) / 1000000); if (t) { int nops = alg->ishash ? count : 2*count; #if 0 t /= threads; printf("%6.3lf sec, %7d %6s crypts, %7d bytes, %8.0lf byte/sec, %7.1lf Mb/sec\n", t, nops, alg->name, size, (double)nops*size / t, (double)nops*size / t * 8 / 1024 / 1024); #else nops *= threads; printf("%8.3lf sec, %7d %6s crypts, %7d bytes, %8.0lf byte/sec, %7.1lf Mb/sec\n", t, nops, alg->name, size, (double)nops*size / t, (double)nops*size / t * 8 / 1024 / 1024); #endif } #ifdef __FreeBSD__ if (profile) { struct cryptostats stats; size_t slen = sizeof (stats); if (sysctlbyname("debug.crypto_timing", NULL, NULL, &otiming, sizeof (otiming)) < 0) perror("debug.crypto_timing"); if (sysctlbyname("kern.crypto_stats", &stats, &slen, NULL, 0) < 0) perror("kern.cryptostats"); if (stats.cs_invoke.count) { printt("dispatch->invoke", &stats.cs_invoke); printt("invoke->done", &stats.cs_done); printt("done->cb", &stats.cs_cb); printt("cb->finis", &stats.cs_finis); } } #endif fflush(stdout); } int main(int argc, char **argv) { struct alg *alg = NULL; int count = 1; int sizes[128], nsizes = 0; u_long cmd = CIOCGSESSION2; int testall = 0; int maxthreads = 1; int profile = 0; int i, ch; while ((ch = getopt(argc, argv, "cpzsva:bd:t:")) != -1) { switch (ch) { #ifdef CIOCGSSESSION case 's': cmd = CIOCGSSESSION; break; #endif case 'v': verbose++; break; case 'a': alg = getalgbyname(optarg); if (alg == NULL) { if (streq(optarg, "rijndael")) alg = getalgbyname("aes"); else usage(argv[0]); } break; case 'd': crid = crlookup(optarg); break; case 't': maxthreads = atoi(optarg); break; case 'z': testall = 1; break; case 'p': profile = 1; break; case 'b': opflags |= COP_F_BATCH; break; case 'c': verify = 1; break; default: usage(argv[0]); } } argc -= optind, argv += optind; if (argc > 0) count = atoi(argv[0]); while (argc > 1) { int s = atoi(argv[1]); if (nsizes < nitems(sizes)) { sizes[nsizes++] = s; } else { printf("Too many sizes, ignoring %u\n", s); } argc--, argv++; } if (nsizes == 0) { if (alg) sizes[nsizes++] = alg->blocksize; else sizes[nsizes++] = 8; if (testall) { while (sizes[nsizes-1] < 8*1024) { sizes[nsizes] = sizes[nsizes-1]<<1; nsizes++; } } } if (testall) { for (i = 0; i < nitems(algorithms); i++) { int j; alg = &algorithms[i]; for (j = 0; j < nsizes; j++) runtests(alg, count, sizes[j], cmd, maxthreads, profile); } } else { if (alg == NULL) alg = getalgbycode(CRYPTO_3DES_CBC); for (i = 0; i < nsizes; i++) runtests(alg, count, sizes[i], cmd, maxthreads, profile); } return (0); } void hexdump(char *p, int n) { int i, off; for (off = 0; n > 0; off += 16, n -= 16) { printf("%s%04x:", off == 0 ? "\n" : "", off); i = (n >= 16 ? 16 : n); do { printf(" %02x", *p++ & 0xff); } while (--i); printf("\n"); } } Index: projects/netbsd-tests-update-12/usr.bin/Makefile =================================================================== --- projects/netbsd-tests-update-12/usr.bin/Makefile (revision 305171) +++ projects/netbsd-tests-update-12/usr.bin/Makefile (revision 305172) @@ -1,310 +1,310 @@ # From: @(#)Makefile 8.3 (Berkeley) 1/7/94 # $FreeBSD$ .include # XXX MISSING: deroff diction graph learn plot # spell spline struct xsend # XXX Use GNU versions: diff ld patch # Moved to secure: bdes # SUBDIR= alias \ apply \ asa \ awk \ banner \ basename \ brandelf \ bsdcat \ bsdiff \ bzip2 \ bzip2recover \ cap_mkdb \ chat \ chpass \ cksum \ cmp \ col \ colldef \ colrm \ column \ comm \ compress \ cpuset \ csplit \ ctlstat \ cut \ dirname \ dpv \ du \ elf2aout \ elfdump \ enigma \ env \ expand \ false \ fetch \ find \ fmt \ fold \ fstat \ fsync \ gcore \ gencat \ getconf \ getent \ getopt \ grep \ gzip \ head \ hexdump \ id \ ident \ ipcrm \ ipcs \ join \ jot \ keylogin \ keylogout \ killall \ ktrace \ ktrdump \ lam \ lastcomm \ ldd \ leave \ less \ lessecho \ lesskey \ limits \ locale \ localedef \ lock \ lockf \ logger \ login \ logins \ logname \ look \ lorder \ lsvfs \ lzmainfo \ m4 \ mandoc \ mesg \ minigzip \ ministat \ mkdep \ mkfifo \ mkimg \ mklocale \ mktemp \ mkuzip \ mt \ ncal \ netstat \ newgrp \ nfsstat \ nice \ nl \ numactl \ nohup \ opieinfo \ opiekey \ opiepasswd \ pagesize \ passwd \ paste \ patch \ pathchk \ perror \ pr \ printenv \ printf \ procstat \ protect \ rctl \ renice \ resizewin \ rev \ revoke \ rpcinfo \ rs \ rup \ rusers \ rwall \ script \ sdiff \ sed \ send-pr \ seq \ shar \ showmount \ sockstat \ soelim \ sort \ split \ stat \ stdbuf \ su \ systat \ tabs \ tail \ tar \ tcopy \ tee \ time \ timeout \ tip \ top \ touch \ tput \ tr \ true \ truncate \ tset \ tsort \ tty \ uname \ unexpand \ uniq \ unzip \ units \ unvis \ uudecode \ uuencode \ vis \ vmstat \ w \ wall \ wc \ what \ whereis \ which \ whois \ write \ xargs \ xinstall \ xo \ xz \ xzdec \ yes # NB: keep these sorted by MK_* knobs SUBDIR.${MK_AT}+= at SUBDIR.${MK_ATM}+= atm SUBDIR.${MK_BLUETOOTH}+= bluetooth SUBDIR.${MK_BSD_CPIO}+= cpio SUBDIR.${MK_CALENDAR}+= calendar SUBDIR.${MK_CLANG}+= clang SUBDIR.${MK_EE}+= ee SUBDIR.${MK_FILE}+= file SUBDIR.${MK_FINGER}+= finger SUBDIR.${MK_FTP}+= ftp SUBDIR.${MK_GAMES}+= caesar SUBDIR.${MK_GAMES}+= factor SUBDIR.${MK_GAMES}+= fortune SUBDIR.${MK_GAMES}+= grdc SUBDIR.${MK_GAMES}+= morse SUBDIR.${MK_GAMES}+= number SUBDIR.${MK_GAMES}+= pom SUBDIR.${MK_GAMES}+= primes SUBDIR.${MK_GAMES}+= random .if ${MK_GPL_DTC} != "yes" .if ${COMPILER_FEATURES:Mc++11} SUBDIR+= dtc .endif .endif SUBDIR.${MK_GROFF}+= vgrind SUBDIR.${MK_HESIOD}+= hesinfo SUBDIR.${MK_ICONV}+= iconv SUBDIR.${MK_ICONV}+= mkcsmapper SUBDIR.${MK_ICONV}+= mkesdb SUBDIR.${MK_ISCSI}+= iscsictl SUBDIR.${MK_KDUMP}+= kdump SUBDIR.${MK_KDUMP}+= truss SUBDIR.${MK_KERBEROS_SUPPORT}+= compile_et SUBDIR.${MK_LDNS_UTILS}+= drill SUBDIR.${MK_LDNS_UTILS}+= host SUBDIR.${MK_LOCATE}+= locate # XXX msgs? SUBDIR.${MK_MAIL}+= biff SUBDIR.${MK_MAIL}+= from SUBDIR.${MK_MAIL}+= mail SUBDIR.${MK_MAIL}+= msgs SUBDIR.${MK_MAKE}+= bmake SUBDIR.${MK_MAN_UTILS}+= catman .if ${MK_MANDOCDB} == "no" # AND SUBDIR.${MK_MAN_UTILS}+= makewhatis .endif SUBDIR.${MK_MAN_UTILS}+= man SUBDIR.${MK_NETCAT}+= nc SUBDIR.${MK_NIS}+= ypcat SUBDIR.${MK_NIS}+= ypmatch SUBDIR.${MK_NIS}+= ypwhich SUBDIR.${MK_OPENSSH}+= ssh-copy-id SUBDIR.${MK_OPENSSL}+= bc SUBDIR.${MK_OPENSSL}+= chkey SUBDIR.${MK_OPENSSL}+= dc SUBDIR.${MK_OPENSSL}+= newkey SUBDIR.${MK_QUOTAS}+= quota SUBDIR.${MK_RCMDS}+= rlogin SUBDIR.${MK_RCMDS}+= rsh SUBDIR.${MK_RCMDS}+= ruptime SUBDIR.${MK_RCMDS}+= rwho SUBDIR.${MK_SENDMAIL}+= vacation SUBDIR.${MK_TALK}+= talk SUBDIR.${MK_TELNET}+= telnet SUBDIR.${MK_TESTS}+= tests SUBDIR.${MK_TEXTPROC}+= checknr SUBDIR.${MK_TEXTPROC}+= colcrt SUBDIR.${MK_TEXTPROC}+= ul SUBDIR.${MK_TFTP}+= tftp SUBDIR.${MK_TOOLCHAIN}+= addr2line SUBDIR.${MK_TOOLCHAIN}+= ar SUBDIR.${MK_TOOLCHAIN}+= c89 SUBDIR.${MK_TOOLCHAIN}+= c99 SUBDIR.${MK_TOOLCHAIN}+= ctags SUBDIR.${MK_TOOLCHAIN}+= cxxfilt SUBDIR.${MK_TOOLCHAIN}+= elfcopy SUBDIR.${MK_TOOLCHAIN}+= file2c # ARM64TODO gprof does not build # RISCVTODO gprof does not build .if ${MACHINE_ARCH} != "aarch64" && ${MACHINE_CPUARCH} != "riscv" SUBDIR.${MK_TOOLCHAIN}+= gprof .endif SUBDIR.${MK_TOOLCHAIN}+= indent SUBDIR.${MK_TOOLCHAIN}+= lex SUBDIR.${MK_TOOLCHAIN}+= mkstr SUBDIR.${MK_TOOLCHAIN}+= nm SUBDIR.${MK_TOOLCHAIN}+= readelf SUBDIR.${MK_TOOLCHAIN}+= rpcgen SUBDIR.${MK_TOOLCHAIN}+= unifdef SUBDIR.${MK_TOOLCHAIN}+= size SUBDIR.${MK_TOOLCHAIN}+= strings .if ${MACHINE_ARCH} != "aarch64" # ARM64TODO xlint does not build SUBDIR.${MK_TOOLCHAIN}+= xlint .endif SUBDIR.${MK_TOOLCHAIN}+= xstr SUBDIR.${MK_TOOLCHAIN}+= yacc SUBDIR.${MK_VI}+= vi SUBDIR.${MK_VT}+= vtfontcvt SUBDIR.${MK_USB}+= usbhidaction SUBDIR.${MK_USB}+= usbhidctl SUBDIR.${MK_UTMPX}+= last -.if ${MACHINE_CPUARCH} != "riscv" # RISCVTODO users does not build +.if ${MK_CXX} != "no" SUBDIR.${MK_UTMPX}+= users .endif SUBDIR.${MK_UTMPX}+= who SUBDIR.${MK_SVN}+= svn SUBDIR.${MK_SVNLITE}+= svn .include SUBDIR:= ${SUBDIR:O:u} SUBDIR_PARALLEL= .include Index: projects/netbsd-tests-update-12/usr.bin/bsdcat/tests/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.bin/bsdcat/tests/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/usr.bin/bsdcat/tests/Makefile.depend (revision 305172) @@ -0,0 +1,25 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + include/xlocale \ + lib/${CSU_DIR} \ + lib/libarchive \ + lib/libbz2 \ + lib/libc \ + lib/libcompiler_rt \ + lib/libexpat \ + lib/liblzma \ + lib/libthr \ + lib/libz \ + secure/lib/libcrypto \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/usr.bin/bsdcat/tests/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/usr.bin/newkey/generic.c =================================================================== --- projects/netbsd-tests-update-12/usr.bin/newkey/generic.c (revision 305171) +++ projects/netbsd-tests-update-12/usr.bin/newkey/generic.c (revision 305172) @@ -1,132 +1,132 @@ /* * Sun RPC is a product of Sun Microsystems, Inc. and is provided for * unrestricted use provided that this legend is included on all tape * media and as a part of the software program in whole or part. Users * may copy or modify Sun RPC without charge, but are not authorized * to license or distribute it to anyone else except as part of a product or * program developed by the user or with the express written consent of * Sun Microsystems, Inc. * * SUN RPC IS PROVIDED AS IS WITH NO WARRANTIES OF ANY KIND INCLUDING THE * WARRANTIES OF DESIGN, MERCHANTIBILITY AND FITNESS FOR A PARTICULAR * PURPOSE, OR ARISING FROM A COURSE OF DEALING, USAGE OR TRADE PRACTICE. * * Sun RPC is provided with no support and without any obligation on the * part of Sun Microsystems, Inc. to assist in its use, correction, * modification or enhancement. * * SUN MICROSYSTEMS, INC. SHALL HAVE NO LIABILITY WITH RESPECT TO THE * INFRINGEMENT OF COPYRIGHTS, TRADE SECRETS OR ANY PATENTS BY SUN RPC * OR ANY PART THEREOF. * * In no event will Sun Microsystems, Inc. be liable for any lost revenue * or profits or other special, indirect and consequential damages, even if * Sun has been advised of the possibility of such damages. * * Sun Microsystems, Inc. * 2550 Garcia Avenue * Mountain View, California 94043 */ #if !defined(lint) && defined(SCCSIDS) #if 0 static char sccsid[] = "@(#)generic.c 1.2 91/03/11 Copyr 1986 Sun Micro"; #endif #endif /* * Copyright (C) 1986, Sun Microsystems, Inc. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include "extern.h" static void adjust(char[], char *); static void getseed(char *, int, unsigned char *); /* * Generate a seed */ static void getseed(char *seed, int seedsize, unsigned char *pass) { int i; for (i = 0; i < seedsize; i++) { seed[i] = (arc4random() & 0xff) ^ pass[i % 8]; } } /* * Generate a random public/secret key pair */ void genkeys(char *public, char *secret, char *pass) { unsigned int i; # define BASEBITS (8*sizeof (short) - 1) # define BASE (1 << BASEBITS) MINT *pk = mp_itom(0); MINT *sk = mp_itom(0); MINT *tmp; - MINT *base = mp_itom(BASE); + MINT *base = mp_itom((short)BASE); MINT *root = mp_itom(PROOT); MINT *modulus = mp_xtom(HEXMODULUS); short r; unsigned short seed[KEYSIZE/BASEBITS + 1]; char *xkey; getseed((char *)seed, sizeof (seed), (u_char *)pass); for (i = 0; i < KEYSIZE/BASEBITS + 1; i++) { r = seed[i] % BASE; tmp = mp_itom(r); mp_mult(sk, base, sk); mp_madd(sk, tmp, sk); mp_mfree(tmp); } tmp = mp_itom(0); mp_mdiv(sk, modulus, tmp, sk); mp_mfree(tmp); mp_pow(root, sk, modulus, pk); xkey = mp_mtox(sk); adjust(secret, xkey); xkey = mp_mtox(pk); adjust(public, xkey); mp_mfree(sk); mp_mfree(base); mp_mfree(pk); mp_mfree(root); mp_mfree(modulus); } /* * Adjust the input key so that it is 0-filled on the left */ static void adjust(char keyout[HEXKEYBYTES+1], char *keyin) { char *p; char *s; for (p = keyin; *p; p++) ; for (s = keyout + HEXKEYBYTES; p >= keyin; p--, s--) { *s = *p; } while (s >= keyout) { *s-- = '0'; } } Index: projects/netbsd-tests-update-12/usr.bin/nfsstat/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.bin/nfsstat/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/usr.bin/nfsstat/Makefile.depend (revision 305172) @@ -1,18 +1,21 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/libc \ lib/libcompiler_rt \ + lib/libdevstat \ + lib/libelf \ + lib/libkvm \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/usr.bin/sdiff/tests/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.bin/sdiff/tests/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/usr.bin/sdiff/tests/Makefile.depend (revision 305172) @@ -0,0 +1,11 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/usr.bin/sdiff/tests/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/usr.bin/tar/tests/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.bin/tar/tests/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/usr.bin/tar/tests/Makefile.depend (revision 305172) @@ -1,19 +1,25 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ gnu/lib/csu \ gnu/lib/libgcc \ include \ include/xlocale \ lib/${CSU_DIR} \ lib/libarchive \ + lib/libbz2 \ lib/libc \ lib/libcompiler_rt \ + lib/libexpat \ + lib/liblzma \ + lib/libthr \ + lib/libz \ + secure/lib/libcrypto \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/usr.bin/xinstall/tests/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.bin/xinstall/tests/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/usr.bin/xinstall/tests/Makefile.depend (revision 305172) @@ -0,0 +1,11 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/usr.bin/xinstall/tests/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/usr.sbin/bhyve/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.sbin/bhyve/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/usr.sbin/bhyve/Makefile.depend (revision 305172) @@ -1,22 +1,24 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ gnu/lib/csu \ gnu/lib/libgcc \ include \ + include/arpa \ include/xlocale \ lib/${CSU_DIR} \ lib/libc \ lib/libcompiler_rt \ lib/libmd \ lib/libthr \ lib/libutil \ lib/libvmmapi \ + lib/libz \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/usr.sbin/bhyve/pci_e82545.c =================================================================== --- projects/netbsd-tests-update-12/usr.sbin/bhyve/pci_e82545.c (revision 305171) +++ projects/netbsd-tests-update-12/usr.sbin/bhyve/pci_e82545.c (revision 305172) @@ -1,2372 +1,2372 @@ /* * Copyright (c) 2016 Alexander Motin * Copyright (c) 2015 Peter Grehan * Copyright (c) 2013 Jeremiah Lott, Avere Systems * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer * in this position and unchanged. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "e1000_regs.h" #include "e1000_defines.h" #include "mii.h" #include "bhyverun.h" #include "pci_emul.h" #include "mevent.h" /* Hardware/register definitions XXX: move some to common code. */ #define E82545_VENDOR_ID_INTEL 0x8086 #define E82545_DEV_ID_82545EM_COPPER 0x100F #define E82545_SUBDEV_ID 0x1008 #define E82545_REVISION_4 4 #define E82545_MDIC_DATA_MASK 0x0000FFFF #define E82545_MDIC_OP_MASK 0x0c000000 #define E82545_MDIC_IE 0x20000000 #define E82545_EECD_FWE_DIS 0x00000010 /* Flash writes disabled */ #define E82545_EECD_FWE_EN 0x00000020 /* Flash writes enabled */ #define E82545_EECD_FWE_MASK 0x00000030 /* Flash writes mask */ #define E82545_BAR_REGISTER 0 #define E82545_BAR_REGISTER_LEN (128*1024) #define E82545_BAR_FLASH 1 #define E82545_BAR_FLASH_LEN (64*1024) #define E82545_BAR_IO 2 #define E82545_BAR_IO_LEN 8 #define E82545_IOADDR 0x00000000 #define E82545_IODATA 0x00000004 #define E82545_IO_REGISTER_MAX 0x0001FFFF #define E82545_IO_FLASH_BASE 0x00080000 #define E82545_IO_FLASH_MAX 0x000FFFFF #define E82545_ARRAY_ENTRY(reg, offset) (reg + (offset<<2)) #define E82545_RAR_MAX 15 #define E82545_MTA_MAX 127 #define E82545_VFTA_MAX 127 /* Slightly modified from the driver versions, hardcoded for 3 opcode bits, * followed by 6 address bits. * TODO: make opcode bits and addr bits configurable? * NVM Commands - Microwire */ #define E82545_NVM_OPCODE_BITS 3 #define E82545_NVM_ADDR_BITS 6 #define E82545_NVM_DATA_BITS 16 #define E82545_NVM_OPADDR_BITS (E82545_NVM_OPCODE_BITS + E82545_NVM_ADDR_BITS) #define E82545_NVM_ADDR_MASK ((1 << E82545_NVM_ADDR_BITS)-1) #define E82545_NVM_OPCODE_MASK \ (((1 << E82545_NVM_OPCODE_BITS) - 1) << E82545_NVM_ADDR_BITS) #define E82545_NVM_OPCODE_READ (0x6 << E82545_NVM_ADDR_BITS) /* read */ #define E82545_NVM_OPCODE_WRITE (0x5 << E82545_NVM_ADDR_BITS) /* write */ #define E82545_NVM_OPCODE_ERASE (0x7 << E82545_NVM_ADDR_BITS) /* erase */ #define E82545_NVM_OPCODE_EWEN (0x4 << E82545_NVM_ADDR_BITS) /* wr-enable */ #define E82545_NVM_EEPROM_SIZE 64 /* 64 * 16-bit values == 128K */ #define E1000_ICR_SRPD 0x00010000 /* This is an arbitrary number. There is no hard limit on the chip. */ #define I82545_MAX_TXSEGS 64 /* Legacy receive descriptor */ struct e1000_rx_desc { uint64_t buffer_addr; /* Address of the descriptor's data buffer */ uint16_t length; /* Length of data DMAed into data buffer */ uint16_t csum; /* Packet checksum */ uint8_t status; /* Descriptor status */ uint8_t errors; /* Descriptor Errors */ uint16_t special; }; /* Transmit descriptor types */ #define E1000_TXD_MASK (E1000_TXD_CMD_DEXT | 0x00F00000) #define E1000_TXD_TYP_L (0) #define E1000_TXD_TYP_C (E1000_TXD_CMD_DEXT | E1000_TXD_DTYP_C) #define E1000_TXD_TYP_D (E1000_TXD_CMD_DEXT | E1000_TXD_DTYP_D) /* Legacy transmit descriptor */ struct e1000_tx_desc { uint64_t buffer_addr; /* Address of the descriptor's data buffer */ union { uint32_t data; struct { uint16_t length; /* Data buffer length */ uint8_t cso; /* Checksum offset */ uint8_t cmd; /* Descriptor control */ } flags; } lower; union { uint32_t data; struct { uint8_t status; /* Descriptor status */ uint8_t css; /* Checksum start */ uint16_t special; } fields; } upper; }; /* Context descriptor */ struct e1000_context_desc { union { uint32_t ip_config; struct { uint8_t ipcss; /* IP checksum start */ uint8_t ipcso; /* IP checksum offset */ uint16_t ipcse; /* IP checksum end */ } ip_fields; } lower_setup; union { uint32_t tcp_config; struct { uint8_t tucss; /* TCP checksum start */ uint8_t tucso; /* TCP checksum offset */ uint16_t tucse; /* TCP checksum end */ } tcp_fields; } upper_setup; uint32_t cmd_and_length; union { uint32_t data; struct { uint8_t status; /* Descriptor status */ uint8_t hdr_len; /* Header length */ uint16_t mss; /* Maximum segment size */ } fields; } tcp_seg_setup; }; /* Data descriptor */ struct e1000_data_desc { uint64_t buffer_addr; /* Address of the descriptor's buffer address */ union { uint32_t data; struct { uint16_t length; /* Data buffer length */ uint8_t typ_len_ext; uint8_t cmd; } flags; } lower; union { uint32_t data; struct { uint8_t status; /* Descriptor status */ uint8_t popts; /* Packet Options */ uint16_t special; } fields; } upper; }; union e1000_tx_udesc { struct e1000_tx_desc td; struct e1000_context_desc cd; struct e1000_data_desc dd; }; /* Tx checksum info for a packet. */ struct ck_info { int ck_valid; /* ck_info is valid */ uint8_t ck_start; /* start byte of cksum calcuation */ uint8_t ck_off; /* offset of cksum insertion */ uint16_t ck_len; /* length of cksum calc: 0 is to packet-end */ }; /* * Debug printf */ static int e82545_debug = 0; #define DPRINTF(msg,params...) if (e82545_debug) fprintf(stderr, "e82545: " msg, params) #define WPRINTF(msg,params...) fprintf(stderr, "e82545: " msg, params) #define MIN(a,b) (((a)<(b))?(a):(b)) #define MAX(a,b) (((a)>(b))?(a):(b)) /* s/w representation of the RAL/RAH regs */ struct eth_uni { int eu_valid; int eu_addrsel; struct ether_addr eu_eth; }; struct e82545_softc { struct pci_devinst *esc_pi; struct vmctx *esc_ctx; struct mevent *esc_mevp; struct mevent *esc_mevpitr; pthread_mutex_t esc_mtx; struct ether_addr esc_mac; int esc_tapfd; /* General */ uint32_t esc_CTRL; /* x0000 device ctl */ uint32_t esc_FCAL; /* x0028 flow ctl addr lo */ uint32_t esc_FCAH; /* x002C flow ctl addr hi */ uint32_t esc_FCT; /* x0030 flow ctl type */ uint32_t esc_VET; /* x0038 VLAN eth type */ uint32_t esc_FCTTV; /* x0170 flow ctl tx timer */ uint32_t esc_LEDCTL; /* x0E00 LED control */ uint32_t esc_PBA; /* x1000 pkt buffer allocation */ /* Interrupt control */ int esc_irq_asserted; uint32_t esc_ICR; /* x00C0 cause read/clear */ uint32_t esc_ITR; /* x00C4 intr throttling */ uint32_t esc_ICS; /* x00C8 cause set */ uint32_t esc_IMS; /* x00D0 mask set/read */ uint32_t esc_IMC; /* x00D8 mask clear */ /* Transmit */ union e1000_tx_udesc *esc_txdesc; struct e1000_context_desc esc_txctx; pthread_t esc_tx_tid; pthread_cond_t esc_tx_cond; int esc_tx_enabled; int esc_tx_active; uint32_t esc_TXCW; /* x0178 transmit config */ uint32_t esc_TCTL; /* x0400 transmit ctl */ uint32_t esc_TIPG; /* x0410 inter-packet gap */ uint16_t esc_AIT; /* x0458 Adaptive Interframe Throttle */ uint64_t esc_tdba; /* verified 64-bit desc table addr */ uint32_t esc_TDBAL; /* x3800 desc table addr, low bits */ uint32_t esc_TDBAH; /* x3804 desc table addr, hi 32-bits */ uint32_t esc_TDLEN; /* x3808 # descriptors in bytes */ uint16_t esc_TDH; /* x3810 desc table head idx */ uint16_t esc_TDHr; /* internal read version of TDH */ uint16_t esc_TDT; /* x3818 desc table tail idx */ uint32_t esc_TIDV; /* x3820 intr delay */ uint32_t esc_TXDCTL; /* x3828 desc control */ uint32_t esc_TADV; /* x382C intr absolute delay */ /* L2 frame acceptance */ struct eth_uni esc_uni[16]; /* 16 x unicast MAC addresses */ uint32_t esc_fmcast[128]; /* Multicast filter bit-match */ uint32_t esc_fvlan[128]; /* VLAN 4096-bit filter */ /* Receive */ struct e1000_rx_desc *esc_rxdesc; pthread_cond_t esc_rx_cond; int esc_rx_enabled; int esc_rx_active; int esc_rx_loopback; uint32_t esc_RCTL; /* x0100 receive ctl */ uint32_t esc_FCRTL; /* x2160 flow cntl thresh, low */ uint32_t esc_FCRTH; /* x2168 flow cntl thresh, hi */ uint64_t esc_rdba; /* verified 64-bit desc table addr */ uint32_t esc_RDBAL; /* x2800 desc table addr, low bits */ uint32_t esc_RDBAH; /* x2804 desc table addr, hi 32-bits*/ uint32_t esc_RDLEN; /* x2808 #descriptors */ uint16_t esc_RDH; /* x2810 desc table head idx */ uint16_t esc_RDT; /* x2818 desc table tail idx */ uint32_t esc_RDTR; /* x2820 intr delay */ uint32_t esc_RXDCTL; /* x2828 desc control */ uint32_t esc_RADV; /* x282C intr absolute delay */ uint32_t esc_RSRPD; /* x2C00 recv small packet detect */ uint32_t esc_RXCSUM; /* x5000 receive cksum ctl */ /* IO Port register access */ uint32_t io_addr; /* Shadow copy of MDIC */ uint32_t mdi_control; /* Shadow copy of EECD */ uint32_t eeprom_control; /* Latest NVM in/out */ uint16_t nvm_data; uint16_t nvm_opaddr; /* stats */ uint32_t missed_pkt_count; /* dropped for no room in rx queue */ uint32_t pkt_rx_by_size[6]; uint32_t pkt_tx_by_size[6]; uint32_t good_pkt_rx_count; uint32_t bcast_pkt_rx_count; uint32_t mcast_pkt_rx_count; uint32_t good_pkt_tx_count; uint32_t bcast_pkt_tx_count; uint32_t mcast_pkt_tx_count; uint32_t oversize_rx_count; uint32_t tso_tx_count; uint64_t good_octets_rx; uint64_t good_octets_tx; uint64_t missed_octets; /* counts missed and oversized */ uint8_t nvm_bits:6; /* number of bits remaining in/out */ uint8_t nvm_mode:2; #define E82545_NVM_MODE_OPADDR 0x0 #define E82545_NVM_MODE_DATAIN 0x1 #define E82545_NVM_MODE_DATAOUT 0x2 /* EEPROM data */ uint16_t eeprom_data[E82545_NVM_EEPROM_SIZE]; }; static void e82545_reset(struct e82545_softc *sc, int dev); static void e82545_rx_enable(struct e82545_softc *sc); static void e82545_rx_disable(struct e82545_softc *sc); static void e82545_tap_callback(int fd, enum ev_type type, void *param); static void e82545_tx_start(struct e82545_softc *sc); static void e82545_tx_enable(struct e82545_softc *sc); static void e82545_tx_disable(struct e82545_softc *sc); static inline int e82545_size_stat_index(uint32_t size) { if (size <= 64) { return 0; } else if (size >= 1024) { return 5; } else { /* should be 1-4 */ return (ffs(size) - 6); } } static void e82545_init_eeprom(struct e82545_softc *sc) { uint16_t checksum, i; /* mac addr */ sc->eeprom_data[NVM_MAC_ADDR] = ((uint16_t)sc->esc_mac.octet[0]) | (((uint16_t)sc->esc_mac.octet[1]) << 8); sc->eeprom_data[NVM_MAC_ADDR+1] = ((uint16_t)sc->esc_mac.octet[2]) | (((uint16_t)sc->esc_mac.octet[3]) << 8); sc->eeprom_data[NVM_MAC_ADDR+2] = ((uint16_t)sc->esc_mac.octet[4]) | (((uint16_t)sc->esc_mac.octet[5]) << 8); /* pci ids */ sc->eeprom_data[NVM_SUB_DEV_ID] = E82545_SUBDEV_ID; sc->eeprom_data[NVM_SUB_VEN_ID] = E82545_VENDOR_ID_INTEL; sc->eeprom_data[NVM_DEV_ID] = E82545_DEV_ID_82545EM_COPPER; sc->eeprom_data[NVM_VEN_ID] = E82545_VENDOR_ID_INTEL; /* fill in the checksum */ checksum = 0; for (i = 0; i < NVM_CHECKSUM_REG; i++) { checksum += sc->eeprom_data[i]; } checksum = NVM_SUM - checksum; sc->eeprom_data[NVM_CHECKSUM_REG] = checksum; DPRINTF("eeprom checksum: 0x%x\r\n", checksum); } static void e82545_write_mdi(struct e82545_softc *sc, uint8_t reg_addr, uint8_t phy_addr, uint32_t data) { DPRINTF("Write mdi reg:0x%x phy:0x%x data: 0x%x\r\n", reg_addr, phy_addr, data); } static uint32_t e82545_read_mdi(struct e82545_softc *sc, uint8_t reg_addr, uint8_t phy_addr) { //DPRINTF("Read mdi reg:0x%x phy:0x%x\r\n", reg_addr, phy_addr); switch (reg_addr) { case PHY_STATUS: return (MII_SR_LINK_STATUS | MII_SR_AUTONEG_CAPS | MII_SR_AUTONEG_COMPLETE); case PHY_AUTONEG_ADV: return NWAY_AR_SELECTOR_FIELD; case PHY_LP_ABILITY: return 0; case PHY_1000T_STATUS: return (SR_1000T_LP_FD_CAPS | SR_1000T_REMOTE_RX_STATUS | SR_1000T_LOCAL_RX_STATUS); case PHY_ID1: return (M88E1011_I_PHY_ID >> 16) & 0xFFFF; case PHY_ID2: return (M88E1011_I_PHY_ID | E82545_REVISION_4) & 0xFFFF; default: DPRINTF("Unknown mdi read reg:0x%x phy:0x%x\r\n", reg_addr, phy_addr); return 0; } /* not reached */ } static void e82545_eecd_strobe(struct e82545_softc *sc) { /* Microwire state machine */ /* DPRINTF("eeprom state machine srtobe " "0x%x 0x%x 0x%x 0x%x\r\n", sc->nvm_mode, sc->nvm_bits, sc->nvm_opaddr, sc->nvm_data);*/ if (sc->nvm_bits == 0) { DPRINTF("eeprom state machine not expecting data! " "0x%x 0x%x 0x%x 0x%x\r\n", sc->nvm_mode, sc->nvm_bits, sc->nvm_opaddr, sc->nvm_data); return; } sc->nvm_bits--; if (sc->nvm_mode == E82545_NVM_MODE_DATAOUT) { /* shifting out */ if (sc->nvm_data & 0x8000) { sc->eeprom_control |= E1000_EECD_DO; } else { sc->eeprom_control &= ~E1000_EECD_DO; } sc->nvm_data <<= 1; if (sc->nvm_bits == 0) { /* read done, back to opcode mode. */ sc->nvm_opaddr = 0; sc->nvm_mode = E82545_NVM_MODE_OPADDR; sc->nvm_bits = E82545_NVM_OPADDR_BITS; } } else if (sc->nvm_mode == E82545_NVM_MODE_DATAIN) { /* shifting in */ sc->nvm_data <<= 1; if (sc->eeprom_control & E1000_EECD_DI) { sc->nvm_data |= 1; } if (sc->nvm_bits == 0) { /* eeprom write */ uint16_t op = sc->nvm_opaddr & E82545_NVM_OPCODE_MASK; uint16_t addr = sc->nvm_opaddr & E82545_NVM_ADDR_MASK; if (op != E82545_NVM_OPCODE_WRITE) { DPRINTF("Illegal eeprom write op 0x%x\r\n", sc->nvm_opaddr); } else if (addr >= E82545_NVM_EEPROM_SIZE) { DPRINTF("Illegal eeprom write addr 0x%x\r\n", sc->nvm_opaddr); } else { DPRINTF("eeprom write eeprom[0x%x] = 0x%x\r\n", addr, sc->nvm_data); sc->eeprom_data[addr] = sc->nvm_data; } /* back to opcode mode */ sc->nvm_opaddr = 0; sc->nvm_mode = E82545_NVM_MODE_OPADDR; sc->nvm_bits = E82545_NVM_OPADDR_BITS; } } else if (sc->nvm_mode == E82545_NVM_MODE_OPADDR) { sc->nvm_opaddr <<= 1; if (sc->eeprom_control & E1000_EECD_DI) { sc->nvm_opaddr |= 1; } if (sc->nvm_bits == 0) { uint16_t op = sc->nvm_opaddr & E82545_NVM_OPCODE_MASK; switch (op) { case E82545_NVM_OPCODE_EWEN: DPRINTF("eeprom write enable: 0x%x\r\n", sc->nvm_opaddr); /* back to opcode mode */ sc->nvm_opaddr = 0; sc->nvm_mode = E82545_NVM_MODE_OPADDR; sc->nvm_bits = E82545_NVM_OPADDR_BITS; break; case E82545_NVM_OPCODE_READ: { uint16_t addr = sc->nvm_opaddr & E82545_NVM_ADDR_MASK; sc->nvm_mode = E82545_NVM_MODE_DATAOUT; sc->nvm_bits = E82545_NVM_DATA_BITS; if (addr < E82545_NVM_EEPROM_SIZE) { sc->nvm_data = sc->eeprom_data[addr]; DPRINTF("eeprom read: eeprom[0x%x] = 0x%x\r\n", addr, sc->nvm_data); } else { DPRINTF("eeprom illegal read: 0x%x\r\n", sc->nvm_opaddr); sc->nvm_data = 0; } break; } case E82545_NVM_OPCODE_WRITE: sc->nvm_mode = E82545_NVM_MODE_DATAIN; sc->nvm_bits = E82545_NVM_DATA_BITS; sc->nvm_data = 0; break; default: DPRINTF("eeprom unknown op: 0x%x\r\r", sc->nvm_opaddr); /* back to opcode mode */ sc->nvm_opaddr = 0; sc->nvm_mode = E82545_NVM_MODE_OPADDR; sc->nvm_bits = E82545_NVM_OPADDR_BITS; } } } else { DPRINTF("eeprom state machine wrong state! " "0x%x 0x%x 0x%x 0x%x\r\n", sc->nvm_mode, sc->nvm_bits, sc->nvm_opaddr, sc->nvm_data); } } static void e82545_itr_callback(int fd, enum ev_type type, void *param) { uint32_t new; struct e82545_softc *sc = param; pthread_mutex_lock(&sc->esc_mtx); new = sc->esc_ICR & sc->esc_IMS; if (new && !sc->esc_irq_asserted) { DPRINTF("itr callback: lintr assert %x\r\n", new); sc->esc_irq_asserted = 1; pci_lintr_assert(sc->esc_pi); } else { mevent_delete(sc->esc_mevpitr); sc->esc_mevpitr = NULL; } pthread_mutex_unlock(&sc->esc_mtx); } static void e82545_icr_assert(struct e82545_softc *sc, uint32_t bits) { uint32_t new; DPRINTF("icr assert: 0x%x\r\n", bits); /* * An interrupt is only generated if bits are set that * aren't already in the ICR, these bits are unmasked, * and there isn't an interrupt already pending. */ new = bits & ~sc->esc_ICR & sc->esc_IMS; sc->esc_ICR |= bits; if (new == 0) { DPRINTF("icr assert: masked %x, ims %x\r\n", new, sc->esc_IMS); } else if (sc->esc_mevpitr != NULL) { DPRINTF("icr assert: throttled %x, ims %x\r\n", new, sc->esc_IMS); } else if (!sc->esc_irq_asserted) { DPRINTF("icr assert: lintr assert %x\r\n", new); sc->esc_irq_asserted = 1; pci_lintr_assert(sc->esc_pi); if (sc->esc_ITR != 0) { sc->esc_mevpitr = mevent_add( (sc->esc_ITR + 3905) / 3906, /* 256ns -> 1ms */ EVF_TIMER, e82545_itr_callback, sc); } } } static void e82545_ims_change(struct e82545_softc *sc, uint32_t bits) { uint32_t new; /* * Changing the mask may allow previously asserted * but masked interrupt requests to generate an interrupt. */ new = bits & sc->esc_ICR & ~sc->esc_IMS; sc->esc_IMS |= bits; if (new == 0) { DPRINTF("ims change: masked %x, ims %x\r\n", new, sc->esc_IMS); } else if (sc->esc_mevpitr != NULL) { DPRINTF("ims change: throttled %x, ims %x\r\n", new, sc->esc_IMS); } else if (!sc->esc_irq_asserted) { DPRINTF("ims change: lintr assert %x\n\r", new); sc->esc_irq_asserted = 1; pci_lintr_assert(sc->esc_pi); if (sc->esc_ITR != 0) { sc->esc_mevpitr = mevent_add( (sc->esc_ITR + 3905) / 3906, /* 256ns -> 1ms */ EVF_TIMER, e82545_itr_callback, sc); } } } static void e82545_icr_deassert(struct e82545_softc *sc, uint32_t bits) { DPRINTF("icr deassert: 0x%x\r\n", bits); sc->esc_ICR &= ~bits; /* * If there are no longer any interrupt sources and there * was an asserted interrupt, clear it */ if (sc->esc_irq_asserted && !(sc->esc_ICR & sc->esc_IMS)) { DPRINTF("icr deassert: lintr deassert %x\r\n", bits); pci_lintr_deassert(sc->esc_pi); sc->esc_irq_asserted = 0; } } static void e82545_intr_write(struct e82545_softc *sc, uint32_t offset, uint32_t value) { DPRINTF("intr_write: off %x, val %x\n\r", offset, value); switch (offset) { case E1000_ICR: e82545_icr_deassert(sc, value); break; case E1000_ITR: sc->esc_ITR = value; break; case E1000_ICS: sc->esc_ICS = value; /* not used: store for debug */ e82545_icr_assert(sc, value); break; case E1000_IMS: e82545_ims_change(sc, value); break; case E1000_IMC: sc->esc_IMC = value; /* for debug */ sc->esc_IMS &= ~value; // XXX clear interrupts if all ICR bits now masked // and interrupt was pending ? break; default: break; } } static uint32_t e82545_intr_read(struct e82545_softc *sc, uint32_t offset) { uint32_t retval; retval = 0; DPRINTF("intr_read: off %x\n\r", offset); switch (offset) { case E1000_ICR: retval = sc->esc_ICR; sc->esc_ICR = 0; e82545_icr_deassert(sc, ~0); break; case E1000_ITR: retval = sc->esc_ITR; break; case E1000_ICS: /* write-only register */ break; case E1000_IMS: retval = sc->esc_IMS; break; case E1000_IMC: /* write-only register */ break; default: break; } return (retval); } static void e82545_devctl(struct e82545_softc *sc, uint32_t val) { sc->esc_CTRL = val & ~E1000_CTRL_RST; if (val & E1000_CTRL_RST) { DPRINTF("e1k: s/w reset, ctl %x\n", val); e82545_reset(sc, 1); } /* XXX check for phy reset ? */ } static void e82545_rx_update_rdba(struct e82545_softc *sc) { /* XXX verify desc base/len within phys mem range */ sc->esc_rdba = (uint64_t)sc->esc_RDBAH << 32 | sc->esc_RDBAL; /* Cache host mapping of guest descriptor array */ sc->esc_rxdesc = paddr_guest2host(sc->esc_ctx, sc->esc_rdba, sc->esc_RDLEN); } static void e82545_rx_ctl(struct e82545_softc *sc, uint32_t val) { int on; on = ((val & E1000_RCTL_EN) == E1000_RCTL_EN); /* Save RCTL after stripping reserved bits 31:27,24,21,14,11:10,0 */ sc->esc_RCTL = val & ~0xF9204c01; DPRINTF("rx_ctl - %s RCTL %x, val %x\n", on ? "on" : "off", sc->esc_RCTL, val); /* state change requested */ if (on != sc->esc_rx_enabled) { if (on) { /* Catch disallowed/unimplemented settings */ //assert(!(val & E1000_RCTL_LBM_TCVR)); if (sc->esc_RCTL & E1000_RCTL_LBM_TCVR) { sc->esc_rx_loopback = 1; } else { sc->esc_rx_loopback = 0; } e82545_rx_update_rdba(sc); e82545_rx_enable(sc); } else { e82545_rx_disable(sc); sc->esc_rx_loopback = 0; sc->esc_rdba = 0; sc->esc_rxdesc = NULL; } } } static void e82545_tx_update_tdba(struct e82545_softc *sc) { /* XXX verify desc base/len within phys mem range */ sc->esc_tdba = (uint64_t)sc->esc_TDBAH << 32 | sc->esc_TDBAL; /* Cache host mapping of guest descriptor array */ sc->esc_txdesc = paddr_guest2host(sc->esc_ctx, sc->esc_tdba, sc->esc_TDLEN); } static void e82545_tx_ctl(struct e82545_softc *sc, uint32_t val) { int on; on = ((val & E1000_TCTL_EN) == E1000_TCTL_EN); /* ignore TCTL_EN settings that don't change state */ if (on == sc->esc_tx_enabled) return; if (on) { e82545_tx_update_tdba(sc); e82545_tx_enable(sc); } else { e82545_tx_disable(sc); sc->esc_tdba = 0; sc->esc_txdesc = NULL; } /* Save TCTL value after stripping reserved bits 31:25,23,2,0 */ sc->esc_TCTL = val & ~0xFE800005; } int e82545_bufsz(uint32_t rctl) { switch (rctl & (E1000_RCTL_BSEX | E1000_RCTL_SZ_256)) { case (E1000_RCTL_SZ_2048): return (2048); case (E1000_RCTL_SZ_1024): return (1024); case (E1000_RCTL_SZ_512): return (512); case (E1000_RCTL_SZ_256): return (256); case (E1000_RCTL_BSEX|E1000_RCTL_SZ_16384): return (16384); case (E1000_RCTL_BSEX|E1000_RCTL_SZ_8192): return (8192); case (E1000_RCTL_BSEX|E1000_RCTL_SZ_4096): return (4096); } return (256); /* Forbidden value. */ } static uint8_t dummybuf[2048]; /* XXX one packet at a time until this is debugged */ static void e82545_tap_callback(int fd, enum ev_type type, void *param) { struct e82545_softc *sc = param; struct e1000_rx_desc *rxd; struct iovec vec[64]; int left, len, lim, maxpktsz, maxpktdesc, bufsz, i, n, size; uint32_t cause = 0; uint16_t *tp, tag, head; pthread_mutex_lock(&sc->esc_mtx); DPRINTF("rx_run: head %x, tail %x\r\n", sc->esc_RDH, sc->esc_RDT); if (!sc->esc_rx_enabled || sc->esc_rx_loopback) { DPRINTF("rx disabled (!%d || %d) -- packet(s) dropped\r\n", sc->esc_rx_enabled, sc->esc_rx_loopback); while (read(sc->esc_tapfd, dummybuf, sizeof(dummybuf)) > 0) { } goto done1; } bufsz = e82545_bufsz(sc->esc_RCTL); maxpktsz = (sc->esc_RCTL & E1000_RCTL_LPE) ? 16384 : 1522; maxpktdesc = (maxpktsz + bufsz - 1) / bufsz; size = sc->esc_RDLEN / 16; head = sc->esc_RDH; left = (size + sc->esc_RDT - head) % size; if (left < maxpktdesc) { DPRINTF("rx overflow (%d < %d) -- packet(s) dropped\r\n", left, maxpktdesc); while (read(sc->esc_tapfd, dummybuf, sizeof(dummybuf)) > 0) { } goto done1; } sc->esc_rx_active = 1; pthread_mutex_unlock(&sc->esc_mtx); for (lim = size / 4; lim > 0 && left >= maxpktdesc; lim -= n) { /* Grab rx descriptor pointed to by the head pointer */ for (i = 0; i < maxpktdesc; i++) { rxd = &sc->esc_rxdesc[(head + i) % size]; vec[i].iov_base = paddr_guest2host(sc->esc_ctx, rxd->buffer_addr, bufsz); vec[i].iov_len = bufsz; } len = readv(sc->esc_tapfd, vec, maxpktdesc); if (len <= 0) { DPRINTF("tap: readv() returned %d\n", len); goto done; } /* * Adjust the packet length based on whether the CRC needs * to be stripped or if the packet is less than the minimum * eth packet size. */ if (len < ETHER_MIN_LEN - ETHER_CRC_LEN) len = ETHER_MIN_LEN - ETHER_CRC_LEN; if (!(sc->esc_RCTL & E1000_RCTL_SECRC)) len += ETHER_CRC_LEN; n = (len + bufsz - 1) / bufsz; DPRINTF("packet read %d bytes, %d segs, head %d\r\n", len, n, head); /* Apply VLAN filter. */ tp = (uint16_t *)vec[0].iov_base + 6; if ((sc->esc_RCTL & E1000_RCTL_VFE) && (ntohs(tp[0]) == sc->esc_VET)) { tag = ntohs(tp[1]) & 0x0fff; if ((sc->esc_fvlan[tag >> 5] & (1 << (tag & 0x1f))) != 0) { DPRINTF("known VLAN %d\r\n", tag); } else { DPRINTF("unknown VLAN %d\r\n", tag); n = 0; continue; } } /* Update all consumed descriptors. */ for (i = 0; i < n - 1; i++) { rxd = &sc->esc_rxdesc[(head + i) % size]; rxd->length = bufsz; rxd->csum = 0; rxd->errors = 0; rxd->special = 0; rxd->status = E1000_RXD_STAT_DD; } rxd = &sc->esc_rxdesc[(head + i) % size]; rxd->length = len % bufsz; rxd->csum = 0; rxd->errors = 0; rxd->special = 0; /* XXX signal no checksum for now */ rxd->status = E1000_RXD_STAT_PIF | E1000_RXD_STAT_IXSM | E1000_RXD_STAT_EOP | E1000_RXD_STAT_DD; /* Schedule receive interrupts. */ if (len <= sc->esc_RSRPD) { cause |= E1000_ICR_SRPD | E1000_ICR_RXT0; } else { /* XXX: RDRT and RADV timers should be here. */ cause |= E1000_ICR_RXT0; } head = (head + n) % size; left -= n; } done: pthread_mutex_lock(&sc->esc_mtx); sc->esc_rx_active = 0; if (sc->esc_rx_enabled == 0) pthread_cond_signal(&sc->esc_rx_cond); sc->esc_RDH = head; /* Respect E1000_RCTL_RDMTS */ left = (size + sc->esc_RDT - head) % size; if (left < (size >> (((sc->esc_RCTL >> 8) & 3) + 1))) cause |= E1000_ICR_RXDMT0; /* Assert all accumulated interrupts. */ if (cause != 0) e82545_icr_assert(sc, cause); done1: DPRINTF("rx_run done: head %x, tail %x\r\n", sc->esc_RDH, sc->esc_RDT); pthread_mutex_unlock(&sc->esc_mtx); } static uint16_t e82545_carry(uint32_t sum) { sum = (sum & 0xFFFF) + (sum >> 16); if (sum > 0xFFFF) sum -= 0xFFFF; return (sum); } static uint16_t e82545_buf_checksum(uint8_t *buf, int len) { int i; uint32_t sum = 0; /* Checksum all the pairs of bytes first... */ for (i = 0; i < (len & ~1U); i += 2) sum += *((u_int16_t *)(buf + i)); /* * If there's a single byte left over, checksum it, too. * Network byte order is big-endian, so the remaining byte is * the high byte. */ if (i < len) sum += htons(buf[i] << 8); return (e82545_carry(sum)); } static uint16_t e82545_iov_checksum(struct iovec *iov, int iovcnt, int off, int len) { int now, odd; uint32_t sum = 0, s; /* Skip completely unneeded vectors. */ while (iovcnt > 0 && iov->iov_len <= off && off > 0) { off -= iov->iov_len; iov++; iovcnt--; } /* Calculate checksum of requested range. */ odd = 0; while (len > 0 && iovcnt > 0) { now = MIN(len, iov->iov_len - off); s = e82545_buf_checksum(iov->iov_base + off, now); sum += odd ? (s << 8) : s; odd ^= (now & 1); len -= now; off = 0; iov++; iovcnt--; } return (e82545_carry(sum)); } /* * Return the transmit descriptor type. */ int e82545_txdesc_type(uint32_t lower) { int type; type = 0; if (lower & E1000_TXD_CMD_DEXT) type = lower & E1000_TXD_MASK; return (type); } static void e82545_transmit_checksum(struct iovec *iov, int iovcnt, struct ck_info *ck) { uint16_t cksum; int cklen; DPRINTF("tx cksum: iovcnt/s/off/len %d/%d/%d/%d\r\n", iovcnt, ck->ck_start, ck->ck_off, ck->ck_len); cklen = ck->ck_len ? ck->ck_len - ck->ck_start + 1 : INT_MAX; cksum = e82545_iov_checksum(iov, iovcnt, ck->ck_start, cklen); *(uint16_t *)((uint8_t *)iov[0].iov_base + ck->ck_off) = ~cksum; } static void e82545_transmit_backend(struct e82545_softc *sc, struct iovec *iov, int iovcnt) { if (sc->esc_tapfd == -1) return; (void) writev(sc->esc_tapfd, iov, iovcnt); } static void e82545_transmit_done(struct e82545_softc *sc, uint16_t head, uint16_t tail, uint16_t dsize, int *tdwb) { union e1000_tx_udesc *dsc; for ( ; head != tail; head = (head + 1) % dsize) { dsc = &sc->esc_txdesc[head]; if (dsc->td.lower.data & E1000_TXD_CMD_RS) { dsc->td.upper.data |= E1000_TXD_STAT_DD; *tdwb = 1; } } } static int e82545_transmit(struct e82545_softc *sc, uint16_t head, uint16_t tail, uint16_t dsize, uint16_t *rhead, int *tdwb) { uint8_t *hdr, *hdrp; struct iovec iovb[I82545_MAX_TXSEGS + 2]; struct iovec tiov[I82545_MAX_TXSEGS + 2]; struct e1000_context_desc *cd; struct ck_info ckinfo[2]; struct iovec *iov; union e1000_tx_udesc *dsc; int desc, dtype, len, ntype, iovcnt, tlen, hdrlen, vlen, tcp, tso; int mss, paylen, seg, tiovcnt, left, now, nleft, nnow, pv, pvoff; uint32_t tcpsum, tcpseq; uint16_t ipcs, tcpcs, ipid, ohead; ckinfo[0].ck_valid = ckinfo[1].ck_valid = 0; iovcnt = 0; tlen = 0; ntype = 0; tso = 0; ohead = head; /* iovb[0/1] may be used for writable copy of headers. */ iov = &iovb[2]; for (desc = 0; ; desc++, head = (head + 1) % dsize) { if (head == tail) { *rhead = head; return (0); } dsc = &sc->esc_txdesc[head]; dtype = e82545_txdesc_type(dsc->td.lower.data); if (desc == 0) { switch (dtype) { case E1000_TXD_TYP_C: DPRINTF("tx ctxt desc idx %d: %016jx " "%08x%08x\r\n", head, dsc->td.buffer_addr, dsc->td.upper.data, dsc->td.lower.data); /* Save context and return */ sc->esc_txctx = dsc->cd; goto done; case E1000_TXD_TYP_L: DPRINTF("tx legacy desc idx %d: %08x%08x\r\n", head, dsc->td.upper.data, dsc->td.lower.data); /* * legacy cksum start valid in first descriptor */ ntype = dtype; ckinfo[0].ck_start = dsc->td.upper.fields.css; break; case E1000_TXD_TYP_D: DPRINTF("tx data desc idx %d: %08x%08x\r\n", head, dsc->td.upper.data, dsc->td.lower.data); ntype = dtype; break; default: break; } } else { /* Descriptor type must be consistent */ assert(dtype == ntype); DPRINTF("tx next desc idx %d: %08x%08x\r\n", head, dsc->td.upper.data, dsc->td.lower.data); } len = (dtype == E1000_TXD_TYP_L) ? dsc->td.lower.flags.length : dsc->dd.lower.data & 0xFFFFF; if (len > 0) { /* Strip checksum supplied by guest. */ if ((dsc->td.lower.data & E1000_TXD_CMD_EOP) != 0 && (dsc->td.lower.data & E1000_TXD_CMD_IFCS) == 0) len -= 2; tlen += len; if (iovcnt < I82545_MAX_TXSEGS) { iov[iovcnt].iov_base = paddr_guest2host( sc->esc_ctx, dsc->td.buffer_addr, len); iov[iovcnt].iov_len = len; } iovcnt++; } /* * Pull out info that is valid in the final descriptor * and exit descriptor loop. */ if (dsc->td.lower.data & E1000_TXD_CMD_EOP) { if (dtype == E1000_TXD_TYP_L) { if (dsc->td.lower.data & E1000_TXD_CMD_IC) { ckinfo[0].ck_valid = 1; ckinfo[0].ck_off = dsc->td.lower.flags.cso; ckinfo[0].ck_len = 0; } } else { cd = &sc->esc_txctx; if (dsc->dd.lower.data & E1000_TXD_CMD_TSE) tso = 1; if (dsc->dd.upper.fields.popts & E1000_TXD_POPTS_IXSM) ckinfo[0].ck_valid = 1; if (dsc->dd.upper.fields.popts & E1000_TXD_POPTS_IXSM || tso) { ckinfo[0].ck_start = cd->lower_setup.ip_fields.ipcss; ckinfo[0].ck_off = cd->lower_setup.ip_fields.ipcso; ckinfo[0].ck_len = cd->lower_setup.ip_fields.ipcse; } if (dsc->dd.upper.fields.popts & E1000_TXD_POPTS_TXSM) ckinfo[1].ck_valid = 1; if (dsc->dd.upper.fields.popts & E1000_TXD_POPTS_TXSM || tso) { ckinfo[1].ck_start = cd->upper_setup.tcp_fields.tucss; ckinfo[1].ck_off = cd->upper_setup.tcp_fields.tucso; ckinfo[1].ck_len = cd->upper_setup.tcp_fields.tucse; } } break; } } if (iovcnt > I82545_MAX_TXSEGS) { WPRINTF("tx too many descriptors (%d > %d) -- dropped\r\n", iovcnt, I82545_MAX_TXSEGS); goto done; } hdrlen = vlen = 0; /* Estimate writable space for VLAN header insertion. */ if ((sc->esc_CTRL & E1000_CTRL_VME) && (dsc->td.lower.data & E1000_TXD_CMD_VLE)) { hdrlen = ETHER_ADDR_LEN*2; vlen = ETHER_VLAN_ENCAP_LEN; } if (!tso) { /* Estimate required writable space for checksums. */ if (ckinfo[0].ck_valid) hdrlen = MAX(hdrlen, ckinfo[0].ck_off + 2); if (ckinfo[1].ck_valid) hdrlen = MAX(hdrlen, ckinfo[1].ck_off + 2); /* Round up writable space to the first vector. */ if (hdrlen != 0 && iov[0].iov_len > hdrlen && iov[0].iov_len < hdrlen + 100) hdrlen = iov[0].iov_len; } else { /* In case of TSO header length provided by software. */ hdrlen = sc->esc_txctx.tcp_seg_setup.fields.hdr_len; } /* Allocate, fill and prepend writable header vector. */ if (hdrlen != 0) { hdr = __builtin_alloca(hdrlen + vlen); hdr += vlen; for (left = hdrlen, hdrp = hdr; left > 0; left -= now, hdrp += now) { now = MIN(left, iov->iov_len); memcpy(hdrp, iov->iov_base, now); iov->iov_base += now; iov->iov_len -= now; if (iov->iov_len == 0) { iov++; iovcnt--; } } iov--; iovcnt++; iov->iov_base = hdr; iov->iov_len = hdrlen; } /* Insert VLAN tag. */ if (vlen != 0) { hdr -= ETHER_VLAN_ENCAP_LEN; memmove(hdr, hdr + ETHER_VLAN_ENCAP_LEN, ETHER_ADDR_LEN*2); hdrlen += ETHER_VLAN_ENCAP_LEN; hdr[ETHER_ADDR_LEN*2 + 0] = sc->esc_VET >> 8; hdr[ETHER_ADDR_LEN*2 + 1] = sc->esc_VET & 0xff; hdr[ETHER_ADDR_LEN*2 + 2] = dsc->td.upper.fields.special >> 8; hdr[ETHER_ADDR_LEN*2 + 3] = dsc->td.upper.fields.special & 0xff; iov->iov_base = hdr; iov->iov_len += ETHER_VLAN_ENCAP_LEN; /* Correct checksum offsets after VLAN tag insertion. */ ckinfo[0].ck_start += ETHER_VLAN_ENCAP_LEN; ckinfo[0].ck_off += ETHER_VLAN_ENCAP_LEN; if (ckinfo[0].ck_len != 0) ckinfo[0].ck_len += ETHER_VLAN_ENCAP_LEN; ckinfo[1].ck_start += ETHER_VLAN_ENCAP_LEN; ckinfo[1].ck_off += ETHER_VLAN_ENCAP_LEN; if (ckinfo[1].ck_len != 0) ckinfo[1].ck_len += ETHER_VLAN_ENCAP_LEN; } /* Simple non-TSO case. */ if (!tso) { /* Calculate checksums and transmit. */ if (ckinfo[0].ck_valid) e82545_transmit_checksum(iov, iovcnt, &ckinfo[0]); if (ckinfo[1].ck_valid) e82545_transmit_checksum(iov, iovcnt, &ckinfo[1]); e82545_transmit_backend(sc, iov, iovcnt); goto done; } /* Doing TSO. */ tcp = (sc->esc_txctx.cmd_and_length & E1000_TXD_CMD_TCP) != 0; mss = sc->esc_txctx.tcp_seg_setup.fields.mss; paylen = (sc->esc_txctx.cmd_and_length & 0x000fffff); DPRINTF("tx %s segmentation offload %d+%d/%d bytes %d iovs\r\n", tcp ? "TCP" : "UDP", hdrlen, paylen, mss, iovcnt); ipid = ntohs(*(uint16_t *)&hdr[ckinfo[0].ck_start + 4]); tcpseq = ntohl(*(uint32_t *)&hdr[ckinfo[1].ck_start + 4]); ipcs = *(uint16_t *)&hdr[ckinfo[0].ck_off]; tcpcs = 0; if (ckinfo[1].ck_valid) /* Save partial pseudo-header checksum. */ tcpcs = *(uint16_t *)&hdr[ckinfo[1].ck_off]; pv = 1; pvoff = 0; for (seg = 0, left = paylen; left > 0; seg++, left -= now) { now = MIN(left, mss); /* Construct IOVs for the segment. */ /* Include whole original header. */ tiov[0].iov_base = hdr; tiov[0].iov_len = hdrlen; tiovcnt = 1; /* Include respective part of payload IOV. */ for (nleft = now; pv < iovcnt && nleft > 0; nleft -= nnow) { nnow = MIN(nleft, iov[pv].iov_len - pvoff); tiov[tiovcnt].iov_base = iov[pv].iov_base + pvoff; tiov[tiovcnt++].iov_len = nnow; if (pvoff + nnow == iov[pv].iov_len) { pv++; pvoff = 0; } else pvoff += nnow; } DPRINTF("tx segment %d %d+%d bytes %d iovs\r\n", seg, hdrlen, now, tiovcnt); /* Update IP header. */ if (sc->esc_txctx.cmd_and_length & E1000_TXD_CMD_IP) { /* IPv4 -- set length and ID */ *(uint16_t *)&hdr[ckinfo[0].ck_start + 2] = htons(hdrlen - ckinfo[0].ck_start + now); *(uint16_t *)&hdr[ckinfo[0].ck_start + 4] = htons(ipid + seg); } else { /* IPv6 -- set length */ *(uint16_t *)&hdr[ckinfo[0].ck_start + 4] = htons(hdrlen - ckinfo[0].ck_start - 40 + now); } /* Update pseudo-header checksum. */ tcpsum = tcpcs; tcpsum += htons(hdrlen - ckinfo[1].ck_start + now); /* Update TCP/UDP headers. */ if (tcp) { /* Update sequence number and FIN/PUSH flags. */ *(uint32_t *)&hdr[ckinfo[1].ck_start + 4] = htonl(tcpseq + paylen - left); if (now < left) { hdr[ckinfo[1].ck_start + 13] &= ~(TH_FIN | TH_PUSH); } } else { /* Update payload length. */ *(uint32_t *)&hdr[ckinfo[1].ck_start + 4] = hdrlen - ckinfo[1].ck_start + now; } /* Calculate checksums and transmit. */ if (ckinfo[0].ck_valid) { *(uint16_t *)&hdr[ckinfo[0].ck_off] = ipcs; e82545_transmit_checksum(tiov, tiovcnt, &ckinfo[0]); } if (ckinfo[1].ck_valid) { *(uint16_t *)&hdr[ckinfo[1].ck_off] = e82545_carry(tcpsum); e82545_transmit_checksum(tiov, tiovcnt, &ckinfo[1]); } e82545_transmit_backend(sc, tiov, tiovcnt); } done: head = (head + 1) % dsize; e82545_transmit_done(sc, ohead, head, dsize, tdwb); *rhead = head; return (desc + 1); } static void e82545_tx_run(struct e82545_softc *sc) { uint32_t cause; uint16_t head, rhead, tail, size; int lim, tdwb, sent; head = sc->esc_TDH; tail = sc->esc_TDT; size = sc->esc_TDLEN / 16; DPRINTF("tx_run: head %x, rhead %x, tail %x\r\n", sc->esc_TDH, sc->esc_TDHr, sc->esc_TDT); pthread_mutex_unlock(&sc->esc_mtx); rhead = head; tdwb = 0; for (lim = size / 4; sc->esc_tx_enabled && lim > 0; lim -= sent) { sent = e82545_transmit(sc, head, tail, size, &rhead, &tdwb); if (sent == 0) break; head = rhead; } pthread_mutex_lock(&sc->esc_mtx); sc->esc_TDH = head; sc->esc_TDHr = rhead; cause = 0; if (tdwb) cause |= E1000_ICR_TXDW; if (lim != size / 4 && sc->esc_TDH == sc->esc_TDT) cause |= E1000_ICR_TXQE; if (cause) e82545_icr_assert(sc, cause); DPRINTF("tx_run done: head %x, rhead %x, tail %x\r\n", sc->esc_TDH, sc->esc_TDHr, sc->esc_TDT); } static void * e82545_tx_thread(void *param) { struct e82545_softc *sc = param; pthread_mutex_lock(&sc->esc_mtx); for (;;) { while (!sc->esc_tx_enabled || sc->esc_TDHr == sc->esc_TDT) { if (sc->esc_tx_enabled && sc->esc_TDHr != sc->esc_TDT) break; sc->esc_tx_active = 0; if (sc->esc_tx_enabled == 0) pthread_cond_signal(&sc->esc_tx_cond); pthread_cond_wait(&sc->esc_tx_cond, &sc->esc_mtx); } sc->esc_tx_active = 1; /* Process some tx descriptors. Lock dropped inside. */ e82545_tx_run(sc); } } static void e82545_tx_start(struct e82545_softc *sc) { if (sc->esc_tx_active == 0) pthread_cond_signal(&sc->esc_tx_cond); } static void e82545_tx_enable(struct e82545_softc *sc) { sc->esc_tx_enabled = 1; } static void e82545_tx_disable(struct e82545_softc *sc) { sc->esc_tx_enabled = 0; while (sc->esc_tx_active) pthread_cond_wait(&sc->esc_tx_cond, &sc->esc_mtx); } static void e82545_rx_enable(struct e82545_softc *sc) { sc->esc_rx_enabled = 1; } static void e82545_rx_disable(struct e82545_softc *sc) { sc->esc_rx_enabled = 0; while (sc->esc_rx_active) pthread_cond_wait(&sc->esc_rx_cond, &sc->esc_mtx); } static void e82545_write_ra(struct e82545_softc *sc, int reg, uint32_t wval) { struct eth_uni *eu; int idx; idx = reg >> 1; assert(idx < 15); eu = &sc->esc_uni[idx]; if (reg & 0x1) { /* RAH */ eu->eu_valid = ((wval & E1000_RAH_AV) == E1000_RAH_AV); eu->eu_addrsel = (wval >> 16) & 0x3; eu->eu_eth.octet[5] = wval >> 8; eu->eu_eth.octet[4] = wval; } else { /* RAL */ eu->eu_eth.octet[3] = wval >> 24; eu->eu_eth.octet[2] = wval >> 16; eu->eu_eth.octet[1] = wval >> 8; eu->eu_eth.octet[0] = wval; } } static uint32_t e82545_read_ra(struct e82545_softc *sc, int reg) { struct eth_uni *eu; uint32_t retval; int idx; idx = reg >> 1; assert(idx < 15); eu = &sc->esc_uni[idx]; if (reg & 0x1) { /* RAH */ retval = (eu->eu_valid << 31) | (eu->eu_addrsel << 16) | (eu->eu_eth.octet[5] << 8) | eu->eu_eth.octet[4]; } else { /* RAL */ retval = (eu->eu_eth.octet[3] << 24) | (eu->eu_eth.octet[2] << 16) | (eu->eu_eth.octet[1] << 8) | eu->eu_eth.octet[0]; } return (retval); } static void e82545_write_register(struct e82545_softc *sc, uint32_t offset, uint32_t value) { int ridx; if (offset & 0x3) { DPRINTF("Unaligned register write offset:0x%x value:0x%x\r\n", offset, value); return; } DPRINTF("Register write: 0x%x value: 0x%x\r\n", offset, value); switch (offset) { case E1000_CTRL: case E1000_CTRL_DUP: e82545_devctl(sc, value); break; case E1000_FCAL: sc->esc_FCAL = value; break; case E1000_FCAH: sc->esc_FCAH = value & ~0xFFFF0000; break; case E1000_FCT: sc->esc_FCT = value & ~0xFFFF0000; break; case E1000_VET: sc->esc_VET = value & ~0xFFFF0000; break; case E1000_FCTTV: sc->esc_FCTTV = value & ~0xFFFF0000; break; case E1000_LEDCTL: sc->esc_LEDCTL = value & ~0x30303000; break; case E1000_PBA: sc->esc_PBA = value & 0x0000FF80; break; case E1000_ICR: case E1000_ITR: case E1000_ICS: case E1000_IMS: case E1000_IMC: e82545_intr_write(sc, offset, value); break; case E1000_RCTL: e82545_rx_ctl(sc, value); break; case E1000_FCRTL: sc->esc_FCRTL = value & ~0xFFFF0007; break; case E1000_FCRTH: sc->esc_FCRTH = value & ~0xFFFF0007; break; case E1000_RDBAL(0): sc->esc_RDBAL = value & ~0xF; if (sc->esc_rx_enabled) { /* Apparently legal: update cached address */ e82545_rx_update_rdba(sc); } break; case E1000_RDBAH(0): assert(!sc->esc_rx_enabled); sc->esc_RDBAH = value; break; case E1000_RDLEN(0): assert(!sc->esc_rx_enabled); sc->esc_RDLEN = value & ~0xFFF0007F; break; case E1000_RDH(0): /* XXX should only ever be zero ? Range check ? */ sc->esc_RDH = value; break; case E1000_RDT(0): /* XXX if this opens up the rx ring, do something ? */ sc->esc_RDT = value; break; case E1000_RDTR: /* ignore FPD bit 31 */ sc->esc_RDTR = value & ~0xFFFF0000; break; case E1000_RXDCTL(0): sc->esc_RXDCTL = value & ~0xFEC0C0C0; break; case E1000_RADV: sc->esc_RADV = value & ~0xFFFF0000; break; case E1000_RSRPD: sc->esc_RSRPD = value & ~0xFFFFF000; break; case E1000_RXCSUM: sc->esc_RXCSUM = value & ~0xFFFFF800; break; case E1000_TXCW: sc->esc_TXCW = value & ~0x3FFF0000; break; case E1000_TCTL: e82545_tx_ctl(sc, value); break; case E1000_TIPG: sc->esc_TIPG = value; break; case E1000_AIT: sc->esc_AIT = value; break; case E1000_TDBAL(0): sc->esc_TDBAL = value & ~0xF; if (sc->esc_tx_enabled) { /* Apparently legal */ e82545_tx_update_tdba(sc); } break; case E1000_TDBAH(0): //assert(!sc->esc_tx_enabled); sc->esc_TDBAH = value; break; case E1000_TDLEN(0): //assert(!sc->esc_tx_enabled); sc->esc_TDLEN = value & ~0xFFF0007F; break; case E1000_TDH(0): //assert(!sc->esc_tx_enabled); /* XXX should only ever be zero ? Range check ? */ sc->esc_TDHr = sc->esc_TDH = value; break; case E1000_TDT(0): /* XXX range check ? */ sc->esc_TDT = value; if (sc->esc_tx_enabled) e82545_tx_start(sc); break; case E1000_TIDV: sc->esc_TIDV = value & ~0xFFFF0000; break; case E1000_TXDCTL(0): //assert(!sc->esc_tx_enabled); sc->esc_TXDCTL = value & ~0xC0C0C0; break; case E1000_TADV: sc->esc_TADV = value & ~0xFFFF0000; break; case E1000_RAL(0) ... E1000_RAH(15): /* convert to u32 offset */ ridx = (offset - E1000_RAL(0)) >> 2; e82545_write_ra(sc, ridx, value); break; case E1000_MTA ... (E1000_MTA + (127*4)): sc->esc_fmcast[(offset - E1000_MTA) >> 2] = value; break; case E1000_VFTA ... (E1000_VFTA + (127*4)): sc->esc_fvlan[(offset - E1000_VFTA) >> 2] = value; break; case E1000_EECD: { //DPRINTF("EECD write 0x%x -> 0x%x\r\n", sc->eeprom_control, value); /* edge triggered low->high */ uint32_t eecd_strobe = ((sc->eeprom_control & E1000_EECD_SK) ? 0 : (value & E1000_EECD_SK)); uint32_t eecd_mask = (E1000_EECD_SK|E1000_EECD_CS| E1000_EECD_DI|E1000_EECD_REQ); sc->eeprom_control &= ~eecd_mask; sc->eeprom_control |= (value & eecd_mask); /* grant/revoke immediately */ if (value & E1000_EECD_REQ) { sc->eeprom_control |= E1000_EECD_GNT; } else { sc->eeprom_control &= ~E1000_EECD_GNT; } if (eecd_strobe && (sc->eeprom_control & E1000_EECD_CS)) { e82545_eecd_strobe(sc); } return; } case E1000_MDIC: { uint8_t reg_addr = (uint8_t)((value & E1000_MDIC_REG_MASK) >> E1000_MDIC_REG_SHIFT); uint8_t phy_addr = (uint8_t)((value & E1000_MDIC_PHY_MASK) >> E1000_MDIC_PHY_SHIFT); sc->mdi_control = (value & ~(E1000_MDIC_ERROR|E1000_MDIC_DEST)); if ((value & E1000_MDIC_READY) != 0) { DPRINTF("Incorrect MDIC ready bit: 0x%x\r\n", value); return; } switch (value & E82545_MDIC_OP_MASK) { case E1000_MDIC_OP_READ: sc->mdi_control &= ~E82545_MDIC_DATA_MASK; sc->mdi_control |= e82545_read_mdi(sc, reg_addr, phy_addr); break; case E1000_MDIC_OP_WRITE: e82545_write_mdi(sc, reg_addr, phy_addr, value & E82545_MDIC_DATA_MASK); break; default: DPRINTF("Unknown MDIC op: 0x%x\r\n", value); return; } /* TODO: barrier? */ sc->mdi_control |= E1000_MDIC_READY; if (value & E82545_MDIC_IE) { // TODO: generate interrupt } return; } case E1000_MANC: case E1000_STATUS: return; default: DPRINTF("Unknown write register: 0x%x value:%x\r\n", offset, value); return; } } static uint32_t e82545_read_register(struct e82545_softc *sc, uint32_t offset) { uint32_t retval; int ridx; if (offset & 0x3) { DPRINTF("Unaligned register read offset:0x%x\r\n", offset); return 0; } DPRINTF("Register read: 0x%x\r\n", offset); switch (offset) { case E1000_CTRL: retval = sc->esc_CTRL; break; case E1000_STATUS: retval = E1000_STATUS_FD | E1000_STATUS_LU | E1000_STATUS_SPEED_1000; break; case E1000_FCAL: retval = sc->esc_FCAL; break; case E1000_FCAH: retval = sc->esc_FCAH; break; case E1000_FCT: retval = sc->esc_FCT; break; case E1000_VET: retval = sc->esc_VET; break; case E1000_FCTTV: retval = sc->esc_FCTTV; break; case E1000_LEDCTL: retval = sc->esc_LEDCTL; break; case E1000_PBA: retval = sc->esc_PBA; break; case E1000_ICR: case E1000_ITR: case E1000_ICS: case E1000_IMS: case E1000_IMC: retval = e82545_intr_read(sc, offset); break; case E1000_RCTL: retval = sc->esc_RCTL; break; case E1000_FCRTL: retval = sc->esc_FCRTL; break; case E1000_FCRTH: retval = sc->esc_FCRTH; break; case E1000_RDBAL(0): retval = sc->esc_RDBAL; break; case E1000_RDBAH(0): retval = sc->esc_RDBAH; break; case E1000_RDLEN(0): retval = sc->esc_RDLEN; break; case E1000_RDH(0): retval = sc->esc_RDH; break; case E1000_RDT(0): retval = sc->esc_RDT; break; case E1000_RDTR: retval = sc->esc_RDTR; break; case E1000_RXDCTL(0): retval = sc->esc_RXDCTL; break; case E1000_RADV: retval = sc->esc_RADV; break; case E1000_RSRPD: retval = sc->esc_RSRPD; break; case E1000_RXCSUM: retval = sc->esc_RXCSUM; break; case E1000_TXCW: retval = sc->esc_TXCW; break; case E1000_TCTL: retval = sc->esc_TCTL; break; case E1000_TIPG: retval = sc->esc_TIPG; break; case E1000_AIT: retval = sc->esc_AIT; break; case E1000_TDBAL(0): retval = sc->esc_TDBAL; break; case E1000_TDBAH(0): retval = sc->esc_TDBAH; break; case E1000_TDLEN(0): retval = sc->esc_TDLEN; break; case E1000_TDH(0): retval = sc->esc_TDH; break; case E1000_TDT(0): retval = sc->esc_TDT; break; case E1000_TIDV: retval = sc->esc_TIDV; break; case E1000_TXDCTL(0): retval = sc->esc_TXDCTL; break; case E1000_TADV: retval = sc->esc_TADV; break; case E1000_RAL(0) ... E1000_RAH(15): /* convert to u32 offset */ ridx = (offset - E1000_RAL(0)) >> 2; retval = e82545_read_ra(sc, ridx); break; case E1000_MTA ... (E1000_MTA + (127*4)): retval = sc->esc_fmcast[(offset - E1000_MTA) >> 2]; break; case E1000_VFTA ... (E1000_VFTA + (127*4)): retval = sc->esc_fvlan[(offset - E1000_VFTA) >> 2]; break; case E1000_EECD: //DPRINTF("EECD read %x\r\n", sc->eeprom_control); retval = sc->eeprom_control; break; case E1000_MDIC: retval = sc->mdi_control; break; case E1000_MANC: retval = 0; break; /* stats that we emulate. */ case E1000_MPC: retval = sc->missed_pkt_count; break; case E1000_PRC64: retval = sc->pkt_rx_by_size[0]; break; case E1000_PRC127: retval = sc->pkt_rx_by_size[1]; break; case E1000_PRC255: retval = sc->pkt_rx_by_size[2]; break; case E1000_PRC511: retval = sc->pkt_rx_by_size[3]; break; case E1000_PRC1023: retval = sc->pkt_rx_by_size[4]; break; case E1000_PRC1522: retval = sc->pkt_rx_by_size[5]; break; case E1000_GPRC: retval = sc->good_pkt_rx_count; break; case E1000_BPRC: retval = sc->bcast_pkt_rx_count; break; case E1000_MPRC: retval = sc->mcast_pkt_rx_count; break; case E1000_GPTC: case E1000_TPT: retval = sc->good_pkt_tx_count; break; case E1000_GORCL: retval = (uint32_t)sc->good_octets_rx; break; case E1000_GORCH: retval = (uint32_t)(sc->good_octets_rx >> 32); break; case E1000_TOTL: case E1000_GOTCL: retval = (uint32_t)sc->good_octets_tx; break; case E1000_TOTH: case E1000_GOTCH: retval = (uint32_t)(sc->good_octets_tx >> 32); break; case E1000_ROC: retval = sc->oversize_rx_count; break; case E1000_TORL: retval = (uint32_t)(sc->good_octets_rx + sc->missed_octets); break; case E1000_TORH: retval = (uint32_t)((sc->good_octets_rx + sc->missed_octets) >> 32); break; case E1000_TPR: retval = sc->good_pkt_rx_count + sc->missed_pkt_count + sc->oversize_rx_count; break; case E1000_PTC64: retval = sc->pkt_tx_by_size[0]; break; case E1000_PTC127: retval = sc->pkt_tx_by_size[1]; break; case E1000_PTC255: retval = sc->pkt_tx_by_size[2]; break; case E1000_PTC511: retval = sc->pkt_tx_by_size[3]; break; case E1000_PTC1023: retval = sc->pkt_tx_by_size[4]; break; case E1000_PTC1522: retval = sc->pkt_tx_by_size[5]; break; case E1000_MPTC: retval = sc->mcast_pkt_tx_count; break; case E1000_BPTC: retval = sc->bcast_pkt_tx_count; break; case E1000_TSCTC: retval = sc->tso_tx_count; break; /* stats that are always 0. */ case E1000_CRCERRS: case E1000_ALGNERRC: case E1000_SYMERRS: case E1000_RXERRC: case E1000_SCC: case E1000_ECOL: case E1000_MCC: case E1000_LATECOL: case E1000_COLC: case E1000_DC: case E1000_TNCRS: case E1000_SEC: case E1000_CEXTERR: case E1000_RLEC: case E1000_XONRXC: case E1000_XONTXC: case E1000_XOFFRXC: case E1000_XOFFTXC: case E1000_FCRUC: case E1000_RNBC: case E1000_RUC: case E1000_RFC: case E1000_RJC: case E1000_MGTPRC: case E1000_MGTPDC: case E1000_MGTPTC: case E1000_TSCTFC: retval = 0; break; default: DPRINTF("Unknown read register: 0x%x\r\n", offset); retval = 0; break; } return (retval); } static void e82545_write(struct vmctx *ctx, int vcpu, struct pci_devinst *pi, int baridx, uint64_t offset, int size, uint64_t value) { struct e82545_softc *sc; //DPRINTF("Write bar:%d offset:0x%lx value:0x%lx size:%d\r\n", baridx, offset, value, size); sc = pi->pi_arg; pthread_mutex_lock(&sc->esc_mtx); switch (baridx) { case E82545_BAR_IO: switch (offset) { case E82545_IOADDR: if (size != 4) { DPRINTF("Wrong io addr write sz:%d value:0x%lx\r\n", size, value); } else sc->io_addr = (uint32_t)value; break; case E82545_IODATA: if (size != 4) { DPRINTF("Wrong io data write size:%d value:0x%lx\r\n", size, value); } else if (sc->io_addr > E82545_IO_REGISTER_MAX) { DPRINTF("Non-register io write addr:0x%x value:0x%lx\r\n", sc->io_addr, value); } else e82545_write_register(sc, sc->io_addr, (uint32_t)value); break; default: DPRINTF("Unknown io bar write offset:0x%lx value:0x%lx size:%d\r\n", offset, value, size); break; } break; case E82545_BAR_REGISTER: if (size != 4) { DPRINTF("Wrong register write size:%d offset:0x%lx value:0x%lx\r\n", size, offset, value); } else e82545_write_register(sc, (uint32_t)offset, (uint32_t)value); break; default: DPRINTF("Unknown write bar:%d off:0x%lx val:0x%lx size:%d\r\n", baridx, offset, value, size); } pthread_mutex_unlock(&sc->esc_mtx); } static uint64_t e82545_read(struct vmctx *ctx, int vcpu, struct pci_devinst *pi, int baridx, uint64_t offset, int size) { struct e82545_softc *sc; uint64_t retval; //DPRINTF("Read bar:%d offset:0x%lx size:%d\r\n", baridx, offset, size); sc = pi->pi_arg; retval = 0; pthread_mutex_lock(&sc->esc_mtx); switch (baridx) { case E82545_BAR_IO: switch (offset) { case E82545_IOADDR: if (size != 4) { DPRINTF("Wrong io addr read sz:%d\r\n", size); } else retval = sc->io_addr; break; case E82545_IODATA: if (size != 4) { DPRINTF("Wrong io data read sz:%d\r\n", size); } if (sc->io_addr > E82545_IO_REGISTER_MAX) { DPRINTF("Non-register io read addr:0x%x\r\n", sc->io_addr); } else retval = e82545_read_register(sc, sc->io_addr); break; default: DPRINTF("Unknown io bar read offset:0x%lx size:%d\r\n", offset, size); break; } break; case E82545_BAR_REGISTER: if (size != 4) { DPRINTF("Wrong register read size:%d offset:0x%lx\r\n", size, offset); } else retval = e82545_read_register(sc, (uint32_t)offset); break; default: DPRINTF("Unknown read bar:%d offset:0x%lx size:%d\r\n", baridx, offset, size); break; } pthread_mutex_unlock(&sc->esc_mtx); return (retval); } static void e82545_reset(struct e82545_softc *sc, int drvr) { int i; e82545_rx_disable(sc); e82545_tx_disable(sc); /* clear outstanding interrupts */ if (sc->esc_irq_asserted) pci_lintr_deassert(sc->esc_pi); /* misc */ if (!drvr) { sc->esc_FCAL = 0; sc->esc_FCAH = 0; sc->esc_FCT = 0; sc->esc_VET = 0; sc->esc_FCTTV = 0; } sc->esc_LEDCTL = 0x07061302; sc->esc_PBA = 0x00100030; /* start nvm in opcode mode. */ sc->nvm_opaddr = 0; sc->nvm_mode = E82545_NVM_MODE_OPADDR; sc->nvm_bits = E82545_NVM_OPADDR_BITS; sc->eeprom_control = E1000_EECD_PRES | E82545_EECD_FWE_EN; e82545_init_eeprom(sc); /* interrupt */ sc->esc_ICR = 0; sc->esc_ITR = 250; sc->esc_ICS = 0; sc->esc_IMS = 0; sc->esc_IMC = 0; /* L2 filters */ if (!drvr) { memset(sc->esc_fvlan, 0, sizeof(sc->esc_fvlan)); memset(sc->esc_fmcast, 0, sizeof(sc->esc_fmcast)); memset(sc->esc_uni, 0, sizeof(sc->esc_uni)); /* XXX not necessary on 82545 ?? */ sc->esc_uni[0].eu_valid = 1; memcpy(sc->esc_uni[0].eu_eth.octet, sc->esc_mac.octet, ETHER_ADDR_LEN); } else { /* Clear RAH valid bits */ for (i = 0; i < 16; i++) sc->esc_uni[i].eu_valid = 0; } /* receive */ if (!drvr) { sc->esc_RDBAL = 0; sc->esc_RDBAH = 0; } sc->esc_RCTL = 0; sc->esc_FCRTL = 0; sc->esc_FCRTH = 0; sc->esc_RDLEN = 0; sc->esc_RDH = 0; sc->esc_RDT = 0; sc->esc_RDTR = 0; sc->esc_RXDCTL = (1 << 24) | (1 << 16); /* default GRAN/WTHRESH */ sc->esc_RADV = 0; sc->esc_RXCSUM = 0; /* transmit */ if (!drvr) { sc->esc_TDBAL = 0; sc->esc_TDBAH = 0; sc->esc_TIPG = 0; sc->esc_AIT = 0; sc->esc_TIDV = 0; sc->esc_TADV = 0; } sc->esc_tdba = 0; sc->esc_txdesc = NULL; sc->esc_TXCW = 0; sc->esc_TCTL = 0; sc->esc_TDLEN = 0; sc->esc_TDT = 0; sc->esc_TDHr = sc->esc_TDH = 0; sc->esc_TXDCTL = 0; } static void e82545_open_tap(struct e82545_softc *sc, char *opts) { char tbuf[80]; if (opts == NULL) { sc->esc_tapfd = -1; return; } strcpy(tbuf, "/dev/"); strlcat(tbuf, opts, sizeof(tbuf)); sc->esc_tapfd = open(tbuf, O_RDWR); if (sc->esc_tapfd == -1) { DPRINTF("unable to open tap device %s\n", opts); exit(1); } /* * Set non-blocking and register for read * notifications with the event loop */ int opt = 1; if (ioctl(sc->esc_tapfd, FIONBIO, &opt) < 0) { WPRINTF("tap device O_NONBLOCK failed: %d\n", errno); close(sc->esc_tapfd); sc->esc_tapfd = -1; } sc->esc_mevp = mevent_add(sc->esc_tapfd, EVF_READ, e82545_tap_callback, sc); if (sc->esc_mevp == NULL) { DPRINTF("Could not register mevent %d\n", EVF_READ); close(sc->esc_tapfd); sc->esc_tapfd = -1; } } static int e82545_parsemac(char *mac_str, uint8_t *mac_addr) { struct ether_addr *ea; char *tmpstr; char zero_addr[ETHER_ADDR_LEN] = { 0, 0, 0, 0, 0, 0 }; tmpstr = strsep(&mac_str,"="); if ((mac_str != NULL) && (!strcmp(tmpstr,"mac"))) { ea = ether_aton(mac_str); if (ea == NULL || ETHER_IS_MULTICAST(ea->octet) || memcmp(ea->octet, zero_addr, ETHER_ADDR_LEN) == 0) { fprintf(stderr, "Invalid MAC %s\n", mac_str); return (1); } else memcpy(mac_addr, ea->octet, ETHER_ADDR_LEN); } return (0); } static int e82545_init(struct vmctx *ctx, struct pci_devinst *pi, char *opts) { DPRINTF("Loading with options: %s\r\n", opts); MD5_CTX mdctx; unsigned char digest[16]; char nstr[80]; struct e82545_softc *sc; char *devname; char *vtopts; int mac_provided; /* Setup our softc */ - sc = calloc(sizeof(*sc), 1); + sc = calloc(1, sizeof(*sc)); pi->pi_arg = sc; sc->esc_pi = pi; sc->esc_ctx = ctx; pthread_mutex_init(&sc->esc_mtx, NULL); pthread_cond_init(&sc->esc_rx_cond, NULL); pthread_cond_init(&sc->esc_tx_cond, NULL); pthread_create(&sc->esc_tx_tid, NULL, e82545_tx_thread, sc); snprintf(nstr, sizeof(nstr), "e82545-%d:%d tx", pi->pi_slot, pi->pi_func); pthread_set_name_np(sc->esc_tx_tid, nstr); pci_set_cfgdata16(pi, PCIR_DEVICE, E82545_DEV_ID_82545EM_COPPER); pci_set_cfgdata16(pi, PCIR_VENDOR, E82545_VENDOR_ID_INTEL); pci_set_cfgdata8(pi, PCIR_CLASS, PCIC_NETWORK); pci_set_cfgdata8(pi, PCIR_SUBCLASS, PCIS_NETWORK_ETHERNET); pci_set_cfgdata16(pi, PCIR_SUBDEV_0, E82545_SUBDEV_ID); pci_set_cfgdata16(pi, PCIR_SUBVEND_0, E82545_VENDOR_ID_INTEL); pci_set_cfgdata8(pi, PCIR_HDRTYPE, PCIM_HDRTYPE_NORMAL); pci_set_cfgdata8(pi, PCIR_INTPIN, 0x1); /* TODO: this card also supports msi, but the freebsd driver for it * does not, so I have not implemented it. */ pci_lintr_request(pi); pci_emul_alloc_bar(pi, E82545_BAR_REGISTER, PCIBAR_MEM32, E82545_BAR_REGISTER_LEN); pci_emul_alloc_bar(pi, E82545_BAR_FLASH, PCIBAR_MEM32, E82545_BAR_FLASH_LEN); pci_emul_alloc_bar(pi, E82545_BAR_IO, PCIBAR_IO, E82545_BAR_IO_LEN); /* * Attempt to open the tap device and read the MAC address * if specified. Copied from virtio-net, slightly modified. */ mac_provided = 0; sc->esc_tapfd = -1; if (opts != NULL) { int err; devname = vtopts = strdup(opts); (void) strsep(&vtopts, ","); if (vtopts != NULL) { err = e82545_parsemac(vtopts, sc->esc_mac.octet); if (err != 0) { free(devname); return (err); } mac_provided = 1; } if (strncmp(devname, "tap", 3) == 0 || strncmp(devname, "vmnet", 5) == 0) e82545_open_tap(sc, devname); free(devname); } /* * The default MAC address is the standard NetApp OUI of 00-a0-98, * followed by an MD5 of the PCI slot/func number and dev name */ if (!mac_provided) { snprintf(nstr, sizeof(nstr), "%d-%d-%s", pi->pi_slot, pi->pi_func, vmname); MD5Init(&mdctx); MD5Update(&mdctx, nstr, strlen(nstr)); MD5Final(digest, &mdctx); sc->esc_mac.octet[0] = 0x00; sc->esc_mac.octet[1] = 0xa0; sc->esc_mac.octet[2] = 0x98; sc->esc_mac.octet[3] = digest[0]; sc->esc_mac.octet[4] = digest[1]; sc->esc_mac.octet[5] = digest[2]; } /* H/w initiated reset */ e82545_reset(sc, 0); return (0); } struct pci_devemu pci_de_e82545 = { .pe_emu = "e1000", .pe_init = e82545_init, .pe_barwrite = e82545_write, .pe_barread = e82545_read }; PCI_EMUL_SET(pci_de_e82545); Index: projects/netbsd-tests-update-12/usr.sbin/extattr/tests/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.sbin/extattr/tests/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/usr.sbin/extattr/tests/Makefile.depend (revision 305172) @@ -0,0 +1,11 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/usr.sbin/extattr/tests/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/usr.sbin/pmcstudy/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.sbin/pmcstudy/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/usr.sbin/pmcstudy/Makefile.depend (revision 305172) @@ -0,0 +1,18 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + include/xlocale \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libcompiler_rt \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/usr.sbin/pmcstudy/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12/usr.sbin/pw/tests/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.sbin/pw/tests/Makefile.depend (revision 305171) +++ projects/netbsd-tests-update-12/usr.sbin/pw/tests/Makefile.depend (revision 305172) @@ -1,11 +1,18 @@ # $FreeBSD$ # Autogenerated - do NOT edit! DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libcompiler_rt \ + lib/libcrypt \ .include .if ${DEP_RELDIR} == ${_DEP_RELDIR} # local dependencies - needed for -jN in clean tree .endif Index: projects/netbsd-tests-update-12/usr.sbin/ypldap/Makefile.depend =================================================================== --- projects/netbsd-tests-update-12/usr.sbin/ypldap/Makefile.depend (nonexistent) +++ projects/netbsd-tests-update-12/usr.sbin/ypldap/Makefile.depend (revision 305172) @@ -0,0 +1,25 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + gnu/lib/csu \ + gnu/lib/libgcc \ + include \ + include/arpa \ + include/rpc \ + include/rpcsvc \ + include/xlocale \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libcompiler_rt \ + lib/libevent \ + lib/libopenbsd \ + lib/librpcsvc \ + lib/libutil \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/netbsd-tests-update-12/usr.sbin/ypldap/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/netbsd-tests-update-12 =================================================================== --- projects/netbsd-tests-update-12 (revision 305171) +++ projects/netbsd-tests-update-12 (revision 305172) Property changes on: projects/netbsd-tests-update-12 ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head:r305042-305170