Page MenuHomeFreeBSD
Files F144289818

D40466.id125528.diff
No OneTemporary
Actions

D40466.id125528.diff
View Options

Index: sys/conf/files
===================================================================
--- sys/conf/files
+++ sys/conf/files
@@ -1911,6 +1911,19 @@
dev/hwpmc/hwpmc_logging.c optional hwpmc
dev/hwpmc/hwpmc_mod.c optional hwpmc
dev/hwpmc/hwpmc_soft.c optional hwpmc
+dev/hwt/hwt.c optional hwt
+dev/hwt/hwt_backend.c optional hwt
+dev/hwt/hwt_config.c optional hwt
+dev/hwt/hwt_context.c optional hwt
+dev/hwt/hwt_contexthash.c optional hwt
+dev/hwt/hwt_cpu.c optional hwt
+dev/hwt/hwt_hook.c optional hwt
+dev/hwt/hwt_ioctl.c optional hwt
+dev/hwt/hwt_owner.c optional hwt
+dev/hwt/hwt_ownerhash.c optional hwt
+dev/hwt/hwt_record.c optional hwt
+dev/hwt/hwt_thread.c optional hwt
+dev/hwt/hwt_vm.c optional hwt
dev/ichiic/ig4_acpi.c optional ig4 acpi iicbus
dev/ichiic/ig4_iic.c optional ig4 iicbus
dev/ichiic/ig4_pci.c optional ig4 pci iicbus
@@ -3289,6 +3302,8 @@
dev/tcp_log/tcp_log_dev.c optional tcp_blackbox inet | tcp_blackbox inet6
dev/tdfx/tdfx_pci.c optional tdfx pci
dev/ti/if_ti.c optional ti pci
+dev/twe/twe.c optional twe
+dev/twe/twe_freebsd.c optional twe
dev/tws/tws.c optional tws
dev/tws/tws_cam.c optional tws
dev/tws/tws_hdm.c optional tws
@@ -3892,6 +3907,7 @@
kern/kern_flag.c standard
kern/kern_fork.c standard
kern/kern_hhook.c standard
+kern/kern_hwt.c standard
kern/kern_idle.c standard
kern/kern_intr.c standard
kern/kern_jail.c standard
Index: sys/conf/options
===================================================================
--- sys/conf/options
+++ sys/conf/options
@@ -906,6 +906,9 @@
HWPMC_HOOKS
HWPMC_MIPS_BACKTRACE opt_hwpmc_hooks.h
+# Hardware Trace (HWT) framework options
+HWT_HOOKS
+
# 802.11 support layer
IEEE80211_DEBUG opt_wlan.h
IEEE80211_DEBUG_REFCNT opt_wlan.h
Index: sys/dev/hwt/hwt.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt.c
@@ -0,0 +1,229 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/*
+ * Hardware Tracing framework.
+ *
+ * The framework manages hardware tracing units that collect information
+ * about software execution and store it as events in highly compressed format
+ * into DRAM. The events cover information about control flow changes of a
+ * program, whether branches taken or not, exceptions taken, timing information,
+ * cycles elapsed and more. That allows us to restore entire program flow of a
+ * given application without performance impact.
+ *
+ * Design overview.
+ *
+ * The framework provides character devices for mmap(2) and ioctl(2) system
+ * calls to allow user to manage CPU (hardware) tracing units.
+ *
+ * /dev/hwt:
+ * .ioctl:
+ * hwt_ioctl():
+ * a) HWT_IOC_ALLOC
+ * Allocates kernel tracing context ctx for a given pid,
+ * and for required mode of operation (CPU or Thread mode),
+ * returns unique ident.
+ * Creates a new character device for ctx management.
+ *
+ * /dev/hwt_%d[_%d], ident[, thread_id]
+ * .mmap
+ * Maps tracing buffers of the corresponding thread to userspace.
+ * .ioctl
+ * hwt_thread_ioctl():
+ * a) HWT_IOC_START
+ * Enables tracing unit for a given context.
+ * b) HWT_IOC_RECORD_GET
+ * Transfers (small) record entries collected during program
+ * execution for a given context to userspace, such as mmaping
+ * tables of executable and dynamic libraries, interpreter,
+ * tid of threads created, etc.
+ * c) HWT_IOC_SET_CONFIG
+ * This allows to set backend-specific configuration of the
+ * trace unit.
+ * d) HWT_IOC_WAKEUP
+ * This wakes up a thread that is currently sleeping.
+ * e) HWT_IOC_BUFPTR_GET
+ * Transfers current hardware pointer in the filling buffer
+ * to userspace.
+ *
+ * HWT context lifecycle:
+ * 1. User invokes HWT_IOC_ALLOC ioctl with information about pid to trace and
+ * size of the buffers for the trace data to allocate.
+ * Some architectures may have different tracing units supported, so user
+ * also provides backend name to use for this context, e.g. "coresight".
+ * 2. Kernel allocates context, lookups the proc for the given pid. Creates
+ * first hwt_thread in the context and allocates trace buffers for it.
+ * Immediately, kernel initializes tracing backend.
+ * Kernel creates character device and returns unique identificator of
+ * trace context.
+ * 3. User opens new character device to operate with the new context.
+ * User invokes HWT_IOC_START ioctl, kernel marks context as RUNNING.
+ * At this point any HWT hook invocation by scheduler enables/disables
+ * tracing for threads associated with the context (threads of the proc).
+ * Any new threads creation (of the target proc) procedures will be invoking
+ * corresponding hooks in HWT framework, so that new hwt_thread and buffers
+ * allocated, character device for mmap(2) created on the fly.
+ * 4. User issues HWT_IOC_RECORD_GET ioctl to fetch information about mmaping
+ * tables and threads created during application startup.
+ * 5. User mmaps tracing buffers of each thread to userspace (using
+ * /dev/hwt_%d_%d % (ident, thread_id) character devices).
+ * 6. User can repeat 4 if expected thread is not yet created during target
+ * application execution.
+ * 7. User issues HWT_IOC_BUFPTR_GET ioctl to get current filling level of the
+ * hardware buffer of a given thread.
+ * 8. User invokes trace decoder library to process available data and see the
+ * results in human readable form.
+ * 9. User repeates 7 if needed.
+ */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/rwlock.h>
+
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_owner.h>
+#include <dev/hwt/hwt_ownerhash.h>
+#include <dev/hwt/hwt_backend.h>
+#include <dev/hwt/hwt_ioctl.h>
+#include <dev/hwt/hwt_hook.h>
+
+#define HWT_DEBUG
+#undef HWT_DEBUG
+
+#ifdef HWT_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static eventhandler_tag hwt_exit_tag;
+static struct cdev *hwt_cdev;
+static struct cdevsw hwt_cdevsw = {
+ .d_version = D_VERSION,
+ .d_name = "hwt",
+ .d_mmap_single = NULL,
+ .d_ioctl = hwt_ioctl
+};
+
+static void
+hwt_process_exit(void *arg __unused, struct proc *p)
+{
+ struct hwt_owner *ho;
+
+ /* Stop HWTs associated with exiting owner, if any. */
+ ho = hwt_ownerhash_lookup(p);
+ if (ho)
+ hwt_owner_shutdown(ho);
+}
+
+static int
+hwt_load(void)
+{
+ struct make_dev_args args;
+ int error;
+
+ make_dev_args_init(&args);
+ args.mda_devsw = &hwt_cdevsw;
+ args.mda_flags = MAKEDEV_CHECKNAME | MAKEDEV_WAITOK;
+ args.mda_uid = UID_ROOT;
+ args.mda_gid = GID_WHEEL;
+ args.mda_mode = 0660;
+ args.mda_si_drv1 = NULL;
+
+ hwt_backend_load();
+ hwt_ctx_load();
+ hwt_contexthash_load();
+ hwt_ownerhash_load();
+
+ error = make_dev_s(&args, &hwt_cdev, "hwt");
+ if (error != 0)
+ return (error);
+
+ hwt_exit_tag = EVENTHANDLER_REGISTER(process_exit, hwt_process_exit,
+ NULL, EVENTHANDLER_PRI_ANY);
+
+ hwt_hook_load();
+
+ return (0);
+}
+
+static int
+hwt_unload(void)
+{
+
+ hwt_hook_unload();
+ EVENTHANDLER_DEREGISTER(process_exit, hwt_exit_tag);
+ destroy_dev(hwt_cdev);
+ hwt_ownerhash_unload();
+ hwt_contexthash_unload();
+ hwt_ctx_unload();
+ hwt_backend_unload();
+
+ return (0);
+}
+
+static int
+hwt_modevent(module_t mod, int type, void *data)
+{
+ int error;
+
+ switch (type) {
+ case MOD_LOAD:
+ error = hwt_load();
+ break;
+ case MOD_UNLOAD:
+ error = hwt_unload();
+ break;
+ default:
+ error = 0;
+ break;
+ }
+
+ return (error);
+}
+
+static moduledata_t hwt_mod = {
+ "hwt",
+ hwt_modevent,
+ NULL
+};
+
+DECLARE_MODULE(hwt, hwt_mod, SI_SUB_DRIVERS, SI_ORDER_FIRST);
+MODULE_VERSION(hwt, 1);
Index: sys/dev/hwt/hwt_backend.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_backend.h
@@ -0,0 +1,72 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_BACKEND_H_
+#define _DEV_HWT_HWT_BACKEND_H_
+
+struct hwt_backend_ops {
+ int (*hwt_backend_init)(struct hwt_context *);
+ void (*hwt_backend_deinit)(void);
+ int (*hwt_backend_configure)(struct hwt_context *, int cpu_id,
+ int thread_id);
+ void (*hwt_backend_enable)(int cpu_id);
+ void (*hwt_backend_disable)(int cpu_id);
+ int (*hwt_backend_read)(int cpu_id, int *curpage,
+ vm_offset_t *curpage_offset);
+
+ /* Debugging only. */
+ void (*hwt_backend_dump)(int cpu_id);
+};
+
+struct hwt_backend {
+ const char *name;
+ struct hwt_backend_ops *ops;
+};
+
+int hwt_backend_init(struct hwt_context *ctx);
+void hwt_backend_deinit(struct hwt_context *ctx);
+int hwt_backend_configure(struct hwt_context *ctx, int cpu_id, int thread_id);
+void hwt_backend_enable(struct hwt_context *ctx, int cpu_id);
+void hwt_backend_disable(struct hwt_context *ctx, int cpu_id);
+void hwt_backend_dump(struct hwt_context *ctx, int cpu_id);
+int hwt_backend_read(struct hwt_context *ctx, int cpu_id, int *curpage,
+ vm_offset_t *curpage_offset);
+int hwt_backend_register(struct hwt_backend *);
+int hwt_backend_unregister(struct hwt_backend *);
+struct hwt_backend * hwt_backend_lookup(const char *name);
+
+void hwt_backend_load(void);
+void hwt_backend_unload(void);
+
+#define HWT_BACKEND_LOCK() mtx_lock_spin(&hwt_backend_mtx)
+#define HWT_BACKEND_UNLOCK() mtx_unlock_spin(&hwt_backend_mtx)
+
+#endif /* !_DEV_HWT_HWT_BACKEND_H_ */
+
Index: sys/dev/hwt/hwt_backend.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_backend.c
@@ -0,0 +1,230 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/* Hardware Trace (HWT) framework. */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_backend.h>
+
+#define HWT_BACKEND_DEBUG
+#undef HWT_BACKEND_DEBUG
+
+#ifdef HWT_BACKEND_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static struct mtx hwt_backend_mtx;
+
+struct hwt_backend_entry {
+ struct hwt_backend *backend;
+ LIST_ENTRY(hwt_backend_entry) next;
+};
+
+static LIST_HEAD(, hwt_backend_entry) hwt_backends;
+
+static MALLOC_DEFINE(M_HWT_BACKEND, "hwt_backend", "HWT backend");
+
+int
+hwt_backend_init(struct hwt_context *ctx)
+{
+ int error;
+
+ dprintf("%s\n", __func__);
+
+ error = ctx->hwt_backend->ops->hwt_backend_init(ctx);
+
+ return (error);
+}
+
+void
+hwt_backend_deinit(struct hwt_context *ctx)
+{
+
+ dprintf("%s\n", __func__);
+
+ ctx->hwt_backend->ops->hwt_backend_deinit();
+}
+
+int
+hwt_backend_configure(struct hwt_context *ctx, int cpu_id, int thread_id)
+{
+ int error;
+
+ dprintf("%s\n", __func__);
+
+ error = ctx->hwt_backend->ops->hwt_backend_configure(ctx, cpu_id,
+ thread_id);
+
+ return (error);
+}
+
+void
+hwt_backend_enable(struct hwt_context *ctx, int cpu_id)
+{
+
+ dprintf("%s\n", __func__);
+
+ ctx->hwt_backend->ops->hwt_backend_enable(cpu_id);
+}
+
+void
+hwt_backend_disable(struct hwt_context *ctx, int cpu_id)
+{
+
+ dprintf("%s\n", __func__);
+
+ ctx->hwt_backend->ops->hwt_backend_disable(cpu_id);
+}
+
+void __unused
+hwt_backend_dump(struct hwt_context *ctx, int cpu_id)
+{
+
+ dprintf("%s\n", __func__);
+
+ ctx->hwt_backend->ops->hwt_backend_dump(cpu_id);
+}
+
+int
+hwt_backend_read(struct hwt_context *ctx, int cpu_id, int *curpage,
+ vm_offset_t *curpage_offset)
+{
+ int error;
+
+ dprintf("%s\n", __func__);
+
+ error = ctx->hwt_backend->ops->hwt_backend_read(cpu_id, curpage,
+ curpage_offset);
+
+ return (error);
+}
+
+struct hwt_backend *
+hwt_backend_lookup(const char *name)
+{
+ struct hwt_backend_entry *entry;
+ struct hwt_backend *backend;
+
+ HWT_BACKEND_LOCK();
+ LIST_FOREACH(entry, &hwt_backends, next) {
+ backend = entry->backend;
+ if (strcmp(backend->name, name) == 0) {
+ HWT_BACKEND_UNLOCK();
+ return (backend);
+ }
+ }
+ HWT_BACKEND_UNLOCK();
+
+ return (NULL);
+}
+
+int
+hwt_backend_register(struct hwt_backend *backend)
+{
+ struct hwt_backend_entry *entry;
+
+ if (backend == NULL ||
+ backend->name == NULL ||
+ backend->ops == NULL)
+ return (EINVAL);
+
+#if 0
+ mtx_init(&entry->mtx, "BKND", NULL, MTX_DEF);
+#endif
+
+ entry = malloc(sizeof(struct hwt_backend_entry), M_HWT_BACKEND,
+ M_WAITOK | M_ZERO);
+ entry->backend = backend;
+
+ HWT_BACKEND_LOCK();
+ LIST_INSERT_HEAD(&hwt_backends, entry, next);
+ HWT_BACKEND_UNLOCK();
+
+ return (0);
+}
+
+int
+hwt_backend_unregister(struct hwt_backend *backend)
+{
+ struct hwt_backend_entry *entry, *tmp;
+
+ if (backend == NULL)
+ return (EINVAL);
+
+ /* TODO: check if not in use */
+
+ HWT_BACKEND_LOCK();
+ LIST_FOREACH_SAFE(entry, &hwt_backends, next, tmp) {
+ if (entry->backend == backend) {
+ LIST_REMOVE(entry, next);
+ HWT_BACKEND_UNLOCK();
+ free(entry, M_HWT_BACKEND);
+ return (0);
+ }
+ }
+ HWT_BACKEND_UNLOCK();
+
+ return (ENOENT);
+}
+
+void
+hwt_backend_load(void)
+{
+
+ mtx_init(&hwt_backend_mtx, "hwt backend", NULL, MTX_SPIN);
+ LIST_INIT(&hwt_backends);
+}
+
+void
+hwt_backend_unload(void)
+{
+
+ /* TODO: ensure all unregistered */
+
+ mtx_destroy(&hwt_backend_mtx);
+}
Index: sys/dev/hwt/hwt_config.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_config.h
@@ -0,0 +1,38 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_CONFIG_H_
+#define _DEV_HWT_HWT_CONFIG_H_
+
+int hwt_config_set(struct thread *td, struct hwt_context *ctx,
+ struct hwt_set_config *sconf);
+void hwt_config_free(struct hwt_context *ctx);
+
+#endif /* !_DEV_HWT_HWT_CONFIG_H_ */
Index: sys/dev/hwt/hwt_config.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_config.c
@@ -0,0 +1,107 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mutex.h>
+#include <sys/hwt.h>
+
+#include <vm/vm.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_record.h>
+
+#define HWT_MAXCONFIGSIZE PAGE_SIZE
+
+#define HWT_CONFIG_DEBUG
+#undef HWT_CONFIG_DEBUG
+
+#ifdef HWT_CONFIG_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static MALLOC_DEFINE(M_HWT_CONFIG, "hwt_config", "HWT config");
+
+int
+hwt_config_set(struct thread *td, struct hwt_context *ctx,
+ struct hwt_set_config *sconf)
+{
+ size_t config_size;
+ void *old_config;
+ void *config;
+ int error;
+
+ config_size = sconf->config_size;
+ if (config_size == 0)
+ return (0);
+
+ if (config_size > HWT_MAXCONFIGSIZE)
+ return (EFBIG);
+
+ config = malloc(config_size, M_HWT_CONFIG, M_WAITOK | M_ZERO);
+
+ error = copyin(sconf->config, config, config_size);
+ if (error) {
+ free(config, M_HWT_CONFIG);
+ return (error);
+ }
+
+ HWT_CTX_LOCK(ctx);
+ old_config = ctx->config;
+ ctx->config = config;
+ ctx->config_size = sconf->config_size;
+ ctx->config_version = sconf->config_version;
+ HWT_CTX_UNLOCK(ctx);
+
+ if (old_config != NULL)
+ free(old_config, M_HWT_CONFIG);
+
+ return (error);
+}
+
+void
+hwt_config_free(struct hwt_context *ctx)
+{
+
+ if (ctx->config == NULL)
+ return;
+
+ free(ctx->config, M_HWT_CONFIG);
+
+ ctx->config = NULL;
+}
Index: sys/dev/hwt/hwt_context.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_context.h
@@ -0,0 +1,77 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_CONTEXT_H_
+#define _DEV_HWT_HWT_CONTEXT_H_
+
+struct hwt_context {
+ LIST_HEAD(, hwt_record_entry) records;
+
+ LIST_ENTRY(hwt_context) next_hch; /* Entry in contexthash. */
+ LIST_ENTRY(hwt_context) next_hwts; /* Entry in ho->hwts. */
+
+ int mode;
+ int ident;
+
+ /* CPU mode. */
+ cpuset_t cpu_map;
+ TAILQ_HEAD(, hwt_cpu) cpus;
+
+ /* Thread mode. */
+ struct proc *proc; /* Target proc. */
+ pid_t pid; /* Target pid. */
+ TAILQ_HEAD(, hwt_thread) threads;
+ int thread_counter;
+ int pause_on_mmap;
+
+ size_t bufsize; /* Trace bufsize for each vm.*/
+ void *config;
+ size_t config_size;
+ int config_version;
+
+ struct hwt_owner *hwt_owner;
+ struct hwt_backend *hwt_backend;
+
+ struct mtx mtx;
+ int state;
+#define CTX_STATE_RUNNING (1 << 0)
+};
+
+#define HWT_CTX_LOCK(ctx) mtx_lock_spin(&(ctx)->mtx)
+#define HWT_CTX_UNLOCK(ctx) mtx_unlock_spin(&(ctx)->mtx)
+#define HWT_CTX_ASSERT_LOCKED(ctx) mtx_assert(&(ctx)->mtx, MA_OWNED)
+
+struct hwt_context * hwt_ctx_alloc(void);
+void hwt_ctx_free(struct hwt_context *ctx);
+
+void hwt_ctx_load(void);
+void hwt_ctx_unload(void);
+
+#endif /* !_DEV_HWT_HWT_CONTEXT_H_ */
Index: sys/dev/hwt/hwt_context.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_context.c
@@ -0,0 +1,140 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/bitstring.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_owner.h>
+
+#define HWT_DEBUG
+#undef HWT_DEBUG
+
+#ifdef HWT_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static MALLOC_DEFINE(M_HWT_CTX, "hwt_ctx", "Hardware Trace");
+
+static bitstr_t *ident_set;
+static int ident_set_size;
+static struct mtx ident_set_mutex;
+
+static int
+hwt_ctx_ident_alloc(int *new_ident)
+{
+
+ mtx_lock_spin(&ident_set_mutex);
+ bit_ffc(ident_set, ident_set_size, new_ident);
+ if (*new_ident == -1) {
+ mtx_unlock_spin(&ident_set_mutex);
+ return (ENOMEM);
+ }
+ bit_set(ident_set, *new_ident);
+ mtx_unlock_spin(&ident_set_mutex);
+
+ return (0);
+}
+
+static void
+hwt_ctx_ident_free(int ident)
+{
+
+ mtx_lock_spin(&ident_set_mutex);
+ bit_clear(ident_set, ident);
+ mtx_unlock_spin(&ident_set_mutex);
+}
+
+struct hwt_context *
+hwt_ctx_alloc(void)
+{
+ struct hwt_context *ctx;
+ int error;
+
+ ctx = malloc(sizeof(struct hwt_context), M_HWT_CTX, M_WAITOK | M_ZERO);
+ ctx->thread_counter = 0;
+
+ LIST_INIT(&ctx->records);
+ TAILQ_INIT(&ctx->threads);
+ TAILQ_INIT(&ctx->cpus);
+ mtx_init(&ctx->mtx, "ctx", NULL, MTX_SPIN);
+
+ hwt_ctx_ident_alloc(&ctx->ident);
+
+ error = hwt_ctx_ident_alloc(&ctx->ident);
+ if (error) {
+ printf("could not allocate ident bit str\n");
+ return (NULL);
+ }
+
+ return (ctx);
+}
+
+void
+hwt_ctx_free(struct hwt_context *ctx)
+{
+
+ hwt_config_free(ctx);
+ hwt_ctx_ident_free(ctx->ident);
+ free(ctx, M_HWT_CTX);
+}
+
+void
+hwt_ctx_load(void)
+{
+
+ ident_set_size = (1 << 8);
+ ident_set = bit_alloc(ident_set_size, M_HWT_CTX, M_WAITOK);
+ mtx_init(&ident_set_mutex, "ident set", NULL, MTX_SPIN);
+}
+
+void
+hwt_ctx_unload(void)
+{
+
+ mtx_destroy(&ident_set_mutex);
+ free(ident_set, M_HWT_CTX);
+}
Index: sys/dev/hwt/hwt_contexthash.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_contexthash.h
@@ -0,0 +1,44 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_CONTEXTHASH_H_
+#define _DEV_HWT_HWT_CONTEXTHASH_H_
+
+struct hwt_context * hwt_contexthash_lookup(struct proc *p);
+void hwt_contexthash_insert(struct hwt_context *ctx);
+void hwt_contexthash_remove(struct hwt_context *ctx);
+
+void hwt_contexthash_load(void);
+void hwt_contexthash_unload(void);
+
+#define HWT_CTXHASH_LOCK() mtx_lock_spin(&hwt_contexthash_mtx)
+#define HWT_CTXHASH_UNLOCK() mtx_unlock_spin(&hwt_contexthash_mtx)
+
+#endif /* !_DEV_HWT_HWT_CONTEXTHASH_H_ */
Index: sys/dev/hwt/hwt_contexthash.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_contexthash.c
@@ -0,0 +1,139 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+
+#define HWT_DEBUG
+#undef HWT_DEBUG
+
+#ifdef HWT_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+#define HWT_CONTEXTHASH_SIZE 1024
+
+static MALLOC_DEFINE(M_HWT_CONTEXTHASH, "hwt_chash", "Hardware Trace");
+
+/*
+ * Hash function. Discard the lower 2 bits of the pointer since
+ * these are always zero for our uses. The hash multiplier is
+ * round((2^LONG_BIT) * ((sqrt(5)-1)/2)).
+ */
+
+#define _HWT_HM 11400714819323198486u /* hash multiplier */
+#define HWT_HASH_PTR(P, M) ((((unsigned long) (P) >> 2) * _HWT_HM) & (M))
+
+static struct mtx hwt_contexthash_mtx;
+static u_long hwt_contexthashmask;
+static LIST_HEAD(hwt_contexthash, hwt_context) *hwt_contexthash;
+
+/*
+ * To use by hwt_switch_in/out() and hwt_record() only.
+ * This function returns mtx locked.
+ */
+struct hwt_context *
+hwt_contexthash_lookup(struct proc *p)
+{
+ struct hwt_contexthash *hch;
+ struct hwt_context *ctx;
+ int hindex;
+
+ hindex = HWT_HASH_PTR(p, hwt_contexthashmask);
+ hch = &hwt_contexthash[hindex];
+
+ HWT_CTXHASH_LOCK();
+ LIST_FOREACH(ctx, hch, next_hch) {
+ if (ctx->proc == p) {
+ HWT_CTX_LOCK(ctx);
+ HWT_CTXHASH_UNLOCK();
+ return (ctx);
+ }
+ }
+ HWT_CTXHASH_UNLOCK();
+
+ return (NULL);
+}
+
+void
+hwt_contexthash_insert(struct hwt_context *ctx)
+{
+ struct hwt_contexthash *hch;
+ int hindex;
+
+ hindex = HWT_HASH_PTR(ctx->proc, hwt_contexthashmask);
+ hch = &hwt_contexthash[hindex];
+
+ HWT_CTXHASH_LOCK();
+ LIST_INSERT_HEAD(hch, ctx, next_hch);
+ HWT_CTXHASH_UNLOCK();
+}
+
+void
+hwt_contexthash_remove(struct hwt_context *ctx)
+{
+
+ HWT_CTXHASH_LOCK();
+ LIST_REMOVE(ctx, next_hch);
+ HWT_CTXHASH_UNLOCK();
+}
+
+void
+hwt_contexthash_load(void)
+{
+
+ hwt_contexthash = hashinit(HWT_CONTEXTHASH_SIZE, M_HWT_CONTEXTHASH,
+ &hwt_contexthashmask);
+ mtx_init(&hwt_contexthash_mtx, "hwt ctx hash", "hwt ctx", MTX_SPIN);
+}
+
+void
+hwt_contexthash_unload(void)
+{
+
+ mtx_destroy(&hwt_contexthash_mtx);
+ hashdestroy(hwt_contexthash, M_HWT_CONTEXTHASH, hwt_contexthashmask);
+}
Index: sys/dev/hwt/hwt_cpu.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_cpu.h
@@ -0,0 +1,46 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_CPU_H_
+#define _DEV_HWT_HWT_CPU_H_
+
+struct hwt_cpu {
+ int cpu_id;
+ struct hwt_vm *vm;
+ TAILQ_ENTRY(hwt_cpu) next;
+};
+
+struct hwt_cpu * hwt_cpu_alloc(void);
+void hwt_cpu_free(struct hwt_cpu *cpu);
+
+struct hwt_cpu * hwt_cpu_first(struct hwt_context *ctx);
+void hwt_cpu_insert(struct hwt_context *ctx, struct hwt_cpu *cpu);
+
+#endif /* !_DEV_HWT_HWT_CPU_H_ */
Index: sys/dev/hwt/hwt_cpu.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_cpu.c
@@ -0,0 +1,98 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/lock.h>
+#include <sys/malloc.h>
+#include <sys/mutex.h>
+#include <sys/hwt.h>
+
+#include <vm/vm.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_record.h>
+#include <dev/hwt/hwt_cpu.h>
+
+#define HWT_CPU_DEBUG
+#undef HWT_CPU_DEBUG
+
+#ifdef HWT_CPU_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static MALLOC_DEFINE(M_HWT_CPU, "hwt_cpu", "HWT cpu");
+
+struct hwt_cpu *
+hwt_cpu_alloc(void)
+{
+ struct hwt_cpu *cpu;
+
+ cpu = malloc(sizeof(struct hwt_cpu), M_HWT_CPU, M_WAITOK | M_ZERO);
+
+ return (cpu);
+}
+
+void
+hwt_cpu_free(struct hwt_cpu *cpu)
+{
+
+ free(cpu, M_HWT_CPU);
+}
+
+struct hwt_cpu *
+hwt_cpu_first(struct hwt_context *ctx)
+{
+ struct hwt_cpu *cpu;
+
+ HWT_CTX_ASSERT_LOCKED(ctx);
+
+ cpu = TAILQ_FIRST(&ctx->cpus);
+
+ KASSERT(cpu != NULL, ("cpu is NULL"));
+
+ return (cpu);
+}
+
+void
+hwt_cpu_insert(struct hwt_context *ctx, struct hwt_cpu *cpu)
+{
+
+ HWT_CTX_ASSERT_LOCKED(ctx);
+
+ TAILQ_INSERT_TAIL(&ctx->cpus, cpu, next);
+}
Index: sys/dev/hwt/hwt_hook.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_hook.h
@@ -0,0 +1,56 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#include <sys/hwt_record.h>
+
+#ifndef _DEV_HWT_HWT_HOOK_H_
+#define _DEV_HWT_HWT_HOOK_H_
+
+#define HWT_SWITCH_IN 0
+#define HWT_SWITCH_OUT 1
+#define HWT_THREAD_EXIT 2
+#define HWT_THREAD_CREATE 3
+#define HWT_THREAD_SET_NAME 4
+#define HWT_RECORD 5
+#define HWT_MMAP 6
+#define HWT_EXEC 7
+
+#define HWT_CALL_HOOK(td, func, arg) \
+do { \
+ if (hwt_hook != NULL) \
+ (hwt_hook)((td), (func), (arg)); \
+} while (0)
+
+extern void (*hwt_hook)(struct thread *td, int func, void *arg);
+
+void hwt_hook_load(void);
+void hwt_hook_unload(void);
+
+#endif /* !_DEV_HWT_HWT_HOOK_H_ */
Index: sys/dev/hwt/hwt_hook.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_hook.c
@@ -0,0 +1,259 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/* Hardware Trace (HWT) framework. */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/refcount.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_owner.h>
+#include <dev/hwt/hwt_backend.h>
+#include <dev/hwt/hwt_record.h>
+
+#define HWT_DEBUG
+#undef HWT_DEBUG
+
+#ifdef HWT_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static void
+hwt_switch_in(struct thread *td)
+{
+ struct hwt_context *ctx;
+ struct hwt_thread *thr;
+ struct proc *p;
+ int cpu_id;
+
+ p = td->td_proc;
+
+ cpu_id = PCPU_GET(cpuid);
+
+ ctx = hwt_contexthash_lookup(p);
+ if (ctx == NULL)
+ return;
+
+ if (ctx->state != CTX_STATE_RUNNING) {
+ HWT_CTX_UNLOCK(ctx);
+ return;
+ }
+
+ thr = hwt_thread_lookup(ctx, td);
+ if (thr == NULL) {
+ HWT_CTX_UNLOCK(ctx);
+ return;
+ }
+
+ dprintf("%s: thr %p index %d tid %d on cpu_id %d\n", __func__, thr,
+ thr->thread_id, td->td_tid, cpu_id);
+
+ hwt_backend_configure(ctx, cpu_id, thr->thread_id);
+ hwt_backend_enable(ctx, cpu_id);
+
+ HWT_THR_UNLOCK(thr);
+}
+
+static void
+hwt_switch_out(struct thread *td)
+{
+ struct hwt_context *ctx;
+ struct hwt_thread *thr;
+ struct proc *p;
+ int cpu_id;
+
+ p = td->td_proc;
+
+ cpu_id = PCPU_GET(cpuid);
+
+ ctx = hwt_contexthash_lookup(p);
+ if (ctx == NULL)
+ return;
+
+ if (ctx->state != CTX_STATE_RUNNING) {
+ HWT_CTX_UNLOCK(ctx);
+ return;
+ }
+ thr = hwt_thread_lookup(ctx, td);
+ if (thr == NULL) {
+ HWT_CTX_UNLOCK(ctx);
+ return;
+ }
+
+ dprintf("%s: thr %p index %d tid %d on cpu_id %d\n", __func__, thr,
+ thr->thread_id, td->td_tid, cpu_id);
+
+ hwt_backend_disable(ctx, cpu_id);
+ HWT_THR_UNLOCK(thr);
+}
+
+static void
+hwt_thread_exit(struct thread *td)
+{
+ struct hwt_context *ctx;
+ struct hwt_thread *thr;
+ struct proc *p;
+ int cpu_id;
+
+ p = td->td_proc;
+
+ cpu_id = PCPU_GET(cpuid);
+
+ ctx = hwt_contexthash_lookup(p);
+ if (ctx == NULL)
+ return;
+
+ if (ctx->state != CTX_STATE_RUNNING) {
+ HWT_CTX_UNLOCK(ctx);
+ return;
+ }
+ thr = hwt_thread_lookup(ctx, td);
+ if (thr == NULL) {
+ HWT_CTX_UNLOCK(ctx);
+ return;
+ }
+
+ thr->state = HWT_THREAD_STATE_EXITED;
+
+ dprintf("%s: thr %p index %d tid %d on cpu_id %d\n", __func__, thr,
+ thr->thread_id, td->td_tid, cpu_id);
+
+ hwt_backend_disable(ctx, cpu_id);
+ HWT_THR_UNLOCK(thr);
+}
+
+static void
+hwt_hook_mmap(struct thread *td)
+{
+ struct hwt_context *ctx;
+ struct hwt_thread *thr;
+ struct proc *p;
+ int pause;
+
+ p = td->td_proc;
+
+ ctx = hwt_contexthash_lookup(p);
+ if (ctx == NULL)
+ return;
+
+ /* The ctx state could be any here. */
+
+ pause = ctx->pause_on_mmap ? 1 : 0;
+
+ thr = hwt_thread_lookup(ctx, td);
+ if (thr == NULL) {
+ HWT_CTX_UNLOCK(ctx);
+ return;
+ }
+
+ /*
+ * msleep(9) atomically releases the mtx lock, so take refcount
+ * to ensure that thr is not destroyed.
+ */
+ refcount_acquire(&thr->refcnt);
+
+ if (pause)
+ msleep_spin(thr, &thr->mtx, "hwt-mmap", 0);
+
+ HWT_THR_UNLOCK(thr);
+
+ if (refcount_release(&thr->refcnt))
+ hwt_thread_free(thr);
+}
+
+static void
+hwt_hook_handler(struct thread *td, int func, void *arg)
+{
+ struct proc *p;
+
+ p = td->td_proc;
+ if ((p->p_flag2 & P2_HWT) == 0)
+ return;
+
+ switch (func) {
+ case HWT_SWITCH_IN:
+ hwt_switch_in(td);
+ break;
+ case HWT_SWITCH_OUT:
+ hwt_switch_out(td);
+ break;
+ case HWT_THREAD_CREATE:
+ hwt_thread_create(td);
+ break;
+ case HWT_THREAD_SET_NAME:
+ /* TODO. */
+ break;
+ case HWT_THREAD_EXIT:
+ hwt_thread_exit(td);
+ break;
+ case HWT_EXEC:
+ hwt_record(td, arg);
+ hwt_hook_mmap(td);
+ break;
+ case HWT_MMAP:
+ hwt_record(td, arg);
+ hwt_hook_mmap(td);
+ break;
+ case HWT_RECORD:
+ hwt_record(td, arg);
+ break;
+ };
+}
+
+void
+hwt_hook_load(void)
+{
+
+ hwt_hook = hwt_hook_handler;
+}
+
+void
+hwt_hook_unload(void)
+{
+
+ hwt_hook = NULL;
+}
Index: sys/dev/hwt/hwt_ioctl.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_ioctl.h
@@ -0,0 +1,37 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_IOCTL_H
+#define _DEV_HWT_HWT_IOCTL_H
+
+int hwt_ioctl(struct cdev *dev, u_long cmd, caddr_t addr, int flags,
+ struct thread *td);
+
+#endif /* !_DEV_HWT_HWT_IOCTL_H */
Index: sys/dev/hwt/hwt_ioctl.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_ioctl.c
@@ -0,0 +1,374 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/* Hardware Trace (HWT) framework. */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/rwlock.h>
+#include <sys/smp.h>
+#include <sys/hwt.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_cpu.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_owner.h>
+#include <dev/hwt/hwt_ownerhash.h>
+#include <dev/hwt/hwt_backend.h>
+#include <dev/hwt/hwt_record.h>
+#include <dev/hwt/hwt_ioctl.h>
+#include <dev/hwt/hwt_vm.h>
+
+#define HWT_IOCTL_DEBUG
+#undef HWT_IOCTL_DEBUG
+
+#ifdef HWT_IOCTL_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+/* No real reason for these limitations just sanity checks. */
+#define HWT_MAXBUFSIZE (32UL * 1024 * 1024 * 1024) /* 32 GB */
+
+static MALLOC_DEFINE(M_HWT_IOCTL, "hwt_ioctl", "Hardware Trace");
+
+/*
+ * Check if owner process *o can trace target process *t.
+ */
+
+static int
+hwt_priv_check(struct proc *o, struct proc *t)
+{
+ struct ucred *oc, *tc;
+ int error;
+ int i;
+
+ PROC_LOCK(o);
+ oc = o->p_ucred;
+ crhold(oc);
+ PROC_UNLOCK(o);
+
+ PROC_LOCK_ASSERT(t, MA_OWNED);
+ tc = t->p_ucred;
+ crhold(tc);
+
+ error = 0;
+
+ /*
+ * The effective uid of the HWT owner should match at least one
+ * of the effective / real / saved uids of the target process.
+ */
+
+ if (oc->cr_uid != tc->cr_uid &&
+ oc->cr_uid != tc->cr_svuid &&
+ oc->cr_uid != tc->cr_ruid) {
+ error = EPERM;
+ goto done;
+ }
+
+ /*
+ * Everyone of the target's group ids must be in the owner's
+ * group list.
+ */
+ for (i = 0; i < tc->cr_ngroups; i++)
+ if (!groupmember(tc->cr_groups[i], oc)) {
+ error = EPERM;
+ goto done;
+ }
+
+ /* Check the read and saved GIDs too. */
+ if (!groupmember(tc->cr_rgid, oc) ||
+ !groupmember(tc->cr_svgid, oc)) {
+ error = EPERM;
+ goto done;
+ }
+
+done:
+ crfree(tc);
+ crfree(oc);
+
+ return (error);
+}
+
+static int
+hwt_ioctl_alloc_mode_thread(struct thread *td, struct hwt_owner *ho,
+ struct hwt_backend *backend, struct hwt_alloc *halloc)
+{
+ char path[MAXPATHLEN];
+ struct hwt_context *ctx;
+ struct hwt_thread *thr;
+ struct proc *p;
+ int thread_id;
+ int error;
+
+ /* Check if the owner have this pid configured already. */
+ ctx = hwt_owner_lookup_ctx(ho, halloc->pid);
+ if (ctx)
+ return (EEXIST);
+
+ /* Allocate a new HWT context. */
+ ctx = hwt_ctx_alloc();
+ ctx->bufsize = halloc->bufsize;
+ ctx->pid = halloc->pid;
+ ctx->hwt_backend = backend;
+ ctx->hwt_owner = ho;
+ ctx->mode = HWT_MODE_THREAD;
+
+ error = copyout(&ctx->ident, halloc->ident, sizeof(int));
+ if (error) {
+ hwt_ctx_free(ctx);
+ return (error);
+ }
+
+ thread_id = atomic_fetchadd_int(&ctx->thread_counter, 1);
+
+ /* Allocate first thread and buffers. */
+ sprintf(path, "hwt_%d_%d", ctx->ident, thread_id);
+ error = hwt_thread_alloc(&thr, path, ctx->bufsize);
+ if (error) {
+ hwt_ctx_free(ctx);
+ return (error);
+ }
+ thr->vm->ctx = ctx;
+
+ /* Since we done with malloc, now get the victim proc. */
+ p = pfind(halloc->pid);
+ if (p == NULL) {
+ hwt_thread_free(thr);
+ hwt_ctx_free(ctx);
+ return (ENXIO);
+ }
+
+ /* Ensure we can trace it. */
+ error = hwt_priv_check(td->td_proc, p);
+ if (error) {
+ hwt_thread_free(thr);
+ hwt_ctx_free(ctx);
+ PROC_UNLOCK(p);
+ return (error);
+ }
+ p->p_flag2 |= P2_HWT;
+ thr->td = FIRST_THREAD_IN_PROC(p);
+ ctx->proc = p;
+ PROC_UNLOCK(p);
+
+ /* All good. */
+ thr->ctx = ctx;
+ thr->thread_id = thread_id;
+
+ HWT_CTX_LOCK(ctx);
+ hwt_thread_insert(ctx, thr);
+ HWT_CTX_UNLOCK(ctx);
+
+ error = hwt_backend_init(ctx);
+ if (error) {
+ hwt_thread_free(thr);
+ hwt_ctx_free(ctx);
+ /* TODO: remove P2_HWT from proc, if it is still there. */
+ return (error);
+ }
+
+ /* hwt_owner_insert_ctx? */
+ mtx_lock(&ho->mtx);
+ LIST_INSERT_HEAD(&ho->hwts, ctx, next_hwts);
+ mtx_unlock(&ho->mtx);
+
+ /*
+ * Hooks are now in action after this, but the ctx is not in RUNNING
+ * state.
+ */
+ hwt_contexthash_insert(ctx);
+
+ return (0);
+}
+
+static int
+hwt_ioctl_alloc_mode_cpu(struct thread *td, struct hwt_owner *ho,
+ struct hwt_backend *backend, struct hwt_alloc *halloc)
+{
+ struct hwt_context *ctx;
+ struct hwt_cpu *cpu;
+ struct hwt_vm *vm;
+ char path[MAXPATHLEN];
+ int error;
+ int cpu_id;
+ int cpu_count;
+
+ cpu_count = 0;
+
+ CPU_FOREACH(cpu_id) {
+ if (!CPU_ISSET(cpu_id, &halloc->cpu_map))
+ continue;
+ /* Ensure CPU is not halted. */
+ if (CPU_ISSET(cpu_id, &hlt_cpus_mask))
+ return (ENXIO);
+#if 0
+ /* TODO: Check if the owner have this cpu configured already. */
+ ctx = hwt_owner_lookup_ctx_by_cpu(ho, halloc->cpu);
+ if (ctx)
+ return (EEXIST);
+#endif
+
+ cpu_count++;
+ }
+
+ if (cpu_count == 0)
+ return (ENODEV);
+
+ /* Allocate a new HWT context. */
+ ctx = hwt_ctx_alloc();
+ ctx->bufsize = halloc->bufsize;
+ ctx->hwt_backend = backend;
+ ctx->hwt_owner = ho;
+ ctx->mode = HWT_MODE_CPU;
+ ctx->cpu_map = halloc->cpu_map;
+
+ error = copyout(&ctx->ident, halloc->ident, sizeof(int));
+ if (error) {
+ hwt_ctx_free(ctx);
+ return (error);
+ }
+
+ CPU_FOREACH(cpu_id) {
+ if (!CPU_ISSET(cpu_id, &halloc->cpu_map))
+ continue;
+
+ sprintf(path, "hwt_%d_%d", ctx->ident, cpu_id);
+ error = hwt_vm_alloc(ctx->bufsize, path, &vm);
+ if (error) {
+ /* TODO: remove all allocated cpus. */
+ hwt_ctx_free(ctx);
+ return (error);
+ }
+
+ cpu = hwt_cpu_alloc();
+ cpu->cpu_id = cpu_id;
+ cpu->vm = vm;
+
+ vm->cpu = cpu;
+ vm->ctx = ctx;
+
+ HWT_CTX_LOCK(ctx);
+ hwt_cpu_insert(ctx, cpu);
+ HWT_CTX_UNLOCK(ctx);
+ }
+
+ error = hwt_backend_init(ctx);
+ if (error) {
+ /* TODO: remove all allocated cpus. */
+ hwt_ctx_free(ctx);
+ return (error);
+ }
+
+ /* hwt_owner_insert_ctx? */
+ mtx_lock(&ho->mtx);
+ LIST_INSERT_HEAD(&ho->hwts, ctx, next_hwts);
+ mtx_unlock(&ho->mtx);
+
+ hwt_record_kernel_objects(ctx);
+
+ return (0);
+}
+
+static int
+hwt_ioctl_alloc(struct thread *td, struct hwt_alloc *halloc)
+{
+ char backend_name[HWT_BACKEND_MAXNAMELEN];
+ struct hwt_backend *backend;
+ struct hwt_owner *ho;
+ int error;
+
+ if (halloc->bufsize > HWT_MAXBUFSIZE)
+ return (EINVAL);
+ if (halloc->bufsize % PAGE_SIZE)
+ return (EINVAL);
+ if (halloc->backend_name == NULL)
+ return (EINVAL);
+
+ error = copyinstr(halloc->backend_name, (void *)backend_name,
+ HWT_BACKEND_MAXNAMELEN, NULL);
+ if (error)
+ return (error);
+
+ backend = hwt_backend_lookup(backend_name);
+ if (backend == NULL)
+ return (ENODEV);
+
+ /* First get the owner. */
+ ho = hwt_ownerhash_lookup(td->td_proc);
+ if (ho == NULL) {
+ /* Create a new owner. */
+ ho = hwt_owner_alloc(td->td_proc);
+ if (ho == NULL)
+ return (ENOMEM);
+ hwt_ownerhash_insert(ho);
+ }
+
+ switch (halloc->mode) {
+ case HWT_MODE_THREAD:
+ error = hwt_ioctl_alloc_mode_thread(td, ho, backend, halloc);
+ break;
+ case HWT_MODE_CPU:
+ error = hwt_ioctl_alloc_mode_cpu(td, ho, backend, halloc);
+ break;
+ default:
+ error = ENXIO;
+ };
+
+ return (error);
+}
+
+int
+hwt_ioctl(struct cdev *dev, u_long cmd, caddr_t addr, int flags,
+ struct thread *td)
+{
+ int error;
+
+ switch (cmd) {
+ case HWT_IOC_ALLOC:
+ /* Allocate HWT context. */
+ error = hwt_ioctl_alloc(td, (struct hwt_alloc *)addr);
+ return (error);
+ default:
+ return (ENXIO);
+ };
+}
Index: sys/dev/hwt/hwt_owner.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_owner.h
@@ -0,0 +1,47 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_OWNER_H_
+#define _DEV_HWT_HWT_OWNER_H_
+
+struct hwt_owner {
+ struct proc *p;
+ struct mtx mtx; /* Protects hwts. */
+ LIST_HEAD(, hwt_context) hwts; /* Owned HWTs. */
+ LIST_ENTRY(hwt_owner) next; /* Entry in hwt owner hash. */
+};
+
+
+struct hwt_context * hwt_owner_lookup_ctx(struct hwt_owner *ho, pid_t pid);
+struct hwt_owner * hwt_owner_alloc(struct proc *p);
+void hwt_owner_shutdown(struct hwt_owner *ho);
+struct hwt_context * hwt_owner_lookup_ctx_by_cpu(struct hwt_owner *ho, int cpu);
+
+#endif /* !_DEV_HWT_HWT_OWNER_H_ */
Index: sys/dev/hwt/hwt_owner.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_owner.c
@@ -0,0 +1,218 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/refcount.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_cpu.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_owner.h>
+#include <dev/hwt/hwt_ownerhash.h>
+#include <dev/hwt/hwt_backend.h>
+#include <dev/hwt/hwt_vm.h>
+#include <dev/hwt/hwt_record.h>
+
+#define HWT_DEBUG
+#undef HWT_DEBUG
+
+#ifdef HWT_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static MALLOC_DEFINE(M_HWT_OWNER, "hwt_owner", "Hardware Trace");
+
+struct hwt_context *
+hwt_owner_lookup_ctx(struct hwt_owner *ho, pid_t pid)
+{
+ struct hwt_context *ctx;
+
+ mtx_lock(&ho->mtx);
+ LIST_FOREACH(ctx, &ho->hwts, next_hwts) {
+ if (ctx->pid == pid) {
+ mtx_unlock(&ho->mtx);
+ return (ctx);
+ }
+ }
+ mtx_unlock(&ho->mtx);
+
+ return (NULL);
+}
+
+#if 0
+struct hwt_context *
+hwt_owner_lookup_ctx_by_cpu(struct hwt_owner *ho, int cpu)
+{
+ struct hwt_context *ctx;
+
+ mtx_lock(&ho->mtx);
+ LIST_FOREACH(ctx, &ho->hwts, next_hwts) {
+ if (ctx->cpu == cpu) {
+ mtx_unlock(&ho->mtx);
+ return (ctx);
+ }
+ }
+ mtx_unlock(&ho->mtx);
+
+ return (NULL);
+}
+#endif
+
+struct hwt_owner *
+hwt_owner_alloc(struct proc *p)
+{
+ struct hwt_owner *ho;
+
+ ho = malloc(sizeof(struct hwt_owner), M_HWT_OWNER,
+ M_WAITOK | M_ZERO);
+ ho->p = p;
+
+ LIST_INIT(&ho->hwts);
+ mtx_init(&ho->mtx, "hwts", NULL, MTX_DEF);
+
+ return (ho);
+}
+
+static void
+hwt_owner_free_cpus(struct hwt_context *ctx)
+{
+ struct hwt_cpu *cpu;
+
+ do {
+ HWT_CTX_LOCK(ctx);
+ cpu = TAILQ_FIRST(&ctx->cpus);
+ if (cpu)
+ TAILQ_REMOVE(&ctx->cpus, cpu, next);
+ HWT_CTX_UNLOCK(ctx);
+
+ if (cpu == NULL)
+ break;
+
+ hwt_vm_free(cpu->vm);
+ hwt_cpu_free(cpu);
+ } while (1);
+}
+
+static void
+hwt_owner_free_threads(struct hwt_context *ctx)
+{
+ struct hwt_thread *thr;
+
+ dprintf("%s: remove threads\n", __func__);
+
+ do {
+ HWT_CTX_LOCK(ctx);
+ thr = TAILQ_FIRST(&ctx->threads);
+ if (thr) {
+ TAILQ_REMOVE(&ctx->threads, thr, next);
+ HWT_THR_LOCK(thr);
+ }
+ HWT_CTX_UNLOCK(ctx);
+
+ if (thr == NULL)
+ break;
+
+ wakeup(thr);
+
+ HWT_THR_UNLOCK(thr);
+
+ if (refcount_release(&thr->refcnt))
+ hwt_thread_free(thr);
+ } while (1);
+}
+
+void
+hwt_owner_shutdown(struct hwt_owner *ho)
+{
+ struct hwt_context *ctx;
+ struct hwt_thread *thr;
+
+ dprintf("%s: stopping hwt owner\n", __func__);
+
+ while (1) {
+ mtx_lock(&ho->mtx);
+ ctx = LIST_FIRST(&ho->hwts);
+ if (ctx)
+ LIST_REMOVE(ctx, next_hwts);
+ mtx_unlock(&ho->mtx);
+
+ if (ctx == NULL)
+ break;
+
+ if (ctx->mode == HWT_MODE_THREAD)
+ hwt_contexthash_remove(ctx);
+
+ /*
+ * It could be that a hook has this ctx locked right here.
+ */
+
+ HWT_CTX_LOCK(ctx);
+ ctx->state = 0;
+ /*
+ * Ensure hook invocation is now completed.
+ */
+ TAILQ_FOREACH(thr, &ctx->threads, next) {
+ HWT_THR_LOCK(thr);
+ HWT_THR_UNLOCK(thr);
+ }
+ HWT_CTX_UNLOCK(ctx);
+
+ /* Note that a thread could be still sleeping on msleep_spin. */
+
+ hwt_backend_deinit(ctx);
+
+ if (ctx->mode == HWT_MODE_CPU)
+ hwt_owner_free_cpus(ctx);
+ else
+ hwt_owner_free_threads(ctx);
+
+ hwt_record_free_all(ctx);
+ hwt_ctx_free(ctx);
+ }
+
+ hwt_ownerhash_remove(ho);
+ free(ho, M_HWT_OWNER);
+}
Index: sys/dev/hwt/hwt_ownerhash.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_ownerhash.h
@@ -0,0 +1,44 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_OWNERHASH_H_
+#define _DEV_HWT_HWT_OWNERHASH_H_
+
+struct hwt_owner * hwt_ownerhash_lookup(struct proc *p);
+void hwt_ownerhash_insert(struct hwt_owner *ho);
+void hwt_ownerhash_remove(struct hwt_owner *ho);
+
+void hwt_ownerhash_load(void);
+void hwt_ownerhash_unload(void);
+
+#define HWT_OWNERHASH_LOCK() mtx_lock_spin(&hwt_ownerhash_mtx)
+#define HWT_OWNERHASH_UNLOCK() mtx_unlock_spin(&hwt_ownerhash_mtx)
+
+#endif /* !_DEV_HWT_HWT_OWNERHASH_H_ */
Index: sys/dev/hwt/hwt_ownerhash.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_ownerhash.c
@@ -0,0 +1,147 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+
+#include <dev/hwt/hwt_owner.h>
+#include <dev/hwt/hwt_ownerhash.h>
+
+#define HWT_DEBUG
+#undef HWT_DEBUG
+
+#ifdef HWT_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+#define HWT_OWNERHASH_SIZE 1024
+
+static MALLOC_DEFINE(M_HWT_OWNERHASH, "hwt_ohash", "Hardware Trace");
+
+/*
+ * Hash function. Discard the lower 2 bits of the pointer since
+ * these are always zero for our uses. The hash multiplier is
+ * round((2^LONG_BIT) * ((sqrt(5)-1)/2)).
+ */
+
+#define _HWT_HM 11400714819323198486u /* hash multiplier */
+#define HWT_HASH_PTR(P, M) ((((unsigned long) (P) >> 2) * _HWT_HM) & (M))
+
+static struct mtx hwt_ownerhash_mtx;
+static u_long hwt_ownerhashmask;
+static LIST_HEAD(hwt_ownerhash, hwt_owner) *hwt_ownerhash;
+
+struct hwt_owner *
+hwt_ownerhash_lookup(struct proc *p)
+{
+ struct hwt_ownerhash *hoh;
+ struct hwt_owner *ho;
+ int hindex;
+
+ hindex = HWT_HASH_PTR(p, hwt_ownerhashmask);
+ hoh = &hwt_ownerhash[hindex];
+
+ HWT_OWNERHASH_LOCK();
+ LIST_FOREACH(ho, hoh, next) {
+ if (ho->p == p) {
+ HWT_OWNERHASH_UNLOCK();
+ return (ho);
+ }
+ }
+ HWT_OWNERHASH_UNLOCK();
+
+ return (NULL);
+}
+
+void
+hwt_ownerhash_insert(struct hwt_owner *ho)
+{
+ struct hwt_ownerhash *hoh;
+ int hindex;
+
+ hindex = HWT_HASH_PTR(ho->p, hwt_ownerhashmask);
+ hoh = &hwt_ownerhash[hindex];
+
+ HWT_OWNERHASH_LOCK();
+ LIST_INSERT_HEAD(hoh, ho, next);
+ HWT_OWNERHASH_UNLOCK();
+}
+
+void
+hwt_ownerhash_remove(struct hwt_owner *ho)
+{
+
+ /* Destroy hwt owner. */
+ HWT_OWNERHASH_LOCK();
+ LIST_REMOVE(ho, next);
+ HWT_OWNERHASH_UNLOCK();
+}
+
+void
+hwt_ownerhash_load(void)
+{
+
+ hwt_ownerhash = hashinit(HWT_OWNERHASH_SIZE, M_HWT_OWNERHASH,
+ &hwt_ownerhashmask);
+ mtx_init(&hwt_ownerhash_mtx, "hwt-owner-hash", "hwt-owner", MTX_SPIN);
+}
+
+void
+hwt_ownerhash_unload(void)
+{
+#if 0
+ struct hwt_ownerhash *hoh;
+ struct hwt_owner *ho, *tmp;
+
+ HWT_OWNERHASH_LOCK();
+ for (hoh = hwt_ownerhash;
+ hoh <= &hwt_ownerhash[hwt_ownerhashmask];
+ hoh++) {
+ LIST_FOREACH_SAFE(ho, hoh, next, tmp) {
+ LIST_REMOVE(ho, next);
+ }
+ }
+ HWT_OWNERHASH_UNLOCK();
+#endif
+ mtx_destroy(&hwt_ownerhash_mtx);
+ hashdestroy(hwt_ownerhash, M_HWT_OWNERHASH, hwt_ownerhashmask);
+}
Index: sys/dev/hwt/hwt_record.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_record.h
@@ -0,0 +1,42 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_RECORD_H_
+#define _DEV_HWT_HWT_RECORD_H_
+
+int hwt_record_send(struct hwt_context *ctx, struct hwt_record_get *record_get);
+void hwt_record(struct thread *td, struct hwt_record_entry *ent);
+
+struct hwt_record_entry * hwt_record_entry_alloc(void);
+void hwt_record_entry_free(struct hwt_record_entry *entry);
+void hwt_record_kernel_objects(struct hwt_context *ctx);
+void hwt_record_free_all(struct hwt_context *ctx);
+
+#endif /* !_DEV_HWT_HWT_RECORD_H_ */
Index: sys/dev/hwt/hwt_record.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_record.c
@@ -0,0 +1,213 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mutex.h>
+#include <sys/hwt.h>
+#include <sys/linker.h>
+#include <sys/pmckern.h> /* linker_hwpmc_list_objects */
+
+#include <vm/vm.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_record.h>
+
+#define HWT_RECORD_DEBUG
+#undef HWT_RECORD_DEBUG
+
+#ifdef HWT_RECORD_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static MALLOC_DEFINE(M_HWT_RECORD, "hwt_record", "Hardware Trace");
+
+void
+hwt_record(struct thread *td, struct hwt_record_entry *ent)
+{
+ struct hwt_record_entry *entry;
+ struct hwt_context *ctx;
+ struct proc *p;
+
+ p = td->td_proc;
+
+ KASSERT(ent != NULL, ("ent is NULL"));
+ KASSERT(ent->fullpath != NULL, ("fullpath is NULL"));
+
+ entry = malloc(sizeof(struct hwt_record_entry), M_HWT_RECORD, M_WAITOK);
+ entry->record_type = ent->record_type;
+ entry->thread_id = -1;
+ entry->fullpath = strdup(ent->fullpath, M_HWT_RECORD);
+ entry->addr = ent->addr;
+
+ ctx = hwt_contexthash_lookup(p);
+ if (ctx == NULL) {
+ free(entry->fullpath, M_HWT_RECORD);
+ free(entry, M_HWT_RECORD);
+ return;
+ }
+ LIST_INSERT_HEAD(&ctx->records, entry, next);
+ HWT_CTX_UNLOCK(ctx);
+}
+
+struct hwt_record_entry *
+hwt_record_entry_alloc(void)
+{
+ struct hwt_record_entry *entry;
+
+ entry = malloc(sizeof(struct hwt_record_entry), M_HWT_RECORD,
+ M_WAITOK | M_ZERO);
+
+ return (entry);
+}
+
+void
+hwt_record_entry_free(struct hwt_record_entry *entry)
+{
+
+ free(entry, M_HWT_RECORD);
+}
+
+static int
+hwt_record_grab(struct hwt_context *ctx,
+ struct hwt_record_user_entry *user_entry, int nitems_req)
+{
+ struct hwt_record_entry *entry;
+ int i;
+
+ for (i = 0; i < nitems_req; i++) {
+ HWT_CTX_LOCK(ctx);
+ entry = LIST_FIRST(&ctx->records);
+ if (entry)
+ LIST_REMOVE(entry, next);
+ HWT_CTX_UNLOCK(ctx);
+
+ if (entry == NULL)
+ break;
+
+ user_entry[i].addr = entry->addr;
+ user_entry[i].record_type = entry->record_type;
+ user_entry[i].thread_id = entry->thread_id;
+ if (entry->fullpath != NULL) {
+ strncpy(user_entry[i].fullpath, entry->fullpath,
+ MAXPATHLEN);
+ free(entry->fullpath, M_HWT_RECORD);
+ }
+
+ free(entry, M_HWT_RECORD);
+ }
+
+ return (i);
+}
+
+void
+hwt_record_free_all(struct hwt_context *ctx)
+{
+ struct hwt_record_entry *entry;
+
+ while (1) {
+ HWT_CTX_LOCK(ctx);
+ entry = LIST_FIRST(&ctx->records);
+ if (entry)
+ LIST_REMOVE(entry, next);
+ HWT_CTX_UNLOCK(ctx);
+
+ if (entry == NULL)
+ break;
+
+ if (entry->fullpath != NULL)
+ free(entry->fullpath, M_HWT_RECORD);
+
+ free(entry, M_HWT_RECORD);
+ }
+}
+
+int
+hwt_record_send(struct hwt_context *ctx, struct hwt_record_get *record_get)
+{
+ struct hwt_record_user_entry *user_entry;
+ int nitems_req;
+ int error;
+ int i;
+
+ nitems_req = 0;
+
+ error = copyin(record_get->nentries, &nitems_req, sizeof(int));
+ if (error)
+ return (error);
+
+ if (nitems_req < 1 || nitems_req > 1024)
+ return (ENXIO);
+
+ user_entry = malloc(sizeof(struct hwt_record_user_entry) * nitems_req,
+ M_HWT_RECORD, M_WAITOK | M_ZERO);
+
+ i = hwt_record_grab(ctx, user_entry, nitems_req);
+ if (i > 0)
+ error = copyout(user_entry, record_get->records,
+ sizeof(struct hwt_record_user_entry) * i);
+
+ if (error == 0)
+ error = copyout(&i, record_get->nentries, sizeof(int));
+
+ free(user_entry, M_HWT_RECORD);
+
+ return (error);
+}
+
+void
+hwt_record_kernel_objects(struct hwt_context *ctx)
+{
+ struct hwt_record_entry *entry;
+ struct pmckern_map_in *kobase;
+ int i;
+
+ kobase = linker_hwpmc_list_objects();
+ for (i = 0; kobase[i].pm_file != NULL; i++) {
+ entry = malloc(sizeof(struct hwt_record_entry), M_HWT_RECORD,
+ M_WAITOK);
+ entry->record_type = HWT_RECORD_KERNEL;
+ entry->fullpath = strdup(kobase[i].pm_file, M_HWT_RECORD);
+ entry->addr = kobase[i].pm_address;
+
+ HWT_CTX_LOCK(ctx);
+ LIST_INSERT_HEAD(&ctx->records, entry, next);
+ HWT_CTX_UNLOCK(ctx);
+ }
+ free(kobase, M_LINKER);
+}
Index: sys/dev/hwt/hwt_thread.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_thread.h
@@ -0,0 +1,64 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_THREAD_H_
+#define _DEV_HWT_HWT_THREAD_H_
+
+struct hwt_thread {
+ struct hwt_vm *vm;
+ struct hwt_context *ctx;
+ struct thread *td;
+ TAILQ_ENTRY(hwt_thread) next;
+ int thread_id;
+ int state;
+#define HWT_THREAD_STATE_EXITED (1 << 0)
+ struct mtx mtx;
+ u_int refcnt;
+ int cpu_id; /* last cpu_id */
+};
+
+/* Thread allocation. */
+int hwt_thread_alloc(struct hwt_thread **thr0, char *path, size_t bufsize);
+int hwt_thread_create(struct thread *td);
+
+/* Thread de-allocation. */
+void hwt_thread_free(struct hwt_thread *thr);
+
+/* Thread list mgt. */
+void hwt_thread_insert(struct hwt_context *ctx, struct hwt_thread *thr);
+struct hwt_thread * hwt_thread_first(struct hwt_context *ctx);
+struct hwt_thread * hwt_thread_lookup(struct hwt_context *ctx,
+ struct thread *td);
+
+#define HWT_THR_LOCK(thr) mtx_lock_spin(&(thr)->mtx)
+#define HWT_THR_UNLOCK(thr) mtx_unlock_spin(&(thr)->mtx)
+#define HWT_THR_ASSERT_LOCKED(thr) mtx_assert(&(thr)->mtx, MA_OWNED)
+
+#endif /* !_DEV_HWT_HWT_THREAD_H_ */
Index: sys/dev/hwt/hwt_thread.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_thread.c
@@ -0,0 +1,215 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/refcount.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+
+#include <vm/vm.h>
+#include <vm/vm_param.h>
+#include <vm/vm_kern.h>
+#include <vm/vm_page.h>
+#include <vm/vm_object.h>
+#include <vm/vm_pager.h>
+#include <vm/vm_pageout.h>
+#include <vm/vm_phys.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_owner.h>
+#include <dev/hwt/hwt_ownerhash.h>
+#include <dev/hwt/hwt_backend.h>
+#include <dev/hwt/hwt_vm.h>
+#include <dev/hwt/hwt_record.h>
+
+#define HWT_THREAD_DEBUG
+#undef HWT_THREAD_DEBUG
+
+#ifdef HWT_THREAD_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static MALLOC_DEFINE(M_HWT_THREAD, "hwt_thread", "Hardware Trace");
+
+struct hwt_thread *
+hwt_thread_first(struct hwt_context *ctx)
+{
+ struct hwt_thread *thr;
+
+ HWT_CTX_ASSERT_LOCKED(ctx);
+
+ thr = TAILQ_FIRST(&ctx->threads);
+
+ KASSERT(thr != NULL, ("thr is NULL"));
+
+ return (thr);
+}
+
+/*
+ * To use by hwt_switch_in/out() only.
+ */
+struct hwt_thread *
+hwt_thread_lookup(struct hwt_context *ctx, struct thread *td)
+{
+ struct hwt_thread *thr;
+
+ HWT_CTX_ASSERT_LOCKED(ctx);
+
+ TAILQ_FOREACH(thr, &ctx->threads, next) {
+ if (thr->td == td) {
+ HWT_THR_LOCK(thr);
+ HWT_CTX_UNLOCK(ctx);
+ return (thr);
+ }
+ }
+
+ /*
+ * We are here because the hook on thread creation failed to allocate
+ * a thread.
+ */
+
+ return (NULL);
+}
+
+int
+hwt_thread_alloc(struct hwt_thread **thr0, char *path, size_t bufsize)
+{
+ struct hwt_thread *thr;
+ struct hwt_vm *vm;
+ int error;
+
+ error = hwt_vm_alloc(bufsize, path, &vm);
+ if (error)
+ return (error);
+
+ thr = malloc(sizeof(struct hwt_thread), M_HWT_THREAD,
+ M_WAITOK | M_ZERO);
+ thr->vm = vm;
+
+ mtx_init(&thr->mtx, "thr", NULL, MTX_SPIN);
+
+ refcount_init(&thr->refcnt, 1);
+
+ vm->thr = thr;
+
+ *thr0 = thr;
+
+ return (0);
+}
+
+void
+hwt_thread_free(struct hwt_thread *thr)
+{
+
+ hwt_vm_free(thr->vm);
+
+ free(thr, M_HWT_THREAD);
+}
+
+void
+hwt_thread_insert(struct hwt_context *ctx, struct hwt_thread *thr)
+{
+
+ HWT_CTX_ASSERT_LOCKED(ctx);
+
+ TAILQ_INSERT_TAIL(&ctx->threads, thr, next);
+}
+
+/*
+ * This is called by hooks only.
+ * TODO: Move to hwt_hook.c ?
+ */
+int
+hwt_thread_create(struct thread *td)
+{
+ struct hwt_record_entry *entry;
+ struct hwt_context *ctx;
+ struct hwt_thread *thr;
+ struct proc *p;
+ size_t bufsize;
+ char path[MAXPATHLEN];
+ int error;
+ int thread_id;
+
+ p = td->td_proc;
+
+ /* 1. First find our ctx and collect some information from it. */
+ ctx = hwt_contexthash_lookup(p);
+ if (ctx == NULL)
+ return (ENXIO);
+ bufsize = ctx->bufsize;
+ thread_id = atomic_fetchadd_int(&ctx->thread_counter, 1);
+ sprintf(path, "hwt_%d_%d", ctx->ident, thread_id);
+ HWT_CTX_UNLOCK(ctx);
+
+ /* 2. Now we can allocate some memory. */
+ error = hwt_thread_alloc(&thr, path, bufsize);
+ if (error) {
+ printf("%s: could not allocate thread, error %d\n",
+ __func__, error);
+ return (error);
+ }
+
+ entry = hwt_record_entry_alloc();
+ entry->record_type = HWT_RECORD_THREAD_CREATE;
+ entry->thread_id = thread_id;
+
+ /* 3. Take ctx again, as it may gone during previous step. */
+ ctx = hwt_contexthash_lookup(p);
+ if (ctx == NULL) {
+ hwt_record_entry_free(entry);
+ hwt_thread_free(thr);
+ return (ENXIO);
+ }
+ thr->vm->ctx = ctx;
+ thr->ctx = ctx;
+ thr->thread_id = thread_id;
+ thr->td = td;
+ TAILQ_INSERT_TAIL(&ctx->threads, thr, next);
+ LIST_INSERT_HEAD(&ctx->records, entry, next);
+ HWT_CTX_UNLOCK(ctx);
+
+ return (0);
+}
Index: sys/dev/hwt/hwt_vm.h
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_vm.h
@@ -0,0 +1,48 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _DEV_HWT_HWT_VM_H_
+#define _DEV_HWT_HWT_VM_H_
+
+struct hwt_vm {
+ vm_page_t *pages;
+ int npages;
+ vm_object_t obj;
+ struct cdev *cdev;
+
+ struct hwt_context *ctx;
+ struct hwt_cpu *cpu; /* cpu mode only. */
+ struct hwt_thread *thr; /* thr mode only. */
+};
+
+int hwt_vm_alloc(size_t bufsize, char *path, struct hwt_vm **vm0);
+void hwt_vm_free(struct hwt_vm *vm);
+
+#endif /* !_DEV_HWT_HWT_VM_H_ */
Index: sys/dev/hwt/hwt_vm.c
===================================================================
--- /dev/null
+++ sys/dev/hwt/hwt_vm.c
@@ -0,0 +1,433 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/refcount.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+#include <sys/smp.h>
+
+#include <vm/vm.h>
+#include <vm/vm_param.h>
+#include <vm/vm_kern.h>
+#include <vm/vm_page.h>
+#include <vm/vm_object.h>
+#include <vm/vm_pager.h>
+#include <vm/vm_pageout.h>
+#include <vm/vm_phys.h>
+
+#include <dev/hwt/hwt_hook.h>
+#include <dev/hwt/hwt_context.h>
+#include <dev/hwt/hwt_contexthash.h>
+#include <dev/hwt/hwt_config.h>
+#include <dev/hwt/hwt_cpu.h>
+#include <dev/hwt/hwt_owner.h>
+#include <dev/hwt/hwt_ownerhash.h>
+#include <dev/hwt/hwt_thread.h>
+#include <dev/hwt/hwt_backend.h>
+#include <dev/hwt/hwt_vm.h>
+#include <dev/hwt/hwt_record.h>
+
+#define HWT_THREAD_DEBUG
+#undef HWT_THREAD_DEBUG
+
+#ifdef HWT_THREAD_DEBUG
+#define dprintf(fmt, ...) printf(fmt, ##__VA_ARGS__)
+#else
+#define dprintf(fmt, ...)
+#endif
+
+static MALLOC_DEFINE(M_HWT_VM, "hwt_vm", "Hardware Trace");
+
+static int
+hwt_vm_fault(vm_object_t vm_obj, vm_ooffset_t offset,
+ int prot, vm_page_t *mres)
+{
+
+ return (0);
+}
+
+static int
+hwt_vm_ctor(void *handle, vm_ooffset_t size, vm_prot_t prot,
+ vm_ooffset_t foff, struct ucred *cred, u_short *color)
+{
+
+ *color = 0;
+
+ return (0);
+}
+
+static void
+hwt_vm_dtor(void *handle)
+{
+
+}
+
+static struct cdev_pager_ops hwt_vm_pager_ops = {
+ .cdev_pg_fault = hwt_vm_fault,
+ .cdev_pg_ctor = hwt_vm_ctor,
+ .cdev_pg_dtor = hwt_vm_dtor
+};
+
+static int
+hwt_vm_alloc_pages(struct hwt_vm *vm)
+{
+ vm_paddr_t low, high, boundary;
+ vm_memattr_t memattr;
+ vm_pointer_t va;
+ int alignment;
+ vm_page_t m;
+ int pflags;
+ int tries;
+ int i;
+
+ alignment = PAGE_SIZE;
+ low = 0;
+ high = -1UL;
+ boundary = 0;
+ pflags = VM_ALLOC_NORMAL | VM_ALLOC_NOBUSY | VM_ALLOC_WIRED |
+ VM_ALLOC_ZERO;
+ memattr = VM_MEMATTR_DEVICE;
+
+ vm->obj = cdev_pager_allocate(vm, OBJT_MGTDEVICE,
+ &hwt_vm_pager_ops, vm->npages * PAGE_SIZE, PROT_READ, 0,
+ curthread->td_ucred);
+
+ for (i = 0; i < vm->npages; i++) {
+ tries = 0;
+retry:
+ m = vm_page_alloc_noobj_contig(pflags, 1, low, high,
+ alignment, boundary, memattr);
+ if (m == NULL) {
+ if (tries < 3) {
+ if (!vm_page_reclaim_contig(pflags, 1, low,
+ high, alignment, boundary))
+ vm_wait(NULL);
+ tries++;
+ goto retry;
+ }
+
+ return (ENOMEM);
+ }
+
+#if 0
+ /* TODO: could not clean device memory on arm64. */
+ if ((m->flags & PG_ZERO) == 0)
+ pmap_zero_page(m);
+#endif
+
+ va = PHYS_TO_DMAP(VM_PAGE_TO_PHYS(m));
+ cpu_dcache_wb_range(va, PAGE_SIZE);
+
+ m->valid = VM_PAGE_BITS_ALL;
+ m->oflags &= ~VPO_UNMANAGED;
+ m->flags |= PG_FICTITIOUS;
+ vm->pages[i] = m;
+
+ VM_OBJECT_WLOCK(vm->obj);
+ vm_page_insert(m, vm->obj, i);
+ VM_OBJECT_WUNLOCK(vm->obj);
+ }
+
+ return (0);
+}
+
+static int
+hwt_vm_open(struct cdev *cdev, int oflags, int devtype, struct thread *td)
+{
+
+ dprintf("%s\n", __func__);
+
+ return (0);
+}
+
+static int
+hwt_vm_mmap_single(struct cdev *cdev, vm_ooffset_t *offset,
+ vm_size_t mapsize, struct vm_object **objp, int nprot)
+{
+ struct hwt_vm *vm;
+
+ vm = cdev->si_drv1;
+
+ if (nprot != PROT_READ || *offset != 0)
+ return (ENXIO);
+
+ *objp = vm->obj;
+
+ return (0);
+}
+
+static void
+hwt_vm_start_cpu_mode(struct hwt_context *ctx)
+{
+ int cpu_id;
+
+ CPU_FOREACH(cpu_id) {
+ if (!CPU_ISSET(cpu_id, &ctx->cpu_map))
+ continue;
+
+ /* Ensure CPU is not halted. */
+ if (CPU_ISSET(cpu_id, &hlt_cpus_mask))
+ return;
+
+ hwt_backend_configure(ctx, cpu_id, cpu_id);
+ hwt_backend_enable(ctx, cpu_id);
+ }
+}
+
+static int
+hwt_vm_ioctl(struct cdev *dev, u_long cmd, caddr_t addr, int flags,
+ struct thread *td)
+{
+ struct hwt_record_get *rget;
+ struct hwt_set_config *sconf;
+ struct hwt_bufptr_get *ptr_get;
+
+ struct hwt_context *ctx;
+ struct hwt_vm *vm;
+ struct hwt_owner *ho;
+
+ vm_offset_t curpage_offset;
+ int cpu_id;
+ int curpage;
+ int error;
+
+ vm = dev->si_drv1;
+ KASSERT(vm != NULL, ("si_drv1 is NULL"));
+
+ ctx = vm->ctx;
+
+ /* Ensure process is registered owner of this HWT. */
+ ho = hwt_ownerhash_lookup(td->td_proc);
+ if (ho == NULL)
+ return (ENXIO);
+
+ if (ctx->hwt_owner != ho)
+ return (EPERM);
+
+ switch (cmd) {
+ case HWT_IOC_START:
+ dprintf("%s: start tracing\n", __func__);
+
+ HWT_CTX_LOCK(ctx);
+ if (ctx->state == CTX_STATE_RUNNING) {
+ /* Already running ? */
+ HWT_CTX_UNLOCK(ctx);
+ return (ENXIO);
+ }
+ ctx->state = CTX_STATE_RUNNING;
+ HWT_CTX_UNLOCK(ctx);
+
+ if (ctx->mode == HWT_MODE_CPU)
+ hwt_vm_start_cpu_mode(ctx);
+ else {
+ /*
+ * Tracing backend will be configured and enabled
+ * during hook invocation. See hwt_hook.c.
+ */
+ }
+
+ break;
+
+ case HWT_IOC_STOP:
+ /* TODO */
+ break;
+
+ case HWT_IOC_RECORD_GET:
+ rget = (struct hwt_record_get *)addr;
+ error = hwt_record_send(ctx, rget);
+ if (error)
+ return (error);
+ break;
+
+ case HWT_IOC_SET_CONFIG:
+ sconf = (struct hwt_set_config *)addr;
+ error = hwt_config_set(td, ctx, sconf);
+ if (error)
+ return (error);
+ ctx->pause_on_mmap = sconf->pause_on_mmap ? 1 : 0;
+ break;
+
+ case HWT_IOC_WAKEUP:
+
+ if (ctx->mode == HWT_MODE_CPU)
+ return (ENXIO);
+
+ KASSERT(vm->thr != NULL, ("thr is NULL"));
+
+ wakeup(vm->thr);
+
+ break;
+
+ case HWT_IOC_BUFPTR_GET:
+ ptr_get = (struct hwt_bufptr_get *)addr;
+
+ if (ctx->mode == HWT_MODE_THREAD)
+ cpu_id = vm->thr->cpu_id;
+ else
+ cpu_id = vm->cpu->cpu_id;
+
+ error = hwt_backend_read(ctx, cpu_id, &curpage,
+ &curpage_offset);
+ if (error)
+ return (error);
+
+ error = copyout(&curpage, ptr_get->curpage, sizeof(int));
+ if (error)
+ return (error);
+ error = copyout(&curpage_offset, ptr_get->curpage_offset,
+ sizeof(vm_offset_t));
+ if (error)
+ return (error);
+ break;
+ default:
+ break;
+ }
+
+ return (0);
+}
+
+static struct cdevsw hwt_vm_cdevsw = {
+ .d_version = D_VERSION,
+ .d_name = "hwt",
+ .d_open = hwt_vm_open,
+ .d_mmap_single = hwt_vm_mmap_single,
+ .d_ioctl = hwt_vm_ioctl,
+};
+
+static int
+hwt_vm_create_cdev(struct hwt_vm *vm, char *path)
+{
+ struct make_dev_args args;
+ int error;
+
+ dprintf("%s: path %s\n", __func__, path);
+
+ make_dev_args_init(&args);
+ args.mda_devsw = &hwt_vm_cdevsw;
+ args.mda_flags = MAKEDEV_CHECKNAME | MAKEDEV_WAITOK;
+ args.mda_uid = UID_ROOT;
+ args.mda_gid = GID_WHEEL;
+ args.mda_mode = 0660;
+ args.mda_si_drv1 = vm;
+
+ error = make_dev_s(&args, &vm->cdev, "%s", path);
+ if (error != 0)
+ return (error);
+
+ return (0);
+}
+
+static int
+hwt_vm_alloc_buffers(struct hwt_vm *vm)
+{
+ int error;
+
+ vm->pages = malloc(sizeof(struct vm_page *) * vm->npages,
+ M_HWT_VM, M_WAITOK | M_ZERO);
+
+ error = hwt_vm_alloc_pages(vm);
+ if (error) {
+ printf("%s: could not alloc pages\n", __func__);
+ return (error);
+ }
+
+ return (0);
+}
+
+static void
+hwt_vm_destroy_buffers(struct hwt_vm *vm)
+{
+ vm_page_t m;
+ int i;
+
+ VM_OBJECT_WLOCK(vm->obj);
+ for (i = 0; i < vm->npages; i++) {
+ m = vm->pages[i];
+ if (m == NULL)
+ break;
+
+ vm_page_busy_acquire(m, 0);
+ cdev_pager_free_page(vm->obj, m);
+ m->flags &= ~PG_FICTITIOUS;
+ vm_page_unwire_noq(m);
+ vm_page_free(m);
+
+ }
+ vm_pager_deallocate(vm->obj);
+ VM_OBJECT_WUNLOCK(vm->obj);
+
+ free(vm->pages, M_HWT_VM);
+}
+
+void
+hwt_vm_free(struct hwt_vm *vm)
+{
+
+ if (vm->cdev)
+ destroy_dev_sched(vm->cdev);
+ hwt_vm_destroy_buffers(vm);
+ free(vm, M_HWT_VM);
+}
+
+int
+hwt_vm_alloc(size_t bufsize, char *path, struct hwt_vm **vm0)
+{
+ struct hwt_vm *vm;
+ int error;
+
+ vm = malloc(sizeof(struct hwt_vm), M_HWT_VM, M_WAITOK | M_ZERO);
+ vm->npages = bufsize / PAGE_SIZE;
+
+ error = hwt_vm_alloc_buffers(vm);
+ if (error) {
+ free(vm, M_HWT_VM);
+ return (error);
+ }
+
+ error = hwt_vm_create_cdev(vm, path);
+ if (error) {
+ hwt_vm_free(vm);
+ return (error);
+ }
+
+ *vm0 = vm;
+
+ return (0);
+}
Index: sys/kern/genoffset.c
===================================================================
--- sys/kern/genoffset.c
+++ sys/kern/genoffset.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2018, Matthew Macy <mmacy@freebsd.org>
*
Index: sys/kern/genoffset.sh
===================================================================
--- sys/kern/genoffset.sh
+++ sys/kern/genoffset.sh
@@ -1,6 +1,6 @@
#!/bin/sh
-# SPDX-License-Identifier: BSD-2-Clause
+# SPDX-License-Identifier: BSD-2-Clause-FreeBSD
#
# Copyright (c) 2000, Bruce Evans <bde@freebsd.org>
# Copyright (c) 2018, Jeff Roberson <jeff@freebsd.org>
Index: sys/kern/imgact_aout.c
===================================================================
--- sys/kern/imgact_aout.c
+++ sys/kern/imgact_aout.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1993, David Greenman
* All rights reserved.
Index: sys/kern/imgact_elf.c
===================================================================
--- sys/kern/imgact_elf.c
+++ sys/kern/imgact_elf.c
@@ -35,6 +35,7 @@
__FBSDID("$FreeBSD$");
#include "opt_capsicum.h"
+#include "opt_hwt_hooks.h"
#include <sys/param.h>
#include <sys/capsicum.h>
@@ -84,6 +85,10 @@
#include <vm/vm_object.h>
#include <vm/vm_extern.h>
+#ifdef HWT_HOOKS
+#include <dev/hwt/hwt_hook.h>
+#endif
+
#if __has_feature(capabilities)
#include <cheri/cheri.h>
#endif
@@ -1603,6 +1608,17 @@
imgp->interp_start = 0;
imgp->interp_end = 0;
+#ifdef HWT_HOOKS
+ /* HWT: record main binary. */
+ struct hwt_record_entry ent;
+ if (td->td_proc->p_flag2 & P2_HWT) {
+ ent.fullpath = imgp->execpath;
+ ent.addr = (uintptr_t)entry;
+ ent.record_type = HWT_RECORD_EXECUTABLE;
+ HWT_CALL_HOOK(td, HWT_EXEC, &ent);
+ }
+#endif
+
if (interp != NULL) {
VOP_UNLOCK(imgp->vp);
if ((map->flags & MAP_ASLR) != 0) {
@@ -1618,6 +1634,17 @@
vn_lock(imgp->vp, LK_SHARED | LK_RETRY);
if (error != 0)
goto ret;
+
+#ifdef HWT_HOOKS
+ /* HWT: Record interp. */
+ struct hwt_record_entry ent;
+ if (td->td_proc->p_flag2 & P2_HWT) {
+ ent.fullpath = interp;
+ ent.addr = (uintptr_t)imgp->entry_addr;
+ ent.record_type = HWT_RECORD_INTERP;
+ HWT_CALL_HOOK(td, HWT_EXEC, &ent);
+ }
+#endif
} else
addr = imgp->et_dyn_addr;
Index: sys/kern/imgact_elf32.c
===================================================================
--- sys/kern/imgact_elf32.c
+++ sys/kern/imgact_elf32.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Doug Rabson
* All rights reserved.
Index: sys/kern/imgact_elf64.c
===================================================================
--- sys/kern/imgact_elf64.c
+++ sys/kern/imgact_elf64.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Doug Rabson
* All rights reserved.
Index: sys/kern/imgact_shell.c
===================================================================
--- sys/kern/imgact_shell.c
+++ sys/kern/imgact_shell.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1993, David Greenman
* All rights reserved.
Index: sys/kern/init_main.c
===================================================================
--- sys/kern/init_main.c
+++ sys/kern/init_main.c
@@ -261,7 +261,6 @@
* Perform a bubble sort of the system initialization objects by
* their subsystem (primary key) and order (secondary key).
*/
- TSENTER2("bubblesort");
for (sipp = sysinit; sipp < sysinit_end; sipp++) {
for (xipp = sipp + 1; xipp < sysinit_end; xipp++) {
if ((*sipp)->subsystem < (*xipp)->subsystem ||
@@ -273,7 +272,6 @@
*xipp = save;
}
}
- TSEXIT2("bubblesort");
last = SI_SUB_COPYRIGHT;
#if defined(VERBOSE_SYSINIT)
Index: sys/kern/kern_alq.c
===================================================================
--- sys/kern/kern_alq.c
+++ sys/kern/kern_alq.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002, Jeffrey Roberson <jeff@freebsd.org>
* Copyright (c) 2008-2009, Lawrence Stewart <lstewart@freebsd.org>
Index: sys/kern/kern_boottrace.c
===================================================================
--- sys/kern/kern_boottrace.c
+++ sys/kern/kern_boottrace.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008-2022 NetApp, Inc.
*
Index: sys/kern/kern_clocksource.c
===================================================================
--- sys/kern/kern_clocksource.c
+++ sys/kern/kern_clocksource.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010-2013 Alexander Motin <mav@FreeBSD.org>
* All rights reserved.
Index: sys/kern/kern_condvar.c
===================================================================
--- sys/kern/kern_condvar.c
+++ sys/kern/kern_condvar.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Jake Burkholder <jake@freebsd.org>.
* All rights reserved.
Index: sys/kern/kern_conf.c
===================================================================
--- sys/kern/kern_conf.c
+++ sys/kern/kern_conf.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999-2002 Poul-Henning Kamp
* All rights reserved.
Index: sys/kern/kern_context.c
===================================================================
--- sys/kern/kern_context.c
+++ sys/kern/kern_context.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Daniel M. Eischen <deischen@freebsd.org>
* All rights reserved.
Index: sys/kern/kern_cpu.c
===================================================================
--- sys/kern/kern_cpu.c
+++ sys/kern/kern_cpu.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004-2007 Nate Lawson (SDG)
* All rights reserved.
Index: sys/kern/kern_cpuset.c
===================================================================
--- sys/kern/kern_cpuset.c
+++ sys/kern/kern_cpuset.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/kern/kern_ctf.c
===================================================================
--- sys/kern/kern_ctf.c
+++ sys/kern/kern_ctf.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 John Birrell <jb@freebsd.org>
* All rights reserved.
Index: sys/kern/kern_dtrace.c
===================================================================
--- sys/kern/kern_dtrace.c
+++ sys/kern/kern_dtrace.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007-2008 John Birrell <jb@FreeBSD.org>
* All rights reserved.
Index: sys/kern/kern_environment.c
===================================================================
--- sys/kern/kern_environment.c
+++ sys/kern/kern_environment.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1998 Michael Smith
* All rights reserved.
Index: sys/kern/kern_et.c
===================================================================
--- sys/kern/kern_et.c
+++ sys/kern/kern_et.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010-2013 Alexander Motin <mav@FreeBSD.org>
* All rights reserved.
Index: sys/kern/kern_event.c
===================================================================
--- sys/kern/kern_event.c
+++ sys/kern/kern_event.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999,2000,2001 Jonathan Lemon <jlemon@FreeBSD.org>
* Copyright 2004 John-Mark Gurney <jmg@FreeBSD.org>
Index: sys/kern/kern_exec.c
===================================================================
--- sys/kern/kern_exec.c
+++ sys/kern/kern_exec.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1993, David Greenman
* All rights reserved.
Index: sys/kern/kern_fail.c
===================================================================
--- sys/kern/kern_fail.c
+++ sys/kern/kern_fail.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009 Isilon Inc http://www.isilon.com/
*
Index: sys/kern/kern_ffclock.c
===================================================================
--- sys/kern/kern_ffclock.c
+++ sys/kern/kern_ffclock.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2011 The University of Melbourne
* All rights reserved.
Index: sys/kern/kern_fork.c
===================================================================
--- sys/kern/kern_fork.c
+++ sys/kern/kern_fork.c
@@ -1179,7 +1179,7 @@
#ifdef KTRACE
if (KTRPOINT(td, KTR_SYSRET))
- ktrsysret(td->td_sa.code, 0, 0);
+ ktrsysret(SYS_fork, 0, 0);
#endif
}
Index: sys/kern/kern_hhook.c
===================================================================
--- sys/kern/kern_hhook.c
+++ sys/kern/kern_hhook.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010,2013 Lawrence Stewart <lstewart@freebsd.org>
* Copyright (c) 2010 The FreeBSD Foundation
Index: sys/kern/kern_hwt.c
===================================================================
--- /dev/null
+++ sys/kern/kern_hwt.c
@@ -0,0 +1,46 @@
+/*-
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <sys/param.h>
+#include <sys/eventhandler.h>
+#include <sys/ioccom.h>
+#include <sys/conf.h>
+#include <sys/proc.h>
+#include <sys/kernel.h>
+#include <sys/malloc.h>
+#include <sys/mman.h>
+#include <sys/module.h>
+#include <sys/mutex.h>
+#include <sys/rwlock.h>
+#include <sys/hwt.h>
+
+#include <dev/hwt/hwt_hook.h>
+
+void __read_mostly (*hwt_hook)(struct thread *td, int func, void *arg) = NULL;
Index: sys/kern/kern_idle.c
===================================================================
--- sys/kern/kern_idle.c
+++ sys/kern/kern_idle.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2000-2004 The FreeBSD Project. All rights reserved.
*
Index: sys/kern/kern_intr.c
===================================================================
--- sys/kern/kern_intr.c
+++ sys/kern/kern_intr.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997, Stefan Esser <se@freebsd.org>
* All rights reserved.
Index: sys/kern/kern_jail.c
===================================================================
--- sys/kern/kern_jail.c
+++ sys/kern/kern_jail.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Poul-Henning Kamp.
* Copyright (c) 2008 Bjoern A. Zeeb.
Index: sys/kern/kern_kcov.c
===================================================================
--- sys/kern/kern_kcov.c
+++ sys/kern/kern_kcov.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2018 The FreeBSD Foundation. All rights reserved.
* Copyright (C) 2018, 2019 Andrew Turner
Index: sys/kern/kern_khelp.c
===================================================================
--- sys/kern/kern_khelp.c
+++ sys/kern/kern_khelp.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010,2013 Lawrence Stewart <lstewart@freebsd.org>
* Copyright (c) 2010 The FreeBSD Foundation
Index: sys/kern/kern_kthread.c
===================================================================
--- sys/kern/kern_kthread.c
+++ sys/kern/kern_kthread.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Peter Wemm <peter@FreeBSD.org>
* All rights reserved.
Index: sys/kern/kern_ktr.c
===================================================================
--- sys/kern/kern_ktr.c
+++ sys/kern/kern_ktr.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 John Baldwin <jhb@FreeBSD.org>
*
Index: sys/kern/kern_linker.c
===================================================================
--- sys/kern/kern_linker.c
+++ sys/kern/kern_linker.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997-2000 Doug Rabson
* All rights reserved.
@@ -32,6 +32,7 @@
#include "opt_ddb.h"
#include "opt_kld.h"
#include "opt_hwpmc_hooks.h"
+#include "opt_hwt_hooks.h"
#include <sys/param.h>
#include <sys/systm.h>
@@ -66,7 +67,7 @@
#include "linker_if.h"
-#ifdef HWPMC_HOOKS
+#if defined(HWPMC_HOOKS) || defined(HWT_HOOKS)
#include <sys/pmckern.h>
#endif
@@ -2131,7 +2132,7 @@
return (filename);
}
-#ifdef HWPMC_HOOKS
+#if defined(HWPMC_HOOKS) || defined(HWT_HOOKS)
/*
* Inform hwpmc about the set of kernel modules currently loaded.
*/
Index: sys/kern/kern_lock.c
===================================================================
--- sys/kern/kern_lock.c
+++ sys/kern/kern_lock.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Attilio Rao <attilio@FreeBSD.org>
* All rights reserved.
Index: sys/kern/kern_lockstat.c
===================================================================
--- sys/kern/kern_lockstat.c
+++ sys/kern/kern_lockstat.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2008-2009 Stacey Son <sson@FreeBSD.org>
*
Index: sys/kern/kern_loginclass.c
===================================================================
--- sys/kern/kern_loginclass.c
+++ sys/kern/kern_loginclass.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2011 The FreeBSD Foundation
*
Index: sys/kern/kern_mbuf.c
===================================================================
--- sys/kern/kern_mbuf.c
+++ sys/kern/kern_mbuf.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004, 2005,
* Bosko Milekic <bmilekic@FreeBSD.org>. All rights reserved.
Index: sys/kern/kern_module.c
===================================================================
--- sys/kern/kern_module.c
+++ sys/kern/kern_module.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997 Doug Rabson
* All rights reserved.
Index: sys/kern/kern_mtxpool.c
===================================================================
--- sys/kern/kern_mtxpool.c
+++ sys/kern/kern_mtxpool.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Matthew Dillon. All Rights Reserved.
*
Index: sys/kern/kern_osd.c
===================================================================
--- sys/kern/kern_osd.c
+++ sys/kern/kern_osd.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007 Pawel Jakub Dawidek <pjd@FreeBSD.org>
* All rights reserved.
Index: sys/kern/kern_pmc.c
===================================================================
--- sys/kern/kern_pmc.c
+++ sys/kern/kern_pmc.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003-2008 Joseph Koshy
* Copyright (c) 2007 The FreeBSD Foundation
Index: sys/kern/kern_poll.c
===================================================================
--- sys/kern/kern_poll.c
+++ sys/kern/kern_poll.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001-2002 Luigi Rizzo
*
Index: sys/kern/kern_priv.c
===================================================================
--- sys/kern/kern_priv.c
+++ sys/kern/kern_priv.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 nCircle Network Security, Inc.
* Copyright (c) 2009 Robert N. M. Watson
Index: sys/kern/kern_racct.c
===================================================================
--- sys/kern/kern_racct.c
+++ sys/kern/kern_racct.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010 The FreeBSD Foundation
*
Index: sys/kern/kern_rangelock.c
===================================================================
--- sys/kern/kern_rangelock.c
+++ sys/kern/kern_rangelock.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009 Konstantin Belousov <kib@FreeBSD.org>
* All rights reserved.
Index: sys/kern/kern_rctl.c
===================================================================
--- sys/kern/kern_rctl.c
+++ sys/kern/kern_rctl.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010 The FreeBSD Foundation
*
Index: sys/kern/kern_rwlock.c
===================================================================
--- sys/kern/kern_rwlock.c
+++ sys/kern/kern_rwlock.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 John Baldwin <jhb@FreeBSD.org>
*
Index: sys/kern/kern_sdt.c
===================================================================
--- sys/kern/kern_sdt.c
+++ sys/kern/kern_sdt.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2006-2008 John Birrell <jb@FreeBSD.org>
*
Index: sys/kern/kern_sema.c
===================================================================
--- sys/kern/kern_sema.c
+++ sys/kern/kern_sema.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2001 Jason Evans <jasone@freebsd.org>. All rights reserved.
*
Index: sys/kern/kern_sharedpage.c
===================================================================
--- sys/kern/kern_sharedpage.c
+++ sys/kern/kern_sharedpage.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010, 2012 Konstantin Belousov <kib@FreeBSD.org>
* Copyright (c) 2015 The FreeBSD Foundation
Index: sys/kern/kern_switch.c
===================================================================
--- sys/kern/kern_switch.c
+++ sys/kern/kern_switch.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Jake Burkholder <jake@FreeBSD.org>
* All rights reserved.
Index: sys/kern/kern_sx.c
===================================================================
--- sys/kern/kern_sx.c
+++ sys/kern/kern_sx.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007 Attilio Rao <attilio@freebsd.org>
* Copyright (c) 2001 Jason Evans <jasone@freebsd.org>
Index: sys/kern/kern_syscalls.c
===================================================================
--- sys/kern/kern_syscalls.c
+++ sys/kern/kern_syscalls.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Assar Westerlund
* All rights reserved.
Index: sys/kern/kern_thr.c
===================================================================
--- sys/kern/kern_thr.c
+++ sys/kern/kern_thr.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
@@ -31,6 +31,7 @@
#include "opt_posix.h"
#include "opt_hwpmc_hooks.h"
+#include "opt_hwt_hooks.h"
#include <sys/param.h>
#include <sys/kernel.h>
#include <sys/lock.h>
@@ -59,6 +60,9 @@
#ifdef HWPMC_HOOKS
#include <sys/pmckern.h>
#endif
+#ifdef HWT_HOOKS
+#include <dev/hwt/hwt_hook.h>
+#endif
#include <machine/frame.h>
@@ -269,6 +273,10 @@
PMC_CALL_HOOK_UNLOCKED(newtd, PMC_FN_THR_CREATE_LOG, NULL);
#endif
+#ifdef HWT_HOOKS
+ HWT_CALL_HOOK(newtd, HWT_THREAD_CREATE, NULL);
+#endif
+
tidhash_add(newtd);
/* ignore timesharing class */
@@ -611,6 +619,9 @@
if (PMC_PROC_IS_USING_PMCS(p) || PMC_SYSTEM_SAMPLING_ACTIVE())
PMC_CALL_HOOK_UNLOCKED(ttd, PMC_FN_THR_CREATE_LOG, NULL);
#endif
+#ifdef HWT_HOOKS
+ HWT_CALL_HOOK(ttd, HWT_THREAD_SET_NAME, NULL);
+#endif
#ifdef KTR
sched_clear_tdname(ttd);
#endif
Index: sys/kern/kern_thread.c
===================================================================
--- sys/kern/kern_thread.c
+++ sys/kern/kern_thread.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2001 Julian Elischer <julian@freebsd.org>.
* All rights reserved.
@@ -30,6 +30,7 @@
#include "opt_witness.h"
#include "opt_hwpmc_hooks.h"
+#include "opt_hwt_hooks.h"
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
@@ -63,6 +64,9 @@
#ifdef HWPMC_HOOKS
#include <sys/pmckern.h>
#endif
+#ifdef HWT_HOOKS
+#include <dev/hwt/hwt_hook.h>
+#endif
#include <sys/priv.h>
#include <security/audit/audit.h>
@@ -89,7 +93,7 @@
"struct thread KBI td_flags");
_Static_assert(offsetof(struct thread, td_pflags) == 0x114,
"struct thread KBI td_pflags");
-_Static_assert(offsetof(struct thread, td_frame) == 0x4b8,
+_Static_assert(offsetof(struct thread, td_frame) == 0x4b0,
"struct thread KBI td_frame");
_Static_assert(offsetof(struct thread, td_emuldata) == 0x6c0,
"struct thread KBI td_emuldata");
@@ -109,9 +113,9 @@
"struct thread KBI td_flags");
_Static_assert(offsetof(struct thread, td_pflags) == 0xa8,
"struct thread KBI td_pflags");
-_Static_assert(offsetof(struct thread, td_frame) == 0x314,
+_Static_assert(offsetof(struct thread, td_frame) == 0x30c,
"struct thread KBI td_frame");
-_Static_assert(offsetof(struct thread, td_emuldata) == 0x358,
+_Static_assert(offsetof(struct thread, td_emuldata) == 0x350,
"struct thread KBI td_emuldata");
_Static_assert(offsetof(struct proc, p_flag) == 0x6c,
"struct proc KBI p_flag");
@@ -971,6 +975,11 @@
} else if (PMC_SYSTEM_SAMPLING_ACTIVE())
PMC_CALL_HOOK_UNLOCKED(td, PMC_FN_THR_EXIT_LOG, NULL);
#endif
+
+#ifdef HWT_HOOKS
+ HWT_CALL_HOOK(td, HWT_THREAD_EXIT, NULL);
+#endif
+
PROC_UNLOCK(p);
PROC_STATLOCK(p);
thread_lock(td);
Index: sys/kern/kern_umtx.c
===================================================================
--- sys/kern/kern_umtx.c
+++ sys/kern/kern_umtx.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2015, 2016 The FreeBSD Foundation
* Copyright (c) 2004, David Xu <davidxu@freebsd.org>
Index: sys/kern/kern_uuid.c
===================================================================
--- sys/kern/kern_uuid.c
+++ sys/kern/kern_uuid.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Marcel Moolenaar
* All rights reserved.
Index: sys/kern/link_elf.c
===================================================================
--- sys/kern/link_elf.c
+++ sys/kern/link_elf.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1998-2000 Doug Rabson
* All rights reserved.
Index: sys/kern/link_elf_obj.c
===================================================================
--- sys/kern/link_elf_obj.c
+++ sys/kern/link_elf_obj.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1998-2000 Doug Rabson
* Copyright (c) 2004 Peter Wemm
Index: sys/kern/sched_ule.c
===================================================================
--- sys/kern/sched_ule.c
+++ sys/kern/sched_ule.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002-2007, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
@@ -41,6 +41,7 @@
__FBSDID("$FreeBSD$");
#include "opt_hwpmc_hooks.h"
+#include "opt_hwt_hooks.h"
#include "opt_sched.h"
#include <sys/param.h>
@@ -70,6 +71,10 @@
#include <sys/pmckern.h>
#endif
+#ifdef HWT_HOOKS
+#include <dev/hwt/hwt_hook.h>
+#endif
+
#ifdef KDTRACE_HOOKS
#include <sys/dtrace_bsd.h>
int __read_mostly dtrace_vtime_active;
@@ -2294,6 +2299,12 @@
if (dtrace_vtime_active)
(*dtrace_vtime_switch_func)(newtd);
#endif
+
+#ifdef HWT_HOOKS
+ HWT_CALL_HOOK(td, HWT_SWITCH_OUT, NULL);
+ HWT_CALL_HOOK(newtd, HWT_SWITCH_IN, NULL);
+#endif
+
td->td_oncpu = NOCPU;
cpu_switch(td, newtd, mtx);
cpuid = td->td_oncpu = PCPU_GET(cpuid);
@@ -3123,6 +3134,10 @@
newtd = sched_throw_grab(tdq);
+#ifdef HWT_HOOKS
+ HWT_CALL_HOOK(newtd, HWT_SWITCH_IN, NULL);
+#endif
+
/* doesn't return */
cpu_throw(NULL, newtd);
}
@@ -3149,6 +3164,10 @@
newtd = sched_throw_grab(tdq);
+#ifdef HWT_HOOKS
+ HWT_CALL_HOOK(newtd, HWT_SWITCH_IN, NULL);
+#endif
+
/* doesn't return */
cpu_switch(td, newtd, TDQ_LOCKPTR(tdq));
}
Index: sys/kern/subr_acl_nfs4.c
===================================================================
--- sys/kern/subr_acl_nfs4.c
+++ sys/kern/subr_acl_nfs4.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008-2010 Edward Tomasz Napierała <trasz@FreeBSD.org>
*
Index: sys/kern/subr_acl_posix1e.c
===================================================================
--- sys/kern/subr_acl_posix1e.c
+++ sys/kern/subr_acl_posix1e.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999-2006 Robert N. M. Watson
* All rights reserved.
Index: sys/kern/subr_atomic64.c
===================================================================
--- sys/kern/subr_atomic64.c
+++ sys/kern/subr_atomic64.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019 Justin Hibbits
*
Index: sys/kern/subr_boot.c
===================================================================
--- sys/kern/subr_boot.c
+++ sys/kern/subr_boot.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1998 Michael Smith <msmith@freebsd.org>
* All Rights Reserved.
Index: sys/kern/subr_bufring.c
===================================================================
--- sys/kern/subr_bufring.c
+++ sys/kern/subr_bufring.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007, 2008 Kip Macy <kmacy@freebsd.org>
* All rights reserved.
Index: sys/kern/subr_bus.c
===================================================================
--- sys/kern/subr_bus.c
+++ sys/kern/subr_bus.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997,1998,2003 Doug Rabson
* All rights reserved.
Index: sys/kern/subr_bus_dma.c
===================================================================
--- sys/kern/subr_bus_dma.c
+++ sys/kern/subr_bus_dma.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2012 EMC Corp.
* All rights reserved.
Index: sys/kern/subr_busdma_bounce.c
===================================================================
--- sys/kern/subr_busdma_bounce.c
+++ sys/kern/subr_busdma_bounce.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997, 1998 Justin T. Gibbs.
* All rights reserved.
Index: sys/kern/subr_busdma_bufalloc.c
===================================================================
--- sys/kern/subr_busdma_bufalloc.c
+++ sys/kern/subr_busdma_bufalloc.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2012 Ian Lepore
* All rights reserved.
Index: sys/kern/subr_capability.c
===================================================================
--- sys/kern/subr_capability.c
+++ sys/kern/subr_capability.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 FreeBSD Foundation
*
Index: sys/kern/subr_compressor.c
===================================================================
--- sys/kern/subr_compressor.c
+++ sys/kern/subr_compressor.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014, 2017 Mark Johnston <markj@FreeBSD.org>
* Copyright (c) 2017 Conrad Meyer <cem@FreeBSD.org>
Index: sys/kern/subr_counter.c
===================================================================
--- sys/kern/subr_counter.c
+++ sys/kern/subr_counter.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2012 Gleb Smirnoff <glebius@FreeBSD.org>
* All rights reserved.
Index: sys/kern/subr_coverage.c
===================================================================
--- sys/kern/subr_coverage.c
+++ sys/kern/subr_coverage.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2018 The FreeBSD Foundation. All rights reserved.
* Copyright (C) 2018, 2019 Andrew Turner
Index: sys/kern/subr_dummy_vdso_tc.c
===================================================================
--- sys/kern/subr_dummy_vdso_tc.c
+++ sys/kern/subr_dummy_vdso_tc.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2012 Konstantin Belousov <kib@FreeBSD.ORG>.
* All rights reserved.
Index: sys/kern/subr_epoch.c
===================================================================
--- sys/kern/subr_epoch.c
+++ sys/kern/subr_epoch.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2018, Matthew Macy <mmacy@freebsd.org>
*
Index: sys/kern/subr_eventhandler.c
===================================================================
--- sys/kern/subr_eventhandler.c
+++ sys/kern/subr_eventhandler.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Michael Smith <msmith@freebsd.org>
* All rights reserved.
Index: sys/kern/subr_fattime.c
===================================================================
--- sys/kern/subr_fattime.c
+++ sys/kern/subr_fattime.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 Poul-Henning Kamp
* All rights reserved.
Index: sys/kern/subr_firmware.c
===================================================================
--- sys/kern/subr_firmware.c
+++ sys/kern/subr_firmware.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005-2008, Sam Leffler <sam@errno.com>
* All rights reserved.
Index: sys/kern/subr_hints.c
===================================================================
--- sys/kern/subr_hints.c
+++ sys/kern/subr_hints.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000,2001 Peter Wemm <peter@FreeBSD.org>
* All rights reserved.
Index: sys/kern/subr_kdb.c
===================================================================
--- sys/kern/subr_kdb.c
+++ sys/kern/subr_kdb.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 The FreeBSD Project
* All rights reserved.
Index: sys/kern/subr_kobj.c
===================================================================
--- sys/kern/subr_kobj.c
+++ sys/kern/subr_kobj.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000,2003 Doug Rabson
* All rights reserved.
Index: sys/kern/subr_lock.c
===================================================================
--- sys/kern/subr_lock.c
+++ sys/kern/subr_lock.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 John Baldwin <jhb@FreeBSD.org>
*
@@ -121,14 +121,14 @@
lock_delay(struct lock_delay_arg *la)
{
struct lock_delay_config *lc = la->config;
- u_int i;
+ u_short i;
for (i = la->delay; i > 0; i--)
cpu_spinwait();
la->spin_cnt += la->delay;
la->delay <<= 1;
- if (__predict_false(la->delay > (u_int)lc->max))
+ if (__predict_false(la->delay > lc->max))
la->delay = lc->max;
}
@@ -148,7 +148,9 @@
{
lc->base = 1;
- lc->max = min(lock_roundup_2(mp_ncpus) * 256, SHRT_MAX);
+ lc->max = lock_roundup_2(mp_ncpus) * 256;
+ if (lc->max > 32678)
+ lc->max = 32678;
}
struct lock_delay_config __read_frequently locks_delay;
Index: sys/kern/subr_module.c
===================================================================
--- sys/kern/subr_module.c
+++ sys/kern/subr_module.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1998 Michael Smith
* All rights reserved.
Index: sys/kern/subr_msgbuf.c
===================================================================
--- sys/kern/subr_msgbuf.c
+++ sys/kern/subr_msgbuf.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003 Ian Dowse. All rights reserved.
*
Index: sys/kern/subr_pctrie.c
===================================================================
--- sys/kern/subr_pctrie.c
+++ sys/kern/subr_pctrie.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 EMC Corp.
* Copyright (c) 2011 Jeffrey Roberson <jeff@freebsd.org>
Index: sys/kern/subr_physmem.c
===================================================================
--- sys/kern/subr_physmem.c
+++ sys/kern/subr_physmem.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014 Ian Lepore <ian@freebsd.org>
* All rights reserved.
Index: sys/kern/subr_pidctrl.c
===================================================================
--- sys/kern/subr_pidctrl.c
+++ sys/kern/subr_pidctrl.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/kern/subr_power.c
===================================================================
--- sys/kern/subr_power.c
+++ sys/kern/subr_power.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Mitsuru IWASAKI
* All rights reserved.
Index: sys/kern/subr_prng.c
===================================================================
--- sys/kern/subr_prng.c
+++ sys/kern/subr_prng.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2020 Conrad Meyer <cem@FreeBSD.org>. All rights reserved.
*
Index: sys/kern/subr_rangeset.c
===================================================================
--- sys/kern/subr_rangeset.c
+++ sys/kern/subr_rangeset.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019 The FreeBSD Foundation
*
Index: sys/kern/subr_sbuf.c
===================================================================
--- sys/kern/subr_sbuf.c
+++ sys/kern/subr_sbuf.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000-2008 Poul-Henning Kamp
* Copyright (c) 2000-2008 Dag-Erling Coïdan Smørgrav
Index: sys/kern/subr_sleepqueue.c
===================================================================
--- sys/kern/subr_sleepqueue.c
+++ sys/kern/subr_sleepqueue.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 John Baldwin <jhb@FreeBSD.org>
*
Index: sys/kern/subr_smp.c
===================================================================
--- sys/kern/subr_smp.c
+++ sys/kern/subr_smp.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001, John Baldwin <jhb@FreeBSD.org>.
*
@@ -630,6 +630,8 @@
smp_rendezvous_cpus(all_cpus, setup_func, action_func, teardown_func, arg);
}
+static struct cpu_group group[MAXCPU * MAX_CACHE_LEVELS + 1];
+
static void
smp_topo_fill(struct cpu_group *cg)
{
@@ -645,14 +647,7 @@
smp_topo(void)
{
char cpusetbuf[CPUSETBUFSIZ], cpusetbuf2[CPUSETBUFSIZ];
- static struct cpu_group *top = NULL;
-
- /*
- * The first call to smp_topo() is guaranteed to occur
- * during the kernel boot while we are still single-threaded.
- */
- if (top != NULL)
- return (top);
+ struct cpu_group *top;
/*
* Check for a fake topology request for debugging purposes.
@@ -718,14 +713,9 @@
struct cpu_group *
smp_topo_alloc(u_int count)
{
- static struct cpu_group *group = NULL;
static u_int index;
u_int curr;
- if (group == NULL) {
- group = mallocarray((mp_maxid + 1) * MAX_CACHE_LEVELS + 1,
- sizeof(*group), M_DEVBUF, M_WAITOK | M_ZERO);
- }
curr = index;
index += count;
return (&group[curr]);
@@ -736,7 +726,7 @@
{
struct cpu_group *top;
- top = smp_topo_alloc(1);
+ top = &group[0];
top->cg_parent = NULL;
top->cg_child = NULL;
top->cg_mask = all_cpus;
@@ -790,9 +780,9 @@
int i;
cpu = 0;
+ top = &group[0];
packages = mp_ncpus / count;
- top = smp_topo_alloc(1 + packages);
- top->cg_child = child = top + 1;
+ top->cg_child = child = &group[1];
top->cg_level = CG_SHARE_NONE;
for (i = 0; i < packages; i++, child++)
cpu = smp_topo_addleaf(top, child, share, count, flags, cpu);
@@ -811,9 +801,8 @@
int j;
cpu = 0;
- top = smp_topo_alloc(1 + mp_ncpus / (l2count * l1count) +
- mp_ncpus / l1count);
- l2g = top + 1;
+ top = &group[0];
+ l2g = &group[1];
top->cg_child = l2g;
top->cg_level = CG_SHARE_NONE;
top->cg_children = mp_ncpus / (l2count * l1count);
@@ -986,8 +975,7 @@
error = 0;
if ((prio & PDROP) == 0) {
- gen = mallocarray(sizeof(u_int), mp_maxid + 1, M_TEMP,
- M_WAITOK);
+ gen = malloc(sizeof(u_int) * MAXCPU, M_TEMP, M_WAITOK);
for (cpu = 0; cpu <= mp_maxid; cpu++) {
if (!CPU_ISSET(cpu, &map) || CPU_ABSENT(cpu))
continue;
Index: sys/kern/subr_smr.c
===================================================================
--- sys/kern/subr_smr.c
+++ sys/kern/subr_smr.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019,2020 Jeffrey Roberson <jeff@FreeBSD.org>
*
Index: sys/kern/subr_stack.c
===================================================================
--- sys/kern/subr_stack.c
+++ sys/kern/subr_stack.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 Antoine Brodin
* All rights reserved.
Index: sys/kern/subr_taskqueue.c
===================================================================
--- sys/kern/subr_taskqueue.c
+++ sys/kern/subr_taskqueue.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Doug Rabson
* All rights reserved.
Index: sys/kern/subr_terminal.c
===================================================================
--- sys/kern/subr_terminal.c
+++ sys/kern/subr_terminal.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009 The FreeBSD Foundation
*
Index: sys/kern/subr_unit.c
===================================================================
--- sys/kern/subr_unit.c
+++ sys/kern/subr_unit.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 Poul-Henning Kamp
* All rights reserved.
Index: sys/kern/subr_vmem.c
===================================================================
--- sys/kern/subr_vmem.c
+++ sys/kern/subr_vmem.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c)2006,2007,2008,2009 YAMAMOTO Takashi,
* Copyright (c) 2013 EMC Corp.
Index: sys/kern/sys_capability.c
===================================================================
--- sys/kern/sys_capability.c
+++ sys/kern/sys_capability.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008-2011 Robert N. M. Watson
* Copyright (c) 2010-2011 Jonathan Anderson
Index: sys/kern/sys_eventfd.c
===================================================================
--- sys/kern/sys_eventfd.c
+++ sys/kern/sys_eventfd.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014 Dmitry Chagin <dchagin@FreeBSD.org>
*
Index: sys/kern/sys_getrandom.c
===================================================================
--- sys/kern/sys_getrandom.c
+++ sys/kern/sys_getrandom.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2018 Conrad Meyer <cem@FreeBSD.org>
* All rights reserved.
Index: sys/kern/sys_procdesc.c
===================================================================
--- sys/kern/sys_procdesc.c
+++ sys/kern/sys_procdesc.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009, 2016 Robert N. M. Watson
* All rights reserved.
Index: sys/kern/sysv_msg.c
===================================================================
--- sys/kern/sysv_msg.c
+++ sys/kern/sysv_msg.c
@@ -17,7 +17,7 @@
* This software is provided ``AS IS'' without any warranties of any kind.
*/
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003-2005 McAfee, Inc.
* Copyright (c) 2016-2017 Robert N. M. Watson
Index: sys/kern/sysv_sem.c
===================================================================
--- sys/kern/sysv_sem.c
+++ sys/kern/sysv_sem.c
@@ -6,7 +6,7 @@
* This software is provided ``AS IS'' without any warranties of any kind.
*/
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003-2005 McAfee, Inc.
* Copyright (c) 2016-2017 Robert N. M. Watson
Index: sys/kern/sysv_shm.c
===================================================================
--- sys/kern/sysv_shm.c
+++ sys/kern/sysv_shm.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-4-Clause AND BSD-2-Clause
+ * SPDX-License-Identifier: BSD-4-Clause AND BSD-2-Clause-FreeBSD
*
* Copyright (c) 1994 Adam Glass and Charles Hannum. All rights reserved.
*
Index: sys/kern/tty.c
===================================================================
--- sys/kern/tty.c
+++ sys/kern/tty.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/kern/tty_inq.c
===================================================================
--- sys/kern/tty_inq.c
+++ sys/kern/tty_inq.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/kern/tty_outq.c
===================================================================
--- sys/kern/tty_outq.c
+++ sys/kern/tty_outq.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/kern/tty_pts.c
===================================================================
--- sys/kern/tty_pts.c
+++ sys/kern/tty_pts.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/kern/tty_tty.c
===================================================================
--- sys/kern/tty_tty.c
+++ sys/kern/tty_tty.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003 Poul-Henning Kamp. All rights reserved.
*
Index: sys/kern/tty_ttydisc.c
===================================================================
--- sys/kern/tty_ttydisc.c
+++ sys/kern/tty_ttydisc.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/kern/uipc_accf.c
===================================================================
--- sys/kern/uipc_accf.c
+++ sys/kern/uipc_accf.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Paycounter, Inc.
* Copyright (c) 2005 Robert N. M. Watson
Index: sys/kern/uipc_debug.c
===================================================================
--- sys/kern/uipc_debug.c
+++ sys/kern/uipc_debug.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007 Robert N. M. Watson
* All rights reserved.
Index: sys/kern/uipc_ktls.c
===================================================================
--- sys/kern/uipc_ktls.c
+++ sys/kern/uipc_ktls.c
@@ -89,9 +89,9 @@
int lastallocfail;
} __aligned(CACHE_LINE_SIZE);
-struct ktls_reclaim_thread {
+struct ktls_alloc_thread {
uint64_t wakeups;
- uint64_t reclaims;
+ uint64_t allocs;
struct thread *td;
int running;
};
@@ -99,7 +99,7 @@
struct ktls_domain_info {
int count;
int cpu[MAXCPU];
- struct ktls_reclaim_thread reclaim_td;
+ struct ktls_alloc_thread alloc_td;
};
struct ktls_domain_info ktls_domains[MAXMEMDOM];
@@ -155,10 +155,10 @@
&ktls_sw_buffer_cache, 1,
"Enable caching of output buffers for SW encryption");
-static int ktls_max_reclaim = 1024;
-SYSCTL_INT(_kern_ipc_tls, OID_AUTO, max_reclaim, CTLFLAG_RWTUN,
- &ktls_max_reclaim, 128,
- "Max number of 16k buffers to reclaim in thread context");
+static int ktls_max_alloc = 128;
+SYSCTL_INT(_kern_ipc_tls, OID_AUTO, max_alloc, CTLFLAG_RWTUN,
+ &ktls_max_alloc, 128,
+ "Max number of 16k buffers to allocate in thread context");
static COUNTER_U64_DEFINE_EARLY(ktls_tasks_active);
SYSCTL_COUNTER_U64(_kern_ipc_tls, OID_AUTO, tasks_active, CTLFLAG_RD,
@@ -304,7 +304,7 @@
static void ktls_reset_receive_tag(void *context, int pending);
static void ktls_reset_send_tag(void *context, int pending);
static void ktls_work_thread(void *ctx);
-static void ktls_reclaim_thread(void *ctx);
+static void ktls_alloc_thread(void *ctx);
static u_int
ktls_get_cpu(struct socket *so)
@@ -459,12 +459,12 @@
continue;
if (CPU_EMPTY(&cpuset_domain[domain]))
continue;
- error = kproc_kthread_add(ktls_reclaim_thread,
+ error = kproc_kthread_add(ktls_alloc_thread,
&ktls_domains[domain], &ktls_proc,
- &ktls_domains[domain].reclaim_td.td,
- 0, 0, "KTLS", "reclaim_%d", domain);
+ &ktls_domains[domain].alloc_td.td,
+ 0, 0, "KTLS", "alloc_%d", domain);
if (error) {
- printf("Can't add KTLS reclaim thread %d error %d\n",
+ printf("Can't add KTLS alloc thread %d error %d\n",
domain, error);
return (error);
}
@@ -1720,12 +1720,10 @@
CURVNET_SET(inp->inp_vnet);
tp = tcp_drop(tp, ECONNABORTED);
CURVNET_RESTORE();
- if (tp != NULL) {
+ if (tp != NULL)
counter_u64_add(ktls_ifnet_reset_dropped, 1);
- INP_WUNLOCK(inp);
- }
- } else
- INP_WUNLOCK(inp);
+ }
+ INP_WUNLOCK(inp);
NET_EPOCH_EXIT(et);
counter_u64_add(ktls_ifnet_reset_failed, 1);
@@ -2709,9 +2707,9 @@
* see an old value of running == true.
*/
if (!VM_DOMAIN_EMPTY(domain)) {
- running = atomic_load_int(&ktls_domains[domain].reclaim_td.running);
+ running = atomic_load_int(&ktls_domains[domain].alloc_td.running);
if (!running)
- wakeup(&ktls_domains[domain].reclaim_td);
+ wakeup(&ktls_domains[domain].alloc_td);
}
}
return (buf);
@@ -3126,51 +3124,65 @@
}
static void
-ktls_reclaim_thread(void *ctx)
+ktls_alloc_thread(void *ctx)
{
struct ktls_domain_info *ktls_domain = ctx;
- struct ktls_reclaim_thread *sc = &ktls_domain->reclaim_td;
+ struct ktls_alloc_thread *sc = &ktls_domain->alloc_td;
+ void **buf;
struct sysctl_oid *oid;
char name[80];
- int error, domain;
+ int domain, error, i, nbufs;
domain = ktls_domain - ktls_domains;
if (bootverbose)
- printf("Starting KTLS reclaim thread for domain %d\n", domain);
+ printf("Starting KTLS alloc thread for domain %d\n", domain);
error = ktls_bind_domain(domain);
if (error)
- printf("Unable to bind KTLS reclaim thread for domain %d: error %d\n",
+ printf("Unable to bind KTLS alloc thread for domain %d: error %d\n",
domain, error);
snprintf(name, sizeof(name), "domain%d", domain);
oid = SYSCTL_ADD_NODE(NULL, SYSCTL_STATIC_CHILDREN(_kern_ipc_tls), OID_AUTO,
name, CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, "");
- SYSCTL_ADD_U64(NULL, SYSCTL_CHILDREN(oid), OID_AUTO, "reclaims",
- CTLFLAG_RD, &sc->reclaims, 0, "buffers reclaimed");
+ SYSCTL_ADD_U64(NULL, SYSCTL_CHILDREN(oid), OID_AUTO, "allocs",
+ CTLFLAG_RD, &sc->allocs, 0, "buffers allocated");
SYSCTL_ADD_U64(NULL, SYSCTL_CHILDREN(oid), OID_AUTO, "wakeups",
CTLFLAG_RD, &sc->wakeups, 0, "thread wakeups");
SYSCTL_ADD_INT(NULL, SYSCTL_CHILDREN(oid), OID_AUTO, "running",
CTLFLAG_RD, &sc->running, 0, "thread running");
+ buf = NULL;
+ nbufs = 0;
for (;;) {
atomic_store_int(&sc->running, 0);
tsleep(sc, PZERO | PNOLOCK, "-", 0);
atomic_store_int(&sc->running, 1);
sc->wakeups++;
+ if (nbufs != ktls_max_alloc) {
+ free(buf, M_KTLS);
+ nbufs = atomic_load_int(&ktls_max_alloc);
+ buf = malloc(sizeof(void *) * nbufs, M_KTLS,
+ M_WAITOK | M_ZERO);
+ }
/*
- * Below we attempt to reclaim ktls_max_reclaim
- * buffers using vm_page_reclaim_contig_domain_ext().
- * We do this here, as this function can take several
- * seconds to scan all of memory and it does not
- * matter if this thread pauses for a while. If we
- * block a ktls worker thread, we risk developing
- * backlogs of buffers to be encrypted, leading to
- * surges of traffic and potential NIC output drops.
+ * Below we allocate nbufs with different allocation
+ * flags than we use when allocating normally during
+ * encryption in the ktls worker thread. We specify
+ * M_NORECLAIM in the worker thread. However, we omit
+ * that flag here and add M_WAITOK so that the VM
+ * system is permitted to perform expensive work to
+ * defragment memory. We do this here, as it does not
+ * matter if this thread blocks. If we block a ktls
+ * worker thread, we risk developing backlogs of
+ * buffers to be encrypted, leading to surges of
+ * traffic and potential NIC output drops.
*/
- if (!vm_page_reclaim_contig_domain_ext(domain, VM_ALLOC_NORMAL,
- atop(ktls_maxlen), 0, ~0ul, PAGE_SIZE, 0, ktls_max_reclaim)) {
- vm_wait_domain(domain);
- } else {
- sc->reclaims += ktls_max_reclaim;
+ for (i = 0; i < nbufs; i++) {
+ buf[i] = uma_zalloc(ktls_buffer_zone, M_WAITOK);
+ sc->allocs++;
+ }
+ for (i = 0; i < nbufs; i++) {
+ uma_zfree(ktls_buffer_zone, buf[i]);
+ buf[i] = NULL;
}
}
}
Index: sys/kern/uipc_mqueue.c
===================================================================
--- sys/kern/uipc_mqueue.c
+++ sys/kern/uipc_mqueue.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 David Xu <davidxu@freebsd.org>
* Copyright (c) 2016-2017 Robert N. M. Watson
Index: sys/kern/uipc_sem.c
===================================================================
--- sys/kern/uipc_sem.c
+++ sys/kern/uipc_sem.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Alfred Perlstein <alfred@FreeBSD.org>
* Copyright (c) 2003-2005 SPARTA, Inc.
Index: sys/kern/uipc_shm.c
===================================================================
--- sys/kern/uipc_shm.c
+++ sys/kern/uipc_shm.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006, 2011, 2016-2017 Robert N. M. Watson
* Copyright 2020 The FreeBSD Foundation
Index: sys/kern/vfs_acl.c
===================================================================
--- sys/kern/vfs_acl.c
+++ sys/kern/vfs_acl.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999-2006, 2016-2017 Robert N. M. Watson
* All rights reserved.
Index: sys/kern/vfs_aio.c
===================================================================
--- sys/kern/vfs_aio.c
+++ sys/kern/vfs_aio.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997 John S. Dyson. All rights reserved.
*
Index: sys/kern/vfs_bio.c
===================================================================
--- sys/kern/vfs_bio.c
+++ sys/kern/vfs_bio.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 Poul-Henning Kamp
* Copyright (c) 1994,1997 John S. Dyson
Index: sys/kern/vfs_extattr.c
===================================================================
--- sys/kern/vfs_extattr.c
+++ sys/kern/vfs_extattr.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999-2001 Robert N. M. Watson
* All rights reserved.
Index: sys/kern/vfs_hash.c
===================================================================
--- sys/kern/vfs_hash.c
+++ sys/kern/vfs_hash.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 Poul-Henning Kamp
* All rights reserved.
Index: sys/kern/vfs_vnops.c
===================================================================
--- sys/kern/vfs_vnops.c
+++ sys/kern/vfs_vnops.c
@@ -46,6 +46,7 @@
__FBSDID("$FreeBSD$");
#include "opt_hwpmc_hooks.h"
+#include "opt_hwt_hooks.h"
#include <sys/param.h>
#include <sys/systm.h>
@@ -97,6 +98,10 @@
#include <sys/pmckern.h>
#endif
+#ifdef HWT_HOOKS
+#include <dev/hwt/hwt_hook.h>
+#endif
+
static fo_rdwr_t vn_read;
static fo_rdwr_t vn_write;
static fo_rdwr_t vn_io_fault;
@@ -2922,6 +2927,25 @@
}
}
#endif
+
+#ifdef HWT_HOOKS
+ /* HWT: record dynamic libs. */
+ struct hwt_record_entry ent;
+ char *fullpath;
+ char *freepath;
+
+ if ((prot & VM_PROT_EXECUTE) != 0 && error == 0) {
+ if (vn_fullpath(vp, &fullpath, &freepath) == 0) {
+ ent.fullpath = fullpath;
+ ent.addr = (uintptr_t) *addr;
+ ent.record_type = HWT_RECORD_MMAP;
+ HWT_CALL_HOOK(td, HWT_MMAP, &ent);
+ if (freepath != NULL)
+ free(freepath, M_TEMP);
+ }
+ }
+#endif
+
return (error);
}
Index: sys/modules/hwt/Makefile
===================================================================
--- /dev/null
+++ sys/modules/hwt/Makefile
@@ -0,0 +1,23 @@
+# $FreeBSD$
+
+.PATH: ${SRCTOP}/sys/dev/hwt
+
+KMOD = hwt
+SRCS = \
+ hwt.c \
+ hwt_backend.c \
+ hwt_config.c \
+ hwt_context.c \
+ hwt_contexthash.c \
+ hwt_cpu.c \
+ hwt_hook.c \
+ hwt_ioctl.c \
+ hwt_owner.c \
+ hwt_ownerhash.c \
+ hwt_record.c \
+ hwt_thread.c \
+ hwt_vm.c
+
+# EXPORT_SYMS= hwt_backend_register
+
+.include <bsd.kmod.mk>
Index: sys/sys/_atomic64e.h
===================================================================
--- sys/sys/_atomic64e.h
+++ sys/sys/_atomic64e.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019 Justin Hibbits
*
Index: sys/sys/_atomic_subword.h
===================================================================
--- sys/sys/_atomic_subword.h
+++ sys/sys/_atomic_subword.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019 Kyle Evans <kevans@FreeBSD.org>
*
Index: sys/sys/_bitset.h
===================================================================
--- sys/sys/_bitset.h
+++ sys/sys/_bitset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/_blockcount.h
===================================================================
--- sys/sys/_blockcount.h
+++ sys/sys/_blockcount.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2020 The FreeBSD Foundation
*
Index: sys/sys/_bus_dma.h
===================================================================
--- sys/sys/_bus_dma.h
+++ sys/sys/_bus_dma.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2006 John-Mark Gurney.
* All rights reserved.
Index: sys/sys/_cpuset.h
===================================================================
--- sys/sys/_cpuset.h
+++ sys/sys/_cpuset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/_domainset.h
===================================================================
--- sys/sys/_domainset.h
+++ sys/sys/_domainset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/_eventhandler.h
===================================================================
--- sys/sys/_eventhandler.h
+++ sys/sys/_eventhandler.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Michael Smith <msmith@freebsd.org>
* All rights reserved.
Index: sys/sys/_ffcounter.h
===================================================================
--- sys/sys/_ffcounter.h
+++ sys/sys/_ffcounter.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2011 The University of Melbourne
* All rights reserved.
Index: sys/sys/_lockmgr.h
===================================================================
--- sys/sys/_lockmgr.h
+++ sys/sys/_lockmgr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Attilio Rao <attilio@FreeBSD.org>
* All rights reserved.
Index: sys/sys/_null.h
===================================================================
--- sys/sys/_null.h
+++ sys/sys/_null.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003 Marcel Moolenaar
* All rights reserved.
Index: sys/sys/_pctrie.h
===================================================================
--- sys/sys/_pctrie.h
+++ sys/sys/_pctrie.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 EMC Corp.
* Copyright (c) 2011 Jeffrey Roberson <jeff@freebsd.org>
Index: sys/sys/_rangeset.h
===================================================================
--- sys/sys/_rangeset.h
+++ sys/sys/_rangeset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019 The FreeBSD Foundation
*
Index: sys/sys/_rwlock.h
===================================================================
--- sys/sys/_rwlock.h
+++ sys/sys/_rwlock.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 John Baldwin <jhb@FreeBSD.org>
*
Index: sys/sys/_semaphore.h
===================================================================
--- sys/sys/_semaphore.h
+++ sys/sys/_semaphore.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Alfred Perlstein <alfred@FreeBSD.org>
* All rights reserved.
Index: sys/sys/_smr.h
===================================================================
--- sys/sys/_smr.h
+++ sys/sys/_smr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019, 2020 Jeffrey Roberson <jeff@FreeBSD.org>
*
Index: sys/sys/_stack.h
===================================================================
--- sys/sys/_stack.h
+++ sys/sys/_stack.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 Antoine Brodin
* All rights reserved.
Index: sys/sys/_stdint.h
===================================================================
--- sys/sys/_stdint.h
+++ sys/sys/_stdint.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2011 David E. O'Brien <obrien@FreeBSD.org>
* Copyright (c) 2001 Mike Barcroft <mike@FreeBSD.org>
Index: sys/sys/_sx.h
===================================================================
--- sys/sys/_sx.h
+++ sys/sys/_sx.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007 Attilio Rao <attilio@freebsd.org>
* All rights reserved.
Index: sys/sys/_task.h
===================================================================
--- sys/sys/_task.h
+++ sys/sys/_task.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Doug Rabson
* All rights reserved.
Index: sys/sys/_timeval.h
===================================================================
--- sys/sys/_timeval.h
+++ sys/sys/_timeval.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Mike Barcroft <mike@FreeBSD.org>
* All rights reserved.
Index: sys/sys/_tls_variant_i.h
===================================================================
--- sys/sys/_tls_variant_i.h
+++ sys/sys/_tls_variant_i.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 David Xu <davidxu@freebsd.org>.
* All rights reserved.
Index: sys/sys/_types.h
===================================================================
--- sys/sys/_types.h
+++ sys/sys/_types.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Mike Barcroft <mike@FreeBSD.org>
* All rights reserved.
Index: sys/sys/_umtx.h
===================================================================
--- sys/sys/_umtx.h
+++ sys/sys/_umtx.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010, David Xu <davidxu@freebsd.org>
* All rights reserved.
Index: sys/sys/_unrhdr.h
===================================================================
--- sys/sys/_unrhdr.h
+++ sys/sys/_unrhdr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 Poul-Henning Kamp
* All rights reserved.
Index: sys/sys/aac_ioctl.h
===================================================================
--- sys/sys/aac_ioctl.h
+++ sys/sys/aac_ioctl.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Michael Smith
* Copyright (c) 2000 Scott Long
Index: sys/sys/abi_compat.h
===================================================================
--- sys/sys/abi_compat.h
+++ sys/sys/abi_compat.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Doug Rabson
* All rights reserved.
Index: sys/sys/acl.h
===================================================================
--- sys/sys/acl.h
+++ sys/sys/acl.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999-2001 Robert N. M. Watson
* Copyright (c) 2008 Edward Tomasz Napierała <trasz@FreeBSD.org>
Index: sys/sys/agpio.h
===================================================================
--- sys/sys/agpio.h
+++ sys/sys/agpio.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Doug Rabson
* All rights reserved.
Index: sys/sys/aio.h
===================================================================
--- sys/sys/aio.h
+++ sys/sys/aio.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997 John S. Dyson. All rights reserved.
*
Index: sys/sys/arb.h
===================================================================
--- sys/sys/arb.h
+++ sys/sys/arb.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2002 Niels Provos <provos@citi.umich.edu>
* Copyright 2018-2019 Netflix, Inc.
Index: sys/sys/ata.h
===================================================================
--- sys/sys/ata.h
+++ sys/sys/ata.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 - 2008 Søren Schmidt <sos@FreeBSD.org>
* All rights reserved.
Index: sys/sys/atomic_common.h
===================================================================
--- sys/sys/atomic_common.h
+++ sys/sys/atomic_common.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017 The FreeBSD Foundation
*
Index: sys/sys/auxv.h
===================================================================
--- sys/sys/auxv.h
+++ sys/sys/auxv.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017 Michal Meloun
* All rights reserved.
Index: sys/sys/backlight.h
===================================================================
--- sys/sys/backlight.h
+++ sys/sys/backlight.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2020 Emmanuel Vadot <manu@FreeBSD.org>
*
Index: sys/sys/bitset.h
===================================================================
--- sys/sys/bitset.h
+++ sys/sys/bitset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/blockcount.h
===================================================================
--- sys/sys/blockcount.h
+++ sys/sys/blockcount.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 John Baldwin <jhb@FreeBSD.org>
* Copyright (c) 2020 The FreeBSD Foundation
Index: sys/sys/boot.h
===================================================================
--- sys/sys/boot.h
+++ sys/sys/boot.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014 Roger Pau Monné <roger.pau@citrix.com>
* All rights reserved.
Index: sys/sys/boottrace.h
===================================================================
--- sys/sys/boottrace.h
+++ sys/sys/boottrace.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008-2022 NetApp, Inc.
*
Index: sys/sys/buf_ring.h
===================================================================
--- sys/sys/buf_ring.h
+++ sys/sys/buf_ring.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007-2009 Kip Macy <kmacy@freebsd.org>
* All rights reserved.
Index: sys/sys/bufobj.h
===================================================================
--- sys/sys/bufobj.h
+++ sys/sys/bufobj.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 Poul-Henning Kamp
* All rights reserved.
Index: sys/sys/bus.h
===================================================================
--- sys/sys/bus.h
+++ sys/sys/bus.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997,1998,2003 Doug Rabson
* All rights reserved.
Index: sys/sys/bus_dma.h
===================================================================
--- sys/sys/bus_dma.h
+++ sys/sys/bus_dma.h
@@ -1,7 +1,7 @@
/* $NetBSD: bus.h,v 1.12 1997/10/01 08:25:15 fvdl Exp $ */
/*-
- * SPDX-License-Identifier: (BSD-2-Clause AND BSD-4-Clause)
+ * SPDX-License-Identifier: (BSD-2-Clause-NetBSD AND BSD-4-Clause)
*
* Copyright (c) 1996, 1997 The NetBSD Foundation, Inc.
* All rights reserved.
Index: sys/sys/bus_dma_internal.h
===================================================================
--- sys/sys/bus_dma_internal.h
+++ sys/sys/bus_dma_internal.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017 Jason A. Harmening.
* All rights reserved.
Index: sys/sys/busdma_bufalloc.h
===================================================================
--- sys/sys/busdma_bufalloc.h
+++ sys/sys/busdma_bufalloc.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2012 Ian Lepore
* All rights reserved.
Index: sys/sys/caprights.h
===================================================================
--- sys/sys/caprights.h
+++ sys/sys/caprights.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 FreeBSD Foundation
*
Index: sys/sys/capsicum.h
===================================================================
--- sys/sys/capsicum.h
+++ sys/sys/capsicum.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008-2010, 2015 Robert N. M. Watson
* Copyright (c) 2012 FreeBSD Foundation
Index: sys/sys/clock.h
===================================================================
--- sys/sys/clock.h
+++ sys/sys/clock.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1996 The NetBSD Foundation, Inc.
* All rights reserved.
Index: sys/sys/cnv.h
===================================================================
--- sys/sys/cnv.h
+++ sys/sys/cnv.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2016 Adam Starak <starak.adam@gmail.com>
* All rights reserved.
Index: sys/sys/compressor.h
===================================================================
--- sys/sys/compressor.h
+++ sys/sys/compressor.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014, 2017 Mark Johnston <markj@FreeBSD.org>
*
Index: sys/sys/condvar.h
===================================================================
--- sys/sys/condvar.h
+++ sys/sys/condvar.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Jake Burkholder <jake@freebsd.org>.
* All rights reserved.
Index: sys/sys/counter.h
===================================================================
--- sys/sys/counter.h
+++ sys/sys/counter.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2012 Gleb Smirnoff <glebius@FreeBSD.org>
* All rights reserved.
Index: sys/sys/coverage.h
===================================================================
--- sys/sys/coverage.h
+++ sys/sys/coverage.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2018 The FreeBSD Foundation. All rights reserved.
* Copyright (C) 2018, 2019 Andrew Turner.
Index: sys/sys/cpu.h
===================================================================
--- sys/sys/cpu.h
+++ sys/sys/cpu.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005-2007 Nate Lawson (SDG)
* All rights reserved.
Index: sys/sys/cpuctl.h
===================================================================
--- sys/sys/cpuctl.h
+++ sys/sys/cpuctl.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006-2008 Stanislav Sedov <stas@FreeBSD.org>
* All rights reserved.
Index: sys/sys/cpuset.h
===================================================================
--- sys/sys/cpuset.h
+++ sys/sys/cpuset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/crc16.h
===================================================================
--- sys/sys/crc16.h
+++ sys/sys/crc16.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017, Fedor Uporov
* All rights reserved.
Index: sys/sys/devmap.h
===================================================================
--- sys/sys/devmap.h
+++ sys/sys/devmap.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 Ian Lepore <ian@freebsd.org>
* All rights reserved.
Index: sys/sys/disk_zone.h
===================================================================
--- sys/sys/disk_zone.h
+++ sys/sys/disk_zone.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2015 Spectra Logic Corporation
* All rights reserved.
Index: sys/sys/dnv.h
===================================================================
--- sys/sys/dnv.h
+++ sys/sys/dnv.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 The FreeBSD Foundation
*
Index: sys/sys/domainset.h
===================================================================
--- sys/sys/domainset.h
+++ sys/sys/domainset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/dtrace_bsd.h
===================================================================
--- sys/sys/dtrace_bsd.h
+++ sys/sys/dtrace_bsd.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007-2008 John Birrell (jb@freebsd.org)
* All rights reserved.
Index: sys/sys/elf.h
===================================================================
--- sys/sys/elf.h
+++ sys/sys/elf.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 David E. O'Brien.
* All rights reserved.
Index: sys/sys/elf32.h
===================================================================
--- sys/sys/elf32.h
+++ sys/sys/elf32.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1996-1998 John D. Polstra.
* All rights reserved.
Index: sys/sys/elf64.h
===================================================================
--- sys/sys/elf64.h
+++ sys/sys/elf64.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1996-1998 John D. Polstra.
* All rights reserved.
Index: sys/sys/elf_common.h
===================================================================
--- sys/sys/elf_common.h
+++ sys/sys/elf_common.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017, 2018 Dell EMC
* Copyright (c) 2000, 2001, 2008, 2011, David E. O'Brien
Index: sys/sys/elf_generic.h
===================================================================
--- sys/sys/elf_generic.h
+++ sys/sys/elf_generic.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1998 John D. Polstra.
* All rights reserved.
Index: sys/sys/endian.h
===================================================================
--- sys/sys/endian.h
+++ sys/sys/endian.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Thomas Moestl <tmm@FreeBSD.org>
* All rights reserved.
Index: sys/sys/epoch.h
===================================================================
--- sys/sys/epoch.h
+++ sys/sys/epoch.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2018, Matthew Macy <mmacy@freebsd.org>
*
Index: sys/sys/event.h
===================================================================
--- sys/sys/event.h
+++ sys/sys/event.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999,2000,2001 Jonathan Lemon <jlemon@FreeBSD.org>
* All rights reserved.
Index: sys/sys/eventfd.h
===================================================================
--- sys/sys/eventfd.h
+++ sys/sys/eventfd.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2020 Val Packett
*
Index: sys/sys/eventhandler.h
===================================================================
--- sys/sys/eventhandler.h
+++ sys/sys/eventhandler.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Michael Smith <msmith@freebsd.org>
* All rights reserved.
Index: sys/sys/eventvar.h
===================================================================
--- sys/sys/eventvar.h
+++ sys/sys/eventvar.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999,2000 Jonathan Lemon <jlemon@FreeBSD.org>
* All rights reserved.
Index: sys/sys/extattr.h
===================================================================
--- sys/sys/extattr.h
+++ sys/sys/extattr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999-2001 Robert N. M. Watson
* All rights reserved.
Index: sys/sys/fail.h
===================================================================
--- sys/sys/fail.h
+++ sys/sys/fail.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009-2019 Dell EMC Isilon http://www.isilon.com/
*
Index: sys/sys/fdcio.h
===================================================================
--- sys/sys/fdcio.h
+++ sys/sys/fdcio.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 1992-1994,2001 by Joerg Wunsch, Dresden
* All rights reserved.
Index: sys/sys/firmware.h
===================================================================
--- sys/sys/firmware.h
+++ sys/sys/firmware.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005, Sam Leffler <sam@errno.com>
* All rights reserved.
Index: sys/sys/font.h
===================================================================
--- sys/sys/font.h
+++ sys/sys/font.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009, 2013 The FreeBSD Foundation
*
Index: sys/sys/gpio.h
===================================================================
--- sys/sys/gpio.h
+++ sys/sys/gpio.h
@@ -1,7 +1,7 @@
/* $NetBSD: gpio.h,v 1.7 2009/09/25 20:27:50 mbalmer Exp $ */
/* $OpenBSD: gpio.h,v 1.7 2008/11/26 14:51:20 mbalmer Exp $ */
/*-
- * SPDX-License-Identifier: (BSD-2-Clause AND ISC)
+ * SPDX-License-Identifier: (BSD-2-Clause-FreeBSD AND ISC)
*
* Copyright (c) 2009, Oleksandr Tymoshenko <gonzo@FreeBSD.org>
* All rights reserved.
Index: sys/sys/gpt.h
===================================================================
--- sys/sys/gpt.h
+++ sys/sys/gpt.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Marcel Moolenaar
* All rights reserved.
Index: sys/sys/gtaskqueue.h
===================================================================
--- sys/sys/gtaskqueue.h
+++ sys/sys/gtaskqueue.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014 Jeffrey Roberson <jeff@freebsd.org>
* Copyright (c) 2016 Matthew Macy <mmacy@mattmacy.io>
Index: sys/sys/hash.h
===================================================================
--- sys/sys/hash.h
+++ sys/sys/hash.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Tobias Weingartner
* All rights reserved.
Index: sys/sys/hhook.h
===================================================================
--- sys/sys/hhook.h
+++ sys/sys/hhook.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010,2013 Lawrence Stewart <lstewart@freebsd.org>
* Copyright (c) 2010 The FreeBSD Foundation
Index: sys/sys/hwt.h
===================================================================
--- /dev/null
+++ sys/sys/hwt.h
@@ -0,0 +1,103 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+/* User-visible header. */
+
+#include <sys/param.h>
+#include <sys/cpuset.h>
+#include <sys/types.h>
+#include <sys/hwt_record.h>
+
+#ifndef _SYS_HWT_H_
+#define _SYS_HWT_H_
+
+#define HWT_MAGIC 0x42
+#define HWT_IOC_ALLOC _IOW(HWT_MAGIC, 0x00, struct hwt_alloc)
+#define HWT_IOC_START _IOW(HWT_MAGIC, 0x01, struct hwt_start)
+#define HWT_IOC_STOP _IOW(HWT_MAGIC, 0x02, struct hwt_stop)
+#define HWT_IOC_RECORD_GET _IOW(HWT_MAGIC, 0x03, struct hwt_record_get)
+#define HWT_IOC_BUFPTR_GET _IOW(HWT_MAGIC, 0x04, struct hwt_bufptr_get)
+#define HWT_IOC_SET_CONFIG _IOW(HWT_MAGIC, 0x05, struct hwt_set_config)
+#define HWT_IOC_WAKEUP _IOW(HWT_MAGIC, 0x06, struct hwt_wakeup)
+
+#define HWT_BACKEND_MAXNAMELEN 256
+
+#define HWT_MODE_THREAD 1
+#define HWT_MODE_CPU 2
+
+struct hwt_alloc {
+ size_t bufsize;
+ int mode;
+ pid_t pid; /* thread mode */
+ cpuset_t cpu_map; /* cpu mode only */
+ const char *backend_name;
+ int *ident;
+} __aligned(16);
+
+struct hwt_start {
+ int reserved;
+} __aligned(16);
+
+struct hwt_stop {
+ int reserved;
+} __aligned(16);
+
+struct hwt_wakeup {
+ int reserved;
+} __aligned(16);
+
+struct hwt_record_user_entry {
+ enum hwt_record_type record_type;
+ char fullpath[MAXPATHLEN];
+ uintptr_t addr;
+ int thread_id;
+} __aligned(16);
+
+struct hwt_record_get {
+ struct hwt_record_user_entry *records;
+ int *nentries;
+} __aligned(16);
+
+struct hwt_bufptr_get {
+ int *curpage;
+ vm_offset_t *curpage_offset;
+} __aligned(16);
+
+struct hwt_set_config {
+ /* Configuration of ctx. */
+ int pause_on_mmap;
+
+ /* The following passed to backend as is. */
+ void *config;
+ size_t config_size;
+ int config_version;
+} __aligned(16);
+
+#endif /* !_SYS_HWT_H_ */
Index: sys/sys/hwt_record.h
===================================================================
--- /dev/null
+++ sys/sys/hwt_record.h
@@ -0,0 +1,56 @@
+/*-
+ * Copyright (c) 2023 Ruslan Bukin <br@bsdpad.com>
+ *
+ * This work was supported by Innovate UK project 105694, "Digital Security
+ * by Design (DSbD) Technology Platform Prototype".
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+/* User-visible header. */
+
+#ifndef _SYS_HWT_RECORD_H_
+#define _SYS_HWT_RECORD_H_
+
+enum hwt_record_type {
+ HWT_RECORD_MMAP,
+ HWT_RECORD_MUNMAP,
+ HWT_RECORD_EXECUTABLE,
+ HWT_RECORD_INTERP,
+ HWT_RECORD_THREAD_CREATE,
+ HWT_RECORD_THREAD_SET_NAME,
+ HWT_RECORD_KERNEL,
+};
+
+#ifdef _KERNEL
+struct hwt_record_entry {
+ enum hwt_record_type record_type;
+ LIST_ENTRY(hwt_record_entry) next;
+ char *fullpath;
+ int thread_id;
+ uintptr_t addr;
+};
+#endif
+
+#endif /* !_SYS_HWT_RECORD_H_ */
Index: sys/sys/iconv.h
===================================================================
--- sys/sys/iconv.h
+++ sys/sys/iconv.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000-2001 Boris Popov
* All rights reserved.
Index: sys/sys/imgact_binmisc.h
===================================================================
--- sys/sys/imgact_binmisc.h
+++ sys/sys/imgact_binmisc.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 Stacey D. Son
*
Index: sys/sys/interrupt.h
===================================================================
--- sys/sys/interrupt.h
+++ sys/sys/interrupt.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997, Stefan Esser <se@freebsd.org>
* All rights reserved.
Index: sys/sys/intr.h
===================================================================
--- sys/sys/intr.h
+++ sys/sys/intr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2015-2016 Svatopluk Kraus
* Copyright (c) 2015-2016 Michal Meloun
Index: sys/sys/iov.h
===================================================================
--- sys/sys/iov.h
+++ sys/sys/iov.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013-2015 Sandvine Inc.
* All rights reserved.
Index: sys/sys/iov_schema.h
===================================================================
--- sys/sys/iov_schema.h
+++ sys/sys/iov_schema.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014-2015 Sandvine Inc.
* All rights reserved.
Index: sys/sys/ipmi.h
===================================================================
--- sys/sys/ipmi.h
+++ sys/sys/ipmi.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 IronPort Systems Inc. <ambrisko@ironport.com>
* All rights reserved.
Index: sys/sys/jail.h
===================================================================
--- sys/sys/jail.h
+++ sys/sys/jail.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Poul-Henning Kamp.
* Copyright (c) 2009 James Gritton.
Index: sys/sys/kcov.h
===================================================================
--- sys/sys/kcov.h
+++ sys/sys/kcov.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2018 The FreeBSD Foundation. All rights reserved.
* Copyright (C) 2018, 2019 Andrew Turner.
Index: sys/sys/kdb.h
===================================================================
--- sys/sys/kdb.h
+++ sys/sys/kdb.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 Marcel Moolenaar
* All rights reserved.
Index: sys/sys/kenv.h
===================================================================
--- sys/sys/kenv.h
+++ sys/sys/kenv.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Maxime Henrion <mux@FreeBSD.org>
* All rights reserved.
Index: sys/sys/khelp.h
===================================================================
--- sys/sys/khelp.h
+++ sys/sys/khelp.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010 Lawrence Stewart <lstewart@freebsd.org>
* Copyright (c) 2010 The FreeBSD Foundation
Index: sys/sys/kobj.h
===================================================================
--- sys/sys/kobj.h
+++ sys/sys/kobj.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000,2003 Doug Rabson
* All rights reserved.
Index: sys/sys/kpilite.h
===================================================================
--- sys/sys/kpilite.h
+++ sys/sys/kpilite.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2018, Matthew Macy <mmacy@freebsd.org>
*
Index: sys/sys/ksem.h
===================================================================
--- sys/sys/ksem.h
+++ sys/sys/ksem.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Alfred Perlstein <alfred@FreeBSD.org>
* All rights reserved.
Index: sys/sys/kthread.h
===================================================================
--- sys/sys/kthread.h
+++ sys/sys/kthread.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Peter Wemm <peter@FreeBSD.org>
* All rights reserved.
Index: sys/sys/ktls.h
===================================================================
--- sys/sys/ktls.h
+++ sys/sys/ktls.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014-2019 Netflix Inc.
*
Index: sys/sys/linker.h
===================================================================
--- sys/sys/linker.h
+++ sys/sys/linker.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997-2000 Doug Rabson
* All rights reserved.
Index: sys/sys/linker_set.h
===================================================================
--- sys/sys/linker_set.h
+++ sys/sys/linker_set.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 John D. Polstra
* Copyright (c) 1999,2001 Peter Wemm <peter@FreeBSD.org>
Index: sys/sys/lock.h
===================================================================
--- sys/sys/lock.h
+++ sys/sys/lock.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997 Berkeley Software Design, Inc. All rights reserved.
*
@@ -182,7 +182,7 @@
struct lock_delay_arg {
struct lock_delay_config *config;
- u_int delay;
+ u_short delay;
u_int spin_cnt;
};
Index: sys/sys/lock_profile.h
===================================================================
--- sys/sys/lock_profile.h
+++ sys/sys/lock_profile.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 Kip Macy kmacy@FreeBSD.org
* Copyright (c) 2006 Kris Kennaway kris@FreeBSD.org
Index: sys/sys/lockmgr.h
===================================================================
--- sys/sys/lockmgr.h
+++ sys/sys/lockmgr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Attilio Rao <attilio@FreeBSD.org>
* All rights reserved.
Index: sys/sys/lockstat.h
===================================================================
--- sys/sys/lockstat.h
+++ sys/sys/lockstat.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008-2009 Stacey Son <sson@FreeBSD.org>
*
Index: sys/sys/loginclass.h
===================================================================
--- sys/sys/loginclass.h
+++ sys/sys/loginclass.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2011 The FreeBSD Foundation
*
Index: sys/sys/mac.h
===================================================================
--- sys/sys/mac.h
+++ sys/sys/mac.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999-2002 Robert N. M. Watson
* Copyright (c) 2001-2005 Networks Associates Technology, Inc.
Index: sys/sys/mchain.h
===================================================================
--- sys/sys/mchain.h
+++ sys/sys/mchain.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000, 2001 Boris Popov
* All rights reserved.
Index: sys/sys/memdesc.h
===================================================================
--- sys/sys/memdesc.h
+++ sys/sys/memdesc.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2012 EMC Corp.
* All rights reserved.
Index: sys/sys/module.h
===================================================================
--- sys/sys/module.h
+++ sys/sys/module.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997 Doug Rabson
* All rights reserved.
Index: sys/sys/module_khelp.h
===================================================================
--- sys/sys/module_khelp.h
+++ sys/sys/module_khelp.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010 Lawrence Stewart <lstewart@freebsd.org>
* Copyright (c) 2010 The FreeBSD Foundation
Index: sys/sys/mqueue.h
===================================================================
--- sys/sys/mqueue.h
+++ sys/sys/mqueue.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 David Xu <davidxu@freebsd.org>
* All rights reserved.
Index: sys/sys/nv_namespace.h
===================================================================
--- sys/sys/nv_namespace.h
+++ sys/sys/nv_namespace.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2022 Rubicon Communications, LLC (Netgate)
* All rights reserved.
Index: sys/sys/osd.h
===================================================================
--- sys/sys/osd.h
+++ sys/sys/osd.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007 Pawel Jakub Dawidek <pjd@FreeBSD.org>
* All rights reserved.
Index: sys/sys/pciio.h
===================================================================
--- sys/sys/pciio.h
+++ sys/sys/pciio.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1997, Stefan Esser <se@FreeBSD.ORG>
* Copyright (c) 1997, 1998, 1999, Kenneth D. Merry <ken@FreeBSD.ORG>
Index: sys/sys/pctrie.h
===================================================================
--- sys/sys/pctrie.h
+++ sys/sys/pctrie.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 EMC Corp.
* Copyright (c) 2011 Jeffrey Roberson <jeff@freebsd.org>
Index: sys/sys/physmem.h
===================================================================
--- sys/sys/physmem.h
+++ sys/sys/physmem.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014 Ian Lepore <ian@freebsd.org>
* All rights reserved.
Index: sys/sys/pidctrl.h
===================================================================
--- sys/sys/pidctrl.h
+++ sys/sys/pidctrl.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/pmc.h
===================================================================
--- sys/sys/pmc.h
+++ sys/sys/pmc.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003-2008, Joseph Koshy
* Copyright (c) 2007 The FreeBSD Foundation
@@ -82,50 +82,62 @@
*
* Please keep the pmc(3) manual page in sync with this list.
*/
-#define __PMC_CPUS() \
- __PMC_CPU(AMD_K7, 0x00, "AMD K7") \
- __PMC_CPU(AMD_K8, 0x01, "AMD K8") \
- __PMC_CPU(INTEL_CORE, 0x87, "Intel Core Solo/Duo") \
- __PMC_CPU(INTEL_CORE2, 0x88, "Intel Core2") \
- __PMC_CPU(INTEL_CORE2EXTREME, 0x89, "Intel Core2 Extreme") \
- __PMC_CPU(INTEL_ATOM, 0x8A, "Intel Atom") \
- __PMC_CPU(INTEL_COREI7, 0x8B, "Intel Core i7") \
- __PMC_CPU(INTEL_WESTMERE, 0x8C, "Intel Westmere") \
- __PMC_CPU(INTEL_SANDYBRIDGE, 0x8D, "Intel Sandy Bridge") \
- __PMC_CPU(INTEL_IVYBRIDGE, 0x8E, "Intel Ivy Bridge") \
- __PMC_CPU(INTEL_SANDYBRIDGE_XEON, 0x8F, "Intel Sandy Bridge Xeon") \
- __PMC_CPU(INTEL_IVYBRIDGE_XEON, 0x90, "Intel Ivy Bridge Xeon") \
- __PMC_CPU(INTEL_HASWELL, 0x91, "Intel Haswell") \
- __PMC_CPU(INTEL_ATOM_SILVERMONT, 0x92, "Intel Atom Silvermont") \
- __PMC_CPU(INTEL_NEHALEM_EX, 0x93, "Intel Nehalem Xeon 7500") \
- __PMC_CPU(INTEL_WESTMERE_EX, 0x94, "Intel Westmere Xeon E7") \
- __PMC_CPU(INTEL_HASWELL_XEON, 0x95, "Intel Haswell Xeon E5 v3") \
- __PMC_CPU(INTEL_BROADWELL, 0x96, "Intel Broadwell") \
- __PMC_CPU(INTEL_BROADWELL_XEON, 0x97, "Intel Broadwell Xeon") \
- __PMC_CPU(INTEL_SKYLAKE, 0x98, "Intel Skylake") \
- __PMC_CPU(INTEL_SKYLAKE_XEON, 0x99, "Intel Skylake Xeon") \
- __PMC_CPU(INTEL_ATOM_GOLDMONT, 0x9A, "Intel Atom Goldmont") \
- __PMC_CPU(INTEL_ICELAKE, 0x9B, "Intel Icelake") \
- __PMC_CPU(INTEL_ICELAKE_XEON, 0x9C, "Intel Icelake Xeon") \
- __PMC_CPU(INTEL_ALDERLAKE, 0x9D, "Intel Alderlake") \
- __PMC_CPU(INTEL_ATOM_GOLDMONT_P, 0x9E, "Intel Atom Goldmont Plus") \
- __PMC_CPU(INTEL_ATOM_TREMONT, 0x9F, "Intel Atom Tremont") \
- __PMC_CPU(INTEL_XSCALE, 0x100, "Intel XScale") \
- __PMC_CPU(PPC_7450, 0x300, "PowerPC MPC7450") \
- __PMC_CPU(PPC_E500, 0x340, "PowerPC e500 Core") \
- __PMC_CPU(PPC_970, 0x380, "IBM PowerPC 970") \
- __PMC_CPU(PPC_POWER8, 0x390, "IBM POWER8") \
- __PMC_CPU(GENERIC, 0x400, "Generic") \
- __PMC_CPU(ARMV7_CORTEX_A5, 0x500, "ARMv7 Cortex A5") \
- __PMC_CPU(ARMV7_CORTEX_A7, 0x501, "ARMv7 Cortex A7") \
- __PMC_CPU(ARMV7_CORTEX_A8, 0x502, "ARMv7 Cortex A8") \
- __PMC_CPU(ARMV7_CORTEX_A9, 0x503, "ARMv7 Cortex A9") \
- __PMC_CPU(ARMV7_CORTEX_A15, 0x504, "ARMv7 Cortex A15") \
- __PMC_CPU(ARMV7_CORTEX_A17, 0x505, "ARMv7 Cortex A17") \
- __PMC_CPU(ARMV8_CORTEX_A53, 0x600, "ARMv8 Cortex A53") \
- __PMC_CPU(ARMV8_CORTEX_A57, 0x601, "ARMv8 Cortex A57") \
- __PMC_CPU(ARMV8_CORTEX_A76, 0x602, "ARMv8 Cortex A76") \
- __PMC_CPU(ARMV8_RAINIER, 0x6ff, "ARMv8 Rainier")
+
+#define __PMC_CPUS() \
+ __PMC_CPU(AMD_K7, 0x00, "AMD K7") \
+ __PMC_CPU(AMD_K8, 0x01, "AMD K8") \
+ __PMC_CPU(INTEL_P5, 0x80, "Intel Pentium") \
+ __PMC_CPU(INTEL_P6, 0x81, "Intel Pentium Pro") \
+ __PMC_CPU(INTEL_CL, 0x82, "Intel Celeron") \
+ __PMC_CPU(INTEL_PII, 0x83, "Intel Pentium II") \
+ __PMC_CPU(INTEL_PIII, 0x84, "Intel Pentium III") \
+ __PMC_CPU(INTEL_PM, 0x85, "Intel Pentium M") \
+ __PMC_CPU(INTEL_PIV, 0x86, "Intel Pentium IV") \
+ __PMC_CPU(INTEL_CORE, 0x87, "Intel Core Solo/Duo") \
+ __PMC_CPU(INTEL_CORE2, 0x88, "Intel Core2") \
+ __PMC_CPU(INTEL_CORE2EXTREME, 0x89, "Intel Core2 Extreme") \
+ __PMC_CPU(INTEL_ATOM, 0x8A, "Intel Atom") \
+ __PMC_CPU(INTEL_COREI7, 0x8B, "Intel Core i7") \
+ __PMC_CPU(INTEL_WESTMERE, 0x8C, "Intel Westmere") \
+ __PMC_CPU(INTEL_SANDYBRIDGE, 0x8D, "Intel Sandy Bridge") \
+ __PMC_CPU(INTEL_IVYBRIDGE, 0x8E, "Intel Ivy Bridge") \
+ __PMC_CPU(INTEL_SANDYBRIDGE_XEON, 0x8F, "Intel Sandy Bridge Xeon") \
+ __PMC_CPU(INTEL_IVYBRIDGE_XEON, 0x90, "Intel Ivy Bridge Xeon") \
+ __PMC_CPU(INTEL_HASWELL, 0x91, "Intel Haswell") \
+ __PMC_CPU(INTEL_ATOM_SILVERMONT, 0x92, "Intel Atom Silvermont") \
+ __PMC_CPU(INTEL_NEHALEM_EX, 0x93, "Intel Nehalem Xeon 7500") \
+ __PMC_CPU(INTEL_WESTMERE_EX, 0x94, "Intel Westmere Xeon E7") \
+ __PMC_CPU(INTEL_HASWELL_XEON, 0x95, "Intel Haswell Xeon E5 v3") \
+ __PMC_CPU(INTEL_BROADWELL, 0x96, "Intel Broadwell") \
+ __PMC_CPU(INTEL_BROADWELL_XEON, 0x97, "Intel Broadwell Xeon") \
+ __PMC_CPU(INTEL_SKYLAKE, 0x98, "Intel Skylake") \
+ __PMC_CPU(INTEL_SKYLAKE_XEON, 0x99, "Intel Skylake Xeon") \
+ __PMC_CPU(INTEL_ATOM_GOLDMONT, 0x9A, "Intel Atom Goldmont") \
+ __PMC_CPU(INTEL_ICELAKE, 0x9B, "Intel Icelake") \
+ __PMC_CPU(INTEL_ICELAKE_XEON, 0x9C, "Intel Icelake Xeon") \
+ __PMC_CPU(INTEL_ALDERLAKE, 0x9D, "Intel Alderlake") \
+ __PMC_CPU(INTEL_ATOM_GOLDMONT_P, 0x9E, "Intel Atom Goldmont Plus") \
+ __PMC_CPU(INTEL_ATOM_TREMONT, 0x9F, "Intel Atom Tremont") \
+ __PMC_CPU(INTEL_XSCALE, 0x100, "Intel XScale") \
+ __PMC_CPU(MIPS_24K, 0x200, "MIPS 24K") \
+ __PMC_CPU(MIPS_OCTEON, 0x201, "Cavium Octeon") \
+ __PMC_CPU(MIPS_74K, 0x202, "MIPS 74K") \
+ __PMC_CPU(MIPS_BERI, 0x203, "BERI") \
+ __PMC_CPU(PPC_7450, 0x300, "PowerPC MPC7450") \
+ __PMC_CPU(PPC_E500, 0x340, "PowerPC e500 Core") \
+ __PMC_CPU(PPC_970, 0x380, "IBM PowerPC 970") \
+ __PMC_CPU(PPC_POWER8, 0x390, "IBM POWER8") \
+ __PMC_CPU(GENERIC, 0x400, "Generic") \
+ __PMC_CPU(ARMV7_CORTEX_A5, 0x500, "ARMv7 Cortex A5") \
+ __PMC_CPU(ARMV7_CORTEX_A7, 0x501, "ARMv7 Cortex A7") \
+ __PMC_CPU(ARMV7_CORTEX_A8, 0x502, "ARMv7 Cortex A8") \
+ __PMC_CPU(ARMV7_CORTEX_A9, 0x503, "ARMv7 Cortex A9") \
+ __PMC_CPU(ARMV7_CORTEX_A15, 0x504, "ARMv7 Cortex A15") \
+ __PMC_CPU(ARMV7_CORTEX_A17, 0x505, "ARMv7 Cortex A17") \
+ __PMC_CPU(ARMV8_CORTEX_A53, 0x600, "ARMv8 Cortex A53") \
+ __PMC_CPU(ARMV8_CORTEX_A57, 0x601, "ARMv8 Cortex A57") \
+ __PMC_CPU(ARMV8_CORTEX_A76, 0x602, "ARMv8 Cortex A76") \
+ __PMC_CPU(ARMV8_RAINIER, 0x6ff, "ARMv8 Rainier")
enum pmc_cputype {
#undef __PMC_CPU
@@ -139,25 +151,33 @@
/*
* Classes of PMCs
*/
-#define __PMC_CLASSES() \
- __PMC_CLASS(TSC, 0x00, "CPU Timestamp counter") \
- __PMC_CLASS(K7, 0x01, "AMD K7 performance counters") \
- __PMC_CLASS(K8, 0x02, "AMD K8 performance counters") \
- __PMC_CLASS(IAF, 0x06, "Intel Core2/Atom, fixed function") \
- __PMC_CLASS(IAP, 0x07, "Intel Core...Atom, programmable") \
- __PMC_CLASS(UCF, 0x08, "Intel Uncore fixed function") \
- __PMC_CLASS(UCP, 0x09, "Intel Uncore programmable") \
- __PMC_CLASS(XSCALE, 0x0A, "Intel XScale counters") \
- __PMC_CLASS(PPC7450, 0x0D, "Motorola MPC7450 class") \
- __PMC_CLASS(PPC970, 0x0E, "IBM PowerPC 970 class") \
- __PMC_CLASS(SOFT, 0x0F, "Software events") \
- __PMC_CLASS(ARMV7, 0x10, "ARMv7") \
- __PMC_CLASS(ARMV8, 0x11, "ARMv8") \
- __PMC_CLASS(E500, 0x13, "Freescale e500 class") \
- __PMC_CLASS(POWER8, 0x15, "IBM POWER8 class") \
- __PMC_CLASS(DMC620_PMU_CD2, 0x16, "ARM DMC620 Memory Controller PMU CLKDIV2") \
- __PMC_CLASS(DMC620_PMU_C, 0x17, "ARM DMC620 Memory Controller PMU CLK") \
- __PMC_CLASS(CMN600_PMU, 0x18, "Arm CoreLink CMN600 Coherent Mesh Network PMU")
+
+#define __PMC_CLASSES() \
+ __PMC_CLASS(TSC, 0x00, "CPU Timestamp counter") \
+ __PMC_CLASS(K7, 0x01, "AMD K7 performance counters") \
+ __PMC_CLASS(K8, 0x02, "AMD K8 performance counters") \
+ __PMC_CLASS(P5, 0x03, "Intel Pentium counters") \
+ __PMC_CLASS(P6, 0x04, "Intel Pentium Pro counters") \
+ __PMC_CLASS(P4, 0x05, "Intel Pentium-IV counters") \
+ __PMC_CLASS(IAF, 0x06, "Intel Core2/Atom, fixed function") \
+ __PMC_CLASS(IAP, 0x07, "Intel Core...Atom, programmable") \
+ __PMC_CLASS(UCF, 0x08, "Intel Uncore fixed function") \
+ __PMC_CLASS(UCP, 0x09, "Intel Uncore programmable") \
+ __PMC_CLASS(XSCALE, 0x0A, "Intel XScale counters") \
+ __PMC_CLASS(MIPS24K, 0x0B, "MIPS 24K") \
+ __PMC_CLASS(OCTEON, 0x0C, "Cavium Octeon") \
+ __PMC_CLASS(PPC7450, 0x0D, "Motorola MPC7450 class") \
+ __PMC_CLASS(PPC970, 0x0E, "IBM PowerPC 970 class") \
+ __PMC_CLASS(SOFT, 0x0F, "Software events") \
+ __PMC_CLASS(ARMV7, 0x10, "ARMv7") \
+ __PMC_CLASS(ARMV8, 0x11, "ARMv8") \
+ __PMC_CLASS(MIPS74K, 0x12, "MIPS 74K") \
+ __PMC_CLASS(E500, 0x13, "Freescale e500 class") \
+ __PMC_CLASS(BERI, 0x14, "MIPS BERI") \
+ __PMC_CLASS(POWER8, 0x15, "IBM POWER8 class") \
+ __PMC_CLASS(DMC620_PMU_CD2, 0x16, "ARM DMC620 Memory Controller PMU CLKDIV2") \
+ __PMC_CLASS(DMC620_PMU_C, 0x17, "ARM DMC620 Memory Controller PMU CLK") \
+ __PMC_CLASS(CMN600_PMU, 0x18, "Arm CoreLink CMN600 Coherent Mesh Network PMU")
enum pmc_class {
#undef __PMC_CLASS
@@ -1046,6 +1066,10 @@
* Machine dependent methods.
*/
+ /* per-cpu initialization and finalization */
+ int (*pmd_pcpu_init)(struct pmc_mdep *_md, int _cpu);
+ int (*pmd_pcpu_fini)(struct pmc_mdep *_md, int _cpu);
+
/* thread context switch in/out */
int (*pmd_switch_in)(struct pmc_cpu *_p, struct pmc_process *_pp);
int (*pmd_switch_out)(struct pmc_cpu *_p, struct pmc_process *_pp);
Index: sys/sys/pmckern.h
===================================================================
--- sys/sys/pmckern.h
+++ sys/sys/pmckern.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003-2007, Joseph Koshy
* Copyright (c) 2007 The FreeBSD Foundation
Index: sys/sys/pmclog.h
===================================================================
--- sys/sys/pmclog.h
+++ sys/sys/pmclog.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005-2007, Joseph Koshy
* Copyright (c) 2007 The FreeBSD Foundation
Index: sys/sys/power.h
===================================================================
--- sys/sys/power.h
+++ sys/sys/power.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Mitsuru IWASAKI
* All rights reserved.
Index: sys/sys/priv.h
===================================================================
--- sys/sys/priv.h
+++ sys/sys/priv.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 nCircle Network Security, Inc.
* All rights reserved.
Index: sys/sys/proc.h
===================================================================
--- sys/sys/proc.h
+++ sys/sys/proc.h
@@ -271,11 +271,11 @@
const char *td_wmesg; /* (t) Reason for sleep. */
volatile u_char td_owepreempt; /* (k*) Preempt on last critical_exit */
u_char td_tsqueue; /* (t) Turnstile queue blocked on. */
- u_char td_stopsched; /* (k) Scheduler stopped. */
- int td_locks; /* (k) Debug: count of non-spin locks */
- int td_rw_rlocks; /* (k) Count of rwlock read locks. */
- int td_sx_slocks; /* (k) Count of sx shared locks. */
- int td_lk_slocks; /* (k) Count of lockmgr shared locks. */
+ short td_locks; /* (k) Debug: count of non-spin locks */
+ short td_rw_rlocks; /* (k) Count of rwlock read locks. */
+ short td_sx_slocks; /* (k) Count of sx shared locks. */
+ short td_lk_slocks; /* (k) Count of lockmgr shared locks. */
+ short td_stopsched; /* (k) Scheduler stopped. */
struct turnstile *td_blocked; /* (t) Lock thread is blocked on. */
const char *td_lockname; /* (t) Name of lock blocked on. */
LIST_HEAD(, turnstile) td_contested; /* (q) Contested locks. */
@@ -886,6 +886,7 @@
#define P2_WEXIT 0x00040000 /* exit just started, no
external thread_single() is
permitted */
+#define P2_HWT 0x00080000 /* Process is using HWT. */
/* Flags protected by proctree_lock, kept in p_treeflags. */
#define P_TREE_ORPHANED 0x00000001 /* Reparented, on orphan list */
Index: sys/sys/procctl.h
===================================================================
--- sys/sys/procctl.h
+++ sys/sys/procctl.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 Hudson River Trading LLC
* Copyright (c) 2014, 2016 The FreeBSD Foundation
Index: sys/sys/procdesc.h
===================================================================
--- sys/sys/procdesc.h
+++ sys/sys/procdesc.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009 Robert N. M. Watson
* All rights reserved.
Index: sys/sys/procfs.h
===================================================================
--- sys/sys/procfs.h
+++ sys/sys/procfs.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1998 John D. Polstra.
* All rights reserved.
Index: sys/sys/ptio.h
===================================================================
--- sys/sys/ptio.h
+++ sys/sys/ptio.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 1999 Kenneth D. Merry.
* All rights reserved.
Index: sys/sys/racct.h
===================================================================
--- sys/sys/racct.h
+++ sys/sys/racct.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010 The FreeBSD Foundation
*
Index: sys/sys/random.h
===================================================================
--- sys/sys/random.h
+++ sys/sys/random.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000-2015, 2017 Mark R. V. Murray
* All rights reserved.
Index: sys/sys/rangelock.h
===================================================================
--- sys/sys/rangelock.h
+++ sys/sys/rangelock.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009 Konstantin Belousov <kib@FreeBSD.org>
* All rights reserved.
Index: sys/sys/rangeset.h
===================================================================
--- sys/sys/rangeset.h
+++ sys/sys/rangeset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019 The FreeBSD Foundation
*
Index: sys/sys/rctl.h
===================================================================
--- sys/sys/rctl.h
+++ sys/sys/rctl.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010 The FreeBSD Foundation
*
Index: sys/sys/refcount.h
===================================================================
--- sys/sys/refcount.h
+++ sys/sys/refcount.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 John Baldwin <jhb@FreeBSD.org>
*
Index: sys/sys/reg.h
===================================================================
--- sys/sys/reg.h
+++ sys/sys/reg.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2014,2019 Andrew Turner
* Copyright (c) 2014-2015 The FreeBSD Foundation
Index: sys/sys/regression.h
===================================================================
--- sys/sys/regression.h
+++ sys/sys/regression.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Robert N. M. Watson
* All rights reserved.
Index: sys/sys/runq.h
===================================================================
--- sys/sys/runq.h
+++ sys/sys/runq.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Jake Burkholder <jake@FreeBSD.org>
* All rights reserved.
Index: sys/sys/rwlock.h
===================================================================
--- sys/sys/rwlock.h
+++ sys/sys/rwlock.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 John Baldwin <jhb@FreeBSD.org>
*
Index: sys/sys/sbuf.h
===================================================================
--- sys/sys/sbuf.h
+++ sys/sys/sbuf.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000-2008 Poul-Henning Kamp
* Copyright (c) 2000-2008 Dag-Erling Coïdan Smørgrav
Index: sys/sys/sched.h
===================================================================
--- sys/sys/sched.h
+++ sys/sys/sched.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: (BSD-4-Clause AND BSD-2-Clause)
+ * SPDX-License-Identifier: (BSD-4-Clause AND BSD-2-Clause-FreeBSD)
*
* Copyright (c) 1996, 1997
* HD Associates, Inc. All rights reserved.
Index: sys/sys/sdt.h
===================================================================
--- sys/sys/sdt.h
+++ sys/sys/sdt.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2006-2008 John Birrell <jb@FreeBSD.org>
*
Index: sys/sys/sema.h
===================================================================
--- sys/sys/sema.h
+++ sys/sys/sema.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (C) 2001 Jason Evans <jasone@freebsd.org>. All rights reserved.
*
Index: sys/sys/serial.h
===================================================================
--- sys/sys/serial.h
+++ sys/sys/serial.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 Poul-Henning Kamp
* All rights reserved.
Index: sys/sys/sf_buf.h
===================================================================
--- sys/sys/sf_buf.h
+++ sys/sys/sf_buf.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2014 Gleb Smirnoff <glebius@FreeBSD.org>
* Copyright (c) 2003-2004 Alan L. Cox <alc@cs.rice.edu>
Index: sys/sys/sleepqueue.h
===================================================================
--- sys/sys/sleepqueue.h
+++ sys/sys/sleepqueue.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2004 John Baldwin <jhb@FreeBSD.org>
*
Index: sys/sys/slicer.h
===================================================================
--- sys/sys/slicer.h
+++ sys/sys/slicer.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2012 Semihalf.
* All rights reserved.
Index: sys/sys/smr.h
===================================================================
--- sys/sys/smr.h
+++ sys/sys/smr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019, 2020 Jeffrey Roberson <jeff@FreeBSD.org>
*
Index: sys/sys/smr_types.h
===================================================================
--- sys/sys/smr_types.h
+++ sys/sys/smr_types.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2019, 2020 Jeffrey Roberson <jeff@FreeBSD.org>
*
Index: sys/sys/sndstat.h
===================================================================
--- sys/sys/sndstat.h
+++ sys/sys/sndstat.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2020 The FreeBSD Foundation
*
Index: sys/sys/specialfd.h
===================================================================
--- sys/sys/specialfd.h
+++ sys/sys/specialfd.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2020 Val Packett
*
Index: sys/sys/spigenio.h
===================================================================
--- sys/sys/spigenio.h
+++ sys/sys/spigenio.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Doug Rabson
* All rights reserved.
Index: sys/sys/stack.h
===================================================================
--- sys/sys/stack.h
+++ sys/sys/stack.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005 Antoine Brodin
* All rights reserved.
Index: sys/sys/stdatomic.h
===================================================================
--- sys/sys/stdatomic.h
+++ sys/sys/stdatomic.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2011 Ed Schouten <ed@FreeBSD.org>
* David Chisnall <theraven@FreeBSD.org>
Index: sys/sys/stddef.h
===================================================================
--- sys/sys/stddef.h
+++ sys/sys/stddef.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Maxime Henrion <mux@FreeBSD.org>
* All rights reserved.
Index: sys/sys/stdint.h
===================================================================
--- sys/sys/stdint.h
+++ sys/sys/stdint.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2001 Mike Barcroft <mike@FreeBSD.org>
* All rights reserved.
Index: sys/sys/sx.h
===================================================================
--- sys/sys/sx.h
+++ sys/sys/sx.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2007 Attilio Rao <attilio@freebsd.org>
* Copyright (c) 2001 Jason Evans <jasone@freebsd.org>
Index: sys/sys/syscallsubr.h
===================================================================
--- sys/sys/syscallsubr.h
+++ sys/sys/syscallsubr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Ian Dowse. All rights reserved.
*
Index: sys/sys/taskqueue.h
===================================================================
--- sys/sys/taskqueue.h
+++ sys/sys/taskqueue.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Doug Rabson
* All rights reserved.
Index: sys/sys/terminal.h
===================================================================
--- sys/sys/terminal.h
+++ sys/sys/terminal.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009 The FreeBSD Foundation
*
Index: sys/sys/thr.h
===================================================================
--- sys/sys/thr.h
+++ sys/sys/thr.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/tiio.h
===================================================================
--- sys/sys/tiio.h
+++ sys/sys/tiio.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: (BSD-2-Clause AND BSD-4-Clause)
+ * SPDX-License-Identifier: (BSD-2-Clause-FreeBSD AND BSD-4-Clause)
*
* Copyright (c) 1999, 2000 Kenneth D. Merry.
* All rights reserved.
Index: sys/sys/timeet.h
===================================================================
--- sys/sys/timeet.h
+++ sys/sys/timeet.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2010-2013 Alexander Motin <mav@FreeBSD.org>
* All rights reserved.
Index: sys/sys/timeffc.h
===================================================================
--- sys/sys/timeffc.h
+++ sys/sys/timeffc.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2011 The University of Melbourne
* All rights reserved.
Index: sys/sys/tree.h
===================================================================
--- sys/sys/tree.h
+++ sys/sys/tree.h
@@ -3,7 +3,7 @@
/* $FreeBSD$ */
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2002 Niels Provos <provos@citi.umich.edu>
* All rights reserved.
Index: sys/sys/tty.h
===================================================================
--- sys/sys/tty.h
+++ sys/sys/tty.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/sys/ttydevsw.h
===================================================================
--- sys/sys/ttydevsw.h
+++ sys/sys/ttydevsw.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/sys/ttydisc.h
===================================================================
--- sys/sys/ttydisc.h
+++ sys/sys/ttydisc.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/sys/ttyhook.h
===================================================================
--- sys/sys/ttyhook.h
+++ sys/sys/ttyhook.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/sys/ttyqueue.h
===================================================================
--- sys/sys/ttyqueue.h
+++ sys/sys/ttyqueue.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2008 Ed Schouten <ed@FreeBSD.org>
* All rights reserved.
Index: sys/sys/turnstile.h
===================================================================
--- sys/sys/turnstile.h
+++ sys/sys/turnstile.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 John Baldwin <jhb@FreeBSD.org>
*
Index: sys/sys/umtx.h
===================================================================
--- sys/sys/umtx.h
+++ sys/sys/umtx.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/umtxvar.h
===================================================================
--- sys/sys/umtxvar.h
+++ sys/sys/umtxvar.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/sys/uuid.h
===================================================================
--- sys/sys/uuid.h
+++ sys/sys/uuid.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002 Marcel Moolenaar
* All rights reserved.
Index: sys/sys/vdso.h
===================================================================
--- sys/sys/vdso.h
+++ sys/sys/vdso.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright 2012 Konstantin Belousov <kib@FreeBSD.ORG>.
* All rights reserved.
Index: sys/sys/vmem.h
===================================================================
--- sys/sys/vmem.h
+++ sys/sys/vmem.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c)2006 YAMAMOTO Takashi,
* All rights reserved.
Index: sys/sys/watchdog.h
===================================================================
--- sys/sys/watchdog.h
+++ sys/sys/watchdog.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2003 Poul-Henning Kamp
* Copyright (c) 2013 iXsystems.com,
Index: sys/vm/_vm_phys.h
===================================================================
--- sys/vm/_vm_phys.h
+++ sys/vm/_vm_phys.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002-2006 Rice University
* Copyright (c) 2007 Alan L. Cox <alc@cs.rice.edu>
Index: sys/vm/_vm_radix.h
===================================================================
--- sys/vm/_vm_radix.h
+++ sys/vm/_vm_radix.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 EMC Corp.
* Copyright (c) 2011 Jeffrey Roberson <jeff@freebsd.org>
Index: sys/vm/memguard.h
===================================================================
--- sys/vm/memguard.h
+++ sys/vm/memguard.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005,
* Bosko Milekic <bmilekic@FreeBSD.org>. All rights reserved.
Index: sys/vm/memguard.c
===================================================================
--- sys/vm/memguard.c
+++ sys/vm/memguard.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2005, Bosko Milekic <bmilekic@FreeBSD.org>.
* Copyright (c) 2010 Isilon Systems, Inc. (http://www.isilon.com/)
Index: sys/vm/phys_pager.c
===================================================================
--- sys/vm/phys_pager.c
+++ sys/vm/phys_pager.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2000 Peter Wemm
*
Index: sys/vm/redzone.h
===================================================================
--- sys/vm/redzone.h
+++ sys/vm/redzone.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 Pawel Jakub Dawidek <pjd@FreeBSD.org>
* All rights reserved.
Index: sys/vm/redzone.c
===================================================================
--- sys/vm/redzone.c
+++ sys/vm/redzone.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2006 Pawel Jakub Dawidek <pjd@FreeBSD.org>
* All rights reserved.
Index: sys/vm/sg_pager.c
===================================================================
--- sys/vm/sg_pager.c
+++ sys/vm/sg_pager.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2009 Hudson River Trading LLC
* Written by: John H. Baldwin <jhb@FreeBSD.org>
Index: sys/vm/uma.h
===================================================================
--- sys/vm/uma.h
+++ sys/vm/uma.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002, 2003, 2004, 2005 Jeffrey Roberson <jeff@FreeBSD.org>
* Copyright (c) 2004, 2005 Bosko Milekic <bmilekic@FreeBSD.org>
Index: sys/vm/uma_core.c
===================================================================
--- sys/vm/uma_core.c
+++ sys/vm/uma_core.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002-2019 Jeffrey Roberson <jeff@FreeBSD.org>
* Copyright (c) 2004, 2005 Bosko Milekic <bmilekic@FreeBSD.org>
Index: sys/vm/uma_dbg.h
===================================================================
--- sys/vm/uma_dbg.h
+++ sys/vm/uma_dbg.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002, 2003, 2004, 2005 Jeffrey Roberson <jeff@FreeBSD.org>
* Copyright (c) 2004, 2005 Bosko Milekic <bmilekic@FreeBSD.org>
Index: sys/vm/uma_dbg.c
===================================================================
--- sys/vm/uma_dbg.c
+++ sys/vm/uma_dbg.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002, 2003, 2004, 2005 Jeffrey Roberson <jeff@FreeBSD.org>
* Copyright (c) 2004, 2005 Bosko Milekic <bmilekic@FreeBSD.org>
Index: sys/vm/uma_int.h
===================================================================
--- sys/vm/uma_int.h
+++ sys/vm/uma_int.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002-2019 Jeffrey Roberson <jeff@FreeBSD.org>
* Copyright (c) 2004, 2005 Bosko Milekic <bmilekic@FreeBSD.org>
Index: sys/vm/vm_domainset.h
===================================================================
--- sys/vm/vm_domainset.h
+++ sys/vm/vm_domainset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/vm/vm_domainset.c
===================================================================
--- sys/vm/vm_domainset.c
+++ sys/vm/vm_domainset.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2017, Jeffrey Roberson <jeff@freebsd.org>
* All rights reserved.
Index: sys/vm/vm_dumpset.h
===================================================================
--- sys/vm/vm_dumpset.h
+++ sys/vm/vm_dumpset.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2020, Scott Phillips <scottph@freebsd.org>
*
Index: sys/vm/vm_mmap.c
===================================================================
--- sys/vm/vm_mmap.c
+++ sys/vm/vm_mmap.c
@@ -110,6 +110,10 @@
#include <sys/pmckern.h>
#endif
+#ifdef HWT_HOOKS
+#include <dev/hwt/hwt_hook.h>
+#endif
+
#if __has_feature(capabilities)
#include <cheri/cheric.h>
#endif
@@ -1054,6 +1058,16 @@
#endif
rv = vm_map_remove_locked(map, addr, addr + size);
+#ifdef HWT_HOOKS
+ struct hwt_record_entry ent;
+ if (rv == KERN_SUCCESS) {
+ ent.addr = (uintptr_t) addr;
+ ent.size = (size_t) size;
+ ent.record_type = HWT_RECORD_MUNMAP;
+ HWT_CALL_HOOK(td, HWT_RECORD, &ent);
+ }
+#endif
+
#ifdef HWPMC_HOOKS
if (rv == KERN_SUCCESS && __predict_false(pmc_handled)) {
/* downgrade the lock to prevent a LOR with the pmc-sx lock */
Index: sys/vm/vm_page.h
===================================================================
--- sys/vm/vm_page.h
+++ sys/vm/vm_page.h
@@ -686,9 +686,6 @@
vm_paddr_t high, u_long alignment, vm_paddr_t boundary);
bool vm_page_reclaim_contig_domain(int domain, int req, u_long npages,
vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary);
-bool vm_page_reclaim_contig_domain_ext(int domain, int req, u_long npages,
- vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary,
- int desired_runs);
void vm_page_reference(vm_page_t m);
#define VPR_TRYFREE 0x01
#define VPR_NOREUSE 0x02
Index: sys/vm/vm_page.c
===================================================================
--- sys/vm/vm_page.c
+++ sys/vm/vm_page.c
@@ -2997,7 +2997,9 @@
#define NRUNS 16
-#define RUN_INDEX(count, nruns) ((count) % (nruns))
+CTASSERT(powerof2(NRUNS));
+
+#define RUN_INDEX(count) ((count) & (NRUNS - 1))
#define MIN_RECLAIM 8
@@ -3025,42 +3027,19 @@
* must be a power of two.
*/
bool
-vm_page_reclaim_contig_domain_ext(int domain, int req, u_long npages,
- vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary,
- int desired_runs)
+vm_page_reclaim_contig_domain(int domain, int req, u_long npages,
+ vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary)
{
struct vm_domain *vmd;
vm_paddr_t curr_low;
- vm_page_t m_run, _m_runs[NRUNS], *m_runs;
+ vm_page_t m_run, m_runs[NRUNS];
u_long count, minalign, reclaimed;
- int error, i, min_reclaim, nruns, options, req_class;
- bool ret;
+ int error, i, options, req_class;
KASSERT(npages > 0, ("npages is 0"));
KASSERT(powerof2(alignment), ("alignment is not a power of 2"));
KASSERT(powerof2(boundary), ("boundary is not a power of 2"));
- ret = false;
-
- /*
- * If the caller wants to reclaim multiple runs, try to allocate
- * space to store the runs. If that fails, fall back to the old
- * behavior of just reclaiming MIN_RECLAIM pages.
- */
- if (desired_runs > 1)
- m_runs = malloc((NRUNS + desired_runs) * sizeof(*m_runs),
- M_TEMP, M_NOWAIT);
- else
- m_runs = NULL;
-
- if (m_runs == NULL) {
- m_runs = _m_runs;
- nruns = NRUNS;
- } else {
- nruns = NRUNS + desired_runs - 1;
- }
- min_reclaim = MAX(desired_runs * npages, MIN_RECLAIM);
-
/*
* The caller will attempt an allocation after some runs have been
* reclaimed and added to the vm_phys buddy lists. Due to limitations
@@ -3089,7 +3068,7 @@
if (count < npages + vmd->vmd_free_reserved || (count < npages +
vmd->vmd_interrupt_free_min && req_class == VM_ALLOC_SYSTEM) ||
(count < npages && req_class == VM_ALLOC_INTERRUPT))
- goto done;
+ return (false);
/*
* Scan up to three times, relaxing the restrictions ("options") on
@@ -3108,29 +3087,27 @@
if (m_run == NULL)
break;
curr_low = VM_PAGE_TO_PHYS(m_run) + ptoa(npages);
- m_runs[RUN_INDEX(count, nruns)] = m_run;
+ m_runs[RUN_INDEX(count)] = m_run;
count++;
}
/*
* Reclaim the highest runs in LIFO (descending) order until
* the number of reclaimed pages, "reclaimed", is at least
- * "min_reclaim". Reset "reclaimed" each time because each
+ * MIN_RECLAIM. Reset "reclaimed" each time because each
* reclamation is idempotent, and runs will (likely) recur
* from one scan to the next as restrictions are relaxed.
*/
reclaimed = 0;
- for (i = 0; count > 0 && i < nruns; i++) {
+ for (i = 0; count > 0 && i < NRUNS; i++) {
count--;
- m_run = m_runs[RUN_INDEX(count, nruns)];
+ m_run = m_runs[RUN_INDEX(count)];
error = vm_page_reclaim_run(req_class, domain, npages,
m_run, high);
if (error == 0) {
reclaimed += npages;
- if (reclaimed >= min_reclaim) {
- ret = true;
- goto done;
- }
+ if (reclaimed >= MIN_RECLAIM)
+ return (true);
}
}
@@ -3142,23 +3119,9 @@
options = VPSC_NOSUPER;
else if (options == VPSC_NOSUPER)
options = VPSC_ANY;
- else if (options == VPSC_ANY) {
- ret = reclaimed != 0;
- goto done;
- }
+ else if (options == VPSC_ANY)
+ return (reclaimed != 0);
}
-done:
- if (m_runs != _m_runs)
- free(m_runs, M_TEMP);
- return (ret);
-}
-
-bool
-vm_page_reclaim_contig_domain(int domain, int req, u_long npages,
- vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary)
-{
- return (vm_page_reclaim_contig_domain_ext(domain, req, npages, low, high,
- alignment, boundary, 1));
}
bool
Index: sys/vm/vm_phys.h
===================================================================
--- sys/vm/vm_phys.h
+++ sys/vm/vm_phys.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002-2006 Rice University
* Copyright (c) 2007 Alan L. Cox <alc@cs.rice.edu>
Index: sys/vm/vm_phys.c
===================================================================
--- sys/vm/vm_phys.c
+++ sys/vm/vm_phys.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002-2006 Rice University
* Copyright (c) 2007 Alan L. Cox <alc@cs.rice.edu>
Index: sys/vm/vm_radix.h
===================================================================
--- sys/vm/vm_radix.h
+++ sys/vm/vm_radix.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 EMC Corp.
* Copyright (c) 2011 Jeffrey Roberson <jeff@freebsd.org>
Index: sys/vm/vm_radix.c
===================================================================
--- sys/vm/vm_radix.c
+++ sys/vm/vm_radix.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2013 EMC Corp.
* Copyright (c) 2011 Jeffrey Roberson <jeff@freebsd.org>
Index: sys/vm/vm_reserv.h
===================================================================
--- sys/vm/vm_reserv.h
+++ sys/vm/vm_reserv.h
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002-2006 Rice University
* Copyright (c) 2007-2008 Alan L. Cox <alc@cs.rice.edu>
Index: sys/vm/vm_reserv.c
===================================================================
--- sys/vm/vm_reserv.c
+++ sys/vm/vm_reserv.c
@@ -1,5 +1,5 @@
/*-
- * SPDX-License-Identifier: BSD-2-Clause
+ * SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
* Copyright (c) 2002-2006 Rice University
* Copyright (c) 2007-2011 Alan L. Cox <alc@cs.rice.edu>

File Metadata

Mime Type
text/plain
Expires
Sun, Feb 8, 4:21 PM (12 h, 4 m)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
28472521
Default Alt Text
D40466.id125528.diff (245 KB)

Event Timeline