Index: share/man/man4/Makefile =================================================================== --- share/man/man4/Makefile +++ share/man/man4/Makefile @@ -81,6 +81,7 @@ bhndb_pci.4 \ blackhole.4 \ bnxt.4 \ + boottrace.4 \ bpf.4 \ bridge.4 \ bwi.4 \ Index: share/man/man4/boottrace.4 =================================================================== --- /dev/null +++ share/man/man4/boottrace.4 @@ -0,0 +1,210 @@ +.\" SPDX-License-Identifier: BSD-2-Clause-FreeBSD +.\" +.\" Copyright (c) 2022 NetApp, Inc. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.Dd November 25, 2021 +.Dt BOOTTRACE 4 +.Os +.Sh NAME +.Nm boottrace +.\" XXX: This is not great that boot-time sometimes refers to the whole boot +.\" process and sometimes only to the part of the whole boot process before +.\" init(8) in this manual page. +.Nd Boot-time, run-time, and shutdown-time tracing facility +.Sh DESCRIPTION +.Nm +is a kernel-userspace interface for capturing trace events +during system boot and shutdown. +This includes kernel initialization and +.Xr rc 8 . +.Pp +Event annotations are present in the boot and shutdown paths in the +kernel, in some key system utilities +.Po +.Xr init 8 , +.Xr shutdown 8 , +.Xr reboot 8 +. Pc , +and in +.Xr rc 8 +scripts. +This gives a complete log of events covering the entire boot process. +.Pp +.Nm +is unconditionally compiled into the kernel +and is disabled by default. +.Sh EVENT TABLES +Events are stored in three event tables: boot-time events, run-time events, +and shutdown events. +Events are stored in the boot-time table, until the +time that control is handed over to init(8). +After this point, +events are stored in the run-time table. +The run-time table wraps around when it fills up. +Upon initiating a shutdown, a reboot, or a kernel panic +events will be stored in +the shutdown-time table. +.\" XXX: Should we make distinction here between system tunables and sysctl variables? +.Sh LOADER TUNABLES +Tunables can be set at the +.Xr loader 8 +prompt before booting the kernel or stored in +.Xr loader.conf 5 . +.Nm +features the following loader tunables: +.Bl -tag -width indent +.It Va kern.boottrace.dotrace_kernel +Set to +.Ql 1 +to enable tracing of kernel events. +Default: +.Ql 1 +.Pq enabled . +.It Va kern.boottrace.dotrace_user +Set to +.Ql 1 +to enable tracing of userspace events. +Default: +.Ql 1 +.Pq enabled . +.El +.Sh SYSCTL VARIABLES +The following variables are available as both +.Xr sysctl 8 +variables and +.Xr loader 8 +tunables: +.Bl -tag -width indent +.It Va kern.boottrace.boottrace +Create a new trace event and write it to the boot-time table. +.\" XXX: I checked the example boottrace_log.txt sent to the mailing list and the +.\" format was completely different. I'm not sure this part is still +.\" up-to-date. +.\" The event will be logged as +.\" .Ql ${procname}: name . +.It Va kern.boottrace.enabled +Set to +.Ql 1 +to enable tracing. +This is a read-only +.Xr sysctl 8 +variable. +Default: +.Ql 0 +.Pq disabled . +.It Va kern.boottrace.log +Show the events stored in boot-time and run-time +tables. +This +is an opaque +.Xr sysctl 8 +variable. +.It Va kern.boottrace.runtrace +Same as +.Va kern.boottrace.boottrace , +but write to the run-time table. +.It Va kern.boottrace.shuttrace +Same as +.Va kern.boottrace.boottrace , +but write to the shutdown-time table. +.It Va kern.boottrace.shutdown_trace +Log shutdown-time events to the console before the system halts. +.It Va kern.boottrace.shutdown_trace_threshold +Set a time threshold for logging shutdown-time events in milliseconds. +An event is ignored +if the time difference to the previous event is less than +the threshold value. +Default: +.Ql 0 +.Pq logs all events . +.El +.Sh EXAMPLES +Here is a sample output of +.Va kern.boottrace.log +.Po shortened with +.Dq [...] +for readability +.Pc : +.Bd -literal +CPU msecs delta process event PID CPUtime IBlks OBlks + 0 44872811 0 kernel sysinit 0x2100001 0 0.00 0 0 + 0 44872812 1 kernel sysinit 0x2110000 0 0.00 0 0 + 0 44872812 0 kernel sysinit 0x2140000 0 0.00 0 0 +[...] + 0 44872817 0 kernel sysinit 0x2800000 0 0.00 0 0 + 0 44873820 1003 kernel sysinit 0x2880000 0 0.00 0 0 + 0 44873820 0 kernel sysinit 0x2888000 0 0.00 0 0 +[...] + 1 44875735 0 kernel sysinit 0xfffffff 0 0.00 0 0 + 1 44875735 0 swapper mi_startup done 0 0.00 0 0 + 0 44875750 15 init init(8) starting... 1 0.00 0 0 + 0 44875751 1 init /etc/rc starting... 1 0.00 0 0 + 0 44875831 80 boottrace /etc/rc.d/rctl start 26 0.00 0 0 + 1 44875839 8 boottrace /etc/rc.d/rctl done 26 0.00 2 0 +[...] + 0 44876446 0 boottrace /etc/rc.d/netif start 390 0.00 0 0 + 1 44881116 4670 boottrace /etc/rc.d/netif done 390 0.12 34 0 +[...] + 0 44882866 1 boottrace /etc/rc.d/securelevel start 1679 0.00 0 0 + 0 44882872 6 boottrace /etc/rc.d/securelevel done 1679 0.00 0 0 + 1 44882879 7 init /etc/rc finished 1 2.22 743 15 +Total measured time: 10068 msecs + + +CPU msecs delta process event PID CPUtime IBlks OBlks + 1 44882880 0 init multi-user start 1 2.22 743 15 + 0 44918215 35335 kldload hwpmc.ko: sysinit 0xd800000 1698 0.00 0 0 +Total measured time: 35335 msecs +.Ed +.Sh SEE ALSO +.Xr sysctl 9 +.Sh HISTORY +.Nm +first appeared in +.Fx 14.0 . +Before that, +NetApp used +.Nm +internally for years +for diagnosing slow devices and subsystems. +.Sh AUTHORS +This manual page was written by +.An Mateusz Piotrowski Aq Mt 0mp@FreeBSD.org . +.Sh BUGS +There is some overlap in functionality between +.Nm +and the +existing boot-time event tracing facility, TSLOG. +TSLOG captures +kernel events from early boot to mountroot, at a +slightly finer granularity. +.Nm boottrace's +scope is wider, and it is currently +initialized slightly later, at SI_SUB_CPU. +For this reason, I think it's fine +for the two to co-exist. +TSLog is nicely self-contained, +and extending it beyond its initial purpose +would likely be messier than adding this as +a separate facility.