diff --git a/share/man/man4/hwpmc.4 b/share/man/man4/hwpmc.4 index c394af90cd8d..3ae3ce726395 100644 --- a/share/man/man4/hwpmc.4 +++ b/share/man/man4/hwpmc.4 @@ -1,818 +1,920 @@ .\" Copyright (c) 2003-2008 Joseph Koshy -.\" Copyright (c) 2007 The FreeBSD Foundation -.\" All rights reserved. +.\" Copyright (c) 2007,2023 The FreeBSD Foundation .\" .\" Portions of this software were developed by A. Joseph Koshy under .\" sponsorship from the FreeBSD Foundation and Google, Inc. .\" +.\" Portions of this documentation were written by Mitchell Horne +.\" under sponsorship from the FreeBSD Foundation. +.\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" .Dd June 16, 2023 .Dt HWPMC 4 .Os .Sh NAME .Nm hwpmc .Nd "Hardware Performance Monitoring Counter support" .Sh SYNOPSIS The following option must be present in the kernel configuration file: .Bd -ragged -offset indent .Cd "options HWPMC_HOOKS" .Ed .Pp Additionally, for i386 systems: .Bd -ragged -offset indent .Cd "device apic" .Ed .Pp To load the driver as a module at boot time, place the following line in .Xr rc.conf 5 : .Bd -literal -offset indent kld_list="${kld_list} hwpmc" .Ed .Pp Alternatively, to compile this driver into the kernel: .Bd -ragged -offset indent .Cd "device hwpmc" .Ed +.Pp +To enable debugging features +.Po see +.Sx DEBUGGING +.Pc : +.Bd -ragged -offset indent +.Cd "options KTR" +.Cd "options KTR_COMPILE=(KTR_SUBSYS)" +.Cd "options KTR_MASK=(KTR_SUBSYS)" +.Cd "options HWPMC_DEBUG" +.Ed .Sh DESCRIPTION The .Nm driver virtualizes the hardware performance monitoring facilities in modern CPUs and provides support for using these facilities from user level processes. .Pp The driver supports multi-processor systems. .Pp PMCs are allocated using the .Dv PMC_OP_PMCALLOCATE request. A successful .Dv PMC_OP_PMCALLOCATE request will return a handle to the requesting process. Subsequent operations on the allocated PMC use this handle to denote the specific PMC. A process that has successfully allocated a PMC is termed an .Dq "owner process" . .Pp PMCs may be allocated with process or system scope. .Bl -tag -width ".Em Process-scope" .It Em "Process-scope" The PMC is active only when a thread belonging to a process it is attached to is scheduled on a CPU. .It Em "System-scope" The PMC operates independently of processes and measures hardware events for the system as a whole. .El .Pp PMCs may be allocated for counting or for sampling: .Bl -tag -width ".Em Counting" .It Em Counting In counting modes, the PMCs count hardware events. These counts are retrievable using the .Dv PMC_OP_PMCREAD system call on all architectures. Some architectures offer faster methods of reading these counts. .It Em Sampling In sampling modes, the PMCs are configured to sample the CPU instruction pointer (and optionally to capture the call chain leading up to the sampled instruction pointer) after a configurable number of hardware events have been observed. Instruction pointer samples and call chain records are usually directed to a log file for subsequent analysis. .El .Pp Scope and operational mode are orthogonal; a PMC may thus be configured to operate in one of the following four modes: .Bl -tag -width indent .It Process-scope, counting These PMCs count hardware events whenever a thread in their attached process is scheduled on a CPU. These PMCs normally count from zero, but the initial count may be set using the .Dv PMC_OP_SETCOUNT operation. Applications can read the value of the PMC anytime using the .Dv PMC_OP_PMCRW operation. .It Process-scope, sampling These PMCs sample the target processes instruction pointer after they have seen the configured number of hardware events. The PMCs only count events when a thread belonging to their attached process is active. The desired frequency of sampling is set using the .Dv PMC_OP_SETCOUNT operation prior to starting the PMC. Log files are configured using the .Dv PMC_OP_CONFIGURELOG operation. .It System-scope, counting These PMCs count hardware events seen by them independent of the processes that are executing. The current count on these PMCs can be read using the .Dv PMC_OP_PMCRW request. These PMCs normally count from zero, but the initial count may be set using the .Dv PMC_OP_SETCOUNT operation. .It System-scope, sampling These PMCs will periodically sample the instruction pointer of the CPU they are allocated on, and will write the sample to a log for further processing. The desired frequency of sampling is set using the .Dv PMC_OP_SETCOUNT operation prior to starting the PMC. Log files are configured using the .Dv PMC_OP_CONFIGURELOG operation. .Pp System-wide statistical sampling can only be enabled by a process with super-user privileges. .El .Pp Processes are allowed to allocate as many PMCs as the hardware and current operating conditions permit. Processes may mix allocations of system-wide and process-private PMCs. Multiple processes may be using PMCs simultaneously. .Pp Allocated PMCs are started using the .Dv PMC_OP_PMCSTART operation, and stopped using the .Dv PMC_OP_PMCSTOP operation. Stopping and starting a PMC is permitted at any time the owner process has a valid handle to the PMC. .Pp Process-private PMCs need to be attached to a target process before they can be used. Attaching a process to a PMC is done using the .Dv PMC_OP_PMCATTACH operation. An already attached PMC may be detached from its target process using the converse .Dv PMC_OP_PMCDETACH operation. Issuing a .Dv PMC_OP_PMCSTART operation on an as yet unattached PMC will cause it to be attached to its owner process. The following rules determine whether a given process may attach a PMC to another target process: .Bl -bullet -compact .It A non-jailed process with super-user privileges is allowed to attach to any other process in the system. .It Other processes are only allowed to attach to targets that they would be able to attach to for debugging (as determined by .Xr p_candebug 9 ) . .El .Pp PMCs are released using .Dv PMC_OP_PMCRELEASE . After a successful .Dv PMC_OP_PMCRELEASE operation the handle to the PMC will become invalid. .Ss Modifier Flags The .Dv PMC_OP_PMCALLOCATE operation supports the following flags that modify the behavior of an allocated PMC: .Bl -tag -width indent .It Dv PMC_F_CALLCHAIN This modifier informs sampling PMCs to record a callchain when capturing a sample. The maximum depth to which call chains are recorded is specified by the .Va "kern.hwpmc.callchaindepth" kernel tunable. .It Dv PMC_F_DESCENDANTS This modifier is valid only for a PMC being allocated in process-private mode. It signifies that the PMC will track hardware events for its target process and the target's current and future descendants. .It Dv PMC_F_LOG_PROCCSW This modifier is valid only for a PMC being allocated in process-private mode. When this modifier is present, at every context switch, .Nm will log a record containing the number of hardware events seen by the target process when it was scheduled on the CPU. .It Dv PMC_F_LOG_PROCEXIT This modifier is valid only for a PMC being allocated in process-private mode. With this modifier present, .Nm will maintain per-process counts for each target process attached to a PMC. At process exit time, a record containing the target process' PID and the accumulated per-process count for that process will be written to the configured log file. .El .Pp Modifiers .Dv PMC_F_LOG_PROCEXIT and .Dv PMC_F_LOG_PROCCSW may be used in combination with modifier .Dv PMC_F_DESCENDANTS to track the behavior of complex pipelines of processes. PMCs with modifiers .Dv PMC_F_LOG_PROCEXIT and .Dv PMC_F_LOG_PROCCSW cannot be started until their owner process has configured a log file. .Ss Signals The .Nm driver may deliver signals to processes that have allocated PMCs: .Bl -tag -width ".Dv SIGBUS" .It Dv SIGIO A .Dv PMC_OP_PMCRW operation was attempted on a process-private PMC that does not have attached target processes. .It Dv SIGBUS The .Nm driver is being unloaded from the kernel. .El .Ss PMC ROW DISPOSITIONS A PMC row is defined as the set of PMC resources at the same hardware address in the CPUs in a system. Since process scope PMCs need to move between CPUs following their target threads, allocation of a process scope PMC reserves all PMCs in a PMC row for use only with process scope PMCs. Accordingly a PMC row will be in one of the following dispositions: .Bl -tag -width ".Dv PMC_DISP_STANDALONE" -compact .It Dv PMC_DISP_FREE Hardware counters in this row are free and may be use to satisfy either of system scope or process scope allocation requests. .It Dv PMC_DISP_THREAD Hardware counters in this row are in use by process scope PMCs and are only available for process scope allocation requests. .It Dv PMC_DISP_STANDALONE Some hardware counters in this row have been administratively disabled or are in use by system scope PMCs. Non-disabled hardware counters in such a row may be used for satisfying system scope allocation requests. No process scope PMCs will use hardware counters in this row. .El .Sh COMPATIBILITY The API and ABI documented in this manual page may change in the future. This interface is intended to be consumed by the .Xr pmc 3 library; other consumers are unsupported. Applications targeting PMCs should use the .Xr pmc 3 library API. .Sh PROGRAMMING API The .Nm driver operates using a system call number that is dynamically allotted to it when it is loaded into the kernel. .Pp The .Nm driver supports the following operations: .Bl -tag -width indent .It Dv PMC_OP_CONFIGURELOG Configure a log file for PMCs that require a log file. The .Nm driver will write log data to this file asynchronously. If it encounters an error, logging will be stopped and the error code encountered will be saved for subsequent retrieval by a .Dv PMC_OP_FLUSHLOG request. .It Dv PMC_OP_FLUSHLOG Transfer buffered log data inside .Nm to a configured output file. This operation returns to the caller after the write operation has returned. The returned error code reflects any pending error state inside .Nm . .It Dv PMC_OP_GETCPUINFO Retrieve information about the highest possible CPU number for the system, and the number of hardware performance monitoring counters available per CPU. .It Dv PMC_OP_GETDRIVERSTATS Retrieve module statistics (for analyzing the behavior of .Nm itself). .It Dv PMC_OP_GETMODULEVERSION Retrieve the version number of API. .It Dv PMC_OP_GETPMCINFO Retrieve information about the current state of the PMCs on a given CPU. .It Dv PMC_OP_PMCADMIN Set the administrative state (i.e., whether enabled or disabled) for the hardware PMCs managed by the .Nm driver. The invoking process needs to possess the .Dv PRIV_PMC_MANAGE privilege. .It Dv PMC_OP_PMCALLOCATE Allocate and configure a PMC. On successful allocation, a handle to the PMC (a 32 bit value) is returned. .It Dv PMC_OP_PMCATTACH Attach a process mode PMC to a target process. The PMC will be active whenever a thread in the target process is scheduled on a CPU. .Pp If the .Dv PMC_F_DESCENDANTS flag had been specified at PMC allocation time, then the PMC is attached to all current and future descendants of the target process. .It Dv PMC_OP_PMCDETACH Detach a PMC from its target process. .It Dv PMC_OP_PMCRELEASE Release a PMC. .It Dv PMC_OP_PMCRW Read and write a PMC. This operation is valid only for PMCs configured in counting modes. .It Dv PMC_OP_SETCOUNT Set the initial count (for counting mode PMCs) or the desired sampling rate (for sampling mode PMCs). .It Dv PMC_OP_PMCSTART Start a PMC. .It Dv PMC_OP_PMCSTOP Stop a PMC. .It Dv PMC_OP_WRITELOG Insert a timestamped user record into the log file. .El .Ss i386 Specific API Some i386 family CPUs support the RDPMC instruction which allows a user process to read a PMC value without needing to invoke a .Dv PMC_OP_PMCRW operation. On such CPUs, the machine address associated with an allocated PMC is retrievable using the .Dv PMC_OP_PMCX86GETMSR system call. .Bl -tag -width indent .It Dv PMC_OP_PMCX86GETMSR Retrieve the MSR (machine specific register) number associated with the given PMC handle. .Pp The PMC needs to be in process-private mode and allocated without the .Dv PMC_F_DESCENDANTS modifier flag, and should be attached only to its owner process at the time of the call. .El .Ss amd64 Specific API AMD64 CPUs support the RDPMC instruction which allows a user process to read a PMC value without needing to invoke a .Dv PMC_OP_PMCRW operation. The machine address associated with an allocated PMC is retrievable using the .Dv PMC_OP_PMCX86GETMSR system call. .Bl -tag -width indent .It Dv PMC_OP_PMCX86GETMSR Retrieve the MSR (machine specific register) number associated with the given PMC handle. .Pp The PMC needs to be in process-private mode and allocated without the .Dv PMC_F_DESCENDANTS modifier flag, and should be attached only to its owner process at the time of the call. .El .Sh SYSCTL VARIABLES AND LOADER TUNABLES The behavior of .Nm is influenced by the following .Xr sysctl 8 and .Xr loader 8 tunables: .Bl -tag -width indent .It Va kern.hwpmc.callchaindepth Pq integer, read-only The maximum number of call chain records to capture per sample. The default is 8. .It Va kern.hwpmc.debugflags Pq string, read-write (Only available if the .Nm driver was compiled with .Fl DDEBUG . ) Control the verbosity of debug messages from the .Nm driver. .It Va kern.hwpmc.hashsize Pq integer, read-only The number of rows in the hash tables used to keep track of owner and target processes. The default is 16. .It Va kern.hwpmc.logbuffersize Pq integer, read-only The size in kilobytes of each log buffer used by .Nm Ns 's logging function. The default buffer size is 4KB. .It Va kern.hwpmc.mincount Pq integer, read-write The minimum sampling rate for sampling mode PMCs. The default count is 1000 events. .It Va kern.hwpmc.mtxpoolsize Pq integer, read-only The size of the spin mutex pool used by the PMC driver. The default is 32. .It Va kern.hwpmc.nbuffers_pcpu Pq integer, read-only The number of log buffers used by .Nm for logging. The default is 64. .It Va kern.hwpmc.nsamples Pq integer, read-only The number of entries in the per-CPU ring buffer used during sampling. The default is 512. .It Va security.bsd.unprivileged_syspmcs Pq boolean, read-write If set to non-zero, allow unprivileged processes to allocate system-wide PMCs. The default value is 0. .It Va security.bsd.unprivileged_proc_debug Pq boolean, read-write If set to 0, the .Nm driver will only allow privileged processes to attach PMCs to other processes. .El .Pp These variables may be set in the kernel environment using .Xr kenv 1 before .Nm is loaded. .Sh IMPLEMENTATION NOTES .Ss SMP Symmetry The kernel driver requires all physical CPUs in an SMP system to have identical performance monitoring counter hardware. .Ss Sparse CPU Numbering On platforms that sparsely number CPUs and which support hot-plugging of CPUs, requests that specify non-existent or disabled CPUs will fail with an error. Applications allocating system-scope PMCs need to be aware of the possibility of such transient failures. .Ss x86 TSC Handling Historically, on the x86 architecture, .Fx has permitted user processes running at a processor CPL of 3 to read the TSC using the RDTSC instruction. The .Nm driver preserves this behavior. .Ss Intel P4/HTT Handling On CPUs with HTT support, Intel P4 PMCs are capable of qualifying only a subset of hardware events on a per-logical CPU basis. Consequently, if HTT is enabled on a system with Intel Pentium P4 PMCs, then the .Nm driver will reject allocation requests for process-private PMCs that request counting of hardware events that cannot be counted separately for each logical CPU. .Sh DIAGNOSTICS .Bl -diag .It "hwpmc: [class/npmc/capabilities]..." Announce the presence of .Va npmc PMCs of class .Va class , with capabilities described by bit string .Va capabilities . .It "hwpmc: kernel version (0x%x) does not match module version (0x%x)." The module loading process failed because a version mismatch was detected between the currently executing kernel and the module being loaded. .It "hwpmc: this kernel has not been compiled with 'options HWPMC_HOOKS'." The module loading process failed because the currently executing kernel was not configured with the required configuration option .Dv HWPMC_HOOKS . .It "hwpmc: tunable hashsize=%d must be greater than zero." A negative value was supplied for tunable .Va kern.hwpmc.hashsize . .It "hwpmc: tunable logbuffersize=%d must be greater than zero." A negative value was supplied for tunable .Va kern.hwpmc.logbuffersize . .It "hwpmc: tunable nlogbuffers=%d must be greater than zero." A negative value was supplied for tunable .Va kern.hwpmc.nlogbuffers . .It "hwpmc: tunable nsamples=%d out of range." The value for tunable .Va kern.hwpmc.nsamples was negative or greater than 65535. .El +.Sh DEBUGGING +The +.Nm +module can be configured to record trace entries using the +.Xr ktr 4 +interface. +This is useful for debugging the driver's functionality, primarily during +development. +This debugging functionality is not enabled by default, and requires +recompiling the kernel and +.Nm +module after adding the following to the kernel config: +.Bd -literal -offset indent +.Cd options KTR +.Cd options KTR_COMPILE=(KTR_SUBSYS) +.Cd options KTR_MASK=(KTR_SUBSYS) +.Cd options HWPMC_DEBUG +.Ed +.Pp +This alone is not enough to enable tracing; one must also configure the +.Va kern.hwpmc.debugflags +.Xr sysctl 8 +variable, which provides fine-grained control over which types of events are +logged to the trace buffer. +.Pp +.Nm +trace events are grouped by 'major' and 'minor' flag types. +The major flag groups are as follows: +.Pp +.Bl -tag -compact -offset indent +.It CPU +CPU events +.It CSW +Context switch events +.It LOG +Logging events +.It MDP +Machine-dependent/class-dependent events +.It MOD +Miscellaneous events +.It OWN +PMC owner events +.It PMC +PMC management events +.It PRC +Process events +.It SAM +Sampling events +.El +.Pp +The minor flags for each major flag group can vary. +For the full list of flag groups, consult +.In sys/pmc.h . +.Pp +The +.Va kern.hwpmc.debugflags +variable is a string with a custom format. +The string should contain a space-separated list of event specifiers. +Each event specifier consists of the major flag name, followed by an equal sign +(=), followed by a comma-separated list of minor event types. +To track all events for a major group, an asterisk (*) can be given instead. +.Pp +For example, to trace all allocation and release events, set +.Va debugflags +as follows: +.Bd -literal -offset indent +kern.hwpmc.debugflags="pmc=all,rel mdp=all,rel" +.Ed +.Pp +To trace all process events, as well as context switches: +.Bd -literal -offset indent +kern.hwpmc.debugflags="prc=* csw=*" +.Ed +.Pp +To disable all trace events, set the variable to an empty string. +.Bd -literal -offset indent +kern.hwpmc.debugflags="" +.Ed +.Pp +Trace events are recorded by +.Xr ktr 4 , +and can be inspected at run-time using the +.Xr ktrdump 8 +utility, or at the +.Xr ddb 4 +prompt after a panic with the 'show ktr' command. .Sh ERRORS A command issued to the .Nm driver may fail with the following errors: .Bl -tag -width Er .It Bq Er EAGAIN Helper process creation failed for a .Dv PMC_OP_CONFIGURELOG request due to a temporary resource shortage in the kernel. .It Bq Er EBUSY A .Dv PMC_OP_CONFIGURELOG operation was requested while an existing log was active. .It Bq Er EBUSY A DISABLE operation was requested using the .Dv PMC_OP_PMCADMIN request for a set of hardware resources currently in use for process-private PMCs. .It Bq Er EBUSY A .Dv PMC_OP_PMCADMIN operation was requested on an active system mode PMC. .It Bq Er EBUSY A .Dv PMC_OP_PMCATTACH operation was requested for a target process that already had another PMC using the same hardware resources attached to it. .It Bq Er EBUSY A .Dv PMC_OP_PMCRW request writing a new value was issued on a PMC that was active. .It Bq Er EBUSY A .Dv PMC_OP_PMCSETCOUNT request was issued on a PMC that was active. .It Bq Er EDOOFUS A .Dv PMC_OP_PMCSTART operation was requested without a log file being configured for a PMC allocated with .Dv PMC_F_LOG_PROCCSW and .Dv PMC_F_LOG_PROCEXIT modifiers. .It Bq Er EDOOFUS A .Dv PMC_OP_PMCSTART operation was requested on a system-wide sampling PMC without a log file being configured. .It Bq Er EEXIST A .Dv PMC_OP_PMCATTACH request was reissued for a target process that already is the target of this PMC. .It Bq Er EFAULT A bad address was passed in to the driver. .It Bq Er EINVAL An invalid PMC handle was specified. .It Bq Er EINVAL An invalid CPU number was passed in for a .Dv PMC_OP_GETPMCINFO operation. .It Bq Er EINVAL The .Ar pm_flags argument to a .Dv PMC_OP_CONFIGURELOG request contained unknown flags. .It Bq Er EINVAL A .Dv PMC_OP_CONFIGURELOG request to de-configure a log file was issued without a log file being configured. .It Bq Er EINVAL A .Dv PMC_OP_FLUSHLOG request was issued without a log file being configured. .It Bq Er EINVAL An invalid CPU number was passed in for a .Dv PMC_OP_PMCADMIN operation. .It Bq Er EINVAL An invalid operation request was passed in for a .Dv PMC_OP_PMCADMIN operation. .It Bq Er EINVAL An invalid PMC ID was passed in for a .Dv PMC_OP_PMCADMIN operation. .It Bq Er EINVAL A suitable PMC matching the parameters passed in to a .Dv PMC_OP_PMCALLOCATE request could not be allocated. .It Bq Er EINVAL An invalid PMC mode was requested during a .Dv PMC_OP_PMCALLOCATE request. .It Bq Er EINVAL An invalid CPU number was specified during a .Dv PMC_OP_PMCALLOCATE request. .It Bq Er EINVAL A CPU other than .Dv PMC_CPU_ANY was specified in a .Dv PMC_OP_PMCALLOCATE request for a process-private PMC. .It Bq Er EINVAL A CPU number of .Dv PMC_CPU_ANY was specified in a .Dv PMC_OP_PMCALLOCATE request for a system-wide PMC. .It Bq Er EINVAL The .Ar pm_flags argument to an .Dv PMC_OP_PMCALLOCATE request contained unknown flags. .It Bq Er EINVAL (On Intel Pentium 4 CPUs with HTT support) A .Dv PMC_OP_PMCALLOCATE request for a process-private PMC was issued for an event that does not support counting on a per-logical CPU basis. .It Bq Er EINVAL A PMC allocated for system-wide operation was specified with a .Dv PMC_OP_PMCATTACH or .Dv PMC_OP_PMCDETACH request. .It Bq Er EINVAL The .Ar pm_pid argument to a .Dv PMC_OP_PMCATTACH or .Dv PMC_OP_PMCDETACH request specified an illegal process ID. .It Bq Er EINVAL A .Dv PMC_OP_PMCDETACH request was issued for a PMC not attached to the target process. .It Bq Er EINVAL Argument .Ar pm_flags to a .Dv PMC_OP_PMCRW request contained illegal flags. .It Bq Er EINVAL A .Dv PMC_OP_PMCX86GETMSR operation was requested for a PMC not in process-virtual mode, or for a PMC that is not solely attached to its owner process, or for a PMC that was allocated with flag .Dv PMC_F_DESCENDANTS . .It Bq Er EINVAL A .Dv PMC_OP_WRITELOG request was issued for an owner process without a log file configured. .It Bq Er ENOMEM The system was not able to allocate kernel memory. .It Bq Er ENOSYS (On i386 and amd64 architectures) A .Dv PMC_OP_PMCX86GETMSR operation was requested for hardware that does not support reading PMCs directly with the RDPMC instruction. .It Bq Er ENXIO A .Dv PMC_OP_GETPMCINFO operation was requested for an absent or disabled CPU. .It Bq Er ENXIO A .Dv PMC_OP_PMCALLOCATE operation specified allocation of a system-wide PMC on an absent or disabled CPU. .It Bq Er ENXIO A .Dv PMC_OP_PMCSTART or .Dv PMC_OP_PMCSTOP request was issued for a system-wide PMC that was allocated on a CPU that is currently absent or disabled. .It Bq Er EOPNOTSUPP A .Dv PMC_OP_PMCALLOCATE request was issued for PMC capabilities not supported by the specified PMC class. .It Bq Er EOPNOTSUPP (i386 architectures) A sampling mode PMC was requested on a CPU lacking an APIC. .It Bq Er EPERM A .Dv PMC_OP_PMCADMIN request was issued by a process without super-user privilege or by a jailed super-user process. .It Bq Er EPERM A .Dv PMC_OP_PMCATTACH operation was issued for a target process that the current process does not have permission to attach to. .It Bq Er EPERM (i386 and amd64 architectures) A .Dv PMC_OP_PMCATTACH operation was issued on a PMC whose MSR has been retrieved using .Dv PMC_OP_PMCX86GETMSR . .It Bq Er ESRCH A process issued a PMC operation request without having allocated any PMCs. .It Bq Er ESRCH A process issued a PMC operation request after the PMC was detached from all of its target processes. .It Bq Er ESRCH A .Dv PMC_OP_PMCATTACH or .Dv PMC_OP_PMCDETACH request specified a non-existent process ID. .It Bq Er ESRCH The target process for a .Dv PMC_OP_PMCDETACH operation is not being monitored by .Nm . .El .Sh SEE ALSO .Xr kenv 1 , .Xr pmc 3 , .Xr pmclog 3 , +.Xr ddb 4 , +.Xr ktr 4 , .Xr kldload 8 , +.Xr ktrdump 8 , .Xr pmccontrol 8 , .Xr pmcstat 8 , .Xr sysctl 8 , .Xr kproc_create 9 , .Xr p_candebug 9 .Sh HISTORY The .Nm driver first appeared in .Fx 6.0 . .Sh AUTHORS The .Nm driver was written by .An Joseph Koshy Aq Mt jkoshy@FreeBSD.org . .Sh BUGS The driver samples the state of the kernel's logical processor support at the time of initialization (i.e., at module load time). On CPUs supporting logical processors, the driver could misbehave if logical processors are subsequently enabled or disabled while the driver is active. .Pp On the i386 architecture, the driver requires that the local APIC on the CPU be enabled for sampling mode to be supported. Many single-processor motherboards keep the APIC disabled in BIOS; on such systems .Nm will not support sampling PMCs. .Sh SECURITY CONSIDERATIONS PMCs may be used to monitor the actual behavior of the system on hardware. In situations where this constitutes an undesirable information leak, the following options are available: .Bl -enum .It Set the .Xr sysctl 8 tunable .Va security.bsd.unprivileged_syspmcs to 0. This ensures that unprivileged processes cannot allocate system-wide PMCs and thus cannot observe the hardware behavior of the system as a whole. This tunable may also be set at boot time using .Xr loader 8 , or with .Xr kenv 1 prior to loading the .Nm driver into the kernel. .It Set the .Xr sysctl 8 tunable .Va security.bsd.unprivileged_proc_debug to 0. This will ensure that an unprivileged process cannot attach a PMC to any process other than itself and thus cannot observe the hardware behavior of other processes with the same credentials. .El .Pp System administrators should note that on IA-32 platforms .Fx makes the content of the IA-32 TSC counter available to all processes via the RDTSC instruction. diff --git a/sys/sys/pmc.h b/sys/sys/pmc.h index cbb82408dcbf..d1b688df6c2b 100644 --- a/sys/sys/pmc.h +++ b/sys/sys/pmc.h @@ -1,1230 +1,1234 @@ /*- * SPDX-License-Identifier: BSD-2-Clause * * Copyright (c) 2003-2008, Joseph Koshy * Copyright (c) 2007 The FreeBSD Foundation * All rights reserved. * * Portions of this software were developed by A. Joseph Koshy under * sponsorship from the FreeBSD Foundation and Google, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _SYS_PMC_H_ #define _SYS_PMC_H_ #include #include #include #include #include #ifdef _KERNEL #include #include #endif #define PMC_MODULE_NAME "hwpmc" #define PMC_NAME_MAX 64 /* HW counter name size */ #define PMC_CLASS_MAX 8 /* max #classes of PMCs per-system */ /* * Kernel<->userland API version number [MMmmpppp] * * Major numbers are to be incremented when an incompatible change to * the ABI occurs that older clients will not be able to handle. * * Minor numbers are incremented when a backwards compatible change * occurs that allows older correct programs to run unchanged. For * example, when support for a new PMC type is added. * * The patch version is incremented for every bug fix. */ #define PMC_VERSION_MAJOR 0x0A #define PMC_VERSION_MINOR 0x00 #define PMC_VERSION_PATCH 0x0000 #define PMC_VERSION (PMC_VERSION_MAJOR << 24 | \ PMC_VERSION_MINOR << 16 | PMC_VERSION_PATCH) #define PMC_CPUID_LEN 64 /* cpu model name for pmu lookup */ extern char pmc_cpuid[PMC_CPUID_LEN]; /* * Kinds of CPUs known. * * We keep track of CPU variants that need to be distinguished in * some way for PMC operations. CPU names are grouped by manufacturer * and numbered sparsely in order to minimize changes to the ABI involved * when new CPUs are added. * * Please keep the pmc(3) manual page in sync with this list. */ #define __PMC_CPUS() \ __PMC_CPU(AMD_K7, 0x00, "AMD K7") \ __PMC_CPU(AMD_K8, 0x01, "AMD K8") \ __PMC_CPU(INTEL_CORE, 0x87, "Intel Core Solo/Duo") \ __PMC_CPU(INTEL_CORE2, 0x88, "Intel Core2") \ __PMC_CPU(INTEL_CORE2EXTREME, 0x89, "Intel Core2 Extreme") \ __PMC_CPU(INTEL_ATOM, 0x8A, "Intel Atom") \ __PMC_CPU(INTEL_COREI7, 0x8B, "Intel Core i7") \ __PMC_CPU(INTEL_WESTMERE, 0x8C, "Intel Westmere") \ __PMC_CPU(INTEL_SANDYBRIDGE, 0x8D, "Intel Sandy Bridge") \ __PMC_CPU(INTEL_IVYBRIDGE, 0x8E, "Intel Ivy Bridge") \ __PMC_CPU(INTEL_SANDYBRIDGE_XEON, 0x8F, "Intel Sandy Bridge Xeon") \ __PMC_CPU(INTEL_IVYBRIDGE_XEON, 0x90, "Intel Ivy Bridge Xeon") \ __PMC_CPU(INTEL_HASWELL, 0x91, "Intel Haswell") \ __PMC_CPU(INTEL_ATOM_SILVERMONT, 0x92, "Intel Atom Silvermont") \ __PMC_CPU(INTEL_NEHALEM_EX, 0x93, "Intel Nehalem Xeon 7500") \ __PMC_CPU(INTEL_WESTMERE_EX, 0x94, "Intel Westmere Xeon E7") \ __PMC_CPU(INTEL_HASWELL_XEON, 0x95, "Intel Haswell Xeon E5 v3") \ __PMC_CPU(INTEL_BROADWELL, 0x96, "Intel Broadwell") \ __PMC_CPU(INTEL_BROADWELL_XEON, 0x97, "Intel Broadwell Xeon") \ __PMC_CPU(INTEL_SKYLAKE, 0x98, "Intel Skylake") \ __PMC_CPU(INTEL_SKYLAKE_XEON, 0x99, "Intel Skylake Xeon") \ __PMC_CPU(INTEL_ATOM_GOLDMONT, 0x9A, "Intel Atom Goldmont") \ __PMC_CPU(INTEL_ICELAKE, 0x9B, "Intel Icelake") \ __PMC_CPU(INTEL_ICELAKE_XEON, 0x9C, "Intel Icelake Xeon") \ __PMC_CPU(INTEL_ALDERLAKE, 0x9D, "Intel Alderlake") \ __PMC_CPU(INTEL_ATOM_GOLDMONT_P, 0x9E, "Intel Atom Goldmont Plus") \ __PMC_CPU(INTEL_ATOM_TREMONT, 0x9F, "Intel Atom Tremont") \ __PMC_CPU(INTEL_XSCALE, 0x100, "Intel XScale") \ __PMC_CPU(PPC_7450, 0x300, "PowerPC MPC7450") \ __PMC_CPU(PPC_E500, 0x340, "PowerPC e500 Core") \ __PMC_CPU(PPC_970, 0x380, "IBM PowerPC 970") \ __PMC_CPU(PPC_POWER8, 0x390, "IBM POWER8") \ __PMC_CPU(GENERIC, 0x400, "Generic") \ __PMC_CPU(ARMV7_CORTEX_A5, 0x500, "ARMv7 Cortex A5") \ __PMC_CPU(ARMV7_CORTEX_A7, 0x501, "ARMv7 Cortex A7") \ __PMC_CPU(ARMV7_CORTEX_A8, 0x502, "ARMv7 Cortex A8") \ __PMC_CPU(ARMV7_CORTEX_A9, 0x503, "ARMv7 Cortex A9") \ __PMC_CPU(ARMV7_CORTEX_A15, 0x504, "ARMv7 Cortex A15") \ __PMC_CPU(ARMV7_CORTEX_A17, 0x505, "ARMv7 Cortex A17") \ __PMC_CPU(ARMV8_CORTEX_A53, 0x600, "ARMv8 Cortex A53") \ __PMC_CPU(ARMV8_CORTEX_A57, 0x601, "ARMv8 Cortex A57") \ __PMC_CPU(ARMV8_CORTEX_A76, 0x602, "ARMv8 Cortex A76") enum pmc_cputype { #undef __PMC_CPU #define __PMC_CPU(S,V,D) PMC_CPU_##S = V, __PMC_CPUS() }; #define PMC_CPU_FIRST PMC_CPU_AMD_K7 #define PMC_CPU_LAST PMC_CPU_ARMV8_CORTEX_A76 /* * Classes of PMCs */ #define __PMC_CLASSES() \ __PMC_CLASS(TSC, 0x00, "CPU Timestamp counter") \ __PMC_CLASS(K7, 0x01, "AMD K7 performance counters") \ __PMC_CLASS(K8, 0x02, "AMD K8 performance counters") \ __PMC_CLASS(IAF, 0x06, "Intel Core2/Atom, fixed function") \ __PMC_CLASS(IAP, 0x07, "Intel Core...Atom, programmable") \ __PMC_CLASS(UCF, 0x08, "Intel Uncore fixed function") \ __PMC_CLASS(UCP, 0x09, "Intel Uncore programmable") \ __PMC_CLASS(XSCALE, 0x0A, "Intel XScale counters") \ __PMC_CLASS(PPC7450, 0x0D, "Motorola MPC7450 class") \ __PMC_CLASS(PPC970, 0x0E, "IBM PowerPC 970 class") \ __PMC_CLASS(SOFT, 0x0F, "Software events") \ __PMC_CLASS(ARMV7, 0x10, "ARMv7") \ __PMC_CLASS(ARMV8, 0x11, "ARMv8") \ __PMC_CLASS(E500, 0x13, "Freescale e500 class") \ __PMC_CLASS(POWER8, 0x15, "IBM POWER8 class") \ __PMC_CLASS(DMC620_PMU_CD2, 0x16, "ARM DMC620 Memory Controller PMU CLKDIV2") \ __PMC_CLASS(DMC620_PMU_C, 0x17, "ARM DMC620 Memory Controller PMU CLK") \ __PMC_CLASS(CMN600_PMU, 0x18, "Arm CoreLink CMN600 Coherent Mesh Network PMU") enum pmc_class { #undef __PMC_CLASS #define __PMC_CLASS(S,V,D) PMC_CLASS_##S = V, __PMC_CLASSES() }; #define PMC_CLASS_FIRST PMC_CLASS_TSC #define PMC_CLASS_LAST PMC_CLASS_CMN600_PMU /* * A PMC can be in the following states: * * Hardware states: * DISABLED -- administratively prohibited from being used. * FREE -- HW available for use * Software states: * ALLOCATED -- allocated * STOPPED -- allocated, but not counting events * RUNNING -- allocated, and in operation; 'pm_runcount' * holds the number of CPUs using this PMC at * a given instant * DELETED -- being destroyed */ #define __PMC_HWSTATES() \ __PMC_STATE(DISABLED) \ __PMC_STATE(FREE) #define __PMC_SWSTATES() \ __PMC_STATE(ALLOCATED) \ __PMC_STATE(STOPPED) \ __PMC_STATE(RUNNING) \ __PMC_STATE(DELETED) #define __PMC_STATES() \ __PMC_HWSTATES() \ __PMC_SWSTATES() enum pmc_state { #undef __PMC_STATE #define __PMC_STATE(S) PMC_STATE_##S, __PMC_STATES() __PMC_STATE(MAX) }; #define PMC_STATE_FIRST PMC_STATE_DISABLED #define PMC_STATE_LAST PMC_STATE_DELETED /* * An allocated PMC may used as a 'global' counter or as a * 'thread-private' one. Each such mode of use can be in either * statistical sampling mode or in counting mode. Thus a PMC in use * * SS i.e., SYSTEM STATISTICAL -- system-wide statistical profiling * SC i.e., SYSTEM COUNTER -- system-wide counting mode * TS i.e., THREAD STATISTICAL -- thread virtual, statistical profiling * TC i.e., THREAD COUNTER -- thread virtual, counting mode * * Statistical profiling modes rely on the PMC periodically delivering * a interrupt to the CPU (when the configured number of events have * been measured), so the PMC must have the ability to generate * interrupts. * * In counting modes, the PMC counts its configured events, with the * value of the PMC being read whenever needed by its owner process. * * The thread specific modes "virtualize" the PMCs -- the PMCs appear * to be thread private and count events only when the profiled thread * actually executes on the CPU. * * The system-wide "global" modes keep the PMCs running all the time * and are used to measure the behaviour of the whole system. */ #define __PMC_MODES() \ __PMC_MODE(SS, 0) \ __PMC_MODE(SC, 1) \ __PMC_MODE(TS, 2) \ __PMC_MODE(TC, 3) enum pmc_mode { #undef __PMC_MODE #define __PMC_MODE(M,N) PMC_MODE_##M = N, __PMC_MODES() }; #define PMC_MODE_FIRST PMC_MODE_SS #define PMC_MODE_LAST PMC_MODE_TC #define PMC_IS_COUNTING_MODE(mode) \ ((mode) == PMC_MODE_SC || (mode) == PMC_MODE_TC) #define PMC_IS_SYSTEM_MODE(mode) \ ((mode) == PMC_MODE_SS || (mode) == PMC_MODE_SC) #define PMC_IS_SAMPLING_MODE(mode) \ ((mode) == PMC_MODE_SS || (mode) == PMC_MODE_TS) #define PMC_IS_VIRTUAL_MODE(mode) \ ((mode) == PMC_MODE_TS || (mode) == PMC_MODE_TC) /* * PMC row disposition */ #define __PMC_DISPOSITIONS(N) \ __PMC_DISP(STANDALONE) /* global/disabled counters */ \ __PMC_DISP(FREE) /* free/available */ \ __PMC_DISP(THREAD) /* thread-virtual PMCs */ \ __PMC_DISP(UNKNOWN) /* sentinel */ enum pmc_disp { #undef __PMC_DISP #define __PMC_DISP(D) PMC_DISP_##D , __PMC_DISPOSITIONS() }; #define PMC_DISP_FIRST PMC_DISP_STANDALONE #define PMC_DISP_LAST PMC_DISP_THREAD /* * Counter capabilities * * __PMC_CAPS(NAME, VALUE, DESCRIPTION) */ #define __PMC_CAPS() \ __PMC_CAP(INTERRUPT, 0, "generate interrupts") \ __PMC_CAP(USER, 1, "count user-mode events") \ __PMC_CAP(SYSTEM, 2, "count system-mode events") \ __PMC_CAP(EDGE, 3, "do edge detection of events") \ __PMC_CAP(THRESHOLD, 4, "ignore events below a threshold") \ __PMC_CAP(READ, 5, "read PMC counter") \ __PMC_CAP(WRITE, 6, "reprogram PMC counter") \ __PMC_CAP(INVERT, 7, "invert comparison sense") \ __PMC_CAP(QUALIFIER, 8, "further qualify monitored events") \ __PMC_CAP(PRECISE, 9, "perform precise sampling") \ __PMC_CAP(TAGGING, 10, "tag upstream events") \ __PMC_CAP(CASCADE, 11, "cascade counters") \ __PMC_CAP(SYSWIDE, 12, "system wide counter") \ __PMC_CAP(DOMWIDE, 13, "NUMA domain wide counter") enum pmc_caps { #undef __PMC_CAP #define __PMC_CAP(NAME, VALUE, DESCR) PMC_CAP_##NAME = (1 << VALUE) , __PMC_CAPS() }; #define PMC_CAP_FIRST PMC_CAP_INTERRUPT #define PMC_CAP_LAST PMC_CAP_DOMWIDE /* * PMC Event Numbers * * These are generated from the definitions in "dev/hwpmc/pmc_events.h". */ enum pmc_event { #undef __PMC_EV #undef __PMC_EV_BLOCK #define __PMC_EV_BLOCK(C,V) PMC_EV_ ## C ## __BLOCK_START = (V) - 1 , #define __PMC_EV(C,N) PMC_EV_ ## C ## _ ## N , __PMC_EVENTS() }; /* * PMC SYSCALL INTERFACE */ /* * "PMC_OPS" -- these are the commands recognized by the kernel * module, and are used when performing a system call from userland. */ #define __PMC_OPS() \ __PMC_OP(CONFIGURELOG, "Set log file") \ __PMC_OP(FLUSHLOG, "Flush log file") \ __PMC_OP(GETCPUINFO, "Get system CPU information") \ __PMC_OP(GETDRIVERSTATS, "Get driver statistics") \ __PMC_OP(GETMODULEVERSION, "Get module version") \ __PMC_OP(GETPMCINFO, "Get per-cpu PMC information") \ __PMC_OP(PMCADMIN, "Set PMC state") \ __PMC_OP(PMCALLOCATE, "Allocate and configure a PMC") \ __PMC_OP(PMCATTACH, "Attach a PMC to a process") \ __PMC_OP(PMCDETACH, "Detach a PMC from a process") \ __PMC_OP(PMCGETMSR, "Get a PMC's hardware address") \ __PMC_OP(PMCRELEASE, "Release a PMC") \ __PMC_OP(PMCRW, "Read/Set a PMC") \ __PMC_OP(PMCSETCOUNT, "Set initial count/sampling rate") \ __PMC_OP(PMCSTART, "Start a PMC") \ __PMC_OP(PMCSTOP, "Stop a PMC") \ __PMC_OP(WRITELOG, "Write a cookie to the log file") \ __PMC_OP(CLOSELOG, "Close log file") \ __PMC_OP(GETDYNEVENTINFO, "Get dynamic events list") enum pmc_ops { #undef __PMC_OP #define __PMC_OP(N, D) PMC_OP_##N, __PMC_OPS() }; /* * Flags used in operations on PMCs. */ #define PMC_F_UNUSED1 0x00000001 /* unused */ #define PMC_F_DESCENDANTS 0x00000002 /*OP ALLOCATE track descendants */ #define PMC_F_LOG_PROCCSW 0x00000004 /*OP ALLOCATE track ctx switches */ #define PMC_F_LOG_PROCEXIT 0x00000008 /*OP ALLOCATE log proc exits */ #define PMC_F_NEWVALUE 0x00000010 /*OP RW write new value */ #define PMC_F_OLDVALUE 0x00000020 /*OP RW get old value */ /* V2 API */ #define PMC_F_CALLCHAIN 0x00000080 /*OP ALLOCATE capture callchains */ #define PMC_F_USERCALLCHAIN 0x00000100 /*OP ALLOCATE use userspace stack */ /* internal flags */ #define PMC_F_ATTACHED_TO_OWNER 0x00010000 /*attached to owner*/ #define PMC_F_NEEDS_LOGFILE 0x00020000 /*needs log file */ #define PMC_F_ATTACH_DONE 0x00040000 /*attached at least once */ #define PMC_CALLCHAIN_DEPTH_MAX 512 #define PMC_CC_F_USERSPACE 0x01 /*userspace callchain*/ /* * Cookies used to denote allocated PMCs, and the values of PMCs. */ typedef uint32_t pmc_id_t; typedef uint64_t pmc_value_t; #define PMC_ID_INVALID (~ (pmc_id_t) 0) /* * PMC IDs have the following format: * * +-----------------------+-------+-----------+ * | CPU | PMC MODE | CLASS | ROW INDEX | * +-----------------------+-------+-----------+ * * where CPU is 12 bits, MODE 4, CLASS 8, and ROW INDEX 8 Field 'CPU' * is set to the requested CPU for system-wide PMCs or PMC_CPU_ANY for * process-mode PMCs. Field 'PMC MODE' is the allocated PMC mode. * Field 'PMC CLASS' is the class of the PMC. Field 'ROW INDEX' is the * row index for the PMC. * * The 'ROW INDEX' ranges over 0..NWPMCS where NHWPMCS is the total * number of hardware PMCs on this cpu. */ #define PMC_ID_TO_ROWINDEX(ID) ((ID) & 0xFF) #define PMC_ID_TO_CLASS(ID) (((ID) & 0xFF00) >> 8) #define PMC_ID_TO_MODE(ID) (((ID) & 0xF0000) >> 16) #define PMC_ID_TO_CPU(ID) (((ID) & 0xFFF00000) >> 20) #define PMC_ID_MAKE_ID(CPU,MODE,CLASS,ROWINDEX) \ ((((CPU) & 0xFFF) << 20) | (((MODE) & 0xF) << 16) | \ (((CLASS) & 0xFF) << 8) | ((ROWINDEX) & 0xFF)) /* * Data structures for system calls supported by the pmc driver. */ /* * OP PMCALLOCATE * * Allocate a PMC on the named CPU. */ #define PMC_CPU_ANY ~0 struct pmc_op_pmcallocate { uint32_t pm_caps; /* PMC_CAP_* */ uint32_t pm_cpu; /* CPU number or PMC_CPU_ANY */ enum pmc_class pm_class; /* class of PMC desired */ enum pmc_event pm_ev; /* [enum pmc_event] desired */ uint32_t pm_flags; /* additional modifiers PMC_F_* */ enum pmc_mode pm_mode; /* desired mode */ pmc_id_t pm_pmcid; /* [return] process pmc id */ pmc_value_t pm_count; /* initial/sample count */ union pmc_md_op_pmcallocate pm_md; /* MD layer extensions */ }; /* * OP PMCADMIN * * Set the administrative state (i.e., whether enabled or disabled) of * a PMC 'pm_pmc' on CPU 'pm_cpu'. Note that 'pm_pmc' specifies an * absolute PMC number and need not have been first allocated by the * calling process. */ struct pmc_op_pmcadmin { int pm_cpu; /* CPU# */ uint32_t pm_flags; /* flags */ int pm_pmc; /* PMC# */ enum pmc_state pm_state; /* desired state */ }; /* * OP PMCATTACH / OP PMCDETACH * * Attach/detach a PMC and a process. */ struct pmc_op_pmcattach { pmc_id_t pm_pmc; /* PMC to attach to */ pid_t pm_pid; /* target process */ }; /* * OP PMCSETCOUNT * * Set the sampling rate (i.e., the reload count) for statistical counters. * 'pm_pmcid' need to have been previously allocated using PMCALLOCATE. */ struct pmc_op_pmcsetcount { pmc_value_t pm_count; /* initial/sample count */ pmc_id_t pm_pmcid; /* PMC id to set */ }; /* * OP PMCRW * * Read the value of a PMC named by 'pm_pmcid'. 'pm_pmcid' needs * to have been previously allocated using PMCALLOCATE. */ struct pmc_op_pmcrw { uint32_t pm_flags; /* PMC_F_{OLD,NEW}VALUE*/ pmc_id_t pm_pmcid; /* pmc id */ pmc_value_t pm_value; /* new&returned value */ }; /* * OP GETPMCINFO * * retrieve PMC state for a named CPU. The caller is expected to * allocate 'npmc' * 'struct pmc_info' bytes of space for the return * values. */ struct pmc_info { char pm_name[PMC_NAME_MAX]; /* pmc name */ enum pmc_class pm_class; /* enum pmc_class */ int pm_enabled; /* whether enabled */ enum pmc_disp pm_rowdisp; /* FREE, THREAD or STANDLONE */ pid_t pm_ownerpid; /* owner, or -1 */ enum pmc_mode pm_mode; /* current mode [enum pmc_mode] */ enum pmc_event pm_event; /* current event */ uint32_t pm_flags; /* current flags */ pmc_value_t pm_reloadcount; /* sampling counters only */ }; struct pmc_op_getpmcinfo { int32_t pm_cpu; /* 0 <= cpu < mp_maxid */ struct pmc_info pm_pmcs[]; /* space for 'npmc' structures */ }; /* * OP GETCPUINFO * * Retrieve system CPU information. */ struct pmc_classinfo { enum pmc_class pm_class; /* class id */ uint32_t pm_caps; /* counter capabilities */ uint32_t pm_width; /* width of the PMC */ uint32_t pm_num; /* number of PMCs in class */ }; struct pmc_op_getcpuinfo { enum pmc_cputype pm_cputype; /* what kind of CPU */ uint32_t pm_ncpu; /* max CPU number */ uint32_t pm_npmc; /* #PMCs per CPU */ uint32_t pm_nclass; /* #classes of PMCs */ struct pmc_classinfo pm_classes[PMC_CLASS_MAX]; }; /* * OP CONFIGURELOG * * Configure a log file for writing system-wide statistics to. */ struct pmc_op_configurelog { int pm_flags; int pm_logfd; /* logfile fd (or -1) */ }; /* * OP GETDRIVERSTATS * * Retrieve pmc(4) driver-wide statistics. */ #ifdef _KERNEL struct pmc_driverstats { counter_u64_t pm_intr_ignored; /* #interrupts ignored */ counter_u64_t pm_intr_processed; /* #interrupts processed */ counter_u64_t pm_intr_bufferfull; /* #interrupts with ENOSPC */ counter_u64_t pm_syscalls; /* #syscalls */ counter_u64_t pm_syscall_errors; /* #syscalls with errors */ counter_u64_t pm_buffer_requests; /* #buffer requests */ counter_u64_t pm_buffer_requests_failed; /* #failed buffer requests */ counter_u64_t pm_log_sweeps; /* #sample buffer processing passes */ counter_u64_t pm_merges; /* merged k+u */ counter_u64_t pm_overwrites; /* UR overwrites */ }; #endif struct pmc_op_getdriverstats { unsigned int pm_intr_ignored; /* #interrupts ignored */ unsigned int pm_intr_processed; /* #interrupts processed */ unsigned int pm_intr_bufferfull; /* #interrupts with ENOSPC */ unsigned int pm_syscalls; /* #syscalls */ unsigned int pm_syscall_errors; /* #syscalls with errors */ unsigned int pm_buffer_requests; /* #buffer requests */ unsigned int pm_buffer_requests_failed; /* #failed buffer requests */ unsigned int pm_log_sweeps; /* #sample buffer processing passes */ }; /* * OP RELEASE / OP START / OP STOP * * Simple operations on a PMC id. */ struct pmc_op_simple { pmc_id_t pm_pmcid; }; /* * OP WRITELOG * * Flush the current log buffer and write 4 bytes of user data to it. */ struct pmc_op_writelog { uint32_t pm_userdata; }; /* * OP GETMSR * * Retrieve the machine specific address associated with the allocated * PMC. This number can be used subsequently with a read-performance-counter * instruction. */ struct pmc_op_getmsr { uint32_t pm_msr; /* machine specific address */ pmc_id_t pm_pmcid; /* allocated pmc id */ }; /* * OP GETDYNEVENTINFO * * Retrieve a PMC dynamic class events list. */ struct pmc_dyn_event_descr { char pm_ev_name[PMC_NAME_MAX]; enum pmc_event pm_ev_code; }; struct pmc_op_getdyneventinfo { enum pmc_class pm_class; unsigned int pm_nevent; struct pmc_dyn_event_descr pm_events[PMC_EV_DYN_COUNT]; }; #ifdef _KERNEL #include #include #include #include #define PMC_HASH_SIZE 1024 #define PMC_MTXPOOL_SIZE 2048 #define PMC_LOG_BUFFER_SIZE 256 #define PMC_NLOGBUFFERS_PCPU 32 #define PMC_NSAMPLES 256 #define PMC_CALLCHAIN_DEPTH 128 #define PMC_THREADLIST_MAX 128 #define PMC_SYSCTL_NAME_PREFIX "kern." PMC_MODULE_NAME "." /* * Locking keys * * (b) - pmc_bufferlist_mtx (spin lock) * (k) - pmc_kthread_mtx (sleep lock) * (o) - po->po_mtx (spin lock) * (g) - global_epoch_preempt (epoch) * (p) - pmc_sx (sx) */ /* * PMC commands */ struct pmc_syscall_args { register_t pmop_code; /* one of PMC_OP_* */ void *pmop_data; /* syscall parameter */ }; /* * Interface to processor specific s1tuff */ /* * struct pmc_descr * * Machine independent (i.e., the common parts) of a human readable * PMC description. */ struct pmc_descr { char pd_name[PMC_NAME_MAX]; /* name */ uint32_t pd_caps; /* capabilities */ enum pmc_class pd_class; /* class of the PMC */ uint32_t pd_width; /* width in bits */ }; /* * struct pmc_target * * This structure records all the target processes associated with a * PMC. */ struct pmc_target { LIST_ENTRY(pmc_target) pt_next; struct pmc_process *pt_process; /* target descriptor */ }; /* * struct pmc * * Describes each allocated PMC. * * Each PMC has precisely one owner, namely the process that allocated * the PMC. * * A PMC may be attached to multiple target processes. The * 'pm_targets' field links all the target processes being monitored * by this PMC. * * The 'pm_savedvalue' field is protected by a mutex. * * On a multi-cpu machine, multiple target threads associated with a * process-virtual PMC could be concurrently executing on different * CPUs. The 'pm_runcount' field is atomically incremented every time * the PMC gets scheduled on a CPU and atomically decremented when it * get descheduled. Deletion of a PMC is only permitted when this * field is '0'. * */ struct pmc_pcpu_state { uint32_t pps_overflowcnt; /* count overflow interrupts */ uint8_t pps_stalled; uint8_t pps_cpustate; } __aligned(CACHE_LINE_SIZE); struct pmc { LIST_HEAD(,pmc_target) pm_targets; /* list of target processes */ LIST_ENTRY(pmc) pm_next; /* owner's list */ /* * System-wide PMCs are allocated on a CPU and are not moved * around. For system-wide PMCs we record the CPU the PMC was * allocated on in the 'CPU' field of the pmc ID. * * Virtual PMCs run on whichever CPU is currently executing * their targets' threads. For these PMCs we need to save * their current PMC counter values when they are taken off * CPU. */ union { pmc_value_t pm_savedvalue; /* Virtual PMCS */ } pm_gv; /* * For sampling mode PMCs, we keep track of the PMC's "reload * count", which is the counter value to be loaded in when * arming the PMC for the next counting session. For counting * modes on PMCs that are read-only (e.g., the x86 TSC), we * keep track of the initial value at the start of * counting-mode operation. */ union { pmc_value_t pm_reloadcount; /* sampling PMC modes */ pmc_value_t pm_initial; /* counting PMC modes */ } pm_sc; struct pmc_pcpu_state *pm_pcpu_state; volatile cpuset_t pm_cpustate; /* CPUs where PMC should be active */ uint32_t pm_caps; /* PMC capabilities */ enum pmc_event pm_event; /* event being measured */ uint32_t pm_flags; /* additional flags PMC_F_... */ struct pmc_owner *pm_owner; /* owner thread state */ counter_u64_t pm_runcount; /* #cpus currently on */ enum pmc_state pm_state; /* current PMC state */ /* * The PMC ID field encodes the row-index for the PMC, its * mode, class and the CPU# associated with the PMC. */ pmc_id_t pm_id; /* allocated PMC id */ enum pmc_class pm_class; /* md extensions */ union pmc_md_pmc pm_md; }; /* * Accessor macros for 'struct pmc' */ #define PMC_TO_MODE(P) PMC_ID_TO_MODE((P)->pm_id) #define PMC_TO_CLASS(P) PMC_ID_TO_CLASS((P)->pm_id) #define PMC_TO_ROWINDEX(P) PMC_ID_TO_ROWINDEX((P)->pm_id) #define PMC_TO_CPU(P) PMC_ID_TO_CPU((P)->pm_id) /* * struct pmc_threadpmcstate * * Record per-PMC, per-thread state. */ struct pmc_threadpmcstate { pmc_value_t pt_pmcval; /* per-thread reload count */ }; /* * struct pmc_thread * * Record a 'target' thread being profiled. */ struct pmc_thread { LIST_ENTRY(pmc_thread) pt_next; /* linked list */ struct thread *pt_td; /* target thread */ struct pmc_threadpmcstate pt_pmcs[]; /* per-PMC state */ }; /* * struct pmc_process * * Record a 'target' process being profiled. * * The target process being profiled could be different from the owner * process which allocated the PMCs. Each target process descriptor * is associated with NHWPMC 'struct pmc *' pointers. Each PMC at a * given hardware row-index 'n' will use slot 'n' of the 'pp_pmcs[]' * array. The size of this structure is thus PMC architecture * dependent. * */ struct pmc_targetstate { struct pmc *pp_pmc; /* target PMC */ pmc_value_t pp_pmcval; /* per-process value */ }; struct pmc_process { LIST_ENTRY(pmc_process) pp_next; /* hash chain */ LIST_HEAD(,pmc_thread) pp_tds; /* list of threads */ struct mtx *pp_tdslock; /* lock on pp_tds thread list */ int pp_refcnt; /* reference count */ uint32_t pp_flags; /* flags PMC_PP_* */ struct proc *pp_proc; /* target process */ struct pmc_targetstate pp_pmcs[]; /* NHWPMCs */ }; #define PMC_PP_ENABLE_MSR_ACCESS 0x00000001 /* * struct pmc_owner * * We associate a PMC with an 'owner' process. * * A process can be associated with 0..NCPUS*NHWPMC PMCs during its * lifetime, where NCPUS is the numbers of CPUS in the system and * NHWPMC is the number of hardware PMCs per CPU. These are * maintained in the list headed by the 'po_pmcs' to save on space. * */ struct pmc_owner { LIST_ENTRY(pmc_owner) po_next; /* hash chain */ CK_LIST_ENTRY(pmc_owner) po_ssnext; /* (g/p) list of SS PMC owners */ LIST_HEAD(, pmc) po_pmcs; /* owned PMC list */ TAILQ_HEAD(, pmclog_buffer) po_logbuffers; /* (o) logbuffer list */ struct mtx po_mtx; /* spin lock for (o) */ struct proc *po_owner; /* owner proc */ uint32_t po_flags; /* (k) flags PMC_PO_* */ struct proc *po_kthread; /* (k) helper kthread */ struct file *po_file; /* file reference */ int po_error; /* recorded error */ short po_sscount; /* # SS PMCs owned */ short po_logprocmaps; /* global mappings done */ struct pmclog_buffer *po_curbuf[MAXCPU]; /* current log buffer */ }; #define PMC_PO_OWNS_LOGFILE 0x00000001 /* has a log file */ #define PMC_PO_SHUTDOWN 0x00000010 /* in the process of shutdown */ #define PMC_PO_INITIAL_MAPPINGS_DONE 0x00000020 /* * struct pmc_hw -- describe the state of the PMC hardware * * When in use, a HW PMC is associated with one allocated 'struct pmc' * pointed to by field 'phw_pmc'. When inactive, this field is NULL. * * On an SMP box, one or more HW PMC's in process virtual mode with * the same 'phw_pmc' could be executing on different CPUs. In order * to handle this case correctly, we need to ensure that only * incremental counts get added to the saved value in the associated * 'struct pmc'. The 'phw_save' field is used to keep the saved PMC * value at the time the hardware is started during this context * switch (i.e., the difference between the new (hardware) count and * the saved count is atomically added to the count field in 'struct * pmc' at context switch time). * */ struct pmc_hw { uint32_t phw_state; /* see PHW_* macros below */ struct pmc *phw_pmc; /* current thread PMC */ }; #define PMC_PHW_RI_MASK 0x000000FF #define PMC_PHW_CPU_SHIFT 8 #define PMC_PHW_CPU_MASK 0x0000FF00 #define PMC_PHW_FLAGS_SHIFT 16 #define PMC_PHW_FLAGS_MASK 0xFFFF0000 #define PMC_PHW_INDEX_TO_STATE(ri) ((ri) & PMC_PHW_RI_MASK) #define PMC_PHW_STATE_TO_INDEX(state) ((state) & PMC_PHW_RI_MASK) #define PMC_PHW_CPU_TO_STATE(cpu) (((cpu) << PMC_PHW_CPU_SHIFT) & \ PMC_PHW_CPU_MASK) #define PMC_PHW_STATE_TO_CPU(state) (((state) & PMC_PHW_CPU_MASK) >> \ PMC_PHW_CPU_SHIFT) #define PMC_PHW_FLAGS_TO_STATE(flags) (((flags) << PMC_PHW_FLAGS_SHIFT) & \ PMC_PHW_FLAGS_MASK) #define PMC_PHW_STATE_TO_FLAGS(state) (((state) & PMC_PHW_FLAGS_MASK) >> \ PMC_PHW_FLAGS_SHIFT) #define PMC_PHW_FLAG_IS_ENABLED (PMC_PHW_FLAGS_TO_STATE(0x01)) #define PMC_PHW_FLAG_IS_SHAREABLE (PMC_PHW_FLAGS_TO_STATE(0x02)) /* * struct pmc_sample * * Space for N (tunable) PC samples and associated control data. */ struct pmc_sample { uint16_t ps_nsamples; /* callchain depth */ uint16_t ps_nsamples_actual; uint16_t ps_cpu; /* cpu number */ uint16_t ps_flags; /* other flags */ lwpid_t ps_tid; /* thread id */ pid_t ps_pid; /* process PID or -1 */ int ps_ticks; /* ticks at sample time */ /* pad */ struct thread *ps_td; /* which thread */ struct pmc *ps_pmc; /* interrupting PMC */ uintptr_t *ps_pc; /* (const) callchain start */ uint64_t ps_tsc; /* tsc value */ }; #define PMC_SAMPLE_FREE ((uint16_t) 0) #define PMC_USER_CALLCHAIN_PENDING ((uint16_t) 0xFFFF) struct pmc_samplebuffer { volatile uint64_t ps_prodidx; /* producer index */ volatile uint64_t ps_considx; /* consumer index */ uintptr_t *ps_callchains; /* all saved call chains */ struct pmc_sample ps_samples[]; /* array of sample entries */ }; #define PMC_CONS_SAMPLE(psb) \ (&(psb)->ps_samples[(psb)->ps_considx & pmc_sample_mask]) #define PMC_CONS_SAMPLE_OFF(psb, off) \ (&(psb)->ps_samples[(off) & pmc_sample_mask]) #define PMC_PROD_SAMPLE(psb) \ (&(psb)->ps_samples[(psb)->ps_prodidx & pmc_sample_mask]) /* * struct pmc_cpustate * * A CPU is modelled as a collection of HW PMCs with space for additional * flags. */ struct pmc_cpu { uint32_t pc_state; /* physical cpu number + flags */ struct pmc_samplebuffer *pc_sb[3]; /* space for samples */ struct pmc_hw *pc_hwpmcs[]; /* 'npmc' pointers */ }; #define PMC_PCPU_CPU_MASK 0x000000FF #define PMC_PCPU_FLAGS_MASK 0xFFFFFF00 #define PMC_PCPU_FLAGS_SHIFT 8 #define PMC_PCPU_STATE_TO_CPU(S) ((S) & PMC_PCPU_CPU_MASK) #define PMC_PCPU_STATE_TO_FLAGS(S) (((S) & PMC_PCPU_FLAGS_MASK) >> PMC_PCPU_FLAGS_SHIFT) #define PMC_PCPU_FLAGS_TO_STATE(F) (((F) << PMC_PCPU_FLAGS_SHIFT) & PMC_PCPU_FLAGS_MASK) #define PMC_PCPU_CPU_TO_STATE(C) ((C) & PMC_PCPU_CPU_MASK) #define PMC_PCPU_FLAG_HTT (PMC_PCPU_FLAGS_TO_STATE(0x1)) /* * struct pmc_binding * * CPU binding information. */ struct pmc_binding { int pb_bound; /* is bound? */ int pb_cpu; /* if so, to which CPU */ u_char pb_priority; /* Thread active priority. */ }; struct pmc_mdep; /* * struct pmc_classdep * * PMC class-dependent operations. */ struct pmc_classdep { uint32_t pcd_caps; /* class capabilities */ enum pmc_class pcd_class; /* class id */ int pcd_num; /* number of PMCs */ int pcd_ri; /* row index of the first PMC in class */ int pcd_width; /* width of the PMC */ /* configuring/reading/writing the hardware PMCs */ int (*pcd_config_pmc)(int _cpu, int _ri, struct pmc *_pm); int (*pcd_get_config)(int _cpu, int _ri, struct pmc **_ppm); int (*pcd_read_pmc)(int _cpu, int _ri, struct pmc *_pm, pmc_value_t *_value); int (*pcd_write_pmc)(int _cpu, int _ri, struct pmc *_pm, pmc_value_t _value); /* pmc allocation/release */ int (*pcd_allocate_pmc)(int _cpu, int _ri, struct pmc *_t, const struct pmc_op_pmcallocate *_a); int (*pcd_release_pmc)(int _cpu, int _ri, struct pmc *_pm); /* starting and stopping PMCs */ int (*pcd_start_pmc)(int _cpu, int _ri, struct pmc *_pm); int (*pcd_stop_pmc)(int _cpu, int _ri, struct pmc *_pm); /* description */ int (*pcd_describe)(int _cpu, int _ri, struct pmc_info *_pi, struct pmc **_ppmc); /* class-dependent initialization & finalization */ int (*pcd_pcpu_init)(struct pmc_mdep *_md, int _cpu); int (*pcd_pcpu_fini)(struct pmc_mdep *_md, int _cpu); /* machine-specific interface */ int (*pcd_get_msr)(int _ri, uint32_t *_msr); }; /* * struct pmc_mdep * * Machine dependent bits needed per CPU type. */ struct pmc_mdep { uint32_t pmd_cputype; /* from enum pmc_cputype */ uint32_t pmd_npmc; /* number of PMCs per CPU */ uint32_t pmd_nclass; /* number of PMC classes present */ /* * Machine dependent methods. */ /* thread context switch in/out */ int (*pmd_switch_in)(struct pmc_cpu *_p, struct pmc_process *_pp); int (*pmd_switch_out)(struct pmc_cpu *_p, struct pmc_process *_pp); /* handle a PMC interrupt */ int (*pmd_intr)(struct trapframe *_tf); /* * PMC class dependent information. */ struct pmc_classdep pmd_classdep[]; }; /* * Per-CPU state. This is an array of 'mp_ncpu' pointers * to struct pmc_cpu descriptors. */ extern struct pmc_cpu **pmc_pcpu; /* driver statistics */ extern struct pmc_driverstats pmc_stats; #if defined(HWPMC_DEBUG) /* HWPMC_DEBUG without KTR will compile but is a no-op. */ #if !defined(KTR) || !defined(KTR_COMPILE) || ((KTR_COMPILE & KTR_SUBSYS) == 0) #error "HWPMC_DEBUG requires KTR and KTR_COMPILE=KTR_SUBSYS -- see ktr(4)" #endif #include #define __pmcdbg_used /* unused variable annotation */ -/* debug flags, major flag groups */ +/* + * Debug flags, major flag groups. + * + * Please keep the DEBUGGING section of the hwpmc(4) man page in sync. + */ struct pmc_debugflags { int pdb_CPU; int pdb_CSW; int pdb_LOG; int pdb_MDP; int pdb_MOD; int pdb_OWN; int pdb_PMC; int pdb_PRC; int pdb_SAM; }; extern struct pmc_debugflags pmc_debugflags; #define KTR_PMC KTR_SUBSYS #define PMC_DEBUG_STRSIZE 128 #define PMC_DEBUG_DEFAULT_FLAGS { 0, 0, 0, 0, 0, 0, 0, 0, 0 } #define PMCDBG0(M, N, L, F) do { \ if (pmc_debugflags.pdb_ ## M & (1 << PMC_DEBUG_MIN_ ## N)) \ CTR0(KTR_PMC, #M ":" #N ":" #L ": " F); \ } while (0) #define PMCDBG1(M, N, L, F, p1) do { \ if (pmc_debugflags.pdb_ ## M & (1 << PMC_DEBUG_MIN_ ## N)) \ CTR1(KTR_PMC, #M ":" #N ":" #L ": " F, p1); \ } while (0) #define PMCDBG2(M, N, L, F, p1, p2) do { \ if (pmc_debugflags.pdb_ ## M & (1 << PMC_DEBUG_MIN_ ## N)) \ CTR2(KTR_PMC, #M ":" #N ":" #L ": " F, p1, p2); \ } while (0) #define PMCDBG3(M, N, L, F, p1, p2, p3) do { \ if (pmc_debugflags.pdb_ ## M & (1 << PMC_DEBUG_MIN_ ## N)) \ CTR3(KTR_PMC, #M ":" #N ":" #L ": " F, p1, p2, p3); \ } while (0) #define PMCDBG4(M, N, L, F, p1, p2, p3, p4) do { \ if (pmc_debugflags.pdb_ ## M & (1 << PMC_DEBUG_MIN_ ## N)) \ CTR4(KTR_PMC, #M ":" #N ":" #L ": " F, p1, p2, p3, p4);\ } while (0) #define PMCDBG5(M, N, L, F, p1, p2, p3, p4, p5) do { \ if (pmc_debugflags.pdb_ ## M & (1 << PMC_DEBUG_MIN_ ## N)) \ CTR5(KTR_PMC, #M ":" #N ":" #L ": " F, p1, p2, p3, p4, \ p5); \ } while (0) #define PMCDBG6(M, N, L, F, p1, p2, p3, p4, p5, p6) do { \ if (pmc_debugflags.pdb_ ## M & (1 << PMC_DEBUG_MIN_ ## N)) \ CTR6(KTR_PMC, #M ":" #N ":" #L ": " F, p1, p2, p3, p4, \ p5, p6); \ } while (0) /* Major numbers */ #define PMC_DEBUG_MAJ_CPU 0 /* cpu switches */ #define PMC_DEBUG_MAJ_CSW 1 /* context switches */ #define PMC_DEBUG_MAJ_LOG 2 /* logging */ #define PMC_DEBUG_MAJ_MDP 3 /* machine dependent */ #define PMC_DEBUG_MAJ_MOD 4 /* misc module infrastructure */ #define PMC_DEBUG_MAJ_OWN 5 /* owner */ #define PMC_DEBUG_MAJ_PMC 6 /* pmc management */ #define PMC_DEBUG_MAJ_PRC 7 /* processes */ #define PMC_DEBUG_MAJ_SAM 8 /* sampling */ /* Minor numbers */ /* Common (8 bits) */ #define PMC_DEBUG_MIN_ALL 0 /* allocation */ #define PMC_DEBUG_MIN_REL 1 /* release */ #define PMC_DEBUG_MIN_OPS 2 /* ops: start, stop, ... */ #define PMC_DEBUG_MIN_INI 3 /* init */ #define PMC_DEBUG_MIN_FND 4 /* find */ /* MODULE */ #define PMC_DEBUG_MIN_PMH 14 /* pmc_hook */ #define PMC_DEBUG_MIN_PMS 15 /* pmc_syscall */ /* OWN */ #define PMC_DEBUG_MIN_ORM 8 /* owner remove */ #define PMC_DEBUG_MIN_OMR 9 /* owner maybe remove */ /* PROCESSES */ #define PMC_DEBUG_MIN_TLK 8 /* link target */ #define PMC_DEBUG_MIN_TUL 9 /* unlink target */ #define PMC_DEBUG_MIN_EXT 10 /* process exit */ #define PMC_DEBUG_MIN_EXC 11 /* process exec */ #define PMC_DEBUG_MIN_FRK 12 /* process fork */ #define PMC_DEBUG_MIN_ATT 13 /* attach/detach */ #define PMC_DEBUG_MIN_SIG 14 /* signalling */ /* CONTEXT SWITCHES */ #define PMC_DEBUG_MIN_SWI 8 /* switch in */ #define PMC_DEBUG_MIN_SWO 9 /* switch out */ /* PMC */ #define PMC_DEBUG_MIN_REG 8 /* pmc register */ #define PMC_DEBUG_MIN_ALR 9 /* allocate row */ /* MACHINE DEPENDENT LAYER */ #define PMC_DEBUG_MIN_REA 8 /* read */ #define PMC_DEBUG_MIN_WRI 9 /* write */ #define PMC_DEBUG_MIN_CFG 10 /* config */ #define PMC_DEBUG_MIN_STA 11 /* start */ #define PMC_DEBUG_MIN_STO 12 /* stop */ #define PMC_DEBUG_MIN_INT 13 /* interrupts */ /* CPU */ #define PMC_DEBUG_MIN_BND 8 /* bind */ #define PMC_DEBUG_MIN_SEL 9 /* select */ /* LOG */ #define PMC_DEBUG_MIN_GTB 8 /* get buf */ #define PMC_DEBUG_MIN_SIO 9 /* schedule i/o */ #define PMC_DEBUG_MIN_FLS 10 /* flush */ #define PMC_DEBUG_MIN_SAM 11 /* sample */ #define PMC_DEBUG_MIN_CLO 12 /* close */ #else #define __pmcdbg_used __unused #define PMCDBG0(M, N, L, F) /* nothing */ #define PMCDBG1(M, N, L, F, p1) #define PMCDBG2(M, N, L, F, p1, p2) #define PMCDBG3(M, N, L, F, p1, p2, p3) #define PMCDBG4(M, N, L, F, p1, p2, p3, p4) #define PMCDBG5(M, N, L, F, p1, p2, p3, p4, p5) #define PMCDBG6(M, N, L, F, p1, p2, p3, p4, p5, p6) #endif /* declare a dedicated memory pool */ MALLOC_DECLARE(M_PMC); /* * Functions */ struct pmc_mdep *pmc_md_initialize(void); /* MD init function */ void pmc_md_finalize(struct pmc_mdep *_md); /* MD fini function */ int pmc_getrowdisp(int _ri); int pmc_process_interrupt(int _ring, struct pmc *_pm, struct trapframe *_tf); int pmc_save_kernel_callchain(uintptr_t *_cc, int _maxsamples, struct trapframe *_tf); int pmc_save_user_callchain(uintptr_t *_cc, int _maxsamples, struct trapframe *_tf); void pmc_restore_cpu_binding(struct pmc_binding *pb); void pmc_save_cpu_binding(struct pmc_binding *pb); void pmc_select_cpu(int cpu); struct pmc_mdep *pmc_mdep_alloc(int nclasses); void pmc_mdep_free(struct pmc_mdep *md); uint64_t pmc_rdtsc(void); #endif /* _KERNEL */ #endif /* _SYS_PMC_H_ */