Changeset View
Changeset View
Standalone View
Standalone View
share/man/man4/numa.4
Show All 28 Lines | |||||
.Os | .Os | ||||
.Sh NAME | .Sh NAME | ||||
.Nm NUMA | .Nm NUMA | ||||
.Nd Non-Uniform Memory Access | .Nd Non-Uniform Memory Access | ||||
.Sh SYNOPSIS | .Sh SYNOPSIS | ||||
.Cd options SMP | .Cd options SMP | ||||
.Cd options MAXMEMDOM=16 | .Cd options MAXMEMDOM=16 | ||||
.Pp | .Pp | ||||
.In sys/numa.h | |||||
.In sys/cpuset.h | .In sys/cpuset.h | ||||
.In sys/bus.h | .In sys/bus.h | ||||
.Sh DESCRIPTION | .Sh DESCRIPTION | ||||
Non-Uniform Memory Access is a computer architecture design which | Non-Uniform Memory Access is a computer architecture design which | ||||
involves unequal costs between processors, memory and IO devices | involves unequal costs between processors, memory and IO devices | ||||
in a given system. | in a given system. | ||||
.Pp | .Pp | ||||
In a | In a | ||||
.Nm | .Nm | ||||
architecture, the latency to access specific memory or IO devices | architecture, the latency to access specific memory or IO devices | ||||
depends upon which processor the memory or device is attached to. | depends upon which processor the memory or device is attached to. | ||||
Accessing memory local to a processor is faster than accessing memory | Accessing memory local to a processor is faster than accessing memory | ||||
that is connected to one of the other processors. | that is connected to one of the other processors. | ||||
.Pp | .Pp | ||||
.Nm | .Nm | ||||
is enabled when the | is enabled when the | ||||
.Cd MAXMEMDOM | .Cd NUMA | ||||
option is used in a kernel configuration | option is used in a kernel configuration | ||||
file and is set to a value greater than 1. | file and the | ||||
.Cd MAXMEMDOM | |||||
option is set to a value greater than 1. | |||||
.Pp | .Pp | ||||
Thread and process | Thread and process | ||||
.Nm | .Nm | ||||
policies are controlled with the | policies are controlled with the | ||||
.Xr numa_setaffinity 2 | .Xr cpuset_getdomain 2 | ||||
and | and | ||||
.Xr numa_getaffinity 2 | .Xr cpuset_setdomain 2 | ||||
syscalls. | syscalls. | ||||
.Pp | |||||
The | The | ||||
.Xr numactl 1 | .Xr cpuset 1 | ||||
tool is available for starting processes with a non-default | tool is available for starting processes with a non-default | ||||
policy, or to change the policy of an existing thread or process. | policy, or to change the policy of an existing thread or process. | ||||
.Pp | .Pp | ||||
Systems with non-uniform access to I/O devices may mark those devices | Systems with non-uniform access to I/O devices may mark those devices | ||||
with the local VM domain identifier. | with the local VM domain identifier. | ||||
Drivers can find out their local domain information by calling | Drivers can find out their local domain information by calling | ||||
.Xr bus_get_domain 9 . | .Xr bus_get_domain 9 . | ||||
.Ss MIB Variables | .Ss MIB Variables | ||||
The operation of | The operation of | ||||
.Nm | .Nm | ||||
is controlled and exposes information with these | is controlled and exposes information with these | ||||
.Xr sysctl 8 | .Xr sysctl 8 | ||||
MIB variables: | MIB variables: | ||||
.Pp | .Pp | ||||
.Bl -tag -width indent -compact | .Bl -tag -width indent -compact | ||||
.It Va vm.ndomains | .It Va vm.ndomains | ||||
The number of VM domains which have been detected. | The number of VM domains which have been detected. | ||||
.Pp | .Pp | ||||
.It Va vm.default_policy | |||||
The default VM domain allocation policy. | |||||
Defaults to "first-touch-rr". | |||||
The valid values are "first-touch", "first-touch-rr", | |||||
"rr", where "rr" is a short-hand for "round-robin." | |||||
See | |||||
.Xr numa_setaffinity 2 | |||||
for more information about the available policies. | |||||
.Pp | |||||
.It Va vm.phys_locality | .It Va vm.phys_locality | ||||
A table indicating the relative cost of each VM domain to each other. | A table indicating the relative cost of each VM domain to each other. | ||||
A value of 10 indicates equal cost. | A value of 10 indicates equal cost. | ||||
A value of -1 means the locality map is not available or no | A value of -1 means the locality map is not available or no | ||||
locality information is available. | locality information is available. | ||||
.Pp | .Pp | ||||
.It Va vm.phys_segs | .It Va vm.phys_segs | ||||
The map of physical memory, grouped by VM domain. | The map of physical memory, grouped by VM domain. | ||||
.El | .El | ||||
.Sh IMPLEMENTATION NOTES | .Sh IMPLEMENTATION NOTES | ||||
The current | The current | ||||
.Nm | .Nm | ||||
implementation is VM-focused. | implementation is VM-focused. | ||||
The hardware | The hardware | ||||
.Nm | .Nm | ||||
domains are mapped into a contiguous, non-sparse | domains are mapped into a contiguous, non-sparse | ||||
VM domain space, starting from 0. | VM domain space, starting from 0. | ||||
Thus, VM domain information (for example, the domain identifier) is not | Thus, VM domain information (for example, the domain identifier) is not | ||||
necessarily the same as is found in the hardware specific information. | necessarily the same as is found in the hardware specific information. | ||||
.Pp | |||||
The | |||||
.Nm | |||||
allocation policies are implemented as a policy and iterator in | |||||
.Pa sys/vm/vm_domain.c | |||||
and | |||||
.Pa sys/vm/vm_domain.h . | |||||
Policy information is available in both struct thread and struct proc. | Policy information is available in both struct thread and struct proc. | ||||
Processes inherit | |||||
.Nm | |||||
policy from parent processes and threads inherit | |||||
.Nm | |||||
policy from parent threads. | |||||
Note that threads do not explicitly inherit their | |||||
.Nm | |||||
policy from processes. | |||||
Instead, if no thread policy is set, the system | |||||
will fall back to the process policy. | |||||
.Pp | |||||
For now, | |||||
.Nm | |||||
domain policies only influence physical page allocation in | |||||
.Pa sys/vm/vm_phys.c . | |||||
This is useful for userland memory allocation, but not for kernel | |||||
and driver memory allocation. | |||||
These features will be implemented in future work. | |||||
.Sh SEE ALSO | .Sh SEE ALSO | ||||
.Xr numactl 1 , | .Xr cpuset 1 , | ||||
.Xr numa_getaffinity 2 , | .Xr cpuset_getaffinity 2 , | ||||
.Xr numa_setaffinity 2 , | .Xr cpuset_setaffinity 2 , | ||||
.Xr bus_get_domain 9 | .Xr bus_get_domain 9 | ||||
.Sh HISTORY | .Sh HISTORY | ||||
.Nm | .Nm | ||||
first appeared in | first appeared in | ||||
.Fx 9.0 | .Fx 9.0 | ||||
as a first-touch allocation policy with a fail-over to round-robin allocation | as a first-touch allocation policy with a fail-over to round-robin allocation | ||||
and was not configurable. | and was not configurable. | ||||
It was then modified in | It was then modified in | ||||
.Fx 10.0 | .Fx 10.0 | ||||
to implement a round-robin allocation policy and was also not configurable. | to implement a round-robin allocation policy and was also not configurable. | ||||
.Pp | .Pp | ||||
The | The | ||||
.Xr numa_getaffinity 2 | .Xr numa_getaffinity 2 | ||||
and | and | ||||
.Xr numa_setaffinity 2 | .Xr numa_setaffinity 2 | ||||
syscalls first appeared in | syscalls and the | ||||
.Fx 11.0 . | |||||
.Pp | |||||
The | |||||
.Xr numactl 1 | .Xr numactl 1 | ||||
tool first appeared in | tool first appeared in | ||||
.Fx 11.0 . | .Fx 11.0 | ||||
and were removed in | |||||
.Fx 12.0 . | |||||
Current implementation appeared in | |||||
.Fx 12.0 . | |||||
.Pp | |||||
.Sh AUTHORS | .Sh AUTHORS | ||||
This manual page written by | This manual page written by | ||||
.An Adrian Chadd Aq Mt adrian@FreeBSD.org . | .An Adrian Chadd Aq Mt adrian@FreeBSD.org . | ||||
.Sh NOTES | .Sh NOTES | ||||
No statistics are kept to indicate how often | No statistics are kept to indicate how often | ||||
.Nm | .Nm | ||||
allocation policies succeed or fail. | allocation policies succeed or fail. |