Changeset View
Changeset View
Standalone View
Standalone View
share/man/man9/epoch.9
Show All 20 Lines | |||||||||
.\" SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER | .\" SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER | ||||||||
.\" CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT | .\" CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT | ||||||||
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY | .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY | ||||||||
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH | .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH | ||||||||
.\" DAMAGE. | .\" DAMAGE. | ||||||||
.\" | .\" | ||||||||
.\" $FreeBSD$ | .\" $FreeBSD$ | ||||||||
.\" | .\" | ||||||||
.Dd April 30, 2020 | .Dd May 21, 2021 | ||||||||
.Dt EPOCH 9 | .Dt EPOCH 9 | ||||||||
.Os | .Os | ||||||||
.Sh NAME | .Sh NAME | ||||||||
.Nm epoch , | .Nm epoch , | ||||||||
.Nm epoch_context , | .Nm epoch_context , | ||||||||
.Nm epoch_alloc , | .Nm epoch_alloc , | ||||||||
.Nm epoch_free , | .Nm epoch_free , | ||||||||
.Nm epoch_enter , | .Nm epoch_enter , | ||||||||
.Nm epoch_exit , | .Nm epoch_exit , | ||||||||
.Nm epoch_wait , | .Nm epoch_wait , | ||||||||
.Nm epoch_enter_preempt , | .Nm epoch_enter_preempt , | ||||||||
.Nm epoch_exit_preempt , | .Nm epoch_exit_preempt , | ||||||||
.Nm epoch_wait_preempt , | .Nm epoch_wait_preempt , | ||||||||
.Nm epoch_enter_sleepable , | |||||||||
.Nm epoch_exit_sleepable , | |||||||||
.Nm epoch_wait_sleepable , | |||||||||
.Nm epoch_call , | .Nm epoch_call , | ||||||||
.Nm epoch_drain_callbacks , | .Nm epoch_drain_callbacks , | ||||||||
.Nm in_epoch , | .Nm in_epoch , | ||||||||
.Nm in_epoch_verbose , | .Nm in_epoch_verbose , | ||||||||
.Nd kernel epoch based reclamation | .Nd kernel epoch based reclamation | ||||||||
.Sh SYNOPSIS | .Sh SYNOPSIS | ||||||||
.In sys/param.h | .In sys/param.h | ||||||||
.In sys/proc.h | .In sys/proc.h | ||||||||
Show All 27 Lines | |||||||||
.Fn epoch_wait "epoch_t epoch" | .Fn epoch_wait "epoch_t epoch" | ||||||||
.Ft void | .Ft void | ||||||||
.Fn epoch_enter_preempt "epoch_t epoch" "epoch_tracker_t et" | .Fn epoch_enter_preempt "epoch_t epoch" "epoch_tracker_t et" | ||||||||
.Ft void | .Ft void | ||||||||
.Fn epoch_exit_preempt "epoch_t epoch" "epoch_tracker_t et" | .Fn epoch_exit_preempt "epoch_t epoch" "epoch_tracker_t et" | ||||||||
.Ft void | .Ft void | ||||||||
.Fn epoch_wait_preempt "epoch_t epoch" | .Fn epoch_wait_preempt "epoch_t epoch" | ||||||||
.Ft void | .Ft void | ||||||||
.Fn epoch_enter_sleepable "epoch_t epoch" "epoch_tracker_t et" | |||||||||
.Ft void | |||||||||
.Fn epoch_exit_sleepable "epoch_t epoch" "epoch_tracker_t et" | |||||||||
.Ft void | |||||||||
.Fn epoch_wait_sleepable "epoch_t epoch" | |||||||||
.Ft void | |||||||||
.Fn epoch_call "epoch_t epoch" "epoch_callback_t callback" "epoch_context_t ctx" | .Fn epoch_call "epoch_t epoch" "epoch_callback_t callback" "epoch_context_t ctx" | ||||||||
.Ft void | .Ft void | ||||||||
.Fn epoch_drain_callbacks "epoch_t epoch" | .Fn epoch_drain_callbacks "epoch_t epoch" | ||||||||
.Ft int | .Ft int | ||||||||
.Fn in_epoch "epoch_t epoch" | .Fn in_epoch "epoch_t epoch" | ||||||||
.Ft int | .Ft int | ||||||||
.Fn in_epoch_verbose "epoch_t epoch" "int dump_onfail" | .Fn in_epoch_verbose "epoch_t epoch" "int dump_onfail" | ||||||||
.Sh DESCRIPTION | .Sh DESCRIPTION | ||||||||
Epochs are used to guarantee liveness and immutability of data by | Epochs are used to guarantee liveness and immutability of data by | ||||||||
deferring reclamation and mutation until a grace period has elapsed. | deferring reclamation and mutation until a grace period has elapsed. | ||||||||
Epochs do not have any lock ordering issues. | Epochs do not have any lock ordering issues. | ||||||||
Entering and leaving an epoch section will never block. | Entering and leaving an epoch section will never block. | ||||||||
.Pp | .Pp | ||||||||
Epochs are allocated with | Epochs are allocated with | ||||||||
.Fn epoch_alloc . | .Fn epoch_alloc . | ||||||||
The | The | ||||||||
.Fa name | .Fa name | ||||||||
argument is used for debugging convenience when the | argument is used for debugging convenience when the | ||||||||
.Cd EPOCH_TRACE | .Cd EPOCH_TRACE | ||||||||
kernel option is configured. | kernel option is configured. | ||||||||
By default, epochs do not allow preemption during sections. | By default, epochs do not allow preemption during sections. | ||||||||
By default mutexes cannot be held across | By default mutexes cannot be held across | ||||||||
.Fn epoch_wait_preempt . | .Fn epoch_wait_preempt | ||||||||
and | |||||||||
.Fn epoch_wait_sleepable . | |||||||||
The | The | ||||||||
.Fa flags | .Fa flags | ||||||||
specified are formed by | specified are formed by | ||||||||
.Em OR Ns 'ing | .Em OR Ns 'ing | ||||||||
the following values: | the following values: | ||||||||
.Bl -tag -offset indent -width Ds | .Bl -tag -offset indent -width Ds | ||||||||
.It Dv EPOCH_LOCKED | .It Dv EPOCH_LOCKED | ||||||||
Permit holding mutexes across | Permit holding mutexes across | ||||||||
.Fn epoch_wait_preempt | .Fn epoch_wait_preempt . | ||||||||
(requires | Requires | ||||||||
.Dv EPOCH_PREEMPT ) . | .Dv EPOCH_PREEMPT | ||||||||
or | |||||||||
.Dv EPOCH_SLEEPABLE . | |||||||||
When doing this one must be cautious of creating a situation where a deadlock | When doing this one must be cautious of creating a situation where a deadlock | ||||||||
is possible. | is possible. | ||||||||
.It Dv EPOCH_CRITICAL | |||||||||
The default non-preemptable | |||||||||
pauamma_gundo.comUnsubmitted Not Done Inline Actions
pauamma_gundo.com: | |||||||||
.Vt epoch | |||||||||
type. | |||||||||
.It Dv EPOCH_PREEMPT | .It Dv EPOCH_PREEMPT | ||||||||
The | The | ||||||||
.Vt epoch | .Vt epoch | ||||||||
will allow preemption during sections. | will allow preemption during sections. | ||||||||
Only non-sleepable locks may be acquired during a preemptible epoch. | Only non-sleepable locks may be acquired during a preemptible epoch. | ||||||||
The functions | The functions | ||||||||
.Fn epoch_enter_preempt , | .Fn epoch_enter_preempt , | ||||||||
.Fn epoch_exit_preempt , | .Fn epoch_exit_preempt , | ||||||||
and | and | ||||||||
.Fn epoch_wait_preempt | .Fn epoch_wait_preempt | ||||||||
must be used in place of | must be used in place of | ||||||||
.Fn epoch_enter , | .Fn epoch_enter , | ||||||||
.Fn epoch_exit , | .Fn epoch_exit , | ||||||||
and | and | ||||||||
.Fn epoch_wait , | .Fn epoch_wait , | ||||||||
respectively. | respectively. | ||||||||
.It Dv EPOCH_SLEEPABLE | |||||||||
The | |||||||||
.Vt epoch | |||||||||
will allow preemption and sleeping during sections. | |||||||||
The functions | |||||||||
.Fn epoch_enter_sleepable , | |||||||||
.Fn epoch_exit_sleepable , | |||||||||
and | |||||||||
.Fn epoch_wait_sleepable | |||||||||
must be used in place of | |||||||||
.Fn epoch_enter , | |||||||||
.Fn epoch_exit , | |||||||||
and | |||||||||
.Fn epoch_wait , | |||||||||
respectively. | |||||||||
.El | .El | ||||||||
.Pp | .Pp | ||||||||
.Vt epoch Ns s | .Vt epoch Ns s | ||||||||
are freed with | are freed with | ||||||||
.Fn epoch_free . | .Fn epoch_free . | ||||||||
.Pp | .Pp | ||||||||
Threads indicate the start of an epoch critical section by calling | Threads indicate the start of an epoch critical section by calling | ||||||||
.Fn epoch_enter | .Fn epoch_enter , | ||||||||
(or | |||||||||
.Fn epoch_enter_preempt | .Fn epoch_enter_preempt | ||||||||
for preemptible epochs). | for preemptible epochs or | ||||||||
.Fn epoch_enter_sleepable | |||||||||
for sleepable epochs. | |||||||||
Threads call | Threads call | ||||||||
.Fn epoch_exit | .Fn epoch_exit , | ||||||||
(or | |||||||||
.Fn epoch_exit_preempt | .Fn epoch_exit_preempt | ||||||||
for preemptible epochs) | for preemptible epochs or | ||||||||
to indicate the end of a critical section. | .Fn epoch_exit_sleepable | ||||||||
for sleepable epochs, to indicate the end of a critical section. | |||||||||
.Vt struct epoch_tracker Ns s | .Vt struct epoch_tracker Ns s | ||||||||
are stack objects whose pointers are passed to | are stack objects whose pointers are passed to | ||||||||
.Fn epoch_enter_preempt | .Fn epoch_enter_preempt , | ||||||||
.Fn epoch_exit_preempt , | |||||||||
.Fn epoch_enter_sleepable | |||||||||
and | and | ||||||||
.Fn epoch_exit_preempt | .Fn epoch_exit_sleepable , | ||||||||
(much like | much like the | ||||||||
.Vt struct rm_priotracker ) . | .Vt struct rm_priotracker . | ||||||||
.Pp | .Pp | ||||||||
Threads can defer work until a grace period has expired since any thread has | Threads can defer work until a grace period has expired since any thread has | ||||||||
entered the epoch either synchronously or asynchronously. | entered the epoch either synchronously or asynchronously. | ||||||||
.Fn epoch_call | .Fn epoch_call | ||||||||
defers work asynchronously by invoking the provided | defers work asynchronously by invoking the provided | ||||||||
.Fa callback | .Fa callback | ||||||||
at a later time. | at a later time. | ||||||||
.Fn epoch_wait | .Fn epoch_wait , | ||||||||
(or | .Fn epoch_wait_preempt | ||||||||
.Fn epoch_wait_preempt ) | or | ||||||||
.Fn epoch_wait_sleepable | |||||||||
blocks the current thread until the grace period has expired and the work can be | blocks the current thread until the grace period has expired and the work can be | ||||||||
done safely. | done safely. | ||||||||
.Pp | .Pp | ||||||||
Default, non-preemptible epoch wait | Default, non-preemptible epoch wait, | ||||||||
.Fn ( epoch_wait ) | .Fn epoch_wait , | ||||||||
is guaranteed to have much shorter completion times relative to | is guaranteed to have much shorter completion times relative to | ||||||||
preemptible epoch wait | preemptible epoch wait, | ||||||||
.Fn ( epoch_wait_preempt ) . | .Fn epoch_wait_preempt . | ||||||||
(In the default type, none of the threads in an epoch section will be preempted | In turn the preemptible epoch wait, | ||||||||
before completing its section.) | .Fn epoch_wait_preempt , | ||||||||
is guaranteed to have much shorter completion times relative to | |||||||||
sleepable epoch wait, | |||||||||
.Fn epoch_wait_sleepable . | |||||||||
In the default type, none of the threads in an epoch section will be preempted | |||||||||
before completing its section. | |||||||||
.Pp | .Pp | ||||||||
INVARIANTS can assert that a thread is in an epoch by using | INVARIANTS can assert that a thread is in an epoch by using | ||||||||
.Fn in_epoch . | .Fn in_epoch . | ||||||||
.Fn in_epoch "epoch" | .Fn in_epoch "epoch" | ||||||||
is equivalent to invoking | is equivalent to invoking | ||||||||
.Fn in_epoch_verbose "epoch" "0" . | .Fn in_epoch_verbose "epoch" "0" . | ||||||||
If | If | ||||||||
.Cd EPOCH_TRACE | .Cd EPOCH_TRACE | ||||||||
is enabled, | is enabled, | ||||||||
.Fn in_epoch_verbose "epoch" "1" | .Fn in_epoch_verbose "epoch" "1" | ||||||||
provides additional verbose debugging information. | provides additional verbose debugging information. | ||||||||
.Pp | .Pp | ||||||||
The epoch API currently does not support sleeping in epoch_preempt sections. | |||||||||
A caller should never call | A caller should never call | ||||||||
.Fn epoch_wait | .Fn epoch_wait , | ||||||||
.Fn epoch_wait_preempt | |||||||||
or | |||||||||
.Fn epoch_wait_sleepable | |||||||||
in the middle of an epoch section for the same epoch as this will lead to a deadlock. | in the middle of an epoch section for the same epoch as this will lead to a deadlock. | ||||||||
.Pp | .Pp | ||||||||
The | The | ||||||||
.Fn epoch_drain_callbacks | .Fn epoch_drain_callbacks | ||||||||
function is used to drain all pending callbacks which have been invoked by prior | function is used to drain all pending callbacks which have been invoked by prior | ||||||||
.Fn epoch_call | .Fn epoch_call | ||||||||
function calls on the same epoch. | function calls on the same epoch. | ||||||||
This function is useful when there are shared memory structure(s) | This function is useful when there are shared memory structure(s) | ||||||||
▲ Show 20 Lines • Show All 72 Lines • ▼ Show 20 Lines | |||||||||
.Xr timeout 9 | .Xr timeout 9 | ||||||||
.Sh HISTORY | .Sh HISTORY | ||||||||
The | The | ||||||||
.Nm | .Nm | ||||||||
framework first appeared in | framework first appeared in | ||||||||
.Fx 11.0 . | .Fx 11.0 . | ||||||||
.Sh CAVEATS | .Sh CAVEATS | ||||||||
One must be cautious when using | One must be cautious when using | ||||||||
.Fn epoch_wait_preempt . | .Fn epoch_wait_preempt | ||||||||
and | |||||||||
.Fn epoch_wait_sleepable . | |||||||||
Threads are pinned during epoch sections, so if a thread in a section is then | Threads are pinned during epoch sections, so if a thread in a section is then | ||||||||
preempted by a higher priority compute bound thread on that CPU, it can be | preempted by a higher priority compute bound thread on that CPU, it can be | ||||||||
prevented from leaving the section indefinitely. | prevented from leaving the section indefinitely. | ||||||||
.Pp | .Pp | ||||||||
Epochs are not a straight replacement for read locks. | Epochs are not a straight replacement for read locks. | ||||||||
Callers must use safe list and tailq traversal routines in an epoch (see ck_queue). | Callers must use safe list and tailq traversal routines in an epoch (see ck_queue). | ||||||||
When modifying a list referenced from an epoch section safe removal | When modifying a list referenced from an epoch section safe removal | ||||||||
routines must be used and the caller can no longer modify a list entry | routines must be used and the caller can no longer modify a list entry | ||||||||
in place. | in place. | ||||||||
An item to be modified must be handled with copy on write | An item to be modified must be handled with copy on write | ||||||||
and frees must be deferred until after a grace period has elapsed. | and frees must be deferred until after a grace period has elapsed. |