D58173.diff
No OneTemporary
Actions

Size

84 KB

Referenced Files

None

Subscribers

None

D58173.diff
View Options

	diff --git a/documentation/content/en/books/handbook/jails/_index.adoc b/documentation/content/en/books/handbook/jails/_index.adoc
	--- a/documentation/content/en/books/handbook/jails/_index.adoc
	+++ b/documentation/content/en/books/handbook/jails/_index.adoc
	@@ -80,9 +80,13 @@
	* The different ways to configure the network for a jail.
	* The jail configuration file.
	* How to create the different types of jail.
	+* How to configure network address translation for jails.
	* How to start, stop, and restart a jail.
	* The basics of jail administration, both from inside and outside the jail.
	* How to upgrade the different types of jail.
	+* How to limit the resources a jail can use.
	+* The additional per-jail security and feature options.
	+* How to delegate ZFS datasets to a jail.
	* An incomplete list of the different FreeBSD jail managers.

	[[jail-types]]
	@@ -116,10 +120,13 @@
	[[thin-jails]]
	=== Thin Jails

	-A thin jail shares the base system using OpenZFS snapshots or NullFS mounts from a template.
	-Only a minimal subset of base system is duplicated for each thin jail, resulting in less resource consumption compared to a thick jail.
	-However, this also means that thin jails have less isolation and independence compared to thick jails.
	-Changes in shared components could potentially affect multiple thin jails simultaneously.
	+A thin jail shares the base system from a template, using either OpenZFS snapshots or NullFS mounts.
	+Only a minimal subset of the base system is duplicated for each thin jail, resulting in less resource consumption compared to a thick jail.
	+The two mechanisms behave differently once the jails exist.
	+With the OpenZFS method, each thin jail is created as a clone of a template snapshot: an independent copy-on-write copy that shares unchanged blocks with the template but diverges as it is modified.
	+Later changes to the template do not propagate to existing clones, and each clone is maintained and updated individually, much like a thick jail.
	+With the NullFS method, each thin jail mounts the template read-only at runtime, so the base system is shared live and updating the template updates every jail that mounts it.
	+This shared, read-only base is also what gives NullFS thin jails less isolation and independence than thick jails, since changes in the shared components affect every jail using them simultaneously.

	In summary, a FreeBSD Thin Jail is a type of FreeBSD Jail that replicates a substantial portion, but not all, of the base system within the isolated environment.

	@@ -130,15 +137,16 @@
	This makes it possible to run more jails on the same hardware without consuming excessive resources.
	* Faster Deployment: Creating and launching thin jails is generally faster compared to thick jails.
	This can be particularly advantageous when rapidly deploying multiple instances.
	-* Unified Maintenance: Since thin jails share the majority of their base system with the host system, updates and maintenance of common base system components (such as libraries and binaries) only need to be done once on the host.
	+* Unified Maintenance: With the NullFS method, where the thin jails share a single read-only copy of the base system, updates and maintenance of common base system components (such as libraries and binaries) only need to be done once on the template.
	This simplifies the maintenance process compared to maintaining an individual base system for each thick jail.
	+OpenZFS-clone thin jails do not share this benefit: because each clone is independent, updates must be applied to each jail individually, as with a thick jail.
	* Shared Resources: Thin jails can more easily share common resources such as libraries and binaries with the host system.
	This can potentially lead to more efficient disk caching and improved performance for applications within the jail.

	Disadvantages of Thin Jails:

	* Reduced Isolation: The primary disadvantage of thin jails is that they offer less isolation compared to thick jails.
	-Since they share a significant portion of the template's base system, vulnerabilities or issues affecting shared components could potentially impact multiple jails simultaneously.
	+With the NullFS method in particular, jails share a live copy of the template's base system, so vulnerabilities or issues affecting those shared components could potentially impact multiple jails simultaneously.
	* Security Concerns: The reduced isolation in thin jails could pose security risks, as a compromise in one jail might have a greater potential to affect other jails or the host system.
	* Dependency Conflicts: If multiple thin jails require different versions of the same libraries or software, managing dependencies can become complex.
	In some cases, this might require additional effort to ensure compatibility.
	@@ -150,7 +158,7 @@
	A service jail shares the complete filesystem tree directly with the host (the jail root path is [.filename]#/#) and as such can access and modify any file on the host, and shares the same user accounts with the host.
	By default it has no access to the network or other resources which are restricted in jails, but they can be configured to re-use the network of the host and to remove some of the jail-restrictions.
	The use case for service jails is automatic confinement of services/daemons inside a jail with minimal configuration, and without any knowledge of the files needed by such service/daemon.
	-Service jails exist since FreeBSD 15.
	+Service jails are available since FreeBSD 15.0.

	Advantages of Service Jails:

	@@ -168,7 +176,7 @@

	Most of the configuration of jails which is discussed below is not needed for service jails.
	To understand how jails work, it is recommended to understand those configuration possibilities.
	-The details about what is needed to configure a service jail is in crossref:jails[service-jails-config, Configuring service jails].
	+The details about what is needed to configure a service jail are in the crossref:jails[service-jails-config, Service Jails] section.

	[[vnet-jails]]
	=== VNET Jails
	@@ -217,15 +225,42 @@
	With `jail_parallel_start`, all configured jails will be started in the background.
	====

	+By default every jail with an entry in the configuration files is started at boot.
	+To start only selected jails, list their names in `jail_list`:
	+
	+[source,shell]
	+....
	+# sysrc jail_list="www db"
	+....
	+
	+The names in `jail_list` are separated by spaces and must match the jail names defined in [.filename]#/etc/jail.conf# or [.filename]#/etc/jail.conf.d/#.
	+Only the listed jails are started at boot, and they are started in the order given, so any jail left out of `jail_list` has to be started by hand.
	+When `jail_list` is left empty, all jails defined in the configuration files are started.
	+
	+To stop the jails in the reverse of their `jail_list` order at shutdown, set `jail_reverse_stop`:
	+
	+[source,shell]
	+....
	+# sysrc jail_reverse_stop="YES"
	+....
	+
	+[NOTE]
	+====
	+With `jail_parallel_start` enabled, jails start in the background and the `jail_list` order is no longer guaranteed.
	+When one jail must be running before another starts, express that relationship with the `depend` parameter in [.filename]#/etc/jail.conf# instead of relying on list order.
	+A jail's dependencies are created automatically before it starts, and stopped in the opposite order, regardless of `jail_list`.
	+The `depend` parameter is described in crossref:jails[jail-configuration-files,Jail Configuration Files].
	+====
	+
	[[jails-networking]]
	=== Networking

	Networking for FreeBSD jails can be configured several different ways:

	Host Networking Mode (IP Sharing)::
	-In host networking mode, a jail shares the same networking stack as the host system.
	-When a jail is created in host networking mode it uses the same network interface and IP address.
	-This means that the jail does not have a separate IP address, and its network traffic is associated with the host's IP.
	+In host networking mode a jail shares the host's network stack instead of receiving its own.
	+Sharing the stack does not mean sharing the address: the common setup gives the jail its own dedicated `ip4.addr` (and `ip6.addr`) that man:jail[8] adds as an alias on the host interface, so the jail and the host answer on different addresses over the same stack.
	+Only a jail configured with `ip4 = inherit` shares the host's addresses wholesale and answers on the host's own address, gaining unrestricted access to every address the host holds.

	Virtual Networks (VNET)::
	Virtual Networks are a feature of FreeBSD jails that offer more advanced and flexible networking solutions than a basic networking mode like host networking.
	@@ -236,6 +271,71 @@
	man:netgraph[4] is a versatile kernel framework for creating custom network configurations.
	It can be used to define how network traffic flows between jails and the host system and between different jails.

	+==== Where a Jail's IP Address Comes From
	+
	+A shared-stack jail reaches the network through an address on a host interface.
	+The `interface` parameter names that interface, and for every address in `ip4.addr` and `ip6.addr` man:jail[8] manages the alias itself: as jail(8) puts it, "an alias for each address will be added to the interface before the jail is created, and will be removed from the interface after the jail is removed".
	+Nothing on the host has to pre-configure the address.
	+
	+The interface can also be given per address, so a single line ties an address to the interface that should carry it:
	+
	+[.programlisting]
	+....
	+ip4.addr = "em0\|192.168.1.151/24";
	+....
	+
	+When an address must persist independently of the jail — because another service on the host relies on it, or because the `interface` parameter is not used — assign it on the host as an rc.conf alias and leave it out of jail(8)'s management:
	+
	+[source,shell]
	+....
	+# sysrc ifconfig_em0_aliases="inet 192.168.1.151/32"
	+....
	+
	+The older, numbered form `ifconfig_em0_alias0="inet 192.168.1.151/32"` still works, but the `_aliases` form is preferred for new configurations because it avoids the fragile requirement that the numbered aliases form an unbroken `alias0`, `alias1`, `alias2` sequence.
	+
	+==== Loopback Addresses in Shared-IP Jails
	+
	+A shared-stack jail has no loopback interface of its own.
	+The FreeBSD kernel compensates for this: in a non-VNET jail, whenever `127.0.0.1` or `::1` is not itself one of the jail's own addresses, the kernel silently rewrites both `bind()` and `connect()` on the loopback address to the jail's first `ip4.addr` or `ip6.addr`.
	+This behavior is not documented in man:jail[8], but it matters, because it carries a security consequence.
	+
	+[WARNING]
	+====
	+Do not assume that binding a service to `localhost` confines it to the jail.
	+In a shared-IP jail a daemon told to listen on `127.0.0.1` actually binds the jail's first address, which is frequently a LAN-facing one, and so becomes reachable from the network.
	+====
	+
	+The standard mitigation is to give the jail a private loopback address and list it first.
	+Clone a dedicated loopback interface on the host and assign it an address in the `127.0.0.0/8` range (or a private RFC 1918 address):
	+
	+[source,shell]
	+....
	+# sysrc cloned_interfaces="lo1"
	+# sysrc ifconfig_lo1="inet 127.0.1.1/32"
	+....
	+
	+Then list that address first in the jail's `ip4.addr`, ahead of any routable address, so the loopback remap lands on the private address instead of a LAN-facing one.
	+
	+==== Preparing Host Services
	+
	+Each jail alias address shares the host's ports.
	+A host daemon bound to the wildcard address — shown as `*:port` in listener listings — is therefore listening on every jail alias too, and it occupies that port before a jail's own service can claim it.
	+Before starting any jails, audit what the host already listens on:
	+
	+[source,shell]
	+....
	+# sockstat -4 -6 -l
	+....
	+
	+Restrict each daemon that binds a wildcard address so it uses only the host's own address and leaves the jail addresses free:
	+
	+* man:syslogd[8]: set `syslogd_flags="-ss"` in [.filename]#/etc/rc.conf#.
	+A single `-s` already runs syslogd in secure mode so it does not listen for remote log messages; per syslogd(8), if `-s` "is specified twice, no network socket will be opened at all", which also stops syslogd from sending to remote hosts.
	+* man:sshd[8]: pin `ListenAddress` to the host's address in [.filename]#/etc/ssh/sshd_config# instead of leaving it on the default wildcard.
	+* man:ntpd[8] and man:sendmail[8]: bind them to the host address as well through their own configuration.
	+
	+For firewall-level control over which addresses a jail may use, see crossref:firewalls[firewalls,Firewalls].
	+
	[[host-configuration-directories]]
	=== Setting Up the Jail Directory Tree

	@@ -265,6 +365,9 @@
	In this case, `zroot` was used for the parent dataset, but other datasets could have been used.
	====

	+Creating a separate child dataset per jail is what later enables per-jail snapshots, clones, quotas, and dataset delegation.
	+Because each jail directory is then its own dataset, OpenZFS can both cap and guarantee its disk use with the `quota` and `reservation` properties; see crossref:jails[jail-resource-limits,Jail Resource Limits].
	+
	When using UFS, execute the following commands to create the directories:

	[source,shell]
	@@ -314,7 +417,7 @@

	# NETWORK
	ip4.addr = 192.168.1.151; <.>
	- ip6.addr = ::ffff:c0a8:197 <.>
	+ ip6.addr = "2001:db8::151"; <.>
	interface = em0; <.>
	}
	....
	@@ -328,21 +431,137 @@
	<.> `allow.raw_sockets` - Allow creating raw sockets inside the jail.
	Setting this parameter allows utilities like man:ping[8] and man:traceroute[8] to operate inside the jail.
	<.> `exec.clean` - Run commands in a clean environment.
	-<.> `mount.devfs` - Mount a man:devfs[5] filesystem on the chrooted [.filename]#/dev# directory, and apply the ruleset in the devfs_ruleset parameter to restrict the devices visible inside the jail.
	+<.> `mount.devfs` - Mount a man:devfs[5] filesystem on the chrooted [.filename]#/dev# directory to restrict the devices visible inside the jail.
	+It applies the ruleset named by the `devfs_ruleset` parameter, or, when that parameter is left at its default of `0`, the `mount.devfs` fallback of ruleset 4, `devfsrules_jail`, which hides every host device except a minimal, jail-safe set (see crossref:jails[jail-conf-devfs, Devfs Rulesets]).
	<.> `host.hostname` - The hostname of the jail.
	<.> `path` - The directory which is to be the root of the jail.
	Any commands that are run inside the jail, either by jail or from man:jexec[8], are run from this directory.
	-<.> `ip4.addr` - IPv4 address.
	-There are two configuration possibilities for IPv4.
	-The first is to establish an IP or a list of IPs as has been done in the example.
	-The other is to use `ip4` instead and set the `inherit` value to inherit the host's IP address.
	-<.> `ip6.addr` - IPv6 address.
	-There are two configuration possibilities for IPv6.
	-The first is to establish an IP or a list of IPs as has been done in the example.
	-The other is to use `ip6` instead and set the `inherit` value to inherit the host's IP address.
	+<.> `ip4.addr` - One IPv4 address, or a comma-separated list of addresses, assigned to the jail.
	+Setting `ip4.addr` implies `ip4 = new`; `ip4` itself is not left at a standalone default.
	+The `ip4` parameter accepts three values: `new` restricts the jail to the addresses listed in `ip4.addr`, `inherit` gives the jail unrestricted access to every IPv4 address of the host, and `disable` turns off IPv4 in the jail entirely.
	+A jail with neither `ip4` nor `ip4.addr` set behaves like `inherit`, and `interface` and `ip4.addr` have no effect under `inherit`.
	+In a jail with several addresses, `ip4.saddrsel` lets the kernel choose the source address for outgoing connections on unbound sockets, while `ip4.nosaddrsel` forces the first address in the list.
	+<.> `ip6.addr` - One IPv6 address, or a comma-separated list of addresses, assigned to the jail.
	+The `ip6` parameter takes the same `new`, `inherit`, and `disable` values as `ip4`, with `ip6.saddrsel` and `ip6.nosaddrsel` controlling source-address selection.
	+Use `ip6 = disable` for an IPv4-only jail, or `ip4 = disable` for an IPv6-only one.
	<.> `interface` - A network interface to add the jail's IP addresses.
	Usually the host interface.

	+[NOTE]
	+====
	+A jail has no virtual console: man:init[8] and man:getty[8] do not run inside it, so there is nothing to attach to interactively.
	+The `exec.consolelog` file captures only the standard output and standard error of the `exec.*` commands, that is, the output of the [.filename]#/etc/rc# startup and [.filename]#/etc/rc.shutdown# scripts.
	+It is not an interactive console; for that, use man:jexec[8] (see crossref:jails[access-jail, Access a Jail]) or run man:sshd[8] inside the jail.
	+The jail's own daemons send their later output to the jail's syslog.
	+====
	+
	+[TIP]
	+====
	+The `exec.consolelog` files grow without bound.
	+Rotate them with man:newsyslog[8] by adding a single glob entry, for example in [.filename]#/etc/newsyslog.conf.d/jail#:
	+
	+[.programlisting]
	+....
	+/var/log/jail_console_.log 640 7 100 J
	+....
	+====
	+
	+==== Global Settings, Wildcards, and Variables
	+
	+Parameters set outside any jail block are wildcard defaults: man:jail.conf[5] applies them to every jail, exactly as if they were written in a `*` wildcard section.
	+Setting the shared parameters once keeps each jail block short.
	+
	+A name preceded by a dollar sign, and optionally enclosed in braces, is replaced with the value of that parameter or variable.
	+The built-in `${name}` expands to the jail's own name, and a custom variable is defined just like a parameter but with a leading dollar sign.
	+
	+The following [.filename]#/etc/jail.conf# defines a common header once and overrides only what differs per jail:
	+
	+[.programlisting]
	+....
	+# Shared defaults applied to every jail
	+exec.start = "/bin/sh /etc/rc";
	+exec.stop = "/bin/sh /etc/rc.shutdown";
	+exec.consolelog = "/var/log/jail_console_${name}.log";
	+exec.clean;
	+mount.devfs;
	+host.hostname = "${name}";
	+path = "/usr/local/jails/containers/${name}";
	+interface = em0;
	+
	+www {
	+ ip4.addr = 192.168.1.151;
	+}
	+
	+db {
	+ host.hostname = "database.example.com";
	+ ip4.addr = 192.168.1.152;
	+}
	+....
	+
	+Here `www` inherits everything from the header and sets only its address, while `db` also overrides `host.hostname`.
	+
	+[[jail-conf-hooks]]
	+==== Command Hooks
	+
	+man:jail[8] runs a jail through a sequence of lifecycle hooks, and it matters whether each one runs in the host environment or inside the jail.
	+In creation order:
	+
	+`exec.prestart`:: Runs in the host environment before the jail is created.
	+`exec.created`:: Runs in the host environment after the jail exists but before any process starts inside it.
	+`exec.start`:: Runs inside the jail, typically `/bin/sh /etc/rc` to bring up its services.
	+`exec.poststart`:: Runs in the host environment after the jail has started.
	+
	+Shutdown runs the mirror image:
	+
	+`exec.prestop`:: Runs in the host environment before the jail is stopped.
	+`exec.stop`:: Runs inside the jail, typically `/bin/sh /etc/rc.shutdown`.
	+`exec.poststop`:: Runs in the host environment after the jail has stopped, to tear down whatever `exec.prestart` set up.
	+
	+A hook may be assigned more than one command by appending with `+=`:
	+
	+[.programlisting]
	+....
	+exec.poststart += "logger jail ${name} started";
	+....
	+
	+`exec.timeout` caps how long man:jail[8] waits for a command to finish, and `exec.consolelog` captures the output of these commands.
	+The VNET section (crossref:jails[creating-vnet-jail, VNET Jails]) uses `exec.prestart` and `exec.poststop` to build and tear down each jail's network plumbing on the host.
	+
	+Where hooks order commands within a single jail, the `depend` parameter orders whole jails.
	+Writing `depend = otherjail;` tells man:jail[8] that this jail requires `otherjail`: the dependency is created first, up to the completion of its last `exec.poststart` command, before this jail is created, and the jails are torn down in the reverse order.
	+
	+[[jail-conf-devfs]]
	+==== Devfs Rulesets
	+
	+Every jail that sets `mount.devfs` gets a devfs ruleset that decides which device nodes appear under its [.filename]#/dev#, either the one named by the `devfs_ruleset` parameter or, when that parameter is unset, the `mount.devfs` default of ruleset 4.
	+man:devfs.rules[5] defines named rulesets in [.filename]#/etc/defaults/devfs.rules#.
	+Two of them are meant for jails: `devfsrules_jail`, ruleset 4, is the standard minimal device set for a jail, and `devfsrules_jail_vnet`, ruleset 5, adds [.filename]#/dev/pf# so a VNET jail can run its own firewall.
	+The VNET jail examples set `devfs_ruleset = 5` and the Linux jail example sets `devfs_ruleset = 4`; the classic and thin-jail examples set only `mount.devfs` and so use its default of ruleset 4.
	+
	+To expose an extra device, write a custom ruleset in [.filename]#/etc/devfs.rules# that includes one of the stock rulesets and unhides what it needs.
	+The following ruleset starts from `devfsrules_jail` and additionally unhides the man:bpf[4] devices:
	+
	+[.programlisting]
	+....
	+[devfsrules_jail_bpf=100]
	+add include $devfsrules_jail
	+add path 'bpf*' unhide
	+....
	+
	+Reload the rules, then reference the new ruleset by its number in the jail's configuration:
	+
	+[source,shell]
	+....
	+# service devfs restart
	+....
	+
	+[.programlisting]
	+....
	+devfs_ruleset = 100;
	+....
	+
	+Rulesets in [.filename]#/etc/devfs.rules# override same-numbered rulesets in [.filename]#/etc/defaults/devfs.rules#; otherwise the two files are effectively merged.
	+
	More information about configuration variables can be found in man:jail[8] and man:jail.conf[5].

	[[classic-jail]]
	@@ -362,7 +581,7 @@

	[source,shell,subs=attributes]
	....
	-# fetch https://download.freebsd.org/ftp/releases/amd64/amd64/{rel-latest}-RELEASE/base.txz -o /usr/local/jails/media/{rel-latest}-RELEASE-base.txz
	+# fetch https://download.freebsd.org/releases/amd64/amd64/{rel-latest}-RELEASE/base.txz -o /usr/local/jails/media/{rel-latest}-RELEASE-base.txz
	....

	Once the download is complete, it will be necessary to extract the contents into the jail directory.
	@@ -427,6 +646,116 @@

	More information on how to manage jails can be found in the section crossref:jails[jail-management, Jail Management].

	+[[creating-jail-userland]]
	+=== Obtaining the Jail Userland
	+
	+The fetch-and-extract procedure shown above is the traditional way to populate a jail's root directory, but it is not the only one.
	+The same userland can be installed with man:bsdinstall[8], from base-system packages, or from a locally built source tree.
	+
	+man:bsdinstall[8] provides a dedicated `jail` target that sets up a new userland at the given directory for use with man:jail[8].
	+It fetches and extracts the distribution sets without installing a kernel, skipping the disk partitioning and network steps of a normal installation, which makes it a one-command, scriptable alternative to the manual fetch and man:tar[1] sequence:
	+
	+[source,shell]
	+....
	+# bsdinstall jail /usr/local/jails/containers/classic
	+....
	+
	+The `jail` target installs the base distribution ([.filename]#base.txz#) by default.
	+On FreeBSD 15.0 and later man:bsdinstall[8] can also populate the root from base-system packages with its `pkgbase` target; the pkg-based method is described next.
	+
	+Starting with FreeBSD 15.0 the base system is also published as packages, so a jail userland can be installed entirely with man:pkg[8] from the `FreeBSD-base` repository.
	+Point man:pkg[8] at the jail's root directory with the global `--rootdir` option and install the jail variant of the base set:
	+
	+[source,shell]
	+....
	+# pkg --rootdir /usr/local/jails/containers/classic install -r FreeBSD-base FreeBSD-set-base-jail
	+....
	+
	+Use `FreeBSD-set-minimal-jail` instead of `FreeBSD-set-base-jail` for a minimal userland.
	+A jail installed this way is afterwards patched and upgraded with man:pkg[8] rather than man:freebsd-update[8]; see crossref:jails[jail-upgrade-pkgbase, Upgrading pkgbase Jails].
	+
	+[NOTE]
	+====
	+Base-system packages are a technology preview in FreeBSD 15.0.
	+The `FreeBSD-base` repository is defined in [.filename]#/etc/pkg/FreeBSD.conf# but disabled by default, so select it explicitly with `install -r FreeBSD-base`.
	+The target root directory and its [.filename]#var/cache/pkg# and [.filename]#var/db/pkg# subdirectories may need to be created first.
	+The traditional [.filename]#base.txz# distribution is still published for FreeBSD 15.0, so the fetch-and-extract method continues to work there as well.
	+====
	+
	+A userland built from a source tree can be installed into a jail's root directory with the `DESTDIR` variable, which is the way to run a `-STABLE` or custom-built userland in a jail.
	+After building the world, install it and the remaining distribution files into the jail directory:
	+
	+[source,shell]
	+....
	+# cd /usr/src
	+# make buildworld
	+# make installworld DESTDIR=/usr/local/jails/containers/classic
	+# make distribution DESTDIR=/usr/local/jails/containers/classic
	+....
	+
	+Updates then go through `installworld` and `distribution` with the same `DESTDIR` (and man:etcupdate[8] with `-D` for [.filename]#/etc#) rather than man:freebsd-update[8].
	+See crossref:cutting-edge[makeworld, Updating FreeBSD from Source] for building the world from source.
	+
	+[[classic-jail-clone]]
	+=== Cloning a Jail as a Template
	+
	+Once a thick jail is fully configured, with packages installed, services enabled, and local configuration in place, it can serve as a template image: a template from which many identical jails are deployed almost instantly.
	+
	+On OpenZFS this is done by snapshotting the configured jail's dataset and cloning it.
	+This requires the source jail to live on its own dataset, so create it with `zfs create` rather than the `mkdir` used in the classic-jail procedure above:
	+
	+[source,shell]
	+....
	+# zfs create -p zroot/jails/containers/classic
	+....
	+
	+Stop the jail before snapshotting so the image is consistent, then snapshot and clone the dataset once for each new jail:
	+
	+[source,shell]
	+....
	+# service jail stop classic
	+# zfs snapshot zroot/jails/containers/classic@template
	+# zfs clone zroot/jails/containers/classic@template zroot/jails/containers/web1
	+....
	+
	+On a UFS host, copy the configured jail's directory tree instead:
	+
	+[source,shell]
	+....
	+# cp -a /usr/local/jails/containers/classic /usr/local/jails/containers/web1
	+....
	+
	+Each clone needs its own entry in [.filename]#/etc/jail.conf#, or in [.filename]#jail.conf.d#, with a unique name, path, and IP address.
	+Because the `path` and `host.hostname` parameters in the example above are derived from the block name with `${name}`, only the block name and the address differ:
	+
	+[.programlisting]
	+....
	+web1 {
	+ # STARTUP/LOGGING
	+ exec.start = "/bin/sh /etc/rc";
	+ exec.stop = "/bin/sh /etc/rc.shutdown";
	+ exec.consolelog = "/var/log/jail_console_${name}.log";
	+
	+ # PERMISSIONS
	+ allow.raw_sockets;
	+ exec.clean;
	+ mount.devfs;
	+
	+ # HOSTNAME/PATH
	+ host.hostname = "${name}";
	+ path = "/usr/local/jails/containers/${name}";
	+
	+ # NETWORK
	+ ip4.addr = 192.168.1.152;
	+ interface = em0;
	+}
	+....
	+
	+A template-image clone differs from a thin jail (crossref:jails[thin-jail, Thin Jails]).
	+A thin jail keeps sharing a read-only base and is maintained once at the template, whereas each template-image clone is a full, independently writable copy that is patched and upgraded on its own as described in crossref:jails[jail-upgrading, Jail Upgrading].
	+An OpenZFS clone starts as a space-efficient copy-on-write copy that shares blocks with the snapshot, and it keeps a dependency on that origin snapshot until `zfs promote` severs it.
	+For a fully independent copy, or to replicate a jail to another pool or host, use `zfs send` piped to `zfs receive` instead of `zfs clone`.
	+
	[[thin-jail]]
	== Thin Jails

	@@ -440,7 +769,7 @@

	Due to the good integration between FreeBSD and OpenZFS it is very easy to create new Thin Jails using OpenZFS Snapshots.

	-To create a Thin Jail using OpenZFS Snapshots the first step is to create the jail directory tree by following the instructions in crossref:jails[host-configuration-directories, "Setting up the Jail Directory Tree"].
	+To create a Thin Jail using OpenZFS Snapshots the first step is to create the jail directory tree by following the instructions in crossref:jails[host-configuration-directories, "Setting Up the Jail Directory Tree"].

	Next, create a template.
	Templates will only be used to create new jails.
	@@ -457,7 +786,7 @@

	[source,shell,subs=attributes]
	....
	-# fetch https://download.freebsd.org/ftp/releases/amd64/amd64/{rel-latest}-RELEASE/base.txz -o /usr/local/jails/media/{rel-latest}-RELEASE-base.txz
	+# fetch https://download.freebsd.org/releases/amd64/amd64/{rel-latest}-RELEASE/base.txz -o /usr/local/jails/media/{rel-latest}-RELEASE-base.txz
	....

	Once the download is complete, it will be necessary to extract the contents in the template directory by executing the following command:
	@@ -561,7 +890,7 @@

	[source,shell,subs=attributes]
	....
	-# fetch https://download.freebsd.org/ftp/releases/amd64/amd64/{rel-latest}-RELEASE/base.txz -o /usr/local/jails/media/{rel-latest}-RELEASE-base.txz
	+# fetch https://download.freebsd.org/releases/amd64/amd64/{rel-latest}-RELEASE/base.txz -o /usr/local/jails/media/{rel-latest}-RELEASE-base.txz
	....

	Once the download is complete, it will be necessary to extract the contents in the template directory by executing the following command:
	@@ -685,7 +1014,7 @@
	}
	....

	-Then the create the [.filename]#/usr/local/jails/thinjail-nullfs-base.fstab# file as follows:
	+Then create the [.filename]#/usr/local/jails/thinjail-nullfs-base.fstab# file as follows:

	[.programlisting,subs=attributes]
	....
	@@ -693,6 +1022,11 @@
	/usr/local/jails/containers/thinjail /usr/local/jails/thinjail-nullfs-base/skeleton nullfs rw 0 0
	....

	+The [.filename]#fstab# file controls how man:mount_nullfs[8] presents the base template and the skeleton inside the jail.
	+The base template is mounted read-only (`ro`) so that every NullFS jail shares one immutable copy of the base system.
	+The per-jail skeleton is mounted read-write (`rw`), which is where each jail keeps its own writable [.filename]#/etc#, [.filename]#/var#, [.filename]#/usr/local#, and other local directories.
	+Because all jails share that single read-only base template, updating the one template updates every NullFS jail at once, in contrast to independent OpenZFS clones which must each be updated separately.
	+
	Execute the following command to start the jail:

	[source,shell]
	@@ -701,7 +1035,7 @@
	....

	[[creating-vnet-jail]]
	-=== Creating a VNET Jail
	+== VNET Jails

	FreeBSD VNET Jails have their own distinct networking stack, including interfaces, IP addresses, routing tables, and firewall rules.

	@@ -719,6 +1053,13 @@
	bridge0
	....

	+[WARNING]
	+====
	+Since FreeBSD 15.0 the default value of the `net.link.bridge.member_ifaddrs` sysctl is `0`, and man:ifconfig[8] refuses with `EINVAL` to add an interface that still carries an IP address to a bridge, or to assign an address to a bridge member.
	+On such a host, assign the host's address to `bridge0` itself rather than to `em0`, exactly as the [.filename]#/etc/rc.conf# block below does.
	+As a temporary compatibility escape hatch, setting `net.link.bridge.member_ifaddrs=1` restores the old behavior, but this sysctl is slated for removal in FreeBSD 16.0.
	+====
	+
	With the `bridge` created, it will be necessary to attach it to the `em0` interface and bring both of them up by executing the following commands:

	[source,shell]
	@@ -741,7 +1082,7 @@

	The next step is to create the jail as indicated above.

	-Either the crossref:jails[classic-jail, Classic Jail (Thick Jail)] procedure and the
	+Either the crossref:jails[classic-jail, Classic Jail (Thick Jail)] procedure or the
	crossref:jails[thin-jail, Thin Jails] procedure can be used.
	The only thing that will change is the configuration in the [.filename]#/etc/jail.conf# file.

	@@ -782,20 +1123,87 @@
	exec.prestart += "/sbin/ifconfig ${bridge} addm ${epair}a up";
	exec.start += "/sbin/ifconfig ${epair}b ${ip} up";
	exec.start += "/sbin/route add default ${gateway}";
	- exec.start += "/bin/sh /etc/rc";
	- exec.stop = "/bin/sh /etc/rc.shutdown";
	+ exec.start += "/bin/sh /etc/rc";
	+ exec.stop = "/bin/sh /etc/rc.shutdown";
	exec.poststop = "/sbin/ifconfig ${bridge} deletem ${epair}a";
	exec.poststop += "/sbin/ifconfig ${epair}a destroy";
	}
	....

	-<.> Represents the IP of the Jail, it must be unique.
	+<.> A per-jail unique number used to derive both the epair device name (`epair${id}`) and the last octet of the jail's address (`192.168.1.${id}`); give each jail on the bridge a different value.
	<.> Refers to the bridge created previously.

	+The `vnet.interface` parameter names the interface that man:jail[8] moves into the jail's network stack once the jail is created, here the jail side of the epair, `${epair}b`.
	+That interface disappears from the host while the jail runs and is automatically released back to the host when the jail stops.
	+A physical network card or an SR-IOV virtual function can be handed to a jail the same way, giving it dedicated networking without a bridge or an epair.
	+
	+The `devfs_ruleset = 5` line applies the `devfsrules_jail_vnet` ruleset from [.filename]#/etc/defaults/devfs.rules#, which extends the default jail ruleset by exposing [.filename]#/dev/pf#.
	+A VNET jail can therefore run its own man:pf[4] firewall with `pf_enable="YES"` and a private [.filename]#pf.conf#, independent of the host.
	+See crossref:jails[jail-conf-devfs, Devfs Rulesets] for how rulesets are defined and crossref:firewalls[firewalls, Firewalls] for pf itself.
	+
	+[[jail-vnet-jib]]
	+=== Automating VNET Networking with jib
	+
	+Wiring up an epair and attaching it to the bridge by hand in `exec.prestart` and `exec.poststop`, as shown above, becomes tedious across many jails.
	+The base system ships a helper script, [.filename]#/usr/share/examples/jails/jib#, that performs the same plumbing automatically.
	+
	+Reference it from the jail's hooks instead of the five hand-rolled man:ifconfig[8] lines:
	+
	+[.programlisting]
	+....
	+ exec.prestart += "jib addm ${name} em0";
	+ exec.poststop += "jib destroy ${name}";
	+
	+ vnet.interface = "e0b_${name}";
	+....
	+
	+`jib addm` creates an epair whose host side is named `e0a_<name>` and whose jail side is `e0b_<name>`, then attaches the host side to a bridge that it creates automatically, named after the member interface (for `em0` the bridge is `em0bridge`).
	+Name that jail side in `vnet.interface`.
	+When the jail stops, `jib destroy` removes the epair.
	+The script also provides a `jib show` subcommand to list the interfaces it manages.
	+
	+[[jail-vnet-jng]]
	+=== Netgraph VNET Networking with jng
	+
	+The third networking mode mentioned in the crossref:jails[jails-networking, Networking] overview uses man:netgraph[4] instead of man:bridge[4] and epair.
	+The base system ships a second helper, [.filename]#/usr/share/examples/jails/jng#, that builds an `ng_bridge` connected to an `ng_eiface` for each jail:
	+
	+[.programlisting]
	+....
	+ exec.prestart += "jng bridge ${name} em0";
	+ exec.poststop += "jng shutdown ${name}";
	+
	+ vnet.interface = "ng0_${name}";
	+....
	+
	+`jng bridge` creates the netgraph nodes and a jail-side interface named `ng0_<name>`, which is named in `vnet.interface`; `jng shutdown` tears the topology down when the jail stops.
	+The result is functionally equivalent to the epair-and-bridge setup, but built on netgraph nodes rather than man:bridge[4].
	+
	+[[jail-vnet-firewall]]
	+=== Running a Firewall and DHCP inside a VNET Jail
	+
	+Because `devfs_ruleset = 5` exposes [.filename]#/dev/pf#, a VNET jail can run its own firewall.
	+Enable it inside the jail exactly as on any host, with `pf_enable="YES"` in the jail's [.filename]#/etc/rc.conf# and its own [.filename]#/etc/pf.conf#.
	+
	+Ruleset 5 does not expose [.filename]#/dev/bpf#, however.
	+A VNET jail that obtains its address over DHCP, for example with `ifconfig_e0b_myjail="SYNCDHCP"` in its [.filename]#/etc/rc.conf#, runs man:dhclient[8], which needs bpf and therefore fails under ruleset 5.
	+Following the recipe in crossref:jails[jail-conf-devfs,Devfs Rulesets], define a custom ruleset in [.filename]#/etc/devfs.rules# that starts from `devfsrules_jail_vnet` and additionally unhides the bpf devices:
	+
	+[.programlisting]
	+....
	+[devfsrules_jail_vnet_dhcp=101]
	+add include $devfsrules_jail_vnet
	+add path 'bpf*' unhide
	+....
	+
	+After running `service devfs restart` on the host, point the jail at the new ruleset with `devfs_ruleset = 101;`.
	+
	+This applies to both FreeBSD 14.x and 15.0.
	+
	[[creating-linux-jail]]
	-=== Creating a Linux Jail
	+== Linux Jails

	-FreeBSD can run Linux inside a jail using crossref:linuxemu[linuxemu,Linux Binary Compatibility] and man:debootstrap[8].
	+FreeBSD can run a Linux userland inside a jail using crossref:linuxemu[linuxemu,Linux Binary Compatibility] and package:sysutils/debootstrap[].
	Jails do not have a kernel.
	They run on the host's kernel.
	Therefore it is necessary to enable Linux Binary Compatibility in the host system.
	@@ -814,18 +1222,21 @@
	# service linux start
	....

	-The next step will be to create a jail as indicated above, for example in
	-crossref:jails[creating-thin-jail-openzfs-snapshots, Creating a Thin Jail Using OpenZFS Snapshots], but without performing the configuration.
	-FreeBSD Linux jails require a specific configuration that will be detailed below.
	+The next step is to create a jail as indicated above, for example as in crossref:jails[creating-thin-jail-openzfs-snapshots, Creating a Thin Jail Using OpenZFS Snapshots], but without performing the configuration.
	+FreeBSD Linux jails require a specific configuration that is detailed below.

	-Create an empty [.filename]#/etc/jail.conf# to prevent failures when trying to stop the jail:
	+If the host does not already have an [.filename]#/etc/jail.conf# — for example on a system whose jails so far have been created only from the command line — create an empty one so that man:service[8] can stop the jail later:

	[source,shell]
	....
	# touch /etc/jail.conf
	....

	-Once the jail has been created as explained above, execute the following command to perform required configuration for the jail and start it:
	+The man:service[8] jail script reads [.filename]#/etc/jail.conf# when stopping a jail, so the stop command fails if the file does not exist.
	+If an [.filename]#/etc/jail.conf# is already present from an earlier section, this step is not needed.
	+
	+The following man:jail[8] command performs the required configuration for the jail and starts it directly, without a [.filename]#/etc/jail.conf# entry.
	+The `-c` flag creates a new jail and `-m` modifies an existing one; combined as `-cm`, man:jail[8] creates the jail if it does not yet exist and updates it if it does:

	[source,shell]
	....
	@@ -834,7 +1245,7 @@
	host.hostname="ubuntu.example.com" \
	path="/usr/local/jails/containers/ubuntu" \
	interface="em0" \
	- ip4.addr="192.168.1.150" \
	+ ip4.addr="192.168.1.155" \
	exec.start="/bin/sh /etc/rc" \
	exec.stop="/bin/sh /etc/rc.shutdown" \
	mount.devfs \
	@@ -849,16 +1260,16 @@
	enforce_statfs=1
	....

	-To access the jail, it will be necessary to install package:sysutils/debootstrap[].
	-
	-Execute the following command to access the FreeBSD Linux jail:
	+Access the newly created jail as `root` with man:jexec[8]:

	[source,shell]
	....
	# jexec -u root ubuntu sh
	....

	-Inside the jail, execute the following commands to install man:pkg[7] and package:sysutils/debootstrap[] to prepare the Ubuntu environment:
	+Inside the jail, install package:sysutils/debootstrap[] and use it to build the Ubuntu userland under [.filename]#/compat/ubuntu#.
	+package:sysutils/debootstrap[] is the tool that builds the Linux userland; it is not required to enter the jail afterward.
	+Running man:pkg[7] for the first time bootstraps it automatically:

	[source,shell]
	....
	@@ -866,14 +1277,19 @@
	# debootstrap jammy /compat/ubuntu
	....

	-When the process has finished and the message `Base system installed successfully` is displayed on the console,
	-it will be necessary to stop the jail from the host system by executing the following command:
	+`jammy` selects Ubuntu 22.04 LTS.
	+A newer LTS such as `noble` (Ubuntu 24.04) can be selected instead, provided it runs under the linuxulator.
	+
	+When the process has finished and the message `Base system installed successfully` is displayed on the console, stop the jail from the host system.
	+Because this jail is not enabled in [.filename]#/etc/rc.conf# and not yet listed in [.filename]#/etc/jail.conf#, use the `one`-prefixed man:service[8] action, which runs even for a jail that is not enabled:

	[source,shell]
	....
	# service jail onestop ubuntu
	....

	+The `onestart` and `onestop` actions are the counterparts of `start` and `stop` for jails that are not enabled through `jail_enable` or listed in `jail_list`.
	+
	Then add an entry in [.filename]#/etc/jail.conf# for the Linux jail:

	[.programlisting]
	@@ -909,6 +1325,13 @@
	}
	....

	+[TIP]
	+====
	+The jail can also present a specific Linux kernel identity to the programs running inside it.
	+The `linux` parameter selects whether the jail inherits the host's Linux emulation environment (`inherit`) or gets its own (`new`), and `linux.osname`, `linux.osrelease`, and `linux.oss_version` set the OS name, release, and OSS version that Linux binaries see.
	+For example, `linux.osrelease=6.1.0` makes man:uname[1] and version-checking Linux programs inside the jail report that kernel release instead of the host-wide `compat.linux.osrelease` value.
	+====
	+
	Then the jail can be started as usual with the following command:

	[source,shell]
	@@ -925,16 +1348,67 @@

	More information can be found in the chapter crossref:linuxemu[linuxemu,Linux Binary Compatibility].

	+[[jail-nat]]
	+== NAT and Port Forwarding for Jails
	+
	+A jail that is given a private address — on a cloned man:lo[4] interface such as `lo1`, on a private bridge, or on a VNET epair on an internal network — has no route to the internet of its own.
	+The host must translate the jail's traffic to a routable address, and, to reach a service running inside such a jail from outside, redirect incoming connections to the jail.
	+
	+First, allow the host to forward packets between its interfaces.
	+Enable it at boot and immediately with the following commands:
	+
	+[source,shell]
	+....
	+# sysrc gateway_enable="YES"
	+# sysctl net.inet.ip.forwarding=1
	+....
	+
	+For IPv6, use `ipv6_gateway_enable="YES"` and `net.inet6.ip6.forwarding=1`.
	+
	+Then configure Network Address Translation on the host with man:pf[4].
	+The following [.filename]#/etc/pf.conf# gives every jail on the private `192.168.0.0/24` network outbound access through the host's external interface `em0`, and publishes a jailed web server on the host's port 80:
	+
	+[.programlisting]
	+....
	+ext_if = "em0"
	+
	+# Translate outbound jail traffic to the host address
	+nat on $ext_if from 192.168.0.0/24 to any -> ($ext_if)
	+
	+# Forward inbound web traffic to the jailed server
	+rdr on $ext_if proto tcp to port 80 -> 192.168.0.151
	+....
	+
	+The `rdr` target must be the jail's own address.
	+
	+Enable and start man:pf[4] with the following commands:
	+
	+[source,shell]
	+....
	+# sysrc pf_enable="YES"
	+# service pf start
	+....
	+
	+[NOTE]
	+====
	+FreeBSD 15.0 adds the OpenBSD-style inline `nat-to` and `rdr-to` syntax, which attaches translation to a `pass` or `match` rule, for example `match out on $ext_if from 192.168.0.0/24 to any nat-to ($ext_if)`.
	+The classic `nat on` and `rdr on` rules shown above work on both FreeBSD 14.x and 15.x.
	+====
	+
	+For the full treatment of man:pf[4] and man:ipfw[8], including complete rule sets and man:ipfw[8]-based NAT, see crossref:firewalls[firewalls,Firewalls].
	+
	[[service-jails-config]]
	-=== Configuring Service Jails
	+== Service Jails

	+Service jails are available since FreeBSD 15.0.
	A service jail is configured completely via [.filename]#/etc/rc.conf# or man:sysrc[8].
	The base system services are service jails ready.
	-They contain a config line which enables networking or lift other restrictions of jails.
	+They contain a config line which enables networking or lifts other restrictions of jails.
	Base system services which do not make sense to run inside jails are configured to not be started as a service jail, even if enabled in [.filename]#/etc/rc.conf#.
	-Some examples of such a service are services which want to mount or unmount something in the start of stop method, or only configure something like a route, or firewall, or the like.
	+Some examples of such a service are services which want to mount or unmount something in the start or stop method, or only configure something like a route, or firewall, or the like.

	-Third party services may or may not be service jails ready. To check if a service is service jail ready, the following command can be used:
	+Third party services may or may not be service jails ready.
	+To check if a service is service jail ready, the following command can be used:

	[source,shell]
	....
	@@ -966,7 +1440,7 @@
	If it is not stopped, the rc framework will not detect the correct state of the service and will not be able to do what is requested.

	Service jails are managed only via man:rc.conf[5]/man:sysrc[8] and the man:service[8] command.
	-The jail utilities, like man:jls[8] as described in crossref:jails[jail-management,Jail Management] can be used to investigate the operation, but the man:jail[8] command is not supposed to be used to manage them.
	+The jail utilities, like man:jls[8] as described in crossref:jails[jail-management,Jail Management], can be used to investigate the operation, but the man:jail[8] command is not supposed to be used to manage them.

	[[jail-management]]
	== Jail Management
	@@ -988,7 +1462,7 @@

	....
	JID IP Address Hostname Path
	- 1 192.168.250.70 classic /usr/local/jails/containers/classic
	+ 1 192.168.1.151 classic /usr/local/jails/containers/classic
	....

	man:jls[8] supports the `--libxo` argument, which through the man:libxo[3] library allows other types of formats to be displayed, such as `JSON`, `HTML`, etc.
	@@ -1003,54 +1477,117 @@
	The output should be similar to the following:

	....
	-{"__version": "2", "jail-information": {"jail": [{"jid":1,"ipv4":"192.168.250.70","hostname":"classic","path":"/usr/local/jails/containers/classic"}]}}
	+{"__version": "2", "jail-information": {"jail": [{"jid":1,"ipv4":"192.168.1.151","hostname":"classic","path":"/usr/local/jails/containers/classic"}]}}
	....

	-[[start-jail]]
	-=== Start, Restart, and Stop a Jail
	-
	-man:service[8] is used to start, reboot, or stop a jail on the host.
	-
	-For example, to start a jail, run the following command:
	+The default listing is only a small part of what man:jls[8] can report.
	+Add the `-v` flag for a verbose, multi-line summary of each jail that includes its state and its assigned cpuset ID:

	[source,shell]
	....
	-# service jail start jailname
	+# jls -v
	....

	-Change the `start` argument to `restart` or `stop` to perform other actions on the jail.
	-
	-[[destroy-jail]]
	-=== Destroy a Jail
	-
	-Destroying a jail is not as simple as stopping the jail using man:service[8] and removing the jail directory and [.filename]#/etc/jail.conf# entry.
	-
	-FreeBSD takes system security very seriously.
	-For this reason there are certain files that not even the root user can delete.
	-This functionality is known as File Flags.
	-
	-The first step is to stop the desired jail executing the following command:
	+Use `-h` to print a header line with the field names above the values of the requested parameters:

	[source,shell]
	....
	-# service jail stop jailname
	+# jls -h jid name path
	....

	-The second step is to remove these flags with man:chflags[1] by executing the following command, in which `classic` is the name of the jail to remove:
	+To query specific parameters of a single jail, name it with `-j` and list the parameters to print:

	[source,shell]
	....
	-# chflags -R 0 /usr/local/jails/containers/classic
	+# jls -j classic path host.hostname ip4.addr
	....

	-The third step is to delete the directory where the jail was:
	+The output should be similar to the following:

	-[source,shell]
	+....
	+/usr/local/jails/containers/classic classic 192.168.1.151
	+....
	+
	+Add `-d` to include jails that are shutting down, which are otherwise hidden while in the dying state:
	+
	+[source,shell]
	+....
	+# jls -d
	+....
	+
	+[[start-jail]]
	+=== Start, Restart, and Stop a Jail
	+
	+man:service[8] is used to start, reboot, or stop a jail on the host.
	+
	+For example, to start a jail, run the following command:
	+
	+[source,shell]
	+....
	+# service jail start jailname
	+....
	+
	+Change the `start` argument to `restart` or `stop` to perform other actions on the jail.
	+
	+Stopping a jail is more than killing the processes inside it.
	+When a jail is stopped, man:jail[8] first runs the commands in its `exec.stop` parameter — typically `/bin/sh /etc/rc.shutdown` — inside the jail so that its services shut down cleanly.
	+It then sends `SIGTERM` to any processes still running in the jail and waits up to `stop.timeout` seconds (10 by default) for them to exit.
	+Once that time has passed, the jail is removed, which kills whatever remains with `SIGKILL`.
	+If `stop.timeout` is set to zero, no `SIGTERM` is sent and the jail is removed immediately.
	+This is why a jail with a slow-stopping daemon can appear to hang for ten seconds while it stops; raise `stop.timeout` in [.filename]#/etc/jail.conf# for a jail that runs a database or another service that needs longer to shut down gracefully.
	+
	+[[destroy-jail]]
	+=== Destroy a Jail
	+
	+Destroying a jail is not as simple as stopping the jail using man:service[8] and removing the jail directory and [.filename]#/etc/jail.conf# entry.
	+
	+FreeBSD takes system security very seriously.
	+For this reason there are certain files that not even the root user can delete.
	+This functionality is known as File Flags.
	+On a jail whose files live on a UFS file system, these flags must be cleared before the jail directory can be removed.
	+
	+The first step is to stop the desired jail executing the following command:
	+
	+[source,shell]
	+....
	+# service jail stop classic
	+....
	+
	+The second step is to remove these flags with man:chflags[1] by executing the following command, in which `classic` is the name of the jail to remove:
	+
	+[source,shell]
	+....
	+# chflags -R 0 /usr/local/jails/containers/classic
	+....
	+
	+The third step is to delete the directory where the jail was:
	+
	+[source,shell]
	....
	# rm -rf /usr/local/jails/containers/classic
	....

	-Finally, it will be necessary to remove the jail entry in [.filename]#/etc/jail.conf# or in [.filename]#jail.conf.d#.
	+The two steps above apply to a jail stored on a UFS file system.
	+A jail backed by its own ZFS dataset — as created in crossref:jails[host-configuration-directories,Setting Up the Jail Directory Tree] — is removed by destroying the dataset instead.
	+Dataset destruction is not blocked by file flags, so the man:chflags[1] step is unnecessary in this case:
	+
	+[source,shell]
	+....
	+# service jail stop classic
	+# zfs destroy zroot/jails/containers/classic
	+....
	+
	+Add `-r` to also destroy any snapshots the dataset holds.
	+
	+[NOTE]
	+====
	+A ZFS thin jail created by cloning a template snapshot (see crossref:jails[thin-jail,Thin Jails]) depends on that snapshot.
	+While any clone still exists, man:zfs[8] refuses to destroy the template snapshot and prints the list of dependent clones.
	+To retire a template, first destroy every jail cloned from it, or run `zfs promote` on a clone so it no longer depends on the template.
	+====
	+
	+Finally, in either case, remove the jail entry in [.filename]#/etc/jail.conf# or in [.filename]#/etc/jail.conf.d#.
	+For a NullFS thin jail, also delete its [.filename]#.fstab# file and its now-empty mountpoint directory.

	[[handle-packages-jail]]
	=== Handle Packages in a Jail
	@@ -1066,6 +1603,22 @@

	For more information on working with packages in FreeBSD, see crossref:ports[ports,"Installing Applications: Packages and Ports"].

	+[[jail-services-host]]
	+=== Managing Services in a Jail from the Host
	+
	+Services inside a jail can be enabled and controlled from the host without first entering the jail with man:jexec[8].
	+Both man:service[8] and man:sysrc[8] accept a `-j` argument that names the target jail by ID or name and operates on that jail's man:rc[8] configuration.
	+
	+Continuing the package example above, enable and start nginx in the `classic` jail directly from the host:
	+
	+[source,shell]
	+....
	+# sysrc -j classic nginx_enable=YES
	+# service -j classic nginx start
	+....
	+
	+These commands have the same effect as running man:sysrc[8] and man:service[8] through man:jexec[8], but they keep host-side administration and scripting uniform with an unjailed system.
	+
	[[access-jail]]
	=== Access a Jail

	@@ -1078,6 +1631,11 @@
	# jexec -u root jailname
	....

	+The `-u` flag selects the user to run as, resolving the name against the host password database.
	+To resolve the user against the jail's own password database instead, use `-U`.
	+When no command is given, as here, man:jexec[8] starts that user's login shell inside the jail.
	+Add `-l` to run it in a clean login environment that keeps only `HOME`, `SHELL`, `TERM`, and `USER`, resets `PATH` to [.filename]#/bin:/usr/bin#, and behaves like a fresh login; `-d` sets the working directory, which defaults to the jail root.
	+
	When gaining access to the jail, the message configured in man:motd[5] will be displayed.

	[[execute-commands-jail]]
	@@ -1092,11 +1650,189 @@
	# jexec -l jailname service nginx stop
	....

	+[[jail-view-processes]]
	+=== Viewing Jailed Processes and Sockets from the Host
	+
	+From the host, the processes and sockets of every jail are visible and can be filtered by jail.
	+
	+man:ps[1] accepts `-J` to restrict its output to a single jail, given by JID or name:
	+
	+[source,shell]
	+....
	+# ps -J classic
	+....
	+
	+The output should be similar to the following, where the `J` in the `STAT` column marks a process that is running in a jail:
	+
	+....
	+ PID TT STAT TIME COMMAND
	+2841 - IsJ 0:00.02 /usr/sbin/syslogd -ss
	+2903 - IsJ 0:00.00 nginx: master process /usr/local/sbin/nginx
	+....
	+
	+Use `ps -J 0` to show only the host's own processes.
	+The `jail` and `jid` keywords can also be added to the `-o` output format to show which jail each process belongs to.
	+For example, to list every process on the host together with its jail:
	+
	+[source,shell]
	+....
	+# ps ax -o pid,jail,command
	+....
	+
	+man:top[1] takes `-J` to filter its live display to one jail, which implies `-j` and adds the `JID` column:
	+
	+[source,shell]
	+....
	+# top -J classic
	+....
	+
	+man:sockstat[1] lists the sockets belonging to a jail with `-j`, which is the natural way to check which ports a jail's daemons have actually bound after starting a service in it:
	+
	+[source,shell]
	+....
	+# sockstat -4 -6 -l -j classic
	+....
	+
	+From inside a jail the picture is reversed: a jailed process sees only its own jail's processes and sockets, never those of the host or of other jails.
	+
	+[[jail-manage-cli]]
	+=== Managing Jails Directly with the jail Utility
	+
	+The man:service[8] wrapper shown above is the usual way to run jails at boot, but man:jail[8] can also create, modify, and remove jails directly, which is useful for scripting and for testing a configuration without the rc framework.
	+
	+Create a configured jail by name:
	+
	+[source,shell]
	+....
	+# jail -c classic
	+....
	+
	+Remove a running jail; this also removes any child jails it created and kills the processes inside it:
	+
	+[source,shell]
	+....
	+# jail -r classic
	+....
	+
	+The `-m` flag modifies the parameters of an already-running jail without restarting it.
	+For example, to add a second IPv4 address to a running jail:
	+
	+[source,shell]
	+....
	+# jail -m name=classic ip4.addr+=192.168.250.71
	+....
	+
	+man:jail[8] notes that "Some parameters may not be changed on a running jail"; in practice this includes `path` and `vnet`, which require the jail to be restarted.
	+
	+Finally, `-e` exhibits all configured non-wildcard jails and their parameters without creating, modifying, or removing anything, which is handy for verifying a configuration file.
	+It takes a string used to separate the parameters:
	+
	+[source,shell]
	+....
	+# jail -e ,
	+....
	+
	+[[jail-backup]]
	+=== Backing Up and Migrating a Jail
	+
	+A jail is largely self-contained, so backing one up or moving it to another host is mostly a matter of copying its files and its configuration.
	+
	+Stop the jail first for a consistent copy; a ZFS-backed jail can instead be snapshotted while running.
	+For a jail on its own ZFS dataset, take a recursive snapshot and stream it to the destination host:
	+
	+[source,shell]
	+....
	+# service jail stop classic
	+# zfs snapshot -r zroot/jails/containers/classic@migrate
	+# zfs send -R zroot/jails/containers/classic@migrate \| ssh newhost zfs receive zroot/jails/containers/classic
	+....
	+
	+The full mechanics of man:zfs-send[8] and man:zfs-receive[8], including incremental transfers, are covered in crossref:zfs[zfs-zfs-send,Replication].
	+
	+For a jail on a UFS file system, use man:tar[1] with `-p` so that ownership, permissions, and file flags are preserved:
	+
	+[source,shell]
	+....
	+# tar -cpf classic.tar -C /usr/local/jails/containers classic
	+....
	+
	+Extract the archive as `root` on the destination host with `tar -xpf` so the flags survive.
	+
	+Finally, copy the jail's block from [.filename]#/etc/jail.conf# — or its file in [.filename]#/etc/jail.conf.d# — to the new host, adjust any host-specific parameters such as `interface`, `ip4.addr`, or the bridge used by a VNET jail, and start the jail there.
	+
	+[CAUTION]
	+====
	+The destination host must not run an older FreeBSD release than the jail's userland, following the same rule that a jail can never run a newer version than its host.
	+A NullFS thin jail is not self-contained: its template and skeleton must exist on the destination host too, so migrate those as well or convert the jail to a thick copy first.
	+Any man:rctl[8] rules kept in [.filename]#/etc/rctl.conf# must be carried over separately.
	+====
	+
	+[[jail-inside]]
	+== Administration from Inside a Jail
	+
	+Most jail administration is done from the host, but some tasks are performed from within the jail itself, and a script running inside a jail sometimes needs to know that it is jailed.
	+This section describes what an administrator sees and can do from inside a jail.
	+
	+A process can detect whether it is running inside a jail by reading the `security.jail.jailed` sysctl, which is `1` inside a jail and `0` on the host:
	+
	+[source,shell]
	+....
	+# sysctl security.jail.jailed
	+security.jail.jailed: 1
	+....
	+
	+Inside a jail the environment is deliberately restricted:
	+
	+* The host's view of the system is hidden.
	+A jailed process sees only its own jail's processes, sockets, and file systems, never those of the host or of other jails.
	+* Kernel state is largely read-only.
	+Kernel tunables cannot be written from inside a jail, man:kldload[8] and man:kldunload[8] are not permitted, and man:dmesg[8] and the kernel message buffer are unavailable unless `allow.read_msgbuf` is set.
	+* Privileged operations that would cross the jail boundary are denied unless the host explicitly allows them.
	+Creating raw sockets — as man:ping[8] and man:traceroute[8] require — needs `allow.raw_sockets` (see the parameter description in crossref:jails[jail-configuration-files,Jail Configuration Files]), mounting file systems needs `allow.mount` and its per-filesystem children, and man:chflags[1] is disabled by default (see crossref:jails[jail-upgrading,Jail Upgrading]).
	+
	+A handful of per-jail sysctl variables can be changed from inside the jail without affecting the host, including `kern.hostname`, `kern.domainname`, `kern.hostid`, `kern.hostuuid`, `kern.securelevel`, and `security.bsd.suser_enabled`.
	+The current set of available jail parameters can be listed with man:sysctl[8]:
	+
	+[source,shell]
	+....
	+# sysctl -d security.jail.param
	+....
	+
	+Managing the jail's own services from inside works exactly as it does on an unjailed system: man:service[8] and man:sysrc[8] operate on the jail's own man:rc[8] configuration.
	+This is the in-jail counterpart to the host-side crossref:jails[jail-services-host,service -j and sysrc -j] shown above.
	+
	[[jail-upgrading]]
	== Jail Upgrading

	Upgrading FreeBSD Jails ensures that the isolated environments remain secure, up-to-date, and in line with the latest features and improvements available in the FreeBSD ecosystem.

	+The right tool depends on how the jail's userland was installed.
	+Jails installed from a distribution set -- the [.filename]#base.txz# extracted in the creation sections, or one produced by man:bsdinstall[8] -- are updated from the host with man:freebsd-update[8] through its `-j` flag, on both FreeBSD 14.x and 15.x.
	+Jails installed from base packages, available from FreeBSD 15.0, are updated with man:pkg[8] instead; man:freebsd-update[8] does not apply to them.
	+Jails built from source are updated with the source procedure (`make installworld` and `make distribution`).
	+Never mix these mechanisms on the same jail.
	+
	+[NOTE]
	+====
	+Support for distribution sets is planned for removal in FreeBSD 16, but will continue, together with man:freebsd-update[8], for the lifetime of the FreeBSD 15 stable branch.
	+Base packages are offered as a technology preview in FreeBSD 15.0 and are expected to become the standard method in future releases, so pkgbase is the forward-looking path.
	+====
	+
	+[TIP]
	+====
	+On OpenZFS, snapshot the jail's dataset before upgrading, so a failed upgrade can be undone instantly:
	+
	+[source,shell]
	+....
	+# zfs snapshot zroot/jails/containers/classic@preupgrade
	+....
	+
	+If the upgrade fails, stop the jail and run `zfs rollback zroot/jails/containers/classic@preupgrade` to restore the previous state.
	+This requires the jail to be its own dataset: the OpenZFS thin-jail clone already is, but a classic jail created with man:mkdir[1] is not, so create it with `zfs create` instead.
	+For a NullFS thin jail, snapshot the template dataset instead.
	+man:zfs-rollback[8] reverts only to the most recent snapshot unless `-r` is given to destroy any newer ones.
	+====
	+
	[[jails-updating]]
	=== Upgrading a Classic Jail or a Thin Jail using OpenZFS Snapshots

	@@ -1120,11 +1856,11 @@
	In case of upgrade from one version to another, it is easier to create a new jail than to upgrade completely.
	====

	-For example to upgrade from 13.1-RELEASE to 13.2-RELEASE, execute the following commands on the host:
	+For example, to upgrade the `classic` jail to {rel-latest}-RELEASE, execute the following commands on the host:

	-[source,shell]
	+[source,shell,subs=attributes]
	....
	-# freebsd-update -j classic -r 13.2-RELEASE upgrade
	+# freebsd-update -j classic -r {rel-latest}-RELEASE upgrade
	# freebsd-update -j classic install
	# service jail restart classic
	# freebsd-update -j classic install
	@@ -1133,8 +1869,9 @@

	[NOTE]
	====
	-It is necessary to execute the `install` step two times.
	-The first one upgrades the kernel, and the second one upgrades the rest of the components.
	+The `install` step is run twice on purpose.
	+A jail has no kernel of its own -- it runs the host's kernel -- so nothing here upgrades a kernel.
	+The first `install` writes the new userland into the jail; after the jail restarts on the updated files, the second `install` finishes replacing the components that could not be changed while their previous versions were still in use.
	====

	Then, if it was a major version upgrade, reinstall all installed packages and restart the jail again.
	@@ -1144,10 +1881,50 @@

	[source,shell]
	....
	-# pkg -j jailname upgrade -f
	-# service jail restart jailname
	+# pkg -j classic upgrade -f
	+# service jail restart classic
	+....
	+
	+[[jail-upgrade-zfs-template]]
	+=== Upgrading OpenZFS Thin Jails
	+
	+The thin jails created in crossref:jails[creating-thin-jail-openzfs-snapshots,"Creating a Thin Jail Using OpenZFS Snapshots"] are OpenZFS clones of a template snapshot, but a clone is independent of that template once it exists.
	+Refreshing the template does not update jails that were already cloned from it, and updating a clone does not update the template.
	+There are therefore two ways to keep these jails current.
	+
	+The first option is to update each clone in place with man:freebsd-update[8], exactly as for a classic jail, as described in crossref:jails[jails-updating,"Upgrading a Classic Jail or a Thin Jail using OpenZFS Snapshots"].
	+This is the simplest approach for a handful of jails, but every clone that is patched diverges from the template snapshot: the changed blocks are duplicated per jail, which erodes the space savings, and a major upgrade rewrites so much of the userland that most of the sharing is lost.
	+
	+The second option keeps the sharing intact by refreshing the template and re-cloning.
	+Update the template to the latest patch level on the host:
	+
	+[source,shell,subs=attributes]
	+....
	+# freebsd-update -b /usr/local/jails/templates/{rel-latest}-RELEASE/ fetch install
	....

	+[NOTE]
	+====
	+man:freebsd-update[8] with `-b` reads the running release from the host's `uname -r`, not from the template.
	+If the template runs an older release than the host, add `--currently-running <template-release>` so it patches the template's release instead of the host's, as shown in crossref:jails[upgrading-thin-jail,"Upgrading a Thin Jail Using NullFS"].
	+====
	+
	+Then take a new snapshot and clone future jails from it:
	+
	+[source,shell,subs=attributes]
	+....
	+# zfs snapshot zroot/jails/templates/{rel-latest}-RELEASE@base-p2
	+# zfs clone zroot/jails/templates/{rel-latest}-RELEASE@base-p2 zroot/jails/containers/newjail
	+....
	+
	+Existing clones keep the base they were created from until they are re-created from the new snapshot.
	+For a major or minor version upgrade, create a fresh template dataset for the new release following crossref:jails[creating-thin-jail-openzfs-snapshots,"Creating a Thin Jail Using OpenZFS Snapshots"], rather than upgrading the existing template in place, and clone new jails from its `@base` snapshot.
	+
	+[TIP]
	+====
	+Keep each jail's application data and configuration on separate datasets, or in the jail's local storage, so that re-creating a jail from an updated template snapshot is cheap and preferable to patching every clone individually.
	+====
	+
	[[upgrading-thin-jail]]
	=== Upgrading a Thin Jail Using NullFS

	@@ -1157,26 +1934,58 @@

	To update the template to the latest patch release of the version of FreeBSD it is running, execute the following commands on the host:

	-[source,shell]
	+[source,shell,subs=attributes]
	....
	-# freebsd-update -b /usr/local/jails/templates/13.1-RELEASE-base/ fetch install
	+# freebsd-update -b /usr/local/jails/templates/{rel-latest}-RELEASE-base/ fetch install
	# service jail restart
	....

	To upgrade the template to a new major or minor version, first upgrade the host system as described in crossref:cutting-edge[freebsdupdate-upgrade,"Performing Major and Minor Version Upgrades"].
	Once the host has been upgraded and rebooted, the template can then be upgraded.

	-For example, to upgrade from 13.1-RELEASE to 13.2-RELEASE, execute the following commands on the host:
	+Because the host was upgraded first, man:freebsd-update[8] with `-b` would otherwise detect the host's new release rather than the template's older one: `-b` reads the running version from the host's `uname -r`, unlike `-j`, which detects the jail's own userland.
	+Pass `--currently-running <template-release>` so the template is patched from its actual release.
	+The following example upgrades a template running 14.2-RELEASE to 14.3-RELEASE; substitute the releases that apply:

	[source,shell]
	....
	-# freebsd-update -b /usr/local/jails/templates/13.1-RELEASE-base/ -r 13.2-RELEASE upgrade
	-# freebsd-update -b /usr/local/jails/templates/13.1-RELEASE-base/ install
	+# freebsd-update -b /usr/local/jails/templates/14.2-RELEASE-base/ --currently-running 14.2-RELEASE -r 14.3-RELEASE upgrade
	+# freebsd-update -b /usr/local/jails/templates/14.2-RELEASE-base/ install
	# service jail restart
	-# freebsd-update -b /usr/local/jails/templates/13.1-RELEASE-base/ install
	+# freebsd-update -b /usr/local/jails/templates/14.2-RELEASE-base/ install
	# service jail restart
	....

	+[[jail-upgrade-pkgbase]]
	+=== Upgrading pkgbase Jails
	+
	+Jails installed from base packages, available from FreeBSD 15.0, are upgraded with man:pkg[8] against the `FreeBSD-base` repository rather than with man:freebsd-update[8].
	+
	+The `FreeBSD-base` repository is defined in [.filename]#/etc/pkg/FreeBSD.conf# but is disabled by default on FreeBSD 15.0, so enable it before upgrading.
	+
	+From the host, point man:pkg[8] at the jail's root directory with `-r` (`--rootdir`):
	+
	+[source,shell]
	+....
	+# pkg -r /usr/local/jails/containers/pkgjail update
	+# pkg -r /usr/local/jails/containers/pkgjail upgrade
	+# service jail restart pkgjail
	+....
	+
	+The same upgrade can be run from inside the jail with man:jexec[8] and a plain `pkg upgrade`:
	+
	+[source,shell]
	+....
	+# jexec -l pkgjail pkg upgrade
	+# service jail restart pkgjail
	+....
	+
	+man:pkg[8] performs a three-way merge on tagged configuration files under [.filename]#/etc#.
	+When a merge cannot be completed automatically it leaves the new version as [.filename]#<file>.pkgnew#, and when it takes over a file that was not previously packaged it saves the old copy as [.filename]#<file>.pkgsave#.
	+Review [.filename]#/etc# after an upgrade and reconcile any such files by hand.
	+Do not run man:etcupdate[8] on a pkgbase system; man:pkg[8] handles configuration merging itself.
	+See crossref:cutting-edge[cutting-edge,"Updating and Upgrading FreeBSD"] for the host-side base-package workflow.
	+
	[[jail-resource-limits]]
	== Jail Resource Limits

	@@ -1193,9 +2002,13 @@

	[.programlisting]
	....
	-rctl -a jail:<jailname>:resource:action=amount/percentage
	+rctl -a jail:<jailname>:resource:action=amount[/per]
	....

	+The optional `per` component is not a percentage; it is the accounting entity the limit is applied to, one of `process`, `user`, `loginclass`, or `jail`.
	+For example, `deny=2G/jail` limits the sum for the whole jail, while `deny=2G/process` applies the same figure to each process individually.
	+A percentage is only meaningful for the `pcpu` resource, whose amount is expressed as a percentage of a single CPU core.
	+
	For example, to limit the maximum RAM that a jail can access, run the following command:

	[source,shell]
	@@ -1210,8 +2023,342 @@
	jail:classic:memoryuse:deny=2G/jail
	....

	+man:rctl[8] can limit far more than memory.
	+The resources most relevant to a jail include `pcpu` (percentage of a single CPU core), `maxproc` (number of processes), `openfiles` (open file descriptors), `vmemoryuse` (address space), and `swapuse` (swap space).
	+Each of them is set the same way:
	+
	+[source,shell]
	+....
	+# rctl -a jail:classic:pcpu:deny=80
	+# rctl -a jail:classic:maxproc:deny=1000
	+# rctl -a jail:classic:openfiles:deny=5000
	+# rctl -a jail:classic:vmemoryuse:deny=4G
	+....
	+
	+Disk throughput is limited with `readbps` and `writebps`, but these resources do not support the `deny` action; pair them with `throttle` instead:
	+
	+[source,shell]
	+....
	+# rctl -a jail:classic:writebps:throttle=1m
	+....
	+
	+The action after the resource selects what happens when the limit is reached: `deny` blocks the operation, `log` records it to the system log, and `devctl` sends a notification through man:devd[8].
	+
	+Report the current resource usage of a jail with the `-u` flag, adding `-h` for human-readable units:
	+
	+[source,shell]
	+....
	+# rctl -hu jail:classic
	+....
	+
	+List the rules that apply to a jail:
	+
	+[source,shell]
	+....
	+# rctl jail:classic
	+....
	+
	+Remove every rule that matches a jail with `-r`:
	+
	+[source,shell]
	+....
	+# rctl -r jail:classic
	+....
	+
	More information on resource limits can be found in the security chapter in the crossref:security[security-resourcelimits,"Resource Limits section"].

	+[[jail-cpu-pinning]]
	+=== CPU Pinning with cpuset
	+
	+man:rctl[8] with `pcpu` caps how much CPU time a jail may consume, but it does not tie the jail to particular cores; man:cpuset[1] assigns specific cores instead.
	+
	+Restrict a running jail to a set of CPUs by its name or JID:
	+
	+[source,shell]
	+....
	+# cpuset -l 0-3 -j classic
	+....
	+
	+Query the cores a jail is currently allowed to use with `-g`:
	+
	+[source,shell]
	+....
	+# cpuset -g -j classic
	+....
	+
	+The assigned set is also reported in the cpuset column of `jls -v`.
	+There is no jail.conf parameter for CPU affinity, so pin the jail at start time by running man:cpuset[1] from an `exec.created` or `exec.poststart` hook, both of which run in the host environment:
	+
	+[.programlisting]
	+....
	+exec.poststart = "cpuset -l 0-3 -j classic";
	+....
	+
	+=== Limiting Disk Space
	+
	+man:rctl[8] has no resource for filesystem space.
	+When each jail lives in its own ZFS dataset, cap and guarantee its disk usage with ZFS quotas and reservations instead:
	+
	+[source,shell]
	+....
	+# zfs set quota=20G zroot/jails/containers/classic
	+# zfs set reservation=5G zroot/jails/containers/classic
	+....
	+
	+This requires the jail to be a dataset of its own.
	+The ZFS thin jails created earlier already are, but the Classic Jail procedure creates the jail directory with `mkdir` inside the shared [.filename]#containers# dataset, so create it with `zfs create` first for these commands to apply.
	+See crossref:jails[host-configuration-directories,"Setting Up the Jail Directory Tree"] for the dataset layout and man:zfs-set[8] for the `quota` and `reservation` properties.
	+
	+[[jail-more-options]]
	+== More Jail Options
	+
	+man:jail[8] exposes many more parameters than the examples in this chapter use.
	+A jail is a strong isolation boundary, but it is not an absolute one.
	+Every `allow.*` relaxation trades isolation for capability and moves the jail's privileged users a step closer to the privileges of the host's `root`.
	+Grant a jail only the permissions the workload inside it genuinely needs, and leave everything else at its more restrictive default.
	+The full list is in man:jail[8] and man:jail.conf[5]; the parameters below are the ones an administrator reaches for most often.
	+
	+=== The allow.* Permissions
	+
	+The `allow.*` parameters are booleans that hand individual privileges to the jail.
	+Per man:jail[8], "With the exception of allow.set_hostname and allow.reserved_ports, these boolean parameters are off by default", so a jail starts locked down and each capability must be granted explicitly.
	+
	+.Common `allow.*` parameters
	+[cols="2,1,3", options="header"]
	+\|===
	+\| Parameter \| Default \| Effect
	+
	+\| `allow.set_hostname`
	+\| On
	+\| The jail may change its own hostname.
	+
	+\| `allow.reserved_ports`
	+\| On
	+\| Jailed `root` may bind the reserved ports below 1024.
	+
	+\| `allow.raw_sockets`
	+\| Off
	+\| Enables raw sockets, so man:ping[8] and man:traceroute[8] work inside the jail.
	+
	+\| `allow.chflags`
	+\| Off
	+\| Jailed `root` may manipulate system file flags as it would on the host.
	+
	+\| `allow.mount`
	+\| Off
	+\| The jail may mount and unmount filesystems, together with the `allow.mount.*` children.
	+
	+\| `allow.quotas`
	+\| Off
	+\| The jail may administer filesystem quotas.
	+
	+\| `allow.read_msgbuf`
	+\| Off
	+\| Jailed processes may read the kernel message buffer with man:dmesg[8].
	+
	+\| `allow.socket_af`
	+\| Off
	+\| Allows socket address families other than IPv4, IPv6, local, and route.
	+
	+\| `allow.mlock`
	+\| Off
	+\| Jailed processes may lock physical memory with man:mlock[2].
	+
	+\| `allow.unprivileged_proc_debug`
	+\| Off
	+\| Unprivileged users may use debugging facilities on their own processes.
	+
	+\| `allow.vmm`
	+\| Off
	+\| The jail may use the man:vmm[4] hypervisor to run man:bhyve[8] guests.
	+
	+\| `allow.nfsd`
	+\| Off
	+\| The jail may run an NFS server.
	+\|===
	+
	+Weigh each relaxation against what it exposes: `allow.raw_sockets` lets jailed processes forge packets, `allow.chflags` lets jailed `root` clear the immutable and append-only flags the host relies on, and `allow.mount` opens up the filesystem namespace.
	+
	+Running an NFS server inside a jail, highlighted in the FreeBSD 14.0 release notes, needs `allow.nfsd` set, a VNET jail whose root directory is itself a filesystem mount point, and `enforce_statfs` not set to `0`.
	+
	+=== System V IPC
	+
	+Databases and other software that rely on System V shared memory, semaphores, and message queues are controlled per module with the `sysvmsg`, `sysvsem`, and `sysvshm` parameters.
	+Each takes one of three values: `inherit` makes all of the host's IPC objects visible to the jail, `new` gives the jail its own private key namespace, and `disable` makes the related system calls fail.
	+
	+For the classic case of running PostgreSQL in a jail, `new` is the right choice, giving the database its own shared-memory namespace without exposing the host's:
	+
	+[.programlisting]
	+....
	+sysvmsg = new;
	+sysvsem = new;
	+sysvshm = new;
	+....
	+
	+The older `allow.sysvipc` parameter is deprecated.
	+It is equivalent to setting all three module parameters to `inherit`, which shares the host-wide IPC namespace with the jail, so prefer the per-module parameters above.
	+
	+=== Mounting Filesystems
	+
	+`allow.mount` lets a jail mount filesystems, and a family of child parameters selects which types: `allow.mount.devfs`, `allow.mount.fdescfs`, `allow.mount.fusefs`, `allow.mount.nullfs`, `allow.mount.procfs`, `allow.mount.linprocfs`, `allow.mount.linsysfs`, `allow.mount.tmpfs`, and `allow.mount.zfs`.
	+Per man:jail[8], each child "is effective only together with allow.mount and only when enforce_statfs is set to a value lower than 2".
	+The convenience parameters `mount.devfs`, `mount.fdescfs`, and `mount.procfs` mount those filesystems for the jail automatically, and `mount.fstab` points at an fstab file of additional mounts.
	+
	+The `enforce_statfs` parameter controls both which mount points a jail can see and, through the rule above, whether the `allow.mount.*` permissions take effect.
	+It takes three values:
	+
	+.`enforce_statfs` values
	+[cols="1,4", options="header"]
	+\|===
	+\| Value \| Mount points visible to the jail
	+
	+\| `0`
	+\| All mount points on the host, with no restrictions.
	+
	+\| `1`
	+\| Only the mount points below the jail's root directory.
	+
	+\| `2` (default)
	+\| Only the single mount point where the jail's root directory is located.
	+\|===
	+
	+Because the `allow.mount.*` permissions require a value lower than `2`, a jail that must mount its own filesystems is typically configured with `enforce_statfs = 1`.
	+
	+=== Per-Jail securelevel
	+
	+The `securelevel` parameter sets the jail's `kern.securelevel` and raises the kernel's immutable-flag and device protections inside the jail.
	+Per man:jail[8], "A jail never has a lower securelevel than its parent system, but by setting this parameter it may have a higher one".
	+A network-facing jail can therefore run at a hardened `securelevel = 3` while the host stays at its default.
	+
	+=== Hierarchical Jails
	+
	+A jail may itself create child jails, which is how nested or hierarchical jail hierarchies are built.
	+The `children.max` parameter sets how many child jails a jail may create; it defaults to `0`, meaning no child jails are allowed, and the read-only `children.cur` reports how many currently exist.
	+A child can never exceed the privileges of its parent: per man:jail[8], "Jailed processes are not allowed to confer greater permissions than they themselves are given".
	+This makes it possible to run a jail manager inside a jail.
	+
	+=== Empty and Persistent Jails
	+
	+By default a jail exists only while it has running processes; when its last process exits, the jail disappears.
	+Setting the `persist` parameter, in the words of man:jail[8], "allows a jail to exist without any processes".
	+This is what keeps a jail alive after it has been created empty and before anything is populated into it, and what anchors a VNET jail that only holds network state.
	+A jail can be created empty and kept alive for experimentation:
	+
	+[source,shell]
	+....
	+# jail -c name=test path=/rescue persist
	+....
	+
	+Clear the flag with `nopersist` to let the jail be removed once idle again.
	+
	+[[jail-osrelease]]
	+=== Presenting a Different OS Release
	+
	+A jail's userland may be older than the host kernel, but tools inside it still read the host's version through man:uname[1] unless told otherwise.
	+The `osrelease` parameter sets the string reported by the jail's `kern.osrelease` and `uname -r`, and `osreldate` sets the number reported by `kern.osreldate` and `uname -K`.
	+Setting them so that an older userland reports its own release keeps man:uname[1], man:sysctl[8] consumers, and version-sniffing build tooling behaving correctly, including man:freebsd-update[8] run inside the jail.
	+Package management is unaffected either way, because man:pkg[8] derives its ABI from the ELF notes of the jail's own binaries rather than from man:uname[1].
	+This complements the rule that a jail's userland can never be newer than the host kernel; see crossref:jails[jail-upgrading,"Jail Upgrading"].
	+
	+[[jails-zfs]]
	+== Jails and ZFS
	+
	+A jail can be given control of one or more ZFS datasets, allowing the jailed root user to create, snapshot, clone, and roll back child datasets without any involvement from the host administrator.
	+This is the recommended way to give a jail its own writable, snapshot-capable storage.
	+
	+The host retains ownership of the dataset and only delegates management of the delegated tree and its descendants to the jail.
	+The `jailed` property, the `zfs jail` subcommand, and the interaction between the two are described in crossref:zfs[zfs-zfs-jail,ZFS and Jails].
	+
	+=== Delegating a Dataset Manually
	+
	+This method works on all supported releases of FreeBSD.
	+
	+On the host, create the dataset to delegate and mark it as jailed:
	+
	+[source,shell]
	+....
	+# zfs create zroot/jails/data
	+# zfs set jailed=on zroot/jails/data
	+....
	+
	+Setting the `jailed` property tells ZFS that the dataset will be managed from inside a jail, so the host stops mounting it automatically.
	+
	+Next, allow the jail to mount ZFS filesystems and attach the dataset when the jail is created.
	+Add the following to the jail's section in [.filename]#/etc/jail.conf#:
	+
	+[.programlisting]
	+....
	+allow.mount;
	+allow.mount.zfs;
	+enforce_statfs = 1;
	+exec.created += "zfs jail ${name} zroot/jails/data";
	+....
	+
	+The `allow.mount.zfs` permission takes effect only together with `allow.mount` and an `enforce_statfs` value below 2; see crossref:jails[jail-more-options,More Jail Options] for the mount permissions and how they interact with `enforce_statfs`.
	+Because a delegated dataset is detached when the jail stops, the `exec.created` hook re-attaches it on every start, where `${name}` expands to the jail name.
	+The same attachment can be performed against an already running jail with `zfs jail <jid\|name> zroot/jails/data`.
	+
	+[NOTE]
	+====
	+The jail needs access to [.filename]#/dev/zfs#.
	+The default devfs ruleset for jails (`devfsrules_jail`, ruleset 4) already unhides it, so no custom devfs ruleset is required for ZFS delegation.
	+====
	+
	+Inside the jail, mount the delegated datasets and manage them like any other:
	+
	+[source,shell]
	+....
	+# zfs mount -a
	+# zfs create zroot/jails/data/work
	+# zfs snapshot zroot/jails/data/work@monday
	+....
	+
	+The jailed root user can create, snapshot, clone, and roll back datasets under the delegated tree, but the `quota` property of the delegated dataset can only be set from the host.
	+
	+=== Attaching Datasets Automatically
	+
	+[NOTE]
	+====
	+The `zfs.dataset` parameter requires FreeBSD 15.0.
	+It is not present in the man:jail[8] of FreeBSD 14.3.
	+====
	+
	+Starting with FreeBSD 15.0, man:jail[8] can attach datasets on its own, removing the need for the manual `jailed` and `zfs jail` steps.
	+List the datasets to delegate in the `zfs.dataset` parameter:
	+
	+[.programlisting]
	+....
	+allow.mount;
	+allow.mount.zfs;
	+enforce_statfs = 1;
	+zfs.dataset = "zroot/jails/data";
	+....
	+
	+man:jail[8] attaches each listed dataset when the jail starts, so the `exec.created` hook shown above is no longer needed.
	+The parameter requires `allow.mount.zfs` to be set.
	+
	+For unmaintained jail managers that do not yet understand `zfs.dataset`, the rc.d/jail startup script also honors the legacy variable `jail_<jailname>_zfs_dataset` in [.filename]#/etc/rc.conf#.
	+
	+=== Accessing Snapshots from Inside a Jail
	+
	+The `zfs.mount_snapshot` parameter, available on FreeBSD 14.x and 15.x, lets jailed users reach snapshot contents without delegating a whole dataset.
	+When set to `1`, jailed users may read the contents of a filesystem's snapshots under its [.filename]#.zfs/snapshot# directory:
	+
	+[.programlisting]
	+....
	+zfs.mount_snapshot = 1;
	+....
	+
	+This is convenient for restoring individual files from host-taken snapshots.
	+If `allow.mount.zfs` is also set, the snapshots may additionally be mounted inside the jail.
	+The dataset's `snapdir` property controls whether the [.filename]#.zfs# directory is visible.
	+
	+[WARNING]
	+====
	+A delegated dataset combined with `allow.mount.zfs` lets the jailed root user set mount-related ZFS properties and mount filesystems inside the jail.
	+Grant these permissions deliberately, and only to jails that are trusted to manage their own storage.
	+====
	+
	[[jail-managers-and-containers]]
	== Jail Managers and Containers

	@@ -1255,3 +2402,6 @@
	\| link:https://erdgeist.org/arts/software/ezjail/[Documentation]

	\|===
	+
	+FreeBSD has published official OCI base-system images since FreeBSD 14.3-RELEASE, and OCI (Docker-style) containers -- which are built on the same jail technology and managed with the Podman toolkit -- are covered in the crossref:containers[containers,Containers] chapter, which complements the native jails described here.
	+

File Metadata

Mime Type: text/plain
Expires: Sun, Jul 12, 11:13 PM (16 h, 24 m)
Storage Engine: blob
Storage Format: Raw Data
Storage Handle: 35004738
Default Alt Text: D58173.diff (84 KB)

D58173.diffNo OneTemporaryActions

D58173.diffView Options

File Metadata

Event Timeline

D58173.diff
No OneTemporary
Actions

D58173.diff
View Options