diff --git a/documentation/content/en/books/handbook/jails/_index.adoc b/documentation/content/en/books/handbook/jails/_index.adoc --- a/documentation/content/en/books/handbook/jails/_index.adoc +++ b/documentation/content/en/books/handbook/jails/_index.adoc @@ -1,17 +1,17 @@ --- -title: Chapter 17. Jails +title: Chapter 17. Jails and Containers part: Part III. System Administration prev: books/handbook/security next: books/handbook/mac description: Jails improve on the concept of the traditional chroot environment in several ways -tags: ["jails", "creating", "managing", "updating", "ezjail"] +tags: ["jails", "creating", "managing", "updating"] showBookMenu: true weight: 21 path: "/books/handbook/" --- [[jails]] -= Jails += Jails and Containers :doctype: book :toc: macro :toclevels: 1 @@ -48,6 +48,23 @@ include::../../../../../shared/asciidoctor.adoc[] endif::[] +// Related bugs to this chapter: 203641, 200905, 209157, 248150, 263330, 264317, 226282, 239861, 166358, 178221, 265410 +// Documentation checked +// https://man.freebsd.org/cgi/man.cgi?query=jail&sektion=8 +// https://wiki.freebsd.org/Jails +// https://man.freebsd.org/cgi/man.cgi?rctl(8) +// https://vermaden.wordpress.com/2023/06/28/freebsd-jails-containers/ +// https://clinta.github.io/freebsd-jails-the-hard-way/ +// https://marcocetica.com/posts/understanding-freebsd-jail/ +// https://jacob.ludriks.com/2017/06/07/FreeBSD-Thin-Jails/ +// Jails and VNET a guide - Derik J. Ramirez +// https://wb-hk.blogspot.com/2016/04/freebsd-jails-4-thin-jails-using-nullfs.html +// https://blog.uidrafter.com/freebsd-jails-network-setup +// https://forums.freebsd.org/threads/setting-up-a-debian-linux-jail-on-freebsd.68434/ +// https://github.com/msimerson/Mail-Toaster-6/wiki/Linux-Jails +// https://etherealwake.com/2021/08/freebsd-jail-networking/ +// https://wiki.freebsd.org/TomMarcoen/JailNetworking + [[jails-synopsis]] == Synopsis @@ -60,1171 +77,775 @@ This creates a safe environment, separate from the rest of the system. Processes created in the chrooted environment can not access files or resources outside of it. For that reason, compromising a service running in a chrooted environment should not allow the attacker to compromise the entire system. + However, a chroot has several limitations. It is suited to easy tasks which do not require much flexibility or complex, advanced features. Over time, many ways have been found to escape from a chrooted environment, making it a less than ideal solution for securing services. Jails improve on the concept of the traditional chroot environment in several ways. + In a traditional chroot environment, processes are only limited in the part of the file system they can access. The rest of the system resources, system users, running processes, and the networking subsystem are shared by the chrooted processes and the processes of the host system. Jails expand this model by virtualizing access to the file system, the set of users, and the networking subsystem. More fine-grained controls are available for tuning the access of a jailed environment. Jails can be considered as a type of operating system-level virtualization. -A jail is characterized by four elements: - -* A directory subtree: the starting point from which a jail is entered. Once inside the jail, a process is not permitted to escape outside of this subtree. -* A hostname: which will be used by the jail. -* An IP address: which is assigned to the jail. The IP address of a jail is often an alias address for an existing network interface. -* A command: the path name of an executable to run inside the jail. The path is relative to the root directory of the jail environment. - -Jails have their own set of users and their own `root` account which are limited to the jail environment. -The `root` account of a jail is not allowed to perform operations to the system outside of the associated jail environment. - -This chapter provides an overview of the terminology and commands for managing FreeBSD jails. -Jails are a powerful tool for both system administrators, and advanced users. - -After reading this chapter, you will know: +This chapter covers: * What a jail is and what purpose it may serve in FreeBSD installations. * How to build, start, and stop a jail. * The basics of jail administration, both from inside and outside the jail. -[IMPORTANT] -==== -Jails are a powerful tool, but they are not a security panacea. -While it is not possible for a jailed process to break out on its own, there are several ways in which an unprivileged user outside the jail can cooperate with a privileged user inside the jail to obtain elevated privileges in the host environment. +[[jail-types]] +== Jail types -Most of these attacks can be mitigated by ensuring that the jail root is not accessible to unprivileged users in the host environment. -As a general rule, untrusted users with privileged access to a jail should not be given access to the host environment. -==== +Some administrators divide jails into a different types, although the underlying technology is the same. +Each administrator will have to assess what type of jail to create in each case depending on the problem they have to solve. -[[jails-terms]] -== Terms Related to Jails +Below can be found a list of the different types, their characteristics, etc. -To facilitate better understanding of parts of the FreeBSD system related to jails, their internals and the way they interact with the rest of FreeBSD, the following terms are used further in this chapter: +[[thick-jails]] +=== Thick Jails -man:chroot[8] (command):: -Utility, which uses man:chroot[2] FreeBSD system call to change the root directory of a process and all its descendants. +A thick jail is a more traditional form of FreeBSD Jail. +In a thick jail, a complete copy of the base system is replicated within the jail's environment. +This means that the jail has its own separate instance of the FreeBSD base system, including libraries, executables, and configuration files. +The jail can be thought of as an almost complete standalone FreeBSD installation, but running within the confines of the host system. +This isolation ensures that the processes within the jail are kept separate from those on the host and other jails. -man:chroot[2] (environment):: -The environment of processes running in a "chroot". -This includes resources such as the part of the file system which is visible, user and group IDs which are available, network interfaces and other IPC mechanisms, etc. +Advantages of Thick Jails: -man:jail[8] (command):: -The system administration utility which allows launching of processes within a jail environment. +* High degree of isolation: Processes within the jail are isolated from the host system and other jails. +* Independence: Thick jails can have different versions of libraries, configurations, and software than the host system or other jails. +* Security: Since the jail contains its own base system, vulnerabilities or issues affecting the jail environment won't directly impact the host or other jails. -host (system, process, user, etc.):: -The controlling system of a jail environment. -The host system has access to all the hardware resources available, and can control processes both outside of and inside a jail environment. -One of the important differences of the host system from a jail is that the limitations which apply to superuser processes inside a jail are not enforced for processes of the host system. +Disadvantages of Thick Jails: -hosted (system, process, user, etc.):: -A process, user or other entity, whose access to resources is restricted by a FreeBSD jail. +* Resource overhead: Because each jail maintains its own separate base system, it consumes more resources compared to thin jails. +* Maintenance: Each jail requires its own maintenance and updates for its base system components. -[[jails-build]] -== Creating and Controlling Jails +[[thin-jails]] +=== Thin Jails -Some administrators divide jails into the following two types: "complete" jails, which resemble a real FreeBSD system, and "service" jails, dedicated to one application or service, possibly running with privileges. -This is only a conceptual division and the process of building a jail is not affected by it. -When creating a "complete" jail there are two options for the source of the userland: use prebuilt binaries (such as those supplied on an install media) or build from source. +A thin jail shares the base system using OpenZFS snapshots o NullFS mounts. +Only a minimal subset of base system is duplicated for each thin jail, resulting in less resource consumption compared to a thick jail. +However, this also means that thin jails have less isolation and independence compared to thick jails. +Changes in shared components could potentially affect multiple thin jails simultaneously. -=== Installing a Jail +In summary, a FreeBSD Thick Jail is a type of FreeBSD Jail that replicates a substantial portion of the base system within the isolated environment. +This provides a high level of isolation and independence, but also requires more resources and maintenance compared to thin jails. -[[jails-install-internet]] -==== To install a Jail from the Internet +Advantages of Thin Jails: -The man:bsdinstall[8] tool can be used to fetch and install the binaries needed for a jail. -This will walk through the picking of a mirror, which distributions will be installed into the destination directory, and some basic configuration of the jail: +* Resource Efficiency: Thin jails are more resource-efficient compared to thick jails. Since they share most base system, they consume less disk space and memory. This makes it possible to run more jails on the same hardware without consuming excessive resources. +* Faster Deployment: Creating and launching thin jails is generally faster compared to thick jails. This can be particularly advantageous when you need to rapidly deploy multiple instances. +* Unified Maintenance: Since thin jails share the majority of their base system with the host system, updates and maintenance of common base system components (such as libraries and binaries) only need to be done once on the host. This simplifies the maintenance process compared to maintaining individual base system for each thick jail. +* Reduced Disk Usage: Thin jails require less disk space compared to thick jails because they don't store redundant copies of the entire base system. This can be important when disk space is a concern. +* Shared Resources: Thin jails can more easily share common resources such as libraries and binaries with the host system. This can potentially lead to more efficient disk caching and improved performance for applications within the jail. -[source,shell] -.... -# bsdinstall jail /here/is/the/jail -.... +Disadvantages of Thin Jails: -Once the command is complete, the next step is configuring the host to run the jail. +* Reduced Isolation: The primary disadvantage of thin jails is that they offer less isolation compared to thick jails. Since they share a significant portion of the host system's base system, vulnerabilities or issues affecting shared components could potentially impact multiple jails simultaneously. +* Limited Independence: Thin jails might be limited in terms of running different versions of libraries or software compared to the host system. This could be a concern if the applications within the jail require specific versions that differ from the host. +* Security Concerns: The reduced isolation in thin jails could pose security risks, as a compromise in one jail might have a greater potential to affect other jails or the host system. +* Dependency Conflicts: If multiple thin jails require different versions of the same libraries or software, managing dependencies can become complex. In some cases, this might require additional effort to ensure compatibility. +* Compatibility Challenges: Applications within a thin jail might encounter compatibility issues if they assume a certain base system environment that differs from the shared components provided by the host system. -[[jails-install-iso]] -==== To install a Jail from an ISO +[[vnet-jails]] +=== VNET Jails -To install the userland from installation media, first create the root directory for the jail. -This can be done by setting the `DESTDIR` variable to the proper location. +A FreeBSD VNET Jail is a specialized type of jail in the FreeBSD operating system that leverages the VNET framework for network virtualization. +VNET stands for "Virtual NETwork," and it allows to create isolated network environments within a jail. +This means that each VNET Jail has its own separate network stack, including interfaces, routing tables, and firewall rules, distinct from the host system and other jails. +This enhances network isolation and security, as well as provides greater flexibility in configuring network settings within each jail. +This allows for more advanced network configurations and better isolation of services while sharing the same underlying hardware. -Start a shell and define `DESTDIR`: +[[linux-jails]] +=== Linux Jails -[source,shell] -.... -# sh -# export DESTDIR=/here/is/the/jail -.... +A FreeBSD Linux Jail is a feature in the FreeBSD operating system that enables to run Linux binaries and applications within a FreeBSD jail. +This functionality is achieved by incorporating a compatibility layer that allows certain Linux system calls and libraries to be translated and executed on the FreeBSD kernel. +The purpose of a Linux Jail is to facilitate the execution of Linux software on a FreeBSD system without needing a separate Linux virtual machine or environment. -Mount the install media as covered in man:mdconfig[8] when using the install ISO: - -[source,shell] -.... -# mount -t cd9660 /dev/`mdconfig -f cdimage.iso` /mnt -# cd /mnt/usr/freebsd-dist/ -.... +[[host-configuration]] +== Host Configuration -Extract the binaries from the tarballs on the install media into the declared destination. -Minimally, only the base set needs to be extracted, but a complete install can be performed when preferred. +Before creating any jail on the host system it is necessary to perform certain configurations and obtain some information from the host system. -To install just the base system: +It will be necessary to configure the man:jail[8] utility, create the necessary directories to configure and install the jails, obtain information from the host's network and check if the host uses OpenZFS or UFS as the file system. -[source,shell] -.... -# tar -xf base.txz -C $DESTDIR -.... +[TIP] +==== +The FreeBSD host can run any other FreeBSD version in a jail as long as its not newer then the host system version. +==== -To install everything except the kernel: +[[host-configuration-jail-Utility]] +=== Jail Utility -[source,shell] -.... -# for set in base ports; do tar -xf $set.txz -C $DESTDIR ; done -.... +The man:jail[8] utility creates new jails, or modifies or removes existing jails. -[[jails-install-source]] -==== To build and install a Jail from source +To be able to execute jails on the host it is necessary to activate the man:jail[8] service. -The man:jail[8] manual page explains the procedure for building a jail: +Enable man:jail[8] service in [.filename]#/etc/rc.conf# to start at system boot: [source,shell] .... -# setenv D /here/is/the/jail -# mkdir -p $D <.> -# cd /usr/src -# make buildworld <.> -# make installworld DESTDIR=$D <.> -# make distribution DESTDIR=$D <.> -# mount -t devfs devfs $D/dev <.> +# sysrc jail_enable="YES" +# sysrc jail_parallel_start="YES" .... -<.> Selecting a location for a jail is the best starting point. This is where the jail will physically reside within the file system of the jail's host. A good choice can be [.filename]#/usr/jail/jailname#, where _jailname_ is the hostname identifying the jail. Usually, [.filename]#/usr/# has enough space for the jail file system, which for "complete" jails is, essentially, a replication of every file present in a default installation of the FreeBSD base system. +[TIP] +==== +With `jail_parallel_start` all configured jails will be started in the background. +==== -<.> If you have already rebuilt your userland using `make world` or `make buildworld`, you can skip this step and install your existing userland into the new jail. +[[jails-networking]] +=== Networking -<.> This command will populate the directory subtree chosen as jail's physical location on the file system with the necessary binaries, libraries, manual pages and so on. -<.> The `distribution` target for make installs every needed configuration file. In simple words, it installs every installable file of [.filename]#/usr/src/etc/# to the [.filename]#/etc# directory of the jail environment: [.filename]#$D/etc/#. +The FreeBSD jails network configuration supports different ways of configuration: -<.> Mounting the man:devfs[8] file system inside a jail is not required. On the other hand, any, or almost any application requires access to at least one device, depending on the purpose of the given application. It is very important to control access to devices from inside a jail, as improper settings could permit an attacker to do nasty things in the jail. Control over man:devfs[8] is managed through rulesets which are described in the man:devfs[8] and man:devfs.conf[5] manual pages. +Host Networking Mode (IP Sharing):: +In host networking mode, a jail shares the same networking stack as the host system. +When a jail is created in host networking mode it uses the same network interface and IP address. +This means that the jail doesn't have a separate IP address, and its network traffic is associated with the host's IP. -=== Configuring the Host +Virtual Networks (VNET):: +Virtual Networks are a feature in FreeBSD jails that provides a more advanced and flexible networking solution compared to the basic networking mode like host networking. +VNET allows to create isolated network stacks for each jail, providing them with their own separate IP addresses, routing tables, and network interfaces. +This offers a higher level of network isolation and allows jails to function as if they are running on separate virtual machines. -Once a jail is installed, it can be started by using the man:jail[8] utility. -The man:jail[8] utility takes four mandatory arguments which are described in the <>. -Other arguments may be specified too, e.g., to run the jailed process with the credentials of a specific user. -The `_command_` argument depends on the type of the jail; for a _virtual system_, [.filename]#/etc/rc# is a good choice, since it will replicate the startup sequence of a real FreeBSD system. -For a _service_ jail, it depends on the service or application that will run within the jail. +The netgraph system:: +Netgraph (man:netgraph[4]) is a kernel framework that provides a way to model and connect various networking components using nodes and links. +It's a versatile technology that allows you to create complex networking configurations and protocols by connecting different nodes together. +Netgraph is often used in networking applications and services to implement custom networking behaviors. ++ +When using Netgraph within FreeBSD jails, custom networking topologies can be set up by creating and configuring Netgraph nodes and links. +This allows to define how network traffic flows between jails and the host system, as well as between different jails. -Jails are often started at boot time and the FreeBSD [.filename]#rc# mechanism provides an easy way to do this. +[[host-configuration-directories]] +=== Setting up the Jail Directory Tree -[.procedure] -* Configure jail parameters in [.filename]#jail.conf#: -+ -[.programlisting] -.... -www { - host.hostname = www.example.org; # Hostname - ip4.addr = 192.168.0.10; # IP address of the jail - path = "/usr/jail/www"; # Path to the jail - mount.devfs; # Mount devfs inside the jail - exec.start = "/bin/sh /etc/rc"; # Start command - exec.stop = "/bin/sh /etc/rc.shutdown"; # Stop command -} -.... +There is no specific place to put the files for the jails. -+ -Configure jails to start at boot time in [.filename]#rc.conf#: -+ -[.programlisting] -.... -jail_enable="YES" # Set to NO to disable starting of any jails -.... -+ -The default startup of jails configured in man:jail.conf[5], will run the [.filename]#/etc/rc# script of the jail, which assumes the jail is a complete virtual system. -For service jails, the default startup command of the jail should be changed, by setting the `exec.start` option appropriately. -+ -[NOTE] -==== -For a full list of available options, please see the man:jail.conf[5] manual page. -==== +Some administrators use [.filename]#/jail#, others [.filename]#/usr/jail#, and still others [.filename]#/usr/local/jails#. +In this chapter [.filename]#/usr/local/jails# will be used. + +Apart from [.filename]#/usr/local/jails# another directories will be created: -man:service[8] can be used to start or stop a jail by hand, if an entry for it exists in [.filename]#jail.conf#: +* [.filename]#media# will contain the compressed files of the downloaded userlands. +* [.filename]#releases# will contain the releases used as the base to create the templates. +* [.filename]#templates# will contain the templates in case of using Thin Jails. + +In case of using OpenZFS execute the following commands to create the dataset: [source,shell] .... -# service jail start www -# service jail stop www +# zfs create -o mountpoint=/usr/local/jails zroot/jails +# zfs create -p zroot/jails/media +# zfs create -p zroot/jails/releases +# zfs create -p zroot/jails/templates .... -Jails can be shut down with man:jexec[8]. -Use man:jls[8] to identify the jail's `JID`, then use man:jexec[8] to run the shutdown script in that jail. +In case of using UFS execute the following commands to create the directories: [source,shell] .... -# jls - JID IP Address Hostname Path - 3 192.168.0.10 www /usr/jail/www -# jexec 3 /etc/rc.shutdown +# mkdir /usr/local/jails/ +# mkdir /usr/local/jails/media +# mkdir /usr/local/jails/releases +# mkdir /usr/local/jails/templates .... -More information about this can be found in the man:jail[8] manual page. - -[[jails-tuning]] -== Fine Tuning and Administration - -There are several options which can be set for any jail, and various ways of combining a host FreeBSD system with jails, to produce higher level applications. -This section presents: +[[jail-configuration-files]] +=== Jail Configuration Files -* Some of the options available for tuning the behavior and security restrictions implemented by a jail installation. -* Some of the high-level applications for jail management, which are available through the FreeBSD Ports Collection, and can be used to implement overall jail-based solutions. +There are 2 ways to configure the jails. -[[jails-tuning-utilities]] -=== System Tools for Jail Tuning in FreeBSD +The first one is to add an entry for each jail to the file [.filename]#/etc/jail.conf#. +The other option is to create a file for each jail in the directory [.filename]#/etc/jail.conf.d/#. -Fine tuning of a jail's configuration is mostly done by setting man:sysctl[8] variables. -A special subtree of sysctl exists as a basis for organizing all the relevant options: the `security.jail.*` hierarchy of FreeBSD kernel options. -Here is a list of the main jail-related sysctls, complete with their default value. -Names should be self-explanatory, but for more information about them, please refer to the man:jail[8] and man:sysctl[8] manual pages. +There is no right or wrong option, each administrator must choose the one that best suits their needs. -* `security.jail.set_hostname_allowed: 1` -* `security.jail.socket_unixiproute_only: 1` -* `security.jail.sysvipc_allowed: 0` -* `security.jail.enforce_statfs: 2` -* `security.jail.allow_raw_sockets: 0` -* `security.jail.chflags_allowed: 0` -* `security.jail.jailed: 0` +In case a host system has few jails, an entry for each jail can be added in the file [.filename]#/etc/jail.conf#. +If the host system has many jails, it is good idea to have one configuration file for each jail in the [.filename]#/etc/jail.conf.d/# directory. -These variables can be used by the system administrator of the _host system_ to add or remove some of the limitations imposed by default on the `root` user. -Note that there are some limitations which cannot be removed. -The `root` user is not allowed to mount or unmount file systems from within a man:jail[8]. -The `root` inside a jail may not load or unload man:devfs[8] rulesets, set firewall rules, or do many other administrative tasks which require modifications of in-kernel data, such as setting the `securelevel` of the kernel. +The vast majority of jails share certain variables which can be defined globally in the [.filename]#/etc/jail.conf# file. -The base system of FreeBSD contains a basic set of tools for viewing information about the active jails, and attaching to a jail to run administrative commands. -The man:jls[8] and man:jexec[8] commands are part of the base FreeBSD system, and can be used to perform the following simple tasks: +A typical [.filename]#/etc/jail.conf# with globally defined variables for all jails and a configured jail would look like this: -* Print a list of active jails and their corresponding jail identifier (JID), IP address, hostname and path. -* Attach to a running jail, from its host system, and run a command inside the jail or perform administrative tasks inside the jail itself. This is especially useful when the `root` user wants to cleanly shut down a jail. The man:jexec[8] utility can also be used to start a shell in a jail to do administration in it; for example: -+ -[source,shell] +[.programlisting] .... -# jexec 1 tcsh +exec.start = "/bin/sh /etc/rc"; <.> +exec.stop = "/bin/sh /etc/rc.shutdown"; <.> +exec.clean; <.> +mount.devfs; <.> + +jailname { <.> + # General data + host.hostname = "${name}"; <.> + path = "/usr/local/jails/${name}"; <.> + + # Logs + exec.consolelog = "/var/log/jail_console_${name}.log"; <.> + + # Network section + ip4.addr = 192.168.1.151; <.> + interface = em0; <.> + + # Security + allow.raw_sockets = 1; <.> +} .... -[[jails-tuning-admintools]] -=== High-Level Administrative Tools in the FreeBSD Ports Collection +<.> `exec.start` - Command(s) to run in the jail environment when a jail is created. A typical command to run is "/bin/sh /etc/rc". +<.> `exec.stop` - Command(s) to run in the jail environment before a jail is removed. A typical command to run is "/bin/sh /etc/rc.shutdown". +<.> `exec.clean` - Run commands in a clean environment. +<.> `mount.devfs` - Mount a man:devfs[5] filesystem on the chrooted [.filename]#/dev# directory, and apply the ruleset in the devfs_ruleset parameter to restrict the devices visible inside the jail. +<.> `jailname` - Name of the jail. +<.> `host.hostname` - The hostname of the jail. +<.> `path` - The directory which is to be the root of the jail. Any commands run inside the jail, either by jail or from man:jexec[8], are run from this directory. +<.> `exec.consolelog` - A file to direct command output (stdout and stderr) to. +<.> `ip4.addr` - IPv4 address. +<.> `interface` - A network interface to add the jail's IP addresses. Usually the host interface. +<.> `allow.raw_sockets` - Allow to create raw sockets. Setting this parameter allows utilities like man:ping[8] and man:traceroute[8] to operate inside the jail. -Among the many third-party utilities for jail administration, one of the most complete and useful is package:sysutils/ezjail[]. -It is a set of scripts that contribute to man:jail[8] management. -Please refer to <> for more information. +More information about configuration variables can be found in man:jail[8] and man:jail.conf[5]. -[[jails-updating]] -=== Keeping Jails Patched and up to Date +[[classic-jail]] +== Classic Jail (Thick Jail) -Jails should be kept up to date from the host operating system as attempting to patch userland from within the jail may likely fail as the default behavior in FreeBSD is to disallow the use of man:chflags[1] in a jail which prevents the replacement of some files. -It is possible to change this behavior but it is recommended to use man:freebsd-update[8] to maintain jails instead. -Use `-b` to specify the path of the jail to be updated. +As previously indicated, a conceptual division could be made and classify "Thick Jails" as the classic FreeBSD jails. +This jails resemble a real FreeBSD system. +They can be managed more or less like a normal host system and updated independently. -To update the jail to the latest patch release of the version of FreeBSD it is already running, then execute the following commands on the host: +[[creating-classic-jail]] +=== Creating a Classic Jail + +Officially, a jail only needs a hostname, a root directory, an IP address and a userland. + +The userland for the jail can be obtained from the official FreeBSD download servers. + +Execute the following command to download the userland: [source,shell] .... -# freebsd-update -b /here/is/the/jail fetch -# freebsd-update -b /here/is/the/jail install +# fetch https://download.freebsd.org/ftp/releases/amd64/amd64/13.2-RELEASE/base.txz -o /usr/local/jails/media/13.2-RELEASE-base.txz .... -To upgrade the jail to a new major or minor version, first upgrade the host system as described in crossref:cutting-edge[freebsdupdate-upgrade,“Performing Major and Minor Version Upgrades”]. -Once the host has been upgraded and rebooted, the jail can then be upgraded. -For example to upgrade from 12.2-RELEASE to 12.3-RELEASE, on the host run: +Once the download is complete it will be necessary to unzip the contents in the jail directory. + +Execute the following commands to extract the userland into jail's directory: [source,shell] .... -# freebsd-update -b /here/is/the/jail --currently-running 12.2-RELEASE -r 12.3-RELEASE upgrade -# freebsd-update -b /here/is/the/jail install -# service jail restart myjail -# freebsd-update -b /here/is/the/jail install +# mkdir /usr/local/jails/classic +# tar -xf /usr/local/jails/media/13.2-RELEASE-base.txz -C /usr/local/jails/classic --unlink .... -Then, if it was a major version upgrade, reinstall all installed packages and restart the jail again. -This is required because the ABI version changes when upgrading between major versions of FreeBSD. -From the host: +With the userland extracted in the jail directory, will be necessary to copy the timezone and the DNS servers files: [source,shell] .... -# pkg -j myjail upgrade -f -# service jail restart myjail +# cp /etc/resolv.conf /usr/local/jails/templates/13.2-RELEASE/etc/resolv.conf +# cp /etc/localtime /usr/local/jails/templates/13.2-RELEASE/etc/localtime .... -[[jails-application]] -== Updating Multiple Jails - -The management of multiple jails can become problematic because every jail has to be rebuilt from scratch whenever it is upgraded. -This can be time consuming and tedious if a lot of jails are created and manually updated. - -This section demonstrates one method to resolve this issue by safely sharing as much as is possible between jails using read-only man:mount_nullfs[8] mounts, so that updating is simpler. -This makes it more attractive to put single services, such as HTTP, DNS, and SMTP, into individual jails. -Additionally, it provides a simple way to add, remove, and upgrade jails. - -[NOTE] -==== -Simpler solutions exist, such as ezjail, which provides an easier method of administering FreeBSD jails but is less versatile than this setup. -ezjail is covered in more detail in <>. -==== +Once the jail has been created, the next step will be to configure it, which is explained in <>. -The goals of the setup described in this section are: +[[thin-jail]] +== Thin Jails -* Create a simple and easy to understand jail structure that does not require running a full installworld on each and every jail. -* Make it easy to add new jails or remove existing ones. -* Make it easy to update or upgrade existing jails. -* Make it possible to run a customized FreeBSD branch. -* Be paranoid about security, reducing as much as possible the possibility of compromise. -* Save space and inodes, as much as possible. +Although the Thin Jails use the same technology, the creation is different. +Thin jails can be created using OpenZFS snapshots or using templates and NullFS. +The use of OpenZFS snapshots and templates using NullFS have certain advantages over classic jails, +such as being able to create them faster from snapshots or being able to update multiple jails using NullFS. -This design relies on a single, read-only master template which is mounted into each jail and one read-write device per jail. -A device can be a separate physical disc, a partition, or a vnode backed memory device. -This example uses read-write nullfs mounts. +[[creating-thin-jail-openzfs-snapshots]] +=== Creating a Thin Jail using OpenZFS Snapshots -The file system layout is as follows: +Due to the good integration between FreeBSD and OpenZFS it is very easy to create new Think Jails using OpenZFS Snapshots. -* The jails are based under the [.filename]#/home# partition. -* Each jail will be mounted under the [.filename]#/home/j# directory. -* The template for each jail and the read-only partition for all of the jails is [.filename]#/home/j/mroot#. -* A blank directory will be created for each jail under the [.filename]#/home/j# directory. -* Each jail will have a [.filename]#/s# directory that will be linked to the read-write portion of the system. -* Each jail will have its own read-write system that is based upon [.filename]#/home/j/skel#. -* The read-write portion of each jail will be created in [.filename]#/home/js#. +To create a Thin Jail using OpenZFS Snapshots the first step will be to create a template. -[[jails-service-jails-template]] -=== Creating the Template +The templates will only be used to create new jails, for this reason they are created in "read-only" mode, in order to create jails easily with an immutable base. -This section describes the steps needed to create the master template. +To create the dataset to save the template execute the following command: -It is recommended to first update the host FreeBSD system to the latest -RELEASE branch using the instructions in crossref:cutting-edge[makeworld,“Updating FreeBSD from Source”]. -Additionally, this template uses the package:sysutils/cpdup[] package or port and link:{handbook}mirrors/#git[Git] will be used to download the FreeBSD Ports Collection. - -[.procedure] -. First, create a directory structure for the read-only file system which will contain the FreeBSD binaries for the jails. Then, change directory to the FreeBSD source tree and install the read-only file system to the jail template: -+ [source,shell] .... -# mkdir /home/j /home/j/mroot -# cd /usr/src -# make installworld DESTDIR=/home/j/mroot +# zfs create -p zroot/jails/templates/13.2-RELEASE .... -. Next, prepare a FreeBSD Ports Collection for the jails as well as a FreeBSD source tree, which is required for mergemaster: -+ -[source,shell] -.... -# cd /home/j/mroot -# mkdir usr/ports -# git clone -o freebsd https://git.FreeBSD.org/ports.git /home/j/mroot/usr/ports -# cpdup /usr/src /home/j/mroot/usr/src -.... +Then execute the following command to download the userland: -. Create a skeleton for the read-write portion of the system: -+ [source,shell] .... -# mkdir /home/j/skel /home/j/skel/home /home/j/skel/usr-X11R6 /home/j/skel/distfiles -# mv etc /home/j/skel -# mv usr/local /home/j/skel/usr-local -# mv tmp /home/j/skel -# mv var /home/j/skel -# mv root /home/j/skel +# fetch https://download.freebsd.org/ftp/releases/amd64/amd64/13.2-RELEASE/base.txz -o /usr/local/jails/media/13.2-RELEASE-base.txz .... -. Use mergemaster to install missing configuration files. Then, remove the extra directories that mergemaster creates: -+ -[source,shell] -.... -# mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i -# cd /home/j/skel -# rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev -.... +Once the download is complete it will be necessary to unzip the contents in the template directory executing the following command: -. Now, symlink the read-write file system to the read-only file system. Ensure that the symlinks are created in the correct [.filename]#s/# locations as the creation of directories in the wrong locations will cause the installation to fail. -+ [source,shell] .... -# cd /home/j/mroot -# mkdir s -# ln -s s/etc etc -# ln -s s/home home -# ln -s s/root root -# ln -s ../s/usr-local usr/local -# ln -s ../s/usr-X11R6 usr/X11R6 -# ln -s ../../s/distfiles usr/ports/distfiles -# ln -s s/tmp tmp -# ln -s s/var var +# tar -xf /usr/local/jails/media/13.2-RELEASE-base.txz -C /usr/local/jails/templates/13.2-RELEASE --unlink .... -. As a last step, create a generic [.filename]#/home/j/skel/etc/make.conf# containing this line: -+ -[.programlisting] -.... -WRKDIRPREFIX?= /s/portbuild -.... -+ -This makes it possible to compile FreeBSD ports inside each jail. -Remember that the ports directory is part of the read-only system. -The custom path for `WRKDIRPREFIX` allows builds to be done in the read-write portion of every jail. - -[[jails-service-jails-creating]] -=== Creating Jails - -The jail template can now be used to setup and configure the jails in [.filename]#/etc/rc.conf#. -This example demonstrates the creation of 3 jails: `NS`, `MAIL` and `WWW`. +With the userland extracted in the templates directory, will be necessary to copy the timezone and the DNS servers files to the template directory executing the following command: -[.procedure] -. Add the following lines to [.filename]#/etc/fstab#, so that the read-only template for the jails and the read-write space will be available in the respective jails: -+ -[.programlisting] -.... -/home/j/mroot /home/j/ns nullfs ro 0 0 -/home/j/mroot /home/j/mail nullfs ro 0 0 -/home/j/mroot /home/j/www nullfs ro 0 0 -/home/js/ns /home/j/ns/s nullfs rw 0 0 -/home/js/mail /home/j/mail/s nullfs rw 0 0 -/home/js/www /home/j/www/s nullfs rw 0 0 -.... -+ -To prevent fsck from checking nullfs mounts during boot and dump from backing up the read-only nullfs mounts of the jails, the last two columns are both set to `0`. -. Configure the jails in [.filename]#/etc/rc.conf#: -+ -[.programlisting] -.... -jail_enable="YES" -jail_set_hostname_allow="NO" -jail_list="ns mail www" -jail_ns_hostname="ns.example.org" -jail_ns_ip="192.168.3.17" -jail_ns_rootdir="/usr/home/j/ns" -jail_ns_devfs_enable="YES" -jail_mail_hostname="mail.example.org" -jail_mail_ip="192.168.3.18" -jail_mail_rootdir="/usr/home/j/mail" -jail_mail_devfs_enable="YES" -jail_www_hostname="www.example.org" -jail_www_ip="62.123.43.14" -jail_www_rootdir="/usr/home/j/www" -jail_www_devfs_enable="YES" -.... -+ -The `jail__name__rootdir` variable is set to [.filename]#/usr/home# instead of [.filename]#/home# because the physical path of [.filename]#/home# on a default FreeBSD installation is [.filename]#/usr/home#. -The `jail__name__rootdir` variable must _not_ be set to a path which includes a symbolic link, otherwise the jails will refuse to start. -. Create the required mount points for the read-only file system of each jail: -+ [source,shell] .... -# mkdir /home/j/ns /home/j/mail /home/j/www +# cp /etc/resolv.conf /usr/local/jails/templates/13.2-RELEASE/etc/resolv.conf +# cp /etc/localtime /usr/local/jails/templates/13.2-RELEASE/etc/localtime .... -. Install the read-write template into each jail using package:sysutils/cpdup[]: -+ -[source,shell] -.... -# mkdir /home/js -# cpdup /home/j/skel /home/js/ns -# cpdup /home/j/skel /home/js/mail -# cpdup /home/j/skel /home/js/www -.... +With the files moved to the template, the next thing to do is update to the latest patch level executing the following command: -. In this phase, the jails are built and prepared to run. First, mount the required file systems for each jail, and then start them: -+ [source,shell] .... -# mount -a -# service jail start +# freebsd-update -b /usr/local/jails/templates/13.2-RELEASE/ fetch install .... -The jails should be running now. -To check if they have started correctly, use `jls`. -Its output should be similar to the following: +Once the update is finished the template will be ready. -[source,shell] -.... -# jls - JID IP Address Hostname Path - 3 192.168.3.17 ns.example.org /home/j/ns - 2 192.168.3.18 mail.example.org /home/j/mail - 1 62.123.43.14 www.example.org /home/j/www -.... - -At this point, it should be possible to log onto each jail, add new users, or configure daemons. -The `JID` column indicates the jail identification number of each running jail. -Use the following command to perform administrative tasks in the jail whose JID is `3`: +To create the OpenZFS Snapshot from the template execute the following command: [source,shell] .... -# jexec 3 tcsh +# zfs snapshot zroot/jails/templates/13.2-RELEASE@base .... -[[jails-service-jails-upgrading]] -=== Upgrading +Once the OpenZFS Snapshot has been created infinite jails can be created using the OpenZFS clone function. -The design of this setup provides an easy way to upgrade existing jails while minimizing their downtime. -Also, it provides a way to roll back to the older version should a problem occur. +To create the Thin Jail execute the following command: -[.procedure] -. The first step is to upgrade the host system. Then, create a new temporary read-only template in [.filename]#/home/j/mroot2#. -+ -[source,shell] -.... -# mkdir /home/j/mroot2 -# cd /usr/src -# make installworld DESTDIR=/home/j/mroot2 -# cd /home/j/mroot2 -# cpdup /usr/src usr/src -# mkdir s -.... -+ -The `installworld` creates a few unnecessary directories, which should be removed: -+ [source,shell] .... -# chflags -R 0 var -# rm -R etc var root usr/local tmp +# zfs clone zroot/jails/templates/13.2-RELEASE@base zroot/jails/thinjail .... -. Recreate the read-write symlinks for the master file system: -+ -[source,shell] -.... -# ln -s s/etc etc -# ln -s s/root root -# ln -s s/home home -# ln -s ../s/usr-local usr/local -# ln -s ../s/usr-X11R6 usr/X11R6 -# ln -s s/tmp tmp -# ln -s s/var var -.... +Once the jail has been created, the next step will be to configure it, which is explained in <>. -. Next, stop the jails: -+ -[source,shell] -.... -# service jail stop -.... +[[creating-thin-jail-nullfs]] +=== Creating a Thin Jail using NullFS -. Unmount the original file systems as the read-write systems are attached to the read-only system ([.filename]#/s#): -+ -[source,shell] -.... -# umount /home/j/ns/s -# umount /home/j/ns -# umount /home/j/mail/s -# umount /home/j/mail -# umount /home/j/www/s -# umount /home/j/www -.... +// TODO -. Move the old read-only file system and replace it with the new one. This will serve as a backup and archive of the old read-only file system should something go wrong. The naming convention used here corresponds to when a new read-only file system has been created. Move the original FreeBSD Ports Collection over to the new file system to save some space and inodes: -+ -[source,shell] -.... -# cd /home/j -# mv mroot mroot.20060601 -# mv mroot2 mroot -# mv mroot.20060601/usr/ports mroot/usr -.... +[[creating-vnet-jail]] +=== Creating a VNET Jail -. At this point the new read-only template is ready, so the only remaining task is to remount the file systems and start the jails: -+ -[source,shell] -.... -# mount -a -# service jail start -.... +FreeBSD VNET Jails have its own distinct networking stack, including interfaces, IP addresses, routing tables, and firewall rules. -Use `jls` to check if the jails started correctly. -Run `mergemaster` in each jail to update the configuration files. -[[jails-ezjail]] -== Managing Jails with ezjail -Creating and managing multiple jails can quickly become tedious and error-prone. -Dirk Engling's ezjail automates and greatly simplifies many jail tasks. -A _basejail_ is created as a template. -Additional jails use man:mount_nullfs[8] to share many of the basejail directories without using additional disk space. -Each additional jail takes only a few megabytes of disk space before applications are installed. -Upgrading the copy of the userland in the basejail automatically upgrades all of the other jails. -Additional benefits and features are described in detail on the ezjail web site, https://erdgeist.org/arts/software/ezjail/[]. +[[creating-linux-jail]] +=== Creating a Linux Jail -[[jails-ezjail-install]] -=== Installing ezjail +FreeBSD can run Linux inside a jail using the crossref:linuxemu[linuxemu,Linux Binary Compatibility] and man:debootstrap[8]. +Jails do not have a kernel, they run on the host's kernel. +Therefore it is necessary to enable the Linux Binary Compatibility in the host system. -Installing ezjail consists of adding a loopback interface for use in jails, installing the port or package, and enabling the service. +To enable the Linux ABI at boot time, execute the following command: -[[jails-ezjail-install-procedure]] -[.procedure] -. To keep jail loopback traffic off the host's loopback network interface `lo0`, a second loopback interface is created by adding an entry to [.filename]#/etc/rc.conf#: -+ -[.programlisting] -.... -cloned_interfaces="lo1" -.... -+ -The second loopback interface `lo1` will be created when the system starts. -It can also be created manually without a restart: -+ [source,shell] .... -# service netif cloneup -Created clone interfaces: lo1. -.... -+ -Jails can be allowed to use aliases of this secondary loopback interface without interfering with the host. -+ -Inside a jail, access to the loopback address `127.0.0.1` is redirected to the first IP address assigned to the jail. -To make the jail loopback correspond with the new `lo1` interface, that interface must be specified first in the list of interfaces and IP addresses given when creating a new jail. -+ -Give each jail a unique loopback address in the `127.0.0.0/8` netblock. -. Install package:sysutils/ezjail[]: -+ -[source,shell] -.... -# cd /usr/ports/sysutils/ezjail -# make install clean +# sysrc linux_enable="YES" .... -. Enable ezjail by adding this line to [.filename]#/etc/rc.conf#: -+ -[.programlisting] -.... -ezjail_enable="YES" -.... +Once enabled, it can be started without rebooting executing the following command: -. The service will automatically start on system boot. It can be started immediately for the current session: -+ [source,shell] .... -# service ezjail start +# service linux start .... -[[jails-ezjail-initialsetup]] -=== Initial Setup - -With ezjail installed, the basejail directory structure can be created and populated. -This step is only needed once on the jail host computer. +The next step will be to create a jail as indicated above, for example in <>, but *without* performing the configuration. +The FreeBSD Linux jails require a specific configuration that will be detailed below. -In both of these examples, `-p` causes the ports tree to be retrieved with man:portsnap[8] into the basejail. -That single copy of the ports directory will be shared by all the jails. -Using a separate copy of the ports directory for jails isolates them from the host. -The ezjailFAQ explains in more detail: http://erdgeist.org/arts/software/ezjail/#FAQ[]. +With the machine created as explained above, execute the following command to perform some configurations to the machine and start it: -[[jails-ezjail-initialsetup-procedure]] -[.procedure] - -. To Populate the Jail with FreeBSD-RELEASE -+ -For a basejail based on the FreeBSD RELEASE matching that of the host computer, use `install`. -For example, on a host computer running FreeBSD 13-STABLE, the latest RELEASE version of FreeBSD 13 will be installed in the jail) -+ [source,shell] .... -# ezjail-admin install -p +# jail -cm \ + name=ubuntu \ + host.hostname="ubuntu.example.com" \ + path="/usr/local/jails/ubuntu" \ + interface="em0" \ + ip4.addr="192.168.1.150" \ + exec.start="/bin/sh /etc/rc" \ + exec.stop="/bin/sh /etc/rc.shutdown" \ + mount.devfs \ + devfs_ruleset=4 \ + allow.mount \ + allow.mount.devfs \ + allow.mount.fdescfs \ + allow.mount.procfs \ + allow.mount.linprocfs \ + allow.mount.linsysfs \ + allow.mount.tmpfs \ + enforce_statfs=1 .... -. To Populate the Jail with `installworld` -+ -The basejail can be installed from binaries created by `buildworld` on the host with `ezjail-admin update`. -+ -In this example, FreeBSD 10-STABLE has been built from source. -The jail directories are created. -Then `installworld` is executed, installing the host's [.filename]#/usr/obj# into the basejail. -+ +Then it will be necesary to access the jail to install package:sysutils/debootstrap[]. + +Execute the following command to access the FreeBSD Linux jail: + [source,shell] .... -# ezjail-admin update -i -p +# jexec -l ubuntu login -f root .... -+ -The host's [.filename]#/usr/src# is used by default. -A different source directory on the host can be specified with `-s` and a path, or set with `ezjail_sourcetree` in [.filename]#/usr/local/etc/ezjail.conf#. -[TIP] -==== -The basejail's ports tree is shared by other jails. -However, downloaded distfiles are stored in the jail that downloaded them. -By default, these files are stored in [.filename]#/var/ports/distfiles# within each jail. -[.filename]#/var/ports# inside each jail is also used as a work directory when building ports. -==== - -[TIP] -==== -The FTP protocol is used by default to download packages for the installation of the basejail. -Firewall or proxy configurations can prevent or interfere with FTP transfers. -The HTTP protocol works differently and avoids these problems. -It can be chosen by specifying a full URL for a particular download mirror in [.filename]#/usr/local/etc/ezjail.conf#: +Inside the jail execute the following commands to install package:sysutils/debootstrap[] and prepare the Ubuntu environment: -[.programlisting] +[source,shell] .... -ezjail_ftphost=http://ftp.FreeBSD.org +# pkg install debootstrap +# debootstrap jammy /compat/ubuntu .... -See the crossref:mirrors[mirrors,mirrors] section for a list of sites. -==== - -[[jails-ezjail-create]] -=== Creating and Starting a New Jail - -New jails are created with `ezjail-admin create`. -In these examples, the `lo1` loopback interface is used as described above. +When the process has finished and is displayed on the console `Base system installed successfully`, +it will be necessary to stop the jail from the host system by executing the following command: -[[jails-ezjail-create-steps]] -[.procedure] -.Procedure: Create and Start a New Jail -. Create the jail, specifying a name and the loopback and network interfaces to use, along with their IP addresses. In this example, the jail is named `dnsjail`. -+ [source,shell] .... -# ezjail-admin create dnsjail 'lo1|127.0.1.1,em0|192.168.1.50' +# jail -r ubuntu .... -+ -[TIP] -==== -Most network services run in jails without problems. -A few network services, most notably man:ping[8], use _raw network sockets_. -In jails, raw network sockets are disabled by default for security. -Services that require them will not work. -Occasionally, a jail genuinely needs raw sockets. -For example, network monitoring applications often use man:ping[8] to check the availability of other computers. -When raw network sockets are actually needed in a jail, they can be enabled by editing the ezjail configuration file for the individual jail, [.filename]#/usr/local/etc/ezjail/jailname#. -Modify the `parameters` entry: +Then add an entry in [.filename]#/etc/jail.conf# for the Linux jail: [.programlisting] .... -export jail_jailname_parameters="allow.raw_sockets=1" +exec.start = "/bin/sh /etc/rc"; +exec.stop = "/bin/sh /etc/rc.shutdown"; +exec.clean; +mount.devfs; + +ubuntu { + host.hostname = "${name}"; + path = "/usr/local/jails/${name}"; + + devfs_ruleset=4; + + exec.consolelog = "/var/log/jail_console_${name}.log"; + + ip4.addr = 192.168.1.151; + interface = em0; + + mount += "devfs $path/compat/ubuntu/dev devfs rw 0 0"; + mount += "tmpfs $path/compat/ubuntu/dev/shm tmpfs rw,size=1g,mode=1777 0 0"; + mount += "fdescfs $path/compat/ubuntu/dev/fd fdescfs rw,linrdlnk 0 0"; + mount += "linprocfs $path/compat/ubuntu/proc linprocfs rw 0 0"; + mount += "linsysfs $path/compat/ubuntu/sys linsysfs rw 0 0"; + mount += "/tmp $path/compat/ubuntu/tmp nullfs rw 0 0"; + mount += "/home $path/compat/ubuntu/home nullfs rw 0 0"; +} .... -Do not enable raw network sockets unless services in the jail actually require them. -==== - -. Start the jail: -+ -[source,shell] -.... -# ezjail-admin start dnsjail -.... +Then the jail can be started as usual with the following command: -. Use a console on the jail: -+ [source,shell] .... -# ezjail-admin console dnsjail +# service jail start ubuntu .... -The jail is operating and additional configuration can be completed. -Typical settings added at this point include: +And access to the Ubuntu environment using the following command: -[.procedure] -. Set the `root` Password -+ -Connect to the jail and set the `root` user's password: -+ [source,shell] .... -# ezjail-admin console dnsjail -# passwd -Changing local password for root -New Password: -Retype New Password: +# jexec ubuntu chroot /compat/ubuntu /bin/bash .... -. Time Zone Configuration -+ -The jail's time zone can be set with man:tzsetup[8]. -To avoid spurious error messages, the man:adjkerntz[8] entry in [.filename]#/etc/crontab# can be commented or removed. -This job attempts to update the computer's hardware clock with time zone changes, but jails are not allowed to access that hardware. -. DNS Servers -+ -Enter domain name server lines in [.filename]#/etc/resolv.conf# so DNS works in the jail. -. Edit [.filename]#/etc/hosts# -+ -Change the address and add the jail name to the `localhost` entries in [.filename]#/etc/hosts#. -. Configure [.filename]#/etc/rc.conf# -+ -Enter configuration settings in [.filename]#/etc/rc.conf#. -This is much like configuring a full computer. -The host name and IP address are not set here. -Those values are already provided by the jail configuration. +More information can be found in the chapter crossref:linuxemu[linuxemu,Linux Binary Compatibility]. -With the jail configured, the applications for which the jail was created can be installed. +[[configuring-jail]] +=== Configuring a Jail -[TIP] -==== -Some ports must be built with special options to be used in a jail. -For example, both of the network monitoring plugin packages package:net-mgmt/nagios-plugins[] and package:net-mgmt/monitoring-plugins[] have a `JAIL` option which must be enabled for them to work correctly inside a jail. -==== +When the userland has been downloaded, extracted, and jailed using the various methods outlined above, the next step is to configure the jail. +More information about the jail configuration file can be obtained at <> -[[jails-ezjail-update]] -=== Updating Jails +It will be necessary to add an entry to the configuration file [.filename]#/etc/jail.conf# with the data of the jail. -[[jails-ezjail-update-os]] -==== Updating the Operating System +An example would be the following: -Because the basejail's copy of the userland is shared by the other jails, updating the basejail automatically updates all of the other jails. -Either source or binary updates can be used. +[.programlisting] +.... +exec.start = "/bin/sh /etc/rc"; +exec.stop = "/bin/sh /etc/rc.shutdown"; +exec.clean; +mount.devfs; -To build the world from source on the host, then install it in the basejail, use: +jailname { + host.hostname = "${name}"; + path = "/usr/local/jails/${name}"; + + exec.consolelog = "/var/log/jail_console_${name}.log"; -[source,shell] -.... -# ezjail-admin update -b + ip4.addr = 192.168.1.151; + interface = em0; +} .... -If the world has already been compiled on the host, install it in the basejail with: +And then execute the following command to start the jail: [source,shell] .... -# ezjail-admin update -i +# service jail start jailname .... -Binary updates use man:freebsd-update[8]. -These updates have the same limitations as if man:freebsd-update[8] were being run directly. -The most important one is that only -RELEASE versions of FreeBSD are available with this method. +More information on how to manage jails can be found in the section <<>>. // TODO: link to the jails management section -Update the basejail to the latest patched release of the version of FreeBSD on the host. -For example, updating from RELEASE-p1 to RELEASE-p2. +[[jail-management]] +== Jail Management -[source,shell] -.... -# ezjail-admin update -u -.... +With the jail already created on the host, there are different procedures that can be done, such as starting it, rebooting it, deleting it, installing software, etc. +In this section the different actions that can be done with the jails created in the host will be described. + +[[list-running-jails]] +=== List Running Jails -To upgrade the basejail to a new version, first upgrade the host system as described in crossref:cutting-edge[freebsdupdate-upgrade,“Performing Major and Minor Version Upgrades”]. -Once the host has been upgraded and rebooted, the basejail can then be upgraded. -man:freebsd-update[8] has no way of determining which version is currently installed in the basejail, so the original version must be specified. -Use man:file[1] to determine the original version in the basejail: +To list the different jails that are running on the host system, the command man:jls[8] can be used: [source,shell] .... -# file /usr/jails/basejail/bin/sh -/usr/jails/basejail/bin/sh: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.0, FreeBSD-style, stripped +# jls .... -Now use this information to perform the upgrade from `13.0-RELEASE` to the current version of the host system: +The output should be similar to the following: -[source,shell] .... -# ezjail-admin update -U -s 13.0-RELEASE + JID IP Address Hostname Path + 1 192.168.250.70 classic /usr/local/jails/classic .... -After updating the basejail, man:mergemaster[8] must be run to update each jail's configuration files. - -How to use man:mergemaster[8] depends on the purpose and trustworthiness of a jail. -If a jail's services or users are not trusted, then man:mergemaster[8] should only be run from within that jail: +man:jls[8] supports the `--libxo` argument, which through the man:libxo[3] library allows other types of formats to be displayed, such as `JSON`, `HTML`, etc. -[[jails-ezjail-update-mergemaster-untrusted]] -.man:mergemaster[8] on Untrusted Jail -[example] -==== -Delete the link from the jail's [.filename]#/usr/src# into the basejail and create a new [.filename]#/usr/src# in the jail as a mountpoint. -Mount the host computer's [.filename]#/usr/src# read-only on the jail's new [.filename]#/usr/src# mountpoint: +For example, execute the following command to get the `JSON` output: [source,shell] .... -# rm /usr/jails/jailname/usr/src -# mkdir /usr/jails/jailname/usr/src -# mount -t nullfs -o ro /usr/src /usr/jails/jailname/usr/src +# jls --libxo=json .... -Get a console in the jail: +The output should be similar to the following: -[source,shell] .... -# ezjail-admin console jailname +{"__version": "2", "jail-information": {"jail": [{"jid":1,"ipv4":"192.168.250.70","hostname":"classic","path":"/usr/local/jails/classic"}]}} .... -Inside the jail, run `mergemaster`. -Then exit the jail console: +[[start-jail]] +=== Start, Restart and Stop a Jail -[source,shell] -.... -# cd /usr/src -# mergemaster -U -# exit -.... +man:service[8] will be used to start, reboot, or stop a jail on the host. -Finally, unmount the jail's [.filename]#/usr/src#: +For example, to start a jalil, run the following command: [source,shell] .... -# umount /usr/jails/jailname/usr/src +# service jail start jailname .... -==== +Change the `start` argument to `restart` or `stop` to perform other actions on the jail. -[[jails-ezjail-update-mergemaster-trusted]] -.man:mergemaster[8] on Trusted Jail -[example] -==== +[[destroy-jail]] +=== Destroy a Jail -If the users and services in a jail are trusted, man:mergemaster[8] can be run from the host: +Destroying a jail is not as simple as stopping the jail using man:service[8], removing the jail directory and the [.filename]#/etc/jail.conf# entry. -[source,shell] -.... -# mergemaster -U -D /usr/jails/jailname -.... - -==== +FreeBSD takes system security very seriously. +For this reason there are certain files that not even the root user can delete. +This functionality is known as File Flags. -[TIP] -==== -After a major version update it is recommended by package:sysutils/ezjail[] to make sure your `pkg` is of the correct version. -Therefore enter: +The first step will be to stop the desired jail executing the following command: [source,shell] .... -# pkg-static upgrade -f pkg +# service jail stop jailname .... -to upgrade or downgrade to the appropriate version. -==== - -[[jails-ezjail-update-ports]] -==== Updating Ports - -The ports tree in the basejail is shared by the other jails. -Updating that copy of the ports tree gives the other jails the updated version also. - -The basejail ports tree is updated with man:portsnap[8]: +The second step will be to remove these flags with man:chflags[1] executing the following command, in which classic is the name of the jail to remove: [source,shell] .... -# ezjail-admin update -P +# chflags -R 0 /usr/local/jails/classic .... -[[jails-ezjail-control]] -=== Controlling Jails - -[[jails-ezjail-control-stop-start]] -==== Stopping and Starting Jails - -ezjail automatically starts jails when the computer is started. -Jails can be manually stopped and restarted with `stop` and `start`: +The third step will be to delete the directory where the jail was: [source,shell] .... -# ezjail-admin stop sambajail -Stopping jails: sambajail. +# rm -rf /usr/local/jails/classic .... -By default, jails are started automatically when the host computer starts. -Autostarting can be disabled with `config`: +Finally, it will be necessary to remove the jail entry in the file [.filename]#/etc/jail.conf#. -[source,shell] -.... -# ezjail-admin config -r norun seldomjail -.... +[[handle-packages-jail]] +=== Handle Packages in a Jail -This takes effect the next time the host computer is started. -A jail that is already running will not be stopped. +The man:pkg[8] tool supports the `-j` argument in order to handle packages installed inside the jail. -Enabling autostart is very similar: +For example, to install package:nginx-lite[] in the jail, the next command will be executed *from the host*: [source,shell] .... -# ezjail-admin config -r run oftenjail +# pkg -j classic install nginx-lite .... -[[jails-ezjail-control-backup]] -==== Archiving and Restoring Jails +For more information on working with packages on FreeBSD see the chapter // TODO: Link to the chapter -Use `archive` to create a [.filename]#.tar.gz# archive of a jail. -The file name is composed from the name of the jail and the current date. -Archive files are written to the archive directory, [.filename]#/usr/jails/ezjail_archives#. -A different archive directory can be chosen by setting `ezjail_archivedir` in the configuration file. +[[access-jail]] +=== Access a Jail -The archive file can be copied elsewhere as a backup, or an existing jail can be restored from it with `restore`. -A new jail can be created from the archive, providing a convenient way to clone existing jails. +While it has been stated above that it is best to manage jails from the host system, a jail can be accessed with man:jexec[8]. -Stop and archive a jail named `wwwserver`: +The jail can be accessed by running man:jexec[8] from the host: [source,shell] .... -# ezjail-admin stop wwwserver -Stopping jails: wwwserver. -# ezjail-admin archive wwwserver -# ls /usr/jails/ezjail-archives/ -wwwserver-201407271153.13.tar.gz +# jexec -l jailname login -f root .... -Create a new jail named `wwwserver-clone` from the archive created in the previous step. -Use the [.filename]#em1# interface and assign a new IP address to avoid conflict with the original: +When gaining access to the jail, the message configured in man:motd[5] will be displayed. + +[[execute-commands-jail]] +=== Execute commands in a Jail + +To execute a command from the host system in a jail the man:jexec[8] can be used. + +For example, to stop a service that is running inside a jail, the command will be executed: [source,shell] .... -# ezjail-admin create -a /usr/jails/ezjail_archives/wwwserver-201407271153.13.tar.gz wwwserver-clone 'lo1|127.0.3.1,em1|192.168.1.51' +# jexec -l jailname service stop nginx .... -[[jails-ezjail-example-bind]] -=== Full Example: BIND in a Jail +[[jail-upgrading]] +== Jail Upgrading -Putting the BINDDNS server in a jail improves security by isolating it. -This example creates a simple caching-only name server. +// TODO: brief introduction -* The jail will be called `dns1`. -* The jail will use IP address `192.168.1.240` on the host's `re0` interface. -* The upstream ISP's DNS servers are at `10.0.0.62` and `10.0.0.61`. -* The basejail has already been created and a ports tree installed as shown in <>. - -[[jails-ezjail-example-bind-steps]] -.Running BIND in a Jail -[example] -==== +[[jails-updating]] +=== Updating or Upgrading a Classic Jail -Create a cloned loopback interface by adding a line to [.filename]#/etc/rc.conf#: +Jails *must be kept up to date from the host* operating system as attempting to patch userland from within the jail may likely fail as the default behavior in FreeBSD is to disallow the use of man:chflags[1] in a jail which prevents the replacement of some files. -[.programlisting] -.... -cloned_interfaces="lo1" -.... - -Immediately create the new loopback interface: +To update the jail to the latest patch release of the version of FreeBSD it is already running, then execute the following commands on the host: [source,shell] .... -# service netif cloneup -Created clone interfaces: lo1. +# freebsd-update -j classic fetch install +# service jail restart classic .... -Create the jail: +To upgrade the jail to a new major or minor version, first upgrade the host system as described in crossref:cutting-edge[freebsdupdate-upgrade,"Performing Major and Minor Version Upgrades"]. +Once the host has been upgraded and rebooted, the jail can then be upgraded. -[source,shell] -.... -# ezjail-admin create dns1 'lo1|127.0.2.1,re0|192.168.1.240' -.... +[TIP] +==== +In case of upgrade from one version to another, it is easier to create a new jail than to upgrade completely. +==== -Start the jail, connect to a console running on it, and perform some basic configuration: +For example to upgrade from 13.1-RELEASE to 13.2-RELEASE, execute the following commands on the host: [source,shell] .... -# ezjail-admin start dns1 -# ezjail-admin console dns1 -# passwd -Changing local password for root -New Password: -Retype New Password: -# tzsetup -# sed -i .bak -e '/adjkerntz/ s/^/#/' /etc/crontab -# sed -i .bak -e 's/127.0.0.1/127.0.2.1/g; s/localhost.my.domain/dns1.my.domain dns1/' /etc/hosts +# freebsd-update -j classic -r 13.2-RELEASE upgrade +# freebsd-update -j classic install +# service jail restart classic +# freebsd-update -j classic install .... -Temporarily set the upstream DNS servers in [.filename]#/etc/resolv.conf# so ports can be downloaded: - -[.programlisting] -.... -nameserver 10.0.0.62 -nameserver 10.0.0.61 -.... +Then, if it was a major version upgrade, reinstall all installed packages and restart the jail again. +This is required because the ABI version changes when upgrading between major versions of FreeBSD. -Still using the jail console, install package:dns/bind99[]. +From the host: [source,shell] .... -# make -C /usr/ports/dns/bind99 install clean +# pkg -j myjail upgrade -f +# service jail restart myjail .... -Configure the name server by editing [.filename]#/usr/local/etc/namedb/named.conf#. +[[upgrading-thin-jail]] +=== Upgrading a Thin Jail -Create an Access Control List (ACL) of addresses and networks that are permitted to send DNS queries to this name server. -This section is added just before the `options` section already in the file: +// TODO -[.programlisting] -.... -... -// or cause huge amounts of useless Internet traffic. +[[jail-filesystems]] +== Jail Filesystems -acl "trusted" { - 192.168.1.0/24; - localhost; - localnets; -}; +// TODO -options { -... -.... +[[jail-resources-limits]] +== Jail Resources Limits -Use the jail IP address in the `listen-on` setting to accept DNS queries from other computers on the network: +Controlling the resources that a jail uses from the host system is a task to be taken into account by the system administrator. -[.programlisting] -.... - listen-on { 192.168.1.240; }; -.... +man:rctl[8] allows you to manage the resources that a jail can use from the host system. -A simple caching-only DNS name server is created by changing the `forwarders` section. -The original file contains: +[TIP] +==== +The `kern.racct.enable` tunable must be enabled at [.filename]#/boot/loader.conf#. +==== -[.programlisting] -.... -/* - forwarders { - 127.0.0.1; - }; -*/ -.... +The syntax to limit the resources of a jail would be `rctl -a jail::resource:action=amount/percentage`. -Uncomment the section by removing the `/\*` and `*/` lines. -Enter the IP addresses of the upstream DNS servers. -Immediately after the `forwarders` section, add references to the `trusted` ACL defined earlier: +For example, to limit the maximum RAM that a jail can access run the following command: -[.programlisting] +[source,shell] .... - forwarders { - 10.0.0.62; - 10.0.0.61; - }; - - allow-query { any; }; - allow-recursion { trusted; }; - allow-query-cache { trusted; }; +# rctl -a jail:classic:memoryuse:deny=2G .... -Enable the service in [.filename]#/etc/rc.conf#: +To make the limitation persistent across reboots of the host system, it will be necessary to add the rule to the [.filename]#/etc/rctl.conf# file as follows: [.programlisting] .... -named_enable="YES" +jail:classic:memoryuse:deny=2G/jail .... -Start and test the name server: +More information on resource limits can be found in the security chapter in the crossref:security[security-resourcelimits,"Resource Limits section"]. -[source,shell] -.... -# service named start -wrote key file "/usr/local/etc/namedb/rndc.key" -Starting named. -# /usr/local/bin/dig @192.168.1.240 freebsd.org -.... +[[jail-managers-and-container]] +== Jail Managers and Containers -A response that includes +As previously explained, the different FreeBSD Jails can be created and configured manually. +But FreeBSD has some third-party utilities to make configuration and administration easier. -[source,shell] -.... -;; Got answer; -.... +Below is an incomplete list of the different FreeBSD Jail managers: -shows that the new DNS server is working. -A long delay followed by a response including +.Jail Managers +[options="header", cols="1,1,1,1"] +|=== +| Name | License | Package | Documentation -[source,shell] -.... -;; connection timed out; no servers could be reached -.... +| BastilleBSD +| BSD-3 +| package:sysutils/bastille[] +| link:https://bastille.readthedocs.io/en/latest/[Documentation] -shows a problem. -Check the configuration settings and make sure any local firewalls allow the new DNS access to the upstream DNS servers. +| pot +| BSD-3 +| package:sysutils/pot[] +| link:https://pot.pizzamig.dev/[Documentation] -The new DNS server can use itself for local name resolution, just like other local computers. -Set the address of the DNS server in the client computer's [.filename]#/etc/resolv.conf#: +| cbsd +| BSD-2 +| package:sysutils/cbsd[] +| link:https://www.bsdstore.ru/en/docs.html[Documentation] -[.programlisting] -.... -nameserver 192.168.1.240 -.... +| AppJail +| BSD-3 +| package:sysutils/appjail[], for devel package:sysutils/appjail-devel[] +| link:https://github.com/DtxdF/AppJail#getting-started[Documentation] -A local DHCP server can be configured to provide this address for a local DNS server, providing automatic configuration on DHCP clients. -==== +| iocage +| BSD-2 +| package:sysutils/iocage[] +| link:https://iocage.readthedocs.io/en/latest/[Documentation] + +| ezjail +| link:https://erdgeist.org/beerware.html[Beer Ware] +| package:sysutils/ezjail[] +| link:https://erdgeist.org/arts/software/ezjail/[Documentation] + +|===