diff --git a/en_US.ISO8859-1/books/handbook/jails/chapter.sgml b/en_US.ISO8859-1/books/handbook/jails/chapter.sgml index de1c723620..77f40321cc 100644 --- a/en_US.ISO8859-1/books/handbook/jails/chapter.sgml +++ b/en_US.ISO8859-1/books/handbook/jails/chapter.sgml @@ -1,487 +1,942 @@ Matteo Riondato Contributed by Jails jails Synopsis This chapter will provide an explanation of what &os; jails are and how to use them. Jails, sometimes referred to as an enhanced replacement of chroot environments, are a very powerful tool for system administrators, but their basic usage can also be useful for advanced users. After reading this chapter, you will know: What a jail is, and what purpose it may serve in &os; installations. How to build, start, and stop a jail. The basics of jail administration, both from inside and outside the jail. Other sources of useful information about jails are: The &man.jail.8; manual page. This is the full reference of the jail utility — the administrative tool which can be used in &os; to start, stop, and control &os; jails. The mailing lists and their archives. The archives of the &a.questions; and other mailing lists hosted by the &a.mailman.lists; already contain a wealth of material for jails. It should always be engaging to search the archives, or post a new question to the &a.questions.name; mailing list. Terms Related to Jails To facilitate better understanding of parts of the &os; system related to jails, their internals and the way they interact with the rest of &os;, the following terms are used further in this chapter: &man.chroot.2; (command) A system call of &os;, which changes the root directory of a process and all its descendants. &man.chroot.2; (environment) The environment of processes running in a chroot. This includes resources such as the part of the file system which is visible, user and group IDs which are available, network interfaces and other IPC mechanisms, etc. &man.jail.8; (command) The system administration utility which allows launching of processes within a jail environment. host (system, process, user, etc.) The controlling system of a jail environment. The host system has access to all the hardware resources available, and can control processes both outside of and inside a jail environment. One of the important differences of the host system from a jail is that the limitations which apply to superuser processes inside a jail are not enforced for processes of the host system. hosted (system, process, user, etc.) A process, user or other entity, whose access to resources is restricted by an &os; jail. Introduction Since system administration is a difficult and perplexing task, many powerful tools were developed to make life easier for the administrator. These tools mostly provide enhancements of some sort to the way systems are installed, configured and maintained. Part of the tasks which an administrator is expected to do is to properly configure the security of a system, so that it can continue serving its real purpose, without allowing security violations. One of the tools which can be used to enhance the security of a &os; system are jails. Jails were introduced in &os; 4.X by &a.phk;, but were greatly improved in &os; 5.X to make them a powerful and flexible subsystem. Their development still goes on, enhancing their usefulness, performance, reliability, and security. What is a Jail BSD-like operating systems have had &man.chroot.2; since the time of 4.2BSD. The &man.chroot.8; utility can be used to change the root directory of a set of processes, creating a safe environment, separate from the rest of the system. Processes created in the chrooted environment can not access files or resources outside of it. For that reason, compromising a service running in a chrooted environment should not allow the attacker to compromise the entire system. The &man.chroot.8; utility is good for easy tasks, which do not require a lot of flexibility or complex and advanced features. Since the inception of the chroot concept, however, many ways have been found to escape from a chrooted environment and, although they have been fixed in modern versions of the &os; kernel, it was clear that &man.chroot.2; was not the ideal solution for securing services. A new subsystem had to be implemented. This is one of the main reasons why jails were developed. Jails improve on the concept of the traditional &man.chroot.2; environment, in several ways. In a traditional &man.chroot.2; environment, processes are only limited in the part of the file system they can access. The rest of the system resources (like the set of system users, the running processes, or the networking subsystem) are shared by the chrooted processes and the processes of the host system. Jails expand this model by virtualizing not only access to the file system, but also the set of users, the networking subsystem of the &os; kernel and a few other things. A more complete set of fine-grained controls available for tuning the access of a jailed environment is described in . A jail is characterized by four elements: A directory subtree — the starting point from which a jail is entered. Once inside the jail, a process is not permitted to escape outside of this subtree. Traditional security issues which plagued the original &man.chroot.2; design will not affect &os; jails. A hostname — the hostname which will be used within the jail. Jails are mainly used for hosting network services, therefore having a descriptive hostname for each jail can really help the system administrator. An IP address — this will be assigned to the jail and cannot be changed in any way during the jail's life span. The IP address of a jail is usually an alias address for an existing network interface, but this is not strictly necessary. A command — the path name of an executable to run inside the jail. This is relative to the root directory of the jail environment, and may vary a lot, depending on the type of the specific jail environment. Apart from these, jails can have their own set of users and their own root user. Naturally, the powers of the root user are limited within the jail environment and, from the point of view of the host system, the jail root user is not an omnipotent user. In addition, the root user of a jail is not allowed to perform critical operations to the system outside of the associated &man.jail.8; environment. More information about capabilities and restrictions of the root user will be discussed in below. Creating and Controlling Jails Some administrators divide jails into the following two types: complete jails, which resemble a real &os; system, and service jails, dedicated to one application or service, possibly running with privileges. This is only a conceptual division and the process of building a jail is not affected by it. The &man.jail.8; manual page is quite clear about the procedure for building a jail: &prompt.root; setenv D /here/is/the/jail &prompt.root; mkdir -p $D &prompt.root; cd /usr/src &prompt.root; make world DESTDIR=$D &prompt.root; cd etc/ This step is not required on &os; 6.0 and later. &prompt.root; make distribution DESTDIR=$D &prompt.root; mount_devfs devfs $D/dev Selecting a location for a jail is the best starting point. This is where the jail will physically reside within the file system of the jail's host. A good choice can be /usr/jail/jailname, where jailname is the hostname identifying the jail. The /usr/ file system usually has enough space for the jail file system, which for complete jails is, essentially, a replication of every file present in a default installation of the &os; base system. This command will populate the directory subtree chosen as jail's physical location on the file system with the necessary binaries, libraries, manual pages and so on. Everything is done in the typical &os; style — first everything is built/compiled, then installed to the destination path. The distribution target for make installs every needed configuration file. In simple words, it installs every installable file of /usr/src/etc/ to the /etc directory of the jail environment: $D/etc/. Mounting the &man.devfs.8; file system inside a jail is not required. On the other hand, any, or almost any application requires access to at least one device, depending on the purpose of the given application. It is very important to control access to devices from inside a jail, as improper settings could permit an attacker to do nasty things in the jail. Control over &man.devfs.8; is managed through rulesets which are described in the &man.devfs.8; and &man.devfs.conf.5; manual pages. Once a jail is installed, it can be started by using the &man.jail.8; utility. The &man.jail.8; utility takes four mandatory arguments which are described in the . Other arguments may be specified too, e.g., to run the jailed process with the credentials of a specific user. The argument depends on the type of the jail; for a virtual system, /etc/rc is a good choice, since it will replicate the startup sequence of a real &os; system. For a service jail, it depends on the service or application that will run within the jail. Jails are often started at boot time and the &os; rc mechanism provides an easy way to do this. A list of the jails which are enabled to start at boot time should be added to the &man.rc.conf.5; file: jail_enable="YES" # Set to NO to disable starting of any jails jail_list="www" # Space separated list of names of jails For each jail listed in jail_list, a group of &man.rc.conf.5; settings, which describe the particular jail, should be added: jail_www_rootdir="/usr/jail/www" # jail's root directory jail_www_hostname="www.example.org" # jail's hostname jail_www_ip="192.168.0.10" # jail's IP address jail_www_devfs_enable="YES" # mount devfs in the jail jail_www_devfs_ruleset="www_ruleset" # devfs ruleset to apply to jail The default startup of jails configured in &man.rc.conf.5;, will run the /etc/rc script of the jail, which assumes the jail is a complete virtual system. For service jails, the default startup command of the jail should be changed, by setting the jail_jailname_exec_start option appropriately. For a full list of available options, please see the &man.rc.conf.5; manual page. The /etc/rc.d/jail script can be used to start or stop a jail by hand, if an entry for it exists in rc.conf: &prompt.root; /etc/rc.d/jail start www &prompt.root; /etc/rc.d/jail stop www A clean way to shut down a &man.jail.8; is not available at the moment. This is because commands normally used to accomplish a clean system shutdown cannot be used inside a jail. The best way to shut down a jail is to run the following command from within the jail itself or using the &man.jexec.8; utility from outside the jail: &prompt.root; sh /etc/rc.shutdown More information about this can be found in the &man.jail.8; manual page. Fine Tuning and Administration There are several options which can be set for any jail, and various ways of combining a host &os; system with jails, to produce higher level applications. This section presents: Some of the options available for tuning the behavior and security restrictions implemented by a jail installation. Some of the high-level applications for jail management, which are available through the &os; Ports Collection, and can be used to implement overall jail-based solutions. System tools for jail tuning in &os; Fine tuning of a jail's configuration is mostly done by setting &man.sysctl.8; variables. A special subtree of sysctl exists as a basis for organizing all the relevant options: the security.jail.* hierarchy of &os; kernel options. Here is a list of the main jail-related sysctls, complete with their default value. Names should be self-explanatory, but for more information about them, please refer to the &man.jail.8; and &man.sysctl.8; manual pages. security.jail.set_hostname_allowed: 1 security.jail.socket_unixiproute_only: 1 security.jail.sysvipc_allowed: 0 security.jail.enforce_statfs: 2 security.jail.allow_raw_sockets: 0 security.jail.chflags_allowed: 0 security.jail.jailed: 0 These variables can be used by the system administrator of the host system to add or remove some of the limitations imposed by default on the root user. Note that there are some limitations which cannot be removed. The root user is not allowed to mount or unmount file systems from within a &man.jail.8;. The root inside a jail may not load or unload &man.devfs.8; rulesets, set firewall rules, or do many other administrative tasks which require modifications of in-kernel data, such as setting the securelevel of the kernel. The base system of &os; contains a basic set of tools for viewing information about the active jails, and attaching to a jail to run administrative commands. The &man.jls.8; and &man.jexec.8; commands are part of the base &os; system, and can be used to perform the following simple tasks: Print a list of active jails and their corresponding jail identifier (JID), IP address, hostname and path. Attach to a running jail, from its host system, and run a command inside the jail or perform administrative tasks inside the jail itself. This is especially useful when the root user wants to cleanly shut down a jail. The &man.jexec.8; utility can also be used to start a shell in a jail to do administration in it; for example: &prompt.root; jexec 1 tcsh High-level administrative tools in &os; Ports Collection Among the many third-party utilities for jail administration, one of the most complete and useful is sysutils/jailutils. It is a set of small applications that contribute to &man.jail.8; management. Please refer to its web page for more information. + + + Application of Jails + + + + + + Daniel + Gerzo + Contributed by + + + + + + Service Jails + + This section is based upon an idea originally presented by + &a.simon; at , and an + updated article written by Ken Tom + locals@gmail.com. This section illustrates how + to set up a &os; system that adds an additional layer of + security, using the &man.jail.8; feature. It is also assumed + that the given system is at least running RELENG_6_0 and the + information provided earlier in this chapter has been well + understood. + + + Design + + One of the major problems with jails is the management of + their upgrade process. This tends to be a problem because + every jail has to be rebuilt from scratch whenever it is + updated. This is usually not a problem for a single jail, + since the update process is fairly simple, but can be quite + time consuming and tedious if a lot of jails are + created. + + + This setup requires advanced experience with &os; and + usage of its features. If the presented steps below look + too complicated, it is advised to take a look at a simpler + system such as sysutils/ezjail, which provides + an easier method of administering &os; jails and is not as + sophisticated as this setup. + + + This idea has been presented to resolve such issues by + sharing as much as is possible between jails, in a safe way + — using read-only &man.mount.nullfs.8; mounts, so that + updating will be be simpler, and putting single services into + individual jails will become more attractive. Additionally, + it provides a simple way to add or remove jails as well as a + way to upgrade them. + + + Examples of services in this context are: an + HTTP server, a DNS + server, a SMTP server, and so forth. + + + The goals of the setup described in this section + are: + + + + Create a simple and easy to understand jail structure. + This implies not having to run a full + installworld on each and every jail. + + + Make it easy to add new jails or remove existing + ones. + + + Make it easy to update or upgrade existing + jails. + + + Make it possible to run a customized &os; + branch. + + + Be paranoid about security, reducing as much as + possible the possibility of compromise. + + + Save space and inodes, as much as possible. + + + + As it has been already mentioned, this design relies + heavily on having a single master template which is read-only + (known as nullfs) mounted into each + jail and one read-write device per jail. A device can be a + separate physical disc, a partition, or a vnode backed + &man.md.4; device. In this example, we will use read-write + nullfs mounts. + + The file system layout is described in the following + list: + + + + Each jail will be mounted under the /home/j directory. + + + /home/j/mroot is + the template for each jail and the read-only partition for + all of the jails. + + + A blank directory will be created for each jail under + the /home/j + directory. + + + Each jail will have a /s directory, that will be + linked to the read-write portion of the system. + + + Each jail shall have its own read-write system that is + based upon /home/j/skel. + + + Each jailspace (read-write portion of each jail) shall + be created in /home/js. + + + + + This assumes that the jails are based under the + /home partition. This + can, of course, be changed to anything else, but this change + will have to be reflected in each of the examples + below. + + + + + + Creating the Template + + This section will describe the steps needed to create the + master template that will be the read-only portion for the + jails to use. + + It is always a good idea to update the &os; system to the + latest -RELEASE branch. Check the corresponding Handbook + Chapter + to accomplish this task. In the case the update is not + feasible, the buildworld will be required in order to be able + to proceed. Additionally, the sysutils/cpdup package will be + required. We will use the &man.portsnap.8; utility to + download the &os; Ports Collection. The Handbook Portsnap Chapter + is always good reading for newcomers. + + + + First, create a directory structure for the read-only + file system which will contain the &os; binaries for our + jails, then change directory to the &os; source tree and + install the read-only file system to the jail + template: + + &prompt.root; mkdir /home/j /home/j/mroot +&prompt.root; cd /usr/src +&prompt.root; make installworld DESTDIR=/home/j/mroot + + + Next, prepare a &os; Ports Collection for the jails as + well as a &os; source tree, which is required for + mergemaster: + + &prompt.root; cd /home/j/mroot +&prompt.root; mkdir usr/ports +&prompt.root; portsnap -p /home/j/mroot/usr/ports fetch extract +&prompt.root; cpdup /usr/src /home/j/mroot/usr/src + + + Create a skeleton for the read-write portion of the + system: + + &prompt.root; mkdir /home/j/skel /home/j/skel/home /home/j/skel/usr-X11R6 /home/j/skel/distfiles +&prompt.root; mv etc /home/j/skel +&prompt.root; mv usr/local /home/j/skel/usr-local +&prompt.root; mv tmp /home/j/skel +&prompt.root; mv var /home/j/skel +&prompt.root; mv root /home/j/skel + + + Use mergemaster to install + missing configuration files. Then get rid of the extra + directories that mergemaster + creates: + + &prompt.root; mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i +&prompt.root; cd /home/j/skel +&prompt.root; rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev + + + Now, symlink the read-write file system to the + read-only file system. Please make sure that the symlinks + are created in the correct s/ locations. Real + directories or the creation of directories in the wrong + locations will cause the installation to fail. + + &prompt.root; cd /home/j/mroot +&prompt.root; mkdir s +&prompt.root; ln -s s/etc etc +&prompt.root; ln -s s/home home +&prompt.root; ln -s s/root root +&prompt.root; ln -s ../s/usr-local usr/local +&prompt.root; ln -s ../s/usr-X11R6 usr/X11R6 +&prompt.root; ln -s ../../s/distfiles usr/ports/distfiles +&prompt.root; ln -s s/tmp tmp +&prompt.root; ln -s s/var var + + + As a last step, create a generic + /home/j/skel/etc/make.conf with its + contents as shown below: + + WRKDIRPREFIX?= /s/portbuild + + + Having WRKDIRPREFIX set up this + way will make it possible to compile &os; ports inside + each jail. Remember that the ports directory is part of + the read-only system. The custom path for + WRKDIRPREFIX allows builds to be done + in the read-write portion of every jail. + + + + + + Creating Jails + + Now that we have a complete &os; jail template, we can + setup and configure the jails in + /etc/rc.conf. This example demonstrates + the creation of 3 jails: NS, + MAIL and WWW. + + + + Put the following lines into the + /etc/fstab file, so that the + read-only template for the jails and the read-write space + will be available in the respective jails: + + /home/j/mroot /home/j/ns nullfs ro 0 0 +/home/j/mroot /home/j/mail nullfs ro 0 0 +/home/j/mroot /home/j/www nullfs ro 0 0 +/home/js/ns /home/j/ns/s nullfs rw 0 0 +/home/js/mail /home/j/mail/s nullfs rw 0 0 +/home/js/www /home/j/www/s nullfs rw 0 0 + + + Partitions marked with a 0 pass number are not + checked by &man.fsck.8; during boot, and partitions + marked with a 0 dump number are not backed up by + &man.dump.8;. We do not want + fsck to check + nullfs mounts or + dump to back up the read-only + nullfs mounts of the jails. This is why they are marked + with 0 0 in the last two columns of + each fstab entry above. + + + + Configure the jails in + /etc/rc.conf: + + jail_enable="YES" +jail_set_hostname_allow="NO" +jail_list="ns mail www" +jail_ns_hostname="ns.example.org" +jail_ns_ip="192.168.3.17" +jail_ns_rootdir="/home/j/ns" +jail_ns_devfs_enable="YES" +jail_mail_hostname="mail.example.org" +jail_mail_ip="192.168.3.18" +jail_mail_rootdir="/home/j/mail" +jail_mail_devfs_enable="YES" +jail_www_hostname="www.example.org" +jail_www_ip="62.123.43.14" +jail_www_rootdir="/home/j/www" +jail_www_devfs_enable="YES" + + + Create the required mount points for the read-only + file system of each jail: + + &prompt.root; mkdir /home/j/ns /home/j/mail /home/j/www + + + Install the read-write template into each jail. Note + the use of sysutils/cpdup, which helps to + ensure that a correct copy is done of each + directory: + + + &prompt.root; mkdir /home/js +&prompt.root; cpdup /home/j/skel /home/js/ns +&prompt.root; cpdup /home/j/skel /home/js/mail +&prompt.root; cpdup /home/j/skel /home/js/www + + + In this phase, the jails are built and prepared to + run. First, mount the required file systems for each + jail, and then start them using the + /etc/rc.d/jail script: + + &prompt.root; mount -a +&prompt.root; /etc/rc.d/jail start + + + + The jails should be running now. To check if they have + started correctly, use the &man.jls.8; command. Its output + should be similar to the following: + + &prompt.root; jls + JID IP Address Hostname Path + 3 192.168.3.17 ns.example.org /home/j/ns + 2 192.168.3.18 mail.example.org /home/j/mail + 1 62.123.43.14 www.example.org /home/j/www + + At this point, it should be possible to log onto each + jail, add new users or configure daemons. The + JID column indicates the jail + indentification number of each running jail. Use the + following command in order to perform administrative tasks in + the jail whose JID is 3: + + &prompt.root; jexec 3 tcsh + + + + Upgrading + + In time, there will be a need to upgrade the system to a + newer version of &os;, either because of a security issue, or + because new features have been implemented which are useful + for the existing jails. The design of this setup provides an + easy way to upgrade existing jails. Additionally, it + minimizes their downtime, as the jails will be brought down + only in the very last minute. Also, it provides a way to roll + back to the older versions should any problems occur. + + + + The first step is to upgrade the host system in the + usual manner. Then create a new temporary read-only + template in /home/j/mroot2. + + &prompt.root; mkdir /home/j/mroot2 +&prompt.root; cd /usr/src +&prompt.root; make installworld DESTDIR=/home/j/mroot2 +&prompt.root; cd /home/j/mroot2 +&prompt.root; cpdup /usr/src usr/src +&prompt.root; mkdir s + + The installworld run creates + a few unnecessary directories, which should be + removed: + + &prompt.root; chflags -R 0 var +&prompt.root; rm -R etc var root usr/local tmp + + + Recreate the read-write symlinks for the master file + system: + + &prompt.root; ln -s s/etc etc +&prompt.root; ln -s s/root root +&prompt.root; ln -s s/home home +&prompt.root; ln -s ../s/usr-local usr/local +&prompt.root; ln -s ../s/usr-X11R6 usr/X11R6 +&prompt.root; ln -s s/tmp tmp +&prompt.root; ln -s s/var var + + + The right time to stop the jails is now: + + &prompt.root; /etc/rc.d/jail stop + + + Unmount the original file systems: + + + &prompt.root; umount /home/j/ns/s +&prompt.root; umount /home/j/ns +&prompt.root; umount /home/j/mail/s +&prompt.root; umount /home/j/mail +&prompt.root; umount /home/j/www/s +&prompt.root; umount /home/j/www + + + The read-write systems are attached to the read-only + system (/s) and + must be unmounted first. + + + + Move the old read-only file system and replace it with + the new one. This will serve a backup and archive of the + old read-only file system should something go wrong. The + naming convention used here corresponds to when a new + read-only file system has been created. Move the original + &os; Ports Collection over to the new file system to save + some space and inodes: + + &prompt.root; cd /home/j +&prompt.root; mv mroot mroot.20060601 +&prompt.root; mv mroot2 mroot +&prompt.root; mv mroot.20060601/usr/ports mroot/usr + + + At this point the new read-only template is ready, so + the only remaining task is to remount the file systems and + start the jails: + + &prompt.root; mount -a +&prompt.root; /etc/rc.d/jail start + + + + Use &man.jls.8; to check if the jails started correctly. + Do not forget to run mergemaster in each jail. The + configuration files will need to be updated as well as the + rc.d scripts. + + +