Index: en_US.ISO8859-1/books/porters-handbook/testing/chapter.xml
===================================================================
--- en_US.ISO8859-1/books/porters-handbook/testing/chapter.xml
+++ en_US.ISO8859-1/books/porters-handbook/testing/chapter.xml
@@ -187,6 +187,677 @@
+
+ <application>Poudriere</application>
+
+ As a ports contributor, poudriere
+ is one of the most important and helpful testing and build
+ tools. Its main features comprise:
+
+
+
+ Bulk building of the entire ports tree, specific subsets
+ of the ports tree or a single port including its
+ dependencies
+
+
+
+ Automatic packaging of build results
+
+
+
+ Generation of build log files per port
+
+
+
+ Maintenance of a signed &man.pkg.8; repository
+
+
+
+ Testing of port builds before submitting a patch to the
+ &os; bug tracker or committing to the ports tree
+
+
+
+ Testing for successful build of using different
+ options
+
+
+
+ Because poudriere performs its
+ building in a clean &man.jail.8; environment and uses
+ &man.zfs.8; features, it has several advantages over traditional
+ testing on the host system:
+
+
+
+ No pollution of the host environment: No leftover files,
+ no accidental removals, no changes of existing configuration
+ files etc.
+
+
+
+ Verify pkg-plist for missing or superfluous
+ entries
+
+
+
+ Ports committers tend to ask for a
+ poudriere log alongside a patch
+ submission to assess whether the patch is ready for
+ integration into the ports tree
+
+
+
+ It is also quite straightforward to set up and use, and has
+ no dependencies except any supported FreeBSD release. The
+ following sections contain information how to install, set up
+ and run poudriere as part of the
+ normal workflow of a ports contributor.
+
+ This article assumes this file tree layout, which is the
+ default in &os;. Substitute any local changes accordingly.
+ The ports tree, represented by ${PORTSDIR} is
+ located in /usr/ports, and
+ ${LOCALBASE} is identical to
+ ${PREFIX} and set to
+ /usr/local.
+
+
+ Installing <application>Poudriere</application>
+
+ Poudriere is available in the
+ ports tree in ports-mgmt/poudriere. It can be
+ installed using &man.pkg.8; or from ports:
+
+
+ &prompt.root; pkg install poudriere
+or
+&prompt.root; make -C /usr/ports/ports-mgmt/poudriere install clean
+
+ There is also a work-in-progress version of poudriere
+ which will eventually become the next release. It is
+ available in ports-mgmt/poudriere-devel. This
+ development version often has newer interesting features, it
+ may be more useful than the regular one.
+
+
+
+
+
+ Setting Up <application>Poudriere</application>
+
+ The port installs a default configuration file,
+ /usr/local/etc/poudriere.conf. Each
+ parameter is well documented, but for starters, here is a
+ minimal example config file:
+
+ ZPOOL=tank
+ZROOTFS=/poudriere
+BASEFS=/poudriere
+DISTFILES_CACHE=/usr/ports/distfiles
+RESOLV_CONF=/etc/resolv.conf
+FREEBSD_HOST=ftp://ftp.freebsd.org
+SVN_HOST=svn0.eu.FreeBSD.org
+
+
+
+ ZPOOL
+
+
+ The name of the zfs storage pool which poudriere
+ shall use. Must be listed in the output of
+ zpool status.
+
+
+
+
+ ZROOTFS
+
+
+ The root of poudriere-managed file systems. This
+ entry will cause poudriere to
+ create &man.zfs.8; file systems under
+ tank/poudriere/something.
+
+
+
+
+ BASEFS
+
+
+ The root mount point for
+ poudriere file systems. This
+ entry will cause poudriere to mount
+ tank/poudriere to
+ /poudriere.
+
+
+
+
+ DISTFILES_CACHE
+
+
+ This entry defines where distfiles are stored. In
+ this example, poudriere and the host share the distfiles
+ store location. This avoids downloading tarballs which
+ are already present on the system.
+
+
+
+
+ RESOLV_CONF
+
+
+ This entry cause poudriere to use the system-wide
+ resolv.conf inside its build jails.
+ This is required so that poudriere can fetch the
+ distfiles during installation and if no proxy is used.
+ Refer to the default configuration file for configuring
+ a proxy.
+
+
+
+
+ FREEBSD_HOST
+
+
+ The server from where jails are installed and
+ updated when not using
+ subversion. Choose a server
+ location which is close, for example
+ ftp.au.freebsd.org if the machine is
+ located in Australia.
+
+
+
+
+ SVN_HOST
+
+
+ The server from where jails are installed and
+ updated when using
+ subversion. Again, choose a
+ server location close by.
+
+
+
+
+
+
+
+ Creating Poudriere Jails
+
+ Create the base jails which poudriere will use for
+ building:
+
+ &prompt.root; poudriere jail -c -j 93Ramd64 -v 9.3-RELEASE -a amd64
+
+ This will fetch a 9.3-RELEASE for
+ amd64 from the ftp server given by
+ FREEBSD_HOST in
+ poudriere.conf, create the zfs file
+ system tank/poudriere/jails/p093Ramd64,
+ mount it on /poudriere/jails/p093Ramd64
+ and extract the 9.3-RELEASE tarballs into
+ this file system.
+
+ &prompt.root; poudriere jail -c -j 10i386 -v stable/10 -a i386 -m svn
+
+ This will create
+ tank/poudriere/jails/10i386, mount it on
+ /poudriere/jails/10i386, then check out
+ the tip of the subversion branch of
+ &os;-10-STABLE from
+ SVN_HOST in
+ poudriere.conf into
+ /poudriere/jails/10i386/usr/src, then
+ complete a buildworld and install
+ it into /poudriere/jails/10i386.
+
+
+ If a specific svn revision is needed, append it to the
+ version string. for example poudriere jail -c -j
+ 10i386 -v stable/10@123456 -a i386 -m
+ svn.
+
+
+
+ While it is possible to build a newer version of &os; on
+ an older version, it will not run most of the time. For
+ example, if a stable/10 jail is needed,
+ the host will have to run stable/10 too,
+ running 10.0-RELEASE will not be
+ enough.
+
+
+ To get a listing about the jails currently known to
+ poudriere, poudriere jail -l can be
+ used:
+
+ &prompt.root; poudriere jail -l
+JAILNAME VERSION ARCH METHOD
+93Ramd64 9.3-RELEASE amd64 ftp
+10i386 10.0-STABLE i386 svn
+
+
+
+
+ Keeping Poudriere Jails Updated
+
+ Managing updates is very straightforward. The command
+ poudriere jail -u -j
+ JAILNAME will update
+ the specified jail to the latest version available. For the
+ releases, this will update to the latest patchlevel with
+ freebsd-update. For the
+ STABLE versions will update to the latest
+ svn revision on the stable branch.
+
+
+ For jails employing the svn method,
+ it is helpful to add -J
+ NumberOfParallelBuildJobs
+ to speed up the build.
+
+
+
+
+ Setting Up Ports Trees for Use with Poudriere
+
+ There are multiple ways to use ports trees in poudriere,
+ the most straightforward way is to have poudriere create a
+ default ports tree for itself:
+
+ &prompt.root; poudriere ports -c
+
+ This command will create tank/poudriere/ports/default,
+ mount it on /poudriere/ports/default and populate it using
+ portsnap(8). Afterwards it is included in the list of known
+ ports trees:
+
+ &prompt.root; poudriere ports -l
+PORTSTREE METHOD PATH
+default portsnap /poudriere/ports/default
+
+
+ Note that the "default" ports tree is special. Each of
+ the build commands explained later will implicitly use this
+ ports tree unless specifically specified otherwise. To use
+ another tree, add -p
+ treename to the
+ commands.
+
+
+ While useful for regular bulk bulds, as a ports
+ contributor having this default ports tree with the portsnap
+ method may not be the best way to deal with local
+ modifications. Just akin the creation of jails, it is
+ possible to use a different method for creating the ports
+ tree. To add an additional ports tree for testing local
+ modifications and ports development, checking out the tree via
+ svn is possible:
+
+ &prompt.root; poudriere ports -c -m svn -p subversive
+
+ Will create
+ tank/poudriere/ports/subversive, mount it
+ on /poudriere/ports/subversive and
+ populate it using svn. Again, afterwards
+ it is included in the list of known ports trees:
+
+ &prompt.root; poudriere ports -l
+PORTSTREE METHOD PATH
+default portsnap /poudriere/ports/default
+subversive svn /poudriere/ports/subversive
+
+
+ The svn method allows extra
+ qualifiers to tell subversion
+ exactly how to fetch data. This is explained in the
+ poudriere manpage. For instance, poudriere ports
+ -c -m svn+https -p subversive enforces the
+ checkout using https.
+
+
+
+
+
+
+ Using Manually Managed Ports Trees with Poudriere
+
+ Depending on the workflow, it can be extremely helpful to
+ use ports trees which are maintained manually. For instance,
+ if there is local copy of the ports tree in
+ /work/ports, point poudriere to the
+ location:
+
+ &prompt.root; poudriere ports -c -F -f none -M /work/ports -p development
+
+ This will be listed in the table of known trees:
+
+ &prompt.root; poudriere ports -l
+PORTSTREE METHOD PATH
+development - /work/ports
+
+
+ The dash in the column METHOD means
+ that poudriere will not update or change this ports tree
+ ever. It is completely up to the user to maintain this
+ tree, including all local modifications that may be used for
+ testing new ports and submitting patches.
+
+
+
+
+
+ Keeping Poudriere Ports Trees Updated
+
+ As straightforward as with jails described earlier:
+
+ &prompt.root; poudriere ports -u -p PORTSTREE
+
+ Will update the given
+ PORTSTREE, one tree given by the
+ output of poudriere -l, to the latest
+ revision available on the official servers.
+
+
+ Ports trees without a method, see , cannot be
+ updated like this. They must be updated manually by the
+ porter.
+
+
+
+
+
+ Testing Ports
+
+ After jails and ports trees have been set up, the result
+ of a contributor's modifications to the ports tree can be
+ tested.
+
+ For example, local modifications to the www/firefox port located in
+ /work/ports/www/firefox, can be tested in
+ the previously created 9.3-RELEASE jail with poudriere with
+ the command:
+
+ &prompt.root; poudriere testport -j 93Ramd64 -p development -o www/firefox
+
+ This will build all dependencies of firefox. If they have
+ been built previously and are still up-to-date, those
+ pre-built packages will be installed. It will be using the
+ default options in jails. Then it will build firefox
+ itself.
+
+ During the build process it will log the complete build of
+ every port it builds to
+ /poudriere/data/logs/bulk/93Ri386-development/build-time/logs.
+
+ The directory name 93Ri386-development
+ is derived from the arguments to -j and
+ -p, respectively. For convenience, it will
+ also maintain a symbolic link
+ /poudriere/data/logs/bulk/93Ri386-development/latest
+ pointing to the latest build-time
+ folder. Also in this folder there is an index.html for
+ observing the build process via a web browser.
+
+ By default, after the attempt to build, successful or not,
+ poudriere will clean up the jails and leave the log files in
+ the folders mentioned above for further investigation. In
+ order to investigate, keeping the jails running after the
+ build can be achieved by adding -i to the
+ testport command:
+
+ &prompt.root; poudriere testport -j 93Ramd64 -p development -i -o www/firefox
+
+ When poudriere is done with the build, and wether it built
+ the port successfully or not, it will end up with a shell
+ within the jail. It will then be possible to investigate
+ further. It is also possible to ask poudriere to leave the
+ jail running after it finishes. This can be achieved with
+ -I. Poudriere will output the command to
+ run when finished the jail is no longer needed. It is then
+ possible to &man.jexec.8; into it:
+
+ &prompt.root; poudriere testport -j 93Ramd64 -p development -I -o www/firefox
+[...]
+====>> Installing local Pkg repository to /usr/local/etc/pkg/repos
+====>> Leaving jail 93Ramd64-development-n running, mounted at /poudriere/data/.m/93Ramd64-development/ref for interactive run testing
+====>> To enter jail: jexec 93Ramd64-development-n env -i TERM=$TERM /usr/bin/login -fp root
+====>> To stop jail: poudriere jail -k -j 93Ramd64 -p development
+&prompt.root; jexec 93Ramd64-development-n env -i TERM=$TERM /usr/bin/login -fp root
+&prompt.root; [do some stuff in the jail]
+&prompt.root; exit
+&prompt.root; poudriere jail -k -j 93Ramd64 -p development
+====>> Umounting file systems
+
+ An integral part of the &os; ports build infrastructure is
+ the possibility to tweak ports to personal preferences by
+ using options. These can be tested with poudriere as well.
+ Adding the -c:
+
+ &prompt.root; poudriere testport -c -o www/firefox
+
+ Will present the port configuration dialog, before the
+ port is built. All depedencies will still be built with
+ default options, only the one given after in the -o
+ category/portname
+ argument will use the specified options. Testing dependent
+ ports with non-default options can be accomplished using sets,
+ see .
+
+
+ When testing ports where pkg-plist
+ is altered during build depending on the selected options,
+ it is recommended to perform a test run with all
+ options selected and one with all
+ options deselected.
+
+
+
+
+ Using Sets
+
+ For all actions involving builds, a so-called set can be
+ specified using -z
+ setname. A set refers
+ to a fully independent build. This allows, for instance, the
+ usage of the testport subcommand with non-standard options for
+ the dependent ports.
+
+ To use sets, poudriere expects an existing directory
+ structure akin to PORT_DBDIR, defaults to
+ /var/db/ports in its configuration
+ directory. This directory is then nullfs-mounted into the
+ jails where the ports and their dependencies are built.
+ Usually a suitable starting point can be obtained by
+ recursively copying the existing PORT_DBDIR
+ to
+ /usr/local/etc/poudriere.d/jailname-portname-setname-options.
+ This is described in detail in &man.poudriere.8;. For
+ instance, testing www/firefox in a specific set named
+ devset, add the -z
+ devset parameter to the testport command:
+
+ &prompt.root; poudriere testport -j 93Ramd64 -p development -z devset -o www/firefox
+
+ This will look for the existence of these directories in
+ this order:
+
+
+
+ /usr/local/etc/poudriere.d/93Ramd64-development-devset-options
+
+
+
+ /usr/local/etc/poudriere.d/93Ramd64-devset-options
+
+
+
+ /usr/local/etc/poudriere.d/93Ramd64-development-options
+
+
+
+ /usr/local/etc/poudriere.d/devset-options
+
+
+
+ /usr/local/etc/poudriere.d/development-options
+
+
+
+ /usr/local/etc/poudriere.d/93Ramd64-options
+
+
+
+ /usr/local/etc/poudriere.d/options
+
+
+
+ From this list, poudriere will nullfs-mount the
+ first existing directory tree into the
+ /var/db/ports directory of the build
+ jails. Hence, all custom options will be used for all the
+ ports during this run of poudriere testport.
+
+ Once the directory structure for a set is provided, it is
+ also possible to alter the options for a particular port for
+ this set using poudriere, for instance:
+
+ &prompt.root; poudriere options -c www/firefox -z devset
+
+ This will show the configuration dialog for www/firefox where options can be
+ edited. Afterwards it will save the selected options to the
+ set devset.
+
+
+ poudriere is very flexible in the configuration of
+ options. They can be set for particular jails, ports trees
+ and for multiple ports by one command. Refer to
+ &man.poudriere.8; for details.
+
+
+
+
+
+ Providing a Custom <filename>make.conf</filename>
+ File
+
+ Similar to using sets, poudriere will also use a custom
+ make.conf if it is provided. No special
+ command line argument is necessary. Instead, poudriere will
+ look for existing files matching a name scheme derived from
+ the command line. For instance:
+
+ &prompt.root; poudriere testport -j 93Ramd64 -p development -z devset -o www/firefox
+
+ will cause poudriere to check for the existence of the
+ following files in the order:
+
+
+
+ /usr/local/etc/poudriere.d/make.conf
+
+
+
+ /usr/local/etc/poudriere.d/devset-make.conf
+
+
+
+ /usr/local/etc/poudriere.d/development-make.conf
+
+
+
+ /usr/local/etc/poudriere.d/93Ramd64-make.conf
+
+
+
+ /usr/local/etc/poudriere.d/93Ramd64-development-make.conf
+
+
+
+ /usr/local/etc/poudriere.d/93Ramd64-devset-make.conf
+
+
+
+ /usr/local/etc/poudriere.d/93Ramd64-development-devset-make.conf
+
+
+
+ Unlike with sets, all of the found files will be appended,
+ in that order, into one
+ make.conf inside the build jails. It is
+ hence possible to have general make variables, intended to
+ affect all builds in
+ /usr/local/etc/poudriere.d/make.conf.
+ Special variables, intended to affect only certain jails or
+ sets can be set in specialised make.conf
+ files, such as
+ /usr/local/etc/poudriere.d/93Ramd64-development-devset-make.conf.
+
+
+ Using <filename>make.conf</filename> to Change Default
+ <application>Perl</application>
+
+ To build a set with a non default
+ Perl version, for example,
+ 5.20, using a set named
+ perl5-20, create a
+ perl5-20-make.conf with this
+ line:
+
+ DEFAULT_VERSIONS+= perl=5.20
+
+
+ Note the use of += so that if the
+ variable is already set in the default
+ make.conf its content will not be
+ overwritten.
+
+
+
+
+
+
+ Pruning no Longer Needed Distfiles
+
+ The poudriere tool comes with a built-in mechanism to
+ remove outdated distfiles that are no longer used in any port
+ of a given tree. The command
+
+ &prompt.root; poudriere distclean -p portstree
+
+ will scan the distfiles folder,
+ DISTFILES_CACHE in
+ poudriere.conf, versus the ports tree
+ given by the -p
+ portstree argument and
+ prompt for removal of those distfiles. To skip the prompt and
+ remove all unused files unconditionally, the
+ -y argument can be added:
+
+ &prompt.root; poudriere distclean -p portstree -y
+
+
+
+
+
+
Tinderbox
@@ -202,23 +873,4 @@
Tinderbox
website for more details.
-
-
- Poudriere
-
- As a ports contributor, consider installing
- poudriere. It is a powerful
- system for building and testing ports.
- Poudriere can be installed with
- ports-mgmt/poudriere.
-
- There is also a ports-mgmt/poudriere-devel that often
- has newer features that are mostly helpful when testing
- ports.
-
- Visit the Poudriere
- website for more details.
-