diff --git a/documentation/content/en/books/handbook/cutting-edge/_index.adoc b/documentation/content/en/books/handbook/cutting-edge/_index.adoc index ee11b2951b..fe9e5503fb 100644 --- a/documentation/content/en/books/handbook/cutting-edge/_index.adoc +++ b/documentation/content/en/books/handbook/cutting-edge/_index.adoc @@ -1,1030 +1,1030 @@ --- title: Chapter 24. Updating and Upgrading FreeBSD part: Part III. System Administration prev: books/handbook/l10n next: books/handbook/dtrace --- [[updating-upgrading]] = Updating and Upgrading FreeBSD :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :source-highlighter: rouge :experimental: :skip-front-matter: :xrefstyle: basic :relfileprefix: ../ :outfilesuffix: :sectnumoffset: 24 ifeval::["{backend}" == "html5"] :imagesdir: ../../../../images/books/handbook/cutting-edge/ endif::[] ifeval::["{backend}" == "pdf"] :imagesdir: ../../../../static/images/books/handbook/cutting-edge/ endif::[] ifeval::["{backend}" == "epub3"] :imagesdir: ../../../../static/images/books/handbook/cutting-edge/ endif::[] include::shared/authors.adoc[] include::shared/releases.adoc[] include::shared/en/mailing-lists.adoc[] include::shared/en/teams.adoc[] include::shared/en/urls.adoc[] toc::[] [[updating-upgrading-synopsis]] == Synopsis FreeBSD is under constant development between releases. Some people prefer to use the officially released versions, while others prefer to keep in sync with the latest developments. However, even official releases are often updated with security and other critical fixes. Regardless of the version used, FreeBSD provides all the necessary tools to keep the system updated, and allows for easy upgrades between versions. This chapter describes how to track the development system and the basic tools for keeping a FreeBSD system up-to-date. After reading this chapter, you will know: * How to keep a FreeBSD system up-to-date with freebsd-update or Git. * How to compare the state of an installed system against a known pristine copy. * How to keep the installed documentation up-to-date with Git or documentation ports. * The difference between the two development branches: FreeBSD-STABLE and FreeBSD-CURRENT. * How to rebuild and reinstall the entire base system. Before reading this chapter, you should: * Properly set up the network connection (crossref:advanced-networking[advanced-networking,Advanced Networking]). * Know how to install additional third-party software (crossref:ports[ports,Installing Applications: Packages and Ports]). [NOTE] ==== Throughout this chapter, `git` is used to obtain and update FreeBSD sources. Optionally, the package:devel/git[] port or package may be used. ==== [[updating-upgrading-freebsdupdate]] == FreeBSD Update Applying security patches in a timely manner and upgrading to a newer release of an operating system are important aspects of ongoing system administration. FreeBSD includes a utility called `freebsd-update` which can be used to perform both these tasks. This utility supports binary security and errata updates to FreeBSD, without the need to manually compile and install the patch or a new kernel. Binary updates are available for all architectures and releases currently supported by the security team. The list of supported releases and their estimated end-of-life dates are listed at https://www.FreeBSD.org/security/[https://www.FreeBSD.org/security/]. This utility also supports operating system upgrades to minor point releases as well as upgrades to another release branch. Before upgrading to a new release, review its release announcement as it contains important information pertinent to the release. Release announcements are available from https://www.FreeBSD.org/releases/[https://www.FreeBSD.org/releases/]. [NOTE] ==== If a `crontab` utilizing the features of man:freebsd-update[8] exists, it must be disabled before upgrading the operating system. ==== This section describes the configuration file used by `freebsd-update`, demonstrates how to apply a security patch and how to upgrade to a minor or major operating system release, and discusses some of the considerations when upgrading the operating system. [[freebsdupdate-config-file]] === The Configuration File The default configuration file for `freebsd-update` works as-is. Some users may wish to tweak the default configuration in [.filename]#/etc/freebsd-update.conf#, allowing better control of the process. The comments in this file explain the available options, but the following may require a bit more explanation: [.programlisting] .... # Components of the base system which should be kept updated. Components world kernel .... This parameter controls which parts of FreeBSD will be kept up-to-date. The default is to update the entire base system and the kernel. Individual components can instead be specified, such as `src/base` or `src/sys`. However, the best option is to leave this at the default as changing it to include specific items requires every needed item to be listed. Over time, this could have disastrous consequences as source code and binaries may become out of sync. [.programlisting] .... # Paths which start with anything matching an entry in an IgnorePaths # statement will be ignored. IgnorePaths /boot/kernel/linker.hints .... To leave specified directories, such as [.filename]#/bin# or [.filename]#/sbin#, untouched during the update process, add their paths to this statement. This option may be used to prevent `freebsd-update` from overwriting local modifications. [.programlisting] .... # Paths which start with anything matching an entry in an UpdateIfUnmodified # statement will only be updated if the contents of the file have not been # modified by the user (unless changes are merged; see below). UpdateIfUnmodified /etc/ /var/ /root/ /.cshrc /.profile .... This option will only update unmodified configuration files in the specified directories. Any changes made by the user will prevent the automatic updating of these files. There is another option, `KeepModifiedMetadata`, which will instruct `freebsd-update` to save the changes during the merge. [.programlisting] .... # When upgrading to a new FreeBSD release, files which match MergeChanges # will have any local changes merged into the version from the new release. MergeChanges /etc/ /var/named/etc/ /boot/device.hints .... List of directories with configuration files that `freebsd-update` should attempt to merge. The file merge process is a series of man:diff[1] patches similar to man:mergemaster[8], but with fewer options. Merges are either accepted, open an editor, or cause `freebsd-update` to abort. When in doubt, backup [.filename]#/etc# and just accept the merges. See man:mergemaster[8] for more information about `mergemaster`. [.programlisting] .... # Directory in which to store downloaded updates and temporary # files used by FreeBSD Update. # WorkDir /var/db/freebsd-update .... This directory is where all patches and temporary files are placed. In cases where the user is doing a version upgrade, this location should have at least a gigabyte of disk space available. [.programlisting] .... # When upgrading between releases, should the list of Components be # read strictly (StrictComponents yes) or merely as a list of components # which *might* be installed of which FreeBSD Update should figure out # which actually are installed and upgrade those (StrictComponents no)? # StrictComponents no .... When this option is set to `yes`, `freebsd-update` will assume that the `Components` list is complete and will not attempt to make changes outside of the list. Effectively, `freebsd-update` will attempt to update every file which belongs to the `Components` list. [[freebsdupdate-security-patches]] === Applying Security Patches The process of applying FreeBSD security patches has been simplified, allowing an administrator to keep a system fully patched using `freebsd-update`. More information about FreeBSD security advisories can be found in crossref:security[security-advisories,"FreeBSD Security Advisories"]. FreeBSD security patches may be downloaded and installed using the following commands. The first command will determine if any outstanding patches are available, and if so, will list the files that will be modifed if the patches are applied. The second command will apply the patches. [source,shell] .... # freebsd-update fetch # freebsd-update install .... If the update applies any kernel patches, the system will need a reboot in order to boot into the patched kernel. If the patch was applied to any running binaries, the affected applications should be restarted so that the patched version of the binary is used. [NOTE] ==== Usually, the user needs to be prepared to reboot the system. To know if a reboot is required by a kernel update, execute the commands `freebsd-version -k` and `uname -r` and if it differs a reboot is required. ==== The system can be configured to automatically check for updates once every day by adding this entry to [.filename]#/etc/crontab#: [.programlisting] .... @daily root freebsd-update cron .... If patches exist, they will automatically be downloaded but will not be applied. The `root` user will be sent an email so that the patches may be reviewed and manually installed with `freebsd-update install`. If anything goes wrong, `freebsd-update` has the ability to roll back the last set of changes with the following command: [source,shell] .... # freebsd-update rollback Uninstalling updates... done. .... Again, the system should be restarted if the kernel or any kernel modules were modified and any affected binaries should be restarted. Only the [.filename]#GENERIC# kernel can be automatically updated by `freebsd-update`. If a custom kernel is installed, it will have to be rebuilt and reinstalled after `freebsd-update` finishes installing the updates. The default kernel name is _GENERIC_. The man:uname[1] command may be used to verify its installation. [NOTE] ==== Always keep a copy of the [.filename]#GENERIC# kernel in [.filename]#/boot/GENERIC#. It will be helpful in diagnosing a variety of problems and in performing version upgrades. Refer to <> for instructions on how to get a copy of the [.filename]#GENERIC# kernel. ==== Unless the default configuration in [.filename]#/etc/freebsd-update.conf# has been changed, `freebsd-update` will install the updated kernel sources along with the rest of the updates. Rebuilding and reinstalling a new custom kernel can then be performed in the usual way. The updates distributed by `freebsd-update` do not always involve the kernel. It is not necessary to rebuild a custom kernel if the kernel sources have not been modified by `freebsd-update install`. However, `freebsd-update` will always update [.filename]#/usr/src/sys/conf/newvers.sh#. The current patch level, as indicated by the `-p` number reported by `uname -r`, is obtained from this file. Rebuilding a custom kernel, even if nothing else changed, allows `uname` to accurately report the current patch level of the system. This is particularly helpful when maintaining multiple systems, as it allows for a quick assessment of the updates installed in each one. [[freebsdupdate-upgrade]] === Performing Major and Minor Version Upgrades Upgrades from one minor version of FreeBSD to another, like from FreeBSD 9.0 to FreeBSD 9.1, are called _minor version_ upgrades. _Major version_ upgrades occur when FreeBSD is upgraded from one major version to another, like from FreeBSD 9.X to FreeBSD 10.X. Both types of upgrades can be performed by providing `freebsd-update` with a release version target. [NOTE] ==== If the system is running a custom kernel, make sure that a copy of the [.filename]#GENERIC# kernel exists in [.filename]#/boot/GENERIC# before starting the upgrade. Refer to <> for instructions on how to get a copy of the [.filename]#GENERIC# kernel. ==== The following command, when run on a FreeBSD 9.0 system, will upgrade it to FreeBSD 9.1: [source,shell] .... # freebsd-update -r 9.1-RELEASE upgrade .... After the command has been received, `freebsd-update` will evaluate the configuration file and current system in an attempt to gather the information necessary to perform the upgrade. A screen listing will display which components have and have not been detected. For example: [source,shell] .... Looking up update.FreeBSD.org mirrors... 1 mirrors found. Fetching metadata signature for 9.0-RELEASE from update1.FreeBSD.org... done. Fetching metadata index... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/smp src/base src/bin src/contrib src/crypto src/etc src/games src/gnu src/include src/krb5 src/lib src/libexec src/release src/rescue src/sbin src/secure src/share src/sys src/tools src/ubin src/usbin world/base world/info world/lib32 world/manpages The following components of FreeBSD do not seem to be installed: kernel/generic world/catpages world/dict world/doc world/games world/proflibs Does this look reasonable (y/n)? y .... At this point, `freebsd-update` will attempt to download all files required for the upgrade. In some cases, the user may be prompted with questions regarding what to install or how to proceed. When using a custom kernel, the above step will produce a warning similar to the following: [source,shell] .... WARNING: This system is running a "MYKERNEL" kernel, which is not a kernel configuration distributed as part of FreeBSD 9.0-RELEASE. This kernel will not be updated: you MUST update the kernel manually before running "/usr/sbin/freebsd-update install" .... This warning may be safely ignored at this point. The updated [.filename]#GENERIC# kernel will be used as an intermediate step in the upgrade process. Once all the patches have been downloaded to the local system, they will be applied. This process may take a while, depending on the speed and workload of the machine. Configuration files will then be merged. The merging process requires some user intervention as a file may be merged or an editor may appear on screen for a manual merge. The results of every successful merge will be shown to the user as the process continues. A failed or ignored merge will cause the process to abort. Users may wish to make a backup of [.filename]#/etc# and manually merge important files, such as [.filename]#master.passwd# or [.filename]#group# at a later time. [NOTE] ==== The system is not being altered yet as all patching and merging is happening in another directory. Once all patches have been applied successfully, all configuration files have been merged and it seems the process will go smoothly, the changes can be committed to disk by the user using the following command: [source,shell] .... # freebsd-update install .... ==== The kernel and kernel modules will be patched first. If the system is running with a custom kernel, use man:nextboot[8] to set the kernel for the next boot to the updated [.filename]#/boot/GENERIC#: [source,shell] .... # nextboot -k GENERIC .... [WARNING] ==== Before rebooting with the [.filename]#GENERIC# kernel, make sure it contains all the drivers required for the system to boot properly and connect to the network, if the machine being updated is accessed remotely. In particular, if the running custom kernel contains built-in functionality usually provided by kernel modules, make sure to temporarily load these modules into the [.filename]#GENERIC# kernel using the [.filename]#/boot/loader.conf# facility. It is recommended to disable non-essential services as well as any disk and network mounts until the upgrade process is complete. ==== The machine should now be restarted with the updated kernel: [source,shell] .... # shutdown -r now .... Once the system has come back online, restart `freebsd-update` using the following command. Since the state of the process has been saved, `freebsd-update` will not start from the beginning, but will instead move on to the next phase and remove all old shared libraries and object files. [source,shell] .... # freebsd-update install .... [NOTE] ==== Depending upon whether any library version numbers were bumped, there may only be two install phases instead of three. ==== The upgrade is now complete. If this was a major version upgrade, reinstall all ports and packages as described in <>. [[freebsd-update-custom-kernel-9x]] ==== Custom Kernels with FreeBSD 9.X and Later Before using `freebsd-update`, ensure that a copy of the [.filename]#GENERIC# kernel exists in [.filename]#/boot/GENERIC#. If a custom kernel has only been built once, the kernel in [.filename]#/boot/kernel.old# is the `GENERIC` kernel. Simply rename this directory to [.filename]#/boot/GENERIC#. If a custom kernel has been built more than once or if it is unknown how many times the custom kernel has been built, obtain a copy of the `GENERIC` kernel that matches the current version of the operating system. If physical access to the system is available, a copy of the `GENERIC` kernel can be installed from the installation media: [source,shell] .... # mount /cdrom # cd /cdrom/usr/freebsd-dist # tar -C/ -xvf kernel.txz boot/kernel/kernel .... Alternately, the `GENERIC` kernel may be rebuilt and installed from source: [source,shell] .... # cd /usr/src # make kernel __MAKE_CONF=/dev/null SRCCONF=/dev/null .... For this kernel to be identified as the `GENERIC` kernel by `freebsd-update`, the [.filename]#GENERIC# configuration file must not have been modified in any way. It is also suggested that the kernel is built without any other special options. Rebooting into the [.filename]#GENERIC# kernel is not required as `freebsd-update` only needs [.filename]#/boot/GENERIC# to exist. [[freebsdupdate-portsrebuild]] ==== Upgrading Packages After a Major Version Upgrade Generally, installed applications will continue to work without problems after minor version upgrades. Major versions use different Application Binary Interfaces (ABIs), which will break most third-party applications. After a major version upgrade, all installed packages and ports need to be upgraded. Packages can be upgraded using `pkg upgrade`. To upgrade installed ports, use a utility such as package:ports-mgmt/portmaster[]. A forced upgrade of all installed packages will replace the packages with fresh versions from the repository even if the version number has not increased. This is required because of the ABI version change when upgrading between major versions of FreeBSD. The forced upgrade can be accomplished by performing: [source,shell] .... # pkg-static upgrade -f .... A rebuild of all installed applications can be accomplished with this command: [source,shell] .... # portmaster -af .... This command will display the configuration screens for each application that has configurable options and wait for the user to interact with those screens. To prevent this behavior, and use only the default options, include `-G` in the above command. Once the software upgrades are complete, finish the upgrade process with a final call to `freebsd-update` in order to tie up all the loose ends in the upgrade process: [source,shell] .... # freebsd-update install .... If the [.filename]#GENERIC# kernel was temporarily used, this is the time to build and install a new custom kernel using the instructions in crossref:kernelconfig[kernelconfig,Configuring the FreeBSD Kernel]. Reboot the machine into the new FreeBSD version. The upgrade process is now complete. [[freebsdupdate-system-comparison]] === System State Comparison The state of the installed FreeBSD version against a known good copy can be tested using `freebsd-update IDS`. This command evaluates the current version of system utilities, libraries, and configuration files and can be used as a built-in Intrusion Detection System (IDS). [WARNING] ==== This command is not a replacement for a real IDS such as package:security/snort[]. As `freebsd-update` stores data on disk, the possibility of tampering is evident. While this possibility may be reduced using `kern.securelevel` and by storing the `freebsd-update` data on a read-only file system when not in use, a better solution would be to compare the system against a secure disk, such as a DVD or securely stored external USB disk device. An alternative method for providing IDS functionality using a built-in utility is described in crossref:security[security-ids,"Binary Verification"] ==== To begin the comparison, specify the output file to save the results to: [source,shell] .... # freebsd-update IDS >> outfile.ids .... The system will now be inspected and a lengthy listing of files, along with the SHA256 hash values for both the known value in the release and the current installation, will be sent to the specified output file. The entries in the listing are extremely long, but the output format may be easily parsed. For instance, to obtain a list of all files which differ from those in the release, issue the following command: [source,shell] .... # cat outfile.ids | awk '{ print $1 }' | more /etc/master.passwd /etc/motd /etc/passwd /etc/pf.conf .... This sample output has been truncated as many more files exist. Some files have natural modifications. For example, [.filename]#/etc/passwd# will be modified if users have been added to the system. Kernel modules may differ as `freebsd-update` may have updated them. To exclude specific files or directories, add them to the `IDSIgnorePaths` option in [.filename]#/etc/freebsd-update.conf#. [[updating-upgrading-documentation]] == Updating the Documentation Set Documentation is an integral part of the FreeBSD operating system. While an up-to-date version of the FreeBSD documentation is always available on the FreeBSD web site (link:https://docs.FreeBSD.org[Documentation Portal]), it can be handy to have an up-to-date, local copy of the FreeBSD website, handbooks, FAQ, and articles. This section describes how to use either source or the FreeBSD Ports Collection to keep a local copy of the FreeBSD documentation up-to-date. For information on editing and submitting corrections to the documentation, refer to the FreeBSD Documentation Project Primer for New Contributors (link:{fdp-primer}[FreeBSD Documentation Project Primer for New Contributors]). [[updating-installed-documentation]] === Updating Documentation from Source Rebuilding the FreeBSD documentation from source requires a collection of tools which are not part of the FreeBSD base system. The required tools can be installed following link:{fdp-primer}#overview-quick-start[these steps] from the FreeBSD Documentation Project Primer. Once installed, use `git` to fetch a clean copy of the documentation source: [source,shell] .... # git clone https://git.FreeBSD.org/doc.git /usr/doc .... The initial download of the documentation sources may take a while. Let it run until it completes. Future updates of the documentation sources may be fetched by running: [source,shell] .... # git pull .... Once an up-to-date snapshot of the documentation sources has been fetched to [.filename]#/usr/doc#, everything is ready for an update of the installed documentation. A full update may be performed by typing: [source,shell] .... # cd /usr/doc # make .... [[current-stable]] == Tracking a Development Branch FreeBSD has two development branches: FreeBSD-CURRENT and FreeBSD-STABLE. This section provides an explanation of each branch and its intended audience, as well as how to keep a system up-to-date with each respective branch. [[current]] === Using FreeBSD-CURRENT FreeBSD-CURRENT is the "bleeding edge" of FreeBSD development and FreeBSD-CURRENT users are expected to have a high degree of technical skill. Less technical users who wish to track a development branch should track FreeBSD-STABLE instead. FreeBSD-CURRENT is the very latest source code for FreeBSD and includes works in progress, experimental changes, and transitional mechanisms that might or might not be present in the next official release. While many FreeBSD developers compile the FreeBSD-CURRENT source code daily, there are short periods of time when the source may not be buildable. These problems are resolved as quickly as possible, but whether or not FreeBSD-CURRENT brings disaster or new functionality can be a matter of when the source code was synced. FreeBSD-CURRENT is made available for three primary interest groups: . Members of the FreeBSD community who are actively working on some part of the source tree. . Members of the FreeBSD community who are active testers. They are willing to spend time solving problems, making topical suggestions on changes and the general direction of FreeBSD, and submitting patches. . Users who wish to keep an eye on things, use the current source for reference purposes, or make the occasional comment or code contribution. FreeBSD-CURRENT should _not_ be considered a fast-track to getting new features before the next release as pre-release features are not yet fully tested and most likely contain bugs. It is not a quick way of getting bug fixes as any given commit is just as likely to introduce new bugs as to fix existing ones. FreeBSD-CURRENT is not in any way "officially supported". To track FreeBSD-CURRENT: . Join the {freebsd-current} and the {dev-commits-src-main} lists. This is _essential_ in order to see the comments that people are making about the current state of the system and to receive important bulletins about the current state of FreeBSD-CURRENT. + The {dev-commits-src-main} list records the commit log entry for each change as it is made, along with any pertinent information on possible side effects. + To join these lists, go to {mailman-lists}, click on the list to subscribe to, and follow the instructions. In order to track changes to the whole source tree, not just the changes to FreeBSD-CURRENT, subscribe to the {dev-commits-src-all}. . Synchronize with the FreeBSD-CURRENT sources. Typically, `git` is used to check out the -CURRENT code from the `main` branch of one of the Git mirror sites listed in crossref:mirrors[git-mirrors,“Git Mirror Sites”]. . Due to the size of the repository, some users choose to only synchronize the sections of source that interest them or which they are contributing patches to. However, users that plan to compile the operating system from source must download _all_ of FreeBSD-CURRENT, not just selected portions. + Before compiling FreeBSD-CURRENT, read [.filename]#/usr/src/Makefile# very carefully and follow the instructions in <>. Read the {freebsd-current} and [.filename]#/usr/src/UPDATING# to stay up-to-date on other bootstrapping procedures that sometimes become necessary on the road to the next release. . Be active! FreeBSD-CURRENT users are encouraged to submit their suggestions for enhancements or bug fixes. Suggestions with accompanying code are always welcome. [[stable]] === Using FreeBSD-STABLE FreeBSD-STABLE is the development branch from which major releases are made. Changes go into this branch at a slower pace and with the general assumption that they have first been tested in FreeBSD-CURRENT. This is _still_ a development branch and, at any given time, the sources for FreeBSD-STABLE may or may not be suitable for general use. It is simply another engineering development track, not a resource for end-users. Users who do not have the resources to perform testing should instead run the most recent release of FreeBSD. Those interested in tracking or contributing to the FreeBSD development process, especially as it relates to the next release of FreeBSD, should consider following FreeBSD-STABLE. While the FreeBSD-STABLE branch should compile and run at all times, this cannot be guaranteed. Since more people run FreeBSD-STABLE than FreeBSD-CURRENT, it is inevitable that bugs and corner cases will sometimes be found in FreeBSD-STABLE that were not apparent in FreeBSD-CURRENT. For this reason, one should not blindly track FreeBSD-STABLE. It is particularly important _not_ to update any production servers to FreeBSD-STABLE without thoroughly testing the code in a development or testing environment. To track FreeBSD-STABLE: . Join the {freebsd-stable} in order to stay informed of build dependencies that may appear in FreeBSD-STABLE or any other issues requiring special attention. Developers will also make announcements in this mailing list when they are contemplating some controversial fix or update, giving the users a chance to respond if they have any issues to raise concerning the proposed change. + Join the relevant git list for the branch being tracked. For example, users tracking the {betarel-current-major}-STABLE branch should join the {dev-commits-src-branches}. This list records the commit log entry for each change as it is made, along with any pertinent information on possible side effects. + To join these lists, go to {mailman-lists}, click on the list to subscribe to, and follow the instructions. In order to track changes for the whole source tree, subscribe to {dev-commits-src-all}. . To install a new FreeBSD-STABLE system, install the most recent FreeBSD-STABLE release from the crossref:mirrors[mirrors,FreeBSD mirror sites] or use a monthly snapshot built from FreeBSD-STABLE. Refer to link:https://www.FreeBSD.org/snapshots/[www.freebsd.org/snapshots] for more information about snapshots. + To compile or upgrade to an existing FreeBSD system to FreeBSD-STABLE, use `git` to check out the source for the desired branch. Branch names, such as `stable/9`, are listed at link:https://www.FreeBSD.org/releng/[www.freebsd.org/releng]. . Before compiling or upgrading to FreeBSD-STABLE , read [.filename]#/usr/src/Makefile# carefully and follow the instructions in <>. Read the {freebsd-stable} and [.filename]#/usr/src/UPDATING# to keep up-to-date on other bootstrapping procedures that sometimes become necessary on the road to the next release. [[makeworld]] == Updating FreeBSD from Source Updating FreeBSD by compiling from source offers several advantages over binary updates. Code can be built with options to take advantage of specific hardware. Parts of the base system can be built with non-default settings, or left out entirely where they are not needed or desired. The build process takes longer to update a system than just installing binary updates, but allows complete customization to produce a tailored version of FreeBSD. [[updating-src-quick-start]] === Quick Start This is a quick reference for the typical steps used to update FreeBSD by building from source. Later sections describe the process in more detail. [.procedure] ==== * Update and Build + [source,shell] .... # git pull /usr/src <.> check /usr/src/UPDATING <.> # cd /usr/src <.> # make -j4 buildworld <.> # make -j4 kernel <.> # shutdown -r now <.> # etcupdate -p <.> # cd /usr/src <.> # make installworld <.> # etcupdate -B <.> # shutdown -r now <.> .... <.> Get the latest version of the source. See <> for more information on obtaining and updating source. <.> Check [.filename]#/usr/src/UPDATING# for any manual steps required before or after building from source. <.> Go to the source directory. <.> Compile the world, everything except the kernel. <.> Compile and install the kernel. This is equivalent to `make buildkernel installkernel`. <.> Reboot the system to the new kernel. <.> Update and merge configuraton files in [.filename]#/etc/# required before installworld. <.> Go to the source directory. <.> Install the world. <.> Update and merge configuration files in [.filename]#/etc/#. <.> Restart the system to use the newly-built world and kernel. ==== [[updating-src-preparing]] === Preparing for a Source Update Read [.filename]#/usr/src/UPDATING#. Any manual steps that must be performed before or after an update are described in this file. [[updating-src-obtaining-src]] === Updating the Source FreeBSD source code is located in [.filename]#/usr/src/#. -The preferred method of updating this source is through the Subversion version control system. +The preferred method of updating this source is through the Git version control system. Verify that the source code is under version control: [source,shell] .... # cd /usr/src # git remote --v origin ssh://git@gitrepo.FreeBSD.org/doc.git (fetch) origin ssh://git@gitrepo.FreeBSD.org/doc.git (push) .... This indicates that [.filename]#/usr/src/# is under version control and can be updated with man:git[1]: [[synching]] [source,shell] .... # git pull /usr/src .... The update process can take some time if the directory has not been updated recently. After it finishes, the source code is up to date and the build process described in the next section can begin. [NOTE] ==== *Obtaining the Source:* + If the output says `fatal: not a git repository`, the files there are missing or were installed with a different method. A new checkout of the source is required. [[updating-src-obtaining-src-repopath]] .FreeBSD Versions and Repository Branchs [cols="10%,10%,80%", options="header"] |=== | uname -r Output | Repository Path | Description |`_X.Y_-RELEASE` |`releng/_X.Y_` |The Release version plus only critical security and bug fix patches. This branch is recommended for most users. |`_X.Y_-STABLE` |`stable/_X_` | The Release version plus all additional development on that branch. _STABLE_ refers to the Applications Binary Interface (ABI) not changing, so software compiled for earlier versions still runs. For example, software compiled to run on FreeBSD 10.1 will still run on FreeBSD 10-STABLE compiled later. STABLE branches occasionally have bugs or incompatibilities which might affect users, although these are typically fixed quickly. |`_X_-CURRENT` |`main/` |The latest unreleased development version of FreeBSD. The CURRENT branch can have major bugs or incompatibilities and is recommended only for advanced users. |=== Determine which version of FreeBSD is being used with man:uname[1]: [source,shell] .... # uname -r 10.3-RELEASE .... Based on <>, the source used to update `10.3-RELEASE` has a repository path of `releng/10.3`. That path is used when checking out the source: [source,shell] .... # mv /usr/src /usr/src.bak <.> # git clone --branch releng/10.3 ssh://git@gitrepo.FreeBSD.org/src.git /usr/src <.> .... <.> Move the old directory out of the way. If there are no local modifications in this directory, it can be deleted. <.> The path from <> is added to the repository URL. The third parameter is the destination directory for the source code on the local system. ==== [[updating-src-building]] === Building from Source The _world_, or all of the operating system except the kernel, is compiled. This is done first to provide up-to-date tools to build the kernel. Then the kernel itself is built: [source,shell] .... # cd /usr/src # make buildworld # make buildkernel .... The compiled code is written to [.filename]#/usr/obj#. These are the basic steps. Additional options to control the build are described below. [[updating-src-building-clean-build]] ==== Performing a Clean Build Some versions of the FreeBSD build system leave previously-compiled code in the temporary object directory, [.filename]#/usr/obj#. This can speed up later builds by avoiding recompiling code that has not changed. To force a clean rebuild of everything, use `cleanworld` before starting a build: [source,shell] .... # make cleanworld .... [[updating-src-building-jobs]] ==== Setting the Number of Jobs Increasing the number of build jobs on multi-core processors can improve build speed. Determine the number of cores with `sysctl hw.ncpu`. Processors vary, as do the build systems used with different versions of FreeBSD, so testing is the only sure method to tell how a different number of jobs affects the build speed. For a starting point, consider values between half and double the number of cores. The number of jobs is specified with `-j`. [[updating-src-building-jobs-example]] .Increasing the Number of Build Jobs [example] ==== Building the world and kernel with four jobs: [source,shell] .... # make -j4 buildworld buildkernel .... ==== [[updating-src-building-only-kernel]] ==== Building Only the Kernel A `buildworld` must be completed if the source code has changed. After that, a `buildkernel` to build a kernel can be run at any time. To build just the kernel: [source,shell] .... # cd /usr/src # make buildkernel .... [[updating-src-building-custom-kernel]] ==== Building a Custom Kernel The standard FreeBSD kernel is based on a _kernel config file_ called [.filename]#GENERIC#. The [.filename]#GENERIC# kernel includes the most commonly-needed device drivers and options. Sometimes it is useful or necessary to build a custom kernel, adding or removing device drivers or options to fit a specific need. For example, someone developing a small embedded computer with severely limited RAM could remove unneeded device drivers or options to make the kernel slightly smaller. Kernel config files are located in [.filename]#/usr/src/sys/arch/conf/#, where _arch_ is the output from `uname -m`. On most computers, that is `amd64`, giving a config file directory of [.filename]#/usr/src/sys/amd64/conf/#. [TIP] ==== [.filename]#/usr/src# can be deleted or recreated, so it is preferable to keep custom kernel config files in a separate directory, like [.filename]#/root#. Link the kernel config file into the [.filename]#conf# directory. If that directory is deleted or overwritten, the kernel config can be re-linked into the new one. ==== A custom config file can be created by copying the [.filename]#GENERIC# config file. In this example, the new custom kernel is for a storage server, so is named [.filename]#STORAGESERVER#: [source,shell] .... # cp /usr/src/sys/amd64/conf/GENERIC /root/STORAGESERVER # cd /usr/src/sys/amd64/conf # ln -s /root/STORAGESERVER . .... [.filename]#/root/STORAGESERVER# is then edited, adding or removing devices or options as shown in man:config[5]. The custom kernel is built by setting `KERNCONF` to the kernel config file on the command line: [source,shell] .... # make buildkernel KERNCONF=STORAGESERVER .... [[updating-src-installing]] === Installing the Compiled Code After the `buildworld` and `buildkernel` steps have been completed, the new kernel and world are installed: [source,shell] .... # cd /usr/src # make installkernel # shutdown -r now # cd /usr/src # make installworld # shutdown -r now .... If a custom kernel was built, `KERNCONF` must also be set to use the new custom kernel: [source,shell] .... # cd /usr/src # make installkernel KERNCONF=STORAGESERVER # shutdown -r now # cd /usr/src # make installworld # shutdown -r now .... [[updating-src-completing]] === Completing the Update A few final tasks complete the update. Any modified configuration files are merged with the new versions, outdated libraries are located and removed, then the system is restarted. [[updating-src-completing-merge-etcupdate]] ==== Merging Configuration Files with man:etcupdate[8] man:etcupdate[8] is a tool for managing updates to files that are not updated as part of an installworld such as files located in [.filename]#/etc/#. It manages updates by doing a three-way merge of changes made to these files against the local versions. It is also designed to minimize the amount of user intervention, in contrast to man:mergemaster[8]'s interactive prompts. [NOTE] ==== In general, man:etcupdate[8] does not need any specific arguments for its job. There is however a handy in between command for sanity checking what will be done the first time man:etcupdate[8] is used: [source,shell] .... # etcupdate diff .... This command allows the user to audit configuration changes. ==== If man:etcupdate[8] is not able to merge a file automatically, the merge conflicts can be resolved with manual interaction by issuing: [source,shell] .... # etcupdate resolve .... [WARNING] ==== When switching from man:mergemaster[8] to man:etcupdate[8], the first run might merge changes incorrectly generating spurious conflicts. To prevent this, perform the following steps *before* updating sources and building the new world: [source,shell] .... # etcupdate bootstrap <.> # etcupdate diff <.> .... <.> Bootstrap the database of stock [.filename]#/etc# files, for more information see man:etcupdate[8]. <.> Check the diff after bootstrapping. Trim any local changes that are no longer needed to reduce the chance of conflicts in future updates. ==== [[updating-src-completing-merge-mergemaster]] ==== Merging Configuration Files with man:mergemaster[8] man:mergemaster[8] provides a way to merge changes that have been made to system configuration files with new versions of those files. man:mergemaster[8] is an alternative to the preferred man:etcupdate[8] With `-Ui`, man:mergemaster[8] automatically updates files that have not been user-modified and installs new files that are not already present: [source,shell] .... # mergemaster -Ui .... If a file must be manually merged, an interactive display allows the user to choose which portions of the files are kept. See man:mergemaster[8] for more information. [[updating-src-completing-check-old]] ==== Checking for Outdated Files and Libraries Some obsolete files or directories can remain after an update. These files can be located: [source,shell] .... # make check-old .... and deleted: [source,shell] .... # make delete-old .... Some obsolete libraries can also remain. These can be detected with: [source,shell] .... # make check-old-libs .... and deleted with [source,shell] .... # make delete-old-libs .... Programs which were still using those old libraries will stop working when the library has been deleted. These programs must be rebuilt or replaced after deleting the old libraries. [TIP] ==== When all the old files or directories are known to be safe to delete, pressing kbd:[y] and kbd:[Enter] to delete each file can be avoided by setting `BATCH_DELETE_OLD_FILES` in the command. For example: [source,shell] .... # make BATCH_DELETE_OLD_FILES=yes delete-old-libs .... ==== [[updating-src-completing-restart]] ==== Restarting After the Update The last step after updating is to restart the computer so all the changes take effect: [source,shell] .... # shutdown -r now .... [[small-lan]] == Tracking for Multiple Machines When multiple machines need to track the same source tree, it is a waste of disk space, network bandwidth, and CPU cycles to have each system download the sources and rebuild everything. The solution is to have one machine do most of the work, while the rest of the machines mount that work via NFS. This section outlines a method of doing so. For more information about using NFS, refer to crossref:network-servers[network-nfs,"Network File System (NFS)"]. First, identify a set of machines which will run the same set of binaries, known as a _build set_. Each machine can have a custom kernel, but will run the same userland binaries. From that set, choose a machine to be the _build machine_ that the world and kernel are built on. Ideally, this is a fast machine that has sufficient spare CPU to run `make buildworld` and `make buildkernel`. Select a machine to be the _test machine_, which will test software updates before they are put into production. This _must_ be a machine that can afford to be down for an extended period of time. It can be the build machine, but need not be. All the machines in this build set need to mount [.filename]#/usr/obj# and [.filename]#/usr/src# from the build machine via NFS. For multiple build sets, [.filename]#/usr/src# should be on one build machine, and NFS mounted on the rest. Ensure that [.filename]#/etc/make.conf# and [.filename]#/etc/src.conf# on all the machines in the build set agree with the build machine. That means that the build machine must build all the parts of the base system that any machine in the build set is going to install. Also, each build machine should have its kernel name set with `KERNCONF` in [.filename]#/etc/make.conf#, and the build machine should list them all in its `KERNCONF`, listing its own kernel first. The build machine must have the kernel configuration files for each machine in its [.filename]#/usr/src/sys/arch/conf#. On the build machine, build the kernel and world as described in <>, but do not install anything on the build machine. Instead, install the built kernel on the test machine. On the test machine, mount [.filename]#/usr/src# and [.filename]#/usr/obj# via NFS. Then, run `shutdown now` to go to single-user mode in order to install the new kernel and world and run `mergemaster` as usual. When done, reboot to return to normal multi-user operations. After verifying that everything on the test machine is working properly, use the same procedure to install the new software on each of the other machines in the build set. The same methodology can be used for the ports tree. The first step is to share [.filename]#/usr/ports# via NFS to all the machines in the build set. To configure [.filename]#/etc/make.conf# to share distfiles, set `DISTDIR` to a common shared directory that is writable by whichever user `root` is mapped to by the NFS mount. Each machine should set `WRKDIRPREFIX` to a local build directory, if ports are to be built locally. Alternately, if the build system is to build and distribute packages to the machines in the build set, set `PACKAGES` on the build system to a directory similar to `DISTDIR`. diff --git a/documentation/content/en/books/handbook/mirrors/_index.adoc b/documentation/content/en/books/handbook/mirrors/_index.adoc index dc6d122428..f0be132b3c 100644 --- a/documentation/content/en/books/handbook/mirrors/_index.adoc +++ b/documentation/content/en/books/handbook/mirrors/_index.adoc @@ -1,927 +1,928 @@ --- title: Appendix A. Obtaining FreeBSD part: Part V. Appendices prev: books/handbook/partv next: books/handbook/bibliography --- [appendix] [[mirrors]] = Obtaining FreeBSD :doctype: book :icons: font :sectnums: :sectnumlevels: 6 :source-highlighter: rouge :experimental: :skip-front-matter: :xrefstyle: basic :relfileprefix: ../ :outfilesuffix: :sectnumoffset: A include::shared/mirrors.adoc[] include::shared/authors.adoc[] include::shared/releases.adoc[] include::shared/en/mailing-lists.adoc[] include::shared/en/teams.adoc[] include::shared/en/urls.adoc[] [[mirrors-cdrom]] == CD and DVD Sets FreeBSD CD and DVD sets are available from several online retailers: * FreeBSD Mall, Inc. + 2420 Sand Creek Rd C-1 #347 + Brentwood, CA + 94513 + USA + Phone: +1 925 240-6652 + Fax: +1 925 674-0821 + Email: + WWW: https://www.freebsdmall.com * Getlinux + 78 Rue de la Croix Rochopt + Épinay-sous-Sénart + 91860 + France + Email: + WWW: http://www.getlinux.fr/ * Dr. Hinner EDV + Kochelseestr. 11 + D-81371 München + Germany + Phone: (0177) 428 419 0 + Email: + WWW: http://www.hinner.de/linux/freebsd.html * Linux Center + Galernaya Street, 55 + Saint-Petersburg + 190000 + Russia + Phone: +7-812-309-06-86 + Email: + WWW: http://linuxcenter.ru/shop/freebsd [[mirrors-ftp]] == FTP Sites The official sources for FreeBSD are available via anonymous FTP from a worldwide set of mirror sites. The site link:ftp://ftp.FreeBSD.org/pub/FreeBSD/[ftp://ftp.FreeBSD.org/pub/FreeBSD/] is available via HTTP and FTP. It is made up of many machines operated by the project cluster administrators and behind GeoDNS to direct users to the closest available mirror. Additionally, FreeBSD is available via anonymous FTP from the following mirror sites. When obtaining FreeBSD via anonymous FTP, please try to use a nearby site. The mirror sites listed as "Primary Mirror Sites" typically have the entire FreeBSD archive (all the currently available versions for each of the architectures) but faster download speeds are probably available from a site that is in your country or region. The regional sites carry the most recent versions for the most popular architecture(s) but might not carry the entire FreeBSD archive. All sites provide access via anonymous FTP but some sites also provide access via other methods. The access methods available for each site are provided in parentheses after the hostname. <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>, <>. (as of UTC) [[central]] *{central}* {central-ftp} (ftp / ftpv6 / {central-http} / {central-httpv6}) [[primary]] *{mirrors-primary}* In case of problems, please contact the hostmaster `<{mirrors-primary-email}>` for this domain. * {mirrors-primary-ftp1} (ftp) * {mirrors-primary-ftp2} (ftp) * {mirrors-primary-ftp3} (ftp) * {mirrors-primary-ftp4} (ftp / ftpv6 / {mirrors-primary-ftp4-http} / {mirrors-primary-ftp4-httpv6}) * {mirrors-primary-ftp5} (ftp) * {mirrors-primary-ftp6} (ftp) * {mirrors-primary-ftp7} (ftp) * {mirrors-primary-ftp10} (ftp / ftpv6 / {mirrors-primary-ftp10-http} / {mirrors-primary-ftp10-httpv6}) * {mirrors-primary-ftp11} (ftp) * {mirrors-primary-ftp13} (ftp) * {mirrors-primary-ftp14} (ftp / {mirrors-primary-ftp14-http}) [[armenia]] *{mirrors-armenia}* In case of problems, please contact the hostmaster `<{mirrors-armenia-email}>` for this domain. * {mirrors-armenia-ftp} (ftp / {mirrors-armenia-ftp-http} / rsync) [[australia]] *{mirrors-australia}* In case of problems, please contact the hostmaster `<{mirrors-australia-email}>` for this domain. * {mirrors-australia-ftp} (ftp) * {mirrors-australia-ftp2} (ftp) * {mirrors-australia-ftp3} (ftp) [[austria]] *{mirrors-austria}* In case of problems, please contact the hostmaster `<{mirrors-austria-email}>` for this domain. * {mirrors-austria-ftp} (ftp / ftpv6 / {mirrors-austria-ftp-http} / {mirrors-austria-ftp-httpv6}) [[brazil]] *{mirrors-brazil}* In case of problems, please contact the hostmaster `<{mirrors-brazil-email}>` for this domain. * {mirrors-brazil-ftp2} (ftp / {mirrors-brazil-ftp2-http}) * {mirrors-brazil-ftp3} (ftp / rsync) * {mirrors-brazil-ftp4} (ftp) [[czech-republic]] *{mirrors-czech}* In case of problems, please contact the hostmaster `<{mirrors-czech-email}>` for this domain. * {mirrors-czech-ftp} (ftp / {mirrors-czech-ftpv6} / {mirrors-czech-ftp-http} / {mirrors-czech-ftp-httpv6} / rsync / rsyncv6) * {mirrors-czech-ftp2} (ftp / {mirrors-czech-ftp2-http}) [[denmark]] *{mirrors-denmark}* In case of problems, please contact the hostmaster `<{mirrors-denmark-email}>` for this domain. * {mirrors-denmark-ftp} (ftp / ftpv6 / {mirrors-denmark-ftp-http} / {mirrors-denmark-ftp-httpv6}) [[estonia]] *{mirrors-estonia}* In case of problems, please contact the hostmaster `<{mirrors-estonia-email}>` for this domain. * {mirrors-estonia-ftp} (ftp) [[finland]] *{mirrors-finland}* In case of problems, please contact the hostmaster `<{mirrors-finland-email}>` for this domain. * {mirrors-finland-ftp} (ftp) [[france]] *{mirrors-france}* In case of problems, please contact the hostmaster `<{mirrors-france-email}>` for this domain. * {mirrors-france-ftp} (ftp) * {mirrors-france-ftp1} (ftp / {mirrors-france-ftp1-http} / rsync) * {mirrors-france-ftp3} (ftp) * {mirrors-france-ftp5} (ftp) * {mirrors-france-ftp6} (ftp / rsync) * {mirrors-france-ftp7} (ftp) * {mirrors-france-ftp8} (ftp) [[germany]] *{mirrors-germany}* In case of problems, please contact the hostmaster `<{mirrors-germany-email}>` for this domain. * ftp://ftp.de.FreeBSD.org/pub/FreeBSD/ (ftp) * ftp://ftp1.de.FreeBSD.org/freebsd/ (ftp / http://www1.de.FreeBSD.org/freebsd/ / rsync://rsync3.de.FreeBSD.org/freebsd/) * ftp://ftp2.de.FreeBSD.org/pub/FreeBSD/ (ftp / http://ftp2.de.FreeBSD.org/pub/FreeBSD/ / rsync) * ftp://ftp4.de.FreeBSD.org/FreeBSD/ (ftp / http://ftp4.de.FreeBSD.org/pub/FreeBSD/) * ftp://ftp5.de.FreeBSD.org/pub/FreeBSD/ (ftp) * ftp://ftp7.de.FreeBSD.org/pub/FreeBSD/ (ftp / http://ftp7.de.FreeBSD.org/pub/FreeBSD/) * ftp://ftp8.de.FreeBSD.org/pub/FreeBSD/ (ftp) [[greece]] *{mirrors-greece}* In case of problems, please contact the hostmaster `<{mirrors-greece-email}>` for this domain. * {mirrors-greece-ftp} (ftp) * {mirrors-greece-ftp2} (ftp) [[hong-kong]] *{mirrors-hongkong}* {mirrors-hongkong-ftp} (ftp) [[ireland]] *{mirrors-ireland}* In case of problems, please contact the hostmaster `<{mirrors-ireland-email}>` for this domain. * {mirrors-ireland-ftp} (ftp / rsync) [[japan]] *{mirrors-japan}* In case of problems, please contact the hostmaster `<{mirrors-japan-email}>` for this domain. * {mirrors-japan-ftp} (ftp) * {mirrors-japan-ftp2} (ftp) * {mirrors-japan-ftp3} (ftp) * {mirrors-japan-ftp4} (ftp) * {mirrors-japan-ftp5} (ftp) * {mirrors-japan-ftp6} (ftp) * {mirrors-japan-ftp7} (ftp) * {mirrors-japan-ftp8} (ftp) * {mirrors-japan-ftp9} (ftp) [[korea]] *{mirrors-korea}* In case of problems, please contact the hostmaster `<{mirrors-korea-email}>` for this domain. * {mirrors-korea-ftp} (ftp / rsync) * {mirrors-korea-ftp2} (ftp / {mirrors-korea-ftp2-http}) [[latvia]] *{mirrors-latvia}* In case of problems, please contact the hostmaster `<{mirrors-latvia-email}>` for this domain. * {mirrors-latvia-ftp} (ftp / {mirrors-latvia-ftp-http}) [[lithuania]] *{mirrors-lithuania}* In case of problems, please contact the hostmaster `<{mirrors-lithuania-email}>` for this domain. * {mirrors-lithuania-ftp} (ftp / {mirrors-lithuania-ftp-http}) [[netherlands]] *{mirrors-netherlands}* In case of problems, please contact the hostmaster `<{mirrors-netherlands-email}>` for this domain. * {mirrors-netherlands-ftp} (ftp / {mirrors-netherlands-ftp-http} / rsync) * {mirrors-netherlands-ftp2} (ftp) [[new-zealand]] *{mirrors-new-zealand}* * {mirrors-new-zealand-ftp} (ftp / {mirrors-new-zealand-ftp-http}) [[norway]] *{mirrors-norway}* In case of problems, please contact the hostmaster `<{mirrors-norway-email}>` for this domain. * {mirrors-norway-ftp} (ftp / rsync) [[poland]] *{mirrors-poland}* In case of problems, please contact the hostmaster `<{mirrors-poland-email}>` for this domain. * {mirrors-poland-ftp} (ftp) * ftp2.pl.FreeBSD.org [[russia]] *{mirrors-russia}* In case of problems, please contact the hostmaster `<{mirrors-russia-email}>` for this domain. * {mirrors-russia-ftp} (ftp / {mirrors-russia-ftp-http} / rsync) * {mirrors-russia-ftp2} (ftp / {mirrors-russia-ftp2-http} / rsync) * {mirrors-russia-ftp4} (ftp) * {mirrors-russia-ftp5} (ftp / {mirrors-russia-ftp5-http} / rsync) * {mirrors-russia-ftp6} (ftp) [[saudi-arabia]] *{mirrors-saudi-arabia}* In case of problems, please contact the hostmaster `<{mirrors-saudi-arabia-email}>` for this domain. * {mirrors-saudi-arabia-ftp} (ftp) [[slovenia]] *{mirrors-slovenia}* In case of problems, please contact the hostmaster `<{mirrors-slovenia-email}>` for this domain. * {mirrors-slovenia-ftp} (ftp) [[south-africa]] *{mirrors-south-africa}* In case of problems, please contact the hostmaster `<{mirrors-south-africa-email}>` for this domain. * {mirrors-south-africa-ftp} (ftp) * {mirrors-south-africa-ftp2} (ftp) * {mirrors-south-africa-ftp4} (ftp) [[spain]] *{mirrors-spain}* In case of problems, please contact the hostmaster `<{mirrors-spain-email}>` for this domain. * {mirrors-spain-ftp} (ftp / {mirrors-spain-ftp-http}) * {mirrors-spain-ftp3} (ftp) [[sweden]] *{mirrors-sweden}* In case of problems, please contact the hostmaster `<{mirrors-sweden-email}>` for this domain. * {mirrors-sweden-ftp} (ftp) * {mirrors-sweden-ftp2} (ftp / {mirrors-sweden-ftp2-rsync}) * {mirrors-sweden-ftp3} (ftp) * {mirrors-sweden-ftp4} (ftp / {mirrors-sweden-ftp4v6} / {mirrors-sweden-ftp4-http} / {mirrors-sweden-ftp4-httpv6} / {mirrors-sweden-ftp4-rsync} / {mirrors-sweden-ftp4-rsyncv6}) * {mirrors-sweden-ftp6} (ftp / {mirrors-sweden-ftp6-http}) [[switzerland]] *{mirrors-switzerland}* In case of problems, please contact the hostmaster `<{mirrors-switzerland-email}>` for this domain. * {mirrors-switzerland-ftp} (ftp / {mirrors-switzerland-ftp-http}) [[taiwan]] *{mirrors-taiwan}* In case of problems, please contact the hostmaster `<{mirrors-taiwan-email}>` for this domain. * {mirrors-taiwan-ftp} (ftp / {mirrors-taiwan-ftpv6} / rsync / rsyncv6) * {mirrors-taiwan-ftp2} (ftp / {mirrors-taiwan-ftp2v6} / {mirrors-taiwan-ftp2-http} / {mirrors-taiwan-ftp2-httpv6} / rsync / rsyncv6) * {mirrors-taiwan-ftp4} (ftp) * {mirrors-taiwan-ftp5} (ftp) * {mirrors-taiwan-ftp6} (ftp / {mirrors-taiwan-ftp6v6} / rsync) * {mirrors-taiwan-ftp7} (ftp) * {mirrors-taiwan-ftp8} (ftp) * {mirrors-taiwan-ftp11} (ftp / {mirrors-taiwan-ftp11-http}) * {mirrors-taiwan-ftp12} (ftp) * {mirrors-taiwan-ftp13} (ftp) * {mirrors-taiwan-ftp14} (ftp) * {mirrors-taiwan-ftp15} (ftp) [[ukraine]] *{mirrors-ukraine}* * {mirrors-ukraine-ftp} (ftp / {mirrors-ukraine-ftp-http}) * {mirrors-ukraine-ftp6} (ftp / {mirrors-ukraine-ftp6-http} / {mirrors-ukraine-ftp6-rsync}) * {mirrors-ukraine-ftp7} (ftp) [[uk]] *{mirrors-uk}* In case of problems, please contact the hostmaster `<{mirrors-uk-email}>` for this domain. * {mirrors-uk-ftp} (ftp) * {mirrors-uk-ftp2} (ftp / {mirrors-uk-ftp2-rsync}) * {mirrors-uk-ftp3} (ftp) * {mirrors-uk-ftp4} (ftp) * {mirrors-uk-ftp5} (ftp) [[usa]] *{mirrors-us}* In case of problems, please contact the hostmaster `<{mirrors-us-email}>` for this domain. * {mirrors-us-ftp} (ftp) * {mirrors-us-ftp2} (ftp) * {mirrors-us-ftp3} (ftp) * {mirrors-us-ftp4} (ftp / ftpv6 / {mirrors-us-ftp4-http} / {mirrors-us-ftp4-httpv6}) * {mirrors-us-ftp5} (ftp) * {mirrors-us-ftp6} (ftp) * {mirrors-us-ftp8} (ftp) * {mirrors-us-ftp10} (ftp) * {mirrors-us-ftp11} (ftp) * {mirrors-us-ftp13} (ftp / {mirrors-us-ftp13-http} / rsync) * {mirrors-us-ftp14} (ftp / {mirrors-us-ftp14-http}) * {mirrors-us-ftp15} (ftp) [[git]] == Using Git [[git-intro]] === Introduction As of December 2020, FreeBSD uses git as the primary version control system for storing all of FreeBSD's base source code and documentation. [NOTE] ==== Git is generally a developer tool. Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `portsnap` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection. In April 2021, the ports tree will migrate to git. ==== This section demonstrates how to install Git on a FreeBSD system and use it to create a local copy of a FreeBSD repository. Additional information on the use of Git is included. [[git-ssl-certificates]] === Root SSL Certificates Older FreeBSD systems do not have proper root certificates. Installing package:security/ca_root_nss[] on these systems allows Git to verify the identity of HTTPS repository servers. The root SSL certificates can be installed from a port: [source,shell] .... # cd /usr/ports/security/ca_root_nss # make install clean .... or as a package: [source,shell] .... # pkg install ca_root_nss .... [[git-install]] === Installation -Git can be installed from the Ports Collection: +Git can be installed as a package: [source,shell] .... -# cd /usr/ports/devel/git -# make install clean +# pkg install git .... -Git can also be installed as a package: +Git can also be installed from the Ports Collection: [source,shell] .... -# pkg install git +# cd /usr/ports/devel/git +# make install clean .... [[git-usage]] === Running Git To fetch a clean copy of the sources into a local directory, use `git`. This directory of files is called the _working tree_. [WARNING] ==== Move or delete an existing destination directory before using `git clone` for the first time. Cloning over an existing non-git directory will fail. ==== Git uses URLs to designate a repository, taking the form of _protocol://hostname/path_. The first component of the path is the FreeBSD repository to access. There are two different repositories, `src` for the FreeBSD systerm source code, and `doc` for documentation. In the future a third repository, 'ports' will be available for the FreeBSD ports code. For example, the URL `https://git.FreeBSD.org/src.git` specifies the main branch of the src repository, using the `https` protocol. .Handy URL Table [options="header,foooter"] |======================================================= |Item | Git URL | Web-based repository browser src | `https://cgit.freebsd.org/src` | Read-only src repo via HTTPS | `https://git.freebsd.org/src.git` | Read-only src repo via anon-ssh | `ssh://anongit@git.freebsd.org/src.git` | Read/write src repo for committers | `ssh://git@gitrepo.freebsd.org/src.git` (*) | Web-based repository browser doc | `https://cgit.freebsd.org/doc` | Read-only doc repo via HTTPS | `https://git.freebsd.org/doc.git` | Read-only doc repo via anon-ssh | `ssh://anongit@git.freebsd.org/doc.git` | Read/write doc repo for committers | `ssh://git@gitrepo.freebsd.org/doc.git` (*) |======================================================= - (*) `git` is a special user on the repository server which will map your registered ssh key in FreeBSD.org to your identity, no need to change it. [WARNING] ==== Sometime after the switch to git is complete, `gitrepo.freebsd.org` will change to simply `repo.freebsd.org`. ==== To get started, clone a copy of the FreeBSD repository: [source,shell] .... # git clone [ -b branch ] https://git.FreeBSD.org/repo.git wcdir .... where: * _repo_ is one of the Project repositories: `src`, `ports`, or `doc`. * _branch_ depends on the repository used. `ports` and `doc` are mostly updated in the `main` branch, while `src` maintains the latest version of -CURRENT under `main` and the respective latest versions of the -STABLE branches under `stable/12` (12._x_) and `stable/13` (13._x_). * _wcdir_ is the target directory where the contents of the specified branch should be placed. This is usually [.filename]#/usr/ports# for `ports`, [.filename]#/usr/src# for `src`, and [.filename]#/usr/doc# for `doc`. This example checks out the `main` branch of the system sources from the FreeBSD repository using the HTTPS protocol, placing the local working copy in [.filename]#/usr/src#. If [.filename]#/usr/src# is already present but was not created by `git`, remember to rename or delete it before the checkout. Git will refuse to do anything otherwise. [source,shell] .... # git clone https://git.FreeBSD.org/src.git /usr/src .... Because the initial checkout must download the full branch of the remote repository, it can take a while. Please be patient. After the initial checkout, the local working copy can be updated by running: [source,shell] .... # cd wcdir # git pull --rebase .... To update [.filename]#/usr/src# created in the example above, use: [source,shell] .... # cd /usr/src # git pull --rebase .... The update is much quicker than a checkout, only transferring files that have changed. There are also external mirrors maintained by project members available, please refer to the "External mirrors" section. === SSH related information - `ssh://${user}@${url}/${repo}.git` can be written as `${user}@${url}:${repo}.git`, i.e., following two URLs are both valid for passing to git: - `ssh://anongit@git.freebsd.org/${repo}.git` - `anongit@git.freebsd.org:${repo}.git` As well as the read-write repo: - `ssh://git@(git)repo.freebsd.org/${repo}.git` - `git@(git)repo.freebsd.org:${repo}.git` - gitrepo.FreeBSD.org host key fingerprints: - ECDSA key fingerprint is `SHA256:seWO5D27ySURcx4bknTNKlC1mgai0whP443PAKEvvZA` - ED25519 key fingerprint is `SHA256:lNR6i4BEOaaUhmDHBA1WJsO7H3KtvjE2r5q4sOxtIWo` - RSA key fingerprint is `SHA256:f453CUEFXEJAXlKeEHV+ajJfeEfx9MdKQUD7lIscnQI` - git.FreeBSD.org host key fingerprints: - ECDSA key fingerprint is `SHA256:/UlirUAsGiitupxmtsn7f9b7zCWd0vCs4Yo/tpVWP9w` - ED25519 key fingerprint is `SHA256:y1ljKrKMD3lDObRUG3xJ9gXwEIuqnh306tSyFd1tuZE` - RSA key fingerprint is `SHA256:jBe6FQGoH4HjvrIVM23dcnLZk9kmpdezR/CvQzm7rJM` These are also published as SSHFP records in DNS. === Web-based repository browser The FreeBSD project currently uses cgit as the web-based repository browser: https://cgit.freebsd.org/ The URL of the indivirual repository is at: https://cgit.freebsd.org/${repo}/ === For Users Using `git clone` and `git pull` from the official distributed mirrors is recommended. The GeoDNS should direct you to the nearest mirror to you. === For Developers This section describes the read-write access for committers to push the commits from developers or contributors. For read-only access, please refer to the users section above. ==== Daily use * Clone the repository: [source,shell] .... % git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/${repo}.git .... Then you should have the official mirrors as your remote: [source,shell] .... % git remote -v freebsd https://git.freebsd.org/${repo}.git (fetch) freebsd https://git.freebsd.org/${repo}.git (push) .... * Configure the FreeBSD committer data: The commit hook in repo.freebsd.org checks the "Commit" field matches the committer's information in FreeBSD.org. The easiest way to get the suggested config is by executing `/usr/local/bin/gen-gitconfig.sh` script on freefall: [source,shell] .... % gen-gitconfig.sh [...] % git config user.name (your name in gecos) % git config user.email (your login)@FreeBSD.org .... * Set the push URL: [source,shell] .... % git remote set-url --push freebsd git@gitrepo.freebsd.org:${repo}.git .... Then you should have separated fetch and push URLs as the most efficient setup: [source,shell] .... % git remote -v freebsd https://git.freebsd.org/${repo}.git (fetch) freebsd git@gitrepo.freebsd.org:${repo}.git (push) .... Again, note that `gitrepo.freebsd.org` will be canonicalized to `repo.freebsd.org` in the future. * Install commit message template hook: [source,shell] .... % fetch https://cgit.freebsd.org/src/plain/tools/tools/git/hooks/prepare-commit-msg -o .git/hooks % chmod 755 .git/hooks/prepare-commit-msg .... ==== "admin" branch The `access` and `mentors` files are stored in an orphan branch, `internal/admin`, in each repository. Following example is how to check out the `internal/admin` branch to a local branch named `admin`: [source,shell] .... % git config --add remote.freebsd.fetch '+refs/internal/*:refs/internal/*' % git fetch % git checkout -b admin internal/admin .... Alternatively, you can add a worktree for the `admin` branch: [source,shell] .... git worktree add -b admin ../${repo}-admin internal/admin .... For browsing `internal/admin` branch on web: https://cgit.freebsd.org/${repo}/log/?h=internal/admin For pushing, either specify the full refspec: [source,shell] .... git push freebsd HEAD:refs/internal/admin .... Or set `push.default` to `upstream` which will make `git push` to push the current branch back to its upstream by default, which is more suitable for our workflow: [source,shell] .... git config push.default upstream .... [WARNING] ==== These internal details may change often. ==== === External mirrors Those mirrors are not hosted in FreeBSD.org but still maintained by the project members. Users and developers are welcome to pull or browse repositories on those mirrors. The project workflow with those mirrors are still under discussion. ==== Codeberg - doc: https://codeberg.org/FreeBSD/freebsd-doc - src: https://codeberg.org/FreeBSD/freebsd-src ==== GitHub - doc: https://github.com/freebsd/freebsd-doc - src: https://github.com/freebsd/freebsd-src ==== GitLab - doc: https://gitlab.com/FreeBSD/freebsd-doc - src: https://gitlab.com/FreeBSD/freebsd-src === Mailing lists General usage and questions about git in the FreeBSD project: [freebsd-git](https://lists.freebsd.org/mailman/listinfo/freebsd-git) Commit messages will be sent to the following mailing lists: - https://lists.freebsd.org/mailman/listinfo/dev-commits-doc-all[dev-commits-doc-all]: All changes to the doc repository - https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-all[dev-commits-ports-all]: All changes to the ports repository - https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-main[dev-commits-ports-main]: All changes to the "main" branch of the ports repository - https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-branches[dev-commits-ports-branches]: All changes to the quarterly branches of the ports repository - https://lists.freebsd.org/mailman/listinfo/dev-commits-src-all[dev-commits-src-all]: All changes to the src repository - https://lists.freebsd.org/mailman/listinfo/dev-commits-src-main[dev-commits-src-main]: All changes to the "main" branch of the src repository (the FreeBSD-CURRENT branch) - https://lists.freebsd.org/mailman/listinfo/dev-commits-src-branches[dev-commits-src-branches]: All changes to all stable branches of the src repository For more information, please refer to the "Commit message lists" section of C.2. "Mailing Lists" in handbook: https://www.freebsd.org/doc/en/books/handbook/eresources-mail.html [[svn]] == Using Subversion [[svn-intro]] === Introduction -As of July 2012, FreeBSD uses Subversion as the only version control system for storing all of FreeBSD's Ports Collection. As of December 2020, FreeBSD uses git as the primary version control system for storing all of FreeBSD's source code and documentation. Changes from the git repo on the `stable/11`, `stable/12` and related releng branches are exported to the subversion repository. This export will continue through the life of these branches. +From July 2012 to March 2021, FreeBSD used Subversion as the only version control system for storing all of FreeBSD's Ports Collection. +As of April 2021, FreeBSD uses git as the only version control system for storing all of FreeBSD's Ports Collection. [NOTE] ==== Subversion is generally a developer tool. Users may prefer to use `freebsd-update` (crossref:cutting-edge[updating-upgrading-freebsdupdate,“FreeBSD Update”]) to update the FreeBSD base system, and `portsnap` (crossref:ports[ports-using,“Using the Ports Collection”]) to update the FreeBSD Ports Collection. After March 2021, subversion use is only for legacy branches (`stable/11` and `stable/12`). ==== This section demonstrates how to install Subversion on a FreeBSD system and use it to create a local copy of a FreeBSD repository. Additional information on the use of Subversion is included. [[svn-ssl-certificates]] === Root SSL Certificates FreeBSD systems older than 12._x_ do not have proper root certificates. Installing package:security/ca_root_nss[] on these systems allows Subversion to verify the identity of HTTPS repository servers. The root SSL certificates can be installed from a port: [source,shell] .... # cd /usr/ports/security/ca_root_nss # make install clean .... or as a package: [source,shell] .... # pkg install ca_root_nss .... [[svn-svnlite]] === Svnlite A lightweight version of Subversion is already installed on FreeBSD as `svnlite`. The port or package version of Subversion is only needed if the Python or Perl API is needed, or if a later version of Subversion is desired. The only difference from normal Subversion use is that the command name is `svnlite`. [[svn-install]] === Installation If `svnlite` is unavailable or the full version of Subversion is needed, then it must be installed. Subversion can be installed from the Ports Collection: [source,shell] .... # cd /usr/ports/devel/subversion # make install clean .... Subversion can also be installed as a package: [source,shell] .... # pkg install subversion .... [[svn-usage]] === Running Subversion To fetch a clean copy of the sources into a local directory, use `svn`. The files in this directory are called a _local working copy_. [WARNING] ==== Move or delete an existing destination directory before using `checkout` for the first time. Checkout over an existing non-`svn` directory can cause conflicts between the existing files and those brought in from the repository. ==== -Subversion uses URLs to designate a repository, taking the form of _protocol://hostname/path_. The first component of the path is the FreeBSD repository to access. There are three different repositories, `base` for the FreeBSD base system source code, `ports` for the Ports Collection, and `doc` for documentation. For example, the URL `https://svn.FreeBSD.org/ports/head/` specifies the main branch of the ports repository, using the `https` protocol. +Subversion uses URLs to designate a repository, taking the form of _protocol://hostname/path_. The first component of the path is the FreeBSD repository to access. There are three different repositories, `base` for the FreeBSD base system source code, `ports` for the Ports Collection, and `doc` for documentation. For example, the URL `https://svn.FreeBSD.org/base/head/` specifies the main branch of the ports repository, using the `https` protocol. A checkout from a given repository is performed with a command like this: [source,shell] .... # svn checkout https://svn.FreeBSD.org/repository/branch lwcdir .... where: * _repository_ is one of the Project repositories: `base`, `ports`, or `doc`. * _branch_ depends on the repository used. `ports` and `doc` are mostly updated in the `head` branch, while `base` maintains the latest version of -CURRENT under `head` and the respective latest versions of the -STABLE branches under `stable/9` (9._x_) and `stable/10` (10._x_). * _lwcdir_ is the target directory where the contents of the specified branch should be placed. This is usually [.filename]#/usr/ports# for `ports`, [.filename]#/usr/src# for `base`, and [.filename]#/usr/doc# for `doc`. -This example checks out the Ports Collection from the FreeBSD repository using the HTTPS protocol, placing the local working copy in [.filename]#/usr/ports#. If [.filename]#/usr/ports# is already present but was not created by `svn`, remember to rename or delete it before the checkout. +This example checks out the Source Tree from the FreeBSD repository using the HTTPS protocol, placing the local working copy in [.filename]#/usr/src#. If [.filename]#/usr/src# is already present but was not created by `svn`, remember to rename or delete it before the checkout. [source,shell] .... -# svn checkout https://svn.FreeBSD.org/ports/head /usr/ports +# svn checkout https://svn.FreeBSD.org/base/head /usr/src .... Because the initial checkout must download the full branch of the remote repository, it can take a while. Please be patient. After the initial checkout, the local working copy can be updated by running: [source,shell] .... # svn update lwcdir .... To update [.filename]#/usr/ports# created in the example above, use: [source,shell] .... -# svn update /usr/ports +# svn update /usr/src .... The update is much quicker than a checkout, only transferring files that have changed. An alternate way of updating the local working copy after checkout is provided by the [.filename]#Makefile# in the [.filename]#/usr/ports#, [.filename]#/usr/src#, and [.filename]#/usr/doc# directories. Set `SVN_UPDATE` and use the `update` target. For example, to update [.filename]#/usr/src#: [source,shell] .... # cd /usr/src # make update SVN_UPDATE=yes .... [[git-mirrors]] === Git Mirror Sites The FreeBSD Git repository is: [.programlisting] .... https://git.freebsd.org/ .... This is a publicly accessible mirror network that uses GeoDNS to select an appropriate back end server. To view the FreeBSD Subversion repositories through a browser, use https://cgit.freebsd.org[https://cgit.freebsd.org]. [[svn-mirrors]] === Subversion Mirror Sites The FreeBSD Subversion repository is: [.programlisting] .... svn.FreeBSD.org .... This is a publicly accessible mirror network that uses GeoDNS to select an appropriate back end server. To view the FreeBSD Subversion repositories through a browser, use https://svnweb.FreeBSD.org/[https://svnweb.FreeBSD.org/]. HTTPS is the preferred protocol, but the [.filename]#security/ca_root_nss# package will need to be installed in order to automatically validate certificates. === For More Information For other information about using Subversion, please see the "Subversion Book", titled http://svnbook.red-bean.com/[Version Control with Subversion], or the http://subversion.apache.org/docs/[Subversion Documentation]. [[mirrors-rsync]] == Using rsync These sites make FreeBSD available through the rsync protocol. The rsync utility transfers only the differences between two sets of files. This is useful for mirror sites of the FreeBSD FTP server. The rsync suite is available for many operating systems, on FreeBSD, see the package:net/rsync[] port or use the package. Czech Republic:: rsync://ftp.cz.FreeBSD.org/ + Available collections: ** ftp: A partial mirror of the FreeBSD FTP server. ** FreeBSD: A full mirror of the FreeBSD FTP server. Netherlands:: rsync://ftp.nl.FreeBSD.org/ + Available collections: ** FreeBSD: A full mirror of the FreeBSD FTP server. Russia:: rsync://ftp.mtu.ru/ + Available collections: ** FreeBSD: A full mirror of the FreeBSD FTP server. ** FreeBSD-Archive: The mirror of FreeBSD Archive FTP server. Sweden:: rsync://ftp4.se.freebsd.org/ + Available collections: ** FreeBSD: A full mirror of the FreeBSD FTP server. Taiwan:: rsync://ftp.tw.FreeBSD.org/ + rsync://ftp2.tw.FreeBSD.org/ + rsync://ftp6.tw.FreeBSD.org/ + Available collections: ** FreeBSD: A full mirror of the FreeBSD FTP server. United Kingdom:: rsync://rsync.mirrorservice.org/ + Available collections: ** ftp.freebsd.org: A full mirror of the FreeBSD FTP server. United States of America:: rsync://ftp-master.FreeBSD.org/ + This server may only be used by FreeBSD primary mirror sites. + Available collections: ** FreeBSD: The master archive of the FreeBSD FTP server. ** acl: The FreeBSD master ACL list. + rsync://ftp13.FreeBSD.org/ + Available collections: ** FreeBSD: A full mirror of the FreeBSD FTP server. diff --git a/documentation/content/en/books/handbook/ports/_index.adoc b/documentation/content/en/books/handbook/ports/_index.adoc index 2da3bfaabc..7df5c7c6d4 100644 --- a/documentation/content/en/books/handbook/ports/_index.adoc +++ b/documentation/content/en/books/handbook/ports/_index.adoc @@ -1,1148 +1,1148 @@ --- title: "Chapter 4. Installing Applications: Packages and Ports" part: Part I. Getting Started prev: books/handbook/basics next: books/handbook/x11 --- [[ports]] = Installing Applications: Packages and Ports :doctype: book :toc: macro :toclevels: 1 :icons: font :sectnums: :sectnumlevels: 6 :source-highlighter: rouge :experimental: :skip-front-matter: :xrefstyle: basic :relfileprefix: ../ :outfilesuffix: :sectnumoffset: 4 ifeval::["{backend}" == "html5"] :imagesdir: ../../../../images/books/handbook/ports/ endif::[] ifeval::["{backend}" == "pdf"] :imagesdir: ../../../../static/images/books/handbook/ports/ endif::[] ifeval::["{backend}" == "epub3"] :imagesdir: ../../../../static/images/books/handbook/ports/ endif::[] include::shared/authors.adoc[] include::shared/releases.adoc[] include::shared/en/mailing-lists.adoc[] include::shared/en/teams.adoc[] include::shared/en/urls.adoc[] toc::[] [[ports-synopsis]] == Synopsis FreeBSD is bundled with a rich collection of system tools as part of the base system. In addition, FreeBSD provides two complementary technologies for installing third-party software: the FreeBSD Ports Collection, for installing from source, and packages, for installing from pre-built binaries. Either method may be used to install software from local media or from the network. After reading this chapter, you will know: * The difference between binary packages and ports. * How to find third-party software that has been ported to FreeBSD. * How to manage binary packages using pkg. * How to build third-party software from source using the Ports Collection. * How to find the files installed with the application for post-installation configuration. * What to do if a software installation fails. [[ports-overview]] == Overview of Software Installation The typical steps for installing third-party software on a UNIX(R) system include: [.procedure] . Find and download the software, which might be distributed in source code format or as a binary. . Unpack the software from its distribution format. This is typically a tarball compressed with a program such as man:compress[1], man:gzip[1], man:bzip2[1] or man:xz[1]. . Locate the documentation in [.filename]#INSTALL#, [.filename]#README# or some file in a [.filename]#doc/# subdirectory and read up on how to install the software. . If the software was distributed in source format, compile it. This may involve editing a [.filename]#Makefile# or running a `configure` script. . Test and install the software. A FreeBSD _port_ is a collection of files designed to automate the process of compiling an application from source code. The files that comprise a port contain all the necessary information to automatically download, extract, patch, compile, and install the application. If the software has not already been adapted and tested on FreeBSD, the source code might need editing in order for it to install and run properly. However, over link:https://www.FreeBSD.org/ports/[{numports}] third-party applications have already been ported to FreeBSD. When feasible, these applications are made available for download as pre-compiled _packages_. Packages can be manipulated with the FreeBSD package management commands. Both packages and ports understand dependencies. If a package or port is used to install an application and a dependent library is not already installed, the library will automatically be installed first. A FreeBSD package contains pre-compiled copies of all the commands for an application, as well as any configuration files and documentation. A package can be manipulated with the man:pkg[8] commands, such as `pkg install`. While the two technologies are similar, packages and ports each have their own strengths. Select the technology that meets your requirements for installing a particular application. .Package Benefits * A compressed package tarball is typically smaller than the compressed tarball containing the source code for the application. * Packages do not require compilation time. For large applications, such as Mozilla, KDE, or GNOME, this can be important on a slow system. * Packages do not require any understanding of the process involved in compiling software on FreeBSD. .Port Benefits * Packages are normally compiled with conservative options because they have to run on the maximum number of systems. By compiling from the port, one can change the compilation options. * Some applications have compile-time options relating to which features are installed. For example, Apache can be configured with a wide variety of different built-in options. + In some cases, multiple packages will exist for the same application to specify certain settings. For example, Ghostscript is available as a [.filename]#ghostscript# package and a [.filename]#ghostscript-nox11# package, depending on whether or not Xorg is installed. Creating multiple packages rapidly becomes impossible if an application has more than one or two different compile-time options. * The licensing conditions of some software forbid binary distribution. Such software must be distributed as source code which must be compiled by the end-user. * Some people do not trust binary distributions or prefer to read through source code in order to look for potential problems. * Source code is needed in order to apply custom patches. To keep track of updated ports, subscribe to the {freebsd-ports} and the {freebsd-ports-bugs}. [WARNING] ==== Before installing any application, check https://vuxml.freebsd.org/[] for security issues related to the application or type `pkg audit -F` to check all installed applications for known vulnerabilities. ==== The remainder of this chapter explains how to use packages and ports to install and manage third-party software on FreeBSD. [[ports-finding-applications]] == Finding Software FreeBSD's list of available applications is growing all the time. There are a number of ways to find software to install: * The FreeBSD web site maintains an up-to-date searchable list of all the available applications, at link:https://www.FreeBSD.org/ports/[https://www.FreeBSD.org/ports/]. The ports can be searched by application name or by software category. * Dan Langille maintains http://www.FreshPorts.org/[FreshPorts.org] which provides a comprehensive search utility and also tracks changes to the applications in the Ports Collection. Registered users can create a customized watch list in order to receive an automated email when their watched ports are updated. * If finding a particular application becomes challenging, try searching a site like http://www.sourceforge.net/[SourceForge.net] or http://www.github.com/[GitHub.com] then check back at the link:https://www.FreeBSD.org/ports/[FreeBSD site] to see if the application has been ported. * To search the binary package repository for an application: + [source,shell] .... # pkg search subversion git-subversion-1.9.2 java-subversion-1.8.8_2 p5-subversion-1.8.8_2 py27-hgsubversion-1.6 py27-subversion-1.8.8_2 ruby-subversion-1.8.8_2 subversion-1.8.8_2 subversion-book-4515 subversion-static-1.8.8_2 subversion16-1.6.23_4 subversion17-1.7.16_2 .... + Package names include the version number and, in the case of ports based on python, the version number of the version of python the package was built with. Some ports also have multiple versions available. In the case of Subversion, there are different versions available, as well as different compile options. In this case, the statically linked version of Subversion. When indicating which package to install, it is best to specify the application by the port origin, which is the path in the ports tree. Repeat the `pkg search` with `-o` to list the origin of each package: + [source,shell] .... # pkg search -o subversion devel/git-subversion java/java-subversion devel/p5-subversion devel/py-hgsubversion devel/py-subversion devel/ruby-subversion devel/subversion16 devel/subversion17 devel/subversion devel/subversion-book devel/subversion-static .... + Searching by shell globs, regular expressions, exact match, by description, or any other field in the repository database is also supported by `pkg search`. After installing package:ports-mgmt/pkg[] or package:ports-mgmt/pkg-devel[], see man:pkg-search[8] for more details. * If the Ports Collection is already installed, there are several methods to query the local version of the ports tree. To find out which category a port is in, type `whereis _file_`, where _file_ is the program to be installed: + [source,shell] .... # whereis lsof lsof: /usr/ports/sysutils/lsof .... + Alternately, an man:echo[1] statement can be used: + [source,shell] .... # echo /usr/ports/*/*lsof* /usr/ports/sysutils/lsof .... + Note that this will also return any matched files downloaded into the [.filename]#/usr/ports/distfiles# directory. * Another way to find software is by using the Ports Collection's built-in search mechanism. To use the search feature, cd to [.filename]#/usr/ports# then run `make search name=program-name` where _program-name_ is the name of the software. For example, to search for `lsof`: + [source,shell] .... # cd /usr/ports # make search name=lsof Port: lsof-4.88.d,8 Path: /usr/ports/sysutils/lsof Info: Lists information about open files (similar to fstat(1)) Maint: ler@lerctr.org Index: sysutils B-deps: R-deps: .... + [TIP] ==== The built-in search mechanism uses a file of index information. If a message indicates that the [.filename]#INDEX# is required, run `make fetchindex` to download the current index file. With the [.filename]#INDEX# present, `make search` will be able to perform the requested search. ==== + The "Path:" line indicates where to find the port. + To receive less information, use the `quicksearch` feature: + [source,shell] .... # cd /usr/ports # make quicksearch name=lsof Port: lsof-4.88.d,8 Path: /usr/ports/sysutils/lsof Info: Lists information about open files (similar to fstat(1)) .... + For more in-depth searching, use `make search key=_string_` or `make quicksearch key=_string_`, where _string_ is some text to search for. The text can be in comments, descriptions, or dependencies in order to find ports which relate to a particular subject when the name of the program is unknown. + When using `search` or `quicksearch`, the search string is case-insensitive. Searching for "LSOF" will yield the same results as searching for "lsof". [[pkgng-intro]] == Using pkg for Binary Package Management pkg is the next generation replacement for the traditional FreeBSD package management tools, offering many features that make dealing with binary packages faster and easier. For sites wishing to only use prebuilt binary packages from the FreeBSD mirrors, managing packages with pkg can be sufficient. However, for those sites building from source or using their own repositories, a separate <> will be needed. Since pkg only works with binary packages, it is not a replacement for such tools. Those tools can be used to install software from both binary packages and the Ports Collection, while pkg installs only binary packages. [[pkgng-initial-setup]] === Getting Started with pkg FreeBSD includes a bootstrap utility which can be used to download and install pkg and its manual pages. This utility is designed to work with versions of FreeBSD starting with 10._X_. [NOTE] ==== Not all FreeBSD versions and architectures support this bootstrap process. The current list is at https://pkg.freebsd.org/[]. For other cases, pkg must instead be installed from the Ports Collection or as a binary package. ==== To bootstrap the system, run: [source,shell] .... # /usr/sbin/pkg .... You must have a working Internet connection for the bootstrap process to succeed. Otherwise, to install the port, run: [source,shell] .... # cd /usr/ports/ports-mgmt/pkg # make # make install clean .... When upgrading an existing system that originally used the older pkg_* tools, the database must be converted to the new format, so that the new tools are aware of the already installed packages. Once pkg has been installed, the package database must be converted from the traditional format to the new format by running this command: [source,shell] .... # pkg2ng .... [NOTE] ==== This step is not required for new installations that do not yet have any third-party software installed. ==== [IMPORTANT] ==== This step is not reversible. Once the package database has been converted to the pkg format, the traditional `pkg_*` tools should no longer be used. ==== [NOTE] ==== The package database conversion may emit errors as the contents are converted to the new version. Generally, these errors can be safely ignored. However, a list of software that was not successfully converted is shown after `pkg2ng` finishes. These applications must be manually reinstalled. ==== To ensure that the Ports Collection registers new software with pkg instead of the traditional packages database, FreeBSD versions earlier than 10._X_ require this line in [.filename]#/etc/make.conf#: [.programlisting] .... WITH_PKGNG= yes .... By default, pkg uses the binary packages from the FreeBSD package mirrors (the _repository_). For information about building a custom package repository, see <>. Additional pkg configuration options are described in man:pkg.conf[5]. Usage information for pkg is available in the man:pkg[8] manual page or by running `pkg` without additional arguments. Each pkg command argument is documented in a command-specific manual page. To read the manual page for `pkg install`, for example, run either of these commands: [source,shell] .... # pkg help install .... [source,shell] .... # man pkg-install .... The rest of this section demonstrates common binary package management tasks which can be performed using pkg. Each demonstrated command provides many switches to customize its use. Refer to a command's help or man page for details and more examples. [[quarterly-latest-branch]] === Quarterly and Latest Ports Branches The `Quarterly` branch provides users with a more predictable and stable experience for port and package installation and upgrades. This is done essentially by only allowing non-feature updates. Quarterly branches aim to receive security fixes (that may be version updates, or backports of commits), bug fixes and ports compliance or framework changes. The Quarterly branch is cut from HEAD at the beginning of every (yearly) quarter in January, April, July, and October. Branches are named according to the year (YYYY) and quarter (Q1-4) they are created in. For example, the quarterly branch created in January 2016, is named 2016Q1. And the `Latest` branch provides the latest versions of the packages to the users. To switch from quarterly to latest run the following commands: [source,shell] .... # mkdir -p /usr/local/etc/pkg/repos # cp /etc/pkg/FreeBSD.conf /usr/local/etc/pkg/repos/FreeBSD.conf .... Edit the file [.filename]#/usr/local/etc/pkg/repos/FreeBSD.conf# and change the string _quarterly_ to _latest_ in the `url:` line. The result should be similar to the following: [.programlisting] .... FreeBSD: { url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest", mirror_type: "srv", signature_type: "fingerprints", fingerprints: "/usr/share/keys/pkg", enabled: yes } .... And finally run this command to update from the new (latest) repository metadata. [source,shell] .... # pkg update -f .... [[pkgng-pkg-info]] === Obtaining Information About Installed Packages Information about the packages installed on a system can be viewed by running `pkg info` which, when run without any switches, will list the package version for either all installed packages or the specified package. For example, to see which version of pkg is installed, run: [source,shell] .... # pkg info pkg pkg-1.1.4_1 .... [[pkgng-installing-deinstalling]] === Installing and Removing Packages To install a binary package use the following command, where _packagename_ is the name of the package to install: [source,shell] .... # pkg install packagename .... This command uses repository data to determine which version of the software to install and if it has any uninstalled dependencies. For example, to install curl: [source,shell] .... # pkg install curl Updating repository catalogue /usr/local/tmp/All/curl-7.31.0_1.txz 100% of 1181 kB 1380 kBps 00m01s /usr/local/tmp/All/ca_root_nss-3.15.1_1.txz 100% of 288 kB 1700 kBps 00m00s Updating repository catalogue The following 2 packages will be installed: Installing ca_root_nss: 3.15.1_1 Installing curl: 7.31.0_1 The installation will require 3 MB more space 0 B to be downloaded Proceed with installing packages [y/N]: y Checking integrity... done [1/2] Installing ca_root_nss-3.15.1_1... done [2/2] Installing curl-7.31.0_1... done Cleaning up cache files...Done .... The new package and any additional packages that were installed as dependencies can be seen in the installed packages list: [source,shell] .... # pkg info ca_root_nss-3.15.1_1 The root certificate bundle from the Mozilla Project curl-7.31.0_1 Non-interactive tool to get files from FTP, GOPHER, HTTP(S) servers pkg-1.1.4_6 New generation package manager .... Packages that are no longer needed can be removed with `pkg delete`. For example: [source,shell] .... # pkg delete curl The following packages will be deleted: curl-7.31.0_1 The deletion will free 3 MB Proceed with deleting packages [y/N]: y [1/1] Deleting curl-7.31.0_1... done .... [[pkgng-upgrading]] === Upgrading Installed Packages Installed packages can be upgraded to their latest versions by running: [source,shell] .... # pkg upgrade .... This command will compare the installed versions with those available in the repository catalogue and upgrade them from the repository. [[pkgng-auditing]] === Auditing Installed Packages Software vulnerabilities are regularly discovered in third-party applications. To address this, pkg includes a built-in auditing mechanism. To determine if there are any known vulnerabilities for the software installed on the system, run: [source,shell] .... # pkg audit -F .... [[pkgng-autoremove]] === Automatically Removing Unused Packages Removing a package may leave behind dependencies which are no longer required. Unneeded packages that were installed as dependencies (leaf packages) can be automatically detected and removed using: [source,shell] .... # pkg autoremove Packages to be autoremoved: ca_root_nss-3.15.1_1 The autoremoval will free 723 kB Proceed with autoremoval of packages [y/N]: y Deinstalling ca_root_nss-3.15.1_1... done .... Packages installed as dependencies are called _automatic_ packages. Non-automatic packages, i.e the packages that were explicity installed not as a dependency to another package, can be listed using: [source,shell] .... # pkg prime-list nginx openvpn sudo .... `pkg prime-list` is an alias command declared in [.filename]#/usr/local/etc/pkg.conf#. There are many others that can be used to query the package database of the system. For instance, command `pkg prime-origins` can be used to get the origin port directory of the list mentioned above: [source,shell] .... # pkg prime-origins www/nginx security/openvpn security/sudo .... This list can be used to rebuild all packages installed on a system using build tools such as package:ports-mgmt/poudriere[] or package:ports-mgmt/synth[]. Marking an installed package as automatic can be done using: [source,shell] .... # pkg set -A 1 devel/cmake .... Once a package is a leaf package and is marked as automatic, it gets selected by `pkg autoremove`. Marking an installed package as _not_ automatic can be done using: [source,shell] .... # pkg set -A 0 devel/cmake .... [[pkgng-backup]] === Restoring the Package Database Unlike the traditional package management system, pkg includes its own package database backup mechanism. This functionality is enabled by default. [TIP] ==== To disable the periodic script from backing up the package database, set `daily_backup_pkgdb_enable="NO"` in man:periodic.conf[5]. ==== To restore the contents of a previous package database backup, run the following command replacing _/path/to/pkg.sql_ with the location of the backup: [source,shell] .... # pkg backup -r /path/to/pkg.sql .... [NOTE] ==== If restoring a backup taken by the periodic script, it must be decompressed prior to being restored. ==== To run a manual backup of the pkg database, run the following command, replacing _/path/to/pkg.sql_ with a suitable file name and location: [source,shell] .... # pkg backup -d /path/to/pkg.sql .... [[pkgng-clean]] === Removing Stale Packages By default, pkg stores binary packages in a cache directory defined by `PKG_CACHEDIR` in man:pkg.conf[5]. Only copies of the latest installed packages are kept. Older versions of pkg kept all previous packages. To remove these outdated binary packages, run: [source,shell] .... # pkg clean .... The entire cache may be cleared by running: [source,shell] .... # pkg clean -a .... [[pkgng-set]] === Modifying Package Metadata Software within the FreeBSD Ports Collection can undergo major version number changes. To address this, pkg has a built-in command to update package origins. This can be useful, for example, if package:lang/php5[] is renamed to package:lang/php53[] so that package:lang/php5[] can now represent version `5.4`. To change the package origin for the above example, run: [source,shell] .... # pkg set -o lang/php5:lang/php53 .... As another example, to update package:lang/ruby18[] to package:lang/ruby19[], run: [source,shell] .... # pkg set -o lang/ruby18:lang/ruby19 .... As a final example, to change the origin of the [.filename]#libglut# shared libraries from package:graphics/libglut[] to package:graphics/freeglut[], run: [source,shell] .... # pkg set -o graphics/libglut:graphics/freeglut .... [NOTE] ==== When changing package origins, it is important to reinstall packages that are dependent on the package with the modified origin. To force a reinstallation of dependent packages, run: [source,shell] .... # pkg install -Rf graphics/freeglut .... ==== [[ports-using]] == Using the Ports Collection The Ports Collection is a set of [.filename]#Makefiles#, patches, and description files. Each set of these files is used to compile and install an individual application on FreeBSD, and is called a _port_. By default, the Ports Collection itself is stored as a subdirectory of [.filename]#/usr/ports#. Before an application can be compiled using a port, the Ports Collection must first be installed. If it was not installed during the installation of FreeBSD, use one of the following methods to install it: [[ports-using-portsnap-method]] [.procedure] **** *Procedure: Portsnap Method* The base system of FreeBSD includes Portsnap. This is a fast and user-friendly tool for retrieving the Ports Collection and is the recommended choice for most users. This utility connects to a FreeBSD site, verifies the secure key, and downloads a new copy of the Ports Collection. The key is used to verify the integrity of all downloaded files. . To download a compressed snapshot of the Ports Collection into [.filename]#/var/db/portsnap#: + [source,shell] .... # portsnap fetch .... + . When running Portsnap for the first time, extract the snapshot into [.filename]#/usr/ports#: + [source,shell] .... # portsnap extract .... + . After the first use of Portsnap has been completed as shown above, [.filename]#/usr/ports# can be updated as needed by running: + [source,shell] .... # portsnap fetch # portsnap update .... + When using `fetch`, the `extract` or the `update` operation may be run consecutively, like so: + [source,shell] .... # portsnap fetch update .... **** -[[ports-using-subversion-method]] +[[ports-using-git-method]] [.procedure] **** -*Procedure: Subversion Method* +*Procedure: Git Method* -If more control over the ports tree is needed or if local changes need to be maintained, Subversion can be used to obtain the Ports Collection. Refer to link:{committers-guide}#subversion-primer[the Subversion Primer] for a detailed description of Subversion. +If more control over the ports tree is needed or if local changes need to be maintained, Git can be used to obtain the Ports Collection. Refer to link:{committers-guide}#git-primer[the Git Primer] for a detailed description of Git. -. Subversion must be installed before it can be used to check out the ports tree. If a copy of the ports tree is already present, install Subversion like this: +. Git must be installed before it can be used to check out the ports tree. If a copy of the ports tree is already present, install Git like this: + [source,shell] .... -# cd /usr/ports/devel/subversion +# cd /usr/ports/devel/git # make install clean .... + -If the ports tree is not available, or pkg is being used to manage packages, Subversion can be installed as a package: +If the ports tree is not available, or pkg is being used to manage packages, Git can be installed as a package: + [source,shell] .... -# pkg install subversion +# pkg install git .... + . Check out a copy of the ports tree: + [source,shell] .... -# svn checkout https://svn.FreeBSD.org/ports/head /usr/ports +# git clone https://git.FreeBSD.org/ports.git /usr/ports .... + -. As needed, update [.filename]#/usr/ports# after the initial Subversion checkout: +. As needed, update [.filename]#/usr/ports# after the initial Git checkout: + [source,shell] .... -# svn update /usr/ports +# git -C /usr/ports pull .... **** The Ports Collection contains directories for software categories. Inside each category are subdirectories for individual applications. Each application subdirectory contains a set of files that tells FreeBSD how to compile and install that program, called a _ports skeleton_. Each port skeleton includes these files and directories: * [.filename]#Makefile#: contains statements that specify how the application should be compiled and where its components should be installed. * [.filename]#distinfo#: contains the names and checksums of the files that must be downloaded to build the port. * [.filename]#files/#: this directory contains any patches needed for the program to compile and install on FreeBSD. This directory may also contain other files used to build the port. * [.filename]#pkg-descr#: provides a more detailed description of the program. * [.filename]#pkg-plist#: a list of all the files that will be installed by the port. It also tells the ports system which files to remove upon deinstallation. Some ports include [.filename]#pkg-message# or other files to handle special situations. For more details on these files, and on ports in general, refer to the link:{porters-handbook}[FreeBSD Porter's Handbook]. The port does not include the actual source code, also known as a [.filename]#distfile#. The extract portion of building a port will automatically save the downloaded source to [.filename]#/usr/ports/distfiles#. [[ports-skeleton]] === Installing Ports This section provides basic instructions on using the Ports Collection to install or remove software. The detailed description of available `make` targets and environment variables is available in man:ports[7]. [WARNING] ==== Before compiling any port, be sure to update the Ports Collection as described in the previous section. Since the installation of any third-party software can introduce security vulnerabilities, it is recommended to first check https://vuxml.freebsd.org/[] for known security issues related to the port. Alternately, run `pkg audit -F` before installing a new port. This command can be configured to automatically perform a security audit and an update of the vulnerability database during the daily security system check. For more information, refer to man:pkg-audit[8] and man:periodic[8]. ==== Using the Ports Collection assumes a working Internet connection. It also requires superuser privilege. To compile and install the port, change to the directory of the port to be installed, then type `make install` at the prompt. Messages will indicate the progress: [source,shell] .... # cd /usr/ports/sysutils/lsof # make install >> lsof_4.88D.freebsd.tar.gz doesn't seem to exist in /usr/ports/distfiles/. >> Attempting to fetch from ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/. ===> Extracting for lsof-4.88 ... [extraction output snipped] ... >> Checksum OK for lsof_4.88D.freebsd.tar.gz. ===> Patching for lsof-4.88.d,8 ===> Applying FreeBSD patches for lsof-4.88.d,8 ===> Configuring for lsof-4.88.d,8 ... [configure output snipped] ... ===> Building for lsof-4.88.d,8 ... [compilation output snipped] ... ===> Installing for lsof-4.88.d,8 ... [installation output snipped] ... ===> Generating temporary packing list ===> Compressing manual pages for lsof-4.88.d,8 ===> Registering installation for lsof-4.88.d,8 ===> SECURITY NOTE: This port has installed the following binaries which execute with increased privileges. /usr/local/sbin/lsof # .... Since `lsof` is a program that runs with increased privileges, a security warning is displayed as it is installed. Once the installation is complete, the prompt will be returned. Some shells keep a cache of the commands that are available in the directories listed in the `PATH` environment variable, to speed up lookup operations for the executable file of these commands. Users of the `tcsh` shell should type `rehash` so that a newly installed command can be used without specifying its full path. Use `hash -r` instead for the `sh` shell. Refer to the documentation for the shell for more information. During installation, a working subdirectory is created which contains all the temporary files used during compilation. Removing this directory saves disk space and minimizes the chance of problems later when upgrading to the newer version of the port: [source,shell] .... # make clean ===> Cleaning for lsof-88.d,8 # .... [NOTE] ==== To save this extra step, instead use `make install clean` when compiling the port. ==== ==== Customizing Ports Installation Some ports provide build options which can be used to enable or disable application components, provide security options, or allow for other customizations. Examples include package:www/firefox[], package:security/gpgme[], and package:mail/sylpheed-claws[]. If the port depends upon other ports which have configurable options, it may pause several times for user interaction as the default behavior is to prompt the user to select options from a menu. To avoid this and do all of the configuration in one batch, run `make config-recursive` within the port skeleton. Then, run `make install [clean]` to compile and install the port. [TIP] ==== When using `config-recursive`, the list of ports to configure are gathered by the `all-depends-list` target. It is recommended to run `make config-recursive` until all dependent ports options have been defined, and ports options screens no longer appear, to be certain that all dependency options have been configured. ==== There are several ways to revisit a port's build options menu in order to add, remove, or change these options after a port has been built. One method is to `cd` into the directory containing the port and type `make config`. Another option is to use `make showconfig`. Another option is to execute `make rmconfig` which will remove all selected options and allow you to start over. All of these options, and others, are explained in great detail in man:ports[7]. The ports system uses man:fetch[1] to download the source files, which supports various environment variables. The `FTP_PASSIVE_MODE`, `FTP_PROXY`, and `FTP_PASSWORD` variables may need to be set if the FreeBSD system is behind a firewall or FTP/HTTP proxy. See man:fetch[3] for the complete list of supported variables. For users who cannot be connected to the Internet all the time, `make fetch` can be run within [.filename]#/usr/ports#, to fetch all distfiles, or within a category, such as [.filename]#/usr/ports/net#, or within the specific port skeleton. Note that if a port has any dependencies, running this command in a category or ports skeleton will _not_ fetch the distfiles of ports from another category. Instead, use `make fetch-recursive` to also fetch the distfiles for all the dependencies of a port. In rare cases, such as when an organization has a local distfiles repository, the `MASTER_SITES` variable can be used to override the download locations specified in the [.filename]#Makefile#. When using, specify the alternate location: [source,shell] .... # cd /usr/ports/directory # make MASTER_SITE_OVERRIDE= \ ftp://ftp.organization.org/pub/FreeBSD/ports/distfiles/ fetch .... The `WRKDIRPREFIX` and `PREFIX` variables can override the default working and target directories. For example: [source,shell] .... # make WRKDIRPREFIX=/usr/home/example/ports install .... will compile the port in [.filename]#/usr/home/example/ports# and install everything under [.filename]#/usr/local#. [source,shell] .... # make PREFIX=/usr/home/example/local install .... will compile the port in [.filename]#/usr/ports# and install it in [.filename]#/usr/home/example/local#. And: [source,shell] .... # make WRKDIRPREFIX=../ports PREFIX=../local install .... will combine the two. These can also be set as environmental variables. Refer to the manual page for your shell for instructions on how to set an environmental variable. [[ports-removing]] === Removing Installed Ports Installed ports can be uninstalled using `pkg delete`. Examples for using this command can be found in the man:pkg-delete[8] manual page. Alternately, `make deinstall` can be run in the port's directory: [source,shell] .... # cd /usr/ports/sysutils/lsof # make deinstall ===> Deinstalling for sysutils/lsof ===> Deinstalling Deinstallation has been requested for the following 1 packages: lsof-4.88.d,8 The deinstallation will free 229 kB [1/1] Deleting lsof-4.88.d,8... done .... It is recommended to read the messages as the port is uninstalled. If the port has any applications that depend upon it, this information will be displayed but the uninstallation will proceed. In such cases, it may be better to reinstall the application in order to prevent broken dependencies. [[ports-upgrading]] === Upgrading Ports Over time, newer versions of software become available in the Ports Collection. This section describes how to determine which software can be upgraded and how to perform the upgrade. -To determine if newer versions of installed ports are available, ensure that the latest version of the ports tree is installed, using the updating command described in either <> or <>. On FreeBSD 10 and later, or if the system has been converted to pkg, the following command will list the installed ports which are out of date: +To determine if newer versions of installed ports are available, ensure that the latest version of the ports tree is installed, using the updating command described in either <> or <>. On FreeBSD 10 and later, or if the system has been converted to pkg, the following command will list the installed ports which are out of date: [source,shell] .... # pkg version -l "<" .... For FreeBSD 9._X_ and lower, the following command will list the installed ports that are out of date: [source,shell] .... # pkg_version -l "<" .... [IMPORTANT] ==== Before attempting an upgrade, read [.filename]#/usr/ports/UPDATING# from the top of the file to the date closest to the last time ports were upgraded or the system was installed. This file describes various issues and additional steps users may encounter and need to perform when updating a port, including such things as file format changes, changes in locations of configuration files, or any incompatibilities with previous versions. Make note of any instructions which match any of the ports that need upgrading and follow these instructions when performing the upgrade. ==== [[ports-upgrading-tools]] ==== Tools to Upgrade and Manage Ports The Ports Collection contains several utilities to perform the actual upgrade. Each has its strengths and weaknesses. Historically, most installations used either Portmaster or Portupgrade. Synth is a newer alternative. [NOTE] ==== The choice of which tool is best for a particular system is up to the system administrator. It is recommended practice to back up your data before using any of these tools. ==== [[portmaster]] ==== Upgrading Ports Using Portmaster package:ports-mgmt/portmaster[] is a very small utility for upgrading installed ports. It is designed to use the tools installed with the FreeBSD base system without depending on other ports or databases. To install this utility as a port: [source,shell] .... # cd /usr/ports/ports-mgmt/portmaster # make install clean .... Portmaster defines four categories of ports: * Root port: has no dependencies and is not a dependency of any other ports. * Trunk port: has no dependencies, but other ports depend upon it. * Branch port: has dependencies and other ports depend upon it. * Leaf port: has dependencies but no other ports depend upon it. To list these categories and search for updates: [source,shell] .... # portmaster -L ===>>> Root ports (No dependencies, not depended on) ===>>> ispell-3.2.06_18 ===>>> screen-4.0.3 ===>>> New version available: screen-4.0.3_1 ===>>> tcpflow-0.21_1 ===>>> 7 root ports ... ===>>> Branch ports (Have dependencies, are depended on) ===>>> apache22-2.2.3 ===>>> New version available: apache22-2.2.8 ... ===>>> Leaf ports (Have dependencies, not depended on) ===>>> automake-1.9.6_2 ===>>> bash-3.1.17 ===>>> New version available: bash-3.2.33 ... ===>>> 32 leaf ports ===>>> 137 total installed ports ===>>> 83 have new versions available .... This command is used to upgrade all outdated ports: [source,shell] .... # portmaster -a .... [NOTE] ==== By default, Portmaster makes a backup package before deleting the existing port. If the installation of the new version is successful, Portmaster deletes the backup. Using `-b` instructs Portmaster not to automatically delete the backup. Adding `-i` starts Portmaster in interactive mode, prompting for confirmation before upgrading each port. Many other options are available. Read through the manual page for man:portmaster[8] for details regarding their usage. ==== If errors are encountered during the upgrade process, add `-f` to upgrade and rebuild all ports: [source,shell] .... # portmaster -af .... Portmaster can also be used to install new ports on the system, upgrading all dependencies before building and installing the new port. To use this function, specify the location of the port in the Ports Collection: [source,shell] .... # portmaster shells/bash .... More information about package:ports-mgmt/portmaster[] may be found in its [.filename]#pkg-descr#. [[portupgrade]] ==== Upgrading Ports Using Portupgrade package:ports-mgmt/portupgrade[] is another utility that can be used to upgrade ports. It installs a suite of applications which can be used to manage ports. However, it is dependent upon Ruby. To install the port: [source,shell] .... # cd /usr/ports/ports-mgmt/portupgrade # make install clean .... Before performing an upgrade using this utility, it is recommended to scan the list of installed ports using `pkgdb -F` and to fix all the inconsistencies it reports. To upgrade all the outdated ports installed on the system, use `portupgrade -a`. Alternately, include `-i` to be asked for confirmation of every individual upgrade: [source,shell] .... # portupgrade -ai .... To upgrade only a specified application instead of all available ports, use `portupgrade _pkgname_`. It is very important to include `-R` to first upgrade all the ports required by the given application: [source,shell] .... # portupgrade -R firefox .... If `-P` is included, Portupgrade searches for available packages in the local directories listed in `PKG_PATH`. If none are available locally, it then fetches packages from a remote site. If packages can not be found locally or fetched remotely, Portupgrade will use ports. To avoid using ports entirely, specify `-PP`. This last set of options tells Portupgrade to abort if no packages are available: [source,shell] .... # portupgrade -PP gnome3 .... To just fetch the port distfiles, or packages, if `-P` is specified, without building or installing anything, use `-F`. For further information on all of the available switches, refer to the manual page for `portupgrade`. More information about package:ports-mgmt/portupgrade[] may be found in its [.filename]#pkg-descr#. [[ports-disk-space]] === Ports and Disk Space Using the Ports Collection will use up disk space over time. After building and installing a port, running `make clean` within the ports skeleton will clean up the temporary [.filename]#work# directory. If Portmaster is used to install a port, it will automatically remove this directory unless `-K` is specified. If Portupgrade is installed, this command will remove all [.filename]#work# directories found within the local copy of the Ports Collection: [source,shell] .... # portsclean -C .... In addition, outdated source distribution files accumulate in [.filename]#/usr/ports/distfiles# over time. To use Portupgrade to delete all the distfiles that are no longer referenced by any ports: [source,shell] .... # portsclean -D .... Portupgrade can remove all distfiles not referenced by any port currently installed on the system: [source,shell] .... # portsclean -DD .... If Portmaster is installed, use: [source,shell] .... # portmaster --clean-distfiles .... By default, this command is interactive and prompts the user to confirm if a distfile should be deleted. In addition to these commands, package:ports-mgmt/pkg_cutleaves[] automates the task of removing installed ports that are no longer needed. [[ports-poudriere]] == Building Packages with Poudriere Poudriere is a `BSD`-licensed utility for creating and testing FreeBSD packages. It uses FreeBSD jails to set up isolated compilation environments. These jails can be used to build packages for versions of FreeBSD that are different from the system on which it is installed, and also to build packages for i386 if the host is an amd64 system. Once the packages are built, they are in a layout identical to the official mirrors. These packages are usable by man:pkg[8] and other package management tools. Poudriere is installed using the package:ports-mgmt/poudriere[] package or port. The installation includes a sample configuration file [.filename]#/usr/local/etc/poudriere.conf.sample#. Copy this file to [.filename]#/usr/local/etc/poudriere.conf#. Edit the copied file to suit the local configuration. While `ZFS` is not required on the system running poudriere, it is beneficial. When `ZFS` is used, `ZPOOL` must be specified in [.filename]#/usr/local/etc/poudriere.conf# and `FREEBSD_HOST` should be set to a nearby mirror. Defining `CCACHE_DIR` enables the use of package:devel/ccache[] to cache compilation and reduce build times for frequently-compiled code. It may be convenient to put poudriere datasets in an isolated tree mounted at [.filename]#/poudriere#. Defaults for the other configuration values are adequate. The number of processor cores detected is used to define how many builds will run in parallel. Supply enough virtual memory, either with `RAM` or swap space. If virtual memory runs out, the compilation jails will stop and be torn down, resulting in weird error messages. [[poudriere-initialization]] === Initialize Jails and Port Trees After configuration, initialize poudriere so that it installs a jail with the required FreeBSD tree and a ports tree. Specify a name for the jail using `-j` and the FreeBSD version with `-v`. On systems running FreeBSD/amd64, the architecture can be set with `-a` to either `i386` or `amd64`. The default is the architecture shown by `uname`. [source,shell] .... # poudriere jail -c -j 11amd64 -v 11.4-RELEASE [00:00:00] Creating 11amd64 fs at /poudriere/jails/11amd64... done [00:00:00] Using pre-distributed MANIFEST for FreeBSD 11.4-RELEASE amd64 [00:00:00] Fetching base for FreeBSD 11.4-RELEASE amd64 /poudriere/jails/11amd64/fromftp/base.txz 125 MB 4110 kBps 31s [00:00:33] Extracting base... done [00:00:54] Fetching src for FreeBSD 11.4-RELEASE amd64 /poudriere/jails/11amd64/fromftp/src.txz 154 MB 4178 kBps 38s [00:01:33] Extracting src... done [00:02:31] Fetching lib32 for FreeBSD 11.4-RELEASE amd64 /poudriere/jails/11amd64/fromftp/lib32.txz 24 MB 3969 kBps 06s [00:02:38] Extracting lib32... done [00:02:42] Cleaning up... done [00:02:42] Recording filesystem state for clean... done [00:02:42] Upgrading using ftp /etc/resolv.conf -> /poudriere/jails/11amd64/etc/resolv.conf Looking up update.FreeBSD.org mirrors... 3 mirrors found. Fetching public key from update4.freebsd.org... done. Fetching metadata signature for 11.4-RELEASE from update4.freebsd.org... done. Fetching metadata index... done. Fetching 2 metadata files... done. Inspecting system... done. Preparing to download files... done. Fetching 124 patches.....10....20....30....40....50....60....70....80....90....100....110....120.. done. Applying patches... done. Fetching 6 files... done. The following files will be added as part of updating to 11.4-RELEASE-p1: /usr/src/contrib/unbound/.github /usr/src/contrib/unbound/.github/FUNDING.yml /usr/src/contrib/unbound/contrib/drop2rpz /usr/src/contrib/unbound/contrib/unbound_portable.service.in /usr/src/contrib/unbound/services/rpz.c /usr/src/contrib/unbound/services/rpz.h /usr/src/lib/libc/tests/gen/spawnp_enoexec.sh The following files will be updated as part of updating to 11.4-RELEASE-p1: […] Installing updates...Scanning //usr/share/certs/blacklisted for certificates... Scanning //usr/share/certs/trusted for certificates... done. 11.4-RELEASE-p1 [00:04:06] Recording filesystem state for clean... done [00:04:07] Jail 11amd64 11.4-RELEASE-p1 amd64 is ready to be used .... [source,shell] .... -# poudriere ports -c -p local -m svn+https +# poudriere ports -c -p local -m git+https [00:00:00] Creating local fs at /poudriere/ports/local... done [00:00:00] Checking out the ports tree... done .... On a single computer, poudriere can build ports with multiple configurations, in multiple jails, and from different port trees. Custom configurations for these combinations are called _sets_. See the CUSTOMIZATION section of man:poudriere[8] for details after package:ports-mgmt/poudriere[] or package:ports-mgmt/poudriere-devel[] is installed. The basic configuration shown here puts a single jail-, port-, and set-specific [.filename]#make.conf# in [.filename]#/usr/local/etc/poudriere.d#. The filename in this example is created by combining the jail name, port name, and set name: [.filename]#10amd64-local-workstation-make.conf#. The system [.filename]#make.conf# and this new file are combined at build time to create the [.filename]#make.conf# used by the build jail. Packages to be built are entered in [.filename]#10amd64-local-workstation-pkglist#: [.programlisting] .... editors/emacs devel/git ports-mgmt/pkg ... .... Options and dependencies for the specified ports are configured: [source,shell] .... # poudriere options -j 10amd64 -p local -z workstation -f 10amd64-local-workstation-pkglist .... Finally, packages are built and a package repository is created: [source,shell] .... # poudriere bulk -j 10amd64 -p local -z workstation -f 10amd64-local-workstation-pkglist .... While running, pressing kbd:[Ctrl+t] displays the current state of the build. Poudriere also builds files in [.filename]#/poudriere/logs/bulk/jailname# that can be used with a web server to display build information. After completion, the new packages are now available for installation from the poudriere repository. For more information on using poudriere, see man:poudriere[8] and the main web site, https://github.com/freebsd/poudriere/wiki[]. === Configuring pkg Clients to Use a Poudriere Repository While it is possible to use both a custom repository along side of the official repository, sometimes it is useful to disable the official repository. This is done by creating a configuration file that overrides and disables the official configuration file. Create [.filename]#/usr/local/etc/pkg/repos/FreeBSD.conf# that contains the following: [.programlisting] .... FreeBSD: { enabled: no } .... Usually it is easiest to serve a poudriere repository to the client machines via HTTP. Set up a webserver to serve up the package directory, for instance: [.filename]#/usr/local/poudriere/data/packages/10amd64#, where [.filename]#10amd64# is the name of the build. If the URL to the package repository is: `http://pkg.example.com/10amd64`, then the repository configuration file in [.filename]#/usr/local/etc/pkg/repos/custom.conf# would look like: [.programlisting] .... custom: { url: "http://pkg.example.com/10amd64", enabled: yes, } .... [[ports-nextsteps]] == Post-Installation Considerations Regardless of whether the software was installed from a binary package or port, most third-party applications require some level of configuration after installation. The following commands and locations can be used to help determine what was installed with the application. * Most applications install at least one default configuration file in [.filename]#/usr/local/etc#. In cases where an application has a large number of configuration files, a subdirectory will be created to hold them. Often, sample configuration files are installed which end with a suffix such as [.filename]#.sample#. The configuration files should be reviewed and possibly edited to meet the system's needs. To edit a sample file, first copy it without the [.filename]#.sample# extension. * Applications which provide documentation will install it into [.filename]#/usr/local/shared/doc# and many applications also install manual pages. This documentation should be consulted before continuing. * Some applications run services which must be added to [.filename]#/etc/rc.conf# before starting the application. These applications usually install a startup script in [.filename]#/usr/local/etc/rc.d#. See crossref:config[configtuning-starting-services,Starting Services] for more information. + [NOTE] ==== By design, applications do not run their startup script upon installation, nor do they run their stop script upon deinstallation or upgrade. This decision is left to the individual system administrator. ==== * Users of man:csh[1] should run `rehash` to rebuild the known binary list in the shells `PATH`. * Use `pkg info` to determine which files, man pages, and binaries were installed with the application. [[ports-broken]] == Dealing with Broken Ports When a port does not build or install, try the following: . Search to see if there is a fix pending for the port in the link:https://www.FreeBSD.org/support/[Problem Report database]. If so, implementing the proposed fix may fix the issue. . Ask the maintainer of the port for help. Type `make maintainer` in the ports skeleton or read the port's [.filename]#Makefile# to find the maintainer's email address. Remember to include the `$FreeBSD:` line from the port's [.filename]#Makefile# and the output leading up to the error in the email to the maintainer. + [NOTE] ==== Some ports are not maintained by an individual but instead by a group maintainer represented by a link:{mailing-list-faq}[mailing list]. Many, but not all, of these addresses look like mailto:freebsd-listname@FreeBSD.org[freebsd-listname@FreeBSD.org]. Please take this into account when sending an email. In particular, ports maintained by mailto:ports@FreeBSD.org[ports@FreeBSD.org] are not maintained by a specific individual. Instead, any fixes and support come from the general community who subscribe to that mailing list. More volunteers are always needed! ==== + If there is no response to the email, use Bugzilla to submit a bug report using the instructions in link:{problem-reports}[Writing FreeBSD Problem Reports]. . Fix it! The link:{porters-handbook}[Porter's Handbook] includes detailed information on the ports infrastructure so that you can fix the occasional broken port or even submit your own! . Install the package instead of the port using the instructions in <>.