Page MenuHomeFreeBSD

D61505.1795105548.diff
No OneTemporary

Authored By
Unknown
Size
38 KB
Referenced Files
None
Subscribers
None

D61505.1795105548.diff

diff --git a/documentation/content/en/articles/freebsd-update-server/_index.adoc b/documentation/content/en/articles/freebsd-update-server/_index.adoc
--- a/documentation/content/en/articles/freebsd-update-server/_index.adoc
+++ b/documentation/content/en/articles/freebsd-update-server/_index.adoc
@@ -1,15 +1,15 @@
---
-title: Build Your Own FreeBSD Update Server
+title: Building a FreeBSD Base System Package Repository
authors:
- - author: Jason Helfman
- email: jgh@FreeBSD.org
-copyright: 2009-2011, 2013 Jason Helfman
-description: Building your own freebsd-update server allows a system administrator to perform fast updates for a number of machines from a local mirror
+ - author: Muhammad Moinur Rahman
+ email: bofh@FreeBSD.org
+copyright: 2026 Muhammad Moinur Rahman
+description: Building and signing a private repository of FreeBSD base system packages allows a system administrator to update a number of machines from a local, trusted source
trademarks: ["freebsd", "amd", "intel", "general"]
-tags: ["FreeBSD", "Update", "Server", "internal"]
+tags: ["FreeBSD", "pkgbase", "pkg", "Update", "Server", "internal"]
---
-= Build Your Own FreeBSD Update Server
+= Building a FreeBSD Base System Package Repository
:doctype: article
:toc: macro
:toclevels: 1
@@ -39,602 +39,265 @@
include::../../../../../shared/asciidoctor.adoc[]
endif::[]
-
-[WARNING]
-====
-The instructions in this article refer to an older version of FreeBSD and may
-not work properly on recent versions of the OS. With the availability of
-pkgbase, the freebsd-update utility is scheduled to be removed from FreeBSD in
-the future. When that happens, this article is either updated to reflect the
-new procedures or removed entirely.
-====
-
[.abstract-title]
Abstract
-This article describes building an internal FreeBSD Update Server.
-The https://github.com/freebsd/freebsd-update-build/[freebsd-update-server] is written by `{cperciva}`, Security Officer Emeritus of FreeBSD.
-For users that think it is convenient to update their systems against an official update server, building their own FreeBSD Update Server may help to extend its functionality by supporting manually-tweaked FreeBSD releases or by providing a local mirror that will allow faster updates for a number of machines.
+This article walks through building a private repository of FreeBSD base system packages, signing it, and pointing machines at it.
+It is written for administrators looking after more than a handful of FreeBSD hosts.
+A local repository makes it possible to try a security patch on a couple of test machines before it gets anywhere near production, to ship a custom kernel configuration or local fixes, and to let every machine update over the LAN instead of the Internet.
'''
toc::[]
-[[acknowledgments]]
-== Acknowledgments
-
-This article was subsequently printed at https://people.freebsd.org/~jgh/files/fus/BSD_03_2010_EN.pdf[BSD Magazine].
-
[[introduction]]
== Introduction
-Experienced users or administrators are often responsible for several machines or environments.
-They understand the difficult demands and challenges of maintaining such an infrastructure.
-Running a FreeBSD Update Server makes it easier to deploy security and software patches to selected test machines before rolling them out to production.
-It also means a number of systems can be updated from the local network rather than a potentially slower Internet connection.
-This article outlines the steps involved in creating an internal FreeBSD Update Server.
-
-[[prerequisites]]
-== Prerequisites
+For a long time, the way to run a private update service was freebsd-update-build, the server side of man:freebsd-update[8].
+It worked, but it was fiddly.
+Among other things, it built the world twice, 400 days apart, just to find out which files contained timestamps.
-To build an internal FreeBSD Update Server some requirements should be met.
+Since FreeBSD 15.0, the base system is also available as packages managed by man:pkg[8], described in man:freebsd-base[7] and usually called _pkgbase_.
+These packages are meant to replace both the distribution sets used by man:bsdinstall[8] and man:freebsd-update[8] itself.
+The good news for anyone running a private update server is that the source tree now does most of the work.
+It builds the packages, signs the catalogue, and on later builds keeps every package whose contents did not change.
+What gets build is a plain directory tree that any web server can serve.
-* A running FreeBSD system.
-+
[NOTE]
====
-At a minimum, updates require building on a FreeBSD release greater than or equal to the target release version for distribution.
+man:freebsd-base[7] is a technology preview in FreeBSD 15.
+Read the extref:{handbook}cutting-edge[Updating FreeBSD with packages, pkgbase] section of the Handbook before converting production systems.
====
-* A user account with at least 4 GB of available space. This will allow the creation of updates for 7.1 and 7.2, but the exact space requirements may change from version to version.
-* An man:ssh[1] account on a remote machine to upload distributed updates.
-* A web server, like extref:{handbook}network-servers[Apache, network-apache], with over half of the space required for the build. For instance, test builds for 7.1 and 7.2 consume a total amount of 4 GB, and the webserver space needed to distribute these updates is 2.6 GB.
-* Basic knowledge of shell scripting with POSIX(R) shell, man:sh[1].
-
-[[Configuration]]
-== Configuration: Installation & Setup
-
-Download the https://github.com/freebsd/freebsd-update-build/[freebsd-update-server] software by installing package:devel/git[] and package:security/ca_root_nss[], and execute:
-[source,shell]
-....
-% git clone https://github.com/freebsd/freebsd-update-build.git freebsd-update-server
-....
-
-Update [.filename]#scripts/build.conf# appropriately.
-It is sourced during all build operations.
-
-Here is the default [.filename]#build.conf#, which should be modified to suit your environment.
-
-[.programlisting]
-....
-# Main configuration file for FreeBSD Update builds. The
-# release-specific configuration data is lower down in
-# the scripts tree.
-
-# Location from which to fetch releases
-export FTP=ftp://ftp2.freebsd.org/pub/FreeBSD/releases <.>
-
-# Host platform
-export HOSTPLATFORM=`uname -m`
-
-# Host name to use inside jails
-export BUILDHOSTNAME=${HOSTPLATFORM}-builder.daemonology.net <.>
+[[prerequisites]]
+== Prerequisites
-# Location of SSH key
-export SSHKEY=/root/.ssh/id_dsa <.>
+The following are needed:
-# SSH account into which files are uploaded
-MASTERACCT=builder@wadham.daemonology.net <.>
+* A FreeBSD build host.
+It can run an older release than the one being built; [.filename]#/usr/src/UPDATING# lists the oldest build host that is supported.
+* Room under [.filename]#/usr/obj# for a full `buildworld` and `buildkernel`, plus one package set for every release and patch level kept in the repository.
+* package:devel/git[] to fetch the source tree.
+* A web server, such as package:www/nginx[], with a TLS certificate the clients trust.
+* Clients running FreeBSD 15.0 or later.
+FreeBSD 14 works too, with the caveats listed in the Handbook.
-# Directory into which files are uploaded
-MASTERDIR=update-master.freebsd.org <.>
-....
+[[signing-key]]
+== Creating the Signing Key
-Parameters for consideration would be:
+Start with the key, because everything else depends on it.
+Clients only accept a repository whose catalogue is signed by a key they trust.
+Do not skip this step, even on a "trusted" internal network: anyone who can tamper with an unsigned base system repository can replace the kernel and every binary on every machine that updates from it.
-<.> This is the location where ISO images are downloaded from (by the `fetchiso()` subroutine of [.filename]#scripts/build.subr#). The location configured is not limited to FTP URIs. Any URI scheme supported by standard man:fetch[1] utility should work fine.
-Customizations to the `fetchiso()` code can be installed by copying the default [.filename]#build.subr# script to the release and architecture-specific area at [.filename]#scripts/RELEASE/ARCHITECTURE/build.subr# and applying local changes.
+Create an EdDSA key pair on the build host with man:pkg-key[8]:
-<.> The name of the build host. This information will be displayed on updated systems when issuing:
-+
[source,shell]
....
-% uname -v
+# mkdir -m 0700 /root/pkgbase-keys
+# cd /root/pkgbase-keys
+# pkg key --create -t eddsa repo.key > repo.pub
....
-+
-<.> The SSH key for uploading files to the update server. A key pair can be created by typing `ssh-keygen -t dsa`. This parameter is optional; standard password authentication will be used as a fallback authentication method when `SSHKEY` is not defined.
-The man:ssh-keygen[1] manual page has more detailed information about SSH and the appropriate steps for creating and using one.
-
-<.> Account for uploading files to the update server.
-<.> Directory on the update server where files are uploaded to.
+man:pkg-key[8] sets the mode of [.filename]#repo.key# to `0400` automatically.
+Back it up like any other signing key; losing it means distributing a new public key to every client.
+Get [.filename]#repo.pub# onto the clients through a channel that is already trusted, such as a configuration management system.
+Downloading it from the very repository server it is supposed to protect defeats the purpose.
-The default [.filename]#build.conf# shipped with the freebsd-update-server sources is suitable for building i386 releases of FreeBSD. As an example of building an update server for other architectures, the following steps outline the configuration changes needed for amd64:
-
-[.procedure]
+[NOTE]
====
-. Create a build environment for amd64:
-+
-[source,shell]
-....
-% mkdir -p /usr/local/freebsd-update-server/scripts/7.2-RELEASE/amd64
-....
-
-. Install a [.filename]#build.conf# in the newly created build directory. The build configuration options for FreeBSD 7.2-RELEASE on amd64 should be similar to:
-+
-[.programlisting]
-....
-# SHA256 hash of RELEASE disc1.iso image.
-export RELH=1ea1f6f652d7c5f5eab7ef9f8edbed50cb664b08ed761850f95f48e86cc71ef5 <.>
-# Components of the world, source, and kernels
-export WORLDPARTS="base catpages dict doc games info manpages proflibs lib32"
-export SOURCEPARTS="base bin contrib crypto etc games gnu include krb5 \
- lib libexec release rescue sbin secure share sys tools \
- ubin usbin cddl"
-export KERNELPARTS="generic"
-
-# EOL date
-export EOL=1275289200 <.>
-....
-+
-<.> The man:sha256[1] hash key for the desired release, is published within the respective link:https://www.FreeBSD.org/releases/[release announcement].
-<.> To generate the "End of Life" number for [.filename]#build.conf#, refer to the "Estimated EOL" posted on the link:https://www.FreeBSD.org/security/security/[FreeBSD Security Website]. The value of `EOL` can be derived from the date listed on the web site, using the man:date[1] utility, for example:
-+
-[source,shell]
-....
-% date -j -f '%Y%m%d-%H%M%S' '20090401-000000' '+%s'
-....
+EdDSA needs a man:pkg[8] on every client that supports the `eddsa` signer; man:pkg-repo[8] lists the signer types a given version knows about.
+If older clients are still around, create an RSA key with man:openssl[1] as shown in man:pkg-repo[8], and use the `rsa:` prefix wherever this article uses `eddsa:`.
====
-[[build]]
-== Building Update Code
-
-The first step is to run [.filename]#scripts/make.sh#.
-This will build some binaries, create directories, and generate an RSA signing key used for approving builds.
-In this step, a passphrase will have to be supplied for the final creation of the signing key.
-
-[source,shell]
-....
-# sh scripts/make.sh
-cc -O2 -fno-strict-aliasing -pipe findstamps.c -o findstamps
-findstamps.c: In function 'usage':
-findstamps.c:45: warning: incompatible implicit declaration of built-in function 'exit'
-cc -O2 -fno-strict-aliasing -pipe unstamp.c -o unstamp
-install findstamps ../bin
-install unstamp ../bin
-rm -f findstamps unstamp
-Generating RSA private key, 4096 bit long modulus
-................................................................................++
-...................++
-e is 65537 (0x10001)
-
-Public key fingerprint:
-27ef53e48dc869eea6c3136091cc6ab8589f967559824779e855d58a2294de9e
-
-Encrypting signing key for root
-enter aes-256-cbc encryption password:
-Verifying - enter aes-256-cbc encryption password:
-....
+To keep the private key off the build host altogether, man:pkg-repo[8] can call an external `signing_command` instead, for example over man:ssh[1] to a dedicated signing host.
+The `PKG_REPO_SIGNING_KEY` variable used below accepts that form too.
+The EXAMPLES section of man:pkg-repo[8] describes the protocol and the matching `FINGERPRINTS` setup on the clients.
-[NOTE]
-====
-Keep a note of the generated key fingerprint.
-This value is required in [.filename]#/etc/freebsd-update.conf# for binary updates.
-====
+[[build]]
+== Building the Packages
-At this point, we are ready to stage a build.
+Fetch the source tree for the release to build.
+The examples here use the `releng/15.2` branch, which carries 15.2-RELEASE and its security and errata patches:
[source,shell]
....
-# cd /usr/local/freebsd-update-server
-# sh scripts/init.sh amd64 7.2-RELEASE
+# git clone -b releng/15.2 https://git.FreeBSD.org/src.git /usr/src
....
-What follows is a sample of an _initial_ build run.
+Build the world and the kernel, then create and sign the packages:
[source,shell]
....
-# sh scripts/init.sh amd64 7.2-RELEASE
-Mon Aug 24 16:04:36 PDT 2009 Starting fetch for FreeBSD/amd64 7.2-RELEASE
-/usr/local/freebsd-update-server/work/7.2-RELE100 of 588 MB 359 kBps 00m00s
-Mon Aug 24 16:32:38 PDT 2009 Verifying disc1 hash for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 16:32:44 PDT 2009 Extracting components for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 16:34:05 PDT 2009 Constructing world+src image for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 16:35:57 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 23:36:24 UTC 2009 Building world for FreeBSD/amd64 7.2-RELEASE
-Tue Aug 25 00:31:29 UTC 2009 Distributing world for FreeBSD/amd64 7.2-RELEASE
-Tue Aug 25 00:32:36 UTC 2009 Building and distributing kernels for FreeBSD/amd64 7.2-RELEASE
-Tue Aug 25 00:44:44 UTC 2009 Constructing world components for FreeBSD/amd64 7.2-RELEASE
-Tue Aug 25 00:44:56 UTC 2009 Distributing source for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 17:46:18 PDT 2009 Moving components into staging area for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 17:46:33 PDT 2009 Identifying extra documentation for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 17:47:13 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 17:47:18 PDT 2009 Indexing release for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 17:50:44 PDT 2009 Indexing world0 for FreeBSD/amd64 7.2-RELEASE
-
-Files built but not released:
-Files released but not built:
-Files which differ by more than contents:
-Files which differ between release and build:
-kernel|generic|/GENERIC/hptrr.ko
-kernel|generic|/GENERIC/kernel
-src|sys|/sys/conf/newvers.sh
-world|base|/boot/loader
-world|base|/boot/pxeboot
-world|base|/etc/mail/freebsd.cf
-world|base|/etc/mail/freebsd.submit.cf
-world|base|/etc/mail/sendmail.cf
-world|base|/etc/mail/submit.cf
-world|base|/lib/libcrypto.so.5
-world|base|/usr/bin/ntpq
-world|base|/usr/lib/libalias.a
-world|base|/usr/lib/libalias_cuseeme.a
-world|base|/usr/lib/libalias_dummy.a
-world|base|/usr/lib/libalias_ftp.a
-...
+# cd /usr/src
+# make -j "$(sysctl -n hw.ncpu)" buildworld buildkernel
+# make update-packages PKG_REPO_SIGNING_KEY=eddsa:/root/pkgbase-keys/repo.key
....
-Then the build of the world is performed again, with world patches.
-A more detailed explanation may be found in [.filename]#scripts/build.subr#.
+Once that finishes, the packages and the signed catalogue are in [.filename]#/usr/obj/usr/src/repo/FreeBSD:15:amd64/15.2/#.
+Next to that directory is a symbolic link named [.filename]#latest#, which always points to the newest package set.
+On this first run, `update-packages` has nothing to compare against, so it prints `Bootstrapping repository, not checking for new packages`.
+That is expected.
-[WARNING]
-====
-During this second build cycle, the network time protocol daemon, man:ntpd[8], is turned off.
-Per `{cperciva}`, Security Officer Emeritus of FreeBSD, "the https://github.com/freebsd/freebsd-update-build/[freebsd-update-server] build code needs to identify timestamps which are stored in files so that they can be ignored when comparing builds to determine which files need to be updated.
-This timestamp-finding works by doing two builds 400 days apart and comparing the results."
-====
-
-[source,shell]
-....
-Mon Aug 24 17:54:07 PDT 2009 Extracting world+src for FreeBSD/amd64 7.2-RELEASE
-Wed Sep 29 00:54:34 UTC 2010 Building world for FreeBSD/amd64 7.2-RELEASE
-Wed Sep 29 01:49:42 UTC 2010 Distributing world for FreeBSD/amd64 7.2-RELEASE
-Wed Sep 29 01:50:50 UTC 2010 Building and distributing kernels for FreeBSD/amd64 7.2-RELEASE
-Wed Sep 29 02:02:56 UTC 2010 Constructing world components for FreeBSD/amd64 7.2-RELEASE
-Wed Sep 29 02:03:08 UTC 2010 Distributing source for FreeBSD/amd64 7.2-RELEASE
-Tue Sep 28 19:04:31 PDT 2010 Moving components into staging area for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 19:04:46 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 19:04:51 PDT 2009 Indexing world1 for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 19:08:04 PDT 2009 Locating build stamps for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 19:10:19 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 19:10:19 PDT 2009 Preparing to copy files into staging area for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 19:10:20 PDT 2009 Copying data files into staging area for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 12:16:57 PDT 2009 Copying metadata files into staging area for FreeBSD/amd64 7.2-RELEASE
-Mon Aug 24 12:16:59 PDT 2009 Constructing metadata index and tag for FreeBSD/amd64 7.2-RELEASE
-
-Files found which include build stamps:
-kernel|generic|/GENERIC/hptrr.ko
-kernel|generic|/GENERIC/kernel
-world|base|/boot/loader
-world|base|/boot/pxeboot
-world|base|/etc/mail/freebsd.cf
-world|base|/etc/mail/freebsd.submit.cf
-world|base|/etc/mail/sendmail.cf
-world|base|/etc/mail/submit.cf
-world|base|/lib/libcrypto.so.5
-world|base|/usr/bin/ntpq
-world|base|/usr/include/osreldate.h
-world|base|/usr/lib/libalias.a
-world|base|/usr/lib/libalias_cuseeme.a
-world|base|/usr/lib/libalias_dummy.a
-world|base|/usr/lib/libalias_ftp.a
-...
-....
+[[patch]]
+=== Building a Patch Release
-Finally, the build completes.
+When a link:https://www.FreeBSD.org/security/advisories/[security advisory] or link:https://www.FreeBSD.org/security/notices/[errata notice] comes out, the fix lands on the `releng` branch and the patch level in [.filename]#sys/conf/newvers.sh# is increased by one.
+Updating the tree and building again is all it takes:
[source,shell]
....
-Values of build stamps, excluding library archive headers:
-v1.2 (Aug 25 2009 00:40:36)
-v1.2 (Aug 25 2009 00:38:22)
-@()FreeBSD 7.2-RELEASE 0: Tue Aug 25 00:38:29 UTC 2009
-FreeBSD 7.2-RELEASE 0: Tue Aug 25 00:38:29 UTC 2009
- root@server.myhost.com:/usr/obj/usr/src/sys/GENERIC
-7.2-RELEASE
-Mon Aug 24 23:55:25 UTC 2009
-Mon Aug 24 23:55:25 UTC 2009
- built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009
- built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009
- built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009
- built by root@server.myhost.com on Tue Aug 25 00:16:15 UTC 2009
-Mon Aug 24 23:46:47 UTC 2009
-ntpq 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1)
- * Copyright (c) 1992-2009 The FreeBSD Project.
-Mon Aug 24 23:46:47 UTC 2009
-Mon Aug 24 23:55:40 UTC 2009
-Aug 25 2009
-ntpd 4.2.4p5-a Mon Aug 24 23:55:52 UTC 2009 (1)
-ntpdate 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1)
-ntpdc 4.2.4p5-a Mon Aug 24 23:55:53 UTC 2009 (1)
-Tue Aug 25 00:21:21 UTC 2009
-Tue Aug 25 00:21:21 UTC 2009
-Tue Aug 25 00:21:21 UTC 2009
-Mon Aug 24 23:46:47 UTC 2009
-
-FreeBSD/amd64 7.2-RELEASE initialization build complete. Please
-review the list of build stamps printed above to confirm that
-they look sensible, then run
- sh -e approve.sh amd64 7.2-RELEASE
-to sign the release.
+# cd /usr/src
+# git pull
+# make -j "$(sysctl -n hw.ncpu)" buildworld buildkernel
+# make update-packages PKG_REPO_SIGNING_KEY=eddsa:/root/pkgbase-keys/repo.key
....
-Approve the build if everything is correct.
-More information on determining this can be found in the distributed source file named [.filename]#USAGE#. Execute [.filename]#scripts/approve.sh#, as directed.
-This will sign the release, and move components into a staging area suitable for uploading.
+This produces a new package set, such as [.filename]#15.2p1/#, and moves [.filename]#latest# to it.
-[source,shell]
-....
-# cd /usr/local/freebsd-update-server
-# sh scripts/mountkey.sh
-....
+This is where `update-packages` earns its name.
+It compares each new package with the package of the same name in the previous set.
+If the contents are identical, it throws the new one away and copies in the old one, old version number and all.
+As a result, clients only download the packages a patch actually touched, not the whole base system.
-[source,shell]
-....
-# sh -e scripts/approve.sh amd64 7.2-RELEASE
-Wed Aug 26 12:50:06 PDT 2009 Signing build for FreeBSD/amd64 7.2-RELEASE
-Wed Aug 26 12:50:06 PDT 2009 Copying files to patch source directories for FreeBSD/amd64 7.2-RELEASE
-Wed Aug 26 12:50:06 PDT 2009 Copying files to upload staging area for FreeBSD/amd64 7.2-RELEASE
-Wed Aug 26 12:50:07 PDT 2009 Updating databases for FreeBSD/amd64 7.2-RELEASE
-Wed Aug 26 12:50:07 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.2-RELEASE
-....
+That comparison only works because builds are reproducible, which is the default.
+Leave `WITHOUT_REPRODUCIBLE_BUILD` out of man:src.conf[5]; otherwise every package looks different on every build.
-After the approval process is complete, the upload procedure may be started.
+Each package set is complete on its own, so old ones can be deleted once no client needs them anymore.
-[source,shell]
-....
-# cd /usr/local/freebsd-update-server
-# sh scripts/upload.sh amd64 7.2-RELEASE
-....
+[[local-changes]]
+=== Building Local Changes
-[NOTE]
-====
-In the event update code needs to be re-uploaded, this may be done by changing to the public distributions directory for the target release and updating attributes of the _uploaded_ file.
+There is one catch when patching the tree locally.
+`update-packages` takes the package version from the release and patch level, so a locally modified build at the same patch level gets the same version as the set before it, and clients will not install it.
+To publish a local change, set `PKG_VERSION` to something man:pkg[8] sorts after the current version.
+`pkg version -t` is a quick way to check:
[source,shell]
....
-# cd /usr/local/freebsd-update-server/pub/7.2-RELEASE/amd64
-# touch -t 200801010101.01 uploaded
+% pkg version -t 15.2p1 15.2p1.1
+<
+# make update-packages PKG_VERSION=15.2p1.1 PKG_REPO_SIGNING_KEY=eddsa:/root/pkgbase-keys/repo.key
....
-====
+The next official patch level, `15.2p2`, still sorts after `15.2p1.1`, so clients return to the regular sequence on their own.
-The uploaded files will need to be in the document root of the webserver in order for updates to be distributed.
-The exact configuration will vary depending on the web server used.
-For the Apache web server, please refer to the extref:{handbook}network-servers[Configuration of Apache servers, network-apache] section in the Handbook.
-
-Update client's `KeyPrint` and `ServerName` in [.filename]#/etc/freebsd-update.conf#, and perform updates as instructed in the extref:{handbook}cutting-edge[FreeBSD Update, updating-upgrading-freebsdupdate] section of the Handbook.
-
-[IMPORTANT]
+[WARNING]
====
-In order for FreeBSD Update Server to work properly, updates for both the _current_ release and the release _one wants to upgrade to_ need to be built.
-This is necessary for determining the differences of files between releases.
-For example, when upgrading a FreeBSD system from 7.1-RELEASE to 7.2-RELEASE, updates will need to be built and uploaded to your distribution server for both versions.
+Check where every applied patch comes from.
+Signing the repository proves that the packages came from the build host; it says nothing about whether the source they were built from deserves trust.
====
-For reference, the entire run of link:../../source/articles/freebsd-update-server/init.txt[init.sh] is attached.
+[[publish]]
+== Publishing the Repository
-[[patch]]
-== Building a Patch
-
-Every time a link:https://www.FreeBSD.org/security/advisories/[security advisory] or link:https://www.FreeBSD.org/security/notices/[security notice] is announced, a patch update can be built.
-
-For this example, 7.1-RELEASE will be used.
-
-A couple of assumptions are made for a different release build:
-
-* Setup the correct directory structure for the initial build.
-* Perform an initial build for 7.1-RELEASE.
-
-Create the patch directory of the respective release under [.filename]#/usr/local/freebsd-update-server/patches/#.
+The build host and the web server do not have to be the same machine, and it is usually better if they are not.
+Copy the repository to the web server's document root, keeping the [.filename]#latest# link intact.
+package:net/rsync[] does this well:
[source,shell]
....
-% mkdir -p /usr/local/freebsd-update-server/patches/7.1-RELEASE/
-% cd /usr/local/freebsd-update-server/patches/7.1-RELEASE
+# rsync -a --delete --delay-updates /usr/obj/usr/src/repo/ www@pkg.example.org:/usr/local/www/pkgbase/
....
-As an example, take the patch for man:named[8].
-Read the advisory, and grab the necessary file from link:https://www.FreeBSD.org/security/advisories/[FreeBSD Security Advisories].
-More information on interpreting the advisory, can be found in the extref:{handbook}security[FreeBSD Handbook, security-advisories].
+`--delay-updates` holds the new files back until the end of the transfer, so a client that updates halfway through does not see a half-copied package set.
-In the https://security.freebsd.org/advisories/FreeBSD-SA-09:12.bind.asc[security brief], this advisory is called `SA-09:12.bind`.
-After downloading the file, it is required to rename the file to an appropriate patch level.
-It is suggested to keep this consistent with official FreeBSD patch levels, but its name may be freely chosen.
-For this build, let us follow the currently established practice of FreeBSD and call this `p7`. Rename the file:
+The web server only has to serve static files.
+A minimal package:www/nginx[] configuration looks like this:
-[source,shell]
-....
-% cd /usr/local/freebsd-update-server/patches/7.1-RELEASE/; mv bind.patch 7-SA-09:12.bind
+[.programlisting]
....
+server {
+ listen 443 ssl;
+ server_name pkg.example.org;
-[NOTE]
-====
-When running a patch level build, it is assumed that previous patches are in place.
-When a patch build is run, it will run all patches contained in the patch directory.
+ ssl_certificate /usr/local/etc/nginx/pkg.example.org.crt;
+ ssl_certificate_key /usr/local/etc/nginx/pkg.example.org.key;
-There can be custom patches added to any build. Use the number zero, or any other number.
-====
+ root /usr/local/www/pkgbase;
+}
+....
-[WARNING]
-====
+Use HTTPS even though the catalogue is signed.
+A signature tells a client who produced a catalogue, not whether it is the newest one.
+Without HTTPS, an attacker on the network could keep serving an older catalogue, perfectly validly signed, that is missing the latest security fixes.
-It is up to the administrator of the FreeBSD Update Server to take appropriate measures to verify the authenticity of every patch.
-====
+[[clients]]
+== Configuring Clients
-At this point, a _diff_ is ready to be built.
-The software checks first to see if a [.filename]#scripts/init.sh# has been run on the respective release prior to running the diff build.
+On each client, install the public key:
[source,shell]
....
-# cd /usr/local/freebsd-update-server
-# sh scripts/diff.sh amd64 7.1-RELEASE 7
+# mkdir -p /usr/local/etc/pkg/keys
+# install -m 0444 repo.pub /usr/local/etc/pkg/keys/pkgbase-local.pub
....
-What follows is a sample of a _differential_ build run.
+Then create [.filename]#/usr/local/etc/pkg/repos/FreeBSD-base.conf#:
-[source,shell]
+[.programlisting]
....
-# sh -e scripts/diff.sh amd64 7.1-RELEASE 7
-Wed Aug 26 10:09:59 PDT 2009 Extracting world+src for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 17:10:25 UTC 2009 Building world for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 18:05:11 UTC 2009 Distributing world for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 18:06:16 UTC 2009 Building and distributing kernels for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 18:17:50 UTC 2009 Constructing world components for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 18:18:02 UTC 2009 Distributing source for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 11:19:23 PDT 2009 Moving components into staging area for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 11:19:37 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 11:19:42 PDT 2009 Indexing world0 for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 11:23:02 PDT 2009 Extracting world+src for FreeBSD/amd64 7.1-RELEASE-p7
-Thu Sep 30 18:23:29 UTC 2010 Building world for FreeBSD/amd64 7.1-RELEASE-p7
-Thu Sep 30 19:18:15 UTC 2010 Distributing world for FreeBSD/amd64 7.1-RELEASE-p7
-Thu Sep 30 19:19:18 UTC 2010 Building and distributing kernels for FreeBSD/amd64 7.1-RELEASE-p7
-Thu Sep 30 19:30:52 UTC 2010 Constructing world components for FreeBSD/amd64 7.1-RELEASE-p7
-Thu Sep 30 19:31:03 UTC 2010 Distributing source for FreeBSD/amd64 7.1-RELEASE-p7
-Thu Sep 30 12:32:25 PDT 2010 Moving components into staging area for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:32:39 PDT 2009 Extracting extra docs for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:32:43 PDT 2009 Indexing world1 for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:35:54 PDT 2009 Locating build stamps for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:36:58 PDT 2009 Reverting changes due to build stamps for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:37:14 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:37:14 PDT 2009 Preparing to copy files into staging area for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:37:15 PDT 2009 Copying data files into staging area for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:43:23 PDT 2009 Copying metadata files into staging area for FreeBSD/amd64 7.1-RELEASE-p7
-Wed Aug 26 12:43:25 PDT 2009 Constructing metadata index and tag for FreeBSD/amd64 7.1-RELEASE-p7
-...
-Files found which include build stamps:
-kernel|generic|/GENERIC/hptrr.ko
-kernel|generic|/GENERIC/kernel
-world|base|/boot/loader
-world|base|/boot/pxeboot
-world|base|/etc/mail/freebsd.cf
-world|base|/etc/mail/freebsd.submit.cf
-world|base|/etc/mail/sendmail.cf
-world|base|/etc/mail/submit.cf
-world|base|/lib/libcrypto.so.5
-world|base|/usr/bin/ntpq
-world|base|/usr/include/osreldate.h
-world|base|/usr/lib/libalias.a
-world|base|/usr/lib/libalias_cuseeme.a
-world|base|/usr/lib/libalias_dummy.a
-world|base|/usr/lib/libalias_ftp.a
-...
-Values of build stamps, excluding library archive headers:
-v1.2 (Aug 26 2009 18:13:46)
-v1.2 (Aug 26 2009 18:11:44)
-@()FreeBSD 7.1-RELEASE-p7 0: Wed Aug 26 18:11:50 UTC 2009
-FreeBSD 7.1-RELEASE-p7 0: Wed Aug 26 18:11:50 UTC 2009
- root@server.myhost.com:/usr/obj/usr/src/sys/GENERIC
-7.1-RELEASE-p7
-Wed Aug 26 17:29:15 UTC 2009
-Wed Aug 26 17:29:15 UTC 2009
- built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009
- built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009
- built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009
- built by root@server.myhost.com on Wed Aug 26 17:49:58 UTC 2009
-Wed Aug 26 17:20:39 UTC 2009
-ntpq 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1)
- * Copyright (c) 1992-2009 The FreeBSD Project.
-Wed Aug 26 17:20:39 UTC 2009
-Wed Aug 26 17:29:30 UTC 2009
-Aug 26 2009
-ntpd 4.2.4p5-a Wed Aug 26 17:29:41 UTC 2009 (1)
-ntpdate 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1)
-ntpdc 4.2.4p5-a Wed Aug 26 17:29:42 UTC 2009 (1)
-Wed Aug 26 17:55:02 UTC 2009
-Wed Aug 26 17:55:02 UTC 2009
-Wed Aug 26 17:55:02 UTC 2009
-Wed Aug 26 17:20:39 UTC 2009
-...
+FreeBSD-base: {
+ url: "https://pkg.example.org/${ABI}/latest",
+ mirror_type: "none",
+ signature_type: "pubkey",
+ pubkey: "/usr/local/etc/pkg/keys/pkgbase-local.pub",
+ enabled: yes
+}
....
-Updates are printed, and approval is requested.
+The trick here is reusing the `FreeBSD-base` repository name from [.filename]#/etc/pkg/FreeBSD.conf#.
+These settings replace the official ones, so everything the Handbook says about `FreeBSD-base` works unchanged against the local repository.
+man:pkg[8] fills in `+${ABI}+` itself, for example with `FreeBSD:15:amd64`.
+
+Make sure the client accepts the signature:
[source,shell]
....
-New updates:
-kernel|generic|/GENERIC/kernel.symbols|f|0|0|0555|0|7c8dc176763f96ced0a57fc04e7c1b8d793f27e006dd13e0b499e1474ac47e10|
-kernel|generic|/GENERIC/kernel|f|0|0|0555|0|33197e8cf15bbbac263d17f39c153c9d489348c2c534f7ca1120a1183dec67b1|
-kernel|generic|/|d|0|0|0755|0||
-src|base|/|d|0|0|0755|0||
-src|bin|/|d|0|0|0755|0||
-src|cddl|/|d|0|0|0755|0||
-src|contrib|/contrib/bind9/bin/named/update.c|f|0|10000|0644|0|4d434abf0983df9bc47435670d307fa882ef4b348ed8ca90928d250f42ea0757|
-src|contrib|/contrib/bind9/lib/dns/openssldsa_link.c|f|0|10000|0644|0|c6805c39f3da2a06dd3f163f26c314a4692d4cd9a2d929c0acc88d736324f550|
-src|contrib|/contrib/bind9/lib/dns/opensslrsa_link.c|f|0|10000|0644|0|fa0f7417ee9da42cc8d0fd96ad24e7a34125e05b5ae075bd6e3238f1c022a712|
-...
-FreeBSD/amd64 7.1-RELEASE update build complete. Please review
-the list of build stamps printed above and the list of updated
-files to confirm that they look sensible, then run
- sh -e approve.sh amd64 7.1-RELEASE
-to sign the build.
+# pkg update -f -r FreeBSD-base
....
-Follow the same process as noted before for approving a build:
+A host that still uses distribution sets has to be converted to base system packages first; the extref:{handbook}cutting-edge[Handbook, pkgbase] explains how.
+From then on, updating from the local repository is one command:
[source,shell]
....
-# sh -e scripts/approve.sh amd64 7.1-RELEASE
-Wed Aug 26 12:50:06 PDT 2009 Signing build for FreeBSD/amd64 7.1-RELEASE
-Wed Aug 26 12:50:06 PDT 2009 Copying files to patch source directories for FreeBSD/amd64 7.1-RELEASE
-Wed Aug 26 12:50:06 PDT 2009 Copying files to upload staging area for FreeBSD/amd64 7.1-RELEASE
-Wed Aug 26 12:50:07 PDT 2009 Updating databases for FreeBSD/amd64 7.1-RELEASE
-Wed Aug 26 12:50:07 PDT 2009 Cleaning staging area for FreeBSD/amd64 7.1-RELEASE
-
-The FreeBSD/amd64 7.1-RELEASE update build has been signed and is
-ready to be uploaded. Remember to run
- sh -e umountkey.sh
-to unmount the decrypted key once you have finished signing all
-the new builds.
+# pkg upgrade -r FreeBSD-base
....
-After approving the build, upload the software:
+[[major-upgrades]]
+== Major Upgrades
-[source,shell]
-....
-# cd /usr/local/freebsd-update-server
-# sh scripts/upload.sh amd64 7.1-RELEASE
-....
+Package sets are stored per ABI, so one repository can hold several major versions side by side.
+To prepare for an upgrade, build the target branch, for example `releng/16.0`, in its own source tree and object directory, and publish it to the same document root.
+Its packages end up under [.filename]#FreeBSD:16:amd64/#.
-For reference, the entire run of link:../../source/articles/freebsd-update-server/diff.txt[diff.sh] is attached.
+Clients switch to the new major version by overriding `ABI`, as described in extref:{handbook}cutting-edge[Major Upgrades, pkgbase-major-upgrade].
+Because the client configuration already uses `+${ABI}+` in its URL, there is nothing else to change.
[[tips]]
== Tips
-* If a custom release is built using the native `make release` extref:{releng}[procedure, release-build], freebsd-update-server code will work from your release. As an example, a release without ports or documentation can be built by clearing functionality pertaining to documentation subroutines `findextradocs ()`, `addextradocs ()` and altering the download location in `fetchiso ()`, respectively, in [.filename]#scripts/build.subr#. As a last step, change the man:sha256[1] hash in [.filename]#build.conf# under your respective release and architecture and you are ready to build off your custom release.
+* To package custom kernels, list them in `KERNCONF` for both `buildkernel` and `update-packages`.
+Each kernel becomes a package named after its configuration file, such as `FreeBSD-kernel-mykernel`.
+
-[.programlisting]
+[source,shell]
....
-# Compare ${WORKDIR}/release and ${WORKDIR}/$1, identify which parts
-# of the world|doc subcomponent are missing from the latter, and
-# build a tarball out of them.
-findextradocs () {
-}
-# Add extra docs to ${WORKDIR}/$1
-addextradocs () {
-}
+# make -j "$(sysctl -n hw.ncpu)" buildworld buildkernel KERNCONF="GENERIC MYKERNEL"
+# make update-packages KERNCONF="GENERIC MYKERNEL" PKG_REPO_SIGNING_KEY=eddsa:/root/pkgbase-keys/repo.key
....
-* Adding `-j _NUMBER_` flags to `buildworld` and `obj` targets in the [.filename]#scripts/build.subr# script may speed up processing depending on the hardware used, however it is not necessary. Using these flags in other targets is not recommended, as it may cause the build to become unreliable.
-+
-[.programlisting]
-....
- # Build the world
- log "Building world"
- cd /usr/src &&
- make -j 2 ${COMPATFLAGS} buildworld 2>&1
- # Distribute the world
- log "Distributing world"
- cd /usr/src/release &&
- make -j 2 obj &&
- make ${COMPATFLAGS} release.1 release.2 2>&1
-....
+* package:devel/ccache[] makes rebuilds after small patches a lot faster.
-* Create an appropriate extref:{handbook}network-servers[DNS, network-dns] SRV record for the update server, and put others behind it with variable weights. Using this facility will provide update mirrors, however this tip is not necessary unless you wish to provide a redundant service.
+* To spread the load over several mirrors, publish an extref:{handbook}network-servers[DNS, network-dns] SRV record and set `mirror_type: "srv"` with a `pkg+https://` URL in the client configuration, as man:pkg.conf[5] describes.
+
[.programlisting]
....
-_http._tcp.update.myserver.com. IN SRV 0 2 80 host1.myserver.com.
- IN SRV 0 1 80 host2.myserver.com.
- IN SRV 0 0 80 host3.myserver.com.
+_https._tcp.pkg.example.org. IN SRV 0 2 443 host1.example.org.
+ IN SRV 0 1 443 host2.example.org.
....
+
+[[acknowledgments]]
+== Acknowledgments
+
+An earlier version of this article, written by Jason Helfman, described running a private man:freebsd-update[8] server with freebsd-update-build.
+That version was also printed in https://people.freebsd.org/~jgh/files/fus/BSD_03_2010_EN.pdf[BSD Magazine].

File Metadata

Mime Type
text/plain
Expires
Thu, Nov 19, 4:25 PM (21 h, 3 m)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
41746000
Default Alt Text
D61505.1795105548.diff (38 KB)

Event Timeline