Index: head/en_US.ISO8859-1/articles/Makefile =================================================================== --- head/en_US.ISO8859-1/articles/Makefile (revision 50118) +++ head/en_US.ISO8859-1/articles/Makefile (revision 50119) @@ -1,44 +1,45 @@ # $FreeBSD$ SUBDIR = SUBDIR+= bsdl-gpl SUBDIR+= building-products SUBDIR+= committers-guide SUBDIR+= contributing SUBDIR+= contributors SUBDIR+= cups SUBDIR+= explaining-bsd SUBDIR+= filtering-bridges SUBDIR+= fonts SUBDIR+= freebsd-questions +SUBDIR+= freebsd-releng SUBDIR+= freebsd-update-server SUBDIR+= geom-class SUBDIR+= gjournal-desktop SUBDIR+= hubs SUBDIR+= ipsec-must SUBDIR+= ldap-auth SUBDIR+= leap-seconds SUBDIR+= linux-emulation SUBDIR+= linux-users SUBDIR+= mailing-list-faq SUBDIR+= nanobsd SUBDIR+= new-users SUBDIR+= p4-primer SUBDIR+= pam SUBDIR+= pgpkeys SUBDIR+= port-mentor-guidelines SUBDIR+= pr-guidelines SUBDIR+= problem-reports SUBDIR+= rc-scripting SUBDIR+= relaydelay SUBDIR+= releng SUBDIR+= remote-install SUBDIR+= serial-uart SUBDIR+= solid-state SUBDIR+= vinum SUBDIR+= vm-design # ROOT_SYMLINKS+= new-users DOC_PREFIX?= ${.CURDIR}/../.. .include "${DOC_PREFIX}/share/mk/doc.project.mk" Index: head/en_US.ISO8859-1/articles/freebsd-releng/Makefile =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-releng/Makefile (nonexistent) +++ head/en_US.ISO8859-1/articles/freebsd-releng/Makefile (revision 50119) @@ -0,0 +1,20 @@ +# +# $FreeBSD$ +# +# Article: FreeBSD Release Engineering + +DOC?= article + +FORMATS?= html html-split + +INSTALL_COMPRESSED?=gz +INSTALL_ONLY_COMPRESSED?= + +SRCS= article.xml + +CSS_SHEET_ADDITIONS=extra.css + +URL_RELPREFIX?= ../../../.. +DOC_PREFIX?= ${.CURDIR}/../../.. + +.include "${DOC_PREFIX}/share/mk/doc.project.mk" Property changes on: head/en_US.ISO8859-1/articles/freebsd-releng/Makefile ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Index: head/en_US.ISO8859-1/articles/freebsd-releng/article.xml =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-releng/article.xml (nonexistent) +++ head/en_US.ISO8859-1/articles/freebsd-releng/article.xml (revision 50119) @@ -0,0 +1,415 @@ + + + + + + + +head/"> +stable/"> +stable/11/"> +releng/"> +releng/11.0/"> +release/11.0.0/"> +11.0"> + + + + + + + +]> +
+ + + &os; Release Engineering + + + &tm-attrib.freebsd; + &tm-attrib.intel; + &tm-attrib.general; + + + $FreeBSD$ + + + This article describes the release engineering process of + the &os; Project. + + + + + Introduction to the &os; Release Engineering + Process + + Development of &os; has a very specific workflow. In + general, all changes to the &os; base system are committed to + the &branch.head; branch, which reflects the top of the source + tree. + + After a reasonable testing period, changes can then be + merged to the &branch.stable; branches. The default minimum + timeframe before merging to &branch.stable; branches is three + (3) days. + + Although a general rule to wait a minimum of three days + before merging from &branch.head;, there are a few special + circumstances where an immediate merge may be necessary, such as + a critical security fix, or a bug fix that directly inhibits the + release build process. + + After several months, and the number of changes in the + &branch.stable; branch have grown significantly, it is time to + release the next version of &os;. These releases have been + historically referred to as point + releases. + + In between releases from the &branch.stable; branches, + approximately every two (2) years, a release will be cut + directly from &branch.head;. These releases have been + historically referred to as dot-zero + releases. + + This article will highlight the workflow and + responsibilities of the &team.re; for both + dot-zero and point' + releases. + + The following sections of this article describe: + + + + + + + General information and preparation before + starting the release cycle. + + + + + + + + Terminology and general information, such as the + code slush and code + freeze, used throughout this document. + + + + + + + + The Release Engineering process for a + dot-zero release. + + + + + + + + The Release Engineering process for a + point release. + + + + + + + + Information related to the specific procedures to + build installation medium. + + + + + + + + Procedures to publish installation medium. + + + + + + + + Wrapping up the release cycle. + + + + + + + General Information and Preparation + + Approximately two months before the start of the release + cycle, the &team.re; decides on a schedule for the release. + The schedule includes the various milestone points of the + release cycle, such as freeze dates, branch dates, and build + dates. For example: + + + + + + Milestone + Anticipated Date + + + + + + &branch.head; slush: + May 27, 2016 + + + + &branch.head; freeze: + June 10, 2016 + + + + &branch.head; KBI freeze: + June 24, 2016 + + + + doc/ tree slush [1]: + June 24, 2016 + + + + Ports quarterly branch [2]: + July 1, 2016 + + + + &branch.stablex; branch: + July 8, 2016 + + + + doc/ tree tag [3]: + July 8, 2016 + + + + BETA1 build starts: + July 8, 2016 + + + + &branch.head; thaw: + July 9, 2016 + + + + BETA2 build starts: + July 15, 2016 + + + + BETA3 build starts [*]: + July 22, 2016 + + + + &branch.relengx; branch: + July 29, 2016 + + + + RC1 build starts: + July 29, 2016 + + + + &branch.stablex; thaw: + July 30, 2016 + + + + RC2 build starts: + August 5, 2016 + + + + Final Ports package builds [4]: + August 6, 2016 + + + + Ports release tag: + August 12, 2016 + + + + RC3 build starts [*]: + August 12, 2016 + + + + RELEASE build starts: + August 19, 2016 + + + + RELEASE announcement: + September 2, 2016 + + + + + + + Items marked with "[*]" are "as + needed". + + + + + The doc/ tree slush is coordinated by + the &team.doceng;. + + + + The Ports quarterly branch used is determined by when + the final RC build is planned. A new + quarterly branch is created on the first day of the quarter, + so this metric should be used when taking the release cycle + milestones into account. The quarterly branch is created by + the &team.portmgr;. + + + + The doc/ tree is tagged by the + &team.doceng;. + + + + The final Ports package build is done by the + &team.portmgr; after the final (or what is expected to be + final) RC build. + + + + + If the release is being created from an existing + &branch.stable; branch, the KBI + freeze date can be excluded, since the KBI + is already considered frozen on established + &branch.stable; branches. + + + When writing the release cycle schedule, a number of things + need to be taken into consideration, in particular milestones + where the target date depends on predefined milestones upon + which there is a dependency. For example, the Ports Collection + release tag originates from the active quarterly branch at the + time of the last RC. This in part defines + which quarterly branch is used, when the release tag can happen, + and what revision of the ports tree is used for the final + RELEASE build. + + After general agreement on the schedule, the &team.re; + emails the schedule to the &os; Developers. + + It is somewhat typical that many developers will inform + the &team.re; about various works-in-progress. In some cases, + an extension for the in-progress work will be requested, and + in other cases, a request for blanket approval + to a particular subset of the tree will be made. + + When such requests are made, it is important to make sure + timelines (even if estimated) are discussed. For blanket + approvals, the length of time for the blanket approval should + be made clear. For example, a &os; developer may request + blanket approvals from the start of the code slush until the + start of the RC builds. + + Depending on the underlying set of code in question, and + the overall impact the set of code has on &os; as a whole, such + requests may be approved or denied by the &team.re;. + + The same applies to work-in-progress extensions. For + example, in-progress work for a new device driver that is + otherwise isolated from the rest of the tree may be granted + an extension. A new scheduler, however, may not be feasible, + especially if such dramatic changes do not exist in another + branch. + + The schedule is also added to the Project website, in the + doc/ repository, in + head/en_US.ISO8859-1/htdocs/releases/&branch.revision;R/schedule.xml. + This file is continuously updated as the release cycle + progresses. + + + In most cases, the schedule.xml can + be copied from a prior release and updated accordingly. + + + The schedule is also linked from + head/en_US.ISO8859-1/htdocs/releng/index.xml. + + Approximately one month prior to the scheduled code + slush, the &team.re; sends a reminder email to the + &os; Developers. + + Once the first builds of the release cycle are available, + update the beta.local.where entity in + head/en_US.ISO8859-1/htdocs/releases/&branch.revision;R/schedule.xml. + replacing IGNORE with + INCLUDE. + + + If two parallel release cycles are happening at once, the + beta2.local.where entity may be used + instead. + + + + &release.terminology; + &release.major.version; + &release.minor.version; + &release.building; + &release.mirrors; + + + Wrapping up the Release Cycle + + This section describes general post-release tasks. + + + Post-Release Errata Notices + + As the release cycle approaches conclusion, it is common + to have several EN (Errata Notice) + candidates to address issues that were discovered late in the + cycle. Following the release, the &team.re; and the + &team.secteam; revisit changes that were not approved prior to + the final release, and depending on the scope of the change in + question, may issue an EN. + + + + Handoff to the &team.secteam; + + Roughly two weeks following the release, the Release + Engineer updates svnadmin/conf/approvers + changing the approver column from re to + (so|security-officer) for the + &branch.relengx; branch. + + + +
Property changes on: head/en_US.ISO8859-1/articles/freebsd-releng/article.xml ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/xml \ No newline at end of property Index: head/en_US.ISO8859-1/articles/freebsd-releng/extra.css =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-releng/extra.css (nonexistent) +++ head/en_US.ISO8859-1/articles/freebsd-releng/extra.css (revision 50119) @@ -0,0 +1,7 @@ +/* + * $FreeBSD$ + */ + +DIV.TITLEPAGE { + text-align: center; +} Property changes on: head/en_US.ISO8859-1/articles/freebsd-releng/extra.css ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Index: head/en_US.ISO8859-1/articles/freebsd-releng/releng-building.xml =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-releng/releng-building.xml (nonexistent) +++ head/en_US.ISO8859-1/articles/freebsd-releng/releng-building.xml (revision 50119) @@ -0,0 +1,247 @@ + + + + Building &os; Installation Media + + This section describes the general procedures producing &os; + development snapshots and releases. + + + Release Build Scripts + + This section describes the build scripts used by &team.re; + to produce development snapshots and releases. + + + The <filename>release.sh</filename> Script + + Prior to &os; 9.0-RELEASE, + src/release/Makefile was updated to + support &man.bsdinstall.8;, and the + src/release/generate-release.sh script + was introduced as a wrapper to automate invoking the + &man.release.7; targets. + + Prior to &os; 9.2-RELEASE, + src/release/release.sh was introduced, + which heavily based on + src/release/generate-release.sh included + support to specify configuration files to override various + options and environment variables. Support for configuration + files provided support for cross building each architecture + for a release by specifying a separate configuration file for + each invocation. + + As a brief example of using + src/release/release.sh to build a single + release in /scratch: + + &prompt.root; /bin/sh /usr/src/release/release.sh + + As a brief example of using + src/release/release.sh to build a single, + cross-built release using a different target directory, create + a custom release.conf containing: + + # release.sh configuration for powerpc/powerpc64 +CHROOTDIR="/scratch-powerpc64" +TARGET="powerpc" +TARGET_ARCH="powerpc64" +KERNEL="GENERIC64" + + Then invoke src/release/release.sh + as: + + &prompt.root; /bin/sh /usr/src/release/release.sh -c $HOME/release.conf + + See &man.release.7; and + src/release/release.conf.sample for more + details and example usage. + + + + The <filename>thermite.sh</filename> Wrapper + Script + + In order to make cross building the full set of + architectures supported on a given branch faster, easier, and + reduce human error factors, a wrapper script around + src/release/release.sh was written to + iterate through the various combinations of architectures and + invoke src/release/release.sh using + a configuration file specific to that architecture. + + The wrapper script is called + thermite.sh, which is available in the + &os; Subversion repository at + svn://svn.freebsd.org/base/user/gjb/thermite/, + in addition to configuration files used to build + &branch.head; and &branch.stablex; development + snapshots. + + Using thermite.sh is covered in and . + + Each architecture and individual kernel have their own + configuration file used by release.sh. + Each branch has its own defaults-X.conf + configuration which contains entries common throughout each + architecture, where overrides or special variables are set + and/or overridden in the per-build files. + + The per-build configuration file naming scheme is in the + form of + ${revision}-${TARGET_ARCH}-${KERNCONF}-${type}.conf, + where the uppercase variables are equivalent to what + &man.make.1; uses in the build system, and lowercase variables + are set within the configuration files, mapping to the major + version of the respective branch. + + Each branch also has its own + builds-X.conf configuration, which is + used by thermite.sh. The + thermite.sh script iterates through each + ${revision}, ${TARGET_ARCH}, + ${KERNCONF}, and ${type} value, creating + a master list of what to build. However, a given + combination from the list will only be built if the + respective configuration file exists, which is where the + naming scheme above is relevant. + + There are two paths of file sourcing: + + + + builds-11.conf + -> main.conf + This controls thermite.sh + behavior + + + + 11-amd64-GENERIC-snap.conf + -> + defaults-11.conf + -> main.conf + This controls release/release.sh + behavior within the build &man.chroot.8; + + + + + The + builds-11.conf, + defaults-11.conf, + and main.conf configuration files exist + to reduce repetition between the various per-build + files. + + + + + + Building &os; Development Snapshots + + The official release build machines have a specific + filesystem layout, which using ZFS, + thermite.sh takes heavy advantage of with + clones and snapshots, ensuring a pristine build + environment. + + The build scripts reside in /releng/scripts-snapshot/scripts + or /releng/scripts-release/scripts + respectfully, to avoid collisions between an + RC build from a releng branch versus + a STABLE snapshot from the respective stable + branch. + + A separate dataset exists for the final build images, + /snap/ftp. This + directory contains both snapshots and releases directories. + They are only used if the EVERYTHINGISFINE + variable is defined in main.conf. + + + The EVERYTHINGISFINE variable name was + chosen to avoid colliding with a variable that might be + possibly set in the user environment, accidentally enabling + the behavior that depends on it being defined. + + + As thermite.sh iterates through the + master list of combinations and locates the per-build + configuration file, a ZFS dataset is created + under /releng, such as + /releng/12-amd64-GENERIC-snap. + The src/, ports/, and + doc/ trees are checked out to separate + ZFS datasets, such as /releng/12-src-snap, which are + then cloned and mounted into the respective build datasets. + This is done to avoid checking out a given tree more than + once. + + Assuming these filesystem paths, + thermite.sh would be invoked as: + + &prompt.root; cd /releng/scripts-snapshot/scripts +&prompt.root; ./setrev.sh -b &branch.stablex; +&prompt.root; ./zfs-setup.sh -c ./builds-11.conf +&prompt.root; ./thermite.sh -c ./builds-11.conf + + + + Building &os; Releases + + Similar to building &os; development snapshots, + thermite.sh would be invoked the same way. + The difference between development snapshots and release builds, + BETA and RC, included, is + that the &man.chroot.8; configuration files must be named with + release instead of snap as + the "type", as mentioned above. + + In addition, the BUILDTYPE and + types must be changed from + snap to release in + defaults-11.conf + and + builds-11.conf, + respectively. + + When building BETA, + RC, and the final RELEASE, + also statically set BUILDSVNREV to the + revision on the branch reflecting the name change, + BUILDDATE to the date the builds are started + in YYYYMMDD format. If the + doc/ and ports/ trees have + been tagged, also set PORTBRANCH and + DOCBRANCH to the relevant tag path in the + Subversion repository, replacing HEAD with + the last changed revision. Also set + releasesrc in + builds-11.conf + to the relevant branch, such as &branch.stablex; or + &branch.relengx;. + + After building the final RELEASE, the + &branch.relengx; branch is tagged as &branch.releasex; using the + revision from which the RELEASE was built. + Similar to creating the &branch.stablex; and &branch.relengx; + branches, this is done with svn cp. From the + repository root: + + &prompt.user; svn cp ^/&branch.relengx;@r306420 &branch.releasex; +&prompt.user; svn commit &branch.releasex; + + Property changes on: head/en_US.ISO8859-1/articles/freebsd-releng/releng-building.xml ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/xml \ No newline at end of property Index: head/en_US.ISO8859-1/articles/freebsd-releng/releng-major-version.xml =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-releng/releng-major-version.xml (nonexistent) +++ head/en_US.ISO8859-1/articles/freebsd-releng/releng-major-version.xml (revision 50119) @@ -0,0 +1,196 @@ + + + + Release from &branch.head; + + + This section describes the general procedures of the &os; + release cycle from the &branch.head; branch. + + + &os; <quote><literal>ALPHA</literal></quote> Builds + + Starting with the &os; 10.0-RELEASE cycle, the notion + of ALPHA builds was + introduced. Unlike the BETA and + RC builds, ALPHA builds + are not included in the &os; Release schedule. + + The idea behind ALPHA builds is to + provide regular &os;-provided builds before the creation of the + &branch.stable; branch. + + &os; ALPHA snapshots should be built + approximately once a week. + + For the first ALPHA build, the + BRANCH value in + sys/conf/newvers.sh needs to be changed + from CURRENT to ALPHA1. + For subsequent ALPHA builds, increment each + ALPHAN value by + one. + + See for information on + building the ALPHA images. + + + + Creating the &branch.stablex; Branch + + When creating the &branch.stable; branch, several changes + are required in both the new &branch.stable; branch and the + &branch.head; branch. The files listed are relative to the + repository root. To create the new &branch.stablex; branch + in Subversion: + + &prompt.user; svn cp head &branch.stablex; + + Once the &branch.stablex; branch has been committed, make + the following edits: + + + + + + File to Edit + What to Change + + + + + + stable/11/UPDATING + Update the &os; version, and remove the notice + about WITNESS + + + + stable/11/contrib/jemalloc/include/jemalloc/jemalloc_FreeBSD.h + #ifndef MALLOC_PRODUCTION +#define MALLOC_PRODUCTION +#endif + + + + stable/11/sys/*/conf/GENERIC* + Remove debugging support + + + + stable/11/release/release.conf.sample + Update SRCBRANCH + + + + stable/11/sys/*/conf/GENERIC-NODEBUG + Remove these kernel configurations + + + + stable/11/sys/conf/newvers.sh + Update the BRANCH value to + reflect BETA1 + + + + + + Then in the &branch.head; branch, which will now become + a new major version: + + + + + + File to Edit + What to Change + + + + + + head/UPDATING + Update the &os; version + + + + head/gnu/usr.bin/groff/tmac/mdoc.local.in + Add the new &os; version + + + + head/sys/conf/newvers.sh + Update the BRANCH value to + reflect CURRENT, and increment + REVISION + + + + head/Makefile.inc1 + Update TARGET_TRIPLE + + + + head/sys/sys/param.h + Update __FreeBSD_version + + + + head/contrib/llvm/tools/clang/lib/Basic/Targets.cpp + Update + __FreeBSD_cc_version + + + + head/gnu/usr.bin/cc/cc_tools/freebsd-native.h + Update FBSD_MAJOR and + FBSD_CC_VER + + + + head/contrib/gcc/config.gcc + Append the + freebsd<version>.h + section + + + + head/release/Makefile + Remove the + debug.witness.trace entries + + + + head/release/doc/en_US.ISO8859-1/readme/article.xml + Replace &a.current; with &a.stable; + + + + head/release/doc/share/xml/release.ent + + + + ?> + + + head/lib/clang/clang.build.mk + Uncomment -DNDEBUG + + + + head/lib/clang/freebsd_cc_version.h + Update + FREEBSD_CC_VERSION + + + + + + Property changes on: head/en_US.ISO8859-1/articles/freebsd-releng/releng-major-version.xml ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/xml \ No newline at end of property Index: head/en_US.ISO8859-1/articles/freebsd-releng/releng-minor-version.xml =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-releng/releng-minor-version.xml (nonexistent) +++ head/en_US.ISO8859-1/articles/freebsd-releng/releng-minor-version.xml (revision 50119) @@ -0,0 +1,174 @@ + + + + Release from &branch.stable; + + This section describes the general procedures of the &os; + release cycle from an extablished &branch.stable; branch. + + + &os; <literal>stable</literal> Branch Code Slush + + In preparation for the code freeze on + a stable branch, several files need to be + updated to reflect the release cycle is officially in + progress. These files are all relative to the top-most level of + the stable branch: + + + + + + File to Edit + What to Change + + + + + + gnu/usr.bin/groff/tmac/mdoc.local.in + Add the new &os; version + + + + sys/conf/newvers.sh + Update the BRANCH value to + reflect PRERELEASE + + + + Makefile.inc1 + Update TARGET_TRIPLE + + + + + + + + &os; <literal>BETA</literal> Builds + + Following the code slush, the next phase of the release + cycle is the code freeze. This is the point at which all + commits to the stable branch require explicit approval from + the &team.re;. This is enforced by pre-commit hooks in the + Subversion repository by editing + base/svnadmin/conf/approvers to include + a regular expression matching the &branch.stablex; branch for + the release: + + ^/&branch.stablex; re + + + There are two general exceptions to requiring commit + approval during the release cycle. The first is any change + that needs to be committed by the Release Engineer in order + to proceed with the day-to-day workflow of the release cycle, + the other is security fixes that may occur during the release + cycle. + + + Once the code freeze is in effect, the next build from the + branch is labeled BETA1. This is done by + updating the BRANCH value in + sys/conf/newvers.sh from + PRERELEASE to + BETA1. + + Once this is done, the first set of BETA + builds are started. Subsequent BETA builds + do not require updates to any files other than + sys/conf/newvers.sh, incrementing the + BETA build number. + + + + Creating the &branch.relengx; Branch + + When the first RC (Release Candidate) + build is ready to begin, the &branch.releng; branch is created. + This is a multi-step process that must be done in a specific + order, in order to avoid anomalies such as overlaps with + __FreeBSD_version values, for example. The + paths listed below are relative to the repository root. The + order of commits and what to change are: + + &prompt.user; svn cp &branch.stablex; &branch.relengx; + + + + + + File to Edit + What to Change + + + + + + releng/11.0/sys/conf/newvers.sh + Change + BETAX + to RC1 + + + + releng/11.0/sys/sys/param.h + Update __FreeBSD_version + + + + releng/11.0/etc/pkg/FreeBSD.conf + Replace latest with + quarterly as the default package + repository location + + + + releng/11.0/release/pkg_repos/release-dvd.conf + Replace latest with + quarterly as the default package + repository location + + + + stable/11/sys/conf/newver.sh + Update + BETAX with + PRERELEASE + + + + stable/11/sys/sys/param.h + Update __FreeBSD_version + + + + svnadmin/conf/approvers + Add a new approvers line for the releng + branch as was done for the stable branch + + + + + + &prompt.user; svn propdel -R svn:mergeinfo &branch.relengx; +&prompt.user; svn commit &branch.relengx; +&prompt.user; svn commit &branch.stablex; + + Now that two new __FreeBSD_version values + exist, also update + head/en_US.ISO8859-1/books/porters-handbook/versions/chapter.xml + in the Documentation Project repository. + + After the first RC build has completed + and tested, the &branch.stable; branch can be + thawed by removing (or commenting) the + ^/&branch.stablex; entry in + svnadmin/conf/approvers. + + Property changes on: head/en_US.ISO8859-1/articles/freebsd-releng/releng-minor-version.xml ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/xml \ No newline at end of property Index: head/en_US.ISO8859-1/articles/freebsd-releng/releng-mirrors.xml =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-releng/releng-mirrors.xml (nonexistent) +++ head/en_US.ISO8859-1/articles/freebsd-releng/releng-mirrors.xml (revision 50119) @@ -0,0 +1,113 @@ + + + + Publishing &os; Installation Media to Project Mirrors + + This section describes the procedure to publish &os; + development snapshots and releases to the Project mirrors. + + + Staging &os; Installation Media Images + + Staging &os; snapshots and releases is a two part + process: + + + + Creating the directory structure to match the hierarchy + on ftp-master + + If EVERYTHINGISFINE is defined in the + build configuration files, main.conf in + the case of the build scripts referenced above, this happens + automatically in the &man.chroot.8; after the build is + complete, creating the directory structure in ${DESTDIR}/R/ftp-stage + with a path structure matching what is expected on + ftp-master. This is equivalent to + running the following in the &man.chroot.8; directly: + + &prompt.root; make -C /usr/src/release -f Makefile.mirrors EVERYTHINGISFINE=1 ftp-stage + + After each architecture is built, + thermite.sh will + rsync the ${DESTDIR}/R/ftp-stage + from the build &man.chroot.8; to /snap/ftp/snapshots or + /snap/ftp/releases on + the build host, respectively. + + + + Copying the files to a staging directory on + ftp-master before moving the files + into pub/ to begin + propagation to the Project mirrors + + Once all builds have finished, /snap/ftp/snapshots, or + /snap/ftp/releases + for a release, is polled by + ftp-master using + rsync to /archive/tmp/snapshots or + /snap/ftp/releases, + respectively. + + + On ftp-master in the &os; + Project infrastructure, this step requires + root level access, as this step must + be executed as the archive user. + + + + + + + Publishing &os; Installation Media + + Once the images are staged in /archive/tmp/, they are ready to + be made public by putting them in /archive/pub/FreeBSD. In order + to reduce propagation time, &man.pax.1; is used to create hard + links from /archive/tmp + to /archive/pub/FreeBSD. + + + In order for this to be effective, both /archive/tmp and /archive/pub must reside on the + same logical filesystem. + + + There is a caveat, however, where + rsync must be used after &man.pax.1; + in order to correct the symbolic links in pub/FreeBSD/snapshots/ISO-IMAGES + which &man.pax.1; will replace with a hard link, increasing the + propagation time. + + + As with the staging steps, this requires + root level access, as this step must be + executed as the archive user. + + + As the archive user: + + &prompt.user; cd /archive/tmp/snapshots +&prompt.user; pax -r -w -l . /archive/pub/FreeBSD/snapshots +&prompt.user; /usr/local/bin/rsync -avH /archive/tmp/snapshots/* /archive/pub/FreeBSD/snapshots/ + + Replace snapshots with + releases as appropriate. + + Property changes on: head/en_US.ISO8859-1/articles/freebsd-releng/releng-mirrors.xml ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/xml \ No newline at end of property Index: head/en_US.ISO8859-1/articles/freebsd-releng/releng-terminology.xml =================================================================== --- head/en_US.ISO8859-1/articles/freebsd-releng/releng-terminology.xml (nonexistent) +++ head/en_US.ISO8859-1/articles/freebsd-releng/releng-terminology.xml (revision 50119) @@ -0,0 +1,61 @@ + + + + Release Engineering Terminology + + This section describes some of the terminology used throughout + the rest of this document. + + + The Code Slush + + Although the code slush is not a hard freeze on the tree, + the &team.re; requests that bugs in the existing code base take + priority over new features. + + The code slush does not enforce commit approvals to the + branch. + + + + The Code Freeze + + The code freeze marks the point in time where all commits to + the branch require explicit approval from the &team.re;. + + The &os; Subversion repository + contains several hooks to perform sanity checks before any + commit is actually committed to the tree. One of these hooks + will evaluate if committing to a particular branch requires + specific approval. + + To enforce commit approvals by the &team.re;, the Release + Engineer updates + base/svnadmin/conf/approvers, and commits + the change back to the repository. Once this is done, any + change to the branch must include an Approved by: + line in the commit message. + + The Approved by: line must match the second + column in base/svnadmin/conf/approvers, + otherwise the commit will be rejected by the repository + hooks. + + + + The <acronym>KBI</acronym>/<acronym>KPI</acronym> + Freeze + + KBI/KPI stability + implies that the caller of a function across two different + releases of software that implement the function results in the + same end state. The caller, whether it is a process, thread, or + function, expects the function to operate in a certain way, + otherwise the KBI/KPI + stability on the branch is broken. + + Property changes on: head/en_US.ISO8859-1/articles/freebsd-releng/releng-terminology.xml ___________________________________________________________________ Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/xml \ No newline at end of property Index: head/en_US.ISO8859-1/articles/releng/article.xml =================================================================== --- head/en_US.ISO8859-1/articles/releng/article.xml (revision 50118) +++ head/en_US.ISO8859-1/articles/releng/article.xml (revision 50119) @@ -1,1043 +1,1044 @@
&os; Release Engineering November 2001 BSDCon Europe MurrayStokely I've been involved in the development of &os; based products since 1997 at Walnut Creek CDROM, BSDi, and now Wind River Systems. &os; 4.4 was the first official release of &os; that I played a significant part in.
murray@FreeBSD.org http://www.FreeBSD.org/~murray/
&tm-attrib.freebsd; &tm-attrib.intel; &tm-attrib.general; $FreeBSD$ - - 2013/02/26: This document is partially outdated and does not - accurately describe the current release procedures of the - &os; Release Engineering team. The &os; Release - Engineering team is currently reviewing this document and - will publish updated content soon. - - + + This document is outdated and does not accurately + describe the current release procedures of the &os; + Release Engineering team. It is retained for historical + purposes. The current procedures used by the &os; Release + Engineering team are available in the &os; + Release Engineering article. + This paper describes the approach used by the &os; release engineering team to make production quality releases of the &os; Operating System. It details the methodology used for the official &os; releases and describes the tools available for those interested in producing customized &os; releases for corporate rollouts or commercial productization. Introduction The development of &os; is a very open process. &os; is comprised of contributions from thousands of people around the world. The &os; Project provides Subversion Subversion, http://subversion.apache.org access to the general public so that others can have access to log messages, diffs (patches) between development branches, and other productivity enhancements that formal source code management provides. This has been a huge help in attracting more talented developers to &os;. However, I think everyone would agree that chaos would soon manifest if write access to the main repository was opened up to everyone on the Internet. Therefore only a select group of nearly 300 people are given write access to the Subversion repository. These committers FreeBSD committers are usually the people who do the bulk of &os; development. An elected Core Team &os; Core Team of developers provide some level of direction over the project. The rapid pace of &os; development makes the main development branch unsuitable for the everyday use by the general public. In particular, stabilizing efforts are required for polishing the development system into a production quality release. To solve this conflict, development continues on several parallel tracks. The main development branch is the HEAD or trunk of our Subversion tree, known as &os;-CURRENT or -CURRENT for short. A set of more stable branches are maintained, known as &os;-STABLE or -STABLE for short. All branches live in a master Subversion repository maintained by the &os; Project. &os;-CURRENT is the bleeding-edge of &os; development where all new changes first enter the system. &os;-STABLE is the development branch from which major releases are made. Changes go into this branch at a different pace, and with the general assumption that they have first gone into &os;-CURRENT and have been thoroughly tested by our user community. The term stable in the name of the branch refers to the presumed Application Binary Interface stability, which is promised by the project. This means that a user application compiled on an older version of the system from the same branch works on a newer system from the same branch. The ABI stability has improved greatly from the compared to previous releases. In most cases, binaries from the older STABLE systems run unmodified on newer systems, including HEAD, assuming that the system management interfaces are not used. In the interim period between releases, weekly snapshots are built automatically by the &os; Project build machines and made available for download from ftp://ftp.FreeBSD.org/pub/FreeBSD/snapshots/. The widespread availability of binary release snapshots, and the tendency of our user community to keep up with -STABLE development with Subversion and make buildworld Rebuilding "world" helps to keep &os;-STABLE in a very reliable condition even before the quality assurance activities ramp up pending a major release. In addition to installation ISO snapshots, weekly virtual machine images are also provided for use with VirtualBox, qemu, or other popular emulation software. The virtual machine images can be downloaded from ftp://ftp.FreeBSD.org/pub/FreeBSD/snapshots/VM-IMAGES/. The virtual machine images are approximately 150MB &man.xz.1; compressed, and contain a 10GB sparse filesystem when attached to a virtual machine. Bug reports and feature requests are continuously submitted by users throughout the release cycle. Problems reports are entered into our Bugzilla database through the web interface provided at https://www.freebsd.org/support/bugreports.html. To service our most conservative users, individual release branches were introduced with &os; 4.3. These release branches are created shortly before a final release is made. After the release goes out, only the most critical security fixes and additions are merged onto the release branch. In addition to source updates via Subversion, binary patchkits are available to keep systems on the releng/X.Y branches updated. What this article describes The following sections of this article describe: The different phases of the release engineering process leading up to the actual system build. The actual build process. How the base release may be extended by third parties. Some of the lessons learned through the release of &os; 4.4. Future directions of development. Release Process New releases of &os; are released from the -STABLE branch at approximately four month intervals. The &os; release process begins to ramp up 70-80 days before the anticipated release date when the release engineer sends an email to the development mailing lists to remind developers that they only have 15 days to integrate new changes before the code freeze. During this time, many developers perform what have become known as MFC sweeps. MFC stands for Merge From CURRENT and it describes the process of merging a tested change from our -CURRENT development branch to our -STABLE branch. Project policy requires any change to be first applied to trunk, and merged to the -STABLE branches after sufficient external testing was done by -CURRENT users (developers are expected to extensively test the change before committing to -CURRENT, but it is impossible for a person to exercise all usages of the general-purpose operating system). Minimal MFC period is 3 days, which is typically used only for trivial or critical bugfixes. Code Review Sixty days before the anticipated release, the source repository enters a code freeze. During this time, all commits to the -STABLE branch must be approved by &a.re;. The approval process is technically enforced by a pre-commit hook. The kinds of changes that are allowed during this period include: Bug fixes. Documentation updates. Security-related fixes of any kind. Minor changes to device drivers, such as adding new Device IDs. Driver updates from the vendors. Any additional change that the release engineering team feels is justified, given the potential risk. Shortly after the code freeze is started, a BETA1 image is built and released for widespread testing. During the code freeze, at least one beta image or release candidate is released every two weeks until the final release is ready. During the days preceeding the final release, the release engineering team is in constant communication with the security-officer team, the documentation maintainers, and the port maintainers to ensure that all of the different components required for a successful release are available. After the quality of the BETA images is satisfying enough, and no large and potentially risky changes are planned, the release branch is created and Release Candidate (RC) images are built from the release branch, instead of the BETA images from the STABLE branch. Also, the freeze on the STABLE branch is lifted and release branch enters a hard code freeze where it becomes much harder to justify new changes to the system unless a serious bug-fix or security issue is involved. Final Release Checklist When several BETA images have been made available for widespread testing and all major issues have been resolved, the final release polishing can begin. Creating the Release Branch In all examples below, $FSVN refers to the location of the &os; Subversion repository, svn+ssh://svn.FreeBSD.org/base/. The layout of &os; branches in Subversion is described in the Committer's Guide. The first step in creating a branch is to identify the revision of the stable/X sources that you want to branch from. &prompt.root; svn log -v $FSVN/stable/9 The next step is to create the release branch &prompt.root; svn cp $FSVN/stable/9@REVISION $FSVN/releng/9.2 This branch can be checked out: &prompt.root; svn co $FSVN/releng/9.2 src Creating the releng branch and release tags is done by the Release Engineering Team. &os; Development Branch &os; 3.x STABLE Branch &os; 4.x STABLE Branch &os; 5.x STABLE Branch &os; 6.x STABLE Branch &os; 7.x STABLE Branch &os; 8.x STABLE Branch &os; 9.x STABLE Branch Bumping up the Version Number Before the final release can be tagged, built, and released, the following files need to be modified to reflect the correct version of &os;: doc/en_US.ISO8859-1/books/handbook/mirrors/chapter.xml doc/en_US.ISO8859-1/books/porters-handbook/book.xml doc/en_US.ISO8859-1/htdocs/cgi/ports.cgi ports/Tools/scripts/release/config doc/share/xml/freebsd.ent src/Makefile.inc1 src/UPDATING src/gnu/usr.bin/groff/tmac/mdoc.local src/release/Makefile src/release/doc/en_US.ISO8859-1/share/xml/release.dsl src/release/doc/share/examples/Makefile.relnotesng src/release/doc/share/xml/release.ent src/sys/conf/newvers.sh src/sys/sys/param.h src/usr.sbin/pkg_install/add/main.c doc/en_US.ISO8859-1/htdocs/search/opensearch/man.xml The release notes and errata files also need to be adjusted for the new release (on the release branch) and truncated appropriately (on the stable/current branch): src/release/doc/en_US.ISO8859-1/relnotes/common/new.xml src/release/doc/en_US.ISO8859-1/errata/article.xml Sysinstall should be updated to note the number of available ports and the amount of disk space required for the Ports Collection. &os; Ports Collection http://www.FreeBSD.org/ports This information is currently kept in src/usr.sbin/sysinstall/dist.c. After the release has been built, a number of files should be updated to announce the release to the world. These files are relative to head/ within the doc/ subversion tree. share/images/articles/releng/branches-relengX.pic head/share/xml/release.ent en_US.ISO8859-1/htdocs/releases/* en_US.ISO8859-1/htdocs/releng/index.xml share/xml/news.xml Additionally, update the BSD Family Tree file: src/share/misc/bsd-family-tree Creating the Release Tag When the final release is ready, the following command will create the release/9.2.0 tag. &prompt.root; svn cp $FSVN/releng/9.2 $FSVN/release/9.2.0 The Documentation and Ports managers are responsible for tagging their respective trees with the tags/RELEASE_9_2_0 tag. When the Subversion svn cp command is used to create a release tag, this identifies the source at a specific point in time. By creating tags, we ensure that future release builders will always be able to use the exact same source we used to create the official &os; Project releases. Release Building &os; releases can be built by anyone with a fast machine and access to a source repository. (That should be everyone, since we offer Subversion access ! See the Subversion section in the Handbook for details.) The only special requirement is that the &man.md.4; device must be available. If the device is not loaded into your kernel, then the kernel module should be automatically loaded when &man.mdconfig.8; is executed during the boot media creation phase. All of the tools necessary to build a release are available from the Subversion repository in src/release. These tools aim to provide a consistent way to build &os; releases. A complete release can actually be built with only a single command, including the creation of ISO images suitable for burning to CDROM or DVD, and an FTP install directory. &man.release.7; fully documents the src/release/generate-release.sh script which is used to build a release. generate-release.sh is a wrapper around the Makefile target: make release. Building a Release &man.release.7; documents the exact commands required to build a &os; release. The following sequences of commands can build an 9.2.0 release: &prompt.root; cd /usr/src/release &prompt.root; sh generate-release.sh release/9.2.0 /local3/release After running these commands, all prepared release files are available in /local3/release/R directory. The release Makefile can be broken down into several distinct steps. Creation of a sanitized system environment in a separate directory hierarchy with make installworld. Checkout from Subversion of a clean version of the system source, documentation, and ports into the release build hierarchy. Population of /etc and /dev in the chrooted environment. chroot into the release build hierarchy, to make it harder for the outside environment to taint this build. make world in the chrooted environment. Build of Kerberos-related binaries. Build GENERIC kernel. Creation of a staging directory tree where the binary distributions will be built and packaged. Build and installation of the documentation toolchain needed to convert the documentation source (SGML) into HTML and text documents that will accompany the release. Build and installation of the actual documentation (user manuals, tutorials, release notes, hardware compatibility lists, and so on.) Package up distribution tarballs of the binaries and sources. Create FTP installation hierarchy. (optionally) Create ISO images for CDROM/DVD media. For more information about the release build infrastructure, please see &man.release.7;. It is important to remove any site-specific settings from /etc/make.conf. For example, it would be unwise to distribute binaries that were built on a system with CPUTYPE set to a specific processor. Contributed Software (<quote>ports</quote>) The &os; Ports collection is a collection of over &os.numports; third-party software packages available for &os;. The &a.portmgr; is responsible for maintaining a consistent ports tree that can be used to create the binary packages that accompany official &os; releases. Release ISOs Starting with &os; 4.4, the &os; Project decided to release all four ISO images that were previously sold on the BSDi/Wind River Systems/FreeBSD Mall official CDROM distributions. Each of the four discs must contain a README.TXT file that explains the contents of the disc, a CDROM.INF file that provides meta-data for the disc so that &man.sysinstall.8; can validate and use the contents, and a filename.txt file that provides a manifest for the disc. This manifest can be created with a simple command: /stage/cdrom&prompt.root; find . -type f | sed -e 's/^\.\///' | sort > filename.txt The specific requirements of each CD are outlined below. Disc 1 The first disc is almost completely created by make release. The only changes that should be made to the disc1 directory are the addition of a tools directory, and as many popular third party software packages as will fit on the disc. The tools directory contains software that allow users to create installation floppies from other operating systems. This disc should be made bootable so that users of modern PCs do not need to create installation floppy disks. If a custom kernel of &os; is to be included, then &man.sysinstall.8; and &man.release.7; must be updated to include installation instructions. The relevant code is contained in src/release and src/usr.sbin/sysinstall. Specifically, the file src/release/Makefile, and dist.c, dist.h, menus.c, install.c, and Makefile will need to be updated under src/usr.sbin/sysinstall. Optionally, you may choose to update sysinstall.8. Disc 2 The second disc is also largely created by make release. This disc contains a live filesystem that can be used from &man.sysinstall.8; to troubleshoot a &os; installation. This disc should be bootable and should also contain a compressed copy of the CVS repository in the CVSROOT directory and commercial software demos in the commerce directory. Multi-volume support Sysinstall supports multiple volume package installations. This requires that each disc have an INDEX file containing all of the packages on all volumes of a set, along with an extra field that indicates which volume that particular package is on. Each volume in the set must also have the CD_VOLUME variable set in the cdrom.inf file so that sysinstall can tell which volume is which. When a user attempts to install a package that is not on the current disc, sysinstall will prompt the user to insert the appropriate one. Distribution FTP Sites When the release has been thoroughly tested and packaged for distribution, the master FTP site must be updated. The official &os; public FTP sites are all mirrors of a master server that is open only to other FTP sites. This site is known as ftp-master. When the release is ready, the following files must be modified on ftp-master: /pub/FreeBSD/releases/arch/X.Y-RELEASE/ The installable FTP directory as output from make release. /pub/FreeBSD/ports/arch/packages-X.Y-release/ The complete package build for this release. /pub/FreeBSD/releases/arch/X.Y-RELEASE/tools A symlink to ../../../tools. /pub/FreeBSD/releases/arch/X.Y-RELEASE/packages A symlink to ../../../ports/arch/packages-X.Y-release. /pub/FreeBSD/releases/arch/ISO-IMAGES/X.Y/X.Y-RELEASE-arch-*.iso The ISO images. The * is disc1, disc2, etc. Only if there is a disc1 and there is an alternative first installation CD (for example a stripped-down install with no windowing system) there may be a mini as well. For more information about the distribution mirror architecture of the &os; FTP sites, please see the Mirroring &os; article. It may take many hours to two days after updating ftp-master before a majority of the Tier-1 FTP sites have the new software depending on whether or not a package set got loaded at the same time. It is imperative that the release engineers coordinate with the &a.mirror-announce; before announcing the general availability of new software on the FTP sites. Ideally the release package set should be loaded at least four days prior to release day. The release bits should be loaded between 24 and 48 hours before the planned release time with other file permissions turned off. This will allow the mirror sites to download it but the general public will not be able to download it from the mirror sites. Mail should be sent to &a.mirror-announce; at the time the release bits get posted saying the release has been staged and giving the time that the mirror sites should begin allowing access. Be sure to include a time zone with the time, for example make it relative to GMT. CD-ROM Replication Coming soon: Tips for sending &os; ISOs to a replicator and quality assurance measures to be taken. Extensibility Although &os; forms a complete operating system, there is nothing that forces you to use the system exactly as we have packaged it up for distribution. We have tried to design the system to be as extensible as possible so that it can serve as a platform that other commercial products can be built on top of. The only rule we have about this is that if you are going to distribute &os; with non-trivial changes, we encourage you to document your enhancements! The &os; community can only help support users of the software we provide. We certainly encourage innovation in the form of advanced installation and administration tools, for example, but we cannot be expected to answer questions about it. Scripting <command>sysinstall</command> The &os; system installation and configuration tool, &man.sysinstall.8;, can be scripted to provide automated installs for large sites. This functionality can be used in conjunction with &intel; PXE &url.books.handbook;/network-pxe-nfs.html to bootstrap systems from the network. Lessons Learned from &os; 4.4 The release engineering process for 4.4 formally began on August 1st, 2001. After that date all commits to the RELENG_4 branch of &os; had to be explicitly approved by the &a.re;. The first release candidate for the x86 architecture was released on August 16, followed by 4 more release candidates leading up to the final release on September 18th. The security officer was very involved in the last week of the process as several security issues were found in the earlier release candidates. A total of over 500 emails were sent to the &a.re; in little over a month. Our user community has made it very clear that the security and stability of a &os; release should not be sacrificed for any self-imposed deadlines or target release dates. The &os; Project has grown tremendously over its lifetime and the need for standardized release engineering procedures has never been more apparent. This will become even more important as &os; is ported to new platforms. Future Directions It is imperative for our release engineering activities to scale with our growing userbase. Along these lines we are working very hard to document the procedures involved in producing &os; releases. Parallelism - Certain portions of the release build are actually embarrassingly parallel. Most of the tasks are very I/O intensive, so having multiple high-speed disk drives is actually more important than using multiple processors in speeding up the make release process. If multiple disks are used for different hierarchies in the &man.chroot.2; environment, then the CVS checkout of the ports and doc trees can be happening simultaneously as the make world on another disk. Using a RAID solution (hardware or software) can significantly decrease the overall build time. Cross-building releases - Building IA-64 or Alpha release on x86 hardware? make TARGET=ia64 release. Regression Testing - We need better automated correctness testing for &os;. Installation Tools - Our installation program has long since outlived its intended life span. Several projects are under development to provide a more advanced installation mechanism. The libh project was one such project that aimed to provide an intelligent new package framework and GUI installation program. Acknowledgements I would like to thank Jordan Hubbard for giving me the opportunity to take on some of the release engineering responsibilities for &os; 4.4 and also for all of his work throughout the years making &os; what it is today. Of course the release would not have been possible without all of the release-related work done by &a.asami.email;, &a.steve.email;, &a.bmah.email;, &a.nik.email;, &a.obrien.email;, &a.kris.email;, &a.jhb.email; and the rest of the &os; development community. I would also like to thank &a.rgrimes.email;, &a.phk.email;, and others who worked on the release engineering tools in the very early days of &os;. This article was influenced by release engineering documents from the CSRG Marshall Kirk McKusick, Michael J. Karels, and Keith Bostic: The Release Engineering of 4.3BSD , the NetBSD Project , NetBSD Developer Documentation: Release Engineering http://www.NetBSD.org/developers/releng/index.html , and John Baldwin's proposed release engineering process notes. John Baldwin's &os; Release Engineering Proposal http://people.FreeBSD.org/~jhb/docs/releng.txt
Index: head/en_US.ISO8859-1 =================================================================== --- head/en_US.ISO8859-1 (revision 50118) +++ head/en_US.ISO8859-1 (revision 50119) Property changes on: head/en_US.ISO8859-1 ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /user/gjb/releng-rewrite/en_US.ISO8859-1:r43296-43297,43932-43947,43950,49890,49945-49948,49954-49955,49961,50064,50077-50081,50097,50100,50103-50106,50108-50113,50115-50118 Index: head/share/xml/urls.ent =================================================================== --- head/share/xml/urls.ent (revision 50118) +++ head/share/xml/urls.ent (revision 50119) @@ -1,116 +1,118 @@ + + Index: head/share =================================================================== --- head/share (revision 50118) +++ head/share (revision 50119) Property changes on: head/share ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /user/gjb/releng-rewrite/share:r50116