diff --git a/usr.sbin/etcupdate/etcupdate.8 b/usr.sbin/etcupdate/etcupdate.8 index 24afbca12951..5f13ed1d6733 100644 --- a/usr.sbin/etcupdate/etcupdate.8 +++ b/usr.sbin/etcupdate/etcupdate.8 @@ -1,901 +1,907 @@ .\" Copyright (c) 2010-2013 Hudson River Trading LLC .\" Written by: John H. Baldwin .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" $FreeBSD$ .\" -.Dd November 27, 2018 +.Dd April 20, 2021 .Dt ETCUPDATE 8 .Os .Sh NAME .Nm etcupdate .Nd "manage updates to system files not updated by installworld" .Sh SYNOPSIS .Nm .Op Fl npBF .Op Fl d Ar workdir .Op Fl r | Fl s Ar source | Fl t Ar tarball .Op Fl A Ar patterns .Op Fl D Ar destdir .Op Fl I Ar patterns .Op Fl L Ar logfile .Op Fl M Ar options .Nm .Cm build .Op Fl B .Op Fl d Ar workdir .Op Fl s Ar source .Op Fl L Ar logfile .Op Fl M Ar options .Ar tarball .Nm .Cm diff .Op Fl d Ar workdir .Op Fl D Ar destdir .Op Fl I Ar patterns .Op Fl L Ar logfile .Nm .Cm extract .Op Fl B .Op Fl d Ar workdir .Op Fl s Ar source | Fl t Ar tarball .Op Fl L Ar logfile .Op Fl M Ar options .Nm .Cm resolve .Op Fl p .Op Fl d Ar workdir .Op Fl D Ar destdir .Op Fl L Ar logfile .Nm +.Cm revert +.Op Fl d Ar workdir +.Op Fl D Ar destdir +.Op Fl L Ar logfile +.Ar +.Nm .Cm status .Op Fl d Ar workdir .Op Fl D Ar destdir .Sh DESCRIPTION The .Nm utility is a tool for managing updates to files that are not updated as part of .Sq make installworld such as files in .Pa /etc . It manages updates by doing a three-way merge of changes made to these files against the local versions. It is also designed to minimize the amount of user intervention with the goal of simplifying upgrades for clusters of machines. .Pp To perform a three-way merge, .Nm keeps copies of the current and previous versions of files that it manages. These copies are stored in two trees known as the .Dq current and .Dq previous trees. During a merge, .Nm compares the .Dq current and .Dq previous copies of each file to determine which changes need to be merged into the local version of each file. If a file can be updated without generating a conflict, .Nm will update the file automatically. If the local changes to a file conflict with the changes made to a file in the source tree, then a merge conflict is generated. The conflict must be resolved after the merge has finished. The .Nm utility will not perform a new merge until all conflicts from an earlier merge are resolved. .Sh MODES The .Nm utility supports several modes of operation. The mode is specified via an optional command argument. If present, the command must be the first argument on the command line. If a command is not specified, the default mode is used. .Ss Default Mode The default mode merges changes from the source tree to the destination directory. First, it updates the .Dq current and .Dq previous trees. Next, it compares the two trees merging changes into the destination directory. Finally, it displays warnings for any conditions it could not handle automatically. .Pp If the .Fl r option is not specified, then the first step taken is to update the .Dq current and .Dq previous trees. If a .Dq current tree already exists, then that tree is saved as the .Dq previous tree. An older .Dq previous tree is removed if it exists. By default the new .Dq current tree is built from a source tree. However, if a tarball is specified via the .Fl t option, then the tree is extracted from that tarball instead. .Pp Next, .Nm compares the files in the .Dq current and .Dq previous trees. If a file was removed from the .Dq current tree, then it will be removed from the destination directory only if it does not have any local modifications. If a file was added to the .Dq current tree, then it will be copied to the destination directory only if it would not clobber an existing file. If a file is changed in the .Dq current tree, then .Nm will attempt to merge the changes into the version of the file in the destination directory. If the merge encounters conflicts, then a version of the file with conflict markers will be saved for future resolution. If the merge does not encounter conflicts, then the merged version of the file will be saved in the destination directory. If .Nm is not able to safely merge in changes to a file other than a merge conflict, it will generate a warning. .Pp For each file that is updated a line will be output with a leading character to indicate the action taken. The possible actions follow: .Pp .Bl -tag -width "A" -compact -offset indent .It A Added .It C Conflict .It D Deleted .It M Merged .It U Updated .El .Pp Finally, if any warnings were encountered they are displayed after the merge has completed. .Pp Note that for certain files .Nm will perform post-install actions any time that the file is updated. Specifically, .Xr pwd_mkdb 8 is invoked if .Pa /etc/master.passwd is changed, .Xr cap_mkdb 1 is invoked to update .Pa /etc/login.conf.db if .Pa /etc/login.conf is changed, .Xr newaliases 1 is invoked if .Pa /etc/mail/aliases is changed, .Xr services_mkdb 8 is invoked if .Pa /etc/services is changed, .Xr tzsetup 8 is invoked if .Pa /etc/localtime is changed and if .Fa /var/db/zoneinfo exists, and .Pa /etc/rc.d/motd is invoked if .Pa /etc/motd is changed. One exception is that if .Pa /etc/mail/aliases is changed and the destination directory is not the default, then a warning will be issued instead. This is due to a limitation of the .Xr newaliases 1 command. Similarly, if .Pa /etc/motd is changed and the destination directory is not the default, then .Pa /etc/rc.d/motd will not be executed due to a limitation of that script. In this case no warning is issued as the result of .Pa /etc/rc.d/motd is merely cosmetic and will be corrected on the next reboot. .Ss Build Mode The .Cm build mode is used to build a tarball that contains a snapshot of a .Dq current tree. This tarball can be used by the default and extract modes. Using a tarball can allow .Nm to perform a merge without requiring a source tree that matches the currently installed world. The .Fa tarball argument specifies the name of the file to create. The file will be a .Xr tar 5 file compressed with .Xr bzip2 1 . .Ss Diff Mode The .Cm diff mode compares the versions of files in the destination directory to the .Dq current tree and generates a unified format diff of the changes. This can be used to determine which files have been locally modified and how. Note that .Nm does not manage files that are not maintained in the source tree such as .Pa /etc/fstab and .Pa /etc/rc.conf . .Ss Extract Mode The .Cm extract mode generates a new .Dq current tree. Unlike the default mode, it does not save any existing .Dq current tree and does not modify any existing .Dq previous tree. The new .Dq current tree can either be built from a source tree or extracted from a tarball. .Ss Resolve Mode The .Cm resolve mode is used to resolve any conflicts encountered during a merge. In this mode, .Nm iterates over any existing conflicts prompting the user for actions to take on each conflicted file. For each file, the following actions are available: .Pp .Bl -tag -width "(tf) theirs-full" -compact .It (p) postpone Ignore this conflict for now. .It (df) diff-full Show all changes made to the merged file as a unified diff. .It (e) edit Change the merged file in an editor. .It (r) resolved Install the merged version of the file into the destination directory. .It (mf) mine-full Use the version of the file in the destination directory and ignore any changes made to the file in the .Dq current tree. .It (tf) theirs-full Use the version of the file from the .Dq current tree and discard any local changes made to the file. .It (h) help Display the list of commands. .El +.Ss Revert Mode +The +.Cm revert +mode is used to restore the stock versions of files. +In this mode, +.Nm +installs the stock version of requested files. +This mode cannot be used to restore directories, only individual files. .Ss Status Mode The .Cm status mode shows a summary of the results of the most recent merge. First it lists any files for which there are unresolved conflicts. Next it lists any warnings generated during the last merge. If the last merge did not generate any conflicts or warnings, then nothing will be output. .Sh OPTIONS The following options are available. Note that most options do not apply to all modes. .Bl -tag -width ".Fl A Ar patterns" .It Fl A Ar patterns Always install the new version of any files that match any of the patterns listed in .Ar patterns . Each pattern is evaluated as an .Xr sh 1 shell pattern. This option may be specified multiple times to specify multiple patterns. Multiple space-separated patterns may also be specified in a single option. Note that ignored files specified via the .Ev IGNORE_FILES variable or the .Fl I option will not be installed. .It Fl B Do not build generated files in a private object tree. Instead, reuse the generated files from a previously built object tree that matches the source tree. This can be useful to avoid gratuitous conflicts in .Xr sendmail 8 configuration files when bootstrapping. It can also be useful for building a tarball that matches a specific world build. .It Fl D Ar destdir Specify an alternate destination directory as the target of a merge. This is analogous to the .Dv DESTDIR variable used with .Sq make installworld . The default destination directory is an empty string which results in merges updating .Pa /etc on the local machine. .It Fl d Ar workdir Specify an alternate directory to use as the work directory. The work directory is used to store the .Dq current and .Dq previous trees as well as unresolved conflicts. The default work directory is .Pa /var/db/etcupdate . .It Fl F Ignore changes in the FreeBSD ID string when comparing files in the destination directory to files in either of the .Dq current or .Dq previous trees. In .Cm diff mode, this reduces noise due to FreeBSD ID string changes in the output. During an update this can simplify handling for harmless conflicts caused by FreeBSD ID string changes. .Pp Specifically, if a file in the destination directory is identical to the same file in the .Dq previous tree modulo the FreeBSD ID string, then the file is treated as if it was unmodified and the .Dq current version of the file will be installed. Similarly, if a file in the destination directory is identical to the same file in the .Dq current tree modulo the FreeBSD ID string, then the .Dq current version of the file will be installed to update the ID string. If the .Dq previous and .Dq current versions of the file are identical, then .Nm will not change the file in the destination directory. .Pp Due to limitations in the .Xr diff 1 command, this option may not have an effect if there are other changes in a file that are close to the FreeBSD ID string. .It Fl I Ar patterns Ignore any files that match any of the patterns listed in .Ar patterns . No warnings or other messages will be generated for those files during a merge. Each pattern is evaluated as an .Xr sh 1 shell pattern. This option may be specified multiple times to specify multiple patterns. Multiple space-separated patterns may also be specified in a single option. .It Fl L Ar logfile Specify an alternate path for the log file. The .Nm utility logs each command that it invokes along with the standard output and standard error to this file. By default the log file is stored in a file named .Pa log in the work directory. .It Fl M Ar options Pass .Ar options as additional parameters to .Xr make 1 when building a .Dq current tree. This can be used for to set the .Dv TARGET or .Dv TARGET_ARCH variables for a cross-build. .It Fl n Enable .Dq dry-run mode. Do not merge any changes to the destination directory. Instead, report what actions would be taken during a merge. Note that the existing .Dq current and .Dq previous trees will not be changed. If the .Fl r option is not specified, then a temporary .Dq current tree will be extracted to perform the comparison. .It Fl p Enable .Dq pre-world mode. Only merge changes to files that are necessary to successfully run .Sq make installworld or .Sq make installkernel . When this flag is enabled, the existing .Dq current and .Dq previous trees are left alone. Instead, a temporary tree is populated with the necessary files. This temporary tree is compared against the .Dq current tree. This allows a normal update to be run after .Sq make installworld has completed. Any conflicts generated during a .Dq pre-world update should be resolved by a .Dq pre-world .Cm resolve . .It Fl r Do not update the .Dq current and .Dq previous trees during a merge. This can be used to .Dq re-run a previous merge operation. .It Fl s Ar source Specify an alternate source tree to use when building or extracting a .Dq current tree. The default source tree is .Pa /usr/src . .It Fl t Ar tarball Extract a new .Dq current tree from a tarball previously generated by the .Cm build command rather than building the tree from a source tree. .El .Sh CONFIG FILE The .Nm utility can also be configured by setting variables in an optional configuration file named .Pa /etc/etcupdate.conf . Note that command line options override settings in the configuration file. The configuration file is executed by .Xr sh 1 , so it uses that syntax to set configuration variables. The following variables can be set: .Bl -tag -width ".Ev ALWAYS_INSTALL" .It Ev ALWAYS_INSTALL Always install files that match any of the patterns listed in this variable similar to the .Fl A option. .It Ev DESTDIR Specify an alternate destination directory similar to the .Fl D option. .It Ev EDITOR Specify a program to edit merge conflicts. .It Ev FREEBSD_ID Ignore changes in the FreeBSD ID string similar to the .Fl F option. This is enabled by setting the variable to a non-empty value. .It Ev IGNORE_FILES Ignore files that match any of the patterns listed in this variable similar to the .Fl I option. .It Ev LOGFILE Specify an alternate path for the log file similar to the .Fl L option. .It Ev MAKE_OPTIONS Pass additional options to .Xr make 1 when building a .Dq current tree similar to the .Fl M option. .It Ev SRCDIR Specify an alternate source tree similar to the .Fl s option. .It Ev WORKDIR Specify an alternate work directory similar to the .Fl d option. .El .Sh ENVIRONMENT The .Nm utility uses the program identified in the .Ev EDITOR environment variable to edit merge conflicts. If .Ev EDITOR is not set, .Xr vi 1 is used as the default editor. .Sh FILES .Bl -tag -width ".Pa /var/db/etcupdate/log" -compact .It Pa /etc/etcupdate.conf Optional config file. .It Pa /var/db/etcupdate Default work directory used to store trees and other data. .It Pa /var/db/etcupdate/log Default log file. .El .Sh EXIT STATUS .Ex -std .Sh EXAMPLES To compare the files in .Pa /etc with the stock versions: .Pp .Dl "etcupdate diff" .Pp To merge changes after an upgrade via the buildworld and installworld process: .Pp .Dl "etcupdate" .Pp To resolve any conflicts generated during a merge: .Pp .Dl "etcupdate resolve" .Ss Bootstrapping The .Nm utility may need to be bootstrapped before it can be used. The .Cm diff command will fail with an error about a missing reference tree if bootstrapping is needed. .Pp Bootstrapping .Nm requires a source tree that matches the currently installed world. The easiest way to ensure this is to bootstrap .Nm before updating the source tree to start the next world upgrade cycle. First, generate a reference tree: .Pp .Dl "etcupdate extract" .Pp Second, use the .Cm diff command to compare the reference tree to your current files in .Pa /etc . Undesired differences should be removed using an editor, .Xr patch 1 , or by copying files from the reference tree .Po located at .Pa /var/db/etcupdate/current by default .Pc . .Pp If the tree at .Pa /usr/src is already newer than the currently installed world, a new tree matching the currently installed world can be checked out to a temporary location. The reference tree for .Nm can then be generated via: .Pp .Dl "etcupdate extract -s /path/to/tree" .Pp The .Cm diff command can be used as above to remove undesired differences. Afterwards, the changes in the tree at .Pa /usr/src can be merged via a regular merge. .Sh DIAGNOSTICS The following warning messages may be generated during a merge. Note that several of these warnings cover obscure cases that should occur rarely if at all in practice. For example, if a file changes from a file to a directory in the .Dq current tree and the file was modified in the destination directory, then a warning will be triggered. In general, when a warning references a pathname, the corresponding file in the destination directory is not changed by a merge operation. .Bl -diag .It "Directory mismatch: ()" An attempt was made to create a directory at .Pa path but an existing file of type .Dq type already exists for that path name. .It "Modified link changed: ( became )" The target of a symbolic link named .Pa file was changed from .Dq old to .Dq new in the .Dq current tree. The symbolic link has been modified to point to a target that is neither .Dq old nor .Dq new in the destination directory. .It "Modified mismatch: ( vs )" A file named .Pa file of type .Dq new was modified in the .Dq current tree, but the file exists as a different type .Dq dest in the destination directory. .It "Modified changed: ( became )" A file named .Pa file changed type from .Dq old in the .Dq previous tree to type .Dq new in the .Dq current tree. The file in the destination directory of type .Dq type has been modified, so it could not be merged automatically. .It "Modified remains: " The file of type .Dq type named .Pa file has been removed from the .Dq current tree, but it has been locally modified. The modified version of the file remains in the destination directory. .It "Needs update: /etc/localtime (required manual update via tzsetup(8))" The .Fa /var/db/zoneinfo file does not exist, so .Nm was not able to refresh .Fa /etc/localtime from its source file in .Fa /usr/share/zoneinfo . Running .Xr tzsetup 8 will both refresh .Fa /etc/localtime and generate .Fa /var/db/zoneinfo permitting future updates to refresh .Fa /etc/localtime automatically. .It "Needs update: /etc/mail/aliases.db (required manual update via newaliases(1))" The file .Pa /etc/mail/aliases was updated during a merge with a non-empty destination directory. Due to a limitation of the .Xr newaliases 1 command, .Nm was not able to automatically update the corresponding aliases database. .It "New file mismatch: ( vs )" A new file named .Pa file of type .Dq new has been added to the .Dq current tree. A file of that name already exists in the destination directory, but it is of a different type .Dq dest . .It "New link conflict: ( vs )" A symbolic link named .Pa file has been added to the .Dq current tree that links to .Dq new . A symbolic link of the same name already exists in the destination directory, but it links to a different target .Dq dest . .It "Non-empty directory remains: " The directory .Pa file was removed from the .Dq current tree, but it contains additional files in the destination directory. These additional files as well as the directory remain. .It "Remove mismatch: ( became )" A file named .Pa file changed from type .Dq old in the .Dq previous tree to type .Dq new in the .Dq current tree, but it has been removed in the destination directory. .It "Removed file changed: " A file named .Pa file was modified in the .Dq current tree, but it has been removed in the destination directory. .It "Removed link changed: ( became )" The target of a symbolic link named .Pa file was changed from .Dq old to .Dq new in the .Dq current tree, but it has been removed in the destination directory. .El .Sh SEE ALSO .Xr cap_mkdb 1 , .Xr diff 1 , .Xr make 1 , .Xr newaliases 1 , .Xr sh 1 , .Xr mergemaster 8 , .Xr pwd_mkdb 8 , .Xr services_mkdb 8 , .Xr tzsetup 8 .Sh HISTORY The .Nm utility first appeared in .Fx 10.0 . .Sh AUTHORS The .Nm utility was written by .An John Baldwin Aq Mt jhb@FreeBSD.org . .Sh BUGS Rerunning a merge does not automatically delete conflicts left over from a previous merge. Any conflicts must be resolved before the merge can be rerun. It is not clear if this is a feature or a bug. .Pp There is no way to easily automate conflict resolution for specific files. For example, one can imagine a syntax along the lines of .Pp .Dl "etcupdate resolve tf /some/file" .Pp to resolve a specific conflict in an automated fashion. .Pp -It might be nice to have something like a -.Sq revert -command to replace a locally modified version of a file with the stock -version of the file. -For example: -.Pp -.Dl "etcupdate revert /etc/mail/freebsd.cf" -.Pp Bootstrapping .Nm often results in gratuitous diffs in .Pa /etc/mail/*.cf that cause conflicts in the first merge. If an object tree that matches the source tree is present when bootstrapping, then passing the .Fl B flag to the .Cm extract command can work around this. diff --git a/usr.sbin/etcupdate/etcupdate.sh b/usr.sbin/etcupdate/etcupdate.sh index dc26bc0c8aab..254a72abae08 100755 --- a/usr.sbin/etcupdate/etcupdate.sh +++ b/usr.sbin/etcupdate/etcupdate.sh @@ -1,1846 +1,1887 @@ #!/bin/sh # # SPDX-License-Identifier: BSD-2-Clause-FreeBSD # # Copyright (c) 2010-2013 Hudson River Trading LLC # Written by: John H. Baldwin # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # $FreeBSD$ # This is a tool to manage updating files that are not updated as part # of 'make installworld' such as files in /etc. Unlike other tools, # this one is specifically tailored to assisting with mass upgrades. # To that end it does not require user intervention while running. # # Theory of operation: # # The most reliable way to update changes to files that have local # modifications is to perform a three-way merge between the original # unmodified file, the new version of the file, and the modified file. # This requires having all three versions of the file available when # performing an update. # # To that end, etcupdate uses a strategy where the current unmodified # tree is kept in WORKDIR/current and the previous unmodified tree is # kept in WORKDIR/old. When performing a merge, a new tree is built # if needed and then the changes are merged into DESTDIR. Any files # with unresolved conflicts after the merge are left in a tree rooted # at WORKDIR/conflicts. # # To provide extra flexibility, etcupdate can also build tarballs of # root trees that can later be used. It can also use a tarball as the # source of a new tree instead of building it from /usr/src. # Global settings. These can be adjusted by config files and in some # cases by command line options. # TODO: # - automatable conflict resolution -# - a 'revert' command to make a file "stock" usage() { cat < etcupdate diff [-d workdir] [-D destdir] [-I patterns] [-L logfile] etcupdate extract [-B] [-d workdir] [-s source | -t tarball] [-L logfile] [-M options] etcupdate resolve [-p] [-d workdir] [-D destdir] [-L logfile] + etcupdate revert [-d workdir] [-D destdir] [-L logfile] file ... etcupdate status [-d workdir] [-D destdir] EOF exit 1 } # Used to write a message prepended with '>>>' to the logfile. log() { echo ">>>" "$@" >&3 } # Used for assertion conditions that should never happen. panic() { echo "PANIC:" "$@" exit 10 } # Used to write a warning message. These are saved to the WARNINGS # file with " " prepended. warn() { echo -n " " >> $WARNINGS echo "$@" >> $WARNINGS } # Output a horizontal rule using the passed-in character. Matches the # length used for Index lines in CVS and SVN diffs. # # $1 - character rule() { jot -b "$1" -s "" 67 } # Output a text description of a specified file's type. # # $1 - file pathname. file_type() { stat -f "%HT" $1 | tr "[:upper:]" "[:lower:]" } # Returns true (0) if a file exists # # $1 - file pathname. exists() { [ -e $1 -o -L $1 ] } # Returns true (0) if a file should be ignored, false otherwise. # # $1 - file pathname ignore() { local pattern - set -o noglob for pattern in $IGNORE_FILES; do set +o noglob case $1 in $pattern) return 0 ;; esac set -o noglob done # Ignore /.cshrc and /.profile if they are hardlinked to the # same file in /root. This ensures we only compare those # files once in that case. case $1 in /.cshrc|/.profile) if [ ${DESTDIR}$1 -ef ${DESTDIR}/root$1 ]; then return 0 fi ;; *) ;; esac return 1 } # Returns true (0) if the new version of a file should always be # installed rather than attempting to do a merge. # # $1 - file pathname always_install() { local pattern - set -o noglob for pattern in $ALWAYS_INSTALL; do set +o noglob case $1 in $pattern) return 0 ;; esac set -o noglob done return 1 } # Build a new tree. This runs inside a subshell to trap SIGINT. # # $1 - directory to store new tree in build_tree() ( local destdir dir file make make="make $MAKE_OPTIONS -DNO_FILEMON" log "Building tree at $1 with $make" exec >&3 2>&1 trap 'return 1' INT mkdir -p $1/usr/obj destdir=`realpath $1` if [ -n "$preworld" ]; then # Build a limited tree that only contains files that are # crucial to installworld. for file in $PREWORLD_FILES; do name=$(basename $file) mkdir -p $1/etc || return 1 cp -p $SRCDIR/$file $1/etc/$name || return 1 done elif ! [ -n "$nobuild" ]; then (cd $SRCDIR; $make DESTDIR=$destdir distrib-dirs && MAKEOBJDIRPREFIX=$destdir/usr/obj $make _obj SUBDIR_OVERRIDE=etc && MAKEOBJDIRPREFIX=$destdir/usr/obj $make everything SUBDIR_OVERRIDE=etc && MAKEOBJDIRPREFIX=$destdir/usr/obj $make DESTDIR=$destdir distribution) || \ return 1 else (cd $SRCDIR; $make DESTDIR=$destdir distrib-dirs && $make DESTDIR=$destdir distribution) || return 1 fi chflags -R noschg $1 || return 1 rm -rf $1/usr/obj || return 1 # Purge auto-generated files. Only the source files need to # be updated after which these files are regenerated. rm -f $1/etc/*.db $1/etc/passwd $1/var/db/services.db || return 1 # Remove empty files. These just clutter the output of 'diff'. find $1 -type f -size 0 -delete || return 1 # Trim empty directories. find -d $1 -type d -empty -delete || return 1 return 0 ) # Generate a new tree. If tarball is set, then the tree is # extracted from the tarball. Otherwise the tree is built from a # source tree. # # $1 - directory to store new tree in extract_tree() { local files # If we have a tarball, extract that into the new directory. if [ -n "$tarball" ]; then files= if [ -n "$preworld" ]; then files="$PREWORLD_FILES" fi if ! (mkdir -p $1 && tar xf $tarball -C $1 $files) \ >&3 2>&1; then echo "Failed to extract new tree." remove_tree $1 exit 1 fi else if ! build_tree $1; then echo "Failed to build new tree." remove_tree $1 exit 1 fi fi } # Forcefully remove a tree. Returns true (0) if the operation succeeds. # # $1 - path to tree remove_tree() { rm -rf $1 >&3 2>&1 if [ -e $1 ]; then chflags -R noschg $1 >&3 2>&1 rm -rf $1 >&3 2>&1 fi [ ! -e $1 ] } # Return values for compare() COMPARE_EQUAL=0 COMPARE_ONLYFIRST=1 COMPARE_ONLYSECOND=2 COMPARE_DIFFTYPE=3 COMPARE_DIFFLINKS=4 COMPARE_DIFFFILES=5 # Compare two files/directories/symlinks. Note that this does not # recurse into subdirectories. Instead, if two nodes are both # directories, they are assumed to be equivalent. # # Returns true (0) if the nodes are identical. If only one of the two # nodes are present, return one of the COMPARE_ONLY* constants. If # the nodes are different, return one of the COMPARE_DIFF* constants # to indicate the type of difference. # # $1 - first node # $2 - second node compare() { local first second # If the first node doesn't exist, then check for the second # node. Note that -e will fail for a symbolic link that # points to a missing target. if ! exists $1; then if exists $2; then return $COMPARE_ONLYSECOND else return $COMPARE_EQUAL fi elif ! exists $2; then return $COMPARE_ONLYFIRST fi # If the two nodes are different file types fail. first=`stat -f "%Hp" $1` second=`stat -f "%Hp" $2` if [ "$first" != "$second" ]; then return $COMPARE_DIFFTYPE fi # If both are symlinks, compare the link values. if [ -L $1 ]; then first=`readlink $1` second=`readlink $2` if [ "$first" = "$second" ]; then return $COMPARE_EQUAL else return $COMPARE_DIFFLINKS fi fi # If both are files, compare the file contents. if [ -f $1 ]; then if cmp -s $1 $2; then return $COMPARE_EQUAL else return $COMPARE_DIFFFILES fi fi # As long as the two nodes are the same type of file, consider # them equivalent. return $COMPARE_EQUAL } # Returns true (0) if the only difference between two regular files is a # change in the FreeBSD ID string. # # $1 - path of first file # $2 - path of second file fbsdid_only() { diff -qI '\$FreeBSD.*\$' $1 $2 >/dev/null 2>&1 } # This is a wrapper around compare that will return COMPARE_EQUAL if # the only difference between two regular files is a change in the # FreeBSD ID string. It only makes this adjustment if the -F flag has # been specified. # # $1 - first node # $2 - second node compare_fbsdid() { local cmp compare $1 $2 cmp=$? if [ -n "$FREEBSD_ID" -a "$cmp" -eq $COMPARE_DIFFFILES ] && \ fbsdid_only $1 $2; then return $COMPARE_EQUAL fi return $cmp } # Returns true (0) if a directory is empty. # # $1 - pathname of the directory to check empty_dir() { local contents contents=`ls -A $1` [ -z "$contents" ] } # Returns true (0) if one directories contents are a subset of the # other. This will recurse to handle subdirectories and compares # individual files in the trees. Its purpose is to quiet spurious # directory warnings for dryrun invocations. # # $1 - first directory (sub) # $2 - second directory (super) dir_subset() { local contents file if ! [ -d $1 -a -d $2 ]; then return 1 fi # Ignore files that are present in the second directory but not # in the first. contents=`ls -A $1` for file in $contents; do if ! compare $1/$file $2/$file; then return 1 fi if [ -d $1/$file ]; then if ! dir_subset $1/$file $2/$file; then return 1 fi fi done return 0 } # Returns true (0) if a directory in the destination tree is empty. # If this is a dryrun, then this returns true as long as the contents # of the directory are a subset of the contents in the old tree # (meaning that the directory would be empty in a non-dryrun when this # was invoked) to quiet spurious warnings. # # $1 - pathname of the directory to check relative to DESTDIR. empty_destdir() { if [ -n "$dryrun" ]; then dir_subset $DESTDIR/$1 $OLDTREE/$1 return fi empty_dir $DESTDIR/$1 } # Output a diff of two directory entries with the same relative name # in different trees. Note that as with compare(), this does not # recurse into subdirectories. If the nodes are identical, nothing is # output. # # $1 - first tree # $2 - second tree # $3 - node name # $4 - label for first tree # $5 - label for second tree diffnode() { local first second file old new diffargs if [ -n "$FREEBSD_ID" ]; then diffargs="-I \\\$FreeBSD.*\\\$" else diffargs="" fi compare_fbsdid $1/$3 $2/$3 case $? in $COMPARE_EQUAL) ;; $COMPARE_ONLYFIRST) echo echo "Removed: $3" echo ;; $COMPARE_ONLYSECOND) echo echo "Added: $3" echo ;; $COMPARE_DIFFTYPE) first=`file_type $1/$3` second=`file_type $2/$3` echo echo "Node changed from a $first to a $second: $3" echo ;; $COMPARE_DIFFLINKS) first=`readlink $1/$file` second=`readlink $2/$file` echo echo "Link changed: $file" rule "=" echo "-$first" echo "+$second" echo ;; $COMPARE_DIFFFILES) echo "Index: $3" rule "=" diff -u $diffargs -L "$3 ($4)" $1/$3 -L "$3 ($5)" $2/$3 ;; esac } # Run one-off commands after an update has completed. These commands # are not tied to a specific file, so they cannot be handled by # post_install_file(). post_update() { local args # None of these commands should be run for a pre-world update. if [ -n "$preworld" ]; then return fi # If /etc/localtime exists and is not a symlink and /var/db/zoneinfo # exists, run tzsetup -r to refresh /etc/localtime. if [ -f ${DESTDIR}/etc/localtime -a \ ! -L ${DESTDIR}/etc/localtime ]; then if [ -f ${DESTDIR}/var/db/zoneinfo ]; then if [ -n "${DESTDIR}" ]; then args="-C ${DESTDIR}" else args="" fi log "tzsetup -r ${args}" if [ -z "$dryrun" ]; then tzsetup -r ${args} >&3 2>&1 fi else warn "Needs update: /etc/localtime (required" \ "manual update via tzsetup(8))" fi fi } # Create missing parent directories of a node in a target tree # preserving the owner, group, and permissions from a specified # template tree. # # $1 - template tree # $2 - target tree # $3 - pathname of the node (relative to both trees) install_dirs() { local args dir dir=`dirname $3` # Nothing to do if the parent directory exists. This also # catches the degenerate cases when the path is just a simple # filename. if [ -d ${2}$dir ]; then return 0 fi # If non-directory file exists with the desired directory # name, then fail. if exists ${2}$dir; then # If this is a dryrun and we are installing the # directory in the DESTDIR and the file in the DESTDIR # matches the file in the old tree, then fake success # to quiet spurious warnings. if [ -n "$dryrun" -a "$2" = "$DESTDIR" ]; then if compare $OLDTREE/$dir $DESTDIR/$dir; then return 0 fi fi args=`file_type ${2}$dir` warn "Directory mismatch: ${2}$dir ($args)" return 1 fi # Ensure the parent directory of the directory is present # first. if ! install_dirs $1 "$2" $dir; then return 1 fi # Format attributes from template directory as install(1) # arguments. args=`stat -f "-o %Su -g %Sg -m %0Mp%0Lp" $1/$dir` log "install -d $args ${2}$dir" if [ -z "$dryrun" ]; then install -d $args ${2}$dir >&3 2>&1 fi return 0 } # Perform post-install fixups for a file. This largely consists of # regenerating any files that depend on the newly installed file. # # $1 - pathname of the updated file (relative to DESTDIR) post_install_file() { case $1 in /etc/mail/aliases) # Grr, newaliases only works for an empty DESTDIR. if [ -z "$DESTDIR" ]; then log "newaliases" if [ -z "$dryrun" ]; then newaliases >&3 2>&1 fi else NEWALIAS_WARN=yes fi ;; /usr/share/certs/trusted/* | /usr/share/certs/blacklisted/*) log "certctl rehash" if [ -z "$dryrun" ]; then env DESTDIR=${DESTDIR} certctl rehash >&3 2>&1 fi ;; /etc/login.conf) log "cap_mkdb ${DESTDIR}$1" if [ -z "$dryrun" ]; then cap_mkdb ${DESTDIR}$1 >&3 2>&1 fi ;; /etc/master.passwd) log "pwd_mkdb -p -d $DESTDIR/etc ${DESTDIR}$1" if [ -z "$dryrun" ]; then pwd_mkdb -p -d $DESTDIR/etc ${DESTDIR}$1 \ >&3 2>&1 fi ;; /etc/motd) # /etc/rc.d/motd hardcodes the /etc/motd path. # Don't warn about non-empty DESTDIR's since this # change is only cosmetic anyway. if [ -z "$DESTDIR" ]; then log "sh /etc/rc.d/motd start" if [ -z "$dryrun" ]; then sh /etc/rc.d/motd start >&3 2>&1 fi fi ;; /etc/services) log "services_mkdb -q -o $DESTDIR/var/db/services.db" \ "${DESTDIR}$1" if [ -z "$dryrun" ]; then services_mkdb -q -o $DESTDIR/var/db/services.db \ ${DESTDIR}$1 >&3 2>&1 fi ;; esac } # Install the "new" version of a file. Returns true if it succeeds # and false otherwise. # # $1 - pathname of the file to install (relative to DESTDIR) install_new() { if ! install_dirs $NEWTREE "$DESTDIR" $1; then return 1 fi log "cp -Rp ${NEWTREE}$1 ${DESTDIR}$1" if [ -z "$dryrun" ]; then cp -Rp ${NEWTREE}$1 ${DESTDIR}$1 >&3 2>&1 fi post_install_file $1 return 0 } # Install the "resolved" version of a file. Returns true if it succeeds # and false otherwise. # # $1 - pathname of the file to install (relative to DESTDIR) install_resolved() { # This should always be present since the file is already # there (it caused a conflict). However, it doesn't hurt to # just be safe. if ! install_dirs $NEWTREE "$DESTDIR" $1; then return 1 fi log "cp -Rp ${CONFLICTS}$1 ${DESTDIR}$1" cp -Rp ${CONFLICTS}$1 ${DESTDIR}$1 >&3 2>&1 post_install_file $1 return 0 } # Generate a conflict file when a "new" file conflicts with an # existing file in DESTDIR. # # $1 - pathname of the file that conflicts (relative to DESTDIR) new_conflict() { if [ -n "$dryrun" ]; then return fi install_dirs $NEWTREE $CONFLICTS $1 diff --changed-group-format='<<<<<<< (local) %<======= %>>>>>>>> (stock) ' $DESTDIR/$1 $NEWTREE/$1 > $CONFLICTS/$1 } # Remove the "old" version of a file. # # $1 - pathname of the old file to remove (relative to DESTDIR) remove_old() { log "rm -f ${DESTDIR}$1" if [ -z "$dryrun" ]; then rm -f ${DESTDIR}$1 >&3 2>&1 fi echo " D $1" } # Update a file that has no local modifications. # # $1 - pathname of the file to update (relative to DESTDIR) update_unmodified() { local new old # If the old file is a directory, then remove it with rmdir # (this should only happen if the file has changed its type # from a directory to a non-directory). If the directory # isn't empty, then fail. This will be reported as a warning # later. if [ -d $DESTDIR/$1 ]; then if empty_destdir $1; then log "rmdir ${DESTDIR}$1" if [ -z "$dryrun" ]; then rmdir ${DESTDIR}$1 >&3 2>&1 fi else return 1 fi # If both the old and new files are regular files, leave the # existing file. This avoids breaking hard links for /.cshrc # and /.profile. Otherwise, explicitly remove the old file. elif ! [ -f ${DESTDIR}$1 -a -f ${NEWTREE}$1 ]; then log "rm -f ${DESTDIR}$1" if [ -z "$dryrun" ]; then rm -f ${DESTDIR}$1 >&3 2>&1 fi fi # If the new file is a directory, note that the old file has # been removed, but don't do anything else for now. The # directory will be installed if needed when new files within # that directory are installed. if [ -d $NEWTREE/$1 ]; then if empty_dir $NEWTREE/$1; then echo " D $file" else echo " U $file" fi elif install_new $1; then echo " U $file" fi return 0 } # Update the FreeBSD ID string in a locally modified file to match the # FreeBSD ID string from the "new" version of the file. # # $1 - pathname of the file to update (relative to DESTDIR) update_freebsdid() { local new dest file # If the FreeBSD ID string is removed from the local file, # there is nothing to do. In this case, treat the file as # updated. Otherwise, if either file has more than one # FreeBSD ID string, just punt and let the user handle the # conflict manually. new=`grep -c '\$FreeBSD.*\$' ${NEWTREE}$1` dest=`grep -c '\$FreeBSD.*\$' ${DESTDIR}$1` if [ "$dest" -eq 0 ]; then return 0 fi if [ "$dest" -ne 1 -o "$dest" -ne 1 ]; then return 1 fi # If the FreeBSD ID string in the new file matches the FreeBSD ID # string in the local file, there is nothing to do. new=`grep '\$FreeBSD.*\$' ${NEWTREE}$1` dest=`grep '\$FreeBSD.*\$' ${DESTDIR}$1` if [ "$new" = "$dest" ]; then return 0 fi # Build the new file in three passes. First, copy all the # lines preceding the FreeBSD ID string from the local version # of the file. Second, append the FreeBSD ID string line from # the new version. Finally, append all the lines after the # FreeBSD ID string from the local version of the file. file=`mktemp $WORKDIR/etcupdate-XXXXXXX` awk '/\$FreeBSD.*\$/ { exit } { print }' ${DESTDIR}$1 >> $file awk '/\$FreeBSD.*\$/ { print }' ${NEWTREE}$1 >> $file awk '/\$FreeBSD.*\$/ { ok = 1; next } { if (ok) print }' \ ${DESTDIR}$1 >> $file # As an extra sanity check, fail the attempt if the updated # version of the file has any differences aside from the # FreeBSD ID string. if ! fbsdid_only ${DESTDIR}$1 $file; then rm -f $file return 1 fi log "cp $file ${DESTDIR}$1" if [ -z "$dryrun" ]; then cp $file ${DESTDIR}$1 >&3 2>&1 fi rm -f $file post_install_file $1 echo " M $1" return 0 } # Attempt to update a file that has local modifications. This routine # only handles regular files. If the 3-way merge succeeds without # conflicts, the updated file is installed. If the merge fails, the # merged version with conflict markers is left in the CONFLICTS tree. # # $1 - pathname of the file to merge (relative to DESTDIR) merge_file() { local res # Try the merge to see if there is a conflict. diff3 -E -m ${DESTDIR}$1 ${OLDTREE}$1 ${NEWTREE}$1 > /dev/null 2>&3 res=$? case $res in 0) # No conflicts, so just redo the merge to the # real file. log "diff3 -E -m ${DESTDIR}$1 ${OLDTREE}$1 ${NEWTREE}$1" if [ -z "$dryrun" ]; then temp=$(mktemp -t etcupdate) diff3 -E -m ${DESTDIR}$1 ${OLDTREE}$1 ${NEWTREE}$1 > ${temp} # Use "cat >" to preserve metadata. cat ${temp} > ${DESTDIR}$1 rm -f ${temp} fi post_install_file $1 echo " M $1" ;; 1) # Conflicts, save a version with conflict markers in # the conflicts directory. if [ -z "$dryrun" ]; then install_dirs $NEWTREE $CONFLICTS $1 log "diff3 -m -A ${DESTDIR}$1 ${CONFLICTS}$1" diff3 -m -A -L "yours" -L "original" -L "new" \ ${DESTDIR}$1 ${OLDTREE}$1 ${NEWTREE}$1 > \ ${CONFLICTS}$1 fi echo " C $1" ;; *) panic "merge failed with status $res" ;; esac } # Returns true if a file contains conflict markers from a merge conflict. # # $1 - pathname of the file to resolve (relative to DESTDIR) has_conflicts() { egrep -q '^(<{7}|\|{7}|={7}|>{7}) ' $CONFLICTS/$1 } # Attempt to resolve a conflict. The user is prompted to choose an # action for each conflict. If the user edits the file, they are # prompted again for an action. The process is very similar to # resolving conflicts after an update or merge with Perforce or # Subversion. The prompts are modelled on a subset of the available # commands for resolving conflicts with Subversion. # # $1 - pathname of the file to resolve (relative to DESTDIR) resolve_conflict() { local command junk echo "Resolving conflict in '$1':" edit= while true; do # Only display the resolved command if the file # doesn't contain any conflicts. echo -n "Select: (p) postpone, (df) diff-full, (e) edit," if ! has_conflicts $1; then echo -n " (r) resolved," fi echo echo -n " (h) help for more options: " read command case $command in df) diff -u ${DESTDIR}$1 ${CONFLICTS}$1 ;; e) $EDITOR ${CONFLICTS}$1 ;; h) cat </dev/null 2>&1 fi echo " D $dir" else warn "Non-empty directory remains: $dir" fi fi } # Handle a file that exists in both the old and new trees. If the # file has not changed in the old and new trees, there is nothing to # do. If the file in the destination directory matches the new file, # there is nothing to do. If the file in the destination directory # matches the old file, then the new file should be installed. # Everything else becomes some sort of conflict with more detailed # handling. # # $1 - pathname of the file (relative to DESTDIR) handle_modified_file() { local cmp dest file new newdestcmp old file=$1 if ignore $file; then log "IGNORE: modified file $file" return fi compare $OLDTREE/$file $NEWTREE/$file cmp=$? if [ $cmp -eq $COMPARE_EQUAL ]; then return fi if [ $cmp -eq $COMPARE_ONLYFIRST -o $cmp -eq $COMPARE_ONLYSECOND ]; then panic "Changed file now missing" fi compare $NEWTREE/$file $DESTDIR/$file newdestcmp=$? if [ $newdestcmp -eq $COMPARE_EQUAL ]; then return fi # If the only change in the new file versus the destination # file is a change in the FreeBSD ID string and -F is # specified, just install the new file. if [ -n "$FREEBSD_ID" -a $newdestcmp -eq $COMPARE_DIFFFILES ] && \ fbsdid_only $NEWTREE/$file $DESTDIR/$file; then if update_unmodified $file; then return else panic "Updating FreeBSD ID string failed" fi fi # If the local file is the same as the old file, install the # new file. If -F is specified and the only local change is # in the FreeBSD ID string, then install the new file as well. if compare_fbsdid $OLDTREE/$file $DESTDIR/$file; then if update_unmodified $file; then return fi fi # If the file was removed from the dest tree, just whine. if [ $newdestcmp -eq $COMPARE_ONLYFIRST ]; then # If the removed file matches an ALWAYS_INSTALL glob, # then just install the new version of the file. if always_install $file; then log "ALWAYS: adding $file" if ! [ -d $NEWTREE/$file ]; then if install_new $file; then echo " A $file" fi fi return fi # If the only change in the new file versus the old # file is a change in the FreeBSD ID string and -F is # specified, don't warn. if [ -n "$FREEBSD_ID" -a $cmp -eq $COMPARE_DIFFFILES ] && \ fbsdid_only $OLDTREE/$file $NEWTREE/$file; then return fi case $cmp in $COMPARE_DIFFTYPE) old=`file_type $OLDTREE/$file` new=`file_type $NEWTREE/$file` warn "Remove mismatch: $file ($old became $new)" ;; $COMPARE_DIFFLINKS) old=`readlink $OLDTREE/$file` new=`readlink $NEWTREE/$file` warn \ "Removed link changed: $file (\"$old\" became \"$new\")" ;; $COMPARE_DIFFFILES) warn "Removed file changed: $file" ;; esac return fi # Treat the file as unmodified and force install of the new # file if it matches an ALWAYS_INSTALL glob. If the update # attempt fails, then fall through to the normal case so a # warning is generated. if always_install $file; then log "ALWAYS: updating $file" if update_unmodified $file; then return fi fi # If the only change in the new file versus the old file is a # change in the FreeBSD ID string and -F is specified, just # update the FreeBSD ID string in the local file. if [ -n "$FREEBSD_ID" -a $cmp -eq $COMPARE_DIFFFILES ] && \ fbsdid_only $OLDTREE/$file $NEWTREE/$file; then if update_freebsdid $file; then continue fi fi # If the file changed types between the old and new trees but # the files in the new and dest tree are both of the same # type, treat it like an added file just comparing the new and # dest files. if [ $cmp -eq $COMPARE_DIFFTYPE ]; then case $newdestcmp in $COMPARE_DIFFLINKS) new=`readlink $NEWTREE/$file` dest=`readlink $DESTDIR/$file` warn \ "New link conflict: $file (\"$new\" vs \"$dest\")" return ;; $COMPARE_DIFFFILES) new_conflict $file echo " C $file" return ;; esac else # If the file has not changed types between the old # and new trees, but it is a different type in # DESTDIR, then just warn. if [ $newdestcmp -eq $COMPARE_DIFFTYPE ]; then new=`file_type $NEWTREE/$file` dest=`file_type $DESTDIR/$file` warn "Modified mismatch: $file ($new vs $dest)" return fi fi case $cmp in $COMPARE_DIFFTYPE) old=`file_type $OLDTREE/$file` new=`file_type $NEWTREE/$file` dest=`file_type $DESTDIR/$file` warn "Modified $dest changed: $file ($old became $new)" ;; $COMPARE_DIFFLINKS) old=`readlink $OLDTREE/$file` new=`readlink $NEWTREE/$file` warn \ "Modified link changed: $file (\"$old\" became \"$new\")" ;; $COMPARE_DIFFFILES) merge_file $file ;; esac } # Handle a file that has been added in the new tree. If the file does # not exist in DESTDIR, simply copy the file into DESTDIR. If the # file exists in the DESTDIR and is identical to the new version, do # nothing. Otherwise, generate a diff of the two versions of the file # and mark it as a conflict. # # $1 - pathname of the file (relative to DESTDIR) handle_added_file() { local cmp dest file new file=$1 if ignore $file; then log "IGNORE: added file $file" return fi compare $DESTDIR/$file $NEWTREE/$file cmp=$? case $cmp in $COMPARE_EQUAL) return ;; $COMPARE_ONLYFIRST) panic "Added file now missing" ;; $COMPARE_ONLYSECOND) # Ignore new directories. They will be # created as needed when non-directory nodes # are installed. if ! [ -d $NEWTREE/$file ]; then if install_new $file; then echo " A $file" fi fi return ;; esac # Treat the file as unmodified and force install of the new # file if it matches an ALWAYS_INSTALL glob. If the update # attempt fails, then fall through to the normal case so a # warning is generated. if always_install $file; then log "ALWAYS: updating $file" if update_unmodified $file; then return fi fi case $cmp in $COMPARE_DIFFTYPE) new=`file_type $NEWTREE/$file` dest=`file_type $DESTDIR/$file` warn "New file mismatch: $file ($new vs $dest)" ;; $COMPARE_DIFFLINKS) new=`readlink $NEWTREE/$file` dest=`readlink $DESTDIR/$file` warn "New link conflict: $file (\"$new\" vs \"$dest\")" ;; $COMPARE_DIFFFILES) # If the only change in the new file versus # the destination file is a change in the # FreeBSD ID string and -F is specified, just # install the new file. if [ -n "$FREEBSD_ID" ] && \ fbsdid_only $NEWTREE/$file $DESTDIR/$file; then if update_unmodified $file; then return else panic \ "Updating FreeBSD ID string failed" fi fi new_conflict $file echo " C $file" ;; esac } # Main routines for each command # Build a new tree and save it in a tarball. build_cmd() { local dir if [ $# -ne 1 ]; then echo "Missing required tarball." echo usage fi log "build command: $1" # Create a temporary directory to hold the tree dir=`mktemp -d $WORKDIR/etcupdate-XXXXXXX` if [ $? -ne 0 ]; then echo "Unable to create temporary directory." exit 1 fi if ! build_tree $dir; then echo "Failed to build tree." remove_tree $dir exit 1 fi if ! tar cfj $1 -C $dir . >&3 2>&1; then echo "Failed to create tarball." remove_tree $dir exit 1 fi remove_tree $dir } # Output a diff comparing the tree at DESTDIR to the current # unmodified tree. Note that this diff does not include files that # are present in DESTDIR but not in the unmodified tree. diff_cmd() { local file if [ $# -ne 0 ]; then usage fi # Requires an unmodified tree to diff against. if ! [ -d $NEWTREE ]; then echo "Reference tree to diff against unavailable." exit 1 fi # Unfortunately, diff alone does not quite provide the right # level of options that we want, so improvise. for file in `(cd $NEWTREE; find .) | sed -e 's/^\.//'`; do if ignore $file; then continue fi diffnode $NEWTREE "$DESTDIR" $file "stock" "local" done } # Just extract a new tree into NEWTREE either by building a tree or # extracting a tarball. This can be used to bootstrap updates by # initializing the current "stock" tree to match the currently # installed system. # # Unlike 'update', this command does not rotate or preserve an # existing NEWTREE, it just replaces any existing tree. extract_cmd() { if [ $# -ne 0 ]; then usage fi log "extract command: tarball=$tarball" # Create a temporary directory to hold the tree dir=`mktemp -d $WORKDIR/etcupdate-XXXXXXX` if [ $? -ne 0 ]; then echo "Unable to create temporary directory." exit 1 fi extract_tree $dir if [ -d $NEWTREE ]; then if ! remove_tree $NEWTREE; then echo "Unable to remove current tree." remove_tree $dir exit 1 fi fi if ! mv $dir $NEWTREE >&3 2>&1; then echo "Unable to rename temp tree to current tree." remove_tree $dir exit 1 fi } # Resolve conflicts left from an earlier merge. resolve_cmd() { local conflicts if [ $# -ne 0 ]; then usage fi if ! [ -d $CONFLICTS ]; then return fi if ! [ -d $NEWTREE ]; then echo "The current tree is not present to resolve conflicts." exit 1 fi conflicts=`(cd $CONFLICTS; find . ! -type d) | sed -e 's/^\.//'` for file in $conflicts; do resolve_conflict $file done if [ -n "$NEWALIAS_WARN" ]; then warn "Needs update: /etc/mail/aliases.db" \ "(requires manual update via newaliases(1))" echo echo "Warnings:" echo " Needs update: /etc/mail/aliases.db" \ "(requires manual update via newaliases(1))" fi } +# Restore files to the stock version. Only files with a local change +# are restored from the stock version. +revert_cmd() +{ + local cmp file + + if [ $# -eq 0 ]; then + usage + fi + + for file; do + log "revert $file" + + if ! [ -e $NEWTREE/$file ]; then + echo "File $file does not exist in the current tree." + exit 1 + fi + if [ -d $NEWTREE/$file ]; then + echo "File $file is a directory." + exit 1 + fi + + compare $DESTDIR/$file $NEWTREE/$file + cmp=$? + if [ $cmp -eq $COMPARE_EQUAL ]; then + continue + fi + + if update_unmodified $file; then + # If this file had a conflict, clean up the + # conflict. + if [ -e $CONFLICTS/$file ]; then + if ! rm $CONFLICTS/$file >&3 2>&1; then + echo "Failed to remove conflict " \ + "for $file". + fi + fi + fi + done +} + # Report a summary of the previous merge. Specifically, list any # remaining conflicts followed by any warnings from the previous # update. status_cmd() { if [ $# -ne 0 ]; then usage fi if [ -d $CONFLICTS ]; then (cd $CONFLICTS; find . ! -type d) | sed -e 's/^\./ C /' fi if [ -s $WARNINGS ]; then echo "Warnings:" cat $WARNINGS fi } # Perform an actual merge. The new tree can either already exist (if # rerunning a merge), be extracted from a tarball, or generated from a # source tree. update_cmd() { local dir new old if [ $# -ne 0 ]; then usage fi log "update command: rerun=$rerun tarball=$tarball preworld=$preworld" if [ `id -u` -ne 0 ]; then echo "Must be root to update a tree." exit 1 fi # Enforce a sane umask umask 022 # XXX: Should existing conflicts be ignored and removed during # a rerun? # Trim the conflicts tree. Whine if there is anything left. if [ -e $CONFLICTS ]; then find -d $CONFLICTS -type d -empty -delete >&3 2>&1 rmdir $CONFLICTS >&3 2>&1 fi if [ -d $CONFLICTS ]; then echo "Conflicts remain from previous update, aborting." exit 1 fi # Save tree names to use for rotation later. old=$OLDTREE new=$NEWTREE if [ -z "$rerun" ]; then # Extract the new tree to a temporary directory. The # trees are only rotated after a successful update to # avoid races if an update command is interrupted # before it completes. dir=`mktemp -d $WORKDIR/etcupdate-XXXXXXX` if [ $? -ne 0 ]; then echo "Unable to create temporary directory." exit 1 fi # Populate the new tree. extract_tree $dir # Compare the new tree against the previous tree. For # the preworld case OLDTREE already points to the # current stock tree. if [ -z "$preworld" ]; then OLDTREE=$NEWTREE fi NEWTREE=$dir fi if ! [ -d $OLDTREE ]; then cat < $WORKDIR/old.files (cd $NEWTREE; find .) | sed -e 's/^\.//' | sort > $WORKDIR/new.files # Split the files up into three groups using comm. comm -23 $WORKDIR/old.files $WORKDIR/new.files > $WORKDIR/removed.files comm -13 $WORKDIR/old.files $WORKDIR/new.files > $WORKDIR/added.files comm -12 $WORKDIR/old.files $WORKDIR/new.files > $WORKDIR/both.files # Initialize conflicts and warnings handling. rm -f $WARNINGS mkdir -p $CONFLICTS # Ignore removed files for the pre-world case. A pre-world # update uses a stripped-down tree. if [ -n "$preworld" ]; then > $WORKDIR/removed.files fi # The order for the following sections is important. In the # odd case that a directory is converted into a file, the # existing subfiles need to be removed if possible before the # file is converted. Similarly, in the case that a file is # converted into a directory, the file needs to be converted # into a directory if possible before the new files are added. # First, handle removed files. for file in `cat $WORKDIR/removed.files`; do handle_removed_file $file done # For the directory pass, reverse sort the list to effect a # depth-first traversal. This is needed to ensure that if a # directory with subdirectories is removed, the entire # directory is removed if there are no local modifications. for file in `sort -r $WORKDIR/removed.files`; do handle_removed_directory $file done # Second, handle files that exist in both the old and new # trees. for file in `cat $WORKDIR/both.files`; do handle_modified_file $file done # Finally, handle newly added files. for file in `cat $WORKDIR/added.files`; do handle_added_file $file done if [ -n "$NEWALIAS_WARN" ]; then warn "Needs update: /etc/mail/aliases.db" \ "(requires manual update via newaliases(1))" fi # Run any special one-off commands after an update has completed. post_update if [ -s $WARNINGS ]; then echo "Warnings:" cat $WARNINGS fi # Finally, rotate any needed trees. if [ "$new" != "$NEWTREE" ]; then if [ -n "$rerun" ]; then panic "Should not have a temporary directory" fi if [ -z "$dir" ]; then panic "Should have a temporary directory" fi # Rotate the old tree if needed if [ "$old" != "$OLDTREE" ]; then if [ -n "$preworld" ]; then panic "Old tree should be unchanged" fi if ! remove_tree $old; then echo "Unable to remove previous old tree." exit 1 fi if ! mv $OLDTREE $old >&3 2>&1; then echo "Unable to rename old tree." exit 1 fi fi # Rotate the new tree. Remove a previous pre-world # tree if it exists. if [ -d $new ]; then if [ -z "$preworld" ]; then panic "New tree should be rotated to old" fi if ! remove_tree $new; then echo "Unable to remove previous pre-world tree." exit 1 fi fi if ! mv $NEWTREE $new >&3 2>&1; then echo "Unable to rename current tree." exit 1 fi fi } # Determine which command we are executing. A command may be # specified as the first word. If one is not specified then 'update' # is assumed as the default command. command="update" if [ $# -gt 0 ]; then case "$1" in - build|diff|extract|status|resolve) + build|diff|extract|status|resolve|revert) command="$1" shift ;; -*) # If first arg is an option, assume the # default command. ;; *) usage ;; esac fi # Set default variable values. # The path to the source tree used to build trees. SRCDIR=/usr/src # The destination directory where the modified files live. DESTDIR= # Ignore changes in the FreeBSD ID string. FREEBSD_ID= # Files that should always have the new version of the file installed. ALWAYS_INSTALL= # Files to ignore and never update during a merge. IGNORE_FILES= # Flags to pass to 'make' when building a tree. MAKE_OPTIONS= # Include a config file if it exists. Note that command line options # override any settings in the config file. More details are in the # manual, but in general the following variables can be set: # - ALWAYS_INSTALL # - DESTDIR # - EDITOR # - FREEBSD_ID # - IGNORE_FILES # - LOGFILE # - MAKE_OPTIONS # - SRCDIR # - WORKDIR if [ -r /etc/etcupdate.conf ]; then . /etc/etcupdate.conf fi # Parse command line options tarball= rerun= always= dryrun= ignore= nobuild= preworld= while getopts "d:nprs:t:A:BD:FI:L:M:" option; do case "$option" in d) WORKDIR=$OPTARG ;; n) dryrun=YES ;; p) preworld=YES ;; r) rerun=YES ;; s) SRCDIR=$OPTARG ;; t) tarball=$OPTARG ;; A) # To allow this option to be specified # multiple times, accumulate command-line # specified patterns in an 'always' variable # and use that to overwrite ALWAYS_INSTALL # after parsing all options. Need to be # careful here with globbing expansion. set -o noglob always="$always $OPTARG" set +o noglob ;; B) nobuild=YES ;; D) DESTDIR=$OPTARG ;; F) FREEBSD_ID=YES ;; I) # To allow this option to be specified # multiple times, accumulate command-line # specified patterns in an 'ignore' variable # and use that to overwrite IGNORE_FILES after # parsing all options. Need to be careful # here with globbing expansion. set -o noglob ignore="$ignore $OPTARG" set +o noglob ;; L) LOGFILE=$OPTARG ;; M) MAKE_OPTIONS="$OPTARG" ;; *) echo usage ;; esac done shift $((OPTIND - 1)) # Allow -A command line options to override ALWAYS_INSTALL set from # the config file. set -o noglob if [ -n "$always" ]; then ALWAYS_INSTALL="$always" fi # Allow -I command line options to override IGNORE_FILES set from the # config file. if [ -n "$ignore" ]; then IGNORE_FILES="$ignore" fi set +o noglob # Where the "old" and "new" trees are stored. WORKDIR=${WORKDIR:-$DESTDIR/var/db/etcupdate} # Log file for verbose output from program that are run. The log file # is opened on fd '3'. LOGFILE=${LOGFILE:-$WORKDIR/log} # The path of the "old" tree OLDTREE=$WORKDIR/old # The path of the "new" tree NEWTREE=$WORKDIR/current # The path of the "conflicts" tree where files with merge conflicts are saved. CONFLICTS=$WORKDIR/conflicts # The path of the "warnings" file that accumulates warning notes from an update. WARNINGS=$WORKDIR/warnings # Use $EDITOR for resolving conflicts. If it is not set, default to vi. EDITOR=${EDITOR:-/usr/bin/vi} # Files that need to be updated before installworld. PREWORLD_FILES="etc/master.passwd etc/group" # Handle command-specific argument processing such as complaining # about unsupported options. Since the configuration file is always # included, do not complain about extra command line arguments that # may have been set via the config file rather than the command line. case $command in update) if [ -n "$rerun" -a -n "$tarball" ]; then echo "Only one of -r or -t can be specified." echo usage fi if [ -n "$rerun" -a -n "$preworld" ]; then echo "Only one of -p or -r can be specified." echo usage fi ;; - build|diff|status) + build|diff|status|revert) if [ -n "$dryrun" -o -n "$rerun" -o -n "$tarball" -o \ -n "$preworld" ]; then usage fi ;; resolve) if [ -n "$dryrun" -o -n "$rerun" -o -n "$tarball" ]; then usage fi ;; extract) if [ -n "$dryrun" -o -n "$rerun" -o -n "$preworld" ]; then usage fi ;; esac # Pre-world mode uses a different set of trees. It leaves the current # tree as-is so it is still present for a full etcupdate run after the # world install is complete. Instead, it installs a few critical files # into a separate tree. if [ -n "$preworld" ]; then OLDTREE=$NEWTREE NEWTREE=$WORKDIR/preworld fi # Open the log file. Don't truncate it if doing a minor operation so # that a minor operation doesn't lose log info from a major operation. if ! mkdir -p $WORKDIR 2>/dev/null; then echo "Failed to create work directory $WORKDIR" fi case $command in - diff|resolve|status) + diff|resolve|revert|status) exec 3>>$LOGFILE ;; *) exec 3>$LOGFILE ;; esac ${command}_cmd "$@"