Index: releng/8.4/UPDATING =================================================================== --- releng/8.4/UPDATING (revision 282873) +++ releng/8.4/UPDATING (revision 282874) @@ -1,2048 +1,2053 @@ Updating Information for FreeBSD current users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. NOTE TO PEOPLE WHO THINK THAT FreeBSD 8.x IS SLOW ON IA64 OR SUN4V: For ia64 the INVARIANTS and INVARIANT_SUPPORT kernel options were left in the GENERIC kernel because the kernel does not work properly without them. For sun4v all of the normal kernel debugging tools present in HEAD were left in place because sun4v support still needs work to become production ready. +20150513: p28 FreeBSD-EN-15:04.freebsd-update + + Fix bug with freebsd-update(8) that does not ensure the previous + upgrade was completed. [EN-15:04] + 20150407: p27 FreeBSD-SA-15:04.igmp [revised] FreeBSD-SA-15:07.ntp FreeBSD-SA-15:09.ipv6 Improved patch for SA-15:04.igmp. Fix multiple vulnerabilities of ntp. [SA-15:07] Fix Denial of Service with IPv6 Router Advertisements. [SA-15:09] 20150320: p26 Fix patch for SA-15:06.openssl. 20150319: p25 FreeBSD-SA-15:06.openssl Fix multiple vulnerabilities in OpenSSL. [SA-15:06] 20150225: p24 FreeBSD-SA-15:04.igmp FreeBSD-SA-15:05.bind FreeBSD-EN-15:01.vt FreeBSD-EN-15:02.openssl FreeBSD-EN-15:03.freebsd-update Fix integer overflow in IGMP protocol. [SA-15:04] Fix BIND remote denial of service vulnerability. [SA-15:05] Fix vt(4) crash with improper ioctl parameters. [EN-15:01] Updated base system OpenSSL to 0.9.8zd. [EN-15:02] Fix freebsd-update libraries update ordering issue. [EN-15:03] 20150127: p23 FreeBSD-SA-15:02.kmem FreeBSD-SA-15:03.sctp Fix SCTP SCTP_SS_VALUE kernel memory corruption and disclosure vulnerability. [SA-15:02] Fix SCTP stream reset vulnerability. [SA-15:03] 20150114: p22 FreeBSD-SA-15:01.openssl Fix multiple vulnerabilities in OpenSSL. [SA-15:01] 20141223: p21 FreeBSD-SA-14:31.ntp FreeBSD-EN-14:13.freebsd-update Fix multiple vulnerabilities in NTP suite. [SA-14:31] Fix directory deletion issue in freebsd-update. [EN-14:13] 20141210: p20 FreeBSD-SA-14:28.file FreeBSD-SA-14:29.bind Fix multiple vulnerabilities in file(1) and libmagic(3). [SA-14:28] Fix BIND remote denial of service vulnerability. [SA-14:29] 20141104: p19 FreeBSD-SA-14:25.setlogin FreeBSD-SA-14:26.ftp FreeBSD-EN-14:12.zfs Fix kernel stack disclosure in setlogin(2) / getlogin(2). [SA-14:25] Fix remote command execution in ftp(1). [SA-14:26] Fix NFSv4 and ZFS cache consistency issue. [EN-14:12] 20141022: p18 FreeBSD-EN-14:10.tzdata Time zone data file update. [EN-14:10] 20141021: p17 FreeBSD-SA-14:21.routed FreeBSD-SA-14:23.openssl Fix routed(8) remote denial of service vulnerability. [SA-14:21] Fix OpenSSL multiple vulnerabilities. [SA-14:23] 20140916: p16 FreeBSD-SA-14:19.tcp Fix Denial of Service in TCP packet processing. [SA-14:19] 20140909: p15 FreeBSD-SA-14:18.openssl Fix OpenSSL multiple vulnerabilities. [SA-14:18] 20140708: p14 FreeBSD-SA-14:17.kmem FreeBSD-EN-14:09.jail Fix kernel memory disclosure in control messages and SCTP notifications. [SA-14:17] Fix jail fails to start if WITHOUT_INET/WITHOUT_INET6 is use. [EN-14:09] 20140624: p13 FreeBSD-SA-14:16.file FreeBSD-EN-14:08.heimdal Fix multiple vulnerabilities in file(1) and libmagic(3). [SA-14:16] Fix gss_pseudo_random interoperability issue. [EN-14:08] 20140605: p12 FreeBSD-SA-14:14.openssl Fix OpenSSL multiple vulnerabilities. [SA-14:14] 20140603: p11 FreeBSD-SA-14:11.sendmail FreeBSD-SA-14:12.ktrace FreeBSD-EN-14:06.exec Fix sendmail improper close-on-exec flag handling. [SA-14:11] Fix ktrace memory disclosure. [SA-14:12] Fix triple-fault when executing from a threaded process. [EN-14:06] 20140513: p10 FreeBSD-EN-14:03.pkg FreeBSD-EN-14:04.kldxref Add pkg bootstrapping, configuration and public keys. [EN-14:03] Improve build repeatability for kldxref(8). [EN-14:04] 20140430: p9 FreeBSD-SA-14:08.tcp Fix TCP reassembly vulnerability. [SA-14:08] 20140408: p8 FreeBSD-SA-14:05.nfsserver FreeBSD-SA-14:06.openssl Fix deadlock in the NFS server. [SA-14:05] Fix for ECDSA Cache Side-channel Attack in OpenSSL. [SA-14:06] 20140114: p7 FreeBSD-SA-14:01.bsnmpd FreeBSD-SA-14:02.ntpd FreeBSD-SA-14:04.bind FreeBSD-EN-14:01.random FreeBSD-EN-14:02.mmap Fix bsnmpd remote denial of service vulnerability. [SA-14:01] Fix ntpd distributed reflection Denial of Service vulnerability. [SA-14:02] Fix BIND remote denial of service vulnerability. [SA-14:04] Disable hardware RNGs by default. [EN-14:01] Fix incorrect coalescing of stack entry with mmap. [EN-14:02] 20131128: p6 FreeBSD-EN-13:05.freebsd-update Fix error in patch for FreeBSD-EN-13:04.freebsd-update. 20131026: p5 FreeBSD-EN-13:04.freebsd-update Fix multiple freebsd-update bugs that break upgrading to FreeBSD 10.0. 20130910: p4 FreeBSD-SA-13:12.ifioctl FreeBSD-SA-13:13.nullfs In IPv6 and NetATM, stop SIOCSIFADDR, SIOCSIFBRDADDR, SIOCSIFDSTADDR and SIOCSIFNETMASK at the socket layer rather than pass them on to the link layer without validation or credential checks. [SA-13:12] Prevent cross-mount hardlinks between different nullfs mounts of the same underlying filesystem. [SA-13:13] 20130822: p3 FreeBSD-SA-13:09.ip_multicast FreeBSD-SA-13:10.sctp Fix an integer overflow in computing the size of a temporary buffer can result in a buffer which is too small for the requested operation. [13:09] Fix a bug that could lead to kernel memory disclosure with SCTP state cookie. [13:10] 20130726: p2 FreeBSD-SA-13:07.bind Fix Denial of Service vulnerability in named(8). 20130628: p1 FreeBSD-EN-13:01.fxp FreeBSD-EN-13:02.vtnet Fix a problem where dhclient(8) utility tries to initilaize an fxp(4) forever because the driver resets the controller chip twice upon initialization. [EN-13:01] Fix a problem where frames sent to additional MAC addresses are not forwarded to the vtnet(4) interface. [EN-13:02] 20130429: Fix a bug that allows NFS clients to issue READDIR on files. 20130225: A new compression method (lz4) has been merged. Please refer to zpool-features(7) for more information. Please refer to the "ZFS notes" section of this file for information on upgrading boot ZFS pools. 20121130: A new version of ZFS (pool version 5000) has been merged to 8-STABLE. Starting with this version the old system of ZFS pool versioning is superseded by "feature flags". This concept enables forward compatibility against certain future changes in functionality of ZFS pools. The first two read-only compatible "feature flags" for ZFS pools are "com.delphix:async_destroy" and "com.delphix:empty_bpobj". For more information read the new zpool-features(7) manual page. Please refer to the "ZFS notes" section of this file for information on upgrading boot ZFS pools. 20121018: WITH_CTF can now be specified in src.conf (not recommended, there are some problems with static executables), make.conf (would also affect ports which do not use GNU make and do not override the compile targets) or in the kernel config (via "makeoptions WITH_CTF=yes"). When WITH_CTF was specified there before this was silently ignored, so make sure that WITH_CTF is not used in places which could lead to unwanted behavior. 20120913: The random(4) support for the VIA hardware random number generator (`PADLOCK') is no longer enabled unconditionally. Add the PADLOCK_RNG option in the custom kernel config if needed. The GENERIC kernels on i386 and amd64 do include the option, so the change only affects the custom kernel configurations. 20120727: The sparc64 ZFS loader has been changed to no longer try to auto- detect ZFS providers based on diskN aliases but now requires these to be explicitly listed in the OFW boot-device environment variable. 20120411: 8.3-RELEASE. 20120106: A new VOP_ADVISE() was added to support posix_fadvise(2). All filesystem modules must be recompiled. 20111116: A new VOP_ALLOCATE() was added to support posix_fallocate(2). All filesystem modules must be recompiled. 20111101: The broken amd(4) driver has been replaced with esp(4) in the amd64, i386 and pc98 GENERIC kernel configuration files. 20110606: A new version of ZFS (version 28) has been merged. This version does not depend on a python library and the sysutils/py-zfs port is not used anymore. For upgrading your boot pool, please read "ZFS notes" in the COMMON ITEMS: section later in this file. 20110428: IPsec's HMAC_SHA256-512 support has been fixed to be RFC4868 compliant, and will now use half of hash for authentication. This will break interoperability with all stacks (including all older FreeBSD versions) who implement draft-ietf-ipsec-ciph-sha-256-00 (they use 96 bits of hash for authentication). The only workaround with such peers is to use another HMAC algorithm for IPsec ("phase 2") authentication. 20110221: 8.2-RELEASE. 20101126: New version of minidump format for amd64 architecture was introduced in r215872. To analyze vmcore files produced by kernels at or after this version you will need updated userland, libkvm actually, that is able to handle the new version. 20101022: A workaround for a fixed ld bug has been removed in kernel code, so make sure that your system ld is built from sources after revision 211583 from 2010-08-21 (r210245 from 2010-07-19 if building stable/8 kernel on head, r211584 from 2010-08-21 for stable/7). A symptom of incorrect ld version is different addresses for set_pcpu section and __start_set_pcpu symbol in kernel and/or modules. 20100915: A new version of ZFS (version 15) has been merged. This version uses a python library for the following subcommands: zfs allow, zfs unallow, zfs groupspace, zfs userspace. For full functionality of these commands the following port must be installed: sysutils/py-zfs 20100720: 8.1-RELEASE. 20100502: The config(8) command has been updated to maintain compatibility with config files from 8.0-RELEASE. You will need a new version of config to build kernels (this version can be used from 8.0-RELEASE forward). The buildworld target will generate it, so following the instructions in this file for updating will work glitch-free. Merely doing a make buildkernel without first doing a make buildworld (or kernel-toolchain), or attempting to build a kernel using traidtional methods will generate a config version warning, indicating you should update. 20100408: The rc.firewall and rc.firewall6 were unified, and rc.firewall6 and rc.d/ip6fw were removed. According to the removal of rc.d/ip6fw, ipv6_firewall_* rc variables are obsoleted. Instead, the following new rc variables are added to rc.d/ipfw: firewall_client_net_ipv6, firewall_simple_iif_ipv6, firewall_simple_inet_ipv6, firewall_simple_oif_ipv6, firewall_simple_onet_ipv6 The meanings correspond to the relevant IPv4 variables. 20100417: COMPAT_IA32 has been added as an alias for COMPAT_FREEBSD32. A new version of config(8) is required. The error message when you hit this condition is confusing (COMPAT_FREEBSD32 duplicate option), when it should really say "your config is too old to compile this new kernel." 20100406: The kernel option COMPAT_IA32 has been replaced with COMPAT_FREEBSD32 to allow 32-bit compatibility on non-x86 platforms. All kernel configurations on amd64 and ia64 platforms using these options must be modified accordingly. 20100125: Introduce the kernel thread "deadlock resolver" (which can be enabled via the DEADLKRES option, see NOTES for more details) and the sleepq_type() function for sleepqueues. 20091125: 8.0-RELEASE. 20090929: 802.11s D3.03 support was committed. This is incompatible with the previous code, which was based on D3.0. 20090915: ZFS snapshots are now mounted with MNT_IGNORE flag. Use -v option for mount(8) and -a option for df(1) to see them. 20090813: Remove the option STOP_NMI. The default action is now to use NMI only for KDB via the newly introduced function stop_cpus_hard() and maintain stop_cpus() to just use a normal IPI_STOP on ia32 and amd64. 20090803: RELENG_8 branched. 20090719: Bump the shared library version numbers for all libraries that do not use symbol versioning as part of the 8.0-RELEASE cycle. Bump __FreeBSD_version to 800105. 20090714: Due to changes in the implementation of virtual network stack support, all network-related kernel modules must be recompiled. As this change breaks the ABI, bump __FreeBSD_version to 800104. 20090713: The TOE interface to the TCP syncache has been modified to remove struct tcpopt () from the ABI of the network stack. The cxgb driver is the only TOE consumer affected by this change, and needs to be recompiled along with the kernel. As this change breaks the ABI, bump __FreeBSD_version to 800103. 20090712: Padding has been added to struct tcpcb, sackhint and tcpstat in to facilitate future MFCs and bug fixes whilst maintainig the ABI. However, this change breaks the ABI, so bump __FreeBSD_version to 800102. User space tools that rely on the size of any of these structs (e.g. sockstat) need to be recompiled. 20090630: The NFS_LEGACYRPC option has been removed along with the old kernel RPC implementation that this option selected. Kernel configurations may need to be adjusted. 20090629: The network interface device nodes at /dev/net/ have been removed. All ioctl operations can be performed the normal way using routing sockets. The kqueue functionality can generally be replaced with routing sockets. 20090628: The documentation from the FreeBSD Documentation Project (Handbook, FAQ, etc.) is now installed via packages by sysinstall(8) and under the /usr/local/share/doc/freebsd directory instead of /usr/share/doc. 20090624: The ABI of various structures related to the SYSV IPC API have been changed. As a result, the COMPAT_FREEBSD[456] and COMPAT_43 kernel options now all require COMPAT_FREEBSD7. Bump __FreeBSD_version to 800100. 20090622: Layout of struct vnet has changed as routing related variables were moved to their own Vimage module. Modules need to be recompiled. Bump __FreeBSD_version to 800099. 20090619: NGROUPS_MAX and NGROUPS have been increased from 16 to 1023 and 1024 respectively. As long as no more than 16 groups per process are used, no changes should be visible. When more than 16 groups are used, old binaries may fail if they call getgroups() or getgrouplist() with statically sized storage. Recompiling will work around this, but applications should be modified to use dynamically allocated storage for group arrays as POSIX.1-2008 does not cap an implementation's number of supported groups at NGROUPS_MAX+1 as previous versions did. NFS and portalfs mounts may also be affected as the list of groups is truncated to 16. Users of NFS who use more than 16 groups, should take care that negative group permissions are not used on the exported file systems as they will not be reliable unless a GSSAPI based authentication method is used. 20090616: The compiling option ADAPTIVE_LOCKMGRS has been introduced. This option compiles in the support for adaptive spinning for lockmgrs which want to enable it. The lockinit() function now accepts the flag LK_ADAPTIVE in order to make the lock object subject to adaptive spinning when both held in write and read mode. 20090613: The layout of the structure returned by IEEE80211_IOC_STA_INFO has changed. User applications that use this ioctl need to be rebuilt. 20090611: The layout of struct thread has changed. Kernel and modules need to be rebuilt. 20090608: The layout of structs ifnet, domain, protosw and vnet_net has changed. Kernel modules need to be rebuilt. Bump __FreeBSD_version to 800097. 20090602: window(1) has been removed from the base system. It can now be installed from ports. The port is called misc/window. 20090601: The way we are storing and accessing `routing table' entries has changed. Programs reading the FIB, like netstat, need to be re-compiled. 20090601: A new netisr implementation has been added for FreeBSD 8. Network file system modules, such as igmp, ipdivert, and others, should be rebuilt. Bump __FreeBSD_version to 800096. 20090530: Remove the tunable/sysctl debug.mpsafevfs as its initial purpose is no more valid. 20090530: Add VOP_ACCESSX(9). File system modules need to be rebuilt. Bump __FreeBSD_version to 800094. 20090529: Add mnt_xflag field to 'struct mount'. File system modules need to be rebuilt. Bump __FreeBSD_version to 800093. 20090528: The compiling option ADAPTIVE_SX has been retired while it has been introduced the option NO_ADAPTIVE_SX which handles the reversed logic. The KPI for sx_init_flags() changes as accepting flags: SX_ADAPTIVESPIN flag has been retired while the SX_NOADAPTIVE flag has been introduced in order to handle the reversed logic. Bump __FreeBSD_version to 800092. 20090527: Add support for hierarchical jails. Remove global securelevel. Bump __FreeBSD_version to 800091. 20090523: The layout of struct vnet_net has changed, therefore modules need to be rebuilt. Bump __FreeBSD_version to 800090. 20090523: The newly imported zic(8) produces a new format in the output. Please run tzsetup(8) to install the newly created data to /etc/localtime. 20090520: The sysctl tree for the usb stack has renamed from hw.usb2.* to hw.usb.* and is now consistent again with previous releases. 20090520: 802.11 monitor mode support was revised and driver api's were changed. Drivers dependent on net80211 now support DLT_IEEE802_11_RADIO instead of DLT_IEEE802_11. No user-visible data structures were changed but applications that use DLT_IEEE802_11 may require changes. Bump __FreeBSD_version to 800088. 20090430: The layout of the following structs has changed: sysctl_oid, socket, ifnet, inpcbinfo, tcpcb, syncache_head, vnet_inet, vnet_inet6 and vnet_ipfw. Most modules need to be rebuild or panics may be experienced. World rebuild is required for correctly checking networking state from userland. Bump __FreeBSD_version to 800085. 20090429: MLDv2 and Source-Specific Multicast (SSM) have been merged to the IPv6 stack. VIMAGE hooks are in but not yet used. The implementation of SSM within FreeBSD's IPv6 stack closely follows the IPv4 implementation. For kernel developers: * The most important changes are that the ip6_output() and ip6_input() paths no longer take the IN6_MULTI_LOCK, and this lock has been downgraded to a non-recursive mutex. * As with the changes to the IPv4 stack to support SSM, filtering of inbound multicast traffic must now be performed by transport protocols within the IPv6 stack. This does not apply to TCP and SCTP, however, it does apply to UDP in IPv6 and raw IPv6. * The KPIs used by IPv6 multicast are similar to those used by the IPv4 stack, with the following differences: * im6o_mc_filter() is analogous to imo_multicast_filter(). * The legacy KAME entry points in6_joingroup and in6_leavegroup() are shimmed to in6_mc_join() and in6_mc_leave() respectively. * IN6_LOOKUP_MULTI() has been deprecated and removed. * IPv6 relies on MLD for the DAD mechanism. KAME's internal KPIs for MLDv1 have an additional 'timer' argument which is used to jitter the initial membership report for the solicited-node multicast membership on-link. * This is not strictly needed for MLDv2, which already jitters its report transmissions. However, the 'timer' argument is preserved in case MLDv1 is active on the interface. * The KAME linked-list based IPv6 membership implementation has been refactored to use a vector similar to that used by the IPv4 stack. Code which maintains a list of its own multicast memberships internally, e.g. carp, has been updated to reflect the new semantics. * There is a known Lock Order Reversal (LOR) due to in6_setscope() acquiring the IF_AFDATA_LOCK and being called within ip6_output(). Whilst MLDv2 tries to avoid this otherwise benign LOR, it is an implementation constraint which needs to be addressed in HEAD. For application developers: * The changes are broadly similar to those made for the IPv4 stack. * The use of IPv4 and IPv6 multicast socket options on the same socket, using mapped addresses, HAS NOT been tested or supported. * There are a number of issues with the implementation of various IPv6 multicast APIs which need to be resolved in the API surface before the implementation is fully compatible with KAME userland use, and these are mostly to do with interface index treatment. * The literature available discusses the use of either the delta / ASM API with setsockopt(2)/getsockopt(2), or the full-state / ASM API using setsourcefilter(3)/getsourcefilter(3). For more information please refer to RFC 3768, 'Socket Interface Extensions for Multicast Source Filters'. * Applications which use the published RFC 3678 APIs should be fine. For systems administrators: * The mtest(8) utility has been refactored to support IPv6, in addition to IPv4. Interface addresses are no longer accepted as arguments, their names must be used instead. The utility will map the interface name to its first IPv4 address as returned by getifaddrs(3). * The ifmcstat(8) utility has also been updated to print the MLDv2 endpoint state and source filter lists via sysctl(3). * The net.inet6.ip6.mcast.loop sysctl may be tuned to 0 to disable loopback of IPv6 multicast datagrams by default; it defaults to 1 to preserve the existing behaviour. Disabling multicast loopback is recommended for optimal system performance. * The IPv6 MROUTING code has been changed to examine this sysctl instead of attempting to perform a group lookup before looping back forwarded datagrams. Bump __FreeBSD_version to 800084. 20090422: Implement low-level Bluetooth HCI API. Bump __FreeBSD_version to 800083. 20090419: The layout of struct malloc_type, used by modules to register new memory allocation types, has changed. Most modules will need to be rebuilt or panics may be experienced. Bump __FreeBSD_version to 800081. 20090415: Anticipate overflowing inp_flags - add inp_flags2. This changes most offsets in inpcb, so checking v4 connection state will require a world rebuild. Bump __FreeBSD_version to 800080. 20090415: Add an llentry to struct route and struct route_in6. Modules embedding a struct route will need to be recompiled. Bump __FreeBSD_version to 800079. 20090414: The size of rt_metrics_lite and by extension rtentry has changed. Networking administration apps will need to be recompiled. The route command now supports show as an alias for get, weighting of routes, sticky and nostick flags to alter the behavior of stateful load balancing. Bump __FreeBSD_version to 800078. 20090408: Do not use Giant for kbdmux(4) locking. This is wrong and apparently causing more problems than it solves. This will re-open the issue where interrupt handlers may race with kbdmux(4) in polling mode. Typical symptoms include (but not limited to) duplicated and/or missing characters when low level console functions (such as gets) are used while interrupts are enabled (for example geli password prompt, mountroot prompt etc.). Disabling kbdmux(4) may help. 20090407: The size of structs vnet_net, vnet_inet and vnet_ipfw has changed; kernel modules referencing any of the above need to be recompiled. Bump __FreeBSD_version to 800075. 20090320: GEOM_PART has become the default partition slicer for storage devices, replacing GEOM_MBR, GEOM_BSD, GEOM_PC98 and GEOM_GPT slicers. It introduces some changes: MSDOS/EBR: the devices created from MSDOS extended partition entries (EBR) can be named differently than with GEOM_MBR and are now symlinks to devices with offset-based names. fstabs may need to be modified. BSD: the "geometry does not match label" warning is harmless in most cases but it points to problems in file system misalignment with disk geometry. The "c" partition is now implicit, covers the whole top-level drive and cannot be (mis)used by users. General: Kernel dumps are now not allowed to be written to devices whose partition types indicate they are meant to be used for file systems (or, in case of MSDOS partitions, as something else than the "386BSD" type). Most of these changes date approximately from 200812. 20090319: The uscanner(4) driver has been removed from the kernel. This follows Linux removing theirs in 2.6 and making libusb the default interface (supported by sane). 20090319: The multicast forwarding code has been cleaned up. netstat(1) only relies on KVM now for printing bandwidth upcall meters. The IPv4 and IPv6 modules are split into ip_mroute_mod and ip6_mroute_mod respectively. The config(5) options for statically compiling this code remain the same, i.e. 'options MROUTING'. 20090315: Support for the IFF_NEEDSGIANT network interface flag has been removed, which means that non-MPSAFE network device drivers are no longer supported. In particular, if_ar, if_sr, and network device drivers from the old (legacy) USB stack can no longer be built or used. 20090313: POSIX.1 Native Language Support (NLS) has been enabled in libc and a bunch of new language catalog files have also been added. This means that some common libc messages are now localized and they depend on the LC_MESSAGES environmental variable. 20090313: The k8temp(4) driver has been renamed to amdtemp(4) since support for K10 and K11 CPU families was added. 20090309: IGMPv3 and Source-Specific Multicast (SSM) have been merged to the IPv4 stack. VIMAGE hooks are in but not yet used. For kernel developers, the most important changes are that the ip_output() and ip_input() paths no longer take the IN_MULTI_LOCK(), and this lock has been downgraded to a non-recursive mutex. Transport protocols (UDP, Raw IP) are now responsible for filtering inbound multicast traffic according to group membership and source filters. The imo_multicast_filter() KPI exists for this purpose. Transports which do not use multicast (SCTP, TCP) already reject multicast by default. Forwarding and receive performance may improve as a mutex acquisition is no longer needed in the ip_input() low-level input path. in_addmulti() and in_delmulti() are shimmed to new KPIs which exist to support SSM in-kernel. For application developers, it is recommended that loopback of multicast datagrams be disabled for best performance, as this will still cause the lock to be taken for each looped-back datagram transmission. The net.inet.ip.mcast.loop sysctl may be tuned to 0 to disable loopback by default; it defaults to 1 to preserve the existing behaviour. For systems administrators, to obtain best performance with multicast reception and multiple groups, it is always recommended that a card with a suitably precise hash filter is used. Hash collisions will still result in the lock being taken within the transport protocol input path to check group membership. If deploying FreeBSD in an environment with IGMP snooping switches, it is recommended that the net.inet.igmp.sendlocal sysctl remain enabled; this forces 224.0.0.0/24 group membership to be announced via IGMP. The size of 'struct igmpstat' has changed; netstat needs to be recompiled to reflect this. Bump __FreeBSD_version to 800070. 20090309: libusb20.so.1 is now installed as libusb.so.1 and the ports system updated to use it. This requires a buildworld/installworld in order to update the library and dependencies (usbconfig, etc). Its advisable to rebuild all ports which uses libusb. More specific directions are given in the ports collection UPDATING file. Any /etc/libmap.conf entries for libusb are no longer required and can be removed. 20090302: A workaround is committed to allow the creation of System V shared memory segment of size > 2 GB on the 64-bit architectures. Due to a limitation of the existing ABI, the shm_segsz member of the struct shmid_ds, returned by shmctl(IPC_STAT) call is wrong for large segments. Note that limits must be explicitly raised to allow such segments to be created. 20090301: The layout of struct ifnet has changed, requiring a rebuild of all network device driver modules. 20090227: The /dev handling for the new USB stack has changed, a buildworld/installworld is required for libusb20. 20090223: The new USB2 stack has now been permanently moved in and all kernel and module names reverted to their previous values (eg, usb, ehci, ohci, ums, ...). The old usb stack can be compiled in by prefixing the name with the letter 'o', the old usb modules have been removed. Updating entry 20090216 for xorg and 20090215 for libmap may still apply. 20090217: The rc.conf(5) option if_up_delay has been renamed to defaultroute_delay to better reflect its purpose. If you have customized this setting in /etc/rc.conf you need to update it to use the new name. 20090216: xorg 7.4 wants to configure its input devices via hald which does not yet work with USB2. If the keyboard/mouse does not work in xorg then add Option "AllowEmptyInput" "off" to your ServerLayout section. This will cause X to use the configured kbd and mouse sections from your xorg.conf. 20090215: The GENERIC kernels for all architectures now default to the new USB2 stack. No kernel config options or code have been removed so if a problem arises please report it and optionally revert to the old USB stack. If you are loading USB kernel modules or have a custom kernel that includes GENERIC then ensure that usb names are also changed over, eg uftdi -> usb2_serial_ftdi. Older programs linked against the ports libusb 0.1 need to be redirected to the new stack's libusb20. /etc/libmap.conf can be used for this: # Map old usb library to new one for usb2 stack libusb-0.1.so.8 libusb20.so.1 20090209: All USB ethernet devices now attach as interfaces under the name ueN (eg. ue0). This is to provide a predictable name as vendors often change usb chipsets in a product without notice. 20090203: The ichsmb(4) driver has been changed to require SMBus slave addresses be left-justified (xxxxxxx0b) rather than right-justified. All of the other SMBus controller drivers require left-justified slave addresses, so this change makes all the drivers provide the same interface. 20090201: INET6 statistics (struct ip6stat) was updated. netstat(1) needs to be recompiled. 20090119: NTFS has been removed from GENERIC kernel on amd64 to match GENERIC on i386. Should not cause any issues since mount_ntfs(8) will load ntfs.ko module automatically when NTFS support is actually needed, unless ntfs.ko is not installed or security level prohibits loading kernel modules. If either is the case, "options NTFS" has to be added into kernel config. 20090115: TCP Appropriate Byte Counting (RFC 3465) support added to kernel. New field in struct tcpcb breaks ABI, so bump __FreeBSD_version to 800061. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20081225: ng_tty(4) module updated to match the new TTY subsystem. Due to API change, user-level applications must be updated. New API support added to mpd5 CVS and expected to be present in next mpd5.3 release. 20081219: With __FreeBSD_version 800060 the makefs tool is part of the base system (it was a port). 20081216: The afdata and ifnet locks have been changed from mutexes to rwlocks, network modules will need to be re-compiled. 20081214: __FreeBSD_version 800059 incorporates the new arp-v2 rewrite. RTF_CLONING, RTF_LLINFO and RTF_WASCLONED flags are eliminated. The new code reduced struct rtentry{} by 16 bytes on 32-bit architecture and 40 bytes on 64-bit architecture. The userland applications "arp" and "ndp" have been updated accordingly. The output from "netstat -r" shows only routing entries and none of the L2 information. 20081130: __FreeBSD_version 800057 marks the switchover from the binary ath hal to source code. Users must add the line: options AH_SUPPORT_AR5416 to their kernel config files when specifying: device ath_hal The ath_hal module no longer exists; the code is now compiled together with the driver in the ath module. It is now possible to tailor chip support (i.e. reduce the set of chips and thereby the code size); consult ath_hal(4) for details. 20081121: __FreeBSD_version 800054 adds memory barriers to , new interfaces to ifnet to facilitate multiple hardware transmit queues for cards that support them, and a lock-less ring-buffer implementation to enable drivers to more efficiently manage queueing of packets. 20081117: A new version of ZFS (version 13) has been merged to -HEAD. This version has zpool attribute "listsnapshots" off by default, which means "zfs list" does not show snapshots, and is the same as Solaris behavior. 20081028: dummynet(4) ABI has changed. ipfw(8) needs to be recompiled. 20081009: The uhci, ohci, ehci and slhci USB Host controller drivers have been put into separate modules. If you load the usb module separately through loader.conf you will need to load the appropriate *hci module as well. E.g. for a UHCI-based USB 2.0 controller add the following to loader.conf: uhci_load="YES" ehci_load="YES" 20081009: The ABI used by the PMC toolset has changed. Please keep userland (libpmc(3)) and the kernel module (hwpmc(4)) in sync. 20081009: atapci kernel module now includes only generic PCI ATA driver. AHCI driver moved to ataahci kernel module. All vendor-specific code moved into separate kernel modules: ataacard, ataacerlabs, ataadaptec, ataamd, ataati, atacenatek, atacypress, atacyrix, atahighpoint, ataintel, ataite, atajmicron, atamarvell, atamicron, atanational, atanetcell, atanvidia, atapromise, ataserverworks, atasiliconimage, atasis, atavia 20080820: The TTY subsystem of the kernel has been replaced by a new implementation, which provides better scalability and an improved driver model. Most common drivers have been migrated to the new TTY subsystem, while others have not. The following drivers have not yet been ported to the new TTY layer: PCI/ISA: cy, digi, rc, rp, sio USB: ubser, ucycom Line disciplines: ng_h4, ng_tty, ppp, sl, snp Adding these drivers to your kernel configuration file shall cause compilation to fail. 20080818: ntpd has been upgraded to 4.2.4p5. 20080801: OpenSSH has been upgraded to 5.1p1. For many years, FreeBSD's version of OpenSSH preferred DSA over RSA for host and user authentication keys. With this upgrade, we've switched to the vendor's default of RSA over DSA. This may cause upgraded clients to warn about unknown host keys even for previously known hosts. Users should follow the usual procedure for verifying host keys before accepting the RSA key. This can be circumvented by setting the "HostKeyAlgorithms" option to "ssh-dss,ssh-rsa" in ~/.ssh/config or on the ssh command line. Please note that the sequence of keys offered for authentication has been changed as well. You may want to specify IdentityFile in a different order to revert this behavior. 20080713: The sio(4) driver has been removed from the i386 and amd64 kernel configuration files. This means uart(4) is now the default serial port driver on those platforms as well. To prevent collisions with the sio(4) driver, the uart(4) driver uses different names for its device nodes. This means the onboard serial port will now most likely be called "ttyu0" instead of "ttyd0". You may need to reconfigure applications to use the new device names. When using the serial port as a boot console, be sure to update /boot/device.hints and /etc/ttys before booting the new kernel. If you forget to do so, you can still manually specify the hints at the loader prompt: set hint.uart.0.at="isa" set hint.uart.0.port="0x3F8" set hint.uart.0.flags="0x10" set hint.uart.0.irq="4" boot -s 20080609: The gpt(8) utility has been removed. Use gpart(8) to partition disks instead. 20080603: The version that Linuxulator emulates was changed from 2.4.2 to 2.6.16. If you experience any problems with Linux binaries please try to set sysctl compat.linux.osrelease to 2.4.2 and if it fixes the problem contact emulation mailing list. 20080525: ISDN4BSD (I4B) was removed from the src tree. You may need to update a your kernel configuration and remove relevant entries. 20080509: I have checked in code to support multiple routing tables. See the man pages setfib(1) and setfib(2). This is a hopefully backwards compatible version, but to make use of it you need to compile your kernel with options ROUTETABLES=2 (or more up to 16). 20080420: The 802.11 wireless support was redone to enable multi-bss operation on devices that are capable. The underlying device is no longer used directly but instead wlanX devices are cloned with ifconfig. This requires changes to rc.conf files. For example, change: ifconfig_ath0="WPA DHCP" to wlans_ath0=wlan0 ifconfig_wlan0="WPA DHCP" see rc.conf(5) for more details. In addition, mergemaster of /etc/rc.d is highly recommended. Simultaneous update of userland and kernel wouldn't hurt either. As part of the multi-bss changes the wlan_scan_ap and wlan_scan_sta modules were merged into the base wlan module. All references to these modules (e.g. in kernel config files) must be removed. 20080408: psm(4) has gained write(2) support in native operation level. Arbitrary commands can be written to /dev/psm%d and status can be read back from it. Therefore, an application is responsible for status validation and error recovery. It is a no-op in other operation levels. 20080312: Support for KSE threading has been removed from the kernel. To run legacy applications linked against KSE libmap.conf may be used. The following libmap.conf may be used to ensure compatibility with any prior release: libpthread.so.1 libthr.so.1 libpthread.so.2 libthr.so.2 libkse.so.3 libthr.so.3 20080301: The layout of struct vmspace has changed. This affects libkvm and any executables that link against libkvm and use the kvm_getprocs() function. In particular, but not exclusively, it affects ps(1), fstat(1), pkill(1), systat(1), top(1) and w(1). The effects are minimal, but it's advisable to upgrade world nonetheless. 20080229: The latest em driver no longer has support in it for the 82575 adapter, this is now moved to the igb driver. The split was done to make new features that are incompatible with older hardware easier to do. 20080220: The new geom_lvm(4) geom class has been renamed to geom_linux_lvm(4), likewise the kernel option is now GEOM_LINUX_LVM. 20080211: The default NFS mount mode has changed from UDP to TCP for increased reliability. If you rely on (insecurely) NFS mounting across a firewall you may need to update your firewall rules. 20080208: Belatedly note the addition of m_collapse for compacting mbuf chains. 20080126: The fts(3) structures have been changed to use adequate integer types for their members and so to be able to cope with huge file trees. The old fts(3) ABI is preserved through symbol versioning in libc, so third-party binaries using fts(3) should still work, although they will not take advantage of the extended types. At the same time, some third-party software might fail to build after this change due to unportable assumptions made in its source code about fts(3) structure members. Such software should be fixed by its vendor or, in the worst case, in the ports tree. FreeBSD_version 800015 marks this change for the unlikely case that a portable fix is impossible. 20080123: To upgrade to -current after this date, you must be running FreeBSD not older than 6.0-RELEASE. Upgrading to -current from 5.x now requires a stop over at RELENG_6 or RELENG_7 systems. 20071128: The ADAPTIVE_GIANT kernel option has been retired because its functionality is the default now. 20071118: The AT keyboard emulation of sunkbd(4) has been turned on by default. In order to make the special symbols of the Sun keyboards driven by sunkbd(4) work under X these now have to be configured the same way as Sun USB keyboards driven by ukbd(4) (which also does AT keyboard emulation), f.e.: Option "XkbLayout" "us" Option "XkbRules" "xorg" Option "XkbSymbols" "pc(pc105)+sun_vndr/usb(sun_usb)+us" 20071024: It has been decided that it is desirable to provide ABI backwards compatibility to the FreeBSD 4/5/6 versions of the PCIOCGETCONF, PCIOCREAD and PCIOCWRITE IOCTLs, which was broken with the introduction of PCI domain support (see the 20070930 entry). Unfortunately, this required the ABI of PCIOCGETCONF to be broken again in order to be able to provide backwards compatibility to the old version of that IOCTL. Thus consumers of PCIOCGETCONF have to be recompiled again. As for prominent ports this affects neither pciutils nor xorg-server this time, the hal port needs to be rebuilt however. 20071020: The misnamed kthread_create() and friends have been renamed to kproc_create() etc. Many of the callers already used kproc_start().. I will return kthread_create() and friends in a while with implementations that actually create threads, not procs. Renaming corresponds with version 800002. 20071010: RELENG_7 branched. 20071009: Setting WITHOUT_LIBPTHREAD now means WITHOUT_LIBKSE and WITHOUT_LIBTHR are set. 20070930: The PCI code has been made aware of PCI domains. This means that the location strings as used by pciconf(8) etc are now in the following format: pci::[:]. It also means that consumers of potentially need to be recompiled; this includes the hal and xorg-server ports. 20070928: The caching daemon (cached) was renamed to nscd. nscd.conf configuration file should be used instead of cached.conf and nscd_enable, nscd_pidfile and nscd_flags options should be used instead of cached_enable, cached_pidfile and cached_flags in rc.conf. 20070921: The getfacl(1) utility now prints owning user and group name instead of owning uid and gid in the three line comment header. This is the same behavior as getfacl(1) on Solaris and Linux. 20070704: The new IPsec code is now compiled in using the IPSEC option. The IPSEC option now requires "device crypto" be defined in your kernel configuration. The FAST_IPSEC kernel option is now deprecated. 20070702: The packet filter (pf) code has been updated to OpenBSD 4.1 Please note the changed syntax - keep state is now on by default. Also note the fact that ftp-proxy(8) has been changed from bottom up and has been moved from libexec to usr/sbin. Changes in the ALTQ handling also affect users of IPFW's ALTQ capabilities. 20070701: Remove KAME IPsec in favor of FAST_IPSEC, which is now the only IPsec supported by FreeBSD. The new IPsec stack supports both IPv4 and IPv6. The kernel option will change after the code changes have settled in. For now the kernel option IPSEC is deprecated and FAST_IPSEC is the only option, that will change after some settling time. 20070701: The wicontrol(8) utility has been removed from the base system. wi(4) cards should be configured using ifconfig(8), see the man page for more information. 20070612: The i386/amd64 GENERIC kernel now defaults to the nfe(4) driver instead of the nve(4) driver. Please update your configuration accordingly. 20070612: By default, /etc/rc.d/sendmail no longer rebuilds the aliases database if it is missing or older than the aliases file. If desired, set the new rc.conf option sendmail_rebuild_aliases to "YES" to restore that functionality. 20070612: The IPv4 multicast socket code has been considerably modified, and moved to the file sys/netinet/in_mcast.c. Initial support for the RFC 3678 Source-Specific Multicast Socket API has been added to the IPv4 network stack. Strict multicast and broadcast reception is now the default for UDP/IPv4 sockets; the net.inet.udp.strict_mcast_mship sysctl variable has now been removed. The RFC 1724 hack for interface selection has been removed; the use of the Linux-derived ip_mreqn structure with IP_MULTICAST_IF has been added to replace it. Consumers such as routed will soon be updated to reflect this. These changes affect users who are running routed(8) or rdisc(8) from the FreeBSD base system on point-to-point or unnumbered interfaces. 20070610: The net80211 layer has changed significantly and all wireless drivers that depend on it need to be recompiled. Further these changes require that any program that interacts with the wireless support in the kernel be recompiled; this includes: ifconfig, wpa_supplicant, hostapd, and wlanstats. Users must also, for the moment, kldload the wlan_scan_sta and/or wlan_scan_ap modules if they use modules for wireless support. These modules implement scanning support for station and ap modes, respectively. Failure to load the appropriate module before marking a wireless interface up will result in a message to the console and the device not operating properly. 20070610: The pam_nologin(8) module ceases to provide an authentication function and starts providing an account management function. Consequent changes to /etc/pam.d should be brought in using mergemaster(8). Third-party files in /usr/local/etc/pam.d may need manual editing as follows. Locate this line (or similar): auth required pam_nologin.so no_warn and change it according to this example: account required pam_nologin.so no_warn That is, the first word needs to be changed from "auth" to "account". The new line can be moved to the account section within the file for clarity. Not updating pam.conf(5) files will result in nologin(5) ignored by the respective services. 20070529: The ether_ioctl() function has been synchronized with ioctl(2) and ifnet.if_ioctl. Due to that, the size of one of its arguments has changed on 64-bit architectures. All kernel modules using ether_ioctl() need to be rebuilt on such architectures. 20070516: Improved INCLUDE_CONFIG_FILE support has been introduced to the config(8) utility. In order to take advantage of this new functionality, you are expected to recompile and install src/usr.sbin/config. If you don't rebuild config(8), and your kernel configuration depends on INCLUDE_CONFIG_FILE, the kernel build will be broken because of a missing "kernconfstring" symbol. 20070513: Symbol versioning is enabled by default. To disable it, use option WITHOUT_SYMVER. It is not advisable to attempt to disable symbol versioning once it is enabled; your installworld will break because a symbol version-less libc will get installed before the install tools. As a result, the old install tools, which previously had symbol dependencies to FBSD_1.0, will fail because the freshly installed libc will not have them. The default threading library (providing "libpthread") has been changed to libthr. If you wish to have libkse as your default, use option DEFAULT_THREAD_LIB=libkse for the buildworld. 20070423: The ABI breakage in sendmail(8)'s libmilter has been repaired so it is no longer necessary to recompile mail filters (aka, milters). If you recompiled mail filters after the 20070408 note, it is not necessary to recompile them again. 20070417: The new trunk(4) driver has been renamed to lagg(4) as it better reflects its purpose. ifconfig will need to be recompiled. 20070408: sendmail(8) has been updated to version 8.14.1. Mail filters (aka, milters) compiled against the libmilter included in the base operating system should be recompiled. 20070302: Firmwares for ipw(4) and iwi(4) are now included in the base tree. In order to use them one must agree to the respective LICENSE in share/doc/legal and define legal.intel_.license_ack=1 via loader.conf(5) or kenv(1). Make sure to deinstall the now deprecated modules from the respective firmware ports. 20070228: The name resolution/mapping functions addr2ascii(3) and ascii2addr(3) were removed from FreeBSD's libc. These originally came from INRIA IPv6. Nothing in FreeBSD ever used them. They may be regarded as deprecated in previous releases. The AF_LINK support for getnameinfo(3) was merged from NetBSD to replace it as a more portable (and re-entrant) API. 20070224: To support interrupt filtering a modification to the newbus API has occurred, ABI was broken and __FreeBSD_version was bumped to 700031. Please make sure that your kernel and modules are in sync. For more info: http://docs.freebsd.org/cgi/mid.cgi?20070221233124.GA13941 20070224: The IPv6 multicast forwarding code may now be loaded into GENERIC kernels by loading the ip_mroute.ko module. This is built into the module unless WITHOUT_INET6 or WITHOUT_INET6_SUPPORT options are set; see src.conf(5) for more information. 20070214: The output of netstat -r has changed. Without -n, we now only print a "network name" without the prefix length if the network address and mask exactly match a Class A/B/C network, and an entry exists in the nsswitch "networks" map. With -n, we print the full unabbreviated CIDR network prefix in the form "a.b.c.d/p". 0.0.0.0/0 is always printed as "default". This change is in preparation for changes such as equal-cost multipath, and to more generally assist operational deployment of FreeBSD as a modern IPv4 router. 20070210: PIM has been turned on by default in the IPv4 multicast routing code. The kernel option 'PIM' has now been removed. PIM is now built by default if option 'MROUTING' is specified. It may now be loaded into GENERIC kernels by loading the ip_mroute.ko module. 20070207: Support for IPIP tunnels (VIFF_TUNNEL) in IPv4 multicast routing has been removed. Its functionality may be achieved by explicitly configuring gif(4) interfaces and using the 'phyint' keyword in mrouted.conf. XORP does not support source-routed IPv4 multicast tunnels nor the integrated IPIP tunneling, therefore it is not affected by this change. The __FreeBSD_version macro has been bumped to 700030. 20061221: Support for PCI Message Signalled Interrupts has been re-enabled in the bge driver, only for those chips which are believed to support it properly. If there are any problems, MSI can be disabled completely by setting the 'hw.pci.enable_msi' and 'hw.pci.enable_msix' tunables to 0 in the loader. 20061214: Support for PCI Message Signalled Interrupts has been disabled again in the bge driver. Many revisions of the hardware fail to support it properly. Support can be re-enabled by removing the #define of BGE_DISABLE_MSI in "src/sys/dev/bge/if_bge.c". 20061214: Support for PCI Message Signalled Interrupts has been added to the bge driver. If there are any problems, MSI can be disabled completely by setting the 'hw.pci.enable_msi' and 'hw.pci.enable_msix' tunables to 0 in the loader. 20061205: The removal of several facets of the experimental Threading system from the kernel means that the proc and thread structures have changed quite a bit. I suggest all kernel modules that might reference these structures be recompiled.. Especially the linux module. 20061126: Sound infrastructure has been updated with various fixes and improvements. Most of the changes are pretty much transparent, with exceptions of followings: 1) All sound driver specific sysctls (hw.snd.pcm%d.*) have been moved to their own dev sysctl nodes, for example: hw.snd.pcm0.vchans -> dev.pcm.0.vchans 2) /dev/dspr%d.%d has been deprecated. Each channel now has its own chardev in the form of "dsp%d.%d", where is p = playback, r = record and v = virtual, respectively. Users are encouraged to use these devs instead of (old) "/dev/dsp%d.%d". This does not affect those who are using "/dev/dsp". 20061122: geom(4)'s gmirror(8) class metadata structure has been rev'd from v3 to v4. If you update across this point and your metadata is converted for you, you will not be easily able to downgrade since the /boot/kernel.old/geom_mirror.ko kernel module will be unable to read the v4 metadata. You can resolve this by doing from the loader(8) prompt: set vfs.root.mountfrom="ufs:/dev/XXX" where XXX is the root slice of one of the disks that composed the mirror (i.e.: /dev/ad0s1a). You can then rebuild the array the same way you built it originally. 20061122: The following binaries have been disconnected from the build: mount_devfs, mount_ext2fs, mount_fdescfs, mount_procfs, mount_linprocfs, and mount_std. The functionality of these programs has been moved into the mount program. For example, to mount a devfs filesystem, instead of using mount_devfs, use: "mount -t devfs". This does not affect entries in /etc/fstab, since entries in /etc/fstab are always processed with "mount -t fstype". 20061113: Support for PCI Message Signalled Interrupts on i386 and amd64 has been added to the kernel and various drivers will soon be updated to use MSI when it is available. If there are any problems, MSI can be disabled completely by setting the 'hw.pci.enable_msi' and 'hw.pci.enable_msix' tunables to 0 in the loader. 20061110: The MUTEX_PROFILING option has been renamed to LOCK_PROFILING. The lockmgr object layout has been changed as a result of having a lock_object embedded in it. As a consequence all file system kernel modules must be re-compiled. The mutex profiling man page has not yet been updated to reflect this change. 20061026: KSE in the kernel has now been made optional and turned on by default. Use 'nooption KSE' in your kernel config to turn it off. All kernel modules *must* be recompiled after this change. There-after, modules from a KSE kernel should be compatible with modules from a NOKSE kernel due to the temporary padding fields added to 'struct proc'. 20060929: mrouted and its utilities have been removed from the base system. 20060927: Some ioctl(2) command codes have changed. Full backward ABI compatibility is provided if the "options COMPAT_FREEBSD6" is present in the kernel configuration file. Make sure to add this option to your kernel config file, or recompile X.Org and the rest of ports; otherwise they may refuse to work. 20060924: tcpslice has been removed from the base system. 20060913: The sizes of struct tcpcb (and struct xtcpcb) have changed due to the rewrite of TCP syncookies. Tools like netstat, sockstat, and systat needs to be rebuilt. 20060903: libpcap updated to v0.9.4 and tcpdump to v3.9.4 20060816: The IPFIREWALL_FORWARD_EXTENDED option is gone and the behaviour for IPFIREWALL_FORWARD is now as it was before when it was first committed and for years after. The behaviour is now ON. 20060725: enigma(1)/crypt(1) utility has been changed on 64 bit architectures. Now it can decrypt files created from different architectures. Unfortunately, it is no longer able to decrypt a cipher text generated with an older version on 64 bit architectures. If you have such a file, you need old utility to decrypt it. 20060709: The interface version of the i4b kernel part has changed. So after updating the kernel sources and compiling a new kernel, the i4b user space tools in "/usr/src/usr.sbin/i4b" must also be rebuilt, and vice versa. 20060627: The XBOX kernel now defaults to the nfe(4) driver instead of the nve(4) driver. Please update your configuration accordingly. 20060514: The i386-only lnc(4) driver for the AMD Am7900 LANCE and Am79C9xx PCnet family of NICs has been removed. The new le(4) driver serves as an equivalent but cross-platform replacement with the pcn(4) driver still providing performance-optimized support for the subset of AMD Am79C971 PCnet-FAST and greater chips as before. 20060511: The machdep.* sysctls and the adjkerntz utility have been modified a bit. The new adjkerntz utility uses the new sysctl names and sysctlbyname() calls, so it may be impossible to run an old /sbin/adjkerntz utility in single-user mode with a new kernel. Replace the `adjkerntz -i' step before `make installworld' with: /usr/obj/usr/src/sbin/adjkerntz/adjkerntz -i and proceed as usual with the rest of the installworld-stage steps. Otherwise, you risk installing binaries with their timestamp set several hours in the future, especially if you are running with local time set to GMT+X hours. 20060412: The ip6fw utility has been removed. The behavior provided by ip6fw has been in ipfw2 for a good while and the rc.d scripts have been updated to deal with it. There are some rules that might not migrate cleanly. Use rc.firewall6 as a template to rewrite rules. 20060428: The puc(4) driver has been overhauled. The ebus(4) and sbus(4) attachments have been removed. Make sure to configure scc(4) on sparc64. Note also that by default puc(4) will use uart(4) and not sio(4) for serial ports because interrupt handling has been optimized for multi-port serial cards and only uart(4) implements the interface to support it. 20060330: The scc(4) driver replaces puc(4) for Serial Communications Controllers (SCCs) like the Siemens SAB82532 and the Zilog Z8530. On sparc64, it is advised to add scc(4) to the kernel configuration to make sure that the serial ports remain functional. 20060317: Most world/kernel related NO_* build options changed names. New knobs have common prefixes WITHOUT_*/WITH_* (modelled after FreeBSD ports) and should be set in /etc/src.conf (the src.conf(5) manpage is provided). Full backwards compatibility is maintained for the time being though it's highly recommended to start moving old options out of the system-wide /etc/make.conf file into the new /etc/src.conf while also properly renaming them. More conversions will likely follow. Posting to current@: http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html 20060305: The NETSMBCRYPTO kernel option has been retired because its functionality is always included in NETSMB and smbfs.ko now. 20060303: The TDFX_LINUX kernel option was retired and replaced by the tdfx_linux device. The latter can be loaded as the 3dfx_linux.ko kernel module. Loading it alone should suffice to get 3dfx support for Linux apps because it will pull in 3dfx.ko and linux.ko through its dependencies. 20060204: The 'audit' group was added to support the new auditing functionality in the base system. Be sure to follow the directions for updating, including the requirement to run mergemaster -p. 20060201: The kernel ABI to file system modules was changed on i386. Please make sure that your kernel and modules are in sync. 20060118: This actually occured some time ago, but installing the kernel now also installs a bunch of symbol files for the kernel modules. This increases the size of /boot/kernel to about 67Mbytes. You will need twice this if you will eventually back this up to kernel.old on your next install. If you have a shortage of room in your root partition, you should add -DINSTALL_NODEBUG to your make arguments or add INSTALL_NODEBUG="yes" to your /etc/make.conf. 20060113: libc's malloc implementation has been replaced. This change has the potential to uncover application bugs that previously went unnoticed. See the malloc(3) manual page for more details. 20060112: The generic netgraph(4) cookie has been changed. If you upgrade kernel passing this point, you also need to upgrade userland and netgraph(4) utilities like ports/net/mpd or ports/net/mpd4. 20060106: si(4)'s device files now contain the unit number. Uses of {cua,tty}A[0-9a-f] should be replaced by {cua,tty}A0[0-9a-f]. 20060106: The kernel ABI was mostly destroyed due to a change in the size of struct lock_object which is nested in other structures such as mutexes which are nested in all sorts of other structures. Make sure your kernel and modules are in sync. 20051231: The page coloring algorithm in the VM subsystem was converted from tuning with kernel options to autotuning. Please remove any PQ_* option except PQ_NOOPT from your kernel config. 20051211: The net80211-related tools in the tools/tools/ath directory have been moved to tools/tools/net80211 and renamed with a "wlan" prefix. Scripts that use them should be adjusted accordingly. 20051202: Scripts in the local_startup directories (as defined in /etc/defaults/rc.conf) that have the new rc.d semantics will now be run as part of the base system rcorder. If there are errors or problems with one of these local scripts, it could cause boot problems. If you encounter such problems, boot in single user mode, remove that script from the */rc.d directory. Please report the problem to the port's maintainer, and the freebsd-ports@freebsd.org mailing list. 20051129: The nodev mount option was deprecated in RELENG_6 (where it was a no-op), and is now unsupported. If you have nodev or dev listed in /etc/fstab, remove it, otherwise it will result in a mount error. 20051129: ABI between ipfw(4) and ipfw(8) has been changed. You need to rebuild ipfw(8) when rebuilding kernel. 20051108: rp(4)'s device files now contain the unit number. Uses of {cua,tty}R[0-9a-f] should be replaced by {cua,tty}R0[0-9a-f]. 20051029: /etc/rc.d/ppp-user has been renamed to /etc/rc.d/ppp. Its /etc/rc.conf.d configuration file has been `ppp' from the beginning, and hence there is no need to touch it. 20051014: Now most modules get their build-time options from the kernel configuration file. A few modules still have fixed options due to their non-conformant implementation, but they will be corrected eventually. You may need to review the options of the modules in use, explicitly specify the non-default options in the kernel configuration file, and rebuild the kernel and modules afterwards. 20051001: kern.polling.enable sysctl MIB is now deprecated. Use ifconfig(8) to turn polling(4) on your interfaces. 20050927: The old bridge(4) implementation was retired. The new if_bridge(4) serves as a full functional replacement. 20050722: The ai_addrlen of a struct addrinfo was changed to a socklen_t to conform to POSIX-2001. This change broke an ABI compatibility on 64 bit architecture. You have to recompile userland programs that use getaddrinfo(3) on 64 bit architecture. 20050711: RELENG_6 branched here. 20050629: The pccard_ifconfig rc.conf variable has been removed and a new variable, ifconfig_DEFAULT has been introduced. Unlike pccard_ifconfig, ifconfig_DEFAULT applies to ALL interfaces that do not have ifconfig_ifn entries rather than just those in removable_interfaces. 20050616: Some previous versions of PAM have permitted the use of non-absolute paths in /etc/pam.conf or /etc/pam.d/* when referring to third party PAM modules in /usr/local/lib. A change has been made to require the use of absolute paths in order to avoid ambiguity and dependence on library path configuration, which may affect existing configurations. 20050610: Major changes to network interface API. All drivers must be recompiled. Drivers not in the base system will need to be updated to the new APIs. 20050609: Changes were made to kinfo_proc in sys/user.h. Please recompile userland, or commands like `fstat', `pkill', `ps', `top' and `w' will not behave correctly. The API and ABI for hwpmc(4) have changed with the addition of sampling support. Please recompile lib/libpmc(3) and usr.sbin/{pmcstat,pmccontrol}. 20050606: The OpenBSD dhclient was imported in place of the ISC dhclient and the network interface configuration scripts were updated accordingly. If you use DHCP to configure your interfaces, you must now run devd. Also, DNS updating was lost so you will need to find a workaround if you use this feature. The '_dhcp' user was added to support the OpenBSD dhclient. Be sure to run mergemaster -p (like you are supposed to do every time anyway). 20050605: if_bridge was added to the tree. This has changed struct ifnet. Please recompile userland and all network related modules. 20050603: The n_net of a struct netent was changed to an uint32_t, and 1st argument of getnetbyaddr() was changed to an uint32_t, to conform to POSIX-2001. These changes broke an ABI compatibility on 64 bit architecture. With these changes, shlib major of libpcap was bumped. You have to recompile userland programs that use getnetbyaddr(3), getnetbyname(3), getnetent(3) and/or libpcap on 64 bit architecture. 20050528: Kernel parsing of extra options on '#!' first lines of shell scripts has changed. Lines with multiple options likely will fail after this date. For full details, please see http://people.freebsd.org/~gad/Updating-20050528.txt 20050503: The packet filter (pf) code has been updated to OpenBSD 3.7 Please note the changed anchor syntax and the fact that authpf(8) now needs a mounted fdescfs(5) to function. 20050415: The NO_MIXED_MODE kernel option has been removed from the i386 amd64 platforms as its use has been superceded by the new local APIC timer code. Any kernel config files containing this option should be updated. 20050227: The on-disk format of LC_CTYPE files was changed to be machine independent. Please make sure NOT to use NO_CLEAN buildworld when crossing this point. Crossing this point also requires recompile or reinstall of all locale depended packages. 20050225: The ifi_epoch member of struct if_data has been changed to contain the uptime at which the interface was created or the statistics zeroed rather then the wall clock time because wallclock time may go backwards. This should have no impact unless an snmp implementation is using this value (I know of none at this point.) 20050224: The acpi_perf and acpi_throttle drivers are now part of the acpi(4) main module. They are no longer built separately. 20050223: The layout of struct image_params has changed. You have to recompile all compatibility modules (linux, svr4, etc) for use with the new kernel. 20050223: The p4tcc driver has been merged into cpufreq(4). This makes "options CPU_ENABLE_TCC" obsolete. Please load cpufreq.ko or compile in "device cpufreq" to restore this functionality. 20050220: The responsibility of recomputing the file system summary of a SoftUpdates-enabled dirty volume has been transferred to the background fsck. A rebuild of fsck(8) utility is recommended if you have updated the kernel. To get the old behavior (recompute file system summary at mount time), you can set vfs.ffs.compute_summary_at_mount=1 before mounting the new volume. 20050206: The cpufreq import is complete. As part of this, the sysctls for acpi(4) throttling have been removed. The power_profile script has been updated, so you can use performance/economy_cpu_freq in rc.conf(5) to set AC on/offline cpu frequencies. 20050206: NG_VERSION has been increased. Recompiling kernel (or ng_socket.ko) requires recompiling libnetgraph and userland netgraph utilities. 20050114: Support for abbreviated forms of a number of ipfw options is now deprecated. Warnings are printed to stderr indicating the correct full form when a match occurs. Some abbreviations may be supported at a later date based on user feedback. To be considered for support, abbreviations must be in use prior to this commit and unlikely to be confused with current key words. 20041221: By a popular demand, a lot of NOFOO options were renamed to NO_FOO (see bsd.compat.mk for a full list). The old spellings are still supported, but will cause annoying warnings on stderr. Make sure you upgrade properly (see the COMMON ITEMS: section later in this file). 20041219: Auto-loading of ancillary wlan modules such as wlan_wep has been temporarily disabled; you need to statically configure the modules you need into your kernel or explicitly load them prior to use. Specifically, if you intend to use WEP encryption with an 802.11 device load/configure wlan_wep; if you want to use WPA with the ath driver load/configure wlan_tkip, wlan_ccmp, and wlan_xauth as required. 20041213: The behaviour of ppp(8) has changed slightly. If lqr is enabled (``enable lqr''), older versions would revert to LCP ECHO mode on negotiation failure. Now, ``enable echo'' is required for this behaviour. The ppp version number has been bumped to 3.4.2 to reflect the change. 20041201: The wlan support has been updated to split the crypto support into separate modules. For static WEP you must configure the wlan_wep module in your system or build and install the module in place where it can be loaded (the kernel will auto-load the module when a wep key is configured). 20041201: The ath driver has been updated to split the tx rate control algorithm into a separate module. You need to include either ath_rate_onoe or ath_rate_amrr when configuring the kernel. 20041116: Support for systems with an 80386 CPU has been removed. Please use FreeBSD 5.x or earlier on systems with an 80386. 20041110: We have had a hack which would mount the root filesystem R/W if the device were named 'md*'. As part of the vnode work I'm doing I have had to remove this hack. People building systems which use preloaded MD root filesystems may need to insert a "/sbin/mount -u -o rw /dev/md0 /" in their /etc/rc scripts. 20041104: FreeBSD 5.3 shipped here. 20041102: The size of struct tcpcb has changed again due to the removal of RFC1644 T/TCP. You have to recompile userland programs that read kmem for tcp sockets directly (netstat, sockstat, etc.) 20041022: The size of struct tcpcb has changed. You have to recompile userland programs that read kmem for tcp sockets directly (netstat, sockstat, etc.) 20041016: RELENG_5 branched here. For older entries, please see updating in the RELENG_5 branch. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. When upgrading from one major version to another it is generally best to upgrade to the latest code in the currently installed branch first, then do an upgrade to the new branch. This is the best-tested upgrade path, and has the highest probability of being successful. Please try this approach before reporting problems with a major version upgrade. ZFS notes --------- When upgrading the boot ZFS pool to a new version, always follow these two steps: 1.) recompile and reinstall the ZFS boot loader and boot block (this is part of "make buildworld" and "make installworld") 2.) update the ZFS boot block on your boot drive The following example updates the ZFS boot block on the first partition (freebsd-boot) of a GPT partitioned drive ad0: "gpart bootcode -p /boot/gptzfsboot -i 1 ad0" Non-boot pools do not need these updates. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. It is the most failsafe as it uses a /usr/obj tree with a fresh mini-buildworld, make kernel-toolchain make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To test a kernel once --------------------- If you just want to boot a kernel once (because you are not sure if it works, or if you want to boot a known bad kernel to provide debugging information) run make installkernel KERNCONF=YOUR_KERNEL_HERE KODIR=/boot/testkernel nextboot -k testkernel To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a 5.X system. Replace ${arch} with the architecture of your machine (e.g. "i386", "alpha", "amd64", "ia64", "pc98", "sparc64", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. make buildworld make kernel KERNCONF=YOUR_KERNEL_HERE [1] [3] mergemaster -p [5] make installworld mergemaster [4] make delete-old [6] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installworld DESTDIR=${CURRENT_ROOT} make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from 5.x-stable to current ---------------------------------------------- make buildworld [9] make kernel KERNCONF=YOUR_KERNEL_HERE [8] [1] [3] mergemaster -p [5] make installworld mergemaster -i [4] make delete-old [6] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. [5] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. You may need to install a new mergemaster (cd src/usr.sbin/mergemaster && make install) after the buildworld before this step if you last updated from current before 20020224 or from -stable before 20020408. [6] This only deletes old files and directories. Old libraries can be deleted by "make delete-old-libs", but you have to make sure that no program is using those libraries anymore. [8] In order to have a kernel that can run the 4.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD4 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. A similar kernel option COMPAT_FREEBSD5 is required to run the 5.x binaries on more recent kernels. Make sure that you merge any new devices from GENERIC since the last time you updated your kernel config file. [9] When checking out sources, you must include the -P flag to have cvs prune empty directories. If CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. MAKEOBJDIRPREFIX must be defined in an environment variable, and not on the command line, or in /etc/make.conf. buildworld will warn if it is improperly defined. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -current. It is not guaranteed to be a complete list of such breakages, and only contains entries since October 16, 2004. If you need to see UPDATING entries from before that date, you will need to fetch an UPDATING file from an older FreeBSD release. Copyright information: Copyright 1998-2005 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. If you find this document useful, and you want to, you may buy the author a beer. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: releng/8.4/sys/conf/newvers.sh =================================================================== --- releng/8.4/sys/conf/newvers.sh (revision 282873) +++ releng/8.4/sys/conf/newvers.sh (revision 282874) @@ -1,152 +1,152 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="8.4" -BRANCH="RELEASE-p27" +BRANCH="RELEASE-p28" if [ "X${BRANCH_OVERRIDE}" != "X" ]; then BRANCH=${BRANCH_OVERRIDE} fi RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" SYSDIR=$(dirname $0)/.. if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${SYSDIR}/sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/1992-$year/" \ -e 's/\[your name here\]\.* /The FreeBSD Project./' \ -e 's/\[your name\]\.*/The FreeBSD Project./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/*- * Copyright (c) 1992-$year The FreeBSD Project. * All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} t=`date` i=`${MAKE:-make} -V KERN_IDENT` compiler_v=$($(${MAKE:-make} -V CC) -v 2>&1 | grep 'version') for dir in /bin /usr/bin /usr/local/bin; do if [ -x "${dir}/svnversion" ] ; then svnversion=${dir}/svnversion break fi done if [ -d "${SYSDIR}/../.git" ] ; then for dir in /bin /usr/bin /usr/local/bin; do if [ -x "${dir}/git" ] ; then git_cmd="${dir}/git --git-dir=${SYSDIR}/../.git" break fi done fi if [ -n "$svnversion" ] ; then echo "$svnversion" svn=`cd ${SYSDIR} && $svnversion` case "$svn" in [0-9]*) svn=" r${svn}" ;; *) unset svn ;; esac fi if [ -n "$git_cmd" ] ; then git=`$git_cmd rev-parse --verify --short HEAD 2>/dev/null` svn=`$git_cmd svn find-rev $git 2>/dev/null` if [ -n "$svn" ] ; then svn=" r${svn}" git="=${git}" else svn=`$git_cmd log | fgrep 'git-svn-id:' | head -1 | \ sed -n 's/^.*@\([0-9][0-9]*\).*$/\1/p'` if [ -n $svn ] ; then svn=" r${svn}" git="+${git}" else git=" ${git}" fi fi if $git_cmd --work-tree=${SYSDIR}/.. diff-index \ --name-only HEAD | read dummy; then git="${git}-dirty" fi fi cat << EOF > vers.c $COPYRIGHT #define SCCSSTR "@(#)${VERSION} #${v}${svn}${git}: ${t}" #define VERSTR "${VERSION} #${v}${svn}${git}: ${t}\\n ${u}@${h}:${d}\\n" #define RELSTR "${RELEASE}" char sccs[sizeof(SCCSSTR) > 128 ? sizeof(SCCSSTR) : 128] = SCCSSTR; char version[sizeof(VERSTR) > 256 ? sizeof(VERSTR) : 256] = VERSTR; char compiler_version[] = "${compiler_v}"; char ostype[] = "${TYPE}"; char osrelease[sizeof(RELSTR) > 32 ? sizeof(RELSTR) : 32] = RELSTR; int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF echo $((v + 1)) > version Index: releng/8.4/usr.sbin/freebsd-update/freebsd-update.8 =================================================================== --- releng/8.4/usr.sbin/freebsd-update/freebsd-update.8 (revision 282873) +++ releng/8.4/usr.sbin/freebsd-update/freebsd-update.8 (revision 282874) @@ -1,173 +1,192 @@ .\"- .\" Copyright 2006, 2007 Colin Percival .\" All rights reserved .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted providing that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR .\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED .\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY .\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, .\" STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING .\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE .\" POSSIBILITY OF SUCH DAMAGE. .\" .\" $FreeBSD$ .\" -.Dd July 14, 2010 +.Dd March 2, 2015 .Dt FREEBSD-UPDATE 8 .Os FreeBSD .Sh NAME .Nm freebsd-update .Nd fetch and install binary updates to FreeBSD .Sh SYNOPSIS .Nm .Op Fl b Ar basedir .Op Fl d Ar workdir .Op Fl f Ar conffile +.Op Fl F .Op Fl k Ar KEY .Op Fl r Ar newrelease .Op Fl s Ar server .Op Fl t Ar address +.Op Fl -not-running-from-cron .Cm command ... .Sh DESCRIPTION The .Nm tool is used to fetch, install, and rollback binary updates to the FreeBSD base system. Note that updates are only available if they are being built for the FreeBSD release and architecture being used; in particular, the .Fx -Security Team only builds updates for releases shipped in binary form +Security Team only builds updates for releases shipped in binary form by the .Fx Release Engineering Team, e.g., .Fx -7.3-RELEASE and +9.3-RELEASE and .Fx -8.0, but not +10.1-RELEASE, but not .Fx -6.3-STABLE or +9.3-STABLE or .Fx -9.0-CURRENT. +11-CURRENT. .Sh OPTIONS The following options are supported: -.Bl -tag -width "-f conffile" +.Bl -tag -width "-r newrelease" .It Fl b Ar basedir Operate on a system mounted at .Ar basedir . (default: .Pa / , or as given in the configuration file.) .It Fl d Ar workdir Store working files in .Ar workdir . (default: .Pa /var/db/freebsd-update/ , or as given in the configuration file.) .It Fl f Ar conffile Read configuration options from .Ar conffile . (default: .Pa /etc/freebsd-update.conf ) +.It Fl F +Force +.Nm Cm fetch +to proceed where it normally would not, such as an unfinished upgrade .It Fl k Ar KEY Trust an RSA key with SHA256 of .Ar KEY . (default: read value from configuration file.) .It Fl r Ar newrelease Specify the new release to which .Nm should upgrade (upgrade command only). .It Fl s Ar server Fetch files from the specified server or server pool. (default: read value from configuration file.) .It Fl t Ar address Mail output of .Cm cron command, if any, to .Ar address . (default: root, or as given in the configuration file.) +.It Fl -not-running-from-cron +Force +.Nm Cm fetch +to proceed when there is no controlling tty. +This is for use by automated scripts and orchestration tools. +Please do not run +.Nm Cm fetch +from crontab or similar using this flag, see: +.Nm Cm cron .El .Sh COMMANDS The .Cm command can be any one of the following: -.Pp -.Bl -tag -width "-f conffile" +.Bl -tag -width "rollback" .It Cm fetch Based on the currently installed world and the configuration options set, fetch all available binary updates. .It Cm cron Sleep a random amount of time between 1 and 3600 seconds, then download updates as if the .Cm fetch command was used. If updates are downloaded, an email will be sent (to root or a different address if specified via the .Fl t option or in the configuration file). As the name suggests, this command is designed for running from .Xr cron 8 ; the random delay serves to minimize the probability that a large number of machines will simultaneously attempt to fetch updates. .It Cm upgrade Fetch files necessary for upgrading to a new release. Before using this command, make sure that you read the announcement and release notes for the new release in case there are any special steps needed for upgrading. +Note that this command may require up to 500 MB of space in +.Ar workdir +depending on which components of the +.Fx +base system are installed. .It Cm install Install the most recently fetched updates or upgrade. .It Cm rollback Uninstall the most recently installed updates. .It Cm IDS Compare the system against a "known good" index of the installed release. .El .Sh TIPS .Bl -bullet .It If your clock is set to local time, adding the line .Pp .Dl 0 3 * * * root /usr/sbin/freebsd-update cron .Pp to /etc/crontab will check for updates every night. If your clock is set to UTC, please pick a random time other than 3AM, to avoid overly imposing an uneven load on the server(s) hosting the updates. .It In spite of its name, -.Cm +.Nm IDS should not be relied upon as an "Intrusion Detection System", since if the system has been tampered with it cannot be trusted to operate correctly. If you intend to use this command for intrusion-detection purposes, make sure you boot from a secure disk (e.g., a CD). .El .Sh FILES .Bl -tag -width "/etc/freebsd-update.conf" -.It /etc/freebsd-update.conf +.It Pa /etc/freebsd-update.conf Default location of the .Nm configuration file. -.It /var/db/freebsd-update/ +.It Pa /var/db/freebsd-update/ Default location where .Nm stores temporary files and downloaded updates. .El .Sh SEE ALSO .Xr freebsd-update.conf 5 .Sh AUTHORS -.An Colin Percival Aq cperciva@FreeBSD.org +.An Colin Percival Aq Mt cperciva@FreeBSD.org Index: releng/8.4/usr.sbin/freebsd-update/freebsd-update.sh =================================================================== --- releng/8.4/usr.sbin/freebsd-update/freebsd-update.sh (revision 282873) +++ releng/8.4/usr.sbin/freebsd-update/freebsd-update.sh (revision 282874) @@ -1,3224 +1,3291 @@ #!/bin/sh #- # Copyright 2004-2007 Colin Percival # All rights reserved # # Redistribution and use in source and binary forms, with or without # modification, are permitted providing that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY # DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING # IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE # POSSIBILITY OF SUCH DAMAGE. # $FreeBSD$ #### Usage function -- called from command-line handling code. # Usage instructions. Options not listed: # --debug -- don't filter output from utilities # --no-stats -- don't show progress statistics while fetching files usage () { cat < ${LINE}" exit 1 fi done < ${CONFFILE} # Merge the settings read from the configuration file with those # provided at the command line. mergeconfig } # Provide some default parameters default_params () { # Save any parameters already configured, and clear the slate saveconfig nullconfig # Default configurations config_WorkDir /var/db/freebsd-update config_MailTo root config_AllowAdd yes config_AllowDelete yes config_KeepModifiedMetadata yes config_BaseDir / config_VerboseLevel stats config_StrictComponents no config_BackupKernel yes config_BackupKernelDir /boot/kernel.old config_BackupKernelSymbolFiles no # Merge these defaults into the earlier-configured settings mergeconfig } # Set utility output filtering options, based on ${VERBOSELEVEL} fetch_setup_verboselevel () { case ${VERBOSELEVEL} in debug) QUIETREDIR="/dev/stderr" QUIETFLAG=" " STATSREDIR="/dev/stderr" DDSTATS=".." XARGST="-t" NDEBUG=" " ;; nostats) QUIETREDIR="" QUIETFLAG="" STATSREDIR="/dev/null" DDSTATS=".." XARGST="" NDEBUG="" ;; stats) QUIETREDIR="/dev/null" QUIETFLAG="-q" STATSREDIR="/dev/stdout" DDSTATS="" XARGST="" NDEBUG="-n" ;; esac } # Perform sanity checks and set some final parameters # in preparation for fetching files. Figure out which # set of updates should be downloaded: If the user is # running *-p[0-9]+, strip off the last part; if the # user is running -SECURITY, call it -RELEASE. Chdir # into the working directory. -fetch_check_params () { +fetchupgrade_check_params () { export HTTP_USER_AGENT="freebsd-update (${COMMAND}, `uname -r`)" _SERVERNAME_z=\ "SERVERNAME must be given via command line or configuration file." _KEYPRINT_z="Key must be given via -k option or configuration file." _KEYPRINT_bad="Invalid key fingerprint: " _WORKDIR_bad="Directory does not exist or is not writable: " + _WORKDIR_bad2="Directory is not on a persistent filesystem: " if [ -z "${SERVERNAME}" ]; then echo -n "`basename $0`: " echo "${_SERVERNAME_z}" exit 1 fi if [ -z "${KEYPRINT}" ]; then echo -n "`basename $0`: " echo "${_KEYPRINT_z}" exit 1 fi if ! echo "${KEYPRINT}" | grep -qE "^[0-9a-f]{64}$"; then echo -n "`basename $0`: " echo -n "${_KEYPRINT_bad}" echo ${KEYPRINT} exit 1 fi if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then echo -n "`basename $0`: " echo -n "${_WORKDIR_bad}" echo ${WORKDIR} exit 1 fi + case `df -T ${WORKDIR}` in */dev/md[0-9]* | *tmpfs*) + echo -n "`basename $0`: " + echo -n "${_WORKDIR_bad2}" + echo ${WORKDIR} + exit 1 + ;; + esac chmod 700 ${WORKDIR} cd ${WORKDIR} || exit 1 # Generate release number. The s/SECURITY/RELEASE/ bit exists # to provide an upgrade path for FreeBSD Update 1.x users, since # the kernels provided by FreeBSD Update 1.x are always labelled # as X.Y-SECURITY. RELNUM=`uname -r | sed -E 's,-p[0-9]+,,' | sed -E 's,-SECURITY,-RELEASE,'` ARCH=`uname -m` FETCHDIR=${RELNUM}/${ARCH} PATCHDIR=${RELNUM}/${ARCH}/bp # Figure out what directory contains the running kernel BOOTFILE=`sysctl -n kern.bootfile` KERNELDIR=${BOOTFILE%/kernel} if ! [ -d ${KERNELDIR} ]; then echo "Cannot identify running kernel" exit 1 fi # Figure out what kernel configuration is running. We start with # the output of `uname -i`, and then make the following adjustments: # 1. Replace "SMP-GENERIC" with "SMP". Why the SMP kernel config # file says "ident SMP-GENERIC", I don't know... # 2. If the kernel claims to be GENERIC _and_ ${ARCH} is "amd64" # _and_ `sysctl kern.version` contains a line which ends "/SMP", then # we're running an SMP kernel. This mis-identification is a bug # which was fixed in 6.2-STABLE. KERNCONF=`uname -i` if [ ${KERNCONF} = "SMP-GENERIC" ]; then KERNCONF=SMP fi if [ ${KERNCONF} = "GENERIC" ] && [ ${ARCH} = "amd64" ]; then if sysctl kern.version | grep -qE '/SMP$'; then KERNCONF=SMP fi fi # Define some paths BSPATCH=/usr/bin/bspatch SHA256=/sbin/sha256 PHTTPGET=/usr/libexec/phttpget # Set up variables relating to VERBOSELEVEL fetch_setup_verboselevel # Construct a unique name from ${BASEDIR} BDHASH=`echo ${BASEDIR} | sha256 -q` } +# Perform sanity checks etc. before fetching updates. +fetch_check_params () { + fetchupgrade_check_params + + if ! [ -z "${TARGETRELEASE}" ]; then + echo -n "`basename $0`: " + echo -n "-r option is meaningless with 'fetch' command. " + echo "(Did you mean 'upgrade' instead?)" + exit 1 + fi + + # Check that we have updates ready to install + if [ -f ${BDHASH}-install/kerneldone -a $FORCEFETCH -eq 0 ]; then + echo "You have a partially completed upgrade pending" + echo "Run '$0 install' first." + echo "Run '$0 fetch -F' to proceed anyway." + exit 1 + fi +} + # Perform sanity checks etc. before fetching upgrades. upgrade_check_params () { - fetch_check_params + fetchupgrade_check_params # Unless set otherwise, we're upgrading to the same kernel config. NKERNCONF=${KERNCONF} # We need TARGETRELEASE set _TARGETRELEASE_z="Release target must be specified via -r option." if [ -z "${TARGETRELEASE}" ]; then echo -n "`basename $0`: " echo "${_TARGETRELEASE_z}" exit 1 fi # The target release should be != the current release. if [ "${TARGETRELEASE}" = "${RELNUM}" ]; then echo -n "`basename $0`: " echo "Cannot upgrade from ${RELNUM} to itself" exit 1 fi # Turning off AllowAdd or AllowDelete is a bad idea for upgrades. if [ "${ALLOWADD}" = "no" ]; then echo -n "`basename $0`: " echo -n "WARNING: \"AllowAdd no\" is a bad idea " echo "when upgrading between releases." echo fi if [ "${ALLOWDELETE}" = "no" ]; then echo -n "`basename $0`: " echo -n "WARNING: \"AllowDelete no\" is a bad idea " echo "when upgrading between releases." echo fi # Set EDITOR to /usr/bin/vi if it isn't already set : ${EDITOR:='/usr/bin/vi'} } # Perform sanity checks and set some final parameters in # preparation for installing updates. install_check_params () { # Check that we are root. All sorts of things won't work otherwise. if [ `id -u` != 0 ]; then echo "You must be root to run this." exit 1 fi # Check that securelevel <= 0. Otherwise we can't update schg files. if [ `sysctl -n kern.securelevel` -gt 0 ]; then echo "Updates cannot be installed when the system securelevel" echo "is greater than zero." exit 1 fi # Check that we have a working directory _WORKDIR_bad="Directory does not exist or is not writable: " if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then echo -n "`basename $0`: " echo -n "${_WORKDIR_bad}" echo ${WORKDIR} exit 1 fi cd ${WORKDIR} || exit 1 # Construct a unique name from ${BASEDIR} BDHASH=`echo ${BASEDIR} | sha256 -q` # Check that we have updates ready to install if ! [ -L ${BDHASH}-install ]; then echo "No updates are available to install." echo "Run '$0 fetch' first." exit 1 fi if ! [ -f ${BDHASH}-install/INDEX-OLD ] || ! [ -f ${BDHASH}-install/INDEX-NEW ]; then echo "Update manifest is corrupt -- this should never happen." echo "Re-run '$0 fetch'." exit 1 fi # Figure out what directory contains the running kernel BOOTFILE=`sysctl -n kern.bootfile` KERNELDIR=${BOOTFILE%/kernel} if ! [ -d ${KERNELDIR} ]; then echo "Cannot identify running kernel" exit 1 fi } # Perform sanity checks and set some final parameters in # preparation for UNinstalling updates. rollback_check_params () { # Check that we are root. All sorts of things won't work otherwise. if [ `id -u` != 0 ]; then echo "You must be root to run this." exit 1 fi # Check that we have a working directory _WORKDIR_bad="Directory does not exist or is not writable: " if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then echo -n "`basename $0`: " echo -n "${_WORKDIR_bad}" echo ${WORKDIR} exit 1 fi cd ${WORKDIR} || exit 1 # Construct a unique name from ${BASEDIR} BDHASH=`echo ${BASEDIR} | sha256 -q` # Check that we have updates ready to rollback if ! [ -L ${BDHASH}-rollback ]; then echo "No rollback directory found." exit 1 fi if ! [ -f ${BDHASH}-rollback/INDEX-OLD ] || ! [ -f ${BDHASH}-rollback/INDEX-NEW ]; then echo "Update manifest is corrupt -- this should never happen." exit 1 fi } # Perform sanity checks and set some final parameters # in preparation for comparing the system against the # published index. Figure out which index we should # compare against: If the user is running *-p[0-9]+, # strip off the last part; if the user is running # -SECURITY, call it -RELEASE. Chdir into the working # directory. IDS_check_params () { export HTTP_USER_AGENT="freebsd-update (${COMMAND}, `uname -r`)" _SERVERNAME_z=\ "SERVERNAME must be given via command line or configuration file." _KEYPRINT_z="Key must be given via -k option or configuration file." _KEYPRINT_bad="Invalid key fingerprint: " _WORKDIR_bad="Directory does not exist or is not writable: " if [ -z "${SERVERNAME}" ]; then echo -n "`basename $0`: " echo "${_SERVERNAME_z}" exit 1 fi if [ -z "${KEYPRINT}" ]; then echo -n "`basename $0`: " echo "${_KEYPRINT_z}" exit 1 fi if ! echo "${KEYPRINT}" | grep -qE "^[0-9a-f]{64}$"; then echo -n "`basename $0`: " echo -n "${_KEYPRINT_bad}" echo ${KEYPRINT} exit 1 fi if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then echo -n "`basename $0`: " echo -n "${_WORKDIR_bad}" echo ${WORKDIR} exit 1 fi cd ${WORKDIR} || exit 1 # Generate release number. The s/SECURITY/RELEASE/ bit exists # to provide an upgrade path for FreeBSD Update 1.x users, since # the kernels provided by FreeBSD Update 1.x are always labelled # as X.Y-SECURITY. RELNUM=`uname -r | sed -E 's,-p[0-9]+,,' | sed -E 's,-SECURITY,-RELEASE,'` ARCH=`uname -m` FETCHDIR=${RELNUM}/${ARCH} PATCHDIR=${RELNUM}/${ARCH}/bp # Figure out what directory contains the running kernel BOOTFILE=`sysctl -n kern.bootfile` KERNELDIR=${BOOTFILE%/kernel} if ! [ -d ${KERNELDIR} ]; then echo "Cannot identify running kernel" exit 1 fi # Figure out what kernel configuration is running. We start with # the output of `uname -i`, and then make the following adjustments: # 1. Replace "SMP-GENERIC" with "SMP". Why the SMP kernel config # file says "ident SMP-GENERIC", I don't know... # 2. If the kernel claims to be GENERIC _and_ ${ARCH} is "amd64" # _and_ `sysctl kern.version` contains a line which ends "/SMP", then # we're running an SMP kernel. This mis-identification is a bug # which was fixed in 6.2-STABLE. KERNCONF=`uname -i` if [ ${KERNCONF} = "SMP-GENERIC" ]; then KERNCONF=SMP fi if [ ${KERNCONF} = "GENERIC" ] && [ ${ARCH} = "amd64" ]; then if sysctl kern.version | grep -qE '/SMP$'; then KERNCONF=SMP fi fi # Define some paths SHA256=/sbin/sha256 PHTTPGET=/usr/libexec/phttpget # Set up variables relating to VERBOSELEVEL fetch_setup_verboselevel } #### Core functionality -- the actual work gets done here # Use an SRV query to pick a server. If the SRV query doesn't provide # a useful answer, use the server name specified by the user. # Put another way... look up _http._tcp.${SERVERNAME} and pick a server # from that; or if no servers are returned, use ${SERVERNAME}. # This allows a user to specify "portsnap.freebsd.org" (in which case # portsnap will select one of the mirrors) or "portsnap5.tld.freebsd.org" # (in which case portsnap will use that particular server, since there # won't be an SRV entry for that name). # # We ignore the Port field, since we are always going to use port 80. # Fetch the mirror list, but do not pick a mirror yet. Returns 1 if # no mirrors are available for any reason. fetch_pick_server_init () { : > serverlist_tried # Check that host(1) exists (i.e., that the system wasn't built with the # WITHOUT_BIND set) and don't try to find a mirror if it doesn't exist. if ! which -s host; then : > serverlist_full return 1 fi echo -n "Looking up ${SERVERNAME} mirrors... " # Issue the SRV query and pull out the Priority, Weight, and Target fields. # BIND 9 prints "$name has SRV record ..." while BIND 8 prints # "$name server selection ..."; we allow either format. MLIST="_http._tcp.${SERVERNAME}" host -t srv "${MLIST}" | sed -nE "s/${MLIST} (has SRV record|server selection) //p" | cut -f 1,2,4 -d ' ' | sed -e 's/\.$//' | sort > serverlist_full # If no records, give up -- we'll just use the server name we were given. if [ `wc -l < serverlist_full` -eq 0 ]; then echo "none found." return 1 fi # Report how many mirrors we found. echo `wc -l < serverlist_full` "mirrors found." # Generate a random seed for use in picking mirrors. If HTTP_PROXY # is set, this will be used to generate the seed; otherwise, the seed # will be random. if [ -n "${HTTP_PROXY}${http_proxy}" ]; then RANDVALUE=`sha256 -qs "${HTTP_PROXY}${http_proxy}" | tr -d 'a-f' | cut -c 1-9` else RANDVALUE=`jot -r 1 0 999999999` fi } # Pick a mirror. Returns 1 if we have run out of mirrors to try. fetch_pick_server () { # Generate a list of not-yet-tried mirrors sort serverlist_tried | comm -23 serverlist_full - > serverlist # Have we run out of mirrors? if [ `wc -l < serverlist` -eq 0 ]; then echo "No mirrors remaining, giving up." return 1 fi # Find the highest priority level (lowest numeric value). SRV_PRIORITY=`cut -f 1 -d ' ' serverlist | sort -n | head -1` # Add up the weights of the response lines at that priority level. SRV_WSUM=0; while read X; do case "$X" in ${SRV_PRIORITY}\ *) SRV_W=`echo $X | cut -f 2 -d ' '` SRV_WSUM=$(($SRV_WSUM + $SRV_W)) ;; esac done < serverlist # If all the weights are 0, pretend that they are all 1 instead. if [ ${SRV_WSUM} -eq 0 ]; then SRV_WSUM=`grep -E "^${SRV_PRIORITY} " serverlist | wc -l` SRV_W_ADD=1 else SRV_W_ADD=0 fi # Pick a value between 0 and the sum of the weights - 1 SRV_RND=`expr ${RANDVALUE} % ${SRV_WSUM}` # Read through the list of mirrors and set SERVERNAME. Write the line # corresponding to the mirror we selected into serverlist_tried so that # we won't try it again. while read X; do case "$X" in ${SRV_PRIORITY}\ *) SRV_W=`echo $X | cut -f 2 -d ' '` SRV_W=$(($SRV_W + $SRV_W_ADD)) if [ $SRV_RND -lt $SRV_W ]; then SERVERNAME=`echo $X | cut -f 3 -d ' '` echo "$X" >> serverlist_tried break else SRV_RND=$(($SRV_RND - $SRV_W)) fi ;; esac done < serverlist } # Take a list of ${oldhash}|${newhash} and output a list of needed patches, # i.e., those for which we have ${oldhash} and don't have ${newhash}. fetch_make_patchlist () { grep -vE "^([0-9a-f]{64})\|\1$" | tr '|' ' ' | while read X Y; do if [ -f "files/${Y}.gz" ] || [ ! -f "files/${X}.gz" ]; then continue fi echo "${X}|${Y}" done | uniq } # Print user-friendly progress statistics fetch_progress () { LNC=0 while read x; do LNC=$(($LNC + 1)) if [ $(($LNC % 10)) = 0 ]; then echo -n $LNC elif [ $(($LNC % 2)) = 0 ]; then echo -n . fi done echo -n " " } # Function for asking the user if everything is ok continuep () { while read -p "Does this look reasonable (y/n)? " CONTINUE; do case "${CONTINUE}" in y*) return 0 ;; n*) return 1 ;; esac done } # Initialize the working directory workdir_init () { mkdir -p files touch tINDEX.present } # Check that we have a public key with an appropriate hash, or # fetch the key if it doesn't exist. Returns 1 if the key has # not yet been fetched. fetch_key () { if [ -r pub.ssl ] && [ `${SHA256} -q pub.ssl` = ${KEYPRINT} ]; then return 0 fi echo -n "Fetching public key from ${SERVERNAME}... " rm -f pub.ssl fetch ${QUIETFLAG} http://${SERVERNAME}/${FETCHDIR}/pub.ssl \ 2>${QUIETREDIR} || true if ! [ -r pub.ssl ]; then echo "failed." return 1 fi if ! [ `${SHA256} -q pub.ssl` = ${KEYPRINT} ]; then echo "key has incorrect hash." rm -f pub.ssl return 1 fi echo "done." } # Fetch metadata signature, aka "tag". fetch_tag () { echo -n "Fetching metadata signature " echo ${NDEBUG} "for ${RELNUM} from ${SERVERNAME}... " rm -f latest.ssl fetch ${QUIETFLAG} http://${SERVERNAME}/${FETCHDIR}/latest.ssl \ 2>${QUIETREDIR} || true if ! [ -r latest.ssl ]; then echo "failed." return 1 fi openssl rsautl -pubin -inkey pub.ssl -verify \ < latest.ssl > tag.new 2>${QUIETREDIR} || true rm latest.ssl if ! [ `wc -l < tag.new` = 1 ] || ! grep -qE \ "^freebsd-update\|${ARCH}\|${RELNUM}\|[0-9]+\|[0-9a-f]{64}\|[0-9]{10}" \ tag.new; then echo "invalid signature." return 1 fi echo "done." RELPATCHNUM=`cut -f 4 -d '|' < tag.new` TINDEXHASH=`cut -f 5 -d '|' < tag.new` EOLTIME=`cut -f 6 -d '|' < tag.new` } # Sanity-check the patch number in a tag, to make sure that we're not # going to "update" backwards and to prevent replay attacks. fetch_tagsanity () { # Check that we're not going to move from -pX to -pY with Y < X. RELPX=`uname -r | sed -E 's,.*-,,'` if echo ${RELPX} | grep -qE '^p[0-9]+$'; then RELPX=`echo ${RELPX} | cut -c 2-` else RELPX=0 fi if [ "${RELPATCHNUM}" -lt "${RELPX}" ]; then echo echo -n "Files on mirror (${RELNUM}-p${RELPATCHNUM})" echo " appear older than what" echo "we are currently running (`uname -r`)!" echo "Cowardly refusing to proceed any further." return 1 fi # If "tag" exists and corresponds to ${RELNUM}, make sure that # it contains a patch number <= RELPATCHNUM, in order to protect # against rollback (replay) attacks. if [ -f tag ] && grep -qE \ "^freebsd-update\|${ARCH}\|${RELNUM}\|[0-9]+\|[0-9a-f]{64}\|[0-9]{10}" \ tag; then LASTRELPATCHNUM=`cut -f 4 -d '|' < tag` if [ "${RELPATCHNUM}" -lt "${LASTRELPATCHNUM}" ]; then echo echo -n "Files on mirror (${RELNUM}-p${RELPATCHNUM})" echo " are older than the" echo -n "most recently seen updates" echo " (${RELNUM}-p${LASTRELPATCHNUM})." echo "Cowardly refusing to proceed any further." return 1 fi fi } # Fetch metadata index file fetch_metadata_index () { echo ${NDEBUG} "Fetching metadata index... " rm -f ${TINDEXHASH} fetch ${QUIETFLAG} http://${SERVERNAME}/${FETCHDIR}/t/${TINDEXHASH} 2>${QUIETREDIR} if ! [ -f ${TINDEXHASH} ]; then echo "failed." return 1 fi if [ `${SHA256} -q ${TINDEXHASH}` != ${TINDEXHASH} ]; then echo "update metadata index corrupt." return 1 fi echo "done." } # Print an error message about signed metadata being bogus. fetch_metadata_bogus () { echo echo "The update metadata$1 is correctly signed, but" echo "failed an integrity check." echo "Cowardly refusing to proceed any further." return 1 } # Construct tINDEX.new by merging the lines named in $1 from ${TINDEXHASH} # with the lines not named in $@ from tINDEX.present (if that file exists). fetch_metadata_index_merge () { for METAFILE in $@; do if [ `grep -E "^${METAFILE}\|" ${TINDEXHASH} | wc -l` \ -ne 1 ]; then fetch_metadata_bogus " index" return 1 fi grep -E "${METAFILE}\|" ${TINDEXHASH} done | sort > tINDEX.wanted if [ -f tINDEX.present ]; then join -t '|' -v 2 tINDEX.wanted tINDEX.present | sort -m - tINDEX.wanted > tINDEX.new rm tINDEX.wanted else mv tINDEX.wanted tINDEX.new fi } # Sanity check all the lines of tINDEX.new. Even if more metadata lines # are added by future versions of the server, this won't cause problems, # since the only lines which appear in tINDEX.new are the ones which we # specifically grepped out of ${TINDEXHASH}. fetch_metadata_index_sanity () { if grep -qvE '^[0-9A-Z.-]+\|[0-9a-f]{64}$' tINDEX.new; then fetch_metadata_bogus " index" return 1 fi } # Sanity check the metadata file $1. fetch_metadata_sanity () { # Some aliases to save space later: ${P} is a character which can # appear in a path; ${M} is the four numeric metadata fields; and # ${H} is a sha256 hash. - P="[-+./:=%@_[~[:alnum:]]" + P="[-+./:=,%@_[~[:alnum:]]" M="[0-9]+\|[0-9]+\|[0-9]+\|[0-9]+" H="[0-9a-f]{64}" # Check that the first four fields make sense. if gunzip -c < files/$1.gz | grep -qvE "^[a-z]+\|[0-9a-z]+\|${P}+\|[fdL-]\|"; then fetch_metadata_bogus "" return 1 fi # Remove the first three fields. gunzip -c < files/$1.gz | cut -f 4- -d '|' > sanitycheck.tmp # Sanity check entries with type 'f' if grep -E '^f' sanitycheck.tmp | grep -qvE "^f\|${M}\|${H}\|${P}*\$"; then fetch_metadata_bogus "" return 1 fi # Sanity check entries with type 'd' if grep -E '^d' sanitycheck.tmp | grep -qvE "^d\|${M}\|\|\$"; then fetch_metadata_bogus "" return 1 fi # Sanity check entries with type 'L' if grep -E '^L' sanitycheck.tmp | grep -qvE "^L\|${M}\|${P}*\|\$"; then fetch_metadata_bogus "" return 1 fi # Sanity check entries with type '-' if grep -E '^-' sanitycheck.tmp | grep -qvE "^-\|\|\|\|\|\|"; then fetch_metadata_bogus "" return 1 fi # Clean up rm sanitycheck.tmp } # Fetch the metadata index and metadata files listed in $@, # taking advantage of metadata patches where possible. fetch_metadata () { fetch_metadata_index || return 1 fetch_metadata_index_merge $@ || return 1 fetch_metadata_index_sanity || return 1 # Generate a list of wanted metadata patches join -t '|' -o 1.2,2.2 tINDEX.present tINDEX.new | fetch_make_patchlist > patchlist if [ -s patchlist ]; then # Attempt to fetch metadata patches echo -n "Fetching `wc -l < patchlist | tr -d ' '` " echo ${NDEBUG} "metadata patches.${DDSTATS}" tr '|' '-' < patchlist | lam -s "${FETCHDIR}/tp/" - -s ".gz" | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${STATSREDIR} | fetch_progress echo "done." # Attempt to apply metadata patches echo -n "Applying metadata patches... " tr '|' ' ' < patchlist | while read X Y; do if [ ! -f "${X}-${Y}.gz" ]; then continue; fi gunzip -c < ${X}-${Y}.gz > diff gunzip -c < files/${X}.gz > diff-OLD # Figure out which lines are being added and removed grep -E '^-' diff | cut -c 2- | while read PREFIX; do look "${PREFIX}" diff-OLD done | sort > diff-rm grep -E '^\+' diff | cut -c 2- > diff-add # Generate the new file comm -23 diff-OLD diff-rm | sort - diff-add > diff-NEW if [ `${SHA256} -q diff-NEW` = ${Y} ]; then mv diff-NEW files/${Y} gzip -n files/${Y} else mv diff-NEW ${Y}.bad fi rm -f ${X}-${Y}.gz diff rm -f diff-OLD diff-NEW diff-add diff-rm done 2>${QUIETREDIR} echo "done." fi # Update metadata without patches cut -f 2 -d '|' < tINDEX.new | while read Y; do if [ ! -f "files/${Y}.gz" ]; then echo ${Y}; fi done | sort -u > filelist if [ -s filelist ]; then echo -n "Fetching `wc -l < filelist | tr -d ' '` " echo ${NDEBUG} "metadata files... " lam -s "${FETCHDIR}/m/" - -s ".gz" < filelist | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${QUIETREDIR} while read Y; do if ! [ -f ${Y}.gz ]; then echo "failed." return 1 fi if [ `gunzip -c < ${Y}.gz | ${SHA256} -q` = ${Y} ]; then mv ${Y}.gz files/${Y}.gz else echo "metadata is corrupt." return 1 fi done < filelist echo "done." fi # Sanity-check the metadata files. cut -f 2 -d '|' tINDEX.new > filelist while read X; do fetch_metadata_sanity ${X} || return 1 done < filelist # Remove files which are no longer needed cut -f 2 -d '|' tINDEX.present | sort > oldfiles cut -f 2 -d '|' tINDEX.new | sort | comm -13 - oldfiles | lam -s "files/" - -s ".gz" | xargs rm -f rm patchlist filelist oldfiles rm ${TINDEXHASH} # We're done! mv tINDEX.new tINDEX.present mv tag.new tag return 0 } # Extract a subset of a downloaded metadata file containing only the parts # which are listed in COMPONENTS. fetch_filter_metadata_components () { METAHASH=`look "$1|" tINDEX.present | cut -f 2 -d '|'` gunzip -c < files/${METAHASH}.gz > $1.all # Fish out the lines belonging to components we care about. for C in ${COMPONENTS}; do look "`echo ${C} | tr '/' '|'`|" $1.all done > $1 # Remove temporary file. rm $1.all } # Generate a filtered version of the metadata file $1 from the downloaded # file, by fishing out the lines corresponding to components we're trying # to keep updated, and then removing lines corresponding to paths we want # to ignore. fetch_filter_metadata () { # Fish out the lines belonging to components we care about. fetch_filter_metadata_components $1 # Canonicalize directory names by removing any trailing / in # order to avoid listing directories multiple times if they # belong to multiple components. Turning "/" into "" doesn't # matter, since we add a leading "/" when we use paths later. cut -f 3- -d '|' $1 | sed -e 's,/|d|,|d|,' | sed -e 's,/|-|,|-|,' | sort -u > $1.tmp # Figure out which lines to ignore and remove them. for X in ${IGNOREPATHS}; do grep -E "^${X}" $1.tmp done | sort -u | comm -13 - $1.tmp > $1 # Remove temporary files. rm $1.tmp } # Filter the metadata file $1 by adding lines with "/boot/$2" # replaced by ${KERNELDIR} (which is `sysctl -n kern.bootfile` minus the # trailing "/kernel"); and if "/boot/$2" does not exist, remove # the original lines which start with that. # Put another way: Deal with the fact that the FOO kernel is sometimes # installed in /boot/FOO/ and is sometimes installed elsewhere. fetch_filter_kernel_names () { grep ^/boot/$2 $1 | sed -e "s,/boot/$2,${KERNELDIR},g" | sort - $1 > $1.tmp mv $1.tmp $1 if ! [ -d /boot/$2 ]; then grep -v ^/boot/$2 $1 > $1.tmp mv $1.tmp $1 fi } # For all paths appearing in $1 or $3, inspect the system # and generate $2 describing what is currently installed. fetch_inspect_system () { # No errors yet... rm -f .err # Tell the user why his disk is suddenly making lots of noise echo -n "Inspecting system... " # Generate list of files to inspect cat $1 $3 | cut -f 1 -d '|' | sort -u > filelist # Examine each file and output lines of the form # /path/to/file|type|device-inum|user|group|perm|flags|value # sorted by device and inode number. while read F; do # If the symlink/file/directory does not exist, record this. if ! [ -e ${BASEDIR}/${F} ]; then echo "${F}|-||||||" continue fi if ! [ -r ${BASEDIR}/${F} ]; then echo "Cannot read file: ${BASEDIR}/${F}" \ >/dev/stderr touch .err return 1 fi # Otherwise, output an index line. if [ -L ${BASEDIR}/${F} ]; then echo -n "${F}|L|" stat -n -f '%d-%i|%u|%g|%Mp%Lp|%Of|' ${BASEDIR}/${F}; readlink ${BASEDIR}/${F}; elif [ -f ${BASEDIR}/${F} ]; then echo -n "${F}|f|" stat -n -f '%d-%i|%u|%g|%Mp%Lp|%Of|' ${BASEDIR}/${F}; sha256 -q ${BASEDIR}/${F}; elif [ -d ${BASEDIR}/${F} ]; then echo -n "${F}|d|" stat -f '%d-%i|%u|%g|%Mp%Lp|%Of|' ${BASEDIR}/${F}; else echo "Unknown file type: ${BASEDIR}/${F}" \ >/dev/stderr touch .err return 1 fi done < filelist | sort -k 3,3 -t '|' > $2.tmp rm filelist - # Check if an error occured during system inspection + # Check if an error occurred during system inspection if [ -f .err ]; then return 1 fi # Convert to the form # /path/to/file|type|user|group|perm|flags|value|hlink # by resolving identical device and inode numbers into hard links. cut -f 1,3 -d '|' $2.tmp | sort -k 1,1 -t '|' | sort -s -u -k 2,2 -t '|' | join -1 2 -2 3 -t '|' - $2.tmp | awk -F \| -v OFS=\| \ '{ if (($2 == $3) || ($4 == "-")) print $3,$4,$5,$6,$7,$8,$9,"" else print $3,$4,$5,$6,$7,$8,$9,$2 }' | sort > $2 rm $2.tmp # We're finished looking around echo "done." } # For any paths matching ${MERGECHANGES}, compare $1 and $2 and find any # files which differ; generate $3 containing these paths and the old hashes. fetch_filter_mergechanges () { # Pull out the paths and hashes of the files matching ${MERGECHANGES}. for F in $1 $2; do for X in ${MERGECHANGES}; do grep -E "^${X}" ${F} done | cut -f 1,2,7 -d '|' | sort > ${F}-values done # Any line in $2-values which doesn't appear in $1-values and is a # file means that we should list the path in $3. comm -13 $1-values $2-values | fgrep '|f|' | cut -f 1 -d '|' > $2-paths # For each path, pull out one (and only one!) entry from $1-values. # Note that we cannot distinguish which "old" version the user made # changes to; but hopefully any changes which occur due to security # updates will exist in both the "new" version and the version which # the user has installed, so the merging will still work. while read X; do look "${X}|" $1-values | head -1 done < $2-paths > $3 # Clean up rm $1-values $2-values $2-paths } # For any paths matching ${UPDATEIFUNMODIFIED}, remove lines from $[123] # which correspond to lines in $2 with hashes not matching $1 or $3, unless # the paths are listed in $4. For entries in $2 marked "not present" # (aka. type -), remove lines from $[123] unless there is a corresponding # entry in $1. fetch_filter_unmodified_notpresent () { # Figure out which lines of $1 and $3 correspond to bits which # should only be updated if they haven't changed, and fish out # the (path, type, value) tuples. # NOTE: We don't consider a file to be "modified" if it matches # the hash from $3. for X in ${UPDATEIFUNMODIFIED}; do grep -E "^${X}" $1 grep -E "^${X}" $3 done | cut -f 1,2,7 -d '|' | sort > $1-values # Do the same for $2. for X in ${UPDATEIFUNMODIFIED}; do grep -E "^${X}" $2 done | cut -f 1,2,7 -d '|' | sort > $2-values # Any entry in $2-values which is not in $1-values corresponds to # a path which we need to remove from $1, $2, and $3, unless it # that path appears in $4. comm -13 $1-values $2-values | sort -t '|' -k 1,1 > mlines.tmp cut -f 1 -d '|' $4 | sort | join -v 2 -t '|' - mlines.tmp | sort > mlines rm $1-values $2-values mlines.tmp # Any lines in $2 which are not in $1 AND are "not present" lines # also belong in mlines. comm -13 $1 $2 | cut -f 1,2,7 -d '|' | fgrep '|-|' >> mlines # Remove lines from $1, $2, and $3 for X in $1 $2 $3; do sort -t '|' -k 1,1 ${X} > ${X}.tmp cut -f 1 -d '|' < mlines | sort | join -v 2 -t '|' - ${X}.tmp | sort > ${X} rm ${X}.tmp done # Store a list of the modified files, for future reference fgrep -v '|-|' mlines | cut -f 1 -d '|' > modifiedfiles rm mlines } # For each entry in $1 of type -, remove any corresponding # entry from $2 if ${ALLOWADD} != "yes". Remove all entries # of type - from $1. fetch_filter_allowadd () { cut -f 1,2 -d '|' < $1 | fgrep '|-' | cut -f 1 -d '|' > filesnotpresent if [ ${ALLOWADD} != "yes" ]; then sort < $2 | join -v 1 -t '|' - filesnotpresent | sort > $2.tmp mv $2.tmp $2 fi sort < $1 | join -v 1 -t '|' - filesnotpresent | sort > $1.tmp mv $1.tmp $1 rm filesnotpresent } # If ${ALLOWDELETE} != "yes", then remove any entries from $1 # which don't correspond to entries in $2. fetch_filter_allowdelete () { # Produce a lists ${PATH}|${TYPE} for X in $1 $2; do cut -f 1-2 -d '|' < ${X} | sort -u > ${X}.nodes done # Figure out which lines need to be removed from $1. if [ ${ALLOWDELETE} != "yes" ]; then comm -23 $1.nodes $2.nodes > $1.badnodes else : > $1.badnodes fi # Remove the relevant lines from $1 while read X; do look "${X}|" $1 done < $1.badnodes | comm -13 - $1 > $1.tmp mv $1.tmp $1 rm $1.badnodes $1.nodes $2.nodes } # If ${KEEPMODIFIEDMETADATA} == "yes", then for each entry in $2 # with metadata not matching any entry in $1, replace the corresponding # line of $3 with one having the same metadata as the entry in $2. fetch_filter_modified_metadata () { # Fish out the metadata from $1 and $2 for X in $1 $2; do cut -f 1-6 -d '|' < ${X} > ${X}.metadata done # Find the metadata we need to keep if [ ${KEEPMODIFIEDMETADATA} = "yes" ]; then comm -13 $1.metadata $2.metadata > keepmeta else : > keepmeta fi # Extract the lines which we need to remove from $3, and # construct the lines which we need to add to $3. : > $3.remove : > $3.add while read LINE; do NODE=`echo "${LINE}" | cut -f 1-2 -d '|'` look "${NODE}|" $3 >> $3.remove look "${NODE}|" $3 | cut -f 7- -d '|' | lam -s "${LINE}|" - >> $3.add done < keepmeta # Remove the specified lines and add the new lines. sort $3.remove | comm -13 - $3 | sort -u - $3.add > $3.tmp mv $3.tmp $3 rm keepmeta $1.metadata $2.metadata $3.add $3.remove } # Remove lines from $1 and $2 which are identical; # no need to update a file if it isn't changing. fetch_filter_uptodate () { comm -23 $1 $2 > $1.tmp comm -13 $1 $2 > $2.tmp mv $1.tmp $1 mv $2.tmp $2 } # Fetch any "clean" old versions of files we need for merging changes. fetch_files_premerge () { # We only need to do anything if $1 is non-empty. if [ -s $1 ]; then # Tell the user what we're doing echo -n "Fetching files from ${OLDRELNUM} for merging... " # List of files wanted fgrep '|f|' < $1 | cut -f 3 -d '|' | sort -u > files.wanted # Only fetch the files we don't already have while read Y; do if [ ! -f "files/${Y}.gz" ]; then echo ${Y}; fi done < files.wanted > filelist # Actually fetch them lam -s "${OLDFETCHDIR}/f/" - -s ".gz" < filelist | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${QUIETREDIR} # Make sure we got them all, and move them into /files/ while read Y; do if ! [ -f ${Y}.gz ]; then echo "failed." return 1 fi if [ `gunzip -c < ${Y}.gz | ${SHA256} -q` = ${Y} ]; then mv ${Y}.gz files/${Y}.gz else echo "${Y} has incorrect hash." return 1 fi done < filelist echo "done." # Clean up rm filelist files.wanted fi } # Prepare to fetch files: Generate a list of the files we need, # copy the unmodified files we have into /files/, and generate # a list of patches to download. fetch_files_prepare () { # Tell the user why his disk is suddenly making lots of noise echo -n "Preparing to download files... " # Reduce indices to ${PATH}|${HASH} pairs for X in $1 $2 $3; do cut -f 1,2,7 -d '|' < ${X} | fgrep '|f|' | cut -f 1,3 -d '|' | sort > ${X}.hashes done # List of files wanted cut -f 2 -d '|' < $3.hashes | sort -u | while read HASH; do if ! [ -f files/${HASH}.gz ]; then echo ${HASH} fi done > files.wanted # Generate a list of unmodified files comm -12 $1.hashes $2.hashes | sort -k 1,1 -t '|' > unmodified.files # Copy all files into /files/. We only need the unmodified files # for use in patching; but we'll want all of them if the user asks # to rollback the updates later. while read LINE; do F=`echo "${LINE}" | cut -f 1 -d '|'` HASH=`echo "${LINE}" | cut -f 2 -d '|'` # Skip files we already have. if [ -f files/${HASH}.gz ]; then continue fi # Make sure the file hasn't changed. cp "${BASEDIR}/${F}" tmpfile if [ `sha256 -q tmpfile` != ${HASH} ]; then echo echo "File changed while FreeBSD Update running: ${F}" return 1 fi # Place the file into storage. gzip -c < tmpfile > files/${HASH}.gz rm tmpfile done < $2.hashes # Produce a list of patches to download sort -k 1,1 -t '|' $3.hashes | join -t '|' -o 2.2,1.2 - unmodified.files | fetch_make_patchlist > patchlist # Garbage collect rm unmodified.files $1.hashes $2.hashes $3.hashes # We don't need the list of possible old files any more. rm $1 # We're finished making noise echo "done." } # Fetch files. fetch_files () { # Attempt to fetch patches if [ -s patchlist ]; then echo -n "Fetching `wc -l < patchlist | tr -d ' '` " echo ${NDEBUG} "patches.${DDSTATS}" tr '|' '-' < patchlist | lam -s "${PATCHDIR}/" - | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${STATSREDIR} | fetch_progress echo "done." # Attempt to apply patches echo -n "Applying patches... " tr '|' ' ' < patchlist | while read X Y; do if [ ! -f "${X}-${Y}" ]; then continue; fi gunzip -c < files/${X}.gz > OLD bspatch OLD NEW ${X}-${Y} if [ `${SHA256} -q NEW` = ${Y} ]; then mv NEW files/${Y} gzip -n files/${Y} fi rm -f diff OLD NEW ${X}-${Y} done 2>${QUIETREDIR} echo "done." fi # Download files which couldn't be generate via patching while read Y; do if [ ! -f "files/${Y}.gz" ]; then echo ${Y}; fi done < files.wanted > filelist if [ -s filelist ]; then echo -n "Fetching `wc -l < filelist | tr -d ' '` " echo ${NDEBUG} "files... " lam -s "${FETCHDIR}/f/" - -s ".gz" < filelist | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${QUIETREDIR} while read Y; do if ! [ -f ${Y}.gz ]; then echo "failed." return 1 fi if [ `gunzip -c < ${Y}.gz | ${SHA256} -q` = ${Y} ]; then mv ${Y}.gz files/${Y}.gz else echo "${Y} has incorrect hash." return 1 fi done < filelist echo "done." fi # Clean up rm files.wanted filelist patchlist } # Create and populate install manifest directory; and report what updates # are available. fetch_create_manifest () { # If we have an existing install manifest, nuke it. if [ -L "${BDHASH}-install" ]; then rm -r ${BDHASH}-install/ rm ${BDHASH}-install fi # Report to the user if any updates were avoided due to local changes if [ -s modifiedfiles ]; then echo echo -n "The following files are affected by updates, " echo "but no changes have" echo -n "been downloaded because the files have been " echo "modified locally:" cat modifiedfiles fi | $PAGER rm modifiedfiles # If no files will be updated, tell the user and exit if ! [ -s INDEX-PRESENT ] && ! [ -s INDEX-NEW ]; then rm INDEX-PRESENT INDEX-NEW echo echo -n "No updates needed to update system to " echo "${RELNUM}-p${RELPATCHNUM}." return fi # Divide files into (a) removed files, (b) added files, and # (c) updated files. cut -f 1 -d '|' < INDEX-PRESENT | sort > INDEX-PRESENT.flist cut -f 1 -d '|' < INDEX-NEW | sort > INDEX-NEW.flist comm -23 INDEX-PRESENT.flist INDEX-NEW.flist > files.removed comm -13 INDEX-PRESENT.flist INDEX-NEW.flist > files.added comm -12 INDEX-PRESENT.flist INDEX-NEW.flist > files.updated rm INDEX-PRESENT.flist INDEX-NEW.flist # Report removed files, if any if [ -s files.removed ]; then echo echo -n "The following files will be removed " echo "as part of updating to ${RELNUM}-p${RELPATCHNUM}:" cat files.removed fi | $PAGER rm files.removed # Report added files, if any if [ -s files.added ]; then echo echo -n "The following files will be added " echo "as part of updating to ${RELNUM}-p${RELPATCHNUM}:" cat files.added fi | $PAGER rm files.added # Report updated files, if any if [ -s files.updated ]; then echo echo -n "The following files will be updated " echo "as part of updating to ${RELNUM}-p${RELPATCHNUM}:" cat files.updated fi | $PAGER rm files.updated # Create a directory for the install manifest. MDIR=`mktemp -d install.XXXXXX` || return 1 # Populate it mv INDEX-PRESENT ${MDIR}/INDEX-OLD mv INDEX-NEW ${MDIR}/INDEX-NEW # Link it into place ln -s ${MDIR} ${BDHASH}-install } # Warn about any upcoming EoL fetch_warn_eol () { # What's the current time? NOWTIME=`date "+%s"` # When did we last warn about the EoL date? if [ -f lasteolwarn ]; then LASTWARN=`cat lasteolwarn` else LASTWARN=`expr ${NOWTIME} - 63072000` fi # If the EoL time is past, warn. if [ ${EOLTIME} -lt ${NOWTIME} ]; then echo cat <<-EOF WARNING: `uname -sr` HAS PASSED ITS END-OF-LIFE DATE. Any security issues discovered after `date -r ${EOLTIME}` will not have been corrected. EOF return 1 fi # Figure out how long it has been since we last warned about the # upcoming EoL, and how much longer we have left. SINCEWARN=`expr ${NOWTIME} - ${LASTWARN}` TIMELEFT=`expr ${EOLTIME} - ${NOWTIME}` # Don't warn if the EoL is more than 3 months away if [ ${TIMELEFT} -gt 7884000 ]; then return 0 fi # Don't warn if the time remaining is more than 3 times the time # since the last warning. if [ ${TIMELEFT} -gt `expr ${SINCEWARN} \* 3` ]; then return 0 fi # Figure out what time units to use. if [ ${TIMELEFT} -lt 604800 ]; then UNIT="day" SIZE=86400 elif [ ${TIMELEFT} -lt 2678400 ]; then UNIT="week" SIZE=604800 else UNIT="month" SIZE=2678400 fi # Compute the right number of units NUM=`expr ${TIMELEFT} / ${SIZE}` if [ ${NUM} != 1 ]; then UNIT="${UNIT}s" fi # Print the warning echo cat <<-EOF WARNING: `uname -sr` is approaching its End-of-Life date. It is strongly recommended that you upgrade to a newer release within the next ${NUM} ${UNIT}. EOF # Update the stored time of last warning echo ${NOWTIME} > lasteolwarn } # Do the actual work involved in "fetch" / "cron". fetch_run () { workdir_init || return 1 # Prepare the mirror list. fetch_pick_server_init && fetch_pick_server # Try to fetch the public key until we run out of servers. while ! fetch_key; do fetch_pick_server || return 1 done # Try to fetch the metadata index signature ("tag") until we run # out of available servers; and sanity check the downloaded tag. while ! fetch_tag; do fetch_pick_server || return 1 done fetch_tagsanity || return 1 # Fetch the latest INDEX-NEW and INDEX-OLD files. fetch_metadata INDEX-NEW INDEX-OLD || return 1 # Generate filtered INDEX-NEW and INDEX-OLD files containing only # the lines which (a) belong to components we care about, and (b) # don't correspond to paths we're explicitly ignoring. fetch_filter_metadata INDEX-NEW || return 1 fetch_filter_metadata INDEX-OLD || return 1 # Translate /boot/${KERNCONF} into ${KERNELDIR} fetch_filter_kernel_names INDEX-NEW ${KERNCONF} fetch_filter_kernel_names INDEX-OLD ${KERNCONF} # For all paths appearing in INDEX-OLD or INDEX-NEW, inspect the # system and generate an INDEX-PRESENT file. fetch_inspect_system INDEX-OLD INDEX-PRESENT INDEX-NEW || return 1 # Based on ${UPDATEIFUNMODIFIED}, remove lines from INDEX-* which # correspond to lines in INDEX-PRESENT with hashes not appearing # in INDEX-OLD or INDEX-NEW. Also remove lines where the entry in # INDEX-PRESENT has type - and there isn't a corresponding entry in # INDEX-OLD with type -. fetch_filter_unmodified_notpresent \ INDEX-OLD INDEX-PRESENT INDEX-NEW /dev/null # For each entry in INDEX-PRESENT of type -, remove any corresponding # entry from INDEX-NEW if ${ALLOWADD} != "yes". Remove all entries # of type - from INDEX-PRESENT. fetch_filter_allowadd INDEX-PRESENT INDEX-NEW # If ${ALLOWDELETE} != "yes", then remove any entries from # INDEX-PRESENT which don't correspond to entries in INDEX-NEW. fetch_filter_allowdelete INDEX-PRESENT INDEX-NEW # If ${KEEPMODIFIEDMETADATA} == "yes", then for each entry in # INDEX-PRESENT with metadata not matching any entry in INDEX-OLD, # replace the corresponding line of INDEX-NEW with one having the # same metadata as the entry in INDEX-PRESENT. fetch_filter_modified_metadata INDEX-OLD INDEX-PRESENT INDEX-NEW # Remove lines from INDEX-PRESENT and INDEX-NEW which are identical; # no need to update a file if it isn't changing. fetch_filter_uptodate INDEX-PRESENT INDEX-NEW # Prepare to fetch files: Generate a list of the files we need, # copy the unmodified files we have into /files/, and generate # a list of patches to download. fetch_files_prepare INDEX-OLD INDEX-PRESENT INDEX-NEW || return 1 # Fetch files. fetch_files || return 1 # Create and populate install manifest directory; and report what # updates are available. fetch_create_manifest || return 1 # Warn about any upcoming EoL fetch_warn_eol || return 1 } # If StrictComponents is not "yes", generate a new components list # with only the components which appear to be installed. upgrade_guess_components () { if [ "${STRICTCOMPONENTS}" = "no" ]; then # Generate filtered INDEX-ALL with only the components listed # in COMPONENTS. fetch_filter_metadata_components $1 || return 1 # Tell the user why his disk is suddenly making lots of noise echo -n "Inspecting system... " # Look at the files on disk, and assume that a component is # supposed to be present if it is more than half-present. cut -f 1-3 -d '|' < INDEX-ALL | tr '|' ' ' | while read C S F; do if [ -e ${BASEDIR}/${F} ]; then echo "+ ${C}|${S}" fi echo "= ${C}|${S}" done | sort | uniq -c | sed -E 's,^ +,,' > compfreq grep ' = ' compfreq | cut -f 1,3 -d ' ' | sort -k 2,2 -t ' ' > compfreq.total grep ' + ' compfreq | cut -f 1,3 -d ' ' | sort -k 2,2 -t ' ' > compfreq.present join -t ' ' -1 2 -2 2 compfreq.present compfreq.total | while read S P T; do if [ ${P} -gt `expr ${T} / 2` ]; then echo ${S} fi done > comp.present cut -f 2 -d ' ' < compfreq.total > comp.total rm INDEX-ALL compfreq compfreq.total compfreq.present # We're done making noise. echo "done." # Sometimes the kernel isn't installed where INDEX-ALL # thinks that it should be: In particular, it is often in # /boot/kernel instead of /boot/GENERIC or /boot/SMP. To # deal with this, if "kernel|X" is listed in comp.total # (i.e., is a component which would be upgraded if it is # found to be present) we will add it to comp.present. # If "kernel|" is in comp.total but "kernel|X" is # not, we print a warning -- the user is running a kernel # which isn't part of the release. KCOMP=`echo ${KERNCONF} | tr 'A-Z' 'a-z'` grep -E "^kernel\|${KCOMP}\$" comp.total >> comp.present if grep -qE "^kernel\|" comp.total && ! grep -qE "^kernel\|${KCOMP}\$" comp.total; then cat <<-EOF WARNING: This system is running a "${KCOMP}" kernel, which is not a kernel configuration distributed as part of FreeBSD ${RELNUM}. This kernel will not be updated: you MUST update the kernel manually before running "$0 install". EOF fi # Re-sort the list of installed components and generate # the list of non-installed components. sort -u < comp.present > comp.present.tmp mv comp.present.tmp comp.present comm -13 comp.present comp.total > comp.absent # Ask the user to confirm that what we have is correct. To # reduce user confusion, translate "X|Y" back to "X/Y" (as # subcomponents must be listed in the configuration file). echo echo -n "The following components of FreeBSD " echo "seem to be installed:" tr '|' '/' < comp.present | fmt -72 echo echo -n "The following components of FreeBSD " echo "do not seem to be installed:" tr '|' '/' < comp.absent | fmt -72 echo continuep || return 1 echo # Suck the generated list of components into ${COMPONENTS}. # Note that comp.present.tmp is used due to issues with # pipelines and setting variables. COMPONENTS="" tr '|' '/' < comp.present > comp.present.tmp while read C; do COMPONENTS="${COMPONENTS} ${C}" done < comp.present.tmp # Delete temporary files rm comp.present comp.present.tmp comp.absent comp.total fi } # If StrictComponents is not "yes", COMPONENTS contains an entry # corresponding to the currently running kernel, and said kernel # does not exist in the new release, add "kernel/generic" to the # list of components. upgrade_guess_new_kernel () { if [ "${STRICTCOMPONENTS}" = "no" ]; then # Grab the unfiltered metadata file. METAHASH=`look "$1|" tINDEX.present | cut -f 2 -d '|'` gunzip -c < files/${METAHASH}.gz > $1.all # If "kernel/${KCOMP}" is in ${COMPONENTS} and that component # isn't in $1.all, we need to add kernel/generic. for C in ${COMPONENTS}; do if [ ${C} = "kernel/${KCOMP}" ] && ! grep -qE "^kernel\|${KCOMP}\|" $1.all; then COMPONENTS="${COMPONENTS} kernel/generic" NKERNCONF="GENERIC" cat <<-EOF WARNING: This system is running a "${KCOMP}" kernel, which is not a kernel configuration distributed as part of FreeBSD ${RELNUM}. As part of upgrading to FreeBSD ${RELNUM}, this kernel will be replaced with a "generic" kernel. EOF continuep || return 1 fi done # Don't need this any more... rm $1.all fi } # Convert INDEX-OLD (last release) and INDEX-ALL (new release) into # INDEX-OLD and INDEX-NEW files (in the sense of normal upgrades). upgrade_oldall_to_oldnew () { # For each ${F}|... which appears in INDEX-ALL but does not appear # in INDEX-OLD, add ${F}|-|||||| to INDEX-OLD. cut -f 1 -d '|' < $1 | sort -u > $1.paths cut -f 1 -d '|' < $2 | sort -u | comm -13 $1.paths - | lam - -s "|-||||||" | sort - $1 > $1.tmp mv $1.tmp $1 # Remove lines from INDEX-OLD which also appear in INDEX-ALL comm -23 $1 $2 > $1.tmp mv $1.tmp $1 # Remove lines from INDEX-ALL which have a file name not appearing # anywhere in INDEX-OLD (since these must be files which haven't # changed -- if they were new, there would be an entry of type "-"). cut -f 1 -d '|' < $1 | sort -u > $1.paths sort -k 1,1 -t '|' < $2 | join -t '|' - $1.paths | sort > $2.tmp rm $1.paths mv $2.tmp $2 # Rename INDEX-ALL to INDEX-NEW. mv $2 $3 } +# Helper for upgrade_merge: Return zero true iff the two files differ only +# in the contents of their RCS tags. +samef () { + X=`sed -E 's/\\$FreeBSD.*\\$/\$FreeBSD\$/' < $1 | ${SHA256}` + Y=`sed -E 's/\\$FreeBSD.*\\$/\$FreeBSD\$/' < $2 | ${SHA256}` + + if [ $X = $Y ]; then + return 0; + else + return 1; + fi +} + # From the list of "old" files in $1, merge changes in $2 with those in $3, # and update $3 to reflect the hashes of merged files. upgrade_merge () { # We only need to do anything if $1 is non-empty. if [ -s $1 ]; then cut -f 1 -d '|' $1 | sort > $1-paths # Create staging area for merging files rm -rf merge/ while read F; do D=`dirname ${F}` mkdir -p merge/old/${D} mkdir -p merge/${OLDRELNUM}/${D} mkdir -p merge/${RELNUM}/${D} mkdir -p merge/new/${D} done < $1-paths # Copy in files while read F; do # Currently installed file V=`look "${F}|" $2 | cut -f 7 -d '|'` gunzip < files/${V}.gz > merge/old/${F} # Old release if look "${F}|" $1 | fgrep -q "|f|"; then V=`look "${F}|" $1 | cut -f 3 -d '|'` gunzip < files/${V}.gz \ > merge/${OLDRELNUM}/${F} fi # New release if look "${F}|" $3 | cut -f 1,2,7 -d '|' | fgrep -q "|f|"; then V=`look "${F}|" $3 | cut -f 7 -d '|'` gunzip < files/${V}.gz \ > merge/${RELNUM}/${F} fi done < $1-paths # Attempt to automatically merge changes echo -n "Attempting to automatically merge " echo -n "changes in files..." : > failed.merges while read F; do # If the file doesn't exist in the new release, # the result of "merging changes" is having the file # not exist. if ! [ -f merge/${RELNUM}/${F} ]; then continue fi # If the file didn't exist in the old release, we're # going to throw away the existing file and hope that # the version from the new release is what we want. if ! [ -f merge/${OLDRELNUM}/${F} ]; then cp merge/${RELNUM}/${F} merge/new/${F} continue fi # Some files need special treatment. case ${F} in /etc/spwd.db | /etc/pwd.db | /etc/login.conf.db) # Don't merge these -- we're rebuild them # after updates are installed. cp merge/old/${F} merge/new/${F} ;; *) if ! merge -p -L "current version" \ -L "${OLDRELNUM}" -L "${RELNUM}" \ merge/old/${F} \ merge/${OLDRELNUM}/${F} \ merge/${RELNUM}/${F} \ > merge/new/${F} 2>/dev/null; then echo ${F} >> failed.merges fi ;; esac done < $1-paths echo " done." # Ask the user to handle any files which didn't merge. while read F; do + # If the installed file differs from the version in + # the old release only due to RCS tag expansion + # then just use the version in the new release. + if samef merge/old/${F} merge/${OLDRELNUM}/${F}; then + cp merge/${RELNUM}/${F} merge/new/${F} + continue + fi + cat <<-EOF The following file could not be merged automatically: ${F} Press Enter to edit this file in ${EDITOR} and resolve the conflicts manually... EOF read dummy files/${V}.gz echo "${F}|${V}" fi done < $1-paths > newhashes # Pull lines out from $3 which need to be updated to # reflect merged files. while read F; do look "${F}|" $3 done < $1-paths > $3-oldlines # Update lines to reflect merged files join -t '|' -o 1.1,1.2,1.3,1.4,1.5,1.6,2.2,1.8 \ $3-oldlines newhashes > $3-newlines # Remove old lines from $3 and add new lines. sort $3-oldlines | comm -13 - $3 | sort - $3-newlines > $3.tmp mv $3.tmp $3 # Clean up rm $1-paths newhashes $3-oldlines $3-newlines rm -rf merge/ fi # We're done with merging files. rm $1 } # Do the work involved in fetching upgrades to a new release upgrade_run () { workdir_init || return 1 # Prepare the mirror list. fetch_pick_server_init && fetch_pick_server # Try to fetch the public key until we run out of servers. while ! fetch_key; do fetch_pick_server || return 1 done # Try to fetch the metadata index signature ("tag") until we run # out of available servers; and sanity check the downloaded tag. while ! fetch_tag; do fetch_pick_server || return 1 done fetch_tagsanity || return 1 # Fetch the INDEX-OLD and INDEX-ALL. fetch_metadata INDEX-OLD INDEX-ALL || return 1 # If StrictComponents is not "yes", generate a new components list # with only the components which appear to be installed. upgrade_guess_components INDEX-ALL || return 1 # Generate filtered INDEX-OLD and INDEX-ALL files containing only # the components we want and without anything marked as "Ignore". fetch_filter_metadata INDEX-OLD || return 1 fetch_filter_metadata INDEX-ALL || return 1 # Merge the INDEX-OLD and INDEX-ALL files into INDEX-OLD. sort INDEX-OLD INDEX-ALL > INDEX-OLD.tmp mv INDEX-OLD.tmp INDEX-OLD rm INDEX-ALL # Adjust variables for fetching files from the new release. OLDRELNUM=${RELNUM} RELNUM=${TARGETRELEASE} OLDFETCHDIR=${FETCHDIR} FETCHDIR=${RELNUM}/${ARCH} # Try to fetch the NEW metadata index signature ("tag") until we run # out of available servers; and sanity check the downloaded tag. while ! fetch_tag; do fetch_pick_server || return 1 done # Fetch the new INDEX-ALL. fetch_metadata INDEX-ALL || return 1 # If StrictComponents is not "yes", COMPONENTS contains an entry # corresponding to the currently running kernel, and said kernel # does not exist in the new release, add "kernel/generic" to the # list of components. upgrade_guess_new_kernel INDEX-ALL || return 1 # Filter INDEX-ALL to contain only the components we want and without # anything marked as "Ignore". fetch_filter_metadata INDEX-ALL || return 1 # Convert INDEX-OLD (last release) and INDEX-ALL (new release) into # INDEX-OLD and INDEX-NEW files (in the sense of normal upgrades). upgrade_oldall_to_oldnew INDEX-OLD INDEX-ALL INDEX-NEW # Translate /boot/${KERNCONF} or /boot/${NKERNCONF} into ${KERNELDIR} fetch_filter_kernel_names INDEX-NEW ${NKERNCONF} fetch_filter_kernel_names INDEX-OLD ${KERNCONF} # For all paths appearing in INDEX-OLD or INDEX-NEW, inspect the # system and generate an INDEX-PRESENT file. fetch_inspect_system INDEX-OLD INDEX-PRESENT INDEX-NEW || return 1 # Based on ${MERGECHANGES}, generate a file tomerge-old with the # paths and hashes of old versions of files to merge. fetch_filter_mergechanges INDEX-OLD INDEX-PRESENT tomerge-old # Based on ${UPDATEIFUNMODIFIED}, remove lines from INDEX-* which # correspond to lines in INDEX-PRESENT with hashes not appearing # in INDEX-OLD or INDEX-NEW. Also remove lines where the entry in # INDEX-PRESENT has type - and there isn't a corresponding entry in # INDEX-OLD with type -. fetch_filter_unmodified_notpresent \ INDEX-OLD INDEX-PRESENT INDEX-NEW tomerge-old # For each entry in INDEX-PRESENT of type -, remove any corresponding # entry from INDEX-NEW if ${ALLOWADD} != "yes". Remove all entries # of type - from INDEX-PRESENT. fetch_filter_allowadd INDEX-PRESENT INDEX-NEW # If ${ALLOWDELETE} != "yes", then remove any entries from # INDEX-PRESENT which don't correspond to entries in INDEX-NEW. fetch_filter_allowdelete INDEX-PRESENT INDEX-NEW # If ${KEEPMODIFIEDMETADATA} == "yes", then for each entry in # INDEX-PRESENT with metadata not matching any entry in INDEX-OLD, # replace the corresponding line of INDEX-NEW with one having the # same metadata as the entry in INDEX-PRESENT. fetch_filter_modified_metadata INDEX-OLD INDEX-PRESENT INDEX-NEW # Remove lines from INDEX-PRESENT and INDEX-NEW which are identical; # no need to update a file if it isn't changing. fetch_filter_uptodate INDEX-PRESENT INDEX-NEW # Fetch "clean" files from the old release for merging changes. fetch_files_premerge tomerge-old # Prepare to fetch files: Generate a list of the files we need, # copy the unmodified files we have into /files/, and generate # a list of patches to download. fetch_files_prepare INDEX-OLD INDEX-PRESENT INDEX-NEW || return 1 # Fetch patches from to-${RELNUM}/${ARCH}/bp/ PATCHDIR=to-${RELNUM}/${ARCH}/bp fetch_files || return 1 # Merge configuration file changes. upgrade_merge tomerge-old INDEX-PRESENT INDEX-NEW || return 1 # Create and populate install manifest directory; and report what # updates are available. fetch_create_manifest || return 1 # Leave a note behind to tell the "install" command that the kernel # needs to be installed before the world. touch ${BDHASH}-install/kernelfirst + + # Remind the user that they need to run "freebsd-update install" + # to install the downloaded bits, in case they didn't RTFM. + echo "To install the downloaded upgrades, run \"$0 install\"." } # Make sure that all the file hashes mentioned in $@ have corresponding # gzipped files stored in /files/. install_verify () { # Generate a list of hashes cat $@ | cut -f 2,7 -d '|' | grep -E '^f' | cut -f 2 -d '|' | sort -u > filelist # Make sure all the hashes exist while read HASH; do if ! [ -f files/${HASH}.gz ]; then echo -n "Update files missing -- " echo "this should never happen." echo "Re-run '$0 fetch'." return 1 fi done < filelist # Clean up rm filelist } # Remove the system immutable flag from files install_unschg () { # Generate file list cat $@ | cut -f 1 -d '|' > filelist # Remove flags while read F; do if ! [ -e ${BASEDIR}/${F} ]; then continue fi chflags noschg ${BASEDIR}/${F} || return 1 done < filelist # Clean up rm filelist } # Decide which directory name to use for kernel backups. backup_kernel_finddir () { CNT=0 while true ; do # Pathname does not exist, so it is OK use that name # for backup directory. - if [ ! -e $BACKUPKERNELDIR ]; then + if [ ! -e $BASEDIR/$BACKUPKERNELDIR ]; then return 0 fi # If directory do exist, we only use if it has our # marker file. - if [ -d $BACKUPKERNELDIR -a \ - -e $BACKUPKERNELDIR/.freebsd-update ]; then + if [ -d $BASEDIR/$BACKUPKERNELDIR -a \ + -e $BASEDIR/$BACKUPKERNELDIR/.freebsd-update ]; then return 0 fi # We could not use current directory name, so add counter to # the end and try again. CNT=$((CNT + 1)) if [ $CNT -gt 9 ]; then - echo "Could not find valid backup dir ($BACKUPKERNELDIR)" + echo "Could not find valid backup dir ($BASEDIR/$BACKUPKERNELDIR)" exit 1 fi BACKUPKERNELDIR="`echo $BACKUPKERNELDIR | sed -Ee 's/[0-9]\$//'`" BACKUPKERNELDIR="${BACKUPKERNELDIR}${CNT}" done } # Backup the current kernel using hardlinks, if not disabled by user. # Since we delete all files in the directory used for previous backups # we create a marker file called ".freebsd-update" in the directory so # we can determine on the next run that the directory was created by # freebsd-update and we then do not accidentally remove user files in # the unlikely case that the user has created a directory with a # conflicting name. backup_kernel () { # Only make kernel backup is so configured. if [ $BACKUPKERNEL != yes ]; then return 0 fi # Decide which directory name to use for kernel backups. backup_kernel_finddir # Remove old kernel backup files. If $BACKUPKERNELDIR was # "not ours", backup_kernel_finddir would have exited, so # deleting the directory content is as safe as we can make it. - if [ -d $BACKUPKERNELDIR ]; then - rm -fr $BACKUPKERNELDIR + if [ -d $BASEDIR/$BACKUPKERNELDIR ]; then + rm -fr $BASEDIR/$BACKUPKERNELDIR fi # Create directories for backup. - mkdir -p $BACKUPKERNELDIR - mtree -cdn -p "${KERNELDIR}" | \ - mtree -Ue -p "${BACKUPKERNELDIR}" > /dev/null + mkdir -p $BASEDIR/$BACKUPKERNELDIR + mtree -cdn -p "${BASEDIR}/${KERNELDIR}" | \ + mtree -Ue -p "${BASEDIR}/${BACKUPKERNELDIR}" > /dev/null # Mark the directory as having been created by freebsd-update. - touch $BACKUPKERNELDIR/.freebsd-update + touch $BASEDIR/$BACKUPKERNELDIR/.freebsd-update if [ $? -ne 0 ]; then echo "Could not create kernel backup directory" exit 1 fi # Disable pathname expansion to be sure *.symbols is not # expanded. set -f # Use find to ignore symbol files, unless disabled by user. if [ $BACKUPKERNELSYMBOLFILES = yes ]; then FINDFILTER="" else FINDFILTER=-"a ! -name *.symbols" fi # Backup all the kernel files using hardlinks. - (cd $KERNELDIR && find . -type f $FINDFILTER -exec \ - cp -pl '{}' ${BACKUPKERNELDIR}/'{}' \;) + (cd ${BASEDIR}/${KERNELDIR} && find . -type f $FINDFILTER -exec \ + cp -pl '{}' ${BASEDIR}/${BACKUPKERNELDIR}/'{}' \;) # Re-enable patchname expansion. set +f } # Install new files install_from_index () { # First pass: Do everything apart from setting file flags. We # can't set flags yet, because schg inhibits hard linking. sort -k 1,1 -t '|' $1 | tr '|' ' ' | while read FPATH TYPE OWNER GROUP PERM FLAGS HASH LINK; do case ${TYPE} in d) # Create a directory install -d -o ${OWNER} -g ${GROUP} \ -m ${PERM} ${BASEDIR}/${FPATH} ;; f) if [ -z "${LINK}" ]; then # Create a file, without setting flags. gunzip < files/${HASH}.gz > ${HASH} install -S -o ${OWNER} -g ${GROUP} \ -m ${PERM} ${HASH} ${BASEDIR}/${FPATH} rm ${HASH} else # Create a hard link. ln -f ${BASEDIR}/${LINK} ${BASEDIR}/${FPATH} fi ;; L) # Create a symlink ln -sfh ${HASH} ${BASEDIR}/${FPATH} ;; esac done # Perform a second pass, adding file flags. tr '|' ' ' < $1 | while read FPATH TYPE OWNER GROUP PERM FLAGS HASH LINK; do if [ ${TYPE} = "f" ] && ! [ ${FLAGS} = "0" ]; then chflags ${FLAGS} ${BASEDIR}/${FPATH} fi done } # Remove files which we want to delete install_delete () { # Generate list of new files cut -f 1 -d '|' < $2 | sort > newfiles # Generate subindex of old files we want to nuke sort -k 1,1 -t '|' $1 | join -t '|' -v 1 - newfiles | sort -r -k 1,1 -t '|' | cut -f 1,2 -d '|' | tr '|' ' ' > killfiles # Remove the offending bits while read FPATH TYPE; do case ${TYPE} in d) rmdir ${BASEDIR}/${FPATH} ;; f) rm ${BASEDIR}/${FPATH} ;; L) rm ${BASEDIR}/${FPATH} ;; esac done < killfiles # Clean up rm newfiles killfiles } # Install new files, delete old files, and update linker.hints install_files () { # If we haven't already dealt with the kernel, deal with it. if ! [ -f $1/kerneldone ]; then grep -E '^/boot/' $1/INDEX-OLD > INDEX-OLD grep -E '^/boot/' $1/INDEX-NEW > INDEX-NEW # Backup current kernel before installing a new one backup_kernel || return 1 # Install new files install_from_index INDEX-NEW || return 1 # Remove files which need to be deleted install_delete INDEX-OLD INDEX-NEW || return 1 # Update linker.hints if necessary if [ -s INDEX-OLD -o -s INDEX-NEW ]; then - kldxref -R /boot/ 2>/dev/null + kldxref -R ${BASEDIR}/boot/ 2>/dev/null fi # We've finished updating the kernel. touch $1/kerneldone # Do we need to ask for a reboot now? if [ -f $1/kernelfirst ] && [ -s INDEX-OLD -o -s INDEX-NEW ]; then cat <<-EOF Kernel updates have been installed. Please reboot and run "$0 install" again to finish installing updates. EOF exit 0 fi fi # If we haven't already dealt with the world, deal with it. if ! [ -f $1/worlddone ]; then # Create any necessary directories first grep -vE '^/boot/' $1/INDEX-NEW | grep -E '^[^|]+\|d\|' > INDEX-NEW install_from_index INDEX-NEW || return 1 # Install new runtime linker grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '^[^|]+\|d\|' | grep -E '^/libexec/ld-elf[^|]*\.so\.[0-9]+\|' > INDEX-NEW install_from_index INDEX-NEW || return 1 # Install new shared libraries next grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '^[^|]+\|d\|' | grep -vE '^/libexec/ld-elf[^|]*\.so\.[0-9]+\|' | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-NEW install_from_index INDEX-NEW || return 1 # Deal with everything else grep -vE '^/boot/' $1/INDEX-OLD | grep -vE '^[^|]+\|d\|' | grep -vE '^/libexec/ld-elf[^|]*\.so\.[0-9]+\|' | grep -vE '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-OLD grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '^[^|]+\|d\|' | grep -vE '^/libexec/ld-elf[^|]*\.so\.[0-9]+\|' | grep -vE '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-NEW install_from_index INDEX-NEW || return 1 install_delete INDEX-OLD INDEX-NEW || return 1 # Rebuild /etc/spwd.db and /etc/pwd.db if necessary. - if [ /etc/master.passwd -nt /etc/spwd.db ] || - [ /etc/master.passwd -nt /etc/pwd.db ]; then - pwd_mkdb /etc/master.passwd + if [ ${BASEDIR}/etc/master.passwd -nt ${BASEDIR}/etc/spwd.db ] || + [ ${BASEDIR}/etc/master.passwd -nt ${BASEDIR}/etc/pwd.db ]; then + pwd_mkdb -d ${BASEDIR}/etc ${BASEDIR}/etc/master.passwd fi # Rebuild /etc/login.conf.db if necessary. - if [ /etc/login.conf -nt /etc/login.conf.db ]; then - cap_mkdb /etc/login.conf + if [ ${BASEDIR}/etc/login.conf -nt ${BASEDIR}/etc/login.conf.db ]; then + cap_mkdb ${BASEDIR}/etc/login.conf fi # We've finished installing the world and deleting old files # which are not shared libraries. touch $1/worlddone # Do we need to ask the user to portupgrade now? grep -vE '^/boot/' $1/INDEX-NEW | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' | cut -f 1 -d '|' | sort > newfiles if grep -vE '^/boot/' $1/INDEX-OLD | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' | cut -f 1 -d '|' | sort | join -v 1 - newfiles | grep -q .; then cat <<-EOF Completing this upgrade requires removing old shared object files. Please rebuild all installed 3rd party software (e.g., programs installed from the ports tree) and then run "$0 install" again to finish installing updates. EOF rm newfiles exit 0 fi rm newfiles fi # Remove old shared libraries grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '^[^|]+\|d\|' | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-NEW grep -vE '^/boot/' $1/INDEX-OLD | grep -vE '^[^|]+\|d\|' | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-OLD install_delete INDEX-OLD INDEX-NEW || return 1 # Remove old directories grep -vE '^/boot/' $1/INDEX-NEW | grep -E '^[^|]+\|d\|' > INDEX-NEW grep -vE '^/boot/' $1/INDEX-OLD | grep -E '^[^|]+\|d\|' > INDEX-OLD install_delete INDEX-OLD INDEX-NEW || return 1 # Remove temporary files rm INDEX-OLD INDEX-NEW } # Rearrange bits to allow the installed updates to be rolled back install_setup_rollback () { # Remove the "reboot after installing kernel", "kernel updated", and # "finished installing the world" flags if present -- they are # irrelevant when rolling back updates. if [ -f ${BDHASH}-install/kernelfirst ]; then rm ${BDHASH}-install/kernelfirst rm ${BDHASH}-install/kerneldone fi if [ -f ${BDHASH}-install/worlddone ]; then rm ${BDHASH}-install/worlddone fi if [ -L ${BDHASH}-rollback ]; then mv ${BDHASH}-rollback ${BDHASH}-install/rollback fi mv ${BDHASH}-install ${BDHASH}-rollback } # Actually install updates install_run () { echo -n "Installing updates..." # Make sure we have all the files we should have install_verify ${BDHASH}-install/INDEX-OLD \ ${BDHASH}-install/INDEX-NEW || return 1 # Remove system immutable flag from files install_unschg ${BDHASH}-install/INDEX-OLD \ ${BDHASH}-install/INDEX-NEW || return 1 # Install new files, delete old files, and update linker.hints install_files ${BDHASH}-install || return 1 # Rearrange bits to allow the installed updates to be rolled back install_setup_rollback echo " done." } # Rearrange bits to allow the previous set of updates to be rolled back next. rollback_setup_rollback () { if [ -L ${BDHASH}-rollback/rollback ]; then mv ${BDHASH}-rollback/rollback rollback-tmp rm -r ${BDHASH}-rollback/ rm ${BDHASH}-rollback mv rollback-tmp ${BDHASH}-rollback else rm -r ${BDHASH}-rollback/ rm ${BDHASH}-rollback fi } # Install old files, delete new files, and update linker.hints rollback_files () { # Install old shared library files which don't have the same path as # a new shared library file. grep -vE '^/boot/' $1/INDEX-NEW | grep -E '/lib/.*\.so\.[0-9]+\|' | cut -f 1 -d '|' | sort > INDEX-NEW.libs.flist grep -vE '^/boot/' $1/INDEX-OLD | grep -E '/lib/.*\.so\.[0-9]+\|' | sort -k 1,1 -t '|' - | join -t '|' -v 1 - INDEX-NEW.libs.flist > INDEX-OLD install_from_index INDEX-OLD || return 1 # Deal with files which are neither kernel nor shared library grep -vE '^/boot/' $1/INDEX-OLD | grep -vE '/lib/.*\.so\.[0-9]+\|' > INDEX-OLD grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '/lib/.*\.so\.[0-9]+\|' > INDEX-NEW install_from_index INDEX-OLD || return 1 install_delete INDEX-NEW INDEX-OLD || return 1 # Install any old shared library files which we didn't install above. grep -vE '^/boot/' $1/INDEX-OLD | grep -E '/lib/.*\.so\.[0-9]+\|' | sort -k 1,1 -t '|' - | join -t '|' - INDEX-NEW.libs.flist > INDEX-OLD install_from_index INDEX-OLD || return 1 # Delete unneeded shared library files grep -vE '^/boot/' $1/INDEX-OLD | grep -E '/lib/.*\.so\.[0-9]+\|' > INDEX-OLD grep -vE '^/boot/' $1/INDEX-NEW | grep -E '/lib/.*\.so\.[0-9]+\|' > INDEX-NEW install_delete INDEX-NEW INDEX-OLD || return 1 # Deal with kernel files grep -E '^/boot/' $1/INDEX-OLD > INDEX-OLD grep -E '^/boot/' $1/INDEX-NEW > INDEX-NEW install_from_index INDEX-OLD || return 1 install_delete INDEX-NEW INDEX-OLD || return 1 if [ -s INDEX-OLD -o -s INDEX-NEW ]; then kldxref -R /boot/ 2>/dev/null fi # Remove temporary files rm INDEX-OLD INDEX-NEW INDEX-NEW.libs.flist } # Actually rollback updates rollback_run () { echo -n "Uninstalling updates..." # If there are updates waiting to be installed, remove them; we # want the user to re-run 'fetch' after rolling back updates. if [ -L ${BDHASH}-install ]; then rm -r ${BDHASH}-install/ rm ${BDHASH}-install fi # Make sure we have all the files we should have install_verify ${BDHASH}-rollback/INDEX-NEW \ ${BDHASH}-rollback/INDEX-OLD || return 1 # Remove system immutable flag from files install_unschg ${BDHASH}-rollback/INDEX-NEW \ ${BDHASH}-rollback/INDEX-OLD || return 1 # Install old files, delete new files, and update linker.hints rollback_files ${BDHASH}-rollback || return 1 # Remove the rollback directory and the symlink pointing to it; and # rearrange bits to allow the previous set of updates to be rolled # back next. rollback_setup_rollback echo " done." } # Compare INDEX-ALL and INDEX-PRESENT and print warnings about differences. IDS_compare () { # Get all the lines which mismatch in something other than file # flags. We ignore file flags because sysinstall doesn't seem to # set them when it installs FreeBSD; warning about these adds a # very large amount of noise. cut -f 1-5,7-8 -d '|' $1 > $1.noflags sort -k 1,1 -t '|' $1.noflags > $1.sorted cut -f 1-5,7-8 -d '|' $2 | comm -13 $1.noflags - | fgrep -v '|-|||||' | sort -k 1,1 -t '|' | join -t '|' $1.sorted - > INDEX-NOTMATCHING # Ignore files which match IDSIGNOREPATHS. for X in ${IDSIGNOREPATHS}; do grep -E "^${X}" INDEX-NOTMATCHING done | sort -u | comm -13 - INDEX-NOTMATCHING > INDEX-NOTMATCHING.tmp mv INDEX-NOTMATCHING.tmp INDEX-NOTMATCHING # Go through the lines and print warnings. - while read LINE; do - FPATH=`echo "${LINE}" | cut -f 1 -d '|'` - TYPE=`echo "${LINE}" | cut -f 2 -d '|'` - OWNER=`echo "${LINE}" | cut -f 3 -d '|'` - GROUP=`echo "${LINE}" | cut -f 4 -d '|'` - PERM=`echo "${LINE}" | cut -f 5 -d '|'` - HASH=`echo "${LINE}" | cut -f 6 -d '|'` - LINK=`echo "${LINE}" | cut -f 7 -d '|'` - P_TYPE=`echo "${LINE}" | cut -f 8 -d '|'` - P_OWNER=`echo "${LINE}" | cut -f 9 -d '|'` - P_GROUP=`echo "${LINE}" | cut -f 10 -d '|'` - P_PERM=`echo "${LINE}" | cut -f 11 -d '|'` - P_HASH=`echo "${LINE}" | cut -f 12 -d '|'` - P_LINK=`echo "${LINE}" | cut -f 13 -d '|'` - + local IFS='|' + while read FPATH TYPE OWNER GROUP PERM HASH LINK P_TYPE P_OWNER P_GROUP P_PERM P_HASH P_LINK; do # Warn about different object types. if ! [ "${TYPE}" = "${P_TYPE}" ]; then echo -n "${FPATH} is a " case "${P_TYPE}" in f) echo -n "regular file, " ;; d) echo -n "directory, " ;; L) echo -n "symlink, " ;; esac echo -n "but should be a " case "${TYPE}" in f) echo -n "regular file." ;; d) echo -n "directory." ;; L) echo -n "symlink." ;; esac echo # Skip other tests, since they don't make sense if # we're comparing different object types. continue fi # Warn about different owners. if ! [ "${OWNER}" = "${P_OWNER}" ]; then echo -n "${FPATH} is owned by user id ${P_OWNER}, " echo "but should be owned by user id ${OWNER}." fi # Warn about different groups. if ! [ "${GROUP}" = "${P_GROUP}" ]; then echo -n "${FPATH} is owned by group id ${P_GROUP}, " echo "but should be owned by group id ${GROUP}." fi # Warn about different permissions. We do not warn about # different permissions on symlinks, since some archivers # don't extract symlink permissions correctly and they are # ignored anyway. if ! [ "${PERM}" = "${P_PERM}" ] && ! [ "${TYPE}" = "L" ]; then echo -n "${FPATH} has ${P_PERM} permissions, " echo "but should have ${PERM} permissions." fi # Warn about different file hashes / symlink destinations. if ! [ "${HASH}" = "${P_HASH}" ]; then if [ "${TYPE}" = "L" ]; then echo -n "${FPATH} is a symlink to ${P_HASH}, " echo "but should be a symlink to ${HASH}." fi if [ "${TYPE}" = "f" ]; then echo -n "${FPATH} has SHA256 hash ${P_HASH}, " echo "but should have SHA256 hash ${HASH}." fi fi # We don't warn about different hard links, since some # some archivers break hard links, and as long as the # underlying data is correct they really don't matter. done < INDEX-NOTMATCHING # Clean up rm $1 $1.noflags $1.sorted $2 INDEX-NOTMATCHING } # Do the work involved in comparing the system to a "known good" index IDS_run () { workdir_init || return 1 # Prepare the mirror list. fetch_pick_server_init && fetch_pick_server # Try to fetch the public key until we run out of servers. while ! fetch_key; do fetch_pick_server || return 1 done # Try to fetch the metadata index signature ("tag") until we run # out of available servers; and sanity check the downloaded tag. while ! fetch_tag; do fetch_pick_server || return 1 done fetch_tagsanity || return 1 # Fetch INDEX-OLD and INDEX-ALL. fetch_metadata INDEX-OLD INDEX-ALL || return 1 # Generate filtered INDEX-OLD and INDEX-ALL files containing only # the components we want and without anything marked as "Ignore". fetch_filter_metadata INDEX-OLD || return 1 fetch_filter_metadata INDEX-ALL || return 1 # Merge the INDEX-OLD and INDEX-ALL files into INDEX-ALL. sort INDEX-OLD INDEX-ALL > INDEX-ALL.tmp mv INDEX-ALL.tmp INDEX-ALL rm INDEX-OLD # Translate /boot/${KERNCONF} to ${KERNELDIR} fetch_filter_kernel_names INDEX-ALL ${KERNCONF} # Inspect the system and generate an INDEX-PRESENT file. fetch_inspect_system INDEX-ALL INDEX-PRESENT /dev/null || return 1 # Compare INDEX-ALL and INDEX-PRESENT and print warnings about any # differences. IDS_compare INDEX-ALL INDEX-PRESENT } #### Main functions -- call parameter-handling and core functions # Using the command line, configuration file, and defaults, # set all the parameters which are needed later. get_params () { init_params parse_cmdline $@ parse_conffile default_params } # Fetch command. Make sure that we're being called # interactively, then run fetch_check_params and fetch_run cmd_fetch () { - if [ ! -t 0 ]; then + if [ ! -t 0 -a $NOTTYOK -eq 0 ]; then echo -n "`basename $0` fetch should not " echo "be run non-interactively." echo "Run `basename $0` cron instead." exit 1 fi fetch_check_params fetch_run || exit 1 } # Cron command. Make sure the parameters are sensible; wait # rand(3600) seconds; then fetch updates. While fetching updates, # send output to a temporary file; only print that file if the # fetching failed. cmd_cron () { fetch_check_params sleep `jot -r 1 0 3600` TMPFILE=`mktemp /tmp/freebsd-update.XXXXXX` || exit 1 if ! fetch_run >> ${TMPFILE} || ! grep -q "No updates needed" ${TMPFILE} || [ ${VERBOSELEVEL} = "debug" ]; then mail -s "`hostname` security updates" ${MAILTO} < ${TMPFILE} fi rm ${TMPFILE} } # Fetch files for upgrading to a new release. cmd_upgrade () { upgrade_check_params upgrade_run || exit 1 } # Install downloaded updates. cmd_install () { install_check_params install_run || exit 1 } # Rollback most recently installed updates. cmd_rollback () { rollback_check_params rollback_run || exit 1 } # Compare system against a "known good" index. cmd_IDS () { IDS_check_params IDS_run || exit 1 } #### Entry point # Make sure we find utilities from the base system export PATH=/sbin:/bin:/usr/sbin:/usr/bin:${PATH} # Set a pager if the user doesn't if [ -z "$PAGER" ]; then PAGER=/usr/bin/more fi # Set LC_ALL in order to avoid problems with character ranges like [A-Z]. export LC_ALL=C get_params $@ for COMMAND in ${COMMANDS}; do cmd_${COMMAND} done Index: releng/9.3/UPDATING =================================================================== --- releng/9.3/UPDATING (revision 282873) +++ releng/9.3/UPDATING (revision 282874) @@ -1,1782 +1,1787 @@ Updating Information for FreeBSD current users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. These instructions assume that you basically know what you are doing. If not, then please consult the FreeBSD handbook: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. +20150513: p14 FreeBSD-EN-15:04.freebsd-update + + Fix bug with freebsd-update(8) that does not ensure the previous + upgrade was completed. [EN-15:04] + 20150407: p13 FreeBSD-SA-15:04.igmp [revised] FreeBSD-SA-15:07.ntp FreeBSD-SA-15:09.ipv6 Improved patch for SA-15:04.igmp. Fix multiple vulnerabilities of ntp. [SA-15:07] Fix Denial of Service with IPv6 Router Advertisements. [SA-15:09] 20150320: p12 Fix patch for SA-15:06.openssl. 20150319: p11 FreeBSD-SA-15:06.openssl Fix multiple vulnerabilities in OpenSSL. [SA-15:06] 20150225: p10 FreeBSD-SA-15:04.igmp FreeBSD-SA-15:05.bind FreeBSD-EN-15:01.vt FreeBSD-EN-15:02.openssl FreeBSD-EN-15:03.freebsd-update Fix integer overflow in IGMP protocol. [SA-15:04] Fix BIND remote denial of service vulnerability. [SA-15:05] Fix vt(4) crash with improper ioctl parameters. [EN-15:01] Updated base system OpenSSL to 0.9.8zd. [EN-15:02] Fix freebsd-update libraries update ordering issue. [EN-15:03] 20150127: p9 FreeBSD-SA-15:02.kmem FreeBSD-SA-15:03.sctp Fix SCTP SCTP_SS_VALUE kernel memory corruption and disclosure vulnerability. [SA-15:02] Fix SCTP stream reset vulnerability. [SA-15:03] 20150114: p8 FreeBSD-SA-15:01.openssl Fix multiple vulnerabilities in OpenSSL. [SA-15:01] 20141223: p7 FreeBSD-SA-14:31.ntp FreeBSD-EN-14:13.freebsd-update Fix multiple vulnerabilities in NTP suite. [SA-14:31] Fix directory deletion issue in freebsd-update. [EN-14:13] 20141210: p6 FreeBSD-SA-14:28.file FreeBSD-SA-14:29.bind Fix multiple vulnerabilities in file(1) and libmagic(3). [SA-14:28] Fix BIND remote denial of service vulnerability. [SA-14:29] 20141104: p5 FreeBSD-SA-14:25.setlogin FreeBSD-SA-14:26.ftp FreeBSD-EN-14:12.zfs Fix kernel stack disclosure in setlogin(2) / getlogin(2). [SA-14:25] Fix remote command execution in ftp(1). [SA-14:26] Fix NFSv4 and ZFS cache consistency issue. [EN-14:12] 20141022: p4 FreeBSD-EN-14:10.tzdata FreeBSD-EN-14:11.crypt Time zone data file update. [EN-14:10] Change crypt(3) default hashing algorithm back to DES. [EN-14:11] 20141021: p3 FreeBSD-SA-14:20.rtsold FreeBSD-SA-14:21.routed FreeBSD-SA-14:22.namei FreeBSD-SA-14:23.openssl Fix rtsold(8) remote buffer overflow vulnerability. [SA-14:20] Fix routed(8) remote denial of service vulnerability. [SA-14:21] Fix memory leak in sandboxed namei lookup. [SA-14:22] Fix OpenSSL multiple vulnerabilities. [SA-14:23] 20140916: p2 FreeBSD-SA-14:19.tcp Fix Denial of Service in TCP packet processing. [SA-14:19] 20140909: p1 FreeBSD-SA-14:18.openssl Fix OpenSSL multiple vulnerabilities. [SA-14:18] 20140716: 9.3-RELEASE. 20140608: On i386 and amd64 systems, the onifconsole flag is now set by default in /etc/ttys for ttyu0. This causes ttyu0 to be automatically enabled as a login TTY if it is set in the bootloader as an active kernel console. No changes in behavior should result otherwise. To revert to the previous behavior, set ttyu0 to "off" in /etc/ttys. 20140512: Clang and llvm have been upgraded to 3.4.1 release. 20140321: Clang and llvm have been upgraded to 3.4 release. 20140216: The nve(4) driver for NVIDIA nForce MCP Ethernet adapters has been deprecated and will not be part of FreeBSD 11.0 and later releases. If you use this driver, please consider switching to the nfe(4) driver instead. 20131216: The behavior of gss_pseudo_random() for the krb5 mechanism has changed, for applications requesting a longer random string than produced by the underlying enctype's pseudo-random() function. In particular, the random string produced from a session key of enctype aes256-cts-hmac-sha1-96 or aes256-cts-hmac-sha1-96 will be different at the 17th octet and later, after this change. The counter used in the PRF+ construction is now encoded as a big-endian integer in accordance with RFC 4402. __FreeBSD_version is bumped to 902505. 20130930: 9.2-RELEASE. 20130823: Behavior of devfs rules path matching has been changed. Pattern is now always matched against fully qualified devfs path and slash characters must be explicitly matched by slashes in pattern (FNM_PATHNAME). Rulesets involving devfs subdirectories must be reviewed. 20130705: hastctl(8)'s `status' command output changed to terse one-liner format. Scripts using this should switch to `list' command or be rewritten. 20130618: Fix a bug that allowed a tracing process (e.g. gdb) to write to a memory-mapped file in the traced process's address space even if neither the traced process nor the tracing process had write access to that file. 20130605: Added ZFS TRIM support which is enabled by default. To disable ZFS TRIM support set vfs.zfs.trim.enabled=0 in loader.conf. Creating new ZFS pools and adding new devices to existing pools first performs a full device level TRIM which can take a significant amount of time. The sysctl vfs.zfs.vdev.trim_on_init can be set to 0 to disable this behaviour. ZFS TRIM requires the underlying device support BIO_DELETE which is currently provided by methods such as ATA TRIM and SCSI UNMAP via CAM, which are typically supported by SSD's. Stats for ZFS TRIM can be monitored by looking at the sysctl's under kstat.zfs.misc.zio_trim. 20130524: `list' command has been added to hastctl(8). For now, it is full equivalent of `status' command. WARNING: in the near future the output of hastctl's status command will change to more terse format. If you use `hastctl status' for parsing in your scripts, switch to `hastctl list'. 20130430: The mergemaster command now uses the default MAKEOBJDIRPREFIX rather than creating it's own in the temporary directory in order allow access to bootstrapped versions of tools such as install and mtree. When upgrading from version of FreeBSD where the install command does not support -l, you will need to install a new mergemaster command if mergemaster -p is required. This can be accomplished with the command (cd src/usr.sbin/mergemaster && make install). Due to the use of the new -l option to install(1) during build and install, you must take care not to directly set the INSTALL make variable in your /etc/make.conf, /etc/src.conf, or on the command line. If you wish to use the -C flag for all installs you may be able to add INSTALL+=-C to /etc/make.conf or /etc/src.conf. 20130429: Fix a bug that allows NFS clients to issue READDIR on files. 20130315: The install(1) option -M has changed meaning and now takes an argument that is a file or path to append logs to. In the unlikely event that -M was the last option on the command line and the command line contained at least two files and a target directory the first file will have logs appended to it. The -M option served little practical purpose in the last decade so it's used expected to be extremely rare. 20130225: A new compression method (lz4) has been merged to. Please refer to zpool-features(7) for more information. Please refer to the "ZFS notes" section of this file for information on upgrading boot ZFS pools. 20121224: The VFS KBI was changed with the merge of several nullfs optimizations and fixes. All filesystem modules must be recompiled. 20121218: With the addition of auditdistd(8), a new auditdistd user is now depended on during installworld. "mergemaster -p" can be used to add the user prior to installworld, as documented in the handbook. 20121205: 9.1-RELEASE. 20121129: A new version of ZFS (pool version 5000) has been merged to 9-STABLE. Starting with this version the old system of ZFS pool versioning is superseded by "feature flags". This concept enables forward compatibility against certain future changes in functionality of ZFS pools. The first two read-only compatible "feature flags" for ZFS pools are "com.delphix:async_destroy" and "com.delphix:empty_bpobj". For more information read the new zpool-features(7) manual page. Please refer to the "ZFS notes" section of this file for information on upgrading boot ZFS pools. 20121114: The commit introducing bsd.compiler.mk breaks the traditional building of kernels before this point. Add -m ${SRC}/share/mk (for the right value of SRC) to your command lines to work around; update your useland to a point after this; or use the buildkernel/installkernel top-level targets. See also 20120829. 20121102: The IPFIREWALL_FORWARD kernel option has been removed. Its functionality now turned on by default. 20120913: The random(4) support for the VIA hardware random number generator (`PADLOCK') is no longer enabled unconditionally. Add the PADLOCK_RNG option in the custom kernel config if needed. The GENERIC kernels on i386 and amd64 do include the option, so the change only affects the custom kernel configurations. 20120829: The amd64 kernel now uses xsetbv, xrstor instructions. To compile with the traditional method, you must update your system with an installworld before the kernel will build. The documented make buildkernel/installkernel interfaces (coupled with fresh make kernel-toolchain) continue to work. 20120727: The sparc64 ZFS loader has been changed to no longer try to auto- detect ZFS providers based on diskN aliases but now requires these to be explicitly listed in the OFW boot-device environment variable. 20120422: Now unix domain sockets behave "as expected" on nullfs(5). Previously nullfs(5) did not pass through all behaviours to the underlying layer, as a result if we bound to a socket on the lower layer we could connect only to the lower path; if we bound to the upper layer we could connect only to the upper path. The new behavior is one can connect to both the lower and the upper paths regardless what layer path one binds to. 20120109: The acpi_wmi(4) status device /dev/wmistat has been renamed to /dev/wmistat0. 20120106: A new VOP_ADVISE() was added to support posix_fadvise(2). All filesystem modules must be recompiled. 20120106: The interface of the VOP_VPTOCNP(9) changed, now the returned vnode shall be referenced, previously it was required to be only held. All in-tree filesystems are converted. 20120106: 9.0-RELEASE. 20111101: The broken amd(4) driver has been replaced with esp(4) in the amd64, i386 and pc98 GENERIC kernel configuration files. 20110913: This commit modifies vfs_register() so that it uses a hash calculation to set vfc_typenum, which is enabled by default. The first time a system is booted after this change, the vfc_typenum values will change for all file systems. The main effect of this is a change to the NFS server file handles for file systems that use vfc_typenum in their fsid, such as ZFS. It will, however, prevent vfc_typenum from changing when file systems are loaded in a different order for subsequent reboots. To disable this, you can set vfs.typenumhash=0 in /boot/loader.conf until you are ready to remount all NFS clients after a reboot. 20110828: Bump the shared library version numbers for libraries that do not use symbol versioning, have changed the ABI compared to stable/8 and which shared library version was not bumped. Done as part of 9.0-RELEASE cycle. 20110815: During the merge of Capsicum features, the fget(9) KPI was modified. This may require the rebuilding of out-of-tree device drivers -- issues have been reported specifically with the nVidia device driver. __FreeBSD_version is bumped to 900041. Also, there is a period between 20110811 and 20110814 where the special devices /dev/{stdin,stdout,stderr} did not work correctly. Building world from a kernel during that window may not work. 20110628: The packet filter (pf) code has been updated to OpenBSD 4.5. You need to update userland tools to be in sync with kernel. This update breaks backward compatibility with earlier pfsync(4) versions. Care must be taken when updating redundant firewall setups. 20110608: The following sysctls and tunables are retired on x86 platforms: machdep.hlt_cpus machdep.hlt_logical_cpus The following sysctl is retired: machdep.hyperthreading_allowed The sysctls were supposed to provide a way to dynamically offline and online selected CPUs on x86 platforms, but the implementation has not been reliable especially with SCHED_ULE scheduler. machdep.hyperthreading_allowed tunable is still available to ignore hyperthreading CPUs at OS level. Individual CPUs can be disabled using hint.lapic.X.disabled tunable, where X is an APIC ID of a CPU. Be advised, though, that disabling CPUs in non-uniform fashion will result in non-uniform topology and may lead to sub-optimal system performance with SCHED_ULE, which is a default scheduler. 20110607: cpumask_t type is retired and cpuset_t is used in order to describe a mask of CPUs. 20110531: Changes to ifconfig(8) for dynamic address family detection mandate that you are running a kernel of 20110525 or later. Make sure to follow the update procedure to boot a new kernel before installing world. 20110513: Support for sun4v architecture is officially dropped 20110503: Several KPI breaking changes have been committed to the mii(4) layer, the PHY drivers and consequently some Ethernet drivers using mii(4). This means that miibus.ko and the modules of the affected Ethernet drivers need to be recompiled. Note to kernel developers: Given that the OUI bit reversion problem was fixed as part of these changes all mii(4) commits related to OUIs, i.e. to sys/dev/mii/miidevs, PHY driver probing and vendor specific handling, no longer can be merged verbatim to stable/8 and previous branches. 20110430: Users of the Atheros AR71xx SoC code now need to add 'device ar71xx_pci' into their kernel configurations along with 'device pci'. 20110427: The default NFS client is now the new NFS client, so fstype "newnfs" is now "nfs" and the regular/old NFS client is now fstype "oldnfs". Although mounts via fstype "nfs" will usually work without userland changes, it is recommended that the mount(8) and mount_nfs(8) commands be rebuilt from sources and that a link to mount_nfs called mount_oldnfs be created. The new client is compiled into the kernel with "options NFSCL" and this is needed for diskless root file systems. The GENERIC kernel configs have been changed to use NFSCL and NFSD (the new server) instead of NFSCLIENT and NFSSERVER. To use the regular/old client, you can "mount -t oldnfs ...". For a diskless root file system, you must also include a line like: vfs.root.mountfrom="oldnfs:" in the boot/loader.conf on the root fs on the NFS server to make a diskless root fs use the old client. 20110424: The GENERIC kernels for all architectures now default to the new CAM-based ATA stack. It means that all legacy ATA drivers were removed and replaced by respective CAM drivers. If you are using ATA device names in /etc/fstab or other places, make sure to update them respectively (adX -> adaY, acdX -> cdY, afdX -> daY, astX -> saY, where 'Y's are the sequential numbers starting from zero for each type in order of detection, unless configured otherwise with tunables, see cam(4)). There will be symbolic links created in /dev/ to map old adX devices to the respective adaY. They should provide basic compatibility for file systems mounting in most cases, but they do not support old user-level APIs and do not have respective providers in GEOM. Consider using updated management tools with new device names. It is possible to load devices ahci, ata, siis and mvs as modules, but option ATA_CAM should remain in kernel configuration to make ata module work as CAM driver supporting legacy ATA controllers. Device ata still can be used in modular fashion (atacore + ...). Modules atadisk and atapi* are not used and won't affect operation in ATA_CAM mode. Note that to use CAM-based ATA kernel should include CAM devices scbus, pass, da (or explicitly ada), cd and optionally others. All of them are parts of the cam module. ataraid(4) functionality is now supported by the RAID GEOM class. To use it you can load geom_raid kernel module and use graid(8) tool for management. Instead of /dev/arX device names, use /dev/raid/rX. No kernel config options or code have been removed, so if a problem arises, please report it and optionally revert to the old ATA stack. In order to do it you can remove from the kernel config: options ATA_CAM device ahci device mvs device siis , and instead add back: device atadisk # ATA disk drives device ataraid # ATA RAID drives device atapicd # ATAPI CDROM drives device atapifd # ATAPI floppy drives device atapist # ATAPI tape drives 20110423: The default NFS server has been changed to the new server, which was referred to as the experimental server. If you need to switch back to the old NFS server, you must now put the "-o" option on both the mountd and nfsd commands. This can be done using the mountd_flags and nfs_server_flags rc.conf variables until an update to the rc scripts is committed, which is coming soon. 20110418: The GNU Objective-C runtime library (libobjc), and other Objective-C related components have been removed from the base system. If you require an Objective-C library, please use one of the available ports. 20110331: ath(4) has been split into bus- and device- modules. if_ath contains the HAL, the TX rate control and the network device code. if_ath_pci contains the PCI bus glue. For Atheros MIPS embedded systems, if_ath_ahb contains the AHB glue. Users need to load both if_ath_pci and if_ath in order to use ath on everything else. TO REPEAT: if_ath_ahb is not needed for normal users. Normal users only need to load if_ath and if_ath_pci for ath(4) operation. 20110314: As part of the replacement of sysinstall, the process of building release media has changed significantly. For details, please re-read release(7), which has been updated to reflect the new build process. 20110218: GNU binutils 2.17.50 (as of 2007-07-03) has been merged to -HEAD. This is the last available version under GPLv2. It brings a number of new features, such as support for newer x86 CPU's (with SSE-3, SSSE-3, SSE 4.1 and SSE 4.2), better support for powerpc64, a number of new directives, and lots of other small improvements. See the ChangeLog file in contrib/binutils for the full details. 20110218: IPsec's HMAC_SHA256-512 support has been fixed to be RFC4868 compliant, and will now use half of hash for authentication. This will break interoperability with all stacks (including all actual FreeBSD versions) who implement draft-ietf-ipsec-ciph-sha-256-00 (they use 96 bits of hash for authentication). The only workaround with such peers is to use another HMAC algorithm for IPsec ("phase 2") authentication. 20110207: Remove the uio_yield prototype and symbol. This function has been misnamed since it was introduced and should not be globally exposed with this name. The equivalent functionality is now available using kern_yield(curthread->td_user_pri). The function remains undocumented. 20110112: A SYSCTL_[ADD_]UQUAD was added for unsigned uint64_t pointers, symmetric with the existing SYSCTL_[ADD_]QUAD. Type checking for scalar sysctls is defined but disabled. Code that needs UQUAD to pass the type checking that must compile on older systems where the define is not present can check against __FreeBSD_version >= 900030. The system dialog(1) has been replaced with a new version previously in ports as devel/cdialog. dialog(1) is mostly command-line compatible with the previous version, but the libdialog associated with it has a largely incompatible API. As such, the original version of libdialog will be kept temporarily as libodialog, until its base system consumers are replaced or updated. Bump __FreeBSD_version to 900030. 20110103: If you are trying to run make universe on a -stable system, and you get the following warning: "Makefile", line 356: "Target architecture for i386/conf/GENERIC unknown. config(8) likely too old." or something similar to it, then you must upgrade your -stable system to 8.2-Release or newer (really, any time after r210146 7/15/2010 in stable/8) or build the config from the latest stable/8 branch and install it on your system. Prior to this date, building a current universe on 8-stable system from between 7/15/2010 and 1/2/2011 would result in a weird shell parsing error in the first kernel build phase. A new config on those old systems will fix that problem for older versions of -current. 20101228: The TCP stack has been modified to allow Khelp modules to interact with it via helper hook points and store per-connection data in the TCP control block. Bump __FreeBSD_version to 900029. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20101114: Generic IEEE 802.3 annex 31B full duplex flow control support has been added to mii(4) and bge(4), bce(4), msk(4), nfe(4) and stge(4) along with brgphy(4), e1000phy(4) as well as ip1000phy() have been converted to take advantage of it instead of using custom implementations. This means that these drivers now no longer unconditionally advertise support for flow control but only do so if flow control is a selected media option. This was implemented in the generic support that way in order to allow flow control to be switched on and off via ifconfig(8) with the PHY specific default to typically off in order to protect from unwanted effects. Consequently, if you used flow control with one of the above mentioned drivers you now need to explicitly enable it, for example via: ifconfig bge0 media auto mediaopt flowcontrol Along with the above mentioned changes generic support for setting 1000baseT master mode also has been added and brgphy(4), ciphy(4), e1000phy(4) as well as ip1000phy(4) have been converted to take advantage of it. This means that these drivers now no longer take the link0 parameter for selecting master mode but the master media option has to be used instead, for example like in the following: ifconfig bge0 media 1000baseT mediaopt full-duplex,master Selection of master mode now is also available with all other PHY drivers supporting 1000baseT. 20101111: The TCP stack has received a significant update to add support for modularised congestion control and generally improve the clarity of congestion control decisions. Bump __FreeBSD_version to 900025. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20101002: The man(1) utility has been replaced by a new version that no longer uses /etc/manpath.config. Please consult man.conf(5) for how to migrate local entries to the new format. 20100928: The copyright strings printed by login(1) and sshd(8) at the time of a new connection have been removed to follow other operating systems and upstream sshd. 20100915: A workaround for a fixed ld bug has been removed in kernel code, so make sure that your system ld is built from sources after revision 210245 from 2010-07-19 (r211583 if building head kernel on stable/8, r211584 for stable/7; both from 2010-08-21). A symptom of incorrect ld version is different addresses for set_pcpu section and __start_set_pcpu symbol in kernel and/or modules. 20100913: The $ipv6_prefer variable in rc.conf(5) has been split into $ip6addrctl_policy and $ipv6_activate_all_interfaces. The $ip6addrctl_policy is a variable to choose a pre-defined address selection policy set by ip6addrctl(8). A value "ipv4_prefer", "ipv6_prefer" or "AUTO" can be specified. The default is "AUTO". The $ipv6_activate_all_interfaces specifies whether IFDISABLED flag (see an entry of 20090926) is set on an interface with no corresponding $ifconfig_IF_ipv6 line. The default is "NO" for security reason. If you want IPv6 link-local address on all interfaces by default, set this to "YES". The old ipv6_prefer="YES" is equivalent to ipv6_activate_all_interfaces="YES" and ip6addrctl_policy="ipv6_prefer". 20100913: DTrace has grown support for userland tracing. Due to this, DTrace is now i386 and amd64 only. dtruss(1) is now installed by default on those systems and a new kernel module is needed for userland tracing: fasttrap. No changes to your kernel config file are necessary to enable userland tracing, but you might consider adding 'STRIP=' and 'CFLAGS+=-fno-omit-frame-pointer' to your make.conf if you want to have informative userland stack traces in DTrace (ustack). 20100725: The acpi_aiboost(4) driver has been removed in favor of the new aibs(4) driver. You should update your kernel configuration file. 20100722: BSD grep has been imported to the base system and it is built by default. It is completely BSD licensed, highly GNU-compatible, uses less memory than its GNU counterpart and has a small codebase. However, it is slower than its GNU counterpart, which is mostly noticeable for larger searches, for smaller ones it is measurable but not significant. The reason is complex, the most important factor is that we lack a modern and efficient regex library and GNU overcomes this by optimizing the searches internally. Future work on improving the regex performance is planned, for the meantime, users that need better performance, can build GNU grep instead by setting the WITH_GNU_GREP knob. 20100713: Due to the import of powerpc64 support, all existing powerpc kernel configuration files must be updated with a machine directive like this: machine powerpc powerpc In addition, an updated config(8) is required to build powerpc kernels after this change. 20100713: A new version of ZFS (version 15) has been merged to -HEAD. This version uses a python library for the following subcommands: zfs allow, zfs unallow, zfs groupspace, zfs userspace. For full functionality of these commands the following port must be installed: sysutils/py-zfs 20100429: 'vm_page's are now hashed by physical address to an array of mutexes. Currently this is only used to serialize access to hold_count. Over time the page queue mutex will be peeled away. This changes the size of pmap on every architecture. And requires all callers of vm_page_hold and vm_page_unhold to be updated. 20100402: WITH_CTF can now be specified in src.conf (not recommended, there are some problems with static executables), make.conf (would also affect ports which do not use GNU make and do not override the compile targets) or in the kernel config (via "makeoptions WITH_CTF=yes"). When WITH_CTF was specified there before this was silently ignored, so make sure that WITH_CTF is not used in places which could lead to unwanted behavior. 20100311: The kernel option COMPAT_IA32 has been replaced with COMPAT_FREEBSD32 to allow 32-bit compatibility on non-x86 platforms. All kernel configurations on amd64 and ia64 platforms using these options must be modified accordingly. 20100113: The utmp user accounting database has been replaced with utmpx, the user accounting interface standardized by POSIX. Unfortunately the semantics of utmp and utmpx don't match, making it practically impossible to support both interfaces. The user accounting database is used by tools like finger(1), last(1), talk(1), w(1) and ac(8). All applications in the base system use utmpx. This means only local binaries (e.g. from the ports tree) may still use these utmp database files. These applications must be rebuilt to make use of utmpx. After the system has been upgraded, it is safe to remove the old log files (/var/run/utmp, /var/log/lastlog and /var/log/wtmp*), assuming their contents is of no importance anymore. Old wtmp databases can only be used by last(1) and ac(8) after they have been converted to the new format using wtmpcvt(1). 20100108: Introduce the kernel thread "deadlock resolver" (which can be enabled via the DEADLKRES option, see NOTES for more details) and the sleepq_type() function for sleepqueues. 20091202: The rc.firewall and rc.firewall6 were unified, and rc.firewall6 and rc.d/ip6fw were removed. According to the removal of rc.d/ip6fw, ipv6_firewall_* rc variables are obsoleted. Instead, the following new rc variables are added to rc.d/ipfw: firewall_client_net_ipv6, firewall_simple_iif_ipv6, firewall_simple_inet_ipv6, firewall_simple_oif_ipv6, firewall_simple_onet_ipv6, firewall_trusted_ipv6 The meanings correspond to the relevant IPv4 variables. 20091125: 8.0-RELEASE. 20091113: The default terminal emulation for syscons(4) has been changed from cons25 to xterm on all platforms except pc98. This means that the /etc/ttys file needs to be updated to ensure correct operation of applications on the console. The terminal emulation style can be toggled per window by using vidcontrol(1)'s -T flag. The TEKEN_CONS25 kernel configuration options can be used to change the compile-time default back to cons25. To prevent graphical artifacts, make sure the TERM environment variable is set to match the terminal emulation that is being performed by syscons(4). 20091109: The layout of the structure ieee80211req_scan_result has changed. Applications that require wireless scan results (e.g. ifconfig(8)) from net80211 need to be recompiled. Applications such as wpa_supplicant(8) may require a full world build without using NO_CLEAN in order to get synchronized with the new structure. 20091025: The iwn(4) driver has been updated to support the 5000 and 5150 series. There's one kernel module for each firmware. Adding "device iwnfw" to the kernel configuration file means including all three firmware images inside the kernel. If you want to include just the one for your wireless card, use the devices iwn4965fw, iwn5000fw or iwn5150fw. 20090926: The rc.d/network_ipv6, IPv6 configuration script has been integrated into rc.d/netif. The changes are the following: 1. To use IPv6, simply define $ifconfig_IF_ipv6 like $ifconfig_IF for IPv4. For aliases, $ifconfig_IF_aliasN should be used. Note that both variables need the "inet6" keyword at the head. Do not set $ipv6_network_interfaces manually if you do not understand what you are doing. It is not needed in most cases. $ipv6_ifconfig_IF and $ipv6_ifconfig_IF_aliasN still work, but they are obsolete. 2. $ipv6_enable is obsolete. Use $ipv6_prefer and "inet6 accept_rtadv" keyword in ifconfig(8) instead. If you define $ipv6_enable=YES, it means $ipv6_prefer=YES and all configured interfaces have "inet6 accept_rtadv" in the $ifconfig_IF_ipv6. These are for backward compatibility. 3. A new variable $ipv6_prefer has been added. If NO, IPv6 functionality of interfaces with no corresponding $ifconfig_IF_ipv6 is disabled by using "inet6 ifdisabled" flag, and the default address selection policy of ip6addrctl(8) is the IPv4-preferred one (see rc.d/ip6addrctl for more details). Note that if you want to configure IPv6 functionality on the disabled interfaces after boot, first you need to clear the flag by using ifconfig(8) like: ifconfig em0 inet6 -ifdisabled If YES, the default address selection policy is set as IPv6-preferred. The default value of $ipv6_prefer is NO. 4. If your system need to receive Router Advertisement messages, define "inet6 accept_rtadv" in $ifconfig_IF_ipv6. The rc(8) scripts automatically invoke rtsol(8) when the interface becomes UP. The Router Advertisement messages are used for SLAAC (State-Less Address AutoConfiguration). 20090922: 802.11s D3.03 support was committed. This is incompatible with the previous code, which was based on D3.0. 20090912: A sysctl variable net.inet6.ip6.accept_rtadv now sets the default value of a per-interface flag ND6_IFF_ACCEPT_RTADV, not a global knob to control whether accepting Router Advertisement messages or not. Also, a per-interface flag ND6_IFF_AUTO_LINKLOCAL has been added and a sysctl variable net.inet6.ip6.auto_linklocal is its default value. The ifconfig(8) utility now supports these flags. 20090910: ZFS snapshots are now mounted with MNT_IGNORE flag. Use -v option for mount(8) and -a option for df(1) to see them. 20090825: The old tunable hw.bus.devctl_disable has been superseded by hw.bus.devctl_queue. hw.bus.devctl_disable=1 in loader.conf should be replaced by hw.bus.devctl_queue=0. The default for this new tunable is 1000. 20090813: Remove the option STOP_NMI. The default action is now to use NMI only for KDB via the newly introduced function stop_cpus_hard() and maintain stop_cpus() to just use a normal IPI_STOP on ia32 and amd64. 20090803: The stable/8 branch created in subversion. This corresponds to the RELENG_8 branch in CVS. 20090719: Bump the shared library version numbers for all libraries that do not use symbol versioning as part of the 8.0-RELEASE cycle. Bump __FreeBSD_version to 800105. 20090714: Due to changes in the implementation of virtual network stack support, all network-related kernel modules must be recompiled. As this change breaks the ABI, bump __FreeBSD_version to 800104. 20090713: The TOE interface to the TCP syncache has been modified to remove struct tcpopt () from the ABI of the network stack. The cxgb driver is the only TOE consumer affected by this change, and needs to be recompiled along with the kernel. As this change breaks the ABI, bump __FreeBSD_version to 800103. 20090712: Padding has been added to struct tcpcb, sackhint and tcpstat in to facilitate future MFCs and bug fixes whilst maintaining the ABI. However, this change breaks the ABI, so bump __FreeBSD_version to 800102. User space tools that rely on the size of any of these structs (e.g. sockstat) need to be recompiled. 20090630: The NFS_LEGACYRPC option has been removed along with the old kernel RPC implementation that this option selected. Kernel configurations may need to be adjusted. 20090629: The network interface device nodes at /dev/net/ have been removed. All ioctl operations can be performed the normal way using routing sockets. The kqueue functionality can generally be replaced with routing sockets. 20090628: The documentation from the FreeBSD Documentation Project (Handbook, FAQ, etc.) is now installed via packages by sysinstall(8) and under the /usr/local/share/doc/freebsd directory instead of /usr/share/doc. 20090624: The ABI of various structures related to the SYSV IPC API have been changed. As a result, the COMPAT_FREEBSD[456] and COMPAT_43 kernel options now all require COMPAT_FREEBSD7. Bump __FreeBSD_version to 800100. 20090622: Layout of struct vnet has changed as routing related variables were moved to their own Vimage module. Modules need to be recompiled. Bump __FreeBSD_version to 800099. 20090619: NGROUPS_MAX and NGROUPS have been increased from 16 to 1023 and 1024 respectively. As long as no more than 16 groups per process are used, no changes should be visible. When more than 16 groups are used, old binaries may fail if they call getgroups() or getgrouplist() with statically sized storage. Recompiling will work around this, but applications should be modified to use dynamically allocated storage for group arrays as POSIX.1-2008 does not cap an implementation's number of supported groups at NGROUPS_MAX+1 as previous versions did. NFS and portalfs mounts may also be affected as the list of groups is truncated to 16. Users of NFS who use more than 16 groups, should take care that negative group permissions are not used on the exported file systems as they will not be reliable unless a GSSAPI based authentication method is used. 20090616: The compiling option ADAPTIVE_LOCKMGRS has been introduced. This option compiles in the support for adaptive spinning for lockmgrs which want to enable it. The lockinit() function now accepts the flag LK_ADAPTIVE in order to make the lock object subject to adaptive spinning when both held in write and read mode. 20090613: The layout of the structure returned by IEEE80211_IOC_STA_INFO has changed. User applications that use this ioctl need to be rebuilt. 20090611: The layout of struct thread has changed. Kernel and modules need to be rebuilt. 20090608: The layout of structs ifnet, domain, protosw and vnet_net has changed. Kernel modules need to be rebuilt. Bump __FreeBSD_version to 800097. 20090602: window(1) has been removed from the base system. It can now be installed from ports. The port is called misc/window. 20090601: The way we are storing and accessing `routing table' entries has changed. Programs reading the FIB, like netstat, need to be re-compiled. 20090601: A new netisr implementation has been added for FreeBSD 8. Network file system modules, such as igmp, ipdivert, and others, should be rebuilt. Bump __FreeBSD_version to 800096. 20090530: Remove the tunable/sysctl debug.mpsafevfs as its initial purpose is no more valid. 20090530: Add VOP_ACCESSX(9). File system modules need to be rebuilt. Bump __FreeBSD_version to 800094. 20090529: Add mnt_xflag field to 'struct mount'. File system modules need to be rebuilt. Bump __FreeBSD_version to 800093. 20090528: The compiling option ADAPTIVE_SX has been retired while it has been introduced the option NO_ADAPTIVE_SX which handles the reversed logic. The KPI for sx_init_flags() changes as accepting flags: SX_ADAPTIVESPIN flag has been retired while the SX_NOADAPTIVE flag has been introduced in order to handle the reversed logic. Bump __FreeBSD_version to 800092. 20090527: Add support for hierarchical jails. Remove global securelevel. Bump __FreeBSD_version to 800091. 20090523: The layout of struct vnet_net has changed, therefore modules need to be rebuilt. Bump __FreeBSD_version to 800090. 20090523: The newly imported zic(8) produces a new format in the output. Please run tzsetup(8) to install the newly created data to /etc/localtime. 20090520: The sysctl tree for the usb stack has renamed from hw.usb2.* to hw.usb.* and is now consistent again with previous releases. 20090520: 802.11 monitor mode support was revised and driver api's were changed. Drivers dependent on net80211 now support DLT_IEEE802_11_RADIO instead of DLT_IEEE802_11. No user-visible data structures were changed but applications that use DLT_IEEE802_11 may require changes. Bump __FreeBSD_version to 800088. 20090430: The layout of the following structs has changed: sysctl_oid, socket, ifnet, inpcbinfo, tcpcb, syncache_head, vnet_inet, vnet_inet6 and vnet_ipfw. Most modules need to be rebuild or panics may be experienced. World rebuild is required for correctly checking networking state from userland. Bump __FreeBSD_version to 800085. 20090429: MLDv2 and Source-Specific Multicast (SSM) have been merged to the IPv6 stack. VIMAGE hooks are in but not yet used. The implementation of SSM within FreeBSD's IPv6 stack closely follows the IPv4 implementation. For kernel developers: * The most important changes are that the ip6_output() and ip6_input() paths no longer take the IN6_MULTI_LOCK, and this lock has been downgraded to a non-recursive mutex. * As with the changes to the IPv4 stack to support SSM, filtering of inbound multicast traffic must now be performed by transport protocols within the IPv6 stack. This does not apply to TCP and SCTP, however, it does apply to UDP in IPv6 and raw IPv6. * The KPIs used by IPv6 multicast are similar to those used by the IPv4 stack, with the following differences: * im6o_mc_filter() is analogous to imo_multicast_filter(). * The legacy KAME entry points in6_joingroup and in6_leavegroup() are shimmed to in6_mc_join() and in6_mc_leave() respectively. * IN6_LOOKUP_MULTI() has been deprecated and removed. * IPv6 relies on MLD for the DAD mechanism. KAME's internal KPIs for MLDv1 have an additional 'timer' argument which is used to jitter the initial membership report for the solicited-node multicast membership on-link. * This is not strictly needed for MLDv2, which already jitters its report transmissions. However, the 'timer' argument is preserved in case MLDv1 is active on the interface. * The KAME linked-list based IPv6 membership implementation has been refactored to use a vector similar to that used by the IPv4 stack. Code which maintains a list of its own multicast memberships internally, e.g. carp, has been updated to reflect the new semantics. * There is a known Lock Order Reversal (LOR) due to in6_setscope() acquiring the IF_AFDATA_LOCK and being called within ip6_output(). Whilst MLDv2 tries to avoid this otherwise benign LOR, it is an implementation constraint which needs to be addressed in HEAD. For application developers: * The changes are broadly similar to those made for the IPv4 stack. * The use of IPv4 and IPv6 multicast socket options on the same socket, using mapped addresses, HAS NOT been tested or supported. * There are a number of issues with the implementation of various IPv6 multicast APIs which need to be resolved in the API surface before the implementation is fully compatible with KAME userland use, and these are mostly to do with interface index treatment. * The literature available discusses the use of either the delta / ASM API with setsockopt(2)/getsockopt(2), or the full-state / ASM API using setsourcefilter(3)/getsourcefilter(3). For more information please refer to RFC 3768, 'Socket Interface Extensions for Multicast Source Filters'. * Applications which use the published RFC 3678 APIs should be fine. For systems administrators: * The mtest(8) utility has been refactored to support IPv6, in addition to IPv4. Interface addresses are no longer accepted as arguments, their names must be used instead. The utility will map the interface name to its first IPv4 address as returned by getifaddrs(3). * The ifmcstat(8) utility has also been updated to print the MLDv2 endpoint state and source filter lists via sysctl(3). * The net.inet6.ip6.mcast.loop sysctl may be tuned to 0 to disable loopback of IPv6 multicast datagrams by default; it defaults to 1 to preserve the existing behaviour. Disabling multicast loopback is recommended for optimal system performance. * The IPv6 MROUTING code has been changed to examine this sysctl instead of attempting to perform a group lookup before looping back forwarded datagrams. Bump __FreeBSD_version to 800084. 20090422: Implement low-level Bluetooth HCI API. Bump __FreeBSD_version to 800083. 20090419: The layout of struct malloc_type, used by modules to register new memory allocation types, has changed. Most modules will need to be rebuilt or panics may be experienced. Bump __FreeBSD_version to 800081. 20090415: Anticipate overflowing inp_flags - add inp_flags2. This changes most offsets in inpcb, so checking v4 connection state will require a world rebuild. Bump __FreeBSD_version to 800080. 20090415: Add an llentry to struct route and struct route_in6. Modules embedding a struct route will need to be recompiled. Bump __FreeBSD_version to 800079. 20090414: The size of rt_metrics_lite and by extension rtentry has changed. Networking administration apps will need to be recompiled. The route command now supports show as an alias for get, weighting of routes, sticky and nostick flags to alter the behavior of stateful load balancing. Bump __FreeBSD_version to 800078. 20090408: Do not use Giant for kbdmux(4) locking. This is wrong and apparently causing more problems than it solves. This will re-open the issue where interrupt handlers may race with kbdmux(4) in polling mode. Typical symptoms include (but not limited to) duplicated and/or missing characters when low level console functions (such as gets) are used while interrupts are enabled (for example geli password prompt, mountroot prompt etc.). Disabling kbdmux(4) may help. 20090407: The size of structs vnet_net, vnet_inet and vnet_ipfw has changed; kernel modules referencing any of the above need to be recompiled. Bump __FreeBSD_version to 800075. 20090320: GEOM_PART has become the default partition slicer for storage devices, replacing GEOM_MBR, GEOM_BSD, GEOM_PC98 and GEOM_GPT slicers. It introduces some changes: MSDOS/EBR: the devices created from MSDOS extended partition entries (EBR) can be named differently than with GEOM_MBR and are now symlinks to devices with offset-based names. fstabs may need to be modified. BSD: the "geometry does not match label" warning is harmless in most cases but it points to problems in file system misalignment with disk geometry. The "c" partition is now implicit, covers the whole top-level drive and cannot be (mis)used by users. General: Kernel dumps are now not allowed to be written to devices whose partition types indicate they are meant to be used for file systems (or, in case of MSDOS partitions, as something else than the "386BSD" type). Most of these changes date approximately from 200812. 20090319: The uscanner(4) driver has been removed from the kernel. This follows Linux removing theirs in 2.6 and making libusb the default interface (supported by sane). 20090319: The multicast forwarding code has been cleaned up. netstat(1) only relies on KVM now for printing bandwidth upcall meters. The IPv4 and IPv6 modules are split into ip_mroute_mod and ip6_mroute_mod respectively. The config(5) options for statically compiling this code remain the same, i.e. 'options MROUTING'. 20090315: Support for the IFF_NEEDSGIANT network interface flag has been removed, which means that non-MPSAFE network device drivers are no longer supported. In particular, if_ar, if_sr, and network device drivers from the old (legacy) USB stack can no longer be built or used. 20090313: POSIX.1 Native Language Support (NLS) has been enabled in libc and a bunch of new language catalog files have also been added. This means that some common libc messages are now localized and they depend on the LC_MESSAGES environmental variable. 20090313: The k8temp(4) driver has been renamed to amdtemp(4) since support for Family 10 and Family 11 CPU families was added. 20090309: IGMPv3 and Source-Specific Multicast (SSM) have been merged to the IPv4 stack. VIMAGE hooks are in but not yet used. For kernel developers, the most important changes are that the ip_output() and ip_input() paths no longer take the IN_MULTI_LOCK(), and this lock has been downgraded to a non-recursive mutex. Transport protocols (UDP, Raw IP) are now responsible for filtering inbound multicast traffic according to group membership and source filters. The imo_multicast_filter() KPI exists for this purpose. Transports which do not use multicast (SCTP, TCP) already reject multicast by default. Forwarding and receive performance may improve as a mutex acquisition is no longer needed in the ip_input() low-level input path. in_addmulti() and in_delmulti() are shimmed to new KPIs which exist to support SSM in-kernel. For application developers, it is recommended that loopback of multicast datagrams be disabled for best performance, as this will still cause the lock to be taken for each looped-back datagram transmission. The net.inet.ip.mcast.loop sysctl may be tuned to 0 to disable loopback by default; it defaults to 1 to preserve the existing behaviour. For systems administrators, to obtain best performance with multicast reception and multiple groups, it is always recommended that a card with a suitably precise hash filter is used. Hash collisions will still result in the lock being taken within the transport protocol input path to check group membership. If deploying FreeBSD in an environment with IGMP snooping switches, it is recommended that the net.inet.igmp.sendlocal sysctl remain enabled; this forces 224.0.0.0/24 group membership to be announced via IGMP. The size of 'struct igmpstat' has changed; netstat needs to be recompiled to reflect this. Bump __FreeBSD_version to 800070. 20090309: libusb20.so.1 is now installed as libusb.so.1 and the ports system updated to use it. This requires a buildworld/installworld in order to update the library and dependencies (usbconfig, etc). Its advisable to rebuild all ports which uses libusb. More specific directions are given in the ports collection UPDATING file. Any /etc/libmap.conf entries for libusb are no longer required and can be removed. 20090302: A workaround is committed to allow the creation of System V shared memory segment of size > 2 GB on the 64-bit architectures. Due to a limitation of the existing ABI, the shm_segsz member of the struct shmid_ds, returned by shmctl(IPC_STAT) call is wrong for large segments. Note that limits must be explicitly raised to allow such segments to be created. 20090301: The layout of struct ifnet has changed, requiring a rebuild of all network device driver modules. 20090227: The /dev handling for the new USB stack has changed, a buildworld/installworld is required for libusb20. 20090223: The new USB2 stack has now been permanently moved in and all kernel and module names reverted to their previous values (eg, usb, ehci, ohci, ums, ...). The old usb stack can be compiled in by prefixing the name with the letter 'o', the old usb modules have been removed. Updating entry 20090216 for xorg and 20090215 for libmap may still apply. 20090217: The rc.conf(5) option if_up_delay has been renamed to defaultroute_delay to better reflect its purpose. If you have customized this setting in /etc/rc.conf you need to update it to use the new name. 20090216: xorg 7.4 wants to configure its input devices via hald which does not yet work with USB2. If the keyboard/mouse does not work in xorg then add Option "AllowEmptyInput" "off" to your ServerLayout section. This will cause X to use the configured kbd and mouse sections from your xorg.conf. 20090215: The GENERIC kernels for all architectures now default to the new USB2 stack. No kernel config options or code have been removed so if a problem arises please report it and optionally revert to the old USB stack. If you are loading USB kernel modules or have a custom kernel that includes GENERIC then ensure that usb names are also changed over, eg uftdi -> usb2_serial_ftdi. Older programs linked against the ports libusb 0.1 need to be redirected to the new stack's libusb20. /etc/libmap.conf can be used for this: # Map old usb library to new one for usb2 stack libusb-0.1.so.8 libusb20.so.1 20090209: All USB ethernet devices now attach as interfaces under the name ueN (eg. ue0). This is to provide a predictable name as vendors often change usb chipsets in a product without notice. 20090203: The ichsmb(4) driver has been changed to require SMBus slave addresses be left-justified (xxxxxxx0b) rather than right-justified. All of the other SMBus controller drivers require left-justified slave addresses, so this change makes all the drivers provide the same interface. 20090201: INET6 statistics (struct ip6stat) was updated. netstat(1) needs to be recompiled. 20090119: NTFS has been removed from GENERIC kernel on amd64 to match GENERIC on i386. Should not cause any issues since mount_ntfs(8) will load ntfs.ko module automatically when NTFS support is actually needed, unless ntfs.ko is not installed or security level prohibits loading kernel modules. If either is the case, "options NTFS" has to be added into kernel config. 20090115: TCP Appropriate Byte Counting (RFC 3465) support added to kernel. New field in struct tcpcb breaks ABI, so bump __FreeBSD_version to 800061. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20081225: ng_tty(4) module updated to match the new TTY subsystem. Due to API change, user-level applications must be updated. New API support added to mpd5 CVS and expected to be present in next mpd5.3 release. 20081219: With __FreeBSD_version 800060 the makefs tool is part of the base system (it was a port). 20081216: The afdata and ifnet locks have been changed from mutexes to rwlocks, network modules will need to be re-compiled. 20081214: __FreeBSD_version 800059 incorporates the new arp-v2 rewrite. RTF_CLONING, RTF_LLINFO and RTF_WASCLONED flags are eliminated. The new code reduced struct rtentry{} by 16 bytes on 32-bit architecture and 40 bytes on 64-bit architecture. The userland applications "arp" and "ndp" have been updated accordingly. The output from "netstat -r" shows only routing entries and none of the L2 information. 20081130: __FreeBSD_version 800057 marks the switchover from the binary ath hal to source code. Users must add the line: options AH_SUPPORT_AR5416 to their kernel config files when specifying: device ath_hal The ath_hal module no longer exists; the code is now compiled together with the driver in the ath module. It is now possible to tailor chip support (i.e. reduce the set of chips and thereby the code size); consult ath_hal(4) for details. 20081121: __FreeBSD_version 800054 adds memory barriers to , new interfaces to ifnet to facilitate multiple hardware transmit queues for cards that support them, and a lock-less ring-buffer implementation to enable drivers to more efficiently manage queueing of packets. 20081117: A new version of ZFS (version 13) has been merged to -HEAD. This version has zpool attribute "listsnapshots" off by default, which means "zfs list" does not show snapshots, and is the same as Solaris behavior. 20081028: dummynet(4) ABI has changed. ipfw(8) needs to be recompiled. 20081009: The uhci, ohci, ehci and slhci USB Host controller drivers have been put into separate modules. If you load the usb module separately through loader.conf you will need to load the appropriate *hci module as well. E.g. for a UHCI-based USB 2.0 controller add the following to loader.conf: uhci_load="YES" ehci_load="YES" 20081009: The ABI used by the PMC toolset has changed. Please keep userland (libpmc(3)) and the kernel module (hwpmc(4)) in sync. 20081009: atapci kernel module now includes only generic PCI ATA driver. AHCI driver moved to ataahci kernel module. All vendor-specific code moved into separate kernel modules: ataacard, ataacerlabs, ataadaptec, ataamd, ataati, atacenatek, atacypress, atacyrix, atahighpoint, ataintel, ataite, atajmicron, atamarvell, atamicron, atanational, atanetcell, atanvidia, atapromise, ataserverworks, atasiliconimage, atasis, atavia 20080820: The TTY subsystem of the kernel has been replaced by a new implementation, which provides better scalability and an improved driver model. Most common drivers have been migrated to the new TTY subsystem, while others have not. The following drivers have not yet been ported to the new TTY layer: PCI/ISA: cy, digi, rc, rp, sio USB: ubser, ucycom Line disciplines: ng_h4, ng_tty, ppp, sl, snp Adding these drivers to your kernel configuration file shall cause compilation to fail. 20080818: ntpd has been upgraded to 4.2.4p5. 20080801: OpenSSH has been upgraded to 5.1p1. For many years, FreeBSD's version of OpenSSH preferred DSA over RSA for host and user authentication keys. With this upgrade, we've switched to the vendor's default of RSA over DSA. This may cause upgraded clients to warn about unknown host keys even for previously known hosts. Users should follow the usual procedure for verifying host keys before accepting the RSA key. This can be circumvented by setting the "HostKeyAlgorithms" option to "ssh-dss,ssh-rsa" in ~/.ssh/config or on the ssh command line. Please note that the sequence of keys offered for authentication has been changed as well. You may want to specify IdentityFile in a different order to revert this behavior. 20080713: The sio(4) driver has been removed from the i386 and amd64 kernel configuration files. This means uart(4) is now the default serial port driver on those platforms as well. To prevent collisions with the sio(4) driver, the uart(4) driver uses different names for its device nodes. This means the onboard serial port will now most likely be called "ttyu0" instead of "ttyd0". You may need to reconfigure applications to use the new device names. When using the serial port as a boot console, be sure to update /boot/device.hints and /etc/ttys before booting the new kernel. If you forget to do so, you can still manually specify the hints at the loader prompt: set hint.uart.0.at="isa" set hint.uart.0.port="0x3F8" set hint.uart.0.flags="0x10" set hint.uart.0.irq="4" boot -s 20080609: The gpt(8) utility has been removed. Use gpart(8) to partition disks instead. 20080603: The version that Linuxulator emulates was changed from 2.4.2 to 2.6.16. If you experience any problems with Linux binaries please try to set sysctl compat.linux.osrelease to 2.4.2 and if it fixes the problem contact emulation mailing list. 20080525: ISDN4BSD (I4B) was removed from the src tree. You may need to update a your kernel configuration and remove relevant entries. 20080509: I have checked in code to support multiple routing tables. See the man pages setfib(1) and setfib(2). This is a hopefully backwards compatible version, but to make use of it you need to compile your kernel with options ROUTETABLES=2 (or more up to 16). 20080420: The 802.11 wireless support was redone to enable multi-bss operation on devices that are capable. The underlying device is no longer used directly but instead wlanX devices are cloned with ifconfig. This requires changes to rc.conf files. For example, change: ifconfig_ath0="WPA DHCP" to wlans_ath0=wlan0 ifconfig_wlan0="WPA DHCP" see rc.conf(5) for more details. In addition, mergemaster of /etc/rc.d is highly recommended. Simultaneous update of userland and kernel wouldn't hurt either. As part of the multi-bss changes the wlan_scan_ap and wlan_scan_sta modules were merged into the base wlan module. All references to these modules (e.g. in kernel config files) must be removed. 20080408: psm(4) has gained write(2) support in native operation level. Arbitrary commands can be written to /dev/psm%d and status can be read back from it. Therefore, an application is responsible for status validation and error recovery. It is a no-op in other operation levels. 20080312: Support for KSE threading has been removed from the kernel. To run legacy applications linked against KSE libmap.conf may be used. The following libmap.conf may be used to ensure compatibility with any prior release: libpthread.so.1 libthr.so.1 libpthread.so.2 libthr.so.2 libkse.so.3 libthr.so.3 20080301: The layout of struct vmspace has changed. This affects libkvm and any executables that link against libkvm and use the kvm_getprocs() function. In particular, but not exclusively, it affects ps(1), fstat(1), pkill(1), systat(1), top(1) and w(1). The effects are minimal, but it's advisable to upgrade world nonetheless. 20080229: The latest em driver no longer has support in it for the 82575 adapter, this is now moved to the igb driver. The split was done to make new features that are incompatible with older hardware easier to do. 20080220: The new geom_lvm(4) geom class has been renamed to geom_linux_lvm(4), likewise the kernel option is now GEOM_LINUX_LVM. 20080211: The default NFS mount mode has changed from UDP to TCP for increased reliability. If you rely on (insecurely) NFS mounting across a firewall you may need to update your firewall rules. 20080208: Belatedly note the addition of m_collapse for compacting mbuf chains. 20080126: The fts(3) structures have been changed to use adequate integer types for their members and so to be able to cope with huge file trees. The old fts(3) ABI is preserved through symbol versioning in libc, so third-party binaries using fts(3) should still work, although they will not take advantage of the extended types. At the same time, some third-party software might fail to build after this change due to unportable assumptions made in its source code about fts(3) structure members. Such software should be fixed by its vendor or, in the worst case, in the ports tree. FreeBSD_version 800015 marks this change for the unlikely case that a portable fix is impossible. 20080123: To upgrade to -current after this date, you must be running FreeBSD not older than 6.0-RELEASE. Upgrading to -current from 5.x now requires a stop over at RELENG_6 or RELENG_7 systems. 20071128: The ADAPTIVE_GIANT kernel option has been retired because its functionality is the default now. 20071118: The AT keyboard emulation of sunkbd(4) has been turned on by default. In order to make the special symbols of the Sun keyboards driven by sunkbd(4) work under X these now have to be configured the same way as Sun USB keyboards driven by ukbd(4) (which also does AT keyboard emulation), f.e.: Option "XkbLayout" "us" Option "XkbRules" "xorg" Option "XkbSymbols" "pc(pc105)+sun_vndr/usb(sun_usb)+us" 20071024: It has been decided that it is desirable to provide ABI backwards compatibility to the FreeBSD 4/5/6 versions of the PCIOCGETCONF, PCIOCREAD and PCIOCWRITE IOCTLs, which was broken with the introduction of PCI domain support (see the 20070930 entry). Unfortunately, this required the ABI of PCIOCGETCONF to be broken again in order to be able to provide backwards compatibility to the old version of that IOCTL. Thus consumers of PCIOCGETCONF have to be recompiled again. As for prominent ports this affects neither pciutils nor xorg-server this time, the hal port needs to be rebuilt however. 20071020: The misnamed kthread_create() and friends have been renamed to kproc_create() etc. Many of the callers already used kproc_start().. I will return kthread_create() and friends in a while with implementations that actually create threads, not procs. Renaming corresponds with version 800002. 20071010: RELENG_7 branched. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. While generally safe, there are sometimes problems using -j to upgrade. If your upgrade fails with -j, please try again without -j. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. When upgrading from one major version to another it is generally best to upgrade to the latest code in the currently installed branch first, then do an upgrade to the new branch. This is the best-tested upgrade path, and has the highest probability of being successful. Please try this approach before reporting problems with a major version upgrade. ZFS notes --------- When upgrading the boot ZFS pool to a new version, always follow these two steps: 1.) recompile and reinstall the ZFS boot loader and boot block (this is part of "make buildworld" and "make installworld") 2.) update the ZFS boot block on your boot drive The following example updates the ZFS boot block on the first partition (freebsd-boot) of a GPT partitioned drive ad0: "gpart bootcode -p /boot/gptzfsboot -i 1 ad0" Non-boot pools do not need these updates. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. It is the most failsafe as it uses a /usr/obj tree with a fresh mini-buildworld, make kernel-toolchain make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To test a kernel once --------------------- If you just want to boot a kernel once (because you are not sure if it works, or if you want to boot a known bad kernel to provide debugging information) run make installkernel KERNCONF=YOUR_KERNEL_HERE KODIR=/boot/testkernel nextboot -k testkernel To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a CURRENT system. Replace ${arch} with the architecture of your machine (e.g. "i386", "arm", "amd64", "ia64", "pc98", "sparc64", "powerpc", "mips", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. make buildworld make kernel KERNCONF=YOUR_KERNEL_HERE [1] [3] mergemaster -p [5] make installworld mergemaster -i [4] make delete-old [6] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installworld DESTDIR=${CURRENT_ROOT} make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from stable to current ---------------------------------------------- make buildworld [9] make kernel KERNCONF=YOUR_KERNEL_HERE [8] [1] [3] mergemaster -p [5] make installworld mergemaster -i [4] make delete-old [6] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. The -U option is also useful to consider. See mergemaster(8) for more information. [5] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. You may need to install a new mergemaster (cd src/usr.sbin/mergemaster && make install) after the buildworld before this step if you last updated from [78]-stable or 9-stable before 20130430. [6] This only deletes old files and directories. Old libraries can be deleted by "make delete-old-libs", but you have to make sure that no program is using those libraries anymore. [8] In order to have a kernel that can run the 4.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD4 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. A similar kernel option COMPAT_FREEBSD5 is required to run the 5.x binaries on more recent kernels. And so on for COMPAT_FREEBSD6 and COMPAT_FREEBSD7. Make sure that you merge any new devices from GENERIC since the last time you updated your kernel config file. [9] When checking out sources, you must include the -P flag to have cvs prune empty directories. If CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. MAKEOBJDIRPREFIX must be defined in an environment variable, and not on the command line, or in /etc/make.conf. buildworld will warn if it is improperly defined. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -current. It is not guaranteed to be a complete list of such breakages, and only contains entries since October 10, 2007. If you need to see UPDATING entries from before that date, you will need to fetch an UPDATING file from an older FreeBSD release. Copyright information: Copyright 1998-2009 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: releng/9.3/sys/conf/newvers.sh =================================================================== --- releng/9.3/sys/conf/newvers.sh (revision 282873) +++ releng/9.3/sys/conf/newvers.sh (revision 282874) @@ -1,160 +1,160 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="9.3" -BRANCH="RELEASE-p13" +BRANCH="RELEASE-p14" if [ "X${BRANCH_OVERRIDE}" != "X" ]; then BRANCH=${BRANCH_OVERRIDE} fi RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" if [ "X${SYSDIR}" = "X" ]; then SYSDIR=$(dirname $0)/.. fi if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${SYSDIR}/sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/1992-$year/" \ -e 's/\[your name here\]\.* /The FreeBSD Project./' \ -e 's/\[your name\]\.*/The FreeBSD Project./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/*- * Copyright (c) 1992-$year The FreeBSD Project. * All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} t=`date` i=`${MAKE:-make} -V KERN_IDENT` compiler_v=$($(${MAKE:-make} -V CC) -v 2>&1 | grep 'version') for dir in /bin /usr/bin /usr/local/bin; do if [ -x "${dir}/svnversion" ] ; then svnversion=${dir}/svnversion break fi done if [ -d "${SYSDIR}/../.git" ] ; then for dir in /bin /usr/bin /usr/local/bin; do if [ -x "${dir}/git" ] ; then git_cmd="${dir}/git --git-dir=${SYSDIR}/../.git" break fi done fi if [ -n "$svnversion" ] ; then echo "$svnversion" svn=`cd ${SYSDIR} && $svnversion` case "$svn" in [0-9]*) svn=" r${svn}" ;; *) unset svn ;; esac fi if [ -n "$git_cmd" ] ; then git=`$git_cmd rev-parse --verify --short HEAD 2>/dev/null` svn=`$git_cmd svn find-rev $git 2>/dev/null` if [ -n "$svn" ] ; then svn=" r${svn}" git="=${git}" else svn=`$git_cmd log | fgrep 'git-svn-id:' | head -1 | \ sed -n 's/^.*@\([0-9][0-9]*\).*$/\1/p'` if [ -z "$svn" ] ; then svn=`$git_cmd log --format='format:%N' | \ grep '^svn ' | head -1 | \ sed -n 's/^.*revision=\([0-9][0-9]*\).*$/\1/p'` fi if [ -n "$svn" ] ; then svn=" r${svn}" git="+${git}" else git=" ${git}" fi fi if $git_cmd --work-tree=${SYSDIR}/.. diff-index \ --name-only HEAD | read dummy; then git="${git}-dirty" fi fi cat << EOF > vers.c $COPYRIGHT #define SCCSSTR "@(#)${VERSION} #${v}${svn}${git}: ${t}" #define VERSTR "${VERSION} #${v}${svn}${git}: ${t}\\n ${u}@${h}:${d}\\n" #define RELSTR "${RELEASE}" char sccs[sizeof(SCCSSTR) > 128 ? sizeof(SCCSSTR) : 128] = SCCSSTR; char version[sizeof(VERSTR) > 256 ? sizeof(VERSTR) : 256] = VERSTR; char compiler_version[] = "${compiler_v}"; char ostype[] = "${TYPE}"; char osrelease[sizeof(RELSTR) > 32 ? sizeof(RELSTR) : 32] = RELSTR; int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF echo $((v + 1)) > version Index: releng/9.3/usr.sbin/freebsd-update/freebsd-update.8 =================================================================== --- releng/9.3/usr.sbin/freebsd-update/freebsd-update.8 (revision 282873) +++ releng/9.3/usr.sbin/freebsd-update/freebsd-update.8 (revision 282874) @@ -1,177 +1,192 @@ .\"- .\" Copyright 2006, 2007 Colin Percival .\" All rights reserved .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted providing that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR .\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED .\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY .\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, .\" STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING .\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE .\" POSSIBILITY OF SUCH DAMAGE. .\" .\" $FreeBSD$ .\" -.Dd July 14, 2010 +.Dd March 2, 2015 .Dt FREEBSD-UPDATE 8 .Os FreeBSD .Sh NAME .Nm freebsd-update .Nd fetch and install binary updates to FreeBSD .Sh SYNOPSIS .Nm .Op Fl b Ar basedir .Op Fl d Ar workdir .Op Fl f Ar conffile +.Op Fl F .Op Fl k Ar KEY .Op Fl r Ar newrelease .Op Fl s Ar server .Op Fl t Ar address +.Op Fl -not-running-from-cron .Cm command ... .Sh DESCRIPTION The .Nm tool is used to fetch, install, and rollback binary updates to the FreeBSD base system. Note that updates are only available if they are being built for the FreeBSD release and architecture being used; in particular, the .Fx Security Team only builds updates for releases shipped in binary form by the .Fx Release Engineering Team, e.g., .Fx -7.3-RELEASE and +9.3-RELEASE and .Fx -8.0-RELEASE, but not +10.1-RELEASE, but not .Fx -6.3-STABLE or +9.3-STABLE or .Fx -9.0-CURRENT. +11-CURRENT. .Sh OPTIONS The following options are supported: -.Bl -tag -width "-f conffile" +.Bl -tag -width "-r newrelease" .It Fl b Ar basedir Operate on a system mounted at .Ar basedir . (default: .Pa / , or as given in the configuration file.) .It Fl d Ar workdir Store working files in .Ar workdir . (default: .Pa /var/db/freebsd-update/ , or as given in the configuration file.) .It Fl f Ar conffile Read configuration options from .Ar conffile . (default: .Pa /etc/freebsd-update.conf ) +.It Fl F +Force +.Nm Cm fetch +to proceed where it normally would not, such as an unfinished upgrade .It Fl k Ar KEY Trust an RSA key with SHA256 of .Ar KEY . (default: read value from configuration file.) .It Fl r Ar newrelease Specify the new release to which .Nm should upgrade (upgrade command only). .It Fl s Ar server Fetch files from the specified server or server pool. (default: read value from configuration file.) .It Fl t Ar address Mail output of .Cm cron command, if any, to .Ar address . (default: root, or as given in the configuration file.) +.It Fl -not-running-from-cron +Force +.Nm Cm fetch +to proceed when there is no controlling tty. +This is for use by automated scripts and orchestration tools. +Please do not run +.Nm Cm fetch +from crontab or similar using this flag, see: +.Nm Cm cron .El .Sh COMMANDS The .Cm command can be any one of the following: -.Bl -tag -width "-f conffile" +.Bl -tag -width "rollback" .It Cm fetch Based on the currently installed world and the configuration options set, fetch all available binary updates. .It Cm cron Sleep a random amount of time between 1 and 3600 seconds, then download updates as if the .Cm fetch command was used. If updates are downloaded, an email will be sent (to root or a different address if specified via the .Fl t option or in the configuration file). As the name suggests, this command is designed for running from .Xr cron 8 ; the random delay serves to minimize the probability that a large number of machines will simultaneously attempt to fetch updates. .It Cm upgrade Fetch files necessary for upgrading to a new release. Before using this command, make sure that you read the announcement and release notes for the new release in case there are any special steps needed for upgrading. Note that this command may require up to 500 MB of space in .Ar workdir depending on which components of the .Fx base system are installed. .It Cm install Install the most recently fetched updates or upgrade. .It Cm rollback Uninstall the most recently installed updates. .It Cm IDS Compare the system against a "known good" index of the installed release. .El .Sh TIPS .Bl -bullet .It If your clock is set to local time, adding the line .Pp .Dl 0 3 * * * root /usr/sbin/freebsd-update cron .Pp to /etc/crontab will check for updates every night. If your clock is set to UTC, please pick a random time other than 3AM, to avoid overly imposing an uneven load on the server(s) hosting the updates. .It In spite of its name, .Nm IDS should not be relied upon as an "Intrusion Detection System", since if the system has been tampered with it cannot be trusted to operate correctly. If you intend to use this command for intrusion-detection purposes, make sure you boot from a secure disk (e.g., a CD). .El .Sh FILES .Bl -tag -width "/etc/freebsd-update.conf" .It /etc/freebsd-update.conf Default location of the .Nm configuration file. .It /var/db/freebsd-update/ Default location where .Nm stores temporary files and downloaded updates. .El .Sh SEE ALSO .Xr freebsd-update.conf 5 .Sh AUTHORS .An Colin Percival Aq cperciva@FreeBSD.org Index: releng/9.3/usr.sbin/freebsd-update/freebsd-update.sh =================================================================== --- releng/9.3/usr.sbin/freebsd-update/freebsd-update.sh (revision 282873) +++ releng/9.3/usr.sbin/freebsd-update/freebsd-update.sh (revision 282874) @@ -1,3273 +1,3296 @@ #!/bin/sh #- # Copyright 2004-2007 Colin Percival # All rights reserved # # Redistribution and use in source and binary forms, with or without # modification, are permitted providing that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR # IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY # DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING # IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE # POSSIBILITY OF SUCH DAMAGE. # $FreeBSD$ #### Usage function -- called from command-line handling code. # Usage instructions. Options not listed: # --debug -- don't filter output from utilities # --no-stats -- don't show progress statistics while fetching files usage () { cat < ${LINE}" exit 1 fi done < ${CONFFILE} # Merge the settings read from the configuration file with those # provided at the command line. mergeconfig } # Provide some default parameters default_params () { # Save any parameters already configured, and clear the slate saveconfig nullconfig # Default configurations config_WorkDir /var/db/freebsd-update config_MailTo root config_AllowAdd yes config_AllowDelete yes config_KeepModifiedMetadata yes config_BaseDir / config_VerboseLevel stats config_StrictComponents no config_BackupKernel yes config_BackupKernelDir /boot/kernel.old config_BackupKernelSymbolFiles no # Merge these defaults into the earlier-configured settings mergeconfig } # Set utility output filtering options, based on ${VERBOSELEVEL} fetch_setup_verboselevel () { case ${VERBOSELEVEL} in debug) QUIETREDIR="/dev/stderr" QUIETFLAG=" " STATSREDIR="/dev/stderr" DDSTATS=".." XARGST="-t" NDEBUG=" " ;; nostats) QUIETREDIR="" QUIETFLAG="" STATSREDIR="/dev/null" DDSTATS=".." XARGST="" NDEBUG="" ;; stats) QUIETREDIR="/dev/null" QUIETFLAG="-q" STATSREDIR="/dev/stdout" DDSTATS="" XARGST="" NDEBUG="-n" ;; esac } # Perform sanity checks and set some final parameters # in preparation for fetching files. Figure out which # set of updates should be downloaded: If the user is # running *-p[0-9]+, strip off the last part; if the # user is running -SECURITY, call it -RELEASE. Chdir # into the working directory. fetchupgrade_check_params () { export HTTP_USER_AGENT="freebsd-update (${COMMAND}, `uname -r`)" _SERVERNAME_z=\ "SERVERNAME must be given via command line or configuration file." _KEYPRINT_z="Key must be given via -k option or configuration file." _KEYPRINT_bad="Invalid key fingerprint: " _WORKDIR_bad="Directory does not exist or is not writable: " if [ -z "${SERVERNAME}" ]; then echo -n "`basename $0`: " echo "${_SERVERNAME_z}" exit 1 fi if [ -z "${KEYPRINT}" ]; then echo -n "`basename $0`: " echo "${_KEYPRINT_z}" exit 1 fi if ! echo "${KEYPRINT}" | grep -qE "^[0-9a-f]{64}$"; then echo -n "`basename $0`: " echo -n "${_KEYPRINT_bad}" echo ${KEYPRINT} exit 1 fi if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then echo -n "`basename $0`: " echo -n "${_WORKDIR_bad}" echo ${WORKDIR} exit 1 fi chmod 700 ${WORKDIR} cd ${WORKDIR} || exit 1 # Generate release number. The s/SECURITY/RELEASE/ bit exists # to provide an upgrade path for FreeBSD Update 1.x users, since # the kernels provided by FreeBSD Update 1.x are always labelled # as X.Y-SECURITY. RELNUM=`uname -r | sed -E 's,-p[0-9]+,,' | sed -E 's,-SECURITY,-RELEASE,'` ARCH=`uname -m` FETCHDIR=${RELNUM}/${ARCH} PATCHDIR=${RELNUM}/${ARCH}/bp # Figure out what directory contains the running kernel BOOTFILE=`sysctl -n kern.bootfile` KERNELDIR=${BOOTFILE%/kernel} if ! [ -d ${KERNELDIR} ]; then echo "Cannot identify running kernel" exit 1 fi # Figure out what kernel configuration is running. We start with # the output of `uname -i`, and then make the following adjustments: # 1. Replace "SMP-GENERIC" with "SMP". Why the SMP kernel config # file says "ident SMP-GENERIC", I don't know... # 2. If the kernel claims to be GENERIC _and_ ${ARCH} is "amd64" # _and_ `sysctl kern.version` contains a line which ends "/SMP", then # we're running an SMP kernel. This mis-identification is a bug # which was fixed in 6.2-STABLE. KERNCONF=`uname -i` if [ ${KERNCONF} = "SMP-GENERIC" ]; then KERNCONF=SMP fi if [ ${KERNCONF} = "GENERIC" ] && [ ${ARCH} = "amd64" ]; then if sysctl kern.version | grep -qE '/SMP$'; then KERNCONF=SMP fi fi # Define some paths BSPATCH=/usr/bin/bspatch SHA256=/sbin/sha256 PHTTPGET=/usr/libexec/phttpget # Set up variables relating to VERBOSELEVEL fetch_setup_verboselevel # Construct a unique name from ${BASEDIR} BDHASH=`echo ${BASEDIR} | sha256 -q` } # Perform sanity checks etc. before fetching updates. fetch_check_params () { fetchupgrade_check_params if ! [ -z "${TARGETRELEASE}" ]; then echo -n "`basename $0`: " echo -n "-r option is meaningless with 'fetch' command. " echo "(Did you mean 'upgrade' instead?)" exit 1 fi + + # Check that we have updates ready to install + if [ -f ${BDHASH}-install/kerneldone -a $FORCEFETCH -eq 0 ]; then + echo "You have a partially completed upgrade pending" + echo "Run '$0 install' first." + echo "Run '$0 fetch -F' to proceed anyway." + exit 1 + fi } # Perform sanity checks etc. before fetching upgrades. upgrade_check_params () { fetchupgrade_check_params # Unless set otherwise, we're upgrading to the same kernel config. NKERNCONF=${KERNCONF} # We need TARGETRELEASE set _TARGETRELEASE_z="Release target must be specified via -r option." if [ -z "${TARGETRELEASE}" ]; then echo -n "`basename $0`: " echo "${_TARGETRELEASE_z}" exit 1 fi # The target release should be != the current release. if [ "${TARGETRELEASE}" = "${RELNUM}" ]; then echo -n "`basename $0`: " echo "Cannot upgrade from ${RELNUM} to itself" exit 1 fi # Turning off AllowAdd or AllowDelete is a bad idea for upgrades. if [ "${ALLOWADD}" = "no" ]; then echo -n "`basename $0`: " echo -n "WARNING: \"AllowAdd no\" is a bad idea " echo "when upgrading between releases." echo fi if [ "${ALLOWDELETE}" = "no" ]; then echo -n "`basename $0`: " echo -n "WARNING: \"AllowDelete no\" is a bad idea " echo "when upgrading between releases." echo fi # Set EDITOR to /usr/bin/vi if it isn't already set : ${EDITOR:='/usr/bin/vi'} } # Perform sanity checks and set some final parameters in # preparation for installing updates. install_check_params () { # Check that we are root. All sorts of things won't work otherwise. if [ `id -u` != 0 ]; then echo "You must be root to run this." exit 1 fi # Check that securelevel <= 0. Otherwise we can't update schg files. if [ `sysctl -n kern.securelevel` -gt 0 ]; then echo "Updates cannot be installed when the system securelevel" echo "is greater than zero." exit 1 fi # Check that we have a working directory _WORKDIR_bad="Directory does not exist or is not writable: " if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then echo -n "`basename $0`: " echo -n "${_WORKDIR_bad}" echo ${WORKDIR} exit 1 fi cd ${WORKDIR} || exit 1 # Construct a unique name from ${BASEDIR} BDHASH=`echo ${BASEDIR} | sha256 -q` # Check that we have updates ready to install if ! [ -L ${BDHASH}-install ]; then echo "No updates are available to install." echo "Run '$0 fetch' first." exit 1 fi if ! [ -f ${BDHASH}-install/INDEX-OLD ] || ! [ -f ${BDHASH}-install/INDEX-NEW ]; then echo "Update manifest is corrupt -- this should never happen." echo "Re-run '$0 fetch'." exit 1 fi # Figure out what directory contains the running kernel BOOTFILE=`sysctl -n kern.bootfile` KERNELDIR=${BOOTFILE%/kernel} if ! [ -d ${KERNELDIR} ]; then echo "Cannot identify running kernel" exit 1 fi } # Perform sanity checks and set some final parameters in # preparation for UNinstalling updates. rollback_check_params () { # Check that we are root. All sorts of things won't work otherwise. if [ `id -u` != 0 ]; then echo "You must be root to run this." exit 1 fi # Check that we have a working directory _WORKDIR_bad="Directory does not exist or is not writable: " if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then echo -n "`basename $0`: " echo -n "${_WORKDIR_bad}" echo ${WORKDIR} exit 1 fi cd ${WORKDIR} || exit 1 # Construct a unique name from ${BASEDIR} BDHASH=`echo ${BASEDIR} | sha256 -q` # Check that we have updates ready to rollback if ! [ -L ${BDHASH}-rollback ]; then echo "No rollback directory found." exit 1 fi if ! [ -f ${BDHASH}-rollback/INDEX-OLD ] || ! [ -f ${BDHASH}-rollback/INDEX-NEW ]; then echo "Update manifest is corrupt -- this should never happen." exit 1 fi } # Perform sanity checks and set some final parameters # in preparation for comparing the system against the # published index. Figure out which index we should # compare against: If the user is running *-p[0-9]+, # strip off the last part; if the user is running # -SECURITY, call it -RELEASE. Chdir into the working # directory. IDS_check_params () { export HTTP_USER_AGENT="freebsd-update (${COMMAND}, `uname -r`)" _SERVERNAME_z=\ "SERVERNAME must be given via command line or configuration file." _KEYPRINT_z="Key must be given via -k option or configuration file." _KEYPRINT_bad="Invalid key fingerprint: " _WORKDIR_bad="Directory does not exist or is not writable: " if [ -z "${SERVERNAME}" ]; then echo -n "`basename $0`: " echo "${_SERVERNAME_z}" exit 1 fi if [ -z "${KEYPRINT}" ]; then echo -n "`basename $0`: " echo "${_KEYPRINT_z}" exit 1 fi if ! echo "${KEYPRINT}" | grep -qE "^[0-9a-f]{64}$"; then echo -n "`basename $0`: " echo -n "${_KEYPRINT_bad}" echo ${KEYPRINT} exit 1 fi if ! [ -d "${WORKDIR}" -a -w "${WORKDIR}" ]; then echo -n "`basename $0`: " echo -n "${_WORKDIR_bad}" echo ${WORKDIR} exit 1 fi cd ${WORKDIR} || exit 1 # Generate release number. The s/SECURITY/RELEASE/ bit exists # to provide an upgrade path for FreeBSD Update 1.x users, since # the kernels provided by FreeBSD Update 1.x are always labelled # as X.Y-SECURITY. RELNUM=`uname -r | sed -E 's,-p[0-9]+,,' | sed -E 's,-SECURITY,-RELEASE,'` ARCH=`uname -m` FETCHDIR=${RELNUM}/${ARCH} PATCHDIR=${RELNUM}/${ARCH}/bp # Figure out what directory contains the running kernel BOOTFILE=`sysctl -n kern.bootfile` KERNELDIR=${BOOTFILE%/kernel} if ! [ -d ${KERNELDIR} ]; then echo "Cannot identify running kernel" exit 1 fi # Figure out what kernel configuration is running. We start with # the output of `uname -i`, and then make the following adjustments: # 1. Replace "SMP-GENERIC" with "SMP". Why the SMP kernel config # file says "ident SMP-GENERIC", I don't know... # 2. If the kernel claims to be GENERIC _and_ ${ARCH} is "amd64" # _and_ `sysctl kern.version` contains a line which ends "/SMP", then # we're running an SMP kernel. This mis-identification is a bug # which was fixed in 6.2-STABLE. KERNCONF=`uname -i` if [ ${KERNCONF} = "SMP-GENERIC" ]; then KERNCONF=SMP fi if [ ${KERNCONF} = "GENERIC" ] && [ ${ARCH} = "amd64" ]; then if sysctl kern.version | grep -qE '/SMP$'; then KERNCONF=SMP fi fi # Define some paths SHA256=/sbin/sha256 PHTTPGET=/usr/libexec/phttpget # Set up variables relating to VERBOSELEVEL fetch_setup_verboselevel } #### Core functionality -- the actual work gets done here # Use an SRV query to pick a server. If the SRV query doesn't provide # a useful answer, use the server name specified by the user. # Put another way... look up _http._tcp.${SERVERNAME} and pick a server # from that; or if no servers are returned, use ${SERVERNAME}. # This allows a user to specify "portsnap.freebsd.org" (in which case # portsnap will select one of the mirrors) or "portsnap5.tld.freebsd.org" # (in which case portsnap will use that particular server, since there # won't be an SRV entry for that name). # # We ignore the Port field, since we are always going to use port 80. # Fetch the mirror list, but do not pick a mirror yet. Returns 1 if # no mirrors are available for any reason. fetch_pick_server_init () { : > serverlist_tried # Check that host(1) exists (i.e., that the system wasn't built with the # WITHOUT_BIND set) and don't try to find a mirror if it doesn't exist. if ! which -s host; then : > serverlist_full return 1 fi echo -n "Looking up ${SERVERNAME} mirrors... " # Issue the SRV query and pull out the Priority, Weight, and Target fields. # BIND 9 prints "$name has SRV record ..." while BIND 8 prints # "$name server selection ..."; we allow either format. MLIST="_http._tcp.${SERVERNAME}" host -t srv "${MLIST}" | sed -nE "s/${MLIST} (has SRV record|server selection) //p" | cut -f 1,2,4 -d ' ' | sed -e 's/\.$//' | sort > serverlist_full # If no records, give up -- we'll just use the server name we were given. if [ `wc -l < serverlist_full` -eq 0 ]; then echo "none found." return 1 fi # Report how many mirrors we found. echo `wc -l < serverlist_full` "mirrors found." # Generate a random seed for use in picking mirrors. If HTTP_PROXY # is set, this will be used to generate the seed; otherwise, the seed # will be random. if [ -n "${HTTP_PROXY}${http_proxy}" ]; then RANDVALUE=`sha256 -qs "${HTTP_PROXY}${http_proxy}" | tr -d 'a-f' | cut -c 1-9` else RANDVALUE=`jot -r 1 0 999999999` fi } # Pick a mirror. Returns 1 if we have run out of mirrors to try. fetch_pick_server () { # Generate a list of not-yet-tried mirrors sort serverlist_tried | comm -23 serverlist_full - > serverlist # Have we run out of mirrors? if [ `wc -l < serverlist` -eq 0 ]; then echo "No mirrors remaining, giving up." return 1 fi # Find the highest priority level (lowest numeric value). SRV_PRIORITY=`cut -f 1 -d ' ' serverlist | sort -n | head -1` # Add up the weights of the response lines at that priority level. SRV_WSUM=0; while read X; do case "$X" in ${SRV_PRIORITY}\ *) SRV_W=`echo $X | cut -f 2 -d ' '` SRV_WSUM=$(($SRV_WSUM + $SRV_W)) ;; esac done < serverlist # If all the weights are 0, pretend that they are all 1 instead. if [ ${SRV_WSUM} -eq 0 ]; then SRV_WSUM=`grep -E "^${SRV_PRIORITY} " serverlist | wc -l` SRV_W_ADD=1 else SRV_W_ADD=0 fi # Pick a value between 0 and the sum of the weights - 1 SRV_RND=`expr ${RANDVALUE} % ${SRV_WSUM}` # Read through the list of mirrors and set SERVERNAME. Write the line # corresponding to the mirror we selected into serverlist_tried so that # we won't try it again. while read X; do case "$X" in ${SRV_PRIORITY}\ *) SRV_W=`echo $X | cut -f 2 -d ' '` SRV_W=$(($SRV_W + $SRV_W_ADD)) if [ $SRV_RND -lt $SRV_W ]; then SERVERNAME=`echo $X | cut -f 3 -d ' '` echo "$X" >> serverlist_tried break else SRV_RND=$(($SRV_RND - $SRV_W)) fi ;; esac done < serverlist } # Take a list of ${oldhash}|${newhash} and output a list of needed patches, # i.e., those for which we have ${oldhash} and don't have ${newhash}. fetch_make_patchlist () { grep -vE "^([0-9a-f]{64})\|\1$" | tr '|' ' ' | while read X Y; do if [ -f "files/${Y}.gz" ] || [ ! -f "files/${X}.gz" ]; then continue fi echo "${X}|${Y}" done | uniq } # Print user-friendly progress statistics fetch_progress () { LNC=0 while read x; do LNC=$(($LNC + 1)) if [ $(($LNC % 10)) = 0 ]; then echo -n $LNC elif [ $(($LNC % 2)) = 0 ]; then echo -n . fi done echo -n " " } # Function for asking the user if everything is ok continuep () { while read -p "Does this look reasonable (y/n)? " CONTINUE; do case "${CONTINUE}" in y*) return 0 ;; n*) return 1 ;; esac done } # Initialize the working directory workdir_init () { mkdir -p files touch tINDEX.present } # Check that we have a public key with an appropriate hash, or # fetch the key if it doesn't exist. Returns 1 if the key has # not yet been fetched. fetch_key () { if [ -r pub.ssl ] && [ `${SHA256} -q pub.ssl` = ${KEYPRINT} ]; then return 0 fi echo -n "Fetching public key from ${SERVERNAME}... " rm -f pub.ssl fetch ${QUIETFLAG} http://${SERVERNAME}/${FETCHDIR}/pub.ssl \ 2>${QUIETREDIR} || true if ! [ -r pub.ssl ]; then echo "failed." return 1 fi if ! [ `${SHA256} -q pub.ssl` = ${KEYPRINT} ]; then echo "key has incorrect hash." rm -f pub.ssl return 1 fi echo "done." } # Fetch metadata signature, aka "tag". fetch_tag () { echo -n "Fetching metadata signature " echo ${NDEBUG} "for ${RELNUM} from ${SERVERNAME}... " rm -f latest.ssl fetch ${QUIETFLAG} http://${SERVERNAME}/${FETCHDIR}/latest.ssl \ 2>${QUIETREDIR} || true if ! [ -r latest.ssl ]; then echo "failed." return 1 fi openssl rsautl -pubin -inkey pub.ssl -verify \ < latest.ssl > tag.new 2>${QUIETREDIR} || true rm latest.ssl if ! [ `wc -l < tag.new` = 1 ] || ! grep -qE \ "^freebsd-update\|${ARCH}\|${RELNUM}\|[0-9]+\|[0-9a-f]{64}\|[0-9]{10}" \ tag.new; then echo "invalid signature." return 1 fi echo "done." RELPATCHNUM=`cut -f 4 -d '|' < tag.new` TINDEXHASH=`cut -f 5 -d '|' < tag.new` EOLTIME=`cut -f 6 -d '|' < tag.new` } # Sanity-check the patch number in a tag, to make sure that we're not # going to "update" backwards and to prevent replay attacks. fetch_tagsanity () { # Check that we're not going to move from -pX to -pY with Y < X. RELPX=`uname -r | sed -E 's,.*-,,'` if echo ${RELPX} | grep -qE '^p[0-9]+$'; then RELPX=`echo ${RELPX} | cut -c 2-` else RELPX=0 fi if [ "${RELPATCHNUM}" -lt "${RELPX}" ]; then echo echo -n "Files on mirror (${RELNUM}-p${RELPATCHNUM})" echo " appear older than what" echo "we are currently running (`uname -r`)!" echo "Cowardly refusing to proceed any further." return 1 fi # If "tag" exists and corresponds to ${RELNUM}, make sure that # it contains a patch number <= RELPATCHNUM, in order to protect # against rollback (replay) attacks. if [ -f tag ] && grep -qE \ "^freebsd-update\|${ARCH}\|${RELNUM}\|[0-9]+\|[0-9a-f]{64}\|[0-9]{10}" \ tag; then LASTRELPATCHNUM=`cut -f 4 -d '|' < tag` if [ "${RELPATCHNUM}" -lt "${LASTRELPATCHNUM}" ]; then echo echo -n "Files on mirror (${RELNUM}-p${RELPATCHNUM})" echo " are older than the" echo -n "most recently seen updates" echo " (${RELNUM}-p${LASTRELPATCHNUM})." echo "Cowardly refusing to proceed any further." return 1 fi fi } # Fetch metadata index file fetch_metadata_index () { echo ${NDEBUG} "Fetching metadata index... " rm -f ${TINDEXHASH} fetch ${QUIETFLAG} http://${SERVERNAME}/${FETCHDIR}/t/${TINDEXHASH} 2>${QUIETREDIR} if ! [ -f ${TINDEXHASH} ]; then echo "failed." return 1 fi if [ `${SHA256} -q ${TINDEXHASH}` != ${TINDEXHASH} ]; then echo "update metadata index corrupt." return 1 fi echo "done." } # Print an error message about signed metadata being bogus. fetch_metadata_bogus () { echo echo "The update metadata$1 is correctly signed, but" echo "failed an integrity check." echo "Cowardly refusing to proceed any further." return 1 } # Construct tINDEX.new by merging the lines named in $1 from ${TINDEXHASH} # with the lines not named in $@ from tINDEX.present (if that file exists). fetch_metadata_index_merge () { for METAFILE in $@; do if [ `grep -E "^${METAFILE}\|" ${TINDEXHASH} | wc -l` \ -ne 1 ]; then fetch_metadata_bogus " index" return 1 fi grep -E "${METAFILE}\|" ${TINDEXHASH} done | sort > tINDEX.wanted if [ -f tINDEX.present ]; then join -t '|' -v 2 tINDEX.wanted tINDEX.present | sort -m - tINDEX.wanted > tINDEX.new rm tINDEX.wanted else mv tINDEX.wanted tINDEX.new fi } # Sanity check all the lines of tINDEX.new. Even if more metadata lines # are added by future versions of the server, this won't cause problems, # since the only lines which appear in tINDEX.new are the ones which we # specifically grepped out of ${TINDEXHASH}. fetch_metadata_index_sanity () { if grep -qvE '^[0-9A-Z.-]+\|[0-9a-f]{64}$' tINDEX.new; then fetch_metadata_bogus " index" return 1 fi } # Sanity check the metadata file $1. fetch_metadata_sanity () { # Some aliases to save space later: ${P} is a character which can # appear in a path; ${M} is the four numeric metadata fields; and # ${H} is a sha256 hash. P="[-+./:=%@_[~[:alnum:]]" M="[0-9]+\|[0-9]+\|[0-9]+\|[0-9]+" H="[0-9a-f]{64}" # Check that the first four fields make sense. if gunzip -c < files/$1.gz | grep -qvE "^[a-z]+\|[0-9a-z]+\|${P}+\|[fdL-]\|"; then fetch_metadata_bogus "" return 1 fi # Remove the first three fields. gunzip -c < files/$1.gz | cut -f 4- -d '|' > sanitycheck.tmp # Sanity check entries with type 'f' if grep -E '^f' sanitycheck.tmp | grep -qvE "^f\|${M}\|${H}\|${P}*\$"; then fetch_metadata_bogus "" return 1 fi # Sanity check entries with type 'd' if grep -E '^d' sanitycheck.tmp | grep -qvE "^d\|${M}\|\|\$"; then fetch_metadata_bogus "" return 1 fi # Sanity check entries with type 'L' if grep -E '^L' sanitycheck.tmp | grep -qvE "^L\|${M}\|${P}*\|\$"; then fetch_metadata_bogus "" return 1 fi # Sanity check entries with type '-' if grep -E '^-' sanitycheck.tmp | grep -qvE "^-\|\|\|\|\|\|"; then fetch_metadata_bogus "" return 1 fi # Clean up rm sanitycheck.tmp } # Fetch the metadata index and metadata files listed in $@, # taking advantage of metadata patches where possible. fetch_metadata () { fetch_metadata_index || return 1 fetch_metadata_index_merge $@ || return 1 fetch_metadata_index_sanity || return 1 # Generate a list of wanted metadata patches join -t '|' -o 1.2,2.2 tINDEX.present tINDEX.new | fetch_make_patchlist > patchlist if [ -s patchlist ]; then # Attempt to fetch metadata patches echo -n "Fetching `wc -l < patchlist | tr -d ' '` " echo ${NDEBUG} "metadata patches.${DDSTATS}" tr '|' '-' < patchlist | lam -s "${FETCHDIR}/tp/" - -s ".gz" | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${STATSREDIR} | fetch_progress echo "done." # Attempt to apply metadata patches echo -n "Applying metadata patches... " tr '|' ' ' < patchlist | while read X Y; do if [ ! -f "${X}-${Y}.gz" ]; then continue; fi gunzip -c < ${X}-${Y}.gz > diff gunzip -c < files/${X}.gz > diff-OLD # Figure out which lines are being added and removed grep -E '^-' diff | cut -c 2- | while read PREFIX; do look "${PREFIX}" diff-OLD done | sort > diff-rm grep -E '^\+' diff | cut -c 2- > diff-add # Generate the new file comm -23 diff-OLD diff-rm | sort - diff-add > diff-NEW if [ `${SHA256} -q diff-NEW` = ${Y} ]; then mv diff-NEW files/${Y} gzip -n files/${Y} else mv diff-NEW ${Y}.bad fi rm -f ${X}-${Y}.gz diff rm -f diff-OLD diff-NEW diff-add diff-rm done 2>${QUIETREDIR} echo "done." fi # Update metadata without patches cut -f 2 -d '|' < tINDEX.new | while read Y; do if [ ! -f "files/${Y}.gz" ]; then echo ${Y}; fi done | sort -u > filelist if [ -s filelist ]; then echo -n "Fetching `wc -l < filelist | tr -d ' '` " echo ${NDEBUG} "metadata files... " lam -s "${FETCHDIR}/m/" - -s ".gz" < filelist | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${QUIETREDIR} while read Y; do if ! [ -f ${Y}.gz ]; then echo "failed." return 1 fi if [ `gunzip -c < ${Y}.gz | ${SHA256} -q` = ${Y} ]; then mv ${Y}.gz files/${Y}.gz else echo "metadata is corrupt." return 1 fi done < filelist echo "done." fi # Sanity-check the metadata files. cut -f 2 -d '|' tINDEX.new > filelist while read X; do fetch_metadata_sanity ${X} || return 1 done < filelist # Remove files which are no longer needed cut -f 2 -d '|' tINDEX.present | sort > oldfiles cut -f 2 -d '|' tINDEX.new | sort | comm -13 - oldfiles | lam -s "files/" - -s ".gz" | xargs rm -f rm patchlist filelist oldfiles rm ${TINDEXHASH} # We're done! mv tINDEX.new tINDEX.present mv tag.new tag return 0 } # Extract a subset of a downloaded metadata file containing only the parts # which are listed in COMPONENTS. fetch_filter_metadata_components () { METAHASH=`look "$1|" tINDEX.present | cut -f 2 -d '|'` gunzip -c < files/${METAHASH}.gz > $1.all # Fish out the lines belonging to components we care about. for C in ${COMPONENTS}; do look "`echo ${C} | tr '/' '|'`|" $1.all done > $1 # Remove temporary file. rm $1.all } # Generate a filtered version of the metadata file $1 from the downloaded # file, by fishing out the lines corresponding to components we're trying # to keep updated, and then removing lines corresponding to paths we want # to ignore. fetch_filter_metadata () { # Fish out the lines belonging to components we care about. fetch_filter_metadata_components $1 # Canonicalize directory names by removing any trailing / in # order to avoid listing directories multiple times if they # belong to multiple components. Turning "/" into "" doesn't # matter, since we add a leading "/" when we use paths later. cut -f 3- -d '|' $1 | sed -e 's,/|d|,|d|,' | sed -e 's,/|-|,|-|,' | sort -u > $1.tmp # Figure out which lines to ignore and remove them. for X in ${IGNOREPATHS}; do grep -E "^${X}" $1.tmp done | sort -u | comm -13 - $1.tmp > $1 # Remove temporary files. rm $1.tmp } # Filter the metadata file $1 by adding lines with "/boot/$2" # replaced by ${KERNELDIR} (which is `sysctl -n kern.bootfile` minus the # trailing "/kernel"); and if "/boot/$2" does not exist, remove # the original lines which start with that. # Put another way: Deal with the fact that the FOO kernel is sometimes # installed in /boot/FOO/ and is sometimes installed elsewhere. fetch_filter_kernel_names () { grep ^/boot/$2 $1 | sed -e "s,/boot/$2,${KERNELDIR},g" | sort - $1 > $1.tmp mv $1.tmp $1 if ! [ -d /boot/$2 ]; then grep -v ^/boot/$2 $1 > $1.tmp mv $1.tmp $1 fi } # For all paths appearing in $1 or $3, inspect the system # and generate $2 describing what is currently installed. fetch_inspect_system () { # No errors yet... rm -f .err # Tell the user why his disk is suddenly making lots of noise echo -n "Inspecting system... " # Generate list of files to inspect cat $1 $3 | cut -f 1 -d '|' | sort -u > filelist # Examine each file and output lines of the form # /path/to/file|type|device-inum|user|group|perm|flags|value # sorted by device and inode number. while read F; do # If the symlink/file/directory does not exist, record this. if ! [ -e ${BASEDIR}/${F} ]; then echo "${F}|-||||||" continue fi if ! [ -r ${BASEDIR}/${F} ]; then echo "Cannot read file: ${BASEDIR}/${F}" \ >/dev/stderr touch .err return 1 fi # Otherwise, output an index line. if [ -L ${BASEDIR}/${F} ]; then echo -n "${F}|L|" stat -n -f '%d-%i|%u|%g|%Mp%Lp|%Of|' ${BASEDIR}/${F}; readlink ${BASEDIR}/${F}; elif [ -f ${BASEDIR}/${F} ]; then echo -n "${F}|f|" stat -n -f '%d-%i|%u|%g|%Mp%Lp|%Of|' ${BASEDIR}/${F}; sha256 -q ${BASEDIR}/${F}; elif [ -d ${BASEDIR}/${F} ]; then echo -n "${F}|d|" stat -f '%d-%i|%u|%g|%Mp%Lp|%Of|' ${BASEDIR}/${F}; else echo "Unknown file type: ${BASEDIR}/${F}" \ >/dev/stderr touch .err return 1 fi done < filelist | sort -k 3,3 -t '|' > $2.tmp rm filelist # Check if an error occurred during system inspection if [ -f .err ]; then return 1 fi # Convert to the form # /path/to/file|type|user|group|perm|flags|value|hlink # by resolving identical device and inode numbers into hard links. cut -f 1,3 -d '|' $2.tmp | sort -k 1,1 -t '|' | sort -s -u -k 2,2 -t '|' | join -1 2 -2 3 -t '|' - $2.tmp | awk -F \| -v OFS=\| \ '{ if (($2 == $3) || ($4 == "-")) print $3,$4,$5,$6,$7,$8,$9,"" else print $3,$4,$5,$6,$7,$8,$9,$2 }' | sort > $2 rm $2.tmp # We're finished looking around echo "done." } # For any paths matching ${MERGECHANGES}, compare $1 and $2 and find any # files which differ; generate $3 containing these paths and the old hashes. fetch_filter_mergechanges () { # Pull out the paths and hashes of the files matching ${MERGECHANGES}. for F in $1 $2; do for X in ${MERGECHANGES}; do grep -E "^${X}" ${F} done | cut -f 1,2,7 -d '|' | sort > ${F}-values done # Any line in $2-values which doesn't appear in $1-values and is a # file means that we should list the path in $3. comm -13 $1-values $2-values | fgrep '|f|' | cut -f 1 -d '|' > $2-paths # For each path, pull out one (and only one!) entry from $1-values. # Note that we cannot distinguish which "old" version the user made # changes to; but hopefully any changes which occur due to security # updates will exist in both the "new" version and the version which # the user has installed, so the merging will still work. while read X; do look "${X}|" $1-values | head -1 done < $2-paths > $3 # Clean up rm $1-values $2-values $2-paths } # For any paths matching ${UPDATEIFUNMODIFIED}, remove lines from $[123] # which correspond to lines in $2 with hashes not matching $1 or $3, unless # the paths are listed in $4. For entries in $2 marked "not present" # (aka. type -), remove lines from $[123] unless there is a corresponding # entry in $1. fetch_filter_unmodified_notpresent () { # Figure out which lines of $1 and $3 correspond to bits which # should only be updated if they haven't changed, and fish out # the (path, type, value) tuples. # NOTE: We don't consider a file to be "modified" if it matches # the hash from $3. for X in ${UPDATEIFUNMODIFIED}; do grep -E "^${X}" $1 grep -E "^${X}" $3 done | cut -f 1,2,7 -d '|' | sort > $1-values # Do the same for $2. for X in ${UPDATEIFUNMODIFIED}; do grep -E "^${X}" $2 done | cut -f 1,2,7 -d '|' | sort > $2-values # Any entry in $2-values which is not in $1-values corresponds to # a path which we need to remove from $1, $2, and $3, unless it # that path appears in $4. comm -13 $1-values $2-values | sort -t '|' -k 1,1 > mlines.tmp cut -f 1 -d '|' $4 | sort | join -v 2 -t '|' - mlines.tmp | sort > mlines rm $1-values $2-values mlines.tmp # Any lines in $2 which are not in $1 AND are "not present" lines # also belong in mlines. comm -13 $1 $2 | cut -f 1,2,7 -d '|' | fgrep '|-|' >> mlines # Remove lines from $1, $2, and $3 for X in $1 $2 $3; do sort -t '|' -k 1,1 ${X} > ${X}.tmp cut -f 1 -d '|' < mlines | sort | join -v 2 -t '|' - ${X}.tmp | sort > ${X} rm ${X}.tmp done # Store a list of the modified files, for future reference fgrep -v '|-|' mlines | cut -f 1 -d '|' > modifiedfiles rm mlines } # For each entry in $1 of type -, remove any corresponding # entry from $2 if ${ALLOWADD} != "yes". Remove all entries # of type - from $1. fetch_filter_allowadd () { cut -f 1,2 -d '|' < $1 | fgrep '|-' | cut -f 1 -d '|' > filesnotpresent if [ ${ALLOWADD} != "yes" ]; then sort < $2 | join -v 1 -t '|' - filesnotpresent | sort > $2.tmp mv $2.tmp $2 fi sort < $1 | join -v 1 -t '|' - filesnotpresent | sort > $1.tmp mv $1.tmp $1 rm filesnotpresent } # If ${ALLOWDELETE} != "yes", then remove any entries from $1 # which don't correspond to entries in $2. fetch_filter_allowdelete () { # Produce a lists ${PATH}|${TYPE} for X in $1 $2; do cut -f 1-2 -d '|' < ${X} | sort -u > ${X}.nodes done # Figure out which lines need to be removed from $1. if [ ${ALLOWDELETE} != "yes" ]; then comm -23 $1.nodes $2.nodes > $1.badnodes else : > $1.badnodes fi # Remove the relevant lines from $1 while read X; do look "${X}|" $1 done < $1.badnodes | comm -13 - $1 > $1.tmp mv $1.tmp $1 rm $1.badnodes $1.nodes $2.nodes } # If ${KEEPMODIFIEDMETADATA} == "yes", then for each entry in $2 # with metadata not matching any entry in $1, replace the corresponding # line of $3 with one having the same metadata as the entry in $2. fetch_filter_modified_metadata () { # Fish out the metadata from $1 and $2 for X in $1 $2; do cut -f 1-6 -d '|' < ${X} > ${X}.metadata done # Find the metadata we need to keep if [ ${KEEPMODIFIEDMETADATA} = "yes" ]; then comm -13 $1.metadata $2.metadata > keepmeta else : > keepmeta fi # Extract the lines which we need to remove from $3, and # construct the lines which we need to add to $3. : > $3.remove : > $3.add while read LINE; do NODE=`echo "${LINE}" | cut -f 1-2 -d '|'` look "${NODE}|" $3 >> $3.remove look "${NODE}|" $3 | cut -f 7- -d '|' | lam -s "${LINE}|" - >> $3.add done < keepmeta # Remove the specified lines and add the new lines. sort $3.remove | comm -13 - $3 | sort -u - $3.add > $3.tmp mv $3.tmp $3 rm keepmeta $1.metadata $2.metadata $3.add $3.remove } # Remove lines from $1 and $2 which are identical; # no need to update a file if it isn't changing. fetch_filter_uptodate () { comm -23 $1 $2 > $1.tmp comm -13 $1 $2 > $2.tmp mv $1.tmp $1 mv $2.tmp $2 } # Fetch any "clean" old versions of files we need for merging changes. fetch_files_premerge () { # We only need to do anything if $1 is non-empty. if [ -s $1 ]; then # Tell the user what we're doing echo -n "Fetching files from ${OLDRELNUM} for merging... " # List of files wanted fgrep '|f|' < $1 | cut -f 3 -d '|' | sort -u > files.wanted # Only fetch the files we don't already have while read Y; do if [ ! -f "files/${Y}.gz" ]; then echo ${Y}; fi done < files.wanted > filelist # Actually fetch them lam -s "${OLDFETCHDIR}/f/" - -s ".gz" < filelist | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${QUIETREDIR} # Make sure we got them all, and move them into /files/ while read Y; do if ! [ -f ${Y}.gz ]; then echo "failed." return 1 fi if [ `gunzip -c < ${Y}.gz | ${SHA256} -q` = ${Y} ]; then mv ${Y}.gz files/${Y}.gz else echo "${Y} has incorrect hash." return 1 fi done < filelist echo "done." # Clean up rm filelist files.wanted fi } # Prepare to fetch files: Generate a list of the files we need, # copy the unmodified files we have into /files/, and generate # a list of patches to download. fetch_files_prepare () { # Tell the user why his disk is suddenly making lots of noise echo -n "Preparing to download files... " # Reduce indices to ${PATH}|${HASH} pairs for X in $1 $2 $3; do cut -f 1,2,7 -d '|' < ${X} | fgrep '|f|' | cut -f 1,3 -d '|' | sort > ${X}.hashes done # List of files wanted cut -f 2 -d '|' < $3.hashes | sort -u | while read HASH; do if ! [ -f files/${HASH}.gz ]; then echo ${HASH} fi done > files.wanted # Generate a list of unmodified files comm -12 $1.hashes $2.hashes | sort -k 1,1 -t '|' > unmodified.files # Copy all files into /files/. We only need the unmodified files # for use in patching; but we'll want all of them if the user asks # to rollback the updates later. while read LINE; do F=`echo "${LINE}" | cut -f 1 -d '|'` HASH=`echo "${LINE}" | cut -f 2 -d '|'` # Skip files we already have. if [ -f files/${HASH}.gz ]; then continue fi # Make sure the file hasn't changed. cp "${BASEDIR}/${F}" tmpfile if [ `sha256 -q tmpfile` != ${HASH} ]; then echo echo "File changed while FreeBSD Update running: ${F}" return 1 fi # Place the file into storage. gzip -c < tmpfile > files/${HASH}.gz rm tmpfile done < $2.hashes # Produce a list of patches to download sort -k 1,1 -t '|' $3.hashes | join -t '|' -o 2.2,1.2 - unmodified.files | fetch_make_patchlist > patchlist # Garbage collect rm unmodified.files $1.hashes $2.hashes $3.hashes # We don't need the list of possible old files any more. rm $1 # We're finished making noise echo "done." } # Fetch files. fetch_files () { # Attempt to fetch patches if [ -s patchlist ]; then echo -n "Fetching `wc -l < patchlist | tr -d ' '` " echo ${NDEBUG} "patches.${DDSTATS}" tr '|' '-' < patchlist | lam -s "${PATCHDIR}/" - | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${STATSREDIR} | fetch_progress echo "done." # Attempt to apply patches echo -n "Applying patches... " tr '|' ' ' < patchlist | while read X Y; do if [ ! -f "${X}-${Y}" ]; then continue; fi gunzip -c < files/${X}.gz > OLD bspatch OLD NEW ${X}-${Y} if [ `${SHA256} -q NEW` = ${Y} ]; then mv NEW files/${Y} gzip -n files/${Y} fi rm -f diff OLD NEW ${X}-${Y} done 2>${QUIETREDIR} echo "done." fi # Download files which couldn't be generate via patching while read Y; do if [ ! -f "files/${Y}.gz" ]; then echo ${Y}; fi done < files.wanted > filelist if [ -s filelist ]; then echo -n "Fetching `wc -l < filelist | tr -d ' '` " echo ${NDEBUG} "files... " lam -s "${FETCHDIR}/f/" - -s ".gz" < filelist | xargs ${XARGST} ${PHTTPGET} ${SERVERNAME} \ 2>${QUIETREDIR} while read Y; do if ! [ -f ${Y}.gz ]; then echo "failed." return 1 fi if [ `gunzip -c < ${Y}.gz | ${SHA256} -q` = ${Y} ]; then mv ${Y}.gz files/${Y}.gz else echo "${Y} has incorrect hash." return 1 fi done < filelist echo "done." fi # Clean up rm files.wanted filelist patchlist } # Create and populate install manifest directory; and report what updates # are available. fetch_create_manifest () { # If we have an existing install manifest, nuke it. if [ -L "${BDHASH}-install" ]; then rm -r ${BDHASH}-install/ rm ${BDHASH}-install fi # Report to the user if any updates were avoided due to local changes if [ -s modifiedfiles ]; then echo echo -n "The following files are affected by updates, " echo "but no changes have" echo -n "been downloaded because the files have been " echo "modified locally:" cat modifiedfiles fi | $PAGER rm modifiedfiles # If no files will be updated, tell the user and exit if ! [ -s INDEX-PRESENT ] && ! [ -s INDEX-NEW ]; then rm INDEX-PRESENT INDEX-NEW echo echo -n "No updates needed to update system to " echo "${RELNUM}-p${RELPATCHNUM}." return fi # Divide files into (a) removed files, (b) added files, and # (c) updated files. cut -f 1 -d '|' < INDEX-PRESENT | sort > INDEX-PRESENT.flist cut -f 1 -d '|' < INDEX-NEW | sort > INDEX-NEW.flist comm -23 INDEX-PRESENT.flist INDEX-NEW.flist > files.removed comm -13 INDEX-PRESENT.flist INDEX-NEW.flist > files.added comm -12 INDEX-PRESENT.flist INDEX-NEW.flist > files.updated rm INDEX-PRESENT.flist INDEX-NEW.flist # Report removed files, if any if [ -s files.removed ]; then echo echo -n "The following files will be removed " echo "as part of updating to ${RELNUM}-p${RELPATCHNUM}:" cat files.removed fi | $PAGER rm files.removed # Report added files, if any if [ -s files.added ]; then echo echo -n "The following files will be added " echo "as part of updating to ${RELNUM}-p${RELPATCHNUM}:" cat files.added fi | $PAGER rm files.added # Report updated files, if any if [ -s files.updated ]; then echo echo -n "The following files will be updated " echo "as part of updating to ${RELNUM}-p${RELPATCHNUM}:" cat files.updated fi | $PAGER rm files.updated # Create a directory for the install manifest. MDIR=`mktemp -d install.XXXXXX` || return 1 # Populate it mv INDEX-PRESENT ${MDIR}/INDEX-OLD mv INDEX-NEW ${MDIR}/INDEX-NEW # Link it into place ln -s ${MDIR} ${BDHASH}-install } # Warn about any upcoming EoL fetch_warn_eol () { # What's the current time? NOWTIME=`date "+%s"` # When did we last warn about the EoL date? if [ -f lasteolwarn ]; then LASTWARN=`cat lasteolwarn` else LASTWARN=`expr ${NOWTIME} - 63072000` fi # If the EoL time is past, warn. if [ ${EOLTIME} -lt ${NOWTIME} ]; then echo cat <<-EOF WARNING: `uname -sr` HAS PASSED ITS END-OF-LIFE DATE. Any security issues discovered after `date -r ${EOLTIME}` will not have been corrected. EOF return 1 fi # Figure out how long it has been since we last warned about the # upcoming EoL, and how much longer we have left. SINCEWARN=`expr ${NOWTIME} - ${LASTWARN}` TIMELEFT=`expr ${EOLTIME} - ${NOWTIME}` # Don't warn if the EoL is more than 3 months away if [ ${TIMELEFT} -gt 7884000 ]; then return 0 fi # Don't warn if the time remaining is more than 3 times the time # since the last warning. if [ ${TIMELEFT} -gt `expr ${SINCEWARN} \* 3` ]; then return 0 fi # Figure out what time units to use. if [ ${TIMELEFT} -lt 604800 ]; then UNIT="day" SIZE=86400 elif [ ${TIMELEFT} -lt 2678400 ]; then UNIT="week" SIZE=604800 else UNIT="month" SIZE=2678400 fi # Compute the right number of units NUM=`expr ${TIMELEFT} / ${SIZE}` if [ ${NUM} != 1 ]; then UNIT="${UNIT}s" fi # Print the warning echo cat <<-EOF WARNING: `uname -sr` is approaching its End-of-Life date. It is strongly recommended that you upgrade to a newer release within the next ${NUM} ${UNIT}. EOF # Update the stored time of last warning echo ${NOWTIME} > lasteolwarn } # Do the actual work involved in "fetch" / "cron". fetch_run () { workdir_init || return 1 # Prepare the mirror list. fetch_pick_server_init && fetch_pick_server # Try to fetch the public key until we run out of servers. while ! fetch_key; do fetch_pick_server || return 1 done # Try to fetch the metadata index signature ("tag") until we run # out of available servers; and sanity check the downloaded tag. while ! fetch_tag; do fetch_pick_server || return 1 done fetch_tagsanity || return 1 # Fetch the latest INDEX-NEW and INDEX-OLD files. fetch_metadata INDEX-NEW INDEX-OLD || return 1 # Generate filtered INDEX-NEW and INDEX-OLD files containing only # the lines which (a) belong to components we care about, and (b) # don't correspond to paths we're explicitly ignoring. fetch_filter_metadata INDEX-NEW || return 1 fetch_filter_metadata INDEX-OLD || return 1 # Translate /boot/${KERNCONF} into ${KERNELDIR} fetch_filter_kernel_names INDEX-NEW ${KERNCONF} fetch_filter_kernel_names INDEX-OLD ${KERNCONF} # For all paths appearing in INDEX-OLD or INDEX-NEW, inspect the # system and generate an INDEX-PRESENT file. fetch_inspect_system INDEX-OLD INDEX-PRESENT INDEX-NEW || return 1 # Based on ${UPDATEIFUNMODIFIED}, remove lines from INDEX-* which # correspond to lines in INDEX-PRESENT with hashes not appearing # in INDEX-OLD or INDEX-NEW. Also remove lines where the entry in # INDEX-PRESENT has type - and there isn't a corresponding entry in # INDEX-OLD with type -. fetch_filter_unmodified_notpresent \ INDEX-OLD INDEX-PRESENT INDEX-NEW /dev/null # For each entry in INDEX-PRESENT of type -, remove any corresponding # entry from INDEX-NEW if ${ALLOWADD} != "yes". Remove all entries # of type - from INDEX-PRESENT. fetch_filter_allowadd INDEX-PRESENT INDEX-NEW # If ${ALLOWDELETE} != "yes", then remove any entries from # INDEX-PRESENT which don't correspond to entries in INDEX-NEW. fetch_filter_allowdelete INDEX-PRESENT INDEX-NEW # If ${KEEPMODIFIEDMETADATA} == "yes", then for each entry in # INDEX-PRESENT with metadata not matching any entry in INDEX-OLD, # replace the corresponding line of INDEX-NEW with one having the # same metadata as the entry in INDEX-PRESENT. fetch_filter_modified_metadata INDEX-OLD INDEX-PRESENT INDEX-NEW # Remove lines from INDEX-PRESENT and INDEX-NEW which are identical; # no need to update a file if it isn't changing. fetch_filter_uptodate INDEX-PRESENT INDEX-NEW # Prepare to fetch files: Generate a list of the files we need, # copy the unmodified files we have into /files/, and generate # a list of patches to download. fetch_files_prepare INDEX-OLD INDEX-PRESENT INDEX-NEW || return 1 # Fetch files. fetch_files || return 1 # Create and populate install manifest directory; and report what # updates are available. fetch_create_manifest || return 1 # Warn about any upcoming EoL fetch_warn_eol || return 1 } # If StrictComponents is not "yes", generate a new components list # with only the components which appear to be installed. upgrade_guess_components () { if [ "${STRICTCOMPONENTS}" = "no" ]; then # Generate filtered INDEX-ALL with only the components listed # in COMPONENTS. fetch_filter_metadata_components $1 || return 1 # Tell the user why his disk is suddenly making lots of noise echo -n "Inspecting system... " # Look at the files on disk, and assume that a component is # supposed to be present if it is more than half-present. cut -f 1-3 -d '|' < INDEX-ALL | tr '|' ' ' | while read C S F; do if [ -e ${BASEDIR}/${F} ]; then echo "+ ${C}|${S}" fi echo "= ${C}|${S}" done | sort | uniq -c | sed -E 's,^ +,,' > compfreq grep ' = ' compfreq | cut -f 1,3 -d ' ' | sort -k 2,2 -t ' ' > compfreq.total grep ' + ' compfreq | cut -f 1,3 -d ' ' | sort -k 2,2 -t ' ' > compfreq.present join -t ' ' -1 2 -2 2 compfreq.present compfreq.total | while read S P T; do if [ ${P} -gt `expr ${T} / 2` ]; then echo ${S} fi done > comp.present cut -f 2 -d ' ' < compfreq.total > comp.total rm INDEX-ALL compfreq compfreq.total compfreq.present # We're done making noise. echo "done." # Sometimes the kernel isn't installed where INDEX-ALL # thinks that it should be: In particular, it is often in # /boot/kernel instead of /boot/GENERIC or /boot/SMP. To # deal with this, if "kernel|X" is listed in comp.total # (i.e., is a component which would be upgraded if it is # found to be present) we will add it to comp.present. # If "kernel|" is in comp.total but "kernel|X" is # not, we print a warning -- the user is running a kernel # which isn't part of the release. KCOMP=`echo ${KERNCONF} | tr 'A-Z' 'a-z'` grep -E "^kernel\|${KCOMP}\$" comp.total >> comp.present if grep -qE "^kernel\|" comp.total && ! grep -qE "^kernel\|${KCOMP}\$" comp.total; then cat <<-EOF WARNING: This system is running a "${KCOMP}" kernel, which is not a kernel configuration distributed as part of FreeBSD ${RELNUM}. This kernel will not be updated: you MUST update the kernel manually before running "$0 install". EOF fi # Re-sort the list of installed components and generate # the list of non-installed components. sort -u < comp.present > comp.present.tmp mv comp.present.tmp comp.present comm -13 comp.present comp.total > comp.absent # Ask the user to confirm that what we have is correct. To # reduce user confusion, translate "X|Y" back to "X/Y" (as # subcomponents must be listed in the configuration file). echo echo -n "The following components of FreeBSD " echo "seem to be installed:" tr '|' '/' < comp.present | fmt -72 echo echo -n "The following components of FreeBSD " echo "do not seem to be installed:" tr '|' '/' < comp.absent | fmt -72 echo continuep || return 1 echo # Suck the generated list of components into ${COMPONENTS}. # Note that comp.present.tmp is used due to issues with # pipelines and setting variables. COMPONENTS="" tr '|' '/' < comp.present > comp.present.tmp while read C; do COMPONENTS="${COMPONENTS} ${C}" done < comp.present.tmp # Delete temporary files rm comp.present comp.present.tmp comp.absent comp.total fi } # If StrictComponents is not "yes", COMPONENTS contains an entry # corresponding to the currently running kernel, and said kernel # does not exist in the new release, add "kernel/generic" to the # list of components. upgrade_guess_new_kernel () { if [ "${STRICTCOMPONENTS}" = "no" ]; then # Grab the unfiltered metadata file. METAHASH=`look "$1|" tINDEX.present | cut -f 2 -d '|'` gunzip -c < files/${METAHASH}.gz > $1.all # If "kernel/${KCOMP}" is in ${COMPONENTS} and that component # isn't in $1.all, we need to add kernel/generic. for C in ${COMPONENTS}; do if [ ${C} = "kernel/${KCOMP}" ] && ! grep -qE "^kernel\|${KCOMP}\|" $1.all; then COMPONENTS="${COMPONENTS} kernel/generic" NKERNCONF="GENERIC" cat <<-EOF WARNING: This system is running a "${KCOMP}" kernel, which is not a kernel configuration distributed as part of FreeBSD ${RELNUM}. As part of upgrading to FreeBSD ${RELNUM}, this kernel will be replaced with a "generic" kernel. EOF continuep || return 1 fi done # Don't need this any more... rm $1.all fi } # Convert INDEX-OLD (last release) and INDEX-ALL (new release) into # INDEX-OLD and INDEX-NEW files (in the sense of normal upgrades). upgrade_oldall_to_oldnew () { # For each ${F}|... which appears in INDEX-ALL but does not appear # in INDEX-OLD, add ${F}|-|||||| to INDEX-OLD. cut -f 1 -d '|' < $1 | sort -u > $1.paths cut -f 1 -d '|' < $2 | sort -u | comm -13 $1.paths - | lam - -s "|-||||||" | sort - $1 > $1.tmp mv $1.tmp $1 # Remove lines from INDEX-OLD which also appear in INDEX-ALL comm -23 $1 $2 > $1.tmp mv $1.tmp $1 # Remove lines from INDEX-ALL which have a file name not appearing # anywhere in INDEX-OLD (since these must be files which haven't # changed -- if they were new, there would be an entry of type "-"). cut -f 1 -d '|' < $1 | sort -u > $1.paths sort -k 1,1 -t '|' < $2 | join -t '|' - $1.paths | sort > $2.tmp rm $1.paths mv $2.tmp $2 # Rename INDEX-ALL to INDEX-NEW. mv $2 $3 } # Helper for upgrade_merge: Return zero true iff the two files differ only # in the contents of their $FreeBSD$ tags. samef () { X=`sed -E 's/\\$FreeBSD.*\\$/\$FreeBSD\$/' < $1 | ${SHA256}` Y=`sed -E 's/\\$FreeBSD.*\\$/\$FreeBSD\$/' < $2 | ${SHA256}` if [ $X = $Y ]; then return 0; else return 1; fi } # From the list of "old" files in $1, merge changes in $2 with those in $3, # and update $3 to reflect the hashes of merged files. upgrade_merge () { # We only need to do anything if $1 is non-empty. if [ -s $1 ]; then cut -f 1 -d '|' $1 | sort > $1-paths # Create staging area for merging files rm -rf merge/ while read F; do D=`dirname ${F}` mkdir -p merge/old/${D} mkdir -p merge/${OLDRELNUM}/${D} mkdir -p merge/${RELNUM}/${D} mkdir -p merge/new/${D} done < $1-paths # Copy in files while read F; do # Currently installed file V=`look "${F}|" $2 | cut -f 7 -d '|'` gunzip < files/${V}.gz > merge/old/${F} # Old release if look "${F}|" $1 | fgrep -q "|f|"; then V=`look "${F}|" $1 | cut -f 3 -d '|'` gunzip < files/${V}.gz \ > merge/${OLDRELNUM}/${F} fi # New release if look "${F}|" $3 | cut -f 1,2,7 -d '|' | fgrep -q "|f|"; then V=`look "${F}|" $3 | cut -f 7 -d '|'` gunzip < files/${V}.gz \ > merge/${RELNUM}/${F} fi done < $1-paths # Attempt to automatically merge changes echo -n "Attempting to automatically merge " echo -n "changes in files..." : > failed.merges while read F; do # If the file doesn't exist in the new release, # the result of "merging changes" is having the file # not exist. if ! [ -f merge/${RELNUM}/${F} ]; then continue fi # If the file didn't exist in the old release, we're # going to throw away the existing file and hope that # the version from the new release is what we want. if ! [ -f merge/${OLDRELNUM}/${F} ]; then cp merge/${RELNUM}/${F} merge/new/${F} continue fi # Some files need special treatment. case ${F} in /etc/spwd.db | /etc/pwd.db | /etc/login.conf.db) # Don't merge these -- we're rebuild them # after updates are installed. cp merge/old/${F} merge/new/${F} ;; *) if ! merge -p -L "current version" \ -L "${OLDRELNUM}" -L "${RELNUM}" \ merge/old/${F} \ merge/${OLDRELNUM}/${F} \ merge/${RELNUM}/${F} \ > merge/new/${F} 2>/dev/null; then echo ${F} >> failed.merges fi ;; esac done < $1-paths echo " done." # Ask the user to handle any files which didn't merge. while read F; do # If the installed file differs from the version in # the old release only due to $FreeBSD$ tag expansion # then just use the version in the new release. if samef merge/old/${F} merge/${OLDRELNUM}/${F}; then cp merge/${RELNUM}/${F} merge/new/${F} continue fi cat <<-EOF The following file could not be merged automatically: ${F} Press Enter to edit this file in ${EDITOR} and resolve the conflicts manually... EOF read dummy files/${V}.gz echo "${F}|${V}" fi done < $1-paths > newhashes # Pull lines out from $3 which need to be updated to # reflect merged files. while read F; do look "${F}|" $3 done < $1-paths > $3-oldlines # Update lines to reflect merged files join -t '|' -o 1.1,1.2,1.3,1.4,1.5,1.6,2.2,1.8 \ $3-oldlines newhashes > $3-newlines # Remove old lines from $3 and add new lines. sort $3-oldlines | comm -13 - $3 | sort - $3-newlines > $3.tmp mv $3.tmp $3 # Clean up rm $1-paths newhashes $3-oldlines $3-newlines rm -rf merge/ fi # We're done with merging files. rm $1 } # Do the work involved in fetching upgrades to a new release upgrade_run () { workdir_init || return 1 # Prepare the mirror list. fetch_pick_server_init && fetch_pick_server # Try to fetch the public key until we run out of servers. while ! fetch_key; do fetch_pick_server || return 1 done # Try to fetch the metadata index signature ("tag") until we run # out of available servers; and sanity check the downloaded tag. while ! fetch_tag; do fetch_pick_server || return 1 done fetch_tagsanity || return 1 # Fetch the INDEX-OLD and INDEX-ALL. fetch_metadata INDEX-OLD INDEX-ALL || return 1 # If StrictComponents is not "yes", generate a new components list # with only the components which appear to be installed. upgrade_guess_components INDEX-ALL || return 1 # Generate filtered INDEX-OLD and INDEX-ALL files containing only # the components we want and without anything marked as "Ignore". fetch_filter_metadata INDEX-OLD || return 1 fetch_filter_metadata INDEX-ALL || return 1 # Merge the INDEX-OLD and INDEX-ALL files into INDEX-OLD. sort INDEX-OLD INDEX-ALL > INDEX-OLD.tmp mv INDEX-OLD.tmp INDEX-OLD rm INDEX-ALL # Adjust variables for fetching files from the new release. OLDRELNUM=${RELNUM} RELNUM=${TARGETRELEASE} OLDFETCHDIR=${FETCHDIR} FETCHDIR=${RELNUM}/${ARCH} # Try to fetch the NEW metadata index signature ("tag") until we run # out of available servers; and sanity check the downloaded tag. while ! fetch_tag; do fetch_pick_server || return 1 done # Fetch the new INDEX-ALL. fetch_metadata INDEX-ALL || return 1 # If StrictComponents is not "yes", COMPONENTS contains an entry # corresponding to the currently running kernel, and said kernel # does not exist in the new release, add "kernel/generic" to the # list of components. upgrade_guess_new_kernel INDEX-ALL || return 1 # Filter INDEX-ALL to contain only the components we want and without # anything marked as "Ignore". fetch_filter_metadata INDEX-ALL || return 1 # Convert INDEX-OLD (last release) and INDEX-ALL (new release) into # INDEX-OLD and INDEX-NEW files (in the sense of normal upgrades). upgrade_oldall_to_oldnew INDEX-OLD INDEX-ALL INDEX-NEW # Translate /boot/${KERNCONF} or /boot/${NKERNCONF} into ${KERNELDIR} fetch_filter_kernel_names INDEX-NEW ${NKERNCONF} fetch_filter_kernel_names INDEX-OLD ${KERNCONF} # For all paths appearing in INDEX-OLD or INDEX-NEW, inspect the # system and generate an INDEX-PRESENT file. fetch_inspect_system INDEX-OLD INDEX-PRESENT INDEX-NEW || return 1 # Based on ${MERGECHANGES}, generate a file tomerge-old with the # paths and hashes of old versions of files to merge. fetch_filter_mergechanges INDEX-OLD INDEX-PRESENT tomerge-old # Based on ${UPDATEIFUNMODIFIED}, remove lines from INDEX-* which # correspond to lines in INDEX-PRESENT with hashes not appearing # in INDEX-OLD or INDEX-NEW. Also remove lines where the entry in # INDEX-PRESENT has type - and there isn't a corresponding entry in # INDEX-OLD with type -. fetch_filter_unmodified_notpresent \ INDEX-OLD INDEX-PRESENT INDEX-NEW tomerge-old # For each entry in INDEX-PRESENT of type -, remove any corresponding # entry from INDEX-NEW if ${ALLOWADD} != "yes". Remove all entries # of type - from INDEX-PRESENT. fetch_filter_allowadd INDEX-PRESENT INDEX-NEW # If ${ALLOWDELETE} != "yes", then remove any entries from # INDEX-PRESENT which don't correspond to entries in INDEX-NEW. fetch_filter_allowdelete INDEX-PRESENT INDEX-NEW # If ${KEEPMODIFIEDMETADATA} == "yes", then for each entry in # INDEX-PRESENT with metadata not matching any entry in INDEX-OLD, # replace the corresponding line of INDEX-NEW with one having the # same metadata as the entry in INDEX-PRESENT. fetch_filter_modified_metadata INDEX-OLD INDEX-PRESENT INDEX-NEW # Remove lines from INDEX-PRESENT and INDEX-NEW which are identical; # no need to update a file if it isn't changing. fetch_filter_uptodate INDEX-PRESENT INDEX-NEW # Fetch "clean" files from the old release for merging changes. fetch_files_premerge tomerge-old # Prepare to fetch files: Generate a list of the files we need, # copy the unmodified files we have into /files/, and generate # a list of patches to download. fetch_files_prepare INDEX-OLD INDEX-PRESENT INDEX-NEW || return 1 # Fetch patches from to-${RELNUM}/${ARCH}/bp/ PATCHDIR=to-${RELNUM}/${ARCH}/bp fetch_files || return 1 # Merge configuration file changes. upgrade_merge tomerge-old INDEX-PRESENT INDEX-NEW || return 1 # Create and populate install manifest directory; and report what # updates are available. fetch_create_manifest || return 1 # Leave a note behind to tell the "install" command that the kernel # needs to be installed before the world. touch ${BDHASH}-install/kernelfirst # Remind the user that they need to run "freebsd-update install" # to install the downloaded bits, in case they didn't RTFM. echo "To install the downloaded upgrades, run \"$0 install\"." } # Make sure that all the file hashes mentioned in $@ have corresponding # gzipped files stored in /files/. install_verify () { # Generate a list of hashes cat $@ | cut -f 2,7 -d '|' | grep -E '^f' | cut -f 2 -d '|' | sort -u > filelist # Make sure all the hashes exist while read HASH; do if ! [ -f files/${HASH}.gz ]; then echo -n "Update files missing -- " echo "this should never happen." echo "Re-run '$0 fetch'." return 1 fi done < filelist # Clean up rm filelist } # Remove the system immutable flag from files install_unschg () { # Generate file list cat $@ | cut -f 1 -d '|' > filelist # Remove flags while read F; do if ! [ -e ${BASEDIR}/${F} ]; then continue fi chflags noschg ${BASEDIR}/${F} || return 1 done < filelist # Clean up rm filelist } # Decide which directory name to use for kernel backups. backup_kernel_finddir () { CNT=0 while true ; do # Pathname does not exist, so it is OK use that name # for backup directory. if [ ! -e $BACKUPKERNELDIR ]; then return 0 fi # If directory do exist, we only use if it has our # marker file. if [ -d $BACKUPKERNELDIR -a \ -e $BACKUPKERNELDIR/.freebsd-update ]; then return 0 fi # We could not use current directory name, so add counter to # the end and try again. CNT=$((CNT + 1)) if [ $CNT -gt 9 ]; then echo "Could not find valid backup dir ($BACKUPKERNELDIR)" exit 1 fi BACKUPKERNELDIR="`echo $BACKUPKERNELDIR | sed -Ee 's/[0-9]\$//'`" BACKUPKERNELDIR="${BACKUPKERNELDIR}${CNT}" done } # Backup the current kernel using hardlinks, if not disabled by user. # Since we delete all files in the directory used for previous backups # we create a marker file called ".freebsd-update" in the directory so # we can determine on the next run that the directory was created by # freebsd-update and we then do not accidentally remove user files in # the unlikely case that the user has created a directory with a # conflicting name. backup_kernel () { # Only make kernel backup is so configured. if [ $BACKUPKERNEL != yes ]; then return 0 fi # Decide which directory name to use for kernel backups. backup_kernel_finddir # Remove old kernel backup files. If $BACKUPKERNELDIR was # "not ours", backup_kernel_finddir would have exited, so # deleting the directory content is as safe as we can make it. if [ -d $BACKUPKERNELDIR ]; then rm -fr $BACKUPKERNELDIR fi # Create directories for backup. mkdir -p $BACKUPKERNELDIR mtree -cdn -p "${KERNELDIR}" | \ mtree -Ue -p "${BACKUPKERNELDIR}" > /dev/null # Mark the directory as having been created by freebsd-update. touch $BACKUPKERNELDIR/.freebsd-update if [ $? -ne 0 ]; then echo "Could not create kernel backup directory" exit 1 fi # Disable pathname expansion to be sure *.symbols is not # expanded. set -f # Use find to ignore symbol files, unless disabled by user. if [ $BACKUPKERNELSYMBOLFILES = yes ]; then FINDFILTER="" else FINDFILTER=-"a ! -name *.symbols" fi # Backup all the kernel files using hardlinks. (cd $KERNELDIR && find . -type f $FINDFILTER -exec \ cp -pl '{}' ${BACKUPKERNELDIR}/'{}' \;) # Re-enable patchname expansion. set +f } # Install new files install_from_index () { # First pass: Do everything apart from setting file flags. We # can't set flags yet, because schg inhibits hard linking. sort -k 1,1 -t '|' $1 | tr '|' ' ' | while read FPATH TYPE OWNER GROUP PERM FLAGS HASH LINK; do case ${TYPE} in d) # Create a directory install -d -o ${OWNER} -g ${GROUP} \ -m ${PERM} ${BASEDIR}/${FPATH} ;; f) if [ -z "${LINK}" ]; then # Create a file, without setting flags. gunzip < files/${HASH}.gz > ${HASH} install -S -o ${OWNER} -g ${GROUP} \ -m ${PERM} ${HASH} ${BASEDIR}/${FPATH} rm ${HASH} else # Create a hard link. ln -f ${BASEDIR}/${LINK} ${BASEDIR}/${FPATH} fi ;; L) # Create a symlink ln -sfh ${HASH} ${BASEDIR}/${FPATH} ;; esac done # Perform a second pass, adding file flags. tr '|' ' ' < $1 | while read FPATH TYPE OWNER GROUP PERM FLAGS HASH LINK; do if [ ${TYPE} = "f" ] && ! [ ${FLAGS} = "0" ]; then chflags ${FLAGS} ${BASEDIR}/${FPATH} fi done } # Remove files which we want to delete install_delete () { # Generate list of new files cut -f 1 -d '|' < $2 | sort > newfiles # Generate subindex of old files we want to nuke sort -k 1,1 -t '|' $1 | join -t '|' -v 1 - newfiles | sort -r -k 1,1 -t '|' | cut -f 1,2 -d '|' | tr '|' ' ' > killfiles # Remove the offending bits while read FPATH TYPE; do case ${TYPE} in d) rmdir ${BASEDIR}/${FPATH} ;; f) rm ${BASEDIR}/${FPATH} ;; L) rm ${BASEDIR}/${FPATH} ;; esac done < killfiles # Clean up rm newfiles killfiles } # Install new files, delete old files, and update linker.hints install_files () { # If we haven't already dealt with the kernel, deal with it. if ! [ -f $1/kerneldone ]; then grep -E '^/boot/' $1/INDEX-OLD > INDEX-OLD grep -E '^/boot/' $1/INDEX-NEW > INDEX-NEW # Backup current kernel before installing a new one backup_kernel || return 1 # Install new files install_from_index INDEX-NEW || return 1 # Remove files which need to be deleted install_delete INDEX-OLD INDEX-NEW || return 1 # Update linker.hints if necessary if [ -s INDEX-OLD -o -s INDEX-NEW ]; then kldxref -R /boot/ 2>/dev/null fi # We've finished updating the kernel. touch $1/kerneldone # Do we need to ask for a reboot now? if [ -f $1/kernelfirst ] && [ -s INDEX-OLD -o -s INDEX-NEW ]; then cat <<-EOF Kernel updates have been installed. Please reboot and run "$0 install" again to finish installing updates. EOF exit 0 fi fi # If we haven't already dealt with the world, deal with it. if ! [ -f $1/worlddone ]; then # Create any necessary directories first grep -vE '^/boot/' $1/INDEX-NEW | grep -E '^[^|]+\|d\|' > INDEX-NEW install_from_index INDEX-NEW || return 1 # Install new runtime linker grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '^[^|]+\|d\|' | grep -E '^/libexec/ld-elf[^|]*\.so\.[0-9]+\|' > INDEX-NEW install_from_index INDEX-NEW || return 1 # Install new shared libraries next grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '^[^|]+\|d\|' | grep -vE '^/libexec/ld-elf[^|]*\.so\.[0-9]+\|' | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-NEW install_from_index INDEX-NEW || return 1 # Deal with everything else grep -vE '^/boot/' $1/INDEX-OLD | grep -vE '^[^|]+\|d\|' | grep -vE '^/libexec/ld-elf[^|]*\.so\.[0-9]+\|' | grep -vE '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-OLD grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '^[^|]+\|d\|' | grep -vE '^/libexec/ld-elf[^|]*\.so\.[0-9]+\|' | grep -vE '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-NEW install_from_index INDEX-NEW || return 1 install_delete INDEX-OLD INDEX-NEW || return 1 # Rebuild /etc/spwd.db and /etc/pwd.db if necessary. if [ /etc/master.passwd -nt /etc/spwd.db ] || [ /etc/master.passwd -nt /etc/pwd.db ]; then pwd_mkdb /etc/master.passwd fi # Rebuild /etc/login.conf.db if necessary. if [ /etc/login.conf -nt /etc/login.conf.db ]; then cap_mkdb /etc/login.conf fi # We've finished installing the world and deleting old files # which are not shared libraries. touch $1/worlddone # Do we need to ask the user to portupgrade now? grep -vE '^/boot/' $1/INDEX-NEW | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' | cut -f 1 -d '|' | sort > newfiles if grep -vE '^/boot/' $1/INDEX-OLD | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' | cut -f 1 -d '|' | sort | join -v 1 - newfiles | grep -q .; then cat <<-EOF Completing this upgrade requires removing old shared object files. Please rebuild all installed 3rd party software (e.g., programs installed from the ports tree) and then run "$0 install" again to finish installing updates. EOF rm newfiles exit 0 fi rm newfiles fi # Remove old shared libraries grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '^[^|]+\|d\|' | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-NEW grep -vE '^/boot/' $1/INDEX-OLD | grep -vE '^[^|]+\|d\|' | grep -E '^[^|]*/lib/[^|]*\.so\.[0-9]+\|' > INDEX-OLD install_delete INDEX-OLD INDEX-NEW || return 1 # Remove old directories grep -vE '^/boot/' $1/INDEX-NEW | grep -E '^[^|]+\|d\|' > INDEX-NEW grep -vE '^/boot/' $1/INDEX-OLD | grep -E '^[^|]+\|d\|' > INDEX-OLD install_delete INDEX-OLD INDEX-NEW || return 1 # Remove temporary files rm INDEX-OLD INDEX-NEW } # Rearrange bits to allow the installed updates to be rolled back install_setup_rollback () { # Remove the "reboot after installing kernel", "kernel updated", and # "finished installing the world" flags if present -- they are # irrelevant when rolling back updates. if [ -f ${BDHASH}-install/kernelfirst ]; then rm ${BDHASH}-install/kernelfirst rm ${BDHASH}-install/kerneldone fi if [ -f ${BDHASH}-install/worlddone ]; then rm ${BDHASH}-install/worlddone fi if [ -L ${BDHASH}-rollback ]; then mv ${BDHASH}-rollback ${BDHASH}-install/rollback fi mv ${BDHASH}-install ${BDHASH}-rollback } # Actually install updates install_run () { echo -n "Installing updates..." # Make sure we have all the files we should have install_verify ${BDHASH}-install/INDEX-OLD \ ${BDHASH}-install/INDEX-NEW || return 1 # Remove system immutable flag from files install_unschg ${BDHASH}-install/INDEX-OLD \ ${BDHASH}-install/INDEX-NEW || return 1 # Install new files, delete old files, and update linker.hints install_files ${BDHASH}-install || return 1 # Rearrange bits to allow the installed updates to be rolled back install_setup_rollback echo " done." } # Rearrange bits to allow the previous set of updates to be rolled back next. rollback_setup_rollback () { if [ -L ${BDHASH}-rollback/rollback ]; then mv ${BDHASH}-rollback/rollback rollback-tmp rm -r ${BDHASH}-rollback/ rm ${BDHASH}-rollback mv rollback-tmp ${BDHASH}-rollback else rm -r ${BDHASH}-rollback/ rm ${BDHASH}-rollback fi } # Install old files, delete new files, and update linker.hints rollback_files () { # Install old shared library files which don't have the same path as # a new shared library file. grep -vE '^/boot/' $1/INDEX-NEW | grep -E '/lib/.*\.so\.[0-9]+\|' | cut -f 1 -d '|' | sort > INDEX-NEW.libs.flist grep -vE '^/boot/' $1/INDEX-OLD | grep -E '/lib/.*\.so\.[0-9]+\|' | sort -k 1,1 -t '|' - | join -t '|' -v 1 - INDEX-NEW.libs.flist > INDEX-OLD install_from_index INDEX-OLD || return 1 # Deal with files which are neither kernel nor shared library grep -vE '^/boot/' $1/INDEX-OLD | grep -vE '/lib/.*\.so\.[0-9]+\|' > INDEX-OLD grep -vE '^/boot/' $1/INDEX-NEW | grep -vE '/lib/.*\.so\.[0-9]+\|' > INDEX-NEW install_from_index INDEX-OLD || return 1 install_delete INDEX-NEW INDEX-OLD || return 1 # Install any old shared library files which we didn't install above. grep -vE '^/boot/' $1/INDEX-OLD | grep -E '/lib/.*\.so\.[0-9]+\|' | sort -k 1,1 -t '|' - | join -t '|' - INDEX-NEW.libs.flist > INDEX-OLD install_from_index INDEX-OLD || return 1 # Delete unneeded shared library files grep -vE '^/boot/' $1/INDEX-OLD | grep -E '/lib/.*\.so\.[0-9]+\|' > INDEX-OLD grep -vE '^/boot/' $1/INDEX-NEW | grep -E '/lib/.*\.so\.[0-9]+\|' > INDEX-NEW install_delete INDEX-NEW INDEX-OLD || return 1 # Deal with kernel files grep -E '^/boot/' $1/INDEX-OLD > INDEX-OLD grep -E '^/boot/' $1/INDEX-NEW > INDEX-NEW install_from_index INDEX-OLD || return 1 install_delete INDEX-NEW INDEX-OLD || return 1 if [ -s INDEX-OLD -o -s INDEX-NEW ]; then kldxref -R /boot/ 2>/dev/null fi # Remove temporary files rm INDEX-OLD INDEX-NEW INDEX-NEW.libs.flist } # Actually rollback updates rollback_run () { echo -n "Uninstalling updates..." # If there are updates waiting to be installed, remove them; we # want the user to re-run 'fetch' after rolling back updates. if [ -L ${BDHASH}-install ]; then rm -r ${BDHASH}-install/ rm ${BDHASH}-install fi # Make sure we have all the files we should have install_verify ${BDHASH}-rollback/INDEX-NEW \ ${BDHASH}-rollback/INDEX-OLD || return 1 # Remove system immutable flag from files install_unschg ${BDHASH}-rollback/INDEX-NEW \ ${BDHASH}-rollback/INDEX-OLD || return 1 # Install old files, delete new files, and update linker.hints rollback_files ${BDHASH}-rollback || return 1 # Remove the rollback directory and the symlink pointing to it; and # rearrange bits to allow the previous set of updates to be rolled # back next. rollback_setup_rollback echo " done." } # Compare INDEX-ALL and INDEX-PRESENT and print warnings about differences. IDS_compare () { # Get all the lines which mismatch in something other than file # flags. We ignore file flags because sysinstall doesn't seem to # set them when it installs FreeBSD; warning about these adds a # very large amount of noise. cut -f 1-5,7-8 -d '|' $1 > $1.noflags sort -k 1,1 -t '|' $1.noflags > $1.sorted cut -f 1-5,7-8 -d '|' $2 | comm -13 $1.noflags - | fgrep -v '|-|||||' | sort -k 1,1 -t '|' | join -t '|' $1.sorted - > INDEX-NOTMATCHING # Ignore files which match IDSIGNOREPATHS. for X in ${IDSIGNOREPATHS}; do grep -E "^${X}" INDEX-NOTMATCHING done | sort -u | comm -13 - INDEX-NOTMATCHING > INDEX-NOTMATCHING.tmp mv INDEX-NOTMATCHING.tmp INDEX-NOTMATCHING # Go through the lines and print warnings. while read LINE; do FPATH=`echo "${LINE}" | cut -f 1 -d '|'` TYPE=`echo "${LINE}" | cut -f 2 -d '|'` OWNER=`echo "${LINE}" | cut -f 3 -d '|'` GROUP=`echo "${LINE}" | cut -f 4 -d '|'` PERM=`echo "${LINE}" | cut -f 5 -d '|'` HASH=`echo "${LINE}" | cut -f 6 -d '|'` LINK=`echo "${LINE}" | cut -f 7 -d '|'` P_TYPE=`echo "${LINE}" | cut -f 8 -d '|'` P_OWNER=`echo "${LINE}" | cut -f 9 -d '|'` P_GROUP=`echo "${LINE}" | cut -f 10 -d '|'` P_PERM=`echo "${LINE}" | cut -f 11 -d '|'` P_HASH=`echo "${LINE}" | cut -f 12 -d '|'` P_LINK=`echo "${LINE}" | cut -f 13 -d '|'` # Warn about different object types. if ! [ "${TYPE}" = "${P_TYPE}" ]; then echo -n "${FPATH} is a " case "${P_TYPE}" in f) echo -n "regular file, " ;; d) echo -n "directory, " ;; L) echo -n "symlink, " ;; esac echo -n "but should be a " case "${TYPE}" in f) echo -n "regular file." ;; d) echo -n "directory." ;; L) echo -n "symlink." ;; esac echo # Skip other tests, since they don't make sense if # we're comparing different object types. continue fi # Warn about different owners. if ! [ "${OWNER}" = "${P_OWNER}" ]; then echo -n "${FPATH} is owned by user id ${P_OWNER}, " echo "but should be owned by user id ${OWNER}." fi # Warn about different groups. if ! [ "${GROUP}" = "${P_GROUP}" ]; then echo -n "${FPATH} is owned by group id ${P_GROUP}, " echo "but should be owned by group id ${GROUP}." fi # Warn about different permissions. We do not warn about # different permissions on symlinks, since some archivers # don't extract symlink permissions correctly and they are # ignored anyway. if ! [ "${PERM}" = "${P_PERM}" ] && ! [ "${TYPE}" = "L" ]; then echo -n "${FPATH} has ${P_PERM} permissions, " echo "but should have ${PERM} permissions." fi # Warn about different file hashes / symlink destinations. if ! [ "${HASH}" = "${P_HASH}" ]; then if [ "${TYPE}" = "L" ]; then echo -n "${FPATH} is a symlink to ${P_HASH}, " echo "but should be a symlink to ${HASH}." fi if [ "${TYPE}" = "f" ]; then echo -n "${FPATH} has SHA256 hash ${P_HASH}, " echo "but should have SHA256 hash ${HASH}." fi fi # We don't warn about different hard links, since some # some archivers break hard links, and as long as the # underlying data is correct they really don't matter. done < INDEX-NOTMATCHING # Clean up rm $1 $1.noflags $1.sorted $2 INDEX-NOTMATCHING } # Do the work involved in comparing the system to a "known good" index IDS_run () { workdir_init || return 1 # Prepare the mirror list. fetch_pick_server_init && fetch_pick_server # Try to fetch the public key until we run out of servers. while ! fetch_key; do fetch_pick_server || return 1 done # Try to fetch the metadata index signature ("tag") until we run # out of available servers; and sanity check the downloaded tag. while ! fetch_tag; do fetch_pick_server || return 1 done fetch_tagsanity || return 1 # Fetch INDEX-OLD and INDEX-ALL. fetch_metadata INDEX-OLD INDEX-ALL || return 1 # Generate filtered INDEX-OLD and INDEX-ALL files containing only # the components we want and without anything marked as "Ignore". fetch_filter_metadata INDEX-OLD || return 1 fetch_filter_metadata INDEX-ALL || return 1 # Merge the INDEX-OLD and INDEX-ALL files into INDEX-ALL. sort INDEX-OLD INDEX-ALL > INDEX-ALL.tmp mv INDEX-ALL.tmp INDEX-ALL rm INDEX-OLD # Translate /boot/${KERNCONF} to ${KERNELDIR} fetch_filter_kernel_names INDEX-ALL ${KERNCONF} # Inspect the system and generate an INDEX-PRESENT file. fetch_inspect_system INDEX-ALL INDEX-PRESENT /dev/null || return 1 # Compare INDEX-ALL and INDEX-PRESENT and print warnings about any # differences. IDS_compare INDEX-ALL INDEX-PRESENT } #### Main functions -- call parameter-handling and core functions # Using the command line, configuration file, and defaults, # set all the parameters which are needed later. get_params () { init_params parse_cmdline $@ parse_conffile default_params } # Fetch command. Make sure that we're being called # interactively, then run fetch_check_params and fetch_run cmd_fetch () { - if [ ! -t 0 ]; then + if [ ! -t 0 -a $NOTTYOK -eq 0 ]; then echo -n "`basename $0` fetch should not " echo "be run non-interactively." echo "Run `basename $0` cron instead." exit 1 fi fetch_check_params fetch_run || exit 1 } # Cron command. Make sure the parameters are sensible; wait # rand(3600) seconds; then fetch updates. While fetching updates, # send output to a temporary file; only print that file if the # fetching failed. cmd_cron () { fetch_check_params sleep `jot -r 1 0 3600` TMPFILE=`mktemp /tmp/freebsd-update.XXXXXX` || exit 1 if ! fetch_run >> ${TMPFILE} || ! grep -q "No updates needed" ${TMPFILE} || [ ${VERBOSELEVEL} = "debug" ]; then mail -s "`hostname` security updates" ${MAILTO} < ${TMPFILE} fi rm ${TMPFILE} } # Fetch files for upgrading to a new release. cmd_upgrade () { upgrade_check_params upgrade_run || exit 1 } # Install downloaded updates. cmd_install () { install_check_params install_run || exit 1 } # Rollback most recently installed updates. cmd_rollback () { rollback_check_params rollback_run || exit 1 } # Compare system against a "known good" index. cmd_IDS () { IDS_check_params IDS_run || exit 1 } #### Entry point # Make sure we find utilities from the base system export PATH=/sbin:/bin:/usr/sbin:/usr/bin:${PATH} # Set a pager if the user doesn't if [ -z "$PAGER" ]; then PAGER=/usr/bin/more fi # Set LC_ALL in order to avoid problems with character ranges like [A-Z]. export LC_ALL=C get_params $@ for COMMAND in ${COMMANDS}; do cmd_${COMMAND} done