Index: projects/clang700-import/UPDATING =================================================================== --- projects/clang700-import/UPDATING (revision 338689) +++ projects/clang700-import/UPDATING (revision 338690) @@ -1,1820 +1,1822 @@ Updating Information for FreeBSD current users. This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. These instructions assume that you basically know what you are doing. If not, then please consult the FreeBSD handbook: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. NOTE: FreeBSD has switched from gcc to clang. If you have trouble bootstrapping from older versions of FreeBSD, try WITHOUT_CLANG and WITH_GCC to bootstrap to the tip of head, and then rebuild without this option. The bootstrap process from older version of current across the gcc/clang cutover is a bit fragile. NOTE TO PEOPLE WHO THINK THAT FreeBSD 12.x IS SLOW: FreeBSD 12.x has many debugging features turned on, in both the kernel and userland. These features attempt to detect incorrect use of system primitives, and encourage loud failure through extra sanity checking and fail stop semantics. They also substantially impact system performance. If you want to do performance measurement, benchmarking, and optimization, you'll want to turn them off. This includes various WITNESS- related kernel options, INVARIANTS, malloc debugging flags in userland, and various verbose features in the kernel. Many developers choose to disable these features on build machines to maximize performance. (To completely disable malloc debugging, define MALLOC_PRODUCTION in /etc/make.conf, or to merely disable the most expensive debugging functionality run "ln -s 'abort:false,junk:false' /etc/malloc.conf".) 20180826: The Yarrow CSPRNG has been removed from the kernel as it has not been supported by its designers since at least 2003. Fortuna has been the default since FreeBSD-11. 20180822: devctl freeze/thaw have gone into the tree, the rc scripts have been updated to use them and devmatch has been changed. You should update kernel, userland and rc scripts all at the same time. 20180818: The default interpreter has been switched from 4th to Lua. LOADER_DEFAULT_INTERP, documented in build(7), will override the default interpreter. If you have custom FORTH code you will need to set LOADER_DEFAULT_INTERP=4th (valid values are 4th, lua or simp) in src.conf for the build. This will create default hard links between loader and loader_4th instead of loader and loader_lua, the new default. If you are using UEFI it will create the proper hard link to loader.efi. bhyve uses userboot.so. It remains 4th-only until some issues are solved regarding coexisting with multiple versions of FreeBSD are resolved. 20180815: ls(1) now respects the COLORTERM environment variable used in other systems and software to indicate that a colored terminal is both supported and desired. If ls(1) is suddenly emitting colors, they may be disabled again by either removing the unwanted COLORTERM from your environment, or using `ls --color=never`. The ls(1) specific CLICOLOR may not be observed in a future release. 20180808: The default pager for most commands has been changed to "less". To restore the old behavior, set PAGER="more" and MANPAGER="more -s" in your environment. 20180731: The jedec_ts(4) driver has been removed. A superset of its functionality is available in the jedec_dimm(4) driver, and the manpage for that driver includes migration instructions. If you have "device jedec_ts" in your kernel configuration file, it must be removed. 20180730: amd64/GENERIC now has EFI runtime services, EFIRT, enabled by default. This should have no effect if the kernel is booted via BIOS/legacy boot. EFIRT may be disabled via a loader tunable, efi.rt.disabled, if a system has a buggy firmware that prevents a successful boot due to use of runtime services. 20180727: Atmel AT91RM9200 and AT91SAM9, Cavium CNS 11xx and XScale support has been removed from the tree. These ports were obsolete and/or known to be broken for many years. 20180723: loader.efi has been augmented to participate more fully in the UEFI boot manager protocol. loader.efi will now look at the BootXXXX environment variable to determine if a specific kernel or root partition was specified. XXXX is derived from BootCurrent. efibootmgr(8) manages these standard UEFI variables. 20180720: zfsloader's functionality has now been folded into loader. zfsloader is no longer necessary once you've updated your boot blocks. For a transition period, we will install a hardlink for zfsloader to loader to allow a smooth transition until the boot blocks can be updated (hard link because old zfs boot blocks don't understand symlinks). 20180719: ARM64 now have efifb support, if you want to have serial console on your arm64 board when an screen is connected and the bootloader setup a frambuffer for us to use, just add : boot_serial=YES boot_multicons=YES in /boot/loader.conf For Raspberry Pi 3 (RPI) users, this is needed even if you don't have an screen connected as the firmware will setup a framebuffer are that u-boot will expose as an EFI framebuffer. 20180719: New uid:gid added, ntpd:ntpd (123:123). Be sure to run mergemaster or take steps to update /etc/passwd before doing installworld on - existing systems. Also, rc.d/ntpd now starts ntpd(8) as user ntpd + existing systems. Do not skip the "mergemaster -Fp" step before + installworld, as described in the update procedures near the bottom + of this document. Also, rc.d/ntpd now starts ntpd(8) as user ntpd if the new mac_ntpd(4) policy is available, unless ntpd_flags or the ntp config file contain options that change file/dir locations. When such options (e.g., "statsdir" or "crypto") are used, ntpd can still be run as non-root by setting ntpd_user=ntpd in rc.conf, after taking steps to ensure that all required files/dirs are accessible by the ntpd user. 20180717: Big endian arm support has been removed. 20180711: The static environment setup in kernel configs is no longer mutually exclusive with the loader(8) environment by default. In order to restore the previous default behavior of disabling the loader(8) environment if a static environment is present, you must specify loader_env.disabled=1 in the static environment. 20180705: The ABI of syscalls used by management tools like sockstat and netstat has been broken to allow 32-bit binaries to work on 64-bit kernels without modification. These programs will need to match the kernel in order to function. External programs may require minor modifications to accommodate a change of type in structures from pointers to 64-bit virtual addresses. 20180702: On i386 and amd64 atomics are now inlined. Out of tree modules using atomics will need to be rebuilt. 20180701: The '%I' format in the kern.corefile sysctl limits the number of core files that a process can generate to the number stored in the debug.ncores sysctl. The '%I' format is replaced by the single digit index. Previously, if all indexes were taken the kernel would overwrite only a core file with the highest index in a filename. Currently the system will create a new core file if there is a free index or if all slots are taken it will overwrite the oldest one. 20180630: Clang, llvm, lld, lldb, compiler-rt and libc++ have been upgraded to 6.0.1. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using clang 3.5.0 or higher. 20180628: r335753 introduced a new quoting method. However, etc/devd/devmatch.conf needed to be changed to work with it. This change was made with r335763 and requires a mergemaster / etcupdate / etc to update the installed file. 20180612: r334930 changed the interface between the NFS modules, so they all need to be rebuilt. r335018 did a __FreeBSD_version bump for this. 20180530: The kernel / userland interface for devinfo changed, so you'll need a new kernel and userland as a pair for it to work (rebuilding lib/libdevinfo is all that's required). devinfo and devmatch will not work, but everything else will when there's a mismatch. 20180523: The on-disk format for hwpmc callchain records has changed to include threadid corresponding to a given record. This changes the field offsets and thus requires that libpmcstat be rebuilt before using a kernel later than r334108. 20180517: The vxge(4) driver has been removed. This driver was introduced into HEAD one week before the Exar left the Ethernet market and is not known to be used. If you have device vxge in your kernel config file it must be removed. 20180510: The amd64 kernel now requires a ld that supports ifunc to produce a working kernel, either lld or a newer binutils. lld is built by default on amd64, and the 'buildkernel' target uses it automatically. However, it is not the default linker, so building the kernel the traditional way requires LD=ld.lld on the command line (or LD=/usr/local/bin/ld for binutils port/package). lld will soon be default, and this requirement will go away. 20180508: The nxge(4) driver has been removed. This driver was for PCI-X 10g cards made by s2io/Neterion. The company was aquired by Exar and no longer sells or supports Ethernet products. If you have device nxge in your kernel config file it must be removed. 20180504: The tz database (tzdb) has been updated to 2018e. This version more correctly models time stamps in time zones with negative DST such as Europe/Dublin (from 1971 on), Europe/Prague (1946/7), and Africa/Windhoek (1994/2017). This does not affect the UT offsets, only time zone abbreviations and the tm_isdst flag. 20180502: The ixgb(4) driver has been removed. This driver was for an early and uncommon legacy PCI 10GbE for a single ASIC, Intel 82597EX. Intel quickly shifted to the long lived ixgbe family. If you have device ixgb in your kernel config file it must be removed. 20180501: The lmc(4) driver has been removed. This was a WAN interface card that was already reportedly rare in 2003, and had an ambiguous license. If you have device lmc in your kernel config file it must be removed. 20180413: Support for Arcnet networks has been removed. If you have device arcnet or device cm in your kernel config file they must be removed. 20180411: Support for FDDI networks has been removed. If you have device fddi or device fpa in your kernel config file they must be removed. 20180406: In addition to supporting RFC 3164 formatted messages, the syslogd(8) service is now capable of parsing RFC 5424 formatted log messages. The main benefit of using RFC 5424 is that clients may now send log messages with timestamps containing year numbers, microseconds and time zone offsets. Similarly, the syslog(3) C library function has been altered to send RFC 5424 formatted messages to the local system logging daemon. On systems using syslogd(8), this change should have no negative impact, as long as syslogd(8) and the C library are updated at the same time. On systems using a different system logging daemon, it may be necessary to make configuration adjustments, depending on the software used. When using syslog-ng, add the 'syslog-protocol' flag to local input sources to enable parsing of RFC 5424 formatted messages: source src { unix-dgram("/var/run/log" flags(syslog-protocol)); } When using rsyslog, disable the 'SysSock.UseSpecialParser' option of the 'imuxsock' module to let messages be processed by the regular RFC 3164/5424 parsing pipeline: module(load="imuxsock" SysSock.UseSpecialParser="off") Do note that these changes only affect communication between local applications and syslogd(8). The format that syslogd(8) uses to store messages on disk or forward messages to other systems remains unchanged. syslogd(8) still uses RFC 3164 for these purposes. Options to customize this behaviour will be added in the future. Utilities that process log files stored in /var/log are thus expected to continue to function as before. __FreeBSD_version has been incremented to 1200061 to denote this change. 20180328: Support for token ring networks has been removed. If you have "device token" in your kernel config you should remove it. No device drivers supported token ring. 20180323: makefs was modified to be able to tag ISO9660 El Torito boot catalog entries as EFI instead of overloading the i386 tag as done previously. The amd64 mkisoimages.sh script used to build amd64 ISO images for release was updated to use this. This may mean that makefs must be updated before "make cdrom" can be run in the release directory. This should be as simple as: $ cd $SRCDIR/usr.sbin/makefs $ make depend all install 20180212: FreeBSD boot loader enhanced with Lua scripting. It's purely opt-in for now by building WITH_LOADER_LUA and WITHOUT_FORTH in /etc/src.conf. Co-existance for the transition period will come shortly. Booting is a complex environment and test coverage for Lua-enabled loaders has been thin, so it would be prudent to assume it might not work and make provisions for backup boot methods. 20180211: devmatch functionality has been turned on in devd. It will automatically load drivers for unattached devices. This may cause unexpected drivers to be loaded. Please report any problems to current@ and imp@freebsd.org. 20180114: Clang, llvm, lld, lldb, compiler-rt and libc++ have been upgraded to 6.0.0. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using clang 3.5.0 or higher. 20180110: LLVM's lld linker is now used as the FreeBSD/amd64 bootstrap linker. This means it is used to link the kernel and userland libraries and executables, but is not yet installed as /usr/bin/ld by default. To revert to ld.bfd as the bootstrap linker, in /etc/src.conf set WITHOUT_LLD_BOOTSTRAP=yes 20180110: On i386, pmtimer has been removed. Its functionality has been folded into apm. It was a no-op on ACPI in current for a while now (but was still needed on i386 in FreeBSD 11 and earlier). Users may need to remove it from kernel config files. 20180104: The use of RSS hash from the network card aka flowid has been disabled by default for lagg(4) as it's currently incompatible with the lacp and loadbalance protocols. This can be re-enabled by setting the following in loader.conf: net.link.lagg.default_use_flowid="1" 20180102: The SW_WATCHDOG option is no longer necessary to enable the hardclock-based software watchdog if no hardware watchdog is configured. As before, SW_WATCHDOG will cause the software watchdog to be enabled even if a hardware watchdog is configured. 20171215: r326887 fixes the issue described in the 20171214 UPDATING entry. r326888 flips the switch back to building GELI support always. 20171214: r362593 broke ZFS + GELI support for reasons unknown. However, it also broke ZFS support generally, so GELI has been turned off by default as the lesser evil in r326857. If you boot off ZFS and/or GELI, it might not be a good time to update. 20171125: PowerPC users must update loader(8) by rebuilding world before installing a new kernel, as the protocol connecting them has changed. Without the update, loader metadata will not be passed successfully to the kernel and users will have to enter their root partition at the kernel mountroot prompt to continue booting. Newer versions of loader can boot old kernels without issue. 20171110: The LOADER_FIREWIRE_SUPPORT build variable as been renamed to WITH/OUT_LOADER_FIREWIRE. LOADER_{NO_,}GELI_SUPPORT has been renamed to WITH/OUT_LOADER_GELI. 20171106: The naive and non-compliant support of posix_fallocate(2) in ZFS has been removed as of r325320. The system call now returns EINVAL when used on a ZFS file. Although the new behavior complies with the standard, some consumers are not prepared to cope with it. One known victim is lld prior to r325420. 20171102: Building in a FreeBSD src checkout will automatically create object directories now rather than store files in the current directory if 'make obj' was not ran. Calling 'make obj' is no longer necessary. This feature can be disabled by setting WITHOUT_AUTO_OBJ=yes in /etc/src-env.conf (not /etc/src.conf), or passing the option in the environment. 20171101: The default MAKEOBJDIR has changed from /usr/obj/ for native builds, and /usr/obj// for cross-builds, to a unified /usr/obj//. This behavior can be changed to the old format by setting WITHOUT_UNIFIED_OBJDIR=yes in /etc/src-env.conf, the environment, or with -DWITHOUT_UNIFIED_OBJDIR when building. The UNIFIED_OBJDIR option is a transitional feature that will be removed for 12.0 release; please migrate to the new format for any tools by looking up the OBJDIR used by 'make -V .OBJDIR' means rather than hardcoding paths. 20171028: The native-xtools target no longer installs the files by default to the OBJDIR. Use the native-xtools-install target with a DESTDIR to install to ${DESTDIR}/${NXTP} where NXTP defaults to /nxb-bin. 20171021: As part of the boot loader infrastructure cleanup, LOADER_*_SUPPORT options are changing from controlling the build if defined / undefined to controlling the build with explicit 'yes' or 'no' values. They will shift to WITH/WITHOUT options to match other options in the system. 20171010: libstand has turned into a private library for sys/boot use only. It is no longer supported as a public interface outside of sys/boot. 20171005: The arm port has split armv6 into armv6 and armv7. armv7 is now a valid TARGET_ARCH/MACHINE_ARCH setting. If you have an armv7 system and are running a kernel from before r324363, you will need to add MACHINE_ARCH=armv7 to 'make buildworld' to do a native build. 20171003: When building multiple kernels using KERNCONF, non-existent KERNCONF files will produce an error and buildkernel will fail. Previously missing KERNCONF files silently failed giving no indication as to why, only to subsequently discover during installkernel that the desired kernel was never built in the first place. 20170912: The default serial number format for CTL LUNs has changed. This will affect users who use /dev/diskid/* device nodes, or whose FibreChannel or iSCSI clients care about their LUNs' serial numbers. Users who require serial number stability should hardcode serial numbers in /etc/ctl.conf . 20170912: For 32-bit arm compiled for hard-float support, soft-floating point binaries now always get their shared libraries from LD_SOFT_LIBRARY_PATH (in the past, this was only used if /usr/libsoft also existed). Only users with a hard-float ld.so, but soft-float everything else should be affected. 20170826: The geli password typed at boot is now hidden. To restore the previous behavior, see geli(8) for configuration options. 20170825: Move PMTUD blackhole counters to TCPSTATS and remove them from bare sysctl values. Minor nit, but requires a rebuild of both world/kernel to complete. 20170814: "make check" behavior (made in ^/head@r295380) has been changed to execute from a limited sandbox, as opposed to executing from ${TESTSDIR}. Behavioral changes: - The "beforecheck" and "aftercheck" targets are now specified. - ${CHECKDIR} (added in commit noted above) has been removed. - Legacy behavior can be enabled by setting WITHOUT_MAKE_CHECK_USE_SANDBOX in src.conf(5) or the environment. If the limited sandbox mode is enabled, "make check" will execute "make distribution", then install, execute the tests, and clean up the sandbox if successful. The "make distribution" and "make install" targets are typically run as root to set appropriate permissions and ownership at installation time. The end-user should set "WITH_INSTALL_AS_USER" in src.conf(5) or the environment if executing "make check" with limited sandbox mode using an unprivileged user. 20170808: Since the switch to GPT disk labels, fsck for UFS/FFS has been unable to automatically find alternate superblocks. As of r322297, the information needed to find alternate superblocks has been moved to the end of the area reserved for the boot block. Filesystems created with a newfs of this vintage or later will create the recovery information. If you have a filesystem created prior to this change and wish to have a recovery block created for your filesystem, you can do so by running fsck in foreground mode (i.e., do not use the -p or -y options). As it starts, fsck will ask ``SAVE DATA TO FIND ALTERNATE SUPERBLOCKS'' to which you should answer yes. 20170728: As of r321665, an NFSv4 server configuration that services Kerberos mounts or clients that do not support the uid/gid in owner/owner_group string capability, must explicitly enable the nfsuserd daemon by adding nfsuserd_enable="YES" to the machine's /etc/rc.conf file. 20170722: Clang, llvm, lldb, compiler-rt and libc++ have been upgraded to 5.0.0. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using clang 3.5.0 or higher. 20170701: WITHOUT_RCMDS is now the default. Set WITH_RCMDS if you need the r-commands (rlogin, rsh, etc.) to be built with the base system. 20170625: The FreeBSD/powerpc platform now uses a 64-bit type for time_t. This is a very major ABI incompatible change, so users of FreeBSD/powerpc must be careful when performing source upgrades. It is best to run 'make installworld' from an alternate root system, either a live CD/memory stick, or a temporary root partition. Additionally, all ports must be recompiled. powerpc64 is largely unaffected, except in the case of 32-bit compatibility. All 32-bit binaries will be affected. 20170623: Forward compatibility for the "ino64" project have been committed. This will allow most new binaries to run on older kernels in a limited fashion. This prevents many of the common foot-shooting actions in the upgrade as well as the limited ability to roll back the kernel across the ino64 upgrade. Complicated use cases may not work properly, though enough simpler ones work to allow recovery in most situations. 20170620: Switch back to the BSDL dtc (Device Tree Compiler). Set WITH_GPL_DTC if you require the GPL compiler. 20170618: The internal ABI used for communication between the NFS kernel modules was changed by r320085, so __FreeBSD_version was bumped to ensure all the NFS related modules are updated together. 20170617: The ABI of struct event was changed by extending the data member to 64bit and adding ext fields. For upgrade, same precautions as for the entry 20170523 "ino64" must be followed. 20170531: The GNU roff toolchain has been removed from base. To render manpages which are not supported by mandoc(1), man(1) can fallback on GNU roff from ports (and recommends to install it). To render roff(7) documents, consider using GNU roff from ports or the heirloom doctools roff toolchain from ports via pkg install groff or via pkg install heirloom-doctools. 20170524: The ath(4) and ath_hal(4) modules now build piecemeal to allow for smaller runtime footprint builds. This is useful for embedded systems which only require one chipset support. If you load it as a module, make sure this is in /boot/loader.conf: if_ath_load="YES" This will load the HAL, all chip/RF backends and if_ath_pci. If you have if_ath_pci in /boot/loader.conf, ensure it is after if_ath or it will not load any HAL chipset support. If you want to selectively load things (eg on ye cheape ARM/MIPS platforms where RAM is at a premium) you should: * load ath_hal * load the chip modules in question * load ath_rate, ath_dfs * load ath_main * load if_ath_pci and/or if_ath_ahb depending upon your particular bus bind type - this is where probe/attach is done. For further comments/feedback, poke adrian@ . 20170523: The "ino64" 64-bit inode project has been committed, which extends a number of types to 64 bits. Upgrading in place requires care and adherence to the documented upgrade procedure. If using a custom kernel configuration ensure that the COMPAT_FREEBSD11 option is included (as during the upgrade the system will be running the ino64 kernel with the existing world). For the safest in-place upgrade begin by removing previous build artifacts via "rm -rf /usr/obj/*". Then, carefully follow the full procedure documented below under the heading "To rebuild everything and install it on the current system." Specifically, a reboot is required after installing the new kernel before installing world. 20170424: The NATM framework including the en(4), fatm(4), hatm(4), and patm(4) devices has been removed. Consumers should plan a migration before the end-of-life date for FreeBSD 11. 20170420: GNU diff has been replaced by a BSD licensed diff. Some features of GNU diff has not been implemented, if those are needed a newer version of GNU diff is available via the diffutils package under the gdiff name. 20170413: As of r316810 for ipfilter, keep frags is no longer assumed when keep state is specified in a rule. r316810 aligns ipfilter with documentation in man pages separating keep frags from keep state. This allows keep state to be specified without forcing keep frags and allows keep frags to be specified independently of keep state. To maintain previous behaviour, also specify keep frags with keep state (as documented in ipf.conf.5). 20170407: arm64 builds now use the base system LLD 4.0.0 linker by default, instead of requiring that the aarch64-binutils port or package be installed. To continue using aarch64-binutils, set CROSS_BINUTILS_PREFIX=/usr/local/aarch64-freebsd/bin . 20170405: The UDP optimization in entry 20160818 that added the sysctl net.inet.udp.require_l2_bcast has been reverted. L2 broadcast packets will no longer be treated as L3 broadcast packets. 20170331: Binds and sends to the loopback addresses, IPv6 and IPv4, will now use any explicitly assigned loopback address available in the jail instead of using the first assigned address of the jail. 20170329: The ctl.ko module no longer implements the iSCSI target frontend: cfiscsi.ko does instead. If building cfiscsi.ko as a kernel module, the module can be loaded via one of the following methods: - `cfiscsi_load="YES"` in loader.conf(5). - Add `cfiscsi` to `$kld_list` in rc.conf(5). - ctladm(8)/ctld(8), when compiled with iSCSI support (`WITH_ISCSI=yes` in src.conf(5)) Please see cfiscsi(4) for more details. 20170316: The mmcsd.ko module now additionally depends on geom_flashmap.ko. Also, mmc.ko and mmcsd.ko need to be a matching pair built from the same source (previously, the dependency of mmcsd.ko on mmc.ko was missing, but mmcsd.ko now will refuse to load if it is incompatible with mmc.ko). 20170315: The syntax of ipfw(8) named states was changed to avoid ambiguity. If you have used named states in the firewall rules, you need to modify them after installworld and before rebooting. Now named states must be prefixed with colon. 20170311: The old drm (sys/dev/drm/) drivers for i915 and radeon have been removed as the userland we provide cannot use them. The KMS version (sys/dev/drm2) supports the same hardware. 20170302: Clang, llvm, lldb, compiler-rt and libc++ have been upgraded to 4.0.0. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using clang 3.5.0 or higher. 20170221: The code that provides support for ZFS .zfs/ directory functionality has been reimplemented. It's not possible now to create a snapshot by mkdir under .zfs/snapshot/. That should be the only user visible change. 20170216: EISA bus support has been removed. The WITH_EISA option is no longer valid. 20170215: MCA bus support has been removed. 20170127: The WITH_LLD_AS_LD / WITHOUT_LLD_AS_LD build knobs have been renamed WITH_LLD_IS_LD / WITHOUT_LLD_IS_LD, for consistency with CLANG_IS_CC. 20170112: The EM_MULTIQUEUE kernel configuration option is deprecated now that the em(4) driver conforms to iflib specifications. 20170109: The igb(4), em(4) and lem(4) ethernet drivers are now implemented via IFLIB. If you have a custom kernel configuration that excludes em(4) but you use igb(4), you need to re-add em(4) to your custom configuration. 20161217: Clang, llvm, lldb, compiler-rt and libc++ have been upgraded to 3.9.1. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using clang 3.5.0 or higher. 20161124: Clang, llvm, lldb, compiler-rt and libc++ have been upgraded to 3.9.0. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using clang 3.5.0 or higher. 20161119: The layout of the pmap structure has changed for powerpc to put the pmap statistics at the front for all CPU variations. libkvm(3) and all tools that link against it need to be recompiled. 20161030: isl(4) and cyapa(4) drivers now require a new driver, chromebook_platform(4), to work properly on Chromebook-class hardware. On other types of hardware the drivers may need to be configured using device hints. Please see the corresponding manual pages for details. 20161017: The urtwn(4) driver was merged into rtwn(4) and now consists of rtwn(4) main module + rtwn_usb(4) and rtwn_pci(4) bus-specific parts. Also, firmware for RTL8188CE was renamed due to possible name conflict (rtwnrtl8192cU(B) -> rtwnrtl8192cE(B)) 20161015: GNU rcs has been removed from base. It is available as packages: - rcs: Latest GPLv3 GNU rcs version. - rcs57: Copy of the latest version of GNU rcs (GPLv2) before it was removed from base. 20161008: Use of the cc_cdg, cc_chd, cc_hd, or cc_vegas congestion control modules now requires that the kernel configuration contain the TCP_HHOOK option. (This option is included in the GENERIC kernel.) 20161003: The WITHOUT_ELFCOPY_AS_OBJCOPY src.conf(5) knob has been retired. ELF Tool Chain's elfcopy is always installed as /usr/bin/objcopy. 20160924: Relocatable object files with the extension of .So have been renamed to use an extension of .pico instead. The purpose of this change is to avoid a name clash with shared libraries on case-insensitive file systems. On those file systems, foo.So is the same file as foo.so. 20160918: GNU rcs has been turned off by default. It can (temporarily) be built again by adding WITH_RCS knob in src.conf. Otherwise, GNU rcs is available from packages: - rcs: Latest GPLv3 GNU rcs version. - rcs57: Copy of the latest version of GNU rcs (GPLv2) from base. 20160918: The backup_uses_rcs functionality has been removed from rc.subr. 20160908: The queue(3) debugging macro, QUEUE_MACRO_DEBUG, has been split into two separate components, QUEUE_MACRO_DEBUG_TRACE and QUEUE_MACRO_DEBUG_TRASH. Define both for the original QUEUE_MACRO_DEBUG behavior. 20160824: r304787 changed some ioctl interfaces between the iSCSI userspace programs and the kernel. ctladm, ctld, iscsictl, and iscsid must be rebuilt to work with new kernels. __FreeBSD_version has been bumped to 1200005. 20160818: The UDP receive code has been updated to only treat incoming UDP packets that were addressed to an L2 broadcast address as L3 broadcast packets. It is not expected that this will affect any standards-conforming UDP application. The new behaviour can be disabled by setting the sysctl net.inet.udp.require_l2_bcast to 0. 20160818: Remove the openbsd_poll system call. __FreeBSD_version has been bumped because of this. 20160708: The stable/11 branch has been created from head@r302406. 20160622: The libc stub for the pipe(2) system call has been replaced with a wrapper that calls the pipe2(2) system call and the pipe(2) system call is now only implemented by the kernels that include "options COMPAT_FREEBSD10" in their config file (this is the default). Users should ensure that this option is enabled in their kernel or upgrade userspace to r302092 before upgrading their kernel. 20160527: CAM will now strip leading spaces from SCSI disks' serial numbers. This will affect users who create UFS filesystems on SCSI disks using those disk's diskid device nodes. For example, if /etc/fstab previously contained a line like "/dev/diskid/DISK-%20%20%20%20%20%20%20ABCDEFG0123456", you should change it to "/dev/diskid/DISK-ABCDEFG0123456". Users of geom transforms like gmirror may also be affected. ZFS users should generally be fine. 20160523: The bitstring(3) API has been updated with new functionality and improved performance. But it is binary-incompatible with the old API. Objects built with the new headers may not be linked against objects built with the old headers. 20160520: The brk and sbrk functions have been removed from libc on arm64. Binutils from ports has been updated to not link to these functions and should be updated to the latest version before installing a new libc. 20160517: The armv6 port now defaults to hard float ABI. Limited support for running both hardfloat and soft float on the same system is available using the libraries installed with -DWITH_LIBSOFT. This has only been tested as an upgrade path for installworld and packages may fail or need manual intervention to run. New packages will be needed. To update an existing self-hosted armv6hf system, you must add TARGET_ARCH=armv6 on the make command line for both the build and the install steps. 20160510: Kernel modules compiled outside of a kernel build now default to installing to /boot/modules instead of /boot/kernel. Many kernel modules built this way (such as those in ports) already overrode KMODDIR explicitly to install into /boot/modules. However, manually building and installing a module from /sys/modules will now install to /boot/modules instead of /boot/kernel. 20160414: The CAM I/O scheduler has been committed to the kernel. There should be no user visible impact. This does enable NCQ Trim on ada SSDs. While the list of known rogues that claim support for this but actually corrupt data is believed to be complete, be on the lookout for data corruption. The known rogue list is believed to be complete: o Crucial MX100, M550 drives with MU01 firmware. o Micron M510 and M550 drives with MU01 firmware. o Micron M500 prior to MU07 firmware o Samsung 830, 840, and 850 all firmwares o FCCT M500 all firmwares Crucial has firmware http://www.crucial.com/usa/en/support-ssd-firmware with working NCQ TRIM. For Micron branded drives, see your sales rep for updated firmware. Black listed drives will work correctly because these drives work correctly so long as no NCQ TRIMs are sent to them. Given this list is the same as found in Linux, it's believed there are no other rogues in the market place. All other models from the above vendors work. To be safe, if you are at all concerned, you can quirk each of your drives to prevent NCQ from being sent by setting: kern.cam.ada.X.quirks="0x2" in loader.conf. If the drive requires the 4k sector quirk, set the quirks entry to 0x3. 20160330: The FAST_DEPEND build option has been removed and its functionality is now the one true way. The old mkdep(1) style of 'make depend' has been removed. See 20160311 for further details. 20160317: Resource range types have grown from unsigned long to uintmax_t. All drivers, and anything using libdevinfo, need to be recompiled. 20160311: WITH_FAST_DEPEND is now enabled by default for in-tree and out-of-tree builds. It no longer runs mkdep(1) during 'make depend', and the 'make depend' stage can safely be skipped now as it is auto ran when building 'make all' and will generate all SRCS and DPSRCS before building anything else. Dependencies are gathered at compile time with -MF flags kept in separate .depend files per object file. Users should run 'make cleandepend' once if using -DNO_CLEAN to clean out older stale .depend files. 20160306: On amd64, clang 3.8.0 can now insert sections of type AMD64_UNWIND into kernel modules. Therefore, if you load any kernel modules at boot time, please install the boot loaders after you install the kernel, but before rebooting, e.g.: make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installkernel KERNCONF=YOUR_KERNEL_HERE make -C sys/boot install Then follow the usual steps, described in the General Notes section, below. 20160305: Clang, llvm, lldb and compiler-rt have been upgraded to 3.8.0. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using clang 3.5.0 or higher. 20160301: The AIO subsystem is now a standard part of the kernel. The VFS_AIO kernel option and aio.ko kernel module have been removed. Due to stability concerns, asynchronous I/O requests are only permitted on sockets and raw disks by default. To enable asynchronous I/O requests on all file types, set the vfs.aio.enable_unsafe sysctl to a non-zero value. 20160226: The ELF object manipulation tool objcopy is now provided by the ELF Tool Chain project rather than by GNU binutils. It should be a drop-in replacement, with the addition of arm64 support. The (temporary) src.conf knob WITHOUT_ELFCOPY_AS_OBJCOPY knob may be set to obtain the GNU version if necessary. 20160129: Building ZFS pools on top of zvols is prohibited by default. That feature has never worked safely; it's always been prone to deadlocks. Using a zvol as the backing store for a VM guest's virtual disk will still work, even if the guest is using ZFS. Legacy behavior can be restored by setting vfs.zfs.vol.recursive=1. 20160119: The NONE and HPN patches has been removed from OpenSSH. They are still available in the security/openssh-portable port. 20160113: With the addition of ypldap(8), a new _ypldap user is now required during installworld. "mergemaster -p" can be used to add the user prior to installworld, as documented in the handbook. 20151216: The tftp loader (pxeboot) now uses the option root-path directive. As a consequence it no longer looks for a pxeboot.4th file on the tftp server. Instead it uses the regular /boot infrastructure as with the other loaders. 20151211: The code to start recording plug and play data into the modules has been committed. While the old tools will properly build a new kernel, a number of warnings about "unknown metadata record 4" will be produced for an older kldxref. To avoid such warnings, make sure to rebuild the kernel toolchain (or world). Make sure that you have r292078 or later when trying to build 292077 or later before rebuilding. 20151207: Debug data files are now built by default with 'make buildworld' and installed with 'make installworld'. This facilitates debugging but requires more disk space both during the build and for the installed world. Debug files may be disabled by setting WITHOUT_DEBUG_FILES=yes in src.conf(5). 20151130: r291527 changed the internal interface between the nfsd.ko and nfscommon.ko modules. As such, they must both be upgraded to-gether. __FreeBSD_version has been bumped because of this. 20151108: Add support for unicode collation strings leads to a change of order of files listed by ls(1) for example. To get back to the old behaviour, set LC_COLLATE environment variable to "C". Databases administrators will need to reindex their databases given collation results will be different. Due to a bug in install(1) it is recommended to remove the ancient locales before running make installworld. rm -rf /usr/share/locale/* 20151030: The OpenSSL has been upgraded to 1.0.2d. Any binaries requiring libcrypto.so.7 or libssl.so.7 must be recompiled. 20151020: Qlogic 24xx/25xx firmware images were updated from 5.5.0 to 7.3.0. Kernel modules isp_2400_multi and isp_2500_multi were removed and should be replaced with isp_2400 and isp_2500 modules respectively. 20151017: The build previously allowed using 'make -n' to not recurse into sub-directories while showing what commands would be executed, and 'make -n -n' to recursively show commands. Now 'make -n' will recurse and 'make -N' will not. 20151012: If you specify SENDMAIL_MC or SENDMAIL_CF in make.conf, mergemaster and etcupdate will now use this file. A custom sendmail.cf is now updated via this mechanism rather than via installworld. If you had excluded sendmail.cf in mergemaster.rc or etcupdate.conf, you may want to remove the exclusion or change it to "always install". /etc/mail/sendmail.cf is now managed the same way regardless of whether SENDMAIL_MC/SENDMAIL_CF is used. If you are not using SENDMAIL_MC/SENDMAIL_CF there should be no change in behavior. 20151011: Compatibility shims for legacy ATA device names have been removed. It includes ATA_STATIC_ID kernel option, kern.cam.ada.legacy_aliases and kern.geom.raid.legacy_aliases loader tunables, kern.devalias.* environment variables, /dev/ad* and /dev/ar* symbolic links. 20151006: Clang, llvm, lldb, compiler-rt and libc++ have been upgraded to 3.7.0. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using clang 3.5.0 or higher. 20150924: Kernel debug files have been moved to /usr/lib/debug/boot/kernel/, and renamed from .symbols to .debug. This reduces the size requirements on the boot partition or file system and provides consistency with userland debug files. When using the supported kernel installation method the /usr/lib/debug/boot/kernel directory will be renamed (to kernel.old) as is done with /boot/kernel. Developers wishing to maintain the historical behavior of installing debug files in /boot/kernel/ can set KERN_DEBUGDIR="" in src.conf(5). 20150827: The wireless drivers had undergone changes that remove the 'parent interface' from the ifconfig -l output. The rc.d network scripts used to check presence of a parent interface in the list, so old scripts would fail to start wireless networking. Thus, etcupdate(3) or mergemaster(8) run is required after kernel update, to update your rc.d scripts in /etc. 20150827: pf no longer supports 'scrub fragment crop' or 'scrub fragment drop-ovl' These configurations are now automatically interpreted as 'scrub fragment reassemble'. 20150817: Kernel-loadable modules for the random(4) device are back. To use them, the kernel must have device random options RANDOM_LOADABLE kldload(8) can then be used to load random_fortuna.ko or random_yarrow.ko. Please note that due to the indirect function calls that the loadable modules need to provide, the build-in variants will be slightly more efficient. The random(4) kernel option RANDOM_DUMMY has been retired due to unpopularity. It was not all that useful anyway. 20150813: The WITHOUT_ELFTOOLCHAIN_TOOLS src.conf(5) knob has been retired. Control over building the ELF Tool Chain tools is now provided by the WITHOUT_TOOLCHAIN knob. 20150810: The polarity of Pulse Per Second (PPS) capture events with the uart(4) driver has been corrected. Prior to this change the PPS "assert" event corresponded to the trailing edge of a positive PPS pulse and the "clear" event was the leading edge of the next pulse. As the width of a PPS pulse in a typical GPS receiver is on the order of 1 millisecond, most users will not notice any significant difference with this change. Anyone who has compensated for the historical polarity reversal by configuring a negative offset equal to the pulse width will need to remove that workaround. 20150809: The default group assigned to /dev/dri entries has been changed from 'wheel' to 'video' with the id of '44'. If you want to have access to the dri devices please add yourself to the video group with: # pw groupmod video -m $USER 20150806: The menu.rc and loader.rc files will now be replaced during upgrades. Please migrate local changes to menu.rc.local and loader.rc.local instead. 20150805: GNU Binutils versions of addr2line, c++filt, nm, readelf, size, strings and strip have been removed. The src.conf(5) knob WITHOUT_ELFTOOLCHAIN_TOOLS no longer provides the binutils tools. 20150728: As ZFS requires more kernel stack pages than is the default on some architectures e.g. i386, it now warns if KSTACK_PAGES is less than ZFS_MIN_KSTACK_PAGES (which is 4 at the time of writing). Please consider using 'options KSTACK_PAGES=X' where X is greater than or equal to ZFS_MIN_KSTACK_PAGES i.e. 4 in such configurations. 20150706: sendmail has been updated to 8.15.2. Starting with FreeBSD 11.0 and sendmail 8.15, sendmail uses uncompressed IPv6 addresses by default, i.e., they will not contain "::". For example, instead of ::1, it will be 0:0:0:0:0:0:0:1. This permits a zero subnet to have a more specific match, such as different map entries for IPv6:0:0 vs IPv6:0. This change requires that configuration data (including maps, files, classes, custom ruleset, etc.) must use the same format, so make certain such configuration data is upgrading. As a very simple check search for patterns like 'IPv6:[0-9a-fA-F:]*::' and 'IPv6::'. To return to the old behavior, set the m4 option confUSE_COMPRESSED_IPV6_ADDRESSES or the cf option UseCompressedIPv6Addresses. 20150630: The default kernel entropy-processing algorithm is now Fortuna, replacing Yarrow. Assuming you have 'device random' in your kernel config file, the configurations allow a kernel option to override this default. You may choose *ONE* of: options RANDOM_YARROW # Legacy /dev/random algorithm. options RANDOM_DUMMY # Blocking-only driver. If you have neither, you get Fortuna. For most people, read no further, Fortuna will give a /dev/random that works like it always used to, and the difference will be irrelevant. If you remove 'device random', you get *NO* kernel-processed entropy at all. This may be acceptable to folks building embedded systems, but has complications. Carry on reading, and it is assumed you know what you need. *PLEASE* read random(4) and random(9) if you are in the habit of tweaking kernel configs, and/or if you are a member of the embedded community, wanting specific and not-usual behaviour from your security subsystems. NOTE!! If you use RANDOM_DUMMY and/or have no 'device random', you will NOT have a functioning /dev/random, and many cryptographic features will not work, including SSH. You may also find strange behaviour from the random(3) set of library functions, in particular sranddev(3), srandomdev(3) and arc4random(3). The reason for this is that the KERN_ARND sysctl only returns entropy if it thinks it has some to share, and with RANDOM_DUMMY or no 'device random' this will never happen. 20150623: An additional fix for the issue described in the 20150614 sendmail entry below has been committed in revision 284717. 20150616: FreeBSD's old make (fmake) has been removed from the system. It is available as the devel/fmake port or via pkg install fmake. 20150615: The fix for the issue described in the 20150614 sendmail entry below has been committed in revision 284436. The work around described in that entry is no longer needed unless the default setting is overridden by a confDH_PARAMETERS configuration setting of '5' or pointing to a 512 bit DH parameter file. 20150614: ALLOW_DEPRECATED_ATF_TOOLS/ATFFILE support has been removed from atf.test.mk (included from bsd.test.mk). Please upgrade devel/atf and devel/kyua to version 0.20+ and adjust any calling code to work with Kyuafile and kyua. 20150614: The import of openssl to address the FreeBSD-SA-15:10.openssl security advisory includes a change which rejects handshakes with DH parameters below 768 bits. sendmail releases prior to 8.15.2 (not yet released), defaulted to a 512 bit DH parameter setting for client connections. To work around this interoperability, sendmail can be configured to use a 2048 bit DH parameter by: 1. Edit /etc/mail/`hostname`.mc 2. If a setting for confDH_PARAMETERS does not exist or exists and is set to a string beginning with '5', replace it with '2'. 3. If a setting for confDH_PARAMETERS exists and is set to a file path, create a new file with: openssl dhparam -out /path/to/file 2048 4. Rebuild the .cf file: cd /etc/mail/; make; make install 5. Restart sendmail: cd /etc/mail/; make restart A sendmail patch is coming, at which time this file will be updated. 20150604: Generation of legacy formatted entries have been disabled by default in pwd_mkdb(8), as all base system consumers of the legacy formatted entries were converted to use the new format by default when the new, machine independent format have been added and supported since FreeBSD 5.x. Please see the pwd_mkdb(8) manual page for further details. 20150525: Clang and llvm have been upgraded to 3.6.1 release. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using 3.5.0 or higher. 20150521: TI platform code switched to using vendor DTS files and this update may break existing systems running on Beaglebone, Beaglebone Black, and Pandaboard: - dtb files should be regenerated/reinstalled. Filenames are the same but content is different now - GPIO addressing was changed, now each GPIO bank (32 pins per bank) has its own /dev/gpiocX device, e.g. pin 121 on /dev/gpioc0 in old addressing scheme is now pin 25 on /dev/gpioc3. - Pandaboard: /etc/ttys should be updated, serial console device is now /dev/ttyu2, not /dev/ttyu0 20150501: soelim(1) from gnu/usr.bin/groff has been replaced by usr.bin/soelim. If you need the GNU extension from groff soelim(1), install groff from package: pkg install groff, or via ports: textproc/groff. 20150423: chmod, chflags, chown and chgrp now affect symlinks in -R mode as defined in symlink(7); previously symlinks were silently ignored. 20150415: The const qualifier has been removed from iconv(3) to comply with POSIX. The ports tree is aware of this from r384038 onwards. 20150416: Libraries specified by LIBADD in Makefiles must have a corresponding DPADD_ variable to ensure correct dependencies. This is now enforced in src.libnames.mk. 20150324: From legacy ata(4) driver was removed support for SATA controllers supported by more functional drivers ahci(4), siis(4) and mvs(4). Kernel modules ataahci and ataadaptec were removed completely, replaced by ahci and mvs modules respectively. 20150315: Clang, llvm and lldb have been upgraded to 3.6.0 release. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using 3.5.0 or higher. 20150307: The 32-bit PowerPC kernel has been changed to a position-independent executable. This can only be booted with a version of loader(8) newer than January 31, 2015, so make sure to update both world and kernel before rebooting. 20150217: If you are running a -CURRENT kernel since r273872 (Oct 30th, 2014), but before r278950, the RNG was not seeded properly. Immediately upgrade the kernel to r278950 or later and regenerate any keys (e.g. ssh keys or openssl keys) that were generated w/ a kernel from that range. This does not affect programs that directly used /dev/random or /dev/urandom. All userland uses of arc4random(3) are affected. 20150210: The autofs(4) ABI was changed in order to restore binary compatibility with 10.1-RELEASE. The automountd(8) daemon needs to be rebuilt to work with the new kernel. 20150131: The powerpc64 kernel has been changed to a position-independent executable. This can only be booted with a new version of loader(8), so make sure to update both world and kernel before rebooting. 20150118: Clang and llvm have been upgraded to 3.5.1 release. This is a bugfix only release, no new features have been added. Please see the 20141231 entry below for information about prerequisites and upgrading, if you are not already using 3.5.0. 20150107: ELF tools addr2line, elfcopy (strip), nm, size, and strings are now taken from the ELF Tool Chain project rather than GNU binutils. They should be drop-in replacements, with the addition of arm64 support. The WITHOUT_ELFTOOLCHAIN_TOOLS= knob may be used to obtain the binutils tools, if necessary. See 20150805 for updated information. 20150105: The default Unbound configuration now enables remote control using a local socket. Users who have already enabled the local_unbound service should regenerate their configuration by running "service local_unbound setup" as root. 20150102: The GNU texinfo and GNU info pages have been removed. To be able to view GNU info pages please install texinfo from ports. 20141231: Clang, llvm and lldb have been upgraded to 3.5.0 release. As of this release, a prerequisite for building clang, llvm and lldb is a C++11 capable compiler and C++11 standard library. This means that to be able to successfully build the cross-tools stage of buildworld, with clang as the bootstrap compiler, your system compiler or cross compiler should either be clang 3.3 or later, or gcc 4.8 or later, and your system C++ library should be libc++, or libdstdc++ from gcc 4.8 or later. On any standard FreeBSD 10.x or 11.x installation, where clang and libc++ are on by default (that is, on x86 or arm), this should work out of the box. On 9.x installations where clang is enabled by default, e.g. on x86 and powerpc, libc++ will not be enabled by default, so libc++ should be built (with clang) and installed first. If both clang and libc++ are missing, build clang first, then use it to build libc++. On 8.x and earlier installations, upgrade to 9.x first, and then follow the instructions for 9.x above. Sparc64 and mips users are unaffected, as they still use gcc 4.2.1 by default, and do not build clang. Many embedded systems are resource constrained, and will not be able to build clang in a reasonable time, or in some cases at all. In those cases, cross building bootable systems on amd64 is a workaround. This new version of clang introduces a number of new warnings, of which the following are most likely to appear: -Wabsolute-value This warns in two cases, for both C and C++: * When the code is trying to take the absolute value of an unsigned quantity, which is effectively a no-op, and almost never what was intended. The code should be fixed, if at all possible. If you are sure that the unsigned quantity can be safely cast to signed, without loss of information or undefined behavior, you can add an explicit cast, or disable the warning. * When the code is trying to take an absolute value, but the called abs() variant is for the wrong type, which can lead to truncation. If you want to disable the warning instead of fixing the code, please make sure that truncation will not occur, or it might lead to unwanted side-effects. -Wtautological-undefined-compare and -Wundefined-bool-conversion These warn when C++ code is trying to compare 'this' against NULL, while 'this' should never be NULL in well-defined C++ code. However, there is some legacy (pre C++11) code out there, which actively abuses this feature, which was less strictly defined in previous C++ versions. Squid and openjdk do this, for example. The warning can be turned off for C++98 and earlier, but compiling the code in C++11 mode might result in unexpected behavior; for example, the parts of the program that are unreachable could be optimized away. 20141222: The old NFS client and server (kernel options NFSCLIENT, NFSSERVER) kernel sources have been removed. The .h files remain, since some utilities include them. This will need to be fixed later. If "mount -t oldnfs ..." is attempted, it will fail. If the "-o" option on mountd(8), nfsd(8) or nfsstat(1) is used, the utilities will report errors. 20141121: The handling of LOCAL_LIB_DIRS has been altered to skip addition of directories to top level SUBDIR variable when their parent directory is included in LOCAL_DIRS. Users with build systems with such hierarchies and without SUBDIR entries in the parent directory Makefiles should add them or add the directories to LOCAL_DIRS. 20141109: faith(4) and faithd(8) have been removed from the base system. Faith has been obsolete for a very long time. 20141104: vt(4), the new console driver, is enabled by default. It brings support for Unicode and double-width characters, as well as support for UEFI and integration with the KMS kernel video drivers. You may need to update your console settings in /etc/rc.conf, most probably the keymap. During boot, /etc/rc.d/syscons will indicate what you need to do. vt(4) still has issues and lacks some features compared to syscons(4). See the wiki for up-to-date information: https://wiki.freebsd.org/Newcons If you want to keep using syscons(4), you can do so by adding the following line to /boot/loader.conf: kern.vty=sc 20141102: pjdfstest has been integrated into kyua as an opt-in test suite. Please see share/doc/pjdfstest/README for more details on how to execute it. 20141009: gperf has been removed from the base system for architectures that use clang. Ports that require gperf will obtain it from the devel/gperf port. 20140923: pjdfstest has been moved from tools/regression/pjdfstest to contrib/pjdfstest . 20140922: At svn r271982, The default linux compat kernel ABI has been adjusted to 2.6.18 in support of the linux-c6 compat ports infrastructure update. If you wish to continue using the linux-f10 compat ports, add compat.linux.osrelease=2.6.16 to your local sysctl.conf. Users are encouraged to update their linux-compat packages to linux-c6 during their next update cycle. 20140729: The ofwfb driver, used to provide a graphics console on PowerPC when using vt(4), no longer allows mmap() of all physical memory. This will prevent Xorg on PowerPC with some ATI graphics cards from initializing properly unless x11-servers/xorg-server is updated to 1.12.4_8 or newer. 20140723: The xdev targets have been converted to using TARGET and TARGET_ARCH instead of XDEV and XDEV_ARCH. 20140719: The default unbound configuration has been modified to address issues with reverse lookups on networks that use private address ranges. If you use the local_unbound service, run "service local_unbound setup" as root to regenerate your configuration, then "service local_unbound reload" to load the new configuration. 20140709: The GNU texinfo and GNU info pages are not built and installed anymore, WITH_INFO knob has been added to allow to built and install them again. UPDATE: see 20150102 entry on texinfo's removal 20140708: The GNU readline library is now an INTERNALLIB - that is, it is statically linked into consumers (GDB and variants) in the base system, and the shared library is no longer installed. The devel/readline port is available for third party software that requires readline. 20140702: The Itanium architecture (ia64) has been removed from the list of known architectures. This is the first step in the removal of the architecture. 20140701: Commit r268115 has added NFSv4.1 server support, merged from projects/nfsv4.1-server. Since this includes changes to the internal interfaces between the NFS related modules, a full build of the kernel and modules will be necessary. __FreeBSD_version has been bumped. 20140629: The WITHOUT_VT_SUPPORT kernel config knob has been renamed WITHOUT_VT. (The other _SUPPORT knobs have a consistent meaning which differs from the behaviour controlled by this knob.) 20140619: Maximal length of the serial number in CTL was increased from 16 to 64 chars, that breaks ABI. All CTL-related tools, such as ctladm and ctld, need to be rebuilt to work with a new kernel. 20140606: The libatf-c and libatf-c++ major versions were downgraded to 0 and 1 respectively to match the upstream numbers. They were out of sync because, when they were originally added to FreeBSD, the upstream versions were not respected. These libraries are private and not yet built by default, so renumbering them should be a non-issue. However, unclean source trees will yield broken test programs once the operator executes "make delete-old-libs" after a "make installworld". Additionally, the atf-sh binary was made private by moving it into /usr/libexec/. Already-built shell test programs will keep the path to the old binary so they will break after "make delete-old" is run. If you are using WITH_TESTS=yes (not the default), wipe the object tree and rebuild from scratch to prevent spurious test failures. This is only needed once: the misnumbered libraries and misplaced binaries have been added to OptionalObsoleteFiles.inc so they will be removed during a clean upgrade. 20140512: Clang and llvm have been upgraded to 3.4.1 release. 20140508: We bogusly installed src.opts.mk in /usr/share/mk. This file should be removed to avoid issues in the future (and has been added to ObsoleteFiles.inc). 20140505: /etc/src.conf now affects only builds of the FreeBSD src tree. In the past, it affected all builds that used the bsd.*.mk files. The old behavior was a bug, but people may have relied upon it. To get this behavior back, you can .include /etc/src.conf from /etc/make.conf (which is still global and isn't changed). This also changes the behavior of incremental builds inside the tree of individual directories. Set MAKESYSPATH to ".../share/mk" to do that. Although this has survived make universe and some upgrade scenarios, other upgrade scenarios may have broken. At least one form of temporary breakage was fixed with MAKESYSPATH settings for buildworld as well... In cases where MAKESYSPATH isn't working with this setting, you'll need to set it to the full path to your tree. One side effect of all this cleaning up is that bsd.compiler.mk is no longer implicitly included by bsd.own.mk. If you wish to use COMPILER_TYPE, you must now explicitly include bsd.compiler.mk as well. 20140430: The lindev device has been removed since /dev/full has been made a standard device. __FreeBSD_version has been bumped. 20140424: The knob WITHOUT_VI was added to the base system, which controls building ex(1), vi(1), etc. Older releases of FreeBSD required ex(1) in order to reorder files share/termcap and didn't build ex(1) as a build tool, so building/installing with WITH_VI is highly advised for build hosts for older releases. This issue has been fixed in stable/9 and stable/10 in r277022 and r276991, respectively. 20140418: The YES_HESIOD knob has been removed. It has been obsolete for a decade. Please move to using WITH_HESIOD instead or your builds will silently lack HESIOD. 20140405: The uart(4) driver has been changed with respect to its handling of the low-level console. Previously the uart(4) driver prevented any process from changing the baudrate or the CLOCAL and HUPCL control flags. By removing the restrictions, operators can make changes to the serial console port without having to reboot. However, when getty(8) is started on the serial device that is associated with the low-level console, a misconfigured terminal line in /etc/ttys will now have a real impact. Before upgrading the kernel, make sure that /etc/ttys has the serial console device configured as 3wire without baudrate to preserve the previous behaviour. E.g: ttyu0 "/usr/libexec/getty 3wire" vt100 on secure 20140306: Support for libwrap (TCP wrappers) in rpcbind was disabled by default to improve performance. To re-enable it, if needed, run rpcbind with command line option -W. 20140226: Switched back to the GPL dtc compiler due to updates in the upstream dts files not being supported by the BSDL dtc compiler. You will need to rebuild your kernel toolchain to pick up the new compiler. Core dumps may result while building dtb files during a kernel build if you fail to do so. Set WITHOUT_GPL_DTC if you require the BSDL compiler. 20140216: Clang and llvm have been upgraded to 3.4 release. 20140216: The nve(4) driver has been removed. Please use the nfe(4) driver for NVIDIA nForce MCP Ethernet adapters instead. 20140212: An ABI incompatibility crept into the libc++ 3.4 import in r261283. This could cause certain C++ applications using shared libraries built against the previous version of libc++ to crash. The incompatibility has now been fixed, but any C++ applications or shared libraries built between r261283 and r261801 should be recompiled. 20140204: OpenSSH will now ignore errors caused by kernel lacking of Capsicum capability mode support. Please note that enabling the feature in kernel is still highly recommended. 20140131: OpenSSH is now built with sandbox support, and will use sandbox as the default privilege separation method. This requires Capsicum capability mode support in kernel. 20140128: The libelf and libdwarf libraries have been updated to newer versions from upstream. Shared library version numbers for these two libraries were bumped. Any ports or binaries requiring these two libraries should be recompiled. __FreeBSD_version is bumped to 1100006. 20140110: If a Makefile in a tests/ directory was auto-generating a Kyuafile instead of providing an explicit one, this would prevent such Makefile from providing its own Kyuafile in the future during NO_CLEAN builds. This has been fixed in the Makefiles but manual intervention is needed to clean an objdir if you use NO_CLEAN: # find /usr/obj -name Kyuafile | xargs rm -f 20131213: The behavior of gss_pseudo_random() for the krb5 mechanism has changed, for applications requesting a longer random string than produced by the underlying enctype's pseudo-random() function. In particular, the random string produced from a session key of enctype aes256-cts-hmac-sha1-96 or aes256-cts-hmac-sha1-96 will be different at the 17th octet and later, after this change. The counter used in the PRF+ construction is now encoded as a big-endian integer in accordance with RFC 4402. __FreeBSD_version is bumped to 1100004. 20131108: The WITHOUT_ATF build knob has been removed and its functionality has been subsumed into the more generic WITHOUT_TESTS. If you were using the former to disable the build of the ATF libraries, you should change your settings to use the latter. 20131025: The default version of mtree is nmtree which is obtained from NetBSD. The output is generally the same, but may vary slightly. If you found you need identical output adding "-F freebsd9" to the command line should do the trick. For the time being, the old mtree is available as fmtree. 20131014: libbsdyml has been renamed to libyaml and moved to /usr/lib/private. This will break ports-mgmt/pkg. Rebuild the port, or upgrade to pkg 1.1.4_8 and verify bsdyml not linked in, before running "make delete-old-libs": # make -C /usr/ports/ports-mgmt/pkg build deinstall install clean or # pkg install pkg; ldd /usr/local/sbin/pkg | grep bsdyml 20131010: The stable/10 branch has been created in subversion from head revision r256279. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. While generally safe, there are sometimes problems using -j to upgrade. If your upgrade fails with -j, please try again without -j. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. When upgrading from one major version to another it is generally best to upgrade to the latest code in the currently installed branch first, then do an upgrade to the new branch. This is the best-tested upgrade path, and has the highest probability of being successful. Please try this approach if you encounter problems with a major version upgrade. Since the stable 4.x branch point, one has generally been able to upgrade from anywhere in the most recent stable branch to head / current (or even the last couple of stable branches). See the top of this file when there's an exception. When upgrading a live system, having a root shell around before installing anything can help undo problems. Not having a root shell around can lead to problems if pam has changed too much from your starting point to allow continued authentication after the upgrade. This file should be read as a log of events. When a later event changes information of a prior event, the prior event should not be deleted. Instead, a pointer to the entry with the new information should be placed in the old entry. Readers of this file should also sanity check older entries before relying on them blindly. Authors of new entries should write them with this in mind. ZFS notes --------- When upgrading the boot ZFS pool to a new version, always follow these two steps: 1.) recompile and reinstall the ZFS boot loader and boot block (this is part of "make buildworld" and "make installworld") 2.) update the ZFS boot block on your boot drive The following example updates the ZFS boot block on the first partition (freebsd-boot) of a GPT partitioned drive ada0: "gpart bootcode -p /boot/gptzfsboot -i 1 ada0" Non-boot pools do not need these updates. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. It is the most failsafe as it uses a /usr/obj tree with a fresh mini-buildworld, make kernel-toolchain make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To test a kernel once --------------------- If you just want to boot a kernel once (because you are not sure if it works, or if you want to boot a known bad kernel to provide debugging information) run make installkernel KERNCONF=YOUR_KERNEL_HERE KODIR=/boot/testkernel nextboot -k testkernel To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installkernel KERNCONF=YOUR_KERNEL_HERE [1] [3] mergemaster -Fp [5] make installworld mergemaster -Fi [4] make delete-old [6] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installworld DESTDIR=${CURRENT_ROOT} -DDB_FROM_SRC make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from stable to current ---------------------------------------------- make buildworld [9] make buildkernel KERNCONF=YOUR_KERNEL_HERE [8] make installkernel KERNCONF=YOUR_KERNEL_HERE [1] [3] mergemaster -Fp [5] make installworld mergemaster -Fi [4] make delete-old [6] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. The -U option is also useful to consider. See mergemaster(8) for more information. [5] Usually this step is a no-op. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. You may need to install a new mergemaster (cd src/usr.sbin/mergemaster && make install) after the buildworld before this step if you last updated from current before 20130425 or from -stable before 20130430. [6] This only deletes old files and directories. Old libraries can be deleted by "make delete-old-libs", but you have to make sure that no program is using those libraries anymore. [8] In order to have a kernel that can run the 4.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD4 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. A similar kernel option COMPAT_FREEBSD5 is required to run the 5.x binaries on more recent kernels. And so on for COMPAT_FREEBSD6 and COMPAT_FREEBSD7. Make sure that you merge any new devices from GENERIC since the last time you updated your kernel config file. [9] If CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. MAKEOBJDIRPREFIX must be defined in an environment variable, and not on the command line, or in /etc/make.conf. buildworld will warn if it is improperly defined. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -current. It is not guaranteed to be a complete list of such breakages, and only contains entries since September 23, 2011. If you need to see UPDATING entries from before that date, you will need to fetch an UPDATING file from an older FreeBSD release. Copyright information: Copyright 1998-2009 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: projects/clang700-import/bin/dd/misc.c =================================================================== --- projects/clang700-import/bin/dd/misc.c (revision 338689) +++ projects/clang700-import/bin/dd/misc.c (revision 338690) @@ -1,157 +1,157 @@ /*- * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 1991, 1993, 1994 * The Regents of the University of California. All rights reserved. * * This code is derived from software contributed to Berkeley by * Keith Muller of the University of California, San Diego and Lance * Visser of Convex Computer Corporation. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #ifndef lint #if 0 static char sccsid[] = "@(#)misc.c 8.3 (Berkeley) 4/2/94"; #endif #endif /* not lint */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include "dd.h" #include "extern.h" double secs_elapsed(void) { struct timespec end, ts_res; double secs, res; if (clock_gettime(CLOCK_MONOTONIC, &end)) err(1, "clock_gettime"); if (clock_getres(CLOCK_MONOTONIC, &ts_res)) err(1, "clock_getres"); secs = (end.tv_sec - st.start.tv_sec) + \ (end.tv_nsec - st.start.tv_nsec) * 1e-9; res = ts_res.tv_sec + ts_res.tv_nsec * 1e-9; if (secs < res) secs = res; return (secs); } void summary(void) { double secs; if (ddflags & C_NOINFO) return; if (ddflags & C_PROGRESS) fprintf(stderr, "\n"); secs = secs_elapsed(); (void)fprintf(stderr, "%ju+%ju records in\n%ju+%ju records out\n", st.in_full, st.in_part, st.out_full, st.out_part); if (st.swab) (void)fprintf(stderr, "%ju odd length swab %s\n", st.swab, (st.swab == 1) ? "block" : "blocks"); if (st.trunc) (void)fprintf(stderr, "%ju truncated %s\n", st.trunc, (st.trunc == 1) ? "block" : "blocks"); if (!(ddflags & C_NOXFER)) { (void)fprintf(stderr, "%ju bytes transferred in %.6f secs (%.0f bytes/sec)\n", st.bytes, secs, st.bytes / secs); } need_summary = 0; } void progress(void) { static int outlen; char si[4 + 1 + 2 + 1]; /* 123 NUL */ - char iec[4 + 1 + 2 + 1]; /* 123 NUL */ + char iec[4 + 1 + 3 + 1]; /* 123 NUL */ char persec[4 + 1 + 2 + 1]; /* 123 NUL */ char *buf; double secs; secs = secs_elapsed(); humanize_number(si, sizeof(si), (int64_t)st.bytes, "B", HN_AUTOSCALE, HN_DECIMAL | HN_DIVISOR_1000); humanize_number(iec, sizeof(iec), (int64_t)st.bytes, "B", HN_AUTOSCALE, HN_DECIMAL | HN_IEC_PREFIXES); - humanize_number(persec, sizeof(iec), (int64_t)(st.bytes / secs), "B", + humanize_number(persec, sizeof(persec), (int64_t)(st.bytes / secs), "B", HN_AUTOSCALE, HN_DECIMAL | HN_DIVISOR_1000); asprintf(&buf, " %'ju bytes (%s, %s) transferred %.3fs, %s/s", (uintmax_t)st.bytes, si, iec, secs, persec); - outlen = fprintf(stderr, "%-*s\r", outlen, buf); + outlen = fprintf(stderr, "%-*s\r", outlen, buf) - 1; fflush(stderr); free(buf); need_progress = 0; } /* ARGSUSED */ void siginfo_handler(int signo __unused) { need_summary = 1; } /* ARGSUSED */ void sigalarm_handler(int signo __unused) { need_progress = 1; } /* ARGSUSED */ void terminate(int sig) { summary(); _exit(sig == 0 ? 0 : 1); } Index: projects/clang700-import/bin/rm/Makefile =================================================================== --- projects/clang700-import/bin/rm/Makefile (revision 338689) +++ projects/clang700-import/bin/rm/Makefile (revision 338690) @@ -1,10 +1,15 @@ # @(#)Makefile 8.1 (Berkeley) 5/31/93 # $FreeBSD$ +.include + PACKAGE=runtime PROG= rm LINKS= ${BINDIR}/rm ${BINDIR}/unlink MLINKS= rm.1 unlink.1 + +HAS_TESTS= +SUBDIR.${MK_TESTS}+= tests .include Index: projects/clang700-import/bin/rm/rm.1 =================================================================== --- projects/clang700-import/bin/rm/rm.1 (revision 338689) +++ projects/clang700-import/bin/rm/rm.1 (revision 338690) @@ -1,259 +1,264 @@ .\"- .\" Copyright (c) 1990, 1993, 1994 .\" The Regents of the University of California. All rights reserved. .\" .\" This code is derived from software contributed to Berkeley by .\" the Institute of Electrical and Electronics Engineers, Inc. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" 3. Neither the name of the University nor the names of its contributors .\" may be used to endorse or promote products derived from this software .\" without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .\" @(#)rm.1 8.5 (Berkeley) 12/5/94 .\" $FreeBSD$ .\" -.Dd November 7, 2015 +.Dd September 12, 2018 .Dt RM 1 .Os .Sh NAME .Nm rm , .Nm unlink .Nd remove directory entries .Sh SYNOPSIS .Nm .Op Fl f | i .Op Fl dIPRrvWx .Ar .Nm unlink +.Op Fl - .Ar file .Sh DESCRIPTION The .Nm utility attempts to remove the non-directory type files specified on the command line. If the permissions of the file do not permit writing, and the standard input device is a terminal, the user is prompted (on the standard error output) for confirmation. .Pp The options are as follows: .Bl -tag -width indent .It Fl d Attempt to remove directories as well as other types of files. .It Fl f Attempt to remove the files without prompting for confirmation, regardless of the file's permissions. If the file does not exist, do not display a diagnostic message or modify the exit status to reflect an error. The .Fl f option overrides any previous .Fl i options. .It Fl i Request confirmation before attempting to remove each file, regardless of the file's permissions, or whether or not the standard input device is a terminal. The .Fl i option overrides any previous .Fl f options. .It Fl I Request confirmation once if more than three files are being removed or if a directory is being recursively removed. This is a far less intrusive option than .Fl i yet provides almost the same level of protection against mistakes. .It Fl P Overwrite regular files before deleting them. Files are overwritten three times, first with the byte pattern 0xff, then 0x00, and then 0xff again, before they are deleted. Files with multiple links will not be overwritten nor deleted and a warning will be issued. If the .Fl f option is specified, files with multiple links will also be overwritten and deleted. No warning will be issued. .Pp Specifying this flag for a read only file will cause .Nm to generate an error message and exit. The file will not be removed or overwritten. .Pp N.B.: The .Fl P flag is not considered a security feature .Pq see Sx BUGS . .It Fl R Attempt to remove the file hierarchy rooted in each .Ar file argument. The .Fl R option implies the .Fl d option. If the .Fl i option is specified, the user is prompted for confirmation before each directory's contents are processed (as well as before the attempt is made to remove the directory). If the user does not respond affirmatively, the file hierarchy rooted in that directory is skipped. .It Fl r Equivalent to .Fl R . .It Fl v Be verbose when deleting files, showing them as they are removed. .It Fl W Attempt to undelete the named files. Currently, this option can only be used to recover files covered by whiteouts in a union file system (see .Xr undelete 2 ) . .It Fl x When removing a hierarchy, do not cross mount points. .El .Pp The .Nm utility removes symbolic links, not the files referenced by the links. .Pp It is an error to attempt to remove the files .Pa / , .Pa .\& or .Pa .. . .Pp When the utility is called as .Nm unlink , only one argument, which must not be a directory, may be supplied. No options may be supplied in this simple mode of operation, which performs an .Xr unlink 2 operation on the passed argument. +However, the usual option-end delimiter, +.Fl - , +may optionally precede the argument. .Sh EXIT STATUS The .Nm utility exits 0 if all of the named files or file hierarchies were removed, or if the .Fl f option was specified and all of the existing files or file hierarchies were removed. If an error occurs, .Nm exits with a value >0. .Sh NOTES The .Nm command uses .Xr getopt 3 to parse its arguments, which allows it to accept the .Sq Li -- option which will cause it to stop processing flag options at that point. This will allow the removal of file names that begin with a dash .Pq Sq - . For example: .Pp .Dl "rm -- -filename" .Pp The same behavior can be obtained by using an absolute or relative path reference. For example: .Pp .Dl "rm /home/user/-filename" .Dl "rm ./-filename" .Pp When .Fl P is specified with .Fl f the file will be overwritten and removed even if it has hard links. .Sh EXAMPLES Recursively remove all files contained within the .Pa foobar directory hierarchy: .Pp .Dl $ rm -rf foobar .Pp -Either of these commands will remove the file +Any of these commands will remove the file .Pa -f : .Bd -literal -offset indent $ rm -- -f $ rm ./-f +$ unlink -f .Ed .Sh COMPATIBILITY The .Nm utility differs from historical implementations in that the .Fl f option only masks attempts to remove non-existent files instead of masking a large variety of errors. The .Fl v option is non-standard and its use in scripts is not recommended. .Pp Also, historical .Bx implementations prompted on the standard output, not the standard error output. .Sh SEE ALSO .Xr chflags 1 , .Xr rmdir 1 , .Xr undelete 2 , .Xr unlink 2 , .Xr fts 3 , .Xr getopt 3 , .Xr symlink 7 .Sh STANDARDS The .Nm command conforms to .St -p1003.1-2013 . .Pp The simplified .Nm unlink command conforms to .St -susv2 . .Sh HISTORY A .Nm command appeared in .At v1 . .Sh BUGS The .Fl P option assumes that the underlying storage overwrites file blocks when data is written to an existing offset. Several factors including the file system and its backing store could defeat this assumption. This includes, but is not limited to file systems that use a Copy-On-Write strategy (e.g. ZFS or UFS when snapshots are being used), Flash media that are using a wear leveling algorithm, or when the backing datastore does journaling, etc. In addition, only regular files are overwritten, other types of files are not. Index: projects/clang700-import/bin/rm/rm.c =================================================================== --- projects/clang700-import/bin/rm/rm.c (revision 338689) +++ projects/clang700-import/bin/rm/rm.c (revision 338690) @@ -1,646 +1,645 @@ /*- * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 1990, 1993, 1994 * The Regents of the University of California. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #if 0 #ifndef lint static const char copyright[] = "@(#) Copyright (c) 1990, 1993, 1994\n\ The Regents of the University of California. All rights reserved.\n"; #endif /* not lint */ #ifndef lint static char sccsid[] = "@(#)rm.c 8.5 (Berkeley) 4/18/94"; #endif /* not lint */ #endif #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include static int dflag, eval, fflag, iflag, Pflag, vflag, Wflag, stdin_ok; static int rflag, Iflag, xflag; static uid_t uid; static volatile sig_atomic_t info; static int check(const char *, const char *, struct stat *); static int check2(char **); static void checkdot(char **); static void checkslash(char **); static void rm_file(char **); static int rm_overwrite(const char *, struct stat *); static void rm_tree(char **); static void siginfo(int __unused); static void usage(void); /* * rm -- * This rm is different from historic rm's, but is expected to match * POSIX 1003.2 behavior. The most visible difference is that -f * has two specific effects now, ignore non-existent files and force * file removal. */ int main(int argc, char *argv[]) { int ch; char *p; (void)setlocale(LC_ALL, ""); /* * Test for the special case where the utility is called as * "unlink", for which the functionality provided is greatly * simplified. */ if ((p = strrchr(argv[0], '/')) == NULL) p = argv[0]; else ++p; if (strcmp(p, "unlink") == 0) { - while (getopt(argc, argv, "") != -1) + if (argc == 2) + rm_file(&argv[1]); + else if (argc == 3 && strcmp(argv[1], "--") == 0) + rm_file(&argv[2]); + else usage(); - argc -= optind; - argv += optind; - if (argc != 1) - usage(); - rm_file(&argv[0]); exit(eval); } Pflag = rflag = xflag = 0; while ((ch = getopt(argc, argv, "dfiIPRrvWx")) != -1) switch(ch) { case 'd': dflag = 1; break; case 'f': fflag = 1; iflag = 0; break; case 'i': fflag = 0; iflag = 1; break; case 'I': Iflag = 1; break; case 'P': Pflag = 1; break; case 'R': case 'r': /* Compatibility. */ rflag = 1; break; case 'v': vflag = 1; break; case 'W': Wflag = 1; break; case 'x': xflag = 1; break; default: usage(); } argc -= optind; argv += optind; if (argc < 1) { if (fflag) return (0); usage(); } checkdot(argv); checkslash(argv); uid = geteuid(); (void)signal(SIGINFO, siginfo); if (*argv) { stdin_ok = isatty(STDIN_FILENO); if (Iflag) { if (check2(argv) == 0) exit (1); } if (rflag) rm_tree(argv); else rm_file(argv); } exit (eval); } static void rm_tree(char **argv) { FTS *fts; FTSENT *p; int needstat; int flags; int rval; /* * Remove a file hierarchy. If forcing removal (-f), or interactive * (-i) or can't ask anyway (stdin_ok), don't stat the file. */ needstat = !uid || (!fflag && !iflag && stdin_ok); /* * If the -i option is specified, the user can skip on the pre-order * visit. The fts_number field flags skipped directories. */ #define SKIPPED 1 flags = FTS_PHYSICAL; if (!needstat) flags |= FTS_NOSTAT; if (Wflag) flags |= FTS_WHITEOUT; if (xflag) flags |= FTS_XDEV; if (!(fts = fts_open(argv, flags, NULL))) { if (fflag && errno == ENOENT) return; err(1, "fts_open"); } while ((p = fts_read(fts)) != NULL) { switch (p->fts_info) { case FTS_DNR: if (!fflag || p->fts_errno != ENOENT) { warnx("%s: %s", p->fts_path, strerror(p->fts_errno)); eval = 1; } continue; case FTS_ERR: errx(1, "%s: %s", p->fts_path, strerror(p->fts_errno)); case FTS_NS: /* * Assume that since fts_read() couldn't stat the * file, it can't be unlinked. */ if (!needstat) break; if (!fflag || p->fts_errno != ENOENT) { warnx("%s: %s", p->fts_path, strerror(p->fts_errno)); eval = 1; } continue; case FTS_D: /* Pre-order: give user chance to skip. */ if (!fflag && !check(p->fts_path, p->fts_accpath, p->fts_statp)) { (void)fts_set(fts, p, FTS_SKIP); p->fts_number = SKIPPED; } else if (!uid && (p->fts_statp->st_flags & (UF_APPEND|UF_IMMUTABLE)) && !(p->fts_statp->st_flags & (SF_APPEND|SF_IMMUTABLE)) && lchflags(p->fts_accpath, p->fts_statp->st_flags &= ~(UF_APPEND|UF_IMMUTABLE)) < 0) goto err; continue; case FTS_DP: /* Post-order: see if user skipped. */ if (p->fts_number == SKIPPED) continue; break; default: if (!fflag && !check(p->fts_path, p->fts_accpath, p->fts_statp)) continue; } rval = 0; if (!uid && (p->fts_statp->st_flags & (UF_APPEND|UF_IMMUTABLE)) && !(p->fts_statp->st_flags & (SF_APPEND|SF_IMMUTABLE))) rval = lchflags(p->fts_accpath, p->fts_statp->st_flags &= ~(UF_APPEND|UF_IMMUTABLE)); if (rval == 0) { /* * If we can't read or search the directory, may still be * able to remove it. Don't print out the un{read,search}able * message unless the remove fails. */ switch (p->fts_info) { case FTS_DP: case FTS_DNR: rval = rmdir(p->fts_accpath); if (rval == 0 || (fflag && errno == ENOENT)) { if (rval == 0 && vflag) (void)printf("%s\n", p->fts_path); if (rval == 0 && info) { info = 0; (void)printf("%s\n", p->fts_path); } continue; } break; case FTS_W: rval = undelete(p->fts_accpath); if (rval == 0 && (fflag && errno == ENOENT)) { if (vflag) (void)printf("%s\n", p->fts_path); if (info) { info = 0; (void)printf("%s\n", p->fts_path); } continue; } break; case FTS_NS: /* * Assume that since fts_read() couldn't stat * the file, it can't be unlinked. */ if (fflag) continue; /* FALLTHROUGH */ case FTS_F: case FTS_NSOK: if (Pflag) if (!rm_overwrite(p->fts_accpath, p->fts_info == FTS_NSOK ? NULL : p->fts_statp)) continue; /* FALLTHROUGH */ default: rval = unlink(p->fts_accpath); if (rval == 0 || (fflag && errno == ENOENT)) { if (rval == 0 && vflag) (void)printf("%s\n", p->fts_path); if (rval == 0 && info) { info = 0; (void)printf("%s\n", p->fts_path); } continue; } } } err: warn("%s", p->fts_path); eval = 1; } if (!fflag && errno) err(1, "fts_read"); fts_close(fts); } static void rm_file(char **argv) { struct stat sb; int rval; char *f; /* * Remove a file. POSIX 1003.2 states that, by default, attempting * to remove a directory is an error, so must always stat the file. */ while ((f = *argv++) != NULL) { /* Assume if can't stat the file, can't unlink it. */ if (lstat(f, &sb)) { if (Wflag) { sb.st_mode = S_IFWHT|S_IWUSR|S_IRUSR; } else { if (!fflag || errno != ENOENT) { warn("%s", f); eval = 1; } continue; } } else if (Wflag) { warnx("%s: %s", f, strerror(EEXIST)); eval = 1; continue; } if (S_ISDIR(sb.st_mode) && !dflag) { warnx("%s: is a directory", f); eval = 1; continue; } if (!fflag && !S_ISWHT(sb.st_mode) && !check(f, f, &sb)) continue; rval = 0; if (!uid && !S_ISWHT(sb.st_mode) && (sb.st_flags & (UF_APPEND|UF_IMMUTABLE)) && !(sb.st_flags & (SF_APPEND|SF_IMMUTABLE))) rval = lchflags(f, sb.st_flags & ~(UF_APPEND|UF_IMMUTABLE)); if (rval == 0) { if (S_ISWHT(sb.st_mode)) rval = undelete(f); else if (S_ISDIR(sb.st_mode)) rval = rmdir(f); else { if (Pflag) if (!rm_overwrite(f, &sb)) continue; rval = unlink(f); } } if (rval && (!fflag || errno != ENOENT)) { warn("%s", f); eval = 1; } if (vflag && rval == 0) (void)printf("%s\n", f); if (info && rval == 0) { info = 0; (void)printf("%s\n", f); } } } /* * rm_overwrite -- * Overwrite the file 3 times with varying bit patterns. * * XXX * This is a cheap way to *really* delete files. Note that only regular * files are deleted, directories (and therefore names) will remain. * Also, this assumes a fixed-block file system (like FFS, or a V7 or a * System V file system). In a logging or COW file system, you'll have to * have kernel support. */ static int rm_overwrite(const char *file, struct stat *sbp) { struct stat sb, sb2; struct statfs fsb; off_t len; int bsize, fd, wlen; char *buf = NULL; fd = -1; if (sbp == NULL) { if (lstat(file, &sb)) goto err; sbp = &sb; } if (!S_ISREG(sbp->st_mode)) return (1); if (sbp->st_nlink > 1 && !fflag) { warnx("%s (inode %ju): not overwritten due to multiple links", file, (uintmax_t)sbp->st_ino); return (0); } if ((fd = open(file, O_WRONLY|O_NONBLOCK|O_NOFOLLOW, 0)) == -1) goto err; if (fstat(fd, &sb2)) goto err; if (sb2.st_dev != sbp->st_dev || sb2.st_ino != sbp->st_ino || !S_ISREG(sb2.st_mode)) { errno = EPERM; goto err; } if (fstatfs(fd, &fsb) == -1) goto err; bsize = MAX(fsb.f_iosize, 1024); if ((buf = malloc(bsize)) == NULL) err(1, "%s: malloc", file); #define PASS(byte) { \ memset(buf, byte, bsize); \ for (len = sbp->st_size; len > 0; len -= wlen) { \ wlen = len < bsize ? len : bsize; \ if (write(fd, buf, wlen) != wlen) \ goto err; \ } \ } PASS(0xff); if (fsync(fd) || lseek(fd, (off_t)0, SEEK_SET)) goto err; PASS(0x00); if (fsync(fd) || lseek(fd, (off_t)0, SEEK_SET)) goto err; PASS(0xff); if (!fsync(fd) && !close(fd)) { free(buf); return (1); } err: eval = 1; if (buf) free(buf); if (fd != -1) close(fd); warn("%s", file); return (0); } static int check(const char *path, const char *name, struct stat *sp) { int ch, first; char modep[15], *flagsp; /* Check -i first. */ if (iflag) (void)fprintf(stderr, "remove %s? ", path); else { /* * If it's not a symbolic link and it's unwritable and we're * talking to a terminal, ask. Symbolic links are excluded * because their permissions are meaningless. Check stdin_ok * first because we may not have stat'ed the file. */ if (!stdin_ok || S_ISLNK(sp->st_mode) || (!access(name, W_OK) && !(sp->st_flags & (SF_APPEND|SF_IMMUTABLE)) && (!(sp->st_flags & (UF_APPEND|UF_IMMUTABLE)) || !uid))) return (1); strmode(sp->st_mode, modep); if ((flagsp = fflagstostr(sp->st_flags)) == NULL) err(1, "fflagstostr"); if (Pflag) errx(1, "%s: -P was specified, but file is not writable", path); (void)fprintf(stderr, "override %s%s%s/%s %s%sfor %s? ", modep + 1, modep[10] == ' ' ? "" : " ", user_from_uid(sp->st_uid, 0), group_from_gid(sp->st_gid, 0), *flagsp ? flagsp : "", *flagsp ? " " : "", path); free(flagsp); } (void)fflush(stderr); first = ch = getchar(); while (ch != '\n' && ch != EOF) ch = getchar(); return (first == 'y' || first == 'Y'); } #define ISSLASH(a) ((a)[0] == '/' && (a)[1] == '\0') static void checkslash(char **argv) { char **t, **u; int complained; complained = 0; for (t = argv; *t;) { if (ISSLASH(*t)) { if (!complained++) warnx("\"/\" may not be removed"); eval = 1; for (u = t; u[0] != NULL; ++u) u[0] = u[1]; } else { ++t; } } } static int check2(char **argv) { struct stat st; int first; int ch; int fcount = 0; int dcount = 0; int i; const char *dname = NULL; for (i = 0; argv[i]; ++i) { if (lstat(argv[i], &st) == 0) { if (S_ISDIR(st.st_mode)) { ++dcount; dname = argv[i]; /* only used if 1 dir */ } else { ++fcount; } } } first = 0; while (first != 'n' && first != 'N' && first != 'y' && first != 'Y') { if (dcount && rflag) { fprintf(stderr, "recursively remove"); if (dcount == 1) fprintf(stderr, " %s", dname); else fprintf(stderr, " %d dirs", dcount); if (fcount == 1) fprintf(stderr, " and 1 file"); else if (fcount > 1) fprintf(stderr, " and %d files", fcount); } else if (dcount + fcount > 3) { fprintf(stderr, "remove %d files", dcount + fcount); } else { return(1); } fprintf(stderr, "? "); fflush(stderr); first = ch = getchar(); while (ch != '\n' && ch != EOF) ch = getchar(); if (ch == EOF) break; } return (first == 'y' || first == 'Y'); } #define ISDOT(a) ((a)[0] == '.' && (!(a)[1] || ((a)[1] == '.' && !(a)[2]))) static void checkdot(char **argv) { char *p, **save, **t; int complained; complained = 0; for (t = argv; *t;) { if ((p = strrchr(*t, '/')) != NULL) ++p; else p = *t; if (ISDOT(p)) { if (!complained++) warnx("\".\" and \"..\" may not be removed"); eval = 1; for (save = t; (t[0] = t[1]) != NULL; ++t) continue; t = save; } else ++t; } } static void usage(void) { (void)fprintf(stderr, "%s\n%s\n", "usage: rm [-f | -i] [-dIPRrvWx] file ...", - " unlink file"); + " unlink [--] file"); exit(EX_USAGE); } static void siginfo(int sig __unused) { info = 1; } Index: projects/clang700-import/bin/rm/tests/Makefile =================================================================== --- projects/clang700-import/bin/rm/tests/Makefile (nonexistent) +++ projects/clang700-import/bin/rm/tests/Makefile (revision 338690) @@ -0,0 +1,5 @@ +# $FreeBSD$ + +ATF_TESTS_SH+= rm_test + +.include Property changes on: projects/clang700-import/bin/rm/tests/Makefile ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/clang700-import/bin/rm/tests/Makefile.depend =================================================================== --- projects/clang700-import/bin/rm/tests/Makefile.depend (nonexistent) +++ projects/clang700-import/bin/rm/tests/Makefile.depend (revision 338690) @@ -0,0 +1,11 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif Property changes on: projects/clang700-import/bin/rm/tests/Makefile.depend ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/clang700-import/bin/rm/tests/rm_test.sh =================================================================== --- projects/clang700-import/bin/rm/tests/rm_test.sh (nonexistent) +++ projects/clang700-import/bin/rm/tests/rm_test.sh (revision 338690) @@ -0,0 +1,45 @@ +# +# Copyright 2018 Yuri Pankov +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ +# + +atf_test_case unlink_dash_filename +unlink_dash_filename_head() +{ + atf_set "descr" "unlink correctly handles -filename" +} +unlink_dash_filename_body() +{ + touch -- foo bar -foo -bar + atf_check -s exit:0 unlink foo + atf_check -s exit:0 unlink -- bar + atf_check -s exit:0 unlink -foo + atf_check -s exit:0 unlink -- -bar +} + +atf_init_test_cases() +{ + atf_add_test_case unlink_dash_filename +} Property changes on: projects/clang700-import/bin/rm/tests/rm_test.sh ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:executable ## -0,0 +1 ## +* \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: projects/clang700-import/contrib/libarchive/NEWS =================================================================== --- projects/clang700-import/contrib/libarchive/NEWS (revision 338689) +++ projects/clang700-import/contrib/libarchive/NEWS (revision 338690) @@ -1,687 +1,697 @@ +Sep 03, 2018: libarchive 3.3.3 released + +Jul 19, 2018: Avoid super-linear slowdown on malformed mtree files + +Jan 27, 2018: Many fixes for building with Visual Studio + +Oct 19, 2017: NO_OVERWRITE doesn't change existing directory attributes + +Aug 12, 2017: New support for Zstandard read and write filters + Jul 09, 2017: libarchive 3.3.2 released Mar 16, 2017: NFSv4 ACL support for Linux (librichacl) Feb 26, 2017: libarchive 3.3.1 released Security & Feature release Feb 19, 2017: libarchive 3.3.0 released Security & Feature release Jan 29, 2017: Limited NFSv4 ACL support for Mac OS (Darwin) Jan 10, 2017: POSIX.1e and NFSv4 ACL support for Solaris and derivates Dec 27, 2016: NFSv4 ACL read and write support for pax Deprecated functions: archive_entry_acl_text(), archive_entry_acl_text_w() Nov, 2016: libarchive is now being tested by the OSS-Fuzz project Oct 26, 2016: Remove liblzmadec support Oct 23, 2016: libarchive 3.2.2 released Security release Jun 20, 2016: libarchive 3.2.1 released This fixes a handful of security and other critical issues with 3.2.0 May 01, 2016: libarchive 3.2.0 released Apr 09, 2016: libarchive 3.1.901a released Another test release in preparation for 3.2.0 Feb 13, 2016: libarchive 3.1.900a released This is a test release in preparation for 3.2.0 Oct 21, 2015: Preliminary port to OSF Apr 11, 2015: libarchive's issue tracker is now hosted at GitHub. https://github.com/libarchive/libarchive/issues Early 2015: Many fixes to crash and overflow bugs thanks to Hanno Boeck Oct 13, 2014: Zip encryption and decryption support Aug 13, 2014: Add support for lz4 compression. Jun 10, 2014: Add warc format support May 3, 2014: Add experimental Zip streaming extension Apr 6, 2014: Add bsdcat command-line tool Jan 12, 2014: Add Zip64 support Dec 1, 2013: Rewrite Zip write logic Jul 1, 2013: Add ability to detect encrypted entries for many formats (This does not add the ability to *decrypt* those entries, however) Feb 23, 2013: "raw" write support added Feb 09, 2013: libarchive 3.1.2 released Jan 28, 2013: libarchive's new website moved to http://www.libarchive.org. Jan 13, 2013: libarchive 3.1.1 released Jan 13, 2013: libarchive 3.1.0 released Dec 07, 2012: Implement functions to manually set the format and filters used. Nov 11, 2012: Add support for __MACOSX directory in Zip archives, which resource forks are stored in. Oct 20, 2012: Add support for writing v7 tar format. Oct 09, 2012: Add support for grzip compression. Oct 07, 2012: Introduce b64encode filter. Oct 07, 2012: Introduce uuencode filter. Oct 06, 2012: Add support for lzop. Sep 27, 2012: Implement function used to seek within data blocks. (Currently only supported for uncompressed RAR archives). Apr 22, 2012: Add basic archive read and write filter support for lrzip. Mar 27, 2012: libarchive 3.0.4 released Feb 05, 2012: libarchive development now hosted at GitHub. http://libarchive.github.com/ Feb 05, 2012: libarchive's issue tracker remains at Google Code. http://code.google.com/p/libarchive/issues/list Feb 05, 2012: libarchive's mailing lists remain at Google Groups. Dec 24, 2011: libarchive 3.0.2 released Dec 23, 2011: Various fixes merged from FreeBSD Dec 23, 2011: Symlink support in Zip reader and writer Dec 23, 2011: Robustness fixes to 7Zip reader Nov 27, 2011: libarchive 3.0.1b released Nov 26, 2011: 7Zip reader Nov 26, 2011: Small fixes to ISO and Zip to improve robustness with corrupted input Nov 24, 2011: Improve streaming Zip reader's support for uncompressed entries Nov 20, 2011: New seeking Zip reader supports SFX Zip archives Nov 20, 2011: Build fixes on Windows Nov 13, 2011: libarchive 3.0.0a released Nov 06, 2011: Update shared-library version calculations for libarchive 3.x Sep 04, 2011: Fix tar -s; follow GNU tar for controlling hardlink/symlink substitutions Aug 18, 2011: Fix reading ISO images built by NetBSD's mkisofs Aug 15, 2011: Old archive_read_support_compression_XXX functions are deprecated and will disappear in libarchive 4.0. Jun 26, 2011: RAR reader Jun 16, 2011: Add tar:compat-2x option to emulate broken libarchive 2.x handling of pax UTF-8 headers Apr 25, 2011: Refactor read_open() into a collection of single-item setters; support the old interfaces as wrappers Apr 12, 2011: Split disk writer into separate POSIX and Windows implementations Apr 10, 2011: Improvements to character translations on Windows. Mar 30, 2011: More work to return errors instead of calling abort() Mar 23, 2011: Add charset option to many writers to control MBCS filenames Mar 17, 2011: Overhauled support for per-format extension options Mar 17, 2011: Track character set used for mbcs strings, support translating to/from user-specified locale Mar 09, 2011: Recognize mtree files without requiring a signature Mar 06, 2011: Use iconv to convert to/from Unicode instead of making bad assumptions about the C90 character set translation functions Feb 17, 2011: Fixes for AIX, TRU64, and other platforms Dec 22, 2010: CAB reader Dec 20, 2010: LHA/LZH reader Jul 03, 2010: minitar example demonstrates archive_read_disk directory traversal Jun 29, 2010: Many improvements to ISO reader compatibility Jun 26, 2010: Use larger buffers when copy files into an archive Jun 18, 2010: Reimplement Mac OS extensions in libarchive Jun 09, 2010: archive_read_disk now supports traversals May 28, 2010: XAR writer May 16, 2010: Fix ^T handling; don't exit on interrupted reads and writes May 09, 2010: Improved detection of platform-specific crypto support May 04, 2010: lzip read and write filters May 01, 2010: New options: tar --gid --gname --uid --uname Apr 28, 2010: Use Red-black tree for ISO reader/writer to improve performance Apr 17, 2010: Minimal writer for legacy GNU tar format Mar 12, 2010: Don't dereference symlinks on Linux when reading ACLs. Mar 06, 2010: Fix build when an older libarchive is already installed Feb 28, 2010: Relax handling of state failures; misuse by clients now generally results in a sticky ARCHIVE_FATAL rather than a visit to abort() Feb 25, 2010: ISO writer Feb 21, 2010: Split many man pages into smaller chunks. Feb 21, 2010: Performance: Cheat on block sizes when reading archives from disk. Feb 21, 2010: Use int64_t instead of off_t, dev_t, ino_t, uid_t, and gid_t Feb 20, 2010: Document new ACL functions. Feb 19, 2010: Support multiple write filters Feb 07, 2010: Remove some legacy libarchive 1.x APIs Feb 04, 2010: Read afio headers Feb 02, 2010: Archive sparse files compatibly with GNU tar Feb 01, 2010: Integrate Apple extensions for Mac OS extended attributes into bsdtar Jan 31, 2010: Support cpio -V Feb 04, 2010: libarchive 2.8.0 released Jan 17, 2010: Fix error handling for 'echo nonexistent | cpio -o' Jan 17, 2010: Don't use futimes() on Cygwin Jan 02, 2010: libarchive 2.7.902a released (test release for 2.8) Jan 02, 2010: Fix tar/test/test_windows on MinGW Jan 02, 2010: Fix memory leaks in libarchive tests Jan 01, 2010: Fix memory leak when filter startup fails Dec 27, 2009: libarchive 2.7.901a released (test release for 2.8) Aug 04, 2009: libarchive 2.7.1 released Jul 20, 2009: Suppress bogus warning about unxz Jul 19, 2009: Support Cygwin 1.7 Jun 11, 2009: Support lzma/xz files compressed with larger buffer sizes. May 24, 2009: Handle gzip files signed with OpenBSD "gzsig" program. May 07, 2009: Avoid false failures when reading from pipe. Apr 16, 2009: libarchive 2.7.0 released Apr 10, 2009: libarchive 2.6.992a released Apr 09, 2009: Fix SIGPIPE issue building with MSVC. Apr 09, 2009: Fix several minor memory leaks in libarchive and libarchive_test Apr 08, 2009: libarchive 2.6.991a released Apr 07, 2009: Additional tests added to bsdcpio_test Apr 01, 2009: libarchive 2.6.990a released Apr 01, 2009: Use command-line gunzip, bunzip2, unxz, unlzma for decompression if the library is built without suitable libraries. The setup functions return ARCHIVE_WARN in this case so clients can adapt if necessary. Apr 01, 2009: Use getpw*_r and getgr*_r functions for thread-safety. Mar 24, 2009: Add archive_read_next_header2(), which is up to 25% more efficient for some clients; from Brian Harring. Mar 22, 2009: PDF versions of manpages are now included in the distribution. Mar, 2009: Major work to improve Cygwin build by Charles Wilson. Feb/Mar, 2009: Major work on cmake build support, mostly by Michihiro NAKAJIMA. Feb/Mar, 2009: Major work on Visual Studio support by Michihiro NAKAJIMA. All tests now pass. Feb 25, 2009: Fix Debian Bug #516577 Feb 21, 2009: Yacc is no longer needed to build; date parser rewritten in C. Jan/Feb, 2009: Mtree work by Michihiro. Feb, 2009: Joliet support by Andreas Henriksson. Jan/Feb, 2009: New options framework by Michihiro. Feb, 2009: High-res timestamps on Tru64, AIX, and GNU Hurd, by Björn Jacke. Jan 18, 2009: Extended attributes work on FreeBSD and Linux now with pax format. Jan 07, 2009: New archive_read_disk_entry_from_file() knows about ACLs, extended attributes, etc so that bsdtar and bsdcpio don't require such system-specific knowledge. Jan 03, 2009: Read filter system extensively refactored. In particular, read filter pipelines are now built out automatically and individual filters should be much easier to implement. Documentation on the Googlecode Wiki explains how to implement new filters. Dec 28, 2008: Many Windows/Visual Studio fixes from Michihiro NAKAJIMA. Dec 28, 2008: Main libarchive development moved from FreeBSD Perforce server to Google Code. This should make it easier for more people to participate in libarchive development. Dec 28, 2008: libarchive 2.6.0 released Dec 25, 2008: libarchive 2.5.905a released Dec 10, 2008: libarchive 2.5.904a released Dec 04, 2008: libarchive 2.5.903a released Nov 09, 2008: libarchive 2.5.902a released Nov 08, 2008: libarchive 2.5.901a released Nov 08, 2008: Start of pre-release testing for libarchive 2.6 Nov 07, 2008: Read filter refactor: The decompression routines just consume and produce arbitrarily-sized blocks. The reblocking from read_support_compression_none() has been pulled into the read core. Also, the decompression bid now makes multiple passes and stacks read filters. Oct 21, 2008: bsdcpio: New command-line parser. Oct 19, 2008: Internal read_ahead change: short reads are now an error Oct 06, 2008: bsdtar: option parser no longer uses getopt_long(), gives consistent option parsing on all platforms. Sep 19, 2008: Jaakko Heinonen: shar utility built on libarchive Sep 17, 2008: Pedro Giffuni: birthtime support Sep 17, 2008: Miklos Vajna: lzma reader and test. Note: I still have some concerns about the auto-detection (LZMA file format doesn't support auto-detection well), so this is not yet enabled under archive_read_support_compression_all(). For now, you must call archive_read_support_compression_lzma() if you want LZMA read support. Sep 11, 2008: Ivailo Petrov: Many fixes to Windows build, new solution files Jul 26, 2008: archive_entry now tracks which values have not been set. This helps zip extraction (file size is often "unknown") and time restores (tar usually doesn't know atime). Jul 26, 2008: Joerg Sonnenberger: Performance improvements to shar writer Jul 25, 2008: Joerg Sonnenberger: mtree write support Jul 02, 2008: libarchive 2.5.5 released Jul 02, 2008: libarchive 2.5.5b released Jul 01, 2008: bsdcpio is being used by enough people, we can call it 1.0.0 now Jun 20, 2008: bsdcpio: If a -l link fails with EXDEV, copy the file instead Jun 19, 2008: bsdcpio: additional long options for better GNU cpio compat Jun 15, 2008: Many small portability and bugfixes since 2.5.4b. May 25, 2008: libarchive 2.5.4b released May 21, 2008: Joerg Sonnenberger: fix bsdtar hardlink handling for newc format May 21, 2008: More progress on Windows building. Thanks to "Scott" for the Windows makefiles, thanks to Kees Zeelenberg for code contributions. May 21, 2008: Fix a number of non-exploitable integer and buffer overflows, thanks to David Remahl at Apple for pointing these out. May 21, 2008: Colin Percival: SIGINFO or SIGUSR1 to bsdtar prints progress info May 16, 2008: bsdtar's test harness no longer depends on file ordering. This was causing spurious test failures on a lot of systems. Thanks to Bernhard R. Link for the diagnosis. May 14, 2008: Joerg Sonnenberger: -s substitution support for bsdtar May 13, 2008: Joerg Sonnenberger: Many mtree improvements May 11, 2008: Joerg Sonnenberger: fix hardlink extraction when hardlinks have different permissions from original file April 30, 2008: Primary libarchive work has been moved into the FreeBSD project's Perforce repository: http://perforce.freebsd.org/ The libarchive project can be browsed at //depot/user/kientzle/libarchive-portable Direct link: http://preview.tinyurl.com/46mdgr May 04, 2008: libarchive 2.5.3b released * libarchive: Several fixes to link resolver to address bsdcpio crashes * bsdcpio: -p hardlink handling fixes * tar/pax: Ensure ustar dirnames end in '/'; be more careful about measuring filenames when deciding what pathname fields to use * libarchive: Mark which entry strings are set; be accurate about distinguishing empty strings ("") from unset ones (NULL) * tar: Don't crash reading entries with empty filenames * libarchive_test, bsdtar_test, bsdcpio_test: Better defaults: run all tests, delete temp dirs, summarize repeated failures * -no-undefined to libtool for Cygwin * libarchive_test: Skip large file tests on systems with 32-bit off_t * iso9660: Don't bother trying to find the body of an empty file; this works around strange behavior from some ISO9660 writers * tar: allow -r -T to be used together * tar: allow --format with -r or -u * libarchive: Don't build archive.h May 04, 2008: Simplified building: archive.h is no longer constructed This may require additional #if conditionals on some platforms. Mar 30, 2008: libarchive 2.5.1b released Mar 15, 2008: libarchive 2.5.0b released Mar 15, 2008: bsdcpio now seems to correctly write hardlinks into newc, ustar, and old cpio archives. Just a little more testing before bsdcpio 1.0 becomes a reality. Mar 15, 2008: I think the new linkify() interface is finally handling all known hardlink strategies. Mar 15, 2008: Mtree read fixes from Joerg Sonnenberger. Mar 15, 2008: Many new bsdtar and bsdcpio options from Joerg Sonnenberger. Mar 15, 2008: test harnesses no longer require uudecode; they now have built-in decoding logic that decodes the reference files as they are needed. Mar 14, 2008: libarchive 2.4.14 released; identical to 2.4.13 except for a point fix for gname/uname mixup in pax format that was introduced with the UTF-8 fixes. Feb 26, 2008: libarchive 2.4.13 released Feb 25, 2008: Handle path, linkname, gname, or uname that can't be converted to/from UTF-8. Implement "hdrcharset" attribute from SUS-2008. Feb 25, 2008: Fix name clash on NetBSD. Feb 18, 2008: Fix writing empty 'ar' archives, per Kai Wang Feb 18, 2008: [bsdtar] Permit appending on block devices. Feb 09, 2008: New "linkify" resolver to help with newc hardlink writing; bsdcpio still needs to be converted to use this. Feb 02, 2008: Windows compatibility fixes from Ivailo Petrov, Kees Zeelenberg Jan 30, 2008: Ignore hardlink size for non-POSIX tar archives. Jan 22, 2008: libarchive 2.4.12 released Jan 22, 2008: Fix bad padding when writing symlinks to newc cpio archives. Jan 22, 2008: Verify bsdcpio_test by getting it to work against GNU cpio 2.9. bsdcpio_test complains about missing options (-y and -z), format of informational messages (--version, --help), and a minor formatting issue in odc format output. After this update, bsdcpio_test uncovered several more cosmetic issues in bsdcpio, all now fixed. Jan 22, 2008: Experimental support for self-extracting Zip archives. Jan 22, 2008: Extend hardlink restore strategy to work correctly with hardlinks extracted from newc cpio files. (Which store the body only with the last occurrence of a link.) Dec 30, 2007: libarchive 2.4.11 released Dec 30, 2007: Fixed a compile error in bsdcpio on some systems. Dec 29, 2007: libarchive 2.4.10 released Dec 29, 2007: bsdcpio 0.9.0 is ready for wider use. Dec 29, 2007: Completed initial test harness for bsdcpio. Dec 22, 2007: libarchive 2.4.9 released Dec 22, 2007: Implement the remaining options for bsdcpio: -a, -q, -L, -f, pattern selection for -i and -it. Dec 13, 2007: libarchive 2.4.8 released Dec 13, 2007: gzip and bzip2 compression now handle zero-byte writes correctly, Thanks to Damien Golding for bringing this to my attention. Dec 12, 2007: libarchive 2.4.7 released Dec 10, 2007: libarchive 2.4.6 released Dec 09, 2007: tar/test/test_copy.c verifies "tar -c | tar -x" copy pipeline Dec 07, 2007: Fix a couple of minor memory leaks. Dec 04, 2007: libarchive 2.4.5 released Dec 04, 2007: Fix cpio/test/test_write_odc by setting the umask first. Dec 03, 2007: libarchive 2.4.4 released Dec 03, 2007: New configure options --disable-xattr and --disable-acl, thanks to Samuli Suominen. Dec 03, 2007: libarchive 2.4.3 released Dec 03, 2007: Thanks to Lapo Luchini for sending me a ZIP file that libarchive couldn't handle. Fixed a bug in handling of "length at end" flags in ZIP files. Dec 03, 2007: Fixed bsdcpio -help, bsdtar -help tests. Dec 02, 2007: First cut at real bsdtar test harness. Dec 02, 2007: libarchive 2.4.2 released Dec 02, 2007: libarchive 2.4.1 released Dec 02, 2007: Minor fixes, rough cut of mdoc-to-man conversion for man pages. Oct 30, 2007: libarchive 2.4.0 released Oct 30, 2007: Minor compile fix thanks to Joerg Schilling. Oct 30, 2007: Only run the format auction once at the beginning of the archive. This is simpler and supports better error recovery. Oct 29, 2007: Test support for very large entries in tar archives: libarchive_test now exercises entries from 2GB up to 1TB. Oct 27, 2007: libarchive 2.3.5 released Oct 27, 2007: Correct some unnecessary internal data copying in the "compression none" reader and writer; this reduces user time by up to 2/3 in some tests. (Thanks to Jan Psota for publishing his performance test results to GNU tar's bug-tar mailing list; those results pointed me towards this problem.) Oct 27, 2007: Fix for skipping archive entries that are exactly a multiple of 4G on 32-bit platforms. Oct 25, 2007: Fix for reading very large (>8G) tar archives; this was broken when I put in support for new GNU tar sparse formats. Oct 20, 2007: Initial work on new pattern-matching code for cpio; I hope this eventually replaces the code currently in bsdtar. Oct 08, 2007: libarchive 2.3.4 released Oct 05, 2007: Continuing work on bsdcpio test suite. Oct 05, 2007: New cpio.5 manpage, updates to "History" of bsdcpio.1 and bsdtar.1 manpages. Oct 05, 2007: Fix zip reader to immediately return EOF if you try to read body of non-regular file. In particular, this fixes bsdtar extraction of zip archives. Sep 30, 2007: libarchive 2.3.3 released Sep 26, 2007: Rework Makefile.am so that the enable/disable options actually do the right things. Sep 26, 2007: cpio-odc and cpio-newc archives no longer write bodies for non-regular files. Sep 26, 2007: Test harness for bsdcpio is in place, needs more tests written. This is much nicer than the ragtag collection of test scripts that bsdtar has. Sep 20, 2007: libarchive 2.3.2 released Sep 20, 2007: libarchive 2.3.1 broke bsdtar because the archive_write_data() fix was implemented incorrectly. Sep 16, 2007: libarchive 2.3.1 released Sep 16, 2007: Many fixes to bsdcpio 0.3: handle hardlinks with -p, recognize block size on writing, fix a couple of segfaults. Sep 16, 2007: Fixed return value from archive_write_data() when used with archive_write_disk() to match the documentation and other instances of this same function. Sep 15, 2007: Add archive_entry_link_resolver, archive_entry_strmode Sep 11, 2007: libarchive 2.2.8 released Sep 09, 2007: bsdcpio 0.2 supports most (not yet all) of the old POSIX spec. Sep 01, 2007: libarchive 2.2.7 released Aug 31, 2007: Support for reading mtree files, including an mtree.5 manpage (A little experimental still.) Aug 18, 2007: Read gtar 1.17 --posix --sparse entries. Aug 13, 2007: Refined suid/sgid restore handling; it is no longer an error if suid/sgid bits are dropped when you request perm restore but don't request owner restore. Aug 06, 2007: Use --enable-bsdcpio if you want to try bsdcpio Aug 05, 2007: libarchive 2.2.6 released Aug 05, 2007: New configure option --disable-bsdtar, thanks to Joerg Sonnenberger. Aug 05, 2007: Several bug fixes from FreeBSD CVS repo. Jul 13, 2007: libarchive 2.2.5 released Jul 12, 2007: libarchive 2.2.4 released Jul 12, 2007: Thanks to Colin Percival's help in diagnosing and fixing several critical security bugs. Details available at http://security.freebsd.org/advisories/FreeBSD-SA-07:05.libarchive.asc May 26, 2007: libarchive 2.2.3 released May 26, 2007: Fix memory leaks in ZIP reader and shar writer, add some missing system headers to archive_entry.h, dead code cleanup from Colin Percival, more tests for gzip/bzip2, fix an EOF anomaly in bzip2 decompression. May 12, 2007: libarchive 2.2.2 released May 12, 2007: Fix archive_write_disk permission restore by cloning entry passed into write_header so that permission info is still available at finish_entry time. (archive_read_extract() worked okay because it held onto the passed-in entry, but direct consumers of archive_write_disk would break). This required fixing archive_entry_clone(), which now works and has a reasonably complete test case. May 10, 2007: Skeletal cpio implementation. May 06, 2007: libarchive 2.2.1 released May 06, 2007: Flesh out a lot more of test_entry.c so as to catch problems such as the device node breakage before releasing . May 05, 2007: Fix a bad bug introduced in 2.1.9 that broke device node entries in tar archives. May 03, 2007: Move 'struct stat' out of archive_entry core as well. This removes some portability headaches and fixes a bunch of corner cases that arise when manipulating archives on dissimilar systems. Apr 30, 2007: libarchive 2.1.10 released Apr 31, 2007: Minor code cleanup. Apr 24, 2007: libarchive 2.1.9 released Apr 24, 2007: Fix some recently-introduced problems with libraries (Just let automake handle it and it all works much better.) Finish isolating major()/minor()/makedev() in archive_entry.c. Apr 23, 2007: libarchive 2.1.8 released Apr 23, 2007: Minor fixes found from building on MacOS X Apr 22, 2007: libarchive 2.1.7 released Apr 22, 2007: Eliminated all uses of 'struct stat' from the format readers/writers. This should improve portability; 'struct stat' is now only used in archive_entry and in code that actually touches the disk. Apr 17, 2007: libarchive 2.1.6 released Libarchive now compiles and passes all tests on Interix. Apr 16, 2007: libarchive 2.1.5 released Apr 15, 2007: libarchive 2.1b2 released Apr 15, 2007: New libarchive_internals.3 documentation of internal APIs. Not complete, but should prove helpful. Apr 15, 2007: Experimental "read_compress_program" and "write_compress_program" for using libarchive with external compression. Not yet well tested, and likely has portability issues. Feedback appreciated. Apr 14, 2007: libarchive 2.0.31 released Apr 14, 2007: More fixes for Interix, more 'ar' work Apr 14, 2007: libarchive 2.0.30 released Apr 13, 2007: libarchive now enforces trailing '/' on dirs written to tar archives Apr 11, 2007: libarchive 2.0.29 released Apr 11, 2007: Make it easier to statically configure for different platforms. Apr 11, 2007: Updated config.guess, config.sub, libtool Apr 06, 2007: libarchive 2.0.28 released Apr 06, 2007: 'ar' format read/write support thanks to Kai Wang. Apr 01, 2007: libarchive 2.0.27 released Mar 31, 2007: Several minor fixes from Colin Percival and Joerg Sonnenberger. Mar 12, 2007: libarchive 2.0.25 released Mar 12, 2007: Fix broken --unlink flag. Mar 11, 2007: libarchive 2.0.24 released Mar 10, 2007: Correct an ACL blunder that causes any ACL with an entry that refers to a non-existent user or group to not be restored correctly. The fix both makes the parser more tolerant (so that archives created with the buggy ACLs can be read now) and corrects the ACL formatter. Mar 10, 2007: More work on test portability to Linux. Mar 10, 2007: libarchive 2.0.22 released Mar 10, 2007: Header cleanups; added linux/fs.h, removed some unnecessary headers, added #include guards in bsdtar. If you see any obvious compile failures from this, let me know. Mar 10, 2007: Work on bsdtar test scripts: not yet robust enough to enable as part of "make check", but getting better. Mar 10, 2007: libarchive now returns ARCHIVE_FAILED when a header write fails in a way that only affects this item. Less bad than ARCHIVE_FATAL, but worse than ARCHIVE_WARN. Mar 07, 2007: libarchive 2.0.21 released Mar 07, 2007: Add some ACL tests (only for the system-independent portion of the ACL support for now). Mar 07, 2007: tar's ability to read ACLs off disk got turned off for FreeBSD; re-enable it. (ACL restores and libarchive support for storing/reading ACLs from pax archives was unaffected.) Mar 02, 2007: libarchive 2.0.20 released Mar 2, 2007: It's not perfect, but it's pretty good. Libarchive 2.0 is officially out of beta. Feb 28, 2007: libarchive 2.0b17 released Feb 27, 2007: Make the GID restore checks more robust by checking whether the current user has too few or too many privileges. Feb 26, 2007: libarchive 2.0b15 released Feb 26, 2007: Don't lose symlinks when extracting from ISOs. Thanks to Diego "Flameeyes" Pettenò for telling me about the broken testcase on Gentoo that (finally!) led me to the cause of this long-standing bug. Feb 26, 2007: libarchive 2.0b14 released Feb 26, 2007: Fix a broken test on platforms that lack lchmod(). Feb 25, 2007: libarchive 2.0b13 released Feb 25, 2007: Empty archives were being written as empty files, without a proper end-of-archive marker. Fixed. Feb 23, 2007: libarchive 2.0b12 released Feb 22, 2007: Basic security checks added: _EXTRACT_SECURE_NODOTDOT and _EXTRACT_SECURE_SYMLINK. These checks used to be in bsdtar, but they belong down in libarchive where they can be used by other tools and where they can be better optimized. Feb 11, 2007: libarchive 2.0b11 released Feb 10, 2007: Fixed a bunch of errors in libarchive's handling of EXTRACT_PERM and EXTRACT_OWNER, especially relating to SUID and SGID bits. Jan 31, 2007: libarchive 2.0b9 released Jan 31, 2007: Added read support for "empty" archives as a distinct archive format. Bsdtar uses this to handle, e.g., "touch foo.tar; tar -rf foo.tar" Jan 22, 2007: libarchive 2.0b6 released Jan 22, 2007: archive_write_disk API is now in place. It provides a finer-grained interface than archive_read_extract. In particular, you can use it to create objects on disk without having an archive around (just feed it archive_entry objects describing what you want to create), you can override the uname/gname-to-uid/gid lookups (minitar uses this to avoid getpwXXX() and getgrXXX() bloat). Jan 09, 2007: libarchive 2.0a3 released Jan 9, 2007: archive_extract is now much better; it handles the most common cases with a minimal number of system calls. Some features still need a lot of testing, especially corner cases involving objects that already exist on disk. I expect the next round of API overhaul will simplify building test cases. Jan 9, 2007: a number of fixes thanks to Colin Percival, especially corrections to the skip() framework and handling of large files. Jan 9, 2007: Fixes for large ISOs. The code should correctly handle very large ISOs with entries up to 4G. Thanks to Robert Sciuk for pointing out these issues. Sep 05, 2006: libarchive 1.3.1 released Sep 5, 2006: Bump version to 1.3 for new I/O wrappers. Sep 4, 2006: New memory and FILE read/write wrappers. Sep 4, 2006: libarchive test harness is now minimally functional; it's located a few minor bugs in error-handling logic Aug 17, 2006: libarchive 1.2.54 released Aug 17, 2006: Outline ABI changes for libarchive 2.0; these are protected behind #ifdef's until I think I've found everything that needs to change. Aug 17, 2006: Fix error-handling in archive_read/write_close() They weren't returning any errors before. Aug 17, 2006: Fix recursive-add logic to not trigger if it's not set Fixes a bug adding files when writing archive to pipe or when using archive_write_open() directly. Jul 2006: New "skip" handling improves performance extracting single files from large uncompressed archives. Mar 21, 2006: 1.2.52 released Mar 21, 2006: Fix -p on platforms that don't have platform-specific extended attribute code. Mar 20, 2006: Add NEWS file; fill in some older history from other files. I'll try to keep this file up-to-date from now on. OLDER NEWS SUMMARIES Mar 19, 2006: libarchive 1.2.51 released Mar 18, 2006: Many fixes to extended attribute support, including a redesign of the storage format to simplify debugging. Mar 12, 2006: Remove 'tp' support; it was a fun idea, but not worth spending much time on. Mar 11, 2006: Incorporated Jaakko Heinonen's still-experimental support for extended attributes (Currently Linux-only.). Mar 11, 2006: Reorganized distribution package: There is now one tar.gz file that builds both libarchive and bsdtar. Feb 13, 2006: Minor bug fixes: correctly read cpio device entries, write Pax attribute entry names. Nov 7, 2005: Experimental 'tp' format support in libarchive. Feedback appreciated; this is not enabled by archive_read_support_format_all() yet as I'm not quite content with the format detection heuristics. Nov 7, 2005: Some more portability improvements thanks to Darin Broady, minor bugfixes. Oct 12, 2005: Use GNU libtool to build shared libraries on many systems. Aug 9, 2005: Correctly detect that MacOS X does not have POSIX ACLs. Apr 17, 2005: Kees Zeelenberg has ported libarchive and bsdtar to Windows: http://gnuwin32.sourceforge.net/ Apr 11, 2005: Extended Zip/Zip64 support thanks to Dan Nelson. -L/-h fix from Jaakko Heinonen. Mar 12, 2005: archive_read_extract can now handle very long pathnames (I've tested with pathnames up to 1MB). Mar 12, 2005: Marcus Geiger has written an article about libarchive http://xsnil.antbear.org/2005/02/05/archive-mit-libarchive-verarbeiten/ including examples of using it from Objective-C. His MoinX http://moinx.antbear.org/ desktop Wiki uses libarchive for archiving and restoring Wiki pages. Jan 22, 2005: Preliminary ZIP extraction support, new directory-walking code for bsdtar. Jan 16, 2005: ISO9660 extraction code added; manpage corrections. May 22, 2004: Many gtar-compatible long options have been added; almost all FreeBSD ports extract correctly with bsdtar. May 18, 2004: bsdtar can read Solaris, HP-UX, Unixware, star, gtar, and pdtar archives. Index: projects/clang700-import/contrib/libarchive/README.md =================================================================== --- projects/clang700-import/contrib/libarchive/README.md (revision 338689) +++ projects/clang700-import/contrib/libarchive/README.md (revision 338690) @@ -1,222 +1,224 @@ # Welcome to libarchive! The libarchive project develops a portable, efficient C library that can read and write streaming archives in a variety of formats. It also includes implementations of the common `tar`, `cpio`, and `zcat` command-line tools that use the libarchive library. ## Questions? Issues? * http://www.libarchive.org is the home for ongoing libarchive development, including documentation, and links to the libarchive mailing lists. * To report an issue, use the issue tracker at https://github.com/libarchive/libarchive/issues * To submit an enhancement to libarchive, please submit a pull request via GitHub: https://github.com/libarchive/libarchive/pulls ## Contents of the Distribution This distribution bundle includes the following major components: * **libarchive**: a library for reading and writing streaming archives * **tar**: the 'bsdtar' program is a full-featured 'tar' implementation built on libarchive * **cpio**: the 'bsdcpio' program is a different interface to essentially the same functionality * **cat**: the 'bsdcat' program is a simple replacement tool for zcat, bzcat, xzcat, and such * **examples**: Some small example programs that you may find useful. * **examples/minitar**: a compact sample demonstrating use of libarchive. * **contrib**: Various items sent to me by third parties; please contact the authors with any questions. The top-level directory contains the following information files: * **NEWS** - highlights of recent changes * **COPYING** - what you can do with this * **INSTALL** - installation instructions * **README** - this file * **CMakeLists.txt** - input for "cmake" build tool, see INSTALL * **configure** - configuration script, see INSTALL for details. If your copy of the source lacks a `configure` script, you can try to construct it by running the script in `build/autogen.sh` (or use `cmake`). The following files in the top-level directory are used by the 'configure' script: * `Makefile.am`, `aclocal.m4`, `configure.ac` - used to build this distribution, only needed by maintainers * `Makefile.in`, `config.h.in` - templates used by configure script ## Documentation In addition to the informational articles and documentation in the online [libarchive Wiki](https://github.com/libarchive/libarchive/wiki), the distribution also includes a number of manual pages: * bsdtar.1 explains the use of the bsdtar program * bsdcpio.1 explains the use of the bsdcpio program * bsdcat.1 explains the use of the bsdcat program * libarchive.3 gives an overview of the library as a whole * archive_read.3, archive_write.3, archive_write_disk.3, and archive_read_disk.3 provide detailed calling sequences for the read and write APIs * archive_entry.3 details the "struct archive_entry" utility class * archive_internals.3 provides some insight into libarchive's internal structure and operation. * libarchive-formats.5 documents the file formats supported by the library * cpio.5, mtree.5, and tar.5 provide detailed information about these popular archive formats, including hard-to-find details about modern cpio and tar variants. The manual pages above are provided in the 'doc' directory in a number of different formats. You should also read the copious comments in `archive.h` and the source code for the sample programs for more details. Please let us know about any errors or omissions you find. ## Supported Formats Currently, the library automatically detects and reads the following fomats: * Old V7 tar archives * POSIX ustar * GNU tar format (including GNU long filenames, long link names, and sparse files) * Solaris 9 extended tar format (including ACLs) * POSIX pax interchange format * POSIX octet-oriented cpio * SVR4 ASCII cpio * POSIX octet-oriented cpio * Binary cpio (big-endian or little-endian) * ISO9660 CD-ROM images (with optional Rockridge or Joliet extensions) * ZIP archives (with uncompressed or "deflate" compressed entries, including support for encrypted Zip archives) * GNU and BSD 'ar' archives * 'mtree' format * 7-Zip archives * Microsoft CAB format * LHA and LZH archives * RAR archives (with some limitations due to RAR's proprietary status) * XAR archives The library also detects and handles any of the following before evaluating the archive: * uuencoded files * files with RPM wrapper * gzip compression * bzip2 compression * compress/LZW compression * lzma, lzip, and xz compression * lz4 compression * lzop compression + * zstandard compression The library can create archives in any of the following formats: * POSIX ustar * POSIX pax interchange format * "restricted" pax format, which will create ustar archives except for entries that require pax extensions (for long filenames, ACLs, etc). * Old GNU tar format * Old V7 tar format * POSIX octet-oriented cpio * SVR4 "newc" cpio * shar archives * ZIP archives (with uncompressed or "deflate" compressed entries) * GNU and BSD 'ar' archives * 'mtree' format * ISO9660 format * 7-Zip archives * XAR archives When creating archives, the result can be filtered with any of the following: * uuencode * gzip compression * bzip2 compression * compress/LZW compression * lzma, lzip, and xz compression * lz4 compression * lzop compression + * zstandard compression ## Notes about the Library Design The following notes address many of the most common questions we are asked about libarchive: * This is a heavily stream-oriented system. That means that it is optimized to read or write the archive in a single pass from beginning to end. For example, this allows libarchive to process archives too large to store on disk by processing them on-the-fly as they are read from or written to a network or tape drive. This also makes libarchive useful for tools that need to produce archives on-the-fly (such as webservers that provide archived contents of a users account). * In-place modification and random access to the contents of an archive are not directly supported. For some formats, this is not an issue: For example, tar.gz archives are not designed for random access. In some other cases, libarchive can re-open an archive and scan it from the beginning quickly enough to provide the needed abilities even without true random access. Of course, some applications do require true random access; those applications should consider alternatives to libarchive. * The library is designed to be extended with new compression and archive formats. The only requirement is that the format be readable or writable as a stream and that each archive entry be independent. There are articles on the libarchive Wiki explaining how to extend libarchive. * On read, compression and format are always detected automatically. -* The same API is used for all formats; in particular, it's very +* The same API is used for all formats; it should be very easy for software using libarchive to transparently handle any of libarchive's archiving formats. * Libarchive's automatic support for decompression can be used without archiving by explicitly selecting the "raw" and "empty" formats. * I've attempted to minimize static link pollution. If you don't explicitly invoke a particular feature (such as support for a particular compression or format), it won't get pulled in to statically-linked programs. In particular, if you don't explicitly enable a particular compression or decompression support, you won't need to link against the corresponding compression or decompression libraries. This also reduces the size of statically-linked binaries in environments where that matters. * The library is generally _thread safe_ depending on the platform: it does not define any global variables of its own. However, some platforms do not provide fully thread-safe versions of key C library functions. On those platforms, libarchive will use the non-thread-safe functions. Patches to improve this are of great interest to us. * In particular, libarchive's modules to read or write a directory tree do use `chdir()` to optimize the directory traversals. This can cause problems for programs that expect to do disk access from multiple threads. Of course, those modules are completely optional and you can use the rest of libarchive without them. * The library is _not_ thread aware, however. It does no locking or thread management of any kind. If you create a libarchive object and need to access it from multiple threads, you will need to provide your own locking. * On read, the library accepts whatever blocks you hand it. Your read callback is free to pass the library a byte at a time or mmap the entire archive and give it to the library at once. On write, the library always produces correctly-blocked output. * The object-style approach allows you to have multiple archive streams open at once. bsdtar uses this in its "@archive" extension. * The archive itself is read/written using callback functions. You can read an archive directly from an in-memory buffer or write it to a socket, if you wish. There are some utility functions to provide easy-to-use "open file," etc, capabilities. * The read/write APIs are designed to allow individual entries to be read or written to any data source: You can create a block of data in memory and add it to a tar archive without first writing a temporary file. You can also read an entry from an archive and write the data directly to a socket. If you want to read/write entries to disk, there are convenience functions to make this especially easy. * Note: The "pax interchange format" is a POSIX standard extended tar format that should be used when the older _ustar_ format is not appropriate. It has many advantages over other tar formats (including the legacy GNU tar format) and is widely supported by current tar implementations. Index: projects/clang700-import/contrib/libarchive/cpio/test/test_basic.c =================================================================== --- projects/clang700-import/contrib/libarchive/cpio/test/test_basic.c (revision 338689) +++ projects/clang700-import/contrib/libarchive/cpio/test/test_basic.c (revision 338690) @@ -1,208 +1,238 @@ /*- * Copyright (c) 2003-2007 Tim Kientzle * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "test.h" __FBSDID("$FreeBSD$"); static void verify_files(const char *msg) { /* * Verify unpacked files. */ /* Regular file with 2 links. */ failure(msg); assertIsReg("file", 0644); failure(msg); assertFileSize("file", 10); failure(msg); assertFileNLinks("file", 2); /* Another name for the same file. */ failure(msg); assertIsHardlink("linkfile", "file"); /* Symlink */ if (canSymlink()) assertIsSymlink("symlink", "file"); /* Another file with 1 link and different permissions. */ failure(msg); assertIsReg("file2", 0777); failure(msg); assertFileSize("file2", 10); failure(msg); assertFileNLinks("file2", 1); /* dir */ assertIsDir("dir", 0775); } static void basic_cpio(const char *target, const char *pack_options, const char *unpack_options, const char *se, const char *se2) { int r; if (!assertMakeDir(target, 0775)) return; /* Use the cpio program to create an archive. */ r = systemf("%s -R 1000:1000 -o %s < filelist >%s/archive 2>%s/pack.err", testprog, pack_options, target, target); failure("Error invoking %s -o %s", testprog, pack_options); assertEqualInt(r, 0); assertChdir(target); /* Verify stderr. */ failure("Expected: %s, options=%s", se, pack_options); assertTextFileContents(se, "pack.err"); /* * Use cpio to unpack the archive into another directory. */ r = systemf("%s -i %s< archive >unpack.out 2>unpack.err", testprog, unpack_options); failure("Error invoking %s -i %s", testprog, unpack_options); assertEqualInt(r, 0); /* Verify stderr. */ failure("Error invoking %s -i %s in dir %s", testprog, unpack_options, target); assertTextFileContents(se2, "unpack.err"); verify_files(pack_options); assertChdir(".."); } static void passthrough(const char *target) { int r; if (!assertMakeDir(target, 0775)) return; /* * Use cpio passthrough mode to copy files to another directory. */ r = systemf("%s -p %s %s/stdout 2>%s/stderr", testprog, target, target, target); failure("Error invoking %s -p", testprog); assertEqualInt(r, 0); assertChdir(target); /* Verify stderr. */ failure("Error invoking %s -p in dir %s", testprog, target); assertTextFileContents("1 block\n", "stderr"); verify_files("passthrough"); assertChdir(".."); } DEFINE_TEST(test_basic) { FILE *filelist; const char *msg; char result[1024]; assertUmask(0); /* * Create an assortment of files on disk. */ filelist = fopen("filelist", "w"); memset(result, 0, sizeof(result)); /* File with 10 bytes content. */ assertMakeFile("file", 0644, "1234567890"); fprintf(filelist, "file\n"); - if (is_LargeInode("file")) + if (is_LargeInode("file")) { strncat(result, - "bsdcpio: file: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: file: large inode number truncated: ", sizeof(result) - strlen(result) -1); + strncat(result, + strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, + "\n", + sizeof(result) - strlen(result) -1); + } /* hardlink to above file. */ assertMakeHardlink("linkfile", "file"); fprintf(filelist, "linkfile\n"); - if (is_LargeInode("linkfile")) + if (is_LargeInode("linkfile")) { strncat(result, - "bsdcpio: linkfile: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: linkfile: large inode number truncated: ", sizeof(result) - strlen(result) -1); + strncat(result, + strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, + "\n", + sizeof(result) - strlen(result) -1); + } /* Symlink to above file. */ if (canSymlink()) { assertMakeSymlink("symlink", "file"); fprintf(filelist, "symlink\n"); - if (is_LargeInode("symlink")) + if (is_LargeInode("symlink")) { strncat(result, - "bsdcpio: symlink: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: symlink: large inode number truncated: ", sizeof(result) - strlen(result) -1); + strncat(result, + strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, + "\n", + sizeof(result) - strlen(result) -1); + } } /* Another file with different permissions. */ assertMakeFile("file2", 0777, "1234567890"); fprintf(filelist, "file2\n"); - if (is_LargeInode("file2")) + if (is_LargeInode("file2")) { strncat(result, - "bsdcpio: file2: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: file2: large inode number truncated: ", sizeof(result) - strlen(result) -1); + strncat(result, + strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, + "\n", + sizeof(result) - strlen(result) -1); + } /* Directory. */ assertMakeDir("dir", 0775); fprintf(filelist, "dir\n"); - if (is_LargeInode("dir")) + if (is_LargeInode("dir")) { strncat(result, - "bsdcpio: dir: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: dir: large inode number truncated: ", sizeof(result) - strlen(result) -1); + strncat(result, + strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, + "\n", + sizeof(result) - strlen(result) -1); + } strncat(result, "2 blocks\n", sizeof(result) - strlen(result) -1); /* All done. */ fclose(filelist); assertUmask(022); /* Archive/dearchive with a variety of options. */ msg = canSymlink() ? "2 blocks\n" : "1 block\n"; basic_cpio("copy", "", "", msg, msg); basic_cpio("copy_odc", "--format=odc", "", msg, msg); basic_cpio("copy_newc", "-H newc", "", result, "2 blocks\n"); basic_cpio("copy_cpio", "-H odc", "", msg, msg); msg = canSymlink() ? "9 blocks\n" : "8 blocks\n"; basic_cpio("copy_ustar", "-H ustar", "", msg, msg); /* Copy in one step using -p */ passthrough("passthrough"); } Index: projects/clang700-import/contrib/libarchive/cpio/test/test_format_newc.c =================================================================== --- projects/clang700-import/contrib/libarchive/cpio/test/test_format_newc.c (revision 338689) +++ projects/clang700-import/contrib/libarchive/cpio/test/test_format_newc.c (revision 338690) @@ -1,329 +1,345 @@ /*- * Copyright (c) 2003-2007 Tim Kientzle * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "test.h" __FBSDID("$FreeBSD$"); /* Number of bytes needed to pad 'n' to multiple of 'block', assuming * that 'block' is a power of two. This trick can be more easily * remembered as -n & (block - 1), but many compilers quite reasonably * warn about "-n" when n is an unsigned value. (~(n) + 1) is the * same thing, but written in a way that won't offend anyone. */ #define PAD(n, block) ((~(n) + 1) & ((block) - 1)) static int is_hex(const char *p, size_t l) { while (l > 0) { if ((*p >= '0' && *p <= '9') || (*p >= 'a' && *p <= 'f') || (*p >= 'A' && *p <= 'F')) { --l; ++p; } else return (0); } return (1); } static int from_hex(const char *p, size_t l) { int r = 0; while (l > 0) { r *= 16; if (*p >= 'a' && *p <= 'f') r += *p + 10 - 'a'; else if (*p >= 'A' && *p <= 'F') r += *p + 10 - 'A'; else r += *p - '0'; --l; ++p; } return (r); } #if !defined(_WIN32) || defined(__CYGWIN__) static int nlinks(const char *p) { struct stat st; assertEqualInt(0, stat(p, &st)); return st.st_nlink; } #endif DEFINE_TEST(test_format_newc) { FILE *list; int r; int devmajor, devminor, ino, gid; int uid = -1; time_t t, t2, now; char *p, *e; size_t s, fs, ns; char result[1024]; assertUmask(0); #if !defined(_WIN32) uid = getuid(); #endif /* * Create an assortment of files. * TODO: Extend this to cover more filetypes. */ list = fopen("list", "w"); /* "file1" */ assertMakeFile("file1", 0644, "1234567890"); fprintf(list, "file1\n"); /* "hardlink" */ assertMakeHardlink("hardlink", "file1"); fprintf(list, "hardlink\n"); /* Another hardlink, but this one won't be archived. */ assertMakeHardlink("hardlink2", "file1"); /* "symlink" */ if (canSymlink()) { assertMakeSymlink("symlink", "file1"); fprintf(list, "symlink\n"); } /* "dir" */ assertMakeDir("dir", 0775); fprintf(list, "dir\n"); /* Setup result message. */ memset(result, 0, sizeof(result)); - if (is_LargeInode("file1")) + if (is_LargeInode("file1")) { strncat(result, - "bsdcpio: file1: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: file1: large inode number truncated: ", sizeof(result) - strlen(result) -1); - if (canSymlink() && is_LargeInode("symlink")) + strncat(result, strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, "\n", + sizeof(result) - strlen(result) -1); + } + if (canSymlink() && is_LargeInode("symlink")) { strncat(result, - "bsdcpio: symlink: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: symlink: large inode number truncated: ", sizeof(result) - strlen(result) -1); - if (is_LargeInode("dir")) + strncat(result, strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, "\n", + sizeof(result) - strlen(result) -1); + } + if (is_LargeInode("dir")) { strncat(result, - "bsdcpio: dir: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: dir: large inode number truncated: ", sizeof(result) - strlen(result) -1); - if (is_LargeInode("hardlink")) + strncat(result, strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, "\n", + sizeof(result) - strlen(result) -1); + } + if (is_LargeInode("hardlink")) { strncat(result, - "bsdcpio: hardlink: large inode number truncated: " - "Numerical result out of range\n", + "bsdcpio: hardlink: large inode number truncated: ", sizeof(result) - strlen(result) -1); + strncat(result, strerror(ERANGE), + sizeof(result) - strlen(result) -1); + strncat(result, "\n", + sizeof(result) - strlen(result) -1); + } /* Record some facts about what we just created: */ now = time(NULL); /* They were all created w/in last two seconds. */ /* Use the cpio program to create an archive. */ fclose(list); r = systemf("%s -o --format=newc newc.out 2>newc.err", testprog); if (!assertEqualInt(r, 0)) return; /* Verify that nothing went to stderr. */ if (canSymlink()) { strncat(result, "2 blocks\n", sizeof(result) - strlen(result) -1); } else { strncat(result, "1 block\n", sizeof(result) - strlen(result) -1); } assertTextFileContents(result, "newc.err"); /* Verify that stdout is a well-formed cpio file in "newc" format. */ p = slurpfile(&s, "newc.out"); assertEqualInt(s, canSymlink() ? 1024 : 512); e = p; /* * Some of these assertions could be stronger, but it's * a little tricky because they depend on the local environment. */ /* First entry is "file1" */ assert(is_hex(e, 110)); /* Entire header is octal digits. */ assertEqualMem(e + 0, "070701", 6); /* Magic */ ino = from_hex(e + 6, 8); /* ino */ #if defined(_WIN32) && !defined(__CYGWIN__) /* Group members bits and others bits do not work. */ assertEqualInt(0x8180, from_hex(e + 14, 8) & 0xffc0); /* Mode */ #else assertEqualInt(0x81a4, from_hex(e + 14, 8)); /* Mode */ #endif if (uid < 0) uid = from_hex(e + 22, 8); assertEqualInt(from_hex(e + 22, 8), uid); /* uid */ gid = from_hex(e + 30, 8); /* gid */ assertEqualMem(e + 38, "00000003", 8); /* nlink */ t = from_hex(e + 46, 8); /* mtime */ failure("t=0x%08x now=0x%08x=%d", t, now, now); assert(t <= now); /* File wasn't created in future. */ failure("t=0x%08x now - 2=0x%08x = %d", t, now - 2, now - 2); assert(t >= now - 2); /* File was created w/in last 2 secs. */ failure("newc format stores body only with last appearance of a link\n" " first appearance should be empty, so this file size\n" " field should be zero"); assertEqualInt(0, from_hex(e + 54, 8)); /* File size */ fs = from_hex(e + 54, 8); fs += PAD(fs, 4); devmajor = from_hex(e + 62, 8); /* devmajor */ devminor = from_hex(e + 70, 8); /* devminor */ assert(is_hex(e + 78, 8)); /* rdevmajor */ assert(is_hex(e + 86, 8)); /* rdevminor */ assertEqualMem(e + 94, "00000006", 8); /* Name size */ ns = from_hex(e + 94, 8); ns += PAD(ns + 2, 4); assertEqualInt(0, from_hex(e + 102, 8)); /* check field */ assertEqualMem(e + 110, "file1\0", 6); /* Name contents */ /* Since there's another link, no file contents here. */ /* But add in file size so that an error here doesn't cascade. */ e += 110 + fs + ns; if (canSymlink()) { /* "symlink" pointing to "file1" */ assert(is_hex(e, 110)); assertEqualMem(e + 0, "070701", 6); /* Magic */ assert(is_hex(e + 6, 8)); /* ino */ assertEqualInt(0xa1ff, from_hex(e + 14, 8)); /* Mode */ assertEqualInt(from_hex(e + 22, 8), uid); /* uid */ assertEqualInt(gid, from_hex(e + 30, 8)); /* gid */ assertEqualMem(e + 38, "00000001", 8); /* nlink */ t2 = from_hex(e + 46, 8); /* mtime */ failure("First entry created at t=0x%08x this entry created at t2=0x%08x", t, t2); assert(t2 == t || t2 == t + 1); /* Almost same as first entry. */ assertEqualMem(e + 54, "00000005", 8); /* File size */ fs = from_hex(e + 54, 8); fs += PAD(fs, 4); assertEqualInt(devmajor, from_hex(e + 62, 8)); /* devmajor */ assertEqualInt(devminor, from_hex(e + 70, 8)); /* devminor */ assert(is_hex(e + 78, 8)); /* rdevmajor */ assert(is_hex(e + 86, 8)); /* rdevminor */ assertEqualMem(e + 94, "00000008", 8); /* Name size */ ns = from_hex(e + 94, 8); ns += PAD(ns + 2, 4); assertEqualInt(0, from_hex(e + 102, 8)); /* check field */ assertEqualMem(e + 110, "symlink\0\0\0", 10); /* Name contents */ assertEqualMem(e + 110 + ns, "file1\0\0\0", 8); /* symlink target */ e += 110 + fs + ns; } /* "dir" */ assert(is_hex(e, 110)); assertEqualMem(e + 0, "070701", 6); /* Magic */ assert(is_hex(e + 6, 8)); /* ino */ #if defined(_WIN32) && !defined(__CYGWIN__) /* Group members bits and others bits do not work. */ assertEqualInt(0x41c0, from_hex(e + 14, 8) & 0xffc0); /* Mode */ #else /* Mode: sgid bit sometimes propagates from parent dirs, ignore it. */ assertEqualInt(040775, from_hex(e + 14, 8) & ~02000); #endif assertEqualInt(uid, from_hex(e + 22, 8)); /* uid */ assertEqualInt(gid, from_hex(e + 30, 8)); /* gid */ #if !defined(_WIN32) || defined(__CYGWIN__) assertEqualInt(nlinks("dir"), from_hex(e + 38, 8)); /* nlinks */ #endif t2 = from_hex(e + 46, 8); /* mtime */ failure("First entry created at t=0x%08x this entry created at t2=0x%08x", t, t2); assert(t2 == t || t2 == t + 1); /* Almost same as first entry. */ assertEqualMem(e + 54, "00000000", 8); /* File size */ fs = from_hex(e + 54, 8); fs += PAD(fs, 4); assertEqualInt(devmajor, from_hex(e + 62, 8)); /* devmajor */ assertEqualInt(devminor, from_hex(e + 70, 8)); /* devminor */ assert(is_hex(e + 78, 8)); /* rdevmajor */ assert(is_hex(e + 86, 8)); /* rdevminor */ assertEqualMem(e + 94, "00000004", 8); /* Name size */ ns = from_hex(e + 94, 8); ns += PAD(ns + 2, 4); assertEqualInt(0, from_hex(e + 102, 8)); /* check field */ assertEqualMem(e + 110, "dir\0\0\0", 6); /* Name contents */ e += 110 + fs + ns; /* Hardlink identical to "file1" */ /* Since we only wrote two of the three links to this * file, this link should get deferred by the hardlink logic. */ assert(is_hex(e, 110)); assertEqualMem(e + 0, "070701", 6); /* Magic */ failure("If these aren't the same, then the hardlink detection failed to match them."); assertEqualInt(ino, from_hex(e + 6, 8)); /* ino */ #if defined(_WIN32) && !defined(__CYGWIN__) /* Group members bits and others bits do not work. */ assertEqualInt(0x8180, from_hex(e + 14, 8) & 0xffc0); /* Mode */ #else assertEqualInt(0x81a4, from_hex(e + 14, 8)); /* Mode */ #endif assertEqualInt(from_hex(e + 22, 8), uid); /* uid */ assertEqualInt(gid, from_hex(e + 30, 8)); /* gid */ assertEqualMem(e + 38, "00000003", 8); /* nlink */ t2 = from_hex(e + 46, 8); /* mtime */ failure("First entry created at t=0x%08x this entry created at t2=0x%08x", t, t2); assert(t2 == t || t2 == t + 1); /* Almost same as first entry. */ assertEqualInt(10, from_hex(e + 54, 8)); /* File size */ fs = from_hex(e + 54, 8); fs += PAD(fs, 4); assertEqualInt(devmajor, from_hex(e + 62, 8)); /* devmajor */ assertEqualInt(devminor, from_hex(e + 70, 8)); /* devminor */ assert(is_hex(e + 78, 8)); /* rdevmajor */ assert(is_hex(e + 86, 8)); /* rdevminor */ assertEqualMem(e + 94, "00000009", 8); /* Name size */ ns = from_hex(e + 94, 8); ns += PAD(ns + 2, 4); assertEqualInt(0, from_hex(e + 102, 8)); /* check field */ assertEqualMem(e + 110, "hardlink\0\0", 10); /* Name contents */ assertEqualMem(e + 110 + ns, "1234567890\0\0", 12); /* File contents */ e += 110 + ns + fs; /* Last entry is end-of-archive marker. */ assert(is_hex(e, 110)); assertEqualMem(e + 0, "070701", 6); /* Magic */ assertEqualMem(e + 8, "00000000", 8); /* ino */ assertEqualMem(e + 14, "00000000", 8); /* mode */ assertEqualMem(e + 22, "00000000", 8); /* uid */ assertEqualMem(e + 30, "00000000", 8); /* gid */ assertEqualMem(e + 38, "00000001", 8); /* nlink */ assertEqualMem(e + 46, "00000000", 8); /* mtime */ assertEqualMem(e + 54, "00000000", 8); /* size */ assertEqualMem(e + 62, "00000000", 8); /* devmajor */ assertEqualMem(e + 70, "00000000", 8); /* devminor */ assertEqualMem(e + 78, "00000000", 8); /* rdevmajor */ assertEqualMem(e + 86, "00000000", 8); /* rdevminor */ assertEqualInt(11, from_hex(e + 94, 8)); /* name size */ assertEqualMem(e + 102, "00000000", 8); /* check field */ assertEqualMem(e + 110, "TRAILER!!!\0\0", 12); /* Name */ free(p); } Index: projects/clang700-import/contrib/libarchive/libarchive/archive.h =================================================================== --- projects/clang700-import/contrib/libarchive/libarchive/archive.h (revision 338689) +++ projects/clang700-import/contrib/libarchive/libarchive/archive.h (revision 338690) @@ -1,1191 +1,1191 @@ /*- * Copyright (c) 2003-2010 Tim Kientzle * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ #ifndef ARCHIVE_H_INCLUDED #define ARCHIVE_H_INCLUDED /* * The version number is expressed as a single integer that makes it * easy to compare versions at build time: for version a.b.c, the * version number is printf("%d%03d%03d",a,b,c). For example, if you * know your application requires version 2.12.108 or later, you can * assert that ARCHIVE_VERSION_NUMBER >= 2012108. */ /* Note: Compiler will complain if this does not match archive_entry.h! */ -#define ARCHIVE_VERSION_NUMBER 3003002 +#define ARCHIVE_VERSION_NUMBER 3003003 #include #include /* for wchar_t */ #include /* For FILE * */ #include /* For time_t */ /* * Note: archive.h is for use outside of libarchive; the configuration * headers (config.h, archive_platform.h, etc.) are purely internal. * Do NOT use HAVE_XXX configuration macros to control the behavior of * this header! If you must conditionalize, use predefined compiler and/or * platform macros. */ #if defined(__BORLANDC__) && __BORLANDC__ >= 0x560 # include #elif !defined(__WATCOMC__) && !defined(_MSC_VER) && !defined(__INTERIX) && !defined(__BORLANDC__) && !defined(_SCO_DS) && !defined(__osf__) # include #endif /* Get appropriate definitions of 64-bit integer */ #if !defined(__LA_INT64_T_DEFINED) /* Older code relied on the __LA_INT64_T macro; after 4.0 we'll switch to the typedef exclusively. */ # if ARCHIVE_VERSION_NUMBER < 4000000 #define __LA_INT64_T la_int64_t # endif #define __LA_INT64_T_DEFINED # if defined(_WIN32) && !defined(__CYGWIN__) && !defined(__WATCOMC__) typedef __int64 la_int64_t; # else # include /* ssize_t */ # if defined(_SCO_DS) || defined(__osf__) typedef long long la_int64_t; # else typedef int64_t la_int64_t; # endif # endif #endif /* The la_ssize_t should match the type used in 'struct stat' */ #if !defined(__LA_SSIZE_T_DEFINED) /* Older code relied on the __LA_SSIZE_T macro; after 4.0 we'll switch to the typedef exclusively. */ # if ARCHIVE_VERSION_NUMBER < 4000000 #define __LA_SSIZE_T la_ssize_t # endif #define __LA_SSIZE_T_DEFINED # if defined(_WIN32) && !defined(__CYGWIN__) && !defined(__WATCOMC__) # if defined(_SSIZE_T_DEFINED) || defined(_SSIZE_T_) typedef ssize_t la_ssize_t; # elif defined(_WIN64) typedef __int64 la_ssize_t; # else typedef long la_ssize_t; # endif # else # include /* ssize_t */ typedef ssize_t la_ssize_t; # endif #endif /* Large file support for Android */ #ifdef __ANDROID__ #include "android_lf.h" #endif /* * On Windows, define LIBARCHIVE_STATIC if you're building or using a * .lib. The default here assumes you're building a DLL. Only * libarchive source should ever define __LIBARCHIVE_BUILD. */ #if ((defined __WIN32__) || (defined _WIN32) || defined(__CYGWIN__)) && (!defined LIBARCHIVE_STATIC) # ifdef __LIBARCHIVE_BUILD # ifdef __GNUC__ # define __LA_DECL __attribute__((dllexport)) extern # else # define __LA_DECL __declspec(dllexport) # endif # else # ifdef __GNUC__ # define __LA_DECL # else # define __LA_DECL __declspec(dllimport) # endif # endif #else /* Static libraries or non-Windows needs no special declaration. */ # define __LA_DECL #endif #if defined(__GNUC__) && __GNUC__ >= 3 && !defined(__MINGW32__) #define __LA_PRINTF(fmtarg, firstvararg) \ __attribute__((__format__ (__printf__, fmtarg, firstvararg))) #else #define __LA_PRINTF(fmtarg, firstvararg) /* nothing */ #endif #if defined(__GNUC__) && __GNUC__ >= 3 && __GNUC_MINOR__ >= 1 # define __LA_DEPRECATED __attribute__((deprecated)) #else # define __LA_DEPRECATED #endif #ifdef __cplusplus extern "C" { #endif /* * The version number is provided as both a macro and a function. * The macro identifies the installed header; the function identifies * the library version (which may not be the same if you're using a * dynamically-linked version of the library). Of course, if the * header and library are very different, you should expect some * strangeness. Don't do that. */ __LA_DECL int archive_version_number(void); /* * Textual name/version of the library, useful for version displays. */ -#define ARCHIVE_VERSION_ONLY_STRING "3.3.2" +#define ARCHIVE_VERSION_ONLY_STRING "3.3.3" #define ARCHIVE_VERSION_STRING "libarchive " ARCHIVE_VERSION_ONLY_STRING __LA_DECL const char * archive_version_string(void); /* * Detailed textual name/version of the library and its dependencies. * This has the form: * "libarchive x.y.z zlib/a.b.c liblzma/d.e.f ... etc ..." * the list of libraries described here will vary depending on how * libarchive was compiled. */ __LA_DECL const char * archive_version_details(void); /* * Returns NULL if libarchive was compiled without the associated library. * Otherwise, returns the version number that libarchive was compiled * against. */ __LA_DECL const char * archive_zlib_version(void); __LA_DECL const char * archive_liblzma_version(void); __LA_DECL const char * archive_bzlib_version(void); __LA_DECL const char * archive_liblz4_version(void); __LA_DECL const char * archive_libzstd_version(void); /* Declare our basic types. */ struct archive; struct archive_entry; /* * Error codes: Use archive_errno() and archive_error_string() * to retrieve details. Unless specified otherwise, all functions * that return 'int' use these codes. */ #define ARCHIVE_EOF 1 /* Found end of archive. */ #define ARCHIVE_OK 0 /* Operation was successful. */ #define ARCHIVE_RETRY (-10) /* Retry might succeed. */ #define ARCHIVE_WARN (-20) /* Partial success. */ /* For example, if write_header "fails", then you can't push data. */ #define ARCHIVE_FAILED (-25) /* Current operation cannot complete. */ /* But if write_header is "fatal," then this archive is dead and useless. */ #define ARCHIVE_FATAL (-30) /* No more operations are possible. */ /* * As far as possible, archive_errno returns standard platform errno codes. * Of course, the details vary by platform, so the actual definitions * here are stored in "archive_platform.h". The symbols are listed here * for reference; as a rule, clients should not need to know the exact * platform-dependent error code. */ /* Unrecognized or invalid file format. */ /* #define ARCHIVE_ERRNO_FILE_FORMAT */ /* Illegal usage of the library. */ /* #define ARCHIVE_ERRNO_PROGRAMMER_ERROR */ /* Unknown or unclassified error. */ /* #define ARCHIVE_ERRNO_MISC */ /* * Callbacks are invoked to automatically read/skip/write/open/close the * archive. You can provide your own for complex tasks (like breaking * archives across multiple tapes) or use standard ones built into the * library. */ /* Returns pointer and size of next block of data from archive. */ typedef la_ssize_t archive_read_callback(struct archive *, void *_client_data, const void **_buffer); /* Skips at most request bytes from archive and returns the skipped amount. * This may skip fewer bytes than requested; it may even skip zero bytes. * If you do skip fewer bytes than requested, libarchive will invoke your * read callback and discard data as necessary to make up the full skip. */ typedef la_int64_t archive_skip_callback(struct archive *, void *_client_data, la_int64_t request); /* Seeks to specified location in the file and returns the position. * Whence values are SEEK_SET, SEEK_CUR, SEEK_END from stdio.h. * Return ARCHIVE_FATAL if the seek fails for any reason. */ typedef la_int64_t archive_seek_callback(struct archive *, void *_client_data, la_int64_t offset, int whence); /* Returns size actually written, zero on EOF, -1 on error. */ typedef la_ssize_t archive_write_callback(struct archive *, void *_client_data, const void *_buffer, size_t _length); typedef int archive_open_callback(struct archive *, void *_client_data); typedef int archive_close_callback(struct archive *, void *_client_data); /* Switches from one client data object to the next/prev client data object. * This is useful for reading from different data blocks such as a set of files * that make up one large file. */ typedef int archive_switch_callback(struct archive *, void *_client_data1, void *_client_data2); /* * Returns a passphrase used for encryption or decryption, NULL on nothing * to do and give it up. */ typedef const char *archive_passphrase_callback(struct archive *, void *_client_data); /* * Codes to identify various stream filters. */ #define ARCHIVE_FILTER_NONE 0 #define ARCHIVE_FILTER_GZIP 1 #define ARCHIVE_FILTER_BZIP2 2 #define ARCHIVE_FILTER_COMPRESS 3 #define ARCHIVE_FILTER_PROGRAM 4 #define ARCHIVE_FILTER_LZMA 5 #define ARCHIVE_FILTER_XZ 6 #define ARCHIVE_FILTER_UU 7 #define ARCHIVE_FILTER_RPM 8 #define ARCHIVE_FILTER_LZIP 9 #define ARCHIVE_FILTER_LRZIP 10 #define ARCHIVE_FILTER_LZOP 11 #define ARCHIVE_FILTER_GRZIP 12 #define ARCHIVE_FILTER_LZ4 13 #define ARCHIVE_FILTER_ZSTD 14 #if ARCHIVE_VERSION_NUMBER < 4000000 #define ARCHIVE_COMPRESSION_NONE ARCHIVE_FILTER_NONE #define ARCHIVE_COMPRESSION_GZIP ARCHIVE_FILTER_GZIP #define ARCHIVE_COMPRESSION_BZIP2 ARCHIVE_FILTER_BZIP2 #define ARCHIVE_COMPRESSION_COMPRESS ARCHIVE_FILTER_COMPRESS #define ARCHIVE_COMPRESSION_PROGRAM ARCHIVE_FILTER_PROGRAM #define ARCHIVE_COMPRESSION_LZMA ARCHIVE_FILTER_LZMA #define ARCHIVE_COMPRESSION_XZ ARCHIVE_FILTER_XZ #define ARCHIVE_COMPRESSION_UU ARCHIVE_FILTER_UU #define ARCHIVE_COMPRESSION_RPM ARCHIVE_FILTER_RPM #define ARCHIVE_COMPRESSION_LZIP ARCHIVE_FILTER_LZIP #define ARCHIVE_COMPRESSION_LRZIP ARCHIVE_FILTER_LRZIP #endif /* * Codes returned by archive_format. * * Top 16 bits identifies the format family (e.g., "tar"); lower * 16 bits indicate the variant. This is updated by read_next_header. * Note that the lower 16 bits will often vary from entry to entry. * In some cases, this variation occurs as libarchive learns more about * the archive (for example, later entries might utilize extensions that * weren't necessary earlier in the archive; in this case, libarchive * will change the format code to indicate the extended format that * was used). In other cases, it's because different tools have * modified the archive and so different parts of the archive * actually have slightly different formats. (Both tar and cpio store * format codes in each entry, so it is quite possible for each * entry to be in a different format.) */ #define ARCHIVE_FORMAT_BASE_MASK 0xff0000 #define ARCHIVE_FORMAT_CPIO 0x10000 #define ARCHIVE_FORMAT_CPIO_POSIX (ARCHIVE_FORMAT_CPIO | 1) #define ARCHIVE_FORMAT_CPIO_BIN_LE (ARCHIVE_FORMAT_CPIO | 2) #define ARCHIVE_FORMAT_CPIO_BIN_BE (ARCHIVE_FORMAT_CPIO | 3) #define ARCHIVE_FORMAT_CPIO_SVR4_NOCRC (ARCHIVE_FORMAT_CPIO | 4) #define ARCHIVE_FORMAT_CPIO_SVR4_CRC (ARCHIVE_FORMAT_CPIO | 5) #define ARCHIVE_FORMAT_CPIO_AFIO_LARGE (ARCHIVE_FORMAT_CPIO | 6) #define ARCHIVE_FORMAT_SHAR 0x20000 #define ARCHIVE_FORMAT_SHAR_BASE (ARCHIVE_FORMAT_SHAR | 1) #define ARCHIVE_FORMAT_SHAR_DUMP (ARCHIVE_FORMAT_SHAR | 2) #define ARCHIVE_FORMAT_TAR 0x30000 #define ARCHIVE_FORMAT_TAR_USTAR (ARCHIVE_FORMAT_TAR | 1) #define ARCHIVE_FORMAT_TAR_PAX_INTERCHANGE (ARCHIVE_FORMAT_TAR | 2) #define ARCHIVE_FORMAT_TAR_PAX_RESTRICTED (ARCHIVE_FORMAT_TAR | 3) #define ARCHIVE_FORMAT_TAR_GNUTAR (ARCHIVE_FORMAT_TAR | 4) #define ARCHIVE_FORMAT_ISO9660 0x40000 #define ARCHIVE_FORMAT_ISO9660_ROCKRIDGE (ARCHIVE_FORMAT_ISO9660 | 1) #define ARCHIVE_FORMAT_ZIP 0x50000 #define ARCHIVE_FORMAT_EMPTY 0x60000 #define ARCHIVE_FORMAT_AR 0x70000 #define ARCHIVE_FORMAT_AR_GNU (ARCHIVE_FORMAT_AR | 1) #define ARCHIVE_FORMAT_AR_BSD (ARCHIVE_FORMAT_AR | 2) #define ARCHIVE_FORMAT_MTREE 0x80000 #define ARCHIVE_FORMAT_RAW 0x90000 #define ARCHIVE_FORMAT_XAR 0xA0000 #define ARCHIVE_FORMAT_LHA 0xB0000 #define ARCHIVE_FORMAT_CAB 0xC0000 #define ARCHIVE_FORMAT_RAR 0xD0000 #define ARCHIVE_FORMAT_7ZIP 0xE0000 #define ARCHIVE_FORMAT_WARC 0xF0000 /* * Codes returned by archive_read_format_capabilities(). * * This list can be extended with values between 0 and 0xffff. * The original purpose of this list was to let different archive * format readers expose their general capabilities in terms of * encryption. */ #define ARCHIVE_READ_FORMAT_CAPS_NONE (0) /* no special capabilities */ #define ARCHIVE_READ_FORMAT_CAPS_ENCRYPT_DATA (1<<0) /* reader can detect encrypted data */ #define ARCHIVE_READ_FORMAT_CAPS_ENCRYPT_METADATA (1<<1) /* reader can detect encryptable metadata (pathname, mtime, etc.) */ /* * Codes returned by archive_read_has_encrypted_entries(). * * In case the archive does not support encryption detection at all * ARCHIVE_READ_FORMAT_ENCRYPTION_UNSUPPORTED is returned. If the reader * for some other reason (e.g. not enough bytes read) cannot say if * there are encrypted entries, ARCHIVE_READ_FORMAT_ENCRYPTION_DONT_KNOW * is returned. */ #define ARCHIVE_READ_FORMAT_ENCRYPTION_UNSUPPORTED -2 #define ARCHIVE_READ_FORMAT_ENCRYPTION_DONT_KNOW -1 /*- * Basic outline for reading an archive: * 1) Ask archive_read_new for an archive reader object. * 2) Update any global properties as appropriate. * In particular, you'll certainly want to call appropriate * archive_read_support_XXX functions. * 3) Call archive_read_open_XXX to open the archive * 4) Repeatedly call archive_read_next_header to get information about * successive archive entries. Call archive_read_data to extract * data for entries of interest. * 5) Call archive_read_free to end processing. */ __LA_DECL struct archive *archive_read_new(void); /* * The archive_read_support_XXX calls enable auto-detect for this * archive handle. They also link in the necessary support code. * For example, if you don't want bzlib linked in, don't invoke * support_compression_bzip2(). The "all" functions provide the * obvious shorthand. */ #if ARCHIVE_VERSION_NUMBER < 4000000 __LA_DECL int archive_read_support_compression_all(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_bzip2(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_compress(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_gzip(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_lzip(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_lzma(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_none(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_program(struct archive *, const char *command) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_program_signature (struct archive *, const char *, const void * /* match */, size_t) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_rpm(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_uu(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_read_support_compression_xz(struct archive *) __LA_DEPRECATED; #endif __LA_DECL int archive_read_support_filter_all(struct archive *); __LA_DECL int archive_read_support_filter_bzip2(struct archive *); __LA_DECL int archive_read_support_filter_compress(struct archive *); __LA_DECL int archive_read_support_filter_gzip(struct archive *); __LA_DECL int archive_read_support_filter_grzip(struct archive *); __LA_DECL int archive_read_support_filter_lrzip(struct archive *); __LA_DECL int archive_read_support_filter_lz4(struct archive *); __LA_DECL int archive_read_support_filter_lzip(struct archive *); __LA_DECL int archive_read_support_filter_lzma(struct archive *); __LA_DECL int archive_read_support_filter_lzop(struct archive *); __LA_DECL int archive_read_support_filter_none(struct archive *); __LA_DECL int archive_read_support_filter_program(struct archive *, const char *command); __LA_DECL int archive_read_support_filter_program_signature (struct archive *, const char * /* cmd */, const void * /* match */, size_t); __LA_DECL int archive_read_support_filter_rpm(struct archive *); __LA_DECL int archive_read_support_filter_uu(struct archive *); __LA_DECL int archive_read_support_filter_xz(struct archive *); __LA_DECL int archive_read_support_filter_zstd(struct archive *); __LA_DECL int archive_read_support_format_7zip(struct archive *); __LA_DECL int archive_read_support_format_all(struct archive *); __LA_DECL int archive_read_support_format_ar(struct archive *); __LA_DECL int archive_read_support_format_by_code(struct archive *, int); __LA_DECL int archive_read_support_format_cab(struct archive *); __LA_DECL int archive_read_support_format_cpio(struct archive *); __LA_DECL int archive_read_support_format_empty(struct archive *); __LA_DECL int archive_read_support_format_gnutar(struct archive *); __LA_DECL int archive_read_support_format_iso9660(struct archive *); __LA_DECL int archive_read_support_format_lha(struct archive *); __LA_DECL int archive_read_support_format_mtree(struct archive *); __LA_DECL int archive_read_support_format_rar(struct archive *); __LA_DECL int archive_read_support_format_raw(struct archive *); __LA_DECL int archive_read_support_format_tar(struct archive *); __LA_DECL int archive_read_support_format_warc(struct archive *); __LA_DECL int archive_read_support_format_xar(struct archive *); /* archive_read_support_format_zip() enables both streamable and seekable * zip readers. */ __LA_DECL int archive_read_support_format_zip(struct archive *); /* Reads Zip archives as stream from beginning to end. Doesn't * correctly handle SFX ZIP files or ZIP archives that have been modified * in-place. */ __LA_DECL int archive_read_support_format_zip_streamable(struct archive *); /* Reads starting from central directory; requires seekable input. */ __LA_DECL int archive_read_support_format_zip_seekable(struct archive *); /* Functions to manually set the format and filters to be used. This is * useful to bypass the bidding process when the format and filters to use * is known in advance. */ __LA_DECL int archive_read_set_format(struct archive *, int); __LA_DECL int archive_read_append_filter(struct archive *, int); __LA_DECL int archive_read_append_filter_program(struct archive *, const char *); __LA_DECL int archive_read_append_filter_program_signature (struct archive *, const char *, const void * /* match */, size_t); /* Set various callbacks. */ __LA_DECL int archive_read_set_open_callback(struct archive *, archive_open_callback *); __LA_DECL int archive_read_set_read_callback(struct archive *, archive_read_callback *); __LA_DECL int archive_read_set_seek_callback(struct archive *, archive_seek_callback *); __LA_DECL int archive_read_set_skip_callback(struct archive *, archive_skip_callback *); __LA_DECL int archive_read_set_close_callback(struct archive *, archive_close_callback *); /* Callback used to switch between one data object to the next */ __LA_DECL int archive_read_set_switch_callback(struct archive *, archive_switch_callback *); /* This sets the first data object. */ __LA_DECL int archive_read_set_callback_data(struct archive *, void *); /* This sets data object at specified index */ __LA_DECL int archive_read_set_callback_data2(struct archive *, void *, unsigned int); /* This adds a data object at the specified index. */ __LA_DECL int archive_read_add_callback_data(struct archive *, void *, unsigned int); /* This appends a data object to the end of list */ __LA_DECL int archive_read_append_callback_data(struct archive *, void *); /* This prepends a data object to the beginning of list */ __LA_DECL int archive_read_prepend_callback_data(struct archive *, void *); /* Opening freezes the callbacks. */ __LA_DECL int archive_read_open1(struct archive *); /* Convenience wrappers around the above. */ __LA_DECL int archive_read_open(struct archive *, void *_client_data, archive_open_callback *, archive_read_callback *, archive_close_callback *); __LA_DECL int archive_read_open2(struct archive *, void *_client_data, archive_open_callback *, archive_read_callback *, archive_skip_callback *, archive_close_callback *); /* * A variety of shortcuts that invoke archive_read_open() with * canned callbacks suitable for common situations. The ones that * accept a block size handle tape blocking correctly. */ /* Use this if you know the filename. Note: NULL indicates stdin. */ __LA_DECL int archive_read_open_filename(struct archive *, const char *_filename, size_t _block_size); /* Use this for reading multivolume files by filenames. * NOTE: Must be NULL terminated. Sorting is NOT done. */ __LA_DECL int archive_read_open_filenames(struct archive *, const char **_filenames, size_t _block_size); __LA_DECL int archive_read_open_filename_w(struct archive *, const wchar_t *_filename, size_t _block_size); /* archive_read_open_file() is a deprecated synonym for ..._open_filename(). */ __LA_DECL int archive_read_open_file(struct archive *, const char *_filename, size_t _block_size) __LA_DEPRECATED; /* Read an archive that's stored in memory. */ __LA_DECL int archive_read_open_memory(struct archive *, const void * buff, size_t size); /* A more involved version that is only used for internal testing. */ __LA_DECL int archive_read_open_memory2(struct archive *a, const void *buff, size_t size, size_t read_size); /* Read an archive that's already open, using the file descriptor. */ __LA_DECL int archive_read_open_fd(struct archive *, int _fd, size_t _block_size); /* Read an archive that's already open, using a FILE *. */ /* Note: DO NOT use this with tape drives. */ __LA_DECL int archive_read_open_FILE(struct archive *, FILE *_file); /* Parses and returns next entry header. */ __LA_DECL int archive_read_next_header(struct archive *, struct archive_entry **); /* Parses and returns next entry header using the archive_entry passed in */ __LA_DECL int archive_read_next_header2(struct archive *, struct archive_entry *); /* * Retrieve the byte offset in UNCOMPRESSED data where last-read * header started. */ __LA_DECL la_int64_t archive_read_header_position(struct archive *); /* * Returns 1 if the archive contains at least one encrypted entry. * If the archive format not support encryption at all * ARCHIVE_READ_FORMAT_ENCRYPTION_UNSUPPORTED is returned. * If for any other reason (e.g. not enough data read so far) * we cannot say whether there are encrypted entries, then * ARCHIVE_READ_FORMAT_ENCRYPTION_DONT_KNOW is returned. * In general, this function will return values below zero when the * reader is uncertain or totally incapable of encryption support. * When this function returns 0 you can be sure that the reader * supports encryption detection but no encrypted entries have * been found yet. * * NOTE: If the metadata/header of an archive is also encrypted, you * cannot rely on the number of encrypted entries. That is why this * function does not return the number of encrypted entries but# * just shows that there are some. */ __LA_DECL int archive_read_has_encrypted_entries(struct archive *); /* * Returns a bitmask of capabilities that are supported by the archive format reader. * If the reader has no special capabilities, ARCHIVE_READ_FORMAT_CAPS_NONE is returned. */ __LA_DECL int archive_read_format_capabilities(struct archive *); /* Read data from the body of an entry. Similar to read(2). */ __LA_DECL la_ssize_t archive_read_data(struct archive *, void *, size_t); /* Seek within the body of an entry. Similar to lseek(2). */ __LA_DECL la_int64_t archive_seek_data(struct archive *, la_int64_t, int); /* * A zero-copy version of archive_read_data that also exposes the file offset * of each returned block. Note that the client has no way to specify * the desired size of the block. The API does guarantee that offsets will * be strictly increasing and that returned blocks will not overlap. */ __LA_DECL int archive_read_data_block(struct archive *a, const void **buff, size_t *size, la_int64_t *offset); /*- * Some convenience functions that are built on archive_read_data: * 'skip': skips entire entry * 'into_buffer': writes data into memory buffer that you provide * 'into_fd': writes data to specified filedes */ __LA_DECL int archive_read_data_skip(struct archive *); __LA_DECL int archive_read_data_into_fd(struct archive *, int fd); /* * Set read options. */ /* Apply option to the format only. */ __LA_DECL int archive_read_set_format_option(struct archive *_a, const char *m, const char *o, const char *v); /* Apply option to the filter only. */ __LA_DECL int archive_read_set_filter_option(struct archive *_a, const char *m, const char *o, const char *v); /* Apply option to both the format and the filter. */ __LA_DECL int archive_read_set_option(struct archive *_a, const char *m, const char *o, const char *v); /* Apply option string to both the format and the filter. */ __LA_DECL int archive_read_set_options(struct archive *_a, const char *opts); /* * Add a decryption passphrase. */ __LA_DECL int archive_read_add_passphrase(struct archive *, const char *); __LA_DECL int archive_read_set_passphrase_callback(struct archive *, void *client_data, archive_passphrase_callback *); /*- * Convenience function to recreate the current entry (whose header * has just been read) on disk. * * This does quite a bit more than just copy data to disk. It also: * - Creates intermediate directories as required. * - Manages directory permissions: non-writable directories will * be initially created with write permission enabled; when the * archive is closed, dir permissions are edited to the values specified * in the archive. * - Checks hardlinks: hardlinks will not be extracted unless the * linked-to file was also extracted within the same session. (TODO) */ /* The "flags" argument selects optional behavior, 'OR' the flags you want. */ /* Default: Do not try to set owner/group. */ #define ARCHIVE_EXTRACT_OWNER (0x0001) /* Default: Do obey umask, do not restore SUID/SGID/SVTX bits. */ #define ARCHIVE_EXTRACT_PERM (0x0002) /* Default: Do not restore mtime/atime. */ #define ARCHIVE_EXTRACT_TIME (0x0004) /* Default: Replace existing files. */ #define ARCHIVE_EXTRACT_NO_OVERWRITE (0x0008) /* Default: Try create first, unlink only if create fails with EEXIST. */ #define ARCHIVE_EXTRACT_UNLINK (0x0010) /* Default: Do not restore ACLs. */ #define ARCHIVE_EXTRACT_ACL (0x0020) /* Default: Do not restore fflags. */ #define ARCHIVE_EXTRACT_FFLAGS (0x0040) /* Default: Do not restore xattrs. */ #define ARCHIVE_EXTRACT_XATTR (0x0080) /* Default: Do not try to guard against extracts redirected by symlinks. */ /* Note: With ARCHIVE_EXTRACT_UNLINK, will remove any intermediate symlink. */ #define ARCHIVE_EXTRACT_SECURE_SYMLINKS (0x0100) /* Default: Do not reject entries with '..' as path elements. */ #define ARCHIVE_EXTRACT_SECURE_NODOTDOT (0x0200) /* Default: Create parent directories as needed. */ #define ARCHIVE_EXTRACT_NO_AUTODIR (0x0400) /* Default: Overwrite files, even if one on disk is newer. */ #define ARCHIVE_EXTRACT_NO_OVERWRITE_NEWER (0x0800) /* Detect blocks of 0 and write holes instead. */ #define ARCHIVE_EXTRACT_SPARSE (0x1000) /* Default: Do not restore Mac extended metadata. */ /* This has no effect except on Mac OS. */ #define ARCHIVE_EXTRACT_MAC_METADATA (0x2000) /* Default: Use HFS+ compression if it was compressed. */ /* This has no effect except on Mac OS v10.6 or later. */ #define ARCHIVE_EXTRACT_NO_HFS_COMPRESSION (0x4000) /* Default: Do not use HFS+ compression if it was not compressed. */ /* This has no effect except on Mac OS v10.6 or later. */ #define ARCHIVE_EXTRACT_HFS_COMPRESSION_FORCED (0x8000) /* Default: Do not reject entries with absolute paths */ #define ARCHIVE_EXTRACT_SECURE_NOABSOLUTEPATHS (0x10000) /* Default: Do not clear no-change flags when unlinking object */ #define ARCHIVE_EXTRACT_CLEAR_NOCHANGE_FFLAGS (0x20000) __LA_DECL int archive_read_extract(struct archive *, struct archive_entry *, int flags); __LA_DECL int archive_read_extract2(struct archive *, struct archive_entry *, struct archive * /* dest */); __LA_DECL void archive_read_extract_set_progress_callback(struct archive *, void (*_progress_func)(void *), void *_user_data); /* Record the dev/ino of a file that will not be written. This is * generally set to the dev/ino of the archive being read. */ __LA_DECL void archive_read_extract_set_skip_file(struct archive *, la_int64_t, la_int64_t); /* Close the file and release most resources. */ __LA_DECL int archive_read_close(struct archive *); /* Release all resources and destroy the object. */ /* Note that archive_read_free will call archive_read_close for you. */ __LA_DECL int archive_read_free(struct archive *); #if ARCHIVE_VERSION_NUMBER < 4000000 /* Synonym for archive_read_free() for backwards compatibility. */ __LA_DECL int archive_read_finish(struct archive *) __LA_DEPRECATED; #endif /*- * To create an archive: * 1) Ask archive_write_new for an archive writer object. * 2) Set any global properties. In particular, you should set * the compression and format to use. * 3) Call archive_write_open to open the file (most people * will use archive_write_open_file or archive_write_open_fd, * which provide convenient canned I/O callbacks for you). * 4) For each entry: * - construct an appropriate struct archive_entry structure * - archive_write_header to write the header * - archive_write_data to write the entry data * 5) archive_write_close to close the output * 6) archive_write_free to cleanup the writer and release resources */ __LA_DECL struct archive *archive_write_new(void); __LA_DECL int archive_write_set_bytes_per_block(struct archive *, int bytes_per_block); __LA_DECL int archive_write_get_bytes_per_block(struct archive *); /* XXX This is badly misnamed; suggestions appreciated. XXX */ __LA_DECL int archive_write_set_bytes_in_last_block(struct archive *, int bytes_in_last_block); __LA_DECL int archive_write_get_bytes_in_last_block(struct archive *); /* The dev/ino of a file that won't be archived. This is used * to avoid recursively adding an archive to itself. */ __LA_DECL int archive_write_set_skip_file(struct archive *, la_int64_t, la_int64_t); #if ARCHIVE_VERSION_NUMBER < 4000000 __LA_DECL int archive_write_set_compression_bzip2(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_write_set_compression_compress(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_write_set_compression_gzip(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_write_set_compression_lzip(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_write_set_compression_lzma(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_write_set_compression_none(struct archive *) __LA_DEPRECATED; __LA_DECL int archive_write_set_compression_program(struct archive *, const char *cmd) __LA_DEPRECATED; __LA_DECL int archive_write_set_compression_xz(struct archive *) __LA_DEPRECATED; #endif /* A convenience function to set the filter based on the code. */ __LA_DECL int archive_write_add_filter(struct archive *, int filter_code); __LA_DECL int archive_write_add_filter_by_name(struct archive *, const char *name); __LA_DECL int archive_write_add_filter_b64encode(struct archive *); __LA_DECL int archive_write_add_filter_bzip2(struct archive *); __LA_DECL int archive_write_add_filter_compress(struct archive *); __LA_DECL int archive_write_add_filter_grzip(struct archive *); __LA_DECL int archive_write_add_filter_gzip(struct archive *); __LA_DECL int archive_write_add_filter_lrzip(struct archive *); __LA_DECL int archive_write_add_filter_lz4(struct archive *); __LA_DECL int archive_write_add_filter_lzip(struct archive *); __LA_DECL int archive_write_add_filter_lzma(struct archive *); __LA_DECL int archive_write_add_filter_lzop(struct archive *); __LA_DECL int archive_write_add_filter_none(struct archive *); __LA_DECL int archive_write_add_filter_program(struct archive *, const char *cmd); __LA_DECL int archive_write_add_filter_uuencode(struct archive *); __LA_DECL int archive_write_add_filter_xz(struct archive *); __LA_DECL int archive_write_add_filter_zstd(struct archive *); /* A convenience function to set the format based on the code or name. */ __LA_DECL int archive_write_set_format(struct archive *, int format_code); __LA_DECL int archive_write_set_format_by_name(struct archive *, const char *name); /* To minimize link pollution, use one or more of the following. */ __LA_DECL int archive_write_set_format_7zip(struct archive *); __LA_DECL int archive_write_set_format_ar_bsd(struct archive *); __LA_DECL int archive_write_set_format_ar_svr4(struct archive *); __LA_DECL int archive_write_set_format_cpio(struct archive *); __LA_DECL int archive_write_set_format_cpio_newc(struct archive *); __LA_DECL int archive_write_set_format_gnutar(struct archive *); __LA_DECL int archive_write_set_format_iso9660(struct archive *); __LA_DECL int archive_write_set_format_mtree(struct archive *); __LA_DECL int archive_write_set_format_mtree_classic(struct archive *); /* TODO: int archive_write_set_format_old_tar(struct archive *); */ __LA_DECL int archive_write_set_format_pax(struct archive *); __LA_DECL int archive_write_set_format_pax_restricted(struct archive *); __LA_DECL int archive_write_set_format_raw(struct archive *); __LA_DECL int archive_write_set_format_shar(struct archive *); __LA_DECL int archive_write_set_format_shar_dump(struct archive *); __LA_DECL int archive_write_set_format_ustar(struct archive *); __LA_DECL int archive_write_set_format_v7tar(struct archive *); __LA_DECL int archive_write_set_format_warc(struct archive *); __LA_DECL int archive_write_set_format_xar(struct archive *); __LA_DECL int archive_write_set_format_zip(struct archive *); __LA_DECL int archive_write_set_format_filter_by_ext(struct archive *a, const char *filename); __LA_DECL int archive_write_set_format_filter_by_ext_def(struct archive *a, const char *filename, const char * def_ext); __LA_DECL int archive_write_zip_set_compression_deflate(struct archive *); __LA_DECL int archive_write_zip_set_compression_store(struct archive *); __LA_DECL int archive_write_open(struct archive *, void *, archive_open_callback *, archive_write_callback *, archive_close_callback *); __LA_DECL int archive_write_open_fd(struct archive *, int _fd); __LA_DECL int archive_write_open_filename(struct archive *, const char *_file); __LA_DECL int archive_write_open_filename_w(struct archive *, const wchar_t *_file); /* A deprecated synonym for archive_write_open_filename() */ __LA_DECL int archive_write_open_file(struct archive *, const char *_file) __LA_DEPRECATED; __LA_DECL int archive_write_open_FILE(struct archive *, FILE *); /* _buffSize is the size of the buffer, _used refers to a variable that * will be updated after each write into the buffer. */ __LA_DECL int archive_write_open_memory(struct archive *, void *_buffer, size_t _buffSize, size_t *_used); /* * Note that the library will truncate writes beyond the size provided * to archive_write_header or pad if the provided data is short. */ __LA_DECL int archive_write_header(struct archive *, struct archive_entry *); __LA_DECL la_ssize_t archive_write_data(struct archive *, const void *, size_t); /* This interface is currently only available for archive_write_disk handles. */ __LA_DECL la_ssize_t archive_write_data_block(struct archive *, const void *, size_t, la_int64_t); __LA_DECL int archive_write_finish_entry(struct archive *); __LA_DECL int archive_write_close(struct archive *); /* Marks the archive as FATAL so that a subsequent free() operation * won't try to close() cleanly. Provides a fast abort capability * when the client discovers that things have gone wrong. */ __LA_DECL int archive_write_fail(struct archive *); /* This can fail if the archive wasn't already closed, in which case * archive_write_free() will implicitly call archive_write_close(). */ __LA_DECL int archive_write_free(struct archive *); #if ARCHIVE_VERSION_NUMBER < 4000000 /* Synonym for archive_write_free() for backwards compatibility. */ __LA_DECL int archive_write_finish(struct archive *) __LA_DEPRECATED; #endif /* * Set write options. */ /* Apply option to the format only. */ __LA_DECL int archive_write_set_format_option(struct archive *_a, const char *m, const char *o, const char *v); /* Apply option to the filter only. */ __LA_DECL int archive_write_set_filter_option(struct archive *_a, const char *m, const char *o, const char *v); /* Apply option to both the format and the filter. */ __LA_DECL int archive_write_set_option(struct archive *_a, const char *m, const char *o, const char *v); /* Apply option string to both the format and the filter. */ __LA_DECL int archive_write_set_options(struct archive *_a, const char *opts); /* * Set a encryption passphrase. */ __LA_DECL int archive_write_set_passphrase(struct archive *_a, const char *p); __LA_DECL int archive_write_set_passphrase_callback(struct archive *, void *client_data, archive_passphrase_callback *); /*- * ARCHIVE_WRITE_DISK API * * To create objects on disk: * 1) Ask archive_write_disk_new for a new archive_write_disk object. * 2) Set any global properties. In particular, you probably * want to set the options. * 3) For each entry: * - construct an appropriate struct archive_entry structure * - archive_write_header to create the file/dir/etc on disk * - archive_write_data to write the entry data * 4) archive_write_free to cleanup the writer and release resources * * In particular, you can use this in conjunction with archive_read() * to pull entries out of an archive and create them on disk. */ __LA_DECL struct archive *archive_write_disk_new(void); /* This file will not be overwritten. */ __LA_DECL int archive_write_disk_set_skip_file(struct archive *, la_int64_t, la_int64_t); /* Set flags to control how the next item gets created. * This accepts a bitmask of ARCHIVE_EXTRACT_XXX flags defined above. */ __LA_DECL int archive_write_disk_set_options(struct archive *, int flags); /* * The lookup functions are given uname/uid (or gname/gid) pairs and * return a uid (gid) suitable for this system. These are used for * restoring ownership and for setting ACLs. The default functions * are naive, they just return the uid/gid. These are small, so reasonable * for applications that don't need to preserve ownership; they * are probably also appropriate for applications that are doing * same-system backup and restore. */ /* * The "standard" lookup functions use common system calls to lookup * the uname/gname, falling back to the uid/gid if the names can't be * found. They cache lookups and are reasonably fast, but can be very * large, so they are not used unless you ask for them. In * particular, these match the specifications of POSIX "pax" and old * POSIX "tar". */ __LA_DECL int archive_write_disk_set_standard_lookup(struct archive *); /* * If neither the default (naive) nor the standard (big) functions suit * your needs, you can write your own and register them. Be sure to * include a cleanup function if you have allocated private data. */ __LA_DECL int archive_write_disk_set_group_lookup(struct archive *, void * /* private_data */, la_int64_t (*)(void *, const char *, la_int64_t), void (* /* cleanup */)(void *)); __LA_DECL int archive_write_disk_set_user_lookup(struct archive *, void * /* private_data */, la_int64_t (*)(void *, const char *, la_int64_t), void (* /* cleanup */)(void *)); __LA_DECL la_int64_t archive_write_disk_gid(struct archive *, const char *, la_int64_t); __LA_DECL la_int64_t archive_write_disk_uid(struct archive *, const char *, la_int64_t); /* * ARCHIVE_READ_DISK API * * This is still evolving and somewhat experimental. */ __LA_DECL struct archive *archive_read_disk_new(void); /* The names for symlink modes here correspond to an old BSD * command-line argument convention: -L, -P, -H */ /* Follow all symlinks. */ __LA_DECL int archive_read_disk_set_symlink_logical(struct archive *); /* Follow no symlinks. */ __LA_DECL int archive_read_disk_set_symlink_physical(struct archive *); /* Follow symlink initially, then not. */ __LA_DECL int archive_read_disk_set_symlink_hybrid(struct archive *); /* TODO: Handle Linux stat32/stat64 ugliness. */ __LA_DECL int archive_read_disk_entry_from_file(struct archive *, struct archive_entry *, int /* fd */, const struct stat *); /* Look up gname for gid or uname for uid. */ /* Default implementations are very, very stupid. */ __LA_DECL const char *archive_read_disk_gname(struct archive *, la_int64_t); __LA_DECL const char *archive_read_disk_uname(struct archive *, la_int64_t); /* "Standard" implementation uses getpwuid_r, getgrgid_r and caches the * results for performance. */ __LA_DECL int archive_read_disk_set_standard_lookup(struct archive *); /* You can install your own lookups if you like. */ __LA_DECL int archive_read_disk_set_gname_lookup(struct archive *, void * /* private_data */, const char *(* /* lookup_fn */)(void *, la_int64_t), void (* /* cleanup_fn */)(void *)); __LA_DECL int archive_read_disk_set_uname_lookup(struct archive *, void * /* private_data */, const char *(* /* lookup_fn */)(void *, la_int64_t), void (* /* cleanup_fn */)(void *)); /* Start traversal. */ __LA_DECL int archive_read_disk_open(struct archive *, const char *); __LA_DECL int archive_read_disk_open_w(struct archive *, const wchar_t *); /* * Request that current entry be visited. If you invoke it on every * directory, you'll get a physical traversal. This is ignored if the * current entry isn't a directory or a link to a directory. So, if * you invoke this on every returned path, you'll get a full logical * traversal. */ __LA_DECL int archive_read_disk_descend(struct archive *); __LA_DECL int archive_read_disk_can_descend(struct archive *); __LA_DECL int archive_read_disk_current_filesystem(struct archive *); __LA_DECL int archive_read_disk_current_filesystem_is_synthetic(struct archive *); __LA_DECL int archive_read_disk_current_filesystem_is_remote(struct archive *); /* Request that the access time of the entry visited by traversal be restored. */ __LA_DECL int archive_read_disk_set_atime_restored(struct archive *); /* * Set behavior. The "flags" argument selects optional behavior. */ /* Request that the access time of the entry visited by traversal be restored. * This is the same as archive_read_disk_set_atime_restored. */ #define ARCHIVE_READDISK_RESTORE_ATIME (0x0001) /* Default: Do not skip an entry which has nodump flags. */ #define ARCHIVE_READDISK_HONOR_NODUMP (0x0002) /* Default: Skip a mac resource fork file whose prefix is "._" because of * using copyfile. */ #define ARCHIVE_READDISK_MAC_COPYFILE (0x0004) /* Default: Traverse mount points. */ #define ARCHIVE_READDISK_NO_TRAVERSE_MOUNTS (0x0008) /* Default: Xattrs are read from disk. */ #define ARCHIVE_READDISK_NO_XATTR (0x0010) /* Default: ACLs are read from disk. */ #define ARCHIVE_READDISK_NO_ACL (0x0020) /* Default: File flags are read from disk. */ #define ARCHIVE_READDISK_NO_FFLAGS (0x0040) __LA_DECL int archive_read_disk_set_behavior(struct archive *, int flags); /* * Set archive_match object that will be used in archive_read_disk to * know whether an entry should be skipped. The callback function * _excluded_func will be invoked when an entry is skipped by the result * of archive_match. */ __LA_DECL int archive_read_disk_set_matching(struct archive *, struct archive *_matching, void (*_excluded_func) (struct archive *, void *, struct archive_entry *), void *_client_data); __LA_DECL int archive_read_disk_set_metadata_filter_callback(struct archive *, int (*_metadata_filter_func)(struct archive *, void *, struct archive_entry *), void *_client_data); /* Simplified cleanup interface; * This calls archive_read_free() or archive_write_free() as needed. */ __LA_DECL int archive_free(struct archive *); /* * Accessor functions to read/set various information in * the struct archive object: */ /* Number of filters in the current filter pipeline. */ /* Filter #0 is the one closest to the format, -1 is a synonym for the * last filter, which is always the pseudo-filter that wraps the * client callbacks. */ __LA_DECL int archive_filter_count(struct archive *); __LA_DECL la_int64_t archive_filter_bytes(struct archive *, int); __LA_DECL int archive_filter_code(struct archive *, int); __LA_DECL const char * archive_filter_name(struct archive *, int); #if ARCHIVE_VERSION_NUMBER < 4000000 /* These don't properly handle multiple filters, so are deprecated and * will eventually be removed. */ /* As of libarchive 3.0, this is an alias for archive_filter_bytes(a, -1); */ __LA_DECL la_int64_t archive_position_compressed(struct archive *) __LA_DEPRECATED; /* As of libarchive 3.0, this is an alias for archive_filter_bytes(a, 0); */ __LA_DECL la_int64_t archive_position_uncompressed(struct archive *) __LA_DEPRECATED; /* As of libarchive 3.0, this is an alias for archive_filter_name(a, 0); */ __LA_DECL const char *archive_compression_name(struct archive *) __LA_DEPRECATED; /* As of libarchive 3.0, this is an alias for archive_filter_code(a, 0); */ __LA_DECL int archive_compression(struct archive *) __LA_DEPRECATED; #endif __LA_DECL int archive_errno(struct archive *); __LA_DECL const char *archive_error_string(struct archive *); __LA_DECL const char *archive_format_name(struct archive *); __LA_DECL int archive_format(struct archive *); __LA_DECL void archive_clear_error(struct archive *); __LA_DECL void archive_set_error(struct archive *, int _err, const char *fmt, ...) __LA_PRINTF(3, 4); __LA_DECL void archive_copy_error(struct archive *dest, struct archive *src); __LA_DECL int archive_file_count(struct archive *); /* * ARCHIVE_MATCH API */ __LA_DECL struct archive *archive_match_new(void); __LA_DECL int archive_match_free(struct archive *); /* * Test if archive_entry is excluded. * This is a convenience function. This is the same as calling all * archive_match_path_excluded, archive_match_time_excluded * and archive_match_owner_excluded. */ __LA_DECL int archive_match_excluded(struct archive *, struct archive_entry *); /* * Test if pathname is excluded. The conditions are set by following functions. */ __LA_DECL int archive_match_path_excluded(struct archive *, struct archive_entry *); /* Add exclusion pathname pattern. */ __LA_DECL int archive_match_exclude_pattern(struct archive *, const char *); __LA_DECL int archive_match_exclude_pattern_w(struct archive *, const wchar_t *); /* Add exclusion pathname pattern from file. */ __LA_DECL int archive_match_exclude_pattern_from_file(struct archive *, const char *, int _nullSeparator); __LA_DECL int archive_match_exclude_pattern_from_file_w(struct archive *, const wchar_t *, int _nullSeparator); /* Add inclusion pathname pattern. */ __LA_DECL int archive_match_include_pattern(struct archive *, const char *); __LA_DECL int archive_match_include_pattern_w(struct archive *, const wchar_t *); /* Add inclusion pathname pattern from file. */ __LA_DECL int archive_match_include_pattern_from_file(struct archive *, const char *, int _nullSeparator); __LA_DECL int archive_match_include_pattern_from_file_w(struct archive *, const wchar_t *, int _nullSeparator); /* * How to get statistic information for inclusion patterns. */ /* Return the amount number of unmatched inclusion patterns. */ __LA_DECL int archive_match_path_unmatched_inclusions(struct archive *); /* Return the pattern of unmatched inclusion with ARCHIVE_OK. * Return ARCHIVE_EOF if there is no inclusion pattern. */ __LA_DECL int archive_match_path_unmatched_inclusions_next( struct archive *, const char **); __LA_DECL int archive_match_path_unmatched_inclusions_next_w( struct archive *, const wchar_t **); /* * Test if a file is excluded by its time stamp. * The conditions are set by following functions. */ __LA_DECL int archive_match_time_excluded(struct archive *, struct archive_entry *); /* * Flags to tell a matching type of time stamps. These are used for * following functions. */ /* Time flag: mtime to be tested. */ #define ARCHIVE_MATCH_MTIME (0x0100) /* Time flag: ctime to be tested. */ #define ARCHIVE_MATCH_CTIME (0x0200) /* Comparison flag: Match the time if it is newer than. */ #define ARCHIVE_MATCH_NEWER (0x0001) /* Comparison flag: Match the time if it is older than. */ #define ARCHIVE_MATCH_OLDER (0x0002) /* Comparison flag: Match the time if it is equal to. */ #define ARCHIVE_MATCH_EQUAL (0x0010) /* Set inclusion time. */ __LA_DECL int archive_match_include_time(struct archive *, int _flag, time_t _sec, long _nsec); /* Set inclusion time by a date string. */ __LA_DECL int archive_match_include_date(struct archive *, int _flag, const char *_datestr); __LA_DECL int archive_match_include_date_w(struct archive *, int _flag, const wchar_t *_datestr); /* Set inclusion time by a particular file. */ __LA_DECL int archive_match_include_file_time(struct archive *, int _flag, const char *_pathname); __LA_DECL int archive_match_include_file_time_w(struct archive *, int _flag, const wchar_t *_pathname); /* Add exclusion entry. */ __LA_DECL int archive_match_exclude_entry(struct archive *, int _flag, struct archive_entry *); /* * Test if a file is excluded by its uid ,gid, uname or gname. * The conditions are set by following functions. */ __LA_DECL int archive_match_owner_excluded(struct archive *, struct archive_entry *); /* Add inclusion uid, gid, uname and gname. */ __LA_DECL int archive_match_include_uid(struct archive *, la_int64_t); __LA_DECL int archive_match_include_gid(struct archive *, la_int64_t); __LA_DECL int archive_match_include_uname(struct archive *, const char *); __LA_DECL int archive_match_include_uname_w(struct archive *, const wchar_t *); __LA_DECL int archive_match_include_gname(struct archive *, const char *); __LA_DECL int archive_match_include_gname_w(struct archive *, const wchar_t *); /* Utility functions */ /* Convenience function to sort a NULL terminated list of strings */ __LA_DECL int archive_utility_string_sort(char **); #ifdef __cplusplus } #endif /* These are meaningless outside of this header. */ #undef __LA_DECL #endif /* !ARCHIVE_H_INCLUDED */ Index: projects/clang700-import/contrib/libarchive/libarchive/archive_cryptor.c =================================================================== --- projects/clang700-import/contrib/libarchive/libarchive/archive_cryptor.c (revision 338689) +++ projects/clang700-import/contrib/libarchive/libarchive/archive_cryptor.c (revision 338690) @@ -1,450 +1,450 @@ /*- * Copyright (c) 2014 Michihiro NAKAJIMA * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "archive_platform.h" #ifdef HAVE_STRING_H #include #endif #include "archive.h" #include "archive_cryptor_private.h" /* * On systems that do not support any recognized crypto libraries, * this file will normally define no usable symbols. * * But some compilers and linkers choke on empty object files, so * define a public symbol that will always exist. This could * be removed someday if this file gains another always-present * symbol definition. */ int __libarchive_cryptor_build_hack(void) { return 0; } #ifdef ARCHIVE_CRYPTOR_USE_Apple_CommonCrypto static int pbkdf2_sha1(const char *pw, size_t pw_len, const uint8_t *salt, size_t salt_len, unsigned rounds, uint8_t *derived_key, size_t derived_key_len) { CCKeyDerivationPBKDF(kCCPBKDF2, (const char *)pw, pw_len, salt, salt_len, kCCPRFHmacAlgSHA1, rounds, derived_key, derived_key_len); return 0; } #elif defined(_WIN32) && !defined(__CYGWIN__) && defined(HAVE_BCRYPT_H) #ifdef _MSC_VER #pragma comment(lib, "Bcrypt.lib") #endif static int pbkdf2_sha1(const char *pw, size_t pw_len, const uint8_t *salt, size_t salt_len, unsigned rounds, uint8_t *derived_key, size_t derived_key_len) { NTSTATUS status; BCRYPT_ALG_HANDLE hAlg; status = BCryptOpenAlgorithmProvider(&hAlg, BCRYPT_SHA1_ALGORITHM, MS_PRIMITIVE_PROVIDER, BCRYPT_ALG_HANDLE_HMAC_FLAG); if (!BCRYPT_SUCCESS(status)) return -1; status = BCryptDeriveKeyPBKDF2(hAlg, (PUCHAR)(uintptr_t)pw, (ULONG)pw_len, (PUCHAR)(uintptr_t)salt, (ULONG)salt_len, rounds, (PUCHAR)derived_key, (ULONG)derived_key_len, 0); BCryptCloseAlgorithmProvider(hAlg, 0); return (BCRYPT_SUCCESS(status)) ? 0: -1; } #elif defined(HAVE_LIBNETTLE) && defined(HAVE_NETTLE_PBKDF2_H) static int pbkdf2_sha1(const char *pw, size_t pw_len, const uint8_t *salt, size_t salt_len, unsigned rounds, uint8_t *derived_key, size_t derived_key_len) { pbkdf2_hmac_sha1((unsigned)pw_len, (const uint8_t *)pw, rounds, salt_len, salt, derived_key_len, derived_key); return 0; } #elif defined(HAVE_LIBCRYPTO) && defined(HAVE_PKCS5_PBKDF2_HMAC_SHA1) static int pbkdf2_sha1(const char *pw, size_t pw_len, const uint8_t *salt, size_t salt_len, unsigned rounds, uint8_t *derived_key, size_t derived_key_len) { PKCS5_PBKDF2_HMAC_SHA1(pw, pw_len, salt, salt_len, rounds, derived_key_len, derived_key); return 0; } #else /* Stub */ static int pbkdf2_sha1(const char *pw, size_t pw_len, const uint8_t *salt, size_t salt_len, unsigned rounds, uint8_t *derived_key, size_t derived_key_len) { (void)pw; /* UNUSED */ (void)pw_len; /* UNUSED */ (void)salt; /* UNUSED */ (void)salt_len; /* UNUSED */ (void)rounds; /* UNUSED */ (void)derived_key; /* UNUSED */ (void)derived_key_len; /* UNUSED */ return -1; /* UNSUPPORTED */ } #endif #ifdef ARCHIVE_CRYPTOR_USE_Apple_CommonCrypto # if MAC_OS_X_VERSION_MAX_ALLOWED < 1090 # define kCCAlgorithmAES kCCAlgorithmAES128 # endif static int aes_ctr_init(archive_crypto_ctx *ctx, const uint8_t *key, size_t key_len) { CCCryptorStatus r; ctx->key_len = key_len; memcpy(ctx->key, key, key_len); memset(ctx->nonce, 0, sizeof(ctx->nonce)); ctx->encr_pos = AES_BLOCK_SIZE; r = CCCryptorCreateWithMode(kCCEncrypt, kCCModeECB, kCCAlgorithmAES, ccNoPadding, NULL, key, key_len, NULL, 0, 0, 0, &ctx->ctx); return (r == kCCSuccess)? 0: -1; } static int aes_ctr_encrypt_counter(archive_crypto_ctx *ctx) { CCCryptorRef ref = ctx->ctx; CCCryptorStatus r; r = CCCryptorReset(ref, NULL); - if (r != kCCSuccess) + if (r != kCCSuccess && r != kCCUnimplemented) return -1; r = CCCryptorUpdate(ref, ctx->nonce, AES_BLOCK_SIZE, ctx->encr_buf, AES_BLOCK_SIZE, NULL); return (r == kCCSuccess)? 0: -1; } static int aes_ctr_release(archive_crypto_ctx *ctx) { memset(ctx->key, 0, ctx->key_len); memset(ctx->nonce, 0, sizeof(ctx->nonce)); return 0; } #elif defined(_WIN32) && !defined(__CYGWIN__) && defined(HAVE_BCRYPT_H) static int aes_ctr_init(archive_crypto_ctx *ctx, const uint8_t *key, size_t key_len) { BCRYPT_ALG_HANDLE hAlg; BCRYPT_KEY_HANDLE hKey; DWORD keyObj_len, aes_key_len; PBYTE keyObj; ULONG result; NTSTATUS status; BCRYPT_KEY_LENGTHS_STRUCT key_lengths; ctx->hAlg = NULL; ctx->hKey = NULL; ctx->keyObj = NULL; switch (key_len) { case 16: aes_key_len = 128; break; case 24: aes_key_len = 192; break; case 32: aes_key_len = 256; break; default: return -1; } status = BCryptOpenAlgorithmProvider(&hAlg, BCRYPT_AES_ALGORITHM, MS_PRIMITIVE_PROVIDER, 0); if (!BCRYPT_SUCCESS(status)) return -1; status = BCryptGetProperty(hAlg, BCRYPT_KEY_LENGTHS, (PUCHAR)&key_lengths, sizeof(key_lengths), &result, 0); if (!BCRYPT_SUCCESS(status)) { BCryptCloseAlgorithmProvider(hAlg, 0); return -1; } if (key_lengths.dwMinLength > aes_key_len || key_lengths.dwMaxLength < aes_key_len) { BCryptCloseAlgorithmProvider(hAlg, 0); return -1; } status = BCryptGetProperty(hAlg, BCRYPT_OBJECT_LENGTH, (PUCHAR)&keyObj_len, sizeof(keyObj_len), &result, 0); if (!BCRYPT_SUCCESS(status)) { BCryptCloseAlgorithmProvider(hAlg, 0); return -1; } keyObj = (PBYTE)HeapAlloc(GetProcessHeap(), 0, keyObj_len); if (keyObj == NULL) { BCryptCloseAlgorithmProvider(hAlg, 0); return -1; } status = BCryptSetProperty(hAlg, BCRYPT_CHAINING_MODE, (PUCHAR)BCRYPT_CHAIN_MODE_ECB, sizeof(BCRYPT_CHAIN_MODE_ECB), 0); if (!BCRYPT_SUCCESS(status)) { BCryptCloseAlgorithmProvider(hAlg, 0); HeapFree(GetProcessHeap(), 0, keyObj); return -1; } status = BCryptGenerateSymmetricKey(hAlg, &hKey, keyObj, keyObj_len, (PUCHAR)(uintptr_t)key, (ULONG)key_len, 0); if (!BCRYPT_SUCCESS(status)) { BCryptCloseAlgorithmProvider(hAlg, 0); HeapFree(GetProcessHeap(), 0, keyObj); return -1; } ctx->hAlg = hAlg; ctx->hKey = hKey; ctx->keyObj = keyObj; ctx->keyObj_len = keyObj_len; ctx->encr_pos = AES_BLOCK_SIZE; return 0; } static int aes_ctr_encrypt_counter(archive_crypto_ctx *ctx) { NTSTATUS status; ULONG result; status = BCryptEncrypt(ctx->hKey, (PUCHAR)ctx->nonce, AES_BLOCK_SIZE, NULL, NULL, 0, (PUCHAR)ctx->encr_buf, AES_BLOCK_SIZE, &result, 0); return BCRYPT_SUCCESS(status) ? 0 : -1; } static int aes_ctr_release(archive_crypto_ctx *ctx) { if (ctx->hAlg != NULL) { BCryptCloseAlgorithmProvider(ctx->hAlg, 0); ctx->hAlg = NULL; BCryptDestroyKey(ctx->hKey); ctx->hKey = NULL; HeapFree(GetProcessHeap(), 0, ctx->keyObj); ctx->keyObj = NULL; } memset(ctx, 0, sizeof(*ctx)); return 0; } #elif defined(HAVE_LIBNETTLE) && defined(HAVE_NETTLE_AES_H) static int aes_ctr_init(archive_crypto_ctx *ctx, const uint8_t *key, size_t key_len) { ctx->key_len = key_len; memcpy(ctx->key, key, key_len); memset(ctx->nonce, 0, sizeof(ctx->nonce)); ctx->encr_pos = AES_BLOCK_SIZE; memset(&ctx->ctx, 0, sizeof(ctx->ctx)); return 0; } static int aes_ctr_encrypt_counter(archive_crypto_ctx *ctx) { aes_set_encrypt_key(&ctx->ctx, ctx->key_len, ctx->key); aes_encrypt(&ctx->ctx, AES_BLOCK_SIZE, ctx->encr_buf, ctx->nonce); return 0; } static int aes_ctr_release(archive_crypto_ctx *ctx) { memset(ctx, 0, sizeof(*ctx)); return 0; } #elif defined(HAVE_LIBCRYPTO) static int aes_ctr_init(archive_crypto_ctx *ctx, const uint8_t *key, size_t key_len) { if ((ctx->ctx = EVP_CIPHER_CTX_new()) == NULL) return -1; switch (key_len) { case 16: ctx->type = EVP_aes_128_ecb(); break; case 24: ctx->type = EVP_aes_192_ecb(); break; case 32: ctx->type = EVP_aes_256_ecb(); break; default: ctx->type = NULL; return -1; } ctx->key_len = key_len; memcpy(ctx->key, key, key_len); memset(ctx->nonce, 0, sizeof(ctx->nonce)); ctx->encr_pos = AES_BLOCK_SIZE; EVP_CIPHER_CTX_init(ctx->ctx); return 0; } static int aes_ctr_encrypt_counter(archive_crypto_ctx *ctx) { int outl = 0; int r; r = EVP_EncryptInit_ex(ctx->ctx, ctx->type, NULL, ctx->key, NULL); if (r == 0) return -1; r = EVP_EncryptUpdate(ctx->ctx, ctx->encr_buf, &outl, ctx->nonce, AES_BLOCK_SIZE); if (r == 0 || outl != AES_BLOCK_SIZE) return -1; return 0; } static int aes_ctr_release(archive_crypto_ctx *ctx) { EVP_CIPHER_CTX_free(ctx->ctx); memset(ctx->key, 0, ctx->key_len); memset(ctx->nonce, 0, sizeof(ctx->nonce)); return 0; } #else #define ARCHIVE_CRYPTOR_STUB /* Stub */ static int aes_ctr_init(archive_crypto_ctx *ctx, const uint8_t *key, size_t key_len) { (void)ctx; /* UNUSED */ (void)key; /* UNUSED */ (void)key_len; /* UNUSED */ return -1; } static int aes_ctr_encrypt_counter(archive_crypto_ctx *ctx) { (void)ctx; /* UNUSED */ return -1; } static int aes_ctr_release(archive_crypto_ctx *ctx) { (void)ctx; /* UNUSED */ return 0; } #endif #ifdef ARCHIVE_CRYPTOR_STUB static int aes_ctr_update(archive_crypto_ctx *ctx, const uint8_t * const in, size_t in_len, uint8_t * const out, size_t *out_len) { (void)ctx; /* UNUSED */ (void)in; /* UNUSED */ (void)in_len; /* UNUSED */ (void)out; /* UNUSED */ (void)out_len; /* UNUSED */ aes_ctr_encrypt_counter(ctx); /* UNUSED */ /* Fix unused function warning */ return -1; } #else static void aes_ctr_increase_counter(archive_crypto_ctx *ctx) { uint8_t *const nonce = ctx->nonce; int j; for (j = 0; j < 8; j++) { if (++nonce[j]) break; } } static int aes_ctr_update(archive_crypto_ctx *ctx, const uint8_t * const in, size_t in_len, uint8_t * const out, size_t *out_len) { uint8_t *const ebuf = ctx->encr_buf; unsigned pos = ctx->encr_pos; unsigned max = (unsigned)((in_len < *out_len)? in_len: *out_len); unsigned i; for (i = 0; i < max; ) { if (pos == AES_BLOCK_SIZE) { aes_ctr_increase_counter(ctx); if (aes_ctr_encrypt_counter(ctx) != 0) return -1; while (max -i >= AES_BLOCK_SIZE) { for (pos = 0; pos < AES_BLOCK_SIZE; pos++) out[i+pos] = in[i+pos] ^ ebuf[pos]; i += AES_BLOCK_SIZE; aes_ctr_increase_counter(ctx); if (aes_ctr_encrypt_counter(ctx) != 0) return -1; } pos = 0; if (i >= max) break; } out[i] = in[i] ^ ebuf[pos++]; i++; } ctx->encr_pos = pos; *out_len = i; return 0; } #endif /* ARCHIVE_CRYPTOR_STUB */ const struct archive_cryptor __archive_cryptor = { &pbkdf2_sha1, &aes_ctr_init, &aes_ctr_update, &aes_ctr_release, &aes_ctr_init, &aes_ctr_update, &aes_ctr_release, }; Index: projects/clang700-import/contrib/libarchive/libarchive/archive_entry.h =================================================================== --- projects/clang700-import/contrib/libarchive/libarchive/archive_entry.h (revision 338689) +++ projects/clang700-import/contrib/libarchive/libarchive/archive_entry.h (revision 338690) @@ -1,703 +1,703 @@ /*- * Copyright (c) 2003-2008 Tim Kientzle * Copyright (c) 2016 Martin Matuska * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * $FreeBSD$ */ #ifndef ARCHIVE_ENTRY_H_INCLUDED #define ARCHIVE_ENTRY_H_INCLUDED /* Note: Compiler will complain if this does not match archive.h! */ -#define ARCHIVE_VERSION_NUMBER 3003002 +#define ARCHIVE_VERSION_NUMBER 3003003 /* * Note: archive_entry.h is for use outside of libarchive; the * configuration headers (config.h, archive_platform.h, etc.) are * purely internal. Do NOT use HAVE_XXX configuration macros to * control the behavior of this header! If you must conditionalize, * use predefined compiler and/or platform macros. */ #include #include /* for wchar_t */ #include #include #if defined(_WIN32) && !defined(__CYGWIN__) #include #endif /* Get a suitable 64-bit integer type. */ #if !defined(__LA_INT64_T_DEFINED) # if ARCHIVE_VERSION_NUMBER < 4000000 #define __LA_INT64_T la_int64_t # endif #define __LA_INT64_T_DEFINED # if defined(_WIN32) && !defined(__CYGWIN__) && !defined(__WATCOMC__) typedef __int64 la_int64_t; # else #include # if defined(_SCO_DS) || defined(__osf__) typedef long long la_int64_t; # else typedef int64_t la_int64_t; # endif # endif #endif /* The la_ssize_t should match the type used in 'struct stat' */ #if !defined(__LA_SSIZE_T_DEFINED) /* Older code relied on the __LA_SSIZE_T macro; after 4.0 we'll switch to the typedef exclusively. */ # if ARCHIVE_VERSION_NUMBER < 4000000 #define __LA_SSIZE_T la_ssize_t # endif #define __LA_SSIZE_T_DEFINED # if defined(_WIN32) && !defined(__CYGWIN__) && !defined(__WATCOMC__) # if defined(_SSIZE_T_DEFINED) || defined(_SSIZE_T_) typedef ssize_t la_ssize_t; # elif defined(_WIN64) typedef __int64 la_ssize_t; # else typedef long la_ssize_t; # endif # else # include /* ssize_t */ typedef ssize_t la_ssize_t; # endif #endif /* Get a suitable definition for mode_t */ #if ARCHIVE_VERSION_NUMBER >= 3999000 /* Switch to plain 'int' for libarchive 4.0. It's less broken than 'mode_t' */ # define __LA_MODE_T int #elif defined(_WIN32) && !defined(__CYGWIN__) && !defined(__BORLANDC__) && !defined(__WATCOMC__) # define __LA_MODE_T unsigned short #else # define __LA_MODE_T mode_t #endif /* Large file support for Android */ #ifdef __ANDROID__ #include "android_lf.h" #endif /* * On Windows, define LIBARCHIVE_STATIC if you're building or using a * .lib. The default here assumes you're building a DLL. Only * libarchive source should ever define __LIBARCHIVE_BUILD. */ #if ((defined __WIN32__) || (defined _WIN32) || defined(__CYGWIN__)) && (!defined LIBARCHIVE_STATIC) # ifdef __LIBARCHIVE_BUILD # ifdef __GNUC__ # define __LA_DECL __attribute__((dllexport)) extern # else # define __LA_DECL __declspec(dllexport) # endif # else # ifdef __GNUC__ # define __LA_DECL # else # define __LA_DECL __declspec(dllimport) # endif # endif #else /* Static libraries on all platforms and shared libraries on non-Windows. */ # define __LA_DECL #endif #if defined(__GNUC__) && __GNUC__ >= 3 && __GNUC_MINOR__ >= 1 # define __LA_DEPRECATED __attribute__((deprecated)) #else # define __LA_DEPRECATED #endif #ifdef __cplusplus extern "C" { #endif /* * Description of an archive entry. * * You can think of this as "struct stat" with some text fields added in. * * TODO: Add "comment", "charset", and possibly other entries that are * supported by "pax interchange" format. However, GNU, ustar, cpio, * and other variants don't support these features, so they're not an * excruciatingly high priority right now. * * TODO: "pax interchange" format allows essentially arbitrary * key/value attributes to be attached to any entry. Supporting * such extensions may make this library useful for special * applications (e.g., a package manager could attach special * package-management attributes to each entry). */ struct archive; struct archive_entry; /* * File-type constants. These are returned from archive_entry_filetype() * and passed to archive_entry_set_filetype(). * * These values match S_XXX defines on every platform I've checked, * including Windows, AIX, Linux, Solaris, and BSD. They're * (re)defined here because platforms generally don't define the ones * they don't support. For example, Windows doesn't define S_IFLNK or * S_IFBLK. Instead of having a mass of conditional logic and system * checks to define any S_XXX values that aren't supported locally, * I've just defined a new set of such constants so that * libarchive-based applications can manipulate and identify archive * entries properly even if the hosting platform can't store them on * disk. * * These values are also used directly within some portable formats, * such as cpio. If you find a platform that varies from these, the * correct solution is to leave these alone and translate from these * portable values to platform-native values when entries are read from * or written to disk. */ /* * In libarchive 4.0, we can drop the casts here. * They're needed to work around Borland C's broken mode_t. */ #define AE_IFMT ((__LA_MODE_T)0170000) #define AE_IFREG ((__LA_MODE_T)0100000) #define AE_IFLNK ((__LA_MODE_T)0120000) #define AE_IFSOCK ((__LA_MODE_T)0140000) #define AE_IFCHR ((__LA_MODE_T)0020000) #define AE_IFBLK ((__LA_MODE_T)0060000) #define AE_IFDIR ((__LA_MODE_T)0040000) #define AE_IFIFO ((__LA_MODE_T)0010000) /* * Basic object manipulation */ __LA_DECL struct archive_entry *archive_entry_clear(struct archive_entry *); /* The 'clone' function does a deep copy; all of the strings are copied too. */ __LA_DECL struct archive_entry *archive_entry_clone(struct archive_entry *); __LA_DECL void archive_entry_free(struct archive_entry *); __LA_DECL struct archive_entry *archive_entry_new(void); /* * This form of archive_entry_new2() will pull character-set * conversion information from the specified archive handle. The * older archive_entry_new(void) form is equivalent to calling * archive_entry_new2(NULL) and will result in the use of an internal * default character-set conversion. */ __LA_DECL struct archive_entry *archive_entry_new2(struct archive *); /* * Retrieve fields from an archive_entry. * * There are a number of implicit conversions among these fields. For * example, if a regular string field is set and you read the _w wide * character field, the entry will implicitly convert narrow-to-wide * using the current locale. Similarly, dev values are automatically * updated when you write devmajor or devminor and vice versa. * * In addition, fields can be "set" or "unset." Unset string fields * return NULL, non-string fields have _is_set() functions to test * whether they've been set. You can "unset" a string field by * assigning NULL; non-string fields have _unset() functions to * unset them. * * Note: There is one ambiguity in the above; string fields will * also return NULL when implicit character set conversions fail. * This is usually what you want. */ __LA_DECL time_t archive_entry_atime(struct archive_entry *); __LA_DECL long archive_entry_atime_nsec(struct archive_entry *); __LA_DECL int archive_entry_atime_is_set(struct archive_entry *); __LA_DECL time_t archive_entry_birthtime(struct archive_entry *); __LA_DECL long archive_entry_birthtime_nsec(struct archive_entry *); __LA_DECL int archive_entry_birthtime_is_set(struct archive_entry *); __LA_DECL time_t archive_entry_ctime(struct archive_entry *); __LA_DECL long archive_entry_ctime_nsec(struct archive_entry *); __LA_DECL int archive_entry_ctime_is_set(struct archive_entry *); __LA_DECL dev_t archive_entry_dev(struct archive_entry *); __LA_DECL int archive_entry_dev_is_set(struct archive_entry *); __LA_DECL dev_t archive_entry_devmajor(struct archive_entry *); __LA_DECL dev_t archive_entry_devminor(struct archive_entry *); __LA_DECL __LA_MODE_T archive_entry_filetype(struct archive_entry *); __LA_DECL void archive_entry_fflags(struct archive_entry *, unsigned long * /* set */, unsigned long * /* clear */); __LA_DECL const char *archive_entry_fflags_text(struct archive_entry *); __LA_DECL la_int64_t archive_entry_gid(struct archive_entry *); __LA_DECL const char *archive_entry_gname(struct archive_entry *); __LA_DECL const char *archive_entry_gname_utf8(struct archive_entry *); __LA_DECL const wchar_t *archive_entry_gname_w(struct archive_entry *); __LA_DECL const char *archive_entry_hardlink(struct archive_entry *); __LA_DECL const char *archive_entry_hardlink_utf8(struct archive_entry *); __LA_DECL const wchar_t *archive_entry_hardlink_w(struct archive_entry *); __LA_DECL la_int64_t archive_entry_ino(struct archive_entry *); __LA_DECL la_int64_t archive_entry_ino64(struct archive_entry *); __LA_DECL int archive_entry_ino_is_set(struct archive_entry *); __LA_DECL __LA_MODE_T archive_entry_mode(struct archive_entry *); __LA_DECL time_t archive_entry_mtime(struct archive_entry *); __LA_DECL long archive_entry_mtime_nsec(struct archive_entry *); __LA_DECL int archive_entry_mtime_is_set(struct archive_entry *); __LA_DECL unsigned int archive_entry_nlink(struct archive_entry *); __LA_DECL const char *archive_entry_pathname(struct archive_entry *); __LA_DECL const char *archive_entry_pathname_utf8(struct archive_entry *); __LA_DECL const wchar_t *archive_entry_pathname_w(struct archive_entry *); __LA_DECL __LA_MODE_T archive_entry_perm(struct archive_entry *); __LA_DECL dev_t archive_entry_rdev(struct archive_entry *); __LA_DECL dev_t archive_entry_rdevmajor(struct archive_entry *); __LA_DECL dev_t archive_entry_rdevminor(struct archive_entry *); __LA_DECL const char *archive_entry_sourcepath(struct archive_entry *); __LA_DECL const wchar_t *archive_entry_sourcepath_w(struct archive_entry *); __LA_DECL la_int64_t archive_entry_size(struct archive_entry *); __LA_DECL int archive_entry_size_is_set(struct archive_entry *); __LA_DECL const char *archive_entry_strmode(struct archive_entry *); __LA_DECL const char *archive_entry_symlink(struct archive_entry *); __LA_DECL const char *archive_entry_symlink_utf8(struct archive_entry *); __LA_DECL const wchar_t *archive_entry_symlink_w(struct archive_entry *); __LA_DECL la_int64_t archive_entry_uid(struct archive_entry *); __LA_DECL const char *archive_entry_uname(struct archive_entry *); __LA_DECL const char *archive_entry_uname_utf8(struct archive_entry *); __LA_DECL const wchar_t *archive_entry_uname_w(struct archive_entry *); __LA_DECL int archive_entry_is_data_encrypted(struct archive_entry *); __LA_DECL int archive_entry_is_metadata_encrypted(struct archive_entry *); __LA_DECL int archive_entry_is_encrypted(struct archive_entry *); /* * Set fields in an archive_entry. * * Note: Before libarchive 2.4, there were 'set' and 'copy' versions * of the string setters. 'copy' copied the actual string, 'set' just * stored the pointer. In libarchive 2.4 and later, strings are * always copied. */ __LA_DECL void archive_entry_set_atime(struct archive_entry *, time_t, long); __LA_DECL void archive_entry_unset_atime(struct archive_entry *); #if defined(_WIN32) && !defined(__CYGWIN__) __LA_DECL void archive_entry_copy_bhfi(struct archive_entry *, BY_HANDLE_FILE_INFORMATION *); #endif __LA_DECL void archive_entry_set_birthtime(struct archive_entry *, time_t, long); __LA_DECL void archive_entry_unset_birthtime(struct archive_entry *); __LA_DECL void archive_entry_set_ctime(struct archive_entry *, time_t, long); __LA_DECL void archive_entry_unset_ctime(struct archive_entry *); __LA_DECL void archive_entry_set_dev(struct archive_entry *, dev_t); __LA_DECL void archive_entry_set_devmajor(struct archive_entry *, dev_t); __LA_DECL void archive_entry_set_devminor(struct archive_entry *, dev_t); __LA_DECL void archive_entry_set_filetype(struct archive_entry *, unsigned int); __LA_DECL void archive_entry_set_fflags(struct archive_entry *, unsigned long /* set */, unsigned long /* clear */); /* Returns pointer to start of first invalid token, or NULL if none. */ /* Note that all recognized tokens are processed, regardless. */ __LA_DECL const char *archive_entry_copy_fflags_text(struct archive_entry *, const char *); __LA_DECL const wchar_t *archive_entry_copy_fflags_text_w(struct archive_entry *, const wchar_t *); __LA_DECL void archive_entry_set_gid(struct archive_entry *, la_int64_t); __LA_DECL void archive_entry_set_gname(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_gname_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_gname(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_gname_w(struct archive_entry *, const wchar_t *); __LA_DECL int archive_entry_update_gname_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_hardlink(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_hardlink_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_hardlink(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_hardlink_w(struct archive_entry *, const wchar_t *); __LA_DECL int archive_entry_update_hardlink_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_ino(struct archive_entry *, la_int64_t); __LA_DECL void archive_entry_set_ino64(struct archive_entry *, la_int64_t); __LA_DECL void archive_entry_set_link(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_link_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_link(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_link_w(struct archive_entry *, const wchar_t *); __LA_DECL int archive_entry_update_link_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_mode(struct archive_entry *, __LA_MODE_T); __LA_DECL void archive_entry_set_mtime(struct archive_entry *, time_t, long); __LA_DECL void archive_entry_unset_mtime(struct archive_entry *); __LA_DECL void archive_entry_set_nlink(struct archive_entry *, unsigned int); __LA_DECL void archive_entry_set_pathname(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_pathname_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_pathname(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_pathname_w(struct archive_entry *, const wchar_t *); __LA_DECL int archive_entry_update_pathname_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_perm(struct archive_entry *, __LA_MODE_T); __LA_DECL void archive_entry_set_rdev(struct archive_entry *, dev_t); __LA_DECL void archive_entry_set_rdevmajor(struct archive_entry *, dev_t); __LA_DECL void archive_entry_set_rdevminor(struct archive_entry *, dev_t); __LA_DECL void archive_entry_set_size(struct archive_entry *, la_int64_t); __LA_DECL void archive_entry_unset_size(struct archive_entry *); __LA_DECL void archive_entry_copy_sourcepath(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_sourcepath_w(struct archive_entry *, const wchar_t *); __LA_DECL void archive_entry_set_symlink(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_symlink_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_symlink(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_symlink_w(struct archive_entry *, const wchar_t *); __LA_DECL int archive_entry_update_symlink_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_uid(struct archive_entry *, la_int64_t); __LA_DECL void archive_entry_set_uname(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_uname_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_uname(struct archive_entry *, const char *); __LA_DECL void archive_entry_copy_uname_w(struct archive_entry *, const wchar_t *); __LA_DECL int archive_entry_update_uname_utf8(struct archive_entry *, const char *); __LA_DECL void archive_entry_set_is_data_encrypted(struct archive_entry *, char is_encrypted); __LA_DECL void archive_entry_set_is_metadata_encrypted(struct archive_entry *, char is_encrypted); /* * Routines to bulk copy fields to/from a platform-native "struct * stat." Libarchive used to just store a struct stat inside of each * archive_entry object, but this created issues when trying to * manipulate archives on systems different than the ones they were * created on. * * TODO: On Linux and other LFS systems, provide both stat32 and * stat64 versions of these functions and all of the macro glue so * that archive_entry_stat is magically defined to * archive_entry_stat32 or archive_entry_stat64 as appropriate. */ __LA_DECL const struct stat *archive_entry_stat(struct archive_entry *); __LA_DECL void archive_entry_copy_stat(struct archive_entry *, const struct stat *); /* * Storage for Mac OS-specific AppleDouble metadata information. * Apple-format tar files store a separate binary blob containing * encoded metadata with ACL, extended attributes, etc. * This provides a place to store that blob. */ __LA_DECL const void * archive_entry_mac_metadata(struct archive_entry *, size_t *); __LA_DECL void archive_entry_copy_mac_metadata(struct archive_entry *, const void *, size_t); /* * ACL routines. This used to simply store and return text-format ACL * strings, but that proved insufficient for a number of reasons: * = clients need control over uname/uid and gname/gid mappings * = there are many different ACL text formats * = would like to be able to read/convert archives containing ACLs * on platforms that lack ACL libraries * * This last point, in particular, forces me to implement a reasonably * complete set of ACL support routines. */ /* * Permission bits. */ #define ARCHIVE_ENTRY_ACL_EXECUTE 0x00000001 #define ARCHIVE_ENTRY_ACL_WRITE 0x00000002 #define ARCHIVE_ENTRY_ACL_READ 0x00000004 #define ARCHIVE_ENTRY_ACL_READ_DATA 0x00000008 #define ARCHIVE_ENTRY_ACL_LIST_DIRECTORY 0x00000008 #define ARCHIVE_ENTRY_ACL_WRITE_DATA 0x00000010 #define ARCHIVE_ENTRY_ACL_ADD_FILE 0x00000010 #define ARCHIVE_ENTRY_ACL_APPEND_DATA 0x00000020 #define ARCHIVE_ENTRY_ACL_ADD_SUBDIRECTORY 0x00000020 #define ARCHIVE_ENTRY_ACL_READ_NAMED_ATTRS 0x00000040 #define ARCHIVE_ENTRY_ACL_WRITE_NAMED_ATTRS 0x00000080 #define ARCHIVE_ENTRY_ACL_DELETE_CHILD 0x00000100 #define ARCHIVE_ENTRY_ACL_READ_ATTRIBUTES 0x00000200 #define ARCHIVE_ENTRY_ACL_WRITE_ATTRIBUTES 0x00000400 #define ARCHIVE_ENTRY_ACL_DELETE 0x00000800 #define ARCHIVE_ENTRY_ACL_READ_ACL 0x00001000 #define ARCHIVE_ENTRY_ACL_WRITE_ACL 0x00002000 #define ARCHIVE_ENTRY_ACL_WRITE_OWNER 0x00004000 #define ARCHIVE_ENTRY_ACL_SYNCHRONIZE 0x00008000 #define ARCHIVE_ENTRY_ACL_PERMS_POSIX1E \ (ARCHIVE_ENTRY_ACL_EXECUTE \ | ARCHIVE_ENTRY_ACL_WRITE \ | ARCHIVE_ENTRY_ACL_READ) #define ARCHIVE_ENTRY_ACL_PERMS_NFS4 \ (ARCHIVE_ENTRY_ACL_EXECUTE \ | ARCHIVE_ENTRY_ACL_READ_DATA \ | ARCHIVE_ENTRY_ACL_LIST_DIRECTORY \ | ARCHIVE_ENTRY_ACL_WRITE_DATA \ | ARCHIVE_ENTRY_ACL_ADD_FILE \ | ARCHIVE_ENTRY_ACL_APPEND_DATA \ | ARCHIVE_ENTRY_ACL_ADD_SUBDIRECTORY \ | ARCHIVE_ENTRY_ACL_READ_NAMED_ATTRS \ | ARCHIVE_ENTRY_ACL_WRITE_NAMED_ATTRS \ | ARCHIVE_ENTRY_ACL_DELETE_CHILD \ | ARCHIVE_ENTRY_ACL_READ_ATTRIBUTES \ | ARCHIVE_ENTRY_ACL_WRITE_ATTRIBUTES \ | ARCHIVE_ENTRY_ACL_DELETE \ | ARCHIVE_ENTRY_ACL_READ_ACL \ | ARCHIVE_ENTRY_ACL_WRITE_ACL \ | ARCHIVE_ENTRY_ACL_WRITE_OWNER \ | ARCHIVE_ENTRY_ACL_SYNCHRONIZE) /* * Inheritance values (NFS4 ACLs only); included in permset. */ #define ARCHIVE_ENTRY_ACL_ENTRY_INHERITED 0x01000000 #define ARCHIVE_ENTRY_ACL_ENTRY_FILE_INHERIT 0x02000000 #define ARCHIVE_ENTRY_ACL_ENTRY_DIRECTORY_INHERIT 0x04000000 #define ARCHIVE_ENTRY_ACL_ENTRY_NO_PROPAGATE_INHERIT 0x08000000 #define ARCHIVE_ENTRY_ACL_ENTRY_INHERIT_ONLY 0x10000000 #define ARCHIVE_ENTRY_ACL_ENTRY_SUCCESSFUL_ACCESS 0x20000000 #define ARCHIVE_ENTRY_ACL_ENTRY_FAILED_ACCESS 0x40000000 #define ARCHIVE_ENTRY_ACL_INHERITANCE_NFS4 \ (ARCHIVE_ENTRY_ACL_ENTRY_FILE_INHERIT \ | ARCHIVE_ENTRY_ACL_ENTRY_DIRECTORY_INHERIT \ | ARCHIVE_ENTRY_ACL_ENTRY_NO_PROPAGATE_INHERIT \ | ARCHIVE_ENTRY_ACL_ENTRY_INHERIT_ONLY \ | ARCHIVE_ENTRY_ACL_ENTRY_SUCCESSFUL_ACCESS \ | ARCHIVE_ENTRY_ACL_ENTRY_FAILED_ACCESS \ | ARCHIVE_ENTRY_ACL_ENTRY_INHERITED) /* We need to be able to specify combinations of these. */ #define ARCHIVE_ENTRY_ACL_TYPE_ACCESS 0x00000100 /* POSIX.1e only */ #define ARCHIVE_ENTRY_ACL_TYPE_DEFAULT 0x00000200 /* POSIX.1e only */ #define ARCHIVE_ENTRY_ACL_TYPE_ALLOW 0x00000400 /* NFS4 only */ #define ARCHIVE_ENTRY_ACL_TYPE_DENY 0x00000800 /* NFS4 only */ #define ARCHIVE_ENTRY_ACL_TYPE_AUDIT 0x00001000 /* NFS4 only */ #define ARCHIVE_ENTRY_ACL_TYPE_ALARM 0x00002000 /* NFS4 only */ #define ARCHIVE_ENTRY_ACL_TYPE_POSIX1E (ARCHIVE_ENTRY_ACL_TYPE_ACCESS \ | ARCHIVE_ENTRY_ACL_TYPE_DEFAULT) #define ARCHIVE_ENTRY_ACL_TYPE_NFS4 (ARCHIVE_ENTRY_ACL_TYPE_ALLOW \ | ARCHIVE_ENTRY_ACL_TYPE_DENY \ | ARCHIVE_ENTRY_ACL_TYPE_AUDIT \ | ARCHIVE_ENTRY_ACL_TYPE_ALARM) /* Tag values mimic POSIX.1e */ #define ARCHIVE_ENTRY_ACL_USER 10001 /* Specified user. */ #define ARCHIVE_ENTRY_ACL_USER_OBJ 10002 /* User who owns the file. */ #define ARCHIVE_ENTRY_ACL_GROUP 10003 /* Specified group. */ #define ARCHIVE_ENTRY_ACL_GROUP_OBJ 10004 /* Group who owns the file. */ #define ARCHIVE_ENTRY_ACL_MASK 10005 /* Modify group access (POSIX.1e only) */ #define ARCHIVE_ENTRY_ACL_OTHER 10006 /* Public (POSIX.1e only) */ #define ARCHIVE_ENTRY_ACL_EVERYONE 10107 /* Everyone (NFS4 only) */ /* * Set the ACL by clearing it and adding entries one at a time. * Unlike the POSIX.1e ACL routines, you must specify the type * (access/default) for each entry. Internally, the ACL data is just * a soup of entries. API calls here allow you to retrieve just the * entries of interest. This design (which goes against the spirit of * POSIX.1e) is useful for handling archive formats that combine * default and access information in a single ACL list. */ __LA_DECL void archive_entry_acl_clear(struct archive_entry *); __LA_DECL int archive_entry_acl_add_entry(struct archive_entry *, int /* type */, int /* permset */, int /* tag */, int /* qual */, const char * /* name */); __LA_DECL int archive_entry_acl_add_entry_w(struct archive_entry *, int /* type */, int /* permset */, int /* tag */, int /* qual */, const wchar_t * /* name */); /* * To retrieve the ACL, first "reset", then repeatedly ask for the * "next" entry. The want_type parameter allows you to request only * certain types of entries. */ __LA_DECL int archive_entry_acl_reset(struct archive_entry *, int /* want_type */); __LA_DECL int archive_entry_acl_next(struct archive_entry *, int /* want_type */, int * /* type */, int * /* permset */, int * /* tag */, int * /* qual */, const char ** /* name */); __LA_DECL int archive_entry_acl_next_w(struct archive_entry *, int /* want_type */, int * /* type */, int * /* permset */, int * /* tag */, int * /* qual */, const wchar_t ** /* name */); /* * Construct a text-format ACL. The flags argument is a bitmask that * can include any of the following: * * Flags only for archive entries with POSIX.1e ACL: * ARCHIVE_ENTRY_ACL_TYPE_ACCESS - Include POSIX.1e "access" entries. * ARCHIVE_ENTRY_ACL_TYPE_DEFAULT - Include POSIX.1e "default" entries. * ARCHIVE_ENTRY_ACL_STYLE_MARK_DEFAULT - Include "default:" before each * default ACL entry. * ARCHIVE_ENTRY_ACL_STYLE_SOLARIS - Output only one colon after "other" and * "mask" entries. * * Flags only for archive entries with NFSv4 ACL: * ARCHIVE_ENTRY_ACL_STYLE_COMPACT - Do not output the minus character for * unset permissions and flags in NFSv4 ACL permission and flag fields * * Flags for for archive entries with POSIX.1e ACL or NFSv4 ACL: * ARCHIVE_ENTRY_ACL_STYLE_EXTRA_ID - Include extra numeric ID field in * each ACL entry. * ARCHIVE_ENTRY_ACL_STYLE_SEPARATOR_COMMA - Separate entries with comma * instead of newline. */ #define ARCHIVE_ENTRY_ACL_STYLE_EXTRA_ID 0x00000001 #define ARCHIVE_ENTRY_ACL_STYLE_MARK_DEFAULT 0x00000002 #define ARCHIVE_ENTRY_ACL_STYLE_SOLARIS 0x00000004 #define ARCHIVE_ENTRY_ACL_STYLE_SEPARATOR_COMMA 0x00000008 #define ARCHIVE_ENTRY_ACL_STYLE_COMPACT 0x00000010 __LA_DECL wchar_t *archive_entry_acl_to_text_w(struct archive_entry *, la_ssize_t * /* len */, int /* flags */); __LA_DECL char *archive_entry_acl_to_text(struct archive_entry *, la_ssize_t * /* len */, int /* flags */); __LA_DECL int archive_entry_acl_from_text_w(struct archive_entry *, const wchar_t * /* wtext */, int /* type */); __LA_DECL int archive_entry_acl_from_text(struct archive_entry *, const char * /* text */, int /* type */); /* Deprecated constants */ #define OLD_ARCHIVE_ENTRY_ACL_STYLE_EXTRA_ID 1024 #define OLD_ARCHIVE_ENTRY_ACL_STYLE_MARK_DEFAULT 2048 /* Deprecated functions */ __LA_DECL const wchar_t *archive_entry_acl_text_w(struct archive_entry *, int /* flags */) __LA_DEPRECATED; __LA_DECL const char *archive_entry_acl_text(struct archive_entry *, int /* flags */) __LA_DEPRECATED; /* Return bitmask of ACL types in an archive entry */ __LA_DECL int archive_entry_acl_types(struct archive_entry *); /* Return a count of entries matching 'want_type' */ __LA_DECL int archive_entry_acl_count(struct archive_entry *, int /* want_type */); /* Return an opaque ACL object. */ /* There's not yet anything clients can actually do with this... */ struct archive_acl; __LA_DECL struct archive_acl *archive_entry_acl(struct archive_entry *); /* * extended attributes */ __LA_DECL void archive_entry_xattr_clear(struct archive_entry *); __LA_DECL void archive_entry_xattr_add_entry(struct archive_entry *, const char * /* name */, const void * /* value */, size_t /* size */); /* * To retrieve the xattr list, first "reset", then repeatedly ask for the * "next" entry. */ __LA_DECL int archive_entry_xattr_count(struct archive_entry *); __LA_DECL int archive_entry_xattr_reset(struct archive_entry *); __LA_DECL int archive_entry_xattr_next(struct archive_entry *, const char ** /* name */, const void ** /* value */, size_t *); /* * sparse */ __LA_DECL void archive_entry_sparse_clear(struct archive_entry *); __LA_DECL void archive_entry_sparse_add_entry(struct archive_entry *, la_int64_t /* offset */, la_int64_t /* length */); /* * To retrieve the xattr list, first "reset", then repeatedly ask for the * "next" entry. */ __LA_DECL int archive_entry_sparse_count(struct archive_entry *); __LA_DECL int archive_entry_sparse_reset(struct archive_entry *); __LA_DECL int archive_entry_sparse_next(struct archive_entry *, la_int64_t * /* offset */, la_int64_t * /* length */); /* * Utility to match up hardlinks. * * The 'struct archive_entry_linkresolver' is a cache of archive entries * for files with multiple links. Here's how to use it: * 1. Create a lookup object with archive_entry_linkresolver_new() * 2. Tell it the archive format you're using. * 3. Hand each archive_entry to archive_entry_linkify(). * That function will return 0, 1, or 2 entries that should * be written. * 4. Call archive_entry_linkify(resolver, NULL) until * no more entries are returned. * 5. Call archive_entry_linkresolver_free(resolver) to free resources. * * The entries returned have their hardlink and size fields updated * appropriately. If an entry is passed in that does not refer to * a file with multiple links, it is returned unchanged. The intention * is that you should be able to simply filter all entries through * this machine. * * To make things more efficient, be sure that each entry has a valid * nlinks value. The hardlink cache uses this to track when all links * have been found. If the nlinks value is zero, it will keep every * name in the cache indefinitely, which can use a lot of memory. * * Note that archive_entry_size() is reset to zero if the file * body should not be written to the archive. Pay attention! */ struct archive_entry_linkresolver; /* * There are three different strategies for marking hardlinks. * The descriptions below name them after the best-known * formats that rely on each strategy: * * "Old cpio" is the simplest, it always returns any entry unmodified. * As far as I know, only cpio formats use this. Old cpio archives * store every link with the full body; the onus is on the dearchiver * to detect and properly link the files as they are restored. * "tar" is also pretty simple; it caches a copy the first time it sees * any link. Subsequent appearances are modified to be hardlink * references to the first one without any body. Used by all tar * formats, although the newest tar formats permit the "old cpio" strategy * as well. This strategy is very simple for the dearchiver, * and reasonably straightforward for the archiver. * "new cpio" is trickier. It stores the body only with the last * occurrence. The complication is that we might not * see every link to a particular file in a single session, so * there's no easy way to know when we've seen the last occurrence. * The solution here is to queue one link until we see the next. * At the end of the session, you can enumerate any remaining * entries by calling archive_entry_linkify(NULL) and store those * bodies. If you have a file with three links l1, l2, and l3, * you'll get the following behavior if you see all three links: * linkify(l1) => NULL (the resolver stores l1 internally) * linkify(l2) => l1 (resolver stores l2, you write l1) * linkify(l3) => l2, l3 (all links seen, you can write both). * If you only see l1 and l2, you'll get this behavior: * linkify(l1) => NULL * linkify(l2) => l1 * linkify(NULL) => l2 (at end, you retrieve remaining links) * As the name suggests, this strategy is used by newer cpio variants. * It's noticeably more complex for the archiver, slightly more complex * for the dearchiver than the tar strategy, but makes it straightforward * to restore a file using any link by simply continuing to scan until * you see a link that is stored with a body. In contrast, the tar * strategy requires you to rescan the archive from the beginning to * correctly extract an arbitrary link. */ __LA_DECL struct archive_entry_linkresolver *archive_entry_linkresolver_new(void); __LA_DECL void archive_entry_linkresolver_set_strategy( struct archive_entry_linkresolver *, int /* format_code */); __LA_DECL void archive_entry_linkresolver_free(struct archive_entry_linkresolver *); __LA_DECL void archive_entry_linkify(struct archive_entry_linkresolver *, struct archive_entry **, struct archive_entry **); __LA_DECL struct archive_entry *archive_entry_partial_links( struct archive_entry_linkresolver *res, unsigned int *links); #ifdef __cplusplus } #endif /* This is meaningless outside of this header. */ #undef __LA_DECL #endif /* !ARCHIVE_ENTRY_H_INCLUDED */ Index: projects/clang700-import/contrib/libarchive/libarchive/archive_pack_dev.c =================================================================== --- projects/clang700-import/contrib/libarchive/libarchive/archive_pack_dev.c (revision 338689) +++ projects/clang700-import/contrib/libarchive/libarchive/archive_pack_dev.c (revision 338690) @@ -1,329 +1,332 @@ /* $NetBSD: pack_dev.c,v 1.12 2013/06/14 16:28:20 tsutsui Exp $ */ /*- * Copyright (c) 1998, 2001 The NetBSD Foundation, Inc. * All rights reserved. * * This code is derived from software contributed to The NetBSD Foundation * by Charles M. Hannum. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ /* Originally from NetBSD's mknod(8) source. */ #include "archive_platform.h" #if HAVE_SYS_CDEFS_H #include #endif #if !defined(lint) __RCSID("$NetBSD$"); #endif /* not lint */ #ifdef HAVE_LIMITS_H #include #endif #include #ifdef HAVE_STDLIB_H #include #endif #ifdef HAVE_STRING_H #include #endif #ifdef HAVE_SYS_TYPES_H #include #endif #ifdef HAVE_SYS_STAT_H #include #endif +#ifdef HAVE_SYS_SYSMACROS_H +#include +#endif #ifdef HAVE_UNISTD_H #include #endif #include "archive_pack_dev.h" static pack_t pack_netbsd; static pack_t pack_freebsd; static pack_t pack_8_8; static pack_t pack_12_20; static pack_t pack_14_18; static pack_t pack_8_24; static pack_t pack_bsdos; static int compare_format(const void *, const void *); static const char iMajorError[] = "invalid major number"; static const char iMinorError[] = "invalid minor number"; static const char tooManyFields[] = "too many fields for format"; /* This is blatantly stolen from libarchive/archive_entry.c, * in an attempt to get this to play nice on MinGW... */ #if !defined(HAVE_MAJOR) && !defined(major) /* Replacement for major/minor/makedev. */ #define major(x) ((int)(0x00ff & ((x) >> 8))) #define minor(x) ((int)(0xffff00ff & (x))) #define makedev(maj,min) ((0xff00 & ((maj)<<8)) | (0xffff00ff & (min))) #endif /* Play games to come up with a suitable makedev() definition. */ #ifdef __QNXNTO__ /* QNX. */ #include #define apd_makedev(maj, min) makedev(ND_LOCAL_NODE, (maj), (min)) #elif defined makedev /* There's a "makedev" macro. */ #define apd_makedev(maj, min) makedev((maj), (min)) #elif defined mkdev || ((defined _WIN32 || defined __WIN32__) && !defined(__CYGWIN__)) /* Windows. */ #define apd_makedev(maj, min) mkdev((maj), (min)) #else /* There's a "makedev" function. */ #define apd_makedev(maj, min) makedev((maj), (min)) #endif /* exported */ dev_t pack_native(int n, unsigned long numbers[], const char **error) { dev_t dev = 0; if (n == 2) { dev = apd_makedev(numbers[0], numbers[1]); if ((unsigned long)major(dev) != numbers[0]) *error = iMajorError; else if ((unsigned long)minor(dev) != numbers[1]) *error = iMinorError; } else *error = tooManyFields; return (dev); } static dev_t pack_netbsd(int n, unsigned long numbers[], const char **error) { dev_t dev = 0; if (n == 2) { dev = makedev_netbsd(numbers[0], numbers[1]); if ((unsigned long)major_netbsd(dev) != numbers[0]) *error = iMajorError; else if ((unsigned long)minor_netbsd(dev) != numbers[1]) *error = iMinorError; } else *error = tooManyFields; return (dev); } #define major_freebsd(x) ((int32_t)(((x) & 0x0000ff00) >> 8)) #define minor_freebsd(x) ((int32_t)(((x) & 0xffff00ff) >> 0)) #define makedev_freebsd(x,y) ((dev_t)((((x) << 8) & 0x0000ff00) | \ (((y) << 0) & 0xffff00ff))) static dev_t pack_freebsd(int n, unsigned long numbers[], const char **error) { dev_t dev = 0; if (n == 2) { dev = makedev_freebsd(numbers[0], numbers[1]); if ((unsigned long)major_freebsd(dev) != numbers[0]) *error = iMajorError; if ((unsigned long)minor_freebsd(dev) != numbers[1]) *error = iMinorError; } else *error = tooManyFields; return (dev); } #define major_8_8(x) ((int32_t)(((x) & 0x0000ff00) >> 8)) #define minor_8_8(x) ((int32_t)(((x) & 0x000000ff) >> 0)) #define makedev_8_8(x,y) ((dev_t)((((x) << 8) & 0x0000ff00) | \ (((y) << 0) & 0x000000ff))) static dev_t pack_8_8(int n, unsigned long numbers[], const char **error) { dev_t dev = 0; if (n == 2) { dev = makedev_8_8(numbers[0], numbers[1]); if ((unsigned long)major_8_8(dev) != numbers[0]) *error = iMajorError; if ((unsigned long)minor_8_8(dev) != numbers[1]) *error = iMinorError; } else *error = tooManyFields; return (dev); } #define major_12_20(x) ((int32_t)(((x) & 0xfff00000) >> 20)) #define minor_12_20(x) ((int32_t)(((x) & 0x000fffff) >> 0)) #define makedev_12_20(x,y) ((dev_t)((((x) << 20) & 0xfff00000) | \ (((y) << 0) & 0x000fffff))) static dev_t pack_12_20(int n, unsigned long numbers[], const char **error) { dev_t dev = 0; if (n == 2) { dev = makedev_12_20(numbers[0], numbers[1]); if ((unsigned long)major_12_20(dev) != numbers[0]) *error = iMajorError; if ((unsigned long)minor_12_20(dev) != numbers[1]) *error = iMinorError; } else *error = tooManyFields; return (dev); } #define major_14_18(x) ((int32_t)(((x) & 0xfffc0000) >> 18)) #define minor_14_18(x) ((int32_t)(((x) & 0x0003ffff) >> 0)) #define makedev_14_18(x,y) ((dev_t)((((x) << 18) & 0xfffc0000) | \ (((y) << 0) & 0x0003ffff))) static dev_t pack_14_18(int n, unsigned long numbers[], const char **error) { dev_t dev = 0; if (n == 2) { dev = makedev_14_18(numbers[0], numbers[1]); if ((unsigned long)major_14_18(dev) != numbers[0]) *error = iMajorError; if ((unsigned long)minor_14_18(dev) != numbers[1]) *error = iMinorError; } else *error = tooManyFields; return (dev); } #define major_8_24(x) ((int32_t)(((x) & 0xff000000) >> 24)) #define minor_8_24(x) ((int32_t)(((x) & 0x00ffffff) >> 0)) #define makedev_8_24(x,y) ((dev_t)((((x) << 24) & 0xff000000) | \ (((y) << 0) & 0x00ffffff))) static dev_t pack_8_24(int n, unsigned long numbers[], const char **error) { dev_t dev = 0; if (n == 2) { dev = makedev_8_24(numbers[0], numbers[1]); if ((unsigned long)major_8_24(dev) != numbers[0]) *error = iMajorError; if ((unsigned long)minor_8_24(dev) != numbers[1]) *error = iMinorError; } else *error = tooManyFields; return (dev); } #define major_12_12_8(x) ((int32_t)(((x) & 0xfff00000) >> 20)) #define unit_12_12_8(x) ((int32_t)(((x) & 0x000fff00) >> 8)) #define subunit_12_12_8(x) ((int32_t)(((x) & 0x000000ff) >> 0)) #define makedev_12_12_8(x,y,z) ((dev_t)((((x) << 20) & 0xfff00000) | \ (((y) << 8) & 0x000fff00) | \ (((z) << 0) & 0x000000ff))) static dev_t pack_bsdos(int n, unsigned long numbers[], const char **error) { dev_t dev = 0; if (n == 2) { dev = makedev_12_20(numbers[0], numbers[1]); if ((unsigned long)major_12_20(dev) != numbers[0]) *error = iMajorError; if ((unsigned long)minor_12_20(dev) != numbers[1]) *error = iMinorError; } else if (n == 3) { dev = makedev_12_12_8(numbers[0], numbers[1], numbers[2]); if ((unsigned long)major_12_12_8(dev) != numbers[0]) *error = iMajorError; if ((unsigned long)unit_12_12_8(dev) != numbers[1]) *error = "invalid unit number"; if ((unsigned long)subunit_12_12_8(dev) != numbers[2]) *error = "invalid subunit number"; } else *error = tooManyFields; return (dev); } /* list of formats and pack functions */ /* this list must be sorted lexically */ static const struct format { const char *name; pack_t *pack; } formats[] = { {"386bsd", pack_8_8}, {"4bsd", pack_8_8}, {"bsdos", pack_bsdos}, {"freebsd", pack_freebsd}, {"hpux", pack_8_24}, {"isc", pack_8_8}, {"linux", pack_8_8}, {"native", pack_native}, {"netbsd", pack_netbsd}, {"osf1", pack_12_20}, {"sco", pack_8_8}, {"solaris", pack_14_18}, {"sunos", pack_8_8}, {"svr3", pack_8_8}, {"svr4", pack_14_18}, {"ultrix", pack_8_8}, }; static int compare_format(const void *key, const void *element) { const char *name; const struct format *format; name = key; format = element; return (strcmp(name, format->name)); } pack_t * pack_find(const char *name) { struct format *format; format = bsearch(name, formats, sizeof(formats)/sizeof(formats[0]), sizeof(formats[0]), compare_format); if (format == 0) return (NULL); return (format->pack); } Index: projects/clang700-import/contrib/libarchive =================================================================== --- projects/clang700-import/contrib/libarchive (revision 338689) +++ projects/clang700-import/contrib/libarchive (revision 338690) Property changes on: projects/clang700-import/contrib/libarchive ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,2 ## Merged /head/contrib/libarchive:r338015-338689 Merged /vendor/libarchive/dist:r338519 Index: projects/clang700-import/contrib/llvm/tools/lld/ELF/Config.h =================================================================== --- projects/clang700-import/contrib/llvm/tools/lld/ELF/Config.h (revision 338689) +++ projects/clang700-import/contrib/llvm/tools/lld/ELF/Config.h (revision 338690) @@ -1,285 +1,286 @@ //===- Config.h -------------------------------------------------*- C++ -*-===// // // The LLVM Linker // // This file is distributed under the University of Illinois Open Source // License. See LICENSE.TXT for details. // //===----------------------------------------------------------------------===// #ifndef LLD_ELF_CONFIG_H #define LLD_ELF_CONFIG_H #include "lld/Common/ErrorHandler.h" #include "llvm/ADT/MapVector.h" #include "llvm/ADT/StringRef.h" #include "llvm/ADT/StringSet.h" #include "llvm/BinaryFormat/ELF.h" #include "llvm/Support/CachePruning.h" #include "llvm/Support/CodeGen.h" #include "llvm/Support/Endian.h" #include namespace lld { namespace elf { class InputFile; class InputSectionBase; enum ELFKind { ELFNoneKind, ELF32LEKind, ELF32BEKind, ELF64LEKind, ELF64BEKind }; // For --build-id. enum class BuildIdKind { None, Fast, Md5, Sha1, Hexstring, Uuid }; // For --discard-{all,locals,none}. enum class DiscardPolicy { Default, All, Locals, None }; // For --icf={none,safe,all}. enum class ICFLevel { None, Safe, All }; // For --strip-{all,debug}. enum class StripPolicy { None, All, Debug }; // For --unresolved-symbols. enum class UnresolvedPolicy { ReportError, Warn, Ignore, IgnoreAll }; // For --orphan-handling. enum class OrphanHandlingPolicy { Place, Warn, Error }; // For --sort-section and linkerscript sorting rules. enum class SortSectionPolicy { Default, None, Alignment, Name, Priority }; // For --target2 enum class Target2Policy { Abs, Rel, GotRel }; // For tracking ARM Float Argument PCS enum class ARMVFPArgKind { Default, Base, VFP, ToolChain }; struct SymbolVersion { llvm::StringRef Name; bool IsExternCpp; bool HasWildcard; }; // This struct contains symbols version definition that // can be found in version script if it is used for link. struct VersionDefinition { llvm::StringRef Name; uint16_t Id = 0; std::vector Globals; size_t NameOff = 0; // Offset in the string table }; // This struct contains the global configuration for the linker. // Most fields are direct mapping from the command line options // and such fields have the same name as the corresponding options. // Most fields are initialized by the driver. struct Configuration { uint8_t OSABI = 0; llvm::CachePruningPolicy ThinLTOCachePolicy; llvm::StringMap SectionStartMap; llvm::StringRef Chroot; llvm::StringRef DynamicLinker; llvm::StringRef DwoDir; llvm::StringRef Entry; llvm::StringRef Emulation; llvm::StringRef Fini; llvm::StringRef Init; llvm::StringRef LTOAAPipeline; llvm::StringRef LTONewPmPasses; llvm::StringRef LTOObjPath; llvm::StringRef LTOSampleProfile; llvm::StringRef MapFile; llvm::StringRef OutputFile; llvm::StringRef OptRemarksFilename; llvm::StringRef ProgName; llvm::StringRef SoName; llvm::StringRef Sysroot; llvm::StringRef ThinLTOCacheDir; llvm::StringRef ThinLTOIndexOnlyArg; std::pair ThinLTOObjectSuffixReplace; std::pair ThinLTOPrefixReplace; std::string Rpath; std::vector VersionDefinitions; std::vector AuxiliaryList; std::vector FilterList; std::vector SearchPaths; std::vector SymbolOrderingFile; std::vector Undefined; std::vector DynamicList; std::vector VersionScriptGlobals; std::vector VersionScriptLocals; std::vector BuildIdVector; llvm::MapVector, uint64_t> CallGraphProfile; bool AllowMultipleDefinition; bool AndroidPackDynRelocs; bool ARMHasBlx = false; bool ARMHasMovtMovw = false; bool ARMJ1J2BranchEncoding = false; bool AsNeeded = false; bool Bsymbolic; bool BsymbolicFunctions; bool CheckSections; bool CompressDebugSections; bool Cref; bool DefineCommon; bool Demangle = true; bool DisableVerify; bool EhFrameHdr; bool EmitRelocs; bool EnableNewDtags; bool ExecuteOnly; bool ExportDynamic; bool FixCortexA53Errata843419; bool GcSections; bool GdbIndex; bool GnuHash = false; bool GnuUnique; bool HasDynamicList = false; bool HasDynSymTab; bool IgnoreDataAddressEquality; bool IgnoreFunctionAddressEquality; bool LTODebugPassManager; bool LTONewPassManager; bool MergeArmExidx; bool MipsN32Abi = false; bool NoinhibitExec; bool Nostdlib; bool OFormatBinary; bool Omagic; bool OptRemarksWithHotness; bool Pie; bool PrintGcSections; bool PrintIcfSections; bool Relocatable; bool RelrPackDynRelocs; bool SaveTemps; bool SingleRoRx; bool Shared; bool Static = false; bool SysvHash = false; bool Target1Rel; bool Trace; bool ThinLTOEmitImportsFiles; bool ThinLTOIndexOnly; bool UndefinedVersion; bool UseAndroidRelrTags = false; bool WarnBackrefs; bool WarnCommon; bool WarnMissingEntry; bool WarnSymbolOrdering; bool WriteAddends; bool ZCombreloc; bool ZCopyreloc; bool ZExecstack; bool ZHazardplt; bool ZIfuncnoplt; bool ZInitfirst; + bool ZInterpose; bool ZKeepTextSectionPrefix; bool ZNodelete; bool ZNodlopen; bool ZNow; bool ZOrigin; bool ZRelro; bool ZRodynamic; bool ZText; bool ZRetpolineplt; bool ZWxneeded; DiscardPolicy Discard; ICFLevel ICF; OrphanHandlingPolicy OrphanHandling; SortSectionPolicy SortSection; StripPolicy Strip; UnresolvedPolicy UnresolvedSymbols; Target2Policy Target2; ARMVFPArgKind ARMVFPArgs = ARMVFPArgKind::Default; BuildIdKind BuildId = BuildIdKind::None; ELFKind EKind = ELFNoneKind; uint16_t DefaultSymbolVersion = llvm::ELF::VER_NDX_GLOBAL; uint16_t EMachine = llvm::ELF::EM_NONE; llvm::Optional ImageBase; uint64_t MaxPageSize; uint64_t MipsGotSize; uint64_t ZStackSize; unsigned LTOPartitions; unsigned LTOO; unsigned Optimize; unsigned ThinLTOJobs; // The following config options do not directly correspond to any // particualr command line options. // True if we need to pass through relocations in input files to the // output file. Usually false because we consume relocations. bool CopyRelocs; // True if the target is ELF64. False if ELF32. bool Is64; // True if the target is little-endian. False if big-endian. bool IsLE; // endianness::little if IsLE is true. endianness::big otherwise. llvm::support::endianness Endianness; // True if the target is the little-endian MIPS64. // // The reason why we have this variable only for the MIPS is because // we use this often. Some ELF headers for MIPS64EL are in a // mixed-endian (which is horrible and I'd say that's a serious spec // bug), and we need to know whether we are reading MIPS ELF files or // not in various places. // // (Note that MIPS64EL is not a typo for MIPS64LE. This is the official // name whatever that means. A fun hypothesis is that "EL" is short for // little-endian written in the little-endian order, but I don't know // if that's true.) bool IsMips64EL; // Holds set of ELF header flags for the target. uint32_t EFlags = 0; // The ELF spec defines two types of relocation table entries, RELA and // REL. RELA is a triplet of (offset, info, addend) while REL is a // tuple of (offset, info). Addends for REL are implicit and read from // the location where the relocations are applied. So, REL is more // compact than RELA but requires a bit of more work to process. // // (From the linker writer's view, this distinction is not necessary. // If the ELF had chosen whichever and sticked with it, it would have // been easier to write code to process relocations, but it's too late // to change the spec.) // // Each ABI defines its relocation type. IsRela is true if target // uses RELA. As far as we know, all 64-bit ABIs are using RELA. A // few 32-bit ABIs are using RELA too. bool IsRela; // True if we are creating position-independent code. bool Pic; // 4 for ELF32, 8 for ELF64. int Wordsize; }; // The only instance of Configuration struct. extern Configuration *Config; static inline void errorOrWarn(const Twine &Msg) { if (!Config->NoinhibitExec) error(Msg); else warn(Msg); } } // namespace elf } // namespace lld #endif Index: projects/clang700-import/contrib/llvm/tools/lld/ELF/Driver.cpp =================================================================== --- projects/clang700-import/contrib/llvm/tools/lld/ELF/Driver.cpp (revision 338689) +++ projects/clang700-import/contrib/llvm/tools/lld/ELF/Driver.cpp (revision 338690) @@ -1,1520 +1,1521 @@ //===- Driver.cpp ---------------------------------------------------------===// // // The LLVM Linker // // This file is distributed under the University of Illinois Open Source // License. See LICENSE.TXT for details. // //===----------------------------------------------------------------------===// // // The driver drives the entire linking process. It is responsible for // parsing command line options and doing whatever it is instructed to do. // // One notable thing in the LLD's driver when compared to other linkers is // that the LLD's driver is agnostic on the host operating system. // Other linkers usually have implicit default values (such as a dynamic // linker path or library paths) for each host OS. // // I don't think implicit default values are useful because they are // usually explicitly specified by the compiler driver. They can even // be harmful when you are doing cross-linking. Therefore, in LLD, we // simply trust the compiler driver to pass all required options and // don't try to make effort on our side. // //===----------------------------------------------------------------------===// #include "Driver.h" #include "Config.h" #include "Filesystem.h" #include "ICF.h" #include "InputFiles.h" #include "InputSection.h" #include "LinkerScript.h" #include "MarkLive.h" #include "OutputSections.h" #include "ScriptParser.h" #include "SymbolTable.h" #include "Symbols.h" #include "SyntheticSections.h" #include "Target.h" #include "Writer.h" #include "lld/Common/Args.h" #include "lld/Common/Driver.h" #include "lld/Common/ErrorHandler.h" #include "lld/Common/Memory.h" #include "lld/Common/Strings.h" #include "lld/Common/TargetOptionsCommandFlags.h" #include "lld/Common/Threads.h" #include "lld/Common/Version.h" #include "llvm/ADT/SetVector.h" #include "llvm/ADT/StringExtras.h" #include "llvm/ADT/StringSwitch.h" #include "llvm/Support/CommandLine.h" #include "llvm/Support/Compression.h" #include "llvm/Support/LEB128.h" #include "llvm/Support/Path.h" #include "llvm/Support/TarWriter.h" #include "llvm/Support/TargetSelect.h" #include "llvm/Support/raw_ostream.h" #include #include using namespace llvm; using namespace llvm::ELF; using namespace llvm::object; using namespace llvm::sys; using namespace lld; using namespace lld::elf; Configuration *elf::Config; LinkerDriver *elf::Driver; static void setConfigs(opt::InputArgList &Args); bool elf::link(ArrayRef Args, bool CanExitEarly, raw_ostream &Error) { errorHandler().LogName = sys::path::filename(Args[0]); errorHandler().ErrorLimitExceededMsg = "too many errors emitted, stopping now (use " "-error-limit=0 to see all errors)"; errorHandler().ErrorOS = &Error; errorHandler().ExitEarly = CanExitEarly; errorHandler().ColorDiagnostics = Error.has_colors(); InputSections.clear(); OutputSections.clear(); Tar = nullptr; BinaryFiles.clear(); BitcodeFiles.clear(); ObjectFiles.clear(); SharedFiles.clear(); Config = make(); Driver = make(); Script = make(); Symtab = make(); Config->ProgName = Args[0]; Driver->main(Args); // Exit immediately if we don't need to return to the caller. // This saves time because the overhead of calling destructors // for all globally-allocated objects is not negligible. if (CanExitEarly) exitLld(errorCount() ? 1 : 0); freeArena(); return !errorCount(); } // Parses a linker -m option. static std::tuple parseEmulation(StringRef Emul) { uint8_t OSABI = 0; StringRef S = Emul; if (S.endswith("_fbsd")) { S = S.drop_back(5); OSABI = ELFOSABI_FREEBSD; } std::pair Ret = StringSwitch>(S) .Cases("aarch64elf", "aarch64linux", "aarch64_elf64_le_vec", {ELF64LEKind, EM_AARCH64}) .Cases("armelf", "armelf_linux_eabi", {ELF32LEKind, EM_ARM}) .Case("elf32_x86_64", {ELF32LEKind, EM_X86_64}) .Cases("elf32btsmip", "elf32btsmipn32", {ELF32BEKind, EM_MIPS}) .Cases("elf32ltsmip", "elf32ltsmipn32", {ELF32LEKind, EM_MIPS}) .Case("elf32ppc", {ELF32BEKind, EM_PPC}) .Case("elf64btsmip", {ELF64BEKind, EM_MIPS}) .Case("elf64ltsmip", {ELF64LEKind, EM_MIPS}) .Case("elf64ppc", {ELF64BEKind, EM_PPC64}) .Case("elf64lppc", {ELF64LEKind, EM_PPC64}) .Cases("elf_amd64", "elf_x86_64", {ELF64LEKind, EM_X86_64}) .Case("elf_i386", {ELF32LEKind, EM_386}) .Case("elf_iamcu", {ELF32LEKind, EM_IAMCU}) .Default({ELFNoneKind, EM_NONE}); if (Ret.first == ELFNoneKind) error("unknown emulation: " + Emul); return std::make_tuple(Ret.first, Ret.second, OSABI); } // Returns slices of MB by parsing MB as an archive file. // Each slice consists of a member file in the archive. std::vector> static getArchiveMembers( MemoryBufferRef MB) { std::unique_ptr File = CHECK(Archive::create(MB), MB.getBufferIdentifier() + ": failed to parse archive"); std::vector> V; Error Err = Error::success(); bool AddToTar = File->isThin() && Tar; for (const ErrorOr &COrErr : File->children(Err)) { Archive::Child C = CHECK(COrErr, MB.getBufferIdentifier() + ": could not get the child of the archive"); MemoryBufferRef MBRef = CHECK(C.getMemoryBufferRef(), MB.getBufferIdentifier() + ": could not get the buffer for a child of the archive"); if (AddToTar) Tar->append(relativeToRoot(check(C.getFullName())), MBRef.getBuffer()); V.push_back(std::make_pair(MBRef, C.getChildOffset())); } if (Err) fatal(MB.getBufferIdentifier() + ": Archive::children failed: " + toString(std::move(Err))); // Take ownership of memory buffers created for members of thin archives. for (std::unique_ptr &MB : File->takeThinBuffers()) make>(std::move(MB)); return V; } // Opens a file and create a file object. Path has to be resolved already. void LinkerDriver::addFile(StringRef Path, bool WithLOption) { using namespace sys::fs; Optional Buffer = readFile(Path); if (!Buffer.hasValue()) return; MemoryBufferRef MBRef = *Buffer; if (InBinary) { Files.push_back(make(MBRef)); return; } switch (identify_magic(MBRef.getBuffer())) { case file_magic::unknown: readLinkerScript(MBRef); return; case file_magic::archive: { // Handle -whole-archive. if (InWholeArchive) { for (const auto &P : getArchiveMembers(MBRef)) Files.push_back(createObjectFile(P.first, Path, P.second)); return; } std::unique_ptr File = CHECK(Archive::create(MBRef), Path + ": failed to parse archive"); // If an archive file has no symbol table, it is likely that a user // is attempting LTO and using a default ar command that doesn't // understand the LLVM bitcode file. It is a pretty common error, so // we'll handle it as if it had a symbol table. if (!File->isEmpty() && !File->hasSymbolTable()) { for (const auto &P : getArchiveMembers(MBRef)) Files.push_back(make(P.first, Path, P.second)); return; } // Handle the regular case. Files.push_back(make(std::move(File))); return; } case file_magic::elf_shared_object: if (Config->Relocatable) { error("attempted static link of dynamic object " + Path); return; } // DSOs usually have DT_SONAME tags in their ELF headers, and the // sonames are used to identify DSOs. But if they are missing, // they are identified by filenames. We don't know whether the new // file has a DT_SONAME or not because we haven't parsed it yet. // Here, we set the default soname for the file because we might // need it later. // // If a file was specified by -lfoo, the directory part is not // significant, as a user did not specify it. This behavior is // compatible with GNU. Files.push_back( createSharedFile(MBRef, WithLOption ? path::filename(Path) : Path)); return; case file_magic::bitcode: case file_magic::elf_relocatable: if (InLib) Files.push_back(make(MBRef, "", 0)); else Files.push_back(createObjectFile(MBRef)); break; default: error(Path + ": unknown file type"); } } // Add a given library by searching it from input search paths. void LinkerDriver::addLibrary(StringRef Name) { if (Optional Path = searchLibrary(Name)) addFile(*Path, /*WithLOption=*/true); else error("unable to find library -l" + Name); } // This function is called on startup. We need this for LTO since // LTO calls LLVM functions to compile bitcode files to native code. // Technically this can be delayed until we read bitcode files, but // we don't bother to do lazily because the initialization is fast. static void initLLVM() { InitializeAllTargets(); InitializeAllTargetMCs(); InitializeAllAsmPrinters(); InitializeAllAsmParsers(); } // Some command line options or some combinations of them are not allowed. // This function checks for such errors. static void checkOptions(opt::InputArgList &Args) { // The MIPS ABI as of 2016 does not support the GNU-style symbol lookup // table which is a relatively new feature. if (Config->EMachine == EM_MIPS && Config->GnuHash) error("the .gnu.hash section is not compatible with the MIPS target."); if (Config->FixCortexA53Errata843419 && Config->EMachine != EM_AARCH64) error("--fix-cortex-a53-843419 is only supported on AArch64 targets."); if (Config->Pie && Config->Shared) error("-shared and -pie may not be used together"); if (!Config->Shared && !Config->FilterList.empty()) error("-F may not be used without -shared"); if (!Config->Shared && !Config->AuxiliaryList.empty()) error("-f may not be used without -shared"); if (!Config->Relocatable && !Config->DefineCommon) error("-no-define-common not supported in non relocatable output"); if (Config->Relocatable) { if (Config->Shared) error("-r and -shared may not be used together"); if (Config->GcSections) error("-r and --gc-sections may not be used together"); if (Config->GdbIndex) error("-r and --gdb-index may not be used together"); if (Config->ICF != ICFLevel::None) error("-r and --icf may not be used together"); if (Config->Pie) error("-r and -pie may not be used together"); } if (Config->ExecuteOnly) { if (Config->EMachine != EM_AARCH64) error("-execute-only is only supported on AArch64 targets"); if (Config->SingleRoRx && !Script->HasSectionsCommand) error("-execute-only and -no-rosegment cannot be used together"); } } static const char *getReproduceOption(opt::InputArgList &Args) { if (auto *Arg = Args.getLastArg(OPT_reproduce)) return Arg->getValue(); return getenv("LLD_REPRODUCE"); } static bool hasZOption(opt::InputArgList &Args, StringRef Key) { for (auto *Arg : Args.filtered(OPT_z)) if (Key == Arg->getValue()) return true; return false; } static bool getZFlag(opt::InputArgList &Args, StringRef K1, StringRef K2, bool Default) { for (auto *Arg : Args.filtered_reverse(OPT_z)) { if (K1 == Arg->getValue()) return true; if (K2 == Arg->getValue()) return false; } return Default; } static bool isKnown(StringRef S) { return S == "combreloc" || S == "copyreloc" || S == "defs" || S == "execstack" || S == "hazardplt" || S == "ifunc-noplt" || S == "initfirst" || S == "keep-text-section-prefix" || S == "lazy" || S == "muldefs" || S == "nocombreloc" || S == "nocopyreloc" || S == "nodelete" || S == "nodlopen" || S == "noexecstack" || S == "nokeep-text-section-prefix" || S == "norelro" || S == "notext" || S == "now" || S == "origin" || S == "relro" || S == "retpolineplt" || S == "rodynamic" || S == "text" || S == "wxneeded" || S.startswith("max-page-size=") || S.startswith("stack-size="); } // Report an error for an unknown -z option. static void checkZOptions(opt::InputArgList &Args) { for (auto *Arg : Args.filtered(OPT_z)) if (!isKnown(Arg->getValue())) error("unknown -z value: " + StringRef(Arg->getValue())); } void LinkerDriver::main(ArrayRef ArgsArr) { ELFOptTable Parser; opt::InputArgList Args = Parser.parse(ArgsArr.slice(1)); // Interpret this flag early because error() depends on them. errorHandler().ErrorLimit = args::getInteger(Args, OPT_error_limit, 20); // Handle -help if (Args.hasArg(OPT_help)) { printHelp(); return; } // Handle -v or -version. // // A note about "compatible with GNU linkers" message: this is a hack for // scripts generated by GNU Libtool 2.4.6 (released in February 2014 and // still the newest version in March 2017) or earlier to recognize LLD as // a GNU compatible linker. As long as an output for the -v option // contains "GNU" or "with BFD", they recognize us as GNU-compatible. // // This is somewhat ugly hack, but in reality, we had no choice other // than doing this. Considering the very long release cycle of Libtool, // it is not easy to improve it to recognize LLD as a GNU compatible // linker in a timely manner. Even if we can make it, there are still a // lot of "configure" scripts out there that are generated by old version // of Libtool. We cannot convince every software developer to migrate to // the latest version and re-generate scripts. So we have this hack. if (Args.hasArg(OPT_v) || Args.hasArg(OPT_version)) message(getLLDVersion() + " (compatible with GNU linkers)"); // The behavior of -v or --version is a bit strange, but this is // needed for compatibility with GNU linkers. if (Args.hasArg(OPT_v) && !Args.hasArg(OPT_INPUT)) return; if (Args.hasArg(OPT_version)) return; if (const char *Path = getReproduceOption(Args)) { // Note that --reproduce is a debug option so you can ignore it // if you are trying to understand the whole picture of the code. Expected> ErrOrWriter = TarWriter::create(Path, path::stem(Path)); if (ErrOrWriter) { Tar = ErrOrWriter->get(); Tar->append("response.txt", createResponseFile(Args)); Tar->append("version.txt", getLLDVersion() + "\n"); make>(std::move(*ErrOrWriter)); } else { error(Twine("--reproduce: failed to open ") + Path + ": " + toString(ErrOrWriter.takeError())); } } readConfigs(Args); checkZOptions(Args); initLLVM(); createFiles(Args); if (errorCount()) return; inferMachineType(); setConfigs(Args); checkOptions(Args); if (errorCount()) return; switch (Config->EKind) { case ELF32LEKind: link(Args); return; case ELF32BEKind: link(Args); return; case ELF64LEKind: link(Args); return; case ELF64BEKind: link(Args); return; default: llvm_unreachable("unknown Config->EKind"); } } static std::string getRpath(opt::InputArgList &Args) { std::vector V = args::getStrings(Args, OPT_rpath); return llvm::join(V.begin(), V.end(), ":"); } // Determines what we should do if there are remaining unresolved // symbols after the name resolution. static UnresolvedPolicy getUnresolvedSymbolPolicy(opt::InputArgList &Args) { if (Args.hasArg(OPT_relocatable)) return UnresolvedPolicy::IgnoreAll; UnresolvedPolicy ErrorOrWarn = Args.hasFlag(OPT_error_unresolved_symbols, OPT_warn_unresolved_symbols, true) ? UnresolvedPolicy::ReportError : UnresolvedPolicy::Warn; // Process the last of -unresolved-symbols, -no-undefined or -z defs. for (auto *Arg : llvm::reverse(Args)) { switch (Arg->getOption().getID()) { case OPT_unresolved_symbols: { StringRef S = Arg->getValue(); if (S == "ignore-all" || S == "ignore-in-object-files") return UnresolvedPolicy::Ignore; if (S == "ignore-in-shared-libs" || S == "report-all") return ErrorOrWarn; error("unknown --unresolved-symbols value: " + S); continue; } case OPT_no_undefined: return ErrorOrWarn; case OPT_z: if (StringRef(Arg->getValue()) == "defs") return ErrorOrWarn; continue; } } // -shared implies -unresolved-symbols=ignore-all because missing // symbols are likely to be resolved at runtime using other DSOs. if (Config->Shared) return UnresolvedPolicy::Ignore; return ErrorOrWarn; } static Target2Policy getTarget2(opt::InputArgList &Args) { StringRef S = Args.getLastArgValue(OPT_target2, "got-rel"); if (S == "rel") return Target2Policy::Rel; if (S == "abs") return Target2Policy::Abs; if (S == "got-rel") return Target2Policy::GotRel; error("unknown --target2 option: " + S); return Target2Policy::GotRel; } static bool isOutputFormatBinary(opt::InputArgList &Args) { if (auto *Arg = Args.getLastArg(OPT_oformat)) { StringRef S = Arg->getValue(); if (S == "binary") return true; if (S.startswith("elf")) return false; error("unknown --oformat value: " + S); } return false; } static DiscardPolicy getDiscard(opt::InputArgList &Args) { if (Args.hasArg(OPT_relocatable)) return DiscardPolicy::None; auto *Arg = Args.getLastArg(OPT_discard_all, OPT_discard_locals, OPT_discard_none); if (!Arg) return DiscardPolicy::Default; if (Arg->getOption().getID() == OPT_discard_all) return DiscardPolicy::All; if (Arg->getOption().getID() == OPT_discard_locals) return DiscardPolicy::Locals; return DiscardPolicy::None; } static StringRef getDynamicLinker(opt::InputArgList &Args) { auto *Arg = Args.getLastArg(OPT_dynamic_linker, OPT_no_dynamic_linker); if (!Arg || Arg->getOption().getID() == OPT_no_dynamic_linker) return ""; return Arg->getValue(); } static ICFLevel getICF(opt::InputArgList &Args) { auto *Arg = Args.getLastArg(OPT_icf_none, OPT_icf_safe, OPT_icf_all); if (!Arg || Arg->getOption().getID() == OPT_icf_none) return ICFLevel::None; if (Arg->getOption().getID() == OPT_icf_safe) return ICFLevel::Safe; return ICFLevel::All; } static StripPolicy getStrip(opt::InputArgList &Args) { if (Args.hasArg(OPT_relocatable)) return StripPolicy::None; auto *Arg = Args.getLastArg(OPT_strip_all, OPT_strip_debug); if (!Arg) return StripPolicy::None; if (Arg->getOption().getID() == OPT_strip_all) return StripPolicy::All; return StripPolicy::Debug; } static uint64_t parseSectionAddress(StringRef S, const opt::Arg &Arg) { uint64_t VA = 0; if (S.startswith("0x")) S = S.drop_front(2); if (!to_integer(S, VA, 16)) error("invalid argument: " + toString(Arg)); return VA; } static StringMap getSectionStartMap(opt::InputArgList &Args) { StringMap Ret; for (auto *Arg : Args.filtered(OPT_section_start)) { StringRef Name; StringRef Addr; std::tie(Name, Addr) = StringRef(Arg->getValue()).split('='); Ret[Name] = parseSectionAddress(Addr, *Arg); } if (auto *Arg = Args.getLastArg(OPT_Ttext)) Ret[".text"] = parseSectionAddress(Arg->getValue(), *Arg); if (auto *Arg = Args.getLastArg(OPT_Tdata)) Ret[".data"] = parseSectionAddress(Arg->getValue(), *Arg); if (auto *Arg = Args.getLastArg(OPT_Tbss)) Ret[".bss"] = parseSectionAddress(Arg->getValue(), *Arg); return Ret; } static SortSectionPolicy getSortSection(opt::InputArgList &Args) { StringRef S = Args.getLastArgValue(OPT_sort_section); if (S == "alignment") return SortSectionPolicy::Alignment; if (S == "name") return SortSectionPolicy::Name; if (!S.empty()) error("unknown --sort-section rule: " + S); return SortSectionPolicy::Default; } static OrphanHandlingPolicy getOrphanHandling(opt::InputArgList &Args) { StringRef S = Args.getLastArgValue(OPT_orphan_handling, "place"); if (S == "warn") return OrphanHandlingPolicy::Warn; if (S == "error") return OrphanHandlingPolicy::Error; if (S != "place") error("unknown --orphan-handling mode: " + S); return OrphanHandlingPolicy::Place; } // Parse --build-id or --build-id=