Index: releng/10.1/UPDATING =================================================================== --- releng/10.1/UPDATING (revision 284192) +++ releng/10.1/UPDATING (revision 284193) @@ -1,2221 +1,2230 @@ Updating Information for FreeBSD current users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. These instructions assume that you basically know what you are doing. If not, then please consult the FreeBSD handbook: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. NOTE: FreeBSD has switched from gcc to clang. If you have trouble bootstrapping from older versions of FreeBSD, try WITHOUT_CLANG to bootstrap to the tip of stable/10, and then rebuild without this option. The bootstrap process from older version of current is a bit fragile. +20150609: p29 FreeBSD-EN-15:06.file + FreeBSD-EN-15:07.zfs + + Updated base system file(1) to 5.22 to address multiple denial + of service issues. [EN-15:06] + + Improved reliability of ZFS when TRIM/UNMAP and/or L2ARC is used. + [EN-15:07] + 20150513: p10 FreeBSD-EN-15:04.freebsd-update FreeBSD-EN-15:05.ufs Fix bug with freebsd-update(8) that does not ensure the previous upgrade was completed. [EN-15:04] Fix deadlock on reboot with UFS tuned with SU+J. [EN-15:05] 20150407: p9 FreeBSD-SA-15:04.igmp [revised] FreeBSD-SA-15:07.ntp FreeBSD-SA-15:08.bsdinstall FreeBSD-SA-15:09.ipv6 Improved patch for SA-15:04.igmp. Fix multiple vulnerabilities of ntp. [SA-15:07] Fix bsdinstall(8) insecure default GELI keyfile permissions. [SA-15:08] Fix Denial of Service with IPv6 Router Advertisements. [SA-15:09] 20150320: p8 Fix patch for SA-15:06.openssl. 20150319: p7 FreeBSD-SA-15:06.openssl Fix multiple vulnerabilities in OpenSSL. [SA-15:06] 20150225: p6 FreeBSD-SA-15:04.igmp FreeBSD-EN-15:01.vt FreeBSD-EN-15:02.openssl FreeBSD-EN-15:03.freebsd-update Fix integer overflow in IGMP protocol. [SA-15:04] Fix vt(4) crash with improper ioctl parameters. [EN-15:01] Updated base system OpenSSL to 1.0.1l. [EN-15:02] Fix freebsd-update libraries update ordering issue. [EN-15:03] 20150127: p5 FreeBSD-SA-15:02.kmem FreeBSD-SA-15:03.sctp Fix SCTP SCTP_SS_VALUE kernel memory corruption and disclosure vulnerability. [SA-15:02] Fix SCTP stream reset vulnerability. [SA-15:03] 20150114: p4 FreeBSD-SA-15:01.openssl Fix multiple vulnerabilities in OpenSSL. [SA-15:01] 20141223: p3 FreeBSD-SA-14:31.ntp FreeBSD-EN-14:13.freebsd-update Fix multiple vulnerabilities in NTP suite. [SA-14:31] Fix directory deletion issue in freebsd-update. [EN-14:13] 20141217: p2 FreeBSD-SA-14:30.unbound Fix unbound remote denial of service vulnerability. 20141210: p1 FreeBSD-SA-14:27.stdio FreeBSD-SA-14:28.file Fix buffer overflow in stdio. [SA-14:27] Fix multiple vulnerabilities in file(1) and libmagic(3). [SA-14:28] 20140904: The ofwfb driver, used to provide a graphics console on PowerPC when using vt(4), no longer allows mmap() of all of physical memory. This will prevent Xorg on PowerPC with some ATI graphics cards from initializing properly unless x11-servers/xorg-server is updated to 1.12.4_8 or newer. 20140831: The libatf-c and libatf-c++ major versions were downgraded to 0 and 1 respectively to match the upstream numbers. They were out of sync because, when they were originally added to FreeBSD, the upstream versions were not respected. These libraries are private and not yet built by default, so renumbering them should be a non-issue. However, unclean source trees will yield broken test programs once the operator executes "make delete-old-libs" after a "make installworld". Additionally, the atf-sh binary was made private by moving it into /usr/libexec/. Already-built shell test programs will keep the path to the old binary so they will break after "make delete-old" is run. If you are using WITH_TESTS=yes (not the default), wipe the object tree and rebuild from scratch to prevent spurious test failures. This is only needed once: the misnumbered libraries and misplaced binaries have been added to OptionalObsoleteFiles.inc so they will be removed during a clean upgrade. 20140814: The ixgbe tunables now match their sysctl counterparts, for example: hw.ixgbe.enable_aim => hw.ix.enable_aim Anyone using ixgbe tunables should ensure they update /boot/loader.conf. 20140801: The NFSv4.1 server committed by r269398 changes the internal function call interfaces used between the NFS and krpc modules. As such, __FreeBSD_version was bumped. 20140729: The default unbound configuration has been modified to address issues with reverse lookups on networks that use private address ranges. If you use the local_unbound service, run "service local_unbound setup" as root to regenerate your configuration, then "service local_unbound reload" to load the new configuration. 20140717: It is no longer necessary to include the dwarf version in your DEBUG options in your kernel config file. The bug that required it to be placed in the config file has bene fixed. DEBUG should now just contain -g. The build system will automatically update things to do the right thing. 20140715: Several ABI breaking changes were merged to CTL and new iSCSI code. All CTL and iSCSI-related tools, such as ctladm, ctld, iscsid and iscsictl need to be rebuilt to work with a new kernel. 20140708: The WITHOUT_VT_SUPPORT kernel config knob has been renamed WITHOUT_VT. (The other _SUPPORT knobs have a consistent meaning which differs from the behaviour controlled by this knob.) 20140608: On i386 and amd64 systems, the onifconsole flag is now set by default in /etc/ttys for ttyu0. This causes ttyu0 to be automatically enabled as a login TTY if it is set in the bootloader as an active kernel console. No changes in behavior should result otherwise. To revert to the previous behavior, set ttyu0 to "off" in /etc/ttys. 20140512: Clang and llvm have been upgraded to 3.4.1 release. 20140321: Clang and llvm have been upgraded to 3.4 release. 20140306: If a Makefile in a tests/ directory was auto-generating a Kyuafile instead of providing an explicit one, this would prevent such Makefile from providing its own Kyuafile in the future during NO_CLEAN builds. This has been fixed in the Makefiles but manual intervention is needed to clean an objdir if you use NO_CLEAN: # find /usr/obj -name Kyuafile | xargs rm -f 20140303: OpenSSH will now ignore errors caused by kernel lacking of Capsicum capability mode support. Please note that enabling the feature in kernel is still highly recommended. 20140227: OpenSSH is now built with sandbox support, and will use sandbox as the default privilege separation method. This requires Capsicum capability mode support in kernel. 20140216: The nve(4) driver for NVIDIA nForce MCP Ethernet adapters has been deprecated and will not be part of FreeBSD 11.0 and later releases. If you use this driver, please consider switching to the nfe(4) driver instead. 20140120: 10.0-RELEASE. 20131216: The behavior of gss_pseudo_random() for the krb5 mechanism has changed, for applications requesting a longer random string than produced by the underlying enctype's pseudo-random() function. In particular, the random string produced from a session key of enctype aes256-cts-hmac-sha1-96 or aes256-cts-hmac-sha1-96 will be different at the 17th octet and later, after this change. The counter used in the PRF+ construction is now encoded as a big-endian integer in accordance with RFC 4402. __FreeBSD_version is bumped to 1000701. 20131108: The WITHOUT_ATF build knob has been removed and its functionality has been subsumed into the more generic WITHOUT_TESTS. If you were using the former to disable the build of the ATF libraries, you should change your settings to use the latter. 20131031: The default version of mtree is nmtree which is obtained from NetBSD. The output is generally the same, but may vary slightly. If you found you need identical output adding "-F freebsd9" to the command line should do the trick. For the time being, the old mtree is available as fmtree. 20131014: libbsdyml has been renamed to libyaml and moved to /usr/lib/private. This will break ports-mgmt/pkg. Rebuild the port, or upgrade to pkg 1.1.4_8 and verify bsdyml not linked in, before running "make delete-old-libs": # make -C /usr/ports/ports-mgmt/pkg build deinstall install clean or # pkg install pkg; ldd /usr/local/sbin/pkg | grep bsdyml 20131010: The rc.d/jail script has been updated to support jail(8) configuration file. The "jail__*" rc.conf(5) variables for per-jail configuration are automatically converted to /var/run/jail..conf before the jail(8) utility is invoked. This is transparently backward compatible. See below about some incompatibilities and rc.conf(5) manual page for more details. These variables are now deprecated in favor of jail(8) configuration file. One can use "rc.d/jail config " command to generate a jail(8) configuration file in /var/run/jail..conf without running the jail(8) utility. The default pathname of the configuration file is /etc/jail.conf and can be specified by using $jail_conf or $jail__conf variables. Please note that jail_devfs_ruleset accepts an integer at this moment. Please consider to rewrite the ruleset name with an integer. 20130930: BIND has been removed from the base system. If all you need is a local resolver, simply enable and start the local_unbound service instead. Otherwise, several versions of BIND are available in the ports tree. The dns/bind99 port is one example. With this change, nslookup(1) and dig(1) are no longer in the base system. Users should instead use host(1) and drill(1) which are in the base system. Alternatively, nslookup and dig can be obtained by installing the dns/bind-tools port. 20130916: With the addition of unbound(8), a new unbound user is now required during installworld. "mergemaster -p" can be used to add the user prior to installworld, as documented in the handbook. 20130911: OpenSSH is now built with DNSSEC support, and will by default silently trust signed SSHFP records. This can be controlled with the VerifyHostKeyDNS client configuration setting. DNSSEC support can be disabled entirely with the WITHOUT_LDNS option in src.conf. 20130906: The GNU Compiler Collection and C++ standard library (libstdc++) are no longer built by default on platforms where clang is the system compiler. You can enable them with the WITH_GCC and WITH_GNUCXX options in src.conf. 20130905: The PROCDESC kernel option is now part of the GENERIC kernel configuration and is required for the rwhod(8) to work. If you are using custom kernel configuration, you should include 'options PROCDESC'. 20130905: The API and ABI related to the Capsicum framework was modified in backward incompatible way. The userland libraries and programs have to be recompiled to work with the new kernel. This includes the following libraries and programs, but the whole buildworld is advised: libc, libprocstat, dhclient, tcpdump, hastd, hastctl, kdump, procstat, rwho, rwhod, uniq. 20130903: AES-NI intrinsic support has been added to gcc. The AES-NI module has been updated to use this support. A new gcc is required to build the aesni module on both i386 and amd64. 20130821: The PADLOCK_RNG and RDRAND_RNG kernel options are now devices. Thus "device padlock_rng" and "device rdrand_rng" should be used instead of "options PADLOCK_RNG" & "options RDRAND_RNG". 20130813: WITH_ICONV has been split into two feature sets. WITH_ICONV now enables just the iconv* functionality and is now on by default. WITH_LIBICONV_COMPAT enables the libiconv api and link time compatability. Set WITHOUT_ICONV to build the old way. If you have been using WITH_ICONV before, you will very likely need to turn on WITH_LIBICONV_COMPAT. 20130806: INVARIANTS option now enables DEBUG for code with OpenSolaris and Illumos origin, including ZFS. If you have INVARIANTS in your kernel configuration, then there is no need to set DEBUG or ZFS_DEBUG explicitly. DEBUG used to enable witness(9) tracking of OpenSolaris (mostly ZFS) locks if WITNESS option was set. Because that generated a lot of witness(9) reports and all of them were believed to be false positives, this is no longer done. New option OPENSOLARIS_WITNESS can be used to achieve the previous behavior. 20130806: Timer values in IPv6 data structures now use time_uptime instead of time_second. Although this is not a user-visible functional change, userland utilities which directly use them---ndp(8), rtadvd(8), and rtsold(8) in the base system---need to be updated to r253970 or later. 20130802: find -delete can now delete the pathnames given as arguments, instead of only files found below them or if the pathname did not contain any slashes. Formerly, the following error message would result: find: -delete: : relative path potentially not safe Deleting the pathnames given as arguments can be prevented without error messages using -mindepth 1 or by changing directory and passing "." as argument to find. This works in the old as well as the new version of find. 20130726: Behavior of devfs rules path matching has been changed. Pattern is now always matched against fully qualified devfs path and slash characters must be explicitly matched by slashes in pattern (FNM_PATHNAME). Rulesets involving devfs subdirectories must be reviewed. 20130716: The default ARM ABI has changed to the ARM EABI. The old ABI is incompatible with the ARM EABI and all programs and modules will need to be rebuilt to work with a new kernel. To keep using the old ABI ensure the WITHOUT_ARM_EABI knob is set. NOTE: Support for the old ABI will be removed in the future and users are advised to upgrade. 20130709: pkg_install has been disconnected from the build if you really need it you should add WITH_PKGTOOLS in your src.conf(5). 20130709: Most of network statistics structures were changed to be able keep 64-bits counters. Thus all tools, that work with networking statistics, must be rebuilt (netstat(1), bsnmpd(1), etc.) 20130629: Fix targets that run multiple make's to use && rather than ; so that subsequent steps depend on success of previous. NOTE: if building 'universe' with -j* on stable/8 or stable/9 it would be better to start the build using bmake, to avoid overloading the machine. 20130618: Fix a bug that allowed a tracing process (e.g. gdb) to write to a memory-mapped file in the traced process's address space even if neither the traced process nor the tracing process had write access to that file. 20130615: CVS has been removed from the base system. An exact copy of the code is available from the devel/cvs port. 20130613: Some people report the following error after the switch to bmake: make: illegal option -- J usage: make [-BPSXeiknpqrstv] [-C directory] [-D variable] ... *** [buildworld] Error code 2 this likely due to an old instance of make in ${MAKEPATH} (${MAKEOBJDIRPREFIX}${.CURDIR}/make.${MACHINE}) which src/Makefile will use that blindly, if it exists, so if you see the above error: rm -rf `make -V MAKEPATH` should resolve it. 20130516: Use bmake by default. Whereas before one could choose to build with bmake via -DWITH_BMAKE one must now use -DWITHOUT_BMAKE to use the old make. The goal is to remove these knobs for 10-RELEASE. It is worth noting that bmake (like gmake) treats the command line as the unit of failure, rather than statements within the command line. Thus '(cd some/where && dosomething)' is safer than 'cd some/where; dosomething'. The '()' allows consistent behavior in parallel build. 20130429: Fix a bug that allows NFS clients to issue READDIR on files. 20130426: The WITHOUT_IDEA option has been removed because the IDEA patent expired. 20130426: The sysctl which controls TRIM support under ZFS has been renamed from vfs.zfs.trim_disable -> vfs.zfs.trim.enabled and has been enabled by default. 20130425: The mergemaster command now uses the default MAKEOBJDIRPREFIX rather than creating it's own in the temporary directory in order allow access to bootstrapped versions of tools such as install and mtree. When upgrading from version of FreeBSD where the install command does not support -l, you will need to install a new mergemaster command if mergemaster -p is required. This can be accomplished with the command (cd src/usr.sbin/mergemaster && make install). 20130404: Legacy ATA stack, disabled and replaced by new CAM-based one since FreeBSD 9.0, completely removed from the sources. Kernel modules atadisk and atapi*, user-level tools atacontrol and burncd are removed. Kernel option `options ATA_CAM` is now permanently enabled and removed. 20130319: SOCK_CLOEXEC and SOCK_NONBLOCK flags have been added to socket(2) and socketpair(2). Software, in particular Kerberos, may automatically detect and use these during building. The resulting binaries will not work on older kernels. 20130308: CTL_DISABLE has also been added to the sparc64 GENERIC (for further information, see the respective 20130304 entry). 20130304: Recent commits to callout(9) changed the size of struct callout, so the KBI is probably heavily disturbed. Also, some functions in callout(9)/sleep(9)/sleepqueue(9)/condvar(9) KPIs were replaced by macros. Every kernel module using it won't load, so rebuild is requested. The ctl device has been re-enabled in GENERIC for i386 and amd64, but does not initialize by default (because of the new CTL_DISABLE option) to save memory. To re-enable it, remove the CTL_DISABLE option from the kernel config file or set kern.cam.ctl.disable=0 in /boot/loader.conf. 20130301: The ctl device has been disabled in GENERIC for i386 and amd64. This was done due to the extra memory being allocated at system initialisation time by the ctl driver which was only used if a CAM target device was created. This makes a FreeBSD system unusable on 128MB or less of RAM. 20130208: A new compression method (lz4) has been merged to -HEAD. Please refer to zpool-features(7) for more information. Please refer to the "ZFS notes" section of this file for information on upgrading boot ZFS pools. 20130129: A BSD-licensed patch(1) variant has been added and is installed as bsdpatch, being the GNU version the default patch. To inverse the logic and use the BSD-licensed one as default, while having the GNU version installed as gnupatch, rebuild and install world with the WITH_BSD_PATCH knob set. 20130121: Due to the use of the new -l option to install(1) during build and install, you must take care not to directly set the INSTALL make variable in your /etc/make.conf, /etc/src.conf, or on the command line. If you wish to use the -C flag for all installs you may be able to add INSTALL+=-C to /etc/make.conf or /etc/src.conf. 20130118: The install(1) option -M has changed meaning and now takes an argument that is a file or path to append logs to. In the unlikely event that -M was the last option on the command line and the command line contained at least two files and a target directory the first file will have logs appended to it. The -M option served little practical purpose in the last decade so its use is expected to be extremely rare. 20121223: After switching to Clang as the default compiler some users of ZFS on i386 systems started to experience stack overflow kernel panics. Please consider using 'options KSTACK_PAGES=4' in such configurations. 20121222: GEOM_LABEL now mangles label names read from file system metadata. Mangling affect labels containing spaces, non-printable characters, '%' or '"'. Device names in /etc/fstab and other places may need to be updated. 20121217: By default, only the 10 most recent kernel dumps will be saved. To restore the previous behaviour (no limit on the number of kernel dumps stored in the dump directory) add the following line to /etc/rc.conf: savecore_flags="" 20121201: With the addition of auditdistd(8), a new auditdistd user is now required during installworld. "mergemaster -p" can be used to add the user prior to installworld, as documented in the handbook. 20121117: The sin6_scope_id member variable in struct sockaddr_in6 is now filled by the kernel before passing the structure to the userland via sysctl or routing socket. This means the KAME-specific embedded scope id in sin6_addr.s6_addr[2] is always cleared in userland application. This behavior can be controlled by net.inet6.ip6.deembed_scopeid. __FreeBSD_version is bumped to 1000025. 20121105: On i386 and amd64 systems WITH_CLANG_IS_CC is now the default. This means that the world and kernel will be compiled with clang and that clang will be installed as /usr/bin/cc, /usr/bin/c++, and /usr/bin/cpp. To disable this behavior and revert to building with gcc, compile with WITHOUT_CLANG_IS_CC. Really old versions of current may need to bootstrap WITHOUT_CLANG first if the clang build fails (its compatibility window doesn't extend to the 9 stable branch point). 20121102: The IPFIREWALL_FORWARD kernel option has been removed. Its functionality now turned on by default. 20121023: The ZERO_COPY_SOCKET kernel option has been removed and split into SOCKET_SEND_COW and SOCKET_RECV_PFLIP. NB: SOCKET_SEND_COW uses the VM page based copy-on-write mechanism which is not safe and may result in kernel crashes. NB: The SOCKET_RECV_PFLIP mechanism is useless as no current driver supports disposeable external page sized mbuf storage. Proper replacements for both zero-copy mechanisms are under consideration and will eventually lead to complete removal of the two kernel options. 20121023: The IPv4 network stack has been converted to network byte order. The following modules need to be recompiled together with kernel: carp(4), divert(4), gif(4), siftr(4), gre(4), pf(4), ipfw(4), ng_ipfw(4), stf(4). 20121022: Support for non-MPSAFE filesystems was removed from VFS. The VFS_VERSION was bumped, all filesystem modules shall be recompiled. 20121018: All the non-MPSAFE filesystems have been disconnected from the build. The full list includes: codafs, hpfs, ntfs, nwfs, portalfs, smbfs, xfs. 20121016: The interface cloning API and ABI has changed. The following modules need to be recompiled together with kernel: ipfw(4), pfsync(4), pflog(4), usb(4), wlan(4), stf(4), vlan(4), disc(4), edsc(4), if_bridge(4), gif(4), tap(4), faith(4), epair(4), enc(4), tun(4), if_lagg(4), gre(4). 20121015: The sdhci driver was split in two parts: sdhci (generic SD Host Controller logic) and sdhci_pci (actual hardware driver). No kernel config modifications are required, but if you load sdhc as a module you must switch to sdhci_pci instead. 20121014: Import the FUSE kernel and userland support into base system. 20121013: The GNU sort(1) program has been removed since the BSD-licensed sort(1) has been the default for quite some time and no serious problems have been reported. The corresponding WITH_GNU_SORT knob has also gone. 20121006: The pfil(9) API/ABI for AF_INET family has been changed. Packet filtering modules: pf(4), ipfw(4), ipfilter(4) need to be recompiled with new kernel. 20121001: The net80211(4) ABI has been changed to allow for improved driver PS-POLL and power-save support. All wireless drivers need to be recompiled to work with the new kernel. 20120913: The random(4) support for the VIA hardware random number generator (`PADLOCK') is no longer enabled unconditionally. Add the padlock_rng device in the custom kernel config if needed. The GENERIC kernels on i386 and amd64 do include the device, so the change only affects the custom kernel configurations. 20120908: The pf(4) packet filter ABI has been changed. pfctl(8) and snmp_pf module need to be recompiled to work with new kernel. 20120828: A new ZFS feature flag "com.delphix:empty_bpobj" has been merged to -HEAD. Pools that have empty_bpobj in active state can not be imported read-write with ZFS implementations that do not support this feature. For more information read the zpool-features(5) manual page. 20120727: The sparc64 ZFS loader has been changed to no longer try to auto- detect ZFS providers based on diskN aliases but now requires these to be explicitly listed in the OFW boot-device environment variable. 20120712: The OpenSSL has been upgraded to 1.0.1c. Any binaries requiring libcrypto.so.6 or libssl.so.6 must be recompiled. Also, there are configuration changes. Make sure to merge /etc/ssl/openssl.cnf. 20120712: The following sysctls and tunables have been renamed for consistency with other variables: kern.cam.da.da_send_ordered -> kern.cam.da.send_ordered kern.cam.ada.ada_send_ordered -> kern.cam.ada.send_ordered 20120628: The sort utility has been replaced with BSD sort. For now, GNU sort is also available as "gnusort" or the default can be set back to GNU sort by setting WITH_GNU_SORT. In this case, BSD sort will be installed as "bsdsort". 20120611: A new version of ZFS (pool version 5000) has been merged to -HEAD. Starting with this version the old system of ZFS pool versioning is superseded by "feature flags". This concept enables forward compatibility against certain future changes in functionality of ZFS pools. The first read-only compatible "feature flag" for ZFS pools is named "com.delphix:async_destroy". For more information read the new zpool-features(5) manual page. Please refer to the "ZFS notes" section of this file for information on upgrading boot ZFS pools. 20120417: The malloc(3) implementation embedded in libc now uses sources imported as contrib/jemalloc. The most disruptive API change is to /etc/malloc.conf. If your system has an old-style /etc/malloc.conf, delete it prior to installworld, and optionally re-create it using the new format after rebooting. See malloc.conf(5) for details (specifically the TUNING section and the "opt.*" entries in the MALLCTL NAMESPACE section). 20120328: Big-endian MIPS TARGET_ARCH values no longer end in "eb". mips64eb is now spelled mips64. mipsn32eb is now spelled mipsn32. mipseb is now spelled mips. This is to aid compatibility with third-party software that expects this naming scheme in uname(3). Little-endian settings are unchanged. If you are updating a big-endian mips64 machine from before this change, you may need to set MACHINE_ARCH=mips64 in your environment before the new build system will recognize your machine. 20120306: Disable by default the option VFS_ALLOW_NONMPSAFE for all supported platforms. 20120229: Now unix domain sockets behave "as expected" on nullfs(5). Previously nullfs(5) did not pass through all behaviours to the underlying layer, as a result if we bound to a socket on the lower layer we could connect only to the lower path; if we bound to the upper layer we could connect only to the upper path. The new behavior is one can connect to both the lower and the upper paths regardless what layer path one binds to. 20120211: The getifaddrs upgrade path broken with 20111215 has been restored. If you have upgraded in between 20111215 and 20120209 you need to recompile libc again with your kernel. You still need to recompile world to be able to configure CARP but this restriction already comes from 20111215. 20120114: The set_rcvar() function has been removed from /etc/rc.subr. All base and ports rc.d scripts have been updated, so if you have a port installed with a script in /usr/local/etc/rc.d you can either hand-edit the rcvar= line, or reinstall the port. An easy way to handle the mass-update of /etc/rc.d: rm /etc/rc.d/* && mergemaster -i 20120109: panic(9) now stops other CPUs in the SMP systems, disables interrupts on the current CPU and prevents other threads from running. This behavior can be reverted using the kern.stop_scheduler_on_panic tunable/sysctl. The new behavior can be incompatible with kern.sync_on_panic. 20111215: The carp(4) facility has been changed significantly. Configuration of the CARP protocol via ifconfig(8) has changed, as well as format of CARP events submitted to devd(8) has changed. See manual pages for more information. The arpbalance feature of carp(4) is currently not supported anymore. Size of struct in_aliasreq, struct in6_aliasreq has changed. User utilities using SIOCAIFADDR, SIOCAIFADDR_IN6, e.g. ifconfig(8), need to be recompiled. 20111122: The acpi_wmi(4) status device /dev/wmistat has been renamed to /dev/wmistat0. 20111108: The option VFS_ALLOW_NONMPSAFE option has been added in order to explicitely support non-MPSAFE filesystems. It is on by default for all supported platform at this present time. 20111101: The broken amd(4) driver has been replaced with esp(4) in the amd64, i386 and pc98 GENERIC kernel configuration files. 20110930: sysinstall has been removed 20110923: The stable/9 branch created in subversion. This corresponds to the RELENG_9 branch in CVS. 20110913: This commit modifies vfs_register() so that it uses a hash calculation to set vfc_typenum, which is enabled by default. The first time a system is booted after this change, the vfc_typenum values will change for all file systems. The main effect of this is a change to the NFS server file handles for file systems that use vfc_typenum in their fsid, such as ZFS. It will, however, prevent vfc_typenum from changing when file systems are loaded in a different order for subsequent reboots. To disable this, you can set vfs.typenumhash=0 in /boot/loader.conf until you are ready to remount all NFS clients after a reboot. 20110828: Bump the shared library version numbers for libraries that do not use symbol versioning, have changed the ABI compared to stable/8 and which shared library version was not bumped. Done as part of 9.0-RELEASE cycle. 20110815: During the merge of Capsicum features, the fget(9) KPI was modified. This may require the rebuilding of out-of-tree device drivers -- issues have been reported specifically with the nVidia device driver. __FreeBSD_version is bumped to 900041. Also, there is a period between 20110811 and 20110814 where the special devices /dev/{stdin,stdout,stderr} did not work correctly. Building world from a kernel during that window may not work. 20110628: The packet filter (pf) code has been updated to OpenBSD 4.5. You need to update userland tools to be in sync with kernel. This update breaks backward compatibility with earlier pfsync(4) versions. Care must be taken when updating redundant firewall setups. 20110608: The following sysctls and tunables are retired on x86 platforms: machdep.hlt_cpus machdep.hlt_logical_cpus The following sysctl is retired: machdep.hyperthreading_allowed The sysctls were supposed to provide a way to dynamically offline and online selected CPUs on x86 platforms, but the implementation has not been reliable especially with SCHED_ULE scheduler. machdep.hyperthreading_allowed tunable is still available to ignore hyperthreading CPUs at OS level. Individual CPUs can be disabled using hint.lapic.X.disabled tunable, where X is an APIC ID of a CPU. Be advised, though, that disabling CPUs in non-uniform fashion will result in non-uniform topology and may lead to sub-optimal system performance with SCHED_ULE, which is a default scheduler. 20110607: cpumask_t type is retired and cpuset_t is used in order to describe a mask of CPUs. 20110531: Changes to ifconfig(8) for dynamic address family detection mandate that you are running a kernel of 20110525 or later. Make sure to follow the update procedure to boot a new kernel before installing world. 20110513: Support for sun4v architecture is officially dropped 20110503: Several KPI breaking changes have been committed to the mii(4) layer, the PHY drivers and consequently some Ethernet drivers using mii(4). This means that miibus.ko and the modules of the affected Ethernet drivers need to be recompiled. Note to kernel developers: Given that the OUI bit reversion problem was fixed as part of these changes all mii(4) commits related to OUIs, i.e. to sys/dev/mii/miidevs, PHY driver probing and vendor specific handling, no longer can be merged verbatim to stable/8 and previous branches. 20110430: Users of the Atheros AR71xx SoC code now need to add 'device ar71xx_pci' into their kernel configurations along with 'device pci'. 20110427: The default NFS client is now the new NFS client, so fstype "newnfs" is now "nfs" and the regular/old NFS client is now fstype "oldnfs". Although mounts via fstype "nfs" will usually work without userland changes, it is recommended that the mount(8) and mount_nfs(8) commands be rebuilt from sources and that a link to mount_nfs called mount_oldnfs be created. The new client is compiled into the kernel with "options NFSCL" and this is needed for diskless root file systems. The GENERIC kernel configs have been changed to use NFSCL and NFSD (the new server) instead of NFSCLIENT and NFSSERVER. To use the regular/old client, you can "mount -t oldnfs ...". For a diskless root file system, you must also include a line like: vfs.root.mountfrom="oldnfs:" in the boot/loader.conf on the root fs on the NFS server to make a diskless root fs use the old client. 20110424: The GENERIC kernels for all architectures now default to the new CAM-based ATA stack. It means that all legacy ATA drivers were removed and replaced by respective CAM drivers. If you are using ATA device names in /etc/fstab or other places, make sure to update them respectively (adX -> adaY, acdX -> cdY, afdX -> daY, astX -> saY, where 'Y's are the sequential numbers starting from zero for each type in order of detection, unless configured otherwise with tunables, see cam(4)). There will be symbolic links created in /dev/ to map old adX devices to the respective adaY. They should provide basic compatibility for file systems mounting in most cases, but they do not support old user-level APIs and do not have respective providers in GEOM. Consider using updated management tools with new device names. It is possible to load devices ahci, ata, siis and mvs as modules, but option ATA_CAM should remain in kernel configuration to make ata module work as CAM driver supporting legacy ATA controllers. Device ata still can be used in modular fashion (atacore + ...). Modules atadisk and atapi* are not used and won't affect operation in ATA_CAM mode. Note that to use CAM-based ATA kernel should include CAM devices scbus, pass, da (or explicitly ada), cd and optionally others. All of them are parts of the cam module. ataraid(4) functionality is now supported by the RAID GEOM class. To use it you can load geom_raid kernel module and use graid(8) tool for management. Instead of /dev/arX device names, use /dev/raid/rX. No kernel config options or code have been removed, so if a problem arises, please report it and optionally revert to the old ATA stack. In order to do it you can remove from the kernel config: options ATA_CAM device ahci device mvs device siis , and instead add back: device atadisk # ATA disk drives device ataraid # ATA RAID drives device atapicd # ATAPI CDROM drives device atapifd # ATAPI floppy drives device atapist # ATAPI tape drives 20110423: The default NFS server has been changed to the new server, which was referred to as the experimental server. If you need to switch back to the old NFS server, you must now put the "-o" option on both the mountd and nfsd commands. This can be done using the mountd_flags and nfs_server_flags rc.conf variables until an update to the rc scripts is committed, which is coming soon. 20110418: The GNU Objective-C runtime library (libobjc), and other Objective-C related components have been removed from the base system. If you require an Objective-C library, please use one of the available ports. 20110331: ath(4) has been split into bus- and device- modules. if_ath contains the HAL, the TX rate control and the network device code. if_ath_pci contains the PCI bus glue. For Atheros MIPS embedded systems, if_ath_ahb contains the AHB glue. Users need to load both if_ath_pci and if_ath in order to use ath on everything else. TO REPEAT: if_ath_ahb is not needed for normal users. Normal users only need to load if_ath and if_ath_pci for ath(4) operation. 20110314: As part of the replacement of sysinstall, the process of building release media has changed significantly. For details, please re-read release(7), which has been updated to reflect the new build process. 20110218: GNU binutils 2.17.50 (as of 2007-07-03) has been merged to -HEAD. This is the last available version under GPLv2. It brings a number of new features, such as support for newer x86 CPU's (with SSE-3, SSSE-3, SSE 4.1 and SSE 4.2), better support for powerpc64, a number of new directives, and lots of other small improvements. See the ChangeLog file in contrib/binutils for the full details. 20110218: IPsec's HMAC_SHA256-512 support has been fixed to be RFC4868 compliant, and will now use half of hash for authentication. This will break interoperability with all stacks (including all actual FreeBSD versions) who implement draft-ietf-ipsec-ciph-sha-256-00 (they use 96 bits of hash for authentication). The only workaround with such peers is to use another HMAC algorithm for IPsec ("phase 2") authentication. 20110207: Remove the uio_yield prototype and symbol. This function has been misnamed since it was introduced and should not be globally exposed with this name. The equivalent functionality is now available using kern_yield(curthread->td_user_pri). The function remains undocumented. 20110112: A SYSCTL_[ADD_]UQUAD was added for unsigned uint64_t pointers, symmetric with the existing SYSCTL_[ADD_]QUAD. Type checking for scalar sysctls is defined but disabled. Code that needs UQUAD to pass the type checking that must compile on older systems where the define is not present can check against __FreeBSD_version >= 900030. The system dialog(1) has been replaced with a new version previously in ports as devel/cdialog. dialog(1) is mostly command-line compatible with the previous version, but the libdialog associated with it has a largely incompatible API. As such, the original version of libdialog will be kept temporarily as libodialog, until its base system consumers are replaced or updated. Bump __FreeBSD_version to 900030. 20110103: If you are trying to run make universe on a -stable system, and you get the following warning: "Makefile", line 356: "Target architecture for i386/conf/GENERIC unknown. config(8) likely too old." or something similar to it, then you must upgrade your -stable system to 8.2-Release or newer (really, any time after r210146 7/15/2010 in stable/8) or build the config from the latest stable/8 branch and install it on your system. Prior to this date, building a current universe on 8-stable system from between 7/15/2010 and 1/2/2011 would result in a weird shell parsing error in the first kernel build phase. A new config on those old systems will fix that problem for older versions of -current. 20101228: The TCP stack has been modified to allow Khelp modules to interact with it via helper hook points and store per-connection data in the TCP control block. Bump __FreeBSD_version to 900029. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20101114: Generic IEEE 802.3 annex 31B full duplex flow control support has been added to mii(4) and bge(4), bce(4), msk(4), nfe(4) and stge(4) along with brgphy(4), e1000phy(4) as well as ip1000phy() have been converted to take advantage of it instead of using custom implementations. This means that these drivers now no longer unconditionally advertise support for flow control but only do so if flow control is a selected media option. This was implemented in the generic support that way in order to allow flow control to be switched on and off via ifconfig(8) with the PHY specific default to typically off in order to protect from unwanted effects. Consequently, if you used flow control with one of the above mentioned drivers you now need to explicitly enable it, for example via: ifconfig bge0 media auto mediaopt flowcontrol Along with the above mentioned changes generic support for setting 1000baseT master mode also has been added and brgphy(4), ciphy(4), e1000phy(4) as well as ip1000phy(4) have been converted to take advantage of it. This means that these drivers now no longer take the link0 parameter for selecting master mode but the master media option has to be used instead, for example like in the following: ifconfig bge0 media 1000baseT mediaopt full-duplex,master Selection of master mode now is also available with all other PHY drivers supporting 1000baseT. 20101111: The TCP stack has received a significant update to add support for modularised congestion control and generally improve the clarity of congestion control decisions. Bump __FreeBSD_version to 900025. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20101002: The man(1) utility has been replaced by a new version that no longer uses /etc/manpath.config. Please consult man.conf(5) for how to migrate local entries to the new format. 20100928: The copyright strings printed by login(1) and sshd(8) at the time of a new connection have been removed to follow other operating systems and upstream sshd. 20100915: A workaround for a fixed ld bug has been removed in kernel code, so make sure that your system ld is built from sources after revision 210245 from 2010-07-19 (r211583 if building head kernel on stable/8, r211584 for stable/7; both from 2010-08-21). A symptom of incorrect ld version is different addresses for set_pcpu section and __start_set_pcpu symbol in kernel and/or modules. 20100913: The $ipv6_prefer variable in rc.conf(5) has been split into $ip6addrctl_policy and $ipv6_activate_all_interfaces. The $ip6addrctl_policy is a variable to choose a pre-defined address selection policy set by ip6addrctl(8). A value "ipv4_prefer", "ipv6_prefer" or "AUTO" can be specified. The default is "AUTO". The $ipv6_activate_all_interfaces specifies whether IFDISABLED flag (see an entry of 20090926) is set on an interface with no corresponding $ifconfig_IF_ipv6 line. The default is "NO" for security reason. If you want IPv6 link-local address on all interfaces by default, set this to "YES". The old ipv6_prefer="YES" is equivalent to ipv6_activate_all_interfaces="YES" and ip6addrctl_policy="ipv6_prefer". 20100913: DTrace has grown support for userland tracing. Due to this, DTrace is now i386 and amd64 only. dtruss(1) is now installed by default on those systems and a new kernel module is needed for userland tracing: fasttrap. No changes to your kernel config file are necessary to enable userland tracing, but you might consider adding 'STRIP=' and 'CFLAGS+=-fno-omit-frame-pointer' to your make.conf if you want to have informative userland stack traces in DTrace (ustack). 20100725: The acpi_aiboost(4) driver has been removed in favor of the new aibs(4) driver. You should update your kernel configuration file. 20100722: BSD grep has been imported to the base system and it is built by default. It is completely BSD licensed, highly GNU-compatible, uses less memory than its GNU counterpart and has a small codebase. However, it is slower than its GNU counterpart, which is mostly noticeable for larger searches, for smaller ones it is measurable but not significant. The reason is complex, the most important factor is that we lack a modern and efficient regex library and GNU overcomes this by optimizing the searches internally. Future work on improving the regex performance is planned, for the meantime, users that need better performance, can build GNU grep instead by setting the WITH_GNU_GREP knob. 20100713: Due to the import of powerpc64 support, all existing powerpc kernel configuration files must be updated with a machine directive like this: machine powerpc powerpc In addition, an updated config(8) is required to build powerpc kernels after this change. 20100713: A new version of ZFS (version 15) has been merged to -HEAD. This version uses a python library for the following subcommands: zfs allow, zfs unallow, zfs groupspace, zfs userspace. For full functionality of these commands the following port must be installed: sysutils/py-zfs 20100429: 'vm_page's are now hashed by physical address to an array of mutexes. Currently this is only used to serialize access to hold_count. Over time the page queue mutex will be peeled away. This changes the size of pmap on every architecture. And requires all callers of vm_page_hold and vm_page_unhold to be updated. 20100402: WITH_CTF can now be specified in src.conf (not recommended, there are some problems with static executables), make.conf (would also affect ports which do not use GNU make and do not override the compile targets) or in the kernel config (via "makeoptions WITH_CTF=yes"). When WITH_CTF was specified there before this was silently ignored, so make sure that WITH_CTF is not used in places which could lead to unwanted behavior. 20100311: The kernel option COMPAT_IA32 has been replaced with COMPAT_FREEBSD32 to allow 32-bit compatibility on non-x86 platforms. All kernel configurations on amd64 and ia64 platforms using these options must be modified accordingly. 20100113: The utmp user accounting database has been replaced with utmpx, the user accounting interface standardized by POSIX. Unfortunately the semantics of utmp and utmpx don't match, making it practically impossible to support both interfaces. The user accounting database is used by tools like finger(1), last(1), talk(1), w(1) and ac(8). All applications in the base system use utmpx. This means only local binaries (e.g. from the ports tree) may still use these utmp database files. These applications must be rebuilt to make use of utmpx. After the system has been upgraded, it is safe to remove the old log files (/var/run/utmp, /var/log/lastlog and /var/log/wtmp*), assuming their contents is of no importance anymore. Old wtmp databases can only be used by last(1) and ac(8) after they have been converted to the new format using wtmpcvt(1). 20100108: Introduce the kernel thread "deadlock resolver" (which can be enabled via the DEADLKRES option, see NOTES for more details) and the sleepq_type() function for sleepqueues. 20091202: The rc.firewall and rc.firewall6 were unified, and rc.firewall6 and rc.d/ip6fw were removed. According to the removal of rc.d/ip6fw, ipv6_firewall_* rc variables are obsoleted. Instead, the following new rc variables are added to rc.d/ipfw: firewall_client_net_ipv6, firewall_simple_iif_ipv6, firewall_simple_inet_ipv6, firewall_simple_oif_ipv6, firewall_simple_onet_ipv6, firewall_trusted_ipv6 The meanings correspond to the relevant IPv4 variables. 20091125: 8.0-RELEASE. 20091113: The default terminal emulation for syscons(4) has been changed from cons25 to xterm on all platforms except pc98. This means that the /etc/ttys file needs to be updated to ensure correct operation of applications on the console. The terminal emulation style can be toggled per window by using vidcontrol(1)'s -T flag. The TEKEN_CONS25 kernel configuration options can be used to change the compile-time default back to cons25. To prevent graphical artifacts, make sure the TERM environment variable is set to match the terminal emulation that is being performed by syscons(4). 20091109: The layout of the structure ieee80211req_scan_result has changed. Applications that require wireless scan results (e.g. ifconfig(8)) from net80211 need to be recompiled. Applications such as wpa_supplicant(8) may require a full world build without using NO_CLEAN in order to get synchronized with the new structure. 20091025: The iwn(4) driver has been updated to support the 5000 and 5150 series. There's one kernel module for each firmware. Adding "device iwnfw" to the kernel configuration file means including all three firmware images inside the kernel. If you want to include just the one for your wireless card, use the devices iwn4965fw, iwn5000fw or iwn5150fw. 20090926: The rc.d/network_ipv6, IPv6 configuration script has been integrated into rc.d/netif. The changes are the following: 1. To use IPv6, simply define $ifconfig_IF_ipv6 like $ifconfig_IF for IPv4. For aliases, $ifconfig_IF_aliasN should be used. Note that both variables need the "inet6" keyword at the head. Do not set $ipv6_network_interfaces manually if you do not understand what you are doing. It is not needed in most cases. $ipv6_ifconfig_IF and $ipv6_ifconfig_IF_aliasN still work, but they are obsolete. 2. $ipv6_enable is obsolete. Use $ipv6_prefer and "inet6 accept_rtadv" keyword in ifconfig(8) instead. If you define $ipv6_enable=YES, it means $ipv6_prefer=YES and all configured interfaces have "inet6 accept_rtadv" in the $ifconfig_IF_ipv6. These are for backward compatibility. 3. A new variable $ipv6_prefer has been added. If NO, IPv6 functionality of interfaces with no corresponding $ifconfig_IF_ipv6 is disabled by using "inet6 ifdisabled" flag, and the default address selection policy of ip6addrctl(8) is the IPv4-preferred one (see rc.d/ip6addrctl for more details). Note that if you want to configure IPv6 functionality on the disabled interfaces after boot, first you need to clear the flag by using ifconfig(8) like: ifconfig em0 inet6 -ifdisabled If YES, the default address selection policy is set as IPv6-preferred. The default value of $ipv6_prefer is NO. 4. If your system need to receive Router Advertisement messages, define "inet6 accept_rtadv" in $ifconfig_IF_ipv6. The rc(8) scripts automatically invoke rtsol(8) when the interface becomes UP. The Router Advertisement messages are used for SLAAC (State-Less Address AutoConfiguration). 20090922: 802.11s D3.03 support was committed. This is incompatible with the previous code, which was based on D3.0. 20090912: A sysctl variable net.inet6.ip6.accept_rtadv now sets the default value of a per-interface flag ND6_IFF_ACCEPT_RTADV, not a global knob to control whether accepting Router Advertisement messages or not. Also, a per-interface flag ND6_IFF_AUTO_LINKLOCAL has been added and a sysctl variable net.inet6.ip6.auto_linklocal is its default value. The ifconfig(8) utility now supports these flags. 20090910: ZFS snapshots are now mounted with MNT_IGNORE flag. Use -v option for mount(8) and -a option for df(1) to see them. 20090825: The old tunable hw.bus.devctl_disable has been superseded by hw.bus.devctl_queue. hw.bus.devctl_disable=1 in loader.conf should be replaced by hw.bus.devctl_queue=0. The default for this new tunable is 1000. 20090813: Remove the option STOP_NMI. The default action is now to use NMI only for KDB via the newly introduced function stop_cpus_hard() and maintain stop_cpus() to just use a normal IPI_STOP on ia32 and amd64. 20090803: The stable/8 branch created in subversion. This corresponds to the RELENG_8 branch in CVS. 20090719: Bump the shared library version numbers for all libraries that do not use symbol versioning as part of the 8.0-RELEASE cycle. Bump __FreeBSD_version to 800105. 20090714: Due to changes in the implementation of virtual network stack support, all network-related kernel modules must be recompiled. As this change breaks the ABI, bump __FreeBSD_version to 800104. 20090713: The TOE interface to the TCP syncache has been modified to remove struct tcpopt () from the ABI of the network stack. The cxgb driver is the only TOE consumer affected by this change, and needs to be recompiled along with the kernel. As this change breaks the ABI, bump __FreeBSD_version to 800103. 20090712: Padding has been added to struct tcpcb, sackhint and tcpstat in to facilitate future MFCs and bug fixes whilst maintaining the ABI. However, this change breaks the ABI, so bump __FreeBSD_version to 800102. User space tools that rely on the size of any of these structs (e.g. sockstat) need to be recompiled. 20090630: The NFS_LEGACYRPC option has been removed along with the old kernel RPC implementation that this option selected. Kernel configurations may need to be adjusted. 20090629: The network interface device nodes at /dev/net/ have been removed. All ioctl operations can be performed the normal way using routing sockets. The kqueue functionality can generally be replaced with routing sockets. 20090628: The documentation from the FreeBSD Documentation Project (Handbook, FAQ, etc.) is now installed via packages by sysinstall(8) and under the /usr/local/share/doc/freebsd directory instead of /usr/share/doc. 20090624: The ABI of various structures related to the SYSV IPC API have been changed. As a result, the COMPAT_FREEBSD[456] and COMPAT_43 kernel options now all require COMPAT_FREEBSD7. Bump __FreeBSD_version to 800100. 20090622: Layout of struct vnet has changed as routing related variables were moved to their own Vimage module. Modules need to be recompiled. Bump __FreeBSD_version to 800099. 20090619: NGROUPS_MAX and NGROUPS have been increased from 16 to 1023 and 1024 respectively. As long as no more than 16 groups per process are used, no changes should be visible. When more than 16 groups are used, old binaries may fail if they call getgroups() or getgrouplist() with statically sized storage. Recompiling will work around this, but applications should be modified to use dynamically allocated storage for group arrays as POSIX.1-2008 does not cap an implementation's number of supported groups at NGROUPS_MAX+1 as previous versions did. NFS and portalfs mounts may also be affected as the list of groups is truncated to 16. Users of NFS who use more than 16 groups, should take care that negative group permissions are not used on the exported file systems as they will not be reliable unless a GSSAPI based authentication method is used. 20090616: The compiling option ADAPTIVE_LOCKMGRS has been introduced. This option compiles in the support for adaptive spinning for lockmgrs which want to enable it. The lockinit() function now accepts the flag LK_ADAPTIVE in order to make the lock object subject to adaptive spinning when both held in write and read mode. 20090613: The layout of the structure returned by IEEE80211_IOC_STA_INFO has changed. User applications that use this ioctl need to be rebuilt. 20090611: The layout of struct thread has changed. Kernel and modules need to be rebuilt. 20090608: The layout of structs ifnet, domain, protosw and vnet_net has changed. Kernel modules need to be rebuilt. Bump __FreeBSD_version to 800097. 20090602: window(1) has been removed from the base system. It can now be installed from ports. The port is called misc/window. 20090601: The way we are storing and accessing `routing table' entries has changed. Programs reading the FIB, like netstat, need to be re-compiled. 20090601: A new netisr implementation has been added for FreeBSD 8. Network file system modules, such as igmp, ipdivert, and others, should be rebuilt. Bump __FreeBSD_version to 800096. 20090530: Remove the tunable/sysctl debug.mpsafevfs as its initial purpose is no more valid. 20090530: Add VOP_ACCESSX(9). File system modules need to be rebuilt. Bump __FreeBSD_version to 800094. 20090529: Add mnt_xflag field to 'struct mount'. File system modules need to be rebuilt. Bump __FreeBSD_version to 800093. 20090528: The compiling option ADAPTIVE_SX has been retired while it has been introduced the option NO_ADAPTIVE_SX which handles the reversed logic. The KPI for sx_init_flags() changes as accepting flags: SX_ADAPTIVESPIN flag has been retired while the SX_NOADAPTIVE flag has been introduced in order to handle the reversed logic. Bump __FreeBSD_version to 800092. 20090527: Add support for hierarchical jails. Remove global securelevel. Bump __FreeBSD_version to 800091. 20090523: The layout of struct vnet_net has changed, therefore modules need to be rebuilt. Bump __FreeBSD_version to 800090. 20090523: The newly imported zic(8) produces a new format in the output. Please run tzsetup(8) to install the newly created data to /etc/localtime. 20090520: The sysctl tree for the usb stack has renamed from hw.usb2.* to hw.usb.* and is now consistent again with previous releases. 20090520: 802.11 monitor mode support was revised and driver api's were changed. Drivers dependent on net80211 now support DLT_IEEE802_11_RADIO instead of DLT_IEEE802_11. No user-visible data structures were changed but applications that use DLT_IEEE802_11 may require changes. Bump __FreeBSD_version to 800088. 20090430: The layout of the following structs has changed: sysctl_oid, socket, ifnet, inpcbinfo, tcpcb, syncache_head, vnet_inet, vnet_inet6 and vnet_ipfw. Most modules need to be rebuild or panics may be experienced. World rebuild is required for correctly checking networking state from userland. Bump __FreeBSD_version to 800085. 20090429: MLDv2 and Source-Specific Multicast (SSM) have been merged to the IPv6 stack. VIMAGE hooks are in but not yet used. The implementation of SSM within FreeBSD's IPv6 stack closely follows the IPv4 implementation. For kernel developers: * The most important changes are that the ip6_output() and ip6_input() paths no longer take the IN6_MULTI_LOCK, and this lock has been downgraded to a non-recursive mutex. * As with the changes to the IPv4 stack to support SSM, filtering of inbound multicast traffic must now be performed by transport protocols within the IPv6 stack. This does not apply to TCP and SCTP, however, it does apply to UDP in IPv6 and raw IPv6. * The KPIs used by IPv6 multicast are similar to those used by the IPv4 stack, with the following differences: * im6o_mc_filter() is analogous to imo_multicast_filter(). * The legacy KAME entry points in6_joingroup and in6_leavegroup() are shimmed to in6_mc_join() and in6_mc_leave() respectively. * IN6_LOOKUP_MULTI() has been deprecated and removed. * IPv6 relies on MLD for the DAD mechanism. KAME's internal KPIs for MLDv1 have an additional 'timer' argument which is used to jitter the initial membership report for the solicited-node multicast membership on-link. * This is not strictly needed for MLDv2, which already jitters its report transmissions. However, the 'timer' argument is preserved in case MLDv1 is active on the interface. * The KAME linked-list based IPv6 membership implementation has been refactored to use a vector similar to that used by the IPv4 stack. Code which maintains a list of its own multicast memberships internally, e.g. carp, has been updated to reflect the new semantics. * There is a known Lock Order Reversal (LOR) due to in6_setscope() acquiring the IF_AFDATA_LOCK and being called within ip6_output(). Whilst MLDv2 tries to avoid this otherwise benign LOR, it is an implementation constraint which needs to be addressed in HEAD. For application developers: * The changes are broadly similar to those made for the IPv4 stack. * The use of IPv4 and IPv6 multicast socket options on the same socket, using mapped addresses, HAS NOT been tested or supported. * There are a number of issues with the implementation of various IPv6 multicast APIs which need to be resolved in the API surface before the implementation is fully compatible with KAME userland use, and these are mostly to do with interface index treatment. * The literature available discusses the use of either the delta / ASM API with setsockopt(2)/getsockopt(2), or the full-state / ASM API using setsourcefilter(3)/getsourcefilter(3). For more information please refer to RFC 3768, 'Socket Interface Extensions for Multicast Source Filters'. * Applications which use the published RFC 3678 APIs should be fine. For systems administrators: * The mtest(8) utility has been refactored to support IPv6, in addition to IPv4. Interface addresses are no longer accepted as arguments, their names must be used instead. The utility will map the interface name to its first IPv4 address as returned by getifaddrs(3). * The ifmcstat(8) utility has also been updated to print the MLDv2 endpoint state and source filter lists via sysctl(3). * The net.inet6.ip6.mcast.loop sysctl may be tuned to 0 to disable loopback of IPv6 multicast datagrams by default; it defaults to 1 to preserve the existing behaviour. Disabling multicast loopback is recommended for optimal system performance. * The IPv6 MROUTING code has been changed to examine this sysctl instead of attempting to perform a group lookup before looping back forwarded datagrams. Bump __FreeBSD_version to 800084. 20090422: Implement low-level Bluetooth HCI API. Bump __FreeBSD_version to 800083. 20090419: The layout of struct malloc_type, used by modules to register new memory allocation types, has changed. Most modules will need to be rebuilt or panics may be experienced. Bump __FreeBSD_version to 800081. 20090415: Anticipate overflowing inp_flags - add inp_flags2. This changes most offsets in inpcb, so checking v4 connection state will require a world rebuild. Bump __FreeBSD_version to 800080. 20090415: Add an llentry to struct route and struct route_in6. Modules embedding a struct route will need to be recompiled. Bump __FreeBSD_version to 800079. 20090414: The size of rt_metrics_lite and by extension rtentry has changed. Networking administration apps will need to be recompiled. The route command now supports show as an alias for get, weighting of routes, sticky and nostick flags to alter the behavior of stateful load balancing. Bump __FreeBSD_version to 800078. 20090408: Do not use Giant for kbdmux(4) locking. This is wrong and apparently causing more problems than it solves. This will re-open the issue where interrupt handlers may race with kbdmux(4) in polling mode. Typical symptoms include (but not limited to) duplicated and/or missing characters when low level console functions (such as gets) are used while interrupts are enabled (for example geli password prompt, mountroot prompt etc.). Disabling kbdmux(4) may help. 20090407: The size of structs vnet_net, vnet_inet and vnet_ipfw has changed; kernel modules referencing any of the above need to be recompiled. Bump __FreeBSD_version to 800075. 20090320: GEOM_PART has become the default partition slicer for storage devices, replacing GEOM_MBR, GEOM_BSD, GEOM_PC98 and GEOM_GPT slicers. It introduces some changes: MSDOS/EBR: the devices created from MSDOS extended partition entries (EBR) can be named differently than with GEOM_MBR and are now symlinks to devices with offset-based names. fstabs may need to be modified. BSD: the "geometry does not match label" warning is harmless in most cases but it points to problems in file system misalignment with disk geometry. The "c" partition is now implicit, covers the whole top-level drive and cannot be (mis)used by users. General: Kernel dumps are now not allowed to be written to devices whose partition types indicate they are meant to be used for file systems (or, in case of MSDOS partitions, as something else than the "386BSD" type). Most of these changes date approximately from 200812. 20090319: The uscanner(4) driver has been removed from the kernel. This follows Linux removing theirs in 2.6 and making libusb the default interface (supported by sane). 20090319: The multicast forwarding code has been cleaned up. netstat(1) only relies on KVM now for printing bandwidth upcall meters. The IPv4 and IPv6 modules are split into ip_mroute_mod and ip6_mroute_mod respectively. The config(5) options for statically compiling this code remain the same, i.e. 'options MROUTING'. 20090315: Support for the IFF_NEEDSGIANT network interface flag has been removed, which means that non-MPSAFE network device drivers are no longer supported. In particular, if_ar, if_sr, and network device drivers from the old (legacy) USB stack can no longer be built or used. 20090313: POSIX.1 Native Language Support (NLS) has been enabled in libc and a bunch of new language catalog files have also been added. This means that some common libc messages are now localized and they depend on the LC_MESSAGES environmental variable. 20090313: The k8temp(4) driver has been renamed to amdtemp(4) since support for Family 10 and Family 11 CPU families was added. 20090309: IGMPv3 and Source-Specific Multicast (SSM) have been merged to the IPv4 stack. VIMAGE hooks are in but not yet used. For kernel developers, the most important changes are that the ip_output() and ip_input() paths no longer take the IN_MULTI_LOCK(), and this lock has been downgraded to a non-recursive mutex. Transport protocols (UDP, Raw IP) are now responsible for filtering inbound multicast traffic according to group membership and source filters. The imo_multicast_filter() KPI exists for this purpose. Transports which do not use multicast (SCTP, TCP) already reject multicast by default. Forwarding and receive performance may improve as a mutex acquisition is no longer needed in the ip_input() low-level input path. in_addmulti() and in_delmulti() are shimmed to new KPIs which exist to support SSM in-kernel. For application developers, it is recommended that loopback of multicast datagrams be disabled for best performance, as this will still cause the lock to be taken for each looped-back datagram transmission. The net.inet.ip.mcast.loop sysctl may be tuned to 0 to disable loopback by default; it defaults to 1 to preserve the existing behaviour. For systems administrators, to obtain best performance with multicast reception and multiple groups, it is always recommended that a card with a suitably precise hash filter is used. Hash collisions will still result in the lock being taken within the transport protocol input path to check group membership. If deploying FreeBSD in an environment with IGMP snooping switches, it is recommended that the net.inet.igmp.sendlocal sysctl remain enabled; this forces 224.0.0.0/24 group membership to be announced via IGMP. The size of 'struct igmpstat' has changed; netstat needs to be recompiled to reflect this. Bump __FreeBSD_version to 800070. 20090309: libusb20.so.1 is now installed as libusb.so.1 and the ports system updated to use it. This requires a buildworld/installworld in order to update the library and dependencies (usbconfig, etc). Its advisable to rebuild all ports which uses libusb. More specific directions are given in the ports collection UPDATING file. Any /etc/libmap.conf entries for libusb are no longer required and can be removed. 20090302: A workaround is committed to allow the creation of System V shared memory segment of size > 2 GB on the 64-bit architectures. Due to a limitation of the existing ABI, the shm_segsz member of the struct shmid_ds, returned by shmctl(IPC_STAT) call is wrong for large segments. Note that limits must be explicitly raised to allow such segments to be created. 20090301: The layout of struct ifnet has changed, requiring a rebuild of all network device driver modules. 20090227: The /dev handling for the new USB stack has changed, a buildworld/installworld is required for libusb20. 20090223: The new USB2 stack has now been permanently moved in and all kernel and module names reverted to their previous values (eg, usb, ehci, ohci, ums, ...). The old usb stack can be compiled in by prefixing the name with the letter 'o', the old usb modules have been removed. Updating entry 20090216 for xorg and 20090215 for libmap may still apply. 20090217: The rc.conf(5) option if_up_delay has been renamed to defaultroute_delay to better reflect its purpose. If you have customized this setting in /etc/rc.conf you need to update it to use the new name. 20090216: xorg 7.4 wants to configure its input devices via hald which does not yet work with USB2. If the keyboard/mouse does not work in xorg then add Option "AllowEmptyInput" "off" to your ServerLayout section. This will cause X to use the configured kbd and mouse sections from your xorg.conf. 20090215: The GENERIC kernels for all architectures now default to the new USB2 stack. No kernel config options or code have been removed so if a problem arises please report it and optionally revert to the old USB stack. If you are loading USB kernel modules or have a custom kernel that includes GENERIC then ensure that usb names are also changed over, eg uftdi -> usb2_serial_ftdi. Older programs linked against the ports libusb 0.1 need to be redirected to the new stack's libusb20. /etc/libmap.conf can be used for this: # Map old usb library to new one for usb2 stack libusb-0.1.so.8 libusb20.so.1 20090209: All USB ethernet devices now attach as interfaces under the name ueN (eg. ue0). This is to provide a predictable name as vendors often change usb chipsets in a product without notice. 20090203: The ichsmb(4) driver has been changed to require SMBus slave addresses be left-justified (xxxxxxx0b) rather than right-justified. All of the other SMBus controller drivers require left-justified slave addresses, so this change makes all the drivers provide the same interface. 20090201: INET6 statistics (struct ip6stat) was updated. netstat(1) needs to be recompiled. 20090119: NTFS has been removed from GENERIC kernel on amd64 to match GENERIC on i386. Should not cause any issues since mount_ntfs(8) will load ntfs.ko module automatically when NTFS support is actually needed, unless ntfs.ko is not installed or security level prohibits loading kernel modules. If either is the case, "options NTFS" has to be added into kernel config. 20090115: TCP Appropriate Byte Counting (RFC 3465) support added to kernel. New field in struct tcpcb breaks ABI, so bump __FreeBSD_version to 800061. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20081225: ng_tty(4) module updated to match the new TTY subsystem. Due to API change, user-level applications must be updated. New API support added to mpd5 CVS and expected to be present in next mpd5.3 release. 20081219: With __FreeBSD_version 800060 the makefs tool is part of the base system (it was a port). 20081216: The afdata and ifnet locks have been changed from mutexes to rwlocks, network modules will need to be re-compiled. 20081214: __FreeBSD_version 800059 incorporates the new arp-v2 rewrite. RTF_CLONING, RTF_LLINFO and RTF_WASCLONED flags are eliminated. The new code reduced struct rtentry{} by 16 bytes on 32-bit architecture and 40 bytes on 64-bit architecture. The userland applications "arp" and "ndp" have been updated accordingly. The output from "netstat -r" shows only routing entries and none of the L2 information. 20081130: __FreeBSD_version 800057 marks the switchover from the binary ath hal to source code. Users must add the line: options AH_SUPPORT_AR5416 to their kernel config files when specifying: device ath_hal The ath_hal module no longer exists; the code is now compiled together with the driver in the ath module. It is now possible to tailor chip support (i.e. reduce the set of chips and thereby the code size); consult ath_hal(4) for details. 20081121: __FreeBSD_version 800054 adds memory barriers to , new interfaces to ifnet to facilitate multiple hardware transmit queues for cards that support them, and a lock-less ring-buffer implementation to enable drivers to more efficiently manage queueing of packets. 20081117: A new version of ZFS (version 13) has been merged to -HEAD. This version has zpool attribute "listsnapshots" off by default, which means "zfs list" does not show snapshots, and is the same as Solaris behavior. 20081028: dummynet(4) ABI has changed. ipfw(8) needs to be recompiled. 20081009: The uhci, ohci, ehci and slhci USB Host controller drivers have been put into separate modules. If you load the usb module separately through loader.conf you will need to load the appropriate *hci module as well. E.g. for a UHCI-based USB 2.0 controller add the following to loader.conf: uhci_load="YES" ehci_load="YES" 20081009: The ABI used by the PMC toolset has changed. Please keep userland (libpmc(3)) and the kernel module (hwpmc(4)) in sync. 20081009: atapci kernel module now includes only generic PCI ATA driver. AHCI driver moved to ataahci kernel module. All vendor-specific code moved into separate kernel modules: ataacard, ataacerlabs, ataadaptec, ataamd, ataati, atacenatek, atacypress, atacyrix, atahighpoint, ataintel, ataite, atajmicron, atamarvell, atamicron, atanational, atanetcell, atanvidia, atapromise, ataserverworks, atasiliconimage, atasis, atavia 20080820: The TTY subsystem of the kernel has been replaced by a new implementation, which provides better scalability and an improved driver model. Most common drivers have been migrated to the new TTY subsystem, while others have not. The following drivers have not yet been ported to the new TTY layer: PCI/ISA: cy, digi, rc, rp, sio USB: ubser, ucycom Line disciplines: ng_h4, ng_tty, ppp, sl, snp Adding these drivers to your kernel configuration file shall cause compilation to fail. 20080818: ntpd has been upgraded to 4.2.4p5. 20080801: OpenSSH has been upgraded to 5.1p1. For many years, FreeBSD's version of OpenSSH preferred DSA over RSA for host and user authentication keys. With this upgrade, we've switched to the vendor's default of RSA over DSA. This may cause upgraded clients to warn about unknown host keys even for previously known hosts. Users should follow the usual procedure for verifying host keys before accepting the RSA key. This can be circumvented by setting the "HostKeyAlgorithms" option to "ssh-dss,ssh-rsa" in ~/.ssh/config or on the ssh command line. Please note that the sequence of keys offered for authentication has been changed as well. You may want to specify IdentityFile in a different order to revert this behavior. 20080713: The sio(4) driver has been removed from the i386 and amd64 kernel configuration files. This means uart(4) is now the default serial port driver on those platforms as well. To prevent collisions with the sio(4) driver, the uart(4) driver uses different names for its device nodes. This means the onboard serial port will now most likely be called "ttyu0" instead of "ttyd0". You may need to reconfigure applications to use the new device names. When using the serial port as a boot console, be sure to update /boot/device.hints and /etc/ttys before booting the new kernel. If you forget to do so, you can still manually specify the hints at the loader prompt: set hint.uart.0.at="isa" set hint.uart.0.port="0x3F8" set hint.uart.0.flags="0x10" set hint.uart.0.irq="4" boot -s 20080609: The gpt(8) utility has been removed. Use gpart(8) to partition disks instead. 20080603: The version that Linuxulator emulates was changed from 2.4.2 to 2.6.16. If you experience any problems with Linux binaries please try to set sysctl compat.linux.osrelease to 2.4.2 and if it fixes the problem contact emulation mailing list. 20080525: ISDN4BSD (I4B) was removed from the src tree. You may need to update a your kernel configuration and remove relevant entries. 20080509: I have checked in code to support multiple routing tables. See the man pages setfib(1) and setfib(2). This is a hopefully backwards compatible version, but to make use of it you need to compile your kernel with options ROUTETABLES=2 (or more up to 16). 20080420: The 802.11 wireless support was redone to enable multi-bss operation on devices that are capable. The underlying device is no longer used directly but instead wlanX devices are cloned with ifconfig. This requires changes to rc.conf files. For example, change: ifconfig_ath0="WPA DHCP" to wlans_ath0=wlan0 ifconfig_wlan0="WPA DHCP" see rc.conf(5) for more details. In addition, mergemaster of /etc/rc.d is highly recommended. Simultaneous update of userland and kernel wouldn't hurt either. As part of the multi-bss changes the wlan_scan_ap and wlan_scan_sta modules were merged into the base wlan module. All references to these modules (e.g. in kernel config files) must be removed. 20080408: psm(4) has gained write(2) support in native operation level. Arbitrary commands can be written to /dev/psm%d and status can be read back from it. Therefore, an application is responsible for status validation and error recovery. It is a no-op in other operation levels. 20080312: Support for KSE threading has been removed from the kernel. To run legacy applications linked against KSE libmap.conf may be used. The following libmap.conf may be used to ensure compatibility with any prior release: libpthread.so.1 libthr.so.1 libpthread.so.2 libthr.so.2 libkse.so.3 libthr.so.3 20080301: The layout of struct vmspace has changed. This affects libkvm and any executables that link against libkvm and use the kvm_getprocs() function. In particular, but not exclusively, it affects ps(1), fstat(1), pkill(1), systat(1), top(1) and w(1). The effects are minimal, but it's advisable to upgrade world nonetheless. 20080229: The latest em driver no longer has support in it for the 82575 adapter, this is now moved to the igb driver. The split was done to make new features that are incompatible with older hardware easier to do. 20080220: The new geom_lvm(4) geom class has been renamed to geom_linux_lvm(4), likewise the kernel option is now GEOM_LINUX_LVM. 20080211: The default NFS mount mode has changed from UDP to TCP for increased reliability. If you rely on (insecurely) NFS mounting across a firewall you may need to update your firewall rules. 20080208: Belatedly note the addition of m_collapse for compacting mbuf chains. 20080126: The fts(3) structures have been changed to use adequate integer types for their members and so to be able to cope with huge file trees. The old fts(3) ABI is preserved through symbol versioning in libc, so third-party binaries using fts(3) should still work, although they will not take advantage of the extended types. At the same time, some third-party software might fail to build after this change due to unportable assumptions made in its source code about fts(3) structure members. Such software should be fixed by its vendor or, in the worst case, in the ports tree. FreeBSD_version 800015 marks this change for the unlikely case that a portable fix is impossible. 20080123: To upgrade to -current after this date, you must be running FreeBSD not older than 6.0-RELEASE. Upgrading to -current from 5.x now requires a stop over at RELENG_6 or RELENG_7 systems. 20071128: The ADAPTIVE_GIANT kernel option has been retired because its functionality is the default now. 20071118: The AT keyboard emulation of sunkbd(4) has been turned on by default. In order to make the special symbols of the Sun keyboards driven by sunkbd(4) work under X these now have to be configured the same way as Sun USB keyboards driven by ukbd(4) (which also does AT keyboard emulation), f.e.: Option "XkbLayout" "us" Option "XkbRules" "xorg" Option "XkbSymbols" "pc(pc105)+sun_vndr/usb(sun_usb)+us" 20071024: It has been decided that it is desirable to provide ABI backwards compatibility to the FreeBSD 4/5/6 versions of the PCIOCGETCONF, PCIOCREAD and PCIOCWRITE IOCTLs, which was broken with the introduction of PCI domain support (see the 20070930 entry). Unfortunately, this required the ABI of PCIOCGETCONF to be broken again in order to be able to provide backwards compatibility to the old version of that IOCTL. Thus consumers of PCIOCGETCONF have to be recompiled again. As for prominent ports this affects neither pciutils nor xorg-server this time, the hal port needs to be rebuilt however. 20071020: The misnamed kthread_create() and friends have been renamed to kproc_create() etc. Many of the callers already used kproc_start().. I will return kthread_create() and friends in a while with implementations that actually create threads, not procs. Renaming corresponds with version 800002. 20071010: RELENG_7 branched. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. While generally safe, there are sometimes problems using -j to upgrade. If your upgrade fails with -j, please try again without -j. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. When upgrading from one major version to another it is generally best to upgrade to the latest code in the currently installed branch first, then do an upgrade to the new branch. This is the best-tested upgrade path, and has the highest probability of being successful. Please try this approach before reporting problems with a major version upgrade. When upgrading a live system, having a root shell around before installing anything can help undo problems. Not having a root shell around can lead to problems if pam has changed too much from your starting point to allow continued authentication after the upgrade. ZFS notes --------- When upgrading the boot ZFS pool to a new version, always follow these two steps: 1.) recompile and reinstall the ZFS boot loader and boot block (this is part of "make buildworld" and "make installworld") 2.) update the ZFS boot block on your boot drive The following example updates the ZFS boot block on the first partition (freebsd-boot) of a GPT partitioned drive ad0: "gpart bootcode -p /boot/gptzfsboot -i 1 ad0" Non-boot pools do not need these updates. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. It is the most failsafe as it uses a /usr/obj tree with a fresh mini-buildworld, make kernel-toolchain make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To test a kernel once --------------------- If you just want to boot a kernel once (because you are not sure if it works, or if you want to boot a known bad kernel to provide debugging information) run make installkernel KERNCONF=YOUR_KERNEL_HERE KODIR=/boot/testkernel nextboot -k testkernel To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a CURRENT system. Replace ${arch} with the architecture of your machine (e.g. "i386", "arm", "amd64", "ia64", "pc98", "sparc64", "powerpc", "mips", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. make buildworld make kernel KERNCONF=YOUR_KERNEL_HERE [1] [3] mergemaster -p [5] make installworld mergemaster -i [4] make delete-old [6] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installworld DESTDIR=${CURRENT_ROOT} make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from stable to current ---------------------------------------------- make buildworld [9] make kernel KERNCONF=YOUR_KERNEL_HERE [8] [1] [3] mergemaster -p [5] make installworld mergemaster -i [4] make delete-old [6] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. The -U option is also useful to consider. See mergemaster(8) for more information. [5] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. You may need to install a new mergemaster (cd src/usr.sbin/mergemaster && make install) after the buildworld before this step if you last updated from current before 20130425 or from -stable before 20130430. [6] This only deletes old files and directories. Old libraries can be deleted by "make delete-old-libs", but you have to make sure that no program is using those libraries anymore. [8] In order to have a kernel that can run the 4.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD4 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. A similar kernel option COMPAT_FREEBSD5 is required to run the 5.x binaries on more recent kernels. And so on for COMPAT_FREEBSD6 and COMPAT_FREEBSD7. Make sure that you merge any new devices from GENERIC since the last time you updated your kernel config file. [9] When checking out sources, you must include the -P flag to have cvs prune empty directories. If CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. MAKEOBJDIRPREFIX must be defined in an environment variable, and not on the command line, or in /etc/make.conf. buildworld will warn if it is improperly defined. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -current. It is not guaranteed to be a complete list of such breakages, and only contains entries since October 10, 2007. If you need to see UPDATING entries from before that date, you will need to fetch an UPDATING file from an older FreeBSD release. Copyright information: Copyright 1998-2009 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: releng/10.1/contrib/file/ChangeLog =================================================================== --- releng/10.1/contrib/file/ChangeLog (revision 284192) +++ releng/10.1/contrib/file/ChangeLog (revision 284193) @@ -1,1395 +1,1489 @@ +2015-01-02 15:15 Christos Zoulas + + * release 5.22 + +2015-01-01 12:01 Christos Zoulas + + * add indirect relative for TIFF/Exif + +2014-12-16 18:10 Christos Zoulas + + * restructure elf note printing to avoid repeated messages + * add note limit, suggested by Alexander Cherepanov + +2014-12-16 16:53 Christos Zoulas + + * Bail out on partial pread()'s (Alexander Cherepanov) + * Fix incorrect bounds check in file_printable (Alexander Cherepanov) + +2014-12-11 20:01 Christos Zoulas + + * PR/405: ignore SIGPIPE from uncompress programs + * change printable -> file_printable and use it in + more places for safety + * in ELF, instead of "(uses dynamic libraries)" when PT_INTERP + is present print the interpreter name. + +2014-12-10 20:01 Christos Zoulas + + * release 5.21 + +2014-11-27 18:40 Christos Zoulas + + * Allow setting more parameters from the command line. + * Split name/use and indirect magic recursion limits. + +2014-11-27 11:12 Christos Zoulas + + * Adjust ELF parameters and the default recursion + level. + * Allow setting the recursion level dynamically. + +2014-11-24 8:55 Christos Zoulas + + * The following fixes resulted from Thomas Jarosch's fuzzing + tests that revealed severe performance issues on pathological + input: + - limit number of elf program and sections processing + - abort elf note processing quickly + - reduce the number of recursion levels from 20 to 10 + - preserve error messages in indirect magic handling + + This is tracked as CVE-2014-8116 and CVE-2014-8117 + +2014-11-12 10:30 Christos Zoulas + + * fix bogus free in the user buffer case. + +2014-11-11 12:35 Christos Zoulas + + * fix out of bounds read for pascal strings + * fix memory leak (not freeing the head of each mlist) + +2014-11-07 10:25 Christos Zoulas + + * When printing strings from a file, convert them to printable + on a byte by byte basis, so that we don't get issues with + locale's trying to interpret random byte streams as UTF-8 and + having printf error out with EILSEQ. + +2014-10-17 11:48 Christos Zoulas + + * fix bounds in note reading (Francisco Alonso / Red Hat) + +2014-10-11 15:02 Christos Zoulas + + * fix autoconf glue for setlocale and locale_t; some OS's + have locale_t in xlocale.h + +2014-10-10 15:01 Christos Zoulas + + * release 5.20 + +2014-08-17 10:01 Christos Zoulas + + * recognize encrypted CDF documents + +2014-08-04 9:18 Christos Zoulas + + * add magic_load_buffers from Brooks Davis + +2014-07-24 16:40 Christos Zoulas + + * add thumbs.db support + 2014-06-12 12:28 Christos Zoulas * release 5.19 2014-06-09 9:04 Christos Zoulas * Misc buffer overruns and missing buffer size tests in cdf parsing (Francisco Alonso, Jan Kaluza) 2014-06-02 14:50 Christos Zoulas * Enforce limit of 8K on regex searches that have no limits * Allow the l modifier for regex to mean line count. Default to byte count. If line count is specified, assume a max of 80 characters per line to limit the byte count. * Don't allow conversions to be used for dates, allowing the mask field to be used as an offset. 2014-05-30 12:51 Christos Zoulas * Make the range operator limit the length of the regex search. 2014-05-14 19:23 Christos Zoulas * PR/347: Windows fixes * PR/352: Hangul word processor recognition * PR/354: Encoding irregularities in text files 2014-05-06 6:12 Christos Zoulas * Fix uninitialized title in CDF files (Jan Kaluza) 2014-05-04 14:55 Christos Zoulas * PR/351: Fix compilation of empty files 2014-04-30 17:39 Christos Zoulas * Fix integer formats: We don't specify 'l' or 'h' and 'hh' specifiers anymore, only 'll' for quads and nothing for the rest. This is so that magic writing is simpler. 2014-04-01 15:25 Christos Zoulas * PR/341: Jan Kaluza, fix memory leak * PR/342: Jan Kaluza, fix out of bounds read 2014-03-28 15:25 Christos Zoulas * Fix issue with long formats not matching fmtcheck 2014-03-26 11:25 Christos Zoulas * release 5.18 2014-03-15 17:45 Christos Zoulas * add fmtcheck(3) for those who don't have it 2014-03-14 15:12 Christos Zoulas * prevent mime entries from being attached to magic entries with no descriptions * adjust magic strength for regex type * remove superfluous ascmagic with encoding test 2014-03-06 12:01 Christos Zoulas * fix regression fix echo -ne "\012\013\014" | file -i - which printed "binary" instead of "application/octet-stream" * add size_t overflow check for magic file size 2014-02-27 16:01 Christos Zoulas * experimental support for matching with CFD CLSID 2014-02-18 13:04 Kimmo Suominen (kimmo@suominen.com) * Cache old LC_CTYPE locale before setting it to "C", so we can use it to restore LC_CTYPE instead of asking setlocale() to scan the environment variables. 2014-02-12 18:21 Christos Zoulas * Count recursion levels through indirect magic 2014-02-11 10:40 Christos Zoulas * Prevent infinite recursion on files with indirect offsets of 0 2014-01-30 21:00 Christos Zoulas * Add -E flag that makes file print filesystem errors to stderr and exit. 2014-01-08 17:20 Christos Zoulas * mime printing could print results from multiple magic entries if there were multiple matches. * in some cases overflow was not detected when computing offsets in softmagic. 2013-12-05 12:00 Christos Zoulas * use strcasestr() to for cdf strings * reset to the "C" locale while doing regex operations, or case insensitive comparisons; this is provisional 2013-11-19 20:10 Christos Zoulas * always leave magic file loaded, don't unload for magic_check, etc. * fix default encoding to binary instead of unknown which broke recently * handle empty and one byte files, less specially so that --mime-encoding does not break completely. ` 2013-11-06 14:40 Christos Zoulas * fix erroneous non-zero exit code from non-existant file and message 2013-10-29 14:25 Christos Zoulas * add CDF MSI file detection (Guy Helmer) 2013-09-03 11:56 Christos Zoulas * Don't mix errors and regular output if there was an error * in magic_descriptor() don't close the file and try to restore its position 2013-05-30 17:25 Christos Zoulas * Don't treat magic as an error if offset was past EOF (Christoph Biedl) 2013-05-28 17:25 Christos Zoulas * Fix spacing issues in softmagic and elf (Jan Kaluza) 2013-05-02 18:00 Christos Zoulas * Fix segmentation fault with multiple magic_load commands. 2013-04-22 11:20 Christos Zoulas * The way "default" was implemented was not very useful because the "if something was printed at that level" was not easily controlled by the user, and the format was bound to a string which is too restrictive. Add a "clear" for that level keyword and make "default" void. This way one can do: >>13 clear x >>13 lelong 1 foo >>13 lelong 2 bar >>13 default x >>>13 lelong x unknown %x 2013-03-25 13:20 Christos Zoulas * disallow strength setting in "name" entries 2013-03-06 21:24 Christos Zoulas * fix recursive magic separator printing 2013-02-26 19:28 Christos Zoulas * limit recursion level for mget * fix pread() related breakage in cdf * handle offsets properly in recursive "use" 2013-02-18 10:39 Christos Zoulas * add elf reading of debug info to determine if file is stripped (Jan Kaluza) * use pread() 2013-01-25 18:05 Christos Zoulas * change mime description size from 64 to 80 to accommodate OOXML. 2013-01-11 14:50 Christos Zoulas * Warn about inconsistent continuation levels. * Change fsmagic to add a space after it prints. 2013-01-10 21:00 Christos Zoulas * Make getline public so that file can link against it. Perhaps it is better to rename it, or hide it differently. Fixes builds on platforms that do not provide it. 2013-01-07 16:30 Christos Zoulas * Add SuS d{,1,2,4,8}, u{,1,2,4,8} and document what long, int, short, etc is (Guy Harris) 2013-01-06 11:20 Christos Zoulas * add magic_version function and constant * Redo memory allocation and de-allocation. (prevents double frees on non mmap platforms) * Fix bug with name/use having to do with passing found state from the parent to the child and back. 2012-12-19 8:47 Christos Zoulas * Only print elf capabilities for archs we know (Jan Kaluza) 2012-10-30 19:14 Christos Zoulas * Add "name" and "use" file types in order to look inside mach-o files. 2012-09-06 10:40 Christos Zoulas * make --version exit 0 (Matthew Schultz) * add string/T (Jan Kaluza) 2012-08-09 2:15 Christos Zoulas * add z and t modifiers for our own vasprintf * search for $HOME/.magic.mgc if it is there first * fix reads from a pipe, and preserve errno 2012-05-15 13:12 Christos Zoulas * use ctime_r, asctime_r 2012-04-06 17:18 Christos Zoulas * Fixes for indirect offsets to handle apple disk formats 2012-04-03 18:26 Christos Zoulas * Add windows date field types * More info for windows shortcuts (incomplete) 2012-02-20 17:33 Christos Zoulas * Fix CDF parsing issues found by CERT's fuzzing tool (Will Dormann) 2011-12-15 12:17 Chris Metcalf * Support Tilera architectures (tile64, tilepro, tilegx). 2011-12-16 16:33 Reuben Thomas * Add magic for /usr/bin/env Perl scripts * Weaken generic script magic to avoid clashing with language-specific magic. 2011-12-08 13:37 Reuben Thomas * Simplify if (p) free(p) to free(p). 2011-12-08 13:07 Reuben Thomas * Remove hardwired token finding (names.h), turning it into soft magic. Patterns are either anchored regexs or search/8192. English language detection and PL/1 detection have been removed as they were too fragile. -e tokens is still accepted for backwards compatibility. * Move 3ds patterns (which are commented out anyway) into autodesk (they were, oddly, in c-lang). 2011-12-06 00:16 Reuben Thomas * Tweak strength of generic hash-bang detectors to be less than specific ones. * Make an inconsistent description of Python scripts consistent. 2011-12-05 23:58 Reuben Thomas * Fix minor error in file(1). 2011-11-05 00:00 Reuben Thomas * Fix issue #150 (I hope). 2011-09-22 12:57 Christos Zoulas * Python3 binding fixes from Kelly Anderson 2011-09-20 11:32 Christos Zoulas * If a string type magic entry is marked as text or binary only match text files against text entries and binary files against binary entries. 2011-09-01 12:12 Christos Zoulas * Don't wait for any subprocess, just the one we forked. 2011-08-26 16:40 Christos Zoulas * If the application name is not set in a cdf file, try to see if it has a directory with the application name on it. 2011-08-17 14:32 Christos Zoulas * Fix ELF lseek(2) madness. Inspired by PR/134 by Jan Kaluza 2011-08-14 09:03 Christos Zoulas * Don't use variable string formats. 2011-07-12 12:32 Reuben Thomas * Fix detection of Zip files (Mantis #128). * Make some minor improvements to file(1). * Rename MIME types for filesystem objects for consistency with xdg-utils. Typically this means that application/x-foo becomes inode/foo, but some names also change slightly, e.g. application/x-character-device becomes inode/chardevice. 2011-05-10 20:57 Christos Zoulas * fix mingw compilation (Abradoks) 2011-05-10 20:57 Christos Zoulas * remove patchlevel.h * Fix read past allocated memory caused by double-incrementing a pointer in a loop (reported by Roberto Maar) 2011-03-30 15:45 Christos Zoulas * Fix cdf string buffer setting (Sven Anders) 2011-03-20 16:35 Christos Zoulas * Eliminate MAXPATHLEN and use dynamic allocation for path and file buffers. 2011-03-15 18:15 Christos Zoulas * binary tests on magic entries with masks could spuriously get converted to ascii. 2011-03-12 18:06 Reuben Thomas * Improve file.man (remove BUGS, present email addresses consistently). 2011-03-07 19:38 Christos Zoulas * add lrzip support (from Ville Skytta) 2011-02-10 16:36 Christos Zoulas * fix CDF bounds checking (Guy Helmer) 2011-02-10 12:03 Christos Zoulas * add cdf_ctime() that prints a meaningful error when time cannot be converted. 2011-02-02 20:40 Christos Zoulas * help and version output to stdout. * When matching softmagic for ascii files, don't just print the softmagic classification, keep going and print the text classification too. This fixes broken troff files when we moved them from keyword recognition to softmagic (they stopped printing "with CRLF" etc.) Reported by Doug McIlroy. 2011-01-16 19:31 Reuben Thomas * Fix two potential buffer overruns in apprentice_list. 2011-01-14 22:33 Reuben Thomas * New Python binding in pure Python. * Update libmagic(3). 2011-01-06 21:40 Reuben Thomas * Fix Python bindings (including recent Python 3 compatibility update). 2011-01-04 18:43 Reuben Thomas * magic/Makefile.am: make it easier to recover from magic build failures. * Fix pstring length specifier parsing to avoid generating invalid magic files. * Add pstring length "J" (for "JPEG") to specify that the length include itself. * Fix JPEG comment parsing at last using pstring/HJ! * Ignore section 5 man pages in doc/.cvsignore. 2010-12-22 13:12 Christos Zoulas * Add pstring/BHhLl to specify the type of the length of pascal strings. 2010-11-26 18:39 Reuben Thomas * Fix "-e soft": it was ignored when softmagic was called during asciimagic. * Improve comments and use "unsigned char" in tar.h/is_tar.c. 2010-11-05 17:26 Reuben Thomas * Make bug reporting addresses more visible. 2010-11-01 18:35 Reuben Thomas * Add tcl magic from Gustaf Neumann 2010-10-24 10:42 Christos Zoulas * Fix the whitespace comparing code (Christopher Chittleborough) 2010-10-06 21:05 Christos Zoulas * allow string/t to work (Jan Kaluza) 2010-09-20 22:11 Reuben Thomas * Apply some patches from Ubuntu and Fedora. 2010-09-20 21:16 Reuben Thomas * Apply all patches from Debian package 5.04-6 which have not already been applied and are not Debian-specific. 2010-09-20 15:24 Reuben Thomas * Minor security fix to softmagic.c (don't use untrusted string as printf format). 2010-07-21 12:20 Christos Zoulas * MINGW32 portability from LRN * Don't warn about escaping magic regex chars when we are in a regex. 2010-07-19 10:55 Christos Zoulas * Only try to print prpsinfo for core files. (Jan Kaluza) 2010-04-22 12:55 Christos Zoulas * Try more elf offsets for Debian core files. (Arnaud Giersch) 2010-02-20 15:18 Reuben Thomas * Clarify which sort of CDF we mean. 2010-02-14 22:58 Reuben Thomas * Re-jig Zip file type magic so that unsupported special Zip types (those with "mimetype" at offset 30) can be recognized. 2010-02-02 21:50 Reuben Thomas * Add support for OCF (EPUB) files (application/epub+zip) 2010-01-28 18:25 Christos Zoulas * Fix core-dump from unbound loop: https://bugzilla.redhat.com/show_bug.cgi?id=533245 2010-01-22 15:45 Christos Zoulas * print proper mime for crystal reports file * print the last summary information of a cdf document, not the first so that nested documents print the right info 2010-01-16 18:42 Charles Longeau * bring back some fixes from OpenBSD: - make gcc2 builds file - fix typos in a magic file comment 2009-11-17 18:35 Christos Zoulas * ctime/asctime can return NULL on some OS's although they should not (Toshit Antani) 2009-09-14 13:49 Christos Zoulas * Centralize magic path handling routines and remove the special-casing from file.c so that the python module for example comes up with the same magic path (Fixes ~/.magic handling) (from Gab) 2009-09-11 23:38 Reuben Thomas * When magic argument is a directory, read the files in strcmp-sorted order (fixes Debian bug #488562 and our own FIXME). 2009-09-11 13:11 Reuben Thomas * Combine overlapping epoc and psion magic files into one (epoc). * Add some more EPOC MIME types. 2009-08-19 15:55 Christos Zoulas * Fix 3 bugs (From Ian Darwin): - file_showstr could move one past the end of the array - parse_apple did not nul terminate the string in the overflow case - parse_mime truncated the wrong string in the overflow case 2009-08-12 12:28 Robert Byrnes * Include Localstuff when compiling magic. 2009-07-15 10:05 Christos Zoulas * Fix logic for including mygetopts.h * Make cdf.c compile again with debugging * Add the necessary field handling for crystal reports files to work 2009-06-23 01:34 Reuben Thomas * Stop "(if" identifying Lisp files, that's plain dumb! 2009-06-09 22:13 Reuben Thomas * Add a couple of missing MP3 MIME types. 2009-05-27 23:00 Reuben Thomas * Add full range of hash-bang tests for Python and Ruby. * Add MIME types for Python and Ruby scripts. 2009-05-13 10:44 Christos Zoulas * off by one in parsing hw capabilities in elf (Cheng Renquan) 2009-05-08 13:40 Christos Zoulas * lint fixes and more from NetBSD 2009-05-06 10:25 Christos Zoulas * Avoid null dereference in cdf code (Drew Yao) * More cdf bounds checks and overflow checks 2009-05-01 18:37 Christos Zoulas * Buffer overflow fixes from Drew Yao 2009-04-30 17:10 Christos Zoulas * Fix more cdf lossage. All the documents I have right now print the correct information. 2009-03-27 18:43 Christos Zoulas * don't print \012- separators in the same magic entry if it consists of multiple magic printing lines. 2009-03-23 10:20 Christos Zoulas * Avoid file descriptor leak in compress code from (Daniel Novotny) 2009-03-18 16:50 Christos Zoulas * Allow escaping of relation characters, so that we can say \^[A-Z] and the ^ is not eaten as a relation char. * Fix troff and fortran to their previous glory using regex. This was broken since their removel from ascmagic. 2009-03-10 16:50 Christos Zoulas * don't use strlen in strndup() (Toby Peterson) 2009-03-10 7:45 Christos Zoulas * avoid c99 syntax. 2009-02-23 15:45 Christos Zoulas * make the cdf code use the buffer first if available, and then the fd code. 2009-02-13 13:45 Christos Zoulas * look for struct option to determine if getopt.h is usable for IRIX. * sanitize cdf document strings 2009-02-04 13:25 Christos Zoulas * fix OS/2 warnings. 2008-12-12 15:50 Christos Zoulas * fix initial offset calculation for non 4K sector files * add loop limits to avoid DoS attacks by constructing looping sector references. 2008-12-03 13:05 Christos Zoulas * fix memory botches on cdf file parsing. * exit with non-zero value for any error, not just for the last file processed. 2008-11-09 20:42 Charles Longeau * Replace all str{cpy,cat} functions with strl{cpy,cat} * Ensure that strl{cpy,cat} are included in libmagic, as needed. 2008-11-06 18:18 Christos Zoulas * Handle ID3 format files. 2008-11-06 23:00 Reuben Thomas * Fix --mime, --mime-type and --mime-encoding under new scheme. * Rename "ascii" to "text" and add "encoding" test. * Return a precise ("utf-16le" or "utf-16be") MIME charset for UTF-16. * Fix error in comment caused by automatic indentation adding words! 2008-11-06 10:35 Christos Zoulas * use memchr instead of strchr because the string might not be NUL terminated (Scott MacVicar) 2008-11-03 07:31 Reuben Thomas * Fix a printf with a non-literal format string. * Fix formatting and punctuation of help for "--apple". 2008-10-30 11:00 Reuben Thomas * Correct words counts in comments of struct magic. * Fix handle_annotation to allow both Apple and MIME types to be printed, and to return correct code if MIME type is printed (1, not 0) or if there's an error (-1 not 1). * Fix output of charset for MIME type (precede with semi-colon; fixes Debian bug #501460). * Fix potential attacks via conversion specifications in magic strings. * Add a FIXME for Debian bug #488562 (magic files should be read in a defined order, by sorting the names). 2008-10-18 16:45 Christos Zoulas * Added APPLE file creator/type 2008-10-12 10:20 Christos Zoulas * Added CDF parsing 2008-10-09 16:40 Christos Zoulas * filesystem and msdos patches (Joerg Jenderek) 2008-10-09 13:20 Christos Zoulas * correct --exclude documentation issues: remove troff and fortran and rename "token" to "tokens". (Randy McMurchy) 2008-10-01 10:30 Christos Zoulas * Read ~/.magic in addition to the default magic file not instead of, as documented in the man page. 2008-09-10 21:30 Reuben Thomas * Comment out graphviz patterns, as they match too many files. 2008-08-30 12:54 Christos Zoulas * Don't eat trailing \n in magic enties. * Cast defines to allow compilation using a c++ compiler. 2008-08-25 23:56 Reuben Thomas * Add text/x-lua MIME type for Lua scripts. * Escape { in regex in graphviz patterns. 2008-07-26 00:59 Reuben Thomas * Add MIME types for special files. * Use access to give more accurate information for files that can't be opened. * Add a TODO list. 2008-07-02 11:15 Christos Zoulas * add !:strength op to adjust magic strength (experimental) 2008-06-16 21:41 Reuben Thomas * Fix automake error in configure.ac. * Add MIME type for Psion Sketch files. 2008-06-05 08:59 Christos Zoulas * Don't print warnings about bad namesize in stripped binaries with PT_NOTE is still there, and the actual note is gone (Jakub Jelinek) 2008-05-28 15:12 Robert Byrnes * magic/Magdir/elf: Note invalid byte order for little-endian SPARC32PLUS. Add SPARC V9 vendor extensions and memory model. * src/elfclass.h: Pass target machine to doshn (for Solaris hardware capabilities). * src/readelf.c (doshn): Add support for Solaris hardware/software capabilities. * src/readelf.h: Ditto. * src/vasprintf.c (dispatch): Add support for ll modifier. 2008-05-16 10:25 Christos Zoulas * Fix compiler warnings. * remove stray printf, and fix a vprintf bug. (Martin Dorey) 2008-05-06 00:13 Robert Byrnes * src/Makefile.am: Ensure that getopt_long and [v]asprintf are included in libmagic, as needed. Remove unnecessary EXTRA_DIST. * src/Makefile.in: Rerun automake. * src/vasprintf.c (dispatch): Fix variable precision bug: be sure to step past '*'. * src/vasprintf.c (core): Remove unreachable code. * src/apprentice.c (set_test_type): Add cast to avoid compiler warning. 2008-04-22 23:45 Christos Zoulas * Add magic submission guidelines (Abel Cheung) * split msdos and windows magic (Abel Cheung) 2008-04-04 11:00 Christos Zoulas * >= <= is not supported, so fix the magic and warn about it. reported by: Thien-Thi Nguyen 2008-03-27 16:16 Robert Byrnes * src/readelf.c (donote): ELF core file command name/line bug fixes and enhancements: Try larger offsets first to avoid false matches from earlier data that happen to look like strings; this primarily affected SunOS 5.x 32-bit Intel core files. Add support for command line (instead of just short name) for SunOS 5.x. Add information about NT_PSINFO for SunOS 5.x. Only trim whitespace from end of command line. 2007-02-11 01:36 Reuben Thomas * Change strength of ! from MULT to 0, as it matches almost anything (Reuben Thomas) * Debian fixes (Reuben Thomas) 2007-02-11 00:17 Reuben Thomas * Clarify UTF-8 BOM message (Reuben Thomas) * Add HTML comment to token list in names.h 2007-02-04 15:50 Christos Zoulas * Debian fixes (Reuben Thomas) 2007-02-04 11:31 Christos Zoulas * !:mime annotations in magic files (Reuben Thomas) 2007-01-29 15:35 Christos Zoulas * zero out utime/utimes structs (Gavin Atkinson) 2007-01-26 13:45 Christos Zoulas * reduce writable data from Diego "Flameeyes" Petten 2007-12-28 15:06 Christos Zoulas * strtof detection * remove bogus regex magic that could cause a DoS * better mismatch version message 2007-12-27 11:35 Christos Zoulas * bring back some fixes from OpenBSD * treat ELF dynamic objects as executables * fix gcc warnings 2007-12-01 19:55 Christos Zoulas * make sure we have zlib.h and libz to compile the builtin decompress code 2007-10-28 20:48 Christos Zoulas * float and double magic support (Behan Webster) 2007-10-28 20:48 Christos Zoulas * Convert fortran to a soft test (Reuben Thomas) 2007-10-23 5:25 Christos Zoulas * Add --with-filename, and --no-filename (Reuben Thomas) 2007-10-23 3:59 Christos Zoulas * Rest of the mime split (Reuben Thomas) * Make usage message generated from the flags so that they stay consistent (Reuben Thomas) 2007-10-20 3:06 Christos Zoulas * typo in comment, missing ifdef QUICK, remove unneeded code (Charles Longeau) 2007-10-17 3:33 Christos Zoulas * Fix problem printing -\012 in some entries * Separate magic type and encoding flags (Reuben Thomas) 2007-10-09 3:55 Christos Zoulas * configure fix for int64 and strndup (Reuben Thomas) 2007-09-26 4:45 Christos Zoulas * Add magic_descriptor() function. * Fix regression in elf reading code where the core name was not being printed. * Don't convert NUL's to spaces in {l,b}estring16 (Daniel Dawson) 2007-08-19 6:30 Christos Zoulas * Make mime format consistent so that it can be easily parsed: mimetype [charset=character-set] [encoding=encoding-mime-type] Remove spurious extra text from some MIME type printouts (mostly in is_tar). Fix one case where -i produced nothing at all (for a 1-byte file, which is now classed as application/octet-stream). Remove 7/8bit classifications, since they were arbitrary and not based on the file data. This work was done by Reuben Thomas 2007-05-24 10:00 Christos Zoulas * Fix another integer overflow (Colin Percival) 2007-03-26 13:58 Christos Zoulas * make sure that all of struct magic_set is initialized appropriately (Brett) 2007-03-25 17:44 Christos Zoulas * reset left bytes in the buffer (Dmitry V. Levin) * compilation failed with COMPILE_ONLY and ENABLE_CONDITIONALS (Peter Avalos) 2007-03-15 10:51 Christos Zoulas * fix fortran and nroff reversed tests (Dmitry V. Levin) * fix exclude option (Dmitry V. Levin) 2007-02-08 17:30 Christos Zoulas * fix integer underflow in file_printf which can lead to to exploitable heap overflow (Jean-Sebastien Guay-Lero) 2007-02-05 11:35 Christos Zoulas * make socket/pipe reading more robust 2007-01-25 16:01 Christos Zoulas * Centralize all the tests in file_buffer. * Add exclude flag. 2007-01-18 05:29 Anon Ymous * Move the "type" detection code from parse() into its own table driven routine. This avoids maintaining multiple lists in file.h. * Add an optional conditional field (ust before the type field). This code is wrapped in "#ifdef ENABLE_CONDITIONALS" as it is likely to go away. 2007-01-16 23:24 Anon Ymous * Fix an initialization bug in check_mem(). 2007-01-16 14:58 Anon Ymous * Add a "default" type to print a message if nothing previously matched at that level or since the last default at that level. This is useful for setting up switch-like statements. It can also be used to do if/else constructions without a redundant second test. * Fix the "x" special case test so that one can test for that string with "=x". * Allow "search" to search the entire buffer if the "/N" search count is missing. * Make "regex" work! It now starts its search at the specified offset and takes an (optional) "/N" line count to specify the search range; otherwise it searches to the end of the file. The match is now grabbed correctly for format strings and the offset set to the end of the match. * Add a "/s" flag to "regex" and "search" to set the offset to the start of the match. By default the offset is set to the end of the match, as it is with other tests. This is mostly useful for "regex". * Make "search", "string" and "pstring" use the same file_strncmp() routine so that they support the same flags; "bestring16" and "lestring16" call the same routine, but with flags = 0. Also add a "/C" flag (in analogy to "/c") to ignore the case on uppercase (lowercase) characters in the test string. * Strict adherence to C style string escapes. A warnings are printed when compiling. Note: previously "\a" was incorrectly translated to 'a' instead of an (i.e., BELL, typically 0x07). * Make this compile with "-Wall -Wextra" and all the warning flags used with WARNS=4 in the NetBSD source. Also make it pass lint. * Many "cleanups" and hopefully not too many new bugs! 2007-01-16 14:56 Anon Ymous * make several more files compile with gcc warnings on and also make them pass lint. 2007-01-16 14:54 Anon Ymous * fix a puts()/putc() usage goof in file.c * make file.c compile with gcc warnings and pass lint 2006-12-11 16:49 Christos Zoulas * fix byteswapping issue * report the number of bytes we tried to allocate when allocation fails * add a few missed cases in the strength routine 2006-12-08 16:32 Christos Zoulas * store and print the line number of the magic entry for debugging. * if the magic entry did not print anything, don't treat it as a match * change the magic strength algorithm to take into account the relationship op. * fix a bug in search where we could accidentally return a match. * propagate the error return from match to file_softmagic. 2006-11-25 13:35 Christos Zoulas * Don't store the current offset in the magic struct, because it needs to be restored and it was not done properly all the time. Bug found by: Arkadiusz Miskiewicz * Fix problem in the '\0' separator; and don't print it as an additional separator; print it as the only separator. 2006-11-17 10:51 Christos Zoulas * Added a -0 option to print a '\0' separator Etienne Buira 2006-10-31 15:14 Christos Zoulas * Check offset before copying (Mike Frysinger) * merge duplicated code * add quad date support * make sure that we nul terminate desc (Ryoji Kanai) * don't process elf notes multiple times * allow -z to report empty compressed files * use calloc to initialize the ascii buffers (Jos van den Oever) 2006-06-08 11:11 Christos Zoulas * QNX fixes (Mike Gorchak) * Add quad support. * FIFO checks (Dr. Werner Fink) * Linux ELF fixes (Dr. Werner Fink) * Magic format checks (Dr. Werner Fink) * Magic format function improvent (Karl Chen) 2006-05-03 11:11 Christos Zoulas * Pick up some elf changes and some constant fixes from SUSE * Identify gnu tar vs. posix tar * When keep going, don't print spurious newlines (Radek Vokal) 2006-04-01 12:02 Christos Zoulas * Use calloc instead of malloc (Mike Frysinger) * Fix configure script to detect wctypes.h (Mike Frysinger) 2006-03-02 16:06 Christos Zoulas * Print empty if the file is (Mike Frysinger) * Don't try to read past the end of the buffer (Mike Frysinger) * Sort magic entries by strength [experimental] 2005-11-29 13:26 Christos Zoulas * Use iswprint() to convert the output string. (Bastien Nocera) 2005-10-31 8:54 Christos Zoulas * Fix regression where the core info was not completely processed (Radek Vokal) 2005-10-20 11:15 Christos Zoulas * Middle Endian magic (Diomidis Spinellis) 2005-10-17 11:15 Christos Zoulas * Open with O_BINARY for CYGWIN (Corinna Vinschen) * Don't close stdin (Arkadiusz Miskiewicz) * Look for note sections in non executables. 2005-09-20 13:33 Christos Zoulas * Don't print SVR4 Style in core files multiple times (Radek Vokal) 2005-08-27 04:09 Christos Zoulas * Cygwin changes Corinna Vinschen 2005-08-18 09:53 Christos Zoulas * Remove erroreous mention of /etc/magic in the file man page This is gentoo bug 101639. (Mike Frysinger) * Cross-compile support and detection (Mike Frysinger) 2005-08-12 10:17 Christos Zoulas * Add -h flag and dereference symlinks if POSIXLY_CORRECT is set. 2005-07-29 13:57 Christos Zoulas * Avoid search and regex buffer overflows (Kelledin) 2005-07-12 11:48 Christos Zoulas * Provide stub implementations for {v,}nsprintf() for older OS's that don't have them. * Change mbstate_t autoconf detection macro from AC_MBSTATE_T to AC_TYPE_MBSTATE_T. 2005-06-25 11:48 Christos Zoulas * Dynamically allocate the string buffers and make the default read size 256K. 2005-06-01 00:00 Joerg Sonnenberger * Dragonfly ELF note support 2005-03-14 00:00 Giuliano Bertoletti * Avoid NULL pointer dereference in time conversion. 2005-03-06 00:00 Joerg Walter * Add indirect magic offset support, and search mode. 2005-01-12 00:00 Stepan Kasal * src/ascmagic.c (file_ascmagic): Fix three bugs about text files: If a CRLF text file happens to have CR at offset HOWMANY - 1 (currently 0xffff), it should not be counted as CR line terminator. If a line has length exactly MAXLINELEN, it should not yet be treated as a ``very long line'', as MAXLINELEN is ``longest sane line length''. With CRLF, the line length was not computed correctly, and even lines of length MAXLINELEN - 1 were treated as ``very long''. 2004-12-07 14:15 Christos Zoulas * bzip2 needs a lot of input buffer space on some files before it can begin uncompressing. This makes file -z fail on some bz2 files. Fix it by giving it a copy of the file descriptor to read as much as it wants if we have access to it. 2004-11-24 12:39 Christos Zoulas * Stack smash fix, and ELF more conservative reading. Jakub Bogusz 2004-11-20 18:50 Christos Zoulas * New FreeBSD version parsing code: Jon Noack * Hackish support for ucs16 strings 2004-11-13 03:07 Christos Zoulas * print the file name and line number in syntax errors. 2004 10-12 10:50 Christos Zoulas * Fix stack overwriting on 0 length strings: Tim Waugh Ned Ludd 2004-09-27 11:30 Christos Zoulas * Remove 3rd and 4th copyright clause; approved by Ian Darwin. * Fix small memory leaks; caught by: Tamas Sarlos 2004-07-24 16:33 Christos Zoulas * magic.mime update Danny Milosavljevic * FreeBSD version update Oliver Eikemeier * utime/utimes detection Ian Lance Taylor * errors reading elf magic Jakub Bogusz 2004-04-12 10:55 Christos Zoulas * make sure that magic formats match magic types during compilation * fix broken sgi magic file 2004-04-06 20:36 Christos Zoulas * detect present of mbstate_t Petter Reinholdtsen * magic fixes 2004-03-22 15:25 Christos Zoulas * Lots of mime fixes (Joerg Ostertag) * FreeBSD ELF version handling (Edwin Groothuis) * correct cleanup in all cases; don't just close the file. (Christos Zoulas) * add gettext message catalogue support (Michael Piefel) * better printout for unreadable files (Michael Piefel) * compensate for missing MAXPATHLEN (Michael Piefel) * add wide character string length computation (Michael Piefel) * Avoid infinite loops caused by bad elf alignments or name and description note sizes. Reported by (Mikael Magnusson) 2004-03-09 13:55 Christos Zoulas * Fix possible memory leak on error and add missing regfree (Dmitry V. Levin) 2003-12-23 12:12 Christos Zoulas * fix -k flag (Maciej W. Rozycki) 2003-11-18 14:10 Christos Zoulas * Try to give us much info as possible on corrupt elf files. (Willy Tarreau) * Updated python bindings (Brett Funderburg) 2003-11-11 15:03 Christos Zoulas * Include file.h first, because it includes config.h breaks largefile test macros otherwise. (Paul Eggert via Lars Hecking ) 2003-10-14 21:39 Christos Zoulas * Python bindings (Brett Funderburg) * Don't lookup past the end of the buffer (Chad Hanson) * Add MAGIC_ERROR and api on magic_errno() 2003-10-08 12:40 Christos Zoulas * handle error conditions from compile as fatal (Antti Kantee) * handle magic filename parsing sanely * more magic fixes. * fix a memory leak (Illes Marton) * describe magic file handling (Bryan Henderson) 2003-09-12 15:09 Christos Zoulas * update magic files. * remove largefile support from file.h; it breaks things on most OS's 2003-08-10 10:25 Christos Zoulas * fix unmapping'ing of mmaped files. 2003-07-10 12:03 Christos Zoulas * don't exit with -1 on error; always exit 1 (Marty Leisner) * restore utimes code. 2003-06-10 17:03 Christos Zoulas * make sure we don't access uninitialized memory. * pass lint * #ifdef __cplusplus in magic.h 2003-05-25 19:23 Christos Zoulas * rename cvs magic file to revision to deal with case insensitive filesystems. 2003-05-23 17:03 Christos Zoulas * documentation fixes from Michael Piefel * magic fixes (various) * revert basename magic in .mgc name determination * buffer protection in uncompress, signness issues, close files Maciej W. Rozycki * fix zsh magic 2003-04-04 16:59 Christos Zoulas * fix operand sort order in string. 2003-04-02 17:30 Christos Zoulas * cleanup namespace in magic.h 2003-04-02 13:50 Christos Zoulas * Magic additions (Alex Ott) * Fix bug that broke VPATH compilation (Peter Breitenlohner) 2003-03-28 16:03 Christos Zoulas * remove packed attribute from magic struct. * make the magic struct properly aligned. * bump version number of compiled files to 2. 2003-03-27 13:10 Christos Zoulas * separate tar detection and run it before softmagic. * fix reversed symlink test. * fix version printing. * make separator a string instead of a char. * update manual page and sort options. 2003-03-26 11:00 Christos Zoulas * Pass lint * make NULL in magic_file mean stdin * Fix "-" argument to file to pass NULL to magic_file * avoid pointer casts by using memcpy * rename magic_buf -> magic_buffer * keep only the first error * manual page: new sentence, new line * fix typo in api function (magic_buf -> magic_buffer) Index: releng/10.1/contrib/file/README =================================================================== --- releng/10.1/contrib/file/README (revision 284192) +++ releng/10.1/contrib/file/README (revision 284193) @@ -1,126 +1,134 @@ ## README for file(1) Command ## - @(#) $File: README,v 1.48 2014/03/07 13:55:30 christos Exp $ + @(#) $File: README,v 1.49 2015/01/02 20:23:04 christos Exp $ Mailing List: file@mx.gw.com Mailing List archives: http://mx.gw.com/pipermail/file/ Bug tracker: http://bugs.gw.com/ E-mail: christos@astron.com [![Build Status](https://travis-ci.org/file/file.png?branch=master)](https://travis-ci.org/file/file) Phone: Do not even think of telephoning me about this program. Send cash first! This is Release 5.x of Ian Darwin's (copyright but distributable) file(1) command, an implementation of the Unix File(1) command. It knows the 'magic number' of several thousands of file types. This version is the standard "file" command for Linux, *BSD, and other systems. (See "patchlevel.h" for the exact release number). You can download the latest version of the original sources for file from: ftp://ftp.astron.com/pub/file/ A public read-only git repository of the same sources is available at: https://github.com/file/file -The major changes for 5.x are CDF file parsing, indirect magic, and -overhaul in mime and ascii encoding handling. +The major changes for 5.x are CDF file parsing, indirect magic, name/use +(recursion) and overhaul in mime and ascii encoding handling. The major feature of 4.x is the refactoring of the code into a library, and the re-write of the file command in terms of that library. The library itself, libmagic can be used by 3rd party programs that wish to identify file types without having to fork() and exec() file. The prime contributor for 4.0 was Mans Rullgard. UNIX is a trademark of UNIX System Laboratories. The prime contributor to Release 3.8 was Guy Harris, who put in megachanges including byte-order independence. The prime contributor to Release 3.0 was Christos Zoulas, who put in hundreds of lines of source code changes, including his own ANSIfication of the code (I liked my own ANSIfication better, but his (__P()) is the "Berkeley standard" way of doing it, and I wanted UCB to include the code...), his HP-like "indirection" (a feature of the HP file command, I think), and his mods that finally got the uncompress (-z) mode finished and working. This release has compiled in numerous environments; see PORTING for a list and problems. This fine freeware file(1) follows the USG (System V) model of the file command, rather than the Research (V7) version or the V7-derived 4.[23] Berkeley one. That is, the file /etc/magic contains much of the ritual information that is the source of this program's power. My version knows a little more magic (including tar archives) than System V; the /etc/magic parsing seems to be compatible with the (poorly documented) System V /etc/magic format (with one exception; see the man page). In addition, the /etc/magic file is built from a subdirectory for easier(?) maintenance. I will act as a clearinghouse for magic numbers assigned to all sorts of data files that are in reasonable circulation. Send your magic numbers, in magic(5) format please, to the maintainer, Christos Zoulas. COPYING - read this first. README - read this second (you are currently reading this file). INSTALL - read on how to install - src/apprentice.c - parses /etc/magic to learn magic +src/asctime_r.c - replacement for OS's that don't have it. src/apptype.c - used for OS/2 specific application type magic src/asprintf.c - replacement for OS's that don't have it. src/ascmagic.c - third & last set of tests, based on hardwired assumptions. -src/asctime_r.c - for systems that don't have it. -src/asprintf.c - for systems that don't have it. -src/cdf.c - parser for Microsoft Compound Document Files +src/asctime_r.c - replacement for OS's that don't have it. +src/asprintf.c - replacement for OS's that don't have it. +src/cdf.[ch] - parser for Microsoft Compound Document Files src/cdf_time.c - time converter for CDF. src/compress.c - handles decompressing files to look inside. -src/ctime_r.c - for systems that don't have it. +src/ctime_r.c - replacement for OS's that don't have it. +src/elfclass.h - common code for elf 32/64. src/encoding.c - handles unicode encodings src/file.c - the main program src/file.h - header file +src/file_opts.h - list of options +src/fmtcheck.c - replacement for OS's that don't have it. src/fsmagic.c - first set of tests the program runs, based on filesystem info src/funcs.c - utilility functions -src/getopt_long.c - for systems that don't have it. -src/getline.c - for systems that don't have it. +src/getline.c - replacement for OS's that don't have it. +src/getopt_long.c - replacement for OS's that don't have it. src/is_tar.c, tar.h - knows about tarchives (courtesy John Gilmore). src/names.h - header file for ascmagic.c +src/magic.h.in - source file for magic.h src/magic.c - the libmagic api +src/pread.c - replacement for OS's that don't have it. src/print.c - print results, errors, warnings. src/readcdf.c - CDF wrapper. src/readelf.[ch] - Stand-alone elf parsing code. src/softmagic.c - 2nd set of tests, based on /etc/magic -src/strlcat.c - for systems that don't have it. -src/strlcpy.c - for systems that don't have it. +src/mygetopt.h - replacement for OS's that don't have it. +src/strcasestr.c - replacement for OS's that don't have it. +src/strlcat.c - replacement for OS's that don't have it. +src/strlcpy.c - replacement for OS's that don't have it. +src/tar.h - tar file definitions src/vasprintf.c - for systems that don't have it. doc/file.man - man page for the command doc/magic.man - man page for the magic file, courtesy Guy Harris. Install as magic.4 on USG and magic.5 on V7 or Berkeley; cf Makefile. Magdir - directory of /etc/magic pieces ------------------------------------------------------------------------------ If you submit a new magic entry please make sure you read the following guidelines: - Initial match is preferably at least 32 bits long, and is a _unique_ match - If this is not feasible, use additional check - Match of <= 16 bits are not accepted - Delay printing string as much as possible, don't print output too early - Avoid printf arbitrary byte as string, which can be a source of crash and buffer overflow - Provide complete information with entry: * One line short summary * Optional long description * File extension, if applicable * Full name and contact method (for discussion when entry has problem) * Further reference, such as documentation of format ------------------------------------------------------------------------------ Parts of this software were developed at SoftQuad Inc., developers of SGML/HTML/XML publishing software, in Toronto, Canada. SoftQuad was swallowed up by Corel in 2002 and does not exist any longer. Index: releng/10.1/contrib/file/TODO =================================================================== --- releng/10.1/contrib/file/TODO (revision 284192) +++ releng/10.1/contrib/file/TODO (revision 284193) @@ -1,17 +1,19 @@ Most TODOs live in the TODO section of doc/file.man (i.e. file(1)). They are more visible there, so please add any further TODOs to that file, not here. More speculative material can live here. (This change was made when Reuben Thomas noticed that all the bugs listed in the BUGS section of the man page had been fixed!) --- It would be nice to simplify file considerably. For example, reimplement the apprentice and non-pattern magic methods in Python, and compile the magic patterns to a giant regex (or something similar; maybe using Ragel (http://www.complang.org/ragel/)) so that only a small amount of C is needed (because fast execution is typically only required for soft magic, not the more detailed information given by hard-wired routines). In this regard, note that hplip, which is BSD-licensed, has a magic reimplementation in Python. + +Read the kerberos magic entry for more ideas. Index: releng/10.1/contrib/file/config.h.in =================================================================== --- releng/10.1/contrib/file/config.h.in (revision 284192) +++ releng/10.1/contrib/file/config.h.in (revision 284193) @@ -1,388 +1,403 @@ /* config.h.in. Generated from configure.ac by autoheader. */ /* Define if building universal (internal helper macro) */ #undef AC_APPLE_UNIVERSAL_BUILD /* Define in built-in ELF support is used */ #undef BUILTIN_ELF /* Define for ELF core file support */ #undef ELFCORE /* Define to 1 if you have the `asctime_r' function. */ #undef HAVE_ASCTIME_R /* Define to 1 if you have the `asprintf' function. */ #undef HAVE_ASPRINTF /* Define to 1 if you have the `ctime_r' function. */ #undef HAVE_CTIME_R /* HAVE_DAYLIGHT */ #undef HAVE_DAYLIGHT /* Define to 1 if you have the declaration of `daylight', and to 0 if you don't. */ #undef HAVE_DECL_DAYLIGHT /* Define to 1 if you have the declaration of `tzname', and to 0 if you don't. */ #undef HAVE_DECL_TZNAME /* Define to 1 if you have the header file. */ #undef HAVE_DLFCN_H /* Define to 1 if you have the header file. */ #undef HAVE_ERR_H /* Define to 1 if you have the header file. */ #undef HAVE_FCNTL_H /* Define to 1 if you have the `fmtcheck' function. */ #undef HAVE_FMTCHECK /* Define to 1 if you have the `fork' function. */ #undef HAVE_FORK +/* Define to 1 if you have the `freelocale' function. */ +#undef HAVE_FREELOCALE + /* Define to 1 if fseeko (and presumably ftello) exists and is declared. */ #undef HAVE_FSEEKO /* Define to 1 if you have the `getline' function. */ #undef HAVE_GETLINE /* Define to 1 if you have the header file. */ #undef HAVE_GETOPT_H /* Define to 1 if you have the `getopt_long' function. */ #undef HAVE_GETOPT_LONG /* Define to 1 if you have the `getpagesize' function. */ #undef HAVE_GETPAGESIZE /* Define to 1 if the system has the type `intptr_t'. */ #undef HAVE_INTPTR_T /* Define to 1 if you have the header file. */ #undef HAVE_INTTYPES_H /* Define to 1 if you have the `gnurx' library (-lgnurx). */ #undef HAVE_LIBGNURX /* Define to 1 if you have the `z' library (-lz). */ #undef HAVE_LIBZ /* Define to 1 if you have the header file. */ #undef HAVE_LIMITS_H /* Define to 1 if you have the header file. */ #undef HAVE_LOCALE_H /* Define to 1 if mbrtowc and mbstate_t are properly declared. */ #undef HAVE_MBRTOWC /* Define to 1 if declares mbstate_t. */ #undef HAVE_MBSTATE_T /* Define to 1 if you have the header file. */ #undef HAVE_MEMORY_H /* Define to 1 if you have the `mkostemp' function. */ #undef HAVE_MKOSTEMP /* Define to 1 if you have the `mkstemp' function. */ #undef HAVE_MKSTEMP /* Define to 1 if you have a working `mmap' system call. */ #undef HAVE_MMAP +/* Define to 1 if you have the `newlocale' function. */ +#undef HAVE_NEWLOCALE + /* Define to 1 if you have the `pread' function. */ #undef HAVE_PREAD +/* Define to 1 if you have the `setlocale' function. */ +#undef HAVE_SETLOCALE + /* Define to 1 if you have the header file. */ #undef HAVE_STDDEF_H /* Define to 1 if you have the header file. */ #undef HAVE_STDINT_H /* Define to 1 if you have the header file. */ #undef HAVE_STDLIB_H /* Define to 1 if you have the `strcasestr' function. */ #undef HAVE_STRCASESTR /* Define to 1 if you have the `strerror' function. */ #undef HAVE_STRERROR /* Define to 1 if you have the header file. */ #undef HAVE_STRINGS_H /* Define to 1 if you have the header file. */ #undef HAVE_STRING_H /* Define to 1 if you have the `strlcat' function. */ #undef HAVE_STRLCAT /* Define to 1 if you have the `strlcpy' function. */ #undef HAVE_STRLCPY /* Define to 1 if you have the `strndup' function. */ #undef HAVE_STRNDUP /* Define to 1 if you have the `strtof' function. */ #undef HAVE_STRTOF /* Define to 1 if you have the `strtoul' function. */ #undef HAVE_STRTOUL /* HAVE_STRUCT_OPTION */ #undef HAVE_STRUCT_OPTION /* Define to 1 if `st_rdev' is a member of `struct stat'. */ #undef HAVE_STRUCT_STAT_ST_RDEV /* Define to 1 if `tm_gmtoff' is a member of `struct tm'. */ #undef HAVE_STRUCT_TM_TM_GMTOFF /* Define to 1 if `tm_zone' is a member of `struct tm'. */ #undef HAVE_STRUCT_TM_TM_ZONE /* Define to 1 if you have the header file. */ #undef HAVE_SYS_MMAN_H /* Define to 1 if you have the header file. */ #undef HAVE_SYS_PARAM_H /* Define to 1 if you have the header file. */ #undef HAVE_SYS_STAT_H /* Define to 1 if you have the header file. */ #undef HAVE_SYS_TIME_H /* Define to 1 if you have the header file. */ #undef HAVE_SYS_TYPES_H /* Define to 1 if you have the header file. */ #undef HAVE_SYS_UTIME_H /* Define to 1 if you have that is POSIX.1 compatible. */ #undef HAVE_SYS_WAIT_H /* HAVE_TM_ISDST */ #undef HAVE_TM_ISDST /* HAVE_TM_ZONE */ #undef HAVE_TM_ZONE /* HAVE_TZNAME */ #undef HAVE_TZNAME /* Define to 1 if the system has the type `uintptr_t'. */ #undef HAVE_UINTPTR_T /* Define to 1 if you have the header file. */ #undef HAVE_UNISTD_H +/* Define to 1 if you have the `uselocale' function. */ +#undef HAVE_USELOCALE + /* Define to 1 if you have the `utime' function. */ #undef HAVE_UTIME /* Define to 1 if you have the `utimes' function. */ #undef HAVE_UTIMES /* Define to 1 if you have the header file. */ #undef HAVE_UTIME_H /* Define to 1 if you have the `vasprintf' function. */ #undef HAVE_VASPRINTF /* Define to 1 if you have the `vfork' function. */ #undef HAVE_VFORK /* Define to 1 if you have the header file. */ #undef HAVE_VFORK_H /* Define to 1 or 0, depending whether the compiler supports simple visibility declarations. */ #undef HAVE_VISIBILITY /* Define to 1 if you have the header file. */ #undef HAVE_WCHAR_H /* Define to 1 if you have the header file. */ #undef HAVE_WCTYPE_H /* Define to 1 if you have the `wcwidth' function. */ #undef HAVE_WCWIDTH /* Define to 1 if `fork' works. */ #undef HAVE_WORKING_FORK /* Define to 1 if `vfork' works. */ #undef HAVE_WORKING_VFORK + +/* Define to 1 if you have the header file. */ +#undef HAVE_XLOCALE_H /* Define to 1 if you have the header file. */ #undef HAVE_ZLIB_H /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR /* Define to 1 if `major', `minor', and `makedev' are declared in . */ #undef MAJOR_IN_MKDEV /* Define to 1 if `major', `minor', and `makedev' are declared in . */ #undef MAJOR_IN_SYSMACROS /* Name of package */ #undef PACKAGE /* Define to the address where bug reports for this package should be sent. */ #undef PACKAGE_BUGREPORT /* Define to the full name of this package. */ #undef PACKAGE_NAME /* Define to the full name and version of this package. */ #undef PACKAGE_STRING /* Define to the one symbol short name of this package. */ #undef PACKAGE_TARNAME /* Define to the home page for this package. */ #undef PACKAGE_URL /* Define to the version of this package. */ #undef PACKAGE_VERSION /* Define to 1 if you have the ANSI C header files. */ #undef STDC_HEADERS /* Define to 1 if your declares `struct tm'. */ #undef TM_IN_SYS_TIME /* Enable extensions on AIX 3, Interix. */ #ifndef _ALL_SOURCE # undef _ALL_SOURCE #endif /* Enable GNU extensions on systems that have them. */ #ifndef _GNU_SOURCE # undef _GNU_SOURCE #endif /* Enable threading extensions on Solaris. */ #ifndef _POSIX_PTHREAD_SEMANTICS # undef _POSIX_PTHREAD_SEMANTICS #endif /* Enable extensions on HP NonStop. */ #ifndef _TANDEM_SOURCE # undef _TANDEM_SOURCE #endif /* Enable general extensions on Solaris. */ #ifndef __EXTENSIONS__ # undef __EXTENSIONS__ #endif /* Version number of package */ #undef VERSION /* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most significant byte first (like Motorola and SPARC, unlike Intel). */ #if defined AC_APPLE_UNIVERSAL_BUILD # if defined __BIG_ENDIAN__ # define WORDS_BIGENDIAN 1 # endif #else # ifndef WORDS_BIGENDIAN # undef WORDS_BIGENDIAN # endif #endif /* Enable large inode numbers on Mac OS X 10.5. */ #ifndef _DARWIN_USE_64_BIT_INODE # define _DARWIN_USE_64_BIT_INODE 1 #endif /* Number of bits in a file offset, on hosts where this is settable. */ #undef _FILE_OFFSET_BITS /* Define to 1 to make fseeko visible on some hosts (e.g. glibc 2.2). */ #undef _LARGEFILE_SOURCE /* Define for large files, on AIX-style hosts. */ #undef _LARGE_FILES /* Define to 1 if on MINIX. */ #undef _MINIX /* Define to 2 if the system does not provide POSIX.1 features except with this defined. */ #undef _POSIX_1_SOURCE /* Define to 1 if you need to in order for `stat' and other things to work. */ #undef _POSIX_SOURCE /* Define for Solaris 2.5.1 so the uint32_t typedef from , , or is not used. If the typedef were allowed, the #define below would cause a syntax error. */ #undef _UINT32_T /* Define for Solaris 2.5.1 so the uint64_t typedef from , , or is not used. If the typedef were allowed, the #define below would cause a syntax error. */ #undef _UINT64_T /* Define for Solaris 2.5.1 so the uint8_t typedef from , , or is not used. If the typedef were allowed, the #define below would cause a syntax error. */ #undef _UINT8_T /* Define to empty if `const' does not conform to ANSI C. */ #undef const /* Define to the type of a signed integer type of width exactly 32 bits if such a type exists and the standard includes do not define it. */ #undef int32_t /* Define to the type of a signed integer type of width exactly 64 bits if such a type exists and the standard includes do not define it. */ #undef int64_t /* Define to the type of a signed integer type wide enough to hold a pointer, if such a type exists, and if the system does not define it. */ #undef intptr_t /* Define to a type if does not define. */ #undef mbstate_t /* Define to `long int' if does not define. */ #undef off_t /* Define to `int' if does not define. */ #undef pid_t /* Define to `unsigned int' if does not define. */ #undef size_t /* Define to the type of an unsigned integer type of width exactly 16 bits if such a type exists and the standard includes do not define it. */ #undef uint16_t /* Define to the type of an unsigned integer type of width exactly 32 bits if such a type exists and the standard includes do not define it. */ #undef uint32_t /* Define to the type of an unsigned integer type of width exactly 64 bits if such a type exists and the standard includes do not define it. */ #undef uint64_t /* Define to the type of an unsigned integer type of width exactly 8 bits if such a type exists and the standard includes do not define it. */ #undef uint8_t /* Define to the type of an unsigned integer type wide enough to hold a pointer, if such a type exists, and if the system does not define it. */ #undef uintptr_t /* Define as `fork' if `vfork' does not work. */ #undef vfork Index: releng/10.1/contrib/file/configure =================================================================== --- releng/10.1/contrib/file/configure (revision 284192) +++ releng/10.1/contrib/file/configure (revision 284193) @@ -1,16837 +1,16837 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for file 5.19. +# Generated by GNU Autoconf 2.69 for file 5.22. # # Report bugs to . # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. # # # This configure script is free software; the Free Software Foundation # gives unlimited permission to copy, distribute and modify it. ## -------------------- ## ## M4sh Initialization. ## ## -------------------- ## # Be more Bourne compatible DUALCASE=1; export DUALCASE # for MKS sh if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which # is contrary to our usage. Disable this feature. alias -g '${1+"$@"}'='"$@"' setopt NO_GLOB_SUBST else case `(set -o) 2>/dev/null` in #( *posix*) : set -o posix ;; #( *) : ;; esac fi as_nl=' ' export as_nl # Printing a long string crashes Solaris 7 /usr/bin/printf. as_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo # Prefer a ksh shell builtin over an external printf program on Solaris, # but without wasting forks for bash or zsh. if test -z "$BASH_VERSION$ZSH_VERSION" \ && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='print -r --' as_echo_n='print -rn --' elif (test "X`printf %s $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='printf %s\n' as_echo_n='printf %s' else if test "X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`" = "X-n $as_echo"; then as_echo_body='eval /usr/ucb/echo -n "$1$as_nl"' as_echo_n='/usr/ucb/echo -n' else as_echo_body='eval expr "X$1" : "X\\(.*\\)"' as_echo_n_body='eval arg=$1; case $arg in #( *"$as_nl"*) expr "X$arg" : "X\\(.*\\)$as_nl"; arg=`expr "X$arg" : ".*$as_nl\\(.*\\)"`;; esac; expr "X$arg" : "X\\(.*\\)" | tr -d "$as_nl" ' export as_echo_n_body as_echo_n='sh -c $as_echo_n_body as_echo' fi export as_echo_body as_echo='sh -c $as_echo_body as_echo' fi # The user is always right. if test "${PATH_SEPARATOR+set}" != set; then PATH_SEPARATOR=: (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && { (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 || PATH_SEPARATOR=';' } fi # IFS # We need space, tab and new line, in precisely that order. Quoting is # there to prevent editors from complaining about space-tab. # (If _AS_PATH_WALK were called with IFS unset, it would disable word # splitting by setting IFS to empty value.) IFS=" "" $as_nl" # Find who we are. Look in the path if we contain no directory separator. as_myself= case $0 in #(( *[\\/]* ) as_myself=$0 ;; *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. test -r "$as_dir/$0" && as_myself=$as_dir/$0 && break done IFS=$as_save_IFS ;; esac # We did not find ourselves, most probably we were run as `sh COMMAND' # in which case we are not to be found in the path. if test "x$as_myself" = x; then as_myself=$0 fi if test ! -f "$as_myself"; then $as_echo "$as_myself: error: cannot find myself; rerun with an absolute file name" >&2 exit 1 fi # Unset variables that we do not need and which cause bugs (e.g. in # pre-3.0 UWIN ksh). But do not cause bugs in bash 2.01; the "|| exit 1" # suppresses any "Segmentation fault" message there. '((' could # trigger a bug in pdksh 5.2.14. for as_var in BASH_ENV ENV MAIL MAILPATH do eval test x\${$as_var+set} = xset \ && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || : done PS1='$ ' PS2='> ' PS4='+ ' # NLS nuisances. LC_ALL=C export LC_ALL LANGUAGE=C export LANGUAGE # CDPATH. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH # Use a proper internal environment variable to ensure we don't fall # into an infinite loop, continuously re-executing ourselves. if test x"${_as_can_reexec}" != xno && test "x$CONFIG_SHELL" != x; then _as_can_reexec=no; export _as_can_reexec; # We cannot yet assume a decent shell, so we have to provide a # neutralization value for shells without unset; and this also # works around shells that cannot unset nonexistent variables. # Preserve -v and -x to the replacement shell. BASH_ENV=/dev/null ENV=/dev/null (unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV case $- in # (((( *v*x* | *x*v* ) as_opts=-vx ;; *v* ) as_opts=-v ;; *x* ) as_opts=-x ;; * ) as_opts= ;; esac exec $CONFIG_SHELL $as_opts "$as_myself" ${1+"$@"} # Admittedly, this is quite paranoid, since all the known shells bail # out after a failed `exec'. $as_echo "$0: could not re-execute with $CONFIG_SHELL" >&2 as_fn_exit 255 fi # We don't want this to propagate to other subprocesses. { _as_can_reexec=; unset _as_can_reexec;} if test "x$CONFIG_SHELL" = x; then as_bourne_compatible="if test -n \"\${ZSH_VERSION+set}\" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: # Pre-4.2 versions of Zsh do word splitting on \${1+\"\$@\"}, which # is contrary to our usage. Disable this feature. alias -g '\${1+\"\$@\"}'='\"\$@\"' setopt NO_GLOB_SUBST else case \`(set -o) 2>/dev/null\` in #( *posix*) : set -o posix ;; #( *) : ;; esac fi " as_required="as_fn_return () { (exit \$1); } as_fn_success () { as_fn_return 0; } as_fn_failure () { as_fn_return 1; } as_fn_ret_success () { return 0; } as_fn_ret_failure () { return 1; } exitcode=0 as_fn_success || { exitcode=1; echo as_fn_success failed.; } as_fn_failure && { exitcode=1; echo as_fn_failure succeeded.; } as_fn_ret_success || { exitcode=1; echo as_fn_ret_success failed.; } as_fn_ret_failure && { exitcode=1; echo as_fn_ret_failure succeeded.; } if ( set x; as_fn_ret_success y && test x = \"\$1\" ); then : else exitcode=1; echo positional parameters were not saved. fi test x\$exitcode = x0 || exit 1 test -x / || exit 1" as_suggested=" as_lineno_1=";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested" as_lineno_1a=\$LINENO as_lineno_2=";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested" as_lineno_2a=\$LINENO eval 'test \"x\$as_lineno_1'\$as_run'\" != \"x\$as_lineno_2'\$as_run'\" && test \"x\`expr \$as_lineno_1'\$as_run' + 1\`\" = \"x\$as_lineno_2'\$as_run'\"' || exit 1 test \$(( 1 + 1 )) = 2 || exit 1 test -n \"\${ZSH_VERSION+set}\${BASH_VERSION+set}\" || ( ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' ECHO=\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO ECHO=\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO PATH=/empty FPATH=/empty; export PATH FPATH test \"X\`printf %s \$ECHO\`\" = \"X\$ECHO\" \\ || test \"X\`print -r -- \$ECHO\`\" = \"X\$ECHO\" ) || exit 1" if (eval "$as_required") 2>/dev/null; then : as_have_required=yes else as_have_required=no fi if test x$as_have_required = xyes && (eval "$as_suggested") 2>/dev/null; then : else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR as_found=false for as_dir in /bin$PATH_SEPARATOR/usr/bin$PATH_SEPARATOR$PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. as_found=: case $as_dir in #( /*) for as_base in sh bash ksh sh5; do # Try only shells that exist, to save several forks. as_shell=$as_dir/$as_base if { test -f "$as_shell" || test -f "$as_shell.exe"; } && { $as_echo "$as_bourne_compatible""$as_required" | as_run=a "$as_shell"; } 2>/dev/null; then : CONFIG_SHELL=$as_shell as_have_required=yes if { $as_echo "$as_bourne_compatible""$as_suggested" | as_run=a "$as_shell"; } 2>/dev/null; then : break 2 fi fi done;; esac as_found=false done $as_found || { if { test -f "$SHELL" || test -f "$SHELL.exe"; } && { $as_echo "$as_bourne_compatible""$as_required" | as_run=a "$SHELL"; } 2>/dev/null; then : CONFIG_SHELL=$SHELL as_have_required=yes fi; } IFS=$as_save_IFS if test "x$CONFIG_SHELL" != x; then : export CONFIG_SHELL # We cannot yet assume a decent shell, so we have to provide a # neutralization value for shells without unset; and this also # works around shells that cannot unset nonexistent variables. # Preserve -v and -x to the replacement shell. BASH_ENV=/dev/null ENV=/dev/null (unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV case $- in # (((( *v*x* | *x*v* ) as_opts=-vx ;; *v* ) as_opts=-v ;; *x* ) as_opts=-x ;; * ) as_opts= ;; esac exec $CONFIG_SHELL $as_opts "$as_myself" ${1+"$@"} # Admittedly, this is quite paranoid, since all the known shells bail # out after a failed `exec'. $as_echo "$0: could not re-execute with $CONFIG_SHELL" >&2 exit 255 fi if test x$as_have_required = xno; then : $as_echo "$0: This script requires a shell more modern than all" $as_echo "$0: the shells that I found on your system." if test x${ZSH_VERSION+set} = xset ; then $as_echo "$0: In particular, zsh $ZSH_VERSION has bugs and should" $as_echo "$0: be upgraded to zsh 4.3.4 or later." else $as_echo "$0: Please tell bug-autoconf@gnu.org and $0: christos@astron.com about your system, including any $0: error possibly output before this message. Then install $0: a modern shell, or manually run the script under such a $0: shell if you do have one." fi exit 1 fi fi fi SHELL=${CONFIG_SHELL-/bin/sh} export SHELL # Unset more variables known to interfere with behavior of common tools. CLICOLOR_FORCE= GREP_OPTIONS= unset CLICOLOR_FORCE GREP_OPTIONS ## --------------------- ## ## M4sh Shell Functions. ## ## --------------------- ## # as_fn_unset VAR # --------------- # Portably unset VAR. as_fn_unset () { { eval $1=; unset $1;} } as_unset=as_fn_unset # as_fn_set_status STATUS # ----------------------- # Set $? to STATUS, without forking. as_fn_set_status () { return $1 } # as_fn_set_status # as_fn_exit STATUS # ----------------- # Exit the shell with STATUS, even in a "trap 0" or "set -e" context. as_fn_exit () { set +e as_fn_set_status $1 exit $1 } # as_fn_exit # as_fn_mkdir_p # ------------- # Create "$as_dir" as a directory, including parents if necessary. as_fn_mkdir_p () { case $as_dir in #( -*) as_dir=./$as_dir;; esac test -d "$as_dir" || eval $as_mkdir_p || { as_dirs= while :; do case $as_dir in #( *\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'( *) as_qdir=$as_dir;; esac as_dirs="'$as_qdir' $as_dirs" as_dir=`$as_dirname -- "$as_dir" || $as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$as_dir" : 'X\(//\)[^/]' \| \ X"$as_dir" : 'X\(//\)$' \| \ X"$as_dir" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$as_dir" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` test -d "$as_dir" && break done test -z "$as_dirs" || eval "mkdir $as_dirs" } || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir" } # as_fn_mkdir_p # as_fn_executable_p FILE # ----------------------- # Test if FILE is an executable regular file. as_fn_executable_p () { test -f "$1" && test -x "$1" } # as_fn_executable_p # as_fn_append VAR VALUE # ---------------------- # Append the text in VALUE to the end of the definition contained in VAR. Take # advantage of any shell optimizations that allow amortized linear growth over # repeated appends, instead of the typical quadratic growth present in naive # implementations. if (eval "as_var=1; as_var+=2; test x\$as_var = x12") 2>/dev/null; then : eval 'as_fn_append () { eval $1+=\$2 }' else as_fn_append () { eval $1=\$$1\$2 } fi # as_fn_append # as_fn_arith ARG... # ------------------ # Perform arithmetic evaluation on the ARGs, and store the result in the # global $as_val. Take advantage of shells that can avoid forks. The arguments # must be portable across $(()) and expr. if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then : eval 'as_fn_arith () { as_val=$(( $* )) }' else as_fn_arith () { as_val=`expr "$@" || test $? -eq 1` } fi # as_fn_arith # as_fn_error STATUS ERROR [LINENO LOG_FD] # ---------------------------------------- # Output "`basename $0`: error: ERROR" to stderr. If LINENO and LOG_FD are # provided, also output the error to LOG_FD, referencing LINENO. Then exit the # script with STATUS, using 1 if that was 0. as_fn_error () { as_status=$1; test $as_status -eq 0 && as_status=1 if test "$4"; then as_lineno=${as_lineno-"$3"} as_lineno_stack=as_lineno_stack=$as_lineno_stack $as_echo "$as_me:${as_lineno-$LINENO}: error: $2" >&$4 fi $as_echo "$as_me: error: $2" >&2 as_fn_exit $as_status } # as_fn_error if expr a : '\(a\)' >/dev/null 2>&1 && test "X`expr 00001 : '.*\(...\)'`" = X001; then as_expr=expr else as_expr=false fi if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then as_basename=basename else as_basename=false fi if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then as_dirname=dirname else as_dirname=false fi as_me=`$as_basename -- "$0" || $as_expr X/"$0" : '.*/\([^/][^/]*\)/*$' \| \ X"$0" : 'X\(//\)$' \| \ X"$0" : 'X\(/\)' \| . 2>/dev/null || $as_echo X/"$0" | sed '/^.*\/\([^/][^/]*\)\/*$/{ s//\1/ q } /^X\/\(\/\/\)$/{ s//\1/ q } /^X\/\(\/\).*/{ s//\1/ q } s/.*/./; q'` # Avoid depending upon Character Ranges. as_cr_letters='abcdefghijklmnopqrstuvwxyz' as_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ' as_cr_Letters=$as_cr_letters$as_cr_LETTERS as_cr_digits='0123456789' as_cr_alnum=$as_cr_Letters$as_cr_digits as_lineno_1=$LINENO as_lineno_1a=$LINENO as_lineno_2=$LINENO as_lineno_2a=$LINENO eval 'test "x$as_lineno_1'$as_run'" != "x$as_lineno_2'$as_run'" && test "x`expr $as_lineno_1'$as_run' + 1`" = "x$as_lineno_2'$as_run'"' || { # Blame Lee E. McMahon (1931-1989) for sed's syntax. :-) sed -n ' p /[$]LINENO/= ' <$as_myself | sed ' s/[$]LINENO.*/&-/ t lineno b :lineno N :loop s/[$]LINENO\([^'$as_cr_alnum'_].*\n\)\(.*\)/\2\1\2/ t loop s/-\n.*// ' >$as_me.lineno && chmod +x "$as_me.lineno" || { $as_echo "$as_me: error: cannot create $as_me.lineno; rerun with a POSIX shell" >&2; as_fn_exit 1; } # If we had to re-execute with $CONFIG_SHELL, we're ensured to have # already done that, so ensure we don't try to do so again and fall # in an infinite loop. This has already happened in practice. _as_can_reexec=no; export _as_can_reexec # Don't try to exec as it changes $[0], causing all sort of problems # (the dirname of $[0] is not the place where we might find the # original and so on. Autoconf is especially sensitive to this). . "./$as_me.lineno" # Exit status is that of the last command. exit } ECHO_C= ECHO_N= ECHO_T= case `echo -n x` in #((((( -n*) case `echo 'xy\c'` in *c*) ECHO_T=' ';; # ECHO_T is single tab character. xy) ECHO_C='\c';; *) echo `echo ksh88 bug on AIX 6.1` > /dev/null ECHO_T=' ';; esac;; *) ECHO_N='-n';; esac rm -f conf$$ conf$$.exe conf$$.file if test -d conf$$.dir; then rm -f conf$$.dir/conf$$.file else rm -f conf$$.dir mkdir conf$$.dir 2>/dev/null fi if (echo >conf$$.file) 2>/dev/null; then if ln -s conf$$.file conf$$ 2>/dev/null; then as_ln_s='ln -s' # ... but there are two gotchas: # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail. # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable. # In both cases, we have to default to `cp -pR'. ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe || as_ln_s='cp -pR' elif ln conf$$.file conf$$ 2>/dev/null; then as_ln_s=ln else as_ln_s='cp -pR' fi else as_ln_s='cp -pR' fi rm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file rmdir conf$$.dir 2>/dev/null if mkdir -p . 2>/dev/null; then as_mkdir_p='mkdir -p "$as_dir"' else test -d ./-p && rmdir ./-p as_mkdir_p=false fi as_test_x='test -x' as_executable_p=as_fn_executable_p # Sed expression to map a string onto a valid CPP name. as_tr_cpp="eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'" # Sed expression to map a string onto a valid variable name. as_tr_sh="eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'" SHELL=${CONFIG_SHELL-/bin/sh} test -n "$DJDIR" || exec 7<&0 &1 # Name of the host. # hostname on some systems (SVR3.2, old GNU/Linux) returns a bogus exit status, # so uname gets run too. ac_hostname=`(hostname || uname -n) 2>/dev/null | sed 1q` # # Initializations. # ac_default_prefix=/usr/local ac_clean_files= ac_config_libobj_dir=. LIBOBJS= cross_compiling=no subdirs= MFLAGS= MAKEFLAGS= # Identity of this package. PACKAGE_NAME='file' PACKAGE_TARNAME='file' -PACKAGE_VERSION='5.19' -PACKAGE_STRING='file 5.19' +PACKAGE_VERSION='5.22' +PACKAGE_STRING='file 5.22' PACKAGE_BUGREPORT='christos@astron.com' PACKAGE_URL='' # Factoring default headers for most tests. ac_includes_default="\ #include #ifdef HAVE_SYS_TYPES_H # include #endif #ifdef HAVE_SYS_STAT_H # include #endif #ifdef STDC_HEADERS # include # include #else # ifdef HAVE_STDLIB_H # include # endif #endif #ifdef HAVE_STRING_H # if !defined STDC_HEADERS && defined HAVE_MEMORY_H # include # endif # include #endif #ifdef HAVE_STRINGS_H # include #endif #ifdef HAVE_INTTYPES_H # include #endif #ifdef HAVE_STDINT_H # include #endif #ifdef HAVE_UNISTD_H # include #endif" ac_header_list= ac_subst_vars='am__EXEEXT_FALSE am__EXEEXT_TRUE LTLIBOBJS IS_CROSS_COMPILE_FALSE IS_CROSS_COMPILE_TRUE LIBOBJS HAVE_VISIBILITY CFLAG_VISIBILITY OTOOL64 OTOOL LIPO NMEDIT DSYMUTIL MANIFEST_TOOL RANLIB ac_ct_AR AR DLLTOOL OBJDUMP NM ac_ct_DUMPBIN DUMPBIN LD FGREP SED LIBTOOL LN_S EGREP GREP CPP am__fastdepCC_FALSE am__fastdepCC_TRUE CCDEPMODE am__nodep AMDEPBACKSLASH AMDEP_FALSE AMDEP_TRUE am__quote am__include DEPDIR OBJEXT EXEEXT ac_ct_CC CPPFLAGS LDFLAGS CFLAGS CC WARNINGS FSECT5_FALSE FSECT5_TRUE fsect pkgdatadir MINGW_FALSE MINGW_TRUE MINGW host_os host_vendor host_cpu host build_os build_vendor build_cpu build AM_BACKSLASH AM_DEFAULT_VERBOSITY AM_DEFAULT_V AM_V am__untar am__tar AMTAR am__leading_dot SET_MAKE AWK mkdir_p MKDIR_P INSTALL_STRIP_PROGRAM STRIP install_sh MAKEINFO AUTOHEADER AUTOMAKE AUTOCONF ACLOCAL VERSION PACKAGE CYGPATH_W am__isrc INSTALL_DATA INSTALL_SCRIPT INSTALL_PROGRAM target_alias host_alias build_alias LIBS ECHO_T ECHO_N ECHO_C DEFS mandir localedir libdir psdir pdfdir dvidir htmldir infodir docdir oldincludedir includedir localstatedir sharedstatedir sysconfdir datadir datarootdir libexecdir sbindir bindir program_transform_name prefix exec_prefix PACKAGE_URL PACKAGE_BUGREPORT PACKAGE_STRING PACKAGE_VERSION PACKAGE_TARNAME PACKAGE_NAME PATH_SEPARATOR SHELL' ac_subst_files='' ac_user_opts=' enable_option_checking enable_silent_rules enable_elf enable_elf_core enable_fsect_man5 enable_dependency_tracking enable_static with_pic enable_shared enable_fast_install with_gnu_ld with_sysroot enable_libtool_lock enable_largefile enable_warnings ' ac_precious_vars='build_alias host_alias target_alias CC CFLAGS LDFLAGS LIBS CPPFLAGS CPP' # Initialize some variables set by options. ac_init_help= ac_init_version=false ac_unrecognized_opts= ac_unrecognized_sep= # The variables have the same names as the options, with # dashes changed to underlines. cache_file=/dev/null exec_prefix=NONE no_create= no_recursion= prefix=NONE program_prefix=NONE program_suffix=NONE program_transform_name=s,x,x, silent= site= srcdir= verbose= x_includes=NONE x_libraries=NONE # Installation directory options. # These are left unexpanded so users can "make install exec_prefix=/foo" # and all the variables that are supposed to be based on exec_prefix # by default will actually change. # Use braces instead of parens because sh, perl, etc. also accept them. # (The list follows the same order as the GNU Coding Standards.) bindir='${exec_prefix}/bin' sbindir='${exec_prefix}/sbin' libexecdir='${exec_prefix}/libexec' datarootdir='${prefix}/share' datadir='${datarootdir}' sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' infodir='${datarootdir}/info' htmldir='${docdir}' dvidir='${docdir}' pdfdir='${docdir}' psdir='${docdir}' libdir='${exec_prefix}/lib' localedir='${datarootdir}/locale' mandir='${datarootdir}/man' ac_prev= ac_dashdash= for ac_option do # If the previous option needs an argument, assign it. if test -n "$ac_prev"; then eval $ac_prev=\$ac_option ac_prev= continue fi case $ac_option in *=?*) ac_optarg=`expr "X$ac_option" : '[^=]*=\(.*\)'` ;; *=) ac_optarg= ;; *) ac_optarg=yes ;; esac # Accept the important Cygnus configure options, so we can diagnose typos. case $ac_dashdash$ac_option in --) ac_dashdash=yes ;; -bindir | --bindir | --bindi | --bind | --bin | --bi) ac_prev=bindir ;; -bindir=* | --bindir=* | --bindi=* | --bind=* | --bin=* | --bi=*) bindir=$ac_optarg ;; -build | --build | --buil | --bui | --bu) ac_prev=build_alias ;; -build=* | --build=* | --buil=* | --bui=* | --bu=*) build_alias=$ac_optarg ;; -cache-file | --cache-file | --cache-fil | --cache-fi \ | --cache-f | --cache- | --cache | --cach | --cac | --ca | --c) ac_prev=cache_file ;; -cache-file=* | --cache-file=* | --cache-fil=* | --cache-fi=* \ | --cache-f=* | --cache-=* | --cache=* | --cach=* | --cac=* | --ca=* | --c=*) cache_file=$ac_optarg ;; --config-cache | -C) cache_file=config.cache ;; -datadir | --datadir | --datadi | --datad) ac_prev=datadir ;; -datadir=* | --datadir=* | --datadi=* | --datad=*) datadir=$ac_optarg ;; -datarootdir | --datarootdir | --datarootdi | --datarootd | --dataroot \ | --dataroo | --dataro | --datar) ac_prev=datarootdir ;; -datarootdir=* | --datarootdir=* | --datarootdi=* | --datarootd=* \ | --dataroot=* | --dataroo=* | --dataro=* | --datar=*) datarootdir=$ac_optarg ;; -disable-* | --disable-*) ac_useropt=`expr "x$ac_option" : 'x-*disable-\(.*\)'` # Reject names that are not valid shell variable names. expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && as_fn_error $? "invalid feature name: $ac_useropt" ac_useropt_orig=$ac_useropt ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` case $ac_user_opts in *" "enable_$ac_useropt" "*) ;; *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--disable-$ac_useropt_orig" ac_unrecognized_sep=', ';; esac eval enable_$ac_useropt=no ;; -docdir | --docdir | --docdi | --doc | --do) ac_prev=docdir ;; -docdir=* | --docdir=* | --docdi=* | --doc=* | --do=*) docdir=$ac_optarg ;; -dvidir | --dvidir | --dvidi | --dvid | --dvi | --dv) ac_prev=dvidir ;; -dvidir=* | --dvidir=* | --dvidi=* | --dvid=* | --dvi=* | --dv=*) dvidir=$ac_optarg ;; -enable-* | --enable-*) ac_useropt=`expr "x$ac_option" : 'x-*enable-\([^=]*\)'` # Reject names that are not valid shell variable names. expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && as_fn_error $? "invalid feature name: $ac_useropt" ac_useropt_orig=$ac_useropt ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` case $ac_user_opts in *" "enable_$ac_useropt" "*) ;; *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--enable-$ac_useropt_orig" ac_unrecognized_sep=', ';; esac eval enable_$ac_useropt=\$ac_optarg ;; -exec-prefix | --exec_prefix | --exec-prefix | --exec-prefi \ | --exec-pref | --exec-pre | --exec-pr | --exec-p | --exec- \ | --exec | --exe | --ex) ac_prev=exec_prefix ;; -exec-prefix=* | --exec_prefix=* | --exec-prefix=* | --exec-prefi=* \ | --exec-pref=* | --exec-pre=* | --exec-pr=* | --exec-p=* | --exec-=* \ | --exec=* | --exe=* | --ex=*) exec_prefix=$ac_optarg ;; -gas | --gas | --ga | --g) # Obsolete; use --with-gas. with_gas=yes ;; -help | --help | --hel | --he | -h) ac_init_help=long ;; -help=r* | --help=r* | --hel=r* | --he=r* | -hr*) ac_init_help=recursive ;; -help=s* | --help=s* | --hel=s* | --he=s* | -hs*) ac_init_help=short ;; -host | --host | --hos | --ho) ac_prev=host_alias ;; -host=* | --host=* | --hos=* | --ho=*) host_alias=$ac_optarg ;; -htmldir | --htmldir | --htmldi | --htmld | --html | --htm | --ht) ac_prev=htmldir ;; -htmldir=* | --htmldir=* | --htmldi=* | --htmld=* | --html=* | --htm=* \ | --ht=*) htmldir=$ac_optarg ;; -includedir | --includedir | --includedi | --included | --include \ | --includ | --inclu | --incl | --inc) ac_prev=includedir ;; -includedir=* | --includedir=* | --includedi=* | --included=* | --include=* \ | --includ=* | --inclu=* | --incl=* | --inc=*) includedir=$ac_optarg ;; -infodir | --infodir | --infodi | --infod | --info | --inf) ac_prev=infodir ;; -infodir=* | --infodir=* | --infodi=* | --infod=* | --info=* | --inf=*) infodir=$ac_optarg ;; -libdir | --libdir | --libdi | --libd) ac_prev=libdir ;; -libdir=* | --libdir=* | --libdi=* | --libd=*) libdir=$ac_optarg ;; -libexecdir | --libexecdir | --libexecdi | --libexecd | --libexec \ | --libexe | --libex | --libe) ac_prev=libexecdir ;; -libexecdir=* | --libexecdir=* | --libexecdi=* | --libexecd=* | --libexec=* \ | --libexe=* | --libex=* | --libe=*) libexecdir=$ac_optarg ;; -localedir | --localedir | --localedi | --localed | --locale) ac_prev=localedir ;; -localedir=* | --localedir=* | --localedi=* | --localed=* | --locale=*) localedir=$ac_optarg ;; -localstatedir | --localstatedir | --localstatedi | --localstated \ | --localstate | --localstat | --localsta | --localst | --locals) ac_prev=localstatedir ;; -localstatedir=* | --localstatedir=* | --localstatedi=* | --localstated=* \ | --localstate=* | --localstat=* | --localsta=* | --localst=* | --locals=*) localstatedir=$ac_optarg ;; -mandir | --mandir | --mandi | --mand | --man | --ma | --m) ac_prev=mandir ;; -mandir=* | --mandir=* | --mandi=* | --mand=* | --man=* | --ma=* | --m=*) mandir=$ac_optarg ;; -nfp | --nfp | --nf) # Obsolete; use --without-fp. with_fp=no ;; -no-create | --no-create | --no-creat | --no-crea | --no-cre \ | --no-cr | --no-c | -n) no_create=yes ;; -no-recursion | --no-recursion | --no-recursio | --no-recursi \ | --no-recurs | --no-recur | --no-recu | --no-rec | --no-re | --no-r) no_recursion=yes ;; -oldincludedir | --oldincludedir | --oldincludedi | --oldincluded \ | --oldinclude | --oldinclud | --oldinclu | --oldincl | --oldinc \ | --oldin | --oldi | --old | --ol | --o) ac_prev=oldincludedir ;; -oldincludedir=* | --oldincludedir=* | --oldincludedi=* | --oldincluded=* \ | --oldinclude=* | --oldinclud=* | --oldinclu=* | --oldincl=* | --oldinc=* \ | --oldin=* | --oldi=* | --old=* | --ol=* | --o=*) oldincludedir=$ac_optarg ;; -prefix | --prefix | --prefi | --pref | --pre | --pr | --p) ac_prev=prefix ;; -prefix=* | --prefix=* | --prefi=* | --pref=* | --pre=* | --pr=* | --p=*) prefix=$ac_optarg ;; -program-prefix | --program-prefix | --program-prefi | --program-pref \ | --program-pre | --program-pr | --program-p) ac_prev=program_prefix ;; -program-prefix=* | --program-prefix=* | --program-prefi=* \ | --program-pref=* | --program-pre=* | --program-pr=* | --program-p=*) program_prefix=$ac_optarg ;; -program-suffix | --program-suffix | --program-suffi | --program-suff \ | --program-suf | --program-su | --program-s) ac_prev=program_suffix ;; -program-suffix=* | --program-suffix=* | --program-suffi=* \ | --program-suff=* | --program-suf=* | --program-su=* | --program-s=*) program_suffix=$ac_optarg ;; -program-transform-name | --program-transform-name \ | --program-transform-nam | --program-transform-na \ | --program-transform-n | --program-transform- \ | --program-transform | --program-transfor \ | --program-transfo | --program-transf \ | --program-trans | --program-tran \ | --progr-tra | --program-tr | --program-t) ac_prev=program_transform_name ;; -program-transform-name=* | --program-transform-name=* \ | --program-transform-nam=* | --program-transform-na=* \ | --program-transform-n=* | --program-transform-=* \ | --program-transform=* | --program-transfor=* \ | --program-transfo=* | --program-transf=* \ | --program-trans=* | --program-tran=* \ | --progr-tra=* | --program-tr=* | --program-t=*) program_transform_name=$ac_optarg ;; -pdfdir | --pdfdir | --pdfdi | --pdfd | --pdf | --pd) ac_prev=pdfdir ;; -pdfdir=* | --pdfdir=* | --pdfdi=* | --pdfd=* | --pdf=* | --pd=*) pdfdir=$ac_optarg ;; -psdir | --psdir | --psdi | --psd | --ps) ac_prev=psdir ;; -psdir=* | --psdir=* | --psdi=* | --psd=* | --ps=*) psdir=$ac_optarg ;; -q | -quiet | --quiet | --quie | --qui | --qu | --q \ | -silent | --silent | --silen | --sile | --sil) silent=yes ;; -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ | --sbi=* | --sb=*) sbindir=$ac_optarg ;; -sharedstatedir | --sharedstatedir | --sharedstatedi \ | --sharedstated | --sharedstate | --sharedstat | --sharedsta \ | --sharedst | --shareds | --shared | --share | --shar \ | --sha | --sh) ac_prev=sharedstatedir ;; -sharedstatedir=* | --sharedstatedir=* | --sharedstatedi=* \ | --sharedstated=* | --sharedstate=* | --sharedstat=* | --sharedsta=* \ | --sharedst=* | --shareds=* | --shared=* | --share=* | --shar=* \ | --sha=* | --sh=*) sharedstatedir=$ac_optarg ;; -site | --site | --sit) ac_prev=site ;; -site=* | --site=* | --sit=*) site=$ac_optarg ;; -srcdir | --srcdir | --srcdi | --srcd | --src | --sr) ac_prev=srcdir ;; -srcdir=* | --srcdir=* | --srcdi=* | --srcd=* | --src=* | --sr=*) srcdir=$ac_optarg ;; -sysconfdir | --sysconfdir | --sysconfdi | --sysconfd | --sysconf \ | --syscon | --sysco | --sysc | --sys | --sy) ac_prev=sysconfdir ;; -sysconfdir=* | --sysconfdir=* | --sysconfdi=* | --sysconfd=* | --sysconf=* \ | --syscon=* | --sysco=* | --sysc=* | --sys=* | --sy=*) sysconfdir=$ac_optarg ;; -target | --target | --targe | --targ | --tar | --ta | --t) ac_prev=target_alias ;; -target=* | --target=* | --targe=* | --targ=* | --tar=* | --ta=* | --t=*) target_alias=$ac_optarg ;; -v | -verbose | --verbose | --verbos | --verbo | --verb) verbose=yes ;; -version | --version | --versio | --versi | --vers | -V) ac_init_version=: ;; -with-* | --with-*) ac_useropt=`expr "x$ac_option" : 'x-*with-\([^=]*\)'` # Reject names that are not valid shell variable names. expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && as_fn_error $? "invalid package name: $ac_useropt" ac_useropt_orig=$ac_useropt ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` case $ac_user_opts in *" "with_$ac_useropt" "*) ;; *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--with-$ac_useropt_orig" ac_unrecognized_sep=', ';; esac eval with_$ac_useropt=\$ac_optarg ;; -without-* | --without-*) ac_useropt=`expr "x$ac_option" : 'x-*without-\(.*\)'` # Reject names that are not valid shell variable names. expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && as_fn_error $? "invalid package name: $ac_useropt" ac_useropt_orig=$ac_useropt ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` case $ac_user_opts in *" "with_$ac_useropt" "*) ;; *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--without-$ac_useropt_orig" ac_unrecognized_sep=', ';; esac eval with_$ac_useropt=no ;; --x) # Obsolete; use --with-x. with_x=yes ;; -x-includes | --x-includes | --x-include | --x-includ | --x-inclu \ | --x-incl | --x-inc | --x-in | --x-i) ac_prev=x_includes ;; -x-includes=* | --x-includes=* | --x-include=* | --x-includ=* | --x-inclu=* \ | --x-incl=* | --x-inc=* | --x-in=* | --x-i=*) x_includes=$ac_optarg ;; -x-libraries | --x-libraries | --x-librarie | --x-librari \ | --x-librar | --x-libra | --x-libr | --x-lib | --x-li | --x-l) ac_prev=x_libraries ;; -x-libraries=* | --x-libraries=* | --x-librarie=* | --x-librari=* \ | --x-librar=* | --x-libra=* | --x-libr=* | --x-lib=* | --x-li=* | --x-l=*) x_libraries=$ac_optarg ;; -*) as_fn_error $? "unrecognized option: \`$ac_option' Try \`$0 --help' for more information" ;; *=*) ac_envvar=`expr "x$ac_option" : 'x\([^=]*\)='` # Reject names that are not valid shell variable names. case $ac_envvar in #( '' | [0-9]* | *[!_$as_cr_alnum]* ) as_fn_error $? "invalid variable name: \`$ac_envvar'" ;; esac eval $ac_envvar=\$ac_optarg export $ac_envvar ;; *) # FIXME: should be removed in autoconf 3.0. $as_echo "$as_me: WARNING: you should use --build, --host, --target" >&2 expr "x$ac_option" : ".*[^-._$as_cr_alnum]" >/dev/null && $as_echo "$as_me: WARNING: invalid host type: $ac_option" >&2 : "${build_alias=$ac_option} ${host_alias=$ac_option} ${target_alias=$ac_option}" ;; esac done if test -n "$ac_prev"; then ac_option=--`echo $ac_prev | sed 's/_/-/g'` as_fn_error $? "missing argument to $ac_option" fi if test -n "$ac_unrecognized_opts"; then case $enable_option_checking in no) ;; fatal) as_fn_error $? "unrecognized options: $ac_unrecognized_opts" ;; *) $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2 ;; esac fi # Check all directory arguments for consistency. for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes. case $ac_val in */ ) ac_val=`expr "X$ac_val" : 'X\(.*[^/]\)' \| "X$ac_val" : 'X\(.*\)'` eval $ac_var=\$ac_val;; esac # Be sure to have absolute directory names. case $ac_val in [\\/$]* | ?:[\\/]* ) continue;; NONE | '' ) case $ac_var in *prefix ) continue;; esac;; esac as_fn_error $? "expected an absolute directory name for --$ac_var: $ac_val" done # There might be people who depend on the old broken behavior: `$host' # used to hold the argument of --host etc. # FIXME: To remove some day. build=$build_alias host=$host_alias target=$target_alias # FIXME: To remove some day. if test "x$host_alias" != x; then if test "x$build_alias" = x; then cross_compiling=maybe elif test "x$build_alias" != "x$host_alias"; then cross_compiling=yes fi fi ac_tool_prefix= test -n "$host_alias" && ac_tool_prefix=$host_alias- test "$silent" = yes && exec 6>/dev/null ac_pwd=`pwd` && test -n "$ac_pwd" && ac_ls_di=`ls -di .` && ac_pwd_ls_di=`cd "$ac_pwd" && ls -di .` || as_fn_error $? "working directory cannot be determined" test "X$ac_ls_di" = "X$ac_pwd_ls_di" || as_fn_error $? "pwd does not report name of working directory" # Find the source files, if location was not specified. if test -z "$srcdir"; then ac_srcdir_defaulted=yes # Try the directory containing this script, then the parent directory. ac_confdir=`$as_dirname -- "$as_myself" || $as_expr X"$as_myself" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$as_myself" : 'X\(//\)[^/]' \| \ X"$as_myself" : 'X\(//\)$' \| \ X"$as_myself" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$as_myself" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` srcdir=$ac_confdir if test ! -r "$srcdir/$ac_unique_file"; then srcdir=.. fi else ac_srcdir_defaulted=no fi if test ! -r "$srcdir/$ac_unique_file"; then test "$ac_srcdir_defaulted" = yes && srcdir="$ac_confdir or .." as_fn_error $? "cannot find sources ($ac_unique_file) in $srcdir" fi ac_msg="sources are in $srcdir, but \`cd $srcdir' does not work" ac_abs_confdir=`( cd "$srcdir" && test -r "./$ac_unique_file" || as_fn_error $? "$ac_msg" pwd)` # When building in place, set srcdir=. if test "$ac_abs_confdir" = "$ac_pwd"; then srcdir=. fi # Remove unnecessary trailing slashes from srcdir. # Double slashes in file names in object file debugging info # mess up M-x gdb in Emacs. case $srcdir in */) srcdir=`expr "X$srcdir" : 'X\(.*[^/]\)' \| "X$srcdir" : 'X\(.*\)'`;; esac for ac_var in $ac_precious_vars; do eval ac_env_${ac_var}_set=\${${ac_var}+set} eval ac_env_${ac_var}_value=\$${ac_var} eval ac_cv_env_${ac_var}_set=\${${ac_var}+set} eval ac_cv_env_${ac_var}_value=\$${ac_var} done # # Report the --help message. # if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures file 5.19 to adapt to many kinds of systems. +\`configure' configures file 5.22 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... To assign environment variables (e.g., CC, CFLAGS...), specify them as VAR=VALUE. See below for descriptions of some of the useful variables. Defaults for the options are specified in brackets. Configuration: -h, --help display this help and exit --help=short display options specific to this package --help=recursive display the short help of all the included packages -V, --version display version information and exit -q, --quiet, --silent do not print \`checking ...' messages --cache-file=FILE cache test results in FILE [disabled] -C, --config-cache alias for \`--cache-file=config.cache' -n, --no-create do not create output files --srcdir=DIR find the sources in DIR [configure dir or \`..'] Installation directories: --prefix=PREFIX install architecture-independent files in PREFIX [$ac_default_prefix] --exec-prefix=EPREFIX install architecture-dependent files in EPREFIX [PREFIX] By default, \`make install' will install all the files in \`$ac_default_prefix/bin', \`$ac_default_prefix/lib' etc. You can specify an installation prefix other than \`$ac_default_prefix' using \`--prefix', for instance \`--prefix=\$HOME'. For better control, use the options below. Fine tuning of the installation directories: --bindir=DIR user executables [EPREFIX/bin] --sbindir=DIR system admin executables [EPREFIX/sbin] --libexecdir=DIR program executables [EPREFIX/libexec] --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include] --datarootdir=DIR read-only arch.-independent data root [PREFIX/share] --datadir=DIR read-only architecture-independent data [DATAROOTDIR] --infodir=DIR info documentation [DATAROOTDIR/info] --localedir=DIR locale-dependent data [DATAROOTDIR/locale] --mandir=DIR man documentation [DATAROOTDIR/man] --docdir=DIR documentation root [DATAROOTDIR/doc/file] --htmldir=DIR html documentation [DOCDIR] --dvidir=DIR dvi documentation [DOCDIR] --pdfdir=DIR pdf documentation [DOCDIR] --psdir=DIR ps documentation [DOCDIR] _ACEOF cat <<\_ACEOF Program names: --program-prefix=PREFIX prepend PREFIX to installed program names --program-suffix=SUFFIX append SUFFIX to installed program names --program-transform-name=PROGRAM run sed PROGRAM on installed program names System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] _ACEOF fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of file 5.19:";; + short | recursive ) echo "Configuration of file 5.22:";; esac cat <<\_ACEOF Optional Features: --disable-option-checking ignore unrecognized --enable/--with options --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] --enable-silent-rules less verbose build output (undo: "make V=1") --disable-silent-rules verbose build output (undo: "make V=0") --disable-elf disable builtin ELF support --disable-elf-core disable ELF core file support --enable-fsect-man5 enable file formats in man section 5 --enable-dependency-tracking do not reject slow dependency extractors --disable-dependency-tracking speeds up one-time build --enable-static[=PKGS] build static libraries [default=no] --enable-shared[=PKGS] build shared libraries [default=yes] --enable-fast-install[=PKGS] optimize for fast installation [default=yes] --disable-libtool-lock avoid locking (might break parallel builds) --disable-largefile omit support for large files --disable-warnings disable compiler warnings Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-pic[=PKGS] try to use only PIC/non-PIC objects [default=use both] --with-gnu-ld assume the C compiler uses GNU ld [default=no] --with-sysroot=DIR Search for dependent libraries within DIR (or the compiler's sysroot if not specified). Some influential environment variables: CC C compiler command CFLAGS C compiler flags LDFLAGS linker flags, e.g. -L if you have libraries in a nonstandard directory LIBS libraries to pass to the linker, e.g. -l CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I if you have headers in a nonstandard directory CPP C preprocessor Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. Report bugs to . _ACEOF ac_status=$? fi if test "$ac_init_help" = "recursive"; then # If there are subdirs, report their specific --help. for ac_dir in : $ac_subdirs_all; do test "x$ac_dir" = x: && continue test -d "$ac_dir" || { cd "$srcdir" && ac_pwd=`pwd` && srcdir=. && test -d "$ac_dir"; } || continue ac_builddir=. case "$ac_dir" in .) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_dir_suffix=/`$as_echo "$ac_dir" | sed 's|^\.[\\/]||'` # A ".." for each directory in $ac_dir_suffix. ac_top_builddir_sub=`$as_echo "$ac_dir_suffix" | sed 's|/[^\\/]*|/..|g;s|/||'` case $ac_top_builddir_sub in "") ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_top_build_prefix=$ac_top_builddir_sub/ ;; esac ;; esac ac_abs_top_builddir=$ac_pwd ac_abs_builddir=$ac_pwd$ac_dir_suffix # for backward compatibility: ac_top_builddir=$ac_top_build_prefix case $srcdir in .) # We are building in place. ac_srcdir=. ac_top_srcdir=$ac_top_builddir_sub ac_abs_top_srcdir=$ac_pwd ;; [\\/]* | ?:[\\/]* ) # Absolute name. ac_srcdir=$srcdir$ac_dir_suffix; ac_top_srcdir=$srcdir ac_abs_top_srcdir=$srcdir ;; *) # Relative name. ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix ac_top_srcdir=$ac_top_build_prefix$srcdir ac_abs_top_srcdir=$ac_pwd/$srcdir ;; esac ac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix cd "$ac_dir" || { ac_status=$?; continue; } # Check for guested configure. if test -f "$ac_srcdir/configure.gnu"; then echo && $SHELL "$ac_srcdir/configure.gnu" --help=recursive elif test -f "$ac_srcdir/configure"; then echo && $SHELL "$ac_srcdir/configure" --help=recursive else $as_echo "$as_me: WARNING: no configuration information is in $ac_dir" >&2 fi || ac_status=$? cd "$ac_pwd" || { ac_status=$?; break; } done fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -file configure 5.19 +file configure 5.22 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. This configure script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it. _ACEOF exit fi ## ------------------------ ## ## Autoconf initialization. ## ## ------------------------ ## # ac_fn_c_try_compile LINENO # -------------------------- # Try to compile conftest.$ac_ext, and return whether this succeeded. ac_fn_c_try_compile () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack rm -f conftest.$ac_objext if { { ac_try="$ac_compile" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_compile") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { test -z "$ac_c_werror_flag" || test ! -s conftest.err } && test -s conftest.$ac_objext; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_c_try_compile # ac_fn_c_try_cpp LINENO # ---------------------- # Try to preprocess conftest.$ac_ext, and return whether this succeeded. ac_fn_c_try_cpp () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack if { { ac_try="$ac_cpp conftest.$ac_ext" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_cpp conftest.$ac_ext") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } > conftest.i && { test -z "$ac_c_preproc_warn_flag$ac_c_werror_flag" || test ! -s conftest.err }; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_c_try_cpp # ac_fn_c_check_header_mongrel LINENO HEADER VAR INCLUDES # ------------------------------------------------------- # Tests whether HEADER exists, giving a warning if it cannot be compiled using # the include files in INCLUDES and setting the cache variable VAR # accordingly. ac_fn_c_check_header_mongrel () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack if eval \${$3+:} false; then : { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } else # Is the header compilable? { $as_echo "$as_me:${as_lineno-$LINENO}: checking $2 usability" >&5 $as_echo_n "checking $2 usability... " >&6; } cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $4 #include <$2> _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_header_compiler=yes else ac_header_compiler=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_header_compiler" >&5 $as_echo "$ac_header_compiler" >&6; } # Is the header present? { $as_echo "$as_me:${as_lineno-$LINENO}: checking $2 presence" >&5 $as_echo_n "checking $2 presence... " >&6; } cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include <$2> _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : ac_header_preproc=yes else ac_header_preproc=no fi rm -f conftest.err conftest.i conftest.$ac_ext { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_header_preproc" >&5 $as_echo "$ac_header_preproc" >&6; } # So? What about this header? case $ac_header_compiler:$ac_header_preproc:$ac_c_preproc_warn_flag in #(( yes:no: ) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: accepted by the compiler, rejected by the preprocessor!" >&5 $as_echo "$as_me: WARNING: $2: accepted by the compiler, rejected by the preprocessor!" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 $as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} ;; no:yes:* ) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: present but cannot be compiled" >&5 $as_echo "$as_me: WARNING: $2: present but cannot be compiled" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: check for missing prerequisite headers?" >&5 $as_echo "$as_me: WARNING: $2: check for missing prerequisite headers?" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: see the Autoconf documentation" >&5 $as_echo "$as_me: WARNING: $2: see the Autoconf documentation" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&5 $as_echo "$as_me: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 $as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} ( $as_echo "## ---------------------------------- ## ## Report this to christos@astron.com ## ## ---------------------------------- ##" ) | sed "s/^/$as_me: WARNING: /" >&2 ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else eval "$3=\$ac_header_compiler" fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } fi eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_header_mongrel # ac_fn_c_try_run LINENO # ---------------------- # Try to link conftest.$ac_ext, and return whether this succeeded. Assumes # that executables *can* be run. ac_fn_c_try_run () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack if { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { ac_try='./conftest$ac_exeext' { { case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_try") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; }; then : ac_retval=0 else $as_echo "$as_me: program exited with status $ac_status" >&5 $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=$ac_status fi rm -rf conftest.dSYM conftest_ipa8_conftest.oo eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_c_try_run # ac_fn_c_check_header_compile LINENO HEADER VAR INCLUDES # ------------------------------------------------------- # Tests whether HEADER exists and can be compiled using the include files in # INCLUDES, setting the cache variable VAR accordingly. ac_fn_c_check_header_compile () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $4 #include <$2> _ACEOF if ac_fn_c_try_compile "$LINENO"; then : eval "$3=yes" else eval "$3=no" fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_header_compile # ac_fn_c_try_link LINENO # ----------------------- # Try to link conftest.$ac_ext, and return whether this succeeded. ac_fn_c_try_link () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack rm -f conftest.$ac_objext conftest$ac_exeext if { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>conftest.err ac_status=$? if test -s conftest.err; then grep -v '^ *+' conftest.err >conftest.er1 cat conftest.er1 >&5 mv -f conftest.er1 conftest.err fi $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && { test -z "$ac_c_werror_flag" || test ! -s conftest.err } && test -s conftest$ac_exeext && { test "$cross_compiling" = yes || test -x conftest$ac_exeext }; then : ac_retval=0 else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 ac_retval=1 fi # Delete the IPA/IPO (Inter Procedural Analysis/Optimization) information # created by the PGI compiler (conftest_ipa8_conftest.oo), as it would # interfere with the next link command; also delete a directory that is # left behind by Apple's compiler. We do this before executing the actions. rm -rf conftest.dSYM conftest_ipa8_conftest.oo eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno as_fn_set_status $ac_retval } # ac_fn_c_try_link # ac_fn_c_check_func LINENO FUNC VAR # ---------------------------------- # Tests whether FUNC exists, setting the cache variable VAR accordingly ac_fn_c_check_func () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Define $2 to an innocuous variant, in case declares $2. For example, HP-UX 11i declares gettimeofday. */ #define $2 innocuous_$2 /* System header to define __stub macros and hopefully few prototypes, which can conflict with char $2 (); below. Prefer to if __STDC__ is defined, since exists even on freestanding compilers. */ #ifdef __STDC__ # include #else # include #endif #undef $2 /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char $2 (); /* The GNU C library defines this for functions which it implements to always fail with ENOSYS. Some functions are actually named something starting with __ and the normal name is an alias. */ #if defined __stub_$2 || defined __stub___$2 choke me #endif int main () { return $2 (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : eval "$3=yes" else eval "$3=no" fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_func # ac_fn_c_check_type LINENO TYPE VAR INCLUDES # ------------------------------------------- # Tests whether TYPE exists after having included INCLUDES, setting cache # variable VAR accordingly. ac_fn_c_check_type () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 $as_echo_n "checking for $2... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else eval "$3=no" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $4 int main () { if (sizeof ($2)) return 0; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $4 int main () { if (sizeof (($2))) return 0; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : else eval "$3=yes" fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_type # ac_fn_c_check_member LINENO AGGR MEMBER VAR INCLUDES # ---------------------------------------------------- # Tries to find if the field MEMBER exists in type AGGR, after including # INCLUDES, setting cache variable VAR accordingly. ac_fn_c_check_member () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2.$3" >&5 $as_echo_n "checking for $2.$3... " >&6; } if eval \${$4+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $5 int main () { static $2 ac_aggr; if (ac_aggr.$3) return 0; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : eval "$4=yes" else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $5 int main () { static $2 ac_aggr; if (sizeof ac_aggr.$3) return 0; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : eval "$4=yes" else eval "$4=no" fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi eval ac_res=\$$4 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_member # ac_fn_c_check_decl LINENO SYMBOL VAR INCLUDES # --------------------------------------------- # Tests whether SYMBOL is declared in INCLUDES, setting cache variable VAR # accordingly. ac_fn_c_check_decl () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack as_decl_name=`echo $2|sed 's/ *(.*//'` as_decl_use=`echo $2|sed -e 's/(/((/' -e 's/)/) 0&/' -e 's/,/) 0& (/g'` { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $as_decl_name is declared" >&5 $as_echo_n "checking whether $as_decl_name is declared... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $4 int main () { #ifndef $as_decl_name #ifdef __cplusplus (void) $as_decl_use; #else (void) $as_decl_name; #endif #endif ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : eval "$3=yes" else eval "$3=no" fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_decl # ac_fn_c_find_uintX_t LINENO BITS VAR # ------------------------------------ # Finds an unsigned integer type with width BITS, setting cache variable VAR # accordingly. ac_fn_c_find_uintX_t () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack { $as_echo "$as_me:${as_lineno-$LINENO}: checking for uint$2_t" >&5 $as_echo_n "checking for uint$2_t... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else eval "$3=no" # Order is important - never check a type that is potentially smaller # than half of the expected target width. for ac_type in uint$2_t 'unsigned int' 'unsigned long int' \ 'unsigned long long int' 'unsigned short int' 'unsigned char'; do cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default int main () { static int test_array [1 - 2 * !((($ac_type) -1 >> ($2 / 2 - 1)) >> ($2 / 2 - 1) == 3)]; test_array [0] = 0; return test_array [0]; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : case $ac_type in #( uint$2_t) : eval "$3=yes" ;; #( *) : eval "$3=\$ac_type" ;; esac fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext if eval test \"x\$"$3"\" = x"no"; then : else break fi done fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_find_uintX_t # ac_fn_c_find_intX_t LINENO BITS VAR # ----------------------------------- # Finds a signed integer type with width BITS, setting cache variable VAR # accordingly. ac_fn_c_find_intX_t () { as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack { $as_echo "$as_me:${as_lineno-$LINENO}: checking for int$2_t" >&5 $as_echo_n "checking for int$2_t... " >&6; } if eval \${$3+:} false; then : $as_echo_n "(cached) " >&6 else eval "$3=no" # Order is important - never check a type that is potentially smaller # than half of the expected target width. for ac_type in int$2_t 'int' 'long int' \ 'long long int' 'short int' 'signed char'; do cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default enum { N = $2 / 2 - 1 }; int main () { static int test_array [1 - 2 * !(0 < ($ac_type) ((((($ac_type) 1 << N) << N) - 1) * 2 + 1))]; test_array [0] = 0; return test_array [0]; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default enum { N = $2 / 2 - 1 }; int main () { static int test_array [1 - 2 * !(($ac_type) ((((($ac_type) 1 << N) << N) - 1) * 2 + 1) < ($ac_type) ((((($ac_type) 1 << N) << N) - 1) * 2 + 2))]; test_array [0] = 0; return test_array [0]; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : else case $ac_type in #( int$2_t) : eval "$3=yes" ;; #( *) : eval "$3=\$ac_type" ;; esac fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext if eval test \"x\$"$3"\" = x"no"; then : else break fi done fi eval ac_res=\$$3 { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 $as_echo "$ac_res" >&6; } eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_find_intX_t cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by file $as_me 5.19, which was +It was created by file $as_me 5.22, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ _ACEOF exec 5>>config.log { cat <<_ASUNAME ## --------- ## ## Platform. ## ## --------- ## hostname = `(hostname || uname -n) 2>/dev/null | sed 1q` uname -m = `(uname -m) 2>/dev/null || echo unknown` uname -r = `(uname -r) 2>/dev/null || echo unknown` uname -s = `(uname -s) 2>/dev/null || echo unknown` uname -v = `(uname -v) 2>/dev/null || echo unknown` /usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null || echo unknown` /bin/uname -X = `(/bin/uname -X) 2>/dev/null || echo unknown` /bin/arch = `(/bin/arch) 2>/dev/null || echo unknown` /usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null || echo unknown` /usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null || echo unknown` /usr/bin/hostinfo = `(/usr/bin/hostinfo) 2>/dev/null || echo unknown` /bin/machine = `(/bin/machine) 2>/dev/null || echo unknown` /usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null || echo unknown` /bin/universe = `(/bin/universe) 2>/dev/null || echo unknown` _ASUNAME as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. $as_echo "PATH: $as_dir" done IFS=$as_save_IFS } >&5 cat >&5 <<_ACEOF ## ----------- ## ## Core tests. ## ## ----------- ## _ACEOF # Keep a trace of the command line. # Strip out --no-create and --no-recursion so they do not pile up. # Strip out --silent because we don't want to record it for future runs. # Also quote any args containing shell meta-characters. # Make two passes to allow for proper duplicate-argument suppression. ac_configure_args= ac_configure_args0= ac_configure_args1= ac_must_keep_next=false for ac_pass in 1 2 do for ac_arg do case $ac_arg in -no-create | --no-c* | -n | -no-recursion | --no-r*) continue ;; -q | -quiet | --quiet | --quie | --qui | --qu | --q \ | -silent | --silent | --silen | --sile | --sil) continue ;; *\'*) ac_arg=`$as_echo "$ac_arg" | sed "s/'/'\\\\\\\\''/g"` ;; esac case $ac_pass in 1) as_fn_append ac_configure_args0 " '$ac_arg'" ;; 2) as_fn_append ac_configure_args1 " '$ac_arg'" if test $ac_must_keep_next = true; then ac_must_keep_next=false # Got value, back to normal. else case $ac_arg in *=* | --config-cache | -C | -disable-* | --disable-* \ | -enable-* | --enable-* | -gas | --g* | -nfp | --nf* \ | -q | -quiet | --q* | -silent | --sil* | -v | -verb* \ | -with-* | --with-* | -without-* | --without-* | --x) case "$ac_configure_args0 " in "$ac_configure_args1"*" '$ac_arg' "* ) continue ;; esac ;; -* ) ac_must_keep_next=true ;; esac fi as_fn_append ac_configure_args " '$ac_arg'" ;; esac done done { ac_configure_args0=; unset ac_configure_args0;} { ac_configure_args1=; unset ac_configure_args1;} # When interrupted or exit'd, cleanup temporary files, and complete # config.log. We remove comments because anyway the quotes in there # would cause problems or look ugly. # WARNING: Use '\'' to represent an apostrophe within the trap. # WARNING: Do not start the trap code with a newline, due to a FreeBSD 4.0 bug. trap 'exit_status=$? # Save into config.log some information that might help in debugging. { echo $as_echo "## ---------------- ## ## Cache variables. ## ## ---------------- ##" echo # The following way of writing the cache mishandles newlines in values, ( for ac_var in `(set) 2>&1 | sed -n '\''s/^\([a-zA-Z_][a-zA-Z0-9_]*\)=.*/\1/p'\''`; do eval ac_val=\$$ac_var case $ac_val in #( *${as_nl}*) case $ac_var in #( *_cv_*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline" >&5 $as_echo "$as_me: WARNING: cache variable $ac_var contains a newline" >&2;} ;; esac case $ac_var in #( _ | IFS | as_nl) ;; #( BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #( *) { eval $ac_var=; unset $ac_var;} ;; esac ;; esac done (set) 2>&1 | case $as_nl`(ac_space='\'' '\''; set) 2>&1` in #( *${as_nl}ac_space=\ *) sed -n \ "s/'\''/'\''\\\\'\'''\''/g; s/^\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\)=\\(.*\\)/\\1='\''\\2'\''/p" ;; #( *) sed -n "/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p" ;; esac | sort ) echo $as_echo "## ----------------- ## ## Output variables. ## ## ----------------- ##" echo for ac_var in $ac_subst_vars do eval ac_val=\$$ac_var case $ac_val in *\'\''*) ac_val=`$as_echo "$ac_val" | sed "s/'\''/'\''\\\\\\\\'\'''\''/g"`;; esac $as_echo "$ac_var='\''$ac_val'\''" done | sort echo if test -n "$ac_subst_files"; then $as_echo "## ------------------- ## ## File substitutions. ## ## ------------------- ##" echo for ac_var in $ac_subst_files do eval ac_val=\$$ac_var case $ac_val in *\'\''*) ac_val=`$as_echo "$ac_val" | sed "s/'\''/'\''\\\\\\\\'\'''\''/g"`;; esac $as_echo "$ac_var='\''$ac_val'\''" done | sort echo fi if test -s confdefs.h; then $as_echo "## ----------- ## ## confdefs.h. ## ## ----------- ##" echo cat confdefs.h echo fi test "$ac_signal" != 0 && $as_echo "$as_me: caught signal $ac_signal" $as_echo "$as_me: exit $exit_status" } >&5 rm -f core *.core core.conftest.* && rm -f -r conftest* confdefs* conf$$* $ac_clean_files && exit $exit_status ' 0 for ac_signal in 1 2 13 15; do trap 'ac_signal='$ac_signal'; as_fn_exit 1' $ac_signal done ac_signal=0 # confdefs.h avoids OS command line length limits that DEFS can exceed. rm -f -r conftest* confdefs.h $as_echo "/* confdefs.h */" > confdefs.h # Predefined preprocessor variables. cat >>confdefs.h <<_ACEOF #define PACKAGE_NAME "$PACKAGE_NAME" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_TARNAME "$PACKAGE_TARNAME" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_VERSION "$PACKAGE_VERSION" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_STRING "$PACKAGE_STRING" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_BUGREPORT "$PACKAGE_BUGREPORT" _ACEOF cat >>confdefs.h <<_ACEOF #define PACKAGE_URL "$PACKAGE_URL" _ACEOF # Let the site file select an alternate cache file if it wants to. # Prefer an explicitly selected file to automatically selected ones. ac_site_file1=NONE ac_site_file2=NONE if test -n "$CONFIG_SITE"; then # We do not want a PATH search for config.site. case $CONFIG_SITE in #(( -*) ac_site_file1=./$CONFIG_SITE;; */*) ac_site_file1=$CONFIG_SITE;; *) ac_site_file1=./$CONFIG_SITE;; esac elif test "x$prefix" != xNONE; then ac_site_file1=$prefix/share/config.site ac_site_file2=$prefix/etc/config.site else ac_site_file1=$ac_default_prefix/share/config.site ac_site_file2=$ac_default_prefix/etc/config.site fi for ac_site_file in "$ac_site_file1" "$ac_site_file2" do test "x$ac_site_file" = xNONE && continue if test /dev/null != "$ac_site_file" && test -r "$ac_site_file"; then { $as_echo "$as_me:${as_lineno-$LINENO}: loading site script $ac_site_file" >&5 $as_echo "$as_me: loading site script $ac_site_file" >&6;} sed 's/^/| /' "$ac_site_file" >&5 . "$ac_site_file" \ || { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "failed to load site script $ac_site_file See \`config.log' for more details" "$LINENO" 5; } fi done if test -r "$cache_file"; then # Some versions of bash will fail to source /dev/null (special files # actually), so we avoid doing that. DJGPP emulates it as a regular file. if test /dev/null != "$cache_file" && test -f "$cache_file"; then { $as_echo "$as_me:${as_lineno-$LINENO}: loading cache $cache_file" >&5 $as_echo "$as_me: loading cache $cache_file" >&6;} case $cache_file in [\\/]* | ?:[\\/]* ) . "$cache_file";; *) . "./$cache_file";; esac fi else { $as_echo "$as_me:${as_lineno-$LINENO}: creating cache $cache_file" >&5 $as_echo "$as_me: creating cache $cache_file" >&6;} >$cache_file fi as_fn_append ac_header_list " stdlib.h" as_fn_append ac_header_list " unistd.h" as_fn_append ac_header_list " sys/param.h" # Check that the precious variables saved in the cache have kept the same # value. ac_cache_corrupted=false for ac_var in $ac_precious_vars; do eval ac_old_set=\$ac_cv_env_${ac_var}_set eval ac_new_set=\$ac_env_${ac_var}_set eval ac_old_val=\$ac_cv_env_${ac_var}_value eval ac_new_val=\$ac_env_${ac_var}_value case $ac_old_set,$ac_new_set in set,) { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' was set to \`$ac_old_val' in the previous run" >&5 $as_echo "$as_me: error: \`$ac_var' was set to \`$ac_old_val' in the previous run" >&2;} ac_cache_corrupted=: ;; ,set) { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' was not set in the previous run" >&5 $as_echo "$as_me: error: \`$ac_var' was not set in the previous run" >&2;} ac_cache_corrupted=: ;; ,);; *) if test "x$ac_old_val" != "x$ac_new_val"; then # differences in whitespace do not lead to failure. ac_old_val_w=`echo x $ac_old_val` ac_new_val_w=`echo x $ac_new_val` if test "$ac_old_val_w" != "$ac_new_val_w"; then { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' has changed since the previous run:" >&5 $as_echo "$as_me: error: \`$ac_var' has changed since the previous run:" >&2;} ac_cache_corrupted=: else { $as_echo "$as_me:${as_lineno-$LINENO}: warning: ignoring whitespace changes in \`$ac_var' since the previous run:" >&5 $as_echo "$as_me: warning: ignoring whitespace changes in \`$ac_var' since the previous run:" >&2;} eval $ac_var=\$ac_old_val fi { $as_echo "$as_me:${as_lineno-$LINENO}: former value: \`$ac_old_val'" >&5 $as_echo "$as_me: former value: \`$ac_old_val'" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: current value: \`$ac_new_val'" >&5 $as_echo "$as_me: current value: \`$ac_new_val'" >&2;} fi;; esac # Pass precious variables to config.status. if test "$ac_new_set" = set; then case $ac_new_val in *\'*) ac_arg=$ac_var=`$as_echo "$ac_new_val" | sed "s/'/'\\\\\\\\''/g"` ;; *) ac_arg=$ac_var=$ac_new_val ;; esac case " $ac_configure_args " in *" '$ac_arg' "*) ;; # Avoid dups. Use of quotes ensures accuracy. *) as_fn_append ac_configure_args " '$ac_arg'" ;; esac fi done if $ac_cache_corrupted; then { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} { $as_echo "$as_me:${as_lineno-$LINENO}: error: changes in the environment can compromise the build" >&5 $as_echo "$as_me: error: changes in the environment can compromise the build" >&2;} as_fn_error $? "run \`make distclean' and/or \`rm $cache_file' and start over" "$LINENO" 5 fi ## -------------------- ## ## Main body of script. ## ## -------------------- ## ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu am__api_version='1.14' ac_aux_dir= for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do if test -f "$ac_dir/install-sh"; then ac_aux_dir=$ac_dir ac_install_sh="$ac_aux_dir/install-sh -c" break elif test -f "$ac_dir/install.sh"; then ac_aux_dir=$ac_dir ac_install_sh="$ac_aux_dir/install.sh -c" break elif test -f "$ac_dir/shtool"; then ac_aux_dir=$ac_dir ac_install_sh="$ac_aux_dir/shtool install -c" break fi done if test -z "$ac_aux_dir"; then as_fn_error $? "cannot find install-sh, install.sh, or shtool in \"$srcdir\" \"$srcdir/..\" \"$srcdir/../..\"" "$LINENO" 5 fi # These three variables are undocumented and unsupported, # and are intended to be withdrawn in a future Autoconf release. # They can cause serious problems if a builder's source tree is in a directory # whose full name contains unusual characters. ac_config_guess="$SHELL $ac_aux_dir/config.guess" # Please don't use this var. ac_config_sub="$SHELL $ac_aux_dir/config.sub" # Please don't use this var. ac_configure="$SHELL $ac_aux_dir/configure" # Please don't use this var. # Find a good install program. We prefer a C program (faster), # so one script is as good as another. But avoid the broken or # incompatible versions: # SysV /etc/install, /usr/sbin/install # SunOS /usr/etc/install # IRIX /sbin/install # AIX /bin/install # AmigaOS /C/install, which installs bootblocks on floppy discs # AIX 4 /usr/bin/installbsd, which doesn't work without a -g flag # AFS /usr/afsws/bin/install, which mishandles nonexistent args # SVR4 /usr/ucb/install, which tries to use the nonexistent group "staff" # OS/2's system install, which has a completely different semantic # ./install, which can be erroneously created by make from ./install.sh. # Reject install programs that cannot install multiple files. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for a BSD-compatible install" >&5 $as_echo_n "checking for a BSD-compatible install... " >&6; } if test -z "$INSTALL"; then if ${ac_cv_path_install+:} false; then : $as_echo_n "(cached) " >&6 else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. # Account for people who put trailing slashes in PATH elements. case $as_dir/ in #(( ./ | .// | /[cC]/* | \ /etc/* | /usr/sbin/* | /usr/etc/* | /sbin/* | /usr/afsws/bin/* | \ ?:[\\/]os2[\\/]install[\\/]* | ?:[\\/]OS2[\\/]INSTALL[\\/]* | \ /usr/ucb/* ) ;; *) # OSF1 and SCO ODT 3.0 have their own names for install. # Don't use installbsd from OSF since it installs stuff as root # by default. for ac_prog in ginstall scoinst install; do for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_prog$ac_exec_ext"; then if test $ac_prog = install && grep dspmsg "$as_dir/$ac_prog$ac_exec_ext" >/dev/null 2>&1; then # AIX install. It has an incompatible calling convention. : elif test $ac_prog = install && grep pwplus "$as_dir/$ac_prog$ac_exec_ext" >/dev/null 2>&1; then # program-specific install script used by HP pwplus--don't use. : else rm -rf conftest.one conftest.two conftest.dir echo one > conftest.one echo two > conftest.two mkdir conftest.dir if "$as_dir/$ac_prog$ac_exec_ext" -c conftest.one conftest.two "`pwd`/conftest.dir" && test -s conftest.one && test -s conftest.two && test -s conftest.dir/conftest.one && test -s conftest.dir/conftest.two then ac_cv_path_install="$as_dir/$ac_prog$ac_exec_ext -c" break 3 fi fi fi done done ;; esac done IFS=$as_save_IFS rm -rf conftest.one conftest.two conftest.dir fi if test "${ac_cv_path_install+set}" = set; then INSTALL=$ac_cv_path_install else # As a last resort, use the slow shell script. Don't cache a # value for INSTALL within a source directory, because that will # break other packages using the cache if that directory is # removed, or if the value is a relative name. INSTALL=$ac_install_sh fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $INSTALL" >&5 $as_echo "$INSTALL" >&6; } # Use test -z because SunOS4 sh mishandles braces in ${var-val}. # It thinks the first close brace ends the variable substitution. test -z "$INSTALL_PROGRAM" && INSTALL_PROGRAM='${INSTALL}' test -z "$INSTALL_SCRIPT" && INSTALL_SCRIPT='${INSTALL}' test -z "$INSTALL_DATA" && INSTALL_DATA='${INSTALL} -m 644' { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether build environment is sane" >&5 $as_echo_n "checking whether build environment is sane... " >&6; } # Reject unsafe characters in $srcdir or the absolute working directory # name. Accept space and tab only in the latter. am_lf=' ' case `pwd` in *[\\\"\#\$\&\'\`$am_lf]*) as_fn_error $? "unsafe absolute working directory name" "$LINENO" 5;; esac case $srcdir in *[\\\"\#\$\&\'\`$am_lf\ \ ]*) as_fn_error $? "unsafe srcdir value: '$srcdir'" "$LINENO" 5;; esac # Do 'set' in a subshell so we don't clobber the current shell's # arguments. Must try -L first in case configure is actually a # symlink; some systems play weird games with the mod time of symlinks # (eg FreeBSD returns the mod time of the symlink's containing # directory). if ( am_has_slept=no for am_try in 1 2; do echo "timestamp, slept: $am_has_slept" > conftest.file set X `ls -Lt "$srcdir/configure" conftest.file 2> /dev/null` if test "$*" = "X"; then # -L didn't work. set X `ls -t "$srcdir/configure" conftest.file` fi if test "$*" != "X $srcdir/configure conftest.file" \ && test "$*" != "X conftest.file $srcdir/configure"; then # If neither matched, then we have a broken ls. This can happen # if, for instance, CONFIG_SHELL is bash and it inherits a # broken ls alias from the environment. This has actually # happened. Such a system could not be considered "sane". as_fn_error $? "ls -t appears to fail. Make sure there is not a broken alias in your environment" "$LINENO" 5 fi if test "$2" = conftest.file || test $am_try -eq 2; then break fi # Just in case. sleep 1 am_has_slept=yes done test "$2" = conftest.file ) then # Ok. : else as_fn_error $? "newly created file is older than distributed files! Check your system clock" "$LINENO" 5 fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } # If we didn't sleep, we still need to ensure time stamps of config.status and # generated files are strictly newer. am_sleep_pid= if grep 'slept: no' conftest.file >/dev/null 2>&1; then ( sleep 1 ) & am_sleep_pid=$! fi rm -f conftest.file test "$program_prefix" != NONE && program_transform_name="s&^&$program_prefix&;$program_transform_name" # Use a double $ so make ignores it. test "$program_suffix" != NONE && program_transform_name="s&\$&$program_suffix&;$program_transform_name" # Double any \ or $. # By default was `s,x,x', remove it if useless. ac_script='s/[\\$]/&&/g;s/;s,x,x,$//' program_transform_name=`$as_echo "$program_transform_name" | sed "$ac_script"` # expand $ac_aux_dir to an absolute path am_aux_dir=`cd $ac_aux_dir && pwd` if test x"${MISSING+set}" != xset; then case $am_aux_dir in *\ * | *\ *) MISSING="\${SHELL} \"$am_aux_dir/missing\"" ;; *) MISSING="\${SHELL} $am_aux_dir/missing" ;; esac fi # Use eval to expand $SHELL if eval "$MISSING --is-lightweight"; then am_missing_run="$MISSING " else am_missing_run= { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: 'missing' script is too old or missing" >&5 $as_echo "$as_me: WARNING: 'missing' script is too old or missing" >&2;} fi if test x"${install_sh}" != xset; then case $am_aux_dir in *\ * | *\ *) install_sh="\${SHELL} '$am_aux_dir/install-sh'" ;; *) install_sh="\${SHELL} $am_aux_dir/install-sh" esac fi # Installed binaries are usually stripped using 'strip' when the user # run "make install-strip". However 'strip' might not be the right # tool to use in cross-compilation environments, therefore Automake # will honor the 'STRIP' environment variable to overrule this program. if test "$cross_compiling" != no; then if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}strip", so it can be a program name with args. set dummy ${ac_tool_prefix}strip; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_STRIP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$STRIP"; then ac_cv_prog_STRIP="$STRIP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_STRIP="${ac_tool_prefix}strip" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi STRIP=$ac_cv_prog_STRIP if test -n "$STRIP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $STRIP" >&5 $as_echo "$STRIP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_STRIP"; then ac_ct_STRIP=$STRIP # Extract the first word of "strip", so it can be a program name with args. set dummy strip; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_STRIP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_STRIP"; then ac_cv_prog_ac_ct_STRIP="$ac_ct_STRIP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_STRIP="strip" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_STRIP=$ac_cv_prog_ac_ct_STRIP if test -n "$ac_ct_STRIP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_STRIP" >&5 $as_echo "$ac_ct_STRIP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_STRIP" = x; then STRIP=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac STRIP=$ac_ct_STRIP fi else STRIP="$ac_cv_prog_STRIP" fi fi INSTALL_STRIP_PROGRAM="\$(install_sh) -c -s" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for a thread-safe mkdir -p" >&5 $as_echo_n "checking for a thread-safe mkdir -p... " >&6; } if test -z "$MKDIR_P"; then if ${ac_cv_path_mkdir+:} false; then : $as_echo_n "(cached) " >&6 else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH$PATH_SEPARATOR/opt/sfw/bin do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in mkdir gmkdir; do for ac_exec_ext in '' $ac_executable_extensions; do as_fn_executable_p "$as_dir/$ac_prog$ac_exec_ext" || continue case `"$as_dir/$ac_prog$ac_exec_ext" --version 2>&1` in #( 'mkdir (GNU coreutils) '* | \ 'mkdir (coreutils) '* | \ 'mkdir (fileutils) '4.1*) ac_cv_path_mkdir=$as_dir/$ac_prog$ac_exec_ext break 3;; esac done done done IFS=$as_save_IFS fi test -d ./--version && rmdir ./--version if test "${ac_cv_path_mkdir+set}" = set; then MKDIR_P="$ac_cv_path_mkdir -p" else # As a last resort, use the slow shell script. Don't cache a # value for MKDIR_P within a source directory, because that will # break other packages using the cache if that directory is # removed, or if the value is a relative name. MKDIR_P="$ac_install_sh -d" fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MKDIR_P" >&5 $as_echo "$MKDIR_P" >&6; } for ac_prog in gawk mawk nawk awk do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_AWK+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$AWK"; then ac_cv_prog_AWK="$AWK" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_AWK="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi AWK=$ac_cv_prog_AWK if test -n "$AWK"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $AWK" >&5 $as_echo "$AWK" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$AWK" && break done { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${MAKE-make} sets \$(MAKE)" >&5 $as_echo_n "checking whether ${MAKE-make} sets \$(MAKE)... " >&6; } set x ${MAKE-make} ac_make=`$as_echo "$2" | sed 's/+/p/g; s/[^a-zA-Z0-9_]/_/g'` if eval \${ac_cv_prog_make_${ac_make}_set+:} false; then : $as_echo_n "(cached) " >&6 else cat >conftest.make <<\_ACEOF SHELL = /bin/sh all: @echo '@@@%%%=$(MAKE)=@@@%%%' _ACEOF # GNU make sometimes prints "make[1]: Entering ...", which would confuse us. case `${MAKE-make} -f conftest.make 2>/dev/null` in *@@@%%%=?*=@@@%%%*) eval ac_cv_prog_make_${ac_make}_set=yes;; *) eval ac_cv_prog_make_${ac_make}_set=no;; esac rm -f conftest.make fi if eval test \$ac_cv_prog_make_${ac_make}_set = yes; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } SET_MAKE= else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } SET_MAKE="MAKE=${MAKE-make}" fi rm -rf .tst 2>/dev/null mkdir .tst 2>/dev/null if test -d .tst; then am__leading_dot=. else am__leading_dot=_ fi rmdir .tst 2>/dev/null # Check whether --enable-silent-rules was given. if test "${enable_silent_rules+set}" = set; then : enableval=$enable_silent_rules; fi case $enable_silent_rules in # ((( yes) AM_DEFAULT_VERBOSITY=0;; no) AM_DEFAULT_VERBOSITY=1;; *) AM_DEFAULT_VERBOSITY=1;; esac am_make=${MAKE-make} { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $am_make supports nested variables" >&5 $as_echo_n "checking whether $am_make supports nested variables... " >&6; } if ${am_cv_make_support_nested_variables+:} false; then : $as_echo_n "(cached) " >&6 else if $as_echo 'TRUE=$(BAR$(V)) BAR0=false BAR1=true V=1 am__doit: @$(TRUE) .PHONY: am__doit' | $am_make -f - >/dev/null 2>&1; then am_cv_make_support_nested_variables=yes else am_cv_make_support_nested_variables=no fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $am_cv_make_support_nested_variables" >&5 $as_echo "$am_cv_make_support_nested_variables" >&6; } if test $am_cv_make_support_nested_variables = yes; then AM_V='$(V)' AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)' else AM_V=$AM_DEFAULT_VERBOSITY AM_DEFAULT_V=$AM_DEFAULT_VERBOSITY fi AM_BACKSLASH='\' if test "`cd $srcdir && pwd`" != "`pwd`"; then # Use -I$(srcdir) only when $(srcdir) != ., so that make's output # is not polluted with repeated "-I." am__isrc=' -I$(srcdir)' # test to see if srcdir already configured if test -f $srcdir/config.status; then as_fn_error $? "source directory already configured; run \"make distclean\" there first" "$LINENO" 5 fi fi # test whether we have cygpath if test -z "$CYGPATH_W"; then if (cygpath --version) >/dev/null 2>/dev/null; then CYGPATH_W='cygpath -w' else CYGPATH_W=echo fi fi # Define the identity of the package. PACKAGE='file' - VERSION='5.19' + VERSION='5.22' cat >>confdefs.h <<_ACEOF #define PACKAGE "$PACKAGE" _ACEOF cat >>confdefs.h <<_ACEOF #define VERSION "$VERSION" _ACEOF # Some tools Automake needs. ACLOCAL=${ACLOCAL-"${am_missing_run}aclocal-${am__api_version}"} AUTOCONF=${AUTOCONF-"${am_missing_run}autoconf"} AUTOMAKE=${AUTOMAKE-"${am_missing_run}automake-${am__api_version}"} AUTOHEADER=${AUTOHEADER-"${am_missing_run}autoheader"} MAKEINFO=${MAKEINFO-"${am_missing_run}makeinfo"} # For better backward compatibility. To be removed once Automake 1.9.x # dies out for good. For more background, see: # # mkdir_p='$(MKDIR_P)' # We need awk for the "check" target. The system "awk" is bad on # some platforms. # Always define AMTAR for backward compatibility. Yes, it's still used # in the wild :-( We should find a proper way to deprecate it ... AMTAR='$${TAR-tar}' # We'll loop over all known methods to create a tar archive until one works. _am_tools='gnutar pax cpio none' am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -' # POSIX will say in a future version that running "rm -f" with no argument # is OK; and we want to be able to make that assumption in our Makefile # recipes. So use an aggressive probe to check that the usage we want is # actually supported "in the wild" to an acceptable degree. # See automake bug#10828. # To make any issue more visible, cause the running configure to be aborted # by default if the 'rm' program in use doesn't match our expectations; the # user can still override this though. if rm -f && rm -fr && rm -rf; then : OK; else cat >&2 <<'END' Oops! Your 'rm' program seems unable to run without file operands specified on the command line, even when the '-f' option is present. This is contrary to the behaviour of most rm programs out there, and not conforming with the upcoming POSIX standard: Please tell bug-automake@gnu.org about your system, including the value of your $PATH and any error possibly output before this message. This can help us improve future automake versions. END if test x"$ACCEPT_INFERIOR_RM_PROGRAM" = x"yes"; then echo 'Configuration will proceed anyway, since you have set the' >&2 echo 'ACCEPT_INFERIOR_RM_PROGRAM variable to "yes"' >&2 echo >&2 else cat >&2 <<'END' Aborting the configuration process, to ensure you take notice of the issue. You can download and install GNU coreutils to get an 'rm' implementation that behaves properly: . If you want to complete the configuration process using your problematic 'rm' anyway, export the environment variable ACCEPT_INFERIOR_RM_PROGRAM to "yes", and re-run configure. END as_fn_error $? "Your 'rm' program is bad, sorry." "$LINENO" 5 fi fi # Check whether --enable-silent-rules was given. if test "${enable_silent_rules+set}" = set; then : enableval=$enable_silent_rules; fi case $enable_silent_rules in # ((( yes) AM_DEFAULT_VERBOSITY=0;; no) AM_DEFAULT_VERBOSITY=1;; *) AM_DEFAULT_VERBOSITY=0;; esac am_make=${MAKE-make} { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $am_make supports nested variables" >&5 $as_echo_n "checking whether $am_make supports nested variables... " >&6; } if ${am_cv_make_support_nested_variables+:} false; then : $as_echo_n "(cached) " >&6 else if $as_echo 'TRUE=$(BAR$(V)) BAR0=false BAR1=true V=1 am__doit: @$(TRUE) .PHONY: am__doit' | $am_make -f - >/dev/null 2>&1; then am_cv_make_support_nested_variables=yes else am_cv_make_support_nested_variables=no fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $am_cv_make_support_nested_variables" >&5 $as_echo "$am_cv_make_support_nested_variables" >&6; } if test $am_cv_make_support_nested_variables = yes; then AM_V='$(V)' AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)' else AM_V=$AM_DEFAULT_VERBOSITY AM_DEFAULT_V=$AM_DEFAULT_VERBOSITY fi AM_BACKSLASH='\' ac_config_headers="$ac_config_headers config.h" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for builtin ELF support" >&5 $as_echo_n "checking for builtin ELF support... " >&6; } # Check whether --enable-elf was given. if test "${enable_elf+set}" = set; then : enableval=$enable_elf; if test "${enableval}" = yes; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } $as_echo "#define BUILTIN_ELF 1" >>confdefs.h else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi else # enable by default { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } $as_echo "#define BUILTIN_ELF 1" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ELF core file support" >&5 $as_echo_n "checking for ELF core file support... " >&6; } # Check whether --enable-elf-core was given. if test "${enable_elf_core+set}" = set; then : enableval=$enable_elf_core; if test "${enableval}" = yes; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } $as_echo "#define ELFCORE 1" >>confdefs.h else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi else # enable by default { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } $as_echo "#define ELFCORE 1" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for file formats in man section 5" >&5 $as_echo_n "checking for file formats in man section 5... " >&6; } # Check whether --enable-fsect-man5 was given. if test "${enable_fsect_man5+set}" = set; then : enableval=$enable_fsect_man5; if test "${enableval}" = yes; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } fsect=5 else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fsect=4 fi else # disable by default { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fsect=4 fi # Make sure we can run config.sub. $SHELL "$ac_aux_dir/config.sub" sun4 >/dev/null 2>&1 || as_fn_error $? "cannot run $SHELL $ac_aux_dir/config.sub" "$LINENO" 5 { $as_echo "$as_me:${as_lineno-$LINENO}: checking build system type" >&5 $as_echo_n "checking build system type... " >&6; } if ${ac_cv_build+:} false; then : $as_echo_n "(cached) " >&6 else ac_build_alias=$build_alias test "x$ac_build_alias" = x && ac_build_alias=`$SHELL "$ac_aux_dir/config.guess"` test "x$ac_build_alias" = x && as_fn_error $? "cannot guess build type; you must specify one" "$LINENO" 5 ac_cv_build=`$SHELL "$ac_aux_dir/config.sub" $ac_build_alias` || as_fn_error $? "$SHELL $ac_aux_dir/config.sub $ac_build_alias failed" "$LINENO" 5 fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_build" >&5 $as_echo "$ac_cv_build" >&6; } case $ac_cv_build in *-*-*) ;; *) as_fn_error $? "invalid value of canonical build" "$LINENO" 5;; esac build=$ac_cv_build ac_save_IFS=$IFS; IFS='-' set x $ac_cv_build shift build_cpu=$1 build_vendor=$2 shift; shift # Remember, the first character of IFS is used to create $*, # except with old shells: build_os=$* IFS=$ac_save_IFS case $build_os in *\ *) build_os=`echo "$build_os" | sed 's/ /-/g'`;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking host system type" >&5 $as_echo_n "checking host system type... " >&6; } if ${ac_cv_host+:} false; then : $as_echo_n "(cached) " >&6 else if test "x$host_alias" = x; then ac_cv_host=$ac_cv_build else ac_cv_host=`$SHELL "$ac_aux_dir/config.sub" $host_alias` || as_fn_error $? "$SHELL $ac_aux_dir/config.sub $host_alias failed" "$LINENO" 5 fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_host" >&5 $as_echo "$ac_cv_host" >&6; } case $ac_cv_host in *-*-*) ;; *) as_fn_error $? "invalid value of canonical host" "$LINENO" 5;; esac host=$ac_cv_host ac_save_IFS=$IFS; IFS='-' set x $ac_cv_host shift host_cpu=$1 host_vendor=$2 shift; shift # Remember, the first character of IFS is used to create $*, # except with old shells: host_os=$* IFS=$ac_save_IFS case $host_os in *\ *) host_os=`echo "$host_os" | sed 's/ /-/g'`;; esac case "$host_os" in mingw32*) MINGW=1 ;; *) MINGW=0 ;; esac if test "$MINGW" = 1; then MINGW_TRUE= MINGW_FALSE='#' else MINGW_TRUE='#' MINGW_FALSE= fi pkgdatadir='$(datadir)/misc' if test x$fsect = x5; then FSECT5_TRUE= FSECT5_FALSE='#' else FSECT5_TRUE='#' FSECT5_FALSE= fi DEPDIR="${am__leading_dot}deps" ac_config_commands="$ac_config_commands depfiles" am_make=${MAKE-make} cat > confinc << 'END' am__doit: @echo this is the am__doit target .PHONY: am__doit END # If we don't find an include directive, just comment out the code. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for style of include used by $am_make" >&5 $as_echo_n "checking for style of include used by $am_make... " >&6; } am__include="#" am__quote= _am_result=none # First try GNU make style include. echo "include confinc" > confmf # Ignore all kinds of additional output from 'make'. case `$am_make -s -f confmf 2> /dev/null` in #( *the\ am__doit\ target*) am__include=include am__quote= _am_result=GNU ;; esac # Now try BSD make style include. if test "$am__include" = "#"; then echo '.include "confinc"' > confmf case `$am_make -s -f confmf 2> /dev/null` in #( *the\ am__doit\ target*) am__include=.include am__quote="\"" _am_result=BSD ;; esac fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $_am_result" >&5 $as_echo "$_am_result" >&6; } rm -f confinc confmf # Check whether --enable-dependency-tracking was given. if test "${enable_dependency_tracking+set}" = set; then : enableval=$enable_dependency_tracking; fi if test "x$enable_dependency_tracking" != xno; then am_depcomp="$ac_aux_dir/depcomp" AMDEPBACKSLASH='\' am__nodep='_no' fi if test "x$enable_dependency_tracking" != xno; then AMDEP_TRUE= AMDEP_FALSE='#' else AMDEP_TRUE='#' AMDEP_FALSE= fi ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}gcc", so it can be a program name with args. set dummy ${ac_tool_prefix}gcc; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CC"; then ac_cv_prog_CC="$CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_CC="${ac_tool_prefix}gcc" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi CC=$ac_cv_prog_CC if test -n "$CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 $as_echo "$CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_CC"; then ac_ct_CC=$CC # Extract the first word of "gcc", so it can be a program name with args. set dummy gcc; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_CC"; then ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_CC="gcc" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_CC=$ac_cv_prog_ac_ct_CC if test -n "$ac_ct_CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_CC" >&5 $as_echo "$ac_ct_CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_CC" = x; then CC="" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac CC=$ac_ct_CC fi else CC="$ac_cv_prog_CC" fi if test -z "$CC"; then if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}cc", so it can be a program name with args. set dummy ${ac_tool_prefix}cc; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CC"; then ac_cv_prog_CC="$CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_CC="${ac_tool_prefix}cc" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi CC=$ac_cv_prog_CC if test -n "$CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 $as_echo "$CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi fi if test -z "$CC"; then # Extract the first word of "cc", so it can be a program name with args. set dummy cc; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CC"; then ac_cv_prog_CC="$CC" # Let the user override the test. else ac_prog_rejected=no as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then if test "$as_dir/$ac_word$ac_exec_ext" = "/usr/ucb/cc"; then ac_prog_rejected=yes continue fi ac_cv_prog_CC="cc" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS if test $ac_prog_rejected = yes; then # We found a bogon in the path, so make sure we never use it. set dummy $ac_cv_prog_CC shift if test $# != 0; then # We chose a different compiler from the bogus one. # However, it has the same basename, so the bogon will be chosen # first if we set CC to just the basename; use the full file name. shift ac_cv_prog_CC="$as_dir/$ac_word${1+' '}$@" fi fi fi fi CC=$ac_cv_prog_CC if test -n "$CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 $as_echo "$CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$CC"; then if test -n "$ac_tool_prefix"; then for ac_prog in cl.exe do # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. set dummy $ac_tool_prefix$ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$CC"; then ac_cv_prog_CC="$CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_CC="$ac_tool_prefix$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi CC=$ac_cv_prog_CC if test -n "$CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 $as_echo "$CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$CC" && break done fi if test -z "$CC"; then ac_ct_CC=$CC for ac_prog in cl.exe do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_CC+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_CC"; then ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_CC="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_CC=$ac_cv_prog_ac_ct_CC if test -n "$ac_ct_CC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_CC" >&5 $as_echo "$ac_ct_CC" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$ac_ct_CC" && break done if test "x$ac_ct_CC" = x; then CC="" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac CC=$ac_ct_CC fi fi fi test -z "$CC" && { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "no acceptable C compiler found in \$PATH See \`config.log' for more details" "$LINENO" 5; } # Provide some information about the compiler. $as_echo "$as_me:${as_lineno-$LINENO}: checking for C compiler version" >&5 set X $ac_compile ac_compiler=$2 for ac_option in --version -v -V -qversion; do { { ac_try="$ac_compiler $ac_option >&5" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_compiler $ac_option >&5") 2>conftest.err ac_status=$? if test -s conftest.err; then sed '10a\ ... rest of stderr output deleted ... 10q' conftest.err >conftest.er1 cat conftest.er1 >&5 fi rm -f conftest.er1 conftest.err $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } done cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF ac_clean_files_save=$ac_clean_files ac_clean_files="$ac_clean_files a.out a.out.dSYM a.exe b.out" # Try to create an executable without -o first, disregard a.out. # It will help us diagnose broken compilers, and finding out an intuition # of exeext. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the C compiler works" >&5 $as_echo_n "checking whether the C compiler works... " >&6; } ac_link_default=`$as_echo "$ac_link" | sed 's/ -o *conftest[^ ]*//'` # The possible output files: ac_files="a.out conftest.exe conftest a.exe a_out.exe b.out conftest.*" ac_rmfiles= for ac_file in $ac_files do case $ac_file in *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;; * ) ac_rmfiles="$ac_rmfiles $ac_file";; esac done rm -f $ac_rmfiles if { { ac_try="$ac_link_default" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link_default") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then : # Autoconf-2.13 could set the ac_cv_exeext variable to `no'. # So ignore a value of `no', otherwise this would lead to `EXEEXT = no' # in a Makefile. We should not override ac_cv_exeext if it was cached, # so that the user can short-circuit this test for compilers unknown to # Autoconf. for ac_file in $ac_files '' do test -f "$ac_file" || continue case $ac_file in *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;; [ab].out ) # We found the default executable, but exeext='' is most # certainly right. break;; *.* ) if test "${ac_cv_exeext+set}" = set && test "$ac_cv_exeext" != no; then :; else ac_cv_exeext=`expr "$ac_file" : '[^.]*\(\..*\)'` fi # We set ac_cv_exeext here because the later test for it is not # safe: cross compilers may not add the suffix if given an `-o' # argument, so we may need to know it at that point already. # Even if this section looks crufty: it has the advantage of # actually working. break;; * ) break;; esac done test "$ac_cv_exeext" = no && ac_cv_exeext= else ac_file='' fi if test -z "$ac_file"; then : { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error 77 "C compiler cannot create executables See \`config.log' for more details" "$LINENO" 5; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for C compiler default output file name" >&5 $as_echo_n "checking for C compiler default output file name... " >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_file" >&5 $as_echo "$ac_file" >&6; } ac_exeext=$ac_cv_exeext rm -f -r a.out a.out.dSYM a.exe conftest$ac_cv_exeext b.out ac_clean_files=$ac_clean_files_save { $as_echo "$as_me:${as_lineno-$LINENO}: checking for suffix of executables" >&5 $as_echo_n "checking for suffix of executables... " >&6; } if { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then : # If both `conftest.exe' and `conftest' are `present' (well, observable) # catch `conftest.exe'. For instance with Cygwin, `ls conftest' will # work properly (i.e., refer to `conftest.exe'), while it won't with # `rm'. for ac_file in conftest.exe conftest conftest.*; do test -f "$ac_file" || continue case $ac_file in *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;; *.* ) ac_cv_exeext=`expr "$ac_file" : '[^.]*\(\..*\)'` break;; * ) break;; esac done else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "cannot compute suffix of executables: cannot compile and link See \`config.log' for more details" "$LINENO" 5; } fi rm -f conftest conftest$ac_cv_exeext { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_exeext" >&5 $as_echo "$ac_cv_exeext" >&6; } rm -f conftest.$ac_ext EXEEXT=$ac_cv_exeext ac_exeext=$EXEEXT cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { FILE *f = fopen ("conftest.out", "w"); return ferror (f) || fclose (f) != 0; ; return 0; } _ACEOF ac_clean_files="$ac_clean_files conftest.out" # Check that the compiler produces executables we can run. If not, either # the compiler is broken, or we cross compile. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we are cross compiling" >&5 $as_echo_n "checking whether we are cross compiling... " >&6; } if test "$cross_compiling" != yes; then { { ac_try="$ac_link" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_link") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } if { ac_try='./conftest$ac_cv_exeext' { { case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_try") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; }; then cross_compiling=no else if test "$cross_compiling" = maybe; then cross_compiling=yes else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "cannot run C compiled programs. If you meant to cross compile, use \`--host'. See \`config.log' for more details" "$LINENO" 5; } fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $cross_compiling" >&5 $as_echo "$cross_compiling" >&6; } rm -f conftest.$ac_ext conftest$ac_cv_exeext conftest.out ac_clean_files=$ac_clean_files_save { $as_echo "$as_me:${as_lineno-$LINENO}: checking for suffix of object files" >&5 $as_echo_n "checking for suffix of object files... " >&6; } if ${ac_cv_objext+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF rm -f conftest.o conftest.obj if { { ac_try="$ac_compile" case "(($ac_try" in *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; *) ac_try_echo=$ac_try;; esac eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" $as_echo "$ac_try_echo"; } >&5 (eval "$ac_compile") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then : for ac_file in conftest.o conftest.obj conftest.*; do test -f "$ac_file" || continue; case $ac_file in *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM ) ;; *) ac_cv_objext=`expr "$ac_file" : '.*\.\(.*\)'` break;; esac done else $as_echo "$as_me: failed program was:" >&5 sed 's/^/| /' conftest.$ac_ext >&5 { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "cannot compute suffix of object files: cannot compile See \`config.log' for more details" "$LINENO" 5; } fi rm -f conftest.$ac_cv_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_objext" >&5 $as_echo "$ac_cv_objext" >&6; } OBJEXT=$ac_cv_objext ac_objext=$OBJEXT { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we are using the GNU C compiler" >&5 $as_echo_n "checking whether we are using the GNU C compiler... " >&6; } if ${ac_cv_c_compiler_gnu+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { #ifndef __GNUC__ choke me #endif ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_compiler_gnu=yes else ac_compiler_gnu=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_cv_c_compiler_gnu=$ac_compiler_gnu fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_c_compiler_gnu" >&5 $as_echo "$ac_cv_c_compiler_gnu" >&6; } if test $ac_compiler_gnu = yes; then GCC=yes else GCC= fi ac_test_CFLAGS=${CFLAGS+set} ac_save_CFLAGS=$CFLAGS { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CC accepts -g" >&5 $as_echo_n "checking whether $CC accepts -g... " >&6; } if ${ac_cv_prog_cc_g+:} false; then : $as_echo_n "(cached) " >&6 else ac_save_c_werror_flag=$ac_c_werror_flag ac_c_werror_flag=yes ac_cv_prog_cc_g=no CFLAGS="-g" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_prog_cc_g=yes else CFLAGS="" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : else ac_c_werror_flag=$ac_save_c_werror_flag CFLAGS="-g" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_prog_cc_g=yes fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_c_werror_flag=$ac_save_c_werror_flag fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_g" >&5 $as_echo "$ac_cv_prog_cc_g" >&6; } if test "$ac_test_CFLAGS" = set; then CFLAGS=$ac_save_CFLAGS elif test $ac_cv_prog_cc_g = yes; then if test "$GCC" = yes; then CFLAGS="-g -O2" else CFLAGS="-g" fi else if test "$GCC" = yes; then CFLAGS="-O2" else CFLAGS= fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $CC option to accept ISO C89" >&5 $as_echo_n "checking for $CC option to accept ISO C89... " >&6; } if ${ac_cv_prog_cc_c89+:} false; then : $as_echo_n "(cached) " >&6 else ac_cv_prog_cc_c89=no ac_save_CC=$CC cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include struct stat; /* Most of the following tests are stolen from RCS 5.7's src/conf.sh. */ struct buf { int x; }; FILE * (*rcsopen) (struct buf *, struct stat *, int); static char *e (p, i) char **p; int i; { return p[i]; } static char *f (char * (*g) (char **, int), char **p, ...) { char *s; va_list v; va_start (v,p); s = g (p, va_arg (v,int)); va_end (v); return s; } /* OSF 4.0 Compaq cc is some sort of almost-ANSI by default. It has function prototypes and stuff, but not '\xHH' hex character constants. These don't provoke an error unfortunately, instead are silently treated as 'x'. The following induces an error, until -std is added to get proper ANSI mode. Curiously '\x00'!='x' always comes out true, for an array size at least. It's necessary to write '\x00'==0 to get something that's true only with -std. */ int osf4_cc_array ['\x00' == 0 ? 1 : -1]; /* IBM C 6 for AIX is almost-ANSI by default, but it replaces macro parameters inside strings and character constants. */ #define FOO(x) 'x' int xlc6_cc_array[FOO(a) == 'x' ? 1 : -1]; int test (int i, double x); struct s1 {int (*f) (int a);}; struct s2 {int (*f) (double a);}; int pairnames (int, char **, FILE *(*)(struct buf *, struct stat *, int), int, int); int argc; char **argv; int main () { return f (e, argv, 0) != argv[0] || f (e, argv, 1) != argv[1]; ; return 0; } _ACEOF for ac_arg in '' -qlanglvl=extc89 -qlanglvl=ansi -std \ -Ae "-Aa -D_HPUX_SOURCE" "-Xc -D__EXTENSIONS__" do CC="$ac_save_CC $ac_arg" if ac_fn_c_try_compile "$LINENO"; then : ac_cv_prog_cc_c89=$ac_arg fi rm -f core conftest.err conftest.$ac_objext test "x$ac_cv_prog_cc_c89" != "xno" && break done rm -f conftest.$ac_ext CC=$ac_save_CC fi # AC_CACHE_VAL case "x$ac_cv_prog_cc_c89" in x) { $as_echo "$as_me:${as_lineno-$LINENO}: result: none needed" >&5 $as_echo "none needed" >&6; } ;; xno) { $as_echo "$as_me:${as_lineno-$LINENO}: result: unsupported" >&5 $as_echo "unsupported" >&6; } ;; *) CC="$CC $ac_cv_prog_cc_c89" { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_c89" >&5 $as_echo "$ac_cv_prog_cc_c89" >&6; } ;; esac if test "x$ac_cv_prog_cc_c89" != xno; then : fi ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CC understands -c and -o together" >&5 $as_echo_n "checking whether $CC understands -c and -o together... " >&6; } if ${am_cv_prog_cc_c_o+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF # Make sure it works both with $CC and with simple cc. # Following AC_PROG_CC_C_O, we do the test twice because some # compilers refuse to overwrite an existing .o file with -o, # though they will create one. am_cv_prog_cc_c_o=yes for am_i in 1 2; do if { echo "$as_me:$LINENO: $CC -c conftest.$ac_ext -o conftest2.$ac_objext" >&5 ($CC -c conftest.$ac_ext -o conftest2.$ac_objext) >&5 2>&5 ac_status=$? echo "$as_me:$LINENO: \$? = $ac_status" >&5 (exit $ac_status); } \ && test -f conftest2.$ac_objext; then : OK else am_cv_prog_cc_c_o=no break fi done rm -f core conftest* unset am_i fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $am_cv_prog_cc_c_o" >&5 $as_echo "$am_cv_prog_cc_c_o" >&6; } if test "$am_cv_prog_cc_c_o" != yes; then # Losing compiler, so override with the script. # FIXME: It is wrong to rewrite CC. # But if we don't then we get into trouble of one sort or another. # A longer-term fix would be to have automake use am__CC in this case, # and then we could set am__CC="\$(top_srcdir)/compile \$(CC)" CC="$am_aux_dir/compile $CC" fi ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu depcc="$CC" am_compiler_list= { $as_echo "$as_me:${as_lineno-$LINENO}: checking dependency style of $depcc" >&5 $as_echo_n "checking dependency style of $depcc... " >&6; } if ${am_cv_CC_dependencies_compiler_type+:} false; then : $as_echo_n "(cached) " >&6 else if test -z "$AMDEP_TRUE" && test -f "$am_depcomp"; then # We make a subdir and do the tests there. Otherwise we can end up # making bogus files that we don't know about and never remove. For # instance it was reported that on HP-UX the gcc test will end up # making a dummy file named 'D' -- because '-MD' means "put the output # in D". rm -rf conftest.dir mkdir conftest.dir # Copy depcomp to subdir because otherwise we won't find it if we're # using a relative directory. cp "$am_depcomp" conftest.dir cd conftest.dir # We will build objects and dependencies in a subdirectory because # it helps to detect inapplicable dependency modes. For instance # both Tru64's cc and ICC support -MD to output dependencies as a # side effect of compilation, but ICC will put the dependencies in # the current directory while Tru64 will put them in the object # directory. mkdir sub am_cv_CC_dependencies_compiler_type=none if test "$am_compiler_list" = ""; then am_compiler_list=`sed -n 's/^#*\([a-zA-Z0-9]*\))$/\1/p' < ./depcomp` fi am__universal=false case " $depcc " in #( *\ -arch\ *\ -arch\ *) am__universal=true ;; esac for depmode in $am_compiler_list; do # Setup a source with many dependencies, because some compilers # like to wrap large dependency lists on column 80 (with \), and # we should not choose a depcomp mode which is confused by this. # # We need to recreate these files for each test, as the compiler may # overwrite some of them when testing with obscure command lines. # This happens at least with the AIX C compiler. : > sub/conftest.c for i in 1 2 3 4 5 6; do echo '#include "conftst'$i'.h"' >> sub/conftest.c # Using ": > sub/conftst$i.h" creates only sub/conftst1.h with # Solaris 10 /bin/sh. echo '/* dummy */' > sub/conftst$i.h done echo "${am__include} ${am__quote}sub/conftest.Po${am__quote}" > confmf # We check with '-c' and '-o' for the sake of the "dashmstdout" # mode. It turns out that the SunPro C++ compiler does not properly # handle '-M -o', and we need to detect this. Also, some Intel # versions had trouble with output in subdirs. am__obj=sub/conftest.${OBJEXT-o} am__minus_obj="-o $am__obj" case $depmode in gcc) # This depmode causes a compiler race in universal mode. test "$am__universal" = false || continue ;; nosideeffect) # After this tag, mechanisms are not by side-effect, so they'll # only be used when explicitly requested. if test "x$enable_dependency_tracking" = xyes; then continue else break fi ;; msvc7 | msvc7msys | msvisualcpp | msvcmsys) # This compiler won't grok '-c -o', but also, the minuso test has # not run yet. These depmodes are late enough in the game, and # so weak that their functioning should not be impacted. am__obj=conftest.${OBJEXT-o} am__minus_obj= ;; none) break ;; esac if depmode=$depmode \ source=sub/conftest.c object=$am__obj \ depfile=sub/conftest.Po tmpdepfile=sub/conftest.TPo \ $SHELL ./depcomp $depcc -c $am__minus_obj sub/conftest.c \ >/dev/null 2>conftest.err && grep sub/conftst1.h sub/conftest.Po > /dev/null 2>&1 && grep sub/conftst6.h sub/conftest.Po > /dev/null 2>&1 && grep $am__obj sub/conftest.Po > /dev/null 2>&1 && ${MAKE-make} -s -f confmf > /dev/null 2>&1; then # icc doesn't choke on unknown options, it will just issue warnings # or remarks (even with -Werror). So we grep stderr for any message # that says an option was ignored or not supported. # When given -MP, icc 7.0 and 7.1 complain thusly: # icc: Command line warning: ignoring option '-M'; no argument required # The diagnosis changed in icc 8.0: # icc: Command line remark: option '-MP' not supported if (grep 'ignoring option' conftest.err || grep 'not supported' conftest.err) >/dev/null 2>&1; then :; else am_cv_CC_dependencies_compiler_type=$depmode break fi fi done cd .. rm -rf conftest.dir else am_cv_CC_dependencies_compiler_type=none fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $am_cv_CC_dependencies_compiler_type" >&5 $as_echo "$am_cv_CC_dependencies_compiler_type" >&6; } CCDEPMODE=depmode=$am_cv_CC_dependencies_compiler_type if test "x$enable_dependency_tracking" != xno \ && test "$am_cv_CC_dependencies_compiler_type" = gcc3; then am__fastdepCC_TRUE= am__fastdepCC_FALSE='#' else am__fastdepCC_TRUE='#' am__fastdepCC_FALSE= fi case $ac_cv_prog_cc_stdc in #( no) : ac_cv_prog_cc_c99=no; ac_cv_prog_cc_c89=no ;; #( *) : { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $CC option to accept ISO C99" >&5 $as_echo_n "checking for $CC option to accept ISO C99... " >&6; } if ${ac_cv_prog_cc_c99+:} false; then : $as_echo_n "(cached) " >&6 else ac_cv_prog_cc_c99=no ac_save_CC=$CC cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #include #include #include // Check varargs macros. These examples are taken from C99 6.10.3.5. #define debug(...) fprintf (stderr, __VA_ARGS__) #define showlist(...) puts (#__VA_ARGS__) #define report(test,...) ((test) ? puts (#test) : printf (__VA_ARGS__)) static void test_varargs_macros (void) { int x = 1234; int y = 5678; debug ("Flag"); debug ("X = %d\n", x); showlist (The first, second, and third items.); report (x>y, "x is %d but y is %d", x, y); } // Check long long types. #define BIG64 18446744073709551615ull #define BIG32 4294967295ul #define BIG_OK (BIG64 / BIG32 == 4294967297ull && BIG64 % BIG32 == 0) #if !BIG_OK your preprocessor is broken; #endif #if BIG_OK #else your preprocessor is broken; #endif static long long int bignum = -9223372036854775807LL; static unsigned long long int ubignum = BIG64; struct incomplete_array { int datasize; double data[]; }; struct named_init { int number; const wchar_t *name; double average; }; typedef const char *ccp; static inline int test_restrict (ccp restrict text) { // See if C++-style comments work. // Iterate through items via the restricted pointer. // Also check for declarations in for loops. for (unsigned int i = 0; *(text+i) != '\0'; ++i) continue; return 0; } // Check varargs and va_copy. static void test_varargs (const char *format, ...) { va_list args; va_start (args, format); va_list args_copy; va_copy (args_copy, args); const char *str; int number; float fnumber; while (*format) { switch (*format++) { case 's': // string str = va_arg (args_copy, const char *); break; case 'd': // int number = va_arg (args_copy, int); break; case 'f': // float fnumber = va_arg (args_copy, double); break; default: break; } } va_end (args_copy); va_end (args); } int main () { // Check bool. _Bool success = false; // Check restrict. if (test_restrict ("String literal") == 0) success = true; char *restrict newvar = "Another string"; // Check varargs. test_varargs ("s, d' f .", "string", 65, 34.234); test_varargs_macros (); // Check flexible array members. struct incomplete_array *ia = malloc (sizeof (struct incomplete_array) + (sizeof (double) * 10)); ia->datasize = 10; for (int i = 0; i < ia->datasize; ++i) ia->data[i] = i * 1.234; // Check named initializers. struct named_init ni = { .number = 34, .name = L"Test wide string", .average = 543.34343, }; ni.number = 58; int dynamic_array[ni.number]; dynamic_array[ni.number - 1] = 543; // work around unused variable warnings return (!success || bignum == 0LL || ubignum == 0uLL || newvar[0] == 'x' || dynamic_array[ni.number - 1] != 543); ; return 0; } _ACEOF for ac_arg in '' -std=gnu99 -std=c99 -c99 -AC99 -D_STDC_C99= -qlanglvl=extc99 do CC="$ac_save_CC $ac_arg" if ac_fn_c_try_compile "$LINENO"; then : ac_cv_prog_cc_c99=$ac_arg fi rm -f core conftest.err conftest.$ac_objext test "x$ac_cv_prog_cc_c99" != "xno" && break done rm -f conftest.$ac_ext CC=$ac_save_CC fi # AC_CACHE_VAL case "x$ac_cv_prog_cc_c99" in x) { $as_echo "$as_me:${as_lineno-$LINENO}: result: none needed" >&5 $as_echo "none needed" >&6; } ;; xno) { $as_echo "$as_me:${as_lineno-$LINENO}: result: unsupported" >&5 $as_echo "unsupported" >&6; } ;; *) CC="$CC $ac_cv_prog_cc_c99" { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_c99" >&5 $as_echo "$ac_cv_prog_cc_c99" >&6; } ;; esac if test "x$ac_cv_prog_cc_c99" != xno; then : ac_cv_prog_cc_stdc=$ac_cv_prog_cc_c99 else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $CC option to accept ISO C89" >&5 $as_echo_n "checking for $CC option to accept ISO C89... " >&6; } if ${ac_cv_prog_cc_c89+:} false; then : $as_echo_n "(cached) " >&6 else ac_cv_prog_cc_c89=no ac_save_CC=$CC cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include struct stat; /* Most of the following tests are stolen from RCS 5.7's src/conf.sh. */ struct buf { int x; }; FILE * (*rcsopen) (struct buf *, struct stat *, int); static char *e (p, i) char **p; int i; { return p[i]; } static char *f (char * (*g) (char **, int), char **p, ...) { char *s; va_list v; va_start (v,p); s = g (p, va_arg (v,int)); va_end (v); return s; } /* OSF 4.0 Compaq cc is some sort of almost-ANSI by default. It has function prototypes and stuff, but not '\xHH' hex character constants. These don't provoke an error unfortunately, instead are silently treated as 'x'. The following induces an error, until -std is added to get proper ANSI mode. Curiously '\x00'!='x' always comes out true, for an array size at least. It's necessary to write '\x00'==0 to get something that's true only with -std. */ int osf4_cc_array ['\x00' == 0 ? 1 : -1]; /* IBM C 6 for AIX is almost-ANSI by default, but it replaces macro parameters inside strings and character constants. */ #define FOO(x) 'x' int xlc6_cc_array[FOO(a) == 'x' ? 1 : -1]; int test (int i, double x); struct s1 {int (*f) (int a);}; struct s2 {int (*f) (double a);}; int pairnames (int, char **, FILE *(*)(struct buf *, struct stat *, int), int, int); int argc; char **argv; int main () { return f (e, argv, 0) != argv[0] || f (e, argv, 1) != argv[1]; ; return 0; } _ACEOF for ac_arg in '' -qlanglvl=extc89 -qlanglvl=ansi -std \ -Ae "-Aa -D_HPUX_SOURCE" "-Xc -D__EXTENSIONS__" do CC="$ac_save_CC $ac_arg" if ac_fn_c_try_compile "$LINENO"; then : ac_cv_prog_cc_c89=$ac_arg fi rm -f core conftest.err conftest.$ac_objext test "x$ac_cv_prog_cc_c89" != "xno" && break done rm -f conftest.$ac_ext CC=$ac_save_CC fi # AC_CACHE_VAL case "x$ac_cv_prog_cc_c89" in x) { $as_echo "$as_me:${as_lineno-$LINENO}: result: none needed" >&5 $as_echo "none needed" >&6; } ;; xno) { $as_echo "$as_me:${as_lineno-$LINENO}: result: unsupported" >&5 $as_echo "unsupported" >&6; } ;; *) CC="$CC $ac_cv_prog_cc_c89" { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_c89" >&5 $as_echo "$ac_cv_prog_cc_c89" >&6; } ;; esac if test "x$ac_cv_prog_cc_c89" != xno; then : ac_cv_prog_cc_stdc=$ac_cv_prog_cc_c89 else ac_cv_prog_cc_stdc=no fi fi ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $CC option to accept ISO Standard C" >&5 $as_echo_n "checking for $CC option to accept ISO Standard C... " >&6; } if ${ac_cv_prog_cc_stdc+:} false; then : $as_echo_n "(cached) " >&6 fi case $ac_cv_prog_cc_stdc in #( no) : { $as_echo "$as_me:${as_lineno-$LINENO}: result: unsupported" >&5 $as_echo "unsupported" >&6; } ;; #( '') : { $as_echo "$as_me:${as_lineno-$LINENO}: result: none needed" >&5 $as_echo "none needed" >&6; } ;; #( *) : { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_stdc" >&5 $as_echo "$ac_cv_prog_cc_stdc" >&6; } ;; esac ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to run the C preprocessor" >&5 $as_echo_n "checking how to run the C preprocessor... " >&6; } # On Suns, sometimes $CPP names a directory. if test -n "$CPP" && test -d "$CPP"; then CPP= fi if test -z "$CPP"; then if ${ac_cv_prog_CPP+:} false; then : $as_echo_n "(cached) " >&6 else # Double quotes because CPP needs to be expanded for CPP in "$CC -E" "$CC -E -traditional-cpp" "/lib/cpp" do ac_preproc_ok=false for ac_c_preproc_warn_flag in '' yes do # Use a header file that comes with gcc, so configuring glibc # with a fresh cross-compiler works. # Prefer to if __STDC__ is defined, since # exists even on freestanding compilers. # On the NeXT, cc -E runs the code through the compiler's parser, # not just through cpp. "Syntax error" is here to catch this case. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifdef __STDC__ # include #else # include #endif Syntax error _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : else # Broken: fails on valid input. continue fi rm -f conftest.err conftest.i conftest.$ac_ext # OK, works on sane cases. Now check whether nonexistent headers # can be detected and how. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : # Broken: success on invalid input. continue else # Passes both tests. ac_preproc_ok=: break fi rm -f conftest.err conftest.i conftest.$ac_ext done # Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. rm -f conftest.i conftest.err conftest.$ac_ext if $ac_preproc_ok; then : break fi done ac_cv_prog_CPP=$CPP fi CPP=$ac_cv_prog_CPP else ac_cv_prog_CPP=$CPP fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CPP" >&5 $as_echo "$CPP" >&6; } ac_preproc_ok=false for ac_c_preproc_warn_flag in '' yes do # Use a header file that comes with gcc, so configuring glibc # with a fresh cross-compiler works. # Prefer to if __STDC__ is defined, since # exists even on freestanding compilers. # On the NeXT, cc -E runs the code through the compiler's parser, # not just through cpp. "Syntax error" is here to catch this case. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifdef __STDC__ # include #else # include #endif Syntax error _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : else # Broken: fails on valid input. continue fi rm -f conftest.err conftest.i conftest.$ac_ext # OK, works on sane cases. Now check whether nonexistent headers # can be detected and how. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if ac_fn_c_try_cpp "$LINENO"; then : # Broken: success on invalid input. continue else # Passes both tests. ac_preproc_ok=: break fi rm -f conftest.err conftest.i conftest.$ac_ext done # Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. rm -f conftest.i conftest.err conftest.$ac_ext if $ac_preproc_ok; then : else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "C preprocessor \"$CPP\" fails sanity check See \`config.log' for more details" "$LINENO" 5; } fi ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu { $as_echo "$as_me:${as_lineno-$LINENO}: checking for grep that handles long lines and -e" >&5 $as_echo_n "checking for grep that handles long lines and -e... " >&6; } if ${ac_cv_path_GREP+:} false; then : $as_echo_n "(cached) " >&6 else if test -z "$GREP"; then ac_path_GREP_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in grep ggrep; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_GREP="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_GREP" || continue # Check for GNU ac_path_GREP and select it if it is found. # Check for GNU $ac_path_GREP case `"$ac_path_GREP" --version 2>&1` in *GNU*) ac_cv_path_GREP="$ac_path_GREP" ac_path_GREP_found=:;; *) ac_count=0 $as_echo_n 0123456789 >"conftest.in" while : do cat "conftest.in" "conftest.in" >"conftest.tmp" mv "conftest.tmp" "conftest.in" cp "conftest.in" "conftest.nl" $as_echo 'GREP' >> "conftest.nl" "$ac_path_GREP" -e 'GREP$' -e '-(cannot match)-' < "conftest.nl" >"conftest.out" 2>/dev/null || break diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break as_fn_arith $ac_count + 1 && ac_count=$as_val if test $ac_count -gt ${ac_path_GREP_max-0}; then # Best one so far, save it but keep looking for a better one ac_cv_path_GREP="$ac_path_GREP" ac_path_GREP_max=$ac_count fi # 10*(2^10) chars as input seems more than enough test $ac_count -gt 10 && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out;; esac $ac_path_GREP_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_GREP"; then as_fn_error $? "no acceptable grep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 fi else ac_cv_path_GREP=$GREP fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_GREP" >&5 $as_echo "$ac_cv_path_GREP" >&6; } GREP="$ac_cv_path_GREP" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for egrep" >&5 $as_echo_n "checking for egrep... " >&6; } if ${ac_cv_path_EGREP+:} false; then : $as_echo_n "(cached) " >&6 else if echo a | $GREP -E '(a|b)' >/dev/null 2>&1 then ac_cv_path_EGREP="$GREP -E" else if test -z "$EGREP"; then ac_path_EGREP_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in egrep; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_EGREP="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_EGREP" || continue # Check for GNU ac_path_EGREP and select it if it is found. # Check for GNU $ac_path_EGREP case `"$ac_path_EGREP" --version 2>&1` in *GNU*) ac_cv_path_EGREP="$ac_path_EGREP" ac_path_EGREP_found=:;; *) ac_count=0 $as_echo_n 0123456789 >"conftest.in" while : do cat "conftest.in" "conftest.in" >"conftest.tmp" mv "conftest.tmp" "conftest.in" cp "conftest.in" "conftest.nl" $as_echo 'EGREP' >> "conftest.nl" "$ac_path_EGREP" 'EGREP$' < "conftest.nl" >"conftest.out" 2>/dev/null || break diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break as_fn_arith $ac_count + 1 && ac_count=$as_val if test $ac_count -gt ${ac_path_EGREP_max-0}; then # Best one so far, save it but keep looking for a better one ac_cv_path_EGREP="$ac_path_EGREP" ac_path_EGREP_max=$ac_count fi # 10*(2^10) chars as input seems more than enough test $ac_count -gt 10 && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out;; esac $ac_path_EGREP_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_EGREP"; then as_fn_error $? "no acceptable egrep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 fi else ac_cv_path_EGREP=$EGREP fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_EGREP" >&5 $as_echo "$ac_cv_path_EGREP" >&6; } EGREP="$ac_cv_path_EGREP" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ANSI C header files" >&5 $as_echo_n "checking for ANSI C header files... " >&6; } if ${ac_cv_header_stdc+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #include #include int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_header_stdc=yes else ac_cv_header_stdc=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext if test $ac_cv_header_stdc = yes; then # SunOS 4.x string.h does not declare mem*, contrary to ANSI. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | $EGREP "memchr" >/dev/null 2>&1; then : else ac_cv_header_stdc=no fi rm -f conftest* fi if test $ac_cv_header_stdc = yes; then # ISC 2.0.2 stdlib.h does not declare free, contrary to ANSI. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | $EGREP "free" >/dev/null 2>&1; then : else ac_cv_header_stdc=no fi rm -f conftest* fi if test $ac_cv_header_stdc = yes; then # /bin/cc in Irix-4.0.5 gets non-ANSI ctype macros unless using -ansi. if test "$cross_compiling" = yes; then : : else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #if ((' ' & 0x0FF) == 0x020) # define ISLOWER(c) ('a' <= (c) && (c) <= 'z') # define TOUPPER(c) (ISLOWER(c) ? 'A' + ((c) - 'a') : (c)) #else # define ISLOWER(c) \ (('a' <= (c) && (c) <= 'i') \ || ('j' <= (c) && (c) <= 'r') \ || ('s' <= (c) && (c) <= 'z')) # define TOUPPER(c) (ISLOWER(c) ? ((c) | 0x40) : (c)) #endif #define XOR(e, f) (((e) && !(f)) || (!(e) && (f))) int main () { int i; for (i = 0; i < 256; i++) if (XOR (islower (i), ISLOWER (i)) || toupper (i) != TOUPPER (i)) return 2; return 0; } _ACEOF if ac_fn_c_try_run "$LINENO"; then : else ac_cv_header_stdc=no fi rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ conftest.$ac_objext conftest.beam conftest.$ac_ext fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_header_stdc" >&5 $as_echo "$ac_cv_header_stdc" >&6; } if test $ac_cv_header_stdc = yes; then $as_echo "#define STDC_HEADERS 1" >>confdefs.h fi # On IRIX 5.3, sys/types and inttypes.h are conflicting. for ac_header in sys/types.h sys/stat.h stdlib.h string.h memory.h strings.h \ inttypes.h stdint.h unistd.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_compile "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default " if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 _ACEOF fi done ac_fn_c_check_header_mongrel "$LINENO" "minix/config.h" "ac_cv_header_minix_config_h" "$ac_includes_default" if test "x$ac_cv_header_minix_config_h" = xyes; then : MINIX=yes else MINIX= fi if test "$MINIX" = yes; then $as_echo "#define _POSIX_SOURCE 1" >>confdefs.h $as_echo "#define _POSIX_1_SOURCE 2" >>confdefs.h $as_echo "#define _MINIX 1" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether it is safe to define __EXTENSIONS__" >&5 $as_echo_n "checking whether it is safe to define __EXTENSIONS__... " >&6; } if ${ac_cv_safe_to_define___extensions__+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ # define __EXTENSIONS__ 1 $ac_includes_default int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_safe_to_define___extensions__=yes else ac_cv_safe_to_define___extensions__=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_safe_to_define___extensions__" >&5 $as_echo "$ac_cv_safe_to_define___extensions__" >&6; } test $ac_cv_safe_to_define___extensions__ = yes && $as_echo "#define __EXTENSIONS__ 1" >>confdefs.h $as_echo "#define _ALL_SOURCE 1" >>confdefs.h $as_echo "#define _GNU_SOURCE 1" >>confdefs.h $as_echo "#define _POSIX_PTHREAD_SEMANTICS 1" >>confdefs.h $as_echo "#define _TANDEM_SOURCE 1" >>confdefs.h { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether byte ordering is bigendian" >&5 $as_echo_n "checking whether byte ordering is bigendian... " >&6; } if ${ac_cv_c_bigendian+:} false; then : $as_echo_n "(cached) " >&6 else ac_cv_c_bigendian=unknown # See if we're dealing with a universal compiler. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #ifndef __APPLE_CC__ not a universal capable compiler #endif typedef int dummy; _ACEOF if ac_fn_c_try_compile "$LINENO"; then : # Check for potential -arch flags. It is not universal unless # there are at least two -arch flags with different values. ac_arch= ac_prev= for ac_word in $CC $CFLAGS $CPPFLAGS $LDFLAGS; do if test -n "$ac_prev"; then case $ac_word in i?86 | x86_64 | ppc | ppc64) if test -z "$ac_arch" || test "$ac_arch" = "$ac_word"; then ac_arch=$ac_word else ac_cv_c_bigendian=universal break fi ;; esac ac_prev= elif test "x$ac_word" = "x-arch"; then ac_prev=arch fi done fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext if test $ac_cv_c_bigendian = unknown; then # See if sys/param.h defines the BYTE_ORDER macro. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include int main () { #if ! (defined BYTE_ORDER && defined BIG_ENDIAN \ && defined LITTLE_ENDIAN && BYTE_ORDER && BIG_ENDIAN \ && LITTLE_ENDIAN) bogus endian macros #endif ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : # It does; now see whether it defined to BIG_ENDIAN or not. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include int main () { #if BYTE_ORDER != BIG_ENDIAN not big endian #endif ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_c_bigendian=yes else ac_cv_c_bigendian=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi if test $ac_cv_c_bigendian = unknown; then # See if defines _LITTLE_ENDIAN or _BIG_ENDIAN (e.g., Solaris). cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { #if ! (defined _LITTLE_ENDIAN || defined _BIG_ENDIAN) bogus endian macros #endif ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : # It does; now see whether it defined to _BIG_ENDIAN or not. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { #ifndef _BIG_ENDIAN not big endian #endif ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_c_bigendian=yes else ac_cv_c_bigendian=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi if test $ac_cv_c_bigendian = unknown; then # Compile a test program. if test "$cross_compiling" = yes; then : # Try to guess by grepping values from an object file. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ short int ascii_mm[] = { 0x4249, 0x4765, 0x6E44, 0x6961, 0x6E53, 0x7953, 0 }; short int ascii_ii[] = { 0x694C, 0x5454, 0x656C, 0x6E45, 0x6944, 0x6E61, 0 }; int use_ascii (int i) { return ascii_mm[i] + ascii_ii[i]; } short int ebcdic_ii[] = { 0x89D3, 0xE3E3, 0x8593, 0x95C5, 0x89C4, 0x9581, 0 }; short int ebcdic_mm[] = { 0xC2C9, 0xC785, 0x95C4, 0x8981, 0x95E2, 0xA8E2, 0 }; int use_ebcdic (int i) { return ebcdic_mm[i] + ebcdic_ii[i]; } extern int foo; int main () { return use_ascii (foo) == use_ebcdic (foo); ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : if grep BIGenDianSyS conftest.$ac_objext >/dev/null; then ac_cv_c_bigendian=yes fi if grep LiTTleEnDian conftest.$ac_objext >/dev/null ; then if test "$ac_cv_c_bigendian" = unknown; then ac_cv_c_bigendian=no else # finding both strings is unlikely to happen, but who knows? ac_cv_c_bigendian=unknown fi fi fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default int main () { /* Are we little or big endian? From Harbison&Steele. */ union { long int l; char c[sizeof (long int)]; } u; u.l = 1; return u.c[sizeof (long int) - 1] == 1; ; return 0; } _ACEOF if ac_fn_c_try_run "$LINENO"; then : ac_cv_c_bigendian=no else ac_cv_c_bigendian=yes fi rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ conftest.$ac_objext conftest.beam conftest.$ac_ext fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_c_bigendian" >&5 $as_echo "$ac_cv_c_bigendian" >&6; } case $ac_cv_c_bigendian in #( yes) $as_echo "#define WORDS_BIGENDIAN 1" >>confdefs.h ;; #( no) ;; #( universal) $as_echo "#define AC_APPLE_UNIVERSAL_BUILD 1" >>confdefs.h ;; #( *) as_fn_error $? "unknown endianness presetting ac_cv_c_bigendian=no (or yes) will help" "$LINENO" 5 ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ln -s works" >&5 $as_echo_n "checking whether ln -s works... " >&6; } LN_S=$as_ln_s if test "$LN_S" = "ln -s"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no, using $LN_S" >&5 $as_echo "no, using $LN_S" >&6; } fi case `pwd` in *\ * | *\ *) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&5 $as_echo "$as_me: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&2;} ;; esac macro_version='2.4.2' macro_revision='1.3337' ltmain="$ac_aux_dir/ltmain.sh" # Backslashify metacharacters that are still active within # double-quoted strings. sed_quote_subst='s/\(["`$\\]\)/\\\1/g' # Same as above, but do not quote variable references. double_quote_subst='s/\(["`\\]\)/\\\1/g' # Sed substitution to delay expansion of an escaped shell variable in a # double_quote_subst'ed string. delay_variable_subst='s/\\\\\\\\\\\$/\\\\\\$/g' # Sed substitution to delay expansion of an escaped single quote. delay_single_quote_subst='s/'\''/'\'\\\\\\\'\''/g' # Sed substitution to avoid accidental globbing in evaled expressions no_glob_subst='s/\*/\\\*/g' ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO$ECHO { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to print strings" >&5 $as_echo_n "checking how to print strings... " >&6; } # Test print first, because it will be a builtin if present. if test "X`( print -r -- -n ) 2>/dev/null`" = X-n && \ test "X`print -r -- $ECHO 2>/dev/null`" = "X$ECHO"; then ECHO='print -r --' elif test "X`printf %s $ECHO 2>/dev/null`" = "X$ECHO"; then ECHO='printf %s\n' else # Use this function as a fallback that always works. func_fallback_echo () { eval 'cat <<_LTECHO_EOF $1 _LTECHO_EOF' } ECHO='func_fallback_echo' fi # func_echo_all arg... # Invoke $ECHO with all args, space-separated. func_echo_all () { $ECHO "" } case "$ECHO" in printf*) { $as_echo "$as_me:${as_lineno-$LINENO}: result: printf" >&5 $as_echo "printf" >&6; } ;; print*) { $as_echo "$as_me:${as_lineno-$LINENO}: result: print -r" >&5 $as_echo "print -r" >&6; } ;; *) { $as_echo "$as_me:${as_lineno-$LINENO}: result: cat" >&5 $as_echo "cat" >&6; } ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking for a sed that does not truncate output" >&5 $as_echo_n "checking for a sed that does not truncate output... " >&6; } if ${ac_cv_path_SED+:} false; then : $as_echo_n "(cached) " >&6 else ac_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/ for ac_i in 1 2 3 4 5 6 7; do ac_script="$ac_script$as_nl$ac_script" done echo "$ac_script" 2>/dev/null | sed 99q >conftest.sed { ac_script=; unset ac_script;} if test -z "$SED"; then ac_path_SED_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in sed gsed; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_SED="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_SED" || continue # Check for GNU ac_path_SED and select it if it is found. # Check for GNU $ac_path_SED case `"$ac_path_SED" --version 2>&1` in *GNU*) ac_cv_path_SED="$ac_path_SED" ac_path_SED_found=:;; *) ac_count=0 $as_echo_n 0123456789 >"conftest.in" while : do cat "conftest.in" "conftest.in" >"conftest.tmp" mv "conftest.tmp" "conftest.in" cp "conftest.in" "conftest.nl" $as_echo '' >> "conftest.nl" "$ac_path_SED" -f conftest.sed < "conftest.nl" >"conftest.out" 2>/dev/null || break diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break as_fn_arith $ac_count + 1 && ac_count=$as_val if test $ac_count -gt ${ac_path_SED_max-0}; then # Best one so far, save it but keep looking for a better one ac_cv_path_SED="$ac_path_SED" ac_path_SED_max=$ac_count fi # 10*(2^10) chars as input seems more than enough test $ac_count -gt 10 && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out;; esac $ac_path_SED_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_SED"; then as_fn_error $? "no acceptable sed could be found in \$PATH" "$LINENO" 5 fi else ac_cv_path_SED=$SED fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_SED" >&5 $as_echo "$ac_cv_path_SED" >&6; } SED="$ac_cv_path_SED" rm -f conftest.sed test -z "$SED" && SED=sed Xsed="$SED -e 1s/^X//" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for fgrep" >&5 $as_echo_n "checking for fgrep... " >&6; } if ${ac_cv_path_FGREP+:} false; then : $as_echo_n "(cached) " >&6 else if echo 'ab*c' | $GREP -F 'ab*c' >/dev/null 2>&1 then ac_cv_path_FGREP="$GREP -F" else if test -z "$FGREP"; then ac_path_FGREP_found=false # Loop through the user's path and test for each of PROGNAME-LIST as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_prog in fgrep; do for ac_exec_ext in '' $ac_executable_extensions; do ac_path_FGREP="$as_dir/$ac_prog$ac_exec_ext" as_fn_executable_p "$ac_path_FGREP" || continue # Check for GNU ac_path_FGREP and select it if it is found. # Check for GNU $ac_path_FGREP case `"$ac_path_FGREP" --version 2>&1` in *GNU*) ac_cv_path_FGREP="$ac_path_FGREP" ac_path_FGREP_found=:;; *) ac_count=0 $as_echo_n 0123456789 >"conftest.in" while : do cat "conftest.in" "conftest.in" >"conftest.tmp" mv "conftest.tmp" "conftest.in" cp "conftest.in" "conftest.nl" $as_echo 'FGREP' >> "conftest.nl" "$ac_path_FGREP" FGREP < "conftest.nl" >"conftest.out" 2>/dev/null || break diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break as_fn_arith $ac_count + 1 && ac_count=$as_val if test $ac_count -gt ${ac_path_FGREP_max-0}; then # Best one so far, save it but keep looking for a better one ac_cv_path_FGREP="$ac_path_FGREP" ac_path_FGREP_max=$ac_count fi # 10*(2^10) chars as input seems more than enough test $ac_count -gt 10 && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out;; esac $ac_path_FGREP_found && break 3 done done done IFS=$as_save_IFS if test -z "$ac_cv_path_FGREP"; then as_fn_error $? "no acceptable fgrep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 fi else ac_cv_path_FGREP=$FGREP fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_FGREP" >&5 $as_echo "$ac_cv_path_FGREP" >&6; } FGREP="$ac_cv_path_FGREP" test -z "$GREP" && GREP=grep # Check whether --with-gnu-ld was given. if test "${with_gnu_ld+set}" = set; then : withval=$with_gnu_ld; test "$withval" = no || with_gnu_ld=yes else with_gnu_ld=no fi ac_prog=ld if test "$GCC" = yes; then # Check if gcc -print-prog-name=ld gives a path. { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ld used by $CC" >&5 $as_echo_n "checking for ld used by $CC... " >&6; } case $host in *-*-mingw*) # gcc leaves a trailing carriage return which upsets mingw ac_prog=`($CC -print-prog-name=ld) 2>&5 | tr -d '\015'` ;; *) ac_prog=`($CC -print-prog-name=ld) 2>&5` ;; esac case $ac_prog in # Accept absolute paths. [\\/]* | ?:[\\/]*) re_direlt='/[^/][^/]*/\.\./' # Canonicalize the pathname of ld ac_prog=`$ECHO "$ac_prog"| $SED 's%\\\\%/%g'` while $ECHO "$ac_prog" | $GREP "$re_direlt" > /dev/null 2>&1; do ac_prog=`$ECHO $ac_prog| $SED "s%$re_direlt%/%"` done test -z "$LD" && LD="$ac_prog" ;; "") # If it fails, then pretend we aren't using GCC. ac_prog=ld ;; *) # If it is relative, then search for the first ld in PATH. with_gnu_ld=unknown ;; esac elif test "$with_gnu_ld" = yes; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for GNU ld" >&5 $as_echo_n "checking for GNU ld... " >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for non-GNU ld" >&5 $as_echo_n "checking for non-GNU ld... " >&6; } fi if ${lt_cv_path_LD+:} false; then : $as_echo_n "(cached) " >&6 else if test -z "$LD"; then lt_save_ifs="$IFS"; IFS=$PATH_SEPARATOR for ac_dir in $PATH; do IFS="$lt_save_ifs" test -z "$ac_dir" && ac_dir=. if test -f "$ac_dir/$ac_prog" || test -f "$ac_dir/$ac_prog$ac_exeext"; then lt_cv_path_LD="$ac_dir/$ac_prog" # Check to see if the program is GNU ld. I'd rather use --version, # but apparently some variants of GNU ld only accept -v. # Break only if it was the GNU/non-GNU ld that we prefer. case `"$lt_cv_path_LD" -v 2>&1 &5 $as_echo "$LD" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -z "$LD" && as_fn_error $? "no acceptable ld found in \$PATH" "$LINENO" 5 { $as_echo "$as_me:${as_lineno-$LINENO}: checking if the linker ($LD) is GNU ld" >&5 $as_echo_n "checking if the linker ($LD) is GNU ld... " >&6; } if ${lt_cv_prog_gnu_ld+:} false; then : $as_echo_n "(cached) " >&6 else # I'd rather use --version here, but apparently some GNU lds only accept -v. case `$LD -v 2>&1 &5 $as_echo "$lt_cv_prog_gnu_ld" >&6; } with_gnu_ld=$lt_cv_prog_gnu_ld { $as_echo "$as_me:${as_lineno-$LINENO}: checking for BSD- or MS-compatible name lister (nm)" >&5 $as_echo_n "checking for BSD- or MS-compatible name lister (nm)... " >&6; } if ${lt_cv_path_NM+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$NM"; then # Let the user override the test. lt_cv_path_NM="$NM" else lt_nm_to_check="${ac_tool_prefix}nm" if test -n "$ac_tool_prefix" && test "$build" = "$host"; then lt_nm_to_check="$lt_nm_to_check nm" fi for lt_tmp_nm in $lt_nm_to_check; do lt_save_ifs="$IFS"; IFS=$PATH_SEPARATOR for ac_dir in $PATH /usr/ccs/bin/elf /usr/ccs/bin /usr/ucb /bin; do IFS="$lt_save_ifs" test -z "$ac_dir" && ac_dir=. tmp_nm="$ac_dir/$lt_tmp_nm" if test -f "$tmp_nm" || test -f "$tmp_nm$ac_exeext" ; then # Check to see if the nm accepts a BSD-compat flag. # Adding the `sed 1q' prevents false positives on HP-UX, which says: # nm: unknown option "B" ignored # Tru64's nm complains that /dev/null is an invalid object file case `"$tmp_nm" -B /dev/null 2>&1 | sed '1q'` in */dev/null* | *'Invalid file or object type'*) lt_cv_path_NM="$tmp_nm -B" break ;; *) case `"$tmp_nm" -p /dev/null 2>&1 | sed '1q'` in */dev/null*) lt_cv_path_NM="$tmp_nm -p" break ;; *) lt_cv_path_NM=${lt_cv_path_NM="$tmp_nm"} # keep the first match, but continue # so that we can try to find one that supports BSD flags ;; esac ;; esac fi done IFS="$lt_save_ifs" done : ${lt_cv_path_NM=no} fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_path_NM" >&5 $as_echo "$lt_cv_path_NM" >&6; } if test "$lt_cv_path_NM" != "no"; then NM="$lt_cv_path_NM" else # Didn't find any BSD compatible name lister, look for dumpbin. if test -n "$DUMPBIN"; then : # Let the user override the test. else if test -n "$ac_tool_prefix"; then for ac_prog in dumpbin "link -dump" do # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. set dummy $ac_tool_prefix$ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_DUMPBIN+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$DUMPBIN"; then ac_cv_prog_DUMPBIN="$DUMPBIN" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_DUMPBIN="$ac_tool_prefix$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi DUMPBIN=$ac_cv_prog_DUMPBIN if test -n "$DUMPBIN"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DUMPBIN" >&5 $as_echo "$DUMPBIN" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$DUMPBIN" && break done fi if test -z "$DUMPBIN"; then ac_ct_DUMPBIN=$DUMPBIN for ac_prog in dumpbin "link -dump" do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_DUMPBIN+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_DUMPBIN"; then ac_cv_prog_ac_ct_DUMPBIN="$ac_ct_DUMPBIN" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_DUMPBIN="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_DUMPBIN=$ac_cv_prog_ac_ct_DUMPBIN if test -n "$ac_ct_DUMPBIN"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DUMPBIN" >&5 $as_echo "$ac_ct_DUMPBIN" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$ac_ct_DUMPBIN" && break done if test "x$ac_ct_DUMPBIN" = x; then DUMPBIN=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac DUMPBIN=$ac_ct_DUMPBIN fi fi case `$DUMPBIN -symbols /dev/null 2>&1 | sed '1q'` in *COFF*) DUMPBIN="$DUMPBIN -symbols" ;; *) DUMPBIN=: ;; esac fi if test "$DUMPBIN" != ":"; then NM="$DUMPBIN" fi fi test -z "$NM" && NM=nm { $as_echo "$as_me:${as_lineno-$LINENO}: checking the name lister ($NM) interface" >&5 $as_echo_n "checking the name lister ($NM) interface... " >&6; } if ${lt_cv_nm_interface+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_nm_interface="BSD nm" echo "int some_variable = 0;" > conftest.$ac_ext (eval echo "\"\$as_me:$LINENO: $ac_compile\"" >&5) (eval "$ac_compile" 2>conftest.err) cat conftest.err >&5 (eval echo "\"\$as_me:$LINENO: $NM \\\"conftest.$ac_objext\\\"\"" >&5) (eval "$NM \"conftest.$ac_objext\"" 2>conftest.err > conftest.out) cat conftest.err >&5 (eval echo "\"\$as_me:$LINENO: output\"" >&5) cat conftest.out >&5 if $GREP 'External.*some_variable' conftest.out > /dev/null; then lt_cv_nm_interface="MS dumpbin" fi rm -f conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_nm_interface" >&5 $as_echo "$lt_cv_nm_interface" >&6; } # find the maximum length of command line arguments { $as_echo "$as_me:${as_lineno-$LINENO}: checking the maximum length of command line arguments" >&5 $as_echo_n "checking the maximum length of command line arguments... " >&6; } if ${lt_cv_sys_max_cmd_len+:} false; then : $as_echo_n "(cached) " >&6 else i=0 teststring="ABCD" case $build_os in msdosdjgpp*) # On DJGPP, this test can blow up pretty badly due to problems in libc # (any single argument exceeding 2000 bytes causes a buffer overrun # during glob expansion). Even if it were fixed, the result of this # check would be larger than it should be. lt_cv_sys_max_cmd_len=12288; # 12K is about right ;; gnu*) # Under GNU Hurd, this test is not required because there is # no limit to the length of command line arguments. # Libtool will interpret -1 as no limit whatsoever lt_cv_sys_max_cmd_len=-1; ;; cygwin* | mingw* | cegcc*) # On Win9x/ME, this test blows up -- it succeeds, but takes # about 5 minutes as the teststring grows exponentially. # Worse, since 9x/ME are not pre-emptively multitasking, # you end up with a "frozen" computer, even though with patience # the test eventually succeeds (with a max line length of 256k). # Instead, let's just punt: use the minimum linelength reported by # all of the supported platforms: 8192 (on NT/2K/XP). lt_cv_sys_max_cmd_len=8192; ;; mint*) # On MiNT this can take a long time and run out of memory. lt_cv_sys_max_cmd_len=8192; ;; amigaos*) # On AmigaOS with pdksh, this test takes hours, literally. # So we just punt and use a minimum line length of 8192. lt_cv_sys_max_cmd_len=8192; ;; netbsd* | freebsd* | openbsd* | darwin* | dragonfly*) # This has been around since 386BSD, at least. Likely further. if test -x /sbin/sysctl; then lt_cv_sys_max_cmd_len=`/sbin/sysctl -n kern.argmax` elif test -x /usr/sbin/sysctl; then lt_cv_sys_max_cmd_len=`/usr/sbin/sysctl -n kern.argmax` else lt_cv_sys_max_cmd_len=65536 # usable default for all BSDs fi # And add a safety zone lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` ;; interix*) # We know the value 262144 and hardcode it with a safety zone (like BSD) lt_cv_sys_max_cmd_len=196608 ;; os2*) # The test takes a long time on OS/2. lt_cv_sys_max_cmd_len=8192 ;; osf*) # Dr. Hans Ekkehard Plesser reports seeing a kernel panic running configure # due to this test when exec_disable_arg_limit is 1 on Tru64. It is not # nice to cause kernel panics so lets avoid the loop below. # First set a reasonable default. lt_cv_sys_max_cmd_len=16384 # if test -x /sbin/sysconfig; then case `/sbin/sysconfig -q proc exec_disable_arg_limit` in *1*) lt_cv_sys_max_cmd_len=-1 ;; esac fi ;; sco3.2v5*) lt_cv_sys_max_cmd_len=102400 ;; sysv5* | sco5v6* | sysv4.2uw2*) kargmax=`grep ARG_MAX /etc/conf/cf.d/stune 2>/dev/null` if test -n "$kargmax"; then lt_cv_sys_max_cmd_len=`echo $kargmax | sed 's/.*[ ]//'` else lt_cv_sys_max_cmd_len=32768 fi ;; *) lt_cv_sys_max_cmd_len=`(getconf ARG_MAX) 2> /dev/null` if test -n "$lt_cv_sys_max_cmd_len"; then lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` else # Make teststring a little bigger before we do anything with it. # a 1K string should be a reasonable start. for i in 1 2 3 4 5 6 7 8 ; do teststring=$teststring$teststring done SHELL=${SHELL-${CONFIG_SHELL-/bin/sh}} # If test is not a shell built-in, we'll probably end up computing a # maximum length that is only half of the actual maximum length, but # we can't tell. while { test "X"`env echo "$teststring$teststring" 2>/dev/null` \ = "X$teststring$teststring"; } >/dev/null 2>&1 && test $i != 17 # 1/2 MB should be enough do i=`expr $i + 1` teststring=$teststring$teststring done # Only check the string length outside the loop. lt_cv_sys_max_cmd_len=`expr "X$teststring" : ".*" 2>&1` teststring= # Add a significant safety factor because C++ compilers can tack on # massive amounts of additional arguments before passing them to the # linker. It appears as though 1/2 is a usable value. lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 2` fi ;; esac fi if test -n $lt_cv_sys_max_cmd_len ; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_sys_max_cmd_len" >&5 $as_echo "$lt_cv_sys_max_cmd_len" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: none" >&5 $as_echo "none" >&6; } fi max_cmd_len=$lt_cv_sys_max_cmd_len : ${CP="cp -f"} : ${MV="mv -f"} : ${RM="rm -f"} { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the shell understands some XSI constructs" >&5 $as_echo_n "checking whether the shell understands some XSI constructs... " >&6; } # Try some XSI features xsi_shell=no ( _lt_dummy="a/b/c" test "${_lt_dummy##*/},${_lt_dummy%/*},${_lt_dummy#??}"${_lt_dummy%"$_lt_dummy"}, \ = c,a/b,b/c, \ && eval 'test $(( 1 + 1 )) -eq 2 \ && test "${#_lt_dummy}" -eq 5' ) >/dev/null 2>&1 \ && xsi_shell=yes { $as_echo "$as_me:${as_lineno-$LINENO}: result: $xsi_shell" >&5 $as_echo "$xsi_shell" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the shell understands \"+=\"" >&5 $as_echo_n "checking whether the shell understands \"+=\"... " >&6; } lt_shell_append=no ( foo=bar; set foo baz; eval "$1+=\$2" && test "$foo" = barbaz ) \ >/dev/null 2>&1 \ && lt_shell_append=yes { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_shell_append" >&5 $as_echo "$lt_shell_append" >&6; } if ( (MAIL=60; unset MAIL) || exit) >/dev/null 2>&1; then lt_unset=unset else lt_unset=false fi # test EBCDIC or ASCII case `echo X|tr X '\101'` in A) # ASCII based system # \n is not interpreted correctly by Solaris 8 /usr/ucb/tr lt_SP2NL='tr \040 \012' lt_NL2SP='tr \015\012 \040\040' ;; *) # EBCDIC based system lt_SP2NL='tr \100 \n' lt_NL2SP='tr \r\n \100\100' ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to convert $build file names to $host format" >&5 $as_echo_n "checking how to convert $build file names to $host format... " >&6; } if ${lt_cv_to_host_file_cmd+:} false; then : $as_echo_n "(cached) " >&6 else case $host in *-*-mingw* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_host_file_cmd=func_convert_file_msys_to_w32 ;; *-*-cygwin* ) lt_cv_to_host_file_cmd=func_convert_file_cygwin_to_w32 ;; * ) # otherwise, assume *nix lt_cv_to_host_file_cmd=func_convert_file_nix_to_w32 ;; esac ;; *-*-cygwin* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_host_file_cmd=func_convert_file_msys_to_cygwin ;; *-*-cygwin* ) lt_cv_to_host_file_cmd=func_convert_file_noop ;; * ) # otherwise, assume *nix lt_cv_to_host_file_cmd=func_convert_file_nix_to_cygwin ;; esac ;; * ) # unhandled hosts (and "normal" native builds) lt_cv_to_host_file_cmd=func_convert_file_noop ;; esac fi to_host_file_cmd=$lt_cv_to_host_file_cmd { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_to_host_file_cmd" >&5 $as_echo "$lt_cv_to_host_file_cmd" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to convert $build file names to toolchain format" >&5 $as_echo_n "checking how to convert $build file names to toolchain format... " >&6; } if ${lt_cv_to_tool_file_cmd+:} false; then : $as_echo_n "(cached) " >&6 else #assume ordinary cross tools, or native build. lt_cv_to_tool_file_cmd=func_convert_file_noop case $host in *-*-mingw* ) case $build in *-*-mingw* ) # actually msys lt_cv_to_tool_file_cmd=func_convert_file_msys_to_w32 ;; esac ;; esac fi to_tool_file_cmd=$lt_cv_to_tool_file_cmd { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_to_tool_file_cmd" >&5 $as_echo "$lt_cv_to_tool_file_cmd" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $LD option to reload object files" >&5 $as_echo_n "checking for $LD option to reload object files... " >&6; } if ${lt_cv_ld_reload_flag+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_ld_reload_flag='-r' fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_reload_flag" >&5 $as_echo "$lt_cv_ld_reload_flag" >&6; } reload_flag=$lt_cv_ld_reload_flag case $reload_flag in "" | " "*) ;; *) reload_flag=" $reload_flag" ;; esac reload_cmds='$LD$reload_flag -o $output$reload_objs' case $host_os in cygwin* | mingw* | pw32* | cegcc*) if test "$GCC" != yes; then reload_cmds=false fi ;; darwin*) if test "$GCC" = yes; then reload_cmds='$LTCC $LTCFLAGS -nostdlib ${wl}-r -o $output$reload_objs' else reload_cmds='$LD$reload_flag -o $output$reload_objs' fi ;; esac if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}objdump", so it can be a program name with args. set dummy ${ac_tool_prefix}objdump; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_OBJDUMP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$OBJDUMP"; then ac_cv_prog_OBJDUMP="$OBJDUMP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_OBJDUMP="${ac_tool_prefix}objdump" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi OBJDUMP=$ac_cv_prog_OBJDUMP if test -n "$OBJDUMP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OBJDUMP" >&5 $as_echo "$OBJDUMP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_OBJDUMP"; then ac_ct_OBJDUMP=$OBJDUMP # Extract the first word of "objdump", so it can be a program name with args. set dummy objdump; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_OBJDUMP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_OBJDUMP"; then ac_cv_prog_ac_ct_OBJDUMP="$ac_ct_OBJDUMP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_OBJDUMP="objdump" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_OBJDUMP=$ac_cv_prog_ac_ct_OBJDUMP if test -n "$ac_ct_OBJDUMP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OBJDUMP" >&5 $as_echo "$ac_ct_OBJDUMP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_OBJDUMP" = x; then OBJDUMP="false" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac OBJDUMP=$ac_ct_OBJDUMP fi else OBJDUMP="$ac_cv_prog_OBJDUMP" fi test -z "$OBJDUMP" && OBJDUMP=objdump { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to recognize dependent libraries" >&5 $as_echo_n "checking how to recognize dependent libraries... " >&6; } if ${lt_cv_deplibs_check_method+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_file_magic_cmd='$MAGIC_CMD' lt_cv_file_magic_test_file= lt_cv_deplibs_check_method='unknown' # Need to set the preceding variable on all platforms that support # interlibrary dependencies. # 'none' -- dependencies not supported. # `unknown' -- same as none, but documents that we really don't know. # 'pass_all' -- all dependencies passed with no checks. # 'test_compile' -- check by making test program. # 'file_magic [[regex]]' -- check by looking for files in library path # which responds to the $file_magic_cmd with a given extended regex. # If you have `file' or equivalent on your system and you're not sure # whether `pass_all' will *always* work, you probably want this one. case $host_os in aix[4-9]*) lt_cv_deplibs_check_method=pass_all ;; beos*) lt_cv_deplibs_check_method=pass_all ;; bsdi[45]*) lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (shared object|dynamic lib)' lt_cv_file_magic_cmd='/usr/bin/file -L' lt_cv_file_magic_test_file=/shlib/libc.so ;; cygwin*) # func_win32_libid is a shell function defined in ltmain.sh lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' lt_cv_file_magic_cmd='func_win32_libid' ;; mingw* | pw32*) # Base MSYS/MinGW do not provide the 'file' command needed by # func_win32_libid shell function, so use a weaker test based on 'objdump', # unless we find 'file', for example because we are cross-compiling. # func_win32_libid assumes BSD nm, so disallow it if using MS dumpbin. if ( test "$lt_cv_nm_interface" = "BSD nm" && file / ) >/dev/null 2>&1; then lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' lt_cv_file_magic_cmd='func_win32_libid' else # Keep this pattern in sync with the one in func_win32_libid. lt_cv_deplibs_check_method='file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)' lt_cv_file_magic_cmd='$OBJDUMP -f' fi ;; cegcc*) # use the weaker test based on 'objdump'. See mingw*. lt_cv_deplibs_check_method='file_magic file format pe-arm-.*little(.*architecture: arm)?' lt_cv_file_magic_cmd='$OBJDUMP -f' ;; darwin* | rhapsody*) lt_cv_deplibs_check_method=pass_all ;; freebsd* | dragonfly*) if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then case $host_cpu in i*86 ) # Not sure whether the presence of OpenBSD here was a mistake. # Let's accept both of them until this is cleared up. lt_cv_deplibs_check_method='file_magic (FreeBSD|OpenBSD|DragonFly)/i[3-9]86 (compact )?demand paged shared library' lt_cv_file_magic_cmd=/usr/bin/file lt_cv_file_magic_test_file=`echo /usr/lib/libc.so.*` ;; esac else lt_cv_deplibs_check_method=pass_all fi ;; gnu*) lt_cv_deplibs_check_method=pass_all ;; haiku*) lt_cv_deplibs_check_method=pass_all ;; hpux10.20* | hpux11*) lt_cv_file_magic_cmd=/usr/bin/file case $host_cpu in ia64*) lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|ELF-[0-9][0-9]) shared object file - IA64' lt_cv_file_magic_test_file=/usr/lib/hpux32/libc.so ;; hppa*64*) lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|ELF[ -][0-9][0-9])(-bit)?( [LM]SB)? shared object( file)?[, -]* PA-RISC [0-9]\.[0-9]' lt_cv_file_magic_test_file=/usr/lib/pa20_64/libc.sl ;; *) lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|PA-RISC[0-9]\.[0-9]) shared library' lt_cv_file_magic_test_file=/usr/lib/libc.sl ;; esac ;; interix[3-9]*) # PIC code is broken on Interix 3.x, that's why |\.a not |_pic\.a here lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so|\.a)$' ;; irix5* | irix6* | nonstopux*) case $LD in *-32|*"-32 ") libmagic=32-bit;; *-n32|*"-n32 ") libmagic=N32;; *-64|*"-64 ") libmagic=64-bit;; *) libmagic=never-match;; esac lt_cv_deplibs_check_method=pass_all ;; # This must be glibc/ELF. linux* | k*bsd*-gnu | kopensolaris*-gnu) lt_cv_deplibs_check_method=pass_all ;; netbsd*) if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|_pic\.a)$' else lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so|_pic\.a)$' fi ;; newos6*) lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (executable|dynamic lib)' lt_cv_file_magic_cmd=/usr/bin/file lt_cv_file_magic_test_file=/usr/lib/libnls.so ;; *nto* | *qnx*) lt_cv_deplibs_check_method=pass_all ;; openbsd*) if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`" || test "$host_os-$host_cpu" = "openbsd2.8-powerpc"; then lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|\.so|_pic\.a)$' else lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|_pic\.a)$' fi ;; osf3* | osf4* | osf5*) lt_cv_deplibs_check_method=pass_all ;; rdos*) lt_cv_deplibs_check_method=pass_all ;; solaris*) lt_cv_deplibs_check_method=pass_all ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) lt_cv_deplibs_check_method=pass_all ;; sysv4 | sysv4.3*) case $host_vendor in motorola) lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (shared object|dynamic lib) M[0-9][0-9]* Version [0-9]' lt_cv_file_magic_test_file=`echo /usr/lib/libc.so*` ;; ncr) lt_cv_deplibs_check_method=pass_all ;; sequent) lt_cv_file_magic_cmd='/bin/file' lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [LM]SB (shared object|dynamic lib )' ;; sni) lt_cv_file_magic_cmd='/bin/file' lt_cv_deplibs_check_method="file_magic ELF [0-9][0-9]*-bit [LM]SB dynamic lib" lt_cv_file_magic_test_file=/lib/libc.so ;; siemens) lt_cv_deplibs_check_method=pass_all ;; pc) lt_cv_deplibs_check_method=pass_all ;; esac ;; tpf*) lt_cv_deplibs_check_method=pass_all ;; esac fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_deplibs_check_method" >&5 $as_echo "$lt_cv_deplibs_check_method" >&6; } file_magic_glob= want_nocaseglob=no if test "$build" = "$host"; then case $host_os in mingw* | pw32*) if ( shopt | grep nocaseglob ) >/dev/null 2>&1; then want_nocaseglob=yes else file_magic_glob=`echo aAbBcCdDeEfFgGhHiIjJkKlLmMnNoOpPqQrRsStTuUvVwWxXyYzZ | $SED -e "s/\(..\)/s\/[\1]\/[\1]\/g;/g"` fi ;; esac fi file_magic_cmd=$lt_cv_file_magic_cmd deplibs_check_method=$lt_cv_deplibs_check_method test -z "$deplibs_check_method" && deplibs_check_method=unknown if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}dlltool", so it can be a program name with args. set dummy ${ac_tool_prefix}dlltool; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_DLLTOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$DLLTOOL"; then ac_cv_prog_DLLTOOL="$DLLTOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_DLLTOOL="${ac_tool_prefix}dlltool" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi DLLTOOL=$ac_cv_prog_DLLTOOL if test -n "$DLLTOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DLLTOOL" >&5 $as_echo "$DLLTOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_DLLTOOL"; then ac_ct_DLLTOOL=$DLLTOOL # Extract the first word of "dlltool", so it can be a program name with args. set dummy dlltool; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_DLLTOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_DLLTOOL"; then ac_cv_prog_ac_ct_DLLTOOL="$ac_ct_DLLTOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_DLLTOOL="dlltool" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_DLLTOOL=$ac_cv_prog_ac_ct_DLLTOOL if test -n "$ac_ct_DLLTOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DLLTOOL" >&5 $as_echo "$ac_ct_DLLTOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_DLLTOOL" = x; then DLLTOOL="false" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac DLLTOOL=$ac_ct_DLLTOOL fi else DLLTOOL="$ac_cv_prog_DLLTOOL" fi test -z "$DLLTOOL" && DLLTOOL=dlltool { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to associate runtime and link libraries" >&5 $as_echo_n "checking how to associate runtime and link libraries... " >&6; } if ${lt_cv_sharedlib_from_linklib_cmd+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_sharedlib_from_linklib_cmd='unknown' case $host_os in cygwin* | mingw* | pw32* | cegcc*) # two different shell functions defined in ltmain.sh # decide which to use based on capabilities of $DLLTOOL case `$DLLTOOL --help 2>&1` in *--identify-strict*) lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib ;; *) lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib_fallback ;; esac ;; *) # fallback: assume linklib IS sharedlib lt_cv_sharedlib_from_linklib_cmd="$ECHO" ;; esac fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_sharedlib_from_linklib_cmd" >&5 $as_echo "$lt_cv_sharedlib_from_linklib_cmd" >&6; } sharedlib_from_linklib_cmd=$lt_cv_sharedlib_from_linklib_cmd test -z "$sharedlib_from_linklib_cmd" && sharedlib_from_linklib_cmd=$ECHO if test -n "$ac_tool_prefix"; then for ac_prog in ar do # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. set dummy $ac_tool_prefix$ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_AR+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$AR"; then ac_cv_prog_AR="$AR" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_AR="$ac_tool_prefix$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi AR=$ac_cv_prog_AR if test -n "$AR"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $AR" >&5 $as_echo "$AR" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$AR" && break done fi if test -z "$AR"; then ac_ct_AR=$AR for ac_prog in ar do # Extract the first word of "$ac_prog", so it can be a program name with args. set dummy $ac_prog; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_AR+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_AR"; then ac_cv_prog_ac_ct_AR="$ac_ct_AR" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_AR="$ac_prog" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_AR=$ac_cv_prog_ac_ct_AR if test -n "$ac_ct_AR"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_AR" >&5 $as_echo "$ac_ct_AR" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi test -n "$ac_ct_AR" && break done if test "x$ac_ct_AR" = x; then AR="false" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac AR=$ac_ct_AR fi fi : ${AR=ar} : ${AR_FLAGS=cru} { $as_echo "$as_me:${as_lineno-$LINENO}: checking for archiver @FILE support" >&5 $as_echo_n "checking for archiver @FILE support... " >&6; } if ${lt_cv_ar_at_file+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_ar_at_file=no cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : echo conftest.$ac_objext > conftest.lst lt_ar_try='$AR $AR_FLAGS libconftest.a @conftest.lst >&5' { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$lt_ar_try\""; } >&5 (eval $lt_ar_try) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } if test "$ac_status" -eq 0; then # Ensure the archiver fails upon bogus file names. rm -f conftest.$ac_objext libconftest.a { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$lt_ar_try\""; } >&5 (eval $lt_ar_try) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } if test "$ac_status" -ne 0; then lt_cv_ar_at_file=@ fi fi rm -f conftest.* libconftest.a fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ar_at_file" >&5 $as_echo "$lt_cv_ar_at_file" >&6; } if test "x$lt_cv_ar_at_file" = xno; then archiver_list_spec= else archiver_list_spec=$lt_cv_ar_at_file fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}strip", so it can be a program name with args. set dummy ${ac_tool_prefix}strip; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_STRIP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$STRIP"; then ac_cv_prog_STRIP="$STRIP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_STRIP="${ac_tool_prefix}strip" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi STRIP=$ac_cv_prog_STRIP if test -n "$STRIP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $STRIP" >&5 $as_echo "$STRIP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_STRIP"; then ac_ct_STRIP=$STRIP # Extract the first word of "strip", so it can be a program name with args. set dummy strip; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_STRIP+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_STRIP"; then ac_cv_prog_ac_ct_STRIP="$ac_ct_STRIP" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_STRIP="strip" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_STRIP=$ac_cv_prog_ac_ct_STRIP if test -n "$ac_ct_STRIP"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_STRIP" >&5 $as_echo "$ac_ct_STRIP" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_STRIP" = x; then STRIP=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac STRIP=$ac_ct_STRIP fi else STRIP="$ac_cv_prog_STRIP" fi test -z "$STRIP" && STRIP=: if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}ranlib", so it can be a program name with args. set dummy ${ac_tool_prefix}ranlib; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_RANLIB+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$RANLIB"; then ac_cv_prog_RANLIB="$RANLIB" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_RANLIB="${ac_tool_prefix}ranlib" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi RANLIB=$ac_cv_prog_RANLIB if test -n "$RANLIB"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $RANLIB" >&5 $as_echo "$RANLIB" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_RANLIB"; then ac_ct_RANLIB=$RANLIB # Extract the first word of "ranlib", so it can be a program name with args. set dummy ranlib; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_RANLIB+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_RANLIB"; then ac_cv_prog_ac_ct_RANLIB="$ac_ct_RANLIB" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_RANLIB="ranlib" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_RANLIB=$ac_cv_prog_ac_ct_RANLIB if test -n "$ac_ct_RANLIB"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_RANLIB" >&5 $as_echo "$ac_ct_RANLIB" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_RANLIB" = x; then RANLIB=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac RANLIB=$ac_ct_RANLIB fi else RANLIB="$ac_cv_prog_RANLIB" fi test -z "$RANLIB" && RANLIB=: # Determine commands to create old-style static archives. old_archive_cmds='$AR $AR_FLAGS $oldlib$oldobjs' old_postinstall_cmds='chmod 644 $oldlib' old_postuninstall_cmds= if test -n "$RANLIB"; then case $host_os in openbsd*) old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB -t \$tool_oldlib" ;; *) old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB \$tool_oldlib" ;; esac old_archive_cmds="$old_archive_cmds~\$RANLIB \$tool_oldlib" fi case $host_os in darwin*) lock_old_archive_extraction=yes ;; *) lock_old_archive_extraction=no ;; esac # If no C compiler was specified, use CC. LTCC=${LTCC-"$CC"} # If no C compiler flags were specified, use CFLAGS. LTCFLAGS=${LTCFLAGS-"$CFLAGS"} # Allow CC to be a program name with arguments. compiler=$CC # Check for command to grab the raw symbol name followed by C symbol from nm. { $as_echo "$as_me:${as_lineno-$LINENO}: checking command to parse $NM output from $compiler object" >&5 $as_echo_n "checking command to parse $NM output from $compiler object... " >&6; } if ${lt_cv_sys_global_symbol_pipe+:} false; then : $as_echo_n "(cached) " >&6 else # These are sane defaults that work on at least a few old systems. # [They come from Ultrix. What could be older than Ultrix?!! ;)] # Character class describing NM global symbol codes. symcode='[BCDEGRST]' # Regexp to match symbols that can be accessed directly from C. sympat='\([_A-Za-z][_A-Za-z0-9]*\)' # Define system-specific variables. case $host_os in aix*) symcode='[BCDT]' ;; cygwin* | mingw* | pw32* | cegcc*) symcode='[ABCDGISTW]' ;; hpux*) if test "$host_cpu" = ia64; then symcode='[ABCDEGRST]' fi ;; irix* | nonstopux*) symcode='[BCDEGRST]' ;; osf*) symcode='[BCDEGQRST]' ;; solaris*) symcode='[BDRT]' ;; sco3.2v5*) symcode='[DT]' ;; sysv4.2uw2*) symcode='[DT]' ;; sysv5* | sco5v6* | unixware* | OpenUNIX*) symcode='[ABDT]' ;; sysv4) symcode='[DFNSTU]' ;; esac # If we're using GNU nm, then use its standard symbol codes. case `$NM -V 2>&1` in *GNU* | *'with BFD'*) symcode='[ABCDGIRSTW]' ;; esac # Transform an extracted symbol line into a proper C declaration. # Some systems (esp. on ia64) link data and code symbols differently, # so use this general approach. lt_cv_sys_global_symbol_to_cdecl="sed -n -e 's/^T .* \(.*\)$/extern int \1();/p' -e 's/^$symcode* .* \(.*\)$/extern char \1;/p'" # Transform an extracted symbol line into symbol name and symbol address lt_cv_sys_global_symbol_to_c_name_address="sed -n -e 's/^: \([^ ]*\)[ ]*$/ {\\\"\1\\\", (void *) 0},/p' -e 's/^$symcode* \([^ ]*\) \([^ ]*\)$/ {\"\2\", (void *) \&\2},/p'" lt_cv_sys_global_symbol_to_c_name_address_lib_prefix="sed -n -e 's/^: \([^ ]*\)[ ]*$/ {\\\"\1\\\", (void *) 0},/p' -e 's/^$symcode* \([^ ]*\) \(lib[^ ]*\)$/ {\"\2\", (void *) \&\2},/p' -e 's/^$symcode* \([^ ]*\) \([^ ]*\)$/ {\"lib\2\", (void *) \&\2},/p'" # Handle CRLF in mingw tool chain opt_cr= case $build_os in mingw*) opt_cr=`$ECHO 'x\{0,1\}' | tr x '\015'` # option cr in regexp ;; esac # Try without a prefix underscore, then with it. for ac_symprfx in "" "_"; do # Transform symcode, sympat, and symprfx into a raw symbol and a C symbol. symxfrm="\\1 $ac_symprfx\\2 \\2" # Write the raw and C identifiers. if test "$lt_cv_nm_interface" = "MS dumpbin"; then # Fake it for dumpbin and say T for any non-static function # and D for any global variable. # Also find C++ and __fastcall symbols from MSVC++, # which start with @ or ?. lt_cv_sys_global_symbol_pipe="$AWK '"\ " {last_section=section; section=\$ 3};"\ " /^COFF SYMBOL TABLE/{for(i in hide) delete hide[i]};"\ " /Section length .*#relocs.*(pick any)/{hide[last_section]=1};"\ " \$ 0!~/External *\|/{next};"\ " / 0+ UNDEF /{next}; / UNDEF \([^|]\)*()/{next};"\ " {if(hide[section]) next};"\ " {f=0}; \$ 0~/\(\).*\|/{f=1}; {printf f ? \"T \" : \"D \"};"\ " {split(\$ 0, a, /\||\r/); split(a[2], s)};"\ " s[1]~/^[@?]/{print s[1], s[1]; next};"\ " s[1]~prfx {split(s[1],t,\"@\"); print t[1], substr(t[1],length(prfx))}"\ " ' prfx=^$ac_symprfx" else lt_cv_sys_global_symbol_pipe="sed -n -e 's/^.*[ ]\($symcode$symcode*\)[ ][ ]*$ac_symprfx$sympat$opt_cr$/$symxfrm/p'" fi lt_cv_sys_global_symbol_pipe="$lt_cv_sys_global_symbol_pipe | sed '/ __gnu_lto/d'" # Check to see that the pipe works correctly. pipe_works=no rm -f conftest* cat > conftest.$ac_ext <<_LT_EOF #ifdef __cplusplus extern "C" { #endif char nm_test_var; void nm_test_func(void); void nm_test_func(void){} #ifdef __cplusplus } #endif int main(){nm_test_var='a';nm_test_func();return(0);} _LT_EOF if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then # Now try to grab the symbols. nlist=conftest.nm if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$NM conftest.$ac_objext \| "$lt_cv_sys_global_symbol_pipe" \> $nlist\""; } >&5 (eval $NM conftest.$ac_objext \| "$lt_cv_sys_global_symbol_pipe" \> $nlist) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && test -s "$nlist"; then # Try sorting and uniquifying the output. if sort "$nlist" | uniq > "$nlist"T; then mv -f "$nlist"T "$nlist" else rm -f "$nlist"T fi # Make sure that we snagged all the symbols we need. if $GREP ' nm_test_var$' "$nlist" >/dev/null; then if $GREP ' nm_test_func$' "$nlist" >/dev/null; then cat <<_LT_EOF > conftest.$ac_ext /* Keep this code in sync between libtool.m4, ltmain, lt_system.h, and tests. */ #if defined(_WIN32) || defined(__CYGWIN__) || defined(_WIN32_WCE) /* DATA imports from DLLs on WIN32 con't be const, because runtime relocations are performed -- see ld's documentation on pseudo-relocs. */ # define LT_DLSYM_CONST #elif defined(__osf__) /* This system does not cope well with relocations in const data. */ # define LT_DLSYM_CONST #else # define LT_DLSYM_CONST const #endif #ifdef __cplusplus extern "C" { #endif _LT_EOF # Now generate the symbol file. eval "$lt_cv_sys_global_symbol_to_cdecl"' < "$nlist" | $GREP -v main >> conftest.$ac_ext' cat <<_LT_EOF >> conftest.$ac_ext /* The mapping between symbol names and symbols. */ LT_DLSYM_CONST struct { const char *name; void *address; } lt__PROGRAM__LTX_preloaded_symbols[] = { { "@PROGRAM@", (void *) 0 }, _LT_EOF $SED "s/^$symcode$symcode* \(.*\) \(.*\)$/ {\"\2\", (void *) \&\2},/" < "$nlist" | $GREP -v main >> conftest.$ac_ext cat <<\_LT_EOF >> conftest.$ac_ext {0, (void *) 0} }; /* This works around a problem in FreeBSD linker */ #ifdef FREEBSD_WORKAROUND static const void *lt_preloaded_setup() { return lt__PROGRAM__LTX_preloaded_symbols; } #endif #ifdef __cplusplus } #endif _LT_EOF # Now try linking the two files. mv conftest.$ac_objext conftstm.$ac_objext lt_globsym_save_LIBS=$LIBS lt_globsym_save_CFLAGS=$CFLAGS LIBS="conftstm.$ac_objext" CFLAGS="$CFLAGS$lt_prog_compiler_no_builtin_flag" if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 (eval $ac_link) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && test -s conftest${ac_exeext}; then pipe_works=yes fi LIBS=$lt_globsym_save_LIBS CFLAGS=$lt_globsym_save_CFLAGS else echo "cannot find nm_test_func in $nlist" >&5 fi else echo "cannot find nm_test_var in $nlist" >&5 fi else echo "cannot run $lt_cv_sys_global_symbol_pipe" >&5 fi else echo "$progname: failed program was:" >&5 cat conftest.$ac_ext >&5 fi rm -rf conftest* conftst* # Do not use the global_symbol_pipe unless it works. if test "$pipe_works" = yes; then break else lt_cv_sys_global_symbol_pipe= fi done fi if test -z "$lt_cv_sys_global_symbol_pipe"; then lt_cv_sys_global_symbol_to_cdecl= fi if test -z "$lt_cv_sys_global_symbol_pipe$lt_cv_sys_global_symbol_to_cdecl"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: failed" >&5 $as_echo "failed" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: ok" >&5 $as_echo "ok" >&6; } fi # Response file support. if test "$lt_cv_nm_interface" = "MS dumpbin"; then nm_file_list_spec='@' elif $NM --help 2>/dev/null | grep '[@]FILE' >/dev/null; then nm_file_list_spec='@' fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for sysroot" >&5 $as_echo_n "checking for sysroot... " >&6; } # Check whether --with-sysroot was given. if test "${with_sysroot+set}" = set; then : withval=$with_sysroot; else with_sysroot=no fi lt_sysroot= case ${with_sysroot} in #( yes) if test "$GCC" = yes; then lt_sysroot=`$CC --print-sysroot 2>/dev/null` fi ;; #( /*) lt_sysroot=`echo "$with_sysroot" | sed -e "$sed_quote_subst"` ;; #( no|'') ;; #( *) { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${with_sysroot}" >&5 $as_echo "${with_sysroot}" >&6; } as_fn_error $? "The sysroot must be an absolute path." "$LINENO" 5 ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${lt_sysroot:-no}" >&5 $as_echo "${lt_sysroot:-no}" >&6; } # Check whether --enable-libtool-lock was given. if test "${enable_libtool_lock+set}" = set; then : enableval=$enable_libtool_lock; fi test "x$enable_libtool_lock" != xno && enable_libtool_lock=yes # Some flags need to be propagated to the compiler or linker for good # libtool support. case $host in ia64-*-hpux*) # Find out which ABI we are using. echo 'int i;' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then case `/usr/bin/file conftest.$ac_objext` in *ELF-32*) HPUX_IA64_MODE="32" ;; *ELF-64*) HPUX_IA64_MODE="64" ;; esac fi rm -rf conftest* ;; *-*-irix6*) # Find out which ABI we are using. echo '#line '$LINENO' "configure"' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then if test "$lt_cv_prog_gnu_ld" = yes; then case `/usr/bin/file conftest.$ac_objext` in *32-bit*) LD="${LD-ld} -melf32bsmip" ;; *N32*) LD="${LD-ld} -melf32bmipn32" ;; *64-bit*) LD="${LD-ld} -melf64bmip" ;; esac else case `/usr/bin/file conftest.$ac_objext` in *32-bit*) LD="${LD-ld} -32" ;; *N32*) LD="${LD-ld} -n32" ;; *64-bit*) LD="${LD-ld} -64" ;; esac fi fi rm -rf conftest* ;; x86_64-*kfreebsd*-gnu|x86_64-*linux*|ppc*-*linux*|powerpc*-*linux*| \ s390*-*linux*|s390*-*tpf*|sparc*-*linux*) # Find out which ABI we are using. echo 'int i;' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then case `/usr/bin/file conftest.o` in *32-bit*) case $host in x86_64-*kfreebsd*-gnu) LD="${LD-ld} -m elf_i386_fbsd" ;; x86_64-*linux*) LD="${LD-ld} -m elf_i386" ;; ppc64-*linux*|powerpc64-*linux*) LD="${LD-ld} -m elf32ppclinux" ;; s390x-*linux*) LD="${LD-ld} -m elf_s390" ;; sparc64-*linux*) LD="${LD-ld} -m elf32_sparc" ;; esac ;; *64-bit*) case $host in x86_64-*kfreebsd*-gnu) LD="${LD-ld} -m elf_x86_64_fbsd" ;; x86_64-*linux*) LD="${LD-ld} -m elf_x86_64" ;; ppc*-*linux*|powerpc*-*linux*) LD="${LD-ld} -m elf64ppc" ;; s390*-*linux*|s390*-*tpf*) LD="${LD-ld} -m elf64_s390" ;; sparc*-*linux*) LD="${LD-ld} -m elf64_sparc" ;; esac ;; esac fi rm -rf conftest* ;; *-*-sco3.2v5*) # On SCO OpenServer 5, we need -belf to get full-featured binaries. SAVE_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -belf" { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the C compiler needs -belf" >&5 $as_echo_n "checking whether the C compiler needs -belf... " >&6; } if ${lt_cv_cc_needs_belf+:} false; then : $as_echo_n "(cached) " >&6 else ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_cv_cc_needs_belf=yes else lt_cv_cc_needs_belf=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_cc_needs_belf" >&5 $as_echo "$lt_cv_cc_needs_belf" >&6; } if test x"$lt_cv_cc_needs_belf" != x"yes"; then # this is probably gcc 2.8.0, egcs 1.0 or newer; no need for -belf CFLAGS="$SAVE_CFLAGS" fi ;; *-*solaris*) # Find out which ABI we are using. echo 'int i;' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then case `/usr/bin/file conftest.o` in *64-bit*) case $lt_cv_prog_gnu_ld in yes*) case $host in i?86-*-solaris*) LD="${LD-ld} -m elf_x86_64" ;; sparc*-*-solaris*) LD="${LD-ld} -m elf64_sparc" ;; esac # GNU ld 2.21 introduced _sol2 emulations. Use them if available. if ${LD-ld} -V | grep _sol2 >/dev/null 2>&1; then LD="${LD-ld}_sol2" fi ;; *) if ${LD-ld} -64 -r -o conftest2.o conftest.o >/dev/null 2>&1; then LD="${LD-ld} -64" fi ;; esac ;; esac fi rm -rf conftest* ;; esac need_locks="$enable_libtool_lock" if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}mt", so it can be a program name with args. set dummy ${ac_tool_prefix}mt; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_MANIFEST_TOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$MANIFEST_TOOL"; then ac_cv_prog_MANIFEST_TOOL="$MANIFEST_TOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_MANIFEST_TOOL="${ac_tool_prefix}mt" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi MANIFEST_TOOL=$ac_cv_prog_MANIFEST_TOOL if test -n "$MANIFEST_TOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MANIFEST_TOOL" >&5 $as_echo "$MANIFEST_TOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_MANIFEST_TOOL"; then ac_ct_MANIFEST_TOOL=$MANIFEST_TOOL # Extract the first word of "mt", so it can be a program name with args. set dummy mt; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_MANIFEST_TOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_MANIFEST_TOOL"; then ac_cv_prog_ac_ct_MANIFEST_TOOL="$ac_ct_MANIFEST_TOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_MANIFEST_TOOL="mt" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_MANIFEST_TOOL=$ac_cv_prog_ac_ct_MANIFEST_TOOL if test -n "$ac_ct_MANIFEST_TOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_MANIFEST_TOOL" >&5 $as_echo "$ac_ct_MANIFEST_TOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_MANIFEST_TOOL" = x; then MANIFEST_TOOL=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac MANIFEST_TOOL=$ac_ct_MANIFEST_TOOL fi else MANIFEST_TOOL="$ac_cv_prog_MANIFEST_TOOL" fi test -z "$MANIFEST_TOOL" && MANIFEST_TOOL=mt { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $MANIFEST_TOOL is a manifest tool" >&5 $as_echo_n "checking if $MANIFEST_TOOL is a manifest tool... " >&6; } if ${lt_cv_path_mainfest_tool+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_path_mainfest_tool=no echo "$as_me:$LINENO: $MANIFEST_TOOL '-?'" >&5 $MANIFEST_TOOL '-?' 2>conftest.err > conftest.out cat conftest.err >&5 if $GREP 'Manifest Tool' conftest.out > /dev/null; then lt_cv_path_mainfest_tool=yes fi rm -f conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_path_mainfest_tool" >&5 $as_echo "$lt_cv_path_mainfest_tool" >&6; } if test "x$lt_cv_path_mainfest_tool" != xyes; then MANIFEST_TOOL=: fi case $host_os in rhapsody* | darwin*) if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}dsymutil", so it can be a program name with args. set dummy ${ac_tool_prefix}dsymutil; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_DSYMUTIL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$DSYMUTIL"; then ac_cv_prog_DSYMUTIL="$DSYMUTIL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_DSYMUTIL="${ac_tool_prefix}dsymutil" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi DSYMUTIL=$ac_cv_prog_DSYMUTIL if test -n "$DSYMUTIL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DSYMUTIL" >&5 $as_echo "$DSYMUTIL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_DSYMUTIL"; then ac_ct_DSYMUTIL=$DSYMUTIL # Extract the first word of "dsymutil", so it can be a program name with args. set dummy dsymutil; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_DSYMUTIL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_DSYMUTIL"; then ac_cv_prog_ac_ct_DSYMUTIL="$ac_ct_DSYMUTIL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_DSYMUTIL="dsymutil" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_DSYMUTIL=$ac_cv_prog_ac_ct_DSYMUTIL if test -n "$ac_ct_DSYMUTIL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DSYMUTIL" >&5 $as_echo "$ac_ct_DSYMUTIL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_DSYMUTIL" = x; then DSYMUTIL=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac DSYMUTIL=$ac_ct_DSYMUTIL fi else DSYMUTIL="$ac_cv_prog_DSYMUTIL" fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}nmedit", so it can be a program name with args. set dummy ${ac_tool_prefix}nmedit; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_NMEDIT+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$NMEDIT"; then ac_cv_prog_NMEDIT="$NMEDIT" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_NMEDIT="${ac_tool_prefix}nmedit" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi NMEDIT=$ac_cv_prog_NMEDIT if test -n "$NMEDIT"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $NMEDIT" >&5 $as_echo "$NMEDIT" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_NMEDIT"; then ac_ct_NMEDIT=$NMEDIT # Extract the first word of "nmedit", so it can be a program name with args. set dummy nmedit; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_NMEDIT+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_NMEDIT"; then ac_cv_prog_ac_ct_NMEDIT="$ac_ct_NMEDIT" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_NMEDIT="nmedit" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_NMEDIT=$ac_cv_prog_ac_ct_NMEDIT if test -n "$ac_ct_NMEDIT"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_NMEDIT" >&5 $as_echo "$ac_ct_NMEDIT" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_NMEDIT" = x; then NMEDIT=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac NMEDIT=$ac_ct_NMEDIT fi else NMEDIT="$ac_cv_prog_NMEDIT" fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}lipo", so it can be a program name with args. set dummy ${ac_tool_prefix}lipo; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_LIPO+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$LIPO"; then ac_cv_prog_LIPO="$LIPO" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_LIPO="${ac_tool_prefix}lipo" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi LIPO=$ac_cv_prog_LIPO if test -n "$LIPO"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LIPO" >&5 $as_echo "$LIPO" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_LIPO"; then ac_ct_LIPO=$LIPO # Extract the first word of "lipo", so it can be a program name with args. set dummy lipo; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_LIPO+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_LIPO"; then ac_cv_prog_ac_ct_LIPO="$ac_ct_LIPO" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_LIPO="lipo" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_LIPO=$ac_cv_prog_ac_ct_LIPO if test -n "$ac_ct_LIPO"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_LIPO" >&5 $as_echo "$ac_ct_LIPO" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_LIPO" = x; then LIPO=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac LIPO=$ac_ct_LIPO fi else LIPO="$ac_cv_prog_LIPO" fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}otool", so it can be a program name with args. set dummy ${ac_tool_prefix}otool; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_OTOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$OTOOL"; then ac_cv_prog_OTOOL="$OTOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_OTOOL="${ac_tool_prefix}otool" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi OTOOL=$ac_cv_prog_OTOOL if test -n "$OTOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OTOOL" >&5 $as_echo "$OTOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_OTOOL"; then ac_ct_OTOOL=$OTOOL # Extract the first word of "otool", so it can be a program name with args. set dummy otool; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_OTOOL+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_OTOOL"; then ac_cv_prog_ac_ct_OTOOL="$ac_ct_OTOOL" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_OTOOL="otool" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_OTOOL=$ac_cv_prog_ac_ct_OTOOL if test -n "$ac_ct_OTOOL"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OTOOL" >&5 $as_echo "$ac_ct_OTOOL" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_OTOOL" = x; then OTOOL=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac OTOOL=$ac_ct_OTOOL fi else OTOOL="$ac_cv_prog_OTOOL" fi if test -n "$ac_tool_prefix"; then # Extract the first word of "${ac_tool_prefix}otool64", so it can be a program name with args. set dummy ${ac_tool_prefix}otool64; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_OTOOL64+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$OTOOL64"; then ac_cv_prog_OTOOL64="$OTOOL64" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_OTOOL64="${ac_tool_prefix}otool64" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi OTOOL64=$ac_cv_prog_OTOOL64 if test -n "$OTOOL64"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OTOOL64" >&5 $as_echo "$OTOOL64" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi if test -z "$ac_cv_prog_OTOOL64"; then ac_ct_OTOOL64=$OTOOL64 # Extract the first word of "otool64", so it can be a program name with args. set dummy otool64; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } if ${ac_cv_prog_ac_ct_OTOOL64+:} false; then : $as_echo_n "(cached) " >&6 else if test -n "$ac_ct_OTOOL64"; then ac_cv_prog_ac_ct_OTOOL64="$ac_ct_OTOOL64" # Let the user override the test. else as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then ac_cv_prog_ac_ct_OTOOL64="otool64" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi done done IFS=$as_save_IFS fi fi ac_ct_OTOOL64=$ac_cv_prog_ac_ct_OTOOL64 if test -n "$ac_ct_OTOOL64"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OTOOL64" >&5 $as_echo "$ac_ct_OTOOL64" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test "x$ac_ct_OTOOL64" = x; then OTOOL64=":" else case $cross_compiling:$ac_tool_warned in yes:) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 $as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} ac_tool_warned=yes ;; esac OTOOL64=$ac_ct_OTOOL64 fi else OTOOL64="$ac_cv_prog_OTOOL64" fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -single_module linker flag" >&5 $as_echo_n "checking for -single_module linker flag... " >&6; } if ${lt_cv_apple_cc_single_mod+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_apple_cc_single_mod=no if test -z "${LT_MULTI_MODULE}"; then # By default we will add the -single_module flag. You can override # by either setting the environment variable LT_MULTI_MODULE # non-empty at configure time, or by adding -multi_module to the # link flags. rm -rf libconftest.dylib* echo "int foo(void){return 1;}" > conftest.c echo "$LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ -dynamiclib -Wl,-single_module conftest.c" >&5 $LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ -dynamiclib -Wl,-single_module conftest.c 2>conftest.err _lt_result=$? # If there is a non-empty error log, and "single_module" # appears in it, assume the flag caused a linker warning if test -s conftest.err && $GREP single_module conftest.err; then cat conftest.err >&5 # Otherwise, if the output was created with a 0 exit code from # the compiler, it worked. elif test -f libconftest.dylib && test $_lt_result -eq 0; then lt_cv_apple_cc_single_mod=yes else cat conftest.err >&5 fi rm -rf libconftest.dylib* rm -f conftest.* fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_apple_cc_single_mod" >&5 $as_echo "$lt_cv_apple_cc_single_mod" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -exported_symbols_list linker flag" >&5 $as_echo_n "checking for -exported_symbols_list linker flag... " >&6; } if ${lt_cv_ld_exported_symbols_list+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_ld_exported_symbols_list=no save_LDFLAGS=$LDFLAGS echo "_main" > conftest.sym LDFLAGS="$LDFLAGS -Wl,-exported_symbols_list,conftest.sym" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_cv_ld_exported_symbols_list=yes else lt_cv_ld_exported_symbols_list=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LDFLAGS="$save_LDFLAGS" fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_exported_symbols_list" >&5 $as_echo "$lt_cv_ld_exported_symbols_list" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -force_load linker flag" >&5 $as_echo_n "checking for -force_load linker flag... " >&6; } if ${lt_cv_ld_force_load+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_ld_force_load=no cat > conftest.c << _LT_EOF int forced_loaded() { return 2;} _LT_EOF echo "$LTCC $LTCFLAGS -c -o conftest.o conftest.c" >&5 $LTCC $LTCFLAGS -c -o conftest.o conftest.c 2>&5 echo "$AR cru libconftest.a conftest.o" >&5 $AR cru libconftest.a conftest.o 2>&5 echo "$RANLIB libconftest.a" >&5 $RANLIB libconftest.a 2>&5 cat > conftest.c << _LT_EOF int main() { return 0;} _LT_EOF echo "$LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a" >&5 $LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a 2>conftest.err _lt_result=$? if test -s conftest.err && $GREP force_load conftest.err; then cat conftest.err >&5 elif test -f conftest && test $_lt_result -eq 0 && $GREP forced_load conftest >/dev/null 2>&1 ; then lt_cv_ld_force_load=yes else cat conftest.err >&5 fi rm -f conftest.err libconftest.a conftest conftest.c rm -rf conftest.dSYM fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_force_load" >&5 $as_echo "$lt_cv_ld_force_load" >&6; } case $host_os in rhapsody* | darwin1.[012]) _lt_dar_allow_undefined='${wl}-undefined ${wl}suppress' ;; darwin1.*) _lt_dar_allow_undefined='${wl}-flat_namespace ${wl}-undefined ${wl}suppress' ;; darwin*) # darwin 5.x on # if running on 10.5 or later, the deployment target defaults # to the OS version, if on x86, and 10.4, the deployment # target defaults to 10.4. Don't you love it? case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in 10.0,*86*-darwin8*|10.0,*-darwin[91]*) _lt_dar_allow_undefined='${wl}-undefined ${wl}dynamic_lookup' ;; 10.[012]*) _lt_dar_allow_undefined='${wl}-flat_namespace ${wl}-undefined ${wl}suppress' ;; 10.*) _lt_dar_allow_undefined='${wl}-undefined ${wl}dynamic_lookup' ;; esac ;; esac if test "$lt_cv_apple_cc_single_mod" = "yes"; then _lt_dar_single_mod='$single_module' fi if test "$lt_cv_ld_exported_symbols_list" = "yes"; then _lt_dar_export_syms=' ${wl}-exported_symbols_list,$output_objdir/${libname}-symbols.expsym' else _lt_dar_export_syms='~$NMEDIT -s $output_objdir/${libname}-symbols.expsym ${lib}' fi if test "$DSYMUTIL" != ":" && test "$lt_cv_ld_force_load" = "no"; then _lt_dsymutil='~$DSYMUTIL $lib || :' else _lt_dsymutil= fi ;; esac for ac_header in dlfcn.h do : ac_fn_c_check_header_compile "$LINENO" "dlfcn.h" "ac_cv_header_dlfcn_h" "$ac_includes_default " if test "x$ac_cv_header_dlfcn_h" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_DLFCN_H 1 _ACEOF fi done # Set options # Check whether --enable-static was given. if test "${enable_static+set}" = set; then : enableval=$enable_static; p=${PACKAGE-default} case $enableval in yes) enable_static=yes ;; no) enable_static=no ;; *) enable_static=no # Look at the argument we got. We use all the common list separators. lt_save_ifs="$IFS"; IFS="${IFS}$PATH_SEPARATOR," for pkg in $enableval; do IFS="$lt_save_ifs" if test "X$pkg" = "X$p"; then enable_static=yes fi done IFS="$lt_save_ifs" ;; esac else enable_static=no fi # Check whether --with-pic was given. if test "${with_pic+set}" = set; then : withval=$with_pic; lt_p=${PACKAGE-default} case $withval in yes|no) pic_mode=$withval ;; *) pic_mode=default # Look at the argument we got. We use all the common list separators. lt_save_ifs="$IFS"; IFS="${IFS}$PATH_SEPARATOR," for lt_pkg in $withval; do IFS="$lt_save_ifs" if test "X$lt_pkg" = "X$lt_p"; then pic_mode=yes fi done IFS="$lt_save_ifs" ;; esac else pic_mode=default fi test -z "$pic_mode" && pic_mode=yes enable_dlopen=no enable_win32_dll=no # Check whether --enable-shared was given. if test "${enable_shared+set}" = set; then : enableval=$enable_shared; p=${PACKAGE-default} case $enableval in yes) enable_shared=yes ;; no) enable_shared=no ;; *) enable_shared=no # Look at the argument we got. We use all the common list separators. lt_save_ifs="$IFS"; IFS="${IFS}$PATH_SEPARATOR," for pkg in $enableval; do IFS="$lt_save_ifs" if test "X$pkg" = "X$p"; then enable_shared=yes fi done IFS="$lt_save_ifs" ;; esac else enable_shared=yes fi # Check whether --enable-fast-install was given. if test "${enable_fast_install+set}" = set; then : enableval=$enable_fast_install; p=${PACKAGE-default} case $enableval in yes) enable_fast_install=yes ;; no) enable_fast_install=no ;; *) enable_fast_install=no # Look at the argument we got. We use all the common list separators. lt_save_ifs="$IFS"; IFS="${IFS}$PATH_SEPARATOR," for pkg in $enableval; do IFS="$lt_save_ifs" if test "X$pkg" = "X$p"; then enable_fast_install=yes fi done IFS="$lt_save_ifs" ;; esac else enable_fast_install=yes fi # This can be used to rebuild libtool when needed LIBTOOL_DEPS="$ltmain" # Always use our own libtool. LIBTOOL='$(SHELL) $(top_builddir)/libtool' test -z "$LN_S" && LN_S="ln -s" if test -n "${ZSH_VERSION+set}" ; then setopt NO_GLOB_SUBST fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for objdir" >&5 $as_echo_n "checking for objdir... " >&6; } if ${lt_cv_objdir+:} false; then : $as_echo_n "(cached) " >&6 else rm -f .libs 2>/dev/null mkdir .libs 2>/dev/null if test -d .libs; then lt_cv_objdir=.libs else # MS-DOS does not allow filenames that begin with a dot. lt_cv_objdir=_libs fi rmdir .libs 2>/dev/null fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_objdir" >&5 $as_echo "$lt_cv_objdir" >&6; } objdir=$lt_cv_objdir cat >>confdefs.h <<_ACEOF #define LT_OBJDIR "$lt_cv_objdir/" _ACEOF case $host_os in aix3*) # AIX sometimes has problems with the GCC collect2 program. For some # reason, if we set the COLLECT_NAMES environment variable, the problems # vanish in a puff of smoke. if test "X${COLLECT_NAMES+set}" != Xset; then COLLECT_NAMES= export COLLECT_NAMES fi ;; esac # Global variables: ofile=libtool can_build_shared=yes # All known linkers require a `.a' archive for static linking (except MSVC, # which needs '.lib'). libext=a with_gnu_ld="$lt_cv_prog_gnu_ld" old_CC="$CC" old_CFLAGS="$CFLAGS" # Set sane defaults for various variables test -z "$CC" && CC=cc test -z "$LTCC" && LTCC=$CC test -z "$LTCFLAGS" && LTCFLAGS=$CFLAGS test -z "$LD" && LD=ld test -z "$ac_objext" && ac_objext=o for cc_temp in $compiler""; do case $cc_temp in compile | *[\\/]compile | ccache | *[\\/]ccache ) ;; distcc | *[\\/]distcc | purify | *[\\/]purify ) ;; \-*) ;; *) break;; esac done cc_basename=`$ECHO "$cc_temp" | $SED "s%.*/%%; s%^$host_alias-%%"` # Only perform the check for file, if the check method requires it test -z "$MAGIC_CMD" && MAGIC_CMD=file case $deplibs_check_method in file_magic*) if test "$file_magic_cmd" = '$MAGIC_CMD'; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ${ac_tool_prefix}file" >&5 $as_echo_n "checking for ${ac_tool_prefix}file... " >&6; } if ${lt_cv_path_MAGIC_CMD+:} false; then : $as_echo_n "(cached) " >&6 else case $MAGIC_CMD in [\\/*] | ?:[\\/]*) lt_cv_path_MAGIC_CMD="$MAGIC_CMD" # Let the user override the test with a path. ;; *) lt_save_MAGIC_CMD="$MAGIC_CMD" lt_save_ifs="$IFS"; IFS=$PATH_SEPARATOR ac_dummy="/usr/bin$PATH_SEPARATOR$PATH" for ac_dir in $ac_dummy; do IFS="$lt_save_ifs" test -z "$ac_dir" && ac_dir=. if test -f $ac_dir/${ac_tool_prefix}file; then lt_cv_path_MAGIC_CMD="$ac_dir/${ac_tool_prefix}file" if test -n "$file_magic_test_file"; then case $deplibs_check_method in "file_magic "*) file_magic_regex=`expr "$deplibs_check_method" : "file_magic \(.*\)"` MAGIC_CMD="$lt_cv_path_MAGIC_CMD" if eval $file_magic_cmd \$file_magic_test_file 2> /dev/null | $EGREP "$file_magic_regex" > /dev/null; then : else cat <<_LT_EOF 1>&2 *** Warning: the command libtool uses to detect shared libraries, *** $file_magic_cmd, produces output that libtool cannot recognize. *** The result is that libtool may fail to recognize shared libraries *** as such. This will affect the creation of libtool libraries that *** depend on shared libraries, but programs linked with such libtool *** libraries will work regardless of this problem. Nevertheless, you *** may want to report the problem to your system manager and/or to *** bug-libtool@gnu.org _LT_EOF fi ;; esac fi break fi done IFS="$lt_save_ifs" MAGIC_CMD="$lt_save_MAGIC_CMD" ;; esac fi MAGIC_CMD="$lt_cv_path_MAGIC_CMD" if test -n "$MAGIC_CMD"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MAGIC_CMD" >&5 $as_echo "$MAGIC_CMD" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi if test -z "$lt_cv_path_MAGIC_CMD"; then if test -n "$ac_tool_prefix"; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for file" >&5 $as_echo_n "checking for file... " >&6; } if ${lt_cv_path_MAGIC_CMD+:} false; then : $as_echo_n "(cached) " >&6 else case $MAGIC_CMD in [\\/*] | ?:[\\/]*) lt_cv_path_MAGIC_CMD="$MAGIC_CMD" # Let the user override the test with a path. ;; *) lt_save_MAGIC_CMD="$MAGIC_CMD" lt_save_ifs="$IFS"; IFS=$PATH_SEPARATOR ac_dummy="/usr/bin$PATH_SEPARATOR$PATH" for ac_dir in $ac_dummy; do IFS="$lt_save_ifs" test -z "$ac_dir" && ac_dir=. if test -f $ac_dir/file; then lt_cv_path_MAGIC_CMD="$ac_dir/file" if test -n "$file_magic_test_file"; then case $deplibs_check_method in "file_magic "*) file_magic_regex=`expr "$deplibs_check_method" : "file_magic \(.*\)"` MAGIC_CMD="$lt_cv_path_MAGIC_CMD" if eval $file_magic_cmd \$file_magic_test_file 2> /dev/null | $EGREP "$file_magic_regex" > /dev/null; then : else cat <<_LT_EOF 1>&2 *** Warning: the command libtool uses to detect shared libraries, *** $file_magic_cmd, produces output that libtool cannot recognize. *** The result is that libtool may fail to recognize shared libraries *** as such. This will affect the creation of libtool libraries that *** depend on shared libraries, but programs linked with such libtool *** libraries will work regardless of this problem. Nevertheless, you *** may want to report the problem to your system manager and/or to *** bug-libtool@gnu.org _LT_EOF fi ;; esac fi break fi done IFS="$lt_save_ifs" MAGIC_CMD="$lt_save_MAGIC_CMD" ;; esac fi MAGIC_CMD="$lt_cv_path_MAGIC_CMD" if test -n "$MAGIC_CMD"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MAGIC_CMD" >&5 $as_echo "$MAGIC_CMD" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi else MAGIC_CMD=: fi fi fi ;; esac # Use C for the default configuration in the libtool script lt_save_CC="$CC" ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu # Source file extension for C test sources. ac_ext=c # Object file extension for compiled C test sources. objext=o objext=$objext # Code to be used in simple compile tests lt_simple_compile_test_code="int some_variable = 0;" # Code to be used in simple link tests lt_simple_link_test_code='int main(){return(0);}' # If no C compiler was specified, use CC. LTCC=${LTCC-"$CC"} # If no C compiler flags were specified, use CFLAGS. LTCFLAGS=${LTCFLAGS-"$CFLAGS"} # Allow CC to be a program name with arguments. compiler=$CC # Save the default compiler, since it gets overwritten when the other # tags are being tested, and _LT_TAGVAR(compiler, []) is a NOP. compiler_DEFAULT=$CC # save warnings/boilerplate of simple test code ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" >conftest.$ac_ext eval "$ac_compile" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err _lt_compiler_boilerplate=`cat conftest.err` $RM conftest* ac_outfile=conftest.$ac_objext echo "$lt_simple_link_test_code" >conftest.$ac_ext eval "$ac_link" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err _lt_linker_boilerplate=`cat conftest.err` $RM -r conftest* ## CAVEAT EMPTOR: ## There is no encapsulation within the following macros, do not change ## the running order or otherwise move them around unless you know exactly ## what you are doing... if test -n "$compiler"; then lt_prog_compiler_no_builtin_flag= if test "$GCC" = yes; then case $cc_basename in nvcc*) lt_prog_compiler_no_builtin_flag=' -Xcompiler -fno-builtin' ;; *) lt_prog_compiler_no_builtin_flag=' -fno-builtin' ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -fno-rtti -fno-exceptions" >&5 $as_echo_n "checking if $compiler supports -fno-rtti -fno-exceptions... " >&6; } if ${lt_cv_prog_compiler_rtti_exceptions+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_rtti_exceptions=no ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-fno-rtti -fno-exceptions" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_rtti_exceptions=yes fi fi $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_rtti_exceptions" >&5 $as_echo "$lt_cv_prog_compiler_rtti_exceptions" >&6; } if test x"$lt_cv_prog_compiler_rtti_exceptions" = xyes; then lt_prog_compiler_no_builtin_flag="$lt_prog_compiler_no_builtin_flag -fno-rtti -fno-exceptions" else : fi fi lt_prog_compiler_wl= lt_prog_compiler_pic= lt_prog_compiler_static= if test "$GCC" = yes; then lt_prog_compiler_wl='-Wl,' lt_prog_compiler_static='-static' case $host_os in aix*) # All AIX code is PIC. if test "$host_cpu" = ia64; then # AIX 5 now supports IA64 processor lt_prog_compiler_static='-Bstatic' fi ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support lt_prog_compiler_pic='-fPIC' ;; m68k) # FIXME: we need at least 68020 code to build shared libraries, but # adding the `-m68020' flag to GCC prevents building anything better, # like `-m68040'. lt_prog_compiler_pic='-m68020 -resident32 -malways-restore-a4' ;; esac ;; beos* | irix5* | irix6* | nonstopux* | osf3* | osf4* | osf5*) # PIC is the default for these OSes. ;; mingw* | cygwin* | pw32* | os2* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). # Although the cygwin gcc ignores -fPIC, still need this for old-style # (--disable-auto-import) libraries lt_prog_compiler_pic='-DDLL_EXPORT' ;; darwin* | rhapsody*) # PIC is the default on this platform # Common symbols not allowed in MH_DYLIB files lt_prog_compiler_pic='-fno-common' ;; haiku*) # PIC is the default for Haiku. # The "-static" flag exists, but is broken. lt_prog_compiler_static= ;; hpux*) # PIC is the default for 64-bit PA HP-UX, but not for 32-bit # PA HP-UX. On IA64 HP-UX, PIC is the default but the pic flag # sets the default TLS model and affects inlining. case $host_cpu in hppa*64*) # +Z the default ;; *) lt_prog_compiler_pic='-fPIC' ;; esac ;; interix[3-9]*) # Interix 3.x gcc -fpic/-fPIC options generate broken code. # Instead, we relocate shared libraries at runtime. ;; msdosdjgpp*) # Just because we use GCC doesn't mean we suddenly get shared libraries # on systems that don't support them. lt_prog_compiler_can_build_shared=no enable_shared=no ;; *nto* | *qnx*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. lt_prog_compiler_pic='-fPIC -shared' ;; sysv4*MP*) if test -d /usr/nec; then lt_prog_compiler_pic=-Kconform_pic fi ;; *) lt_prog_compiler_pic='-fPIC' ;; esac case $cc_basename in nvcc*) # Cuda Compiler Driver 2.2 lt_prog_compiler_wl='-Xlinker ' if test -n "$lt_prog_compiler_pic"; then lt_prog_compiler_pic="-Xcompiler $lt_prog_compiler_pic" fi ;; esac else # PORTME Check for flag to pass linker flags through the system compiler. case $host_os in aix*) lt_prog_compiler_wl='-Wl,' if test "$host_cpu" = ia64; then # AIX 5 now supports IA64 processor lt_prog_compiler_static='-Bstatic' else lt_prog_compiler_static='-bnso -bI:/lib/syscalls.exp' fi ;; mingw* | cygwin* | pw32* | os2* | cegcc*) # This hack is so that the source file can tell whether it is being # built for inclusion in a dll (and should export symbols for example). lt_prog_compiler_pic='-DDLL_EXPORT' ;; hpux9* | hpux10* | hpux11*) lt_prog_compiler_wl='-Wl,' # PIC is the default for IA64 HP-UX and 64-bit HP-UX, but # not for PA HP-UX. case $host_cpu in hppa*64*|ia64*) # +Z the default ;; *) lt_prog_compiler_pic='+Z' ;; esac # Is there a better lt_prog_compiler_static that works with the bundled CC? lt_prog_compiler_static='${wl}-a ${wl}archive' ;; irix5* | irix6* | nonstopux*) lt_prog_compiler_wl='-Wl,' # PIC (with -KPIC) is the default. lt_prog_compiler_static='-non_shared' ;; linux* | k*bsd*-gnu | kopensolaris*-gnu) case $cc_basename in # old Intel for x86_64 which still supported -KPIC. ecc*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-static' ;; # icc used to be incompatible with GCC. # ICC 10 doesn't accept -KPIC any more. icc* | ifort*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fPIC' lt_prog_compiler_static='-static' ;; # Lahey Fortran 8.1. lf95*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='--shared' lt_prog_compiler_static='--static' ;; nagfor*) # NAG Fortran compiler lt_prog_compiler_wl='-Wl,-Wl,,' lt_prog_compiler_pic='-PIC' lt_prog_compiler_static='-Bstatic' ;; pgcc* | pgf77* | pgf90* | pgf95* | pgfortran*) # Portland Group compilers (*not* the Pentium gcc compiler, # which looks to be a dead project) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fpic' lt_prog_compiler_static='-Bstatic' ;; ccc*) lt_prog_compiler_wl='-Wl,' # All Alpha code is PIC. lt_prog_compiler_static='-non_shared' ;; xl* | bgxl* | bgf* | mpixl*) # IBM XL C 8.0/Fortran 10.1, 11.1 on PPC and BlueGene lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-qpic' lt_prog_compiler_static='-qstaticlink' ;; *) case `$CC -V 2>&1 | sed 5q` in *Sun\ Ceres\ Fortran* | *Sun*Fortran*\ [1-7].* | *Sun*Fortran*\ 8.[0-3]*) # Sun Fortran 8.3 passes all unrecognized flags to the linker lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' lt_prog_compiler_wl='' ;; *Sun\ F* | *Sun*Fortran*) lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' lt_prog_compiler_wl='-Qoption ld ' ;; *Sun\ C*) # Sun C 5.9 lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' lt_prog_compiler_wl='-Wl,' ;; *Intel*\ [CF]*Compiler*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fPIC' lt_prog_compiler_static='-static' ;; *Portland\ Group*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-fpic' lt_prog_compiler_static='-Bstatic' ;; esac ;; esac ;; newsos6) lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' ;; *nto* | *qnx*) # QNX uses GNU C++, but need to define -shared option too, otherwise # it will coredump. lt_prog_compiler_pic='-fPIC -shared' ;; osf3* | osf4* | osf5*) lt_prog_compiler_wl='-Wl,' # All OSF/1 code is PIC. lt_prog_compiler_static='-non_shared' ;; rdos*) lt_prog_compiler_static='-non_shared' ;; solaris*) lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' case $cc_basename in f77* | f90* | f95* | sunf77* | sunf90* | sunf95*) lt_prog_compiler_wl='-Qoption ld ';; *) lt_prog_compiler_wl='-Wl,';; esac ;; sunos4*) lt_prog_compiler_wl='-Qoption ld ' lt_prog_compiler_pic='-PIC' lt_prog_compiler_static='-Bstatic' ;; sysv4 | sysv4.2uw2* | sysv4.3*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' ;; sysv4*MP*) if test -d /usr/nec ;then lt_prog_compiler_pic='-Kconform_pic' lt_prog_compiler_static='-Bstatic' fi ;; sysv5* | unixware* | sco3.2v5* | sco5v6* | OpenUNIX*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_pic='-KPIC' lt_prog_compiler_static='-Bstatic' ;; unicos*) lt_prog_compiler_wl='-Wl,' lt_prog_compiler_can_build_shared=no ;; uts4*) lt_prog_compiler_pic='-pic' lt_prog_compiler_static='-Bstatic' ;; *) lt_prog_compiler_can_build_shared=no ;; esac fi case $host_os in # For platforms which do not support PIC, -DPIC is meaningless: *djgpp*) lt_prog_compiler_pic= ;; *) lt_prog_compiler_pic="$lt_prog_compiler_pic -DPIC" ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $compiler option to produce PIC" >&5 $as_echo_n "checking for $compiler option to produce PIC... " >&6; } if ${lt_cv_prog_compiler_pic+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_pic=$lt_prog_compiler_pic fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_pic" >&5 $as_echo "$lt_cv_prog_compiler_pic" >&6; } lt_prog_compiler_pic=$lt_cv_prog_compiler_pic # # Check to make sure the PIC flag actually works. # if test -n "$lt_prog_compiler_pic"; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler PIC flag $lt_prog_compiler_pic works" >&5 $as_echo_n "checking if $compiler PIC flag $lt_prog_compiler_pic works... " >&6; } if ${lt_cv_prog_compiler_pic_works+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_pic_works=no ac_outfile=conftest.$ac_objext echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="$lt_prog_compiler_pic -DPIC" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_pic_works=yes fi fi $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_pic_works" >&5 $as_echo "$lt_cv_prog_compiler_pic_works" >&6; } if test x"$lt_cv_prog_compiler_pic_works" = xyes; then case $lt_prog_compiler_pic in "" | " "*) ;; *) lt_prog_compiler_pic=" $lt_prog_compiler_pic" ;; esac else lt_prog_compiler_pic= lt_prog_compiler_can_build_shared=no fi fi # # Check to make sure the static flag actually works. # wl=$lt_prog_compiler_wl eval lt_tmp_static_flag=\"$lt_prog_compiler_static\" { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler static flag $lt_tmp_static_flag works" >&5 $as_echo_n "checking if $compiler static flag $lt_tmp_static_flag works... " >&6; } if ${lt_cv_prog_compiler_static_works+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_static_works=no save_LDFLAGS="$LDFLAGS" LDFLAGS="$LDFLAGS $lt_tmp_static_flag" echo "$lt_simple_link_test_code" > conftest.$ac_ext if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then # The linker can only warn and ignore the option if not recognized # So say no if there are warnings if test -s conftest.err; then # Append any errors to the config.log. cat conftest.err 1>&5 $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_static_works=yes fi else lt_cv_prog_compiler_static_works=yes fi fi $RM -r conftest* LDFLAGS="$save_LDFLAGS" fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_static_works" >&5 $as_echo "$lt_cv_prog_compiler_static_works" >&6; } if test x"$lt_cv_prog_compiler_static_works" = xyes; then : else lt_prog_compiler_static= fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -c -o file.$ac_objext" >&5 $as_echo_n "checking if $compiler supports -c -o file.$ac_objext... " >&6; } if ${lt_cv_prog_compiler_c_o+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_c_o=no $RM -r conftest 2>/dev/null mkdir conftest cd conftest mkdir out echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-o out/conftest2.$ac_objext" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then lt_cv_prog_compiler_c_o=yes fi fi chmod u+w . 2>&5 $RM conftest* # SGI C++ compiler will create directory out/ii_files/ for # template instantiation test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files $RM out/* && rmdir out cd .. $RM -r conftest $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_c_o" >&5 $as_echo "$lt_cv_prog_compiler_c_o" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -c -o file.$ac_objext" >&5 $as_echo_n "checking if $compiler supports -c -o file.$ac_objext... " >&6; } if ${lt_cv_prog_compiler_c_o+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler_c_o=no $RM -r conftest 2>/dev/null mkdir conftest cd conftest mkdir out echo "$lt_simple_compile_test_code" > conftest.$ac_ext lt_compiler_flag="-o out/conftest2.$ac_objext" # Insert the option either (1) after the last *FLAGS variable, or # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 echo "$as_me:$LINENO: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then lt_cv_prog_compiler_c_o=yes fi fi chmod u+w . 2>&5 $RM conftest* # SGI C++ compiler will create directory out/ii_files/ for # template instantiation test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files $RM out/* && rmdir out cd .. $RM -r conftest $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_c_o" >&5 $as_echo "$lt_cv_prog_compiler_c_o" >&6; } hard_links="nottested" if test "$lt_cv_prog_compiler_c_o" = no && test "$need_locks" != no; then # do not overwrite the value of need_locks provided by the user { $as_echo "$as_me:${as_lineno-$LINENO}: checking if we can lock with hard links" >&5 $as_echo_n "checking if we can lock with hard links... " >&6; } hard_links=yes $RM conftest* ln conftest.a conftest.b 2>/dev/null && hard_links=no touch conftest.a ln conftest.a conftest.b 2>&5 || hard_links=no ln conftest.a conftest.b 2>/dev/null && hard_links=no { $as_echo "$as_me:${as_lineno-$LINENO}: result: $hard_links" >&5 $as_echo "$hard_links" >&6; } if test "$hard_links" = no; then { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: \`$CC' does not support \`-c -o', so \`make -j' may be unsafe" >&5 $as_echo "$as_me: WARNING: \`$CC' does not support \`-c -o', so \`make -j' may be unsafe" >&2;} need_locks=warn fi else need_locks=no fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the $compiler linker ($LD) supports shared libraries" >&5 $as_echo_n "checking whether the $compiler linker ($LD) supports shared libraries... " >&6; } runpath_var= allow_undefined_flag= always_export_symbols=no archive_cmds= archive_expsym_cmds= compiler_needs_object=no enable_shared_with_static_runtimes=no export_dynamic_flag_spec= export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' hardcode_automatic=no hardcode_direct=no hardcode_direct_absolute=no hardcode_libdir_flag_spec= hardcode_libdir_separator= hardcode_minus_L=no hardcode_shlibpath_var=unsupported inherit_rpath=no link_all_deplibs=unknown module_cmds= module_expsym_cmds= old_archive_from_new_cmds= old_archive_from_expsyms_cmds= thread_safe_flag_spec= whole_archive_flag_spec= # include_expsyms should be a list of space-separated symbols to be *always* # included in the symbol list include_expsyms= # exclude_expsyms can be an extended regexp of symbols to exclude # it will be wrapped by ` (' and `)$', so one must not match beginning or # end of line. Example: `a|bc|.*d.*' will exclude the symbols `a' and `bc', # as well as any symbol that contains `d'. exclude_expsyms='_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*' # Although _GLOBAL_OFFSET_TABLE_ is a valid symbol C name, most a.out # platforms (ab)use it in PIC code, but their linkers get confused if # the symbol is explicitly referenced. Since portable code cannot # rely on this symbol name, it's probably fine to never include it in # preloaded symbol tables. # Exclude shared library initialization/finalization symbols. extract_expsyms_cmds= case $host_os in cygwin* | mingw* | pw32* | cegcc*) # FIXME: the MSVC++ port hasn't been tested in a loooong time # When not using gcc, we currently assume that we are using # Microsoft Visual C++. if test "$GCC" != yes; then with_gnu_ld=no fi ;; interix*) # we just hope/assume this is gcc and not c89 (= MSVC++) with_gnu_ld=yes ;; openbsd*) with_gnu_ld=no ;; esac ld_shlibs=yes # On some targets, GNU ld is compatible enough with the native linker # that we're better off using the native interface for both. lt_use_gnu_ld_interface=no if test "$with_gnu_ld" = yes; then case $host_os in aix*) # The AIX port of GNU ld has always aspired to compatibility # with the native linker. However, as the warning in the GNU ld # block says, versions before 2.19.5* couldn't really create working # shared libraries, regardless of the interface used. case `$LD -v 2>&1` in *\ \(GNU\ Binutils\)\ 2.19.5*) ;; *\ \(GNU\ Binutils\)\ 2.[2-9]*) ;; *\ \(GNU\ Binutils\)\ [3-9]*) ;; *) lt_use_gnu_ld_interface=yes ;; esac ;; *) lt_use_gnu_ld_interface=yes ;; esac fi if test "$lt_use_gnu_ld_interface" = yes; then # If archive_cmds runs LD, not CC, wlarc should be empty wlarc='${wl}' # Set some defaults for GNU ld with shared library support. These # are reset later if shared libraries are not supported. Putting them # here allows them to be overridden if necessary. runpath_var=LD_RUN_PATH hardcode_libdir_flag_spec='${wl}-rpath ${wl}$libdir' export_dynamic_flag_spec='${wl}--export-dynamic' # ancient GNU ld didn't support --whole-archive et. al. if $LD --help 2>&1 | $GREP 'no-whole-archive' > /dev/null; then whole_archive_flag_spec="$wlarc"'--whole-archive$convenience '"$wlarc"'--no-whole-archive' else whole_archive_flag_spec= fi supports_anon_versioning=no case `$LD -v 2>&1` in *GNU\ gold*) supports_anon_versioning=yes ;; *\ [01].* | *\ 2.[0-9].* | *\ 2.10.*) ;; # catch versions < 2.11 *\ 2.11.93.0.2\ *) supports_anon_versioning=yes ;; # RH7.3 ... *\ 2.11.92.0.12\ *) supports_anon_versioning=yes ;; # Mandrake 8.2 ... *\ 2.11.*) ;; # other 2.11 versions *) supports_anon_versioning=yes ;; esac # See if GNU ld supports shared libraries. case $host_os in aix[3-9]*) # On AIX/PPC, the GNU linker is very broken if test "$host_cpu" != ia64; then ld_shlibs=no cat <<_LT_EOF 1>&2 *** Warning: the GNU linker, at least up to release 2.19, is reported *** to be unable to reliably create shared libraries on AIX. *** Therefore, libtool is disabling shared libraries support. If you *** really care for shared libraries, you may want to install binutils *** 2.20 or above, or modify your PATH so that a non-GNU linker is found. *** You will then need to restart the configuration process. _LT_EOF fi ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' archive_expsym_cmds='' ;; m68k) archive_cmds='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' hardcode_libdir_flag_spec='-L$libdir' hardcode_minus_L=yes ;; esac ;; beos*) if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then allow_undefined_flag=unsupported # Joseph Beckenbach says some releases of gcc # support --undefined. This deserves some investigation. FIXME archive_cmds='$CC -nostart $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' else ld_shlibs=no fi ;; cygwin* | mingw* | pw32* | cegcc*) # _LT_TAGVAR(hardcode_libdir_flag_spec, ) is actually meaningless, # as there is no search path for DLLs. hardcode_libdir_flag_spec='-L$libdir' export_dynamic_flag_spec='${wl}--export-all-symbols' allow_undefined_flag=unsupported always_export_symbols=no enable_shared_with_static_runtimes=yes export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[BCDGRS][ ]/s/.*[ ]\([^ ]*\)/\1 DATA/;s/^.*[ ]__nm__\([^ ]*\)[ ][^ ]*/\1 DATA/;/^I[ ]/d;/^[AITW][ ]/s/.* //'\'' | sort | uniq > $export_symbols' exclude_expsyms='[_]+GLOBAL_OFFSET_TABLE_|[_]+GLOBAL__[FID]_.*|[_]+head_[A-Za-z0-9_]+_dll|[A-Za-z0-9_]+_dll_iname' if $LD --help 2>&1 | $GREP 'auto-import' > /dev/null; then archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags -o $output_objdir/$soname ${wl}--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' # If the export-symbols file already is a .def file (1st line # is EXPORTS), use it as is; otherwise, prepend... archive_expsym_cmds='if test "x`$SED 1q $export_symbols`" = xEXPORTS; then cp $export_symbols $output_objdir/$soname.def; else echo EXPORTS > $output_objdir/$soname.def; cat $export_symbols >> $output_objdir/$soname.def; fi~ $CC -shared $output_objdir/$soname.def $libobjs $deplibs $compiler_flags -o $output_objdir/$soname ${wl}--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' else ld_shlibs=no fi ;; haiku*) archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' link_all_deplibs=yes ;; interix[3-9]*) hardcode_direct=no hardcode_shlibpath_var=no hardcode_libdir_flag_spec='${wl}-rpath,$libdir' export_dynamic_flag_spec='${wl}-E' # Hack: On Interix 3.x, we cannot compile PIC because of a broken gcc. # Instead, shared libraries are loaded at an image base (0x10000000 by # default) and relocated if they conflict, which is a slow very memory # consuming and fragmenting process. To avoid this, we pick a random, # 256 KiB-aligned image base between 0x50000000 and 0x6FFC0000 at link # time. Moving up from 0x10000000 also allows more sbrk(2) space. archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-h,$soname ${wl}--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' archive_expsym_cmds='sed "s,^,_," $export_symbols >$output_objdir/$soname.expsym~$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-h,$soname ${wl}--retain-symbols-file,$output_objdir/$soname.expsym ${wl}--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' ;; gnu* | linux* | tpf* | k*bsd*-gnu | kopensolaris*-gnu) tmp_diet=no if test "$host_os" = linux-dietlibc; then case $cc_basename in diet\ *) tmp_diet=yes;; # linux-dietlibc with static linking (!diet-dyn) esac fi if $LD --help 2>&1 | $EGREP ': supported targets:.* elf' > /dev/null \ && test "$tmp_diet" = no then tmp_addflag=' $pic_flag' tmp_sharedflag='-shared' case $cc_basename,$host_cpu in pgcc*) # Portland Group C compiler whole_archive_flag_spec='${wl}--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` ${wl}--no-whole-archive' tmp_addflag=' $pic_flag' ;; pgf77* | pgf90* | pgf95* | pgfortran*) # Portland Group f77 and f90 compilers whole_archive_flag_spec='${wl}--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` ${wl}--no-whole-archive' tmp_addflag=' $pic_flag -Mnomain' ;; ecc*,ia64* | icc*,ia64*) # Intel C compiler on ia64 tmp_addflag=' -i_dynamic' ;; efc*,ia64* | ifort*,ia64*) # Intel Fortran compiler on ia64 tmp_addflag=' -i_dynamic -nofor_main' ;; ifc* | ifort*) # Intel Fortran compiler tmp_addflag=' -nofor_main' ;; lf95*) # Lahey Fortran 8.1 whole_archive_flag_spec= tmp_sharedflag='--shared' ;; xl[cC]* | bgxl[cC]* | mpixl[cC]*) # IBM XL C 8.0 on PPC (deal with xlf below) tmp_sharedflag='-qmkshrobj' tmp_addflag= ;; nvcc*) # Cuda Compiler Driver 2.2 whole_archive_flag_spec='${wl}--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` ${wl}--no-whole-archive' compiler_needs_object=yes ;; esac case `$CC -V 2>&1 | sed 5q` in *Sun\ C*) # Sun C 5.9 whole_archive_flag_spec='${wl}--whole-archive`new_convenience=; for conv in $convenience\"\"; do test -z \"$conv\" || new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` ${wl}--no-whole-archive' compiler_needs_object=yes tmp_sharedflag='-G' ;; *Sun\ F*) # Sun Fortran 8.3 tmp_sharedflag='-G' ;; esac archive_cmds='$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' if test "x$supports_anon_versioning" = xyes; then archive_expsym_cmds='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-version-script ${wl}$output_objdir/$libname.ver -o $lib' fi case $cc_basename in xlf* | bgf* | bgxlf* | mpixlf*) # IBM XL Fortran 10.1 on PPC cannot create shared libs itself whole_archive_flag_spec='--whole-archive$convenience --no-whole-archive' hardcode_libdir_flag_spec='${wl}-rpath ${wl}$libdir' archive_cmds='$LD -shared $libobjs $deplibs $linker_flags -soname $soname -o $lib' if test "x$supports_anon_versioning" = xyes; then archive_expsym_cmds='echo "{ global:" > $output_objdir/$libname.ver~ cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ echo "local: *; };" >> $output_objdir/$libname.ver~ $LD -shared $libobjs $deplibs $linker_flags -soname $soname -version-script $output_objdir/$libname.ver -o $lib' fi ;; esac else ld_shlibs=no fi ;; netbsd*) if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then archive_cmds='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib' wlarc= else archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-retain-symbols-file $wl$export_symbols -o $lib' fi ;; solaris*) if $LD -v 2>&1 | $GREP 'BFD 2\.8' > /dev/null; then ld_shlibs=no cat <<_LT_EOF 1>&2 *** Warning: The releases 2.8.* of the GNU linker cannot reliably *** create shared libraries on Solaris systems. Therefore, libtool *** is disabling shared libraries support. We urge you to upgrade GNU *** binutils to release 2.9.1 or newer. Another option is to modify *** your PATH or compiler configuration so that the native linker is *** used, and then restart. _LT_EOF elif $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-retain-symbols-file $wl$export_symbols -o $lib' else ld_shlibs=no fi ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX*) case `$LD -v 2>&1` in *\ [01].* | *\ 2.[0-9].* | *\ 2.1[0-5].*) ld_shlibs=no cat <<_LT_EOF 1>&2 *** Warning: Releases of the GNU linker prior to 2.16.91.0.3 can not *** reliably create shared libraries on SCO systems. Therefore, libtool *** is disabling shared libraries support. We urge you to upgrade GNU *** binutils to release 2.16.91.0.3 or newer. Another option is to modify *** your PATH or compiler configuration so that the native linker is *** used, and then restart. _LT_EOF ;; *) # For security reasons, it is highly recommended that you always # use absolute paths for naming shared libraries, and exclude the # DT_RUNPATH tag from executables and libraries. But doing so # requires that you compile everything twice, which is a pain. if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then hardcode_libdir_flag_spec='${wl}-rpath ${wl}$libdir' archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' archive_expsym_cmds='$CC -shared $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-retain-symbols-file $wl$export_symbols -o $lib' else ld_shlibs=no fi ;; esac ;; sunos4*) archive_cmds='$LD -assert pure-text -Bshareable -o $lib $libobjs $deplibs $linker_flags' wlarc= hardcode_direct=yes hardcode_shlibpath_var=no ;; *) if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname ${wl}-retain-symbols-file $wl$export_symbols -o $lib' else ld_shlibs=no fi ;; esac if test "$ld_shlibs" = no; then runpath_var= hardcode_libdir_flag_spec= export_dynamic_flag_spec= whole_archive_flag_spec= fi else # PORTME fill in a description of your system's linker (not GNU ld) case $host_os in aix3*) allow_undefined_flag=unsupported always_export_symbols=yes archive_expsym_cmds='$LD -o $output_objdir/$soname $libobjs $deplibs $linker_flags -bE:$export_symbols -T512 -H512 -bM:SRE~$AR $AR_FLAGS $lib $output_objdir/$soname' # Note: this linker hardcodes the directories in LIBPATH if there # are no directories specified by -L. hardcode_minus_L=yes if test "$GCC" = yes && test -z "$lt_prog_compiler_static"; then # Neither direct hardcoding nor static linking is supported with a # broken collect2. hardcode_direct=unsupported fi ;; aix[4-9]*) if test "$host_cpu" = ia64; then # On IA64, the linker does run time linking by default, so we don't # have to do anything special. aix_use_runtimelinking=no exp_sym_flag='-Bexport' no_entry_flag="" else # If we're using GNU nm, then we don't want the "-C" option. # -C means demangle to AIX nm, but means don't demangle with GNU nm # Also, AIX nm treats weak defined symbols like other global # defined symbols, whereas GNU nm marks them as "W". if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then export_symbols_cmds='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && (substr(\$ 3,1,1) != ".")) { print \$ 3 } }'\'' | sort -u > $export_symbols' else export_symbols_cmds='$NM -BCpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B")) && (substr(\$ 3,1,1) != ".")) { print \$ 3 } }'\'' | sort -u > $export_symbols' fi aix_use_runtimelinking=no # Test if we are trying to use run time linking or normal # AIX style linking. If -brtl is somewhere in LDFLAGS, we # need to do runtime linking. case $host_os in aix4.[23]|aix4.[23].*|aix[5-9]*) for ld_flag in $LDFLAGS; do if (test $ld_flag = "-brtl" || test $ld_flag = "-Wl,-brtl"); then aix_use_runtimelinking=yes break fi done ;; esac exp_sym_flag='-bexport' no_entry_flag='-bnoentry' fi # When large executables or shared objects are built, AIX ld can # have problems creating the table of contents. If linking a library # or program results in "error TOC overflow" add -mminimal-toc to # CXXFLAGS/CFLAGS for g++/gcc. In the cases where that is not # enough to fix the problem, add -Wl,-bbigtoc to LDFLAGS. archive_cmds='' hardcode_direct=yes hardcode_direct_absolute=yes hardcode_libdir_separator=':' link_all_deplibs=yes file_list_spec='${wl}-f,' if test "$GCC" = yes; then case $host_os in aix4.[012]|aix4.[012].*) # We only want to do this on AIX 4.2 and lower, the check # below for broken collect2 doesn't work under 4.3+ collect2name=`${CC} -print-prog-name=collect2` if test -f "$collect2name" && strings "$collect2name" | $GREP resolve_lib_name >/dev/null then # We have reworked collect2 : else # We have old collect2 hardcode_direct=unsupported # It fails to find uninstalled libraries when the uninstalled # path is not listed in the libpath. Setting hardcode_minus_L # to unsupported forces relinking hardcode_minus_L=yes hardcode_libdir_flag_spec='-L$libdir' hardcode_libdir_separator= fi ;; esac shared_flag='-shared' if test "$aix_use_runtimelinking" = yes; then shared_flag="$shared_flag "'${wl}-G' fi else # not using gcc if test "$host_cpu" = ia64; then # VisualAge C++, Version 5.5 for AIX 5L for IA-64, Beta 3 Release # chokes on -Wl,-G. The following line is correct: shared_flag='-G' else if test "$aix_use_runtimelinking" = yes; then shared_flag='${wl}-G' else shared_flag='${wl}-bM:SRE' fi fi fi export_dynamic_flag_spec='${wl}-bexpall' # It seems that -bexpall does not export symbols beginning with # underscore (_), so it is better to generate a list of symbols to export. always_export_symbols=yes if test "$aix_use_runtimelinking" = yes; then # Warning - without using the other runtime loading flags (-brtl), # -berok will link without error, but may produce a broken library. allow_undefined_flag='-berok' # Determine the default libpath from the value encoded in an # empty executable. if test "${lt_cv_aix_libpath+set}" = set; then aix_libpath=$lt_cv_aix_libpath else if ${lt_cv_aix_libpath_+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_aix_libpath_sed=' /Import File Strings/,/^$/ { /^0/ { s/^0 *\([^ ]*\) *$/\1/ p } }' lt_cv_aix_libpath_=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` # Check for a 64-bit object if we didn't find anything. if test -z "$lt_cv_aix_libpath_"; then lt_cv_aix_libpath_=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test -z "$lt_cv_aix_libpath_"; then lt_cv_aix_libpath_="/usr/lib:/lib" fi fi aix_libpath=$lt_cv_aix_libpath_ fi hardcode_libdir_flag_spec='${wl}-blibpath:$libdir:'"$aix_libpath" archive_expsym_cmds='$CC -o $output_objdir/$soname $libobjs $deplibs '"\${wl}$no_entry_flag"' $compiler_flags `if test "x${allow_undefined_flag}" != "x"; then func_echo_all "${wl}${allow_undefined_flag}"; else :; fi` '"\${wl}$exp_sym_flag:\$export_symbols $shared_flag" else if test "$host_cpu" = ia64; then hardcode_libdir_flag_spec='${wl}-R $libdir:/usr/lib:/lib' allow_undefined_flag="-z nodefs" archive_expsym_cmds="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs '"\${wl}$no_entry_flag"' $compiler_flags ${wl}${allow_undefined_flag} '"\${wl}$exp_sym_flag:\$export_symbols" else # Determine the default libpath from the value encoded in an # empty executable. if test "${lt_cv_aix_libpath+set}" = set; then aix_libpath=$lt_cv_aix_libpath else if ${lt_cv_aix_libpath_+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_aix_libpath_sed=' /Import File Strings/,/^$/ { /^0/ { s/^0 *\([^ ]*\) *$/\1/ p } }' lt_cv_aix_libpath_=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` # Check for a 64-bit object if we didn't find anything. if test -z "$lt_cv_aix_libpath_"; then lt_cv_aix_libpath_=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext if test -z "$lt_cv_aix_libpath_"; then lt_cv_aix_libpath_="/usr/lib:/lib" fi fi aix_libpath=$lt_cv_aix_libpath_ fi hardcode_libdir_flag_spec='${wl}-blibpath:$libdir:'"$aix_libpath" # Warning - without using the other run time loading flags, # -berok will link without error, but may produce a broken library. no_undefined_flag=' ${wl}-bernotok' allow_undefined_flag=' ${wl}-berok' if test "$with_gnu_ld" = yes; then # We only use this code for GNU lds that support --whole-archive. whole_archive_flag_spec='${wl}--whole-archive$convenience ${wl}--no-whole-archive' else # Exported symbols can be pulled into shared objects from archives whole_archive_flag_spec='$convenience' fi archive_cmds_need_lc=yes # This is similar to how AIX traditionally builds its shared libraries. archive_expsym_cmds="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs ${wl}-bnoentry $compiler_flags ${wl}-bE:$export_symbols${allow_undefined_flag}~$AR $AR_FLAGS $output_objdir/$libname$release.a $output_objdir/$soname' fi fi ;; amigaos*) case $host_cpu in powerpc) # see comment about AmigaOS4 .so support archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags ${wl}-soname $wl$soname -o $lib' archive_expsym_cmds='' ;; m68k) archive_cmds='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' hardcode_libdir_flag_spec='-L$libdir' hardcode_minus_L=yes ;; esac ;; bsdi[45]*) export_dynamic_flag_spec=-rdynamic ;; cygwin* | mingw* | pw32* | cegcc*) # When not using gcc, we currently assume that we are using # Microsoft Visual C++. # hardcode_libdir_flag_spec is actually meaningless, as there is # no search path for DLLs. case $cc_basename in cl*) # Native MSVC hardcode_libdir_flag_spec=' ' allow_undefined_flag=unsupported always_export_symbols=yes file_list_spec='@' # Tell ltmain to make .lib files, not .a files. libext=lib # Tell ltmain to make .dll files, not .so files. shrext_cmds=".dll" # FIXME: Setting linknames here is a bad hack. archive_cmds='$CC -o $output_objdir/$soname $libobjs $compiler_flags $deplibs -Wl,-dll~linknames=' archive_expsym_cmds='if test "x`$SED 1q $export_symbols`" = xEXPORTS; then sed -n -e 's/\\\\\\\(.*\\\\\\\)/-link\\\ -EXPORT:\\\\\\\1/' -e '1\\\!p' < $export_symbols > $output_objdir/$soname.exp; else sed -e 's/\\\\\\\(.*\\\\\\\)/-link\\\ -EXPORT:\\\\\\\1/' < $export_symbols > $output_objdir/$soname.exp; fi~ $CC -o $tool_output_objdir$soname $libobjs $compiler_flags $deplibs "@$tool_output_objdir$soname.exp" -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~ linknames=' # The linker will not automatically build a static lib if we build a DLL. # _LT_TAGVAR(old_archive_from_new_cmds, )='true' enable_shared_with_static_runtimes=yes exclude_expsyms='_NULL_IMPORT_DESCRIPTOR|_IMPORT_DESCRIPTOR_.*' export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[BCDGRS][ ]/s/.*[ ]\([^ ]*\)/\1,DATA/'\'' | $SED -e '\''/^[AITW][ ]/s/.*[ ]//'\'' | sort | uniq > $export_symbols' # Don't use ranlib old_postinstall_cmds='chmod 644 $oldlib' postlink_cmds='lt_outputfile="@OUTPUT@"~ lt_tool_outputfile="@TOOL_OUTPUT@"~ case $lt_outputfile in *.exe|*.EXE) ;; *) lt_outputfile="$lt_outputfile.exe" lt_tool_outputfile="$lt_tool_outputfile.exe" ;; esac~ if test "$MANIFEST_TOOL" != ":" && test -f "$lt_outputfile.manifest"; then $MANIFEST_TOOL -manifest "$lt_tool_outputfile.manifest" -outputresource:"$lt_tool_outputfile" || exit 1; $RM "$lt_outputfile.manifest"; fi' ;; *) # Assume MSVC wrapper hardcode_libdir_flag_spec=' ' allow_undefined_flag=unsupported # Tell ltmain to make .lib files, not .a files. libext=lib # Tell ltmain to make .dll files, not .so files. shrext_cmds=".dll" # FIXME: Setting linknames here is a bad hack. archive_cmds='$CC -o $lib $libobjs $compiler_flags `func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link -dll~linknames=' # The linker will automatically build a .lib file if we build a DLL. old_archive_from_new_cmds='true' # FIXME: Should let the user specify the lib program. old_archive_cmds='lib -OUT:$oldlib$oldobjs$old_deplibs' enable_shared_with_static_runtimes=yes ;; esac ;; darwin* | rhapsody*) archive_cmds_need_lc=no hardcode_direct=no hardcode_automatic=yes hardcode_shlibpath_var=unsupported if test "$lt_cv_ld_force_load" = "yes"; then whole_archive_flag_spec='`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience ${wl}-force_load,$conv\"; done; func_echo_all \"$new_convenience\"`' else whole_archive_flag_spec='' fi link_all_deplibs=yes allow_undefined_flag="$_lt_dar_allow_undefined" case $cc_basename in ifort*) _lt_dar_can_shared=yes ;; *) _lt_dar_can_shared=$GCC ;; esac if test "$_lt_dar_can_shared" = "yes"; then output_verbose_link_cmd=func_echo_all archive_cmds="\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod${_lt_dsymutil}" module_cmds="\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags${_lt_dsymutil}" archive_expsym_cmds="sed 's,^,_,' < \$export_symbols > \$output_objdir/\${libname}-symbols.expsym~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring ${_lt_dar_single_mod}${_lt_dar_export_syms}${_lt_dsymutil}" module_expsym_cmds="sed -e 's,^,_,' < \$export_symbols > \$output_objdir/\${libname}-symbols.expsym~\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags${_lt_dar_export_syms}${_lt_dsymutil}" else ld_shlibs=no fi ;; dgux*) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_libdir_flag_spec='-L$libdir' hardcode_shlibpath_var=no ;; # FreeBSD 2.2.[012] allows us to include c++rt0.o to get C++ constructor # support. Future versions do this automatically, but an explicit c++rt0.o # does not break anything, and helps significantly (at the cost of a little # extra space). freebsd2.2*) archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags /usr/lib/c++rt0.o' hardcode_libdir_flag_spec='-R$libdir' hardcode_direct=yes hardcode_shlibpath_var=no ;; # Unfortunately, older versions of FreeBSD 2 do not have this feature. freebsd2.*) archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' hardcode_direct=yes hardcode_minus_L=yes hardcode_shlibpath_var=no ;; # FreeBSD 3 and greater uses gcc -shared to do shared libraries. freebsd* | dragonfly*) archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' hardcode_libdir_flag_spec='-R$libdir' hardcode_direct=yes hardcode_shlibpath_var=no ;; hpux9*) if test "$GCC" = yes; then archive_cmds='$RM $output_objdir/$soname~$CC -shared $pic_flag ${wl}+b ${wl}$install_libdir -o $output_objdir/$soname $libobjs $deplibs $compiler_flags~test $output_objdir/$soname = $lib || mv $output_objdir/$soname $lib' else archive_cmds='$RM $output_objdir/$soname~$LD -b +b $install_libdir -o $output_objdir/$soname $libobjs $deplibs $linker_flags~test $output_objdir/$soname = $lib || mv $output_objdir/$soname $lib' fi hardcode_libdir_flag_spec='${wl}+b ${wl}$libdir' hardcode_libdir_separator=: hardcode_direct=yes # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. hardcode_minus_L=yes export_dynamic_flag_spec='${wl}-E' ;; hpux10*) if test "$GCC" = yes && test "$with_gnu_ld" = no; then archive_cmds='$CC -shared $pic_flag ${wl}+h ${wl}$soname ${wl}+b ${wl}$install_libdir -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags' fi if test "$with_gnu_ld" = no; then hardcode_libdir_flag_spec='${wl}+b ${wl}$libdir' hardcode_libdir_separator=: hardcode_direct=yes hardcode_direct_absolute=yes export_dynamic_flag_spec='${wl}-E' # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. hardcode_minus_L=yes fi ;; hpux11*) if test "$GCC" = yes && test "$with_gnu_ld" = no; then case $host_cpu in hppa*64*) archive_cmds='$CC -shared ${wl}+h ${wl}$soname -o $lib $libobjs $deplibs $compiler_flags' ;; ia64*) archive_cmds='$CC -shared $pic_flag ${wl}+h ${wl}$soname ${wl}+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' ;; *) archive_cmds='$CC -shared $pic_flag ${wl}+h ${wl}$soname ${wl}+b ${wl}$install_libdir -o $lib $libobjs $deplibs $compiler_flags' ;; esac else case $host_cpu in hppa*64*) archive_cmds='$CC -b ${wl}+h ${wl}$soname -o $lib $libobjs $deplibs $compiler_flags' ;; ia64*) archive_cmds='$CC -b ${wl}+h ${wl}$soname ${wl}+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' ;; *) # Older versions of the 11.00 compiler do not understand -b yet # (HP92453-01 A.11.01.20 doesn't, HP92453-01 B.11.X.35175-35176.GP does) { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $CC understands -b" >&5 $as_echo_n "checking if $CC understands -b... " >&6; } if ${lt_cv_prog_compiler__b+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_prog_compiler__b=no save_LDFLAGS="$LDFLAGS" LDFLAGS="$LDFLAGS -b" echo "$lt_simple_link_test_code" > conftest.$ac_ext if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then # The linker can only warn and ignore the option if not recognized # So say no if there are warnings if test -s conftest.err; then # Append any errors to the config.log. cat conftest.err 1>&5 $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler__b=yes fi else lt_cv_prog_compiler__b=yes fi fi $RM -r conftest* LDFLAGS="$save_LDFLAGS" fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler__b" >&5 $as_echo "$lt_cv_prog_compiler__b" >&6; } if test x"$lt_cv_prog_compiler__b" = xyes; then archive_cmds='$CC -b ${wl}+h ${wl}$soname ${wl}+b ${wl}$install_libdir -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags' fi ;; esac fi if test "$with_gnu_ld" = no; then hardcode_libdir_flag_spec='${wl}+b ${wl}$libdir' hardcode_libdir_separator=: case $host_cpu in hppa*64*|ia64*) hardcode_direct=no hardcode_shlibpath_var=no ;; *) hardcode_direct=yes hardcode_direct_absolute=yes export_dynamic_flag_spec='${wl}-E' # hardcode_minus_L: Not really in the search PATH, # but as the default location of the library. hardcode_minus_L=yes ;; esac fi ;; irix5* | irix6* | nonstopux*) if test "$GCC" = yes; then archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-soname ${wl}$soname `test -n "$verstring" && func_echo_all "${wl}-set_version ${wl}$verstring"` ${wl}-update_registry ${wl}${output_objdir}/so_locations -o $lib' # Try to use the -exported_symbol ld option, if it does not # work, assume that -exports_file does not work either and # implicitly export all symbols. # This should be the same for all languages, so no per-tag cache variable. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the $host_os linker accepts -exported_symbol" >&5 $as_echo_n "checking whether the $host_os linker accepts -exported_symbol... " >&6; } if ${lt_cv_irix_exported_symbol+:} false; then : $as_echo_n "(cached) " >&6 else save_LDFLAGS="$LDFLAGS" LDFLAGS="$LDFLAGS -shared ${wl}-exported_symbol ${wl}foo ${wl}-update_registry ${wl}/dev/null" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int foo (void) { return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : lt_cv_irix_exported_symbol=yes else lt_cv_irix_exported_symbol=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LDFLAGS="$save_LDFLAGS" fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_irix_exported_symbol" >&5 $as_echo "$lt_cv_irix_exported_symbol" >&6; } if test "$lt_cv_irix_exported_symbol" = yes; then archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags ${wl}-soname ${wl}$soname `test -n "$verstring" && func_echo_all "${wl}-set_version ${wl}$verstring"` ${wl}-update_registry ${wl}${output_objdir}/so_locations ${wl}-exports_file ${wl}$export_symbols -o $lib' fi else archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry ${output_objdir}/so_locations -o $lib' archive_expsym_cmds='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry ${output_objdir}/so_locations -exports_file $export_symbols -o $lib' fi archive_cmds_need_lc='no' hardcode_libdir_flag_spec='${wl}-rpath ${wl}$libdir' hardcode_libdir_separator=: inherit_rpath=yes link_all_deplibs=yes ;; netbsd*) if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out else archive_cmds='$LD -shared -o $lib $libobjs $deplibs $linker_flags' # ELF fi hardcode_libdir_flag_spec='-R$libdir' hardcode_direct=yes hardcode_shlibpath_var=no ;; newsos6) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_direct=yes hardcode_libdir_flag_spec='${wl}-rpath ${wl}$libdir' hardcode_libdir_separator=: hardcode_shlibpath_var=no ;; *nto* | *qnx*) ;; openbsd*) if test -f /usr/libexec/ld.so; then hardcode_direct=yes hardcode_shlibpath_var=no hardcode_direct_absolute=yes if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`" || test "$host_os-$host_cpu" = "openbsd2.8-powerpc"; then archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags ${wl}-retain-symbols-file,$export_symbols' hardcode_libdir_flag_spec='${wl}-rpath,$libdir' export_dynamic_flag_spec='${wl}-E' else case $host_os in openbsd[01].* | openbsd2.[0-7] | openbsd2.[0-7].*) archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' hardcode_libdir_flag_spec='-R$libdir' ;; *) archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' hardcode_libdir_flag_spec='${wl}-rpath,$libdir' ;; esac fi else ld_shlibs=no fi ;; os2*) hardcode_libdir_flag_spec='-L$libdir' hardcode_minus_L=yes allow_undefined_flag=unsupported archive_cmds='$ECHO "LIBRARY $libname INITINSTANCE" > $output_objdir/$libname.def~$ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~echo DATA >> $output_objdir/$libname.def~echo " SINGLE NONSHARED" >> $output_objdir/$libname.def~echo EXPORTS >> $output_objdir/$libname.def~emxexp $libobjs >> $output_objdir/$libname.def~$CC -Zdll -Zcrtdll -o $lib $libobjs $deplibs $compiler_flags $output_objdir/$libname.def' old_archive_from_new_cmds='emximp -o $output_objdir/$libname.a $output_objdir/$libname.def' ;; osf3*) if test "$GCC" = yes; then allow_undefined_flag=' ${wl}-expect_unresolved ${wl}\*' archive_cmds='$CC -shared${allow_undefined_flag} $libobjs $deplibs $compiler_flags ${wl}-soname ${wl}$soname `test -n "$verstring" && func_echo_all "${wl}-set_version ${wl}$verstring"` ${wl}-update_registry ${wl}${output_objdir}/so_locations -o $lib' else allow_undefined_flag=' -expect_unresolved \*' archive_cmds='$CC -shared${allow_undefined_flag} $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry ${output_objdir}/so_locations -o $lib' fi archive_cmds_need_lc='no' hardcode_libdir_flag_spec='${wl}-rpath ${wl}$libdir' hardcode_libdir_separator=: ;; osf4* | osf5*) # as osf3* with the addition of -msym flag if test "$GCC" = yes; then allow_undefined_flag=' ${wl}-expect_unresolved ${wl}\*' archive_cmds='$CC -shared${allow_undefined_flag} $pic_flag $libobjs $deplibs $compiler_flags ${wl}-msym ${wl}-soname ${wl}$soname `test -n "$verstring" && func_echo_all "${wl}-set_version ${wl}$verstring"` ${wl}-update_registry ${wl}${output_objdir}/so_locations -o $lib' hardcode_libdir_flag_spec='${wl}-rpath ${wl}$libdir' else allow_undefined_flag=' -expect_unresolved \*' archive_cmds='$CC -shared${allow_undefined_flag} $libobjs $deplibs $compiler_flags -msym -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry ${output_objdir}/so_locations -o $lib' archive_expsym_cmds='for i in `cat $export_symbols`; do printf "%s %s\\n" -exported_symbol "\$i" >> $lib.exp; done; printf "%s\\n" "-hidden">> $lib.exp~ $CC -shared${allow_undefined_flag} ${wl}-input ${wl}$lib.exp $compiler_flags $libobjs $deplibs -soname $soname `test -n "$verstring" && $ECHO "-set_version $verstring"` -update_registry ${output_objdir}/so_locations -o $lib~$RM $lib.exp' # Both c and cxx compiler support -rpath directly hardcode_libdir_flag_spec='-rpath $libdir' fi archive_cmds_need_lc='no' hardcode_libdir_separator=: ;; solaris*) no_undefined_flag=' -z defs' if test "$GCC" = yes; then wlarc='${wl}' archive_cmds='$CC -shared $pic_flag ${wl}-z ${wl}text ${wl}-h ${wl}$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -shared $pic_flag ${wl}-z ${wl}text ${wl}-M ${wl}$lib.exp ${wl}-h ${wl}$soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' else case `$CC -V 2>&1` in *"Compilers 5.0"*) wlarc='' archive_cmds='$LD -G${allow_undefined_flag} -h $soname -o $lib $libobjs $deplibs $linker_flags' archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $LD -G${allow_undefined_flag} -M $lib.exp -h $soname -o $lib $libobjs $deplibs $linker_flags~$RM $lib.exp' ;; *) wlarc='${wl}' archive_cmds='$CC -G${allow_undefined_flag} -h $soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ $CC -G${allow_undefined_flag} -M $lib.exp -h $soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' ;; esac fi hardcode_libdir_flag_spec='-R$libdir' hardcode_shlibpath_var=no case $host_os in solaris2.[0-5] | solaris2.[0-5].*) ;; *) # The compiler driver will combine and reorder linker options, # but understands `-z linker_flag'. GCC discards it without `$wl', # but is careful enough not to reorder. # Supported since Solaris 2.6 (maybe 2.5.1?) if test "$GCC" = yes; then whole_archive_flag_spec='${wl}-z ${wl}allextract$convenience ${wl}-z ${wl}defaultextract' else whole_archive_flag_spec='-z allextract$convenience -z defaultextract' fi ;; esac link_all_deplibs=yes ;; sunos4*) if test "x$host_vendor" = xsequent; then # Use $CC to link under sequent, because it throws in some extra .o # files that make .init and .fini sections work. archive_cmds='$CC -G ${wl}-h $soname -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$LD -assert pure-text -Bstatic -o $lib $libobjs $deplibs $linker_flags' fi hardcode_libdir_flag_spec='-L$libdir' hardcode_direct=yes hardcode_minus_L=yes hardcode_shlibpath_var=no ;; sysv4) case $host_vendor in sni) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_direct=yes # is this really true??? ;; siemens) ## LD is ld it makes a PLAMLIB ## CC just makes a GrossModule. archive_cmds='$LD -G -o $lib $libobjs $deplibs $linker_flags' reload_cmds='$CC -r -o $output$reload_objs' hardcode_direct=no ;; motorola) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_direct=no #Motorola manual says yes, but my tests say they lie ;; esac runpath_var='LD_RUN_PATH' hardcode_shlibpath_var=no ;; sysv4.3*) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_shlibpath_var=no export_dynamic_flag_spec='-Bexport' ;; sysv4*MP*) if test -d /usr/nec; then archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_shlibpath_var=no runpath_var=LD_RUN_PATH hardcode_runpath_var=yes ld_shlibs=yes fi ;; sysv4*uw2* | sysv5OpenUNIX* | sysv5UnixWare7.[01].[10]* | unixware7* | sco3.2v5.0.[024]*) no_undefined_flag='${wl}-z,text' archive_cmds_need_lc=no hardcode_shlibpath_var=no runpath_var='LD_RUN_PATH' if test "$GCC" = yes; then archive_cmds='$CC -shared ${wl}-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -shared ${wl}-Bexport:$export_symbols ${wl}-h,$soname -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$CC -G ${wl}-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -G ${wl}-Bexport:$export_symbols ${wl}-h,$soname -o $lib $libobjs $deplibs $compiler_flags' fi ;; sysv5* | sco3.2v5* | sco5v6*) # Note: We can NOT use -z defs as we might desire, because we do not # link with -lc, and that would cause any symbols used from libc to # always be unresolved, which means just about no library would # ever link correctly. If we're not using GNU ld we use -z text # though, which does catch some bad symbols but isn't as heavy-handed # as -z defs. no_undefined_flag='${wl}-z,text' allow_undefined_flag='${wl}-z,nodefs' archive_cmds_need_lc=no hardcode_shlibpath_var=no hardcode_libdir_flag_spec='${wl}-R,$libdir' hardcode_libdir_separator=':' link_all_deplibs=yes export_dynamic_flag_spec='${wl}-Bexport' runpath_var='LD_RUN_PATH' if test "$GCC" = yes; then archive_cmds='$CC -shared ${wl}-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -shared ${wl}-Bexport:$export_symbols ${wl}-h,$soname -o $lib $libobjs $deplibs $compiler_flags' else archive_cmds='$CC -G ${wl}-h,$soname -o $lib $libobjs $deplibs $compiler_flags' archive_expsym_cmds='$CC -G ${wl}-Bexport:$export_symbols ${wl}-h,$soname -o $lib $libobjs $deplibs $compiler_flags' fi ;; uts4*) archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' hardcode_libdir_flag_spec='-L$libdir' hardcode_shlibpath_var=no ;; *) ld_shlibs=no ;; esac if test x$host_vendor = xsni; then case $host in sysv4 | sysv4.2uw2* | sysv4.3* | sysv5*) export_dynamic_flag_spec='${wl}-Blargedynsym' ;; esac fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ld_shlibs" >&5 $as_echo "$ld_shlibs" >&6; } test "$ld_shlibs" = no && can_build_shared=no with_gnu_ld=$with_gnu_ld # # Do we need to explicitly link libc? # case "x$archive_cmds_need_lc" in x|xyes) # Assume -lc should be added archive_cmds_need_lc=yes if test "$enable_shared" = yes && test "$GCC" = yes; then case $archive_cmds in *'~'*) # FIXME: we may have to deal with multi-command sequences. ;; '$CC '*) # Test whether the compiler implicitly links with -lc since on some # systems, -lgcc has to come before -lc. If gcc already passes -lc # to ld, don't add -lc before -lgcc. { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether -lc should be explicitly linked in" >&5 $as_echo_n "checking whether -lc should be explicitly linked in... " >&6; } if ${lt_cv_archive_cmds_need_lc+:} false; then : $as_echo_n "(cached) " >&6 else $RM conftest* echo "$lt_simple_compile_test_code" > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } 2>conftest.err; then soname=conftest lib=conftest libobjs=conftest.$ac_objext deplibs= wl=$lt_prog_compiler_wl pic_flag=$lt_prog_compiler_pic compiler_flags=-v linker_flags=-v verstring= output_objdir=. libname=conftest lt_save_allow_undefined_flag=$allow_undefined_flag allow_undefined_flag= if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$archive_cmds 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1\""; } >&5 (eval $archive_cmds 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } then lt_cv_archive_cmds_need_lc=no else lt_cv_archive_cmds_need_lc=yes fi allow_undefined_flag=$lt_save_allow_undefined_flag else cat conftest.err 1>&5 fi $RM conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_archive_cmds_need_lc" >&5 $as_echo "$lt_cv_archive_cmds_need_lc" >&6; } archive_cmds_need_lc=$lt_cv_archive_cmds_need_lc ;; esac fi ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: checking dynamic linker characteristics" >&5 $as_echo_n "checking dynamic linker characteristics... " >&6; } if test "$GCC" = yes; then case $host_os in darwin*) lt_awk_arg="/^libraries:/,/LR/" ;; *) lt_awk_arg="/^libraries:/" ;; esac case $host_os in mingw* | cegcc*) lt_sed_strip_eq="s,=\([A-Za-z]:\),\1,g" ;; *) lt_sed_strip_eq="s,=/,/,g" ;; esac lt_search_path_spec=`$CC -print-search-dirs | awk $lt_awk_arg | $SED -e "s/^libraries://" -e $lt_sed_strip_eq` case $lt_search_path_spec in *\;*) # if the path contains ";" then we assume it to be the separator # otherwise default to the standard path separator (i.e. ":") - it is # assumed that no part of a normal pathname contains ";" but that should # okay in the real world where ";" in dirpaths is itself problematic. lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED 's/;/ /g'` ;; *) lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED "s/$PATH_SEPARATOR/ /g"` ;; esac # Ok, now we have the path, separated by spaces, we can step through it # and add multilib dir if necessary. lt_tmp_lt_search_path_spec= lt_multi_os_dir=`$CC $CPPFLAGS $CFLAGS $LDFLAGS -print-multi-os-directory 2>/dev/null` for lt_sys_path in $lt_search_path_spec; do if test -d "$lt_sys_path/$lt_multi_os_dir"; then lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path/$lt_multi_os_dir" else test -d "$lt_sys_path" && \ lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path" fi done lt_search_path_spec=`$ECHO "$lt_tmp_lt_search_path_spec" | awk ' BEGIN {RS=" "; FS="/|\n";} { lt_foo=""; lt_count=0; for (lt_i = NF; lt_i > 0; lt_i--) { if ($lt_i != "" && $lt_i != ".") { if ($lt_i == "..") { lt_count++; } else { if (lt_count == 0) { lt_foo="/" $lt_i lt_foo; } else { lt_count--; } } } } if (lt_foo != "") { lt_freq[lt_foo]++; } if (lt_freq[lt_foo] == 1) { print lt_foo; } }'` # AWK program above erroneously prepends '/' to C:/dos/paths # for these hosts. case $host_os in mingw* | cegcc*) lt_search_path_spec=`$ECHO "$lt_search_path_spec" |\ $SED 's,/\([A-Za-z]:\),\1,g'` ;; esac sys_lib_search_path_spec=`$ECHO "$lt_search_path_spec" | $lt_NL2SP` else sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" fi library_names_spec= libname_spec='lib$name' soname_spec= shrext_cmds=".so" postinstall_cmds= postuninstall_cmds= finish_cmds= finish_eval= shlibpath_var= shlibpath_overrides_runpath=unknown version_type=none dynamic_linker="$host_os ld.so" sys_lib_dlsearch_path_spec="/lib /usr/lib" need_lib_prefix=unknown hardcode_into_libs=no # when you set need_version to no, make sure it does not cause -set_version # flags to be left without arguments need_version=unknown case $host_os in aix3*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='${libname}${release}${shared_ext}$versuffix $libname.a' shlibpath_var=LIBPATH # AIX 3 has no versioning support, so we append a major version to the name. soname_spec='${libname}${release}${shared_ext}$major' ;; aix[4-9]*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no hardcode_into_libs=yes if test "$host_cpu" = ia64; then # AIX 5 supports IA64 library_names_spec='${libname}${release}${shared_ext}$major ${libname}${release}${shared_ext}$versuffix $libname${shared_ext}' shlibpath_var=LD_LIBRARY_PATH else # With GCC up to 2.95.x, collect2 would create an import file # for dependence libraries. The import file would start with # the line `#! .'. This would cause the generated library to # depend on `.', always an invalid library. This was fixed in # development snapshots of GCC prior to 3.0. case $host_os in aix4 | aix4.[01] | aix4.[01].*) if { echo '#if __GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 97)' echo ' yes ' echo '#endif'; } | ${CC} -E - | $GREP yes > /dev/null; then : else can_build_shared=no fi ;; esac # AIX (on Power*) has no versioning support, so currently we can not hardcode correct # soname into executable. Probably we can add versioning support to # collect2, so additional links can be useful in future. if test "$aix_use_runtimelinking" = yes; then # If using run time linking (on AIX 4.2 or later) use lib.so # instead of lib.a to let people know that these are not # typical AIX shared libraries. library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' else # We preserve .a as extension for shared libraries through AIX4.2 # and later when we are not doing run time linking. library_names_spec='${libname}${release}.a $libname.a' soname_spec='${libname}${release}${shared_ext}$major' fi shlibpath_var=LIBPATH fi ;; amigaos*) case $host_cpu in powerpc) # Since July 2007 AmigaOS4 officially supports .so libraries. # When compiling the executable, add -use-dynld -Lsobjs: to the compileline. library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' ;; m68k) library_names_spec='$libname.ixlibrary $libname.a' # Create ${libname}_ixlibrary.a entries in /sys/libs. finish_eval='for lib in `ls $libdir/*.ixlibrary 2>/dev/null`; do libname=`func_echo_all "$lib" | $SED '\''s%^.*/\([^/]*\)\.ixlibrary$%\1%'\''`; test $RM /sys/libs/${libname}_ixlibrary.a; $show "cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a"; cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a || exit 1; done' ;; esac ;; beos*) library_names_spec='${libname}${shared_ext}' dynamic_linker="$host_os ld.so" shlibpath_var=LIBRARY_PATH ;; bsdi[45]*) version_type=linux # correct to gnu/linux during the next big refactor need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' finish_cmds='PATH="\$PATH:/sbin" ldconfig $libdir' shlibpath_var=LD_LIBRARY_PATH sys_lib_search_path_spec="/shlib /usr/lib /usr/X11/lib /usr/contrib/lib /lib /usr/local/lib" sys_lib_dlsearch_path_spec="/shlib /usr/lib /usr/local/lib" # the default ld.so.conf also contains /usr/contrib/lib and # /usr/X11R6/lib (/usr/X11 is a link to /usr/X11R6), but let us allow # libtool to hard-code these into programs ;; cygwin* | mingw* | pw32* | cegcc*) version_type=windows shrext_cmds=".dll" need_version=no need_lib_prefix=no case $GCC,$cc_basename in yes,*) # gcc library_names_spec='$libname.dll.a' # DLL is installed to $(libdir)/../bin by postinstall_cmds postinstall_cmds='base_file=`basename \${file}`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\${base_file}'\''i; echo \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname~ chmod a+x \$dldir/$dlname~ if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; fi' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' shlibpath_overrides_runpath=yes case $host_os in cygwin*) # Cygwin DLLs use 'cyg' prefix rather than 'lib' soname_spec='`echo ${libname} | sed -e 's/^lib/cyg/'``echo ${release} | $SED -e 's/[.]/-/g'`${versuffix}${shared_ext}' sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/lib/w32api" ;; mingw* | cegcc*) # MinGW DLLs use traditional 'lib' prefix soname_spec='${libname}`echo ${release} | $SED -e 's/[.]/-/g'`${versuffix}${shared_ext}' ;; pw32*) # pw32 DLLs use 'pw' prefix rather than 'lib' library_names_spec='`echo ${libname} | sed -e 's/^lib/pw/'``echo ${release} | $SED -e 's/[.]/-/g'`${versuffix}${shared_ext}' ;; esac dynamic_linker='Win32 ld.exe' ;; *,cl*) # Native MSVC libname_spec='$name' soname_spec='${libname}`echo ${release} | $SED -e 's/[.]/-/g'`${versuffix}${shared_ext}' library_names_spec='${libname}.dll.lib' case $build_os in mingw*) sys_lib_search_path_spec= lt_save_ifs=$IFS IFS=';' for lt_path in $LIB do IFS=$lt_save_ifs # Let DOS variable expansion print the short 8.3 style file name. lt_path=`cd "$lt_path" 2>/dev/null && cmd //C "for %i in (".") do @echo %~si"` sys_lib_search_path_spec="$sys_lib_search_path_spec $lt_path" done IFS=$lt_save_ifs # Convert to MSYS style. sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | sed -e 's|\\\\|/|g' -e 's| \\([a-zA-Z]\\):| /\\1|g' -e 's|^ ||'` ;; cygwin*) # Convert to unix form, then to dos form, then back to unix form # but this time dos style (no spaces!) so that the unix form looks # like /cygdrive/c/PROGRA~1:/cygdr... sys_lib_search_path_spec=`cygpath --path --unix "$LIB"` sys_lib_search_path_spec=`cygpath --path --dos "$sys_lib_search_path_spec" 2>/dev/null` sys_lib_search_path_spec=`cygpath --path --unix "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` ;; *) sys_lib_search_path_spec="$LIB" if $ECHO "$sys_lib_search_path_spec" | $GREP ';[c-zC-Z]:/' >/dev/null; then # It is most probably a Windows format PATH. sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e 's/;/ /g'` else sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` fi # FIXME: find the short name or the path components, as spaces are # common. (e.g. "Program Files" -> "PROGRA~1") ;; esac # DLL is installed to $(libdir)/../bin by postinstall_cmds postinstall_cmds='base_file=`basename \${file}`~ dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\${base_file}'\''i; echo \$dlname'\''`~ dldir=$destdir/`dirname \$dlpath`~ test -d \$dldir || mkdir -p \$dldir~ $install_prog $dir/$dlname \$dldir/$dlname' postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ dlpath=$dir/\$dldll~ $RM \$dlpath' shlibpath_overrides_runpath=yes dynamic_linker='Win32 link.exe' ;; *) # Assume MSVC wrapper library_names_spec='${libname}`echo ${release} | $SED -e 's/[.]/-/g'`${versuffix}${shared_ext} $libname.lib' dynamic_linker='Win32 ld.exe' ;; esac # FIXME: first we should search . and the directory the executable is in shlibpath_var=PATH ;; darwin* | rhapsody*) dynamic_linker="$host_os dyld" version_type=darwin need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${major}$shared_ext ${libname}$shared_ext' soname_spec='${libname}${release}${major}$shared_ext' shlibpath_overrides_runpath=yes shlibpath_var=DYLD_LIBRARY_PATH shrext_cmds='`test .$module = .yes && echo .so || echo .dylib`' sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/local/lib" sys_lib_dlsearch_path_spec='/usr/local/lib /lib /usr/lib' ;; dgux*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname$shared_ext' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH ;; freebsd* | dragonfly*) # DragonFly does not have aout. When/if they implement a new # versioning mechanism, adjust this. if test -x /usr/bin/objformat; then objformat=`/usr/bin/objformat` else case $host_os in freebsd[23].*) objformat=aout ;; *) objformat=elf ;; esac fi version_type=freebsd-$objformat case $version_type in freebsd-elf*) library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext} $libname${shared_ext}' need_version=no need_lib_prefix=no ;; freebsd-*) library_names_spec='${libname}${release}${shared_ext}$versuffix $libname${shared_ext}$versuffix' need_version=yes ;; esac shlibpath_var=LD_LIBRARY_PATH case $host_os in freebsd2.*) shlibpath_overrides_runpath=yes ;; freebsd3.[01]* | freebsdelf3.[01]*) shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; freebsd3.[2-9]* | freebsdelf3.[2-9]* | \ freebsd4.[0-5] | freebsdelf4.[0-5] | freebsd4.1.1 | freebsdelf4.1.1) shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; *) # from 4.6 on, and DragonFly shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; esac ;; gnu*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}${major} ${libname}${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; haiku*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no dynamic_linker="$host_os runtime_loader" library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}${major} ${libname}${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LIBRARY_PATH shlibpath_overrides_runpath=yes sys_lib_dlsearch_path_spec='/boot/home/config/lib /boot/common/lib /boot/system/lib' hardcode_into_libs=yes ;; hpux9* | hpux10* | hpux11*) # Give a soname corresponding to the major version so that dld.sl refuses to # link against other versions. version_type=sunos need_lib_prefix=no need_version=no case $host_cpu in ia64*) shrext_cmds='.so' hardcode_into_libs=yes dynamic_linker="$host_os dld.so" shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' if test "X$HPUX_IA64_MODE" = X32; then sys_lib_search_path_spec="/usr/lib/hpux32 /usr/local/lib/hpux32 /usr/local/lib" else sys_lib_search_path_spec="/usr/lib/hpux64 /usr/local/lib/hpux64" fi sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec ;; hppa*64*) shrext_cmds='.sl' hardcode_into_libs=yes dynamic_linker="$host_os dld.sl" shlibpath_var=LD_LIBRARY_PATH # How should we handle SHLIB_PATH shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' sys_lib_search_path_spec="/usr/lib/pa20_64 /usr/ccs/lib/pa20_64" sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec ;; *) shrext_cmds='.sl' dynamic_linker="$host_os dld.sl" shlibpath_var=SHLIB_PATH shlibpath_overrides_runpath=no # +s is required to enable SHLIB_PATH library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' ;; esac # HP-UX runs *really* slowly unless shared libraries are mode 555, ... postinstall_cmds='chmod 555 $lib' # or fails outright, so override atomically: install_override_mode=555 ;; interix[3-9]*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' dynamic_linker='Interix 3.x ld.so.1 (PE, like ELF)' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; irix5* | irix6* | nonstopux*) case $host_os in nonstopux*) version_type=nonstopux ;; *) if test "$lt_cv_prog_gnu_ld" = yes; then version_type=linux # correct to gnu/linux during the next big refactor else version_type=irix fi ;; esac need_lib_prefix=no need_version=no soname_spec='${libname}${release}${shared_ext}$major' library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${release}${shared_ext} $libname${shared_ext}' case $host_os in irix5* | nonstopux*) libsuff= shlibsuff= ;; *) case $LD in # libtool.m4 will add one of these switches to LD *-32|*"-32 "|*-melf32bsmip|*"-melf32bsmip ") libsuff= shlibsuff= libmagic=32-bit;; *-n32|*"-n32 "|*-melf32bmipn32|*"-melf32bmipn32 ") libsuff=32 shlibsuff=N32 libmagic=N32;; *-64|*"-64 "|*-melf64bmip|*"-melf64bmip ") libsuff=64 shlibsuff=64 libmagic=64-bit;; *) libsuff= shlibsuff= libmagic=never-match;; esac ;; esac shlibpath_var=LD_LIBRARY${shlibsuff}_PATH shlibpath_overrides_runpath=no sys_lib_search_path_spec="/usr/lib${libsuff} /lib${libsuff} /usr/local/lib${libsuff}" sys_lib_dlsearch_path_spec="/usr/lib${libsuff} /lib${libsuff}" hardcode_into_libs=yes ;; # No shared lib support for Linux oldld, aout, or coff. linux*oldld* | linux*aout* | linux*coff*) dynamic_linker=no ;; # This must be glibc/ELF. linux* | k*bsd*-gnu | kopensolaris*-gnu) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' finish_cmds='PATH="\$PATH:/sbin" ldconfig -n $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no # Some binutils ld are patched to set DT_RUNPATH if ${lt_cv_shlibpath_overrides_runpath+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_shlibpath_overrides_runpath=no save_LDFLAGS=$LDFLAGS save_libdir=$libdir eval "libdir=/foo; wl=\"$lt_prog_compiler_wl\"; \ LDFLAGS=\"\$LDFLAGS $hardcode_libdir_flag_spec\"" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : if ($OBJDUMP -p conftest$ac_exeext) 2>/dev/null | grep "RUNPATH.*$libdir" >/dev/null; then : lt_cv_shlibpath_overrides_runpath=yes fi fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LDFLAGS=$save_LDFLAGS libdir=$save_libdir fi shlibpath_overrides_runpath=$lt_cv_shlibpath_overrides_runpath # This implies no fast_install, which is unacceptable. # Some rework will be needed to allow for fast_install # before this can be enabled. hardcode_into_libs=yes # Append ld.so.conf contents to the search path if test -f /etc/ld.so.conf; then lt_ld_extra=`awk '/^include / { system(sprintf("cd /etc; cat %s 2>/dev/null", \$2)); skip = 1; } { if (!skip) print \$0; skip = 0; }' < /etc/ld.so.conf | $SED -e 's/#.*//;/^[ ]*hwcap[ ]/d;s/[:, ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;s/"//g;/^$/d' | tr '\n' ' '` sys_lib_dlsearch_path_spec="/lib /usr/lib $lt_ld_extra" fi # We used to test for /lib/ld.so.1 and disable shared libraries on # powerpc, because MkLinux only supported shared libraries with the # GNU dynamic linker. Since this was broken with cross compilers, # most powerpc-linux boxes support dynamic linking these days and # people can always --disable-shared, the test was removed, and we # assume the GNU/Linux dynamic linker is in use. dynamic_linker='GNU/Linux ld.so' ;; netbsd*) version_type=sunos need_lib_prefix=no need_version=no if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${shared_ext}$versuffix' finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' dynamic_linker='NetBSD (a.out) ld.so' else library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major ${libname}${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' dynamic_linker='NetBSD ld.elf_so' fi shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes ;; newsos6) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes ;; *nto* | *qnx*) version_type=qnx need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes dynamic_linker='ldqnx.so' ;; openbsd*) version_type=sunos sys_lib_dlsearch_path_spec="/usr/lib" need_lib_prefix=no # Some older versions of OpenBSD (3.3 at least) *do* need versioned libs. case $host_os in openbsd3.3 | openbsd3.3.*) need_version=yes ;; *) need_version=no ;; esac library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${shared_ext}$versuffix' finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' shlibpath_var=LD_LIBRARY_PATH if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`" || test "$host_os-$host_cpu" = "openbsd2.8-powerpc"; then case $host_os in openbsd2.[89] | openbsd2.[89].*) shlibpath_overrides_runpath=no ;; *) shlibpath_overrides_runpath=yes ;; esac else shlibpath_overrides_runpath=yes fi ;; os2*) libname_spec='$name' shrext_cmds=".dll" need_lib_prefix=no library_names_spec='$libname${shared_ext} $libname.a' dynamic_linker='OS/2 ld.exe' shlibpath_var=LIBPATH ;; osf3* | osf4* | osf5*) version_type=osf need_lib_prefix=no need_version=no soname_spec='${libname}${release}${shared_ext}$major' library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' shlibpath_var=LD_LIBRARY_PATH sys_lib_search_path_spec="/usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /usr/local/lib /var/shlib" sys_lib_dlsearch_path_spec="$sys_lib_search_path_spec" ;; rdos*) dynamic_linker=no ;; solaris*) version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes # ldd complains unless libraries are executable postinstall_cmds='chmod +x $lib' ;; sunos4*) version_type=sunos library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${shared_ext}$versuffix' finish_cmds='PATH="\$PATH:/usr/etc" ldconfig $libdir' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes if test "$with_gnu_ld" = yes; then need_lib_prefix=no fi need_version=yes ;; sysv4 | sysv4.3*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH case $host_vendor in sni) shlibpath_overrides_runpath=no need_lib_prefix=no runpath_var=LD_RUN_PATH ;; siemens) need_lib_prefix=no ;; motorola) need_lib_prefix=no need_version=no shlibpath_overrides_runpath=no sys_lib_search_path_spec='/lib /usr/lib /usr/ccs/lib' ;; esac ;; sysv4*MP*) if test -d /usr/nec ;then version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='$libname${shared_ext}.$versuffix $libname${shared_ext}.$major $libname${shared_ext}' soname_spec='$libname${shared_ext}.$major' shlibpath_var=LD_LIBRARY_PATH fi ;; sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) version_type=freebsd-elf need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext} $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=yes hardcode_into_libs=yes if test "$with_gnu_ld" = yes; then sys_lib_search_path_spec='/usr/local/lib /usr/gnu/lib /usr/ccs/lib /usr/lib /lib' else sys_lib_search_path_spec='/usr/ccs/lib /usr/lib' case $host_os in sco3.2v5*) sys_lib_search_path_spec="$sys_lib_search_path_spec /lib" ;; esac fi sys_lib_dlsearch_path_spec='/usr/lib' ;; tpf*) # TPF is a cross-target only. Preferred cross-host = GNU/Linux. version_type=linux # correct to gnu/linux during the next big refactor need_lib_prefix=no need_version=no library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' shlibpath_var=LD_LIBRARY_PATH shlibpath_overrides_runpath=no hardcode_into_libs=yes ;; uts4*) version_type=linux # correct to gnu/linux during the next big refactor library_names_spec='${libname}${release}${shared_ext}$versuffix ${libname}${release}${shared_ext}$major $libname${shared_ext}' soname_spec='${libname}${release}${shared_ext}$major' shlibpath_var=LD_LIBRARY_PATH ;; *) dynamic_linker=no ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: $dynamic_linker" >&5 $as_echo "$dynamic_linker" >&6; } test "$dynamic_linker" = no && can_build_shared=no variables_saved_for_relink="PATH $shlibpath_var $runpath_var" if test "$GCC" = yes; then variables_saved_for_relink="$variables_saved_for_relink GCC_EXEC_PREFIX COMPILER_PATH LIBRARY_PATH" fi if test "${lt_cv_sys_lib_search_path_spec+set}" = set; then sys_lib_search_path_spec="$lt_cv_sys_lib_search_path_spec" fi if test "${lt_cv_sys_lib_dlsearch_path_spec+set}" = set; then sys_lib_dlsearch_path_spec="$lt_cv_sys_lib_dlsearch_path_spec" fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to hardcode library paths into programs" >&5 $as_echo_n "checking how to hardcode library paths into programs... " >&6; } hardcode_action= if test -n "$hardcode_libdir_flag_spec" || test -n "$runpath_var" || test "X$hardcode_automatic" = "Xyes" ; then # We can hardcode non-existent directories. if test "$hardcode_direct" != no && # If the only mechanism to avoid hardcoding is shlibpath_var, we # have to relink, otherwise we might link with an installed library # when we should be linking with a yet-to-be-installed one ## test "$_LT_TAGVAR(hardcode_shlibpath_var, )" != no && test "$hardcode_minus_L" != no; then # Linking always hardcodes the temporary library directory. hardcode_action=relink else # We can link without hardcoding, and we can hardcode nonexisting dirs. hardcode_action=immediate fi else # We cannot hardcode anything, or else we can only hardcode existing # directories. hardcode_action=unsupported fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $hardcode_action" >&5 $as_echo "$hardcode_action" >&6; } if test "$hardcode_action" = relink || test "$inherit_rpath" = yes; then # Fast installation is not supported enable_fast_install=no elif test "$shlibpath_overrides_runpath" = yes || test "$enable_shared" = no; then # Fast installation is not necessary enable_fast_install=needless fi if test "x$enable_dlopen" != xyes; then enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown else lt_cv_dlopen=no lt_cv_dlopen_libs= case $host_os in beos*) lt_cv_dlopen="load_add_on" lt_cv_dlopen_libs= lt_cv_dlopen_self=yes ;; mingw* | pw32* | cegcc*) lt_cv_dlopen="LoadLibrary" lt_cv_dlopen_libs= ;; cygwin*) lt_cv_dlopen="dlopen" lt_cv_dlopen_libs= ;; darwin*) # if libdl is installed we need to link against it { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -ldl" >&5 $as_echo_n "checking for dlopen in -ldl... " >&6; } if ${ac_cv_lib_dl_dlopen+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-ldl $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char dlopen (); int main () { return dlopen (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_dl_dlopen=yes else ac_cv_lib_dl_dlopen=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dl_dlopen" >&5 $as_echo "$ac_cv_lib_dl_dlopen" >&6; } if test "x$ac_cv_lib_dl_dlopen" = xyes; then : lt_cv_dlopen="dlopen" lt_cv_dlopen_libs="-ldl" else lt_cv_dlopen="dyld" lt_cv_dlopen_libs= lt_cv_dlopen_self=yes fi ;; *) ac_fn_c_check_func "$LINENO" "shl_load" "ac_cv_func_shl_load" if test "x$ac_cv_func_shl_load" = xyes; then : lt_cv_dlopen="shl_load" else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for shl_load in -ldld" >&5 $as_echo_n "checking for shl_load in -ldld... " >&6; } if ${ac_cv_lib_dld_shl_load+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-ldld $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char shl_load (); int main () { return shl_load (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_dld_shl_load=yes else ac_cv_lib_dld_shl_load=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dld_shl_load" >&5 $as_echo "$ac_cv_lib_dld_shl_load" >&6; } if test "x$ac_cv_lib_dld_shl_load" = xyes; then : lt_cv_dlopen="shl_load" lt_cv_dlopen_libs="-ldld" else ac_fn_c_check_func "$LINENO" "dlopen" "ac_cv_func_dlopen" if test "x$ac_cv_func_dlopen" = xyes; then : lt_cv_dlopen="dlopen" else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -ldl" >&5 $as_echo_n "checking for dlopen in -ldl... " >&6; } if ${ac_cv_lib_dl_dlopen+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-ldl $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char dlopen (); int main () { return dlopen (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_dl_dlopen=yes else ac_cv_lib_dl_dlopen=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dl_dlopen" >&5 $as_echo "$ac_cv_lib_dl_dlopen" >&6; } if test "x$ac_cv_lib_dl_dlopen" = xyes; then : lt_cv_dlopen="dlopen" lt_cv_dlopen_libs="-ldl" else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -lsvld" >&5 $as_echo_n "checking for dlopen in -lsvld... " >&6; } if ${ac_cv_lib_svld_dlopen+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-lsvld $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char dlopen (); int main () { return dlopen (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_svld_dlopen=yes else ac_cv_lib_svld_dlopen=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_svld_dlopen" >&5 $as_echo "$ac_cv_lib_svld_dlopen" >&6; } if test "x$ac_cv_lib_svld_dlopen" = xyes; then : lt_cv_dlopen="dlopen" lt_cv_dlopen_libs="-lsvld" else { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dld_link in -ldld" >&5 $as_echo_n "checking for dld_link in -ldld... " >&6; } if ${ac_cv_lib_dld_dld_link+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-ldld $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char dld_link (); int main () { return dld_link (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_dld_dld_link=yes else ac_cv_lib_dld_dld_link=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dld_dld_link" >&5 $as_echo "$ac_cv_lib_dld_dld_link" >&6; } if test "x$ac_cv_lib_dld_dld_link" = xyes; then : lt_cv_dlopen="dld_link" lt_cv_dlopen_libs="-ldld" fi fi fi fi fi fi ;; esac if test "x$lt_cv_dlopen" != xno; then enable_dlopen=yes else enable_dlopen=no fi case $lt_cv_dlopen in dlopen) save_CPPFLAGS="$CPPFLAGS" test "x$ac_cv_header_dlfcn_h" = xyes && CPPFLAGS="$CPPFLAGS -DHAVE_DLFCN_H" save_LDFLAGS="$LDFLAGS" wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $export_dynamic_flag_spec\" save_LIBS="$LIBS" LIBS="$lt_cv_dlopen_libs $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether a program can dlopen itself" >&5 $as_echo_n "checking whether a program can dlopen itself... " >&6; } if ${lt_cv_dlopen_self+:} false; then : $as_echo_n "(cached) " >&6 else if test "$cross_compiling" = yes; then : lt_cv_dlopen_self=cross else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF #line $LINENO "configure" #include "confdefs.h" #if HAVE_DLFCN_H #include #endif #include #ifdef RTLD_GLOBAL # define LT_DLGLOBAL RTLD_GLOBAL #else # ifdef DL_GLOBAL # define LT_DLGLOBAL DL_GLOBAL # else # define LT_DLGLOBAL 0 # endif #endif /* We may have to define LT_DLLAZY_OR_NOW in the command line if we find out it does not work in some platform. */ #ifndef LT_DLLAZY_OR_NOW # ifdef RTLD_LAZY # define LT_DLLAZY_OR_NOW RTLD_LAZY # else # ifdef DL_LAZY # define LT_DLLAZY_OR_NOW DL_LAZY # else # ifdef RTLD_NOW # define LT_DLLAZY_OR_NOW RTLD_NOW # else # ifdef DL_NOW # define LT_DLLAZY_OR_NOW DL_NOW # else # define LT_DLLAZY_OR_NOW 0 # endif # endif # endif # endif #endif /* When -fvisbility=hidden is used, assume the code has been annotated correspondingly for the symbols needed. */ #if defined(__GNUC__) && (((__GNUC__ == 3) && (__GNUC_MINOR__ >= 3)) || (__GNUC__ > 3)) int fnord () __attribute__((visibility("default"))); #endif int fnord () { return 42; } int main () { void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); int status = $lt_dlunknown; if (self) { if (dlsym (self,"fnord")) status = $lt_dlno_uscore; else { if (dlsym( self,"_fnord")) status = $lt_dlneed_uscore; else puts (dlerror ()); } /* dlclose (self); */ } else puts (dlerror ()); return status; } _LT_EOF if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 (eval $ac_link) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && test -s conftest${ac_exeext} 2>/dev/null; then (./conftest; exit; ) >&5 2>/dev/null lt_status=$? case x$lt_status in x$lt_dlno_uscore) lt_cv_dlopen_self=yes ;; x$lt_dlneed_uscore) lt_cv_dlopen_self=yes ;; x$lt_dlunknown|x*) lt_cv_dlopen_self=no ;; esac else : # compilation failed lt_cv_dlopen_self=no fi fi rm -fr conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_dlopen_self" >&5 $as_echo "$lt_cv_dlopen_self" >&6; } if test "x$lt_cv_dlopen_self" = xyes; then wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $lt_prog_compiler_static\" { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether a statically linked program can dlopen itself" >&5 $as_echo_n "checking whether a statically linked program can dlopen itself... " >&6; } if ${lt_cv_dlopen_self_static+:} false; then : $as_echo_n "(cached) " >&6 else if test "$cross_compiling" = yes; then : lt_cv_dlopen_self_static=cross else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF #line $LINENO "configure" #include "confdefs.h" #if HAVE_DLFCN_H #include #endif #include #ifdef RTLD_GLOBAL # define LT_DLGLOBAL RTLD_GLOBAL #else # ifdef DL_GLOBAL # define LT_DLGLOBAL DL_GLOBAL # else # define LT_DLGLOBAL 0 # endif #endif /* We may have to define LT_DLLAZY_OR_NOW in the command line if we find out it does not work in some platform. */ #ifndef LT_DLLAZY_OR_NOW # ifdef RTLD_LAZY # define LT_DLLAZY_OR_NOW RTLD_LAZY # else # ifdef DL_LAZY # define LT_DLLAZY_OR_NOW DL_LAZY # else # ifdef RTLD_NOW # define LT_DLLAZY_OR_NOW RTLD_NOW # else # ifdef DL_NOW # define LT_DLLAZY_OR_NOW DL_NOW # else # define LT_DLLAZY_OR_NOW 0 # endif # endif # endif # endif #endif /* When -fvisbility=hidden is used, assume the code has been annotated correspondingly for the symbols needed. */ #if defined(__GNUC__) && (((__GNUC__ == 3) && (__GNUC_MINOR__ >= 3)) || (__GNUC__ > 3)) int fnord () __attribute__((visibility("default"))); #endif int fnord () { return 42; } int main () { void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); int status = $lt_dlunknown; if (self) { if (dlsym (self,"fnord")) status = $lt_dlno_uscore; else { if (dlsym( self,"_fnord")) status = $lt_dlneed_uscore; else puts (dlerror ()); } /* dlclose (self); */ } else puts (dlerror ()); return status; } _LT_EOF if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 (eval $ac_link) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; } && test -s conftest${ac_exeext} 2>/dev/null; then (./conftest; exit; ) >&5 2>/dev/null lt_status=$? case x$lt_status in x$lt_dlno_uscore) lt_cv_dlopen_self_static=yes ;; x$lt_dlneed_uscore) lt_cv_dlopen_self_static=yes ;; x$lt_dlunknown|x*) lt_cv_dlopen_self_static=no ;; esac else : # compilation failed lt_cv_dlopen_self_static=no fi fi rm -fr conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_dlopen_self_static" >&5 $as_echo "$lt_cv_dlopen_self_static" >&6; } fi CPPFLAGS="$save_CPPFLAGS" LDFLAGS="$save_LDFLAGS" LIBS="$save_LIBS" ;; esac case $lt_cv_dlopen_self in yes|no) enable_dlopen_self=$lt_cv_dlopen_self ;; *) enable_dlopen_self=unknown ;; esac case $lt_cv_dlopen_self_static in yes|no) enable_dlopen_self_static=$lt_cv_dlopen_self_static ;; *) enable_dlopen_self_static=unknown ;; esac fi striplib= old_striplib= { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether stripping libraries is possible" >&5 $as_echo_n "checking whether stripping libraries is possible... " >&6; } if test -n "$STRIP" && $STRIP -V 2>&1 | $GREP "GNU strip" >/dev/null; then test -z "$old_striplib" && old_striplib="$STRIP --strip-debug" test -z "$striplib" && striplib="$STRIP --strip-unneeded" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else # FIXME - insert some real tests, host_os isn't really good enough case $host_os in darwin*) if test -n "$STRIP" ; then striplib="$STRIP -x" old_striplib="$STRIP -S" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi ;; *) { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } ;; esac fi # Report which library types will actually be built { $as_echo "$as_me:${as_lineno-$LINENO}: checking if libtool supports shared libraries" >&5 $as_echo_n "checking if libtool supports shared libraries... " >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: result: $can_build_shared" >&5 $as_echo "$can_build_shared" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build shared libraries" >&5 $as_echo_n "checking whether to build shared libraries... " >&6; } test "$can_build_shared" = "no" && enable_shared=no # On AIX, shared libraries and static libraries use the same namespace, and # are all built from PIC. case $host_os in aix3*) test "$enable_shared" = yes && enable_static=no if test -n "$RANLIB"; then archive_cmds="$archive_cmds~\$RANLIB \$lib" postinstall_cmds='$RANLIB $lib' fi ;; aix[4-9]*) if test "$host_cpu" != ia64 && test "$aix_use_runtimelinking" = no ; then test "$enable_shared" = yes && enable_static=no fi ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: result: $enable_shared" >&5 $as_echo "$enable_shared" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build static libraries" >&5 $as_echo_n "checking whether to build static libraries... " >&6; } # Make sure either enable_shared or enable_static is yes. test "$enable_shared" = yes || enable_static=yes { $as_echo "$as_me:${as_lineno-$LINENO}: result: $enable_static" >&5 $as_echo "$enable_static" >&6; } fi ac_ext=c ac_cpp='$CPP $CPPFLAGS' ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' ac_compiler_gnu=$ac_cv_c_compiler_gnu CC="$lt_save_CC" ac_config_commands="$ac_config_commands libtool" # Only expand once: CFLAG_VISIBILITY= HAVE_VISIBILITY=0 if test -n "$GCC"; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the -Werror option is usable" >&5 $as_echo_n "checking whether the -Werror option is usable... " >&6; } if ${gl_cv_cc_vis_werror+:} false; then : $as_echo_n "(cached) " >&6 else gl_save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : gl_cv_cc_vis_werror=yes else gl_cv_cc_vis_werror=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext CFLAGS="$gl_save_CFLAGS" fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $gl_cv_cc_vis_werror" >&5 $as_echo "$gl_cv_cc_vis_werror" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: checking for simple visibility declarations" >&5 $as_echo_n "checking for simple visibility declarations... " >&6; } if ${gl_cv_cc_visibility+:} false; then : $as_echo_n "(cached) " >&6 else gl_save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -fvisibility=hidden" if test $gl_cv_cc_vis_werror = yes; then CFLAGS="$CFLAGS -Werror" fi cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ extern __attribute__((__visibility__("hidden"))) int hiddenvar; extern __attribute__((__visibility__("default"))) int exportedvar; extern __attribute__((__visibility__("hidden"))) int hiddenfunc (void); extern __attribute__((__visibility__("default"))) int exportedfunc (void); void dummyfunc (void) {} int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : gl_cv_cc_visibility=yes else gl_cv_cc_visibility=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext CFLAGS="$gl_save_CFLAGS" fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $gl_cv_cc_visibility" >&5 $as_echo "$gl_cv_cc_visibility" >&6; } if test $gl_cv_cc_visibility = yes; then CFLAG_VISIBILITY="-fvisibility=hidden" HAVE_VISIBILITY=1 fi fi cat >>confdefs.h <<_ACEOF #define HAVE_VISIBILITY $HAVE_VISIBILITY _ACEOF { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ANSI C header files" >&5 $as_echo_n "checking for ANSI C header files... " >&6; } if ${ac_cv_header_stdc+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #include #include int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_header_stdc=yes else ac_cv_header_stdc=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext if test $ac_cv_header_stdc = yes; then # SunOS 4.x string.h does not declare mem*, contrary to ANSI. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | $EGREP "memchr" >/dev/null 2>&1; then : else ac_cv_header_stdc=no fi rm -f conftest* fi if test $ac_cv_header_stdc = yes; then # ISC 2.0.2 stdlib.h does not declare free, contrary to ANSI. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include _ACEOF if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | $EGREP "free" >/dev/null 2>&1; then : else ac_cv_header_stdc=no fi rm -f conftest* fi if test $ac_cv_header_stdc = yes; then # /bin/cc in Irix-4.0.5 gets non-ANSI ctype macros unless using -ansi. if test "$cross_compiling" = yes; then : : else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #if ((' ' & 0x0FF) == 0x020) # define ISLOWER(c) ('a' <= (c) && (c) <= 'z') # define TOUPPER(c) (ISLOWER(c) ? 'A' + ((c) - 'a') : (c)) #else # define ISLOWER(c) \ (('a' <= (c) && (c) <= 'i') \ || ('j' <= (c) && (c) <= 'r') \ || ('s' <= (c) && (c) <= 'z')) # define TOUPPER(c) (ISLOWER(c) ? ((c) | 0x40) : (c)) #endif #define XOR(e, f) (((e) && !(f)) || (!(e) && (f))) int main () { int i; for (i = 0; i < 256; i++) if (XOR (islower (i), ISLOWER (i)) || toupper (i) != TOUPPER (i)) return 2; return 0; } _ACEOF if ac_fn_c_try_run "$LINENO"; then : else ac_cv_header_stdc=no fi rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ conftest.$ac_objext conftest.beam conftest.$ac_ext fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_header_stdc" >&5 $as_echo "$ac_cv_header_stdc" >&6; } if test $ac_cv_header_stdc = yes; then $as_echo "#define STDC_HEADERS 1" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether sys/types.h defines makedev" >&5 $as_echo_n "checking whether sys/types.h defines makedev... " >&6; } if ${ac_cv_header_sys_types_h_makedev+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { return makedev(0, 0); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_header_sys_types_h_makedev=yes else ac_cv_header_sys_types_h_makedev=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_header_sys_types_h_makedev" >&5 $as_echo "$ac_cv_header_sys_types_h_makedev" >&6; } if test $ac_cv_header_sys_types_h_makedev = no; then ac_fn_c_check_header_mongrel "$LINENO" "sys/mkdev.h" "ac_cv_header_sys_mkdev_h" "$ac_includes_default" if test "x$ac_cv_header_sys_mkdev_h" = xyes; then : $as_echo "#define MAJOR_IN_MKDEV 1" >>confdefs.h fi if test $ac_cv_header_sys_mkdev_h = no; then ac_fn_c_check_header_mongrel "$LINENO" "sys/sysmacros.h" "ac_cv_header_sys_sysmacros_h" "$ac_includes_default" if test "x$ac_cv_header_sys_sysmacros_h" = xyes; then : $as_echo "#define MAJOR_IN_SYSMACROS 1" >>confdefs.h fi fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for sys/wait.h that is POSIX.1 compatible" >&5 $as_echo_n "checking for sys/wait.h that is POSIX.1 compatible... " >&6; } if ${ac_cv_header_sys_wait_h+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include #ifndef WEXITSTATUS # define WEXITSTATUS(stat_val) ((unsigned int) (stat_val) >> 8) #endif #ifndef WIFEXITED # define WIFEXITED(stat_val) (((stat_val) & 255) == 0) #endif int main () { int s; wait (&s); s = WIFEXITED (s) ? WEXITSTATUS (s) : 1; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_header_sys_wait_h=yes else ac_cv_header_sys_wait_h=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_header_sys_wait_h" >&5 $as_echo "$ac_cv_header_sys_wait_h" >&6; } if test $ac_cv_header_sys_wait_h = yes; then $as_echo "#define HAVE_SYS_WAIT_H 1" >>confdefs.h fi for ac_header in stdint.h fcntl.h locale.h stdint.h inttypes.h unistd.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 _ACEOF fi done for ac_header in stddef.h utime.h wchar.h wctype.h limits.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 _ACEOF fi done -for ac_header in getopt.h err.h +for ac_header in getopt.h err.h xlocale.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 _ACEOF fi done for ac_header in sys/mman.h sys/stat.h sys/types.h sys/utime.h sys/time.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 _ACEOF fi done for ac_header in zlib.h do : ac_fn_c_check_header_mongrel "$LINENO" "zlib.h" "ac_cv_header_zlib_h" "$ac_includes_default" if test "x$ac_cv_header_zlib_h" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_ZLIB_H 1 _ACEOF fi done { $as_echo "$as_me:${as_lineno-$LINENO}: checking for an ANSI C-conforming const" >&5 $as_echo_n "checking for an ANSI C-conforming const... " >&6; } if ${ac_cv_c_const+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ int main () { #ifndef __cplusplus /* Ultrix mips cc rejects this sort of thing. */ typedef int charset[2]; const charset cs = { 0, 0 }; /* SunOS 4.1.1 cc rejects this. */ char const *const *pcpcc; char **ppc; /* NEC SVR4.0.2 mips cc rejects this. */ struct point {int x, y;}; static struct point const zero = {0,0}; /* AIX XL C 1.02.0.0 rejects this. It does not let you subtract one const X* pointer from another in an arm of an if-expression whose if-part is not a constant expression */ const char *g = "string"; pcpcc = &g + (g ? g-g : 0); /* HPUX 7.0 cc rejects these. */ ++pcpcc; ppc = (char**) pcpcc; pcpcc = (char const *const *) ppc; { /* SCO 3.2v4 cc rejects this sort of thing. */ char tx; char *t = &tx; char const *s = 0 ? (char *) 0 : (char const *) 0; *t++ = 0; if (s) return 0; } { /* Someone thinks the Sun supposedly-ANSI compiler will reject this. */ int x[] = {25, 17}; const int *foo = &x[0]; ++foo; } { /* Sun SC1.0 ANSI compiler rejects this -- but not the above. */ typedef const int *iptr; iptr p = 0; ++p; } { /* AIX XL C 1.02.0.0 rejects this sort of thing, saying "k.c", line 2.27: 1506-025 (S) Operand must be a modifiable lvalue. */ struct s { int j; const int *ap[3]; } bx; struct s *b = &bx; b->j = 5; } { /* ULTRIX-32 V3.1 (Rev 9) vcc rejects this */ const int foo = 10; if (!foo) return 0; } return !cs[0] && !zero.x; #endif ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_c_const=yes else ac_cv_c_const=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_c_const" >&5 $as_echo "$ac_cv_c_const" >&6; } if test $ac_cv_c_const = no; then $as_echo "#define const /**/" >>confdefs.h fi ac_fn_c_check_type "$LINENO" "off_t" "ac_cv_type_off_t" "$ac_includes_default" if test "x$ac_cv_type_off_t" = xyes; then : else cat >>confdefs.h <<_ACEOF #define off_t long int _ACEOF fi ac_fn_c_check_type "$LINENO" "size_t" "ac_cv_type_size_t" "$ac_includes_default" if test "x$ac_cv_type_size_t" = xyes; then : else cat >>confdefs.h <<_ACEOF #define size_t unsigned int _ACEOF fi ac_fn_c_check_member "$LINENO" "struct stat" "st_rdev" "ac_cv_member_struct_stat_st_rdev" "$ac_includes_default" if test "x$ac_cv_member_struct_stat_st_rdev" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_STRUCT_STAT_ST_RDEV 1 _ACEOF fi ac_fn_c_check_member "$LINENO" "struct tm" "tm_gmtoff" "ac_cv_member_struct_tm_tm_gmtoff" "$ac_includes_default" if test "x$ac_cv_member_struct_tm_tm_gmtoff" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_STRUCT_TM_TM_GMTOFF 1 _ACEOF fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether struct tm is in sys/time.h or time.h" >&5 $as_echo_n "checking whether struct tm is in sys/time.h or time.h... " >&6; } if ${ac_cv_struct_tm+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include int main () { struct tm tm; int *p = &tm.tm_sec; return !p; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_struct_tm=time.h else ac_cv_struct_tm=sys/time.h fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_struct_tm" >&5 $as_echo "$ac_cv_struct_tm" >&6; } if test $ac_cv_struct_tm = sys/time.h; then $as_echo "#define TM_IN_SYS_TIME 1" >>confdefs.h fi ac_fn_c_check_member "$LINENO" "struct tm" "tm_zone" "ac_cv_member_struct_tm_tm_zone" "#include #include <$ac_cv_struct_tm> " if test "x$ac_cv_member_struct_tm_tm_zone" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_STRUCT_TM_TM_ZONE 1 _ACEOF fi if test "$ac_cv_member_struct_tm_tm_zone" = yes; then $as_echo "#define HAVE_TM_ZONE 1" >>confdefs.h else ac_fn_c_check_decl "$LINENO" "tzname" "ac_cv_have_decl_tzname" "#include " if test "x$ac_cv_have_decl_tzname" = xyes; then : ac_have_decl=1 else ac_have_decl=0 fi cat >>confdefs.h <<_ACEOF #define HAVE_DECL_TZNAME $ac_have_decl _ACEOF { $as_echo "$as_me:${as_lineno-$LINENO}: checking for tzname" >&5 $as_echo_n "checking for tzname... " >&6; } if ${ac_cv_var_tzname+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #if !HAVE_DECL_TZNAME extern char *tzname[]; #endif int main () { return tzname[0][0]; ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_var_tzname=yes else ac_cv_var_tzname=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_var_tzname" >&5 $as_echo "$ac_cv_var_tzname" >&6; } if test $ac_cv_var_tzname = yes; then $as_echo "#define HAVE_TZNAME 1" >>confdefs.h fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for tm_zone in struct tm" >&5 $as_echo_n "checking for tm_zone in struct tm... " >&6; } if ${ac_cv_struct_tm_zone+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include <$ac_cv_struct_tm> int main () { struct tm tm; tm.tm_zone; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_struct_tm_zone=yes else ac_cv_struct_tm_zone=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_struct_tm_zone" >&5 $as_echo "$ac_cv_struct_tm_zone" >&6; } if test "$ac_cv_struct_tm_zone" = yes; then $as_echo "#define HAVE_TM_ZONE 1" >>confdefs.h fi # On SGI, apparently tzname is a #define, but that's ok, AC_CHECK_DECL will # consider it declared and we won't give our own extern. ac_fn_c_check_decl "$LINENO" "tzname" "ac_cv_have_decl_tzname" "#include " if test "x$ac_cv_have_decl_tzname" = xyes; then : ac_have_decl=1 else ac_have_decl=0 fi cat >>confdefs.h <<_ACEOF #define HAVE_DECL_TZNAME $ac_have_decl _ACEOF { $as_echo "$as_me:${as_lineno-$LINENO}: checking for tzname" >&5 $as_echo_n "checking for tzname... " >&6; } if ${ac_cv_var_tzname+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #if !HAVE_DECL_TZNAME extern char *tzname[]; #endif int main () { return tzname[0][0]; ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_var_tzname=yes else ac_cv_var_tzname=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_var_tzname" >&5 $as_echo "$ac_cv_var_tzname" >&6; } if test $ac_cv_var_tzname = yes; then $as_echo "#define HAVE_TZNAME 1" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for tm_isdst in struct tm" >&5 $as_echo_n "checking for tm_isdst in struct tm... " >&6; } if ${ac_cv_struct_tm_isdst+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #include <$ac_cv_struct_tm> int main () { struct tm tm; tm.tm_isdst; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_struct_tm_isdst=yes else ac_cv_struct_tm_isdst=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_struct_tm_isdst" >&5 $as_echo "$ac_cv_struct_tm_isdst" >&6; } if test "$ac_cv_struct_tm_isdst" = yes; then $as_echo "#define HAVE_TM_ISDST 1" >>confdefs.h fi ac_fn_c_check_decl "$LINENO" "daylight" "ac_cv_have_decl_daylight" "#include " if test "x$ac_cv_have_decl_daylight" = xyes; then : ac_have_decl=1 else ac_have_decl=0 fi cat >>confdefs.h <<_ACEOF #define HAVE_DECL_DAYLIGHT $ac_have_decl _ACEOF { $as_echo "$as_me:${as_lineno-$LINENO}: checking for daylight" >&5 $as_echo_n "checking for daylight... " >&6; } if ${ac_cv_var_daylight+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include #if !HAVE_DECL_DAYLIGHT extern int daylight; #endif int main () { atoi(daylight); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_var_daylight=yes else ac_cv_var_daylight=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_var_daylight" >&5 $as_echo "$ac_cv_var_daylight" >&6; } if test $ac_cv_var_daylight = yes; then $as_echo "#define HAVE_DAYLIGHT 1" >>confdefs.h fi # Check whether --enable-largefile was given. if test "${enable_largefile+set}" = set; then : enableval=$enable_largefile; fi if test "$enable_largefile" != no; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for special C compiler options needed for large files" >&5 $as_echo_n "checking for special C compiler options needed for large files... " >&6; } if ${ac_cv_sys_largefile_CC+:} false; then : $as_echo_n "(cached) " >&6 else ac_cv_sys_largefile_CC=no if test "$GCC" != yes; then ac_save_CC=$CC while :; do # IRIX 6.2 and later do not support large files by default, # so use the C compiler's -n32 option if that helps. cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include /* Check that off_t can represent 2**63 - 1 correctly. We can't simply define LARGE_OFF_T to be 9223372036854775807, since some C++ compilers masquerading as C compilers incorrectly reject 9223372036854775807. */ #define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 && LARGE_OFF_T % 2147483647 == 1) ? 1 : -1]; int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : break fi rm -f core conftest.err conftest.$ac_objext CC="$CC -n32" if ac_fn_c_try_compile "$LINENO"; then : ac_cv_sys_largefile_CC=' -n32'; break fi rm -f core conftest.err conftest.$ac_objext break done CC=$ac_save_CC rm -f conftest.$ac_ext fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sys_largefile_CC" >&5 $as_echo "$ac_cv_sys_largefile_CC" >&6; } if test "$ac_cv_sys_largefile_CC" != no; then CC=$CC$ac_cv_sys_largefile_CC fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for _FILE_OFFSET_BITS value needed for large files" >&5 $as_echo_n "checking for _FILE_OFFSET_BITS value needed for large files... " >&6; } if ${ac_cv_sys_file_offset_bits+:} false; then : $as_echo_n "(cached) " >&6 else while :; do cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include /* Check that off_t can represent 2**63 - 1 correctly. We can't simply define LARGE_OFF_T to be 9223372036854775807, since some C++ compilers masquerading as C compilers incorrectly reject 9223372036854775807. */ #define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 && LARGE_OFF_T % 2147483647 == 1) ? 1 : -1]; int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_sys_file_offset_bits=no; break fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #define _FILE_OFFSET_BITS 64 #include /* Check that off_t can represent 2**63 - 1 correctly. We can't simply define LARGE_OFF_T to be 9223372036854775807, since some C++ compilers masquerading as C compilers incorrectly reject 9223372036854775807. */ #define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 && LARGE_OFF_T % 2147483647 == 1) ? 1 : -1]; int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_sys_file_offset_bits=64; break fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_cv_sys_file_offset_bits=unknown break done fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sys_file_offset_bits" >&5 $as_echo "$ac_cv_sys_file_offset_bits" >&6; } case $ac_cv_sys_file_offset_bits in #( no | unknown) ;; *) cat >>confdefs.h <<_ACEOF #define _FILE_OFFSET_BITS $ac_cv_sys_file_offset_bits _ACEOF ;; esac rm -rf conftest* if test $ac_cv_sys_file_offset_bits = unknown; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for _LARGE_FILES value needed for large files" >&5 $as_echo_n "checking for _LARGE_FILES value needed for large files... " >&6; } if ${ac_cv_sys_large_files+:} false; then : $as_echo_n "(cached) " >&6 else while :; do cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include /* Check that off_t can represent 2**63 - 1 correctly. We can't simply define LARGE_OFF_T to be 9223372036854775807, since some C++ compilers masquerading as C compilers incorrectly reject 9223372036854775807. */ #define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 && LARGE_OFF_T % 2147483647 == 1) ? 1 : -1]; int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_sys_large_files=no; break fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #define _LARGE_FILES 1 #include /* Check that off_t can represent 2**63 - 1 correctly. We can't simply define LARGE_OFF_T to be 9223372036854775807, since some C++ compilers masquerading as C compilers incorrectly reject 9223372036854775807. */ #define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 && LARGE_OFF_T % 2147483647 == 1) ? 1 : -1]; int main () { ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_sys_large_files=1; break fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext ac_cv_sys_large_files=unknown break done fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sys_large_files" >&5 $as_echo "$ac_cv_sys_large_files" >&6; } case $ac_cv_sys_large_files in #( no | unknown) ;; *) cat >>confdefs.h <<_ACEOF #define _LARGE_FILES $ac_cv_sys_large_files _ACEOF ;; esac rm -rf conftest* fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for _LARGEFILE_SOURCE value needed for large files" >&5 $as_echo_n "checking for _LARGEFILE_SOURCE value needed for large files... " >&6; } if ${ac_cv_sys_largefile_source+:} false; then : $as_echo_n "(cached) " >&6 else while :; do cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include /* for off_t */ #include int main () { int (*fp) (FILE *, off_t, int) = fseeko; return fseeko (stdin, 0, 0) && fp (stdin, 0, 0); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_sys_largefile_source=no; break fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #define _LARGEFILE_SOURCE 1 #include /* for off_t */ #include int main () { int (*fp) (FILE *, off_t, int) = fseeko; return fseeko (stdin, 0, 0) && fp (stdin, 0, 0); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_sys_largefile_source=1; break fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext ac_cv_sys_largefile_source=unknown break done fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sys_largefile_source" >&5 $as_echo "$ac_cv_sys_largefile_source" >&6; } case $ac_cv_sys_largefile_source in #( no | unknown) ;; *) cat >>confdefs.h <<_ACEOF #define _LARGEFILE_SOURCE $ac_cv_sys_largefile_source _ACEOF ;; esac rm -rf conftest* # We used to try defining _XOPEN_SOURCE=500 too, to work around a bug # in glibc 2.1.3, but that breaks too many other things. # If you want fseeko and ftello with glibc, upgrade to a fixed glibc. if test $ac_cv_sys_largefile_source != unknown; then $as_echo "#define HAVE_FSEEKO 1" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for mbstate_t" >&5 $as_echo_n "checking for mbstate_t... " >&6; } if ${ac_cv_type_mbstate_t+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default # include int main () { mbstate_t x; return sizeof x; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_type_mbstate_t=yes else ac_cv_type_mbstate_t=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_type_mbstate_t" >&5 $as_echo "$ac_cv_type_mbstate_t" >&6; } if test $ac_cv_type_mbstate_t = yes; then $as_echo "#define HAVE_MBSTATE_T 1" >>confdefs.h else $as_echo "#define mbstate_t int" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for struct option in getopt" >&5 $as_echo_n "checking for struct option in getopt... " >&6; } if ${ac_cv_struct_option_getopt_h+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { struct option op; op.name; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : ac_cv_struct_option_getopt_h=yes else ac_cv_struct_option_getopt_h=no fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_struct_option_getopt_h" >&5 $as_echo "$ac_cv_struct_option_getopt_h" >&6; } if test "$ac_cv_struct_option_getopt_h" = yes; then $as_echo "#define HAVE_STRUCT_OPTION 1" >>confdefs.h fi ac_fn_c_check_type "$LINENO" "pid_t" "ac_cv_type_pid_t" "$ac_includes_default" if test "x$ac_cv_type_pid_t" = xyes; then : else cat >>confdefs.h <<_ACEOF #define pid_t int _ACEOF fi ac_fn_c_find_uintX_t "$LINENO" "8" "ac_cv_c_uint8_t" case $ac_cv_c_uint8_t in #( no|yes) ;; #( *) $as_echo "#define _UINT8_T 1" >>confdefs.h cat >>confdefs.h <<_ACEOF #define uint8_t $ac_cv_c_uint8_t _ACEOF ;; esac ac_fn_c_find_uintX_t "$LINENO" "16" "ac_cv_c_uint16_t" case $ac_cv_c_uint16_t in #( no|yes) ;; #( *) cat >>confdefs.h <<_ACEOF #define uint16_t $ac_cv_c_uint16_t _ACEOF ;; esac ac_fn_c_find_uintX_t "$LINENO" "32" "ac_cv_c_uint32_t" case $ac_cv_c_uint32_t in #( no|yes) ;; #( *) $as_echo "#define _UINT32_T 1" >>confdefs.h cat >>confdefs.h <<_ACEOF #define uint32_t $ac_cv_c_uint32_t _ACEOF ;; esac ac_fn_c_find_intX_t "$LINENO" "32" "ac_cv_c_int32_t" case $ac_cv_c_int32_t in #( no|yes) ;; #( *) cat >>confdefs.h <<_ACEOF #define int32_t $ac_cv_c_int32_t _ACEOF ;; esac ac_fn_c_find_uintX_t "$LINENO" "64" "ac_cv_c_uint64_t" case $ac_cv_c_uint64_t in #( no|yes) ;; #( *) $as_echo "#define _UINT64_T 1" >>confdefs.h cat >>confdefs.h <<_ACEOF #define uint64_t $ac_cv_c_uint64_t _ACEOF ;; esac ac_fn_c_find_intX_t "$LINENO" "64" "ac_cv_c_int64_t" case $ac_cv_c_int64_t in #( no|yes) ;; #( *) cat >>confdefs.h <<_ACEOF #define int64_t $ac_cv_c_int64_t _ACEOF ;; esac ac_fn_c_check_type "$LINENO" "intptr_t" "ac_cv_type_intptr_t" "$ac_includes_default" if test "x$ac_cv_type_intptr_t" = xyes; then : $as_echo "#define HAVE_INTPTR_T 1" >>confdefs.h else for ac_type in 'int' 'long int' 'long long int'; do cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default int main () { static int test_array [1 - 2 * !(sizeof (void *) <= sizeof ($ac_type))]; test_array [0] = 0; return test_array [0]; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : cat >>confdefs.h <<_ACEOF #define intptr_t $ac_type _ACEOF ac_type= fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext test -z "$ac_type" && break done fi ac_fn_c_check_type "$LINENO" "uintptr_t" "ac_cv_type_uintptr_t" "$ac_includes_default" if test "x$ac_cv_type_uintptr_t" = xyes; then : $as_echo "#define HAVE_UINTPTR_T 1" >>confdefs.h else for ac_type in 'unsigned int' 'unsigned long int' \ 'unsigned long long int'; do cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default int main () { static int test_array [1 - 2 * !(sizeof (void *) <= sizeof ($ac_type))]; test_array [0] = 0; return test_array [0]; ; return 0; } _ACEOF if ac_fn_c_try_compile "$LINENO"; then : cat >>confdefs.h <<_ACEOF #define uintptr_t $ac_type _ACEOF ac_type= fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext test -z "$ac_type" && break done fi for ac_header in $ac_header_list do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_compile "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default " if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 _ACEOF fi done for ac_func in getpagesize do : ac_fn_c_check_func "$LINENO" "getpagesize" "ac_cv_func_getpagesize" if test "x$ac_cv_func_getpagesize" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_GETPAGESIZE 1 _ACEOF fi done { $as_echo "$as_me:${as_lineno-$LINENO}: checking for working mmap" >&5 $as_echo_n "checking for working mmap... " >&6; } if ${ac_cv_func_mmap_fixed_mapped+:} false; then : $as_echo_n "(cached) " >&6 else if test "$cross_compiling" = yes; then : ac_cv_func_mmap_fixed_mapped=no else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default /* malloc might have been renamed as rpl_malloc. */ #undef malloc /* Thanks to Mike Haertel and Jim Avera for this test. Here is a matrix of mmap possibilities: mmap private not fixed mmap private fixed at somewhere currently unmapped mmap private fixed at somewhere already mapped mmap shared not fixed mmap shared fixed at somewhere currently unmapped mmap shared fixed at somewhere already mapped For private mappings, we should verify that changes cannot be read() back from the file, nor mmap's back from the file at a different address. (There have been systems where private was not correctly implemented like the infamous i386 svr4.0, and systems where the VM page cache was not coherent with the file system buffer cache like early versions of FreeBSD and possibly contemporary NetBSD.) For shared mappings, we should conversely verify that changes get propagated back to all the places they're supposed to be. Grep wants private fixed already mapped. The main things grep needs to know about mmap are: * does it exist and is it safe to write into the mmap'd area * how to use it (BSD variants) */ #include #include #if !defined STDC_HEADERS && !defined HAVE_STDLIB_H char *malloc (); #endif /* This mess was copied from the GNU getpagesize.h. */ #ifndef HAVE_GETPAGESIZE # ifdef _SC_PAGESIZE # define getpagesize() sysconf(_SC_PAGESIZE) # else /* no _SC_PAGESIZE */ # ifdef HAVE_SYS_PARAM_H # include # ifdef EXEC_PAGESIZE # define getpagesize() EXEC_PAGESIZE # else /* no EXEC_PAGESIZE */ # ifdef NBPG # define getpagesize() NBPG * CLSIZE # ifndef CLSIZE # define CLSIZE 1 # endif /* no CLSIZE */ # else /* no NBPG */ # ifdef NBPC # define getpagesize() NBPC # else /* no NBPC */ # ifdef PAGESIZE # define getpagesize() PAGESIZE # endif /* PAGESIZE */ # endif /* no NBPC */ # endif /* no NBPG */ # endif /* no EXEC_PAGESIZE */ # else /* no HAVE_SYS_PARAM_H */ # define getpagesize() 8192 /* punt totally */ # endif /* no HAVE_SYS_PARAM_H */ # endif /* no _SC_PAGESIZE */ #endif /* no HAVE_GETPAGESIZE */ int main () { char *data, *data2, *data3; const char *cdata2; int i, pagesize; int fd, fd2; pagesize = getpagesize (); /* First, make a file with some known garbage in it. */ data = (char *) malloc (pagesize); if (!data) return 1; for (i = 0; i < pagesize; ++i) *(data + i) = rand (); umask (0); fd = creat ("conftest.mmap", 0600); if (fd < 0) return 2; if (write (fd, data, pagesize) != pagesize) return 3; close (fd); /* Next, check that the tail of a page is zero-filled. File must have non-zero length, otherwise we risk SIGBUS for entire page. */ fd2 = open ("conftest.txt", O_RDWR | O_CREAT | O_TRUNC, 0600); if (fd2 < 0) return 4; cdata2 = ""; if (write (fd2, cdata2, 1) != 1) return 5; data2 = (char *) mmap (0, pagesize, PROT_READ | PROT_WRITE, MAP_SHARED, fd2, 0L); if (data2 == MAP_FAILED) return 6; for (i = 0; i < pagesize; ++i) if (*(data2 + i)) return 7; close (fd2); if (munmap (data2, pagesize)) return 8; /* Next, try to mmap the file at a fixed address which already has something else allocated at it. If we can, also make sure that we see the same garbage. */ fd = open ("conftest.mmap", O_RDWR); if (fd < 0) return 9; if (data2 != mmap (data2, pagesize, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_FIXED, fd, 0L)) return 10; for (i = 0; i < pagesize; ++i) if (*(data + i) != *(data2 + i)) return 11; /* Finally, make sure that changes to the mapped area do not percolate back to the file as seen by read(). (This is a bug on some variants of i386 svr4.0.) */ for (i = 0; i < pagesize; ++i) *(data2 + i) = *(data2 + i) + 1; data3 = (char *) malloc (pagesize); if (!data3) return 12; if (read (fd, data3, pagesize) != pagesize) return 13; for (i = 0; i < pagesize; ++i) if (*(data + i) != *(data3 + i)) return 14; close (fd); return 0; } _ACEOF if ac_fn_c_try_run "$LINENO"; then : ac_cv_func_mmap_fixed_mapped=yes else ac_cv_func_mmap_fixed_mapped=no fi rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ conftest.$ac_objext conftest.beam conftest.$ac_ext fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_func_mmap_fixed_mapped" >&5 $as_echo "$ac_cv_func_mmap_fixed_mapped" >&6; } if test $ac_cv_func_mmap_fixed_mapped = yes; then $as_echo "#define HAVE_MMAP 1" >>confdefs.h fi rm -f conftest.mmap conftest.txt for ac_header in vfork.h do : ac_fn_c_check_header_mongrel "$LINENO" "vfork.h" "ac_cv_header_vfork_h" "$ac_includes_default" if test "x$ac_cv_header_vfork_h" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_VFORK_H 1 _ACEOF fi done for ac_func in fork vfork do : as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var" if eval test \"x\$"$as_ac_var"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_func" | $as_tr_cpp` 1 _ACEOF fi done if test "x$ac_cv_func_fork" = xyes; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for working fork" >&5 $as_echo_n "checking for working fork... " >&6; } if ${ac_cv_func_fork_works+:} false; then : $as_echo_n "(cached) " >&6 else if test "$cross_compiling" = yes; then : ac_cv_func_fork_works=cross else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ $ac_includes_default int main () { /* By Ruediger Kuhlmann. */ return fork () < 0; ; return 0; } _ACEOF if ac_fn_c_try_run "$LINENO"; then : ac_cv_func_fork_works=yes else ac_cv_func_fork_works=no fi rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ conftest.$ac_objext conftest.beam conftest.$ac_ext fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_func_fork_works" >&5 $as_echo "$ac_cv_func_fork_works" >&6; } else ac_cv_func_fork_works=$ac_cv_func_fork fi if test "x$ac_cv_func_fork_works" = xcross; then case $host in *-*-amigaos* | *-*-msdosdjgpp*) # Override, as these systems have only a dummy fork() stub ac_cv_func_fork_works=no ;; *) ac_cv_func_fork_works=yes ;; esac { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: result $ac_cv_func_fork_works guessed because of cross compilation" >&5 $as_echo "$as_me: WARNING: result $ac_cv_func_fork_works guessed because of cross compilation" >&2;} fi ac_cv_func_vfork_works=$ac_cv_func_vfork if test "x$ac_cv_func_vfork" = xyes; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for working vfork" >&5 $as_echo_n "checking for working vfork... " >&6; } if ${ac_cv_func_vfork_works+:} false; then : $as_echo_n "(cached) " >&6 else if test "$cross_compiling" = yes; then : ac_cv_func_vfork_works=cross else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Thanks to Paul Eggert for this test. */ $ac_includes_default #include #ifdef HAVE_VFORK_H # include #endif /* On some sparc systems, changes by the child to local and incoming argument registers are propagated back to the parent. The compiler is told about this with #include , but some compilers (e.g. gcc -O) don't grok . Test for this by using a static variable whose address is put into a register that is clobbered by the vfork. */ static void #ifdef __cplusplus sparc_address_test (int arg) # else sparc_address_test (arg) int arg; #endif { static pid_t child; if (!child) { child = vfork (); if (child < 0) { perror ("vfork"); _exit(2); } if (!child) { arg = getpid(); write(-1, "", 0); _exit (arg); } } } int main () { pid_t parent = getpid (); pid_t child; sparc_address_test (0); child = vfork (); if (child == 0) { /* Here is another test for sparc vfork register problems. This test uses lots of local variables, at least as many local variables as main has allocated so far including compiler temporaries. 4 locals are enough for gcc 1.40.3 on a Solaris 4.1.3 sparc, but we use 8 to be safe. A buggy compiler should reuse the register of parent for one of the local variables, since it will think that parent can't possibly be used any more in this routine. Assigning to the local variable will thus munge parent in the parent process. */ pid_t p = getpid(), p1 = getpid(), p2 = getpid(), p3 = getpid(), p4 = getpid(), p5 = getpid(), p6 = getpid(), p7 = getpid(); /* Convince the compiler that p..p7 are live; otherwise, it might use the same hardware register for all 8 local variables. */ if (p != p1 || p != p2 || p != p3 || p != p4 || p != p5 || p != p6 || p != p7) _exit(1); /* On some systems (e.g. IRIX 3.3), vfork doesn't separate parent from child file descriptors. If the child closes a descriptor before it execs or exits, this munges the parent's descriptor as well. Test for this by closing stdout in the child. */ _exit(close(fileno(stdout)) != 0); } else { int status; struct stat st; while (wait(&status) != child) ; return ( /* Was there some problem with vforking? */ child < 0 /* Did the child fail? (This shouldn't happen.) */ || status /* Did the vfork/compiler bug occur? */ || parent != getpid() /* Did the file descriptor bug occur? */ || fstat(fileno(stdout), &st) != 0 ); } } _ACEOF if ac_fn_c_try_run "$LINENO"; then : ac_cv_func_vfork_works=yes else ac_cv_func_vfork_works=no fi rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ conftest.$ac_objext conftest.beam conftest.$ac_ext fi fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_func_vfork_works" >&5 $as_echo "$ac_cv_func_vfork_works" >&6; } fi; if test "x$ac_cv_func_fork_works" = xcross; then ac_cv_func_vfork_works=$ac_cv_func_vfork { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: result $ac_cv_func_vfork_works guessed because of cross compilation" >&5 $as_echo "$as_me: WARNING: result $ac_cv_func_vfork_works guessed because of cross compilation" >&2;} fi if test "x$ac_cv_func_vfork_works" = xyes; then $as_echo "#define HAVE_WORKING_VFORK 1" >>confdefs.h else $as_echo "#define vfork fork" >>confdefs.h fi if test "x$ac_cv_func_fork_works" = xyes; then $as_echo "#define HAVE_WORKING_FORK 1" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether mbrtowc and mbstate_t are properly declared" >&5 $as_echo_n "checking whether mbrtowc and mbstate_t are properly declared... " >&6; } if ${ac_cv_func_mbrtowc+:} false; then : $as_echo_n "(cached) " >&6 else cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ #include int main () { wchar_t wc; char const s[] = ""; size_t n = 1; mbstate_t state; return ! (sizeof state && (mbrtowc) (&wc, s, n, &state)); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_func_mbrtowc=yes else ac_cv_func_mbrtowc=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_func_mbrtowc" >&5 $as_echo "$ac_cv_func_mbrtowc" >&6; } if test $ac_cv_func_mbrtowc = yes; then $as_echo "#define HAVE_MBRTOWC 1" >>confdefs.h fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for gcc compiler warnings" >&5 $as_echo_n "checking for gcc compiler warnings... " >&6; } # Check whether --enable-warnings was given. if test "${enable_warnings+set}" = set; then : enableval=$enable_warnings; if test "${enableval}" = no -o "$GCC" = no; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } WARNINGS= else { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } WARNINGS="-Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith \ -Wmissing-declarations -Wredundant-decls -Wnested-externs \ -Wsign-compare -Wreturn-type -Wswitch -Wshadow \ -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -Wformat=2" fi else if test "$GCC" = yes; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } WARNINGS="-Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith \ -Wmissing-declarations -Wredundant-decls -Wnested-externs \ -Wsign-compare -Wreturn-type -Wswitch -Wshadow \ -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -Wformat=2" else WARNINGS= { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi fi -for ac_func in strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof +for ac_func in strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale setlocale do : as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var" if eval test \"x\$"$as_ac_var"\" = x"yes"; then : cat >>confdefs.h <<_ACEOF #define `$as_echo "HAVE_$ac_func" | $as_tr_cpp` 1 _ACEOF fi done ac_fn_c_check_func "$LINENO" "getopt_long" "ac_cv_func_getopt_long" if test "x$ac_cv_func_getopt_long" = xyes; then : $as_echo "#define HAVE_GETOPT_LONG 1" >>confdefs.h else case " $LIBOBJS " in *" getopt_long.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS getopt_long.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "asprintf" "ac_cv_func_asprintf" if test "x$ac_cv_func_asprintf" = xyes; then : $as_echo "#define HAVE_ASPRINTF 1" >>confdefs.h else case " $LIBOBJS " in *" asprintf.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS asprintf.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "vasprintf" "ac_cv_func_vasprintf" if test "x$ac_cv_func_vasprintf" = xyes; then : $as_echo "#define HAVE_VASPRINTF 1" >>confdefs.h else case " $LIBOBJS " in *" vasprintf.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS vasprintf.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "strlcpy" "ac_cv_func_strlcpy" if test "x$ac_cv_func_strlcpy" = xyes; then : $as_echo "#define HAVE_STRLCPY 1" >>confdefs.h else case " $LIBOBJS " in *" strlcpy.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS strlcpy.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "strlcat" "ac_cv_func_strlcat" if test "x$ac_cv_func_strlcat" = xyes; then : $as_echo "#define HAVE_STRLCAT 1" >>confdefs.h else case " $LIBOBJS " in *" strlcat.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS strlcat.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "getline" "ac_cv_func_getline" if test "x$ac_cv_func_getline" = xyes; then : $as_echo "#define HAVE_GETLINE 1" >>confdefs.h else case " $LIBOBJS " in *" getline.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS getline.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "ctime_r" "ac_cv_func_ctime_r" if test "x$ac_cv_func_ctime_r" = xyes; then : $as_echo "#define HAVE_CTIME_R 1" >>confdefs.h else case " $LIBOBJS " in *" ctime_r.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS ctime_r.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "asctime_r" "ac_cv_func_asctime_r" if test "x$ac_cv_func_asctime_r" = xyes; then : $as_echo "#define HAVE_ASCTIME_R 1" >>confdefs.h else case " $LIBOBJS " in *" asctime_r.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS asctime_r.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "pread" "ac_cv_func_pread" if test "x$ac_cv_func_pread" = xyes; then : $as_echo "#define HAVE_PREAD 1" >>confdefs.h else case " $LIBOBJS " in *" pread.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS pread.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "strcasestr" "ac_cv_func_strcasestr" if test "x$ac_cv_func_strcasestr" = xyes; then : $as_echo "#define HAVE_STRCASESTR 1" >>confdefs.h else case " $LIBOBJS " in *" strcasestr.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS strcasestr.$ac_objext" ;; esac fi ac_fn_c_check_func "$LINENO" "fmtcheck" "ac_cv_func_fmtcheck" if test "x$ac_cv_func_fmtcheck" = xyes; then : $as_echo "#define HAVE_FMTCHECK 1" >>confdefs.h else case " $LIBOBJS " in *" fmtcheck.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS fmtcheck.$ac_objext" ;; esac fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking for gzopen in -lz" >&5 $as_echo_n "checking for gzopen in -lz... " >&6; } if ${ac_cv_lib_z_gzopen+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-lz $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char gzopen (); int main () { return gzopen (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_z_gzopen=yes else ac_cv_lib_z_gzopen=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_z_gzopen" >&5 $as_echo "$ac_cv_lib_z_gzopen" >&6; } if test "x$ac_cv_lib_z_gzopen" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_LIBZ 1 _ACEOF LIBS="-lz $LIBS" fi if test "$MINGW" = 1; then { $as_echo "$as_me:${as_lineno-$LINENO}: checking for regexec in -lgnurx" >&5 $as_echo_n "checking for regexec in -lgnurx... " >&6; } if ${ac_cv_lib_gnurx_regexec+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS LIBS="-lgnurx $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char regexec (); int main () { return regexec (); ; return 0; } _ACEOF if ac_fn_c_try_link "$LINENO"; then : ac_cv_lib_gnurx_regexec=yes else ac_cv_lib_gnurx_regexec=no fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext conftest.$ac_ext LIBS=$ac_check_lib_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_gnurx_regexec" >&5 $as_echo "$ac_cv_lib_gnurx_regexec" >&6; } if test "x$ac_cv_lib_gnurx_regexec" = xyes; then : cat >>confdefs.h <<_ACEOF #define HAVE_LIBGNURX 1 _ACEOF LIBS="-lgnurx $LIBS" else as_fn_error $? "libgnurx is required to build file(1) with MinGW" "$LINENO" 5 fi fi if test "$cross_compiling" = yes; then IS_CROSS_COMPILE_TRUE= IS_CROSS_COMPILE_FALSE='#' else IS_CROSS_COMPILE_TRUE='#' IS_CROSS_COMPILE_FALSE= fi ac_config_files="$ac_config_files Makefile src/Makefile magic/Makefile tests/Makefile doc/Makefile python/Makefile" cat >confcache <<\_ACEOF # This file is a shell script that caches the results of configure # tests run on this system so they can be shared between configure # scripts and configure runs, see configure's option --config-cache. # It is not useful on other systems. If it contains results you don't # want to keep, you may remove or edit it. # # config.status only pays attention to the cache file if you give it # the --recheck option to rerun configure. # # `ac_cv_env_foo' variables (set or unset) will be overridden when # loading this file, other *unset* `ac_cv_foo' will be assigned the # following values. _ACEOF # The following way of writing the cache mishandles newlines in values, # but we know of no workaround that is simple, portable, and efficient. # So, we kill variables containing newlines. # Ultrix sh set writes to stderr and can't be redirected directly, # and sets the high bit in the cache file unless we assign to the vars. ( for ac_var in `(set) 2>&1 | sed -n 's/^\([a-zA-Z_][a-zA-Z0-9_]*\)=.*/\1/p'`; do eval ac_val=\$$ac_var case $ac_val in #( *${as_nl}*) case $ac_var in #( *_cv_*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline" >&5 $as_echo "$as_me: WARNING: cache variable $ac_var contains a newline" >&2;} ;; esac case $ac_var in #( _ | IFS | as_nl) ;; #( BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #( *) { eval $ac_var=; unset $ac_var;} ;; esac ;; esac done (set) 2>&1 | case $as_nl`(ac_space=' '; set) 2>&1` in #( *${as_nl}ac_space=\ *) # `set' does not quote correctly, so add quotes: double-quote # substitution turns \\\\ into \\, and sed turns \\ into \. sed -n \ "s/'/'\\\\''/g; s/^\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\)=\\(.*\\)/\\1='\\2'/p" ;; #( *) # `set' quotes correctly as required by POSIX, so do not add quotes. sed -n "/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p" ;; esac | sort ) | sed ' /^ac_cv_env_/b end t clear :clear s/^\([^=]*\)=\(.*[{}].*\)$/test "${\1+set}" = set || &/ t end s/^\([^=]*\)=\(.*\)$/\1=${\1=\2}/ :end' >>confcache if diff "$cache_file" confcache >/dev/null 2>&1; then :; else if test -w "$cache_file"; then if test "x$cache_file" != "x/dev/null"; then { $as_echo "$as_me:${as_lineno-$LINENO}: updating cache $cache_file" >&5 $as_echo "$as_me: updating cache $cache_file" >&6;} if test ! -f "$cache_file" || test -h "$cache_file"; then cat confcache >"$cache_file" else case $cache_file in #( */* | ?:*) mv -f confcache "$cache_file"$$ && mv -f "$cache_file"$$ "$cache_file" ;; #( *) mv -f confcache "$cache_file" ;; esac fi fi else { $as_echo "$as_me:${as_lineno-$LINENO}: not updating unwritable cache $cache_file" >&5 $as_echo "$as_me: not updating unwritable cache $cache_file" >&6;} fi fi rm -f confcache test "x$prefix" = xNONE && prefix=$ac_default_prefix # Let make expand exec_prefix. test "x$exec_prefix" = xNONE && exec_prefix='${prefix}' DEFS=-DHAVE_CONFIG_H ac_libobjs= ac_ltlibobjs= U= for ac_i in : $LIBOBJS; do test "x$ac_i" = x: && continue # 1. Remove the extension, and $U if already installed. ac_script='s/\$U\././;s/\.o$//;s/\.obj$//' ac_i=`$as_echo "$ac_i" | sed "$ac_script"` # 2. Prepend LIBOBJDIR. When used with automake>=1.10 LIBOBJDIR # will be set to the directory where LIBOBJS objects are built. as_fn_append ac_libobjs " \${LIBOBJDIR}$ac_i\$U.$ac_objext" as_fn_append ac_ltlibobjs " \${LIBOBJDIR}$ac_i"'$U.lo' done LIBOBJS=$ac_libobjs LTLIBOBJS=$ac_ltlibobjs { $as_echo "$as_me:${as_lineno-$LINENO}: checking that generated files are newer than configure" >&5 $as_echo_n "checking that generated files are newer than configure... " >&6; } if test -n "$am_sleep_pid"; then # Hide warnings about reused PIDs. wait $am_sleep_pid 2>/dev/null fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: done" >&5 $as_echo "done" >&6; } if test -n "$EXEEXT"; then am__EXEEXT_TRUE= am__EXEEXT_FALSE='#' else am__EXEEXT_TRUE='#' am__EXEEXT_FALSE= fi if test -z "${MINGW_TRUE}" && test -z "${MINGW_FALSE}"; then as_fn_error $? "conditional \"MINGW\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${FSECT5_TRUE}" && test -z "${FSECT5_FALSE}"; then as_fn_error $? "conditional \"FSECT5\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${AMDEP_TRUE}" && test -z "${AMDEP_FALSE}"; then as_fn_error $? "conditional \"AMDEP\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${am__fastdepCC_TRUE}" && test -z "${am__fastdepCC_FALSE}"; then as_fn_error $? "conditional \"am__fastdepCC\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi if test -z "${IS_CROSS_COMPILE_TRUE}" && test -z "${IS_CROSS_COMPILE_FALSE}"; then as_fn_error $? "conditional \"IS_CROSS_COMPILE\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi : "${CONFIG_STATUS=./config.status}" ac_write_fail=0 ac_clean_files_save=$ac_clean_files ac_clean_files="$ac_clean_files $CONFIG_STATUS" { $as_echo "$as_me:${as_lineno-$LINENO}: creating $CONFIG_STATUS" >&5 $as_echo "$as_me: creating $CONFIG_STATUS" >&6;} as_write_fail=0 cat >$CONFIG_STATUS <<_ASEOF || as_write_fail=1 #! $SHELL # Generated by $as_me. # Run this file to recreate the current configuration. # Compiler output produced by configure, useful for debugging # configure, is in config.log if it exists. debug=false ac_cs_recheck=false ac_cs_silent=false SHELL=\${CONFIG_SHELL-$SHELL} export SHELL _ASEOF cat >>$CONFIG_STATUS <<\_ASEOF || as_write_fail=1 ## -------------------- ## ## M4sh Initialization. ## ## -------------------- ## # Be more Bourne compatible DUALCASE=1; export DUALCASE # for MKS sh if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : emulate sh NULLCMD=: # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which # is contrary to our usage. Disable this feature. alias -g '${1+"$@"}'='"$@"' setopt NO_GLOB_SUBST else case `(set -o) 2>/dev/null` in #( *posix*) : set -o posix ;; #( *) : ;; esac fi as_nl=' ' export as_nl # Printing a long string crashes Solaris 7 /usr/bin/printf. as_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo # Prefer a ksh shell builtin over an external printf program on Solaris, # but without wasting forks for bash or zsh. if test -z "$BASH_VERSION$ZSH_VERSION" \ && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='print -r --' as_echo_n='print -rn --' elif (test "X`printf %s $as_echo`" = "X$as_echo") 2>/dev/null; then as_echo='printf %s\n' as_echo_n='printf %s' else if test "X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`" = "X-n $as_echo"; then as_echo_body='eval /usr/ucb/echo -n "$1$as_nl"' as_echo_n='/usr/ucb/echo -n' else as_echo_body='eval expr "X$1" : "X\\(.*\\)"' as_echo_n_body='eval arg=$1; case $arg in #( *"$as_nl"*) expr "X$arg" : "X\\(.*\\)$as_nl"; arg=`expr "X$arg" : ".*$as_nl\\(.*\\)"`;; esac; expr "X$arg" : "X\\(.*\\)" | tr -d "$as_nl" ' export as_echo_n_body as_echo_n='sh -c $as_echo_n_body as_echo' fi export as_echo_body as_echo='sh -c $as_echo_body as_echo' fi # The user is always right. if test "${PATH_SEPARATOR+set}" != set; then PATH_SEPARATOR=: (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && { (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 || PATH_SEPARATOR=';' } fi # IFS # We need space, tab and new line, in precisely that order. Quoting is # there to prevent editors from complaining about space-tab. # (If _AS_PATH_WALK were called with IFS unset, it would disable word # splitting by setting IFS to empty value.) IFS=" "" $as_nl" # Find who we are. Look in the path if we contain no directory separator. as_myself= case $0 in #(( *[\\/]* ) as_myself=$0 ;; *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. test -r "$as_dir/$0" && as_myself=$as_dir/$0 && break done IFS=$as_save_IFS ;; esac # We did not find ourselves, most probably we were run as `sh COMMAND' # in which case we are not to be found in the path. if test "x$as_myself" = x; then as_myself=$0 fi if test ! -f "$as_myself"; then $as_echo "$as_myself: error: cannot find myself; rerun with an absolute file name" >&2 exit 1 fi # Unset variables that we do not need and which cause bugs (e.g. in # pre-3.0 UWIN ksh). But do not cause bugs in bash 2.01; the "|| exit 1" # suppresses any "Segmentation fault" message there. '((' could # trigger a bug in pdksh 5.2.14. for as_var in BASH_ENV ENV MAIL MAILPATH do eval test x\${$as_var+set} = xset \ && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || : done PS1='$ ' PS2='> ' PS4='+ ' # NLS nuisances. LC_ALL=C export LC_ALL LANGUAGE=C export LANGUAGE # CDPATH. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH # as_fn_error STATUS ERROR [LINENO LOG_FD] # ---------------------------------------- # Output "`basename $0`: error: ERROR" to stderr. If LINENO and LOG_FD are # provided, also output the error to LOG_FD, referencing LINENO. Then exit the # script with STATUS, using 1 if that was 0. as_fn_error () { as_status=$1; test $as_status -eq 0 && as_status=1 if test "$4"; then as_lineno=${as_lineno-"$3"} as_lineno_stack=as_lineno_stack=$as_lineno_stack $as_echo "$as_me:${as_lineno-$LINENO}: error: $2" >&$4 fi $as_echo "$as_me: error: $2" >&2 as_fn_exit $as_status } # as_fn_error # as_fn_set_status STATUS # ----------------------- # Set $? to STATUS, without forking. as_fn_set_status () { return $1 } # as_fn_set_status # as_fn_exit STATUS # ----------------- # Exit the shell with STATUS, even in a "trap 0" or "set -e" context. as_fn_exit () { set +e as_fn_set_status $1 exit $1 } # as_fn_exit # as_fn_unset VAR # --------------- # Portably unset VAR. as_fn_unset () { { eval $1=; unset $1;} } as_unset=as_fn_unset # as_fn_append VAR VALUE # ---------------------- # Append the text in VALUE to the end of the definition contained in VAR. Take # advantage of any shell optimizations that allow amortized linear growth over # repeated appends, instead of the typical quadratic growth present in naive # implementations. if (eval "as_var=1; as_var+=2; test x\$as_var = x12") 2>/dev/null; then : eval 'as_fn_append () { eval $1+=\$2 }' else as_fn_append () { eval $1=\$$1\$2 } fi # as_fn_append # as_fn_arith ARG... # ------------------ # Perform arithmetic evaluation on the ARGs, and store the result in the # global $as_val. Take advantage of shells that can avoid forks. The arguments # must be portable across $(()) and expr. if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then : eval 'as_fn_arith () { as_val=$(( $* )) }' else as_fn_arith () { as_val=`expr "$@" || test $? -eq 1` } fi # as_fn_arith if expr a : '\(a\)' >/dev/null 2>&1 && test "X`expr 00001 : '.*\(...\)'`" = X001; then as_expr=expr else as_expr=false fi if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then as_basename=basename else as_basename=false fi if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then as_dirname=dirname else as_dirname=false fi as_me=`$as_basename -- "$0" || $as_expr X/"$0" : '.*/\([^/][^/]*\)/*$' \| \ X"$0" : 'X\(//\)$' \| \ X"$0" : 'X\(/\)' \| . 2>/dev/null || $as_echo X/"$0" | sed '/^.*\/\([^/][^/]*\)\/*$/{ s//\1/ q } /^X\/\(\/\/\)$/{ s//\1/ q } /^X\/\(\/\).*/{ s//\1/ q } s/.*/./; q'` # Avoid depending upon Character Ranges. as_cr_letters='abcdefghijklmnopqrstuvwxyz' as_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ' as_cr_Letters=$as_cr_letters$as_cr_LETTERS as_cr_digits='0123456789' as_cr_alnum=$as_cr_Letters$as_cr_digits ECHO_C= ECHO_N= ECHO_T= case `echo -n x` in #((((( -n*) case `echo 'xy\c'` in *c*) ECHO_T=' ';; # ECHO_T is single tab character. xy) ECHO_C='\c';; *) echo `echo ksh88 bug on AIX 6.1` > /dev/null ECHO_T=' ';; esac;; *) ECHO_N='-n';; esac rm -f conf$$ conf$$.exe conf$$.file if test -d conf$$.dir; then rm -f conf$$.dir/conf$$.file else rm -f conf$$.dir mkdir conf$$.dir 2>/dev/null fi if (echo >conf$$.file) 2>/dev/null; then if ln -s conf$$.file conf$$ 2>/dev/null; then as_ln_s='ln -s' # ... but there are two gotchas: # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail. # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable. # In both cases, we have to default to `cp -pR'. ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe || as_ln_s='cp -pR' elif ln conf$$.file conf$$ 2>/dev/null; then as_ln_s=ln else as_ln_s='cp -pR' fi else as_ln_s='cp -pR' fi rm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file rmdir conf$$.dir 2>/dev/null # as_fn_mkdir_p # ------------- # Create "$as_dir" as a directory, including parents if necessary. as_fn_mkdir_p () { case $as_dir in #( -*) as_dir=./$as_dir;; esac test -d "$as_dir" || eval $as_mkdir_p || { as_dirs= while :; do case $as_dir in #( *\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'( *) as_qdir=$as_dir;; esac as_dirs="'$as_qdir' $as_dirs" as_dir=`$as_dirname -- "$as_dir" || $as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$as_dir" : 'X\(//\)[^/]' \| \ X"$as_dir" : 'X\(//\)$' \| \ X"$as_dir" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$as_dir" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` test -d "$as_dir" && break done test -z "$as_dirs" || eval "mkdir $as_dirs" } || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir" } # as_fn_mkdir_p if mkdir -p . 2>/dev/null; then as_mkdir_p='mkdir -p "$as_dir"' else test -d ./-p && rmdir ./-p as_mkdir_p=false fi # as_fn_executable_p FILE # ----------------------- # Test if FILE is an executable regular file. as_fn_executable_p () { test -f "$1" && test -x "$1" } # as_fn_executable_p as_test_x='test -x' as_executable_p=as_fn_executable_p # Sed expression to map a string onto a valid CPP name. as_tr_cpp="eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'" # Sed expression to map a string onto a valid variable name. as_tr_sh="eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'" exec 6>&1 ## ----------------------------------- ## ## Main body of $CONFIG_STATUS script. ## ## ----------------------------------- ## _ASEOF test $as_write_fail = 0 && chmod +x $CONFIG_STATUS || ac_write_fail=1 cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # Save the log message, to keep $0 and so on meaningful, and to # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by file $as_me 5.19, which was +This file was extended by file $as_me 5.22, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES CONFIG_HEADERS = $CONFIG_HEADERS CONFIG_LINKS = $CONFIG_LINKS CONFIG_COMMANDS = $CONFIG_COMMANDS $ $0 $@ on `(hostname || uname -n) 2>/dev/null | sed 1q` " _ACEOF case $ac_config_files in *" "*) set x $ac_config_files; shift; ac_config_files=$*;; esac case $ac_config_headers in *" "*) set x $ac_config_headers; shift; ac_config_headers=$*;; esac cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 # Files that config.status was made for. config_files="$ac_config_files" config_headers="$ac_config_headers" config_commands="$ac_config_commands" _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 ac_cs_usage="\ \`$as_me' instantiates files and other configuration actions from templates according to the current configuration. Unless the files and actions are specified as TAGs, all are instantiated by default. Usage: $0 [OPTION]... [TAG]... -h, --help print this help, then exit -V, --version print version number and configuration settings, then exit --config print configuration, then exit -q, --quiet, --silent do not print progress messages -d, --debug don't remove temporary files --recheck update $as_me by reconfiguring in the same conditions --file=FILE[:TEMPLATE] instantiate the configuration file FILE --header=FILE[:TEMPLATE] instantiate the configuration header FILE Configuration files: $config_files Configuration headers: $config_headers Configuration commands: $config_commands Report bugs to ." _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -file config.status 5.19 +file config.status 5.22 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" Copyright (C) 2012 Free Software Foundation, Inc. This config.status script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it." ac_pwd='$ac_pwd' srcdir='$srcdir' INSTALL='$INSTALL' MKDIR_P='$MKDIR_P' AWK='$AWK' test -n "\$AWK" || AWK=awk _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # The default lists apply if the user does not specify any file. ac_need_defaults=: while test $# != 0 do case $1 in --*=?*) ac_option=`expr "X$1" : 'X\([^=]*\)='` ac_optarg=`expr "X$1" : 'X[^=]*=\(.*\)'` ac_shift=: ;; --*=) ac_option=`expr "X$1" : 'X\([^=]*\)='` ac_optarg= ac_shift=: ;; *) ac_option=$1 ac_optarg=$2 ac_shift=shift ;; esac case $ac_option in # Handling of the options. -recheck | --recheck | --rechec | --reche | --rech | --rec | --re | --r) ac_cs_recheck=: ;; --version | --versio | --versi | --vers | --ver | --ve | --v | -V ) $as_echo "$ac_cs_version"; exit ;; --config | --confi | --conf | --con | --co | --c ) $as_echo "$ac_cs_config"; exit ;; --debug | --debu | --deb | --de | --d | -d ) debug=: ;; --file | --fil | --fi | --f ) $ac_shift case $ac_optarg in *\'*) ac_optarg=`$as_echo "$ac_optarg" | sed "s/'/'\\\\\\\\''/g"` ;; '') as_fn_error $? "missing file argument" ;; esac as_fn_append CONFIG_FILES " '$ac_optarg'" ac_need_defaults=false;; --header | --heade | --head | --hea ) $ac_shift case $ac_optarg in *\'*) ac_optarg=`$as_echo "$ac_optarg" | sed "s/'/'\\\\\\\\''/g"` ;; esac as_fn_append CONFIG_HEADERS " '$ac_optarg'" ac_need_defaults=false;; --he | --h) # Conflict between --help and --header as_fn_error $? "ambiguous option: \`$1' Try \`$0 --help' for more information.";; --help | --hel | -h ) $as_echo "$ac_cs_usage"; exit ;; -q | -quiet | --quiet | --quie | --qui | --qu | --q \ | -silent | --silent | --silen | --sile | --sil | --si | --s) ac_cs_silent=: ;; # This is an error. -*) as_fn_error $? "unrecognized option: \`$1' Try \`$0 --help' for more information." ;; *) as_fn_append ac_config_targets " $1" ac_need_defaults=false ;; esac shift done ac_configure_extra_args= if $ac_cs_silent; then exec 6>/dev/null ac_configure_extra_args="$ac_configure_extra_args --silent" fi _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 if \$ac_cs_recheck; then set X $SHELL '$0' $ac_configure_args \$ac_configure_extra_args --no-create --no-recursion shift \$as_echo "running CONFIG_SHELL=$SHELL \$*" >&6 CONFIG_SHELL='$SHELL' export CONFIG_SHELL exec "\$@" fi _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 exec 5>>config.log { echo sed 'h;s/./-/g;s/^.../## /;s/...$/ ##/;p;x;p;x' <<_ASBOX ## Running $as_me. ## _ASBOX $as_echo "$ac_log" } >&5 _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 # # INIT-COMMANDS # AMDEP_TRUE="$AMDEP_TRUE" ac_aux_dir="$ac_aux_dir" # The HP-UX ksh and POSIX shell print the target directory to stdout # if CDPATH is set. (unset CDPATH) >/dev/null 2>&1 && unset CDPATH sed_quote_subst='$sed_quote_subst' double_quote_subst='$double_quote_subst' delay_variable_subst='$delay_variable_subst' macro_version='`$ECHO "$macro_version" | $SED "$delay_single_quote_subst"`' macro_revision='`$ECHO "$macro_revision" | $SED "$delay_single_quote_subst"`' enable_static='`$ECHO "$enable_static" | $SED "$delay_single_quote_subst"`' pic_mode='`$ECHO "$pic_mode" | $SED "$delay_single_quote_subst"`' enable_shared='`$ECHO "$enable_shared" | $SED "$delay_single_quote_subst"`' enable_fast_install='`$ECHO "$enable_fast_install" | $SED "$delay_single_quote_subst"`' SHELL='`$ECHO "$SHELL" | $SED "$delay_single_quote_subst"`' ECHO='`$ECHO "$ECHO" | $SED "$delay_single_quote_subst"`' PATH_SEPARATOR='`$ECHO "$PATH_SEPARATOR" | $SED "$delay_single_quote_subst"`' host_alias='`$ECHO "$host_alias" | $SED "$delay_single_quote_subst"`' host='`$ECHO "$host" | $SED "$delay_single_quote_subst"`' host_os='`$ECHO "$host_os" | $SED "$delay_single_quote_subst"`' build_alias='`$ECHO "$build_alias" | $SED "$delay_single_quote_subst"`' build='`$ECHO "$build" | $SED "$delay_single_quote_subst"`' build_os='`$ECHO "$build_os" | $SED "$delay_single_quote_subst"`' SED='`$ECHO "$SED" | $SED "$delay_single_quote_subst"`' Xsed='`$ECHO "$Xsed" | $SED "$delay_single_quote_subst"`' GREP='`$ECHO "$GREP" | $SED "$delay_single_quote_subst"`' EGREP='`$ECHO "$EGREP" | $SED "$delay_single_quote_subst"`' FGREP='`$ECHO "$FGREP" | $SED "$delay_single_quote_subst"`' LD='`$ECHO "$LD" | $SED "$delay_single_quote_subst"`' NM='`$ECHO "$NM" | $SED "$delay_single_quote_subst"`' LN_S='`$ECHO "$LN_S" | $SED "$delay_single_quote_subst"`' max_cmd_len='`$ECHO "$max_cmd_len" | $SED "$delay_single_quote_subst"`' ac_objext='`$ECHO "$ac_objext" | $SED "$delay_single_quote_subst"`' exeext='`$ECHO "$exeext" | $SED "$delay_single_quote_subst"`' lt_unset='`$ECHO "$lt_unset" | $SED "$delay_single_quote_subst"`' lt_SP2NL='`$ECHO "$lt_SP2NL" | $SED "$delay_single_quote_subst"`' lt_NL2SP='`$ECHO "$lt_NL2SP" | $SED "$delay_single_quote_subst"`' lt_cv_to_host_file_cmd='`$ECHO "$lt_cv_to_host_file_cmd" | $SED "$delay_single_quote_subst"`' lt_cv_to_tool_file_cmd='`$ECHO "$lt_cv_to_tool_file_cmd" | $SED "$delay_single_quote_subst"`' reload_flag='`$ECHO "$reload_flag" | $SED "$delay_single_quote_subst"`' reload_cmds='`$ECHO "$reload_cmds" | $SED "$delay_single_quote_subst"`' OBJDUMP='`$ECHO "$OBJDUMP" | $SED "$delay_single_quote_subst"`' deplibs_check_method='`$ECHO "$deplibs_check_method" | $SED "$delay_single_quote_subst"`' file_magic_cmd='`$ECHO "$file_magic_cmd" | $SED "$delay_single_quote_subst"`' file_magic_glob='`$ECHO "$file_magic_glob" | $SED "$delay_single_quote_subst"`' want_nocaseglob='`$ECHO "$want_nocaseglob" | $SED "$delay_single_quote_subst"`' DLLTOOL='`$ECHO "$DLLTOOL" | $SED "$delay_single_quote_subst"`' sharedlib_from_linklib_cmd='`$ECHO "$sharedlib_from_linklib_cmd" | $SED "$delay_single_quote_subst"`' AR='`$ECHO "$AR" | $SED "$delay_single_quote_subst"`' AR_FLAGS='`$ECHO "$AR_FLAGS" | $SED "$delay_single_quote_subst"`' archiver_list_spec='`$ECHO "$archiver_list_spec" | $SED "$delay_single_quote_subst"`' STRIP='`$ECHO "$STRIP" | $SED "$delay_single_quote_subst"`' RANLIB='`$ECHO "$RANLIB" | $SED "$delay_single_quote_subst"`' old_postinstall_cmds='`$ECHO "$old_postinstall_cmds" | $SED "$delay_single_quote_subst"`' old_postuninstall_cmds='`$ECHO "$old_postuninstall_cmds" | $SED "$delay_single_quote_subst"`' old_archive_cmds='`$ECHO "$old_archive_cmds" | $SED "$delay_single_quote_subst"`' lock_old_archive_extraction='`$ECHO "$lock_old_archive_extraction" | $SED "$delay_single_quote_subst"`' CC='`$ECHO "$CC" | $SED "$delay_single_quote_subst"`' CFLAGS='`$ECHO "$CFLAGS" | $SED "$delay_single_quote_subst"`' compiler='`$ECHO "$compiler" | $SED "$delay_single_quote_subst"`' GCC='`$ECHO "$GCC" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_pipe='`$ECHO "$lt_cv_sys_global_symbol_pipe" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_to_cdecl='`$ECHO "$lt_cv_sys_global_symbol_to_cdecl" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_to_c_name_address='`$ECHO "$lt_cv_sys_global_symbol_to_c_name_address" | $SED "$delay_single_quote_subst"`' lt_cv_sys_global_symbol_to_c_name_address_lib_prefix='`$ECHO "$lt_cv_sys_global_symbol_to_c_name_address_lib_prefix" | $SED "$delay_single_quote_subst"`' nm_file_list_spec='`$ECHO "$nm_file_list_spec" | $SED "$delay_single_quote_subst"`' lt_sysroot='`$ECHO "$lt_sysroot" | $SED "$delay_single_quote_subst"`' objdir='`$ECHO "$objdir" | $SED "$delay_single_quote_subst"`' MAGIC_CMD='`$ECHO "$MAGIC_CMD" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_no_builtin_flag='`$ECHO "$lt_prog_compiler_no_builtin_flag" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_pic='`$ECHO "$lt_prog_compiler_pic" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_wl='`$ECHO "$lt_prog_compiler_wl" | $SED "$delay_single_quote_subst"`' lt_prog_compiler_static='`$ECHO "$lt_prog_compiler_static" | $SED "$delay_single_quote_subst"`' lt_cv_prog_compiler_c_o='`$ECHO "$lt_cv_prog_compiler_c_o" | $SED "$delay_single_quote_subst"`' need_locks='`$ECHO "$need_locks" | $SED "$delay_single_quote_subst"`' MANIFEST_TOOL='`$ECHO "$MANIFEST_TOOL" | $SED "$delay_single_quote_subst"`' DSYMUTIL='`$ECHO "$DSYMUTIL" | $SED "$delay_single_quote_subst"`' NMEDIT='`$ECHO "$NMEDIT" | $SED "$delay_single_quote_subst"`' LIPO='`$ECHO "$LIPO" | $SED "$delay_single_quote_subst"`' OTOOL='`$ECHO "$OTOOL" | $SED "$delay_single_quote_subst"`' OTOOL64='`$ECHO "$OTOOL64" | $SED "$delay_single_quote_subst"`' libext='`$ECHO "$libext" | $SED "$delay_single_quote_subst"`' shrext_cmds='`$ECHO "$shrext_cmds" | $SED "$delay_single_quote_subst"`' extract_expsyms_cmds='`$ECHO "$extract_expsyms_cmds" | $SED "$delay_single_quote_subst"`' archive_cmds_need_lc='`$ECHO "$archive_cmds_need_lc" | $SED "$delay_single_quote_subst"`' enable_shared_with_static_runtimes='`$ECHO "$enable_shared_with_static_runtimes" | $SED "$delay_single_quote_subst"`' export_dynamic_flag_spec='`$ECHO "$export_dynamic_flag_spec" | $SED "$delay_single_quote_subst"`' whole_archive_flag_spec='`$ECHO "$whole_archive_flag_spec" | $SED "$delay_single_quote_subst"`' compiler_needs_object='`$ECHO "$compiler_needs_object" | $SED "$delay_single_quote_subst"`' old_archive_from_new_cmds='`$ECHO "$old_archive_from_new_cmds" | $SED "$delay_single_quote_subst"`' old_archive_from_expsyms_cmds='`$ECHO "$old_archive_from_expsyms_cmds" | $SED "$delay_single_quote_subst"`' archive_cmds='`$ECHO "$archive_cmds" | $SED "$delay_single_quote_subst"`' archive_expsym_cmds='`$ECHO "$archive_expsym_cmds" | $SED "$delay_single_quote_subst"`' module_cmds='`$ECHO "$module_cmds" | $SED "$delay_single_quote_subst"`' module_expsym_cmds='`$ECHO "$module_expsym_cmds" | $SED "$delay_single_quote_subst"`' with_gnu_ld='`$ECHO "$with_gnu_ld" | $SED "$delay_single_quote_subst"`' allow_undefined_flag='`$ECHO "$allow_undefined_flag" | $SED "$delay_single_quote_subst"`' no_undefined_flag='`$ECHO "$no_undefined_flag" | $SED "$delay_single_quote_subst"`' hardcode_libdir_flag_spec='`$ECHO "$hardcode_libdir_flag_spec" | $SED "$delay_single_quote_subst"`' hardcode_libdir_separator='`$ECHO "$hardcode_libdir_separator" | $SED "$delay_single_quote_subst"`' hardcode_direct='`$ECHO "$hardcode_direct" | $SED "$delay_single_quote_subst"`' hardcode_direct_absolute='`$ECHO "$hardcode_direct_absolute" | $SED "$delay_single_quote_subst"`' hardcode_minus_L='`$ECHO "$hardcode_minus_L" | $SED "$delay_single_quote_subst"`' hardcode_shlibpath_var='`$ECHO "$hardcode_shlibpath_var" | $SED "$delay_single_quote_subst"`' hardcode_automatic='`$ECHO "$hardcode_automatic" | $SED "$delay_single_quote_subst"`' inherit_rpath='`$ECHO "$inherit_rpath" | $SED "$delay_single_quote_subst"`' link_all_deplibs='`$ECHO "$link_all_deplibs" | $SED "$delay_single_quote_subst"`' always_export_symbols='`$ECHO "$always_export_symbols" | $SED "$delay_single_quote_subst"`' export_symbols_cmds='`$ECHO "$export_symbols_cmds" | $SED "$delay_single_quote_subst"`' exclude_expsyms='`$ECHO "$exclude_expsyms" | $SED "$delay_single_quote_subst"`' include_expsyms='`$ECHO "$include_expsyms" | $SED "$delay_single_quote_subst"`' prelink_cmds='`$ECHO "$prelink_cmds" | $SED "$delay_single_quote_subst"`' postlink_cmds='`$ECHO "$postlink_cmds" | $SED "$delay_single_quote_subst"`' file_list_spec='`$ECHO "$file_list_spec" | $SED "$delay_single_quote_subst"`' variables_saved_for_relink='`$ECHO "$variables_saved_for_relink" | $SED "$delay_single_quote_subst"`' need_lib_prefix='`$ECHO "$need_lib_prefix" | $SED "$delay_single_quote_subst"`' need_version='`$ECHO "$need_version" | $SED "$delay_single_quote_subst"`' version_type='`$ECHO "$version_type" | $SED "$delay_single_quote_subst"`' runpath_var='`$ECHO "$runpath_var" | $SED "$delay_single_quote_subst"`' shlibpath_var='`$ECHO "$shlibpath_var" | $SED "$delay_single_quote_subst"`' shlibpath_overrides_runpath='`$ECHO "$shlibpath_overrides_runpath" | $SED "$delay_single_quote_subst"`' libname_spec='`$ECHO "$libname_spec" | $SED "$delay_single_quote_subst"`' library_names_spec='`$ECHO "$library_names_spec" | $SED "$delay_single_quote_subst"`' soname_spec='`$ECHO "$soname_spec" | $SED "$delay_single_quote_subst"`' install_override_mode='`$ECHO "$install_override_mode" | $SED "$delay_single_quote_subst"`' postinstall_cmds='`$ECHO "$postinstall_cmds" | $SED "$delay_single_quote_subst"`' postuninstall_cmds='`$ECHO "$postuninstall_cmds" | $SED "$delay_single_quote_subst"`' finish_cmds='`$ECHO "$finish_cmds" | $SED "$delay_single_quote_subst"`' finish_eval='`$ECHO "$finish_eval" | $SED "$delay_single_quote_subst"`' hardcode_into_libs='`$ECHO "$hardcode_into_libs" | $SED "$delay_single_quote_subst"`' sys_lib_search_path_spec='`$ECHO "$sys_lib_search_path_spec" | $SED "$delay_single_quote_subst"`' sys_lib_dlsearch_path_spec='`$ECHO "$sys_lib_dlsearch_path_spec" | $SED "$delay_single_quote_subst"`' hardcode_action='`$ECHO "$hardcode_action" | $SED "$delay_single_quote_subst"`' enable_dlopen='`$ECHO "$enable_dlopen" | $SED "$delay_single_quote_subst"`' enable_dlopen_self='`$ECHO "$enable_dlopen_self" | $SED "$delay_single_quote_subst"`' enable_dlopen_self_static='`$ECHO "$enable_dlopen_self_static" | $SED "$delay_single_quote_subst"`' old_striplib='`$ECHO "$old_striplib" | $SED "$delay_single_quote_subst"`' striplib='`$ECHO "$striplib" | $SED "$delay_single_quote_subst"`' LTCC='$LTCC' LTCFLAGS='$LTCFLAGS' compiler='$compiler_DEFAULT' # A function that is used when there is no print builtin or printf. func_fallback_echo () { eval 'cat <<_LTECHO_EOF \$1 _LTECHO_EOF' } # Quote evaled strings. for var in SHELL \ ECHO \ PATH_SEPARATOR \ SED \ GREP \ EGREP \ FGREP \ LD \ NM \ LN_S \ lt_SP2NL \ lt_NL2SP \ reload_flag \ OBJDUMP \ deplibs_check_method \ file_magic_cmd \ file_magic_glob \ want_nocaseglob \ DLLTOOL \ sharedlib_from_linklib_cmd \ AR \ AR_FLAGS \ archiver_list_spec \ STRIP \ RANLIB \ CC \ CFLAGS \ compiler \ lt_cv_sys_global_symbol_pipe \ lt_cv_sys_global_symbol_to_cdecl \ lt_cv_sys_global_symbol_to_c_name_address \ lt_cv_sys_global_symbol_to_c_name_address_lib_prefix \ nm_file_list_spec \ lt_prog_compiler_no_builtin_flag \ lt_prog_compiler_pic \ lt_prog_compiler_wl \ lt_prog_compiler_static \ lt_cv_prog_compiler_c_o \ need_locks \ MANIFEST_TOOL \ DSYMUTIL \ NMEDIT \ LIPO \ OTOOL \ OTOOL64 \ shrext_cmds \ export_dynamic_flag_spec \ whole_archive_flag_spec \ compiler_needs_object \ with_gnu_ld \ allow_undefined_flag \ no_undefined_flag \ hardcode_libdir_flag_spec \ hardcode_libdir_separator \ exclude_expsyms \ include_expsyms \ file_list_spec \ variables_saved_for_relink \ libname_spec \ library_names_spec \ soname_spec \ install_override_mode \ finish_eval \ old_striplib \ striplib; do case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in *[\\\\\\\`\\"\\\$]*) eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED \\"\\\$sed_quote_subst\\"\\\`\\\\\\"" ;; *) eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" ;; esac done # Double-quote double-evaled strings. for var in reload_cmds \ old_postinstall_cmds \ old_postuninstall_cmds \ old_archive_cmds \ extract_expsyms_cmds \ old_archive_from_new_cmds \ old_archive_from_expsyms_cmds \ archive_cmds \ archive_expsym_cmds \ module_cmds \ module_expsym_cmds \ export_symbols_cmds \ prelink_cmds \ postlink_cmds \ postinstall_cmds \ postuninstall_cmds \ finish_cmds \ sys_lib_search_path_spec \ sys_lib_dlsearch_path_spec; do case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in *[\\\\\\\`\\"\\\$]*) eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED -e \\"\\\$double_quote_subst\\" -e \\"\\\$sed_quote_subst\\" -e \\"\\\$delay_variable_subst\\"\\\`\\\\\\"" ;; *) eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" ;; esac done ac_aux_dir='$ac_aux_dir' xsi_shell='$xsi_shell' lt_shell_append='$lt_shell_append' # See if we are running on zsh, and set the options which allow our # commands through without removal of \ escapes INIT. if test -n "\${ZSH_VERSION+set}" ; then setopt NO_GLOB_SUBST fi PACKAGE='$PACKAGE' VERSION='$VERSION' TIMESTAMP='$TIMESTAMP' RM='$RM' ofile='$ofile' _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # Handling of arguments. for ac_config_target in $ac_config_targets do case $ac_config_target in "config.h") CONFIG_HEADERS="$CONFIG_HEADERS config.h" ;; "depfiles") CONFIG_COMMANDS="$CONFIG_COMMANDS depfiles" ;; "libtool") CONFIG_COMMANDS="$CONFIG_COMMANDS libtool" ;; "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;; "src/Makefile") CONFIG_FILES="$CONFIG_FILES src/Makefile" ;; "magic/Makefile") CONFIG_FILES="$CONFIG_FILES magic/Makefile" ;; "tests/Makefile") CONFIG_FILES="$CONFIG_FILES tests/Makefile" ;; "doc/Makefile") CONFIG_FILES="$CONFIG_FILES doc/Makefile" ;; "python/Makefile") CONFIG_FILES="$CONFIG_FILES python/Makefile" ;; *) as_fn_error $? "invalid argument: \`$ac_config_target'" "$LINENO" 5;; esac done # If the user did not use the arguments to specify the items to instantiate, # then the envvar interface is used. Set only those that are not. # We use the long form for the default assignment because of an extremely # bizarre bug on SunOS 4.1.3. if $ac_need_defaults; then test "${CONFIG_FILES+set}" = set || CONFIG_FILES=$config_files test "${CONFIG_HEADERS+set}" = set || CONFIG_HEADERS=$config_headers test "${CONFIG_COMMANDS+set}" = set || CONFIG_COMMANDS=$config_commands fi # Have a temporary directory for convenience. Make it in the build tree # simply because there is no reason against having it here, and in addition, # creating and moving files from /tmp can sometimes cause problems. # Hook for its removal unless debugging. # Note that there is a small window in which the directory will not be cleaned: # after its creation but before its name has been assigned to `$tmp'. $debug || { tmp= ac_tmp= trap 'exit_status=$? : "${ac_tmp:=$tmp}" { test ! -d "$ac_tmp" || rm -fr "$ac_tmp"; } && exit $exit_status ' 0 trap 'as_fn_exit 1' 1 2 13 15 } # Create a (secure) tmp directory for tmp files. { tmp=`(umask 077 && mktemp -d "./confXXXXXX") 2>/dev/null` && test -d "$tmp" } || { tmp=./conf$$-$RANDOM (umask 077 && mkdir "$tmp") } || as_fn_error $? "cannot create a temporary directory in ." "$LINENO" 5 ac_tmp=$tmp # Set up the scripts for CONFIG_FILES section. # No need to generate them if there are no CONFIG_FILES. # This happens for instance with `./config.status config.h'. if test -n "$CONFIG_FILES"; then ac_cr=`echo X | tr X '\015'` # On cygwin, bash can eat \r inside `` if the user requested igncr. # But we know of no other shell where ac_cr would be empty at this # point, so we can use a bashism as a fallback. if test "x$ac_cr" = x; then eval ac_cr=\$\'\\r\' fi ac_cs_awk_cr=`$AWK 'BEGIN { print "a\rb" }' /dev/null` if test "$ac_cs_awk_cr" = "a${ac_cr}b"; then ac_cs_awk_cr='\\r' else ac_cs_awk_cr=$ac_cr fi echo 'BEGIN {' >"$ac_tmp/subs1.awk" && _ACEOF { echo "cat >conf$$subs.awk <<_ACEOF" && echo "$ac_subst_vars" | sed 's/.*/&!$&$ac_delim/' && echo "_ACEOF" } >conf$$subs.sh || as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 ac_delim_num=`echo "$ac_subst_vars" | grep -c '^'` ac_delim='%!_!# ' for ac_last_try in false false false false false :; do . ./conf$$subs.sh || as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 ac_delim_n=`sed -n "s/.*$ac_delim\$/X/p" conf$$subs.awk | grep -c X` if test $ac_delim_n = $ac_delim_num; then break elif $ac_last_try; then as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 else ac_delim="$ac_delim!$ac_delim _$ac_delim!! " fi done rm -f conf$$subs.sh cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 cat >>"\$ac_tmp/subs1.awk" <<\\_ACAWK && _ACEOF sed -n ' h s/^/S["/; s/!.*/"]=/ p g s/^[^!]*!// :repl t repl s/'"$ac_delim"'$// t delim :nl h s/\(.\{148\}\)..*/\1/ t more1 s/["\\]/\\&/g; s/^/"/; s/$/\\n"\\/ p n b repl :more1 s/["\\]/\\&/g; s/^/"/; s/$/"\\/ p g s/.\{148\}// t nl :delim h s/\(.\{148\}\)..*/\1/ t more2 s/["\\]/\\&/g; s/^/"/; s/$/"/ p b :more2 s/["\\]/\\&/g; s/^/"/; s/$/"\\/ p g s/.\{148\}// t delim ' >$CONFIG_STATUS || ac_write_fail=1 rm -f conf$$subs.awk cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 _ACAWK cat >>"\$ac_tmp/subs1.awk" <<_ACAWK && for (key in S) S_is_set[key] = 1 FS = "" } { line = $ 0 nfields = split(line, field, "@") substed = 0 len = length(field[1]) for (i = 2; i < nfields; i++) { key = field[i] keylen = length(key) if (S_is_set[key]) { value = S[key] line = substr(line, 1, len) "" value "" substr(line, len + keylen + 3) len += length(value) + length(field[++i]) substed = 1 } else len += 1 + keylen } print line } _ACAWK _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 if sed "s/$ac_cr//" < /dev/null > /dev/null 2>&1; then sed "s/$ac_cr\$//; s/$ac_cr/$ac_cs_awk_cr/g" else cat fi < "$ac_tmp/subs1.awk" > "$ac_tmp/subs.awk" \ || as_fn_error $? "could not setup config files machinery" "$LINENO" 5 _ACEOF # VPATH may cause trouble with some makes, so we remove sole $(srcdir), # ${srcdir} and @srcdir@ entries from VPATH if srcdir is ".", strip leading and # trailing colons and then remove the whole line if VPATH becomes empty # (actually we leave an empty line to preserve line numbers). if test "x$srcdir" = x.; then ac_vpsub='/^[ ]*VPATH[ ]*=[ ]*/{ h s/// s/^/:/ s/[ ]*$/:/ s/:\$(srcdir):/:/g s/:\${srcdir}:/:/g s/:@srcdir@:/:/g s/^:*// s/:*$// x s/\(=[ ]*\).*/\1/ G s/\n// s/^[^=]*=[ ]*$// }' fi cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 fi # test -n "$CONFIG_FILES" # Set up the scripts for CONFIG_HEADERS section. # No need to generate them if there are no CONFIG_HEADERS. # This happens for instance with `./config.status Makefile'. if test -n "$CONFIG_HEADERS"; then cat >"$ac_tmp/defines.awk" <<\_ACAWK || BEGIN { _ACEOF # Transform confdefs.h into an awk script `defines.awk', embedded as # here-document in config.status, that substitutes the proper values into # config.h.in to produce config.h. # Create a delimiter string that does not exist in confdefs.h, to ease # handling of long lines. ac_delim='%!_!# ' for ac_last_try in false false :; do ac_tt=`sed -n "/$ac_delim/p" confdefs.h` if test -z "$ac_tt"; then break elif $ac_last_try; then as_fn_error $? "could not make $CONFIG_HEADERS" "$LINENO" 5 else ac_delim="$ac_delim!$ac_delim _$ac_delim!! " fi done # For the awk script, D is an array of macro values keyed by name, # likewise P contains macro parameters if any. Preserve backslash # newline sequences. ac_word_re=[_$as_cr_Letters][_$as_cr_alnum]* sed -n ' s/.\{148\}/&'"$ac_delim"'/g t rset :rset s/^[ ]*#[ ]*define[ ][ ]*/ / t def d :def s/\\$// t bsnl s/["\\]/\\&/g s/^ \('"$ac_word_re"'\)\(([^()]*)\)[ ]*\(.*\)/P["\1"]="\2"\ D["\1"]=" \3"/p s/^ \('"$ac_word_re"'\)[ ]*\(.*\)/D["\1"]=" \2"/p d :bsnl s/["\\]/\\&/g s/^ \('"$ac_word_re"'\)\(([^()]*)\)[ ]*\(.*\)/P["\1"]="\2"\ D["\1"]=" \3\\\\\\n"\\/p t cont s/^ \('"$ac_word_re"'\)[ ]*\(.*\)/D["\1"]=" \2\\\\\\n"\\/p t cont d :cont n s/.\{148\}/&'"$ac_delim"'/g t clear :clear s/\\$// t bsnlc s/["\\]/\\&/g; s/^/"/; s/$/"/p d :bsnlc s/["\\]/\\&/g; s/^/"/; s/$/\\\\\\n"\\/p b cont ' >$CONFIG_STATUS || ac_write_fail=1 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 for (key in D) D_is_set[key] = 1 FS = "" } /^[\t ]*#[\t ]*(define|undef)[\t ]+$ac_word_re([\t (]|\$)/ { line = \$ 0 split(line, arg, " ") if (arg[1] == "#") { defundef = arg[2] mac1 = arg[3] } else { defundef = substr(arg[1], 2) mac1 = arg[2] } split(mac1, mac2, "(") #) macro = mac2[1] prefix = substr(line, 1, index(line, defundef) - 1) if (D_is_set[macro]) { # Preserve the white space surrounding the "#". print prefix "define", macro P[macro] D[macro] next } else { # Replace #undef with comments. This is necessary, for example, # in the case of _POSIX_SOURCE, which is predefined and required # on some systems where configure will not decide to define it. if (defundef == "undef") { print "/*", prefix defundef, macro, "*/" next } } } { print } _ACAWK _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 as_fn_error $? "could not setup config headers machinery" "$LINENO" 5 fi # test -n "$CONFIG_HEADERS" eval set X " :F $CONFIG_FILES :H $CONFIG_HEADERS :C $CONFIG_COMMANDS" shift for ac_tag do case $ac_tag in :[FHLC]) ac_mode=$ac_tag; continue;; esac case $ac_mode$ac_tag in :[FHL]*:*);; :L* | :C*:*) as_fn_error $? "invalid tag \`$ac_tag'" "$LINENO" 5;; :[FH]-) ac_tag=-:-;; :[FH]*) ac_tag=$ac_tag:$ac_tag.in;; esac ac_save_IFS=$IFS IFS=: set x $ac_tag IFS=$ac_save_IFS shift ac_file=$1 shift case $ac_mode in :L) ac_source=$1;; :[FH]) ac_file_inputs= for ac_f do case $ac_f in -) ac_f="$ac_tmp/stdin";; *) # Look for the file first in the build tree, then in the source tree # (if the path is not absolute). The absolute path cannot be DOS-style, # because $ac_f cannot contain `:'. test -f "$ac_f" || case $ac_f in [\\/$]*) false;; *) test -f "$srcdir/$ac_f" && ac_f="$srcdir/$ac_f";; esac || as_fn_error 1 "cannot find input file: \`$ac_f'" "$LINENO" 5;; esac case $ac_f in *\'*) ac_f=`$as_echo "$ac_f" | sed "s/'/'\\\\\\\\''/g"`;; esac as_fn_append ac_file_inputs " '$ac_f'" done # Let's still pretend it is `configure' which instantiates (i.e., don't # use $as_me), people would be surprised to read: # /* config.h. Generated by config.status. */ configure_input='Generated from '` $as_echo "$*" | sed 's|^[^:]*/||;s|:[^:]*/|, |g' `' by configure.' if test x"$ac_file" != x-; then configure_input="$ac_file. $configure_input" { $as_echo "$as_me:${as_lineno-$LINENO}: creating $ac_file" >&5 $as_echo "$as_me: creating $ac_file" >&6;} fi # Neutralize special characters interpreted by sed in replacement strings. case $configure_input in #( *\&* | *\|* | *\\* ) ac_sed_conf_input=`$as_echo "$configure_input" | sed 's/[\\\\&|]/\\\\&/g'`;; #( *) ac_sed_conf_input=$configure_input;; esac case $ac_tag in *:-:* | *:-) cat >"$ac_tmp/stdin" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 ;; esac ;; esac ac_dir=`$as_dirname -- "$ac_file" || $as_expr X"$ac_file" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$ac_file" : 'X\(//\)[^/]' \| \ X"$ac_file" : 'X\(//\)$' \| \ X"$ac_file" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$ac_file" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` as_dir="$ac_dir"; as_fn_mkdir_p ac_builddir=. case "$ac_dir" in .) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_dir_suffix=/`$as_echo "$ac_dir" | sed 's|^\.[\\/]||'` # A ".." for each directory in $ac_dir_suffix. ac_top_builddir_sub=`$as_echo "$ac_dir_suffix" | sed 's|/[^\\/]*|/..|g;s|/||'` case $ac_top_builddir_sub in "") ac_top_builddir_sub=. ac_top_build_prefix= ;; *) ac_top_build_prefix=$ac_top_builddir_sub/ ;; esac ;; esac ac_abs_top_builddir=$ac_pwd ac_abs_builddir=$ac_pwd$ac_dir_suffix # for backward compatibility: ac_top_builddir=$ac_top_build_prefix case $srcdir in .) # We are building in place. ac_srcdir=. ac_top_srcdir=$ac_top_builddir_sub ac_abs_top_srcdir=$ac_pwd ;; [\\/]* | ?:[\\/]* ) # Absolute name. ac_srcdir=$srcdir$ac_dir_suffix; ac_top_srcdir=$srcdir ac_abs_top_srcdir=$srcdir ;; *) # Relative name. ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix ac_top_srcdir=$ac_top_build_prefix$srcdir ac_abs_top_srcdir=$ac_pwd/$srcdir ;; esac ac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix case $ac_mode in :F) # # CONFIG_FILE # case $INSTALL in [\\/$]* | ?:[\\/]* ) ac_INSTALL=$INSTALL ;; *) ac_INSTALL=$ac_top_build_prefix$INSTALL ;; esac ac_MKDIR_P=$MKDIR_P case $MKDIR_P in [\\/$]* | ?:[\\/]* ) ;; */*) ac_MKDIR_P=$ac_top_build_prefix$MKDIR_P ;; esac _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # If the template does not know about datarootdir, expand it. # FIXME: This hack should be removed a few years after 2.60. ac_datarootdir_hack=; ac_datarootdir_seen= ac_sed_dataroot=' /datarootdir/ { p q } /@datadir@/p /@docdir@/p /@infodir@/p /@localedir@/p /@mandir@/p' case `eval "sed -n \"\$ac_sed_dataroot\" $ac_file_inputs"` in *datarootdir*) ac_datarootdir_seen=yes;; *@datadir@*|*@docdir@*|*@infodir@*|*@localedir@*|*@mandir@*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting" >&5 $as_echo "$as_me: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting" >&2;} _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_datarootdir_hack=' s&@datadir@&$datadir&g s&@docdir@&$docdir&g s&@infodir@&$infodir&g s&@localedir@&$localedir&g s&@mandir@&$mandir&g s&\\\${datarootdir}&$datarootdir&g' ;; esac _ACEOF # Neutralize VPATH when `$srcdir' = `.'. # Shell code in configure.ac might set extrasub. # FIXME: do we really want to maintain this feature? cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_sed_extra="$ac_vpsub $extrasub _ACEOF cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 :t /@[a-zA-Z_][a-zA-Z_0-9]*@/!b s|@configure_input@|$ac_sed_conf_input|;t t s&@top_builddir@&$ac_top_builddir_sub&;t t s&@top_build_prefix@&$ac_top_build_prefix&;t t s&@srcdir@&$ac_srcdir&;t t s&@abs_srcdir@&$ac_abs_srcdir&;t t s&@top_srcdir@&$ac_top_srcdir&;t t s&@abs_top_srcdir@&$ac_abs_top_srcdir&;t t s&@builddir@&$ac_builddir&;t t s&@abs_builddir@&$ac_abs_builddir&;t t s&@abs_top_builddir@&$ac_abs_top_builddir&;t t s&@INSTALL@&$ac_INSTALL&;t t s&@MKDIR_P@&$ac_MKDIR_P&;t t $ac_datarootdir_hack " eval sed \"\$ac_sed_extra\" "$ac_file_inputs" | $AWK -f "$ac_tmp/subs.awk" \ >$ac_tmp/out || as_fn_error $? "could not create $ac_file" "$LINENO" 5 test -z "$ac_datarootdir_hack$ac_datarootdir_seen" && { ac_out=`sed -n '/\${datarootdir}/p' "$ac_tmp/out"`; test -n "$ac_out"; } && { ac_out=`sed -n '/^[ ]*datarootdir[ ]*:*=/p' \ "$ac_tmp/out"`; test -z "$ac_out"; } && { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $ac_file contains a reference to the variable \`datarootdir' which seems to be undefined. Please make sure it is defined" >&5 $as_echo "$as_me: WARNING: $ac_file contains a reference to the variable \`datarootdir' which seems to be undefined. Please make sure it is defined" >&2;} rm -f "$ac_tmp/stdin" case $ac_file in -) cat "$ac_tmp/out" && rm -f "$ac_tmp/out";; *) rm -f "$ac_file" && mv "$ac_tmp/out" "$ac_file";; esac \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 ;; :H) # # CONFIG_HEADER # if test x"$ac_file" != x-; then { $as_echo "/* $configure_input */" \ && eval '$AWK -f "$ac_tmp/defines.awk"' "$ac_file_inputs" } >"$ac_tmp/config.h" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 if diff "$ac_file" "$ac_tmp/config.h" >/dev/null 2>&1; then { $as_echo "$as_me:${as_lineno-$LINENO}: $ac_file is unchanged" >&5 $as_echo "$as_me: $ac_file is unchanged" >&6;} else rm -f "$ac_file" mv "$ac_tmp/config.h" "$ac_file" \ || as_fn_error $? "could not create $ac_file" "$LINENO" 5 fi else $as_echo "/* $configure_input */" \ && eval '$AWK -f "$ac_tmp/defines.awk"' "$ac_file_inputs" \ || as_fn_error $? "could not create -" "$LINENO" 5 fi # Compute "$ac_file"'s index in $config_headers. _am_arg="$ac_file" _am_stamp_count=1 for _am_header in $config_headers :; do case $_am_header in $_am_arg | $_am_arg:* ) break ;; * ) _am_stamp_count=`expr $_am_stamp_count + 1` ;; esac done echo "timestamp for $_am_arg" >`$as_dirname -- "$_am_arg" || $as_expr X"$_am_arg" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$_am_arg" : 'X\(//\)[^/]' \| \ X"$_am_arg" : 'X\(//\)$' \| \ X"$_am_arg" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$_am_arg" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'`/stamp-h$_am_stamp_count ;; :C) { $as_echo "$as_me:${as_lineno-$LINENO}: executing $ac_file commands" >&5 $as_echo "$as_me: executing $ac_file commands" >&6;} ;; esac case $ac_file$ac_mode in "depfiles":C) test x"$AMDEP_TRUE" != x"" || { # Older Autoconf quotes --file arguments for eval, but not when files # are listed without --file. Let's play safe and only enable the eval # if we detect the quoting. case $CONFIG_FILES in *\'*) eval set x "$CONFIG_FILES" ;; *) set x $CONFIG_FILES ;; esac shift for mf do # Strip MF so we end up with the name of the file. mf=`echo "$mf" | sed -e 's/:.*$//'` # Check whether this is an Automake generated Makefile or not. # We used to match only the files named 'Makefile.in', but # some people rename them; so instead we look at the file content. # Grep'ing the first line is not enough: some people post-process # each Makefile.in and add a new line on top of each file to say so. # Grep'ing the whole file is not good either: AIX grep has a line # limit of 2048, but all sed's we know have understand at least 4000. if sed -n 's,^#.*generated by automake.*,X,p' "$mf" | grep X >/dev/null 2>&1; then dirpart=`$as_dirname -- "$mf" || $as_expr X"$mf" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$mf" : 'X\(//\)[^/]' \| \ X"$mf" : 'X\(//\)$' \| \ X"$mf" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$mf" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` else continue fi # Extract the definition of DEPDIR, am__include, and am__quote # from the Makefile without running 'make'. DEPDIR=`sed -n 's/^DEPDIR = //p' < "$mf"` test -z "$DEPDIR" && continue am__include=`sed -n 's/^am__include = //p' < "$mf"` test -z "$am__include" && continue am__quote=`sed -n 's/^am__quote = //p' < "$mf"` # Find all dependency output files, they are included files with # $(DEPDIR) in their names. We invoke sed twice because it is the # simplest approach to changing $(DEPDIR) to its actual value in the # expansion. for file in `sed -n " s/^$am__include $am__quote\(.*(DEPDIR).*\)$am__quote"'$/\1/p' <"$mf" | \ sed -e 's/\$(DEPDIR)/'"$DEPDIR"'/g'`; do # Make sure the directory exists. test -f "$dirpart/$file" && continue fdir=`$as_dirname -- "$file" || $as_expr X"$file" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ X"$file" : 'X\(//\)[^/]' \| \ X"$file" : 'X\(//\)$' \| \ X"$file" : 'X\(/\)' \| . 2>/dev/null || $as_echo X"$file" | sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ s//\1/ q } /^X\(\/\/\)[^/].*/{ s//\1/ q } /^X\(\/\/\)$/{ s//\1/ q } /^X\(\/\).*/{ s//\1/ q } s/.*/./; q'` as_dir=$dirpart/$fdir; as_fn_mkdir_p # echo "creating $dirpart/$file" echo '# dummy' > "$dirpart/$file" done done } ;; "libtool":C) # See if we are running on zsh, and set the options which allow our # commands through without removal of \ escapes. if test -n "${ZSH_VERSION+set}" ; then setopt NO_GLOB_SUBST fi cfgfile="${ofile}T" trap "$RM \"$cfgfile\"; exit 1" 1 2 15 $RM "$cfgfile" cat <<_LT_EOF >> "$cfgfile" #! $SHELL # `$ECHO "$ofile" | sed 's%^.*/%%'` - Provide generalized library-building support services. # Generated automatically by $as_me ($PACKAGE$TIMESTAMP) $VERSION # Libtool was configured on host `(hostname || uname -n) 2>/dev/null | sed 1q`: # NOTE: Changes made to this file will be lost: look at ltmain.sh. # # Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2003, 2004, 2005, # 2006, 2007, 2008, 2009, 2010, 2011 Free Software # Foundation, Inc. # Written by Gordon Matzigkeit, 1996 # # This file is part of GNU Libtool. # # GNU Libtool is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License as # published by the Free Software Foundation; either version 2 of # the License, or (at your option) any later version. # # As a special exception to the GNU General Public License, # if you distribute this file as part of a program or library that # is built using GNU Libtool, you may include this file under the # same distribution terms that you use for the rest of that program. # # GNU Libtool is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with GNU Libtool; see the file COPYING. If not, a copy # can be downloaded from http://www.gnu.org/licenses/gpl.html, or # obtained by writing to the Free Software Foundation, Inc., # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. # The names of the tagged configurations supported by this script. available_tags="" # ### BEGIN LIBTOOL CONFIG # Which release of libtool.m4 was used? macro_version=$macro_version macro_revision=$macro_revision # Whether or not to build static libraries. build_old_libs=$enable_static # What type of objects to build. pic_mode=$pic_mode # Whether or not to build shared libraries. build_libtool_libs=$enable_shared # Whether or not to optimize for fast installation. fast_install=$enable_fast_install # Shell to use when invoking shell scripts. SHELL=$lt_SHELL # An echo program that protects backslashes. ECHO=$lt_ECHO # The PATH separator for the build system. PATH_SEPARATOR=$lt_PATH_SEPARATOR # The host system. host_alias=$host_alias host=$host host_os=$host_os # The build system. build_alias=$build_alias build=$build build_os=$build_os # A sed program that does not truncate output. SED=$lt_SED # Sed that helps us avoid accidentally triggering echo(1) options like -n. Xsed="\$SED -e 1s/^X//" # A grep program that handles long lines. GREP=$lt_GREP # An ERE matcher. EGREP=$lt_EGREP # A literal string matcher. FGREP=$lt_FGREP # A BSD- or MS-compatible name lister. NM=$lt_NM # Whether we need soft or hard links. LN_S=$lt_LN_S # What is the maximum length of a command? max_cmd_len=$max_cmd_len # Object file suffix (normally "o"). objext=$ac_objext # Executable file suffix (normally ""). exeext=$exeext # whether the shell understands "unset". lt_unset=$lt_unset # turn spaces into newlines. SP2NL=$lt_lt_SP2NL # turn newlines into spaces. NL2SP=$lt_lt_NL2SP # convert \$build file names to \$host format. to_host_file_cmd=$lt_cv_to_host_file_cmd # convert \$build files to toolchain format. to_tool_file_cmd=$lt_cv_to_tool_file_cmd # An object symbol dumper. OBJDUMP=$lt_OBJDUMP # Method to check whether dependent libraries are shared objects. deplibs_check_method=$lt_deplibs_check_method # Command to use when deplibs_check_method = "file_magic". file_magic_cmd=$lt_file_magic_cmd # How to find potential files when deplibs_check_method = "file_magic". file_magic_glob=$lt_file_magic_glob # Find potential files using nocaseglob when deplibs_check_method = "file_magic". want_nocaseglob=$lt_want_nocaseglob # DLL creation program. DLLTOOL=$lt_DLLTOOL # Command to associate shared and link libraries. sharedlib_from_linklib_cmd=$lt_sharedlib_from_linklib_cmd # The archiver. AR=$lt_AR # Flags to create an archive. AR_FLAGS=$lt_AR_FLAGS # How to feed a file listing to the archiver. archiver_list_spec=$lt_archiver_list_spec # A symbol stripping program. STRIP=$lt_STRIP # Commands used to install an old-style archive. RANLIB=$lt_RANLIB old_postinstall_cmds=$lt_old_postinstall_cmds old_postuninstall_cmds=$lt_old_postuninstall_cmds # Whether to use a lock for old archive extraction. lock_old_archive_extraction=$lock_old_archive_extraction # A C compiler. LTCC=$lt_CC # LTCC compiler flags. LTCFLAGS=$lt_CFLAGS # Take the output of nm and produce a listing of raw symbols and C names. global_symbol_pipe=$lt_lt_cv_sys_global_symbol_pipe # Transform the output of nm in a proper C declaration. global_symbol_to_cdecl=$lt_lt_cv_sys_global_symbol_to_cdecl # Transform the output of nm in a C name address pair. global_symbol_to_c_name_address=$lt_lt_cv_sys_global_symbol_to_c_name_address # Transform the output of nm in a C name address pair when lib prefix is needed. global_symbol_to_c_name_address_lib_prefix=$lt_lt_cv_sys_global_symbol_to_c_name_address_lib_prefix # Specify filename containing input files for \$NM. nm_file_list_spec=$lt_nm_file_list_spec # The root where to search for dependent libraries,and in which our libraries should be installed. lt_sysroot=$lt_sysroot # The name of the directory that contains temporary libtool files. objdir=$objdir # Used to examine libraries when file_magic_cmd begins with "file". MAGIC_CMD=$MAGIC_CMD # Must we lock files when doing compilation? need_locks=$lt_need_locks # Manifest tool. MANIFEST_TOOL=$lt_MANIFEST_TOOL # Tool to manipulate archived DWARF debug symbol files on Mac OS X. DSYMUTIL=$lt_DSYMUTIL # Tool to change global to local symbols on Mac OS X. NMEDIT=$lt_NMEDIT # Tool to manipulate fat objects and archives on Mac OS X. LIPO=$lt_LIPO # ldd/readelf like tool for Mach-O binaries on Mac OS X. OTOOL=$lt_OTOOL # ldd/readelf like tool for 64 bit Mach-O binaries on Mac OS X 10.4. OTOOL64=$lt_OTOOL64 # Old archive suffix (normally "a"). libext=$libext # Shared library suffix (normally ".so"). shrext_cmds=$lt_shrext_cmds # The commands to extract the exported symbol list from a shared archive. extract_expsyms_cmds=$lt_extract_expsyms_cmds # Variables whose values should be saved in libtool wrapper scripts and # restored at link time. variables_saved_for_relink=$lt_variables_saved_for_relink # Do we need the "lib" prefix for modules? need_lib_prefix=$need_lib_prefix # Do we need a version for libraries? need_version=$need_version # Library versioning type. version_type=$version_type # Shared library runtime path variable. runpath_var=$runpath_var # Shared library path variable. shlibpath_var=$shlibpath_var # Is shlibpath searched before the hard-coded library search path? shlibpath_overrides_runpath=$shlibpath_overrides_runpath # Format of library name prefix. libname_spec=$lt_libname_spec # List of archive names. First name is the real one, the rest are links. # The last name is the one that the linker finds with -lNAME library_names_spec=$lt_library_names_spec # The coded name of the library, if different from the real name. soname_spec=$lt_soname_spec # Permission mode override for installation of shared libraries. install_override_mode=$lt_install_override_mode # Command to use after installation of a shared archive. postinstall_cmds=$lt_postinstall_cmds # Command to use after uninstallation of a shared archive. postuninstall_cmds=$lt_postuninstall_cmds # Commands used to finish a libtool library installation in a directory. finish_cmds=$lt_finish_cmds # As "finish_cmds", except a single script fragment to be evaled but # not shown. finish_eval=$lt_finish_eval # Whether we should hardcode library paths into libraries. hardcode_into_libs=$hardcode_into_libs # Compile-time system search path for libraries. sys_lib_search_path_spec=$lt_sys_lib_search_path_spec # Run-time system search path for libraries. sys_lib_dlsearch_path_spec=$lt_sys_lib_dlsearch_path_spec # Whether dlopen is supported. dlopen_support=$enable_dlopen # Whether dlopen of programs is supported. dlopen_self=$enable_dlopen_self # Whether dlopen of statically linked programs is supported. dlopen_self_static=$enable_dlopen_self_static # Commands to strip libraries. old_striplib=$lt_old_striplib striplib=$lt_striplib # The linker used to build libraries. LD=$lt_LD # How to create reloadable object files. reload_flag=$lt_reload_flag reload_cmds=$lt_reload_cmds # Commands used to build an old-style archive. old_archive_cmds=$lt_old_archive_cmds # A language specific compiler. CC=$lt_compiler # Is the compiler the GNU compiler? with_gcc=$GCC # Compiler flag to turn off builtin functions. no_builtin_flag=$lt_lt_prog_compiler_no_builtin_flag # Additional compiler flags for building library objects. pic_flag=$lt_lt_prog_compiler_pic # How to pass a linker flag through the compiler. wl=$lt_lt_prog_compiler_wl # Compiler flag to prevent dynamic linking. link_static_flag=$lt_lt_prog_compiler_static # Does compiler simultaneously support -c and -o options? compiler_c_o=$lt_lt_cv_prog_compiler_c_o # Whether or not to add -lc for building shared libraries. build_libtool_need_lc=$archive_cmds_need_lc # Whether or not to disallow shared libs when runtime libs are static. allow_libtool_libs_with_static_runtimes=$enable_shared_with_static_runtimes # Compiler flag to allow reflexive dlopens. export_dynamic_flag_spec=$lt_export_dynamic_flag_spec # Compiler flag to generate shared objects directly from archives. whole_archive_flag_spec=$lt_whole_archive_flag_spec # Whether the compiler copes with passing no objects directly. compiler_needs_object=$lt_compiler_needs_object # Create an old-style archive from a shared archive. old_archive_from_new_cmds=$lt_old_archive_from_new_cmds # Create a temporary old-style archive to link instead of a shared archive. old_archive_from_expsyms_cmds=$lt_old_archive_from_expsyms_cmds # Commands used to build a shared archive. archive_cmds=$lt_archive_cmds archive_expsym_cmds=$lt_archive_expsym_cmds # Commands used to build a loadable module if different from building # a shared archive. module_cmds=$lt_module_cmds module_expsym_cmds=$lt_module_expsym_cmds # Whether we are building with GNU ld or not. with_gnu_ld=$lt_with_gnu_ld # Flag that allows shared libraries with undefined symbols to be built. allow_undefined_flag=$lt_allow_undefined_flag # Flag that enforces no undefined symbols. no_undefined_flag=$lt_no_undefined_flag # Flag to hardcode \$libdir into a binary during linking. # This must work even if \$libdir does not exist hardcode_libdir_flag_spec=$lt_hardcode_libdir_flag_spec # Whether we need a single "-rpath" flag with a separated argument. hardcode_libdir_separator=$lt_hardcode_libdir_separator # Set to "yes" if using DIR/libNAME\${shared_ext} during linking hardcodes # DIR into the resulting binary. hardcode_direct=$hardcode_direct # Set to "yes" if using DIR/libNAME\${shared_ext} during linking hardcodes # DIR into the resulting binary and the resulting library dependency is # "absolute",i.e impossible to change by setting \${shlibpath_var} if the # library is relocated. hardcode_direct_absolute=$hardcode_direct_absolute # Set to "yes" if using the -LDIR flag during linking hardcodes DIR # into the resulting binary. hardcode_minus_L=$hardcode_minus_L # Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR # into the resulting binary. hardcode_shlibpath_var=$hardcode_shlibpath_var # Set to "yes" if building a shared library automatically hardcodes DIR # into the library and all subsequent libraries and executables linked # against it. hardcode_automatic=$hardcode_automatic # Set to yes if linker adds runtime paths of dependent libraries # to runtime path list. inherit_rpath=$inherit_rpath # Whether libtool must link a program against all its dependency libraries. link_all_deplibs=$link_all_deplibs # Set to "yes" if exported symbols are required. always_export_symbols=$always_export_symbols # The commands to list exported symbols. export_symbols_cmds=$lt_export_symbols_cmds # Symbols that should not be listed in the preloaded symbols. exclude_expsyms=$lt_exclude_expsyms # Symbols that must always be exported. include_expsyms=$lt_include_expsyms # Commands necessary for linking programs (against libraries) with templates. prelink_cmds=$lt_prelink_cmds # Commands necessary for finishing linking programs. postlink_cmds=$lt_postlink_cmds # Specify filename containing input files. file_list_spec=$lt_file_list_spec # How to hardcode a shared library path into an executable. hardcode_action=$hardcode_action # ### END LIBTOOL CONFIG _LT_EOF case $host_os in aix3*) cat <<\_LT_EOF >> "$cfgfile" # AIX sometimes has problems with the GCC collect2 program. For some # reason, if we set the COLLECT_NAMES environment variable, the problems # vanish in a puff of smoke. if test "X${COLLECT_NAMES+set}" != Xset; then COLLECT_NAMES= export COLLECT_NAMES fi _LT_EOF ;; esac ltmain="$ac_aux_dir/ltmain.sh" # We use sed instead of cat because bash on DJGPP gets confused if # if finds mixed CR/LF and LF-only lines. Since sed operates in # text mode, it properly converts lines to CR/LF. This bash problem # is reportedly fixed, but why not run on old versions too? sed '$q' "$ltmain" >> "$cfgfile" \ || (rm -f "$cfgfile"; exit 1) if test x"$xsi_shell" = xyes; then sed -e '/^func_dirname ()$/,/^} # func_dirname /c\ func_dirname ()\ {\ \ case ${1} in\ \ */*) func_dirname_result="${1%/*}${2}" ;;\ \ * ) func_dirname_result="${3}" ;;\ \ esac\ } # Extended-shell func_dirname implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_basename ()$/,/^} # func_basename /c\ func_basename ()\ {\ \ func_basename_result="${1##*/}"\ } # Extended-shell func_basename implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_dirname_and_basename ()$/,/^} # func_dirname_and_basename /c\ func_dirname_and_basename ()\ {\ \ case ${1} in\ \ */*) func_dirname_result="${1%/*}${2}" ;;\ \ * ) func_dirname_result="${3}" ;;\ \ esac\ \ func_basename_result="${1##*/}"\ } # Extended-shell func_dirname_and_basename implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_stripname ()$/,/^} # func_stripname /c\ func_stripname ()\ {\ \ # pdksh 5.2.14 does not do ${X%$Y} correctly if both X and Y are\ \ # positional parameters, so assign one to ordinary parameter first.\ \ func_stripname_result=${3}\ \ func_stripname_result=${func_stripname_result#"${1}"}\ \ func_stripname_result=${func_stripname_result%"${2}"}\ } # Extended-shell func_stripname implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_split_long_opt ()$/,/^} # func_split_long_opt /c\ func_split_long_opt ()\ {\ \ func_split_long_opt_name=${1%%=*}\ \ func_split_long_opt_arg=${1#*=}\ } # Extended-shell func_split_long_opt implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_split_short_opt ()$/,/^} # func_split_short_opt /c\ func_split_short_opt ()\ {\ \ func_split_short_opt_arg=${1#??}\ \ func_split_short_opt_name=${1%"$func_split_short_opt_arg"}\ } # Extended-shell func_split_short_opt implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_lo2o ()$/,/^} # func_lo2o /c\ func_lo2o ()\ {\ \ case ${1} in\ \ *.lo) func_lo2o_result=${1%.lo}.${objext} ;;\ \ *) func_lo2o_result=${1} ;;\ \ esac\ } # Extended-shell func_lo2o implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_xform ()$/,/^} # func_xform /c\ func_xform ()\ {\ func_xform_result=${1%.*}.lo\ } # Extended-shell func_xform implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_arith ()$/,/^} # func_arith /c\ func_arith ()\ {\ func_arith_result=$(( $* ))\ } # Extended-shell func_arith implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_len ()$/,/^} # func_len /c\ func_len ()\ {\ func_len_result=${#1}\ } # Extended-shell func_len implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: fi if test x"$lt_shell_append" = xyes; then sed -e '/^func_append ()$/,/^} # func_append /c\ func_append ()\ {\ eval "${1}+=\\${2}"\ } # Extended-shell func_append implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: sed -e '/^func_append_quoted ()$/,/^} # func_append_quoted /c\ func_append_quoted ()\ {\ \ func_quote_for_eval "${2}"\ \ eval "${1}+=\\\\ \\$func_quote_for_eval_result"\ } # Extended-shell func_append_quoted implementation' "$cfgfile" > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: # Save a `func_append' function call where possible by direct use of '+=' sed -e 's%func_append \([a-zA-Z_]\{1,\}\) "%\1+="%g' $cfgfile > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: else # Save a `func_append' function call even when '+=' is not available sed -e 's%func_append \([a-zA-Z_]\{1,\}\) "%\1="$\1%g' $cfgfile > $cfgfile.tmp \ && mv -f "$cfgfile.tmp" "$cfgfile" \ || (rm -f "$cfgfile" && cp "$cfgfile.tmp" "$cfgfile" && rm -f "$cfgfile.tmp") test 0 -eq $? || _lt_function_replace_fail=: fi if test x"$_lt_function_replace_fail" = x":"; then { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Unable to substitute extended shell functions in $ofile" >&5 $as_echo "$as_me: WARNING: Unable to substitute extended shell functions in $ofile" >&2;} fi mv -f "$cfgfile" "$ofile" || (rm -f "$ofile" && cp "$cfgfile" "$ofile" && rm -f "$cfgfile") chmod +x "$ofile" ;; esac done # for ac_tag as_fn_exit 0 _ACEOF ac_clean_files=$ac_clean_files_save test $ac_write_fail = 0 || as_fn_error $? "write failure creating $CONFIG_STATUS" "$LINENO" 5 # configure is writing to config.log, and then calls config.status. # config.status does its own redirection, appending to config.log. # Unfortunately, on DOS this fails, as config.log is still kept open # by configure, so config.status won't be able to write to it; its # output is simply discarded. So we exec the FD to /dev/null, # effectively closing config.log, so it can be properly (re)opened and # appended to by config.status. When coming back to configure, we # need to make the FD available again. if test "$no_create" != yes; then ac_cs_success=: ac_config_status_args= test "$silent" = yes && ac_config_status_args="$ac_config_status_args --quiet" exec 5>/dev/null $SHELL $CONFIG_STATUS $ac_config_status_args || ac_cs_success=false exec 5>>config.log # Use ||, not &&, to avoid exiting from the if with $? = 1, which # would make configure fail if this is the last instruction. $ac_cs_success || as_fn_exit 1 fi if test -n "$ac_unrecognized_opts" && test "$enable_option_checking" != no; then { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: unrecognized options: $ac_unrecognized_opts" >&5 $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2;} fi Index: releng/10.1/contrib/file/configure.ac =================================================================== --- releng/10.1/contrib/file/configure.ac (revision 284192) +++ releng/10.1/contrib/file/configure.ac (revision 284193) @@ -1,156 +1,156 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT([file],[5.19],[christos@astron.com]) +AC_INIT([file],[5.22],[christos@astron.com]) AM_INIT_AUTOMAKE([subdir-objects foreign]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_MACRO_DIR([m4]) AC_MSG_CHECKING(for builtin ELF support) AC_ARG_ENABLE(elf, [ --disable-elf disable builtin ELF support], [if test "${enableval}" = yes; then AC_MSG_RESULT(yes) AC_DEFINE([BUILTIN_ELF], 1, [Define if built-in ELF support is used]) else AC_MSG_RESULT(no) fi], [ # enable by default AC_MSG_RESULT(yes) AC_DEFINE([BUILTIN_ELF], 1, [Define in built-in ELF support is used]) ]) AC_MSG_CHECKING(for ELF core file support) AC_ARG_ENABLE(elf-core, [ --disable-elf-core disable ELF core file support], [if test "${enableval}" = yes; then AC_MSG_RESULT(yes) AC_DEFINE([ELFCORE], 1, [Define for ELF core file support]) else AC_MSG_RESULT(no) fi], [ # enable by default AC_MSG_RESULT(yes) AC_DEFINE([ELFCORE], 1, [Define for ELF core file support]) ]) AC_MSG_CHECKING(for file formats in man section 5) AC_ARG_ENABLE(fsect-man5, [ --enable-fsect-man5 enable file formats in man section 5], [if test "${enableval}" = yes; then AC_MSG_RESULT(yes) fsect=5 else AC_MSG_RESULT(no) fsect=4 fi], [ # disable by default AC_MSG_RESULT(no) fsect=4 ]) AC_CANONICAL_HOST case "$host_os" in mingw32*) MINGW=1 ;; *) MINGW=0 ;; esac AC_SUBST(MINGW) AM_CONDITIONAL(MINGW, test "$MINGW" = 1) AC_SUBST([pkgdatadir], ['$(datadir)/misc']) AC_SUBST(fsect) AM_CONDITIONAL(FSECT5, test x$fsect = x5) AC_SUBST(WARNINGS) dnl Checks for programs. AC_PROG_CC_STDC AC_USE_SYSTEM_EXTENSIONS AM_PROG_CC_C_O AC_C_BIGENDIAN AC_PROG_INSTALL AC_PROG_LN_S LT_INIT([disable-static pic-only]) gl_VISIBILITY dnl Checks for headers AC_HEADER_STDC AC_HEADER_MAJOR AC_HEADER_SYS_WAIT AC_CHECK_HEADERS(stdint.h fcntl.h locale.h stdint.h inttypes.h unistd.h) AC_CHECK_HEADERS(stddef.h utime.h wchar.h wctype.h limits.h) -AC_CHECK_HEADERS(getopt.h err.h) +AC_CHECK_HEADERS(getopt.h err.h xlocale.h) AC_CHECK_HEADERS(sys/mman.h sys/stat.h sys/types.h sys/utime.h sys/time.h) AC_CHECK_HEADERS(zlib.h) dnl Checks for typedefs, structures, and compiler characteristics. AC_C_CONST AC_TYPE_OFF_T AC_TYPE_SIZE_T AC_CHECK_MEMBERS([struct stat.st_rdev]) AC_CHECK_MEMBERS([struct tm.tm_gmtoff]) AC_STRUCT_TIMEZONE AC_STRUCT_TIMEZONE_DAYLIGHT AC_SYS_LARGEFILE AC_FUNC_FSEEKO AC_TYPE_MBSTATE_T AC_STRUCT_OPTION_GETOPT_H AC_TYPE_PID_T AC_TYPE_UINT8_T AC_TYPE_UINT16_T AC_TYPE_UINT32_T AC_TYPE_INT32_T AC_TYPE_UINT64_T AC_TYPE_INT64_T AC_TYPE_INTPTR_T AC_TYPE_UINTPTR_T AC_FUNC_MMAP AC_FUNC_FORK AC_FUNC_MBRTOWC AC_MSG_CHECKING(for gcc compiler warnings) AC_ARG_ENABLE(warnings, [ --disable-warnings disable compiler warnings], [if test "${enableval}" = no -o "$GCC" = no; then AC_MSG_RESULT(no) WARNINGS= else AC_MSG_RESULT(yes) WARNINGS="-Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith \ -Wmissing-declarations -Wredundant-decls -Wnested-externs \ -Wsign-compare -Wreturn-type -Wswitch -Wshadow \ -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -Wformat=2" fi], [ if test "$GCC" = yes; then AC_MSG_RESULT(yes) WARNINGS="-Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith \ -Wmissing-declarations -Wredundant-decls -Wnested-externs \ -Wsign-compare -Wreturn-type -Wswitch -Wshadow \ -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -Wformat=2" else WARNINGS= AC_MSG_RESULT(no) fi]) dnl Checks for functions -AC_CHECK_FUNCS(strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof) +AC_CHECK_FUNCS(strerror strndup strtoul mkstemp mkostemp utimes utime wcwidth strtof newlocale uselocale freelocale setlocale) dnl Provide implementation of some required functions if necessary AC_REPLACE_FUNCS(getopt_long asprintf vasprintf strlcpy strlcat getline ctime_r asctime_r pread strcasestr fmtcheck) dnl Checks for libraries AC_CHECK_LIB(z,gzopen) if test "$MINGW" = 1; then AC_CHECK_LIB(gnurx,regexec,,AC_MSG_ERROR([libgnurx is required to build file(1) with MinGW])) fi dnl See if we are cross-compiling AM_CONDITIONAL(IS_CROSS_COMPILE, test "$cross_compiling" = yes) AC_CONFIG_FILES([Makefile src/Makefile magic/Makefile tests/Makefile doc/Makefile python/Makefile]) AC_OUTPUT Index: releng/10.1/contrib/file/doc/file.man =================================================================== --- releng/10.1/contrib/file/doc/file.man (revision 284192) +++ releng/10.1/contrib/file/doc/file.man (revision 284193) @@ -1,623 +1,635 @@ -.\" $File: file.man,v 1.106 2014/03/07 23:11:51 christos Exp $ -.Dd January 30, 2014 +.\" $File: file.man,v 1.111 2014/12/16 23:18:40 christos Exp $ +.Dd December 16, 2014 .Dt FILE __CSECTION__ .Os .Sh NAME .Nm file .Nd determine file type .Sh SYNOPSIS .Nm .Bk -words .Op Fl bcEhiklLNnprsvz0 .Op Fl Fl apple .Op Fl Fl mime-encoding .Op Fl Fl mime-type .Op Fl e Ar testname .Op Fl F Ar separator .Op Fl f Ar namefile .Op Fl m Ar magicfiles +.Op Fl P Ar name=value .Ar .Ek .Nm .Fl C .Op Fl m Ar magicfiles .Nm .Op Fl Fl help .Sh DESCRIPTION This manual page documents version __VERSION__ of the .Nm command. .Pp .Nm tests each argument in an attempt to classify it. There are three sets of tests, performed in this order: filesystem tests, magic tests, and language tests. The .Em first test that succeeds causes the file type to be printed. .Pp The type printed will usually contain one of the words .Em text (the file contains only printing characters and a few common control characters and is probably safe to read on an .Dv ASCII terminal), .Em executable (the file contains the result of compiling a program in a form understandable to some .Tn UNIX kernel or another), or .Em data meaning anything else (data is usually .Dq binary or non-printable). Exceptions are well-known file formats (core files, tar archives) that are known to contain binary data. When modifying magic files or the program itself, make sure to .Em "preserve these keywords" . Users depend on knowing that all the readable files in a directory have the word .Dq text printed. Don't do as Berkeley did and change .Dq shell commands text to .Dq shell script . .Pp The filesystem tests are based on examining the return from a .Xr stat 2 system call. The program checks to see if the file is empty, or if it's some sort of special file. Any known file types appropriate to the system you are running on (sockets, symbolic links, or named pipes (FIFOs) on those systems that implement them) are intuited if they are defined in the system header file .In sys/stat.h . .Pp The magic tests are used to check for files with data in particular fixed formats. The canonical example of this is a binary executable (compiled program) .Dv a.out file, whose format is defined in .In elf.h , .In a.out.h and possibly .In exec.h in the standard include directory. These files have a .Dq "magic number" stored in a particular place near the beginning of the file that tells the .Tn UNIX operating system that the file is a binary executable, and which of several types thereof. The concept of a .Dq "magic" has been applied by extension to data files. Any file with some invariant identifier at a small fixed offset into the file can usually be described in this way. The information identifying these files is read from the compiled magic file .Pa __MAGIC__.mgc , or the files in the directory .Pa __MAGIC__ if the compiled file does not exist. In addition, if .Pa $HOME/.magic.mgc or .Pa $HOME/.magic exists, it will be used in preference to the system magic files. .Pp If a file does not match any of the entries in the magic file, it is examined to see if it seems to be a text file. ASCII, ISO-8859-x, non-ISO 8-bit extended-ASCII character sets (such as those used on Macintosh and IBM PC systems), UTF-8-encoded Unicode, UTF-16-encoded Unicode, and EBCDIC character sets can be distinguished by the different ranges and sequences of bytes that constitute printable text in each set. If a file passes any of these tests, its character set is reported. ASCII, ISO-8859-x, UTF-8, and extended-ASCII files are identified as .Dq text because they will be mostly readable on nearly any terminal; UTF-16 and EBCDIC are only .Dq character data because, while they contain text, it is text that will require translation before it can be read. In addition, .Nm will attempt to determine other characteristics of text-type files. If the lines of a file are terminated by CR, CRLF, or NEL, instead of the Unix-standard LF, this will be reported. Files that contain embedded escape sequences or overstriking will also be identified. .Pp Once .Nm has determined the character set used in a text-type file, it will attempt to determine in what language the file is written. The language tests look for particular strings (cf. .In names.h ) that can appear anywhere in the first few blocks of a file. For example, the keyword .Em .br indicates that the file is most likely a .Xr troff 1 input file, just as the keyword .Em struct indicates a C program. These tests are less reliable than the previous two groups, so they are performed last. The language test routines also test for some miscellany (such as .Xr tar 1 archives). .Pp Any file that cannot be identified as having been written in any of the character sets listed above is simply said to be .Dq data . .Sh OPTIONS .Bl -tag -width indent .It Fl Fl apple Causes the file command to output the file type and creator code as used by older MacOS versions. The code consists of eight letters, the first describing the file type, the latter the creator. .It Fl b , Fl Fl brief Do not prepend filenames to output lines (brief mode). .It Fl C , Fl Fl compile Write a .Pa magic.mgc output file that contains a pre-parsed version of the magic file or directory. .It Fl c , Fl Fl checking-printout Cause a checking printout of the parsed form of the magic file. This is usually used in conjunction with the .Fl m flag to debug a new magic file before installing it. .It Fl E On filesystem errors (file not found etc), instead of handling the error as regular output as POSIX mandates and keep going, issue an error message and exit. .It Fl e , Fl Fl exclude Ar testname Exclude the test named in .Ar testname from the list of tests made to determine the file type. Valid test names are: .Bl -tag -width compress .It apptype .Dv EMX application type (only on EMX). .It ascii Various types of text files (this test will try to guess the text encoding, irrespective of the setting of the .Sq encoding option). .It encoding Different text encodings for soft magic tests. .It tokens Ignored for backwards compatibility. .It cdf Prints details of Compound Document Files. .It compress Checks for, and looks inside, compressed files. .It elf Prints ELF file details. .It soft Consults magic files. .It tar Examines tar files. .El .It Fl F , Fl Fl separator Ar separator Use the specified string as the separator between the filename and the file result returned. Defaults to .Sq \&: . .It Fl f , Fl Fl files-from Ar namefile Read the names of the files to be examined from .Ar namefile (one per line) before the argument list. Either .Ar namefile or at least one filename argument must be present; to test the standard input, use .Sq - as a filename argument. Please note that .Ar namefile is unwrapped and the enclosed filenames are processed when this option is encountered and before any further options processing is done. This allows one to process multiple lists of files with different command line arguments on the same .Nm invocation. Thus if you want to set the delimiter, you need to do it before you specify the list of files, like: .Dq Fl F Ar @ Fl f Ar namefile , instead of: .Dq Fl f Ar namefile Fl F Ar @ . .It Fl h , Fl Fl no-dereference option causes symlinks not to be followed (on systems that support symbolic links). This is the default if the environment variable .Dv POSIXLY_CORRECT is not defined. .It Fl i , Fl Fl mime Causes the file command to output mime type strings rather than the more traditional human readable ones. Thus it may say .Sq text/plain; charset=us-ascii rather than .Dq ASCII text . .It Fl Fl mime-type , Fl Fl mime-encoding Like .Fl i , but print only the specified element(s). .It Fl k , Fl Fl keep-going Don't stop at the first match, keep going. Subsequent matches will be have the string .Sq "\[rs]012\- " prepended. (If you want a newline, see the .Fl r option.) The magic pattern with the highest strength (see the .Fl l option) comes first. .It Fl l , Fl Fl list Shows a list of patterns and their strength sorted descending by .Xr magic 4 strength which is used for the matching (see also the .Fl k option). .It Fl L , Fl Fl dereference option causes symlinks to be followed, as the like-named option in .Xr ls 1 (on systems that support symbolic links). This is the default if the environment variable .Ev POSIXLY_CORRECT is defined. .It Fl m , Fl Fl magic-file Ar magicfiles Specify an alternate list of files and directories containing magic. This can be a single item, or a colon-separated list. If a compiled magic file is found alongside a file or directory, it will be used instead. .It Fl N , Fl Fl no-pad Don't pad filenames so that they align in the output. .It Fl n , Fl Fl no-buffer Force stdout to be flushed after checking each file. This is only useful if checking a list of files. It is intended to be used by programs that want filetype output from a pipe. .It Fl p , Fl Fl preserve-date On systems that support .Xr utime 3 or .Xr utimes 2 , attempt to preserve the access time of files analyzed, to pretend that .Nm never read them. +.It Fl P , Fl Fl parameter Ar name=value +Set various parameter limits. +.Bl -column "elf_phnum" "Default" "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" -offset indent +.It Sy "Name" Ta Sy "Default" Ta Sy "Explanation" +.It Li indir Ta 15 Ta recursion limit for indirect magic +.It Li name Ta 30 Ta use count limit for name/use magic +.It Li elf_notes Ta 256 Ta max ELF notes processed +.It Li elf_phnum Ta 128 Ta max ELF program sections processed +.It Li elf_shnum Ta 32768 Ta max ELF sections processed +.El .It Fl r , Fl Fl raw Don't translate unprintable characters to \eooo. Normally .Nm translates unprintable characters to their octal representation. .It Fl s , Fl Fl special-files Normally, .Nm only attempts to read and determine the type of argument files which .Xr stat 2 reports are ordinary files. This prevents problems, because reading special files may have peculiar consequences. Specifying the .Fl s option causes .Nm to also read argument files which are block or character special files. This is useful for determining the filesystem types of the data in raw disk partitions, which are block special files. This option also causes .Nm to disregard the file size as reported by .Xr stat 2 since on some systems it reports a zero size for raw disk partitions. .It Fl v , Fl Fl version Print the version of the program and exit. .It Fl z , Fl Fl uncompress Try to look inside compressed files. .It Fl 0 , Fl Fl print0 Output a null character .Sq \e0 after the end of the filename. Nice to .Xr cut 1 the output. This does not affect the separator, which is still printed. .It Fl -help Print a help message and exit. .El .Sh FILES .Bl -tag -width __MAGIC__.mgc -compact .It Pa __MAGIC__.mgc Default compiled list of magic. .It Pa __MAGIC__ Directory containing default magic files. .El .Sh ENVIRONMENT The environment variable .Ev MAGIC can be used to set the default magic file name. If that variable is set, then .Nm will not attempt to open .Pa $HOME/.magic . .Nm adds .Dq Pa .mgc to the value of this variable as appropriate. However, .Pa file has to exist in order for .Pa file.mime to be considered. The environment variable .Ev POSIXLY_CORRECT controls (on systems that support symbolic links), whether .Nm will attempt to follow symlinks or not. If set, then .Nm follows symlink, otherwise it does not. This is also controlled by the .Fl L and .Fl h options. .Sh SEE ALSO .Xr magic __FSECTION__ , .Xr hexdump 1 , .Xr od 1 , .Xr strings 1 , +.Xr fstyp 8 .Sh STANDARDS CONFORMANCE This program is believed to exceed the System V Interface Definition of FILE(CMD), as near as one can determine from the vague language contained therein. Its behavior is mostly compatible with the System V program of the same name. This version knows more magic, however, so it will produce different (albeit more accurate) output in many cases. .\" URL: http://www.opengroup.org/onlinepubs/009695399/utilities/file.html .Pp The one significant difference between this version and System V is that this version treats any white space as a delimiter, so that spaces in pattern strings must be escaped. For example, .Bd -literal -offset indent \*[Gt]10 string language impress\ (imPRESS data) .Ed .Pp in an existing magic file would have to be changed to .Bd -literal -offset indent \*[Gt]10 string language\e impress (imPRESS data) .Ed .Pp In addition, in this version, if a pattern string contains a backslash, it must be escaped. For example .Bd -literal -offset indent 0 string \ebegindata Andrew Toolkit document .Ed .Pp in an existing magic file would have to be changed to .Bd -literal -offset indent 0 string \e\ebegindata Andrew Toolkit document .Ed .Pp SunOS releases 3.2 and later from Sun Microsystems include a .Nm command derived from the System V one, but with some extensions. This version differs from Sun's only in minor ways. It includes the extension of the .Sq \*[Am] operator, used as, for example, .Bd -literal -offset indent \*[Gt]16 long\*[Am]0x7fffffff \*[Gt]0 not stripped .Ed .Sh MAGIC DIRECTORY The magic file entries have been collected from various sources, mainly USENET, and contributed by various authors. Christos Zoulas (address below) will collect additional or corrected magic file entries. A consolidation of magic file entries will be distributed periodically. .Pp The order of entries in the magic file is significant. Depending on what system you are using, the order that they are put together may be incorrect. If your old .Nm command uses a magic file, keep the old magic file around for comparison purposes (rename it to .Pa __MAGIC__.orig ) . .Sh EXAMPLES .Bd -literal -offset indent $ file file.c file /dev/{wd0a,hda} file.c: C program text file: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped /dev/wd0a: block special (0/0) /dev/hda: block special (3/0) $ file -s /dev/wd0{b,d} /dev/wd0b: data /dev/wd0d: x86 boot sector $ file -s /dev/hda{,1,2,3,4,5,6,7,8,9,10} /dev/hda: x86 boot sector /dev/hda1: Linux/i386 ext2 filesystem /dev/hda2: x86 boot sector /dev/hda3: x86 boot sector, extended partition table /dev/hda4: Linux/i386 ext2 filesystem /dev/hda5: Linux/i386 swap file /dev/hda6: Linux/i386 swap file /dev/hda7: Linux/i386 swap file /dev/hda8: Linux/i386 swap file /dev/hda9: empty /dev/hda10: empty $ file -i file.c file /dev/{wd0a,hda} file.c: text/x-c file: application/x-executable /dev/hda: application/x-not-regular-file /dev/wd0a: application/x-not-regular-file .Ed .Sh HISTORY There has been a .Nm command in every .Dv UNIX since at least Research Version 4 (man page dated November, 1973). The System V version introduced one significant major change: the external list of magic types. This slowed the program down slightly but made it a lot more flexible. .Pp This program, based on the System V version, was written by Ian Darwin .Aq ian@darwinsys.com without looking at anybody else's source code. .Pp John Gilmore revised the code extensively, making it better than the first version. Geoff Collyer found several inadequacies and provided some magic file entries. Contributions by the .Sq \*[Am] operator by Rob McMahon, .Aq cudcv@warwick.ac.uk , 1989. .Pp Guy Harris, .Aq guy@netapp.com , made many changes from 1993 to the present. 1989. .Pp Primary development and maintenance from 1990 to the present by Christos Zoulas .Aq christos@astron.com . .Pp Altered by Chris Lowth .Aq chris@lowth.com , 2000: handle the .Fl i option to output mime type strings, using an alternative magic file and internal logic. .Pp Altered by Eric Fischer .Aq enf@pobox.com , July, 2000, to identify character codes and attempt to identify the languages of non-ASCII files. .Pp Altered by Reuben Thomas .Aq rrt@sc3d.org , 2007-2011, to improve MIME support, merge MIME and non-MIME magic, support directories as well as files of magic, apply many bug fixes, update and fix a lot of magic, improve the build system, improve the documentation, and rewrite the Python bindings in pure Python. .Pp The list of contributors to the .Sq magic directory (magic files) is too long to include here. You know who you are; thank you. Many contributors are listed in the source files. .Sh LEGAL NOTICE Copyright (c) Ian F. Darwin, Toronto, Canada, 1986-1999. Covered by the standard Berkeley Software Distribution copyright; see the file COPYING in the source distribution. .Pp The files .Pa tar.h and .Pa is_tar.c were written by John Gilmore from his public-domain .Xr tar 1 program, and are not covered by the above license. .Sh RETURN CODE .Nm returns 0 on success, and non-zero on error. .Sh BUGS .Pp Please report bugs and send patches to the bug tracker at .Pa http://bugs.gw.com/ or the mailing list at .Aq file@mx.gw.com (visit .Pa http://mx.gw.com/mailman/listinfo/file first to subscribe). .Sh TODO .Pp Fix output so that tests for MIME and APPLE flags are not needed all over the place, and actual output is only done in one place. This needs a design. Suggestion: push possible outputs on to a list, then pick the last-pushed (most specific, one hopes) value at the end, or use a default if the list is empty. This should not slow down evaluation. .Pp Continue to squash all magic bugs. See Debian BTS for a good source. .Pp Store arbitrarily long strings, for example for %s patterns, so that they can be printed out. Fixes Debian bug #271672. Would require more complex store/load code in apprentice. .Pp Add syntax for relative offsets after current level (Debian bug #466037). .Pp Make file -ki work, i.e. give multiple MIME types. .Pp Add a zip library so we can peek inside Office2007 documents to figure out what they are. .Pp Add an option to print URLs for the sources of the file descriptions. .Pp Combine script searches and add a way to map executable names to MIME types (e.g. have a magic value for !:mime which causes the resulting string to be looked up in a table). This would avoid adding the same magic repeatedly for each new hash-bang interpreter. .Pp Fix .Dq name and .Dq use to check for consistency at compile time (duplicate .Dq name , .Dq use pointing to undefined .Dq name ). Make .Dq name / .Dq use more efficient by keeping a sorted list of names. Special-case ^ to flip endianness in the parser so that it does not have to be escaped, and document it. .Sh AVAILABILITY You can obtain the original author's latest version by anonymous FTP on .Pa ftp.astron.com in the directory .Pa /pub/file/file-X.YZ.tar.gz . Index: releng/10.1/contrib/file/doc/libmagic.man =================================================================== --- releng/10.1/contrib/file/doc/libmagic.man (revision 284192) +++ releng/10.1/contrib/file/doc/libmagic.man (revision 284193) @@ -1,320 +1,383 @@ -.\" $File: libmagic.man,v 1.28 2014/03/02 14:47:16 christos Exp $ +.\" $File: libmagic.man,v 1.34 2014/12/16 23:18:40 christos Exp $ .\" .\" Copyright (c) Christos Zoulas 2003. .\" All Rights Reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice immediately at the beginning of the file, without modification, .\" this list of conditions, and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" -.Dd January 6, 2012 +.Dd December 16, 2014 .Dt LIBMAGIC 3 .Os .Sh NAME .Nm magic_open , .Nm magic_close , .Nm magic_error , .Nm magic_errno , .Nm magic_descriptor , .Nm magic_buffer , .Nm magic_setflags , .Nm magic_check , .Nm magic_compile , .Nm magic_list , .Nm magic_load , +.Nm magic_load_buffers , +.Nm magic_setparam , +.Nm magic_getparam , .Nm magic_version .Nd Magic number recognition library .Sh LIBRARY .Lb libmagic .Sh SYNOPSIS .In magic.h .Ft magic_t .Fn magic_open "int flags" .Ft void .Fn magic_close "magic_t cookie" .Ft const char * .Fn magic_error "magic_t cookie" .Ft int .Fn magic_errno "magic_t cookie" .Ft const char * .Fn magic_descriptor "magic_t cookie" "int fd" .Ft const char * .Fn magic_file "magic_t cookie" "const char *filename" .Ft const char * .Fn magic_buffer "magic_t cookie" "const void *buffer" "size_t length" .Ft int .Fn magic_setflags "magic_t cookie" "int flags" .Ft int .Fn magic_check "magic_t cookie" "const char *filename" .Ft int .Fn magic_compile "magic_t cookie" "const char *filename" .Ft int .Fn magic_list "magic_t cookie" "const char *filename" .Ft int .Fn magic_load "magic_t cookie" "const char *filename" .Ft int +.Fn magic_load_buffers "magic_t cookie" "void **buffers" "size_t *sizes" "size_t nbuffers" +.Ft int +.Fn magic_getparam "magic_t cookie" "int param" "void *value" +.Ft int +.Fn magic_setparam "magic_t cookie" "int param" "const void *value" +.Ft int .Fn magic_version "void" .Sh DESCRIPTION These functions operate on the magic database file which is described in .Xr magic __FSECTION__ . .Pp The function .Fn magic_open creates a magic cookie pointer and returns it. It returns .Dv NULL if there was an error allocating the magic cookie. The .Ar flags argument specifies how the other magic functions should behave: .Bl -tag -width MAGIC_COMPRESS .It Dv MAGIC_NONE No special handling. .It Dv MAGIC_DEBUG Print debugging messages to stderr. .It Dv MAGIC_SYMLINK If the file queried is a symlink, follow it. .It Dv MAGIC_COMPRESS If the file is compressed, unpack it and look at the contents. .It Dv MAGIC_DEVICES If the file is a block or character special device, then open the device and try to look in its contents. .It Dv MAGIC_MIME_TYPE Return a MIME type string, instead of a textual description. .It Dv MAGIC_MIME_ENCODING Return a MIME encoding, instead of a textual description. .It Dv MAGIC_MIME A shorthand for MAGIC_MIME_TYPE | MAGIC_MIME_ENCODING. .It Dv MAGIC_CONTINUE Return all matches, not just the first. .It Dv MAGIC_CHECK Check the magic database for consistency and print warnings to stderr. .It Dv MAGIC_PRESERVE_ATIME On systems that support .Xr utime 3 or .Xr utimes 2 , attempt to preserve the access time of files analysed. .It Dv MAGIC_RAW Don't translate unprintable characters to a \eooo octal representation. .It Dv MAGIC_ERROR Treat operating system errors while trying to open files and follow symlinks as real errors, instead of printing them in the magic buffer. .It Dv MAGIC_APPLE Return the Apple creator and type. .It Dv MAGIC_NO_CHECK_APPTYPE Don't check for .Dv EMX application type (only on EMX). .It Dv MAGIC_NO_CHECK_CDF Don't get extra information on MS Composite Document Files. .It Dv MAGIC_NO_CHECK_COMPRESS Don't look inside compressed files. .It Dv MAGIC_NO_CHECK_ELF Don't print ELF details. .It Dv MAGIC_NO_CHECK_ENCODING Don't check text encodings. .It Dv MAGIC_NO_CHECK_SOFT Don't consult magic files. .It Dv MAGIC_NO_CHECK_TAR Don't examine tar files. .It Dv MAGIC_NO_CHECK_TEXT Don't check for various types of text files. .It Dv MAGIC_NO_CHECK_TOKENS Don't look for known tokens inside ascii files. .El .Pp The .Fn magic_close function closes the .Xr magic __FSECTION__ database and deallocates any resources used. .Pp The .Fn magic_error function returns a textual explanation of the last error, or .Dv NULL if there was no error. .Pp The .Fn magic_errno function returns the last operating system error number .Pq Xr errno 2 that was encountered by a system call. .Pp The .Fn magic_file function returns a textual description of the contents of the .Ar filename argument, or .Dv NULL if an error occurred. If the .Ar filename is .Dv NULL , then stdin is used. .Pp The .Fn magic_descriptor function returns a textual description of the contents of the .Ar fd argument, or .Dv NULL if an error occurred. .Pp The .Fn magic_buffer function returns a textual description of the contents of the .Ar buffer argument with .Ar length bytes size. .Pp The .Fn magic_setflags function sets the .Ar flags described above. Note that using both MIME flags together can also return extra information on the charset. .Pp The .Fn magic_check function can be used to check the validity of entries in the colon separated database files passed in as .Ar filename , or .Dv NULL for the default database. It returns 0 on success and \-1 on failure. .Pp The .Fn magic_compile function can be used to compile the the colon separated list of database files passed in as .Ar filename , or .Dv NULL for the default database. It returns 0 on success and \-1 on failure. The compiled files created are named from the .Xr basename 1 of each file argument with .Dq .mgc appended to it. .Pp The .Fn magic_list function dumps all magic entries in a human readable format, dumping first the entries that are matched against binary files and then the ones that match text files. It takes and optional .Fa filename argument which is a colon separated list of database files, or .Dv NULL for the default database. .Pp The .Fn magic_load function must be used to load the the colon separated list of database files passed in as .Ar filename , or .Dv NULL for the default database file before any magic queries can performed. .Pp The default database file is named by the MAGIC environment variable. If that variable is not set, the default database file name is __MAGIC__. .Fn magic_load adds .Dq .mgc to the database filename as appropriate. +.Pp +The +.Fn magic_load_buffers +function takes an array of size +.Fa nbuffers +of +.Fa buffers +with a respective size for each in the array of +.Fa sizes +loaded with the contents of the magic databases from the filesystem. +This function can be used in environment where the magic library does +not have direct access to the filesystem, but can access the magic +database via shared memory or other IPC means. +.Pp +The +.Fn magic_getparam +and +.Fn magic_setparam +allow getting and setting various limits related to the the magic +library. +.Bl -column "MAGIC_PARAM_ELF_PHNUM_MAX" "size_t" "Default" -offset indent +.It Sy "Parameter" Ta Sy "Type" Ta Sy "Default" +.It Li MAGIC_PARAM_INDIR_MAX Ta size_t Ta 15 +.It Li MAGIC_PARAM_NAME_MAX Ta size_t Ta 30 +.It Li MAGIC_PARAM_ELF_NOTES_MAX Ta size_t Ta 256 +.It Li MAGIC_PARAM_ELF_PHNUM_MAX Ta size_t Ta 128 +.It Li MAGIC_PARAM_ELF_SHNUM_MAX Ta size_t Ta 32768 +.El +.Pp +The +.Dv MAGIC_PARAM_INDIR_RECURSION +parameter controls how many levels of recursion will be followed for +indirect magic entries. +.Pp +The +.Dv MAGIC_PARAM_NAME_RECURSION +parameter controls how many levels of recursion will be followed for +for name/use calls. +.Pp +The +.Dv MAGIC_PARAM_NAME_MAX +parameter controls the maximum number of calls for name/use. +.Pp +The +.Dv MAGIC_PARAM_NOTES_MAX +parameter controls how many ELF notes will be processed. +.Pp +The +.Dv MAGIC_PARAM_PHNUM_MAX +parameter controls how many ELF program sections will be processed. +.Pp +The +.Dv MAGIC_PARAM_SHNUM_MAX +parameter controls how many ELF sections will be processed. .Pp The .Fn magic_version command returns the version number of this library which is compiled into the shared library using the constant .Dv MAGIC_VERSION from .In magic.h . This can be used by client programs to verify that the version they compile against is the same as the version that they run against. .Sh RETURN VALUES The function .Fn magic_open returns a magic cookie on success and .Dv NULL on failure setting errno to an appropriate value. It will set errno to .Er EINVAL if an unsupported value for flags was given. The .Fn magic_list , .Fn magic_load , .Fn magic_compile , and .Fn magic_check functions return 0 on success and \-1 on failure. The .Fn magic_buffer , .Fn magic_getpath , and .Fn magic_file , functions return a string on success and .Dv NULL on failure. The .Fn magic_error function returns a textual description of the errors of the above functions, or .Dv NULL if there was no error. The .Fn magic_version always returns the version number of the library. Finally, .Fn magic_setflags returns \-1 on systems that don't support .Xr utime 3 , or .Xr utimes 2 when .Dv MAGIC_PRESERVE_ATIME is set. .Sh FILES .Bl -tag -width __MAGIC__.mgc -compact .It Pa __MAGIC__ The non-compiled default magic database. .It Pa __MAGIC__.mgc The compiled default magic database. .El .Sh SEE ALSO .Xr file __CSECTION__ , .Xr magic __FSECTION__ .Sh AUTHORS .An M\(oans Rullg\(oard Initial libmagic implementation, and configuration. .An Christos Zoulas API cleanup, error code and allocation handling. Index: releng/10.1/contrib/file/doc/magic.man =================================================================== --- releng/10.1/contrib/file/doc/magic.man (revision 284192) +++ releng/10.1/contrib/file/doc/magic.man (revision 284193) @@ -1,713 +1,718 @@ -.\" $File: magic.man,v 1.84 2014/06/03 19:01:34 christos Exp $ -.Dd June 3, 2014 +.\" $File: magic.man,v 1.85 2015/01/01 17:07:34 christos Exp $ +.Dd January 1, 2015 .Dt MAGIC __FSECTION__ .Os .\" install as magic.4 on USG, magic.5 on V7, Berkeley and Linux systems. .Sh NAME .Nm magic .Nd file command's magic pattern file .Sh DESCRIPTION This manual page documents the format of the magic file as used by the .Xr file __CSECTION__ command, version __VERSION__. The .Xr file __CSECTION__ command identifies the type of a file using, among other tests, a test for whether the file contains certain .Dq "magic patterns" . The file .Pa __MAGIC__ specifies what patterns are to be tested for, what message or MIME type to print if a particular pattern is found, and additional information to extract from the file. .Pp Each line of the file specifies a test to be performed. A test compares the data starting at a particular offset in the file with a byte value, a string or a numeric value. If the test succeeds, a message is printed. The line consists of the following fields: .Bl -tag -width ".Dv message" .It Dv offset A number specifying the offset, in bytes, into the file of the data which is to be tested. .It Dv type The type of the data to be tested. The possible values are: .Bl -tag -width ".Dv lestring16" .It Dv byte A one-byte value. .It Dv short A two-byte value in this machine's native byte order. .It Dv long A four-byte value in this machine's native byte order. .It Dv quad An eight-byte value in this machine's native byte order. .It Dv float A 32-bit single precision IEEE floating point number in this machine's native byte order. .It Dv double A 64-bit double precision IEEE floating point number in this machine's native byte order. .It Dv string A string of bytes. The string type specification can be optionally followed by /[WwcCtbT]*. The .Dq W flag compacts whitespace in the target, which must contain at least one whitespace character. If the magic has .Dv n consecutive blanks, the target needs at least .Dv n consecutive blanks to match. The .Dq w flag treats every blank in the magic as an optional blank. The .Dq c flag specifies case insensitive matching: lower case characters in the magic match both lower and upper case characters in the target, whereas upper case characters in the magic only match upper case characters in the target. The .Dq C flag specifies case insensitive matching: upper case characters in the magic match both lower and upper case characters in the target, whereas lower case characters in the magic only match upper case characters in the target. To do a complete case insensitive match, specify both .Dq c and .Dq C . The .Dq t flag forces the test to be done for text files, while the .Dq b flag forces the test to be done for binary files. The .Dq T flag causes the string to be trimmed, i.e. leading and trailing whitespace is deleted before the string is printed. .It Dv pstring A Pascal-style string where the first byte/short/int is interpreted as the unsigned length. The length defaults to byte and can be specified as a modifier. The following modifiers are supported: .Bl -tag -compact -width B .It B A byte length (default). .It H A 2 byte big endian length. .It h A 2 byte big little length. .It L A 4 byte big endian length. .It l A 4 byte big little length. .It J The length includes itself in its count. .El The string is not NUL terminated. .Dq J is used rather than the more valuable .Dq I because this type of length is a feature of the JPEG format. .It Dv date A four-byte value interpreted as a UNIX date. .It Dv qdate A eight-byte value interpreted as a UNIX date. .It Dv ldate A four-byte value interpreted as a UNIX-style date, but interpreted as local time rather than UTC. .It Dv qldate An eight-byte value interpreted as a UNIX-style date, but interpreted as local time rather than UTC. .It Dv qwdate An eight-byte value interpreted as a Windows-style date. .It Dv beid3 A 32-bit ID3 length in big-endian byte order. .It Dv beshort A two-byte value in big-endian byte order. .It Dv belong A four-byte value in big-endian byte order. .It Dv bequad An eight-byte value in big-endian byte order. .It Dv befloat A 32-bit single precision IEEE floating point number in big-endian byte order. .It Dv bedouble A 64-bit double precision IEEE floating point number in big-endian byte order. .It Dv bedate A four-byte value in big-endian byte order, interpreted as a Unix date. .It Dv beqdate An eight-byte value in big-endian byte order, interpreted as a Unix date. .It Dv beldate A four-byte value in big-endian byte order, interpreted as a UNIX-style date, but interpreted as local time rather than UTC. .It Dv beqldate An eight-byte value in big-endian byte order, interpreted as a UNIX-style date, but interpreted as local time rather than UTC. .It Dv beqwdate An eight-byte value in big-endian byte order, interpreted as a Windows-style date. .It Dv bestring16 A two-byte unicode (UCS16) string in big-endian byte order. .It Dv leid3 A 32-bit ID3 length in little-endian byte order. .It Dv leshort A two-byte value in little-endian byte order. .It Dv lelong A four-byte value in little-endian byte order. .It Dv lequad An eight-byte value in little-endian byte order. .It Dv lefloat A 32-bit single precision IEEE floating point number in little-endian byte order. .It Dv ledouble A 64-bit double precision IEEE floating point number in little-endian byte order. .It Dv ledate A four-byte value in little-endian byte order, interpreted as a UNIX date. .It Dv leqdate An eight-byte value in little-endian byte order, interpreted as a UNIX date. .It Dv leldate A four-byte value in little-endian byte order, interpreted as a UNIX-style date, but interpreted as local time rather than UTC. .It Dv leqldate An eight-byte value in little-endian byte order, interpreted as a UNIX-style date, but interpreted as local time rather than UTC. .It Dv leqwdate An eight-byte value in little-endian byte order, interpreted as a Windows-style date. .It Dv lestring16 A two-byte unicode (UCS16) string in little-endian byte order. .It Dv melong A four-byte value in middle-endian (PDP-11) byte order. .It Dv medate A four-byte value in middle-endian (PDP-11) byte order, interpreted as a UNIX date. .It Dv meldate A four-byte value in middle-endian (PDP-11) byte order, interpreted as a UNIX-style date, but interpreted as local time rather than UTC. .It Dv indirect Starting at the given offset, consult the magic database again. +The offset of th +.Dv indirect +magic is by default absolute in the file, but one can specify +.Dv /r +to indicate that the offset is relative from the beginning of the entry. .It Dv name Define a .Dq named magic instance that can be called from another .Dv use magic entry, like a subroutine call. Named instance direct magic offsets are relative to the offset of the previous matched entry, but indirect offsets are relative to the beginning of the file as usual. Named magic entries always match. .It Dv use Recursively call the named magic starting from the current offset. If the name of the referenced begins with a .Dv ^ then the endianness of the magic is switched; if the magic mentioned .Dv leshort for example, it is treated as .Dv beshort and vice versa. This is useful to avoid duplicating the rules for different endianness. .It Dv regex A regular expression match in extended POSIX regular expression syntax (like egrep). Regular expressions can take exponential time to process, and their performance is hard to predict, so their use is discouraged. When used in production environments, their performance should be carefully checked. The size of the string to search should also be limited by specifying .Dv / , to avoid performance issues scanning long files. The type specification can also be optionally followed by .Dv /[c][s][l] . The .Dq c flag makes the match case insensitive, while the .Dq s flag update the offset to the start offset of the match, rather than the end. The .Dq l modifier, changes the limit of length to mean number of lines instead of a byte count. Lines are delimited by the platforms native line delimiter. When a line count is specified, an implicit byte count also computed assuming each line is 80 characters long. If neither a byte or line count is specified, the search is limited automatically to 8KiB. .Dv ^ and .Dv $ match the beginning and end of individual lines, respectively, not beginning and end of file. .It Dv search A literal string search starting at the given offset. The same modifier flags can be used as for string patterns. The search expression must contain the range in the form .Dv /number, that is the number of positions at which the match will be attempted, starting from the start offset. This is suitable for searching larger binary expressions with variable offsets, using .Dv \e escapes for special characters. The order of modifier and number is not relevant. .It Dv default This is intended to be used with the test .Em x (which is always true) and it has no type. It matches when no other test at that continuation level has matched before. Clearing that matched tests for a continuation level, can be done using the .Dv clear test. .It Dv clear This test is always true and clears the match flag for that continuation level. It is intended to be used with the .Dv default test. .El .Pp For compatibility with the Single .Ux Standard, the type specifiers .Dv dC and .Dv d1 are equivalent to .Dv byte , the type specifiers .Dv uC and .Dv u1 are equivalent to .Dv ubyte , the type specifiers .Dv dS and .Dv d2 are equivalent to .Dv short , the type specifiers .Dv uS and .Dv u2 are equivalent to .Dv ushort , the type specifiers .Dv dI , .Dv dL , and .Dv d4 are equivalent to .Dv long , the type specifiers .Dv uI , .Dv uL , and .Dv u4 are equivalent to .Dv ulong , the type specifier .Dv d8 is equivalent to .Dv quad , the type specifier .Dv u8 is equivalent to .Dv uquad , and the type specifier .Dv s is equivalent to .Dv string . In addition, the type specifier .Dv dQ is equivalent to .Dv quad and the type specifier .Dv uQ is equivalent to .Dv uquad . .Pp Each top-level magic pattern (see below for an explanation of levels) is classified as text or binary according to the types used. Types .Dq regex and .Dq search are classified as text tests, unless non-printable characters are used in the pattern. All other tests are classified as binary. A top-level pattern is considered to be a test text when all its patterns are text patterns; otherwise, it is considered to be a binary pattern. When matching a file, binary patterns are tried first; if no match is found, and the file looks like text, then its encoding is determined and the text patterns are tried. .Pp The numeric types may optionally be followed by .Dv \*[Am] and a numeric value, to specify that the value is to be AND'ed with the numeric value before any comparisons are done. Prepending a .Dv u to the type indicates that ordered comparisons should be unsigned. .It Dv test The value to be compared with the value from the file. If the type is numeric, this value is specified in C form; if it is a string, it is specified as a C string with the usual escapes permitted (e.g. \en for new-line). .Pp Numeric values may be preceded by a character indicating the operation to be performed. It may be .Dv = , to specify that the value from the file must equal the specified value, .Dv \*[Lt] , to specify that the value from the file must be less than the specified value, .Dv \*[Gt] , to specify that the value from the file must be greater than the specified value, .Dv \*[Am] , to specify that the value from the file must have set all of the bits that are set in the specified value, .Dv ^ , to specify that the value from the file must have clear any of the bits that are set in the specified value, or .Dv ~ , the value specified after is negated before tested. .Dv x , to specify that any value will match. If the character is omitted, it is assumed to be .Dv = . Operators .Dv \*[Am] , .Dv ^ , and .Dv ~ don't work with floats and doubles. The operator .Dv !\& specifies that the line matches if the test does .Em not succeed. .Pp Numeric values are specified in C form; e.g. .Dv 13 is decimal, .Dv 013 is octal, and .Dv 0x13 is hexadecimal. .Pp Numeric operations are not performed on date types, instead the numeric value is interpreted as an offset. .Pp For string values, the string from the file must match the specified string. The operators .Dv = , .Dv \*[Lt] and .Dv \*[Gt] (but not .Dv \*[Am] ) can be applied to strings. The length used for matching is that of the string argument in the magic file. This means that a line can match any non-empty string (usually used to then print the string), with .Em \*[Gt]\e0 (because all non-empty strings are greater than the empty string). .Pp Dates are treated as numerical values in the respective internal representation. .Pp The special test .Em x always evaluates to true. .It Dv message The message to be printed if the comparison succeeds. If the string contains a .Xr printf 3 format specification, the value from the file (with any specified masking performed) is printed using the message as the format string. If the string begins with .Dq \eb , the message printed is the remainder of the string with no whitespace added before it: multiple matches are normally separated by a single space. .El .Pp An APPLE 4+4 character APPLE creator and type can be specified as: .Bd -literal -offset indent !:apple CREATYPE .Ed .Pp A MIME type is given on a separate line, which must be the next non-blank or comment line after the magic line that identifies the file type, and has the following format: .Bd -literal -offset indent !:mime MIMETYPE .Ed .Pp i.e. the literal string .Dq !:mime followed by the MIME type. .Pp An optional strength can be supplied on a separate line which refers to the current magic description using the following format: .Bd -literal -offset indent !:strength OP VALUE .Ed .Pp The operand .Dv OP can be: .Dv + , .Dv - , .Dv * , or .Dv / and .Dv VALUE is a constant between 0 and 255. This constant is applied using the specified operand to the currently computed default magic strength. .Pp Some file formats contain additional information which is to be printed along with the file type or need additional tests to determine the true file type. These additional tests are introduced by one or more .Em \*[Gt] characters preceding the offset. The number of .Em \*[Gt] on the line indicates the level of the test; a line with no .Em \*[Gt] at the beginning is considered to be at level 0. Tests are arranged in a tree-like hierarchy: if the test on a line at level .Em n succeeds, all following tests at level .Em n+1 are performed, and the messages printed if the tests succeed, until a line with level .Em n (or less) appears. For more complex files, one can use empty messages to get just the "if/then" effect, in the following way: .Bd -literal -offset indent 0 string MZ \*[Gt]0x18 leshort \*[Lt]0x40 MS-DOS executable \*[Gt]0x18 leshort \*[Gt]0x3f extended PC executable (e.g., MS Windows) .Ed .Pp Offsets do not need to be constant, but can also be read from the file being examined. If the first character following the last .Em \*[Gt] is a .Em \&( then the string after the parenthesis is interpreted as an indirect offset. That means that the number after the parenthesis is used as an offset in the file. The value at that offset is read, and is used again as an offset in the file. Indirect offsets are of the form: .Em (( x [.[bislBISL]][+\-][ y ]) . The value of .Em x is used as an offset in the file. A byte, id3 length, short or long is read at that offset depending on the .Em [bislBISLm] type specifier. The capitalized types interpret the number as a big endian value, whereas the small letter versions interpret the number as a little endian value; the .Em m type interprets the number as a middle endian (PDP-11) value. To that number the value of .Em y is added and the result is used as an offset in the file. The default type if one is not specified is long. .Pp That way variable length structures can be examined: .Bd -literal -offset indent # MS Windows executables are also valid MS-DOS executables 0 string MZ \*[Gt]0x18 leshort \*[Lt]0x40 MZ executable (MS-DOS) # skip the whole block below if it is not an extended executable \*[Gt]0x18 leshort \*[Gt]0x3f \*[Gt]\*[Gt](0x3c.l) string PE\e0\e0 PE executable (MS-Windows) \*[Gt]\*[Gt](0x3c.l) string LX\e0\e0 LX executable (OS/2) .Ed .Pp This strategy of examining has a drawback: You must make sure that you eventually print something, or users may get empty output (like, when there is neither PE\e0\e0 nor LE\e0\e0 in the above example) .Pp If this indirect offset cannot be used directly, simple calculations are possible: appending .Em [+-*/%\*[Am]|^]number inside parentheses allows one to modify the value read from the file before it is used as an offset: .Bd -literal -offset indent # MS Windows executables are also valid MS-DOS executables 0 string MZ # sometimes, the value at 0x18 is less that 0x40 but there's still an # extended executable, simply appended to the file \*[Gt]0x18 leshort \*[Lt]0x40 \*[Gt]\*[Gt](4.s*512) leshort 0x014c COFF executable (MS-DOS, DJGPP) \*[Gt]\*[Gt](4.s*512) leshort !0x014c MZ executable (MS-DOS) .Ed .Pp Sometimes you do not know the exact offset as this depends on the length or position (when indirection was used before) of preceding fields. You can specify an offset relative to the end of the last up-level field using .Sq \*[Am] as a prefix to the offset: .Bd -literal -offset indent 0 string MZ \*[Gt]0x18 leshort \*[Gt]0x3f \*[Gt]\*[Gt](0x3c.l) string PE\e0\e0 PE executable (MS-Windows) # immediately following the PE signature is the CPU type \*[Gt]\*[Gt]\*[Gt]\*[Am]0 leshort 0x14c for Intel 80386 \*[Gt]\*[Gt]\*[Gt]\*[Am]0 leshort 0x184 for DEC Alpha .Ed .Pp Indirect and relative offsets can be combined: .Bd -literal -offset indent 0 string MZ \*[Gt]0x18 leshort \*[Lt]0x40 \*[Gt]\*[Gt](4.s*512) leshort !0x014c MZ executable (MS-DOS) # if it's not COFF, go back 512 bytes and add the offset taken # from byte 2/3, which is yet another way of finding the start # of the extended executable \*[Gt]\*[Gt]\*[Gt]\*[Am](2.s-514) string LE LE executable (MS Windows VxD driver) .Ed .Pp Or the other way around: .Bd -literal -offset indent 0 string MZ \*[Gt]0x18 leshort \*[Gt]0x3f \*[Gt]\*[Gt](0x3c.l) string LE\e0\e0 LE executable (MS-Windows) # at offset 0x80 (-4, since relative offsets start at the end # of the up-level match) inside the LE header, we find the absolute # offset to the code area, where we look for a specific signature \*[Gt]\*[Gt]\*[Gt](\*[Am]0x7c.l+0x26) string UPX \eb, UPX compressed .Ed .Pp Or even both! .Bd -literal -offset indent 0 string MZ \*[Gt]0x18 leshort \*[Gt]0x3f \*[Gt]\*[Gt](0x3c.l) string LE\e0\e0 LE executable (MS-Windows) # at offset 0x58 inside the LE header, we find the relative offset # to a data area where we look for a specific signature \*[Gt]\*[Gt]\*[Gt]\*[Am](\*[Am]0x54.l-3) string UNACE \eb, ACE self-extracting archive .Ed .Pp If you have to deal with offset/length pairs in your file, even the second value in a parenthesized expression can be taken from the file itself, using another set of parentheses. Note that this additional indirect offset is always relative to the start of the main indirect offset. .Bd -literal -offset indent 0 string MZ \*[Gt]0x18 leshort \*[Gt]0x3f \*[Gt]\*[Gt](0x3c.l) string PE\e0\e0 PE executable (MS-Windows) # search for the PE section called ".idata"... \*[Gt]\*[Gt]\*[Gt]\*[Am]0xf4 search/0x140 .idata # ...and go to the end of it, calculated from start+length; # these are located 14 and 10 bytes after the section name \*[Gt]\*[Gt]\*[Gt]\*[Gt](\*[Am]0xe.l+(-4)) string PK\e3\e4 \eb, ZIP self-extracting archive .Ed .Pp If you have a list of known avalues at a particular continuation level, and you want to provide a switch-like default case: .Bd -literal -offset indent # clear that continuation level match \*[Gt]18 clear \*[Gt]18 lelong 1 one \*[Gt]18 lelong 2 two \*[Gt]18 default x # print default match \*[Gt]\*[Gt]18 lelong x unmatched 0x%x .Ed .Sh SEE ALSO .Xr file __CSECTION__ \- the command that reads this file. .Sh BUGS The formats .Dv long , .Dv belong , .Dv lelong , .Dv melong , .Dv short , .Dv beshort , and .Dv leshort do not depend on the length of the C data types .Dv short and .Dv long on the platform, even though the Single .Ux Specification implies that they do. However, as OS X Mountain Lion has passed the Single .Ux Specification validation suite, and supplies a version of .Xr file __CSECTION__ in which they do not depend on the sizes of the C data types and that is built for a 64-bit environment in which .Dv long is 8 bytes rather than 4 bytes, presumably the validation suite does not test whether, for example .Dv long refers to an item with the same size as the C data type .Dv long . There should probably be .Dv type names .Dv int8 , .Dv uint8 , .Dv int16 , .Dv uint16 , .Dv int32 , .Dv uint32 , .Dv int64 , and .Dv uint64 , and specified-byte-order variants of them, to make it clearer that those types have specified widths. .\" .\" From: guy@sun.uucp (Guy Harris) .\" Newsgroups: net.bugs.usg .\" Subject: /etc/magic's format isn't well documented .\" Message-ID: <2752@sun.uucp> .\" Date: 3 Sep 85 08:19:07 GMT .\" Organization: Sun Microsystems, Inc. .\" Lines: 136 .\" .\" Here's a manual page for the format accepted by the "file" made by adding .\" the changes I posted to the S5R2 version. .\" .\" Modified for Ian Darwin's version of the file command. Index: releng/10.1/contrib/file/magic/Magdir/rinex =================================================================== --- releng/10.1/contrib/file/magic/Magdir/rinex (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/rinex (nonexistent) @@ -1,44 +0,0 @@ - -#------------------------------------------------------------------------------ -# $File: rinex,v 1.4 2011/05/03 01:44:17 christos Exp $ -# rinex: file(1) magic for RINEX files -# http://igscb.jpl.nasa.gov/igscb/data/format/rinex210.txt -# ftp://cddis.gsfc.nasa.gov/pub/reports/formats/rinex300.pdf -# data for testing: ftp://cddis.gsfc.nasa.gov/pub/gps/data -60 string RINEX ->80 search/256 XXRINEXB RINEX Data, GEO SBAS Broadcast ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/broadcast ->80 search/256 XXRINEXD RINEX Data, Observation (Hatanaka comp) ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/observation ->80 search/256 XXRINEXC RINEX Data, Clock ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/clock ->80 search/256 XXRINEXH RINEX Data, GEO SBAS Navigation ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/navigation ->80 search/256 XXRINEXG RINEX Data, GLONASS Navigation ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/navigation ->80 search/256 XXRINEXL RINEX Data, Galileo Navigation ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/navigation ->80 search/256 XXRINEXM RINEX Data, Meteorological ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/meteorological ->80 search/256 XXRINEXN RINEX Data, Navigation ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/navigation ->80 search/256 XXRINEXO RINEX Data, Observation ->>&32 string x \b, date %15.15s ->>5 string x \b, version %6.6s -!:mime rinex/observation Index: releng/10.1/contrib/file/magic/Magdir/android =================================================================== --- releng/10.1/contrib/file/magic/Magdir/android (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/android (revision 284193) @@ -1,100 +1,139 @@ #------------------------------------------------------------ -# $File: android,v 1.4 2014/06/03 19:01:34 christos Exp $ +# $File: android,v 1.7 2014/11/10 05:08:23 christos Exp $ # Various android related magic entries #------------------------------------------------------------ # Dalvik .dex format. http://retrodev.com/android/dexformat.html # From "Mike Fleming" # Fixed to avoid regexec 17 errors on some dex files # From "Tim Strazzere" 0 string dex\n >0 regex dex\n[0-9]{2}\0 Dalvik dex file >4 string >000 version %s 0 string dey\n >0 regex dey\n[0-9]{2}\0 Dalvik dex file (optimized for host) >4 string >000 version %s -# http://android.stackexchange.com/questions/23357/\ -# is-there-a-way-to-look-inside-and-modify-an-adb-backup-created-file/\ -# 23608#23608 -0 string ANDROID\040BACKUP\n Android Backup ->15 string 1\n \b, version 1 ->17 string 0\n \b, uncompressed ->17 string 1\n \b, compressed ->19 string none\n \b, unencrypted ->19 string AES-256\n \b, encrypted AES-256 - # Android bootimg format # From https://android.googlesource.com/\ # platform/system/core/+/master/mkbootimg/bootimg.h 0 string ANDROID! Android bootimg +>1024 string LOKI\01 \b, LOKI'd >8 lelong >0 \b, kernel >>12 lelong >0 \b (0x%x) >16 lelong >0 \b, ramdisk >>20 lelong >0 \b (0x%x) >24 lelong >0 \b, second stage >>28 lelong >0 \b (0x%x) >36 lelong >0 \b, page size: %d >38 string >0 \b, name: %s >64 string >0 \b, cmdline (%s) -# Dalvik .dex format. http://retrodev.com/android/dexformat.html -# From "Mike Fleming" -# Fixed to avoid regexec 17 errors on some dex files -# From "Tim Strazzere" -0 string dex\n ->0 regex dex\n[0-9]{2}\0 Dalvik dex file ->4 string >000 version %s -0 string dey\n ->0 regex dey\n[0-9]{2}\0 Dalvik dex file (optimized for host) ->4 string >000 version %s -# http://android.stackexchange.com/questions/23357/\ -# is-there-a-way-to-look-inside-and-modify-an-adb-backup-created-file/\ -# 23608#23608 -0 string ANDROID\040BACKUP\n Android Backup ->15 string 1\n \b, version 1 ->17 string 0\n \b, uncompressed ->17 string 1\n \b, compressed ->19 string none\n \b, unencrypted ->19 string AES-256\n \b, encrypted AES-256 - -# Android bootimg format -# From https://android.googlesource.com/\ -# platform/system/core/+/master/mkbootimg/bootimg.h -0 string ANDROID! Android bootimg ->8 lelong >0 \b, kernel ->>12 lelong >0 \b (0x%x) ->16 lelong >0 \b, ramdisk ->>20 lelong >0 \b (0x%x) ->24 lelong >0 \b, second stage ->>28 lelong >0 \b (0x%x) ->36 lelong >0 \b, page size: %d ->38 string >0 \b, name: %s ->64 string >0 \b, cmdline (%s) - # Android Backup archive # From: Ariel Shkedi # File extension: .ab # No mime-type defined # URL: https://github.com/android/platform_frameworks_base/blob/\ # 0bacfd2ba68d21a68a3df345b830bc2a1e515b5a/services/java/com/\ # android/server/BackupManagerService.java#L2367 # After the header comes a tar file # If compressed, the entire tar file is compressed with JAVA deflate # # Include the version number hardcoded with the magic string to avoid # false positives 0 string/b ANDROID\ BACKUP\n1\n Android Backup >17 string 0\n \b, Not-Compressed >17 string 1\n \b, Compressed # any string as long as it's not the word none (which is matched below) >>19 regex/1l \^([^n\n]|n[^o]|no[^n]|non[^e]|none.+).* \b, Encrypted (%s) >>19 string none\n \b, Not-Encrypted # Commented out because they don't seem useful to print # (but they are part of the header - the tar file comes after them): #>>>&1 regex/1l .* \b, Password salt: %s #>>>>&1 regex/1l .* \b, Master salt: %s #>>>>>&1 regex/1l .* \b, PBKDF2 rounds: %s #>>>>>>&1 regex/1l .* \b, IV: %s #>>>>>>>&1 regex/1l .* \b, Key: %s + +# *.pit files by Joerg Jenderek +# http://forum.xda-developers.com/showthread.php?p=9122369 +# http://forum.xda-developers.com/showthread.php?t=816449 +# Partition Information Table for Samsung's smartphone with Android +# used by flash software Odin +0 ulelong 0x12349876 +# 1st pit entry marker +>0x01C ulequad&0xFFFFFFFCFFFFFFFC =0x0000000000000000 +# minimal 13 and maximal 18 PIT entries found +>>4 ulelong <128 Partition Information Table for Samsung smartphone +>>>4 ulelong x \b, %d entries +# 1. pit entry +>>>4 ulelong >0 \b; #1 +>>>0x01C use PIT-entry +>>>4 ulelong >1 \b; #2 +>>>0x0A0 use PIT-entry +>>>4 ulelong >2 \b; #3 +>>>0x124 use PIT-entry +>>>4 ulelong >3 \b; #4 +>>>0x1A8 use PIT-entry +>>>4 ulelong >4 \b; #5 +>>>0x22C use PIT-entry +>>>4 ulelong >5 \b; #6 +>>>0x2B0 use PIT-entry +>>>4 ulelong >6 \b; #7 +>>>0x334 use PIT-entry +>>>4 ulelong >7 \b; #8 +>>>0x3B8 use PIT-entry +>>>4 ulelong >8 \b; #9 +>>>0x43C use PIT-entry +>>>4 ulelong >9 \b; #10 +>>>0x4C0 use PIT-entry +>>>4 ulelong >10 \b; #11 +>>>0x544 use PIT-entry +>>>4 ulelong >11 \b; #12 +>>>0x5C8 use PIT-entry +>>>4 ulelong >12 \b; #13 +>>>>0x64C use PIT-entry +# 14. pit entry +>>>4 ulelong >13 \b; #14 +>>>>0x6D0 use PIT-entry +>>>4 ulelong >14 \b; #15 +>>>0x754 use PIT-entry +>>>4 ulelong >15 \b; #16 +>>>0x7D8 use PIT-entry +>>>4 ulelong >16 \b; #17 +>>>0x85C use PIT-entry +# 18. pit entry +>>>4 ulelong >17 \b; #18 +>>>0x8E0 use PIT-entry + +0 name PIT-entry +# garbage value implies end of pit entries +>0x00 ulequad&0xFFFFFFFCFFFFFFFC =0x0000000000000000 +# skip empty partition name +>>0x24 ubyte !0 +# partition name +>>>0x24 string >\0 %-.32s +# flags +>>>0x0C ulelong&0x00000002 2 \b+RW +# partition ID: +# 0~IPL,MOVINAND,GANG;1~PIT,GPT;2~HIDDEN;3~SBL,HIDDEN;4~SBL2,HIDDEN;5~BOOT;6~KENREl,RECOVER,misc;7~RECOVER +# ;11~MODEM;20~efs;21~PARAM;22~FACTORY,SYSTEM;23~DBDATAFS,USERDATA;24~CACHE;80~BOOTLOADER;81~TZSW +>>>0x08 ulelong x (0x%x) +# filename +>>>0x44 string >\0 "%-.64s" +#>>>0x18 ulelong >0 +# blocksize in 512 byte units ? +#>>>>0x18 ulelong x \b, %db +# partition size in blocks ? +#>>>>0x22 ulelong x \b*%d + +# Android bootimg format +# From https://android.googlesource.com/\ +# platform/system/core/+/master/libsparse/sparse_format.h +0 lelong 0xed26ff3a Android sparse image +>4 leshort x \b, version: %d +>6 leshort x \b.%d +>16 lelong x \b, Total of %d +>12 lelong x \b %d-byte output blocks in +>20 lelong x \b %d input chunks. Index: releng/10.1/contrib/file/magic/Magdir/animation =================================================================== --- releng/10.1/contrib/file/magic/Magdir/animation (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/animation (revision 284193) @@ -1,896 +1,1008 @@ #------------------------------------------------------------------------------ -# $File: animation,v 1.53 2014/04/30 21:41:02 christos Exp $ +# $File: animation,v 1.56 2014/10/23 23:12:51 christos Exp $ # animation: file(1) magic for animation/movie formats # # animation formats # MPEG, FLI, DL originally from vax@ccwf.cc.utexas.edu (VaX#n8) # FLC, SGI, Apple originally from Daniel Quinlan (quinlan@yggdrasil.com) # SGI and Apple formats 0 string MOVI Silicon Graphics movie file !:mime video/x-sgi-movie 4 string moov Apple QuickTime !:mime video/quicktime >12 string mvhd \b movie (fast start) >12 string mdra \b URL >12 string cmov \b movie (fast start, compressed header) >12 string rmra \b multiple URLs 4 string mdat Apple QuickTime movie (unoptimized) !:mime video/quicktime #4 string wide Apple QuickTime movie (unoptimized) #!:mime video/quicktime #4 string skip Apple QuickTime movie (modified) #!:mime video/quicktime #4 string free Apple QuickTime movie (modified) #!:mime video/quicktime 4 string idsc Apple QuickTime image (fast start) !:mime image/x-quicktime #4 string idat Apple QuickTime image (unoptimized) #!:mime image/x-quicktime 4 string pckg Apple QuickTime compressed archive !:mime application/x-quicktime-player 4 string/W jP JPEG 2000 image !:mime image/jp2 +# http://www.ftyps.com/ with local additions 4 string ftyp ISO Media ->8 string isom \b, MPEG v4 system, version 1 -!:mime video/mp4 ->8 string iso2 \b, MPEG v4 system, part 12 revision ->8 string mp41 \b, MPEG v4 system, version 1 -!:mime video/mp4 ->8 string mp42 \b, MPEG v4 system, version 2 -!:mime video/mp4 ->8 string mp7t \b, MPEG v4 system, MPEG v7 XML ->8 string mp7b \b, MPEG v4 system, MPEG v7 binary XML ->8 string/W jp2 \b, JPEG 2000 -!:mime image/jp2 +>8 string 3g2 \b, MPEG v4 system, 3GPP2 +!:mime video/3gpp2 +>>11 byte 4 \b v4 (H.263/AMR GSM 6.10) +>>11 byte 5 \b v5 (H.263/AMR GSM 6.10) +>>11 byte 6 \b v6 (ITU H.264/AMR GSM 6.10) +>>11 byte a \b C.S0050-0 V1.0 +>>11 byte b \b C.S0050-0-A V1.0.0 +>>11 byte c \b C.S0050-0-B V1.0 >8 string 3ge \b, MPEG v4 system, 3GPP !:mime video/3gpp +>>11 byte 6 \b, Release 6 MBMS Extended Presentations +>>11 byte 7 \b, Release 7 MBMS Extended Presentations >8 string 3gg \b, MPEG v4 system, 3GPP +>11 byte 6 \b, Release 6 General Profile !:mime video/3gpp >8 string 3gp \b, MPEG v4 system, 3GPP +>11 byte 1 \b, Release %d (non existent) +>11 byte 2 \b, Release %d (non existent) +>11 byte 3 \b, Release %d (non existent) +>11 byte 4 \b, Release %d +>11 byte 5 \b, Release %d +>11 byte 6 \b, Release %d +>11 byte 7 \b, Release %d Streaming Servers !:mime video/3gpp >8 string 3gs \b, MPEG v4 system, 3GPP +>11 byte 7 \b, Release %d Streaming Servers !:mime video/3gpp ->8 string 3g2 \b, MPEG v4 system, 3GPP2 +>8 string avc1 \b, MPEG v4 system, 3GPP JVT AVC [ISO 14496-12:2005] +!:mime video/mp4 +>8 string/W qt \b, Apple QuickTime movie +!:mime video/quicktime +>8 string CAEP \b, Canon Digital Camera +>8 string caqv \b, Casio Digital Camera +>8 string CDes \b, Convergent Design +>8 string da0a \b, DMB MAF w/ MPEG Layer II aud, MOT slides, DLS, JPG/PNG/MNG +>8 string da0b \b, DMB MAF, ext DA0A, with 3GPP timed text, DID, TVA, REL, IPMP +>8 string da1a \b, DMB MAF audio with ER-BSAC audio, JPG/PNG/MNG images +>8 string da1b \b, DMB MAF, ext da1a, with 3GPP timed text, DID, TVA, REL, IPMP +>8 string da2a \b, DMB MAF aud w/ HE-AAC v2 aud, MOT slides, DLS, JPG/PNG/MNG +>8 string da2b \b, DMB MAF, ext da2a, with 3GPP timed text, DID, TVA, REL, IPMP +>8 string da3a \b, DMB MAF aud with HE-AAC aud, JPG/PNG/MNG images +>8 string da3b \b, DMB MAF, ext da3a w/ BIFS, 3GPP, DID, TVA, REL, IPMP +>8 string dmb1 \b, DMB MAF supporting all the components defined in the spec +>8 string dmpf \b, Digital Media Project +>8 string drc1 \b, Dirac (wavelet compression), encap in ISO base media (MP4) +>8 string dv1a \b, DMB MAF vid w/ AVC vid, ER-BSAC aud, BIFS, JPG/PNG/MNG, TS +>8 string dv1b \b, DMB MAF, ext dv1a, with 3GPP timed text, DID, TVA, REL, IPMP +>8 string dv2a \b, DMB MAF vid w/ AVC vid, HE-AAC v2 aud, BIFS, JPG/PNG/MNG, TS +>8 string dv2b \b, DMB MAF, ext dv2a, with 3GPP timed text, DID, TVA, REL, IPMP +>8 string dv3a \b, DMB MAF vid w/ AVC vid, HE-AAC aud, BIFS, JPG/PNG/MNG, TS +>8 string dv3b \b, DMB MAF, ext dv3a, with 3GPP timed text, DID, TVA, REL, IPMP +>8 string dvr1 \b, DVB (.DVB) over RTP +!:mime video/vnd.dvb.file +>8 string dvt1 \b, DVB (.DVB) over MPEG-2 Transport Stream +!:mime video/vnd.dvb.file +>8 string F4V \b, Video for Adobe Flash Player 9+ (.F4V) +!:mime video/mp4 +>8 string F4P \b, Protected Video for Adobe Flash Player 9+ (.F4P) +!:mime video/mp4 +>8 string F4A \b, Audio for Adobe Flash Player 9+ (.F4A) +!:mime audio/mp4 +>8 string F4B \b, Audio Book for Adobe Flash Player 9+ (.F4B) +!:mime audio/mp4 +>8 string isc2 \b, ISMACryp 2.0 Encrypted File +# ?/enc-isoff-generic +>8 string iso2 \b, MP4 Base Media v2 [ISO 14496-12:2005] +!:mime video/mp4 +>8 string isom \b, MP4 Base Media v1 [IS0 14496-12:2003] +!:mime video/mp4 +>8 string/W jp2 \b, JPEG 2000 +!:mime image/jp2 +>8 string JP2 \b, JPEG 2000 Image (.JP2) [ISO 15444-1 ?] +!:mime image/jp2 +>8 string JP20 \b, Unknown, from GPAC samples (prob non-existent) +>8 string jpm \b, JPEG 2000 Compound Image (.JPM) [ISO 15444-6] +!:mime image/jpm +>8 string jpx \b, JPEG 2000 w/ extensions (.JPX) [ISO 15444-2] +!:mime image/jpx +>8 string KDDI \b, 3GPP2 EZmovie for KDDI 3G cellphones !:mime video/3gpp2 ->>11 byte 4 \b v4 (H.263/AMR GSM 6.10) ->>11 byte 5 \b v5 (H.263/AMR GSM 6.10) ->>11 byte 6 \b v6 (ITU H.264/AMR GSM 6.10) +>8 string M4A \b, Apple iTunes ALAC/AAC-LC (.M4A) Audio +!:mime audio/x-m4a +>8 string M4B \b, Apple iTunes ALAC/AAC-LC (.M4B) Audio Book +!:mime audio/mp4 +>8 string M4P \b, Apple iTunes ALAC/AAC-LC (.M4P) AES Protected Audio +!:mime video/mp4 +>8 string M4V \b, Apple iTunes Video (.M4V) Video +!:mime video/x-m4v +>8 string M4VH \b, Apple TV (.M4V) +!:mime video/x-m4v +>8 string M4VP \b, Apple iPhone (.M4V) +!:mime video/x-m4v +>8 string mj2s \b, Motion JPEG 2000 [ISO 15444-3] Simple Profile +!:mime video/mj2 +>8 string mjp2 \b, Motion JPEG 2000 [ISO 15444-3] General Profile +!:mime video/mj2 +>8 string mmp4 \b, MPEG-4/3GPP Mobile Profile (.MP4 / .3GP) (for NTT) +!:mime video/mp4 +>8 string mobi \b, MPEG-4, MOBI format +!:mime video/mp4 +>8 string mp21 \b, MPEG-21 [ISO/IEC 21000-9] +>8 string mp41 \b, MP4 v1 [ISO 14496-1:ch13] +!:mime video/mp4 +>8 string mp42 \b, MP4 v2 [ISO 14496-14] +!:mime video/mp4 +>8 string mp71 \b, MP4 w/ MPEG-7 Metadata [per ISO 14496-12] +>8 string mp7t \b, MPEG v4 system, MPEG v7 XML +>8 string mp7b \b, MPEG v4 system, MPEG v7 binary XML >8 string mmp4 \b, MPEG v4 system, 3GPP Mobile !:mime video/mp4 ->8 string avc1 \b, MPEG v4 system, 3GPP JVT AVC -!:mime video/3gpp ->8 string/W M4A \b, MPEG v4 system, iTunes AAC-LC +>8 string MPPI \b, Photo Player, MAF [ISO/IEC 23000-3] +>8 string mqt \b, Sony / Mobile QuickTime (.MQV) US Pat 7,477,830 +!:mime video/quicktime +>8 string MSNV \b, MPEG-4 (.MP4) for SonyPSP !:mime audio/mp4 ->8 string/W M4V \b, MPEG v4 system, iTunes AVC-LC +>8 string NDAS \b, MP4 v2 [ISO 14496-14] Nero Digital AAC Audio +!:mime audio/mp4 +>8 string NDSC \b, MPEG-4 (.MP4) Nero Cinema Profile !:mime video/mp4 ->8 string/W M4P \b, MPEG v4 system, iTunes AES encrypted ->8 string/W M4B \b, MPEG v4 system, iTunes bookmarked ->8 string/W qt \b, Apple QuickTime movie +>8 string NDSH \b, MPEG-4 (.MP4) Nero HDTV Profile +!:mime video/mp4 +>8 string NDSM \b, MPEG-4 (.MP4) Nero Mobile Profile +!:mime video/mp4 +>8 string NDSP \b, MPEG-4 (.MP4) Nero Portable Profile +!:mime video/mp4 +>8 string NDSS \b, MPEG-4 (.MP4) Nero Standard Profile +!:mime video/mp4 +>8 string NDXC \b, H.264/MPEG-4 AVC (.MP4) Nero Cinema Profile +!:mime video/mp4 +>8 string NDXH \b, H.264/MPEG-4 AVC (.MP4) Nero HDTV Profile +!:mime video/mp4 +>8 string NDXM \b, H.264/MPEG-4 AVC (.MP4) Nero Mobile Profile +!:mime video/mp4 +>8 string NDXP \b, H.264/MPEG-4 AVC (.MP4) Nero Portable Profile +!:mime video/mp4 +>8 string NDXS \b, H.264/MPEG-4 AVC (.MP4) Nero Standard Profile +!:mime video/mp4 +>8 string odcf \b, OMA DCF DRM Format 2.0 (OMA-TS-DRM-DCF-V2_0-20060303-A) +>8 string opf2 \b, OMA PDCF DRM Format 2.1 (OMA-TS-DRM-DCF-V2_1-20070724-C) +>8 string opx2 \b, OMA PDCF DRM + XBS ext (OMA-TS-DRM_XBS-V1_0-20070529-C) +>8 string pana \b, Panasonic Digital Camera +>8 string qt \b, Apple QuickTime (.MOV/QT) !:mime video/quicktime +>8 string ROSS \b, Ross Video +>8 string sdv \b, SD Memory Card Video +>8 string ssc1 \b, Samsung stereo, single stream (patent pending) +>8 string ssc2 \b, Samsung stereo, dual stream (patent pending) # MPEG sequences # Scans for all common MPEG header start codes 0 belong 0x00000001 >4 byte&0x1F 0x07 JVT NAL sequence, H.264 video >>5 byte 66 \b, baseline >>5 byte 77 \b, main >>5 byte 88 \b, extended >>7 byte x \b @ L %u 0 belong&0xFFFFFF00 0x00000100 >3 byte 0xBA MPEG sequence !:mime video/mpeg >>4 byte &0x40 \b, v2, program multiplex >>4 byte ^0x40 \b, v1, system multiplex >3 byte 0xBB MPEG sequence, v1/2, multiplex (missing pack header) >3 byte&0x1F 0x07 MPEG sequence, H.264 video >>4 byte 66 \b, baseline >>4 byte 77 \b, main >>4 byte 88 \b, extended >>6 byte x \b @ L %u # GRR too general as it catches also FoxPro Memo example NG.FPT >3 byte 0xB0 MPEG sequence, v4 # TODO: maybe this extra line exclude FoxPro Memo example NG.FPT starting with 000001b0 00000100 00000000 #>>4 byte !0 MPEG sequence, v4 !:mime video/mpeg4-generic >>5 belong 0x000001B5 >>>9 byte &0x80 >>>>10 byte&0xF0 16 \b, video >>>>10 byte&0xF0 32 \b, still texture >>>>10 byte&0xF0 48 \b, mesh >>>>10 byte&0xF0 64 \b, face >>>9 byte&0xF8 8 \b, video >>>9 byte&0xF8 16 \b, still texture >>>9 byte&0xF8 24 \b, mesh >>>9 byte&0xF8 32 \b, face >>4 byte 1 \b, simple @ L1 >>4 byte 2 \b, simple @ L2 >>4 byte 3 \b, simple @ L3 >>4 byte 4 \b, simple @ L0 >>4 byte 17 \b, simple scalable @ L1 >>4 byte 18 \b, simple scalable @ L2 >>4 byte 33 \b, core @ L1 >>4 byte 34 \b, core @ L2 >>4 byte 50 \b, main @ L2 >>4 byte 51 \b, main @ L3 >>4 byte 53 \b, main @ L4 >>4 byte 66 \b, n-bit @ L2 >>4 byte 81 \b, scalable texture @ L1 >>4 byte 97 \b, simple face animation @ L1 >>4 byte 98 \b, simple face animation @ L2 >>4 byte 99 \b, simple face basic animation @ L1 >>4 byte 100 \b, simple face basic animation @ L2 >>4 byte 113 \b, basic animation text @ L1 >>4 byte 114 \b, basic animation text @ L2 >>4 byte 129 \b, hybrid @ L1 >>4 byte 130 \b, hybrid @ L2 >>4 byte 145 \b, advanced RT simple @ L! >>4 byte 146 \b, advanced RT simple @ L2 >>4 byte 147 \b, advanced RT simple @ L3 >>4 byte 148 \b, advanced RT simple @ L4 >>4 byte 161 \b, core scalable @ L1 >>4 byte 162 \b, core scalable @ L2 >>4 byte 163 \b, core scalable @ L3 >>4 byte 177 \b, advanced coding efficiency @ L1 >>4 byte 178 \b, advanced coding efficiency @ L2 >>4 byte 179 \b, advanced coding efficiency @ L3 >>4 byte 180 \b, advanced coding efficiency @ L4 >>4 byte 193 \b, advanced core @ L1 >>4 byte 194 \b, advanced core @ L2 >>4 byte 209 \b, advanced scalable texture @ L1 >>4 byte 210 \b, advanced scalable texture @ L2 >>4 byte 211 \b, advanced scalable texture @ L3 >>4 byte 225 \b, simple studio @ L1 >>4 byte 226 \b, simple studio @ L2 >>4 byte 227 \b, simple studio @ L3 >>4 byte 228 \b, simple studio @ L4 >>4 byte 229 \b, core studio @ L1 >>4 byte 230 \b, core studio @ L2 >>4 byte 231 \b, core studio @ L3 >>4 byte 232 \b, core studio @ L4 >>4 byte 240 \b, advanced simple @ L0 >>4 byte 241 \b, advanced simple @ L1 >>4 byte 242 \b, advanced simple @ L2 >>4 byte 243 \b, advanced simple @ L3 >>4 byte 244 \b, advanced simple @ L4 >>4 byte 245 \b, advanced simple @ L5 >>4 byte 247 \b, advanced simple @ L3b >>4 byte 248 \b, FGS @ L0 >>4 byte 249 \b, FGS @ L1 >>4 byte 250 \b, FGS @ L2 >>4 byte 251 \b, FGS @ L3 >>4 byte 252 \b, FGS @ L4 >>4 byte 253 \b, FGS @ L5 >3 byte 0xB5 MPEG sequence, v4 !:mime video/mpeg4-generic >>4 byte &0x80 >>>5 byte&0xF0 16 \b, video (missing profile header) >>>5 byte&0xF0 32 \b, still texture (missing profile header) >>>5 byte&0xF0 48 \b, mesh (missing profile header) >>>5 byte&0xF0 64 \b, face (missing profile header) >>4 byte&0xF8 8 \b, video (missing profile header) >>4 byte&0xF8 16 \b, still texture (missing profile header) >>4 byte&0xF8 24 \b, mesh (missing profile header) >>4 byte&0xF8 32 \b, face (missing profile header) >3 byte 0xB3 MPEG sequence !:mime video/mpeg >>12 belong 0x000001B8 \b, v1, progressive Y'CbCr 4:2:0 video >>12 belong 0x000001B2 \b, v1, progressive Y'CbCr 4:2:0 video >>12 belong 0x000001B5 \b, v2, >>>16 byte&0x0F 1 \b HP >>>16 byte&0x0F 2 \b Spt >>>16 byte&0x0F 3 \b SNR >>>16 byte&0x0F 4 \b MP >>>16 byte&0x0F 5 \b SP >>>17 byte&0xF0 64 \b@HL >>>17 byte&0xF0 96 \b@H-14 >>>17 byte&0xF0 128 \b@ML >>>17 byte&0xF0 160 \b@LL >>>17 byte &0x08 \b progressive >>>17 byte ^0x08 \b interlaced >>>17 byte&0x06 2 \b Y'CbCr 4:2:0 video >>>17 byte&0x06 4 \b Y'CbCr 4:2:2 video >>>17 byte&0x06 6 \b Y'CbCr 4:4:4 video >>11 byte &0x02 >>>75 byte &0x01 >>>>140 belong 0x000001B8 \b, v1, progressive Y'CbCr 4:2:0 video >>>>140 belong 0x000001B2 \b, v1, progressive Y'CbCr 4:2:0 video >>>>140 belong 0x000001B5 \b, v2, >>>>>144 byte&0x0F 1 \b HP >>>>>144 byte&0x0F 2 \b Spt >>>>>144 byte&0x0F 3 \b SNR >>>>>144 byte&0x0F 4 \b MP >>>>>144 byte&0x0F 5 \b SP >>>>>145 byte&0xF0 64 \b@HL >>>>>145 byte&0xF0 96 \b@H-14 >>>>>145 byte&0xF0 128 \b@ML >>>>>145 byte&0xF0 160 \b@LL >>>>>145 byte &0x08 \b progressive >>>>>145 byte ^0x08 \b interlaced >>>>>145 byte&0x06 2 \b Y'CbCr 4:2:0 video >>>>>145 byte&0x06 4 \b Y'CbCr 4:2:2 video >>>>>145 byte&0x06 6 \b Y'CbCr 4:4:4 video >>76 belong 0x000001B8 \b, v1, progressive Y'CbCr 4:2:0 video >>76 belong 0x000001B2 \b, v1, progressive Y'CbCr 4:2:0 video >>76 belong 0x000001B5 \b, v2, >>>80 byte&0x0F 1 \b HP >>>80 byte&0x0F 2 \b Spt >>>80 byte&0x0F 3 \b SNR >>>80 byte&0x0F 4 \b MP >>>80 byte&0x0F 5 \b SP >>>81 byte&0xF0 64 \b@HL >>>81 byte&0xF0 96 \b@H-14 >>>81 byte&0xF0 128 \b@ML >>>81 byte&0xF0 160 \b@LL >>>81 byte &0x08 \b progressive >>>81 byte ^0x08 \b interlaced >>>81 byte&0x06 2 \b Y'CbCr 4:2:0 video >>>81 byte&0x06 4 \b Y'CbCr 4:2:2 video >>>81 byte&0x06 6 \b Y'CbCr 4:4:4 video >>4 belong&0xFFFFFF00 0x78043800 \b, HD-TV 1920P >>>7 byte&0xF0 0x10 \b, 16:9 >>4 belong&0xFFFFFF00 0x50002D00 \b, SD-TV 1280I >>>7 byte&0xF0 0x10 \b, 16:9 >>4 belong&0xFFFFFF00 0x30024000 \b, PAL Capture >>>7 byte&0xF0 0x10 \b, 4:3 >>4 beshort&0xFFF0 0x2C00 \b, 4CIF >>>5 beshort&0x0FFF 0x01E0 \b NTSC >>>5 beshort&0x0FFF 0x0240 \b PAL >>>7 byte&0xF0 0x20 \b, 4:3 >>>7 byte&0xF0 0x30 \b, 16:9 >>>7 byte&0xF0 0x40 \b, 11:5 >>>7 byte&0xF0 0x80 \b, PAL 4:3 >>>7 byte&0xF0 0xC0 \b, NTSC 4:3 >>4 belong&0xFFFFFF00 0x2801E000 \b, LD-TV 640P >>>7 byte&0xF0 0x10 \b, 4:3 >>4 belong&0xFFFFFF00 0x1400F000 \b, 320x240 >>>7 byte&0xF0 0x10 \b, 4:3 >>4 belong&0xFFFFFF00 0x0F00A000 \b, 240x160 >>>7 byte&0xF0 0x10 \b, 4:3 >>4 belong&0xFFFFFF00 0x0A007800 \b, 160x120 >>>7 byte&0xF0 0x10 \b, 4:3 >>4 beshort&0xFFF0 0x1600 \b, CIF >>>5 beshort&0x0FFF 0x00F0 \b NTSC >>>5 beshort&0x0FFF 0x0120 \b PAL >>>7 byte&0xF0 0x20 \b, 4:3 >>>7 byte&0xF0 0x30 \b, 16:9 >>>7 byte&0xF0 0x40 \b, 11:5 >>>7 byte&0xF0 0x80 \b, PAL 4:3 >>>7 byte&0xF0 0xC0 \b, NTSC 4:3 >>>5 beshort&0x0FFF 0x0240 \b PAL 625 >>>>7 byte&0xF0 0x20 \b, 4:3 >>>>7 byte&0xF0 0x30 \b, 16:9 >>>>7 byte&0xF0 0x40 \b, 11:5 >>4 beshort&0xFFF0 0x2D00 \b, CCIR/ITU >>>5 beshort&0x0FFF 0x01E0 \b NTSC 525 >>>5 beshort&0x0FFF 0x0240 \b PAL 625 >>>7 byte&0xF0 0x20 \b, 4:3 >>>7 byte&0xF0 0x30 \b, 16:9 >>>7 byte&0xF0 0x40 \b, 11:5 >>4 beshort&0xFFF0 0x1E00 \b, SVCD >>>5 beshort&0x0FFF 0x01E0 \b NTSC 525 >>>5 beshort&0x0FFF 0x0240 \b PAL 625 >>>7 byte&0xF0 0x20 \b, 4:3 >>>7 byte&0xF0 0x30 \b, 16:9 >>>7 byte&0xF0 0x40 \b, 11:5 >>7 byte&0x0F 1 \b, 23.976 fps >>7 byte&0x0F 2 \b, 24 fps >>7 byte&0x0F 3 \b, 25 fps >>7 byte&0x0F 4 \b, 29.97 fps >>7 byte&0x0F 5 \b, 30 fps >>7 byte&0x0F 6 \b, 50 fps >>7 byte&0x0F 7 \b, 59.94 fps >>7 byte&0x0F 8 \b, 60 fps >>11 byte &0x04 \b, Constrained # MPEG ADTS Audio (*.mpx/mxa/aac) # from dreesen@math.fu-berlin.de # modified to fully support MPEG ADTS # MP3, M1A # modified by Joerg Jenderek # GRR the original test are too common for many DOS files # so don't accept as MP3 until we've tested the rate 0 beshort&0xFFFE 0xFFFA # rates >2 byte&0xF0 0x10 MPEG ADTS, layer III, v1, 32 kbps !:mime audio/mpeg >2 byte&0xF0 0x20 MPEG ADTS, layer III, v1, 40 kbps !:mime audio/mpeg >2 byte&0xF0 0x30 MPEG ADTS, layer III, v1, 48 kbps !:mime audio/mpeg >2 byte&0xF0 0x40 MPEG ADTS, layer III, v1, 56 kbps !:mime audio/mpeg >2 byte&0xF0 0x50 MPEG ADTS, layer III, v1, 64 kbps !:mime audio/mpeg >2 byte&0xF0 0x60 MPEG ADTS, layer III, v1, 80 kbps !:mime audio/mpeg >2 byte&0xF0 0x70 MPEG ADTS, layer III, v1, 96 kbps !:mime audio/mpeg >2 byte&0xF0 0x80 MPEG ADTS, layer III, v1, 112 kbps !:mime audio/mpeg >2 byte&0xF0 0x90 MPEG ADTS, layer III, v1, 128 kbps !:mime audio/mpeg >2 byte&0xF0 0xA0 MPEG ADTS, layer III, v1, 160 kbps !:mime audio/mpeg >2 byte&0xF0 0xB0 MPEG ADTS, layer III, v1, 192 kbps !:mime audio/mpeg >2 byte&0xF0 0xC0 MPEG ADTS, layer III, v1, 224 kbps !:mime audio/mpeg >2 byte&0xF0 0xD0 MPEG ADTS, layer III, v1, 256 kbps !:mime audio/mpeg >2 byte&0xF0 0xE0 MPEG ADTS, layer III, v1, 320 kbps !:mime audio/mpeg # timing >2 byte&0x0C 0x00 \b, 44.1 kHz >2 byte&0x0C 0x04 \b, 48 kHz >2 byte&0x0C 0x08 \b, 32 kHz # channels/options >3 byte&0xC0 0x00 \b, Stereo >3 byte&0xC0 0x40 \b, JntStereo >3 byte&0xC0 0x80 \b, 2x Monaural >3 byte&0xC0 0xC0 \b, Monaural #>1 byte ^0x01 \b, Data Verify #>2 byte &0x02 \b, Packet Pad #>2 byte &0x01 \b, Custom Flag #>3 byte &0x08 \b, Copyrighted #>3 byte &0x04 \b, Original Source #>3 byte&0x03 1 \b, NR: 50/15 ms #>3 byte&0x03 3 \b, NR: CCIT J.17 # MP2, M1A 0 beshort&0xFFFE 0xFFFC MPEG ADTS, layer II, v1 !:mime audio/mpeg # rates >2 byte&0xF0 0x10 \b, 32 kbps >2 byte&0xF0 0x20 \b, 48 kbps >2 byte&0xF0 0x30 \b, 56 kbps >2 byte&0xF0 0x40 \b, 64 kbps >2 byte&0xF0 0x50 \b, 80 kbps >2 byte&0xF0 0x60 \b, 96 kbps >2 byte&0xF0 0x70 \b, 112 kbps >2 byte&0xF0 0x80 \b, 128 kbps >2 byte&0xF0 0x90 \b, 160 kbps >2 byte&0xF0 0xA0 \b, 192 kbps >2 byte&0xF0 0xB0 \b, 224 kbps >2 byte&0xF0 0xC0 \b, 256 kbps >2 byte&0xF0 0xD0 \b, 320 kbps >2 byte&0xF0 0xE0 \b, 384 kbps # timing >2 byte&0x0C 0x00 \b, 44.1 kHz >2 byte&0x0C 0x04 \b, 48 kHz >2 byte&0x0C 0x08 \b, 32 kHz # channels/options >3 byte&0xC0 0x00 \b, Stereo >3 byte&0xC0 0x40 \b, JntStereo >3 byte&0xC0 0x80 \b, 2x Monaural >3 byte&0xC0 0xC0 \b, Monaural #>1 byte ^0x01 \b, Data Verify #>2 byte &0x02 \b, Packet Pad #>2 byte &0x01 \b, Custom Flag #>3 byte &0x08 \b, Copyrighted #>3 byte &0x04 \b, Original Source #>3 byte&0x03 1 \b, NR: 50/15 ms #>3 byte&0x03 3 \b, NR: CCIT J.17 # MPA, M1A # updated by Joerg Jenderek # GRR the original test are too common for many DOS files, so test 32 <= kbits <= 448 # GRR this test is still too general as it catches a BOM of UTF-16 files (0xFFFE) # FIXME: Almost all little endian UTF-16 text with BOM are clobbered by these entries #0 beshort&0xFFFE 0xFFFE #>2 ubyte&0xF0 >0x0F #>>2 ubyte&0xF0 <0xE1 MPEG ADTS, layer I, v1 ## rate #>>>2 byte&0xF0 0x10 \b, 32 kbps #>>>2 byte&0xF0 0x20 \b, 64 kbps #>>>2 byte&0xF0 0x30 \b, 96 kbps #>>>2 byte&0xF0 0x40 \b, 128 kbps #>>>2 byte&0xF0 0x50 \b, 160 kbps #>>>2 byte&0xF0 0x60 \b, 192 kbps #>>>2 byte&0xF0 0x70 \b, 224 kbps #>>>2 byte&0xF0 0x80 \b, 256 kbps #>>>2 byte&0xF0 0x90 \b, 288 kbps #>>>2 byte&0xF0 0xA0 \b, 320 kbps #>>>2 byte&0xF0 0xB0 \b, 352 kbps #>>>2 byte&0xF0 0xC0 \b, 384 kbps #>>>2 byte&0xF0 0xD0 \b, 416 kbps #>>>2 byte&0xF0 0xE0 \b, 448 kbps ## timing #>>>2 byte&0x0C 0x00 \b, 44.1 kHz #>>>2 byte&0x0C 0x04 \b, 48 kHz #>>>2 byte&0x0C 0x08 \b, 32 kHz ## channels/options #>>>3 byte&0xC0 0x00 \b, Stereo #>>>3 byte&0xC0 0x40 \b, JntStereo #>>>3 byte&0xC0 0x80 \b, 2x Monaural #>>>3 byte&0xC0 0xC0 \b, Monaural ##>1 byte ^0x01 \b, Data Verify ##>2 byte &0x02 \b, Packet Pad ##>2 byte &0x01 \b, Custom Flag ##>3 byte &0x08 \b, Copyrighted ##>3 byte &0x04 \b, Original Source ##>3 byte&0x03 1 \b, NR: 50/15 ms ##>3 byte&0x03 3 \b, NR: CCIT J.17 # MP3, M2A 0 beshort&0xFFFE 0xFFF2 MPEG ADTS, layer III, v2 !:mime audio/mpeg # rate >2 byte&0xF0 0x10 \b, 8 kbps >2 byte&0xF0 0x20 \b, 16 kbps >2 byte&0xF0 0x30 \b, 24 kbps >2 byte&0xF0 0x40 \b, 32 kbps >2 byte&0xF0 0x50 \b, 40 kbps >2 byte&0xF0 0x60 \b, 48 kbps >2 byte&0xF0 0x70 \b, 56 kbps >2 byte&0xF0 0x80 \b, 64 kbps >2 byte&0xF0 0x90 \b, 80 kbps >2 byte&0xF0 0xA0 \b, 96 kbps >2 byte&0xF0 0xB0 \b, 112 kbps >2 byte&0xF0 0xC0 \b, 128 kbps >2 byte&0xF0 0xD0 \b, 144 kbps >2 byte&0xF0 0xE0 \b, 160 kbps # timing >2 byte&0x0C 0x00 \b, 22.05 kHz >2 byte&0x0C 0x04 \b, 24 kHz >2 byte&0x0C 0x08 \b, 16 kHz # channels/options >3 byte&0xC0 0x00 \b, Stereo >3 byte&0xC0 0x40 \b, JntStereo >3 byte&0xC0 0x80 \b, 2x Monaural >3 byte&0xC0 0xC0 \b, Monaural #>1 byte ^0x01 \b, Data Verify #>2 byte &0x02 \b, Packet Pad #>2 byte &0x01 \b, Custom Flag #>3 byte &0x08 \b, Copyrighted #>3 byte &0x04 \b, Original Source #>3 byte&0x03 1 \b, NR: 50/15 ms #>3 byte&0x03 3 \b, NR: CCIT J.17 # MP2, M2A 0 beshort&0xFFFE 0xFFF4 MPEG ADTS, layer II, v2 !:mime audio/mpeg # rate >2 byte&0xF0 0x10 \b, 8 kbps >2 byte&0xF0 0x20 \b, 16 kbps >2 byte&0xF0 0x30 \b, 24 kbps >2 byte&0xF0 0x40 \b, 32 kbps >2 byte&0xF0 0x50 \b, 40 kbps >2 byte&0xF0 0x60 \b, 48 kbps >2 byte&0xF0 0x70 \b, 56 kbps >2 byte&0xF0 0x80 \b, 64 kbps >2 byte&0xF0 0x90 \b, 80 kbps >2 byte&0xF0 0xA0 \b, 96 kbps >2 byte&0xF0 0xB0 \b, 112 kbps >2 byte&0xF0 0xC0 \b, 128 kbps >2 byte&0xF0 0xD0 \b, 144 kbps >2 byte&0xF0 0xE0 \b, 160 kbps # timing >2 byte&0x0C 0x00 \b, 22.05 kHz >2 byte&0x0C 0x04 \b, 24 kHz >2 byte&0x0C 0x08 \b, 16 kHz # channels/options >3 byte&0xC0 0x00 \b, Stereo >3 byte&0xC0 0x40 \b, JntStereo >3 byte&0xC0 0x80 \b, 2x Monaural >3 byte&0xC0 0xC0 \b, Monaural #>1 byte ^0x01 \b, Data Verify #>2 byte &0x02 \b, Packet Pad #>2 byte &0x01 \b, Custom Flag #>3 byte &0x08 \b, Copyrighted #>3 byte &0x04 \b, Original Source #>3 byte&0x03 1 \b, NR: 50/15 ms #>3 byte&0x03 3 \b, NR: CCIT J.17 # MPA, M2A 0 beshort&0xFFFE 0xFFF6 MPEG ADTS, layer I, v2 !:mime audio/mpeg # rate >2 byte&0xF0 0x10 \b, 32 kbps >2 byte&0xF0 0x20 \b, 48 kbps >2 byte&0xF0 0x30 \b, 56 kbps >2 byte&0xF0 0x40 \b, 64 kbps >2 byte&0xF0 0x50 \b, 80 kbps >2 byte&0xF0 0x60 \b, 96 kbps >2 byte&0xF0 0x70 \b, 112 kbps >2 byte&0xF0 0x80 \b, 128 kbps >2 byte&0xF0 0x90 \b, 144 kbps >2 byte&0xF0 0xA0 \b, 160 kbps >2 byte&0xF0 0xB0 \b, 176 kbps >2 byte&0xF0 0xC0 \b, 192 kbps >2 byte&0xF0 0xD0 \b, 224 kbps >2 byte&0xF0 0xE0 \b, 256 kbps # timing >2 byte&0x0C 0x00 \b, 22.05 kHz >2 byte&0x0C 0x04 \b, 24 kHz >2 byte&0x0C 0x08 \b, 16 kHz # channels/options >3 byte&0xC0 0x00 \b, Stereo >3 byte&0xC0 0x40 \b, JntStereo >3 byte&0xC0 0x80 \b, 2x Monaural >3 byte&0xC0 0xC0 \b, Monaural #>1 byte ^0x01 \b, Data Verify #>2 byte &0x02 \b, Packet Pad #>2 byte &0x01 \b, Custom Flag #>3 byte &0x08 \b, Copyrighted #>3 byte &0x04 \b, Original Source #>3 byte&0x03 1 \b, NR: 50/15 ms #>3 byte&0x03 3 \b, NR: CCIT J.17 # MP3, M25A 0 beshort&0xFFFE 0xFFE2 MPEG ADTS, layer III, v2.5 !:mime audio/mpeg # rate >2 byte&0xF0 0x10 \b, 8 kbps >2 byte&0xF0 0x20 \b, 16 kbps >2 byte&0xF0 0x30 \b, 24 kbps >2 byte&0xF0 0x40 \b, 32 kbps >2 byte&0xF0 0x50 \b, 40 kbps >2 byte&0xF0 0x60 \b, 48 kbps >2 byte&0xF0 0x70 \b, 56 kbps >2 byte&0xF0 0x80 \b, 64 kbps >2 byte&0xF0 0x90 \b, 80 kbps >2 byte&0xF0 0xA0 \b, 96 kbps >2 byte&0xF0 0xB0 \b, 112 kbps >2 byte&0xF0 0xC0 \b, 128 kbps >2 byte&0xF0 0xD0 \b, 144 kbps >2 byte&0xF0 0xE0 \b, 160 kbps # timing >2 byte&0x0C 0x00 \b, 11.025 kHz >2 byte&0x0C 0x04 \b, 12 kHz >2 byte&0x0C 0x08 \b, 8 kHz # channels/options >3 byte&0xC0 0x00 \b, Stereo >3 byte&0xC0 0x40 \b, JntStereo >3 byte&0xC0 0x80 \b, 2x Monaural >3 byte&0xC0 0xC0 \b, Monaural #>1 byte ^0x01 \b, Data Verify #>2 byte &0x02 \b, Packet Pad #>2 byte &0x01 \b, Custom Flag #>3 byte &0x08 \b, Copyrighted #>3 byte &0x04 \b, Original Source #>3 byte&0x03 1 \b, NR: 50/15 ms #>3 byte&0x03 3 \b, NR: CCIT J.17 # AAC (aka MPEG-2 NBC audio) and MPEG-4 audio # Stored AAC streams (instead of the MP4 format) 0 string ADIF MPEG ADIF, AAC !:mime audio/x-hx-aac-adif >4 byte &0x80 >>13 byte &0x10 \b, VBR >>13 byte ^0x10 \b, CBR >>16 byte&0x1E 0x02 \b, single stream >>16 byte&0x1E 0x04 \b, 2 streams >>16 byte&0x1E 0x06 \b, 3 streams >>16 byte &0x08 \b, 4 or more streams >>16 byte &0x10 \b, 8 or more streams >>4 byte &0x80 \b, Copyrighted >>13 byte &0x40 \b, Original Source >>13 byte &0x20 \b, Home Flag >4 byte ^0x80 >>4 byte &0x10 \b, VBR >>4 byte ^0x10 \b, CBR >>7 byte&0x1E 0x02 \b, single stream >>7 byte&0x1E 0x04 \b, 2 streams >>7 byte&0x1E 0x06 \b, 3 streams >>7 byte &0x08 \b, 4 or more streams >>7 byte &0x10 \b, 8 or more streams >>4 byte &0x40 \b, Original Stream(s) >>4 byte &0x20 \b, Home Source # Live or stored single AAC stream (used with MPEG-2 systems) 0 beshort&0xFFF6 0xFFF0 MPEG ADTS, AAC !:mime audio/x-hx-aac-adts >1 byte &0x08 \b, v2 >1 byte ^0x08 \b, v4 # profile >>2 byte &0xC0 \b LTP >2 byte&0xc0 0x00 \b Main >2 byte&0xc0 0x40 \b LC >2 byte&0xc0 0x80 \b SSR # timing >2 byte&0x3c 0x00 \b, 96 kHz >2 byte&0x3c 0x04 \b, 88.2 kHz >2 byte&0x3c 0x08 \b, 64 kHz >2 byte&0x3c 0x0c \b, 48 kHz >2 byte&0x3c 0x10 \b, 44.1 kHz >2 byte&0x3c 0x14 \b, 32 kHz >2 byte&0x3c 0x18 \b, 24 kHz >2 byte&0x3c 0x1c \b, 22.05 kHz >2 byte&0x3c 0x20 \b, 16 kHz >2 byte&0x3c 0x24 \b, 12 kHz >2 byte&0x3c 0x28 \b, 11.025 kHz >2 byte&0x3c 0x2c \b, 8 kHz # channels >2 beshort&0x01c0 0x0040 \b, monaural >2 beshort&0x01c0 0x0080 \b, stereo >2 beshort&0x01c0 0x00c0 \b, stereo + center >2 beshort&0x01c0 0x0100 \b, stereo+center+LFE >2 beshort&0x01c0 0x0140 \b, surround >2 beshort&0x01c0 0x0180 \b, surround + LFE >2 beshort &0x01C0 \b, surround + side #>1 byte ^0x01 \b, Data Verify #>2 byte &0x02 \b, Custom Flag #>3 byte &0x20 \b, Original Stream #>3 byte &0x10 \b, Home Source #>3 byte &0x08 \b, Copyrighted # Live MPEG-4 audio streams (instead of RTP FlexMux) 0 beshort&0xFFE0 0x56E0 MPEG-4 LOAS !:mime audio/x-mp4a-latm #>1 beshort&0x1FFF x \b, %hu byte packet >3 byte&0xE0 0x40 >>4 byte&0x3C 0x04 \b, single stream >>4 byte&0x3C 0x08 \b, 2 streams >>4 byte&0x3C 0x0C \b, 3 streams >>4 byte &0x08 \b, 4 or more streams >>4 byte &0x20 \b, 8 or more streams >3 byte&0xC0 0 >>4 byte&0x78 0x08 \b, single stream >>4 byte&0x78 0x10 \b, 2 streams >>4 byte&0x78 0x18 \b, 3 streams >>4 byte &0x20 \b, 4 or more streams >>4 byte &0x40 \b, 8 or more streams # This magic isn't strong enough (matches plausible ISO-8859-1 text) #0 beshort 0x4DE1 MPEG-4 LO-EP audio stream #!:mime audio/x-mp4a-latm # Summary: FLI animation format # Created by: Daniel Quinlan # Modified by (1): Abel Cheung (avoid over-generic detection) 4 leshort 0xAF11 # standard FLI always has 320x200 resolution and 8 bit color >8 leshort 320 >>10 leshort 200 >>>12 leshort 8 FLI animation, 320x200x8 !:mime video/x-fli >>>>6 leshort x \b, %d frames # frame speed is multiple of 1/70s >>>>16 leshort x \b, %d/70s per frame # Summary: FLC animation format # Created by: Daniel Quinlan # Modified by (1): Abel Cheung (avoid over-generic detection) 4 leshort 0xAF12 # standard FLC always use 8 bit color >12 leshort 8 FLC animation !:mime video/x-flc >>8 leshort x \b, %d >>10 leshort x \bx%dx8 >>6 uleshort x \b, %d frames >>16 uleshort x \b, %dms per frame # DL animation format # XXX - collision with most `mips' magic # # I couldn't find a real magic number for these, however, this # -appears- to work. Note that it might catch other files, too, so be # careful! # # Note that title and author appear in the two 20-byte chunks # at decimal offsets 2 and 22, respectively, but they are XOR'ed with # 255 (hex FF)! The DL format is really bad. # #0 byte 1 DL version 1, medium format (160x100, 4 images/screen) #!:mime video/x-unknown #>42 byte x - %d screens, #>43 byte x %d commands #0 byte 2 DL version 2 #!:mime video/x-unknown #>1 byte 1 - large format (320x200,1 image/screen), #>1 byte 2 - medium format (160x100,4 images/screen), #>1 byte >2 - unknown format, #>42 byte x %d screens, #>43 byte x %d commands # Based on empirical evidence, DL version 3 have several nulls following the # \003. Most of them start with non-null values at hex offset 0x34 or so. #0 string \3\0\0\0\0\0\0\0\0\0\0\0 DL version 3 # iso 13818 transport stream # # from Oskar Schirmer Feb 3, 2001 (ISO 13818.1) # syncbyte 8 bit 0x47 # error_ind 1 bit - # payload_start 1 bit 1 # priority 1 bit - # PID 13 bit 0x0000 # scrambling 2 bit - # adaptfld_ctrl 2 bit 1 or 3 # conti_count 4 bit - 0 belong&0xFF5FFF10 0x47400010 >188 byte 0x47 MPEG transport stream data # DIF digital video file format 0 belong&0xffffff00 0x1f070000 DIF >4 byte &0x01 (DVCPRO) movie file >4 byte ^0x01 (DV) movie file >3 byte &0x80 (PAL) >3 byte ^0x80 (NTSC) # Microsoft Advanced Streaming Format (ASF) 0 belong 0x3026b275 Microsoft ASF !:mime video/x-ms-asf # MNG Video Format, 0 string \x8aMNG MNG video data, !:mime video/x-mng >4 belong !0x0d0a1a0a CORRUPTED, >4 belong 0x0d0a1a0a >>16 belong x %d x >>20 belong x %d # JNG Video Format, 0 string \x8bJNG JNG video data, !:mime video/x-jng >4 belong !0x0d0a1a0a CORRUPTED, >4 belong 0x0d0a1a0a >>16 belong x %d x >>20 belong x %d # Vivo video (Wolfram Kleff) 3 string \x0D\x0AVersion:Vivo Vivo video data # VRML (Virtual Reality Modelling Language) 0 string/w #VRML\ V1.0\ ascii VRML 1 file !:mime model/vrml 0 string/w #VRML\ V2.0\ utf8 ISO/IEC 14772 VRML 97 file !:mime model/vrml # X3D (Extensible 3D) [http://www.web3d.org/specifications/x3d-3.0.dtd] # From Michel Briand 0 string/t \20 search/1000/cw \, 2002-10-03 # 0 string HVQM4 %s >6 string >\0 v%s >0 byte x GameCube movie, >0x34 ubeshort x %d x >0x36 ubeshort x %d, >0x26 ubeshort x %dus, >0x42 ubeshort 0 no audio >0x42 ubeshort >0 %dHz audio # From: "Stefan A. Haubenthal" 0 string DVDVIDEO-VTS Video title set, >0x21 byte x v%x 0 string DVDVIDEO-VMG Video manager, >0x21 byte x v%x # From: Behan Webster # NuppelVideo used by Mythtv (*.nuv) # Note: there are two identical stanzas here differing only in the # initial string matched. It used to be done with a regex, but we're # trying to get rid of those. 0 string NuppelVideo MythTV NuppelVideo >12 string x v%s >20 lelong x (%d >24 lelong x \bx%d), >36 string P \bprogressive, >36 string I \binterlaced, >40 ledouble x \baspect:%.2f, >48 ledouble x \bfps:%.2f 0 string MythTV MythTV NuppelVideo >12 string x v%s >20 lelong x (%d >24 lelong x \bx%d), >36 string P \bprogressive, >36 string I \binterlaced, >40 ledouble x \baspect:%.2f, >48 ledouble x \bfps:%.2f # MPEG file # MPEG sequences # FIXME: This section is from the old magic.mime file and needs # integrating with the rest #0 belong 0x000001BA #>4 byte &0x40 #!:mime video/mp2p #>4 byte ^0x40 #!:mime video/mpeg #0 belong 0x000001BB #!:mime video/mpeg #0 belong 0x000001B0 #!:mime video/mp4v-es #0 belong 0x000001B5 #!:mime video/mp4v-es #0 belong 0x000001B3 #!:mime video/mpv #0 belong&0xFF5FFF10 0x47400010 #!:mime video/mp2t #0 belong 0x00000001 #>4 byte&0x1F 0x07 #!:mime video/h264 # Type: Bink Video # Extension: .bik # URL: http://wiki.multimedia.cx/index.php?title=Bink_Container # From: 2008-07-18 0 string BIK Bink Video >3 regex =[a-z] rev.%s #>4 ulelong x size %d >20 ulelong x \b, %d >24 ulelong x \bx%d >8 ulelong x \b, %d frames >32 ulelong x at rate %d/ >28 ulelong >1 \b%d >40 ulelong =0 \b, no audio >40 ulelong !0 \b, %d audio track >>40 ulelong !1 \bs # follow properties of the first audio track only >>48 uleshort x %dHz >>51 byte&0x20 0 mono >>51 byte&0x20 !0 stereo #>>51 byte&0x10 0 FFT #>>51 byte&0x10 !0 DCT # Type: NUT Container # URL: http://wiki.multimedia.cx/index.php?title=NUT # From: Adam Buchbinder 0 string nut/multimedia\ container\0 NUT multimedia container # Type: Nullsoft Video (NSV) # URL: http://wiki.multimedia.cx/index.php?title=Nullsoft_Video # From: Mike Melanson 0 string NSVf Nullsoft Video # Type: REDCode Video # URL: http://www.red.com/ ; http://wiki.multimedia.cx/index.php?title=REDCode # From: Mike Melanson 4 string RED1 REDCode Video # Type: MTV Multimedia File # URL: http://wiki.multimedia.cx/index.php?title=MTV # From: Mike Melanson 0 string AMVS MTV Multimedia File # Type: ARMovie # URL: http://wiki.multimedia.cx/index.php?title=ARMovie # From: Mike Melanson 0 string ARMovie\012 ARMovie # Type: Interplay MVE Movie # URL: http://wiki.multimedia.cx/index.php?title=Interplay_MVE # From: Mike Melanson 0 string Interplay\040MVE\040File\032 Interplay MVE Movie # Type: Windows Television DVR File # URL: http://wiki.multimedia.cx/index.php?title=WTV # From: Mike Melanson # This takes the form of a Windows-style GUID 0 bequad 0xB7D800203749DA11 >8 bequad 0xA64E0007E95EAD8D Windows Television DVR Media # Type: Sega FILM/CPK Multimedia # URL: http://wiki.multimedia.cx/index.php?title=Sega_FILM # From: Mike Melanson 0 string FILM Sega FILM/CPK Multimedia, >32 belong x %d x >28 belong x %d # Type: Nintendo THP Multimedia # URL: http://wiki.multimedia.cx/index.php?title=THP # From: Mike Melanson 0 string THP\0 Nintendo THP Multimedia # Type: BBC Dirac Video # URL: http://wiki.multimedia.cx/index.php?title=Dirac # From: Mike Melanson 0 string BBCD BBC Dirac Video # Type: RAD Game Tools Smacker Multimedia # URL: http://wiki.multimedia.cx/index.php?title=Smacker # From: Mike Melanson 0 string SMK RAD Game Tools Smacker Multimedia >3 byte x version %c, >4 lelong x %d x >8 lelong x %d, >12 lelong x %d frames Index: releng/10.1/contrib/file/magic/Magdir/archive =================================================================== --- releng/10.1/contrib/file/magic/Magdir/archive (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/archive (revision 284193) @@ -1,987 +1,956 @@ #------------------------------------------------------------------------------ -# $File: archive,v 1.87 2014/06/03 19:15:58 christos Exp $ +# $File: archive,v 1.88 2014/08/16 10:42:17 christos Exp $ # archive: file(1) magic for archive formats (see also "msdos" for self- # extracting compressed archives) # # cpio, ar, arc, arj, hpack, lha/lharc, rar, squish, uc2, zip, zoo, etc. # pre-POSIX "tar" archives are handled in the C code. # POSIX tar archives 257 string ustar\0 POSIX tar archive !:mime application/x-tar # encoding: posix 257 string ustar\040\040\0 GNU tar archive !:mime application/x-tar # encoding: gnu # Incremental snapshot gnu-tar format from: # http://www.gnu.org/software/tar/manual/html_node/Snapshot-Files.html 0 string GNU\ tar- GNU tar incremental snapshot data >&0 regex [0-9]\.[0-9]+-[0-9]+ version %s # cpio archives # # Yes, the top two "cpio archive" formats *are* supposed to just be "short". # The idea is to indicate archives produced on machines with the same # byte order as the machine running "file" with "cpio archive", and # to indicate archives produced on machines with the opposite byte order # from the machine running "file" with "byte-swapped cpio archive". # # The SVR4 "cpio(4)" hints that there are additional formats, but they # are defined as "short"s; I think all the new formats are # character-header formats and thus are strings, not numbers. 0 short 070707 cpio archive !:mime application/x-cpio 0 short 0143561 byte-swapped cpio archive !:mime application/x-cpio # encoding: swapped 0 string 070707 ASCII cpio archive (pre-SVR4 or odc) 0 string 070701 ASCII cpio archive (SVR4 with no CRC) 0 string 070702 ASCII cpio archive (SVR4 with CRC) # # Various archive formats used by various versions of the "ar" # command. # # # Original UNIX archive formats. # They were written with binary values in host byte order, and # the magic number was a host "int", which might have been 16 bits # or 32 bits. We don't say "PDP-11" or "VAX", as there might have # been ports to little-endian 16-bit-int or 32-bit-int platforms # (x86?) using some of those formats; if none existed, feel free # to use "PDP-11" for little-endian 16-bit and "VAX" for little-endian # 32-bit. There might have been big-endian ports of that sort as # well. # 0 leshort 0177555 very old 16-bit-int little-endian archive 0 beshort 0177555 very old 16-bit-int big-endian archive 0 lelong 0177555 very old 32-bit-int little-endian archive 0 belong 0177555 very old 32-bit-int big-endian archive 0 leshort 0177545 old 16-bit-int little-endian archive >2 string __.SYMDEF random library 0 beshort 0177545 old 16-bit-int big-endian archive >2 string __.SYMDEF random library 0 lelong 0177545 old 32-bit-int little-endian archive >4 string __.SYMDEF random library 0 belong 0177545 old 32-bit-int big-endian archive >4 string __.SYMDEF random library # # From "pdp" (but why a 4-byte quantity?) # 0 lelong 0x39bed PDP-11 old archive 0 lelong 0x39bee PDP-11 4.0 archive # # XXX - what flavor of APL used this, and was it a variant of # some ar archive format? It's similar to, but not the same # as, the APL workspace magic numbers in pdp. # 0 long 0100554 apl workspace # # System V Release 1 portable(?) archive format. # 0 string = System V Release 1 ar archive !:mime application/x-archive # # Debian package; it's in the portable archive format, and needs to go # before the entry for regular portable archives, as it's recognized as # a portable archive whose first member has a name beginning with # "debian". # 0 string =!\ndebian >8 string debian-split part of multipart Debian package !:mime application/vnd.debian.binary-package >8 string debian-binary Debian binary package !:mime application/vnd.debian.binary-package >8 string !debian >68 string >\0 (format %s) # These next two lines do not work, because a bzip2 Debian archive # still uses gzip for the control.tar (first in the archive). Only # data.tar varies, and the location of its filename varies too. # file/libmagic does not current have support for ascii-string based # (offsets) as of 2005-09-15. #>81 string bz2 \b, uses bzip2 compression #>84 string gz \b, uses gzip compression #>136 ledate x created: %s # # MIPS archive; they're in the portable archive format, and need to go # before the entry for regular portable archives, as it's recognized as # a portable archive whose first member has a name beginning with # "__________E". # 0 string =!\n__________E MIPS archive !:mime application/x-archive >20 string U with MIPS Ucode members >21 string L with MIPSEL members >21 string B with MIPSEB members >19 string L and an EL hash table >19 string B and an EB hash table >22 string X -- out of date 0 search/1 -h- Software Tools format archive text # # BSD/SVR2-and-later portable archive formats. # 0 string =! current ar archive !:mime application/x-archive >8 string __.SYMDEF random library >68 string __.SYMDEF\ SORTED random library # # "Thin" archive, as can be produced by GNU ar. # 0 string =!\n thin archive with >68 belong 0 no symbol entries >68 belong 1 %d symbol entry >68 belong >1 %d symbol entries # ARC archiver, from Daniel Quinlan (quinlan@yggdrasil.com) # # The first byte is the magic (0x1a), byte 2 is the compression type for # the first file (0x01 through 0x09), and bytes 3 to 15 are the MS-DOS # filename of the first file (null terminated). Since some types collide # we only test some types on basis of frequency: 0x08 (83%), 0x09 (5%), # 0x02 (5%), 0x03 (3%), 0x04 (2%), 0x06 (2%). 0x01 collides with terminfo. 0 lelong&0x8080ffff 0x0000081a ARC archive data, dynamic LZW !:mime application/x-arc 0 lelong&0x8080ffff 0x0000091a ARC archive data, squashed !:mime application/x-arc 0 lelong&0x8080ffff 0x0000021a ARC archive data, uncompressed !:mime application/x-arc 0 lelong&0x8080ffff 0x0000031a ARC archive data, packed !:mime application/x-arc 0 lelong&0x8080ffff 0x0000041a ARC archive data, squeezed !:mime application/x-arc 0 lelong&0x8080ffff 0x0000061a ARC archive data, crunched !:mime application/x-arc # [JW] stuff taken from idarc, obviously ARC successors: 0 lelong&0x8080ffff 0x00000a1a PAK archive data !:mime application/x-arc 0 lelong&0x8080ffff 0x0000141a ARC+ archive data !:mime application/x-arc 0 lelong&0x8080ffff 0x0000481a HYP archive data !:mime application/x-arc # Acorn archive formats (Disaster prone simpleton, m91dps@ecs.ox.ac.uk) # I can't create either SPARK or ArcFS archives so I have not tested this stuff # [GRR: the original entries collide with ARC, above; replaced with combined # version (not tested)] #0 byte 0x1a RISC OS archive (spark format) 0 string \032archive RISC OS archive (ArcFS format) 0 string Archive\000 RISC OS archive (ArcFS format) # All these were taken from idarc, many could not be verified. Unfortunately, # there were many low-quality sigs, i.e. easy to trigger false positives. # Please notify me of any real-world fishy/ambiguous signatures and I'll try # to get my hands on the actual archiver and see if I find something better. [JW] # probably many can be enhanced by finding some 0-byte or control char near the start # idarc calls this Crush/Uncompressed... *shrug* 0 string CRUSH Crush archive data # Squeeze It (.sqz) 0 string HLSQZ Squeeze It archive data # SQWEZ 0 string SQWEZ SQWEZ archive data # HPack (.hpk) 0 string HPAK HPack archive data # HAP 0 string \x91\x33HF HAP archive data # MD/MDCD 0 string MDmd MDCD archive data # LIM 0 string LIM\x1a LIM archive data # SAR 3 string LH5 SAR archive data # BSArc/BS2 0 string \212\3SB\020\0 BSArc/BS2 archive data # Bethesda Softworks Archive (Oblivion) 0 string BSA\0 BSArc archive data >4 lelong x version %d # MAR 2 string =-ah MAR archive data # ACB #0 belong&0x00f800ff 0x00800000 ACB archive data # CPZ # TODO, this is what idarc says: 0 string \0\0\0 CPZ archive data # JRC 0 string JRchive JRC archive data # Quantum 0 string DS\0 Quantum archive data # ReSOF 0 string PK\3\6 ReSOF archive data # QuArk 0 string 7\4 QuArk archive data # YAC 14 string YC YAC archive data # X1 0 string X1 X1 archive data 0 string XhDr X1 archive data # CDC Codec (.dqt) 0 belong&0xffffe000 0x76ff2000 CDC Codec archive data # AMGC 0 string \xad6" AMGC archive data # NuLIB 0 string N\xc3\xb5F\xc3\xa9lx\xc3\xa5 NuLIB archive data # PakLeo 0 string LEOLZW PAKLeo archive data # ChArc 0 string SChF ChArc archive data # PSA 0 string PSA PSA archive data # CrossePAC 0 string DSIGDCC CrossePAC archive data # Freeze 0 string \x1f\x9f\x4a\x10\x0a Freeze archive data # KBoom 0 string \xc2\xa8MP\xc2\xa8 KBoom archive data # NSQ, must go after CDC Codec 0 string \x76\xff NSQ archive data # DPA 0 string Dirk\ Paehl DPA archive data # BA # TODO: idarc says "bytes 0-2 == bytes 3-5" # TTComp 0 string \0\6 TTComp archive data # ESP, could this conflict with Easy Software Products' (e.g.ESP ghostscript) documentation? 0 string ESP ESP archive data # ZPack 0 string \1ZPK\1 ZPack archive data # Sky 0 string \xbc\x40 Sky archive data # UFA 0 string UFA UFA archive data # Dry 0 string =-H2O DRY archive data # FoxSQZ 0 string FOXSQZ FoxSQZ archive data # AR7 0 string ,AR7 AR7 archive data # PPMZ 0 string PPMZ PPMZ archive data # MS Compress 4 string \x88\xf0\x27 MS Compress archive data # updated by Joerg Jenderek >9 string \0 >>0 string KWAJ >>>7 string \321\003 MS Compress archive data >>>>14 ulong >0 \b, original size: %d bytes >>>>18 ubyte >0x65 >>>>>18 string x \b, was %.8s >>>>>(10.b-4) string x \b.%.3s # MP3 (archiver, not lossy audio compression) 0 string MP3\x1a MP3-Archiver archive data # ZET 0 string OZ\xc3\x9d ZET archive data # TSComp 0 string \x65\x5d\x13\x8c\x08\x01\x03\x00 TSComp archive data # ARQ 0 string gW\4\1 ARQ archive data # Squash 3 string OctSqu Squash archive data # Terse 0 string \5\1\1\0 Terse archive data # PUCrunch 0 string \x01\x08\x0b\x08\xef\x00\x9e\x32\x30\x36\x31 PUCrunch archive data # UHarc 0 string UHA UHarc archive data # ABComp 0 string \2AB ABComp archive data 0 string \3AB2 ABComp archive data # CMP 0 string CO\0 CMP archive data # Splint 0 string \x93\xb9\x06 Splint archive data # InstallShield 0 string \x13\x5d\x65\x8c InstallShield Z archive Data # Gather 1 string GTH Gather archive data # BOA 0 string BOA BOA archive data # RAX 0 string ULEB\xa RAX archive data # Xtreme 0 string ULEB\0 Xtreme archive data # Pack Magic 0 string @\xc3\xa2\1\0 Pack Magic archive data # BTS 0 belong&0xfeffffff 0x1a034465 BTS archive data # ELI 5750 0 string Ora\ ELI 5750 archive data # QFC 0 string \x1aFC\x1a QFC archive data 0 string \x1aQF\x1a QFC archive data # PRO-PACK 0 string RNC PRO-PACK archive data # 777 0 string 777 777 archive data # LZS221 0 string sTaC LZS221 archive data # HPA 0 string HPA HPA archive data # Arhangel 0 string LG Arhangel archive data # EXP1, uses bzip2 0 string 0123456789012345BZh EXP1 archive data # IMP 0 string IMP\xa IMP archive data # NRV 0 string \x00\x9E\x6E\x72\x76\xFF NRV archive data # Squish 0 string \x73\xb2\x90\xf4 Squish archive data # Par 0 string PHILIPP Par archive data 0 string PAR Par archive data # HIT 0 string UB HIT archive data # SBX 0 belong&0xfffff000 0x53423000 SBX archive data # NaShrink 0 string NSK NaShrink archive data # SAPCAR 0 string #\ CAR\ archive\ header SAPCAR archive data 0 string CAR\ 2.00RG SAPCAR archive data # Disintegrator 0 string DST Disintegrator archive data # ASD 0 string ASD ASD archive data # InstallShield CAB 0 string ISc( InstallShield CAB # TOP4 0 string T4\x1a TOP4 archive data # BatComp left out: sig looks like COM executable # so TODO: get real 4dos batcomp file and find sig # BlakHole 0 string BH\5\7 BlakHole archive data # BIX 0 string BIX0 BIX archive data # ChiefLZA 0 string ChfLZ ChiefLZA archive data # Blink 0 string Blink Blink archive data # Logitech Compress 0 string \xda\xfa Logitech Compress archive data # ARS-Sfx (FIXME: really a SFX? then goto COM/EXE) 1 string (C)\ STEPANYUK ARS-Sfx archive data # AKT/AKT32 0 string AKT32 AKT32 archive data 0 string AKT AKT archive data # NPack 0 string MSTSM NPack archive data # PFT 0 string \0\x50\0\x14 PFT archive data # SemOne 0 string SEM SemOne archive data # PPMD 0 string \x8f\xaf\xac\x84 PPMD archive data # FIZ 0 string FIZ FIZ archive data # MSXiE 0 belong&0xfffff0f0 0x4d530000 MSXiE archive data # DeepFreezer 0 belong&0xfffffff0 0x797a3030 DeepFreezer archive data # DC 0 string =2 byte x \b, version %i >3 byte x \b.%i # ZZip archiver (.zz) 0 string ZZ\ \0\0 ZZip archive data 0 string ZZ0 ZZip archive data # PAQ archiver (.paq) 0 string \xaa\x40\x5f\x77\x1f\xe5\x82\x0d PAQ archive data 0 string PAQ PAQ archive data >3 byte&0xf0 0x30 >>3 byte x (v%c) # JAR archiver (.j), this is the successor to ARJ, not Java's JAR (which is essentially ZIP) 0xe string \x1aJar\x1b JAR (ARJ Software, Inc.) archive data 0 string JARCS JAR (ARJ Software, Inc.) archive data # ARJ archiver (jason@jarthur.Claremont.EDU) 0 leshort 0xea60 ARJ archive data !:mime application/x-arj >5 byte x \b, v%d, >8 byte &0x04 multi-volume, >8 byte &0x10 slash-switched, >8 byte &0x20 backup, >34 string x original name: %s, >7 byte 0 os: MS-DOS >7 byte 1 os: PRIMOS >7 byte 2 os: Unix >7 byte 3 os: Amiga >7 byte 4 os: Macintosh >7 byte 5 os: OS/2 >7 byte 6 os: Apple ][ GS >7 byte 7 os: Atari ST >7 byte 8 os: NeXT >7 byte 9 os: VAX/VMS >3 byte >0 %d] # [JW] idarc says this is also possible 2 leshort 0xea60 ARJ archive data # HA archiver (Greg Roelofs, newt@uchicago.edu) # This is a really bad format. A file containing HAWAII will match this... #0 string HA HA archive data, #>2 leshort =1 1 file, #>2 leshort >1 %hu files, #>4 byte&0x0f =0 first is type CPY #>4 byte&0x0f =1 first is type ASC #>4 byte&0x0f =2 first is type HSC #>4 byte&0x0f =0x0e first is type DIR #>4 byte&0x0f =0x0f first is type SPECIAL # suggestion: at least identify small archives (<1024 files) 0 belong&0xffff00fc 0x48410000 HA archive data >2 leshort =1 1 file, >2 leshort >1 %u files, >4 byte&0x0f =0 first is type CPY >4 byte&0x0f =1 first is type ASC >4 byte&0x0f =2 first is type HSC >4 byte&0x0f =0x0e first is type DIR >4 byte&0x0f =0x0f first is type SPECIAL # HPACK archiver (Peter Gutmann, pgut1@cs.aukuni.ac.nz) 0 string HPAK HPACK archive data # JAM Archive volume format, by Dmitry.Kohmanyuk@UA.net 0 string \351,\001JAM\ JAM archive, >7 string >\0 version %.4s >0x26 byte =0x27 - >>0x2b string >\0 label %.11s, >>0x27 lelong x serial %08x, >>0x36 string >\0 fstype %.8s # LHARC/LHA archiver (Greg Roelofs, newt@uchicago.edu) 2 string -lh0- LHarc 1.x/ARX archive data [lh0] !:mime application/x-lharc 2 string -lh1- LHarc 1.x/ARX archive data [lh1] !:mime application/x-lharc 2 string -lz4- LHarc 1.x archive data [lz4] !:mime application/x-lharc 2 string -lz5- LHarc 1.x archive data [lz5] !:mime application/x-lharc # [never seen any but the last; -lh4- reported in comp.compression:] 2 string -lzs- LHa/LZS archive data [lzs] !:mime application/x-lha 2 string -lh\40- LHa 2.x? archive data [lh ] !:mime application/x-lha 2 string -lhd- LHa 2.x? archive data [lhd] !:mime application/x-lha 2 string -lh2- LHa 2.x? archive data [lh2] !:mime application/x-lha 2 string -lh3- LHa 2.x? archive data [lh3] !:mime application/x-lha 2 string -lh4- LHa (2.x) archive data [lh4] !:mime application/x-lha 2 string -lh5- LHa (2.x) archive data [lh5] !:mime application/x-lha 2 string -lh6- LHa (2.x) archive data [lh6] !:mime application/x-lha 2 string -lh7- LHa (2.x)/LHark archive data [lh7] !:mime application/x-lha >20 byte x - header level %d # taken from idarc [JW] 2 string -lZ PUT archive data 2 string -lz LZS archive data 2 string -sw1- Swag archive data # RAR archiver (Greg Roelofs, newt@uchicago.edu) 0 string Rar! RAR archive data, !:mime application/x-rar >44 byte x v%0x, >10 byte >0 flags: >>10 byte &0x01 Archive volume, >>10 byte &0x02 Commented, >>10 byte &0x04 Locked, >>10 byte &0x08 Solid, >>10 byte &0x20 Authenticated, >35 byte 0 os: MS-DOS >35 byte 1 os: OS/2 >35 byte 2 os: Win32 >35 byte 3 os: Unix # some old version? idarc says: 0 string RE\x7e\x5e RAR archive data # SQUISH archiver (Greg Roelofs, newt@uchicago.edu) 0 string SQSH squished archive data (Acorn RISCOS) # UC2 archiver (Greg Roelofs, newt@uchicago.edu) # [JW] see exe section for self-extracting version 0 string UC2\x1a UC2 archive data # PKZIP multi-volume archive 0 string PK\x07\x08PK\x03\x04 Zip multi-volume archive data, at least PKZIP v2.50 to extract !:mime application/zip # Zip archives (Greg Roelofs, c/o zip-bugs@wkuvx1.wku.edu) 0 string PK\005\006 Zip archive data (empty) 0 string PK\003\004 # Specialised zip formats which start with a member named 'mimetype' # (stored uncompressed, with no 'extra field') containing the file's MIME type. # Check for have 8-byte name, 0-byte extra field, name "mimetype", and # contents starting with "application/": >26 string \x8\0\0\0mimetypeapplication/ # KOffice / OpenOffice & StarOffice / OpenDocument formats # From: Abel Cheung # KOffice (1.2 or above) formats # (mimetype contains "application/vnd.kde.") >>50 string vnd.kde. KOffice (>=1.2) >>>58 string karbon Karbon document >>>58 string kchart KChart document >>>58 string kformula KFormula document >>>58 string kivio Kivio document >>>58 string kontour Kontour document >>>58 string kpresenter KPresenter document >>>58 string kspread KSpread document >>>58 string kword KWord document # OpenOffice formats (for OpenOffice 1.x / StarOffice 6/7) # (mimetype contains "application/vnd.sun.xml.") >>50 string vnd.sun.xml. OpenOffice.org 1.x >>>62 string writer Writer >>>>68 byte !0x2e document >>>>68 string .template template >>>>68 string .global global document >>>62 string calc Calc >>>>66 byte !0x2e spreadsheet >>>>66 string .template template >>>62 string draw Draw >>>>66 byte !0x2e document >>>>66 string .template template >>>62 string impress Impress >>>>69 byte !0x2e presentation >>>>69 string .template template >>>62 string math Math document >>>62 string base Database file # OpenDocument formats (for OpenOffice 2.x / StarOffice >= 8) # http://lists.oasis-open.org/archives/office/200505/msg00006.html # (mimetype contains "application/vnd.oasis.opendocument.") >>50 string vnd.oasis.opendocument. OpenDocument >>>73 string text >>>>77 byte !0x2d Text !:mime application/vnd.oasis.opendocument.text >>>>77 string -template Text Template !:mime application/vnd.oasis.opendocument.text-template >>>>77 string -web HTML Document Template !:mime application/vnd.oasis.opendocument.text-web >>>>77 string -master Master Document !:mime application/vnd.oasis.opendocument.text-master >>>73 string graphics >>>>81 byte !0x2d Drawing !:mime application/vnd.oasis.opendocument.graphics >>>>81 string -template Template !:mime application/vnd.oasis.opendocument.graphics-template >>>73 string presentation >>>>85 byte !0x2d Presentation !:mime application/vnd.oasis.opendocument.presentation >>>>85 string -template Template !:mime application/vnd.oasis.opendocument.presentation-template >>>73 string spreadsheet >>>>84 byte !0x2d Spreadsheet !:mime application/vnd.oasis.opendocument.spreadsheet >>>>84 string -template Template !:mime application/vnd.oasis.opendocument.spreadsheet-template >>>73 string chart >>>>78 byte !0x2d Chart !:mime application/vnd.oasis.opendocument.chart >>>>78 string -template Template !:mime application/vnd.oasis.opendocument.chart-template >>>73 string formula >>>>80 byte !0x2d Formula !:mime application/vnd.oasis.opendocument.formula >>>>80 string -template Template !:mime application/vnd.oasis.opendocument.formula-template >>>73 string database Database !:mime application/vnd.oasis.opendocument.database >>>73 string image >>>>78 byte !0x2d Image !:mime application/vnd.oasis.opendocument.image >>>>78 string -template Template !:mime application/vnd.oasis.opendocument.image-template # EPUB (OEBPS) books using OCF (OEBPS Container Format) # http://www.idpf.org/ocf/ocf1.0/download/ocf10.htm, section 4. # From: Ralf Brown >>50 string epub+zip EPUB document !:mime application/epub+zip # Catch other ZIP-with-mimetype formats # In a ZIP file, the bytes immediately after a member's contents are # always "PK". The 2 regex rules here print the "mimetype" member's # contents up to the first 'P'. Luckily, most MIME types don't contain # any capital 'P's. This is a kludge. # (mimetype contains "application/") >>50 string !epub+zip >>>50 string !vnd.oasis.opendocument. >>>>50 string !vnd.sun.xml. >>>>>50 string !vnd.kde. >>>>>>38 regex [!-OQ-~]+ Zip data (MIME type "%s"?) !:mime application/zip # (mimetype contents other than "application/*") >26 string \x8\0\0\0mimetype >>38 string !application/ >>>38 regex [!-OQ-~]+ Zip data (MIME type "%s"?) !:mime application/zip # Java Jar files >(26.s+30) leshort 0xcafe Java archive data (JAR) !:mime application/java-archive # Generic zip archives (Greg Roelofs, c/o zip-bugs@wkuvx1.wku.edu) # Next line excludes specialized formats: >(26.s+30) leshort !0xcafe >>26 string !\x8\0\0\0mimetype Zip archive data !:mime application/zip >>>4 byte 0x09 \b, at least v0.9 to extract >>>4 byte 0x0a \b, at least v1.0 to extract >>>4 byte 0x0b \b, at least v1.1 to extract >>>4 byte 0x14 \b, at least v2.0 to extract >>>4 byte 0x2d \b, at least v3.0 to extract >>>0x161 string WINZIP \b, WinZIP self-extracting # StarView Metafile # From Pierre Ducroquet 0 string VCLMTF StarView MetaFile >6 beshort x \b, version %d >8 belong x \b, size %d # Zoo archiver 20 lelong 0xfdc4a7dc Zoo archive data !:mime application/x-zoo >4 byte >48 \b, v%c. >>6 byte >47 \b%c >>>7 byte >47 \b%c >32 byte >0 \b, modify: v%d >>33 byte x \b.%d+ >42 lelong 0xfdc4a7dc \b, >>70 byte >0 extract: v%d >>>71 byte x \b.%d+ # Shell archives 10 string #\ This\ is\ a\ shell\ archive shell archive text !:mime application/octet-stream # # LBR. NB: May conflict with the questionable # "binary Computer Graphics Metafile" format. # 0 string \0\ \ \ \ \ \ \ \ \ \ \ \0\0 LBR archive data # # PMA (CP/M derivative of LHA) # 2 string -pm0- PMarc archive data [pm0] 2 string -pm1- PMarc archive data [pm1] 2 string -pm2- PMarc archive data [pm2] 2 string -pms- PMarc SFX archive (CP/M, DOS) 5 string -pc1- PopCom compressed executable (CP/M) # From Rafael Laboissiere # The Project Revision Control System (see # http://prcs.sourceforge.net) generates a packaged project # file which is recognized by the following entry: 0 leshort 0xeb81 PRCS packaged project # Microsoft cabinets # by David Necas (Yeti) #0 string MSCF\0\0\0\0 Microsoft cabinet file data, #>25 byte x v%d #>24 byte x \b.%d # MPi: All CABs have version 1.3, so this is pointless. # Better magic in debian-additions. # GTKtalog catalogs # by David Necas (Yeti) 4 string gtktalog\ GTKtalog catalog data, >13 string 3 version 3 >>14 beshort 0x677a (gzipped) >>14 beshort !0x677a (not gzipped) >13 string >3 version %s ############################################################################ # Parity archive reconstruction file, the 'par' file format now used on Usenet. 0 string PAR\0 PARity archive data >48 leshort =0 - Index file >48 leshort >0 - file number %d # Felix von Leitner 0 string d8:announce BitTorrent file !:mime application/x-bittorrent # Atari MSA archive - Teemu Hukkanen 0 beshort 0x0e0f Atari MSA archive data >2 beshort x \b, %d sectors per track >4 beshort 0 \b, 1 sided >4 beshort 1 \b, 2 sided >6 beshort x \b, starting track: %d >8 beshort x \b, ending track: %d # Alternate ZIP string (amc@arwen.cs.berkeley.edu) 0 string PK00PK\003\004 Zip archive data # ACE archive (from http://www.wotsit.org/download.asp?f=ace) # by Stefan `Sec` Zehl 7 string **ACE** ACE archive data >15 byte >0 version %d >16 byte =0x00 \b, from MS-DOS >16 byte =0x01 \b, from OS/2 >16 byte =0x02 \b, from Win/32 >16 byte =0x03 \b, from Unix >16 byte =0x04 \b, from MacOS >16 byte =0x05 \b, from WinNT >16 byte =0x06 \b, from Primos >16 byte =0x07 \b, from AppleGS >16 byte =0x08 \b, from Atari >16 byte =0x09 \b, from Vax/VMS >16 byte =0x0A \b, from Amiga >16 byte =0x0B \b, from Next >14 byte x \b, version %d to extract >5 leshort &0x0080 \b, multiple volumes, >>17 byte x \b (part %d), >5 leshort &0x0002 \b, contains comment >5 leshort &0x0200 \b, sfx >5 leshort &0x0400 \b, small dictionary >5 leshort &0x0800 \b, multi-volume >5 leshort &0x1000 \b, contains AV-String >>30 string \x16*UNREGISTERED\x20VERSION* (unregistered) >5 leshort &0x2000 \b, with recovery record >5 leshort &0x4000 \b, locked >5 leshort &0x8000 \b, solid # Date in MS-DOS format (whatever that is) #>18 lelong x Created on # sfArk : compression program for Soundfonts (sf2) by Dirk Jagdmann # 0x1A string sfArk sfArk compressed Soundfont >0x15 string 2 >>0x1 string >\0 Version %s >>0x2A string >\0 : %s # DR-DOS 7.03 Packed File *.??_ 0 string Packed\ File\ Personal NetWare Packed File >12 string x \b, was "%.12s" # EET archive # From: Tilman Sauerbeck 0 belong 0x1ee7ff00 EET archive !:mime application/x-eet # rzip archives 0 string RZIP rzip compressed data >4 byte x - version %d >5 byte x \b.%d >6 belong x (%d bytes) # From: "Robert Dale" 0 belong 123 dar archive, >4 belong x label "%.8x >>8 belong x %.8x >>>12 beshort x %.4x" >14 byte 0x54 end slice >14 beshort 0x4e4e multi-part >14 beshort 0x4e53 multi-part, with -S # Symbian installation files # http://www.thouky.co.uk/software/psifs/sis.html # http://developer.symbian.com/main/downloads/papers/SymbianOSv91/softwareinstallsis.pdf 8 lelong 0x10000419 Symbian installation file !:mime application/vnd.symbian.install >4 lelong 0x1000006D (EPOC release 3/4/5) >4 lelong 0x10003A12 (EPOC release 6) 0 lelong 0x10201A7A Symbian installation file (Symbian OS 9.x) !:mime x-epoc/x-sisx-app # From "Nelson A. de Oliveira" 0 string MPQ\032 MoPaQ (MPQ) archive # From: Dirk Jagdmann # xar archive format: http://code.google.com/p/xar/ 0 string xar! xar archive >6 beshort x - version %d # From: "Nelson A. de Oliveira" # .kgb 0 string KGB_arch KGB Archiver file >10 string x with compression level %.1s # xar (eXtensible ARchiver) archive # From: "David Remahl" 0 string xar! xar archive #>4 beshort x header size %d >6 beshort x version %d, #>8 quad x compressed TOC: %d, #>16 quad x uncompressed TOC: %d, >24 belong 0 no checksum >24 belong 1 SHA-1 checksum >24 belong 2 MD5 checksum # Type: Parity Archive # From: Daniel van Eeden 0 string PAR2 Parity Archive Volume Set # Bacula volume format. (Volumes always start with a block header.) # URL: http://bacula.org/3.0.x-manuals/en/developers/developers/Block_Header.html # From: Adam Buchbinder 12 string BB02 Bacula volume >20 bedate x \b, started %s # ePub is XHTML + XML inside a ZIP archive. The first member of the # archive must be an uncompressed file called 'mimetype' with contents # 'application/epub+zip' # From: "Michael Gorny" # ZPAQ: http://mattmahoney.net/dc/zpaq.html 0 string zPQ ZPAQ stream >3 byte x \b, level %d # BBeB ebook, unencrypted (LRF format) # URL: http://www.sven.de/librie/Librie/LrfFormat # From: Adam Buchbinder 0 string L\0R\0F\0\0\0 BBeB ebook data, unencrypted >8 beshort x \b, version %d >36 byte 1 \b, front-to-back >36 byte 16 \b, back-to-front >42 beshort x \b, (%dx, >44 beshort x %d) # Symantec GHOST image by Joerg Jenderek at May 2014 # http://us.norton.com/ghost/ # http://www.garykessler.net/library/file_sigs.html 0 ubelong&0xFFFFf7f0 0xFEEF0100 Norton GHost image # *.GHO >2 ubyte&0x08 0x00 \b, first file # *.GHS or *.[0-9] with cns program option >2 ubyte&0x08 0x08 \b, split file # part of split index interesting for *.ghs >>4 ubyte x id=0x%x # compression tag minus one equals numeric compression command line switch z[1-9] >3 ubyte 0 \b, no compression >3 ubyte 2 \b, fast compression (Z1) >3 ubyte 3 \b, medium compression (Z2) >3 ubyte >3 >>3 ubyte <11 \b, compression (Z%d-1) >2 ubyte&0x08 0x00 # ~ 30 byte password field only for *.gho >>12 ubequad !0 \b, password protected >>44 ubyte !1 # 1~Image All, sector-by-sector only for *.gho >>>10 ubyte 1 \b, sector copy # 1~Image Boot track only for *.gho >>>43 ubyte 1 \b, boot track # 1~Image Disc only for *.gho implies Image Boot track and sector copy >>44 ubyte 1 \b, disc sector copy # optional image description only *.gho >>0xff string >\0 "%-.254s" # look for DOS sector end sequence >0xE08 search/7776 \x55\xAA >>&-512 indirect x \b; contains -# Symantec GHOST image by Joerg Jenderek at May 2014 -# http://us.norton.com/ghost/ -# http://www.garykessler.net/library/file_sigs.html -0 ubelong&0xFFFFf7f0 0xFEEF0100 Norton GHost image -# *.GHO ->2 ubyte&0x08 0x00 \b, first file -# *.GHS or *.[0-9] with cns program option ->2 ubyte&0x08 0x08 \b, split file -# part of split index interesting for *.ghs ->>4 ubyte x id=0x%x -# compression tag minus one equals numeric compression command line switch z[1-9] ->3 ubyte 0 \b, no compression ->3 ubyte 2 \b, fast compression (Z1) ->3 ubyte 3 \b, medium compression (Z2) ->3 ubyte >3 ->>3 ubyte <11 \b, compression (Z%d-1) ->2 ubyte&0x08 0x00 -# ~ 30 byte password field only for *.gho ->>12 ubequad !0 \b, password protected ->>44 ubyte !1 -# 1~Image All, sector-by-sector only for *.gho ->>>10 ubyte 1 \b, sector copy -# 1~Image Boot track only for *.gho ->>>43 ubyte 1 \b, boot track -# 1~Image Disc only for *.gho implies Image Boot track and sector copy ->>44 ubyte 1 \b, disc sector copy -# optional image description only *.gho ->>0xff string >\0 "%-.254s" -# look for DOS sector end sequence ->0xE08 search/7776 \x55\xAA ->>&-512 indirect x \b; contains Index: releng/10.1/contrib/file/magic/Magdir/blender =================================================================== --- releng/10.1/contrib/file/magic/Magdir/blender (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/blender (revision 284193) @@ -1,39 +1,39 @@ #------------------------------------------------------------------------------ -# $File: blender,v 1.5 2009/09/19 16:28:08 christos Exp $ +# $File: blender,v 1.6 2014/08/30 08:34:17 christos Exp $ # blender: file(1) magic for Blender 3D related files # # Native format rule v1.2. For questions use the developers list # http://lists.blender.org/mailman/listinfo/bf-committers # GLOB chunk was moved near start and provides subversion info since 2.42 0 string =BLENDER Blender3D, >7 string =_ saved as 32-bits >>8 string =v little endian >>>9 byte x with version %c. >>>10 byte x \b%c >>>11 byte x \b%c >>>0x40 string =GLOB \b. >>>>0x58 leshort x \b%.4d >>8 string =V big endian >>>9 byte x with version %c. >>>10 byte x \b%c >>>11 byte x \b%c >>>0x40 string =GLOB \b. >>>>0x58 beshort x \b%.4d >7 string =- saved as 64-bits >>8 string =v little endian >>9 byte x with version %c. >>10 byte x \b%c >>11 byte x \b%c >>0x44 string =GLOB \b. >>>0x60 leshort x \b%.4d >>8 string =V big endian >>>9 byte x with version %c. >>>10 byte x \b%c >>>11 byte x \b%c >>>0x44 string =GLOB \b. >>>>0x60 beshort x \b%.4d -# Scripts that run in the embeded Python interpreter +# Scripts that run in the embedded Python interpreter 0 string #!BPY Blender3D BPython script Index: releng/10.1/contrib/file/magic/Magdir/cafebabe =================================================================== --- releng/10.1/contrib/file/magic/Magdir/cafebabe (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/cafebabe (revision 284193) @@ -1,63 +1,63 @@ #------------------------------------------------------------------------------ -# $File: cafebabe,v 1.16 2014/04/30 21:41:02 christos Exp $ +# $File: cafebabe,v 1.17 2015/01/01 17:07:00 christos Exp $ # Cafe Babes unite! # # Since Java bytecode and Mach-O universal binaries have the same magic number, # the test must be performed in the same "magic" sequence to get both right. # The long at offset 4 in a Mach-O universal binary tells the number of # architectures; the short at offset 4 in a Java bytecode file is the JVM minor # version and the short at offset 6 is the JVM major version. Since there are only # only 18 labeled Mach-O architectures at current, and the first released # Java class format was version 43.0, we can safely choose any number # between 18 and 39 to test the number of architectures against # (and use as a hack). Let's not use 18, because the Mach-O people # might add another one or two as time goes by... # ### JAVA START ### 0 belong 0xcafebabe >4 belong >30 compiled Java class data, !:mime application/x-java-applet >>6 beshort x version %d. >>4 beshort x \b%d # Which is which? #>>4 belong 0x032d (Java 1.0) #>>4 belong 0x032d (Java 1.1) >>4 belong 0x002e (Java 1.2) >>4 belong 0x002f (Java 1.3) >>4 belong 0x0030 (Java 1.4) >>4 belong 0x0031 (Java 1.5) >>4 belong 0x0032 (Java 1.6) 0 belong 0xcafed00d JAR compressed with pack200, >5 byte x version %d. >4 byte x \b%d !:mime application/x-java-pack200 0 belong 0xcafed00d JAR compressed with pack200, >5 byte x version %d. >4 byte x \b%d !:mime application/x-java-pack200 ### JAVA END ### ### MACH-O START ### 0 name mach-o \b [ >0 use mach-o-cpu \b ->&(8.L) indirect \b: +>(8.L) indirect \b: >0 belong x \b] 0 belong 0xcafebabe >4 belong 1 Mach-O universal binary with 1 architecture: >>8 use mach-o \b >4 belong >1 >>4 belong <20 Mach-O universal binary with %d architectures: >>>8 use mach-o \b >>>28 use mach-o \b >>4 belong >2 >>>48 use mach-o \b >>4 belong >3 >>>68 use mach-o \b ### MACH-O END ### Index: releng/10.1/contrib/file/magic/Magdir/commands =================================================================== --- releng/10.1/contrib/file/magic/Magdir/commands (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/commands (revision 284193) @@ -1,110 +1,110 @@ #------------------------------------------------------------------------------ -# $File: commands,v 1.50 2014/05/30 16:48:44 christos Exp $ +# $File: commands,v 1.51 2014/09/27 00:12:55 christos Exp $ # commands: file(1) magic for various shells and interpreters # #0 string/w : shell archive or script for antique kernel text 0 string/wt #!\ /bin/sh POSIX shell script text executable !:mime text/x-shellscript 0 string/wb #!\ /bin/sh POSIX shell script executable (binary data) !:mime text/x-shellscript 0 string/wt #!\ /bin/csh C shell script text executable !:mime text/x-shellscript # korn shell magic, sent by George Wu, gwu@clyde.att.com 0 string/wt #!\ /bin/ksh Korn shell script text executable !:mime text/x-shellscript 0 string/wb #!\ /bin/ksh Korn shell script executable (binary data) !:mime text/x-shellscript 0 string/wt #!\ /bin/tcsh Tenex C shell script text executable !:mime text/x-shellscript 0 string/wt #!\ /usr/bin/tcsh Tenex C shell script text executable !:mime text/x-shellscript 0 string/wt #!\ /usr/local/tcsh Tenex C shell script text executable !:mime text/x-shellscript 0 string/wt #!\ /usr/local/bin/tcsh Tenex C shell script text executable !:mime text/x-shellscript # # zsh/ash/ae/nawk/gawk magic from cameron@cs.unsw.oz.au (Cameron Simpson) 0 string/wt #!\ /bin/zsh Paul Falstad's zsh script text executable !:mime text/x-shellscript 0 string/wt #!\ /usr/bin/zsh Paul Falstad's zsh script text executable !:mime text/x-shellscript 0 string/wt #!\ /usr/local/bin/zsh Paul Falstad's zsh script text executable !:mime text/x-shellscript 0 string/wt #!\ /usr/local/bin/ash Neil Brown's ash script text executable !:mime text/x-shellscript 0 string/wt #!\ /usr/local/bin/ae Neil Brown's ae script text executable !:mime text/x-shellscript 0 string/wt #!\ /bin/nawk new awk script text executable !:mime text/x-nawk 0 string/wt #!\ /usr/bin/nawk new awk script text executable !:mime text/x-nawk 0 string/wt #!\ /usr/local/bin/nawk new awk script text executable !:mime text/x-nawk 0 string/wt #!\ /bin/gawk GNU awk script text executable !:mime text/x-gawk 0 string/wt #!\ /usr/bin/gawk GNU awk script text executable !:mime text/x-gawk 0 string/wt #!\ /usr/local/bin/gawk GNU awk script text executable !:mime text/x-gawk # 0 string/wt #!\ /bin/awk awk script text executable !:mime text/x-awk 0 string/wt #!\ /usr/bin/awk awk script text executable !:mime text/x-awk -0 regex/4096 =^\\s{0,100}BEGIN\\s{0,100}[{] awk script text +0 regex/4096 =^\\s{0,100}BEGIN\\s{0,100}[{] awk or perl script text # AT&T Bell Labs' Plan 9 shell 0 string/wt #!\ /bin/rc Plan 9 rc shell script text executable # bash shell magic, from Peter Tobias (tobias@server.et-inf.fho-emden.de) 0 string/wt #!\ /bin/bash Bourne-Again shell script text executable !:mime text/x-shellscript 0 string/wb #!\ /bin/bash Bourne-Again shell script executable (binary data) !:mime text/x-shellscript 0 string/wt #!\ /usr/bin/bash Bourne-Again shell script text executable !:mime text/x-shellscript 0 string/wb #!\ /usr/bin/bash Bourne-Again shell script executable (binary data) !:mime text/x-shellscript 0 string/wt #!\ /usr/local/bash Bourne-Again shell script text executable !:mime text/x-shellscript 0 string/wb #!\ /usr/local/bash Bourne-Again shell script executable (binary data) !:mime text/x-shellscript 0 string/wt #!\ /usr/local/bin/bash Bourne-Again shell script text executable !:mime text/x-shellscript 0 string/wb #!\ /usr/local/bin/bash Bourne-Again shell script executable (binary data) !:mime text/x-shellscript # PHP scripts # Ulf Harnhammar 0 search/1/c = 0 string =24 regex [0-9.]+ \b, version %s !:mime text/x-php 0 string Zend\x00 PHP script Zend Optimizer data 0 string/t $! DCL command file # Type: Pdmenu # URL: http://packages.debian.org/pdmenu # From: Edward Betts 0 string #!/usr/bin/pdmenu Pdmenu configuration file text Index: releng/10.1/contrib/file/magic/Magdir/compress =================================================================== --- releng/10.1/contrib/file/magic/Magdir/compress (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/compress (revision 284193) @@ -1,253 +1,263 @@ #------------------------------------------------------------------------------ -# $File: compress,v 1.58 2014/05/07 19:36:59 christos Exp $ +# $File: compress,v 1.62 2014/09/13 14:27:12 christos Exp $ # compress: file(1) magic for pure-compression formats (no archives) # # compress, gzip, pack, compact, huf, squeeze, crunch, freeze, yabba, etc. # # Formats for various forms of compressed data # Formats for "compress" proper have been moved into "compress.c", # because it tries to uncompress it to figure out what's inside. # standard unix compress 0 string \037\235 compress'd data !:mime application/x-compress !:apple LZIVZIVU >2 byte&0x80 >0 block compressed >2 byte&0x1f x %d bits # gzip (GNU zip, not to be confused with Info-ZIP or PKWARE zip archiver) # Edited by Chris Chittleborough , March 2002 # * Original filename is only at offset 10 if "extra field" absent # * Produce shorter output - notably, only report compression methods # other than 8 ("deflate", the only method defined in RFC 1952). 0 string \037\213 gzip compressed data !:mime application/x-gzip !:strength * 2 >2 byte <8 \b, reserved method >2 byte >8 \b, unknown method >3 byte &0x01 \b, ASCII >3 byte &0x02 \b, has CRC >3 byte &0x04 \b, extra field >3 byte&0xC =0x08 >>10 string x \b, was "%s" >3 byte &0x10 \b, has comment >3 byte &0x20 \b, encrypted >4 ledate >0 \b, last modified: %s >8 byte 2 \b, max compression >8 byte 4 \b, max speed >9 byte =0x00 \b, from FAT filesystem (MS-DOS, OS/2, NT) >9 byte =0x01 \b, from Amiga >9 byte =0x02 \b, from VMS >9 byte =0x03 \b, from Unix >9 byte =0x04 \b, from VM/CMS >9 byte =0x05 \b, from Atari >9 byte =0x06 \b, from HPFS filesystem (OS/2, NT) >9 byte =0x07 \b, from MacOS >9 byte =0x08 \b, from Z-System >9 byte =0x09 \b, from CP/M >9 byte =0x0A \b, from TOPS/20 >9 byte =0x0B \b, from NTFS filesystem (NT) >9 byte =0x0C \b, from QDOS >9 byte =0x0D \b, from Acorn RISCOS # packed data, Huffman (minimum redundancy) codes on a byte-by-byte basis 0 string \037\036 packed data !:mime application/octet-stream >2 belong >1 \b, %d characters originally >2 belong =1 \b, %d character originally # # This magic number is byte-order-independent. 0 short 0x1f1f old packed data !:mime application/octet-stream # XXX - why *two* entries for "compacted data", one of which is # byte-order independent, and one of which is byte-order dependent? # 0 short 0x1fff compacted data !:mime application/octet-stream # This string is valid for SunOS (BE) and a matching "short" is listed # in the Ultrix (LE) magic file. 0 string \377\037 compacted data !:mime application/octet-stream 0 short 0145405 huf output !:mime application/octet-stream # bzip2 0 string BZh bzip2 compressed data !:mime application/x-bzip2 >3 byte >47 \b, block size = %c00k # lzip 0 string LZIP lzip compressed data !:mime application/x-lzip >4 byte x \b, version: %d # squeeze and crunch # Michael Haardt 0 beshort 0x76FF squeezed data, >4 string x original name %s 0 beshort 0x76FE crunched data, >2 string x original name %s 0 beshort 0x76FD LZH compressed data, >2 string x original name %s # Freeze 0 string \037\237 frozen file 2.1 0 string \037\236 frozen file 1.0 (or gzip 0.5) # SCO compress -H (LZH) 0 string \037\240 SCO compress -H (LZH) data # European GSM 06.10 is a provisional standard for full-rate speech # transcoding, prI-ETS 300 036, which uses RPE/LTP (residual pulse # excitation/long term prediction) coding at 13 kbit/s. # # There's only a magic nibble (4 bits); that nibble repeats every 33 # bytes. This isn't suited for use, but maybe we can use it someday. # # This will cause very short GSM files to be declared as data and # mismatches to be declared as data too! #0 byte&0xF0 0xd0 data #>33 byte&0xF0 0xd0 #>66 byte&0xF0 0xd0 #>99 byte&0xF0 0xd0 #>132 byte&0xF0 0xd0 GSM 06.10 compressed audio # bzip a block-sorting file compressor # by Julian Seward and others # #0 string BZ bzip compressed data #>2 byte x \b, version: %c #>3 string =1 \b, compression block size 100k #>3 string =2 \b, compression block size 200k #>3 string =3 \b, compression block size 300k #>3 string =4 \b, compression block size 400k #>3 string =5 \b, compression block size 500k #>3 string =6 \b, compression block size 600k #>3 string =7 \b, compression block size 700k #>3 string =8 \b, compression block size 800k #>3 string =9 \b, compression block size 900k # lzop from 0 string \x89\x4c\x5a\x4f\x00\x0d\x0a\x1a\x0a lzop compressed data >9 beshort <0x0940 >>9 byte&0xf0 =0x00 - version 0. >>9 beshort&0x0fff x \b%03x, >>13 byte 1 LZO1X-1, >>13 byte 2 LZO1X-1(15), >>13 byte 3 LZO1X-999, ## >>22 bedate >0 last modified: %s, >>14 byte =0x00 os: MS-DOS >>14 byte =0x01 os: Amiga >>14 byte =0x02 os: VMS >>14 byte =0x03 os: Unix >>14 byte =0x05 os: Atari >>14 byte =0x06 os: OS/2 >>14 byte =0x07 os: MacOS >>14 byte =0x0A os: Tops/20 >>14 byte =0x0B os: WinNT >>14 byte =0x0E os: Win32 >9 beshort >0x0939 >>9 byte&0xf0 =0x00 - version 0. >>9 byte&0xf0 =0x10 - version 1. >>9 byte&0xf0 =0x20 - version 2. >>9 beshort&0x0fff x \b%03x, >>15 byte 1 LZO1X-1, >>15 byte 2 LZO1X-1(15), >>15 byte 3 LZO1X-999, ## >>25 bedate >0 last modified: %s, >>17 byte =0x00 os: MS-DOS >>17 byte =0x01 os: Amiga >>17 byte =0x02 os: VMS >>17 byte =0x03 os: Unix >>17 byte =0x05 os: Atari >>17 byte =0x06 os: OS/2 >>17 byte =0x07 os: MacOS >>17 byte =0x0A os: Tops/20 >>17 byte =0x0B os: WinNT >>17 byte =0x0E os: Win32 # 4.3BSD-Quasijarus Strong Compression # http://minnie.tuhs.org/Quasijarus/compress.html 0 string \037\241 Quasijarus strong compressed data # From: Cory Dikkers 0 string XPKF Amiga xpkf.library compressed data 0 string PP11 Power Packer 1.1 compressed data 0 string PP20 Power Packer 2.0 compressed data, >4 belong 0x09090909 fast compression >4 belong 0x090A0A0A mediocre compression >4 belong 0x090A0B0B good compression >4 belong 0x090A0C0C very good compression >4 belong 0x090A0C0D best compression # 7-zip archiver, from Thomas Klausner (wiz@danbala.tuwien.ac.at) # http://www.7-zip.org or DOC/7zFormat.txt # 0 string 7z\274\257\047\034 7-zip archive data, >6 byte x version %d >7 byte x \b.%d !:mime application/x-7z-compressed # Type: LZMA 0 lelong&0xffffff =0x5d >12 leshort 0xff LZMA compressed data, !:mime application/x-lzma >>5 lequad =0xffffffffffffffff streamed >>5 lequad !0xffffffffffffffff non-streamed, size %lld >12 leshort 0 LZMA compressed data, >>5 lequad =0xffffffffffffffff streamed >>5 lequad !0xffffffffffffffff non-streamed, size %lld # http://tukaani.org/xz/xz-file-format.txt 0 ustring \xFD7zXZ\x00 XZ compressed data !:mime application/x-xz # https://github.com/ckolivas/lrzip/blob/master/doc/magic.header.txt 0 string LRZI LRZIP compressed data >4 byte x - version %d >5 byte x \b.%d !:mime application/x-lrzip # http://fastcompression.blogspot.fi/2013/04/lz4-streaming-format-final.html 0 lelong 0x184d2204 LZ4 compressed data (v1.4+) !:mime application/x-lz4 # Added by osm0sis@xda-developers.com 0 lelong 0x184c2103 LZ4 compressed data (v1.0-v1.3) !:mime application/x-lz4 0 lelong 0x184c2102 LZ4 compressed data (v0.1-v0.9) !:mime application/x-lz4 # AFX compressed files (Wolfram Kleff) 2 string -afx- AFX compressed file data # Supplementary magic data for the file(1) command to support # rzip(1). The format is described in magic(5). # # Copyright (C) 2003 by Andrew Tridgell. You may do whatever you want with # this file. # 0 string RZIP rzip compressed data >4 byte x - version %d >5 byte x \b.%d >6 belong x (%d bytes) 0 string ArC\x01 FreeArc archive # Type: DACT compressed files 0 long 0x444354C3 DACT compressed data >4 byte >-1 (version %i. >5 byte >-1 %i. >6 byte >-1 %i) >7 long >0 , original size: %i bytes >15 long >30 , block size: %i bytes # Valve Pack (VPK) files 0 lelong 0x55aa1234 Valve Pak file >0x4 lelong x \b, version %u >0x8 lelong x \b, %u entries # Snappy framing format # http://code.google.com/p/snappy/source/browse/trunk/framing_format.txt 0 string \377\006\0\0sNaPpY snappy framed data !:mime application/x-snappy-framed + +# qpress, http://www.quicklz.com/ +0 string qpress10 qpress compressed data +!:mime application/x-qpress + +# Zlib https://www.ietf.org/rfc/rfc6713.txt +0 beshort%31 =0 +>0 byte&0xf =8 +>>0 byte&0x80 =0 zlib compressed data +!:mime application/zlib Index: releng/10.1/contrib/file/magic/Magdir/database =================================================================== --- releng/10.1/contrib/file/magic/Magdir/database (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/database (revision 284193) @@ -1,490 +1,541 @@ #------------------------------------------------------------------------------ -# $File: database,v 1.41 2014/06/03 19:17:27 christos Exp $ +# $File: database,v 1.43 2014/10/28 15:47:39 christos Exp $ # database: file(1) magic for various databases # # extracted from header/code files by Graeme Wilford (eep2gw@ee.surrey.ac.uk) # # # GDBM magic numbers # Will be maintained as part of the GDBM distribution in the future. # -0 belong 0x13579ace GNU dbm 1.x or ndbm database, big endian +0 belong 0x13579acd GNU dbm 1.x or ndbm database, big endian, 32-bit !:mime application/x-gdbm -0 lelong 0x13579ace GNU dbm 1.x or ndbm database, little endian +0 belong 0x13579ace GNU dbm 1.x or ndbm database, big endian, old !:mime application/x-gdbm +0 belong 0x13579acf GNU dbm 1.x or ndbm database, big endian, 64-bit +!:mime application/x-gdbm +0 lelong 0x13579acd GNU dbm 1.x or ndbm database, little endian, 32-bit +!:mime application/x-gdbm +0 lelong 0x13579ace GNU dbm 1.x or ndbm database, little endian, old +!:mime application/x-gdbm +0 lelong 0x13579acf GNU dbm 1.x or ndbm database, little endian, 64-bit +!:mime application/x-gdbm 0 string GDBM GNU dbm 2.x database !:mime application/x-gdbm # # Berkeley DB # # Ian Darwin's file /etc/magic files: big/little-endian version. # # Hash 1.85/1.86 databases store metadata in network byte order. # Btree 1.85/1.86 databases store the metadata in host byte order. # Hash and Btree 2.X and later databases store the metadata in host byte order. 0 long 0x00061561 Berkeley DB !:mime application/x-dbm >8 belong 4321 >>4 belong >2 1.86 >>4 belong <3 1.85 >>4 belong >0 (Hash, version %d, native byte-order) >8 belong 1234 >>4 belong >2 1.86 >>4 belong <3 1.85 >>4 belong >0 (Hash, version %d, little-endian) 0 belong 0x00061561 Berkeley DB >8 belong 4321 >>4 belong >2 1.86 >>4 belong <3 1.85 >>4 belong >0 (Hash, version %d, big-endian) >8 belong 1234 >>4 belong >2 1.86 >>4 belong <3 1.85 >>4 belong >0 (Hash, version %d, native byte-order) 0 long 0x00053162 Berkeley DB 1.85/1.86 >4 long >0 (Btree, version %d, native byte-order) 0 belong 0x00053162 Berkeley DB 1.85/1.86 >4 belong >0 (Btree, version %d, big-endian) 0 lelong 0x00053162 Berkeley DB 1.85/1.86 >4 lelong >0 (Btree, version %d, little-endian) 12 long 0x00061561 Berkeley DB >16 long >0 (Hash, version %d, native byte-order) 12 belong 0x00061561 Berkeley DB >16 belong >0 (Hash, version %d, big-endian) 12 lelong 0x00061561 Berkeley DB >16 lelong >0 (Hash, version %d, little-endian) 12 long 0x00053162 Berkeley DB >16 long >0 (Btree, version %d, native byte-order) 12 belong 0x00053162 Berkeley DB >16 belong >0 (Btree, version %d, big-endian) 12 lelong 0x00053162 Berkeley DB >16 lelong >0 (Btree, version %d, little-endian) 12 long 0x00042253 Berkeley DB >16 long >0 (Queue, version %d, native byte-order) 12 belong 0x00042253 Berkeley DB >16 belong >0 (Queue, version %d, big-endian) 12 lelong 0x00042253 Berkeley DB >16 lelong >0 (Queue, version %d, little-endian) # From Max Bowsher. 12 long 0x00040988 Berkeley DB >16 long >0 (Log, version %d, native byte-order) 12 belong 0x00040988 Berkeley DB >16 belong >0 (Log, version %d, big-endian) 12 lelong 0x00040988 Berkeley DB >16 lelong >0 (Log, version %d, little-endian) # # # Round Robin Database Tool by Tobias Oetiker 0 string/b RRD\0 RRDTool DB >4 string/b x version %s >>10 short !0 16bit aligned >>>10 bedouble 8.642135e+130 big-endian >>>>18 short x 32bit long (m68k) >>10 short 0 >>>12 long !0 32bit aligned >>>>12 bedouble 8.642135e+130 big-endian >>>>>20 long 0 64bit long >>>>>20 long !0 32bit long >>>>12 ledouble 8.642135e+130 little-endian >>>>>24 long 0 64bit long >>>>>24 long !0 32bit long (i386) >>>>12 string \x43\x2b\x1f\x5b\x2f\x25\xc0\xc7 middle-endian >>>>>24 short !0 32bit long (arm) >>8 quad 0 64bit aligned >>>16 bedouble 8.642135e+130 big-endian >>>>24 long 0 64bit long (s390x) >>>>24 long !0 32bit long (hppa/mips/ppc/s390/SPARC) >>>16 ledouble 8.642135e+130 little-endian >>>>28 long 0 64bit long (alpha/amd64/ia64) >>>>28 long !0 32bit long (armel/mipsel) #---------------------------------------------------------------------- # ROOT: file(1) magic for ROOT databases # 0 string root\0 ROOT file >4 belong x Version %d >33 belong x (Compression: %d) # XXX: Weak magic. # Alex Ott ## Paradox file formats #2 leshort 0x0800 Paradox #>0x39 byte 3 v. 3.0 #>0x39 byte 4 v. 3.5 #>0x39 byte 9 v. 4.x #>0x39 byte 10 v. 5.x #>0x39 byte 11 v. 5.x #>0x39 byte 12 v. 7.x #>>0x04 byte 0 indexed .DB data file #>>0x04 byte 1 primary index .PX file #>>0x04 byte 2 non-indexed .DB data file #>>0x04 byte 3 non-incrementing secondary index .Xnn file #>>0x04 byte 4 secondary index .Ynn file #>>0x04 byte 5 incrementing secondary index .Xnn file #>>0x04 byte 6 non-incrementing secondary index .XGn file #>>0x04 byte 7 secondary index .YGn file #>>>0x04 byte 8 incrementing secondary index .XGn file ## XBase database files # updated by Joerg Jenderek at Feb 2013 # http://www.dbase.com/Knowledgebase/INT/db7_file_fmt.htm # http://www.clicketyclick.dk/databases/xbase/format/dbf.html # http://home.f1.htw-berlin.de/scheibl/db/intern/dBase.htm # inspect VVYYMMDD , where 1<= MM <= 12 and 1<= DD <= 31 0 ubelong&0x0000FFFF <0x00000C20 # skip Infocom game Z-machine >2 ubyte >0 # skip Androids *.xml >>3 ubyte >0 >>>3 ubyte <32 # 1 < version VV >>>>0 ubyte >1 # skip HELP.CA3 by test for reserved byte ( NULL ) >>>>>27 ubyte 0 # reserved bytes not always 0 ; also found 0x3901 (T4.DBF) ,0x7101 (T5.DBF,T6.DBF) #>>>>>30 ubeshort x 30NULL?%x # possible production flag,tag numbers(<=0x30),tag length(<=0x20), reserved (NULL) >>>>>>24 ubelong&0xffFFFFff >0x01302000 # .DBF or .MDX >>>>>>24 ubelong&0xffFFFFff <0x01302001 # for Xbase Database file (*.DBF) reserved (NULL) for multi-user >>>>>>>24 ubelong&0xffFFFFff =0 # test for 2 reserved NULL bytes,transaction and encryption byte flag >>>>>>>>12 ubelong&0xFFFFfEfE 0 # test for MDX flag >>>>>>>>>28 ubyte x >>>>>>>>>28 ubyte&0xf8 0 # header size >= 32 >>>>>>>>>>8 uleshort >31 # skip PIC15736.PCX by test for language driver name or field name >>>>>>>>>>>32 ubyte >0 #!:mime application/x-dbf; charset=unknown-8bit ?? #!:mime application/x-dbase >>>>>>>>>>>>0 use xbase-type # database file >>>>>>>>>>>>0 ubyte x \b DBF >>>>>>>>>>>>4 lelong 0 \b, no records >>>>>>>>>>>>4 lelong >0 \b, %d record # plural s appended >>>>>>>>>>>>>4 lelong >1 \bs # http://www.clicketyclick.dk/databases/xbase/format/dbf_check.html#CHECK_DBF # 1 <= record size <= 4000 (dBase 3,4) or 32 * KB (=0x8000) >>>>>>>>>>>>10 uleshort x * %d # file size = records * record size + header size >>>>>>>>>>>>1 ubyte x \b, update-date >>>>>>>>>>>>1 use xbase-date # http://msdn.microsoft.com/de-de/library/cc483186(v=vs.71).aspx #>>>>>>>>>>>>29 ubyte =0 \b, codepage ID=0x%x # 2~cp850 , 3~cp1252 , 0x1b~?? ; what code page is 0x1b ? >>>>>>>>>>>>29 ubyte >0 \b, codepage ID=0x%x #>>>>>>>>>>>>28 ubyte&0x01 0 \b, no index file >>>>>>>>>>>>28 ubyte&0x01 1 \b, with index file .MDX >>>>>>>>>>>>28 ubyte&0x02 2 \b, with memo .FPT >>>>>>>>>>>>28 ubyte&0x04 4 \b, DataBaseContainer # 1st record offset + 1 = header size >>>>>>>>>>>>8 uleshort >0 >>>>>>>>>>>>(8.s+1) ubyte >0 >>>>>>>>>>>>>8 uleshort >0 \b, at offset %d >>>>>>>>>>>>>(8.s+1) ubyte >0 >>>>>>>>>>>>>>&-1 string >\0 1st record "%s" # for multiple index files (*.MDX) Production flag,tag numbers(<=0x30),tag length(<=0x20), reserverd (NULL) >>>>>>>24 ubelong&0x0133f7ff >0 # test for reserved NULL byte ->>>>>>>>47 ubyte x +>>>>>>>>47 ubyte 0 # test for valid TAG key format (0x10 or 0) >>>>>>>>>559 ubyte&0xeF 0 # test MM <= 12 ->>>>>>>>>45 ubeshort <0x0C20 ->>>>>>>>>>45 ubyte >0 ->>>>>>>>>>>46 ubyte <32 ->>>>>>>>>>>>46 ubyte >0 +>>>>>>>>>>45 ubeshort <0x0C20 +>>>>>>>>>>>45 ubyte >0 +>>>>>>>>>>>>46 ubyte <32 +>>>>>>>>>>>>>46 ubyte >0 #!:mime application/x-mdx ->>>>>>>>>>>>>0 use xbase-type ->>>>>>>>>>>>>0 ubyte x \b MDX ->>>>>>>>>>>>>1 ubyte x \b, creation-date ->>>>>>>>>>>>>1 use xbase-date ->>>>>>>>>>>>>44 ubyte x \b, update-date ->>>>>>>>>>>>>44 use xbase-date +>>>>>>>>>>>>>>0 use xbase-type +>>>>>>>>>>>>>>0 ubyte x \b MDX +>>>>>>>>>>>>>>1 ubyte x \b, creation-date +>>>>>>>>>>>>>>1 use xbase-date +>>>>>>>>>>>>>>44 ubyte x \b, update-date +>>>>>>>>>>>>>>44 use xbase-date # No.of tags in use (1,2,5,12) ->>>>>>>>>>>>>28 uleshort x \b, %d +>>>>>>>>>>>>>>28 uleshort x \b, %d # No. of entries in tag (0x30) ->>>>>>>>>>>>>25 ubyte x \b/%d tags +>>>>>>>>>>>>>>25 ubyte x \b/%d tags # Length of tag ->>>>>>>>>>>>>26 ubyte x * %d +>>>>>>>>>>>>>>26 ubyte x * %d # 1st tag name_ >>>>>>>>>>>>>548 string x \b, 1st tag "%.11s" # 2nd tag name #>>>>>>>>>>>>(26.b+548) string x \b, 2nd tag "%.11s" # # Print the xBase names of different version variants 0 name xbase-type >0 ubyte <2 # 1 < version >0 ubyte >1 >>0 ubyte 0x02 FoxBase # FoxBase+/dBaseIII+, no memo >>0 ubyte 0x03 FoxBase+/dBase III !:mime application/x-dbf # dBASE IV no memo file >>0 ubyte 0x04 dBase IV !:mime application/x-dbf # dBASE V no memo file >>0 ubyte 0x05 dBase V !:mime application/x-dbf >>0 ubyte 0x30 Visual FoxPro !:mime application/x-dbf >>0 ubyte 0x31 Visual FoxPro, autoincrement !:mime application/x-dbf # Visual FoxPro, with field type Varchar or Varbinary >>0 ubyte 0x32 Visual FoxPro, with field type Varchar !:mime application/x-dbf # dBASE IV SQL, no memo;dbv memo var size (Flagship) >>0 ubyte 0x43 dBase IV, with SQL table !:mime application/x-dbf # http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0x62 dBase IV, with SQL table #!:mime application/x-dbf # dBASE IV, with memo!! >>0 ubyte 0x7b dBase IV, with memo !:mime application/x-dbf # http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0x82 dBase IV, with SQL system #!:mime application/x-dbf # FoxBase+/dBaseIII+ with memo .DBT! >>0 ubyte 0x83 FoxBase+/dBase III, with memo .DBT !:mime application/x-dbf # VISUAL OBJECTS (first 1.0 versions) for the Dbase III files (NTX clipper driver); memo file >>0 ubyte 0x87 VISUAL OBJECTS, with memo file !:mime application/x-dbf # http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0x8A FoxBase+/dBase III, with memo .DBT #!:mime application/x-dbf # dBASE IV with memo! >>0 ubyte 0x8B dBase IV, with memo .DBT !:mime application/x-dbf # dBase IV with SQL Table,no memo? >>0 ubyte 0x8E dBase IV, with SQL table !:mime application/x-dbf # .dbv and .dbt memo (Flagship)? >>0 ubyte 0xB3 Flagship # http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0xCA dBase IV with memo .DBT #!:mime application/x-dbf # dBASE IV with SQL table, with memo .DBT >>0 ubyte 0xCB dBase IV with SQL table, with memo .DBT !:mime application/x-dbf # HiPer-Six format;Clipper SIX, with SMT memo file >>0 ubyte 0xE5 Clipper SIX with memo !:mime application/x-dbf # http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0xF4 dBase IV, with SQL table, with memo #!:mime application/x-dbf >>0 ubyte 0xF5 FoxPro with memo !:mime application/x-dbf # http://msdn.microsoft.com/en-US/library/st4a0s68(v=vs.80).aspx #>>0 ubyte 0xFA FoxPro 2.x, with memo #!:mime application/x-dbf # unknown version (should not happen) >>0 default x xBase !:mime application/x-dbf >>>0 ubyte x (0x%x) # flags in version byte # DBT flag (with dBASE III memo .DBT)!! # >>0 ubyte&0x80 >0 DBT_FLAG=%x # memo flag ?? # >>0 ubyte&0x08 >0 MEMO_FLAG=%x # SQL flag ?? # >>0 ubyte&0x70 >0 SQL_FLAG=%x # test and print the date of xBase .DBF .MDX 0 name xbase-date # inspect YYMMDD , where 1<= MM <= 12 and 1<= DD <= 31 >0 ubelong x >1 ubyte <13 >>1 ubyte >0 >>>2 ubyte >0 >>>>2 ubyte <32 >>>>>0 ubyte x # YY is interpreted as 20YY or 19YY >>>>>>0 ubyte <100 \b %.2d # YY is interpreted 1900+YY; TODO: display yy or 20yy instead 1YY >>>>>>0 ubyte >99 \b %d >>>>>1 ubyte x \b-%d >>>>>2 ubyte x \b-%d # dBase memo files .DBT or .FPT # http://msdn.microsoft.com/en-us/library/8599s21w(v=vs.80).aspx 16 ubyte <4 >16 ubyte !2 >>16 ubyte !1 # next free block index is positive >>>0 ulelong >0 # skip many JPG. ZIP, BZ2 by test for reserved bytes NULL , 0|2 , 0|1 , low byte of block size >>>>17 ubelong&0xFFfdFE00 0x00000000 # skip many RAR by test for low byte 0 ,high byte 0|2|even of block size, 0|a|e|d7 , 0|64h >>>>>20 ubelong&0xFF01209B 0x00000000 # dBASE III >>>>>>16 ubyte 3 # dBASE III DBT ->>>>>>>0 use xbase-memo-print -# dBASE IV DBT , FoxPro FPT or many PNG , ZIP , DBF garbage +>>>>>>>0 use dbase3-memo-print +# dBASE III DBT without version, dBASE IV DBT , FoxPro FPT , or many ZIP , DBF garbage >>>>>>16 ubyte 0 -# dBASE IV DBT with DBF name or DBF garbage ->>>>>>>8 ubelong >0x40000000 -# skip DBF and catch dBASE IV DBT with DBF name and with non big index of next free block ->>>>>>>>0 ulelong <0x01010002 ->>>>>>>>>0 use xbase-memo-print ->>>>>>>8 ubelong 0 +# unusual dBASE III DBT like angest.dbt, dBASE IV DBT with block size 0 , FoxPro FPT , or garbage PCX DBF +>>>>>>>20 uleshort 0 +# FoxPro FPT , unusual dBASE III DBT like biblio.dbt or garbage +>>>>>>>>8 ulong =0 +>>>>>>>>>6 ubeshort >0 +# skip emacs.PIF +>>>>>>>>>>4 ushort 0 +>>>>>>>>>>>0 use foxpro-memo-print +# dBASE III DBT , garbage +>>>>>>>>>6 ubeshort 0 # skip MM*DD*.bin by test for for reserved NULL byte ->>>>>>>>508 ubelong 0 -# real memo files should contain text here ->>>>>>>>>520 ubelong >0x20202019 ->>>>>>>>>>520 ubelong <0xFEFEFEFF ->>>>>>>>>>>0 use xbase-memo-print -# garbage PCX , ZIP , JAR , XPI ->>>>>>>8 default x +>>>>>>>>>>510 ubeshort 0 +# skip TK-DOS11.img image by looking for memo text +>>>>>>>>>>>512 ubelong <0xfeffff03 +# skip EFI executables by looking for memo text +>>>>>>>>>>>>512 ubelong >0x1F202020 +>>>>>>>>>>>>>513 ubyte >0 +# unusual dBASE III DBT like adressen.dbt +>>>>>>>>>>>>>>0 use dbase3-memo-print +# dBASE III DBT like angest.dbt, or garbage PCX DBF +>>>>>>>>8 ubelong !0 +# skip PCX and some DBF by test for for reserved NULL bytes +>>>>>>>>>510 ubeshort 0 +# skip some DBF by test of invalid version +>>>>>>>>>>0 ubyte >5 +>>>>>>>>>>>0 ubyte <48 +>>>>>>>>>>>>0 use dbase3-memo-print +# dBASE IV DBT with positive block size +>>>>>>>20 uleshort >0 +>>>>>>>>0 use dbase4-memo-print -# Print the information of dBase DBT or FoxPro FPT memo files -0 name xbase-memo-print ->0 ubyte x -# test version -# memo file ->>16 ubyte 3 dBase III DBT ->>16 ubyte 0 ->>>512 ubelong <0x00000003 FoxPro FPT -# Size of blocks for FoxPro ->>>>6 ubeshort x \b, blocks size %u -# Number of next available block for appending data for FoxPro ->>>>0 ubelong =0 \b, next free block index %u ->>>>0 ubelong !0 \b, next free block index %u ->>>512 default x dBase IV DBT -# DBF file name without extension ->>>>8 string >\0 \b of %-.8s.DBF -# size of blocks ; not reliable 0x2020204C -#>>>>4 ulelong =0 \b, blocks size %u ->>>>4 ulelong !0 \b, blocks size %u -# Block length found 0 , 512 -#>>>>20 uleshort =0 \b, block length %u ->>>>20 uleshort !0 \b, block length %u +# Print the information of dBase III DBT memo file +0 name dbase3-memo-print +>0 ubyte x dBase III DBT +# instead 3 as version number 0 for unusual examples like biblio.dbt +>16 ubyte !3 \b, version number %u # Number of next available block for appending data ->>>>0 ulelong =0 \b, next free block index %u ->>>>0 ulelong !0 \b, next free block index %u ->>512 ubelong x ->>>512 ubelong =0xFFFF0800 ->>>>520 string >\0 \b, 1st used item "%s" -# FoxPro ->>>512 ubelong <3 -# FoxPro memo ->>>>512 ubelong =1 ->>>>520 string >\0 \b, 1st used item "%s" ->>>512 default x -# may be deleted memo field ->>>>512 string >\0 \b, 1st item "%s" +#>0 lelong =0 \b, next free block index %u +>0 lelong !0 \b, next free block index %u +# no positiv block length +#>20 uleshort =0 \b, block length %u +>20 uleshort !0 \b, block length %u +# dBase III memo field terminated by \032\032 +>512 string >\0 \b, 1st item "%s" +# Print the information of dBase IV DBT memo file +0 name dbase4-memo-print +>0 lelong x dBase IV DBT +# 8 character shorted main name of coresponding dBASE IV DBF file +>8 ubelong >0x20000000 +# skip unusual like for angest.dbt +>>20 uleshort >0 +>>>8 string >\0 \b of %-.8s.DBF +# value 0 implies 512 as size +#>4 ulelong =0 \b, blocks size %u +# size of blocks not reliable like 0x2020204C in angest.dbt +>4 ulelong !0 +>>4 ulelong&0x0000003f 0 \b, blocks size %u +# dBase IV DBT with positive block length (found 512 , 1024) +>20 uleshort >0 \b, block length %u +# next available block +#>0 lelong =0 \b, next free block index %u +>0 lelong !0 \b, next free block index %u +>20 uleshort >0 +>>(20.s) ubelong x +>>>&-4 use dbase4-memofield-print +# unusual dBase IV DBT without block length (implies 512 as length) +>20 uleshort =0 +>>512 ubelong x +>>>&-4 use dbase4-memofield-print +# Print the information of dBase IV memo field +0 name dbase4-memofield-print +# free dBase IV memo field +>0 ubelong !0xFFFF0800 +>>0 lelong x \b, next free block %u +>>4 lelong x \b, next used block %u +# used dBase IV memo field +>0 ubelong =0xFFFF0800 +# length of memo field +>>4 lelong x \b, field length %d +>>>8 string >\0 \b, 1st used item "%s" +# Print the information of FoxPro FPT memo file +0 name foxpro-memo-print +>0 belong x FoxPro FPT +# Size of blocks for FoxPro ( 64,256 ) +>6 ubeshort x \b, blocks size %u +# next available block +#>0 belong =0 \b, next free block index %u +>0 belong !0 \b, next free block index %u +# field type ( 0~picture, 1~memo, 2~object ) +>512 ubelong <3 \b, field type %u +# length of memo field +>512 ubelong 1 +>>516 belong >0 \b, field length %d +>>>520 string >\0 \b, 1st item "%s" # TODO: # DBASE index file *.NDX # DBASE Compound Index file *.CDX # dBASE IV Printer Driver *.PRF ## End of XBase database stuff # MS Access database 4 string Standard\ Jet\ DB Microsoft Access Database !:mime application/x-msaccess 4 string Standard\ ACE\ DB Microsoft Access Database !:mime application/x-msaccess # TDB database from Samba et al - Martin Pool 0 string TDB\ file TDB database >32 lelong 0x2601196D version 6, little-endian >>36 lelong x hash size %d bytes # SE Linux policy database 0 lelong 0xf97cff8c SE Linux policy >16 lelong x v%d >20 lelong 1 MLS >24 lelong x %d symbols >28 lelong x %d ocons # ICE authority file data (Wolfram Kleff) 2 string ICE ICE authority data # X11 Xauthority file (Wolfram Kleff) 10 string MIT-MAGIC-COOKIE-1 X11 Xauthority data 11 string MIT-MAGIC-COOKIE-1 X11 Xauthority data 12 string MIT-MAGIC-COOKIE-1 X11 Xauthority data 13 string MIT-MAGIC-COOKIE-1 X11 Xauthority data 14 string MIT-MAGIC-COOKIE-1 X11 Xauthority data 15 string MIT-MAGIC-COOKIE-1 X11 Xauthority data 16 string MIT-MAGIC-COOKIE-1 X11 Xauthority data 17 string MIT-MAGIC-COOKIE-1 X11 Xauthority data 18 string MIT-MAGIC-COOKIE-1 X11 Xauthority data # From: Maxime Henrion # PostgreSQL's custom dump format, Maxime Henrion 0 string PGDMP PostgreSQL custom database dump >5 byte x - v%d >6 byte x \b.%d >5 beshort <0x101 \b-0 >5 beshort >0x100 >>7 byte x \b-%d # Type: Advanced Data Format (ADF) database # URL: http://www.grc.nasa.gov/WWW/cgns/adf/ # From: Nicolas Chauvat 0 string @(#)ADF\ Database CGNS Advanced Data Format # Tokyo Cabinet magic data # http://tokyocabinet.sourceforge.net/index.html 0 string ToKyO\ CaBiNeT\n Tokyo Cabinet >14 string x \b (%s) >32 byte 0 \b, Hash !:mime application/x-tokyocabinet-hash >32 byte 1 \b, B+ tree !:mime application/x-tokyocabinet-btree >32 byte 2 \b, Fixed-length !:mime application/x-tokyocabinet-fixed >32 byte 3 \b, Table !:mime application/x-tokyocabinet-table >33 byte &1 \b, [open] >33 byte &2 \b, [fatal] >34 byte x \b, apow=%d >35 byte x \b, fpow=%d >36 byte &0x01 \b, [large] >36 byte &0x02 \b, [deflate] >36 byte &0x04 \b, [bzip] >36 byte &0x08 \b, [tcbs] >36 byte &0x10 \b, [excodec] >40 lequad x \b, bnum=%lld >48 lequad x \b, rnum=%lld >56 lequad x \b, fsiz=%lld # Type: QDBM Quick Database Manager # From: Benoit Sibaud 0 string \\[depot\\]\n\f Quick Database Manager, little endian 0 string \\[DEPOT\\]\n\f Quick Database Manager, big endian # Type: TokyoCabinet database # URL: http://tokyocabinet.sourceforge.net/ # From: Benoit Sibaud 0 string ToKyO\ CaBiNeT\n TokyoCabinet database >14 string x (version %s) # From: Stephane Blondon http://www.yaal.fr # Database file for Zope (done by FileStorage) 0 string FS21 Zope Object Database File Storage (data) # Cache file for the database of Zope (done by ClientStorage) 0 string ZEC3 Zope Object Database Client Cache File (data) # IDA (Interactive Disassembler) database 0 string IDA1 IDA (Interactive Disassembler) database Index: releng/10.1/contrib/file/magic/Magdir/elf =================================================================== --- releng/10.1/contrib/file/magic/Magdir/elf (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/elf (revision 284193) @@ -1,324 +1,325 @@ #------------------------------------------------------------------------------ -# $File: elf,v 1.67 2014/06/12 13:52:48 christos Exp $ +# $File: elf,v 1.68 2014/09/19 19:05:57 christos Exp $ # elf: file(1) magic for ELF executables # # We have to check the byte order flag to see what byte order all the # other stuff in the header is in. # # What're the correct byte orders for the nCUBE and the Fujitsu VPP500? # # Created by: unknown # Modified by (1): Daniel Quinlan # Modified by (2): Peter Tobias (core support) # Modified by (3): Christian 'Dr. Disk' Hechelmann (fix of core support) # Modified by (4): (VMS Itanium) # Modified by (5): Matthias Urlichs (Listing of many architectures) 0 name elf-le >16 leshort 0 no file type, !:mime application/octet-stream >16 leshort 1 relocatable, !:mime application/x-object >16 leshort 2 executable, !:mime application/x-executable >16 leshort 3 shared object, !:mime application/x-sharedlib >16 leshort 4 core file !:mime application/x-coredump # Core file detection is not reliable. #>>>(0x38+0xcc) string >\0 of '%s' #>>>(0x38+0x10) lelong >0 (signal %d), >16 leshort &0xff00 processor-specific, >18 clear x >18 leshort 0 no machine, >18 leshort 1 AT&T WE32100, >18 leshort 2 SPARC, >18 leshort 3 Intel 80386, >18 leshort 4 Motorola m68k, >>4 byte 1 >>>36 lelong &0x01000000 68000, >>>36 lelong &0x00810000 CPU32, >>>36 lelong 0 68020, >18 leshort 5 Motorola m88k, >18 leshort 6 Intel 80486, >18 leshort 7 Intel 80860, # The official e_machine number for MIPS is now #8, regardless of endianness. # The second number (#10) will be deprecated later. For now, we still # say something if #10 is encountered, but only gory details for #8. >18 leshort 8 MIPS, >>4 byte 1 >>>36 lelong &0x20 N32 >18 leshort 10 MIPS, >>4 byte 1 >>>36 lelong &0x20 N32 >18 leshort 8 # only for 32-bit >>4 byte 1 >>>36 lelong&0xf0000000 0x00000000 MIPS-I >>>36 lelong&0xf0000000 0x10000000 MIPS-II >>>36 lelong&0xf0000000 0x20000000 MIPS-III >>>36 lelong&0xf0000000 0x30000000 MIPS-IV >>>36 lelong&0xf0000000 0x40000000 MIPS-V >>>36 lelong&0xf0000000 0x50000000 MIPS32 >>>36 lelong&0xf0000000 0x60000000 MIPS64 >>>36 lelong&0xf0000000 0x70000000 MIPS32 rel2 >>>36 lelong&0xf0000000 0x80000000 MIPS64 rel2 # only for 64-bit >>4 byte 2 >>>48 lelong&0xf0000000 0x00000000 MIPS-I >>>48 lelong&0xf0000000 0x10000000 MIPS-II >>>48 lelong&0xf0000000 0x20000000 MIPS-III >>>48 lelong&0xf0000000 0x30000000 MIPS-IV >>>48 lelong&0xf0000000 0x40000000 MIPS-V >>>48 lelong&0xf0000000 0x50000000 MIPS32 >>>48 lelong&0xf0000000 0x60000000 MIPS64 >>>48 lelong&0xf0000000 0x70000000 MIPS32 rel2 >>>48 lelong&0xf0000000 0x80000000 MIPS64 rel2 >18 leshort 9 Amdahl, >18 leshort 10 MIPS (deprecated), >18 leshort 11 RS6000, >18 leshort 15 PA-RISC, # only for 32-bit >>4 byte 1 >>>38 leshort 0x0214 2.0 >>>36 leshort &0x0008 (LP64) # only for 64-bit >>4 byte 2 >>>50 leshort 0x0214 2.0 >>>48 leshort &0x0008 (LP64) >18 leshort 16 nCUBE, >18 leshort 17 Fujitsu VPP500, >18 leshort 18 SPARC32PLUS, # only for 32-bit >>4 byte 1 >>>36 lelong&0xffff00 0x000100 V8+ Required, >>>36 lelong&0xffff00 0x000200 Sun UltraSPARC1 Extensions Required, >>>36 lelong&0xffff00 0x000400 HaL R1 Extensions Required, >>>36 lelong&0xffff00 0x000800 Sun UltraSPARC3 Extensions Required, >18 leshort 19 Intel 80960, >18 leshort 20 PowerPC or cisco 4500, >18 leshort 21 64-bit PowerPC or cisco 7500, >18 leshort 22 IBM S/390, >18 leshort 23 Cell SPU, >18 leshort 24 cisco SVIP, >18 leshort 25 cisco 7200, >18 leshort 36 NEC V800 or cisco 12000, >18 leshort 37 Fujitsu FR20, >18 leshort 38 TRW RH-32, >18 leshort 39 Motorola RCE, >18 leshort 40 ARM, >>4 byte 1 >>>36 lelong&0xff000000 0x04000000 EABI4 >>>36 lelong&0xff000000 0x05000000 EABI5 >>>36 lelong &0x00800000 BE8 >>>36 lelong &0x00400000 LE8 >18 leshort 41 Alpha, >18 leshort 42 Renesas SH, >18 leshort 43 SPARC V9, >>4 byte 2 >>>48 lelong&0xffff00 0x000200 Sun UltraSPARC1 Extensions Required, >>>48 lelong&0xffff00 0x000400 HaL R1 Extensions Required, >>>48 lelong&0xffff00 0x000800 Sun UltraSPARC3 Extensions Required, >>>48 lelong&0x3 0 total store ordering, >>>48 lelong&0x3 1 partial store ordering, >>>48 lelong&0x3 2 relaxed memory ordering, >18 leshort 44 Siemens Tricore Embedded Processor, >18 leshort 45 Argonaut RISC Core, Argonaut Technologies Inc., >18 leshort 46 Renesas H8/300, >18 leshort 47 Renesas H8/300H, >18 leshort 48 Renesas H8S, >18 leshort 49 Renesas H8/500, >18 leshort 50 IA-64, >18 leshort 51 Stanford MIPS-X, >18 leshort 52 Motorola Coldfire, >18 leshort 53 Motorola M68HC12, >18 leshort 54 Fujitsu MMA, >18 leshort 55 Siemens PCP, >18 leshort 56 Sony nCPU, >18 leshort 57 Denso NDR1, >18 leshort 58 Start*Core, >18 leshort 59 Toyota ME16, >18 leshort 60 ST100, >18 leshort 61 Tinyj emb., >18 leshort 62 x86-64, >18 leshort 63 Sony DSP, >18 leshort 64 DEC PDP-10, >18 leshort 65 DEC PDP-11, >18 leshort 66 FX66, >18 leshort 67 ST9+ 8/16 bit, >18 leshort 68 ST7 8 bit, >18 leshort 69 MC68HC16, >18 leshort 70 MC68HC11, >18 leshort 71 MC68HC08, >18 leshort 72 MC68HC05, >18 leshort 73 SGI SVx or Cray NV1, >18 leshort 74 ST19 8 bit, >18 leshort 75 Digital VAX, >18 leshort 76 Axis cris, >18 leshort 77 Infineon 32-bit embedded, >18 leshort 78 Element 14 64-bit DSP, >18 leshort 79 LSI Logic 16-bit DSP, >18 leshort 80 MMIX, >18 leshort 81 Harvard machine-independent, >18 leshort 82 SiTera Prism, >18 leshort 83 Atmel AVR 8-bit, >18 leshort 84 Fujitsu FR30, >18 leshort 85 Mitsubishi D10V, >18 leshort 86 Mitsubishi D30V, >18 leshort 87 NEC v850, >18 leshort 88 Renesas M32R, >18 leshort 89 Matsushita MN10300, >18 leshort 90 Matsushita MN10200, >18 leshort 91 picoJava, >18 leshort 92 OpenRISC, >18 leshort 93 ARC Cores Tangent-A5, >18 leshort 94 Tensilica Xtensa, >18 leshort 95 Alphamosaic VideoCore, >18 leshort 96 Thompson Multimedia, >18 leshort 97 NatSemi 32k, >18 leshort 98 Tenor Network TPC, >18 leshort 99 Trebia SNP 1000, >18 leshort 100 STMicroelectronics ST200, >18 leshort 101 Ubicom IP2022, >18 leshort 102 MAX Processor, >18 leshort 103 NatSemi CompactRISC, >18 leshort 104 Fujitsu F2MC16, >18 leshort 105 TI msp430, >18 leshort 106 Analog Devices Blackfin, >18 leshort 107 S1C33 Family of Seiko Epson, >18 leshort 108 Sharp embedded, >18 leshort 109 Arca RISC, >18 leshort 110 PKU-Unity Ltd., >18 leshort 111 eXcess: 16/32/64-bit, >18 leshort 112 Icera Deep Execution Processor, >18 leshort 113 Altera Nios II, >18 leshort 114 NatSemi CRX, >18 leshort 115 Motorola XGATE, >18 leshort 116 Infineon C16x/XC16x, >18 leshort 117 Renesas M16C series, >18 leshort 118 Microchip dsPIC30F, >18 leshort 119 Freescale RISC core, >18 leshort 120 Renesas M32C series, >18 leshort 131 Altium TSK3000 core, >18 leshort 132 Freescale RS08, >18 leshort 134 Cyan Technology eCOG2, >18 leshort 135 Sunplus S+core7 RISC, >18 leshort 136 New Japan Radio (NJR) 24-bit DSP, >18 leshort 137 Broadcom VideoCore III, >18 leshort 138 LatticeMico32, >18 leshort 139 Seiko Epson C17 family, >18 leshort 140 TI TMS320C6000 DSP family, >18 leshort 141 TI TMS320C2000 DSP family, >18 leshort 142 TI TMS320C55x DSP family, >18 leshort 160 STMicroelectronics 64bit VLIW DSP, >18 leshort 161 Cypress M8C, >18 leshort 162 Renesas R32C series, >18 leshort 163 NXP TriMedia family, >18 leshort 164 QUALCOMM DSP6, >18 leshort 165 Intel 8051 and variants, >18 leshort 166 STMicroelectronics STxP7x family, >18 leshort 167 Andes embedded RISC, >18 leshort 168 Cyan eCOG1X family, >18 leshort 169 Dallas MAXQ30, >18 leshort 170 New Japan Radio (NJR) 16-bit DSP, >18 leshort 171 M2000 Reconfigurable RISC, >18 leshort 172 Cray NV2 vector architecture, >18 leshort 173 Renesas RX family, >18 leshort 174 META, >18 leshort 175 MCST Elbrus, >18 leshort 176 Cyan Technology eCOG16 family, >18 leshort 177 NatSemi CompactRISC, >18 leshort 178 Freescale Extended Time Processing Unit, >18 leshort 179 Infineon SLE9X, >18 leshort 180 Intel L1OM, >18 leshort 181 Intel K1OM, >18 leshort 183 ARM aarch64, >18 leshort 185 Atmel 32-bit family, >18 leshort 186 STMicroeletronics STM8 8-bit, >18 leshort 187 Tilera TILE64, >18 leshort 188 Tilera TILEPro, >18 leshort 189 Xilinx MicroBlaze 32-bit RISC, >18 leshort 190 NVIDIA CUDA architecture, >18 leshort 191 Tilera TILE-Gx, >18 leshort 197 Renesas RL78 family, >18 leshort 199 Renesas 78K0R, >18 leshort 200 Freescale 56800EX, >18 leshort 201 Beyond BA1, >18 leshort 202 Beyond BA2, >18 leshort 203 XMOS xCORE, >18 leshort 204 Microchip 8-bit PIC(r), >18 leshort 210 KM211 KM32, >18 leshort 211 KM211 KMX32, >18 leshort 212 KM211 KMX16, >18 leshort 213 KM211 KMX8, >18 leshort 214 KM211 KVARC, >18 leshort 215 Paneve CDP, >18 leshort 216 Cognitive Smart Memory, >18 leshort 217 iCelero CoolEngine, >18 leshort 218 Nanoradio Optimized RISC, +>18 leshort 243 UCB RISC-V, >18 leshort 0x1057 AVR (unofficial), >18 leshort 0x1059 MSP430 (unofficial), >18 leshort 0x1223 Adapteva Epiphany (unofficial), >18 leshort 0x2530 Morpho MT (unofficial), >18 leshort 0x3330 FR30 (unofficial), >18 leshort 0x3426 OpenRISC (obsolete), >18 leshort 0x4688 Infineon C166 (unofficial), >18 leshort 0x5441 Cygnus FRV (unofficial), >18 leshort 0x5aa5 DLX (unofficial), >18 leshort 0x7650 Cygnus D10V (unofficial), >18 leshort 0x7676 Cygnus D30V (unofficial), >18 leshort 0x8217 Ubicom IP2xxx (unofficial), >18 leshort 0x8472 OpenRISC (obsolete), >18 leshort 0x9025 Cygnus PowerPC (unofficial), >18 leshort 0x9026 Alpha (unofficial), >18 leshort 0x9041 Cygnus M32R (unofficial), >18 leshort 0x9080 Cygnus V850 (unofficial), >18 leshort 0xa390 IBM S/390 (obsolete), >18 leshort 0xabc7 Old Xtensa (unofficial), >18 leshort 0xad45 xstormy16 (unofficial), >18 leshort 0xbaab Old MicroBlaze (unofficial),, >18 leshort 0xbeef Cygnus MN10300 (unofficial), >18 leshort 0xdead Cygnus MN10200 (unofficial), >18 leshort 0xf00d Toshiba MeP (unofficial), >18 leshort 0xfeb0 Renesas M32C (unofficial), >18 leshort 0xfeba Vitesse IQ2000 (unofficial), >18 leshort 0xfebb NIOS (unofficial), >18 leshort 0xfeed Moxie (unofficial), >18 default x >>18 leshort x *unknown arch 0x%x* >20 lelong 0 invalid version >20 lelong 1 version 1 0 string \177ELF ELF !:strength *2 >4 byte 0 invalid class >4 byte 1 32-bit >4 byte 2 64-bit >5 byte 0 invalid byte order >5 byte 1 LSB >>0 use elf-le >5 byte 2 MSB >>0 use \^elf-le # Up to now only 0, 1 and 2 are defined; I've seen a file with 0x83, it seemed # like proper ELF, but extracting the string had bad results. >4 byte <0x80 >>8 string >\0 (%s) >8 string \0 >>7 byte 0 (SYSV) >>7 byte 1 (HP-UX) >>7 byte 2 (NetBSD) >>7 byte 3 (GNU/Linux) >>7 byte 4 (GNU/Hurd) >>7 byte 5 (86Open) >>7 byte 6 (Solaris) >>7 byte 7 (Monterey) >>7 byte 8 (IRIX) >>7 byte 9 (FreeBSD) >>7 byte 10 (Tru64) >>7 byte 11 (Novell Modesto) >>7 byte 12 (OpenBSD) >8 string \2 >>7 byte 13 (OpenVMS) >>7 byte 97 (ARM) >>7 byte 255 (embedded) Index: releng/10.1/contrib/file/magic/Magdir/filesystems =================================================================== --- releng/10.1/contrib/file/magic/Magdir/filesystems (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/filesystems (revision 284193) @@ -1,2279 +1,2345 @@ #------------------------------------------------------------------------------ -# $File: filesystems,v 1.95 2014/06/03 19:17:27 christos Exp $ +# $File: filesystems,v 1.108 2015/01/01 17:43:47 christos Exp $ # filesystems: file(1) magic for different filesystems # 0 name partid >0 ubyte 0x00 Unused >0 ubyte 0x01 12-bit FAT >0 ubyte 0x02 XENIX / >0 ubyte 0x03 XENIX /usr >0 ubyte 0x04 16-bit FAT, less than 32M >0 ubyte 0x05 extended partition >0 ubyte 0x06 16-bit FAT, more than 32M >0 ubyte 0x07 OS/2 HPFS, NTFS, QNX2, Adv. UNIX >0 ubyte 0x08 AIX or os, or etc. >0 ubyte 0x09 AIX boot partition or Coherent >0 ubyte 0x0a O/2 boot manager or Coherent swap >0 ubyte 0x0b 32-bit FAT >0 ubyte 0x0c 32-bit FAT, LBA-mapped >0 ubyte 0x0d 7XXX, LBA-mapped >0 ubyte 0x0e 16-bit FAT, LBA-mapped >0 ubyte 0x0f extended partition, LBA-mapped >0 ubyte 0x10 OPUS >0 ubyte 0x11 OS/2 DOS 12-bit FAT >0 ubyte 0x12 Compaq diagnostics >0 ubyte 0x14 OS/2 DOS 16-bit FAT <32M >0 ubyte 0x16 OS/2 DOS 16-bit FAT >=32M >0 ubyte 0x17 OS/2 hidden IFS >0 ubyte 0x18 AST Windows swapfile >0 ubyte 0x19 Willowtech Photon coS >0 ubyte 0x1b hidden win95 fat 32 >0 ubyte 0x1c hidden win95 fat 32 lba >0 ubyte 0x1d hidden win95 fat 16 lba >0 ubyte 0x20 Willowsoft OFS1 >0 ubyte 0x21 reserved >0 ubyte 0x23 reserved >0 ubyte 0x24 NEC DOS >0 ubyte 0x26 reserved >0 ubyte 0x31 reserved >0 ubyte 0x32 Alien Internet Services NOS >0 ubyte 0x33 reserved >0 ubyte 0x34 reserved >0 ubyte 0x35 JFS on OS2 >0 ubyte 0x36 reserved >0 ubyte 0x38 Theos >0 ubyte 0x39 Plan 9, or Theos spanned >0 ubyte 0x3a Theos ver 4 4gb partition >0 ubyte 0x3b Theos ve 4 extended partition >0 ubyte 0x3c PartitionMagic recovery >0 ubyte 0x3d Hidden Netware >0 ubyte 0x40 VENIX 286 or LynxOS >0 ubyte 0x41 PReP >0 ubyte 0x42 linux swap sharing DRDOS disk >0 ubyte 0x43 linux sharing DRDOS disk >0 ubyte 0x44 GoBack change utility >0 ubyte 0x45 Boot US Boot manager >0 ubyte 0x46 EUMEL/Elan or Ergos 3 >0 ubyte 0x47 EUMEL/Elan or Ergos 3 >0 ubyte 0x48 EUMEL/Elan or Ergos 3 >0 ubyte 0x4a ALFX/THIN filesystem for DOS >0 ubyte 0x4c Oberon partition >0 ubyte 0x4d QNX4.x >0 ubyte 0x4e QNX4.x 2nd part >0 ubyte 0x4f QNX4.x 3rd part >0 ubyte 0x50 DM (disk manager) >0 ubyte 0x51 DM6 Aux1 (or Novell) >0 ubyte 0x52 CP/M or Microport SysV/AT >0 ubyte 0x53 DM6 Aux3 >0 ubyte 0x54 DM6 DDO >0 ubyte 0x55 EZ-Drive (disk manager) >0 ubyte 0x56 Golden Bow (disk manager) >0 ubyte 0x57 Drive PRO >0 ubyte 0x5c Priam Edisk (disk manager) >0 ubyte 0x61 SpeedStor >0 ubyte 0x63 GNU HURD or Mach or Sys V/386 >0 ubyte 0x64 Novell Netware 2.xx or Speedstore >0 ubyte 0x65 Novell Netware 3.xx >0 ubyte 0x66 Novell 386 Netware >0 ubyte 0x67 Novell >0 ubyte 0x68 Novell >0 ubyte 0x69 Novell >0 ubyte 0x70 DiskSecure Multi-Boot >0 ubyte 0x71 reserved >0 ubyte 0x73 reserved >0 ubyte 0x74 reserved >0 ubyte 0x75 PC/IX >0 ubyte 0x76 reserved >0 ubyte 0x77 M2FS/M2CS partition >0 ubyte 0x78 XOSL boot loader filesystem >0 ubyte 0x80 MINIX until 1.4a >0 ubyte 0x81 MINIX since 1.4b >0 ubyte 0x82 Linux swap or Solaris >0 ubyte 0x83 Linux native >0 ubyte 0x84 OS/2 hidden C: drive >0 ubyte 0x85 Linux extended partition >0 ubyte 0x86 NT FAT volume set >0 ubyte 0x87 NTFS volume set or HPFS mirrored >0 ubyte 0x8a Linux Kernel AiR-BOOT partition >0 ubyte 0x8b Legacy Fault tolerant FAT32 >0 ubyte 0x8c Legacy Fault tolerant FAT32 ext >0 ubyte 0x8d Hidden free FDISK FAT12 >0 ubyte 0x8e Linux Logical Volume Manager >0 ubyte 0x90 Hidden free FDISK FAT16 >0 ubyte 0x91 Hidden free FDISK DOS EXT >0 ubyte 0x92 Hidden free FDISK FAT16 Big >0 ubyte 0x93 Amoeba filesystem >0 ubyte 0x94 Amoeba bad block table >0 ubyte 0x95 MIT EXOPC native partitions >0 ubyte 0x97 Hidden free FDISK FAT32 >0 ubyte 0x98 Datalight ROM-DOS Super-Boot >0 ubyte 0x99 Mylex EISA SCSI >0 ubyte 0x9a Hidden free FDISK FAT16 LBA >0 ubyte 0x9b Hidden free FDISK EXT LBA >0 ubyte 0x9f BSDI? >0 ubyte 0xa0 IBM Thinkpad hibernation >0 ubyte 0xa1 HP Volume expansion (SpeedStor) >0 ubyte 0xa3 HP Volume expansion (SpeedStor) >0 ubyte 0xa4 HP Volume expansion (SpeedStor) >0 ubyte 0xa5 386BSD partition type >0 ubyte 0xa6 OpenBSD partition type >0 ubyte 0xa7 NeXTSTEP 486 >0 ubyte 0xa8 Apple UFS >0 ubyte 0xa9 NetBSD partition type >0 ubyte 0xaa Olivetty Fat12 1.44MB Service part >0 ubyte 0xab Apple Boot >0 ubyte 0xae SHAG OS filesystem >0 ubyte 0xaf Apple HFS >0 ubyte 0xb0 BootStar Dummy >0 ubyte 0xb1 reserved >0 ubyte 0xb3 reserved >0 ubyte 0xb4 reserved >0 ubyte 0xb6 reserved >0 ubyte 0xb7 BSDI BSD/386 filesystem >0 ubyte 0xb8 BSDI BSD/386 swap >0 ubyte 0xbb Boot Wizard Hidden >0 ubyte 0xbe Solaris 8 partition type >0 ubyte 0xbf Solaris partition type >0 ubyte 0xc0 CTOS >0 ubyte 0xc1 DRDOS/sec (FAT-12) >0 ubyte 0xc2 Hidden Linux >0 ubyte 0xc3 Hidden Linux swap >0 ubyte 0xc4 DRDOS/sec (FAT-16, < 32M) >0 ubyte 0xc5 DRDOS/sec (EXT) >0 ubyte 0xc6 DRDOS/sec (FAT-16, >= 32M) >0 ubyte 0xc7 Syrinx (Cyrnix?) or HPFS disabled >0 ubyte 0xc8 Reserved for DR-DOS 8.0+ >0 ubyte 0xc9 Reserved for DR-DOS 8.0+ >0 ubyte 0xca Reserved for DR-DOS 8.0+ >0 ubyte 0xcb DR-DOS 7.04+ Secured FAT32 CHS >0 ubyte 0xcc DR-DOS 7.04+ Secured FAT32 LBA >0 ubyte 0xcd CTOS Memdump >0 ubyte 0xce DR-DOS 7.04+ FAT16X LBA >0 ubyte 0xcf DR-DOS 7.04+ EXT LBA >0 ubyte 0xd0 REAL/32 secure big partition >0 ubyte 0xd1 Old Multiuser DOS FAT12 >0 ubyte 0xd4 Old Multiuser DOS FAT16 Small >0 ubyte 0xd5 Old Multiuser DOS Extended >0 ubyte 0xd6 Old Multiuser DOS FAT16 Big >0 ubyte 0xd8 CP/M 86 >0 ubyte 0xdb CP/M or Concurrent CP/M >0 ubyte 0xdd Hidden CTOS Memdump >0 ubyte 0xde Dell PowerEdge Server utilities >0 ubyte 0xdf DG/UX virtual disk manager >0 ubyte 0xe0 STMicroelectronics ST AVFS >0 ubyte 0xe1 DOS access or SpeedStor 12-bit >0 ubyte 0xe3 DOS R/O or Storage Dimensions >0 ubyte 0xe4 SpeedStor 16-bit FAT < 1024 cyl. >0 ubyte 0xe5 reserved >0 ubyte 0xe6 reserved >0 ubyte 0xeb BeOS >0 ubyte 0xee GPT Protective MBR >0 ubyte 0xef EFI system partition >0 ubyte 0xf0 Linux PA-RISC boot loader >0 ubyte 0xf1 SpeedStor or Storage Dimensions >0 ubyte 0xf2 DOS 3.3+ Secondary >0 ubyte 0xf3 reserved >0 ubyte 0xf4 SpeedStor large partition >0 ubyte 0xf5 Prologue multi-volumen partition >0 ubyte 0xf6 reserved >0 ubyte 0xf9 pCache: ext2/ext3 persistent cache >0 ubyte 0xfa Bochs x86 emulator >0 ubyte 0xfb VMware File System >0 ubyte 0xfc VMware Swap >0 ubyte 0xfd Linux RAID partition persistent sb >0 ubyte 0xfe LANstep or IBM PS/2 IML >0 ubyte 0xff Xenix Bad Block Table 0 string \366\366\366\366 PC formatted floppy with no filesystem # Sun disk labels # From /usr/include/sun/dklabel.h: 0774 beshort 0xdabe # modified by Joerg Jenderek, because original test # succeeds for Cabinet archive dao360.dl_ with negative blocks >0770 long >0 Sun disk label >>0 string x '%s >>>31 string >\0 \b%s >>>>63 string >\0 \b%s >>>>>95 string >\0 \b%s >>0 string x \b' >>0734 short >0 %d rpm, >>0736 short >0 %d phys cys, >>0740 short >0 %d alts/cyl, >>0746 short >0 %d interleave, >>0750 short >0 %d data cyls, >>0752 short >0 %d alt cyls, >>0754 short >0 %d heads/partition, >>0756 short >0 %d sectors/track, >>0764 long >0 start cyl %d, >>0770 long x %d blocks # Is there a boot block written 1 sector in? >512 belong&077777777 0600407 \b, boot block present # Joerg Jenderek: Smart Boot Manager backup file is 25 (MSDOS) or 41 (LINUX) byte header + first sectors of disk # (http://btmgr.sourceforge.net/docs/user-guide-3.html) 0 string SBMBAKUP_ Smart Boot Manager backup file >9 string x \b, version %-5.5s >>14 string =_ >>>15 string x %-.1s >>>>16 string =_ \b. >>>>>17 string x \b%-.1s >>>>>>18 string =_ \b. >>>>>>>19 string x \b%-.1s >>>22 ubyte 0 >>>>21 ubyte x \b, from drive 0x%x >>>22 ubyte >0 >>>>21 string x \b, from drive %s >>>535 search/17 \x55\xAA >>>>&-512 indirect x \b; contains # updated by Joerg Jenderek at Nov 2012 # DOS Emulator image is 128 byte, null right padded header + harddisc image 0 string DOSEMU\0 >0x27E leshort 0xAA55 #offset is 128 >>19 ubyte 128 >>>(19.b-1) ubyte 0x0 DOS Emulator image >>>>7 ulelong >0 \b, %u heads >>>>11 ulelong >0 \b, %d sectors/track >>>>15 ulelong >0 \b, %d cylinders >>>>128 indirect x \b; contains # added by Joerg Jenderek at Nov 2012 # http://www.thenakedpc.com/articles/v04/08/0408-05.html # Symantec (Peter Norton) Image.dat file consists of variable header, bootrecord, part of FAT and root directory data 0 string PNCIHISK\0 Norton Utilities disc image data # real x86 boot sector with jump instruction >509 search/1026 \x55\xAA\xeb >>&-1 indirect x \b; contains # http://file-extension.net/seeker/file_extension_dat 0 string PNCIUNDO Norton Disk Doctor UnDo file # # DOS/MBR boot sector updated by Joerg Jenderek at Sep 2007,May 2011,2013 # for any allowed sector sizes 30 search/481 \x55\xAA # to display DOS/MBR boot sector (40) before old one (strength=50+21),Syslinux bootloader (71),SYSLINUX MBR (37+36),NetBSD mbr (110),AdvanceMAME mbr (111) # DOS BPB information (70) and after DOS floppy (120) like in previous file version -!:strength +72 +!:strength +65 # for sector sizes < 512 Bytes >11 uleshort <512 >>(11.s-2) uleshort 0xAA55 DOS/MBR boot sector # for sector sizes with 512 or more Bytes >0x1FE leshort 0xAA55 DOS/MBR boot sector + # keep old DOS/MBR boot sector as dummy for mbr and bootloader displaying # only for sector sizes with 512 or more Bytes -0x1FE leshort 0xAA55 +0x1FE leshort 0xAA55 DOS/MBR boot sector # # to display information (50) before DOS BPB (strength=70) and after DOS floppy (120) like in old file version -!:strength +21 ->2 string OSBS \b, OS/BS MBR +!:strength +65 +>2 string OSBS OS/BS MBR # added by Joerg Jenderek at Feb 2013 according to http://thestarman.pcministry.com/asm/mbr/ # and http://en.wikipedia.org/wiki/Master_Boot_Record # test for nearly all MS-DOS Master Boot Record initial program loader (IPL) is now done by # characteristic assembler instructions: xor ax,ax;mov ss,ax;mov sp,7c00 >0 search/2 \x33\xc0\x8e\xd0\xbc\x00\x7c MS-MBR # Microsoft Windows 95A and early ( http://thestarman.pcministry.com/asm/mbr/STDMBR.htm ) # assembler instructions: mov si,sp;push ax;pop es;push ax;pop ds;sti;cld >>8 ubequad 0x8bf45007501ffbfc # http://thestarman.pcministry.com/asm/mbr/200MBR.htm >>>0x16 ubyte 0xF3 \b,DOS 2 >>>>219 regex Author\ -\ Author: # found "David Litton" , "A Pehrsson " >>>>>&0 string x "%s" >>>0x16 ubyte 0xF2 # NEC MS-DOS 3.30 Rev. 3 . See http://thestarman.pcministry.com/asm/mbr/DOS33MBR.htm # assembler instructions: mov di,077c;cmp word ptrl[di],a55a;jnz >>>>0x22 ubequad 0xbf7c07813d5aa575 \b,NEC 3.3 # version MS-DOS 3.30 til MS-Windows 95A (WinVer=4.00.1111) >>>>0x22 default x \b,D0S version 3.3-7.0 # error messages are printed by assembler instructions: mov si,06nn;...;int 10 (0xBEnn06;...) # where nn is string offset varying for different languages # "Invalid partition table" nn=0x8b for english version >>>>>(0x49.b) string Invalid\ partition\ table english >>>>>(0x49.b) string Ung\201ltige\ Partitionstabelle german >>>>>(0x49.b) string Table\ de\ partition\ invalide french >>>>>(0x49.b) string Tabela\ de\ parti\207ao\ inv\240lida portuguese >>>>>(0x49.b) string Tabla\ de\ partici\242n\ no\ v\240lida spanish >>>>>(0x49.b) string Tavola\ delle\ partizioni\ non\ valida italian >>>>>0x49 ubyte >0 at offset 0x%x >>>>>>(0x49.b) string >\0 "%s" # "Error loading operating system" nn=0xa3 for english version # "Fehler beim Laden des Betriebssystems" nn=0xa7 for german version # "Erreur en chargeant syst\212me d'exploitation" nn=0xa7 for french version # "Erro na inicializa\207ao do sistema operacional" nn=0xa7 for portuguese Brazilian version # "Error al cargar sistema operativo" nn=0xa8 for spanish version # "Errore durante il caricamento del sistema operativo" nn=0xae for italian version >>>>>0x74 ubyte >0 at offset 0x%x >>>>>>(0x74.b) string >\0 "%s" # "Missing operating system" nn=0xc2 for english version # "Betriebssystem fehlt" nn=0xcd for german version # "Syst\212me d'exploitation absent" nn=0xd2 for french version # "Sistema operacional nao encontrado" nn=0xd4 for portuguese Brazilian version # "Falta sistema operativo" nn=0xca for spanish version # "Sistema operativo mancante" nn=0xe2 for italian version >>>>>0x79 ubyte >0 at offset 0x%x >>>>>>(0x79.b) string >\0 "%s" # Microsoft Windows 95B to XP (http://thestarman.pcministry.com/asm/mbr/95BMEMBR.htm) # assembler instructions: push ax;pop es;push ax;pop ds;cld;mov si,7c1b >>8 ubequad 0x5007501ffcbe1b7c # assembler instructions: rep;movsb;retf;mov si,07be;mov cl,04 >>>24 ubequad 0xf3a4cbbebe07b104 9M # "Invalid partition table" nn=0x10F for english version # "Ung\201ltige Partitionstabelle" nn=0x10F for german version # "Table de partition erron\202e" nn=0x10F for french version # "\216\257\245\340\240\346\250\256\255\255\240\357 \341\250\341\342\245\254\240 \255\245 \255\240\251\244\245\255\240" nn=0x10F for russian version >>>>(0x3C.b+0x0FF) string Invalid\ partition\ table english >>>>(0x3C.b+0x0FF) string Ung\201ltige\ Partitionstabelle german >>>>(0x3C.b+0x0FF) string Table\ de\ partition\ erron\202e french >>>>(0x3C.b+0x0FF) string \215\245\257\340\240\242\250\253\354\255\240\357\ \342\240\241\253\250\346\240 russian >>>>0x3C ubyte x at offset 0x%x+0xFF >>>>(0x3C.b+0x0FF) string >\0 "%s" # "Error loading operating system" nn=0x127 for english version # "Fehler beim Laden des Betriebssystems" nn=0x12b for german version # "Erreur lors du chargement du syst\212me d'exploitation" nn=0x12a for french version # "\216\350\250\241\252\240 \257\340\250 \247\240\243\340\343\247\252\245 \256\257\245\340\240\346\250\256\255\255\256\251 \341\250\341\342\245\254\353" nn=0x12d for russian version >>>>0xBD ubyte x at offset 0x1%x >>>>(0xBD.b+0x100) string >\0 "%s" # "Missing operating system" nn=0x146 for english version # "Betriebssystem fehlt" nn=0x151 for german version # "Syst\212me d'exploitation manquant" nn=0x15e for french version # "\216\257\245\340\240\346\250\256\255\255\240\357 \341\250\341\342\245\254\240 \255\245 \255\240\251\244\245\255\240" nn=0x156 for russian version >>>>0xA9 ubyte x at offset 0x1%x >>>>(0xA9.b+0x100) string >\0 "%s" # http://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm # assembler instructions: rep;movsb;retf;mov BP,07be;mov cl,04 >>>24 ubequad 0xf3a4cbbdbe07b104 XP # where xxyyzz are lower bits from offsets of error messages varying for different languages >>>>0x1B4 ubelong&0x00FFFFFF 0x002c4463 english >>>>0x1B4 ubelong&0x00FFFFFF 0x002c486e german # "Invalid partition table" xx=0x12C for english version # "Ung\201ltige Partitionstabelle" xx=0x12C for german version >>>>0x1b5 ubyte >0 at offset 0x1%x >>>>(0x1b5.b+0x100) string >\0 "%s" # "Error loading operating system" yy=0x144 for english version # "Fehler beim Laden des Betriebssystems" yy=0x148 for german version >>>>0x1b6 ubyte >0 at offset 0x1%x >>>>(0x1b6.b+0x100) string >\0 "%s" # "Missing operating system" zz=0x163 for english version # "Betriebssystem nicht vorhanden" zz=0x16e for german version >>>>0x1b7 ubyte >0 at offset 0x1%x >>>>(0x1b7.b+0x100) string >\0 "%s" # Microsoft Windows Vista or 7 # assembler instructions: ..;mov ds,ax;mov si,7c00;mov di,..00 >>8 ubequad 0xc08ed8be007cbf00 # Microsoft Windows Vista (http://thestarman.pcministry.com/asm/mbr/VistaMBR.htm) # assembler instructions: jnz 0729;cmp ebx,"TCPA" >>>0xEC ubequad 0x753b6681fb544350 Vista # where xxyyzz are lower bits from offsets of error messages varying for different languages >>>>0x1B4 ubelong&0x00FFFFFF 0x00627a99 english #>>>>0x1B4 ubelong&0x00FFFFFF ? german # "Invalid partition table" xx=0x162 for english version # "Ung\201ltige Partitionstabelle" xx=0x1?? for german version >>>>0x1b5 ubyte >0 at offset 0x1%x >>>>(0x1b5.b+0x100) string >\0 "%s" # "Error loading operating system" yy=0x17a for english version # "Fehler beim Laden des Betriebssystems" yy= 0x1?? for german version >>>>0x1b6 ubyte >0 at offset 0x1%x >>>>(0x1b6.b+0x100) string >\0 "%s" # "Missing operating system" zz=0x199 for english version # "Betriebssystem nicht vorhanden" zz=0x1?? for german version >>>>0x1b7 ubyte >0 at offset 0x1%x >>>>(0x1b7.b+0x100) string >\0 "%s" # Microsoft Windows 7 (http://thestarman.pcministry.com/asm/mbr/W7MBR.htm) # assembler instructions: cmp ebx,"TCPA";cmp >>>0xEC ubequad 0x6681fb5443504175 Windows 7 # where xxyyzz are lower bits from offsets of error messages varying for different languages >>>>0x1B4 ubelong&0x00FFFFFF 0x00637b9a english #>>>>0x1B4 ubelong&0x00FFFFFF ? german # "Invalid partition table" xx=0x163 for english version # "Ung\201ltige Partitionstabelle" xx=0x1?? for german version >>>>0x1b5 ubyte >0 at offset 0x1%x >>>>(0x1b5.b+0x100) string >\0 "%s" # "Error loading operating system" yy=0x17b for english version # "Fehler beim Laden des Betriebssystems" yy=0x1?? for german version >>>>0x1b6 ubyte >0 at offset 0x1%x >>>>(0x1b6.b+0x100) string >\0 "%s" # "Missing operating system" zz=0x19a for english version # "Betriebssystem nicht vorhanden" zz=0x1?? for german version >>>>0x1b7 ubyte >0 at offset 0x1%x >>>>(0x1b7.b+0x100) string >\0 "%s" # http://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm#DiskSigs # http://en.wikipedia.org/wiki/MBR_disk_signature#ID >>0x1b8 ulelong >0 \b, disk signature 0x%-.4x # driveID/timestamp for Win 95B,98,98SE and ME. See http://thestarman.pcministry.com/asm/mbr/mystery.htm >>0xDA uleshort 0 >>>0xDC ulelong >0 \b, created # physical drive number (0x80-0xFF) when the Windows wrote that byte to the drive >>>>0xDC ubyte x with driveID 0x%x # hours, minutes and seconds >>>>0xDf ubyte x at %x >>>>0xDe ubyte x \b:%x >>>>0xDd ubyte x \b:%x # special case for Microsoft MS-DOS 3.21 spanish # assembler instructions: cli;mov $0x30,%ax;mov %ax,%ss;mov >0 ubequad 0xfab830008ed0bc00 # assembler instructions: $0x1f00,%sp;mov $0x80cb,%di;add %cl,(%bx,%si);in (%dx),%ax;mov >>8 ubequad 0x1fbfcb800008ed8 MS-MBR,D0S version 3.21 spanish # Microsoft MBR IPL end # dr-dos with some upper-, lowercase variants >0x9D string Invalid\ partition\ table$ >>181 string No\ Operating\ System$ >>>201 string Operating\ System\ load\ error$ \b, DR-DOS MBR, Version 7.01 to 7.03 >0x9D string Invalid\ partition\ table$ >>181 string No\ operating\ system$ >>>201 string Operating\ system\ load\ error$ \b, DR-DOS MBR, Version 7.01 to 7.03 >342 string Invalid\ partition\ table$ >>366 string No\ operating\ system$ >>>386 string Operating\ system\ load\ error$ \b, DR-DOS MBR, version 7.01 to 7.03 >295 string NEWLDR\0 >>302 string Bad\ PT\ $ >>>310 string No\ OS\ $ >>>>317 string OS\ load\ err$ >>>>>329 string Moved\ or\ missing\ IBMBIO.LDR\n\r >>>>>>358 string Press\ any\ key\ to\ continue.\n\r$ >>>>>>>387 string Copyright\ (c)\ 1984,1998 >>>>>>>>411 string Caldera\ Inc.\0 \b, DR-DOS MBR (IBMBIO.LDR) # # tests for different MS-DOS Master Boot Records (MBR) moved and merged # #>0x145 string Default:\ F \b, FREE-DOS MBR #>0x14B string Default:\ F \b, FREE-DOS 1.0 MBR >0x145 search/7 Default:\ F \b, FREE-DOS MBR #>>313 string F0\ .\ .\ . #>>>322 string disk\ 1 #>>>>382 string FAT3 >64 string no\ active\ partition\ found >>96 string read\ error\ while\ reading\ drive \b, FREE-DOS Beta 0.9 MBR # Ranish Partition Manager http://www.ranish.com/part/ >387 search/4 \0\ Error!\r >>378 search/7 Virus! >>>397 search/4 Booting\ >>>>408 search/4 HD1/\0 \b, Ranish MBR ( >>>>>416 string Writing\ changes... \b2.37 >>>>>>438 ubyte x \b,0x%x dots >>>>>>440 ubyte >0 \b,virus check >>>>>>441 ubyte >0 \b,partition %c #2.38,2.42,2.44 >>>>>416 string !Writing\ changes... \b >>>>>>418 ubyte 1 \bvirus check, >>>>>>419 ubyte x \b0x%x seconds >>>>>>420 ubyte&0x0F >0 \b,partition >>>>>>>420 ubyte&0x0F <5 \b %x >>>>>>>420 ubyte&0x0F 0Xf \b ask >>>>>420 ubyte x \b) # # SYSLINUX MBR moved # http://www.acronis.de/ >362 string MBR\ Error\ \0\r >>376 string ress\ any\ key\ to\ >>>392 string boot\ from\ floppy...\0 \b, Acronis MBR # added by Joerg Jenderek # http://www.visopsys.org/ # http://partitionlogic.org.uk/ >309 string No\ bootable\ partition\ found\r >>339 string I/O\ Error\ reading\ boot\ sector\r \b, Visopsys MBR >349 string No\ bootable\ partition\ found\r >>379 string I/O\ Error\ reading\ boot\ sector\r \b, simple Visopsys MBR # bootloader, bootmanager >0x40 string SBML # label with 11 characters of FAT 12 bit filesystem >>43 string SMART\ BTMGR >>>430 string SBMK\ Bad!\r \b, Smart Boot Manager # OEM-ID not always "SBM" #>>>>3 strings SBM >>>>6 string >\0 \b, version %s >382 string XOSLLOADXCF \b, eXtended Operating System Loader >6 string LILO \b, LInux i386 boot LOader >>120 string LILO \b, version 22.3.4 SuSe >>172 string LILO \b, version 22.5.8 Debian # updated by Joerg Jenderek at Oct 2008 # variables according to grub-0.97/stage1/stage1.S or # http://www.gnu.org/software/grub/manual/grub.html#Embedded-data # usual values are marked with comments to get only informations of strange GRUB loaders >342 search/60 \0Geom\0 #>0 ulelong x %x=0x009048EB , 0x2a9048EB 0 >>0x41 ubyte <2 >>>0x3E ubyte >2 \b; GRand Unified Bootloader # 0x3 for 0.5.95,0.93,0.94,0.96 0x4 for 1.90 >>>>0x3E ubyte x \b, stage1 version 0x%x #If it is 0xFF, use a drive passed by BIOS >>>>0x40 ubyte <0xFF \b, boot drive 0x%x # in most case 0,1,0x2e for GRUB 0.5.95 >>>>0x41 ubyte >0 \b, LBA flag 0x%x >>>>0x42 uleshort <0x8000 \b, stage2 address 0x%x #>>>>0x42 uleshort =0x8000 \b, stage2 address 0x%x (usual) >>>>0x42 uleshort >0x8000 \b, stage2 address 0x%x #>>>>0x44 ulelong =1 \b, 1st sector stage2 0x%x (default) >>>>0x44 ulelong >1 \b, 1st sector stage2 0x%x >>>>0x48 uleshort <0x800 \b, stage2 segment 0x%x #>>>>0x48 uleshort =0x800 \b, stage2 segment 0x%x (usual) >>>>0x48 uleshort >0x800 \b, stage2 segment 0x%x >>>>402 string Geom\0Hard\ Disk\0Read\0\ Error\0 >>>>>394 string stage1 \b, GRUB version 0.5.95 >>>>382 string Geom\0Hard\ Disk\0Read\0\ Error\0 >>>>>376 string GRUB\ \0 \b, GRUB version 0.93 or 1.94 >>>>383 string Geom\0Hard\ Disk\0Read\0\ Error\0 >>>>>377 string GRUB\ \0 \b, GRUB version 0.94 >>>>385 string Geom\0Hard\ Disk\0Read\0\ Error\0 >>>>>379 string GRUB\ \0 \b, GRUB version 0.95 or 0.96 >>>>391 string Geom\0Hard\ Disk\0Read\0\ Error\0 >>>>>385 string GRUB\ \0 \b, GRUB version 0.97 # unknown version >>>343 string Geom\0Read\0\ Error\0 >>>>321 string Loading\ stage1.5 \b, GRUB version x.y >>>380 string Geom\0Hard\ Disk\0Read\0\ Error\0 >>>>374 string GRUB\ \0 \b, GRUB version n.m # SYSLINUX bootloader moved >395 string chksum\0\ ERROR!\0 \b, Gujin bootloader # http://www.bcdwb.de/bcdw/index_e.htm >3 string BCDL >>498 string BCDL\ \ \ \ BIN \b, Bootable CD Loader (1.50Z) # mbr partition table entries updated by Joerg Jenderek at Sep 2013 # skip Norton Utilities disc image data >3 string !IHISK # skip Linux style boot sector starting with assember instructions mov 0x7c0,ax; >>0 belong !0xb8c0078e # not Linux kernel >>>514 string !HdrS # not BeOS >>>>422 string !Be\ Boot\ Loader +>>>>>32769 string CD001 +>>>>>>0 use cdrom # jump over BPB instruction implies DOS bootsector or AdvanceMAME mbr >>>>>0 ubelong&0xFD000000 =0xE9000000 # AdvanceMAME mbr >>>>>>(1.b+2) ubequad 0xfa31c08ed88ec08e >>>>>>>446 use partition-table # mbr, Norton Utilities disc image data, or 2nd,etc. sector of x86 bootloader >>>>>0 ubelong&0xFD000000 !0xE9000000 # skip FSInfosector >>>>>>0 string !RRaA # skip 3rd sector of MS x86 bootloader with assember instructions cli;MOVZX EAX,BYTE PTR [BP+10];MOV ECX, # http://thestarman.pcministry.com/asm/mbr/MSWIN41.htm >>>>>>>0 ubequad !0xfa660fb64610668b # skip 13rd sector of MS x86 bootloader >>>>>>>>0 ubequad !0x660fb64610668b4e # skip sector starting with DOS new line >>>>>>>>>0 string !\r\n # allowed active flag 0,80h-FFh >>>>>>>>>>446 ubyte 0 >>>>>>>>>>>446 use partition-table >>>>>>>>>>446 ubyte >0x7F >>>>>>>>>>>446 use partition-table # TODO: test for extended bootrecord (ebr) moved and merged with mbr partition table entries # mbr partition table entries end # http://www.acronis.de/ #FAT label=ACRONIS\ SZ #OEM-ID=BOOTWIZ0 >442 string Non-system\ disk,\ >>459 string press\ any\ key...\x7\0 \b, Acronis Startup Recovery Loader # updated by Joerg Jenderek at Nov 2012, Sep 2013 # DOS names like F11.SYS or BOOTWIZ.SYS are 8 right space padded bytes+3 bytes # display 1 space >>>447 ubyte x \b >>>477 use DOS-filename # >185 string FDBOOT\ Version\ >>204 string \rNo\ Systemdisk.\ >>>220 string Booting\ from\ harddisk.\n\r >>>245 string Cannot\ load\ from\ harddisk.\n\r >>>>273 string Insert\ Systemdisk\ >>>>>291 string and\ press\ any\ key.\n\r \b, FDBOOT harddisk Bootloader >>>>>>200 string >\0 \b, version %-3s >242 string Bootsector\ from\ C.H.\ Hochst\204 # http://freecode.com/projects/dosfstools dosfstools-n.m/src/mkdosfs.c # updated by Joerg Jenderek at Nov 2012. Use search directive with offset instead of string # skip name "C.H. Hochstaetter" partly because it is sometimes written without umlaut >242 search/127 Bootsector\ from\ C.H.\ Hochst >>278 search/127 No\ Systemdisk.\ Booting\ from\ harddisk # followed by variants with point,CR-NL or NL-CR >>>208 search/261 Cannot\ load\ from\ harddisk. # followed by variants CR-NL or NL-CR >>>>236 search/235 Insert\ Systemdisk\ and\ press\ any\ key. # followed by variants with point,CR-NL or NL-CR >>>>>180 search/96 Disk\ formatted\ with\ WinImage\ \b, WinImage harddisk Bootloader # followed by string like "6.50 (c) 1993-2004 Gilles Vollant" >>>>>>&0 string x \b, version %-4.4s >(1.b+2) ubyte 0xe >>(1.b+3) ubyte 0x1f >>>(1.b+4) ubyte 0xbe # message offset found at (1.b+5) is 0x77 for FAT32 or 0x5b for others >>>>(1.b+5) ubyte&0xd3 0x53 >>>>>(1.b+6) ubyte 0x7c # assembler instructions: lodsb;and al,al;jz 0xb;push si;mov ah, >>>>>>(1.b+7) ubyte 0xac >>>>>>>(1.b+8) ubyte 0x22 >>>>>>>>(1.b+9) ubyte 0xc0 >>>>>>>>>(1.b+10) ubyte 0x74 >>>>>>>>>>(1.b+11) ubyte 0x0b >>>>>>>>>>>(1.b+12) ubyte 0x56 >>>>>>>>>>>>(1.b+13) ubyte 0xb4 \b, mkdosfs boot message display # FAT1X version >>>>>>>>>>>>>(1.b+5) ubyte 0x5b >>>>>>>>>>>>>>0x5b string >\0 "%-s" # FAT32 version >>>>>>>>>>>>>(1.b+5) ubyte 0x77 >>>>>>>>>>>>>>0x77 string >\0 "%-s" >214 string Please\ try\ to\ install\ FreeDOS\ \b, DOS Emulator boot message display #>>244 string from\ dosemu-freedos-*-bin.tgz\r #>>>170 string Sorry,\ could\ not\ load\ an\ #>>>>195 string operating\ system.\r\n # >103 string This\ is\ not\ a\ bootable\ disk.\ >>132 string Please\ insert\ a\ bootable\ >>>157 string floppy\ and\r\n >>>>169 string press\ any\ key\ to\ try\ again...\r \b, FREE-DOS message display # >66 string Solaris\ Boot\ Sector >>99 string Incomplete\ MDBoot\ load. >>>89 string Version \b, Sun Solaris Bootloader >>>>97 byte x version %c # >408 string OS/2\ !!\ SYS01475\r\0 >>429 string OS/2\ !!\ SYS02025\r\0 >>>450 string OS/2\ !!\ SYS02027\r\0 >>>469 string OS2BOOT\ \ \ \ \b, IBM OS/2 Warp bootloader # >409 string OS/2\ !!\ SYS01475\r\0 >>430 string OS/2\ !!\ SYS02025\r\0 >>>451 string OS/2\ !!\ SYS02027\r\0 >>>470 string OS2BOOT\ \ \ \ \b, IBM OS/2 Warp Bootloader >112 string This\ disk\ is\ not\ bootable\r >>142 string If\ you\ wish\ to\ make\ it\ bootable >>>176 string run\ the\ DOS\ program\ SYS\ >>>200 string after\ the\r >>>>216 string system\ has\ been\ loaded\r\n >>>>>242 string Please\ insert\ a\ DOS\ diskette\ >>>>>271 string into\r\n\ the\ drive\ and\ >>>>>>292 string strike\ any\ key...\0 \b, IBM OS/2 Warp message display # XP >430 string NTLDR\ is\ missing\xFF\r\n >>449 string Disk\ error\xFF\r\n >>>462 string Press\ any\ key\ to\ restart\r \b, Microsoft Windows XP Bootloader # DOS names like NTLDR,CMLDR,$LDR$ are 8 right space padded bytes+3 bytes >>>>417 ubyte&0xDF >0 >>>>>417 string x %-.5s >>>>>>422 ubyte&0xDF >0 >>>>>>>422 string x \b%-.3s >>>>>425 ubyte&0xDF >0 >>>>>>425 string >\ \b.%-.3s # >>>>371 ubyte >0x20 >>>>>368 ubyte&0xDF >0 >>>>>>368 string x %-.5s >>>>>>>373 ubyte&0xDF >0 >>>>>>>>373 string x \b%-.3s >>>>>>376 ubyte&0xDF >0 >>>>>>>376 string x \b.%-.3s # >430 string NTLDR\ nicht\ gefunden\xFF\r\n >>453 string Datentr\204gerfehler\xFF\r\n >>>473 string Neustart\ mit\ beliebiger\ Taste\r \b, Microsoft Windows XP Bootloader (german) >>>>417 ubyte&0xDF >0 >>>>>417 string x %-.5s >>>>>>422 ubyte&0xDF >0 >>>>>>>422 string x \b%-.3s >>>>>425 ubyte&0xDF >0 >>>>>>425 string >\ \b.%-.3s # offset variant >>>>379 string \0 >>>>>368 ubyte&0xDF >0 >>>>>>368 string x %-.5s >>>>>>>373 ubyte&0xDF >0 >>>>>>>>373 string x \b%-.3s # >430 string NTLDR\ fehlt\xFF\r\n >>444 string Datentr\204gerfehler\xFF\r\n >>>464 string Neustart\ mit\ beliebiger\ Taste\r \b, Microsoft Windows XP Bootloader (2.german) >>>>417 ubyte&0xDF >0 >>>>>417 string x %-.5s >>>>>>422 ubyte&0xDF >0 >>>>>>>422 string x \b%-.3s >>>>>425 ubyte&0xDF >0 >>>>>>425 string >\ \b.%-.3s # variant >>>>371 ubyte >0x20 >>>>>368 ubyte&0xDF >0 >>>>>>368 string x %-.5s >>>>>>>373 ubyte&0xDF >0 >>>>>>>>373 string x \b%-.3s >>>>>>376 ubyte&0xDF >0 >>>>>>>376 string x \b.%-.3s # >430 string NTLDR\ fehlt\xFF\r\n >>444 string Medienfehler\xFF\r\n >>>459 string Neustart:\ Taste\ dr\201cken\r \b, Microsoft Windows XP Bootloader (3.german) >>>>371 ubyte >0x20 >>>>>368 ubyte&0xDF >0 >>>>>>368 string x %-.5s >>>>>>>373 ubyte&0xDF >0 >>>>>>>>373 string x \b%-.3s >>>>>>376 ubyte&0xDF >0 >>>>>>>376 string x \b.%-.3s # variant >>>>417 ubyte&0xDF >0 >>>>>417 string x %-.5s >>>>>>422 ubyte&0xDF >0 >>>>>>>422 string x \b%-.3s >>>>>425 ubyte&0xDF >0 >>>>>>425 string >\ \b.%-.3s # >430 string Datentr\204ger\ entfernen\xFF\r\n >>454 string Medienfehler\xFF\r\n >>>469 string Neustart:\ Taste\ dr\201cken\r \b, Microsoft Windows XP Bootloader (4.german) >>>>379 string \0 >>>>>368 ubyte&0xDF >0 >>>>>>368 string x %-.5s >>>>>>>373 ubyte&0xDF >0 >>>>>>>>373 string x \b%-.3s >>>>>>376 ubyte&0xDF >0 >>>>>>>376 string x \b.%-.3s # variant >>>>417 ubyte&0xDF >0 >>>>>417 string x %-.5s >>>>>>422 ubyte&0xDF >0 >>>>>>>422 string x \b%-.3s >>>>>425 ubyte&0xDF >0 >>>>>>425 string >\ \b.%-.3s # #>3 string NTFS\ \ \ \ >389 string Fehler\ beim\ Lesen\ >>407 string des\ Datentr\204gers >>>426 string NTLDR\ fehlt >>>>440 string NTLDR\ ist\ komprimiert >>>>>464 string Neustart\ mit\ Strg+Alt+Entf\r \b, Microsoft Windows XP Bootloader NTFS (german) #>3 string NTFS\ \ \ \ >313 string A\ disk\ read\ error\ occurred.\r >>345 string A\ kernel\ file\ is\ missing\ >>>370 string from\ the\ disk.\r >>>>484 string NTLDR\ is\ compressed >>>>>429 string Insert\ a\ system\ diskette\ >>>>>>454 string and\ restart\r\nthe\ system.\r \b, Microsoft Windows XP Bootloader NTFS # DOS loader variants different languages,offsets >472 ubyte&0xDF >0 >>389 string Invalid\ system\ disk\xFF\r\n >>>411 string Disk\ I/O\ error >>>>428 string Replace\ the\ disk,\ and\ >>>>>455 string press\ any\ key \b, Microsoft Windows 98 Bootloader #IO.SYS >>>>>>472 ubyte&0xDF >0 >>>>>>>472 string x \b %-.2s >>>>>>>>474 ubyte&0xDF >0 >>>>>>>>>474 string x \b%-.5s >>>>>>>>>>479 ubyte&0xDF >0 >>>>>>>>>>>479 string x \b%-.1s >>>>>>>480 ubyte&0xDF >0 >>>>>>>>480 string x \b.%-.3s #MSDOS.SYS >>>>>>>483 ubyte&0xDF >0 \b+ >>>>>>>>483 string x \b%-.5s >>>>>>>>>488 ubyte&0xDF >0 >>>>>>>>>>488 string x \b%-.3s >>>>>>>>491 ubyte&0xDF >0 >>>>>>>>>491 string x \b.%-.3s # >>390 string Invalid\ system\ disk\xFF\r\n >>>412 string Disk\ I/O\ error\xFF\r\n >>>>429 string Replace\ the\ disk,\ and\ >>>>>451 string then\ press\ any\ key\r \b, Microsoft Windows 98 Bootloader >>388 string Ungueltiges\ System\ \xFF\r\n >>>410 string E/A-Fehler\ \ \ \ \xFF\r\n >>>>427 string Datentraeger\ wechseln\ und\ >>>>>453 string Taste\ druecken\r \b, Microsoft Windows 95/98/ME Bootloader (german) #WINBOOT.SYS only not spaces (0xDF) >>>>>>497 ubyte&0xDF >0 >>>>>>>497 string x %-.5s >>>>>>>>502 ubyte&0xDF >0 >>>>>>>>>502 string x \b%-.1s >>>>>>>>>>503 ubyte&0xDF >0 >>>>>>>>>>>503 string x \b%-.1s >>>>>>>>>>>>504 ubyte&0xDF >0 >>>>>>>>>>>>>504 string x \b%-.1s >>>>>>505 ubyte&0xDF >0 >>>>>>>505 string x \b.%-.3s #IO.SYS >>>>>>472 ubyte&0xDF >0 or >>>>>>>472 string x \b %-.2s >>>>>>>>474 ubyte&0xDF >0 >>>>>>>>>474 string x \b%-.5s >>>>>>>>>>479 ubyte&0xDF >0 >>>>>>>>>>>479 string x \b%-.1s >>>>>>>480 ubyte&0xDF >0 >>>>>>>>480 string x \b.%-.3s #MSDOS.SYS >>>>>>>483 ubyte&0xDF >0 \b+ >>>>>>>>483 string x \b%-.5s >>>>>>>>>488 ubyte&0xDF >0 >>>>>>>>>>488 string x \b%-.3s >>>>>>>>491 ubyte&0xDF >0 >>>>>>>>>491 string x \b.%-.3s # >>390 string Ungueltiges\ System\ \xFF\r\n >>>412 string E/A-Fehler\ \ \ \ \xFF\r\n >>>>429 string Datentraeger\ wechseln\ und\ >>>>>455 string Taste\ druecken\r \b, Microsoft Windows 95/98/ME Bootloader (German) #WINBOOT.SYS only not spaces (0xDF) >>>>>>497 ubyte&0xDF >0 >>>>>>>497 string x %-.7s >>>>>>>>504 ubyte&0xDF >0 >>>>>>>>>504 string x \b%-.1s >>>>>>505 ubyte&0xDF >0 >>>>>>>505 string x \b.%-.3s #IO.SYS >>>>>>472 ubyte&0xDF >0 or >>>>>>>472 string x \b %-.2s >>>>>>>>474 ubyte&0xDF >0 >>>>>>>>>474 string x \b%-.6s >>>>>>>480 ubyte&0xDF >0 >>>>>>>>480 string x \b.%-.3s #MSDOS.SYS >>>>>>>483 ubyte&0xDF >0 \b+ >>>>>>>>483 string x \b%-.5s >>>>>>>>>488 ubyte&0xDF >0 >>>>>>>>>>488 string x \b%-.3s >>>>>>>>491 ubyte&0xDF >0 >>>>>>>>>491 string x \b.%-.3s # >>389 string Ungueltiges\ System\ \xFF\r\n >>>411 string E/A-Fehler\ \ \ \ \xFF\r\n >>>>428 string Datentraeger\ wechseln\ und\ >>>>>454 string Taste\ druecken\r \b, Microsoft Windows 95/98/ME Bootloader (GERMAN) # DOS names like IO.SYS,WINBOOT.SYS,MSDOS.SYS,WINBOOT.INI are 8 right space padded bytes+3 bytes >>>>>>472 string x %-.2s >>>>>>>474 ubyte&0xDF >0 >>>>>>>>474 string x \b%-.5s >>>>>>>>479 ubyte&0xDF >0 >>>>>>>>>479 string x \b%-.1s >>>>>>480 ubyte&0xDF >0 >>>>>>>480 string x \b.%-.3s >>>>>>483 ubyte&0xDF >0 \b+ >>>>>>>483 string x \b%-.5s >>>>>>>488 ubyte&0xDF >0 >>>>>>>>488 string x \b%-.2s >>>>>>>>490 ubyte&0xDF >0 >>>>>>>>>490 string x \b%-.1s >>>>>>>491 ubyte&0xDF >0 >>>>>>>>491 string x \b.%-.3s >479 ubyte&0xDF >0 >>416 string Kein\ System\ oder\ >>>433 string Laufwerksfehler >>>>450 string Wechseln\ und\ Taste\ dr\201cken \b, Microsoft DOS Bootloader (german) #IO.SYS >>>>>479 string x \b %-.2s >>>>>>481 ubyte&0xDF >0 >>>>>>>481 string x \b%-.6s >>>>>487 ubyte&0xDF >0 >>>>>>487 string x \b.%-.3s #MSDOS.SYS >>>>>>490 ubyte&0xDF >0 \b+ >>>>>>>490 string x \b%-.5s >>>>>>>>495 ubyte&0xDF >0 >>>>>>>>>495 string x \b%-.3s >>>>>>>498 ubyte&0xDF >0 >>>>>>>>498 string x \b.%-.3s # >376 search/41 Non-System\ disk\ or\ >>395 search/41 disk\ error\r >>>407 search/41 Replace\ and\ >>>>419 search/41 press\ \b, >>>>419 search/41 strike\ \b, old >>>>426 search/41 any\ key\ when\ ready\r MS or PC-DOS bootloader #449 Disk\ Boot\ failure\r MS 3.21 #466 Boot\ Failure\r MS 3.30 >>>>>468 search/18 \0 #IO.SYS,IBMBIO.COM >>>>>>&0 string x \b %-.2s >>>>>>>&-20 ubyte&0xDF >0 >>>>>>>>&-1 string x \b%-.4s >>>>>>>>>&-16 ubyte&0xDF >0 >>>>>>>>>>&-1 string x \b%-.2s >>>>>>&8 ubyte&0xDF >0 \b. >>>>>>>&-1 string x \b%-.3s #MSDOS.SYS,IBMDOS.COM >>>>>>&11 ubyte&0xDF >0 \b+ >>>>>>>&-1 string x \b%-.5s >>>>>>>>&-6 ubyte&0xDF >0 >>>>>>>>>&-1 string x \b%-.1s >>>>>>>>>>&-5 ubyte&0xDF >0 >>>>>>>>>>>&-1 string x \b%-.2s >>>>>>>&7 ubyte&0xDF >0 \b. >>>>>>>>&-1 string x \b%-.3s >441 string Cannot\ load\ from\ harddisk.\n\r >>469 string Insert\ Systemdisk\ >>>487 string and\ press\ any\ key.\n\r \b, MS (2.11) DOS bootloader #>43 string \224R-LOADER\ \ SYS =label >54 string SYS >>324 string VASKK >>>495 string NEWLDR\0 \b, DR-DOS Bootloader (LOADER.SYS) # >98 string Press\ a\ key\ to\ retry\0\r >>120 string Cannot\ find\ file\ \0\r >>>139 string Disk\ read\ error\0\r >>>>156 string Loading\ ...\0 \b, DR-DOS (3.41) Bootloader #DRBIOS.SYS >>>>>44 ubyte&0xDF >0 >>>>>>44 string x \b %-.6s >>>>>>>50 ubyte&0xDF >0 >>>>>>>>50 string x \b%-.2s >>>>>>52 ubyte&0xDF >0 >>>>>>>52 string x \b.%-.3s # >70 string IBMBIO\ \ COM >>472 string Cannot\ load\ DOS!\ >>>489 string Any\ key\ to\ retry \b, DR-DOS Bootloader >>471 string Cannot\ load\ DOS\ >>487 string press\ key\ to\ retry \b, Open-DOS Bootloader #?? >444 string KERNEL\ \ SYS >>314 string BOOT\ error! \b, FREE-DOS Bootloader >499 string KERNEL\ \ SYS >>305 string BOOT\ err!\0 \b, Free-DOS Bootloader >449 string KERNEL\ \ SYS >>319 string BOOT\ error! \b, FREE-DOS 0.5 Bootloader # >449 string Loading\ FreeDOS >>0x1AF ulelong >0 \b, FREE-DOS 0.95,1.0 Bootloader >>>497 ubyte&0xDF >0 >>>>497 string x \b %-.6s >>>>>503 ubyte&0xDF >0 >>>>>>503 string x \b%-.1s >>>>>>>504 ubyte&0xDF >0 >>>>>>>>504 string x \b%-.1s >>>>505 ubyte&0xDF >0 >>>>>505 string x \b.%-.3s # >331 string Error!.0 \b, FREE-DOS 1.0 bootloader # >125 string Loading\ FreeDOS...\r >>311 string BOOT\ error!\r \b, FREE-DOS bootloader >>>441 ubyte&0xDF >0 >>>>441 string x \b %-.6s >>>>>447 ubyte&0xDF >0 >>>>>>447 string x \b%-.1s >>>>>>>448 ubyte&0xDF >0 >>>>>>>>448 string x \b%-.1s >>>>449 ubyte&0xDF >0 >>>>>449 string x \b.%-.3s >124 string FreeDOS\0 >>331 string \ err\0 \b, FREE-DOS BETa 0.9 Bootloader # DOS names like KERNEL.SYS,KERNEL16.SYS,KERNEL32.SYS,METAKERN.SYS are 8 right space padded bytes+3 bytes >>>497 ubyte&0xDF >0 >>>>497 string x \b %-.6s >>>>>503 ubyte&0xDF >0 >>>>>>503 string x \b%-.1s >>>>>>>504 ubyte&0xDF >0 >>>>>>>>504 string x \b%-.1s >>>>505 ubyte&0xDF >0 >>>>>505 string x \b.%-.3s >>333 string \ err\0 \b, FREE-DOS BEta 0.9 Bootloader >>>497 ubyte&0xDF >0 >>>>497 string x \b %-.6s >>>>>503 ubyte&0xDF >0 >>>>>>503 string x \b%-.1s >>>>>>>504 ubyte&0xDF >0 >>>>>>>>504 string x \b%-.1s >>>>505 ubyte&0xDF >0 >>>>>505 string x \b.%-.3s >>334 string \ err\0 \b, FREE-DOS Beta 0.9 Bootloader >>>497 ubyte&0xDF >0 >>>>497 string x \b %-.6s >>>>>503 ubyte&0xDF >0 >>>>>>503 string x \b%-.1s >>>>>>>504 ubyte&0xDF >0 >>>>>>>>504 string x \b%-.1s >>>>505 ubyte&0xDF >0 >>>>>505 string x \b.%-.3s >336 string Error!\ >>343 string Hit\ a\ key\ to\ reboot. \b, FREE-DOS Beta 0.9sr1 Bootloader >>>497 ubyte&0xDF >0 >>>>497 string x \b %-.6s >>>>>503 ubyte&0xDF >0 >>>>>>503 string x \b%-.1s >>>>>>>504 ubyte&0xDF >0 >>>>>>>>504 string x \b%-.1s >>>>505 ubyte&0xDF >0 >>>>>505 string x \b.%-.3s # added by Joerg Jenderek # http://www.visopsys.org/ # http://partitionlogic.org.uk/ # OEM-ID=Visopsys >478 ulelong 0 >>(1.b+326) string I/O\ Error\ reading\ >>>(1.b+344) string Visopsys\ loader\r >>>>(1.b+361) string Press\ any\ key\ to\ continue.\r \b, Visopsys loader # http://alexfru.chat.ru/epm.html#bootprog >494 ubyte >0x4D >>495 string >E >>>495 string >>>3 string BootProg # It just looks for a program file name at the root directory # and loads corresponding file with following execution. # DOS names like STARTUP.BIN,STARTUPC.COM,STARTUPE.EXE are 8 right space padded bytes+3 bytes >>>>499 ubyte&0xDF >0 \b, COM/EXE Bootloader >>>>>499 use DOS-filename #If the boot sector fails to read any other sector, #it prints a very short message ("RE") to the screen and hangs the computer. #If the boot sector fails to find needed program in the root directory, #it also hangs with another message ("NF"). >>>>>492 string RENF \b, FAT (12 bit) >>>>>495 string RENF \b, FAT (16 bit) #If the boot sector fails to read any other sector, #it prints a very short message ("RE") to the screen and hangs the computer. # x86 bootloader end # added by Joerg Jenderek at Feb 2013 according to http://thestarman.pcministry.com/asm/mbr/MSWIN41.htm#FSINFO # and http://en.wikipedia.org/wiki/File_Allocation_Table#FS_Information_Sector >0 string RRaA >>0x1E4 string rrAa \b, FSInfosector #>>0x1FC uleshort =0 SHOULD BE ZERO >>>0x1E8 ulelong <0xffffffff \b, %u free clusters >>>0x1EC ulelong <0xffffffff \b, last allocated cluster %u # updated by Joerg Jenderek at Sep 2007 >3 ubyte 0 #no active flag >>446 ubyte 0 # partition 1 not empty >>>450 ubyte >0 # partitions 3,4 empty >>>>482 ubyte 0 >>>>>498 ubyte 0 # partition 2 ID=0,5,15 >>>>>>466 ubyte <0x10 >>>>>>>466 ubyte 0x05 \b, extended partition table >>>>>>>466 ubyte 0x0F \b, extended partition table (LBA) ->>>>>>>466 ubyte 0x0 \b, extended partition table (last) +>>>>>>>466 ubyte 0x0 \b, extended partition table (last) # DOS x86 sector separated and moved from "DOS/MBR boot sector" by Joerg Jenderek at May 2011 >0x200 lelong 0x82564557 \b, BSD disklabel # by Joerg Jenderek at Apr 2013 # Print the DOS filenames from directory entry form with 8 right space padded bytes + 3 bytes for extension # like IO.SYS. MSDOS.SYS , KERNEL.SYS , DRBIO.SYS 0 name DOS-filename # space=0x20 (00100000b) means empty >0 ubyte&0xDF >0 >>0 ubyte x \b%c >>>1 ubyte&0xDF >0 >>>>1 ubyte x \b%c >>>>>2 ubyte&0xDF >0 >>>>>>2 ubyte x \b%c >>>>>>>3 ubyte&0xDF >0 >>>>>>>>3 ubyte x \b%c >>>>>>>>>4 ubyte&0xDF >0 >>>>>>>>>>4 ubyte x \b%c >>>>>>>>>>>5 ubyte&0xDF >0 >>>>>>>>>>>>5 ubyte x \b%c >>>>>>>>>>>>>6 ubyte&0xDF >0 >>>>>>>>>>>>>>6 ubyte x \b%c >>>>>>>>>>>>>>>7 ubyte&0xDF >0 >>>>>>>>>>>>>>>>7 ubyte x \b%c # DOS filename extension >>8 ubyte&0xDF >0 \b. >>>8 ubyte x \b%c >>>>9 ubyte&0xDF >0 >>>>>9 ubyte x \b%c >>>>>>10 ubyte&0xDF >0 >>>>>>>10 ubyte x \b%c # Print 2 following DOS filenames from directory entry form # like IO.SYS+MSDOS.SYS or ibmbio.com+ibmdos.com 0 name 2xDOS-filename # display 1 space >0 ubyte x \b >0 use DOS-filename >11 ubyte x \b+ >11 use DOS-filename # http://en.wikipedia.org/wiki/Master_boot_record#PTE # display standard partition table 0 name partition-table #>0 ubyte x PARTITION-TABLE # test and display 1st til 4th partition table entry >0 use partition-entry-test >16 use partition-entry-test >32 use partition-entry-test >48 use partition-entry-test # test for entry of partition table 0 name partition-entry-test # partition type ID > 0 >4 ubyte >0 # active flag 0 >>0 ubyte 0 >>>0 use partition-entry # active flag 0x80, 0x81, ... >>0 ubyte >0x7F >>>0 use partition-entry # Print entry of partition table 0 name partition-entry # partition type ID > 0 >4 ubyte >0 \b; partition >>64 leshort 0xAA55 1 >>48 leshort 0xAA55 2 >>32 leshort 0xAA55 3 >>16 leshort 0xAA55 4 >>4 ubyte x : ID=0x%x >>0 ubyte&0x80 0x80 \b, active >>0 ubyte >0x80 0x%x >>1 ubyte x \b, start-CHS ( >>1 use partition-chs >>5 ubyte x \b), end-CHS ( >>5 use partition-chs >>8 ulelong x \b), startsector %u >>12 ulelong x \b, %u sectors # Print cylinder,head,sector (CHS) of partition entry 0 name partition-chs # cylinder >1 ubyte x \b0x >1 ubyte&0xC0 0x40 \b1 >1 ubyte&0xC0 0x80 \b2 >1 ubyte&0xC0 0xC0 \b3 >2 ubyte x \b%x # head >0 ubyte x \b,%u # sector >1 ubyte&0x3F x \b,%u # FATX 0 string FATX FATX filesystem data # romfs filesystems - Juan Cespedes 0 string -rom1fs- romfs filesystem, version 1 >8 belong x %d bytes, >16 string x named %s. # netboot image - Juan Cespedes 0 lelong 0x1b031336L Netboot image, >4 lelong&0xFFFFFF00 0 >>4 lelong&0x100 0x000 mode 2 >>4 lelong&0x100 0x100 mode 3 >4 lelong&0xFFFFFF00 !0 unknown mode 0x18b string OS/2 OS/2 Boot Manager # updated by Joerg Jenderek at Oct 2008 and Sep 2012 # http://syslinux.zytor.com/iso.php # tested with versions 1.47,1.48,1.49,1.50,1.62,1.76,2.00,2.10;3.00,3.11,3.31,;3.70,3.71,3.73,3.75,3.80,3.82,3.84,3.86,4.01,4.03 and 4.05 # assembler instructions: cli;jmp 0:7Cyy (yy=0x40,0x5e,0x6c,0x6e,0x77);nop;nop 0 ulequad&0x909000007cc0eafa 0x909000007c40eafa >631 search/689 ISOLINUX\ isolinux Loader >>&0 string x (version %-4.4s) # http://syslinux.zytor.com/pxe.php # assembler instructions: jmp 7C05 0 ulelong 0x007c05ea pxelinux loader (version 2.13 or older) # assembler instructions: pushfd;pushad 0 ulelong 0x60669c66 pxelinux loader # assembler instructions: jmp 05 0 ulelong 0xc00005ea pxelinux loader (version 3.70 or newer) # http://syslinux.zytor.com/wiki/index.php/SYSLINUX 0 string LDLINUX\ SYS\ SYSLINUX loader >12 string x (older version %-4.4s) 0 string \r\nSYSLINUX\ SYSLINUX loader >11 string x (version %-4.4s) # syslinux updated and separated from "DOS/MBR boot sector" by Joerg Jenderek at Sep 2012 # assembler instructions: jmp yy (yy=0x3c,0x58);nop;"SYSLINUX" 0 ulelong&0x80909bEB 0x009018EB # OEM-ID not always "SYSLINUX" >434 search/47 Boot\ failed # followed by \r\n\0 or :\ >>482 search/132 \0LDLINUX\ SYS Syslinux bootloader (version 2.13 or older) >>1 ubyte 0x58 Syslinux bootloader (version 3.0-3.9) >459 search/30 Boot\ error\r\n\0 >>1 ubyte 0x58 Syslinux bootloader (version 3.10 or newer) # SYSLINUX MBR updated and separated from "DOS/MBR boot sector" by Joerg Jenderek at Sep 2012 # assembler instructions: mov di,0600h;mov cx,0100h 16 search/4 \xbf\x00\x06\xb9\x00\x01 # to display SYSLINUX MBR (36) before old DOS/MBR boot sector one with partition table (strength=50+21) !:strength +36 >94 search/249 Missing\ operating\ system # followed by \r for versions older 3.35 , .\r for versions newer 3.52 and point for other # skip Ranish MBR >>408 search/4 HD1/\0 >>408 default x >>>250 search/118 \0Operating\ system\ load SYSLINUX MBR # followed by "ing " or space >>>>292 search/98 error >>>>>&0 string \r (version 3.35 or older) >>>>>&0 string .\r (version 3.52 or newer) >>>>>&0 default x (version 3.36-3.51 ) >368 search/106 \0Disk\ error\ on\ boot\r\n SYSLINUX GPT-MBR >>156 search/10 \0Boot\ partition\ not\ found\r\n >>>270 search/10 \0OS\ not\ bootable\r\n (version 3.86 or older) >>174 search/10 \0Missing\ OS\r\n >>>189 search/10 \0Multiple\ active\ partitions\r\n (version 4.00 or newer) # SYSLINUX END # NetBSD mbr variants (master-boot-code version 1.22) added by Joerg Jenderek at Nov 2012 # assembler instructions: xor ax,ax;mov ax,ss;mov sp,0x7c00;mov ax, 0 ubequad 0x31c08ed0bc007c8e # mbr_bootsel magic before partition table not reliable with small ipl fragments #>444 uleshort 0xb5e1 >0004 uleshort x # ERRorTeXT >>181 search/166 Error\ \0\r\n NetBSD mbr # NT Drive Serial Number http://thestarman.pcministry.com/asm/mbr/Win2kmbr.htm#DS >>>0x1B8 ubelong >0 \b,Serial 0x%-.8x # BOOTSEL definitions contains assembler instructions: int 0x13;pop dx;push dx;push dx >>>0xbb search/71 \xcd\x13\x5a\x52\x52 \b,bootselector # BOOT_EXTENDED definitions contains assembler instructions: # xchg ecx,edx;addl ecx,edx;movw lba_info,si;movb 0x42,ah;pop dx;push dx;int 0x13 >>>0x96 search/1 \x66\x87\xca\x66\x01\xca\x66\x89\x16\x3a\x07\xbe\x32\x07\xb4\x42\x5a\x52\xcd\x13 \b,boot extended # COM_PORT_VAL definitions contains assembler instructions: outb al,dx;add 5,dl;inb %dx;test 0x40,al >>>0x130 search/55 \xee\x80\xc2\x05\xec\xa8\x40 \b,serial IO # not TERSE_ERROR >>>196 search/106 No\ active\ partition\0 >>>>&0 string Disk\ read\ error\0 >>>>>&0 string No\ operating\ system\0 \b,verbose # not NO_CHS definitions contains assembler instructions: pop dx;push dx;movb $8,ah;int0x13 >>>0x7d search/7 \x5a\x52\xb4\x08\xcd\x13 \b,CHS # not NO_LBA_CHECK definitions contains assembler instructions: movw 0x55aa,bx;movb 0x41,ah;pop dx;push dx;int 0x13 >>>0xa4 search/84 \xbb\xaa\x55\xb4\x41\x5a\x52\xcd\x13 \b,LBA-check # assembler instructions: movw nametab,bx >>>0x26 search/21 \xBB\x94\x07 # not NO_BANNER definitions contains assembler instructions: mov banner,si;call message_crlf >>>>&-9 ubequad&0xBE00f0E800febb94 0xBE0000E80000bb94 >>>>>181 search/166 Error\ \0 # "a: disk" , "Fn: diskn" or "NetBSD MBR boot" >>>>>>&3 string x \b,"%s" +>>>446 use partition-table # Andrea Mazzoleni AdvanceCD mbr loader of http://advancemame.sourceforge.net/boot-readme.html # added by Joerg Jenderek at Nov 2012 for versions 1.3 - 1.4 # assembler instructions: jmp short 0x58;nop;ASCII 0 ubequad&0xeb58908000000000 0xeb58900000000000 # assembler instructions: cli;xor ax,ax;mov ds,ax;mov es,ax;mov ss, >(1.b+2) ubequad 0xfa31c08ed88ec08e # Error messages at end of code >>376 string No\ operating\ system\r\n\0 >>>398 string Disk\ error\r\n\0FDD\0HDD\0 >>>>419 string \ EBIOS\r\n\0 AdvanceMAME mbr # Neil Turton mbr loader variant of http://www.chiark.greenend.org.uk/~neilt/mbr/ # added by Joerg Jenderek at Mar 2011 for versions 1.0.0 - 1.1.11 # for 1st version assembler instructions: cld;xor ax,ax;mov DS,ax;MOV ES,AX;mov SI, # or cld;xor ax,ax;mov SS,ax;XOR SP,SP;mov DS, 0 ulequad&0xcE1b40D48EC031FC 0x8E0000D08EC031FC # pointer to the data starting with Neil Turton signature string >(0x1BC.s) string NDTmbr >>&-14 string 1234F\0 Turton mbr ( # parameters also viewed by install-mbr --list >>>(0x1BC.s+7) ubyte x \b%u<= >>>(0x1BC.s+9) ubyte x \bVersion<=%u #>>>(0x1BC.s+8) ubyte x asm_flag_%x >>>(0x1BC.s+8) ubyte&1 1 \b,Y2K-Fix # variant used by testdisk of http://www.cgsecurity.org/wiki/Menu_MBRCode >>>(0x1BC.s+8) ubyte&2 2 \b,TestDisk #0x1~1,..,0x8~4,0x10~F,0x80~A enabled #>>>(0x1BC.s+10) ubyte x \b,flags 0x%x #0x0~1,0x1~2,...,0x3~4,0x4~F,0x7~D default boot #>>>(0x1BC.s+11) ubyte x \b,cfg_def 0x%x # for older versions >>>(0x1BC.s+9) ubyte <2 #>>>>(0x1BC.s+12) ubyte 18 \b,%hhu/18 seconds >>>>(0x1BC.s+12) ubyte !18 \b,%u/18 seconds # floppy A: or B: >>>>(0x1BC.s+13) ubyte <2 \b,floppy 0x%x >>>>(0x1BC.s+13) ubyte >1 # 1st hard disc #>>>>>(0x1BC.s+13) ubyte 0x80 \b,drive 0x%x # not 1st hard disc >>>>>(0x1BC.s+13) ubyte !0x80 \b,drive 0x%x # for version >= 2 maximal timeout can be 65534 >>>(0x1BC.s+9) ubyte >1 #>>>>(0x1BC.s+12) uleshort 18 \b,%u/18 seconds >>>>(0x1BC.s+12) uleshort !18 \b,%u/18 seconds # floppy A: or B: >>>>(0x1BC.s+14) ubyte <2 \b,floppy 0x%x >>>>(0x1BC.s+14) ubyte >1 # 1st hard disc #>>>>>(0x1BC.s+14) ubyte 0x80 \b,drive 0x%x # not 1st hard disc >>>>>(0x1BC.s+14) ubyte !0x80 \b,drive 0x%x >>>0 ubyte x \b) # added by Joerg Jenderek # In the second sector (+0x200) are variables according to grub-0.97/stage2/asm.S or # grub-1.94/kern/i386/pc/startup.S # http://www.gnu.org/software/grub/manual/grub.html#Embedded-data # usual values are marked with comments to get only informations of strange GRUB loaders 0x200 uleshort 0x70EA # found only version 3.{1,2} >0x206 ubeshort >0x0300 # GRUB version (0.5.)95,0.93,0.94,0.96,0.97 > "00" >>0x212 ubyte >0x29 >>>0x213 ubyte >0x29 # not iso9660_stage1_5 #>>>0 ulelong&0x00BE5652 0x00BE5652 >>>>0x213 ubyte >0x29 GRand Unified Bootloader # config_file for stage1_5 is 0xffffffff + default "/boot/grub/stage2" >>>>0x217 ubyte 0xFF stage1_5 >>>>0x217 ubyte <0xFF stage2 >>>>0x206 ubyte x \b version %u >>>>0x207 ubyte x \b.%u # module_size for 1.94 >>>>0x208 ulelong <0xffffff \b, installed partition %u #>>>>0x208 ulelong =0xffffff \b, %lu (default) >>>>0x208 ulelong >0xffffff \b, installed partition %u # GRUB 0.5.95 unofficial >>>>0x20C ulelong&0x2E300000 0x2E300000 # 0=stage2 1=ffs 2=e2fs 3=fat 4=minix 5=reiserfs >>>>>0x20C ubyte x \b, identifier 0x%x #>>>>>0x20D ubyte =0 \b, LBA flag 0x%x (default) >>>>>0x20D ubyte >0 \b, LBA flag 0x%x # GRUB version as string >>>>>0x20E string >\0 \b, GRUB version %-s # for stage1_5 is 0xffffffff + config_file "/boot/grub/stage2" default >>>>>>0x215 ulong 0xffffffff >>>>>>>0x219 string >\0 \b, configuration file %-s >>>>>>0x215 ulong !0xffffffff >>>>>>>0x215 string >\0 \b, configuration file %-s # newer GRUB versions >>>>0x20C ulelong&0x2E300000 !0x2E300000 ##>>>>>0x20C ulelong =0 \b, saved entry %d (usual) >>>>>0x20C ulelong >0 \b, saved entry %d # for 1.94 contains kernel image size # for 0.93,0.94,0.96,0.97 # 0=stage2 1=ffs 2=e2fs 3=fat 4=minix 5=reiserfs 6=vstafs 7=jfs 8=xfs 9=iso9660 a=ufs2 >>>>>0x210 ubyte x \b, identifier 0x%x # The flag for LBA forcing is in most cases 0 #>>>>>0x211 ubyte =0 \b, LBA flag 0x%x (default) >>>>>0x211 ubyte >0 \b, LBA flag 0x%x # GRUB version as string >>>>>0x212 string >\0 \b, GRUB version %-s # for stage1_5 is 0xffffffff + config_file "/boot/grub/stage2" default >>>>>0x217 ulong 0xffffffff >>>>>>0x21b string >\0 \b, configuration file %-s >>>>>0x217 ulong !0xffffffff >>>>>>0x217 string >\0 \b, configuration file %-s # DOS x86 sector updated and separated from "DOS/MBR boot sector" by Joerg Jenderek at May 2011 # JuMP short bootcodeoffset NOP assembler instructions will usually be EB xx 90 # over BIOS parameter block (BPB) # http://thestarman.pcministry.com/asm/2bytejumps.htm#FWD # older drives may use Near JuMP instruction E9 xx xx # minimal short forward jump found 0x29 for bootloaders or 0x0 # maximal short forward jump is 0x7f # OEM-ID is empty or contain readable bytes -0 ulelong&0x804000E9 0x000000E9 +0 ulelong&0x804000E9 0x000000E9 +!:strength +60 # mtools-3.9.8/msdos.h # usual values are marked with comments to get only informations of strange FAT systems # valid sectorsize must be a power of 2 from 32 to 32768 ->11 uleshort&0xf001f 0 +>11 uleshort&0x001f 0 >>11 uleshort <32769 >>>11 uleshort >31 >>>>21 ubyte&0xf0 0xF0 ->>>>>0 ubyte 0xEB +>>>>>0 ubyte 0xEB DOS/MBR boot sector >>>>>>1 ubyte x \b, code offset 0x%x+2 >>>>>0 ubyte 0xE9 ->>>>>>1 uleshort x \b, code offset 0x%x+2 +>>>>>>1 uleshort x \b, code offset 0x%x+3 >>>>>3 string >\0 \b, OEM-ID "%-.8s" #http://mirror.href.com/thestarman/asm/debug/debug2.htm#IHC >>>>>>8 string IHC \b cached by Windows 9M >>>>>11 uleshort >512 \b, Bytes/sector %u #>>>>>11 uleshort =512 \b, Bytes/sector %u=512 (usual) >>>>>11 uleshort <512 \b, Bytes/sector %u >>>>>13 ubyte >1 \b, sectors/cluster %u #>>>>>13 ubyte =1 \b, sectors/cluster %u (usual on Floppies) ->>>>>82 string FAT32 +# for lazy FAT32 implementation like Transcend digital photo frame PF830 +>>>>>82 string/c fat32 >>>>>>14 uleshort !32 \b, reserved sectors %u #>>>>>>14 uleshort =32 \b, reserved sectors %u (usual Fat32) ->>>>>82 string !FAT32 +>>>>>82 string/c !fat32 >>>>>>14 uleshort >1 \b, reserved sectors %u #>>>>>>14 uleshort =1 \b, reserved sectors %u (usual FAT12,FAT16) #>>>>>>14 uleshort 0 \b, reserved sectors %u (usual NTFS) >>>>>16 ubyte >2 \b, FATs %u #>>>>>16 ubyte =2 \b, FATs %u (usual) >>>>>16 ubyte =1 \b, FAT %u >>>>>16 ubyte >0 >>>>>17 uleshort >0 \b, root entries %u #>>>>>17 uleshort =0 \b, root entries %hu=0 (usual Fat32) >>>>>19 uleshort >0 \b, sectors %u (volumes <=32 MB) #>>>>>19 uleshort =0 \b, sectors %hu=0 (usual Fat32) >>>>>21 ubyte >0xF0 \b, Media descriptor 0x%x #>>>>>21 ubyte =0xF0 \b, Media descriptor 0x%x (usual floppy) >>>>>21 ubyte <0xF0 \b, Media descriptor 0x%x >>>>>22 uleshort >0 \b, sectors/FAT %u #>>>>>22 uleshort =0 \b, sectors/FAT %hu=0 (usual Fat32) >>>>>24 uleshort x \b, sectors/track %u >>>>>26 ubyte >2 \b, heads %u #>>>>>26 ubyte =2 \b, heads %u (usual floppy) >>>>>26 ubyte =1 \b, heads %u # valid only for sector sizes with more then 32 Bytes >>>>>11 uleshort >32 -# skip for Digital Research DOS (version 3.41) 1440 kB Bootdisk ->>>>>>38 ubyte !0x70 +# http://en.wikipedia.org/wiki/Design_of_the_FAT_file_system#Extended_BIOS_Parameter_Block +# skip for values 2,2Ah,70h,73h,DFh +# and continue for extended boot signature values 0,28h,29h,80h +>>>>>>38 ubyte&0x56 =0 >>>>>>>28 ulelong >0 \b, hidden sectors %u #>>>>>>>28 ulelong =0 \b, hidden sectors %u (usual floppy) >>>>>>>32 ulelong >0 \b, sectors %u (volumes > 32 MB) #>>>>>>>32 ulelong =0 \b, sectors %u (volumes > 32 MB) # FAT<32 bit specific ->>>>>>>82 string !FAT32 +>>>>>>>82 string/c !fat32 #>>>>>>>>36 ubyte 0x80 \b, physical drive 0x%x=0x80 (usual harddisk) #>>>>>>>>36 ubyte 0 \b, physical drive 0x%x=0 (usual floppy) >>>>>>>>36 ubyte !0x80 >>>>>>>>>36 ubyte !0 \b, physical drive 0x%x +# VGA-copy CRC or +# in Windows NT bit 0 is a dirty flag to request chkdsk at boot time. bit 1 requests surface scan too >>>>>>>>37 ubyte >0 \b, reserved 0x%x #>>>>>>>>37 ubyte =0 \b, reserved 0x%x -# value is 0x80 for NTFS +# extended boot signatur value is 0x80 for NTFS, 0x28 or 0x29 for others >>>>>>>>38 ubyte !0x29 \b, dos < 4.0 BootSector (0x%x) ->>>>>>>>38 ubyte =0x29 +>>>>>>>>38 ubyte&0xFE =0x28 >>>>>>>>>39 ulelong x \b, serial number 0x%x +>>>>>>>>38 ubyte =0x29 >>>>>>>>>43 string >>>>>>>>43 string >NO\ NAME \b, label: "%11.11s" >>>>>>>>>43 string =NO\ NAME \b, unlabeled # there exist some old floppies without word FAT at offset 54 # a word like "FATnm " is only a hint for a FAT size on nm-bits # Normally the number of clusters is calculated by the values of BPP. # if it is small enough FAT is 12 bit, if it is too big enough FAT is 32 bit, # otherwise FAT is 16 bit. # http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/determining-fat-widths.html ->>>>>>54 string FAT \b, FAT ->>>>>>>54 string FAT12 \b (12 bit) ->>>>>>>54 string FAT16 \b (16 bit) +>>>>>82 string/c !fat32 +>>>>>>54 string FAT12 \b, FAT (12 bit) +>>>>>>54 string FAT16 \b, FAT (16 bit) +>>>>>>54 default x +# determinate FAT bit size by media descriptor +# small floppies implies FAT12 +>>>>>>>21 ubyte <0xF0 \b, FAT (12 bit by descriptor) +# with media descriptor F0h floppy or maybe superfloppy with FAT16 +>>>>>>>21 ubyte =0xF0 +# superfloppy (many sectors) implies FAT16 +>>>>>>>>32 ulelong >0xFFFF \b, FAT (16 bit by descriptor+sectors) +# no superfloppy with media descriptor F0h implies FAT12 +>>>>>>>>32 default x \b, FAT (12 bit by descriptor+sectors) +# with media descriptor F8h floppy or hard disc with FAT12 or FAT16 +>>>>>>>21 ubyte =0xF8 +# 360 KiB with media descriptor F8h, 9 sectors per track ,single sided floppy implies FAT12 +>>>>>>>>19 ubequad 0xd002f80300090001 \b, FAT (12 bit by descriptor+geometry) +# hard disc with FAT12 or FAT16 +>>>>>>>>19 default x \b, FAT (1Y bit by descriptor) +# with media descriptor FAh floppy, RAM disc with FAT12 or FAT16 or Tandy hard disc +>>>>>>>21 ubyte =0xFA +# 320 KiB with media descriptor FAh, 8 sectors per track ,single sided floppy implies FAT12 +>>>>>>>>19 ubequad 0x8002fa0200080001 \b, FAT (12 bit by descriptor+geometry) +# RAM disc with FAT12 or FAT16 or Tandy hard disc +>>>>>>>>19 default x \b, FAT (1Y bit by descriptor) +# others are floppy +>>>>>>>21 default x \b, FAT (12 bit by descriptor) # FAT32 bit specific ->>>>>82 string FAT32 \b, FAT (32 bit) +>>>>>82 string/c fat32 \b, FAT (32 bit) >>>>>>36 ulelong x \b, sectors/FAT %u # http://technet.microsoft.com/en-us/library/cc977221.aspx >>>>>>40 uleshort >0 \b, extension flags 0x%x #>>>>>>40 uleshort =0 \b, extension flags %hu >>>>>>42 uleshort >0 \b, fsVersion %u #>>>>>>42 uleshort =0 \b, fsVersion %u (usual) >>>>>>44 ulelong >2 \b, rootdir cluster %u #>>>>>>44 ulelong =2 \b, rootdir cluster %u #>>>>>>44 ulelong =1 \b, rootdir cluster %u >>>>>>48 uleshort >1 \b, infoSector %u #>>>>>>48 uleshort =1 \b, infoSector %u (usual) >>>>>>48 uleshort <1 \b, infoSector %u ->>>>>>50 uleshort >6 \b, Backup boot sector %u +# 0 or 0xFFFF instead of usual 6 means no backup sector +>>>>>>50 uleshort =0xFFFF \b, no Backup boot sector +>>>>>>50 uleshort =0 \b, no Backup boot sector #>>>>>>50 uleshort =6 \b, Backup boot sector %u (usual) ->>>>>>50 uleshort <6 \b, Backup boot sector %u +>>>>>>50 default x +>>>>>>>50 uleshort x \b, Backup boot sector %u # corrected by Joerg Jenderek at Feb 2011 according to http://thestarman.pcministry.com/asm/mbr/MSWIN41.htm#FSINFO >>>>>>52 ulelong >0 \b, reserved1 0x%x >>>>>>56 ulelong >0 \b, reserved2 0x%x >>>>>>60 ulelong >0 \b, reserved3 0x%x # same structure as FAT1X #>>>>>>64 ubyte =0x80 \b, physical drive 0x%x=80 (usual harddisk) #>>>>>>64 ubyte =0 \b, physical drive 0x%x=0 (usual floppy) >>>>>>64 ubyte !0x80 >>>>>>>64 ubyte >0 \b, physical drive 0x%x # in Windows NT bit 0 is a dirty flag to request chkdsk at boot time. bit 1 requests surface scan too >>>>>>65 ubyte >0 \b, reserved 0x%x >>>>>>66 ubyte !0x29 \b, dos < 4.0 BootSector (0x%x) >>>>>>66 ubyte =0x29 >>>>>>>67 ulelong x \b, serial number 0x%x >>>>>>>71 string >>>>>>71 string >NO\ NAME \b, label: "%11.11s" >>>>>>>71 string =NO\ NAME \b, unlabeled # additional tests for floppy image added by Joerg Jenderek # no fixed disk >>>>>21 ubyte !0xF8 # floppy media with 12 bit FAT >>>>>>54 string !FAT16 # test for FAT after bootsector >>>>>>>(11.s) ulelong&0x00ffffF0 0x00ffffF0 \b, followed by FAT # floppy image !:mime application/x-ima # NTFS specific added by Joerg Jenderek at Mar 2011 according to http://thestarman.pcministry.com/asm/mbr/NTFSBR.htm # and http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/bios-parameter-block.html # 0 FATs >>>>>16 ubyte =0 # 0 root entries >>>>>>17 uleshort =0 # 0 DOS sectors >>>>>>>19 uleshort =0 # 0 sectors/FAT # dos < 4.0 BootSector value found is 0x80 #38 ubyte =0x80 \b, dos < 4.0 BootSector (0x%x) >>>>>>>>22 uleshort =0 \b; NTFS >>>>>>>>>24 uleshort >0 \b, sectors/track %u >>>>>>>>>36 ulelong !0x800080 \b, physical drive 0x%x >>>>>>>>>40 ulequad >0 \b, sectors %lld >>>>>>>>>48 ulequad >0 \b, $MFT start cluster %lld >>>>>>>>>56 ulequad >0 \b, $MFTMirror start cluster %lld # Values 0 to 127 represent MFT record sizes of 0 to 127 clusters. # Values 128 to 255 represent MFT record sizes of 2^(256-N) bytes. >>>>>>>>>64 lelong <256 >>>>>>>>>>64 lelong <128 \b, clusters/RecordSegment %d >>>>>>>>>>64 ubyte >127 \b, bytes/RecordSegment 2^(-1*%i) # Values 0 to 127 represent index block sizes of 0 to 127 clusters. # Values 128 to 255 represent index block sizes of 2^(256-N) byte >>>>>>>>>68 ulelong <256 >>>>>>>>>>68 ulelong <128 \b, clusters/index block %d #>>>>>>>>>>68 ulelong >127 \b, bytes/index block 2^(256-%d) >>>>>>>>>>68 ubyte >127 \b, bytes/index block 2^(-1*%i) >>>>>>>>>72 ulequad x \b, serial number 0%llx >>>>>>>>>80 ulelong >0 \b, checksum 0x%x #>>>>>>>>>80 ulelong =0 \b, checksum 0x%x=0 (usual) >>>>>>>>>0x258 ulelong&0x00009090 =0x00009090 >>>>>>>>>>&-92 indirect x \b; contains # For 2nd NTFS sector added by Joerg Jenderek at Jan 2013 # http://thestarman.pcministry.com/asm/mbr/NTFSbrHexEd.htm # unused assembler instructions JMP y2;NOP;NOP 0x056 ulelong&0xFFFF0FFF 0x909002EB # unicode loadername terminated by CTRL-D >(0.s*2) ulelong&0xFFFFFF00 0x00040000 # loadernames are NTLDR,CMLDR,PELDR,$LDR$ or BOOTMGR >>0x002 lestring16 x Microsoft Windows XP/VISTA bootloader %-5.5s >>0x12 string $ >>>0x0c lestring16 x \b%-2.2s ### DOS,NTFS boot sectors end 9564 lelong 0x00011954 Unix Fast File system [v1] (little-endian), >8404 string x last mounted on %s, #>9504 ledate x last checked at %s, >8224 ledate x last written at %s, >8401 byte x clean flag %d, >8228 lelong x number of blocks %d, >8232 lelong x number of data blocks %d, >8236 lelong x number of cylinder groups %d, >8240 lelong x block size %d, >8244 lelong x fragment size %d, >8252 lelong x minimum percentage of free blocks %d, >8256 lelong x rotational delay %dms, >8260 lelong x disk rotational speed %drps, >8320 lelong 0 TIME optimization >8320 lelong 1 SPACE optimization 42332 lelong 0x19540119 Unix Fast File system [v2] (little-endian) >&-1164 string x last mounted on %s, >&-696 string >\0 volume name %s, >&-304 leqldate x last written at %s, >&-1167 byte x clean flag %d, >&-1168 byte x readonly flag %d, >&-296 lequad x number of blocks %lld, >&-288 lequad x number of data blocks %lld, >&-1332 lelong x number of cylinder groups %d, >&-1328 lelong x block size %d, >&-1324 lelong x fragment size %d, >&-180 lelong x average file size %d, >&-176 lelong x average number of files in dir %d, >&-272 lequad x pending blocks to free %lld, >&-264 lelong x pending inodes to free %d, >&-664 lequad x system-wide uuid %0llx, >&-1316 lelong x minimum percentage of free blocks %d, >&-1248 lelong 0 TIME optimization >&-1248 lelong 1 SPACE optimization 66908 lelong 0x19540119 Unix Fast File system [v2] (little-endian) >&-1164 string x last mounted on %s, >&-696 string >\0 volume name %s, >&-304 leqldate x last written at %s, >&-1167 byte x clean flag %d, >&-1168 byte x readonly flag %d, >&-296 lequad x number of blocks %lld, >&-288 lequad x number of data blocks %lld, >&-1332 lelong x number of cylinder groups %d, >&-1328 lelong x block size %d, >&-1324 lelong x fragment size %d, >&-180 lelong x average file size %d, >&-176 lelong x average number of files in dir %d, >&-272 lequad x pending blocks to free %lld, >&-264 lelong x pending inodes to free %d, >&-664 lequad x system-wide uuid %0llx, >&-1316 lelong x minimum percentage of free blocks %d, >&-1248 lelong 0 TIME optimization >&-1248 lelong 1 SPACE optimization 9564 belong 0x00011954 Unix Fast File system [v1] (big-endian), >7168 belong 0x4c41424c Apple UFS Volume >>7186 string x named %s, >>7176 belong x volume label version %d, >>7180 bedate x created on %s, >8404 string x last mounted on %s, #>9504 bedate x last checked at %s, >8224 bedate x last written at %s, >8401 byte x clean flag %d, >8228 belong x number of blocks %d, >8232 belong x number of data blocks %d, >8236 belong x number of cylinder groups %d, >8240 belong x block size %d, >8244 belong x fragment size %d, >8252 belong x minimum percentage of free blocks %d, >8256 belong x rotational delay %dms, >8260 belong x disk rotational speed %drps, >8320 belong 0 TIME optimization >8320 belong 1 SPACE optimization 42332 belong 0x19540119 Unix Fast File system [v2] (big-endian) >&-1164 string x last mounted on %s, >&-696 string >\0 volume name %s, >&-304 beqldate x last written at %s, >&-1167 byte x clean flag %d, >&-1168 byte x readonly flag %d, >&-296 bequad x number of blocks %lld, >&-288 bequad x number of data blocks %lld, >&-1332 belong x number of cylinder groups %d, >&-1328 belong x block size %d, >&-1324 belong x fragment size %d, >&-180 belong x average file size %d, >&-176 belong x average number of files in dir %d, >&-272 bequad x pending blocks to free %lld, >&-264 belong x pending inodes to free %d, >&-664 bequad x system-wide uuid %0llx, >&-1316 belong x minimum percentage of free blocks %d, >&-1248 belong 0 TIME optimization >&-1248 belong 1 SPACE optimization 66908 belong 0x19540119 Unix Fast File system [v2] (big-endian) >&-1164 string x last mounted on %s, >&-696 string >\0 volume name %s, >&-304 beqldate x last written at %s, >&-1167 byte x clean flag %d, >&-1168 byte x readonly flag %d, >&-296 bequad x number of blocks %lld, >&-288 bequad x number of data blocks %lld, >&-1332 belong x number of cylinder groups %d, >&-1328 belong x block size %d, >&-1324 belong x fragment size %d, >&-180 belong x average file size %d, >&-176 belong x average number of files in dir %d, >&-272 bequad x pending blocks to free %lld, >&-264 belong x pending inodes to free %d, >&-664 bequad x system-wide uuid %0llx, >&-1316 belong x minimum percentage of free blocks %d, >&-1248 belong 0 TIME optimization >&-1248 belong 1 SPACE optimization # ext2/ext3 filesystems - Andreas Dilger # ext4 filesystem - Eric Sandeen # volume label and UUID Russell Coker # http://etbe.coker.com.au/2008/07/08/label-vs-uuid-vs-device/ 0x438 leshort 0xEF53 Linux >0x44c lelong x rev %d >0x43e leshort x \b.%d # No journal? ext2 >0x45c lelong ^0x0000004 ext2 filesystem data >>0x43a leshort ^0x0000001 (mounted or unclean) # Has a journal? ext3 or ext4 >0x45c lelong &0x0000004 # and small INCOMPAT? >>0x460 lelong <0x0000040 # and small RO_COMPAT? >>>0x464 lelong <0x0000008 ext3 filesystem data # else large RO_COMPAT? >>>0x464 lelong >0x0000007 ext4 filesystem data # else large INCOMPAT? >>0x460 lelong >0x000003f ext4 filesystem data >0x468 belong x \b, UUID=%08x >0x46c beshort x \b-%04x >0x46e beshort x \b-%04x >0x470 beshort x \b-%04x >0x472 belong x \b-%08x >0x476 beshort x \b%04x >0x478 string >0 \b, volume name "%s" # General flags for any ext* fs >0x460 lelong &0x0000004 (needs journal recovery) >0x43a leshort &0x0000002 (errors) # INCOMPAT flags >0x460 lelong &0x0000001 (compressed) #>0x460 lelong &0x0000002 (filetype) #>0x460 lelong &0x0000010 (meta bg) >0x460 lelong &0x0000040 (extents) >0x460 lelong &0x0000080 (64bit) #>0x460 lelong &0x0000100 (mmp) #>0x460 lelong &0x0000200 (flex bg) # RO_INCOMPAT flags #>0x464 lelong &0x0000001 (sparse super) >0x464 lelong &0x0000002 (large files) >0x464 lelong &0x0000008 (huge files) #>0x464 lelong &0x0000010 (gdt checksum) #>0x464 lelong &0x0000020 (many subdirs) #>0x463 lelong &0x0000040 (extra isize) # Minix filesystems - Juan Cespedes 0x410 leshort 0x137f !:strength / 2 >0x402 beshort < 100 >0x402 beshort > -1 Minix filesystem, V1, %d zones >0x1e string minix \b, bootable 0x410 beshort 0x137f !:strength / 2 >0x402 beshort < 100 >0x402 beshort > -1 Minix filesystem, V1 (big endian), %d zones >0x1e string minix \b, bootable 0x410 leshort 0x138f !:strength / 2 >0x402 beshort < 100 >0x402 beshort > -1 Minix filesystem, V1, 30 char names, %d zones >0x1e string minix \b, bootable 0x410 beshort 0x138f !:strength / 2 >0x402 beshort < 100 >0x402 beshort > -1 Minix filesystem, V1, 30 char names (big endian), %d zones >0x1e string minix \b, bootable 0x410 leshort 0x2468 >0x402 beshort < 100 >>0x402 beshort > -1 Minix filesystem, V2, %d zones >0x1e string minix \b, bootable 0x410 beshort 0x2468 >0x402 beshort < 100 >0x402 beshort > -1 Minix filesystem, V2 (big endian), %d zones >0x1e string minix \b, bootable 0x410 leshort 0x2478 >0x402 beshort < 100 >0x402 beshort > -1 Minix filesystem, V2, 30 char names, %d zones >0x1e string minix \b, bootable 0x410 leshort 0x2478 >0x402 beshort < 100 >0x402 beshort > -1 Minix filesystem, V2, 30 char names, %d zones >0x1e string minix \b, bootable 0x410 beshort 0x2478 >0x402 beshort !0 Minix filesystem, V2, 30 char names (big endian), %d zones >0x1e string minix \b, bootable 0x410 leshort 0x4d5a >0x402 beshort !0 Minix filesystem, V3, %d zones >0x1e string minix \b, bootable # SGI disk labels - Nathan Scott 0 belong 0x0BE5A941 SGI disk label (volume header) # SGI XFS filesystem - Nathan Scott 0 belong 0x58465342 SGI XFS filesystem data >0x4 belong x (blksz %d, >0x68 beshort x inosz %d, >0x64 beshort ^0x2004 v1 dirs) >0x64 beshort &0x2004 v2 dirs) ############################################################################ # Minix-ST kernel floppy 0x800 belong 0x46fc2700 Atari-ST Minix kernel image # http://en.wikipedia.org/wiki/BIOS_parameter_block # floppies with valid BPB and any instruction at beginning >19 string \240\005\371\005\0\011\0\2\0 \b, 720k floppy >19 string \320\002\370\005\0\011\0\1\0 \b, 360k floppy ############################################################################ # Hmmm, is this a better way of detecting _standard_ floppy images ? 19 string \320\002\360\003\0\011\0\1\0 DOS floppy 360k >0x1FE leshort 0xAA55 \b, DOS/MBR hard disk boot sector 19 string \240\005\371\003\0\011\0\2\0 DOS floppy 720k >0x1FE leshort 0xAA55 \b, DOS/MBR hard disk boot sector 19 string \100\013\360\011\0\022\0\2\0 DOS floppy 1440k >0x1FE leshort 0xAA55 \b, DOS/MBR hard disk boot sector 19 string \240\005\371\005\0\011\0\2\0 DOS floppy 720k, IBM >0x1FE leshort 0xAA55 \b, DOS/MBR hard disk boot sector 19 string \100\013\371\005\0\011\0\2\0 DOS floppy 1440k, mkdosfs >0x1FE leshort 0xAA55 \b, DOS/MBR hard disk boot sector 19 string \320\002\370\005\0\011\0\1\0 Atari-ST floppy 360k 19 string \240\005\371\005\0\011\0\2\0 Atari-ST floppy 720k # | | | | | # | | | | heads # | | | sectors/track # | | sectors/FAT # | media descriptor # BPB: sectors # Valid media descriptor bytes for MS-DOS: # # Byte Capacity Media Size and Type # ------------------------------------------------- # # F0 2.88 MB 3.5-inch, 2-sided, 36-sector # F0 1.44 MB 3.5-inch, 2-sided, 18-sector # F9 720K 3.5-inch, 2-sided, 9-sector # F9 1.2 MB 5.25-inch, 2-sided, 15-sector # FD 360K 5.25-inch, 2-sided, 9-sector # FF 320K 5.25-inch, 2-sided, 8-sector # FC 180K 5.25-inch, 1-sided, 9-sector # FE 160K 5.25-inch, 1-sided, 8-sector # FE 250K 8-inch, 1-sided, single-density # FD 500K 8-inch, 2-sided, single-density # FE 1.2 MB 8-inch, 2-sided, double-density # F8 ----- Fixed disk # # FC xxxK Apricot 70x1x9 boot disk. # # Originally a bitmap: # xxxxxxx0 Not two sided # xxxxxxx1 Double sided # xxxxxx0x Not 8 SPT # xxxxxx1x 8 SPT # xxxxx0xx Not Removable drive # xxxxx1xx Removable drive # 11111xxx Must be one. # # But now it's rather random: # 111111xx Low density disk # 00 SS, Not 8 SPT # 01 DS, Not 8 SPT # 10 SS, 8 SPT # 11 DS, 8 SPT # # 11111001 Double density 3 1/2 floppy disk, high density 5 1/4 # 11110000 High density 3 1/2 floppy disk # 11111000 Hard disk any format # # all FAT12 (strength=70) floppies with sectorsize 512 added by Joerg Jenderek at Jun 2013 # http://en.wikipedia.org/wiki/File_Allocation_Table#Exceptions # Too Weak. #512 ubelong&0xE0ffff00 0xE0ffff00 # without valid Media descriptor in place of BPB, cases with are done at other places #>21 ubyte <0xE5 floppy with old FAT filesystem # but valid Media descriptor at begin of FAT #>>512 ubyte =0xed 720k #>>512 ubyte =0xf0 1440k #>>512 ubyte =0xf8 720k #>>512 ubyte =0xf9 1220k #>>512 ubyte =0xfa 320k #>>512 ubyte =0xfb 640k #>>512 ubyte =0xfc 180k # look like an an old DOS directory entry #>>>0xA0E ubequad 0 #>>>>0xA00 ubequad !0 #!:mime application/x-ima #>>512 ubyte =0xfd # look for 2nd FAT at different location to distinguish between 360k and 500k #>>>0x600 ubelong&0xE0ffff00 0xE0ffff00 360k #>>>0x500 ubelong&0xE0ffff00 0xE0ffff00 500k #>>>0xA0E ubequad 0 #!:mime application/x-ima #>>512 ubyte =0xfe #>>>0x400 ubelong&0xE0ffff00 0xE0ffff00 160k #>>>>0x60E ubequad 0 #>>>>>0x600 ubequad !0 #!:mime application/x-ima #>>>0xC00 ubelong&0xE0ffff00 0xE0ffff00 1200k #>>512 ubyte =0xff 320k #>>>0x60E ubequad 0 #>>>>0x600 ubequad !0 #!:mime application/x-ima #>>512 ubyte x \b, Media descriptor 0x%x # without x86 jump instruction #>>0 ulelong&0x804000E9 !0x000000E9 # assembler instructions: CLI;MOV SP,1E7;MOV AX;07c0;MOV #>>>0 ubequad 0xfabce701b8c0078e \b, MS-DOS 1.12 bootloader # IOSYS.COM+MSDOS.COM #>>>>0xc4 use 2xDOS-filename #>>0 ulelong&0x804000E9 =0x000000E9 # only x86 short jump instruction found #>>>0 ubyte =0xEB #>>>>1 ubyte x \b, code offset 0x%x+2 # http://thestarman.pcministry.com/DOS/ibm100/Boot.htm # assembler instructions: CLI;MOV AX,CS;MOV DS,AX;MOV DX,0 #>>>>(1.b+2) ubequad 0xfa8cc88ed8ba0000 \b, PC-DOS 1.0 bootloader # ibmbio.com+ibmdos.com #>>>>>0x176 use DOS-filename #>>>>>0x181 ubyte x \b+ #>>>>>0x182 use DOS-filename # http://thestarman.pcministry.com/DOS/ibm110/Boot.htm # assembler instructions: CLI;MOV AX,CS;MOV DS,AX;XOR DX,DX;MOV #>>>>(1.b+2) ubequad 0xfa8cc88ed833d28e \b, PC-DOS 1.1 bootloader # ibmbio.com+ibmdos.com #>>>>>0x18b use DOS-filename #>>>>>0x196 ubyte x \b+ #>>>>>0x197 use DOS-filename # http://en.wikipedia.org/wiki/Zenith_Data_Systems # assembler instructions: MOV BX,07c0;MOV SS,BX;MOV SP,01c6 #>>>>(1.b+2) ubequad 0xbbc0078ed3bcc601 \b, Zenith Data Systems MS-DOS 1.25 bootloader # IO.SYS+MSDOS.SYS #>>>>>0x20 use 2xDOS-filename # http://en.wikipedia.org/wiki/Corona_Data_Systems # assembler instructions: MOV AX,CS;MOV DS,AX;CLI;MOV SS,AX; #>>>>(1.b+2) ubequad 0x8cc88ed8fa8ed0bc \b, MS-DOS 1.25 bootloader # IO.SYS+MSDOS.SYS #>>>>>0x69 use 2xDOS-filename # assembler instructions: CLI;PUSH CS;POP SS;MOV SP,7c00; #>>>>(1.b+2) ubequad 0xfa0e17bc007cb860 \b, MS-DOS 2.11 bootloader # defect IO.SYS+MSDOS.SYS ? #>>>>>0x162 use 2xDOS-filename -# CDROM Filesystems -# Modified for UDF by gerardo.cacciari@gmail.com -32769 string CD001 # -!:mime application/x-iso9660-image +0 name cdrom >38913 string !NSR0 ISO 9660 CD-ROM filesystem data +!:mime application/x-iso9660-image >38913 string NSR0 UDF filesystem data +!:mime application/x-iso9660-image >>38917 string 1 (version 1.0) >>38917 string 2 (version 1.5) >>38917 string 3 (version 2.0) >>38917 byte >0x33 (unknown version, ID 0x%X) >>38917 byte <0x31 (unknown version, ID 0x%X) # "application id" which appears to be used as a volume label >32808 string/T >\0 '%s' >34816 string \000CD001\001EL\ TORITO\ SPECIFICATION (bootable) 37633 string CD001 ISO 9660 CD-ROM filesystem data (raw 2352 byte sectors) !:mime application/x-iso9660-image 32777 string CDROM High Sierra CD-ROM filesystem data +# CDROM Filesystems +# https://en.wikipedia.org/wiki/ISO_9660 +# Modified for UDF by gerardo.cacciari@gmail.com +32769 string CD001 +# mime line at that position does not work +# to display CD-ROM (70=81-11) after MBR (113=40+72+1), partition-table (71=50+21) and before Apple Driver Map (51) +!:strength -11 +# to display CD-ROM (114=81+33) before MBR (113=40+72+1), partition-table (71=50+21) and Apple Driver Map (51) +# does not work +#!:strength +33 +>0 use cdrom + # .cso files 0 string CISO Compressed ISO CD image # cramfs filesystem - russell@coker.com.au 0 lelong 0x28cd3d45 Linux Compressed ROM File System data, little endian >4 lelong x size %u >8 lelong &1 version #2 >8 lelong &2 sorted_dirs >8 lelong &4 hole_support >32 lelong x CRC 0x%x, >36 lelong x edition %u, >40 lelong x %u blocks, >44 lelong x %u files 0 belong 0x28cd3d45 Linux Compressed ROM File System data, big endian >4 belong x size %u >8 belong &1 version #2 >8 belong &2 sorted_dirs >8 belong &4 hole_support >32 belong x CRC 0x%x, >36 belong x edition %u, >40 belong x %u blocks, >44 belong x %u files # reiserfs - russell@coker.com.au 0x10034 string ReIsErFs ReiserFS V3.5 0x10034 string ReIsEr2Fs ReiserFS V3.6 0x10034 string ReIsEr3Fs ReiserFS V3.6.19 >0x1002c leshort x block size %d >0x10032 leshort &2 (mounted or unclean) >0x10000 lelong x num blocks %d >0x10040 lelong 1 tea hash >0x10040 lelong 2 yura hash >0x10040 lelong 3 r5 hash # JFFS - russell@coker.com.au 0 lelong 0x34383931 Linux Journalled Flash File system, little endian 0 belong 0x34383931 Linux Journalled Flash File system, big endian # EST flat binary format (which isn't, but anyway) # From: Mark Brown 0 string ESTFBINR EST flat binary # Aculab VoIP firmware # From: Mark Brown 0 string VoIP\ Startup\ and Aculab VoIP firmware >35 string x format %s # From: Mark Brown [old] # From: Behan Webster 0 belong 0x27051956 u-boot legacy uImage, >32 string x %s, >28 byte 0 Invalid os/ >28 byte 1 OpenBSD/ >28 byte 2 NetBSD/ >28 byte 3 FreeBSD/ >28 byte 4 4.4BSD/ >28 byte 5 Linux/ >28 byte 6 SVR4/ >28 byte 7 Esix/ >28 byte 8 Solaris/ >28 byte 9 Irix/ >28 byte 10 SCO/ >28 byte 11 Dell/ >28 byte 12 NCR/ >28 byte 13 LynxOS/ >28 byte 14 VxWorks/ >28 byte 15 pSOS/ >28 byte 16 QNX/ >28 byte 17 Firmware/ >28 byte 18 RTEMS/ >28 byte 19 ARTOS/ >28 byte 20 Unity OS/ >28 byte 21 INTEGRITY/ >29 byte 0 \bInvalid CPU, >29 byte 1 \bAlpha, >29 byte 2 \bARM, >29 byte 3 \bIntel x86, >29 byte 4 \bIA64, >29 byte 5 \bMIPS, >29 byte 6 \bMIPS 64-bit, >29 byte 7 \bPowerPC, >29 byte 8 \bIBM S390, >29 byte 9 \bSuperH, >29 byte 10 \bSparc, >29 byte 11 \bSparc 64-bit, >29 byte 12 \bM68K, >29 byte 13 \bNios-32, >29 byte 14 \bMicroBlaze, >29 byte 15 \bNios-II, >29 byte 16 \bBlackfin, >29 byte 17 \bAVR32, >29 byte 18 \bSTMicroelectronics ST200, >30 byte 0 Invalid Image >30 byte 1 Standalone Program >30 byte 2 OS Kernel Image >30 byte 3 RAMDisk Image >30 byte 4 Multi-File Image >30 byte 5 Firmware Image >30 byte 6 Script File >30 byte 7 Filesystem Image (any type) >30 byte 8 Binary Flat Device Tree BLOB >31 byte 0 (Not compressed), >31 byte 1 (gzip), >31 byte 2 (bzip2), >31 byte 3 (lzma), >12 belong x %d bytes, >8 bedate x %s, >16 belong x Load Address: 0x%08X, >20 belong x Entry Point: 0x%08X, >4 belong x Header CRC: 0x%08X, >24 belong x Data CRC: 0x%08X # JFFS2 file system 0 leshort 0x1984 Linux old jffs2 filesystem data little endian 0 leshort 0x1985 Linux jffs2 filesystem data little endian # Squashfs 0 string sqsh Squashfs filesystem, big endian, >28 beshort x version %d. >30 beshort x \b%d, >28 beshort <3 >>8 belong x %d bytes, >28 beshort >2 >>28 beshort <4 >>>63 bequad x %lld bytes, >>28 beshort >3 >>>40 bequad x %lld bytes, #>>67 belong x %d bytes, >4 belong x %d inodes, >28 beshort <2 >>32 beshort x blocksize: %d bytes, >28 beshort >1 >>28 beshort <4 >>>51 belong x blocksize: %d bytes, >>28 beshort >3 >>>12 belong x blocksize: %d bytes, >28 beshort <4 >>39 bedate x created: %s >28 beshort >3 >>8 bedate x created: %s 0 string hsqs Squashfs filesystem, little endian, >28 leshort x version %d. >30 leshort x \b%d, >28 leshort <3 >>8 lelong x %d bytes, >28 leshort >2 >>28 leshort <4 >>>63 lequad x %lld bytes, >>28 leshort >3 >>>40 lequad x %lld bytes, #>>63 lelong x %d bytes, >4 lelong x %d inodes, >28 leshort <2 >>32 leshort x blocksize: %d bytes, >28 leshort >1 >>28 leshort <4 >>>51 lelong x blocksize: %d bytes, >>28 leshort >3 >>>12 lelong x blocksize: %d bytes, >28 leshort <4 >>39 ledate x created: %s >28 leshort >3 >>8 ledate x created: %s # AFS Dump Magic # From: Ty Sarna 0 string \x01\xb3\xa1\x13\x22 AFS Dump >&0 belong x (v%d) >>&0 byte 0x76 >>>&0 belong x Vol %d, >>>>&0 byte 0x6e >>>>>&0 string x %s >>>>>>&1 byte 0x74 >>>>>>>&0 beshort 2 >>>>>>>>&4 bedate x on: %s >>>>>>>>&0 bedate =0 full dump >>>>>>>>&0 bedate !0 incremental since: %s #---------------------------------------------------------- #delta ISO Daniel Novotny (dnovotny@redhat.com) 0 string DISO Delta ISO data +!:strength +50 >4 belong x version %d # VMS backup savesets - gerardo.cacciari@gmail.com # 4 string \x01\x00\x01\x00\x01\x00 >(0.s+16) string \x01\x01 >>&(&0.b+8) byte 0x42 OpenVMS backup saveset data >>>40 lelong x (block size %d, >>>49 string >\0 original name '%s', >>>2 short 1024 VAX generated) >>>2 short 2048 AXP generated) >>>2 short 4096 I64 generated) # Summary: Oracle Clustered Filesystem # Created by: Aaron Botsis 8 string OracleCFS Oracle Clustered Filesystem, >4 long x rev %d >0 long x \b.%d, >560 string x label: %.64s, >136 string x mountpoint: %.128s # Summary: Oracle ASM tagged volume # Created by: Aaron Botsis 32 string ORCLDISK Oracle ASM Volume, >40 string x Disk Name: %0.12s 32 string ORCLCLRD Oracle ASM Volume (cleared), >40 string x Disk Name: %0.12s # Oracle Clustered Filesystem - Aaron Botsis 8 string OracleCFS Oracle Clustered Filesystem, >4 long x rev %d >0 long x \b.%d, >560 string x label: %.64s, >136 string x mountpoint: %.128s # Oracle ASM tagged volume - Aaron Botsis 32 string ORCLDISK Oracle ASM Volume, >40 string x Disk Name: %0.12s 32 string ORCLCLRD Oracle ASM Volume (cleared), >40 string x Disk Name: %0.12s # Compaq/HP RILOE floppy image # From: Dirk Jagdmann 0 string CPQRFBLO Compaq/HP RILOE floppy image #------------------------------------------------------------------------------ # Files-11 On-Disk Structure (File system for various RSX-11 and VMS flavours). # These bits come from LBN 1 (home block) of ODS-1, ODS-2 and ODS-5 volumes, # which is mapped to VBN 2 of [000000]INDEXF.SYS;1 - gerardo.cacciari@gmail.com # 1008 string DECFILE11 Files-11 On-Disk Structure ->525 byte x Level %d >525 byte x (ODS-%d); >1017 string A RSX-11, VAX/VMS or OpenVMS VAX file system; >1017 string B >>525 byte 2 VAX/VMS or OpenVMS file system; >>525 byte 5 OpenVMS Alpha or Itanium file system; >984 string x volume label is '%-12.12s' # From: Thomas Klausner # http://filext.com/file-extension/DAA # describes the daa file format. The magic would be: 0 string DAA\x0\x0\x0\x0\x0 PowerISO Direct-Access-Archive # From Albert Cahalan # really le32 operation,destination,payloadsize (but quite predictable) # 01 00 00 00 00 00 00 c0 00 02 00 00 0 string \1\0\0\0\0\0\0\300\0\2\0\0 Marvell Libertas firmware # From Eric Sandeen # GFS2 0x10000 belong 0x01161970 >0x10018 belong 0x0000051d GFS1 Filesystem >>0x10024 belong x (blocksize %d, >>0x10060 string >\0 lockproto %s) >0x10018 belong 0x00000709 GFS2 Filesystem >>0x10024 belong x (blocksize %d, >>0x10060 string >\0 lockproto %s) # BTRFS 0x10040 string _BHRfS_M BTRFS Filesystem >0x1012b string >\0 (label "%s", >0x10090 lelong x sectorsize %d, >0x10094 lelong x nodesize %d, >0x10098 lelong x leafsize %d) # dvdisaster's .ecc # From: "Nelson A. de Oliveira" 0 string *dvdisaster* dvdisaster error correction file # xfs metadump image # mb_magic XFSM at 0; superblock magic XFSB at 1 << mb_blocklog # but can we do the << ? For now it's always 512 (0x200) anyway. 0 string XFSM >0x200 string XFSB XFS filesystem metadump image # Type: CROM filesystem # From: Werner Fink 0 string CROMFS CROMFS >6 string >\0 \b version %2.2s, >8 ulequad >0 \b block data at %lld, >16 ulequad >0 \b fblock table at %lld, >24 ulequad >0 \b inode table at %lld, >32 ulequad >0 \b root at %lld, >40 ulelong >0 \b fblock size = %d, >44 ulelong >0 \b block size = %d, >48 ulequad >0 \b bytes = %lld # Type: xfs metadump image # From: Daniel Novotny # mb_magic XFSM at 0; superblock magic XFSB at 1 << mb_blocklog # but can we do the << ? For now it's always 512 (0x200) anyway. 0 string XFSM >0x200 string XFSB XFS filesystem metadump image # Type: delta ISO # From: Daniel Novotny 0 string DISO Delta ISO data, >4 belong x version %d # JFS2 (Journaling File System) image. (Old JFS1 has superblock at 0x1000.) # See linux/fs/jfs/jfs_superblock.h for layout; see jfs_filsys.h for flags. # From: Adam Buchbinder 0x8000 string JFS1 # Because it's text-only magic, check a binary value (version) to be sure. # Should always be 2, but mkfs.jfs writes it as 1. Needs to be 2 or 1 to be # mountable. >&0 lelong <3 JFS2 filesystem image # Label is followed by a UUID; we have to limit string length to avoid # appending the UUID in the case of a 16-byte label. >>&144 regex [\x20-\x7E]{1,16} (label "%s") >>&0 lequad x \b, %lld blocks >>&8 lelong x \b, blocksize %d >>&32 lelong&0x00000006 >0 (dirty) >>&36 lelong >0 (compressed) # LFS 0 lelong 0x070162 LFS filesystem image >4 lelong 1 version 1, >>8 lelong x \b blocks %u, >>12 lelong x \b blocks per segment %u, >4 lelong 2 version 2, >>8 lelong x \b fragments %u, >>12 lelong x \b bytes per segment %u, >16 lelong x \b disk blocks %u, >20 lelong x \b block size %u, >24 lelong x \b fragment size %u, >28 lelong x \b fragments per block %u, >32 lelong x \b start for free list %u, >36 lelong x \b number of free blocks %d, >40 lelong x \b number of files %u, >44 lelong x \b blocks available for writing %d, >48 lelong x \b inodes in cache %d, >52 lelong x \b inode file disk address 0x%x, >56 lelong x \b inode file inode number %u, >60 lelong x \b address of last segment written 0x%x, >64 lelong x \b address of next segment to write 0x%x, >68 lelong x \b address of current segment written 0x%x 0 string td\000 floppy image data (TeleDisk, compressed) 0 string TD\000 floppy image data (TeleDisk) 0 string CQ\024 floppy image data (CopyQM, >16 leshort x %d sectors, >18 leshort x %d heads.) 0 string ACT\020Apricot\020disk\020image\032\004 floppy image data (ApriDisk) 0 beshort 0xAA58 floppy image data (IBM SaveDskF, old) 0 beshort 0xAA59 floppy image data (IBM SaveDskF) 0 beshort 0xAA5A floppy image data (IBM SaveDskF, compressed) 0 string \074CPM_Disk\076 disk image data (YAZE) # ReFS # Richard W.M. Jones 0 string \0\0\0ReFS\0 ReFS filesystem image # EFW encase image file format: # Gregoire Passault # http://www.forensicswiki.org/wiki/Encase_image_file_format 0 string EVF\x09\x0d\x0a\xff\x00 EWF/Expert Witness/EnCase image file format # UBIfs # Linux kernel sources: fs/ubifs/ubifs-media.h -0 belong 0x31181006 ->0x16 short 0 UBIfs image ->0x08 lequad x \b, sequence number %llu ->0x10 leshort x \b, length %u ->0x04 lelong x \b, CRC 0x%08x +0 lelong 0x06101831 +>0x16 leshort 0 UBIfs image +>0x08 lequad x \b, sequence number %llu +>0x10 leshort x \b, length %u +>0x04 lelong x \b, CRC 0x%08x -0 belong 0x55424923 ->0x04 short <2 ->0x05 string \0\0\0 ->0x1c string \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 ->0x04 short x UBI image, version %u +0 lelong 0x23494255 +>0x04 leshort <2 +>0x05 string \0\0\0 +>0x1c string \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 +>0x04 leshort x UBI image, version %u + +# NEC PC-88 2D disk image +# From Fabio R. Schmidlin +0x20 ulelong&0xFFFFFEFF 0x2A0 +>0x10 string \0\0\0\0\0\0\0\0\0\0 +>>0x280 string \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 +>>>0x1A ubyte&0xEF 0 +>>>>0x1B ubyte&0x8F 0 +>>>>>0x1B ubyte&70 <0x40 +>>>>>>0x1C ulelong >0x21 +>>>>>>>0 regex [[:print:]]* NEC PC-88 disk image, name=%s +>>>>>>>>0x1B ubyte 0 \b, media=2D +>>>>>>>>0x1B ubyte 0x10 \b, media=2DD +>>>>>>>>0x1B ubyte 0x20 \b, media=2HD +>>>>>>>>0x1B ubyte 0x30 \b, media=1D +>>>>>>>>0x1B ubyte 0x40 \b, media=1DD +>>>>>>>>0x1A ubyte 0x10 \b, write-protected Index: releng/10.1/contrib/file/magic/Magdir/images =================================================================== --- releng/10.1/contrib/file/magic/Magdir/images (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/images (revision 284193) @@ -1,897 +1,1101 @@ #------------------------------------------------------------------------------ -# $File: images,v 1.91 2014/04/30 21:41:02 christos Exp $ +# $File: images,v 1.102 2015/01/02 02:36:35 christos Exp $ # images: file(1) magic for image formats (see also "iff", and "c-lang" for # XPM bitmaps) # # originally from jef@helios.ee.lbl.gov (Jef Poskanzer), # additions by janl@ifi.uio.no as well as others. Jan also suggested # merging several one- and two-line files into here. # # little magic: PCX (first byte is 0x0a) # Targa - matches `povray', `ppmtotga' and `xv' outputs # by Philippe De Muyter # at 2, byte ImgType must be 1, 2, 3, 9, 10 or 11 # at 1, byte CoMapType must be 1 if ImgType is 1 or 9, 0 otherwise # at 3, leshort Index is 0 for povray, ppmtotga and xv outputs # `xv' recognizes only a subset of the following (RGB with pixelsize = 24) # `tgatoppm' recognizes a superset (Index may be anything) 1 belong&0xfff7ffff 0x01010000 Targa image data - Map !:strength + 2 >2 byte&8 8 - RLE >12 leshort >0 %d x >14 leshort >0 %d 1 belong&0xfff7ffff 0x00020000 Targa image data - RGB !:strength + 2 >2 byte&8 8 - RLE >12 leshort >0 %d x >14 leshort >0 %d 1 belong&0xfff7ffff 0x00030000 Targa image data - Mono !:strength + 2 >2 byte&8 8 - RLE >12 leshort >0 %d x >14 leshort >0 %d # PBMPLUS images # The next byte following the magic is always whitespace. # strength is changed to try these patterns before "x86 boot sector" 0 search/1 P1 >3 regex =[0-9]{0,50}\ [0-9]{0,50} Netpbm PBM image text >3 regex =[0-9]{1,50}\ \b, size = %sx >>3 regex =\ [0-9]{1,50} \b%s !:strength + 45 !:mime image/x-portable-bitmap 0 search/1 P2 >3 regex =[0-9]{0,50}\ [0-9]{0,50} Netpbm PGM image text >3 regex =[0-9]{1,50}\ \b, size = %sx >>3 regex =\ [0-9]{1,50} \b%s !:strength + 45 !:mime image/x-portable-greymap 0 search/1 P3 >3 regex =[0-9]{0,50}\ [0-9]{0,50} Netpbm PPM image text >3 regex =[0-9]{1,50}\ \b, size = %sx >>3 regex =\ [0-9]{1,50} \b%s !:strength + 45 !:mime image/x-portable-pixmap 0 string P4 >3 regex =[0-9]{0,50}\ [0-9]{0,50} Netpbm PBM "rawbits" image data >3 regex =[0-9]{1,50}\ \b, size = %sx >>3 regex =\ [0-9]{1,50} \b%s !:strength + 45 !:mime image/x-portable-bitmap 0 string P5 >3 regex =[0-9]{0,50}\ [0-9]{0,50} Netpbm PGM "rawbits" image data >3 regex =[0-9]{1,50}\ \b, size = %sx >>3 regex =\ [0-9]{1,50} \b%s !:strength + 45 !:mime image/x-portable-greymap 0 string P6 >3 regex =[0-9]{0,50}\ [0-9]{0,50} Netpbm PPM "rawbits" image data >3 regex =[0-9]{1,50}\ \b, size = %sx >>3 regex =\ [0-9]{1,50} \b%s !:strength + 45 !:mime image/x-portable-pixmap 0 string P7 Netpbm PAM image file !:mime image/x-portable-pixmap # From: bryanh@giraffe-data.com (Bryan Henderson) 0 string \117\072 Solitaire Image Recorder format >4 string \013 MGI Type 11 >4 string \021 MGI Type 17 0 string .MDA MicroDesign data >21 byte 48 version 2 >21 byte 51 version 3 0 string .MDP MicroDesign page data >21 byte 48 version 2 >21 byte 51 version 3 # NIFF (Navy Interchange File Format, a modification of TIFF) images # [GRR: this *must* go before TIFF] 0 string IIN1 NIFF image data !:mime image/x-niff # Canon RAW version 1 (CRW) files are a type of Canon Image File Format # (CIFF) file. These are apparently all little-endian. # From: Adam Buchbinder # URL: http://www.sno.phy.queensu.ca/~phil/exiftool/canon_raw.html 0 string II\x1a\0\0\0HEAPCCDR Canon CIFF raw image data !:mime image/x-canon-crw >16 leshort x \b, version %d. >14 leshort x \b%d # Canon RAW version 2 (CR2) files are a kind of TIFF with an extra magic # number. Put this above the TIFF test to make sure we detect them. # These are apparently all little-endian. # From: Adam Buchbinder # URL: http://libopenraw.freedesktop.org/wiki/Canon_CR2 0 string II\x2a\0\x10\0\0\0CR Canon CR2 raw image data !:mime image/x-canon-cr2 >10 byte x \b, version %d. >11 byte x \b%d # Tag Image File Format, from Daniel Quinlan (quinlan@yggdrasil.com) # The second word of TIFF files is the TIFF version number, 42, which has # never changed. The TIFF specification recommends testing for it. 0 string MM\x00\x2a TIFF image data, big-endian !:mime image/tiff +>(4.L) use \^tiff_ifd 0 string II\x2a\x00 TIFF image data, little-endian !:mime image/tiff +>(4.l) use tiff_ifd +0 name tiff_ifd +>0 leshort x \b, direntries=%d +>2 use tiff_entry + +0 name tiff_entry +# NewSubFileType +>0 leshort 0xfe +>>12 use tiff_entry +>0 leshort 0x100 +>>4 lelong 1 +>>>12 use tiff_entry +>>>8 leshort x \b, width=%d +>0 leshort 0x101 +>>4 lelong 1 +>>>8 leshort x \b, height=%d +>>>12 use tiff_entry +>0 leshort 0x102 +>>8 leshort x \b, bps=%d +>>12 use tiff_entry +>0 leshort 0x103 +>>4 lelong 1 \b, compression= +>>>8 leshort 1 \bnone +>>>8 leshort 2 \bhuffman +>>>8 leshort 3 \bbi-level group 3 +>>>8 leshort 4 \bbi-level group 4 +>>>8 leshort 5 \bLZW +>>>8 leshort 6 \bJPEG (old) +>>>8 leshort 7 \bJPEG +>>>8 leshort 8 \bdeflate +>>>8 leshort 9 \bJBIG, ITU-T T.85 +>>>8 leshort 0xa \bJBIG, ITU-T T.43 +>>>8 leshort 0x7ffe \bNeXT RLE 2-bit +>>>8 leshort 0x8005 \bPackBits (Macintosh RLE) +>>>8 leshort 0x8029 \bThunderscan RLE +>>>8 leshort 0x807f \bRasterPadding (CT or MP) +>>>8 leshort 0x8080 \bRLE (Line Work) +>>>8 leshort 0x8081 \bRLE (High-Res Cont-Tone) +>>>8 leshort 0x8082 \bRLE (Binary Line Work) +>>>8 leshort 0x80b2 \bDeflate (PKZIP) +>>>8 leshort 0x80b3 \bKodak DCS +>>>8 leshort 0x8765 \bJBIG +>>>8 leshort 0x8798 \bJPEG2000 +>>>8 leshort 0x8799 \bNikon NEF Compressed +>>>8 default x +>>>>8 leshort x \b(unknown 0x%x) +>>>12 use tiff_entry +>0 leshort 0x106 \b, PhotometricIntepretation= +>>8 leshort 0 \bWhiteIsZero +>>8 leshort 1 \bBlackIsZero +>>8 leshort 2 \bRGB +>>8 leshort 3 \bRGB Palette +>>8 leshort 4 \bTransparency Mask +>>8 leshort 5 \bCMYK +>>8 leshort 6 \bYCbCr +>>8 leshort 8 \bCIELab +>>>8 leshort x \b(unknown=0x%x) +>>12 use tiff_entry +# FillOrder +>0 leshort 0x10a +>>4 lelong 1 +>>>12 use tiff_entry +# DocumentName +>0 leshort 0x10d +>>(8.l) string x \b, name=%s +>>>12 use tiff_entry +# ImageDescription +>0 leshort 0x10e +>>(8.l) string x \b, description=%s +>>>12 use tiff_entry +# Make +>0 leshort 0x10f +>>(8.l) string x \b, manufacturer=%s +>>>12 use tiff_entry +# Model +>0 leshort 0x110 +>>(8.l) string x \b, model=%s +>>>12 use tiff_entry +# StripOffsets +>0 leshort 0x111 +>>12 use tiff_entry +# Orientation +>0 leshort 0x112 \b, orientation= +>>8 leshort 1 \bupper-left +>>8 leshort 3 \blower-right +>>8 leshort 6 \bupper-right +>>8 leshort 8 \blower-left +>>8 leshort 9 \bundefined +>>8 default x +>>>8 leshort x \b[*%d*] +>>12 use tiff_entry +# XResolution +>0 leshort 0x11a +>>8 lelong x \b, xresolution=%d +>>12 use tiff_entry +# YResolution +>0 leshort 0x11b +>>8 lelong x \b, yresolution=%d +>>12 use tiff_entry +# ResolutionUnit +>0 leshort 0x128 +>>8 leshort x \b, resolutionunit=%d +>>12 use tiff_entry +# Software +>0 leshort 0x131 +>>(8.l) string x \b, software=%s +>>12 use tiff_entry +# Datetime +>0 leshort 0x132 +>>(8.l) string x \b, datetime=%s +>>12 use tiff_entry +# HostComputer +>0 leshort 0x13c +>>(8.l) string x \b, hostcomputer=%s +>>12 use tiff_entry +# WhitePoint +>0 leshort 0x13e +>>12 use tiff_entry +# PrimaryChromaticities +>0 leshort 0x13f +>>12 use tiff_entry +# YCbCrCoefficients +>0 leshort 0x211 +>>12 use tiff_entry +# YCbCrPositioning +>0 leshort 0x213 +>>12 use tiff_entry +# ReferenceBlackWhite +>0 leshort 0x214 +>>12 use tiff_entry +# Copyright +>0 leshort 0x8298 +>>(8.l) string x \b, copyright=%s +>>12 use tiff_entry +# ExifOffset +>0 leshort 0x8769 +>>12 use tiff_entry +# GPS IFD +>0 leshort 0x8825 \b, GPS-Data +>>12 use tiff_entry + +#>0 leshort x \b, unknown=0x%x +#>>12 use tiff_entry + 0 string MM\x00\x2b Big TIFF image data, big-endian !:mime image/tiff 0 string II\x2b\x00 Big TIFF image data, little-endian !:mime image/tiff # PNG [Portable Network Graphics, or "PNG's Not GIF"] images # (Greg Roelofs, newt@uchicago.edu) # (Albert Cahalan, acahalan@cs.uml.edu) # # 137 P N G \r \n ^Z \n [4-byte length] H E A D [HEAD data] [HEAD crc] ... # 0 string \x89PNG\x0d\x0a\x1a\x0a PNG image data !:mime image/png >16 belong x \b, %d x >20 belong x %d, >24 byte x %d-bit >25 byte 0 grayscale, >25 byte 2 \b/color RGB, >25 byte 3 colormap, >25 byte 4 gray+alpha, >25 byte 6 \b/color RGBA, #>26 byte 0 deflate/32K, >28 byte 0 non-interlaced >28 byte 1 interlaced # possible GIF replacements; none yet released! # (Greg Roelofs, newt@uchicago.edu) # # GRR 950115: this was mine ("Zip GIF"): 0 string GIF94z ZIF image (GIF+deflate alpha) !:mime image/x-unknown # # GRR 950115: this is Jeremy Wohl's Free Graphics Format (better): # 0 string FGF95a FGF image (GIF+deflate beta) !:mime image/x-unknown # # GRR 950115: this is Thomas Boutell's Portable Bitmap Format proposal # (best; not yet implemented): # 0 string PBF PBF image (deflate compression) !:mime image/x-unknown # GIF 0 string GIF8 GIF image data !:mime image/gif !:apple 8BIMGIFf >4 string 7a \b, version 8%s, >4 string 9a \b, version 8%s, >6 leshort >0 %d x >8 leshort >0 %d #>10 byte &0x80 color mapped, #>10 byte&0x07 =0x00 2 colors #>10 byte&0x07 =0x01 4 colors #>10 byte&0x07 =0x02 8 colors #>10 byte&0x07 =0x03 16 colors #>10 byte&0x07 =0x04 32 colors #>10 byte&0x07 =0x05 64 colors #>10 byte&0x07 =0x06 128 colors #>10 byte&0x07 =0x07 256 colors # ITC (CMU WM) raster files. It is essentially a byte-reversed Sun raster, # 1 plane, no encoding. 0 string \361\0\100\273 CMU window manager raster image data >4 lelong >0 %d x >8 lelong >0 %d, >12 lelong >0 %d-bit # Magick Image File Format 0 string id=ImageMagick MIFF image data # Artisan 0 long 1123028772 Artisan image data >4 long 1 \b, rectangular 24-bit >4 long 2 \b, rectangular 8-bit with colormap >4 long 3 \b, rectangular 32-bit (24-bit with matte) # FIG (Facility for Interactive Generation of figures), an object-based format 0 search/1 #FIG FIG image text >5 string x \b, version %.3s # PHIGS 0 string ARF_BEGARF PHIGS clear text archive 0 string @(#)SunPHIGS SunPHIGS # version number follows, in the form m.n >40 string SunBin binary >32 string archive archive # GKS (Graphics Kernel System) 0 string GKSM GKS Metafile >24 string SunGKS \b, SunGKS # CGM image files 0 string BEGMF clear text Computer Graphics Metafile # MGR bitmaps (Michael Haardt, u31b3hs@pool.informatik.rwth-aachen.de) 0 string yz MGR bitmap, modern format, 8-bit aligned 0 string zz MGR bitmap, old format, 1-bit deep, 16-bit aligned 0 string xz MGR bitmap, old format, 1-bit deep, 32-bit aligned 0 string yx MGR bitmap, modern format, squeezed # Fuzzy Bitmap (FBM) images 0 string %bitmap\0 FBM image data >30 long 0x31 \b, mono >30 long 0x33 \b, color # facsimile data 1 string PC\ Research,\ Inc group 3 fax data >29 byte 0 \b, normal resolution (204x98 DPI) >29 byte 1 \b, fine resolution (204x196 DPI) # From: Herbert Rosmanith 0 string Sfff structured fax file # From: Joerg Jenderek # most files with the extension .EPA and some with .BMP 0 string \x11\x06 Award BIOS Logo, 136 x 84 !:mime image/x-award-bioslogo 0 string \x11\x09 Award BIOS Logo, 136 x 126 !:mime image/x-award-bioslogo #0 string \x07\x1f BIOS Logo corrupted? # http://www.blackfiveservices.co.uk/awbmtools.shtml # http://biosgfx.narod.ru/v3/ # http://biosgfx.narod.ru/abr-2/ 0 string AWBM >4 leshort <1981 Award BIOS bitmap !:mime image/x-award-bmp # image width is a multiple of 4 >>4 leshort&0x0003 0 >>>4 leshort x \b, %d >>>6 leshort x x %d >>4 leshort&0x0003 >0 \b, >>>4 leshort&0x0003 =1 >>>>4 leshort x %d+3 >>>4 leshort&0x0003 =2 >>>>4 leshort x %d+2 >>>4 leshort&0x0003 =3 >>>>4 leshort x %d+1 >>>6 leshort x x %d # at offset 8 starts imagedata followed by "RGB " marker # PC bitmaps (OS/2, Windows BMP files) (Greg Roelofs, newt@uchicago.edu) # http://en.wikipedia.org/wiki/BMP_file_format#DIB_header_.\ # 28bitmap_information_header.29 0 string BM >14 leshort 12 PC bitmap, OS/2 1.x format !:mime image/x-ms-bmp >>18 leshort x \b, %d x >>20 leshort x %d >14 leshort 64 PC bitmap, OS/2 2.x format !:mime image/x-ms-bmp >>18 leshort x \b, %d x >>20 leshort x %d >14 leshort 40 PC bitmap, Windows 3.x format !:mime image/x-ms-bmp >>18 lelong x \b, %d x >>22 lelong x %d x >>28 leshort x %d >14 leshort 124 PC bitmap, Windows 98/2000 and newer format !:mime image/x-ms-bmp >>18 lelong x \b, %d x >>22 lelong x %d x >>28 leshort x %d >14 leshort 108 PC bitmap, Windows 95/NT4 and newer format !:mime image/x-ms-bmp >>18 lelong x \b, %d x >>22 lelong x %d x >>28 leshort x %d >14 leshort 128 PC bitmap, Windows NT/2000 format !:mime image/x-ms-bmp >>18 lelong x \b, %d x >>22 lelong x %d x >>28 leshort x %d # Too simple - MPi #0 string IC PC icon data #0 string PI PC pointer image data #0 string CI PC color icon data #0 string CP PC color pointer image data # Conflicts with other entries [BABYL] #0 string BA PC bitmap array data # XPM icons (Greg Roelofs, newt@uchicago.edu) 0 search/1 /*\ XPM\ */ X pixmap image text !:mime image/x-xpmi # Utah Raster Toolkit RLE images (janl@ifi.uio.no) 0 leshort 0xcc52 RLE image data, >6 leshort x %d x >8 leshort x %d >2 leshort >0 \b, lower left corner: %d >4 leshort >0 \b, lower right corner: %d >10 byte&0x1 =0x1 \b, clear first >10 byte&0x2 =0x2 \b, no background >10 byte&0x4 =0x4 \b, alpha channel >10 byte&0x8 =0x8 \b, comment >11 byte >0 \b, %d color channels >12 byte >0 \b, %d bits per pixel >13 byte >0 \b, %d color map channels # image file format (Robert Potter, potter@cs.rochester.edu) 0 string Imagefile\ version- iff image data # this adds the whole header (inc. version number), informative but longish >10 string >\0 %s # Sun raster images, from Daniel Quinlan (quinlan@yggdrasil.com) 0 belong 0x59a66a95 Sun raster image data >4 belong >0 \b, %d x >8 belong >0 %d, >12 belong >0 %d-bit, #>16 belong >0 %d bytes long, >20 belong 0 old format, #>20 belong 1 standard, >20 belong 2 compressed, >20 belong 3 RGB, >20 belong 4 TIFF, >20 belong 5 IFF, >20 belong 0xffff reserved for testing, >24 belong 0 no colormap >24 belong 1 RGB colormap >24 belong 2 raw colormap #>28 belong >0 colormap is %d bytes long # SGI image file format, from Daniel Quinlan (quinlan@yggdrasil.com) # # See # http://reality.sgi.com/grafica/sgiimage.html # 0 beshort 474 SGI image data #>2 byte 0 \b, verbatim >2 byte 1 \b, RLE #>3 byte 1 \b, normal precision >3 byte 2 \b, high precision >4 beshort x \b, %d-D >6 beshort x \b, %d x >8 beshort x %d >10 beshort x \b, %d channel >10 beshort !1 \bs >80 string >0 \b, "%s" 0 string IT01 FIT image data >4 belong x \b, %d x >8 belong x %d x >12 belong x %d # 0 string IT02 FIT image data >4 belong x \b, %d x >8 belong x %d x >12 belong x %d # 2048 string PCD_IPI Kodak Photo CD image pack file >0xe02 byte&0x03 0x00 , landscape mode >0xe02 byte&0x03 0x01 , portrait mode >0xe02 byte&0x03 0x02 , landscape mode >0xe02 byte&0x03 0x03 , portrait mode 0 string PCD_OPA Kodak Photo CD overview pack file # FITS format. Jeff Uphoff # FITS is the Flexible Image Transport System, the de facto standard for # data and image transfer, storage, etc., for the astronomical community. # (FITS floating point formats are big-endian.) 0 string SIMPLE\ \ = FITS image data >109 string 8 \b, 8-bit, character or unsigned binary integer >108 string 16 \b, 16-bit, two's complement binary integer >107 string \ 32 \b, 32-bit, two's complement binary integer >107 string -32 \b, 32-bit, floating point, single precision >107 string -64 \b, 64-bit, floating point, double precision # other images 0 string This\ is\ a\ BitMap\ file Lisp Machine bit-array-file # From SunOS 5.5.1 "/etc/magic" - appeared right before Sun raster image # stuff. # 0 beshort 0x1010 PEX Binary Archive # DICOM medical imaging data 128 string DICM DICOM medical imaging data !:mime application/dicom # XWD - X Window Dump file. # As described in /usr/X11R6/include/X11/XWDFile.h # used by the xwd program. # Bradford Castalia, idaeim, 1/01 # updated by Adam Buchbinder, 2/09 # The following assumes version 7 of the format; the first long is the length # of the header, which is at least 25 4-byte longs, and the one at offset 8 # is a constant which is always either 1 or 2. Offset 12 is the pixmap depth, # which is a maximum of 32. 0 belong >100 >8 belong <3 >>12 belong <33 >>>4 belong 7 XWD X Window Dump image data !:mime image/x-xwindowdump >>>>100 string >\0 \b, "%s" >>>>16 belong x \b, %dx >>>>20 belong x \b%dx >>>>12 belong x \b%d # PDS - Planetary Data System # These files use Parameter Value Language in the header section. # Unfortunately, there is no certain magic, but the following # strings have been found to be most likely. 0 string NJPL1I00 PDS (JPL) image data 2 string NJPL1I PDS (JPL) image data 0 string CCSD3ZF PDS (CCSD) image data 2 string CCSD3Z PDS (CCSD) image data 0 string PDS_ PDS image data 0 string LBLSIZE= PDS (VICAR) image data # pM8x: ATARI STAD compressed bitmap format # # from Oskar Schirmer Feb 2, 2001 # p M 8 5/6 xx yy zz data... # Atari ST STAD bitmap is always 640x400, bytewise runlength compressed. # bytes either run horizontally (pM85) or vertically (pM86). yy is the # most frequent byte, xx and zz are runlength escape codes, where xx is # used for runs of yy. # 0 string pM85 Atari ST STAD bitmap image data (hor) >5 byte 0x00 (white background) >5 byte 0xFF (black background) 0 string pM86 Atari ST STAD bitmap image data (vert) >5 byte 0x00 (white background) >5 byte 0xFF (black background) # Gurkan Sengun , www.linuks.mine.nu # http://www.atarimax.com/jindroush.atari.org/afmtatr.html 0 leshort 0x0296 Atari ATR image # XXX: # This is bad magic 0x5249 == 'RI' conflicts with RIFF and other # magic. # SGI RICE image file #0 beshort 0x5249 RICE image #>2 beshort x v%d #>4 beshort x (%d x #>6 beshort x %d) #>8 beshort 0 8 bit #>8 beshort 1 10 bit #>8 beshort 2 12 bit #>8 beshort 3 13 bit #>10 beshort 0 4:2:2 #>10 beshort 1 4:2:2:4 #>10 beshort 2 4:4:4 #>10 beshort 3 4:4:4:4 #>12 beshort 1 RGB #>12 beshort 2 CCIR601 #>12 beshort 3 RP175 #>12 beshort 4 YUV # PCX image files # From: Dan Fandrich # updated by Joerg Jenderek at Feb 2013 by http://de.wikipedia.org/wiki/PCX # http://web.archive.org/web/20100206055706/http://www.qzx.com/pc-gpe/pcx.txt # GRR: original test was still too general as it catches xbase examples T5.DBT,T6.DBT with 0xa000000 # test for bytes 0x0a,version byte (0,2,3,4,5),compression byte flag(0,1), bit depth (>0) of PCX or T5.DBT,T6.DBT 0 ubelong&0xffF8fe00 0x0a000000 # for PCX bit depth > 0 >3 ubyte >0 # test for valid versions >>1 ubyte <6 >>>1 ubyte !1 PCX !:mime image/x-pcx #!:mime image/pcx >>>>1 ubyte 0 ver. 2.5 image data >>>>1 ubyte 2 ver. 2.8 image data, with palette >>>>1 ubyte 3 ver. 2.8 image data, without palette >>>>1 ubyte 4 for Windows image data >>>>1 ubyte 5 ver. 3.0 image data >>>>4 uleshort x bounding box [%d, >>>>6 uleshort x %d] - >>>>8 uleshort x [%d, >>>>10 uleshort x %d], >>>>65 ubyte >1 %d planes each of >>>>3 ubyte x %d-bit >>>>68 byte 1 colour, >>>>68 byte 2 grayscale, # this should not happen >>>>68 default x image, >>>>12 leshort >0 %d x >>>>>14 uleshort x %d dpi, >>>>2 byte 0 uncompressed >>>>2 byte 1 RLE compressed # Adobe Photoshop # From: Asbjoern Sloth Toennesen 0 string 8BPS Adobe Photoshop Image !:mime image/vnd.adobe.photoshop >4 beshort 2 (PSB) >18 belong x \b, %d x >14 belong x %d, >24 beshort 0 bitmap >24 beshort 1 grayscale >>12 beshort 2 with alpha >24 beshort 2 indexed >24 beshort 3 RGB >>12 beshort 4 \bA >24 beshort 4 CMYK >>12 beshort 5 \bA >24 beshort 7 multichannel >24 beshort 8 duotone >24 beshort 9 lab >12 beshort > 1 >>12 beshort x \b, %dx >12 beshort 1 \b, >22 beshort x %d-bit channel >12 beshort > 1 \bs # XV thumbnail indicator (ThMO) 0 string P7\ 332 XV thumbnail image data # NITF is defined by United States MIL-STD-2500A 0 string NITF National Imagery Transmission Format >25 string >\0 dated %.14s # GEM Image: Version 1, Headerlen 8 (Wolfram Kleff) 0 belong 0x00010008 GEM Image data >12 beshort x %d x >14 beshort x %d, >4 beshort x %d planes, >8 beshort x %d x >10 beshort x %d pixelsize # GEM Metafile (Wolfram Kleff) 0 lelong 0x0018FFFF GEM Metafile data >4 leshort x version %d # # SMJPEG. A custom Motion JPEG format used by Loki Entertainment # Software Torbjorn Andersson . # 0 string \0\nSMJPEG SMJPEG >8 belong x %d.x data # According to the specification you could find any number of _TXT # headers here, but I can't think of any way of handling that. None of # the SMJPEG files I tried it on used this feature. Even if such a # file is encountered the output should still be reasonable. >16 string _SND \b, >>24 beshort >0 %d Hz >>26 byte 8 8-bit >>26 byte 16 16-bit >>28 string NONE uncompressed # >>28 string APCM ADPCM compressed >>27 byte 1 mono >>28 byte 2 stereo # Help! Isn't there any way to avoid writing this part twice? >>32 string _VID \b, # >>>48 string JFIF JPEG >>>40 belong >0 %d frames >>>44 beshort >0 (%d x >>>46 beshort >0 %d) >16 string _VID \b, # >>32 string JFIF JPEG >>24 belong >0 %d frames >>28 beshort >0 (%d x >>30 beshort >0 %d) 0 string Paint\ Shop\ Pro\ Image\ File Paint Shop Pro Image File # "thumbnail file" (icon) # descended from "xv", but in use by other applications as well (Wolfram Kleff) 0 string P7\ 332 XV "thumbnail file" (icon) data # taken from fkiss: ( ?) 0 string KiSS KISS/GS >4 byte 16 color >>5 byte x %d bit >>8 leshort x %d colors >>10 leshort x %d groups >4 byte 32 cell >>5 byte x %d bit >>8 leshort x %d x >>10 leshort x %d >>12 leshort x +%d >>14 leshort x +%d # Webshots (www.webshots.com), by John Harrison 0 string C\253\221g\230\0\0\0 Webshots Desktop .wbz file # Hercules DASD image files # From Jan Jaeger 0 string CKD_P370 Hercules CKD DASD image file >8 long x \b, %d heads per cylinder >12 long x \b, track size %d bytes >16 byte x \b, device type 33%2.2X 0 string CKD_C370 Hercules compressed CKD DASD image file >8 long x \b, %d heads per cylinder >12 long x \b, track size %d bytes >16 byte x \b, device type 33%2.2X 0 string CKD_S370 Hercules CKD DASD shadow file >8 long x \b, %d heads per cylinder >12 long x \b, track size %d bytes >16 byte x \b, device type 33%2.2X # Squeak images and programs - etoffi@softhome.net 0 string \146\031\0\0 Squeak image data 0 search/1 'From\040Squeak Squeak program text # partimage: file(1) magic for PartImage files (experimental, incomplete) # Author: Hans-Joachim Baader 0 string PaRtImAgE-VoLuMe PartImage >0x0020 string 0.6.1 file version %s >>0x0060 lelong >-1 volume %d #>>0x0064 8 byte identifier #>>0x007c reserved >>0x0200 string >\0 type %s >>0x1400 string >\0 device %s, >>0x1600 string >\0 original filename %s, # Some fields omitted >>0x2744 lelong 0 not compressed >>0x2744 lelong 1 gzip compressed >>0x2744 lelong 2 bzip2 compressed >>0x2744 lelong >2 compressed with unknown algorithm >0x0020 string >0.6.1 file version %s >0x0020 string <0.6.1 file version %s # DCX is multi-page PCX, using a simple header of up to 1024 # offsets for the respective PCX components. # From: Joerg Wunsch 0 lelong 987654321 DCX multi-page PCX image data # Simon Walton # Kodak Cineon format for scanned negatives # http://www.kodak.com/US/en/motion/support/dlad/ 0 lelong 0xd75f2a80 Cineon image data >200 belong >0 \b, %d x >204 belong >0 %d # Bio-Rad .PIC is an image format used by microscope control systems # and related image processing software used by biologists. # From: Vebjorn Ljosa # BOOL values are two-byte integers; use them to rule out false positives. # http://web.archive.org/web/20050317223257/www.cs.ubc.ca/spider/ladic/text/biorad.txt # Samples: http://www.loci.wisc.edu/software/sample-data 14 leshort <2 >62 leshort <2 >>54 leshort 12345 Bio-Rad .PIC Image File >>>0 leshort >0 %d x >>>2 leshort >0 %d, >>>4 leshort =1 1 image in file >>>4 leshort >1 %d images in file # From Jan "Yenya" Kasprzak # The description of *.mrw format can be found at # http://www.dalibor.cz/minolta/raw_file_format.htm 0 string \000MRM Minolta Dimage camera raw image data # Summary: DjVu image / document # Extension: .djvu # Reference: http://djvu.org/docs/DjVu3Spec.djvu # Submitted by: Stephane Loeuillet # Modified by (1): Abel Cheung 0 string AT&TFORM >12 string DJVM DjVu multiple page document !:mime image/vnd.djvu >12 string DJVU DjVu image or single page document !:mime image/vnd.djvu >12 string DJVI DjVu shared document !:mime image/vnd.djvu >12 string THUM DjVu page thumbnails !:mime image/vnd.djvu # Originally by Marc Espie # Modified by Robert Minsk # http://www.openexr.com/openexrfilelayout.pdf 0 lelong 20000630 OpenEXR image data, !:mime image/x-exr >4 lelong&0x000000ff x version %d, >4 lelong ^0x00000200 storage: scanline >4 lelong &0x00000200 storage: tiled >8 search/0x1000 compression\0 \b, compression: >>&16 byte 0 none >>&16 byte 1 rle >>&16 byte 2 zips >>&16 byte 3 zip >>&16 byte 4 piz >>&16 byte 5 pxr24 >>&16 byte 6 b44 >>&16 byte 7 b44a >>&16 byte >7 unknown >8 search/0x1000 dataWindow\0 \b, dataWindow: >>&10 lelong x (%d >>&14 lelong x %d)- >>&18 lelong x \b(%d >>&22 lelong x %d) >8 search/0x1000 displayWindow\0 \b, displayWindow: >>&10 lelong x (%d >>&14 lelong x %d)- >>&18 lelong x \b(%d >>&22 lelong x %d) >8 search/0x1000 lineOrder\0 \b, lineOrder: >>&14 byte 0 increasing y >>&14 byte 1 decreasing y >>&14 byte 2 random y >>&14 byte >2 unknown # SMPTE Digital Picture Exchange Format, SMPTE DPX # # ANSI/SMPTE 268M-1994, SMPTE Standard for File Format for Digital # Moving-Picture Exchange (DPX), v1.0, 18 February 1994 # Robert Minsk 0 string SDPX DPX image data, big-endian, !:mime image/x-dpx >768 beshort <4 >>772 belong x %dx >>776 belong x \b%d, >768 beshort >3 >>776 belong x %dx >>772 belong x \b%d, >768 beshort 0 left to right/top to bottom >768 beshort 1 right to left/top to bottom >768 beshort 2 left to right/bottom to top >768 beshort 3 right to left/bottom to top >768 beshort 4 top to bottom/left to right >768 beshort 5 top to bottom/right to left >768 leshort 6 bottom to top/left to right >768 leshort 7 bottom to top/right to left # From: Tom Hilinski # http://www.unidata.ucar.edu/packages/netcdf/ 0 string CDF\001 NetCDF Data Format data #----------------------------------------------------------------------- # Hierarchical Data Format, used to facilitate scientific data exchange # specifications at http://hdf.ncsa.uiuc.edu/ 0 belong 0x0e031301 Hierarchical Data Format (version 4) data !:mime application/x-hdf 0 string \211HDF\r\n\032\n Hierarchical Data Format (version 5) data !:mime application/x-hdf 512 string \211HDF\r\n\032\n Hierarchical Data Format (version 5) with 512 bytes user block !:mime application/x-hdf 1024 string \211HDF\r\n\032\n Hierarchical Data Format (version 5) with 1k user block !:mime application/x-hdf 2048 string \211HDF\r\n\032\n Hierarchical Data Format (version 5) with 2k user block !:mime application/x-hdf 4096 string \211HDF\r\n\032\n Hierarchical Data Format (version 5) with 4k user block !:mime application/x-hdf # From: Tobias Burnus # Xara (for a while: Corel Xara) is a graphic package, see # http://www.xara.com/ for Windows and as GPL application for Linux 0 string XARA\243\243 Xara graphics file # http://www.cartesianinc.com/Tech/ 0 string CPC\262 Cartesian Perceptual Compression image !:mime image/x-cpi # From Albert Cahalan # puredigital used it for the CVS disposable camcorder #8 lelong 4 ZBM bitmap image data #>4 leshort x %u x #>6 leshort x %u # From Albert Cahalan # uncompressed 5:6:5 HighColor image for OLPC XO firmware icons 0 string C565 OLPC firmware icon image data >4 leshort x %u x >6 leshort x %u # Applied Images - Image files from Cytovision # Gustavo Junior Alves 0 string \xce\xda\xde\xfa Cytovision Metaphases file 0 string \xed\xad\xef\xac Cytovision Karyotype file 0 string \x0b\x00\x03\x00 Cytovision FISH Probe file 0 string \xed\xfe\xda\xbe Cytovision FLEX file 0 string \xed\xab\xed\xfe Cytovision FLEX file 0 string \xad\xfd\xea\xad Cytovision RATS file # Wavelet Scalar Quantization format used in gray-scale fingerprint images # From Tano M Fotang 0 string \xff\xa0\xff\xa8\x00 Wavelet Scalar Quantization image data # Type: PCO B16 image files # URL: http://www.pco.de/fileadmin/user_upload/db/download/MA_CWDCOPIE_0412b.pdf # From: Florian Philipp # Extension: .b16 # Description: Pixel image format produced by PCO Camware, typically used # together with PCO cameras. # Note: Different versions exist for e.g. 8 bit and 16 bit images. # Documentation is incomplete. 0 string/b PCO- PCO B16 image data >12 lelong x \b, %dx >16 lelong x \b%d >20 lelong 0 \b, short header >20 lelong -1 \b, extended header >>24 lelong 0 \b, grayscale >>>36 lelong 0 linear LUT >>>36 lelong 1 logarithmic LUT >>>28 lelong x [%d >>>32 lelong x \b,%d] >>24 lelong 1 \b, color >>>64 lelong 0 linear LUT >>>64 lelong 1 logarithmic LUT >>>40 lelong x r[%d >>>44 lelong x \b,%d] >>>48 lelong x g[%d >>>52 lelong x \b,%d] >>>56 lelong x b[%d >>>60 lelong x \b,%d] # Polar Monitor Bitmap (.pmb) used as logo for Polar Electro watches # From: Markus Heidelberg 0 string/t [BitmapInfo2] Polar Monitor Bitmap text !:mime image/x-polar-monitor-bitmap # From: Rick Richardson 0 string GARMIN\ BITMAP\ 01 Garmin Bitmap file # Type: Ulead Photo Explorer5 (.pe5) # URL: http://www.jisyo.com/cgibin/view.cgi?EXT=pe5 (Japanese) # From: Simon Horman 0 string IIO2H Ulead Photo Explorer5 # Type: X11 cursor # URL: http://webcvs.freedesktop.org/mime/shared-mime-info/freedesktop.org.xml.in?view=markup # From: Mathias Brodala 0 string Xcur X11 cursor # Type: Olympus ORF raw images. # URL: http://libopenraw.freedesktop.org/wiki/Olympus_ORF # From: Adam Buchbinder 0 string MMOR Olympus ORF raw image data, big-endian !:mime image/x-olympus-orf 0 string IIRO Olympus ORF raw image data, little-endian !:mime image/x-olympus-orf 0 string IIRS Olympus ORF raw image data, little-endian !:mime image/x-olympus-orf # Type: files used in modern AVCHD camcoders to store clip information # Extension: .cpi # From: Alexander Danilov 0 string HDMV0100 AVCHD Clip Information # From: Adam Buchbinder # URL: http://local.wasp.uwa.edu.au/~pbourke/dataformats/pic/ # Radiance HDR; usually has .pic or .hdr extension. 0 string #?RADIANCE\n Radiance HDR image data #!mime image/vnd.radiance # From: Adam Buchbinder # URL: http://www.mpi-inf.mpg.de/resources/pfstools/pfs_format_spec.pdf # Used by the pfstools packages. The regex matches for the image size could # probably use some work. The MIME type is made up; if there's one in # actual common use, it should replace the one below. 0 string PFS1\x0a PFS HDR image data #!mime image/x-pfs >1 regex [0-9]*\ \b, %s >>1 regex \ [0-9]{4} \bx%s # Type: Foveon X3F # URL: http://www.photofo.com/downloads/x3f-raw-format.pdf # From: Adam Buchbinder # Note that the MIME type isn't defined anywhere that I can find; if # there's a canonical type for this format, it should replace this one. 0 string FOVb Foveon X3F raw image data !:mime image/x-x3f >6 leshort x \b, version %d. >4 leshort x \b%d >28 lelong x \b, %dx >32 lelong x \b%d # Paint.NET file # From Adam Buchbinder 0 string PDN3 Paint.NET image data !:mime image/x-paintnet # Not really an image. # From: "Tano M. Fotang" 0 string \x46\x4d\x52\x00 ISO/IEC 19794-2 Format Minutiae Record (FMR) # WEBP https://developers.google.com/speed/webp/docs/riff_container -0 string RIFF ->8 string WEBP Web/P image data ->>4 lelong x \b, %d bytes +#0 string RIFF +#>8 string WEBP Web/P image data +#>>4 lelong x \b, %d bytes + +# doc: http://www.shikino.co.jp/eng/products/images/FLOWER.jpg.zip +# example: http://www.shikino.co.jp/eng/products/images/FLOWER.wdp.zip +90 bequad 0x574D50484F544F00 JPEG-XR Image +>98 byte&0x08 =0x08 \b, hard tiling +>99 byte&0x80 =0x80 \b, tiling present +>99 byte&0x40 =0x40 \b, codestream present +>99 byte&0x38 x \b, spatial xform= +>99 byte&0x38 0x00 \bTL +>99 byte&0x38 0x08 \bBL +>99 byte&0x38 0x10 \bTR +>99 byte&0x38 0x18 \bBR +>99 byte&0x38 0x20 \bBT +>99 byte&0x38 0x28 \bRB +>99 byte&0x38 0x30 \bLT +>99 byte&0x38 0x38 \bLB +>100 byte&0x80 =0x80 \b, short header +>>102 beshort+1 x \b, %d +>>104 beshort+1 x \bx%d +>100 byte&0x80 =0x00 \b, long header +>>102 belong+1 x \b, %x +>>106 belong+1 x \bx%x +>101 beshort&0xf x \b, bitdepth= +>>101 beshort&0xf 0x0 \b1-WHITE=1 +>>101 beshort&0xf 0x1 \b8 +>>101 beshort&0xf 0x2 \b16 +>>101 beshort&0xf 0x3 \b16-SIGNED +>>101 beshort&0xf 0x4 \b16-FLOAT +>>101 beshort&0xf 0x5 \b(reserved 5) +>>101 beshort&0xf 0x6 \b32-SIGNED +>>101 beshort&0xf 0x7 \b32-FLOAT +>>101 beshort&0xf 0x8 \b5 +>>101 beshort&0xf 0x9 \b10 +>>101 beshort&0xf 0xa \b5-6-5 +>>101 beshort&0xf 0xb \b(reserved %d) +>>101 beshort&0xf 0xc \b(reserved %d) +>>101 beshort&0xf 0xd \b(reserved %d) +>>101 beshort&0xf 0xe \b(reserved %d) +>>101 beshort&0xf 0xf \b1-BLACK=1 +>101 beshort&0xf0 x \b, colorfmt= +>>101 beshort&0xf0 0x00 \bYONLY +>>101 beshort&0xf0 0x10 \bYUV240 +>>101 beshort&0xf0 0x20 \bYWV422 +>>101 beshort&0xf0 0x30 \bYWV444 +>>101 beshort&0xf0 0x40 \bCMYK +>>101 beshort&0xf0 0x50 \bCMYKDIRECT +>>101 beshort&0xf0 0x60 \bNCOMPONENT +>>101 beshort&0xf0 0x70 \bRGB +>>101 beshort&0xf0 0x80 \bRGBE +>>101 beshort&0xf0 >0x80 \b(reserved 0x%x) + +# From: Johan van der Knijff +# +# BPG (Better Portable Graphics) format +# http://bellard.org/bpg/ +# http://fileformats.archiveteam.org/wiki/BPG +# +0 string \x42\x50\x47\xFB BPG (Better Portable Graphics) +!:mime image/bpg Index: releng/10.1/contrib/file/magic/Magdir/jpeg =================================================================== --- releng/10.1/contrib/file/magic/Magdir/jpeg (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/jpeg (revision 284193) @@ -1,174 +1,115 @@ #------------------------------------------------------------------------------ -# $File: jpeg,v 1.19 2013/02/04 15:50:03 christos Exp $ +# $File: jpeg,v 1.25 2015/01/02 16:56:50 christos Exp $ # JPEG images # SunOS 5.5.1 had # # 0 string \377\330\377\340 JPEG file # 0 string \377\330\377\356 JPG file # # both of which turn into "JPEG image data" here. # 0 beshort 0xffd8 JPEG image data !:mime image/jpeg !:apple 8BIMJPEG !:strength +2 >6 string JFIF \b, JFIF standard # The following added by Erik Rossen 1999-09-06 # in a vain attempt to add image size reporting for JFIF. Note that these # tests are not fool-proof since some perfectly valid JPEGs are currently # impossible to specify in magic(4) format. # First, a little JFIF version info: >>11 byte x \b %d. >>12 byte x \b%02d # Next, the resolution or aspect ratio of the image: -#>>13 byte 0 \b, aspect ratio -#>>13 byte 1 \b, resolution (DPI) -#>>13 byte 2 \b, resolution (DPCM) -#>>4 beshort x \b, segment length %d +>>13 byte 0 \b, aspect ratio +>>13 byte 1 \b, resolution (DPI) +>>13 byte 2 \b, resolution (DPCM) +>>14 beshort x \b, density %dx +>>16 beshort x \b%d +>>4 beshort x \b, segment length %d # Next, show thumbnail info, if it exists: >>18 byte !0 \b, thumbnail %dx >>>19 byte x \b%d +>6 string Exif \b, Exif standard: [ +>>12 indirect/r x +>>12 string x \b] -# EXIF moved down here to avoid reporting a bogus version number, -# and EXIF version number printing added. -# - Patrik R=E5dman ->6 string Exif \b, EXIF standard -# Look for EXIF IFD offset in IFD 0, and then look for EXIF version tag in EXIF IFD. -# All possible combinations of entries have to be enumerated, since no looping -# is possible. And both endians are possible... -# The combinations included below are from real-world JPEGs. -# Little-endian ->>12 string II -# IFD 0 Entry #5: ->>>70 leshort 0x8769 -# EXIF IFD Entry #1: ->>>>(78.l+14) leshort 0x9000 ->>>>>(78.l+23) byte x %c ->>>>>(78.l+24) byte x \b.%c ->>>>>(78.l+25) byte !0x30 \b%c -# IFD 0 Entry #9: ->>>118 leshort 0x8769 -# EXIF IFD Entry #3: ->>>>(126.l+38) leshort 0x9000 ->>>>>(126.l+47) byte x %c ->>>>>(126.l+48) byte x \b.%c ->>>>>(126.l+49) byte !0x30 \b%c -# IFD 0 Entry #10 ->>>130 leshort 0x8769 -# EXIF IFD Entry #3: ->>>>(138.l+38) leshort 0x9000 ->>>>>(138.l+47) byte x %c ->>>>>(138.l+48) byte x \b.%c ->>>>>(138.l+49) byte !0x30 \b%c -# EXIF IFD Entry #4: ->>>>(138.l+50) leshort 0x9000 ->>>>>(138.l+59) byte x %c ->>>>>(138.l+60) byte x \b.%c ->>>>>(138.l+61) byte !0x30 \b%c -# EXIF IFD Entry #5: ->>>>(138.l+62) leshort 0x9000 ->>>>>(138.l+71) byte x %c ->>>>>(138.l+72) byte x \b.%c ->>>>>(138.l+73) byte !0x30 \b%c -# IFD 0 Entry #11 ->>>142 leshort 0x8769 -# EXIF IFD Entry #3: ->>>>(150.l+38) leshort 0x9000 ->>>>>(150.l+47) byte x %c ->>>>>(150.l+48) byte x \b.%c ->>>>>(150.l+49) byte !0x30 \b%c -# EXIF IFD Entry #4: ->>>>(150.l+50) leshort 0x9000 ->>>>>(150.l+59) byte x %c ->>>>>(150.l+60) byte x \b.%c ->>>>>(150.l+61) byte !0x30 \b%c -# EXIF IFD Entry #5: ->>>>(150.l+62) leshort 0x9000 ->>>>>(150.l+71) byte x %c ->>>>>(150.l+72) byte x \b.%c ->>>>>(150.l+73) byte !0x30 \b%c -# Big-endian ->>12 string MM -# IFD 0 Entry #9: ->>>118 beshort 0x8769 -# EXIF IFD Entry #1: ->>>>(126.L+14) beshort 0x9000 ->>>>>(126.L+23) byte x %c ->>>>>(126.L+24) byte x \b.%c ->>>>>(126.L+25) byte !0x30 \b%c -# EXIF IFD Entry #3: ->>>>(126.L+38) beshort 0x9000 ->>>>>(126.L+47) byte x %c ->>>>>(126.L+48) byte x \b.%c ->>>>>(126.L+49) byte !0x30 \b%c -# IFD 0 Entry #10 ->>>130 beshort 0x8769 -# EXIF IFD Entry #3: ->>>>(138.L+38) beshort 0x9000 ->>>>>(138.L+47) byte x %c ->>>>>(138.L+48) byte x \b.%c ->>>>>(138.L+49) byte !0x30 \b%c -# EXIF IFD Entry #5: ->>>>(138.L+62) beshort 0x9000 ->>>>>(138.L+71) byte x %c ->>>>>(138.L+72) byte x \b.%c ->>>>>(138.L+73) byte !0x30 \b%c -# IFD 0 Entry #11 ->>>142 beshort 0x8769 -# EXIF IFD Entry #4: ->>>>(150.L+50) beshort 0x9000 ->>>>>(150.L+59) byte x %c ->>>>>(150.L+60) byte x \b.%c ->>>>>(150.L+61) byte !0x30 \b%c -# Here things get sticky. We can do ONE MORE marker segment with -# indirect addressing, and that's all. It would be great if we could -# do pointer arithemetic like in an assembler language. Christos? -# And if there was some sort of looping construct to do searches, plus a few -# named accumulators, it would be even more effective... -# At least we can show a comment if no other segments got inserted before: ->(4.S+5) byte 0xFE \b, comment: ->>(4.S+6) pstring/HJ x "%s" -# Or, we can show the encoding type (I've included only the three most common) -# and image dimensions if we are lucky and the SOFn (image segment) is here: ->(4.S+5) byte 0xC0 \b, baseline ->>(4.S+6) byte x \b, precision %d ->>(4.S+7) beshort x \b, %dx ->>(4.S+9) beshort x \b%d ->(4.S+5) byte 0xC1 \b, extended sequential ->>(4.S+6) byte x \b, precision %d ->>(4.S+7) beshort x \b, %dx ->>(4.S+9) beshort x \b%d ->(4.S+5) byte 0xC2 \b, progressive ->>(4.S+6) byte x \b, precision %d ->>(4.S+7) beshort x \b, %dx ->>(4.S+9) beshort x \b%d -# I've commented-out quantisation table reporting. I doubt anyone cares yet. -#>(4.S+5) byte 0xDB \b, quantisation table -#>>(4.S+6) beshort x \b length=%d -#>14 beshort x \b, %d x -#>16 beshort x \b %d +# Jump to the first segment +>(4.S+4) use jpeg_segment + +# This uses recursion... +0 name jpeg_segment +>0 beshort 0xFFFE +>>(2.S+2) use jpeg_segment +>>2 pstring/HJ x \b, comment: "%s" + +>0 beshort 0xFFC0 +>>(2.S+2) use jpeg_segment +>>4 byte x \b, baseline, precision %d +>>7 beshort x \b, %dx +>>5 beshort x \b%d +>>9 byte x \b, frames %d + +>0 beshort 0xFFC1 +>>(2.S+2) use jpeg_segment +>>4 byte x \b, extended sequential, precision %d +>>7 beshort x \b, %dx +>>5 beshort x \b%d +>>9 byte x \b, frames %d + +>0 beshort 0xFFC2 +>>(2.S+2) use jpeg_segment +>>4 byte x \b, progressive, precision %d +>>7 beshort x \b, %dx +>>5 beshort x \b%d +>>9 byte x \b, frames %d + +# Define Huffman Tables +>0 beshort 0xFFC4 +>>(2.S+2) use jpeg_segment + +>0 beshort 0xFFE1 +#>>(2.S+2) use jpeg_segment +>>4 string Exif \b, Exif Standard: [ +>>>10 indirect/r x +>>>10 string x \b] + +# Application specific markers +>0 beshort&0xFFE0 =0xFFE0 +>>(2.S+2) use jpeg_segment + +# DB: Define Quantization tables +# DD: Define Restart interval [XXX: wrong here, it is 4 bytes] +# D8: Start of image +# D9: End of image +# Dn: Restart +>0 beshort&0xFFD0 =0xFFD0 +>>(2.S+2) use jpeg_segment + +#>0 beshort x unknown 0x%x +#>>(2.S+2) use jpeg_segment # HSI is Handmade Software's proprietary JPEG encoding scheme 0 string hsi1 JPEG image data, HSI proprietary # From: David Santinoli 0 string \x00\x00\x00\x0C\x6A\x50\x20\x20\x0D\x0A\x87\x0A JPEG 2000 # From: Johan van der Knijff # Added sub-entries for JP2, JPX, JPM and MJ2 formats; added mimetypes # https://github.com/bitsgalore/jp2kMagic # # Now read value of 'Brand' field, which yields a few possibilities: >20 string \x6a\x70\x32\x20 Part 1 (JP2) !:mime image/jp2 >20 string \x6a\x70\x78\x20 Part 2 (JPX) !:mime image/jpx >20 string \x6a\x70\x6d\x20 Part 6 (JPM) !:mime image/jpm >20 string \x6d\x6a\x70\x32 Part 3 (MJ2) !:mime video/mj2 # Type: JPEG 2000 codesream # From: Mathieu Malaterre 0 belong 0xff4fff51 JPEG 2000 codestream 45 beshort 0xff52 Index: releng/10.1/contrib/file/magic/Magdir/kerberos =================================================================== --- releng/10.1/contrib/file/magic/Magdir/kerberos (nonexistent) +++ releng/10.1/contrib/file/magic/Magdir/kerberos (revision 284193) @@ -0,0 +1,45 @@ + +#------------------------------------------------------------------------------ +# $File: kerberos,v 1.1 2014/12/10 18:45:43 christos Exp $ +# kerberos: MIT kerberos file binary formats +# + +# This magic entry is for demonstration purposes and could be improved +# if the following features were implemented in file: +# +# Strings inside [[ .. ]] in the descriptions have special meanings and +# are not printed. +# +# - Provide some form of iteration in number of components +# [[${counter}=%d]] in the description +# then append +# [${counter}--] in the offset of the entries +# - Provide a way to round the next offset +# Add [R:4] after the offset? +# - Provide a way to have optional entries +# XXX: Syntax: +# - Provide a way to "save" entries to print them later. +# if the description is [[${name}=%s]], then nothing is +# printed and a subsequent entry in the same magic file +# can refer to ${name} +# - Provide a way to format strings as hex values +# +# http://www.gnu.org/software/shishi/manual/html_node/\ +# The-Keytab-Binary-File-Format.html +# + +0 name keytab_entry +#>0 beshort x \b, size=%d +#>2 beshort x \b, components=%d +>4 pstring/H x \b, realm=%s +>>&0 pstring/H x \b, principal=%s/ +>>>&0 pstring/H x \b%s +>>>>&0 belong x \b, type=%d +>>>>>&0 bedate x \b, date=%s +>>>>>>&0 byte x \b, kvno=%u +#>>>>>>>&0 pstring/H x +#>>>>>>>>&0 belong x +#>>>>>>>>>>&0 use keytab_entry + +0 belong 0x05020000 Kerberos Keytab file +>4 use keytab_entry Index: releng/10.1/contrib/file/magic/Magdir/linux =================================================================== --- releng/10.1/contrib/file/magic/Magdir/linux (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/linux (revision 284193) @@ -1,415 +1,434 @@ #------------------------------------------------------------------------------ -# $File: linux,v 1.57 2014/05/20 20:10:17 christos Exp $ +# $File: linux,v 1.59 2014/11/03 21:03:36 christos Exp $ # linux: file(1) magic for Linux files # # Values for Linux/i386 binaries, from Daniel Quinlan # The following basic Linux magic is useful for reference, but using # "long" magic is a better practice in order to avoid collisions. # # 2 leshort 100 Linux/i386 # >0 leshort 0407 impure executable (OMAGIC) # >0 leshort 0410 pure executable (NMAGIC) # >0 leshort 0413 demand-paged executable (ZMAGIC) # >0 leshort 0314 demand-paged executable (QMAGIC) # 0 lelong 0x00640107 Linux/i386 impure executable (OMAGIC) >16 lelong 0 \b, stripped 0 lelong 0x00640108 Linux/i386 pure executable (NMAGIC) >16 lelong 0 \b, stripped 0 lelong 0x0064010b Linux/i386 demand-paged executable (ZMAGIC) >16 lelong 0 \b, stripped 0 lelong 0x006400cc Linux/i386 demand-paged executable (QMAGIC) >16 lelong 0 \b, stripped # 0 string \007\001\000 Linux/i386 object file >20 lelong >0x1020 \b, DLL library # Linux-8086 stuff: 0 string \01\03\020\04 Linux-8086 impure executable >28 long !0 not stripped 0 string \01\03\040\04 Linux-8086 executable >28 long !0 not stripped # 0 string \243\206\001\0 Linux-8086 object file # 0 string \01\03\020\20 Minix-386 impure executable >28 long !0 not stripped 0 string \01\03\040\20 Minix-386 executable >28 long !0 not stripped 0 string \01\03\04\20 Minix-386 NSYM/GNU executable >28 long !0 not stripped # core dump file, from Bill Reynolds 216 lelong 0421 Linux/i386 core file !:strength / 2 >220 string >\0 of '%s' >200 lelong >0 (signal %d) # # LILO boot/chain loaders, from Daniel Quinlan # this can be overridden by the DOS executable (COM) entry 2 string LILO Linux/i386 LILO boot/chain loader # # Linux make config build file, from Ole Aamot # Updated by Ken Sharp 28 string make\ config Linux make config build file (old) 49 search/70 Kernel\ Configuration Linux make config build file # # PSF fonts, from H. Peter Anvin # Updated by Adam Buchbinder # See: http://www.win.tue.nl/~aeb/linux/kbd/font-formats-1.html 0 leshort 0x0436 Linux/i386 PC Screen Font v1 data, >2 byte&0x01 0 256 characters, >2 byte&0x01 !0 512 characters, >2 byte&0x02 0 no directory, >2 byte&0x02 !0 Unicode directory, >3 byte >0 8x%d 0 string \x72\xb5\x4a\x86\x00\x00 Linux/i386 PC Screen Font v2 data, >16 lelong x %d characters, >12 lelong&0x01 0 no directory, >12 lelong&0x01 !0 Unicode directory, >24 lelong x %d >28 lelong x \bx%d # Linux swap file, from Daniel Quinlan 4086 string SWAP-SPACE Linux/i386 swap file # From: Jeff Bailey # Linux swap file with swsusp1 image, from Jeff Bailey 4076 string SWAPSPACE2S1SUSPEND Linux/i386 swap file (new style) with SWSUSP1 image # From: James Hunt 4076 string SWAPSPACE2LINHIB0001 Linux/i386 swap file (new style) (compressed hibernate) # according to man page of mkswap (8) March 1999 # volume label and UUID Russell Coker # http://etbe.coker.com.au/2008/07/08/label-vs-uuid-vs-device/ 4086 string SWAPSPACE2 Linux/i386 swap file (new style), >0x400 long x version %d (4K pages), >0x404 long x size %d pages, >1052 string \0 no label, >1052 string >\0 LABEL=%s, >0x40c belong x UUID=%08x >0x410 beshort x \b-%04x >0x412 beshort x \b-%04x >0x414 beshort x \b-%04x >0x416 belong x \b-%08x >0x41a beshort x \b%04x # From Daniel Novotny # swap file for PowerPC 65526 string SWAPSPACE2 Linux/ppc swap file 16374 string SWAPSPACE2 Linux/ia64 swap file # # Linux kernel boot images, from Albert Cahalan # and others such as Axel Kohlmeyer # and Nicolas Lichtmaier # All known start with: b8 c0 07 8e d8 b8 00 90 8e c0 b9 00 01 29 f6 29 # Linux kernel boot images (i386 arch) (Wolfram Kleff) 514 string HdrS Linux kernel !:strength + 55 >510 leshort 0xAA55 x86 boot executable >>518 leshort >0x1ff >>>529 byte 0 zImage, >>>529 byte 1 bzImage, >>>526 lelong >0 >>>>(526.s+0x200) string >\0 version %s, >>498 leshort 1 RO-rootFS, >>498 leshort 0 RW-rootFS, >>508 leshort >0 root_dev 0x%X, >>502 leshort >0 swap_dev 0x%X, >>504 leshort >0 RAMdisksize %u KB, >>506 leshort 0xFFFF Normal VGA >>506 leshort 0xFFFE Extended VGA >>506 leshort 0xFFFD Prompt for Videomode >>506 leshort >0 Video mode %d # This also matches new kernels, which were caught above by "HdrS". 0 belong 0xb8c0078e Linux kernel >0x1e3 string Loading version 1.3.79 or older >0x1e9 string Loading from prehistoric times # System.map files - Nicolas Lichtmaier 8 search/1 \ A\ _text Linux kernel symbol map text # LSM entries - Nicolas Lichtmaier 0 search/1 Begin3 Linux Software Map entry text 0 search/1 Begin4 Linux Software Map entry text (new format) # From Matt Zimmerman, enhanced for v3 by Matthew Palmer 0 belong 0x4f4f4f4d User-mode Linux COW file >4 belong <3 \b, version %d >>8 string >\0 \b, backing file %s >4 belong >2 \b, version %d >>32 string >\0 \b, backing file %s ############################################################################ # Linux kernel versions 0 string \xb8\xc0\x07\x8e\xd8\xb8\x00\x90 Linux >497 leshort 0 x86 boot sector >>514 belong 0x8e of a kernel from the dawn of time! >>514 belong 0x908ed8b4 version 0.99-1.1.42 >>514 belong 0x908ed8b8 for memtest86 >497 leshort !0 x86 kernel >>504 leshort >0 RAMdisksize=%u KB >>502 leshort >0 swap=0x%X >>508 leshort >0 root=0x%X >>>498 leshort 1 \b-ro >>>498 leshort 0 \b-rw >>506 leshort 0xFFFF vga=normal >>506 leshort 0xFFFE vga=extended >>506 leshort 0xFFFD vga=ask >>506 leshort >0 vga=%d >>514 belong 0x908ed881 version 1.1.43-1.1.45 >>514 belong 0x15b281cd >>>0xa8e belong 0x55AA5a5a version 1.1.46-1.2.13,1.3.0 >>>0xa99 belong 0x55AA5a5a version 1.3.1,2 >>>0xaa3 belong 0x55AA5a5a version 1.3.3-1.3.30 >>>0xaa6 belong 0x55AA5a5a version 1.3.31-1.3.41 >>>0xb2b belong 0x55AA5a5a version 1.3.42-1.3.45 >>>0xaf7 belong 0x55AA5a5a version 1.3.46-1.3.72 >>514 string HdrS >>>518 leshort >0x1FF >>>>529 byte 0 \b, zImage >>>>529 byte 1 \b, bzImage >>>>(526.s+0x200) string >\0 \b, version %s # Linux boot sector thefts. 0 belong 0xb8c0078e Linux >0x1e6 belong 0x454c4b53 ELKS Kernel >0x1e6 belong !0x454c4b53 style boot sector ############################################################################ # Linux S390 kernel image # Created by: Jan Kaluza 8 string \x02\x00\x00\x18\x60\x00\x00\x50\x02\x00\x00\x68\x60\x00\x00\x50\x40\x40\x40\x40\x40\x40\x40\x40 Linux S390 >0x00010000 search/b/4096 \x00\x0a\x00\x00\x8b\xad\xcc\xcc # 64bit >>&0 string \xc1\x00\xef\xe3\xf0\x68\x00\x00 Z10 64bit kernel >>&0 string \xc1\x00\xef\xc3\x00\x00\x00\x00 Z9-109 64bit kernel >>&0 string \xc0\x00\x20\x00\x00\x00\x00\x00 Z990 64bit kernel >>&0 string \x00\x00\x00\x00\x00\x00\x00\x00 Z900 64bit kernel # 32bit >>&0 string \x81\x00\xc8\x80\x00\x00\x00\x00 Z10 32bit kernel >>&0 string \x81\x00\xc8\x80\x00\x00\x00\x00 Z9-109 32bit kernel >>&0 string \x80\x00\x20\x00\x00\x00\x00\x00 Z990 32bit kernel >>&0 string \x80\x00\x00\x00\x00\x00\x00\x00 Z900 32bit kernel # Linux ARM compressed kernel image # From: Kevin Cernekee 36 lelong 0x016f2818 Linux kernel ARM boot executable zImage (little-endian) 36 belong 0x016f2818 Linux kernel ARM boot executable zImage (big-endian) ############################################################################ # Linux 8086 executable 0 lelong&0xFF0000FF 0xC30000E9 Linux-Dev86 executable, headerless >5 string . >>4 string >\0 \b, libc version %s 0 lelong&0xFF00FFFF 0x4000301 Linux-8086 executable >2 byte&0x01 !0 \b, unmapped zero page >2 byte&0x20 0 \b, impure >2 byte&0x20 !0 >>2 byte&0x10 !0 \b, A_EXEC >2 byte&0x02 !0 \b, A_PAL >2 byte&0x04 !0 \b, A_NSYM >2 byte&0x08 !0 \b, A_STAND >2 byte&0x40 !0 \b, A_PURE >2 byte&0x80 !0 \b, A_TOVLY >28 long !0 \b, not stripped >37 string . >>36 string >\0 \b, libc version %s # 0 lelong&0xFF00FFFF 0x10000301 ld86 I80386 executable # 0 lelong&0xFF00FFFF 0xB000301 ld86 M68K executable # 0 lelong&0xFF00FFFF 0xC000301 ld86 NS16K executable # 0 lelong&0xFF00FFFF 0x17000301 ld86 SPARC executable # SYSLINUX boot logo files (from 'ppmtolss16' sources) # http://www.syslinux.org/wiki/index.php/SYSLINUX#Display_graphic_from_filename: # file extension .lss .16 0 lelong =0x1413f33d SYSLINUX' LSS16 image data # syslinux-4.05/mime/image/x-lss16.xml !:mime image/x-lss16 >4 leshort x \b, width %d >6 leshort x \b, height %d 0 string OOOM User-Mode-Linux's Copy-On-Write disk image >4 belong x version %d # SE Linux policy database # From: Mike Frysinger 0 lelong 0xf97cff8c SE Linux policy >16 lelong x v%d >20 lelong 1 MLS >24 lelong x %d symbols >28 lelong x %d ocons # Linux Logical Volume Manager (LVM) # Emmanuel VARAGNAT # # System ID, UUID and volume group name are 128 bytes long # but they should never be full and initialized with zeros... # # LVM1 # 0x0 string HM\001 LVM1 (Linux Logical Volume Manager), version 1 >0x12c string >\0 , System ID: %s 0x0 string HM\002 LVM1 (Linux Logical Volume Manager), version 2 >0x12c string >\0 , System ID: %s # LVM2 # # It seems that the label header can be in one the four first sector # of the disk... (from _find_labeller in lib/label/label.c of LVM2) # # 0x200 seems to be the common case 0x218 string LVM2\ 001 LVM2 PV (Linux Logical Volume Manager) # read the offset to add to the start of the header, and the header # start in 0x200 >&(&-12.l-0x21) byte x # display UUID in LVM format + display all 32 bytes (instead of max string length: 31) >>&0x0 string >\x2f \b, UUID: %.6s >>&0x6 string >\x2f \b-%.4s >>&0xa string >\x2f \b-%.4s >>&0xe string >\x2f \b-%.4s >>&0x12 string >\x2f \b-%.4s >>&0x16 string >\x2f \b-%.4s >>&0x1a string >\x2f \b-%.6s >>&0x20 lequad x \b, size: %lld 0x018 string LVM2\ 001 LVM2 PV (Linux Logical Volume Manager) >&(&-12.l-0x21) byte x # display UUID in LVM format + display all 32 bytes (instead of max string length: 31) >>&0x0 string >\x2f \b, UUID: %.6s >>&0x6 string >\x2f \b-%.4s >>&0xa string >\x2f \b-%.4s >>&0xe string >\x2f \b-%.4s >>&0x12 string >\x2f \b-%.4s >>&0x16 string >\x2f \b-%.4s >>&0x1a string >\x2f \b-%.6s >>&0x20 lequad x \b, size: %lld 0x418 string LVM2\ 001 LVM2 PV (Linux Logical Volume Manager) >&(&-12.l-0x21) byte x # display UUID in LVM format + display all 32 bytes (instead of max string length: 31) >>&0x0 string >\x2f \b, UUID: %.6s >>&0x6 string >\x2f \b-%.4s >>&0xa string >\x2f \b-%.4s >>&0xe string >\x2f \b-%.4s >>&0x12 string >\x2f \b-%.4s >>&0x16 string >\x2f \b-%.4s >>&0x1a string >\x2f \b-%.6s >>&0x20 lequad x \b, size: %lld 0x618 string LVM2\ 001 LVM2 PV (Linux Logical Volume Manager) >&(&-12.l-0x21) byte x # display UUID in LVM format + display all 32 bytes (instead of max string length: 31) >>&0x0 string >\x2f \b, UUID: %.6s >>&0x6 string >\x2f \b-%.4s >>&0xa string >\x2f \b-%.4s >>&0xe string >\x2f \b-%.4s >>&0x12 string >\x2f \b-%.4s >>&0x16 string >\x2f \b-%.4s >>&0x1a string >\x2f \b-%.6s >>&0x20 lequad x \b, size: %lld # LVM snapshot # from Jason Farrel 0 string SnAp LVM Snapshot (CopyOnWrite store) >4 lelong !0 - valid, >4 lelong 0 - invalid, >8 lelong x version %d, >12 lelong x chunk_size %d # SE Linux policy database 0 lelong 0xf97cff8c SE Linux policy >16 lelong x v%d >20 lelong 1 MLS >24 lelong x %d symbols >28 lelong x %d ocons # LUKS: Linux Unified Key Setup, On-Disk Format, http://luks.endorphin.org/spec # Anthon van der Neut (anthon@mnt.org) 0 string LUKS\xba\xbe LUKS encrypted file, >6 beshort x ver %d >8 string x [%s, >40 string x %s, >72 string x %s] >168 string x UUID: %s # Summary: Xen saved domain file # Created by: Radek Vokal 0 string LinuxGuestRecord Xen saved domain >20 search/256 (name >>&1 string x (name %s) # Type: Xen, the virtual machine monitor # From: Radek Vokal 0 string LinuxGuestRecord Xen saved domain #>2 regex \(name\ [^)]*\) %s >20 search/256 (name (name >>&1 string x %s...) # Systemd journald files # See http://www.freedesktop.org/wiki/Software/systemd/journal-files/. # From: Zbigniew Jedrzejewski-Szmek # check magic 0 string LPKSHHRH # check that state is one of known values >16 ubyte&252 0 # check that each half of three unique id128s is non-zero >>24 ubequad >0 >>>32 ubequad >0 >>>>40 ubequad >0 >>>>>48 ubequad >0 >>>>>>56 ubequad >0 >>>>>>>64 ubequad >0 Journal file !:mime application/octet-stream # provide more info >>>>>>>>184 leqdate 0 empty >>>>>>>>16 ubyte 0 \b, offline >>>>>>>>16 ubyte 1 \b, online >>>>>>>>16 ubyte 2 \b, archived >>>>>>>>8 ulelong&1 1 \b, sealed >>>>>>>>12 ulelong&1 1 \b, compressed # BCache backing and cache devices # From: Gabriel de Perthuis 0x1008 lequad 8 >0x1018 string \xc6\x85\x73\xf6\x4e\x1a\x45\xca\x82\x65\xf5\x7f\x48\xba\x6d\x81 BCache >>0x1010 ulequad 0 cache device >>0x1010 ulequad 1 backing device >>0x1010 ulequad 3 cache device >>0x1010 ulequad 4 backing device >>0x1048 string >0 \b, label "%.32s" >>0x1028 ubelong x \b, uuid %08x >>0x102c ubeshort x \b-%04x >>0x102e ubeshort x \b-%04x >>0x1030 ubeshort x \b-%04x >>0x1032 ubelong x \b-%08x >>0x1036 ubeshort x \b%04x >>0x1038 ubelong x \b, set uuid %08x >>0x103c ubeshort x \b-%04x >>0x103e ubeshort x \b-%04x >>0x1040 ubeshort x \b-%04x >>0x1042 ubelong x \b-%08x >>0x1046 ubeshort x \b%04x # Linux device tree: # File format description can be found in the Linux kernel sources at # Documentation/devicetree/booting-without-of.txt # From Christoph Biedl 0 belong 0xd00dfeed # structure and strings must be within blob >&(8.L) byte x >>&(12.L) byte x >>>20 belong >1 Device Tree Blob version %d >>>>4 belong x \b, size=%d >>>>20 belong >1 >>>>>28 belong x \b, boot CPU=%d >>>>20 belong >2 >>>>>32 belong x \b, string block size=%d >>>>20 belong >16 >>>>>36 belong x \b, DT structure block size=%d +# glibc locale archive as defined in glibc locale/locarchive.h +0 lelong 0xde020109 locale archive +>24 lelong x %d strings + +# Summary: Database file for mlocate +# Description: A database file as used by mlocate, a fast implementation +# of locate/updatedb. It uses merging to reuse the existing +# database and avoid rereading most of the filesystem. It's +# the default version of locate on Arch Linux (and others). +# File path: /var/lib/mlocate/mlocate.db by default (but configurable) +# Site: https://fedorahosted.org/mlocate/ +# Format docs: http://linux.die.net/man/5/mlocate.db +# Type: mlocate database file +# URL: https://fedorahosted.org/mlocate/ +# From: Wander Nauta +0 string \0mlocate mlocate database +>12 byte x \b, version %d +>13 byte 1 \b, require visibility +>16 string x \b, root %s Index: releng/10.1/contrib/file/magic/Magdir/macintosh =================================================================== --- releng/10.1/contrib/file/magic/Magdir/macintosh (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/macintosh (revision 284193) @@ -1,393 +1,377 @@ #------------------------------------------------------------------------------ -# $File: macintosh,v 1.23 2013/11/19 18:47:58 christos Exp $ +# $File: macintosh,v 1.25 2014/09/03 13:34:16 christos Exp $ # macintosh description # # BinHex is the Macintosh ASCII-encoded file format (see also "apple") # Daniel Quinlan, quinlan@yggdrasil.com 11 string must\ be\ converted\ with\ BinHex BinHex binary text !:mime application/mac-binhex40 >41 string x \b, version %.3s # Stuffit archives are the de facto standard of compression for Macintosh # files obtained from most archives. (franklsm@tuns.ca) 0 string SIT! StuffIt Archive (data) !:mime application/x-stuffit !:apple SIT!SIT! >2 string x : %s 0 string SITD StuffIt Deluxe (data) >2 string x : %s 0 string Seg StuffIt Deluxe Segment (data) >2 string x : %s # Newer StuffIt archives (grant@netbsd.org) 0 string StuffIt StuffIt Archive !:mime application/x-stuffit !:apple SIT!SIT! #>162 string >0 : %s # Macintosh Applications and Installation binaries (franklsm@tuns.ca) # GRR: Too weak #0 string APPL Macintosh Application (data) #>2 string x \b: %s # Macintosh System files (franklsm@tuns.ca) # GRR: Too weak #0 string zsys Macintosh System File (data) #0 string FNDR Macintosh Finder (data) #0 string libr Macintosh Library (data) #>2 string x : %s #0 string shlb Macintosh Shared Library (data) #>2 string x : %s #0 string cdev Macintosh Control Panel (data) #>2 string x : %s #0 string INIT Macintosh Extension (data) #>2 string x : %s #0 string FFIL Macintosh Truetype Font (data) #>2 string x : %s #0 string LWFN Macintosh Postscript Font (data) #>2 string x : %s # Additional Macintosh Files (franklsm@tuns.ca) # GRR: Too weak #0 string PACT Macintosh Compact Pro Archive (data) #>2 string x : %s #0 string ttro Macintosh TeachText File (data) #>2 string x : %s #0 string TEXT Macintosh TeachText File (data) #>2 string x : %s #0 string PDF Macintosh PDF File (data) #>2 string x : %s # MacBinary format (Eric Fischer, enf@pobox.com) # # Unfortunately MacBinary doesn't really have a magic number prior # to the MacBinary III format. The checksum is really the way to # do it, but the magic file format isn't up to the challenge. # # 0 byte 0 # 1 byte # filename length # 2 string # filename # 65 string # file type # 69 string # file creator # 73 byte # Finder flags # 74 byte 0 # 75 beshort # vertical posn in window # 77 beshort # horiz posn in window # 79 beshort # window or folder ID # 81 byte # protected? # 82 byte 0 # 83 belong # length of data segment # 87 belong # length of resource segment # 91 belong # file creation date # 95 belong # file modification date # 99 beshort # length of comment after resource # 101 byte # new Finder flags # 102 string mBIN # (only in MacBinary III) # 106 byte # char. code of file name # 107 byte # still more Finder flags # 116 belong # total file length # 120 beshort # length of add'l header # 122 byte 129 # for MacBinary II # 122 byte 130 # for MacBinary III # 123 byte 129 # minimum version that can read fmt # 124 beshort # checksum # # This attempts to use the version numbers as a magic number, requiring # that the first one be 0x80, 0x81, 0x82, or 0x83, and that the second # be 0x81. This works for the files I have, but maybe not for everyone's. # Unfortunately, this magic is quite weak - MPi #122 beshort&0xFCFF 0x8081 Macintosh MacBinary data # MacBinary I doesn't have the version number field at all, but MacBinary II # has been in use since 1987 so I hope there aren't many really old files # floating around that this will miss. The original spec calls for using # the nulls in 0, 74, and 82 as the magic number. # # Another possibility, that would also work for MacBinary I, is to use # the assumption that 65-72 will all be ASCII (0x20-0x7F), that 73 will # have bits 1 (changed), 2 (busy), 3 (bozo), and 6 (invisible) unset, # and that 74 will be 0. So something like # # 71 belong&0x80804EFF 0x00000000 Macintosh MacBinary data # # >73 byte&0x01 0x01 \b, inited # >73 byte&0x02 0x02 \b, changed # >73 byte&0x04 0x04 \b, busy # >73 byte&0x08 0x08 \b, bozo # >73 byte&0x10 0x10 \b, system # >73 byte&0x10 0x20 \b, bundle # >73 byte&0x10 0x40 \b, invisible # >73 byte&0x10 0x80 \b, locked #>65 string x \b, type "%4.4s" #>65 string 8BIM (PhotoShop) #>65 string ALB3 (PageMaker 3) #>65 string ALB4 (PageMaker 4) #>65 string ALT3 (PageMaker 3) #>65 string APPL (application) #>65 string AWWP (AppleWorks word processor) #>65 string CIRC (simulated circuit) #>65 string DRWG (MacDraw) #>65 string EPSF (Encapsulated PostScript) #>65 string FFIL (font suitcase) #>65 string FKEY (function key) #>65 string FNDR (Macintosh Finder) #>65 string GIFf (GIF image) #>65 string Gzip (GNU gzip) #>65 string INIT (system extension) #>65 string LIB\ (library) #>65 string LWFN (PostScript font) #>65 string MSBC (Microsoft BASIC) #>65 string PACT (Compact Pro archive) #>65 string PDF\ (Portable Document Format) #>65 string PICT (picture) #>65 string PNTG (MacPaint picture) #>65 string PREF (preferences) #>65 string PROJ (Think C project) #>65 string QPRJ (Think Pascal project) #>65 string SCFL (Defender scores) #>65 string SCRN (startup screen) #>65 string SITD (StuffIt Deluxe) #>65 string SPn3 (SuperPaint) #>65 string STAK (HyperCard stack) #>65 string Seg\ (StuffIt segment) #>65 string TARF (Unix tar archive) #>65 string TEXT (ASCII) #>65 string TIFF (TIFF image) #>65 string TOVF (Eudora table of contents) #>65 string WDBN (Microsoft Word word processor) #>65 string WORD (MacWrite word processor) #>65 string XLS\ (Microsoft Excel) #>65 string ZIVM (compress (.Z)) #>65 string ZSYS (Pre-System 7 system file) #>65 string acf3 (Aldus FreeHand) #>65 string cdev (control panel) -#>65 string dfil (Desk Acessory suitcase) +#>65 string dfil (Desk Accessory suitcase) #>65 string libr (library) #>65 string nX^d (WriteNow word processor) #>65 string nX^w (WriteNow dictionary) #>65 string rsrc (resource) #>65 string scbk (Scrapbook) #>65 string shlb (shared library) #>65 string ttro (SimpleText read-only) #>65 string zsys (system file) #>69 string x \b, creator "%4.4s" # Somewhere, Apple has a repository of registered Creator IDs. These are # just the ones that I happened to have files from and was able to identify. #>69 string 8BIM (Adobe Photoshop) #>69 string ALD3 (PageMaker 3) #>69 string ALD4 (PageMaker 4) #>69 string ALFA (Alpha editor) #>69 string APLS (Apple Scanner) #>69 string APSC (Apple Scanner) #>69 string BRKL (Brickles) #>69 string BTFT (BitFont) #>69 string CCL2 (Common Lisp 2) #>69 string CCL\ (Common Lisp) #>69 string CDmo (The Talking Moose) #>69 string CPCT (Compact Pro) #>69 string CSOm (Eudora) #>69 string DMOV (Font/DA Mover) #>69 string DSIM (DigSim) #>69 string EDIT (Macintosh Edit) #>69 string ERIK (Macintosh Finder) #>69 string EXTR (self-extracting archive) #>69 string Gzip (GNU gzip) #>69 string KAHL (Think C) #>69 string LWFU (LaserWriter Utility) #>69 string LZIV (compress) #>69 string MACA (MacWrite) #>69 string MACS (Macintosh operating system) #>69 string MAcK (MacKnowledge terminal emulator) #>69 string MLND (Defender) #>69 string MPNT (MacPaint) #>69 string MSBB (Microsoft BASIC (binary)) #>69 string MSWD (Microsoft Word) #>69 string NCSA (NCSA Telnet) #>69 string PJMM (Think Pascal) #>69 string PSAL (Hunt the Wumpus) #>69 string PSI2 (Apple File Exchange) #>69 string R*ch (BBEdit) #>69 string RMKR (Resource Maker) #>69 string RSED (Resource Editor) #>69 string Rich (BBEdit) #>69 string SIT! (StuffIt) #>69 string SPNT (SuperPaint) #>69 string Unix (NeXT Mac filesystem) #>69 string VIM! (Vim editor) #>69 string WILD (HyperCard) #>69 string XCEL (Microsoft Excel) #>69 string aCa2 (Fontographer) #>69 string aca3 (Aldus FreeHand) #>69 string dosa (Macintosh MS-DOS file system) #>69 string movr (Font/DA Mover) #>69 string nX^n (WriteNow) #>69 string pdos (Apple ProDOS file system) #>69 string scbk (Scrapbook) #>69 string ttxt (SimpleText) #>69 string ufox (Foreign File Access) # Just in case... 102 string mBIN MacBinary III data with surprising version number # sas magic from Bruce Foster (bef@nwu.edu) # #0 string SAS SAS #>8 string x %s 0 string SAS SAS >24 string DATA data file >24 string CATALOG catalog >24 string INDEX data file index >24 string VIEW data view # sas 7+ magic from Reinhold Koch (reinhold.koch@roche.com) # 0x54 string SAS SAS 7+ >0x9C string DATA data file >0x9C string CATALOG catalog >0x9C string INDEX data file index >0x9C string VIEW data view # spss magic for SPSS system and portable files, # from Bruce Foster (bef@nwu.edu). 0 long 0xc1e2c3c9 SPSS Portable File >40 string x %s 0 string $FL2 SPSS System File >24 string x %s 0 string $FL3 SPSS System File >24 string x %s # Macintosh filesystem data # From "Tom N Harris" # Fixed HFS+ and Partition map magic: Ethan Benson # The MacOS epoch begins on 1 Jan 1904 instead of 1 Jan 1970, so these # entries depend on the data arithmetic added after v.35 # There's also some Pascal strings in here, ditto... # The boot block signature, according to IM:Files, is # "for HFS volumes, this field always contains the value 0x4C4B." # But if this is true for MFS or HFS+ volumes, I don't know. # Alternatively, the boot block is supposed to be zeroed if it's # unused, so a simply >0 should suffice. 0x400 beshort 0xD2D7 Macintosh MFS data >0 beshort 0x4C4B (bootable) >0x40a beshort &0x8000 (locked) >0x402 beldate-0x7C25B080 x created: %s, >0x406 beldate-0x7C25B080 >0 last backup: %s, >0x414 belong x block size: %d, >0x412 beshort x number of blocks: %d, >0x424 pstring x volume name: %s +# *.hfs updated by Joerg Jenderek +# http://en.wikipedia.org/wiki/Hierarchical_File_System # "BD" gives many false positives -#0x400 beshort 0x4244 Macintosh HFS data -#>0 beshort 0x4C4B (bootable) -#>0x40a beshort &0x8000 (locked) -#>0x40a beshort ^0x0100 (mounted) -#>0x40a beshort &0x0200 (spared blocks) -#>0x40a beshort &0x0800 (unclean) -#>0x47C beshort 0x482B (Embedded HFS+ Volume) -#>0x402 beldate-0x7C25B080 x created: %s, -#>0x406 beldate-0x7C25B080 x last modified: %s, -#>0x440 beldate-0x7C25B080 >0 last backup: %s, -#>0x414 belong x block size: %d, -#>0x412 beshort x number of blocks: %d, -#>0x424 pstring x volume name: %s +0x400 beshort 0x4244 +# ftp://ftp.mars.org/pub/hfs/hfsutils-3.2.6.tar.gz/hfsutils-3.2.6/libhfs/apple.h +# first block of volume bit map (always 3) +>0x40e ubeshort 0x0003 +# maximal length of volume name is 27 +>>0x424 ubyte <28 Macintosh HFS data +#!:mime application/octet-stream +# these mime and apple types are not sure +!:mime application/x-apple-diskimage +#!:apple hfsdINIT +#!:apple MACSdisk +>>>0 beshort 0x4C4B (bootable) +#>>>0 beshort 0x0000 (not bootable) +>>>0x40a beshort &0x8000 (locked) +>>>0x40a beshort ^0x0100 (mounted) +>>>0x40a beshort &0x0200 (spared blocks) +>>>0x40a beshort &0x0800 (unclean) +>>>0x47C beshort 0x482B (Embedded HFS+ Volume) +# http://www.epochconverter.com/ +# 0x7C245F00 seconds ~ 2082758400 ~ 01 Jan 2036 00:00:00 ~ 66 years to 1970 +# 0x7C25B080 seconds ~ 2082844800 ~ 02 Jan 2036 00:00:00 +# construct not working +#>>>0x402 beldate-0x7C25B080 x created: %s, +#>>>0x406 beldate-0x7C25B080 x last modified: %s, +#>>>0x440 beldate-0x7C25B080 >0 last backup: %s, +# found block sizes 200h,1200h,2800h +>>>0x414 belong x block size: %d, +>>>0x412 beshort x number of blocks: %d, +>>>0x424 pstring x volume name: %s 0x400 beshort 0x482B Macintosh HFS Extended >&0 beshort x version %d data >0 beshort 0x4C4B (bootable) >0x404 belong ^0x00000100 (mounted) >&2 belong &0x00000200 (spared blocks) >&2 belong &0x00000800 (unclean) >&2 belong &0x00008000 (locked) >&6 string x last mounted by: '%.4s', # really, that should be treated as a belong and we print a string # based on the value. TN1150 only mentions '8.10' for "MacOS 8.1" >&14 beldate-0x7C25B080 x created: %s, # only the creation date is local time, all other timestamps in HFS+ are UTC. >&18 bedate-0x7C25B080 x last modified: %s, >&22 bedate-0x7C25B080 >0 last backup: %s, >&26 bedate-0x7C25B080 >0 last checked: %s, >&38 belong x block size: %d, >&42 belong x number of blocks: %d, >&46 belong x free blocks: %d -# I don't think this is really necessary since it doesn't do much and -# anything with a valid driver descriptor will also have a valid -# partition map -#0 beshort 0x4552 Apple Device Driver data -#>&24 beshort =1 \b, MacOS - -# Is that the partition type a cstring or a pstring? Well, IM says "strings -# shorter than 32 bytes must be terminated with NULL" so I'll treat it as a -# cstring. Of course, partitions can contain more than four entries, but -# what're you gonna do? -# GRR: This magic is too weak, it is just "PM" -#0x200 beshort 0x504D Apple Partition data -#>0x2 beshort x (block size: %d): -#>0x230 string x first type: %s, -#>0x210 string x name: %s, -#>0x254 belong x number of blocks: %d, -#>0x400 beshort 0x504D -#>>0x430 string x second type: %s, -#>>0x410 string x name: %s, -#>>0x454 belong x number of blocks: %d, -#>>0x600 beshort 0x504D -#>>>0x630 string x third type: %s, -#>>>0x610 string x name: %s, -#>>>0x654 belong x number of blocks: %d, -#>>0x800 beshort 0x504D -#>>>0x830 string x fourth type: %s, -#>>>0x810 string x name: %s, -#>>>0x854 belong x number of blocks: %d, -#>>>0xa00 beshort 0x504D -#>>>>0xa30 string x fifth type: %s, -#>>>>0xa10 string x name: %s, -#>>>>0xa54 belong x number of blocks: %d -#>>>0xc00 beshort 0x504D -#>>>>0xc30 string x sixth type: %s, -#>>>>0xc10 string x name: %s, -#>>>>0xc54 belong x number of blocks: %d ## AFAIK, only the signature is different +# same as Apple Partition Map +# GRR: This magic is too weak, it is just "TS" #0x200 beshort 0x5453 Apple Old Partition data #>0x2 beshort x block size: %d, #>0x230 string x first type: %s, #>0x210 string x name: %s, #>0x254 belong x number of blocks: %d, #>0x400 beshort 0x504D #>>0x430 string x second type: %s, #>>0x410 string x name: %s, #>>0x454 belong x number of blocks: %d, #>>0x800 beshort 0x504D #>>>0x830 string x third type: %s, #>>>0x810 string x name: %s, #>>>0x854 belong x number of blocks: %d, #>>>0xa00 beshort 0x504D #>>>>0xa30 string x fourth type: %s, #>>>>0xa10 string x name: %s, #>>>>0xa54 belong x number of blocks: %d # From: Remi Mommsen 0 string BOMStore Mac OS X bill of materials (BOM) file # From: Adam Buchbinder # URL: http://en.wikipedia.org/wiki/Datafork_TrueType # Derived from the 'fondu' and 'ufond' source code (fondu.sf.net). 'sfnt' is # TrueType; 'POST' is PostScript. 'FONT' and 'NFNT' sometimes appear, but I # don't know what they mean. 0 belong 0x100 >(0x4.L+24) beshort x >>&4 belong 0x73666e74 Mac OSX datafork font, TrueType >>&4 belong 0x464f4e54 Mac OSX datafork font, 'FONT' >>&4 belong 0x4e464e54 Mac OSX datafork font, 'NFNT' >>&4 belong 0x504f5354 Mac OSX datafork font, PostScript Index: releng/10.1/contrib/file/magic/Magdir/meteorological =================================================================== --- releng/10.1/contrib/file/magic/Magdir/meteorological (nonexistent) +++ releng/10.1/contrib/file/magic/Magdir/meteorological (revision 284193) @@ -0,0 +1,49 @@ + +#------------------------------------------------------------------------------ +# $File: meteorological,v 1.1 2014/08/04 06:26:16 christos Exp $ +# rinex: file(1) magic for RINEX files +# http://igscb.jpl.nasa.gov/igscb/data/format/rinex210.txt +# ftp://cddis.gsfc.nasa.gov/pub/reports/formats/rinex300.pdf +# data for testing: ftp://cddis.gsfc.nasa.gov/pub/gps/data +60 string RINEX +>80 search/256 XXRINEXB RINEX Data, GEO SBAS Broadcast +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/broadcast +>80 search/256 XXRINEXD RINEX Data, Observation (Hatanaka comp) +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/observation +>80 search/256 XXRINEXC RINEX Data, Clock +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/clock +>80 search/256 XXRINEXH RINEX Data, GEO SBAS Navigation +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/navigation +>80 search/256 XXRINEXG RINEX Data, GLONASS Navigation +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/navigation +>80 search/256 XXRINEXL RINEX Data, Galileo Navigation +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/navigation +>80 search/256 XXRINEXM RINEX Data, Meteorological +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/meteorological +>80 search/256 XXRINEXN RINEX Data, Navigation +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/navigation +>80 search/256 XXRINEXO RINEX Data, Observation +>>&32 string x \b, date %15.15s +>>5 string x \b, version %6.6s +!:mime rinex/observation + +# https://en.wikipedia.org/wiki/GRIB +0 string GRIB +>7 byte =1 Gridded binary (GRIB) version 1 +>7 byte =2 Gridded binary (GRIB) version 2 Index: releng/10.1/contrib/file/magic/Magdir/msooxml =================================================================== --- releng/10.1/contrib/file/magic/Magdir/msooxml (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/msooxml (revision 284193) @@ -1,36 +1,36 @@ #------------------------------------------------------------------------------ -# $File: msooxml,v 1.4 2014/01/06 18:16:24 rrt Exp $ +# $File: msooxml,v 1.5 2014/08/05 07:38:45 christos Exp $ # msooxml: file(1) magic for Microsoft Office XML # From: Ralf Brown # .docx, .pptx, and .xlsx are XML plus other files inside a ZIP # archive. The first member file is normally "[Content_Types].xml". # but some libreoffice generated files put this later. Perhaps skip # the "[Content_Types].xml" test? # Since MSOOXML doesn't have anything like the uncompressed "mimetype" # file of ePub or OpenDocument, we'll have to scan for a filename # which can distinguish between the three types # start by checking for ZIP local file header signature 0 string PK\003\004 !:strength +10 # make sure the first file is correct ->0x1E regex \[Content_Types\]\.xml|_rels/\.rels +>0x1E regex \\[Content_Types\\]\\.xml|_rels/\\.rels # skip to the second local file header # since some documents include a 520-byte extra field following the file # header, we need to scan for the next header >>(18.l+49) search/2000 PK\003\004 # now skip to the *third* local file header; again, we need to scan due to a # 520-byte extra field following the file header >>>&26 search/1000 PK\003\004 # and check the subdirectory name to determine which type of OOXML # file we have. Correct the mimetype with the registered ones: # http://technet.microsoft.com/en-us/library/cc179224.aspx >>>>&26 string word/ Microsoft Word 2007+ !:mime application/vnd.openxmlformats-officedocument.wordprocessingml.document >>>>&26 string ppt/ Microsoft PowerPoint 2007+ !:mime application/vnd.openxmlformats-officedocument.presentationml.presentation >>>>&26 string xl/ Microsoft Excel 2007+ !:mime application/vnd.openxmlformats-officedocument.spreadsheetml.sheet >>>>&26 default x Microsoft OOXML Index: releng/10.1/contrib/file/magic/Magdir/netbsd =================================================================== --- releng/10.1/contrib/file/magic/Magdir/netbsd (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/netbsd (revision 284193) @@ -1,284 +1,288 @@ #------------------------------------------------------------------------------ -# $File: netbsd,v 1.21 2014/03/29 15:40:34 christos Exp $ +# $File: netbsd,v 1.22 2014/12/08 20:53:52 christos Exp $ # netbsd: file(1) magic for NetBSD objects # # All new-style magic numbers are in network byte order. # The old-style magic numbers are indistinguishable from the same magic # numbers used in other systems, and are handled, for all those systems, # in aout. # 0 belong&0377777777 041400413 a.out NetBSD/i386 demand paged >0 byte &0x80 >>20 lelong <4096 shared library >>20 lelong =4096 dynamically linked executable >>20 lelong >4096 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 041400410 a.out NetBSD/i386 pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 041400407 a.out NetBSD/i386 >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 lelong !0 executable >>20 lelong =0 object file >16 lelong >0 not stripped 0 belong&0377777777 041400507 a.out NetBSD/i386 core >12 string >\0 from '%s' >32 lelong !0 (signal %d) 0 belong&0377777777 041600413 a.out NetBSD/m68k demand paged >0 byte &0x80 >>20 belong <8192 shared library >>20 belong =8192 dynamically linked executable >>20 belong >8192 dynamically linked executable >0 byte ^0x80 executable >16 belong >0 not stripped 0 belong&0377777777 041600410 a.out NetBSD/m68k pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 belong >0 not stripped 0 belong&0377777777 041600407 a.out NetBSD/m68k >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 belong !0 executable >>20 belong =0 object file >16 belong >0 not stripped 0 belong&0377777777 041600507 a.out NetBSD/m68k core >12 string >\0 from '%s' >32 belong !0 (signal %d) 0 belong&0377777777 042000413 a.out NetBSD/m68k4k demand paged >0 byte &0x80 >>20 belong <4096 shared library >>20 belong =4096 dynamically linked executable >>20 belong >4096 dynamically linked executable >0 byte ^0x80 executable >16 belong >0 not stripped 0 belong&0377777777 042000410 a.out NetBSD/m68k4k pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 belong >0 not stripped 0 belong&0377777777 042000407 a.out NetBSD/m68k4k >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 belong !0 executable >>20 belong =0 object file >16 belong >0 not stripped 0 belong&0377777777 042000507 a.out NetBSD/m68k4k core >12 string >\0 from '%s' >32 belong !0 (signal %d) 0 belong&0377777777 042200413 a.out NetBSD/ns32532 demand paged >0 byte &0x80 >>20 lelong <4096 shared library >>20 lelong =4096 dynamically linked executable >>20 lelong >4096 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 042200410 a.out NetBSD/ns32532 pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 042200407 a.out NetBSD/ns32532 >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 lelong !0 executable >>20 lelong =0 object file >16 lelong >0 not stripped 0 belong&0377777777 042200507 a.out NetBSD/ns32532 core >12 string >\0 from '%s' >32 lelong !0 (signal %d) 0 belong&0377777777 045200507 a.out NetBSD/powerpc core >12 string >\0 from '%s' 0 belong&0377777777 042400413 a.out NetBSD/SPARC demand paged >0 byte &0x80 >>20 belong <8192 shared library >>20 belong =8192 dynamically linked executable >>20 belong >8192 dynamically linked executable >0 byte ^0x80 executable >16 belong >0 not stripped 0 belong&0377777777 042400410 a.out NetBSD/SPARC pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 belong >0 not stripped 0 belong&0377777777 042400407 a.out NetBSD/SPARC >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 belong !0 executable >>20 belong =0 object file >16 belong >0 not stripped 0 belong&0377777777 042400507 a.out NetBSD/SPARC core >12 string >\0 from '%s' >32 belong !0 (signal %d) 0 belong&0377777777 042600413 a.out NetBSD/pmax demand paged >0 byte &0x80 >>20 lelong <4096 shared library >>20 lelong =4096 dynamically linked executable >>20 lelong >4096 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 042600410 a.out NetBSD/pmax pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 042600407 a.out NetBSD/pmax >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 lelong !0 executable >>20 lelong =0 object file >16 lelong >0 not stripped 0 belong&0377777777 042600507 a.out NetBSD/pmax core >12 string >\0 from '%s' >32 lelong !0 (signal %d) 0 belong&0377777777 043000413 a.out NetBSD/vax 1k demand paged >0 byte &0x80 >>20 lelong <4096 shared library >>20 lelong =4096 dynamically linked executable >>20 lelong >4096 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 043000410 a.out NetBSD/vax 1k pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 043000407 a.out NetBSD/vax 1k >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 lelong !0 executable >>20 lelong =0 object file >16 lelong >0 not stripped 0 belong&0377777777 043000507 a.out NetBSD/vax 1k core >12 string >\0 from '%s' >32 lelong !0 (signal %d) 0 belong&0377777777 045400413 a.out NetBSD/vax 4k demand paged >0 byte &0x80 >>20 lelong <4096 shared library >>20 lelong =4096 dynamically linked executable >>20 lelong >4096 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 045400410 a.out NetBSD/vax 4k pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 045400407 a.out NetBSD/vax 4k >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 lelong !0 executable >>20 lelong =0 object file >16 lelong >0 not stripped 0 belong&0377777777 045400507 a.out NetBSD/vax 4k core >12 string >\0 from '%s' >32 lelong !0 (signal %d) # NetBSD/alpha does not support (and has never supported) a.out objects, # so no rules are provided for them. NetBSD/alpha ELF objects are # dealt with in "elf". 0 lelong 0x00070185 ECOFF NetBSD/alpha binary >10 leshort 0x0001 not stripped >10 leshort 0x0000 stripped 0 belong&0377777777 043200507 a.out NetBSD/alpha core >12 string >\0 from '%s' >32 lelong !0 (signal %d) 0 belong&0377777777 043400413 a.out NetBSD/mips demand paged >0 byte &0x80 >>20 belong <8192 shared library >>20 belong =8192 dynamically linked executable >>20 belong >8192 dynamically linked executable >0 byte ^0x80 executable >16 belong >0 not stripped 0 belong&0377777777 043400410 a.out NetBSD/mips pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 belong >0 not stripped 0 belong&0377777777 043400407 a.out NetBSD/mips >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 belong !0 executable >>20 belong =0 object file >16 belong >0 not stripped 0 belong&0377777777 043400507 a.out NetBSD/mips core >12 string >\0 from '%s' >32 belong !0 (signal %d) 0 belong&0377777777 043600413 a.out NetBSD/arm32 demand paged >0 byte &0x80 >>20 lelong <4096 shared library >>20 lelong =4096 dynamically linked executable >>20 lelong >4096 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 043600410 a.out NetBSD/arm32 pure >0 byte &0x80 dynamically linked executable >0 byte ^0x80 executable >16 lelong >0 not stripped 0 belong&0377777777 043600407 a.out NetBSD/arm32 >0 byte &0x80 dynamically linked executable >0 byte ^0x80 >>0 byte &0x40 position independent >>20 lelong !0 executable >>20 lelong =0 object file >16 lelong >0 not stripped # NetBSD/arm26 has always used ELF objects, but it shares a core file # format with NetBSD/arm32. 0 belong&0377777777 043600507 a.out NetBSD/arm core >12 string >\0 from '%s' >32 lelong !0 (signal %d) # Kernel core dump format 0 belong&0x0000ffff 0x00008fca NetBSD kernel core file >0 belong&0x03ff0000 0x00000000 \b, Unknown ->0 belong&0x03ff0000 0x00001000 \b, sun 68010/68020 +>0 belong&0x03ff0000 0x00010000 \b, sun 68010/68020 >0 belong&0x03ff0000 0x00020000 \b, sun 68020 >0 belong&0x03ff0000 0x00640000 \b, 386 PC >0 belong&0x03ff0000 0x00860000 \b, i386 BSD >0 belong&0x03ff0000 0x00870000 \b, m68k BSD (8K pages) >0 belong&0x03ff0000 0x00880000 \b, m68k BSD (4K pages) >0 belong&0x03ff0000 0x00890000 \b, ns32532 BSD >0 belong&0x03ff0000 0x008a0000 \b, SPARC/32 BSD >0 belong&0x03ff0000 0x008b0000 \b, pmax BSD >0 belong&0x03ff0000 0x008c0000 \b, vax BSD (1K pages) >0 belong&0x03ff0000 0x008d0000 \b, alpha BSD >0 belong&0x03ff0000 0x008e0000 \b, mips BSD (Big Endian) >0 belong&0x03ff0000 0x008f0000 \b, arm6 BSD >0 belong&0x03ff0000 0x00900000 \b, m68k BSD (2K pages) >0 belong&0x03ff0000 0x00910000 \b, sh3 BSD ->0 belong&0x03ff0000 0x00920000 \b, ppc BSD (Big Endian) ->0 belong&0x03ff0000 0x00930000 \b, vax BSD (4K pages) ->0 belong&0x03ff0000 0x00940000 \b, mips1 BSD ->0 belong&0x03ff0000 0x00950000 \b, mips2 BSD ->0 belong&0x03ff0000 0x00960000 \b, parisc BSD ->0 belong&0x03ff0000 0x00970000 \b, sh5/64 BSD ->0 belong&0x03ff0000 0x00980000 \b, SPARC/64 BSD ->0 belong&0x03ff0000 0x00990000 \b, amd64 BSD ->0 belong&0x03ff0000 0x009a0000 \b, hp200 (68010) BSD ->0 belong&0x03ff0000 0x009b0000 \b, hp300 (68020+68881) BSD ->0 belong&0x03ff0000 0x009b0000 \b, hp300 (68020+68881) BSD ->0 belong&0x03ff0000 0x00c80000 \b, hp200 ->0 belong&0x03ff0000 0x020b0000 \b, hp300 (68020+68881) HP-UX ->0 belong&0x03ff0000 0x020c0000 \b, hp300 (68020+68881) HP-UX +>0 belong&0x03ff0000 0x00950000 \b, ppc BSD (Big Endian) +>0 belong&0x03ff0000 0x00960000 \b, vax BSD (4K pages) +>0 belong&0x03ff0000 0x00970000 \b, mips1 BSD +>0 belong&0x03ff0000 0x00980000 \b, mips2 BSD +>0 belong&0x03ff0000 0x00990000 \b, m88k BSD +>0 belong&0x03ff0000 0x00920000 \b, parisc BSD +>0 belong&0x03ff0000 0x009b0000 \b, sh5/64 BSD +>0 belong&0x03ff0000 0x009c0000 \b, SPARC/64 BSD +>0 belong&0x03ff0000 0x009d0000 \b, amd64 BSD +>0 belong&0x03ff0000 0x009e0000 \b, sh5/32 BSD +>0 belong&0x03ff0000 0x009f0000 \b, ia64 BSD +>0 belong&0x03ff0000 0x00b70000 \b, aarch64 BSD +>0 belong&0x03ff0000 0x00b80000 \b, or1k BSD +>0 belong&0x03ff0000 0x00b90000 \b, Risk-V BSD +>0 belong&0x03ff0000 0x00c80000 \b, hp200 BSD +>0 belong&0x03ff0000 0x012c0000 \b, hp300 BSD +>0 belong&0x03ff0000 0x020b0000 \b, hp800 HP-UX +>0 belong&0x03ff0000 0x020c0000 \b, hp200/hp300 HP-UX >0 belong&0xfc000000 0x04000000 \b, CPU >0 belong&0xfc000000 0x08000000 \b, DATA >0 belong&0xfc000000 0x10000000 \b, STACK >4 leshort x \b, (headersize = %d >6 leshort x \b, segmentsize = %d >6 lelong x \b, segments = %d) Index: releng/10.1/contrib/file/magic/Magdir/pascal =================================================================== --- releng/10.1/contrib/file/magic/Magdir/pascal (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/pascal (revision 284193) @@ -1,10 +1,10 @@ #------------------------------------------------------------------------------ -# $File: pascal,v 1.1 2011/12/08 12:12:46 rrt Exp $ +# $File: pascal,v 1.2 2014/07/14 14:21:33 rrt Exp $ # pascal: file(1) magic for Pascal source # 0 search/8192 (input, Pascal source text !:mime text/x-pascal -0 regex \^program Pascal source text -!:mime text/x-pascal -0 regex \^record Pascal source text -!:mime text/x-pascal +#0 regex \^program Pascal source text +#!:mime text/x-pascal +#0 regex \^record Pascal source text +#!:mime text/x-pascal Index: releng/10.1/contrib/file/magic/Magdir/pgp =================================================================== --- releng/10.1/contrib/file/magic/Magdir/pgp (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/pgp (revision 284193) @@ -1,27 +1,469 @@ #------------------------------------------------------------------------------ -# $File: pgp,v 1.9 2009/09/19 16:28:11 christos Exp $ +# $File: pgp,v 1.11 2014/11/11 21:32:38 christos Exp $ # pgp: file(1) magic for Pretty Good Privacy # see http://lists.gnupg.org/pipermail/gnupg-devel/1999-September/016052.html # 0 beshort 0x9900 PGP key public ring !:mime application/x-pgp-keyring 0 beshort 0x9501 PGP key security ring !:mime application/x-pgp-keyring 0 beshort 0x9500 PGP key security ring !:mime application/x-pgp-keyring 0 beshort 0xa600 PGP encrypted data #!:mime application/pgp-encrypted #0 string -----BEGIN\040PGP text/PGP armored data !:mime text/PGP # encoding: armored data #>15 string PUBLIC\040KEY\040BLOCK- public key block #>15 string MESSAGE- message #>15 string SIGNED\040MESSAGE- signed message #>15 string PGP\040SIGNATURE- signature 2 string ---BEGIN\ PGP\ PUBLIC\ KEY\ BLOCK- PGP public key block !:mime application/pgp-keys +>10 search/100 \n\n +>>&0 use pgp 0 string -----BEGIN\040PGP\40MESSAGE- PGP message !:mime application/pgp +>10 search/100 \n\n +>>&0 use pgp 0 string -----BEGIN\040PGP\40SIGNATURE- PGP signature !:mime application/pgp-signature +>10 search/100 \n\n +>>&0 use pgp + +# Decode the type of the packet based on it's base64 encoding. +# Idea from Mark Martinec +# The specification is in RFC 4880, section 4.2 and 4.3: +# http://tools.ietf.org/html/rfc4880#section-4.2 + +0 name pgp +>0 byte 0x67 Reserved (old) +>0 byte 0x68 Public-Key Encrypted Session Key (old) +>0 byte 0x69 Signature (old) +>0 byte 0x6a Symmetric-Key Encrypted Session Key (old) +>0 byte 0x6b One-Pass Signature (old) +>0 byte 0x6c Secret-Key (old) +>0 byte 0x6d Public-Key (old) +>0 byte 0x6e Secret-Subkey (old) +>0 byte 0x6f Compressed Data (old) +>0 byte 0x70 Symmetrically Encrypted Data (old) +>0 byte 0x71 Marker (old) +>0 byte 0x72 Literal Data (old) +>0 byte 0x73 Trust (old) +>0 byte 0x74 User ID (old) +>0 byte 0x75 Public-Subkey (old) +>0 byte 0x76 Unused (old) +>0 byte 0x77 +>>1 byte&0xc0 0x00 Reserved +>>1 byte&0xc0 0x40 Public-Key Encrypted Session Key +>>1 byte&0xc0 0x80 Signature +>>1 byte&0xc0 0xc0 Symmetric-Key Encrypted Session Key +>0 byte 0x78 +>>1 byte&0xc0 0x00 One-Pass Signature +>>1 byte&0xc0 0x40 Secret-Key +>>1 byte&0xc0 0x80 Public-Key +>>1 byte&0xc0 0xc0 Secret-Subkey +>0 byte 0x79 +>>1 byte&0xc0 0x00 Compressed Data +>>1 byte&0xc0 0x40 Symmetrically Encrypted Data +>>1 byte&0xc0 0x80 Marker +>>1 byte&0xc0 0xc0 Literal Data +>0 byte 0x7a +>>1 byte&0xc0 0x00 Trust +>>1 byte&0xc0 0x40 User ID +>>1 byte&0xc0 0x80 Public-Subkey +>>1 byte&0xc0 0xc0 Unused [z%x] +>0 byte 0x30 +>>1 byte&0xc0 0x00 Unused [0%x] +>>1 byte&0xc0 0x40 User Attribute +>>1 byte&0xc0 0x80 Sym. Encrypted and Integrity Protected Data +>>1 byte&0xc0 0xc0 Modification Detection Code + +# magic signatures to detect PGP crypto material (from stef) +# detects and extracts metadata from: +# - symmetric encrypted packet header +# - RSA (e=65537) secret (sub-)keys + +# 1024b RSA encrypted data + +0 string \x84\x8c\x03 PGP RSA encrypted session key - +>3 lelong x keyid: %X +>7 lelong x %X +>11 byte 0x01 RSA (Encrypt or Sign) 1024b +>11 byte 0x02 RSA Encrypt-Only 1024b +>12 string \x04\x00 +>12 string \x03\xff +>12 string \x03\xfe +>12 string \x03\xfd +>12 string \x03\xfc +>12 string \x03\xfb +>12 string \x03\xfa +>12 string \x03\xf9 +>142 byte 0xd2 . + +# 2048b RSA encrypted data + +0 string \x85\x01\x0c\x03 PGP RSA encrypted session key - +>4 lelong x keyid: %X +>8 lelong x %X +>12 byte 0x01 RSA (Encrypt or Sign) 2048b +>12 byte 0x02 RSA Encrypt-Only 2048b +>13 string \x08\x00 +>13 string \x07\xff +>13 string \x07\xfe +>13 string \x07\xfd +>13 string \x07\xfc +>13 string \x07\xfb +>13 string \x07\xfa +>13 string \x07\xf9 +>271 byte 0xd2 . + +# 3072b RSA encrypted data + +0 string \x85\x01\x8c\x03 PGP RSA encrypted session key - +>4 lelong x keyid: %X +>8 lelong x %X +>12 byte 0x01 RSA (Encrypt or Sign) 3072b +>12 byte 0x02 RSA Encrypt-Only 3072b +>13 string \x0c\x00 +>13 string \x0b\xff +>13 string \x0b\xfe +>13 string \x0b\xfd +>13 string \x0b\xfc +>13 string \x0b\xfb +>13 string \x0b\xfa +>13 string \x0b\xf9 +>399 byte 0xd2 . + +# 3072b RSA encrypted data + +0 string \x85\x02\x0c\x03 PGP RSA encrypted session key - +>4 lelong x keyid: %X +>8 lelong x %X +>12 byte 0x01 RSA (Encrypt or Sign) 4096b +>12 byte 0x02 RSA Encrypt-Only 4096b +>13 string \x10\x00 +>13 string \x0f\xff +>13 string \x0f\xfe +>13 string \x0f\xfd +>13 string \x0f\xfc +>13 string \x0f\xfb +>13 string \x0f\xfa +>13 string \x0f\xf9 +>527 byte 0xd2 . + +# 4096b RSA encrypted data + +0 string \x85\x04\x0c\x03 PGP RSA encrypted session key - +>4 lelong x keyid: %X +>8 lelong x %X +>12 byte 0x01 RSA (Encrypt or Sign) 8129b +>12 byte 0x02 RSA Encrypt-Only 8129b +>13 string \x20\x00 +>13 string \x1f\xff +>13 string \x1f\xfe +>13 string \x1f\xfd +>13 string \x1f\xfc +>13 string \x1f\xfb +>13 string \x1f\xfa +>13 string \x1f\xf9 +>1039 byte 0xd2 . + +# crypto algo mapper + +0 name crypto +>0 byte 0x00 Plaintext or unencrypted data +>0 byte 0x01 IDEA +>0 byte 0x02 TripleDES +>0 byte 0x03 CAST5 (128 bit key) +>0 byte 0x04 Blowfish (128 bit key, 16 rounds) +>0 byte 0x07 AES with 128-bit key +>0 byte 0x08 AES with 192-bit key +>0 byte 0x09 AES with 256-bit key +>0 byte 0x0a Twofish with 256-bit key + +# hash algo mapper + +0 name hash +>0 byte 0x01 MD5 +>0 byte 0x02 SHA-1 +>0 byte 0x03 RIPE-MD/160 +>0 byte 0x08 SHA256 +>0 byte 0x09 SHA384 +>0 byte 0x0a SHA512 +>0 byte 0x0b SHA224 + +# pgp symmetric encrypted data + +0 byte 0x8c PGP symmetric key encrypted data - +>1 byte 0x0d +>1 byte 0x0c +>2 byte 0x04 +>3 use crypto +>4 byte 0x01 salted - +>>5 use hash +>>14 byte 0xd2 . +>>14 byte 0xc9 . +>4 byte 0x03 salted & iterated - +>>5 use hash +>>15 byte 0xd2 . +>>15 byte 0xc9 . + +# encrypted keymaterial needs s2k & can be checksummed/hashed + +0 name chkcrypto +>0 use crypto +>1 byte 0x00 Simple S2K +>1 byte 0x01 Salted S2K +>1 byte 0x03 Salted&Iterated S2K +>2 use hash + +# all PGP keys start with this prolog +# containing version, creation date, and purpose + +0 name keyprolog +>0 byte 0x04 +>1 beldate x created on %s - +>5 byte 0x01 RSA (Encrypt or Sign) +>5 byte 0x02 RSA Encrypt-Only + +# end of secret keys known signature +# contains e=65537 and the prolog to +# the encrypted parameters + +0 name keyend +>0 string \x00\x11\x01\x00\x01 e=65537 +>5 use crypto +>5 byte 0xff checksummed +>>6 use chkcrypto +>5 byte 0xfe hashed +>>6 use chkcrypto + +# PGP secret keys contain also the public parts +# these vary by bitsize of the key + +0 name x1024 +>0 use keyprolog +>6 string \x03\xfe +>6 string \x03\xff +>6 string \x04\x00 +>136 use keyend + +0 name x2048 +>0 use keyprolog +>6 string \x80\x00 +>6 string \x07\xfe +>6 string \x07\xff +>264 use keyend + +0 name x3072 +>0 use keyprolog +>6 string \x0b\xfe +>6 string \x0b\xff +>6 string \x0c\x00 +>392 use keyend + +0 name x4096 +>0 use keyprolog +>6 string \x10\x00 +>6 string \x0f\xfe +>6 string \x0f\xff +>520 use keyend + +# \x00|\x1f[\xfe\xff]).{1024})' +0 name x8192 +>0 use keyprolog +>6 string \x20\x00 +>6 string \x1f\xfe +>6 string \x1f\xff +>1032 use keyend + +# depending on the size of the pkt +# we branch into the proper key size +# signatures defined as x{keysize} + +>0 name pgpkey +>0 string \x01\xd8 1024b +>>2 use x1024 +>0 string \x01\xeb 1024b +>>2 use x1024 +>0 string \x01\xfb 1024b +>>2 use x1024 +>0 string \x01\xfd 1024b +>>2 use x1024 +>0 string \x01\xf3 1024b +>>2 use x1024 +>0 string \x01\xee 1024b +>>2 use x1024 +>0 string \x01\xfe 1024b +>>2 use x1024 +>0 string \x01\xf4 1024b +>>2 use x1024 +>0 string \x02\x0d 1024b +>>2 use x1024 +>0 string \x02\x03 1024b +>>2 use x1024 +>0 string \x02\x05 1024b +>>2 use x1024 +>0 string \x02\x15 1024b +>>2 use x1024 +>0 string \x02\x00 1024b +>>2 use x1024 +>0 string \x02\x10 1024b +>>2 use x1024 +>0 string \x02\x04 1024b +>>2 use x1024 +>0 string \x02\x06 1024b +>>2 use x1024 +>0 string \x02\x16 1024b +>>2 use x1024 +>0 string \x03\x98 2048b +>>2 use x2048 +>0 string \x03\xab 2048b +>>2 use x2048 +>0 string \x03\xbb 2048b +>>2 use x2048 +>0 string \x03\xbd 2048b +>>2 use x2048 +>0 string \x03\xcd 2048b +>>2 use x2048 +>0 string \x03\xb3 2048b +>>2 use x2048 +>0 string \x03\xc3 2048b +>>2 use x2048 +>0 string \x03\xc5 2048b +>>2 use x2048 +>0 string \x03\xd5 2048b +>>2 use x2048 +>0 string \x03\xae 2048b +>>2 use x2048 +>0 string \x03\xbe 2048b +>>2 use x2048 +>0 string \x03\xc0 2048b +>>2 use x2048 +>0 string \x03\xd0 2048b +>>2 use x2048 +>0 string \x03\xb4 2048b +>>2 use x2048 +>0 string \x03\xc4 2048b +>>2 use x2048 +>0 string \x03\xc6 2048b +>>2 use x2048 +>0 string \x03\xd6 2048b +>>2 use x2048 +>0 string \x05X 3072b +>>2 use x3072 +>0 string \x05k 3072b +>>2 use x3072 +>0 string \x05{ 3072b +>>2 use x3072 +>0 string \x05} 3072b +>>2 use x3072 +>0 string \x05\x8d 3072b +>>2 use x3072 +>0 string \x05s 3072b +>>2 use x3072 +>0 string \x05\x83 3072b +>>2 use x3072 +>0 string \x05\x85 3072b +>>2 use x3072 +>0 string \x05\x95 3072b +>>2 use x3072 +>0 string \x05n 3072b +>>2 use x3072 +>0 string \x05\x7e 3072b +>>2 use x3072 +>0 string \x05\x80 3072b +>>2 use x3072 +>0 string \x05\x90 3072b +>>2 use x3072 +>0 string \x05t 3072b +>>2 use x3072 +>0 string \x05\x84 3072b +>>2 use x3072 +>0 string \x05\x86 3072b +>>2 use x3072 +>0 string \x05\x96 3072b +>>2 use x3072 +>0 string \x07[ 4096b +>>2 use x4096 +>0 string \x07\x18 4096b +>>2 use x4096 +>0 string \x07+ 4096b +>>2 use x4096 +>0 string \x07; 4096b +>>2 use x4096 +>0 string \x07= 4096b +>>2 use x4096 +>0 string \x07M 4096b +>>2 use x4096 +>0 string \x073 4096b +>>2 use x4096 +>0 string \x07C 4096b +>>2 use x4096 +>0 string \x07E 4096b +>>2 use x4096 +>0 string \x07U 4096b +>>2 use x4096 +>0 string \x07. 4096b +>>2 use x4096 +>0 string \x07> 4096b +>>2 use x4096 +>0 string \x07@ 4096b +>>2 use x4096 +>0 string \x07P 4096b +>>2 use x4096 +>0 string \x074 4096b +>>2 use x4096 +>0 string \x07D 4096b +>>2 use x4096 +>0 string \x07F 4096b +>>2 use x4096 +>0 string \x07V 4096b +>>2 use x4096 +>0 string \x0e[ 8192b +>>2 use x8192 +>0 string \x0e\x18 8192b +>>2 use x8192 +>0 string \x0e+ 8192b +>>2 use x8192 +>0 string \x0e; 8192b +>>2 use x8192 +>0 string \x0e= 8192b +>>2 use x8192 +>0 string \x0eM 8192b +>>2 use x8192 +>0 string \x0e3 8192b +>>2 use x8192 +>0 string \x0eC 8192b +>>2 use x8192 +>0 string \x0eE 8192b +>>2 use x8192 +>0 string \x0eU 8192b +>>2 use x8192 +>0 string \x0e. 8192b +>>2 use x8192 +>0 string \x0e> 8192b +>>2 use x8192 +>0 string \x0e@ 8192b +>>2 use x8192 +>0 string \x0eP 8192b +>>2 use x8192 +>0 string \x0e4 8192b +>>2 use x8192 +>0 string \x0eD 8192b +>>2 use x8192 +>0 string \x0eF 8192b +>>2 use x8192 +>0 string \x0eV 8192b +>>2 use x8192 + +# PGP RSA (e=65537) secret (sub-)key header + +0 byte 0x95 PGP Secret Key - +>1 use pgpkey +0 byte 0x97 PGP Secret Sub-key - +>1 use pgpkey +0 byte 0x9d PGP Secret Sub-key - +>1 use pgpkey Index: releng/10.1/contrib/file/magic/Magdir/python =================================================================== --- releng/10.1/contrib/file/magic/Magdir/python (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/python (revision 284193) @@ -1,67 +1,68 @@ #------------------------------------------------------------------------------ -# $File: python,v 1.25 2014/05/06 16:08:32 christos Exp $ +# $File: python,v 1.26 2014/08/04 05:58:40 christos Exp $ # python: file(1) magic for python # # Outlook puts """ too for urgent messages # From: David Necas # often the module starts with a multiline string 0 string/t """ Python script text executable # MAGIC as specified in Python/import.c (1.5 to 2.7a0 and 3.1a0, assuming # that Py_UnicodeFlag is off for Python 2) # 20121 ( YEAR - 1995 ) + MONTH + DAY (little endian followed by "\r\n" 0 belong 0x994e0d0a python 1.5/1.6 byte-compiled 0 belong 0x87c60d0a python 2.0 byte-compiled 0 belong 0x2aeb0d0a python 2.1 byte-compiled 0 belong 0x2ded0d0a python 2.2 byte-compiled 0 belong 0x3bf20d0a python 2.3 byte-compiled 0 belong 0x6df20d0a python 2.4 byte-compiled 0 belong 0xb3f20d0a python 2.5 byte-compiled 0 belong 0xd1f20d0a python 2.6 byte-compiled 0 belong 0x03f30d0a python 2.7 byte-compiled 0 belong 0x3b0c0d0a python 3.0 byte-compiled 0 belong 0x4f0c0d0a python 3.1 byte-compiled 0 belong 0x6c0c0d0a python 3.2 byte-compiled 0 belong 0x9e0c0d0a python 3.3 byte-compiled +0 belong 0xee0c0d0a python 3.4 byte-compiled 0 search/1/w #!\ /usr/bin/python Python script text executable !:mime text/x-python 0 search/1/w #!\ /usr/local/bin/python Python script text executable !:mime text/x-python 0 search/1 #!/usr/bin/env\ python Python script text executable !:mime text/x-python 0 search/1 #!\ /usr/bin/env\ python Python script text executable !:mime text/x-python # from module.submodule import func1, func2 0 regex \^from\\s+(\\w|\\.)+\\s+import.*$ Python script text executable !:mime text/x-python # def __init__ (self, ...): 0 search/4096 def\ __init__ >&0 search/64 self Python script text executable !:mime text/x-python # comments #0 search/4096 ''' #>&0 regex .*'''$ Python script text executable #!:mime text/x-python #0 search/4096 """ #>&0 regex .*"""$ Python script text executable #!:mime text/x-python # try: # except: or finally: # block 0 search/4096 try: >&0 regex \^\\s*except.*: Python script text executable !:mime text/x-python >&0 search/4096 finally: Python script text executable !:mime text/x-python # def name(args, args): 0 regex \^(\ |\\t){0,50}def\ {1,50}[a-zA-Z]{1,100} >&0 regex \ {0,50}\\(([a-zA-Z]|,|\ ){1,255}\\):$ Python script text executable !:mime text/x-python Index: releng/10.1/contrib/file/magic/Magdir/qt =================================================================== --- releng/10.1/contrib/file/magic/Magdir/qt (nonexistent) +++ releng/10.1/contrib/file/magic/Magdir/qt (revision 284193) @@ -0,0 +1,19 @@ + +#------------------------------------------------------------------------------ +# $File: qt,v 1.2 2014/12/16 19:49:29 christos Exp $ +# qt: file(1) magic for Qt + +# http://doc.qt.io/qt-5/resources.html +0 string \ Qt Resource Collection file + +# https://qt.gitorious.org/qt/qtbase/source/\ +# 5367fa356233da4c0f28172a8f817791525f5457:\ +# src/tools/rcc/rcc.cpp#L840 +0 string qres\0\0 Qt Binary Resource file +0 search/1024 The\040Resource\040Compiler\040for\040Qt Qt C-code resource file + +# https://qt.gitorious.org/qt/qtbase/source/\ +# 5367fa356233da4c0f28172a8f817791525f5457:\ +# src/corelib/kernel/qtranslator.cpp#L62 +0 string \x3c\xb8\x64\x18\xca\xef\x9c\x95 +>8 string \xcd\x21\x1c\xbf\x60\xa1\xbd\xdd Qt Translation file Index: releng/10.1/contrib/file/magic/Magdir/riff =================================================================== --- releng/10.1/contrib/file/magic/Magdir/riff (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/riff (revision 284193) @@ -1,287 +1,308 @@ #------------------------------------------------------------------------------ -# $File: riff,v 1.27 2014/04/30 21:41:02 christos Exp $ +# $File: riff,v 1.30 2014/09/23 17:02:12 christos Exp $ # riff: file(1) magic for RIFF format # See # # http://www.seanet.com/users/matts/riffmci/riffmci.htm # # audio format tag. Assume limits: max 1024 bit, 128 channels, 1 MHz 0 name riff-wave >0 leshort 1 \b, Microsoft PCM >>14 leshort >0 >>>14 leshort <1024 \b, %d bit >0 leshort 2 \b, Microsoft ADPCM >0 leshort 6 \b, ITU G.711 A-law >0 leshort 7 \b, ITU G.711 mu-law >0 leshort 8 \b, Microsoft DTS >0 leshort 17 \b, IMA ADPCM >0 leshort 20 \b, ITU G.723 ADPCM (Yamaha) >0 leshort 49 \b, GSM 6.10 >0 leshort 64 \b, ITU G.721 ADPCM >0 leshort 80 \b, MPEG >0 leshort 85 \b, MPEG Layer 3 >0 leshort 0x2001 \b, DTS >2 leshort =1 \b, mono >2 leshort =2 \b, stereo >2 leshort >2 >>2 leshort <128 \b, %d channels >4 lelong >0 >>4 lelong <1000000 %d Hz # try to find "fmt " 0 name riff-walk >0 string fmt\x20 >>4 lelong <0x80 >>>8 use riff-wave >0 string LIST >>&(4.l+4) use riff-walk >0 string DISP >>&(4.l+4) use riff-walk >0 string bext >>&(4.l+4) use riff-walk >0 string Fake >>&(4.l+4) use riff-walk >0 string fact >>&(4.l+4) use riff-walk +>0 string VP8 +>>11 byte 0x9d +>>>12 byte 0x01 +>>>>13 byte 0x2a \b, VP8 encoding +>>>>>14 leshort&0x3fff x \b, %d +>>>>>16 leshort&0x3fff x \bx%d, Scaling: +>>>>>14 leshort&0xc000 0x0000 \b [none] +>>>>>14 leshort&0xc000 0x1000 \b [5/4] +>>>>>14 leshort&0xc000 0x2000 \b [5/3] +>>>>>14 leshort&0xc000 0x3000 \b [2] +>>>>>14 leshort&0xc000 0x0000 \bx[none] +>>>>>14 leshort&0xc000 0x1000 \bx[5/4] +>>>>>14 leshort&0xc000 0x2000 \bx[5/3] +>>>>>14 leshort&0xc000 0x3000 \bx[2] +>>>>>15 byte&0x80 =0x00 \b, YUV color +>>>>>15 byte&0x80 =0x80 \b, bad color specification +>>>>>15 byte&0x40 =0x40 \b, no clamping required +>>>>>15 byte&0x40 =0x00 \b, decoders should clamp +#>0 string x we got %s +#>>&(4.l+4) use riff-walk # AVI section extended by Patrik Radman # 0 string RIFF RIFF (little-endian) data # RIFF Palette format >8 string PAL \b, palette >>16 leshort x \b, version %d >>18 leshort x \b, %d entries # RIFF Device Independent Bitmap format >8 string RDIB \b, device-independent bitmap >>16 string BM >>>30 leshort 12 \b, OS/2 1.x format >>>>34 leshort x \b, %d x >>>>36 leshort x %d >>>30 leshort 64 \b, OS/2 2.x format >>>>34 leshort x \b, %d x >>>>36 leshort x %d >>>30 leshort 40 \b, Windows 3.x format >>>>34 lelong x \b, %d x >>>>38 lelong x %d x >>>>44 leshort x %d # RIFF MIDI format >8 string RMID \b, MIDI # RIFF Multimedia Movie File format >8 string RMMP \b, multimedia movie # RIFF wrapper for MP3 >8 string RMP3 \b, MPEG Layer 3 audio # Microsoft WAVE format (*.wav) >8 string WAVE \b, WAVE audio !:mime audio/x-wav >>12 string >\0 >>>12 use riff-walk # Corel Draw Picture >8 string CDRA \b, Corel Draw Picture !:mime image/x-coreldraw >8 string CDR6 \b, Corel Draw Picture, version 6 !:mime image/x-coreldraw >8 string NUNDROOT \b, Steinberg CuBase # AVI == Audio Video Interleave >8 string AVI\040 \b, AVI !:mime video/x-msvideo >>12 string LIST >>>20 string hdrlavih >>>>&36 lelong x \b, %u x >>>>&40 lelong x %u, >>>>&4 lelong >1000000 <1 fps, >>>>&4 lelong 1000000 1.00 fps, >>>>&4 lelong 500000 2.00 fps, >>>>&4 lelong 333333 3.00 fps, >>>>&4 lelong 250000 4.00 fps, >>>>&4 lelong 200000 5.00 fps, >>>>&4 lelong 166667 6.00 fps, >>>>&4 lelong 142857 7.00 fps, >>>>&4 lelong 125000 8.00 fps, >>>>&4 lelong 111111 9.00 fps, >>>>&4 lelong 100000 10.00 fps, # ]9.9,10.1[ >>>>&4 lelong <101010 >>>>>&-4 lelong >99010 >>>>>>&-4 lelong !100000 ~10 fps, >>>>&4 lelong 83333 12.00 fps, # ]11.9,12.1[ >>>>&4 lelong <84034 >>>>>&-4 lelong >82645 >>>>>>&-4 lelong !83333 ~12 fps, >>>>&4 lelong 66667 15.00 fps, # ]14.9,15.1[ >>>>&4 lelong <67114 >>>>>&-4 lelong >66225 >>>>>>&-4 lelong !66667 ~15 fps, >>>>&4 lelong 50000 20.00 fps, >>>>&4 lelong 41708 23.98 fps, >>>>&4 lelong 41667 24.00 fps, # ]23.9,24.1[ >>>>&4 lelong <41841 >>>>>&-4 lelong >41494 >>>>>>&-4 lelong !41708 >>>>>>>&-4 lelong !41667 ~24 fps, >>>>&4 lelong 40000 25.00 fps, # ]24.9,25.1[ >>>>&4 lelong <40161 >>>>>&-4 lelong >39841 >>>>>>&-4 lelong !40000 ~25 fps, >>>>&4 lelong 33367 29.97 fps, >>>>&4 lelong 33333 30.00 fps, # ]29.9,30.1[ >>>>&4 lelong <33445 >>>>>&-4 lelong >33223 >>>>>>&-4 lelong !33367 >>>>>>>&-4 lelong !33333 ~30 fps, >>>>&4 lelong <32224 >30 fps, ##>>>>&4 lelong x (%lu) ##>>>>&20 lelong x %lu frames, # Note: The tests below assume that the AVI has 1 or 2 streams, # "vids" optionally followed by "auds". # (Should cover 99.9% of all AVIs.) # assuming avih length = 56 >>>88 string LIST >>>>96 string strlstrh >>>>>108 string vids video: >>>>>>&0 lelong 0 uncompressed # skip past vids strh >>>>>>(104.l+108) string strf >>>>>>>(104.l+132) lelong 1 RLE 8bpp >>>>>>>(104.l+132) string/c cvid Cinepak >>>>>>>(104.l+132) string/c i263 Intel I.263 >>>>>>>(104.l+132) string/c iv32 Indeo 3.2 >>>>>>>(104.l+132) string/c iv41 Indeo 4.1 >>>>>>>(104.l+132) string/c iv50 Indeo 5.0 >>>>>>>(104.l+132) string/c mp42 Microsoft MPEG-4 v2 >>>>>>>(104.l+132) string/c mp43 Microsoft MPEG-4 v3 >>>>>>>(104.l+132) string/c fmp4 FFMpeg MPEG-4 >>>>>>>(104.l+132) string/c mjpg Motion JPEG >>>>>>>(104.l+132) string/c div3 DivX 3 >>>>>>>>112 string/c div3 Low-Motion >>>>>>>>112 string/c div4 Fast-Motion >>>>>>>(104.l+132) string/c divx DivX 4 >>>>>>>(104.l+132) string/c dx50 DivX 5 >>>>>>>(104.l+132) string/c xvid XviD >>>>>>>(104.l+132) string/c h264 H.264 >>>>>>>(104.l+132) string/c wmv3 Windows Media Video 9 >>>>>>>(104.l+132) string/c h264 X.264 or H.264 >>>>>>>(104.l+132) lelong 0 ##>>>>>>>(104.l+132) string x (%.4s) # skip past first (video) LIST >>>>(92.l+96) string LIST >>>>>(92.l+104) string strlstrh >>>>>>(92.l+116) string auds \b, audio: # auds strh length = 56: >>>>>>>(92.l+172) string strf >>>>>>>>(92.l+180) leshort 0x0001 uncompressed PCM >>>>>>>>(92.l+180) leshort 0x0002 ADPCM >>>>>>>>(92.l+180) leshort 0x0006 aLaw >>>>>>>>(92.l+180) leshort 0x0007 uLaw >>>>>>>>(92.l+180) leshort 0x0050 MPEG-1 Layer 1 or 2 >>>>>>>>(92.l+180) leshort 0x0055 MPEG-1 Layer 3 >>>>>>>>(92.l+180) leshort 0x2000 Dolby AC3 >>>>>>>>(92.l+180) leshort 0x0161 DivX ##>>>>>>>>(92.l+180) leshort x (0x%.4x) >>>>>>>>(92.l+182) leshort 1 (mono, >>>>>>>>(92.l+182) leshort 2 (stereo, >>>>>>>>(92.l+182) leshort >2 (%d channels, >>>>>>>>(92.l+184) lelong x %d Hz) # auds strh length = 64: >>>>>>>(92.l+180) string strf >>>>>>>>(92.l+188) leshort 0x0001 uncompressed PCM >>>>>>>>(92.l+188) leshort 0x0002 ADPCM >>>>>>>>(92.l+188) leshort 0x0055 MPEG-1 Layer 3 >>>>>>>>(92.l+188) leshort 0x2000 Dolby AC3 >>>>>>>>(92.l+188) leshort 0x0161 DivX ##>>>>>>>>(92.l+188) leshort x (0x%.4x) >>>>>>>>(92.l+190) leshort 1 (mono, >>>>>>>>(92.l+190) leshort 2 (stereo, >>>>>>>>(92.l+190) leshort >2 (%d channels, >>>>>>>>(92.l+192) lelong x %d Hz) # Animated Cursor format >8 string ACON \b, animated cursor # SoundFont 2 >8 string sfbk SoundFont/Bank # MPEG-1 wrapped in a RIFF, apparently >8 string CDXA \b, wrapped MPEG-1 (CDXA) >8 string 4XMV \b, 4X Movie file # AMV-type AVI file: http://wiki.multimedia.cx/index.php?title=AMV >8 string AMV\040 \b, AMV +>8 string WEBP \b, Web/P image +>>12 use riff-walk # # XXX - some of the below may only appear in little-endian form. # # Also "MV93" appears to be for one form of Macromedia Director # files, and "GDMF" appears to be another multimedia format. # 0 string RIFX RIFF (big-endian) data # RIFF Palette format >8 string PAL \b, palette >>16 beshort x \b, version %d >>18 beshort x \b, %d entries # RIFF Device Independent Bitmap format >8 string RDIB \b, device-independent bitmap >>16 string BM >>>30 beshort 12 \b, OS/2 1.x format >>>>34 beshort x \b, %d x >>>>36 beshort x %d >>>30 beshort 64 \b, OS/2 2.x format >>>>34 beshort x \b, %d x >>>>36 beshort x %d >>>30 beshort 40 \b, Windows 3.x format >>>>34 belong x \b, %d x >>>>38 belong x %d x >>>>44 beshort x %d # RIFF MIDI format >8 string RMID \b, MIDI # RIFF Multimedia Movie File format >8 string RMMP \b, multimedia movie # Microsoft WAVE format (*.wav) >8 string WAVE \b, WAVE audio >>20 leshort 1 \b, Microsoft PCM >>>34 leshort >0 \b, %d bit >>22 beshort =1 \b, mono >>22 beshort =2 \b, stereo >>22 beshort >2 \b, %d channels >>24 belong >0 %d Hz # Corel Draw Picture >8 string CDRA \b, Corel Draw Picture >8 string CDR6 \b, Corel Draw Picture, version 6 # AVI == Audio Video Interleave >8 string AVI\040 \b, AVI # Animated Cursor format >8 string ACON \b, animated cursor # Notation Interchange File Format (big-endian only) >8 string NIFF \b, Notation Interchange File Format # SoundFont 2 >8 string sfbk SoundFont/Bank #------------------------------------------------------------------------------ # Sony Wave64 # see http://www.vcs.de/fileadmin/user_upload/MBS/PDF/Whitepaper/Informations_about_Sony_Wave64.pdf # 128 bit RIFF-GUID { 66666972-912E-11CF-A5D6-28DB04C10000 } in little-endian 0 string riff\x2E\x91\xCF\x11\xA5\xD6\x28\xDB\x04\xC1\x00\x00 Sony Wave64 RIFF data # 128 bit + total file size (64 bits) so 24 bytes # then WAVE-GUID { 65766177-ACF3-11D3-8CD1-00C04F8EDB8A } >24 string wave\xF3\xAC\xD3\x11\x8C\xD1\x00\xC0\x4F\x8E\xDB\x8A \b, WAVE 64 audio !:mime audio/x-w64 # FMT-GUID { 20746D66-ACF3-11D3-8CD1-00C04F8EDB8A } >>40 search/256 fmt\x20\xF3\xAC\xD3\x11\x8C\xD1\x00\xC0\x4F\x8E\xDB\x8A \b >>>&10 leshort =1 \b, mono >>>&10 leshort =2 \b, stereo >>>&10 leshort >2 \b, %d channels >>>&12 lelong >0 %d Hz #------------------------------------------------------------------------------ # MBWF/RF64 # see EBU TECH 3306 http://tech.ebu.ch/docs/tech/tech3306-2009.pdf 0 string RF64\xff\xff\xff\xffWAVEds64 MBWF/RF64 audio !:mime audio/x-wav >40 search/256 fmt\x20 \b >>&6 leshort =1 \b, mono >>&6 leshort =2 \b, stereo >>&6 leshort >2 \b, %d channels >>&8 lelong >0 %d Hz - Index: releng/10.1/contrib/file/magic/Magdir/sequent =================================================================== --- releng/10.1/contrib/file/magic/Magdir/sequent (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/sequent (revision 284193) @@ -1,45 +1,42 @@ #------------------------------------------------------------------------------ -# $File: sequent,v 1.11 2014/06/02 19:27:54 christos Exp $ +# $File: sequent,v 1.12 2014/08/16 16:07:12 christos Exp $ # sequent: file(1) magic for Sequent machines # # Sequent information updated by Don Dwiggins . # For Sequent's multiprocessor systems (incomplete). 0 lelong 0x00ea BALANCE NS32000 .o >16 lelong >0 not stripped >124 lelong >0 version %d 0 lelong 0x10ea BALANCE NS32000 executable (0 @ 0) >16 lelong >0 not stripped >124 lelong >0 version %d 0 lelong 0x20ea BALANCE NS32000 executable (invalid @ 0) >16 lelong >0 not stripped >124 lelong >0 version %d 0 lelong 0x30ea BALANCE NS32000 standalone executable >16 lelong >0 not stripped >124 lelong >0 version %d # # Symmetry information added by Jason Merrill . # Symmetry magic nums will not be reached if DOS COM comes before them; # byte 0xeb is matched before these get a chance. 0 leshort 0x12eb SYMMETRY i386 .o >16 lelong >0 not stripped >124 lelong >0 version %d 0 leshort 0x22eb SYMMETRY i386 executable (0 @ 0) >16 lelong >0 not stripped >124 lelong >0 version %d 0 leshort 0x32eb SYMMETRY i386 executable (invalid @ 0) ->16 lelong >0 not stripped ->124 lelong >0 version %d -0 leshort 0x42eb SYMMETRY i386 standalone executable >16 lelong >0 not stripped >124 lelong >0 version %d # http://en.wikipedia.org/wiki/Sequent_Computer_Systems # below test line conflicts with MS-DOS 2.11 floppies and Acronis loader #0 leshort 0x42eb SYMMETRY i386 standalone executable 0 leshort 0x42eb # skip unlike negative version >124 lelong >-1 # assuming version 28867614 is very low probable >>124 lelong !28867614 SYMMETRY i386 standalone executable >>>16 lelong >0 not stripped >>>124 lelong >0 version %d Index: releng/10.1/contrib/file/magic/Magdir/sereal =================================================================== --- releng/10.1/contrib/file/magic/Magdir/sereal (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/sereal (revision 284193) @@ -1,24 +1,25 @@ + #------------------------------------------------------------------------------ -# $File: sereal,v 1.1 2014/05/14 23:04:59 christos Exp $ +# $File: sereal,v 1.2 2014/11/11 20:10:49 christos Exp $ # sereal: file(1) magic the Sereal binary serialization format # # From: Ævar Arnfjörð Bjarmason # # See the specification of the format at # https://github.com/Sereal/Sereal/blob/master/sereal_spec.pod#document-header-format # # I'd have liked to do the byte&0xF0 matching against 0, 1, 2 ... by # doing (byte&0xF0)>>4 here, but unfortunately that's not # supported. So when we print out a message about an unknown format # we'll print out e.g. 0x30 instead of the more human-readable # 0x30>>4. # # See https://github.com/Sereal/Sereal/commit/35372ae01d in the # Sereal.git repository for test Sereal data. -0 string \=srl Sereal data +0 string \=srl Sereal data !:mime application/sereal ->4 byte&0x0F x (version %d, ->4 byte&0xF0 0x00 uncompressed) ->4 byte&0xF0 0x10 compressed with non-incremental Snappy) ->4 byte&0xF0 0x20 compressed with incremental Snappy) ->4 byte&0xF0 >0x20 unknown subformat, flag: %d>>4) +>4 byte&0x0F x (version %d, +>4 byte&0xF0 0x00 uncompressed) +>4 byte&0xF0 0x10 compressed with non-incremental Snappy) +>4 byte&0xF0 0x20 compressed with incremental Snappy) +>4 byte&0xF0 >0x20 unknown subformat, flag: %d>>4) Index: releng/10.1/contrib/file/magic/Magdir/ssh =================================================================== --- releng/10.1/contrib/file/magic/Magdir/ssh (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/ssh (revision 284193) @@ -1,11 +1,13 @@ # Type: OpenSSH key files # From: Nicolas Collignon 0 string SSH\ PRIVATE\ KEY OpenSSH RSA1 private key, >28 string >\0 version %s +0 string -----BEGIN\ OPENSSH\ PRIVATE\ KEY----- OpenSSH private key 0 string ssh-dss\ OpenSSH DSA public key 0 string ssh-rsa\ OpenSSH RSA public key 0 string ecdsa-sha2-nistp256 OpenSSH ECDSA public key 0 string ecdsa-sha2-nistp384 OpenSSH ECDSA public key 0 string ecdsa-sha2-nistp521 OpenSSH ECDSA public key +0 string ssh-ed25519 OpenSSH ED25519 public key Index: releng/10.1/contrib/file/magic/Magdir/vms =================================================================== --- releng/10.1/contrib/file/magic/Magdir/vms (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/vms (revision 284193) @@ -1,30 +1,30 @@ #------------------------------------------------------------------------------ -# $File: vms,v 1.7 2013/03/09 22:36:00 christos Exp $ +# $File: vms,v 1.9 2014/08/17 13:47:59 christos Exp $ # vms: file(1) magic for VMS executables (experimental) # # VMS .exe formats, both VAX and AXP (Greg Roelofs, newt@uchicago.edu) # GRR 950122: I'm just guessing on these, based on inspection of the headers # of three executables each for Alpha and VAX architectures. The VAX files # all had headers similar to this: # # 00000 b0 00 30 00 44 00 60 00 00 00 00 00 30 32 30 35 ..0.D.`.....0205 # 00010 01 01 00 00 ff ff ff ff ff ff ff ff 00 00 00 00 ................ # 0 string \xb0\0\x30\0 VMS VAX executable >44032 string PK\003\004 \b, Info-ZIP SFX archive v5.12 w/decryption # # The AXP files all looked like this, except that the byte at offset 0x22 # was 06 in some of them and 07 in others: # # 00000 03 00 00 00 00 00 00 00 ec 02 00 00 10 01 00 00 ................ # 00010 68 00 00 00 98 00 00 00 b8 00 00 00 00 00 00 00 h............... # 00020 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ # 00030 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 ................ # 00040 00 00 00 00 ff ff ff ff ff ff ff ff 02 00 00 00 ................ # # GRR this test is still too general as it catches example adressen.dbt 0 belong 0x03000000 >8 ubelong 0xec020000 VMS Alpha executable >>75264 string PK\003\004 \b, Info-ZIP SFX archive v5.12 w/decryption Index: releng/10.1/contrib/file/magic/Magdir/vorbis =================================================================== --- releng/10.1/contrib/file/magic/Magdir/vorbis (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/vorbis (revision 284193) @@ -1,118 +1,124 @@ #------------------------------------------------------------------------------ -# $File: vorbis,v 1.18 2014/04/30 21:41:02 christos Exp $ +# $File: vorbis,v 1.20 2014/09/23 16:35:08 christos Exp $ # vorbis: file(1) magic for Ogg/Vorbis files # # From Felix von Leitner # Extended by Beni Cherniavsky # Further extended by Greg Wooledge # # Most (everything but the number of channels and bitrate) is commented # out with `##' as it's not interesting to the average user. The most # probable things advanced users would want to uncomment are probably # the number of comments and the encoder version. # # FIXME: The first match has been made a search, so that it can skip # over prepended ID3 tags. This will work for MIME type detection, but # won't work for detecting other properties of the file (they all need # to be made relative to the search). In any case, if the file has ID3 # tags, the ID3 information will be printed, not the Ogg information, # so until that's fixed, this doesn't matter. # FIXME[2]: Disable the above for now, since search assumes text mode. # # --- Ogg Framing --- #0 search/1000 OggS Ogg data 0 string OggS Ogg data -!:mime application/ogg >4 byte !0 UNKNOWN REVISION %u ##>4 byte 0 revision 0 >4 byte 0 ##>>14 lelong x (Serial %lX) # non-Vorbis content: FLAC (Free Lossless Audio Codec, http://flac.sourceforge.net) >>28 string \x7fFLAC \b, FLAC audio # non-Vorbis content: Theora +!:mime audio/ogg >>28 string \x80theora \b, Theora video +!:mime video/ogg # non-Vorbis content: Kate ->>28 string \x80kate\0\0\0\0 \b, Kate +>>28 string \x80kate\0\0\0\0 \b, Kate (Karaoke and Text) +!:mime application/ogg >>>37 ubyte x v%u >>>38 ubyte x \b.%u, >>>40 byte 0 utf8 encoding, >>>40 byte !0 unknown character encoding, >>>60 string >\0 language %s, >>>60 string \0 no language set, >>>76 string >\0 category %s >>>76 string \0 no category set # non-Vorbis content: Skeleton >>28 string fishead\0 \b, Skeleton +!:mime video/ogg >>>36 short x v%u >>>40 short x \b.%u # non-Vorbis content: Speex >>28 string Speex\ \ \ \b, Speex audio +!:mime audio/ogg # non-Vorbis content: OGM >>28 string \x01video\0\0\0 \b, OGM video +!:mime video/ogg >>>37 string/c div3 (DivX 3) >>>37 string/c divx (DivX 4) >>>37 string/c dx50 (DivX 5) >>>37 string/c xvid (XviD) # --- First vorbis packet - general header --- >>28 string \x01vorbis \b, Vorbis audio, +!:mime audio/ogg >>>35 lelong !0 UNKNOWN VERSION %u, ##>>>35 lelong 0 version 0, >>>35 lelong 0 >>>>39 ubyte 1 mono, >>>>39 ubyte 2 stereo, >>>>39 ubyte >2 %u channels, >>>>40 lelong x %u Hz # Minimal, nominal and maximal bitrates specified when encoding >>>>48 string <\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff \b, # The above tests if at least one of these is specified: >>>>>52 lelong !-1 # Vorbis RC2 has a bug which puts -1000 in the min/max bitrate fields # instead of -1. # Vorbis 1.0 uses 0 instead of -1. >>>>>>52 lelong !0 >>>>>>>52 lelong !-1000 >>>>>>>>52 lelong x <%u >>>>>48 lelong !-1 >>>>>>48 lelong x ~%u >>>>>44 lelong !-1 >>>>>>44 lelong !-1000 >>>>>>>44 lelong !0 >>>>>>>>44 lelong x >%u >>>>>48 string <\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff bps # -- Second vorbis header packet - the comments # A kludge to read the vendor string. It's a counted string, not a # zero-terminated one, so file(1) can't read it in a generic way. # libVorbis is the only one existing currently, so I detect specifically # it. The interesting value is the cvs date (8 digits decimal). # Post-RC1 Ogg files have the second header packet (and thus the version) # in a different place, so we must use an indirect offset. >>>(84.b+85) string \x03vorbis >>>>(84.b+96) string/c Xiphophorus\ libVorbis\ I \b, created by: Xiphophorus libVorbis I >>>>>(84.b+120) string >00000000 # Map to beta version numbers: >>>>>>(84.b+120) string <20000508 (>>>>>(84.b+120) string 20000508 (1.0 beta 1 or beta 2) >>>>>>(84.b+120) string >20000508 >>>>>>>(84.b+120) string <20001031 (beta2-3) >>>>>>(84.b+120) string 20001031 (1.0 beta 3) >>>>>>(84.b+120) string >20001031 >>>>>>>(84.b+120) string <20010225 (beta3-4) >>>>>>(84.b+120) string 20010225 (1.0 beta 4) >>>>>>(84.b+120) string >20010225 >>>>>>>(84.b+120) string <20010615 (beta4-RC1) >>>>>>(84.b+120) string 20010615 (1.0 RC1) >>>>>>(84.b+120) string 20010813 (1.0 RC2) >>>>>>(84.b+120) string 20010816 (RC2 - Garf tuned v1) >>>>>>(84.b+120) string 20011014 (RC2 - Garf tuned v2) >>>>>>(84.b+120) string 20011217 (1.0 RC3) >>>>>>(84.b+120) string 20011231 (1.0 RC3) # Some pre-1.0 CVS snapshots still had "Xiphphorus"... >>>>>>(84.b+120) string >20011231 (pre-1.0 CVS) # For the 1.0 release, Xiphophorus is replaced by Xiph.Org >>>>(84.b+96) string/c Xiph.Org\ libVorbis\ I \b, created by: Xiph.Org libVorbis I >>>>>(84.b+117) string >00000000 >>>>>>(84.b+117) string <20020717 (pre-1.0 CVS) >>>>>>(84.b+117) string 20020717 (1.0) >>>>>>(84.b+117) string 20030909 (1.0.1) >>>>>>(84.b+117) string 20040629 (1.1.0 RC1) Index: releng/10.1/contrib/file/magic/Magdir/windows =================================================================== --- releng/10.1/contrib/file/magic/Magdir/windows (revision 284192) +++ releng/10.1/contrib/file/magic/Magdir/windows (revision 284193) @@ -1,336 +1,337 @@ #------------------------------------------------------------------------------ -# $File: windows,v 1.8 2014/04/30 21:41:02 christos Exp $ +# $File: windows,v 1.10 2014/09/24 19:52:46 christos Exp $ # windows: file(1) magic for Microsoft Windows # # This file is mainly reserved for files where programs # using them are run almost always on MS Windows 3.x or # above, or files only used exclusively in Windows OS, # where there is no better category to allocate for. # For example, even though WinZIP almost run on Windows # only, it is better to treat them as "archive" instead. # For format usable in DOS, such as generic executable # format, please specify under "msdos" file. # # Summary: Outlook Express DBX file # Extension: .dbx # Created by: Christophe Monniez 0 string \xCF\xAD\x12\xFE MS Outlook Express DBX file >4 byte =0xC5 \b, message database >4 byte =0xC6 \b, folder database >4 byte =0xC7 \b, account information >4 byte =0x30 \b, offline database # Summary: Windows crash dump # Extension: .dmp # Created by: Andreas Schuster (http://computer.forensikblog.de/) # Reference (1): http://computer.forensikblog.de/en/2008/02/64bit_magic.html # Modified by (1): Abel Cheung (Avoid match with first 4 bytes only) 0 string PAGE >4 string DUMP MS Windows 32bit crash dump >>0x05c byte 0 \b, no PAE >>0x05c byte 1 \b, PAE >>0xf88 lelong 1 \b, full dump >>0xf88 lelong 2 \b, kernel dump >>0xf88 lelong 3 \b, small dump >>0x068 lelong x \b, %d pages >4 string DU64 MS Windows 64bit crash dump >>0xf98 lelong 1 \b, full dump >>0xf98 lelong 2 \b, kernel dump >>0xf98 lelong 3 \b, small dump >>0x090 lequad x \b, %lld pages # Summary: Vista Event Log # Extension: .evtx # Created by: Andreas Schuster (http://computer.forensikblog.de/) # Reference (1): http://computer.forensikblog.de/en/2007/05/some_magic.html 0 string ElfFile\0 MS Windows Vista Event Log >0x2a leshort x \b, %d chunks >>0x10 lelong x \b (no. %d in use) >0x18 lelong >1 \b, next record no. %d >0x18 lelong =1 \b, empty >0x78 lelong &1 \b, DIRTY >0x78 lelong &2 \b, FULL # Summary: Windows 3.1 group files # Extension: .grp # Created by: unknown 0 string \120\115\103\103 MS Windows 3.1 group files # Summary: Old format help files # Extension: .hlp # Created by: Dirk Jagdmann 0 lelong 0x00035f3f MS Windows 3.x help file # Summary: Hyper terminal # Extension: .ht # Created by: unknown 0 string HyperTerminal\ >15 string 1.0\ --\ HyperTerminal\ data\ file MS Windows HyperTerminal profile # http://ithreats.files.wordpress.com/2009/05/\ # lnk_the_windows_shortcut_file_format.pdf # Summary: Windows shortcut # Extension: .lnk # Created by: unknown # 'L' + GUUID 0 string \114\0\0\0\001\024\002\0\0\0\0\0\300\0\0\0\0\0\0\106 MS Windows shortcut >20 lelong&1 1 \b, Item id list present >20 lelong&2 2 \b, Points to a file or directory >20 lelong&4 4 \b, Has Description string >20 lelong&8 8 \b, Has Relative path >20 lelong&16 16 \b, Has Working directory >20 lelong&32 32 \b, Has command line arguments >20 lelong&64 64 \b, Icon >>56 lelong \b number=%d >24 lelong&1 1 \b, Read-Only >24 lelong&2 2 \b, Hidden >24 lelong&4 4 \b, System >24 lelong&8 8 \b, Volume Label >24 lelong&16 16 \b, Directory >24 lelong&32 32 \b, Archive >24 lelong&64 64 \b, Encrypted >24 lelong&128 128 \b, Normal >24 lelong&256 256 \b, Temporary >24 lelong&512 512 \b, Sparse >24 lelong&1024 1024 \b, Reparse point >24 lelong&2048 2048 \b, Compressed >24 lelong&4096 4096 \b, Offline >28 leqwdate x \b, ctime=%s >36 leqwdate x \b, mtime=%s >44 leqwdate x \b, atime=%s >52 lelong x \b, length=%u, window= >60 lelong&1 1 \bhide >60 lelong&2 2 \bnormal >60 lelong&4 4 \bshowminimized >60 lelong&8 8 \bshowmaximized >60 lelong&16 16 \bshownoactivate >60 lelong&32 32 \bminimize >60 lelong&64 64 \bshowminnoactive >60 lelong&128 128 \bshowna >60 lelong&256 256 \brestore >60 lelong&512 512 \bshowdefault #>20 lelong&1 0 #>>20 lelong&2 2 #>>>(72.l-64) pstring/h x \b [%s] #>20 lelong&1 1 #>>20 lelong&2 2 #>>>(72.s) leshort x #>>>&75 pstring/h x \b [%s] # Summary: Outlook Personal Folders # Created by: unknown 0 lelong 0x4E444221 Microsoft Outlook email folder >10 leshort 0x0e (<=2002) >10 leshort 0x17 (>=2003) # Summary: Windows help cache # Created by: unknown 0 string \164\146\115\122\012\000\000\000\001\000\000\000 MS Windows help cache # Summary: IE cache file # Created by: Christophe Monniez 0 string Client\ UrlCache\ MMF Internet Explorer cache file >20 string >\0 version %s # Summary: Registry files # Created by: unknown # Modified by (1): Joerg Jenderek 0 string regf MS Windows registry file, NT/2000 or above 0 string CREG MS Windows 95/98/ME registry file 0 string SHCC3 MS Windows 3.1 registry file # Summary: Windows Registry text # Extension: .reg # Submitted by: Abel Cheung 0 string REGEDIT4\r\n\r\n Windows Registry text (Win95 or above) 0 string Windows\ Registry\ Editor\ >&0 string Version\ 5.00\r\n\r\n Windows Registry text (Win2K or above) # Windows *.INF *.INI files updated by Joerg Jenderek at Apr 2013 -# emtpy ,comment , section , unicode line -0 regex/s \\`(\r\n|;|[[]|\xFF\xFE) +# empty ,comment , section +# PR/383: remove unicode BOM because it is not portable across regex impls +0 regex/s \\`(\\r\\n|;|[[]) # left bracket in section line >&0 search/8192 [ # http://en.wikipedia.org/wiki/Autorun.inf # http://msdn.microsoft.com/en-us/library/windows/desktop/cc144200.aspx >>&0 regex/c \^(autorun)]\r\n >>>&0 ubyte =0x5b INItialization configuration !:mime application/x-wine-extension-ini # From: Pal Tamas # Autorun File >>>&0 ubyte !0x5b Microsoft Windows Autorun file !:mime application/x-setupscript # http://msdn.microsoft.com/en-us/library/windows/hardware/ff549520(v=vs.85).aspx # version strings ASCII coded case-independent for Windows setup information script file >>&0 regex/c \^(version|strings)] Windows setup INFormation !:mime application/x-setupscript #!:mime application/inf #!:mime application/x-wine-extension-inf >>&0 regex/c \^(WinsockCRCList|OEMCPL)] Windows setup INFormation !:mime text/inf # http://www.winfaq.de/faq_html/Content/tip2500/onlinefaq.php?h=tip2653.htm # http://msdn.microsoft.com/en-us/library/windows/desktop/cc144102.aspx # .ShellClassInfo DeleteOnCopy LocalizedFileNames ASCII coded case-independent >>&0 regex/c \^(\.ShellClassInfo|DeleteOnCopy|LocalizedFileNames)] Windows desktop.ini !:mime application/x-wine-extension-ini #!:mime text/plain # http://support.microsoft.com/kb/84709/ >>&0 regex/c \^(don't\ load)] Windows CONTROL.INI !:mime application/x-wine-extension-ini >>&0 regex/c \^(ndishlp\\$|protman\\$|NETBEUI\\$)] Windows PROTOCOL.INI !:mime application/x-wine-extension-ini # http://technet.microsoft.com/en-us/library/cc722567.aspx # http://www.winfaq.de/faq_html/Content/tip0000/onlinefaq.php?h=tip0137.htm >>&0 regex/c \^(windows|Compatibility|embedding)] Windows WIN.INI !:mime application/x-wine-extension-ini # http://en.wikipedia.org/wiki/SYSTEM.INI >>&0 regex/c \^(boot|386enh|drivers)] Windows SYSTEM.INI !:mime application/x-wine-extension-ini # http://www.mdgx.com/newtip6.htm >>&0 regex/c \^(SafeList)] Windows IOS.INI !:mime application/x-wine-extension-ini # http://en.wikipedia.org/wiki/NTLDR Windows Boot Loader information >>&0 regex/c \^(boot\x20loader)] Windows boot.ini !:mime application/x-wine-extension-ini >>>&0 ubyte x # http://en.wikipedia.org/wiki/CONFIG.SYS >>&0 regex/c \^(menu)]\r\n MS-DOS CONFIG.SYS # http://support.microsoft.com/kb/118579/ >>&0 regex/c \^(Paths)]\r\n MS-DOS MSDOS.SYS # VERS string unicoded case-independent >>&0 ubequad&0xFFdfFFdfFFdfFFdf 0x0056004500520053 # ION] string unicoded case-independent >>>&0 ubequad&0xFFdfFFdfFFdfFFff 0x0049004f004e005d Windows setup INFormation !:mime application/x-setupscript # STRI string unicoded case-independent >>&0 ubequad&0xFFdfFFdfFFdfFFdf 0x0053005400520049 # NGS] string unicoded case-independent >>>&0 ubequad&0xFFdfFFdfFFdfFFff 0x004e00470053005D Windows setup INFormation !:mime application/x-setupscript # unknown keyword after opening bracket >>&0 default x >>>&0 search/8192 [ # version Strings FileIdentification >>>>&0 string/c version Windows setup INFormation !:mime application/x-setupscript # VERS string unicoded case-independent >>>>&0 ubequad&0xFFdfFFdfFFdfFFdf 0x0056004500520053 # ION] string unicoded case-independent >>>>>&0 ubequad&0xFFdfFFdfFFdfFFff 0x0049004f004e005d Windows setup INFormation !:mime application/x-setupscript # http://en.wikipedia.org/wiki/Initialization_file Windows Initialization File or other #>>>>&0 default x Generic INItialization configuration #!:mime application/x-wine-extension-ini # Windows Precompiled INF files *.PNF added by Joerg Jenderek at Mar 2013 of _PNF_HEADER inf.h # http://read.pudn.com/downloads3/sourcecode/windows/248345/win2k/private/windows/setup/setupapi/inf.h__.htm # GRR: line below too general as it catches also PDP-11 UNIX/RT ldp 0 leshort&0xFeFe 0x0000 # test for unused null bits in PNF_FLAGs >4 ulelong&0xFCffFe00 0x00000000 # only found 58h for Offset of WinDirPath immediately after _PNF_HEADER structure >>68 ulelong >0x57 # test for zero high byte of InfValueBlockSize, followed by WinDirPath like # C:\WINDOWS (ASCII 0x433a5c.. , unicode 0x43003a005c..) or X:\MININT >>>(68.l-1) ubelong&0xffE0C519 =0x00400018 Windows Precompiled iNF !:mime application/x-pnf # currently only found Major Version=1 and Minor Version=1 #>>>>0 uleshort =0x0101 #>>>>>1 ubyte x \b, version %u #>>>>>0 ubyte x \b.%u >>>>0 uleshort !0x0101 >>>>>1 ubyte x \b, version %u >>>>>0 ubyte x \b.%u # 1 ,2 (windows 98 SE) #>>>>2 uleshort =2 \b, InfStyle %u >>>>2 uleshort !2 \b, InfStyle %u # PNF_FLAG_IS_UNICODE 0x00000001 # PNF_FLAG_HAS_STRINGS 0x00000002 # PNF_FLAG_SRCPATH_IS_URL 0x00000004 # PNF_FLAG_HAS_VOLATILE_DIRIDS 0x00000008 # PNF_FLAG_INF_VERIFIED 0x00000010 # PNF_FLAG_INF_DIGITALLY_SIGNED 0x00000020 # ?? 0x00000100 # ?? 0x01000000 # ?? 0x02000000 >>>>4 ulelong&0x00000001 0x00000001 \b, unicoded >>>>4 ulelong&0x00000020 0x00000020 \b, digitally signed #>>>>8 ulelong x \b, InfSubstValueListOffset 0x%x # many 0, 1 lmouusb.PNF, 2 linkfx10.PNF , f webfdr16.PNF #>>>>12 uleshort x \b, InfSubstValueCount 0x%x # only < 9 found #>>>>14 uleshort x \b, InfVersionDatumCount 0x%x # only found values lower 0x0000ffff #>>>>16 ulelong x \b, InfVersionDataSize 0x%x # only found positive values lower 0x00ffFFff for InfVersionDataOffset >>>>20 ulelong x \b, at 0x%x >>>>4 ulelong&0x00000001 =0x00000001 # case independent: CatalogFile Class DriverVer layoutfile LayoutFile SetupClass signature Signature >>>>>(20.l) lestring16 x "%s" >>>>4 ulelong&0x00000001 !0x00000001 >>>>>(20.l) string x "%s" # FILETIME is number of 100-nanosecond intervals since 1 January 1601 #>>>>24 ulequad x \b, InfVersionLastWriteTime %16.16llx # only found values lower 0x00ffFFff #>>>>32 ulelong x \b, StringTableBlockOffset 0x%x #>>>>36 ulelong x \b, StringTableBlockSize 0x%x #>>>>40 ulelong x \b, InfSectionCount 0x%x #>>>>44 ulelong x \b, InfSectionBlockOffset 0x%x #>>>>48 ulelong x \b, InfSectionBlockSize 0x%x #>>>>52 ulelong x \b, InfLineBlockOffset 0x%x #>>>>56 ulelong x \b, InfLineBlockSize 0x%x #>>>>60 ulelong x \b, InfValueBlockOffset 0x%x #>>>>64 ulelong x \b, InfValueBlockSize 0x%x # WinDirPathOffset #>>>>68 ulelong x \b, at 0x%x >>>>68 ulelong >0x57 >>>>>4 ulelong&0x00000001 =0x00000001 >>>>>>(68.l) ubequad =0x43003a005c005700 # normally unicoded C:\Windows #>>>>>>>(68.l) lestring16 x \b, WinDirPath "%s" >>>>>>(68.l) ubequad !0x43003a005c005700 >>>>>>>(68.l) lestring16 x \b, WinDirPath "%s" >>>>>4 ulelong&0x00000001 !0x00000001 # normally ASCII C:\WINDOWS #>>>>>>(68.l) string =C:\\WINDOWS \b, WinDirPath "%s" >>>>>>(68.l) string !C:\\WINDOWS \b, WinDirPath "%s" # found OsLoaderPathOffset values often 0 , once 70h corelist.PNF, once 68h ASCII machine.PNF #>>>>72 ulelong >0 \b, at 0x%x >>>>72 ulelong >0 \b, >>>>>4 ulelong&0x00000001 =0x00000001 >>>>>>(72.l) lestring16 x OsLoaderPath "%s" >>>>>4 ulelong&0x00000001 !0x00000001 # seldom C:\ instead empty >>>>>>(72.l) string x OsLoaderPath "%s" # 1fdh #>>>>76 uleshort x \b, StringTableHashBucketCount 0x%x >>>>78 uleshort !0x407 \b, LanguageId %x # only 407h found #>>>>78 uleshort =0x407 \b, LanguageId %x # InfSourcePathOffset often 0 #>>>>80 ulelong >0 \b, at 0x%x >>>>80 ulelong >0 \b, >>>>>4 ulelong&0x00000001 =0x00000001 >>>>>>(80.l) lestring16 x SourcePath "%s" >>>>>4 ulelong&0x00000001 !0x00000001 >>>>>>(80.l) string >\0 SourcePath "%s" # OriginalInfNameOffset often 0 #>>>>84 ulelong >0 \b, at 0x%x >>>>84 ulelong >0 \b, >>>>>4 ulelong&0x00000001 =0x00000001 >>>>>>(84.l) lestring16 x InfName "%s" >>>>>4 ulelong&0x00000001 !0x00000001 >>>>>>(84.l) string >\0 InfName "%s" Index: releng/10.1/contrib/file/magic/Makefile.am =================================================================== --- releng/10.1/contrib/file/magic/Makefile.am (revision 284192) +++ releng/10.1/contrib/file/magic/Makefile.am (revision 284193) @@ -1,299 +1,301 @@ # -# $File: Makefile.am,v 1.98 2014/06/03 18:22:34 christos Exp $ +# $File: Makefile.am,v 1.101 2014/12/12 16:48:39 christos Exp $ # MAGIC_FRAGMENT_BASE = Magdir MAGIC_DIR = $(top_srcdir)/magic MAGIC_FRAGMENT_DIR = $(MAGIC_DIR)/$(MAGIC_FRAGMENT_BASE) pkgdata_DATA = magic.mgc EXTRA_DIST = \ $(MAGIC_DIR)/Header \ $(MAGIC_DIR)/Localstuff \ $(MAGIC_FRAGMENT_DIR)/acorn \ $(MAGIC_FRAGMENT_DIR)/adi \ $(MAGIC_FRAGMENT_DIR)/adventure \ $(MAGIC_FRAGMENT_DIR)/allegro \ $(MAGIC_FRAGMENT_DIR)/alliant \ $(MAGIC_FRAGMENT_DIR)/amanda \ $(MAGIC_FRAGMENT_DIR)/amigaos \ $(MAGIC_FRAGMENT_DIR)/android \ $(MAGIC_FRAGMENT_DIR)/animation \ $(MAGIC_FRAGMENT_DIR)/aout \ $(MAGIC_FRAGMENT_DIR)/apl \ $(MAGIC_FRAGMENT_DIR)/apple \ $(MAGIC_FRAGMENT_DIR)/applix \ $(MAGIC_FRAGMENT_DIR)/archive \ $(MAGIC_FRAGMENT_DIR)/assembler \ $(MAGIC_FRAGMENT_DIR)/asterix \ $(MAGIC_FRAGMENT_DIR)/att3b \ $(MAGIC_FRAGMENT_DIR)/audio \ $(MAGIC_FRAGMENT_DIR)/basis \ $(MAGIC_FRAGMENT_DIR)/bflt \ $(MAGIC_FRAGMENT_DIR)/blackberry \ $(MAGIC_FRAGMENT_DIR)/blcr \ $(MAGIC_FRAGMENT_DIR)/blender \ $(MAGIC_FRAGMENT_DIR)/blit \ $(MAGIC_FRAGMENT_DIR)/bout \ $(MAGIC_FRAGMENT_DIR)/bsdi \ $(MAGIC_FRAGMENT_DIR)/bsi \ $(MAGIC_FRAGMENT_DIR)/btsnoop \ $(MAGIC_FRAGMENT_DIR)/c-lang \ $(MAGIC_FRAGMENT_DIR)/c64 \ $(MAGIC_FRAGMENT_DIR)/cad \ $(MAGIC_FRAGMENT_DIR)/cafebabe \ $(MAGIC_FRAGMENT_DIR)/cddb \ $(MAGIC_FRAGMENT_DIR)/chord \ $(MAGIC_FRAGMENT_DIR)/cisco \ $(MAGIC_FRAGMENT_DIR)/citrus \ $(MAGIC_FRAGMENT_DIR)/clarion \ $(MAGIC_FRAGMENT_DIR)/claris \ $(MAGIC_FRAGMENT_DIR)/clipper \ $(MAGIC_FRAGMENT_DIR)/commands \ $(MAGIC_FRAGMENT_DIR)/communications \ $(MAGIC_FRAGMENT_DIR)/compress \ $(MAGIC_FRAGMENT_DIR)/console \ $(MAGIC_FRAGMENT_DIR)/convex \ $(MAGIC_FRAGMENT_DIR)/cracklib \ $(MAGIC_FRAGMENT_DIR)/ctags \ $(MAGIC_FRAGMENT_DIR)/ctf \ $(MAGIC_FRAGMENT_DIR)/cubemap \ $(MAGIC_FRAGMENT_DIR)/cups \ $(MAGIC_FRAGMENT_DIR)/dact \ $(MAGIC_FRAGMENT_DIR)/database \ $(MAGIC_FRAGMENT_DIR)/diamond \ $(MAGIC_FRAGMENT_DIR)/diff \ $(MAGIC_FRAGMENT_DIR)/digital \ $(MAGIC_FRAGMENT_DIR)/dolby \ $(MAGIC_FRAGMENT_DIR)/dump \ $(MAGIC_FRAGMENT_DIR)/dyadic \ $(MAGIC_FRAGMENT_DIR)/ebml \ $(MAGIC_FRAGMENT_DIR)/editors \ $(MAGIC_FRAGMENT_DIR)/efi \ $(MAGIC_FRAGMENT_DIR)/elf \ $(MAGIC_FRAGMENT_DIR)/encore \ $(MAGIC_FRAGMENT_DIR)/epoc \ $(MAGIC_FRAGMENT_DIR)/erlang \ $(MAGIC_FRAGMENT_DIR)/esri \ $(MAGIC_FRAGMENT_DIR)/fcs \ $(MAGIC_FRAGMENT_DIR)/filesystems \ $(MAGIC_FRAGMENT_DIR)/flash \ $(MAGIC_FRAGMENT_DIR)/fonts \ $(MAGIC_FRAGMENT_DIR)/fortran \ $(MAGIC_FRAGMENT_DIR)/frame \ $(MAGIC_FRAGMENT_DIR)/freebsd \ $(MAGIC_FRAGMENT_DIR)/fsav \ $(MAGIC_FRAGMENT_DIR)/fusecompress \ $(MAGIC_FRAGMENT_DIR)/games \ $(MAGIC_FRAGMENT_DIR)/gcc \ $(MAGIC_FRAGMENT_DIR)/geo \ $(MAGIC_FRAGMENT_DIR)/geos \ $(MAGIC_FRAGMENT_DIR)/gimp \ $(MAGIC_FRAGMENT_DIR)/gnome \ $(MAGIC_FRAGMENT_DIR)/gnu \ $(MAGIC_FRAGMENT_DIR)/gnumeric \ $(MAGIC_FRAGMENT_DIR)/gpt \ $(MAGIC_FRAGMENT_DIR)/grace \ $(MAGIC_FRAGMENT_DIR)/graphviz \ $(MAGIC_FRAGMENT_DIR)/gringotts \ $(MAGIC_FRAGMENT_DIR)/guile \ $(MAGIC_FRAGMENT_DIR)/hitachi-sh \ $(MAGIC_FRAGMENT_DIR)/hp \ $(MAGIC_FRAGMENT_DIR)/human68k \ $(MAGIC_FRAGMENT_DIR)/ibm370 \ $(MAGIC_FRAGMENT_DIR)/ibm6000 \ $(MAGIC_FRAGMENT_DIR)/icc \ $(MAGIC_FRAGMENT_DIR)/iff \ $(MAGIC_FRAGMENT_DIR)/images \ $(MAGIC_FRAGMENT_DIR)/inform \ $(MAGIC_FRAGMENT_DIR)/intel \ $(MAGIC_FRAGMENT_DIR)/interleaf \ $(MAGIC_FRAGMENT_DIR)/island \ $(MAGIC_FRAGMENT_DIR)/ispell \ $(MAGIC_FRAGMENT_DIR)/isz \ $(MAGIC_FRAGMENT_DIR)/java \ $(MAGIC_FRAGMENT_DIR)/javascript \ $(MAGIC_FRAGMENT_DIR)/jpeg \ $(MAGIC_FRAGMENT_DIR)/karma \ $(MAGIC_FRAGMENT_DIR)/kde \ $(MAGIC_FRAGMENT_DIR)/keepass \ +$(MAGIC_FRAGMENT_DIR)/kerberos \ $(MAGIC_FRAGMENT_DIR)/kml \ $(MAGIC_FRAGMENT_DIR)/lecter \ $(MAGIC_FRAGMENT_DIR)/lex \ $(MAGIC_FRAGMENT_DIR)/lif \ $(MAGIC_FRAGMENT_DIR)/linux \ $(MAGIC_FRAGMENT_DIR)/lisp \ $(MAGIC_FRAGMENT_DIR)/llvm \ $(MAGIC_FRAGMENT_DIR)/lua \ $(MAGIC_FRAGMENT_DIR)/luks \ $(MAGIC_FRAGMENT_DIR)/m4 \ $(MAGIC_FRAGMENT_DIR)/mach \ $(MAGIC_FRAGMENT_DIR)/macos \ $(MAGIC_FRAGMENT_DIR)/macintosh \ $(MAGIC_FRAGMENT_DIR)/magic \ $(MAGIC_FRAGMENT_DIR)/mail.news \ $(MAGIC_FRAGMENT_DIR)/make \ $(MAGIC_FRAGMENT_DIR)/map \ $(MAGIC_FRAGMENT_DIR)/maple \ $(MAGIC_FRAGMENT_DIR)/marc21 \ $(MAGIC_FRAGMENT_DIR)/mathcad \ $(MAGIC_FRAGMENT_DIR)/mathematica \ $(MAGIC_FRAGMENT_DIR)/matroska \ $(MAGIC_FRAGMENT_DIR)/mcrypt \ $(MAGIC_FRAGMENT_DIR)/mercurial \ $(MAGIC_FRAGMENT_DIR)/metastore \ +$(MAGIC_FRAGMENT_DIR)/meteorological \ $(MAGIC_FRAGMENT_DIR)/mime \ $(MAGIC_FRAGMENT_DIR)/mips \ $(MAGIC_FRAGMENT_DIR)/mirage \ $(MAGIC_FRAGMENT_DIR)/misctools \ $(MAGIC_FRAGMENT_DIR)/mkid \ $(MAGIC_FRAGMENT_DIR)/mlssa \ $(MAGIC_FRAGMENT_DIR)/mmdf \ $(MAGIC_FRAGMENT_DIR)/modem \ $(MAGIC_FRAGMENT_DIR)/motorola \ $(MAGIC_FRAGMENT_DIR)/mozilla \ $(MAGIC_FRAGMENT_DIR)/msdos \ $(MAGIC_FRAGMENT_DIR)/msooxml \ $(MAGIC_FRAGMENT_DIR)/msx \ $(MAGIC_FRAGMENT_DIR)/msvc \ $(MAGIC_FRAGMENT_DIR)/mup \ $(MAGIC_FRAGMENT_DIR)/music \ $(MAGIC_FRAGMENT_DIR)/natinst \ $(MAGIC_FRAGMENT_DIR)/ncr \ $(MAGIC_FRAGMENT_DIR)/neko \ $(MAGIC_FRAGMENT_DIR)/netbsd \ $(MAGIC_FRAGMENT_DIR)/netscape \ $(MAGIC_FRAGMENT_DIR)/netware \ $(MAGIC_FRAGMENT_DIR)/news \ $(MAGIC_FRAGMENT_DIR)/nitpicker \ $(MAGIC_FRAGMENT_DIR)/oasis \ $(MAGIC_FRAGMENT_DIR)/ocaml \ $(MAGIC_FRAGMENT_DIR)/octave \ $(MAGIC_FRAGMENT_DIR)/ole2compounddocs \ $(MAGIC_FRAGMENT_DIR)/olf \ $(MAGIC_FRAGMENT_DIR)/os2 \ $(MAGIC_FRAGMENT_DIR)/os400 \ $(MAGIC_FRAGMENT_DIR)/os9 \ $(MAGIC_FRAGMENT_DIR)/osf1 \ $(MAGIC_FRAGMENT_DIR)/palm \ $(MAGIC_FRAGMENT_DIR)/parix \ $(MAGIC_FRAGMENT_DIR)/parrot \ $(MAGIC_FRAGMENT_DIR)/pascal \ $(MAGIC_FRAGMENT_DIR)/pbf \ $(MAGIC_FRAGMENT_DIR)/pbm \ $(MAGIC_FRAGMENT_DIR)/pdf \ $(MAGIC_FRAGMENT_DIR)/pdp \ $(MAGIC_FRAGMENT_DIR)/perl \ $(MAGIC_FRAGMENT_DIR)/pgf \ $(MAGIC_FRAGMENT_DIR)/pgp \ $(MAGIC_FRAGMENT_DIR)/pkgadd \ $(MAGIC_FRAGMENT_DIR)/plan9 \ $(MAGIC_FRAGMENT_DIR)/plus5 \ $(MAGIC_FRAGMENT_DIR)/printer \ $(MAGIC_FRAGMENT_DIR)/project \ $(MAGIC_FRAGMENT_DIR)/psdbms \ $(MAGIC_FRAGMENT_DIR)/pulsar \ $(MAGIC_FRAGMENT_DIR)/pwsafe \ $(MAGIC_FRAGMENT_DIR)/pyramid \ $(MAGIC_FRAGMENT_DIR)/python \ +$(MAGIC_FRAGMENT_DIR)/qt \ $(MAGIC_FRAGMENT_DIR)/revision \ $(MAGIC_FRAGMENT_DIR)/riff \ -$(MAGIC_FRAGMENT_DIR)/rinex \ $(MAGIC_FRAGMENT_DIR)/rpm \ $(MAGIC_FRAGMENT_DIR)/rtf \ $(MAGIC_FRAGMENT_DIR)/ruby \ $(MAGIC_FRAGMENT_DIR)/sc \ $(MAGIC_FRAGMENT_DIR)/sccs \ $(MAGIC_FRAGMENT_DIR)/scientific \ $(MAGIC_FRAGMENT_DIR)/securitycerts \ $(MAGIC_FRAGMENT_DIR)/selinux \ $(MAGIC_FRAGMENT_DIR)/sendmail \ $(MAGIC_FRAGMENT_DIR)/sequent \ $(MAGIC_FRAGMENT_DIR)/sereal \ $(MAGIC_FRAGMENT_DIR)/sgi \ $(MAGIC_FRAGMENT_DIR)/sgml \ $(MAGIC_FRAGMENT_DIR)/sharc \ $(MAGIC_FRAGMENT_DIR)/sinclair \ $(MAGIC_FRAGMENT_DIR)/sisu \ $(MAGIC_FRAGMENT_DIR)/sketch \ $(MAGIC_FRAGMENT_DIR)/smalltalk \ $(MAGIC_FRAGMENT_DIR)/smile \ $(MAGIC_FRAGMENT_DIR)/sniffer \ $(MAGIC_FRAGMENT_DIR)/softquad \ $(MAGIC_FRAGMENT_DIR)/spec \ $(MAGIC_FRAGMENT_DIR)/spectrum \ $(MAGIC_FRAGMENT_DIR)/sql \ $(MAGIC_FRAGMENT_DIR)/ssh \ $(MAGIC_FRAGMENT_DIR)/ssl \ $(MAGIC_FRAGMENT_DIR)/sun \ $(MAGIC_FRAGMENT_DIR)/symbos \ $(MAGIC_FRAGMENT_DIR)/sysex \ $(MAGIC_FRAGMENT_DIR)/tcl \ $(MAGIC_FRAGMENT_DIR)/teapot \ $(MAGIC_FRAGMENT_DIR)/terminfo \ $(MAGIC_FRAGMENT_DIR)/tex \ $(MAGIC_FRAGMENT_DIR)/tgif \ $(MAGIC_FRAGMENT_DIR)/ti-8x \ $(MAGIC_FRAGMENT_DIR)/timezone \ $(MAGIC_FRAGMENT_DIR)/troff \ $(MAGIC_FRAGMENT_DIR)/tuxedo \ $(MAGIC_FRAGMENT_DIR)/typeset \ $(MAGIC_FRAGMENT_DIR)/unicode \ $(MAGIC_FRAGMENT_DIR)/unknown \ $(MAGIC_FRAGMENT_DIR)/uterus \ $(MAGIC_FRAGMENT_DIR)/uuencode \ $(MAGIC_FRAGMENT_DIR)/varied.out \ $(MAGIC_FRAGMENT_DIR)/varied.script \ $(MAGIC_FRAGMENT_DIR)/vax \ $(MAGIC_FRAGMENT_DIR)/vicar \ $(MAGIC_FRAGMENT_DIR)/virtual \ $(MAGIC_FRAGMENT_DIR)/virtutech \ $(MAGIC_FRAGMENT_DIR)/visx \ $(MAGIC_FRAGMENT_DIR)/vms \ $(MAGIC_FRAGMENT_DIR)/vmware \ $(MAGIC_FRAGMENT_DIR)/vorbis \ $(MAGIC_FRAGMENT_DIR)/vxl \ $(MAGIC_FRAGMENT_DIR)/warc \ $(MAGIC_FRAGMENT_DIR)/weak \ $(MAGIC_FRAGMENT_DIR)/windows \ $(MAGIC_FRAGMENT_DIR)/wireless \ $(MAGIC_FRAGMENT_DIR)/wordprocessors \ $(MAGIC_FRAGMENT_DIR)/wsdl \ $(MAGIC_FRAGMENT_DIR)/xdelta \ $(MAGIC_FRAGMENT_DIR)/xenix \ $(MAGIC_FRAGMENT_DIR)/xilinx \ $(MAGIC_FRAGMENT_DIR)/xo65 \ $(MAGIC_FRAGMENT_DIR)/xwindows \ $(MAGIC_FRAGMENT_DIR)/zfs \ $(MAGIC_FRAGMENT_DIR)/zilog \ $(MAGIC_FRAGMENT_DIR)/zyxel MAGIC = magic.mgc CLEANFILES = ${MAGIC} $(MAGIC_FRAGMENT_DIR)/Localstuff # FIXME: Build file natively as well so that it can be used to compile # the target's magic file; for now we bail if the local version does not match if IS_CROSS_COMPILE FILE_COMPILE = file${EXEEXT} FILE_COMPILE_DEP = else FILE_COMPILE = $(top_builddir)/src/file${EXEEXT} FILE_COMPILE_DEP = $(FILE_COMPILE) endif ${MAGIC}: $(EXTRA_DIST) $(FILE_COMPILE_DEP) @rm -fr magic @mkdir magic && cp -p $(EXTRA_DIST) magic @(if expr "${FILE_COMPILE}" : '.*/.*' > /dev/null; then \ echo "Using ${FILE_COMPILE} to generate ${MAGIC}" > /dev/null; \ else \ v=$$(file --version | sed -e s/file-// -e q); \ if [ "$$v" != "${PACKAGE_VERSION}" ]; then \ echo "Cannot use the installed version of file ($$v) to"; \ echo "cross-compile file ${PACKAGE_VERSION}"; \ echo "Please install file ${PACKAGE_VERSION} locally first"; \ exit 1; \ fi; \ fi) $(FILE_COMPILE) -C -m magic @rm -fr magic Index: releng/10.1/contrib/file/magic/Makefile.in =================================================================== --- releng/10.1/contrib/file/magic/Makefile.in (revision 284192) +++ releng/10.1/contrib/file/magic/Makefile.in (revision 284193) @@ -1,776 +1,778 @@ # Makefile.in generated by automake 1.14 from Makefile.am. # @configure_input@ # Copyright (C) 1994-2013 Free Software Foundation, Inc. # This Makefile.in is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY, to the extent permitted by law; without # even the implied warranty of MERCHANTABILITY or FITNESS FOR A # PARTICULAR PURPOSE. @SET_MAKE@ VPATH = @srcdir@ am__is_gnu_make = test -n '$(MAKEFILE_LIST)' && test -n '$(MAKELEVEL)' am__make_running_with_option = \ case $${target_option-} in \ ?) ;; \ *) echo "am__make_running_with_option: internal error: invalid" \ "target option '$${target_option-}' specified" >&2; \ exit 1;; \ esac; \ has_opt=no; \ sane_makeflags=$$MAKEFLAGS; \ if $(am__is_gnu_make); then \ sane_makeflags=$$MFLAGS; \ else \ case $$MAKEFLAGS in \ *\\[\ \ ]*) \ bs=\\; \ sane_makeflags=`printf '%s\n' "$$MAKEFLAGS" \ | sed "s/$$bs$$bs[$$bs $$bs ]*//g"`;; \ esac; \ fi; \ skip_next=no; \ strip_trailopt () \ { \ flg=`printf '%s\n' "$$flg" | sed "s/$$1.*$$//"`; \ }; \ for flg in $$sane_makeflags; do \ test $$skip_next = yes && { skip_next=no; continue; }; \ case $$flg in \ *=*|--*) continue;; \ -*I) strip_trailopt 'I'; skip_next=yes;; \ -*I?*) strip_trailopt 'I';; \ -*O) strip_trailopt 'O'; skip_next=yes;; \ -*O?*) strip_trailopt 'O';; \ -*l) strip_trailopt 'l'; skip_next=yes;; \ -*l?*) strip_trailopt 'l';; \ -[dEDm]) skip_next=yes;; \ -[JT]) skip_next=yes;; \ esac; \ case $$flg in \ *$$target_option*) has_opt=yes; break;; \ esac; \ done; \ test $$has_opt = yes am__make_dryrun = (target_option=n; $(am__make_running_with_option)) am__make_keepgoing = (target_option=k; $(am__make_running_with_option)) pkgincludedir = $(includedir)/@PACKAGE@ pkglibdir = $(libdir)/@PACKAGE@ pkglibexecdir = $(libexecdir)/@PACKAGE@ am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd install_sh_DATA = $(install_sh) -c -m 644 install_sh_PROGRAM = $(install_sh) -c install_sh_SCRIPT = $(install_sh) -c INSTALL_HEADER = $(INSTALL_DATA) transform = $(program_transform_name) NORMAL_INSTALL = : PRE_INSTALL = : POST_INSTALL = : NORMAL_UNINSTALL = : PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ subdir = magic DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \ $(top_srcdir)/m4/ltoptions.m4 $(top_srcdir)/m4/ltsugar.m4 \ $(top_srcdir)/m4/ltversion.m4 $(top_srcdir)/m4/lt~obsolete.m4 \ $(top_srcdir)/acinclude.m4 $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ $(ACLOCAL_M4) mkinstalldirs = $(install_sh) -d CONFIG_HEADER = $(top_builddir)/config.h CONFIG_CLEAN_FILES = CONFIG_CLEAN_VPATH_FILES = AM_V_P = $(am__v_P_@AM_V@) am__v_P_ = $(am__v_P_@AM_DEFAULT_V@) am__v_P_0 = false am__v_P_1 = : AM_V_GEN = $(am__v_GEN_@AM_V@) am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) am__v_GEN_0 = @echo " GEN " $@; am__v_GEN_1 = AM_V_at = $(am__v_at_@AM_V@) am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) am__v_at_0 = @ am__v_at_1 = SOURCES = DIST_SOURCES = am__can_run_installinfo = \ case $$AM_UPDATE_INFO_DIR in \ n|no|NO) false;; \ *) (install-info --version) >/dev/null 2>&1;; \ esac am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; am__vpath_adj = case $$p in \ $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ *) f=$$p;; \ esac; am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; am__install_max = 40 am__nobase_strip_setup = \ srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` am__nobase_strip = \ for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" am__nobase_list = $(am__nobase_strip_setup); \ for p in $$list; do echo "$$p $$p"; done | \ sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ if (++n[$$2] == $(am__install_max)) \ { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ END { for (dir in files) print dir, files[dir] }' am__base_list = \ sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' am__uninstall_files_from_dir = { \ test -z "$$files" \ || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ $(am__cd) "$$dir" && rm -f $$files; }; \ } am__installdirs = "$(DESTDIR)$(pkgdatadir)" DATA = $(pkgdata_DATA) am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP) DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) pkgdatadir = @pkgdatadir@ ACLOCAL = @ACLOCAL@ AMTAR = @AMTAR@ AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ AR = @AR@ AUTOCONF = @AUTOCONF@ AUTOHEADER = @AUTOHEADER@ AUTOMAKE = @AUTOMAKE@ AWK = @AWK@ CC = @CC@ CCDEPMODE = @CCDEPMODE@ CFLAGS = @CFLAGS@ CFLAG_VISIBILITY = @CFLAG_VISIBILITY@ CPP = @CPP@ CPPFLAGS = @CPPFLAGS@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DLLTOOL = @DLLTOOL@ DSYMUTIL = @DSYMUTIL@ DUMPBIN = @DUMPBIN@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ ECHO_T = @ECHO_T@ EGREP = @EGREP@ EXEEXT = @EXEEXT@ FGREP = @FGREP@ GREP = @GREP@ HAVE_VISIBILITY = @HAVE_VISIBILITY@ INSTALL = @INSTALL@ INSTALL_DATA = @INSTALL_DATA@ INSTALL_PROGRAM = @INSTALL_PROGRAM@ INSTALL_SCRIPT = @INSTALL_SCRIPT@ INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ LD = @LD@ LDFLAGS = @LDFLAGS@ LIBOBJS = @LIBOBJS@ LIBS = @LIBS@ LIBTOOL = @LIBTOOL@ LIPO = @LIPO@ LN_S = @LN_S@ LTLIBOBJS = @LTLIBOBJS@ MAKEINFO = @MAKEINFO@ MANIFEST_TOOL = @MANIFEST_TOOL@ MINGW = @MINGW@ MKDIR_P = @MKDIR_P@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ OTOOL = @OTOOL@ OTOOL64 = @OTOOL64@ PACKAGE = @PACKAGE@ PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ PACKAGE_NAME = @PACKAGE_NAME@ PACKAGE_STRING = @PACKAGE_STRING@ PACKAGE_TARNAME = @PACKAGE_TARNAME@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ RANLIB = @RANLIB@ SED = @SED@ SET_MAKE = @SET_MAKE@ SHELL = @SHELL@ STRIP = @STRIP@ VERSION = @VERSION@ WARNINGS = @WARNINGS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ abs_top_srcdir = @abs_top_srcdir@ ac_ct_AR = @ac_ct_AR@ ac_ct_CC = @ac_ct_CC@ ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ am__include = @am__include@ am__leading_dot = @am__leading_dot@ am__quote = @am__quote@ am__tar = @am__tar@ am__untar = @am__untar@ bindir = @bindir@ build = @build@ build_alias = @build_alias@ build_cpu = @build_cpu@ build_os = @build_os@ build_vendor = @build_vendor@ builddir = @builddir@ datadir = @datadir@ datarootdir = @datarootdir@ docdir = @docdir@ dvidir = @dvidir@ exec_prefix = @exec_prefix@ fsect = @fsect@ host = @host@ host_alias = @host_alias@ host_cpu = @host_cpu@ host_os = @host_os@ host_vendor = @host_vendor@ htmldir = @htmldir@ includedir = @includedir@ infodir = @infodir@ install_sh = @install_sh@ libdir = @libdir@ libexecdir = @libexecdir@ localedir = @localedir@ localstatedir = @localstatedir@ mandir = @mandir@ mkdir_p = @mkdir_p@ oldincludedir = @oldincludedir@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ sysconfdir = @sysconfdir@ target_alias = @target_alias@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ # -# $File: Makefile.am,v 1.98 2014/06/03 18:22:34 christos Exp $ +# $File: Makefile.am,v 1.101 2014/12/12 16:48:39 christos Exp $ # MAGIC_FRAGMENT_BASE = Magdir MAGIC_DIR = $(top_srcdir)/magic MAGIC_FRAGMENT_DIR = $(MAGIC_DIR)/$(MAGIC_FRAGMENT_BASE) pkgdata_DATA = magic.mgc EXTRA_DIST = \ $(MAGIC_DIR)/Header \ $(MAGIC_DIR)/Localstuff \ $(MAGIC_FRAGMENT_DIR)/acorn \ $(MAGIC_FRAGMENT_DIR)/adi \ $(MAGIC_FRAGMENT_DIR)/adventure \ $(MAGIC_FRAGMENT_DIR)/allegro \ $(MAGIC_FRAGMENT_DIR)/alliant \ $(MAGIC_FRAGMENT_DIR)/amanda \ $(MAGIC_FRAGMENT_DIR)/amigaos \ $(MAGIC_FRAGMENT_DIR)/android \ $(MAGIC_FRAGMENT_DIR)/animation \ $(MAGIC_FRAGMENT_DIR)/aout \ $(MAGIC_FRAGMENT_DIR)/apl \ $(MAGIC_FRAGMENT_DIR)/apple \ $(MAGIC_FRAGMENT_DIR)/applix \ $(MAGIC_FRAGMENT_DIR)/archive \ $(MAGIC_FRAGMENT_DIR)/assembler \ $(MAGIC_FRAGMENT_DIR)/asterix \ $(MAGIC_FRAGMENT_DIR)/att3b \ $(MAGIC_FRAGMENT_DIR)/audio \ $(MAGIC_FRAGMENT_DIR)/basis \ $(MAGIC_FRAGMENT_DIR)/bflt \ $(MAGIC_FRAGMENT_DIR)/blackberry \ $(MAGIC_FRAGMENT_DIR)/blcr \ $(MAGIC_FRAGMENT_DIR)/blender \ $(MAGIC_FRAGMENT_DIR)/blit \ $(MAGIC_FRAGMENT_DIR)/bout \ $(MAGIC_FRAGMENT_DIR)/bsdi \ $(MAGIC_FRAGMENT_DIR)/bsi \ $(MAGIC_FRAGMENT_DIR)/btsnoop \ $(MAGIC_FRAGMENT_DIR)/c-lang \ $(MAGIC_FRAGMENT_DIR)/c64 \ $(MAGIC_FRAGMENT_DIR)/cad \ $(MAGIC_FRAGMENT_DIR)/cafebabe \ $(MAGIC_FRAGMENT_DIR)/cddb \ $(MAGIC_FRAGMENT_DIR)/chord \ $(MAGIC_FRAGMENT_DIR)/cisco \ $(MAGIC_FRAGMENT_DIR)/citrus \ $(MAGIC_FRAGMENT_DIR)/clarion \ $(MAGIC_FRAGMENT_DIR)/claris \ $(MAGIC_FRAGMENT_DIR)/clipper \ $(MAGIC_FRAGMENT_DIR)/commands \ $(MAGIC_FRAGMENT_DIR)/communications \ $(MAGIC_FRAGMENT_DIR)/compress \ $(MAGIC_FRAGMENT_DIR)/console \ $(MAGIC_FRAGMENT_DIR)/convex \ $(MAGIC_FRAGMENT_DIR)/cracklib \ $(MAGIC_FRAGMENT_DIR)/ctags \ $(MAGIC_FRAGMENT_DIR)/ctf \ $(MAGIC_FRAGMENT_DIR)/cubemap \ $(MAGIC_FRAGMENT_DIR)/cups \ $(MAGIC_FRAGMENT_DIR)/dact \ $(MAGIC_FRAGMENT_DIR)/database \ $(MAGIC_FRAGMENT_DIR)/diamond \ $(MAGIC_FRAGMENT_DIR)/diff \ $(MAGIC_FRAGMENT_DIR)/digital \ $(MAGIC_FRAGMENT_DIR)/dolby \ $(MAGIC_FRAGMENT_DIR)/dump \ $(MAGIC_FRAGMENT_DIR)/dyadic \ $(MAGIC_FRAGMENT_DIR)/ebml \ $(MAGIC_FRAGMENT_DIR)/editors \ $(MAGIC_FRAGMENT_DIR)/efi \ $(MAGIC_FRAGMENT_DIR)/elf \ $(MAGIC_FRAGMENT_DIR)/encore \ $(MAGIC_FRAGMENT_DIR)/epoc \ $(MAGIC_FRAGMENT_DIR)/erlang \ $(MAGIC_FRAGMENT_DIR)/esri \ $(MAGIC_FRAGMENT_DIR)/fcs \ $(MAGIC_FRAGMENT_DIR)/filesystems \ $(MAGIC_FRAGMENT_DIR)/flash \ $(MAGIC_FRAGMENT_DIR)/fonts \ $(MAGIC_FRAGMENT_DIR)/fortran \ $(MAGIC_FRAGMENT_DIR)/frame \ $(MAGIC_FRAGMENT_DIR)/freebsd \ $(MAGIC_FRAGMENT_DIR)/fsav \ $(MAGIC_FRAGMENT_DIR)/fusecompress \ $(MAGIC_FRAGMENT_DIR)/games \ $(MAGIC_FRAGMENT_DIR)/gcc \ $(MAGIC_FRAGMENT_DIR)/geo \ $(MAGIC_FRAGMENT_DIR)/geos \ $(MAGIC_FRAGMENT_DIR)/gimp \ $(MAGIC_FRAGMENT_DIR)/gnome \ $(MAGIC_FRAGMENT_DIR)/gnu \ $(MAGIC_FRAGMENT_DIR)/gnumeric \ $(MAGIC_FRAGMENT_DIR)/gpt \ $(MAGIC_FRAGMENT_DIR)/grace \ $(MAGIC_FRAGMENT_DIR)/graphviz \ $(MAGIC_FRAGMENT_DIR)/gringotts \ $(MAGIC_FRAGMENT_DIR)/guile \ $(MAGIC_FRAGMENT_DIR)/hitachi-sh \ $(MAGIC_FRAGMENT_DIR)/hp \ $(MAGIC_FRAGMENT_DIR)/human68k \ $(MAGIC_FRAGMENT_DIR)/ibm370 \ $(MAGIC_FRAGMENT_DIR)/ibm6000 \ $(MAGIC_FRAGMENT_DIR)/icc \ $(MAGIC_FRAGMENT_DIR)/iff \ $(MAGIC_FRAGMENT_DIR)/images \ $(MAGIC_FRAGMENT_DIR)/inform \ $(MAGIC_FRAGMENT_DIR)/intel \ $(MAGIC_FRAGMENT_DIR)/interleaf \ $(MAGIC_FRAGMENT_DIR)/island \ $(MAGIC_FRAGMENT_DIR)/ispell \ $(MAGIC_FRAGMENT_DIR)/isz \ $(MAGIC_FRAGMENT_DIR)/java \ $(MAGIC_FRAGMENT_DIR)/javascript \ $(MAGIC_FRAGMENT_DIR)/jpeg \ $(MAGIC_FRAGMENT_DIR)/karma \ $(MAGIC_FRAGMENT_DIR)/kde \ $(MAGIC_FRAGMENT_DIR)/keepass \ +$(MAGIC_FRAGMENT_DIR)/kerberos \ $(MAGIC_FRAGMENT_DIR)/kml \ $(MAGIC_FRAGMENT_DIR)/lecter \ $(MAGIC_FRAGMENT_DIR)/lex \ $(MAGIC_FRAGMENT_DIR)/lif \ $(MAGIC_FRAGMENT_DIR)/linux \ $(MAGIC_FRAGMENT_DIR)/lisp \ $(MAGIC_FRAGMENT_DIR)/llvm \ $(MAGIC_FRAGMENT_DIR)/lua \ $(MAGIC_FRAGMENT_DIR)/luks \ $(MAGIC_FRAGMENT_DIR)/m4 \ $(MAGIC_FRAGMENT_DIR)/mach \ $(MAGIC_FRAGMENT_DIR)/macos \ $(MAGIC_FRAGMENT_DIR)/macintosh \ $(MAGIC_FRAGMENT_DIR)/magic \ $(MAGIC_FRAGMENT_DIR)/mail.news \ $(MAGIC_FRAGMENT_DIR)/make \ $(MAGIC_FRAGMENT_DIR)/map \ $(MAGIC_FRAGMENT_DIR)/maple \ $(MAGIC_FRAGMENT_DIR)/marc21 \ $(MAGIC_FRAGMENT_DIR)/mathcad \ $(MAGIC_FRAGMENT_DIR)/mathematica \ $(MAGIC_FRAGMENT_DIR)/matroska \ $(MAGIC_FRAGMENT_DIR)/mcrypt \ $(MAGIC_FRAGMENT_DIR)/mercurial \ $(MAGIC_FRAGMENT_DIR)/metastore \ +$(MAGIC_FRAGMENT_DIR)/meteorological \ $(MAGIC_FRAGMENT_DIR)/mime \ $(MAGIC_FRAGMENT_DIR)/mips \ $(MAGIC_FRAGMENT_DIR)/mirage \ $(MAGIC_FRAGMENT_DIR)/misctools \ $(MAGIC_FRAGMENT_DIR)/mkid \ $(MAGIC_FRAGMENT_DIR)/mlssa \ $(MAGIC_FRAGMENT_DIR)/mmdf \ $(MAGIC_FRAGMENT_DIR)/modem \ $(MAGIC_FRAGMENT_DIR)/motorola \ $(MAGIC_FRAGMENT_DIR)/mozilla \ $(MAGIC_FRAGMENT_DIR)/msdos \ $(MAGIC_FRAGMENT_DIR)/msooxml \ $(MAGIC_FRAGMENT_DIR)/msx \ $(MAGIC_FRAGMENT_DIR)/msvc \ $(MAGIC_FRAGMENT_DIR)/mup \ $(MAGIC_FRAGMENT_DIR)/music \ $(MAGIC_FRAGMENT_DIR)/natinst \ $(MAGIC_FRAGMENT_DIR)/ncr \ $(MAGIC_FRAGMENT_DIR)/neko \ $(MAGIC_FRAGMENT_DIR)/netbsd \ $(MAGIC_FRAGMENT_DIR)/netscape \ $(MAGIC_FRAGMENT_DIR)/netware \ $(MAGIC_FRAGMENT_DIR)/news \ $(MAGIC_FRAGMENT_DIR)/nitpicker \ $(MAGIC_FRAGMENT_DIR)/oasis \ $(MAGIC_FRAGMENT_DIR)/ocaml \ $(MAGIC_FRAGMENT_DIR)/octave \ $(MAGIC_FRAGMENT_DIR)/ole2compounddocs \ $(MAGIC_FRAGMENT_DIR)/olf \ $(MAGIC_FRAGMENT_DIR)/os2 \ $(MAGIC_FRAGMENT_DIR)/os400 \ $(MAGIC_FRAGMENT_DIR)/os9 \ $(MAGIC_FRAGMENT_DIR)/osf1 \ $(MAGIC_FRAGMENT_DIR)/palm \ $(MAGIC_FRAGMENT_DIR)/parix \ $(MAGIC_FRAGMENT_DIR)/parrot \ $(MAGIC_FRAGMENT_DIR)/pascal \ $(MAGIC_FRAGMENT_DIR)/pbf \ $(MAGIC_FRAGMENT_DIR)/pbm \ $(MAGIC_FRAGMENT_DIR)/pdf \ $(MAGIC_FRAGMENT_DIR)/pdp \ $(MAGIC_FRAGMENT_DIR)/perl \ $(MAGIC_FRAGMENT_DIR)/pgf \ $(MAGIC_FRAGMENT_DIR)/pgp \ $(MAGIC_FRAGMENT_DIR)/pkgadd \ $(MAGIC_FRAGMENT_DIR)/plan9 \ $(MAGIC_FRAGMENT_DIR)/plus5 \ $(MAGIC_FRAGMENT_DIR)/printer \ $(MAGIC_FRAGMENT_DIR)/project \ $(MAGIC_FRAGMENT_DIR)/psdbms \ $(MAGIC_FRAGMENT_DIR)/pulsar \ $(MAGIC_FRAGMENT_DIR)/pwsafe \ $(MAGIC_FRAGMENT_DIR)/pyramid \ $(MAGIC_FRAGMENT_DIR)/python \ +$(MAGIC_FRAGMENT_DIR)/qt \ $(MAGIC_FRAGMENT_DIR)/revision \ $(MAGIC_FRAGMENT_DIR)/riff \ -$(MAGIC_FRAGMENT_DIR)/rinex \ $(MAGIC_FRAGMENT_DIR)/rpm \ $(MAGIC_FRAGMENT_DIR)/rtf \ $(MAGIC_FRAGMENT_DIR)/ruby \ $(MAGIC_FRAGMENT_DIR)/sc \ $(MAGIC_FRAGMENT_DIR)/sccs \ $(MAGIC_FRAGMENT_DIR)/scientific \ $(MAGIC_FRAGMENT_DIR)/securitycerts \ $(MAGIC_FRAGMENT_DIR)/selinux \ $(MAGIC_FRAGMENT_DIR)/sendmail \ $(MAGIC_FRAGMENT_DIR)/sequent \ $(MAGIC_FRAGMENT_DIR)/sereal \ $(MAGIC_FRAGMENT_DIR)/sgi \ $(MAGIC_FRAGMENT_DIR)/sgml \ $(MAGIC_FRAGMENT_DIR)/sharc \ $(MAGIC_FRAGMENT_DIR)/sinclair \ $(MAGIC_FRAGMENT_DIR)/sisu \ $(MAGIC_FRAGMENT_DIR)/sketch \ $(MAGIC_FRAGMENT_DIR)/smalltalk \ $(MAGIC_FRAGMENT_DIR)/smile \ $(MAGIC_FRAGMENT_DIR)/sniffer \ $(MAGIC_FRAGMENT_DIR)/softquad \ $(MAGIC_FRAGMENT_DIR)/spec \ $(MAGIC_FRAGMENT_DIR)/spectrum \ $(MAGIC_FRAGMENT_DIR)/sql \ $(MAGIC_FRAGMENT_DIR)/ssh \ $(MAGIC_FRAGMENT_DIR)/ssl \ $(MAGIC_FRAGMENT_DIR)/sun \ $(MAGIC_FRAGMENT_DIR)/symbos \ $(MAGIC_FRAGMENT_DIR)/sysex \ $(MAGIC_FRAGMENT_DIR)/tcl \ $(MAGIC_FRAGMENT_DIR)/teapot \ $(MAGIC_FRAGMENT_DIR)/terminfo \ $(MAGIC_FRAGMENT_DIR)/tex \ $(MAGIC_FRAGMENT_DIR)/tgif \ $(MAGIC_FRAGMENT_DIR)/ti-8x \ $(MAGIC_FRAGMENT_DIR)/timezone \ $(MAGIC_FRAGMENT_DIR)/troff \ $(MAGIC_FRAGMENT_DIR)/tuxedo \ $(MAGIC_FRAGMENT_DIR)/typeset \ $(MAGIC_FRAGMENT_DIR)/unicode \ $(MAGIC_FRAGMENT_DIR)/unknown \ $(MAGIC_FRAGMENT_DIR)/uterus \ $(MAGIC_FRAGMENT_DIR)/uuencode \ $(MAGIC_FRAGMENT_DIR)/varied.out \ $(MAGIC_FRAGMENT_DIR)/varied.script \ $(MAGIC_FRAGMENT_DIR)/vax \ $(MAGIC_FRAGMENT_DIR)/vicar \ $(MAGIC_FRAGMENT_DIR)/virtual \ $(MAGIC_FRAGMENT_DIR)/virtutech \ $(MAGIC_FRAGMENT_DIR)/visx \ $(MAGIC_FRAGMENT_DIR)/vms \ $(MAGIC_FRAGMENT_DIR)/vmware \ $(MAGIC_FRAGMENT_DIR)/vorbis \ $(MAGIC_FRAGMENT_DIR)/vxl \ $(MAGIC_FRAGMENT_DIR)/warc \ $(MAGIC_FRAGMENT_DIR)/weak \ $(MAGIC_FRAGMENT_DIR)/windows \ $(MAGIC_FRAGMENT_DIR)/wireless \ $(MAGIC_FRAGMENT_DIR)/wordprocessors \ $(MAGIC_FRAGMENT_DIR)/wsdl \ $(MAGIC_FRAGMENT_DIR)/xdelta \ $(MAGIC_FRAGMENT_DIR)/xenix \ $(MAGIC_FRAGMENT_DIR)/xilinx \ $(MAGIC_FRAGMENT_DIR)/xo65 \ $(MAGIC_FRAGMENT_DIR)/xwindows \ $(MAGIC_FRAGMENT_DIR)/zfs \ $(MAGIC_FRAGMENT_DIR)/zilog \ $(MAGIC_FRAGMENT_DIR)/zyxel MAGIC = magic.mgc CLEANFILES = ${MAGIC} $(MAGIC_FRAGMENT_DIR)/Localstuff @IS_CROSS_COMPILE_FALSE@FILE_COMPILE = $(top_builddir)/src/file${EXEEXT} # FIXME: Build file natively as well so that it can be used to compile # the target's magic file; for now we bail if the local version does not match @IS_CROSS_COMPILE_TRUE@FILE_COMPILE = file${EXEEXT} @IS_CROSS_COMPILE_FALSE@FILE_COMPILE_DEP = $(FILE_COMPILE) @IS_CROSS_COMPILE_TRUE@FILE_COMPILE_DEP = all: all-am .SUFFIXES: $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) @for dep in $?; do \ case '$(am__configure_deps)' in \ *$$dep*) \ ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ && { if test -f $@; then exit 0; else break; fi; }; \ exit 1;; \ esac; \ done; \ echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign magic/Makefile'; \ $(am__cd) $(top_srcdir) && \ $(AUTOMAKE) --foreign magic/Makefile .PRECIOUS: Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ *) \ echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ esac; $(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(top_srcdir)/configure: $(am__configure_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(ACLOCAL_M4): $(am__aclocal_m4_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(am__aclocal_m4_deps): mostlyclean-libtool: -rm -f *.lo clean-libtool: -rm -rf .libs _libs install-pkgdataDATA: $(pkgdata_DATA) @$(NORMAL_INSTALL) @list='$(pkgdata_DATA)'; test -n "$(pkgdatadir)" || list=; \ if test -n "$$list"; then \ echo " $(MKDIR_P) '$(DESTDIR)$(pkgdatadir)'"; \ $(MKDIR_P) "$(DESTDIR)$(pkgdatadir)" || exit 1; \ fi; \ for p in $$list; do \ if test -f "$$p"; then d=; else d="$(srcdir)/"; fi; \ echo "$$d$$p"; \ done | $(am__base_list) | \ while read files; do \ echo " $(INSTALL_DATA) $$files '$(DESTDIR)$(pkgdatadir)'"; \ $(INSTALL_DATA) $$files "$(DESTDIR)$(pkgdatadir)" || exit $$?; \ done uninstall-pkgdataDATA: @$(NORMAL_UNINSTALL) @list='$(pkgdata_DATA)'; test -n "$(pkgdatadir)" || list=; \ files=`for p in $$list; do echo $$p; done | sed -e 's|^.*/||'`; \ dir='$(DESTDIR)$(pkgdatadir)'; $(am__uninstall_files_from_dir) tags TAGS: ctags CTAGS: cscope cscopelist: distdir: $(DISTFILES) @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ list='$(DISTFILES)'; \ dist_files=`for file in $$list; do echo $$file; done | \ sed -e "s|^$$srcdirstrip/||;t" \ -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ case $$dist_files in \ */*) $(MKDIR_P) `echo "$$dist_files" | \ sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ sort -u` ;; \ esac; \ for file in $$dist_files; do \ if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ if test -d $$d/$$file; then \ dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ if test -d "$(distdir)/$$file"; then \ find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ fi; \ if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ fi; \ cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ else \ test -f "$(distdir)/$$file" \ || cp -p $$d/$$file "$(distdir)/$$file" \ || exit 1; \ fi; \ done check-am: all-am check: check-am all-am: Makefile $(DATA) installdirs: for dir in "$(DESTDIR)$(pkgdatadir)"; do \ test -z "$$dir" || $(MKDIR_P) "$$dir"; \ done install: install-am install-exec: install-exec-am install-data: install-data-am uninstall: uninstall-am install-am: all-am @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am installcheck: installcheck-am install-strip: if test -z '$(STRIP)'; then \ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ install; \ else \ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ fi mostlyclean-generic: clean-generic: -test -z "$(CLEANFILES)" || rm -f $(CLEANFILES) distclean-generic: -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) maintainer-clean-generic: @echo "This command is intended for maintainers to use" @echo "it deletes files that may require special tools to rebuild." clean: clean-am clean-am: clean-generic clean-libtool mostlyclean-am distclean: distclean-am -rm -f Makefile distclean-am: clean-am distclean-generic dvi: dvi-am dvi-am: html: html-am html-am: info: info-am info-am: install-data-am: install-pkgdataDATA install-dvi: install-dvi-am install-dvi-am: install-exec-am: install-html: install-html-am install-html-am: install-info: install-info-am install-info-am: install-man: install-pdf: install-pdf-am install-pdf-am: install-ps: install-ps-am install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-am -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic mostlyclean: mostlyclean-am mostlyclean-am: mostlyclean-generic mostlyclean-libtool pdf: pdf-am pdf-am: ps: ps-am ps-am: uninstall-am: uninstall-pkgdataDATA .MAKE: install-am install-strip .PHONY: all all-am check check-am clean clean-generic clean-libtool \ cscopelist-am ctags-am distclean distclean-generic \ distclean-libtool distdir dvi dvi-am html html-am info info-am \ install install-am install-data install-data-am install-dvi \ install-dvi-am install-exec install-exec-am install-html \ install-html-am install-info install-info-am install-man \ install-pdf install-pdf-am install-pkgdataDATA install-ps \ install-ps-am install-strip installcheck installcheck-am \ installdirs maintainer-clean maintainer-clean-generic \ mostlyclean mostlyclean-generic mostlyclean-libtool pdf pdf-am \ ps ps-am tags-am uninstall uninstall-am uninstall-pkgdataDATA ${MAGIC}: $(EXTRA_DIST) $(FILE_COMPILE_DEP) @rm -fr magic @mkdir magic && cp -p $(EXTRA_DIST) magic @(if expr "${FILE_COMPILE}" : '.*/.*' > /dev/null; then \ echo "Using ${FILE_COMPILE} to generate ${MAGIC}" > /dev/null; \ else \ v=$$(file --version | sed -e s/file-// -e q); \ if [ "$$v" != "${PACKAGE_VERSION}" ]; then \ echo "Cannot use the installed version of file ($$v) to"; \ echo "cross-compile file ${PACKAGE_VERSION}"; \ echo "Please install file ${PACKAGE_VERSION} locally first"; \ exit 1; \ fi; \ fi) $(FILE_COMPILE) -C -m magic @rm -fr magic # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. .NOEXPORT: Index: releng/10.1/contrib/file/src/Makefile.in =================================================================== --- releng/10.1/contrib/file/src/Makefile.in (revision 284192) +++ releng/10.1/contrib/file/src/Makefile.in (revision 284193) @@ -1,783 +1,783 @@ # Makefile.in generated by automake 1.14 from Makefile.am. # @configure_input@ # Copyright (C) 1994-2013 Free Software Foundation, Inc. # This Makefile.in is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY, to the extent permitted by law; without # even the implied warranty of MERCHANTABILITY or FITNESS FOR A # PARTICULAR PURPOSE. @SET_MAKE@ VPATH = @srcdir@ am__is_gnu_make = test -n '$(MAKEFILE_LIST)' && test -n '$(MAKELEVEL)' am__make_running_with_option = \ case $${target_option-} in \ ?) ;; \ *) echo "am__make_running_with_option: internal error: invalid" \ "target option '$${target_option-}' specified" >&2; \ exit 1;; \ esac; \ has_opt=no; \ sane_makeflags=$$MAKEFLAGS; \ if $(am__is_gnu_make); then \ sane_makeflags=$$MFLAGS; \ else \ case $$MAKEFLAGS in \ *\\[\ \ ]*) \ bs=\\; \ sane_makeflags=`printf '%s\n' "$$MAKEFLAGS" \ | sed "s/$$bs$$bs[$$bs $$bs ]*//g"`;; \ esac; \ fi; \ skip_next=no; \ strip_trailopt () \ { \ flg=`printf '%s\n' "$$flg" | sed "s/$$1.*$$//"`; \ }; \ for flg in $$sane_makeflags; do \ test $$skip_next = yes && { skip_next=no; continue; }; \ case $$flg in \ *=*|--*) continue;; \ -*I) strip_trailopt 'I'; skip_next=yes;; \ -*I?*) strip_trailopt 'I';; \ -*O) strip_trailopt 'O'; skip_next=yes;; \ -*O?*) strip_trailopt 'O';; \ -*l) strip_trailopt 'l'; skip_next=yes;; \ -*l?*) strip_trailopt 'l';; \ -[dEDm]) skip_next=yes;; \ -[JT]) skip_next=yes;; \ esac; \ case $$flg in \ *$$target_option*) has_opt=yes; break;; \ esac; \ done; \ test $$has_opt = yes am__make_dryrun = (target_option=n; $(am__make_running_with_option)) am__make_keepgoing = (target_option=k; $(am__make_running_with_option)) pkgincludedir = $(includedir)/@PACKAGE@ pkglibdir = $(libdir)/@PACKAGE@ pkglibexecdir = $(libexecdir)/@PACKAGE@ am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd install_sh_DATA = $(install_sh) -c -m 644 install_sh_PROGRAM = $(install_sh) -c install_sh_SCRIPT = $(install_sh) -c INSTALL_HEADER = $(INSTALL_DATA) transform = $(program_transform_name) NORMAL_INSTALL = : PRE_INSTALL = : POST_INSTALL = : NORMAL_UNINSTALL = : PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ bin_PROGRAMS = file$(EXEEXT) subdir = src -DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am pread.c \ - ctime_r.c getline.c vasprintf.c asprintf.c asctime_r.c \ - fmtcheck.c strlcpy.c getopt_long.c strcasestr.c strlcat.c \ +DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am ctime_r.c \ + vasprintf.c asctime_r.c asprintf.c strcasestr.c pread.c \ + getline.c strlcpy.c strlcat.c fmtcheck.c getopt_long.c \ $(top_srcdir)/depcomp $(include_HEADERS) ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \ $(top_srcdir)/m4/ltoptions.m4 $(top_srcdir)/m4/ltsugar.m4 \ $(top_srcdir)/m4/ltversion.m4 $(top_srcdir)/m4/lt~obsolete.m4 \ $(top_srcdir)/acinclude.m4 $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ $(ACLOCAL_M4) mkinstalldirs = $(install_sh) -d CONFIG_HEADER = $(top_builddir)/config.h CONFIG_CLEAN_FILES = CONFIG_CLEAN_VPATH_FILES = am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; am__vpath_adj = case $$p in \ $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ *) f=$$p;; \ esac; am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; am__install_max = 40 am__nobase_strip_setup = \ srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` am__nobase_strip = \ for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" am__nobase_list = $(am__nobase_strip_setup); \ for p in $$list; do echo "$$p $$p"; done | \ sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ if (++n[$$2] == $(am__install_max)) \ { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ END { for (dir in files) print dir, files[dir] }' am__base_list = \ sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' am__uninstall_files_from_dir = { \ test -z "$$files" \ || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ $(am__cd) "$$dir" && rm -f $$files; }; \ } am__installdirs = "$(DESTDIR)$(libdir)" "$(DESTDIR)$(bindir)" \ "$(DESTDIR)$(includedir)" LTLIBRARIES = $(lib_LTLIBRARIES) am__DEPENDENCIES_1 = libmagic_la_DEPENDENCIES = $(LTLIBOBJS) $(am__DEPENDENCIES_1) am_libmagic_la_OBJECTS = magic.lo apprentice.lo softmagic.lo \ ascmagic.lo encoding.lo compress.lo is_tar.lo readelf.lo \ print.lo fsmagic.lo funcs.lo apptype.lo cdf.lo cdf_time.lo \ readcdf.lo libmagic_la_OBJECTS = $(am_libmagic_la_OBJECTS) AM_V_lt = $(am__v_lt_@AM_V@) am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) am__v_lt_0 = --silent am__v_lt_1 = libmagic_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ $(libmagic_la_LDFLAGS) $(LDFLAGS) -o $@ PROGRAMS = $(bin_PROGRAMS) am_file_OBJECTS = file.$(OBJEXT) file_OBJECTS = $(am_file_OBJECTS) file_DEPENDENCIES = libmagic.la AM_V_P = $(am__v_P_@AM_V@) am__v_P_ = $(am__v_P_@AM_DEFAULT_V@) am__v_P_0 = false am__v_P_1 = : AM_V_GEN = $(am__v_GEN_@AM_V@) am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) am__v_GEN_0 = @echo " GEN " $@; am__v_GEN_1 = AM_V_at = $(am__v_at_@AM_V@) am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) am__v_at_0 = @ am__v_at_1 = DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) depcomp = $(SHELL) $(top_srcdir)/depcomp am__depfiles_maybe = depfiles am__mv = mv -f COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ $(AM_CFLAGS) $(CFLAGS) AM_V_CC = $(am__v_CC_@AM_V@) am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) am__v_CC_0 = @echo " CC " $@; am__v_CC_1 = CCLD = $(CC) LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ $(AM_LDFLAGS) $(LDFLAGS) -o $@ AM_V_CCLD = $(am__v_CCLD_@AM_V@) am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) am__v_CCLD_0 = @echo " CCLD " $@; am__v_CCLD_1 = SOURCES = $(libmagic_la_SOURCES) $(file_SOURCES) DIST_SOURCES = $(libmagic_la_SOURCES) $(file_SOURCES) am__can_run_installinfo = \ case $$AM_UPDATE_INFO_DIR in \ n|no|NO) false;; \ *) (install-info --version) >/dev/null 2>&1;; \ esac HEADERS = $(include_HEADERS) am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP) # Read a list of newline-separated strings from the standard input, # and print each of them once, without duplicates. Input order is # *not* preserved. am__uniquify_input = $(AWK) '\ BEGIN { nonempty = 0; } \ { items[$$0] = 1; nonempty = 1; } \ END { if (nonempty) { for (i in items) print i; }; } \ ' # Make sure the list of sources is unique. This is necessary because, # e.g., the same source file might be shared among _SOURCES variables # for different programs/libraries. am__define_uniq_tagged_files = \ list='$(am__tagged_files)'; \ unique=`for i in $$list; do \ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ done | $(am__uniquify_input)` ETAGS = etags CTAGS = ctags DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) pkgdatadir = @pkgdatadir@ ACLOCAL = @ACLOCAL@ AMTAR = @AMTAR@ AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ AR = @AR@ AUTOCONF = @AUTOCONF@ AUTOHEADER = @AUTOHEADER@ AUTOMAKE = @AUTOMAKE@ AWK = @AWK@ CC = @CC@ CCDEPMODE = @CCDEPMODE@ CFLAGS = @CFLAGS@ CFLAG_VISIBILITY = @CFLAG_VISIBILITY@ CPP = @CPP@ CPPFLAGS = @CPPFLAGS@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ DLLTOOL = @DLLTOOL@ DSYMUTIL = @DSYMUTIL@ DUMPBIN = @DUMPBIN@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ ECHO_T = @ECHO_T@ EGREP = @EGREP@ EXEEXT = @EXEEXT@ FGREP = @FGREP@ GREP = @GREP@ HAVE_VISIBILITY = @HAVE_VISIBILITY@ INSTALL = @INSTALL@ INSTALL_DATA = @INSTALL_DATA@ INSTALL_PROGRAM = @INSTALL_PROGRAM@ INSTALL_SCRIPT = @INSTALL_SCRIPT@ INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ LD = @LD@ LDFLAGS = @LDFLAGS@ LIBOBJS = @LIBOBJS@ LIBS = @LIBS@ LIBTOOL = @LIBTOOL@ LIPO = @LIPO@ LN_S = @LN_S@ LTLIBOBJS = @LTLIBOBJS@ MAKEINFO = @MAKEINFO@ MANIFEST_TOOL = @MANIFEST_TOOL@ MINGW = @MINGW@ MKDIR_P = @MKDIR_P@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ OTOOL = @OTOOL@ OTOOL64 = @OTOOL64@ PACKAGE = @PACKAGE@ PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ PACKAGE_NAME = @PACKAGE_NAME@ PACKAGE_STRING = @PACKAGE_STRING@ PACKAGE_TARNAME = @PACKAGE_TARNAME@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ RANLIB = @RANLIB@ SED = @SED@ SET_MAKE = @SET_MAKE@ SHELL = @SHELL@ STRIP = @STRIP@ VERSION = @VERSION@ WARNINGS = @WARNINGS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ abs_top_srcdir = @abs_top_srcdir@ ac_ct_AR = @ac_ct_AR@ ac_ct_CC = @ac_ct_CC@ ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ am__include = @am__include@ am__leading_dot = @am__leading_dot@ am__quote = @am__quote@ am__tar = @am__tar@ am__untar = @am__untar@ bindir = @bindir@ build = @build@ build_alias = @build_alias@ build_cpu = @build_cpu@ build_os = @build_os@ build_vendor = @build_vendor@ builddir = @builddir@ datadir = @datadir@ datarootdir = @datarootdir@ docdir = @docdir@ dvidir = @dvidir@ exec_prefix = @exec_prefix@ fsect = @fsect@ host = @host@ host_alias = @host_alias@ host_cpu = @host_cpu@ host_os = @host_os@ host_vendor = @host_vendor@ htmldir = @htmldir@ includedir = @includedir@ infodir = @infodir@ install_sh = @install_sh@ libdir = @libdir@ libexecdir = @libexecdir@ localedir = @localedir@ localstatedir = @localstatedir@ mandir = @mandir@ mkdir_p = @mkdir_p@ oldincludedir = @oldincludedir@ pdfdir = @pdfdir@ prefix = @prefix@ program_transform_name = @program_transform_name@ psdir = @psdir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ srcdir = @srcdir@ sysconfdir = @sysconfdir@ target_alias = @target_alias@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ MAGIC = $(pkgdatadir)/magic lib_LTLIBRARIES = libmagic.la include_HEADERS = magic.h AM_CPPFLAGS = -DMAGIC='"$(MAGIC)"' AM_CFLAGS = $(CFLAG_VISIBILITY) @WARNINGS@ libmagic_la_SOURCES = magic.c apprentice.c softmagic.c ascmagic.c \ encoding.c compress.c is_tar.c readelf.c print.c fsmagic.c \ funcs.c file.h readelf.h tar.h apptype.c \ file_opts.h elfclass.h mygetopt.h cdf.c cdf_time.c readcdf.c cdf.h libmagic_la_LDFLAGS = -no-undefined -version-info 1:0:0 @MINGW_FALSE@MINGWLIBS = @MINGW_TRUE@MINGWLIBS = -lgnurx -lshlwapi libmagic_la_LIBADD = $(LTLIBOBJS) $(MINGWLIBS) file_SOURCES = file.c file_LDADD = libmagic.la CLEANFILES = magic.h EXTRA_DIST = magic.h.in HDR = $(top_srcdir)/src/magic.h.in BUILT_SOURCES = magic.h all: $(BUILT_SOURCES) $(MAKE) $(AM_MAKEFLAGS) all-am .SUFFIXES: .SUFFIXES: .c .lo .o .obj $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) @for dep in $?; do \ case '$(am__configure_deps)' in \ *$$dep*) \ ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ && { if test -f $@; then exit 0; else break; fi; }; \ exit 1;; \ esac; \ done; \ echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign src/Makefile'; \ $(am__cd) $(top_srcdir) && \ $(AUTOMAKE) --foreign src/Makefile .PRECIOUS: Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ *) \ echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ esac; $(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(top_srcdir)/configure: $(am__configure_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(ACLOCAL_M4): $(am__aclocal_m4_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(am__aclocal_m4_deps): install-libLTLIBRARIES: $(lib_LTLIBRARIES) @$(NORMAL_INSTALL) @list='$(lib_LTLIBRARIES)'; test -n "$(libdir)" || list=; \ list2=; for p in $$list; do \ if test -f $$p; then \ list2="$$list2 $$p"; \ else :; fi; \ done; \ test -z "$$list2" || { \ echo " $(MKDIR_P) '$(DESTDIR)$(libdir)'"; \ $(MKDIR_P) "$(DESTDIR)$(libdir)" || exit 1; \ echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(libdir)'"; \ $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(libdir)"; \ } uninstall-libLTLIBRARIES: @$(NORMAL_UNINSTALL) @list='$(lib_LTLIBRARIES)'; test -n "$(libdir)" || list=; \ for p in $$list; do \ $(am__strip_dir) \ echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(libdir)/$$f'"; \ $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(libdir)/$$f"; \ done clean-libLTLIBRARIES: -test -z "$(lib_LTLIBRARIES)" || rm -f $(lib_LTLIBRARIES) @list='$(lib_LTLIBRARIES)'; \ locs=`for p in $$list; do echo $$p; done | \ sed 's|^[^/]*$$|.|; s|/[^/]*$$||; s|$$|/so_locations|' | \ sort -u`; \ test -z "$$locs" || { \ echo rm -f $${locs}; \ rm -f $${locs}; \ } libmagic.la: $(libmagic_la_OBJECTS) $(libmagic_la_DEPENDENCIES) $(EXTRA_libmagic_la_DEPENDENCIES) $(AM_V_CCLD)$(libmagic_la_LINK) -rpath $(libdir) $(libmagic_la_OBJECTS) $(libmagic_la_LIBADD) $(LIBS) install-binPROGRAMS: $(bin_PROGRAMS) @$(NORMAL_INSTALL) @list='$(bin_PROGRAMS)'; test -n "$(bindir)" || list=; \ if test -n "$$list"; then \ echo " $(MKDIR_P) '$(DESTDIR)$(bindir)'"; \ $(MKDIR_P) "$(DESTDIR)$(bindir)" || exit 1; \ fi; \ for p in $$list; do echo "$$p $$p"; done | \ sed 's/$(EXEEXT)$$//' | \ while read p p1; do if test -f $$p \ || test -f $$p1 \ ; then echo "$$p"; echo "$$p"; else :; fi; \ done | \ sed -e 'p;s,.*/,,;n;h' \ -e 's|.*|.|' \ -e 'p;x;s,.*/,,;s/$(EXEEXT)$$//;$(transform);s/$$/$(EXEEXT)/' | \ sed 'N;N;N;s,\n, ,g' | \ $(AWK) 'BEGIN { files["."] = ""; dirs["."] = 1 } \ { d=$$3; if (dirs[d] != 1) { print "d", d; dirs[d] = 1 } \ if ($$2 == $$4) files[d] = files[d] " " $$1; \ else { print "f", $$3 "/" $$4, $$1; } } \ END { for (d in files) print "f", d, files[d] }' | \ while read type dir files; do \ if test "$$dir" = .; then dir=; else dir=/$$dir; fi; \ test -z "$$files" || { \ echo " $(INSTALL_PROGRAM_ENV) $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL_PROGRAM) $$files '$(DESTDIR)$(bindir)$$dir'"; \ $(INSTALL_PROGRAM_ENV) $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL_PROGRAM) $$files "$(DESTDIR)$(bindir)$$dir" || exit $$?; \ } \ ; done uninstall-binPROGRAMS: @$(NORMAL_UNINSTALL) @list='$(bin_PROGRAMS)'; test -n "$(bindir)" || list=; \ files=`for p in $$list; do echo "$$p"; done | \ sed -e 'h;s,^.*/,,;s/$(EXEEXT)$$//;$(transform)' \ -e 's/$$/$(EXEEXT)/' \ `; \ test -n "$$list" || exit 0; \ echo " ( cd '$(DESTDIR)$(bindir)' && rm -f" $$files ")"; \ cd "$(DESTDIR)$(bindir)" && rm -f $$files clean-binPROGRAMS: @list='$(bin_PROGRAMS)'; test -n "$$list" || exit 0; \ echo " rm -f" $$list; \ rm -f $$list || exit $$?; \ test -n "$(EXEEXT)" || exit 0; \ list=`for p in $$list; do echo "$$p"; done | sed 's/$(EXEEXT)$$//'`; \ echo " rm -f" $$list; \ rm -f $$list file$(EXEEXT): $(file_OBJECTS) $(file_DEPENDENCIES) $(EXTRA_file_DEPENDENCIES) @rm -f file$(EXEEXT) $(AM_V_CCLD)$(LINK) $(file_OBJECTS) $(file_LDADD) $(LIBS) mostlyclean-compile: -rm -f *.$(OBJEXT) distclean-compile: -rm -f *.tab.c @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/asctime_r.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/asprintf.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/ctime_r.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/fmtcheck.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/getline.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/getopt_long.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/pread.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/strcasestr.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/strlcat.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/strlcpy.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@$(DEPDIR)/vasprintf.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/apprentice.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/apptype.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascmagic.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/cdf.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/cdf_time.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/compress.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/encoding.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/file.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/fsmagic.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/funcs.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/is_tar.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/magic.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/print.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/readcdf.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/readelf.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/softmagic.Plo@am__quote@ .c.o: @am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ @am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ @am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: @am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ @am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ @am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: @am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ @am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ @am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< mostlyclean-libtool: -rm -f *.lo clean-libtool: -rm -rf .libs _libs install-includeHEADERS: $(include_HEADERS) @$(NORMAL_INSTALL) @list='$(include_HEADERS)'; test -n "$(includedir)" || list=; \ if test -n "$$list"; then \ echo " $(MKDIR_P) '$(DESTDIR)$(includedir)'"; \ $(MKDIR_P) "$(DESTDIR)$(includedir)" || exit 1; \ fi; \ for p in $$list; do \ if test -f "$$p"; then d=; else d="$(srcdir)/"; fi; \ echo "$$d$$p"; \ done | $(am__base_list) | \ while read files; do \ echo " $(INSTALL_HEADER) $$files '$(DESTDIR)$(includedir)'"; \ $(INSTALL_HEADER) $$files "$(DESTDIR)$(includedir)" || exit $$?; \ done uninstall-includeHEADERS: @$(NORMAL_UNINSTALL) @list='$(include_HEADERS)'; test -n "$(includedir)" || list=; \ files=`for p in $$list; do echo $$p; done | sed -e 's|^.*/||'`; \ dir='$(DESTDIR)$(includedir)'; $(am__uninstall_files_from_dir) ID: $(am__tagged_files) $(am__define_uniq_tagged_files); mkid -fID $$unique tags: tags-am TAGS: tags tags-am: $(TAGS_DEPENDENCIES) $(am__tagged_files) set x; \ here=`pwd`; \ $(am__define_uniq_tagged_files); \ shift; \ if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ test -n "$$unique" || unique=$$empty_fix; \ if test $$# -gt 0; then \ $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ "$$@" $$unique; \ else \ $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ $$unique; \ fi; \ fi ctags: ctags-am CTAGS: ctags ctags-am: $(TAGS_DEPENDENCIES) $(am__tagged_files) $(am__define_uniq_tagged_files); \ test -z "$(CTAGS_ARGS)$$unique" \ || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ $$unique GTAGS: here=`$(am__cd) $(top_builddir) && pwd` \ && $(am__cd) $(top_srcdir) \ && gtags -i $(GTAGS_ARGS) "$$here" cscopelist: cscopelist-am cscopelist-am: $(am__tagged_files) list='$(am__tagged_files)'; \ case "$(srcdir)" in \ [\\/]* | ?:[\\/]*) sdir="$(srcdir)" ;; \ *) sdir=$(subdir)/$(srcdir) ;; \ esac; \ for i in $$list; do \ if test -f "$$i"; then \ echo "$(subdir)/$$i"; \ else \ echo "$$sdir/$$i"; \ fi; \ done >> $(top_builddir)/cscope.files distclean-tags: -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags distdir: $(DISTFILES) @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ list='$(DISTFILES)'; \ dist_files=`for file in $$list; do echo $$file; done | \ sed -e "s|^$$srcdirstrip/||;t" \ -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ case $$dist_files in \ */*) $(MKDIR_P) `echo "$$dist_files" | \ sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ sort -u` ;; \ esac; \ for file in $$dist_files; do \ if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ if test -d $$d/$$file; then \ dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ if test -d "$(distdir)/$$file"; then \ find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ fi; \ if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ fi; \ cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ else \ test -f "$(distdir)/$$file" \ || cp -p $$d/$$file "$(distdir)/$$file" \ || exit 1; \ fi; \ done check-am: all-am check: $(BUILT_SOURCES) $(MAKE) $(AM_MAKEFLAGS) check-am all-am: Makefile $(LTLIBRARIES) $(PROGRAMS) $(HEADERS) install-binPROGRAMS: install-libLTLIBRARIES installdirs: for dir in "$(DESTDIR)$(libdir)" "$(DESTDIR)$(bindir)" "$(DESTDIR)$(includedir)"; do \ test -z "$$dir" || $(MKDIR_P) "$$dir"; \ done install: $(BUILT_SOURCES) $(MAKE) $(AM_MAKEFLAGS) install-am install-exec: install-exec-am install-data: install-data-am uninstall: uninstall-am install-am: all-am @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am installcheck: installcheck-am install-strip: if test -z '$(STRIP)'; then \ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ install; \ else \ $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ fi mostlyclean-generic: clean-generic: -test -z "$(CLEANFILES)" || rm -f $(CLEANFILES) distclean-generic: -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) maintainer-clean-generic: @echo "This command is intended for maintainers to use" @echo "it deletes files that may require special tools to rebuild." -test -z "$(BUILT_SOURCES)" || rm -f $(BUILT_SOURCES) clean: clean-am clean-am: clean-binPROGRAMS clean-generic clean-libLTLIBRARIES \ clean-libtool mostlyclean-am distclean: distclean-am -rm -rf $(DEPDIR) ./$(DEPDIR) -rm -f Makefile distclean-am: clean-am distclean-compile distclean-generic \ distclean-tags dvi: dvi-am dvi-am: html: html-am html-am: info: info-am info-am: install-data-am: install-includeHEADERS install-dvi: install-dvi-am install-dvi-am: install-exec-am: install-binPROGRAMS install-libLTLIBRARIES install-html: install-html-am install-html-am: install-info: install-info-am install-info-am: install-man: install-pdf: install-pdf-am install-pdf-am: install-ps: install-ps-am install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-am -rm -rf $(DEPDIR) ./$(DEPDIR) -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic mostlyclean: mostlyclean-am mostlyclean-am: mostlyclean-compile mostlyclean-generic \ mostlyclean-libtool pdf: pdf-am pdf-am: ps: ps-am ps-am: uninstall-am: uninstall-binPROGRAMS uninstall-includeHEADERS \ uninstall-libLTLIBRARIES .MAKE: all check install install-am install-strip .PHONY: CTAGS GTAGS TAGS all all-am check check-am clean \ clean-binPROGRAMS clean-generic clean-libLTLIBRARIES \ clean-libtool cscopelist-am ctags ctags-am distclean \ distclean-compile distclean-generic distclean-libtool \ distclean-tags distdir dvi dvi-am html html-am info info-am \ install install-am install-binPROGRAMS install-data \ install-data-am install-dvi install-dvi-am install-exec \ install-exec-am install-html install-html-am \ install-includeHEADERS install-info install-info-am \ install-libLTLIBRARIES install-man install-pdf install-pdf-am \ install-ps install-ps-am install-strip installcheck \ installcheck-am installdirs maintainer-clean \ maintainer-clean-generic mostlyclean mostlyclean-compile \ mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ tags tags-am uninstall uninstall-am uninstall-binPROGRAMS \ uninstall-includeHEADERS uninstall-libLTLIBRARIES magic.h: ${HDR} sed -e "s/X.YY/$$(echo @VERSION@ | tr -d .)/" < ${HDR} > $@ # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. .NOEXPORT: Index: releng/10.1/contrib/file/src/apprentice.c =================================================================== --- releng/10.1/contrib/file/src/apprentice.c (revision 284192) +++ releng/10.1/contrib/file/src/apprentice.c (revision 284193) @@ -1,3082 +1,3241 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * apprentice - make one pass through /etc/magic, learning its secrets. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: apprentice.c,v 1.211 2014/06/03 19:01:34 christos Exp $") +FILE_RCSID("@(#)$File: apprentice.c,v 1.229 2015/01/01 17:07:34 christos Exp $") #endif /* lint */ #include "magic.h" #include #ifdef HAVE_UNISTD_H #include #endif #ifdef HAVE_STDDEF_H #include #endif #include #include #include #include #ifdef QUICK #include #endif #include #if defined(HAVE_LIMITS_H) #include #endif #ifndef SSIZE_MAX #define MAXMAGIC_SIZE ((ssize_t)0x7fffffff) #else #define MAXMAGIC_SIZE SSIZE_MAX #endif #define EATAB {while (isascii((unsigned char) *l) && \ isspace((unsigned char) *l)) ++l;} #define LOWCASE(l) (isupper((unsigned char) (l)) ? \ tolower((unsigned char) (l)) : (l)) /* * Work around a bug in headers on Digital Unix. * At least confirmed for: OSF1 V4.0 878 */ #if defined(__osf__) && defined(__DECC) #ifdef MAP_FAILED #undef MAP_FAILED #endif #endif #ifndef MAP_FAILED #define MAP_FAILED (void *) -1 #endif #ifndef MAP_FILE #define MAP_FILE 0 #endif #define ALLOC_CHUNK (size_t)10 #define ALLOC_INCR (size_t)200 +#define MAP_TYPE_MMAP 0 +#define MAP_TYPE_MALLOC 1 +#define MAP_TYPE_USER 2 + struct magic_entry { struct magic *mp; uint32_t cont_count; uint32_t max_count; }; struct magic_entry_set { struct magic_entry *me; uint32_t count; uint32_t max; }; struct magic_map { void *p; size_t len; + int type; struct magic *magic[MAGIC_SETS]; uint32_t nmagic[MAGIC_SETS]; }; int file_formats[FILE_NAMES_SIZE]; const size_t file_nformats = FILE_NAMES_SIZE; const char *file_names[FILE_NAMES_SIZE]; const size_t file_nnames = FILE_NAMES_SIZE; private int getvalue(struct magic_set *ms, struct magic *, const char **, int); private int hextoint(int); private const char *getstr(struct magic_set *, struct magic *, const char *, int); private int parse(struct magic_set *, struct magic_entry *, const char *, size_t, int); private void eatsize(const char **); private int apprentice_1(struct magic_set *, const char *, int); private size_t apprentice_magic_strength(const struct magic *); private int apprentice_sort(const void *, const void *); private void apprentice_list(struct mlist *, int ); private struct magic_map *apprentice_load(struct magic_set *, const char *, int); private struct mlist *mlist_alloc(void); private void mlist_free(struct mlist *); private void byteswap(struct magic *, uint32_t); private void bs1(struct magic *); private uint16_t swap2(uint16_t); private uint32_t swap4(uint32_t); private uint64_t swap8(uint64_t); private char *mkdbname(struct magic_set *, const char *, int); +private struct magic_map *apprentice_buf(struct magic_set *, struct magic *, + size_t); private struct magic_map *apprentice_map(struct magic_set *, const char *); +private int check_buffer(struct magic_set *, struct magic_map *, const char *); private void apprentice_unmap(struct magic_map *); private int apprentice_compile(struct magic_set *, struct magic_map *, const char *); private int check_format_type(const char *, int); private int check_format(struct magic_set *, struct magic *); private int get_op(char); private int parse_mime(struct magic_set *, struct magic_entry *, const char *); private int parse_strength(struct magic_set *, struct magic_entry *, const char *); private int parse_apple(struct magic_set *, struct magic_entry *, const char *); private size_t magicsize = sizeof(struct magic); private const char usg_hdr[] = "cont\toffset\ttype\topcode\tmask\tvalue\tdesc"; private struct { const char *name; size_t len; int (*fun)(struct magic_set *, struct magic_entry *, const char *); } bang[] = { #define DECLARE_FIELD(name) { # name, sizeof(# name) - 1, parse_ ## name } DECLARE_FIELD(mime), DECLARE_FIELD(apple), DECLARE_FIELD(strength), #undef DECLARE_FIELD { NULL, 0, NULL } }; #ifdef COMPILE_ONLY int main(int, char *[]); int main(int argc, char *argv[]) { int ret; struct magic_set *ms; char *progname; if ((progname = strrchr(argv[0], '/')) != NULL) progname++; else progname = argv[0]; if (argc != 2) { (void)fprintf(stderr, "Usage: %s file\n", progname); return 1; } if ((ms = magic_open(MAGIC_CHECK)) == NULL) { (void)fprintf(stderr, "%s: %s\n", progname, strerror(errno)); return 1; } ret = magic_compile(ms, argv[1]) == -1 ? 1 : 0; if (ret == 1) (void)fprintf(stderr, "%s: %s\n", progname, magic_error(ms)); magic_close(ms); return ret; } #endif /* COMPILE_ONLY */ struct type_tbl_s { const char name[16]; const size_t len; const int type; const int format; }; /* * XXX - the actual Single UNIX Specification says that "long" means "long", * as in the C data type, but we treat it as meaning "4-byte integer". * Given that the OS X version of file 5.04 did the same, I guess that passes * the actual test; having "long" be dependent on how big a "long" is on * the machine running "file" is silly. */ static const struct type_tbl_s type_tbl[] = { # define XX(s) s, (sizeof(s) - 1) # define XX_NULL "", 0 { XX("invalid"), FILE_INVALID, FILE_FMT_NONE }, { XX("byte"), FILE_BYTE, FILE_FMT_NUM }, { XX("short"), FILE_SHORT, FILE_FMT_NUM }, { XX("default"), FILE_DEFAULT, FILE_FMT_NONE }, { XX("long"), FILE_LONG, FILE_FMT_NUM }, { XX("string"), FILE_STRING, FILE_FMT_STR }, { XX("date"), FILE_DATE, FILE_FMT_STR }, { XX("beshort"), FILE_BESHORT, FILE_FMT_NUM }, { XX("belong"), FILE_BELONG, FILE_FMT_NUM }, { XX("bedate"), FILE_BEDATE, FILE_FMT_STR }, { XX("leshort"), FILE_LESHORT, FILE_FMT_NUM }, { XX("lelong"), FILE_LELONG, FILE_FMT_NUM }, { XX("ledate"), FILE_LEDATE, FILE_FMT_STR }, { XX("pstring"), FILE_PSTRING, FILE_FMT_STR }, { XX("ldate"), FILE_LDATE, FILE_FMT_STR }, { XX("beldate"), FILE_BELDATE, FILE_FMT_STR }, { XX("leldate"), FILE_LELDATE, FILE_FMT_STR }, { XX("regex"), FILE_REGEX, FILE_FMT_STR }, { XX("bestring16"), FILE_BESTRING16, FILE_FMT_STR }, { XX("lestring16"), FILE_LESTRING16, FILE_FMT_STR }, { XX("search"), FILE_SEARCH, FILE_FMT_STR }, { XX("medate"), FILE_MEDATE, FILE_FMT_STR }, { XX("meldate"), FILE_MELDATE, FILE_FMT_STR }, { XX("melong"), FILE_MELONG, FILE_FMT_NUM }, { XX("quad"), FILE_QUAD, FILE_FMT_QUAD }, { XX("lequad"), FILE_LEQUAD, FILE_FMT_QUAD }, { XX("bequad"), FILE_BEQUAD, FILE_FMT_QUAD }, { XX("qdate"), FILE_QDATE, FILE_FMT_STR }, { XX("leqdate"), FILE_LEQDATE, FILE_FMT_STR }, { XX("beqdate"), FILE_BEQDATE, FILE_FMT_STR }, { XX("qldate"), FILE_QLDATE, FILE_FMT_STR }, { XX("leqldate"), FILE_LEQLDATE, FILE_FMT_STR }, { XX("beqldate"), FILE_BEQLDATE, FILE_FMT_STR }, { XX("float"), FILE_FLOAT, FILE_FMT_FLOAT }, { XX("befloat"), FILE_BEFLOAT, FILE_FMT_FLOAT }, { XX("lefloat"), FILE_LEFLOAT, FILE_FMT_FLOAT }, { XX("double"), FILE_DOUBLE, FILE_FMT_DOUBLE }, { XX("bedouble"), FILE_BEDOUBLE, FILE_FMT_DOUBLE }, { XX("ledouble"), FILE_LEDOUBLE, FILE_FMT_DOUBLE }, { XX("leid3"), FILE_LEID3, FILE_FMT_NUM }, { XX("beid3"), FILE_BEID3, FILE_FMT_NUM }, { XX("indirect"), FILE_INDIRECT, FILE_FMT_NUM }, { XX("qwdate"), FILE_QWDATE, FILE_FMT_STR }, { XX("leqwdate"), FILE_LEQWDATE, FILE_FMT_STR }, { XX("beqwdate"), FILE_BEQWDATE, FILE_FMT_STR }, { XX("name"), FILE_NAME, FILE_FMT_NONE }, { XX("use"), FILE_USE, FILE_FMT_NONE }, { XX("clear"), FILE_CLEAR, FILE_FMT_NONE }, { XX_NULL, FILE_INVALID, FILE_FMT_NONE }, }; /* * These are not types, and cannot be preceded by "u" to make them * unsigned. */ static const struct type_tbl_s special_tbl[] = { { XX("name"), FILE_NAME, FILE_FMT_STR }, { XX("use"), FILE_USE, FILE_FMT_STR }, { XX_NULL, FILE_INVALID, FILE_FMT_NONE }, }; # undef XX # undef XX_NULL private int get_type(const struct type_tbl_s *tbl, const char *l, const char **t) { const struct type_tbl_s *p; for (p = tbl; p->len; p++) { if (strncmp(l, p->name, p->len) == 0) { if (t) *t = l + p->len; break; } } return p->type; } private int get_standard_integer_type(const char *l, const char **t) { int type; if (isalpha((unsigned char)l[1])) { switch (l[1]) { case 'C': /* "dC" and "uC" */ type = FILE_BYTE; break; case 'S': /* "dS" and "uS" */ type = FILE_SHORT; break; case 'I': case 'L': /* * "dI", "dL", "uI", and "uL". * * XXX - the actual Single UNIX Specification says * that "L" means "long", as in the C data type, * but we treat it as meaning "4-byte integer". * Given that the OS X version of file 5.04 did * the same, I guess that passes the actual SUS * validation suite; having "dL" be dependent on * how big a "long" is on the machine running * "file" is silly. */ type = FILE_LONG; break; case 'Q': /* "dQ" and "uQ" */ type = FILE_QUAD; break; default: /* "d{anything else}", "u{anything else}" */ return FILE_INVALID; } l += 2; } else if (isdigit((unsigned char)l[1])) { /* * "d{num}" and "u{num}"; we only support {num} values * of 1, 2, 4, and 8 - the Single UNIX Specification * doesn't say anything about whether arbitrary * values should be supported, but both the Solaris 10 * and OS X Mountain Lion versions of file passed the * Single UNIX Specification validation suite, and * neither of them support values bigger than 8 or * non-power-of-2 values. */ if (isdigit((unsigned char)l[2])) { /* Multi-digit, so > 9 */ return FILE_INVALID; } switch (l[1]) { case '1': type = FILE_BYTE; break; case '2': type = FILE_SHORT; break; case '4': type = FILE_LONG; break; case '8': type = FILE_QUAD; break; default: /* XXX - what about 3, 5, 6, or 7? */ return FILE_INVALID; } l += 2; } else { /* * "d" or "u" by itself. */ type = FILE_LONG; ++l; } if (t) *t = l; return type; } private void init_file_tables(void) { static int done = 0; const struct type_tbl_s *p; if (done) return; done++; for (p = type_tbl; p->len; p++) { assert(p->type < FILE_NAMES_SIZE); file_names[p->type] = p->name; file_formats[p->type] = p->format; } assert(p - type_tbl == FILE_NAMES_SIZE); } private int add_mlist(struct mlist *mlp, struct magic_map *map, size_t idx) { struct mlist *ml; + mlp->map = idx == 0 ? map : NULL; if ((ml = CAST(struct mlist *, malloc(sizeof(*ml)))) == NULL) return -1; - ml->map = idx == 0 ? map : NULL; + ml->map = NULL; ml->magic = map->magic[idx]; ml->nmagic = map->nmagic[idx]; mlp->prev->next = ml; ml->prev = mlp->prev; ml->next = mlp; mlp->prev = ml; return 0; } /* * Handle one file or directory. */ private int apprentice_1(struct magic_set *ms, const char *fn, int action) { -#ifndef COMPILE_ONLY - struct mlist *ml; -#endif /* COMPILE_ONLY */ struct magic_map *map; #ifndef COMPILE_ONLY + struct mlist *ml; size_t i; -#endif /* COMPILE_ONLY */ +#endif if (magicsize != FILE_MAGICSIZE) { file_error(ms, 0, "magic element size %lu != %lu", (unsigned long)sizeof(*map->magic[0]), (unsigned long)FILE_MAGICSIZE); return -1; } if (action == FILE_COMPILE) { map = apprentice_load(ms, fn, action); if (map == NULL) return -1; return apprentice_compile(ms, map, fn); } #ifndef COMPILE_ONLY map = apprentice_map(ms, fn); if (map == NULL) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "using regular magic file `%s'", fn); map = apprentice_load(ms, fn, action); if (map == NULL) return -1; } for (i = 0; i < MAGIC_SETS; i++) { if (add_mlist(ms->mlist[i], map, i) == -1) { file_oomem(ms, sizeof(*ml)); - apprentice_unmap(map); - return -1; + goto fail; } } if (action == FILE_LIST) { for (i = 0; i < MAGIC_SETS; i++) { - printf("Set %zu:\nBinary patterns:\n", i); + printf("Set %" SIZE_T_FORMAT "u:\nBinary patterns:\n", + i); apprentice_list(ms->mlist[i], BINTEST); printf("Text patterns:\n"); apprentice_list(ms->mlist[i], TEXTTEST); } } -#endif /* COMPILE_ONLY */ - return 0; +fail: + for (i = 0; i < MAGIC_SETS; i++) { + mlist_free(ms->mlist[i]); + ms->mlist[i] = NULL; + } + return -1; +#else + return 0; +#endif /* COMPILE_ONLY */ } protected void file_ms_free(struct magic_set *ms) { size_t i; if (ms == NULL) return; for (i = 0; i < MAGIC_SETS; i++) mlist_free(ms->mlist[i]); free(ms->o.pbuf); free(ms->o.buf); free(ms->c.li); free(ms); } protected struct magic_set * file_ms_alloc(int flags) { struct magic_set *ms; size_t i, len; if ((ms = CAST(struct magic_set *, calloc((size_t)1, sizeof(struct magic_set)))) == NULL) return NULL; if (magic_setflags(ms, flags) == -1) { errno = EINVAL; goto free; } ms->o.buf = ms->o.pbuf = NULL; len = (ms->c.len = 10) * sizeof(*ms->c.li); if ((ms->c.li = CAST(struct level_info *, malloc(len))) == NULL) goto free; ms->event_flags = 0; ms->error = -1; for (i = 0; i < MAGIC_SETS; i++) ms->mlist[i] = NULL; ms->file = "unknown"; ms->line = 0; + ms->indir_max = FILE_INDIR_MAX; + ms->name_max = FILE_NAME_MAX; + ms->elf_shnum_max = FILE_ELF_SHNUM_MAX; + ms->elf_phnum_max = FILE_ELF_PHNUM_MAX; + ms->elf_notes_max = FILE_ELF_NOTES_MAX; return ms; free: free(ms); return NULL; } private void apprentice_unmap(struct magic_map *map) { if (map == NULL) return; - if (map->p != NULL) { + + switch (map->type) { #ifdef QUICK - if (map->len) + case MAP_TYPE_MMAP: + if (map->p) (void)munmap(map->p, map->len); - else + break; #endif + case MAP_TYPE_MALLOC: free(map->p); - } else { - uint32_t j; - for (j = 0; j < MAGIC_SETS; j++) - free(map->magic[j]); + break; + case MAP_TYPE_USER: + break; + default: + abort(); } free(map); } private struct mlist * mlist_alloc(void) { struct mlist *mlist; if ((mlist = CAST(struct mlist *, calloc(1, sizeof(*mlist)))) == NULL) { return NULL; } mlist->next = mlist->prev = mlist; return mlist; } private void mlist_free(struct mlist *mlist) { - struct mlist *ml; + struct mlist *ml, *next; if (mlist == NULL) return; - for (ml = mlist->next; ml != mlist;) { - struct mlist *next = ml->next; + ml = mlist->next; + for (ml = mlist->next; (next = ml->next) != NULL; ml = next) { if (ml->map) apprentice_unmap(ml->map); free(ml); - ml = next; + if (ml == mlist) + break; } - free(ml); } +#ifndef COMPILE_ONLY +/* void **bufs: an array of compiled magic files */ +protected int +buffer_apprentice(struct magic_set *ms, struct magic **bufs, + size_t *sizes, size_t nbufs) +{ + size_t i, j; + struct mlist *ml; + struct magic_map *map; + + if (nbufs == 0) + return -1; + + if (ms->mlist[0] != NULL) + file_reset(ms); + + init_file_tables(); + + for (i = 0; i < MAGIC_SETS; i++) { + mlist_free(ms->mlist[i]); + if ((ms->mlist[i] = mlist_alloc()) == NULL) { + file_oomem(ms, sizeof(*ms->mlist[i])); + goto fail; + } + } + + for (i = 0; i < nbufs; i++) { + map = apprentice_buf(ms, bufs[i], sizes[i]); + if (map == NULL) + goto fail; + + for (j = 0; j < MAGIC_SETS; j++) { + if (add_mlist(ms->mlist[j], map, j) == -1) { + file_oomem(ms, sizeof(*ml)); + goto fail; + } + } + } + + return 0; +fail: + for (i = 0; i < MAGIC_SETS; i++) { + mlist_free(ms->mlist[i]); + ms->mlist[i] = NULL; + } + return -1; +} +#endif + /* const char *fn: list of magic files and directories */ protected int file_apprentice(struct magic_set *ms, const char *fn, int action) { char *p, *mfn; int file_err, errs = -1; size_t i; if (ms->mlist[0] != NULL) file_reset(ms); if ((fn = magic_getpath(fn, action)) == NULL) return -1; init_file_tables(); if ((mfn = strdup(fn)) == NULL) { file_oomem(ms, strlen(fn)); return -1; } for (i = 0; i < MAGIC_SETS; i++) { mlist_free(ms->mlist[i]); if ((ms->mlist[i] = mlist_alloc()) == NULL) { file_oomem(ms, sizeof(*ms->mlist[i])); - if (i != 0) { - --i; - do - mlist_free(ms->mlist[i]); - while (i != 0); + while (i-- > 0) { + mlist_free(ms->mlist[i]); + ms->mlist[i] = NULL; } free(mfn); return -1; } } fn = mfn; while (fn) { p = strchr(fn, PATHSEP); if (p) *p++ = '\0'; if (*fn == '\0') break; file_err = apprentice_1(ms, fn, action); errs = MAX(errs, file_err); fn = p; } free(mfn); if (errs == -1) { for (i = 0; i < MAGIC_SETS; i++) { mlist_free(ms->mlist[i]); ms->mlist[i] = NULL; } file_error(ms, 0, "could not find any valid magic files!"); return -1; } #if 0 /* * Always leave the database loaded */ if (action == FILE_LOAD) return 0; for (i = 0; i < MAGIC_SETS; i++) { mlist_free(ms->mlist[i]); ms->mlist[i] = NULL; } #endif switch (action) { case FILE_LOAD: case FILE_COMPILE: case FILE_CHECK: case FILE_LIST: return 0; default: file_error(ms, 0, "Invalid action %d", action); return -1; } } /* * Compute the real length of a magic expression, for the purposes * of determining how "strong" a magic expression is (approximating * how specific its matches are): * - magic characters count 0 unless escaped. * - [] expressions count 1 * - {} expressions count 0 * - regular characters or escaped magic characters count 1 * - 0 length expressions count as one */ private size_t nonmagic(const char *str) { const char *p; size_t rv = 0; for (p = str; *p; p++) switch (*p) { case '\\': /* Escaped anything counts 1 */ if (!*++p) p--; rv++; continue; case '?': /* Magic characters count 0 */ case '*': case '.': case '+': case '^': case '$': continue; case '[': /* Bracketed expressions count 1 the ']' */ while (*p && *p != ']') p++; p--; continue; case '{': /* Braced expressions count 0 */ while (*p && *p != '}') p++; if (!*p) p--; continue; default: /* Anything else counts 1 */ rv++; continue; } return rv == 0 ? 1 : rv; /* Return at least 1 */ } /* * Get weight of this magic entry, for sorting purposes. */ private size_t apprentice_magic_strength(const struct magic *m) { #define MULT 10 size_t v, val = 2 * MULT; /* baseline strength */ switch (m->type) { case FILE_DEFAULT: /* make sure this sorts last */ if (m->factor_op != FILE_FACTOR_OP_NONE) abort(); return 0; case FILE_BYTE: val += 1 * MULT; break; case FILE_SHORT: case FILE_LESHORT: case FILE_BESHORT: val += 2 * MULT; break; case FILE_LONG: case FILE_LELONG: case FILE_BELONG: case FILE_MELONG: val += 4 * MULT; break; case FILE_PSTRING: case FILE_STRING: val += m->vallen * MULT; break; case FILE_BESTRING16: case FILE_LESTRING16: val += m->vallen * MULT / 2; break; case FILE_SEARCH: val += m->vallen * MAX(MULT / m->vallen, 1); break; case FILE_REGEX: v = nonmagic(m->value.s); val += v * MAX(MULT / v, 1); break; case FILE_DATE: case FILE_LEDATE: case FILE_BEDATE: case FILE_MEDATE: case FILE_LDATE: case FILE_LELDATE: case FILE_BELDATE: case FILE_MELDATE: case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: val += 4 * MULT; break; case FILE_QUAD: case FILE_BEQUAD: case FILE_LEQUAD: case FILE_QDATE: case FILE_LEQDATE: case FILE_BEQDATE: case FILE_QLDATE: case FILE_LEQLDATE: case FILE_BEQLDATE: case FILE_QWDATE: case FILE_LEQWDATE: case FILE_BEQWDATE: case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: val += 8 * MULT; break; case FILE_INDIRECT: case FILE_NAME: case FILE_USE: break; default: (void)fprintf(stderr, "Bad type %d\n", m->type); abort(); } switch (m->reln) { case 'x': /* matches anything penalize */ case '!': /* matches almost anything penalize */ val = 0; break; case '=': /* Exact match, prefer */ val += MULT; break; case '>': case '<': /* comparison match reduce strength */ val -= 2 * MULT; break; case '^': case '&': /* masking bits, we could count them too */ val -= MULT; break; default: (void)fprintf(stderr, "Bad relation %c\n", m->reln); abort(); } if (val == 0) /* ensure we only return 0 for FILE_DEFAULT */ val = 1; switch (m->factor_op) { case FILE_FACTOR_OP_NONE: break; case FILE_FACTOR_OP_PLUS: val += m->factor; break; case FILE_FACTOR_OP_MINUS: val -= m->factor; break; case FILE_FACTOR_OP_TIMES: val *= m->factor; break; case FILE_FACTOR_OP_DIV: val /= m->factor; break; default: abort(); } /* * Magic entries with no description get a bonus because they depend * on subsequent magic entries to print something. */ if (m->desc[0] == '\0') val++; return val; } /* * Sort callback for sorting entries by "strength" (basically length) */ private int apprentice_sort(const void *a, const void *b) { const struct magic_entry *ma = CAST(const struct magic_entry *, a); const struct magic_entry *mb = CAST(const struct magic_entry *, b); size_t sa = apprentice_magic_strength(ma->mp); size_t sb = apprentice_magic_strength(mb->mp); if (sa == sb) return 0; else if (sa > sb) return -1; else return 1; } /* * Shows sorted patterns list in the order which is used for the matching */ private void apprentice_list(struct mlist *mlist, int mode) { uint32_t magindex = 0; struct mlist *ml; for (ml = mlist->next; ml != mlist; ml = ml->next) { for (magindex = 0; magindex < ml->nmagic; magindex++) { struct magic *m = &ml->magic[magindex]; if ((m->flag & mode) != mode) { /* Skip sub-tests */ while (magindex + 1 < ml->nmagic && ml->magic[magindex + 1].cont_level != 0) ++magindex; continue; /* Skip to next top-level test*/ } /* * Try to iterate over the tree until we find item with * description/mimetype. */ while (magindex + 1 < ml->nmagic && ml->magic[magindex + 1].cont_level != 0 && *ml->magic[magindex].desc == '\0' && *ml->magic[magindex].mimetype == '\0') magindex++; printf("Strength = %3" SIZE_T_FORMAT "u : %s [%s]\n", apprentice_magic_strength(m), ml->magic[magindex].desc, ml->magic[magindex].mimetype); } } } private void set_test_type(struct magic *mstart, struct magic *m) { switch (m->type) { case FILE_BYTE: case FILE_SHORT: case FILE_LONG: case FILE_DATE: case FILE_BESHORT: case FILE_BELONG: case FILE_BEDATE: case FILE_LESHORT: case FILE_LELONG: case FILE_LEDATE: case FILE_LDATE: case FILE_BELDATE: case FILE_LELDATE: case FILE_MEDATE: case FILE_MELDATE: case FILE_MELONG: case FILE_QUAD: case FILE_LEQUAD: case FILE_BEQUAD: case FILE_QDATE: case FILE_LEQDATE: case FILE_BEQDATE: case FILE_QLDATE: case FILE_LEQLDATE: case FILE_BEQLDATE: case FILE_QWDATE: case FILE_LEQWDATE: case FILE_BEQWDATE: case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: mstart->flag |= BINTEST; break; case FILE_STRING: case FILE_PSTRING: case FILE_BESTRING16: case FILE_LESTRING16: /* Allow text overrides */ if (mstart->str_flags & STRING_TEXTTEST) mstart->flag |= TEXTTEST; else mstart->flag |= BINTEST; break; case FILE_REGEX: case FILE_SEARCH: /* Check for override */ if (mstart->str_flags & STRING_BINTEST) mstart->flag |= BINTEST; if (mstart->str_flags & STRING_TEXTTEST) mstart->flag |= TEXTTEST; if (mstart->flag & (TEXTTEST|BINTEST)) break; /* binary test if pattern is not text */ if (file_looks_utf8(m->value.us, (size_t)m->vallen, NULL, NULL) <= 0) mstart->flag |= BINTEST; else mstart->flag |= TEXTTEST; break; case FILE_DEFAULT: /* can't deduce anything; we shouldn't see this at the top level anyway */ break; case FILE_INVALID: default: /* invalid search type, but no need to complain here */ break; } } private int addentry(struct magic_set *ms, struct magic_entry *me, struct magic_entry_set *mset) { size_t i = me->mp->type == FILE_NAME ? 1 : 0; if (mset[i].count == mset[i].max) { struct magic_entry *mp; mset[i].max += ALLOC_INCR; if ((mp = CAST(struct magic_entry *, realloc(mset[i].me, sizeof(*mp) * mset[i].max))) == NULL) { file_oomem(ms, sizeof(*mp) * mset[i].max); return -1; } (void)memset(&mp[mset[i].count], 0, sizeof(*mp) * ALLOC_INCR); mset[i].me = mp; } mset[i].me[mset[i].count++] = *me; memset(me, 0, sizeof(*me)); return 0; } /* * Load and parse one file. */ private void load_1(struct magic_set *ms, int action, const char *fn, int *errs, struct magic_entry_set *mset) { size_t lineno = 0, llen = 0; char *line = NULL; ssize_t len; struct magic_entry me; FILE *f = fopen(ms->file = fn, "r"); if (f == NULL) { if (errno != ENOENT) file_error(ms, errno, "cannot read magic file `%s'", fn); (*errs)++; return; } memset(&me, 0, sizeof(me)); /* read and parse this file */ for (ms->line = 1; (len = getline(&line, &llen, f)) != -1; ms->line++) { if (len == 0) /* null line, garbage, etc */ continue; if (line[len - 1] == '\n') { lineno++; line[len - 1] = '\0'; /* delete newline */ } switch (line[0]) { case '\0': /* empty, do not parse */ case '#': /* comment, do not parse */ continue; case '!': if (line[1] == ':') { size_t i; for (i = 0; bang[i].name != NULL; i++) { if ((size_t)(len - 2) > bang[i].len && memcmp(bang[i].name, line + 2, bang[i].len) == 0) break; } if (bang[i].name == NULL) { file_error(ms, 0, "Unknown !: entry `%s'", line); (*errs)++; continue; } if (me.mp == NULL) { file_error(ms, 0, "No current entry for :!%s type", bang[i].name); (*errs)++; continue; } if ((*bang[i].fun)(ms, &me, line + bang[i].len + 2) != 0) { (*errs)++; continue; } continue; } /*FALLTHROUGH*/ default: again: switch (parse(ms, &me, line, lineno, action)) { case 0: continue; case 1: (void)addentry(ms, &me, mset); goto again; default: (*errs)++; break; } } } if (me.mp) (void)addentry(ms, &me, mset); free(line); (void)fclose(f); } /* * parse a file or directory of files * const char *fn: name of magic file or directory */ private int cmpstrp(const void *p1, const void *p2) { return strcmp(*(char *const *)p1, *(char *const *)p2); } private uint32_t set_text_binary(struct magic_set *ms, struct magic_entry *me, uint32_t nme, uint32_t starttest) { static const char text[] = "text"; static const char binary[] = "binary"; static const size_t len = sizeof(text); uint32_t i = starttest; do { set_test_type(me[starttest].mp, me[i].mp); if ((ms->flags & MAGIC_DEBUG) == 0) continue; (void)fprintf(stderr, "%s%s%s: %s\n", me[i].mp->mimetype, me[i].mp->mimetype[0] == '\0' ? "" : "; ", me[i].mp->desc[0] ? me[i].mp->desc : "(no description)", me[i].mp->flag & BINTEST ? binary : text); if (me[i].mp->flag & BINTEST) { char *p = strstr(me[i].mp->desc, text); if (p && (p == me[i].mp->desc || isspace((unsigned char)p[-1])) && (p + len - me[i].mp->desc == MAXstring || (p[len] == '\0' || isspace((unsigned char)p[len])))) (void)fprintf(stderr, "*** Possible " "binary test for text type\n"); } } while (++i < nme && me[i].mp->cont_level != 0); return i; } private void set_last_default(struct magic_set *ms, struct magic_entry *me, uint32_t nme) { uint32_t i; for (i = 0; i < nme; i++) { if (me[i].mp->cont_level == 0 && me[i].mp->type == FILE_DEFAULT) { while (++i < nme) if (me[i].mp->cont_level == 0) break; if (i != nme) { /* XXX - Ugh! */ ms->line = me[i].mp->lineno; file_magwarn(ms, "level 0 \"default\" did not sort last"); } return; } } } private int coalesce_entries(struct magic_set *ms, struct magic_entry *me, uint32_t nme, struct magic **ma, uint32_t *nma) { uint32_t i, mentrycount = 0; size_t slen; for (i = 0; i < nme; i++) mentrycount += me[i].cont_count; slen = sizeof(**ma) * mentrycount; if ((*ma = CAST(struct magic *, malloc(slen))) == NULL) { file_oomem(ms, slen); return -1; } mentrycount = 0; for (i = 0; i < nme; i++) { (void)memcpy(*ma + mentrycount, me[i].mp, me[i].cont_count * sizeof(**ma)); mentrycount += me[i].cont_count; } *nma = mentrycount; return 0; } private void magic_entry_free(struct magic_entry *me, uint32_t nme) { uint32_t i; if (me == NULL) return; for (i = 0; i < nme; i++) free(me[i].mp); free(me); } private struct magic_map * apprentice_load(struct magic_set *ms, const char *fn, int action) { int errs = 0; uint32_t i, j; size_t files = 0, maxfiles = 0; char **filearr = NULL, *mfn; struct stat st; struct magic_map *map; struct magic_entry_set mset[MAGIC_SETS]; DIR *dir; struct dirent *d; memset(mset, 0, sizeof(mset)); ms->flags |= MAGIC_CHECK; /* Enable checks for parsed files */ if ((map = CAST(struct magic_map *, calloc(1, sizeof(*map)))) == NULL) { file_oomem(ms, sizeof(*map)); return NULL; } /* print silly verbose header for USG compat. */ if (action == FILE_CHECK) (void)fprintf(stderr, "%s\n", usg_hdr); /* load directory or file */ if (stat(fn, &st) == 0 && S_ISDIR(st.st_mode)) { dir = opendir(fn); if (!dir) { errs++; goto out; } while ((d = readdir(dir)) != NULL) { if (asprintf(&mfn, "%s/%s", fn, d->d_name) < 0) { file_oomem(ms, strlen(fn) + strlen(d->d_name) + 2); errs++; closedir(dir); goto out; } if (stat(mfn, &st) == -1 || !S_ISREG(st.st_mode)) { free(mfn); continue; } if (files >= maxfiles) { size_t mlen; maxfiles = (maxfiles + 1) * 2; mlen = maxfiles * sizeof(*filearr); if ((filearr = CAST(char **, realloc(filearr, mlen))) == NULL) { file_oomem(ms, mlen); free(mfn); closedir(dir); errs++; goto out; } } filearr[files++] = mfn; } closedir(dir); qsort(filearr, files, sizeof(*filearr), cmpstrp); for (i = 0; i < files; i++) { load_1(ms, action, filearr[i], &errs, mset); free(filearr[i]); } free(filearr); } else load_1(ms, action, fn, &errs, mset); if (errs) goto out; for (j = 0; j < MAGIC_SETS; j++) { /* Set types of tests */ for (i = 0; i < mset[j].count; ) { if (mset[j].me[i].mp->cont_level != 0) { i++; continue; } i = set_text_binary(ms, mset[j].me, mset[j].count, i); } qsort(mset[j].me, mset[j].count, sizeof(*mset[j].me), apprentice_sort); /* * Make sure that any level 0 "default" line is last * (if one exists). */ set_last_default(ms, mset[j].me, mset[j].count); /* coalesce per file arrays into a single one */ if (coalesce_entries(ms, mset[j].me, mset[j].count, &map->magic[j], &map->nmagic[j]) == -1) { errs++; goto out; } } out: for (j = 0; j < MAGIC_SETS; j++) magic_entry_free(mset[j].me, mset[j].count); if (errs) { apprentice_unmap(map); return NULL; } return map; } /* * extend the sign bit if the comparison is to be signed */ protected uint64_t file_signextend(struct magic_set *ms, struct magic *m, uint64_t v) { if (!(m->flag & UNSIGNED)) { switch(m->type) { /* * Do not remove the casts below. They are * vital. When later compared with the data, * the sign extension must have happened. */ case FILE_BYTE: - v = (char) v; + v = (signed char) v; break; case FILE_SHORT: case FILE_BESHORT: case FILE_LESHORT: v = (short) v; break; case FILE_DATE: case FILE_BEDATE: case FILE_LEDATE: case FILE_MEDATE: case FILE_LDATE: case FILE_BELDATE: case FILE_LELDATE: case FILE_MELDATE: case FILE_LONG: case FILE_BELONG: case FILE_LELONG: case FILE_MELONG: case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: v = (int32_t) v; break; case FILE_QUAD: case FILE_BEQUAD: case FILE_LEQUAD: case FILE_QDATE: case FILE_QLDATE: case FILE_QWDATE: case FILE_BEQDATE: case FILE_BEQLDATE: case FILE_BEQWDATE: case FILE_LEQDATE: case FILE_LEQLDATE: case FILE_LEQWDATE: case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: v = (int64_t) v; break; case FILE_STRING: case FILE_PSTRING: case FILE_BESTRING16: case FILE_LESTRING16: case FILE_REGEX: case FILE_SEARCH: case FILE_DEFAULT: case FILE_INDIRECT: case FILE_NAME: case FILE_USE: case FILE_CLEAR: break; default: if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "cannot happen: m->type=%d\n", m->type); return ~0U; } } return v; } private int string_modifier_check(struct magic_set *ms, struct magic *m) { if ((ms->flags & MAGIC_CHECK) == 0) return 0; if ((m->type != FILE_REGEX || (m->str_flags & REGEX_LINE_COUNT) == 0) && (m->type != FILE_PSTRING && (m->str_flags & PSTRING_LEN) != 0)) { file_magwarn(ms, "'/BHhLl' modifiers are only allowed for pascal strings\n"); return -1; } switch (m->type) { case FILE_BESTRING16: case FILE_LESTRING16: if (m->str_flags != 0) { file_magwarn(ms, "no modifiers allowed for 16-bit strings\n"); return -1; } break; case FILE_STRING: case FILE_PSTRING: if ((m->str_flags & REGEX_OFFSET_START) != 0) { file_magwarn(ms, "'/%c' only allowed on regex and search\n", CHAR_REGEX_OFFSET_START); return -1; } break; case FILE_SEARCH: if (m->str_range == 0) { file_magwarn(ms, "missing range; defaulting to %d\n", STRING_DEFAULT_RANGE); m->str_range = STRING_DEFAULT_RANGE; return -1; } break; case FILE_REGEX: if ((m->str_flags & STRING_COMPACT_WHITESPACE) != 0) { file_magwarn(ms, "'/%c' not allowed on regex\n", CHAR_COMPACT_WHITESPACE); return -1; } if ((m->str_flags & STRING_COMPACT_OPTIONAL_WHITESPACE) != 0) { file_magwarn(ms, "'/%c' not allowed on regex\n", CHAR_COMPACT_OPTIONAL_WHITESPACE); return -1; } break; default: file_magwarn(ms, "coding error: m->type=%d\n", m->type); return -1; } return 0; } private int get_op(char c) { switch (c) { case '&': return FILE_OPAND; case '|': return FILE_OPOR; case '^': return FILE_OPXOR; case '+': return FILE_OPADD; case '-': return FILE_OPMINUS; case '*': return FILE_OPMULTIPLY; case '/': return FILE_OPDIVIDE; case '%': return FILE_OPMODULO; default: return -1; } } #ifdef ENABLE_CONDITIONALS private int get_cond(const char *l, const char **t) { static const struct cond_tbl_s { char name[8]; size_t len; int cond; } cond_tbl[] = { { "if", 2, COND_IF }, { "elif", 4, COND_ELIF }, { "else", 4, COND_ELSE }, { "", 0, COND_NONE }, }; const struct cond_tbl_s *p; for (p = cond_tbl; p->len; p++) { if (strncmp(l, p->name, p->len) == 0 && isspace((unsigned char)l[p->len])) { if (t) *t = l + p->len; break; } } return p->cond; } private int check_cond(struct magic_set *ms, int cond, uint32_t cont_level) { int last_cond; last_cond = ms->c.li[cont_level].last_cond; switch (cond) { case COND_IF: if (last_cond != COND_NONE && last_cond != COND_ELIF) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "syntax error: `if'"); return -1; } last_cond = COND_IF; break; case COND_ELIF: if (last_cond != COND_IF && last_cond != COND_ELIF) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "syntax error: `elif'"); return -1; } last_cond = COND_ELIF; break; case COND_ELSE: if (last_cond != COND_IF && last_cond != COND_ELIF) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "syntax error: `else'"); return -1; } last_cond = COND_NONE; break; case COND_NONE: last_cond = COND_NONE; break; } ms->c.li[cont_level].last_cond = last_cond; return 0; } #endif /* ENABLE_CONDITIONALS */ +private int +parse_indirect_modifier(struct magic_set *ms, struct magic *m, const char **lp) +{ + const char *l = *lp; + + while (!isspace((unsigned char)*++l)) + switch (*l) { + case CHAR_INDIRECT_RELATIVE: + m->str_flags |= INDIRECT_RELATIVE; + break; + default: + if (ms->flags & MAGIC_CHECK) + file_magwarn(ms, "indirect modifier `%c' " + "invalid", *l); + *lp = l; + return -1; + } + *lp = l; + return 0; +} + +private void +parse_op_modifier(struct magic_set *ms, struct magic *m, const char **lp, + int op) +{ + const char *l = *lp; + char *t; + uint64_t val; + + ++l; + m->mask_op |= op; + val = (uint64_t)strtoull(l, &t, 0); + l = t; + m->num_mask = file_signextend(ms, m, val); + eatsize(&l); + *lp = l; +} + +private int +parse_string_modifier(struct magic_set *ms, struct magic *m, const char **lp) +{ + const char *l = *lp; + char *t; + int have_range = 0; + + while (!isspace((unsigned char)*++l)) { + switch (*l) { + case '0': case '1': case '2': + case '3': case '4': case '5': + case '6': case '7': case '8': + case '9': + if (have_range && (ms->flags & MAGIC_CHECK)) + file_magwarn(ms, "multiple ranges"); + have_range = 1; + m->str_range = CAST(uint32_t, strtoul(l, &t, 0)); + if (m->str_range == 0) + file_magwarn(ms, "zero range"); + l = t - 1; + break; + case CHAR_COMPACT_WHITESPACE: + m->str_flags |= STRING_COMPACT_WHITESPACE; + break; + case CHAR_COMPACT_OPTIONAL_WHITESPACE: + m->str_flags |= STRING_COMPACT_OPTIONAL_WHITESPACE; + break; + case CHAR_IGNORE_LOWERCASE: + m->str_flags |= STRING_IGNORE_LOWERCASE; + break; + case CHAR_IGNORE_UPPERCASE: + m->str_flags |= STRING_IGNORE_UPPERCASE; + break; + case CHAR_REGEX_OFFSET_START: + m->str_flags |= REGEX_OFFSET_START; + break; + case CHAR_BINTEST: + m->str_flags |= STRING_BINTEST; + break; + case CHAR_TEXTTEST: + m->str_flags |= STRING_TEXTTEST; + break; + case CHAR_TRIM: + m->str_flags |= STRING_TRIM; + break; + case CHAR_PSTRING_1_LE: +#define SET_LENGTH(a) m->str_flags = (m->str_flags & ~PSTRING_LEN) | (a) + if (m->type != FILE_PSTRING) + goto bad; + SET_LENGTH(PSTRING_1_LE); + break; + case CHAR_PSTRING_2_BE: + if (m->type != FILE_PSTRING) + goto bad; + SET_LENGTH(PSTRING_2_BE); + break; + case CHAR_PSTRING_2_LE: + if (m->type != FILE_PSTRING) + goto bad; + SET_LENGTH(PSTRING_2_LE); + break; + case CHAR_PSTRING_4_BE: + if (m->type != FILE_PSTRING) + goto bad; + SET_LENGTH(PSTRING_4_BE); + break; + case CHAR_PSTRING_4_LE: + switch (m->type) { + case FILE_PSTRING: + case FILE_REGEX: + break; + default: + goto bad; + } + SET_LENGTH(PSTRING_4_LE); + break; + case CHAR_PSTRING_LENGTH_INCLUDES_ITSELF: + if (m->type != FILE_PSTRING) + goto bad; + m->str_flags |= PSTRING_LENGTH_INCLUDES_ITSELF; + break; + default: + bad: + if (ms->flags & MAGIC_CHECK) + file_magwarn(ms, "string modifier `%c' " + "invalid", *l); + goto out; + } + /* allow multiple '/' for readability */ + if (l[1] == '/' && !isspace((unsigned char)l[2])) + l++; + } + if (string_modifier_check(ms, m) == -1) + goto out; + *lp = l; + return 0; +out: + *lp = l; + return -1; +} + /* * parse one line from magic file, put into magic[index++] if valid */ private int parse(struct magic_set *ms, struct magic_entry *me, const char *line, size_t lineno, int action) { #ifdef ENABLE_CONDITIONALS static uint32_t last_cont_level = 0; #endif size_t i; struct magic *m; const char *l = line; char *t; int op; uint32_t cont_level; int32_t diff; cont_level = 0; /* * Parse the offset. */ while (*l == '>') { ++l; /* step over */ cont_level++; } #ifdef ENABLE_CONDITIONALS if (cont_level == 0 || cont_level > last_cont_level) if (file_check_mem(ms, cont_level) == -1) return -1; last_cont_level = cont_level; #endif if (cont_level != 0) { if (me->mp == NULL) { file_magerror(ms, "No current entry for continuation"); return -1; } if (me->cont_count == 0) { file_magerror(ms, "Continuations present with 0 count"); return -1; } m = &me->mp[me->cont_count - 1]; diff = (int32_t)cont_level - (int32_t)m->cont_level; if (diff > 1) file_magwarn(ms, "New continuation level %u is more " "than one larger than current level %u", cont_level, m->cont_level); if (me->cont_count == me->max_count) { struct magic *nm; size_t cnt = me->max_count + ALLOC_CHUNK; if ((nm = CAST(struct magic *, realloc(me->mp, sizeof(*nm) * cnt))) == NULL) { file_oomem(ms, sizeof(*nm) * cnt); return -1; } me->mp = m = nm; me->max_count = CAST(uint32_t, cnt); } m = &me->mp[me->cont_count++]; (void)memset(m, 0, sizeof(*m)); m->cont_level = cont_level; } else { static const size_t len = sizeof(*m) * ALLOC_CHUNK; if (me->mp != NULL) return 1; if ((m = CAST(struct magic *, malloc(len))) == NULL) { file_oomem(ms, len); return -1; } me->mp = m; me->max_count = ALLOC_CHUNK; (void)memset(m, 0, sizeof(*m)); m->factor_op = FILE_FACTOR_OP_NONE; m->cont_level = 0; me->cont_count = 1; } m->lineno = CAST(uint32_t, lineno); if (*l == '&') { /* m->cont_level == 0 checked below. */ ++l; /* step over */ m->flag |= OFFADD; } if (*l == '(') { ++l; /* step over */ m->flag |= INDIR; if (m->flag & OFFADD) m->flag = (m->flag & ~OFFADD) | INDIROFFADD; if (*l == '&') { /* m->cont_level == 0 checked below */ ++l; /* step over */ m->flag |= OFFADD; } } /* Indirect offsets are not valid at level 0. */ if (m->cont_level == 0 && (m->flag & (OFFADD | INDIROFFADD))) if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "relative offset at level 0"); /* get offset, then skip over it */ m->offset = (uint32_t)strtoul(l, &t, 0); if (l == t) if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "offset `%s' invalid", l); l = t; if (m->flag & INDIR) { m->in_type = FILE_LONG; m->in_offset = 0; /* * read [.lbs][+-]nnnnn) */ if (*l == '.') { l++; switch (*l) { case 'l': m->in_type = FILE_LELONG; break; case 'L': m->in_type = FILE_BELONG; break; case 'm': m->in_type = FILE_MELONG; break; case 'h': case 's': m->in_type = FILE_LESHORT; break; case 'H': case 'S': m->in_type = FILE_BESHORT; break; case 'c': case 'b': case 'C': case 'B': m->in_type = FILE_BYTE; break; case 'e': case 'f': case 'g': m->in_type = FILE_LEDOUBLE; break; case 'E': case 'F': case 'G': m->in_type = FILE_BEDOUBLE; break; case 'i': m->in_type = FILE_LEID3; break; case 'I': m->in_type = FILE_BEID3; break; default: if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "indirect offset type `%c' invalid", *l); break; } l++; } m->in_op = 0; if (*l == '~') { m->in_op |= FILE_OPINVERSE; l++; } if ((op = get_op(*l)) != -1) { m->in_op |= op; l++; } if (*l == '(') { m->in_op |= FILE_OPINDIRECT; l++; } if (isdigit((unsigned char)*l) || *l == '-') { m->in_offset = (int32_t)strtol(l, &t, 0); if (l == t) if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "in_offset `%s' invalid", l); l = t; } if (*l++ != ')' || ((m->in_op & FILE_OPINDIRECT) && *l++ != ')')) if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "missing ')' in indirect offset"); } EATAB; #ifdef ENABLE_CONDITIONALS m->cond = get_cond(l, &l); if (check_cond(ms, m->cond, cont_level) == -1) return -1; EATAB; #endif /* * Parse the type. */ if (*l == 'u') { /* * Try it as a keyword type prefixed by "u"; match what * follows the "u". If that fails, try it as an SUS * integer type. */ m->type = get_type(type_tbl, l + 1, &l); if (m->type == FILE_INVALID) { /* * Not a keyword type; parse it as an SUS type, * 'u' possibly followed by a number or C/S/L. */ m->type = get_standard_integer_type(l, &l); } /* It's unsigned. */ if (m->type != FILE_INVALID) m->flag |= UNSIGNED; } else { /* * Try it as a keyword type. If that fails, try it as * an SUS integer type if it begins with "d" or as an * SUS string type if it begins with "s". In any case, * it's not unsigned. */ m->type = get_type(type_tbl, l, &l); if (m->type == FILE_INVALID) { /* * Not a keyword type; parse it as an SUS type, * either 'd' possibly followed by a number or * C/S/L, or just 's'. */ if (*l == 'd') m->type = get_standard_integer_type(l, &l); else if (*l == 's' && !isalpha((unsigned char)l[1])) { m->type = FILE_STRING; ++l; } } } if (m->type == FILE_INVALID) { /* Not found - try it as a special keyword. */ m->type = get_type(special_tbl, l, &l); } if (m->type == FILE_INVALID) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "type `%s' invalid", l); return -1; } /* New-style anding: "0 byte&0x80 =0x80 dynamically linked" */ /* New and improved: ~ & | ^ + - * / % -- exciting, isn't it? */ m->mask_op = 0; if (*l == '~') { if (!IS_STRING(m->type)) m->mask_op |= FILE_OPINVERSE; else if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "'~' invalid for string types"); ++l; } m->str_range = 0; m->str_flags = m->type == FILE_PSTRING ? PSTRING_1_LE : 0; if ((op = get_op(*l)) != -1) { - if (!IS_STRING(m->type)) { - uint64_t val; - ++l; - m->mask_op |= op; - val = (uint64_t)strtoull(l, &t, 0); - l = t; - m->num_mask = file_signextend(ms, m, val); - eatsize(&l); - } - else if (op == FILE_OPDIVIDE) { - int have_range = 0; - while (!isspace((unsigned char)*++l)) { - switch (*l) { - case '0': case '1': case '2': - case '3': case '4': case '5': - case '6': case '7': case '8': - case '9': - if (have_range && - (ms->flags & MAGIC_CHECK)) - file_magwarn(ms, - "multiple ranges"); - have_range = 1; - m->str_range = CAST(uint32_t, - strtoul(l, &t, 0)); - if (m->str_range == 0) - file_magwarn(ms, - "zero range"); - l = t - 1; - break; - case CHAR_COMPACT_WHITESPACE: - m->str_flags |= - STRING_COMPACT_WHITESPACE; - break; - case CHAR_COMPACT_OPTIONAL_WHITESPACE: - m->str_flags |= - STRING_COMPACT_OPTIONAL_WHITESPACE; - break; - case CHAR_IGNORE_LOWERCASE: - m->str_flags |= STRING_IGNORE_LOWERCASE; - break; - case CHAR_IGNORE_UPPERCASE: - m->str_flags |= STRING_IGNORE_UPPERCASE; - break; - case CHAR_REGEX_OFFSET_START: - m->str_flags |= REGEX_OFFSET_START; - break; - case CHAR_BINTEST: - m->str_flags |= STRING_BINTEST; - break; - case CHAR_TEXTTEST: - m->str_flags |= STRING_TEXTTEST; - break; - case CHAR_TRIM: - m->str_flags |= STRING_TRIM; - break; - case CHAR_PSTRING_1_LE: - if (m->type != FILE_PSTRING) - goto bad; - m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_1_LE; - break; - case CHAR_PSTRING_2_BE: - if (m->type != FILE_PSTRING) - goto bad; - m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_2_BE; - break; - case CHAR_PSTRING_2_LE: - if (m->type != FILE_PSTRING) - goto bad; - m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_2_LE; - break; - case CHAR_PSTRING_4_BE: - if (m->type != FILE_PSTRING) - goto bad; - m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_4_BE; - break; - case CHAR_PSTRING_4_LE: - switch (m->type) { - case FILE_PSTRING: - case FILE_REGEX: - break; - default: - goto bad; - } - m->str_flags = (m->str_flags & ~PSTRING_LEN) | PSTRING_4_LE; - break; - case CHAR_PSTRING_LENGTH_INCLUDES_ITSELF: - if (m->type != FILE_PSTRING) - goto bad; - m->str_flags |= PSTRING_LENGTH_INCLUDES_ITSELF; - break; - default: - bad: - if (ms->flags & MAGIC_CHECK) - file_magwarn(ms, - "string extension `%c' " - "invalid", *l); - return -1; - } - /* allow multiple '/' for readability */ - if (l[1] == '/' && - !isspace((unsigned char)l[2])) - l++; + if (IS_STRING(m->type)) { + int r; + + if (op != FILE_OPDIVIDE) { + if (ms->flags & MAGIC_CHECK) + file_magwarn(ms, + "invalid string/indirect op: " + "`%c'", *t); + return -1; } - if (string_modifier_check(ms, m) == -1) + + if (m->type == FILE_INDIRECT) + r = parse_indirect_modifier(ms, m, &l); + else + r = parse_string_modifier(ms, m, &l); + if (r == -1) return -1; - } - else { - if (ms->flags & MAGIC_CHECK) - file_magwarn(ms, "invalid string op: %c", *t); - return -1; - } + } else + parse_op_modifier(ms, m, &l, op); } + /* * We used to set mask to all 1's here, instead let's just not do * anything if mask = 0 (unless you have a better idea) */ EATAB; switch (*l) { case '>': case '<': m->reln = *l; ++l; if (*l == '=') { if (ms->flags & MAGIC_CHECK) { file_magwarn(ms, "%c= not supported", m->reln); return -1; } ++l; } break; /* Old-style anding: "0 byte &0x80 dynamically linked" */ case '&': case '^': case '=': m->reln = *l; ++l; if (*l == '=') { /* HP compat: ignore &= etc. */ ++l; } break; case '!': m->reln = *l; ++l; break; default: m->reln = '='; /* the default relation */ if (*l == 'x' && ((isascii((unsigned char)l[1]) && isspace((unsigned char)l[1])) || !l[1])) { m->reln = *l; ++l; } break; } /* * Grab the value part, except for an 'x' reln. */ if (m->reln != 'x' && getvalue(ms, m, &l, action)) return -1; /* * TODO finish this macro and start using it! * #define offsetcheck {if (offset > HOWMANY-1) * magwarn("offset too big"); } */ /* * Now get last part - the description */ EATAB; if (l[0] == '\b') { ++l; m->flag |= NOSPACE; } else if ((l[0] == '\\') && (l[1] == 'b')) { ++l; ++l; m->flag |= NOSPACE; } for (i = 0; (m->desc[i++] = *l++) != '\0' && i < sizeof(m->desc); ) continue; if (i == sizeof(m->desc)) { m->desc[sizeof(m->desc) - 1] = '\0'; if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "description `%s' truncated", m->desc); } /* * We only do this check while compiling, or if any of the magic * files were not compiled. */ if (ms->flags & MAGIC_CHECK) { if (check_format(ms, m) == -1) return -1; } #ifndef COMPILE_ONLY if (action == FILE_CHECK) { file_mdump(m); } #endif m->mimetype[0] = '\0'; /* initialise MIME type to none */ return 0; } /* * parse a STRENGTH annotation line from magic file, put into magic[index - 1] * if valid */ private int parse_strength(struct magic_set *ms, struct magic_entry *me, const char *line) { const char *l = line; char *el; unsigned long factor; struct magic *m = &me->mp[0]; if (m->factor_op != FILE_FACTOR_OP_NONE) { file_magwarn(ms, "Current entry already has a strength type: %c %d", m->factor_op, m->factor); return -1; } if (m->type == FILE_NAME) { file_magwarn(ms, "%s: Strength setting is not supported in " "\"name\" magic entries", m->value.s); return -1; } EATAB; switch (*l) { case FILE_FACTOR_OP_NONE: case FILE_FACTOR_OP_PLUS: case FILE_FACTOR_OP_MINUS: case FILE_FACTOR_OP_TIMES: case FILE_FACTOR_OP_DIV: m->factor_op = *l++; break; default: file_magwarn(ms, "Unknown factor op `%c'", *l); return -1; } EATAB; factor = strtoul(l, &el, 0); if (factor > 255) { file_magwarn(ms, "Too large factor `%lu'", factor); goto out; } if (*el && !isspace((unsigned char)*el)) { file_magwarn(ms, "Bad factor `%s'", l); goto out; } m->factor = (uint8_t)factor; if (m->factor == 0 && m->factor_op == FILE_FACTOR_OP_DIV) { file_magwarn(ms, "Cannot have factor op `%c' and factor %u", m->factor_op, m->factor); goto out; } return 0; out: m->factor_op = FILE_FACTOR_OP_NONE; m->factor = 0; return -1; } private int +goodchar(unsigned char x, const char *extra) +{ + return (isascii(x) && isalnum(x)) || strchr(extra, x); +} + +private int parse_extra(struct magic_set *ms, struct magic_entry *me, const char *line, - off_t off, size_t len, const char *name, int nt) + off_t off, size_t len, const char *name, const char *extra, int nt) { size_t i; const char *l = line; struct magic *m = &me->mp[me->cont_count == 0 ? 0 : me->cont_count - 1]; char *buf = (char *)m + off; if (buf[0] != '\0') { len = nt ? strlen(buf) : len; file_magwarn(ms, "Current entry already has a %s type " "`%.*s', new type `%s'", name, (int)len, buf, l); return -1; } if (*m->desc == '\0') { file_magwarn(ms, "Current entry does not yet have a " "description for adding a %s type", name); return -1; } EATAB; - for (i = 0; *l && ((isascii((unsigned char)*l) && - isalnum((unsigned char)*l)) || strchr("-+/.", *l)) && - i < len; buf[i++] = *l++) + for (i = 0; *l && i < len && goodchar(*l, extra); buf[i++] = *l++) continue; if (i == len && *l) { if (nt) buf[len - 1] = '\0'; if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "%s type `%s' truncated %" SIZE_T_FORMAT "u", name, line, i); } else { + if (!isspace((unsigned char)*l) && !goodchar(*l, extra)) + file_magwarn(ms, "%s type `%s' has bad char '%c'", + name, line, *l); if (nt) buf[i] = '\0'; } if (i > 0) return 0; - else - return -1; + + file_magerror(ms, "Bad magic entry '%s'", line); + return -1; } /* * Parse an Apple CREATOR/TYPE annotation from magic file and put it into * magic[index - 1] */ private int parse_apple(struct magic_set *ms, struct magic_entry *me, const char *line) { struct magic *m = &me->mp[0]; return parse_extra(ms, me, line, offsetof(struct magic, apple), - sizeof(m->apple), "APPLE", 0); + sizeof(m->apple), "APPLE", "!+-./", 0); } /* * parse a MIME annotation line from magic file, put into magic[index - 1] * if valid */ private int parse_mime(struct magic_set *ms, struct magic_entry *me, const char *line) { struct magic *m = &me->mp[0]; return parse_extra(ms, me, line, offsetof(struct magic, mimetype), - sizeof(m->mimetype), "MIME", 1); + sizeof(m->mimetype), "MIME", "+-/.", 1); } private int check_format_type(const char *ptr, int type) { int quad = 0, h; if (*ptr == '\0') { /* Missing format string; bad */ return -1; } switch (file_formats[type]) { case FILE_FMT_QUAD: quad = 1; /*FALLTHROUGH*/ case FILE_FMT_NUM: if (quad == 0) { switch (type) { case FILE_BYTE: h = 2; break; case FILE_SHORT: case FILE_BESHORT: case FILE_LESHORT: h = 1; break; case FILE_LONG: case FILE_BELONG: case FILE_LELONG: case FILE_MELONG: case FILE_LEID3: case FILE_BEID3: case FILE_INDIRECT: h = 0; break; default: abort(); } } else h = 0; if (*ptr == '-') ptr++; if (*ptr == '.') ptr++; while (isdigit((unsigned char)*ptr)) ptr++; if (*ptr == '.') ptr++; while (isdigit((unsigned char)*ptr)) ptr++; if (quad) { if (*ptr++ != 'l') return -1; if (*ptr++ != 'l') return -1; } switch (*ptr++) { #ifdef STRICT_FORMAT /* "long" formats are int formats for us */ /* so don't accept the 'l' modifier */ case 'l': switch (*ptr++) { case 'i': case 'd': case 'u': case 'o': case 'x': case 'X': return h != 0 ? -1 : 0; default: return -1; } /* * Don't accept h and hh modifiers. They make writing * magic entries more complicated, for very little benefit */ case 'h': if (h-- <= 0) return -1; switch (*ptr++) { case 'h': if (h-- <= 0) return -1; switch (*ptr++) { case 'i': case 'd': case 'u': case 'o': case 'x': case 'X': return 0; default: return -1; } case 'i': case 'd': case 'u': case 'o': case 'x': case 'X': return h != 0 ? -1 : 0; default: return -1; } #endif case 'c': return h != 2 ? -1 : 0; case 'i': case 'd': case 'u': case 'o': case 'x': case 'X': #ifdef STRICT_FORMAT return h != 0 ? -1 : 0; #else return 0; #endif default: return -1; } case FILE_FMT_FLOAT: case FILE_FMT_DOUBLE: if (*ptr == '-') ptr++; if (*ptr == '.') ptr++; while (isdigit((unsigned char)*ptr)) ptr++; if (*ptr == '.') ptr++; while (isdigit((unsigned char)*ptr)) ptr++; switch (*ptr++) { case 'e': case 'E': case 'f': case 'F': case 'g': case 'G': return 0; default: return -1; } case FILE_FMT_STR: if (*ptr == '-') ptr++; while (isdigit((unsigned char )*ptr)) ptr++; if (*ptr == '.') { ptr++; while (isdigit((unsigned char )*ptr)) ptr++; } switch (*ptr++) { case 's': return 0; default: return -1; } default: /* internal error */ abort(); } /*NOTREACHED*/ return -1; } /* * Check that the optional printf format in description matches * the type of the magic. */ private int check_format(struct magic_set *ms, struct magic *m) { char *ptr; for (ptr = m->desc; *ptr; ptr++) if (*ptr == '%') break; if (*ptr == '\0') { /* No format string; ok */ return 1; } assert(file_nformats == file_nnames); if (m->type >= file_nformats) { file_magwarn(ms, "Internal error inconsistency between " "m->type and format strings"); return -1; } if (file_formats[m->type] == FILE_FMT_NONE) { file_magwarn(ms, "No format string for `%s' with description " "`%s'", m->desc, file_names[m->type]); return -1; } ptr++; if (check_format_type(ptr, m->type) == -1) { /* * TODO: this error message is unhelpful if the format * string is not one character long */ file_magwarn(ms, "Printf format `%c' is not valid for type " "`%s' in description `%s'", *ptr ? *ptr : '?', file_names[m->type], m->desc); return -1; } for (; *ptr; ptr++) { if (*ptr == '%') { file_magwarn(ms, "Too many format strings (should have at most one) " "for `%s' with description `%s'", file_names[m->type], m->desc); return -1; } } return 0; } /* * Read a numeric value from a pointer, into the value union of a magic * pointer, according to the magic type. Update the string pointer to point * just after the number read. Return 0 for success, non-zero for failure. */ private int getvalue(struct magic_set *ms, struct magic *m, const char **p, int action) { switch (m->type) { case FILE_BESTRING16: case FILE_LESTRING16: case FILE_STRING: case FILE_PSTRING: case FILE_REGEX: case FILE_SEARCH: case FILE_NAME: case FILE_USE: *p = getstr(ms, m, *p, action == FILE_COMPILE); if (*p == NULL) { if (ms->flags & MAGIC_CHECK) file_magwarn(ms, "cannot get string from `%s'", m->value.s); return -1; } if (m->type == FILE_REGEX) { file_regex_t rx; int rc = file_regcomp(&rx, m->value.s, REG_EXTENDED); if (rc) { if (ms->flags & MAGIC_CHECK) file_regerror(&rx, rc, ms); } file_regfree(&rx); return rc ? -1 : 0; } return 0; case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: if (m->reln != 'x') { char *ep; #ifdef HAVE_STRTOF m->value.f = strtof(*p, &ep); #else m->value.f = (float)strtod(*p, &ep); #endif *p = ep; } return 0; case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: if (m->reln != 'x') { char *ep; m->value.d = strtod(*p, &ep); *p = ep; } return 0; default: if (m->reln != 'x') { char *ep; m->value.q = file_signextend(ms, m, (uint64_t)strtoull(*p, &ep, 0)); *p = ep; eatsize(p); } return 0; } } /* * Convert a string containing C character escapes. Stop at an unescaped * space or tab. * Copy the converted version to "m->value.s", and the length in m->vallen. * Return updated scan pointer as function result. Warn if set. */ private const char * getstr(struct magic_set *ms, struct magic *m, const char *s, int warn) { const char *origs = s; char *p = m->value.s; size_t plen = sizeof(m->value.s); char *origp = p; char *pmax = p + plen - 1; int c; int val; while ((c = *s++) != '\0') { if (isspace((unsigned char) c)) break; if (p >= pmax) { file_error(ms, 0, "string too long: `%s'", origs); return NULL; } if (c == '\\') { switch(c = *s++) { case '\0': if (warn) file_magwarn(ms, "incomplete escape"); goto out; case '\t': if (warn) { file_magwarn(ms, "escaped tab found, use \\t instead"); warn = 0; /* already did */ } /*FALLTHROUGH*/ default: if (warn) { if (isprint((unsigned char)c)) { /* Allow escaping of * ``relations'' */ if (strchr("<>&^=!", c) == NULL && (m->type != FILE_REGEX || strchr("[]().*?^$|{}", c) == NULL)) { file_magwarn(ms, "no " "need to escape " "`%c'", c); } } else { file_magwarn(ms, "unknown escape sequence: " "\\%03o", c); } } /*FALLTHROUGH*/ /* space, perhaps force people to use \040? */ case ' ': #if 0 /* * Other things people escape, but shouldn't need to, * so we disallow them */ case '\'': case '"': case '?': #endif /* Relations */ case '>': case '<': case '&': case '^': case '=': case '!': /* and baskslash itself */ case '\\': *p++ = (char) c; break; case 'a': *p++ = '\a'; break; case 'b': *p++ = '\b'; break; case 'f': *p++ = '\f'; break; case 'n': *p++ = '\n'; break; case 'r': *p++ = '\r'; break; case 't': *p++ = '\t'; break; case 'v': *p++ = '\v'; break; /* \ and up to 3 octal digits */ case '0': case '1': case '2': case '3': case '4': case '5': case '6': case '7': val = c - '0'; c = *s++; /* try for 2 */ if (c >= '0' && c <= '7') { val = (val << 3) | (c - '0'); c = *s++; /* try for 3 */ if (c >= '0' && c <= '7') val = (val << 3) | (c-'0'); else --s; } else --s; *p++ = (char)val; break; /* \x and up to 2 hex digits */ case 'x': val = 'x'; /* Default if no digits */ c = hextoint(*s++); /* Get next char */ if (c >= 0) { val = c; c = hextoint(*s++); if (c >= 0) val = (val << 4) + c; else --s; } else --s; *p++ = (char)val; break; } } else *p++ = (char)c; } out: *p = '\0'; m->vallen = CAST(unsigned char, (p - origp)); if (m->type == FILE_PSTRING) m->vallen += (unsigned char)file_pstring_length_size(m); return s; } /* Single hex char to int; -1 if not a hex char. */ private int hextoint(int c) { if (!isascii((unsigned char) c)) return -1; if (isdigit((unsigned char) c)) return c - '0'; if ((c >= 'a') && (c <= 'f')) return c + 10 - 'a'; if (( c>= 'A') && (c <= 'F')) return c + 10 - 'A'; return -1; } /* * Print a string containing C character escapes. */ protected void file_showstr(FILE *fp, const char *s, size_t len) { char c; for (;;) { if (len == ~0U) { c = *s++; if (c == '\0') break; } else { if (len-- == 0) break; c = *s++; } if (c >= 040 && c <= 0176) /* TODO isprint && !iscntrl */ (void) fputc(c, fp); else { (void) fputc('\\', fp); switch (c) { case '\a': (void) fputc('a', fp); break; case '\b': (void) fputc('b', fp); break; case '\f': (void) fputc('f', fp); break; case '\n': (void) fputc('n', fp); break; case '\r': (void) fputc('r', fp); break; case '\t': (void) fputc('t', fp); break; case '\v': (void) fputc('v', fp); break; default: (void) fprintf(fp, "%.3o", c & 0377); break; } } } } /* * eatsize(): Eat the size spec from a number [eg. 10UL] */ private void eatsize(const char **p) { const char *l = *p; if (LOWCASE(*l) == 'u') l++; switch (LOWCASE(*l)) { case 'l': /* long */ case 's': /* short */ case 'h': /* short */ case 'b': /* char/byte */ case 'c': /* char/byte */ l++; /*FALLTHROUGH*/ default: break; } *p = l; } /* + * handle a buffer containing a compiled file. + */ +private struct magic_map * +apprentice_buf(struct magic_set *ms, struct magic *buf, size_t len) +{ + struct magic_map *map; + + if ((map = CAST(struct magic_map *, calloc(1, sizeof(*map)))) == NULL) { + file_oomem(ms, sizeof(*map)); + return NULL; + } + map->len = len; + map->p = buf; + map->type = MAP_TYPE_USER; + if (check_buffer(ms, map, "buffer") != 0) { + apprentice_unmap(map); + return NULL; + } + return map; +} + +/* * handle a compiled file. */ private struct magic_map * apprentice_map(struct magic_set *ms, const char *fn) { int fd; struct stat st; - uint32_t *ptr; - uint32_t version, entries, nentries; - int needsbyteswap; char *dbname = NULL; struct magic_map *map; - size_t i; fd = -1; if ((map = CAST(struct magic_map *, calloc(1, sizeof(*map)))) == NULL) { file_oomem(ms, sizeof(*map)); goto error; } dbname = mkdbname(ms, fn, 0); if (dbname == NULL) goto error; if ((fd = open(dbname, O_RDONLY|O_BINARY)) == -1) goto error; if (fstat(fd, &st) == -1) { file_error(ms, errno, "cannot stat `%s'", dbname); goto error; } if (st.st_size < 8 || st.st_size > MAXMAGIC_SIZE) { file_error(ms, 0, "file `%s' is too %s", dbname, st.st_size < 8 ? "small" : "large"); goto error; } map->len = (size_t)st.st_size; #ifdef QUICK if ((map->p = mmap(0, (size_t)st.st_size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FILE, fd, (off_t)0)) == MAP_FAILED) { file_error(ms, errno, "cannot map `%s'", dbname); goto error; } + map->type = MAP_TYPE_MMAP; #else if ((map->p = CAST(void *, malloc(map->len))) == NULL) { file_oomem(ms, map->len); goto error; } if (read(fd, map->p, map->len) != (ssize_t)map->len) { file_badread(ms); goto error; } - map->len = 0; + map->type = MAP_TYPE_MALLOC; #define RET 1 #endif (void)close(fd); fd = -1; + + if (check_buffer(ms, map, dbname) != 0) + goto error; + + free(dbname); + return map; + +error: + if (fd != -1) + (void)close(fd); + apprentice_unmap(map); + free(dbname); + return NULL; +} + +private int +check_buffer(struct magic_set *ms, struct magic_map *map, const char *dbname) +{ + uint32_t *ptr; + uint32_t entries, nentries; + uint32_t version; + int i, needsbyteswap; + ptr = CAST(uint32_t *, map->p); if (*ptr != MAGICNO) { if (swap4(*ptr) != MAGICNO) { file_error(ms, 0, "bad magic in `%s'", dbname); - goto error; + return -1; } needsbyteswap = 1; } else needsbyteswap = 0; if (needsbyteswap) version = swap4(ptr[1]); else version = ptr[1]; if (version != VERSIONNO) { file_error(ms, 0, "File %s supports only version %d magic " "files. `%s' is version %d", VERSION, VERSIONNO, dbname, version); - goto error; + return -1; } - entries = (uint32_t)(st.st_size / sizeof(struct magic)); - if ((off_t)(entries * sizeof(struct magic)) != st.st_size) { - file_error(ms, 0, "Size of `%s' %" INT64_T_FORMAT "u is not " + entries = (uint32_t)(map->len / sizeof(struct magic)); + if ((entries * sizeof(struct magic)) != map->len) { + file_error(ms, 0, "Size of `%s' %" SIZE_T_FORMAT "u is not " "a multiple of %" SIZE_T_FORMAT "u", - dbname, (unsigned long long)st.st_size, - sizeof(struct magic)); - goto error; + dbname, map->len, sizeof(struct magic)); + return -1; } map->magic[0] = CAST(struct magic *, map->p) + 1; nentries = 0; for (i = 0; i < MAGIC_SETS; i++) { if (needsbyteswap) map->nmagic[i] = swap4(ptr[i + 2]); else map->nmagic[i] = ptr[i + 2]; if (i != MAGIC_SETS - 1) map->magic[i + 1] = map->magic[i] + map->nmagic[i]; nentries += map->nmagic[i]; } if (entries != nentries + 1) { file_error(ms, 0, "Inconsistent entries in `%s' %u != %u", dbname, entries, nentries + 1); - goto error; + return -1; } if (needsbyteswap) for (i = 0; i < MAGIC_SETS; i++) byteswap(map->magic[i], map->nmagic[i]); - free(dbname); - return map; - -error: - if (fd != -1) - (void)close(fd); - apprentice_unmap(map); - free(dbname); - return NULL; + return 0; } /* * handle an mmaped file. */ private int apprentice_compile(struct magic_set *ms, struct magic_map *map, const char *fn) { static const size_t nm = sizeof(*map->nmagic) * MAGIC_SETS; static const size_t m = sizeof(**map->magic); int fd = -1; size_t len; char *dbname; int rv = -1; uint32_t i; union { struct magic m; uint32_t h[2 + MAGIC_SETS]; } hdr; dbname = mkdbname(ms, fn, 1); if (dbname == NULL) goto out; if ((fd = open(dbname, O_WRONLY|O_CREAT|O_TRUNC|O_BINARY, 0644)) == -1) { file_error(ms, errno, "cannot open `%s'", dbname); goto out; } memset(&hdr, 0, sizeof(hdr)); hdr.h[0] = MAGICNO; hdr.h[1] = VERSIONNO; memcpy(hdr.h + 2, map->nmagic, nm); if (write(fd, &hdr, sizeof(hdr)) != (ssize_t)sizeof(hdr)) { file_error(ms, errno, "error writing `%s'", dbname); goto out; } for (i = 0; i < MAGIC_SETS; i++) { len = m * map->nmagic[i]; if (write(fd, map->magic[i], len) != (ssize_t)len) { file_error(ms, errno, "error writing `%s'", dbname); goto out; } } if (fd != -1) (void)close(fd); rv = 0; out: free(dbname); return rv; } private const char ext[] = ".mgc"; /* * make a dbname */ private char * mkdbname(struct magic_set *ms, const char *fn, int strip) { const char *p, *q; char *buf; if (strip) { if ((p = strrchr(fn, '/')) != NULL) fn = ++p; } for (q = fn; *q; q++) continue; /* Look for .mgc */ for (p = ext + sizeof(ext) - 1; p >= ext && q >= fn; p--, q--) if (*p != *q) break; /* Did not find .mgc, restore q */ if (p >= ext) while (*q) q++; q++; /* Compatibility with old code that looked in .mime */ if (ms->flags & MAGIC_MIME) { if (asprintf(&buf, "%.*s.mime%s", (int)(q - fn), fn, ext) < 0) return NULL; if (access(buf, R_OK) != -1) { ms->flags &= MAGIC_MIME_TYPE; return buf; } free(buf); } if (asprintf(&buf, "%.*s%s", (int)(q - fn), fn, ext) < 0) return NULL; /* Compatibility with old code that looked in .mime */ if (strstr(p, ".mime") != NULL) ms->flags &= MAGIC_MIME_TYPE; return buf; } /* * Byteswap an mmap'ed file if needed */ private void byteswap(struct magic *magic, uint32_t nmagic) { uint32_t i; for (i = 0; i < nmagic; i++) bs1(&magic[i]); } /* * swap a short */ private uint16_t swap2(uint16_t sv) { uint16_t rv; uint8_t *s = (uint8_t *)(void *)&sv; uint8_t *d = (uint8_t *)(void *)&rv; d[0] = s[1]; d[1] = s[0]; return rv; } /* * swap an int */ private uint32_t swap4(uint32_t sv) { uint32_t rv; uint8_t *s = (uint8_t *)(void *)&sv; uint8_t *d = (uint8_t *)(void *)&rv; d[0] = s[3]; d[1] = s[2]; d[2] = s[1]; d[3] = s[0]; return rv; } /* * swap a quad */ private uint64_t swap8(uint64_t sv) { uint64_t rv; uint8_t *s = (uint8_t *)(void *)&sv; uint8_t *d = (uint8_t *)(void *)&rv; #if 0 d[0] = s[3]; d[1] = s[2]; d[2] = s[1]; d[3] = s[0]; d[4] = s[7]; d[5] = s[6]; d[6] = s[5]; d[7] = s[4]; #else d[0] = s[7]; d[1] = s[6]; d[2] = s[5]; d[3] = s[4]; d[4] = s[3]; d[5] = s[2]; d[6] = s[1]; d[7] = s[0]; #endif return rv; } /* * byteswap a single magic entry */ private void bs1(struct magic *m) { m->cont_level = swap2(m->cont_level); m->offset = swap4((uint32_t)m->offset); m->in_offset = swap4((uint32_t)m->in_offset); m->lineno = swap4((uint32_t)m->lineno); if (IS_STRING(m->type)) { m->str_range = swap4(m->str_range); m->str_flags = swap4(m->str_flags); } else { m->value.q = swap8(m->value.q); m->num_mask = swap8(m->num_mask); } } protected size_t file_pstring_length_size(const struct magic *m) { switch (m->str_flags & PSTRING_LEN) { case PSTRING_1_LE: return 1; case PSTRING_2_LE: case PSTRING_2_BE: return 2; case PSTRING_4_LE: case PSTRING_4_BE: return 4; default: abort(); /* Impossible */ return 1; } } protected size_t file_pstring_get_length(const struct magic *m, const char *s) { size_t len = 0; switch (m->str_flags & PSTRING_LEN) { case PSTRING_1_LE: len = *s; break; case PSTRING_2_LE: len = (s[1] << 8) | s[0]; break; case PSTRING_2_BE: len = (s[0] << 8) | s[1]; break; case PSTRING_4_LE: len = (s[3] << 24) | (s[2] << 16) | (s[1] << 8) | s[0]; break; case PSTRING_4_BE: len = (s[0] << 24) | (s[1] << 16) | (s[2] << 8) | s[3]; break; default: abort(); /* Impossible */ } if (m->str_flags & PSTRING_LENGTH_INCLUDES_ITSELF) len -= file_pstring_length_size(m); return len; } protected int file_magicfind(struct magic_set *ms, const char *name, struct mlist *v) { uint32_t i, j; struct mlist *mlist, *ml; mlist = ms->mlist[1]; for (ml = mlist->next; ml != mlist; ml = ml->next) { struct magic *ma = ml->magic; uint32_t nma = ml->nmagic; for (i = 0; i < nma; i++) { if (ma[i].type != FILE_NAME) continue; if (strcmp(ma[i].value.s, name) == 0) { v->magic = &ma[i]; for (j = i + 1; j < nma; j++) if (ma[j].cont_level == 0) break; v->nmagic = j - i; return 0; } } } return -1; } Index: releng/10.1/contrib/file/src/ascmagic.c =================================================================== --- releng/10.1/contrib/file/src/ascmagic.c (revision 284192) +++ releng/10.1/contrib/file/src/ascmagic.c (revision 284193) @@ -1,360 +1,361 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * ASCII magic -- try to detect text encoding. * * Extensively modified by Eric Fischer in July, 2000, * to handle character codes other than ASCII on a unified basis. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: ascmagic.c,v 1.88 2014/02/12 23:20:53 christos Exp $") +FILE_RCSID("@(#)$File: ascmagic.c,v 1.91 2014/11/28 02:46:39 christos Exp $") #endif /* lint */ #include "magic.h" #include #include #include #include #ifdef HAVE_UNISTD_H #include #endif #define MAXLINELEN 300 /* longest sane line length */ #define ISSPC(x) ((x) == ' ' || (x) == '\t' || (x) == '\r' || (x) == '\n' \ || (x) == 0x85 || (x) == '\f') private unsigned char *encode_utf8(unsigned char *, size_t, unichar *, size_t); private size_t trim_nuls(const unsigned char *, size_t); /* * Undo the NUL-termination kindly provided by process() * but leave at least one byte to look at */ private size_t trim_nuls(const unsigned char *buf, size_t nbytes) { while (nbytes > 1 && buf[nbytes - 1] == '\0') nbytes--; return nbytes; } protected int file_ascmagic(struct magic_set *ms, const unsigned char *buf, size_t nbytes, int text) { unichar *ubuf = NULL; size_t ulen = 0; int rv = 1; const char *code = NULL; const char *code_mime = NULL; const char *type = NULL; if (ms->flags & MAGIC_APPLE) return 0; nbytes = trim_nuls(buf, nbytes); /* If file doesn't look like any sort of text, give up. */ if (file_encoding(ms, buf, nbytes, &ubuf, &ulen, &code, &code_mime, &type) == 0) rv = 0; else rv = file_ascmagic_with_encoding(ms, buf, nbytes, ubuf, ulen, code, type, text); free(ubuf); return rv; } protected int file_ascmagic_with_encoding(struct magic_set *ms, const unsigned char *buf, size_t nbytes, unichar *ubuf, size_t ulen, const char *code, const char *type, int text) { unsigned char *utf8_buf = NULL, *utf8_end; size_t mlen, i; int rv = -1; int mime = ms->flags & MAGIC_MIME; const char *subtype = NULL; const char *subtype_mime = NULL; int has_escapes = 0; int has_backspace = 0; int seen_cr = 0; int n_crlf = 0; int n_lf = 0; int n_cr = 0; int n_nel = 0; int executable = 0; size_t last_line_end = (size_t)-1; int has_long_lines = 0; if (ms->flags & MAGIC_APPLE) return 0; nbytes = trim_nuls(buf, nbytes); /* If we have fewer than 2 bytes, give up. */ if (nbytes <= 1) { rv = 0; goto done; } if (ulen > 0 && (ms->flags & MAGIC_NO_CHECK_SOFT) == 0) { /* Convert ubuf to UTF-8 and try text soft magic */ /* malloc size is a conservative overestimate; could be improved, or at least realloced after conversion. */ mlen = ulen * 6; if ((utf8_buf = CAST(unsigned char *, malloc(mlen))) == NULL) { file_oomem(ms, mlen); goto done; } if ((utf8_end = encode_utf8(utf8_buf, mlen, ubuf, ulen)) == NULL) goto done; if ((rv = file_softmagic(ms, utf8_buf, - (size_t)(utf8_end - utf8_buf), 0, TEXTTEST, text)) == 0) + (size_t)(utf8_end - utf8_buf), 0, NULL, + TEXTTEST, text)) == 0) rv = -1; } /* Now try to discover other details about the file. */ for (i = 0; i < ulen; i++) { if (ubuf[i] == '\n') { if (seen_cr) n_crlf++; else n_lf++; last_line_end = i; } else if (seen_cr) n_cr++; seen_cr = (ubuf[i] == '\r'); if (seen_cr) last_line_end = i; if (ubuf[i] == 0x85) { /* X3.64/ECMA-43 "next line" character */ n_nel++; last_line_end = i; } /* If this line is _longer_ than MAXLINELEN, remember it. */ if (i > last_line_end + MAXLINELEN) has_long_lines = 1; if (ubuf[i] == '\033') has_escapes = 1; if (ubuf[i] == '\b') has_backspace = 1; } /* Beware, if the data has been truncated, the final CR could have been followed by a LF. If we have HOWMANY bytes, it indicates that the data might have been truncated, probably even before this function was called. */ if (seen_cr && nbytes < HOWMANY) n_cr++; if (strcmp(type, "binary") == 0) { rv = 0; goto done; } if (mime) { if (!file_printedlen(ms) && (mime & MAGIC_MIME_TYPE) != 0) { if (subtype_mime) { if (file_printf(ms, "%s", subtype_mime) == -1) goto done; } else { if (file_printf(ms, "text/plain") == -1) goto done; } } } else { if (file_printedlen(ms)) { switch (file_replace(ms, " text$", ", ")) { case 0: switch (file_replace(ms, " text executable$", ", ")) { case 0: if (file_printf(ms, ", ") == -1) goto done; break; case -1: goto done; default: executable = 1; break; } break; case -1: goto done; default: break; } } if (file_printf(ms, "%s", code) == -1) goto done; if (subtype) { if (file_printf(ms, " %s", subtype) == -1) goto done; } if (file_printf(ms, " %s", type) == -1) goto done; if (executable) if (file_printf(ms, " executable") == -1) goto done; if (has_long_lines) if (file_printf(ms, ", with very long lines") == -1) goto done; /* * Only report line terminators if we find one other than LF, * or if we find none at all. */ if ((n_crlf == 0 && n_cr == 0 && n_nel == 0 && n_lf == 0) || (n_crlf != 0 || n_cr != 0 || n_nel != 0)) { if (file_printf(ms, ", with") == -1) goto done; if (n_crlf == 0 && n_cr == 0 && n_nel == 0 && n_lf == 0) { if (file_printf(ms, " no") == -1) goto done; } else { if (n_crlf) { if (file_printf(ms, " CRLF") == -1) goto done; if (n_cr || n_lf || n_nel) if (file_printf(ms, ",") == -1) goto done; } if (n_cr) { if (file_printf(ms, " CR") == -1) goto done; if (n_lf || n_nel) if (file_printf(ms, ",") == -1) goto done; } if (n_lf) { if (file_printf(ms, " LF") == -1) goto done; if (n_nel) if (file_printf(ms, ",") == -1) goto done; } if (n_nel) if (file_printf(ms, " NEL") == -1) goto done; } if (file_printf(ms, " line terminators") == -1) goto done; } if (has_escapes) if (file_printf(ms, ", with escape sequences") == -1) goto done; if (has_backspace) if (file_printf(ms, ", with overstriking") == -1) goto done; } rv = 1; done: free(utf8_buf); return rv; } /* * Encode Unicode string as UTF-8, returning pointer to character * after end of string, or NULL if an invalid character is found. */ private unsigned char * encode_utf8(unsigned char *buf, size_t len, unichar *ubuf, size_t ulen) { size_t i; unsigned char *end = buf + len; for (i = 0; i < ulen; i++) { if (ubuf[i] <= 0x7f) { if (end - buf < 1) return NULL; *buf++ = (unsigned char)ubuf[i]; } else if (ubuf[i] <= 0x7ff) { if (end - buf < 2) return NULL; *buf++ = (unsigned char)((ubuf[i] >> 6) + 0xc0); *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); } else if (ubuf[i] <= 0xffff) { if (end - buf < 3) return NULL; *buf++ = (unsigned char)((ubuf[i] >> 12) + 0xe0); *buf++ = (unsigned char)(((ubuf[i] >> 6) & 0x3f) + 0x80); *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); } else if (ubuf[i] <= 0x1fffff) { if (end - buf < 4) return NULL; *buf++ = (unsigned char)((ubuf[i] >> 18) + 0xf0); *buf++ = (unsigned char)(((ubuf[i] >> 12) & 0x3f) + 0x80); *buf++ = (unsigned char)(((ubuf[i] >> 6) & 0x3f) + 0x80); *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); } else if (ubuf[i] <= 0x3ffffff) { if (end - buf < 5) return NULL; *buf++ = (unsigned char)((ubuf[i] >> 24) + 0xf8); *buf++ = (unsigned char)(((ubuf[i] >> 18) & 0x3f) + 0x80); *buf++ = (unsigned char)(((ubuf[i] >> 12) & 0x3f) + 0x80); *buf++ = (unsigned char)(((ubuf[i] >> 6) & 0x3f) + 0x80); *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); } else if (ubuf[i] <= 0x7fffffff) { if (end - buf < 6) return NULL; *buf++ = (unsigned char)((ubuf[i] >> 30) + 0xfc); *buf++ = (unsigned char)(((ubuf[i] >> 24) & 0x3f) + 0x80); *buf++ = (unsigned char)(((ubuf[i] >> 18) & 0x3f) + 0x80); *buf++ = (unsigned char)(((ubuf[i] >> 12) & 0x3f) + 0x80); *buf++ = (unsigned char)(((ubuf[i] >> 6) & 0x3f) + 0x80); *buf++ = (unsigned char)((ubuf[i] & 0x3f) + 0x80); } else /* Invalid character */ return NULL; } return buf; } Index: releng/10.1/contrib/file/src/cdf.c =================================================================== --- releng/10.1/contrib/file/src/cdf.c (revision 284192) +++ releng/10.1/contrib/file/src/cdf.c (revision 284193) @@ -1,1368 +1,1469 @@ /*- * Copyright (c) 2008 Christos Zoulas * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ /* * Parse Composite Document Files, the format used in Microsoft Office * document files before they switched to zipped XML. * Info from: http://sc.openoffice.org/compdocfileformat.pdf * * N.B. This is the "Composite Document File" format, and not the * "Compound Document Format", nor the "Channel Definition Format". */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: cdf.c,v 1.63 2014/06/09 13:04:37 christos Exp $") +FILE_RCSID("@(#)$File: cdf.c,v 1.69 2014/12/04 15:56:46 christos Exp $") #endif #include #ifdef CDF_DEBUG #include #endif #include #include #include #include #include #ifdef HAVE_LIMITS_H #include #endif #ifndef EFTYPE #define EFTYPE EINVAL #endif #include "cdf.h" #ifdef CDF_DEBUG #define DPRINTF(a) printf a, fflush(stdout) #else #define DPRINTF(a) #endif static union { char s[4]; uint32_t u; } cdf_bo; #define NEED_SWAP (cdf_bo.u == (uint32_t)0x01020304) #define CDF_TOLE8(x) ((uint64_t)(NEED_SWAP ? _cdf_tole8(x) : (uint64_t)(x))) #define CDF_TOLE4(x) ((uint32_t)(NEED_SWAP ? _cdf_tole4(x) : (uint32_t)(x))) #define CDF_TOLE2(x) ((uint16_t)(NEED_SWAP ? _cdf_tole2(x) : (uint16_t)(x))) +#define CDF_TOLE(x) (sizeof(x) == 2 ? CDF_TOLE2(x) : (sizeof(x) == 4 ? \ + CDF_TOLE4(x) : CDF_TOLE8(x))) #define CDF_GETUINT32(x, y) cdf_getuint32(x, y) /* * swap a short */ static uint16_t _cdf_tole2(uint16_t sv) { uint16_t rv; uint8_t *s = (uint8_t *)(void *)&sv; uint8_t *d = (uint8_t *)(void *)&rv; d[0] = s[1]; d[1] = s[0]; return rv; } /* * swap an int */ static uint32_t _cdf_tole4(uint32_t sv) { uint32_t rv; uint8_t *s = (uint8_t *)(void *)&sv; uint8_t *d = (uint8_t *)(void *)&rv; d[0] = s[3]; d[1] = s[2]; d[2] = s[1]; d[3] = s[0]; return rv; } /* * swap a quad */ static uint64_t _cdf_tole8(uint64_t sv) { uint64_t rv; uint8_t *s = (uint8_t *)(void *)&sv; uint8_t *d = (uint8_t *)(void *)&rv; d[0] = s[7]; d[1] = s[6]; d[2] = s[5]; d[3] = s[4]; d[4] = s[3]; d[5] = s[2]; d[6] = s[1]; d[7] = s[0]; return rv; } /* * grab a uint32_t from a possibly unaligned address, and return it in * the native host order. */ static uint32_t cdf_getuint32(const uint8_t *p, size_t offs) { uint32_t rv; (void)memcpy(&rv, p + offs * sizeof(uint32_t), sizeof(rv)); return CDF_TOLE4(rv); } #define CDF_UNPACK(a) \ (void)memcpy(&(a), &buf[len], sizeof(a)), len += sizeof(a) #define CDF_UNPACKA(a) \ (void)memcpy((a), &buf[len], sizeof(a)), len += sizeof(a) uint16_t cdf_tole2(uint16_t sv) { return CDF_TOLE2(sv); } uint32_t cdf_tole4(uint32_t sv) { return CDF_TOLE4(sv); } uint64_t cdf_tole8(uint64_t sv) { return CDF_TOLE8(sv); } void cdf_swap_header(cdf_header_t *h) { size_t i; h->h_magic = CDF_TOLE8(h->h_magic); h->h_uuid[0] = CDF_TOLE8(h->h_uuid[0]); h->h_uuid[1] = CDF_TOLE8(h->h_uuid[1]); h->h_revision = CDF_TOLE2(h->h_revision); h->h_version = CDF_TOLE2(h->h_version); h->h_byte_order = CDF_TOLE2(h->h_byte_order); h->h_sec_size_p2 = CDF_TOLE2(h->h_sec_size_p2); h->h_short_sec_size_p2 = CDF_TOLE2(h->h_short_sec_size_p2); h->h_num_sectors_in_sat = CDF_TOLE4(h->h_num_sectors_in_sat); h->h_secid_first_directory = CDF_TOLE4(h->h_secid_first_directory); h->h_min_size_standard_stream = CDF_TOLE4(h->h_min_size_standard_stream); h->h_secid_first_sector_in_short_sat = CDF_TOLE4((uint32_t)h->h_secid_first_sector_in_short_sat); h->h_num_sectors_in_short_sat = CDF_TOLE4(h->h_num_sectors_in_short_sat); h->h_secid_first_sector_in_master_sat = CDF_TOLE4((uint32_t)h->h_secid_first_sector_in_master_sat); h->h_num_sectors_in_master_sat = CDF_TOLE4(h->h_num_sectors_in_master_sat); for (i = 0; i < __arraycount(h->h_master_sat); i++) h->h_master_sat[i] = CDF_TOLE4((uint32_t)h->h_master_sat[i]); } void cdf_unpack_header(cdf_header_t *h, char *buf) { size_t i; size_t len = 0; CDF_UNPACK(h->h_magic); CDF_UNPACKA(h->h_uuid); CDF_UNPACK(h->h_revision); CDF_UNPACK(h->h_version); CDF_UNPACK(h->h_byte_order); CDF_UNPACK(h->h_sec_size_p2); CDF_UNPACK(h->h_short_sec_size_p2); CDF_UNPACKA(h->h_unused0); CDF_UNPACK(h->h_num_sectors_in_sat); CDF_UNPACK(h->h_secid_first_directory); CDF_UNPACKA(h->h_unused1); CDF_UNPACK(h->h_min_size_standard_stream); CDF_UNPACK(h->h_secid_first_sector_in_short_sat); CDF_UNPACK(h->h_num_sectors_in_short_sat); CDF_UNPACK(h->h_secid_first_sector_in_master_sat); CDF_UNPACK(h->h_num_sectors_in_master_sat); for (i = 0; i < __arraycount(h->h_master_sat); i++) CDF_UNPACK(h->h_master_sat[i]); } void cdf_swap_dir(cdf_directory_t *d) { d->d_namelen = CDF_TOLE2(d->d_namelen); d->d_left_child = CDF_TOLE4((uint32_t)d->d_left_child); d->d_right_child = CDF_TOLE4((uint32_t)d->d_right_child); d->d_storage = CDF_TOLE4((uint32_t)d->d_storage); d->d_storage_uuid[0] = CDF_TOLE8(d->d_storage_uuid[0]); d->d_storage_uuid[1] = CDF_TOLE8(d->d_storage_uuid[1]); d->d_flags = CDF_TOLE4(d->d_flags); d->d_created = CDF_TOLE8((uint64_t)d->d_created); d->d_modified = CDF_TOLE8((uint64_t)d->d_modified); d->d_stream_first_sector = CDF_TOLE4((uint32_t)d->d_stream_first_sector); d->d_size = CDF_TOLE4(d->d_size); } void cdf_swap_class(cdf_classid_t *d) { d->cl_dword = CDF_TOLE4(d->cl_dword); d->cl_word[0] = CDF_TOLE2(d->cl_word[0]); d->cl_word[1] = CDF_TOLE2(d->cl_word[1]); } void cdf_unpack_dir(cdf_directory_t *d, char *buf) { size_t len = 0; CDF_UNPACKA(d->d_name); CDF_UNPACK(d->d_namelen); CDF_UNPACK(d->d_type); CDF_UNPACK(d->d_color); CDF_UNPACK(d->d_left_child); CDF_UNPACK(d->d_right_child); CDF_UNPACK(d->d_storage); CDF_UNPACKA(d->d_storage_uuid); CDF_UNPACK(d->d_flags); CDF_UNPACK(d->d_created); CDF_UNPACK(d->d_modified); CDF_UNPACK(d->d_stream_first_sector); CDF_UNPACK(d->d_size); CDF_UNPACK(d->d_unused0); } static int cdf_check_stream_offset(const cdf_stream_t *sst, const cdf_header_t *h, const void *p, size_t tail, int line) { const char *b = (const char *)sst->sst_tab; const char *e = ((const char *)p) + tail; size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); (void)&line; if (e >= b && (size_t)(e - b) <= ss * sst->sst_len) return 0; DPRINTF(("%d: offset begin %p < end %p || %" SIZE_T_FORMAT "u" " > %" SIZE_T_FORMAT "u [%" SIZE_T_FORMAT "u %" SIZE_T_FORMAT "u]\n", line, b, e, (size_t)(e - b), ss * sst->sst_len, ss, sst->sst_len)); errno = EFTYPE; return -1; } static ssize_t cdf_read(const cdf_info_t *info, off_t off, void *buf, size_t len) { size_t siz = (size_t)off + len; if ((off_t)(off + len) != (off_t)siz) { errno = EINVAL; return -1; } if (info->i_buf != NULL && info->i_len >= siz) { (void)memcpy(buf, &info->i_buf[off], len); return (ssize_t)len; } if (info->i_fd == -1) return -1; if (pread(info->i_fd, buf, len, off) != (ssize_t)len) return -1; return (ssize_t)len; } int cdf_read_header(const cdf_info_t *info, cdf_header_t *h) { char buf[512]; (void)memcpy(cdf_bo.s, "\01\02\03\04", 4); if (cdf_read(info, (off_t)0, buf, sizeof(buf)) == -1) return -1; cdf_unpack_header(h, buf); cdf_swap_header(h); if (h->h_magic != CDF_MAGIC) { DPRINTF(("Bad magic 0x%" INT64_T_FORMAT "x != 0x%" INT64_T_FORMAT "x\n", (unsigned long long)h->h_magic, (unsigned long long)CDF_MAGIC)); goto out; } if (h->h_sec_size_p2 > 20) { DPRINTF(("Bad sector size 0x%u\n", h->h_sec_size_p2)); goto out; } if (h->h_short_sec_size_p2 > 20) { DPRINTF(("Bad short sector size 0x%u\n", h->h_short_sec_size_p2)); goto out; } return 0; out: errno = EFTYPE; return -1; } ssize_t cdf_read_sector(const cdf_info_t *info, void *buf, size_t offs, size_t len, const cdf_header_t *h, cdf_secid_t id) { size_t ss = CDF_SEC_SIZE(h); size_t pos = CDF_SEC_POS(h, id); assert(ss == len); return cdf_read(info, (off_t)pos, ((char *)buf) + offs, len); } ssize_t cdf_read_short_sector(const cdf_stream_t *sst, void *buf, size_t offs, size_t len, const cdf_header_t *h, cdf_secid_t id) { size_t ss = CDF_SHORT_SEC_SIZE(h); size_t pos = CDF_SHORT_SEC_POS(h, id); assert(ss == len); if (pos + len > CDF_SEC_SIZE(h) * sst->sst_len) { DPRINTF(("Out of bounds read %" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", pos + len, CDF_SEC_SIZE(h) * sst->sst_len)); return -1; } (void)memcpy(((char *)buf) + offs, ((const char *)sst->sst_tab) + pos, len); return len; } /* * Read the sector allocation table. */ int cdf_read_sat(const cdf_info_t *info, cdf_header_t *h, cdf_sat_t *sat) { size_t i, j, k; size_t ss = CDF_SEC_SIZE(h); cdf_secid_t *msa, mid, sec; size_t nsatpersec = (ss / sizeof(mid)) - 1; for (i = 0; i < __arraycount(h->h_master_sat); i++) if (h->h_master_sat[i] == CDF_SECID_FREE) break; #define CDF_SEC_LIMIT (UINT32_MAX / (4 * ss)) if ((nsatpersec > 0 && h->h_num_sectors_in_master_sat > CDF_SEC_LIMIT / nsatpersec) || i > CDF_SEC_LIMIT) { DPRINTF(("Number of sectors in master SAT too big %u %" SIZE_T_FORMAT "u\n", h->h_num_sectors_in_master_sat, i)); errno = EFTYPE; return -1; } sat->sat_len = h->h_num_sectors_in_master_sat * nsatpersec + i; DPRINTF(("sat_len = %" SIZE_T_FORMAT "u ss = %" SIZE_T_FORMAT "u\n", sat->sat_len, ss)); if ((sat->sat_tab = CAST(cdf_secid_t *, calloc(sat->sat_len, ss))) == NULL) return -1; for (i = 0; i < __arraycount(h->h_master_sat); i++) { if (h->h_master_sat[i] < 0) break; if (cdf_read_sector(info, sat->sat_tab, ss * i, ss, h, h->h_master_sat[i]) != (ssize_t)ss) { DPRINTF(("Reading sector %d", h->h_master_sat[i])); goto out1; } } if ((msa = CAST(cdf_secid_t *, calloc(1, ss))) == NULL) goto out1; mid = h->h_secid_first_sector_in_master_sat; for (j = 0; j < h->h_num_sectors_in_master_sat; j++) { if (mid < 0) goto out; if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Reading master sector loop limit")); errno = EFTYPE; goto out2; } if (cdf_read_sector(info, msa, 0, ss, h, mid) != (ssize_t)ss) { DPRINTF(("Reading master sector %d", mid)); goto out2; } for (k = 0; k < nsatpersec; k++, i++) { sec = CDF_TOLE4((uint32_t)msa[k]); if (sec < 0) goto out; if (i >= sat->sat_len) { DPRINTF(("Out of bounds reading MSA %" SIZE_T_FORMAT "u >= %" SIZE_T_FORMAT "u", i, sat->sat_len)); errno = EFTYPE; goto out2; } if (cdf_read_sector(info, sat->sat_tab, ss * i, ss, h, sec) != (ssize_t)ss) { DPRINTF(("Reading sector %d", CDF_TOLE4(msa[k]))); goto out2; } } mid = CDF_TOLE4((uint32_t)msa[nsatpersec]); } out: sat->sat_len = i; free(msa); return 0; out2: free(msa); out1: free(sat->sat_tab); return -1; } size_t cdf_count_chain(const cdf_sat_t *sat, cdf_secid_t sid, size_t size) { size_t i, j; cdf_secid_t maxsector = (cdf_secid_t)((sat->sat_len * size) / sizeof(maxsector)); DPRINTF(("Chain:")); + if (sid == CDF_SECID_END_OF_CHAIN) { + /* 0-length chain. */ + DPRINTF((" empty\n")); + return 0; + } + for (j = i = 0; sid >= 0; i++, j++) { DPRINTF((" %d", sid)); if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Counting chain loop limit")); errno = EFTYPE; return (size_t)-1; } if (sid >= maxsector) { DPRINTF(("Sector %d >= %d\n", sid, maxsector)); errno = EFTYPE; return (size_t)-1; } sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); } if (i == 0) { DPRINTF((" none, sid: %d\n", sid)); return (size_t)-1; } DPRINTF(("\n")); return i; } int cdf_read_long_sector_chain(const cdf_info_t *info, const cdf_header_t *h, const cdf_sat_t *sat, cdf_secid_t sid, size_t len, cdf_stream_t *scn) { size_t ss = CDF_SEC_SIZE(h), i, j; ssize_t nr; scn->sst_len = cdf_count_chain(sat, sid, ss); scn->sst_dirlen = len; if (scn->sst_len == (size_t)-1) return -1; scn->sst_tab = calloc(scn->sst_len, ss); if (scn->sst_tab == NULL) return -1; for (j = i = 0; sid >= 0; i++, j++) { if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Read long sector chain loop limit")); errno = EFTYPE; goto out; } if (i >= scn->sst_len) { DPRINTF(("Out of bounds reading long sector chain " "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", i, scn->sst_len)); errno = EFTYPE; goto out; } if ((nr = cdf_read_sector(info, scn->sst_tab, i * ss, ss, h, sid)) != (ssize_t)ss) { if (i == scn->sst_len - 1 && nr > 0) { /* Last sector might be truncated */ return 0; } DPRINTF(("Reading long sector chain %d", sid)); goto out; } sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); } return 0; out: free(scn->sst_tab); return -1; } int cdf_read_short_sector_chain(const cdf_header_t *h, const cdf_sat_t *ssat, const cdf_stream_t *sst, cdf_secid_t sid, size_t len, cdf_stream_t *scn) { size_t ss = CDF_SHORT_SEC_SIZE(h), i, j; scn->sst_len = cdf_count_chain(ssat, sid, CDF_SEC_SIZE(h)); scn->sst_dirlen = len; if (sst->sst_tab == NULL || scn->sst_len == (size_t)-1) return -1; scn->sst_tab = calloc(scn->sst_len, ss); if (scn->sst_tab == NULL) return -1; for (j = i = 0; sid >= 0; i++, j++) { if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Read short sector chain loop limit")); errno = EFTYPE; goto out; } if (i >= scn->sst_len) { DPRINTF(("Out of bounds reading short sector chain " "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", i, scn->sst_len)); errno = EFTYPE; goto out; } if (cdf_read_short_sector(sst, scn->sst_tab, i * ss, ss, h, sid) != (ssize_t)ss) { DPRINTF(("Reading short sector chain %d", sid)); goto out; } sid = CDF_TOLE4((uint32_t)ssat->sat_tab[sid]); } return 0; out: free(scn->sst_tab); return -1; } int cdf_read_sector_chain(const cdf_info_t *info, const cdf_header_t *h, const cdf_sat_t *sat, const cdf_sat_t *ssat, const cdf_stream_t *sst, cdf_secid_t sid, size_t len, cdf_stream_t *scn) { if (len < h->h_min_size_standard_stream && sst->sst_tab != NULL) return cdf_read_short_sector_chain(h, ssat, sst, sid, len, scn); else return cdf_read_long_sector_chain(info, h, sat, sid, len, scn); } int cdf_read_dir(const cdf_info_t *info, const cdf_header_t *h, const cdf_sat_t *sat, cdf_dir_t *dir) { size_t i, j; size_t ss = CDF_SEC_SIZE(h), ns, nd; char *buf; cdf_secid_t sid = h->h_secid_first_directory; ns = cdf_count_chain(sat, sid, ss); if (ns == (size_t)-1) return -1; nd = ss / CDF_DIRECTORY_SIZE; dir->dir_len = ns * nd; dir->dir_tab = CAST(cdf_directory_t *, calloc(dir->dir_len, sizeof(dir->dir_tab[0]))); if (dir->dir_tab == NULL) return -1; if ((buf = CAST(char *, malloc(ss))) == NULL) { free(dir->dir_tab); return -1; } for (j = i = 0; i < ns; i++, j++) { if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Read dir loop limit")); errno = EFTYPE; goto out; } if (cdf_read_sector(info, buf, 0, ss, h, sid) != (ssize_t)ss) { DPRINTF(("Reading directory sector %d", sid)); goto out; } for (j = 0; j < nd; j++) { cdf_unpack_dir(&dir->dir_tab[i * nd + j], &buf[j * CDF_DIRECTORY_SIZE]); } sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); } if (NEED_SWAP) for (i = 0; i < dir->dir_len; i++) cdf_swap_dir(&dir->dir_tab[i]); free(buf); return 0; out: free(dir->dir_tab); free(buf); return -1; } int cdf_read_ssat(const cdf_info_t *info, const cdf_header_t *h, const cdf_sat_t *sat, cdf_sat_t *ssat) { size_t i, j; size_t ss = CDF_SEC_SIZE(h); cdf_secid_t sid = h->h_secid_first_sector_in_short_sat; ssat->sat_len = cdf_count_chain(sat, sid, CDF_SEC_SIZE(h)); if (ssat->sat_len == (size_t)-1) return -1; ssat->sat_tab = CAST(cdf_secid_t *, calloc(ssat->sat_len, ss)); if (ssat->sat_tab == NULL) return -1; for (j = i = 0; sid >= 0; i++, j++) { if (j >= CDF_LOOP_LIMIT) { DPRINTF(("Read short sat sector loop limit")); errno = EFTYPE; goto out; } if (i >= ssat->sat_len) { DPRINTF(("Out of bounds reading short sector chain " "%" SIZE_T_FORMAT "u > %" SIZE_T_FORMAT "u\n", i, ssat->sat_len)); errno = EFTYPE; goto out; } if (cdf_read_sector(info, ssat->sat_tab, i * ss, ss, h, sid) != (ssize_t)ss) { DPRINTF(("Reading short sat sector %d", sid)); goto out; } sid = CDF_TOLE4((uint32_t)sat->sat_tab[sid]); } return 0; out: free(ssat->sat_tab); return -1; } int cdf_read_short_stream(const cdf_info_t *info, const cdf_header_t *h, const cdf_sat_t *sat, const cdf_dir_t *dir, cdf_stream_t *scn, const cdf_directory_t **root) { size_t i; const cdf_directory_t *d; *root = NULL; for (i = 0; i < dir->dir_len; i++) if (dir->dir_tab[i].d_type == CDF_DIR_TYPE_ROOT_STORAGE) break; /* If the it is not there, just fake it; some docs don't have it */ if (i == dir->dir_len) goto out; d = &dir->dir_tab[i]; *root = d; /* If the it is not there, just fake it; some docs don't have it */ if (d->d_stream_first_sector < 0) goto out; return cdf_read_long_sector_chain(info, h, sat, d->d_stream_first_sector, d->d_size, scn); out: scn->sst_tab = NULL; scn->sst_len = 0; scn->sst_dirlen = 0; return 0; } static int cdf_namecmp(const char *d, const uint16_t *s, size_t l) { for (; l--; d++, s++) if (*d != CDF_TOLE2(*s)) return (unsigned char)*d - CDF_TOLE2(*s); return 0; } int cdf_read_summary_info(const cdf_info_t *info, const cdf_header_t *h, const cdf_sat_t *sat, const cdf_sat_t *ssat, const cdf_stream_t *sst, const cdf_dir_t *dir, cdf_stream_t *scn) { return cdf_read_user_stream(info, h, sat, ssat, sst, dir, "\05SummaryInformation", scn); } int cdf_read_user_stream(const cdf_info_t *info, const cdf_header_t *h, const cdf_sat_t *sat, const cdf_sat_t *ssat, const cdf_stream_t *sst, const cdf_dir_t *dir, const char *name, cdf_stream_t *scn) { size_t i; const cdf_directory_t *d; size_t name_len = strlen(name) + 1; for (i = dir->dir_len; i > 0; i--) if (dir->dir_tab[i - 1].d_type == CDF_DIR_TYPE_USER_STREAM && cdf_namecmp(name, dir->dir_tab[i - 1].d_name, name_len) == 0) break; if (i == 0) { DPRINTF(("Cannot find user stream `%s'\n", name)); errno = ESRCH; return -1; } d = &dir->dir_tab[i - 1]; return cdf_read_sector_chain(info, h, sat, ssat, sst, d->d_stream_first_sector, d->d_size, scn); } int cdf_read_property_info(const cdf_stream_t *sst, const cdf_header_t *h, uint32_t offs, cdf_property_info_t **info, size_t *count, size_t *maxcount) { const cdf_section_header_t *shp; cdf_section_header_t sh; const uint8_t *p, *q, *e; int16_t s16; int32_t s32; uint32_t u32; int64_t s64; uint64_t u64; cdf_timestamp_t tp; size_t i, o, o4, nelements, j; cdf_property_info_t *inp; if (offs > UINT32_MAX / 4) { errno = EFTYPE; goto out; } shp = CAST(const cdf_section_header_t *, (const void *) ((const char *)sst->sst_tab + offs)); if (cdf_check_stream_offset(sst, h, shp, sizeof(*shp), __LINE__) == -1) goto out; sh.sh_len = CDF_TOLE4(shp->sh_len); #define CDF_SHLEN_LIMIT (UINT32_MAX / 8) if (sh.sh_len > CDF_SHLEN_LIMIT) { errno = EFTYPE; goto out; } sh.sh_properties = CDF_TOLE4(shp->sh_properties); #define CDF_PROP_LIMIT (UINT32_MAX / (4 * sizeof(*inp))) if (sh.sh_properties > CDF_PROP_LIMIT) goto out; DPRINTF(("section len: %u properties %u\n", sh.sh_len, sh.sh_properties)); if (*maxcount) { if (*maxcount > CDF_PROP_LIMIT) goto out; *maxcount += sh.sh_properties; inp = CAST(cdf_property_info_t *, realloc(*info, *maxcount * sizeof(*inp))); } else { *maxcount = sh.sh_properties; inp = CAST(cdf_property_info_t *, malloc(*maxcount * sizeof(*inp))); } if (inp == NULL) goto out; *info = inp; inp += *count; *count += sh.sh_properties; p = CAST(const uint8_t *, (const void *) ((const char *)(const void *)sst->sst_tab + offs + sizeof(sh))); e = CAST(const uint8_t *, (const void *) (((const char *)(const void *)shp) + sh.sh_len)); if (cdf_check_stream_offset(sst, h, e, 0, __LINE__) == -1) goto out; for (i = 0; i < sh.sh_properties; i++) { size_t tail = (i << 1) + 1; + size_t ofs; if (cdf_check_stream_offset(sst, h, p, tail * sizeof(uint32_t), __LINE__) == -1) goto out; - size_t ofs = CDF_GETUINT32(p, tail); + ofs = CDF_GETUINT32(p, tail); q = (const uint8_t *)(const void *) ((const char *)(const void *)p + ofs - 2 * sizeof(uint32_t)); + if (q < p) { + DPRINTF(("Wrapped around %p < %p\n", q, p)); + goto out; + } if (q > e) { DPRINTF(("Ran of the end %p > %p\n", q, e)); goto out; } inp[i].pi_id = CDF_GETUINT32(p, i << 1); inp[i].pi_type = CDF_GETUINT32(q, 0); DPRINTF(("%" SIZE_T_FORMAT "u) id=%x type=%x offs=0x%tx,0x%x\n", i, inp[i].pi_id, inp[i].pi_type, q - p, offs)); if (inp[i].pi_type & CDF_VECTOR) { nelements = CDF_GETUINT32(q, 1); if (nelements == 0) { DPRINTF(("CDF_VECTOR with nelements == 0\n")); goto out; } o = 2; } else { nelements = 1; o = 1; } o4 = o * sizeof(uint32_t); if (inp[i].pi_type & (CDF_ARRAY|CDF_BYREF|CDF_RESERVED)) goto unknown; switch (inp[i].pi_type & CDF_TYPEMASK) { case CDF_NULL: case CDF_EMPTY: break; case CDF_SIGNED16: if (inp[i].pi_type & CDF_VECTOR) goto unknown; (void)memcpy(&s16, &q[o4], sizeof(s16)); inp[i].pi_s16 = CDF_TOLE2(s16); break; case CDF_SIGNED32: if (inp[i].pi_type & CDF_VECTOR) goto unknown; (void)memcpy(&s32, &q[o4], sizeof(s32)); inp[i].pi_s32 = CDF_TOLE4((uint32_t)s32); break; case CDF_BOOL: case CDF_UNSIGNED32: if (inp[i].pi_type & CDF_VECTOR) goto unknown; (void)memcpy(&u32, &q[o4], sizeof(u32)); inp[i].pi_u32 = CDF_TOLE4(u32); break; case CDF_SIGNED64: if (inp[i].pi_type & CDF_VECTOR) goto unknown; (void)memcpy(&s64, &q[o4], sizeof(s64)); inp[i].pi_s64 = CDF_TOLE8((uint64_t)s64); break; case CDF_UNSIGNED64: if (inp[i].pi_type & CDF_VECTOR) goto unknown; (void)memcpy(&u64, &q[o4], sizeof(u64)); inp[i].pi_u64 = CDF_TOLE8((uint64_t)u64); break; case CDF_FLOAT: if (inp[i].pi_type & CDF_VECTOR) goto unknown; (void)memcpy(&u32, &q[o4], sizeof(u32)); u32 = CDF_TOLE4(u32); memcpy(&inp[i].pi_f, &u32, sizeof(inp[i].pi_f)); break; case CDF_DOUBLE: if (inp[i].pi_type & CDF_VECTOR) goto unknown; (void)memcpy(&u64, &q[o4], sizeof(u64)); u64 = CDF_TOLE8((uint64_t)u64); memcpy(&inp[i].pi_d, &u64, sizeof(inp[i].pi_d)); break; case CDF_LENGTH32_STRING: case CDF_LENGTH32_WSTRING: if (nelements > 1) { size_t nelem = inp - *info; if (*maxcount > CDF_PROP_LIMIT || nelements > CDF_PROP_LIMIT) goto out; *maxcount += nelements; inp = CAST(cdf_property_info_t *, realloc(*info, *maxcount * sizeof(*inp))); if (inp == NULL) goto out; *info = inp; inp = *info + nelem; } DPRINTF(("nelements = %" SIZE_T_FORMAT "u\n", nelements)); for (j = 0; j < nelements && i < sh.sh_properties; j++, i++) { uint32_t l = CDF_GETUINT32(q, o); inp[i].pi_str.s_len = l; inp[i].pi_str.s_buf = (const char *) (const void *)(&q[o4 + sizeof(l)]); DPRINTF(("l = %d, r = %" SIZE_T_FORMAT "u, s = %s\n", l, CDF_ROUND(l, sizeof(l)), inp[i].pi_str.s_buf)); if (l & 1) l++; o += l >> 1; if (q + o >= e) goto out; o4 = o * sizeof(uint32_t); } i--; break; case CDF_FILETIME: if (inp[i].pi_type & CDF_VECTOR) goto unknown; (void)memcpy(&tp, &q[o4], sizeof(tp)); inp[i].pi_tp = CDF_TOLE8((uint64_t)tp); break; case CDF_CLIPBOARD: if (inp[i].pi_type & CDF_VECTOR) goto unknown; break; default: unknown: DPRINTF(("Don't know how to deal with %x\n", inp[i].pi_type)); break; } } return 0; out: free(*info); return -1; } int cdf_unpack_summary_info(const cdf_stream_t *sst, const cdf_header_t *h, cdf_summary_info_header_t *ssi, cdf_property_info_t **info, size_t *count) { size_t maxcount; const cdf_summary_info_header_t *si = CAST(const cdf_summary_info_header_t *, sst->sst_tab); const cdf_section_declaration_t *sd = CAST(const cdf_section_declaration_t *, (const void *) ((const char *)sst->sst_tab + CDF_SECTION_DECLARATION_OFFSET)); if (cdf_check_stream_offset(sst, h, si, sizeof(*si), __LINE__) == -1 || cdf_check_stream_offset(sst, h, sd, sizeof(*sd), __LINE__) == -1) return -1; ssi->si_byte_order = CDF_TOLE2(si->si_byte_order); ssi->si_os_version = CDF_TOLE2(si->si_os_version); ssi->si_os = CDF_TOLE2(si->si_os); ssi->si_class = si->si_class; cdf_swap_class(&ssi->si_class); ssi->si_count = CDF_TOLE4(si->si_count); *count = 0; maxcount = 0; *info = NULL; if (cdf_read_property_info(sst, h, CDF_TOLE4(sd->sd_offset), info, count, &maxcount) == -1) return -1; return 0; } +#define extract_catalog_field(f, l) \ + memcpy(&ce[i].f, b + (l), sizeof(ce[i].f)); \ + ce[i].f = CDF_TOLE(ce[i].f) int +cdf_unpack_catalog(const cdf_header_t *h, const cdf_stream_t *sst, + cdf_catalog_t **cat) +{ + size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? + CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); + const char *b = CAST(const char *, sst->sst_tab); + const char *eb = b + ss * sst->sst_len; + size_t nr, i, k; + cdf_catalog_entry_t *ce; + uint16_t reclen; + const uint16_t *np; + + for (nr = 0; b < eb; nr++) { + memcpy(&reclen, b, sizeof(reclen)); + reclen = CDF_TOLE2(reclen); + if (reclen == 0) + break; + b += reclen; + } + *cat = CAST(cdf_catalog_t *, + malloc(sizeof(cdf_catalog_t) + nr * sizeof(*ce))); + (*cat)->cat_num = nr; + ce = (*cat)->cat_e; + b = CAST(const char *, sst->sst_tab); + for (i = 0; i < nr; i++) { + extract_catalog_field(ce_namlen, 0); + extract_catalog_field(ce_num, 2); + extract_catalog_field(ce_timestamp, 6); + reclen = ce[i].ce_namlen; + ce[i].ce_namlen = + sizeof(ce[i].ce_name) / sizeof(ce[i].ce_name[0]) - 1; + if (ce[i].ce_namlen > reclen - 14) + ce[i].ce_namlen = reclen - 14; + np = CAST(const uint16_t *, (b + 16)); + for (k = 0; k < ce[i].ce_namlen; k++) { + ce[i].ce_name[k] = np[k]; + CDF_TOLE2(ce[i].ce_name[k]); + } + ce[i].ce_name[ce[i].ce_namlen] = 0; + b += reclen; + } + return 0; +} + +int cdf_print_classid(char *buf, size_t buflen, const cdf_classid_t *id) { return snprintf(buf, buflen, "%.8x-%.4x-%.4x-%.2x%.2x-" "%.2x%.2x%.2x%.2x%.2x%.2x", id->cl_dword, id->cl_word[0], id->cl_word[1], id->cl_two[0], id->cl_two[1], id->cl_six[0], id->cl_six[1], id->cl_six[2], id->cl_six[3], id->cl_six[4], id->cl_six[5]); } static const struct { uint32_t v; const char *n; } vn[] = { { CDF_PROPERTY_CODE_PAGE, "Code page" }, { CDF_PROPERTY_TITLE, "Title" }, { CDF_PROPERTY_SUBJECT, "Subject" }, { CDF_PROPERTY_AUTHOR, "Author" }, { CDF_PROPERTY_KEYWORDS, "Keywords" }, { CDF_PROPERTY_COMMENTS, "Comments" }, { CDF_PROPERTY_TEMPLATE, "Template" }, { CDF_PROPERTY_LAST_SAVED_BY, "Last Saved By" }, { CDF_PROPERTY_REVISION_NUMBER, "Revision Number" }, { CDF_PROPERTY_TOTAL_EDITING_TIME, "Total Editing Time" }, { CDF_PROPERTY_LAST_PRINTED, "Last Printed" }, { CDF_PROPERTY_CREATE_TIME, "Create Time/Date" }, { CDF_PROPERTY_LAST_SAVED_TIME, "Last Saved Time/Date" }, { CDF_PROPERTY_NUMBER_OF_PAGES, "Number of Pages" }, { CDF_PROPERTY_NUMBER_OF_WORDS, "Number of Words" }, { CDF_PROPERTY_NUMBER_OF_CHARACTERS, "Number of Characters" }, { CDF_PROPERTY_THUMBNAIL, "Thumbnail" }, { CDF_PROPERTY_NAME_OF_APPLICATION, "Name of Creating Application" }, { CDF_PROPERTY_SECURITY, "Security" }, { CDF_PROPERTY_LOCALE_ID, "Locale ID" }, }; int cdf_print_property_name(char *buf, size_t bufsiz, uint32_t p) { size_t i; for (i = 0; i < __arraycount(vn); i++) if (vn[i].v == p) return snprintf(buf, bufsiz, "%s", vn[i].n); return snprintf(buf, bufsiz, "0x%x", p); } int cdf_print_elapsed_time(char *buf, size_t bufsiz, cdf_timestamp_t ts) { int len = 0; int days, hours, mins, secs; ts /= CDF_TIME_PREC; secs = (int)(ts % 60); ts /= 60; mins = (int)(ts % 60); ts /= 60; hours = (int)(ts % 24); ts /= 24; days = (int)ts; if (days) { len += snprintf(buf + len, bufsiz - len, "%dd+", days); if ((size_t)len >= bufsiz) return len; } if (days || hours) { len += snprintf(buf + len, bufsiz - len, "%.2d:", hours); if ((size_t)len >= bufsiz) return len; } len += snprintf(buf + len, bufsiz - len, "%.2d:", mins); if ((size_t)len >= bufsiz) return len; len += snprintf(buf + len, bufsiz - len, "%.2d", secs); return len; } +char * +cdf_u16tos8(char *buf, size_t len, const uint16_t *p) +{ + size_t i; + for (i = 0; i < len && p[i]; i++) + buf[i] = (char)p[i]; + buf[i] = '\0'; + return buf; +} #ifdef CDF_DEBUG void cdf_dump_header(const cdf_header_t *h) { size_t i; #define DUMP(a, b) (void)fprintf(stderr, "%40.40s = " a "\n", # b, h->h_ ## b) #define DUMP2(a, b) (void)fprintf(stderr, "%40.40s = " a " (" a ")\n", # b, \ h->h_ ## b, 1 << h->h_ ## b) DUMP("%d", revision); DUMP("%d", version); DUMP("0x%x", byte_order); DUMP2("%d", sec_size_p2); DUMP2("%d", short_sec_size_p2); DUMP("%d", num_sectors_in_sat); DUMP("%d", secid_first_directory); DUMP("%d", min_size_standard_stream); DUMP("%d", secid_first_sector_in_short_sat); DUMP("%d", num_sectors_in_short_sat); DUMP("%d", secid_first_sector_in_master_sat); DUMP("%d", num_sectors_in_master_sat); for (i = 0; i < __arraycount(h->h_master_sat); i++) { if (h->h_master_sat[i] == CDF_SECID_FREE) break; - (void)fprintf(stderr, "%35.35s[%.3zu] = %d\n", + (void)fprintf(stderr, "%35.35s[%.3" SIZE_T_FORMAT "u] = %d\n", "master_sat", i, h->h_master_sat[i]); } } void cdf_dump_sat(const char *prefix, const cdf_sat_t *sat, size_t size) { size_t i, j, s = size / sizeof(cdf_secid_t); for (i = 0; i < sat->sat_len; i++) { (void)fprintf(stderr, "%s[%" SIZE_T_FORMAT "u]:\n%.6" SIZE_T_FORMAT "u: ", prefix, i, i * s); for (j = 0; j < s; j++) { (void)fprintf(stderr, "%5d, ", CDF_TOLE4(sat->sat_tab[s * i + j])); if ((j + 1) % 10 == 0) (void)fprintf(stderr, "\n%.6" SIZE_T_FORMAT "u: ", i * s + j + 1); } (void)fprintf(stderr, "\n"); } } void cdf_dump(void *v, size_t len) { size_t i, j; unsigned char *p = v; char abuf[16]; (void)fprintf(stderr, "%.4x: ", 0); for (i = 0, j = 0; i < len; i++, p++) { (void)fprintf(stderr, "%.2x ", *p); abuf[j++] = isprint(*p) ? *p : '.'; if (j == 16) { j = 0; abuf[15] = '\0'; (void)fprintf(stderr, "%s\n%.4" SIZE_T_FORMAT "x: ", abuf, i + 1); } } (void)fprintf(stderr, "\n"); } void cdf_dump_stream(const cdf_header_t *h, const cdf_stream_t *sst) { size_t ss = sst->sst_dirlen < h->h_min_size_standard_stream ? CDF_SHORT_SEC_SIZE(h) : CDF_SEC_SIZE(h); cdf_dump(sst->sst_tab, ss * sst->sst_len); } void cdf_dump_dir(const cdf_info_t *info, const cdf_header_t *h, const cdf_sat_t *sat, const cdf_sat_t *ssat, const cdf_stream_t *sst, const cdf_dir_t *dir) { size_t i, j; cdf_directory_t *d; char name[__arraycount(d->d_name)]; cdf_stream_t scn; struct timespec ts; static const char *types[] = { "empty", "user storage", "user stream", "lockbytes", "property", "root storage" }; for (i = 0; i < dir->dir_len; i++) { char buf[26]; d = &dir->dir_tab[i]; for (j = 0; j < sizeof(name); j++) name[j] = (char)CDF_TOLE2(d->d_name[j]); (void)fprintf(stderr, "Directory %" SIZE_T_FORMAT "u: %s\n", i, name); if (d->d_type < __arraycount(types)) (void)fprintf(stderr, "Type: %s\n", types[d->d_type]); else (void)fprintf(stderr, "Type: %d\n", d->d_type); (void)fprintf(stderr, "Color: %s\n", d->d_color ? "black" : "red"); (void)fprintf(stderr, "Left child: %d\n", d->d_left_child); (void)fprintf(stderr, "Right child: %d\n", d->d_right_child); (void)fprintf(stderr, "Flags: 0x%x\n", d->d_flags); cdf_timestamp_to_timespec(&ts, d->d_created); (void)fprintf(stderr, "Created %s", cdf_ctime(&ts.tv_sec, buf)); cdf_timestamp_to_timespec(&ts, d->d_modified); (void)fprintf(stderr, "Modified %s", cdf_ctime(&ts.tv_sec, buf)); (void)fprintf(stderr, "Stream %d\n", d->d_stream_first_sector); (void)fprintf(stderr, "Size %d\n", d->d_size); switch (d->d_type) { case CDF_DIR_TYPE_USER_STORAGE: (void)fprintf(stderr, "Storage: %d\n", d->d_storage); break; case CDF_DIR_TYPE_USER_STREAM: if (sst == NULL) break; if (cdf_read_sector_chain(info, h, sat, ssat, sst, d->d_stream_first_sector, d->d_size, &scn) == -1) { warn("Can't read stream for %s at %d len %d", name, d->d_stream_first_sector, d->d_size); break; } cdf_dump_stream(h, &scn); free(scn.sst_tab); break; default: break; } } } void cdf_dump_property_info(const cdf_property_info_t *info, size_t count) { cdf_timestamp_t tp; struct timespec ts; char buf[64]; size_t i, j; for (i = 0; i < count; i++) { cdf_print_property_name(buf, sizeof(buf), info[i].pi_id); (void)fprintf(stderr, "%" SIZE_T_FORMAT "u) %s: ", i, buf); switch (info[i].pi_type) { case CDF_NULL: break; case CDF_SIGNED16: (void)fprintf(stderr, "signed 16 [%hd]\n", info[i].pi_s16); break; case CDF_SIGNED32: (void)fprintf(stderr, "signed 32 [%d]\n", info[i].pi_s32); break; case CDF_UNSIGNED32: (void)fprintf(stderr, "unsigned 32 [%u]\n", info[i].pi_u32); break; case CDF_FLOAT: (void)fprintf(stderr, "float [%g]\n", info[i].pi_f); break; case CDF_DOUBLE: (void)fprintf(stderr, "double [%g]\n", info[i].pi_d); break; case CDF_LENGTH32_STRING: (void)fprintf(stderr, "string %u [%.*s]\n", info[i].pi_str.s_len, info[i].pi_str.s_len, info[i].pi_str.s_buf); break; case CDF_LENGTH32_WSTRING: (void)fprintf(stderr, "string %u [", info[i].pi_str.s_len); for (j = 0; j < info[i].pi_str.s_len - 1; j++) (void)fputc(info[i].pi_str.s_buf[j << 1], stderr); (void)fprintf(stderr, "]\n"); break; case CDF_FILETIME: tp = info[i].pi_tp; if (tp < 1000000000000000LL) { cdf_print_elapsed_time(buf, sizeof(buf), tp); (void)fprintf(stderr, "timestamp %s\n", buf); } else { char buf[26]; cdf_timestamp_to_timespec(&ts, tp); (void)fprintf(stderr, "timestamp %s", cdf_ctime(&ts.tv_sec, buf)); } break; case CDF_CLIPBOARD: (void)fprintf(stderr, "CLIPBOARD %u\n", info[i].pi_u32); break; default: DPRINTF(("Don't know how to deal with %x\n", info[i].pi_type)); break; } } } void cdf_dump_summary_info(const cdf_header_t *h, const cdf_stream_t *sst) { char buf[128]; cdf_summary_info_header_t ssi; cdf_property_info_t *info; size_t count; (void)&h; if (cdf_unpack_summary_info(sst, h, &ssi, &info, &count) == -1) return; (void)fprintf(stderr, "Endian: %x\n", ssi.si_byte_order); (void)fprintf(stderr, "Os Version %d.%d\n", ssi.si_os_version & 0xff, - ssi.si_os_version >> 8); + ssi.si_os_version >> 8); (void)fprintf(stderr, "Os %d\n", ssi.si_os); cdf_print_classid(buf, sizeof(buf), &ssi.si_class); (void)fprintf(stderr, "Class %s\n", buf); (void)fprintf(stderr, "Count %d\n", ssi.si_count); cdf_dump_property_info(info, count); free(info); } + +void +cdf_dump_catalog(const cdf_header_t *h, const cdf_stream_t *sst) +{ + cdf_catalog_t *cat; + cdf_unpack_catalog(h, sst, &cat); + const cdf_catalog_entry_t *ce = cat->cat_e; + struct timespec ts; + char tbuf[64], sbuf[256]; + size_t i; + + printf("Catalog:\n"); + for (i = 0; i < cat->cat_num; i++) { + cdf_timestamp_to_timespec(&ts, ce[i].ce_timestamp); + printf("\t%d %s %s", ce[i].ce_num, + cdf_u16tos8(sbuf, ce[i].ce_namlen, ce[i].ce_name), + cdf_ctime(&ts.tv_sec, tbuf)); + } + free(cat); +} + #endif #ifdef TEST int main(int argc, char *argv[]) { int i; cdf_header_t h; cdf_sat_t sat, ssat; cdf_stream_t sst, scn; cdf_dir_t dir; cdf_info_t info; + const cdf_directory_t *root; if (argc < 2) { (void)fprintf(stderr, "Usage: %s \n", getprogname()); return -1; } info.i_buf = NULL; info.i_len = 0; for (i = 1; i < argc; i++) { if ((info.i_fd = open(argv[1], O_RDONLY)) == -1) err(1, "Cannot open `%s'", argv[1]); if (cdf_read_header(&info, &h) == -1) err(1, "Cannot read header"); #ifdef CDF_DEBUG cdf_dump_header(&h); #endif if (cdf_read_sat(&info, &h, &sat) == -1) err(1, "Cannot read sat"); #ifdef CDF_DEBUG cdf_dump_sat("SAT", &sat, CDF_SEC_SIZE(&h)); #endif if (cdf_read_ssat(&info, &h, &sat, &ssat) == -1) err(1, "Cannot read ssat"); #ifdef CDF_DEBUG cdf_dump_sat("SSAT", &ssat, CDF_SHORT_SEC_SIZE(&h)); #endif if (cdf_read_dir(&info, &h, &sat, &dir) == -1) err(1, "Cannot read dir"); - if (cdf_read_short_stream(&info, &h, &sat, &dir, &sst) == -1) + if (cdf_read_short_stream(&info, &h, &sat, &dir, &sst, &root) + == -1) err(1, "Cannot read short stream"); #ifdef CDF_DEBUG cdf_dump_stream(&h, &sst); #endif #ifdef CDF_DEBUG cdf_dump_dir(&info, &h, &sat, &ssat, &sst, &dir); #endif if (cdf_read_summary_info(&info, &h, &sat, &ssat, &sst, &dir, &scn) == -1) - err(1, "Cannot read summary info"); + warn("Cannot read summary info"); #ifdef CDF_DEBUG - cdf_dump_summary_info(&h, &scn); + else + cdf_dump_summary_info(&h, &scn); +#endif + if (cdf_read_catalog(&info, &h, &sat, &ssat, &sst, &dir, + &scn) == -1) + warn("Cannot read catalog"); +#ifdef CDF_DEBUG + else + cdf_dump_catalog(&h, &scn); #endif (void)close(info.i_fd); } return 0; } #endif Index: releng/10.1/contrib/file/src/cdf.h =================================================================== --- releng/10.1/contrib/file/src/cdf.h (revision 284192) +++ releng/10.1/contrib/file/src/cdf.h (revision 284193) @@ -1,329 +1,352 @@ /*- * Copyright (c) 2008 Christos Zoulas * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ /* * Parse Composite Document Files, the format used in Microsoft Office * document files before they switched to zipped XML. * Info from: http://sc.openoffice.org/compdocfileformat.pdf * * N.B. This is the "Composite Document File" format, and not the * "Compound Document Format", nor the "Channel Definition Format". */ #ifndef _H_CDF_ #define _H_CDF_ #ifdef WIN32 #include #define timespec timeval #define tv_nsec tv_usec #endif #ifdef __DJGPP__ #define timespec timeval #define tv_nsec tv_usec #endif typedef int32_t cdf_secid_t; #define CDF_LOOP_LIMIT 10000 #define CDF_SECID_NULL 0 #define CDF_SECID_FREE -1 #define CDF_SECID_END_OF_CHAIN -2 #define CDF_SECID_SECTOR_ALLOCATION_TABLE -3 #define CDF_SECID_MASTER_SECTOR_ALLOCATION_TABLE -4 typedef struct { uint64_t h_magic; #define CDF_MAGIC 0xE11AB1A1E011CFD0LL uint64_t h_uuid[2]; uint16_t h_revision; uint16_t h_version; uint16_t h_byte_order; uint16_t h_sec_size_p2; uint16_t h_short_sec_size_p2; uint8_t h_unused0[10]; uint32_t h_num_sectors_in_sat; uint32_t h_secid_first_directory; uint8_t h_unused1[4]; uint32_t h_min_size_standard_stream; cdf_secid_t h_secid_first_sector_in_short_sat; uint32_t h_num_sectors_in_short_sat; cdf_secid_t h_secid_first_sector_in_master_sat; uint32_t h_num_sectors_in_master_sat; cdf_secid_t h_master_sat[436/4]; } cdf_header_t; #define CDF_SEC_SIZE(h) ((size_t)(1 << (h)->h_sec_size_p2)) #define CDF_SEC_POS(h, secid) (CDF_SEC_SIZE(h) + (secid) * CDF_SEC_SIZE(h)) #define CDF_SHORT_SEC_SIZE(h) ((size_t)(1 << (h)->h_short_sec_size_p2)) #define CDF_SHORT_SEC_POS(h, secid) ((secid) * CDF_SHORT_SEC_SIZE(h)) typedef int32_t cdf_dirid_t; #define CDF_DIRID_NULL -1 typedef int64_t cdf_timestamp_t; #define CDF_BASE_YEAR 1601 #define CDF_TIME_PREC 10000000 typedef struct { uint16_t d_name[32]; uint16_t d_namelen; uint8_t d_type; #define CDF_DIR_TYPE_EMPTY 0 #define CDF_DIR_TYPE_USER_STORAGE 1 #define CDF_DIR_TYPE_USER_STREAM 2 #define CDF_DIR_TYPE_LOCKBYTES 3 #define CDF_DIR_TYPE_PROPERTY 4 #define CDF_DIR_TYPE_ROOT_STORAGE 5 uint8_t d_color; #define CDF_DIR_COLOR_READ 0 #define CDF_DIR_COLOR_BLACK 1 cdf_dirid_t d_left_child; cdf_dirid_t d_right_child; cdf_dirid_t d_storage; uint64_t d_storage_uuid[2]; uint32_t d_flags; cdf_timestamp_t d_created; cdf_timestamp_t d_modified; cdf_secid_t d_stream_first_sector; uint32_t d_size; uint32_t d_unused0; } cdf_directory_t; #define CDF_DIRECTORY_SIZE 128 typedef struct { cdf_secid_t *sat_tab; size_t sat_len; } cdf_sat_t; typedef struct { cdf_directory_t *dir_tab; size_t dir_len; } cdf_dir_t; typedef struct { void *sst_tab; size_t sst_len; size_t sst_dirlen; } cdf_stream_t; typedef struct { uint32_t cl_dword; uint16_t cl_word[2]; uint8_t cl_two[2]; uint8_t cl_six[6]; } cdf_classid_t; typedef struct { uint16_t si_byte_order; uint16_t si_zero; uint16_t si_os_version; uint16_t si_os; cdf_classid_t si_class; uint32_t si_count; } cdf_summary_info_header_t; #define CDF_SECTION_DECLARATION_OFFSET 0x1c typedef struct { cdf_classid_t sd_class; uint32_t sd_offset; } cdf_section_declaration_t; typedef struct { uint32_t sh_len; uint32_t sh_properties; } cdf_section_header_t; typedef struct { uint32_t pi_id; uint32_t pi_type; union { uint16_t _pi_u16; int16_t _pi_s16; uint32_t _pi_u32; int32_t _pi_s32; uint64_t _pi_u64; int64_t _pi_s64; cdf_timestamp_t _pi_tp; float _pi_f; double _pi_d; struct { uint32_t s_len; const char *s_buf; } _pi_str; } pi_val; #define pi_u64 pi_val._pi_u64 #define pi_s64 pi_val._pi_s64 #define pi_u32 pi_val._pi_u32 #define pi_s32 pi_val._pi_s32 #define pi_u16 pi_val._pi_u16 #define pi_s16 pi_val._pi_s16 #define pi_f pi_val._pi_f #define pi_d pi_val._pi_d #define pi_tp pi_val._pi_tp #define pi_str pi_val._pi_str } cdf_property_info_t; #define CDF_ROUND(val, by) (((val) + (by) - 1) & ~((by) - 1)) /* Variant type definitions */ #define CDF_EMPTY 0x00000000 #define CDF_NULL 0x00000001 #define CDF_SIGNED16 0x00000002 #define CDF_SIGNED32 0x00000003 #define CDF_FLOAT 0x00000004 #define CDF_DOUBLE 0x00000005 #define CDF_CY 0x00000006 #define CDF_DATE 0x00000007 #define CDF_BSTR 0x00000008 #define CDF_DISPATCH 0x00000009 #define CDF_ERROR 0x0000000a #define CDF_BOOL 0x0000000b #define CDF_VARIANT 0x0000000c #define CDF_UNKNOWN 0x0000000d #define CDF_DECIMAL 0x0000000e #define CDF_SIGNED8 0x00000010 #define CDF_UNSIGNED8 0x00000011 #define CDF_UNSIGNED16 0x00000012 #define CDF_UNSIGNED32 0x00000013 #define CDF_SIGNED64 0x00000014 #define CDF_UNSIGNED64 0x00000015 #define CDF_INT 0x00000016 #define CDF_UINT 0x00000017 #define CDF_VOID 0x00000018 #define CDF_HRESULT 0x00000019 #define CDF_PTR 0x0000001a #define CDF_SAFEARRAY 0x0000001b #define CDF_CARRAY 0x0000001c #define CDF_USERDEFINED 0x0000001d #define CDF_LENGTH32_STRING 0x0000001e #define CDF_LENGTH32_WSTRING 0x0000001f #define CDF_FILETIME 0x00000040 #define CDF_BLOB 0x00000041 #define CDF_STREAM 0x00000042 #define CDF_STORAGE 0x00000043 #define CDF_STREAMED_OBJECT 0x00000044 #define CDF_STORED_OBJECT 0x00000045 #define CDF_BLOB_OBJECT 0x00000046 #define CDF_CLIPBOARD 0x00000047 #define CDF_CLSID 0x00000048 #define CDF_VECTOR 0x00001000 #define CDF_ARRAY 0x00002000 #define CDF_BYREF 0x00004000 #define CDF_RESERVED 0x00008000 #define CDF_ILLEGAL 0x0000ffff #define CDF_ILLEGALMASKED 0x00000fff #define CDF_TYPEMASK 0x00000fff #define CDF_PROPERTY_CODE_PAGE 0x00000001 #define CDF_PROPERTY_TITLE 0x00000002 #define CDF_PROPERTY_SUBJECT 0x00000003 #define CDF_PROPERTY_AUTHOR 0x00000004 #define CDF_PROPERTY_KEYWORDS 0x00000005 #define CDF_PROPERTY_COMMENTS 0x00000006 #define CDF_PROPERTY_TEMPLATE 0x00000007 #define CDF_PROPERTY_LAST_SAVED_BY 0x00000008 #define CDF_PROPERTY_REVISION_NUMBER 0x00000009 #define CDF_PROPERTY_TOTAL_EDITING_TIME 0x0000000a #define CDF_PROPERTY_LAST_PRINTED 0X0000000b #define CDF_PROPERTY_CREATE_TIME 0x0000000c #define CDF_PROPERTY_LAST_SAVED_TIME 0x0000000d #define CDF_PROPERTY_NUMBER_OF_PAGES 0x0000000e #define CDF_PROPERTY_NUMBER_OF_WORDS 0x0000000f #define CDF_PROPERTY_NUMBER_OF_CHARACTERS 0x00000010 #define CDF_PROPERTY_THUMBNAIL 0x00000011 #define CDF_PROPERTY_NAME_OF_APPLICATION 0x00000012 #define CDF_PROPERTY_SECURITY 0x00000013 #define CDF_PROPERTY_LOCALE_ID 0x80000000 typedef struct { int i_fd; const unsigned char *i_buf; size_t i_len; } cdf_info_t; + +typedef struct { + uint16_t ce_namlen; + uint32_t ce_num; + uint64_t ce_timestamp; + uint16_t ce_name[256]; +} cdf_catalog_entry_t; + +typedef struct { + size_t cat_num; + cdf_catalog_entry_t cat_e[0]; +} cdf_catalog_t; + struct timespec; int cdf_timestamp_to_timespec(struct timespec *, cdf_timestamp_t); int cdf_timespec_to_timestamp(cdf_timestamp_t *, const struct timespec *); int cdf_read_header(const cdf_info_t *, cdf_header_t *); void cdf_swap_header(cdf_header_t *); void cdf_unpack_header(cdf_header_t *, char *); void cdf_swap_dir(cdf_directory_t *); void cdf_unpack_dir(cdf_directory_t *, char *); void cdf_swap_class(cdf_classid_t *); ssize_t cdf_read_sector(const cdf_info_t *, void *, size_t, size_t, const cdf_header_t *, cdf_secid_t); ssize_t cdf_read_short_sector(const cdf_stream_t *, void *, size_t, size_t, const cdf_header_t *, cdf_secid_t); int cdf_read_sat(const cdf_info_t *, cdf_header_t *, cdf_sat_t *); size_t cdf_count_chain(const cdf_sat_t *, cdf_secid_t, size_t); int cdf_read_long_sector_chain(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *, cdf_secid_t, size_t, cdf_stream_t *); int cdf_read_short_sector_chain(const cdf_header_t *, const cdf_sat_t *, const cdf_stream_t *, cdf_secid_t, size_t, cdf_stream_t *); int cdf_read_sector_chain(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *, const cdf_sat_t *, const cdf_stream_t *, cdf_secid_t, size_t, cdf_stream_t *); int cdf_read_dir(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *, cdf_dir_t *); int cdf_read_ssat(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *, cdf_sat_t *); int cdf_read_short_stream(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *, const cdf_dir_t *, cdf_stream_t *, const cdf_directory_t **); int cdf_read_property_info(const cdf_stream_t *, const cdf_header_t *, uint32_t, cdf_property_info_t **, size_t *, size_t *); int cdf_read_user_stream(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *, const cdf_sat_t *, const cdf_stream_t *, const cdf_dir_t *, const char *, cdf_stream_t *); +#define cdf_read_catalog(info, header, sat, ssat, stream, dir, scn) \ + cdf_read_user_stream(info, header, sat, ssat, stream, dir, "Catalog", \ + scn) +#define cdf_read_encrypted_package(info, header, sat, ssat, stream, dir, scn) \ + cdf_read_user_stream(info, header, sat, ssat, stream, dir, \ + "EncryptedPackage", scn) int cdf_read_summary_info(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *, const cdf_sat_t *, const cdf_stream_t *, const cdf_dir_t *, cdf_stream_t *); int cdf_unpack_summary_info(const cdf_stream_t *, const cdf_header_t *, cdf_summary_info_header_t *, cdf_property_info_t **, size_t *); +int cdf_unpack_catalog(const cdf_header_t *, const cdf_stream_t *, + cdf_catalog_t **); int cdf_print_classid(char *, size_t, const cdf_classid_t *); int cdf_print_property_name(char *, size_t, uint32_t); int cdf_print_elapsed_time(char *, size_t, cdf_timestamp_t); uint16_t cdf_tole2(uint16_t); uint32_t cdf_tole4(uint32_t); uint64_t cdf_tole8(uint64_t); char *cdf_ctime(const time_t *, char *); +char *cdf_u16tos8(char *, size_t, const uint16_t *); #ifdef CDF_DEBUG void cdf_dump_header(const cdf_header_t *); void cdf_dump_sat(const char *, const cdf_sat_t *, size_t); void cdf_dump(void *, size_t); void cdf_dump_stream(const cdf_header_t *, const cdf_stream_t *); void cdf_dump_dir(const cdf_info_t *, const cdf_header_t *, const cdf_sat_t *, const cdf_sat_t *, const cdf_stream_t *, const cdf_dir_t *); void cdf_dump_property_info(const cdf_property_info_t *, size_t); void cdf_dump_summary_info(const cdf_header_t *, const cdf_stream_t *); +void cdf_dump_catalog(const cdf_header_t *, const cdf_stream_t *); #endif #endif /* _H_CDF_ */ Index: releng/10.1/contrib/file/src/compress.c =================================================================== --- releng/10.1/contrib/file/src/compress.c (revision 284192) +++ releng/10.1/contrib/file/src/compress.c (revision 284193) @@ -1,505 +1,532 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * compress routines: * zmagic() - returns 0 if not recognized, uncompresses and prints * information if recognized * uncompress(method, old, n, newch) - uncompress old into new, * using method, return sizeof new */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: compress.c,v 1.73 2014/01/05 15:55:21 christos Exp $") +FILE_RCSID("@(#)$File: compress.c,v 1.77 2014/12/12 16:33:01 christos Exp $") #endif #include "magic.h" #include #ifdef HAVE_UNISTD_H #include #endif #include #include -#ifndef __MINGW32__ +#include +#if !defined(__MINGW32__) && !defined(WIN32) #include #endif #ifdef HAVE_SYS_WAIT_H #include #endif #if defined(HAVE_SYS_TIME_H) #include #endif #if defined(HAVE_ZLIB_H) && defined(HAVE_LIBZ) #define BUILTIN_DECOMPRESS #include #endif private const struct { const char magic[8]; size_t maglen; const char *argv[3]; int silent; } compr[] = { { "\037\235", 2, { "gzip", "-cdq", NULL }, 1 }, /* compressed */ /* Uncompress can get stuck; so use gzip first if we have it * Idea from Damien Clark, thanks! */ { "\037\235", 2, { "uncompress", "-c", NULL }, 1 }, /* compressed */ { "\037\213", 2, { "gzip", "-cdq", NULL }, 1 }, /* gzipped */ { "\037\236", 2, { "gzip", "-cdq", NULL }, 1 }, /* frozen */ { "\037\240", 2, { "gzip", "-cdq", NULL }, 1 }, /* SCO LZH */ /* the standard pack utilities do not accept standard input */ { "\037\036", 2, { "gzip", "-cdq", NULL }, 0 }, /* packed */ { "PK\3\4", 4, { "gzip", "-cdq", NULL }, 1 }, /* pkzipped, */ /* ...only first file examined */ { "BZh", 3, { "bzip2", "-cd", NULL }, 1 }, /* bzip2-ed */ { "LZIP", 4, { "lzip", "-cdq", NULL }, 1 }, { "\3757zXZ\0",6,{ "xz", "-cd", NULL }, 1 }, /* XZ Utils */ { "LRZI", 4, { "lrzip", "-dqo-", NULL }, 1 }, /* LRZIP */ { "\004\"M\030", 4, { "lz4", "-cd", NULL }, 1 }, /* LZ4 */ }; #define NODATA ((size_t)~0) private ssize_t swrite(int, const void *, size_t); #if HAVE_FORK private size_t ncompr = sizeof(compr) / sizeof(compr[0]); private size_t uncompressbuf(struct magic_set *, int, size_t, const unsigned char *, unsigned char **, size_t); #ifdef BUILTIN_DECOMPRESS private size_t uncompressgzipped(struct magic_set *, const unsigned char *, unsigned char **, size_t); #endif protected int file_zmagic(struct magic_set *ms, int fd, const char *name, const unsigned char *buf, size_t nbytes) { unsigned char *newbuf = NULL; size_t i, nsz; int rv = 0; int mime = ms->flags & MAGIC_MIME; + sig_t osigpipe; if ((ms->flags & MAGIC_COMPRESS) == 0) return 0; + osigpipe = signal(SIGPIPE, SIG_IGN); for (i = 0; i < ncompr; i++) { if (nbytes < compr[i].maglen) continue; if (memcmp(buf, compr[i].magic, compr[i].maglen) == 0 && (nsz = uncompressbuf(ms, fd, i, buf, &newbuf, nbytes)) != NODATA) { ms->flags &= ~MAGIC_COMPRESS; rv = -1; if (file_buffer(ms, -1, name, newbuf, nsz) == -1) goto error; if (mime == MAGIC_MIME || mime == 0) { if (file_printf(ms, mime ? " compressed-encoding=" : " (") == -1) goto error; if (file_buffer(ms, -1, NULL, buf, nbytes) == -1) goto error; if (!mime && file_printf(ms, ")") == -1) goto error; } rv = 1; break; } } error: + (void)signal(SIGPIPE, osigpipe); free(newbuf); ms->flags |= MAGIC_COMPRESS; return rv; } #endif /* * `safe' write for sockets and pipes. */ private ssize_t swrite(int fd, const void *buf, size_t n) { ssize_t rv; size_t rn = n; do switch (rv = write(fd, buf, n)) { case -1: if (errno == EINTR) continue; return -1; default: n -= rv; buf = CAST(const char *, buf) + rv; break; } while (n > 0); return rn; } /* * `safe' read for sockets and pipes. */ protected ssize_t sread(int fd, void *buf, size_t n, int canbepipe __attribute__((__unused__))) { ssize_t rv; #ifdef FIONREAD int t = 0; #endif size_t rn = n; if (fd == STDIN_FILENO) goto nocheck; #ifdef FIONREAD if (canbepipe && (ioctl(fd, FIONREAD, &t) == -1 || t == 0)) { #ifdef FD_ZERO ssize_t cnt; for (cnt = 0;; cnt++) { fd_set check; struct timeval tout = {0, 100 * 1000}; int selrv; FD_ZERO(&check); FD_SET(fd, &check); /* * Avoid soft deadlock: do not read if there * is nothing to read from sockets and pipes. */ selrv = select(fd + 1, &check, NULL, NULL, &tout); if (selrv == -1) { if (errno == EINTR || errno == EAGAIN) continue; } else if (selrv == 0 && cnt >= 5) { return 0; } else break; } #endif (void)ioctl(fd, FIONREAD, &t); } if (t > 0 && (size_t)t < n) { n = t; rn = n; } #endif nocheck: do switch ((rv = read(fd, buf, n))) { case -1: if (errno == EINTR) continue; return -1; case 0: return rn - n; default: n -= rv; buf = ((char *)buf) + rv; break; } while (n > 0); return rn; } protected int file_pipe2file(struct magic_set *ms, int fd, const void *startbuf, size_t nbytes) { char buf[4096]; ssize_t r; int tfd; (void)strlcpy(buf, "/tmp/file.XXXXXX", sizeof buf); #ifndef HAVE_MKSTEMP { char *ptr = mktemp(buf); tfd = open(ptr, O_RDWR|O_TRUNC|O_EXCL|O_CREAT, 0600); r = errno; (void)unlink(ptr); errno = r; } #else { int te; tfd = mkstemp(buf); te = errno; (void)unlink(buf); errno = te; } #endif if (tfd == -1) { file_error(ms, errno, "cannot create temporary file for pipe copy"); return -1; } if (swrite(tfd, startbuf, nbytes) != (ssize_t)nbytes) r = 1; else { while ((r = sread(fd, buf, sizeof(buf), 1)) > 0) if (swrite(tfd, buf, (size_t)r) != r) break; } switch (r) { case -1: file_error(ms, errno, "error copying from pipe to temp file"); return -1; case 0: break; default: file_error(ms, errno, "error while writing to temp file"); return -1; } /* * We duplicate the file descriptor, because fclose on a * tmpfile will delete the file, but any open descriptors * can still access the phantom inode. */ if ((fd = dup2(tfd, fd)) == -1) { file_error(ms, errno, "could not dup descriptor for temp file"); return -1; } (void)close(tfd); if (lseek(fd, (off_t)0, SEEK_SET) == (off_t)-1) { file_badseek(ms); return -1; } return fd; } #if HAVE_FORK #ifdef BUILTIN_DECOMPRESS #define FHCRC (1 << 1) #define FEXTRA (1 << 2) #define FNAME (1 << 3) #define FCOMMENT (1 << 4) private size_t uncompressgzipped(struct magic_set *ms, const unsigned char *old, unsigned char **newch, size_t n) { unsigned char flg = old[3]; size_t data_start = 10; z_stream z; int rc; if (flg & FEXTRA) { if (data_start+1 >= n) return 0; data_start += 2 + old[data_start] + old[data_start + 1] * 256; } if (flg & FNAME) { while(data_start < n && old[data_start]) data_start++; data_start++; } if(flg & FCOMMENT) { while(data_start < n && old[data_start]) data_start++; data_start++; } if(flg & FHCRC) data_start += 2; if (data_start >= n) return 0; if ((*newch = CAST(unsigned char *, malloc(HOWMANY + 1))) == NULL) { return 0; } /* XXX: const castaway, via strchr */ z.next_in = (Bytef *)strchr((const char *)old + data_start, old[data_start]); z.avail_in = CAST(uint32_t, (n - data_start)); z.next_out = *newch; z.avail_out = HOWMANY; z.zalloc = Z_NULL; z.zfree = Z_NULL; z.opaque = Z_NULL; /* LINTED bug in header macro */ rc = inflateInit2(&z, -15); if (rc != Z_OK) { file_error(ms, 0, "zlib: %s", z.msg); return 0; } rc = inflate(&z, Z_SYNC_FLUSH); if (rc != Z_OK && rc != Z_STREAM_END) { file_error(ms, 0, "zlib: %s", z.msg); return 0; } n = (size_t)z.total_out; (void)inflateEnd(&z); /* let's keep the nul-terminate tradition */ (*newch)[n] = '\0'; return n; } #endif private size_t uncompressbuf(struct magic_set *ms, int fd, size_t method, const unsigned char *old, unsigned char **newch, size_t n) { int fdin[2], fdout[2]; + int status; ssize_t r; pid_t pid; #ifdef BUILTIN_DECOMPRESS /* FIXME: This doesn't cope with bzip2 */ if (method == 2) return uncompressgzipped(ms, old, newch, n); #endif (void)fflush(stdout); (void)fflush(stderr); if ((fd != -1 && pipe(fdin) == -1) || pipe(fdout) == -1) { file_error(ms, errno, "cannot create pipe"); return NODATA; } switch (pid = fork()) { case 0: /* child */ (void) close(0); if (fd != -1) { if (dup(fd) == -1) _exit(1); (void) lseek(0, (off_t)0, SEEK_SET); } else { if (dup(fdin[0]) == -1) _exit(1); (void) close(fdin[0]); (void) close(fdin[1]); } (void) close(1); if (dup(fdout[1]) == -1) _exit(1); (void) close(fdout[0]); (void) close(fdout[1]); #ifndef DEBUG if (compr[method].silent) (void)close(2); #endif (void)execvp(compr[method].argv[0], (char *const *)(intptr_t)compr[method].argv); #ifdef DEBUG (void)fprintf(stderr, "exec `%s' failed (%s)\n", compr[method].argv[0], strerror(errno)); #endif exit(1); /*NOTREACHED*/ case -1: file_error(ms, errno, "could not fork"); return NODATA; default: /* parent */ (void) close(fdout[1]); if (fd == -1) { (void) close(fdin[0]); /* * fork again, to avoid blocking because both * pipes filled */ switch (fork()) { case 0: /* child */ (void)close(fdout[0]); if (swrite(fdin[1], old, n) != (ssize_t)n) { #ifdef DEBUG (void)fprintf(stderr, "Write failed (%s)\n", strerror(errno)); #endif exit(1); } exit(0); /*NOTREACHED*/ case -1: #ifdef DEBUG (void)fprintf(stderr, "Fork failed (%s)\n", strerror(errno)); #endif exit(1); /*NOTREACHED*/ default: /* parent */ - break; + if (wait(&status) == -1) { +#ifdef DEBUG + (void)fprintf(stderr, + "Wait failed (%s)\n", + strerror(errno)); +#endif + exit(1); + } + exit(WIFEXITED(status) ? + WEXITSTATUS(status) : 1); + /*NOTREACHED*/ } (void) close(fdin[1]); fdin[1] = -1; } if ((*newch = (unsigned char *) malloc(HOWMANY + 1)) == NULL) { #ifdef DEBUG (void)fprintf(stderr, "Malloc failed (%s)\n", strerror(errno)); #endif - n = 0; + n = NODATA; goto err; } if ((r = sread(fdout[0], *newch, HOWMANY, 0)) <= 0) { #ifdef DEBUG (void)fprintf(stderr, "Read failed (%s)\n", strerror(errno)); #endif free(*newch); - n = 0; + n = NODATA; *newch = NULL; goto err; } else { n = r; } /* NUL terminate, as every buffer is handled here. */ (*newch)[n] = '\0'; err: if (fdin[1] != -1) (void) close(fdin[1]); (void) close(fdout[0]); -#ifdef WNOHANG - while (waitpid(pid, NULL, WNOHANG) != -1) - continue; -#else - (void)wait(NULL); + if (wait(&status) == -1) { +#ifdef DEBUG + (void)fprintf(stderr, "Wait failed (%s)\n", + strerror(errno)); #endif + n = NODATA; + } else if (!WIFEXITED(status)) { +#ifdef DEBUG + (void)fprintf(stderr, "Child not exited (0x%x)\n", + status); +#endif + } else if (WEXITSTATUS(status) != 0) { +#ifdef DEBUG + (void)fprintf(stderr, "Child exited (0x%d)\n", + WEXITSTATUS(status)); +#endif + } + (void) close(fdin[0]); return n; } } #endif Index: releng/10.1/contrib/file/src/elfclass.h =================================================================== --- releng/10.1/contrib/file/src/elfclass.h (revision 284192) +++ releng/10.1/contrib/file/src/elfclass.h (revision 284193) @@ -1,78 +1,82 @@ /* * Copyright (c) Christos Zoulas 2008. * All Rights Reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ if (nbytes <= sizeof(elfhdr)) return 0; u.l = 1; (void)memcpy(&elfhdr, buf, sizeof elfhdr); swap = (u.c[sizeof(int32_t) - 1] + 1) != elfhdr.e_ident[EI_DATA]; type = elf_getu16(swap, elfhdr.e_type); + notecount = ms->elf_notes_max; switch (type) { #ifdef ELFCORE case ET_CORE: phnum = elf_getu16(swap, elfhdr.e_phnum); - if (phnum > MAX_PHNUM) - return toomany(ms, "program", phnum); + if (phnum > ms->elf_phnum_max) + return toomany(ms, "program headers", phnum); flags |= FLAGS_IS_CORE; if (dophn_core(ms, clazz, swap, fd, (off_t)elf_getu(swap, elfhdr.e_phoff), phnum, (size_t)elf_getu16(swap, elfhdr.e_phentsize), - fsize, &flags) == -1) + fsize, &flags, ¬ecount) == -1) return -1; break; #endif case ET_EXEC: case ET_DYN: phnum = elf_getu16(swap, elfhdr.e_phnum); - if (phnum > MAX_PHNUM) + if (phnum > ms->elf_phnum_max) return toomany(ms, "program", phnum); shnum = elf_getu16(swap, elfhdr.e_shnum); - if (shnum > MAX_SHNUM) + if (shnum > ms->elf_shnum_max) return toomany(ms, "section", shnum); if (dophn_exec(ms, clazz, swap, fd, (off_t)elf_getu(swap, elfhdr.e_phoff), phnum, (size_t)elf_getu16(swap, elfhdr.e_phentsize), - fsize, &flags, shnum) == -1) + fsize, shnum, &flags, ¬ecount) == -1) return -1; /*FALLTHROUGH*/ case ET_REL: shnum = elf_getu16(swap, elfhdr.e_shnum); - if (shnum > MAX_SHNUM) - return toomany(ms, "section", shnum); + if (shnum > ms->elf_shnum_max) + return toomany(ms, "section headers", shnum); if (doshn(ms, clazz, swap, fd, (off_t)elf_getu(swap, elfhdr.e_shoff), shnum, (size_t)elf_getu16(swap, elfhdr.e_shentsize), - fsize, &flags, elf_getu16(swap, elfhdr.e_machine), - (int)elf_getu16(swap, elfhdr.e_shstrndx)) == -1) + fsize, elf_getu16(swap, elfhdr.e_machine), + (int)elf_getu16(swap, elfhdr.e_shstrndx), + &flags, ¬ecount) == -1) return -1; break; default: break; } + if (notecount == 0) + return toomany(ms, "notes", ms->elf_notes_max); return 1; Index: releng/10.1/contrib/file/src/encoding.c =================================================================== --- releng/10.1/contrib/file/src/encoding.c (revision 284192) +++ releng/10.1/contrib/file/src/encoding.c (revision 284193) @@ -1,504 +1,503 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * Encoding -- determine the character encoding of a text file. * * Joerg Wunsch wrote the original support for 8-bit * international characters. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: encoding.c,v 1.9 2013/11/19 20:45:50 christos Exp $") +FILE_RCSID("@(#)$File: encoding.c,v 1.10 2014/09/11 12:08:52 christos Exp $") #endif /* lint */ #include "magic.h" #include #include #include private int looks_ascii(const unsigned char *, size_t, unichar *, size_t *); private int looks_utf8_with_BOM(const unsigned char *, size_t, unichar *, size_t *); private int looks_ucs16(const unsigned char *, size_t, unichar *, size_t *); private int looks_latin1(const unsigned char *, size_t, unichar *, size_t *); private int looks_extended(const unsigned char *, size_t, unichar *, size_t *); private void from_ebcdic(const unsigned char *, size_t, unsigned char *); #ifdef DEBUG_ENCODING #define DPRINTF(a) printf a #else #define DPRINTF(a) #endif /* * Try to determine whether text is in some character code we can * identify. Each of these tests, if it succeeds, will leave * the text converted into one-unichar-per-character Unicode in * ubuf, and the number of characters converted in ulen. */ protected int file_encoding(struct magic_set *ms, const unsigned char *buf, size_t nbytes, unichar **ubuf, size_t *ulen, const char **code, const char **code_mime, const char **type) { size_t mlen; int rv = 1, ucs_type; unsigned char *nbuf = NULL; *type = "text"; *ulen = 0; *code = "unknown"; *code_mime = "binary"; mlen = (nbytes + 1) * sizeof((*ubuf)[0]); if ((*ubuf = CAST(unichar *, calloc((size_t)1, mlen))) == NULL) { file_oomem(ms, mlen); goto done; } mlen = (nbytes + 1) * sizeof(nbuf[0]); if ((nbuf = CAST(unsigned char *, calloc((size_t)1, mlen))) == NULL) { file_oomem(ms, mlen); goto done; } if (looks_ascii(buf, nbytes, *ubuf, ulen)) { DPRINTF(("ascii %" SIZE_T_FORMAT "u\n", *ulen)); *code = "ASCII"; *code_mime = "us-ascii"; } else if (looks_utf8_with_BOM(buf, nbytes, *ubuf, ulen) > 0) { DPRINTF(("utf8/bom %" SIZE_T_FORMAT "u\n", *ulen)); *code = "UTF-8 Unicode (with BOM)"; *code_mime = "utf-8"; } else if (file_looks_utf8(buf, nbytes, *ubuf, ulen) > 1) { DPRINTF(("utf8 %" SIZE_T_FORMAT "u\n", *ulen)); - *code = "UTF-8 Unicode (with BOM)"; *code = "UTF-8 Unicode"; *code_mime = "utf-8"; } else if ((ucs_type = looks_ucs16(buf, nbytes, *ubuf, ulen)) != 0) { if (ucs_type == 1) { *code = "Little-endian UTF-16 Unicode"; *code_mime = "utf-16le"; } else { *code = "Big-endian UTF-16 Unicode"; *code_mime = "utf-16be"; } DPRINTF(("ucs16 %" SIZE_T_FORMAT "u\n", *ulen)); } else if (looks_latin1(buf, nbytes, *ubuf, ulen)) { DPRINTF(("latin1 %" SIZE_T_FORMAT "u\n", *ulen)); *code = "ISO-8859"; *code_mime = "iso-8859-1"; } else if (looks_extended(buf, nbytes, *ubuf, ulen)) { DPRINTF(("extended %" SIZE_T_FORMAT "u\n", *ulen)); *code = "Non-ISO extended-ASCII"; *code_mime = "unknown-8bit"; } else { from_ebcdic(buf, nbytes, nbuf); if (looks_ascii(nbuf, nbytes, *ubuf, ulen)) { DPRINTF(("ebcdic %" SIZE_T_FORMAT "u\n", *ulen)); *code = "EBCDIC"; *code_mime = "ebcdic"; } else if (looks_latin1(nbuf, nbytes, *ubuf, ulen)) { DPRINTF(("ebcdic/international %" SIZE_T_FORMAT "u\n", *ulen)); *code = "International EBCDIC"; *code_mime = "ebcdic"; } else { /* Doesn't look like text at all */ DPRINTF(("binary\n")); rv = 0; *type = "binary"; } } done: free(nbuf); return rv; } /* * This table reflects a particular philosophy about what constitutes * "text," and there is room for disagreement about it. * * Version 3.31 of the file command considered a file to be ASCII if * each of its characters was approved by either the isascii() or * isalpha() function. On most systems, this would mean that any * file consisting only of characters in the range 0x00 ... 0x7F * would be called ASCII text, but many systems might reasonably * consider some characters outside this range to be alphabetic, * so the file command would call such characters ASCII. It might * have been more accurate to call this "considered textual on the * local system" than "ASCII." * * It considered a file to be "International language text" if each * of its characters was either an ASCII printing character (according * to the real ASCII standard, not the above test), a character in * the range 0x80 ... 0xFF, or one of the following control characters: * backspace, tab, line feed, vertical tab, form feed, carriage return, * escape. No attempt was made to determine the language in which files * of this type were written. * * * The table below considers a file to be ASCII if all of its characters * are either ASCII printing characters (again, according to the X3.4 * standard, not isascii()) or any of the following controls: bell, * backspace, tab, line feed, form feed, carriage return, esc, nextline. * * I include bell because some programs (particularly shell scripts) * use it literally, even though it is rare in normal text. I exclude * vertical tab because it never seems to be used in real text. I also * include, with hesitation, the X3.64/ECMA-43 control nextline (0x85), * because that's what the dd EBCDIC->ASCII table maps the EBCDIC newline * character to. It might be more appropriate to include it in the 8859 * set instead of the ASCII set, but it's got to be included in *something* * we recognize or EBCDIC files aren't going to be considered textual. * Some old Unix source files use SO/SI (^N/^O) to shift between Greek * and Latin characters, so these should possibly be allowed. But they * make a real mess on VT100-style displays if they're not paired properly, * so we are probably better off not calling them text. * * A file is considered to be ISO-8859 text if its characters are all * either ASCII, according to the above definition, or printing characters * from the ISO-8859 8-bit extension, characters 0xA0 ... 0xFF. * * Finally, a file is considered to be international text from some other * character code if its characters are all either ISO-8859 (according to * the above definition) or characters in the range 0x80 ... 0x9F, which * ISO-8859 considers to be control characters but the IBM PC and Macintosh * consider to be printing characters. */ #define F 0 /* character never appears in text */ #define T 1 /* character appears in plain ASCII text */ #define I 2 /* character appears in ISO-8859 text */ #define X 3 /* character appears in non-ISO extended ASCII (Mac, IBM PC) */ private char text_chars[256] = { /* BEL BS HT LF FF CR */ F, F, F, F, F, F, F, T, T, T, T, F, T, T, F, F, /* 0x0X */ /* ESC */ F, F, F, F, F, F, F, F, F, F, F, T, F, F, F, F, /* 0x1X */ T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, /* 0x2X */ T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, /* 0x3X */ T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, /* 0x4X */ T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, /* 0x5X */ T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, /* 0x6X */ T, T, T, T, T, T, T, T, T, T, T, T, T, T, T, F, /* 0x7X */ /* NEL */ X, X, X, X, X, T, X, X, X, X, X, X, X, X, X, X, /* 0x8X */ X, X, X, X, X, X, X, X, X, X, X, X, X, X, X, X, /* 0x9X */ I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, /* 0xaX */ I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, /* 0xbX */ I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, /* 0xcX */ I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, /* 0xdX */ I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, /* 0xeX */ I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I /* 0xfX */ }; private int looks_ascii(const unsigned char *buf, size_t nbytes, unichar *ubuf, size_t *ulen) { size_t i; *ulen = 0; for (i = 0; i < nbytes; i++) { int t = text_chars[buf[i]]; if (t != T) return 0; ubuf[(*ulen)++] = buf[i]; } return 1; } private int looks_latin1(const unsigned char *buf, size_t nbytes, unichar *ubuf, size_t *ulen) { size_t i; *ulen = 0; for (i = 0; i < nbytes; i++) { int t = text_chars[buf[i]]; if (t != T && t != I) return 0; ubuf[(*ulen)++] = buf[i]; } return 1; } private int looks_extended(const unsigned char *buf, size_t nbytes, unichar *ubuf, size_t *ulen) { size_t i; *ulen = 0; for (i = 0; i < nbytes; i++) { int t = text_chars[buf[i]]; if (t != T && t != I && t != X) return 0; ubuf[(*ulen)++] = buf[i]; } return 1; } /* * Decide whether some text looks like UTF-8. Returns: * * -1: invalid UTF-8 * 0: uses odd control characters, so doesn't look like text * 1: 7-bit text * 2: definitely UTF-8 text (valid high-bit set bytes) * * If ubuf is non-NULL on entry, text is decoded into ubuf, *ulen; * ubuf must be big enough! */ protected int file_looks_utf8(const unsigned char *buf, size_t nbytes, unichar *ubuf, size_t *ulen) { size_t i; int n; unichar c; int gotone = 0, ctrl = 0; if (ubuf) *ulen = 0; for (i = 0; i < nbytes; i++) { if ((buf[i] & 0x80) == 0) { /* 0xxxxxxx is plain ASCII */ /* * Even if the whole file is valid UTF-8 sequences, * still reject it if it uses weird control characters. */ if (text_chars[buf[i]] != T) ctrl = 1; if (ubuf) ubuf[(*ulen)++] = buf[i]; } else if ((buf[i] & 0x40) == 0) { /* 10xxxxxx never 1st byte */ return -1; } else { /* 11xxxxxx begins UTF-8 */ int following; if ((buf[i] & 0x20) == 0) { /* 110xxxxx */ c = buf[i] & 0x1f; following = 1; } else if ((buf[i] & 0x10) == 0) { /* 1110xxxx */ c = buf[i] & 0x0f; following = 2; } else if ((buf[i] & 0x08) == 0) { /* 11110xxx */ c = buf[i] & 0x07; following = 3; } else if ((buf[i] & 0x04) == 0) { /* 111110xx */ c = buf[i] & 0x03; following = 4; } else if ((buf[i] & 0x02) == 0) { /* 1111110x */ c = buf[i] & 0x01; following = 5; } else return -1; for (n = 0; n < following; n++) { i++; if (i >= nbytes) goto done; if ((buf[i] & 0x80) == 0 || (buf[i] & 0x40)) return -1; c = (c << 6) + (buf[i] & 0x3f); } if (ubuf) ubuf[(*ulen)++] = c; gotone = 1; } } done: return ctrl ? 0 : (gotone ? 2 : 1); } /* * Decide whether some text looks like UTF-8 with BOM. If there is no * BOM, return -1; otherwise return the result of looks_utf8 on the * rest of the text. */ private int looks_utf8_with_BOM(const unsigned char *buf, size_t nbytes, unichar *ubuf, size_t *ulen) { if (nbytes > 3 && buf[0] == 0xef && buf[1] == 0xbb && buf[2] == 0xbf) return file_looks_utf8(buf + 3, nbytes - 3, ubuf, ulen); else return -1; } private int looks_ucs16(const unsigned char *buf, size_t nbytes, unichar *ubuf, size_t *ulen) { int bigend; size_t i; if (nbytes < 2) return 0; if (buf[0] == 0xff && buf[1] == 0xfe) bigend = 0; else if (buf[0] == 0xfe && buf[1] == 0xff) bigend = 1; else return 0; *ulen = 0; for (i = 2; i + 1 < nbytes; i += 2) { /* XXX fix to properly handle chars > 65536 */ if (bigend) ubuf[(*ulen)++] = buf[i + 1] + 256 * buf[i]; else ubuf[(*ulen)++] = buf[i] + 256 * buf[i + 1]; if (ubuf[*ulen - 1] == 0xfffe) return 0; if (ubuf[*ulen - 1] < 128 && text_chars[(size_t)ubuf[*ulen - 1]] != T) return 0; } return 1 + bigend; } #undef F #undef T #undef I #undef X /* * This table maps each EBCDIC character to an (8-bit extended) ASCII * character, as specified in the rationale for the dd(1) command in * draft 11.2 (September, 1991) of the POSIX P1003.2 standard. * * Unfortunately it does not seem to correspond exactly to any of the * five variants of EBCDIC documented in IBM's _Enterprise Systems * Architecture/390: Principles of Operation_, SA22-7201-06, Seventh * Edition, July, 1999, pp. I-1 - I-4. * * Fortunately, though, all versions of EBCDIC, including this one, agree * on most of the printing characters that also appear in (7-bit) ASCII. * Of these, only '|', '!', '~', '^', '[', and ']' are in question at all. * * Fortunately too, there is general agreement that codes 0x00 through * 0x3F represent control characters, 0x41 a nonbreaking space, and the * remainder printing characters. * * This is sufficient to allow us to identify EBCDIC text and to distinguish * between old-style and internationalized examples of text. */ private unsigned char ebcdic_to_ascii[] = { 0, 1, 2, 3, 156, 9, 134, 127, 151, 141, 142, 11, 12, 13, 14, 15, 16, 17, 18, 19, 157, 133, 8, 135, 24, 25, 146, 143, 28, 29, 30, 31, 128, 129, 130, 131, 132, 10, 23, 27, 136, 137, 138, 139, 140, 5, 6, 7, 144, 145, 22, 147, 148, 149, 150, 4, 152, 153, 154, 155, 20, 21, 158, 26, ' ', 160, 161, 162, 163, 164, 165, 166, 167, 168, 213, '.', '<', '(', '+', '|', '&', 169, 170, 171, 172, 173, 174, 175, 176, 177, '!', '$', '*', ')', ';', '~', '-', '/', 178, 179, 180, 181, 182, 183, 184, 185, 203, ',', '%', '_', '>', '?', 186, 187, 188, 189, 190, 191, 192, 193, 194, '`', ':', '#', '@', '\'','=', '"', 195, 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 196, 197, 198, 199, 200, 201, 202, 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', '^', 204, 205, 206, 207, 208, 209, 229, 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 210, 211, 212, '[', 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, ']', 230, 231, '{', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 232, 233, 234, 235, 236, 237, '}', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 238, 239, 240, 241, 242, 243, '\\',159, 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 244, 245, 246, 247, 248, 249, '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 250, 251, 252, 253, 254, 255 }; #ifdef notdef /* * The following EBCDIC-to-ASCII table may relate more closely to reality, * or at least to modern reality. It comes from * * http://ftp.s390.ibm.com/products/oe/bpxqp9.html * * and maps the characters of EBCDIC code page 1047 (the code used for * Unix-derived software on IBM's 390 systems) to the corresponding * characters from ISO 8859-1. * * If this table is used instead of the above one, some of the special * cases for the NEL character can be taken out of the code. */ private unsigned char ebcdic_1047_to_8859[] = { 0x00,0x01,0x02,0x03,0x9C,0x09,0x86,0x7F,0x97,0x8D,0x8E,0x0B,0x0C,0x0D,0x0E,0x0F, 0x10,0x11,0x12,0x13,0x9D,0x0A,0x08,0x87,0x18,0x19,0x92,0x8F,0x1C,0x1D,0x1E,0x1F, 0x80,0x81,0x82,0x83,0x84,0x85,0x17,0x1B,0x88,0x89,0x8A,0x8B,0x8C,0x05,0x06,0x07, 0x90,0x91,0x16,0x93,0x94,0x95,0x96,0x04,0x98,0x99,0x9A,0x9B,0x14,0x15,0x9E,0x1A, 0x20,0xA0,0xE2,0xE4,0xE0,0xE1,0xE3,0xE5,0xE7,0xF1,0xA2,0x2E,0x3C,0x28,0x2B,0x7C, 0x26,0xE9,0xEA,0xEB,0xE8,0xED,0xEE,0xEF,0xEC,0xDF,0x21,0x24,0x2A,0x29,0x3B,0x5E, 0x2D,0x2F,0xC2,0xC4,0xC0,0xC1,0xC3,0xC5,0xC7,0xD1,0xA6,0x2C,0x25,0x5F,0x3E,0x3F, 0xF8,0xC9,0xCA,0xCB,0xC8,0xCD,0xCE,0xCF,0xCC,0x60,0x3A,0x23,0x40,0x27,0x3D,0x22, 0xD8,0x61,0x62,0x63,0x64,0x65,0x66,0x67,0x68,0x69,0xAB,0xBB,0xF0,0xFD,0xFE,0xB1, 0xB0,0x6A,0x6B,0x6C,0x6D,0x6E,0x6F,0x70,0x71,0x72,0xAA,0xBA,0xE6,0xB8,0xC6,0xA4, 0xB5,0x7E,0x73,0x74,0x75,0x76,0x77,0x78,0x79,0x7A,0xA1,0xBF,0xD0,0x5B,0xDE,0xAE, 0xAC,0xA3,0xA5,0xB7,0xA9,0xA7,0xB6,0xBC,0xBD,0xBE,0xDD,0xA8,0xAF,0x5D,0xB4,0xD7, 0x7B,0x41,0x42,0x43,0x44,0x45,0x46,0x47,0x48,0x49,0xAD,0xF4,0xF6,0xF2,0xF3,0xF5, 0x7D,0x4A,0x4B,0x4C,0x4D,0x4E,0x4F,0x50,0x51,0x52,0xB9,0xFB,0xFC,0xF9,0xFA,0xFF, 0x5C,0xF7,0x53,0x54,0x55,0x56,0x57,0x58,0x59,0x5A,0xB2,0xD4,0xD6,0xD2,0xD3,0xD5, 0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0xB3,0xDB,0xDC,0xD9,0xDA,0x9F }; #endif /* * Copy buf[0 ... nbytes-1] into out[], translating EBCDIC to ASCII. */ private void from_ebcdic(const unsigned char *buf, size_t nbytes, unsigned char *out) { size_t i; for (i = 0; i < nbytes; i++) { out[i] = ebcdic_to_ascii[buf[i]]; } } Index: releng/10.1/contrib/file/src/file.c =================================================================== --- releng/10.1/contrib/file/src/file.c (revision 284192) +++ releng/10.1/contrib/file/src/file.c (revision 284193) @@ -1,540 +1,594 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * file - find type of a file or files - main program. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: file.c,v 1.153 2014/02/11 15:41:04 christos Exp $") +FILE_RCSID("@(#)$File: file.c,v 1.160 2014/12/16 23:18:40 christos Exp $") #endif /* lint */ #include "magic.h" #include #include #include #ifdef RESTORE_TIME # if (__COHERENT__ >= 0x420) # include # else # ifdef USE_UTIMES # include # else # include # endif # endif #endif #ifdef HAVE_UNISTD_H #include /* for read() */ #endif -#ifdef HAVE_LOCALE_H -#include -#endif #ifdef HAVE_WCHAR_H #include #endif #if defined(HAVE_GETOPT_H) && defined(HAVE_STRUCT_OPTION) #include #ifndef HAVE_GETOPT_LONG int getopt_long(int argc, char * const *argv, const char *optstring, const struct option *longopts, int *longindex); #endif #else #include "mygetopt.h" #endif #ifdef S_IFLNK #define FILE_FLAGS "-bcEhikLlNnprsvz0" #else #define FILE_FLAGS "-bcEiklNnprsvz0" #endif # define USAGE \ "Usage: %s [" FILE_FLAGS \ "] [--apple] [--mime-encoding] [--mime-type]\n" \ " [-e testname] [-F separator] [-f namefile] [-m magicfiles] " \ "file ...\n" \ " %s -C [-m magicfiles]\n" \ " %s [--help]\n" private int /* Global command-line options */ bflag = 0, /* brief output format */ nopad = 0, /* Don't pad output */ nobuffer = 0, /* Do not buffer stdout */ nulsep = 0; /* Append '\0' to the separator */ private const char *separator = ":"; /* Default field separator */ private const struct option long_options[] = { #define OPT(shortname, longname, opt, doc) \ {longname, opt, NULL, shortname}, #define OPT_LONGONLY(longname, opt, doc) \ {longname, opt, NULL, 0}, #include "file_opts.h" #undef OPT #undef OPT_LONGONLY {0, 0, NULL, 0} }; -#define OPTSTRING "bcCde:Ef:F:hiklLm:nNprsvz0" +#define OPTSTRING "bcCde:Ef:F:hiklLm:nNpP:rsvz0" private const struct { const char *name; int value; } nv[] = { { "apptype", MAGIC_NO_CHECK_APPTYPE }, { "ascii", MAGIC_NO_CHECK_ASCII }, { "cdf", MAGIC_NO_CHECK_CDF }, { "compress", MAGIC_NO_CHECK_COMPRESS }, { "elf", MAGIC_NO_CHECK_ELF }, { "encoding", MAGIC_NO_CHECK_ENCODING }, { "soft", MAGIC_NO_CHECK_SOFT }, { "tar", MAGIC_NO_CHECK_TAR }, { "text", MAGIC_NO_CHECK_TEXT }, /* synonym for ascii */ { "tokens", MAGIC_NO_CHECK_TOKENS }, /* OBSOLETE: ignored for backwards compatibility */ }; +private struct { + const char *name; + int tag; + size_t value; +} pm[] = { + { "indir", MAGIC_PARAM_INDIR_MAX, 0 }, + { "name", MAGIC_PARAM_NAME_MAX, 0 }, + { "elf_phnum", MAGIC_PARAM_ELF_PHNUM_MAX, 0 }, + { "elf_shnum", MAGIC_PARAM_ELF_SHNUM_MAX, 0 }, + { "elf_notes", MAGIC_PARAM_ELF_NOTES_MAX, 0 }, +}; + private char *progname; /* used throughout */ private void usage(void); private void docprint(const char *); private void help(void); private int unwrap(struct magic_set *, const char *); private int process(struct magic_set *ms, const char *, int); private struct magic_set *load(const char *, int); +private void setparam(const char *); +private void applyparam(magic_t); /* * main - parse arguments and handle options */ int main(int argc, char *argv[]) { int c; size_t i; int action = 0, didsomefiles = 0, errflg = 0; int flags = 0, e = 0; struct magic_set *magic = NULL; int longindex; const char *magicfile = NULL; /* where the magic is */ /* makes islower etc work for other langs */ +#ifdef HAVE_SETLOCALE (void)setlocale(LC_CTYPE, ""); +#endif #ifdef __EMX__ /* sh-like wildcard expansion! Shouldn't hurt at least ... */ _wildcard(&argc, &argv); #endif if ((progname = strrchr(argv[0], '/')) != NULL) progname++; else progname = argv[0]; #ifdef S_IFLNK flags |= getenv("POSIXLY_CORRECT") ? MAGIC_SYMLINK : 0; #endif while ((c = getopt_long(argc, argv, OPTSTRING, long_options, &longindex)) != -1) switch (c) { case 0 : switch (longindex) { case 0: help(); break; case 10: flags |= MAGIC_APPLE; break; case 11: flags |= MAGIC_MIME_TYPE; break; case 12: flags |= MAGIC_MIME_ENCODING; break; } break; case '0': nulsep = 1; break; case 'b': bflag++; break; case 'c': action = FILE_CHECK; break; case 'C': action = FILE_COMPILE; break; case 'd': flags |= MAGIC_DEBUG|MAGIC_CHECK; break; case 'E': flags |= MAGIC_ERROR; break; case 'e': for (i = 0; i < sizeof(nv) / sizeof(nv[0]); i++) if (strcmp(nv[i].name, optarg) == 0) break; if (i == sizeof(nv) / sizeof(nv[0])) errflg++; else flags |= nv[i].value; break; case 'f': if(action) usage(); if (magic == NULL) if ((magic = load(magicfile, flags)) == NULL) return 1; e |= unwrap(magic, optarg); ++didsomefiles; break; case 'F': separator = optarg; break; case 'i': flags |= MAGIC_MIME; break; case 'k': flags |= MAGIC_CONTINUE; break; case 'l': action = FILE_LIST; break; case 'm': magicfile = optarg; break; case 'n': ++nobuffer; break; case 'N': ++nopad; break; #if defined(HAVE_UTIME) || defined(HAVE_UTIMES) case 'p': flags |= MAGIC_PRESERVE_ATIME; break; #endif + case 'P': + setparam(optarg); + break; case 'r': flags |= MAGIC_RAW; break; + break; case 's': flags |= MAGIC_DEVICES; break; case 'v': if (magicfile == NULL) magicfile = magic_getpath(magicfile, action); (void)fprintf(stdout, "%s-%s\n", progname, VERSION); (void)fprintf(stdout, "magic file from %s\n", magicfile); return 0; case 'z': flags |= MAGIC_COMPRESS; break; #ifdef S_IFLNK case 'L': flags |= MAGIC_SYMLINK; break; case 'h': flags &= ~MAGIC_SYMLINK; break; #endif case '?': default: errflg++; break; } if (errflg) { usage(); } if (e) return e; if (MAGIC_VERSION != magic_version()) (void)fprintf(stderr, "%s: compiled magic version [%d] " "does not match with shared library magic version [%d]\n", progname, MAGIC_VERSION, magic_version()); switch(action) { case FILE_CHECK: case FILE_COMPILE: case FILE_LIST: /* * Don't try to check/compile ~/.magic unless we explicitly * ask for it. */ magic = magic_open(flags|MAGIC_CHECK); if (magic == NULL) { (void)fprintf(stderr, "%s: %s\n", progname, strerror(errno)); return 1; } + + switch(action) { case FILE_CHECK: c = magic_check(magic, magicfile); break; case FILE_COMPILE: c = magic_compile(magic, magicfile); break; case FILE_LIST: c = magic_list(magic, magicfile); break; default: abort(); } if (c == -1) { (void)fprintf(stderr, "%s: %s\n", progname, magic_error(magic)); return 1; } return 0; default: if (magic == NULL) if ((magic = load(magicfile, flags)) == NULL) return 1; - break; + applyparam(magic); } if (optind == argc) { if (!didsomefiles) usage(); } else { size_t j, wid, nw; for (wid = 0, j = (size_t)optind; j < (size_t)argc; j++) { nw = file_mbswidth(argv[j]); if (nw > wid) wid = nw; } /* * If bflag is only set twice, set it depending on * number of files [this is undocumented, and subject to change] */ if (bflag == 2) { bflag = optind >= argc - 1; } for (; optind < argc; optind++) e |= process(magic, argv[optind], wid); } if (magic) magic_close(magic); return e; } +private void +applyparam(magic_t magic) +{ + size_t i; + + for (i = 0; i < __arraycount(pm); i++) { + if (pm[i].value == 0) + continue; + if (magic_setparam(magic, pm[i].tag, &pm[i].value) == -1) { + (void)fprintf(stderr, "%s: Can't set %s %s\n", progname, + pm[i].name, strerror(errno)); + exit(1); + } + } +} + +private void +setparam(const char *p) +{ + size_t i; + char *s; + + if ((s = strchr(p, '=')) == NULL) + goto badparm; + + for (i = 0; i < __arraycount(pm); i++) { + if (strncmp(p, pm[i].name, s - p) != 0) + continue; + pm[i].value = atoi(s + 1); + return; + } +badparm: + (void)fprintf(stderr, "%s: Unknown param %s\n", progname, p); + exit(1); +} private struct magic_set * /*ARGSUSED*/ load(const char *magicfile, int flags) { struct magic_set *magic = magic_open(flags); if (magic == NULL) { (void)fprintf(stderr, "%s: %s\n", progname, strerror(errno)); return NULL; } if (magic_load(magic, magicfile) == -1) { (void)fprintf(stderr, "%s: %s\n", progname, magic_error(magic)); magic_close(magic); return NULL; } return magic; } /* * unwrap -- read a file of filenames, do each one. */ private int unwrap(struct magic_set *ms, const char *fn) { FILE *f; ssize_t len; char *line = NULL; size_t llen = 0; int wid = 0, cwid; int e = 0; if (strcmp("-", fn) == 0) { f = stdin; wid = 1; } else { if ((f = fopen(fn, "r")) == NULL) { (void)fprintf(stderr, "%s: Cannot open `%s' (%s).\n", progname, fn, strerror(errno)); return 1; } while ((len = getline(&line, &llen, f)) > 0) { if (line[len - 1] == '\n') line[len - 1] = '\0'; cwid = file_mbswidth(line); if (cwid > wid) wid = cwid; } rewind(f); } while ((len = getline(&line, &llen, f)) > 0) { if (line[len - 1] == '\n') line[len - 1] = '\0'; e |= process(ms, line, wid); if(nobuffer) (void)fflush(stdout); } free(line); (void)fclose(f); return e; } /* * Called for each input file on the command line (or in a list of files) */ private int process(struct magic_set *ms, const char *inname, int wid) { const char *type; int std_in = strcmp(inname, "-") == 0; if (wid > 0 && !bflag) { (void)printf("%s", std_in ? "/dev/stdin" : inname); if (nulsep) (void)putc('\0', stdout); (void)printf("%s", separator); (void)printf("%*s ", (int) (nopad ? 0 : (wid - file_mbswidth(inname))), ""); } type = magic_file(ms, std_in ? NULL : inname); if (type == NULL) { (void)printf("ERROR: %s\n", magic_error(ms)); return 1; } else { (void)printf("%s\n", type); return 0; } } protected size_t file_mbswidth(const char *s) { #if defined(HAVE_WCHAR_H) && defined(HAVE_MBRTOWC) && defined(HAVE_WCWIDTH) size_t bytesconsumed, old_n, n, width = 0; mbstate_t state; wchar_t nextchar; (void)memset(&state, 0, sizeof(mbstate_t)); old_n = n = strlen(s); while (n > 0) { bytesconsumed = mbrtowc(&nextchar, s, n, &state); if (bytesconsumed == (size_t)(-1) || bytesconsumed == (size_t)(-2)) { /* Something went wrong, return something reasonable */ return old_n; } if (s[0] == '\n') { /* * do what strlen() would do, so that caller * is always right */ width++; } else { int w = wcwidth(nextchar); if (w > 0) width += w; } s += bytesconsumed, n -= bytesconsumed; } return width; #else return strlen(s); #endif } private void usage(void) { (void)fprintf(stderr, USAGE, progname, progname, progname); exit(1); } private void docprint(const char *opts) { size_t i; int comma; char *sp, *p; p = strstr(opts, "%o"); if (p == NULL) { fprintf(stdout, "%s", opts); return; } for (sp = p - 1; sp > opts && *sp == ' '; sp--) continue; fprintf(stdout, "%.*s", (int)(p - opts), opts); comma = 0; for (i = 0; i < __arraycount(nv); i++) { fprintf(stdout, "%s%s", comma++ ? ", " : "", nv[i].name); if (i && i % 5 == 0) { fprintf(stdout, ",\n%*s", (int)(p - sp - 1), ""); comma = 0; } } fprintf(stdout, "%s", opts + (p - opts) + 2); } private void help(void) { (void)fputs( "Usage: file [OPTION...] [FILE...]\n" "Determine type of FILEs.\n" "\n", stdout); #define OPT(shortname, longname, opt, doc) \ fprintf(stdout, " -%c, --" longname, shortname), \ docprint(doc); #define OPT_LONGONLY(longname, opt, doc) \ fprintf(stdout, " --" longname), \ docprint(doc); #include "file_opts.h" #undef OPT #undef OPT_LONGONLY fprintf(stdout, "\nReport bugs to http://bugs.gw.com/\n"); exit(0); } Index: releng/10.1/contrib/file/src/file.h =================================================================== --- releng/10.1/contrib/file/src/file.h (revision 284192) +++ releng/10.1/contrib/file/src/file.h (revision 284193) @@ -1,565 +1,594 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * file.h - definitions for file(1) program - * @(#)$File: file.h,v 1.152 2014/06/03 19:01:34 christos Exp $ + * @(#)$File: file.h,v 1.164 2015/01/01 17:07:34 christos Exp $ */ #ifndef __file_h__ #define __file_h__ #ifdef HAVE_CONFIG_H #include #endif #ifdef WIN32 #ifdef _WIN64 #define SIZE_T_FORMAT "I64" #else #define SIZE_T_FORMAT "" #endif #define INT64_T_FORMAT "I64" #else #define SIZE_T_FORMAT "z" #define INT64_T_FORMAT "ll" #endif #include /* Include that here, to make sure __P gets defined */ #include #include /* For open and flags */ #ifdef HAVE_STDINT_H #ifndef __STDC_LIMIT_MACROS #define __STDC_LIMIT_MACROS #endif #include #endif #ifdef HAVE_INTTYPES_H #include #endif #include #include #include +#ifndef WIN32 #include +#endif /* Do this here and now, because struct stat gets re-defined on solaris */ #include #include #define ENABLE_CONDITIONALS #ifndef MAGIC #define MAGIC "/etc/magic" #endif #if defined(__EMX__) || defined (WIN32) #define PATHSEP ';' #else #define PATHSEP ':' #endif #define private static #if HAVE_VISIBILITY && !defined(WIN32) #define public __attribute__ ((__visibility__("default"))) #ifndef protected #define protected __attribute__ ((__visibility__("hidden"))) #endif #else #define public #ifndef protected #define protected #endif #endif #ifndef __arraycount #define __arraycount(a) (sizeof(a) / sizeof(a[0])) #endif #ifndef __GNUC_PREREQ__ #ifdef __GNUC__ #define __GNUC_PREREQ__(x, y) \ ((__GNUC__ == (x) && __GNUC_MINOR__ >= (y)) || \ (__GNUC__ > (x))) #else #define __GNUC_PREREQ__(x, y) 0 #endif #endif #ifndef __GNUC__ #ifndef __attribute__ #define __attribute__(a) #endif #endif #ifndef MIN #define MIN(a,b) (((a) < (b)) ? (a) : (b)) #endif #ifndef MAX #define MAX(a,b) (((a) > (b)) ? (a) : (b)) #endif #ifndef HOWMANY # define HOWMANY (256 * 1024) /* how much of the file to look at */ #endif #define MAXMAGIS 8192 /* max entries in any one magic file or directory */ #define MAXDESC 64 /* max len of text description/MIME type */ #define MAXMIME 80 /* max len of text MIME type */ #define MAXstring 64 /* max len of "string" types */ #define MAGICNO 0xF11E041C #define VERSIONNO 12 #define FILE_MAGICSIZE 248 #define FILE_LOAD 0 #define FILE_CHECK 1 #define FILE_COMPILE 2 #define FILE_LIST 3 union VALUETYPE { uint8_t b; uint16_t h; uint32_t l; uint64_t q; uint8_t hs[2]; /* 2 bytes of a fixed-endian "short" */ uint8_t hl[4]; /* 4 bytes of a fixed-endian "long" */ uint8_t hq[8]; /* 8 bytes of a fixed-endian "quad" */ char s[MAXstring]; /* the search string or regex pattern */ unsigned char us[MAXstring]; float f; double d; }; struct magic { /* Word 1 */ uint16_t cont_level; /* level of ">" */ uint8_t flag; #define INDIR 0x01 /* if '(...)' appears */ #define OFFADD 0x02 /* if '>&' or '>...(&' appears */ #define INDIROFFADD 0x04 /* if '>&(' appears */ #define UNSIGNED 0x08 /* comparison is unsigned */ #define NOSPACE 0x10 /* suppress space character before output */ #define BINTEST 0x20 /* test is for a binary type (set only for top-level tests) */ #define TEXTTEST 0x40 /* for passing to file_softmagic */ uint8_t factor; /* Word 2 */ uint8_t reln; /* relation (0=eq, '>'=gt, etc) */ uint8_t vallen; /* length of string value, if any */ uint8_t type; /* comparison type (FILE_*) */ uint8_t in_type; /* type of indirection */ #define FILE_INVALID 0 #define FILE_BYTE 1 #define FILE_SHORT 2 #define FILE_DEFAULT 3 #define FILE_LONG 4 #define FILE_STRING 5 #define FILE_DATE 6 #define FILE_BESHORT 7 #define FILE_BELONG 8 #define FILE_BEDATE 9 #define FILE_LESHORT 10 #define FILE_LELONG 11 #define FILE_LEDATE 12 #define FILE_PSTRING 13 #define FILE_LDATE 14 #define FILE_BELDATE 15 #define FILE_LELDATE 16 #define FILE_REGEX 17 #define FILE_BESTRING16 18 #define FILE_LESTRING16 19 #define FILE_SEARCH 20 #define FILE_MEDATE 21 #define FILE_MELDATE 22 #define FILE_MELONG 23 #define FILE_QUAD 24 #define FILE_LEQUAD 25 #define FILE_BEQUAD 26 #define FILE_QDATE 27 #define FILE_LEQDATE 28 #define FILE_BEQDATE 29 #define FILE_QLDATE 30 #define FILE_LEQLDATE 31 #define FILE_BEQLDATE 32 #define FILE_FLOAT 33 #define FILE_BEFLOAT 34 #define FILE_LEFLOAT 35 #define FILE_DOUBLE 36 #define FILE_BEDOUBLE 37 #define FILE_LEDOUBLE 38 #define FILE_BEID3 39 #define FILE_LEID3 40 #define FILE_INDIRECT 41 #define FILE_QWDATE 42 #define FILE_LEQWDATE 43 #define FILE_BEQWDATE 44 #define FILE_NAME 45 #define FILE_USE 46 #define FILE_CLEAR 47 #define FILE_NAMES_SIZE 48 /* size of array to contain all names */ #define IS_STRING(t) \ ((t) == FILE_STRING || \ (t) == FILE_PSTRING || \ (t) == FILE_BESTRING16 || \ (t) == FILE_LESTRING16 || \ (t) == FILE_REGEX || \ (t) == FILE_SEARCH || \ + (t) == FILE_INDIRECT || \ (t) == FILE_NAME || \ (t) == FILE_USE) #define FILE_FMT_NONE 0 #define FILE_FMT_NUM 1 /* "cduxXi" */ #define FILE_FMT_STR 2 /* "s" */ #define FILE_FMT_QUAD 3 /* "ll" */ #define FILE_FMT_FLOAT 4 /* "eEfFgG" */ #define FILE_FMT_DOUBLE 5 /* "eEfFgG" */ /* Word 3 */ uint8_t in_op; /* operator for indirection */ uint8_t mask_op; /* operator for mask */ #ifdef ENABLE_CONDITIONALS uint8_t cond; /* conditional type */ #else uint8_t dummy; #endif uint8_t factor_op; #define FILE_FACTOR_OP_PLUS '+' #define FILE_FACTOR_OP_MINUS '-' #define FILE_FACTOR_OP_TIMES '*' #define FILE_FACTOR_OP_DIV '/' #define FILE_FACTOR_OP_NONE '\0' #define FILE_OPS "&|^+-*/%" #define FILE_OPAND 0 #define FILE_OPOR 1 #define FILE_OPXOR 2 #define FILE_OPADD 3 #define FILE_OPMINUS 4 #define FILE_OPMULTIPLY 5 #define FILE_OPDIVIDE 6 #define FILE_OPMODULO 7 #define FILE_OPS_MASK 0x07 /* mask for above ops */ #define FILE_UNUSED_1 0x08 #define FILE_UNUSED_2 0x10 #define FILE_UNUSED_3 0x20 #define FILE_OPINVERSE 0x40 #define FILE_OPINDIRECT 0x80 #ifdef ENABLE_CONDITIONALS #define COND_NONE 0 #define COND_IF 1 #define COND_ELIF 2 #define COND_ELSE 3 #endif /* ENABLE_CONDITIONALS */ /* Word 4 */ uint32_t offset; /* offset to magic number */ /* Word 5 */ int32_t in_offset; /* offset from indirection */ /* Word 6 */ uint32_t lineno; /* line number in magic file */ /* Word 7,8 */ union { uint64_t _mask; /* for use with numeric and date types */ struct { uint32_t _count; /* repeat/line count */ uint32_t _flags; /* modifier flags */ } _s; /* for use with string types */ } _u; #define num_mask _u._mask #define str_range _u._s._count #define str_flags _u._s._flags /* Words 9-16 */ union VALUETYPE value; /* either number or string */ /* Words 17-32 */ char desc[MAXDESC]; /* description */ /* Words 33-52 */ char mimetype[MAXMIME]; /* MIME type */ /* Words 53-54 */ char apple[8]; }; #define BIT(A) (1 << (A)) #define STRING_COMPACT_WHITESPACE BIT(0) #define STRING_COMPACT_OPTIONAL_WHITESPACE BIT(1) #define STRING_IGNORE_LOWERCASE BIT(2) #define STRING_IGNORE_UPPERCASE BIT(3) #define REGEX_OFFSET_START BIT(4) #define STRING_TEXTTEST BIT(5) #define STRING_BINTEST BIT(6) #define PSTRING_1_BE BIT(7) #define PSTRING_1_LE BIT(7) #define PSTRING_2_BE BIT(8) #define PSTRING_2_LE BIT(9) #define PSTRING_4_BE BIT(10) #define PSTRING_4_LE BIT(11) #define REGEX_LINE_COUNT BIT(11) #define PSTRING_LEN \ (PSTRING_1_BE|PSTRING_2_LE|PSTRING_2_BE|PSTRING_4_LE|PSTRING_4_BE) #define PSTRING_LENGTH_INCLUDES_ITSELF BIT(12) #define STRING_TRIM BIT(13) #define CHAR_COMPACT_WHITESPACE 'W' #define CHAR_COMPACT_OPTIONAL_WHITESPACE 'w' #define CHAR_IGNORE_LOWERCASE 'c' #define CHAR_IGNORE_UPPERCASE 'C' #define CHAR_REGEX_OFFSET_START 's' #define CHAR_TEXTTEST 't' #define CHAR_TRIM 'T' #define CHAR_BINTEST 'b' #define CHAR_PSTRING_1_BE 'B' #define CHAR_PSTRING_1_LE 'B' #define CHAR_PSTRING_2_BE 'H' #define CHAR_PSTRING_2_LE 'h' #define CHAR_PSTRING_4_BE 'L' #define CHAR_PSTRING_4_LE 'l' #define CHAR_PSTRING_LENGTH_INCLUDES_ITSELF 'J' #define STRING_IGNORE_CASE (STRING_IGNORE_LOWERCASE|STRING_IGNORE_UPPERCASE) #define STRING_DEFAULT_RANGE 100 +#define INDIRECT_RELATIVE BIT(0) +#define CHAR_INDIRECT_RELATIVE 'r' /* list of magic entries */ struct mlist { struct magic *magic; /* array of magic entries */ uint32_t nmagic; /* number of entries in array */ void *map; /* internal resources used by entry */ struct mlist *next, *prev; }; #ifdef __cplusplus #define CAST(T, b) static_cast(b) #define RCAST(T, b) reinterpret_cast(b) #else #define CAST(T, b) (T)(b) #define RCAST(T, b) (T)(b) #endif struct level_info { int32_t off; int got_match; #ifdef ENABLE_CONDITIONALS int last_match; int last_cond; /* used for error checking by parse() */ #endif }; #define MAGIC_SETS 2 struct magic_set { struct mlist *mlist[MAGIC_SETS]; /* list of regular entries */ struct cont { size_t len; struct level_info *li; } c; struct out { char *buf; /* Accumulation buffer */ char *pbuf; /* Printable buffer */ } o; uint32_t offset; int error; int flags; /* Control magic tests. */ int event_flags; /* Note things that happened. */ #define EVENT_HAD_ERR 0x01 const char *file; size_t line; /* current magic line number */ /* data for searches */ struct { const char *s; /* start of search in original source */ size_t s_len; /* length of search region */ size_t offset; /* starting offset in source: XXX - should this be off_t? */ size_t rm_len; /* match length */ } search; /* FIXME: Make the string dynamically allocated so that e.g. strings matched in files can be longer than MAXstring */ union VALUETYPE ms_value; /* either number or string */ + uint16_t indir_max; + uint16_t name_max; + uint16_t elf_shnum_max; + uint16_t elf_phnum_max; + uint16_t elf_notes_max; +#define FILE_INDIR_MAX 15 +#define FILE_NAME_MAX 30 +#define FILE_ELF_SHNUM_MAX 32768 +#define FILE_ELF_PHNUM_MAX 128 +#define FILE_ELF_NOTES_MAX 256 }; /* Type for Unicode characters */ typedef unsigned long unichar; struct stat; #define FILE_T_LOCAL 1 #define FILE_T_WINDOWS 2 protected const char *file_fmttime(uint64_t, int, char *); protected struct magic_set *file_ms_alloc(int); protected void file_ms_free(struct magic_set *); protected int file_buffer(struct magic_set *, int, const char *, const void *, size_t); protected int file_fsmagic(struct magic_set *, const char *, struct stat *); protected int file_pipe2file(struct magic_set *, int, const void *, size_t); protected int file_vprintf(struct magic_set *, const char *, va_list) __attribute__((__format__(__printf__, 2, 0))); protected size_t file_printedlen(const struct magic_set *); protected int file_replace(struct magic_set *, const char *, const char *); protected int file_printf(struct magic_set *, const char *, ...) __attribute__((__format__(__printf__, 2, 3))); protected int file_reset(struct magic_set *); protected int file_tryelf(struct magic_set *, int, const unsigned char *, size_t); protected int file_trycdf(struct magic_set *, int, const unsigned char *, size_t); #if HAVE_FORK protected int file_zmagic(struct magic_set *, int, const char *, const unsigned char *, size_t); #endif protected int file_ascmagic(struct magic_set *, const unsigned char *, size_t, int); protected int file_ascmagic_with_encoding(struct magic_set *, const unsigned char *, size_t, unichar *, size_t, const char *, const char *, int); protected int file_encoding(struct magic_set *, const unsigned char *, size_t, unichar **, size_t *, const char **, const char **, const char **); protected int file_is_tar(struct magic_set *, const unsigned char *, size_t); protected int file_softmagic(struct magic_set *, const unsigned char *, size_t, - size_t, int, int); + uint16_t, uint16_t *, int, int); protected int file_apprentice(struct magic_set *, const char *, int); +protected int buffer_apprentice(struct magic_set *, struct magic **, + size_t *, size_t); protected int file_magicfind(struct magic_set *, const char *, struct mlist *); protected uint64_t file_signextend(struct magic_set *, struct magic *, uint64_t); protected void file_badread(struct magic_set *); protected void file_badseek(struct magic_set *); protected void file_oomem(struct magic_set *, size_t); protected void file_error(struct magic_set *, int, const char *, ...) __attribute__((__format__(__printf__, 3, 4))); protected void file_magerror(struct magic_set *, const char *, ...) __attribute__((__format__(__printf__, 2, 3))); protected void file_magwarn(struct magic_set *, const char *, ...) __attribute__((__format__(__printf__, 2, 3))); protected void file_mdump(struct magic *); protected void file_showstr(FILE *, const char *, size_t); protected size_t file_mbswidth(const char *); protected const char *file_getbuffer(struct magic_set *); protected ssize_t sread(int, void *, size_t, int); protected int file_check_mem(struct magic_set *, unsigned int); protected int file_looks_utf8(const unsigned char *, size_t, unichar *, size_t *); protected size_t file_pstring_length_size(const struct magic *); protected size_t file_pstring_get_length(const struct magic *, const char *); +protected char * file_printable(char *, size_t, const char *); #ifdef __EMX__ protected int file_os2_apptype(struct magic_set *, const char *, const void *, size_t); #endif /* __EMX__ */ +#if defined(HAVE_LOCALE_H) +#include +#endif +#if defined(HAVE_XLOCALE_H) +#include +#endif + typedef struct { const char *pat; - char *old_lc_ctype; +#if defined(HAVE_NEWLOCALE) && defined(HAVE_USELOCALE) && defined(HAVE_FREELOCALE) +#define USE_C_LOCALE + locale_t old_lc_ctype; + locale_t c_lc_ctype; +#endif int rc; regex_t rx; } file_regex_t; protected int file_regcomp(file_regex_t *, const char *, int); protected int file_regexec(file_regex_t *, const char *, size_t, regmatch_t *, int); protected void file_regfree(file_regex_t *); protected void file_regerror(file_regex_t *, int, struct magic_set *); typedef struct { char *buf; uint32_t offset; } file_pushbuf_t; protected file_pushbuf_t *file_push_buffer(struct magic_set *); protected char *file_pop_buffer(struct magic_set *, file_pushbuf_t *); #ifndef COMPILE_ONLY extern const char *file_names[]; extern const size_t file_nnames; #endif #ifndef HAVE_STRERROR extern int sys_nerr; extern char *sys_errlist[]; #define strerror(e) \ (((e) >= 0 && (e) < sys_nerr) ? sys_errlist[(e)] : "Unknown error") #endif #ifndef HAVE_STRTOUL #define strtoul(a, b, c) strtol(a, b, c) #endif #ifndef HAVE_PREAD ssize_t pread(int, void *, size_t, off_t); #endif #ifndef HAVE_VASPRINTF int vasprintf(char **, const char *, va_list); #endif #ifndef HAVE_ASPRINTF int asprintf(char **, const char *, ...); #endif #ifndef HAVE_STRLCPY size_t strlcpy(char *, const char *, size_t); #endif #ifndef HAVE_STRLCAT size_t strlcat(char *, const char *, size_t); #endif #ifndef HAVE_STRCASESTR char *strcasestr(const char *, const char *); #endif #ifndef HAVE_GETLINE ssize_t getline(char **, size_t *, FILE *); ssize_t getdelim(char **, size_t *, int, FILE *); #endif #ifndef HAVE_CTIME_R char *ctime_r(const time_t *, char *); #endif #ifndef HAVE_ASCTIME_R char *asctime_r(const struct tm *, char *); #endif #ifndef HAVE_FMTCHECK const char *fmtcheck(const char *, const char *) __attribute__((__format_arg__(2))); #endif #if defined(HAVE_MMAP) && defined(HAVE_SYS_MMAN_H) && !defined(QUICK) #define QUICK #endif #ifndef O_BINARY #define O_BINARY 0 #endif #ifndef __cplusplus #if defined(__GNUC__) && (__GNUC__ >= 3) #define FILE_RCSID(id) \ static const char rcsid[] __attribute__((__used__)) = id; #else #define FILE_RCSID(id) \ static const char *rcsid(const char *p) { \ return rcsid(p = id); \ } #endif #else #define FILE_RCSID(id) #endif #endif /* __file_h__ */ Index: releng/10.1/contrib/file/src/file_opts.h =================================================================== --- releng/10.1/contrib/file/src/file_opts.h (revision 284192) +++ releng/10.1/contrib/file/src/file_opts.h (revision 284193) @@ -1,50 +1,56 @@ /* * Table of command-line options * * The first column specifies the short name, if any, or 0 if none. * The second column specifies the long name. * The third column specifies whether it takes a parameter. * The fourth column is the documentation. * * N.B. The long options' order must correspond to the code in file.c, * and OPTSTRING must be kept up-to-date with the short options. * Pay particular attention to the numbers of long-only options in the * switch statement! */ OPT_LONGONLY("help", 0, " display this help and exit\n") OPT('v', "version", 0, " output version information and exit\n") OPT('m', "magic-file", 1, " LIST use LIST as a colon-separated list of magic\n" " number files\n") OPT('z', "uncompress", 0, " try to look inside compressed files\n") OPT('b', "brief", 0, " do not prepend filenames to output lines\n") OPT('c', "checking-printout", 0, " print the parsed form of the magic file, use in\n" " conjunction with -m to debug a new magic file\n" " before installing it\n") OPT('e', "exclude", 1, " TEST exclude TEST from the list of test to be\n" " performed for file. Valid tests are:\n" " %o\n") OPT('f', "files-from", 1, " FILE read the filenames to be examined from FILE\n") OPT('F', "separator", 1, " STRING use string as separator instead of `:'\n") OPT('i', "mime", 0, " output MIME type strings (--mime-type and\n" " --mime-encoding)\n") OPT_LONGONLY("apple", 0, " output the Apple CREATOR/TYPE\n") OPT_LONGONLY("mime-type", 0, " output the MIME type\n") OPT_LONGONLY("mime-encoding", 0, " output the MIME encoding\n") OPT('k', "keep-going", 0, " don't stop at the first match\n") OPT('l', "list", 0, " list magic strength\n") #ifdef S_IFLNK OPT('L', "dereference", 0, " follow symlinks (default)\n") OPT('h', "no-dereference", 0, " don't follow symlinks\n") #endif OPT('n', "no-buffer", 0, " do not buffer output\n") OPT('N', "no-pad", 0, " do not pad output\n") OPT('0', "print0", 0, " terminate filenames with ASCII NUL\n") #if defined(HAVE_UTIME) || defined(HAVE_UTIMES) OPT('p', "preserve-date", 0, " preserve access times on files\n") #endif +OPT('P', "parameter", 0, " set file engine parameter limits\n" + " indir 15 recursion limit for indirection\n" + " name 30 use limit for name/use magic\n" + " elf_notes 256 max ELF notes processed\n" + " elf_phnum 128 max ELF prog sections processed\n" + " elf_shnum 32768 max ELF sections processed\n") OPT('r', "raw", 0, " don't translate unprintable chars to \\ooo\n") OPT('s', "special-files", 0, " treat special (block/char devices) files as\n" " ordinary ones\n") OPT('C', "compile", 0, " compile file specified by -m\n") OPT('d', "debug", 0, " print debugging messages\n") Index: releng/10.1/contrib/file/src/fsmagic.c =================================================================== --- releng/10.1/contrib/file/src/fsmagic.c (revision 284192) +++ releng/10.1/contrib/file/src/fsmagic.c (revision 284193) @@ -1,407 +1,407 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * fsmagic - magic based on filesystem info - directory, special files, etc. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: fsmagic.c,v 1.73 2014/05/14 23:15:42 christos Exp $") +FILE_RCSID("@(#)$File: fsmagic.c,v 1.75 2014/12/04 15:56:46 christos Exp $") #endif /* lint */ #include "magic.h" #include #ifdef HAVE_UNISTD_H #include #endif #include /* Since major is a function on SVR4, we cannot use `ifndef major'. */ #ifdef MAJOR_IN_MKDEV # include # define HAVE_MAJOR #endif #ifdef MAJOR_IN_SYSMACROS # include # define HAVE_MAJOR #endif #ifdef major /* Might be defined in sys/types.h. */ # define HAVE_MAJOR #endif #ifdef WIN32 # define WIN32_LEAN_AND_MEAN # include #endif #ifndef HAVE_MAJOR # define major(dev) (((dev) >> 8) & 0xff) # define minor(dev) ((dev) & 0xff) #endif #undef HAVE_MAJOR #ifdef S_IFLNK private int bad_link(struct magic_set *ms, int err, char *buf) { int mime = ms->flags & MAGIC_MIME; if ((mime & MAGIC_MIME_TYPE) && file_printf(ms, "inode/symlink") == -1) return -1; else if (!mime) { if (ms->flags & MAGIC_ERROR) { file_error(ms, err, - "broken symbolic link to `%s'", buf); + "broken symbolic link to %s", buf); return -1; } - if (file_printf(ms, "broken symbolic link to `%s'", buf) == -1) + if (file_printf(ms, "broken symbolic link to %s", buf) == -1) return -1; } return 1; } #endif private int handle_mime(struct magic_set *ms, int mime, const char *str) { if ((mime & MAGIC_MIME_TYPE)) { if (file_printf(ms, "inode/%s", str) == -1) return -1; if ((mime & MAGIC_MIME_ENCODING) && file_printf(ms, "; charset=") == -1) return -1; } if ((mime & MAGIC_MIME_ENCODING) && file_printf(ms, "binary") == -1) return -1; return 0; } protected int file_fsmagic(struct magic_set *ms, const char *fn, struct stat *sb) { int ret, did = 0; int mime = ms->flags & MAGIC_MIME; #ifdef S_IFLNK char buf[BUFSIZ+4]; ssize_t nch; struct stat tstatbuf; #endif if (ms->flags & MAGIC_APPLE) return 0; if (fn == NULL) return 0; #define COMMA (did++ ? ", " : "") /* * Fstat is cheaper but fails for files you don't have read perms on. * On 4.2BSD and similar systems, use lstat() to identify symlinks. */ #ifdef S_IFLNK if ((ms->flags & MAGIC_SYMLINK) == 0) ret = lstat(fn, sb); else #endif ret = stat(fn, sb); /* don't merge into if; see "ret =" above */ #ifdef WIN32 { - HANDLE hFile = CreateFile(fn, 0, FILE_SHARE_DELETE | + HANDLE hFile = CreateFile((LPCSTR)fn, 0, FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL); if (hFile != INVALID_HANDLE_VALUE) { /* * Stat failed, but we can still open it - assume it's * a block device, if nothing else. */ if (ret) { sb->st_mode = S_IFBLK; ret = 0; } switch (GetFileType(hFile)) { case FILE_TYPE_CHAR: sb->st_mode |= S_IFCHR; sb->st_mode &= ~S_IFREG; break; case FILE_TYPE_PIPE: sb->st_mode |= S_IFIFO; sb->st_mode &= ~S_IFREG; break; } CloseHandle(hFile); } } #endif if (ret) { if (ms->flags & MAGIC_ERROR) { file_error(ms, errno, "cannot stat `%s'", fn); return -1; } if (file_printf(ms, "cannot open `%s' (%s)", fn, strerror(errno)) == -1) return -1; return 0; } ret = 1; if (!mime) { #ifdef S_ISUID if (sb->st_mode & S_ISUID) if (file_printf(ms, "%ssetuid", COMMA) == -1) return -1; #endif #ifdef S_ISGID if (sb->st_mode & S_ISGID) if (file_printf(ms, "%ssetgid", COMMA) == -1) return -1; #endif #ifdef S_ISVTX if (sb->st_mode & S_ISVTX) if (file_printf(ms, "%ssticky", COMMA) == -1) return -1; #endif } switch (sb->st_mode & S_IFMT) { case S_IFDIR: if (mime) { if (handle_mime(ms, mime, "directory") == -1) return -1; } else if (file_printf(ms, "%sdirectory", COMMA) == -1) return -1; break; #ifdef S_IFCHR case S_IFCHR: /* * If -s has been specified, treat character special files * like ordinary files. Otherwise, just report that they * are block special files and go on to the next file. */ if ((ms->flags & MAGIC_DEVICES) != 0) { ret = 0; break; } if (mime) { if (handle_mime(ms, mime, "chardevice") == -1) return -1; } else { #ifdef HAVE_STRUCT_STAT_ST_RDEV # ifdef dv_unit if (file_printf(ms, "%scharacter special (%d/%d/%d)", COMMA, major(sb->st_rdev), dv_unit(sb->st_rdev), dv_subunit(sb->st_rdev)) == -1) return -1; # else if (file_printf(ms, "%scharacter special (%ld/%ld)", COMMA, (long)major(sb->st_rdev), (long)minor(sb->st_rdev)) == -1) return -1; # endif #else if (file_printf(ms, "%scharacter special", COMMA) == -1) return -1; #endif } break; #endif #ifdef S_IFBLK case S_IFBLK: /* * If -s has been specified, treat block special files * like ordinary files. Otherwise, just report that they * are block special files and go on to the next file. */ if ((ms->flags & MAGIC_DEVICES) != 0) { ret = 0; break; } if (mime) { if (handle_mime(ms, mime, "blockdevice") == -1) return -1; } else { #ifdef HAVE_STRUCT_STAT_ST_RDEV # ifdef dv_unit if (file_printf(ms, "%sblock special (%d/%d/%d)", COMMA, major(sb->st_rdev), dv_unit(sb->st_rdev), dv_subunit(sb->st_rdev)) == -1) return -1; # else if (file_printf(ms, "%sblock special (%ld/%ld)", COMMA, (long)major(sb->st_rdev), (long)minor(sb->st_rdev)) == -1) return -1; # endif #else if (file_printf(ms, "%sblock special", COMMA) == -1) return -1; #endif } break; #endif /* TODO add code to handle V7 MUX and Blit MUX files */ #ifdef S_IFIFO case S_IFIFO: if((ms->flags & MAGIC_DEVICES) != 0) break; if (mime) { if (handle_mime(ms, mime, "fifo") == -1) return -1; } else if (file_printf(ms, "%sfifo (named pipe)", COMMA) == -1) return -1; break; #endif #ifdef S_IFDOOR case S_IFDOOR: if (mime) { if (handle_mime(ms, mime, "door") == -1) return -1; } else if (file_printf(ms, "%sdoor", COMMA) == -1) return -1; break; #endif #ifdef S_IFLNK case S_IFLNK: if ((nch = readlink(fn, buf, BUFSIZ-1)) <= 0) { if (ms->flags & MAGIC_ERROR) { file_error(ms, errno, "unreadable symlink `%s'", fn); return -1; } if (mime) { if (handle_mime(ms, mime, "symlink") == -1) return -1; } else if (file_printf(ms, "%sunreadable symlink `%s' (%s)", COMMA, fn, strerror(errno)) == -1) return -1; break; } buf[nch] = '\0'; /* readlink(2) does not do this */ /* If broken symlink, say so and quit early. */ if (*buf == '/') { if (stat(buf, &tstatbuf) < 0) return bad_link(ms, errno, buf); } else { char *tmp; char buf2[BUFSIZ+BUFSIZ+4]; if ((tmp = strrchr(fn, '/')) == NULL) { tmp = buf; /* in current directory anyway */ } else { if (tmp - fn + 1 > BUFSIZ) { if (ms->flags & MAGIC_ERROR) { file_error(ms, 0, "path too long: `%s'", buf); return -1; } if (mime) { if (handle_mime(ms, mime, "x-path-too-long") == -1) return -1; } else if (file_printf(ms, "%spath too long: `%s'", COMMA, fn) == -1) return -1; break; } /* take dir part */ (void)strlcpy(buf2, fn, sizeof buf2); buf2[tmp - fn + 1] = '\0'; /* plus (rel) link */ (void)strlcat(buf2, buf, sizeof buf2); tmp = buf2; } if (stat(tmp, &tstatbuf) < 0) return bad_link(ms, errno, buf); } /* Otherwise, handle it. */ if ((ms->flags & MAGIC_SYMLINK) != 0) { const char *p; ms->flags &= MAGIC_SYMLINK; p = magic_file(ms, buf); ms->flags |= MAGIC_SYMLINK; if (p == NULL) return -1; } else { /* just print what it points to */ if (mime) { if (handle_mime(ms, mime, "symlink") == -1) return -1; - } else if (file_printf(ms, "%ssymbolic link to `%s'", + } else if (file_printf(ms, "%ssymbolic link to %s", COMMA, buf) == -1) return -1; } break; #endif #ifdef S_IFSOCK #ifndef __COHERENT__ case S_IFSOCK: if (mime) { if (handle_mime(ms, mime, "socket") == -1) return -1; } else if (file_printf(ms, "%ssocket", COMMA) == -1) return -1; break; #endif #endif case S_IFREG: /* * regular file, check next possibility * * If stat() tells us the file has zero length, report here that * the file is empty, so we can skip all the work of opening and * reading the file. * But if the -s option has been given, we skip this * optimization, since on some systems, stat() reports zero * size for raw disk partitions. (If the block special device * really has zero length, the fact that it is empty will be * detected and reported correctly when we read the file.) */ if ((ms->flags & MAGIC_DEVICES) == 0 && sb->st_size == 0) { if (mime) { if (handle_mime(ms, mime, "x-empty") == -1) return -1; } else if (file_printf(ms, "%sempty", COMMA) == -1) return -1; break; } ret = 0; break; default: file_error(ms, 0, "invalid mode 0%o", sb->st_mode); return -1; /*NOTREACHED*/ } if (!mime && did && ret == 0) { if (file_printf(ms, " ") == -1) return -1; } return ret; } Index: releng/10.1/contrib/file/src/funcs.c =================================================================== --- releng/10.1/contrib/file/src/funcs.c (revision 284192) +++ releng/10.1/contrib/file/src/funcs.c (revision 284193) @@ -1,533 +1,558 @@ /* * Copyright (c) Christos Zoulas 2003. * All Rights Reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: funcs.c,v 1.72 2014/05/14 23:15:42 christos Exp $") +FILE_RCSID("@(#)$File: funcs.c,v 1.79 2014/12/16 20:52:49 christos Exp $") #endif /* lint */ #include "magic.h" #include #include #include #include #include #if defined(HAVE_WCHAR_H) #include #endif #if defined(HAVE_WCTYPE_H) #include #endif #if defined(HAVE_LIMITS_H) #include #endif -#if defined(HAVE_LOCALE_H) -#include -#endif #ifndef SIZE_MAX #define SIZE_MAX ((size_t)~0) #endif /* * Like printf, only we append to a buffer. */ protected int file_vprintf(struct magic_set *ms, const char *fmt, va_list ap) { int len; char *buf, *newstr; if (ms->event_flags & EVENT_HAD_ERR) return 0; len = vasprintf(&buf, fmt, ap); if (len < 0) goto out; if (ms->o.buf != NULL) { len = asprintf(&newstr, "%s%s", ms->o.buf, buf); free(buf); if (len < 0) goto out; free(ms->o.buf); buf = newstr; } ms->o.buf = buf; return 0; out: file_error(ms, errno, "vasprintf failed"); return -1; } protected int file_printf(struct magic_set *ms, const char *fmt, ...) { int rv; va_list ap; va_start(ap, fmt); rv = file_vprintf(ms, fmt, ap); va_end(ap); return rv; } /* * error - print best error message possible */ /*VARARGS*/ __attribute__((__format__(__printf__, 3, 0))) private void file_error_core(struct magic_set *ms, int error, const char *f, va_list va, size_t lineno) { /* Only the first error is ok */ if (ms->event_flags & EVENT_HAD_ERR) return; if (lineno != 0) { free(ms->o.buf); ms->o.buf = NULL; file_printf(ms, "line %" SIZE_T_FORMAT "u: ", lineno); } file_vprintf(ms, f, va); if (error > 0) file_printf(ms, " (%s)", strerror(error)); ms->event_flags |= EVENT_HAD_ERR; ms->error = error; } /*VARARGS*/ protected void file_error(struct magic_set *ms, int error, const char *f, ...) { va_list va; va_start(va, f); file_error_core(ms, error, f, va, 0); va_end(va); } /* * Print an error with magic line number. */ /*VARARGS*/ protected void file_magerror(struct magic_set *ms, const char *f, ...) { va_list va; va_start(va, f); file_error_core(ms, 0, f, va, ms->line); va_end(va); } protected void file_oomem(struct magic_set *ms, size_t len) { file_error(ms, errno, "cannot allocate %" SIZE_T_FORMAT "u bytes", len); } protected void file_badseek(struct magic_set *ms) { file_error(ms, errno, "error seeking"); } protected void file_badread(struct magic_set *ms) { file_error(ms, errno, "error reading"); } #ifndef COMPILE_ONLY protected int file_buffer(struct magic_set *ms, int fd, const char *inname __attribute__ ((unused)), const void *buf, size_t nb) { int m = 0, rv = 0, looks_text = 0; int mime = ms->flags & MAGIC_MIME; const unsigned char *ubuf = CAST(const unsigned char *, buf); unichar *u8buf = NULL; size_t ulen; const char *code = NULL; const char *code_mime = "binary"; const char *type = "application/octet-stream"; const char *def = "data"; const char *ftype = NULL; if (nb == 0) { def = "empty"; type = "application/x-empty"; goto simple; } else if (nb == 1) { def = "very short file (no magic)"; goto simple; } if ((ms->flags & MAGIC_NO_CHECK_ENCODING) == 0) { looks_text = file_encoding(ms, ubuf, nb, &u8buf, &ulen, &code, &code_mime, &ftype); } #ifdef __EMX__ if ((ms->flags & MAGIC_NO_CHECK_APPTYPE) == 0 && inname) { switch (file_os2_apptype(ms, inname, buf, nb)) { case -1: return -1; case 0: break; default: return 1; } } #endif #if HAVE_FORK /* try compression stuff */ if ((ms->flags & MAGIC_NO_CHECK_COMPRESS) == 0) if ((m = file_zmagic(ms, fd, inname, ubuf, nb)) != 0) { if ((ms->flags & MAGIC_DEBUG) != 0) (void)fprintf(stderr, "zmagic %d\n", m); goto done_encoding; } #endif /* Check if we have a tar file */ if ((ms->flags & MAGIC_NO_CHECK_TAR) == 0) if ((m = file_is_tar(ms, ubuf, nb)) != 0) { if ((ms->flags & MAGIC_DEBUG) != 0) (void)fprintf(stderr, "tar %d\n", m); goto done; } /* Check if we have a CDF file */ if ((ms->flags & MAGIC_NO_CHECK_CDF) == 0) if ((m = file_trycdf(ms, fd, ubuf, nb)) != 0) { if ((ms->flags & MAGIC_DEBUG) != 0) (void)fprintf(stderr, "cdf %d\n", m); goto done; } /* try soft magic tests */ if ((ms->flags & MAGIC_NO_CHECK_SOFT) == 0) - if ((m = file_softmagic(ms, ubuf, nb, 0, BINTEST, + if ((m = file_softmagic(ms, ubuf, nb, 0, NULL, BINTEST, looks_text)) != 0) { if ((ms->flags & MAGIC_DEBUG) != 0) (void)fprintf(stderr, "softmagic %d\n", m); #ifdef BUILTIN_ELF if ((ms->flags & MAGIC_NO_CHECK_ELF) == 0 && m == 1 && nb > 5 && fd != -1) { /* * We matched something in the file, so this * *might* be an ELF file, and the file is at * least 5 bytes long, so if it's an ELF file * it has at least one byte past the ELF magic * number - try extracting information from the * ELF headers that cannot easily * be * extracted with rules in the magic file. */ if ((m = file_tryelf(ms, fd, ubuf, nb)) != 0) if ((ms->flags & MAGIC_DEBUG) != 0) (void)fprintf(stderr, "elf %d\n", m); } #endif goto done; } /* try text properties */ if ((ms->flags & MAGIC_NO_CHECK_TEXT) == 0) { if ((m = file_ascmagic(ms, ubuf, nb, looks_text)) != 0) { if ((ms->flags & MAGIC_DEBUG) != 0) (void)fprintf(stderr, "ascmagic %d\n", m); goto done; } } simple: /* give up */ m = 1; if ((!mime || (mime & MAGIC_MIME_TYPE)) && file_printf(ms, "%s", mime ? type : def) == -1) { rv = -1; } done: if ((ms->flags & MAGIC_MIME_ENCODING) != 0) { if (ms->flags & MAGIC_MIME_TYPE) if (file_printf(ms, "; charset=") == -1) rv = -1; if (file_printf(ms, "%s", code_mime) == -1) rv = -1; } #if HAVE_FORK done_encoding: #endif free(u8buf); if (rv) return rv; return m; } #endif protected int file_reset(struct magic_set *ms) { if (ms->mlist[0] == NULL) { file_error(ms, 0, "no magic files loaded"); return -1; } if (ms->o.buf) { free(ms->o.buf); ms->o.buf = NULL; } if (ms->o.pbuf) { free(ms->o.pbuf); ms->o.pbuf = NULL; } ms->event_flags &= ~EVENT_HAD_ERR; ms->error = -1; return 0; } #define OCTALIFY(n, o) \ /*LINTED*/ \ (void)(*(n)++ = '\\', \ *(n)++ = (((uint32_t)*(o) >> 6) & 3) + '0', \ *(n)++ = (((uint32_t)*(o) >> 3) & 7) + '0', \ *(n)++ = (((uint32_t)*(o) >> 0) & 7) + '0', \ (o)++) protected const char * file_getbuffer(struct magic_set *ms) { char *pbuf, *op, *np; size_t psize, len; if (ms->event_flags & EVENT_HAD_ERR) return NULL; if (ms->flags & MAGIC_RAW) return ms->o.buf; if (ms->o.buf == NULL) return NULL; /* * 4 is for octal representation, + 1 is for NUL */ len = strlen(ms->o.buf); if (len > (SIZE_MAX - 1) / 4) { file_oomem(ms, len); return NULL; } psize = len * 4 + 1; if ((pbuf = CAST(char *, realloc(ms->o.pbuf, psize))) == NULL) { file_oomem(ms, psize); return NULL; } ms->o.pbuf = pbuf; #if defined(HAVE_WCHAR_H) && defined(HAVE_MBRTOWC) && defined(HAVE_WCWIDTH) { mbstate_t state; wchar_t nextchar; int mb_conv = 1; size_t bytesconsumed; char *eop; (void)memset(&state, 0, sizeof(mbstate_t)); np = ms->o.pbuf; op = ms->o.buf; eop = op + len; while (op < eop) { bytesconsumed = mbrtowc(&nextchar, op, (size_t)(eop - op), &state); if (bytesconsumed == (size_t)(-1) || bytesconsumed == (size_t)(-2)) { mb_conv = 0; break; } if (iswprint(nextchar)) { (void)memcpy(np, op, bytesconsumed); op += bytesconsumed; np += bytesconsumed; } else { while (bytesconsumed-- > 0) OCTALIFY(np, op); } } *np = '\0'; /* Parsing succeeded as a multi-byte sequence */ if (mb_conv != 0) return ms->o.pbuf; } #endif for (np = ms->o.pbuf, op = ms->o.buf; *op;) { if (isprint((unsigned char)*op)) { *np++ = *op++; } else { OCTALIFY(np, op); } } *np = '\0'; return ms->o.pbuf; } protected int file_check_mem(struct magic_set *ms, unsigned int level) { size_t len; if (level >= ms->c.len) { len = (ms->c.len += 20) * sizeof(*ms->c.li); ms->c.li = CAST(struct level_info *, (ms->c.li == NULL) ? malloc(len) : realloc(ms->c.li, len)); if (ms->c.li == NULL) { file_oomem(ms, len); return -1; } } ms->c.li[level].got_match = 0; #ifdef ENABLE_CONDITIONALS ms->c.li[level].last_match = 0; ms->c.li[level].last_cond = COND_NONE; #endif /* ENABLE_CONDITIONALS */ return 0; } protected size_t file_printedlen(const struct magic_set *ms) { return ms->o.buf == NULL ? 0 : strlen(ms->o.buf); } protected int file_replace(struct magic_set *ms, const char *pat, const char *rep) { file_regex_t rx; int rc, rv = -1; rc = file_regcomp(&rx, pat, REG_EXTENDED); if (rc) { file_regerror(&rx, rc, ms); } else { regmatch_t rm; int nm = 0; while (file_regexec(&rx, ms->o.buf, 1, &rm, 0) == 0) { ms->o.buf[rm.rm_so] = '\0'; if (file_printf(ms, "%s%s", rep, rm.rm_eo != 0 ? ms->o.buf + rm.rm_eo : "") == -1) goto out; nm++; } rv = nm; } out: file_regfree(&rx); return rv; } protected int file_regcomp(file_regex_t *rx, const char *pat, int flags) { - rx->old_lc_ctype = setlocale(LC_CTYPE, NULL); +#ifdef USE_C_LOCALE + rx->c_lc_ctype = newlocale(LC_CTYPE_MASK, "C", 0); + assert(rx->c_lc_ctype != NULL); + rx->old_lc_ctype = uselocale(rx->c_lc_ctype); assert(rx->old_lc_ctype != NULL); - rx->old_lc_ctype = strdup(rx->old_lc_ctype); - assert(rx->old_lc_ctype != NULL); +#endif rx->pat = pat; - (void)setlocale(LC_CTYPE, "C"); return rx->rc = regcomp(&rx->rx, pat, flags); } protected int file_regexec(file_regex_t *rx, const char *str, size_t nmatch, regmatch_t* pmatch, int eflags) { assert(rx->rc == 0); return regexec(&rx->rx, str, nmatch, pmatch, eflags); } protected void file_regfree(file_regex_t *rx) { if (rx->rc == 0) regfree(&rx->rx); - (void)setlocale(LC_CTYPE, rx->old_lc_ctype); - free(rx->old_lc_ctype); +#ifdef USE_C_LOCALE + (void)uselocale(rx->old_lc_ctype); + freelocale(rx->c_lc_ctype); +#endif } protected void file_regerror(file_regex_t *rx, int rc, struct magic_set *ms) { char errmsg[512]; (void)regerror(rc, &rx->rx, errmsg, sizeof(errmsg)); file_magerror(ms, "regex error %d for `%s', (%s)", rc, rx->pat, errmsg); } protected file_pushbuf_t * file_push_buffer(struct magic_set *ms) { file_pushbuf_t *pb; if (ms->event_flags & EVENT_HAD_ERR) return NULL; if ((pb = (CAST(file_pushbuf_t *, malloc(sizeof(*pb))))) == NULL) return NULL; pb->buf = ms->o.buf; pb->offset = ms->offset; ms->o.buf = NULL; ms->offset = 0; return pb; } protected char * file_pop_buffer(struct magic_set *ms, file_pushbuf_t *pb) { char *rbuf; if (ms->event_flags & EVENT_HAD_ERR) { free(pb->buf); free(pb); return NULL; } rbuf = ms->o.buf; ms->o.buf = pb->buf; ms->offset = pb->offset; free(pb); return rbuf; +} + +/* + * convert string to ascii printable format. + */ +protected char * +file_printable(char *buf, size_t bufsiz, const char *str) +{ + char *ptr, *eptr; + const unsigned char *s = (const unsigned char *)str; + + for (ptr = buf, eptr = ptr + bufsiz - 1; ptr < eptr && *s; s++) { + if (isprint(*s)) { + *ptr++ = *s; + continue; + } + if (ptr >= eptr - 3) + break; + *ptr++ = '\\'; + *ptr++ = ((*s >> 6) & 7) + '0'; + *ptr++ = ((*s >> 3) & 7) + '0'; + *ptr++ = ((*s >> 0) & 7) + '0'; + } + *ptr = '\0'; + return buf; } Index: releng/10.1/contrib/file/src/getline.c =================================================================== --- releng/10.1/contrib/file/src/getline.c (revision 284192) +++ releng/10.1/contrib/file/src/getline.c (revision 284193) @@ -1,100 +1,104 @@ -/* $NetBSD: fgetln.c,v 1.9 2008/04/29 06:53:03 martin Exp $ */ +/* $NetBSD: getline.c,v 1.2 2014/09/16 17:23:50 christos Exp $ */ /*- * Copyright (c) 2011 The NetBSD Foundation, Inc. * All rights reserved. * * This code is derived from software contributed to The NetBSD Foundation * by Christos Zoulas. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ #include "file.h" #if !HAVE_GETLINE #include #include #include #include #include public ssize_t getdelim(char **buf, size_t *bufsiz, int delimiter, FILE *fp) { char *ptr, *eptr; if (*buf == NULL || *bufsiz == 0) { *bufsiz = BUFSIZ; if ((*buf = malloc(*bufsiz)) == NULL) return -1; } for (ptr = *buf, eptr = *buf + *bufsiz;;) { int c = fgetc(fp); if (c == -1) { - if (feof(fp)) - return ptr == *buf ? -1 : ptr - *buf; - else - return -1; + if (feof(fp)) { + ssize_t diff = (ssize_t)(ptr - *buf); + if (diff != 0) { + *ptr = '\0'; + return diff; + } + } + return -1; } *ptr++ = c; if (c == delimiter) { *ptr = '\0'; return ptr - *buf; } if (ptr + 2 >= eptr) { char *nbuf; size_t nbufsiz = *bufsiz * 2; ssize_t d = ptr - *buf; if ((nbuf = realloc(*buf, nbufsiz)) == NULL) return -1; *buf = nbuf; *bufsiz = nbufsiz; eptr = nbuf + nbufsiz; ptr = nbuf + d; } } } -ssize_t +public ssize_t getline(char **buf, size_t *bufsiz, FILE *fp) { return getdelim(buf, bufsiz, '\n', fp); } #endif #ifdef TEST int main(int argc, char *argv[]) { char *p = NULL; ssize_t len; size_t n = 0; while ((len = getline(&p, &n, stdin)) != -1) - (void)printf("%zd %s", len, p); + (void)printf("%" SIZE_T_FORMAT "d %s", len, p); free(p); return 0; } #endif Index: releng/10.1/contrib/file/src/magic.c =================================================================== --- releng/10.1/contrib/file/src/magic.c (revision 284192) +++ releng/10.1/contrib/file/src/magic.c (revision 284193) @@ -1,524 +1,589 @@ /* * Copyright (c) Christos Zoulas 2003. * All Rights Reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #ifdef WIN32 #include #include #endif #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: magic.c,v 1.84 2014/05/14 23:15:42 christos Exp $") +FILE_RCSID("@(#)$File: magic.c,v 1.91 2014/12/16 23:18:40 christos Exp $") #endif /* lint */ #include "magic.h" #include #include #include #ifdef QUICK #include #endif #ifdef HAVE_LIMITS_H #include /* for PIPE_BUF */ #endif #if defined(HAVE_UTIMES) # include #elif defined(HAVE_UTIME) # if defined(HAVE_SYS_UTIME_H) # include # elif defined(HAVE_UTIME_H) # include # endif #endif #ifdef HAVE_UNISTD_H #include /* for read() */ #endif #ifndef PIPE_BUF /* Get the PIPE_BUF from pathconf */ #ifdef _PC_PIPE_BUF #define PIPE_BUF pathconf(".", _PC_PIPE_BUF) #else #define PIPE_BUF 512 #endif #endif private void close_and_restore(const struct magic_set *, const char *, int, const struct stat *); private int unreadable_info(struct magic_set *, mode_t, const char *); private const char* get_default_magic(void); #ifndef COMPILE_ONLY private const char *file_or_fd(struct magic_set *, const char *, int); #endif #ifndef STDIN_FILENO #define STDIN_FILENO 0 #endif private const char * get_default_magic(void) { static const char hmagic[] = "/.magic/magic.mgc"; static char *default_magic; char *home, *hmagicpath; #ifndef WIN32 struct stat st; if (default_magic) { free(default_magic); default_magic = NULL; } if ((home = getenv("HOME")) == NULL) return MAGIC; if (asprintf(&hmagicpath, "%s/.magic.mgc", home) < 0) return MAGIC; if (stat(hmagicpath, &st) == -1) { free(hmagicpath); if (asprintf(&hmagicpath, "%s/.magic", home) < 0) return MAGIC; if (stat(hmagicpath, &st) == -1) goto out; if (S_ISDIR(st.st_mode)) { free(hmagicpath); if (asprintf(&hmagicpath, "%s/%s", home, hmagic) < 0) return MAGIC; if (access(hmagicpath, R_OK) == -1) goto out; } } if (asprintf(&default_magic, "%s:%s", hmagicpath, MAGIC) < 0) goto out; free(hmagicpath); return default_magic; out: default_magic = NULL; free(hmagicpath); return MAGIC; #else char *hmagicp; char *tmppath = NULL; + LPTSTR dllpath; hmagicpath = NULL; #define APPENDPATH() \ do { \ if (tmppath && access(tmppath, R_OK) != -1) { \ if (hmagicpath == NULL) \ hmagicpath = tmppath; \ else { \ if (asprintf(&hmagicp, "%s%c%s", hmagicpath, \ PATHSEP, tmppath) >= 0) { \ free(hmagicpath); \ hmagicpath = hmagicp; \ } \ free(tmppath); \ } \ tmppath = NULL; \ } \ } while (/*CONSTCOND*/0) if (default_magic) { free(default_magic); default_magic = NULL; } /* First, try to get user-specific magic file */ if ((home = getenv("LOCALAPPDATA")) == NULL) { if ((home = getenv("USERPROFILE")) != NULL) if (asprintf(&tmppath, "%s/Local Settings/Application Data%s", home, hmagic) < 0) tmppath = NULL; } else { if (asprintf(&tmppath, "%s%s", home, hmagic) < 0) tmppath = NULL; } APPENDPATH(); /* Second, try to get a magic file from Common Files */ if ((home = getenv("COMMONPROGRAMFILES")) != NULL) { if (asprintf(&tmppath, "%s%s", home, hmagic) >= 0) APPENDPATH(); } /* Third, try to get magic file relative to dll location */ - LPTSTR dllpath = malloc(sizeof(*dllpath) * (MAX_PATH + 1)); + dllpath = malloc(sizeof(*dllpath) * (MAX_PATH + 1)); dllpath[MAX_PATH] = 0; /* just in case long path gets truncated and not null terminated */ if (GetModuleFileNameA(NULL, dllpath, MAX_PATH)){ PathRemoveFileSpecA(dllpath); if (strlen(dllpath) > 3 && stricmp(&dllpath[strlen(dllpath) - 3], "bin") == 0) { if (asprintf(&tmppath, "%s/../share/misc/magic.mgc", dllpath) >= 0) APPENDPATH(); } else { if (asprintf(&tmppath, "%s/share/misc/magic.mgc", dllpath) >= 0) APPENDPATH(); else if (asprintf(&tmppath, "%s/magic.mgc", dllpath) >= 0) APPENDPATH(); } } /* Don't put MAGIC constant - it likely points to a file within MSys tree */ default_magic = hmagicpath; return default_magic; #endif } public const char * magic_getpath(const char *magicfile, int action) { if (magicfile != NULL) return magicfile; magicfile = getenv("MAGIC"); if (magicfile != NULL) return magicfile; return action == FILE_LOAD ? get_default_magic() : MAGIC; } public struct magic_set * magic_open(int flags) { return file_ms_alloc(flags); } private int unreadable_info(struct magic_set *ms, mode_t md, const char *file) { if (file) { /* We cannot open it, but we were able to stat it. */ if (access(file, W_OK) == 0) if (file_printf(ms, "writable, ") == -1) return -1; if (access(file, X_OK) == 0) if (file_printf(ms, "executable, ") == -1) return -1; } if (S_ISREG(md)) if (file_printf(ms, "regular file, ") == -1) return -1; if (file_printf(ms, "no read permission") == -1) return -1; return 0; } public void magic_close(struct magic_set *ms) { if (ms == NULL) return; file_ms_free(ms); } /* * load a magic file */ public int magic_load(struct magic_set *ms, const char *magicfile) { if (ms == NULL) return -1; return file_apprentice(ms, magicfile, FILE_LOAD); } +#ifndef COMPILE_ONLY +/* + * Install a set of compiled magic buffers. + */ public int +magic_load_buffers(struct magic_set *ms, void **bufs, size_t *sizes, + size_t nbufs) +{ + if (ms == NULL) + return -1; + return buffer_apprentice(ms, (struct magic **)bufs, sizes, nbufs); +} +#endif + +public int magic_compile(struct magic_set *ms, const char *magicfile) { if (ms == NULL) return -1; return file_apprentice(ms, magicfile, FILE_COMPILE); } public int magic_check(struct magic_set *ms, const char *magicfile) { if (ms == NULL) return -1; return file_apprentice(ms, magicfile, FILE_CHECK); } public int magic_list(struct magic_set *ms, const char *magicfile) { if (ms == NULL) return -1; return file_apprentice(ms, magicfile, FILE_LIST); } private void close_and_restore(const struct magic_set *ms, const char *name, int fd, const struct stat *sb) { if (fd == STDIN_FILENO || name == NULL) return; (void) close(fd); if ((ms->flags & MAGIC_PRESERVE_ATIME) != 0) { /* * Try to restore access, modification times if read it. * This is really *bad* because it will modify the status * time of the file... And of course this will affect * backup programs */ #ifdef HAVE_UTIMES struct timeval utsbuf[2]; (void)memset(utsbuf, 0, sizeof(utsbuf)); utsbuf[0].tv_sec = sb->st_atime; utsbuf[1].tv_sec = sb->st_mtime; (void) utimes(name, utsbuf); /* don't care if loses */ #elif defined(HAVE_UTIME_H) || defined(HAVE_SYS_UTIME_H) struct utimbuf utbuf; (void)memset(&utbuf, 0, sizeof(utbuf)); utbuf.actime = sb->st_atime; utbuf.modtime = sb->st_mtime; (void) utime(name, &utbuf); /* don't care if loses */ #endif } } #ifndef COMPILE_ONLY /* * find type of descriptor */ public const char * magic_descriptor(struct magic_set *ms, int fd) { if (ms == NULL) return NULL; return file_or_fd(ms, NULL, fd); } /* * find type of named file */ public const char * magic_file(struct magic_set *ms, const char *inname) { if (ms == NULL) return NULL; return file_or_fd(ms, inname, STDIN_FILENO); } private const char * file_or_fd(struct magic_set *ms, const char *inname, int fd) { int rv = -1; unsigned char *buf; struct stat sb; ssize_t nbytes = 0; /* number of bytes read from a datafile */ int ispipe = 0; off_t pos = (off_t)-1; if (file_reset(ms) == -1) goto out; /* * one extra for terminating '\0', and * some overlapping space for matches near EOF */ #define SLOP (1 + sizeof(union VALUETYPE)) if ((buf = CAST(unsigned char *, malloc(HOWMANY + SLOP))) == NULL) return NULL; switch (file_fsmagic(ms, inname, &sb)) { case -1: /* error */ goto done; case 0: /* nothing found */ break; default: /* matched it and printed type */ rv = 0; goto done; } #ifdef WIN32 /* Place stdin in binary mode, so EOF (Ctrl+Z) doesn't stop early. */ if (fd == STDIN_FILENO) _setmode(STDIN_FILENO, O_BINARY); #endif if (inname == NULL) { if (fstat(fd, &sb) == 0 && S_ISFIFO(sb.st_mode)) ispipe = 1; else pos = lseek(fd, (off_t)0, SEEK_CUR); } else { int flags = O_RDONLY|O_BINARY; int okstat = stat(inname, &sb) == 0; if (okstat && S_ISFIFO(sb.st_mode)) { #ifdef O_NONBLOCK flags |= O_NONBLOCK; #endif ispipe = 1; } errno = 0; if ((fd = open(inname, flags)) < 0) { #ifdef WIN32 /* * Can't stat, can't open. It may have been opened in * fsmagic, so if the user doesn't have read permission, * allow it to say so; otherwise an error was probably * displayed in fsmagic. */ if (!okstat && errno == EACCES) { sb.st_mode = S_IFBLK; okstat = 1; } #endif if (okstat && unreadable_info(ms, sb.st_mode, inname) == -1) goto done; rv = 0; goto done; } #ifdef O_NONBLOCK if ((flags = fcntl(fd, F_GETFL)) != -1) { flags &= ~O_NONBLOCK; (void)fcntl(fd, F_SETFL, flags); } #endif } /* * try looking at the first HOWMANY bytes */ if (ispipe) { ssize_t r = 0; while ((r = sread(fd, (void *)&buf[nbytes], (size_t)(HOWMANY - nbytes), 1)) > 0) { nbytes += r; if (r < PIPE_BUF) break; } if (nbytes == 0) { /* We can not read it, but we were able to stat it. */ if (unreadable_info(ms, sb.st_mode, inname) == -1) goto done; rv = 0; goto done; } } else { /* Windows refuses to read from a big console buffer. */ size_t howmany = #if defined(WIN32) && HOWMANY > 8 * 1024 _isatty(fd) ? 8 * 1024 : #endif HOWMANY; if ((nbytes = read(fd, (char *)buf, howmany)) == -1) { if (inname == NULL && fd != STDIN_FILENO) file_error(ms, errno, "cannot read fd %d", fd); else file_error(ms, errno, "cannot read `%s'", inname == NULL ? "/dev/stdin" : inname); goto done; } } (void)memset(buf + nbytes, 0, SLOP); /* NUL terminate */ if (file_buffer(ms, fd, inname, buf, (size_t)nbytes) == -1) goto done; rv = 0; done: free(buf); if (pos != (off_t)-1) (void)lseek(fd, pos, SEEK_SET); close_and_restore(ms, inname, fd, &sb); out: return rv == 0 ? file_getbuffer(ms) : NULL; } public const char * magic_buffer(struct magic_set *ms, const void *buf, size_t nb) { if (ms == NULL) return NULL; if (file_reset(ms) == -1) return NULL; /* * The main work is done here! * We have the file name and/or the data buffer to be identified. */ if (file_buffer(ms, -1, NULL, buf, nb) == -1) { return NULL; } return file_getbuffer(ms); } #endif public const char * magic_error(struct magic_set *ms) { if (ms == NULL) return "Magic database is not open"; return (ms->event_flags & EVENT_HAD_ERR) ? ms->o.buf : NULL; } public int magic_errno(struct magic_set *ms) { if (ms == NULL) return EINVAL; return (ms->event_flags & EVENT_HAD_ERR) ? ms->error : 0; } public int magic_setflags(struct magic_set *ms, int flags) { if (ms == NULL) return -1; #if !defined(HAVE_UTIME) && !defined(HAVE_UTIMES) if (flags & MAGIC_PRESERVE_ATIME) return -1; #endif ms->flags = flags; return 0; } public int magic_version(void) { return MAGIC_VERSION; +} + +public int +magic_setparam(struct magic_set *ms, int param, const void *val) +{ + switch (param) { + case MAGIC_PARAM_INDIR_MAX: + ms->indir_max = *(const size_t *)val; + return 0; + case MAGIC_PARAM_NAME_MAX: + ms->name_max = *(const size_t *)val; + return 0; + case MAGIC_PARAM_ELF_PHNUM_MAX: + ms->elf_phnum_max = *(const size_t *)val; + return 0; + case MAGIC_PARAM_ELF_SHNUM_MAX: + ms->elf_shnum_max = *(const size_t *)val; + return 0; + case MAGIC_PARAM_ELF_NOTES_MAX: + ms->elf_notes_max = *(const size_t *)val; + return 0; + default: + errno = EINVAL; + return -1; + } +} + +public int +magic_getparam(struct magic_set *ms, int param, void *val) +{ + switch (param) { + case MAGIC_PARAM_INDIR_MAX: + *(size_t *)val = ms->indir_max; + return 0; + case MAGIC_PARAM_NAME_MAX: + *(size_t *)val = ms->name_max; + return 0; + case MAGIC_PARAM_ELF_PHNUM_MAX: + *(size_t *)val = ms->elf_phnum_max; + return 0; + case MAGIC_PARAM_ELF_SHNUM_MAX: + *(size_t *)val = ms->elf_shnum_max; + return 0; + case MAGIC_PARAM_ELF_NOTES_MAX: + *(size_t *)val = ms->elf_notes_max; + return 0; + default: + errno = EINVAL; + return -1; + } } Index: releng/10.1/contrib/file/src/magic.h =================================================================== --- releng/10.1/contrib/file/src/magic.h (revision 284192) +++ releng/10.1/contrib/file/src/magic.h (revision 284193) @@ -1,108 +1,119 @@ /* * Copyright (c) Christos Zoulas 2003. * All Rights Reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #ifndef _MAGIC_H #define _MAGIC_H #include #define MAGIC_NONE 0x000000 /* No flags */ #define MAGIC_DEBUG 0x000001 /* Turn on debugging */ #define MAGIC_SYMLINK 0x000002 /* Follow symlinks */ #define MAGIC_COMPRESS 0x000004 /* Check inside compressed files */ #define MAGIC_DEVICES 0x000008 /* Look at the contents of devices */ #define MAGIC_MIME_TYPE 0x000010 /* Return the MIME type */ #define MAGIC_CONTINUE 0x000020 /* Return all matches */ #define MAGIC_CHECK 0x000040 /* Print warnings to stderr */ #define MAGIC_PRESERVE_ATIME 0x000080 /* Restore access time on exit */ #define MAGIC_RAW 0x000100 /* Don't translate unprintable chars */ #define MAGIC_ERROR 0x000200 /* Handle ENOENT etc as real errors */ #define MAGIC_MIME_ENCODING 0x000400 /* Return the MIME encoding */ #define MAGIC_MIME (MAGIC_MIME_TYPE|MAGIC_MIME_ENCODING) #define MAGIC_APPLE 0x000800 /* Return the Apple creator and type */ #define MAGIC_NO_CHECK_COMPRESS 0x001000 /* Don't check for compressed files */ #define MAGIC_NO_CHECK_TAR 0x002000 /* Don't check for tar files */ #define MAGIC_NO_CHECK_SOFT 0x004000 /* Don't check magic entries */ #define MAGIC_NO_CHECK_APPTYPE 0x008000 /* Don't check application type */ #define MAGIC_NO_CHECK_ELF 0x010000 /* Don't check for elf details */ #define MAGIC_NO_CHECK_TEXT 0x020000 /* Don't check for text files */ #define MAGIC_NO_CHECK_CDF 0x040000 /* Don't check for cdf files */ #define MAGIC_NO_CHECK_TOKENS 0x100000 /* Don't check tokens */ #define MAGIC_NO_CHECK_ENCODING 0x200000 /* Don't check text encodings */ /* No built-in tests; only consult the magic file */ #define MAGIC_NO_CHECK_BUILTIN ( \ MAGIC_NO_CHECK_COMPRESS | \ MAGIC_NO_CHECK_TAR | \ /* MAGIC_NO_CHECK_SOFT | */ \ MAGIC_NO_CHECK_APPTYPE | \ MAGIC_NO_CHECK_ELF | \ MAGIC_NO_CHECK_TEXT | \ MAGIC_NO_CHECK_CDF | \ MAGIC_NO_CHECK_TOKENS | \ MAGIC_NO_CHECK_ENCODING | \ 0 \ ) /* Defined for backwards compatibility (renamed) */ #define MAGIC_NO_CHECK_ASCII MAGIC_NO_CHECK_TEXT /* Defined for backwards compatibility; do nothing */ #define MAGIC_NO_CHECK_FORTRAN 0x000000 /* Don't check ascii/fortran */ #define MAGIC_NO_CHECK_TROFF 0x000000 /* Don't check ascii/troff */ -#define MAGIC_VERSION 518 /* This implementation */ +#define MAGIC_VERSION 521 /* This implementation */ #ifdef __cplusplus extern "C" { #endif typedef struct magic_set *magic_t; magic_t magic_open(int); void magic_close(magic_t); const char *magic_getpath(const char *, int); const char *magic_file(magic_t, const char *); const char *magic_descriptor(magic_t, int); const char *magic_buffer(magic_t, const void *, size_t); const char *magic_error(magic_t); int magic_setflags(magic_t, int); int magic_version(void); int magic_load(magic_t, const char *); +int magic_load_buffers(magic_t, void **, size_t *, size_t); + int magic_compile(magic_t, const char *); int magic_check(magic_t, const char *); int magic_list(magic_t, const char *); int magic_errno(magic_t); + +#define MAGIC_PARAM_INDIR_MAX 0 +#define MAGIC_PARAM_NAME_MAX 1 +#define MAGIC_PARAM_ELF_PHNUM_MAX 2 +#define MAGIC_PARAM_ELF_SHNUM_MAX 3 +#define MAGIC_PARAM_ELF_NOTES_MAX 4 + +int magic_setparam(magic_t, int, const void *); +int magic_getparam(magic_t, int, void *); #ifdef __cplusplus }; #endif #endif /* _MAGIC_H */ Index: releng/10.1/contrib/file/src/magic.h.in =================================================================== --- releng/10.1/contrib/file/src/magic.h.in (revision 284192) +++ releng/10.1/contrib/file/src/magic.h.in (revision 284193) @@ -1,108 +1,119 @@ /* * Copyright (c) Christos Zoulas 2003. * All Rights Reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #ifndef _MAGIC_H #define _MAGIC_H #include #define MAGIC_NONE 0x000000 /* No flags */ #define MAGIC_DEBUG 0x000001 /* Turn on debugging */ #define MAGIC_SYMLINK 0x000002 /* Follow symlinks */ #define MAGIC_COMPRESS 0x000004 /* Check inside compressed files */ #define MAGIC_DEVICES 0x000008 /* Look at the contents of devices */ #define MAGIC_MIME_TYPE 0x000010 /* Return the MIME type */ #define MAGIC_CONTINUE 0x000020 /* Return all matches */ #define MAGIC_CHECK 0x000040 /* Print warnings to stderr */ #define MAGIC_PRESERVE_ATIME 0x000080 /* Restore access time on exit */ #define MAGIC_RAW 0x000100 /* Don't translate unprintable chars */ #define MAGIC_ERROR 0x000200 /* Handle ENOENT etc as real errors */ #define MAGIC_MIME_ENCODING 0x000400 /* Return the MIME encoding */ #define MAGIC_MIME (MAGIC_MIME_TYPE|MAGIC_MIME_ENCODING) #define MAGIC_APPLE 0x000800 /* Return the Apple creator and type */ #define MAGIC_NO_CHECK_COMPRESS 0x001000 /* Don't check for compressed files */ #define MAGIC_NO_CHECK_TAR 0x002000 /* Don't check for tar files */ #define MAGIC_NO_CHECK_SOFT 0x004000 /* Don't check magic entries */ #define MAGIC_NO_CHECK_APPTYPE 0x008000 /* Don't check application type */ #define MAGIC_NO_CHECK_ELF 0x010000 /* Don't check for elf details */ #define MAGIC_NO_CHECK_TEXT 0x020000 /* Don't check for text files */ #define MAGIC_NO_CHECK_CDF 0x040000 /* Don't check for cdf files */ #define MAGIC_NO_CHECK_TOKENS 0x100000 /* Don't check tokens */ #define MAGIC_NO_CHECK_ENCODING 0x200000 /* Don't check text encodings */ /* No built-in tests; only consult the magic file */ #define MAGIC_NO_CHECK_BUILTIN ( \ MAGIC_NO_CHECK_COMPRESS | \ MAGIC_NO_CHECK_TAR | \ /* MAGIC_NO_CHECK_SOFT | */ \ MAGIC_NO_CHECK_APPTYPE | \ MAGIC_NO_CHECK_ELF | \ MAGIC_NO_CHECK_TEXT | \ MAGIC_NO_CHECK_CDF | \ MAGIC_NO_CHECK_TOKENS | \ MAGIC_NO_CHECK_ENCODING | \ 0 \ ) /* Defined for backwards compatibility (renamed) */ #define MAGIC_NO_CHECK_ASCII MAGIC_NO_CHECK_TEXT /* Defined for backwards compatibility; do nothing */ #define MAGIC_NO_CHECK_FORTRAN 0x000000 /* Don't check ascii/fortran */ #define MAGIC_NO_CHECK_TROFF 0x000000 /* Don't check ascii/troff */ #define MAGIC_VERSION X.YY /* This implementation */ #ifdef __cplusplus extern "C" { #endif typedef struct magic_set *magic_t; magic_t magic_open(int); void magic_close(magic_t); const char *magic_getpath(const char *, int); const char *magic_file(magic_t, const char *); const char *magic_descriptor(magic_t, int); const char *magic_buffer(magic_t, const void *, size_t); const char *magic_error(magic_t); int magic_setflags(magic_t, int); int magic_version(void); int magic_load(magic_t, const char *); +int magic_load_buffers(magic_t, void **, size_t *, size_t); + int magic_compile(magic_t, const char *); int magic_check(magic_t, const char *); int magic_list(magic_t, const char *); int magic_errno(magic_t); + +#define MAGIC_PARAM_INDIR_MAX 0 +#define MAGIC_PARAM_NAME_MAX 1 +#define MAGIC_PARAM_ELF_PHNUM_MAX 2 +#define MAGIC_PARAM_ELF_SHNUM_MAX 3 +#define MAGIC_PARAM_ELF_NOTES_MAX 4 + +int magic_setparam(magic_t, int, const void *); +int magic_getparam(magic_t, int, void *); #ifdef __cplusplus }; #endif #endif /* _MAGIC_H */ Index: releng/10.1/contrib/file/src/pread.c =================================================================== --- releng/10.1/contrib/file/src/pread.c (revision 284192) +++ releng/10.1/contrib/file/src/pread.c (revision 284193) @@ -1,14 +1,23 @@ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: pread.c,v 1.2 2013/04/02 16:23:07 christos Exp $") +FILE_RCSID("@(#)$File: pread.c,v 1.3 2014/09/15 19:11:25 christos Exp $") #endif /* lint */ #include #include ssize_t pread(int fd, void *buf, size_t len, off_t off) { - if (lseek(fd, off, SEEK_SET) == (off_t)-1) + off_t old; + ssize_t rv; + + if ((old = lseek(fd, off, SEEK_SET)) == -1) return -1; - return read(fd, buf, len); + if ((rv = read(fd, buf, len)) == -1) + return -1; + + if (lseek(fd, old, SEEK_SET) == -1) + return -1; + + return rv; } Index: releng/10.1/contrib/file/src/readcdf.c =================================================================== --- releng/10.1/contrib/file/src/readcdf.c (revision 284192) +++ releng/10.1/contrib/file/src/readcdf.c (revision 284193) @@ -1,473 +1,522 @@ /*- * Copyright (c) 2008 Christos Zoulas * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGE. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: readcdf.c,v 1.44 2014/05/14 23:22:48 christos Exp $") +FILE_RCSID("@(#)$File: readcdf.c,v 1.49 2014/12/04 15:56:46 christos Exp $") #endif #include #include #include #include #include #include -#if defined(HAVE_LOCALE_H) -#include -#endif #include "cdf.h" #include "magic.h" #define NOTMIME(ms) (((ms)->flags & MAGIC_MIME) == 0) static const struct nv { const char *pattern; const char *mime; } app2mime[] = { { "Word", "msword", }, { "Excel", "vnd.ms-excel", }, { "Powerpoint", "vnd.ms-powerpoint", }, { "Crystal Reports", "x-rpt", }, { "Advanced Installer", "vnd.ms-msi", }, { "InstallShield", "vnd.ms-msi", }, { "Microsoft Patch Compiler", "vnd.ms-msi", }, { "NAnt", "vnd.ms-msi", }, { "Windows Installer", "vnd.ms-msi", }, { NULL, NULL, }, }, name2mime[] = { { "WordDocument", "msword", }, { "PowerPoint", "vnd.ms-powerpoint", }, { "DigitalSignature", "vnd.ms-msi", }, { NULL, NULL, }, }, name2desc[] = { { "WordDocument", "Microsoft Office Word",}, { "PowerPoint", "Microsoft PowerPoint", }, { "DigitalSignature", "Microsoft Installer", }, { NULL, NULL, }, }; static const struct cv { uint64_t clsid[2]; const char *mime; } clsid2mime[] = { { - { 0x00000000000c1084LLU, 0x46000000000000c0LLU }, + { 0x00000000000c1084ULL, 0x46000000000000c0ULL }, "x-msi", }, { { 0, 0 }, NULL, }, }, clsid2desc[] = { { - { 0x00000000000c1084LLU, 0x46000000000000c0LLU }, + { 0x00000000000c1084ULL, 0x46000000000000c0ULL }, "MSI Installer", }, { { 0, 0 }, NULL, }, }; private const char * cdf_clsid_to_mime(const uint64_t clsid[2], const struct cv *cv) { size_t i; for (i = 0; cv[i].mime != NULL; i++) { if (clsid[0] == cv[i].clsid[0] && clsid[1] == cv[i].clsid[1]) return cv[i].mime; } return NULL; } private const char * cdf_app_to_mime(const char *vbuf, const struct nv *nv) { size_t i; const char *rv = NULL; - char *old_lc_ctype; +#ifdef USE_C_LOCALE + locale_t old_lc_ctype, c_lc_ctype; - old_lc_ctype = setlocale(LC_CTYPE, NULL); + c_lc_ctype = newlocale(LC_CTYPE_MASK, "C", 0); + assert(c_lc_ctype != NULL); + old_lc_ctype = uselocale(c_lc_ctype); assert(old_lc_ctype != NULL); - old_lc_ctype = strdup(old_lc_ctype); - assert(old_lc_ctype != NULL); - (void)setlocale(LC_CTYPE, "C"); +#endif for (i = 0; nv[i].pattern != NULL; i++) if (strcasestr(vbuf, nv[i].pattern) != NULL) { rv = nv[i].mime; break; } - (void)setlocale(LC_CTYPE, old_lc_ctype); - free(old_lc_ctype); +#ifdef USE_C_LOCALE + (void)uselocale(old_lc_ctype); + freelocale(c_lc_ctype); +#endif return rv; } private int cdf_file_property_info(struct magic_set *ms, const cdf_property_info_t *info, size_t count, const cdf_directory_t *root_storage) { size_t i; cdf_timestamp_t tp; struct timespec ts; char buf[64]; const char *str = NULL; const char *s; int len; if (!NOTMIME(ms) && root_storage) str = cdf_clsid_to_mime(root_storage->d_storage_uuid, clsid2mime); for (i = 0; i < count; i++) { cdf_print_property_name(buf, sizeof(buf), info[i].pi_id); switch (info[i].pi_type) { case CDF_NULL: break; case CDF_SIGNED16: if (NOTMIME(ms) && file_printf(ms, ", %s: %hd", buf, info[i].pi_s16) == -1) return -1; break; case CDF_SIGNED32: if (NOTMIME(ms) && file_printf(ms, ", %s: %d", buf, info[i].pi_s32) == -1) return -1; break; case CDF_UNSIGNED32: if (NOTMIME(ms) && file_printf(ms, ", %s: %u", buf, info[i].pi_u32) == -1) return -1; break; case CDF_FLOAT: if (NOTMIME(ms) && file_printf(ms, ", %s: %g", buf, info[i].pi_f) == -1) return -1; break; case CDF_DOUBLE: if (NOTMIME(ms) && file_printf(ms, ", %s: %g", buf, info[i].pi_d) == -1) return -1; break; case CDF_LENGTH32_STRING: case CDF_LENGTH32_WSTRING: len = info[i].pi_str.s_len; if (len > 1) { char vbuf[1024]; size_t j, k = 1; if (info[i].pi_type == CDF_LENGTH32_WSTRING) k++; s = info[i].pi_str.s_buf; for (j = 0; j < sizeof(vbuf) && len--; s += k) { if (*s == '\0') break; if (isprint((unsigned char)*s)) vbuf[j++] = *s; } if (j == sizeof(vbuf)) --j; vbuf[j] = '\0'; if (NOTMIME(ms)) { if (vbuf[0]) { if (file_printf(ms, ", %s: %s", buf, vbuf) == -1) return -1; } } else if (str == NULL && info[i].pi_id == CDF_PROPERTY_NAME_OF_APPLICATION) { str = cdf_app_to_mime(vbuf, app2mime); } } break; case CDF_FILETIME: tp = info[i].pi_tp; if (tp != 0) { char tbuf[64]; if (tp < 1000000000000000LL) { cdf_print_elapsed_time(tbuf, sizeof(tbuf), tp); if (NOTMIME(ms) && file_printf(ms, ", %s: %s", buf, tbuf) == -1) return -1; } else { char *c, *ec; cdf_timestamp_to_timespec(&ts, tp); c = cdf_ctime(&ts.tv_sec, tbuf); if (c != NULL && (ec = strchr(c, '\n')) != NULL) *ec = '\0'; if (NOTMIME(ms) && file_printf(ms, ", %s: %s", buf, c) == -1) return -1; } } break; case CDF_CLIPBOARD: break; default: return -1; } } if (!NOTMIME(ms)) { if (str == NULL) return 0; if (file_printf(ms, "application/%s", str) == -1) return -1; } return 1; } private int +cdf_file_catalog(struct magic_set *ms, const cdf_header_t *h, + const cdf_stream_t *sst) +{ + cdf_catalog_t *cat; + size_t i; + char buf[256]; + cdf_catalog_entry_t *ce; + + if (NOTMIME(ms)) { + if (file_printf(ms, "Microsoft Thumbs.db [") == -1) + return -1; + if (cdf_unpack_catalog(h, sst, &cat) == -1) + return -1; + ce = cat->cat_e; + /* skip first entry since it has a , or paren */ + for (i = 1; i < cat->cat_num; i++) + if (file_printf(ms, "%s%s", + cdf_u16tos8(buf, ce[i].ce_namlen, ce[i].ce_name), + i == cat->cat_num - 1 ? "]" : ", ") == -1) { + free(cat); + return -1; + } + free(cat); + } else { + if (file_printf(ms, "application/CDFV2") == -1) + return -1; + } + return 1; +} + +private int cdf_file_summary_info(struct magic_set *ms, const cdf_header_t *h, const cdf_stream_t *sst, const cdf_directory_t *root_storage) { cdf_summary_info_header_t si; cdf_property_info_t *info; size_t count; int m; if (cdf_unpack_summary_info(sst, h, &si, &info, &count) == -1) return -1; if (NOTMIME(ms)) { const char *str; if (file_printf(ms, "Composite Document File V2 Document") == -1) return -1; if (file_printf(ms, ", %s Endian", si.si_byte_order == 0xfffe ? "Little" : "Big") == -1) return -2; switch (si.si_os) { case 2: if (file_printf(ms, ", Os: Windows, Version %d.%d", si.si_os_version & 0xff, (uint32_t)si.si_os_version >> 8) == -1) return -2; break; case 1: if (file_printf(ms, ", Os: MacOS, Version %d.%d", (uint32_t)si.si_os_version >> 8, si.si_os_version & 0xff) == -1) return -2; break; default: if (file_printf(ms, ", Os %d, Version: %d.%d", si.si_os, si.si_os_version & 0xff, (uint32_t)si.si_os_version >> 8) == -1) return -2; break; } if (root_storage) { str = cdf_clsid_to_mime(root_storage->d_storage_uuid, clsid2desc); - if (str) + if (str) { if (file_printf(ms, ", %s", str) == -1) return -2; } } + } m = cdf_file_property_info(ms, info, count, root_storage); free(info); return m == -1 ? -2 : m; } #ifdef notdef private char * format_clsid(char *buf, size_t len, const uint64_t uuid[2]) { snprintf(buf, len, "%.8" PRIx64 "-%.4" PRIx64 "-%.4" PRIx64 "-%.4" PRIx64 "-%.12" PRIx64, - (uuid[0] >> 32) & (uint64_t)0x000000000ffffffffLLU, - (uuid[0] >> 16) & (uint64_t)0x0000000000000ffffLLU, - (uuid[0] >> 0) & (uint64_t)0x0000000000000ffffLLU, - (uuid[1] >> 48) & (uint64_t)0x0000000000000ffffLLU, - (uuid[1] >> 0) & (uint64_t)0x0000fffffffffffffLLU); + (uuid[0] >> 32) & (uint64_t)0x000000000ffffffffULL, + (uuid[0] >> 16) & (uint64_t)0x0000000000000ffffULL, + (uuid[0] >> 0) & (uint64_t)0x0000000000000ffffULL, + (uuid[1] >> 48) & (uint64_t)0x0000000000000ffffULL, + (uuid[1] >> 0) & (uint64_t)0x0000fffffffffffffULL); return buf; } #endif protected int file_trycdf(struct magic_set *ms, int fd, const unsigned char *buf, size_t nbytes) { cdf_info_t info; cdf_header_t h; cdf_sat_t sat, ssat; cdf_stream_t sst, scn; cdf_dir_t dir; int i; const char *expn = ""; const char *corrupt = "corrupt: "; + const cdf_directory_t *root_storage; info.i_fd = fd; info.i_buf = buf; info.i_len = nbytes; if (ms->flags & MAGIC_APPLE) return 0; if (cdf_read_header(&info, &h) == -1) return 0; #ifdef CDF_DEBUG cdf_dump_header(&h); #endif if ((i = cdf_read_sat(&info, &h, &sat)) == -1) { expn = "Can't read SAT"; goto out0; } #ifdef CDF_DEBUG cdf_dump_sat("SAT", &sat, CDF_SEC_SIZE(&h)); #endif if ((i = cdf_read_ssat(&info, &h, &sat, &ssat)) == -1) { expn = "Can't read SSAT"; goto out1; } #ifdef CDF_DEBUG cdf_dump_sat("SSAT", &ssat, CDF_SHORT_SEC_SIZE(&h)); #endif if ((i = cdf_read_dir(&info, &h, &sat, &dir)) == -1) { expn = "Can't read directory"; goto out2; } - const cdf_directory_t *root_storage; if ((i = cdf_read_short_stream(&info, &h, &sat, &dir, &sst, &root_storage)) == -1) { expn = "Cannot read short stream"; goto out3; } #ifdef CDF_DEBUG cdf_dump_dir(&info, &h, &sat, &ssat, &sst, &dir); #endif #ifdef notdef if (root_storage) { if (NOTMIME(ms)) { char clsbuf[128]; if (file_printf(ms, "CLSID %s, ", format_clsid(clsbuf, sizeof(clsbuf), root_storage->d_storage_uuid)) == -1) return -1; } } #endif if ((i = cdf_read_user_stream(&info, &h, &sat, &ssat, &sst, &dir, "FileHeader", &scn)) != -1) { #define HWP5_SIGNATURE "HWP Document File" if (scn.sst_dirlen >= sizeof(HWP5_SIGNATURE) - 1 && memcmp(scn.sst_tab, HWP5_SIGNATURE, sizeof(HWP5_SIGNATURE) - 1) == 0) { if (NOTMIME(ms)) { if (file_printf(ms, "Hangul (Korean) Word Processor File 5.x") == -1) return -1; } else { if (file_printf(ms, "application/x-hwp") == -1) return -1; } i = 1; goto out5; } else { free(scn.sst_tab); scn.sst_tab = NULL; scn.sst_len = 0; scn.sst_dirlen = 0; } } if ((i = cdf_read_summary_info(&info, &h, &sat, &ssat, &sst, &dir, &scn)) == -1) { if (errno == ESRCH) { - corrupt = expn; - expn = "No summary info"; + if ((i = cdf_read_catalog(&info, &h, &sat, &ssat, &sst, + &dir, &scn)) == -1) { + corrupt = expn; + if ((i = cdf_read_encrypted_package(&info, &h, + &sat, &ssat, &sst, &dir, &scn)) == -1) + expn = "No summary info"; + else { + expn = "Encrypted"; + i = -1; + } + goto out4; + } +#ifdef CDF_DEBUG + cdf_dump_catalog(&h, &scn); +#endif + if ((i = cdf_file_catalog(ms, &h, &scn)) + < 0) + expn = "Can't expand catalog"; } else { expn = "Cannot read summary info"; } goto out4; } #ifdef CDF_DEBUG cdf_dump_summary_info(&h, &scn); #endif if ((i = cdf_file_summary_info(ms, &h, &scn, root_storage)) < 0) expn = "Can't expand summary_info"; if (i == 0) { const char *str = NULL; cdf_directory_t *d; char name[__arraycount(d->d_name)]; size_t j, k; for (j = 0; str == NULL && j < dir.dir_len; j++) { d = &dir.dir_tab[j]; for (k = 0; k < sizeof(name); k++) name[k] = (char)cdf_tole2(d->d_name[k]); str = cdf_app_to_mime(name, NOTMIME(ms) ? name2desc : name2mime); } if (NOTMIME(ms)) { if (str != NULL) { if (file_printf(ms, "%s", str) == -1) return -1; i = 1; } } else { if (str == NULL) str = "vnd.ms-office"; if (file_printf(ms, "application/%s", str) == -1) return -1; i = 1; } } out5: free(scn.sst_tab); out4: free(sst.sst_tab); out3: free(dir.dir_tab); out2: free(ssat.sat_tab); out1: free(sat.sat_tab); out0: if (i == -1) { if (NOTMIME(ms)) { if (file_printf(ms, "Composite Document File V2 Document") == -1) return -1; if (*expn) if (file_printf(ms, ", %s%s", corrupt, expn) == -1) return -1; } else { - if (file_printf(ms, "application/CDFV2-corrupt") == -1) + if (file_printf(ms, "application/CDFV2-%s", + *corrupt ? "corrupt" : "encrypted") == -1) return -1; } i = 1; } return i; } Index: releng/10.1/contrib/file/src/readelf.c =================================================================== --- releng/10.1/contrib/file/src/readelf.c (revision 284192) +++ releng/10.1/contrib/file/src/readelf.c (revision 284193) @@ -1,1310 +1,1405 @@ /* * Copyright (c) Christos Zoulas 2003. * All Rights Reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: readelf.c,v 1.103 2014/05/02 02:25:10 christos Exp $") +FILE_RCSID("@(#)$File: readelf.c,v 1.117 2014/12/16 23:29:42 christos Exp $") #endif #ifdef BUILTIN_ELF #include #include #include #ifdef HAVE_UNISTD_H #include #endif #include "readelf.h" #include "magic.h" #ifdef ELFCORE private int dophn_core(struct magic_set *, int, int, int, off_t, int, size_t, - off_t, int *); + off_t, int *, uint16_t *); #endif private int dophn_exec(struct magic_set *, int, int, int, off_t, int, size_t, - off_t, int *, int); + off_t, int, int *, uint16_t *); private int doshn(struct magic_set *, int, int, int, off_t, int, size_t, - off_t, int *, int, int); + off_t, int, int, int *, uint16_t *); private size_t donote(struct magic_set *, void *, size_t, size_t, int, - int, size_t, int *); + int, size_t, int *, uint16_t *); #define ELF_ALIGN(a) ((((a) + align - 1) / align) * align) #define isquote(c) (strchr("'\"`", (c)) != NULL) private uint16_t getu16(int, uint16_t); private uint32_t getu32(int, uint32_t); private uint64_t getu64(int, uint64_t); -#define MAX_PHNUM 256 -#define MAX_SHNUM 1024 +#define MAX_PHNUM 128 +#define MAX_SHNUM 32768 +#define SIZE_UNKNOWN ((off_t)-1) private int toomany(struct magic_set *ms, const char *name, uint16_t num) { - if (file_printf(ms, ", too many %s header sections (%u)", name, num + if (file_printf(ms, ", too many %s (%u)", name, num ) == -1) return -1; return 0; } private uint16_t getu16(int swap, uint16_t value) { union { uint16_t ui; char c[2]; } retval, tmpval; if (swap) { tmpval.ui = value; retval.c[0] = tmpval.c[1]; retval.c[1] = tmpval.c[0]; return retval.ui; } else return value; } private uint32_t getu32(int swap, uint32_t value) { union { uint32_t ui; char c[4]; } retval, tmpval; if (swap) { tmpval.ui = value; retval.c[0] = tmpval.c[3]; retval.c[1] = tmpval.c[2]; retval.c[2] = tmpval.c[1]; retval.c[3] = tmpval.c[0]; return retval.ui; } else return value; } private uint64_t getu64(int swap, uint64_t value) { union { uint64_t ui; char c[8]; } retval, tmpval; if (swap) { tmpval.ui = value; retval.c[0] = tmpval.c[7]; retval.c[1] = tmpval.c[6]; retval.c[2] = tmpval.c[5]; retval.c[3] = tmpval.c[4]; retval.c[4] = tmpval.c[3]; retval.c[5] = tmpval.c[2]; retval.c[6] = tmpval.c[1]; retval.c[7] = tmpval.c[0]; return retval.ui; } else return value; } #define elf_getu16(swap, value) getu16(swap, value) #define elf_getu32(swap, value) getu32(swap, value) #define elf_getu64(swap, value) getu64(swap, value) #define xsh_addr (clazz == ELFCLASS32 \ ? (void *)&sh32 \ : (void *)&sh64) #define xsh_sizeof (clazz == ELFCLASS32 \ ? sizeof(sh32) \ : sizeof(sh64)) #define xsh_size (size_t)(clazz == ELFCLASS32 \ ? elf_getu32(swap, sh32.sh_size) \ : elf_getu64(swap, sh64.sh_size)) #define xsh_offset (off_t)(clazz == ELFCLASS32 \ ? elf_getu32(swap, sh32.sh_offset) \ : elf_getu64(swap, sh64.sh_offset)) #define xsh_type (clazz == ELFCLASS32 \ ? elf_getu32(swap, sh32.sh_type) \ : elf_getu32(swap, sh64.sh_type)) #define xsh_name (clazz == ELFCLASS32 \ ? elf_getu32(swap, sh32.sh_name) \ : elf_getu32(swap, sh64.sh_name)) #define xph_addr (clazz == ELFCLASS32 \ ? (void *) &ph32 \ : (void *) &ph64) #define xph_sizeof (clazz == ELFCLASS32 \ ? sizeof(ph32) \ : sizeof(ph64)) #define xph_type (clazz == ELFCLASS32 \ ? elf_getu32(swap, ph32.p_type) \ : elf_getu32(swap, ph64.p_type)) #define xph_offset (off_t)(clazz == ELFCLASS32 \ ? elf_getu32(swap, ph32.p_offset) \ : elf_getu64(swap, ph64.p_offset)) #define xph_align (size_t)((clazz == ELFCLASS32 \ ? (off_t) (ph32.p_align ? \ elf_getu32(swap, ph32.p_align) : 4) \ : (off_t) (ph64.p_align ? \ elf_getu64(swap, ph64.p_align) : 4))) #define xph_filesz (size_t)((clazz == ELFCLASS32 \ ? elf_getu32(swap, ph32.p_filesz) \ : elf_getu64(swap, ph64.p_filesz))) #define xnh_addr (clazz == ELFCLASS32 \ ? (void *)&nh32 \ : (void *)&nh64) #define xph_memsz (size_t)((clazz == ELFCLASS32 \ ? elf_getu32(swap, ph32.p_memsz) \ : elf_getu64(swap, ph64.p_memsz))) #define xnh_sizeof (clazz == ELFCLASS32 \ ? sizeof nh32 \ : sizeof nh64) #define xnh_type (clazz == ELFCLASS32 \ ? elf_getu32(swap, nh32.n_type) \ : elf_getu32(swap, nh64.n_type)) #define xnh_namesz (clazz == ELFCLASS32 \ ? elf_getu32(swap, nh32.n_namesz) \ : elf_getu32(swap, nh64.n_namesz)) #define xnh_descsz (clazz == ELFCLASS32 \ ? elf_getu32(swap, nh32.n_descsz) \ : elf_getu32(swap, nh64.n_descsz)) #define prpsoffsets(i) (clazz == ELFCLASS32 \ ? prpsoffsets32[i] \ : prpsoffsets64[i]) #define xcap_addr (clazz == ELFCLASS32 \ ? (void *)&cap32 \ : (void *)&cap64) #define xcap_sizeof (clazz == ELFCLASS32 \ ? sizeof cap32 \ : sizeof cap64) #define xcap_tag (clazz == ELFCLASS32 \ ? elf_getu32(swap, cap32.c_tag) \ : elf_getu64(swap, cap64.c_tag)) #define xcap_val (clazz == ELFCLASS32 \ ? elf_getu32(swap, cap32.c_un.c_val) \ : elf_getu64(swap, cap64.c_un.c_val)) #ifdef ELFCORE /* * Try larger offsets first to avoid false matches * from earlier data that happen to look like strings. */ static const size_t prpsoffsets32[] = { #ifdef USE_NT_PSINFO 104, /* SunOS 5.x (command line) */ 88, /* SunOS 5.x (short name) */ #endif /* USE_NT_PSINFO */ 100, /* SunOS 5.x (command line) */ 84, /* SunOS 5.x (short name) */ 44, /* Linux (command line) */ 28, /* Linux 2.0.36 (short name) */ 8, /* FreeBSD */ }; static const size_t prpsoffsets64[] = { #ifdef USE_NT_PSINFO 152, /* SunOS 5.x (command line) */ 136, /* SunOS 5.x (short name) */ #endif /* USE_NT_PSINFO */ 136, /* SunOS 5.x, 64-bit (command line) */ 120, /* SunOS 5.x, 64-bit (short name) */ 56, /* Linux (command line) */ 40, /* Linux (tested on core from 2.4.x, short name) */ 16, /* FreeBSD, 64-bit */ }; #define NOFFSETS32 (sizeof prpsoffsets32 / sizeof prpsoffsets32[0]) #define NOFFSETS64 (sizeof prpsoffsets64 / sizeof prpsoffsets64[0]) #define NOFFSETS (clazz == ELFCLASS32 ? NOFFSETS32 : NOFFSETS64) /* * Look through the program headers of an executable image, searching * for a PT_NOTE section of type NT_PRPSINFO, with a name "CORE" or * "FreeBSD"; if one is found, try looking in various places in its * contents for a 16-character string containing only printable * characters - if found, that string should be the name of the program * that dropped core. Note: right after that 16-character string is, * at least in SunOS 5.x (and possibly other SVR4-flavored systems) and * Linux, a longer string (80 characters, in 5.x, probably other * SVR4-flavored systems, and Linux) containing the start of the * command line for that program. * * SunOS 5.x core files contain two PT_NOTE sections, with the types * NT_PRPSINFO (old) and NT_PSINFO (new). These structs contain the * same info about the command name and command line, so it probably * isn't worthwhile to look for NT_PSINFO, but the offsets are provided * above (see USE_NT_PSINFO), in case we ever decide to do so. The * NT_PRPSINFO and NT_PSINFO sections are always in order and adjacent; * the SunOS 5.x file command relies on this (and prefers the latter). * * The signal number probably appears in a section of type NT_PRSTATUS, * but that's also rather OS-dependent, in ways that are harder to * dissect with heuristics, so I'm not bothering with the signal number. * (I suppose the signal number could be of interest in situations where * you don't have the binary of the program that dropped core; if you * *do* have that binary, the debugger will probably tell you what * signal it was.) */ #define OS_STYLE_SVR4 0 #define OS_STYLE_FREEBSD 1 #define OS_STYLE_NETBSD 2 private const char os_style_names[][8] = { "SVR4", "FreeBSD", "NetBSD", }; -#define FLAGS_DID_CORE 0x01 -#define FLAGS_DID_NOTE 0x02 -#define FLAGS_DID_BUILD_ID 0x04 -#define FLAGS_DID_CORE_STYLE 0x08 -#define FLAGS_IS_CORE 0x10 +#define FLAGS_DID_CORE 0x001 +#define FLAGS_DID_OS_NOTE 0x002 +#define FLAGS_DID_BUILD_ID 0x004 +#define FLAGS_DID_CORE_STYLE 0x008 +#define FLAGS_DID_NETBSD_PAX 0x010 +#define FLAGS_DID_NETBSD_MARCH 0x020 +#define FLAGS_DID_NETBSD_CMODEL 0x040 +#define FLAGS_DID_NETBSD_UNKNOWN 0x080 +#define FLAGS_IS_CORE 0x100 private int dophn_core(struct magic_set *ms, int clazz, int swap, int fd, off_t off, - int num, size_t size, off_t fsize, int *flags) + int num, size_t size, off_t fsize, int *flags, uint16_t *notecount) { Elf32_Phdr ph32; Elf64_Phdr ph64; size_t offset, len; unsigned char nbuf[BUFSIZ]; ssize_t bufsize; if (size != xph_sizeof) { if (file_printf(ms, ", corrupted program header size") == -1) return -1; return 0; } /* * Loop through all the program headers. */ for ( ; num; num--) { - if (pread(fd, xph_addr, xph_sizeof, off) == -1) { + if (pread(fd, xph_addr, xph_sizeof, off) < (ssize_t)xph_sizeof) { file_badread(ms); return -1; } off += size; - if (xph_offset > fsize) { + if (fsize != SIZE_UNKNOWN && xph_offset > fsize) { /* Perhaps warn here */ continue; } if (xph_type != PT_NOTE) continue; /* * This is a PT_NOTE section; loop through all the notes * in the section. */ len = xph_filesz < sizeof(nbuf) ? xph_filesz : sizeof(nbuf); if ((bufsize = pread(fd, nbuf, len, xph_offset)) == -1) { file_badread(ms); return -1; } offset = 0; for (;;) { if (offset >= (size_t)bufsize) break; offset = donote(ms, nbuf, offset, (size_t)bufsize, - clazz, swap, 4, flags); + clazz, swap, 4, flags, notecount); if (offset == 0) break; } } return 0; } #endif static void do_note_netbsd_version(struct magic_set *ms, int swap, void *v) { uint32_t desc; (void)memcpy(&desc, v, sizeof(desc)); desc = elf_getu32(swap, desc); if (file_printf(ms, ", for NetBSD") == -1) return; /* * The version number used to be stuck as 199905, and was thus * basically content-free. Newer versions of NetBSD have fixed * this and now use the encoding of __NetBSD_Version__: * * MMmmrrpp00 * * M = major version * m = minor version * r = release ["",A-Z,Z[A-Z] but numeric] * p = patchlevel */ if (desc > 100000000U) { uint32_t ver_patch = (desc / 100) % 100; uint32_t ver_rel = (desc / 10000) % 100; uint32_t ver_min = (desc / 1000000) % 100; uint32_t ver_maj = desc / 100000000; if (file_printf(ms, " %u.%u", ver_maj, ver_min) == -1) return; if (ver_rel == 0 && ver_patch != 0) { if (file_printf(ms, ".%u", ver_patch) == -1) return; } else if (ver_rel != 0) { while (ver_rel > 26) { if (file_printf(ms, "Z") == -1) return; ver_rel -= 26; } if (file_printf(ms, "%c", 'A' + ver_rel - 1) == -1) return; } } } static void do_note_freebsd_version(struct magic_set *ms, int swap, void *v) { uint32_t desc; (void)memcpy(&desc, v, sizeof(desc)); desc = elf_getu32(swap, desc); if (file_printf(ms, ", for FreeBSD") == -1) return; /* * Contents is __FreeBSD_version, whose relation to OS * versions is defined by a huge table in the Porter's * Handbook. This is the general scheme: * * Releases: * Mmp000 (before 4.10) * Mmi0p0 (before 5.0) * Mmm0p0 * * Development branches: * Mmpxxx (before 4.6) * Mmp1xx (before 4.10) * Mmi1xx (before 5.0) * M000xx (pre-M.0) * Mmm1xx * * M = major version * m = minor version * i = minor version increment (491000 -> 4.10) * p = patchlevel * x = revision * * The first release of FreeBSD to use ELF by default * was version 3.0. */ if (desc == 460002) { if (file_printf(ms, " 4.6.2") == -1) return; } else if (desc < 460100) { if (file_printf(ms, " %d.%d", desc / 100000, desc / 10000 % 10) == -1) return; if (desc / 1000 % 10 > 0) if (file_printf(ms, ".%d", desc / 1000 % 10) == -1) return; if ((desc % 1000 > 0) || (desc % 100000 == 0)) if (file_printf(ms, " (%d)", desc) == -1) return; } else if (desc < 500000) { if (file_printf(ms, " %d.%d", desc / 100000, desc / 10000 % 10 + desc / 1000 % 10) == -1) return; if (desc / 100 % 10 > 0) { if (file_printf(ms, " (%d)", desc) == -1) return; } else if (desc / 10 % 10 > 0) { if (file_printf(ms, ".%d", desc / 10 % 10) == -1) return; } } else { if (file_printf(ms, " %d.%d", desc / 100000, desc / 1000 % 100) == -1) return; if ((desc / 100 % 10 > 0) || (desc % 100000 / 100 == 0)) { if (file_printf(ms, " (%d)", desc) == -1) return; } else if (desc / 10 % 10 > 0) { if (file_printf(ms, ".%d", desc / 10 % 10) == -1) return; } } } -private size_t -donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size, - int clazz, int swap, size_t align, int *flags) +private int +do_bid_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, + int swap __attribute__((__unused__)), uint32_t namesz, uint32_t descsz, + size_t noff, size_t doff, int *flags) { - Elf32_Nhdr nh32; - Elf64_Nhdr nh64; - size_t noff, doff; -#ifdef ELFCORE - int os_style = -1; -#endif - uint32_t namesz, descsz; - unsigned char *nbuf = CAST(unsigned char *, vbuf); - - if (xnh_sizeof + offset > size) { - /* - * We're out of note headers. - */ - return xnh_sizeof + offset; + if (namesz == 4 && strcmp((char *)&nbuf[noff], "GNU") == 0 && + type == NT_GNU_BUILD_ID && (descsz == 16 || descsz == 20)) { + uint8_t desc[20]; + uint32_t i; + *flags |= FLAGS_DID_BUILD_ID; + if (file_printf(ms, ", BuildID[%s]=", descsz == 16 ? "md5/uuid" : + "sha1") == -1) + return 1; + (void)memcpy(desc, &nbuf[doff], descsz); + for (i = 0; i < descsz; i++) + if (file_printf(ms, "%02x", desc[i]) == -1) + return 1; + return 1; } + return 0; +} - (void)memcpy(xnh_addr, &nbuf[offset], xnh_sizeof); - offset += xnh_sizeof; - - namesz = xnh_namesz; - descsz = xnh_descsz; - if ((namesz == 0) && (descsz == 0)) { - /* - * We're out of note headers. - */ - return (offset >= size) ? offset : size; - } - - if (namesz & 0x80000000) { - (void)file_printf(ms, ", bad note name size 0x%lx", - (unsigned long)namesz); - return 0; - } - - if (descsz & 0x80000000) { - (void)file_printf(ms, ", bad note description size 0x%lx", - (unsigned long)descsz); - return 0; - } - - - noff = offset; - doff = ELF_ALIGN(offset + namesz); - - if (offset + namesz > size) { - /* - * We're past the end of the buffer. - */ - return doff; - } - - offset = ELF_ALIGN(doff + descsz); - if (doff + descsz > size) { - /* - * We're past the end of the buffer. - */ - return (offset >= size) ? offset : size; - } - - if ((*flags & (FLAGS_DID_NOTE|FLAGS_DID_BUILD_ID)) == - (FLAGS_DID_NOTE|FLAGS_DID_BUILD_ID)) - goto core; - +private int +do_os_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, + int swap, uint32_t namesz, uint32_t descsz, + size_t noff, size_t doff, int *flags) +{ if (namesz == 5 && strcmp((char *)&nbuf[noff], "SuSE") == 0 && - xnh_type == NT_GNU_VERSION && descsz == 2) { + type == NT_GNU_VERSION && descsz == 2) { + *flags |= FLAGS_DID_OS_NOTE; file_printf(ms, ", for SuSE %d.%d", nbuf[doff], nbuf[doff + 1]); + return 1; } + if (namesz == 4 && strcmp((char *)&nbuf[noff], "GNU") == 0 && - xnh_type == NT_GNU_VERSION && descsz == 16) { + type == NT_GNU_VERSION && descsz == 16) { uint32_t desc[4]; (void)memcpy(desc, &nbuf[doff], sizeof(desc)); + *flags |= FLAGS_DID_OS_NOTE; if (file_printf(ms, ", for GNU/") == -1) - return size; + return 1; switch (elf_getu32(swap, desc[0])) { case GNU_OS_LINUX: if (file_printf(ms, "Linux") == -1) - return size; + return 1; break; case GNU_OS_HURD: if (file_printf(ms, "Hurd") == -1) - return size; + return 1; break; case GNU_OS_SOLARIS: if (file_printf(ms, "Solaris") == -1) - return size; + return 1; break; case GNU_OS_KFREEBSD: if (file_printf(ms, "kFreeBSD") == -1) - return size; + return 1; break; case GNU_OS_KNETBSD: if (file_printf(ms, "kNetBSD") == -1) - return size; + return 1; break; default: if (file_printf(ms, "") == -1) - return size; + return 1; } if (file_printf(ms, " %d.%d.%d", elf_getu32(swap, desc[1]), elf_getu32(swap, desc[2]), elf_getu32(swap, desc[3])) == -1) - return size; - *flags |= FLAGS_DID_NOTE; - return size; + return 1; + return 1; } - if (namesz == 4 && strcmp((char *)&nbuf[noff], "GNU") == 0 && - xnh_type == NT_GNU_BUILD_ID && (descsz == 16 || descsz == 20)) { - uint8_t desc[20]; - uint32_t i; - if (file_printf(ms, ", BuildID[%s]=", descsz == 16 ? "md5/uuid" : - "sha1") == -1) - return size; - (void)memcpy(desc, &nbuf[doff], descsz); - for (i = 0; i < descsz; i++) - if (file_printf(ms, "%02x", desc[i]) == -1) - return size; - *flags |= FLAGS_DID_BUILD_ID; + if (namesz == 7 && strcmp((char *)&nbuf[noff], "NetBSD") == 0) { + if (type == NT_NETBSD_VERSION && descsz == 4) { + *flags |= FLAGS_DID_OS_NOTE; + do_note_netbsd_version(ms, swap, &nbuf[doff]); + return 1; + } } + if (namesz == 8 && strcmp((char *)&nbuf[noff], "FreeBSD") == 0) { + if (type == NT_FREEBSD_VERSION && descsz == 4) { + *flags |= FLAGS_DID_OS_NOTE; + do_note_freebsd_version(ms, swap, &nbuf[doff]); + return 1; + } + } + + if (namesz == 8 && strcmp((char *)&nbuf[noff], "OpenBSD") == 0 && + type == NT_OPENBSD_VERSION && descsz == 4) { + *flags |= FLAGS_DID_OS_NOTE; + if (file_printf(ms, ", for OpenBSD") == -1) + return 1; + /* Content of note is always 0 */ + return 1; + } + + if (namesz == 10 && strcmp((char *)&nbuf[noff], "DragonFly") == 0 && + type == NT_DRAGONFLY_VERSION && descsz == 4) { + uint32_t desc; + *flags |= FLAGS_DID_OS_NOTE; + if (file_printf(ms, ", for DragonFly") == -1) + return 1; + (void)memcpy(&desc, &nbuf[doff], sizeof(desc)); + desc = elf_getu32(swap, desc); + if (file_printf(ms, " %d.%d.%d", desc / 100000, + desc / 10000 % 10, desc % 10000) == -1) + return 1; + return 1; + } + return 0; +} + +private int +do_pax_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, + int swap, uint32_t namesz, uint32_t descsz, + size_t noff, size_t doff, int *flags) +{ if (namesz == 4 && strcmp((char *)&nbuf[noff], "PaX") == 0 && - xnh_type == NT_NETBSD_PAX && descsz == 4) { + type == NT_NETBSD_PAX && descsz == 4) { static const char *pax[] = { "+mprotect", "-mprotect", "+segvguard", "-segvguard", "+ASLR", "-ASLR", }; uint32_t desc; size_t i; int did = 0; + *flags |= FLAGS_DID_NETBSD_PAX; (void)memcpy(&desc, &nbuf[doff], sizeof(desc)); desc = elf_getu32(swap, desc); if (desc && file_printf(ms, ", PaX: ") == -1) - return size; + return 1; for (i = 0; i < __arraycount(pax); i++) { if (((1 << i) & desc) == 0) continue; if (file_printf(ms, "%s%s", did++ ? "," : "", pax[i]) == -1) - return size; + return 1; } + return 1; } + return 0; +} - if (namesz == 7 && strcmp((char *)&nbuf[noff], "NetBSD") == 0) { - switch (xnh_type) { - case NT_NETBSD_VERSION: - if (descsz == 4) { - do_note_netbsd_version(ms, swap, &nbuf[doff]); - *flags |= FLAGS_DID_NOTE; - return size; - } - break; - case NT_NETBSD_MARCH: - if (file_printf(ms, ", compiled for: %.*s", (int)descsz, - (const char *)&nbuf[doff]) == -1) - return size; - break; - case NT_NETBSD_CMODEL: - if (file_printf(ms, ", compiler model: %.*s", - (int)descsz, (const char *)&nbuf[doff]) == -1) - return size; - break; - default: - if (file_printf(ms, ", note=%u", xnh_type) == -1) - return size; - break; - } - return size; - } - - if (namesz == 8 && strcmp((char *)&nbuf[noff], "FreeBSD") == 0) { - if (xnh_type == NT_FREEBSD_VERSION && descsz == 4) { - do_note_freebsd_version(ms, swap, &nbuf[doff]); - *flags |= FLAGS_DID_NOTE; - return size; - } - } - - if (namesz == 8 && strcmp((char *)&nbuf[noff], "OpenBSD") == 0 && - xnh_type == NT_OPENBSD_VERSION && descsz == 4) { - if (file_printf(ms, ", for OpenBSD") == -1) - return size; - /* Content of note is always 0 */ - *flags |= FLAGS_DID_NOTE; - return size; - } - - if (namesz == 10 && strcmp((char *)&nbuf[noff], "DragonFly") == 0 && - xnh_type == NT_DRAGONFLY_VERSION && descsz == 4) { - uint32_t desc; - if (file_printf(ms, ", for DragonFly") == -1) - return size; - (void)memcpy(&desc, &nbuf[doff], sizeof(desc)); - desc = elf_getu32(swap, desc); - if (file_printf(ms, " %d.%d.%d", desc / 100000, - desc / 10000 % 10, desc % 10000) == -1) - return size; - *flags |= FLAGS_DID_NOTE; - return size; - } - -core: +private int +do_core_note(struct magic_set *ms, unsigned char *nbuf, uint32_t type, + int swap, uint32_t namesz, uint32_t descsz, + size_t noff, size_t doff, int *flags, size_t size, int clazz) +{ +#ifdef ELFCORE + int os_style = -1; /* * Sigh. The 2.0.36 kernel in Debian 2.1, at * least, doesn't correctly implement name * sections, in core dumps, as specified by * the "Program Linking" section of "UNIX(R) System * V Release 4 Programmer's Guide: ANSI C and * Programming Support Tools", because my copy * clearly says "The first 'namesz' bytes in 'name' * contain a *null-terminated* [emphasis mine] * character representation of the entry's owner * or originator", but the 2.0.36 kernel code * doesn't include the terminating null in the * name.... */ if ((namesz == 4 && strncmp((char *)&nbuf[noff], "CORE", 4) == 0) || (namesz == 5 && strcmp((char *)&nbuf[noff], "CORE") == 0)) { os_style = OS_STYLE_SVR4; } if ((namesz == 8 && strcmp((char *)&nbuf[noff], "FreeBSD") == 0)) { os_style = OS_STYLE_FREEBSD; } if ((namesz >= 11 && strncmp((char *)&nbuf[noff], "NetBSD-CORE", 11) == 0)) { os_style = OS_STYLE_NETBSD; } -#ifdef ELFCORE - if ((*flags & FLAGS_DID_CORE) != 0) - return size; - if (os_style != -1 && (*flags & FLAGS_DID_CORE_STYLE) == 0) { if (file_printf(ms, ", %s-style", os_style_names[os_style]) == -1) - return size; + return 1; *flags |= FLAGS_DID_CORE_STYLE; } switch (os_style) { case OS_STYLE_NETBSD: - if (xnh_type == NT_NETBSD_CORE_PROCINFO) { + if (type == NT_NETBSD_CORE_PROCINFO) { + char sbuf[512]; uint32_t signo; /* * Extract the program name. It is at * offset 0x7c, and is up to 32-bytes, * including the terminating NUL. */ if (file_printf(ms, ", from '%.31s'", - &nbuf[doff + 0x7c]) == -1) - return size; + file_printable(sbuf, sizeof(sbuf), + (const char *)&nbuf[doff + 0x7c])) == -1) + return 1; /* * Extract the signal number. It is at * offset 0x08. */ (void)memcpy(&signo, &nbuf[doff + 0x08], sizeof(signo)); if (file_printf(ms, " (signal %u)", elf_getu32(swap, signo)) == -1) - return size; + return 1; *flags |= FLAGS_DID_CORE; - return size; + return 1; } break; default: - if (xnh_type == NT_PRPSINFO && *flags & FLAGS_IS_CORE) { + if (type == NT_PRPSINFO && *flags & FLAGS_IS_CORE) { size_t i, j; unsigned char c; /* * Extract the program name. We assume * it to be 16 characters (that's what it * is in SunOS 5.x and Linux). * * Unfortunately, it's at a different offset * in various OSes, so try multiple offsets. * If the characters aren't all printable, * reject it. */ for (i = 0; i < NOFFSETS; i++) { unsigned char *cname, *cp; size_t reloffset = prpsoffsets(i); size_t noffset = doff + reloffset; size_t k; for (j = 0; j < 16; j++, noffset++, reloffset++) { /* * Make sure we're not past * the end of the buffer; if * we are, just give up. */ if (noffset >= size) goto tryanother; /* * Make sure we're not past * the end of the contents; * if we are, this obviously * isn't the right offset. */ if (reloffset >= descsz) goto tryanother; c = nbuf[noffset]; if (c == '\0') { /* * A '\0' at the * beginning is * obviously wrong. * Any other '\0' * means we're done. */ if (j == 0) goto tryanother; else break; } else { /* * A nonprintable * character is also * wrong. */ if (!isprint(c) || isquote(c)) goto tryanother; } } /* * Well, that worked. */ /* * Try next offsets, in case this match is * in the middle of a string. */ - for (k = i + 1 ; k < NOFFSETS ; k++) { + for (k = i + 1 ; k < NOFFSETS; k++) { size_t no; int adjust = 1; if (prpsoffsets(k) >= prpsoffsets(i)) continue; for (no = doff + prpsoffsets(k); no < doff + prpsoffsets(i); no++) adjust = adjust && isprint(nbuf[no]); if (adjust) i = k; } cname = (unsigned char *) &nbuf[doff + prpsoffsets(i)]; for (cp = cname; *cp && isprint(*cp); cp++) continue; /* * Linux apparently appends a space at the end * of the command line: remove it. */ while (cp > cname && isspace(cp[-1])) cp--; if (file_printf(ms, ", from '%.*s'", (int)(cp - cname), cname) == -1) - return size; + return 1; *flags |= FLAGS_DID_CORE; - return size; + return 1; tryanother: ; } } break; } #endif + return 0; +} + +private size_t +donote(struct magic_set *ms, void *vbuf, size_t offset, size_t size, + int clazz, int swap, size_t align, int *flags, uint16_t *notecount) +{ + Elf32_Nhdr nh32; + Elf64_Nhdr nh64; + size_t noff, doff; + uint32_t namesz, descsz; + unsigned char *nbuf = CAST(unsigned char *, vbuf); + + if (*notecount == 0) + return 0; + --*notecount; + + if (xnh_sizeof + offset > size) { + /* + * We're out of note headers. + */ + return xnh_sizeof + offset; + } + + (void)memcpy(xnh_addr, &nbuf[offset], xnh_sizeof); + offset += xnh_sizeof; + + namesz = xnh_namesz; + descsz = xnh_descsz; + if ((namesz == 0) && (descsz == 0)) { + /* + * We're out of note headers. + */ + return (offset >= size) ? offset : size; + } + + if (namesz & 0x80000000) { + (void)file_printf(ms, ", bad note name size 0x%lx", + (unsigned long)namesz); + return 0; + } + + if (descsz & 0x80000000) { + (void)file_printf(ms, ", bad note description size 0x%lx", + (unsigned long)descsz); + return 0; + } + + noff = offset; + doff = ELF_ALIGN(offset + namesz); + + if (offset + namesz > size) { + /* + * We're past the end of the buffer. + */ + return doff; + } + + offset = ELF_ALIGN(doff + descsz); + if (doff + descsz > size) { + /* + * We're past the end of the buffer. + */ + return (offset >= size) ? offset : size; + } + + if ((*flags & FLAGS_DID_OS_NOTE) == 0) { + if (do_os_note(ms, nbuf, xnh_type, swap, + namesz, descsz, noff, doff, flags)) + return size; + } + + if ((*flags & FLAGS_DID_BUILD_ID) == 0) { + if (do_bid_note(ms, nbuf, xnh_type, swap, + namesz, descsz, noff, doff, flags)) + return size; + } + + if ((*flags & FLAGS_DID_NETBSD_PAX) == 0) { + if (do_pax_note(ms, nbuf, xnh_type, swap, + namesz, descsz, noff, doff, flags)) + return size; + } + + if ((*flags & FLAGS_DID_CORE) == 0) { + if (do_core_note(ms, nbuf, xnh_type, swap, + namesz, descsz, noff, doff, flags, size, clazz)) + return size; + } + + if (namesz == 7 && strcmp((char *)&nbuf[noff], "NetBSD") == 0) { + if (descsz > 100) + descsz = 100; + switch (xnh_type) { + case NT_NETBSD_VERSION: + return size; + case NT_NETBSD_MARCH: + if (*flags & FLAGS_DID_NETBSD_MARCH) + return size; + *flags |= FLAGS_DID_NETBSD_MARCH; + if (file_printf(ms, ", compiled for: %.*s", + (int)descsz, (const char *)&nbuf[doff]) == -1) + return size; + break; + case NT_NETBSD_CMODEL: + if (*flags & FLAGS_DID_NETBSD_CMODEL) + return size; + *flags |= FLAGS_DID_NETBSD_CMODEL; + if (file_printf(ms, ", compiler model: %.*s", + (int)descsz, (const char *)&nbuf[doff]) == -1) + return size; + break; + default: + if (*flags & FLAGS_DID_NETBSD_UNKNOWN) + return size; + *flags |= FLAGS_DID_NETBSD_UNKNOWN; + if (file_printf(ms, ", note=%u", xnh_type) == -1) + return size; + break; + } + return size; + } + return offset; } /* SunOS 5.x hardware capability descriptions */ typedef struct cap_desc { uint64_t cd_mask; const char *cd_name; } cap_desc_t; static const cap_desc_t cap_desc_sparc[] = { { AV_SPARC_MUL32, "MUL32" }, { AV_SPARC_DIV32, "DIV32" }, { AV_SPARC_FSMULD, "FSMULD" }, { AV_SPARC_V8PLUS, "V8PLUS" }, { AV_SPARC_POPC, "POPC" }, { AV_SPARC_VIS, "VIS" }, { AV_SPARC_VIS2, "VIS2" }, { AV_SPARC_ASI_BLK_INIT, "ASI_BLK_INIT" }, { AV_SPARC_FMAF, "FMAF" }, { AV_SPARC_FJFMAU, "FJFMAU" }, { AV_SPARC_IMA, "IMA" }, { 0, NULL } }; static const cap_desc_t cap_desc_386[] = { { AV_386_FPU, "FPU" }, { AV_386_TSC, "TSC" }, { AV_386_CX8, "CX8" }, { AV_386_SEP, "SEP" }, { AV_386_AMD_SYSC, "AMD_SYSC" }, { AV_386_CMOV, "CMOV" }, { AV_386_MMX, "MMX" }, { AV_386_AMD_MMX, "AMD_MMX" }, { AV_386_AMD_3DNow, "AMD_3DNow" }, { AV_386_AMD_3DNowx, "AMD_3DNowx" }, { AV_386_FXSR, "FXSR" }, { AV_386_SSE, "SSE" }, { AV_386_SSE2, "SSE2" }, { AV_386_PAUSE, "PAUSE" }, { AV_386_SSE3, "SSE3" }, { AV_386_MON, "MON" }, { AV_386_CX16, "CX16" }, { AV_386_AHF, "AHF" }, { AV_386_TSCP, "TSCP" }, { AV_386_AMD_SSE4A, "AMD_SSE4A" }, { AV_386_POPCNT, "POPCNT" }, { AV_386_AMD_LZCNT, "AMD_LZCNT" }, { AV_386_SSSE3, "SSSE3" }, { AV_386_SSE4_1, "SSE4.1" }, { AV_386_SSE4_2, "SSE4.2" }, { 0, NULL } }; private int doshn(struct magic_set *ms, int clazz, int swap, int fd, off_t off, int num, - size_t size, off_t fsize, int *flags, int mach, int strtab) + size_t size, off_t fsize, int mach, int strtab, int *flags, + uint16_t *notecount) { Elf32_Shdr sh32; Elf64_Shdr sh64; int stripped = 1; size_t nbadcap = 0; void *nbuf; off_t noff, coff, name_off; uint64_t cap_hw1 = 0; /* SunOS 5.x hardware capabilites */ uint64_t cap_sf1 = 0; /* SunOS 5.x software capabilites */ char name[50]; + ssize_t namesize; if (size != xsh_sizeof) { if (file_printf(ms, ", corrupted section header size") == -1) return -1; return 0; } /* Read offset of name section to be able to read section names later */ - if (pread(fd, xsh_addr, xsh_sizeof, off + size * strtab) == -1) { + if (pread(fd, xsh_addr, xsh_sizeof, off + size * strtab) < (ssize_t)xsh_sizeof) { file_badread(ms); return -1; } name_off = xsh_offset; for ( ; num; num--) { /* Read the name of this section. */ - if (pread(fd, name, sizeof(name), name_off + xsh_name) == -1) { + if ((namesize = pread(fd, name, sizeof(name) - 1, name_off + xsh_name)) == -1) { file_badread(ms); return -1; } - name[sizeof(name) - 1] = '\0'; + name[namesize] = '\0'; if (strcmp(name, ".debug_info") == 0) stripped = 0; - if (pread(fd, xsh_addr, xsh_sizeof, off) == -1) { + if (pread(fd, xsh_addr, xsh_sizeof, off) < (ssize_t)xsh_sizeof) { file_badread(ms); return -1; } off += size; /* Things we can determine before we seek */ switch (xsh_type) { case SHT_SYMTAB: #if 0 case SHT_DYNSYM: #endif stripped = 0; break; default: - if (xsh_offset > fsize) { + if (fsize != SIZE_UNKNOWN && xsh_offset > fsize) { /* Perhaps warn here */ continue; } break; } /* Things we can determine when we seek */ switch (xsh_type) { case SHT_NOTE: if ((nbuf = malloc(xsh_size)) == NULL) { file_error(ms, errno, "Cannot allocate memory" " for note"); return -1; } - if (pread(fd, nbuf, xsh_size, xsh_offset) == -1) { + if (pread(fd, nbuf, xsh_size, xsh_offset) < (ssize_t)xsh_size) { file_badread(ms); free(nbuf); return -1; } noff = 0; for (;;) { if (noff >= (off_t)xsh_size) break; noff = donote(ms, nbuf, (size_t)noff, - xsh_size, clazz, swap, 4, flags); + xsh_size, clazz, swap, 4, flags, notecount); if (noff == 0) break; } free(nbuf); break; case SHT_SUNW_cap: switch (mach) { case EM_SPARC: case EM_SPARCV9: case EM_IA_64: case EM_386: case EM_AMD64: break; default: goto skip; } if (nbadcap > 5) break; if (lseek(fd, xsh_offset, SEEK_SET) == (off_t)-1) { file_badseek(ms); return -1; } coff = 0; for (;;) { Elf32_Cap cap32; Elf64_Cap cap64; char cbuf[/*CONSTCOND*/ MAX(sizeof cap32, sizeof cap64)]; if ((coff += xcap_sizeof) > (off_t)xsh_size) break; if (read(fd, cbuf, (size_t)xcap_sizeof) != (ssize_t)xcap_sizeof) { file_badread(ms); return -1; } if (cbuf[0] == 'A') { #ifdef notyet char *p = cbuf + 1; uint32_t len, tag; memcpy(&len, p, sizeof(len)); p += 4; len = getu32(swap, len); if (memcmp("gnu", p, 3) != 0) { if (file_printf(ms, ", unknown capability %.3s", p) == -1) return -1; break; } p += strlen(p) + 1; tag = *p++; memcpy(&len, p, sizeof(len)); p += 4; len = getu32(swap, len); if (tag != 1) { if (file_printf(ms, ", unknown gnu" " capability tag %d", tag) == -1) return -1; break; } // gnu attributes #endif break; } (void)memcpy(xcap_addr, cbuf, xcap_sizeof); switch (xcap_tag) { case CA_SUNW_NULL: break; case CA_SUNW_HW_1: cap_hw1 |= xcap_val; break; case CA_SUNW_SF_1: cap_sf1 |= xcap_val; break; default: if (file_printf(ms, ", with unknown capability " "0x%" INT64_T_FORMAT "x = 0x%" INT64_T_FORMAT "x", (unsigned long long)xcap_tag, (unsigned long long)xcap_val) == -1) return -1; if (nbadcap++ > 2) coff = xsh_size; break; } } /*FALLTHROUGH*/ skip: default: break; } } if (file_printf(ms, ", %sstripped", stripped ? "" : "not ") == -1) return -1; if (cap_hw1) { const cap_desc_t *cdp; switch (mach) { case EM_SPARC: case EM_SPARC32PLUS: case EM_SPARCV9: cdp = cap_desc_sparc; break; case EM_386: case EM_IA_64: case EM_AMD64: cdp = cap_desc_386; break; default: cdp = NULL; break; } if (file_printf(ms, ", uses") == -1) return -1; if (cdp) { while (cdp->cd_name) { if (cap_hw1 & cdp->cd_mask) { if (file_printf(ms, " %s", cdp->cd_name) == -1) return -1; cap_hw1 &= ~cdp->cd_mask; } ++cdp; } if (cap_hw1) if (file_printf(ms, " unknown hardware capability 0x%" INT64_T_FORMAT "x", (unsigned long long)cap_hw1) == -1) return -1; } else { if (file_printf(ms, " hardware capability 0x%" INT64_T_FORMAT "x", (unsigned long long)cap_hw1) == -1) return -1; } } if (cap_sf1) { if (cap_sf1 & SF1_SUNW_FPUSED) { if (file_printf(ms, (cap_sf1 & SF1_SUNW_FPKNWN) ? ", uses frame pointer" : ", not known to use frame pointer") == -1) return -1; } cap_sf1 &= ~SF1_SUNW_MASK; if (cap_sf1) if (file_printf(ms, ", with unknown software capability 0x%" INT64_T_FORMAT "x", (unsigned long long)cap_sf1) == -1) return -1; } return 0; } /* * Look through the program headers of an executable image, searching * for a PT_INTERP section; if one is found, it's dynamically linked, * otherwise it's statically linked. */ private int dophn_exec(struct magic_set *ms, int clazz, int swap, int fd, off_t off, - int num, size_t size, off_t fsize, int *flags, int sh_num) + int num, size_t size, off_t fsize, int sh_num, int *flags, + uint16_t *notecount) { Elf32_Phdr ph32; Elf64_Phdr ph64; const char *linking_style = "statically"; - const char *shared_libraries = ""; + const char *interp = ""; unsigned char nbuf[BUFSIZ]; + char ibuf[BUFSIZ]; ssize_t bufsize; size_t offset, align, len; if (size != xph_sizeof) { if (file_printf(ms, ", corrupted program header size") == -1) return -1; return 0; } for ( ; num; num--) { - if (pread(fd, xph_addr, xph_sizeof, off) == -1) { + if (pread(fd, xph_addr, xph_sizeof, off) < (ssize_t)xph_sizeof) { file_badread(ms); return -1; } off += size; + bufsize = 0; + align = 4; /* Things we can determine before we seek */ switch (xph_type) { case PT_DYNAMIC: linking_style = "dynamically"; break; + case PT_NOTE: + if (sh_num) /* Did this through section headers */ + continue; + if (((align = xph_align) & 0x80000000UL) != 0 || + align < 4) { + if (file_printf(ms, + ", invalid note alignment 0x%lx", + (unsigned long)align) == -1) + return -1; + align = 4; + } + /*FALLTHROUGH*/ case PT_INTERP: - shared_libraries = " (uses shared libs)"; + len = xph_filesz < sizeof(nbuf) ? xph_filesz + : sizeof(nbuf); + bufsize = pread(fd, nbuf, len, xph_offset); + if (bufsize == -1) { + file_badread(ms); + return -1; + } break; default: - if (xph_offset > fsize) { + if (fsize != SIZE_UNKNOWN && xph_offset > fsize) { /* Maybe warn here? */ continue; } break; } /* Things we can determine when we seek */ switch (xph_type) { + case PT_INTERP: + if (bufsize && nbuf[0]) { + nbuf[bufsize - 1] = '\0'; + interp = (const char *)nbuf; + } else + interp = "*empty*"; + break; case PT_NOTE: - if ((align = xph_align) & 0x80000000UL) { - if (file_printf(ms, - ", invalid note alignment 0x%lx", - (unsigned long)align) == -1) - return -1; - align = 4; - } - if (sh_num) - break; /* * This is a PT_NOTE section; loop through all the notes * in the section. */ - len = xph_filesz < sizeof(nbuf) ? xph_filesz - : sizeof(nbuf); - bufsize = pread(fd, nbuf, len, xph_offset); - if (bufsize == -1) { - file_badread(ms); - return -1; - } offset = 0; for (;;) { if (offset >= (size_t)bufsize) break; offset = donote(ms, nbuf, offset, (size_t)bufsize, clazz, swap, align, - flags); + flags, notecount); if (offset == 0) break; } break; default: break; } } - if (file_printf(ms, ", %s linked%s", linking_style, shared_libraries) + if (file_printf(ms, ", %s linked", linking_style) == -1) - return -1; + return -1; + if (interp[0]) + if (file_printf(ms, ", interpreter %s", + file_printable(ibuf, sizeof(ibuf), interp)) == -1) + return -1; return 0; } protected int file_tryelf(struct magic_set *ms, int fd, const unsigned char *buf, size_t nbytes) { union { int32_t l; char c[sizeof (int32_t)]; } u; int clazz; int swap; struct stat st; off_t fsize; int flags = 0; Elf32_Ehdr elf32hdr; Elf64_Ehdr elf64hdr; - uint16_t type, phnum, shnum; + uint16_t type, phnum, shnum, notecount; if (ms->flags & (MAGIC_MIME|MAGIC_APPLE)) return 0; /* * ELF executables have multiple section headers in arbitrary * file locations and thus file(1) cannot determine it from easily. * Instead we traverse thru all section headers until a symbol table * one is found or else the binary is stripped. * Return immediately if it's not ELF (so we avoid pipe2file unless needed). */ if (buf[EI_MAG0] != ELFMAG0 || (buf[EI_MAG1] != ELFMAG1 && buf[EI_MAG1] != OLFMAG1) || buf[EI_MAG2] != ELFMAG2 || buf[EI_MAG3] != ELFMAG3) return 0; /* * If we cannot seek, it must be a pipe, socket or fifo. */ if((lseek(fd, (off_t)0, SEEK_SET) == (off_t)-1) && (errno == ESPIPE)) fd = file_pipe2file(ms, fd, buf, nbytes); if (fstat(fd, &st) == -1) { file_badread(ms); return -1; } - fsize = st.st_size; + if (S_ISREG(st.st_mode) || st.st_size != 0) + fsize = st.st_size; + else + fsize = SIZE_UNKNOWN; clazz = buf[EI_CLASS]; switch (clazz) { case ELFCLASS32: #undef elf_getu #define elf_getu(a, b) elf_getu32(a, b) #undef elfhdr #define elfhdr elf32hdr #include "elfclass.h" case ELFCLASS64: #undef elf_getu #define elf_getu(a, b) elf_getu64(a, b) #undef elfhdr #define elfhdr elf64hdr #include "elfclass.h" default: if (file_printf(ms, ", unknown class %d", clazz) == -1) return -1; break; } return 0; } #endif Index: releng/10.1/contrib/file/src/softmagic.c =================================================================== --- releng/10.1/contrib/file/src/softmagic.c (revision 284192) +++ releng/10.1/contrib/file/src/softmagic.c (revision 284193) @@ -1,2137 +1,2171 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * softmagic - interpret variable magic from MAGIC */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: softmagic.c,v 1.191 2014/06/04 17:36:34 christos Exp $") +FILE_RCSID("@(#)$File: softmagic.c,v 1.206 2015/01/01 17:07:34 christos Exp $") #endif /* lint */ #include "magic.h" #include #include #include #include #include -#if defined(HAVE_LOCALE_H) -#include -#endif - private int match(struct magic_set *, struct magic *, uint32_t, - const unsigned char *, size_t, size_t, int, int, int, int, int *, int *, - int *); + const unsigned char *, size_t, size_t, int, int, int, uint16_t, + uint16_t *, int *, int *, int *); private int mget(struct magic_set *, const unsigned char *, - struct magic *, size_t, size_t, unsigned int, int, int, int, int, int *, - int *, int *); + struct magic *, size_t, size_t, unsigned int, int, int, int, uint16_t, + uint16_t *, int *, int *, int *); private int magiccheck(struct magic_set *, struct magic *); private int32_t mprint(struct magic_set *, struct magic *); private int32_t moffset(struct magic_set *, struct magic *); private void mdebug(uint32_t, const char *, size_t); private int mcopy(struct magic_set *, union VALUETYPE *, int, int, const unsigned char *, uint32_t, size_t, struct magic *); private int mconvert(struct magic_set *, struct magic *, int); private int print_sep(struct magic_set *, int); private int handle_annotation(struct magic_set *, struct magic *); private void cvt_8(union VALUETYPE *, const struct magic *); private void cvt_16(union VALUETYPE *, const struct magic *); private void cvt_32(union VALUETYPE *, const struct magic *); private void cvt_64(union VALUETYPE *, const struct magic *); #define OFFSET_OOB(n, o, i) ((n) < (o) || (i) > ((n) - (o))) -#define MAX_RECURSION_LEVEL 10 - /* * softmagic - lookup one file in parsed, in-memory copy of database * Passed the name and FILE * of one file to be typed. */ /*ARGSUSED1*/ /* nbytes passed for regularity, maybe need later */ protected int file_softmagic(struct magic_set *ms, const unsigned char *buf, size_t nbytes, - size_t level, int mode, int text) + uint16_t indir_level, uint16_t *name_count, int mode, int text) { struct mlist *ml; int rv, printed_something = 0, need_separator = 0; + uint16_t nc; + + if (name_count == NULL) { + nc = 0; + name_count = &nc; + } + for (ml = ms->mlist[0]->next; ml != ms->mlist[0]; ml = ml->next) if ((rv = match(ms, ml->magic, ml->nmagic, buf, nbytes, 0, mode, - text, 0, level, &printed_something, &need_separator, - NULL)) != 0) + text, 0, indir_level, name_count, + &printed_something, &need_separator, NULL)) != 0) return rv; return 0; } #define FILE_FMTDEBUG #ifdef FILE_FMTDEBUG #define F(a, b, c) file_fmtcheck((a), (b), (c), __FILE__, __LINE__) private const char * __attribute__((__format_arg__(3))) file_fmtcheck(struct magic_set *ms, const struct magic *m, const char *def, const char *file, size_t line) { const char *ptr = fmtcheck(m->desc, def); if (ptr == def) file_magerror(ms, - "%s, %zu: format `%s' does not match with `%s'", - file, line, m->desc, def); + "%s, %" SIZE_T_FORMAT "u: format `%s' does not match" + " with `%s'", file, line, m->desc, def); return ptr; } #else #define F(a, b, c) fmtcheck((b)->desc, (c)) #endif /* * Go through the whole list, stopping if you find a match. Process all * the continuations of that match before returning. * * We support multi-level continuations: * * At any time when processing a successful top-level match, there is a * current continuation level; it represents the level of the last * successfully matched continuation. * * Continuations above that level are skipped as, if we see one, it * means that the continuation that controls them - i.e, the * lower-level continuation preceding them - failed to match. * * Continuations below that level are processed as, if we see one, * it means we've finished processing or skipping higher-level * continuations under the control of a successful or unsuccessful * lower-level continuation, and are now seeing the next lower-level * continuation and should process it. The current continuation * level reverts to the level of the one we're seeing. * * Continuations at the current level are processed as, if we see * one, there's no lower-level continuation that may have failed. * * If a continuation matches, we bump the current continuation level * so that higher-level continuations are processed. */ private int match(struct magic_set *ms, struct magic *magic, uint32_t nmagic, const unsigned char *s, size_t nbytes, size_t offset, int mode, int text, - int flip, int recursion_level, int *printed_something, int *need_separator, - int *returnval) + int flip, uint16_t indir_level, uint16_t *name_count, + int *printed_something, int *need_separator, int *returnval) { uint32_t magindex = 0; unsigned int cont_level = 0; int returnvalv = 0, e; /* if a match is found it is set to 1*/ int firstline = 1; /* a flag to print X\n X\n- X */ int print = (ms->flags & (MAGIC_MIME|MAGIC_APPLE)) == 0; if (returnval == NULL) returnval = &returnvalv; if (file_check_mem(ms, cont_level) == -1) return -1; for (magindex = 0; magindex < nmagic; magindex++) { int flush = 0; struct magic *m = &magic[magindex]; if (m->type != FILE_NAME) if ((IS_STRING(m->type) && #define FLT (STRING_BINTEST | STRING_TEXTTEST) ((text && (m->str_flags & FLT) == STRING_BINTEST) || (!text && (m->str_flags & FLT) == STRING_TEXTTEST))) || (m->flag & mode) != mode) { /* Skip sub-tests */ while (magindex + 1 < nmagic && magic[magindex + 1].cont_level != 0 && ++magindex) continue; continue; /* Skip to next top-level test*/ } ms->offset = m->offset; ms->line = m->lineno; /* if main entry matches, print it... */ switch (mget(ms, s, m, nbytes, offset, cont_level, mode, text, - flip, recursion_level + 1, printed_something, - need_separator, returnval)) { + flip, indir_level, name_count, + printed_something, need_separator, returnval)) { case -1: return -1; case 0: flush = m->reln != '!'; break; default: if (m->type == FILE_INDIRECT) *returnval = 1; switch (magiccheck(ms, m)) { case -1: return -1; case 0: flush++; break; default: flush = 0; break; } break; } if (flush) { /* * main entry didn't match, * flush its continuations */ while (magindex < nmagic - 1 && magic[magindex + 1].cont_level != 0) magindex++; continue; } if ((e = handle_annotation(ms, m)) != 0) { *need_separator = 1; *printed_something = 1; *returnval = 1; return e; } /* * If we are going to print something, we'll need to print * a blank before we print something else. */ if (*m->desc) { *need_separator = 1; *printed_something = 1; if (print_sep(ms, firstline) == -1) return -1; } if (print && mprint(ms, m) == -1) return -1; ms->c.li[cont_level].off = moffset(ms, m); /* and any continuations that match */ if (file_check_mem(ms, ++cont_level) == -1) return -1; - while (++magindex < nmagic && - magic[magindex].cont_level != 0) { - m = &magic[magindex]; + while (magindex + 1 < nmagic && + magic[magindex + 1].cont_level != 0) { + m = &magic[++magindex]; ms->line = m->lineno; /* for messages */ if (cont_level < m->cont_level) continue; if (cont_level > m->cont_level) { /* * We're at the end of the level * "cont_level" continuations. */ cont_level = m->cont_level; } ms->offset = m->offset; if (m->flag & OFFADD) { ms->offset += ms->c.li[cont_level - 1].off; } #ifdef ENABLE_CONDITIONALS if (m->cond == COND_ELSE || m->cond == COND_ELIF) { if (ms->c.li[cont_level].last_match == 1) continue; } #endif switch (mget(ms, s, m, nbytes, offset, cont_level, mode, - text, flip, recursion_level + 1, printed_something, - need_separator, returnval)) { + text, flip, indir_level, name_count, + printed_something, need_separator, returnval)) { case -1: return -1; case 0: if (m->reln != '!') continue; flush = 1; break; default: if (m->type == FILE_INDIRECT) *returnval = 1; flush = 0; break; } switch (flush ? 1 : magiccheck(ms, m)) { case -1: return -1; case 0: #ifdef ENABLE_CONDITIONALS ms->c.li[cont_level].last_match = 0; #endif break; default: #ifdef ENABLE_CONDITIONALS ms->c.li[cont_level].last_match = 1; #endif if (m->type == FILE_CLEAR) ms->c.li[cont_level].got_match = 0; else if (ms->c.li[cont_level].got_match) { if (m->type == FILE_DEFAULT) break; } else ms->c.li[cont_level].got_match = 1; if ((e = handle_annotation(ms, m)) != 0) { *need_separator = 1; *printed_something = 1; *returnval = 1; return e; } /* * If we are going to print something, * make sure that we have a separator first. */ if (*m->desc) { if (!*printed_something) { *printed_something = 1; if (print_sep(ms, firstline) == -1) return -1; } } /* * This continuation matched. Print * its message, with a blank before it * if the previous item printed and * this item isn't empty. */ /* space if previous printed */ if (*need_separator && ((m->flag & NOSPACE) == 0) && *m->desc) { if (print && file_printf(ms, " ") == -1) return -1; *need_separator = 0; } if (print && mprint(ms, m) == -1) return -1; ms->c.li[cont_level].off = moffset(ms, m); if (*m->desc) *need_separator = 1; /* * If we see any continuations * at a higher level, * process them. */ if (file_check_mem(ms, ++cont_level) == -1) return -1; break; } } if (*printed_something) { firstline = 0; if (print) *returnval = 1; } if ((ms->flags & MAGIC_CONTINUE) == 0 && *printed_something) { return *returnval; /* don't keep searching */ } } return *returnval; /* This is hit if -k is set or there is no match */ } private int check_fmt(struct magic_set *ms, struct magic *m) { file_regex_t rx; int rc, rv = -1; if (strchr(m->desc, '%') == NULL) return 0; rc = file_regcomp(&rx, "%[-0-9\\.]*s", REG_EXTENDED|REG_NOSUB); if (rc) { file_regerror(&rx, rc, ms); } else { rc = file_regexec(&rx, m->desc, 0, 0, 0); rv = !rc; } file_regfree(&rx); return rv; } #ifndef HAVE_STRNDUP char * strndup(const char *, size_t); char * strndup(const char *str, size_t n) { size_t len; char *copy; for (len = 0; len < n && str[len]; len++) continue; if ((copy = malloc(len + 1)) == NULL) return NULL; (void)memcpy(copy, str, len); copy[len] = '\0'; return copy; } #endif /* HAVE_STRNDUP */ private int32_t mprint(struct magic_set *ms, struct magic *m) { uint64_t v; float vf; double vd; int64_t t = 0; - char buf[128], tbuf[26]; + char buf[128], tbuf[26], sbuf[512]; union VALUETYPE *p = &ms->ms_value; switch (m->type) { case FILE_BYTE: v = file_signextend(ms, m, (uint64_t)p->b); switch (check_fmt(ms, m)) { case -1: return -1; case 1: (void)snprintf(buf, sizeof(buf), "%d", (unsigned char)v); if (file_printf(ms, F(ms, m, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, m, "%d"), (unsigned char) v) == -1) return -1; break; } t = ms->offset + sizeof(char); break; case FILE_SHORT: case FILE_BESHORT: case FILE_LESHORT: v = file_signextend(ms, m, (uint64_t)p->h); switch (check_fmt(ms, m)) { case -1: return -1; case 1: (void)snprintf(buf, sizeof(buf), "%u", (unsigned short)v); if (file_printf(ms, F(ms, m, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, m, "%u"), (unsigned short) v) == -1) return -1; break; } t = ms->offset + sizeof(short); break; case FILE_LONG: case FILE_BELONG: case FILE_LELONG: case FILE_MELONG: v = file_signextend(ms, m, (uint64_t)p->l); switch (check_fmt(ms, m)) { case -1: return -1; case 1: (void)snprintf(buf, sizeof(buf), "%u", (uint32_t) v); if (file_printf(ms, F(ms, m, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, m, "%u"), (uint32_t) v) == -1) return -1; break; } t = ms->offset + sizeof(int32_t); break; case FILE_QUAD: case FILE_BEQUAD: case FILE_LEQUAD: v = file_signextend(ms, m, p->q); switch (check_fmt(ms, m)) { case -1: return -1; case 1: (void)snprintf(buf, sizeof(buf), "%" INT64_T_FORMAT "u", (unsigned long long)v); if (file_printf(ms, F(ms, m, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, m, "%" INT64_T_FORMAT "u"), (unsigned long long) v) == -1) return -1; break; } t = ms->offset + sizeof(int64_t); break; case FILE_STRING: case FILE_PSTRING: case FILE_BESTRING16: case FILE_LESTRING16: if (m->reln == '=' || m->reln == '!') { - if (file_printf(ms, F(ms, m, "%s"), m->value.s) == -1) + if (file_printf(ms, F(ms, m, "%s"), + file_printable(sbuf, sizeof(sbuf), m->value.s)) + == -1) return -1; t = ms->offset + m->vallen; } else { char *str = p->s; /* compute t before we mangle the string? */ t = ms->offset + strlen(str); if (*m->value.s == '\0') str[strcspn(str, "\n")] = '\0'; if (m->str_flags & STRING_TRIM) { char *last; while (isspace((unsigned char)*str)) str++; last = str; while (*last) last++; --last; while (isspace((unsigned char)*last)) last--; *++last = '\0'; } - if (file_printf(ms, F(ms, m, "%s"), str) == -1) + if (file_printf(ms, F(ms, m, "%s"), + file_printable(sbuf, sizeof(sbuf), str)) == -1) return -1; if (m->type == FILE_PSTRING) t += file_pstring_length_size(m); } break; case FILE_DATE: case FILE_BEDATE: case FILE_LEDATE: case FILE_MEDATE: if (file_printf(ms, F(ms, m, "%s"), file_fmttime(p->l + m->num_mask, FILE_T_LOCAL, tbuf)) == -1) return -1; t = ms->offset + sizeof(uint32_t); break; case FILE_LDATE: case FILE_BELDATE: case FILE_LELDATE: case FILE_MELDATE: if (file_printf(ms, F(ms, m, "%s"), file_fmttime(p->l + m->num_mask, 0, tbuf)) == -1) return -1; t = ms->offset + sizeof(uint32_t); break; case FILE_QDATE: case FILE_BEQDATE: case FILE_LEQDATE: if (file_printf(ms, F(ms, m, "%s"), file_fmttime(p->q + m->num_mask, FILE_T_LOCAL, tbuf)) == -1) return -1; t = ms->offset + sizeof(uint64_t); break; case FILE_QLDATE: case FILE_BEQLDATE: case FILE_LEQLDATE: if (file_printf(ms, F(ms, m, "%s"), file_fmttime(p->q + m->num_mask, 0, tbuf)) == -1) return -1; t = ms->offset + sizeof(uint64_t); break; case FILE_QWDATE: case FILE_BEQWDATE: case FILE_LEQWDATE: if (file_printf(ms, F(ms, m, "%s"), file_fmttime(p->q + m->num_mask, FILE_T_WINDOWS, tbuf)) == -1) return -1; t = ms->offset + sizeof(uint64_t); break; case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: vf = p->f; switch (check_fmt(ms, m)) { case -1: return -1; case 1: (void)snprintf(buf, sizeof(buf), "%g", vf); if (file_printf(ms, F(ms, m, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, m, "%g"), vf) == -1) return -1; break; } t = ms->offset + sizeof(float); break; case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: vd = p->d; switch (check_fmt(ms, m)) { case -1: return -1; case 1: (void)snprintf(buf, sizeof(buf), "%g", vd); if (file_printf(ms, F(ms, m, "%s"), buf) == -1) return -1; break; default: if (file_printf(ms, F(ms, m, "%g"), vd) == -1) return -1; break; } t = ms->offset + sizeof(double); break; case FILE_REGEX: { char *cp; int rval; cp = strndup((const char *)ms->search.s, ms->search.rm_len); if (cp == NULL) { file_oomem(ms, ms->search.rm_len); return -1; } - rval = file_printf(ms, F(ms, m, "%s"), cp); + rval = file_printf(ms, F(ms, m, "%s"), + file_printable(sbuf, sizeof(sbuf), cp)); free(cp); if (rval == -1) return -1; if ((m->str_flags & REGEX_OFFSET_START)) t = ms->search.offset; else t = ms->search.offset + ms->search.rm_len; break; } case FILE_SEARCH: - if (file_printf(ms, F(ms, m, "%s"), m->value.s) == -1) + if (file_printf(ms, F(ms, m, "%s"), + file_printable(sbuf, sizeof(sbuf), m->value.s)) == -1) return -1; if ((m->str_flags & REGEX_OFFSET_START)) t = ms->search.offset; else t = ms->search.offset + m->vallen; break; case FILE_DEFAULT: case FILE_CLEAR: if (file_printf(ms, "%s", m->desc) == -1) return -1; t = ms->offset; break; case FILE_INDIRECT: case FILE_USE: case FILE_NAME: t = ms->offset; break; default: file_magerror(ms, "invalid m->type (%d) in mprint()", m->type); return -1; } return (int32_t)t; } private int32_t moffset(struct magic_set *ms, struct magic *m) { switch (m->type) { case FILE_BYTE: return CAST(int32_t, (ms->offset + sizeof(char))); case FILE_SHORT: case FILE_BESHORT: case FILE_LESHORT: return CAST(int32_t, (ms->offset + sizeof(short))); case FILE_LONG: case FILE_BELONG: case FILE_LELONG: case FILE_MELONG: return CAST(int32_t, (ms->offset + sizeof(int32_t))); case FILE_QUAD: case FILE_BEQUAD: case FILE_LEQUAD: return CAST(int32_t, (ms->offset + sizeof(int64_t))); case FILE_STRING: case FILE_PSTRING: case FILE_BESTRING16: case FILE_LESTRING16: if (m->reln == '=' || m->reln == '!') return ms->offset + m->vallen; else { union VALUETYPE *p = &ms->ms_value; uint32_t t; if (*m->value.s == '\0') p->s[strcspn(p->s, "\n")] = '\0'; t = CAST(uint32_t, (ms->offset + strlen(p->s))); if (m->type == FILE_PSTRING) t += (uint32_t)file_pstring_length_size(m); return t; } case FILE_DATE: case FILE_BEDATE: case FILE_LEDATE: case FILE_MEDATE: return CAST(int32_t, (ms->offset + sizeof(uint32_t))); case FILE_LDATE: case FILE_BELDATE: case FILE_LELDATE: case FILE_MELDATE: return CAST(int32_t, (ms->offset + sizeof(uint32_t))); case FILE_QDATE: case FILE_BEQDATE: case FILE_LEQDATE: return CAST(int32_t, (ms->offset + sizeof(uint64_t))); case FILE_QLDATE: case FILE_BEQLDATE: case FILE_LEQLDATE: return CAST(int32_t, (ms->offset + sizeof(uint64_t))); case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: return CAST(int32_t, (ms->offset + sizeof(float))); case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: return CAST(int32_t, (ms->offset + sizeof(double))); case FILE_REGEX: if ((m->str_flags & REGEX_OFFSET_START) != 0) return CAST(int32_t, ms->search.offset); else return CAST(int32_t, (ms->search.offset + ms->search.rm_len)); case FILE_SEARCH: if ((m->str_flags & REGEX_OFFSET_START) != 0) return CAST(int32_t, ms->search.offset); else return CAST(int32_t, (ms->search.offset + m->vallen)); case FILE_CLEAR: case FILE_DEFAULT: case FILE_INDIRECT: return ms->offset; default: return 0; } } private int cvt_flip(int type, int flip) { if (flip == 0) return type; switch (type) { case FILE_BESHORT: return FILE_LESHORT; case FILE_BELONG: return FILE_LELONG; case FILE_BEDATE: return FILE_LEDATE; case FILE_BELDATE: return FILE_LELDATE; case FILE_BEQUAD: return FILE_LEQUAD; case FILE_BEQDATE: return FILE_LEQDATE; case FILE_BEQLDATE: return FILE_LEQLDATE; case FILE_BEQWDATE: return FILE_LEQWDATE; case FILE_LESHORT: return FILE_BESHORT; case FILE_LELONG: return FILE_BELONG; case FILE_LEDATE: return FILE_BEDATE; case FILE_LELDATE: return FILE_BELDATE; case FILE_LEQUAD: return FILE_BEQUAD; case FILE_LEQDATE: return FILE_BEQDATE; case FILE_LEQLDATE: return FILE_BEQLDATE; case FILE_LEQWDATE: return FILE_BEQWDATE; case FILE_BEFLOAT: return FILE_LEFLOAT; case FILE_LEFLOAT: return FILE_BEFLOAT; case FILE_BEDOUBLE: return FILE_LEDOUBLE; case FILE_LEDOUBLE: return FILE_BEDOUBLE; default: return type; } } #define DO_CVT(fld, cast) \ if (m->num_mask) \ switch (m->mask_op & FILE_OPS_MASK) { \ case FILE_OPAND: \ p->fld &= cast m->num_mask; \ break; \ case FILE_OPOR: \ p->fld |= cast m->num_mask; \ break; \ case FILE_OPXOR: \ p->fld ^= cast m->num_mask; \ break; \ case FILE_OPADD: \ p->fld += cast m->num_mask; \ break; \ case FILE_OPMINUS: \ p->fld -= cast m->num_mask; \ break; \ case FILE_OPMULTIPLY: \ p->fld *= cast m->num_mask; \ break; \ case FILE_OPDIVIDE: \ p->fld /= cast m->num_mask; \ break; \ case FILE_OPMODULO: \ p->fld %= cast m->num_mask; \ break; \ } \ if (m->mask_op & FILE_OPINVERSE) \ p->fld = ~p->fld \ private void cvt_8(union VALUETYPE *p, const struct magic *m) { DO_CVT(b, (uint8_t)); } private void cvt_16(union VALUETYPE *p, const struct magic *m) { DO_CVT(h, (uint16_t)); } private void cvt_32(union VALUETYPE *p, const struct magic *m) { DO_CVT(l, (uint32_t)); } private void cvt_64(union VALUETYPE *p, const struct magic *m) { DO_CVT(q, (uint64_t)); } #define DO_CVT2(fld, cast) \ if (m->num_mask) \ switch (m->mask_op & FILE_OPS_MASK) { \ case FILE_OPADD: \ p->fld += cast m->num_mask; \ break; \ case FILE_OPMINUS: \ p->fld -= cast m->num_mask; \ break; \ case FILE_OPMULTIPLY: \ p->fld *= cast m->num_mask; \ break; \ case FILE_OPDIVIDE: \ p->fld /= cast m->num_mask; \ break; \ } \ private void cvt_float(union VALUETYPE *p, const struct magic *m) { DO_CVT2(f, (float)); } private void cvt_double(union VALUETYPE *p, const struct magic *m) { DO_CVT2(d, (double)); } /* * Convert the byte order of the data we are looking at * While we're here, let's apply the mask operation * (unless you have a better idea) */ private int mconvert(struct magic_set *ms, struct magic *m, int flip) { union VALUETYPE *p = &ms->ms_value; uint8_t type; switch (type = cvt_flip(m->type, flip)) { case FILE_BYTE: cvt_8(p, m); return 1; case FILE_SHORT: cvt_16(p, m); return 1; case FILE_LONG: case FILE_DATE: case FILE_LDATE: cvt_32(p, m); return 1; case FILE_QUAD: case FILE_QDATE: case FILE_QLDATE: case FILE_QWDATE: cvt_64(p, m); return 1; case FILE_STRING: case FILE_BESTRING16: case FILE_LESTRING16: { /* Null terminate and eat *trailing* return */ p->s[sizeof(p->s) - 1] = '\0'; return 1; } case FILE_PSTRING: { size_t sz = file_pstring_length_size(m); char *ptr1 = p->s, *ptr2 = ptr1 + sz; size_t len = file_pstring_get_length(m, ptr1); - if (len >= sizeof(p->s)) { + sz = sizeof(p->s) - sz; /* maximum length of string */ + if (len >= sz) { /* * The size of the pascal string length (sz) * is 1, 2, or 4. We need at least 1 byte for NUL * termination, but we've already truncated the * string by p->s, so we need to deduct sz. + * Because we can use one of the bytes of the length + * after we shifted as NUL termination. */ - len = sizeof(p->s) - sz; + len = sz; } while (len--) *ptr1++ = *ptr2++; *ptr1 = '\0'; return 1; } case FILE_BESHORT: p->h = (short)((p->hs[0]<<8)|(p->hs[1])); cvt_16(p, m); return 1; case FILE_BELONG: case FILE_BEDATE: case FILE_BELDATE: p->l = (int32_t) ((p->hl[0]<<24)|(p->hl[1]<<16)|(p->hl[2]<<8)|(p->hl[3])); if (type == FILE_BELONG) cvt_32(p, m); return 1; case FILE_BEQUAD: case FILE_BEQDATE: case FILE_BEQLDATE: case FILE_BEQWDATE: p->q = (uint64_t) (((uint64_t)p->hq[0]<<56)|((uint64_t)p->hq[1]<<48)| ((uint64_t)p->hq[2]<<40)|((uint64_t)p->hq[3]<<32)| ((uint64_t)p->hq[4]<<24)|((uint64_t)p->hq[5]<<16)| ((uint64_t)p->hq[6]<<8)|((uint64_t)p->hq[7])); if (type == FILE_BEQUAD) cvt_64(p, m); return 1; case FILE_LESHORT: p->h = (short)((p->hs[1]<<8)|(p->hs[0])); cvt_16(p, m); return 1; case FILE_LELONG: case FILE_LEDATE: case FILE_LELDATE: p->l = (int32_t) ((p->hl[3]<<24)|(p->hl[2]<<16)|(p->hl[1]<<8)|(p->hl[0])); if (type == FILE_LELONG) cvt_32(p, m); return 1; case FILE_LEQUAD: case FILE_LEQDATE: case FILE_LEQLDATE: case FILE_LEQWDATE: p->q = (uint64_t) (((uint64_t)p->hq[7]<<56)|((uint64_t)p->hq[6]<<48)| ((uint64_t)p->hq[5]<<40)|((uint64_t)p->hq[4]<<32)| ((uint64_t)p->hq[3]<<24)|((uint64_t)p->hq[2]<<16)| ((uint64_t)p->hq[1]<<8)|((uint64_t)p->hq[0])); if (type == FILE_LEQUAD) cvt_64(p, m); return 1; case FILE_MELONG: case FILE_MEDATE: case FILE_MELDATE: p->l = (int32_t) ((p->hl[1]<<24)|(p->hl[0]<<16)|(p->hl[3]<<8)|(p->hl[2])); if (type == FILE_MELONG) cvt_32(p, m); return 1; case FILE_FLOAT: cvt_float(p, m); return 1; case FILE_BEFLOAT: p->l = ((uint32_t)p->hl[0]<<24)|((uint32_t)p->hl[1]<<16)| ((uint32_t)p->hl[2]<<8) |((uint32_t)p->hl[3]); cvt_float(p, m); return 1; case FILE_LEFLOAT: p->l = ((uint32_t)p->hl[3]<<24)|((uint32_t)p->hl[2]<<16)| ((uint32_t)p->hl[1]<<8) |((uint32_t)p->hl[0]); cvt_float(p, m); return 1; case FILE_DOUBLE: cvt_double(p, m); return 1; case FILE_BEDOUBLE: p->q = ((uint64_t)p->hq[0]<<56)|((uint64_t)p->hq[1]<<48)| ((uint64_t)p->hq[2]<<40)|((uint64_t)p->hq[3]<<32)| ((uint64_t)p->hq[4]<<24)|((uint64_t)p->hq[5]<<16)| ((uint64_t)p->hq[6]<<8) |((uint64_t)p->hq[7]); cvt_double(p, m); return 1; case FILE_LEDOUBLE: p->q = ((uint64_t)p->hq[7]<<56)|((uint64_t)p->hq[6]<<48)| ((uint64_t)p->hq[5]<<40)|((uint64_t)p->hq[4]<<32)| ((uint64_t)p->hq[3]<<24)|((uint64_t)p->hq[2]<<16)| ((uint64_t)p->hq[1]<<8) |((uint64_t)p->hq[0]); cvt_double(p, m); return 1; case FILE_REGEX: case FILE_SEARCH: case FILE_DEFAULT: case FILE_CLEAR: case FILE_NAME: case FILE_USE: return 1; default: file_magerror(ms, "invalid type %d in mconvert()", m->type); return 0; } } private void mdebug(uint32_t offset, const char *str, size_t len) { - (void) fprintf(stderr, "mget/%zu @%d: ", len, offset); + (void) fprintf(stderr, "mget/%" SIZE_T_FORMAT "u @%d: ", len, offset); file_showstr(stderr, str, len); (void) fputc('\n', stderr); (void) fputc('\n', stderr); } private int mcopy(struct magic_set *ms, union VALUETYPE *p, int type, int indir, const unsigned char *s, uint32_t offset, size_t nbytes, struct magic *m) { /* * Note: FILE_SEARCH and FILE_REGEX do not actually copy * anything, but setup pointers into the source */ if (indir == 0) { switch (type) { case FILE_SEARCH: ms->search.s = RCAST(const char *, s) + offset; ms->search.s_len = nbytes - offset; ms->search.offset = offset; return 0; case FILE_REGEX: { const char *b; const char *c; const char *last; /* end of search region */ const char *buf; /* start of search region */ const char *end; size_t lines, linecnt, bytecnt; if (s == NULL) { ms->search.s_len = 0; ms->search.s = NULL; return 0; } if (m->str_flags & REGEX_LINE_COUNT) { linecnt = m->str_range; bytecnt = linecnt * 80; } else { linecnt = 0; bytecnt = m->str_range; } if (bytecnt == 0) bytecnt = 8192; if (bytecnt > nbytes) bytecnt = nbytes; buf = RCAST(const char *, s) + offset; end = last = RCAST(const char *, s) + bytecnt; /* mget() guarantees buf <= last */ for (lines = linecnt, b = buf; lines && b < end && ((b = CAST(const char *, memchr(c = b, '\n', CAST(size_t, (end - b))))) || (b = CAST(const char *, memchr(c, '\r', CAST(size_t, (end - c)))))); lines--, b++) { last = b; if (b[0] == '\r' && b[1] == '\n') b++; } if (lines) last = RCAST(const char *, s) + bytecnt; ms->search.s = buf; ms->search.s_len = last - buf; ms->search.offset = offset; ms->search.rm_len = 0; return 0; } case FILE_BESTRING16: case FILE_LESTRING16: { const unsigned char *src = s + offset; const unsigned char *esrc = s + nbytes; char *dst = p->s; char *edst = &p->s[sizeof(p->s) - 1]; if (type == FILE_BESTRING16) src++; /* check that offset is within range */ if (offset >= nbytes) break; for (/*EMPTY*/; src < esrc; src += 2, dst++) { if (dst < edst) *dst = *src; else break; if (*dst == '\0') { if (type == FILE_BESTRING16 ? *(src - 1) != '\0' : *(src + 1) != '\0') *dst = ' '; } } *edst = '\0'; return 0; } case FILE_STRING: /* XXX - these two should not need */ case FILE_PSTRING: /* to copy anything, but do anyway. */ default: break; } } if (offset >= nbytes) { (void)memset(p, '\0', sizeof(*p)); return 0; } if (nbytes - offset < sizeof(*p)) nbytes = nbytes - offset; else nbytes = sizeof(*p); (void)memcpy(p, s + offset, nbytes); /* * the usefulness of padding with zeroes eludes me, it * might even cause problems */ if (nbytes < sizeof(*p)) (void)memset(((char *)(void *)p) + nbytes, '\0', sizeof(*p) - nbytes); return 0; } private int mget(struct magic_set *ms, const unsigned char *s, struct magic *m, size_t nbytes, size_t o, unsigned int cont_level, int mode, int text, - int flip, int recursion_level, int *printed_something, - int *need_separator, int *returnval) + int flip, uint16_t indir_level, uint16_t *name_count, + int *printed_something, int *need_separator, int *returnval) { uint32_t offset = ms->offset; uint32_t lhs; file_pushbuf_t *pb; int rv, oneed_separator, in_type; char *rbuf; union VALUETYPE *p = &ms->ms_value; struct mlist ml; - if (recursion_level >= MAX_RECURSION_LEVEL) { - file_error(ms, 0, "recursion nesting exceeded"); + if (indir_level >= ms->indir_max) { + file_error(ms, 0, "indirect recursion nesting (%hu) exceeded", + indir_level); return -1; } + if (*name_count >= ms->name_max) { + file_error(ms, 0, "name use count (%hu) exceeded", + *name_count); + return -1; + } + if (mcopy(ms, p, m->type, m->flag & INDIR, s, (uint32_t)(offset + o), (uint32_t)nbytes, m) == -1) return -1; if ((ms->flags & MAGIC_DEBUG) != 0) { - fprintf(stderr, "mget(type=%d, flag=%x, offset=%u, o=%zu, " - "nbytes=%zu)\n", m->type, m->flag, offset, o, nbytes); + fprintf(stderr, "mget(type=%d, flag=%x, offset=%u, o=%" + SIZE_T_FORMAT "u, " "nbytes=%" SIZE_T_FORMAT + "u, il=%hu, nc=%hu)\n", + m->type, m->flag, offset, o, nbytes, + indir_level, *name_count); mdebug(offset, (char *)(void *)p, sizeof(union VALUETYPE)); #ifndef COMPILE_ONLY file_mdump(m); #endif } if (m->flag & INDIR) { int off = m->in_offset; if (m->in_op & FILE_OPINDIRECT) { const union VALUETYPE *q = CAST(const union VALUETYPE *, ((const void *)(s + offset + off))); switch (cvt_flip(m->in_type, flip)) { case FILE_BYTE: off = q->b; break; case FILE_SHORT: off = q->h; break; case FILE_BESHORT: off = (short)((q->hs[0]<<8)|(q->hs[1])); break; case FILE_LESHORT: off = (short)((q->hs[1]<<8)|(q->hs[0])); break; case FILE_LONG: off = q->l; break; case FILE_BELONG: case FILE_BEID3: off = (int32_t)((q->hl[0]<<24)|(q->hl[1]<<16)| (q->hl[2]<<8)|(q->hl[3])); break; case FILE_LEID3: case FILE_LELONG: off = (int32_t)((q->hl[3]<<24)|(q->hl[2]<<16)| (q->hl[1]<<8)|(q->hl[0])); break; case FILE_MELONG: off = (int32_t)((q->hl[1]<<24)|(q->hl[0]<<16)| (q->hl[3]<<8)|(q->hl[2])); break; } if ((ms->flags & MAGIC_DEBUG) != 0) fprintf(stderr, "indirect offs=%u\n", off); } switch (in_type = cvt_flip(m->in_type, flip)) { case FILE_BYTE: if (OFFSET_OOB(nbytes, offset, 1)) return 0; if (off) { switch (m->in_op & FILE_OPS_MASK) { case FILE_OPAND: offset = p->b & off; break; case FILE_OPOR: offset = p->b | off; break; case FILE_OPXOR: offset = p->b ^ off; break; case FILE_OPADD: offset = p->b + off; break; case FILE_OPMINUS: offset = p->b - off; break; case FILE_OPMULTIPLY: offset = p->b * off; break; case FILE_OPDIVIDE: offset = p->b / off; break; case FILE_OPMODULO: offset = p->b % off; break; } } else offset = p->b; if (m->in_op & FILE_OPINVERSE) offset = ~offset; break; case FILE_BESHORT: if (OFFSET_OOB(nbytes, offset, 2)) return 0; lhs = (p->hs[0] << 8) | p->hs[1]; if (off) { switch (m->in_op & FILE_OPS_MASK) { case FILE_OPAND: offset = lhs & off; break; case FILE_OPOR: offset = lhs | off; break; case FILE_OPXOR: offset = lhs ^ off; break; case FILE_OPADD: offset = lhs + off; break; case FILE_OPMINUS: offset = lhs - off; break; case FILE_OPMULTIPLY: offset = lhs * off; break; case FILE_OPDIVIDE: offset = lhs / off; break; case FILE_OPMODULO: offset = lhs % off; break; } } else offset = lhs; if (m->in_op & FILE_OPINVERSE) offset = ~offset; break; case FILE_LESHORT: if (OFFSET_OOB(nbytes, offset, 2)) return 0; lhs = (p->hs[1] << 8) | p->hs[0]; if (off) { switch (m->in_op & FILE_OPS_MASK) { case FILE_OPAND: offset = lhs & off; break; case FILE_OPOR: offset = lhs | off; break; case FILE_OPXOR: offset = lhs ^ off; break; case FILE_OPADD: offset = lhs + off; break; case FILE_OPMINUS: offset = lhs - off; break; case FILE_OPMULTIPLY: offset = lhs * off; break; case FILE_OPDIVIDE: offset = lhs / off; break; case FILE_OPMODULO: offset = lhs % off; break; } } else offset = lhs; if (m->in_op & FILE_OPINVERSE) offset = ~offset; break; case FILE_SHORT: if (OFFSET_OOB(nbytes, offset, 2)) return 0; if (off) { switch (m->in_op & FILE_OPS_MASK) { case FILE_OPAND: offset = p->h & off; break; case FILE_OPOR: offset = p->h | off; break; case FILE_OPXOR: offset = p->h ^ off; break; case FILE_OPADD: offset = p->h + off; break; case FILE_OPMINUS: offset = p->h - off; break; case FILE_OPMULTIPLY: offset = p->h * off; break; case FILE_OPDIVIDE: offset = p->h / off; break; case FILE_OPMODULO: offset = p->h % off; break; } } else offset = p->h; if (m->in_op & FILE_OPINVERSE) offset = ~offset; break; case FILE_BELONG: case FILE_BEID3: if (OFFSET_OOB(nbytes, offset, 4)) return 0; lhs = (p->hl[0] << 24) | (p->hl[1] << 16) | (p->hl[2] << 8) | p->hl[3]; if (off) { switch (m->in_op & FILE_OPS_MASK) { case FILE_OPAND: offset = lhs & off; break; case FILE_OPOR: offset = lhs | off; break; case FILE_OPXOR: offset = lhs ^ off; break; case FILE_OPADD: offset = lhs + off; break; case FILE_OPMINUS: offset = lhs - off; break; case FILE_OPMULTIPLY: offset = lhs * off; break; case FILE_OPDIVIDE: offset = lhs / off; break; case FILE_OPMODULO: offset = lhs % off; break; } } else offset = lhs; if (m->in_op & FILE_OPINVERSE) offset = ~offset; break; case FILE_LELONG: case FILE_LEID3: if (OFFSET_OOB(nbytes, offset, 4)) return 0; lhs = (p->hl[3] << 24) | (p->hl[2] << 16) | (p->hl[1] << 8) | p->hl[0]; if (off) { switch (m->in_op & FILE_OPS_MASK) { case FILE_OPAND: offset = lhs & off; break; case FILE_OPOR: offset = lhs | off; break; case FILE_OPXOR: offset = lhs ^ off; break; case FILE_OPADD: offset = lhs + off; break; case FILE_OPMINUS: offset = lhs - off; break; case FILE_OPMULTIPLY: offset = lhs * off; break; case FILE_OPDIVIDE: offset = lhs / off; break; case FILE_OPMODULO: offset = lhs % off; break; } } else offset = lhs; if (m->in_op & FILE_OPINVERSE) offset = ~offset; break; case FILE_MELONG: if (OFFSET_OOB(nbytes, offset, 4)) return 0; lhs = (p->hl[1] << 24) | (p->hl[0] << 16) | (p->hl[3] << 8) | p->hl[2]; if (off) { switch (m->in_op & FILE_OPS_MASK) { case FILE_OPAND: offset = lhs & off; break; case FILE_OPOR: offset = lhs | off; break; case FILE_OPXOR: offset = lhs ^ off; break; case FILE_OPADD: offset = lhs + off; break; case FILE_OPMINUS: offset = lhs - off; break; case FILE_OPMULTIPLY: offset = lhs * off; break; case FILE_OPDIVIDE: offset = lhs / off; break; case FILE_OPMODULO: offset = lhs % off; break; } } else offset = lhs; if (m->in_op & FILE_OPINVERSE) offset = ~offset; break; case FILE_LONG: if (OFFSET_OOB(nbytes, offset, 4)) return 0; if (off) { switch (m->in_op & FILE_OPS_MASK) { case FILE_OPAND: offset = p->l & off; break; case FILE_OPOR: offset = p->l | off; break; case FILE_OPXOR: offset = p->l ^ off; break; case FILE_OPADD: offset = p->l + off; break; case FILE_OPMINUS: offset = p->l - off; break; case FILE_OPMULTIPLY: offset = p->l * off; break; case FILE_OPDIVIDE: offset = p->l / off; break; case FILE_OPMODULO: offset = p->l % off; break; } } else offset = p->l; if (m->in_op & FILE_OPINVERSE) offset = ~offset; break; default: break; } switch (in_type) { case FILE_LEID3: case FILE_BEID3: offset = ((((offset >> 0) & 0x7f) << 0) | (((offset >> 8) & 0x7f) << 7) | (((offset >> 16) & 0x7f) << 14) | (((offset >> 24) & 0x7f) << 21)) + 10; break; default: break; } if (m->flag & INDIROFFADD) { offset += ms->c.li[cont_level-1].off; if (offset == 0) { if ((ms->flags & MAGIC_DEBUG) != 0) fprintf(stderr, "indirect *zero* offset\n"); return 0; } if ((ms->flags & MAGIC_DEBUG) != 0) fprintf(stderr, "indirect +offs=%u\n", offset); } if (mcopy(ms, p, m->type, 0, s, offset, nbytes, m) == -1) return -1; ms->offset = offset; if ((ms->flags & MAGIC_DEBUG) != 0) { mdebug(offset, (char *)(void *)p, sizeof(union VALUETYPE)); #ifndef COMPILE_ONLY file_mdump(m); #endif } } /* Verify we have enough data to match magic type */ switch (m->type) { case FILE_BYTE: if (OFFSET_OOB(nbytes, offset, 1)) return 0; break; case FILE_SHORT: case FILE_BESHORT: case FILE_LESHORT: if (OFFSET_OOB(nbytes, offset, 2)) return 0; break; case FILE_LONG: case FILE_BELONG: case FILE_LELONG: case FILE_MELONG: case FILE_DATE: case FILE_BEDATE: case FILE_LEDATE: case FILE_MEDATE: case FILE_LDATE: case FILE_BELDATE: case FILE_LELDATE: case FILE_MELDATE: case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: if (OFFSET_OOB(nbytes, offset, 4)) return 0; break; case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: if (OFFSET_OOB(nbytes, offset, 8)) return 0; break; case FILE_STRING: case FILE_PSTRING: case FILE_SEARCH: if (OFFSET_OOB(nbytes, offset, m->vallen)) return 0; break; case FILE_REGEX: if (nbytes < offset) return 0; break; case FILE_INDIRECT: + if (m->str_flags & INDIRECT_RELATIVE) + offset += o; if (offset == 0) return 0; if (nbytes < offset) return 0; if ((pb = file_push_buffer(ms)) == NULL) return -1; rv = file_softmagic(ms, s + offset, nbytes - offset, - recursion_level, BINTEST, text); + indir_level + 1, name_count, BINTEST, text); if ((ms->flags & MAGIC_DEBUG) != 0) fprintf(stderr, "indirect @offs=%u[%d]\n", offset, rv); rbuf = file_pop_buffer(ms, pb); if (rbuf == NULL && ms->event_flags & EVENT_HAD_ERR) return -1; if (rv == 1) { if ((ms->flags & (MAGIC_MIME|MAGIC_APPLE)) == 0 && file_printf(ms, F(ms, m, "%u"), offset) == -1) { free(rbuf); return -1; } if (file_printf(ms, "%s", rbuf) == -1) { free(rbuf); return -1; } } free(rbuf); return rv; case FILE_USE: if (nbytes < offset) return 0; rbuf = m->value.s; if (*rbuf == '^') { rbuf++; flip = !flip; } if (file_magicfind(ms, rbuf, &ml) == -1) { file_error(ms, 0, "cannot find entry `%s'", rbuf); return -1; } - + (*name_count)++; oneed_separator = *need_separator; if (m->flag & NOSPACE) *need_separator = 0; rv = match(ms, ml.magic, ml.nmagic, s, nbytes, offset + o, - mode, text, flip, recursion_level, printed_something, - need_separator, returnval); + mode, text, flip, indir_level, name_count, + printed_something, need_separator, returnval); if (rv != 1) *need_separator = oneed_separator; return rv; case FILE_NAME: if (file_printf(ms, "%s", m->desc) == -1) return -1; return 1; case FILE_DEFAULT: /* nothing to check */ case FILE_CLEAR: default: break; } if (!mconvert(ms, m, flip)) return 0; return 1; } private uint64_t file_strncmp(const char *s1, const char *s2, size_t len, uint32_t flags) { /* * Convert the source args to unsigned here so that (1) the * compare will be unsigned as it is in strncmp() and (2) so * the ctype functions will work correctly without extra * casting. */ const unsigned char *a = (const unsigned char *)s1; const unsigned char *b = (const unsigned char *)s2; uint64_t v; /* * What we want here is v = strncmp(s1, s2, len), * but ignoring any nulls. */ v = 0; if (0L == flags) { /* normal string: do it fast */ while (len-- > 0) if ((v = *b++ - *a++) != '\0') break; } else { /* combine the others */ while (len-- > 0) { if ((flags & STRING_IGNORE_LOWERCASE) && islower(*a)) { if ((v = tolower(*b++) - *a++) != '\0') break; } else if ((flags & STRING_IGNORE_UPPERCASE) && isupper(*a)) { if ((v = toupper(*b++) - *a++) != '\0') break; } else if ((flags & STRING_COMPACT_WHITESPACE) && isspace(*a)) { a++; if (isspace(*b++)) { if (!isspace(*a)) while (isspace(*b)) b++; } else { v = 1; break; } } else if ((flags & STRING_COMPACT_OPTIONAL_WHITESPACE) && isspace(*a)) { a++; while (isspace(*b)) b++; } else { if ((v = *b++ - *a++) != '\0') break; } } } return v; } private uint64_t file_strncmp16(const char *a, const char *b, size_t len, uint32_t flags) { /* * XXX - The 16-bit string compare probably needs to be done * differently, especially if the flags are to be supported. * At the moment, I am unsure. */ flags = 0; return file_strncmp(a, b, len, flags); } private int magiccheck(struct magic_set *ms, struct magic *m) { uint64_t l = m->value.q; uint64_t v; float fl, fv; double dl, dv; int matched; union VALUETYPE *p = &ms->ms_value; switch (m->type) { case FILE_BYTE: v = p->b; break; case FILE_SHORT: case FILE_BESHORT: case FILE_LESHORT: v = p->h; break; case FILE_LONG: case FILE_BELONG: case FILE_LELONG: case FILE_MELONG: case FILE_DATE: case FILE_BEDATE: case FILE_LEDATE: case FILE_MEDATE: case FILE_LDATE: case FILE_BELDATE: case FILE_LELDATE: case FILE_MELDATE: v = p->l; break; case FILE_QUAD: case FILE_LEQUAD: case FILE_BEQUAD: case FILE_QDATE: case FILE_BEQDATE: case FILE_LEQDATE: case FILE_QLDATE: case FILE_BEQLDATE: case FILE_LEQLDATE: case FILE_QWDATE: case FILE_BEQWDATE: case FILE_LEQWDATE: v = p->q; break; case FILE_FLOAT: case FILE_BEFLOAT: case FILE_LEFLOAT: fl = m->value.f; fv = p->f; switch (m->reln) { case 'x': matched = 1; break; case '!': matched = fv != fl; break; case '=': matched = fv == fl; break; case '>': matched = fv > fl; break; case '<': matched = fv < fl; break; default: file_magerror(ms, "cannot happen with float: invalid relation `%c'", m->reln); return -1; } return matched; case FILE_DOUBLE: case FILE_BEDOUBLE: case FILE_LEDOUBLE: dl = m->value.d; dv = p->d; switch (m->reln) { case 'x': matched = 1; break; case '!': matched = dv != dl; break; case '=': matched = dv == dl; break; case '>': matched = dv > dl; break; case '<': matched = dv < dl; break; default: file_magerror(ms, "cannot happen with double: invalid relation `%c'", m->reln); return -1; } return matched; case FILE_DEFAULT: case FILE_CLEAR: l = 0; v = 0; break; case FILE_STRING: case FILE_PSTRING: l = 0; v = file_strncmp(m->value.s, p->s, (size_t)m->vallen, m->str_flags); break; case FILE_BESTRING16: case FILE_LESTRING16: l = 0; v = file_strncmp16(m->value.s, p->s, (size_t)m->vallen, m->str_flags); break; case FILE_SEARCH: { /* search ms->search.s for the string m->value.s */ size_t slen; size_t idx; if (ms->search.s == NULL) return 0; slen = MIN(m->vallen, sizeof(m->value.s)); l = 0; v = 0; for (idx = 0; m->str_range == 0 || idx < m->str_range; idx++) { if (slen + idx > ms->search.s_len) break; v = file_strncmp(m->value.s, ms->search.s + idx, slen, m->str_flags); if (v == 0) { /* found match */ ms->search.offset += idx; break; } } break; } case FILE_REGEX: { int rc; file_regex_t rx; + const char *search; if (ms->search.s == NULL) return 0; l = 0; rc = file_regcomp(&rx, m->value.s, REG_EXTENDED|REG_NEWLINE| ((m->str_flags & STRING_IGNORE_CASE) ? REG_ICASE : 0)); if (rc) { file_regerror(&rx, rc, ms); v = (uint64_t)-1; } else { regmatch_t pmatch[1]; size_t slen = ms->search.s_len; #ifndef REG_STARTEND #define REG_STARTEND 0 - char c; - if (slen != 0) - slen--; - c = ms->search.s[slen]; - ((char *)(intptr_t)ms->search.s)[slen] = '\0'; + char *copy; + if (slen != 0) { + copy = malloc(slen); + if (copy == NULL) { + file_error(ms, errno, + "can't allocate %" SIZE_T_FORMAT "u bytes", + slen); + return -1; + } + memcpy(copy, ms->search.s, slen); + copy[--slen] = '\0'; + search = copy; + } else { + search = ms->search.s; + copy = NULL; + } #else + search = ms->search.s; pmatch[0].rm_so = 0; pmatch[0].rm_eo = slen; #endif - rc = file_regexec(&rx, (const char *)ms->search.s, + rc = file_regexec(&rx, (const char *)search, 1, pmatch, REG_STARTEND); #if REG_STARTEND == 0 - ((char *)(intptr_t)ms->search.s)[l] = c; + free(copy); #endif switch (rc) { case 0: ms->search.s += (int)pmatch[0].rm_so; ms->search.offset += (size_t)pmatch[0].rm_so; ms->search.rm_len = (size_t)(pmatch[0].rm_eo - pmatch[0].rm_so); v = 0; break; case REG_NOMATCH: v = 1; break; default: file_regerror(&rx, rc, ms); v = (uint64_t)-1; break; } } file_regfree(&rx); if (v == (uint64_t)-1) return -1; break; } case FILE_INDIRECT: case FILE_USE: case FILE_NAME: return 1; default: file_magerror(ms, "invalid type %d in magiccheck()", m->type); return -1; } v = file_signextend(ms, m, v); switch (m->reln) { case 'x': if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u == *any* = 1\n", (unsigned long long)v); matched = 1; break; case '!': matched = v != l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u != %" INT64_T_FORMAT "u = %d\n", (unsigned long long)v, (unsigned long long)l, matched); break; case '=': matched = v == l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u == %" INT64_T_FORMAT "u = %d\n", (unsigned long long)v, (unsigned long long)l, matched); break; case '>': if (m->flag & UNSIGNED) { matched = v > l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u > %" INT64_T_FORMAT "u = %d\n", (unsigned long long)v, (unsigned long long)l, matched); } else { matched = (int64_t) v > (int64_t) l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "d > %" INT64_T_FORMAT "d = %d\n", (long long)v, (long long)l, matched); } break; case '<': if (m->flag & UNSIGNED) { matched = v < l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "u < %" INT64_T_FORMAT "u = %d\n", (unsigned long long)v, (unsigned long long)l, matched); } else { matched = (int64_t) v < (int64_t) l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "%" INT64_T_FORMAT "d < %" INT64_T_FORMAT "d = %d\n", (long long)v, (long long)l, matched); } break; case '&': matched = (v & l) == l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "((%" INT64_T_FORMAT "x & %" INT64_T_FORMAT "x) == %" INT64_T_FORMAT "x) = %d\n", (unsigned long long)v, (unsigned long long)l, (unsigned long long)l, matched); break; case '^': matched = (v & l) != l; if ((ms->flags & MAGIC_DEBUG) != 0) (void) fprintf(stderr, "((%" INT64_T_FORMAT "x & %" INT64_T_FORMAT "x) != %" INT64_T_FORMAT "x) = %d\n", (unsigned long long)v, (unsigned long long)l, (unsigned long long)l, matched); break; default: file_magerror(ms, "cannot happen: invalid relation `%c'", m->reln); return -1; } return matched; } private int handle_annotation(struct magic_set *ms, struct magic *m) { if (ms->flags & MAGIC_APPLE) { if (file_printf(ms, "%.8s", m->apple) == -1) return -1; return 1; } if ((ms->flags & MAGIC_MIME_TYPE) && m->mimetype[0]) { if (file_printf(ms, "%s", m->mimetype) == -1) return -1; return 1; } return 0; } private int print_sep(struct magic_set *ms, int firstline) { if (ms->flags & MAGIC_MIME) return 0; if (firstline) return 0; /* * we found another match * put a newline and '-' to do some simple formatting */ return file_printf(ms, "\n- "); } Index: releng/10.1/contrib/file/src/vasprintf.c =================================================================== --- releng/10.1/contrib/file/src/vasprintf.c (revision 284192) +++ releng/10.1/contrib/file/src/vasprintf.c (revision 284193) @@ -1,653 +1,657 @@ /* * Copyright (c) Ian F. Darwin 1986-1995. * Software written by Ian F. Darwin and others; * maintained 1995-present by Christos Zoulas and others. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice immediately at the beginning of the file, without modification, * this list of conditions, and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /*########################################################################### # # # vasprintf # # # # Copyright (c) 2002-2005 David TAILLANDIER # # # ###########################################################################*/ /* This software is distributed under the "modified BSD licence". This software is also released with GNU license (GPL) in another file (same source-code, only license differ). Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ==================== Hacked from xnprintf version of 26th February 2005 to provide only vasprintf by Reuben Thomas . ==================== 'printf' function family use the following format string: %[flag][width][.prec][modifier]type %% is the escape sequence to print a '%' % followed by an unknown format will print the characters without trying to do any interpretation flag: none + - # (blank) width: n 0n * prec: none .0 .n .* modifier: F N L h l ll z t ('F' and 'N' are ms-dos/16-bit specific) type: d i o u x X f e g E G c s p n The function needs to allocate memory to store the full text before to actually writting it. i.e if you want to fnprintf() 1000 characters, the functions will allocate 1000 bytes. This behaviour can be modified: you have to customise the code to flush the internal buffer (writing to screen or file) when it reach a given size. Then the buffer can have a shorter length. But what? If you really need to write HUGE string, don't use printf! During the process, some other memory is allocated (1024 bytes minimum) to handle the output of partial sprintf() calls. If you have only 10000 bytes free in memory, you *may* not be able to nprintf() a 8000 bytes-long text. note: if a buffer overflow occurs, exit() is called. This situation should never appear ... but if you want to be *really* sure, you have to modify the code to handle those situations (only one place to modify). A buffer overflow can only occur if your sprintf() do strange things or when you use strange formats. */ #include "file.h" #ifndef lint -FILE_RCSID("@(#)$File: vasprintf.c,v 1.12 2014/05/14 23:09:21 christos Exp $") +FILE_RCSID("@(#)$File: vasprintf.c,v 1.13 2014/12/04 15:56:46 christos Exp $") #endif /* lint */ #include #include #include #include #include #ifdef HAVE_LIMITS_H #include #endif #ifdef HAVE_STDDEF_H #include #endif #define ALLOC_CHUNK 2048 #define ALLOC_SECURITY_MARGIN 1024 /* big value because some platforms have very big 'G' exponent */ #if ALLOC_CHUNK < ALLOC_SECURITY_MARGIN # error !!! ALLOC_CHUNK < ALLOC_SECURITY_MARGIN !!! #endif /* note: to have some interest, ALLOC_CHUNK should be much greater than ALLOC_SECURITY_MARGIN */ /* * To save a lot of push/pop, every variable are stored into this * structure, which is passed among nearly every sub-functions. */ typedef struct { const char * src_string; /* current position into intput string */ char * buffer_base; /* output buffer */ char * dest_string; /* current position into output string */ size_t buffer_len; /* length of output buffer */ size_t real_len; /* real current length of output text */ size_t pseudo_len; /* total length of output text if it were not limited in size */ size_t maxlen; va_list vargs; /* pointer to current position into vargs */ char * sprintf_string; FILE * fprintf_file; } xprintf_struct; /* * Realloc buffer if needed * Return value: 0 = ok * EOF = not enought memory */ static int realloc_buff(xprintf_struct *s, size_t len) { char * ptr; if (len + ALLOC_SECURITY_MARGIN + s->real_len > s->buffer_len) { len += s->real_len + ALLOC_CHUNK; ptr = (char *)realloc((void *)(s->buffer_base), len); if (ptr == NULL) { s->buffer_base = NULL; return EOF; } s->dest_string = ptr + (size_t)(s->dest_string - s->buffer_base); s->buffer_base = ptr; s->buffer_len = len; (s->buffer_base)[s->buffer_len - 1] = 1; /* overflow marker */ } return 0; } /* * Prints 'usual' characters up to next '%' * or up to end of text */ static int usual_char(xprintf_struct * s) { size_t len; len = strcspn(s->src_string, "%"); /* reachs the next '%' or end of input string */ /* note: 'len' is never 0 because the presence of '%' */ /* or end-of-line is checked in the calling function */ if (realloc_buff(s,len) == EOF) return EOF; memcpy(s->dest_string, s->src_string, len); s->src_string += len; s->dest_string += len; s->real_len += len; s->pseudo_len += len; return 0; } /* * Return value: 0 = ok * EOF = error */ static int print_it(xprintf_struct *s, size_t approx_len, const char *format_string, ...) { va_list varg; int vsprintf_len; size_t len; if (realloc_buff(s,approx_len) == EOF) return EOF; va_start(varg, format_string); vsprintf_len = vsprintf(s->dest_string, format_string, varg); va_end(varg); /* Check for overflow */ assert((s->buffer_base)[s->buffer_len - 1] == 1); if (vsprintf_len == EOF) /* must be done *after* overflow-check */ return EOF; s->pseudo_len += vsprintf_len; len = strlen(s->dest_string); s->real_len += len; s->dest_string += len; return 0; } /* * Prints a string (%s) * We need special handling because: * a: the length of the string is unknown * b: when .prec is used, we must not access any extra byte of the * string (of course, if the original sprintf() does... what the * hell, not my problem) * * Return value: 0 = ok * EOF = error */ static int type_s(xprintf_struct *s, int width, int prec, const char *format_string, const char *arg_string) { size_t string_len; if (arg_string == NULL) return print_it(s, (size_t)6, "(null)", 0); /* hand-made strlen() whitch stops when 'prec' is reached. */ /* if 'prec' is -1 then it is never reached. */ string_len = 0; while (arg_string[string_len] != 0 && (size_t)prec != string_len) string_len++; if (width != -1 && string_len < (size_t)width) string_len = (size_t)width; return print_it(s, string_len, format_string, arg_string); } /* * Read a serie of digits. Stop when non-digit is found. * Return value: the value read (between 0 and 32767). * Note: no checks are made against overflow. If the string contain a big * number, then the return value won't be what we want (but, in this case, * the programmer don't know whatr he wants, then no problem). */ static int getint(const char **string) { int i = 0; while (isdigit((unsigned char)**string) != 0) { i = i * 10 + (**string - '0'); (*string)++; } if (i < 0 || i > 32767) i = 32767; /* if we have i==-10 this is not because the number is */ /* negative; this is because the number is big */ return i; } /* * Read a part of the format string. A part is 'usual characters' (ie "blabla") * or '%%' escape sequence (to print a single '%') or any combination of * format specifier (ie "%i" or "%10.2d"). * After the current part is managed, the function returns to caller with * everything ready to manage the following part. * The caller must ensure than the string is not empty, i.e. the first byte * is not zero. * * Return value: 0 = ok * EOF = error */ static int dispatch(xprintf_struct *s) { const char *initial_ptr; char format_string[24]; /* max length may be something like "% +-#032768.32768Ld" */ char *format_ptr; int flag_plus, flag_minus, flag_space, flag_sharp, flag_zero; int width, prec, modifier, approx_width; char type; /* most of those variables are here to rewrite the format string */ #define SRCTXT (s->src_string) #define DESTTXT (s->dest_string) /* incoherent format string. Characters after the '%' will be printed with the next call */ #define INCOHERENT() do {SRCTXT=initial_ptr; return 0;} while (0) /* do/while to avoid */ #define INCOHERENT_TEST() do {if(*SRCTXT==0) INCOHERENT();} while (0) /* a null statement */ /* 'normal' text */ if (*SRCTXT != '%') return usual_char(s); /* we then have a '%' */ SRCTXT++; /* don't check for end-of-string ; this is done later */ /* '%%' escape sequence */ if (*SRCTXT == '%') { if (realloc_buff(s, (size_t)1) == EOF) /* because we can have "%%%%%%%%..." */ return EOF; *DESTTXT = '%'; DESTTXT++; SRCTXT++; (s->real_len)++; (s->pseudo_len)++; return 0; } /* '%' managing */ initial_ptr = SRCTXT; /* save current pointer in case of incorrect */ /* 'decoding'. Points just after the '%' so the '%' */ /* won't be printed in any case, as required. */ /* flag */ flag_plus = flag_minus = flag_space = flag_sharp = flag_zero = 0; for (;; SRCTXT++) { if (*SRCTXT == ' ') flag_space = 1; else if (*SRCTXT == '+') flag_plus = 1; else if (*SRCTXT == '-') flag_minus = 1; else if (*SRCTXT == '#') flag_sharp = 1; else if (*SRCTXT == '0') flag_zero = 1; else break; } INCOHERENT_TEST(); /* here is the first test for end of string */ /* width */ if (*SRCTXT == '*') { /* width given by next argument */ SRCTXT++; width = va_arg(s->vargs, int); if ((size_t)width > 0x3fffU) /* 'size_t' to check against negative values too */ width = 0x3fff; } else if (isdigit((unsigned char)*SRCTXT)) /* width given as ASCII number */ width = getint(&SRCTXT); else width = -1; /* no width specified */ INCOHERENT_TEST(); /* .prec */ if (*SRCTXT == '.') { SRCTXT++; if (*SRCTXT == '*') { /* .prec given by next argument */ SRCTXT++; prec = va_arg(s->vargs, int); if ((size_t)prec >= 0x3fffU) /* 'size_t' to check against negative values too */ prec = 0x3fff; } else { /* .prec given as ASCII number */ if (isdigit((unsigned char)*SRCTXT) == 0) INCOHERENT(); prec = getint(&SRCTXT); } INCOHERENT_TEST(); } else prec = -1; /* no .prec specified */ /* modifier */ switch (*SRCTXT) { case 'L': case 'h': case 'l': case 'z': case 't': modifier = *SRCTXT; SRCTXT++; if (modifier=='l' && *SRCTXT=='l') { SRCTXT++; modifier = 'L'; /* 'll' == 'L' long long == long double */ } /* only for compatibility ; not portable */ INCOHERENT_TEST(); break; default: modifier = -1; /* no modifier specified */ break; } /* type */ type = *SRCTXT; if (strchr("diouxXfegEGcspn",type) == NULL) INCOHERENT(); /* unknown type */ SRCTXT++; /* rewrite format-string */ format_string[0] = '%'; format_ptr = &(format_string[1]); if (flag_plus) { *format_ptr = '+'; format_ptr++; } if (flag_minus) { *format_ptr = '-'; format_ptr++; } if (flag_space) { *format_ptr = ' '; format_ptr++; } if (flag_sharp) { *format_ptr = '#'; format_ptr++; } if (flag_zero) { *format_ptr = '0'; format_ptr++; } /* '0' *must* be the last one */ if (width != -1) { sprintf(format_ptr, "%i", width); format_ptr += strlen(format_ptr); } if (prec != -1) { *format_ptr = '.'; format_ptr++; sprintf(format_ptr, "%i", prec); format_ptr += strlen(format_ptr); } if (modifier != -1) { if (modifier == 'L' && strchr("diouxX",type) != NULL) { *format_ptr = 'l'; format_ptr++; *format_ptr = 'l'; format_ptr++; } else { *format_ptr = modifier; format_ptr++; } } *format_ptr = type; format_ptr++; *format_ptr = 0; /* vague approximation of minimal length if width or prec are specified */ approx_width = width + prec; if (approx_width < 0) /* because width == -1 and/or prec == -1 */ approx_width = 0; switch (type) { /* int */ case 'd': case 'i': case 'o': case 'u': case 'x': case 'X': switch (modifier) { case -1 : return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, int)); case 'L': return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, long long int)); case 'l': return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, long int)); case 'h': return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, int)); case 'z': return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, size_t)); case 't': return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, ptrdiff_t)); /* 'int' instead of 'short int' because default promotion is 'int' */ default: INCOHERENT(); } /* char */ case 'c': if (modifier != -1) INCOHERENT(); return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, int)); /* 'int' instead of 'char' because default promotion is 'int' */ /* math */ case 'e': case 'f': case 'g': case 'E': case 'G': switch (modifier) { case -1 : /* because of default promotion, no modifier means 'l' */ case 'l': return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, double)); case 'L': return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, long double)); default: INCOHERENT(); } /* string */ case 's': return type_s(s, width, prec, format_string, va_arg(s->vargs, const char*)); /* pointer */ case 'p': if (modifier == -1) return print_it(s, (size_t)approx_width, format_string, va_arg(s->vargs, void *)); INCOHERENT(); /* store */ case 'n': if (modifier == -1) { int * p; p = va_arg(s->vargs, int *); if (p != NULL) { *p = s->pseudo_len; return 0; } return EOF; } INCOHERENT(); } /* switch */ INCOHERENT(); /* unknown type */ #undef INCOHERENT #undef INCOHERENT_TEST #undef SRCTXT #undef DESTTXT } /* * Return value: number of *virtually* written characters * EOF = error */ static int core(xprintf_struct *s) { size_t save_len; char *dummy_base; /* basic checks */ if ((int)(s->maxlen) <= 0) /* 'int' to check against some conversion */ return EOF; /* error for example if value is (int)-10 */ s->maxlen--; /* because initial maxlen counts final 0 */ /* note: now 'maxlen' _can_ be zero */ if (s->src_string == NULL) s->src_string = "(null)"; /* struct init and memory allocation */ s->buffer_base = NULL; s->buffer_len = 0; s->real_len = 0; s->pseudo_len = 0; if (realloc_buff(s, (size_t)0) == EOF) return EOF; s->dest_string = s->buffer_base; /* process source string */ for (;;) { /* up to end of source string */ if (*(s->src_string) == 0) { *(s->dest_string) = '\0'; /* final NUL */ break; } if (dispatch(s) == EOF) goto free_EOF; /* up to end of dest string */ if (s->real_len >= s->maxlen) { (s->buffer_base)[s->maxlen] = '\0'; /* final NUL */ break; } } /* for (v)asnprintf */ dummy_base = s->buffer_base; dummy_base = s->buffer_base + s->real_len; save_len = s->real_len; /* process the remaining of source string to compute 'pseudo_len'. We * overwrite again and again, starting at 'dummy_base' because we don't * need the text, only char count. */ while(*(s->src_string) != 0) { /* up to end of source string */ s->real_len = 0; s->dest_string = dummy_base; if (dispatch(s) == EOF) goto free_EOF; } s->buffer_base = (char *)realloc((void *)(s->buffer_base), save_len + 1); if (s->buffer_base == NULL) return EOF; /* should rarely happen because we shrink the buffer */ return s->pseudo_len; free_EOF: free(s->buffer_base); return EOF; } int vasprintf(char **ptr, const char *format_string, va_list vargs) { xprintf_struct s; int retval; s.src_string = format_string; #ifdef va_copy va_copy (s.vargs, vargs); #else -#ifdef __va_copy +# ifdef __va_copy __va_copy (s.vargs, vargs); -#else +# else +# ifdef WIN32 + s.vargs = vargs; +# else memcpy (&s.vargs, &vargs, sizeof (s.va_args)); -#endif /* __va_copy */ +# endif /* WIN32 */ +# endif /* __va_copy */ #endif /* va_copy */ s.maxlen = (size_t)INT_MAX; retval = core(&s); va_end(s.vargs); if (retval == EOF) { *ptr = NULL; return EOF; } *ptr = s.buffer_base; return retval; } Index: releng/10.1/lib/libmagic/config.h =================================================================== --- releng/10.1/lib/libmagic/config.h (revision 284192) +++ releng/10.1/lib/libmagic/config.h (revision 284193) @@ -1,391 +1,422 @@ /* $FreeBSD$ */ +/* Get __FreeBSD_version. */ +#include + +/* Only specific versions of FreeBSD support xlocale */ +#if __FreeBSD_version >= 1000002 || (__FreeBSD_version < 1000000 && __FreeBSD_version >= 900506) +#define FREEBSD_XLOCALE_SUPPORT 1 +#endif + /* config.h. Generated from config.h.in by configure. */ /* config.h.in. Generated from configure.ac by autoheader. */ /* Define if building universal (internal helper macro) */ /* #undef AC_APPLE_UNIVERSAL_BUILD */ /* Define in built-in ELF support is used */ #define BUILTIN_ELF 1 /* Define for ELF core file support */ #define ELFCORE 1 /* Define to 1 if you have the `asctime_r' function. */ #define HAVE_ASCTIME_R 1 /* Define to 1 if you have the `asprintf' function. */ #define HAVE_ASPRINTF 1 /* Define to 1 if you have the `ctime_r' function. */ #define HAVE_CTIME_R 1 /* HAVE_DAYLIGHT */ /* #undef HAVE_DAYLIGHT */ /* Define to 1 if you have the declaration of `daylight', and to 0 if you don't. */ #define HAVE_DECL_DAYLIGHT 0 /* Define to 1 if you have the declaration of `tzname', and to 0 if you don't. */ #define HAVE_DECL_TZNAME 1 /* Define to 1 if you have the header file. */ #define HAVE_DLFCN_H 1 /* Define to 1 if you have the header file. */ #define HAVE_ERR_H 1 /* Define to 1 if you have the header file. */ #define HAVE_FCNTL_H 1 /* Define to 1 if you have the `fmtcheck' function. */ #define HAVE_FMTCHECK 1 /* Define to 1 if you have the `fork' function. */ #define HAVE_FORK 1 +/* Define to 1 if you have the `freelocale' function. */ +#ifdef FREEBSD_XLOCALE_SUPPORT +#define HAVE_FREELOCALE 1 +#endif + /* Define to 1 if fseeko (and presumably ftello) exists and is declared. */ #define HAVE_FSEEKO 1 /* Define to 1 if you have the `getline' function. */ #define HAVE_GETLINE 1 /* Define to 1 if you have the header file. */ #define HAVE_GETOPT_H 1 /* Define to 1 if you have the `getopt_long' function. */ #define HAVE_GETOPT_LONG 1 /* Define to 1 if you have the `getpagesize' function. */ #define HAVE_GETPAGESIZE 1 /* Define to 1 if the system has the type `intptr_t'. */ #define HAVE_INTPTR_T 1 /* Define to 1 if you have the header file. */ #define HAVE_INTTYPES_H 1 /* Define to 1 if you have the `gnurx' library (-lgnurx). */ /* #undef HAVE_LIBGNURX */ /* Define to 1 if you have the `z' library (-lz). */ #define HAVE_LIBZ 1 /* Define to 1 if you have the header file. */ #define HAVE_LIMITS_H 1 /* Define to 1 if you have the header file. */ #define HAVE_LOCALE_H 1 /* Define to 1 if mbrtowc and mbstate_t are properly declared. */ #define HAVE_MBRTOWC 1 /* Define to 1 if declares mbstate_t. */ #define HAVE_MBSTATE_T 1 /* Define to 1 if you have the header file. */ #define HAVE_MEMORY_H 1 /* Define to 1 if you have the `mkostemp' function. */ #define HAVE_MKOSTEMP 1 /* Define to 1 if you have the `mkstemp' function. */ #define HAVE_MKSTEMP 1 /* Define to 1 if you have a working `mmap' system call. */ #define HAVE_MMAP 1 +/* Define to 1 if you have the `newlocale' function. */ +#ifdef FREEBSD_XLOCALE_SUPPORT +#define HAVE_NEWLOCALE 1 +#endif + /* Define to 1 if you have the `pread' function. */ #define HAVE_PREAD 1 +/* Define to 1 if you have the `setlocale' function. */ +#define HAVE_SETLOCALE 1 + /* Define to 1 if you have the header file. */ #define HAVE_STDDEF_H 1 /* Define to 1 if you have the header file. */ #define HAVE_STDINT_H 1 /* Define to 1 if you have the header file. */ #define HAVE_STDLIB_H 1 /* Define to 1 if you have the `strcasestr' function. */ #define HAVE_STRCASESTR 1 /* Define to 1 if you have the `strerror' function. */ #define HAVE_STRERROR 1 /* Define to 1 if you have the header file. */ #define HAVE_STRINGS_H 1 /* Define to 1 if you have the header file. */ #define HAVE_STRING_H 1 /* Define to 1 if you have the `strlcat' function. */ #define HAVE_STRLCAT 1 /* Define to 1 if you have the `strlcpy' function. */ #define HAVE_STRLCPY 1 /* Define to 1 if you have the `strndup' function. */ #define HAVE_STRNDUP 1 /* Define to 1 if you have the `strtof' function. */ #define HAVE_STRTOF 1 /* Define to 1 if you have the `strtoul' function. */ #define HAVE_STRTOUL 1 /* HAVE_STRUCT_OPTION */ #define HAVE_STRUCT_OPTION 1 /* Define to 1 if `st_rdev' is a member of `struct stat'. */ #define HAVE_STRUCT_STAT_ST_RDEV 1 /* Define to 1 if `tm_gmtoff' is a member of `struct tm'. */ #define HAVE_STRUCT_TM_TM_GMTOFF 1 /* Define to 1 if `tm_zone' is a member of `struct tm'. */ #define HAVE_STRUCT_TM_TM_ZONE 1 /* Define to 1 if you have the header file. */ #define HAVE_SYS_MMAN_H 1 /* Define to 1 if you have the header file. */ #define HAVE_SYS_PARAM_H 1 /* Define to 1 if you have the header file. */ #define HAVE_SYS_STAT_H 1 /* Define to 1 if you have the header file. */ #define HAVE_SYS_TIME_H 1 /* Define to 1 if you have the header file. */ #define HAVE_SYS_TYPES_H 1 /* Define to 1 if you have the header file. */ /* #undef HAVE_SYS_UTIME_H */ /* Define to 1 if you have that is POSIX.1 compatible. */ #define HAVE_SYS_WAIT_H 1 /* HAVE_TM_ISDST */ #define HAVE_TM_ISDST 1 /* HAVE_TM_ZONE */ #define HAVE_TM_ZONE 1 /* HAVE_TZNAME */ #define HAVE_TZNAME 1 /* Define to 1 if the system has the type `uintptr_t'. */ #define HAVE_UINTPTR_T 1 /* Define to 1 if you have the header file. */ #define HAVE_UNISTD_H 1 +/* Define to 1 if you have the `uselocale' function. */ +#ifdef FREEBSD_XLOCALE_SUPPORT +#define HAVE_USELOCALE 1 +#endif + /* Define to 1 if you have the `utime' function. */ #define HAVE_UTIME 1 /* Define to 1 if you have the `utimes' function. */ #define HAVE_UTIMES 1 /* Define to 1 if you have the header file. */ #define HAVE_UTIME_H 1 /* Define to 1 if you have the `vasprintf' function. */ #define HAVE_VASPRINTF 1 /* Define to 1 if you have the `vfork' function. */ #define HAVE_VFORK 1 /* Define to 1 if you have the header file. */ /* #undef HAVE_VFORK_H */ /* Define to 1 or 0, depending whether the compiler supports simple visibility declarations. */ #define HAVE_VISIBILITY 1 /* Define to 1 if you have the header file. */ #define HAVE_WCHAR_H 1 /* Define to 1 if you have the header file. */ #define HAVE_WCTYPE_H 1 /* Define to 1 if you have the `wcwidth' function. */ #define HAVE_WCWIDTH 1 /* Define to 1 if `fork' works. */ #define HAVE_WORKING_FORK 1 /* Define to 1 if `vfork' works. */ #define HAVE_WORKING_VFORK 1 +/* Define to 1 if you have the header file. */ +#ifdef FREEBSD_XLOCALE_SUPPORT +#define HAVE_XLOCALE_H 1 +#endif + /* Define to 1 if you have the header file. */ #define HAVE_ZLIB_H 1 /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #define LT_OBJDIR ".libs/" /* Define to 1 if `major', `minor', and `makedev' are declared in . */ /* #undef MAJOR_IN_MKDEV */ /* Define to 1 if `major', `minor', and `makedev' are declared in . */ /* #undef MAJOR_IN_SYSMACROS */ /* Name of package */ #define PACKAGE "file" /* Define to the address where bug reports for this package should be sent. */ #define PACKAGE_BUGREPORT "christos@astron.com" /* Define to the full name of this package. */ #define PACKAGE_NAME "file" /* Define to the full name and version of this package. */ -#define PACKAGE_STRING "file 5.19" +#define PACKAGE_STRING "file 5.22" /* Define to the one symbol short name of this package. */ #define PACKAGE_TARNAME "file" /* Define to the home page for this package. */ #define PACKAGE_URL "" /* Define to the version of this package. */ -#define PACKAGE_VERSION "5.19" +#define PACKAGE_VERSION "5.22" /* Define to 1 if you have the ANSI C header files. */ #define STDC_HEADERS 1 /* Define to 1 if your declares `struct tm'. */ /* #undef TM_IN_SYS_TIME */ /* Enable extensions on AIX 3, Interix. */ #ifndef _ALL_SOURCE # define _ALL_SOURCE 1 #endif /* Enable GNU extensions on systems that have them. */ #ifndef _GNU_SOURCE # define _GNU_SOURCE 1 #endif /* Enable threading extensions on Solaris. */ #ifndef _POSIX_PTHREAD_SEMANTICS # define _POSIX_PTHREAD_SEMANTICS 1 #endif /* Enable extensions on HP NonStop. */ #ifndef _TANDEM_SOURCE # define _TANDEM_SOURCE 1 #endif /* Enable general extensions on Solaris. */ #ifndef __EXTENSIONS__ # define __EXTENSIONS__ 1 #endif /* Version number of package */ -#define VERSION "5.19" +#define VERSION "5.22" /* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most significant byte first (like Motorola and SPARC, unlike Intel). */ #if defined AC_APPLE_UNIVERSAL_BUILD # if defined __BIG_ENDIAN__ # define WORDS_BIGENDIAN 1 # endif #else # ifndef WORDS_BIGENDIAN /* # undef WORDS_BIGENDIAN */ # endif #endif /* Enable large inode numbers on Mac OS X 10.5. */ #ifndef _DARWIN_USE_64_BIT_INODE # define _DARWIN_USE_64_BIT_INODE 1 #endif /* Number of bits in a file offset, on hosts where this is settable. */ /* #undef _FILE_OFFSET_BITS */ /* Define to 1 to make fseeko visible on some hosts (e.g. glibc 2.2). */ /* #undef _LARGEFILE_SOURCE */ /* Define for large files, on AIX-style hosts. */ /* #undef _LARGE_FILES */ /* Define to 1 if on MINIX. */ /* #undef _MINIX */ /* Define to 2 if the system does not provide POSIX.1 features except with this defined. */ /* #undef _POSIX_1_SOURCE */ /* Define to 1 if you need to in order for `stat' and other things to work. */ /* #undef _POSIX_SOURCE */ /* Define for Solaris 2.5.1 so the uint32_t typedef from , , or is not used. If the typedef were allowed, the #define below would cause a syntax error. */ /* #undef _UINT32_T */ /* Define for Solaris 2.5.1 so the uint64_t typedef from , , or is not used. If the typedef were allowed, the #define below would cause a syntax error. */ /* #undef _UINT64_T */ /* Define for Solaris 2.5.1 so the uint8_t typedef from , , or is not used. If the typedef were allowed, the #define below would cause a syntax error. */ /* #undef _UINT8_T */ /* Define to empty if `const' does not conform to ANSI C. */ /* #undef const */ /* Define to the type of a signed integer type of width exactly 32 bits if such a type exists and the standard includes do not define it. */ /* #undef int32_t */ /* Define to the type of a signed integer type of width exactly 64 bits if such a type exists and the standard includes do not define it. */ /* #undef int64_t */ /* Define to the type of a signed integer type wide enough to hold a pointer, if such a type exists, and if the system does not define it. */ /* #undef intptr_t */ /* Define to a type if does not define. */ /* #undef mbstate_t */ /* Define to `long int' if does not define. */ /* #undef off_t */ /* Define to `int' if does not define. */ /* #undef pid_t */ /* Define to `unsigned int' if does not define. */ /* #undef size_t */ /* Define to the type of an unsigned integer type of width exactly 16 bits if such a type exists and the standard includes do not define it. */ /* #undef uint16_t */ /* Define to the type of an unsigned integer type of width exactly 32 bits if such a type exists and the standard includes do not define it. */ /* #undef uint32_t */ /* Define to the type of an unsigned integer type of width exactly 64 bits if such a type exists and the standard includes do not define it. */ /* #undef uint64_t */ /* Define to the type of an unsigned integer type of width exactly 8 bits if such a type exists and the standard includes do not define it. */ /* #undef uint8_t */ /* Define to the type of an unsigned integer type wide enough to hold a pointer, if such a type exists, and if the system does not define it. */ /* #undef uintptr_t */ /* Define as `fork' if `vfork' does not work. */ /* #undef vfork */ Index: releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 284192) +++ releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 284193) @@ -1,5599 +1,5644 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2011, 2014 by Delphix. All rights reserved. * Copyright (c) 2014 by Saso Kiselkov. All rights reserved. * Copyright 2014 Nexenta Systems, Inc. All rights reserved. */ /* * DVA-based Adjustable Replacement Cache * * While much of the theory of operation used here is * based on the self-tuning, low overhead replacement cache * presented by Megiddo and Modha at FAST 2003, there are some * significant differences: * * 1. The Megiddo and Modha model assumes any page is evictable. * Pages in its cache cannot be "locked" into memory. This makes * the eviction algorithm simple: evict the last page in the list. * This also make the performance characteristics easy to reason * about. Our cache is not so simple. At any given moment, some * subset of the blocks in the cache are un-evictable because we * have handed out a reference to them. Blocks are only evictable * when there are no external references active. This makes * eviction far more problematic: we choose to evict the evictable * blocks that are the "lowest" in the list. * * There are times when it is not possible to evict the requested * space. In these circumstances we are unable to adjust the cache * size. To prevent the cache growing unbounded at these times we * implement a "cache throttle" that slows the flow of new data * into the cache until we can make space available. * * 2. The Megiddo and Modha model assumes a fixed cache size. * Pages are evicted when the cache is full and there is a cache * miss. Our model has a variable sized cache. It grows with * high use, but also tries to react to memory pressure from the * operating system: decreasing its size when system memory is * tight. * * 3. The Megiddo and Modha model assumes a fixed page size. All * elements of the cache are therefore exactly the same size. So * when adjusting the cache size following a cache miss, its simply * a matter of choosing a single page to evict. In our model, we * have variable sized cache blocks (rangeing from 512 bytes to * 128K bytes). We therefore choose a set of blocks to evict to make * space for a cache miss that approximates as closely as possible * the space used by the new block. * * See also: "ARC: A Self-Tuning, Low Overhead Replacement Cache" * by N. Megiddo & D. Modha, FAST 2003 */ /* * The locking model: * * A new reference to a cache buffer can be obtained in two * ways: 1) via a hash table lookup using the DVA as a key, * or 2) via one of the ARC lists. The arc_read() interface * uses method 1, while the internal arc algorithms for * adjusting the cache use method 2. We therefore provide two * types of locks: 1) the hash table lock array, and 2) the * arc list locks. * * Buffers do not have their own mutexs, rather they rely on the * hash table mutexs for the bulk of their protection (i.e. most * fields in the arc_buf_hdr_t are protected by these mutexs). * * buf_hash_find() returns the appropriate mutex (held) when it * locates the requested buffer in the hash table. It returns * NULL for the mutex if the buffer was not in the table. * * buf_hash_remove() expects the appropriate hash mutex to be * already held before it is invoked. * * Each arc state also has a mutex which is used to protect the * buffer list associated with the state. When attempting to * obtain a hash table lock while holding an arc list lock you * must use: mutex_tryenter() to avoid deadlock. Also note that * the active state mutex must be held before the ghost state mutex. * * Arc buffers may have an associated eviction callback function. * This function will be invoked prior to removing the buffer (e.g. * in arc_do_user_evicts()). Note however that the data associated * with the buffer may be evicted prior to the callback. The callback * must be made with *no locks held* (to prevent deadlock). Additionally, * the users of callbacks must ensure that their private data is * protected from simultaneous callbacks from arc_clear_callback() * and arc_do_user_evicts(). * * Note that the majority of the performance stats are manipulated * with atomic operations. * * The L2ARC uses the l2arc_buflist_mtx global mutex for the following: * * - L2ARC buflist creation * - L2ARC buflist eviction * - L2ARC write completion, which walks L2ARC buflists * - ARC header destruction, as it removes from L2ARC buflists * - ARC header release, as it removes from L2ARC buflists */ #include #include #include #include #include #include #include #include #include #ifdef _KERNEL #include #endif #include #include #include #include #include #include #ifdef illumos #ifndef _KERNEL /* set with ZFS_DEBUG=watch, to enable watchpoints on frozen buffers */ boolean_t arc_watch = B_FALSE; int arc_procfd; #endif #endif /* illumos */ static kmutex_t arc_reclaim_thr_lock; static kcondvar_t arc_reclaim_thr_cv; /* used to signal reclaim thr */ static uint8_t arc_thread_exit; #define ARC_REDUCE_DNLC_PERCENT 3 uint_t arc_reduce_dnlc_percent = ARC_REDUCE_DNLC_PERCENT; typedef enum arc_reclaim_strategy { ARC_RECLAIM_AGGR, /* Aggressive reclaim strategy */ ARC_RECLAIM_CONS /* Conservative reclaim strategy */ } arc_reclaim_strategy_t; /* * The number of iterations through arc_evict_*() before we * drop & reacquire the lock. */ int arc_evict_iterations = 100; /* number of seconds before growing cache again */ static int arc_grow_retry = 60; /* shift of arc_c for calculating both min and max arc_p */ static int arc_p_min_shift = 4; /* log2(fraction of arc to reclaim) */ static int arc_shrink_shift = 5; /* * minimum lifespan of a prefetch block in clock ticks * (initialized in arc_init()) */ static int arc_min_prefetch_lifespan; /* * If this percent of memory is free, don't throttle. */ int arc_lotsfree_percent = 10; static int arc_dead; extern int zfs_prefetch_disable; /* * The arc has filled available memory and has now warmed up. */ static boolean_t arc_warm; /* * These tunables are for performance analysis. */ uint64_t zfs_arc_max; uint64_t zfs_arc_min; uint64_t zfs_arc_meta_limit = 0; int zfs_arc_grow_retry = 0; int zfs_arc_shrink_shift = 0; int zfs_arc_p_min_shift = 0; int zfs_disable_dup_eviction = 0; uint64_t zfs_arc_average_blocksize = 8 * 1024; /* 8KB */ TUNABLE_QUAD("vfs.zfs.arc_max", &zfs_arc_max); TUNABLE_QUAD("vfs.zfs.arc_min", &zfs_arc_min); TUNABLE_QUAD("vfs.zfs.arc_meta_limit", &zfs_arc_meta_limit); TUNABLE_QUAD("vfs.zfs.arc_average_blocksize", &zfs_arc_average_blocksize); SYSCTL_DECL(_vfs_zfs); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_max, CTLFLAG_RDTUN, &zfs_arc_max, 0, "Maximum ARC size"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_min, CTLFLAG_RDTUN, &zfs_arc_min, 0, "Minimum ARC size"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_average_blocksize, CTLFLAG_RDTUN, &zfs_arc_average_blocksize, 0, "ARC average blocksize"); /* * Note that buffers can be in one of 6 states: * ARC_anon - anonymous (discussed below) * ARC_mru - recently used, currently cached * ARC_mru_ghost - recentely used, no longer in cache * ARC_mfu - frequently used, currently cached * ARC_mfu_ghost - frequently used, no longer in cache * ARC_l2c_only - exists in L2ARC but not other states * When there are no active references to the buffer, they are * are linked onto a list in one of these arc states. These are * the only buffers that can be evicted or deleted. Within each * state there are multiple lists, one for meta-data and one for * non-meta-data. Meta-data (indirect blocks, blocks of dnodes, * etc.) is tracked separately so that it can be managed more * explicitly: favored over data, limited explicitly. * * Anonymous buffers are buffers that are not associated with * a DVA. These are buffers that hold dirty block copies * before they are written to stable storage. By definition, * they are "ref'd" and are considered part of arc_mru * that cannot be freed. Generally, they will aquire a DVA * as they are written and migrate onto the arc_mru list. * * The ARC_l2c_only state is for buffers that are in the second * level ARC but no longer in any of the ARC_m* lists. The second * level ARC itself may also contain buffers that are in any of * the ARC_m* states - meaning that a buffer can exist in two * places. The reason for the ARC_l2c_only state is to keep the * buffer header in the hash table, so that reads that hit the * second level ARC benefit from these fast lookups. */ #define ARCS_LOCK_PAD CACHE_LINE_SIZE struct arcs_lock { kmutex_t arcs_lock; #ifdef _KERNEL unsigned char pad[(ARCS_LOCK_PAD - sizeof (kmutex_t))]; #endif }; /* * must be power of two for mask use to work * */ #define ARC_BUFC_NUMDATALISTS 16 #define ARC_BUFC_NUMMETADATALISTS 16 #define ARC_BUFC_NUMLISTS (ARC_BUFC_NUMMETADATALISTS + ARC_BUFC_NUMDATALISTS) typedef struct arc_state { uint64_t arcs_lsize[ARC_BUFC_NUMTYPES]; /* amount of evictable data */ uint64_t arcs_size; /* total amount of data in this state */ list_t arcs_lists[ARC_BUFC_NUMLISTS]; /* list of evictable buffers */ struct arcs_lock arcs_locks[ARC_BUFC_NUMLISTS] __aligned(CACHE_LINE_SIZE); } arc_state_t; #define ARCS_LOCK(s, i) (&((s)->arcs_locks[(i)].arcs_lock)) /* The 6 states: */ static arc_state_t ARC_anon; static arc_state_t ARC_mru; static arc_state_t ARC_mru_ghost; static arc_state_t ARC_mfu; static arc_state_t ARC_mfu_ghost; static arc_state_t ARC_l2c_only; typedef struct arc_stats { kstat_named_t arcstat_hits; kstat_named_t arcstat_misses; kstat_named_t arcstat_demand_data_hits; kstat_named_t arcstat_demand_data_misses; kstat_named_t arcstat_demand_metadata_hits; kstat_named_t arcstat_demand_metadata_misses; kstat_named_t arcstat_prefetch_data_hits; kstat_named_t arcstat_prefetch_data_misses; kstat_named_t arcstat_prefetch_metadata_hits; kstat_named_t arcstat_prefetch_metadata_misses; kstat_named_t arcstat_mru_hits; kstat_named_t arcstat_mru_ghost_hits; kstat_named_t arcstat_mfu_hits; kstat_named_t arcstat_mfu_ghost_hits; kstat_named_t arcstat_allocated; kstat_named_t arcstat_deleted; kstat_named_t arcstat_stolen; kstat_named_t arcstat_recycle_miss; /* * Number of buffers that could not be evicted because the hash lock * was held by another thread. The lock may not necessarily be held * by something using the same buffer, since hash locks are shared * by multiple buffers. */ kstat_named_t arcstat_mutex_miss; /* * Number of buffers skipped because they have I/O in progress, are * indrect prefetch buffers that have not lived long enough, or are * not from the spa we're trying to evict from. */ kstat_named_t arcstat_evict_skip; kstat_named_t arcstat_evict_l2_cached; kstat_named_t arcstat_evict_l2_eligible; kstat_named_t arcstat_evict_l2_ineligible; kstat_named_t arcstat_hash_elements; kstat_named_t arcstat_hash_elements_max; kstat_named_t arcstat_hash_collisions; kstat_named_t arcstat_hash_chains; kstat_named_t arcstat_hash_chain_max; kstat_named_t arcstat_p; kstat_named_t arcstat_c; kstat_named_t arcstat_c_min; kstat_named_t arcstat_c_max; kstat_named_t arcstat_size; kstat_named_t arcstat_hdr_size; kstat_named_t arcstat_data_size; kstat_named_t arcstat_other_size; kstat_named_t arcstat_l2_hits; kstat_named_t arcstat_l2_misses; kstat_named_t arcstat_l2_feeds; kstat_named_t arcstat_l2_rw_clash; kstat_named_t arcstat_l2_read_bytes; kstat_named_t arcstat_l2_write_bytes; kstat_named_t arcstat_l2_writes_sent; kstat_named_t arcstat_l2_writes_done; kstat_named_t arcstat_l2_writes_error; kstat_named_t arcstat_l2_writes_hdr_miss; kstat_named_t arcstat_l2_evict_lock_retry; kstat_named_t arcstat_l2_evict_reading; kstat_named_t arcstat_l2_free_on_write; + kstat_named_t arcstat_l2_cdata_free_on_write; kstat_named_t arcstat_l2_abort_lowmem; kstat_named_t arcstat_l2_cksum_bad; kstat_named_t arcstat_l2_io_error; kstat_named_t arcstat_l2_size; kstat_named_t arcstat_l2_asize; kstat_named_t arcstat_l2_hdr_size; kstat_named_t arcstat_l2_compress_successes; kstat_named_t arcstat_l2_compress_zeros; kstat_named_t arcstat_l2_compress_failures; kstat_named_t arcstat_l2_write_trylock_fail; kstat_named_t arcstat_l2_write_passed_headroom; kstat_named_t arcstat_l2_write_spa_mismatch; kstat_named_t arcstat_l2_write_in_l2; kstat_named_t arcstat_l2_write_hdr_io_in_progress; kstat_named_t arcstat_l2_write_not_cacheable; kstat_named_t arcstat_l2_write_full; kstat_named_t arcstat_l2_write_buffer_iter; kstat_named_t arcstat_l2_write_pios; kstat_named_t arcstat_l2_write_buffer_bytes_scanned; kstat_named_t arcstat_l2_write_buffer_list_iter; kstat_named_t arcstat_l2_write_buffer_list_null_iter; kstat_named_t arcstat_memory_throttle_count; kstat_named_t arcstat_duplicate_buffers; kstat_named_t arcstat_duplicate_buffers_size; kstat_named_t arcstat_duplicate_reads; } arc_stats_t; static arc_stats_t arc_stats = { { "hits", KSTAT_DATA_UINT64 }, { "misses", KSTAT_DATA_UINT64 }, { "demand_data_hits", KSTAT_DATA_UINT64 }, { "demand_data_misses", KSTAT_DATA_UINT64 }, { "demand_metadata_hits", KSTAT_DATA_UINT64 }, { "demand_metadata_misses", KSTAT_DATA_UINT64 }, { "prefetch_data_hits", KSTAT_DATA_UINT64 }, { "prefetch_data_misses", KSTAT_DATA_UINT64 }, { "prefetch_metadata_hits", KSTAT_DATA_UINT64 }, { "prefetch_metadata_misses", KSTAT_DATA_UINT64 }, { "mru_hits", KSTAT_DATA_UINT64 }, { "mru_ghost_hits", KSTAT_DATA_UINT64 }, { "mfu_hits", KSTAT_DATA_UINT64 }, { "mfu_ghost_hits", KSTAT_DATA_UINT64 }, { "allocated", KSTAT_DATA_UINT64 }, { "deleted", KSTAT_DATA_UINT64 }, { "stolen", KSTAT_DATA_UINT64 }, { "recycle_miss", KSTAT_DATA_UINT64 }, { "mutex_miss", KSTAT_DATA_UINT64 }, { "evict_skip", KSTAT_DATA_UINT64 }, { "evict_l2_cached", KSTAT_DATA_UINT64 }, { "evict_l2_eligible", KSTAT_DATA_UINT64 }, { "evict_l2_ineligible", KSTAT_DATA_UINT64 }, { "hash_elements", KSTAT_DATA_UINT64 }, { "hash_elements_max", KSTAT_DATA_UINT64 }, { "hash_collisions", KSTAT_DATA_UINT64 }, { "hash_chains", KSTAT_DATA_UINT64 }, { "hash_chain_max", KSTAT_DATA_UINT64 }, { "p", KSTAT_DATA_UINT64 }, { "c", KSTAT_DATA_UINT64 }, { "c_min", KSTAT_DATA_UINT64 }, { "c_max", KSTAT_DATA_UINT64 }, { "size", KSTAT_DATA_UINT64 }, { "hdr_size", KSTAT_DATA_UINT64 }, { "data_size", KSTAT_DATA_UINT64 }, { "other_size", KSTAT_DATA_UINT64 }, { "l2_hits", KSTAT_DATA_UINT64 }, { "l2_misses", KSTAT_DATA_UINT64 }, { "l2_feeds", KSTAT_DATA_UINT64 }, { "l2_rw_clash", KSTAT_DATA_UINT64 }, { "l2_read_bytes", KSTAT_DATA_UINT64 }, { "l2_write_bytes", KSTAT_DATA_UINT64 }, { "l2_writes_sent", KSTAT_DATA_UINT64 }, { "l2_writes_done", KSTAT_DATA_UINT64 }, { "l2_writes_error", KSTAT_DATA_UINT64 }, { "l2_writes_hdr_miss", KSTAT_DATA_UINT64 }, { "l2_evict_lock_retry", KSTAT_DATA_UINT64 }, { "l2_evict_reading", KSTAT_DATA_UINT64 }, { "l2_free_on_write", KSTAT_DATA_UINT64 }, + { "l2_cdata_free_on_write", KSTAT_DATA_UINT64 }, { "l2_abort_lowmem", KSTAT_DATA_UINT64 }, { "l2_cksum_bad", KSTAT_DATA_UINT64 }, { "l2_io_error", KSTAT_DATA_UINT64 }, { "l2_size", KSTAT_DATA_UINT64 }, { "l2_asize", KSTAT_DATA_UINT64 }, { "l2_hdr_size", KSTAT_DATA_UINT64 }, { "l2_compress_successes", KSTAT_DATA_UINT64 }, { "l2_compress_zeros", KSTAT_DATA_UINT64 }, { "l2_compress_failures", KSTAT_DATA_UINT64 }, { "l2_write_trylock_fail", KSTAT_DATA_UINT64 }, { "l2_write_passed_headroom", KSTAT_DATA_UINT64 }, { "l2_write_spa_mismatch", KSTAT_DATA_UINT64 }, { "l2_write_in_l2", KSTAT_DATA_UINT64 }, { "l2_write_io_in_progress", KSTAT_DATA_UINT64 }, { "l2_write_not_cacheable", KSTAT_DATA_UINT64 }, { "l2_write_full", KSTAT_DATA_UINT64 }, { "l2_write_buffer_iter", KSTAT_DATA_UINT64 }, { "l2_write_pios", KSTAT_DATA_UINT64 }, { "l2_write_buffer_bytes_scanned", KSTAT_DATA_UINT64 }, { "l2_write_buffer_list_iter", KSTAT_DATA_UINT64 }, { "l2_write_buffer_list_null_iter", KSTAT_DATA_UINT64 }, { "memory_throttle_count", KSTAT_DATA_UINT64 }, { "duplicate_buffers", KSTAT_DATA_UINT64 }, { "duplicate_buffers_size", KSTAT_DATA_UINT64 }, { "duplicate_reads", KSTAT_DATA_UINT64 } }; #define ARCSTAT(stat) (arc_stats.stat.value.ui64) #define ARCSTAT_INCR(stat, val) \ atomic_add_64(&arc_stats.stat.value.ui64, (val)) #define ARCSTAT_BUMP(stat) ARCSTAT_INCR(stat, 1) #define ARCSTAT_BUMPDOWN(stat) ARCSTAT_INCR(stat, -1) #define ARCSTAT_MAX(stat, val) { \ uint64_t m; \ while ((val) > (m = arc_stats.stat.value.ui64) && \ (m != atomic_cas_64(&arc_stats.stat.value.ui64, m, (val)))) \ continue; \ } #define ARCSTAT_MAXSTAT(stat) \ ARCSTAT_MAX(stat##_max, arc_stats.stat.value.ui64) /* * We define a macro to allow ARC hits/misses to be easily broken down by * two separate conditions, giving a total of four different subtypes for * each of hits and misses (so eight statistics total). */ #define ARCSTAT_CONDSTAT(cond1, stat1, notstat1, cond2, stat2, notstat2, stat) \ if (cond1) { \ if (cond2) { \ ARCSTAT_BUMP(arcstat_##stat1##_##stat2##_##stat); \ } else { \ ARCSTAT_BUMP(arcstat_##stat1##_##notstat2##_##stat); \ } \ } else { \ if (cond2) { \ ARCSTAT_BUMP(arcstat_##notstat1##_##stat2##_##stat); \ } else { \ ARCSTAT_BUMP(arcstat_##notstat1##_##notstat2##_##stat);\ } \ } kstat_t *arc_ksp; static arc_state_t *arc_anon; static arc_state_t *arc_mru; static arc_state_t *arc_mru_ghost; static arc_state_t *arc_mfu; static arc_state_t *arc_mfu_ghost; static arc_state_t *arc_l2c_only; /* * There are several ARC variables that are critical to export as kstats -- * but we don't want to have to grovel around in the kstat whenever we wish to * manipulate them. For these variables, we therefore define them to be in * terms of the statistic variable. This assures that we are not introducing * the possibility of inconsistency by having shadow copies of the variables, * while still allowing the code to be readable. */ #define arc_size ARCSTAT(arcstat_size) /* actual total arc size */ #define arc_p ARCSTAT(arcstat_p) /* target size of MRU */ #define arc_c ARCSTAT(arcstat_c) /* target size of cache */ #define arc_c_min ARCSTAT(arcstat_c_min) /* min target cache size */ #define arc_c_max ARCSTAT(arcstat_c_max) /* max target cache size */ #define L2ARC_IS_VALID_COMPRESS(_c_) \ ((_c_) == ZIO_COMPRESS_LZ4 || (_c_) == ZIO_COMPRESS_EMPTY) static int arc_no_grow; /* Don't try to grow cache size */ static uint64_t arc_tempreserve; static uint64_t arc_loaned_bytes; static uint64_t arc_meta_used; static uint64_t arc_meta_limit; static uint64_t arc_meta_max = 0; SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_used, CTLFLAG_RD, &arc_meta_used, 0, "ARC metadata used"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, arc_meta_limit, CTLFLAG_RW, &arc_meta_limit, 0, "ARC metadata limit"); typedef struct l2arc_buf_hdr l2arc_buf_hdr_t; typedef struct arc_callback arc_callback_t; struct arc_callback { void *acb_private; arc_done_func_t *acb_done; arc_buf_t *acb_buf; zio_t *acb_zio_dummy; arc_callback_t *acb_next; }; typedef struct arc_write_callback arc_write_callback_t; struct arc_write_callback { void *awcb_private; arc_done_func_t *awcb_ready; arc_done_func_t *awcb_physdone; arc_done_func_t *awcb_done; arc_buf_t *awcb_buf; }; struct arc_buf_hdr { /* protected by hash lock */ dva_t b_dva; uint64_t b_birth; uint64_t b_cksum0; kmutex_t b_freeze_lock; zio_cksum_t *b_freeze_cksum; void *b_thawed; arc_buf_hdr_t *b_hash_next; arc_buf_t *b_buf; uint32_t b_flags; uint32_t b_datacnt; arc_callback_t *b_acb; kcondvar_t b_cv; /* immutable */ arc_buf_contents_t b_type; uint64_t b_size; uint64_t b_spa; /* protected by arc state mutex */ arc_state_t *b_state; list_node_t b_arc_node; /* updated atomically */ clock_t b_arc_access; /* self protecting */ refcount_t b_refcnt; l2arc_buf_hdr_t *b_l2hdr; list_node_t b_l2node; }; static arc_buf_t *arc_eviction_list; static kmutex_t arc_eviction_mtx; static arc_buf_hdr_t arc_eviction_hdr; static void arc_get_data_buf(arc_buf_t *buf); static void arc_access(arc_buf_hdr_t *buf, kmutex_t *hash_lock); static int arc_evict_needed(arc_buf_contents_t type); static void arc_evict_ghost(arc_state_t *state, uint64_t spa, int64_t bytes); #ifdef illumos static void arc_buf_watch(arc_buf_t *buf); #endif /* illumos */ static boolean_t l2arc_write_eligible(uint64_t spa_guid, arc_buf_hdr_t *ab); #define GHOST_STATE(state) \ ((state) == arc_mru_ghost || (state) == arc_mfu_ghost || \ (state) == arc_l2c_only) /* * Private ARC flags. These flags are private ARC only flags that will show up * in b_flags in the arc_hdr_buf_t. Some flags are publicly declared, and can * be passed in as arc_flags in things like arc_read. However, these flags * should never be passed and should only be set by ARC code. When adding new * public flags, make sure not to smash the private ones. */ #define ARC_IN_HASH_TABLE (1 << 9) /* this buffer is hashed */ #define ARC_IO_IN_PROGRESS (1 << 10) /* I/O in progress for buf */ #define ARC_IO_ERROR (1 << 11) /* I/O failed for buf */ #define ARC_FREED_IN_READ (1 << 12) /* buf freed while in read */ #define ARC_BUF_AVAILABLE (1 << 13) /* block not in active use */ #define ARC_INDIRECT (1 << 14) /* this is an indirect block */ #define ARC_FREE_IN_PROGRESS (1 << 15) /* hdr about to be freed */ #define ARC_L2_WRITING (1 << 16) /* L2ARC write in progress */ #define ARC_L2_EVICTED (1 << 17) /* evicted during I/O */ #define ARC_L2_WRITE_HEAD (1 << 18) /* head of write list */ #define HDR_IN_HASH_TABLE(hdr) ((hdr)->b_flags & ARC_IN_HASH_TABLE) #define HDR_IO_IN_PROGRESS(hdr) ((hdr)->b_flags & ARC_IO_IN_PROGRESS) #define HDR_IO_ERROR(hdr) ((hdr)->b_flags & ARC_IO_ERROR) #define HDR_PREFETCH(hdr) ((hdr)->b_flags & ARC_PREFETCH) #define HDR_FREED_IN_READ(hdr) ((hdr)->b_flags & ARC_FREED_IN_READ) #define HDR_BUF_AVAILABLE(hdr) ((hdr)->b_flags & ARC_BUF_AVAILABLE) #define HDR_FREE_IN_PROGRESS(hdr) ((hdr)->b_flags & ARC_FREE_IN_PROGRESS) #define HDR_L2CACHE(hdr) ((hdr)->b_flags & ARC_L2CACHE) #define HDR_L2_READING(hdr) ((hdr)->b_flags & ARC_IO_IN_PROGRESS && \ (hdr)->b_l2hdr != NULL) #define HDR_L2_WRITING(hdr) ((hdr)->b_flags & ARC_L2_WRITING) #define HDR_L2_EVICTED(hdr) ((hdr)->b_flags & ARC_L2_EVICTED) #define HDR_L2_WRITE_HEAD(hdr) ((hdr)->b_flags & ARC_L2_WRITE_HEAD) /* * Other sizes */ #define HDR_SIZE ((int64_t)sizeof (arc_buf_hdr_t)) #define L2HDR_SIZE ((int64_t)sizeof (l2arc_buf_hdr_t)) /* * Hash table routines */ #define HT_LOCK_PAD CACHE_LINE_SIZE struct ht_lock { kmutex_t ht_lock; #ifdef _KERNEL unsigned char pad[(HT_LOCK_PAD - sizeof (kmutex_t))]; #endif }; #define BUF_LOCKS 256 typedef struct buf_hash_table { uint64_t ht_mask; arc_buf_hdr_t **ht_table; struct ht_lock ht_locks[BUF_LOCKS] __aligned(CACHE_LINE_SIZE); } buf_hash_table_t; static buf_hash_table_t buf_hash_table; #define BUF_HASH_INDEX(spa, dva, birth) \ (buf_hash(spa, dva, birth) & buf_hash_table.ht_mask) #define BUF_HASH_LOCK_NTRY(idx) (buf_hash_table.ht_locks[idx & (BUF_LOCKS-1)]) #define BUF_HASH_LOCK(idx) (&(BUF_HASH_LOCK_NTRY(idx).ht_lock)) #define HDR_LOCK(hdr) \ (BUF_HASH_LOCK(BUF_HASH_INDEX(hdr->b_spa, &hdr->b_dva, hdr->b_birth))) uint64_t zfs_crc64_table[256]; /* * Level 2 ARC */ #define L2ARC_WRITE_SIZE (8 * 1024 * 1024) /* initial write max */ #define L2ARC_HEADROOM 2 /* num of writes */ /* * If we discover during ARC scan any buffers to be compressed, we boost * our headroom for the next scanning cycle by this percentage multiple. */ #define L2ARC_HEADROOM_BOOST 200 #define L2ARC_FEED_SECS 1 /* caching interval secs */ #define L2ARC_FEED_MIN_MS 200 /* min caching interval ms */ #define l2arc_writes_sent ARCSTAT(arcstat_l2_writes_sent) #define l2arc_writes_done ARCSTAT(arcstat_l2_writes_done) /* L2ARC Performance Tunables */ uint64_t l2arc_write_max = L2ARC_WRITE_SIZE; /* default max write size */ uint64_t l2arc_write_boost = L2ARC_WRITE_SIZE; /* extra write during warmup */ uint64_t l2arc_headroom = L2ARC_HEADROOM; /* number of dev writes */ uint64_t l2arc_headroom_boost = L2ARC_HEADROOM_BOOST; uint64_t l2arc_feed_secs = L2ARC_FEED_SECS; /* interval seconds */ uint64_t l2arc_feed_min_ms = L2ARC_FEED_MIN_MS; /* min interval milliseconds */ boolean_t l2arc_noprefetch = B_TRUE; /* don't cache prefetch bufs */ boolean_t l2arc_feed_again = B_TRUE; /* turbo warmup */ boolean_t l2arc_norw = B_TRUE; /* no reads during writes */ SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, l2arc_write_max, CTLFLAG_RW, &l2arc_write_max, 0, "max write size"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, l2arc_write_boost, CTLFLAG_RW, &l2arc_write_boost, 0, "extra write during warmup"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, l2arc_headroom, CTLFLAG_RW, &l2arc_headroom, 0, "number of dev writes"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, l2arc_feed_secs, CTLFLAG_RW, &l2arc_feed_secs, 0, "interval seconds"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, l2arc_feed_min_ms, CTLFLAG_RW, &l2arc_feed_min_ms, 0, "min interval milliseconds"); SYSCTL_INT(_vfs_zfs, OID_AUTO, l2arc_noprefetch, CTLFLAG_RW, &l2arc_noprefetch, 0, "don't cache prefetch bufs"); SYSCTL_INT(_vfs_zfs, OID_AUTO, l2arc_feed_again, CTLFLAG_RW, &l2arc_feed_again, 0, "turbo warmup"); SYSCTL_INT(_vfs_zfs, OID_AUTO, l2arc_norw, CTLFLAG_RW, &l2arc_norw, 0, "no reads during writes"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, anon_size, CTLFLAG_RD, &ARC_anon.arcs_size, 0, "size of anonymous state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, anon_metadata_lsize, CTLFLAG_RD, &ARC_anon.arcs_lsize[ARC_BUFC_METADATA], 0, "size of anonymous state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, anon_data_lsize, CTLFLAG_RD, &ARC_anon.arcs_lsize[ARC_BUFC_DATA], 0, "size of anonymous state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mru_size, CTLFLAG_RD, &ARC_mru.arcs_size, 0, "size of mru state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mru_metadata_lsize, CTLFLAG_RD, &ARC_mru.arcs_lsize[ARC_BUFC_METADATA], 0, "size of metadata in mru state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mru_data_lsize, CTLFLAG_RD, &ARC_mru.arcs_lsize[ARC_BUFC_DATA], 0, "size of data in mru state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mru_ghost_size, CTLFLAG_RD, &ARC_mru_ghost.arcs_size, 0, "size of mru ghost state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mru_ghost_metadata_lsize, CTLFLAG_RD, &ARC_mru_ghost.arcs_lsize[ARC_BUFC_METADATA], 0, "size of metadata in mru ghost state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mru_ghost_data_lsize, CTLFLAG_RD, &ARC_mru_ghost.arcs_lsize[ARC_BUFC_DATA], 0, "size of data in mru ghost state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mfu_size, CTLFLAG_RD, &ARC_mfu.arcs_size, 0, "size of mfu state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mfu_metadata_lsize, CTLFLAG_RD, &ARC_mfu.arcs_lsize[ARC_BUFC_METADATA], 0, "size of metadata in mfu state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mfu_data_lsize, CTLFLAG_RD, &ARC_mfu.arcs_lsize[ARC_BUFC_DATA], 0, "size of data in mfu state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mfu_ghost_size, CTLFLAG_RD, &ARC_mfu_ghost.arcs_size, 0, "size of mfu ghost state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mfu_ghost_metadata_lsize, CTLFLAG_RD, &ARC_mfu_ghost.arcs_lsize[ARC_BUFC_METADATA], 0, "size of metadata in mfu ghost state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, mfu_ghost_data_lsize, CTLFLAG_RD, &ARC_mfu_ghost.arcs_lsize[ARC_BUFC_DATA], 0, "size of data in mfu ghost state"); SYSCTL_UQUAD(_vfs_zfs, OID_AUTO, l2c_only_size, CTLFLAG_RD, &ARC_l2c_only.arcs_size, 0, "size of mru state"); /* * L2ARC Internals */ typedef struct l2arc_dev { vdev_t *l2ad_vdev; /* vdev */ spa_t *l2ad_spa; /* spa */ uint64_t l2ad_hand; /* next write location */ uint64_t l2ad_start; /* first addr on device */ uint64_t l2ad_end; /* last addr on device */ uint64_t l2ad_evict; /* last addr eviction reached */ boolean_t l2ad_first; /* first sweep through */ boolean_t l2ad_writing; /* currently writing */ list_t *l2ad_buflist; /* buffer list */ list_node_t l2ad_node; /* device list node */ } l2arc_dev_t; static list_t L2ARC_dev_list; /* device list */ static list_t *l2arc_dev_list; /* device list pointer */ static kmutex_t l2arc_dev_mtx; /* device list mutex */ static l2arc_dev_t *l2arc_dev_last; /* last device used */ static kmutex_t l2arc_buflist_mtx; /* mutex for all buflists */ static list_t L2ARC_free_on_write; /* free after write buf list */ static list_t *l2arc_free_on_write; /* free after write list ptr */ static kmutex_t l2arc_free_on_write_mtx; /* mutex for list */ static uint64_t l2arc_ndev; /* number of devices */ typedef struct l2arc_read_callback { arc_buf_t *l2rcb_buf; /* read buffer */ spa_t *l2rcb_spa; /* spa */ blkptr_t l2rcb_bp; /* original blkptr */ zbookmark_phys_t l2rcb_zb; /* original bookmark */ int l2rcb_flags; /* original flags */ enum zio_compress l2rcb_compress; /* applied compress */ } l2arc_read_callback_t; typedef struct l2arc_write_callback { l2arc_dev_t *l2wcb_dev; /* device info */ arc_buf_hdr_t *l2wcb_head; /* head of write buflist */ } l2arc_write_callback_t; struct l2arc_buf_hdr { /* protected by arc_buf_hdr mutex */ l2arc_dev_t *b_dev; /* L2ARC device */ uint64_t b_daddr; /* disk address, offset byte */ /* compression applied to buffer data */ enum zio_compress b_compress; /* real alloc'd buffer size depending on b_compress applied */ int b_asize; /* temporary buffer holder for in-flight compressed data */ void *b_tmp_cdata; }; typedef struct l2arc_data_free { /* protected by l2arc_free_on_write_mtx */ void *l2df_data; size_t l2df_size; void (*l2df_func)(void *, size_t); list_node_t l2df_list_node; } l2arc_data_free_t; static kmutex_t l2arc_feed_thr_lock; static kcondvar_t l2arc_feed_thr_cv; static uint8_t l2arc_thread_exit; static void l2arc_read_done(zio_t *zio); static void l2arc_hdr_stat_add(void); static void l2arc_hdr_stat_remove(void); static boolean_t l2arc_compress_buf(l2arc_buf_hdr_t *l2hdr); static void l2arc_decompress_zio(zio_t *zio, arc_buf_hdr_t *hdr, enum zio_compress c); static void l2arc_release_cdata_buf(arc_buf_hdr_t *ab); static uint64_t buf_hash(uint64_t spa, const dva_t *dva, uint64_t birth) { uint8_t *vdva = (uint8_t *)dva; uint64_t crc = -1ULL; int i; ASSERT(zfs_crc64_table[128] == ZFS_CRC64_POLY); for (i = 0; i < sizeof (dva_t); i++) crc = (crc >> 8) ^ zfs_crc64_table[(crc ^ vdva[i]) & 0xFF]; crc ^= (spa>>8) ^ birth; return (crc); } #define BUF_EMPTY(buf) \ ((buf)->b_dva.dva_word[0] == 0 && \ (buf)->b_dva.dva_word[1] == 0 && \ (buf)->b_cksum0 == 0) #define BUF_EQUAL(spa, dva, birth, buf) \ ((buf)->b_dva.dva_word[0] == (dva)->dva_word[0]) && \ ((buf)->b_dva.dva_word[1] == (dva)->dva_word[1]) && \ ((buf)->b_birth == birth) && ((buf)->b_spa == spa) static void buf_discard_identity(arc_buf_hdr_t *hdr) { hdr->b_dva.dva_word[0] = 0; hdr->b_dva.dva_word[1] = 0; hdr->b_birth = 0; hdr->b_cksum0 = 0; } static arc_buf_hdr_t * buf_hash_find(uint64_t spa, const blkptr_t *bp, kmutex_t **lockp) { const dva_t *dva = BP_IDENTITY(bp); uint64_t birth = BP_PHYSICAL_BIRTH(bp); uint64_t idx = BUF_HASH_INDEX(spa, dva, birth); kmutex_t *hash_lock = BUF_HASH_LOCK(idx); arc_buf_hdr_t *buf; mutex_enter(hash_lock); for (buf = buf_hash_table.ht_table[idx]; buf != NULL; buf = buf->b_hash_next) { if (BUF_EQUAL(spa, dva, birth, buf)) { *lockp = hash_lock; return (buf); } } mutex_exit(hash_lock); *lockp = NULL; return (NULL); } /* * Insert an entry into the hash table. If there is already an element * equal to elem in the hash table, then the already existing element * will be returned and the new element will not be inserted. * Otherwise returns NULL. */ static arc_buf_hdr_t * buf_hash_insert(arc_buf_hdr_t *buf, kmutex_t **lockp) { uint64_t idx = BUF_HASH_INDEX(buf->b_spa, &buf->b_dva, buf->b_birth); kmutex_t *hash_lock = BUF_HASH_LOCK(idx); arc_buf_hdr_t *fbuf; uint32_t i; ASSERT(!DVA_IS_EMPTY(&buf->b_dva)); ASSERT(buf->b_birth != 0); ASSERT(!HDR_IN_HASH_TABLE(buf)); *lockp = hash_lock; mutex_enter(hash_lock); for (fbuf = buf_hash_table.ht_table[idx], i = 0; fbuf != NULL; fbuf = fbuf->b_hash_next, i++) { if (BUF_EQUAL(buf->b_spa, &buf->b_dva, buf->b_birth, fbuf)) return (fbuf); } buf->b_hash_next = buf_hash_table.ht_table[idx]; buf_hash_table.ht_table[idx] = buf; buf->b_flags |= ARC_IN_HASH_TABLE; /* collect some hash table performance data */ if (i > 0) { ARCSTAT_BUMP(arcstat_hash_collisions); if (i == 1) ARCSTAT_BUMP(arcstat_hash_chains); ARCSTAT_MAX(arcstat_hash_chain_max, i); } ARCSTAT_BUMP(arcstat_hash_elements); ARCSTAT_MAXSTAT(arcstat_hash_elements); return (NULL); } static void buf_hash_remove(arc_buf_hdr_t *buf) { arc_buf_hdr_t *fbuf, **bufp; uint64_t idx = BUF_HASH_INDEX(buf->b_spa, &buf->b_dva, buf->b_birth); ASSERT(MUTEX_HELD(BUF_HASH_LOCK(idx))); ASSERT(HDR_IN_HASH_TABLE(buf)); bufp = &buf_hash_table.ht_table[idx]; while ((fbuf = *bufp) != buf) { ASSERT(fbuf != NULL); bufp = &fbuf->b_hash_next; } *bufp = buf->b_hash_next; buf->b_hash_next = NULL; buf->b_flags &= ~ARC_IN_HASH_TABLE; /* collect some hash table performance data */ ARCSTAT_BUMPDOWN(arcstat_hash_elements); if (buf_hash_table.ht_table[idx] && buf_hash_table.ht_table[idx]->b_hash_next == NULL) ARCSTAT_BUMPDOWN(arcstat_hash_chains); } /* * Global data structures and functions for the buf kmem cache. */ static kmem_cache_t *hdr_cache; static kmem_cache_t *buf_cache; static void buf_fini(void) { int i; kmem_free(buf_hash_table.ht_table, (buf_hash_table.ht_mask + 1) * sizeof (void *)); for (i = 0; i < BUF_LOCKS; i++) mutex_destroy(&buf_hash_table.ht_locks[i].ht_lock); kmem_cache_destroy(hdr_cache); kmem_cache_destroy(buf_cache); } /* * Constructor callback - called when the cache is empty * and a new buf is requested. */ /* ARGSUSED */ static int hdr_cons(void *vbuf, void *unused, int kmflag) { arc_buf_hdr_t *buf = vbuf; bzero(buf, sizeof (arc_buf_hdr_t)); refcount_create(&buf->b_refcnt); cv_init(&buf->b_cv, NULL, CV_DEFAULT, NULL); mutex_init(&buf->b_freeze_lock, NULL, MUTEX_DEFAULT, NULL); arc_space_consume(sizeof (arc_buf_hdr_t), ARC_SPACE_HDRS); return (0); } /* ARGSUSED */ static int buf_cons(void *vbuf, void *unused, int kmflag) { arc_buf_t *buf = vbuf; bzero(buf, sizeof (arc_buf_t)); mutex_init(&buf->b_evict_lock, NULL, MUTEX_DEFAULT, NULL); arc_space_consume(sizeof (arc_buf_t), ARC_SPACE_HDRS); return (0); } /* * Destructor callback - called when a cached buf is * no longer required. */ /* ARGSUSED */ static void hdr_dest(void *vbuf, void *unused) { arc_buf_hdr_t *buf = vbuf; ASSERT(BUF_EMPTY(buf)); refcount_destroy(&buf->b_refcnt); cv_destroy(&buf->b_cv); mutex_destroy(&buf->b_freeze_lock); arc_space_return(sizeof (arc_buf_hdr_t), ARC_SPACE_HDRS); } /* ARGSUSED */ static void buf_dest(void *vbuf, void *unused) { arc_buf_t *buf = vbuf; mutex_destroy(&buf->b_evict_lock); arc_space_return(sizeof (arc_buf_t), ARC_SPACE_HDRS); } /* * Reclaim callback -- invoked when memory is low. */ /* ARGSUSED */ static void hdr_recl(void *unused) { dprintf("hdr_recl called\n"); /* * umem calls the reclaim func when we destroy the buf cache, * which is after we do arc_fini(). */ if (!arc_dead) cv_signal(&arc_reclaim_thr_cv); } static void buf_init(void) { uint64_t *ct; uint64_t hsize = 1ULL << 12; int i, j; /* * The hash table is big enough to fill all of physical memory * with an average block size of zfs_arc_average_blocksize (default 8K). * By default, the table will take up * totalmem * sizeof(void*) / 8K (1MB per GB with 8-byte pointers). */ while (hsize * zfs_arc_average_blocksize < (uint64_t)physmem * PAGESIZE) hsize <<= 1; retry: buf_hash_table.ht_mask = hsize - 1; buf_hash_table.ht_table = kmem_zalloc(hsize * sizeof (void*), KM_NOSLEEP); if (buf_hash_table.ht_table == NULL) { ASSERT(hsize > (1ULL << 8)); hsize >>= 1; goto retry; } hdr_cache = kmem_cache_create("arc_buf_hdr_t", sizeof (arc_buf_hdr_t), 0, hdr_cons, hdr_dest, hdr_recl, NULL, NULL, 0); buf_cache = kmem_cache_create("arc_buf_t", sizeof (arc_buf_t), 0, buf_cons, buf_dest, NULL, NULL, NULL, 0); for (i = 0; i < 256; i++) for (ct = zfs_crc64_table + i, *ct = i, j = 8; j > 0; j--) *ct = (*ct >> 1) ^ (-(*ct & 1) & ZFS_CRC64_POLY); for (i = 0; i < BUF_LOCKS; i++) { mutex_init(&buf_hash_table.ht_locks[i].ht_lock, NULL, MUTEX_DEFAULT, NULL); } } #define ARC_MINTIME (hz>>4) /* 62 ms */ static void arc_cksum_verify(arc_buf_t *buf) { zio_cksum_t zc; if (!(zfs_flags & ZFS_DEBUG_MODIFY)) return; mutex_enter(&buf->b_hdr->b_freeze_lock); if (buf->b_hdr->b_freeze_cksum == NULL || (buf->b_hdr->b_flags & ARC_IO_ERROR)) { mutex_exit(&buf->b_hdr->b_freeze_lock); return; } fletcher_2_native(buf->b_data, buf->b_hdr->b_size, &zc); if (!ZIO_CHECKSUM_EQUAL(*buf->b_hdr->b_freeze_cksum, zc)) panic("buffer modified while frozen!"); mutex_exit(&buf->b_hdr->b_freeze_lock); } static int arc_cksum_equal(arc_buf_t *buf) { zio_cksum_t zc; int equal; mutex_enter(&buf->b_hdr->b_freeze_lock); fletcher_2_native(buf->b_data, buf->b_hdr->b_size, &zc); equal = ZIO_CHECKSUM_EQUAL(*buf->b_hdr->b_freeze_cksum, zc); mutex_exit(&buf->b_hdr->b_freeze_lock); return (equal); } static void arc_cksum_compute(arc_buf_t *buf, boolean_t force) { if (!force && !(zfs_flags & ZFS_DEBUG_MODIFY)) return; mutex_enter(&buf->b_hdr->b_freeze_lock); if (buf->b_hdr->b_freeze_cksum != NULL) { mutex_exit(&buf->b_hdr->b_freeze_lock); return; } buf->b_hdr->b_freeze_cksum = kmem_alloc(sizeof (zio_cksum_t), KM_SLEEP); fletcher_2_native(buf->b_data, buf->b_hdr->b_size, buf->b_hdr->b_freeze_cksum); mutex_exit(&buf->b_hdr->b_freeze_lock); #ifdef illumos arc_buf_watch(buf); #endif /* illumos */ } #ifdef illumos #ifndef _KERNEL typedef struct procctl { long cmd; prwatch_t prwatch; } procctl_t; #endif /* ARGSUSED */ static void arc_buf_unwatch(arc_buf_t *buf) { #ifndef _KERNEL if (arc_watch) { int result; procctl_t ctl; ctl.cmd = PCWATCH; ctl.prwatch.pr_vaddr = (uintptr_t)buf->b_data; ctl.prwatch.pr_size = 0; ctl.prwatch.pr_wflags = 0; result = write(arc_procfd, &ctl, sizeof (ctl)); ASSERT3U(result, ==, sizeof (ctl)); } #endif } /* ARGSUSED */ static void arc_buf_watch(arc_buf_t *buf) { #ifndef _KERNEL if (arc_watch) { int result; procctl_t ctl; ctl.cmd = PCWATCH; ctl.prwatch.pr_vaddr = (uintptr_t)buf->b_data; ctl.prwatch.pr_size = buf->b_hdr->b_size; ctl.prwatch.pr_wflags = WA_WRITE; result = write(arc_procfd, &ctl, sizeof (ctl)); ASSERT3U(result, ==, sizeof (ctl)); } #endif } #endif /* illumos */ void arc_buf_thaw(arc_buf_t *buf) { if (zfs_flags & ZFS_DEBUG_MODIFY) { if (buf->b_hdr->b_state != arc_anon) panic("modifying non-anon buffer!"); if (buf->b_hdr->b_flags & ARC_IO_IN_PROGRESS) panic("modifying buffer while i/o in progress!"); arc_cksum_verify(buf); } mutex_enter(&buf->b_hdr->b_freeze_lock); if (buf->b_hdr->b_freeze_cksum != NULL) { kmem_free(buf->b_hdr->b_freeze_cksum, sizeof (zio_cksum_t)); buf->b_hdr->b_freeze_cksum = NULL; } if (zfs_flags & ZFS_DEBUG_MODIFY) { if (buf->b_hdr->b_thawed) kmem_free(buf->b_hdr->b_thawed, 1); buf->b_hdr->b_thawed = kmem_alloc(1, KM_SLEEP); } mutex_exit(&buf->b_hdr->b_freeze_lock); #ifdef illumos arc_buf_unwatch(buf); #endif /* illumos */ } void arc_buf_freeze(arc_buf_t *buf) { kmutex_t *hash_lock; if (!(zfs_flags & ZFS_DEBUG_MODIFY)) return; hash_lock = HDR_LOCK(buf->b_hdr); mutex_enter(hash_lock); ASSERT(buf->b_hdr->b_freeze_cksum != NULL || buf->b_hdr->b_state == arc_anon); arc_cksum_compute(buf, B_FALSE); mutex_exit(hash_lock); } static void get_buf_info(arc_buf_hdr_t *ab, arc_state_t *state, list_t **list, kmutex_t **lock) { uint64_t buf_hashid = buf_hash(ab->b_spa, &ab->b_dva, ab->b_birth); if (ab->b_type == ARC_BUFC_METADATA) buf_hashid &= (ARC_BUFC_NUMMETADATALISTS - 1); else { buf_hashid &= (ARC_BUFC_NUMDATALISTS - 1); buf_hashid += ARC_BUFC_NUMMETADATALISTS; } *list = &state->arcs_lists[buf_hashid]; *lock = ARCS_LOCK(state, buf_hashid); } static void add_reference(arc_buf_hdr_t *ab, kmutex_t *hash_lock, void *tag) { ASSERT(MUTEX_HELD(hash_lock)); if ((refcount_add(&ab->b_refcnt, tag) == 1) && (ab->b_state != arc_anon)) { uint64_t delta = ab->b_size * ab->b_datacnt; uint64_t *size = &ab->b_state->arcs_lsize[ab->b_type]; list_t *list; kmutex_t *lock; get_buf_info(ab, ab->b_state, &list, &lock); ASSERT(!MUTEX_HELD(lock)); mutex_enter(lock); ASSERT(list_link_active(&ab->b_arc_node)); list_remove(list, ab); if (GHOST_STATE(ab->b_state)) { ASSERT0(ab->b_datacnt); ASSERT3P(ab->b_buf, ==, NULL); delta = ab->b_size; } ASSERT(delta > 0); ASSERT3U(*size, >=, delta); atomic_add_64(size, -delta); mutex_exit(lock); /* remove the prefetch flag if we get a reference */ if (ab->b_flags & ARC_PREFETCH) ab->b_flags &= ~ARC_PREFETCH; } } static int remove_reference(arc_buf_hdr_t *ab, kmutex_t *hash_lock, void *tag) { int cnt; arc_state_t *state = ab->b_state; ASSERT(state == arc_anon || MUTEX_HELD(hash_lock)); ASSERT(!GHOST_STATE(state)); if (((cnt = refcount_remove(&ab->b_refcnt, tag)) == 0) && (state != arc_anon)) { uint64_t *size = &state->arcs_lsize[ab->b_type]; list_t *list; kmutex_t *lock; get_buf_info(ab, state, &list, &lock); ASSERT(!MUTEX_HELD(lock)); mutex_enter(lock); ASSERT(!list_link_active(&ab->b_arc_node)); list_insert_head(list, ab); ASSERT(ab->b_datacnt > 0); atomic_add_64(size, ab->b_size * ab->b_datacnt); mutex_exit(lock); } return (cnt); } /* * Move the supplied buffer to the indicated state. The mutex * for the buffer must be held by the caller. */ static void arc_change_state(arc_state_t *new_state, arc_buf_hdr_t *ab, kmutex_t *hash_lock) { arc_state_t *old_state = ab->b_state; int64_t refcnt = refcount_count(&ab->b_refcnt); uint64_t from_delta, to_delta; list_t *list; kmutex_t *lock; ASSERT(MUTEX_HELD(hash_lock)); ASSERT3P(new_state, !=, old_state); ASSERT(refcnt == 0 || ab->b_datacnt > 0); ASSERT(ab->b_datacnt == 0 || !GHOST_STATE(new_state)); ASSERT(ab->b_datacnt <= 1 || old_state != arc_anon); from_delta = to_delta = ab->b_datacnt * ab->b_size; /* * If this buffer is evictable, transfer it from the * old state list to the new state list. */ if (refcnt == 0) { if (old_state != arc_anon) { int use_mutex; uint64_t *size = &old_state->arcs_lsize[ab->b_type]; get_buf_info(ab, old_state, &list, &lock); use_mutex = !MUTEX_HELD(lock); if (use_mutex) mutex_enter(lock); ASSERT(list_link_active(&ab->b_arc_node)); list_remove(list, ab); /* * If prefetching out of the ghost cache, * we will have a non-zero datacnt. */ if (GHOST_STATE(old_state) && ab->b_datacnt == 0) { /* ghost elements have a ghost size */ ASSERT(ab->b_buf == NULL); from_delta = ab->b_size; } ASSERT3U(*size, >=, from_delta); atomic_add_64(size, -from_delta); if (use_mutex) mutex_exit(lock); } if (new_state != arc_anon) { int use_mutex; uint64_t *size = &new_state->arcs_lsize[ab->b_type]; get_buf_info(ab, new_state, &list, &lock); use_mutex = !MUTEX_HELD(lock); if (use_mutex) mutex_enter(lock); list_insert_head(list, ab); /* ghost elements have a ghost size */ if (GHOST_STATE(new_state)) { ASSERT(ab->b_datacnt == 0); ASSERT(ab->b_buf == NULL); to_delta = ab->b_size; } atomic_add_64(size, to_delta); if (use_mutex) mutex_exit(lock); } } ASSERT(!BUF_EMPTY(ab)); if (new_state == arc_anon && HDR_IN_HASH_TABLE(ab)) buf_hash_remove(ab); /* adjust state sizes */ if (to_delta) atomic_add_64(&new_state->arcs_size, to_delta); if (from_delta) { ASSERT3U(old_state->arcs_size, >=, from_delta); atomic_add_64(&old_state->arcs_size, -from_delta); } ab->b_state = new_state; /* adjust l2arc hdr stats */ if (new_state == arc_l2c_only) l2arc_hdr_stat_add(); else if (old_state == arc_l2c_only) l2arc_hdr_stat_remove(); } void arc_space_consume(uint64_t space, arc_space_type_t type) { ASSERT(type >= 0 && type < ARC_SPACE_NUMTYPES); switch (type) { case ARC_SPACE_DATA: ARCSTAT_INCR(arcstat_data_size, space); break; case ARC_SPACE_OTHER: ARCSTAT_INCR(arcstat_other_size, space); break; case ARC_SPACE_HDRS: ARCSTAT_INCR(arcstat_hdr_size, space); break; case ARC_SPACE_L2HDRS: ARCSTAT_INCR(arcstat_l2_hdr_size, space); break; } atomic_add_64(&arc_meta_used, space); atomic_add_64(&arc_size, space); } void arc_space_return(uint64_t space, arc_space_type_t type) { ASSERT(type >= 0 && type < ARC_SPACE_NUMTYPES); switch (type) { case ARC_SPACE_DATA: ARCSTAT_INCR(arcstat_data_size, -space); break; case ARC_SPACE_OTHER: ARCSTAT_INCR(arcstat_other_size, -space); break; case ARC_SPACE_HDRS: ARCSTAT_INCR(arcstat_hdr_size, -space); break; case ARC_SPACE_L2HDRS: ARCSTAT_INCR(arcstat_l2_hdr_size, -space); break; } ASSERT(arc_meta_used >= space); if (arc_meta_max < arc_meta_used) arc_meta_max = arc_meta_used; atomic_add_64(&arc_meta_used, -space); ASSERT(arc_size >= space); atomic_add_64(&arc_size, -space); } void * arc_data_buf_alloc(uint64_t size) { if (arc_evict_needed(ARC_BUFC_DATA)) cv_signal(&arc_reclaim_thr_cv); atomic_add_64(&arc_size, size); return (zio_data_buf_alloc(size)); } void arc_data_buf_free(void *buf, uint64_t size) { zio_data_buf_free(buf, size); ASSERT(arc_size >= size); atomic_add_64(&arc_size, -size); } arc_buf_t * arc_buf_alloc(spa_t *spa, int size, void *tag, arc_buf_contents_t type) { arc_buf_hdr_t *hdr; arc_buf_t *buf; ASSERT3U(size, >, 0); hdr = kmem_cache_alloc(hdr_cache, KM_PUSHPAGE); ASSERT(BUF_EMPTY(hdr)); hdr->b_size = size; hdr->b_type = type; hdr->b_spa = spa_load_guid(spa); hdr->b_state = arc_anon; hdr->b_arc_access = 0; buf = kmem_cache_alloc(buf_cache, KM_PUSHPAGE); buf->b_hdr = hdr; buf->b_data = NULL; buf->b_efunc = NULL; buf->b_private = NULL; buf->b_next = NULL; hdr->b_buf = buf; arc_get_data_buf(buf); hdr->b_datacnt = 1; hdr->b_flags = 0; ASSERT(refcount_is_zero(&hdr->b_refcnt)); (void) refcount_add(&hdr->b_refcnt, tag); return (buf); } static char *arc_onloan_tag = "onloan"; /* * Loan out an anonymous arc buffer. Loaned buffers are not counted as in * flight data by arc_tempreserve_space() until they are "returned". Loaned * buffers must be returned to the arc before they can be used by the DMU or * freed. */ arc_buf_t * arc_loan_buf(spa_t *spa, int size) { arc_buf_t *buf; buf = arc_buf_alloc(spa, size, arc_onloan_tag, ARC_BUFC_DATA); atomic_add_64(&arc_loaned_bytes, size); return (buf); } /* * Return a loaned arc buffer to the arc. */ void arc_return_buf(arc_buf_t *buf, void *tag) { arc_buf_hdr_t *hdr = buf->b_hdr; ASSERT(buf->b_data != NULL); (void) refcount_add(&hdr->b_refcnt, tag); (void) refcount_remove(&hdr->b_refcnt, arc_onloan_tag); atomic_add_64(&arc_loaned_bytes, -hdr->b_size); } /* Detach an arc_buf from a dbuf (tag) */ void arc_loan_inuse_buf(arc_buf_t *buf, void *tag) { arc_buf_hdr_t *hdr; ASSERT(buf->b_data != NULL); hdr = buf->b_hdr; (void) refcount_add(&hdr->b_refcnt, arc_onloan_tag); (void) refcount_remove(&hdr->b_refcnt, tag); buf->b_efunc = NULL; buf->b_private = NULL; atomic_add_64(&arc_loaned_bytes, hdr->b_size); } static arc_buf_t * arc_buf_clone(arc_buf_t *from) { arc_buf_t *buf; arc_buf_hdr_t *hdr = from->b_hdr; uint64_t size = hdr->b_size; ASSERT(hdr->b_state != arc_anon); buf = kmem_cache_alloc(buf_cache, KM_PUSHPAGE); buf->b_hdr = hdr; buf->b_data = NULL; buf->b_efunc = NULL; buf->b_private = NULL; buf->b_next = hdr->b_buf; hdr->b_buf = buf; arc_get_data_buf(buf); bcopy(from->b_data, buf->b_data, size); /* * This buffer already exists in the arc so create a duplicate * copy for the caller. If the buffer is associated with user data * then track the size and number of duplicates. These stats will be * updated as duplicate buffers are created and destroyed. */ if (hdr->b_type == ARC_BUFC_DATA) { ARCSTAT_BUMP(arcstat_duplicate_buffers); ARCSTAT_INCR(arcstat_duplicate_buffers_size, size); } hdr->b_datacnt += 1; return (buf); } void arc_buf_add_ref(arc_buf_t *buf, void* tag) { arc_buf_hdr_t *hdr; kmutex_t *hash_lock; /* * Check to see if this buffer is evicted. Callers * must verify b_data != NULL to know if the add_ref * was successful. */ mutex_enter(&buf->b_evict_lock); if (buf->b_data == NULL) { mutex_exit(&buf->b_evict_lock); return; } hash_lock = HDR_LOCK(buf->b_hdr); mutex_enter(hash_lock); hdr = buf->b_hdr; ASSERT3P(hash_lock, ==, HDR_LOCK(hdr)); mutex_exit(&buf->b_evict_lock); ASSERT(hdr->b_state == arc_mru || hdr->b_state == arc_mfu); add_reference(hdr, hash_lock, tag); DTRACE_PROBE1(arc__hit, arc_buf_hdr_t *, hdr); arc_access(hdr, hash_lock); mutex_exit(hash_lock); ARCSTAT_BUMP(arcstat_hits); ARCSTAT_CONDSTAT(!(hdr->b_flags & ARC_PREFETCH), demand, prefetch, hdr->b_type != ARC_BUFC_METADATA, data, metadata, hits); } +static void +arc_buf_free_on_write(void *data, size_t size, + void (*free_func)(void *, size_t)) +{ + l2arc_data_free_t *df; + + df = kmem_alloc(sizeof (l2arc_data_free_t), KM_SLEEP); + df->l2df_data = data; + df->l2df_size = size; + df->l2df_func = free_func; + mutex_enter(&l2arc_free_on_write_mtx); + list_insert_head(l2arc_free_on_write, df); + mutex_exit(&l2arc_free_on_write_mtx); +} + /* * Free the arc data buffer. If it is an l2arc write in progress, * the buffer is placed on l2arc_free_on_write to be freed later. */ static void arc_buf_data_free(arc_buf_t *buf, void (*free_func)(void *, size_t)) { arc_buf_hdr_t *hdr = buf->b_hdr; if (HDR_L2_WRITING(hdr)) { - l2arc_data_free_t *df; - df = kmem_alloc(sizeof (l2arc_data_free_t), KM_SLEEP); - df->l2df_data = buf->b_data; - df->l2df_size = hdr->b_size; - df->l2df_func = free_func; - mutex_enter(&l2arc_free_on_write_mtx); - list_insert_head(l2arc_free_on_write, df); - mutex_exit(&l2arc_free_on_write_mtx); + arc_buf_free_on_write(buf->b_data, hdr->b_size, free_func); ARCSTAT_BUMP(arcstat_l2_free_on_write); } else { free_func(buf->b_data, hdr->b_size); } } /* * Free up buf->b_data and if 'remove' is set, then pull the * arc_buf_t off of the the arc_buf_hdr_t's list and free it. */ static void +arc_buf_l2_cdata_free(arc_buf_hdr_t *hdr) +{ + l2arc_buf_hdr_t *l2hdr = hdr->b_l2hdr; + + ASSERT(MUTEX_HELD(&l2arc_buflist_mtx)); + + if (l2hdr->b_tmp_cdata == NULL) + return; + + ASSERT(HDR_L2_WRITING(hdr)); + arc_buf_free_on_write(l2hdr->b_tmp_cdata, hdr->b_size, + zio_data_buf_free); + ARCSTAT_BUMP(arcstat_l2_cdata_free_on_write); + l2hdr->b_tmp_cdata = NULL; +} + +static void arc_buf_destroy(arc_buf_t *buf, boolean_t recycle, boolean_t remove) { arc_buf_t **bufp; /* free up data associated with the buf */ if (buf->b_data) { arc_state_t *state = buf->b_hdr->b_state; uint64_t size = buf->b_hdr->b_size; arc_buf_contents_t type = buf->b_hdr->b_type; arc_cksum_verify(buf); #ifdef illumos arc_buf_unwatch(buf); #endif /* illumos */ if (!recycle) { if (type == ARC_BUFC_METADATA) { arc_buf_data_free(buf, zio_buf_free); arc_space_return(size, ARC_SPACE_DATA); } else { ASSERT(type == ARC_BUFC_DATA); arc_buf_data_free(buf, zio_data_buf_free); ARCSTAT_INCR(arcstat_data_size, -size); atomic_add_64(&arc_size, -size); } } if (list_link_active(&buf->b_hdr->b_arc_node)) { uint64_t *cnt = &state->arcs_lsize[type]; ASSERT(refcount_is_zero(&buf->b_hdr->b_refcnt)); ASSERT(state != arc_anon); ASSERT3U(*cnt, >=, size); atomic_add_64(cnt, -size); } ASSERT3U(state->arcs_size, >=, size); atomic_add_64(&state->arcs_size, -size); buf->b_data = NULL; /* * If we're destroying a duplicate buffer make sure * that the appropriate statistics are updated. */ if (buf->b_hdr->b_datacnt > 1 && buf->b_hdr->b_type == ARC_BUFC_DATA) { ARCSTAT_BUMPDOWN(arcstat_duplicate_buffers); ARCSTAT_INCR(arcstat_duplicate_buffers_size, -size); } ASSERT(buf->b_hdr->b_datacnt > 0); buf->b_hdr->b_datacnt -= 1; } /* only remove the buf if requested */ if (!remove) return; /* remove the buf from the hdr list */ for (bufp = &buf->b_hdr->b_buf; *bufp != buf; bufp = &(*bufp)->b_next) continue; *bufp = buf->b_next; buf->b_next = NULL; ASSERT(buf->b_efunc == NULL); /* clean up the buf */ buf->b_hdr = NULL; kmem_cache_free(buf_cache, buf); } static void arc_hdr_destroy(arc_buf_hdr_t *hdr) { ASSERT(refcount_is_zero(&hdr->b_refcnt)); ASSERT3P(hdr->b_state, ==, arc_anon); ASSERT(!HDR_IO_IN_PROGRESS(hdr)); l2arc_buf_hdr_t *l2hdr = hdr->b_l2hdr; if (l2hdr != NULL) { boolean_t buflist_held = MUTEX_HELD(&l2arc_buflist_mtx); /* * To prevent arc_free() and l2arc_evict() from * attempting to free the same buffer at the same time, * a FREE_IN_PROGRESS flag is given to arc_free() to * give it priority. l2arc_evict() can't destroy this * header while we are waiting on l2arc_buflist_mtx. * * The hdr may be removed from l2ad_buflist before we * grab l2arc_buflist_mtx, so b_l2hdr is rechecked. */ if (!buflist_held) { mutex_enter(&l2arc_buflist_mtx); l2hdr = hdr->b_l2hdr; } if (l2hdr != NULL) { trim_map_free(l2hdr->b_dev->l2ad_vdev, l2hdr->b_daddr, hdr->b_size, 0); list_remove(l2hdr->b_dev->l2ad_buflist, hdr); + arc_buf_l2_cdata_free(hdr); ARCSTAT_INCR(arcstat_l2_size, -hdr->b_size); ARCSTAT_INCR(arcstat_l2_asize, -l2hdr->b_asize); vdev_space_update(l2hdr->b_dev->l2ad_vdev, -l2hdr->b_asize, 0, 0); kmem_free(l2hdr, sizeof (l2arc_buf_hdr_t)); if (hdr->b_state == arc_l2c_only) l2arc_hdr_stat_remove(); hdr->b_l2hdr = NULL; } if (!buflist_held) mutex_exit(&l2arc_buflist_mtx); } if (!BUF_EMPTY(hdr)) { ASSERT(!HDR_IN_HASH_TABLE(hdr)); buf_discard_identity(hdr); } while (hdr->b_buf) { arc_buf_t *buf = hdr->b_buf; if (buf->b_efunc) { mutex_enter(&arc_eviction_mtx); mutex_enter(&buf->b_evict_lock); ASSERT(buf->b_hdr != NULL); arc_buf_destroy(hdr->b_buf, FALSE, FALSE); hdr->b_buf = buf->b_next; buf->b_hdr = &arc_eviction_hdr; buf->b_next = arc_eviction_list; arc_eviction_list = buf; mutex_exit(&buf->b_evict_lock); mutex_exit(&arc_eviction_mtx); } else { arc_buf_destroy(hdr->b_buf, FALSE, TRUE); } } if (hdr->b_freeze_cksum != NULL) { kmem_free(hdr->b_freeze_cksum, sizeof (zio_cksum_t)); hdr->b_freeze_cksum = NULL; } if (hdr->b_thawed) { kmem_free(hdr->b_thawed, 1); hdr->b_thawed = NULL; } ASSERT(!list_link_active(&hdr->b_arc_node)); ASSERT3P(hdr->b_hash_next, ==, NULL); ASSERT3P(hdr->b_acb, ==, NULL); kmem_cache_free(hdr_cache, hdr); } void arc_buf_free(arc_buf_t *buf, void *tag) { arc_buf_hdr_t *hdr = buf->b_hdr; int hashed = hdr->b_state != arc_anon; ASSERT(buf->b_efunc == NULL); ASSERT(buf->b_data != NULL); if (hashed) { kmutex_t *hash_lock = HDR_LOCK(hdr); mutex_enter(hash_lock); hdr = buf->b_hdr; ASSERT3P(hash_lock, ==, HDR_LOCK(hdr)); (void) remove_reference(hdr, hash_lock, tag); if (hdr->b_datacnt > 1) { arc_buf_destroy(buf, FALSE, TRUE); } else { ASSERT(buf == hdr->b_buf); ASSERT(buf->b_efunc == NULL); hdr->b_flags |= ARC_BUF_AVAILABLE; } mutex_exit(hash_lock); } else if (HDR_IO_IN_PROGRESS(hdr)) { int destroy_hdr; /* * We are in the middle of an async write. Don't destroy * this buffer unless the write completes before we finish * decrementing the reference count. */ mutex_enter(&arc_eviction_mtx); (void) remove_reference(hdr, NULL, tag); ASSERT(refcount_is_zero(&hdr->b_refcnt)); destroy_hdr = !HDR_IO_IN_PROGRESS(hdr); mutex_exit(&arc_eviction_mtx); if (destroy_hdr) arc_hdr_destroy(hdr); } else { if (remove_reference(hdr, NULL, tag) > 0) arc_buf_destroy(buf, FALSE, TRUE); else arc_hdr_destroy(hdr); } } boolean_t arc_buf_remove_ref(arc_buf_t *buf, void* tag) { arc_buf_hdr_t *hdr = buf->b_hdr; kmutex_t *hash_lock = HDR_LOCK(hdr); boolean_t no_callback = (buf->b_efunc == NULL); if (hdr->b_state == arc_anon) { ASSERT(hdr->b_datacnt == 1); arc_buf_free(buf, tag); return (no_callback); } mutex_enter(hash_lock); hdr = buf->b_hdr; ASSERT3P(hash_lock, ==, HDR_LOCK(hdr)); ASSERT(hdr->b_state != arc_anon); ASSERT(buf->b_data != NULL); (void) remove_reference(hdr, hash_lock, tag); if (hdr->b_datacnt > 1) { if (no_callback) arc_buf_destroy(buf, FALSE, TRUE); } else if (no_callback) { ASSERT(hdr->b_buf == buf && buf->b_next == NULL); ASSERT(buf->b_efunc == NULL); hdr->b_flags |= ARC_BUF_AVAILABLE; } ASSERT(no_callback || hdr->b_datacnt > 1 || refcount_is_zero(&hdr->b_refcnt)); mutex_exit(hash_lock); return (no_callback); } int arc_buf_size(arc_buf_t *buf) { return (buf->b_hdr->b_size); } /* * Called from the DMU to determine if the current buffer should be * evicted. In order to ensure proper locking, the eviction must be initiated * from the DMU. Return true if the buffer is associated with user data and * duplicate buffers still exist. */ boolean_t arc_buf_eviction_needed(arc_buf_t *buf) { arc_buf_hdr_t *hdr; boolean_t evict_needed = B_FALSE; if (zfs_disable_dup_eviction) return (B_FALSE); mutex_enter(&buf->b_evict_lock); hdr = buf->b_hdr; if (hdr == NULL) { /* * We are in arc_do_user_evicts(); let that function * perform the eviction. */ ASSERT(buf->b_data == NULL); mutex_exit(&buf->b_evict_lock); return (B_FALSE); } else if (buf->b_data == NULL) { /* * We have already been added to the arc eviction list; * recommend eviction. */ ASSERT3P(hdr, ==, &arc_eviction_hdr); mutex_exit(&buf->b_evict_lock); return (B_TRUE); } if (hdr->b_datacnt > 1 && hdr->b_type == ARC_BUFC_DATA) evict_needed = B_TRUE; mutex_exit(&buf->b_evict_lock); return (evict_needed); } /* * Evict buffers from list until we've removed the specified number of * bytes. Move the removed buffers to the appropriate evict state. * If the recycle flag is set, then attempt to "recycle" a buffer: * - look for a buffer to evict that is `bytes' long. * - return the data block from this buffer rather than freeing it. * This flag is used by callers that are trying to make space for a * new buffer in a full arc cache. * * This function makes a "best effort". It skips over any buffers * it can't get a hash_lock on, and so may not catch all candidates. * It may also return without evicting as much space as requested. */ static void * arc_evict(arc_state_t *state, uint64_t spa, int64_t bytes, boolean_t recycle, arc_buf_contents_t type) { arc_state_t *evicted_state; uint64_t bytes_evicted = 0, skipped = 0, missed = 0; int64_t bytes_remaining; arc_buf_hdr_t *ab, *ab_prev = NULL; list_t *evicted_list, *list, *evicted_list_start, *list_start; kmutex_t *lock, *evicted_lock; kmutex_t *hash_lock; boolean_t have_lock; void *stolen = NULL; arc_buf_hdr_t marker = { 0 }; int count = 0; static int evict_metadata_offset, evict_data_offset; int i, idx, offset, list_count, lists; ASSERT(state == arc_mru || state == arc_mfu); evicted_state = (state == arc_mru) ? arc_mru_ghost : arc_mfu_ghost; if (type == ARC_BUFC_METADATA) { offset = 0; list_count = ARC_BUFC_NUMMETADATALISTS; list_start = &state->arcs_lists[0]; evicted_list_start = &evicted_state->arcs_lists[0]; idx = evict_metadata_offset; } else { offset = ARC_BUFC_NUMMETADATALISTS; list_start = &state->arcs_lists[offset]; evicted_list_start = &evicted_state->arcs_lists[offset]; list_count = ARC_BUFC_NUMDATALISTS; idx = evict_data_offset; } bytes_remaining = evicted_state->arcs_lsize[type]; lists = 0; evict_start: list = &list_start[idx]; evicted_list = &evicted_list_start[idx]; lock = ARCS_LOCK(state, (offset + idx)); evicted_lock = ARCS_LOCK(evicted_state, (offset + idx)); mutex_enter(lock); mutex_enter(evicted_lock); for (ab = list_tail(list); ab; ab = ab_prev) { ab_prev = list_prev(list, ab); bytes_remaining -= (ab->b_size * ab->b_datacnt); /* prefetch buffers have a minimum lifespan */ if (HDR_IO_IN_PROGRESS(ab) || (spa && ab->b_spa != spa) || (ab->b_flags & (ARC_PREFETCH|ARC_INDIRECT) && ddi_get_lbolt() - ab->b_arc_access < arc_min_prefetch_lifespan)) { skipped++; continue; } /* "lookahead" for better eviction candidate */ if (recycle && ab->b_size != bytes && ab_prev && ab_prev->b_size == bytes) continue; /* ignore markers */ if (ab->b_spa == 0) continue; /* * It may take a long time to evict all the bufs requested. * To avoid blocking all arc activity, periodically drop * the arcs_mtx and give other threads a chance to run * before reacquiring the lock. * * If we are looking for a buffer to recycle, we are in * the hot code path, so don't sleep. */ if (!recycle && count++ > arc_evict_iterations) { list_insert_after(list, ab, &marker); mutex_exit(evicted_lock); mutex_exit(lock); kpreempt(KPREEMPT_SYNC); mutex_enter(lock); mutex_enter(evicted_lock); ab_prev = list_prev(list, &marker); list_remove(list, &marker); count = 0; continue; } hash_lock = HDR_LOCK(ab); have_lock = MUTEX_HELD(hash_lock); if (have_lock || mutex_tryenter(hash_lock)) { ASSERT0(refcount_count(&ab->b_refcnt)); ASSERT(ab->b_datacnt > 0); while (ab->b_buf) { arc_buf_t *buf = ab->b_buf; if (!mutex_tryenter(&buf->b_evict_lock)) { missed += 1; break; } if (buf->b_data) { bytes_evicted += ab->b_size; if (recycle && ab->b_type == type && ab->b_size == bytes && !HDR_L2_WRITING(ab)) { stolen = buf->b_data; recycle = FALSE; } } if (buf->b_efunc) { mutex_enter(&arc_eviction_mtx); arc_buf_destroy(buf, buf->b_data == stolen, FALSE); ab->b_buf = buf->b_next; buf->b_hdr = &arc_eviction_hdr; buf->b_next = arc_eviction_list; arc_eviction_list = buf; mutex_exit(&arc_eviction_mtx); mutex_exit(&buf->b_evict_lock); } else { mutex_exit(&buf->b_evict_lock); arc_buf_destroy(buf, buf->b_data == stolen, TRUE); } } if (ab->b_l2hdr) { ARCSTAT_INCR(arcstat_evict_l2_cached, ab->b_size); } else { if (l2arc_write_eligible(ab->b_spa, ab)) { ARCSTAT_INCR(arcstat_evict_l2_eligible, ab->b_size); } else { ARCSTAT_INCR( arcstat_evict_l2_ineligible, ab->b_size); } } if (ab->b_datacnt == 0) { arc_change_state(evicted_state, ab, hash_lock); ASSERT(HDR_IN_HASH_TABLE(ab)); ab->b_flags |= ARC_IN_HASH_TABLE; ab->b_flags &= ~ARC_BUF_AVAILABLE; DTRACE_PROBE1(arc__evict, arc_buf_hdr_t *, ab); } if (!have_lock) mutex_exit(hash_lock); if (bytes >= 0 && bytes_evicted >= bytes) break; if (bytes_remaining > 0) { mutex_exit(evicted_lock); mutex_exit(lock); idx = ((idx + 1) & (list_count - 1)); lists++; goto evict_start; } } else { missed += 1; } } mutex_exit(evicted_lock); mutex_exit(lock); idx = ((idx + 1) & (list_count - 1)); lists++; if (bytes_evicted < bytes) { if (lists < list_count) goto evict_start; else dprintf("only evicted %lld bytes from %x", (longlong_t)bytes_evicted, state); } if (type == ARC_BUFC_METADATA) evict_metadata_offset = idx; else evict_data_offset = idx; if (skipped) ARCSTAT_INCR(arcstat_evict_skip, skipped); if (missed) ARCSTAT_INCR(arcstat_mutex_miss, missed); /* * Note: we have just evicted some data into the ghost state, * potentially putting the ghost size over the desired size. Rather * that evicting from the ghost list in this hot code path, leave * this chore to the arc_reclaim_thread(). */ if (stolen) ARCSTAT_BUMP(arcstat_stolen); return (stolen); } /* * Remove buffers from list until we've removed the specified number of * bytes. Destroy the buffers that are removed. */ static void arc_evict_ghost(arc_state_t *state, uint64_t spa, int64_t bytes) { arc_buf_hdr_t *ab, *ab_prev; arc_buf_hdr_t marker = { 0 }; list_t *list, *list_start; kmutex_t *hash_lock, *lock; uint64_t bytes_deleted = 0; uint64_t bufs_skipped = 0; int count = 0; static int evict_offset; int list_count, idx = evict_offset; int offset, lists = 0; ASSERT(GHOST_STATE(state)); /* * data lists come after metadata lists */ list_start = &state->arcs_lists[ARC_BUFC_NUMMETADATALISTS]; list_count = ARC_BUFC_NUMDATALISTS; offset = ARC_BUFC_NUMMETADATALISTS; evict_start: list = &list_start[idx]; lock = ARCS_LOCK(state, idx + offset); mutex_enter(lock); for (ab = list_tail(list); ab; ab = ab_prev) { ab_prev = list_prev(list, ab); if (ab->b_type > ARC_BUFC_NUMTYPES) panic("invalid ab=%p", (void *)ab); if (spa && ab->b_spa != spa) continue; /* ignore markers */ if (ab->b_spa == 0) continue; hash_lock = HDR_LOCK(ab); /* caller may be trying to modify this buffer, skip it */ if (MUTEX_HELD(hash_lock)) continue; /* * It may take a long time to evict all the bufs requested. * To avoid blocking all arc activity, periodically drop * the arcs_mtx and give other threads a chance to run * before reacquiring the lock. */ if (count++ > arc_evict_iterations) { list_insert_after(list, ab, &marker); mutex_exit(lock); kpreempt(KPREEMPT_SYNC); mutex_enter(lock); ab_prev = list_prev(list, &marker); list_remove(list, &marker); count = 0; continue; } if (mutex_tryenter(hash_lock)) { ASSERT(!HDR_IO_IN_PROGRESS(ab)); ASSERT(ab->b_buf == NULL); ARCSTAT_BUMP(arcstat_deleted); bytes_deleted += ab->b_size; if (ab->b_l2hdr != NULL) { /* * This buffer is cached on the 2nd Level ARC; * don't destroy the header. */ arc_change_state(arc_l2c_only, ab, hash_lock); mutex_exit(hash_lock); } else { arc_change_state(arc_anon, ab, hash_lock); mutex_exit(hash_lock); arc_hdr_destroy(ab); } DTRACE_PROBE1(arc__delete, arc_buf_hdr_t *, ab); if (bytes >= 0 && bytes_deleted >= bytes) break; } else if (bytes < 0) { /* * Insert a list marker and then wait for the * hash lock to become available. Once its * available, restart from where we left off. */ list_insert_after(list, ab, &marker); mutex_exit(lock); mutex_enter(hash_lock); mutex_exit(hash_lock); mutex_enter(lock); ab_prev = list_prev(list, &marker); list_remove(list, &marker); } else { bufs_skipped += 1; } } mutex_exit(lock); idx = ((idx + 1) & (ARC_BUFC_NUMDATALISTS - 1)); lists++; if (lists < list_count) goto evict_start; evict_offset = idx; if ((uintptr_t)list > (uintptr_t)&state->arcs_lists[ARC_BUFC_NUMMETADATALISTS] && (bytes < 0 || bytes_deleted < bytes)) { list_start = &state->arcs_lists[0]; list_count = ARC_BUFC_NUMMETADATALISTS; offset = lists = 0; goto evict_start; } if (bufs_skipped) { ARCSTAT_INCR(arcstat_mutex_miss, bufs_skipped); ASSERT(bytes >= 0); } if (bytes_deleted < bytes) dprintf("only deleted %lld bytes from %p", (longlong_t)bytes_deleted, state); } static void arc_adjust(void) { int64_t adjustment, delta; /* * Adjust MRU size */ adjustment = MIN((int64_t)(arc_size - arc_c), (int64_t)(arc_anon->arcs_size + arc_mru->arcs_size + arc_meta_used - arc_p)); if (adjustment > 0 && arc_mru->arcs_lsize[ARC_BUFC_DATA] > 0) { delta = MIN(arc_mru->arcs_lsize[ARC_BUFC_DATA], adjustment); (void) arc_evict(arc_mru, 0, delta, FALSE, ARC_BUFC_DATA); adjustment -= delta; } if (adjustment > 0 && arc_mru->arcs_lsize[ARC_BUFC_METADATA] > 0) { delta = MIN(arc_mru->arcs_lsize[ARC_BUFC_METADATA], adjustment); (void) arc_evict(arc_mru, 0, delta, FALSE, ARC_BUFC_METADATA); } /* * Adjust MFU size */ adjustment = arc_size - arc_c; if (adjustment > 0 && arc_mfu->arcs_lsize[ARC_BUFC_DATA] > 0) { delta = MIN(adjustment, arc_mfu->arcs_lsize[ARC_BUFC_DATA]); (void) arc_evict(arc_mfu, 0, delta, FALSE, ARC_BUFC_DATA); adjustment -= delta; } if (adjustment > 0 && arc_mfu->arcs_lsize[ARC_BUFC_METADATA] > 0) { int64_t delta = MIN(adjustment, arc_mfu->arcs_lsize[ARC_BUFC_METADATA]); (void) arc_evict(arc_mfu, 0, delta, FALSE, ARC_BUFC_METADATA); } /* * Adjust ghost lists */ adjustment = arc_mru->arcs_size + arc_mru_ghost->arcs_size - arc_c; if (adjustment > 0 && arc_mru_ghost->arcs_size > 0) { delta = MIN(arc_mru_ghost->arcs_size, adjustment); arc_evict_ghost(arc_mru_ghost, 0, delta); } adjustment = arc_mru_ghost->arcs_size + arc_mfu_ghost->arcs_size - arc_c; if (adjustment > 0 && arc_mfu_ghost->arcs_size > 0) { delta = MIN(arc_mfu_ghost->arcs_size, adjustment); arc_evict_ghost(arc_mfu_ghost, 0, delta); } } static void arc_do_user_evicts(void) { static arc_buf_t *tmp_arc_eviction_list; /* * Move list over to avoid LOR */ restart: mutex_enter(&arc_eviction_mtx); tmp_arc_eviction_list = arc_eviction_list; arc_eviction_list = NULL; mutex_exit(&arc_eviction_mtx); while (tmp_arc_eviction_list != NULL) { arc_buf_t *buf = tmp_arc_eviction_list; tmp_arc_eviction_list = buf->b_next; mutex_enter(&buf->b_evict_lock); buf->b_hdr = NULL; mutex_exit(&buf->b_evict_lock); if (buf->b_efunc != NULL) VERIFY0(buf->b_efunc(buf->b_private)); buf->b_efunc = NULL; buf->b_private = NULL; kmem_cache_free(buf_cache, buf); } if (arc_eviction_list != NULL) goto restart; } /* * Flush all *evictable* data from the cache for the given spa. * NOTE: this will not touch "active" (i.e. referenced) data. */ void arc_flush(spa_t *spa) { uint64_t guid = 0; if (spa) guid = spa_load_guid(spa); while (arc_mru->arcs_lsize[ARC_BUFC_DATA]) { (void) arc_evict(arc_mru, guid, -1, FALSE, ARC_BUFC_DATA); if (spa) break; } while (arc_mru->arcs_lsize[ARC_BUFC_METADATA]) { (void) arc_evict(arc_mru, guid, -1, FALSE, ARC_BUFC_METADATA); if (spa) break; } while (arc_mfu->arcs_lsize[ARC_BUFC_DATA]) { (void) arc_evict(arc_mfu, guid, -1, FALSE, ARC_BUFC_DATA); if (spa) break; } while (arc_mfu->arcs_lsize[ARC_BUFC_METADATA]) { (void) arc_evict(arc_mfu, guid, -1, FALSE, ARC_BUFC_METADATA); if (spa) break; } arc_evict_ghost(arc_mru_ghost, guid, -1); arc_evict_ghost(arc_mfu_ghost, guid, -1); mutex_enter(&arc_reclaim_thr_lock); arc_do_user_evicts(); mutex_exit(&arc_reclaim_thr_lock); ASSERT(spa || arc_eviction_list == NULL); } void arc_shrink(void) { if (arc_c > arc_c_min) { uint64_t to_free; #ifdef _KERNEL to_free = arc_c >> arc_shrink_shift; #else to_free = arc_c >> arc_shrink_shift; #endif if (arc_c > arc_c_min + to_free) atomic_add_64(&arc_c, -to_free); else arc_c = arc_c_min; atomic_add_64(&arc_p, -(arc_p >> arc_shrink_shift)); if (arc_c > arc_size) arc_c = MAX(arc_size, arc_c_min); if (arc_p > arc_c) arc_p = (arc_c >> 1); ASSERT(arc_c >= arc_c_min); ASSERT((int64_t)arc_p >= 0); } if (arc_size > arc_c) arc_adjust(); } static int needfree = 0; static int arc_reclaim_needed(void) { #ifdef _KERNEL if (needfree) return (1); /* * Cooperate with pagedaemon when it's time for it to scan * and reclaim some pages. */ if (vm_paging_needed()) return (1); #ifdef sun /* * take 'desfree' extra pages, so we reclaim sooner, rather than later */ extra = desfree; /* * check that we're out of range of the pageout scanner. It starts to * schedule paging if freemem is less than lotsfree and needfree. * lotsfree is the high-water mark for pageout, and needfree is the * number of needed free pages. We add extra pages here to make sure * the scanner doesn't start up while we're freeing memory. */ if (freemem < lotsfree + needfree + extra) return (1); /* * check to make sure that swapfs has enough space so that anon * reservations can still succeed. anon_resvmem() checks that the * availrmem is greater than swapfs_minfree, and the number of reserved * swap pages. We also add a bit of extra here just to prevent * circumstances from getting really dire. */ if (availrmem < swapfs_minfree + swapfs_reserve + extra) return (1); #if defined(__i386) /* * If we're on an i386 platform, it's possible that we'll exhaust the * kernel heap space before we ever run out of available physical * memory. Most checks of the size of the heap_area compare against * tune.t_minarmem, which is the minimum available real memory that we * can have in the system. However, this is generally fixed at 25 pages * which is so low that it's useless. In this comparison, we seek to * calculate the total heap-size, and reclaim if more than 3/4ths of the * heap is allocated. (Or, in the calculation, if less than 1/4th is * free) */ if (btop(vmem_size(heap_arena, VMEM_FREE)) < (btop(vmem_size(heap_arena, VMEM_FREE | VMEM_ALLOC)) >> 2)) return (1); #endif #else /* !sun */ if (kmem_used() > (kmem_size() * 3) / 4) return (1); #endif /* sun */ #else if (spa_get_random(100) == 0) return (1); #endif return (0); } extern kmem_cache_t *zio_buf_cache[]; extern kmem_cache_t *zio_data_buf_cache[]; static void arc_kmem_reap_now(arc_reclaim_strategy_t strat) { size_t i; kmem_cache_t *prev_cache = NULL; kmem_cache_t *prev_data_cache = NULL; #ifdef _KERNEL if (arc_meta_used >= arc_meta_limit) { /* * We are exceeding our meta-data cache limit. * Purge some DNLC entries to release holds on meta-data. */ dnlc_reduce_cache((void *)(uintptr_t)arc_reduce_dnlc_percent); } #if defined(__i386) /* * Reclaim unused memory from all kmem caches. */ kmem_reap(); #endif #endif /* * An aggressive reclamation will shrink the cache size as well as * reap free buffers from the arc kmem caches. */ if (strat == ARC_RECLAIM_AGGR) arc_shrink(); for (i = 0; i < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT; i++) { if (zio_buf_cache[i] != prev_cache) { prev_cache = zio_buf_cache[i]; kmem_cache_reap_now(zio_buf_cache[i]); } if (zio_data_buf_cache[i] != prev_data_cache) { prev_data_cache = zio_data_buf_cache[i]; kmem_cache_reap_now(zio_data_buf_cache[i]); } } kmem_cache_reap_now(buf_cache); kmem_cache_reap_now(hdr_cache); } static void arc_reclaim_thread(void *dummy __unused) { clock_t growtime = 0; arc_reclaim_strategy_t last_reclaim = ARC_RECLAIM_CONS; callb_cpr_t cpr; CALLB_CPR_INIT(&cpr, &arc_reclaim_thr_lock, callb_generic_cpr, FTAG); mutex_enter(&arc_reclaim_thr_lock); while (arc_thread_exit == 0) { if (arc_reclaim_needed()) { if (arc_no_grow) { if (last_reclaim == ARC_RECLAIM_CONS) { last_reclaim = ARC_RECLAIM_AGGR; } else { last_reclaim = ARC_RECLAIM_CONS; } } else { arc_no_grow = TRUE; last_reclaim = ARC_RECLAIM_AGGR; membar_producer(); } /* reset the growth delay for every reclaim */ growtime = ddi_get_lbolt() + (arc_grow_retry * hz); if (needfree && last_reclaim == ARC_RECLAIM_CONS) { /* * If needfree is TRUE our vm_lowmem hook * was called and in that case we must free some * memory, so switch to aggressive mode. */ arc_no_grow = TRUE; last_reclaim = ARC_RECLAIM_AGGR; } arc_kmem_reap_now(last_reclaim); arc_warm = B_TRUE; } else if (arc_no_grow && ddi_get_lbolt() >= growtime) { arc_no_grow = FALSE; } arc_adjust(); if (arc_eviction_list != NULL) arc_do_user_evicts(); #ifdef _KERNEL if (needfree) { needfree = 0; wakeup(&needfree); } #endif /* block until needed, or one second, whichever is shorter */ CALLB_CPR_SAFE_BEGIN(&cpr); (void) cv_timedwait(&arc_reclaim_thr_cv, &arc_reclaim_thr_lock, hz); CALLB_CPR_SAFE_END(&cpr, &arc_reclaim_thr_lock); } arc_thread_exit = 0; cv_broadcast(&arc_reclaim_thr_cv); CALLB_CPR_EXIT(&cpr); /* drops arc_reclaim_thr_lock */ thread_exit(); } /* * Adapt arc info given the number of bytes we are trying to add and * the state that we are comming from. This function is only called * when we are adding new content to the cache. */ static void arc_adapt(int bytes, arc_state_t *state) { int mult; uint64_t arc_p_min = (arc_c >> arc_p_min_shift); if (state == arc_l2c_only) return; ASSERT(bytes > 0); /* * Adapt the target size of the MRU list: * - if we just hit in the MRU ghost list, then increase * the target size of the MRU list. * - if we just hit in the MFU ghost list, then increase * the target size of the MFU list by decreasing the * target size of the MRU list. */ if (state == arc_mru_ghost) { mult = ((arc_mru_ghost->arcs_size >= arc_mfu_ghost->arcs_size) ? 1 : (arc_mfu_ghost->arcs_size/arc_mru_ghost->arcs_size)); mult = MIN(mult, 10); /* avoid wild arc_p adjustment */ arc_p = MIN(arc_c - arc_p_min, arc_p + bytes * mult); } else if (state == arc_mfu_ghost) { uint64_t delta; mult = ((arc_mfu_ghost->arcs_size >= arc_mru_ghost->arcs_size) ? 1 : (arc_mru_ghost->arcs_size/arc_mfu_ghost->arcs_size)); mult = MIN(mult, 10); delta = MIN(bytes * mult, arc_p); arc_p = MAX(arc_p_min, arc_p - delta); } ASSERT((int64_t)arc_p >= 0); if (arc_reclaim_needed()) { cv_signal(&arc_reclaim_thr_cv); return; } if (arc_no_grow) return; if (arc_c >= arc_c_max) return; /* * If we're within (2 * maxblocksize) bytes of the target * cache size, increment the target cache size */ if (arc_size > arc_c - (2ULL << SPA_MAXBLOCKSHIFT)) { atomic_add_64(&arc_c, (int64_t)bytes); if (arc_c > arc_c_max) arc_c = arc_c_max; else if (state == arc_anon) atomic_add_64(&arc_p, (int64_t)bytes); if (arc_p > arc_c) arc_p = arc_c; } ASSERT((int64_t)arc_p >= 0); } /* * Check if the cache has reached its limits and eviction is required * prior to insert. */ static int arc_evict_needed(arc_buf_contents_t type) { if (type == ARC_BUFC_METADATA && arc_meta_used >= arc_meta_limit) return (1); #ifdef sun #ifdef _KERNEL /* * If zio data pages are being allocated out of a separate heap segment, * then enforce that the size of available vmem for this area remains * above about 1/32nd free. */ if (type == ARC_BUFC_DATA && zio_arena != NULL && vmem_size(zio_arena, VMEM_FREE) < (vmem_size(zio_arena, VMEM_ALLOC) >> 5)) return (1); #endif #endif /* sun */ if (arc_reclaim_needed()) return (1); return (arc_size > arc_c); } /* * The buffer, supplied as the first argument, needs a data block. * So, if we are at cache max, determine which cache should be victimized. * We have the following cases: * * 1. Insert for MRU, p > sizeof(arc_anon + arc_mru) -> * In this situation if we're out of space, but the resident size of the MFU is * under the limit, victimize the MFU cache to satisfy this insertion request. * * 2. Insert for MRU, p <= sizeof(arc_anon + arc_mru) -> * Here, we've used up all of the available space for the MRU, so we need to * evict from our own cache instead. Evict from the set of resident MRU * entries. * * 3. Insert for MFU (c - p) > sizeof(arc_mfu) -> * c minus p represents the MFU space in the cache, since p is the size of the * cache that is dedicated to the MRU. In this situation there's still space on * the MFU side, so the MRU side needs to be victimized. * * 4. Insert for MFU (c - p) < sizeof(arc_mfu) -> * MFU's resident set is consuming more space than it has been allotted. In * this situation, we must victimize our own cache, the MFU, for this insertion. */ static void arc_get_data_buf(arc_buf_t *buf) { arc_state_t *state = buf->b_hdr->b_state; uint64_t size = buf->b_hdr->b_size; arc_buf_contents_t type = buf->b_hdr->b_type; arc_adapt(size, state); /* * We have not yet reached cache maximum size, * just allocate a new buffer. */ if (!arc_evict_needed(type)) { if (type == ARC_BUFC_METADATA) { buf->b_data = zio_buf_alloc(size); arc_space_consume(size, ARC_SPACE_DATA); } else { ASSERT(type == ARC_BUFC_DATA); buf->b_data = zio_data_buf_alloc(size); ARCSTAT_INCR(arcstat_data_size, size); atomic_add_64(&arc_size, size); } goto out; } /* * If we are prefetching from the mfu ghost list, this buffer * will end up on the mru list; so steal space from there. */ if (state == arc_mfu_ghost) state = buf->b_hdr->b_flags & ARC_PREFETCH ? arc_mru : arc_mfu; else if (state == arc_mru_ghost) state = arc_mru; if (state == arc_mru || state == arc_anon) { uint64_t mru_used = arc_anon->arcs_size + arc_mru->arcs_size; state = (arc_mfu->arcs_lsize[type] >= size && arc_p > mru_used) ? arc_mfu : arc_mru; } else { /* MFU cases */ uint64_t mfu_space = arc_c - arc_p; state = (arc_mru->arcs_lsize[type] >= size && mfu_space > arc_mfu->arcs_size) ? arc_mru : arc_mfu; } if ((buf->b_data = arc_evict(state, 0, size, TRUE, type)) == NULL) { if (type == ARC_BUFC_METADATA) { buf->b_data = zio_buf_alloc(size); arc_space_consume(size, ARC_SPACE_DATA); } else { ASSERT(type == ARC_BUFC_DATA); buf->b_data = zio_data_buf_alloc(size); ARCSTAT_INCR(arcstat_data_size, size); atomic_add_64(&arc_size, size); } ARCSTAT_BUMP(arcstat_recycle_miss); } ASSERT(buf->b_data != NULL); out: /* * Update the state size. Note that ghost states have a * "ghost size" and so don't need to be updated. */ if (!GHOST_STATE(buf->b_hdr->b_state)) { arc_buf_hdr_t *hdr = buf->b_hdr; atomic_add_64(&hdr->b_state->arcs_size, size); if (list_link_active(&hdr->b_arc_node)) { ASSERT(refcount_is_zero(&hdr->b_refcnt)); atomic_add_64(&hdr->b_state->arcs_lsize[type], size); } /* * If we are growing the cache, and we are adding anonymous * data, and we have outgrown arc_p, update arc_p */ if (arc_size < arc_c && hdr->b_state == arc_anon && arc_anon->arcs_size + arc_mru->arcs_size > arc_p) arc_p = MIN(arc_c, arc_p + size); } ARCSTAT_BUMP(arcstat_allocated); } /* * This routine is called whenever a buffer is accessed. * NOTE: the hash lock is dropped in this function. */ static void arc_access(arc_buf_hdr_t *buf, kmutex_t *hash_lock) { clock_t now; ASSERT(MUTEX_HELD(hash_lock)); if (buf->b_state == arc_anon) { /* * This buffer is not in the cache, and does not * appear in our "ghost" list. Add the new buffer * to the MRU state. */ ASSERT(buf->b_arc_access == 0); buf->b_arc_access = ddi_get_lbolt(); DTRACE_PROBE1(new_state__mru, arc_buf_hdr_t *, buf); arc_change_state(arc_mru, buf, hash_lock); } else if (buf->b_state == arc_mru) { now = ddi_get_lbolt(); /* * If this buffer is here because of a prefetch, then either: * - clear the flag if this is a "referencing" read * (any subsequent access will bump this into the MFU state). * or * - move the buffer to the head of the list if this is * another prefetch (to make it less likely to be evicted). */ if ((buf->b_flags & ARC_PREFETCH) != 0) { if (refcount_count(&buf->b_refcnt) == 0) { ASSERT(list_link_active(&buf->b_arc_node)); } else { buf->b_flags &= ~ARC_PREFETCH; ARCSTAT_BUMP(arcstat_mru_hits); } buf->b_arc_access = now; return; } /* * This buffer has been "accessed" only once so far, * but it is still in the cache. Move it to the MFU * state. */ if (now > buf->b_arc_access + ARC_MINTIME) { /* * More than 125ms have passed since we * instantiated this buffer. Move it to the * most frequently used state. */ buf->b_arc_access = now; DTRACE_PROBE1(new_state__mfu, arc_buf_hdr_t *, buf); arc_change_state(arc_mfu, buf, hash_lock); } ARCSTAT_BUMP(arcstat_mru_hits); } else if (buf->b_state == arc_mru_ghost) { arc_state_t *new_state; /* * This buffer has been "accessed" recently, but * was evicted from the cache. Move it to the * MFU state. */ if (buf->b_flags & ARC_PREFETCH) { new_state = arc_mru; if (refcount_count(&buf->b_refcnt) > 0) buf->b_flags &= ~ARC_PREFETCH; DTRACE_PROBE1(new_state__mru, arc_buf_hdr_t *, buf); } else { new_state = arc_mfu; DTRACE_PROBE1(new_state__mfu, arc_buf_hdr_t *, buf); } buf->b_arc_access = ddi_get_lbolt(); arc_change_state(new_state, buf, hash_lock); ARCSTAT_BUMP(arcstat_mru_ghost_hits); } else if (buf->b_state == arc_mfu) { /* * This buffer has been accessed more than once and is * still in the cache. Keep it in the MFU state. * * NOTE: an add_reference() that occurred when we did * the arc_read() will have kicked this off the list. * If it was a prefetch, we will explicitly move it to * the head of the list now. */ if ((buf->b_flags & ARC_PREFETCH) != 0) { ASSERT(refcount_count(&buf->b_refcnt) == 0); ASSERT(list_link_active(&buf->b_arc_node)); } ARCSTAT_BUMP(arcstat_mfu_hits); buf->b_arc_access = ddi_get_lbolt(); } else if (buf->b_state == arc_mfu_ghost) { arc_state_t *new_state = arc_mfu; /* * This buffer has been accessed more than once but has * been evicted from the cache. Move it back to the * MFU state. */ if (buf->b_flags & ARC_PREFETCH) { /* * This is a prefetch access... * move this block back to the MRU state. */ ASSERT0(refcount_count(&buf->b_refcnt)); new_state = arc_mru; } buf->b_arc_access = ddi_get_lbolt(); DTRACE_PROBE1(new_state__mfu, arc_buf_hdr_t *, buf); arc_change_state(new_state, buf, hash_lock); ARCSTAT_BUMP(arcstat_mfu_ghost_hits); } else if (buf->b_state == arc_l2c_only) { /* * This buffer is on the 2nd Level ARC. */ buf->b_arc_access = ddi_get_lbolt(); DTRACE_PROBE1(new_state__mfu, arc_buf_hdr_t *, buf); arc_change_state(arc_mfu, buf, hash_lock); } else { ASSERT(!"invalid arc state"); } } /* a generic arc_done_func_t which you can use */ /* ARGSUSED */ void arc_bcopy_func(zio_t *zio, arc_buf_t *buf, void *arg) { if (zio == NULL || zio->io_error == 0) bcopy(buf->b_data, arg, buf->b_hdr->b_size); VERIFY(arc_buf_remove_ref(buf, arg)); } /* a generic arc_done_func_t */ void arc_getbuf_func(zio_t *zio, arc_buf_t *buf, void *arg) { arc_buf_t **bufp = arg; if (zio && zio->io_error) { VERIFY(arc_buf_remove_ref(buf, arg)); *bufp = NULL; } else { *bufp = buf; ASSERT(buf->b_data); } } static void arc_read_done(zio_t *zio) { arc_buf_hdr_t *hdr; arc_buf_t *buf; arc_buf_t *abuf; /* buffer we're assigning to callback */ kmutex_t *hash_lock = NULL; arc_callback_t *callback_list, *acb; int freeable = FALSE; buf = zio->io_private; hdr = buf->b_hdr; /* * The hdr was inserted into hash-table and removed from lists * prior to starting I/O. We should find this header, since * it's in the hash table, and it should be legit since it's * not possible to evict it during the I/O. The only possible * reason for it not to be found is if we were freed during the * read. */ if (HDR_IN_HASH_TABLE(hdr)) { ASSERT3U(hdr->b_birth, ==, BP_PHYSICAL_BIRTH(zio->io_bp)); ASSERT3U(hdr->b_dva.dva_word[0], ==, BP_IDENTITY(zio->io_bp)->dva_word[0]); ASSERT3U(hdr->b_dva.dva_word[1], ==, BP_IDENTITY(zio->io_bp)->dva_word[1]); arc_buf_hdr_t *found = buf_hash_find(hdr->b_spa, zio->io_bp, &hash_lock); ASSERT((found == NULL && HDR_FREED_IN_READ(hdr) && hash_lock == NULL) || (found == hdr && DVA_EQUAL(&hdr->b_dva, BP_IDENTITY(zio->io_bp))) || (found == hdr && HDR_L2_READING(hdr))); } hdr->b_flags &= ~ARC_L2_EVICTED; if (l2arc_noprefetch && (hdr->b_flags & ARC_PREFETCH)) hdr->b_flags &= ~ARC_L2CACHE; /* byteswap if necessary */ callback_list = hdr->b_acb; ASSERT(callback_list != NULL); if (BP_SHOULD_BYTESWAP(zio->io_bp) && zio->io_error == 0) { dmu_object_byteswap_t bswap = DMU_OT_BYTESWAP(BP_GET_TYPE(zio->io_bp)); arc_byteswap_func_t *func = BP_GET_LEVEL(zio->io_bp) > 0 ? byteswap_uint64_array : dmu_ot_byteswap[bswap].ob_func; func(buf->b_data, hdr->b_size); } arc_cksum_compute(buf, B_FALSE); #ifdef illumos arc_buf_watch(buf); #endif /* illumos */ if (hash_lock && zio->io_error == 0 && hdr->b_state == arc_anon) { /* * Only call arc_access on anonymous buffers. This is because * if we've issued an I/O for an evicted buffer, we've already * called arc_access (to prevent any simultaneous readers from * getting confused). */ arc_access(hdr, hash_lock); } /* create copies of the data buffer for the callers */ abuf = buf; for (acb = callback_list; acb; acb = acb->acb_next) { if (acb->acb_done) { if (abuf == NULL) { ARCSTAT_BUMP(arcstat_duplicate_reads); abuf = arc_buf_clone(buf); } acb->acb_buf = abuf; abuf = NULL; } } hdr->b_acb = NULL; hdr->b_flags &= ~ARC_IO_IN_PROGRESS; ASSERT(!HDR_BUF_AVAILABLE(hdr)); if (abuf == buf) { ASSERT(buf->b_efunc == NULL); ASSERT(hdr->b_datacnt == 1); hdr->b_flags |= ARC_BUF_AVAILABLE; } ASSERT(refcount_is_zero(&hdr->b_refcnt) || callback_list != NULL); if (zio->io_error != 0) { hdr->b_flags |= ARC_IO_ERROR; if (hdr->b_state != arc_anon) arc_change_state(arc_anon, hdr, hash_lock); if (HDR_IN_HASH_TABLE(hdr)) buf_hash_remove(hdr); freeable = refcount_is_zero(&hdr->b_refcnt); } /* * Broadcast before we drop the hash_lock to avoid the possibility * that the hdr (and hence the cv) might be freed before we get to * the cv_broadcast(). */ cv_broadcast(&hdr->b_cv); if (hash_lock) { mutex_exit(hash_lock); } else { /* * This block was freed while we waited for the read to * complete. It has been removed from the hash table and * moved to the anonymous state (so that it won't show up * in the cache). */ ASSERT3P(hdr->b_state, ==, arc_anon); freeable = refcount_is_zero(&hdr->b_refcnt); } /* execute each callback and free its structure */ while ((acb = callback_list) != NULL) { if (acb->acb_done) acb->acb_done(zio, acb->acb_buf, acb->acb_private); if (acb->acb_zio_dummy != NULL) { acb->acb_zio_dummy->io_error = zio->io_error; zio_nowait(acb->acb_zio_dummy); } callback_list = acb->acb_next; kmem_free(acb, sizeof (arc_callback_t)); } if (freeable) arc_hdr_destroy(hdr); } /* * "Read" the block block at the specified DVA (in bp) via the * cache. If the block is found in the cache, invoke the provided * callback immediately and return. Note that the `zio' parameter * in the callback will be NULL in this case, since no IO was * required. If the block is not in the cache pass the read request * on to the spa with a substitute callback function, so that the * requested block will be added to the cache. * * If a read request arrives for a block that has a read in-progress, * either wait for the in-progress read to complete (and return the * results); or, if this is a read with a "done" func, add a record * to the read to invoke the "done" func when the read completes, * and return; or just return. * * arc_read_done() will invoke all the requested "done" functions * for readers of this block. */ int arc_read(zio_t *pio, spa_t *spa, const blkptr_t *bp, arc_done_func_t *done, void *private, zio_priority_t priority, int zio_flags, uint32_t *arc_flags, const zbookmark_phys_t *zb) { arc_buf_hdr_t *hdr = NULL; arc_buf_t *buf = NULL; kmutex_t *hash_lock = NULL; zio_t *rzio; uint64_t guid = spa_load_guid(spa); ASSERT(!BP_IS_EMBEDDED(bp) || BPE_GET_ETYPE(bp) == BP_EMBEDDED_TYPE_DATA); top: if (!BP_IS_EMBEDDED(bp)) { /* * Embedded BP's have no DVA and require no I/O to "read". * Create an anonymous arc buf to back it. */ hdr = buf_hash_find(guid, bp, &hash_lock); } if (hdr != NULL && hdr->b_datacnt > 0) { *arc_flags |= ARC_CACHED; if (HDR_IO_IN_PROGRESS(hdr)) { if (*arc_flags & ARC_WAIT) { cv_wait(&hdr->b_cv, hash_lock); mutex_exit(hash_lock); goto top; } ASSERT(*arc_flags & ARC_NOWAIT); if (done) { arc_callback_t *acb = NULL; acb = kmem_zalloc(sizeof (arc_callback_t), KM_SLEEP); acb->acb_done = done; acb->acb_private = private; if (pio != NULL) acb->acb_zio_dummy = zio_null(pio, spa, NULL, NULL, NULL, zio_flags); ASSERT(acb->acb_done != NULL); acb->acb_next = hdr->b_acb; hdr->b_acb = acb; add_reference(hdr, hash_lock, private); mutex_exit(hash_lock); return (0); } mutex_exit(hash_lock); return (0); } ASSERT(hdr->b_state == arc_mru || hdr->b_state == arc_mfu); if (done) { add_reference(hdr, hash_lock, private); /* * If this block is already in use, create a new * copy of the data so that we will be guaranteed * that arc_release() will always succeed. */ buf = hdr->b_buf; ASSERT(buf); ASSERT(buf->b_data); if (HDR_BUF_AVAILABLE(hdr)) { ASSERT(buf->b_efunc == NULL); hdr->b_flags &= ~ARC_BUF_AVAILABLE; } else { buf = arc_buf_clone(buf); } } else if (*arc_flags & ARC_PREFETCH && refcount_count(&hdr->b_refcnt) == 0) { hdr->b_flags |= ARC_PREFETCH; } DTRACE_PROBE1(arc__hit, arc_buf_hdr_t *, hdr); arc_access(hdr, hash_lock); if (*arc_flags & ARC_L2CACHE) hdr->b_flags |= ARC_L2CACHE; if (*arc_flags & ARC_L2COMPRESS) hdr->b_flags |= ARC_L2COMPRESS; mutex_exit(hash_lock); ARCSTAT_BUMP(arcstat_hits); ARCSTAT_CONDSTAT(!(hdr->b_flags & ARC_PREFETCH), demand, prefetch, hdr->b_type != ARC_BUFC_METADATA, data, metadata, hits); if (done) done(NULL, buf, private); } else { uint64_t size = BP_GET_LSIZE(bp); arc_callback_t *acb; vdev_t *vd = NULL; uint64_t addr = 0; boolean_t devw = B_FALSE; enum zio_compress b_compress = ZIO_COMPRESS_OFF; uint64_t b_asize = 0; if (hdr == NULL) { /* this block is not in the cache */ arc_buf_hdr_t *exists = NULL; arc_buf_contents_t type = BP_GET_BUFC_TYPE(bp); buf = arc_buf_alloc(spa, size, private, type); hdr = buf->b_hdr; if (!BP_IS_EMBEDDED(bp)) { hdr->b_dva = *BP_IDENTITY(bp); hdr->b_birth = BP_PHYSICAL_BIRTH(bp); hdr->b_cksum0 = bp->blk_cksum.zc_word[0]; exists = buf_hash_insert(hdr, &hash_lock); } if (exists != NULL) { /* somebody beat us to the hash insert */ mutex_exit(hash_lock); buf_discard_identity(hdr); (void) arc_buf_remove_ref(buf, private); goto top; /* restart the IO request */ } /* if this is a prefetch, we don't have a reference */ if (*arc_flags & ARC_PREFETCH) { (void) remove_reference(hdr, hash_lock, private); hdr->b_flags |= ARC_PREFETCH; } if (*arc_flags & ARC_L2CACHE) hdr->b_flags |= ARC_L2CACHE; if (*arc_flags & ARC_L2COMPRESS) hdr->b_flags |= ARC_L2COMPRESS; if (BP_GET_LEVEL(bp) > 0) hdr->b_flags |= ARC_INDIRECT; } else { /* this block is in the ghost cache */ ASSERT(GHOST_STATE(hdr->b_state)); ASSERT(!HDR_IO_IN_PROGRESS(hdr)); ASSERT0(refcount_count(&hdr->b_refcnt)); ASSERT(hdr->b_buf == NULL); /* if this is a prefetch, we don't have a reference */ if (*arc_flags & ARC_PREFETCH) hdr->b_flags |= ARC_PREFETCH; else add_reference(hdr, hash_lock, private); if (*arc_flags & ARC_L2CACHE) hdr->b_flags |= ARC_L2CACHE; if (*arc_flags & ARC_L2COMPRESS) hdr->b_flags |= ARC_L2COMPRESS; buf = kmem_cache_alloc(buf_cache, KM_PUSHPAGE); buf->b_hdr = hdr; buf->b_data = NULL; buf->b_efunc = NULL; buf->b_private = NULL; buf->b_next = NULL; hdr->b_buf = buf; ASSERT(hdr->b_datacnt == 0); hdr->b_datacnt = 1; arc_get_data_buf(buf); arc_access(hdr, hash_lock); } ASSERT(!GHOST_STATE(hdr->b_state)); acb = kmem_zalloc(sizeof (arc_callback_t), KM_SLEEP); acb->acb_done = done; acb->acb_private = private; ASSERT(hdr->b_acb == NULL); hdr->b_acb = acb; hdr->b_flags |= ARC_IO_IN_PROGRESS; if (hdr->b_l2hdr != NULL && (vd = hdr->b_l2hdr->b_dev->l2ad_vdev) != NULL) { devw = hdr->b_l2hdr->b_dev->l2ad_writing; addr = hdr->b_l2hdr->b_daddr; b_compress = hdr->b_l2hdr->b_compress; b_asize = hdr->b_l2hdr->b_asize; /* * Lock out device removal. */ if (vdev_is_dead(vd) || !spa_config_tryenter(spa, SCL_L2ARC, vd, RW_READER)) vd = NULL; } if (hash_lock != NULL) mutex_exit(hash_lock); /* * At this point, we have a level 1 cache miss. Try again in * L2ARC if possible. */ ASSERT3U(hdr->b_size, ==, size); DTRACE_PROBE4(arc__miss, arc_buf_hdr_t *, hdr, blkptr_t *, bp, uint64_t, size, zbookmark_phys_t *, zb); ARCSTAT_BUMP(arcstat_misses); ARCSTAT_CONDSTAT(!(hdr->b_flags & ARC_PREFETCH), demand, prefetch, hdr->b_type != ARC_BUFC_METADATA, data, metadata, misses); #ifdef _KERNEL curthread->td_ru.ru_inblock++; #endif if (vd != NULL && l2arc_ndev != 0 && !(l2arc_norw && devw)) { /* * Read from the L2ARC if the following are true: * 1. The L2ARC vdev was previously cached. * 2. This buffer still has L2ARC metadata. * 3. This buffer isn't currently writing to the L2ARC. * 4. The L2ARC entry wasn't evicted, which may * also have invalidated the vdev. * 5. This isn't prefetch and l2arc_noprefetch is set. */ if (hdr->b_l2hdr != NULL && !HDR_L2_WRITING(hdr) && !HDR_L2_EVICTED(hdr) && !(l2arc_noprefetch && HDR_PREFETCH(hdr))) { l2arc_read_callback_t *cb; DTRACE_PROBE1(l2arc__hit, arc_buf_hdr_t *, hdr); ARCSTAT_BUMP(arcstat_l2_hits); cb = kmem_zalloc(sizeof (l2arc_read_callback_t), KM_SLEEP); cb->l2rcb_buf = buf; cb->l2rcb_spa = spa; cb->l2rcb_bp = *bp; cb->l2rcb_zb = *zb; cb->l2rcb_flags = zio_flags; cb->l2rcb_compress = b_compress; ASSERT(addr >= VDEV_LABEL_START_SIZE && addr + size < vd->vdev_psize - VDEV_LABEL_END_SIZE); /* * l2arc read. The SCL_L2ARC lock will be * released by l2arc_read_done(). * Issue a null zio if the underlying buffer * was squashed to zero size by compression. */ if (b_compress == ZIO_COMPRESS_EMPTY) { rzio = zio_null(pio, spa, vd, l2arc_read_done, cb, zio_flags | ZIO_FLAG_DONT_CACHE | ZIO_FLAG_CANFAIL | ZIO_FLAG_DONT_PROPAGATE | ZIO_FLAG_DONT_RETRY); } else { rzio = zio_read_phys(pio, vd, addr, b_asize, buf->b_data, ZIO_CHECKSUM_OFF, l2arc_read_done, cb, priority, zio_flags | ZIO_FLAG_DONT_CACHE | ZIO_FLAG_CANFAIL | ZIO_FLAG_DONT_PROPAGATE | ZIO_FLAG_DONT_RETRY, B_FALSE); } DTRACE_PROBE2(l2arc__read, vdev_t *, vd, zio_t *, rzio); ARCSTAT_INCR(arcstat_l2_read_bytes, b_asize); if (*arc_flags & ARC_NOWAIT) { zio_nowait(rzio); return (0); } ASSERT(*arc_flags & ARC_WAIT); if (zio_wait(rzio) == 0) return (0); /* l2arc read error; goto zio_read() */ } else { DTRACE_PROBE1(l2arc__miss, arc_buf_hdr_t *, hdr); ARCSTAT_BUMP(arcstat_l2_misses); if (HDR_L2_WRITING(hdr)) ARCSTAT_BUMP(arcstat_l2_rw_clash); spa_config_exit(spa, SCL_L2ARC, vd); } } else { if (vd != NULL) spa_config_exit(spa, SCL_L2ARC, vd); if (l2arc_ndev != 0) { DTRACE_PROBE1(l2arc__miss, arc_buf_hdr_t *, hdr); ARCSTAT_BUMP(arcstat_l2_misses); } } rzio = zio_read(pio, spa, bp, buf->b_data, size, arc_read_done, buf, priority, zio_flags, zb); if (*arc_flags & ARC_WAIT) return (zio_wait(rzio)); ASSERT(*arc_flags & ARC_NOWAIT); zio_nowait(rzio); } return (0); } void arc_set_callback(arc_buf_t *buf, arc_evict_func_t *func, void *private) { ASSERT(buf->b_hdr != NULL); ASSERT(buf->b_hdr->b_state != arc_anon); ASSERT(!refcount_is_zero(&buf->b_hdr->b_refcnt) || func == NULL); ASSERT(buf->b_efunc == NULL); ASSERT(!HDR_BUF_AVAILABLE(buf->b_hdr)); buf->b_efunc = func; buf->b_private = private; } /* * Notify the arc that a block was freed, and thus will never be used again. */ void arc_freed(spa_t *spa, const blkptr_t *bp) { arc_buf_hdr_t *hdr; kmutex_t *hash_lock; uint64_t guid = spa_load_guid(spa); ASSERT(!BP_IS_EMBEDDED(bp)); hdr = buf_hash_find(guid, bp, &hash_lock); if (hdr == NULL) return; if (HDR_BUF_AVAILABLE(hdr)) { arc_buf_t *buf = hdr->b_buf; add_reference(hdr, hash_lock, FTAG); hdr->b_flags &= ~ARC_BUF_AVAILABLE; mutex_exit(hash_lock); arc_release(buf, FTAG); (void) arc_buf_remove_ref(buf, FTAG); } else { mutex_exit(hash_lock); } } /* * Clear the user eviction callback set by arc_set_callback(), first calling * it if it exists. Because the presence of a callback keeps an arc_buf cached * clearing the callback may result in the arc_buf being destroyed. However, * it will not result in the *last* arc_buf being destroyed, hence the data * will remain cached in the ARC. We make a copy of the arc buffer here so * that we can process the callback without holding any locks. * * It's possible that the callback is already in the process of being cleared * by another thread. In this case we can not clear the callback. * * Returns B_TRUE if the callback was successfully called and cleared. */ boolean_t arc_clear_callback(arc_buf_t *buf) { arc_buf_hdr_t *hdr; kmutex_t *hash_lock; arc_evict_func_t *efunc = buf->b_efunc; void *private = buf->b_private; list_t *list, *evicted_list; kmutex_t *lock, *evicted_lock; mutex_enter(&buf->b_evict_lock); hdr = buf->b_hdr; if (hdr == NULL) { /* * We are in arc_do_user_evicts(). */ ASSERT(buf->b_data == NULL); mutex_exit(&buf->b_evict_lock); return (B_FALSE); } else if (buf->b_data == NULL) { /* * We are on the eviction list; process this buffer now * but let arc_do_user_evicts() do the reaping. */ buf->b_efunc = NULL; mutex_exit(&buf->b_evict_lock); VERIFY0(efunc(private)); return (B_TRUE); } hash_lock = HDR_LOCK(hdr); mutex_enter(hash_lock); hdr = buf->b_hdr; ASSERT3P(hash_lock, ==, HDR_LOCK(hdr)); ASSERT3U(refcount_count(&hdr->b_refcnt), <, hdr->b_datacnt); ASSERT(hdr->b_state == arc_mru || hdr->b_state == arc_mfu); buf->b_efunc = NULL; buf->b_private = NULL; if (hdr->b_datacnt > 1) { mutex_exit(&buf->b_evict_lock); arc_buf_destroy(buf, FALSE, TRUE); } else { ASSERT(buf == hdr->b_buf); hdr->b_flags |= ARC_BUF_AVAILABLE; mutex_exit(&buf->b_evict_lock); } mutex_exit(hash_lock); VERIFY0(efunc(private)); return (B_TRUE); } /* * Release this buffer from the cache, making it an anonymous buffer. This * must be done after a read and prior to modifying the buffer contents. * If the buffer has more than one reference, we must make * a new hdr for the buffer. */ void arc_release(arc_buf_t *buf, void *tag) { arc_buf_hdr_t *hdr; kmutex_t *hash_lock = NULL; l2arc_buf_hdr_t *l2hdr; uint64_t buf_size; /* * It would be nice to assert that if it's DMU metadata (level > * 0 || it's the dnode file), then it must be syncing context. * But we don't know that information at this level. */ mutex_enter(&buf->b_evict_lock); hdr = buf->b_hdr; /* this buffer is not on any list */ ASSERT(refcount_count(&hdr->b_refcnt) > 0); if (hdr->b_state == arc_anon) { /* this buffer is already released */ ASSERT(buf->b_efunc == NULL); } else { hash_lock = HDR_LOCK(hdr); mutex_enter(hash_lock); hdr = buf->b_hdr; ASSERT3P(hash_lock, ==, HDR_LOCK(hdr)); } l2hdr = hdr->b_l2hdr; if (l2hdr) { mutex_enter(&l2arc_buflist_mtx); + arc_buf_l2_cdata_free(hdr); hdr->b_l2hdr = NULL; list_remove(l2hdr->b_dev->l2ad_buflist, hdr); } buf_size = hdr->b_size; /* * Do we have more than one buf? */ if (hdr->b_datacnt > 1) { arc_buf_hdr_t *nhdr; arc_buf_t **bufp; uint64_t blksz = hdr->b_size; uint64_t spa = hdr->b_spa; arc_buf_contents_t type = hdr->b_type; uint32_t flags = hdr->b_flags; ASSERT(hdr->b_buf != buf || buf->b_next != NULL); /* * Pull the data off of this hdr and attach it to * a new anonymous hdr. */ (void) remove_reference(hdr, hash_lock, tag); bufp = &hdr->b_buf; while (*bufp != buf) bufp = &(*bufp)->b_next; *bufp = buf->b_next; buf->b_next = NULL; ASSERT3U(hdr->b_state->arcs_size, >=, hdr->b_size); atomic_add_64(&hdr->b_state->arcs_size, -hdr->b_size); if (refcount_is_zero(&hdr->b_refcnt)) { uint64_t *size = &hdr->b_state->arcs_lsize[hdr->b_type]; ASSERT3U(*size, >=, hdr->b_size); atomic_add_64(size, -hdr->b_size); } /* * We're releasing a duplicate user data buffer, update * our statistics accordingly. */ if (hdr->b_type == ARC_BUFC_DATA) { ARCSTAT_BUMPDOWN(arcstat_duplicate_buffers); ARCSTAT_INCR(arcstat_duplicate_buffers_size, -hdr->b_size); } hdr->b_datacnt -= 1; arc_cksum_verify(buf); #ifdef illumos arc_buf_unwatch(buf); #endif /* illumos */ mutex_exit(hash_lock); nhdr = kmem_cache_alloc(hdr_cache, KM_PUSHPAGE); nhdr->b_size = blksz; nhdr->b_spa = spa; nhdr->b_type = type; nhdr->b_buf = buf; nhdr->b_state = arc_anon; nhdr->b_arc_access = 0; nhdr->b_flags = flags & ARC_L2_WRITING; nhdr->b_l2hdr = NULL; nhdr->b_datacnt = 1; nhdr->b_freeze_cksum = NULL; (void) refcount_add(&nhdr->b_refcnt, tag); buf->b_hdr = nhdr; mutex_exit(&buf->b_evict_lock); atomic_add_64(&arc_anon->arcs_size, blksz); } else { mutex_exit(&buf->b_evict_lock); ASSERT(refcount_count(&hdr->b_refcnt) == 1); ASSERT(!list_link_active(&hdr->b_arc_node)); ASSERT(!HDR_IO_IN_PROGRESS(hdr)); if (hdr->b_state != arc_anon) arc_change_state(arc_anon, hdr, hash_lock); hdr->b_arc_access = 0; if (hash_lock) mutex_exit(hash_lock); buf_discard_identity(hdr); arc_buf_thaw(buf); } buf->b_efunc = NULL; buf->b_private = NULL; if (l2hdr) { ARCSTAT_INCR(arcstat_l2_asize, -l2hdr->b_asize); vdev_space_update(l2hdr->b_dev->l2ad_vdev, -l2hdr->b_asize, 0, 0); trim_map_free(l2hdr->b_dev->l2ad_vdev, l2hdr->b_daddr, hdr->b_size, 0); kmem_free(l2hdr, sizeof (l2arc_buf_hdr_t)); ARCSTAT_INCR(arcstat_l2_size, -buf_size); mutex_exit(&l2arc_buflist_mtx); } } int arc_released(arc_buf_t *buf) { int released; mutex_enter(&buf->b_evict_lock); released = (buf->b_data != NULL && buf->b_hdr->b_state == arc_anon); mutex_exit(&buf->b_evict_lock); return (released); } #ifdef ZFS_DEBUG int arc_referenced(arc_buf_t *buf) { int referenced; mutex_enter(&buf->b_evict_lock); referenced = (refcount_count(&buf->b_hdr->b_refcnt)); mutex_exit(&buf->b_evict_lock); return (referenced); } #endif static void arc_write_ready(zio_t *zio) { arc_write_callback_t *callback = zio->io_private; arc_buf_t *buf = callback->awcb_buf; arc_buf_hdr_t *hdr = buf->b_hdr; ASSERT(!refcount_is_zero(&buf->b_hdr->b_refcnt)); callback->awcb_ready(zio, buf, callback->awcb_private); /* * If the IO is already in progress, then this is a re-write * attempt, so we need to thaw and re-compute the cksum. * It is the responsibility of the callback to handle the * accounting for any re-write attempt. */ if (HDR_IO_IN_PROGRESS(hdr)) { mutex_enter(&hdr->b_freeze_lock); if (hdr->b_freeze_cksum != NULL) { kmem_free(hdr->b_freeze_cksum, sizeof (zio_cksum_t)); hdr->b_freeze_cksum = NULL; } mutex_exit(&hdr->b_freeze_lock); } arc_cksum_compute(buf, B_FALSE); hdr->b_flags |= ARC_IO_IN_PROGRESS; } /* * The SPA calls this callback for each physical write that happens on behalf * of a logical write. See the comment in dbuf_write_physdone() for details. */ static void arc_write_physdone(zio_t *zio) { arc_write_callback_t *cb = zio->io_private; if (cb->awcb_physdone != NULL) cb->awcb_physdone(zio, cb->awcb_buf, cb->awcb_private); } static void arc_write_done(zio_t *zio) { arc_write_callback_t *callback = zio->io_private; arc_buf_t *buf = callback->awcb_buf; arc_buf_hdr_t *hdr = buf->b_hdr; ASSERT(hdr->b_acb == NULL); if (zio->io_error == 0) { if (BP_IS_HOLE(zio->io_bp) || BP_IS_EMBEDDED(zio->io_bp)) { buf_discard_identity(hdr); } else { hdr->b_dva = *BP_IDENTITY(zio->io_bp); hdr->b_birth = BP_PHYSICAL_BIRTH(zio->io_bp); hdr->b_cksum0 = zio->io_bp->blk_cksum.zc_word[0]; } } else { ASSERT(BUF_EMPTY(hdr)); } /* * If the block to be written was all-zero or compressed enough to be * embedded in the BP, no write was performed so there will be no * dva/birth/checksum. The buffer must therefore remain anonymous * (and uncached). */ if (!BUF_EMPTY(hdr)) { arc_buf_hdr_t *exists; kmutex_t *hash_lock; ASSERT(zio->io_error == 0); arc_cksum_verify(buf); exists = buf_hash_insert(hdr, &hash_lock); if (exists) { /* * This can only happen if we overwrite for * sync-to-convergence, because we remove * buffers from the hash table when we arc_free(). */ if (zio->io_flags & ZIO_FLAG_IO_REWRITE) { if (!BP_EQUAL(&zio->io_bp_orig, zio->io_bp)) panic("bad overwrite, hdr=%p exists=%p", (void *)hdr, (void *)exists); ASSERT(refcount_is_zero(&exists->b_refcnt)); arc_change_state(arc_anon, exists, hash_lock); mutex_exit(hash_lock); arc_hdr_destroy(exists); exists = buf_hash_insert(hdr, &hash_lock); ASSERT3P(exists, ==, NULL); } else if (zio->io_flags & ZIO_FLAG_NOPWRITE) { /* nopwrite */ ASSERT(zio->io_prop.zp_nopwrite); if (!BP_EQUAL(&zio->io_bp_orig, zio->io_bp)) panic("bad nopwrite, hdr=%p exists=%p", (void *)hdr, (void *)exists); } else { /* Dedup */ ASSERT(hdr->b_datacnt == 1); ASSERT(hdr->b_state == arc_anon); ASSERT(BP_GET_DEDUP(zio->io_bp)); ASSERT(BP_GET_LEVEL(zio->io_bp) == 0); } } hdr->b_flags &= ~ARC_IO_IN_PROGRESS; /* if it's not anon, we are doing a scrub */ if (!exists && hdr->b_state == arc_anon) arc_access(hdr, hash_lock); mutex_exit(hash_lock); } else { hdr->b_flags &= ~ARC_IO_IN_PROGRESS; } ASSERT(!refcount_is_zero(&hdr->b_refcnt)); callback->awcb_done(zio, buf, callback->awcb_private); kmem_free(callback, sizeof (arc_write_callback_t)); } zio_t * arc_write(zio_t *pio, spa_t *spa, uint64_t txg, blkptr_t *bp, arc_buf_t *buf, boolean_t l2arc, boolean_t l2arc_compress, const zio_prop_t *zp, arc_done_func_t *ready, arc_done_func_t *physdone, arc_done_func_t *done, void *private, zio_priority_t priority, int zio_flags, const zbookmark_phys_t *zb) { arc_buf_hdr_t *hdr = buf->b_hdr; arc_write_callback_t *callback; zio_t *zio; ASSERT(ready != NULL); ASSERT(done != NULL); ASSERT(!HDR_IO_ERROR(hdr)); ASSERT((hdr->b_flags & ARC_IO_IN_PROGRESS) == 0); ASSERT(hdr->b_acb == NULL); if (l2arc) hdr->b_flags |= ARC_L2CACHE; if (l2arc_compress) hdr->b_flags |= ARC_L2COMPRESS; callback = kmem_zalloc(sizeof (arc_write_callback_t), KM_SLEEP); callback->awcb_ready = ready; callback->awcb_physdone = physdone; callback->awcb_done = done; callback->awcb_private = private; callback->awcb_buf = buf; zio = zio_write(pio, spa, txg, bp, buf->b_data, hdr->b_size, zp, arc_write_ready, arc_write_physdone, arc_write_done, callback, priority, zio_flags, zb); return (zio); } static int arc_memory_throttle(uint64_t reserve, uint64_t txg) { #ifdef _KERNEL uint64_t available_memory = ptoa((uintmax_t)cnt.v_free_count + cnt.v_cache_count); static uint64_t page_load = 0; static uint64_t last_txg = 0; #ifdef sun #if defined(__i386) available_memory = MIN(available_memory, vmem_size(heap_arena, VMEM_FREE)); #endif #endif /* sun */ if (cnt.v_free_count + cnt.v_cache_count > (uint64_t)physmem * arc_lotsfree_percent / 100) return (0); if (txg > last_txg) { last_txg = txg; page_load = 0; } /* * If we are in pageout, we know that memory is already tight, * the arc is already going to be evicting, so we just want to * continue to let page writes occur as quickly as possible. */ if (curproc == pageproc) { if (page_load > available_memory / 4) return (SET_ERROR(ERESTART)); /* Note: reserve is inflated, so we deflate */ page_load += reserve / 8; return (0); } else if (page_load > 0 && arc_reclaim_needed()) { /* memory is low, delay before restarting */ ARCSTAT_INCR(arcstat_memory_throttle_count, 1); return (SET_ERROR(EAGAIN)); } page_load = 0; #endif return (0); } void arc_tempreserve_clear(uint64_t reserve) { atomic_add_64(&arc_tempreserve, -reserve); ASSERT((int64_t)arc_tempreserve >= 0); } int arc_tempreserve_space(uint64_t reserve, uint64_t txg) { int error; uint64_t anon_size; if (reserve > arc_c/4 && !arc_no_grow) arc_c = MIN(arc_c_max, reserve * 4); if (reserve > arc_c) return (SET_ERROR(ENOMEM)); /* * Don't count loaned bufs as in flight dirty data to prevent long * network delays from blocking transactions that are ready to be * assigned to a txg. */ anon_size = MAX((int64_t)(arc_anon->arcs_size - arc_loaned_bytes), 0); /* * Writes will, almost always, require additional memory allocations * in order to compress/encrypt/etc the data. We therefore need to * make sure that there is sufficient available memory for this. */ error = arc_memory_throttle(reserve, txg); if (error != 0) return (error); /* * Throttle writes when the amount of dirty data in the cache * gets too large. We try to keep the cache less than half full * of dirty blocks so that our sync times don't grow too large. * Note: if two requests come in concurrently, we might let them * both succeed, when one of them should fail. Not a huge deal. */ if (reserve + arc_tempreserve + anon_size > arc_c / 2 && anon_size > arc_c / 4) { dprintf("failing, arc_tempreserve=%lluK anon_meta=%lluK " "anon_data=%lluK tempreserve=%lluK arc_c=%lluK\n", arc_tempreserve>>10, arc_anon->arcs_lsize[ARC_BUFC_METADATA]>>10, arc_anon->arcs_lsize[ARC_BUFC_DATA]>>10, reserve>>10, arc_c>>10); return (SET_ERROR(ERESTART)); } atomic_add_64(&arc_tempreserve, reserve); return (0); } static kmutex_t arc_lowmem_lock; #ifdef _KERNEL static eventhandler_tag arc_event_lowmem = NULL; static void arc_lowmem(void *arg __unused, int howto __unused) { /* Serialize access via arc_lowmem_lock. */ mutex_enter(&arc_lowmem_lock); mutex_enter(&arc_reclaim_thr_lock); needfree = 1; cv_signal(&arc_reclaim_thr_cv); /* * It is unsafe to block here in arbitrary threads, because we can come * here from ARC itself and may hold ARC locks and thus risk a deadlock * with ARC reclaim thread. */ if (curproc == pageproc) { while (needfree) msleep(&needfree, &arc_reclaim_thr_lock, 0, "zfs:lowmem", 0); } mutex_exit(&arc_reclaim_thr_lock); mutex_exit(&arc_lowmem_lock); } #endif void arc_init(void) { int i, prefetch_tunable_set = 0; mutex_init(&arc_reclaim_thr_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&arc_reclaim_thr_cv, NULL, CV_DEFAULT, NULL); mutex_init(&arc_lowmem_lock, NULL, MUTEX_DEFAULT, NULL); /* Convert seconds to clock ticks */ arc_min_prefetch_lifespan = 1 * hz; /* Start out with 1/8 of all memory */ arc_c = kmem_size() / 8; #ifdef sun #ifdef _KERNEL /* * On architectures where the physical memory can be larger * than the addressable space (intel in 32-bit mode), we may * need to limit the cache to 1/8 of VM size. */ arc_c = MIN(arc_c, vmem_size(heap_arena, VMEM_ALLOC | VMEM_FREE) / 8); #endif #endif /* sun */ /* set min cache to 1/32 of all memory, or 16MB, whichever is more */ arc_c_min = MAX(arc_c / 4, 64<<18); /* set max to 1/2 of all memory, or all but 1GB, whichever is more */ if (arc_c * 8 >= 1<<30) arc_c_max = (arc_c * 8) - (1<<30); else arc_c_max = arc_c_min; arc_c_max = MAX(arc_c * 5, arc_c_max); #ifdef _KERNEL /* * Allow the tunables to override our calculations if they are * reasonable (ie. over 16MB) */ if (zfs_arc_max > 64<<18 && zfs_arc_max < kmem_size()) arc_c_max = zfs_arc_max; if (zfs_arc_min > 64<<18 && zfs_arc_min <= arc_c_max) arc_c_min = zfs_arc_min; #endif arc_c = arc_c_max; arc_p = (arc_c >> 1); /* limit meta-data to 1/4 of the arc capacity */ arc_meta_limit = arc_c_max / 4; /* Allow the tunable to override if it is reasonable */ if (zfs_arc_meta_limit > 0 && zfs_arc_meta_limit <= arc_c_max) arc_meta_limit = zfs_arc_meta_limit; if (arc_c_min < arc_meta_limit / 2 && zfs_arc_min == 0) arc_c_min = arc_meta_limit / 2; if (zfs_arc_grow_retry > 0) arc_grow_retry = zfs_arc_grow_retry; if (zfs_arc_shrink_shift > 0) arc_shrink_shift = zfs_arc_shrink_shift; if (zfs_arc_p_min_shift > 0) arc_p_min_shift = zfs_arc_p_min_shift; /* if kmem_flags are set, lets try to use less memory */ if (kmem_debugging()) arc_c = arc_c / 2; if (arc_c < arc_c_min) arc_c = arc_c_min; zfs_arc_min = arc_c_min; zfs_arc_max = arc_c_max; arc_anon = &ARC_anon; arc_mru = &ARC_mru; arc_mru_ghost = &ARC_mru_ghost; arc_mfu = &ARC_mfu; arc_mfu_ghost = &ARC_mfu_ghost; arc_l2c_only = &ARC_l2c_only; arc_size = 0; for (i = 0; i < ARC_BUFC_NUMLISTS; i++) { mutex_init(&arc_anon->arcs_locks[i].arcs_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&arc_mru->arcs_locks[i].arcs_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&arc_mru_ghost->arcs_locks[i].arcs_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&arc_mfu->arcs_locks[i].arcs_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&arc_mfu_ghost->arcs_locks[i].arcs_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&arc_l2c_only->arcs_locks[i].arcs_lock, NULL, MUTEX_DEFAULT, NULL); list_create(&arc_mru->arcs_lists[i], sizeof (arc_buf_hdr_t), offsetof(arc_buf_hdr_t, b_arc_node)); list_create(&arc_mru_ghost->arcs_lists[i], sizeof (arc_buf_hdr_t), offsetof(arc_buf_hdr_t, b_arc_node)); list_create(&arc_mfu->arcs_lists[i], sizeof (arc_buf_hdr_t), offsetof(arc_buf_hdr_t, b_arc_node)); list_create(&arc_mfu_ghost->arcs_lists[i], sizeof (arc_buf_hdr_t), offsetof(arc_buf_hdr_t, b_arc_node)); list_create(&arc_mfu_ghost->arcs_lists[i], sizeof (arc_buf_hdr_t), offsetof(arc_buf_hdr_t, b_arc_node)); list_create(&arc_l2c_only->arcs_lists[i], sizeof (arc_buf_hdr_t), offsetof(arc_buf_hdr_t, b_arc_node)); } buf_init(); arc_thread_exit = 0; arc_eviction_list = NULL; mutex_init(&arc_eviction_mtx, NULL, MUTEX_DEFAULT, NULL); bzero(&arc_eviction_hdr, sizeof (arc_buf_hdr_t)); arc_ksp = kstat_create("zfs", 0, "arcstats", "misc", KSTAT_TYPE_NAMED, sizeof (arc_stats) / sizeof (kstat_named_t), KSTAT_FLAG_VIRTUAL); if (arc_ksp != NULL) { arc_ksp->ks_data = &arc_stats; kstat_install(arc_ksp); } (void) thread_create(NULL, 0, arc_reclaim_thread, NULL, 0, &p0, TS_RUN, minclsyspri); #ifdef _KERNEL arc_event_lowmem = EVENTHANDLER_REGISTER(vm_lowmem, arc_lowmem, NULL, EVENTHANDLER_PRI_FIRST); #endif arc_dead = FALSE; arc_warm = B_FALSE; /* * Calculate maximum amount of dirty data per pool. * * If it has been set by /etc/system, take that. * Otherwise, use a percentage of physical memory defined by * zfs_dirty_data_max_percent (default 10%) with a cap at * zfs_dirty_data_max_max (default 4GB). */ if (zfs_dirty_data_max == 0) { zfs_dirty_data_max = ptob(physmem) * zfs_dirty_data_max_percent / 100; zfs_dirty_data_max = MIN(zfs_dirty_data_max, zfs_dirty_data_max_max); } #ifdef _KERNEL if (TUNABLE_INT_FETCH("vfs.zfs.prefetch_disable", &zfs_prefetch_disable)) prefetch_tunable_set = 1; #ifdef __i386__ if (prefetch_tunable_set == 0) { printf("ZFS NOTICE: Prefetch is disabled by default on i386 " "-- to enable,\n"); printf(" add \"vfs.zfs.prefetch_disable=0\" " "to /boot/loader.conf.\n"); zfs_prefetch_disable = 1; } #else if ((((uint64_t)physmem * PAGESIZE) < (1ULL << 32)) && prefetch_tunable_set == 0) { printf("ZFS NOTICE: Prefetch is disabled by default if less " "than 4GB of RAM is present;\n" " to enable, add \"vfs.zfs.prefetch_disable=0\" " "to /boot/loader.conf.\n"); zfs_prefetch_disable = 1; } #endif /* Warn about ZFS memory and address space requirements. */ if (((uint64_t)physmem * PAGESIZE) < (256 + 128 + 64) * (1 << 20)) { printf("ZFS WARNING: Recommended minimum RAM size is 512MB; " "expect unstable behavior.\n"); } if (kmem_size() < 512 * (1 << 20)) { printf("ZFS WARNING: Recommended minimum kmem_size is 512MB; " "expect unstable behavior.\n"); printf(" Consider tuning vm.kmem_size and " "vm.kmem_size_max\n"); printf(" in /boot/loader.conf.\n"); } #endif } void arc_fini(void) { int i; mutex_enter(&arc_reclaim_thr_lock); arc_thread_exit = 1; cv_signal(&arc_reclaim_thr_cv); while (arc_thread_exit != 0) cv_wait(&arc_reclaim_thr_cv, &arc_reclaim_thr_lock); mutex_exit(&arc_reclaim_thr_lock); arc_flush(NULL); arc_dead = TRUE; if (arc_ksp != NULL) { kstat_delete(arc_ksp); arc_ksp = NULL; } mutex_destroy(&arc_eviction_mtx); mutex_destroy(&arc_reclaim_thr_lock); cv_destroy(&arc_reclaim_thr_cv); for (i = 0; i < ARC_BUFC_NUMLISTS; i++) { list_destroy(&arc_mru->arcs_lists[i]); list_destroy(&arc_mru_ghost->arcs_lists[i]); list_destroy(&arc_mfu->arcs_lists[i]); list_destroy(&arc_mfu_ghost->arcs_lists[i]); list_destroy(&arc_l2c_only->arcs_lists[i]); mutex_destroy(&arc_anon->arcs_locks[i].arcs_lock); mutex_destroy(&arc_mru->arcs_locks[i].arcs_lock); mutex_destroy(&arc_mru_ghost->arcs_locks[i].arcs_lock); mutex_destroy(&arc_mfu->arcs_locks[i].arcs_lock); mutex_destroy(&arc_mfu_ghost->arcs_locks[i].arcs_lock); mutex_destroy(&arc_l2c_only->arcs_locks[i].arcs_lock); } buf_fini(); ASSERT(arc_loaned_bytes == 0); mutex_destroy(&arc_lowmem_lock); #ifdef _KERNEL if (arc_event_lowmem != NULL) EVENTHANDLER_DEREGISTER(vm_lowmem, arc_event_lowmem); #endif } /* * Level 2 ARC * * The level 2 ARC (L2ARC) is a cache layer in-between main memory and disk. * It uses dedicated storage devices to hold cached data, which are populated * using large infrequent writes. The main role of this cache is to boost * the performance of random read workloads. The intended L2ARC devices * include short-stroked disks, solid state disks, and other media with * substantially faster read latency than disk. * * +-----------------------+ * | ARC | * +-----------------------+ * | ^ ^ * | | | * l2arc_feed_thread() arc_read() * | | | * | l2arc read | * V | | * +---------------+ | * | L2ARC | | * +---------------+ | * | ^ | * l2arc_write() | | * | | | * V | | * +-------+ +-------+ * | vdev | | vdev | * | cache | | cache | * +-------+ +-------+ * +=========+ .-----. * : L2ARC : |-_____-| * : devices : | Disks | * +=========+ `-_____-' * * Read requests are satisfied from the following sources, in order: * * 1) ARC * 2) vdev cache of L2ARC devices * 3) L2ARC devices * 4) vdev cache of disks * 5) disks * * Some L2ARC device types exhibit extremely slow write performance. * To accommodate for this there are some significant differences between * the L2ARC and traditional cache design: * * 1. There is no eviction path from the ARC to the L2ARC. Evictions from * the ARC behave as usual, freeing buffers and placing headers on ghost * lists. The ARC does not send buffers to the L2ARC during eviction as * this would add inflated write latencies for all ARC memory pressure. * * 2. The L2ARC attempts to cache data from the ARC before it is evicted. * It does this by periodically scanning buffers from the eviction-end of * the MFU and MRU ARC lists, copying them to the L2ARC devices if they are * not already there. It scans until a headroom of buffers is satisfied, * which itself is a buffer for ARC eviction. If a compressible buffer is * found during scanning and selected for writing to an L2ARC device, we * temporarily boost scanning headroom during the next scan cycle to make * sure we adapt to compression effects (which might significantly reduce * the data volume we write to L2ARC). The thread that does this is * l2arc_feed_thread(), illustrated below; example sizes are included to * provide a better sense of ratio than this diagram: * * head --> tail * +---------------------+----------+ * ARC_mfu |:::::#:::::::::::::::|o#o###o###|-->. # already on L2ARC * +---------------------+----------+ | o L2ARC eligible * ARC_mru |:#:::::::::::::::::::|#o#ooo####|-->| : ARC buffer * +---------------------+----------+ | * 15.9 Gbytes ^ 32 Mbytes | * headroom | * l2arc_feed_thread() * | * l2arc write hand <--[oooo]--' * | 8 Mbyte * | write max * V * +==============================+ * L2ARC dev |####|#|###|###| |####| ... | * +==============================+ * 32 Gbytes * * 3. If an ARC buffer is copied to the L2ARC but then hit instead of * evicted, then the L2ARC has cached a buffer much sooner than it probably * needed to, potentially wasting L2ARC device bandwidth and storage. It is * safe to say that this is an uncommon case, since buffers at the end of * the ARC lists have moved there due to inactivity. * * 4. If the ARC evicts faster than the L2ARC can maintain a headroom, * then the L2ARC simply misses copying some buffers. This serves as a * pressure valve to prevent heavy read workloads from both stalling the ARC * with waits and clogging the L2ARC with writes. This also helps prevent * the potential for the L2ARC to churn if it attempts to cache content too * quickly, such as during backups of the entire pool. * * 5. After system boot and before the ARC has filled main memory, there are * no evictions from the ARC and so the tails of the ARC_mfu and ARC_mru * lists can remain mostly static. Instead of searching from tail of these * lists as pictured, the l2arc_feed_thread() will search from the list heads * for eligible buffers, greatly increasing its chance of finding them. * * The L2ARC device write speed is also boosted during this time so that * the L2ARC warms up faster. Since there have been no ARC evictions yet, * there are no L2ARC reads, and no fear of degrading read performance * through increased writes. * * 6. Writes to the L2ARC devices are grouped and sent in-sequence, so that * the vdev queue can aggregate them into larger and fewer writes. Each * device is written to in a rotor fashion, sweeping writes through * available space then repeating. * * 7. The L2ARC does not store dirty content. It never needs to flush * write buffers back to disk based storage. * * 8. If an ARC buffer is written (and dirtied) which also exists in the * L2ARC, the now stale L2ARC buffer is immediately dropped. * * The performance of the L2ARC can be tweaked by a number of tunables, which * may be necessary for different workloads: * * l2arc_write_max max write bytes per interval * l2arc_write_boost extra write bytes during device warmup * l2arc_noprefetch skip caching prefetched buffers * l2arc_headroom number of max device writes to precache * l2arc_headroom_boost when we find compressed buffers during ARC * scanning, we multiply headroom by this * percentage factor for the next scan cycle, * since more compressed buffers are likely to * be present * l2arc_feed_secs seconds between L2ARC writing * * Tunables may be removed or added as future performance improvements are * integrated, and also may become zpool properties. * * There are three key functions that control how the L2ARC warms up: * * l2arc_write_eligible() check if a buffer is eligible to cache * l2arc_write_size() calculate how much to write * l2arc_write_interval() calculate sleep delay between writes * * These three functions determine what to write, how much, and how quickly * to send writes. */ static boolean_t l2arc_write_eligible(uint64_t spa_guid, arc_buf_hdr_t *ab) { /* * A buffer is *not* eligible for the L2ARC if it: * 1. belongs to a different spa. * 2. is already cached on the L2ARC. * 3. has an I/O in progress (it may be an incomplete read). * 4. is flagged not eligible (zfs property). */ if (ab->b_spa != spa_guid) { ARCSTAT_BUMP(arcstat_l2_write_spa_mismatch); return (B_FALSE); } if (ab->b_l2hdr != NULL) { ARCSTAT_BUMP(arcstat_l2_write_in_l2); return (B_FALSE); } if (HDR_IO_IN_PROGRESS(ab)) { ARCSTAT_BUMP(arcstat_l2_write_hdr_io_in_progress); return (B_FALSE); } if (!HDR_L2CACHE(ab)) { ARCSTAT_BUMP(arcstat_l2_write_not_cacheable); return (B_FALSE); } return (B_TRUE); } static uint64_t l2arc_write_size(void) { uint64_t size; /* * Make sure our globals have meaningful values in case the user * altered them. */ size = l2arc_write_max; if (size == 0) { cmn_err(CE_NOTE, "Bad value for l2arc_write_max, value must " "be greater than zero, resetting it to the default (%d)", L2ARC_WRITE_SIZE); size = l2arc_write_max = L2ARC_WRITE_SIZE; } if (arc_warm == B_FALSE) size += l2arc_write_boost; return (size); } static clock_t l2arc_write_interval(clock_t began, uint64_t wanted, uint64_t wrote) { clock_t interval, next, now; /* * If the ARC lists are busy, increase our write rate; if the * lists are stale, idle back. This is achieved by checking * how much we previously wrote - if it was more than half of * what we wanted, schedule the next write much sooner. */ if (l2arc_feed_again && wrote > (wanted / 2)) interval = (hz * l2arc_feed_min_ms) / 1000; else interval = hz * l2arc_feed_secs; now = ddi_get_lbolt(); next = MAX(now, MIN(now + interval, began + interval)); return (next); } static void l2arc_hdr_stat_add(void) { ARCSTAT_INCR(arcstat_l2_hdr_size, HDR_SIZE + L2HDR_SIZE); ARCSTAT_INCR(arcstat_hdr_size, -HDR_SIZE); } static void l2arc_hdr_stat_remove(void) { ARCSTAT_INCR(arcstat_l2_hdr_size, -(HDR_SIZE + L2HDR_SIZE)); ARCSTAT_INCR(arcstat_hdr_size, HDR_SIZE); } /* * Cycle through L2ARC devices. This is how L2ARC load balances. * If a device is returned, this also returns holding the spa config lock. */ static l2arc_dev_t * l2arc_dev_get_next(void) { l2arc_dev_t *first, *next = NULL; /* * Lock out the removal of spas (spa_namespace_lock), then removal * of cache devices (l2arc_dev_mtx). Once a device has been selected, * both locks will be dropped and a spa config lock held instead. */ mutex_enter(&spa_namespace_lock); mutex_enter(&l2arc_dev_mtx); /* if there are no vdevs, there is nothing to do */ if (l2arc_ndev == 0) goto out; first = NULL; next = l2arc_dev_last; do { /* loop around the list looking for a non-faulted vdev */ if (next == NULL) { next = list_head(l2arc_dev_list); } else { next = list_next(l2arc_dev_list, next); if (next == NULL) next = list_head(l2arc_dev_list); } /* if we have come back to the start, bail out */ if (first == NULL) first = next; else if (next == first) break; } while (vdev_is_dead(next->l2ad_vdev)); /* if we were unable to find any usable vdevs, return NULL */ if (vdev_is_dead(next->l2ad_vdev)) next = NULL; l2arc_dev_last = next; out: mutex_exit(&l2arc_dev_mtx); /* * Grab the config lock to prevent the 'next' device from being * removed while we are writing to it. */ if (next != NULL) spa_config_enter(next->l2ad_spa, SCL_L2ARC, next, RW_READER); mutex_exit(&spa_namespace_lock); return (next); } /* * Free buffers that were tagged for destruction. */ static void l2arc_do_free_on_write() { list_t *buflist; l2arc_data_free_t *df, *df_prev; mutex_enter(&l2arc_free_on_write_mtx); buflist = l2arc_free_on_write; for (df = list_tail(buflist); df; df = df_prev) { df_prev = list_prev(buflist, df); ASSERT(df->l2df_data != NULL); ASSERT(df->l2df_func != NULL); df->l2df_func(df->l2df_data, df->l2df_size); list_remove(buflist, df); kmem_free(df, sizeof (l2arc_data_free_t)); } mutex_exit(&l2arc_free_on_write_mtx); } /* * A write to a cache device has completed. Update all headers to allow * reads from these buffers to begin. */ static void l2arc_write_done(zio_t *zio) { l2arc_write_callback_t *cb; l2arc_dev_t *dev; list_t *buflist; arc_buf_hdr_t *head, *ab, *ab_prev; l2arc_buf_hdr_t *abl2; kmutex_t *hash_lock; int64_t bytes_dropped = 0; cb = zio->io_private; ASSERT(cb != NULL); dev = cb->l2wcb_dev; ASSERT(dev != NULL); head = cb->l2wcb_head; ASSERT(head != NULL); buflist = dev->l2ad_buflist; ASSERT(buflist != NULL); DTRACE_PROBE2(l2arc__iodone, zio_t *, zio, l2arc_write_callback_t *, cb); if (zio->io_error != 0) ARCSTAT_BUMP(arcstat_l2_writes_error); mutex_enter(&l2arc_buflist_mtx); /* * All writes completed, or an error was hit. */ for (ab = list_prev(buflist, head); ab; ab = ab_prev) { ab_prev = list_prev(buflist, ab); abl2 = ab->b_l2hdr; /* * Release the temporary compressed buffer as soon as possible. */ if (abl2->b_compress != ZIO_COMPRESS_OFF) l2arc_release_cdata_buf(ab); hash_lock = HDR_LOCK(ab); if (!mutex_tryenter(hash_lock)) { /* * This buffer misses out. It may be in a stage * of eviction. Its ARC_L2_WRITING flag will be * left set, denying reads to this buffer. */ ARCSTAT_BUMP(arcstat_l2_writes_hdr_miss); continue; } if (zio->io_error != 0) { /* * Error - drop L2ARC entry. */ list_remove(buflist, ab); ARCSTAT_INCR(arcstat_l2_asize, -abl2->b_asize); bytes_dropped += abl2->b_asize; ab->b_l2hdr = NULL; trim_map_free(abl2->b_dev->l2ad_vdev, abl2->b_daddr, ab->b_size, 0); kmem_free(abl2, sizeof (l2arc_buf_hdr_t)); ARCSTAT_INCR(arcstat_l2_size, -ab->b_size); } /* * Allow ARC to begin reads to this L2ARC entry. */ ab->b_flags &= ~ARC_L2_WRITING; mutex_exit(hash_lock); } atomic_inc_64(&l2arc_writes_done); list_remove(buflist, head); kmem_cache_free(hdr_cache, head); mutex_exit(&l2arc_buflist_mtx); vdev_space_update(dev->l2ad_vdev, -bytes_dropped, 0, 0); l2arc_do_free_on_write(); kmem_free(cb, sizeof (l2arc_write_callback_t)); } /* * A read to a cache device completed. Validate buffer contents before * handing over to the regular ARC routines. */ static void l2arc_read_done(zio_t *zio) { l2arc_read_callback_t *cb; arc_buf_hdr_t *hdr; arc_buf_t *buf; kmutex_t *hash_lock; int equal; ASSERT(zio->io_vd != NULL); ASSERT(zio->io_flags & ZIO_FLAG_DONT_PROPAGATE); spa_config_exit(zio->io_spa, SCL_L2ARC, zio->io_vd); cb = zio->io_private; ASSERT(cb != NULL); buf = cb->l2rcb_buf; ASSERT(buf != NULL); hash_lock = HDR_LOCK(buf->b_hdr); mutex_enter(hash_lock); hdr = buf->b_hdr; ASSERT3P(hash_lock, ==, HDR_LOCK(hdr)); /* * If the buffer was compressed, decompress it first. */ if (cb->l2rcb_compress != ZIO_COMPRESS_OFF) l2arc_decompress_zio(zio, hdr, cb->l2rcb_compress); ASSERT(zio->io_data != NULL); /* * Check this survived the L2ARC journey. */ equal = arc_cksum_equal(buf); if (equal && zio->io_error == 0 && !HDR_L2_EVICTED(hdr)) { mutex_exit(hash_lock); zio->io_private = buf; zio->io_bp_copy = cb->l2rcb_bp; /* XXX fix in L2ARC 2.0 */ zio->io_bp = &zio->io_bp_copy; /* XXX fix in L2ARC 2.0 */ arc_read_done(zio); } else { mutex_exit(hash_lock); /* * Buffer didn't survive caching. Increment stats and * reissue to the original storage device. */ if (zio->io_error != 0) { ARCSTAT_BUMP(arcstat_l2_io_error); } else { zio->io_error = SET_ERROR(EIO); } if (!equal) ARCSTAT_BUMP(arcstat_l2_cksum_bad); /* * If there's no waiter, issue an async i/o to the primary * storage now. If there *is* a waiter, the caller must * issue the i/o in a context where it's OK to block. */ if (zio->io_waiter == NULL) { zio_t *pio = zio_unique_parent(zio); ASSERT(!pio || pio->io_child_type == ZIO_CHILD_LOGICAL); zio_nowait(zio_read(pio, cb->l2rcb_spa, &cb->l2rcb_bp, buf->b_data, zio->io_size, arc_read_done, buf, zio->io_priority, cb->l2rcb_flags, &cb->l2rcb_zb)); } } kmem_free(cb, sizeof (l2arc_read_callback_t)); } /* * This is the list priority from which the L2ARC will search for pages to * cache. This is used within loops (0..3) to cycle through lists in the * desired order. This order can have a significant effect on cache * performance. * * Currently the metadata lists are hit first, MFU then MRU, followed by * the data lists. This function returns a locked list, and also returns * the lock pointer. */ static list_t * l2arc_list_locked(int list_num, kmutex_t **lock) { list_t *list = NULL; int idx; ASSERT(list_num >= 0 && list_num < 2 * ARC_BUFC_NUMLISTS); if (list_num < ARC_BUFC_NUMMETADATALISTS) { idx = list_num; list = &arc_mfu->arcs_lists[idx]; *lock = ARCS_LOCK(arc_mfu, idx); } else if (list_num < ARC_BUFC_NUMMETADATALISTS * 2) { idx = list_num - ARC_BUFC_NUMMETADATALISTS; list = &arc_mru->arcs_lists[idx]; *lock = ARCS_LOCK(arc_mru, idx); } else if (list_num < (ARC_BUFC_NUMMETADATALISTS * 2 + ARC_BUFC_NUMDATALISTS)) { idx = list_num - ARC_BUFC_NUMMETADATALISTS; list = &arc_mfu->arcs_lists[idx]; *lock = ARCS_LOCK(arc_mfu, idx); } else { idx = list_num - ARC_BUFC_NUMLISTS; list = &arc_mru->arcs_lists[idx]; *lock = ARCS_LOCK(arc_mru, idx); } ASSERT(!(MUTEX_HELD(*lock))); mutex_enter(*lock); return (list); } /* * Evict buffers from the device write hand to the distance specified in * bytes. This distance may span populated buffers, it may span nothing. * This is clearing a region on the L2ARC device ready for writing. * If the 'all' boolean is set, every buffer is evicted. */ static void l2arc_evict(l2arc_dev_t *dev, uint64_t distance, boolean_t all) { list_t *buflist; l2arc_buf_hdr_t *abl2; arc_buf_hdr_t *ab, *ab_prev; kmutex_t *hash_lock; uint64_t taddr; int64_t bytes_evicted = 0; buflist = dev->l2ad_buflist; if (buflist == NULL) return; if (!all && dev->l2ad_first) { /* * This is the first sweep through the device. There is * nothing to evict. */ return; } if (dev->l2ad_hand >= (dev->l2ad_end - (2 * distance))) { /* * When nearing the end of the device, evict to the end * before the device write hand jumps to the start. */ taddr = dev->l2ad_end; } else { taddr = dev->l2ad_hand + distance; } DTRACE_PROBE4(l2arc__evict, l2arc_dev_t *, dev, list_t *, buflist, uint64_t, taddr, boolean_t, all); top: mutex_enter(&l2arc_buflist_mtx); for (ab = list_tail(buflist); ab; ab = ab_prev) { ab_prev = list_prev(buflist, ab); hash_lock = HDR_LOCK(ab); if (!mutex_tryenter(hash_lock)) { /* * Missed the hash lock. Retry. */ ARCSTAT_BUMP(arcstat_l2_evict_lock_retry); mutex_exit(&l2arc_buflist_mtx); mutex_enter(hash_lock); mutex_exit(hash_lock); goto top; } if (HDR_L2_WRITE_HEAD(ab)) { /* * We hit a write head node. Leave it for * l2arc_write_done(). */ list_remove(buflist, ab); mutex_exit(hash_lock); continue; } if (!all && ab->b_l2hdr != NULL && (ab->b_l2hdr->b_daddr > taddr || ab->b_l2hdr->b_daddr < dev->l2ad_hand)) { /* * We've evicted to the target address, * or the end of the device. */ mutex_exit(hash_lock); break; } if (HDR_FREE_IN_PROGRESS(ab)) { /* * Already on the path to destruction. */ mutex_exit(hash_lock); continue; } if (ab->b_state == arc_l2c_only) { ASSERT(!HDR_L2_READING(ab)); /* * This doesn't exist in the ARC. Destroy. * arc_hdr_destroy() will call list_remove() * and decrement arcstat_l2_size. */ arc_change_state(arc_anon, ab, hash_lock); arc_hdr_destroy(ab); } else { /* * Invalidate issued or about to be issued * reads, since we may be about to write * over this location. */ if (HDR_L2_READING(ab)) { ARCSTAT_BUMP(arcstat_l2_evict_reading); ab->b_flags |= ARC_L2_EVICTED; } /* * Tell ARC this no longer exists in L2ARC. */ if (ab->b_l2hdr != NULL) { abl2 = ab->b_l2hdr; ARCSTAT_INCR(arcstat_l2_asize, -abl2->b_asize); bytes_evicted += abl2->b_asize; ab->b_l2hdr = NULL; + /* + * We are destroying l2hdr, so ensure that + * its compressed buffer, if any, is not leaked. + */ + ASSERT(abl2->b_tmp_cdata == NULL); kmem_free(abl2, sizeof (l2arc_buf_hdr_t)); ARCSTAT_INCR(arcstat_l2_size, -ab->b_size); } list_remove(buflist, ab); /* * This may have been leftover after a * failed write. */ ab->b_flags &= ~ARC_L2_WRITING; } mutex_exit(hash_lock); } mutex_exit(&l2arc_buflist_mtx); vdev_space_update(dev->l2ad_vdev, -bytes_evicted, 0, 0); dev->l2ad_evict = taddr; } /* * Find and write ARC buffers to the L2ARC device. * * An ARC_L2_WRITING flag is set so that the L2ARC buffers are not valid * for reading until they have completed writing. * The headroom_boost is an in-out parameter used to maintain headroom boost * state between calls to this function. * * Returns the number of bytes actually written (which may be smaller than * the delta by which the device hand has changed due to alignment). */ static uint64_t l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, uint64_t target_sz, boolean_t *headroom_boost) { arc_buf_hdr_t *ab, *ab_prev, *head; list_t *list; uint64_t write_asize, write_psize, write_sz, headroom, buf_compress_minsz; void *buf_data; kmutex_t *list_lock; boolean_t full; l2arc_write_callback_t *cb; zio_t *pio, *wzio; uint64_t guid = spa_load_guid(spa); const boolean_t do_headroom_boost = *headroom_boost; int try; ASSERT(dev->l2ad_vdev != NULL); /* Lower the flag now, we might want to raise it again later. */ *headroom_boost = B_FALSE; pio = NULL; write_sz = write_asize = write_psize = 0; full = B_FALSE; head = kmem_cache_alloc(hdr_cache, KM_PUSHPAGE); head->b_flags |= ARC_L2_WRITE_HEAD; ARCSTAT_BUMP(arcstat_l2_write_buffer_iter); /* * We will want to try to compress buffers that are at least 2x the * device sector size. */ buf_compress_minsz = 2 << dev->l2ad_vdev->vdev_ashift; /* * Copy buffers for L2ARC writing. */ mutex_enter(&l2arc_buflist_mtx); for (try = 0; try < 2 * ARC_BUFC_NUMLISTS; try++) { uint64_t passed_sz = 0; list = l2arc_list_locked(try, &list_lock); ARCSTAT_BUMP(arcstat_l2_write_buffer_list_iter); /* * L2ARC fast warmup. * * Until the ARC is warm and starts to evict, read from the * head of the ARC lists rather than the tail. */ if (arc_warm == B_FALSE) ab = list_head(list); else ab = list_tail(list); if (ab == NULL) ARCSTAT_BUMP(arcstat_l2_write_buffer_list_null_iter); headroom = target_sz * l2arc_headroom; if (do_headroom_boost) headroom = (headroom * l2arc_headroom_boost) / 100; for (; ab; ab = ab_prev) { l2arc_buf_hdr_t *l2hdr; kmutex_t *hash_lock; uint64_t buf_sz; if (arc_warm == B_FALSE) ab_prev = list_next(list, ab); else ab_prev = list_prev(list, ab); ARCSTAT_INCR(arcstat_l2_write_buffer_bytes_scanned, ab->b_size); hash_lock = HDR_LOCK(ab); if (!mutex_tryenter(hash_lock)) { ARCSTAT_BUMP(arcstat_l2_write_trylock_fail); /* * Skip this buffer rather than waiting. */ continue; } passed_sz += ab->b_size; if (passed_sz > headroom) { /* * Searched too far. */ mutex_exit(hash_lock); ARCSTAT_BUMP(arcstat_l2_write_passed_headroom); break; } if (!l2arc_write_eligible(guid, ab)) { mutex_exit(hash_lock); continue; } if ((write_sz + ab->b_size) > target_sz) { full = B_TRUE; mutex_exit(hash_lock); ARCSTAT_BUMP(arcstat_l2_write_full); break; } if (pio == NULL) { /* * Insert a dummy header on the buflist so * l2arc_write_done() can find where the * write buffers begin without searching. */ list_insert_head(dev->l2ad_buflist, head); cb = kmem_alloc( sizeof (l2arc_write_callback_t), KM_SLEEP); cb->l2wcb_dev = dev; cb->l2wcb_head = head; pio = zio_root(spa, l2arc_write_done, cb, ZIO_FLAG_CANFAIL); ARCSTAT_BUMP(arcstat_l2_write_pios); } /* * Create and add a new L2ARC header. */ l2hdr = kmem_zalloc(sizeof (l2arc_buf_hdr_t), KM_SLEEP); l2hdr->b_dev = dev; ab->b_flags |= ARC_L2_WRITING; /* * Temporarily stash the data buffer in b_tmp_cdata. * The subsequent write step will pick it up from * there. This is because can't access ab->b_buf * without holding the hash_lock, which we in turn * can't access without holding the ARC list locks * (which we want to avoid during compression/writing). */ l2hdr->b_compress = ZIO_COMPRESS_OFF; l2hdr->b_asize = ab->b_size; l2hdr->b_tmp_cdata = ab->b_buf->b_data; buf_sz = ab->b_size; ab->b_l2hdr = l2hdr; list_insert_head(dev->l2ad_buflist, ab); /* * Compute and store the buffer cksum before * writing. On debug the cksum is verified first. */ arc_cksum_verify(ab->b_buf); arc_cksum_compute(ab->b_buf, B_TRUE); mutex_exit(hash_lock); write_sz += buf_sz; } mutex_exit(list_lock); if (full == B_TRUE) break; } /* No buffers selected for writing? */ if (pio == NULL) { ASSERT0(write_sz); mutex_exit(&l2arc_buflist_mtx); kmem_cache_free(hdr_cache, head); return (0); } /* * Now start writing the buffers. We're starting at the write head * and work backwards, retracing the course of the buffer selector * loop above. */ for (ab = list_prev(dev->l2ad_buflist, head); ab; ab = list_prev(dev->l2ad_buflist, ab)) { l2arc_buf_hdr_t *l2hdr; uint64_t buf_sz; /* * We shouldn't need to lock the buffer here, since we flagged * it as ARC_L2_WRITING in the previous step, but we must take * care to only access its L2 cache parameters. In particular, * ab->b_buf may be invalid by now due to ARC eviction. */ l2hdr = ab->b_l2hdr; l2hdr->b_daddr = dev->l2ad_hand; if ((ab->b_flags & ARC_L2COMPRESS) && l2hdr->b_asize >= buf_compress_minsz) { if (l2arc_compress_buf(l2hdr)) { /* * If compression succeeded, enable headroom * boost on the next scan cycle. */ *headroom_boost = B_TRUE; } } /* * Pick up the buffer data we had previously stashed away * (and now potentially also compressed). */ buf_data = l2hdr->b_tmp_cdata; buf_sz = l2hdr->b_asize; + /* + * If the data has not been compressed, then clear b_tmp_cdata + * to make sure that it points only to a temporary compression + * buffer. + */ + if (!L2ARC_IS_VALID_COMPRESS(l2hdr->b_compress)) + l2hdr->b_tmp_cdata = NULL; + /* Compression may have squashed the buffer to zero length. */ if (buf_sz != 0) { uint64_t buf_p_sz; wzio = zio_write_phys(pio, dev->l2ad_vdev, dev->l2ad_hand, buf_sz, buf_data, ZIO_CHECKSUM_OFF, NULL, NULL, ZIO_PRIORITY_ASYNC_WRITE, ZIO_FLAG_CANFAIL, B_FALSE); DTRACE_PROBE2(l2arc__write, vdev_t *, dev->l2ad_vdev, zio_t *, wzio); (void) zio_nowait(wzio); write_asize += buf_sz; /* * Keep the clock hand suitably device-aligned. */ buf_p_sz = vdev_psize_to_asize(dev->l2ad_vdev, buf_sz); write_psize += buf_p_sz; dev->l2ad_hand += buf_p_sz; } } mutex_exit(&l2arc_buflist_mtx); ASSERT3U(write_asize, <=, target_sz); ARCSTAT_BUMP(arcstat_l2_writes_sent); ARCSTAT_INCR(arcstat_l2_write_bytes, write_asize); ARCSTAT_INCR(arcstat_l2_size, write_sz); ARCSTAT_INCR(arcstat_l2_asize, write_asize); vdev_space_update(dev->l2ad_vdev, write_psize, 0, 0); /* * Bump device hand to the device start if it is approaching the end. * l2arc_evict() will already have evicted ahead for this case. */ if (dev->l2ad_hand >= (dev->l2ad_end - target_sz)) { dev->l2ad_hand = dev->l2ad_start; dev->l2ad_evict = dev->l2ad_start; dev->l2ad_first = B_FALSE; } dev->l2ad_writing = B_TRUE; (void) zio_wait(pio); dev->l2ad_writing = B_FALSE; return (write_asize); } /* * Compresses an L2ARC buffer. * The data to be compressed must be prefilled in l2hdr->b_tmp_cdata and its * size in l2hdr->b_asize. This routine tries to compress the data and * depending on the compression result there are three possible outcomes: * *) The buffer was incompressible. The original l2hdr contents were left * untouched and are ready for writing to an L2 device. * *) The buffer was all-zeros, so there is no need to write it to an L2 * device. To indicate this situation b_tmp_cdata is NULL'ed, b_asize is * set to zero and b_compress is set to ZIO_COMPRESS_EMPTY. * *) Compression succeeded and b_tmp_cdata was replaced with a temporary * data buffer which holds the compressed data to be written, and b_asize * tells us how much data there is. b_compress is set to the appropriate * compression algorithm. Once writing is done, invoke * l2arc_release_cdata_buf on this l2hdr to free this temporary buffer. * * Returns B_TRUE if compression succeeded, or B_FALSE if it didn't (the * buffer was incompressible). */ static boolean_t l2arc_compress_buf(l2arc_buf_hdr_t *l2hdr) { void *cdata; size_t csize, len, rounded; ASSERT(l2hdr->b_compress == ZIO_COMPRESS_OFF); ASSERT(l2hdr->b_tmp_cdata != NULL); len = l2hdr->b_asize; cdata = zio_data_buf_alloc(len); csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, cdata, l2hdr->b_asize); rounded = P2ROUNDUP(csize, (size_t)SPA_MINBLOCKSIZE); if (rounded > csize) { bzero((char *)cdata + csize, rounded - csize); csize = rounded; } if (csize == 0) { /* zero block, indicate that there's nothing to write */ zio_data_buf_free(cdata, len); l2hdr->b_compress = ZIO_COMPRESS_EMPTY; l2hdr->b_asize = 0; l2hdr->b_tmp_cdata = NULL; ARCSTAT_BUMP(arcstat_l2_compress_zeros); return (B_TRUE); } else if (csize > 0 && csize < len) { /* * Compression succeeded, we'll keep the cdata around for * writing and release it afterwards. */ l2hdr->b_compress = ZIO_COMPRESS_LZ4; l2hdr->b_asize = csize; l2hdr->b_tmp_cdata = cdata; ARCSTAT_BUMP(arcstat_l2_compress_successes); return (B_TRUE); } else { /* * Compression failed, release the compressed buffer. * l2hdr will be left unmodified. */ zio_data_buf_free(cdata, len); ARCSTAT_BUMP(arcstat_l2_compress_failures); return (B_FALSE); } } /* * Decompresses a zio read back from an l2arc device. On success, the * underlying zio's io_data buffer is overwritten by the uncompressed * version. On decompression error (corrupt compressed stream), the * zio->io_error value is set to signal an I/O error. * * Please note that the compressed data stream is not checksummed, so * if the underlying device is experiencing data corruption, we may feed * corrupt data to the decompressor, so the decompressor needs to be * able to handle this situation (LZ4 does). */ static void l2arc_decompress_zio(zio_t *zio, arc_buf_hdr_t *hdr, enum zio_compress c) { ASSERT(L2ARC_IS_VALID_COMPRESS(c)); if (zio->io_error != 0) { /* * An io error has occured, just restore the original io * size in preparation for a main pool read. */ zio->io_orig_size = zio->io_size = hdr->b_size; return; } if (c == ZIO_COMPRESS_EMPTY) { /* * An empty buffer results in a null zio, which means we * need to fill its io_data after we're done restoring the * buffer's contents. */ ASSERT(hdr->b_buf != NULL); bzero(hdr->b_buf->b_data, hdr->b_size); zio->io_data = zio->io_orig_data = hdr->b_buf->b_data; } else { ASSERT(zio->io_data != NULL); /* * We copy the compressed data from the start of the arc buffer * (the zio_read will have pulled in only what we need, the * rest is garbage which we will overwrite at decompression) * and then decompress back to the ARC data buffer. This way we * can minimize copying by simply decompressing back over the * original compressed data (rather than decompressing to an * aux buffer and then copying back the uncompressed buffer, * which is likely to be much larger). */ uint64_t csize; void *cdata; csize = zio->io_size; cdata = zio_data_buf_alloc(csize); bcopy(zio->io_data, cdata, csize); if (zio_decompress_data(c, cdata, zio->io_data, csize, hdr->b_size) != 0) zio->io_error = EIO; zio_data_buf_free(cdata, csize); } /* Restore the expected uncompressed IO size. */ zio->io_orig_size = zio->io_size = hdr->b_size; } /* * Releases the temporary b_tmp_cdata buffer in an l2arc header structure. * This buffer serves as a temporary holder of compressed data while * the buffer entry is being written to an l2arc device. Once that is * done, we can dispose of it. */ static void l2arc_release_cdata_buf(arc_buf_hdr_t *ab) { l2arc_buf_hdr_t *l2hdr = ab->b_l2hdr; - if (l2hdr->b_compress == ZIO_COMPRESS_LZ4) { + ASSERT(L2ARC_IS_VALID_COMPRESS(l2hdr->b_compress)); + if (l2hdr->b_compress != ZIO_COMPRESS_EMPTY) { /* * If the data was compressed, then we've allocated a * temporary buffer for it, so now we need to release it. */ ASSERT(l2hdr->b_tmp_cdata != NULL); zio_data_buf_free(l2hdr->b_tmp_cdata, ab->b_size); + l2hdr->b_tmp_cdata = NULL; + } else { + ASSERT(l2hdr->b_tmp_cdata == NULL); } - l2hdr->b_tmp_cdata = NULL; } /* * This thread feeds the L2ARC at regular intervals. This is the beating * heart of the L2ARC. */ static void l2arc_feed_thread(void *dummy __unused) { callb_cpr_t cpr; l2arc_dev_t *dev; spa_t *spa; uint64_t size, wrote; clock_t begin, next = ddi_get_lbolt(); boolean_t headroom_boost = B_FALSE; CALLB_CPR_INIT(&cpr, &l2arc_feed_thr_lock, callb_generic_cpr, FTAG); mutex_enter(&l2arc_feed_thr_lock); while (l2arc_thread_exit == 0) { CALLB_CPR_SAFE_BEGIN(&cpr); (void) cv_timedwait(&l2arc_feed_thr_cv, &l2arc_feed_thr_lock, next - ddi_get_lbolt()); CALLB_CPR_SAFE_END(&cpr, &l2arc_feed_thr_lock); next = ddi_get_lbolt() + hz; /* * Quick check for L2ARC devices. */ mutex_enter(&l2arc_dev_mtx); if (l2arc_ndev == 0) { mutex_exit(&l2arc_dev_mtx); continue; } mutex_exit(&l2arc_dev_mtx); begin = ddi_get_lbolt(); /* * This selects the next l2arc device to write to, and in * doing so the next spa to feed from: dev->l2ad_spa. This * will return NULL if there are now no l2arc devices or if * they are all faulted. * * If a device is returned, its spa's config lock is also * held to prevent device removal. l2arc_dev_get_next() * will grab and release l2arc_dev_mtx. */ if ((dev = l2arc_dev_get_next()) == NULL) continue; spa = dev->l2ad_spa; ASSERT(spa != NULL); /* * If the pool is read-only then force the feed thread to * sleep a little longer. */ if (!spa_writeable(spa)) { next = ddi_get_lbolt() + 5 * l2arc_feed_secs * hz; spa_config_exit(spa, SCL_L2ARC, dev); continue; } /* * Avoid contributing to memory pressure. */ if (arc_reclaim_needed()) { ARCSTAT_BUMP(arcstat_l2_abort_lowmem); spa_config_exit(spa, SCL_L2ARC, dev); continue; } ARCSTAT_BUMP(arcstat_l2_feeds); size = l2arc_write_size(); /* * Evict L2ARC buffers that will be overwritten. */ l2arc_evict(dev, size, B_FALSE); /* * Write ARC buffers. */ wrote = l2arc_write_buffers(spa, dev, size, &headroom_boost); /* * Calculate interval between writes. */ next = l2arc_write_interval(begin, size, wrote); spa_config_exit(spa, SCL_L2ARC, dev); } l2arc_thread_exit = 0; cv_broadcast(&l2arc_feed_thr_cv); CALLB_CPR_EXIT(&cpr); /* drops l2arc_feed_thr_lock */ thread_exit(); } boolean_t l2arc_vdev_present(vdev_t *vd) { l2arc_dev_t *dev; mutex_enter(&l2arc_dev_mtx); for (dev = list_head(l2arc_dev_list); dev != NULL; dev = list_next(l2arc_dev_list, dev)) { if (dev->l2ad_vdev == vd) break; } mutex_exit(&l2arc_dev_mtx); return (dev != NULL); } /* * Add a vdev for use by the L2ARC. By this point the spa has already * validated the vdev and opened it. */ void l2arc_add_vdev(spa_t *spa, vdev_t *vd) { l2arc_dev_t *adddev; ASSERT(!l2arc_vdev_present(vd)); vdev_ashift_optimize(vd); /* * Create a new l2arc device entry. */ adddev = kmem_zalloc(sizeof (l2arc_dev_t), KM_SLEEP); adddev->l2ad_spa = spa; adddev->l2ad_vdev = vd; adddev->l2ad_start = VDEV_LABEL_START_SIZE; adddev->l2ad_end = VDEV_LABEL_START_SIZE + vdev_get_min_asize(vd); adddev->l2ad_hand = adddev->l2ad_start; adddev->l2ad_evict = adddev->l2ad_start; adddev->l2ad_first = B_TRUE; adddev->l2ad_writing = B_FALSE; /* * This is a list of all ARC buffers that are still valid on the * device. */ adddev->l2ad_buflist = kmem_zalloc(sizeof (list_t), KM_SLEEP); list_create(adddev->l2ad_buflist, sizeof (arc_buf_hdr_t), offsetof(arc_buf_hdr_t, b_l2node)); vdev_space_update(vd, 0, 0, adddev->l2ad_end - adddev->l2ad_hand); /* * Add device to global list */ mutex_enter(&l2arc_dev_mtx); list_insert_head(l2arc_dev_list, adddev); atomic_inc_64(&l2arc_ndev); mutex_exit(&l2arc_dev_mtx); } /* * Remove a vdev from the L2ARC. */ void l2arc_remove_vdev(vdev_t *vd) { l2arc_dev_t *dev, *nextdev, *remdev = NULL; /* * Find the device by vdev */ mutex_enter(&l2arc_dev_mtx); for (dev = list_head(l2arc_dev_list); dev; dev = nextdev) { nextdev = list_next(l2arc_dev_list, dev); if (vd == dev->l2ad_vdev) { remdev = dev; break; } } ASSERT(remdev != NULL); /* * Remove device from global list */ list_remove(l2arc_dev_list, remdev); l2arc_dev_last = NULL; /* may have been invalidated */ atomic_dec_64(&l2arc_ndev); mutex_exit(&l2arc_dev_mtx); /* * Clear all buflists and ARC references. L2ARC device flush. */ l2arc_evict(remdev, 0, B_TRUE); list_destroy(remdev->l2ad_buflist); kmem_free(remdev->l2ad_buflist, sizeof (list_t)); kmem_free(remdev, sizeof (l2arc_dev_t)); } void l2arc_init(void) { l2arc_thread_exit = 0; l2arc_ndev = 0; l2arc_writes_sent = 0; l2arc_writes_done = 0; mutex_init(&l2arc_feed_thr_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&l2arc_feed_thr_cv, NULL, CV_DEFAULT, NULL); mutex_init(&l2arc_dev_mtx, NULL, MUTEX_DEFAULT, NULL); mutex_init(&l2arc_buflist_mtx, NULL, MUTEX_DEFAULT, NULL); mutex_init(&l2arc_free_on_write_mtx, NULL, MUTEX_DEFAULT, NULL); l2arc_dev_list = &L2ARC_dev_list; l2arc_free_on_write = &L2ARC_free_on_write; list_create(l2arc_dev_list, sizeof (l2arc_dev_t), offsetof(l2arc_dev_t, l2ad_node)); list_create(l2arc_free_on_write, sizeof (l2arc_data_free_t), offsetof(l2arc_data_free_t, l2df_list_node)); } void l2arc_fini(void) { /* * This is called from dmu_fini(), which is called from spa_fini(); * Because of this, we can assume that all l2arc devices have * already been removed when the pools themselves were removed. */ l2arc_do_free_on_write(); mutex_destroy(&l2arc_feed_thr_lock); cv_destroy(&l2arc_feed_thr_cv); mutex_destroy(&l2arc_dev_mtx); mutex_destroy(&l2arc_buflist_mtx); mutex_destroy(&l2arc_free_on_write_mtx); list_destroy(l2arc_dev_list); list_destroy(l2arc_free_on_write); } void l2arc_start(void) { if (!(spa_mode_global & FWRITE)) return; (void) thread_create(NULL, 0, l2arc_feed_thread, NULL, 0, &p0, TS_RUN, minclsyspri); } void l2arc_stop(void) { if (!(spa_mode_global & FWRITE)) return; mutex_enter(&l2arc_feed_thr_lock); cv_signal(&l2arc_feed_thr_cv); /* kick thread out of startup */ l2arc_thread_exit = 1; while (l2arc_thread_exit != 0) cv_wait(&l2arc_feed_thr_cv, &l2arc_feed_thr_lock); mutex_exit(&l2arc_feed_thr_lock); } Index: releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c =================================================================== --- releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c (revision 284192) +++ releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c (revision 284193) @@ -1,649 +1,647 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright (c) 2012 Pawel Jakub Dawidek . * All rights reserved. */ #include #include #include #include #include /* * Calculate the zio end, upgrading based on ashift which would be * done by zio_vdev_io_start. * * This makes free range consolidation much more effective * than it would otherwise be as well as ensuring that entire * blocks are invalidated by writes. */ #define TRIM_ZIO_END(vd, offset, size) (offset + \ P2ROUNDUP(size, 1ULL << vd->vdev_top->vdev_ashift)) #define TRIM_MAP_SINC(tm, size) \ atomic_add_64(&(tm)->tm_bytes, (size)) #define TRIM_MAP_SDEC(tm, size) \ atomic_add_64(&(tm)->tm_bytes, -(size)) #define TRIM_MAP_QINC(tm) \ atomic_inc_64(&(tm)->tm_pending); \ #define TRIM_MAP_QDEC(tm) \ atomic_dec_64(&(tm)->tm_pending); typedef struct trim_map { list_t tm_head; /* List of segments sorted by txg. */ avl_tree_t tm_queued_frees; /* AVL tree of segments waiting for TRIM. */ avl_tree_t tm_inflight_frees; /* AVL tree of in-flight TRIMs. */ avl_tree_t tm_inflight_writes; /* AVL tree of in-flight writes. */ list_t tm_pending_writes; /* Writes blocked on in-flight frees. */ kmutex_t tm_lock; uint64_t tm_pending; /* Count of pending TRIMs. */ uint64_t tm_bytes; /* Total size in bytes of queued TRIMs. */ } trim_map_t; typedef struct trim_seg { avl_node_t ts_node; /* AVL node. */ list_node_t ts_next; /* List element. */ uint64_t ts_start; /* Starting offset of this segment. */ uint64_t ts_end; /* Ending offset (non-inclusive). */ uint64_t ts_txg; /* Segment creation txg. */ hrtime_t ts_time; /* Segment creation time. */ } trim_seg_t; extern boolean_t zfs_trim_enabled; static u_int trim_txg_delay = 32; static u_int trim_timeout = 30; static u_int trim_max_interval = 1; /* Limit outstanding TRIMs to 2G (max size for a single TRIM request) */ static uint64_t trim_vdev_max_bytes = 2147483648; /* Limit outstanding TRIMs to 64 (max ranges for a single TRIM request) */ static u_int trim_vdev_max_pending = 64; SYSCTL_DECL(_vfs_zfs); SYSCTL_NODE(_vfs_zfs, OID_AUTO, trim, CTLFLAG_RD, 0, "ZFS TRIM"); TUNABLE_INT("vfs.zfs.trim.txg_delay", &trim_txg_delay); SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, txg_delay, CTLFLAG_RWTUN, &trim_txg_delay, 0, "Delay TRIMs by up to this many TXGs"); TUNABLE_INT("vfs.zfs.trim.timeout", &trim_timeout); SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, timeout, CTLFLAG_RWTUN, &trim_timeout, 0, "Delay TRIMs by up to this many seconds"); TUNABLE_INT("vfs.zfs.trim.max_interval", &trim_max_interval); SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, max_interval, CTLFLAG_RWTUN, &trim_max_interval, 0, "Maximum interval between TRIM queue processing (seconds)"); SYSCTL_DECL(_vfs_zfs_vdev); TUNABLE_QUAD("vfs.zfs.vdev.trim_max_bytes", &trim_vdev_max_bytes); SYSCTL_QUAD(_vfs_zfs_vdev, OID_AUTO, trim_max_bytes, CTLFLAG_RWTUN, &trim_vdev_max_bytes, 0, "Maximum pending TRIM bytes for a vdev"); TUNABLE_INT("vfs.zfs.vdev.trim_max_pending", &trim_vdev_max_pending); SYSCTL_UINT(_vfs_zfs_vdev, OID_AUTO, trim_max_pending, CTLFLAG_RWTUN, &trim_vdev_max_pending, 0, "Maximum pending TRIM segments for a vdev"); static void trim_map_vdev_commit_done(spa_t *spa, vdev_t *vd); static int trim_map_seg_compare(const void *x1, const void *x2) { const trim_seg_t *s1 = x1; const trim_seg_t *s2 = x2; if (s1->ts_start < s2->ts_start) { if (s1->ts_end > s2->ts_start) return (0); return (-1); } if (s1->ts_start > s2->ts_start) { if (s1->ts_start < s2->ts_end) return (0); return (1); } return (0); } static int trim_map_zio_compare(const void *x1, const void *x2) { const zio_t *z1 = x1; const zio_t *z2 = x2; if (z1->io_offset < z2->io_offset) { if (z1->io_offset + z1->io_size > z2->io_offset) return (0); return (-1); } if (z1->io_offset > z2->io_offset) { if (z1->io_offset < z2->io_offset + z2->io_size) return (0); return (1); } return (0); } void trim_map_create(vdev_t *vd) { trim_map_t *tm; - ASSERT(vd->vdev_ops->vdev_op_leaf); - - if (!zfs_trim_enabled) - return; + ASSERT(zfs_trim_enabled && !vd->vdev_notrim && + vd->vdev_ops->vdev_op_leaf); tm = kmem_zalloc(sizeof (*tm), KM_SLEEP); mutex_init(&tm->tm_lock, NULL, MUTEX_DEFAULT, NULL); list_create(&tm->tm_head, sizeof (trim_seg_t), offsetof(trim_seg_t, ts_next)); list_create(&tm->tm_pending_writes, sizeof (zio_t), offsetof(zio_t, io_trim_link)); avl_create(&tm->tm_queued_frees, trim_map_seg_compare, sizeof (trim_seg_t), offsetof(trim_seg_t, ts_node)); avl_create(&tm->tm_inflight_frees, trim_map_seg_compare, sizeof (trim_seg_t), offsetof(trim_seg_t, ts_node)); avl_create(&tm->tm_inflight_writes, trim_map_zio_compare, sizeof (zio_t), offsetof(zio_t, io_trim_node)); vd->vdev_trimmap = tm; } void trim_map_destroy(vdev_t *vd) { trim_map_t *tm; trim_seg_t *ts; ASSERT(vd->vdev_ops->vdev_op_leaf); if (!zfs_trim_enabled) return; tm = vd->vdev_trimmap; if (tm == NULL) return; /* * We may have been called before trim_map_vdev_commit_done() * had a chance to run, so do it now to prune the remaining * inflight frees. */ trim_map_vdev_commit_done(vd->vdev_spa, vd); mutex_enter(&tm->tm_lock); while ((ts = list_head(&tm->tm_head)) != NULL) { avl_remove(&tm->tm_queued_frees, ts); list_remove(&tm->tm_head, ts); kmem_free(ts, sizeof (*ts)); TRIM_MAP_SDEC(tm, ts->ts_end - ts->ts_start); TRIM_MAP_QDEC(tm); } mutex_exit(&tm->tm_lock); avl_destroy(&tm->tm_queued_frees); avl_destroy(&tm->tm_inflight_frees); avl_destroy(&tm->tm_inflight_writes); list_destroy(&tm->tm_pending_writes); list_destroy(&tm->tm_head); mutex_destroy(&tm->tm_lock); kmem_free(tm, sizeof (*tm)); vd->vdev_trimmap = NULL; } static void trim_map_segment_add(trim_map_t *tm, uint64_t start, uint64_t end, uint64_t txg) { avl_index_t where; trim_seg_t tsearch, *ts_before, *ts_after, *ts; boolean_t merge_before, merge_after; hrtime_t time; ASSERT(MUTEX_HELD(&tm->tm_lock)); VERIFY(start < end); time = gethrtime(); tsearch.ts_start = start; tsearch.ts_end = end; ts = avl_find(&tm->tm_queued_frees, &tsearch, &where); if (ts != NULL) { if (start < ts->ts_start) trim_map_segment_add(tm, start, ts->ts_start, txg); if (end > ts->ts_end) trim_map_segment_add(tm, ts->ts_end, end, txg); return; } ts_before = avl_nearest(&tm->tm_queued_frees, where, AVL_BEFORE); ts_after = avl_nearest(&tm->tm_queued_frees, where, AVL_AFTER); merge_before = (ts_before != NULL && ts_before->ts_end == start); merge_after = (ts_after != NULL && ts_after->ts_start == end); if (merge_before && merge_after) { TRIM_MAP_SINC(tm, ts_after->ts_start - ts_before->ts_end); TRIM_MAP_QDEC(tm); avl_remove(&tm->tm_queued_frees, ts_before); list_remove(&tm->tm_head, ts_before); ts_after->ts_start = ts_before->ts_start; ts_after->ts_txg = txg; ts_after->ts_time = time; kmem_free(ts_before, sizeof (*ts_before)); } else if (merge_before) { TRIM_MAP_SINC(tm, end - ts_before->ts_end); ts_before->ts_end = end; ts_before->ts_txg = txg; ts_before->ts_time = time; } else if (merge_after) { TRIM_MAP_SINC(tm, ts_after->ts_start - start); ts_after->ts_start = start; ts_after->ts_txg = txg; ts_after->ts_time = time; } else { TRIM_MAP_SINC(tm, end - start); TRIM_MAP_QINC(tm); ts = kmem_alloc(sizeof (*ts), KM_SLEEP); ts->ts_start = start; ts->ts_end = end; ts->ts_txg = txg; ts->ts_time = time; avl_insert(&tm->tm_queued_frees, ts, where); list_insert_tail(&tm->tm_head, ts); } } static void trim_map_segment_remove(trim_map_t *tm, trim_seg_t *ts, uint64_t start, uint64_t end) { trim_seg_t *nts; boolean_t left_over, right_over; ASSERT(MUTEX_HELD(&tm->tm_lock)); left_over = (ts->ts_start < start); right_over = (ts->ts_end > end); TRIM_MAP_SDEC(tm, end - start); if (left_over && right_over) { nts = kmem_alloc(sizeof (*nts), KM_SLEEP); nts->ts_start = end; nts->ts_end = ts->ts_end; nts->ts_txg = ts->ts_txg; nts->ts_time = ts->ts_time; ts->ts_end = start; avl_insert_here(&tm->tm_queued_frees, nts, ts, AVL_AFTER); list_insert_after(&tm->tm_head, ts, nts); TRIM_MAP_QINC(tm); } else if (left_over) { ts->ts_end = start; } else if (right_over) { ts->ts_start = end; } else { avl_remove(&tm->tm_queued_frees, ts); list_remove(&tm->tm_head, ts); TRIM_MAP_QDEC(tm); kmem_free(ts, sizeof (*ts)); } } static void trim_map_free_locked(trim_map_t *tm, uint64_t start, uint64_t end, uint64_t txg) { zio_t zsearch, *zs; ASSERT(MUTEX_HELD(&tm->tm_lock)); zsearch.io_offset = start; zsearch.io_size = end - start; zs = avl_find(&tm->tm_inflight_writes, &zsearch, NULL); if (zs == NULL) { trim_map_segment_add(tm, start, end, txg); return; } if (start < zs->io_offset) trim_map_free_locked(tm, start, zs->io_offset, txg); if (zs->io_offset + zs->io_size < end) trim_map_free_locked(tm, zs->io_offset + zs->io_size, end, txg); } void trim_map_free(vdev_t *vd, uint64_t offset, uint64_t size, uint64_t txg) { trim_map_t *tm = vd->vdev_trimmap; if (!zfs_trim_enabled || vd->vdev_notrim || tm == NULL) return; mutex_enter(&tm->tm_lock); trim_map_free_locked(tm, offset, TRIM_ZIO_END(vd, offset, size), txg); mutex_exit(&tm->tm_lock); } boolean_t trim_map_write_start(zio_t *zio) { vdev_t *vd = zio->io_vd; trim_map_t *tm = vd->vdev_trimmap; trim_seg_t tsearch, *ts; boolean_t left_over, right_over; uint64_t start, end; if (!zfs_trim_enabled || vd->vdev_notrim || tm == NULL) return (B_TRUE); start = zio->io_offset; end = TRIM_ZIO_END(zio->io_vd, start, zio->io_size); tsearch.ts_start = start; tsearch.ts_end = end; mutex_enter(&tm->tm_lock); /* * Checking for colliding in-flight frees. */ ts = avl_find(&tm->tm_inflight_frees, &tsearch, NULL); if (ts != NULL) { list_insert_tail(&tm->tm_pending_writes, zio); mutex_exit(&tm->tm_lock); return (B_FALSE); } ts = avl_find(&tm->tm_queued_frees, &tsearch, NULL); if (ts != NULL) { /* * Loop until all overlapping segments are removed. */ do { trim_map_segment_remove(tm, ts, start, end); ts = avl_find(&tm->tm_queued_frees, &tsearch, NULL); } while (ts != NULL); } avl_add(&tm->tm_inflight_writes, zio); mutex_exit(&tm->tm_lock); return (B_TRUE); } void trim_map_write_done(zio_t *zio) { vdev_t *vd = zio->io_vd; trim_map_t *tm = vd->vdev_trimmap; /* * Don't check for vdev_notrim, since the write could have * started before vdev_notrim was set. */ if (!zfs_trim_enabled || tm == NULL) return; mutex_enter(&tm->tm_lock); /* * Don't fail if the write isn't in the tree, since the write * could have started after vdev_notrim was set. */ if (zio->io_trim_node.avl_child[0] || zio->io_trim_node.avl_child[1] || AVL_XPARENT(&zio->io_trim_node) || tm->tm_inflight_writes.avl_root == &zio->io_trim_node) avl_remove(&tm->tm_inflight_writes, zio); mutex_exit(&tm->tm_lock); } /* * Return the oldest segment (the one with the lowest txg / time) or NULL if: * 1. The list is empty * 2. The first element's txg is greater than txgsafe * 3. The first element's txg is not greater than the txg argument and the * the first element's time is not greater than time argument */ static trim_seg_t * trim_map_first(trim_map_t *tm, uint64_t txg, uint64_t txgsafe, hrtime_t time) { trim_seg_t *ts; ASSERT(MUTEX_HELD(&tm->tm_lock)); VERIFY(txgsafe >= txg); ts = list_head(&tm->tm_head); if (ts != NULL && ts->ts_txg <= txgsafe && (ts->ts_txg <= txg || ts->ts_time <= time || tm->tm_bytes > trim_vdev_max_bytes || tm->tm_pending > trim_vdev_max_pending)) return (ts); return (NULL); } static void trim_map_vdev_commit(spa_t *spa, zio_t *zio, vdev_t *vd) { trim_map_t *tm = vd->vdev_trimmap; trim_seg_t *ts; uint64_t size, offset, txgtarget, txgsafe; hrtime_t timelimit; ASSERT(vd->vdev_ops->vdev_op_leaf); if (tm == NULL) return; timelimit = gethrtime() - trim_timeout * NANOSEC; if (vd->vdev_isl2cache) { txgsafe = UINT64_MAX; txgtarget = UINT64_MAX; } else { txgsafe = MIN(spa_last_synced_txg(spa), spa_freeze_txg(spa)); if (txgsafe > trim_txg_delay) txgtarget = txgsafe - trim_txg_delay; else txgtarget = 0; } mutex_enter(&tm->tm_lock); /* Loop until we have sent all outstanding free's */ while ((ts = trim_map_first(tm, txgtarget, txgsafe, timelimit)) != NULL) { list_remove(&tm->tm_head, ts); avl_remove(&tm->tm_queued_frees, ts); avl_add(&tm->tm_inflight_frees, ts); size = ts->ts_end - ts->ts_start; offset = ts->ts_start; TRIM_MAP_SDEC(tm, size); TRIM_MAP_QDEC(tm); /* * We drop the lock while we call zio_nowait as the IO * scheduler can result in a different IO being run e.g. * a write which would result in a recursive lock. */ mutex_exit(&tm->tm_lock); zio_nowait(zio_trim(zio, spa, vd, offset, size)); mutex_enter(&tm->tm_lock); ts = trim_map_first(tm, txgtarget, txgsafe, timelimit); } mutex_exit(&tm->tm_lock); } static void trim_map_vdev_commit_done(spa_t *spa, vdev_t *vd) { trim_map_t *tm = vd->vdev_trimmap; trim_seg_t *ts; list_t pending_writes; zio_t *zio; uint64_t start, size; void *cookie; ASSERT(vd->vdev_ops->vdev_op_leaf); if (tm == NULL) return; mutex_enter(&tm->tm_lock); if (!avl_is_empty(&tm->tm_inflight_frees)) { cookie = NULL; while ((ts = avl_destroy_nodes(&tm->tm_inflight_frees, &cookie)) != NULL) { kmem_free(ts, sizeof (*ts)); } } list_create(&pending_writes, sizeof (zio_t), offsetof(zio_t, io_trim_link)); list_move_tail(&pending_writes, &tm->tm_pending_writes); mutex_exit(&tm->tm_lock); while ((zio = list_remove_head(&pending_writes)) != NULL) { zio_vdev_io_reissue(zio); zio_execute(zio); } list_destroy(&pending_writes); } static void trim_map_commit(spa_t *spa, zio_t *zio, vdev_t *vd) { int c; if (vd == NULL) return; if (vd->vdev_ops->vdev_op_leaf) { trim_map_vdev_commit(spa, zio, vd); } else { for (c = 0; c < vd->vdev_children; c++) trim_map_commit(spa, zio, vd->vdev_child[c]); } } static void trim_map_commit_done(spa_t *spa, vdev_t *vd) { int c; if (vd == NULL) return; if (vd->vdev_ops->vdev_op_leaf) { trim_map_vdev_commit_done(spa, vd); } else { for (c = 0; c < vd->vdev_children; c++) trim_map_commit_done(spa, vd->vdev_child[c]); } } static void trim_thread(void *arg) { spa_t *spa = arg; zio_t *zio; #ifdef _KERNEL (void) snprintf(curthread->td_name, sizeof(curthread->td_name), "trim %s", spa_name(spa)); #endif for (;;) { mutex_enter(&spa->spa_trim_lock); if (spa->spa_trim_thread == NULL) { spa->spa_trim_thread = curthread; cv_signal(&spa->spa_trim_cv); mutex_exit(&spa->spa_trim_lock); thread_exit(); } (void) cv_timedwait(&spa->spa_trim_cv, &spa->spa_trim_lock, hz * trim_max_interval); mutex_exit(&spa->spa_trim_lock); zio = zio_root(spa, NULL, NULL, ZIO_FLAG_CANFAIL); spa_config_enter(spa, SCL_STATE, FTAG, RW_READER); trim_map_commit(spa, zio, spa->spa_root_vdev); (void) zio_wait(zio); trim_map_commit_done(spa, spa->spa_root_vdev); spa_config_exit(spa, SCL_STATE, FTAG); } } void trim_thread_create(spa_t *spa) { if (!zfs_trim_enabled) return; mutex_init(&spa->spa_trim_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&spa->spa_trim_cv, NULL, CV_DEFAULT, NULL); mutex_enter(&spa->spa_trim_lock); spa->spa_trim_thread = thread_create(NULL, 0, trim_thread, spa, 0, &p0, TS_RUN, minclsyspri); mutex_exit(&spa->spa_trim_lock); } void trim_thread_destroy(spa_t *spa) { if (!zfs_trim_enabled) return; if (spa->spa_trim_thread == NULL) return; mutex_enter(&spa->spa_trim_lock); /* Setting spa_trim_thread to NULL tells the thread to stop. */ spa->spa_trim_thread = NULL; cv_signal(&spa->spa_trim_cv); /* The thread will set it back to != NULL on exit. */ while (spa->spa_trim_thread == NULL) cv_wait(&spa->spa_trim_cv, &spa->spa_trim_lock); spa->spa_trim_thread = NULL; mutex_exit(&spa->spa_trim_lock); cv_destroy(&spa->spa_trim_cv); mutex_destroy(&spa->spa_trim_lock); } void trim_thread_wakeup(spa_t *spa) { if (!zfs_trim_enabled) return; if (spa->spa_trim_thread == NULL) return; mutex_enter(&spa->spa_trim_lock); cv_signal(&spa->spa_trim_cv); mutex_exit(&spa->spa_trim_lock); } Index: releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c =================================================================== --- releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (revision 284192) +++ releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (revision 284193) @@ -1,3494 +1,3493 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2011, 2014 by Delphix. All rights reserved. * Copyright 2013 Martin Matuska . All rights reserved. */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include SYSCTL_DECL(_vfs_zfs); SYSCTL_NODE(_vfs_zfs, OID_AUTO, vdev, CTLFLAG_RW, 0, "ZFS VDEV"); /* * Virtual device management. */ /* * The limit for ZFS to automatically increase a top-level vdev's ashift * from logical ashift to physical ashift. * * Example: one or more 512B emulation child vdevs * child->vdev_ashift = 9 (512 bytes) * child->vdev_physical_ashift = 12 (4096 bytes) * zfs_max_auto_ashift = 11 (2048 bytes) * zfs_min_auto_ashift = 9 (512 bytes) * * On pool creation or the addition of a new top-level vdev, ZFS will * increase the ashift of the top-level vdev to 2048 as limited by * zfs_max_auto_ashift. * * Example: one or more 512B emulation child vdevs * child->vdev_ashift = 9 (512 bytes) * child->vdev_physical_ashift = 12 (4096 bytes) * zfs_max_auto_ashift = 13 (8192 bytes) * zfs_min_auto_ashift = 9 (512 bytes) * * On pool creation or the addition of a new top-level vdev, ZFS will * increase the ashift of the top-level vdev to 4096 to match the * max vdev_physical_ashift. * * Example: one or more 512B emulation child vdevs * child->vdev_ashift = 9 (512 bytes) * child->vdev_physical_ashift = 9 (512 bytes) * zfs_max_auto_ashift = 13 (8192 bytes) * zfs_min_auto_ashift = 12 (4096 bytes) * * On pool creation or the addition of a new top-level vdev, ZFS will * increase the ashift of the top-level vdev to 4096 to match the * zfs_min_auto_ashift. */ static uint64_t zfs_max_auto_ashift = SPA_MAXASHIFT; static uint64_t zfs_min_auto_ashift = SPA_MINASHIFT; static int sysctl_vfs_zfs_max_auto_ashift(SYSCTL_HANDLER_ARGS) { uint64_t val; int err; val = zfs_max_auto_ashift; err = sysctl_handle_64(oidp, &val, 0, req); if (err != 0 || req->newptr == NULL) return (err); if (val > SPA_MAXASHIFT || val < zfs_min_auto_ashift) return (EINVAL); zfs_max_auto_ashift = val; return (0); } SYSCTL_PROC(_vfs_zfs, OID_AUTO, max_auto_ashift, CTLTYPE_U64 | CTLFLAG_MPSAFE | CTLFLAG_RW, 0, sizeof(uint64_t), sysctl_vfs_zfs_max_auto_ashift, "QU", "Max ashift used when optimising for logical -> physical sectors size on " "new top-level vdevs."); static int sysctl_vfs_zfs_min_auto_ashift(SYSCTL_HANDLER_ARGS) { uint64_t val; int err; val = zfs_min_auto_ashift; err = sysctl_handle_64(oidp, &val, 0, req); if (err != 0 || req->newptr == NULL) return (err); if (val < SPA_MINASHIFT || val > zfs_max_auto_ashift) return (EINVAL); zfs_min_auto_ashift = val; return (0); } SYSCTL_PROC(_vfs_zfs, OID_AUTO, min_auto_ashift, CTLTYPE_U64 | CTLFLAG_MPSAFE | CTLFLAG_RW, 0, sizeof(uint64_t), sysctl_vfs_zfs_min_auto_ashift, "QU", "Min ashift used when creating new top-level vdevs."); static vdev_ops_t *vdev_ops_table[] = { &vdev_root_ops, &vdev_raidz_ops, &vdev_mirror_ops, &vdev_replacing_ops, &vdev_spare_ops, #ifdef _KERNEL &vdev_geom_ops, #else &vdev_disk_ops, #endif &vdev_file_ops, &vdev_missing_ops, &vdev_hole_ops, NULL }; /* * Given a vdev type, return the appropriate ops vector. */ static vdev_ops_t * vdev_getops(const char *type) { vdev_ops_t *ops, **opspp; for (opspp = vdev_ops_table; (ops = *opspp) != NULL; opspp++) if (strcmp(ops->vdev_op_type, type) == 0) break; return (ops); } /* * Default asize function: return the MAX of psize with the asize of * all children. This is what's used by anything other than RAID-Z. */ uint64_t vdev_default_asize(vdev_t *vd, uint64_t psize) { uint64_t asize = P2ROUNDUP(psize, 1ULL << vd->vdev_top->vdev_ashift); uint64_t csize; for (int c = 0; c < vd->vdev_children; c++) { csize = vdev_psize_to_asize(vd->vdev_child[c], psize); asize = MAX(asize, csize); } return (asize); } /* * Get the minimum allocatable size. We define the allocatable size as * the vdev's asize rounded to the nearest metaslab. This allows us to * replace or attach devices which don't have the same physical size but * can still satisfy the same number of allocations. */ uint64_t vdev_get_min_asize(vdev_t *vd) { vdev_t *pvd = vd->vdev_parent; /* * If our parent is NULL (inactive spare or cache) or is the root, * just return our own asize. */ if (pvd == NULL) return (vd->vdev_asize); /* * The top-level vdev just returns the allocatable size rounded * to the nearest metaslab. */ if (vd == vd->vdev_top) return (P2ALIGN(vd->vdev_asize, 1ULL << vd->vdev_ms_shift)); /* * The allocatable space for a raidz vdev is N * sizeof(smallest child), * so each child must provide at least 1/Nth of its asize. */ if (pvd->vdev_ops == &vdev_raidz_ops) return (pvd->vdev_min_asize / pvd->vdev_children); return (pvd->vdev_min_asize); } void vdev_set_min_asize(vdev_t *vd) { vd->vdev_min_asize = vdev_get_min_asize(vd); for (int c = 0; c < vd->vdev_children; c++) vdev_set_min_asize(vd->vdev_child[c]); } vdev_t * vdev_lookup_top(spa_t *spa, uint64_t vdev) { vdev_t *rvd = spa->spa_root_vdev; ASSERT(spa_config_held(spa, SCL_ALL, RW_READER) != 0); if (vdev < rvd->vdev_children) { ASSERT(rvd->vdev_child[vdev] != NULL); return (rvd->vdev_child[vdev]); } return (NULL); } vdev_t * vdev_lookup_by_guid(vdev_t *vd, uint64_t guid) { vdev_t *mvd; if (vd->vdev_guid == guid) return (vd); for (int c = 0; c < vd->vdev_children; c++) if ((mvd = vdev_lookup_by_guid(vd->vdev_child[c], guid)) != NULL) return (mvd); return (NULL); } void vdev_add_child(vdev_t *pvd, vdev_t *cvd) { size_t oldsize, newsize; uint64_t id = cvd->vdev_id; vdev_t **newchild; ASSERT(spa_config_held(cvd->vdev_spa, SCL_ALL, RW_WRITER) == SCL_ALL); ASSERT(cvd->vdev_parent == NULL); cvd->vdev_parent = pvd; if (pvd == NULL) return; ASSERT(id >= pvd->vdev_children || pvd->vdev_child[id] == NULL); oldsize = pvd->vdev_children * sizeof (vdev_t *); pvd->vdev_children = MAX(pvd->vdev_children, id + 1); newsize = pvd->vdev_children * sizeof (vdev_t *); newchild = kmem_zalloc(newsize, KM_SLEEP); if (pvd->vdev_child != NULL) { bcopy(pvd->vdev_child, newchild, oldsize); kmem_free(pvd->vdev_child, oldsize); } pvd->vdev_child = newchild; pvd->vdev_child[id] = cvd; cvd->vdev_top = (pvd->vdev_top ? pvd->vdev_top: cvd); ASSERT(cvd->vdev_top->vdev_parent->vdev_parent == NULL); /* * Walk up all ancestors to update guid sum. */ for (; pvd != NULL; pvd = pvd->vdev_parent) pvd->vdev_guid_sum += cvd->vdev_guid_sum; } void vdev_remove_child(vdev_t *pvd, vdev_t *cvd) { int c; uint_t id = cvd->vdev_id; ASSERT(cvd->vdev_parent == pvd); if (pvd == NULL) return; ASSERT(id < pvd->vdev_children); ASSERT(pvd->vdev_child[id] == cvd); pvd->vdev_child[id] = NULL; cvd->vdev_parent = NULL; for (c = 0; c < pvd->vdev_children; c++) if (pvd->vdev_child[c]) break; if (c == pvd->vdev_children) { kmem_free(pvd->vdev_child, c * sizeof (vdev_t *)); pvd->vdev_child = NULL; pvd->vdev_children = 0; } /* * Walk up all ancestors to update guid sum. */ for (; pvd != NULL; pvd = pvd->vdev_parent) pvd->vdev_guid_sum -= cvd->vdev_guid_sum; } /* * Remove any holes in the child array. */ void vdev_compact_children(vdev_t *pvd) { vdev_t **newchild, *cvd; int oldc = pvd->vdev_children; int newc; ASSERT(spa_config_held(pvd->vdev_spa, SCL_ALL, RW_WRITER) == SCL_ALL); for (int c = newc = 0; c < oldc; c++) if (pvd->vdev_child[c]) newc++; newchild = kmem_alloc(newc * sizeof (vdev_t *), KM_SLEEP); for (int c = newc = 0; c < oldc; c++) { if ((cvd = pvd->vdev_child[c]) != NULL) { newchild[newc] = cvd; cvd->vdev_id = newc++; } } kmem_free(pvd->vdev_child, oldc * sizeof (vdev_t *)); pvd->vdev_child = newchild; pvd->vdev_children = newc; } /* * Allocate and minimally initialize a vdev_t. */ vdev_t * vdev_alloc_common(spa_t *spa, uint_t id, uint64_t guid, vdev_ops_t *ops) { vdev_t *vd; vd = kmem_zalloc(sizeof (vdev_t), KM_SLEEP); if (spa->spa_root_vdev == NULL) { ASSERT(ops == &vdev_root_ops); spa->spa_root_vdev = vd; spa->spa_load_guid = spa_generate_guid(NULL); } if (guid == 0 && ops != &vdev_hole_ops) { if (spa->spa_root_vdev == vd) { /* * The root vdev's guid will also be the pool guid, * which must be unique among all pools. */ guid = spa_generate_guid(NULL); } else { /* * Any other vdev's guid must be unique within the pool. */ guid = spa_generate_guid(spa); } ASSERT(!spa_guid_exists(spa_guid(spa), guid)); } vd->vdev_spa = spa; vd->vdev_id = id; vd->vdev_guid = guid; vd->vdev_guid_sum = guid; vd->vdev_ops = ops; vd->vdev_state = VDEV_STATE_CLOSED; vd->vdev_ishole = (ops == &vdev_hole_ops); mutex_init(&vd->vdev_dtl_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&vd->vdev_stat_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&vd->vdev_probe_lock, NULL, MUTEX_DEFAULT, NULL); for (int t = 0; t < DTL_TYPES; t++) { vd->vdev_dtl[t] = range_tree_create(NULL, NULL, &vd->vdev_dtl_lock); } txg_list_create(&vd->vdev_ms_list, offsetof(struct metaslab, ms_txg_node)); txg_list_create(&vd->vdev_dtl_list, offsetof(struct vdev, vdev_dtl_node)); vd->vdev_stat.vs_timestamp = gethrtime(); vdev_queue_init(vd); vdev_cache_init(vd); return (vd); } /* * Allocate a new vdev. The 'alloctype' is used to control whether we are * creating a new vdev or loading an existing one - the behavior is slightly * different for each case. */ int vdev_alloc(spa_t *spa, vdev_t **vdp, nvlist_t *nv, vdev_t *parent, uint_t id, int alloctype) { vdev_ops_t *ops; char *type; uint64_t guid = 0, islog, nparity; vdev_t *vd; ASSERT(spa_config_held(spa, SCL_ALL, RW_WRITER) == SCL_ALL); if (nvlist_lookup_string(nv, ZPOOL_CONFIG_TYPE, &type) != 0) return (SET_ERROR(EINVAL)); if ((ops = vdev_getops(type)) == NULL) return (SET_ERROR(EINVAL)); /* * If this is a load, get the vdev guid from the nvlist. * Otherwise, vdev_alloc_common() will generate one for us. */ if (alloctype == VDEV_ALLOC_LOAD) { uint64_t label_id; if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_ID, &label_id) || label_id != id) return (SET_ERROR(EINVAL)); if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, &guid) != 0) return (SET_ERROR(EINVAL)); } else if (alloctype == VDEV_ALLOC_SPARE) { if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, &guid) != 0) return (SET_ERROR(EINVAL)); } else if (alloctype == VDEV_ALLOC_L2CACHE) { if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, &guid) != 0) return (SET_ERROR(EINVAL)); } else if (alloctype == VDEV_ALLOC_ROOTPOOL) { if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_GUID, &guid) != 0) return (SET_ERROR(EINVAL)); } /* * The first allocated vdev must be of type 'root'. */ if (ops != &vdev_root_ops && spa->spa_root_vdev == NULL) return (SET_ERROR(EINVAL)); /* * Determine whether we're a log vdev. */ islog = 0; (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_IS_LOG, &islog); if (islog && spa_version(spa) < SPA_VERSION_SLOGS) return (SET_ERROR(ENOTSUP)); if (ops == &vdev_hole_ops && spa_version(spa) < SPA_VERSION_HOLES) return (SET_ERROR(ENOTSUP)); /* * Set the nparity property for RAID-Z vdevs. */ nparity = -1ULL; if (ops == &vdev_raidz_ops) { if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_NPARITY, &nparity) == 0) { if (nparity == 0 || nparity > VDEV_RAIDZ_MAXPARITY) return (SET_ERROR(EINVAL)); /* * Previous versions could only support 1 or 2 parity * device. */ if (nparity > 1 && spa_version(spa) < SPA_VERSION_RAIDZ2) return (SET_ERROR(ENOTSUP)); if (nparity > 2 && spa_version(spa) < SPA_VERSION_RAIDZ3) return (SET_ERROR(ENOTSUP)); } else { /* * We require the parity to be specified for SPAs that * support multiple parity levels. */ if (spa_version(spa) >= SPA_VERSION_RAIDZ2) return (SET_ERROR(EINVAL)); /* * Otherwise, we default to 1 parity device for RAID-Z. */ nparity = 1; } } else { nparity = 0; } ASSERT(nparity != -1ULL); vd = vdev_alloc_common(spa, id, guid, ops); vd->vdev_islog = islog; vd->vdev_nparity = nparity; if (nvlist_lookup_string(nv, ZPOOL_CONFIG_PATH, &vd->vdev_path) == 0) vd->vdev_path = spa_strdup(vd->vdev_path); if (nvlist_lookup_string(nv, ZPOOL_CONFIG_DEVID, &vd->vdev_devid) == 0) vd->vdev_devid = spa_strdup(vd->vdev_devid); if (nvlist_lookup_string(nv, ZPOOL_CONFIG_PHYS_PATH, &vd->vdev_physpath) == 0) vd->vdev_physpath = spa_strdup(vd->vdev_physpath); if (nvlist_lookup_string(nv, ZPOOL_CONFIG_FRU, &vd->vdev_fru) == 0) vd->vdev_fru = spa_strdup(vd->vdev_fru); /* * Set the whole_disk property. If it's not specified, leave the value * as -1. */ if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_WHOLE_DISK, &vd->vdev_wholedisk) != 0) vd->vdev_wholedisk = -1ULL; /* * Look for the 'not present' flag. This will only be set if the device * was not present at the time of import. */ (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_NOT_PRESENT, &vd->vdev_not_present); /* * Get the alignment requirement. */ (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_ASHIFT, &vd->vdev_ashift); /* * Retrieve the vdev creation time. */ (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_CREATE_TXG, &vd->vdev_crtxg); /* * If we're a top-level vdev, try to load the allocation parameters. */ if (parent && !parent->vdev_parent && (alloctype == VDEV_ALLOC_LOAD || alloctype == VDEV_ALLOC_SPLIT)) { (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_METASLAB_ARRAY, &vd->vdev_ms_array); (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_METASLAB_SHIFT, &vd->vdev_ms_shift); (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_ASIZE, &vd->vdev_asize); (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_REMOVING, &vd->vdev_removing); } if (parent && !parent->vdev_parent && alloctype != VDEV_ALLOC_ATTACH) { ASSERT(alloctype == VDEV_ALLOC_LOAD || alloctype == VDEV_ALLOC_ADD || alloctype == VDEV_ALLOC_SPLIT || alloctype == VDEV_ALLOC_ROOTPOOL); vd->vdev_mg = metaslab_group_create(islog ? spa_log_class(spa) : spa_normal_class(spa), vd); } /* * If we're a leaf vdev, try to load the DTL object and other state. */ if (vd->vdev_ops->vdev_op_leaf && (alloctype == VDEV_ALLOC_LOAD || alloctype == VDEV_ALLOC_L2CACHE || alloctype == VDEV_ALLOC_ROOTPOOL)) { if (alloctype == VDEV_ALLOC_LOAD) { (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_DTL, &vd->vdev_dtl_object); (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_UNSPARE, &vd->vdev_unspare); } if (alloctype == VDEV_ALLOC_ROOTPOOL) { uint64_t spare = 0; if (nvlist_lookup_uint64(nv, ZPOOL_CONFIG_IS_SPARE, &spare) == 0 && spare) spa_spare_add(vd); } (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_OFFLINE, &vd->vdev_offline); (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_RESILVER_TXG, &vd->vdev_resilver_txg); /* * When importing a pool, we want to ignore the persistent fault * state, as the diagnosis made on another system may not be * valid in the current context. Local vdevs will * remain in the faulted state. */ if (spa_load_state(spa) == SPA_LOAD_OPEN) { (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_FAULTED, &vd->vdev_faulted); (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_DEGRADED, &vd->vdev_degraded); (void) nvlist_lookup_uint64(nv, ZPOOL_CONFIG_REMOVED, &vd->vdev_removed); if (vd->vdev_faulted || vd->vdev_degraded) { char *aux; vd->vdev_label_aux = VDEV_AUX_ERR_EXCEEDED; if (nvlist_lookup_string(nv, ZPOOL_CONFIG_AUX_STATE, &aux) == 0 && strcmp(aux, "external") == 0) vd->vdev_label_aux = VDEV_AUX_EXTERNAL; } } } /* * Add ourselves to the parent's list of children. */ vdev_add_child(parent, vd); *vdp = vd; return (0); } void vdev_free(vdev_t *vd) { spa_t *spa = vd->vdev_spa; /* * vdev_free() implies closing the vdev first. This is simpler than * trying to ensure complicated semantics for all callers. */ vdev_close(vd); ASSERT(!list_link_active(&vd->vdev_config_dirty_node)); ASSERT(!list_link_active(&vd->vdev_state_dirty_node)); /* * Free all children. */ for (int c = 0; c < vd->vdev_children; c++) vdev_free(vd->vdev_child[c]); ASSERT(vd->vdev_child == NULL); ASSERT(vd->vdev_guid_sum == vd->vdev_guid); /* * Discard allocation state. */ if (vd->vdev_mg != NULL) { vdev_metaslab_fini(vd); metaslab_group_destroy(vd->vdev_mg); } ASSERT0(vd->vdev_stat.vs_space); ASSERT0(vd->vdev_stat.vs_dspace); ASSERT0(vd->vdev_stat.vs_alloc); /* * Remove this vdev from its parent's child list. */ vdev_remove_child(vd->vdev_parent, vd); ASSERT(vd->vdev_parent == NULL); /* * Clean up vdev structure. */ vdev_queue_fini(vd); vdev_cache_fini(vd); if (vd->vdev_path) spa_strfree(vd->vdev_path); if (vd->vdev_devid) spa_strfree(vd->vdev_devid); if (vd->vdev_physpath) spa_strfree(vd->vdev_physpath); if (vd->vdev_fru) spa_strfree(vd->vdev_fru); if (vd->vdev_isspare) spa_spare_remove(vd); if (vd->vdev_isl2cache) spa_l2cache_remove(vd); txg_list_destroy(&vd->vdev_ms_list); txg_list_destroy(&vd->vdev_dtl_list); mutex_enter(&vd->vdev_dtl_lock); space_map_close(vd->vdev_dtl_sm); for (int t = 0; t < DTL_TYPES; t++) { range_tree_vacate(vd->vdev_dtl[t], NULL, NULL); range_tree_destroy(vd->vdev_dtl[t]); } mutex_exit(&vd->vdev_dtl_lock); mutex_destroy(&vd->vdev_dtl_lock); mutex_destroy(&vd->vdev_stat_lock); mutex_destroy(&vd->vdev_probe_lock); if (vd == spa->spa_root_vdev) spa->spa_root_vdev = NULL; kmem_free(vd, sizeof (vdev_t)); } /* * Transfer top-level vdev state from svd to tvd. */ static void vdev_top_transfer(vdev_t *svd, vdev_t *tvd) { spa_t *spa = svd->vdev_spa; metaslab_t *msp; vdev_t *vd; int t; ASSERT(tvd == tvd->vdev_top); tvd->vdev_ms_array = svd->vdev_ms_array; tvd->vdev_ms_shift = svd->vdev_ms_shift; tvd->vdev_ms_count = svd->vdev_ms_count; svd->vdev_ms_array = 0; svd->vdev_ms_shift = 0; svd->vdev_ms_count = 0; if (tvd->vdev_mg) ASSERT3P(tvd->vdev_mg, ==, svd->vdev_mg); tvd->vdev_mg = svd->vdev_mg; tvd->vdev_ms = svd->vdev_ms; svd->vdev_mg = NULL; svd->vdev_ms = NULL; if (tvd->vdev_mg != NULL) tvd->vdev_mg->mg_vd = tvd; tvd->vdev_stat.vs_alloc = svd->vdev_stat.vs_alloc; tvd->vdev_stat.vs_space = svd->vdev_stat.vs_space; tvd->vdev_stat.vs_dspace = svd->vdev_stat.vs_dspace; svd->vdev_stat.vs_alloc = 0; svd->vdev_stat.vs_space = 0; svd->vdev_stat.vs_dspace = 0; for (t = 0; t < TXG_SIZE; t++) { while ((msp = txg_list_remove(&svd->vdev_ms_list, t)) != NULL) (void) txg_list_add(&tvd->vdev_ms_list, msp, t); while ((vd = txg_list_remove(&svd->vdev_dtl_list, t)) != NULL) (void) txg_list_add(&tvd->vdev_dtl_list, vd, t); if (txg_list_remove_this(&spa->spa_vdev_txg_list, svd, t)) (void) txg_list_add(&spa->spa_vdev_txg_list, tvd, t); } if (list_link_active(&svd->vdev_config_dirty_node)) { vdev_config_clean(svd); vdev_config_dirty(tvd); } if (list_link_active(&svd->vdev_state_dirty_node)) { vdev_state_clean(svd); vdev_state_dirty(tvd); } tvd->vdev_deflate_ratio = svd->vdev_deflate_ratio; svd->vdev_deflate_ratio = 0; tvd->vdev_islog = svd->vdev_islog; svd->vdev_islog = 0; } static void vdev_top_update(vdev_t *tvd, vdev_t *vd) { if (vd == NULL) return; vd->vdev_top = tvd; for (int c = 0; c < vd->vdev_children; c++) vdev_top_update(tvd, vd->vdev_child[c]); } /* * Add a mirror/replacing vdev above an existing vdev. */ vdev_t * vdev_add_parent(vdev_t *cvd, vdev_ops_t *ops) { spa_t *spa = cvd->vdev_spa; vdev_t *pvd = cvd->vdev_parent; vdev_t *mvd; ASSERT(spa_config_held(spa, SCL_ALL, RW_WRITER) == SCL_ALL); mvd = vdev_alloc_common(spa, cvd->vdev_id, 0, ops); mvd->vdev_asize = cvd->vdev_asize; mvd->vdev_min_asize = cvd->vdev_min_asize; mvd->vdev_max_asize = cvd->vdev_max_asize; mvd->vdev_ashift = cvd->vdev_ashift; mvd->vdev_logical_ashift = cvd->vdev_logical_ashift; mvd->vdev_physical_ashift = cvd->vdev_physical_ashift; mvd->vdev_state = cvd->vdev_state; mvd->vdev_crtxg = cvd->vdev_crtxg; vdev_remove_child(pvd, cvd); vdev_add_child(pvd, mvd); cvd->vdev_id = mvd->vdev_children; vdev_add_child(mvd, cvd); vdev_top_update(cvd->vdev_top, cvd->vdev_top); if (mvd == mvd->vdev_top) vdev_top_transfer(cvd, mvd); return (mvd); } /* * Remove a 1-way mirror/replacing vdev from the tree. */ void vdev_remove_parent(vdev_t *cvd) { vdev_t *mvd = cvd->vdev_parent; vdev_t *pvd = mvd->vdev_parent; ASSERT(spa_config_held(cvd->vdev_spa, SCL_ALL, RW_WRITER) == SCL_ALL); ASSERT(mvd->vdev_children == 1); ASSERT(mvd->vdev_ops == &vdev_mirror_ops || mvd->vdev_ops == &vdev_replacing_ops || mvd->vdev_ops == &vdev_spare_ops); cvd->vdev_ashift = mvd->vdev_ashift; cvd->vdev_logical_ashift = mvd->vdev_logical_ashift; cvd->vdev_physical_ashift = mvd->vdev_physical_ashift; vdev_remove_child(mvd, cvd); vdev_remove_child(pvd, mvd); /* * If cvd will replace mvd as a top-level vdev, preserve mvd's guid. * Otherwise, we could have detached an offline device, and when we * go to import the pool we'll think we have two top-level vdevs, * instead of a different version of the same top-level vdev. */ if (mvd->vdev_top == mvd) { uint64_t guid_delta = mvd->vdev_guid - cvd->vdev_guid; cvd->vdev_orig_guid = cvd->vdev_guid; cvd->vdev_guid += guid_delta; cvd->vdev_guid_sum += guid_delta; } cvd->vdev_id = mvd->vdev_id; vdev_add_child(pvd, cvd); vdev_top_update(cvd->vdev_top, cvd->vdev_top); if (cvd == cvd->vdev_top) vdev_top_transfer(mvd, cvd); ASSERT(mvd->vdev_children == 0); vdev_free(mvd); } int vdev_metaslab_init(vdev_t *vd, uint64_t txg) { spa_t *spa = vd->vdev_spa; objset_t *mos = spa->spa_meta_objset; uint64_t m; uint64_t oldc = vd->vdev_ms_count; uint64_t newc = vd->vdev_asize >> vd->vdev_ms_shift; metaslab_t **mspp; int error; ASSERT(txg == 0 || spa_config_held(spa, SCL_ALLOC, RW_WRITER)); /* * This vdev is not being allocated from yet or is a hole. */ if (vd->vdev_ms_shift == 0) return (0); ASSERT(!vd->vdev_ishole); /* * Compute the raidz-deflation ratio. Note, we hard-code * in 128k (1 << 17) because it is the current "typical" blocksize. * Even if SPA_MAXBLOCKSIZE changes, this algorithm must never change, * or we will inconsistently account for existing bp's. */ vd->vdev_deflate_ratio = (1 << 17) / (vdev_psize_to_asize(vd, 1 << 17) >> SPA_MINBLOCKSHIFT); ASSERT(oldc <= newc); mspp = kmem_zalloc(newc * sizeof (*mspp), KM_SLEEP); if (oldc != 0) { bcopy(vd->vdev_ms, mspp, oldc * sizeof (*mspp)); kmem_free(vd->vdev_ms, oldc * sizeof (*mspp)); } vd->vdev_ms = mspp; vd->vdev_ms_count = newc; for (m = oldc; m < newc; m++) { uint64_t object = 0; if (txg == 0) { error = dmu_read(mos, vd->vdev_ms_array, m * sizeof (uint64_t), sizeof (uint64_t), &object, DMU_READ_PREFETCH); if (error) return (error); } vd->vdev_ms[m] = metaslab_init(vd->vdev_mg, m, object, txg); } if (txg == 0) spa_config_enter(spa, SCL_ALLOC, FTAG, RW_WRITER); /* * If the vdev is being removed we don't activate * the metaslabs since we want to ensure that no new * allocations are performed on this device. */ if (oldc == 0 && !vd->vdev_removing) metaslab_group_activate(vd->vdev_mg); if (txg == 0) spa_config_exit(spa, SCL_ALLOC, FTAG); return (0); } void vdev_metaslab_fini(vdev_t *vd) { uint64_t m; uint64_t count = vd->vdev_ms_count; if (vd->vdev_ms != NULL) { metaslab_group_passivate(vd->vdev_mg); for (m = 0; m < count; m++) { metaslab_t *msp = vd->vdev_ms[m]; if (msp != NULL) metaslab_fini(msp); } kmem_free(vd->vdev_ms, count * sizeof (metaslab_t *)); vd->vdev_ms = NULL; } } typedef struct vdev_probe_stats { boolean_t vps_readable; boolean_t vps_writeable; int vps_flags; } vdev_probe_stats_t; static void vdev_probe_done(zio_t *zio) { spa_t *spa = zio->io_spa; vdev_t *vd = zio->io_vd; vdev_probe_stats_t *vps = zio->io_private; ASSERT(vd->vdev_probe_zio != NULL); if (zio->io_type == ZIO_TYPE_READ) { if (zio->io_error == 0) vps->vps_readable = 1; if (zio->io_error == 0 && spa_writeable(spa)) { zio_nowait(zio_write_phys(vd->vdev_probe_zio, vd, zio->io_offset, zio->io_size, zio->io_data, ZIO_CHECKSUM_OFF, vdev_probe_done, vps, ZIO_PRIORITY_SYNC_WRITE, vps->vps_flags, B_TRUE)); } else { zio_buf_free(zio->io_data, zio->io_size); } } else if (zio->io_type == ZIO_TYPE_WRITE) { if (zio->io_error == 0) vps->vps_writeable = 1; zio_buf_free(zio->io_data, zio->io_size); } else if (zio->io_type == ZIO_TYPE_NULL) { zio_t *pio; vd->vdev_cant_read |= !vps->vps_readable; vd->vdev_cant_write |= !vps->vps_writeable; if (vdev_readable(vd) && (vdev_writeable(vd) || !spa_writeable(spa))) { zio->io_error = 0; } else { ASSERT(zio->io_error != 0); zfs_ereport_post(FM_EREPORT_ZFS_PROBE_FAILURE, spa, vd, NULL, 0, 0); zio->io_error = SET_ERROR(ENXIO); } mutex_enter(&vd->vdev_probe_lock); ASSERT(vd->vdev_probe_zio == zio); vd->vdev_probe_zio = NULL; mutex_exit(&vd->vdev_probe_lock); while ((pio = zio_walk_parents(zio)) != NULL) if (!vdev_accessible(vd, pio)) pio->io_error = SET_ERROR(ENXIO); kmem_free(vps, sizeof (*vps)); } } /* * Determine whether this device is accessible. * * Read and write to several known locations: the pad regions of each * vdev label but the first, which we leave alone in case it contains * a VTOC. */ zio_t * vdev_probe(vdev_t *vd, zio_t *zio) { spa_t *spa = vd->vdev_spa; vdev_probe_stats_t *vps = NULL; zio_t *pio; ASSERT(vd->vdev_ops->vdev_op_leaf); /* * Don't probe the probe. */ if (zio && (zio->io_flags & ZIO_FLAG_PROBE)) return (NULL); /* * To prevent 'probe storms' when a device fails, we create * just one probe i/o at a time. All zios that want to probe * this vdev will become parents of the probe io. */ mutex_enter(&vd->vdev_probe_lock); if ((pio = vd->vdev_probe_zio) == NULL) { vps = kmem_zalloc(sizeof (*vps), KM_SLEEP); vps->vps_flags = ZIO_FLAG_CANFAIL | ZIO_FLAG_PROBE | ZIO_FLAG_DONT_CACHE | ZIO_FLAG_DONT_AGGREGATE | ZIO_FLAG_TRYHARD; if (spa_config_held(spa, SCL_ZIO, RW_WRITER)) { /* * vdev_cant_read and vdev_cant_write can only * transition from TRUE to FALSE when we have the * SCL_ZIO lock as writer; otherwise they can only * transition from FALSE to TRUE. This ensures that * any zio looking at these values can assume that * failures persist for the life of the I/O. That's * important because when a device has intermittent * connectivity problems, we want to ensure that * they're ascribed to the device (ENXIO) and not * the zio (EIO). * * Since we hold SCL_ZIO as writer here, clear both * values so the probe can reevaluate from first * principles. */ vps->vps_flags |= ZIO_FLAG_CONFIG_WRITER; vd->vdev_cant_read = B_FALSE; vd->vdev_cant_write = B_FALSE; } vd->vdev_probe_zio = pio = zio_null(NULL, spa, vd, vdev_probe_done, vps, vps->vps_flags | ZIO_FLAG_DONT_PROPAGATE); /* * We can't change the vdev state in this context, so we * kick off an async task to do it on our behalf. */ if (zio != NULL) { vd->vdev_probe_wanted = B_TRUE; spa_async_request(spa, SPA_ASYNC_PROBE); } } if (zio != NULL) zio_add_child(zio, pio); mutex_exit(&vd->vdev_probe_lock); if (vps == NULL) { ASSERT(zio != NULL); return (NULL); } for (int l = 1; l < VDEV_LABELS; l++) { zio_nowait(zio_read_phys(pio, vd, vdev_label_offset(vd->vdev_psize, l, offsetof(vdev_label_t, vl_pad2)), VDEV_PAD_SIZE, zio_buf_alloc(VDEV_PAD_SIZE), ZIO_CHECKSUM_OFF, vdev_probe_done, vps, ZIO_PRIORITY_SYNC_READ, vps->vps_flags, B_TRUE)); } if (zio == NULL) return (pio); zio_nowait(pio); return (NULL); } static void vdev_open_child(void *arg) { vdev_t *vd = arg; vd->vdev_open_thread = curthread; vd->vdev_open_error = vdev_open(vd); vd->vdev_open_thread = NULL; } boolean_t vdev_uses_zvols(vdev_t *vd) { if (vd->vdev_path && strncmp(vd->vdev_path, ZVOL_DIR, strlen(ZVOL_DIR)) == 0) return (B_TRUE); for (int c = 0; c < vd->vdev_children; c++) if (vdev_uses_zvols(vd->vdev_child[c])) return (B_TRUE); return (B_FALSE); } void vdev_open_children(vdev_t *vd) { taskq_t *tq; int children = vd->vdev_children; /* * in order to handle pools on top of zvols, do the opens * in a single thread so that the same thread holds the * spa_namespace_lock */ if (B_TRUE || vdev_uses_zvols(vd)) { for (int c = 0; c < children; c++) vd->vdev_child[c]->vdev_open_error = vdev_open(vd->vdev_child[c]); return; } tq = taskq_create("vdev_open", children, minclsyspri, children, children, TASKQ_PREPOPULATE); for (int c = 0; c < children; c++) VERIFY(taskq_dispatch(tq, vdev_open_child, vd->vdev_child[c], TQ_SLEEP) != 0); taskq_destroy(tq); } /* * Prepare a virtual device for access. */ int vdev_open(vdev_t *vd) { spa_t *spa = vd->vdev_spa; int error; uint64_t osize = 0; uint64_t max_osize = 0; uint64_t asize, max_asize, psize; uint64_t logical_ashift = 0; uint64_t physical_ashift = 0; ASSERT(vd->vdev_open_thread == curthread || spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); ASSERT(vd->vdev_state == VDEV_STATE_CLOSED || vd->vdev_state == VDEV_STATE_CANT_OPEN || vd->vdev_state == VDEV_STATE_OFFLINE); vd->vdev_stat.vs_aux = VDEV_AUX_NONE; vd->vdev_cant_read = B_FALSE; vd->vdev_cant_write = B_FALSE; + vd->vdev_notrim = B_FALSE; vd->vdev_min_asize = vdev_get_min_asize(vd); /* * If this vdev is not removed, check its fault status. If it's * faulted, bail out of the open. */ if (!vd->vdev_removed && vd->vdev_faulted) { ASSERT(vd->vdev_children == 0); ASSERT(vd->vdev_label_aux == VDEV_AUX_ERR_EXCEEDED || vd->vdev_label_aux == VDEV_AUX_EXTERNAL); vdev_set_state(vd, B_TRUE, VDEV_STATE_FAULTED, vd->vdev_label_aux); return (SET_ERROR(ENXIO)); } else if (vd->vdev_offline) { ASSERT(vd->vdev_children == 0); vdev_set_state(vd, B_TRUE, VDEV_STATE_OFFLINE, VDEV_AUX_NONE); return (SET_ERROR(ENXIO)); } error = vd->vdev_ops->vdev_op_open(vd, &osize, &max_osize, &logical_ashift, &physical_ashift); /* * Reset the vdev_reopening flag so that we actually close * the vdev on error. */ vd->vdev_reopening = B_FALSE; if (zio_injection_enabled && error == 0) error = zio_handle_device_injection(vd, NULL, ENXIO); if (error) { if (vd->vdev_removed && vd->vdev_stat.vs_aux != VDEV_AUX_OPEN_FAILED) vd->vdev_removed = B_FALSE; vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, vd->vdev_stat.vs_aux); return (error); } vd->vdev_removed = B_FALSE; /* * Recheck the faulted flag now that we have confirmed that * the vdev is accessible. If we're faulted, bail. */ if (vd->vdev_faulted) { ASSERT(vd->vdev_children == 0); ASSERT(vd->vdev_label_aux == VDEV_AUX_ERR_EXCEEDED || vd->vdev_label_aux == VDEV_AUX_EXTERNAL); vdev_set_state(vd, B_TRUE, VDEV_STATE_FAULTED, vd->vdev_label_aux); return (SET_ERROR(ENXIO)); } if (vd->vdev_degraded) { ASSERT(vd->vdev_children == 0); vdev_set_state(vd, B_TRUE, VDEV_STATE_DEGRADED, VDEV_AUX_ERR_EXCEEDED); } else { vdev_set_state(vd, B_TRUE, VDEV_STATE_HEALTHY, 0); } /* * For hole or missing vdevs we just return success. */ if (vd->vdev_ishole || vd->vdev_ops == &vdev_missing_ops) return (0); - if (vd->vdev_ops->vdev_op_leaf) { - vd->vdev_notrim = B_FALSE; + if (zfs_trim_enabled && !vd->vdev_notrim && vd->vdev_ops->vdev_op_leaf) trim_map_create(vd); - } for (int c = 0; c < vd->vdev_children; c++) { if (vd->vdev_child[c]->vdev_state != VDEV_STATE_HEALTHY) { vdev_set_state(vd, B_TRUE, VDEV_STATE_DEGRADED, VDEV_AUX_NONE); break; } } osize = P2ALIGN(osize, (uint64_t)sizeof (vdev_label_t)); max_osize = P2ALIGN(max_osize, (uint64_t)sizeof (vdev_label_t)); if (vd->vdev_children == 0) { if (osize < SPA_MINDEVSIZE) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_TOO_SMALL); return (SET_ERROR(EOVERFLOW)); } psize = osize; asize = osize - (VDEV_LABEL_START_SIZE + VDEV_LABEL_END_SIZE); max_asize = max_osize - (VDEV_LABEL_START_SIZE + VDEV_LABEL_END_SIZE); } else { if (vd->vdev_parent != NULL && osize < SPA_MINDEVSIZE - (VDEV_LABEL_START_SIZE + VDEV_LABEL_END_SIZE)) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_TOO_SMALL); return (SET_ERROR(EOVERFLOW)); } psize = 0; asize = osize; max_asize = max_osize; } vd->vdev_psize = psize; /* * Make sure the allocatable size hasn't shrunk. */ if (asize < vd->vdev_min_asize) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_BAD_LABEL); return (SET_ERROR(EINVAL)); } vd->vdev_physical_ashift = MAX(physical_ashift, vd->vdev_physical_ashift); vd->vdev_logical_ashift = MAX(logical_ashift, vd->vdev_logical_ashift); vd->vdev_ashift = MAX(vd->vdev_logical_ashift, vd->vdev_ashift); if (vd->vdev_logical_ashift > SPA_MAXASHIFT) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_ASHIFT_TOO_BIG); return (EINVAL); } if (vd->vdev_asize == 0) { /* * This is the first-ever open, so use the computed values. * For testing purposes, a higher ashift can be requested. */ vd->vdev_asize = asize; vd->vdev_max_asize = max_asize; } else { /* * Make sure the alignment requirement hasn't increased. */ if (vd->vdev_ashift > vd->vdev_top->vdev_ashift && vd->vdev_ops->vdev_op_leaf) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_BAD_LABEL); return (EINVAL); } vd->vdev_max_asize = max_asize; } /* * If all children are healthy and the asize has increased, * then we've experienced dynamic LUN growth. If automatic * expansion is enabled then use the additional space. */ if (vd->vdev_state == VDEV_STATE_HEALTHY && asize > vd->vdev_asize && (vd->vdev_expanding || spa->spa_autoexpand)) vd->vdev_asize = asize; vdev_set_min_asize(vd); /* * Ensure we can issue some IO before declaring the * vdev open for business. */ if (vd->vdev_ops->vdev_op_leaf && (error = zio_wait(vdev_probe(vd, NULL))) != 0) { vdev_set_state(vd, B_TRUE, VDEV_STATE_FAULTED, VDEV_AUX_ERR_EXCEEDED); return (error); } /* * If a leaf vdev has a DTL, and seems healthy, then kick off a * resilver. But don't do this if we are doing a reopen for a scrub, * since this would just restart the scrub we are already doing. */ if (vd->vdev_ops->vdev_op_leaf && !spa->spa_scrub_reopen && vdev_resilver_needed(vd, NULL, NULL)) spa_async_request(spa, SPA_ASYNC_RESILVER); return (0); } /* * Called once the vdevs are all opened, this routine validates the label * contents. This needs to be done before vdev_load() so that we don't * inadvertently do repair I/Os to the wrong device. * * If 'strict' is false ignore the spa guid check. This is necessary because * if the machine crashed during a re-guid the new guid might have been written * to all of the vdev labels, but not the cached config. The strict check * will be performed when the pool is opened again using the mos config. * * This function will only return failure if one of the vdevs indicates that it * has since been destroyed or exported. This is only possible if * /etc/zfs/zpool.cache was readonly at the time. Otherwise, the vdev state * will be updated but the function will return 0. */ int vdev_validate(vdev_t *vd, boolean_t strict) { spa_t *spa = vd->vdev_spa; nvlist_t *label; uint64_t guid = 0, top_guid; uint64_t state; for (int c = 0; c < vd->vdev_children; c++) if (vdev_validate(vd->vdev_child[c], strict) != 0) return (SET_ERROR(EBADF)); /* * If the device has already failed, or was marked offline, don't do * any further validation. Otherwise, label I/O will fail and we will * overwrite the previous state. */ if (vd->vdev_ops->vdev_op_leaf && vdev_readable(vd)) { uint64_t aux_guid = 0; nvlist_t *nvl; uint64_t txg = spa_last_synced_txg(spa) != 0 ? spa_last_synced_txg(spa) : -1ULL; if ((label = vdev_label_read_config(vd, txg)) == NULL) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_BAD_LABEL); return (0); } /* * Determine if this vdev has been split off into another * pool. If so, then refuse to open it. */ if (nvlist_lookup_uint64(label, ZPOOL_CONFIG_SPLIT_GUID, &aux_guid) == 0 && aux_guid == spa_guid(spa)) { vdev_set_state(vd, B_FALSE, VDEV_STATE_CANT_OPEN, VDEV_AUX_SPLIT_POOL); nvlist_free(label); return (0); } if (strict && (nvlist_lookup_uint64(label, ZPOOL_CONFIG_POOL_GUID, &guid) != 0 || guid != spa_guid(spa))) { vdev_set_state(vd, B_FALSE, VDEV_STATE_CANT_OPEN, VDEV_AUX_CORRUPT_DATA); nvlist_free(label); return (0); } if (nvlist_lookup_nvlist(label, ZPOOL_CONFIG_VDEV_TREE, &nvl) != 0 || nvlist_lookup_uint64(nvl, ZPOOL_CONFIG_ORIG_GUID, &aux_guid) != 0) aux_guid = 0; /* * If this vdev just became a top-level vdev because its * sibling was detached, it will have adopted the parent's * vdev guid -- but the label may or may not be on disk yet. * Fortunately, either version of the label will have the * same top guid, so if we're a top-level vdev, we can * safely compare to that instead. * * If we split this vdev off instead, then we also check the * original pool's guid. We don't want to consider the vdev * corrupt if it is partway through a split operation. */ if (nvlist_lookup_uint64(label, ZPOOL_CONFIG_GUID, &guid) != 0 || nvlist_lookup_uint64(label, ZPOOL_CONFIG_TOP_GUID, &top_guid) != 0 || ((vd->vdev_guid != guid && vd->vdev_guid != aux_guid) && (vd->vdev_guid != top_guid || vd != vd->vdev_top))) { vdev_set_state(vd, B_FALSE, VDEV_STATE_CANT_OPEN, VDEV_AUX_CORRUPT_DATA); nvlist_free(label); return (0); } if (nvlist_lookup_uint64(label, ZPOOL_CONFIG_POOL_STATE, &state) != 0) { vdev_set_state(vd, B_FALSE, VDEV_STATE_CANT_OPEN, VDEV_AUX_CORRUPT_DATA); nvlist_free(label); return (0); } nvlist_free(label); /* * If this is a verbatim import, no need to check the * state of the pool. */ if (!(spa->spa_import_flags & ZFS_IMPORT_VERBATIM) && spa_load_state(spa) == SPA_LOAD_OPEN && state != POOL_STATE_ACTIVE) return (SET_ERROR(EBADF)); /* * If we were able to open and validate a vdev that was * previously marked permanently unavailable, clear that state * now. */ if (vd->vdev_not_present) vd->vdev_not_present = 0; } return (0); } /* * Close a virtual device. */ void vdev_close(vdev_t *vd) { spa_t *spa = vd->vdev_spa; vdev_t *pvd = vd->vdev_parent; ASSERT(spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); /* * If our parent is reopening, then we are as well, unless we are * going offline. */ if (pvd != NULL && pvd->vdev_reopening) vd->vdev_reopening = (pvd->vdev_reopening && !vd->vdev_offline); vd->vdev_ops->vdev_op_close(vd); vdev_cache_purge(vd); if (vd->vdev_ops->vdev_op_leaf) trim_map_destroy(vd); /* * We record the previous state before we close it, so that if we are * doing a reopen(), we don't generate FMA ereports if we notice that * it's still faulted. */ vd->vdev_prevstate = vd->vdev_state; if (vd->vdev_offline) vd->vdev_state = VDEV_STATE_OFFLINE; else vd->vdev_state = VDEV_STATE_CLOSED; vd->vdev_stat.vs_aux = VDEV_AUX_NONE; } void vdev_hold(vdev_t *vd) { spa_t *spa = vd->vdev_spa; ASSERT(spa_is_root(spa)); if (spa->spa_state == POOL_STATE_UNINITIALIZED) return; for (int c = 0; c < vd->vdev_children; c++) vdev_hold(vd->vdev_child[c]); if (vd->vdev_ops->vdev_op_leaf) vd->vdev_ops->vdev_op_hold(vd); } void vdev_rele(vdev_t *vd) { spa_t *spa = vd->vdev_spa; ASSERT(spa_is_root(spa)); for (int c = 0; c < vd->vdev_children; c++) vdev_rele(vd->vdev_child[c]); if (vd->vdev_ops->vdev_op_leaf) vd->vdev_ops->vdev_op_rele(vd); } /* * Reopen all interior vdevs and any unopened leaves. We don't actually * reopen leaf vdevs which had previously been opened as they might deadlock * on the spa_config_lock. Instead we only obtain the leaf's physical size. * If the leaf has never been opened then open it, as usual. */ void vdev_reopen(vdev_t *vd) { spa_t *spa = vd->vdev_spa; ASSERT(spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); /* set the reopening flag unless we're taking the vdev offline */ vd->vdev_reopening = !vd->vdev_offline; vdev_close(vd); (void) vdev_open(vd); /* * Call vdev_validate() here to make sure we have the same device. * Otherwise, a device with an invalid label could be successfully * opened in response to vdev_reopen(). */ if (vd->vdev_aux) { (void) vdev_validate_aux(vd); if (vdev_readable(vd) && vdev_writeable(vd) && vd->vdev_aux == &spa->spa_l2cache && !l2arc_vdev_present(vd)) l2arc_add_vdev(spa, vd); } else { (void) vdev_validate(vd, B_TRUE); } /* * Reassess parent vdev's health. */ vdev_propagate_state(vd); } int vdev_create(vdev_t *vd, uint64_t txg, boolean_t isreplacing) { int error; /* * Normally, partial opens (e.g. of a mirror) are allowed. * For a create, however, we want to fail the request if * there are any components we can't open. */ error = vdev_open(vd); if (error || vd->vdev_state != VDEV_STATE_HEALTHY) { vdev_close(vd); return (error ? error : ENXIO); } /* * Recursively load DTLs and initialize all labels. */ if ((error = vdev_dtl_load(vd)) != 0 || (error = vdev_label_init(vd, txg, isreplacing ? VDEV_LABEL_REPLACE : VDEV_LABEL_CREATE)) != 0) { vdev_close(vd); return (error); } return (0); } void vdev_metaslab_set_size(vdev_t *vd) { /* * Aim for roughly 200 metaslabs per vdev. */ vd->vdev_ms_shift = highbit64(vd->vdev_asize / 200); vd->vdev_ms_shift = MAX(vd->vdev_ms_shift, SPA_MAXBLOCKSHIFT); } /* * Maximize performance by inflating the configured ashift for top level * vdevs to be as close to the physical ashift as possible while maintaining * administrator defined limits and ensuring it doesn't go below the * logical ashift. */ void vdev_ashift_optimize(vdev_t *vd) { if (vd == vd->vdev_top) { if (vd->vdev_ashift < vd->vdev_physical_ashift) { vd->vdev_ashift = MIN( MAX(zfs_max_auto_ashift, vd->vdev_ashift), MAX(zfs_min_auto_ashift, vd->vdev_physical_ashift)); } else { /* * Unusual case where logical ashift > physical ashift * so we can't cap the calculated ashift based on max * ashift as that would cause failures. * We still check if we need to increase it to match * the min ashift. */ vd->vdev_ashift = MAX(zfs_min_auto_ashift, vd->vdev_ashift); } } } void vdev_dirty(vdev_t *vd, int flags, void *arg, uint64_t txg) { ASSERT(vd == vd->vdev_top); ASSERT(!vd->vdev_ishole); ASSERT(ISP2(flags)); ASSERT(spa_writeable(vd->vdev_spa)); if (flags & VDD_METASLAB) (void) txg_list_add(&vd->vdev_ms_list, arg, txg); if (flags & VDD_DTL) (void) txg_list_add(&vd->vdev_dtl_list, arg, txg); (void) txg_list_add(&vd->vdev_spa->spa_vdev_txg_list, vd, txg); } void vdev_dirty_leaves(vdev_t *vd, int flags, uint64_t txg) { for (int c = 0; c < vd->vdev_children; c++) vdev_dirty_leaves(vd->vdev_child[c], flags, txg); if (vd->vdev_ops->vdev_op_leaf) vdev_dirty(vd->vdev_top, flags, vd, txg); } /* * DTLs. * * A vdev's DTL (dirty time log) is the set of transaction groups for which * the vdev has less than perfect replication. There are four kinds of DTL: * * DTL_MISSING: txgs for which the vdev has no valid copies of the data * * DTL_PARTIAL: txgs for which data is available, but not fully replicated * * DTL_SCRUB: the txgs that could not be repaired by the last scrub; upon * scrub completion, DTL_SCRUB replaces DTL_MISSING in the range of * txgs that was scrubbed. * * DTL_OUTAGE: txgs which cannot currently be read, whether due to * persistent errors or just some device being offline. * Unlike the other three, the DTL_OUTAGE map is not generally * maintained; it's only computed when needed, typically to * determine whether a device can be detached. * * For leaf vdevs, DTL_MISSING and DTL_PARTIAL are identical: the device * either has the data or it doesn't. * * For interior vdevs such as mirror and RAID-Z the picture is more complex. * A vdev's DTL_PARTIAL is the union of its children's DTL_PARTIALs, because * if any child is less than fully replicated, then so is its parent. * A vdev's DTL_MISSING is a modified union of its children's DTL_MISSINGs, * comprising only those txgs which appear in 'maxfaults' or more children; * those are the txgs we don't have enough replication to read. For example, * double-parity RAID-Z can tolerate up to two missing devices (maxfaults == 2); * thus, its DTL_MISSING consists of the set of txgs that appear in more than * two child DTL_MISSING maps. * * It should be clear from the above that to compute the DTLs and outage maps * for all vdevs, it suffices to know just the leaf vdevs' DTL_MISSING maps. * Therefore, that is all we keep on disk. When loading the pool, or after * a configuration change, we generate all other DTLs from first principles. */ void vdev_dtl_dirty(vdev_t *vd, vdev_dtl_type_t t, uint64_t txg, uint64_t size) { range_tree_t *rt = vd->vdev_dtl[t]; ASSERT(t < DTL_TYPES); ASSERT(vd != vd->vdev_spa->spa_root_vdev); ASSERT(spa_writeable(vd->vdev_spa)); mutex_enter(rt->rt_lock); if (!range_tree_contains(rt, txg, size)) range_tree_add(rt, txg, size); mutex_exit(rt->rt_lock); } boolean_t vdev_dtl_contains(vdev_t *vd, vdev_dtl_type_t t, uint64_t txg, uint64_t size) { range_tree_t *rt = vd->vdev_dtl[t]; boolean_t dirty = B_FALSE; ASSERT(t < DTL_TYPES); ASSERT(vd != vd->vdev_spa->spa_root_vdev); mutex_enter(rt->rt_lock); if (range_tree_space(rt) != 0) dirty = range_tree_contains(rt, txg, size); mutex_exit(rt->rt_lock); return (dirty); } boolean_t vdev_dtl_empty(vdev_t *vd, vdev_dtl_type_t t) { range_tree_t *rt = vd->vdev_dtl[t]; boolean_t empty; mutex_enter(rt->rt_lock); empty = (range_tree_space(rt) == 0); mutex_exit(rt->rt_lock); return (empty); } /* * Returns the lowest txg in the DTL range. */ static uint64_t vdev_dtl_min(vdev_t *vd) { range_seg_t *rs; ASSERT(MUTEX_HELD(&vd->vdev_dtl_lock)); ASSERT3U(range_tree_space(vd->vdev_dtl[DTL_MISSING]), !=, 0); ASSERT0(vd->vdev_children); rs = avl_first(&vd->vdev_dtl[DTL_MISSING]->rt_root); return (rs->rs_start - 1); } /* * Returns the highest txg in the DTL. */ static uint64_t vdev_dtl_max(vdev_t *vd) { range_seg_t *rs; ASSERT(MUTEX_HELD(&vd->vdev_dtl_lock)); ASSERT3U(range_tree_space(vd->vdev_dtl[DTL_MISSING]), !=, 0); ASSERT0(vd->vdev_children); rs = avl_last(&vd->vdev_dtl[DTL_MISSING]->rt_root); return (rs->rs_end); } /* * Determine if a resilvering vdev should remove any DTL entries from * its range. If the vdev was resilvering for the entire duration of the * scan then it should excise that range from its DTLs. Otherwise, this * vdev is considered partially resilvered and should leave its DTL * entries intact. The comment in vdev_dtl_reassess() describes how we * excise the DTLs. */ static boolean_t vdev_dtl_should_excise(vdev_t *vd) { spa_t *spa = vd->vdev_spa; dsl_scan_t *scn = spa->spa_dsl_pool->dp_scan; ASSERT0(scn->scn_phys.scn_errors); ASSERT0(vd->vdev_children); if (vd->vdev_resilver_txg == 0 || range_tree_space(vd->vdev_dtl[DTL_MISSING]) == 0) return (B_TRUE); /* * When a resilver is initiated the scan will assign the scn_max_txg * value to the highest txg value that exists in all DTLs. If this * device's max DTL is not part of this scan (i.e. it is not in * the range (scn_min_txg, scn_max_txg] then it is not eligible * for excision. */ if (vdev_dtl_max(vd) <= scn->scn_phys.scn_max_txg) { ASSERT3U(scn->scn_phys.scn_min_txg, <=, vdev_dtl_min(vd)); ASSERT3U(scn->scn_phys.scn_min_txg, <, vd->vdev_resilver_txg); ASSERT3U(vd->vdev_resilver_txg, <=, scn->scn_phys.scn_max_txg); return (B_TRUE); } return (B_FALSE); } /* * Reassess DTLs after a config change or scrub completion. */ void vdev_dtl_reassess(vdev_t *vd, uint64_t txg, uint64_t scrub_txg, int scrub_done) { spa_t *spa = vd->vdev_spa; avl_tree_t reftree; int minref; ASSERT(spa_config_held(spa, SCL_ALL, RW_READER) != 0); for (int c = 0; c < vd->vdev_children; c++) vdev_dtl_reassess(vd->vdev_child[c], txg, scrub_txg, scrub_done); if (vd == spa->spa_root_vdev || vd->vdev_ishole || vd->vdev_aux) return; if (vd->vdev_ops->vdev_op_leaf) { dsl_scan_t *scn = spa->spa_dsl_pool->dp_scan; mutex_enter(&vd->vdev_dtl_lock); /* * If we've completed a scan cleanly then determine * if this vdev should remove any DTLs. We only want to * excise regions on vdevs that were available during * the entire duration of this scan. */ if (scrub_txg != 0 && (spa->spa_scrub_started || (scn != NULL && scn->scn_phys.scn_errors == 0)) && vdev_dtl_should_excise(vd)) { /* * We completed a scrub up to scrub_txg. If we * did it without rebooting, then the scrub dtl * will be valid, so excise the old region and * fold in the scrub dtl. Otherwise, leave the * dtl as-is if there was an error. * * There's little trick here: to excise the beginning * of the DTL_MISSING map, we put it into a reference * tree and then add a segment with refcnt -1 that * covers the range [0, scrub_txg). This means * that each txg in that range has refcnt -1 or 0. * We then add DTL_SCRUB with a refcnt of 2, so that * entries in the range [0, scrub_txg) will have a * positive refcnt -- either 1 or 2. We then convert * the reference tree into the new DTL_MISSING map. */ space_reftree_create(&reftree); space_reftree_add_map(&reftree, vd->vdev_dtl[DTL_MISSING], 1); space_reftree_add_seg(&reftree, 0, scrub_txg, -1); space_reftree_add_map(&reftree, vd->vdev_dtl[DTL_SCRUB], 2); space_reftree_generate_map(&reftree, vd->vdev_dtl[DTL_MISSING], 1); space_reftree_destroy(&reftree); } range_tree_vacate(vd->vdev_dtl[DTL_PARTIAL], NULL, NULL); range_tree_walk(vd->vdev_dtl[DTL_MISSING], range_tree_add, vd->vdev_dtl[DTL_PARTIAL]); if (scrub_done) range_tree_vacate(vd->vdev_dtl[DTL_SCRUB], NULL, NULL); range_tree_vacate(vd->vdev_dtl[DTL_OUTAGE], NULL, NULL); if (!vdev_readable(vd)) range_tree_add(vd->vdev_dtl[DTL_OUTAGE], 0, -1ULL); else range_tree_walk(vd->vdev_dtl[DTL_MISSING], range_tree_add, vd->vdev_dtl[DTL_OUTAGE]); /* * If the vdev was resilvering and no longer has any * DTLs then reset its resilvering flag and dirty * the top level so that we persist the change. */ if (vd->vdev_resilver_txg != 0 && range_tree_space(vd->vdev_dtl[DTL_MISSING]) == 0 && range_tree_space(vd->vdev_dtl[DTL_OUTAGE]) == 0) { vd->vdev_resilver_txg = 0; vdev_config_dirty(vd->vdev_top); } mutex_exit(&vd->vdev_dtl_lock); if (txg != 0) vdev_dirty(vd->vdev_top, VDD_DTL, vd, txg); return; } mutex_enter(&vd->vdev_dtl_lock); for (int t = 0; t < DTL_TYPES; t++) { /* account for child's outage in parent's missing map */ int s = (t == DTL_MISSING) ? DTL_OUTAGE: t; if (t == DTL_SCRUB) continue; /* leaf vdevs only */ if (t == DTL_PARTIAL) minref = 1; /* i.e. non-zero */ else if (vd->vdev_nparity != 0) minref = vd->vdev_nparity + 1; /* RAID-Z */ else minref = vd->vdev_children; /* any kind of mirror */ space_reftree_create(&reftree); for (int c = 0; c < vd->vdev_children; c++) { vdev_t *cvd = vd->vdev_child[c]; mutex_enter(&cvd->vdev_dtl_lock); space_reftree_add_map(&reftree, cvd->vdev_dtl[s], 1); mutex_exit(&cvd->vdev_dtl_lock); } space_reftree_generate_map(&reftree, vd->vdev_dtl[t], minref); space_reftree_destroy(&reftree); } mutex_exit(&vd->vdev_dtl_lock); } int vdev_dtl_load(vdev_t *vd) { spa_t *spa = vd->vdev_spa; objset_t *mos = spa->spa_meta_objset; int error = 0; if (vd->vdev_ops->vdev_op_leaf && vd->vdev_dtl_object != 0) { ASSERT(!vd->vdev_ishole); error = space_map_open(&vd->vdev_dtl_sm, mos, vd->vdev_dtl_object, 0, -1ULL, 0, &vd->vdev_dtl_lock); if (error) return (error); ASSERT(vd->vdev_dtl_sm != NULL); mutex_enter(&vd->vdev_dtl_lock); /* * Now that we've opened the space_map we need to update * the in-core DTL. */ space_map_update(vd->vdev_dtl_sm); error = space_map_load(vd->vdev_dtl_sm, vd->vdev_dtl[DTL_MISSING], SM_ALLOC); mutex_exit(&vd->vdev_dtl_lock); return (error); } for (int c = 0; c < vd->vdev_children; c++) { error = vdev_dtl_load(vd->vdev_child[c]); if (error != 0) break; } return (error); } void vdev_dtl_sync(vdev_t *vd, uint64_t txg) { spa_t *spa = vd->vdev_spa; range_tree_t *rt = vd->vdev_dtl[DTL_MISSING]; objset_t *mos = spa->spa_meta_objset; range_tree_t *rtsync; kmutex_t rtlock; dmu_tx_t *tx; uint64_t object = space_map_object(vd->vdev_dtl_sm); ASSERT(!vd->vdev_ishole); ASSERT(vd->vdev_ops->vdev_op_leaf); tx = dmu_tx_create_assigned(spa->spa_dsl_pool, txg); if (vd->vdev_detached || vd->vdev_top->vdev_removing) { mutex_enter(&vd->vdev_dtl_lock); space_map_free(vd->vdev_dtl_sm, tx); space_map_close(vd->vdev_dtl_sm); vd->vdev_dtl_sm = NULL; mutex_exit(&vd->vdev_dtl_lock); dmu_tx_commit(tx); return; } if (vd->vdev_dtl_sm == NULL) { uint64_t new_object; new_object = space_map_alloc(mos, tx); VERIFY3U(new_object, !=, 0); VERIFY0(space_map_open(&vd->vdev_dtl_sm, mos, new_object, 0, -1ULL, 0, &vd->vdev_dtl_lock)); ASSERT(vd->vdev_dtl_sm != NULL); } bzero(&rtlock, sizeof(rtlock)); mutex_init(&rtlock, NULL, MUTEX_DEFAULT, NULL); rtsync = range_tree_create(NULL, NULL, &rtlock); mutex_enter(&rtlock); mutex_enter(&vd->vdev_dtl_lock); range_tree_walk(rt, range_tree_add, rtsync); mutex_exit(&vd->vdev_dtl_lock); space_map_truncate(vd->vdev_dtl_sm, tx); space_map_write(vd->vdev_dtl_sm, rtsync, SM_ALLOC, tx); range_tree_vacate(rtsync, NULL, NULL); range_tree_destroy(rtsync); mutex_exit(&rtlock); mutex_destroy(&rtlock); /* * If the object for the space map has changed then dirty * the top level so that we update the config. */ if (object != space_map_object(vd->vdev_dtl_sm)) { zfs_dbgmsg("txg %llu, spa %s, DTL old object %llu, " "new object %llu", txg, spa_name(spa), object, space_map_object(vd->vdev_dtl_sm)); vdev_config_dirty(vd->vdev_top); } dmu_tx_commit(tx); mutex_enter(&vd->vdev_dtl_lock); space_map_update(vd->vdev_dtl_sm); mutex_exit(&vd->vdev_dtl_lock); } /* * Determine whether the specified vdev can be offlined/detached/removed * without losing data. */ boolean_t vdev_dtl_required(vdev_t *vd) { spa_t *spa = vd->vdev_spa; vdev_t *tvd = vd->vdev_top; uint8_t cant_read = vd->vdev_cant_read; boolean_t required; ASSERT(spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); if (vd == spa->spa_root_vdev || vd == tvd) return (B_TRUE); /* * Temporarily mark the device as unreadable, and then determine * whether this results in any DTL outages in the top-level vdev. * If not, we can safely offline/detach/remove the device. */ vd->vdev_cant_read = B_TRUE; vdev_dtl_reassess(tvd, 0, 0, B_FALSE); required = !vdev_dtl_empty(tvd, DTL_OUTAGE); vd->vdev_cant_read = cant_read; vdev_dtl_reassess(tvd, 0, 0, B_FALSE); if (!required && zio_injection_enabled) required = !!zio_handle_device_injection(vd, NULL, ECHILD); return (required); } /* * Determine if resilver is needed, and if so the txg range. */ boolean_t vdev_resilver_needed(vdev_t *vd, uint64_t *minp, uint64_t *maxp) { boolean_t needed = B_FALSE; uint64_t thismin = UINT64_MAX; uint64_t thismax = 0; if (vd->vdev_children == 0) { mutex_enter(&vd->vdev_dtl_lock); if (range_tree_space(vd->vdev_dtl[DTL_MISSING]) != 0 && vdev_writeable(vd)) { thismin = vdev_dtl_min(vd); thismax = vdev_dtl_max(vd); needed = B_TRUE; } mutex_exit(&vd->vdev_dtl_lock); } else { for (int c = 0; c < vd->vdev_children; c++) { vdev_t *cvd = vd->vdev_child[c]; uint64_t cmin, cmax; if (vdev_resilver_needed(cvd, &cmin, &cmax)) { thismin = MIN(thismin, cmin); thismax = MAX(thismax, cmax); needed = B_TRUE; } } } if (needed && minp) { *minp = thismin; *maxp = thismax; } return (needed); } void vdev_load(vdev_t *vd) { /* * Recursively load all children. */ for (int c = 0; c < vd->vdev_children; c++) vdev_load(vd->vdev_child[c]); /* * If this is a top-level vdev, initialize its metaslabs. */ if (vd == vd->vdev_top && !vd->vdev_ishole && (vd->vdev_ashift == 0 || vd->vdev_asize == 0 || vdev_metaslab_init(vd, 0) != 0)) vdev_set_state(vd, B_FALSE, VDEV_STATE_CANT_OPEN, VDEV_AUX_CORRUPT_DATA); /* * If this is a leaf vdev, load its DTL. */ if (vd->vdev_ops->vdev_op_leaf && vdev_dtl_load(vd) != 0) vdev_set_state(vd, B_FALSE, VDEV_STATE_CANT_OPEN, VDEV_AUX_CORRUPT_DATA); } /* * The special vdev case is used for hot spares and l2cache devices. Its * sole purpose it to set the vdev state for the associated vdev. To do this, * we make sure that we can open the underlying device, then try to read the * label, and make sure that the label is sane and that it hasn't been * repurposed to another pool. */ int vdev_validate_aux(vdev_t *vd) { nvlist_t *label; uint64_t guid, version; uint64_t state; if (!vdev_readable(vd)) return (0); if ((label = vdev_label_read_config(vd, -1ULL)) == NULL) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_CORRUPT_DATA); return (-1); } if (nvlist_lookup_uint64(label, ZPOOL_CONFIG_VERSION, &version) != 0 || !SPA_VERSION_IS_SUPPORTED(version) || nvlist_lookup_uint64(label, ZPOOL_CONFIG_GUID, &guid) != 0 || guid != vd->vdev_guid || nvlist_lookup_uint64(label, ZPOOL_CONFIG_POOL_STATE, &state) != 0) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_CORRUPT_DATA); nvlist_free(label); return (-1); } /* * We don't actually check the pool state here. If it's in fact in * use by another pool, we update this fact on the fly when requested. */ nvlist_free(label); return (0); } void vdev_remove(vdev_t *vd, uint64_t txg) { spa_t *spa = vd->vdev_spa; objset_t *mos = spa->spa_meta_objset; dmu_tx_t *tx; tx = dmu_tx_create_assigned(spa_get_dsl(spa), txg); if (vd->vdev_ms != NULL) { metaslab_group_t *mg = vd->vdev_mg; metaslab_group_histogram_verify(mg); metaslab_class_histogram_verify(mg->mg_class); for (int m = 0; m < vd->vdev_ms_count; m++) { metaslab_t *msp = vd->vdev_ms[m]; if (msp == NULL || msp->ms_sm == NULL) continue; mutex_enter(&msp->ms_lock); /* * If the metaslab was not loaded when the vdev * was removed then the histogram accounting may * not be accurate. Update the histogram information * here so that we ensure that the metaslab group * and metaslab class are up-to-date. */ metaslab_group_histogram_remove(mg, msp); VERIFY0(space_map_allocated(msp->ms_sm)); space_map_free(msp->ms_sm, tx); space_map_close(msp->ms_sm); msp->ms_sm = NULL; mutex_exit(&msp->ms_lock); } metaslab_group_histogram_verify(mg); metaslab_class_histogram_verify(mg->mg_class); for (int i = 0; i < RANGE_TREE_HISTOGRAM_SIZE; i++) ASSERT0(mg->mg_histogram[i]); } if (vd->vdev_ms_array) { (void) dmu_object_free(mos, vd->vdev_ms_array, tx); vd->vdev_ms_array = 0; } dmu_tx_commit(tx); } void vdev_sync_done(vdev_t *vd, uint64_t txg) { metaslab_t *msp; boolean_t reassess = !txg_list_empty(&vd->vdev_ms_list, TXG_CLEAN(txg)); ASSERT(!vd->vdev_ishole); while (msp = txg_list_remove(&vd->vdev_ms_list, TXG_CLEAN(txg))) metaslab_sync_done(msp, txg); if (reassess) metaslab_sync_reassess(vd->vdev_mg); } void vdev_sync(vdev_t *vd, uint64_t txg) { spa_t *spa = vd->vdev_spa; vdev_t *lvd; metaslab_t *msp; dmu_tx_t *tx; ASSERT(!vd->vdev_ishole); if (vd->vdev_ms_array == 0 && vd->vdev_ms_shift != 0) { ASSERT(vd == vd->vdev_top); tx = dmu_tx_create_assigned(spa->spa_dsl_pool, txg); vd->vdev_ms_array = dmu_object_alloc(spa->spa_meta_objset, DMU_OT_OBJECT_ARRAY, 0, DMU_OT_NONE, 0, tx); ASSERT(vd->vdev_ms_array != 0); vdev_config_dirty(vd); dmu_tx_commit(tx); } /* * Remove the metadata associated with this vdev once it's empty. */ if (vd->vdev_stat.vs_alloc == 0 && vd->vdev_removing) vdev_remove(vd, txg); while ((msp = txg_list_remove(&vd->vdev_ms_list, txg)) != NULL) { metaslab_sync(msp, txg); (void) txg_list_add(&vd->vdev_ms_list, msp, TXG_CLEAN(txg)); } while ((lvd = txg_list_remove(&vd->vdev_dtl_list, txg)) != NULL) vdev_dtl_sync(lvd, txg); (void) txg_list_add(&spa->spa_vdev_txg_list, vd, TXG_CLEAN(txg)); } uint64_t vdev_psize_to_asize(vdev_t *vd, uint64_t psize) { return (vd->vdev_ops->vdev_op_asize(vd, psize)); } /* * Mark the given vdev faulted. A faulted vdev behaves as if the device could * not be opened, and no I/O is attempted. */ int vdev_fault(spa_t *spa, uint64_t guid, vdev_aux_t aux) { vdev_t *vd, *tvd; spa_vdev_state_enter(spa, SCL_NONE); if ((vd = spa_lookup_by_guid(spa, guid, B_TRUE)) == NULL) return (spa_vdev_state_exit(spa, NULL, ENODEV)); if (!vd->vdev_ops->vdev_op_leaf) return (spa_vdev_state_exit(spa, NULL, ENOTSUP)); tvd = vd->vdev_top; /* * We don't directly use the aux state here, but if we do a * vdev_reopen(), we need this value to be present to remember why we * were faulted. */ vd->vdev_label_aux = aux; /* * Faulted state takes precedence over degraded. */ vd->vdev_delayed_close = B_FALSE; vd->vdev_faulted = 1ULL; vd->vdev_degraded = 0ULL; vdev_set_state(vd, B_FALSE, VDEV_STATE_FAULTED, aux); /* * If this device has the only valid copy of the data, then * back off and simply mark the vdev as degraded instead. */ if (!tvd->vdev_islog && vd->vdev_aux == NULL && vdev_dtl_required(vd)) { vd->vdev_degraded = 1ULL; vd->vdev_faulted = 0ULL; /* * If we reopen the device and it's not dead, only then do we * mark it degraded. */ vdev_reopen(tvd); if (vdev_readable(vd)) vdev_set_state(vd, B_FALSE, VDEV_STATE_DEGRADED, aux); } return (spa_vdev_state_exit(spa, vd, 0)); } /* * Mark the given vdev degraded. A degraded vdev is purely an indication to the * user that something is wrong. The vdev continues to operate as normal as far * as I/O is concerned. */ int vdev_degrade(spa_t *spa, uint64_t guid, vdev_aux_t aux) { vdev_t *vd; spa_vdev_state_enter(spa, SCL_NONE); if ((vd = spa_lookup_by_guid(spa, guid, B_TRUE)) == NULL) return (spa_vdev_state_exit(spa, NULL, ENODEV)); if (!vd->vdev_ops->vdev_op_leaf) return (spa_vdev_state_exit(spa, NULL, ENOTSUP)); /* * If the vdev is already faulted, then don't do anything. */ if (vd->vdev_faulted || vd->vdev_degraded) return (spa_vdev_state_exit(spa, NULL, 0)); vd->vdev_degraded = 1ULL; if (!vdev_is_dead(vd)) vdev_set_state(vd, B_FALSE, VDEV_STATE_DEGRADED, aux); return (spa_vdev_state_exit(spa, vd, 0)); } /* * Online the given vdev. * * If 'ZFS_ONLINE_UNSPARE' is set, it implies two things. First, any attached * spare device should be detached when the device finishes resilvering. * Second, the online should be treated like a 'test' online case, so no FMA * events are generated if the device fails to open. */ int vdev_online(spa_t *spa, uint64_t guid, uint64_t flags, vdev_state_t *newstate) { vdev_t *vd, *tvd, *pvd, *rvd = spa->spa_root_vdev; spa_vdev_state_enter(spa, SCL_NONE); if ((vd = spa_lookup_by_guid(spa, guid, B_TRUE)) == NULL) return (spa_vdev_state_exit(spa, NULL, ENODEV)); if (!vd->vdev_ops->vdev_op_leaf) return (spa_vdev_state_exit(spa, NULL, ENOTSUP)); tvd = vd->vdev_top; vd->vdev_offline = B_FALSE; vd->vdev_tmpoffline = B_FALSE; vd->vdev_checkremove = !!(flags & ZFS_ONLINE_CHECKREMOVE); vd->vdev_forcefault = !!(flags & ZFS_ONLINE_FORCEFAULT); /* XXX - L2ARC 1.0 does not support expansion */ if (!vd->vdev_aux) { for (pvd = vd; pvd != rvd; pvd = pvd->vdev_parent) pvd->vdev_expanding = !!(flags & ZFS_ONLINE_EXPAND); } vdev_reopen(tvd); vd->vdev_checkremove = vd->vdev_forcefault = B_FALSE; if (!vd->vdev_aux) { for (pvd = vd; pvd != rvd; pvd = pvd->vdev_parent) pvd->vdev_expanding = B_FALSE; } if (newstate) *newstate = vd->vdev_state; if ((flags & ZFS_ONLINE_UNSPARE) && !vdev_is_dead(vd) && vd->vdev_parent && vd->vdev_parent->vdev_ops == &vdev_spare_ops && vd->vdev_parent->vdev_child[0] == vd) vd->vdev_unspare = B_TRUE; if ((flags & ZFS_ONLINE_EXPAND) || spa->spa_autoexpand) { /* XXX - L2ARC 1.0 does not support expansion */ if (vd->vdev_aux) return (spa_vdev_state_exit(spa, vd, ENOTSUP)); spa_async_request(spa, SPA_ASYNC_CONFIG_UPDATE); } return (spa_vdev_state_exit(spa, vd, 0)); } static int vdev_offline_locked(spa_t *spa, uint64_t guid, uint64_t flags) { vdev_t *vd, *tvd; int error = 0; uint64_t generation; metaslab_group_t *mg; top: spa_vdev_state_enter(spa, SCL_ALLOC); if ((vd = spa_lookup_by_guid(spa, guid, B_TRUE)) == NULL) return (spa_vdev_state_exit(spa, NULL, ENODEV)); if (!vd->vdev_ops->vdev_op_leaf) return (spa_vdev_state_exit(spa, NULL, ENOTSUP)); tvd = vd->vdev_top; mg = tvd->vdev_mg; generation = spa->spa_config_generation + 1; /* * If the device isn't already offline, try to offline it. */ if (!vd->vdev_offline) { /* * If this device has the only valid copy of some data, * don't allow it to be offlined. Log devices are always * expendable. */ if (!tvd->vdev_islog && vd->vdev_aux == NULL && vdev_dtl_required(vd)) return (spa_vdev_state_exit(spa, NULL, EBUSY)); /* * If the top-level is a slog and it has had allocations * then proceed. We check that the vdev's metaslab group * is not NULL since it's possible that we may have just * added this vdev but not yet initialized its metaslabs. */ if (tvd->vdev_islog && mg != NULL) { /* * Prevent any future allocations. */ metaslab_group_passivate(mg); (void) spa_vdev_state_exit(spa, vd, 0); error = spa_offline_log(spa); spa_vdev_state_enter(spa, SCL_ALLOC); /* * Check to see if the config has changed. */ if (error || generation != spa->spa_config_generation) { metaslab_group_activate(mg); if (error) return (spa_vdev_state_exit(spa, vd, error)); (void) spa_vdev_state_exit(spa, vd, 0); goto top; } ASSERT0(tvd->vdev_stat.vs_alloc); } /* * Offline this device and reopen its top-level vdev. * If the top-level vdev is a log device then just offline * it. Otherwise, if this action results in the top-level * vdev becoming unusable, undo it and fail the request. */ vd->vdev_offline = B_TRUE; vdev_reopen(tvd); if (!tvd->vdev_islog && vd->vdev_aux == NULL && vdev_is_dead(tvd)) { vd->vdev_offline = B_FALSE; vdev_reopen(tvd); return (spa_vdev_state_exit(spa, NULL, EBUSY)); } /* * Add the device back into the metaslab rotor so that * once we online the device it's open for business. */ if (tvd->vdev_islog && mg != NULL) metaslab_group_activate(mg); } vd->vdev_tmpoffline = !!(flags & ZFS_OFFLINE_TEMPORARY); return (spa_vdev_state_exit(spa, vd, 0)); } int vdev_offline(spa_t *spa, uint64_t guid, uint64_t flags) { int error; mutex_enter(&spa->spa_vdev_top_lock); error = vdev_offline_locked(spa, guid, flags); mutex_exit(&spa->spa_vdev_top_lock); return (error); } /* * Clear the error counts associated with this vdev. Unlike vdev_online() and * vdev_offline(), we assume the spa config is locked. We also clear all * children. If 'vd' is NULL, then the user wants to clear all vdevs. */ void vdev_clear(spa_t *spa, vdev_t *vd) { vdev_t *rvd = spa->spa_root_vdev; ASSERT(spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); if (vd == NULL) vd = rvd; vd->vdev_stat.vs_read_errors = 0; vd->vdev_stat.vs_write_errors = 0; vd->vdev_stat.vs_checksum_errors = 0; for (int c = 0; c < vd->vdev_children; c++) vdev_clear(spa, vd->vdev_child[c]); if (vd == rvd) { for (int c = 0; c < spa->spa_l2cache.sav_count; c++) vdev_clear(spa, spa->spa_l2cache.sav_vdevs[c]); for (int c = 0; c < spa->spa_spares.sav_count; c++) vdev_clear(spa, spa->spa_spares.sav_vdevs[c]); } /* * If we're in the FAULTED state or have experienced failed I/O, then * clear the persistent state and attempt to reopen the device. We * also mark the vdev config dirty, so that the new faulted state is * written out to disk. */ if (vd->vdev_faulted || vd->vdev_degraded || !vdev_readable(vd) || !vdev_writeable(vd)) { /* * When reopening in reponse to a clear event, it may be due to * a fmadm repair request. In this case, if the device is * still broken, we want to still post the ereport again. */ vd->vdev_forcefault = B_TRUE; vd->vdev_faulted = vd->vdev_degraded = 0ULL; vd->vdev_cant_read = B_FALSE; vd->vdev_cant_write = B_FALSE; vdev_reopen(vd == rvd ? rvd : vd->vdev_top); vd->vdev_forcefault = B_FALSE; if (vd != rvd && vdev_writeable(vd->vdev_top)) vdev_state_dirty(vd->vdev_top); if (vd->vdev_aux == NULL && !vdev_is_dead(vd)) spa_async_request(spa, SPA_ASYNC_RESILVER); spa_event_notify(spa, vd, ESC_ZFS_VDEV_CLEAR); } /* * When clearing a FMA-diagnosed fault, we always want to * unspare the device, as we assume that the original spare was * done in response to the FMA fault. */ if (!vdev_is_dead(vd) && vd->vdev_parent != NULL && vd->vdev_parent->vdev_ops == &vdev_spare_ops && vd->vdev_parent->vdev_child[0] == vd) vd->vdev_unspare = B_TRUE; } boolean_t vdev_is_dead(vdev_t *vd) { /* * Holes and missing devices are always considered "dead". * This simplifies the code since we don't have to check for * these types of devices in the various code paths. * Instead we rely on the fact that we skip over dead devices * before issuing I/O to them. */ return (vd->vdev_state < VDEV_STATE_DEGRADED || vd->vdev_ishole || vd->vdev_ops == &vdev_missing_ops); } boolean_t vdev_readable(vdev_t *vd) { return (!vdev_is_dead(vd) && !vd->vdev_cant_read); } boolean_t vdev_writeable(vdev_t *vd) { return (!vdev_is_dead(vd) && !vd->vdev_cant_write); } boolean_t vdev_allocatable(vdev_t *vd) { uint64_t state = vd->vdev_state; /* * We currently allow allocations from vdevs which may be in the * process of reopening (i.e. VDEV_STATE_CLOSED). If the device * fails to reopen then we'll catch it later when we're holding * the proper locks. Note that we have to get the vdev state * in a local variable because although it changes atomically, * we're asking two separate questions about it. */ return (!(state < VDEV_STATE_DEGRADED && state != VDEV_STATE_CLOSED) && !vd->vdev_cant_write && !vd->vdev_ishole); } boolean_t vdev_accessible(vdev_t *vd, zio_t *zio) { ASSERT(zio->io_vd == vd); if (vdev_is_dead(vd) || vd->vdev_remove_wanted) return (B_FALSE); if (zio->io_type == ZIO_TYPE_READ) return (!vd->vdev_cant_read); if (zio->io_type == ZIO_TYPE_WRITE) return (!vd->vdev_cant_write); return (B_TRUE); } /* * Get statistics for the given vdev. */ void vdev_get_stats(vdev_t *vd, vdev_stat_t *vs) { spa_t *spa = vd->vdev_spa; vdev_t *rvd = spa->spa_root_vdev; ASSERT(spa_config_held(spa, SCL_ALL, RW_READER) != 0); mutex_enter(&vd->vdev_stat_lock); bcopy(&vd->vdev_stat, vs, sizeof (*vs)); vs->vs_timestamp = gethrtime() - vs->vs_timestamp; vs->vs_state = vd->vdev_state; vs->vs_rsize = vdev_get_min_asize(vd); if (vd->vdev_ops->vdev_op_leaf) vs->vs_rsize += VDEV_LABEL_START_SIZE + VDEV_LABEL_END_SIZE; vs->vs_esize = vd->vdev_max_asize - vd->vdev_asize; vs->vs_configured_ashift = vd->vdev_top != NULL ? vd->vdev_top->vdev_ashift : vd->vdev_ashift; vs->vs_logical_ashift = vd->vdev_logical_ashift; vs->vs_physical_ashift = vd->vdev_physical_ashift; if (vd->vdev_aux == NULL && vd == vd->vdev_top && !vd->vdev_ishole) { vs->vs_fragmentation = vd->vdev_mg->mg_fragmentation; } /* * If we're getting stats on the root vdev, aggregate the I/O counts * over all top-level vdevs (i.e. the direct children of the root). */ if (vd == rvd) { for (int c = 0; c < rvd->vdev_children; c++) { vdev_t *cvd = rvd->vdev_child[c]; vdev_stat_t *cvs = &cvd->vdev_stat; for (int t = 0; t < ZIO_TYPES; t++) { vs->vs_ops[t] += cvs->vs_ops[t]; vs->vs_bytes[t] += cvs->vs_bytes[t]; } cvs->vs_scan_removing = cvd->vdev_removing; } } mutex_exit(&vd->vdev_stat_lock); } void vdev_clear_stats(vdev_t *vd) { mutex_enter(&vd->vdev_stat_lock); vd->vdev_stat.vs_space = 0; vd->vdev_stat.vs_dspace = 0; vd->vdev_stat.vs_alloc = 0; mutex_exit(&vd->vdev_stat_lock); } void vdev_scan_stat_init(vdev_t *vd) { vdev_stat_t *vs = &vd->vdev_stat; for (int c = 0; c < vd->vdev_children; c++) vdev_scan_stat_init(vd->vdev_child[c]); mutex_enter(&vd->vdev_stat_lock); vs->vs_scan_processed = 0; mutex_exit(&vd->vdev_stat_lock); } void vdev_stat_update(zio_t *zio, uint64_t psize) { spa_t *spa = zio->io_spa; vdev_t *rvd = spa->spa_root_vdev; vdev_t *vd = zio->io_vd ? zio->io_vd : rvd; vdev_t *pvd; uint64_t txg = zio->io_txg; vdev_stat_t *vs = &vd->vdev_stat; zio_type_t type = zio->io_type; int flags = zio->io_flags; /* * If this i/o is a gang leader, it didn't do any actual work. */ if (zio->io_gang_tree) return; if (zio->io_error == 0) { /* * If this is a root i/o, don't count it -- we've already * counted the top-level vdevs, and vdev_get_stats() will * aggregate them when asked. This reduces contention on * the root vdev_stat_lock and implicitly handles blocks * that compress away to holes, for which there is no i/o. * (Holes never create vdev children, so all the counters * remain zero, which is what we want.) * * Note: this only applies to successful i/o (io_error == 0) * because unlike i/o counts, errors are not additive. * When reading a ditto block, for example, failure of * one top-level vdev does not imply a root-level error. */ if (vd == rvd) return; ASSERT(vd == zio->io_vd); if (flags & ZIO_FLAG_IO_BYPASS) return; mutex_enter(&vd->vdev_stat_lock); if (flags & ZIO_FLAG_IO_REPAIR) { if (flags & ZIO_FLAG_SCAN_THREAD) { dsl_scan_phys_t *scn_phys = &spa->spa_dsl_pool->dp_scan->scn_phys; uint64_t *processed = &scn_phys->scn_processed; /* XXX cleanup? */ if (vd->vdev_ops->vdev_op_leaf) atomic_add_64(processed, psize); vs->vs_scan_processed += psize; } if (flags & ZIO_FLAG_SELF_HEAL) vs->vs_self_healed += psize; } vs->vs_ops[type]++; vs->vs_bytes[type] += psize; mutex_exit(&vd->vdev_stat_lock); return; } if (flags & ZIO_FLAG_SPECULATIVE) return; /* * If this is an I/O error that is going to be retried, then ignore the * error. Otherwise, the user may interpret B_FAILFAST I/O errors as * hard errors, when in reality they can happen for any number of * innocuous reasons (bus resets, MPxIO link failure, etc). */ if (zio->io_error == EIO && !(zio->io_flags & ZIO_FLAG_IO_RETRY)) return; /* * Intent logs writes won't propagate their error to the root * I/O so don't mark these types of failures as pool-level * errors. */ if (zio->io_vd == NULL && (zio->io_flags & ZIO_FLAG_DONT_PROPAGATE)) return; mutex_enter(&vd->vdev_stat_lock); if (type == ZIO_TYPE_READ && !vdev_is_dead(vd)) { if (zio->io_error == ECKSUM) vs->vs_checksum_errors++; else vs->vs_read_errors++; } if (type == ZIO_TYPE_WRITE && !vdev_is_dead(vd)) vs->vs_write_errors++; mutex_exit(&vd->vdev_stat_lock); if (type == ZIO_TYPE_WRITE && txg != 0 && (!(flags & ZIO_FLAG_IO_REPAIR) || (flags & ZIO_FLAG_SCAN_THREAD) || spa->spa_claiming)) { /* * This is either a normal write (not a repair), or it's * a repair induced by the scrub thread, or it's a repair * made by zil_claim() during spa_load() in the first txg. * In the normal case, we commit the DTL change in the same * txg as the block was born. In the scrub-induced repair * case, we know that scrubs run in first-pass syncing context, * so we commit the DTL change in spa_syncing_txg(spa). * In the zil_claim() case, we commit in spa_first_txg(spa). * * We currently do not make DTL entries for failed spontaneous * self-healing writes triggered by normal (non-scrubbing) * reads, because we have no transactional context in which to * do so -- and it's not clear that it'd be desirable anyway. */ if (vd->vdev_ops->vdev_op_leaf) { uint64_t commit_txg = txg; if (flags & ZIO_FLAG_SCAN_THREAD) { ASSERT(flags & ZIO_FLAG_IO_REPAIR); ASSERT(spa_sync_pass(spa) == 1); vdev_dtl_dirty(vd, DTL_SCRUB, txg, 1); commit_txg = spa_syncing_txg(spa); } else if (spa->spa_claiming) { ASSERT(flags & ZIO_FLAG_IO_REPAIR); commit_txg = spa_first_txg(spa); } ASSERT(commit_txg >= spa_syncing_txg(spa)); if (vdev_dtl_contains(vd, DTL_MISSING, txg, 1)) return; for (pvd = vd; pvd != rvd; pvd = pvd->vdev_parent) vdev_dtl_dirty(pvd, DTL_PARTIAL, txg, 1); vdev_dirty(vd->vdev_top, VDD_DTL, vd, commit_txg); } if (vd != rvd) vdev_dtl_dirty(vd, DTL_MISSING, txg, 1); } } /* * Update the in-core space usage stats for this vdev, its metaslab class, * and the root vdev. */ void vdev_space_update(vdev_t *vd, int64_t alloc_delta, int64_t defer_delta, int64_t space_delta) { int64_t dspace_delta = space_delta; spa_t *spa = vd->vdev_spa; vdev_t *rvd = spa->spa_root_vdev; metaslab_group_t *mg = vd->vdev_mg; metaslab_class_t *mc = mg ? mg->mg_class : NULL; ASSERT(vd == vd->vdev_top); /* * Apply the inverse of the psize-to-asize (ie. RAID-Z) space-expansion * factor. We must calculate this here and not at the root vdev * because the root vdev's psize-to-asize is simply the max of its * childrens', thus not accurate enough for us. */ ASSERT((dspace_delta & (SPA_MINBLOCKSIZE-1)) == 0); ASSERT(vd->vdev_deflate_ratio != 0 || vd->vdev_isl2cache); dspace_delta = (dspace_delta >> SPA_MINBLOCKSHIFT) * vd->vdev_deflate_ratio; mutex_enter(&vd->vdev_stat_lock); vd->vdev_stat.vs_alloc += alloc_delta; vd->vdev_stat.vs_space += space_delta; vd->vdev_stat.vs_dspace += dspace_delta; mutex_exit(&vd->vdev_stat_lock); if (mc == spa_normal_class(spa)) { mutex_enter(&rvd->vdev_stat_lock); rvd->vdev_stat.vs_alloc += alloc_delta; rvd->vdev_stat.vs_space += space_delta; rvd->vdev_stat.vs_dspace += dspace_delta; mutex_exit(&rvd->vdev_stat_lock); } if (mc != NULL) { ASSERT(rvd == vd->vdev_parent); ASSERT(vd->vdev_ms_count != 0); metaslab_class_space_update(mc, alloc_delta, defer_delta, space_delta, dspace_delta); } } /* * Mark a top-level vdev's config as dirty, placing it on the dirty list * so that it will be written out next time the vdev configuration is synced. * If the root vdev is specified (vdev_top == NULL), dirty all top-level vdevs. */ void vdev_config_dirty(vdev_t *vd) { spa_t *spa = vd->vdev_spa; vdev_t *rvd = spa->spa_root_vdev; int c; ASSERT(spa_writeable(spa)); /* * If this is an aux vdev (as with l2cache and spare devices), then we * update the vdev config manually and set the sync flag. */ if (vd->vdev_aux != NULL) { spa_aux_vdev_t *sav = vd->vdev_aux; nvlist_t **aux; uint_t naux; for (c = 0; c < sav->sav_count; c++) { if (sav->sav_vdevs[c] == vd) break; } if (c == sav->sav_count) { /* * We're being removed. There's nothing more to do. */ ASSERT(sav->sav_sync == B_TRUE); return; } sav->sav_sync = B_TRUE; if (nvlist_lookup_nvlist_array(sav->sav_config, ZPOOL_CONFIG_L2CACHE, &aux, &naux) != 0) { VERIFY(nvlist_lookup_nvlist_array(sav->sav_config, ZPOOL_CONFIG_SPARES, &aux, &naux) == 0); } ASSERT(c < naux); /* * Setting the nvlist in the middle if the array is a little * sketchy, but it will work. */ nvlist_free(aux[c]); aux[c] = vdev_config_generate(spa, vd, B_TRUE, 0); return; } /* * The dirty list is protected by the SCL_CONFIG lock. The caller * must either hold SCL_CONFIG as writer, or must be the sync thread * (which holds SCL_CONFIG as reader). There's only one sync thread, * so this is sufficient to ensure mutual exclusion. */ ASSERT(spa_config_held(spa, SCL_CONFIG, RW_WRITER) || (dsl_pool_sync_context(spa_get_dsl(spa)) && spa_config_held(spa, SCL_CONFIG, RW_READER))); if (vd == rvd) { for (c = 0; c < rvd->vdev_children; c++) vdev_config_dirty(rvd->vdev_child[c]); } else { ASSERT(vd == vd->vdev_top); if (!list_link_active(&vd->vdev_config_dirty_node) && !vd->vdev_ishole) list_insert_head(&spa->spa_config_dirty_list, vd); } } void vdev_config_clean(vdev_t *vd) { spa_t *spa = vd->vdev_spa; ASSERT(spa_config_held(spa, SCL_CONFIG, RW_WRITER) || (dsl_pool_sync_context(spa_get_dsl(spa)) && spa_config_held(spa, SCL_CONFIG, RW_READER))); ASSERT(list_link_active(&vd->vdev_config_dirty_node)); list_remove(&spa->spa_config_dirty_list, vd); } /* * Mark a top-level vdev's state as dirty, so that the next pass of * spa_sync() can convert this into vdev_config_dirty(). We distinguish * the state changes from larger config changes because they require * much less locking, and are often needed for administrative actions. */ void vdev_state_dirty(vdev_t *vd) { spa_t *spa = vd->vdev_spa; ASSERT(spa_writeable(spa)); ASSERT(vd == vd->vdev_top); /* * The state list is protected by the SCL_STATE lock. The caller * must either hold SCL_STATE as writer, or must be the sync thread * (which holds SCL_STATE as reader). There's only one sync thread, * so this is sufficient to ensure mutual exclusion. */ ASSERT(spa_config_held(spa, SCL_STATE, RW_WRITER) || (dsl_pool_sync_context(spa_get_dsl(spa)) && spa_config_held(spa, SCL_STATE, RW_READER))); if (!list_link_active(&vd->vdev_state_dirty_node) && !vd->vdev_ishole) list_insert_head(&spa->spa_state_dirty_list, vd); } void vdev_state_clean(vdev_t *vd) { spa_t *spa = vd->vdev_spa; ASSERT(spa_config_held(spa, SCL_STATE, RW_WRITER) || (dsl_pool_sync_context(spa_get_dsl(spa)) && spa_config_held(spa, SCL_STATE, RW_READER))); ASSERT(list_link_active(&vd->vdev_state_dirty_node)); list_remove(&spa->spa_state_dirty_list, vd); } /* * Propagate vdev state up from children to parent. */ void vdev_propagate_state(vdev_t *vd) { spa_t *spa = vd->vdev_spa; vdev_t *rvd = spa->spa_root_vdev; int degraded = 0, faulted = 0; int corrupted = 0; vdev_t *child; if (vd->vdev_children > 0) { for (int c = 0; c < vd->vdev_children; c++) { child = vd->vdev_child[c]; /* * Don't factor holes into the decision. */ if (child->vdev_ishole) continue; if (!vdev_readable(child) || (!vdev_writeable(child) && spa_writeable(spa))) { /* * Root special: if there is a top-level log * device, treat the root vdev as if it were * degraded. */ if (child->vdev_islog && vd == rvd) degraded++; else faulted++; } else if (child->vdev_state <= VDEV_STATE_DEGRADED) { degraded++; } if (child->vdev_stat.vs_aux == VDEV_AUX_CORRUPT_DATA) corrupted++; } vd->vdev_ops->vdev_op_state_change(vd, faulted, degraded); /* * Root special: if there is a top-level vdev that cannot be * opened due to corrupted metadata, then propagate the root * vdev's aux state as 'corrupt' rather than 'insufficient * replicas'. */ if (corrupted && vd == rvd && rvd->vdev_state == VDEV_STATE_CANT_OPEN) vdev_set_state(rvd, B_FALSE, VDEV_STATE_CANT_OPEN, VDEV_AUX_CORRUPT_DATA); } if (vd->vdev_parent) vdev_propagate_state(vd->vdev_parent); } /* * Set a vdev's state. If this is during an open, we don't update the parent * state, because we're in the process of opening children depth-first. * Otherwise, we propagate the change to the parent. * * If this routine places a device in a faulted state, an appropriate ereport is * generated. */ void vdev_set_state(vdev_t *vd, boolean_t isopen, vdev_state_t state, vdev_aux_t aux) { uint64_t save_state; spa_t *spa = vd->vdev_spa; if (state == vd->vdev_state) { vd->vdev_stat.vs_aux = aux; return; } save_state = vd->vdev_state; vd->vdev_state = state; vd->vdev_stat.vs_aux = aux; /* * If we are setting the vdev state to anything but an open state, then * always close the underlying device unless the device has requested * a delayed close (i.e. we're about to remove or fault the device). * Otherwise, we keep accessible but invalid devices open forever. * We don't call vdev_close() itself, because that implies some extra * checks (offline, etc) that we don't want here. This is limited to * leaf devices, because otherwise closing the device will affect other * children. */ if (!vd->vdev_delayed_close && vdev_is_dead(vd) && vd->vdev_ops->vdev_op_leaf) vd->vdev_ops->vdev_op_close(vd); /* * If we have brought this vdev back into service, we need * to notify fmd so that it can gracefully repair any outstanding * cases due to a missing device. We do this in all cases, even those * that probably don't correlate to a repaired fault. This is sure to * catch all cases, and we let the zfs-retire agent sort it out. If * this is a transient state it's OK, as the retire agent will * double-check the state of the vdev before repairing it. */ if (state == VDEV_STATE_HEALTHY && vd->vdev_ops->vdev_op_leaf && vd->vdev_prevstate != state) zfs_post_state_change(spa, vd); if (vd->vdev_removed && state == VDEV_STATE_CANT_OPEN && (aux == VDEV_AUX_OPEN_FAILED || vd->vdev_checkremove)) { /* * If the previous state is set to VDEV_STATE_REMOVED, then this * device was previously marked removed and someone attempted to * reopen it. If this failed due to a nonexistent device, then * keep the device in the REMOVED state. We also let this be if * it is one of our special test online cases, which is only * attempting to online the device and shouldn't generate an FMA * fault. */ vd->vdev_state = VDEV_STATE_REMOVED; vd->vdev_stat.vs_aux = VDEV_AUX_NONE; } else if (state == VDEV_STATE_REMOVED) { vd->vdev_removed = B_TRUE; } else if (state == VDEV_STATE_CANT_OPEN) { /* * If we fail to open a vdev during an import or recovery, we * mark it as "not available", which signifies that it was * never there to begin with. Failure to open such a device * is not considered an error. */ if ((spa_load_state(spa) == SPA_LOAD_IMPORT || spa_load_state(spa) == SPA_LOAD_RECOVER) && vd->vdev_ops->vdev_op_leaf) vd->vdev_not_present = 1; /* * Post the appropriate ereport. If the 'prevstate' field is * set to something other than VDEV_STATE_UNKNOWN, it indicates * that this is part of a vdev_reopen(). In this case, we don't * want to post the ereport if the device was already in the * CANT_OPEN state beforehand. * * If the 'checkremove' flag is set, then this is an attempt to * online the device in response to an insertion event. If we * hit this case, then we have detected an insertion event for a * faulted or offline device that wasn't in the removed state. * In this scenario, we don't post an ereport because we are * about to replace the device, or attempt an online with * vdev_forcefault, which will generate the fault for us. */ if ((vd->vdev_prevstate != state || vd->vdev_forcefault) && !vd->vdev_not_present && !vd->vdev_checkremove && vd != spa->spa_root_vdev) { const char *class; switch (aux) { case VDEV_AUX_OPEN_FAILED: class = FM_EREPORT_ZFS_DEVICE_OPEN_FAILED; break; case VDEV_AUX_CORRUPT_DATA: class = FM_EREPORT_ZFS_DEVICE_CORRUPT_DATA; break; case VDEV_AUX_NO_REPLICAS: class = FM_EREPORT_ZFS_DEVICE_NO_REPLICAS; break; case VDEV_AUX_BAD_GUID_SUM: class = FM_EREPORT_ZFS_DEVICE_BAD_GUID_SUM; break; case VDEV_AUX_TOO_SMALL: class = FM_EREPORT_ZFS_DEVICE_TOO_SMALL; break; case VDEV_AUX_BAD_LABEL: class = FM_EREPORT_ZFS_DEVICE_BAD_LABEL; break; default: class = FM_EREPORT_ZFS_DEVICE_UNKNOWN; } zfs_ereport_post(class, spa, vd, NULL, save_state, 0); } /* Erase any notion of persistent removed state */ vd->vdev_removed = B_FALSE; } else { vd->vdev_removed = B_FALSE; } if (!isopen && vd->vdev_parent) vdev_propagate_state(vd->vdev_parent); } /* * Check the vdev configuration to ensure that it's capable of supporting * a root pool. * * On Solaris, we do not support RAID-Z or partial configuration. In * addition, only a single top-level vdev is allowed and none of the * leaves can be wholedisks. * * For FreeBSD, we can boot from any configuration. There is a * limitation that the boot filesystem must be either uncompressed or * compresses with lzjb compression but I'm not sure how to enforce * that here. */ boolean_t vdev_is_bootable(vdev_t *vd) { #ifdef sun if (!vd->vdev_ops->vdev_op_leaf) { char *vdev_type = vd->vdev_ops->vdev_op_type; if (strcmp(vdev_type, VDEV_TYPE_ROOT) == 0 && vd->vdev_children > 1) { return (B_FALSE); } else if (strcmp(vdev_type, VDEV_TYPE_RAIDZ) == 0 || strcmp(vdev_type, VDEV_TYPE_MISSING) == 0) { return (B_FALSE); } } else if (vd->vdev_wholedisk == 1) { return (B_FALSE); } for (int c = 0; c < vd->vdev_children; c++) { if (!vdev_is_bootable(vd->vdev_child[c])) return (B_FALSE); } #endif /* sun */ return (B_TRUE); } /* * Load the state from the original vdev tree (ovd) which * we've retrieved from the MOS config object. If the original * vdev was offline or faulted then we transfer that state to the * device in the current vdev tree (nvd). */ void vdev_load_log_state(vdev_t *nvd, vdev_t *ovd) { spa_t *spa = nvd->vdev_spa; ASSERT(nvd->vdev_top->vdev_islog); ASSERT(spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); ASSERT3U(nvd->vdev_guid, ==, ovd->vdev_guid); for (int c = 0; c < nvd->vdev_children; c++) vdev_load_log_state(nvd->vdev_child[c], ovd->vdev_child[c]); if (nvd->vdev_ops->vdev_op_leaf) { /* * Restore the persistent vdev state */ nvd->vdev_offline = ovd->vdev_offline; nvd->vdev_faulted = ovd->vdev_faulted; nvd->vdev_degraded = ovd->vdev_degraded; nvd->vdev_removed = ovd->vdev_removed; } } /* * Determine if a log device has valid content. If the vdev was * removed or faulted in the MOS config then we know that * the content on the log device has already been written to the pool. */ boolean_t vdev_log_state_valid(vdev_t *vd) { if (vd->vdev_ops->vdev_op_leaf && !vd->vdev_faulted && !vd->vdev_removed) return (B_TRUE); for (int c = 0; c < vd->vdev_children; c++) if (vdev_log_state_valid(vd->vdev_child[c])) return (B_TRUE); return (B_FALSE); } /* * Expand a vdev if possible. */ void vdev_expand(vdev_t *vd, uint64_t txg) { ASSERT(vd->vdev_top == vd); ASSERT(spa_config_held(vd->vdev_spa, SCL_ALL, RW_WRITER) == SCL_ALL); if ((vd->vdev_asize >> vd->vdev_ms_shift) > vd->vdev_ms_count) { VERIFY(vdev_metaslab_init(vd, txg) == 0); vdev_config_dirty(vd); } } /* * Split a vdev. */ void vdev_split(vdev_t *vd) { vdev_t *cvd, *pvd = vd->vdev_parent; vdev_remove_child(pvd, vd); vdev_compact_children(pvd); cvd = pvd->vdev_child[0]; if (pvd->vdev_children == 1) { vdev_remove_parent(cvd); cvd->vdev_splitting = B_TRUE; } vdev_propagate_state(cvd); } void vdev_deadman(vdev_t *vd) { for (int c = 0; c < vd->vdev_children; c++) { vdev_t *cvd = vd->vdev_child[c]; vdev_deadman(cvd); } if (vd->vdev_ops->vdev_op_leaf) { vdev_queue_t *vq = &vd->vdev_queue; mutex_enter(&vq->vq_lock); if (avl_numnodes(&vq->vq_active_tree) > 0) { spa_t *spa = vd->vdev_spa; zio_t *fio; uint64_t delta; /* * Look at the head of all the pending queues, * if any I/O has been outstanding for longer than * the spa_deadman_synctime we panic the system. */ fio = avl_first(&vq->vq_active_tree); delta = gethrtime() - fio->io_timestamp; if (delta > spa_deadman_synctime(spa)) { zfs_dbgmsg("SLOW IO: zio timestamp %lluns, " "delta %lluns, last io %lluns", fio->io_timestamp, delta, vq->vq_io_complete_ts); fm_panic("I/O to pool '%s' appears to be " "hung on vdev guid %llu at '%s'.", spa_name(spa), (long long unsigned int) vd->vdev_guid, vd->vdev_path); } } mutex_exit(&vq->vq_lock); } } Index: releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c =================================================================== --- releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c (revision 284192) +++ releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c (revision 284193) @@ -1,943 +1,945 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2012, 2014 by Delphix. All rights reserved. * Copyright 2013 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2013 Joyent, Inc. All rights reserved. */ #include #include #include #include #include #include #include #include #include #include /* * Virtual device vector for disks. */ extern ldi_ident_t zfs_li; static void vdev_disk_close(vdev_t *); typedef struct vdev_disk_ldi_cb { list_node_t lcb_next; ldi_callback_id_t lcb_id; } vdev_disk_ldi_cb_t; static void vdev_disk_alloc(vdev_t *vd) { vdev_disk_t *dvd; dvd = vd->vdev_tsd = kmem_zalloc(sizeof (vdev_disk_t), KM_SLEEP); /* * Create the LDI event callback list. */ list_create(&dvd->vd_ldi_cbs, sizeof (vdev_disk_ldi_cb_t), offsetof(vdev_disk_ldi_cb_t, lcb_next)); } static void vdev_disk_free(vdev_t *vd) { vdev_disk_t *dvd = vd->vdev_tsd; vdev_disk_ldi_cb_t *lcb; if (dvd == NULL) return; /* * We have already closed the LDI handle. Clean up the LDI event * callbacks and free vd->vdev_tsd. */ while ((lcb = list_head(&dvd->vd_ldi_cbs)) != NULL) { list_remove(&dvd->vd_ldi_cbs, lcb); (void) ldi_ev_remove_callbacks(lcb->lcb_id); kmem_free(lcb, sizeof (vdev_disk_ldi_cb_t)); } list_destroy(&dvd->vd_ldi_cbs); kmem_free(dvd, sizeof (vdev_disk_t)); vd->vdev_tsd = NULL; } /* ARGSUSED */ static int vdev_disk_off_notify(ldi_handle_t lh, ldi_ev_cookie_t ecookie, void *arg, void *ev_data) { vdev_t *vd = (vdev_t *)arg; vdev_disk_t *dvd = vd->vdev_tsd; /* * Ignore events other than offline. */ if (strcmp(ldi_ev_get_type(ecookie), LDI_EV_OFFLINE) != 0) return (LDI_EV_SUCCESS); /* * All LDI handles must be closed for the state change to succeed, so * call on vdev_disk_close() to do this. * * We inform vdev_disk_close that it is being called from offline * notify context so it will defer cleanup of LDI event callbacks and * freeing of vd->vdev_tsd to the offline finalize or a reopen. */ dvd->vd_ldi_offline = B_TRUE; vdev_disk_close(vd); /* * Now that the device is closed, request that the spa_async_thread * mark the device as REMOVED and notify FMA of the removal. */ zfs_post_remove(vd->vdev_spa, vd); vd->vdev_remove_wanted = B_TRUE; spa_async_request(vd->vdev_spa, SPA_ASYNC_REMOVE); return (LDI_EV_SUCCESS); } /* ARGSUSED */ static void vdev_disk_off_finalize(ldi_handle_t lh, ldi_ev_cookie_t ecookie, int ldi_result, void *arg, void *ev_data) { vdev_t *vd = (vdev_t *)arg; /* * Ignore events other than offline. */ if (strcmp(ldi_ev_get_type(ecookie), LDI_EV_OFFLINE) != 0) return; /* * We have already closed the LDI handle in notify. * Clean up the LDI event callbacks and free vd->vdev_tsd. */ vdev_disk_free(vd); /* * Request that the vdev be reopened if the offline state change was * unsuccessful. */ if (ldi_result != LDI_EV_SUCCESS) { vd->vdev_probe_wanted = B_TRUE; spa_async_request(vd->vdev_spa, SPA_ASYNC_PROBE); } } static ldi_ev_callback_t vdev_disk_off_callb = { .cb_vers = LDI_EV_CB_VERS, .cb_notify = vdev_disk_off_notify, .cb_finalize = vdev_disk_off_finalize }; /* ARGSUSED */ static void vdev_disk_dgrd_finalize(ldi_handle_t lh, ldi_ev_cookie_t ecookie, int ldi_result, void *arg, void *ev_data) { vdev_t *vd = (vdev_t *)arg; /* * Ignore events other than degrade. */ if (strcmp(ldi_ev_get_type(ecookie), LDI_EV_DEGRADE) != 0) return; /* * Degrade events always succeed. Mark the vdev as degraded. * This status is purely informative for the user. */ (void) vdev_degrade(vd->vdev_spa, vd->vdev_guid, 0); } static ldi_ev_callback_t vdev_disk_dgrd_callb = { .cb_vers = LDI_EV_CB_VERS, .cb_notify = NULL, .cb_finalize = vdev_disk_dgrd_finalize }; static void vdev_disk_hold(vdev_t *vd) { ddi_devid_t devid; char *minor; ASSERT(spa_config_held(vd->vdev_spa, SCL_STATE, RW_WRITER)); /* * We must have a pathname, and it must be absolute. */ if (vd->vdev_path == NULL || vd->vdev_path[0] != '/') return; /* * Only prefetch path and devid info if the device has * never been opened. */ if (vd->vdev_tsd != NULL) return; if (vd->vdev_wholedisk == -1ULL) { size_t len = strlen(vd->vdev_path) + 3; char *buf = kmem_alloc(len, KM_SLEEP); (void) snprintf(buf, len, "%ss0", vd->vdev_path); (void) ldi_vp_from_name(buf, &vd->vdev_name_vp); kmem_free(buf, len); } if (vd->vdev_name_vp == NULL) (void) ldi_vp_from_name(vd->vdev_path, &vd->vdev_name_vp); if (vd->vdev_devid != NULL && ddi_devid_str_decode(vd->vdev_devid, &devid, &minor) == 0) { (void) ldi_vp_from_devid(devid, minor, &vd->vdev_devid_vp); ddi_devid_str_free(minor); ddi_devid_free(devid); } } static void vdev_disk_rele(vdev_t *vd) { ASSERT(spa_config_held(vd->vdev_spa, SCL_STATE, RW_WRITER)); if (vd->vdev_name_vp) { VN_RELE_ASYNC(vd->vdev_name_vp, dsl_pool_vnrele_taskq(vd->vdev_spa->spa_dsl_pool)); vd->vdev_name_vp = NULL; } if (vd->vdev_devid_vp) { VN_RELE_ASYNC(vd->vdev_devid_vp, dsl_pool_vnrele_taskq(vd->vdev_spa->spa_dsl_pool)); vd->vdev_devid_vp = NULL; } } static uint64_t vdev_disk_get_space(vdev_t *vd, uint64_t capacity, uint_t blksz) { ASSERT(vd->vdev_wholedisk); vdev_disk_t *dvd = vd->vdev_tsd; dk_efi_t dk_ioc; efi_gpt_t *efi; uint64_t avail_space = 0; int efisize = EFI_LABEL_SIZE * 2; dk_ioc.dki_data = kmem_alloc(efisize, KM_SLEEP); dk_ioc.dki_lba = 1; dk_ioc.dki_length = efisize; dk_ioc.dki_data_64 = (uint64_t)(uintptr_t)dk_ioc.dki_data; efi = dk_ioc.dki_data; if (ldi_ioctl(dvd->vd_lh, DKIOCGETEFI, (intptr_t)&dk_ioc, FKIOCTL, kcred, NULL) == 0) { uint64_t efi_altern_lba = LE_64(efi->efi_gpt_AlternateLBA); if (capacity > efi_altern_lba) avail_space = (capacity - efi_altern_lba) * blksz; } kmem_free(dk_ioc.dki_data, efisize); return (avail_space); } /* * We want to be loud in DEBUG kernels when DKIOCGMEDIAINFOEXT fails, or when * even a fallback to DKIOCGMEDIAINFO fails. */ #ifdef DEBUG #define VDEV_DEBUG(...) cmn_err(CE_NOTE, __VA_ARGS__) #else #define VDEV_DEBUG(...) /* Nothing... */ #endif static int vdev_disk_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, uint64_t *ashift) { spa_t *spa = vd->vdev_spa; vdev_disk_t *dvd = vd->vdev_tsd; ldi_ev_cookie_t ecookie; vdev_disk_ldi_cb_t *lcb; union { struct dk_minfo_ext ude; struct dk_minfo ud; } dks; struct dk_minfo_ext *dkmext = &dks.ude; struct dk_minfo *dkm = &dks.ud; int error; dev_t dev; int otyp; boolean_t validate_devid = B_FALSE; ddi_devid_t devid; uint64_t capacity = 0, blksz = 0, pbsize; /* * We must have a pathname, and it must be absolute. */ if (vd->vdev_path == NULL || vd->vdev_path[0] != '/') { vd->vdev_stat.vs_aux = VDEV_AUX_BAD_LABEL; return (SET_ERROR(EINVAL)); } /* * Reopen the device if it's not currently open. Otherwise, * just update the physical size of the device. */ if (dvd != NULL) { if (dvd->vd_ldi_offline && dvd->vd_lh == NULL) { /* * If we are opening a device in its offline notify * context, the LDI handle was just closed. Clean * up the LDI event callbacks and free vd->vdev_tsd. */ vdev_disk_free(vd); } else { ASSERT(vd->vdev_reopening); goto skip_open; } } /* * Create vd->vdev_tsd. */ vdev_disk_alloc(vd); dvd = vd->vdev_tsd; /* * When opening a disk device, we want to preserve the user's original * intent. We always want to open the device by the path the user gave * us, even if it is one of multiple paths to the save device. But we * also want to be able to survive disks being removed/recabled. * Therefore the sequence of opening devices is: * * 1. Try opening the device by path. For legacy pools without the * 'whole_disk' property, attempt to fix the path by appending 's0'. * * 2. If the devid of the device matches the stored value, return * success. * * 3. Otherwise, the device may have moved. Try opening the device * by the devid instead. */ if (vd->vdev_devid != NULL) { if (ddi_devid_str_decode(vd->vdev_devid, &dvd->vd_devid, &dvd->vd_minor) != 0) { vd->vdev_stat.vs_aux = VDEV_AUX_BAD_LABEL; return (SET_ERROR(EINVAL)); } } error = EINVAL; /* presume failure */ if (vd->vdev_path != NULL) { if (vd->vdev_wholedisk == -1ULL) { size_t len = strlen(vd->vdev_path) + 3; char *buf = kmem_alloc(len, KM_SLEEP); (void) snprintf(buf, len, "%ss0", vd->vdev_path); error = ldi_open_by_name(buf, spa_mode(spa), kcred, &dvd->vd_lh, zfs_li); if (error == 0) { spa_strfree(vd->vdev_path); vd->vdev_path = buf; vd->vdev_wholedisk = 1ULL; } else { kmem_free(buf, len); } } /* * If we have not yet opened the device, try to open it by the * specified path. */ if (error != 0) { error = ldi_open_by_name(vd->vdev_path, spa_mode(spa), kcred, &dvd->vd_lh, zfs_li); } /* * Compare the devid to the stored value. */ if (error == 0 && vd->vdev_devid != NULL && ldi_get_devid(dvd->vd_lh, &devid) == 0) { if (ddi_devid_compare(devid, dvd->vd_devid) != 0) { error = SET_ERROR(EINVAL); (void) ldi_close(dvd->vd_lh, spa_mode(spa), kcred); dvd->vd_lh = NULL; } ddi_devid_free(devid); } /* * If we succeeded in opening the device, but 'vdev_wholedisk' * is not yet set, then this must be a slice. */ if (error == 0 && vd->vdev_wholedisk == -1ULL) vd->vdev_wholedisk = 0; } /* * If we were unable to open by path, or the devid check fails, open by * devid instead. */ if (error != 0 && vd->vdev_devid != NULL) { error = ldi_open_by_devid(dvd->vd_devid, dvd->vd_minor, spa_mode(spa), kcred, &dvd->vd_lh, zfs_li); } /* * If all else fails, then try opening by physical path (if available) * or the logical path (if we failed due to the devid check). While not * as reliable as the devid, this will give us something, and the higher * level vdev validation will prevent us from opening the wrong device. */ if (error) { if (vd->vdev_devid != NULL) validate_devid = B_TRUE; if (vd->vdev_physpath != NULL && (dev = ddi_pathname_to_dev_t(vd->vdev_physpath)) != NODEV) error = ldi_open_by_dev(&dev, OTYP_BLK, spa_mode(spa), kcred, &dvd->vd_lh, zfs_li); /* * Note that we don't support the legacy auto-wholedisk support * as above. This hasn't been used in a very long time and we * don't need to propagate its oddities to this edge condition. */ if (error && vd->vdev_path != NULL) error = ldi_open_by_name(vd->vdev_path, spa_mode(spa), kcred, &dvd->vd_lh, zfs_li); } if (error) { vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; return (error); } /* * Now that the device has been successfully opened, update the devid * if necessary. */ if (validate_devid && spa_writeable(spa) && ldi_get_devid(dvd->vd_lh, &devid) == 0) { if (ddi_devid_compare(devid, dvd->vd_devid) != 0) { char *vd_devid; vd_devid = ddi_devid_str_encode(devid, dvd->vd_minor); zfs_dbgmsg("vdev %s: update devid from %s, " "to %s", vd->vdev_path, vd->vdev_devid, vd_devid); spa_strfree(vd->vdev_devid); vd->vdev_devid = spa_strdup(vd_devid); ddi_devid_str_free(vd_devid); } ddi_devid_free(devid); } /* * Once a device is opened, verify that the physical device path (if * available) is up to date. */ if (ldi_get_dev(dvd->vd_lh, &dev) == 0 && ldi_get_otyp(dvd->vd_lh, &otyp) == 0) { char *physpath, *minorname; physpath = kmem_alloc(MAXPATHLEN, KM_SLEEP); minorname = NULL; if (ddi_dev_pathname(dev, otyp, physpath) == 0 && ldi_get_minor_name(dvd->vd_lh, &minorname) == 0 && (vd->vdev_physpath == NULL || strcmp(vd->vdev_physpath, physpath) != 0)) { if (vd->vdev_physpath) spa_strfree(vd->vdev_physpath); (void) strlcat(physpath, ":", MAXPATHLEN); (void) strlcat(physpath, minorname, MAXPATHLEN); vd->vdev_physpath = spa_strdup(physpath); } if (minorname) kmem_free(minorname, strlen(minorname) + 1); kmem_free(physpath, MAXPATHLEN); } /* * Register callbacks for the LDI offline event. */ if (ldi_ev_get_cookie(dvd->vd_lh, LDI_EV_OFFLINE, &ecookie) == LDI_EV_SUCCESS) { lcb = kmem_zalloc(sizeof (vdev_disk_ldi_cb_t), KM_SLEEP); list_insert_tail(&dvd->vd_ldi_cbs, lcb); (void) ldi_ev_register_callbacks(dvd->vd_lh, ecookie, &vdev_disk_off_callb, (void *) vd, &lcb->lcb_id); } /* * Register callbacks for the LDI degrade event. */ if (ldi_ev_get_cookie(dvd->vd_lh, LDI_EV_DEGRADE, &ecookie) == LDI_EV_SUCCESS) { lcb = kmem_zalloc(sizeof (vdev_disk_ldi_cb_t), KM_SLEEP); list_insert_tail(&dvd->vd_ldi_cbs, lcb); (void) ldi_ev_register_callbacks(dvd->vd_lh, ecookie, &vdev_disk_dgrd_callb, (void *) vd, &lcb->lcb_id); } skip_open: /* * Determine the actual size of the device. */ if (ldi_get_size(dvd->vd_lh, psize) != 0) { vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; return (SET_ERROR(EINVAL)); } *max_psize = *psize; /* * Determine the device's minimum transfer size. * If the ioctl isn't supported, assume DEV_BSIZE. */ if ((error = ldi_ioctl(dvd->vd_lh, DKIOCGMEDIAINFOEXT, (intptr_t)dkmext, FKIOCTL, kcred, NULL)) == 0) { capacity = dkmext->dki_capacity - 1; blksz = dkmext->dki_lbsize; pbsize = dkmext->dki_pbsize; } else if ((error = ldi_ioctl(dvd->vd_lh, DKIOCGMEDIAINFO, (intptr_t)dkm, FKIOCTL, kcred, NULL)) == 0) { VDEV_DEBUG( "vdev_disk_open(\"%s\"): fallback to DKIOCGMEDIAINFO\n", vd->vdev_path); capacity = dkm->dki_capacity - 1; blksz = dkm->dki_lbsize; pbsize = blksz; } else { VDEV_DEBUG("vdev_disk_open(\"%s\"): " "both DKIOCGMEDIAINFO{,EXT} calls failed, %d\n", vd->vdev_path, error); pbsize = DEV_BSIZE; } *ashift = highbit64(MAX(pbsize, SPA_MINBLOCKSIZE)) - 1; if (vd->vdev_wholedisk == 1) { int wce = 1; if (error == 0) { /* * If we have the capability to expand, we'd have * found out via success from DKIOCGMEDIAINFO{,EXT}. * Adjust max_psize upward accordingly since we know * we own the whole disk now. */ *max_psize += vdev_disk_get_space(vd, capacity, blksz); zfs_dbgmsg("capacity change: vdev %s, psize %llu, " "max_psize %llu", vd->vdev_path, *psize, *max_psize); } /* * Since we own the whole disk, try to enable disk write * caching. We ignore errors because it's OK if we can't do it. */ (void) ldi_ioctl(dvd->vd_lh, DKIOCSETWCE, (intptr_t)&wce, FKIOCTL, kcred, NULL); } /* * Clear the nowritecache bit, so that on a vdev_reopen() we will * try again. */ vd->vdev_nowritecache = B_FALSE; return (0); } static void vdev_disk_close(vdev_t *vd) { vdev_disk_t *dvd = vd->vdev_tsd; if (vd->vdev_reopening || dvd == NULL) return; if (dvd->vd_minor != NULL) { ddi_devid_str_free(dvd->vd_minor); dvd->vd_minor = NULL; } if (dvd->vd_devid != NULL) { ddi_devid_free(dvd->vd_devid); dvd->vd_devid = NULL; } if (dvd->vd_lh != NULL) { (void) ldi_close(dvd->vd_lh, spa_mode(vd->vdev_spa), kcred); dvd->vd_lh = NULL; } vd->vdev_delayed_close = B_FALSE; /* * If we closed the LDI handle due to an offline notify from LDI, * don't free vd->vdev_tsd or unregister the callbacks here; * the offline finalize callback or a reopen will take care of it. */ if (dvd->vd_ldi_offline) return; vdev_disk_free(vd); } int vdev_disk_physio(vdev_t *vd, caddr_t data, size_t size, uint64_t offset, int flags, boolean_t isdump) { vdev_disk_t *dvd = vd->vdev_tsd; /* * If the vdev is closed, it's likely in the REMOVED or FAULTED state. * Nothing to be done here but return failure. */ if (dvd == NULL || (dvd->vd_ldi_offline && dvd->vd_lh == NULL)) return (EIO); ASSERT(vd->vdev_ops == &vdev_disk_ops); /* * If in the context of an active crash dump, use the ldi_dump(9F) * call instead of ldi_strategy(9F) as usual. */ if (isdump) { ASSERT3P(dvd, !=, NULL); return (ldi_dump(dvd->vd_lh, data, lbtodb(offset), lbtodb(size))); } return (vdev_disk_ldi_physio(dvd->vd_lh, data, size, offset, flags)); } int vdev_disk_ldi_physio(ldi_handle_t vd_lh, caddr_t data, size_t size, uint64_t offset, int flags) { buf_t *bp; int error = 0; if (vd_lh == NULL) return (SET_ERROR(EINVAL)); ASSERT(flags & B_READ || flags & B_WRITE); bp = getrbuf(KM_SLEEP); bp->b_flags = flags | B_BUSY | B_NOCACHE | B_FAILFAST; bp->b_bcount = size; bp->b_un.b_addr = (void *)data; bp->b_lblkno = lbtodb(offset); bp->b_bufsize = size; error = ldi_strategy(vd_lh, bp); ASSERT(error == 0); if ((error = biowait(bp)) == 0 && bp->b_resid != 0) error = SET_ERROR(EIO); freerbuf(bp); return (error); } static void vdev_disk_io_intr(buf_t *bp) { vdev_buf_t *vb = (vdev_buf_t *)bp; zio_t *zio = vb->vb_io; /* * The rest of the zio stack only deals with EIO, ECKSUM, and ENXIO. * Rather than teach the rest of the stack about other error * possibilities (EFAULT, etc), we normalize the error value here. */ zio->io_error = (geterror(bp) != 0 ? SET_ERROR(EIO) : 0); if (zio->io_error == 0 && bp->b_resid != 0) zio->io_error = SET_ERROR(EIO); kmem_free(vb, sizeof (vdev_buf_t)); zio_interrupt(zio); } static void vdev_disk_ioctl_free(zio_t *zio) { kmem_free(zio->io_vsd, sizeof (struct dk_callback)); } static const zio_vsd_ops_t vdev_disk_vsd_ops = { vdev_disk_ioctl_free, zio_vsd_default_cksum_report }; static void vdev_disk_ioctl_done(void *zio_arg, int error) { zio_t *zio = zio_arg; zio->io_error = error; zio_interrupt(zio); } static int vdev_disk_io_start(zio_t *zio) { vdev_t *vd = zio->io_vd; vdev_disk_t *dvd = vd->vdev_tsd; vdev_buf_t *vb; struct dk_callback *dkc; buf_t *bp; int error; /* * If the vdev is closed, it's likely in the REMOVED or FAULTED state. * Nothing to be done here but return failure. */ if (dvd == NULL || (dvd->vd_ldi_offline && dvd->vd_lh == NULL)) { zio->io_error = SET_ERROR(ENXIO); zio_interrupt(zio); return (ZIO_PIPELINE_STOP); } if (zio->io_type == ZIO_TYPE_IOCTL) { /* XXPOLICY */ if (!vdev_readable(vd)) { zio->io_error = SET_ERROR(ENXIO); zio_interrupt(zio); return (ZIO_PIPELINE_STOP); } switch (zio->io_cmd) { case DKIOCFLUSHWRITECACHE: if (zfs_nocacheflush) break; if (vd->vdev_nowritecache) { zio->io_error = SET_ERROR(ENOTSUP); break; } zio->io_vsd = dkc = kmem_alloc(sizeof (*dkc), KM_SLEEP); zio->io_vsd_ops = &vdev_disk_vsd_ops; dkc->dkc_callback = vdev_disk_ioctl_done; dkc->dkc_flag = FLUSH_VOLATILE; dkc->dkc_cookie = zio; error = ldi_ioctl(dvd->vd_lh, zio->io_cmd, (uintptr_t)dkc, FKIOCTL, kcred, NULL); if (error == 0) { /* * The ioctl will be done asychronously, * and will call vdev_disk_ioctl_done() * upon completion. */ return (ZIO_PIPELINE_STOP); } if (error == ENOTSUP || error == ENOTTY) { /* * If we get ENOTSUP or ENOTTY, we know that * no future attempts will ever succeed. * In this case we set a persistent bit so * that we don't bother with the ioctl in the * future. */ vd->vdev_nowritecache = B_TRUE; } zio->io_error = error; break; default: zio->io_error = SET_ERROR(ENOTSUP); } zio_interrupt(zio); return (ZIO_PIPELINE_STOP); } + ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE); + vb = kmem_alloc(sizeof (vdev_buf_t), KM_SLEEP); vb->vb_io = zio; bp = &vb->vb_buf; bioinit(bp); bp->b_flags = B_BUSY | B_NOCACHE | (zio->io_type == ZIO_TYPE_READ ? B_READ : B_WRITE); if (!(zio->io_flags & (ZIO_FLAG_IO_RETRY | ZIO_FLAG_TRYHARD))) bp->b_flags |= B_FAILFAST; bp->b_bcount = zio->io_size; bp->b_un.b_addr = zio->io_data; bp->b_lblkno = lbtodb(zio->io_offset); bp->b_bufsize = zio->io_size; bp->b_iodone = (int (*)())vdev_disk_io_intr; /* ldi_strategy() will return non-zero only on programming errors */ VERIFY(ldi_strategy(dvd->vd_lh, bp) == 0); return (ZIO_PIPELINE_STOP); } static void vdev_disk_io_done(zio_t *zio) { vdev_t *vd = zio->io_vd; /* * If the device returned EIO, then attempt a DKIOCSTATE ioctl to see if * the device has been removed. If this is the case, then we trigger an * asynchronous removal of the device. Otherwise, probe the device and * make sure it's still accessible. */ if (zio->io_error == EIO && !vd->vdev_remove_wanted) { vdev_disk_t *dvd = vd->vdev_tsd; int state = DKIO_NONE; if (ldi_ioctl(dvd->vd_lh, DKIOCSTATE, (intptr_t)&state, FKIOCTL, kcred, NULL) == 0 && state != DKIO_INSERTED) { /* * We post the resource as soon as possible, instead of * when the async removal actually happens, because the * DE is using this information to discard previous I/O * errors. */ zfs_post_remove(zio->io_spa, vd); vd->vdev_remove_wanted = B_TRUE; spa_async_request(zio->io_spa, SPA_ASYNC_REMOVE); } else if (!vd->vdev_delayed_close) { vd->vdev_delayed_close = B_TRUE; } } } vdev_ops_t vdev_disk_ops = { vdev_disk_open, vdev_disk_close, vdev_default_asize, vdev_disk_io_start, vdev_disk_io_done, NULL, vdev_disk_hold, vdev_disk_rele, VDEV_TYPE_DISK, /* name of this vdev type */ B_TRUE /* leaf vdev */ }; /* * Given the root disk device devid or pathname, read the label from * the device, and construct a configuration nvlist. */ int vdev_disk_read_rootlabel(char *devpath, char *devid, nvlist_t **config) { ldi_handle_t vd_lh; vdev_label_t *label; uint64_t s, size; int l; ddi_devid_t tmpdevid; int error = -1; char *minor_name; /* * Read the device label and build the nvlist. */ if (devid != NULL && ddi_devid_str_decode(devid, &tmpdevid, &minor_name) == 0) { error = ldi_open_by_devid(tmpdevid, minor_name, FREAD, kcred, &vd_lh, zfs_li); ddi_devid_free(tmpdevid); ddi_devid_str_free(minor_name); } if (error && (error = ldi_open_by_name(devpath, FREAD, kcred, &vd_lh, zfs_li))) return (error); if (ldi_get_size(vd_lh, &s)) { (void) ldi_close(vd_lh, FREAD, kcred); return (SET_ERROR(EIO)); } size = P2ALIGN_TYPED(s, sizeof (vdev_label_t), uint64_t); label = kmem_alloc(sizeof (vdev_label_t), KM_SLEEP); *config = NULL; for (l = 0; l < VDEV_LABELS; l++) { uint64_t offset, state, txg = 0; /* read vdev label */ offset = vdev_label_offset(size, l, 0); if (vdev_disk_ldi_physio(vd_lh, (caddr_t)label, VDEV_SKIP_SIZE + VDEV_PHYS_SIZE, offset, B_READ) != 0) continue; if (nvlist_unpack(label->vl_vdev_phys.vp_nvlist, sizeof (label->vl_vdev_phys.vp_nvlist), config, 0) != 0) { *config = NULL; continue; } if (nvlist_lookup_uint64(*config, ZPOOL_CONFIG_POOL_STATE, &state) != 0 || state >= POOL_STATE_DESTROYED) { nvlist_free(*config); *config = NULL; continue; } if (nvlist_lookup_uint64(*config, ZPOOL_CONFIG_POOL_TXG, &txg) != 0 || txg == 0) { nvlist_free(*config); *config = NULL; continue; } break; } kmem_free(label, sizeof (vdev_label_t)); (void) ldi_close(vd_lh, FREAD, kcred); if (*config == NULL) error = SET_ERROR(EIDRM); return (error); } Index: releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c =================================================================== --- releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c (revision 284192) +++ releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c (revision 284193) @@ -1,237 +1,241 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2013 by Delphix. All rights reserved. */ #include #include #include #include #include #include #include /* * Virtual device vector for files. */ static void vdev_file_hold(vdev_t *vd) { ASSERT(vd->vdev_path != NULL); } static void vdev_file_rele(vdev_t *vd) { ASSERT(vd->vdev_path != NULL); } static int vdev_file_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, uint64_t *logical_ashift, uint64_t *physical_ashift) { vdev_file_t *vf; vnode_t *vp; vattr_t vattr; int error; /* * We must have a pathname, and it must be absolute. */ if (vd->vdev_path == NULL || vd->vdev_path[0] != '/') { vd->vdev_stat.vs_aux = VDEV_AUX_BAD_LABEL; return (SET_ERROR(EINVAL)); } /* * Reopen the device if it's not currently open. Otherwise, * just update the physical size of the device. */ if (vd->vdev_tsd != NULL) { ASSERT(vd->vdev_reopening); vf = vd->vdev_tsd; vp = vf->vf_vnode; goto skip_open; } vf = vd->vdev_tsd = kmem_zalloc(sizeof (vdev_file_t), KM_SLEEP); /* * We always open the files from the root of the global zone, even if * we're in a local zone. If the user has gotten to this point, the * administrator has already decided that the pool should be available * to local zone users, so the underlying devices should be as well. */ ASSERT(vd->vdev_path != NULL && vd->vdev_path[0] == '/'); error = vn_openat(vd->vdev_path + 1, UIO_SYSSPACE, spa_mode(vd->vdev_spa) | FOFFMAX, 0, &vp, 0, 0, rootdir, -1); if (error) { vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; kmem_free(vd->vdev_tsd, sizeof (vdev_file_t)); vd->vdev_tsd = NULL; return (error); } vf->vf_vnode = vp; #ifdef _KERNEL /* * Make sure it's a regular file. */ if (vp->v_type != VREG) { #ifdef __FreeBSD__ (void) VOP_CLOSE(vp, spa_mode(vd->vdev_spa), 1, 0, kcred, NULL); #endif vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; #ifdef __FreeBSD__ kmem_free(vd->vdev_tsd, sizeof (vdev_file_t)); vd->vdev_tsd = NULL; #endif return (SET_ERROR(ENODEV)); } #endif /* _KERNEL */ skip_open: /* * Determine the physical size of the file. */ vattr.va_mask = AT_SIZE; vn_lock(vp, LK_SHARED | LK_RETRY); error = VOP_GETATTR(vp, &vattr, kcred); VOP_UNLOCK(vp, 0); if (error) { (void) VOP_CLOSE(vp, spa_mode(vd->vdev_spa), 1, 0, kcred, NULL); vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; kmem_free(vd->vdev_tsd, sizeof (vdev_file_t)); vd->vdev_tsd = NULL; return (error); } + vd->vdev_notrim = B_TRUE; + *max_psize = *psize = vattr.va_size; *logical_ashift = SPA_MINBLOCKSHIFT; *physical_ashift = SPA_MINBLOCKSHIFT; return (0); } static void vdev_file_close(vdev_t *vd) { vdev_file_t *vf = vd->vdev_tsd; if (vd->vdev_reopening || vf == NULL) return; if (vf->vf_vnode != NULL) { (void) VOP_CLOSE(vf->vf_vnode, spa_mode(vd->vdev_spa), 1, 0, kcred, NULL); } vd->vdev_delayed_close = B_FALSE; kmem_free(vf, sizeof (vdev_file_t)); vd->vdev_tsd = NULL; } static int vdev_file_io_start(zio_t *zio) { vdev_t *vd = zio->io_vd; vdev_file_t *vf; vnode_t *vp; ssize_t resid; if (!vdev_readable(vd)) { zio->io_error = SET_ERROR(ENXIO); zio_interrupt(zio); return (ZIO_PIPELINE_STOP); } vf = vd->vdev_tsd; vp = vf->vf_vnode; if (zio->io_type == ZIO_TYPE_IOCTL) { switch (zio->io_cmd) { case DKIOCFLUSHWRITECACHE: zio->io_error = VOP_FSYNC(vp, FSYNC | FDSYNC, kcred, NULL); break; default: zio->io_error = SET_ERROR(ENOTSUP); } zio_interrupt(zio); return (ZIO_PIPELINE_STOP); } + + ASSERT(zio->io_type == ZIO_TYPE_READ || zio->io_type == ZIO_TYPE_WRITE); zio->io_error = vn_rdwr(zio->io_type == ZIO_TYPE_READ ? UIO_READ : UIO_WRITE, vp, zio->io_data, zio->io_size, zio->io_offset, UIO_SYSSPACE, 0, RLIM64_INFINITY, kcred, &resid); if (resid != 0 && zio->io_error == 0) zio->io_error = ENOSPC; zio_interrupt(zio); return (ZIO_PIPELINE_STOP); } /* ARGSUSED */ static void vdev_file_io_done(zio_t *zio) { } vdev_ops_t vdev_file_ops = { vdev_file_open, vdev_file_close, vdev_default_asize, vdev_file_io_start, vdev_file_io_done, NULL, vdev_file_hold, vdev_file_rele, VDEV_TYPE_FILE, /* name of this vdev type */ B_TRUE /* leaf vdev */ }; /* * From userland we access disks just like files. */ #ifndef _KERNEL vdev_ops_t vdev_disk_ops = { vdev_file_open, vdev_file_close, vdev_default_asize, vdev_file_io_start, vdev_file_io_done, NULL, vdev_file_hold, vdev_file_rele, VDEV_TYPE_DISK, /* name of this vdev type */ B_TRUE /* leaf vdev */ }; #endif Index: releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c =================================================================== --- releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision 284192) +++ releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision 284193) @@ -1,898 +1,903 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright (c) 2006 Pawel Jakub Dawidek * All rights reserved. * * Portions Copyright (c) 2012 Martin Matuska */ #include #include #include #include #include #include #include #include #include #include #include #include /* * Virtual device vector for GEOM. */ static g_attrchanged_t vdev_geom_attrchanged; struct g_class zfs_vdev_class = { .name = "ZFS::VDEV", .version = G_VERSION, .attrchanged = vdev_geom_attrchanged, }; DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev); SYSCTL_DECL(_vfs_zfs_vdev); /* Don't send BIO_FLUSH. */ static int vdev_geom_bio_flush_disable = 0; TUNABLE_INT("vfs.zfs.vdev.bio_flush_disable", &vdev_geom_bio_flush_disable); SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, bio_flush_disable, CTLFLAG_RW, &vdev_geom_bio_flush_disable, 0, "Disable BIO_FLUSH"); /* Don't send BIO_DELETE. */ static int vdev_geom_bio_delete_disable = 0; TUNABLE_INT("vfs.zfs.vdev.bio_delete_disable", &vdev_geom_bio_delete_disable); SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, bio_delete_disable, CTLFLAG_RW, &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE"); static void vdev_geom_set_rotation_rate(vdev_t *vd, struct g_consumer *cp) { int error; uint16_t rate; error = g_getattr("GEOM::rotation_rate", cp, &rate); if (error == 0) vd->vdev_rotation_rate = rate; else vd->vdev_rotation_rate = VDEV_RATE_UNKNOWN; } static void vdev_geom_attrchanged(struct g_consumer *cp, const char *attr) { vdev_t *vd; vd = cp->private; if (vd == NULL) return; if (strcmp(attr, "GEOM::rotation_rate") == 0) { vdev_geom_set_rotation_rate(vd, cp); return; } } static void vdev_geom_orphan(struct g_consumer *cp) { vdev_t *vd; g_topology_assert(); vd = cp->private; if (vd == NULL) return; /* * Orphan callbacks occur from the GEOM event thread. * Concurrent with this call, new I/O requests may be * working their way through GEOM about to find out * (only once executed by the g_down thread) that we've * been orphaned from our disk provider. These I/Os * must be retired before we can detach our consumer. * This is most easily achieved by acquiring the * SPA ZIO configuration lock as a writer, but doing * so with the GEOM topology lock held would cause * a lock order reversal. Instead, rely on the SPA's * async removal support to invoke a close on this * vdev once it is safe to do so. */ zfs_post_remove(vd->vdev_spa, vd); vd->vdev_remove_wanted = B_TRUE; spa_async_request(vd->vdev_spa, SPA_ASYNC_REMOVE); } static struct g_consumer * vdev_geom_attach(struct g_provider *pp) { struct g_geom *gp; struct g_consumer *cp; g_topology_assert(); ZFS_LOG(1, "Attaching to %s.", pp->name); /* Do we have geom already? No? Create one. */ LIST_FOREACH(gp, &zfs_vdev_class.geom, geom) { if (gp->flags & G_GEOM_WITHER) continue; if (strcmp(gp->name, "zfs::vdev") != 0) continue; break; } if (gp == NULL) { gp = g_new_geomf(&zfs_vdev_class, "zfs::vdev"); gp->orphan = vdev_geom_orphan; cp = g_new_consumer(gp); if (g_attach(cp, pp) != 0) { g_wither_geom(gp, ENXIO); return (NULL); } if (g_access(cp, 1, 0, 1) != 0) { g_wither_geom(gp, ENXIO); return (NULL); } ZFS_LOG(1, "Created geom and consumer for %s.", pp->name); } else { /* Check if we are already connected to this provider. */ LIST_FOREACH(cp, &gp->consumer, consumer) { if (cp->provider == pp) { ZFS_LOG(1, "Found consumer for %s.", pp->name); break; } } if (cp == NULL) { cp = g_new_consumer(gp); if (g_attach(cp, pp) != 0) { g_destroy_consumer(cp); return (NULL); } if (g_access(cp, 1, 0, 1) != 0) { g_detach(cp); g_destroy_consumer(cp); return (NULL); } ZFS_LOG(1, "Created consumer for %s.", pp->name); } else { if (g_access(cp, 1, 0, 1) != 0) return (NULL); ZFS_LOG(1, "Used existing consumer for %s.", pp->name); } } cp->flags |= G_CF_DIRECT_SEND | G_CF_DIRECT_RECEIVE; return (cp); } static void vdev_geom_detach(void *arg, int flag __unused) { struct g_geom *gp; struct g_consumer *cp; g_topology_assert(); cp = arg; gp = cp->geom; ZFS_LOG(1, "Closing access to %s.", cp->provider->name); g_access(cp, -1, 0, -1); /* Destroy consumer on last close. */ if (cp->acr == 0 && cp->ace == 0) { ZFS_LOG(1, "Destroyed consumer to %s.", cp->provider->name); if (cp->acw > 0) g_access(cp, 0, -cp->acw, 0); g_detach(cp); g_destroy_consumer(cp); } /* Destroy geom if there are no consumers left. */ if (LIST_EMPTY(&gp->consumer)) { ZFS_LOG(1, "Destroyed geom %s.", gp->name); g_wither_geom(gp, ENXIO); } } static uint64_t nvlist_get_guid(nvlist_t *list) { uint64_t value; value = 0; nvlist_lookup_uint64(list, ZPOOL_CONFIG_GUID, &value); return (value); } static int vdev_geom_io(struct g_consumer *cp, int cmd, void *data, off_t offset, off_t size) { struct bio *bp; u_char *p; off_t off, maxio; int error; ASSERT((offset % cp->provider->sectorsize) == 0); ASSERT((size % cp->provider->sectorsize) == 0); bp = g_alloc_bio(); off = offset; offset += size; p = data; maxio = MAXPHYS - (MAXPHYS % cp->provider->sectorsize); error = 0; for (; off < offset; off += maxio, p += maxio, size -= maxio) { bzero(bp, sizeof(*bp)); bp->bio_cmd = cmd; bp->bio_done = NULL; bp->bio_offset = off; bp->bio_length = MIN(size, maxio); bp->bio_data = p; g_io_request(bp, cp); error = biowait(bp, "vdev_geom_io"); if (error != 0) break; } g_destroy_bio(bp); return (error); } static void vdev_geom_taste_orphan(struct g_consumer *cp) { KASSERT(1 == 0, ("%s called while tasting %s.", __func__, cp->provider->name)); } static int vdev_geom_read_config(struct g_consumer *cp, nvlist_t **config) { struct g_provider *pp; vdev_label_t *label; char *p, *buf; size_t buflen; uint64_t psize; off_t offset, size; uint64_t guid, state, txg; int error, l, len; g_topology_assert_not(); pp = cp->provider; ZFS_LOG(1, "Reading config from %s...", pp->name); psize = pp->mediasize; psize = P2ALIGN(psize, (uint64_t)sizeof(vdev_label_t)); size = sizeof(*label) + pp->sectorsize - ((sizeof(*label) - 1) % pp->sectorsize) - 1; guid = 0; label = kmem_alloc(size, KM_SLEEP); buflen = sizeof(label->vl_vdev_phys.vp_nvlist); *config = NULL; for (l = 0; l < VDEV_LABELS; l++) { offset = vdev_label_offset(psize, l, 0); if ((offset % pp->sectorsize) != 0) continue; if (vdev_geom_io(cp, BIO_READ, label, offset, size) != 0) continue; buf = label->vl_vdev_phys.vp_nvlist; if (nvlist_unpack(buf, buflen, config, 0) != 0) continue; if (nvlist_lookup_uint64(*config, ZPOOL_CONFIG_POOL_STATE, &state) != 0 || state > POOL_STATE_L2CACHE) { nvlist_free(*config); *config = NULL; continue; } if (state != POOL_STATE_SPARE && state != POOL_STATE_L2CACHE && (nvlist_lookup_uint64(*config, ZPOOL_CONFIG_POOL_TXG, &txg) != 0 || txg == 0)) { nvlist_free(*config); *config = NULL; continue; } break; } kmem_free(label, size); return (*config == NULL ? ENOENT : 0); } static void resize_configs(nvlist_t ***configs, uint64_t *count, uint64_t id) { nvlist_t **new_configs; uint64_t i; if (id < *count) return; new_configs = kmem_zalloc((id + 1) * sizeof(nvlist_t *), KM_SLEEP); for (i = 0; i < *count; i++) new_configs[i] = (*configs)[i]; if (*configs != NULL) kmem_free(*configs, *count * sizeof(void *)); *configs = new_configs; *count = id + 1; } static void process_vdev_config(nvlist_t ***configs, uint64_t *count, nvlist_t *cfg, const char *name, uint64_t* known_pool_guid) { nvlist_t *vdev_tree; uint64_t pool_guid; uint64_t vdev_guid, known_guid; uint64_t id, txg, known_txg; char *pname; int i; if (nvlist_lookup_string(cfg, ZPOOL_CONFIG_POOL_NAME, &pname) != 0 || strcmp(pname, name) != 0) goto ignore; if (nvlist_lookup_uint64(cfg, ZPOOL_CONFIG_POOL_GUID, &pool_guid) != 0) goto ignore; if (nvlist_lookup_uint64(cfg, ZPOOL_CONFIG_TOP_GUID, &vdev_guid) != 0) goto ignore; if (nvlist_lookup_nvlist(cfg, ZPOOL_CONFIG_VDEV_TREE, &vdev_tree) != 0) goto ignore; if (nvlist_lookup_uint64(vdev_tree, ZPOOL_CONFIG_ID, &id) != 0) goto ignore; VERIFY(nvlist_lookup_uint64(cfg, ZPOOL_CONFIG_POOL_TXG, &txg) == 0); if (*known_pool_guid != 0) { if (pool_guid != *known_pool_guid) goto ignore; } else *known_pool_guid = pool_guid; resize_configs(configs, count, id); if ((*configs)[id] != NULL) { VERIFY(nvlist_lookup_uint64((*configs)[id], ZPOOL_CONFIG_POOL_TXG, &known_txg) == 0); if (txg <= known_txg) goto ignore; nvlist_free((*configs)[id]); } (*configs)[id] = cfg; return; ignore: nvlist_free(cfg); } static int vdev_geom_attach_taster(struct g_consumer *cp, struct g_provider *pp) { int error; if (pp->flags & G_PF_WITHER) return (EINVAL); g_attach(cp, pp); error = g_access(cp, 1, 0, 0); if (error == 0) { if (pp->sectorsize > VDEV_PAD_SIZE || !ISP2(pp->sectorsize)) error = EINVAL; else if (pp->mediasize < SPA_MINDEVSIZE) error = EINVAL; if (error != 0) g_access(cp, -1, 0, 0); } if (error != 0) g_detach(cp); return (error); } static void vdev_geom_detach_taster(struct g_consumer *cp) { g_access(cp, -1, 0, 0); g_detach(cp); } int vdev_geom_read_pool_label(const char *name, nvlist_t ***configs, uint64_t *count) { struct g_class *mp; struct g_geom *gp, *zgp; struct g_provider *pp; struct g_consumer *zcp; nvlist_t *vdev_cfg; uint64_t pool_guid; int error; DROP_GIANT(); g_topology_lock(); zgp = g_new_geomf(&zfs_vdev_class, "zfs::vdev::taste"); /* This orphan function should be never called. */ zgp->orphan = vdev_geom_taste_orphan; zcp = g_new_consumer(zgp); *configs = NULL; *count = 0; pool_guid = 0; LIST_FOREACH(mp, &g_classes, class) { if (mp == &zfs_vdev_class) continue; LIST_FOREACH(gp, &mp->geom, geom) { if (gp->flags & G_GEOM_WITHER) continue; LIST_FOREACH(pp, &gp->provider, provider) { if (pp->flags & G_PF_WITHER) continue; if (vdev_geom_attach_taster(zcp, pp) != 0) continue; g_topology_unlock(); error = vdev_geom_read_config(zcp, &vdev_cfg); g_topology_lock(); vdev_geom_detach_taster(zcp); if (error) continue; ZFS_LOG(1, "successfully read vdev config"); process_vdev_config(configs, count, vdev_cfg, name, &pool_guid); } } } g_destroy_consumer(zcp); g_destroy_geom(zgp); g_topology_unlock(); PICKUP_GIANT(); return (*count > 0 ? 0 : ENOENT); } static uint64_t vdev_geom_read_guid(struct g_consumer *cp) { nvlist_t *config; uint64_t guid; g_topology_assert_not(); guid = 0; if (vdev_geom_read_config(cp, &config) == 0) { guid = nvlist_get_guid(config); nvlist_free(config); } return (guid); } static struct g_consumer * vdev_geom_attach_by_guid(uint64_t guid) { struct g_class *mp; struct g_geom *gp, *zgp; struct g_provider *pp; struct g_consumer *cp, *zcp; uint64_t pguid; g_topology_assert(); zgp = g_new_geomf(&zfs_vdev_class, "zfs::vdev::taste"); /* This orphan function should be never called. */ zgp->orphan = vdev_geom_taste_orphan; zcp = g_new_consumer(zgp); cp = NULL; LIST_FOREACH(mp, &g_classes, class) { if (mp == &zfs_vdev_class) continue; LIST_FOREACH(gp, &mp->geom, geom) { if (gp->flags & G_GEOM_WITHER) continue; LIST_FOREACH(pp, &gp->provider, provider) { if (vdev_geom_attach_taster(zcp, pp) != 0) continue; g_topology_unlock(); pguid = vdev_geom_read_guid(zcp); g_topology_lock(); vdev_geom_detach_taster(zcp); if (pguid != guid) continue; cp = vdev_geom_attach(pp); if (cp == NULL) { printf("ZFS WARNING: Unable to attach to %s.\n", pp->name); continue; } break; } if (cp != NULL) break; } if (cp != NULL) break; } end: g_destroy_consumer(zcp); g_destroy_geom(zgp); return (cp); } static struct g_consumer * vdev_geom_open_by_guid(vdev_t *vd) { struct g_consumer *cp; char *buf; size_t len; g_topology_assert(); ZFS_LOG(1, "Searching by guid [%ju].", (uintmax_t)vd->vdev_guid); cp = vdev_geom_attach_by_guid(vd->vdev_guid); if (cp != NULL) { len = strlen(cp->provider->name) + strlen("/dev/") + 1; buf = kmem_alloc(len, KM_SLEEP); snprintf(buf, len, "/dev/%s", cp->provider->name); spa_strfree(vd->vdev_path); vd->vdev_path = buf; ZFS_LOG(1, "Attach by guid [%ju] succeeded, provider %s.", (uintmax_t)vd->vdev_guid, vd->vdev_path); } else { ZFS_LOG(1, "Search by guid [%ju] failed.", (uintmax_t)vd->vdev_guid); } return (cp); } static struct g_consumer * vdev_geom_open_by_path(vdev_t *vd, int check_guid) { struct g_provider *pp; struct g_consumer *cp; uint64_t guid; g_topology_assert(); cp = NULL; pp = g_provider_by_name(vd->vdev_path + sizeof("/dev/") - 1); if (pp != NULL) { ZFS_LOG(1, "Found provider by name %s.", vd->vdev_path); cp = vdev_geom_attach(pp); if (cp != NULL && check_guid && ISP2(pp->sectorsize) && pp->sectorsize <= VDEV_PAD_SIZE) { g_topology_unlock(); guid = vdev_geom_read_guid(cp); g_topology_lock(); if (guid != vd->vdev_guid) { vdev_geom_detach(cp, 0); cp = NULL; ZFS_LOG(1, "guid mismatch for provider %s: " "%ju != %ju.", vd->vdev_path, (uintmax_t)vd->vdev_guid, (uintmax_t)guid); } else { ZFS_LOG(1, "guid match for provider %s.", vd->vdev_path); } } } return (cp); } static int vdev_geom_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, uint64_t *logical_ashift, uint64_t *physical_ashift) { struct g_provider *pp; struct g_consumer *cp; size_t bufsize; int error; /* * We must have a pathname, and it must be absolute. */ if (vd->vdev_path == NULL || vd->vdev_path[0] != '/') { vd->vdev_stat.vs_aux = VDEV_AUX_BAD_LABEL; return (EINVAL); } vd->vdev_tsd = NULL; DROP_GIANT(); g_topology_lock(); error = 0; /* * If we're creating or splitting a pool, just find the GEOM provider * by its name and ignore GUID mismatches. */ if (vd->vdev_spa->spa_load_state == SPA_LOAD_NONE || vd->vdev_spa->spa_splitting_newspa == B_TRUE) cp = vdev_geom_open_by_path(vd, 0); else { cp = vdev_geom_open_by_path(vd, 1); if (cp == NULL) { /* * The device at vd->vdev_path doesn't have the * expected guid. The disks might have merely * moved around so try all other GEOM providers * to find one with the right guid. */ cp = vdev_geom_open_by_guid(vd); } } if (cp == NULL) { ZFS_LOG(1, "Provider %s not found.", vd->vdev_path); error = ENOENT; } else if (cp->provider->sectorsize > VDEV_PAD_SIZE || !ISP2(cp->provider->sectorsize)) { ZFS_LOG(1, "Provider %s has unsupported sectorsize.", vd->vdev_path); vdev_geom_detach(cp, 0); error = EINVAL; cp = NULL; } else if (cp->acw == 0 && (spa_mode(vd->vdev_spa) & FWRITE) != 0) { int i; for (i = 0; i < 5; i++) { error = g_access(cp, 0, 1, 0); if (error == 0) break; g_topology_unlock(); tsleep(vd, 0, "vdev", hz / 2); g_topology_lock(); } if (error != 0) { printf("ZFS WARNING: Unable to open %s for writing (error=%d).\n", vd->vdev_path, error); vdev_geom_detach(cp, 0); cp = NULL; } } g_topology_unlock(); PICKUP_GIANT(); if (cp == NULL) { vd->vdev_stat.vs_aux = VDEV_AUX_OPEN_FAILED; return (error); } cp->private = vd; vd->vdev_tsd = cp; pp = cp->provider; /* * Determine the actual size of the device. */ *max_psize = *psize = pp->mediasize; /* * Determine the device's minimum transfer size and preferred * transfer size. */ *logical_ashift = highbit(MAX(pp->sectorsize, SPA_MINBLOCKSIZE)) - 1; *physical_ashift = 0; if (pp->stripesize) *physical_ashift = highbit(pp->stripesize) - 1; /* * Clear the nowritecache settings, so that on a vdev_reopen() * we will try again. */ vd->vdev_nowritecache = B_FALSE; if (vd->vdev_physpath != NULL) spa_strfree(vd->vdev_physpath); bufsize = sizeof("/dev/") + strlen(pp->name); vd->vdev_physpath = kmem_alloc(bufsize, KM_SLEEP); snprintf(vd->vdev_physpath, bufsize, "/dev/%s", pp->name); /* * Determine the device's rotation rate. */ vdev_geom_set_rotation_rate(vd, cp); return (0); } static void vdev_geom_close(vdev_t *vd) { struct g_consumer *cp; cp = vd->vdev_tsd; if (cp == NULL) return; vd->vdev_tsd = NULL; vd->vdev_delayed_close = B_FALSE; cp->private = NULL; /* XXX locking */ g_post_event(vdev_geom_detach, cp, M_WAITOK, NULL); } static void vdev_geom_io_intr(struct bio *bp) { vdev_t *vd; zio_t *zio; zio = bp->bio_caller1; vd = zio->io_vd; zio->io_error = bp->bio_error; if (zio->io_error == 0 && bp->bio_resid != 0) zio->io_error = SET_ERROR(EIO); switch(zio->io_error) { case ENOTSUP: /* * If we get ENOTSUP for BIO_FLUSH or BIO_DELETE we know * that future attempts will never succeed. In this case * we set a persistent flag so that we don't bother with * requests in the future. */ switch(bp->bio_cmd) { case BIO_FLUSH: vd->vdev_nowritecache = B_TRUE; break; case BIO_DELETE: vd->vdev_notrim = B_TRUE; break; } break; case ENXIO: if (!vd->vdev_remove_wanted) { /* * If provider's error is set we assume it is being * removed. */ if (bp->bio_to->error != 0) { vd->vdev_remove_wanted = B_TRUE; spa_async_request(zio->io_spa, SPA_ASYNC_REMOVE); } else if (!vd->vdev_delayed_close) { vd->vdev_delayed_close = B_TRUE; } } break; } g_destroy_bio(bp); zio_interrupt(zio); } static int vdev_geom_io_start(zio_t *zio) { vdev_t *vd; struct g_consumer *cp; struct bio *bp; int error; vd = zio->io_vd; switch (zio->io_type) { case ZIO_TYPE_IOCTL: /* XXPOLICY */ if (!vdev_readable(vd)) { zio->io_error = SET_ERROR(ENXIO); } else { switch (zio->io_cmd) { case DKIOCFLUSHWRITECACHE: if (zfs_nocacheflush || vdev_geom_bio_flush_disable) break; if (vd->vdev_nowritecache) { zio->io_error = SET_ERROR(ENOTSUP); break; } goto sendreq; default: zio->io_error = SET_ERROR(ENOTSUP); } } zio_interrupt(zio); return (ZIO_PIPELINE_STOP); case ZIO_TYPE_FREE: if (vd->vdev_notrim) { zio->io_error = SET_ERROR(ENOTSUP); } else if (!vdev_geom_bio_delete_disable) { goto sendreq; } zio_interrupt(zio); return (ZIO_PIPELINE_STOP); } sendreq: + ASSERT(zio->io_type == ZIO_TYPE_READ || + zio->io_type == ZIO_TYPE_WRITE || + zio->io_type == ZIO_TYPE_FREE || + zio->io_type == ZIO_TYPE_IOCTL); + cp = vd->vdev_tsd; if (cp == NULL) { zio->io_error = SET_ERROR(ENXIO); zio_interrupt(zio); return (ZIO_PIPELINE_STOP); } bp = g_alloc_bio(); bp->bio_caller1 = zio; switch (zio->io_type) { case ZIO_TYPE_READ: case ZIO_TYPE_WRITE: bp->bio_cmd = zio->io_type == ZIO_TYPE_READ ? BIO_READ : BIO_WRITE; bp->bio_data = zio->io_data; bp->bio_offset = zio->io_offset; bp->bio_length = zio->io_size; break; case ZIO_TYPE_FREE: bp->bio_cmd = BIO_DELETE; bp->bio_data = NULL; bp->bio_offset = zio->io_offset; bp->bio_length = zio->io_size; break; case ZIO_TYPE_IOCTL: bp->bio_cmd = BIO_FLUSH; bp->bio_flags |= BIO_ORDERED; bp->bio_data = NULL; bp->bio_offset = cp->provider->mediasize; bp->bio_length = 0; break; } bp->bio_done = vdev_geom_io_intr; g_io_request(bp, cp); return (ZIO_PIPELINE_STOP); } static void vdev_geom_io_done(zio_t *zio) { } static void vdev_geom_hold(vdev_t *vd) { } static void vdev_geom_rele(vdev_t *vd) { } vdev_ops_t vdev_geom_ops = { vdev_geom_open, vdev_geom_close, vdev_default_asize, vdev_geom_io_start, vdev_geom_io_done, NULL, vdev_geom_hold, vdev_geom_rele, VDEV_TYPE_DISK, /* name of this vdev type */ B_TRUE /* leaf vdev */ }; Index: releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c =================================================================== --- releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c (revision 284192) +++ releng/10.1/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c (revision 284193) @@ -1,1292 +1,1293 @@ /* * CDDL HEADER START * * The contents of this file are subject to the terms of the * Common Development and Distribution License (the "License"). * You may not use this file except in compliance with the License. * * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE * or http://www.opensolaris.org/os/licensing. * See the License for the specific language governing permissions * and limitations under the License. * * When distributing Covered Code, include this CDDL HEADER in each * file and include the License file at usr/src/OPENSOLARIS.LICENSE. * If applicable, add the following below this CDDL HEADER, with the * fields enclosed by brackets "[]" replaced with your own identifying * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2013 by Delphix. All rights reserved. */ /* * Virtual Device Labels * --------------------- * * The vdev label serves several distinct purposes: * * 1. Uniquely identify this device as part of a ZFS pool and confirm its * identity within the pool. * * 2. Verify that all the devices given in a configuration are present * within the pool. * * 3. Determine the uberblock for the pool. * * 4. In case of an import operation, determine the configuration of the * toplevel vdev of which it is a part. * * 5. If an import operation cannot find all the devices in the pool, * provide enough information to the administrator to determine which * devices are missing. * * It is important to note that while the kernel is responsible for writing the * label, it only consumes the information in the first three cases. The * latter information is only consumed in userland when determining the * configuration to import a pool. * * * Label Organization * ------------------ * * Before describing the contents of the label, it's important to understand how * the labels are written and updated with respect to the uberblock. * * When the pool configuration is altered, either because it was newly created * or a device was added, we want to update all the labels such that we can deal * with fatal failure at any point. To this end, each disk has two labels which * are updated before and after the uberblock is synced. Assuming we have * labels and an uberblock with the following transaction groups: * * L1 UB L2 * +------+ +------+ +------+ * | | | | | | * | t10 | | t10 | | t10 | * | | | | | | * +------+ +------+ +------+ * * In this stable state, the labels and the uberblock were all updated within * the same transaction group (10). Each label is mirrored and checksummed, so * that we can detect when we fail partway through writing the label. * * In order to identify which labels are valid, the labels are written in the * following manner: * * 1. For each vdev, update 'L1' to the new label * 2. Update the uberblock * 3. For each vdev, update 'L2' to the new label * * Given arbitrary failure, we can determine the correct label to use based on * the transaction group. If we fail after updating L1 but before updating the * UB, we will notice that L1's transaction group is greater than the uberblock, * so L2 must be valid. If we fail after writing the uberblock but before * writing L2, we will notice that L2's transaction group is less than L1, and * therefore L1 is valid. * * Another added complexity is that not every label is updated when the config * is synced. If we add a single device, we do not want to have to re-write * every label for every device in the pool. This means that both L1 and L2 may * be older than the pool uberblock, because the necessary information is stored * on another vdev. * * * On-disk Format * -------------- * * The vdev label consists of two distinct parts, and is wrapped within the * vdev_label_t structure. The label includes 8k of padding to permit legacy * VTOC disk labels, but is otherwise ignored. * * The first half of the label is a packed nvlist which contains pool wide * properties, per-vdev properties, and configuration information. It is * described in more detail below. * * The latter half of the label consists of a redundant array of uberblocks. * These uberblocks are updated whenever a transaction group is committed, * or when the configuration is updated. When a pool is loaded, we scan each * vdev for the 'best' uberblock. * * * Configuration Information * ------------------------- * * The nvlist describing the pool and vdev contains the following elements: * * version ZFS on-disk version * name Pool name * state Pool state * txg Transaction group in which this label was written * pool_guid Unique identifier for this pool * vdev_tree An nvlist describing vdev tree. * features_for_read * An nvlist of the features necessary for reading the MOS. * * Each leaf device label also contains the following: * * top_guid Unique ID for top-level vdev in which this is contained * guid Unique ID for the leaf vdev * * The 'vs' configuration follows the format described in 'spa_config.c'. */ #include #include #include #include #include #include #include #include #include #include #include #include #include static boolean_t vdev_trim_on_init = B_TRUE; SYSCTL_DECL(_vfs_zfs_vdev); SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, trim_on_init, CTLFLAG_RW, &vdev_trim_on_init, 0, "Enable/disable full vdev trim on initialisation"); /* * Basic routines to read and write from a vdev label. * Used throughout the rest of this file. */ uint64_t vdev_label_offset(uint64_t psize, int l, uint64_t offset) { ASSERT(offset < sizeof (vdev_label_t)); ASSERT(P2PHASE_TYPED(psize, sizeof (vdev_label_t), uint64_t) == 0); return (offset + l * sizeof (vdev_label_t) + (l < VDEV_LABELS / 2 ? 0 : psize - VDEV_LABELS * sizeof (vdev_label_t))); } /* * Returns back the vdev label associated with the passed in offset. */ int vdev_label_number(uint64_t psize, uint64_t offset) { int l; if (offset >= psize - VDEV_LABEL_END_SIZE) { offset -= psize - VDEV_LABEL_END_SIZE; offset += (VDEV_LABELS / 2) * sizeof (vdev_label_t); } l = offset / sizeof (vdev_label_t); return (l < VDEV_LABELS ? l : -1); } static void vdev_label_read(zio_t *zio, vdev_t *vd, int l, void *buf, uint64_t offset, uint64_t size, zio_done_func_t *done, void *private, int flags) { ASSERT(spa_config_held(zio->io_spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); ASSERT(flags & ZIO_FLAG_CONFIG_WRITER); zio_nowait(zio_read_phys(zio, vd, vdev_label_offset(vd->vdev_psize, l, offset), size, buf, ZIO_CHECKSUM_LABEL, done, private, ZIO_PRIORITY_SYNC_READ, flags, B_TRUE)); } static void vdev_label_write(zio_t *zio, vdev_t *vd, int l, void *buf, uint64_t offset, uint64_t size, zio_done_func_t *done, void *private, int flags) { ASSERT(spa_config_held(zio->io_spa, SCL_ALL, RW_WRITER) == SCL_ALL || (spa_config_held(zio->io_spa, SCL_CONFIG | SCL_STATE, RW_READER) == (SCL_CONFIG | SCL_STATE) && dsl_pool_sync_context(spa_get_dsl(zio->io_spa)))); ASSERT(flags & ZIO_FLAG_CONFIG_WRITER); zio_nowait(zio_write_phys(zio, vd, vdev_label_offset(vd->vdev_psize, l, offset), size, buf, ZIO_CHECKSUM_LABEL, done, private, ZIO_PRIORITY_SYNC_WRITE, flags, B_TRUE)); } /* * Generate the nvlist representing this vdev's config. */ nvlist_t * vdev_config_generate(spa_t *spa, vdev_t *vd, boolean_t getstats, vdev_config_flag_t flags) { nvlist_t *nv = NULL; nv = fnvlist_alloc(); fnvlist_add_string(nv, ZPOOL_CONFIG_TYPE, vd->vdev_ops->vdev_op_type); if (!(flags & (VDEV_CONFIG_SPARE | VDEV_CONFIG_L2CACHE))) fnvlist_add_uint64(nv, ZPOOL_CONFIG_ID, vd->vdev_id); fnvlist_add_uint64(nv, ZPOOL_CONFIG_GUID, vd->vdev_guid); if (vd->vdev_path != NULL) fnvlist_add_string(nv, ZPOOL_CONFIG_PATH, vd->vdev_path); if (vd->vdev_devid != NULL) fnvlist_add_string(nv, ZPOOL_CONFIG_DEVID, vd->vdev_devid); if (vd->vdev_physpath != NULL) fnvlist_add_string(nv, ZPOOL_CONFIG_PHYS_PATH, vd->vdev_physpath); if (vd->vdev_fru != NULL) fnvlist_add_string(nv, ZPOOL_CONFIG_FRU, vd->vdev_fru); if (vd->vdev_nparity != 0) { ASSERT(strcmp(vd->vdev_ops->vdev_op_type, VDEV_TYPE_RAIDZ) == 0); /* * Make sure someone hasn't managed to sneak a fancy new vdev * into a crufty old storage pool. */ ASSERT(vd->vdev_nparity == 1 || (vd->vdev_nparity <= 2 && spa_version(spa) >= SPA_VERSION_RAIDZ2) || (vd->vdev_nparity <= 3 && spa_version(spa) >= SPA_VERSION_RAIDZ3)); /* * Note that we'll add the nparity tag even on storage pools * that only support a single parity device -- older software * will just ignore it. */ fnvlist_add_uint64(nv, ZPOOL_CONFIG_NPARITY, vd->vdev_nparity); } if (vd->vdev_wholedisk != -1ULL) fnvlist_add_uint64(nv, ZPOOL_CONFIG_WHOLE_DISK, vd->vdev_wholedisk); if (vd->vdev_not_present) fnvlist_add_uint64(nv, ZPOOL_CONFIG_NOT_PRESENT, 1); if (vd->vdev_isspare) fnvlist_add_uint64(nv, ZPOOL_CONFIG_IS_SPARE, 1); if (!(flags & (VDEV_CONFIG_SPARE | VDEV_CONFIG_L2CACHE)) && vd == vd->vdev_top) { fnvlist_add_uint64(nv, ZPOOL_CONFIG_METASLAB_ARRAY, vd->vdev_ms_array); fnvlist_add_uint64(nv, ZPOOL_CONFIG_METASLAB_SHIFT, vd->vdev_ms_shift); fnvlist_add_uint64(nv, ZPOOL_CONFIG_ASHIFT, vd->vdev_ashift); fnvlist_add_uint64(nv, ZPOOL_CONFIG_ASIZE, vd->vdev_asize); fnvlist_add_uint64(nv, ZPOOL_CONFIG_IS_LOG, vd->vdev_islog); if (vd->vdev_removing) fnvlist_add_uint64(nv, ZPOOL_CONFIG_REMOVING, vd->vdev_removing); } if (vd->vdev_dtl_sm != NULL) { fnvlist_add_uint64(nv, ZPOOL_CONFIG_DTL, space_map_object(vd->vdev_dtl_sm)); } if (vd->vdev_crtxg) fnvlist_add_uint64(nv, ZPOOL_CONFIG_CREATE_TXG, vd->vdev_crtxg); if (getstats) { vdev_stat_t vs; pool_scan_stat_t ps; vdev_get_stats(vd, &vs); fnvlist_add_uint64_array(nv, ZPOOL_CONFIG_VDEV_STATS, (uint64_t *)&vs, sizeof (vs) / sizeof (uint64_t)); /* provide either current or previous scan information */ if (spa_scan_get_stats(spa, &ps) == 0) { fnvlist_add_uint64_array(nv, ZPOOL_CONFIG_SCAN_STATS, (uint64_t *)&ps, sizeof (pool_scan_stat_t) / sizeof (uint64_t)); } } if (!vd->vdev_ops->vdev_op_leaf) { nvlist_t **child; int c, idx; ASSERT(!vd->vdev_ishole); child = kmem_alloc(vd->vdev_children * sizeof (nvlist_t *), KM_SLEEP); for (c = 0, idx = 0; c < vd->vdev_children; c++) { vdev_t *cvd = vd->vdev_child[c]; /* * If we're generating an nvlist of removing * vdevs then skip over any device which is * not being removed. */ if ((flags & VDEV_CONFIG_REMOVING) && !cvd->vdev_removing) continue; child[idx++] = vdev_config_generate(spa, cvd, getstats, flags); } if (idx) { fnvlist_add_nvlist_array(nv, ZPOOL_CONFIG_CHILDREN, child, idx); } for (c = 0; c < idx; c++) nvlist_free(child[c]); kmem_free(child, vd->vdev_children * sizeof (nvlist_t *)); } else { const char *aux = NULL; if (vd->vdev_offline && !vd->vdev_tmpoffline) fnvlist_add_uint64(nv, ZPOOL_CONFIG_OFFLINE, B_TRUE); if (vd->vdev_resilver_txg != 0) fnvlist_add_uint64(nv, ZPOOL_CONFIG_RESILVER_TXG, vd->vdev_resilver_txg); if (vd->vdev_faulted) fnvlist_add_uint64(nv, ZPOOL_CONFIG_FAULTED, B_TRUE); if (vd->vdev_degraded) fnvlist_add_uint64(nv, ZPOOL_CONFIG_DEGRADED, B_TRUE); if (vd->vdev_removed) fnvlist_add_uint64(nv, ZPOOL_CONFIG_REMOVED, B_TRUE); if (vd->vdev_unspare) fnvlist_add_uint64(nv, ZPOOL_CONFIG_UNSPARE, B_TRUE); if (vd->vdev_ishole) fnvlist_add_uint64(nv, ZPOOL_CONFIG_IS_HOLE, B_TRUE); switch (vd->vdev_stat.vs_aux) { case VDEV_AUX_ERR_EXCEEDED: aux = "err_exceeded"; break; case VDEV_AUX_EXTERNAL: aux = "external"; break; } if (aux != NULL) fnvlist_add_string(nv, ZPOOL_CONFIG_AUX_STATE, aux); if (vd->vdev_splitting && vd->vdev_orig_guid != 0LL) { fnvlist_add_uint64(nv, ZPOOL_CONFIG_ORIG_GUID, vd->vdev_orig_guid); } } return (nv); } /* * Generate a view of the top-level vdevs. If we currently have holes * in the namespace, then generate an array which contains a list of holey * vdevs. Additionally, add the number of top-level children that currently * exist. */ void vdev_top_config_generate(spa_t *spa, nvlist_t *config) { vdev_t *rvd = spa->spa_root_vdev; uint64_t *array; uint_t c, idx; array = kmem_alloc(rvd->vdev_children * sizeof (uint64_t), KM_SLEEP); for (c = 0, idx = 0; c < rvd->vdev_children; c++) { vdev_t *tvd = rvd->vdev_child[c]; if (tvd->vdev_ishole) array[idx++] = c; } if (idx) { VERIFY(nvlist_add_uint64_array(config, ZPOOL_CONFIG_HOLE_ARRAY, array, idx) == 0); } VERIFY(nvlist_add_uint64(config, ZPOOL_CONFIG_VDEV_CHILDREN, rvd->vdev_children) == 0); kmem_free(array, rvd->vdev_children * sizeof (uint64_t)); } /* * Returns the configuration from the label of the given vdev. For vdevs * which don't have a txg value stored on their label (i.e. spares/cache) * or have not been completely initialized (txg = 0) just return * the configuration from the first valid label we find. Otherwise, * find the most up-to-date label that does not exceed the specified * 'txg' value. */ nvlist_t * vdev_label_read_config(vdev_t *vd, uint64_t txg) { spa_t *spa = vd->vdev_spa; nvlist_t *config = NULL; vdev_phys_t *vp; zio_t *zio; uint64_t best_txg = 0; int error = 0; int flags = ZIO_FLAG_CONFIG_WRITER | ZIO_FLAG_CANFAIL | ZIO_FLAG_SPECULATIVE; ASSERT(spa_config_held(spa, SCL_STATE_ALL, RW_WRITER) == SCL_STATE_ALL); if (!vdev_readable(vd)) return (NULL); vp = zio_buf_alloc(sizeof (vdev_phys_t)); retry: for (int l = 0; l < VDEV_LABELS; l++) { nvlist_t *label = NULL; zio = zio_root(spa, NULL, NULL, flags); vdev_label_read(zio, vd, l, vp, offsetof(vdev_label_t, vl_vdev_phys), sizeof (vdev_phys_t), NULL, NULL, flags); if (zio_wait(zio) == 0 && nvlist_unpack(vp->vp_nvlist, sizeof (vp->vp_nvlist), &label, 0) == 0) { uint64_t label_txg = 0; /* * Auxiliary vdevs won't have txg values in their * labels and newly added vdevs may not have been * completely initialized so just return the * configuration from the first valid label we * encounter. */ error = nvlist_lookup_uint64(label, ZPOOL_CONFIG_POOL_TXG, &label_txg); if ((error || label_txg == 0) && !config) { config = label; break; } else if (label_txg <= txg && label_txg > best_txg) { best_txg = label_txg; nvlist_free(config); config = fnvlist_dup(label); } } if (label != NULL) { nvlist_free(label); label = NULL; } } if (config == NULL && !(flags & ZIO_FLAG_TRYHARD)) { flags |= ZIO_FLAG_TRYHARD; goto retry; } zio_buf_free(vp, sizeof (vdev_phys_t)); return (config); } /* * Determine if a device is in use. The 'spare_guid' parameter will be filled * in with the device guid if this spare is active elsewhere on the system. */ static boolean_t vdev_inuse(vdev_t *vd, uint64_t crtxg, vdev_labeltype_t reason, uint64_t *spare_guid, uint64_t *l2cache_guid) { spa_t *spa = vd->vdev_spa; uint64_t state, pool_guid, device_guid, txg, spare_pool; uint64_t vdtxg = 0; nvlist_t *label; if (spare_guid) *spare_guid = 0ULL; if (l2cache_guid) *l2cache_guid = 0ULL; /* * Read the label, if any, and perform some basic sanity checks. */ if ((label = vdev_label_read_config(vd, -1ULL)) == NULL) return (B_FALSE); (void) nvlist_lookup_uint64(label, ZPOOL_CONFIG_CREATE_TXG, &vdtxg); if (nvlist_lookup_uint64(label, ZPOOL_CONFIG_POOL_STATE, &state) != 0 || nvlist_lookup_uint64(label, ZPOOL_CONFIG_GUID, &device_guid) != 0) { nvlist_free(label); return (B_FALSE); } if (state != POOL_STATE_SPARE && state != POOL_STATE_L2CACHE && (nvlist_lookup_uint64(label, ZPOOL_CONFIG_POOL_GUID, &pool_guid) != 0 || nvlist_lookup_uint64(label, ZPOOL_CONFIG_POOL_TXG, &txg) != 0)) { nvlist_free(label); return (B_FALSE); } nvlist_free(label); /* * Check to see if this device indeed belongs to the pool it claims to * be a part of. The only way this is allowed is if the device is a hot * spare (which we check for later on). */ if (state != POOL_STATE_SPARE && state != POOL_STATE_L2CACHE && !spa_guid_exists(pool_guid, device_guid) && !spa_spare_exists(device_guid, NULL, NULL) && !spa_l2cache_exists(device_guid, NULL)) return (B_FALSE); /* * If the transaction group is zero, then this an initialized (but * unused) label. This is only an error if the create transaction * on-disk is the same as the one we're using now, in which case the * user has attempted to add the same vdev multiple times in the same * transaction. */ if (state != POOL_STATE_SPARE && state != POOL_STATE_L2CACHE && txg == 0 && vdtxg == crtxg) return (B_TRUE); /* * Check to see if this is a spare device. We do an explicit check for * spa_has_spare() here because it may be on our pending list of spares * to add. We also check if it is an l2cache device. */ if (spa_spare_exists(device_guid, &spare_pool, NULL) || spa_has_spare(spa, device_guid)) { if (spare_guid) *spare_guid = device_guid; switch (reason) { case VDEV_LABEL_CREATE: case VDEV_LABEL_L2CACHE: return (B_TRUE); case VDEV_LABEL_REPLACE: return (!spa_has_spare(spa, device_guid) || spare_pool != 0ULL); case VDEV_LABEL_SPARE: return (spa_has_spare(spa, device_guid)); } } /* * Check to see if this is an l2cache device. */ if (spa_l2cache_exists(device_guid, NULL)) return (B_TRUE); /* * We can't rely on a pool's state if it's been imported * read-only. Instead we look to see if the pools is marked * read-only in the namespace and set the state to active. */ if ((spa = spa_by_guid(pool_guid, device_guid)) != NULL && spa_mode(spa) == FREAD) state = POOL_STATE_ACTIVE; /* * If the device is marked ACTIVE, then this device is in use by another * pool on the system. */ return (state == POOL_STATE_ACTIVE); } /* * Initialize a vdev label. We check to make sure each leaf device is not in * use, and writable. We put down an initial label which we will later * overwrite with a complete label. Note that it's important to do this * sequentially, not in parallel, so that we catch cases of multiple use of the * same leaf vdev in the vdev we're creating -- e.g. mirroring a disk with * itself. */ int vdev_label_init(vdev_t *vd, uint64_t crtxg, vdev_labeltype_t reason) { spa_t *spa = vd->vdev_spa; nvlist_t *label; vdev_phys_t *vp; char *pad2; uberblock_t *ub; zio_t *zio; char *buf; size_t buflen; int error; uint64_t spare_guid, l2cache_guid; int flags = ZIO_FLAG_CONFIG_WRITER | ZIO_FLAG_CANFAIL; ASSERT(spa_config_held(spa, SCL_ALL, RW_WRITER) == SCL_ALL); for (int c = 0; c < vd->vdev_children; c++) if ((error = vdev_label_init(vd->vdev_child[c], crtxg, reason)) != 0) return (error); /* Track the creation time for this vdev */ vd->vdev_crtxg = crtxg; if (!vd->vdev_ops->vdev_op_leaf || !spa_writeable(spa)) return (0); /* * Dead vdevs cannot be initialized. */ if (vdev_is_dead(vd)) return (SET_ERROR(EIO)); /* * Determine if the vdev is in use. */ if (reason != VDEV_LABEL_REMOVE && reason != VDEV_LABEL_SPLIT && vdev_inuse(vd, crtxg, reason, &spare_guid, &l2cache_guid)) return (SET_ERROR(EBUSY)); /* * If this is a request to add or replace a spare or l2cache device * that is in use elsewhere on the system, then we must update the * guid (which was initialized to a random value) to reflect the * actual GUID (which is shared between multiple pools). */ if (reason != VDEV_LABEL_REMOVE && reason != VDEV_LABEL_L2CACHE && spare_guid != 0ULL) { uint64_t guid_delta = spare_guid - vd->vdev_guid; vd->vdev_guid += guid_delta; for (vdev_t *pvd = vd; pvd != NULL; pvd = pvd->vdev_parent) pvd->vdev_guid_sum += guid_delta; /* * If this is a replacement, then we want to fallthrough to the * rest of the code. If we're adding a spare, then it's already * labeled appropriately and we can just return. */ if (reason == VDEV_LABEL_SPARE) return (0); ASSERT(reason == VDEV_LABEL_REPLACE || reason == VDEV_LABEL_SPLIT); } if (reason != VDEV_LABEL_REMOVE && reason != VDEV_LABEL_SPARE && l2cache_guid != 0ULL) { uint64_t guid_delta = l2cache_guid - vd->vdev_guid; vd->vdev_guid += guid_delta; for (vdev_t *pvd = vd; pvd != NULL; pvd = pvd->vdev_parent) pvd->vdev_guid_sum += guid_delta; /* * If this is a replacement, then we want to fallthrough to the * rest of the code. If we're adding an l2cache, then it's * already labeled appropriately and we can just return. */ if (reason == VDEV_LABEL_L2CACHE) return (0); ASSERT(reason == VDEV_LABEL_REPLACE); } /* * TRIM the whole thing so that we start with a clean slate. * It's just an optimization, so we don't care if it fails. * Don't TRIM if removing so that we don't interfere with zpool * disaster recovery. */ - if (zfs_trim_enabled && vdev_trim_on_init && (reason == VDEV_LABEL_CREATE || - reason == VDEV_LABEL_SPARE || reason == VDEV_LABEL_L2CACHE)) + if (zfs_trim_enabled && vdev_trim_on_init && !vd->vdev_notrim && + (reason == VDEV_LABEL_CREATE || reason == VDEV_LABEL_SPARE || + reason == VDEV_LABEL_L2CACHE)) zio_wait(zio_trim(NULL, spa, vd, 0, vd->vdev_psize)); /* * Initialize its label. */ vp = zio_buf_alloc(sizeof (vdev_phys_t)); bzero(vp, sizeof (vdev_phys_t)); /* * Generate a label describing the pool and our top-level vdev. * We mark it as being from txg 0 to indicate that it's not * really part of an active pool just yet. The labels will * be written again with a meaningful txg by spa_sync(). */ if (reason == VDEV_LABEL_SPARE || (reason == VDEV_LABEL_REMOVE && vd->vdev_isspare)) { /* * For inactive hot spares, we generate a special label that * identifies as a mutually shared hot spare. We write the * label if we are adding a hot spare, or if we are removing an * active hot spare (in which case we want to revert the * labels). */ VERIFY(nvlist_alloc(&label, NV_UNIQUE_NAME, KM_SLEEP) == 0); VERIFY(nvlist_add_uint64(label, ZPOOL_CONFIG_VERSION, spa_version(spa)) == 0); VERIFY(nvlist_add_uint64(label, ZPOOL_CONFIG_POOL_STATE, POOL_STATE_SPARE) == 0); VERIFY(nvlist_add_uint64(label, ZPOOL_CONFIG_GUID, vd->vdev_guid) == 0); } else if (reason == VDEV_LABEL_L2CACHE || (reason == VDEV_LABEL_REMOVE && vd->vdev_isl2cache)) { /* * For level 2 ARC devices, add a special label. */ VERIFY(nvlist_alloc(&label, NV_UNIQUE_NAME, KM_SLEEP) == 0); VERIFY(nvlist_add_uint64(label, ZPOOL_CONFIG_VERSION, spa_version(spa)) == 0); VERIFY(nvlist_add_uint64(label, ZPOOL_CONFIG_POOL_STATE, POOL_STATE_L2CACHE) == 0); VERIFY(nvlist_add_uint64(label, ZPOOL_CONFIG_GUID, vd->vdev_guid) == 0); } else { uint64_t txg = 0ULL; if (reason == VDEV_LABEL_SPLIT) txg = spa->spa_uberblock.ub_txg; label = spa_config_generate(spa, vd, txg, B_FALSE); /* * Add our creation time. This allows us to detect multiple * vdev uses as described above, and automatically expires if we * fail. */ VERIFY(nvlist_add_uint64(label, ZPOOL_CONFIG_CREATE_TXG, crtxg) == 0); } buf = vp->vp_nvlist; buflen = sizeof (vp->vp_nvlist); error = nvlist_pack(label, &buf, &buflen, NV_ENCODE_XDR, KM_SLEEP); if (error != 0) { nvlist_free(label); zio_buf_free(vp, sizeof (vdev_phys_t)); /* EFAULT means nvlist_pack ran out of room */ return (error == EFAULT ? ENAMETOOLONG : EINVAL); } /* * Initialize uberblock template. */ ub = zio_buf_alloc(VDEV_UBERBLOCK_RING); bzero(ub, VDEV_UBERBLOCK_RING); *ub = spa->spa_uberblock; ub->ub_txg = 0; /* Initialize the 2nd padding area. */ pad2 = zio_buf_alloc(VDEV_PAD_SIZE); bzero(pad2, VDEV_PAD_SIZE); /* * Write everything in parallel. */ retry: zio = zio_root(spa, NULL, NULL, flags); for (int l = 0; l < VDEV_LABELS; l++) { vdev_label_write(zio, vd, l, vp, offsetof(vdev_label_t, vl_vdev_phys), sizeof (vdev_phys_t), NULL, NULL, flags); /* * Skip the 1st padding area. * Zero out the 2nd padding area where it might have * left over data from previous filesystem format. */ vdev_label_write(zio, vd, l, pad2, offsetof(vdev_label_t, vl_pad2), VDEV_PAD_SIZE, NULL, NULL, flags); vdev_label_write(zio, vd, l, ub, offsetof(vdev_label_t, vl_uberblock), VDEV_UBERBLOCK_RING, NULL, NULL, flags); } error = zio_wait(zio); if (error != 0 && !(flags & ZIO_FLAG_TRYHARD)) { flags |= ZIO_FLAG_TRYHARD; goto retry; } nvlist_free(label); zio_buf_free(pad2, VDEV_PAD_SIZE); zio_buf_free(ub, VDEV_UBERBLOCK_RING); zio_buf_free(vp, sizeof (vdev_phys_t)); /* * If this vdev hasn't been previously identified as a spare, then we * mark it as such only if a) we are labeling it as a spare, or b) it * exists as a spare elsewhere in the system. Do the same for * level 2 ARC devices. */ if (error == 0 && !vd->vdev_isspare && (reason == VDEV_LABEL_SPARE || spa_spare_exists(vd->vdev_guid, NULL, NULL))) spa_spare_add(vd); if (error == 0 && !vd->vdev_isl2cache && (reason == VDEV_LABEL_L2CACHE || spa_l2cache_exists(vd->vdev_guid, NULL))) spa_l2cache_add(vd); return (error); } /* * ========================================================================== * uberblock load/sync * ========================================================================== */ /* * Consider the following situation: txg is safely synced to disk. We've * written the first uberblock for txg + 1, and then we lose power. When we * come back up, we fail to see the uberblock for txg + 1 because, say, * it was on a mirrored device and the replica to which we wrote txg + 1 * is now offline. If we then make some changes and sync txg + 1, and then * the missing replica comes back, then for a few seconds we'll have two * conflicting uberblocks on disk with the same txg. The solution is simple: * among uberblocks with equal txg, choose the one with the latest timestamp. */ static int vdev_uberblock_compare(uberblock_t *ub1, uberblock_t *ub2) { if (ub1->ub_txg < ub2->ub_txg) return (-1); if (ub1->ub_txg > ub2->ub_txg) return (1); if (ub1->ub_timestamp < ub2->ub_timestamp) return (-1); if (ub1->ub_timestamp > ub2->ub_timestamp) return (1); return (0); } struct ubl_cbdata { uberblock_t *ubl_ubbest; /* Best uberblock */ vdev_t *ubl_vd; /* vdev associated with the above */ }; static void vdev_uberblock_load_done(zio_t *zio) { vdev_t *vd = zio->io_vd; spa_t *spa = zio->io_spa; zio_t *rio = zio->io_private; uberblock_t *ub = zio->io_data; struct ubl_cbdata *cbp = rio->io_private; ASSERT3U(zio->io_size, ==, VDEV_UBERBLOCK_SIZE(vd)); if (zio->io_error == 0 && uberblock_verify(ub) == 0) { mutex_enter(&rio->io_lock); if (ub->ub_txg <= spa->spa_load_max_txg && vdev_uberblock_compare(ub, cbp->ubl_ubbest) > 0) { /* * Keep track of the vdev in which this uberblock * was found. We will use this information later * to obtain the config nvlist associated with * this uberblock. */ *cbp->ubl_ubbest = *ub; cbp->ubl_vd = vd; } mutex_exit(&rio->io_lock); } zio_buf_free(zio->io_data, zio->io_size); } static void vdev_uberblock_load_impl(zio_t *zio, vdev_t *vd, int flags, struct ubl_cbdata *cbp) { for (int c = 0; c < vd->vdev_children; c++) vdev_uberblock_load_impl(zio, vd->vdev_child[c], flags, cbp); if (vd->vdev_ops->vdev_op_leaf && vdev_readable(vd)) { for (int l = 0; l < VDEV_LABELS; l++) { for (int n = 0; n < VDEV_UBERBLOCK_COUNT(vd); n++) { vdev_label_read(zio, vd, l, zio_buf_alloc(VDEV_UBERBLOCK_SIZE(vd)), VDEV_UBERBLOCK_OFFSET(vd, n), VDEV_UBERBLOCK_SIZE(vd), vdev_uberblock_load_done, zio, flags); } } } } /* * Reads the 'best' uberblock from disk along with its associated * configuration. First, we read the uberblock array of each label of each * vdev, keeping track of the uberblock with the highest txg in each array. * Then, we read the configuration from the same vdev as the best uberblock. */ void vdev_uberblock_load(vdev_t *rvd, uberblock_t *ub, nvlist_t **config) { zio_t *zio; spa_t *spa = rvd->vdev_spa; struct ubl_cbdata cb; int flags = ZIO_FLAG_CONFIG_WRITER | ZIO_FLAG_CANFAIL | ZIO_FLAG_SPECULATIVE | ZIO_FLAG_TRYHARD; ASSERT(ub); ASSERT(config); bzero(ub, sizeof (uberblock_t)); *config = NULL; cb.ubl_ubbest = ub; cb.ubl_vd = NULL; spa_config_enter(spa, SCL_ALL, FTAG, RW_WRITER); zio = zio_root(spa, NULL, &cb, flags); vdev_uberblock_load_impl(zio, rvd, flags, &cb); (void) zio_wait(zio); /* * It's possible that the best uberblock was discovered on a label * that has a configuration which was written in a future txg. * Search all labels on this vdev to find the configuration that * matches the txg for our uberblock. */ if (cb.ubl_vd != NULL) *config = vdev_label_read_config(cb.ubl_vd, ub->ub_txg); spa_config_exit(spa, SCL_ALL, FTAG); } /* * On success, increment root zio's count of good writes. * We only get credit for writes to known-visible vdevs; see spa_vdev_add(). */ static void vdev_uberblock_sync_done(zio_t *zio) { uint64_t *good_writes = zio->io_private; if (zio->io_error == 0 && zio->io_vd->vdev_top->vdev_ms_array != 0) atomic_inc_64(good_writes); } /* * Write the uberblock to all labels of all leaves of the specified vdev. */ static void vdev_uberblock_sync(zio_t *zio, uberblock_t *ub, vdev_t *vd, int flags) { uberblock_t *ubbuf; int n; for (int c = 0; c < vd->vdev_children; c++) vdev_uberblock_sync(zio, ub, vd->vdev_child[c], flags); if (!vd->vdev_ops->vdev_op_leaf) return; if (!vdev_writeable(vd)) return; n = ub->ub_txg & (VDEV_UBERBLOCK_COUNT(vd) - 1); ubbuf = zio_buf_alloc(VDEV_UBERBLOCK_SIZE(vd)); bzero(ubbuf, VDEV_UBERBLOCK_SIZE(vd)); *ubbuf = *ub; for (int l = 0; l < VDEV_LABELS; l++) vdev_label_write(zio, vd, l, ubbuf, VDEV_UBERBLOCK_OFFSET(vd, n), VDEV_UBERBLOCK_SIZE(vd), vdev_uberblock_sync_done, zio->io_private, flags | ZIO_FLAG_DONT_PROPAGATE); zio_buf_free(ubbuf, VDEV_UBERBLOCK_SIZE(vd)); } /* Sync the uberblocks to all vdevs in svd[] */ int vdev_uberblock_sync_list(vdev_t **svd, int svdcount, uberblock_t *ub, int flags) { spa_t *spa = svd[0]->vdev_spa; zio_t *zio; uint64_t good_writes = 0; zio = zio_root(spa, NULL, &good_writes, flags); for (int v = 0; v < svdcount; v++) vdev_uberblock_sync(zio, ub, svd[v], flags); (void) zio_wait(zio); /* * Flush the uberblocks to disk. This ensures that the odd labels * are no longer needed (because the new uberblocks and the even * labels are safely on disk), so it is safe to overwrite them. */ zio = zio_root(spa, NULL, NULL, flags); for (int v = 0; v < svdcount; v++) zio_flush(zio, svd[v]); (void) zio_wait(zio); return (good_writes >= 1 ? 0 : EIO); } /* * On success, increment the count of good writes for our top-level vdev. */ static void vdev_label_sync_done(zio_t *zio) { uint64_t *good_writes = zio->io_private; if (zio->io_error == 0) atomic_inc_64(good_writes); } /* * If there weren't enough good writes, indicate failure to the parent. */ static void vdev_label_sync_top_done(zio_t *zio) { uint64_t *good_writes = zio->io_private; if (*good_writes == 0) zio->io_error = SET_ERROR(EIO); kmem_free(good_writes, sizeof (uint64_t)); } /* * We ignore errors for log and cache devices, simply free the private data. */ static void vdev_label_sync_ignore_done(zio_t *zio) { kmem_free(zio->io_private, sizeof (uint64_t)); } /* * Write all even or odd labels to all leaves of the specified vdev. */ static void vdev_label_sync(zio_t *zio, vdev_t *vd, int l, uint64_t txg, int flags) { nvlist_t *label; vdev_phys_t *vp; char *buf; size_t buflen; for (int c = 0; c < vd->vdev_children; c++) vdev_label_sync(zio, vd->vdev_child[c], l, txg, flags); if (!vd->vdev_ops->vdev_op_leaf) return; if (!vdev_writeable(vd)) return; /* * Generate a label describing the top-level config to which we belong. */ label = spa_config_generate(vd->vdev_spa, vd, txg, B_FALSE); vp = zio_buf_alloc(sizeof (vdev_phys_t)); bzero(vp, sizeof (vdev_phys_t)); buf = vp->vp_nvlist; buflen = sizeof (vp->vp_nvlist); if (nvlist_pack(label, &buf, &buflen, NV_ENCODE_XDR, KM_SLEEP) == 0) { for (; l < VDEV_LABELS; l += 2) { vdev_label_write(zio, vd, l, vp, offsetof(vdev_label_t, vl_vdev_phys), sizeof (vdev_phys_t), vdev_label_sync_done, zio->io_private, flags | ZIO_FLAG_DONT_PROPAGATE); } } zio_buf_free(vp, sizeof (vdev_phys_t)); nvlist_free(label); } int vdev_label_sync_list(spa_t *spa, int l, uint64_t txg, int flags) { list_t *dl = &spa->spa_config_dirty_list; vdev_t *vd; zio_t *zio; int error; /* * Write the new labels to disk. */ zio = zio_root(spa, NULL, NULL, flags); for (vd = list_head(dl); vd != NULL; vd = list_next(dl, vd)) { uint64_t *good_writes = kmem_zalloc(sizeof (uint64_t), KM_SLEEP); ASSERT(!vd->vdev_ishole); zio_t *vio = zio_null(zio, spa, NULL, (vd->vdev_islog || vd->vdev_aux != NULL) ? vdev_label_sync_ignore_done : vdev_label_sync_top_done, good_writes, flags); vdev_label_sync(vio, vd, l, txg, flags); zio_nowait(vio); } error = zio_wait(zio); /* * Flush the new labels to disk. */ zio = zio_root(spa, NULL, NULL, flags); for (vd = list_head(dl); vd != NULL; vd = list_next(dl, vd)) zio_flush(zio, vd); (void) zio_wait(zio); return (error); } /* * Sync the uberblock and any changes to the vdev configuration. * * The order of operations is carefully crafted to ensure that * if the system panics or loses power at any time, the state on disk * is still transactionally consistent. The in-line comments below * describe the failure semantics at each stage. * * Moreover, vdev_config_sync() is designed to be idempotent: if it fails * at any time, you can just call it again, and it will resume its work. */ int vdev_config_sync(vdev_t **svd, int svdcount, uint64_t txg, boolean_t tryhard) { spa_t *spa = svd[0]->vdev_spa; uberblock_t *ub = &spa->spa_uberblock; vdev_t *vd; zio_t *zio; int error; int flags = ZIO_FLAG_CONFIG_WRITER | ZIO_FLAG_CANFAIL; /* * Normally, we don't want to try too hard to write every label and * uberblock. If there is a flaky disk, we don't want the rest of the * sync process to block while we retry. But if we can't write a * single label out, we should retry with ZIO_FLAG_TRYHARD before * bailing out and declaring the pool faulted. */ if (tryhard) flags |= ZIO_FLAG_TRYHARD; ASSERT(ub->ub_txg <= txg); /* * If this isn't a resync due to I/O errors, * and nothing changed in this transaction group, * and the vdev configuration hasn't changed, * then there's nothing to do. */ if (ub->ub_txg < txg && uberblock_update(ub, spa->spa_root_vdev, txg) == B_FALSE && list_is_empty(&spa->spa_config_dirty_list)) return (0); if (txg > spa_freeze_txg(spa)) return (0); ASSERT(txg <= spa->spa_final_txg); /* * Flush the write cache of every disk that's been written to * in this transaction group. This ensures that all blocks * written in this txg will be committed to stable storage * before any uberblock that references them. */ zio = zio_root(spa, NULL, NULL, flags); for (vd = txg_list_head(&spa->spa_vdev_txg_list, TXG_CLEAN(txg)); vd; vd = txg_list_next(&spa->spa_vdev_txg_list, vd, TXG_CLEAN(txg))) zio_flush(zio, vd); (void) zio_wait(zio); /* * Sync out the even labels (L0, L2) for every dirty vdev. If the * system dies in the middle of this process, that's OK: all of the * even labels that made it to disk will be newer than any uberblock, * and will therefore be considered invalid. The odd labels (L1, L3), * which have not yet been touched, will still be valid. We flush * the new labels to disk to ensure that all even-label updates * are committed to stable storage before the uberblock update. */ if ((error = vdev_label_sync_list(spa, 0, txg, flags)) != 0) return (error); /* * Sync the uberblocks to all vdevs in svd[]. * If the system dies in the middle of this step, there are two cases * to consider, and the on-disk state is consistent either way: * * (1) If none of the new uberblocks made it to disk, then the * previous uberblock will be the newest, and the odd labels * (which had not yet been touched) will be valid with respect * to that uberblock. * * (2) If one or more new uberblocks made it to disk, then they * will be the newest, and the even labels (which had all * been successfully committed) will be valid with respect * to the new uberblocks. */ if ((error = vdev_uberblock_sync_list(svd, svdcount, ub, flags)) != 0) return (error); /* * Sync out odd labels for every dirty vdev. If the system dies * in the middle of this process, the even labels and the new * uberblocks will suffice to open the pool. The next time * the pool is opened, the first thing we'll do -- before any * user data is modified -- is mark every vdev dirty so that * all labels will be brought up to date. We flush the new labels * to disk to ensure that all odd-label updates are committed to * stable storage before the next transaction group begins. */ if ((error = vdev_label_sync_list(spa, 1, txg, flags)) != 0) return (error); trim_thread_wakeup(spa); return (0); } Index: releng/10.1/sys/conf/newvers.sh =================================================================== --- releng/10.1/sys/conf/newvers.sh (revision 284192) +++ releng/10.1/sys/conf/newvers.sh (revision 284193) @@ -1,224 +1,224 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="10.1" -BRANCH="RELEASE-p10" +BRANCH="RELEASE-p11" if [ "X${BRANCH_OVERRIDE}" != "X" ]; then BRANCH=${BRANCH_OVERRIDE} fi RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" if [ "X${SYSDIR}" = "X" ]; then SYSDIR=$(dirname $0)/.. fi if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${SYSDIR}/sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/1992-$year/" \ -e 's/\[your name here\]\.* /The FreeBSD Project./' \ -e 's/\[your name\]\.*/The FreeBSD Project./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/*- * Copyright (c) 1992-$year The FreeBSD Project. * All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} t=`date` i=`${MAKE:-make} -V KERN_IDENT` compiler_v=$($(${MAKE:-make} -V CC) -v 2>&1 | grep 'version') for dir in /usr/bin /usr/local/bin; do if [ ! -z "${svnversion}" ] ; then break fi if [ -x "${dir}/svnversion" ] && [ -z ${svnversion} ] ; then # Run svnversion from ${dir} on this script; if return code # is not zero, the checkout might not be compatible with the # svnversion being used. ${dir}/svnversion $(realpath ${0}) >/dev/null 2>&1 if [ $? -eq 0 ]; then svnversion=${dir}/svnversion break fi fi done if [ -z "${svnversion}" ] && [ -x /usr/bin/svnliteversion ] ; then /usr/bin/svnliteversion $(realpath ${0}) >/dev/null 2>&1 if [ $? -eq 0 ]; then svnversion=/usr/bin/svnliteversion else svnversion= fi fi for dir in /usr/bin /usr/local/bin; do if [ -x "${dir}/p4" ] && [ -z ${p4_cmd} ] ; then p4_cmd=${dir}/p4 fi done if [ -d "${SYSDIR}/../.git" ] ; then for dir in /usr/bin /usr/local/bin; do if [ -x "${dir}/git" ] ; then git_cmd="${dir}/git --git-dir=${SYSDIR}/../.git" break fi done fi if [ -d "${SYSDIR}/../.hg" ] ; then for dir in /usr/bin /usr/local/bin; do if [ -x "${dir}/hg" ] ; then hg_cmd="${dir}/hg -R ${SYSDIR}/.." break fi done fi if [ -n "$svnversion" ] ; then svn=`cd ${SYSDIR} && $svnversion 2>/dev/null` case "$svn" in [0-9]*) svn=" r${svn}" ;; *) unset svn ;; esac fi if [ -n "$git_cmd" ] ; then git=`$git_cmd rev-parse --verify --short HEAD 2>/dev/null` svn=`$git_cmd svn find-rev $git 2>/dev/null` if [ -n "$svn" ] ; then svn=" r${svn}" git="=${git}" else svn=`$git_cmd log | fgrep 'git-svn-id:' | head -1 | \ sed -n 's/^.*@\([0-9][0-9]*\).*$/\1/p'` if [ -z "$svn" ] ; then svn=`$git_cmd log --format='format:%N' | \ grep '^svn ' | head -1 | \ sed -n 's/^.*revision=\([0-9][0-9]*\).*$/\1/p'` fi if [ -n "$svn" ] ; then svn=" r${svn}" git="+${git}" else git=" ${git}" fi fi git_b=`$git_cmd rev-parse --abbrev-ref HEAD` if [ -n "$git_b" ] ; then git="${git}(${git_b})" fi if $git_cmd --work-tree=${SYSDIR}/.. diff-index \ --name-only HEAD | read dummy; then git="${git}-dirty" fi fi if [ -n "$p4_cmd" ] ; then p4version=`cd ${SYSDIR} && $p4_cmd changes -m1 "./...#have" 2>&1 | \ awk '{ print $2 }'` case "$p4version" in [0-9]*) p4version=" ${p4version}" p4opened=`cd ${SYSDIR} && $p4_cmd opened ./... 2>&1` case "$p4opened" in File*) ;; //*) p4version="${p4version}+edit" ;; esac ;; *) unset p4version ;; esac fi if [ -n "$hg_cmd" ] ; then hg=`$hg_cmd id 2>/dev/null` svn=`$hg_cmd svn info 2>/dev/null | \ awk -F': ' '/Revision/ { print $2 }'` if [ -n "$svn" ] ; then svn=" r${svn}" fi if [ -n "$hg" ] ; then hg=" ${hg}" fi fi cat << EOF > vers.c $COPYRIGHT #define SCCSSTR "@(#)${VERSION} #${v}${svn}${git}${hg}${p4version}: ${t}" #define VERSTR "${VERSION} #${v}${svn}${git}${hg}${p4version}: ${t}\\n ${u}@${h}:${d}\\n" #define RELSTR "${RELEASE}" char sccs[sizeof(SCCSSTR) > 128 ? sizeof(SCCSSTR) : 128] = SCCSSTR; char version[sizeof(VERSTR) > 256 ? sizeof(VERSTR) : 256] = VERSTR; char compiler_version[] = "${compiler_v}"; char ostype[] = "${TYPE}"; char osrelease[sizeof(RELSTR) > 32 ? sizeof(RELSTR) : 32] = RELSTR; int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF echo $((v + 1)) > version