Index: releng/4.11/UPDATING =================================================================== --- releng/4.11/UPDATING (revision 162446) +++ releng/4.11/UPDATING (revision 162447) @@ -1,1290 +1,1293 @@ Updating Information for FreeBSD STABLE users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. A reverse chronology since 4.0 was released is included, followed by the common items quick how-tos, followed by entries for versions of -current prior to 4.0 Release. This is for the 4.11 release branch. All entries since 4.11 are an itemized list of commits to this branch, numbered from the beginning. The security advisories related to various patches contain information on how to build/install a minimal set of binaries and start/stop a minimal number of processes, if possible, for that patch. For those updates that don't have an advisory, or to be safe, you can do a full build and install as described in the COMMON ITEMS section. +20060919: p22 FreeBSD-SA-06:21.gzip + Correct multiple vulnerabilities in gzip(1). + 20060906: p21 FreeBSD-SA-06:19.openssl Correct incorrect PKCS#1 v1.5 padding validation in crypto(3). 20060823: p20 FreeBSD-SA-06:18.ppp Correct buffer overflow in the handling of LCP options in ppp(4). 20060614: p19 FreeBSD-SA-06:17.sendmail Correct a bug in the handling of multipart messages by sendmail(8) which can allow a malformed message to crash a sendmail queue processing process. 20060531: p18 FreeBSD-SA-06:16.smbfs Correct a bug in the handling of backslash characters in smbfs which can allow an attacker to escape from a chroot(2). 20060419: p17 FreeBSD-SA-06:14.fpu Correct a local information leakage bug affecting AMD FPUs. 20060322: p16 FreeBSD-SA-06:11.ipsec, FreeBSD-SA-06:12.opie, FreeBSD-SA-06:13.sendmail Add missing code needed for the detection of IPSec packet replays. [06:11] Correctly identify the user running opiepasswd(1) when the login name differs from the account name. [06:12] Modify timeout handling logic in sendmail(8) to correct a reported signal handling race condition. [06:13] 20060301: p15 FreeBSD-SA-06:10.nfs Correct a remote kernel panic when processing zero-length RPC records via TCP. 20060111: p14 FreeBSD-SA-06:01.texindex, FreeBSD-SA-06:02.ee, FreeBSD-SA-06:03.cpio Correct insecure temporary file usage in texindex. [06:01] Correct insecure temporary file usage in ee. [06:02] Correct a race condition when setting file permissions, sanitize file names by default, and fix a buffer overflow when handling files larger than 4GB in cpio. [06:03] 20051011: p13 FreeBSD-SA-05:21.openssl Correct a man-in-the-middle SSL version rollback vulnerability. 20050907: p12 FreeBSD-SA-05:20.cvsbug Correct insecure temporary file usage. 20050629: p11 FreeBSD-SA-05:14.bzip2, FreeBSD-SA-05:15.tcp Correct bzip2 denial of service and permission race vulnerabilities. Correct TCP connection stall denial of service vulnerabilities. 20050608: p10 FreeBSD-SA-05:11.gzip Correct directory traversal and race condition vulnerabilities in gzip. 20050513: p9 FreeBSD-SA-05:09.htt Add a knob for disabling/enabling HTT. Default off due to information disclosure on multi-user systems. 20050508: p8 FreeBSD-SA-05:08.kmem Correct two issues which were missed from the earlier commit. 20050506: p7 FreeBSD-SA-05:08.kmem Correct several local kernel memory disclosure bugs. 20050506: p6 FreeBSD-SA-05:07.ldt Correctly validate inputs to the i386_get_ldt syscall. 20050506: p5 FreeBSD-SA-05:06.iir Correct overly liberal permissions on /dev/iir. 20050422: p4 FreeBSD-SA-05:05.cvs Correct several vulnerabilities in CVS. 20050414: p3 FreeBSD-SA-05:04.ifconf Zero a buffer in ifconf() in order to avoid accidental disclosure of kernel memory to userland. 20050404: p2 FreeBSD-SA-05:02.sendfile Correct kernel memory disclosure bug in sendfile(2). 20050328: p1 FreeBSD-SA-05:01.telnet Correct buffer overflows in telnet(1). 20050124: FreeBSD 4.11-RELEASE. 20040527: FreeBSD 4.10-RELEASE. 20031028: FreeBSD 4.9-RELEASE. 20031022: Support for HyperThread logical CPUs has now been enabled by default. As a result, the HTT kernel option no longer exists. Instead, the logical CPUs are always started so that they can handle interrupts. However, the extra logical CPUs are prevented from executing user processes by default. To enable the logical CPUs, change the value of the machdep.hlt_logical_cpus from 1 to 0. This value can also be set from the loader as a tunable of the same name. 20030923: Fix a bug in arplookup(), whereby a hostile party on a locally attached network could exhaust kernel memory, and cause a system panic, by sending a flood of spoofed ARP requests. See FreeBSD-SA-03:14.arp. 20030904: If you get errors regarding sendmail.cf files while trying to run mergemaster, you should either install the new version, or run /usr/src/usr.sbin/mergemaster/mergemaster.sh. This is a one-time issue to fix a Makefile compatibility problem. 20030904: Between August 9th and August 30th, a bug existed in the i386 virtual memory system which could cause panics under load. Anyone running a kernel built between those dates is advised to update at the earliest possible convenience. 20030806: All current USB quirks in da(4) have been deprecated and will be removed for 4.9. If this causes failure for your umass(4) devices, enable "options DA_OLD_QUIRKS" in your kernel and send the output of "camcontrol inquiry da0" to scsi@freebsd.org so the quirk can be re-enabled. 20030403: FreeBSD 4.8-RELEASE. 20030214: OpenSSL 0.97 has been imported, and the libcrypto/libssl library versions have been bumped. 20030213: sendmail 8.12.7 has been imported. It has one important change for IPv6 users. The default submit.mc now uses '[127.0.0.1]' instead of 'localhost' meaning only IPv4 is used to connect to the MTA. Users on IPv6-only machines will need to edit /etc/mail/submit.mc appropriately. 20021110: The definitions of the standard file streams (stdin, stdout, and stderr) have changed so that they are no longer compile-time constants. Some older binaries may require updated 3.X compatability libraries (for example, by setting COMPAT3X=yes for a buildworld/installworld). 20021010: FreeBSD 4.7-RELEASE. 20020807: COPY is being deprecated. The 20020703 change was reverted, as it causes far more pain than was expected, and to always compare before installing, please use INSTALL="install -C" again. The -C option is now silently ignored when used with the -d option. 20020704: OpenSSH has been upgraded to 3.4p1 and fully PAMified. Note that the protocol version now defaults to 2 rather than 1, which means clients might get warnings about unknown host keys their hostfiles only list the ssh1 RSA key. See ssh_config(5) and sshd_config(5) for more information on configuring ssh and sshd. 20020703: The install(1) utility no longer allows the -d and -C options to be specified together. "INSTALL=install -C" in make.conf should be replaced with "COPY=-C". 20020615: FreeBSD 4.6-RELEASE. 20020515: The k5su utility installed as part of Kerberos 5 is no longer installed with the set-user-ID bit set by default. Add ENABLE_SUID_K5SU=yes to /etc/make.conf to have it installed with the set-user-ID bit set. 20020501: The ephemeral port range change that was previously mentioned in this document has been reverted, so any changes it required to your system may be reverted as well. Those who missed the change should ignore this notice. 20020421: inetd_enable was changed back to YES, to match the defaults earlier in the 4.0-stable branch. This change restores the daemons that were enabled prior to April 15th. When exec'ing set[ug]id executables, the kernel now ensures that the stdio file descriptors (0..2) are open. See FreeBSD-SA-02:23.stdio. 20020420: sendmail_enable was changed back to YES, to match the defaults earlier in the 4.0-stable branch. 20020415: Many denizens of /etc were MFC'ed. While most of the changes are very benign, some of them have created a bit of a stir. Use the new version of mergemaster to update /etc, either by updating /etc after installworld, or by doing: cd /usr/src/usr.sbin/mergemaster; make -m /usr/src/share/mk all install before you run installworld. Then use the new -C option to double-check your rc.conf settings. You should also double-check settings in [defaults/]rc.conf, syslog.conf, newsyslog.conf, and anything else important to your system. The two most important changes to note: sendmail and inetd were turned off by default. If you need them, please turn them back on by adding sendmail_enable=YES and/or inetd_enable=YES to /etc/rc.conf. 20020411: New sendmail startup scripts have been installed to make it easier to use alternative MTAs with FreeBSD. The rc.conf variable mta_start_script can be set to a script for starting an alternative MTA or to "" to prevent any MTA from being started at boot. 20020404: The sendmail startup code in /etc/rc has been changed to deal with a non-set-user-ID sendmail binary. Setting the rc.conf variable sendmail_enable to "NO" no longer prevents any sendmail daemons from starting. Instead, set sendmail_enable to "NONE" to prevent any MTA from being started at boot. 20020404: Due to the import of sendmail 8.12.2 (see 20020325 entry), a new user and group are required in order for sendmail to run as a set-group-ID binary. A 'make installworld' will use the new user and group to set the owner and group of /var/spool/clientmqueue and will fail if the new user and group do not exist. The 'smmsp' user and group must be merged from src/etc/group and src/etc/master.passwd before using 'make installworld'. 'mergemaster -p' will do this. You may need to install mergemaster before this will work if you are updating from a very old version of stable. This can be done with: cd /usr/src/usr.sbin/mergemaster; make -m /usr/src/share/mk all install 20020325: sendmail 8.12.2 has been imported. The sendmail binary is no longer a set-user-ID root binary and the infrastructure to support command line mail submission has changed. Be sure to run mergemaster (especially for updating /etc/rc, /etc/defaults/rc.conf, and /etc/mail) and read /etc/mail/README for more details. 20020318: The ata(4) driver has been updated from the version in -CURRENT. As a result, some device nodes in /dev have changed; it will be necessary to update MAKEDEV and re-make the device nodes. mergemaster(8) will offer to do this as a part of its upgrade procedure. 20020129: FreeBSD 4.5-RELEASE. 20020105: The locale names have been renamed in -stable to match -current: 1. ISO_* -> ISO* 2. ru_SU* -> ru_RU* 3. DIS_* -> ISO*-15 4. *.ASCII -> *.US-ASCII 20011202: A security hole in OpenSSH involving `UseLogin yes' has been patched. 20011110: Some linux module changes, merged from current, require that you clean out the old compile directory. If you are building with MODULES_WITH_WORLD=yes, then you need to cd to src/sys/modules/linux and run "make cleandir". If not, then you need to cd src/sys/compile/$KERNCONF and do a make modules-clean. 20011102: ipfw/bridge/dummynet features merged from current. 20011018: The kernel Makefile was enhanced to pass -g to modules builds when it is used for kernel building (eg, when the config file has DEBUG=-g). People that have enabled debugging kernels will find they need significantly more disk space in their slash (/) partition. 20010928: The OpenSSH `ChallengeResponseAuthentication' option now defaults to `yes'. This enables S/Key authentication, which can result in users receiving an S/Key challenge when logging in interactively. To disable this behavior, add a line to your /etc/ssh/sshd_config file: `ChallengeResponseAuthentication no' 20010914: 4.4 Released. 20010814: The pci attachment for pcic device was merged from current. You should update your pccardd at the same time as you update your kernel. Note: Interrupts will now be shared between the CardBus bridge and the cards. This is a change over the hand configuration before. 20010811: isdn4bsd was updated to version 1.00.1. for users of the isp (kernel PPP over ISDN) devices, this means that they now _must_ use ispppcontrol instead of spppcontrol to configure and control their isp devices. see /usr/share/examples/isdn for more info. 20010725: The ed driver has been updated. It now allows mii attachments, which means that you must include the miibus in your kernel if you use the ed driver. 20010724: The gif(4) interface has been converted to use cloning rather then allocating a static number of devices a compile time. If you use gif interfaces and configure them by any method other then setting standard variables in /etc/rc.conf you will need to update your system to create devices with "ifconfig gifX create" before attempting to configure them. See the ifconfig(8) man page and /etc/rc.network for more information. 20010611: The TCP_RESTRICT_RST option has been removed. You should entirely remove it from your kernel configuration. The same functionality is available from the blackhole(4) MIB. 20010531: The wd compat devices were removed from the ata driver. Kernels after this date will fail to work with /dev/ nodes that you may have on your disk. /etc/fstab is a common place to find these files hanging out, as are dump devices in /etc/rc.conf. 20010521: Minor repo damange has happened. This may cause problems with cvsup of ports. If you get errors, please see http://www.FreeBSD.org/cgi/query-pr.cgi?pr=27495 at the bottom for details on a workaround. The error message is Updater failed: Cannot delete "/usr/ports/www/jakarta-tomcat/files": Directory not empty 20010516: Size changes in the cred structures introduced a binary incompatibility for modules using the crhold() macro. Modules should be recompiled for kernel later than this date. 20010510: The fxp driver from current has been ported to -stable. This requires kernels that include fxp to also include mii bus devices. See GENERIC for details. 20010421: FreeBSD 4.3 RELEASE 20010402: It appears that PERL_THREADED=true in /etc/make.conf is not compatible with mod_perl. Not sure the exact date of this problem, but it post dates 4.2 and predates 4.3. 20010202: The KERNEL variable for buildkernel and installkernel has been deprecated in favor of KERNCONF. make kernel now combines these two steps. 20010117: Linksys Fast Ethernet PCCARD cards supported by the ed driver now require the addition of flag 0x80000 to their config line in pccard.conf(5). This flag is not optional. These Linksys cards will not be recognized without it. 20010112: Important new FreeBSD-version stuff: PAM support has been worked in, partially from the "Unix" OpenSSH version. This requires adding the following in pam.conf: sshd auth sufficient pam_skey.so sshd auth required pam_unix.so try_first_pass sshd session required pam_permit.so 20010109: ipfw interface changed. Make sure that the userland and kernel match or you won't have the firewall rules you think you do. 20001120: FreeBSD 4.2 RELEASE. 20001020: ****************************** WARNING ****************************** Sendmail has been updated. ****************************** WARNING ****************************** o mail.local(8) is no longer installed as a set-user-id binary. o sendmail(8) is now built with STARTTLS support unless NO_OPENSSL is set. o The default /etc/mail/sendmail.cf disables the SMTP EXPN and VRFY commands. o Now using sendmail's version of vacation(1). o The sendmail cf building tools (contrib/sendmail/cf) are installed in /usr/share/sendmail/cf. o sendmail.cw changed to local-host-names More details can be found at http://people.freebsd.org/~imp/UPDATING/sendmail-20001010 20001009: The ports tree's new layout is in place. Be sure to update your entire ports tree, or you will have problems. 20001006: The perl build procedure no longer installs miniperl, nor uses the installed miniperl. It is recommended that you delete /usr/bin/miniperl. 20000925: FreeBSD 4.1.1 RELEASE. 20000907: Changes to libkvm requires a recompile of the usual suspects. (find /usr/src -name Makefile | xargs egrep -l LIBKVM) This means that your world must match your kernel around this point. 20000904: A new issue with the sendmail upgrade has come to light. /etc/aliases has moved to /etc/mail/aliases. Mergemaster will incorrectly install the default aliases in /etc/mail rather than move the old one from /etc. So you'll need to manually move the file, create a symbolic link, remove the old /etc/aliases.db and run newaliases. For safety sake, you should stop sendmail while doing this and run the upgrade when locally sourced email is not likely to be generated. 20000827: sendmail has been updated from 8.9.3 to 8.11.0. Some of the more visible changes that may immediately affect your configuration include: - New default file locations from src/contrib/sendmail/cf/README - newaliases limited to root and trusted users - MSA port (587) turned on by default - New queue file naming system so can't go from 8.11 -> 8.9 - FEATURE(`rbl') renamed to FEATURE(`dnsbl') - FEATURE(`nullclient') is more full featured - FEATURE(`nouucp') requires an argument: `reject' or `nospecial' - mail.local FreeBSD-only -b option changed to -B - See src/contrib/sendmail/RELEASE_NOTES for more info 20000803: The "installkernel" target has changed slightly. Now even if you override KERNEL e.g. 'make installkernel KERNEL=MYKERNEL' it will install the MYKERNEL file (built with the buildkernel target) as /kernel rather than /MYKERNEL. Those who have updated their /boot/loader.conf files to point to /MYKERNEL should remove that entry or perform manual rename of /kernel to /MYKERNEL. 20000724: FreeBSD 4.1 RELEASE 20000711: If you use CVSUP or CTM to get CVS trees, AND you used to get the old crypto files from internat.freebsd.org AND you check out files from the CVS tree with the cvs command, please read http://people.freebsd.org/~imp/internat.txt for details on potential problems that you might have and how to get around them. If you are merely a mirror, or don't answer yes to each of the clauses above, you needn't worry. 20000707: We started building modules here. IF you don't wish to follow the documented procedure for building kernels, then you'll have to install new mk files from src/share/mk, or define NO_MODULES. 20000706: Binutils were updated. In order to build a kernel after this date, you must follow the updating procedure for building kernels exactly as presented here. You may be able to get away with doing it the old way, but if it breaks, make sure that you've tried the "To build a kernel" section with a fresh /usr/obj first. 20000622: The license on the softupdates is now a standard 2 clause BSD license. You may need to remove your symbolic links that used to be required when updating. 20000501: At approximately this date, an incompatible kernel change was made which means that if you update kernels accross this boundary, you also have to update modules, and vice versa. 20000315: FreeBSD 4.0 RELEASE. Historical entries follow the common items. ****************************** 4.0 RELEASE ****************************** COMMON ITEMS: To build a kernel ----------------- cd /usr/src # If you have not already done so, please buildworld here # You will also need to update your config file to 4.x. Usually # people tend to start with GENERIC from 4.x and hack from there. make buildkernel KERNCONF= make installkernel KERNCONF= # Verify that the new kernel works, it will be installed as # /kernel To rebuild disk /dev entries ---------------------------- MAKEDEV should be copied from src/etc/MAKEDEV to /dev before starting the following: For N in the list of disks MAKEDEV N # eg ad0 for M in the list of slices MAKEDEV NsMa # eg ad0s1a To rebuild everything --------------------- make world Except when it doesn't work :-) To update from 3.x to 4.x stable -------------------------------- *********************************************************** READ -STABLE ARCHIVES FOR THE LAST WEEK OR TWO BEFORE STARTING THIS PROCESS. UPDATING MAY LAG BEHIND A LITTLE A GOTCHAS ARE OFTEN REPORTED IN FREEBSD-STABLE@FREEBSD.ORG BEFORE BEING INCLUDED HERE. IF YOU ARE GOING TO DO REMOTE MACHINES, MAKE SURE THAT THE EXACT VERSION YOU ARE UPDATING THEM TO WORKS ON LOCAL MACHINES BEFORE PROCEEDING. *********************************************************** [3] cd /usr/src [2] make buildworld -DNOPERL cd /usr/src/sbin/mknod make install [1] reboot [6] cd /usr/src cd gnu/usr.bin/texinfo/install-info make install cd ../../../.. ldconfig -R /usr/obj/usr/src/lib/libc make installworld -DNOPERL [5] mergemaster [4] reboot [1] You may need to switch from wd to ad ala 19991210. In addition, you should disable any third party modules such as vmware so that you don't crash your system on reboot. [2] If you have any of the following in your /etc/make.conf, please comment them out before following these instructions and then do a make buildworld + make installworld after you are back to above: MAKE_KERBEROS4 [3] Update to 4.x-stable sources. Make sure that you get all the parts. If you are using cvsup files that are older than a couple of months, be sure they have all the components listed in /usr/share/examples/cvsup. Crypto and secure are now required. [4] This step is not optional. If you fail to do this, you will not have critical system config files updated for their 4.x functionality. This will lead to reduced system functionality, the biggest problem being with /etc/pam.conf not having the entries necessary to allow remote ssh access. [5] Note: This won't upgrade your perl install. You will need to rebuild the world on 4.x to update perl. [6] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd /usr/src adjkerntz -i # if CMOS is wall time To update from 4.0-RELEASE or later to the most current 4.x-STABLE ---------- make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installkernel KERNCONF=YOUR_KERNEL_HERE reboot (in single user) [1] make installworld mergemaster [2] reboot [1] You can often get away without doing this step as the system will be properly updated. During the running of the installworld, however, system components may break and other oddities may happen. Don't do this on systems that aren't otherwise quiet as unpredictable results may happen. If in doubt, reboot into single user. For remote installs, keep a separate kernel around and use a serial console if at all possible. See also note [6] above. [2] If you do not run mergemaster, you will likely hit a number of show stopper problems. The biggest one is that your /etc/pam.conf won't let you log in using ssh. What follows are older entries for those people upgrading from earlier versions of -stable/-current. 20000308: The binary interface of perl to its modules changed. You will need to recompile any perl moudles after an installworld. This includes all ports matching the pattern /usr/ports/*/p5-*. 20000303: CMSG_XXX macros offset in has changed to conform RFC-2292. All affected applications have been corrected. The i386 platform's offsets haven't changed, but the alpha's did. When you build and install new kernel on FreeBSD/alpha, you must also do a make world. 20000225: OpenSSH has been added to FreeBSD. This may conflict with the ports/security/ssh port since it installs binaries into /usr/bin and the port goes into /usr/local/bin. Most paths have /usr/bin in the path before /usr/local/bin, so problems may arise. If you don't want OpenSSH, add NO_OPENSSH=yes to your make.conf. You will also need to enable OpenSSH in /etc/rc.conf if you want to run the new servers. You may need to move your host key and other config files from /usr/local/etc to /etc/ssh. OpenSSH's command line parsing, available options and default settings aren't the same as ssh, so some care needs to be taken in its operation. One should do a full audit of all configuration settings. 20000205: The xinstall problem has kinda sorta been corrected. The following is known to work by the author of UPDATING. It is what he did to update his laptop. It is likely the same effect as the work around listed in 20000129. make buildworld make installworld This will fail cd usr.bin/xinstall make clean all install NOSHARED=YES cd ../.. make installworld If it doesn't work for you, then I'm sure that there will be another 100 messages in -current. 20000204: libipsec version number changed from 2 to 0. The original commit of Jan 6 incorrectly set this to 2. Remove /usr/lib/libipsec.so.2* before the buildworld and rebuild anything that uses libipsec.so after you install the new version. 20000201: The rcmd related functions have been aligned with other BSD implementations. rlogind and rshd have been changed to use the new API. A make world is recommended to keep them in sync with the libraries they use. http://www.FreeBSD.org/cgi/cvsweb.cgi/src/lib/libc/net/rcmd.c has the details. 20000129: {set,get}flags have been added to the tree for rather dubious reasons. IF AND ONLY IF installworld fails: make -k -DNOFSCHG installworld make installworld This issue was resolved Feb 5, 2000. 20000125: rcmd_af() is added for multiple address family support. rsh and rlogin have been updated to use this to support INET6. A make world is recommended to keep them in sync with the libraries they use. 20000124: The default way that virtual tables in our default C++ compiler has changed. We used to use THUNKS for virtual inheritance. Unfortunately there are bugs that The GCC developers thought would be fixed in GCC 2.95. However it isn't. After this change existing applications written in C++ may give errors like below when you try to run them: /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol "__vt_7filebuf" The only fix is to rebuild the application and any C++ libraries used. 20000124: The management of next_writeable has been move from the kernel to userspace. This means that the burncd command must be in sync with the kernel. So make world is your friend. 20000117: GNU texinfo upgraded to 4.0. It is recommended that you use make buildworld to update. 20000117: CAM_VERSION has been increased. Recompile all programs that depend on it (tosha, camcontrol, etc) when moving to kernels past this date. 20000115: pccard ioctls changed with a commit to have the kernel tell pccardd which device it created, rather than having pccardd dictate it to the kernel. Recompile kernel and pccardd/pccardc together. pccardd/pccardc rely on a kernel Makefile that must be installed prior to building them. Make world will automatically take care of this issue, but you need to know if you are building by hand. 20000109: Your kernel config files need to be changed to s/controller/device/ in them. You will need a new version of config to do this. 20000107: chown & chgrp moved again This is a heads up to let you know that you need to rm -f /sbin/chown /bin/chgrp after your next `make world'. Additionally you need to install a new /dev/MAKEDEV (mergemaster(8) will assist you in this). 20000107: SHA-1 password support gone As warned yesterday, I've just removed the ability for libcrypt to understand SHA-1 passwords. This was present but undocumented in the tree for the past few months and was removed so it can be reimplemented properly as part of a revamped libcrypt at a later date. Anyone who was actually making use of the feature will have to update their passwords back to MD5 before they reinstall the library. 20000106: ioctl numbers changed for dvdio ioctls. You will need to recompile anything that uses them. wormcontrol has been deprecated in favor of burncd. 19991223: Building linux and svr4 compatibility into the kernel now requires /usr/bin/genassym. Go to /usr/src/usr.bin/genassym, do "make obj ; make clean depend all install" before building your kernel. 19991218: sendmail.cf has moved from /etc/sendmail.cf to /etc/mail/sendmail.cf. You may need to adjust /etc/rc.conf and /etc/rc to cope as well as moving sendmail.cf. 19991216: ntp 4.0.98 has replaced the ancient xntpd. The daemon name changed from xntpd to ntpd, so you may need to update your /etc/rc.conf file. The ntp.conf files are compatible with the old release, unless you are using a local reference clock. Details about ntp4 can be found at http://www.ntp.org/. xntpd will be cvs removed from the repo in about a week. 19991213: Soren updated the ata driver. Please update to at least this version before submitting bug reports. 19991210: The ata driver has become the primary ata/ide/atapi driver in the kernel. The wd driver is obsolete. You should upgrade your machine to the new ata driver. You will need to follow the directions for updating the devices in the 19991205 entry. You will need to update all occurances of wd to ad in your config files (/etc/rc.conf, /etc/fstab, etc). A compatibility device exists for wd for the time being, but once you have determined that ata is working for you, transition to the new ad devices. 19991205: Block devices are going away. You will need to update your /dev tree with a fresh copy of MAKEDEV for things to always work in the future. fsck is unable to cope with dirty file systems after this change, so it may work for you or it may not. See "To rebuild disk /dev entries" at the end of the file. *** All uses of block devices must be converted to char devices. *** Likely the only thing impacted is dumpdev in /etc/rc.conf 19991204: The dc interface has replaced al, ax, dm, pn and mx. The former have been removed. 19991204: Support for the old 'sd' device names has been removed in favor of 'da'. Please update your /etc/fstab, /etc/rc.conf and any other places you might have sd names cached. Generally these changes are as simple as s=/dev/sd=/dev/da=g, but be careful to make sure that things are really that simple. You may also need to create /dev entries for the da devices as well. See "To rebuild disk /dev entries" at the end of the file. 19991203: BAD144 support has been removed. Cope or replace the hardware. 19991129: ALPHA users take note. All klds will need to be recompiled for kernels build after this date. It is a good time to update since pal.s will be going away soon. 19991126: MFS_ROOT and MFS_ROOT_SIZE are gone, replaced by MD_ROOT and MD_ROOT_SIZE options in the md driver. md driver now deals with the hacks. You should add md driver to kerneles that have MFS_ROOT, et al. See GENERIC or LINT for details. 19991125: The ep changed a while ago and can no longer be used with hard wired addresses in the config file. 19991122: The bridge drivers for sound cards have been committed. Please see LINT for instructions for your card, if it still works at all. 19991113: Gcc 2.95.2 is now the default compiler. 19991030: /etc/radius.conf file format has changed.The format change is as follows. Each server line in the file should now begin with a new field containing either "auth" for RADIUS authentication, or "acct" for RADIUS accounting. Formerly only authentication was supported. If the first field isn't "auth" or "acct" then the code assumes that "auth" is intended. (That's the compatibility hack.) 19991015: PCCARD has been updated to attach pcic to the isa bus. Therefore you will need to modify kernel config files that have pcic/card in them as follows: controller pcic0 at isa? controller pcic1 at isa? controller card0 19990929: The sigset_t datatype has been changed from an integral type to a compound type and can hold 128 signals. Syscalls directly or indirectly using the new sigset_t have been added as to maintain compatibility with existing binaries. A new kernel must be made and installed and booted with before a make world can be done. *************************************************************** RECOMPILE AND REINSTALL KERNEL BEFORE MAKEWORLD *************************************************************** 19990919: New jail syscall format requires recompilation of jail(8) with fresh headers installed (or a make world will do it for you). 19990914: Matt Dillon checked in many vm related things and sent a heads up to -current urging caution and to report vm problems to him. As of the 19th, no killer problems have been reported, but you have been warned. 19990908: The new miibus has been added to the system. If you are using the dm, rl, sf, sis, ste, tl, wb or xl drivers, you need to add "controller miibus0" to your config file. 19990905: /var/cron/log has been moved to /var/log/cron to get all the log files in one place. 19990831: tn3270 has been removed from the base system and added as a port. 19990830: User-visible TCP timers are now expressed in units of 1ms, instead of 500ms, so if you've customized any timer values under ``net.inet.tcp'', multiply them by 500 to preserve TCP's behavior. 19990828: RCS Id tags changed to FreeBSD. This will cause huge cvsup updates. 19990821: On 28-May-1999 libreadline was upgraded from readline-2.2 to readline-4.0. At that time the shared library major version number was bumped from "3" to "4". It has been deemed that the interface change between readline-2.2 and readline-4.0 was not sufficient to warrant the version number bump. Thus I have reverted it back to "3". You will need to perform the below immediately before your next ``make world'': cd /usr/lib ls -l libreadline.so.4 (if you have /usr/lib/libreadline.so.4) mv libreadline.so.4 libreadline.so.3 rm -f libreadline.so ln -s libreadline.so.3 libreadline.so 19990801: Changes to the pccardd kernel interface require that you recompile pccardd for new kernel. 19980725: The ipfw interface to the kernel has changed. You will need to recompile ipfw programs for the new kernel. 19990715: The bpfilter device has been renamed to bpf. You will need to change your config files in order to enable this in newer kernels. 19990704: src/contrib/sys/softupdates is moving to src/sys/contrib/softupdates. Update your symbolic links/etc. 19990702: Major changes have been made to vinum and its interface. See the man page (vinum(8)) for details. Look at the concat, mirror and stripe commands, as well as the SIMPLIFIED CONFIGURATION section. 19990628: Newsyslog.conf has had a minor, but potentially dangerous, change to its username/group syntax. The old syntax was user.group, while the new syntax is user:group. 19990627: Inetd wrapping default has changed. Please see the updated man page for details. 19990623: Compaq Smart Raid driver committed as ida. 19990622: The second phase of syscons cleanup has happened. Some functionality has been made optional. For details, see http://www.freebsd.org/~yokota/sc_update-June.txt Everyone will need to re-config(8) their kernels, but old binaries will work with the new kernel. 19990620: IPFW uid/gid-based filtering support has been committed. This breaks binary compatibility with previous copies of ipfw(8). Any utilities using the ioctl()s of ipfw (especially ipfw(8)) need to be recompiled with the newest headers installed. 19990618: Inetd now wraps all stream-based services, including internals. Syslog "severity" options are honoured. Installed syslog.conf and hosts.allow should be checked. 19990509: Most of the problems with newbus have been corrected, but it is still current and evolving. libcam's ABI has changed. You must recompile the world and any ports that use it. 19990427: Massive changes to SMP went into the tree that should speed things up. However, if you experience problems with SMP machines, you can back off to the PRE_SMP_VMSHARE tag in the CVS repository (or run an MP kernel). 19990420: Pccardd and the pccard bus in general seem to be broken or most people. Soren has patches to make it better for some, but not all people. Work is underway to make it better for all people. 19990416: new bus changes integrated into -current. Many problems were initially reported, but most have been fixed. You'll need a new config and to tweak your kernel config file the way that GENERIC was tweaked. Keep a copy of your old kernel when upgrading in case your new kernel doesn't work (and report the breakage to current@freebsd.org). As of 19990421 the remaining problems/quirks are o sio no longer supports pnp nor pccard. This will be corrected, but the new pccard code needs to be completed. o Some of the sound drivers broke for some people. It seems inconsistent as to who/what/when/where things broke. o Duplicate device entries in your config do not work. Previously they were silently ignored. psm0 seems to be the most common duplicate choice, although some reports have come in from people that wired their scsi disk entries. o You must move the keyboard and mouse attachments from the isa bus to the atkbdc bus. See GENERIC for details. o Machines with multiple host-pci bridges (Intel 450NX) will not probe the second pci bus. o Some probe ordering may have changed, so device naming may change with the new kernel. If you have multiple devices, please double check to make sure they didn't move. This is especially true for cards on different buses. o The bus attachment for vga0 is always isa0 and never pci0 when it is in fact a pci card. It is safe to ignore this. o The format of the boot messages has changed (and is not guaranteed to be constant for a while) so you may see this in some shell scripts that diff dmesg from day to day as well as scripts that parse dmesg output. 19990414: cc -aout sometimes has problems compiling with -g, remove -g from those compilation units affected until this is corrected. cc -aout has problems with producing some threaded libraries, so make world -DWANT_AOUT is not functional at this time. Since make world for a.out isn't functional, you cannot do make aout-to-elf from a pre-egcs a.out system. To work around this problem, make aout-to-elf using 3.1-stable first (or a pre egcs -current) and then a simple make world will take you the rest of the way with a -current source base. 19990413: Make -j works again for make world. As always, use it with care, and be sure to try a make world w/o -j before reporting problems (that is, know if the problem exists w/o -j before reporting the problem with -j). 19990413: Note cc -aout, which had been broken since the conversion to egcs, now supposedly works. If you have rebuilt things like XFree86 a.out libraries, you should rebuild them again or progams that use them, including netscape, will fail to work. 19990409: NOTE: New c++ compiler cannot be used with c++ binaries from the old compiler and vice versa. 19990408: In recent days egcs has been imported into the tree. A number of problems have been discussed in -current. Here is a highlight of a few of the more common ones. With the exception of make -j n, they have been claimed to have been fixed. c++ is broken after make world: This is correct. Because we changed compilers two make worlds are necessary to get C++ working again. The code generated by the old compiler and new compiler are not compatible, so things wind up being undefined. cpp is broken in make world: There was a window where cpp would be built incorrectly. The fix for this is cd src/gnu/usr.bin/cc make clean make all make install and this should correct your problems. See the -current archives for a version of this that does moer cleaning. Make -j n doesn't work. Work contiunes to make this work, I've seen no reports of success as of April 7th. 19990329: Previously, one would define "NOAOUT" to keep from building the legacy a.out bits. Now one would define "WANT_AOUT" to build them. 19990316: The name of the old wd.c and atapi.c based CDROM driver has been changed back to wcd. So update your config file to use "device wcd" instead of "device acd". 19990314: For those of you using the VN device as a loadable module, please be aware that the new VN device has been committed to -4.x and cannot yet be used as a loadable module. This will be fixed in the future. 19990309: New loader.rc mechanism. Please see src/sys/boot/README for details. 19990225: struct proc size changed. Recompile kernel and the usual suspects. Make world if in doubt. 19990214: The nlpt driver has changed names back to be the lpt driver. See ppbus(4) or http://www.freebsd.org/~nsouch/ppbus.html for proper configuration details. 19990210: The 'lpt' driver is now obsolete. Replace it with the 'nlpt' driver and 'ppbus' controller combination. 19990209: New devstat API requires recompilation of libdevstat, systat, iostat, vmstat and rpc.rstatd. A new kernel is also required. make world + building a kernel should do all of this. Any ports that use devstat need to be recompiled as well. 19990125: Linux threads options has gone away (they are now standard in the FreeBSD kernel). A recompile of all libkvm using programs is in order (or better yet a make world). 19990122: On or about this date there was a small window when the boot blocks had some minor problems which seemed to force one to edit /etc/fstab. This has been corrected, if you are seeing this problem, please rebuild and reinstall your boot blocks. 19990121: Vinum has changed. The "vinum read" command has changed. For updates, please see http://www.freebsd.org/cgi/getmsg.cgi?fetch=800363+0+current/freebsd-current for details. Matt Dillon committed a boatload of VM changes, for information please see http://www.freebsd.org/cgi/getmsg.cgi?fetch=886676+0+current/freebsd-current or http://www.freebsd.org/cgi/getmsg.cgi?fetch=827400+0+current/freebsd-current for details. These changes will likely not impact anybody, but large chagnes to the VM need at least a heads up. 19990120: Stable branch created. You might want to consider using this branch. It is tagged with RELENG_3. 19990119: More work on the syscons driver has been committed. Since files moved around, you will need to reconfigure your kernel, and make clean; make depend before rebuilding the kernel. No config file changes are needed. 19990111: New keyboard and video card drivers are introduced as the first stage of console driver reorganization. You are required to update the kernel configuration file and rebuild the kernel. kbdcontrol, vidcontrol, and screen savers also need recompilation. The instruction can be found in http://www.freebsd.org/~yokota/sc_update.txt It has notes on splash screen too. 19990106: Robert Nordier was kind enough to make this page to enable people to install the new bootblocks: http://www.freebsd.org/~rnordier/boot.txt When updating to ELF, make sure that you have updated your /etc/rc files which have different ldconfig invocations for elf and aout. Otherwise things like X will stop working with messages about being unable to find library files. The mergemaster utility in ports/sysutils/mergemaster helps to keep critical files like this in sync, and its use is recommended. 19990104: Information about ELF day (see 19981230) can be found in http://www.freebsd.org/~peter/elfday.html about the upcoming change, its motivations and instructions for doing the upgrade. 19981230: You should install new bootblocks now. The ELF kernel will become default soon. In addition, you'll need to upgrade your userland programs to ELF before then as well. Use make aout-to-elf to upgrade your userland programs (old a.out programs in, for example, /usr/local/bin will continue to work, even after the upgrade). See src/Makefile for more details. 19981224: The old wcd driver has been deleted from the kernel. The driver that replaces it is named acd. You'll need to change this in your configuration files. Failure to do this will result in "ATAPI CD-ROMs not configured" at boot time. The floppy tape driver (ft) has been removed from the kernel, with no replacement driver. 19981202: New groups from 19981201 commented out of mtree, so they aren't strictly needed, but still a good idea to have. 19981201: New user/group added: bind. From src/etc/master.passwd and src/etc/group: % grep /usr/src/etc/group bind:*:53: % grep /usr/src/etc/master.passwd tty:*:4:65533::0:0:Tty Sandbox:/:/sbin/nologin kmem:*:5:65533::0:0:KMem Sandbox:/:/sbin/nologin bind:*:53:53::0:0:Bind Sandbox:/:/sbin/nologin 19981118: PAM integrated into the tree. Requires new /etc/pam.conf file to silence warnings from authentication programs (login, etc). See src/etc/pam.conf for a sample. FORMAT: This file contains a list, in reverse chronologocal order, of major breakages in tracking -stable. Not all things will be listed here, and it only starts on November 18, 1998. If you have an earlier version of FreeBSD, you are on your own to get to November 18, 1998. Please filter your entries through Warner Losh (imp@village.org) so that the style, formatting, etc of this file can be maintained. Copyright information: Copyright 1998-2002 M. Warner Losh. All Rights Reserved. Redistribution, translation and use, with or without modification, in full or in part, are permitted provided that the above copyright notice is retained. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. If you find this document useful, and you want to, you may buy the author a beer. $FreeBSD$ Index: releng/4.11/gnu/usr.bin/gzip/gzip.h =================================================================== --- releng/4.11/gnu/usr.bin/gzip/gzip.h (revision 162446) +++ releng/4.11/gnu/usr.bin/gzip/gzip.h (revision 162447) @@ -1,315 +1,319 @@ /* gzip.h -- common declarations for all gzip modules * Copyright (C) 1992-1993 Jean-loup Gailly. * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. + * + * $FreeBSD$ */ #if defined(__STDC__) || defined(PROTO) # define OF(args) args #else # define OF(args) () #endif #ifdef __STDC__ typedef void *voidp; #else typedef char *voidp; #endif /* I don't like nested includes, but the string and io functions are used * too often */ #include #if !defined(NO_STRING_H) || defined(STDC_HEADERS) # include # if !defined(STDC_HEADERS) && !defined(NO_MEMORY_H) && !defined(__GNUC__) # include # endif # define memzero(s, n) memset ((voidp)(s), 0, (n)) #else # include # define strchr index # define strrchr rindex # define memcpy(d, s, n) bcopy((s), (d), (n)) # define memcmp(s1, s2, n) bcmp((s1), (s2), (n)) # define memzero(s, n) bzero((s), (n)) #endif #ifndef RETSIGTYPE # define RETSIGTYPE void #endif #define local static typedef unsigned char uch; typedef unsigned short ush; typedef unsigned long ulg; /* Return codes from gzip */ #define OK 0 #define ERROR 1 #define WARNING 2 /* Compression methods (see algorithm.doc) */ #define STORED 0 #define COMPRESSED 1 #define PACKED 2 #define LZHED 3 /* methods 4 to 7 reserved */ #define DEFLATED 8 #define MAX_METHODS 9 extern int method; /* compression method */ /* To save memory for 16 bit systems, some arrays are overlaid between * the various modules: * deflate: prev+head window d_buf l_buf outbuf * unlzw: tab_prefix tab_suffix stack inbuf outbuf * inflate: window inbuf * unpack: window inbuf prefix_len * unlzh: left+right window c_table inbuf c_len * For compression, input is done in window[]. For decompression, output * is done in window except for unlzw. */ #ifndef INBUFSIZ # ifdef SMALL_MEM # define INBUFSIZ 0x2000 /* input buffer size */ # else # define INBUFSIZ 0x8000 /* input buffer size */ # endif #endif #define INBUF_EXTRA 64 /* required by unlzw() */ #ifndef OUTBUFSIZ # ifdef SMALL_MEM # define OUTBUFSIZ 8192 /* output buffer size */ # else # define OUTBUFSIZ 16384 /* output buffer size */ # endif #endif #define OUTBUF_EXTRA 2048 /* required by unlzw() */ #ifndef DIST_BUFSIZE # ifdef SMALL_MEM # define DIST_BUFSIZE 0x2000 /* buffer for distances, see trees.c */ # else # define DIST_BUFSIZE 0x8000 /* buffer for distances, see trees.c */ # endif #endif #ifdef DYN_ALLOC # define EXTERN(type, array) extern type * near array # define DECLARE(type, array, size) type * near array # define ALLOC(type, array, size) { \ array = (type*)fcalloc((size_t)(((size)+1L)/2), 2*sizeof(type)); \ if (array == NULL) error("insufficient memory"); \ } # define FREE(array) {if (array != NULL) fcfree(array), array=NULL;} #else # define EXTERN(type, array) extern type array[] # define DECLARE(type, array, size) type array[size] # define ALLOC(type, array, size) # define FREE(array) #endif EXTERN(uch, inbuf); /* input buffer */ EXTERN(uch, outbuf); /* output buffer */ EXTERN(ush, d_buf); /* buffer for distances, see trees.c */ EXTERN(uch, window); /* Sliding window and suffix table (unlzw) */ #define tab_suffix window #ifndef MAXSEG_64K # define tab_prefix prev /* hash link (see deflate.c) */ # define head (prev+WSIZE) /* hash head (see deflate.c) */ EXTERN(ush, tab_prefix); /* prefix code (see unlzw.c) */ #else # define tab_prefix0 prev # define head tab_prefix1 EXTERN(ush, tab_prefix0); /* prefix for even codes */ EXTERN(ush, tab_prefix1); /* prefix for odd codes */ #endif extern unsigned insize; /* valid bytes in inbuf */ extern unsigned inptr; /* index of next byte to be processed in inbuf */ extern unsigned outcnt; /* bytes in output buffer */ extern long bytes_in; /* number of input bytes */ extern long bytes_out; /* number of output bytes */ extern long header_bytes;/* number of bytes in gzip header */ #define isize bytes_in /* for compatibility with old zip sources (to be cleaned) */ extern int ifd; /* input file descriptor */ extern int ofd; /* output file descriptor */ extern char ifname[]; /* input file name or "stdin" */ extern char ofname[]; /* output file name or "stdout" */ extern char *progname; /* program name */ extern long time_stamp; /* original time stamp (modification time) */ extern long ifile_size; /* input file size, -1 for devices (debug only) */ typedef int file_t; /* Do not use stdio */ #define NO_FILE (-1) /* in memory compression */ #define PACK_MAGIC "\037\036" /* Magic header for packed files */ #define GZIP_MAGIC "\037\213" /* Magic header for gzip files, 1F 8B */ #define OLD_GZIP_MAGIC "\037\236" /* Magic header for gzip 0.5 = freeze 1.x */ #define LZH_MAGIC "\037\240" /* Magic header for SCO LZH Compress files*/ #define PKZIP_MAGIC "\120\113\003\004" /* Magic header for pkzip files */ /* gzip flag byte */ #define ASCII_FLAG 0x01 /* bit 0 set: file probably ascii text */ #define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */ #define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */ #define ORIG_NAME 0x08 /* bit 3 set: original file name present */ #define COMMENT 0x10 /* bit 4 set: file comment present */ #define ENCRYPTED 0x20 /* bit 5 set: file is encrypted */ #define RESERVED 0xC0 /* bit 6,7: reserved */ /* internal file attribute */ #define UNKNOWN 0xffff #define BINARY 0 #define ASCII 1 #ifndef WSIZE # define WSIZE 0x8000 /* window size--must be a power of two, and */ #endif /* at least 32K for zip's deflate method */ #define MIN_MATCH 3 #define MAX_MATCH 258 /* The minimum and maximum match lengths */ #define MIN_LOOKAHEAD (MAX_MATCH+MIN_MATCH+1) /* Minimum amount of lookahead, except at the end of the input file. * See deflate.c for comments about the MIN_MATCH+1. */ #define MAX_DIST (WSIZE-MIN_LOOKAHEAD) /* In order to simplify the code, particularly on 16 bit machines, match * distances are limited to MAX_DIST instead of WSIZE. */ extern int decrypt; /* flag to turn on decryption */ extern int exit_code; /* program exit code */ extern int verbose; /* be verbose (-v) */ extern int quiet; /* be quiet (-q) */ extern int level; /* compression level */ extern int test; /* check .z file integrity */ extern int to_stdout; /* output to stdout (-c) */ extern int save_orig_name; /* set if original name must be saved */ + +#define MIN(a,b) ((a) <= (b) ? (a) : (b)) #define get_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(0)) #define try_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(1)) /* put_byte is used for the compressed output, put_ubyte for the * uncompressed output. However unlzw() uses window for its * suffix table instead of its output buffer, so it does not use put_ubyte * (to be cleaned up). */ #define put_byte(c) {outbuf[outcnt++]=(uch)(c); if (outcnt==OUTBUFSIZ)\ flush_outbuf();} #define put_ubyte(c) {window[outcnt++]=(uch)(c); if (outcnt==WSIZE)\ flush_window();} /* Output a 16 bit value, lsb first */ #define put_short(w) \ { if (outcnt < OUTBUFSIZ-2) { \ outbuf[outcnt++] = (uch) ((w) & 0xff); \ outbuf[outcnt++] = (uch) ((ush)(w) >> 8); \ } else { \ put_byte((uch)((w) & 0xff)); \ put_byte((uch)((ush)(w) >> 8)); \ } \ } /* Output a 32 bit value to the bit stream, lsb first */ #define put_long(n) { \ put_short((n) & 0xffff); \ put_short(((ulg)(n)) >> 16); \ } #define seekable() 0 /* force sequential output */ #define translate_eol 0 /* no option -a yet */ #define tolow(c) (isupper(c) ? (c)-'A'+'a' : (c)) /* force to lower case */ /* Macros for getting two-byte and four-byte header values */ #define SH(p) ((ush)(uch)((p)[0]) | ((ush)(uch)((p)[1]) << 8)) #define LG(p) ((ulg)(SH(p)) | ((ulg)(SH((p)+2)) << 16)) /* Diagnostic functions */ #ifdef DEBUG # define Assert(cond,msg) {if(!(cond)) error(msg);} # define Trace(x) fprintf x # define Tracev(x) {if (verbose) fprintf x ;} # define Tracevv(x) {if (verbose>1) fprintf x ;} # define Tracec(c,x) {if (verbose && (c)) fprintf x ;} # define Tracecv(c,x) {if (verbose>1 && (c)) fprintf x ;} #else # define Assert(cond,msg) # define Trace(x) # define Tracev(x) # define Tracevv(x) # define Tracec(c,x) # define Tracecv(c,x) #endif #define WARN(msg) {if (!quiet) fprintf msg ; \ if (exit_code == OK) exit_code = WARNING;} /* in zip.c: */ extern int zip OF((int in, int out)); extern int file_read OF((char *buf, unsigned size)); /* in unzip.c */ extern int unzip OF((int in, int out)); extern int check_zipfile OF((int in)); /* in unpack.c */ extern int unpack OF((int in, int out)); /* in unlzh.c */ extern int unlzh OF((int in, int out)); /* in gzip.c */ RETSIGTYPE abort_gzip OF((void)); /* in deflate.c */ void lm_init OF((int pack_level, ush *flags)); ulg deflate OF((void)); /* in trees.c */ void ct_init OF((ush *attr, int *method)); int ct_tally OF((int dist, int lc)); ulg flush_block OF((char *buf, ulg stored_len, int eof)); /* in bits.c */ void bi_init OF((file_t zipfile)); void send_bits OF((int value, int length)); unsigned bi_reverse OF((unsigned value, int length)); void bi_windup OF((void)); void copy_block OF((char *buf, unsigned len, int header)); extern int (*read_buf) OF((char *buf, unsigned size)); /* in util.c: */ extern int copy OF((int in, int out)); extern ulg updcrc OF((uch *s, unsigned n)); extern void clear_bufs OF((void)); extern int fill_inbuf OF((int eof_ok)); extern void flush_outbuf OF((void)); extern void flush_window OF((void)); extern void write_buf OF((int fd, voidp buf, unsigned cnt)); extern char *strlwr OF((char *s)); extern char *basename OF((char *fname)); extern void make_simple_name OF((char *name)); extern char *add_envopt OF((int *argcp, char ***argvp, char *env)); extern void error OF((char *m)); extern void warn OF((char *a, char *b)); extern void read_error OF((void)); extern void write_error OF((void)); extern void display_ratio OF((long num, long den, FILE *file)); extern voidp xmalloc OF((unsigned int size)); /* in inflate.c */ extern int inflate OF((void)); Index: releng/4.11/gnu/usr.bin/gzip/inflate.c =================================================================== --- releng/4.11/gnu/usr.bin/gzip/inflate.c (revision 162446) +++ releng/4.11/gnu/usr.bin/gzip/inflate.c (revision 162447) @@ -1,958 +1,958 @@ /* inflate.c -- Not copyrighted 1992 by Mark Adler version c10p1, 10 January 1993 */ /* You can do whatever you like with this source file, though I would prefer that if you modify it and redistribute it that you include comments to that effect with your name and the date. Thank you. [The history has been moved to the file ChangeLog.] */ /* Inflate deflated (PKZIP's method 8 compressed) data. The compression method searches for as much of the current string of bytes (up to a length of 258) in the previous 32K bytes. If it doesn't find any matches (of at least length 3), it codes the next byte. Otherwise, it codes the length of the matched string and its distance backwards from the current position. There is a single Huffman code that codes both single bytes (called "literals") and match lengths. A second Huffman code codes the distance information, which follows a length code. Each length or distance code actually represents a base value and a number of "extra" (sometimes zero) bits to get to add to the base value. At the end of each deflated block is a special end-of-block (EOB) literal/ length code. The decoding process is basically: get a literal/length code; if EOB then done; if a literal, emit the decoded byte; if a length then get the distance and emit the referred-to bytes from the sliding window of previously emitted data. There are (currently) three kinds of inflate blocks: stored, fixed, and dynamic. The compressor deals with some chunk of data at a time, and decides which method to use on a chunk-by-chunk basis. A chunk might typically be 32K or 64K. If the chunk is uncompressible, then the "stored" method is used. In this case, the bytes are simply stored as is, eight bits per byte, with none of the above coding. The bytes are preceded by a count, since there is no longer an EOB code. If the data is compressible, then either the fixed or dynamic methods are used. In the dynamic method, the compressed data is preceded by an encoding of the literal/length and distance Huffman codes that are to be used to decode this block. The representation is itself Huffman coded, and so is preceded by a description of that code. These code descriptions take up a little space, and so for small blocks, there is a predefined set of codes, called the fixed codes. The fixed method is used if the block codes up smaller that way (usually for quite small chunks), otherwise the dynamic method is used. In the latter case, the codes are customized to the probabilities in the current block, and so can code it much better than the pre-determined fixed codes. The Huffman codes themselves are decoded using a mutli-level table lookup, in order to maximize the speed of decoding plus the speed of building the decoding tables. See the comments below that precede the lbits and dbits tuning parameters. */ /* Notes beyond the 1.93a appnote.txt: 1. Distance pointers never point before the beginning of the output stream. 2. Distance pointers can point back across blocks, up to 32k away. 3. There is an implied maximum of 7 bits for the bit length table and 15 bits for the actual data. 4. If only one code exists, then it is encoded using one bit. (Zero would be more efficient, but perhaps a little confusing.) If two codes exist, they are coded using one bit each (0 and 1). 5. There is no way of sending zero distance codes--a dummy must be sent if there are none. (History: a pre 2.0 version of PKZIP would store blocks with no distance codes, but this was discovered to be too harsh a criterion.) Valid only for 1.93a. 2.04c does allow zero distance codes, which is sent as one code of zero bits in length. 6. There are up to 286 literal/length codes. Code 256 represents the end-of-block. Note however that the static length tree defines 288 codes just to fill out the Huffman codes. Codes 286 and 287 cannot be used though, since there is no length base or extra bits defined for them. Similarly, there are up to 30 distance codes. However, static trees define 32 codes (all 5 bits) to fill out the Huffman codes, but the last two had better not show up in the data. 7. Unzip can check dynamic Huffman blocks for complete code sets. The exception is that a single code would not be complete (see #4). 8. The five bits following the block type is really the number of literal codes sent minus 257. 9. Length codes 8,16,16 are interpreted as 13 length codes of 8 bits (1+6+6). Therefore, to output three times the length, you output three codes (1+1+1), whereas to output four times the same length, you only need two codes (1+3). Hmm. 10. In the tree reconstruction algorithm, Code = Code + Increment only if BitLength(i) is not zero. (Pretty obvious.) 11. Correction: 4 Bits: # of Bit Length codes - 4 (4 - 19) 12. Note: length code 284 can represent 227-258, but length code 285 really is 258. The last length deserves its own, short code since it gets used a lot in very redundant files. The length 258 is special since 258 - 3 (the min match length) is 255. 13. The literal/length and distance code bit lengths are read as a single stream of lengths. It is possible (and advantageous) for a repeat code (16, 17, or 18) to go across the boundary between the two sets of lengths. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #if defined(STDC_HEADERS) || !defined(NO_STDLIB_H) # include #endif #include "gzip.h" #define slide window /* Huffman code lookup table entry--this entry is four bytes for machines that have 16-bit pointers (e.g. PC's in the small or medium model). Valid extra bits are 0..13. e == 15 is EOB (end of block), e == 16 means that v is a literal, 16 < e < 32 means that v is a pointer to the next table, which codes e - 16 bits, and lastly e == 99 indicates an unused code. If a code with e == 99 is looked up, this implies an error in the data. */ struct huft { uch e; /* number of extra bits or operation */ uch b; /* number of bits in this code or subcode */ union { ush n; /* literal, length base, or distance base */ struct huft *t; /* pointer to next level of table */ } v; }; /* Function prototypes */ int huft_build OF((unsigned *, unsigned, unsigned, ush *, ush *, struct huft **, int *)); int huft_free OF((struct huft *)); int inflate_codes OF((struct huft *, struct huft *, int, int)); int inflate_stored OF((void)); int inflate_fixed OF((void)); int inflate_dynamic OF((void)); int inflate_block OF((int *)); int inflate OF((void)); /* The inflate algorithm uses a sliding 32K byte window on the uncompressed stream to find repeated byte strings. This is implemented here as a circular buffer. The index is updated simply by incrementing and then and'ing with 0x7fff (32K-1). */ /* It is left to other modules to supply the 32K area. It is assumed to be usable as if it were declared "uch slide[32768];" or as just "uch *slide;" and then malloc'ed in the latter case. The definition must be in unzip.h, included above. */ /* unsigned wp; current position in slide */ #define wp outcnt #define flush_output(w) (wp=(w),flush_window()) /* Tables for deflate from PKZIP's appnote.txt. */ static unsigned border[] = { /* Order of the bit length code lengths */ 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; static ush cplens[] = { /* Copy lengths for literal codes 257..285 */ 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0}; /* note: see note #13 above about the 258 in this list. */ static ush cplext[] = { /* Extra bits for literal codes 257..285 */ 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0, 99, 99}; /* 99==invalid */ static ush cpdist[] = { /* Copy offsets for distance codes 0..29 */ 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577}; static ush cpdext[] = { /* Extra bits for distance codes */ 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13}; /* Macros for inflate() bit peeking and grabbing. The usage is: NEEDBITS(j) x = b & mask_bits[j]; DUMPBITS(j) where NEEDBITS makes sure that b has at least j bits in it, and DUMPBITS removes the bits from b. The macros use the variable k for the number of bits in b. Normally, b and k are register variables for speed, and are initialized at the beginning of a routine that uses these macros from a global bit buffer and count. If we assume that EOB will be the longest code, then we will never ask for bits with NEEDBITS that are beyond the end of the stream. So, NEEDBITS should not read any more bytes than are needed to meet the request. Then no bytes need to be "returned" to the buffer at the end of the last block. However, this assumption is not true for fixed blocks--the EOB code is 7 bits, but the other literal/length codes can be 8 or 9 bits. (The EOB code is shorter than other codes because fixed blocks are generally short. So, while a block always has an EOB, many other literal/length codes have a significantly lower probability of showing up at all.) However, by making the first table have a lookup of seven bits, the EOB code will be found in that first lookup, and so will not require that too many bits be pulled from the stream. */ ulg bb; /* bit buffer */ unsigned bk; /* bits in bit buffer */ ush mask_bits[] = { 0x0000, 0x0001, 0x0003, 0x0007, 0x000f, 0x001f, 0x003f, 0x007f, 0x00ff, 0x01ff, 0x03ff, 0x07ff, 0x0fff, 0x1fff, 0x3fff, 0x7fff, 0xffff }; #ifdef CRYPT uch cc; # define NEXTBYTE() \ (decrypt ? (cc = get_byte(), zdecode(cc), cc) : get_byte()) #else # define NEXTBYTE() (uch)get_byte() #endif #define NEEDBITS(n) {while(k<(n)){b|=((ulg)NEXTBYTE())<>=(n);k-=(n);} /* Huffman code decoding is performed using a multi-level table lookup. The fastest way to decode is to simply build a lookup table whose size is determined by the longest code. However, the time it takes to build this table can also be a factor if the data being decoded is not very long. The most common codes are necessarily the shortest codes, so those codes dominate the decoding time, and hence the speed. The idea is you can have a shorter table that decodes the shorter, more probable codes, and then point to subsidiary tables for the longer codes. The time it costs to decode the longer codes is then traded against the time it takes to make longer tables. This results of this trade are in the variables lbits and dbits below. lbits is the number of bits the first level table for literal/ length codes can decode in one step, and dbits is the same thing for the distance codes. Subsequent tables are also less than or equal to those sizes. These values may be adjusted either when all of the codes are shorter than that, in which case the longest code length in bits is used, or when the shortest code is *longer* than the requested table size, in which case the length of the shortest code in bits is used. There are two different values for the two tables, since they code a different number of possibilities each. The literal/length table codes 286 possible values, or in a flat code, a little over eight bits. The distance table codes 30 possible values, or a little less than five bits, flat. The optimum values for speed end up being about one bit more than those, so lbits is 8+1 and dbits is 5+1. The optimum values may differ though from machine to machine, and possibly even between compilers. Your mileage may vary. */ int lbits = 9; /* bits in base literal/length lookup table */ int dbits = 6; /* bits in base distance lookup table */ /* If BMAX needs to be larger than 16, then h and x[] should be ulg. */ #define BMAX 16 /* maximum bit length of any code (16 for explode) */ #define N_MAX 288 /* maximum number of codes in any set */ unsigned hufts; /* track memory usage */ int huft_build(b, n, s, d, e, t, m) unsigned *b; /* code lengths in bits (all assumed <= BMAX) */ unsigned n; /* number of codes (assumed <= N_MAX) */ unsigned s; /* number of simple-valued codes (0..s-1) */ ush *d; /* list of base values for non-simple codes */ ush *e; /* list of extra bits for non-simple codes */ struct huft **t; /* result: starting table */ int *m; /* maximum lookup bits, returns actual */ /* Given a list of code lengths and a maximum table size, make a set of tables to decode that set of codes. Return zero on success, one if the given code set is incomplete (the tables are still built in this case), two if the input is invalid (all zero length codes or an oversubscribed set of lengths), and three if not enough memory. */ { unsigned a; /* counter for codes of length k */ unsigned c[BMAX+1]; /* bit length count table */ unsigned f; /* i repeats in table every f entries */ int g; /* maximum code length */ int h; /* table level */ register unsigned i; /* counter, current code */ register unsigned j; /* counter */ register int k; /* number of bits in current code */ int l; /* bits per table (returned in m) */ register unsigned *p; /* pointer into c[], b[], or v[] */ register struct huft *q; /* points to current table */ struct huft r; /* table entry for structure assignment */ struct huft *u[BMAX]; /* table stack */ unsigned v[N_MAX]; /* values in order of bit length */ register int w; /* bits before this table == (l * h) */ unsigned x[BMAX+1]; /* bit offsets, then code stack */ unsigned *xp; /* pointer into x */ int y; /* number of dummy codes added */ unsigned z; /* number of entries in current table */ /* Generate counts for each bit length */ memzero(c, sizeof(c)); p = b; i = n; do { Tracecv(*p, (stderr, (n-i >= ' ' && n-i <= '~' ? "%c %d\n" : "0x%x %d\n"), n-i, *p)); c[*p]++; /* assume all entries <= BMAX */ p++; /* Can't combine with above line (Solaris bug) */ } while (--i); if (c[0] == n) /* null input--all zero length codes */ { *t = (struct huft *)NULL; *m = 0; - return 0; + return 2; } /* Find minimum and maximum length, bound *m by those */ l = *m; for (j = 1; j <= BMAX; j++) if (c[j]) break; k = j; /* minimum code length */ if ((unsigned)l < j) l = j; for (i = BMAX; i; i--) if (c[i]) break; g = i; /* maximum code length */ if ((unsigned)l > i) l = i; *m = l; /* Adjust last length count to fill out codes, if needed */ for (y = 1 << j; j < i; j++, y <<= 1) if ((y -= c[j]) < 0) return 2; /* bad input: more codes than bits */ if ((y -= c[i]) < 0) return 2; c[i] += y; /* Generate starting offsets into the value table for each length */ x[1] = j = 0; p = c + 1; xp = x + 2; while (--i) { /* note that i == g from above */ *xp++ = (j += *p++); } /* Make a table of values in order of bit lengths */ p = b; i = 0; do { if ((j = *p++) != 0) v[x[j]++] = i; } while (++i < n); n = x[g]; /* set n to length of v */ /* Generate the Huffman codes and for each, make the table entries */ x[0] = i = 0; /* first Huffman code is zero */ p = v; /* grab values in bit order */ h = -1; /* no tables yet--level -1 */ w = -l; /* bits decoded == (l * h) */ u[0] = (struct huft *)NULL; /* just to keep compilers happy */ q = (struct huft *)NULL; /* ditto */ z = 0; /* ditto */ /* go through the bit lengths (k already is bits in shortest code) */ for (; k <= g; k++) { a = c[k]; while (a--) { /* here i is the Huffman code of length k bits for value *p */ /* make tables up to required level */ while (k > w + l) { h++; w += l; /* previous table always l bits */ /* compute minimum size table less than or equal to l bits */ z = (z = g - w) > (unsigned)l ? l : z; /* upper limit on table size */ if ((f = 1 << (j = k - w)) > a + 1) /* try a k-w bit table */ { /* too few codes for k-w bit table */ f -= a + 1; /* deduct codes from patterns left */ xp = c + k; if (j < z) while (++j < z) /* try smaller tables up to z bits */ { if ((f <<= 1) <= *++xp) break; /* enough codes to use up j bits */ f -= *xp; /* else deduct codes from patterns */ } } z = 1 << j; /* table entries for j-bit table */ /* allocate and link in new table */ if ((q = (struct huft *)malloc((z + 1)*sizeof(struct huft))) == (struct huft *)NULL) { if (h) huft_free(u[0]); return 3; /* not enough memory */ } hufts += z + 1; /* track memory usage */ *t = q + 1; /* link to list for huft_free() */ *(t = &(q->v.t)) = (struct huft *)NULL; u[h] = ++q; /* table starts after link */ /* connect to last table, if there is one */ if (h) { x[h] = i; /* save pattern for backing up */ r.b = (uch)l; /* bits to dump before this table */ r.e = (uch)(16 + j); /* bits in this table */ r.v.t = q; /* pointer to this table */ j = i >> (w - l); /* (get around Turbo C bug) */ u[h-1][j] = r; /* connect to last table */ } } /* set up table entry in r */ r.b = (uch)(k - w); if (p >= v + n) r.e = 99; /* out of values--invalid code */ else if (*p < s) { r.e = (uch)(*p < 256 ? 16 : 15); /* 256 is end-of-block code */ r.v.n = (ush)(*p); /* simple code is just the value */ p++; /* one compiler does not like *p++ */ } else { r.e = (uch)e[*p - s]; /* non-simple--look up in lists */ r.v.n = d[*p++ - s]; } /* fill code-like entries with r */ f = 1 << (k - w); for (j = i >> w; j < z; j += f) q[j] = r; /* backwards increment the k-bit code i */ for (j = 1 << (k - 1); i & j; j >>= 1) i ^= j; i ^= j; /* backup over finished tables */ while ((i & ((1 << w) - 1)) != x[h]) { h--; /* don't need to update q */ w -= l; } } } /* Return true (1) if we were given an incomplete table */ return y != 0 && g != 1; } int huft_free(t) struct huft *t; /* table to free */ /* Free the malloc'ed tables built by huft_build(), which makes a linked list of the tables it made, with the links in a dummy first entry of each table. */ { register struct huft *p, *q; /* Go through linked list, freeing from the malloced (t[-1]) address. */ p = t; while (p != (struct huft *)NULL) { q = (--p)->v.t; free((char*)p); p = q; } return 0; } int inflate_codes(tl, td, bl, bd) struct huft *tl, *td; /* literal/length and distance decoder tables */ int bl, bd; /* number of bits decoded by tl[] and td[] */ /* inflate (decompress) the codes in a deflated (compressed) block. Return an error code or zero if it all goes ok. */ { register unsigned e; /* table entry flag/number of extra bits */ unsigned n, d; /* length and index for copy */ unsigned w; /* current window position */ struct huft *t; /* pointer to table entry */ unsigned ml, md; /* masks for bl and bd bits */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* inflate the coded data */ ml = mask_bits[bl]; /* precompute masks for speed */ md = mask_bits[bd]; for (;;) /* do until end of block */ { NEEDBITS((unsigned)bl) if ((e = (t = tl + ((unsigned)b & ml))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) if (e == 16) /* then it's a literal */ { slide[w++] = (uch)t->v.n; Tracevv((stderr, "%c", slide[w-1])); if (w == WSIZE) { flush_output(w); w = 0; } } else /* it's an EOB or a length */ { /* exit if end of block */ if (e == 15) break; /* get length of block to copy */ NEEDBITS(e) n = t->v.n + ((unsigned)b & mask_bits[e]); DUMPBITS(e); /* decode distance of block to copy */ NEEDBITS((unsigned)bd) if ((e = (t = td + ((unsigned)b & md))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) NEEDBITS(e) d = w - t->v.n - ((unsigned)b & mask_bits[e]); DUMPBITS(e) Tracevv((stderr,"\\[%d,%d]", w-d, n)); /* do the copy */ do { n -= (e = (e = WSIZE - ((d &= WSIZE-1) > w ? d : w)) > n ? n : e); #if !defined(NOMEMCPY) && !defined(DEBUG) if (w - d >= e) /* (this test assumes unsigned comparison) */ { memcpy(slide + w, slide + d, e); w += e; d += e; } else /* do it slow to avoid memcpy() overlap */ #endif /* !NOMEMCPY */ do { slide[w++] = slide[d++]; Tracevv((stderr, "%c", slide[w-1])); } while (--e); if (w == WSIZE) { flush_output(w); w = 0; } } while (n); } } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; /* done */ return 0; } int inflate_stored() /* "decompress" an inflated type 0 (stored) block. */ { unsigned n; /* number of bytes in block */ unsigned w; /* current window position */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* go to byte boundary */ n = k & 7; DUMPBITS(n); /* get the length and its complement */ NEEDBITS(16) n = ((unsigned)b & 0xffff); DUMPBITS(16) NEEDBITS(16) if (n != (unsigned)((~b) & 0xffff)) return 1; /* error in compressed data */ DUMPBITS(16) /* read and output the compressed data */ while (n--) { NEEDBITS(8) slide[w++] = (uch)b; if (w == WSIZE) { flush_output(w); w = 0; } DUMPBITS(8) } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; return 0; } int inflate_fixed() /* decompress an inflated type 1 (fixed Huffman codes) block. We should either replace this with a custom decoder, or at least precompute the Huffman tables. */ { int i; /* temporary variable */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned l[288]; /* length list for huft_build */ /* set up literal table */ for (i = 0; i < 144; i++) l[i] = 8; for (; i < 256; i++) l[i] = 9; for (; i < 280; i++) l[i] = 7; for (; i < 288; i++) /* make a complete, but wrong code set */ l[i] = 8; bl = 7; if ((i = huft_build(l, 288, 257, cplens, cplext, &tl, &bl)) != 0) return i; /* set up distance table */ for (i = 0; i < 30; i++) /* make an incomplete code set */ l[i] = 5; bd = 5; if ((i = huft_build(l, 30, 0, cpdist, cpdext, &td, &bd)) > 1) { huft_free(tl); return i; } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_dynamic() /* decompress an inflated type 2 (dynamic Huffman codes) block. */ { int i; /* temporary variables */ unsigned j; unsigned l; /* last length */ unsigned m; /* mask for bit lengths table */ unsigned n; /* number of lengths to get */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned nb; /* number of bit length codes */ unsigned nl; /* number of literal/length codes */ unsigned nd; /* number of distance codes */ #ifdef PKZIP_BUG_WORKAROUND unsigned ll[288+32]; /* literal/length and distance code lengths */ #else unsigned ll[286+30]; /* literal/length and distance code lengths */ #endif register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in table lengths */ NEEDBITS(5) nl = 257 + ((unsigned)b & 0x1f); /* number of literal/length codes */ DUMPBITS(5) NEEDBITS(5) nd = 1 + ((unsigned)b & 0x1f); /* number of distance codes */ DUMPBITS(5) NEEDBITS(4) nb = 4 + ((unsigned)b & 0xf); /* number of bit length codes */ DUMPBITS(4) #ifdef PKZIP_BUG_WORKAROUND if (nl > 288 || nd > 32) #else if (nl > 286 || nd > 30) #endif return 1; /* bad lengths */ /* read in bit-length-code lengths */ for (j = 0; j < nb; j++) { NEEDBITS(3) ll[border[j]] = (unsigned)b & 7; DUMPBITS(3) } for (; j < 19; j++) ll[border[j]] = 0; /* build decoding table for trees--single level, 7 bit lookup */ bl = 7; if ((i = huft_build(ll, 19, 19, NULL, NULL, &tl, &bl)) != 0) { if (i == 1) huft_free(tl); return i; /* incomplete code set */ } if (tl == NULL) /* Grrrhhh */ return 2; /* read in literal and distance code lengths */ n = nl + nd; m = mask_bits[bl]; i = l = 0; while ((unsigned)i < n) { NEEDBITS((unsigned)bl) j = (td = tl + ((unsigned)b & m))->b; DUMPBITS(j) j = td->v.n; if (j < 16) /* length of code in bits (0..15) */ ll[i++] = l = j; /* save last length in l */ else if (j == 16) /* repeat last length 3 to 6 times */ { NEEDBITS(2) j = 3 + ((unsigned)b & 3); DUMPBITS(2) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = l; } else if (j == 17) /* 3 to 10 zero length codes */ { NEEDBITS(3) j = 3 + ((unsigned)b & 7); DUMPBITS(3) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } else /* j == 18: 11 to 138 zero length codes */ { NEEDBITS(7) j = 11 + ((unsigned)b & 0x7f); DUMPBITS(7) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } } /* free decoding table for trees */ huft_free(tl); /* restore the global bit buffer */ bb = b; bk = k; /* build the decoding tables for literal/length and distance codes */ bl = lbits; if ((i = huft_build(ll, nl, 257, cplens, cplext, &tl, &bl)) != 0) { if (i == 1) { fprintf(stderr, " incomplete literal tree\n"); huft_free(tl); } return i; /* incomplete code set */ } bd = dbits; if ((i = huft_build(ll + nl, nd, 0, cpdist, cpdext, &td, &bd)) != 0) { if (i == 1) { fprintf(stderr, " incomplete distance tree\n"); #ifdef PKZIP_BUG_WORKAROUND i = 0; } #else huft_free(td); } huft_free(tl); return i; /* incomplete code set */ #endif } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_block(e) int *e; /* last block flag */ /* decompress an inflated block */ { unsigned t; /* block type */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in last block bit */ NEEDBITS(1) *e = (int)b & 1; DUMPBITS(1) /* read in block type */ NEEDBITS(2) t = (unsigned)b & 3; DUMPBITS(2) /* restore the global bit buffer */ bb = b; bk = k; /* inflate that block type */ if (t == 2) return inflate_dynamic(); if (t == 0) return inflate_stored(); if (t == 1) return inflate_fixed(); /* bad block type */ return 2; } int inflate() /* decompress an inflated entry */ { int e; /* last block flag */ int r; /* result code */ unsigned h; /* maximum struct huft's malloc'ed */ /* initialize window, bit buffer */ wp = 0; bk = 0; bb = 0; /* decompress until the last block */ h = 0; do { hufts = 0; if ((r = inflate_block(&e)) != 0) return r; if (hufts > h) h = hufts; } while (!e); /* Undo too much lookahead. The next read will be byte aligned so we * can discard unused bits in the last meaningful byte. */ while (bk >= 8) { bk -= 8; inptr--; } /* flush out slide */ flush_output(wp); /* return success */ #ifdef DEBUG fprintf(stderr, "<%u> ", h); #endif /* DEBUG */ return 0; } Index: releng/4.11/gnu/usr.bin/gzip/unlzh.c =================================================================== --- releng/4.11/gnu/usr.bin/gzip/unlzh.c (revision 162446) +++ releng/4.11/gnu/usr.bin/gzip/unlzh.c (revision 162447) @@ -1,401 +1,405 @@ /* unlzh.c -- decompress files in SCO compress -H (LZH) format. * The code in this file is directly derived from the public domain 'ar002' * written by Haruhiko Okumura. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #include "gzip.h" #include "lzw.h" /* just for consistency checking */ /* decode.c */ local unsigned decode OF((unsigned count, uch buffer[])); local void decode_start OF((void)); /* huf.c */ local void huf_decode_start OF((void)); local unsigned decode_c OF((void)); local unsigned decode_p OF((void)); local void read_pt_len OF((int nn, int nbit, int i_special)); local void read_c_len OF((void)); /* io.c */ local void fillbuf OF((int n)); local unsigned getbits OF((int n)); local void init_getbits OF((void)); /* maketbl.c */ local void make_table OF((int nchar, uch bitlen[], int tablebits, ush table[])); #define DICBIT 13 /* 12(-lh4-) or 13(-lh5-) */ #define DICSIZ ((unsigned) 1 << DICBIT) #ifndef CHAR_BIT # define CHAR_BIT 8 #endif #ifndef UCHAR_MAX # define UCHAR_MAX 255 #endif #define BITBUFSIZ (CHAR_BIT * 2 * sizeof(char)) /* Do not use CHAR_BIT * sizeof(bitbuf), does not work on machines * for which short is not on 16 bits (Cray). */ /* encode.c and decode.c */ #define MAXMATCH 256 /* formerly F (not more than UCHAR_MAX + 1) */ #define THRESHOLD 3 /* choose optimal value */ /* huf.c */ #define NC (UCHAR_MAX + MAXMATCH + 2 - THRESHOLD) /* alphabet = {0, 1, 2, ..., NC - 1} */ #define CBIT 9 /* $\lfloor \log_2 NC \rfloor + 1$ */ #define CODE_BIT 16 /* codeword length */ #define NP (DICBIT + 1) #define NT (CODE_BIT + 3) #define PBIT 4 /* smallest integer such that (1U << PBIT) > NP */ #define TBIT 5 /* smallest integer such that (1U << TBIT) > NT */ #if NT > NP # define NPT NT #else # define NPT NP #endif /* local ush left[2 * NC - 1]; */ /* local ush right[2 * NC - 1]; */ #define left prev #define right head #if NC > (1<<(BITS-2)) error cannot overlay left+right and prev #endif /* local uch c_len[NC]; */ #define c_len outbuf #if NC > OUTBUFSIZ error cannot overlay c_len and outbuf #endif local uch pt_len[NPT]; local unsigned blocksize; local ush pt_table[256]; /* local ush c_table[4096]; */ #define c_table d_buf #if (DIST_BUFSIZE-1) < 4095 error cannot overlay c_table and d_buf #endif /*********************************************************** io.c -- input/output ***********************************************************/ local ush bitbuf; local unsigned subbitbuf; local int bitcount; local void fillbuf(n) /* Shift bitbuf n bits left, read n bits */ int n; { bitbuf <<= n; while (n > bitcount) { bitbuf |= subbitbuf << (n -= bitcount); subbitbuf = (unsigned)try_byte(); if ((int)subbitbuf == EOF) subbitbuf = 0; bitcount = CHAR_BIT; } bitbuf |= subbitbuf >> (bitcount -= n); } local unsigned getbits(n) int n; { unsigned x; x = bitbuf >> (BITBUFSIZ - n); fillbuf(n); return x; } local void init_getbits() { bitbuf = 0; subbitbuf = 0; bitcount = 0; fillbuf(BITBUFSIZ); } /*********************************************************** maketbl.c -- make table for decoding ***********************************************************/ local void make_table(nchar, bitlen, tablebits, table) int nchar; uch bitlen[]; int tablebits; ush table[]; { ush count[17], weight[17], start[18], *p; unsigned i, k, len, ch, jutbits, avail, nextcode, mask; for (i = 1; i <= 16; i++) count[i] = 0; - for (i = 0; i < (unsigned)nchar; i++) count[bitlen[i]]++; + for (i = 0; i < (unsigned)nchar; i++) { + if (bitlen[i] > 16) + error("Bad table (case a)\n"); + else count[bitlen[i]]++; + } start[1] = 0; for (i = 1; i <= 16; i++) start[i + 1] = start[i] + (count[i] << (16 - i)); - if ((start[17] & 0xffff) != 0) - error("Bad table\n"); + if ((start[17] & 0xffff) != 0 || tablebits > 16) /* 16 for weight below */ + error("Bad table (case b)\n"); jutbits = 16 - tablebits; for (i = 1; i <= (unsigned)tablebits; i++) { start[i] >>= jutbits; weight[i] = (unsigned) 1 << (tablebits - i); } while (i <= 16) { weight[i] = (unsigned) 1 << (16 - i); i++; } i = start[tablebits + 1] >> jutbits; if (i != 0) { - k = 1 << tablebits; - while (i != k) table[i++] = 0; + k = MIN(1 << tablebits, DIST_BUFSIZE); + while (i < k) table[i++] = 0; } avail = nchar; mask = (unsigned) 1 << (15 - tablebits); for (ch = 0; ch < (unsigned)nchar; ch++) { if ((len = bitlen[ch]) == 0) continue; - nextcode = start[len] + weight[len]; + nextcode = MIN(start[len] + weight[len], DIST_BUFSIZE); if (len <= (unsigned)tablebits) { for (i = start[len]; i < nextcode; i++) table[i] = ch; } else { k = start[len]; p = &table[k >> jutbits]; i = len - tablebits; while (i != 0) { if (*p == 0) { right[avail] = left[avail] = 0; *p = avail++; } if (k & mask) p = &right[*p]; else p = &left[*p]; k <<= 1; i--; } *p = ch; } start[len] = nextcode; } } /*********************************************************** huf.c -- static Huffman ***********************************************************/ local void read_pt_len(nn, nbit, i_special) int nn; int nbit; int i_special; { int i, c, n; unsigned mask; n = getbits(nbit); if (n == 0) { c = getbits(nbit); for (i = 0; i < nn; i++) pt_len[i] = 0; for (i = 0; i < 256; i++) pt_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NPT)) { c = bitbuf >> (BITBUFSIZ - 3); if (c == 7) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 3); while (mask & bitbuf) { mask >>= 1; c++; } } fillbuf((c < 7) ? 3 : c - 3); pt_len[i++] = c; if (i == i_special) { c = getbits(2); - while (--c >= 0) pt_len[i++] = 0; + while (--c >= 0 && i < NPT) pt_len[i++] = 0; } } while (i < nn) pt_len[i++] = 0; make_table(nn, pt_len, 8, pt_table); } } local void read_c_len() { int i, c, n; unsigned mask; n = getbits(CBIT); if (n == 0) { c = getbits(CBIT); for (i = 0; i < NC; i++) c_len[i] = 0; for (i = 0; i < 4096; i++) c_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NC)) { c = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (c >= NT) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) c = right[c]; else c = left [c]; mask >>= 1; - } while (c >= NT); + } while (c >= NT && (mask || c != left[c])); } fillbuf((int) pt_len[c]); if (c <= 2) { if (c == 0) c = 1; else if (c == 1) c = getbits(4) + 3; else c = getbits(CBIT) + 20; - while (--c >= 0) c_len[i++] = 0; + while (--c >= 0 && i < NC) c_len[i++] = 0; } else c_len[i++] = c - 2; } while (i < NC) c_len[i++] = 0; make_table(NC, c_len, 12, c_table); } } local unsigned decode_c() { unsigned j, mask; if (blocksize == 0) { blocksize = getbits(16); if (blocksize == 0) { return NC; /* end of file */ } read_pt_len(NT, TBIT, 3); read_c_len(); read_pt_len(NP, PBIT, -1); } blocksize--; j = c_table[bitbuf >> (BITBUFSIZ - 12)]; if (j >= NC) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 12); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NC); + } while (j >= NC && (mask || j != left[j])); } fillbuf((int) c_len[j]); return j; } local unsigned decode_p() { unsigned j, mask; j = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (j >= NP) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NP); + } while (j >= NP && (mask || j != left[j])); } fillbuf((int) pt_len[j]); if (j != 0) j = ((unsigned) 1 << (j - 1)) + getbits((int) (j - 1)); return j; } local void huf_decode_start() { init_getbits(); blocksize = 0; } /*********************************************************** decode.c ***********************************************************/ local int j; /* remaining bytes to copy */ local int done; /* set at end of input */ local void decode_start() { huf_decode_start(); j = 0; done = 0; } /* Decode the input and return the number of decoded bytes put in buffer */ local unsigned decode(count, buffer) unsigned count; uch buffer[]; /* The calling function must keep the number of bytes to be processed. This function decodes either 'count' bytes or 'DICSIZ' bytes, whichever is smaller, into the array 'buffer[]' of size 'DICSIZ' or more. Call decode_start() once for each new file before calling this function. */ { local unsigned i; unsigned r, c; r = 0; while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } for ( ; ; ) { c = decode_c(); if (c == NC) { done = 1; return r; } if (c <= UCHAR_MAX) { buffer[r] = c; - if (++r == count) return r; + if (++r >= count) return r; } else { j = c - (UCHAR_MAX + 1 - THRESHOLD); i = (r - decode_p() - 1) & (DICSIZ - 1); while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } } } } /* =========================================================================== * Unlzh in to out. Return OK or ERROR. */ int unlzh(in, out) int in; int out; { unsigned n; ifd = in; ofd = out; decode_start(); while (!done) { n = decode((unsigned) DICSIZ, window); if (!test && n > 0) { write_buf(out, (char*)window, n); } } return OK; } Index: releng/4.11/gnu/usr.bin/gzip/unpack.c =================================================================== --- releng/4.11/gnu/usr.bin/gzip/unpack.c (revision 162446) +++ releng/4.11/gnu/usr.bin/gzip/unpack.c (revision 162447) @@ -1,239 +1,238 @@ /* unpack.c -- decompress files in pack format. * Copyright (C) 1992-1993 Jean-loup Gailly * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include "tailor.h" #include "gzip.h" #include "crypt.h" -#define MIN(a,b) ((a) <= (b) ? (a) : (b)) /* The arguments must not have side effects. */ #define MAX_BITLEN 25 /* Maximum length of Huffman codes. (Minor modifications to the code * would be needed to support 32 bits codes, but pack never generates * more than 24 bits anyway.) */ #define LITERALS 256 /* Number of literals, excluding the End of Block (EOB) code */ #define MAX_PEEK 12 /* Maximum number of 'peek' bits used to optimize traversal of the * Huffman tree. */ local ulg orig_len; /* original uncompressed length */ local int max_len; /* maximum bit length of Huffman codes */ local uch literal[LITERALS]; /* The literal bytes present in the Huffman tree. The EOB code is not * represented. */ local int lit_base[MAX_BITLEN+1]; /* All literals of a given bit length are contiguous in literal[] and * have contiguous codes. literal[code+lit_base[len]] is the literal * for a code of len bits. */ local int leaves [MAX_BITLEN+1]; /* Number of leaves for each bit length */ local int parents[MAX_BITLEN+1]; /* Number of parents for each bit length */ local int peek_bits; /* Number of peek bits currently used */ /* local uch prefix_len[1 << MAX_PEEK]; */ #define prefix_len outbuf /* For each bit pattern b of peek_bits bits, prefix_len[b] is the length * of the Huffman code starting with a prefix of b (upper bits), or 0 * if all codes of prefix b have more than peek_bits bits. It is not * necessary to have a huge table (large MAX_PEEK) because most of the * codes encountered in the input stream are short codes (by construction). * So for most codes a single lookup will be necessary. */ #if (1< OUTBUFSIZ error cannot overlay prefix_len and outbuf #endif local ulg bitbuf; /* Bits are added on the low part of bitbuf and read from the high part. */ local int valid; /* number of valid bits in bitbuf */ /* all bits above the last valid bit are always zero */ /* Set code to the next 'bits' input bits without skipping them. code * must be the name of a simple variable and bits must not have side effects. * IN assertions: bits <= 25 (so that we still have room for an extra byte * when valid is only 24), and mask = (1<> (valid-(bits))) & (mask); \ } /* Skip the given number of bits (after having peeked at them): */ #define skip_bits(bits) (valid -= (bits)) #define clear_bitbuf() (valid = 0, bitbuf = 0) /* Local functions */ local void read_tree OF((void)); local void build_tree OF((void)); /* =========================================================================== * Read the Huffman tree. */ local void read_tree() { int len; /* bit length */ int base; /* base offset for a sequence of leaves */ int n; /* Read the original input size, MSB first */ orig_len = 0; for (n = 1; n <= 4; n++) orig_len = (orig_len << 8) | (ulg)get_byte(); max_len = (int)get_byte(); /* maximum bit length of Huffman codes */ if (max_len > MAX_BITLEN) { error("invalid compressed data -- Huffman code > 32 bits"); } /* Get the number of leaves at each bit length */ n = 0; for (len = 1; len <= max_len; len++) { leaves[len] = (int)get_byte(); n += leaves[len]; } if (n > LITERALS) { error("too many leaves in Huffman tree"); } Trace((stderr, "orig_len %ld, max_len %d, leaves %d\n", orig_len, max_len, n)); /* There are at least 2 and at most 256 leaves of length max_len. * (Pack arbitrarily rejects empty files and files consisting of * a single byte even repeated.) To fit the last leaf count in a * byte, it is offset by 2. However, the last literal is the EOB * code, and is not transmitted explicitly in the tree, so we must * adjust here by one only. */ leaves[max_len]++; /* Now read the leaves themselves */ base = 0; for (len = 1; len <= max_len; len++) { /* Remember where the literals of this length start in literal[] : */ lit_base[len] = base; /* And read the literals: */ - for (n = leaves[len]; n > 0; n--) { + for (n = leaves[len]; n > 0 && base < LITERALS; n--) { literal[base++] = (uch)get_byte(); } } leaves[max_len]++; /* Now include the EOB code in the Huffman tree */ } /* =========================================================================== * Build the Huffman tree and the prefix table. */ local void build_tree() { int nodes = 0; /* number of nodes (parents+leaves) at current bit length */ int len; /* current bit length */ uch *prefixp; /* pointer in prefix_len */ for (len = max_len; len >= 1; len--) { /* The number of parent nodes at this level is half the total * number of nodes at parent level: */ nodes >>= 1; parents[len] = nodes; /* Update lit_base by the appropriate bias to skip the parent nodes * (which are not represented in the literal array): */ lit_base[len] -= nodes; /* Restore nodes to be parents+leaves: */ nodes += leaves[len]; } /* Construct the prefix table, from shortest leaves to longest ones. * The shortest code is all ones, so we start at the end of the table. */ peek_bits = MIN(max_len, MAX_PEEK); prefixp = &prefix_len[1< prefix_len) *--prefixp = (uch)len; } /* The length of all other codes is unknown: */ while (prefixp > prefix_len) *--prefixp = 0; } /* =========================================================================== * Unpack in to out. This routine does not support the old pack format * with magic header \037\037. * * IN assertions: the buffer inbuf contains already the beginning of * the compressed data, from offsets inptr to insize-1 included. * The magic header has already been checked. The output buffer is cleared. */ int unpack(in, out) int in, out; /* input and output file descriptors */ { int len; /* Bit length of current code */ unsigned eob; /* End Of Block code */ register unsigned peek; /* lookahead bits */ unsigned peek_mask; /* Mask for peek_bits bits */ ifd = in; ofd = out; read_tree(); /* Read the Huffman tree */ build_tree(); /* Build the prefix table */ clear_bitbuf(); /* Initialize bit input */ peek_mask = (1< 0) { peek >>= peek_bits - len; /* discard the extra bits */ } else { /* Code of more than peek_bits bits, we must traverse the tree */ ulg mask = peek_mask; len = peek_bits; do { len++, mask = (mask<<1)+1; look_bits(peek, len, mask); } while (peek < (unsigned)parents[len]); /* loop as long as peek is a parent node */ } /* At this point, peek is the next complete code, of len bits */ if (peek == eob && len == max_len) break; /* end of file? */ put_ubyte(literal[peek+lit_base[len]]); Tracev((stderr,"%02d %04x %c\n", len, peek, literal[peek+lit_base[len]])); skip_bits(len); } /* for (;;) */ flush_window(); Trace((stderr, "bytes_out %ld\n", bytes_out)); if (orig_len != (ulg)bytes_out) { error("invalid compressed data--length error"); } return OK; } Index: releng/4.11/sys/conf/newvers.sh =================================================================== --- releng/4.11/sys/conf/newvers.sh (revision 162446) +++ releng/4.11/sys/conf/newvers.sh (revision 162447) @@ -1,99 +1,99 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 3. All advertising materials mentioning features or use of this software # must display the following acknowledgement: # This product includes software developed by the University of # California, Berkeley and its contributors. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="4.11" -BRANCH="RELEASE-p21" +BRANCH="RELEASE-p22" RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ $(dirname $0)/../sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/$year/" \ -e 's/\[your name here\]\.* /FreeBSD Inc./' \ -e 's/\[your name\]\.*/FreeBSD Inc./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/* * Copyright (c) $year * FreeBSD Inc. All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER-root} d=`pwd` h=`hostname` t=`date` cat << EOF > vers.c $COPYRIGHT char sccspad[32 - 4 /* sizeof(sccs) */] = { '\\0' }; char sccs[4] = { '@', '(', '#', ')' }; char version[] = "${VERSION} #${v}: ${t}\\n ${u}@${h}:${d}\\n"; char ostype[] = "${TYPE}"; char osrelease[] = "${RELEASE}"; int osreldate = ${RELDATE}; EOF echo `expr ${v} + 1` > version Index: releng/5.3/UPDATING =================================================================== --- releng/5.3/UPDATING (revision 162446) +++ releng/5.3/UPDATING (revision 162447) @@ -1,2048 +1,2051 @@ Updating Information for FreeBSD stable users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. Important recent entries: 20040724 (default X changes). +20060919: p34 FreeBSD-SA-06:21.gzip + Correct multiple vulnerabilities in gzip(1). + 20060906: p33 FreeBSD-SA-06:19.openssl, FreeBSD-SA-06:20.bind Correct incorrect PKCS#1 v1.5 padding validation in crypto(3). [06:19] Correct multiple denial-of-service vulnerabilities in BIND related to SIG Query Processing and Excessive Recursive Queries. [06:20] 20060823: p32 FreeBSD-SA-06:18.ppp Correct buffer overflow in the handling of LCP options in ppp(4). 20060614: p31 FreeBSD-SA-06:17.sendmail Correct a bug in the handling of multipart messages by sendmail(8) which can allow a malformed message to crash a sendmail queue processing process. 20060531: p30 FreeBSD-SA-06:15.ypserv, FreeBSD-SA-06:16.smbfs Enable inadvertantly disabled "securenet" access controls in ypserv. [06:15] Correct a bug in the handling of backslash characters in smbfs which can allow an attacker to escape from a chroot(2). [06:16] 20060419: p29 FreeBSD-SA-06:14.fpu Correct a local information leakage bug affecting AMD FPUs. 20060322: p28 FreeBSD-SA-06:11.ipsec, FreeBSD-SA-06:12.opie, FreeBSD-SA-06:13.sendmail Add missing code needed for the detection of IPSec packet replays. [06:11] Correctly identify the user running opiepasswd(1) when the login name differs from the account name. [06:12] Modify timeout handling logic in sendmail(8) to correct a reported signal handling race condition. [06:13] 20060301: p27 FreeBSD-SA-06:09.openssh, FreeBSD-SA-06:10.nfs Correct a remote DoS in OpenSSH when using PAM and privilege separation. [06:09] Correct a remote kernel panic when processing zero-length RPC records via TCP. [06:10] 20060201: p26 FreeBSD-SA-06:08.sack Avoid an infinite loop in sack scoreboard processing which can result from memory exhaustion. 20060125: p25 FreeBSD-SA-06:07.pf Correct an error in pf handling of IP packet fragments which could result in a kernel panic. [06:07] 20060111: p24 FreeBSD-SA-06:01.texindex, FreeBSD-SA-06:02.ee, FreeBSD-SA-06:03.cpio Correct insecure temporary file usage in texindex. [06:01] Correct insecure temporary file usage in ee. [06:02] Correct a race condition when setting file permissions, sanitize file names by default, and fix a buffer overflow when handling files larger than 4GB in cpio. [06:03] 20051011: p23 FreeBSD-SA-05:21.openssl Correct a man-in-the-middle SSL version rollback vulnerability. 20050909: p22 FreeBSD-SA-05:20.cvsbug [revised] Correct additional temporary file usage issues. 20050907: p21 FreeBSD-SA-05:20.cvsbug Correct insecure temporary file usage. 20050727: p20 FreeBSD-SA-05:18.zlib, FreeBSD-SA-05:19.ipsec Correct buffer overflow when handling certain deflated data streams. Correct errors in the implementation of the AES-XCBC-MAC IPsec authentication algorithm. 20050720: p19 FreeBSD-SA-05:17.devfs Correct devfs ruleset bypass. 20050706: p18 FreeBSD-SA-05:16.zlib Correct buffer overflow when handling corrupt deflated data streams. 20050629: p17 FreeBSD-SA-05:14.bzip2, FreeBSD-SA-05:15.tcp Correct bzip2 denial of service and permission race vulnerabilities. Correct TCP connection stall denial of service vulnerabilities. 20050608: p16 FreeBSD-SA-05:10.tcpdump, FreeBSD-SA-05:11.gzip, FreeBSD-SA-05:12.bind9 Correct several denial-of-service vulnerabilities in tcpdump. Correct directory traversal and race condition vulnerabilities in gzip. Correct BIND 9 DNSSEC remote denial of service vulnerability. 20050513: p15 FreeBSD-SA-05:09.htt Add a knob for disabling/enabling HTT. Default off due to information disclosure on multi-user systems. 20050508: p14 FreeBSD-SA-05:08.kmem Correct two issues which were missed from the earlier commit. 20050506: p13 FreeBSD-SA-05:08.kmem Correct several local kernel memory disclosure bugs. 20050506: p12 FreeBSD-SA-05:07.ldt Correctly validate inputs to the i386_get_ldt syscall. 20050506: p11 FreeBSD-SA-05:06.iir Correct overly liberal permissions on /dev/iir. 20050422: p10 FreeBSD-SA-05:05.cvs Correct several vulnerabilities in CVS. 20050414: p9 FreeBSD-SA-05:04.ifconf Zero a buffer in ifconf() in order to avoid accidental disclosure of kernel memory to userland. 20050406: p8 FreeBSD-SA-05:03.amd64 Correctly initialize the I/O permission bitmap on amd64. 20050404: p7 FreeBSD-SA-05:02.sendfile Correct kernel memory disclosure bug in sendfile(2). 20050328: p6 FreeBSD-SA-05:01.telnet Correct buffer overflows in telnet(1). 20050114: p5 FreeBSD-EN-05:03.ipi Avoid more than two pending IPI interrupt vectors per local APIC as this may cause deadlocks. 20050106: p4 FreeBSD-EN-05:02.sk Correct bugs in the sk(4) network driver that could result in data corruption and system crashes on SMP systems. 20050103: p3 FreeBSD-EN-05:01.nfs Correct a bug in nfsrv_create() where a call to nfsrv_access() might be made while holding the NFS server mutex, which resulted in kernel panics under certain load patterns. 20041201: p2 FreeBSD-SA-04:17.procfs Fix a tainted pointer dereference in procfs(5) and linprocfs(5) which could allow a local attacker to panic a system and/or read from kernel memory. 20041118: p1 FreeBSD-SA-04:16.fetch Correct a buffer overflow in fetch(1) which could allow a mallicious server to execute arbitrary code on the client. 20041104: FreeBSD 5.3-RELEASE 20041010: The FreeBSD keyword is no longer a requirement for a valid rc.d script. The rc(8) and rc.shutdown(8) scripts no longer check for the existence of this keyword when ordering rc.d scripts. This change touches most of the file in /etc/rc.d; therefore, if you have not modified any files in that directory it may be easier to rm -rf /etc/rc.d and then use the -i switch with mergemaster(8). 20041009: One of the syscalls the 1:1 threading library libthr uses has changed, thus breaking ABI compatibility. Make sure you rebuild this library with the kernel. 20041003: The pfil API has gained an additional argument to pass an inpcb. You should rebuild all pfil consuming modules: ipfw, ipfilter and pf. 20041001: The following libraries had their version number bumped up: /lib/libm.so.2 -> libm.so.3 /lib/libreadline.so.4 -> libreadline.so.5 /usr/lib/libhistory.so.4 -> libhistory.so.5 /usr/lib/libopie.so.2 -> libopie.so.3 /usr/lib/libpcap.so.2 -> libpcap.so.3 FreeBSD 4.10 versions of these libraries will be added to the compat4x collection. If you expect to be able to run old 4.X executables you will need to remove the old versions of these libraries. However note that any 5.X executables you have built will stop working once you remove those old libraries. You should have all your ports/packages rebuilt before removing the old libraries. 20040928: If enabled, the default is now to run named in a chroot "sandbox." For users with existing configurations in /etc/namedb the migration should be simple. Upgrade your world as usual, then after installworld but before mergemaster do the following: If named is running: /etc/rc.d/named stop cd /etc mv namedb namedb.bak mkdir -p /var/named/etc/namedb cp -Rp namedb.bak/* /var/named/etc/namedb/ mergemaster (with your usual options) If using the generated localhost* files: cd /var/named/etc/namedb /bin/sh make-localhost rm -f localhost-v6.rev localhost.rev /etc/rc.d/syslogd restart /etc/rc.d/named start If you are using a custom configuration, or if you have customised the named_* variables in /etc/rc.conf[.local] then you may have to adjust the instructions accordingly. It is suggested that you carefully examine the new named variables in /etc/defaults/rc.conf and the options in /var/named/etc/namedb/named.conf to see if they might now be more suitable. 20040925: BIND 9 has been imported into the base, and is now fully functional. BIND 8 has now been removed. There are numerous differences between BIND 8 and 9, and users with critical named installations should read the migration documentation in /usr/share/doc/bind9/misc/migration. There is also a new instruction manual in /usr/share/doc/bind9/arm. The key differences that most users will experience are in how picky BIND 9 is about zone file format. If you are using named as a resolving (caching) name server, you will likely not have trouble. The following files are part of the old BIND 8 installation and should be removed: /usr/bin/dnskeygen /usr/bin/dnsquery /usr/libexec/named-xfer /usr/sbin/named.restart /usr/sbin/ndc The following files have moved to /usr/bin, and should be removed from their old locations: /usr/sbin/nslookup /usr/sbin/nsupdate 20040922: PFIL_HOOKS are a fixed part of the network stack now and do not need to be specified in the kernel configuration file anymore. Remove 'options PFIL_HOOKS' from your kernel config file. 20040914: The format of the pflogd(8) logfile "/var/log/pflog" has changed for architectures that have a 64 bit long type to make it compatible to the standard pcap format. In order to prevent corruption move away any old logfile before using a new pflogd(8). 20040913: debug.witness_* has been renamed to debug.witness.*. There are compatibility tunables left in for a few days. Update loader.conf as necessary. 20040907: The kernel and userland debugging options have been turned off by default. 20040902: The default configuration for the network stack has been changed such that it now runs without the Giant lock unless configured otherwise. If you experience network-related instability, you may wish to try setting "debug.mpsafenet=0" or compiling the kernel with "options NET_WITH_GIANT". Details on the netperf project may be found at: http://www.watson.org/~robert/freebsd/netperf/ Including the 20040828 announcement of configuration change details. 20040826: Netgraph changed its message format slightly to align the data portion well on 64 bit machines. Netgraph using utilities (e.g. ngctl, nghook, ppp, mpd, pppoed, bluetooth, ATM) should be recompiled when a new kernel is installed. 20040817: IPFW has been converted to use PFIL_HOOKS. This change is transparent to userland and preserves the ipfw ABI. The ipfw core packet inspection and filtering functions have not been changed, only how ipfw is invoked is different. Note that "option PFIL_HOOKS" is required to use IPFIREWALL compiled into the kernel or as KLD. 20040814: The RANDOM_IP_ID option has been replaced by the sysctl net.inet.ip.random_id. If you had RANDOM_IP_ID in your kernel then you may want to add "net.inet.ip.random_id=1" to /etc/sysctl.conf. 20040807: The size of 'struct ifnet' has changed due to the addition of the if_carp placeholder. All kernel modules implementing network interfaces must be recompiled as a result. 20040806: Module loading has been fixed. Some older installations will drop proper module_path initialization and modules will fail to load properly. If you have a line in /boot/loader.rc that says: "initialize drop", do (i386 only): cp /usr/src/sys/boot/i386/loader/loader.rc /boot/loader.rc chown root:wheel /boot/loader.rc chmod 444 /boot/loader.rc 20040802: making /dev/(null|zero) into a module proved to be too unpopular, so this bit has been revoked from the previous (20040801) entry. 20040801: The /dev/mem, /dev/io /dev/(null/zero) devices are now modules, so you may wish to add them to your kernel config file. See GENERIC for examples. 20040728: System compiler has been upgraded to GCC 3.4.2-pre. As with any major compiler upgrade, there are several issues to be aware of. GCC 3.4.x has broken C++ ABI compatibility with previous releases yet again and users will have to rebuild all their C++ programs with the new compiler. A new unit-at-a-time optimization mode, which is default in this compiler release, is more aggressive in removing unused static symbols. This is the likely cause of 'make buildworld' breakages with non-default CFLAGS where optimization level is set to -O2 or higher. With the upgrade of the system compiler, the kernel has been upgraded to match the new system compiler. This makes it impossible to build a new kernel with the old compiler. Upgrade your system via make buildworld and make kernel (see below) to fix this problem. 20040727: The size of 'struct ifnet' has changed due to the addition of the IFF_NEEDSGIANT flag (and what it implies). All kernel modules implementing network interfaces must be recompiled as a result. 20040716: The sound device drivers are renamed. `sound' is always required, while `snd_*' should be configured accordingly to your hardware. Refer to NOTES for the detail of the drivers. 20040710: __FreeBSD_version bumped to 502122. 20040710: The console initialization on Alpha has been reworked and is now identical to other platforms. This means that the hardcoding of the serial console and the debug port has been removed. As such, hints are now required for the sio(4) driver to become a console or debug port. The NO_SIO option has been decommissioned because of this. 20040710: A revamp of the debugging code in the kernel with some visible changes beyond just the debugging experience: o The DDB option is now specific to the DDB debugger backend and should not be used any more for conditional compilation of debugging code for when debugging is enabled. Use the KDB option for this. o The WITNESS_DDB, DDB_TRACE and DDB_UNATTENDED options have been renamed to WITNESS_KDB, KDB_TRACE and KDB_UNATTENDED respectively. This is in line with the first bullet. o The remote GDB support has been untangled from DDB and needs to be enabled separately now. Use the GDB option for this. o The GDB_REMOTE_CHAT option has been removed. Support for this homegrown feature is discontinued. The GDB remote protocol supports console output and it makes sense to use that. o The DDB_NOKLDSYM option has been removed. The DDB debugger now supports both direct symbol table lookups as well as KLD symbol lookups through the linker. 20040708: Bluetooth code has been marked as non-i386 specific. __FreeBSD_version has been bumped to 502121 to mark this change. 20040702: The native preemption has been added to the kernel scheduler. There is some report that the ULE scheduler was broken in some machines and we encourage users using the ULE scheduler either stick with a known good kernel, or temporarily switch to the 4BSD scheduler as a workaround. 20040630: The netgraph ABI version number has been incremented to indicate an incompatible change in the ABI. Old netgraph nodes will refuse to attach until recompiled. Netgraph now uses mbuf tags to move metadata and this commit removes its home-grown metadata facility. Nodes should just recompile, unless they use metadata, in which case the changes are simple; the file ng_ksocket.c serves as an example of such changes. This also broke i4b, although the compile problem has been papered over. 20040630: ACPI has been updated to disable known-bad BIOS revisions. A message will be printed on the console indicating that ACPI has been disabled automatically and that the user should use a newer BIOS, if possible. If you think ACPI does work on your system and want to override this (i.e., for testing), set hint.acpi.0.disabled="0" at the loader prompt. 20040623: pf was updated to OpenBSD-stable 3.5 and pflogd(8) is privilege separated now. It uses the newly created "_pflogd" user/group combination. If you plan to use pflogd(8) make sure to run mergemaster -p or install the "_pflogd" user and group manually. 20040622: Network interface cloning has been overhauled. This change will require a recompile of modules using cloning and modification of external ones to the new API. __FreeBSD_version has been bumped to 502119 to mark this change. Additionally, users creating stf(4) interfaces via "ifconfig stf" will need to update their scripts as this will create an interface named "stf" instead of "stf0" and ifconfig will not print "stf0" to stdout. 20040621: On 20040524, the /etc/rc.d/nsswitch script was modified to automatically create /etc/nsswitch.conf on startup if it did not already exist. Unfortunately, an error in the man page was carried over to the script, resulting in incorrect nsswitch settings. The simplest remedy is to remove both /etc/nsswitch.conf and /etc/host.conf; they will be recreated during the next reboot. 20040614: The return value of sema_timedwait(9) has been changed to make it consistent with cv_timedwait(9). Be sure to recompile the ips module and any third-party modules which call sema_timedwait. 20040613: ALTQ is now linked to the build. This breaks ABI for struct ifnet. Make sure to recompile modules and any userland that makes use of sizeof(struct ifnet). In order to get the altq headers in place please recompile and reinstall world. 20040607: Splitting kern_thread.c into 2 files (adding kern_kse.c) requires that you re-run config after updating your tree. 20040601: The MIDI drivers have been removed. Until the new module-friendly ones are merged, remove or comment out midi and seq from your kernel configuration. 20040423: Due to a new option in ipfw (versrcreach) the ipfw(8) command needs to be recompiled. Normal accept/reject rules without options are not affected but those with options may break until ipfw(8) is recompiled. 20040420: Due to changes in the callout ABI, kernels compiled after this date may be incompatible with kernel modules compiled prior to 20040406. 20040414: The PCI bus power state stuff has been turned on. If this causes problems for your system, please disable it using the tunable hw.pci.do_powerstate=0. 20040412: The bulk of the pci problems have been fixed, although the floppy drive is still broken. 20040410: A substantial update to the pci bus resource and power management have been committed. Expect a bumpy ride for a few days until the unanticipated problems have been resolved. 20040409: Due to changes in the the Yarrow initialization process, /dev/random needs to be fed before operations requiring temp files can succeed in single user mode. This includes running "make installworld". /dev/random may be fed by running "/etc/rc.d/initrandom start" or with 20040415 source by running "/etc/rc.d/preseedrandom". 20040322: The debug.mpsafenet tunable controls whether the kernel Giant lock is held across the lower levels of the network stack, and by default is turned off. In the few days following 20040322, the behavior of debug.mpsafenet will change such that this tunable controls Giant over all levels of the network stack. If you are currently setting debug.mpsafenet to 1, you should set it back to 0 (the default) again during the change-over. An additional note will be added to UPDATING when sufficient locking is merged to permit this to take place. 20040310: The FreeBSD/sparc64 platform is changing time_t from 32-bits to 64-bits. This is a very major incompatible change, so people using FreeBSD/sparc64 *must* read the UPDATING.64BTT file for detailed instructions on how to make this upgrade. People upgrading FreeBSD on other platforms can ignore this event. 20040308: The packet filter (pf) is now installed with the base system. Make sure to run mergemaster -p before installworld to create required user account ("proxy"). If you do not want to build pf with your system you can use the NO_PF knob in make.conf. Also note that pf requires "options PFIL_HOOKS" in the kernel. The pf system consists of the following three devices: device pf # required device pflog # optional device pfsync # optional 20040303: If you are having trouble with the libc_r -> libpthread transition (see the 20040130 entry), place the following lines at the top of /etc/libmap.conf: libc_r.so.5 libpthread.so.1 libc_r.so libpthread.so This will cause all programs and libraries linked against libc_r to use libpthread instead. 20040226: Some sshd configuration defaults have changed: protocol version 1 is no longer enabled by default, and password authentication is disabled by default if PAM is enabled (which it is by default). OpenSSH clients should not be affected by this; other clients may have to be reconfigured, upgraded or replaced. 20040225: The ABIs defined in and have been updated to support improved reentrancy. Multi-threaded programs that reference the "_res" or "h_errno" symbols may experience some problems if they are not recompiled. Single-threaded programs should remain unaffected. 20040225: routed has been updated in the base system from the vendor sources, routed v2.27, from rhyolite.com. This change means that for users who use RIP's MD5 authentication feature, FreeBSD -CURRENT's routed is now incompatible with previous versions of FreeBSD; however it is now compatible with implementations from Sun, Cisco and other vendors. 20040224: The tcpcb structure has changed and makes a recompile of libkvm and related userland network utilities necessary. 20040222: The cdevsw structure has changed in two externally visible ways. First, the sense of the D_GIANT flag has changed to D_NEEDSGIANT. Second, the d_version field must be filled in with D_VERSION. Drivers outside the tree will need to be updated. 20040207: The /etc/rc.d/ttys script has been removed. It is no longer necessary since devfs has been mandatory for some time. 20040130: libkse has been renamed back to libpthread and is now the default threads library. The gcc -pthread option has also been changed to link to libpthread instead of libc_r. For alpha and sparc64 machines, libkse is not renamed and links are installed so that libpthread points to libc_r. Until the ports system is updated to handle this change, it is recommended that folks install an /etc/libmap.conf(5) that maps libc_r to libpthread. If you have any binaries or libraries linked to libkse, then it is also recommended that you map libkse to libpthread. Anyone that is using nvidia supplied drivers and libraries should use a libmap.conf that maps libpthread to libc_r since their drivers/libraries do not work with libpthread. 20040125: ULE has entered into its probationary period as the default scheduler in GENERIC. For the average user, interactivity is reported to be better in many cases. On SMP machines ULE will be able to make more efficient use of the available parallel resources. If you are not running it now, please switch over, replacing the kernel option SCHED_4BSD with SCHED_ULE. 20040125: Move LongRun support out of identcpu.c, where it hardly belongs, into its own file and make it opt-in, not mandatory, depending on CPU_ENABLE_LONGRUN config(8) option. 20031213: src/lib/libc/gen/initgroups.c:1.8 now causes logins to fail if the login process is unable to successfully set the process credentials to include all groups defined for the user. The current kernel limit is 16 groups; administrators may wish to check that users do not have over 16 groups defined, or they will be unable to log in. 20031203: The ACPI module has been reactivated. It is no longer required to compile ACPI support into kernels statically. 20031112: The statfs structure has been updated with 64-bit fields to allow accurate reporting of multi-terabyte filesystem sizes. You should build world, then build and boot the new kernel BEFORE doing a `installworld' as the new kernel will know about binaries using the old statfs structure, but an old kernel will not know about the new system calls that support the new statfs structure. Note that the backwards compatibility is only present when the kernel is configured with the COMPAT_FREEBSD4 option. Since even /bin/sh will not run with a new kernel without said option you're pretty much dead in the water without it. Make sure you have COMPAT_FREEBSD4! Running an old kernel after a `make world' will cause programs such as `df' that do a statfs system call to fail with a bad system call. Marco Wertejuk also reports that cfsd (ports/security/cfs) needs to be recompiled after these changes are installed. ****************************DANGER******************************* DO NOT make installworld after the buildworld w/o building and installing a new kernel FIRST. You will be unable to build a new kernel otherwise on a system with new binaries and an old kernel. 20031112: Some netgraph string length constants have been changed. This change requires the netgraph kernel modules and all netgraph userland components to be in sync. Especially users who require netgraph to boot need to make sure to have world and kernel in sync before rebooting. 20031111: Hyperthreading logical CPU's are no longer probed by default when using the MP Table. If ACPI is being used, then logical CPUs will be probed if hyperthreading is enabled in the BIOS. If ACPI is not being used and hyperthreading is enabled in the BIOS, logical CPUs can be enabled by building a custom kernel with the option MPTABLE_FORCE_HTT enabled. 20031103: The i386 APIC_IO kernel option has been replaced by 'device apic'. The ACPI module has also been temporarily disabled, so ACPI must be statically compiled into your kernel using 'device acpi' if you wish to use the ACPI driver. 20031031: The API and ABI of struct ifnet have been changed by removing the if_name and if_unit members and replacing them with if_xname, if_dname, and if_dunit. All network drivers and most userland programs which include net/if_var.h must be updated and recompiled. __FreeBSD_version has been bumped to 501113 to reflect this change. 20030928: Changes to the cdevsw default functions have been made to remove the need to specify nullopen() and nullclose() explicitly. __FreeBSD_version bumped to 501110. 20030926: kiconv(3) has been added. mount_msdosfs(8), mount_ntfs(8) and mount_cd9660(8) need to be in sync with kernel. 20030925: Configuring a system to use IPFILTER now requires that PFIL_HOOKS also be explicitly configured. Previously this dependency was magically handled through some cruft in net/pfil.h; but that has been removed. Building a kernel with IPFILTER but not PFIL_HOOKS will fail with obtuse errors in ip_fil.c. 20030923: Fix a bug in arplookup(), whereby a hostile party on a locally attached network could exhaust kernel memory, and cause a system panic, by sending a flood of spoofed ARP requests. See FreeBSD-SA-03:14.arp. 20030915: A change to /etc/defaults/rc.conf now causes inetd to be started with `-C 60' if it is not overridden in /etc/rc.conf. This causes inetd to stop accepting connections from an IP address that exceeds the rate of 60 connections per minute. 20030829: The following rc.d scripts have been removed and should be deleted from your installation: atm2.sh atm3.sh devdb localdaemons network1 network2 network3. Depending on when you last updated world and used mergemaster(8) you may or may not have problems during the rc boot sequence. The simplest solution is an 'rm -rf /etc/rc.d/*' and then 'mergemaster -i'. The atm2.sh atm3.sh and devdb scripts were removed some time ago, so depending on when you installed -CURRENT these scripts may or may not exist on your system. 20030824: ATAng has been committed. You need to build world as sys/ata.h has changed, and userland atacontrol depends on it. If you use ATA SW raids you need "device ataraid" in your kernel config file, as it is no longer pulled in automatically. 20030819: The OFW_NEWPCI option has been turned on in the Sparc64 GENERIC kernel. Among other things, this changes the device enumeration to be closer to Solaris. Be aware that, this can even cause the machine to not boot without manual intervention before the fstab is adjusted. 20030728: All current USB and Firewire quirks in da(4) have been deprecated and will be removed for 5.2. If this causes failure for your umass(4) devices, enable "options DA_OLD_QUIRKS" in your kernel and send the output of "camcontrol inquiry da0" to scsi@freebsd.org so the quirk can be re-enabled. 20030724: Problems with entry 20030714 have been corrected and no known issues with /rescue and -j exist for host systems after this point in time. 20030722: FPU-less support has been removed from FreeBSD. Chances are you won't notice. 386+387 support should still work after this change, but it is now a minimum requirement for the i386 port that you have real FPU hardware. 20030714: Some people are having problems with changes related to /rescue. If you are building -j N, you will need to define NO_RESCUE. Others will need to define it if /rescue has issues with their environment. People should report those issues to current@. 20030711: gcc was upgraded to 3.3. You are advised to not build -DNOCLEAN across this point. Further, it might be a good idea to remove /usr/obj. 20030610: Remove deprecated locale names and transition period code for them, finishing switching to the new scheme. Check your LANG environment variable. 20030609: CCD has been changed to be a fully GEOMified class. Kernel and ccdconfig(8) needs to be in sync, this is particularly important to remember beforehand if your source tree is on a ccd device. Consider making a copy of the old ccdconfig into /boot/kernel.good or wherever you keep your backup kernel. 20030605: There was a small window in which sed(1) was broken. If you happen to have sed(1) installed during that window, which is evidenced by an inability to build world with the failure given below, you need to manually build and install sed(1) (and only sed(1)) before doing anything else. This is a one- time snafu. Typical failure mode: In file included from /usr/src/contrib/binutils/bfd/targets.c:1092: targmatch.h:7:1: null character(s) ignored targmatch.h:12:1: null character(s) ignored targmatch.h:16:1: null character(s) ignored : The window of "sed(1)-uction" is from Wed Jun 4 15:31:55 2003 UTC to Thu Jun 5 12:10:19 2003 UTC (from rev 1.30 to rev 1.31 of usr.bin/sed/process.c). 20030505: Kerberos 5 (Heimdal) is now built by default. Setting MAKE_KERBEROS5 no longer has any effect. If you do NOT want the "base" Kerberos 5, you need to set NO_KERBEROS. 20030502: groff has been updated. If you try to do a buildworld and get an infinite loop in troff, update to May 4th or newer. If you have a newer kernel than userland, you may need to set the OSRELDATE to 500110 in your environment before starting a buildworld. 20030501: The old rc system has been removed. Please report any problems to freebsd-rc@yahoogroups.com, and/or freebsd-current@freebsd.org. Your personal versions of these files will not be removed, so you can continue to use them. However, you should take great care when updating, especially when using mergemaster, since the compatibility code that utilizes these old scripts has also been removed. 20030423: A bug has been fixed in /dev/devctl which would cause devd to hang on boot, were it not for a workaround in devd. The work around in devd will be removed around 20030507. You have until then to upgrade your kernel before updating userland. In general, you should have a userland and kernel that's in sync with each other. However, given the effects of this bug (hang on boot when starting devd), some allowances are made. 20030329: Alphas with libc from between 20030312 and 20030329 exhibit floating point exceptions (FPEs), most notably in awk(1) while upgrading the system through a buildworld. So, to successfully upgrade your Alpha, you must either downgrade your libc.so to a pre-20030312 version, or update /usr/share/mk/bsd.cpu.mk to revision 1.26 which adds -mieee to CFLAGS, then forcibly rebuild and install libc: cd /usr/src/lib/libc && \ make cleandir && make obj && \ make -DNOMAN -DNOPROFILE all && \ make -DNOMAN -DNOPROFILE install 20030208: sendmail 8.12.7 has been imported. It has one important change for IPv6 users. The default submit.mc now uses '[127.0.0.1]' instead of 'localhost' meaning only IPv4 is used to connect to the MTA. Users on IPv6-only machines will need to edit /etc/mail/submit.mc appropriately. 20030128: NODEVFS option has been removed and DEVFS thereby made standard. This makes all references to MAKEDEV obsolete, and they should be removed when convenient. 20030126: The name of the device for the ofw console has changed, sparc64 users must run mergemaster to update their installed /etc/ttys. 20030125: The scheduler framework has grown a second scheduler and consequently you must specify one and only one scheduler in your kernel config. The cvs config files have been updated to use the old scheduler which may be selected via 'options SCHED_4BSD'. If you would like to try the new, much more experimental, scheduler please try 'options SCHED_ULE' and contribute to the arch@ discussion. 20030115: A new version of the wi driver has been imported into the tree. One now must have device wlan in the config file for it to operate properly. In addition, there have been some changes to how wi devices are configured for point to point links to bring it more in line with the former way of doing things, as well as compatibility with NetBSD. 20021222: For a period after the GCC 3.2.1 import (from 12/04 to 12/22), GCC used an incompatible form of ABI for returning structures and unions which FreeBSD's GCC maintainers were not aware of relative to previous versions of FreeBSD. We have gone back to the ABI for now, and any code compiled which is required to interoperate with other code (not built at the same time) returning structs or unions should be rebuilt. 20021216: A name change in /etc/netconfig has been reverted to stay compatible with suns TIRPC and also with NetBSD. You need to run mergemaster after make world. A new libc does still work with an outdated /etc/netconfig for some time, but you'll get a warning. This warning will be removed in 20030301. 20021202: The recent binutils upgrade marks a kernel flag day on sparc64: modules built with the old binutils will not work with new kernels and vice versa. Mismatches will result in panics. Make sure your kernel and modules are in sync. 20021029: The value of IPPROTO_DIVERT has changed. Make sure to keep your kernel, netstat, natd and any third-party DIVERT consumers in sync. 20021024: Old, compatibility slices have been removed in GEOM kernels. This means that you will have to update your /etc/fstab to not use disk devices of the form /dev/ad0a. Instead, you now must specify /dev/ad0s1a, or whatever slice your FreeBSD partition really is on. The old device names have gone away, so if you use them anywhere else, you must also adjust those uses. (This doesn't affect the disks formatted in the ``dangerously-dedicated'' mode.) 20021023: Alphas with kernels from between 20020830 and 20021023 and/or rtld (ld-elf.so.1) older than 20021023 may experience problems with groff while doing a buildworld (kernel: "out of memory", fixed in rev 1.129 of kern/imgact_elf.c; rtld: "too few PT_LOAD segments", fixed in rev 1.8 of libexec/rtld-elf/map_object.c). So, to successfully upgrade your Alpha, you must either upgrade your kernel and rtld first (which might be a bit tricky), or avoid running the bootstrapped groff during the "transitional" buildworld. To avoid running groff during the transitional upgrade run make buildworld with -DNOMAN, -DNO_SHAREDOCS, and -DNO_LPR. 20020831: gcc has been upgraded to 3.2. It is not all binary compatible with earlier versions of gcc for c++ programs. All c++ programs and libraries need to be recompiled. Also, if you encounter g++ issues, rm /usr/include/g++/* before doing an installworld to make sure that stale files are removed. 20020827: Our /etc/termcap now has all the entries from the XFree86 xterm almost unchanged. This means xterm now supports color by default. If you used TERM=xterm-color in the past you now should use TERM=xterm. (xterm-color will lead to benign warnings). 20020815: A "bug" in gcc(1) that was hiding warning in system headers was fixed. It's probably time to add -DNO_WERROR to your make line again. 20020729: COPY is being deprecated. The 20010530 change was reverted, as it causes far more pain than was expected, and to always compare before installing, please use INSTALL="install -C" again. The -C option is now silently ignored when used with the -d option. 20020702: Problems with libc_r clients like KDE and GNOME have been resolved. There are still some minor problems with some signals but the system is stable enough for general use again. SMP is less so than UP but each can successfully complete multiple buildworlds. Libkvm needs to be recompiled due to KSE. 20020701: Now would be a bad time to upgrade. Something in or near the KSE commit totally broke programs using libc_r like KDE and GNOME. 20020511: The k5su utility installed as part of Kerberos 5 is no longer installed with the set-user-ID bit set by default. Add ENABLE_SUID_K5SU=yes to /etc/make.conf to have it installed with the set-user-ID bit set. 20020510: Gcc 3.1 debugging format (cc -g) has changed from STABS to DWARF2. Unfortunately our native GDB (at version 4.18) does not understand the DWARF2 debugging format. Thus you must use `gcc -gstabs+' to generated debugging information for our native GDB. 20020510: Due to the way CVS works, it may not properly update src/contrib/gcc to the 3.1 sources. The easiest fix is to `rm -rf' src/contrib/gcc and then do a cvs update. 20020421: When exec'ing set[ug]id executables, the kernel now ensures that the stdio file descriptors (0..2) are open. See FreeBSD-SA-02:23.stdio. 20020404: New sendmail startup scripts have been installed to make it easier to use alternative MTAs with FreeBSD. Setting the rc.conf variable sendmail_enable to "NO" no longer prevents any sendmail daemons from starting. Instead, either set sendmail_enable to "NONE" or change mta_start_script to a script for starting an alternative MTA. Setting mta_start_script to "" will also prevent any MTA from being started at boot. 20020403: UCONSOLE is no longer a valid kernel option. 20020315: FreeBSD 5.0 DP-1 was basically branched today. 20020225: Warnings are now errors in the kernel. Unless you are a developer, you should add -DNO_WERROR to your make line. 20020217: sendmail 8.12.2 has been imported. The sendmail binary is no longer a set-user-ID root binary and the infrastructure to support command line mail submission has changed. Be sure to run mergemaster (especially for updating /etc/rc, /etc/defaults/rc.conf, and /etc/mail) and read /etc/mail/README for more details. Due to the import of sendmail 8.12.2, a new user and group are required in order for sendmail to run as a set-group-ID binary. A 'make installworld' will use the new user and group to set the owner and group of /var/spool/clientmqueue and will fail if the new user and group do not exist. The 'smmsp' user and group must be merged from src/etc/group and src/etc/master.passwd before using 'make installworld'. 'mergemaster -p' will do this. You may need to install mergemaster before this will work if you are updating from a very old version of current. The updating recipe has changed as of this date. 20020112: The preferred configuration method for PAM is now /etc/pam.d/ rather than /etc/pam.conf. If you have an unmodified pam.conf, just delete it after your next mergemaster run. If you have local modifications, you can use /usr/src/etc/pam.d/convert.pl to incorporate them into your /etc/pam.d. Please see the following url for more details: http://www.freebsd.org/cgi/mid.cgi?db=mid&id= 20011229: If anyone here is already using the new rc.conf(5) variable networkfs_types, please note that it has changed http://www.freebsd.org/cgi/mid.cgi?db=mid&id=<9744.1009655556@axl.seasidesoftware.co.za> 20011220: sys/i4b/driver/i4b_ispppsubr.c has been retired. This file started out its life in the ISDN4BSD project as an offspring from sys/net/if_spppsubr.c, which eventually got a life of its own. All the accumulated features and bug fixes of the i4b version have now been merged back into the base system's version now. The only user-visible change resulting from this is that i4b's sppp(4) interfaces are to be managed with spppcontrol(8) again, since ispppcontrol(8) has been retired as well. (There has never been rc file support for ispppcontrol in -current, but only in -stable. That will be reverted by the time the changes are MFCed.) 20011215: The fdc(4) driver has been updated and now automatically recognizes media in `standard' formats (like 1440 KB and 720 KB for a 3.5" high-density drive) when accessing the default device node (e. g. /dev/fd0). The old variety of floppy device nodes /dev/fd*.* is no longer present by default, devices can be created (in DEVFS) on demand. They will need to be customized then for `odd' densities using fdcontrol(8). 20011209: The bugs in procfs' debugging support code have been fixed, and truss(1) now works again. 20011207: Daily security checks have been split out to use the periodic(8) scripts. Some change in configuration may be necessary. Please see http://www.freebsd.org/cgi/mid.cgi?db=mid&id=<20011207155805.R8975@blossom.cjclark.org> for details. 20011204: sos added VCD/SVCD support to ata driver and that needs the kernel and burncd to be in sync. 20011203: The procfs pseudo-filesystem has now been converted to use the pseudofs framework. If you have 'options PROCFS' in your kernel config, you'll need to add 'options PSEUDOFS' if it's not there already. This change temporarily breaks truss(1); use ktrace(1) instead until the issue has been resolved. 20011202: A security hole in OpenSSH involving `UseLogin yes' has been patched. 20011126: You need to remove /usr/obj/.../usr.bin/tip before rebuilding after this date. You need to do this only once. 20011103: Most of the awk issues have been resolved. Some rough edges may be left, but for the most part things should be back to "normal." For CURRENT's usual definition of "normal." 20011030: Awk has been upgraded to the one true awk from bell labs. Expect choppy waves in the upgrade process. 20011030: The asr driver problem has been resolved. 20011027: Due to changes in other parts of the system, the asr driver now causes the system to panic on boot. Do not use it pending correction. Comment it out of any kernel config file that you try to use from this date forward. 20011025: When crossbuilding, use TARGET=xxx where you used to use MACHINE=xxx. You don't need to set TARGET_ARCH and TARGET, unless you are changing both of them. To cross build pc98 on an alpha, for example, you need to set TARGET=pc98 and TARGET_ARCH=i386. 20011001: The kernel interface that burncd depends on has changed. You must recompile both the kernel and userland applications at the same time. 20010929: When crossbuilding, please set TARGET_ARCH rather than MACHINE_ARCH to indicate the target. In the future, one will set TARGET_MACHINE where you set MACHINE now. At the moment, setting MACHINE alone for same MACHINE_ARCH machines works (eg, you can build pc98 on an i386 machine and vice versa). 20010927: Some weird problems result from using ACPI on some machines. To disable ACPI you can add hint.acpi.0.disabled="1" to /boot/loader.conf (or by putting set X=Y at the boot loader "ok" prompt). Alternatively, you can remove it from /boot/kernel/acpi.ko or use the MODULES_OVERRIDE function in your kernel config file and not list acpi in that list. 20010924: The buildworld has been fixed. You may need to install the 4.x compatibility libraries for some old binaries to work. Add COMPAT4X=true to your /etc/make.conf to get them installed on every installworld, or execute the following to get them installed only once: cd src/lib/compat/compat4x. make all install You will see ``__stdoutp undefined'' until you do this. 20010919: There's a bug in the world build process. The cross-tools are build with the NEW headers, but the OLD libc.a. This leads to all kinds of problems with the new libc. A temporary workaround is to add CFLAGS="-O -pipe -D_OLD_STDIO" before building world when upgrading from 4.x to current. This can be removed afterwards. A proper fix to the buildworld target is needed. 20010918: Peter has committed his new kthread nfs client/server code. NFS may be unstable after this date. 20010912: KSE has hit the tree. Lots of things are now different in the kernel. While a few problems were introduced in the initial commit, most of the major ones have been found and corrected. 20010901: In OLDCARD, CardBus bridges appear to be stable. The work arounds described in the 20010604 entry are now no longer necessary and will be ignored. Most insert/remove problems have been rectified around this date. 20010823: named now runs as user bind and group bind rather than as root. If named_enable is set to YES in /etc/rc.conf, ensure that user bind is available in /etc/passwd (using vipw(8)) and that group bind is available in /etc/group. Also make sure that user or group bind has read (and not write) permission for your name server configuration and that it has read and write permission for your slave zone files and directory. If you wish to continue to run named as root (a less secure alternative), add a line to /etc/rc.conf saying named_flags= 20010709: The PAM libraries have had an API upgrade that is beyond the ability of the shared library major number to handle. It is manifested by PAM-using ports dumping core. The solution is to rebuild those ports. 20010628: The kernel compile module has moved from src/sys/compile/FOO to src/sys/${MACHINE}/compile/FOO. 20010625: The pccard modem issue from 20010613 has been corrected. OLDCARD support is still a little weak in -current. slot 1 is known not to work on some TI based cardbus bridges. Some cardbus bridges do not properly detect insert/removal events. IRQ configuration needs more safety belts. 20010617: Softupdates problems have been corrected. 20010614: Peter ripped out the linkerset support. You must, as always, rerun config after you cvsup if you are using the traditional kernel building methods. 20010613: pccard modems may not work with current after 20010604 date. Some do, others result in panics. *MAKE*SURE* that you update your config and /etc/rc.conf ala the 20010604 entry, or you will have problems (this issue will be fixed, it just hasn't been yet). 20010613: SOFTUPDATES seem to be broken since the middle of May or so. Do not use them in current. You can disable softupdates on all mounted partitions, or remove SOFTUPDATES the kernel config file. 20010612: After Peter's commits to the hints code, people have been noticing that certain devices are attached (or try to) twice. This is due to having both static hints as well as a /boot/device.hints. To work around this issue, please use only one or the other mechanism until this bug is fixed. Please note that a feature of config is that if you have config file FOO and FOO.hints, it automatically adds FOO.hints to the hints.c file, whether you want it to or not. 20010610: Locale names have changed to match other systems better. 20010604: pccard support for pci cards has been committed. You must change your /etc/pccard.conf irq lines. It must match the irq used by pcic device. Interrupt storms may result if you fail to do this. Interrupt storms look a lot like a hang. You must also install a new pccardd, otherwise you will get an interrupt storm at card reset time (just after it tells you what it is). pccardd_flags="-I" is necessary for the time being. It tells pccardd not to ask the kernel if the interrupt is really free or not before using it. You can either change the /etc/pccard.conf irq lines to match pcic, or add "-i X" to the pccardd_flags. 20010530: INSTALL=install -C is being deprecated. If you want to do this, use COPY=-C instead. The former method will be supported for only a limited time. If you see install: warning: the -d and -C options may not be specified together in your makeworld, then you need to migrate towards using COPY=-C. 20010525: It appears that vm is now stable enough to use again. However, there may be other problems, so caution is still urged. alpha definitely is in bad shape. 20010521: Minor repo damage has happened. This may cause problems with cvsup of ports. If you get errors, please see http://www.FreeBSD.org/cgi/query-pr.cgi?pr=27495 at the bottom for details on a workaround. The error message is Updater failed: Cannot delete "/usr/ports/www/jakarta-tomcat/files": Directory not empty 20010520: Vm and/or swapping are busted on -current. Please be patient. 20010519: pccard has had much reorganizational work done to it over the past few days. Everything should still work, but if not, please contact imp@freebsd.org. 20010517: ata ioctl changed. Make sure to recompile both kernel and userland at the same time. 20010517: New ncurses imported. 20010512: DEVFS is now opt out, not opt in. Barring major problems, this will be the only way to go starting July 1. 20010504: OpenSSH has been updated to 2.9. Some defaults are different, including RhostsRSAAuthentication, which changes from yes to no. 20010502: Perl breakage in 20010501 was corrected at 14:18:33 PDT. 20010501: Building perl was broken at 02:25:25 PDT. 20010430: The bug in 20010429 was corrected at 07:35:37 PDT. It is safe to go back in the water. 20010429: A bad bug was committed at 04:48:42 PDT. Don't use kernels after this date, but before the correction date. 20010423: old fsck and new kernel interactions appear to have been fixed. 20010411: fsck and the kernel were changed to handle some optimizations to directory layout. This breaks backward compatibility. Update only if you understand that you must not use the old fsck with the new kernel ever. 20010330: fsck has changed the meaning of the pass column in /etc/fstab. Please see the cvs commit to fsck.8 or the fsck.8 man page for details. It is unclear if changes to /etc/fstab are necessary. 20010319: portmap had changed name to rpcbind for maximum POLA in your current world. /etc/hosts.{allow,deny} needs changes. nfs and other rpc based programs that rely on portmapper will not work without updates to /etc/hosts.{allow,deny} and /etc/netconfig. 20010315: ata subsystem changes. ATA_ENABLE_ATAPI_DMA, ATA_ENABLE_WC and ATA_ENABLE_TAGS are no longer kernel options. They have been replaced by tunables. See ata.4 for details. 20010312: The fxp driver was converted to use miibus. If you compile fxp into your kernel statically, you will need to add miibus. 20010312: The wi device now defaults to BSS (infrastructure) mode instead of ad-hoc. 20010310: /dev/urandom should be a symbolic link to /dev/random now. Users of current not using DEVFS need to run MAKEDEV std. ssh might not work if you don't. 20010303: The ed driver has been updated. It now allows mii attachments, which means that you must include the miibus in your kernel if you use the ed driver. 20010220: The problems with libc have been corrected. It is now mostly safe to go back into the water. 20010211: The size of FILE was changed. This breaks upgrading. If you must upgrade, be prepared for pain. It also breaks almost all binaries that you've compiled on -current. You are warned that before upgrading would be a good time to do a level 0 dump of your system. No, really, I mean it this time. To get to the new system, you'll need to use the following workaround. Hopefully this can be sorted out so that we don't have to move this to the updating section. To get around the installworld problem, do: # cd /usr/src/usr.bin/sed # make install # cd /usr/src # make installworld If that doesn't work, then try: # make -k installworld # make installworld 20010207: DEVFS is now the default. If you use vinum, make sure that you do not include devfs in your kernel as problems result. 20010205: FFS_ROOT and CD9660_ROOT have been removed or deprecated. Remove them from your config. 20010122: ****************************** WARNING ****************************** buildkernel has been changed slightly ****************************** WARNING ****************************** KERNCONF replaces the variable KERNEL for buildkernel. You should update your scripts and make.conf accordingly. 20010119: config has changed to allow DEV_FOO as a replacement for NFOO. This requires a new config to build correctly. 20010116: The kernel option I386_CPU is now mutually exclusive with the other cpu types. If you have an i386 system, be sure that it only had this line. Remove it for all other configurations. 20010110: Changes to the kernel require it and burncd be in sync. 20010102: Everyone who has hw.sndunit set to something in /etc/sysctl.conf, it is now hw.snd.unit. 20010101: ex and vi were broken by some changes to sys/queue.h. If you have a bad vi, you will see make buildworld fail with a core dump while building termcap. You can work around this problem by adding -k to your make buildworld. This will cause the build to complete and install a new vi. Once that's done, you can rebuild again without the -k to pick up anything that might have been ignored by the -k option. Others have suggested that you can just rebuild libc if your vi/ex is dynamically linked, but I've not received any reports of this working. 20001228: There have been some changes to libcrypt in -current. The libscrypt/libdescrypt symlink silliness is gone and the installed libcrypt is fully functional. Be aware of this. 20001218: Linksys Fast Ethernet PCCARD cards supported by the ed driver now require the addition of flag 0x80000 to their config line in pccard.conf(5). This flag is not optional. These Linksys cards will not be recognized without it. 20001205: Important new FreeBSD-version stuff: PAM support has been worked in, partially from the "Unix" OpenSSH version. This requires adding the following in pam.conf: sshd auth sufficient pam_skey.so sshd auth required pam_unix.so try_first_pass sshd session required pam_permit.so 20001031: cvs updated to 1.11. 20001020: The random device needs more entropy, so you need to make sure that you've run mergemaster to get a /etc/rc which will seed /dev/random. If you don't and the system hangs after ldconfig, then banging on the keyboard randomly until it unhangs is one workaround. 20001010: ****************************** WARNING ****************************** Sendmail has been updated. ****************************** WARNING ****************************** o mail.local(8) is no longer installed as a set-user-id binary. o sendmail(8) is now built with STARTTLS support unless NO_OPENSSL is set. o The default /etc/mail/sendmail.cf disables the SMTP EXPN and VRFY commands. o Now using sendmail's version of vacation(1). o The sendmail cf building tools (contrib/sendmail/cf) are installed in /usr/share/sendmail/cf. o sendmail.cw changed to local-host-names More details can be found at http://people.freebsd.org/~imp/UPDATING/sendmail-20001010 20001009: The ports tree's new layout is in place. Be sure to update your entire ports tree, or you will have problems. 20001006: The perl build procedure no longer installs miniperl, nor uses the installed miniperl. It is recommended that you delete /usr/bin/miniperl. 20001005: This weekend the ports tree will be updated to a new layout. It will be in an inconsistent state until noted in the UPDATING file, or with asami-san's message to the relevant mailing lists. With this new layout, you'll need to update the whole tree for anything to work. 20000928: There was a change in the passwd format. Need more information. 20000916: /boot/kernel/kernel.ko -> /boot/kernel/kernel change has taken place. Please update boot loader (not the boot blocks) at the same time as your kernel. 20000914: The new pmtimer device is necessary for laptops. Failure to include the device will cause suspended laptops losing time when they resume. Include device pmtimer in your config file and hint.pmtimer.0.at="isa" to your /boot/device.hints file. 20000911: The random device has been turned into a (pseudo-)device, rather than an option. The supplied kernel config files have been updated. You will need to do something similar in your own kernel config file. Remove: options RANDOMDEV Add: device random If you prefer to load the loadable module, you need to do nothing. 20000909: The random device module has been renamed from randomdev.ko to random.ko. You will need to edit your /boot/loader.conf to reflect this if you load this module at boot time. The line should read: random_load="YES" 20000907: The SMPNG commit has happened. It should work, but if it doesn't, fallback to the PRE_SMPNG CVS tag. There are likely to be a variety of minor issues. Please see 20000905 to make sure you don't have model loading problems which might at first blush appear related to SMP. 20000906: nsswitch has been imported from NetBSD. Among other things, this means that /etc/host.conf is no longer used. See nsswitch.conf(5) instead. Note that at boot time rc.network will attempt to produce a new nsswitch.conf file for you if you don't have one, and you have host.conf. 20000905: The ucred structure changed size. This breaks the interface that mountd uses. Trying to use an older mountd with a newer kernel guarantees a panic. This means that you need to use kernels newer than today only with matching mountd, but you needed to do that anyway with the boot loader changes. 20000905: The boot loader has been updated. The new default kernel is now /boot/kernel/kernel.ko. The new default module location is /boot/kernel. You *MUST* upgrade your boot loader and kernel at the same time. The easiest way to do this is to do the buildworld/buildkernel/ installkernel/installworld dance. Furthermore, you are urged to delete your old /modules directory before booting the new kernel, since kldload will find stale modules in that directory instead of finding them in the correct path, /boot/kernel. The most common complaint that this cures is that the linux module crashes your machine after the update. if [ ! -d /boot/kernel.old ]; then mv /modules.old /boot/kernel.old chflags noschg /kernel.old mv /kernel.old /boot/kernel.old/kernel.ko chflags schg /boot/kernel.old/kernel.ko fi 20000904: A new issue with the sendmail upgrade has come to light. /etc/aliases has moved to /etc/mail/aliases. Mergemaster will incorrectly install the default aliases in /etc/mail rather than move the old one from /etc. So you'll need to manually move the file, create a symbolic link, remove the old /etc/aliases.db and run newaliases. For safety sake, you should stop sendmail while doing this and run the upgrade when locally sourced email is not likely to be generated. 20000825: /boot/device.hints is now required for installkernel to succeed. You should copy GENERIC.hints for your architecture into /boot/device.hints. If and only if you compile hints into your kernel, then this file may be empty. Please note, if you have an empty or missing /boot/device.hints file and you neglected to compile hints into your kernel, no boot messages will appear after the boot loader tries to start the kernel. 20000821: If you do NOT have ``options RANDOMDEV'' in your kernel and you DO want the random device then add randomdev_load="YES" to /boot/loader.conf. 20000812: suidperl is now always built and installed on the system, but with permissions of 511. If you have applications that use this program, you are now required to add ENABLE_SUIDPERL=true to /etc/make.conf. If you forget to do this, chmod 4511 /usr/bin/suidperl will fix this until the next build. 20000812: sendmail has been updated from 8.9.3 to 8.11.0. Some of the more visible changes that may immediately affect your configuration include: - New default file locations from src/contrib/sendmail/cf/README - newaliases limited to root and trusted users - MSA port (587) turned on by default - New queue file naming system so can't go from 8.11 -> 8.9 - FEATURE(`rbl') renamed to FEATURE(`dnsbl') - FEATURE(`nullclient') is more full featured - FEATURE(`nouucp') requires an argument: `reject' or `nospecial' - mail.local FreeBSD-only -b option changed to -B - See src/contrib/sendmail/RELEASE_NOTES for more info 20000810: suidperl (aka sperl) is no longer build by default. You must specifically define BUILD_SUIDPERL to "true" for it to be build. Furthermore, we recommend that you remove /usr/bin/sperl* and /usr/bin/suidperl files from your system unless you have a specific use for it. 20000729: Networking defaults have been tightened. Anybody upgrading /etc/defaults/rc.conf needs to add the following lines to /etc/rc.conf if they want to have the same setup afterwards (unless the variables already are set, of course): # Enable network daemons for user convenience. inetd_enable="YES" portmap_enable="YES" sendmail_enable="YES" 20000728: If you have null_load="YES" in your /boot/loader.conf, you will need to change that to nullfs_load="YES". 20000728: The "installkernel" target has changed slightly. Now even if you override KERNEL e.g. 'make installkernel KERNEL=MYKERNEL' it will install the MYKERNEL file (built with the buildkernel target) as /kernel rather than /MYKERNEL. Those who have updated their /boot/loader.conf files to point to /MYKERNEL should remove that entry or perform manual rename of /kernel to /MYKERNEL. 20000711: If you use CVSUP or CTM to get CVS trees, AND you used to get the old crypto files from internat.freebsd.org AND you check out files from the CVS tree with the cvs command, please read http://people.freebsd.org/~imp/internat.txt for details on potential problems that you might have and how to get around them. If you are merely a mirror, or don't answer yes to each of the clauses above, you needn't worry. 20000711: /etc/security has been updated to print the inode number of setuid programs that have changed. You will see a large spike in the number of changed programs the first time when you run mergemaster to get a new /etc/security. 20000710: /dev/random now has good entropy collection (from the keyboard and sysmouse drivers). Please ensure that either `options RANDOMDEV' is present in your kernel config file or that `randomdev_load="YES"' is in your /boot/loader.conf. If you do not have the /dev/random driver, OpenSSL (and consequently lots of crypto tools (like SSH)) will fail with strange errors. (see below, 20000624). FreeBSD-current is safe again to run Crypto. 20000709: phk made the malloc default options AJ. This may slow things down and uncover other latent bugs in the code. If you need to run at full speed, you can disable this by doing the following: ln -s aj /etc/malloc.conf 20000706: libftpio's version was accidentally bumped a few days ago. This has been corrected. You may need to remove /usr/lib/libftpio.so.6 before doing your next buildworld/installworld pair. It certainly won't hurt to remove it before the update procedure. It will break fetch until a new one is built, but ftp can be used in the interim if needed. 20000705: The crypto packages have changed for the cvsup. This has been done in a backward compatible way, but the old packages will go away at some point in the future. Look at /usr/share/examples/cvsup for details. 20000704: With the new sys/modules/sound/drivers/*, you will need to set SYSDIR until you do an installworld after July 7th. 20000704: rc.shutdown and rc will now call the rc.d scripts with start or stop. This may cause some harmless warnings from older rc.d scripts that haven't been updated. 20000630: The libfetch based version of fetch has gone into the tree. Minor problems may result on some of the less popular sites, which should be reported to des@freebsd.org. 20000625: From approximately this date forward, one must have the crypto system installed in order to build the system and kernel. While not technically strictly true, one should treat it as required and grab the crypto bits. If you are grabbing CVS trees, src-all and cvs-crypto should be treated as if they were required. You should check with the latest collections to make sure that these haven't changed. 20000624: Mark Murray just committed the first parts of a cleanup of /dev/zero, et al. This is also cleaning up /dev/random. The entropy is disconnected, so DO NOT USE VERSIONS OF FREEBSD -CURRENT FROM THIS POINT to 2000710 for cryptographic services until Mark can merge in the fixes to this work in progress. openssh and openssl should not be used to generate keys from this date to the completion of the work. If you must operate at this reduced level of security, add ' options RANDOMDEV' to your kernel or modload the randomdev module. You may also need to copy a new MAKEDEV to /dev and recreate the random and urandom devices. 20000622: The license on the softupdates is now a standard 2 clause BSD license. You may need to remove your symbolic links that used to be required when updating. 20000621: Scott Flatman sent in a decent write-up on the config file update procedure. http://people.freebsd.org/~imp/config-upd.html NOTE: LINT is gone. It has been replaced with NOTES. NOTES isn't buildable. However, you can generate a LINT file: cd /sys//conf && make LINT 20000620: Binutils 2.10 have hit the tree, or will shortly. As soon as they do, the problem noted in 20000522 will be resolved and that workaround will no longer be required. 20000615: phk removed the compatibility creation of wd devices in the ad driver. If you haven't done so already, you must update your fstab, etc to use the ad devices instead of the wd devices. In addition, you'll need to update your boot blocks to a more modern version, if you haven't already done so. Modern here means 4.0 release or newer (although older releases may work). 20000612: Peter took an axe to config(8). Be sure that you read his mail on the topic before even thinking about updating. You will need to create a /boot/device.hints or add a hints directive to your config file to compile them in statically. The format of the config file has changed as well. Please see GENERIC or NEWCARD for examples of the new format. Indirectly, this also breaks USERCONFIG. Unless a newer entry says that it has been fixed, assume that must use the hints mechanism in the loader if you need to use a machine with very old ISA cards in it. 20000522: A new set of binutils went into the tree today. Anybody building a kernel after this point is advised that they need to rebuild their binutils (or better yet do a buildworld/installworld) before building a new kernel. Due to bugs in binutils, using malloc options (eg /etc/malloc.conf or MALLOC_OPTIONS env var) J will cause ld to dump core. It is recommended that you don't set this option until the problem is resolved. 20000513: The ethernet drivers were all updated to clean up the BPF handling. 20000510: The problems with boot blocks on the alphas have been corrected. This will require some care in updating alphas. A new libstand is requires for the boot blocks to build properly. 20000503: Recompile all kld modules. Proper version dependency info is now available. 20000502: Modules have been disconnected from the buildworld tree and connected to the kernel building instead. 20000427: You may need to build gperf cd /usr/src/gnu/usr.bin/gperf && make depend all install when upgrading from 4.0 -> current. The build system now uses an option only in -current. 20000417: The method that we brand ELF binaries has changed to be more acceptable to the binutils maintainers. You will need to rebrand your ELF binaries that aren't native. One problem binary is the Linux ldconfig. After your make world, but before you reboot, you'll need to issue: brandelf -t Linux /compat/linux/sbin/ldconfig if you have Linux compatibility enabled on your machine. 20000320: If you have really bad/marginal IDE drives, you may find they don't work well. Use pio mode instead. The easiest way to cope if you have a problem combination is to add: /sbin/sysctl hw.ata.ata_dma=0 to the start of /etc/rc.conf. 20000319: The ISA and PCI compatibility shims have been connected to the options COMPAT_OLDISA and COMPAT_OLDPCI. If you are using old style PCI or ISA drivers (i.e. tx, voxware, etc.) you must include the appropriate option in your kernel config. Drivers using the shims should be updated or they won't ship with 5.0-RELEASE, targeted for 2001. 20000318: We've entered the traditional post release dumping party. Large kernel changes are being committed and are in the works. It is important to keep the systems' klds and kernel in sync as kernel interfaces and structures are changing. Before reporting kernel panics, make sure that all modules that you are loading are up to date. 20000315: If you are upgrading from an older version of FreeBSD, you need to update your boot blocks as well. 'disklabel -B ad0' will do the trick. This isn't critical until you remove your wd device entries in /dev, at which point your system will not boot. 20000315: 4.0 RELEASE shipped. Please see the 4.0 UPDATING file for how to upgrade to 4.0 from 3.x. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. With a /usr/obj tree with a fresh buildworld, make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a 5.X system. Replace ${arch} with the architecture of your machine (e.g. "i386", "alpha", "amd64", "ia64", "pc98", "sparc64", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. [7] make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installkernel KERNCONF=YOUR_KERNEL_HERE [1] [3] /etc/rc.d/preseedrandom [10] mergemaster -p [5] make installworld mergemaster [4] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make installworld DESTDIR=${CURRENT_ROOT} make buildkernel KERNCONF=YOUR_KERNEL_HERE cp /usr/src/sys/${ARCH}/conf/GENERIC.hints \ ${CURRENT_ROOT}/boot/device.hints # as needed make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cd /usr/src/etc; make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from 4.x-stable to current ---------------------------------------------- # 5.x uses more space than 4.x. Also, the location of kernel # modules has changed. If you are installing 5.0 onto a 4.x # system, you'll need about 30MB of free disk space on your / # partition. If you have less than this, you may encounter difficult # to back out of problems with this procedure. If /tmp is on # the / partition, you may want to completely remove all its content # before upgrading, as this can be a common source of shortage of # space on /. [7] make buildworld [9] make buildkernel KERNCONF=YOUR_KERNEL_HERE [8] cp sys/${MACHINE}/conf/GENERIC.hints /boot/device.hints [2] make installkernel KERNCONF=YOUR_KERNEL_HERE cd sys/boot ; make STRIP="" install [6] [1] [3] /usr/src/etc/rc.d/preseedrandom [10] mergemaster -p [5] rm -rf /usr/include/g++ make installworld mergemaster -i [4] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [2] If you have legacy ISA devices, you may need to create your own device.hints to reflect your unique hardware configuration. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd /usr/src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. For the 4.x -> 5.x upgrade, you will also see many messages about needing to recompile your userland. These are harmless and can be ignored while you proceed to the next step. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. [5] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. You may need to install a new mergemaster (cd src/usr.sbin/mergemaster && make install) after the buildworld before this step if you last updated from current before 20020224 or from -stable before 20020408. [6] 4.x boot loader can be used to boot a 5.x system, however it is difficult to do that at best. If you wish to try, then you should interrupt the boot and at the ok prompt type: ok unload ok boot /boot/kernel/kernel If this fails to work, you must install a new boot loader as described here. [7] Before you upgrade, please make sure that you are not using compatibility slices. These are device names of the form, on i386 and other architectures that use MBR slicing, /dev/ad0a without the actual slice name. Chances are excellent that these will break. You generally must update these entries to use the post FreeBSD 2.x form of /dev/ad0s1a. [8] In order to have a kernel that can run the 4.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD4 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. [9] When checking out sources, you must include the -P flag to have cvs prune empty directories. Also, if CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. In case you would like to avoid installing new packages of everything, you might want to uncomment the "COMPAT4X= YES" entry, so that 4.x compatibility libraries are built which should allow you to continue using your existing software for a while. [10] In order to create temporary files, /dev/random must be initialized by feeding data into it. /usr/src/etc/rc.d/preseedrandom takes care of this. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -current. Not all things will be listed here, and it only starts on March 15, 2000. Updating files can found in previous releases if your system is older than this. Copyright information: Copyright 1998, 2002 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. If you find this document useful, and you want to, you may buy the author a beer. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: releng/5.3/gnu/usr.bin/gzip/gzip.h =================================================================== --- releng/5.3/gnu/usr.bin/gzip/gzip.h (revision 162446) +++ releng/5.3/gnu/usr.bin/gzip/gzip.h (revision 162447) @@ -1,318 +1,320 @@ /* gzip.h -- common declarations for all gzip modules * Copyright (C) 1992-1993 Jean-loup Gailly. * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. * * $FreeBSD$ */ #if defined(__STDC__) || defined(PROTO) # define OF(args) args #else # define OF(args) () #endif #ifdef __STDC__ typedef void *voidp; #else typedef char *voidp; #endif /* I don't like nested includes, but the string and io functions are used * too often */ #include #if !defined(NO_STRING_H) || defined(STDC_HEADERS) # include # if !defined(STDC_HEADERS) && !defined(NO_MEMORY_H) && !defined(__GNUC__) # include # endif # define memzero(s, n) memset ((voidp)(s), 0, (n)) #else # include # define strchr index # define strrchr rindex # define memcpy(d, s, n) bcopy((s), (d), (n)) # define memcmp(s1, s2, n) bcmp((s1), (s2), (n)) # define memzero(s, n) bzero((s), (n)) #endif #ifndef RETSIGTYPE # define RETSIGTYPE void #endif #define local static #include typedef uint8_t uch; typedef uint16_t ush; typedef uint32_t ulg; /* Return codes from gzip */ #define OK 0 #define ERROR 1 #define WARNING 2 /* Compression methods (see algorithm.doc) */ #define STORED 0 #define COMPRESSED 1 #define PACKED 2 #define LZHED 3 /* methods 4 to 7 reserved */ #define DEFLATED 8 #define MAX_METHODS 9 extern int method; /* compression method */ /* To save memory for 16 bit systems, some arrays are overlaid between * the various modules: * deflate: prev+head window d_buf l_buf outbuf * unlzw: tab_prefix tab_suffix stack inbuf outbuf * inflate: window inbuf * unpack: window inbuf prefix_len * unlzh: left+right window c_table inbuf c_len * For compression, input is done in window[]. For decompression, output * is done in window except for unlzw. */ #ifndef INBUFSIZ # ifdef SMALL_MEM # define INBUFSIZ 0x2000 /* input buffer size */ # else # define INBUFSIZ 0x8000 /* input buffer size */ # endif #endif #define INBUF_EXTRA 64 /* required by unlzw() */ #ifndef OUTBUFSIZ # ifdef SMALL_MEM # define OUTBUFSIZ 8192 /* output buffer size */ # else # define OUTBUFSIZ 16384 /* output buffer size */ # endif #endif #define OUTBUF_EXTRA 2048 /* required by unlzw() */ #ifndef DIST_BUFSIZE # ifdef SMALL_MEM # define DIST_BUFSIZE 0x2000 /* buffer for distances, see trees.c */ # else # define DIST_BUFSIZE 0x8000 /* buffer for distances, see trees.c */ # endif #endif #ifdef DYN_ALLOC # define EXTERN(type, array) extern type * near array # define DECLARE(type, array, size) type * near array # define ALLOC(type, array, size) { \ array = (type*)fcalloc((size_t)(((size)+1L)/2), 2*sizeof(type)); \ if (array == NULL) error("insufficient memory"); \ } # define FREE(array) {if (array != NULL) fcfree(array), array=NULL;} #else # define EXTERN(type, array) extern type array[] # define DECLARE(type, array, size) type array[size] # define ALLOC(type, array, size) # define FREE(array) #endif EXTERN(uch, inbuf); /* input buffer */ EXTERN(uch, outbuf); /* output buffer */ EXTERN(ush, d_buf); /* buffer for distances, see trees.c */ EXTERN(uch, window); /* Sliding window and suffix table (unlzw) */ #define tab_suffix window #ifndef MAXSEG_64K # define tab_prefix prev /* hash link (see deflate.c) */ # define head (prev+WSIZE) /* hash head (see deflate.c) */ EXTERN(ush, tab_prefix); /* prefix code (see unlzw.c) */ #else # define tab_prefix0 prev # define head tab_prefix1 EXTERN(ush, tab_prefix0); /* prefix for even codes */ EXTERN(ush, tab_prefix1); /* prefix for odd codes */ #endif extern unsigned insize; /* valid bytes in inbuf */ extern unsigned inptr; /* index of next byte to be processed in inbuf */ extern unsigned outcnt; /* bytes in output buffer */ extern long bytes_in; /* number of input bytes */ extern long bytes_out; /* number of output bytes */ extern long header_bytes;/* number of bytes in gzip header */ #define isize bytes_in /* for compatibility with old zip sources (to be cleaned) */ extern int ifd; /* input file descriptor */ extern int ofd; /* output file descriptor */ extern char ifname[]; /* input file name or "stdin" */ extern char ofname[]; /* output file name or "stdout" */ extern char *progname; /* program name */ extern long time_stamp; /* original time stamp (modification time) */ extern long ifile_size; /* input file size, -1 for devices (debug only) */ typedef int file_t; /* Do not use stdio */ #define NO_FILE (-1) /* in memory compression */ #define PACK_MAGIC "\037\036" /* Magic header for packed files */ #define GZIP_MAGIC "\037\213" /* Magic header for gzip files, 1F 8B */ #define OLD_GZIP_MAGIC "\037\236" /* Magic header for gzip 0.5 = freeze 1.x */ #define LZH_MAGIC "\037\240" /* Magic header for SCO LZH Compress files*/ #define PKZIP_MAGIC "\120\113\003\004" /* Magic header for pkzip files */ /* gzip flag byte */ #define ASCII_FLAG 0x01 /* bit 0 set: file probably ascii text */ #define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */ #define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */ #define ORIG_NAME 0x08 /* bit 3 set: original file name present */ #define COMMENT 0x10 /* bit 4 set: file comment present */ #define ENCRYPTED 0x20 /* bit 5 set: file is encrypted */ #define RESERVED 0xC0 /* bit 6,7: reserved */ /* internal file attribute */ #define UNKNOWN 0xffff #define BINARY 0 #define ASCII 1 #ifndef WSIZE # define WSIZE 0x8000 /* window size--must be a power of two, and */ #endif /* at least 32K for zip's deflate method */ #define MIN_MATCH 3 #define MAX_MATCH 258 /* The minimum and maximum match lengths */ #define MIN_LOOKAHEAD (MAX_MATCH+MIN_MATCH+1) /* Minimum amount of lookahead, except at the end of the input file. * See deflate.c for comments about the MIN_MATCH+1. */ #define MAX_DIST (WSIZE-MIN_LOOKAHEAD) /* In order to simplify the code, particularly on 16 bit machines, match * distances are limited to MAX_DIST instead of WSIZE. */ extern int decrypt; /* flag to turn on decryption */ extern int exit_code; /* program exit code */ extern int verbose; /* be verbose (-v) */ extern int quiet; /* be quiet (-q) */ extern int level; /* compression level */ extern int test; /* check .z file integrity */ extern int to_stdout; /* output to stdout (-c) */ extern int save_orig_name; /* set if original name must be saved */ +#define MIN(a,b) ((a) <= (b) ? (a) : (b)) + #define get_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(0)) #define try_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(1)) /* put_byte is used for the compressed output, put_ubyte for the * uncompressed output. However unlzw() uses window for its * suffix table instead of its output buffer, so it does not use put_ubyte * (to be cleaned up). */ #define put_byte(c) {outbuf[outcnt++]=(uch)(c); if (outcnt==OUTBUFSIZ)\ flush_outbuf();} #define put_ubyte(c) {window[outcnt++]=(uch)(c); if (outcnt==WSIZE)\ flush_window();} /* Output a 16 bit value, lsb first */ #define put_short(w) \ { if (outcnt < OUTBUFSIZ-2) { \ outbuf[outcnt++] = (uch) ((w) & 0xff); \ outbuf[outcnt++] = (uch) ((ush)(w) >> 8); \ } else { \ put_byte((uch)((w) & 0xff)); \ put_byte((uch)((ush)(w) >> 8)); \ } \ } /* Output a 32 bit value to the bit stream, lsb first */ #define put_long(n) { \ put_short((n) & 0xffff); \ put_short(((ulg)(n)) >> 16); \ } #define seekable() 0 /* force sequential output */ #define translate_eol 0 /* no option -a yet */ #define tolow(c) (isupper(c) ? (c)-'A'+'a' : (c)) /* force to lower case */ /* Macros for getting two-byte and four-byte header values */ #define SH(p) ((ush)(uch)((p)[0]) | ((ush)(uch)((p)[1]) << 8)) #define LG(p) ((ulg)(SH(p)) | ((ulg)(SH((p)+2)) << 16)) /* Diagnostic functions */ #ifdef DEBUG # define Assert(cond,msg) {if(!(cond)) error(msg);} # define Trace(x) fprintf x # define Tracev(x) {if (verbose) fprintf x ;} # define Tracevv(x) {if (verbose>1) fprintf x ;} # define Tracec(c,x) {if (verbose && (c)) fprintf x ;} # define Tracecv(c,x) {if (verbose>1 && (c)) fprintf x ;} #else # define Assert(cond,msg) # define Trace(x) # define Tracev(x) # define Tracevv(x) # define Tracec(c,x) # define Tracecv(c,x) #endif #define WARN(msg) {if (!quiet) fprintf msg ; \ if (exit_code == OK) exit_code = WARNING;} /* in zip.c: */ extern int zip OF((int in, int out)); extern int file_read OF((char *buf, unsigned size)); /* in unzip.c */ extern int unzip OF((int in, int out)); extern int check_zipfile OF((int in)); /* in unpack.c */ extern int unpack OF((int in, int out)); /* in unlzh.c */ extern int unlzh OF((int in, int out)); /* in gzip.c */ RETSIGTYPE abort_gzip OF((void)); /* in deflate.c */ void lm_init OF((int pack_level, ush *flags)); ulg deflate OF((void)); /* in trees.c */ void ct_init OF((ush *attr, int *method)); int ct_tally OF((int dist, int lc)); ulg flush_block OF((char *buf, ulg stored_len, int eof)); /* in bits.c */ void bi_init OF((file_t zipfile)); void send_bits OF((int value, int length)); unsigned bi_reverse OF((unsigned value, int length)); void bi_windup OF((void)); void copy_block OF((char *buf, unsigned len, int header)); extern int (*read_buf) OF((char *buf, unsigned size)); /* in util.c: */ extern int copy OF((int in, int out)); extern ulg updcrc OF((uch *s, unsigned n)); extern void clear_bufs OF((void)); extern int fill_inbuf OF((int eof_ok)); extern void flush_outbuf OF((void)); extern void flush_window OF((void)); extern void write_buf OF((int fd, voidp buf, unsigned cnt)); extern char *strlwr OF((char *s)); extern char *basename OF((char *fname)); extern void make_simple_name OF((char *name)); extern char *add_envopt OF((int *argcp, char ***argvp, char *env)); extern void error OF((char *m)); extern void warn OF((char *a, char *b)); extern void read_error OF((void)); extern void write_error OF((void)); extern void display_ratio OF((long num, long den, FILE *file)); extern voidp xmalloc OF((unsigned int size)); /* in inflate.c */ extern int inflate OF((void)); Index: releng/5.3/gnu/usr.bin/gzip/inflate.c =================================================================== --- releng/5.3/gnu/usr.bin/gzip/inflate.c (revision 162446) +++ releng/5.3/gnu/usr.bin/gzip/inflate.c (revision 162447) @@ -1,958 +1,958 @@ /* inflate.c -- Not copyrighted 1992 by Mark Adler version c10p1, 10 January 1993 */ /* You can do whatever you like with this source file, though I would prefer that if you modify it and redistribute it that you include comments to that effect with your name and the date. Thank you. [The history has been moved to the file ChangeLog.] */ /* Inflate deflated (PKZIP's method 8 compressed) data. The compression method searches for as much of the current string of bytes (up to a length of 258) in the previous 32K bytes. If it doesn't find any matches (of at least length 3), it codes the next byte. Otherwise, it codes the length of the matched string and its distance backwards from the current position. There is a single Huffman code that codes both single bytes (called "literals") and match lengths. A second Huffman code codes the distance information, which follows a length code. Each length or distance code actually represents a base value and a number of "extra" (sometimes zero) bits to get to add to the base value. At the end of each deflated block is a special end-of-block (EOB) literal/ length code. The decoding process is basically: get a literal/length code; if EOB then done; if a literal, emit the decoded byte; if a length then get the distance and emit the referred-to bytes from the sliding window of previously emitted data. There are (currently) three kinds of inflate blocks: stored, fixed, and dynamic. The compressor deals with some chunk of data at a time, and decides which method to use on a chunk-by-chunk basis. A chunk might typically be 32K or 64K. If the chunk is uncompressible, then the "stored" method is used. In this case, the bytes are simply stored as is, eight bits per byte, with none of the above coding. The bytes are preceded by a count, since there is no longer an EOB code. If the data is compressible, then either the fixed or dynamic methods are used. In the dynamic method, the compressed data is preceded by an encoding of the literal/length and distance Huffman codes that are to be used to decode this block. The representation is itself Huffman coded, and so is preceded by a description of that code. These code descriptions take up a little space, and so for small blocks, there is a predefined set of codes, called the fixed codes. The fixed method is used if the block codes up smaller that way (usually for quite small chunks), otherwise the dynamic method is used. In the latter case, the codes are customized to the probabilities in the current block, and so can code it much better than the pre-determined fixed codes. The Huffman codes themselves are decoded using a mutli-level table lookup, in order to maximize the speed of decoding plus the speed of building the decoding tables. See the comments below that precede the lbits and dbits tuning parameters. */ /* Notes beyond the 1.93a appnote.txt: 1. Distance pointers never point before the beginning of the output stream. 2. Distance pointers can point back across blocks, up to 32k away. 3. There is an implied maximum of 7 bits for the bit length table and 15 bits for the actual data. 4. If only one code exists, then it is encoded using one bit. (Zero would be more efficient, but perhaps a little confusing.) If two codes exist, they are coded using one bit each (0 and 1). 5. There is no way of sending zero distance codes--a dummy must be sent if there are none. (History: a pre 2.0 version of PKZIP would store blocks with no distance codes, but this was discovered to be too harsh a criterion.) Valid only for 1.93a. 2.04c does allow zero distance codes, which is sent as one code of zero bits in length. 6. There are up to 286 literal/length codes. Code 256 represents the end-of-block. Note however that the static length tree defines 288 codes just to fill out the Huffman codes. Codes 286 and 287 cannot be used though, since there is no length base or extra bits defined for them. Similarly, there are up to 30 distance codes. However, static trees define 32 codes (all 5 bits) to fill out the Huffman codes, but the last two had better not show up in the data. 7. Unzip can check dynamic Huffman blocks for complete code sets. The exception is that a single code would not be complete (see #4). 8. The five bits following the block type is really the number of literal codes sent minus 257. 9. Length codes 8,16,16 are interpreted as 13 length codes of 8 bits (1+6+6). Therefore, to output three times the length, you output three codes (1+1+1), whereas to output four times the same length, you only need two codes (1+3). Hmm. 10. In the tree reconstruction algorithm, Code = Code + Increment only if BitLength(i) is not zero. (Pretty obvious.) 11. Correction: 4 Bits: # of Bit Length codes - 4 (4 - 19) 12. Note: length code 284 can represent 227-258, but length code 285 really is 258. The last length deserves its own, short code since it gets used a lot in very redundant files. The length 258 is special since 258 - 3 (the min match length) is 255. 13. The literal/length and distance code bit lengths are read as a single stream of lengths. It is possible (and advantageous) for a repeat code (16, 17, or 18) to go across the boundary between the two sets of lengths. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #if defined(STDC_HEADERS) || !defined(NO_STDLIB_H) # include #endif #include "gzip.h" #define slide window /* Huffman code lookup table entry--this entry is four bytes for machines that have 16-bit pointers (e.g. PC's in the small or medium model). Valid extra bits are 0..13. e == 15 is EOB (end of block), e == 16 means that v is a literal, 16 < e < 32 means that v is a pointer to the next table, which codes e - 16 bits, and lastly e == 99 indicates an unused code. If a code with e == 99 is looked up, this implies an error in the data. */ struct huft { uch e; /* number of extra bits or operation */ uch b; /* number of bits in this code or subcode */ union { ush n; /* literal, length base, or distance base */ struct huft *t; /* pointer to next level of table */ } v; }; /* Function prototypes */ int huft_build OF((unsigned *, unsigned, unsigned, ush *, ush *, struct huft **, int *)); int huft_free OF((struct huft *)); int inflate_codes OF((struct huft *, struct huft *, int, int)); int inflate_stored OF((void)); int inflate_fixed OF((void)); int inflate_dynamic OF((void)); int inflate_block OF((int *)); int inflate OF((void)); /* The inflate algorithm uses a sliding 32K byte window on the uncompressed stream to find repeated byte strings. This is implemented here as a circular buffer. The index is updated simply by incrementing and then and'ing with 0x7fff (32K-1). */ /* It is left to other modules to supply the 32K area. It is assumed to be usable as if it were declared "uch slide[32768];" or as just "uch *slide;" and then malloc'ed in the latter case. The definition must be in unzip.h, included above. */ /* unsigned wp; current position in slide */ #define wp outcnt #define flush_output(w) (wp=(w),flush_window()) /* Tables for deflate from PKZIP's appnote.txt. */ static unsigned border[] = { /* Order of the bit length code lengths */ 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; static ush cplens[] = { /* Copy lengths for literal codes 257..285 */ 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0}; /* note: see note #13 above about the 258 in this list. */ static ush cplext[] = { /* Extra bits for literal codes 257..285 */ 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0, 99, 99}; /* 99==invalid */ static ush cpdist[] = { /* Copy offsets for distance codes 0..29 */ 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577}; static ush cpdext[] = { /* Extra bits for distance codes */ 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13}; /* Macros for inflate() bit peeking and grabbing. The usage is: NEEDBITS(j) x = b & mask_bits[j]; DUMPBITS(j) where NEEDBITS makes sure that b has at least j bits in it, and DUMPBITS removes the bits from b. The macros use the variable k for the number of bits in b. Normally, b and k are register variables for speed, and are initialized at the beginning of a routine that uses these macros from a global bit buffer and count. If we assume that EOB will be the longest code, then we will never ask for bits with NEEDBITS that are beyond the end of the stream. So, NEEDBITS should not read any more bytes than are needed to meet the request. Then no bytes need to be "returned" to the buffer at the end of the last block. However, this assumption is not true for fixed blocks--the EOB code is 7 bits, but the other literal/length codes can be 8 or 9 bits. (The EOB code is shorter than other codes because fixed blocks are generally short. So, while a block always has an EOB, many other literal/length codes have a significantly lower probability of showing up at all.) However, by making the first table have a lookup of seven bits, the EOB code will be found in that first lookup, and so will not require that too many bits be pulled from the stream. */ ulg bb; /* bit buffer */ unsigned bk; /* bits in bit buffer */ ush mask_bits[] = { 0x0000, 0x0001, 0x0003, 0x0007, 0x000f, 0x001f, 0x003f, 0x007f, 0x00ff, 0x01ff, 0x03ff, 0x07ff, 0x0fff, 0x1fff, 0x3fff, 0x7fff, 0xffff }; #ifdef CRYPT uch cc; # define NEXTBYTE() \ (decrypt ? (cc = get_byte(), zdecode(cc), cc) : get_byte()) #else # define NEXTBYTE() (uch)get_byte() #endif #define NEEDBITS(n) {while(k<(n)){b|=((ulg)NEXTBYTE())<>=(n);k-=(n);} /* Huffman code decoding is performed using a multi-level table lookup. The fastest way to decode is to simply build a lookup table whose size is determined by the longest code. However, the time it takes to build this table can also be a factor if the data being decoded is not very long. The most common codes are necessarily the shortest codes, so those codes dominate the decoding time, and hence the speed. The idea is you can have a shorter table that decodes the shorter, more probable codes, and then point to subsidiary tables for the longer codes. The time it costs to decode the longer codes is then traded against the time it takes to make longer tables. This results of this trade are in the variables lbits and dbits below. lbits is the number of bits the first level table for literal/ length codes can decode in one step, and dbits is the same thing for the distance codes. Subsequent tables are also less than or equal to those sizes. These values may be adjusted either when all of the codes are shorter than that, in which case the longest code length in bits is used, or when the shortest code is *longer* than the requested table size, in which case the length of the shortest code in bits is used. There are two different values for the two tables, since they code a different number of possibilities each. The literal/length table codes 286 possible values, or in a flat code, a little over eight bits. The distance table codes 30 possible values, or a little less than five bits, flat. The optimum values for speed end up being about one bit more than those, so lbits is 8+1 and dbits is 5+1. The optimum values may differ though from machine to machine, and possibly even between compilers. Your mileage may vary. */ int lbits = 9; /* bits in base literal/length lookup table */ int dbits = 6; /* bits in base distance lookup table */ /* If BMAX needs to be larger than 16, then h and x[] should be ulg. */ #define BMAX 16 /* maximum bit length of any code (16 for explode) */ #define N_MAX 288 /* maximum number of codes in any set */ unsigned hufts; /* track memory usage */ int huft_build(b, n, s, d, e, t, m) unsigned *b; /* code lengths in bits (all assumed <= BMAX) */ unsigned n; /* number of codes (assumed <= N_MAX) */ unsigned s; /* number of simple-valued codes (0..s-1) */ ush *d; /* list of base values for non-simple codes */ ush *e; /* list of extra bits for non-simple codes */ struct huft **t; /* result: starting table */ int *m; /* maximum lookup bits, returns actual */ /* Given a list of code lengths and a maximum table size, make a set of tables to decode that set of codes. Return zero on success, one if the given code set is incomplete (the tables are still built in this case), two if the input is invalid (all zero length codes or an oversubscribed set of lengths), and three if not enough memory. */ { unsigned a; /* counter for codes of length k */ unsigned c[BMAX+1]; /* bit length count table */ unsigned f; /* i repeats in table every f entries */ int g; /* maximum code length */ int h; /* table level */ register unsigned i; /* counter, current code */ register unsigned j; /* counter */ register int k; /* number of bits in current code */ int l; /* bits per table (returned in m) */ register unsigned *p; /* pointer into c[], b[], or v[] */ register struct huft *q; /* points to current table */ struct huft r; /* table entry for structure assignment */ struct huft *u[BMAX]; /* table stack */ unsigned v[N_MAX]; /* values in order of bit length */ register int w; /* bits before this table == (l * h) */ unsigned x[BMAX+1]; /* bit offsets, then code stack */ unsigned *xp; /* pointer into x */ int y; /* number of dummy codes added */ unsigned z; /* number of entries in current table */ /* Generate counts for each bit length */ memzero(c, sizeof(c)); p = b; i = n; do { Tracecv(*p, (stderr, (n-i >= ' ' && n-i <= '~' ? "%c %d\n" : "0x%x %d\n"), n-i, *p)); c[*p]++; /* assume all entries <= BMAX */ p++; /* Can't combine with above line (Solaris bug) */ } while (--i); if (c[0] == n) /* null input--all zero length codes */ { *t = (struct huft *)NULL; *m = 0; - return 0; + return 2; } /* Find minimum and maximum length, bound *m by those */ l = *m; for (j = 1; j <= BMAX; j++) if (c[j]) break; k = j; /* minimum code length */ if ((unsigned)l < j) l = j; for (i = BMAX; i; i--) if (c[i]) break; g = i; /* maximum code length */ if ((unsigned)l > i) l = i; *m = l; /* Adjust last length count to fill out codes, if needed */ for (y = 1 << j; j < i; j++, y <<= 1) if ((y -= c[j]) < 0) return 2; /* bad input: more codes than bits */ if ((y -= c[i]) < 0) return 2; c[i] += y; /* Generate starting offsets into the value table for each length */ x[1] = j = 0; p = c + 1; xp = x + 2; while (--i) { /* note that i == g from above */ *xp++ = (j += *p++); } /* Make a table of values in order of bit lengths */ p = b; i = 0; do { if ((j = *p++) != 0) v[x[j]++] = i; } while (++i < n); n = x[g]; /* set n to length of v */ /* Generate the Huffman codes and for each, make the table entries */ x[0] = i = 0; /* first Huffman code is zero */ p = v; /* grab values in bit order */ h = -1; /* no tables yet--level -1 */ w = -l; /* bits decoded == (l * h) */ u[0] = (struct huft *)NULL; /* just to keep compilers happy */ q = (struct huft *)NULL; /* ditto */ z = 0; /* ditto */ /* go through the bit lengths (k already is bits in shortest code) */ for (; k <= g; k++) { a = c[k]; while (a--) { /* here i is the Huffman code of length k bits for value *p */ /* make tables up to required level */ while (k > w + l) { h++; w += l; /* previous table always l bits */ /* compute minimum size table less than or equal to l bits */ z = (z = g - w) > (unsigned)l ? l : z; /* upper limit on table size */ if ((f = 1 << (j = k - w)) > a + 1) /* try a k-w bit table */ { /* too few codes for k-w bit table */ f -= a + 1; /* deduct codes from patterns left */ xp = c + k; if (j < z) while (++j < z) /* try smaller tables up to z bits */ { if ((f <<= 1) <= *++xp) break; /* enough codes to use up j bits */ f -= *xp; /* else deduct codes from patterns */ } } z = 1 << j; /* table entries for j-bit table */ /* allocate and link in new table */ if ((q = (struct huft *)malloc((z + 1)*sizeof(struct huft))) == (struct huft *)NULL) { if (h) huft_free(u[0]); return 3; /* not enough memory */ } hufts += z + 1; /* track memory usage */ *t = q + 1; /* link to list for huft_free() */ *(t = &(q->v.t)) = (struct huft *)NULL; u[h] = ++q; /* table starts after link */ /* connect to last table, if there is one */ if (h) { x[h] = i; /* save pattern for backing up */ r.b = (uch)l; /* bits to dump before this table */ r.e = (uch)(16 + j); /* bits in this table */ r.v.t = q; /* pointer to this table */ j = i >> (w - l); /* (get around Turbo C bug) */ u[h-1][j] = r; /* connect to last table */ } } /* set up table entry in r */ r.b = (uch)(k - w); if (p >= v + n) r.e = 99; /* out of values--invalid code */ else if (*p < s) { r.e = (uch)(*p < 256 ? 16 : 15); /* 256 is end-of-block code */ r.v.n = (ush)(*p); /* simple code is just the value */ p++; /* one compiler does not like *p++ */ } else { r.e = (uch)e[*p - s]; /* non-simple--look up in lists */ r.v.n = d[*p++ - s]; } /* fill code-like entries with r */ f = 1 << (k - w); for (j = i >> w; j < z; j += f) q[j] = r; /* backwards increment the k-bit code i */ for (j = 1 << (k - 1); i & j; j >>= 1) i ^= j; i ^= j; /* backup over finished tables */ while ((i & ((1 << w) - 1)) != x[h]) { h--; /* don't need to update q */ w -= l; } } } /* Return true (1) if we were given an incomplete table */ return y != 0 && g != 1; } int huft_free(t) struct huft *t; /* table to free */ /* Free the malloc'ed tables built by huft_build(), which makes a linked list of the tables it made, with the links in a dummy first entry of each table. */ { register struct huft *p, *q; /* Go through linked list, freeing from the malloced (t[-1]) address. */ p = t; while (p != (struct huft *)NULL) { q = (--p)->v.t; free((char*)p); p = q; } return 0; } int inflate_codes(tl, td, bl, bd) struct huft *tl, *td; /* literal/length and distance decoder tables */ int bl, bd; /* number of bits decoded by tl[] and td[] */ /* inflate (decompress) the codes in a deflated (compressed) block. Return an error code or zero if it all goes ok. */ { register unsigned e; /* table entry flag/number of extra bits */ unsigned n, d; /* length and index for copy */ unsigned w; /* current window position */ struct huft *t; /* pointer to table entry */ unsigned ml, md; /* masks for bl and bd bits */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* inflate the coded data */ ml = mask_bits[bl]; /* precompute masks for speed */ md = mask_bits[bd]; for (;;) /* do until end of block */ { NEEDBITS((unsigned)bl) if ((e = (t = tl + ((unsigned)b & ml))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) if (e == 16) /* then it's a literal */ { slide[w++] = (uch)t->v.n; Tracevv((stderr, "%c", slide[w-1])); if (w == WSIZE) { flush_output(w); w = 0; } } else /* it's an EOB or a length */ { /* exit if end of block */ if (e == 15) break; /* get length of block to copy */ NEEDBITS(e) n = t->v.n + ((unsigned)b & mask_bits[e]); DUMPBITS(e); /* decode distance of block to copy */ NEEDBITS((unsigned)bd) if ((e = (t = td + ((unsigned)b & md))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) NEEDBITS(e) d = w - t->v.n - ((unsigned)b & mask_bits[e]); DUMPBITS(e) Tracevv((stderr,"\\[%d,%d]", w-d, n)); /* do the copy */ do { n -= (e = (e = WSIZE - ((d &= WSIZE-1) > w ? d : w)) > n ? n : e); #if !defined(NOMEMCPY) && !defined(DEBUG) if (w - d >= e) /* (this test assumes unsigned comparison) */ { memcpy(slide + w, slide + d, e); w += e; d += e; } else /* do it slow to avoid memcpy() overlap */ #endif /* !NOMEMCPY */ do { slide[w++] = slide[d++]; Tracevv((stderr, "%c", slide[w-1])); } while (--e); if (w == WSIZE) { flush_output(w); w = 0; } } while (n); } } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; /* done */ return 0; } int inflate_stored() /* "decompress" an inflated type 0 (stored) block. */ { unsigned n; /* number of bytes in block */ unsigned w; /* current window position */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* go to byte boundary */ n = k & 7; DUMPBITS(n); /* get the length and its complement */ NEEDBITS(16) n = ((unsigned)b & 0xffff); DUMPBITS(16) NEEDBITS(16) if (n != (unsigned)((~b) & 0xffff)) return 1; /* error in compressed data */ DUMPBITS(16) /* read and output the compressed data */ while (n--) { NEEDBITS(8) slide[w++] = (uch)b; if (w == WSIZE) { flush_output(w); w = 0; } DUMPBITS(8) } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; return 0; } int inflate_fixed() /* decompress an inflated type 1 (fixed Huffman codes) block. We should either replace this with a custom decoder, or at least precompute the Huffman tables. */ { int i; /* temporary variable */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned l[288]; /* length list for huft_build */ /* set up literal table */ for (i = 0; i < 144; i++) l[i] = 8; for (; i < 256; i++) l[i] = 9; for (; i < 280; i++) l[i] = 7; for (; i < 288; i++) /* make a complete, but wrong code set */ l[i] = 8; bl = 7; if ((i = huft_build(l, 288, 257, cplens, cplext, &tl, &bl)) != 0) return i; /* set up distance table */ for (i = 0; i < 30; i++) /* make an incomplete code set */ l[i] = 5; bd = 5; if ((i = huft_build(l, 30, 0, cpdist, cpdext, &td, &bd)) > 1) { huft_free(tl); return i; } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_dynamic() /* decompress an inflated type 2 (dynamic Huffman codes) block. */ { int i; /* temporary variables */ unsigned j; unsigned l; /* last length */ unsigned m; /* mask for bit lengths table */ unsigned n; /* number of lengths to get */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned nb; /* number of bit length codes */ unsigned nl; /* number of literal/length codes */ unsigned nd; /* number of distance codes */ #ifdef PKZIP_BUG_WORKAROUND unsigned ll[288+32]; /* literal/length and distance code lengths */ #else unsigned ll[286+30]; /* literal/length and distance code lengths */ #endif register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in table lengths */ NEEDBITS(5) nl = 257 + ((unsigned)b & 0x1f); /* number of literal/length codes */ DUMPBITS(5) NEEDBITS(5) nd = 1 + ((unsigned)b & 0x1f); /* number of distance codes */ DUMPBITS(5) NEEDBITS(4) nb = 4 + ((unsigned)b & 0xf); /* number of bit length codes */ DUMPBITS(4) #ifdef PKZIP_BUG_WORKAROUND if (nl > 288 || nd > 32) #else if (nl > 286 || nd > 30) #endif return 1; /* bad lengths */ /* read in bit-length-code lengths */ for (j = 0; j < nb; j++) { NEEDBITS(3) ll[border[j]] = (unsigned)b & 7; DUMPBITS(3) } for (; j < 19; j++) ll[border[j]] = 0; /* build decoding table for trees--single level, 7 bit lookup */ bl = 7; if ((i = huft_build(ll, 19, 19, NULL, NULL, &tl, &bl)) != 0) { if (i == 1) huft_free(tl); return i; /* incomplete code set */ } if (tl == NULL) /* Grrrhhh */ return 2; /* read in literal and distance code lengths */ n = nl + nd; m = mask_bits[bl]; i = l = 0; while ((unsigned)i < n) { NEEDBITS((unsigned)bl) j = (td = tl + ((unsigned)b & m))->b; DUMPBITS(j) j = td->v.n; if (j < 16) /* length of code in bits (0..15) */ ll[i++] = l = j; /* save last length in l */ else if (j == 16) /* repeat last length 3 to 6 times */ { NEEDBITS(2) j = 3 + ((unsigned)b & 3); DUMPBITS(2) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = l; } else if (j == 17) /* 3 to 10 zero length codes */ { NEEDBITS(3) j = 3 + ((unsigned)b & 7); DUMPBITS(3) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } else /* j == 18: 11 to 138 zero length codes */ { NEEDBITS(7) j = 11 + ((unsigned)b & 0x7f); DUMPBITS(7) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } } /* free decoding table for trees */ huft_free(tl); /* restore the global bit buffer */ bb = b; bk = k; /* build the decoding tables for literal/length and distance codes */ bl = lbits; if ((i = huft_build(ll, nl, 257, cplens, cplext, &tl, &bl)) != 0) { if (i == 1) { fprintf(stderr, " incomplete literal tree\n"); huft_free(tl); } return i; /* incomplete code set */ } bd = dbits; if ((i = huft_build(ll + nl, nd, 0, cpdist, cpdext, &td, &bd)) != 0) { if (i == 1) { fprintf(stderr, " incomplete distance tree\n"); #ifdef PKZIP_BUG_WORKAROUND i = 0; } #else huft_free(td); } huft_free(tl); return i; /* incomplete code set */ #endif } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_block(e) int *e; /* last block flag */ /* decompress an inflated block */ { unsigned t; /* block type */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in last block bit */ NEEDBITS(1) *e = (int)b & 1; DUMPBITS(1) /* read in block type */ NEEDBITS(2) t = (unsigned)b & 3; DUMPBITS(2) /* restore the global bit buffer */ bb = b; bk = k; /* inflate that block type */ if (t == 2) return inflate_dynamic(); if (t == 0) return inflate_stored(); if (t == 1) return inflate_fixed(); /* bad block type */ return 2; } int inflate() /* decompress an inflated entry */ { int e; /* last block flag */ int r; /* result code */ unsigned h; /* maximum struct huft's malloc'ed */ /* initialize window, bit buffer */ wp = 0; bk = 0; bb = 0; /* decompress until the last block */ h = 0; do { hufts = 0; if ((r = inflate_block(&e)) != 0) return r; if (hufts > h) h = hufts; } while (!e); /* Undo too much lookahead. The next read will be byte aligned so we * can discard unused bits in the last meaningful byte. */ while (bk >= 8) { bk -= 8; inptr--; } /* flush out slide */ flush_output(wp); /* return success */ #ifdef DEBUG fprintf(stderr, "<%u> ", h); #endif /* DEBUG */ return 0; } Index: releng/5.3/gnu/usr.bin/gzip/unlzh.c =================================================================== --- releng/5.3/gnu/usr.bin/gzip/unlzh.c (revision 162446) +++ releng/5.3/gnu/usr.bin/gzip/unlzh.c (revision 162447) @@ -1,401 +1,405 @@ /* unlzh.c -- decompress files in SCO compress -H (LZH) format. * The code in this file is directly derived from the public domain 'ar002' * written by Haruhiko Okumura. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #include "gzip.h" #include "lzw.h" /* just for consistency checking */ /* decode.c */ local unsigned decode OF((unsigned count, uch buffer[])); local void decode_start OF((void)); /* huf.c */ local void huf_decode_start OF((void)); local unsigned decode_c OF((void)); local unsigned decode_p OF((void)); local void read_pt_len OF((int nn, int nbit, int i_special)); local void read_c_len OF((void)); /* io.c */ local void fillbuf OF((int n)); local unsigned getbits OF((int n)); local void init_getbits OF((void)); /* maketbl.c */ local void make_table OF((int nchar, uch bitlen[], int tablebits, ush table[])); #define DICBIT 13 /* 12(-lh4-) or 13(-lh5-) */ #define DICSIZ ((unsigned) 1 << DICBIT) #ifndef CHAR_BIT # define CHAR_BIT 8 #endif #ifndef UCHAR_MAX # define UCHAR_MAX 255 #endif #define BITBUFSIZ (CHAR_BIT * 2 * sizeof(char)) /* Do not use CHAR_BIT * sizeof(bitbuf), does not work on machines * for which short is not on 16 bits (Cray). */ /* encode.c and decode.c */ #define MAXMATCH 256 /* formerly F (not more than UCHAR_MAX + 1) */ #define THRESHOLD 3 /* choose optimal value */ /* huf.c */ #define NC (UCHAR_MAX + MAXMATCH + 2 - THRESHOLD) /* alphabet = {0, 1, 2, ..., NC - 1} */ #define CBIT 9 /* $\lfloor \log_2 NC \rfloor + 1$ */ #define CODE_BIT 16 /* codeword length */ #define NP (DICBIT + 1) #define NT (CODE_BIT + 3) #define PBIT 4 /* smallest integer such that (1U << PBIT) > NP */ #define TBIT 5 /* smallest integer such that (1U << TBIT) > NT */ #if NT > NP # define NPT NT #else # define NPT NP #endif /* local ush left[2 * NC - 1]; */ /* local ush right[2 * NC - 1]; */ #define left prev #define right head #if NC > (1<<(BITS-2)) error cannot overlay left+right and prev #endif /* local uch c_len[NC]; */ #define c_len outbuf #if NC > OUTBUFSIZ error cannot overlay c_len and outbuf #endif local uch pt_len[NPT]; local unsigned blocksize; local ush pt_table[256]; /* local ush c_table[4096]; */ #define c_table d_buf #if (DIST_BUFSIZE-1) < 4095 error cannot overlay c_table and d_buf #endif /*********************************************************** io.c -- input/output ***********************************************************/ local ush bitbuf; local unsigned subbitbuf; local int bitcount; local void fillbuf(n) /* Shift bitbuf n bits left, read n bits */ int n; { bitbuf <<= n; while (n > bitcount) { bitbuf |= subbitbuf << (n -= bitcount); subbitbuf = (unsigned)try_byte(); if ((int)subbitbuf == EOF) subbitbuf = 0; bitcount = CHAR_BIT; } bitbuf |= subbitbuf >> (bitcount -= n); } local unsigned getbits(n) int n; { unsigned x; x = bitbuf >> (BITBUFSIZ - n); fillbuf(n); return x; } local void init_getbits() { bitbuf = 0; subbitbuf = 0; bitcount = 0; fillbuf(BITBUFSIZ); } /*********************************************************** maketbl.c -- make table for decoding ***********************************************************/ local void make_table(nchar, bitlen, tablebits, table) int nchar; uch bitlen[]; int tablebits; ush table[]; { ush count[17], weight[17], start[18], *p; unsigned i, k, len, ch, jutbits, avail, nextcode, mask; for (i = 1; i <= 16; i++) count[i] = 0; - for (i = 0; i < (unsigned)nchar; i++) count[bitlen[i]]++; + for (i = 0; i < (unsigned)nchar; i++) { + if (bitlen[i] > 16) + error("Bad table (case a)\n"); + else count[bitlen[i]]++; + } start[1] = 0; for (i = 1; i <= 16; i++) start[i + 1] = start[i] + (count[i] << (16 - i)); - if ((start[17] & 0xffff) != 0) - error("Bad table\n"); + if ((start[17] & 0xffff) != 0 || tablebits > 16) /* 16 for weight below */ + error("Bad table (case b)\n"); jutbits = 16 - tablebits; for (i = 1; i <= (unsigned)tablebits; i++) { start[i] >>= jutbits; weight[i] = (unsigned) 1 << (tablebits - i); } while (i <= 16) { weight[i] = (unsigned) 1 << (16 - i); i++; } i = start[tablebits + 1] >> jutbits; if (i != 0) { - k = 1 << tablebits; - while (i != k) table[i++] = 0; + k = MIN(1 << tablebits, DIST_BUFSIZE); + while (i < k) table[i++] = 0; } avail = nchar; mask = (unsigned) 1 << (15 - tablebits); for (ch = 0; ch < (unsigned)nchar; ch++) { if ((len = bitlen[ch]) == 0) continue; - nextcode = start[len] + weight[len]; + nextcode = MIN(start[len] + weight[len], DIST_BUFSIZE); if (len <= (unsigned)tablebits) { for (i = start[len]; i < nextcode; i++) table[i] = ch; } else { k = start[len]; p = &table[k >> jutbits]; i = len - tablebits; while (i != 0) { if (*p == 0) { right[avail] = left[avail] = 0; *p = avail++; } if (k & mask) p = &right[*p]; else p = &left[*p]; k <<= 1; i--; } *p = ch; } start[len] = nextcode; } } /*********************************************************** huf.c -- static Huffman ***********************************************************/ local void read_pt_len(nn, nbit, i_special) int nn; int nbit; int i_special; { int i, c, n; unsigned mask; n = getbits(nbit); if (n == 0) { c = getbits(nbit); for (i = 0; i < nn; i++) pt_len[i] = 0; for (i = 0; i < 256; i++) pt_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NPT)) { c = bitbuf >> (BITBUFSIZ - 3); if (c == 7) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 3); while (mask & bitbuf) { mask >>= 1; c++; } } fillbuf((c < 7) ? 3 : c - 3); pt_len[i++] = c; if (i == i_special) { c = getbits(2); - while (--c >= 0) pt_len[i++] = 0; + while (--c >= 0 && i < NPT) pt_len[i++] = 0; } } while (i < nn) pt_len[i++] = 0; make_table(nn, pt_len, 8, pt_table); } } local void read_c_len() { int i, c, n; unsigned mask; n = getbits(CBIT); if (n == 0) { c = getbits(CBIT); for (i = 0; i < NC; i++) c_len[i] = 0; for (i = 0; i < 4096; i++) c_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NC)) { c = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (c >= NT) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) c = right[c]; else c = left [c]; mask >>= 1; - } while (c >= NT); + } while (c >= NT && (mask || c != left[c])); } fillbuf((int) pt_len[c]); if (c <= 2) { if (c == 0) c = 1; else if (c == 1) c = getbits(4) + 3; else c = getbits(CBIT) + 20; - while (--c >= 0) c_len[i++] = 0; + while (--c >= 0 && i < NC) c_len[i++] = 0; } else c_len[i++] = c - 2; } while (i < NC) c_len[i++] = 0; make_table(NC, c_len, 12, c_table); } } local unsigned decode_c() { unsigned j, mask; if (blocksize == 0) { blocksize = getbits(16); if (blocksize == 0) { return NC; /* end of file */ } read_pt_len(NT, TBIT, 3); read_c_len(); read_pt_len(NP, PBIT, -1); } blocksize--; j = c_table[bitbuf >> (BITBUFSIZ - 12)]; if (j >= NC) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 12); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NC); + } while (j >= NC && (mask || j != left[j])); } fillbuf((int) c_len[j]); return j; } local unsigned decode_p() { unsigned j, mask; j = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (j >= NP) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NP); + } while (j >= NP && (mask || j != left[j])); } fillbuf((int) pt_len[j]); if (j != 0) j = ((unsigned) 1 << (j - 1)) + getbits((int) (j - 1)); return j; } local void huf_decode_start() { init_getbits(); blocksize = 0; } /*********************************************************** decode.c ***********************************************************/ local int j; /* remaining bytes to copy */ local int done; /* set at end of input */ local void decode_start() { huf_decode_start(); j = 0; done = 0; } /* Decode the input and return the number of decoded bytes put in buffer */ local unsigned decode(count, buffer) unsigned count; uch buffer[]; /* The calling function must keep the number of bytes to be processed. This function decodes either 'count' bytes or 'DICSIZ' bytes, whichever is smaller, into the array 'buffer[]' of size 'DICSIZ' or more. Call decode_start() once for each new file before calling this function. */ { local unsigned i; unsigned r, c; r = 0; while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } for ( ; ; ) { c = decode_c(); if (c == NC) { done = 1; return r; } if (c <= UCHAR_MAX) { buffer[r] = c; - if (++r == count) return r; + if (++r >= count) return r; } else { j = c - (UCHAR_MAX + 1 - THRESHOLD); i = (r - decode_p() - 1) & (DICSIZ - 1); while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } } } } /* =========================================================================== * Unlzh in to out. Return OK or ERROR. */ int unlzh(in, out) int in; int out; { unsigned n; ifd = in; ofd = out; decode_start(); while (!done) { n = decode((unsigned) DICSIZ, window); if (!test && n > 0) { write_buf(out, (char*)window, n); } } return OK; } Index: releng/5.3/gnu/usr.bin/gzip/unpack.c =================================================================== --- releng/5.3/gnu/usr.bin/gzip/unpack.c (revision 162446) +++ releng/5.3/gnu/usr.bin/gzip/unpack.c (revision 162447) @@ -1,239 +1,238 @@ /* unpack.c -- decompress files in pack format. * Copyright (C) 1992-1993 Jean-loup Gailly * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include "tailor.h" #include "gzip.h" #include "crypt.h" -#define MIN(a,b) ((a) <= (b) ? (a) : (b)) /* The arguments must not have side effects. */ #define MAX_BITLEN 25 /* Maximum length of Huffman codes. (Minor modifications to the code * would be needed to support 32 bits codes, but pack never generates * more than 24 bits anyway.) */ #define LITERALS 256 /* Number of literals, excluding the End of Block (EOB) code */ #define MAX_PEEK 12 /* Maximum number of 'peek' bits used to optimize traversal of the * Huffman tree. */ local ulg orig_len; /* original uncompressed length */ local int max_len; /* maximum bit length of Huffman codes */ local uch literal[LITERALS]; /* The literal bytes present in the Huffman tree. The EOB code is not * represented. */ local int lit_base[MAX_BITLEN+1]; /* All literals of a given bit length are contiguous in literal[] and * have contiguous codes. literal[code+lit_base[len]] is the literal * for a code of len bits. */ local int leaves [MAX_BITLEN+1]; /* Number of leaves for each bit length */ local int parents[MAX_BITLEN+1]; /* Number of parents for each bit length */ local int peek_bits; /* Number of peek bits currently used */ /* local uch prefix_len[1 << MAX_PEEK]; */ #define prefix_len outbuf /* For each bit pattern b of peek_bits bits, prefix_len[b] is the length * of the Huffman code starting with a prefix of b (upper bits), or 0 * if all codes of prefix b have more than peek_bits bits. It is not * necessary to have a huge table (large MAX_PEEK) because most of the * codes encountered in the input stream are short codes (by construction). * So for most codes a single lookup will be necessary. */ #if (1< OUTBUFSIZ error cannot overlay prefix_len and outbuf #endif local ulg bitbuf; /* Bits are added on the low part of bitbuf and read from the high part. */ local int valid; /* number of valid bits in bitbuf */ /* all bits above the last valid bit are always zero */ /* Set code to the next 'bits' input bits without skipping them. code * must be the name of a simple variable and bits must not have side effects. * IN assertions: bits <= 25 (so that we still have room for an extra byte * when valid is only 24), and mask = (1<> (valid-(bits))) & (mask); \ } /* Skip the given number of bits (after having peeked at them): */ #define skip_bits(bits) (valid -= (bits)) #define clear_bitbuf() (valid = 0, bitbuf = 0) /* Local functions */ local void read_tree OF((void)); local void build_tree OF((void)); /* =========================================================================== * Read the Huffman tree. */ local void read_tree() { int len; /* bit length */ int base; /* base offset for a sequence of leaves */ int n; /* Read the original input size, MSB first */ orig_len = 0; for (n = 1; n <= 4; n++) orig_len = (orig_len << 8) | (ulg)get_byte(); max_len = (int)get_byte(); /* maximum bit length of Huffman codes */ if (max_len > MAX_BITLEN) { error("invalid compressed data -- Huffman code > 32 bits"); } /* Get the number of leaves at each bit length */ n = 0; for (len = 1; len <= max_len; len++) { leaves[len] = (int)get_byte(); n += leaves[len]; } if (n > LITERALS) { error("too many leaves in Huffman tree"); } Trace((stderr, "orig_len %ld, max_len %d, leaves %d\n", orig_len, max_len, n)); /* There are at least 2 and at most 256 leaves of length max_len. * (Pack arbitrarily rejects empty files and files consisting of * a single byte even repeated.) To fit the last leaf count in a * byte, it is offset by 2. However, the last literal is the EOB * code, and is not transmitted explicitly in the tree, so we must * adjust here by one only. */ leaves[max_len]++; /* Now read the leaves themselves */ base = 0; for (len = 1; len <= max_len; len++) { /* Remember where the literals of this length start in literal[] : */ lit_base[len] = base; /* And read the literals: */ - for (n = leaves[len]; n > 0; n--) { + for (n = leaves[len]; n > 0 && base < LITERALS; n--) { literal[base++] = (uch)get_byte(); } } leaves[max_len]++; /* Now include the EOB code in the Huffman tree */ } /* =========================================================================== * Build the Huffman tree and the prefix table. */ local void build_tree() { int nodes = 0; /* number of nodes (parents+leaves) at current bit length */ int len; /* current bit length */ uch *prefixp; /* pointer in prefix_len */ for (len = max_len; len >= 1; len--) { /* The number of parent nodes at this level is half the total * number of nodes at parent level: */ nodes >>= 1; parents[len] = nodes; /* Update lit_base by the appropriate bias to skip the parent nodes * (which are not represented in the literal array): */ lit_base[len] -= nodes; /* Restore nodes to be parents+leaves: */ nodes += leaves[len]; } /* Construct the prefix table, from shortest leaves to longest ones. * The shortest code is all ones, so we start at the end of the table. */ peek_bits = MIN(max_len, MAX_PEEK); prefixp = &prefix_len[1< prefix_len) *--prefixp = (uch)len; } /* The length of all other codes is unknown: */ while (prefixp > prefix_len) *--prefixp = 0; } /* =========================================================================== * Unpack in to out. This routine does not support the old pack format * with magic header \037\037. * * IN assertions: the buffer inbuf contains already the beginning of * the compressed data, from offsets inptr to insize-1 included. * The magic header has already been checked. The output buffer is cleared. */ int unpack(in, out) int in, out; /* input and output file descriptors */ { int len; /* Bit length of current code */ unsigned eob; /* End Of Block code */ register unsigned peek; /* lookahead bits */ unsigned peek_mask; /* Mask for peek_bits bits */ ifd = in; ofd = out; read_tree(); /* Read the Huffman tree */ build_tree(); /* Build the prefix table */ clear_bitbuf(); /* Initialize bit input */ peek_mask = (1< 0) { peek >>= peek_bits - len; /* discard the extra bits */ } else { /* Code of more than peek_bits bits, we must traverse the tree */ ulg mask = peek_mask; len = peek_bits; do { len++, mask = (mask<<1)+1; look_bits(peek, len, mask); } while (peek < (unsigned)parents[len]); /* loop as long as peek is a parent node */ } /* At this point, peek is the next complete code, of len bits */ if (peek == eob && len == max_len) break; /* end of file? */ put_ubyte(literal[peek+lit_base[len]]); Tracev((stderr,"%02d %04x %c\n", len, peek, literal[peek+lit_base[len]])); skip_bits(len); } /* for (;;) */ flush_window(); Trace((stderr, "bytes_out %ld\n", bytes_out)); if (orig_len != (ulg)bytes_out) { error("invalid compressed data--length error"); } return OK; } Index: releng/5.3/sys/conf/newvers.sh =================================================================== --- releng/5.3/sys/conf/newvers.sh (revision 162446) +++ releng/5.3/sys/conf/newvers.sh (revision 162447) @@ -1,97 +1,97 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="5.3" -BRANCH="RELEASE-p33" +BRANCH="RELEASE-p34" RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ $(dirname $0)/../sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/1992-$year/" \ -e 's/\[your name here\]\.* /The FreeBSD Project./' \ -e 's/\[your name\]\.*/The FreeBSD Project./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/* * Copyright (c) 1992-$year The FreeBSD Project. * All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER-root} d=`pwd` h=${HOSTNAME-`hostname`} t=`date` i=`${MAKE:-make} -V KERN_IDENT` cat << EOF > vers.c $COPYRIGHT char sccspad[32 - 4 /* sizeof(sccs) */] = { '\\0' }; char sccs[4] = { '@', '(', '#', ')' }; char version[] = "${VERSION} #${v}: ${t}\\n ${u}@${h}:${d}\\n"; char ostype[] = "${TYPE}"; char osrelease[] = "${RELEASE}"; int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF echo `expr ${v} + 1` > version Index: releng/5.4/UPDATING =================================================================== --- releng/5.4/UPDATING (revision 162446) +++ releng/5.4/UPDATING (revision 162447) @@ -1,2099 +1,2102 @@ Updating Information for FreeBSD stable users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. Important recent entries: 20040724 (default X changes). +20060919: p19 FreeBSD-SA-06:21.gzip + Correct multiple vulnerabilities in gzip(1). + 20060906: p18 FreeBSD-SA-06:19.openssl, FreeBSD-SA-06:20.bind Correct incorrect PKCS#1 v1.5 padding validation in crypto(3). [06:19] Correct multiple denial-of-service vulnerabilities in BIND related to SIG Query Processing and Excessive Recursive Queries. [06:20] 20060823: p17 FreeBSD-SA-06:18.ppp Correct buffer overflow in the handling of LCP options in ppp(4). 20060614: p16 FreeBSD-SA-06:17.sendmail Correct a bug in the handling of multipart messages by sendmail(8) which can allow a malformed message to crash a sendmail queue processing process. 20060531: p15 FreeBSD-SA-06:15.ypserv, FreeBSD-SA-06:16.smbfs Enable inadvertantly disabled "securenet" access controls in ypserv. [06:15] Correct a bug in the handling of backslash characters in smbfs which can allow an attacker to escape from a chroot(2). [06:16] 20060419: p14 FreeBSD-SA-06:14.fpu Correct a local information leakage bug affecting AMD FPUs. 20060322: p13 FreeBSD-SA-06:11.ipsec, FreeBSD-SA-06:12.opie, FreeBSD-SA-06:13.sendmail Add missing code needed for the detection of IPSec packet replays. [06:11] Correctly identify the user running opiepasswd(1) when the login name differs from the account name. [06:12] Modify timeout handling logic in sendmail(8) to correct a reported signal handling race condition. [06:13] 20060301: p12 FreeBSD-SA-06:09.openssh, FreeBSD-SA-06:10.nfs Correct a remote DoS in OpenSSH when using PAM and privilege separation. [06:09] Correct a remote kernel panic when processing zero-length RPC records via TCP. [06:10] 20060201: p11 FreeBSD-SA-06:08.sack Avoid an infinite loop in sack scoreboard processing which can result from memory exhaustion. 20060125: p10 FreeBSD-SA-06:07.pf Correct an error in pf handling of IP packet fragments which could result in a kernel panic. 20060111: p9 FreeBSD-SA-06:01.texindex, FreeBSD-SA-06:02.ee, FreeBSD-SA-06:03.cpio Correct insecure temporary file usage in texindex. [06:01] Correct insecure temporary file usage in ee. [06:02] Correct a race condition when setting file permissions, sanitize file names by default, and fix a buffer overflow when handling files larger than 4GB in cpio. [06:03] 20051011: p8 FreeBSD-SA-05:21.openssl Correct a man-in-the-middle SSL version rollback vulnerability. 20050907: p7 FreeBSD-SA-05:20.cvsbug Correct insecure temporary file usage. 20050727: p6 FreeBSD-SA-05:18.zlib, FreeBSD-SA-05:19.ipsec Correct buffer overflow when handling certain deflated data streams. Correct errors in the implementation of the AES-XCBC-MAC IPsec authentication algorithm. 20050720: p5 FreeBSD-SA-05:17.devfs Correct devfs ruleset bypass. 20050706: p4 FreeBSD-SA-05:16.zlib Correct buffer overflow when handling corrupt deflated data streams. 20050629: p3 FreeBSD-SA-05:13.ipfw, FreeBSD-SA-05:14.bzip2, FreeBSD-SA-05:15.tcp Correct ipfw packet matching errors with address tables. Correct bzip2 denial of service and permission race vulnerabilities. Correct TCP connection stall denial of service vulnerabilities. 20050608: p2 FreeBSD-SA-05:10.tcpdump, FreeBSD-SA-05:11.gzip Correct several denial-of-service vulnerabilities in tcpdump. Correct directory traversal and race condition vulnerabilities in gzip. 20050513: p1 FreeBSD-SA-05:09.htt Add a knob for disabling/enabling HTT. Default off due to information disclosure on multi-user systems. 20050509: FreeBSD 5.4-RELEASE. 20050507: FreeBSD-SA-05:08.kmem Correct two issues which missed from the earlier commit. 20050506: FreeBSD-SA-05:08.kmem Correct several local kernel memory disclosure bugs. 20050506: FreeBSD-SA-05:07.ldt Correctly validate inputs to the i386_get_ldt syscall. 20050506: FreeBSD-SA-05:06.iir Correct overly liberal permissions on /dev/iir. 20050422: FreeBSD-SA-05:05.cvs Correct several vulnerabilities in CVS. 20050414: FreeBSD-SA-05:04.ifconf Zero a buffer in ifconf() in order to avoid accidental disclosure of kernel memory to userland. 20050406: FreeBSD-SA-05:03.amd64 Correctly initialize the I/O permission bitmap on amd64. 20050404: FreeBSD-SA-05:02.sendfile Correct kernel memory disclosure bug in sendfile(2). 20050302: The full packet destination manipulation for 'ipfw fwd' is back with the kernel compile time option: options IPFIREWALL_FORWARD_EXTENDED This option has to be specified in addition to IPFIREWALL_FORWARD. With this option even packets targeted for an IP address local to the host can be redirected. All restrictions to ensure proper behaviour for locally generated packets are turned off. Firewall rules have to be carefully crafted to make sure that things like PMTU discovery do not break. 20050228: The responsibility of recomputing the file system summary of a SoftUpdates-enabled dirty volume has been transferred to the background fsck. A rebuild of fsck(8) utility is recommended if you have updated the kernel. To get the old behavior (recompute file system summary at mount time), you can set vfs.ffs.compute_summary_at_mount=1 before mounting the new volume. 20050228: The ifi_epoch member of struct if_data has been changed to contain the uptime at which the interface was created or the statistics zeroed rather then the wall clock time because wallclock time may go backwards. This should have no impact unless an snmp implementation is using this value (I know of none at this point.) 20050227: The default "world" build no longer supports running on an 80386 CPU. In order to build a world for an 80386 CPU, one needs to set CPUTYPE=i386 in /etc/make.conf. 20050225: The cpufreq framework has been merged. As part of this, the sysctls for acpi(4) throttling have been removed. See cpufreq(4) for the new sysctl interface. The power_profile script has also been updated, so you can use performance/economy_cpu_freq in rc.conf(5) to set AC on/offline cpu frequencies. No new cpufreq drivers have been brought in with this import but drivers from -current can now be built and run on -stable. 20050215: [applicable to sparc64 only] uart(4) is the default serial console now. When using the GENERIC kernel, make sure to start getty(8) processes on ttyu0 and/or ttyu1. Either run mergemaster(8) or edit /etc/ttys manually. 20050203: ppp(8) no longer reverts to sending ECHO requests if LQR is enabled but fails negotiation. To enable the old behaviour, add ``enable echo'' to your config. Additionally, when RADIUS is enabled, ppp now sends both NAS-IP-Address and NAS-Identifier by default. This behaviour can be modified with the ``enable/disable NAS-IP-Address/NAS-Identifier'' command. 20041126: The numbers of arguments for ng_timeout() and ng_untimeout() have changed. If netgraph.ko is rebuilt or if netgraph is statically compiled in kernel, then the following modules must also be rebuilt: ng_hci ng_l2cap ng_source ng_sscop ng_uni 20041024: FreeBSD 5.3-RELEASE 20041023: GENERIC kernel config files for amd64 and i386 no longer have "options SMP". That has been moved to a new config file named SMP for users who want an easy way to build an SMP kernel. 20041010: The FreeBSD keyword is no longer a requirement for a valid rc.d script. The rc(8) and rc.shutdown(8) scripts no longer check for the existence of this keyword when ordering rc.d scripts. This change touches most of the file in /etc/rc.d; therefore, if you have not modified any files in that directory it may be easier to rm -rf /etc/rc.d and then use the -i switch with mergemaster(8). 20041009: One of the syscalls the 1:1 threading library libthr uses has changed, thus breaking ABI compatibility. Make sure you rebuild this library with the kernel. 20041003: The pfil API has gained an additional argument to pass an inpcb. You should rebuild all pfil consuming modules: ipfw, ipfilter and pf. 20041001: The following libraries had their version number bumped up: /lib/libm.so.2 -> libm.so.3 /lib/libreadline.so.4 -> libreadline.so.5 /usr/lib/libhistory.so.4 -> libhistory.so.5 /usr/lib/libopie.so.2 -> libopie.so.3 /usr/lib/libpcap.so.2 -> libpcap.so.3 FreeBSD 4.10 versions of these libraries will be added to the compat4x collection. If you expect to be able to run old 4.X executables you will need to remove the old versions of these libraries. However note that any 5.X executables you have built will stop working once you remove those old libraries. You should have all your ports/packages rebuilt before removing the old libraries. 20040928: If enabled, the default is now to run named in a chroot "sandbox." For users with existing configurations in /etc/namedb the migration should be simple. Upgrade your world as usual, then after installworld but before mergemaster do the following: If named is running: /etc/rc.d/named stop cd /etc mv namedb namedb.bak mkdir -p /var/named/etc/namedb cp -Rp namedb.bak/* /var/named/etc/namedb/ mergemaster (with your usual options) If using the generated localhost* files: cd /var/named/etc/namedb /bin/sh make-localhost rm -f localhost-v6.rev localhost.rev /etc/rc.d/syslogd restart /etc/rc.d/named start If you are using a custom configuration, or if you have customised the named_* variables in /etc/rc.conf[.local] then you may have to adjust the instructions accordingly. It is suggested that you carefully examine the new named variables in /etc/defaults/rc.conf and the options in /var/named/etc/namedb/named.conf to see if they might now be more suitable. 20040925: BIND 9 has been imported into the base, and is now fully functional. BIND 8 has now been removed. There are numerous differences between BIND 8 and 9, and users with critical named installations should read the migration documentation in /usr/share/doc/bind9/misc/migration. There is also a new instruction manual in /usr/share/doc/bind9/arm. The key differences that most users will experience are in how picky BIND 9 is about zone file format. If you are using named as a resolving (caching) name server, you will likely not have trouble. The following files are part of the old BIND 8 installation and should be removed: /usr/bin/dnskeygen /usr/bin/dnsquery /usr/libexec/named-xfer /usr/sbin/named.restart /usr/sbin/ndc The following files have moved to /usr/bin, and should be removed from their old locations: /usr/sbin/nslookup /usr/sbin/nsupdate 20040922: PFIL_HOOKS are a fixed part of the network stack now and do not need to be specified in the kernel configuration file anymore. Remove 'options PFIL_HOOKS' from your kernel config file. 20040914: The format of the pflogd(8) logfile "/var/log/pflog" has changed for architectures that have a 64 bit long type to make it compatible to the standard pcap format. In order to prevent corruption move away any old logfile before using a new pflogd(8). 20040913: debug.witness_* has been renamed to debug.witness.*. There are compatibility tunables left in for a few days. Update loader.conf as necessary. 20040907: The kernel and userland debugging options have been turned off by default. 20040902: The default configuration for the network stack has been changed such that it now runs without the Giant lock unless configured otherwise. If you experience network-related instability, you may wish to try setting "debug.mpsafenet=0" or compiling the kernel with "options NET_WITH_GIANT". Details on the netperf project may be found at: http://www.watson.org/~robert/freebsd/netperf/ Including the 20040828 announcement of configuration change details. 20040826: Netgraph changed its message format slightly to align the data portion well on 64 bit machines. Netgraph using utilities (e.g. ngctl, nghook, ppp, mpd, pppoed, bluetooth, ATM) should be recompiled when a new kernel is installed. 20040817: IPFW has been converted to use PFIL_HOOKS. This change is transparent to userland and preserves the ipfw ABI. The ipfw core packet inspection and filtering functions have not been changed, only how ipfw is invoked is different. Note that "option PFIL_HOOKS" is required to use IPFIREWALL compiled into the kernel or as KLD. 20040814: The RANDOM_IP_ID option has been replaced by the sysctl net.inet.ip.random_id. If you had RANDOM_IP_ID in your kernel then you may want to add "net.inet.ip.random_id=1" to /etc/sysctl.conf. 20040807: The size of 'struct ifnet' has changed due to the addition of the if_carp placeholder. All kernel modules implementing network interfaces must be recompiled as a result. 20040806: Module loading has been fixed. Some older installations will drop proper module_path initialization and modules will fail to load properly. If you have a line in /boot/loader.rc that says: "initialize drop", do (i386 only): cp /usr/src/sys/boot/i386/loader/loader.rc /boot/loader.rc chown root:wheel /boot/loader.rc chmod 444 /boot/loader.rc 20040802: making /dev/(null|zero) into a module proved to be too unpopular, so this bit has been revoked from the previous (20040801) entry. 20040801: The /dev/mem, /dev/io /dev/(null/zero) devices are now modules, so you may wish to add them to your kernel config file. See GENERIC for examples. 20040728: System compiler has been upgraded to GCC 3.4.2-pre. As with any major compiler upgrade, there are several issues to be aware of. GCC 3.4.x has broken C++ ABI compatibility with previous releases yet again and users will have to rebuild all their C++ programs with the new compiler. A new unit-at-a-time optimization mode, which is default in this compiler release, is more aggressive in removing unused static symbols. This is the likely cause of 'make buildworld' breakages with non-default CFLAGS where optimization level is set to -O2 or higher. With the upgrade of the system compiler, the kernel has been upgraded to match the new system compiler. This makes it impossible to build a new kernel with the old compiler. Upgrade your system via make buildworld and make kernel (see below) to fix this problem. 20040727: The size of 'struct ifnet' has changed due to the addition of the IFF_NEEDSGIANT flag (and what it implies). All kernel modules implementing network interfaces must be recompiled as a result. 20040716: The sound device drivers are renamed. `sound' is always required, while `snd_*' should be configured accordingly to your hardware. Refer to NOTES for the detail of the drivers. 20040710: __FreeBSD_version bumped to 502122. 20040710: The console initialization on Alpha has been reworked and is now identical to other platforms. This means that the hardcoding of the serial console and the debug port has been removed. As such, hints are now required for the sio(4) driver to become a console or debug port. The NO_SIO option has been decommissioned because of this. 20040710: A revamp of the debugging code in the kernel with some visible changes beyond just the debugging experience: o The DDB option is now specific to the DDB debugger backend and should not be used any more for conditional compilation of debugging code for when debugging is enabled. Use the KDB option for this. o The WITNESS_DDB, DDB_TRACE and DDB_UNATTENDED options have been renamed to WITNESS_KDB, KDB_TRACE and KDB_UNATTENDED respectively. This is in line with the first bullet. o The remote GDB support has been untangled from DDB and needs to be enabled separately now. Use the GDB option for this. o The GDB_REMOTE_CHAT option has been removed. Support for this homegrown feature is discontinued. The GDB remote protocol supports console output and it makes sense to use that. o The DDB_NOKLDSYM option has been removed. The DDB debugger now supports both direct symbol table lookups as well as KLD symbol lookups through the linker. 20040708: Bluetooth code has been marked as non-i386 specific. __FreeBSD_version has been bumped to 502121 to mark this change. 20040702: The native preemption has been added to the kernel scheduler. There is some report that the ULE scheduler was broken in some machines and we encourage users using the ULE scheduler either stick with a known good kernel, or temporarily switch to the 4BSD scheduler as a workaround. 20040630: The netgraph ABI version number has been incremented to indicate an incompatible change in the ABI. Old netgraph nodes will refuse to attach until recompiled. Netgraph now uses mbuf tags to move metadata and this commit removes its home-grown metadata facility. Nodes should just recompile, unless they use metadata, in which case the changes are simple; the file ng_ksocket.c serves as an example of such changes. This also broke i4b, although the compile problem has been papered over. 20040630: ACPI has been updated to disable known-bad BIOS revisions. A message will be printed on the console indicating that ACPI has been disabled automatically and that the user should use a newer BIOS, if possible. If you think ACPI does work on your system and want to override this (i.e., for testing), set hint.acpi.0.disabled="0" at the loader prompt. 20040623: pf was updated to OpenBSD-stable 3.5 and pflogd(8) is privilege separated now. It uses the newly created "_pflogd" user/group combination. If you plan to use pflogd(8) make sure to run mergemaster -p or install the "_pflogd" user and group manually. 20040622: Network interface cloning has been overhauled. This change will require a recompile of modules using cloning and modification of external ones to the new API. __FreeBSD_version has been bumped to 502119 to mark this change. Additionally, users creating stf(4) interfaces via "ifconfig stf" will need to update their scripts as this will create an interface named "stf" instead of "stf0" and ifconfig will not print "stf0" to stdout. 20040621: On 20040524, the /etc/rc.d/nsswitch script was modified to automatically create /etc/nsswitch.conf on startup if it did not already exist. Unfortunately, an error in the man page was carried over to the script, resulting in incorrect nsswitch settings. The simplest remedy is to remove both /etc/nsswitch.conf and /etc/host.conf; they will be recreated during the next reboot. 20040614: The return value of sema_timedwait(9) has been changed to make it consistent with cv_timedwait(9). Be sure to recompile the ips module and any third-party modules which call sema_timedwait. 20040613: ALTQ is now linked to the build. This breaks ABI for struct ifnet. Make sure to recompile modules and any userland that makes use of sizeof(struct ifnet). In order to get the altq headers in place please recompile and reinstall world. 20040607: Splitting kern_thread.c into 2 files (adding kern_kse.c) requires that you re-run config after updating your tree. 20040601: The MIDI drivers have been removed. Until the new module-friendly ones are merged, remove or comment out midi and seq from your kernel configuration. 20040423: Due to a new option in ipfw (versrcreach) the ipfw(8) command needs to be recompiled. Normal accept/reject rules without options are not affected but those with options may break until ipfw(8) is recompiled. 20040420: Due to changes in the callout ABI, kernels compiled after this date may be incompatible with kernel modules compiled prior to 20040406. 20040414: The PCI bus power state stuff has been turned on. If this causes problems for your system, please disable it using the tunable hw.pci.do_powerstate=0. 20040412: The bulk of the pci problems have been fixed, although the floppy drive is still broken. 20040410: A substantial update to the pci bus resource and power management have been committed. Expect a bumpy ride for a few days until the unanticipated problems have been resolved. 20040409: Due to changes in the the Yarrow initialization process, /dev/random needs to be fed before operations requiring temp files can succeed in single user mode. This includes running "make installworld". /dev/random may be fed by running "/etc/rc.d/initrandom start" or with 20040415 source by running "/etc/rc.d/preseedrandom". 20040322: The debug.mpsafenet tunable controls whether the kernel Giant lock is held across the lower levels of the network stack, and by default is turned off. In the few days following 20040322, the behavior of debug.mpsafenet will change such that this tunable controls Giant over all levels of the network stack. If you are currently setting debug.mpsafenet to 1, you should set it back to 0 (the default) again during the change-over. An additional note will be added to UPDATING when sufficient locking is merged to permit this to take place. 20040310: The FreeBSD/sparc64 platform is changing time_t from 32-bits to 64-bits. This is a very major incompatible change, so people using FreeBSD/sparc64 *must* read the UPDATING.64BTT file for detailed instructions on how to make this upgrade. People upgrading FreeBSD on other platforms can ignore this event. 20040308: The packet filter (pf) is now installed with the base system. Make sure to run mergemaster -p before installworld to create required user account ("proxy"). If you do not want to build pf with your system you can use the NO_PF knob in make.conf. Also note that pf requires "options PFIL_HOOKS" in the kernel. The pf system consists of the following three devices: device pf # required device pflog # optional device pfsync # optional 20040303: If you are having trouble with the libc_r -> libpthread transition (see the 20040130 entry), place the following lines at the top of /etc/libmap.conf: libc_r.so.5 libpthread.so.1 libc_r.so libpthread.so This will cause all programs and libraries linked against libc_r to use libpthread instead. 20040226: Some sshd configuration defaults have changed: protocol version 1 is no longer enabled by default, and password authentication is disabled by default if PAM is enabled (which it is by default). OpenSSH clients should not be affected by this; other clients may have to be reconfigured, upgraded or replaced. 20040225: The ABIs defined in and have been updated to support improved reentrancy. Multi-threaded programs that reference the "_res" or "h_errno" symbols may experience some problems if they are not recompiled. Single-threaded programs should remain unaffected. 20040225: routed has been updated in the base system from the vendor sources, routed v2.27, from rhyolite.com. This change means that for users who use RIP's MD5 authentication feature, FreeBSD -CURRENT's routed is now incompatible with previous versions of FreeBSD; however it is now compatible with implementations from Sun, Cisco and other vendors. 20040224: The tcpcb structure has changed and makes a recompile of libkvm and related userland network utilities necessary. 20040222: The cdevsw structure has changed in two externally visible ways. First, the sense of the D_GIANT flag has changed to D_NEEDSGIANT. Second, the d_version field must be filled in with D_VERSION. Drivers outside the tree will need to be updated. 20040207: The /etc/rc.d/ttys script has been removed. It is no longer necessary since devfs has been mandatory for some time. 20040130: libkse has been renamed back to libpthread and is now the default threads library. The gcc -pthread option has also been changed to link to libpthread instead of libc_r. For alpha and sparc64 machines, libkse is not renamed and links are installed so that libpthread points to libc_r. Until the ports system is updated to handle this change, it is recommended that folks install an /etc/libmap.conf(5) that maps libc_r to libpthread. If you have any binaries or libraries linked to libkse, then it is also recommended that you map libkse to libpthread. Anyone that is using nvidia supplied drivers and libraries should use a libmap.conf that maps libpthread to libc_r since their drivers/libraries do not work with libpthread. 20040125: ULE has entered into its probationary period as the default scheduler in GENERIC. For the average user, interactivity is reported to be better in many cases. On SMP machines ULE will be able to make more efficient use of the available parallel resources. If you are not running it now, please switch over, replacing the kernel option SCHED_4BSD with SCHED_ULE. 20040125: Move LongRun support out of identcpu.c, where it hardly belongs, into its own file and make it opt-in, not mandatory, depending on CPU_ENABLE_LONGRUN config(8) option. 20031213: src/lib/libc/gen/initgroups.c:1.8 now causes logins to fail if the login process is unable to successfully set the process credentials to include all groups defined for the user. The current kernel limit is 16 groups; administrators may wish to check that users do not have over 16 groups defined, or they will be unable to log in. 20031203: The ACPI module has been reactivated. It is no longer required to compile ACPI support into kernels statically. 20031112: The statfs structure has been updated with 64-bit fields to allow accurate reporting of multi-terabyte filesystem sizes. You should build world, then build and boot the new kernel BEFORE doing a `installworld' as the new kernel will know about binaries using the old statfs structure, but an old kernel will not know about the new system calls that support the new statfs structure. Note that the backwards compatibility is only present when the kernel is configured with the COMPAT_FREEBSD4 option. Since even /bin/sh will not run with a new kernel without said option you're pretty much dead in the water without it. Make sure you have COMPAT_FREEBSD4! Running an old kernel after a `make world' will cause programs such as `df' that do a statfs system call to fail with a bad system call. Marco Wertejuk also reports that cfsd (ports/security/cfs) needs to be recompiled after these changes are installed. ****************************DANGER******************************* DO NOT make installworld after the buildworld w/o building and installing a new kernel FIRST. You will be unable to build a new kernel otherwise on a system with new binaries and an old kernel. 20031112: Some netgraph string length constants have been changed. This change requires the netgraph kernel modules and all netgraph userland components to be in sync. Especially users who require netgraph to boot need to make sure to have world and kernel in sync before rebooting. 20031111: Hyperthreading logical CPU's are no longer probed by default when using the MP Table. If ACPI is being used, then logical CPUs will be probed if hyperthreading is enabled in the BIOS. If ACPI is not being used and hyperthreading is enabled in the BIOS, logical CPUs can be enabled by building a custom kernel with the option MPTABLE_FORCE_HTT enabled. 20031103: The i386 APIC_IO kernel option has been replaced by 'device apic'. The ACPI module has also been temporarily disabled, so ACPI must be statically compiled into your kernel using 'device acpi' if you wish to use the ACPI driver. 20031031: The API and ABI of struct ifnet have been changed by removing the if_name and if_unit members and replacing them with if_xname, if_dname, and if_dunit. All network drivers and most userland programs which include net/if_var.h must be updated and recompiled. __FreeBSD_version has been bumped to 501113 to reflect this change. 20030928: Changes to the cdevsw default functions have been made to remove the need to specify nullopen() and nullclose() explicitly. __FreeBSD_version bumped to 501110. 20030926: kiconv(3) has been added. mount_msdosfs(8), mount_ntfs(8) and mount_cd9660(8) need to be in sync with kernel. 20030925: Configuring a system to use IPFILTER now requires that PFIL_HOOKS also be explicitly configured. Previously this dependency was magically handled through some cruft in net/pfil.h; but that has been removed. Building a kernel with IPFILTER but not PFIL_HOOKS will fail with obtuse errors in ip_fil.c. 20030923: Fix a bug in arplookup(), whereby a hostile party on a locally attached network could exhaust kernel memory, and cause a system panic, by sending a flood of spoofed ARP requests. See FreeBSD-SA-03:14.arp. 20030915: A change to /etc/defaults/rc.conf now causes inetd to be started with `-C 60' if it is not overridden in /etc/rc.conf. This causes inetd to stop accepting connections from an IP address that exceeds the rate of 60 connections per minute. 20030829: The following rc.d scripts have been removed and should be deleted from your installation: atm2.sh atm3.sh devdb localdaemons network1 network2 network3. Depending on when you last updated world and used mergemaster(8) you may or may not have problems during the rc boot sequence. The simplest solution is an 'rm -rf /etc/rc.d/*' and then 'mergemaster -i'. The atm2.sh atm3.sh and devdb scripts were removed some time ago, so depending on when you installed -CURRENT these scripts may or may not exist on your system. 20030824: ATAng has been committed. You need to build world as sys/ata.h has changed, and userland atacontrol depends on it. If you use ATA SW raids you need "device ataraid" in your kernel config file, as it is no longer pulled in automatically. 20030819: The OFW_NEWPCI option has been turned on in the Sparc64 GENERIC kernel. Among other things, this changes the device enumeration to be closer to Solaris. Be aware that, this can even cause the machine to not boot without manual intervention before the fstab is adjusted. 20030728: All current USB and Firewire quirks in da(4) have been deprecated and will be removed for 5.2. If this causes failure for your umass(4) devices, enable "options DA_OLD_QUIRKS" in your kernel and send the output of "camcontrol inquiry da0" to scsi@freebsd.org so the quirk can be re-enabled. 20030724: Problems with entry 20030714 have been corrected and no known issues with /rescue and -j exist for host systems after this point in time. 20030722: FPU-less support has been removed from FreeBSD. Chances are you won't notice. 386+387 support should still work after this change, but it is now a minimum requirement for the i386 port that you have real FPU hardware. 20030714: Some people are having problems with changes related to /rescue. If you are building -j N, you will need to define NO_RESCUE. Others will need to define it if /rescue has issues with their environment. People should report those issues to current@. 20030711: gcc was upgraded to 3.3. You are advised to not build -DNOCLEAN across this point. Further, it might be a good idea to remove /usr/obj. 20030610: Remove deprecated locale names and transition period code for them, finishing switching to the new scheme. Check your LANG environment variable. 20030609: CCD has been changed to be a fully GEOMified class. Kernel and ccdconfig(8) needs to be in sync, this is particularly important to remember beforehand if your source tree is on a ccd device. Consider making a copy of the old ccdconfig into /boot/kernel.good or wherever you keep your backup kernel. 20030605: There was a small window in which sed(1) was broken. If you happen to have sed(1) installed during that window, which is evidenced by an inability to build world with the failure given below, you need to manually build and install sed(1) (and only sed(1)) before doing anything else. This is a one- time snafu. Typical failure mode: In file included from /usr/src/contrib/binutils/bfd/targets.c:1092: targmatch.h:7:1: null character(s) ignored targmatch.h:12:1: null character(s) ignored targmatch.h:16:1: null character(s) ignored : The window of "sed(1)-uction" is from Wed Jun 4 15:31:55 2003 UTC to Thu Jun 5 12:10:19 2003 UTC (from rev 1.30 to rev 1.31 of usr.bin/sed/process.c). 20030505: Kerberos 5 (Heimdal) is now built by default. Setting MAKE_KERBEROS5 no longer has any effect. If you do NOT want the "base" Kerberos 5, you need to set NO_KERBEROS. 20030502: groff has been updated. If you try to do a buildworld and get an infinite loop in troff, update to May 4th or newer. If you have a newer kernel than userland, you may need to set the OSRELDATE to 500110 in your environment before starting a buildworld. 20030501: The old rc system has been removed. Please report any problems to freebsd-rc@yahoogroups.com, and/or freebsd-current@freebsd.org. Your personal versions of these files will not be removed, so you can continue to use them. However, you should take great care when updating, especially when using mergemaster, since the compatibility code that utilizes these old scripts has also been removed. 20030423: A bug has been fixed in /dev/devctl which would cause devd to hang on boot, were it not for a workaround in devd. The work around in devd will be removed around 20030507. You have until then to upgrade your kernel before updating userland. In general, you should have a userland and kernel that's in sync with each other. However, given the effects of this bug (hang on boot when starting devd), some allowances are made. 20030329: Alphas with libc from between 20030312 and 20030329 exhibit floating point exceptions (FPEs), most notably in awk(1) while upgrading the system through a buildworld. So, to successfully upgrade your Alpha, you must either downgrade your libc.so to a pre-20030312 version, or update /usr/share/mk/bsd.cpu.mk to revision 1.26 which adds -mieee to CFLAGS, then forcibly rebuild and install libc: cd /usr/src/lib/libc && \ make cleandir && make obj && \ make -DNOMAN -DNOPROFILE all && \ make -DNOMAN -DNOPROFILE install 20030208: sendmail 8.12.7 has been imported. It has one important change for IPv6 users. The default submit.mc now uses '[127.0.0.1]' instead of 'localhost' meaning only IPv4 is used to connect to the MTA. Users on IPv6-only machines will need to edit /etc/mail/submit.mc appropriately. 20030128: NODEVFS option has been removed and DEVFS thereby made standard. This makes all references to MAKEDEV obsolete, and they should be removed when convenient. 20030126: The name of the device for the ofw console has changed, sparc64 users must run mergemaster to update their installed /etc/ttys. 20030125: The scheduler framework has grown a second scheduler and consequently you must specify one and only one scheduler in your kernel config. The cvs config files have been updated to use the old scheduler which may be selected via 'options SCHED_4BSD'. If you would like to try the new, much more experimental, scheduler please try 'options SCHED_ULE' and contribute to the arch@ discussion. 20030115: A new version of the wi driver has been imported into the tree. One now must have device wlan in the config file for it to operate properly. In addition, there have been some changes to how wi devices are configured for point to point links to bring it more in line with the former way of doing things, as well as compatibility with NetBSD. 20021222: For a period after the GCC 3.2.1 import (from 12/04 to 12/22), GCC used an incompatible form of ABI for returning structures and unions which FreeBSD's GCC maintainers were not aware of relative to previous versions of FreeBSD. We have gone back to the ABI for now, and any code compiled which is required to interoperate with other code (not built at the same time) returning structs or unions should be rebuilt. 20021216: A name change in /etc/netconfig has been reverted to stay compatible with suns TIRPC and also with NetBSD. You need to run mergemaster after make world. A new libc does still work with an outdated /etc/netconfig for some time, but you'll get a warning. This warning will be removed in 20030301. 20021202: The recent binutils upgrade marks a kernel flag day on sparc64: modules built with the old binutils will not work with new kernels and vice versa. Mismatches will result in panics. Make sure your kernel and modules are in sync. 20021029: The value of IPPROTO_DIVERT has changed. Make sure to keep your kernel, netstat, natd and any third-party DIVERT consumers in sync. 20021024: Old, compatibility slices have been removed in GEOM kernels. This means that you will have to update your /etc/fstab to not use disk devices of the form /dev/ad0a. Instead, you now must specify /dev/ad0s1a, or whatever slice your FreeBSD partition really is on. The old device names have gone away, so if you use them anywhere else, you must also adjust those uses. (This doesn't affect the disks formatted in the ``dangerously-dedicated'' mode.) 20021023: Alphas with kernels from between 20020830 and 20021023 and/or rtld (ld-elf.so.1) older than 20021023 may experience problems with groff while doing a buildworld (kernel: "out of memory", fixed in rev 1.129 of kern/imgact_elf.c; rtld: "too few PT_LOAD segments", fixed in rev 1.8 of libexec/rtld-elf/map_object.c). So, to successfully upgrade your Alpha, you must either upgrade your kernel and rtld first (which might be a bit tricky), or avoid running the bootstrapped groff during the "transitional" buildworld. To avoid running groff during the transitional upgrade run make buildworld with -DNOMAN, -DNO_SHAREDOCS, and -DNO_LPR. 20020831: gcc has been upgraded to 3.2. It is not all binary compatible with earlier versions of gcc for c++ programs. All c++ programs and libraries need to be recompiled. Also, if you encounter g++ issues, rm /usr/include/g++/* before doing an installworld to make sure that stale files are removed. 20020827: Our /etc/termcap now has all the entries from the XFree86 xterm almost unchanged. This means xterm now supports color by default. If you used TERM=xterm-color in the past you now should use TERM=xterm. (xterm-color will lead to benign warnings). 20020815: A "bug" in gcc(1) that was hiding warning in system headers was fixed. It's probably time to add -DNO_WERROR to your make line again. 20020729: COPY is being deprecated. The 20010530 change was reverted, as it causes far more pain than was expected, and to always compare before installing, please use INSTALL="install -C" again. The -C option is now silently ignored when used with the -d option. 20020702: Problems with libc_r clients like KDE and GNOME have been resolved. There are still some minor problems with some signals but the system is stable enough for general use again. SMP is less so than UP but each can successfully complete multiple buildworlds. Libkvm needs to be recompiled due to KSE. 20020701: Now would be a bad time to upgrade. Something in or near the KSE commit totally broke programs using libc_r like KDE and GNOME. 20020511: The k5su utility installed as part of Kerberos 5 is no longer installed with the set-user-ID bit set by default. Add ENABLE_SUID_K5SU=yes to /etc/make.conf to have it installed with the set-user-ID bit set. 20020510: Gcc 3.1 debugging format (cc -g) has changed from STABS to DWARF2. Unfortunately our native GDB (at version 4.18) does not understand the DWARF2 debugging format. Thus you must use `gcc -gstabs+' to generated debugging information for our native GDB. 20020510: Due to the way CVS works, it may not properly update src/contrib/gcc to the 3.1 sources. The easiest fix is to `rm -rf' src/contrib/gcc and then do a cvs update. 20020421: When exec'ing set[ug]id executables, the kernel now ensures that the stdio file descriptors (0..2) are open. See FreeBSD-SA-02:23.stdio. 20020404: New sendmail startup scripts have been installed to make it easier to use alternative MTAs with FreeBSD. Setting the rc.conf variable sendmail_enable to "NO" no longer prevents any sendmail daemons from starting. Instead, either set sendmail_enable to "NONE" or change mta_start_script to a script for starting an alternative MTA. Setting mta_start_script to "" will also prevent any MTA from being started at boot. 20020403: UCONSOLE is no longer a valid kernel option. 20020315: FreeBSD 5.0 DP-1 was basically branched today. 20020225: Warnings are now errors in the kernel. Unless you are a developer, you should add -DNO_WERROR to your make line. 20020217: sendmail 8.12.2 has been imported. The sendmail binary is no longer a set-user-ID root binary and the infrastructure to support command line mail submission has changed. Be sure to run mergemaster (especially for updating /etc/rc, /etc/defaults/rc.conf, and /etc/mail) and read /etc/mail/README for more details. Due to the import of sendmail 8.12.2, a new user and group are required in order for sendmail to run as a set-group-ID binary. A 'make installworld' will use the new user and group to set the owner and group of /var/spool/clientmqueue and will fail if the new user and group do not exist. The 'smmsp' user and group must be merged from src/etc/group and src/etc/master.passwd before using 'make installworld'. 'mergemaster -p' will do this. You may need to install mergemaster before this will work if you are updating from a very old version of current. The updating recipe has changed as of this date. 20020112: The preferred configuration method for PAM is now /etc/pam.d/ rather than /etc/pam.conf. If you have an unmodified pam.conf, just delete it after your next mergemaster run. If you have local modifications, you can use /usr/src/etc/pam.d/convert.pl to incorporate them into your /etc/pam.d. Please see the following url for more details: http://www.freebsd.org/cgi/mid.cgi?db=mid&id= 20011229: If anyone here is already using the new rc.conf(5) variable networkfs_types, please note that it has changed http://www.freebsd.org/cgi/mid.cgi?db=mid&id=<9744.1009655556@axl.seasidesoftware.co.za> 20011220: sys/i4b/driver/i4b_ispppsubr.c has been retired. This file started out its life in the ISDN4BSD project as an offspring from sys/net/if_spppsubr.c, which eventually got a life of its own. All the accumulated features and bug fixes of the i4b version have now been merged back into the base system's version now. The only user-visible change resulting from this is that i4b's sppp(4) interfaces are to be managed with spppcontrol(8) again, since ispppcontrol(8) has been retired as well. (There has never been rc file support for ispppcontrol in -current, but only in -stable. That will be reverted by the time the changes are MFCed.) 20011215: The fdc(4) driver has been updated and now automatically recognizes media in `standard' formats (like 1440 KB and 720 KB for a 3.5" high-density drive) when accessing the default device node (e. g. /dev/fd0). The old variety of floppy device nodes /dev/fd*.* is no longer present by default, devices can be created (in DEVFS) on demand. They will need to be customized then for `odd' densities using fdcontrol(8). 20011209: The bugs in procfs' debugging support code have been fixed, and truss(1) now works again. 20011207: Daily security checks have been split out to use the periodic(8) scripts. Some change in configuration may be necessary. Please see http://www.freebsd.org/cgi/mid.cgi?db=mid&id=<20011207155805.R8975@blossom.cjclark.org> for details. 20011204: sos added VCD/SVCD support to ata driver and that needs the kernel and burncd to be in sync. 20011203: The procfs pseudo-filesystem has now been converted to use the pseudofs framework. If you have 'options PROCFS' in your kernel config, you'll need to add 'options PSEUDOFS' if it's not there already. This change temporarily breaks truss(1); use ktrace(1) instead until the issue has been resolved. 20011202: A security hole in OpenSSH involving `UseLogin yes' has been patched. 20011126: You need to remove /usr/obj/.../usr.bin/tip before rebuilding after this date. You need to do this only once. 20011103: Most of the awk issues have been resolved. Some rough edges may be left, but for the most part things should be back to "normal." For CURRENT's usual definition of "normal." 20011030: Awk has been upgraded to the one true awk from bell labs. Expect choppy waves in the upgrade process. 20011030: The asr driver problem has been resolved. 20011027: Due to changes in other parts of the system, the asr driver now causes the system to panic on boot. Do not use it pending correction. Comment it out of any kernel config file that you try to use from this date forward. 20011025: When crossbuilding, use TARGET=xxx where you used to use MACHINE=xxx. You don't need to set TARGET_ARCH and TARGET, unless you are changing both of them. To cross build pc98 on an alpha, for example, you need to set TARGET=pc98 and TARGET_ARCH=i386. 20011001: The kernel interface that burncd depends on has changed. You must recompile both the kernel and userland applications at the same time. 20010929: When crossbuilding, please set TARGET_ARCH rather than MACHINE_ARCH to indicate the target. In the future, one will set TARGET_MACHINE where you set MACHINE now. At the moment, setting MACHINE alone for same MACHINE_ARCH machines works (eg, you can build pc98 on an i386 machine and vice versa). 20010927: Some weird problems result from using ACPI on some machines. To disable ACPI you can add hint.acpi.0.disabled="1" to /boot/loader.conf (or by putting set X=Y at the boot loader "ok" prompt). Alternatively, you can remove it from /boot/kernel/acpi.ko or use the MODULES_OVERRIDE function in your kernel config file and not list acpi in that list. 20010924: The buildworld has been fixed. You may need to install the 4.x compatibility libraries for some old binaries to work. Add COMPAT4X=true to your /etc/make.conf to get them installed on every installworld, or execute the following to get them installed only once: cd src/lib/compat/compat4x. make all install You will see ``__stdoutp undefined'' until you do this. 20010919: There's a bug in the world build process. The cross-tools are build with the NEW headers, but the OLD libc.a. This leads to all kinds of problems with the new libc. A temporary workaround is to add CFLAGS="-O -pipe -D_OLD_STDIO" before building world when upgrading from 4.x to current. This can be removed afterwards. A proper fix to the buildworld target is needed. 20010918: Peter has committed his new kthread nfs client/server code. NFS may be unstable after this date. 20010912: KSE has hit the tree. Lots of things are now different in the kernel. While a few problems were introduced in the initial commit, most of the major ones have been found and corrected. 20010901: In OLDCARD, CardBus bridges appear to be stable. The work arounds described in the 20010604 entry are now no longer necessary and will be ignored. Most insert/remove problems have been rectified around this date. 20010823: named now runs as user bind and group bind rather than as root. If named_enable is set to YES in /etc/rc.conf, ensure that user bind is available in /etc/passwd (using vipw(8)) and that group bind is available in /etc/group. Also make sure that user or group bind has read (and not write) permission for your name server configuration and that it has read and write permission for your slave zone files and directory. If you wish to continue to run named as root (a less secure alternative), add a line to /etc/rc.conf saying named_flags= 20010709: The PAM libraries have had an API upgrade that is beyond the ability of the shared library major number to handle. It is manifested by PAM-using ports dumping core. The solution is to rebuild those ports. 20010628: The kernel compile module has moved from src/sys/compile/FOO to src/sys/${MACHINE}/compile/FOO. 20010625: The pccard modem issue from 20010613 has been corrected. OLDCARD support is still a little weak in -current. slot 1 is known not to work on some TI based cardbus bridges. Some cardbus bridges do not properly detect insert/removal events. IRQ configuration needs more safety belts. 20010617: Softupdates problems have been corrected. 20010614: Peter ripped out the linkerset support. You must, as always, rerun config after you cvsup if you are using the traditional kernel building methods. 20010613: pccard modems may not work with current after 20010604 date. Some do, others result in panics. *MAKE*SURE* that you update your config and /etc/rc.conf ala the 20010604 entry, or you will have problems (this issue will be fixed, it just hasn't been yet). 20010613: SOFTUPDATES seem to be broken since the middle of May or so. Do not use them in current. You can disable softupdates on all mounted partitions, or remove SOFTUPDATES the kernel config file. 20010612: After Peter's commits to the hints code, people have been noticing that certain devices are attached (or try to) twice. This is due to having both static hints as well as a /boot/device.hints. To work around this issue, please use only one or the other mechanism until this bug is fixed. Please note that a feature of config is that if you have config file FOO and FOO.hints, it automatically adds FOO.hints to the hints.c file, whether you want it to or not. 20010610: Locale names have changed to match other systems better. 20010604: pccard support for pci cards has been committed. You must change your /etc/pccard.conf irq lines. It must match the irq used by pcic device. Interrupt storms may result if you fail to do this. Interrupt storms look a lot like a hang. You must also install a new pccardd, otherwise you will get an interrupt storm at card reset time (just after it tells you what it is). pccardd_flags="-I" is necessary for the time being. It tells pccardd not to ask the kernel if the interrupt is really free or not before using it. You can either change the /etc/pccard.conf irq lines to match pcic, or add "-i X" to the pccardd_flags. 20010530: INSTALL=install -C is being deprecated. If you want to do this, use COPY=-C instead. The former method will be supported for only a limited time. If you see install: warning: the -d and -C options may not be specified together in your makeworld, then you need to migrate towards using COPY=-C. 20010525: It appears that vm is now stable enough to use again. However, there may be other problems, so caution is still urged. alpha definitely is in bad shape. 20010521: Minor repo damage has happened. This may cause problems with cvsup of ports. If you get errors, please see http://www.FreeBSD.org/cgi/query-pr.cgi?pr=27495 at the bottom for details on a workaround. The error message is Updater failed: Cannot delete "/usr/ports/www/jakarta-tomcat/files": Directory not empty 20010520: Vm and/or swapping are busted on -current. Please be patient. 20010519: pccard has had much reorganizational work done to it over the past few days. Everything should still work, but if not, please contact imp@freebsd.org. 20010517: ata ioctl changed. Make sure to recompile both kernel and userland at the same time. 20010517: New ncurses imported. 20010512: DEVFS is now opt out, not opt in. Barring major problems, this will be the only way to go starting July 1. 20010504: OpenSSH has been updated to 2.9. Some defaults are different, including RhostsRSAAuthentication, which changes from yes to no. 20010502: Perl breakage in 20010501 was corrected at 14:18:33 PDT. 20010501: Building perl was broken at 02:25:25 PDT. 20010430: The bug in 20010429 was corrected at 07:35:37 PDT. It is safe to go back in the water. 20010429: A bad bug was committed at 04:48:42 PDT. Don't use kernels after this date, but before the correction date. 20010423: old fsck and new kernel interactions appear to have been fixed. 20010411: fsck and the kernel were changed to handle some optimizations to directory layout. This breaks backward compatibility. Update only if you understand that you must not use the old fsck with the new kernel ever. 20010330: fsck has changed the meaning of the pass column in /etc/fstab. Please see the cvs commit to fsck.8 or the fsck.8 man page for details. It is unclear if changes to /etc/fstab are necessary. 20010319: portmap had changed name to rpcbind for maximum POLA in your current world. /etc/hosts.{allow,deny} needs changes. nfs and other rpc based programs that rely on portmapper will not work without updates to /etc/hosts.{allow,deny} and /etc/netconfig. 20010315: ata subsystem changes. ATA_ENABLE_ATAPI_DMA, ATA_ENABLE_WC and ATA_ENABLE_TAGS are no longer kernel options. They have been replaced by tunables. See ata.4 for details. 20010312: The fxp driver was converted to use miibus. If you compile fxp into your kernel statically, you will need to add miibus. 20010312: The wi device now defaults to BSS (infrastructure) mode instead of ad-hoc. 20010310: /dev/urandom should be a symbolic link to /dev/random now. Users of current not using DEVFS need to run MAKEDEV std. ssh might not work if you don't. 20010303: The ed driver has been updated. It now allows mii attachments, which means that you must include the miibus in your kernel if you use the ed driver. 20010220: The problems with libc have been corrected. It is now mostly safe to go back into the water. 20010211: The size of FILE was changed. This breaks upgrading. If you must upgrade, be prepared for pain. It also breaks almost all binaries that you've compiled on -current. You are warned that before upgrading would be a good time to do a level 0 dump of your system. No, really, I mean it this time. To get to the new system, you'll need to use the following workaround. Hopefully this can be sorted out so that we don't have to move this to the updating section. To get around the installworld problem, do: # cd /usr/src/usr.bin/sed # make install # cd /usr/src # make installworld If that doesn't work, then try: # make -k installworld # make installworld 20010207: DEVFS is now the default. If you use vinum, make sure that you do not include devfs in your kernel as problems result. 20010205: FFS_ROOT and CD9660_ROOT have been removed or deprecated. Remove them from your config. 20010122: ****************************** WARNING ****************************** buildkernel has been changed slightly ****************************** WARNING ****************************** KERNCONF replaces the variable KERNEL for buildkernel. You should update your scripts and make.conf accordingly. 20010119: config has changed to allow DEV_FOO as a replacement for NFOO. This requires a new config to build correctly. 20010116: The kernel option I386_CPU is now mutually exclusive with the other cpu types. If you have an i386 system, be sure that it only had this line. Remove it for all other configurations. 20010110: Changes to the kernel require it and burncd be in sync. 20010102: Everyone who has hw.sndunit set to something in /etc/sysctl.conf, it is now hw.snd.unit. 20010101: ex and vi were broken by some changes to sys/queue.h. If you have a bad vi, you will see make buildworld fail with a core dump while building termcap. You can work around this problem by adding -k to your make buildworld. This will cause the build to complete and install a new vi. Once that's done, you can rebuild again without the -k to pick up anything that might have been ignored by the -k option. Others have suggested that you can just rebuild libc if your vi/ex is dynamically linked, but I've not received any reports of this working. 20001228: There have been some changes to libcrypt in -current. The libscrypt/libdescrypt symlink silliness is gone and the installed libcrypt is fully functional. Be aware of this. 20001218: Linksys Fast Ethernet PCCARD cards supported by the ed driver now require the addition of flag 0x80000 to their config line in pccard.conf(5). This flag is not optional. These Linksys cards will not be recognized without it. 20001205: Important new FreeBSD-version stuff: PAM support has been worked in, partially from the "Unix" OpenSSH version. This requires adding the following in pam.conf: sshd auth sufficient pam_skey.so sshd auth required pam_unix.so try_first_pass sshd session required pam_permit.so 20001031: cvs updated to 1.11. 20001020: The random device needs more entropy, so you need to make sure that you've run mergemaster to get a /etc/rc which will seed /dev/random. If you don't and the system hangs after ldconfig, then banging on the keyboard randomly until it unhangs is one workaround. 20001010: ****************************** WARNING ****************************** Sendmail has been updated. ****************************** WARNING ****************************** o mail.local(8) is no longer installed as a set-user-id binary. o sendmail(8) is now built with STARTTLS support unless NO_OPENSSL is set. o The default /etc/mail/sendmail.cf disables the SMTP EXPN and VRFY commands. o Now using sendmail's version of vacation(1). o The sendmail cf building tools (contrib/sendmail/cf) are installed in /usr/share/sendmail/cf. o sendmail.cw changed to local-host-names More details can be found at http://people.freebsd.org/~imp/UPDATING/sendmail-20001010 20001009: The ports tree's new layout is in place. Be sure to update your entire ports tree, or you will have problems. 20001006: The perl build procedure no longer installs miniperl, nor uses the installed miniperl. It is recommended that you delete /usr/bin/miniperl. 20001005: This weekend the ports tree will be updated to a new layout. It will be in an inconsistent state until noted in the UPDATING file, or with asami-san's message to the relevant mailing lists. With this new layout, you'll need to update the whole tree for anything to work. 20000928: There was a change in the passwd format. Need more information. 20000916: /boot/kernel/kernel.ko -> /boot/kernel/kernel change has taken place. Please update boot loader (not the boot blocks) at the same time as your kernel. 20000914: The new pmtimer device is necessary for laptops. Failure to include the device will cause suspended laptops losing time when they resume. Include device pmtimer in your config file and hint.pmtimer.0.at="isa" to your /boot/device.hints file. 20000911: The random device has been turned into a (pseudo-)device, rather than an option. The supplied kernel config files have been updated. You will need to do something similar in your own kernel config file. Remove: options RANDOMDEV Add: device random If you prefer to load the loadable module, you need to do nothing. 20000909: The random device module has been renamed from randomdev.ko to random.ko. You will need to edit your /boot/loader.conf to reflect this if you load this module at boot time. The line should read: random_load="YES" 20000907: The SMPNG commit has happened. It should work, but if it doesn't, fallback to the PRE_SMPNG CVS tag. There are likely to be a variety of minor issues. Please see 20000905 to make sure you don't have model loading problems which might at first blush appear related to SMP. 20000906: nsswitch has been imported from NetBSD. Among other things, this means that /etc/host.conf is no longer used. See nsswitch.conf(5) instead. Note that at boot time rc.network will attempt to produce a new nsswitch.conf file for you if you don't have one, and you have host.conf. 20000905: The ucred structure changed size. This breaks the interface that mountd uses. Trying to use an older mountd with a newer kernel guarantees a panic. This means that you need to use kernels newer than today only with matching mountd, but you needed to do that anyway with the boot loader changes. 20000905: The boot loader has been updated. The new default kernel is now /boot/kernel/kernel.ko. The new default module location is /boot/kernel. You *MUST* upgrade your boot loader and kernel at the same time. The easiest way to do this is to do the buildworld/buildkernel/ installkernel/installworld dance. Furthermore, you are urged to delete your old /modules directory before booting the new kernel, since kldload will find stale modules in that directory instead of finding them in the correct path, /boot/kernel. The most common complaint that this cures is that the linux module crashes your machine after the update. if [ ! -d /boot/kernel.old ]; then mv /modules.old /boot/kernel.old chflags noschg /kernel.old mv /kernel.old /boot/kernel.old/kernel.ko chflags schg /boot/kernel.old/kernel.ko fi 20000904: A new issue with the sendmail upgrade has come to light. /etc/aliases has moved to /etc/mail/aliases. Mergemaster will incorrectly install the default aliases in /etc/mail rather than move the old one from /etc. So you'll need to manually move the file, create a symbolic link, remove the old /etc/aliases.db and run newaliases. For safety sake, you should stop sendmail while doing this and run the upgrade when locally sourced email is not likely to be generated. 20000825: /boot/device.hints is now required for installkernel to succeed. You should copy GENERIC.hints for your architecture into /boot/device.hints. If and only if you compile hints into your kernel, then this file may be empty. Please note, if you have an empty or missing /boot/device.hints file and you neglected to compile hints into your kernel, no boot messages will appear after the boot loader tries to start the kernel. 20000821: If you do NOT have ``options RANDOMDEV'' in your kernel and you DO want the random device then add randomdev_load="YES" to /boot/loader.conf. 20000812: suidperl is now always built and installed on the system, but with permissions of 511. If you have applications that use this program, you are now required to add ENABLE_SUIDPERL=true to /etc/make.conf. If you forget to do this, chmod 4511 /usr/bin/suidperl will fix this until the next build. 20000812: sendmail has been updated from 8.9.3 to 8.11.0. Some of the more visible changes that may immediately affect your configuration include: - New default file locations from src/contrib/sendmail/cf/README - newaliases limited to root and trusted users - MSA port (587) turned on by default - New queue file naming system so can't go from 8.11 -> 8.9 - FEATURE(`rbl') renamed to FEATURE(`dnsbl') - FEATURE(`nullclient') is more full featured - FEATURE(`nouucp') requires an argument: `reject' or `nospecial' - mail.local FreeBSD-only -b option changed to -B - See src/contrib/sendmail/RELEASE_NOTES for more info 20000810: suidperl (aka sperl) is no longer build by default. You must specifically define BUILD_SUIDPERL to "true" for it to be build. Furthermore, we recommend that you remove /usr/bin/sperl* and /usr/bin/suidperl files from your system unless you have a specific use for it. 20000729: Networking defaults have been tightened. Anybody upgrading /etc/defaults/rc.conf needs to add the following lines to /etc/rc.conf if they want to have the same setup afterwards (unless the variables already are set, of course): # Enable network daemons for user convenience. inetd_enable="YES" portmap_enable="YES" sendmail_enable="YES" 20000728: If you have null_load="YES" in your /boot/loader.conf, you will need to change that to nullfs_load="YES". 20000728: The "installkernel" target has changed slightly. Now even if you override KERNEL e.g. 'make installkernel KERNEL=MYKERNEL' it will install the MYKERNEL file (built with the buildkernel target) as /kernel rather than /MYKERNEL. Those who have updated their /boot/loader.conf files to point to /MYKERNEL should remove that entry or perform manual rename of /kernel to /MYKERNEL. 20000711: If you use CVSUP or CTM to get CVS trees, AND you used to get the old crypto files from internat.freebsd.org AND you check out files from the CVS tree with the cvs command, please read http://people.freebsd.org/~imp/internat.txt for details on potential problems that you might have and how to get around them. If you are merely a mirror, or don't answer yes to each of the clauses above, you needn't worry. 20000711: /etc/security has been updated to print the inode number of setuid programs that have changed. You will see a large spike in the number of changed programs the first time when you run mergemaster to get a new /etc/security. 20000710: /dev/random now has good entropy collection (from the keyboard and sysmouse drivers). Please ensure that either `options RANDOMDEV' is present in your kernel config file or that `randomdev_load="YES"' is in your /boot/loader.conf. If you do not have the /dev/random driver, OpenSSL (and consequently lots of crypto tools (like SSH)) will fail with strange errors. (see below, 20000624). FreeBSD-current is safe again to run Crypto. 20000709: phk made the malloc default options AJ. This may slow things down and uncover other latent bugs in the code. If you need to run at full speed, you can disable this by doing the following: ln -s aj /etc/malloc.conf 20000706: libftpio's version was accidentally bumped a few days ago. This has been corrected. You may need to remove /usr/lib/libftpio.so.6 before doing your next buildworld/installworld pair. It certainly won't hurt to remove it before the update procedure. It will break fetch until a new one is built, but ftp can be used in the interim if needed. 20000705: The crypto packages have changed for the cvsup. This has been done in a backward compatible way, but the old packages will go away at some point in the future. Look at /usr/share/examples/cvsup for details. 20000704: With the new sys/modules/sound/drivers/*, you will need to set SYSDIR until you do an installworld after July 7th. 20000704: rc.shutdown and rc will now call the rc.d scripts with start or stop. This may cause some harmless warnings from older rc.d scripts that haven't been updated. 20000630: The libfetch based version of fetch has gone into the tree. Minor problems may result on some of the less popular sites, which should be reported to des@freebsd.org. 20000625: From approximately this date forward, one must have the crypto system installed in order to build the system and kernel. While not technically strictly true, one should treat it as required and grab the crypto bits. If you are grabbing CVS trees, src-all and cvs-crypto should be treated as if they were required. You should check with the latest collections to make sure that these haven't changed. 20000624: Mark Murray just committed the first parts of a cleanup of /dev/zero, et al. This is also cleaning up /dev/random. The entropy is disconnected, so DO NOT USE VERSIONS OF FREEBSD -CURRENT FROM THIS POINT to 2000710 for cryptographic services until Mark can merge in the fixes to this work in progress. openssh and openssl should not be used to generate keys from this date to the completion of the work. If you must operate at this reduced level of security, add ' options RANDOMDEV' to your kernel or modload the randomdev module. You may also need to copy a new MAKEDEV to /dev and recreate the random and urandom devices. 20000622: The license on the softupdates is now a standard 2 clause BSD license. You may need to remove your symbolic links that used to be required when updating. 20000621: Scott Flatman sent in a decent write-up on the config file update procedure. http://people.freebsd.org/~imp/config-upd.html NOTE: LINT is gone. It has been replaced with NOTES. NOTES isn't buildable. However, you can generate a LINT file: cd /sys//conf && make LINT 20000620: Binutils 2.10 have hit the tree, or will shortly. As soon as they do, the problem noted in 20000522 will be resolved and that workaround will no longer be required. 20000615: phk removed the compatibility creation of wd devices in the ad driver. If you haven't done so already, you must update your fstab, etc to use the ad devices instead of the wd devices. In addition, you'll need to update your boot blocks to a more modern version, if you haven't already done so. Modern here means 4.0 release or newer (although older releases may work). 20000612: Peter took an axe to config(8). Be sure that you read his mail on the topic before even thinking about updating. You will need to create a /boot/device.hints or add a hints directive to your config file to compile them in statically. The format of the config file has changed as well. Please see GENERIC or NEWCARD for examples of the new format. Indirectly, this also breaks USERCONFIG. Unless a newer entry says that it has been fixed, assume that must use the hints mechanism in the loader if you need to use a machine with very old ISA cards in it. 20000522: A new set of binutils went into the tree today. Anybody building a kernel after this point is advised that they need to rebuild their binutils (or better yet do a buildworld/installworld) before building a new kernel. Due to bugs in binutils, using malloc options (eg /etc/malloc.conf or MALLOC_OPTIONS env var) J will cause ld to dump core. It is recommended that you don't set this option until the problem is resolved. 20000513: The ethernet drivers were all updated to clean up the BPF handling. 20000510: The problems with boot blocks on the alphas have been corrected. This will require some care in updating alphas. A new libstand is requires for the boot blocks to build properly. 20000503: Recompile all kld modules. Proper version dependency info is now available. 20000502: Modules have been disconnected from the buildworld tree and connected to the kernel building instead. 20000427: You may need to build gperf cd /usr/src/gnu/usr.bin/gperf && make depend all install when upgrading from 4.0 -> current. The build system now uses an option only in -current. 20000417: The method that we brand ELF binaries has changed to be more acceptable to the binutils maintainers. You will need to rebrand your ELF binaries that aren't native. One problem binary is the Linux ldconfig. After your make world, but before you reboot, you'll need to issue: brandelf -t Linux /compat/linux/sbin/ldconfig if you have Linux compatibility enabled on your machine. 20000320: If you have really bad/marginal IDE drives, you may find they don't work well. Use pio mode instead. The easiest way to cope if you have a problem combination is to add: /sbin/sysctl hw.ata.ata_dma=0 to the start of /etc/rc.conf. 20000319: The ISA and PCI compatibility shims have been connected to the options COMPAT_OLDISA and COMPAT_OLDPCI. If you are using old style PCI or ISA drivers (i.e. tx, voxware, etc.) you must include the appropriate option in your kernel config. Drivers using the shims should be updated or they won't ship with 5.0-RELEASE, targeted for 2001. 20000318: We've entered the traditional post release dumping party. Large kernel changes are being committed and are in the works. It is important to keep the systems' klds and kernel in sync as kernel interfaces and structures are changing. Before reporting kernel panics, make sure that all modules that you are loading are up to date. 20000315: If you are upgrading from an older version of FreeBSD, you need to update your boot blocks as well. 'disklabel -B ad0' will do the trick. This isn't critical until you remove your wd device entries in /dev, at which point your system will not boot. 20000315: 4.0 RELEASE shipped. Please see the 4.0 UPDATING file for how to upgrade to 4.0 from 3.x. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. With a /usr/obj tree with a fresh buildworld, make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a 5.X system. Replace ${arch} with the architecture of your machine (e.g. "i386", "alpha", "amd64", "ia64", "pc98", "sparc64", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. [7] make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installkernel KERNCONF=YOUR_KERNEL_HERE [1] [3] /etc/rc.d/preseedrandom [10] mergemaster -p [5] make installworld mergemaster [4] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make installworld DESTDIR=${CURRENT_ROOT} make buildkernel KERNCONF=YOUR_KERNEL_HERE cp /usr/src/sys/${ARCH}/conf/GENERIC.hints \ ${CURRENT_ROOT}/boot/device.hints # as needed make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cd /usr/src/etc; make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from 4.x-stable to current ---------------------------------------------- # 5.x uses more space than 4.x. Also, the location of kernel # modules has changed. If you are installing 5.0 onto a 4.x # system, you'll need about 30MB of free disk space on your / # partition. If you have less than this, you may encounter difficult # to back out of problems with this procedure. If /tmp is on # the / partition, you may want to completely remove all its content # before upgrading, as this can be a common source of shortage of # space on /. [7] make buildworld [9] make buildkernel KERNCONF=YOUR_KERNEL_HERE [8] cp sys/${MACHINE}/conf/GENERIC.hints /boot/device.hints [2] make installkernel KERNCONF=YOUR_KERNEL_HERE cd sys/boot ; make STRIP="" install [6] [1] [3] /usr/src/etc/rc.d/preseedrandom [10] mergemaster -p [5] rm -rf /usr/include/g++ make installworld mergemaster -i [4] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [2] If you have legacy ISA devices, you may need to create your own device.hints to reflect your unique hardware configuration. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd /usr/src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. For the 4.x -> 5.x upgrade, you will also see many messages about needing to recompile your userland. These are harmless and can be ignored while you proceed to the next step. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. [5] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. You may need to install a new mergemaster (cd src/usr.sbin/mergemaster && make install) after the buildworld before this step if you last updated from current before 20020224 or from -stable before 20020408. [6] 4.x boot loader can be used to boot a 5.x system, however it is difficult to do that at best. If you wish to try, then you should interrupt the boot and at the ok prompt type: ok unload ok boot /boot/kernel/kernel If this fails to work, you must install a new boot loader as described here. [7] Before you upgrade, please make sure that you are not using compatibility slices. These are device names of the form, on i386 and other architectures that use MBR slicing, /dev/ad0a without the actual slice name. Chances are excellent that these will break. You generally must update these entries to use the post FreeBSD 2.x form of /dev/ad0s1a. [8] In order to have a kernel that can run the 4.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD4 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. [9] When checking out sources, you must include the -P flag to have cvs prune empty directories. Also, if CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. In case you would like to avoid installing new packages of everything, you might want to uncomment the "COMPAT4X= YES" entry, so that 4.x compatibility libraries are built which should allow you to continue using your existing software for a while. [10] In order to create temporary files, /dev/random must be initialized by feeding data into it. /usr/src/etc/rc.d/preseedrandom takes care of this. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -current. Not all things will be listed here, and it only starts on March 15, 2000. Updating files can found in previous releases if your system is older than this. Copyright information: Copyright 1998, 2002 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. If you find this document useful, and you want to, you may buy the author a beer. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: releng/5.4/gnu/usr.bin/gzip/gzip.h =================================================================== --- releng/5.4/gnu/usr.bin/gzip/gzip.h (revision 162446) +++ releng/5.4/gnu/usr.bin/gzip/gzip.h (revision 162447) @@ -1,318 +1,320 @@ /* gzip.h -- common declarations for all gzip modules * Copyright (C) 1992-1993 Jean-loup Gailly. * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. * * $FreeBSD$ */ #if defined(__STDC__) || defined(PROTO) # define OF(args) args #else # define OF(args) () #endif #ifdef __STDC__ typedef void *voidp; #else typedef char *voidp; #endif /* I don't like nested includes, but the string and io functions are used * too often */ #include #if !defined(NO_STRING_H) || defined(STDC_HEADERS) # include # if !defined(STDC_HEADERS) && !defined(NO_MEMORY_H) && !defined(__GNUC__) # include # endif # define memzero(s, n) memset ((voidp)(s), 0, (n)) #else # include # define strchr index # define strrchr rindex # define memcpy(d, s, n) bcopy((s), (d), (n)) # define memcmp(s1, s2, n) bcmp((s1), (s2), (n)) # define memzero(s, n) bzero((s), (n)) #endif #ifndef RETSIGTYPE # define RETSIGTYPE void #endif #define local static #include typedef uint8_t uch; typedef uint16_t ush; typedef uint32_t ulg; /* Return codes from gzip */ #define OK 0 #define ERROR 1 #define WARNING 2 /* Compression methods (see algorithm.doc) */ #define STORED 0 #define COMPRESSED 1 #define PACKED 2 #define LZHED 3 /* methods 4 to 7 reserved */ #define DEFLATED 8 #define MAX_METHODS 9 extern int method; /* compression method */ /* To save memory for 16 bit systems, some arrays are overlaid between * the various modules: * deflate: prev+head window d_buf l_buf outbuf * unlzw: tab_prefix tab_suffix stack inbuf outbuf * inflate: window inbuf * unpack: window inbuf prefix_len * unlzh: left+right window c_table inbuf c_len * For compression, input is done in window[]. For decompression, output * is done in window except for unlzw. */ #ifndef INBUFSIZ # ifdef SMALL_MEM # define INBUFSIZ 0x2000 /* input buffer size */ # else # define INBUFSIZ 0x8000 /* input buffer size */ # endif #endif #define INBUF_EXTRA 64 /* required by unlzw() */ #ifndef OUTBUFSIZ # ifdef SMALL_MEM # define OUTBUFSIZ 8192 /* output buffer size */ # else # define OUTBUFSIZ 16384 /* output buffer size */ # endif #endif #define OUTBUF_EXTRA 2048 /* required by unlzw() */ #ifndef DIST_BUFSIZE # ifdef SMALL_MEM # define DIST_BUFSIZE 0x2000 /* buffer for distances, see trees.c */ # else # define DIST_BUFSIZE 0x8000 /* buffer for distances, see trees.c */ # endif #endif #ifdef DYN_ALLOC # define EXTERN(type, array) extern type * near array # define DECLARE(type, array, size) type * near array # define ALLOC(type, array, size) { \ array = (type*)fcalloc((size_t)(((size)+1L)/2), 2*sizeof(type)); \ if (array == NULL) error("insufficient memory"); \ } # define FREE(array) {if (array != NULL) fcfree(array), array=NULL;} #else # define EXTERN(type, array) extern type array[] # define DECLARE(type, array, size) type array[size] # define ALLOC(type, array, size) # define FREE(array) #endif EXTERN(uch, inbuf); /* input buffer */ EXTERN(uch, outbuf); /* output buffer */ EXTERN(ush, d_buf); /* buffer for distances, see trees.c */ EXTERN(uch, window); /* Sliding window and suffix table (unlzw) */ #define tab_suffix window #ifndef MAXSEG_64K # define tab_prefix prev /* hash link (see deflate.c) */ # define head (prev+WSIZE) /* hash head (see deflate.c) */ EXTERN(ush, tab_prefix); /* prefix code (see unlzw.c) */ #else # define tab_prefix0 prev # define head tab_prefix1 EXTERN(ush, tab_prefix0); /* prefix for even codes */ EXTERN(ush, tab_prefix1); /* prefix for odd codes */ #endif extern unsigned insize; /* valid bytes in inbuf */ extern unsigned inptr; /* index of next byte to be processed in inbuf */ extern unsigned outcnt; /* bytes in output buffer */ extern long bytes_in; /* number of input bytes */ extern long bytes_out; /* number of output bytes */ extern long header_bytes;/* number of bytes in gzip header */ #define isize bytes_in /* for compatibility with old zip sources (to be cleaned) */ extern int ifd; /* input file descriptor */ extern int ofd; /* output file descriptor */ extern char ifname[]; /* input file name or "stdin" */ extern char ofname[]; /* output file name or "stdout" */ extern char *progname; /* program name */ extern long time_stamp; /* original time stamp (modification time) */ extern long ifile_size; /* input file size, -1 for devices (debug only) */ typedef int file_t; /* Do not use stdio */ #define NO_FILE (-1) /* in memory compression */ #define PACK_MAGIC "\037\036" /* Magic header for packed files */ #define GZIP_MAGIC "\037\213" /* Magic header for gzip files, 1F 8B */ #define OLD_GZIP_MAGIC "\037\236" /* Magic header for gzip 0.5 = freeze 1.x */ #define LZH_MAGIC "\037\240" /* Magic header for SCO LZH Compress files*/ #define PKZIP_MAGIC "\120\113\003\004" /* Magic header for pkzip files */ /* gzip flag byte */ #define ASCII_FLAG 0x01 /* bit 0 set: file probably ascii text */ #define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */ #define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */ #define ORIG_NAME 0x08 /* bit 3 set: original file name present */ #define COMMENT 0x10 /* bit 4 set: file comment present */ #define ENCRYPTED 0x20 /* bit 5 set: file is encrypted */ #define RESERVED 0xC0 /* bit 6,7: reserved */ /* internal file attribute */ #define UNKNOWN 0xffff #define BINARY 0 #define ASCII 1 #ifndef WSIZE # define WSIZE 0x8000 /* window size--must be a power of two, and */ #endif /* at least 32K for zip's deflate method */ #define MIN_MATCH 3 #define MAX_MATCH 258 /* The minimum and maximum match lengths */ #define MIN_LOOKAHEAD (MAX_MATCH+MIN_MATCH+1) /* Minimum amount of lookahead, except at the end of the input file. * See deflate.c for comments about the MIN_MATCH+1. */ #define MAX_DIST (WSIZE-MIN_LOOKAHEAD) /* In order to simplify the code, particularly on 16 bit machines, match * distances are limited to MAX_DIST instead of WSIZE. */ extern int decrypt; /* flag to turn on decryption */ extern int exit_code; /* program exit code */ extern int verbose; /* be verbose (-v) */ extern int quiet; /* be quiet (-q) */ extern int level; /* compression level */ extern int test; /* check .z file integrity */ extern int to_stdout; /* output to stdout (-c) */ extern int save_orig_name; /* set if original name must be saved */ +#define MIN(a,b) ((a) <= (b) ? (a) : (b)) + #define get_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(0)) #define try_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(1)) /* put_byte is used for the compressed output, put_ubyte for the * uncompressed output. However unlzw() uses window for its * suffix table instead of its output buffer, so it does not use put_ubyte * (to be cleaned up). */ #define put_byte(c) {outbuf[outcnt++]=(uch)(c); if (outcnt==OUTBUFSIZ)\ flush_outbuf();} #define put_ubyte(c) {window[outcnt++]=(uch)(c); if (outcnt==WSIZE)\ flush_window();} /* Output a 16 bit value, lsb first */ #define put_short(w) \ { if (outcnt < OUTBUFSIZ-2) { \ outbuf[outcnt++] = (uch) ((w) & 0xff); \ outbuf[outcnt++] = (uch) ((ush)(w) >> 8); \ } else { \ put_byte((uch)((w) & 0xff)); \ put_byte((uch)((ush)(w) >> 8)); \ } \ } /* Output a 32 bit value to the bit stream, lsb first */ #define put_long(n) { \ put_short((n) & 0xffff); \ put_short(((ulg)(n)) >> 16); \ } #define seekable() 0 /* force sequential output */ #define translate_eol 0 /* no option -a yet */ #define tolow(c) (isupper(c) ? (c)-'A'+'a' : (c)) /* force to lower case */ /* Macros for getting two-byte and four-byte header values */ #define SH(p) ((ush)(uch)((p)[0]) | ((ush)(uch)((p)[1]) << 8)) #define LG(p) ((ulg)(SH(p)) | ((ulg)(SH((p)+2)) << 16)) /* Diagnostic functions */ #ifdef DEBUG # define Assert(cond,msg) {if(!(cond)) error(msg);} # define Trace(x) fprintf x # define Tracev(x) {if (verbose) fprintf x ;} # define Tracevv(x) {if (verbose>1) fprintf x ;} # define Tracec(c,x) {if (verbose && (c)) fprintf x ;} # define Tracecv(c,x) {if (verbose>1 && (c)) fprintf x ;} #else # define Assert(cond,msg) # define Trace(x) # define Tracev(x) # define Tracevv(x) # define Tracec(c,x) # define Tracecv(c,x) #endif #define WARN(msg) {if (!quiet) fprintf msg ; \ if (exit_code == OK) exit_code = WARNING;} /* in zip.c: */ extern int zip OF((int in, int out)); extern int file_read OF((char *buf, unsigned size)); /* in unzip.c */ extern int unzip OF((int in, int out)); extern int check_zipfile OF((int in)); /* in unpack.c */ extern int unpack OF((int in, int out)); /* in unlzh.c */ extern int unlzh OF((int in, int out)); /* in gzip.c */ RETSIGTYPE abort_gzip OF((void)); /* in deflate.c */ void lm_init OF((int pack_level, ush *flags)); ulg deflate OF((void)); /* in trees.c */ void ct_init OF((ush *attr, int *method)); int ct_tally OF((int dist, int lc)); ulg flush_block OF((char *buf, ulg stored_len, int eof)); /* in bits.c */ void bi_init OF((file_t zipfile)); void send_bits OF((int value, int length)); unsigned bi_reverse OF((unsigned value, int length)); void bi_windup OF((void)); void copy_block OF((char *buf, unsigned len, int header)); extern int (*read_buf) OF((char *buf, unsigned size)); /* in util.c: */ extern int copy OF((int in, int out)); extern ulg updcrc OF((uch *s, unsigned n)); extern void clear_bufs OF((void)); extern int fill_inbuf OF((int eof_ok)); extern void flush_outbuf OF((void)); extern void flush_window OF((void)); extern void write_buf OF((int fd, voidp buf, unsigned cnt)); extern char *strlwr OF((char *s)); extern char *basename OF((char *fname)); extern void make_simple_name OF((char *name)); extern char *add_envopt OF((int *argcp, char ***argvp, char *env)); extern void error OF((char *m)); extern void warn OF((char *a, char *b)); extern void read_error OF((void)); extern void write_error OF((void)); extern void display_ratio OF((long num, long den, FILE *file)); extern voidp xmalloc OF((unsigned int size)); /* in inflate.c */ extern int inflate OF((void)); Index: releng/5.4/gnu/usr.bin/gzip/inflate.c =================================================================== --- releng/5.4/gnu/usr.bin/gzip/inflate.c (revision 162446) +++ releng/5.4/gnu/usr.bin/gzip/inflate.c (revision 162447) @@ -1,958 +1,958 @@ /* inflate.c -- Not copyrighted 1992 by Mark Adler version c10p1, 10 January 1993 */ /* You can do whatever you like with this source file, though I would prefer that if you modify it and redistribute it that you include comments to that effect with your name and the date. Thank you. [The history has been moved to the file ChangeLog.] */ /* Inflate deflated (PKZIP's method 8 compressed) data. The compression method searches for as much of the current string of bytes (up to a length of 258) in the previous 32K bytes. If it doesn't find any matches (of at least length 3), it codes the next byte. Otherwise, it codes the length of the matched string and its distance backwards from the current position. There is a single Huffman code that codes both single bytes (called "literals") and match lengths. A second Huffman code codes the distance information, which follows a length code. Each length or distance code actually represents a base value and a number of "extra" (sometimes zero) bits to get to add to the base value. At the end of each deflated block is a special end-of-block (EOB) literal/ length code. The decoding process is basically: get a literal/length code; if EOB then done; if a literal, emit the decoded byte; if a length then get the distance and emit the referred-to bytes from the sliding window of previously emitted data. There are (currently) three kinds of inflate blocks: stored, fixed, and dynamic. The compressor deals with some chunk of data at a time, and decides which method to use on a chunk-by-chunk basis. A chunk might typically be 32K or 64K. If the chunk is uncompressible, then the "stored" method is used. In this case, the bytes are simply stored as is, eight bits per byte, with none of the above coding. The bytes are preceded by a count, since there is no longer an EOB code. If the data is compressible, then either the fixed or dynamic methods are used. In the dynamic method, the compressed data is preceded by an encoding of the literal/length and distance Huffman codes that are to be used to decode this block. The representation is itself Huffman coded, and so is preceded by a description of that code. These code descriptions take up a little space, and so for small blocks, there is a predefined set of codes, called the fixed codes. The fixed method is used if the block codes up smaller that way (usually for quite small chunks), otherwise the dynamic method is used. In the latter case, the codes are customized to the probabilities in the current block, and so can code it much better than the pre-determined fixed codes. The Huffman codes themselves are decoded using a mutli-level table lookup, in order to maximize the speed of decoding plus the speed of building the decoding tables. See the comments below that precede the lbits and dbits tuning parameters. */ /* Notes beyond the 1.93a appnote.txt: 1. Distance pointers never point before the beginning of the output stream. 2. Distance pointers can point back across blocks, up to 32k away. 3. There is an implied maximum of 7 bits for the bit length table and 15 bits for the actual data. 4. If only one code exists, then it is encoded using one bit. (Zero would be more efficient, but perhaps a little confusing.) If two codes exist, they are coded using one bit each (0 and 1). 5. There is no way of sending zero distance codes--a dummy must be sent if there are none. (History: a pre 2.0 version of PKZIP would store blocks with no distance codes, but this was discovered to be too harsh a criterion.) Valid only for 1.93a. 2.04c does allow zero distance codes, which is sent as one code of zero bits in length. 6. There are up to 286 literal/length codes. Code 256 represents the end-of-block. Note however that the static length tree defines 288 codes just to fill out the Huffman codes. Codes 286 and 287 cannot be used though, since there is no length base or extra bits defined for them. Similarly, there are up to 30 distance codes. However, static trees define 32 codes (all 5 bits) to fill out the Huffman codes, but the last two had better not show up in the data. 7. Unzip can check dynamic Huffman blocks for complete code sets. The exception is that a single code would not be complete (see #4). 8. The five bits following the block type is really the number of literal codes sent minus 257. 9. Length codes 8,16,16 are interpreted as 13 length codes of 8 bits (1+6+6). Therefore, to output three times the length, you output three codes (1+1+1), whereas to output four times the same length, you only need two codes (1+3). Hmm. 10. In the tree reconstruction algorithm, Code = Code + Increment only if BitLength(i) is not zero. (Pretty obvious.) 11. Correction: 4 Bits: # of Bit Length codes - 4 (4 - 19) 12. Note: length code 284 can represent 227-258, but length code 285 really is 258. The last length deserves its own, short code since it gets used a lot in very redundant files. The length 258 is special since 258 - 3 (the min match length) is 255. 13. The literal/length and distance code bit lengths are read as a single stream of lengths. It is possible (and advantageous) for a repeat code (16, 17, or 18) to go across the boundary between the two sets of lengths. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #if defined(STDC_HEADERS) || !defined(NO_STDLIB_H) # include #endif #include "gzip.h" #define slide window /* Huffman code lookup table entry--this entry is four bytes for machines that have 16-bit pointers (e.g. PC's in the small or medium model). Valid extra bits are 0..13. e == 15 is EOB (end of block), e == 16 means that v is a literal, 16 < e < 32 means that v is a pointer to the next table, which codes e - 16 bits, and lastly e == 99 indicates an unused code. If a code with e == 99 is looked up, this implies an error in the data. */ struct huft { uch e; /* number of extra bits or operation */ uch b; /* number of bits in this code or subcode */ union { ush n; /* literal, length base, or distance base */ struct huft *t; /* pointer to next level of table */ } v; }; /* Function prototypes */ int huft_build OF((unsigned *, unsigned, unsigned, ush *, ush *, struct huft **, int *)); int huft_free OF((struct huft *)); int inflate_codes OF((struct huft *, struct huft *, int, int)); int inflate_stored OF((void)); int inflate_fixed OF((void)); int inflate_dynamic OF((void)); int inflate_block OF((int *)); int inflate OF((void)); /* The inflate algorithm uses a sliding 32K byte window on the uncompressed stream to find repeated byte strings. This is implemented here as a circular buffer. The index is updated simply by incrementing and then and'ing with 0x7fff (32K-1). */ /* It is left to other modules to supply the 32K area. It is assumed to be usable as if it were declared "uch slide[32768];" or as just "uch *slide;" and then malloc'ed in the latter case. The definition must be in unzip.h, included above. */ /* unsigned wp; current position in slide */ #define wp outcnt #define flush_output(w) (wp=(w),flush_window()) /* Tables for deflate from PKZIP's appnote.txt. */ static unsigned border[] = { /* Order of the bit length code lengths */ 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; static ush cplens[] = { /* Copy lengths for literal codes 257..285 */ 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0}; /* note: see note #13 above about the 258 in this list. */ static ush cplext[] = { /* Extra bits for literal codes 257..285 */ 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0, 99, 99}; /* 99==invalid */ static ush cpdist[] = { /* Copy offsets for distance codes 0..29 */ 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577}; static ush cpdext[] = { /* Extra bits for distance codes */ 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13}; /* Macros for inflate() bit peeking and grabbing. The usage is: NEEDBITS(j) x = b & mask_bits[j]; DUMPBITS(j) where NEEDBITS makes sure that b has at least j bits in it, and DUMPBITS removes the bits from b. The macros use the variable k for the number of bits in b. Normally, b and k are register variables for speed, and are initialized at the beginning of a routine that uses these macros from a global bit buffer and count. If we assume that EOB will be the longest code, then we will never ask for bits with NEEDBITS that are beyond the end of the stream. So, NEEDBITS should not read any more bytes than are needed to meet the request. Then no bytes need to be "returned" to the buffer at the end of the last block. However, this assumption is not true for fixed blocks--the EOB code is 7 bits, but the other literal/length codes can be 8 or 9 bits. (The EOB code is shorter than other codes because fixed blocks are generally short. So, while a block always has an EOB, many other literal/length codes have a significantly lower probability of showing up at all.) However, by making the first table have a lookup of seven bits, the EOB code will be found in that first lookup, and so will not require that too many bits be pulled from the stream. */ ulg bb; /* bit buffer */ unsigned bk; /* bits in bit buffer */ ush mask_bits[] = { 0x0000, 0x0001, 0x0003, 0x0007, 0x000f, 0x001f, 0x003f, 0x007f, 0x00ff, 0x01ff, 0x03ff, 0x07ff, 0x0fff, 0x1fff, 0x3fff, 0x7fff, 0xffff }; #ifdef CRYPT uch cc; # define NEXTBYTE() \ (decrypt ? (cc = get_byte(), zdecode(cc), cc) : get_byte()) #else # define NEXTBYTE() (uch)get_byte() #endif #define NEEDBITS(n) {while(k<(n)){b|=((ulg)NEXTBYTE())<>=(n);k-=(n);} /* Huffman code decoding is performed using a multi-level table lookup. The fastest way to decode is to simply build a lookup table whose size is determined by the longest code. However, the time it takes to build this table can also be a factor if the data being decoded is not very long. The most common codes are necessarily the shortest codes, so those codes dominate the decoding time, and hence the speed. The idea is you can have a shorter table that decodes the shorter, more probable codes, and then point to subsidiary tables for the longer codes. The time it costs to decode the longer codes is then traded against the time it takes to make longer tables. This results of this trade are in the variables lbits and dbits below. lbits is the number of bits the first level table for literal/ length codes can decode in one step, and dbits is the same thing for the distance codes. Subsequent tables are also less than or equal to those sizes. These values may be adjusted either when all of the codes are shorter than that, in which case the longest code length in bits is used, or when the shortest code is *longer* than the requested table size, in which case the length of the shortest code in bits is used. There are two different values for the two tables, since they code a different number of possibilities each. The literal/length table codes 286 possible values, or in a flat code, a little over eight bits. The distance table codes 30 possible values, or a little less than five bits, flat. The optimum values for speed end up being about one bit more than those, so lbits is 8+1 and dbits is 5+1. The optimum values may differ though from machine to machine, and possibly even between compilers. Your mileage may vary. */ int lbits = 9; /* bits in base literal/length lookup table */ int dbits = 6; /* bits in base distance lookup table */ /* If BMAX needs to be larger than 16, then h and x[] should be ulg. */ #define BMAX 16 /* maximum bit length of any code (16 for explode) */ #define N_MAX 288 /* maximum number of codes in any set */ unsigned hufts; /* track memory usage */ int huft_build(b, n, s, d, e, t, m) unsigned *b; /* code lengths in bits (all assumed <= BMAX) */ unsigned n; /* number of codes (assumed <= N_MAX) */ unsigned s; /* number of simple-valued codes (0..s-1) */ ush *d; /* list of base values for non-simple codes */ ush *e; /* list of extra bits for non-simple codes */ struct huft **t; /* result: starting table */ int *m; /* maximum lookup bits, returns actual */ /* Given a list of code lengths and a maximum table size, make a set of tables to decode that set of codes. Return zero on success, one if the given code set is incomplete (the tables are still built in this case), two if the input is invalid (all zero length codes or an oversubscribed set of lengths), and three if not enough memory. */ { unsigned a; /* counter for codes of length k */ unsigned c[BMAX+1]; /* bit length count table */ unsigned f; /* i repeats in table every f entries */ int g; /* maximum code length */ int h; /* table level */ register unsigned i; /* counter, current code */ register unsigned j; /* counter */ register int k; /* number of bits in current code */ int l; /* bits per table (returned in m) */ register unsigned *p; /* pointer into c[], b[], or v[] */ register struct huft *q; /* points to current table */ struct huft r; /* table entry for structure assignment */ struct huft *u[BMAX]; /* table stack */ unsigned v[N_MAX]; /* values in order of bit length */ register int w; /* bits before this table == (l * h) */ unsigned x[BMAX+1]; /* bit offsets, then code stack */ unsigned *xp; /* pointer into x */ int y; /* number of dummy codes added */ unsigned z; /* number of entries in current table */ /* Generate counts for each bit length */ memzero(c, sizeof(c)); p = b; i = n; do { Tracecv(*p, (stderr, (n-i >= ' ' && n-i <= '~' ? "%c %d\n" : "0x%x %d\n"), n-i, *p)); c[*p]++; /* assume all entries <= BMAX */ p++; /* Can't combine with above line (Solaris bug) */ } while (--i); if (c[0] == n) /* null input--all zero length codes */ { *t = (struct huft *)NULL; *m = 0; - return 0; + return 2; } /* Find minimum and maximum length, bound *m by those */ l = *m; for (j = 1; j <= BMAX; j++) if (c[j]) break; k = j; /* minimum code length */ if ((unsigned)l < j) l = j; for (i = BMAX; i; i--) if (c[i]) break; g = i; /* maximum code length */ if ((unsigned)l > i) l = i; *m = l; /* Adjust last length count to fill out codes, if needed */ for (y = 1 << j; j < i; j++, y <<= 1) if ((y -= c[j]) < 0) return 2; /* bad input: more codes than bits */ if ((y -= c[i]) < 0) return 2; c[i] += y; /* Generate starting offsets into the value table for each length */ x[1] = j = 0; p = c + 1; xp = x + 2; while (--i) { /* note that i == g from above */ *xp++ = (j += *p++); } /* Make a table of values in order of bit lengths */ p = b; i = 0; do { if ((j = *p++) != 0) v[x[j]++] = i; } while (++i < n); n = x[g]; /* set n to length of v */ /* Generate the Huffman codes and for each, make the table entries */ x[0] = i = 0; /* first Huffman code is zero */ p = v; /* grab values in bit order */ h = -1; /* no tables yet--level -1 */ w = -l; /* bits decoded == (l * h) */ u[0] = (struct huft *)NULL; /* just to keep compilers happy */ q = (struct huft *)NULL; /* ditto */ z = 0; /* ditto */ /* go through the bit lengths (k already is bits in shortest code) */ for (; k <= g; k++) { a = c[k]; while (a--) { /* here i is the Huffman code of length k bits for value *p */ /* make tables up to required level */ while (k > w + l) { h++; w += l; /* previous table always l bits */ /* compute minimum size table less than or equal to l bits */ z = (z = g - w) > (unsigned)l ? l : z; /* upper limit on table size */ if ((f = 1 << (j = k - w)) > a + 1) /* try a k-w bit table */ { /* too few codes for k-w bit table */ f -= a + 1; /* deduct codes from patterns left */ xp = c + k; if (j < z) while (++j < z) /* try smaller tables up to z bits */ { if ((f <<= 1) <= *++xp) break; /* enough codes to use up j bits */ f -= *xp; /* else deduct codes from patterns */ } } z = 1 << j; /* table entries for j-bit table */ /* allocate and link in new table */ if ((q = (struct huft *)malloc((z + 1)*sizeof(struct huft))) == (struct huft *)NULL) { if (h) huft_free(u[0]); return 3; /* not enough memory */ } hufts += z + 1; /* track memory usage */ *t = q + 1; /* link to list for huft_free() */ *(t = &(q->v.t)) = (struct huft *)NULL; u[h] = ++q; /* table starts after link */ /* connect to last table, if there is one */ if (h) { x[h] = i; /* save pattern for backing up */ r.b = (uch)l; /* bits to dump before this table */ r.e = (uch)(16 + j); /* bits in this table */ r.v.t = q; /* pointer to this table */ j = i >> (w - l); /* (get around Turbo C bug) */ u[h-1][j] = r; /* connect to last table */ } } /* set up table entry in r */ r.b = (uch)(k - w); if (p >= v + n) r.e = 99; /* out of values--invalid code */ else if (*p < s) { r.e = (uch)(*p < 256 ? 16 : 15); /* 256 is end-of-block code */ r.v.n = (ush)(*p); /* simple code is just the value */ p++; /* one compiler does not like *p++ */ } else { r.e = (uch)e[*p - s]; /* non-simple--look up in lists */ r.v.n = d[*p++ - s]; } /* fill code-like entries with r */ f = 1 << (k - w); for (j = i >> w; j < z; j += f) q[j] = r; /* backwards increment the k-bit code i */ for (j = 1 << (k - 1); i & j; j >>= 1) i ^= j; i ^= j; /* backup over finished tables */ while ((i & ((1 << w) - 1)) != x[h]) { h--; /* don't need to update q */ w -= l; } } } /* Return true (1) if we were given an incomplete table */ return y != 0 && g != 1; } int huft_free(t) struct huft *t; /* table to free */ /* Free the malloc'ed tables built by huft_build(), which makes a linked list of the tables it made, with the links in a dummy first entry of each table. */ { register struct huft *p, *q; /* Go through linked list, freeing from the malloced (t[-1]) address. */ p = t; while (p != (struct huft *)NULL) { q = (--p)->v.t; free((char*)p); p = q; } return 0; } int inflate_codes(tl, td, bl, bd) struct huft *tl, *td; /* literal/length and distance decoder tables */ int bl, bd; /* number of bits decoded by tl[] and td[] */ /* inflate (decompress) the codes in a deflated (compressed) block. Return an error code or zero if it all goes ok. */ { register unsigned e; /* table entry flag/number of extra bits */ unsigned n, d; /* length and index for copy */ unsigned w; /* current window position */ struct huft *t; /* pointer to table entry */ unsigned ml, md; /* masks for bl and bd bits */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* inflate the coded data */ ml = mask_bits[bl]; /* precompute masks for speed */ md = mask_bits[bd]; for (;;) /* do until end of block */ { NEEDBITS((unsigned)bl) if ((e = (t = tl + ((unsigned)b & ml))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) if (e == 16) /* then it's a literal */ { slide[w++] = (uch)t->v.n; Tracevv((stderr, "%c", slide[w-1])); if (w == WSIZE) { flush_output(w); w = 0; } } else /* it's an EOB or a length */ { /* exit if end of block */ if (e == 15) break; /* get length of block to copy */ NEEDBITS(e) n = t->v.n + ((unsigned)b & mask_bits[e]); DUMPBITS(e); /* decode distance of block to copy */ NEEDBITS((unsigned)bd) if ((e = (t = td + ((unsigned)b & md))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) NEEDBITS(e) d = w - t->v.n - ((unsigned)b & mask_bits[e]); DUMPBITS(e) Tracevv((stderr,"\\[%d,%d]", w-d, n)); /* do the copy */ do { n -= (e = (e = WSIZE - ((d &= WSIZE-1) > w ? d : w)) > n ? n : e); #if !defined(NOMEMCPY) && !defined(DEBUG) if (w - d >= e) /* (this test assumes unsigned comparison) */ { memcpy(slide + w, slide + d, e); w += e; d += e; } else /* do it slow to avoid memcpy() overlap */ #endif /* !NOMEMCPY */ do { slide[w++] = slide[d++]; Tracevv((stderr, "%c", slide[w-1])); } while (--e); if (w == WSIZE) { flush_output(w); w = 0; } } while (n); } } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; /* done */ return 0; } int inflate_stored() /* "decompress" an inflated type 0 (stored) block. */ { unsigned n; /* number of bytes in block */ unsigned w; /* current window position */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* go to byte boundary */ n = k & 7; DUMPBITS(n); /* get the length and its complement */ NEEDBITS(16) n = ((unsigned)b & 0xffff); DUMPBITS(16) NEEDBITS(16) if (n != (unsigned)((~b) & 0xffff)) return 1; /* error in compressed data */ DUMPBITS(16) /* read and output the compressed data */ while (n--) { NEEDBITS(8) slide[w++] = (uch)b; if (w == WSIZE) { flush_output(w); w = 0; } DUMPBITS(8) } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; return 0; } int inflate_fixed() /* decompress an inflated type 1 (fixed Huffman codes) block. We should either replace this with a custom decoder, or at least precompute the Huffman tables. */ { int i; /* temporary variable */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned l[288]; /* length list for huft_build */ /* set up literal table */ for (i = 0; i < 144; i++) l[i] = 8; for (; i < 256; i++) l[i] = 9; for (; i < 280; i++) l[i] = 7; for (; i < 288; i++) /* make a complete, but wrong code set */ l[i] = 8; bl = 7; if ((i = huft_build(l, 288, 257, cplens, cplext, &tl, &bl)) != 0) return i; /* set up distance table */ for (i = 0; i < 30; i++) /* make an incomplete code set */ l[i] = 5; bd = 5; if ((i = huft_build(l, 30, 0, cpdist, cpdext, &td, &bd)) > 1) { huft_free(tl); return i; } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_dynamic() /* decompress an inflated type 2 (dynamic Huffman codes) block. */ { int i; /* temporary variables */ unsigned j; unsigned l; /* last length */ unsigned m; /* mask for bit lengths table */ unsigned n; /* number of lengths to get */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned nb; /* number of bit length codes */ unsigned nl; /* number of literal/length codes */ unsigned nd; /* number of distance codes */ #ifdef PKZIP_BUG_WORKAROUND unsigned ll[288+32]; /* literal/length and distance code lengths */ #else unsigned ll[286+30]; /* literal/length and distance code lengths */ #endif register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in table lengths */ NEEDBITS(5) nl = 257 + ((unsigned)b & 0x1f); /* number of literal/length codes */ DUMPBITS(5) NEEDBITS(5) nd = 1 + ((unsigned)b & 0x1f); /* number of distance codes */ DUMPBITS(5) NEEDBITS(4) nb = 4 + ((unsigned)b & 0xf); /* number of bit length codes */ DUMPBITS(4) #ifdef PKZIP_BUG_WORKAROUND if (nl > 288 || nd > 32) #else if (nl > 286 || nd > 30) #endif return 1; /* bad lengths */ /* read in bit-length-code lengths */ for (j = 0; j < nb; j++) { NEEDBITS(3) ll[border[j]] = (unsigned)b & 7; DUMPBITS(3) } for (; j < 19; j++) ll[border[j]] = 0; /* build decoding table for trees--single level, 7 bit lookup */ bl = 7; if ((i = huft_build(ll, 19, 19, NULL, NULL, &tl, &bl)) != 0) { if (i == 1) huft_free(tl); return i; /* incomplete code set */ } if (tl == NULL) /* Grrrhhh */ return 2; /* read in literal and distance code lengths */ n = nl + nd; m = mask_bits[bl]; i = l = 0; while ((unsigned)i < n) { NEEDBITS((unsigned)bl) j = (td = tl + ((unsigned)b & m))->b; DUMPBITS(j) j = td->v.n; if (j < 16) /* length of code in bits (0..15) */ ll[i++] = l = j; /* save last length in l */ else if (j == 16) /* repeat last length 3 to 6 times */ { NEEDBITS(2) j = 3 + ((unsigned)b & 3); DUMPBITS(2) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = l; } else if (j == 17) /* 3 to 10 zero length codes */ { NEEDBITS(3) j = 3 + ((unsigned)b & 7); DUMPBITS(3) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } else /* j == 18: 11 to 138 zero length codes */ { NEEDBITS(7) j = 11 + ((unsigned)b & 0x7f); DUMPBITS(7) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } } /* free decoding table for trees */ huft_free(tl); /* restore the global bit buffer */ bb = b; bk = k; /* build the decoding tables for literal/length and distance codes */ bl = lbits; if ((i = huft_build(ll, nl, 257, cplens, cplext, &tl, &bl)) != 0) { if (i == 1) { fprintf(stderr, " incomplete literal tree\n"); huft_free(tl); } return i; /* incomplete code set */ } bd = dbits; if ((i = huft_build(ll + nl, nd, 0, cpdist, cpdext, &td, &bd)) != 0) { if (i == 1) { fprintf(stderr, " incomplete distance tree\n"); #ifdef PKZIP_BUG_WORKAROUND i = 0; } #else huft_free(td); } huft_free(tl); return i; /* incomplete code set */ #endif } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_block(e) int *e; /* last block flag */ /* decompress an inflated block */ { unsigned t; /* block type */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in last block bit */ NEEDBITS(1) *e = (int)b & 1; DUMPBITS(1) /* read in block type */ NEEDBITS(2) t = (unsigned)b & 3; DUMPBITS(2) /* restore the global bit buffer */ bb = b; bk = k; /* inflate that block type */ if (t == 2) return inflate_dynamic(); if (t == 0) return inflate_stored(); if (t == 1) return inflate_fixed(); /* bad block type */ return 2; } int inflate() /* decompress an inflated entry */ { int e; /* last block flag */ int r; /* result code */ unsigned h; /* maximum struct huft's malloc'ed */ /* initialize window, bit buffer */ wp = 0; bk = 0; bb = 0; /* decompress until the last block */ h = 0; do { hufts = 0; if ((r = inflate_block(&e)) != 0) return r; if (hufts > h) h = hufts; } while (!e); /* Undo too much lookahead. The next read will be byte aligned so we * can discard unused bits in the last meaningful byte. */ while (bk >= 8) { bk -= 8; inptr--; } /* flush out slide */ flush_output(wp); /* return success */ #ifdef DEBUG fprintf(stderr, "<%u> ", h); #endif /* DEBUG */ return 0; } Index: releng/5.4/gnu/usr.bin/gzip/unlzh.c =================================================================== --- releng/5.4/gnu/usr.bin/gzip/unlzh.c (revision 162446) +++ releng/5.4/gnu/usr.bin/gzip/unlzh.c (revision 162447) @@ -1,401 +1,405 @@ /* unlzh.c -- decompress files in SCO compress -H (LZH) format. * The code in this file is directly derived from the public domain 'ar002' * written by Haruhiko Okumura. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #include "gzip.h" #include "lzw.h" /* just for consistency checking */ /* decode.c */ local unsigned decode OF((unsigned count, uch buffer[])); local void decode_start OF((void)); /* huf.c */ local void huf_decode_start OF((void)); local unsigned decode_c OF((void)); local unsigned decode_p OF((void)); local void read_pt_len OF((int nn, int nbit, int i_special)); local void read_c_len OF((void)); /* io.c */ local void fillbuf OF((int n)); local unsigned getbits OF((int n)); local void init_getbits OF((void)); /* maketbl.c */ local void make_table OF((int nchar, uch bitlen[], int tablebits, ush table[])); #define DICBIT 13 /* 12(-lh4-) or 13(-lh5-) */ #define DICSIZ ((unsigned) 1 << DICBIT) #ifndef CHAR_BIT # define CHAR_BIT 8 #endif #ifndef UCHAR_MAX # define UCHAR_MAX 255 #endif #define BITBUFSIZ (CHAR_BIT * 2 * sizeof(char)) /* Do not use CHAR_BIT * sizeof(bitbuf), does not work on machines * for which short is not on 16 bits (Cray). */ /* encode.c and decode.c */ #define MAXMATCH 256 /* formerly F (not more than UCHAR_MAX + 1) */ #define THRESHOLD 3 /* choose optimal value */ /* huf.c */ #define NC (UCHAR_MAX + MAXMATCH + 2 - THRESHOLD) /* alphabet = {0, 1, 2, ..., NC - 1} */ #define CBIT 9 /* $\lfloor \log_2 NC \rfloor + 1$ */ #define CODE_BIT 16 /* codeword length */ #define NP (DICBIT + 1) #define NT (CODE_BIT + 3) #define PBIT 4 /* smallest integer such that (1U << PBIT) > NP */ #define TBIT 5 /* smallest integer such that (1U << TBIT) > NT */ #if NT > NP # define NPT NT #else # define NPT NP #endif /* local ush left[2 * NC - 1]; */ /* local ush right[2 * NC - 1]; */ #define left prev #define right head #if NC > (1<<(BITS-2)) error cannot overlay left+right and prev #endif /* local uch c_len[NC]; */ #define c_len outbuf #if NC > OUTBUFSIZ error cannot overlay c_len and outbuf #endif local uch pt_len[NPT]; local unsigned blocksize; local ush pt_table[256]; /* local ush c_table[4096]; */ #define c_table d_buf #if (DIST_BUFSIZE-1) < 4095 error cannot overlay c_table and d_buf #endif /*********************************************************** io.c -- input/output ***********************************************************/ local ush bitbuf; local unsigned subbitbuf; local int bitcount; local void fillbuf(n) /* Shift bitbuf n bits left, read n bits */ int n; { bitbuf <<= n; while (n > bitcount) { bitbuf |= subbitbuf << (n -= bitcount); subbitbuf = (unsigned)try_byte(); if ((int)subbitbuf == EOF) subbitbuf = 0; bitcount = CHAR_BIT; } bitbuf |= subbitbuf >> (bitcount -= n); } local unsigned getbits(n) int n; { unsigned x; x = bitbuf >> (BITBUFSIZ - n); fillbuf(n); return x; } local void init_getbits() { bitbuf = 0; subbitbuf = 0; bitcount = 0; fillbuf(BITBUFSIZ); } /*********************************************************** maketbl.c -- make table for decoding ***********************************************************/ local void make_table(nchar, bitlen, tablebits, table) int nchar; uch bitlen[]; int tablebits; ush table[]; { ush count[17], weight[17], start[18], *p; unsigned i, k, len, ch, jutbits, avail, nextcode, mask; for (i = 1; i <= 16; i++) count[i] = 0; - for (i = 0; i < (unsigned)nchar; i++) count[bitlen[i]]++; + for (i = 0; i < (unsigned)nchar; i++) { + if (bitlen[i] > 16) + error("Bad table (case a)\n"); + else count[bitlen[i]]++; + } start[1] = 0; for (i = 1; i <= 16; i++) start[i + 1] = start[i] + (count[i] << (16 - i)); - if ((start[17] & 0xffff) != 0) - error("Bad table\n"); + if ((start[17] & 0xffff) != 0 || tablebits > 16) /* 16 for weight below */ + error("Bad table (case b)\n"); jutbits = 16 - tablebits; for (i = 1; i <= (unsigned)tablebits; i++) { start[i] >>= jutbits; weight[i] = (unsigned) 1 << (tablebits - i); } while (i <= 16) { weight[i] = (unsigned) 1 << (16 - i); i++; } i = start[tablebits + 1] >> jutbits; if (i != 0) { - k = 1 << tablebits; - while (i != k) table[i++] = 0; + k = MIN(1 << tablebits, DIST_BUFSIZE); + while (i < k) table[i++] = 0; } avail = nchar; mask = (unsigned) 1 << (15 - tablebits); for (ch = 0; ch < (unsigned)nchar; ch++) { if ((len = bitlen[ch]) == 0) continue; - nextcode = start[len] + weight[len]; + nextcode = MIN(start[len] + weight[len], DIST_BUFSIZE); if (len <= (unsigned)tablebits) { for (i = start[len]; i < nextcode; i++) table[i] = ch; } else { k = start[len]; p = &table[k >> jutbits]; i = len - tablebits; while (i != 0) { if (*p == 0) { right[avail] = left[avail] = 0; *p = avail++; } if (k & mask) p = &right[*p]; else p = &left[*p]; k <<= 1; i--; } *p = ch; } start[len] = nextcode; } } /*********************************************************** huf.c -- static Huffman ***********************************************************/ local void read_pt_len(nn, nbit, i_special) int nn; int nbit; int i_special; { int i, c, n; unsigned mask; n = getbits(nbit); if (n == 0) { c = getbits(nbit); for (i = 0; i < nn; i++) pt_len[i] = 0; for (i = 0; i < 256; i++) pt_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NPT)) { c = bitbuf >> (BITBUFSIZ - 3); if (c == 7) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 3); while (mask & bitbuf) { mask >>= 1; c++; } } fillbuf((c < 7) ? 3 : c - 3); pt_len[i++] = c; if (i == i_special) { c = getbits(2); - while (--c >= 0) pt_len[i++] = 0; + while (--c >= 0 && i < NPT) pt_len[i++] = 0; } } while (i < nn) pt_len[i++] = 0; make_table(nn, pt_len, 8, pt_table); } } local void read_c_len() { int i, c, n; unsigned mask; n = getbits(CBIT); if (n == 0) { c = getbits(CBIT); for (i = 0; i < NC; i++) c_len[i] = 0; for (i = 0; i < 4096; i++) c_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NC)) { c = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (c >= NT) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) c = right[c]; else c = left [c]; mask >>= 1; - } while (c >= NT); + } while (c >= NT && (mask || c != left[c])); } fillbuf((int) pt_len[c]); if (c <= 2) { if (c == 0) c = 1; else if (c == 1) c = getbits(4) + 3; else c = getbits(CBIT) + 20; - while (--c >= 0) c_len[i++] = 0; + while (--c >= 0 && i < NC) c_len[i++] = 0; } else c_len[i++] = c - 2; } while (i < NC) c_len[i++] = 0; make_table(NC, c_len, 12, c_table); } } local unsigned decode_c() { unsigned j, mask; if (blocksize == 0) { blocksize = getbits(16); if (blocksize == 0) { return NC; /* end of file */ } read_pt_len(NT, TBIT, 3); read_c_len(); read_pt_len(NP, PBIT, -1); } blocksize--; j = c_table[bitbuf >> (BITBUFSIZ - 12)]; if (j >= NC) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 12); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NC); + } while (j >= NC && (mask || j != left[j])); } fillbuf((int) c_len[j]); return j; } local unsigned decode_p() { unsigned j, mask; j = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (j >= NP) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NP); + } while (j >= NP && (mask || j != left[j])); } fillbuf((int) pt_len[j]); if (j != 0) j = ((unsigned) 1 << (j - 1)) + getbits((int) (j - 1)); return j; } local void huf_decode_start() { init_getbits(); blocksize = 0; } /*********************************************************** decode.c ***********************************************************/ local int j; /* remaining bytes to copy */ local int done; /* set at end of input */ local void decode_start() { huf_decode_start(); j = 0; done = 0; } /* Decode the input and return the number of decoded bytes put in buffer */ local unsigned decode(count, buffer) unsigned count; uch buffer[]; /* The calling function must keep the number of bytes to be processed. This function decodes either 'count' bytes or 'DICSIZ' bytes, whichever is smaller, into the array 'buffer[]' of size 'DICSIZ' or more. Call decode_start() once for each new file before calling this function. */ { local unsigned i; unsigned r, c; r = 0; while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } for ( ; ; ) { c = decode_c(); if (c == NC) { done = 1; return r; } if (c <= UCHAR_MAX) { buffer[r] = c; - if (++r == count) return r; + if (++r >= count) return r; } else { j = c - (UCHAR_MAX + 1 - THRESHOLD); i = (r - decode_p() - 1) & (DICSIZ - 1); while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } } } } /* =========================================================================== * Unlzh in to out. Return OK or ERROR. */ int unlzh(in, out) int in; int out; { unsigned n; ifd = in; ofd = out; decode_start(); while (!done) { n = decode((unsigned) DICSIZ, window); if (!test && n > 0) { write_buf(out, (char*)window, n); } } return OK; } Index: releng/5.4/gnu/usr.bin/gzip/unpack.c =================================================================== --- releng/5.4/gnu/usr.bin/gzip/unpack.c (revision 162446) +++ releng/5.4/gnu/usr.bin/gzip/unpack.c (revision 162447) @@ -1,239 +1,238 @@ /* unpack.c -- decompress files in pack format. * Copyright (C) 1992-1993 Jean-loup Gailly * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include "tailor.h" #include "gzip.h" #include "crypt.h" -#define MIN(a,b) ((a) <= (b) ? (a) : (b)) /* The arguments must not have side effects. */ #define MAX_BITLEN 25 /* Maximum length of Huffman codes. (Minor modifications to the code * would be needed to support 32 bits codes, but pack never generates * more than 24 bits anyway.) */ #define LITERALS 256 /* Number of literals, excluding the End of Block (EOB) code */ #define MAX_PEEK 12 /* Maximum number of 'peek' bits used to optimize traversal of the * Huffman tree. */ local ulg orig_len; /* original uncompressed length */ local int max_len; /* maximum bit length of Huffman codes */ local uch literal[LITERALS]; /* The literal bytes present in the Huffman tree. The EOB code is not * represented. */ local int lit_base[MAX_BITLEN+1]; /* All literals of a given bit length are contiguous in literal[] and * have contiguous codes. literal[code+lit_base[len]] is the literal * for a code of len bits. */ local int leaves [MAX_BITLEN+1]; /* Number of leaves for each bit length */ local int parents[MAX_BITLEN+1]; /* Number of parents for each bit length */ local int peek_bits; /* Number of peek bits currently used */ /* local uch prefix_len[1 << MAX_PEEK]; */ #define prefix_len outbuf /* For each bit pattern b of peek_bits bits, prefix_len[b] is the length * of the Huffman code starting with a prefix of b (upper bits), or 0 * if all codes of prefix b have more than peek_bits bits. It is not * necessary to have a huge table (large MAX_PEEK) because most of the * codes encountered in the input stream are short codes (by construction). * So for most codes a single lookup will be necessary. */ #if (1< OUTBUFSIZ error cannot overlay prefix_len and outbuf #endif local ulg bitbuf; /* Bits are added on the low part of bitbuf and read from the high part. */ local int valid; /* number of valid bits in bitbuf */ /* all bits above the last valid bit are always zero */ /* Set code to the next 'bits' input bits without skipping them. code * must be the name of a simple variable and bits must not have side effects. * IN assertions: bits <= 25 (so that we still have room for an extra byte * when valid is only 24), and mask = (1<> (valid-(bits))) & (mask); \ } /* Skip the given number of bits (after having peeked at them): */ #define skip_bits(bits) (valid -= (bits)) #define clear_bitbuf() (valid = 0, bitbuf = 0) /* Local functions */ local void read_tree OF((void)); local void build_tree OF((void)); /* =========================================================================== * Read the Huffman tree. */ local void read_tree() { int len; /* bit length */ int base; /* base offset for a sequence of leaves */ int n; /* Read the original input size, MSB first */ orig_len = 0; for (n = 1; n <= 4; n++) orig_len = (orig_len << 8) | (ulg)get_byte(); max_len = (int)get_byte(); /* maximum bit length of Huffman codes */ if (max_len > MAX_BITLEN) { error("invalid compressed data -- Huffman code > 32 bits"); } /* Get the number of leaves at each bit length */ n = 0; for (len = 1; len <= max_len; len++) { leaves[len] = (int)get_byte(); n += leaves[len]; } if (n > LITERALS) { error("too many leaves in Huffman tree"); } Trace((stderr, "orig_len %ld, max_len %d, leaves %d\n", orig_len, max_len, n)); /* There are at least 2 and at most 256 leaves of length max_len. * (Pack arbitrarily rejects empty files and files consisting of * a single byte even repeated.) To fit the last leaf count in a * byte, it is offset by 2. However, the last literal is the EOB * code, and is not transmitted explicitly in the tree, so we must * adjust here by one only. */ leaves[max_len]++; /* Now read the leaves themselves */ base = 0; for (len = 1; len <= max_len; len++) { /* Remember where the literals of this length start in literal[] : */ lit_base[len] = base; /* And read the literals: */ - for (n = leaves[len]; n > 0; n--) { + for (n = leaves[len]; n > 0 && base < LITERALS; n--) { literal[base++] = (uch)get_byte(); } } leaves[max_len]++; /* Now include the EOB code in the Huffman tree */ } /* =========================================================================== * Build the Huffman tree and the prefix table. */ local void build_tree() { int nodes = 0; /* number of nodes (parents+leaves) at current bit length */ int len; /* current bit length */ uch *prefixp; /* pointer in prefix_len */ for (len = max_len; len >= 1; len--) { /* The number of parent nodes at this level is half the total * number of nodes at parent level: */ nodes >>= 1; parents[len] = nodes; /* Update lit_base by the appropriate bias to skip the parent nodes * (which are not represented in the literal array): */ lit_base[len] -= nodes; /* Restore nodes to be parents+leaves: */ nodes += leaves[len]; } /* Construct the prefix table, from shortest leaves to longest ones. * The shortest code is all ones, so we start at the end of the table. */ peek_bits = MIN(max_len, MAX_PEEK); prefixp = &prefix_len[1< prefix_len) *--prefixp = (uch)len; } /* The length of all other codes is unknown: */ while (prefixp > prefix_len) *--prefixp = 0; } /* =========================================================================== * Unpack in to out. This routine does not support the old pack format * with magic header \037\037. * * IN assertions: the buffer inbuf contains already the beginning of * the compressed data, from offsets inptr to insize-1 included. * The magic header has already been checked. The output buffer is cleared. */ int unpack(in, out) int in, out; /* input and output file descriptors */ { int len; /* Bit length of current code */ unsigned eob; /* End Of Block code */ register unsigned peek; /* lookahead bits */ unsigned peek_mask; /* Mask for peek_bits bits */ ifd = in; ofd = out; read_tree(); /* Read the Huffman tree */ build_tree(); /* Build the prefix table */ clear_bitbuf(); /* Initialize bit input */ peek_mask = (1< 0) { peek >>= peek_bits - len; /* discard the extra bits */ } else { /* Code of more than peek_bits bits, we must traverse the tree */ ulg mask = peek_mask; len = peek_bits; do { len++, mask = (mask<<1)+1; look_bits(peek, len, mask); } while (peek < (unsigned)parents[len]); /* loop as long as peek is a parent node */ } /* At this point, peek is the next complete code, of len bits */ if (peek == eob && len == max_len) break; /* end of file? */ put_ubyte(literal[peek+lit_base[len]]); Tracev((stderr,"%02d %04x %c\n", len, peek, literal[peek+lit_base[len]])); skip_bits(len); } /* for (;;) */ flush_window(); Trace((stderr, "bytes_out %ld\n", bytes_out)); if (orig_len != (ulg)bytes_out) { error("invalid compressed data--length error"); } return OK; } Index: releng/5.4/sys/conf/newvers.sh =================================================================== --- releng/5.4/sys/conf/newvers.sh (revision 162446) +++ releng/5.4/sys/conf/newvers.sh (revision 162447) @@ -1,97 +1,97 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="5.4" -BRANCH="RELEASE-p18" +BRANCH="RELEASE-p19" RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ $(dirname $0)/../sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/1992-$year/" \ -e 's/\[your name here\]\.* /The FreeBSD Project./' \ -e 's/\[your name\]\.*/The FreeBSD Project./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/*- * Copyright (c) 1992-$year The FreeBSD Project. * All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} t=`date` i=`${MAKE:-make} -V KERN_IDENT` cat << EOF > vers.c $COPYRIGHT char sccspad[32 - 4 /* sizeof(sccs) */] = { '\\0' }; char sccs[4] = { '@', '(', '#', ')' }; char version[] = "${VERSION} #${v}: ${t}\\n ${u}@${h}:${d}\\n"; char ostype[] = "${TYPE}"; char osrelease[] = "${RELEASE}"; int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF echo `expr ${v} + 1` > version Index: releng/5.5/UPDATING =================================================================== --- releng/5.5/UPDATING (revision 162446) +++ releng/5.5/UPDATING (revision 162447) @@ -1,2076 +1,2079 @@ Updating Information for FreeBSD stable users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. +20060919: p5 FreeBSD-SA-06:21.gzip + Correct multiple vulnerabilities in gzip(1). + 20060906: p4 FreeBSD-SA-06:19.openssl, FreeBSD-SA-06:20.bind Correct incorrect PKCS#1 v1.5 padding validation in crypto(3). [06:19] Correct multiple denial-of-service vulnerabilities in BIND related to SIG Query Processing and Excessive Recursive Queries. [06:20] 20060823: p3 FreeBSD-SA-06:18.ppp Correct buffer overflow in the handling of LCP options in ppp(4). 20060614: p2 FreeBSD-SA-06:17.sendmail Correct a bug in the handling of multipart messages by sendmail(8) which can allow a malformed message to crash a sendmail queue processing process. 20060531: p1 FreeBSD-SA-06:15.ypserv, FreeBSD-SA-06:16.smbfs Enable inadvertantly disabled "securenet" access controls in ypserv. [06:15] Correct a bug in the handling of backslash characters in smbfs which can allow an attacker to escape from a chroot(2). [06:16] 20060525: FreeBSD 5.5-RELEASE 20060127: The i386 loader(8) now defaults to the serial speed set by the previous boot stage, if the comconsole is already in use. If you've changed BOOT_COMCONSOLE_SPEED in make.conf(5) and installed a new loader, but have not rebuilt and reinstalled the boot blocks, then your loader will leave the console at 9600 baud. Either install the new boot blocks, or set comconsole_speed in loader.conf(5). Note that the new boot blocks also support a -S flag described in boot(8). 20060111: Bug fixes to the trimdomain(3) function in libutil may result in slight changes to the host names appearing in log files under relatively rare circumstances. 20051015: The shared library version number of libarchive was accidentally bumped up during an MFC of other changes on October 4, 2005. It was reset to the correct value on October 14, 2005. If you updated a system between those dates you should check to see if /usr/lib/libarchive.so.2 exists and remove it AFTER doing your next update. 20051004: if_bridge has been merged which provides more advanced Ethernet bridging and 802.1d spanning tree support. See if_bridge(4) for configuration details. 20050513: Intel Hyper-Threading is now disabled by default due to a security issue, but can be re-enabled by setting the machdep.hyperthreading_allowed tunable in /boot/loader.conf. 20050509: FreeBSD 5.4-RELEASE 20050302: The full packet destination manipulation for 'ipfw fwd' is back with the kernel compile time option: options IPFIREWALL_FORWARD_EXTENDED This option has to be specified in addition to IPFIREWALL_FORWARD. With this option even packets targeted for an IP address local to the host can be redirected. All restrictions to ensure proper behaviour for locally generated packets are turned off. Firewall rules have to be carefully crafted to make sure that things like PMTU discovery do not break. 20050228: The responsibility of recomputing the file system summary of a SoftUpdates-enabled dirty volume has been transferred to the background fsck. A rebuild of fsck(8) utility is recommended if you have updated the kernel. To get the old behavior (recompute file system summary at mount time), you can set vfs.ffs.compute_summary_at_mount=1 before mounting the new volume. 20050228: The ifi_epoch member of struct if_data has been changed to contain the uptime at which the interface was created or the statistics zeroed rather then the wall clock time because wallclock time may go backwards. This should have no impact unless an snmp implementation is using this value (I know of none at this point.) 20050227: The default "world" build no longer supports running on an 80386 CPU. In order to build a world for an 80386 CPU, one needs to set CPUTYPE=i386 in /etc/make.conf. 20050225: The cpufreq framework has been merged. As part of this, the sysctls for acpi(4) throttling have been removed. See cpufreq(4) for the new sysctl interface. The power_profile script has also been updated, so you can use performance/economy_cpu_freq in rc.conf(5) to set AC on/offline cpu frequencies. No new cpufreq drivers have been brought in with this import but drivers from -current can now be built and run on -stable. 20050215: [applicable to sparc64 only] uart(4) is the default serial console now. When using the GENERIC kernel, make sure to start getty(8) processes on ttyu0 and/or ttyu1. Either run mergemaster(8) or edit /etc/ttys manually. 20050203: ppp(8) no longer reverts to sending ECHO requests if LQR is enabled but fails negotiation. To enable the old behaviour, add ``enable echo'' to your config. Additionally, when RADIUS is enabled, ppp now sends both NAS-IP-Address and NAS-Identifier by default. This behaviour can be modified with the ``enable/disable NAS-IP-Address/NAS-Identifier'' command. 20041126: The numbers of arguments for ng_timeout() and ng_untimeout() have changed. If netgraph.ko is rebuilt or if netgraph is statically compiled in kernel, then the following modules must also be rebuilt: ng_hci ng_l2cap ng_source ng_sscop ng_uni 20041024: FreeBSD 5.3-RELEASE 20041023: GENERIC kernel config files for amd64 and i386 no longer have "options SMP". That has been moved to a new config file named SMP for users who want an easy way to build an SMP kernel. 20041010: The FreeBSD keyword is no longer a requirement for a valid rc.d script. The rc(8) and rc.shutdown(8) scripts no longer check for the existence of this keyword when ordering rc.d scripts. This change touches most of the file in /etc/rc.d; therefore, if you have not modified any files in that directory it may be easier to rm -rf /etc/rc.d and then use the -i switch with mergemaster(8). 20041009: One of the syscalls the 1:1 threading library libthr uses has changed, thus breaking ABI compatibility. Make sure you rebuild this library with the kernel. 20041003: The pfil API has gained an additional argument to pass an inpcb. You should rebuild all pfil consuming modules: ipfw, ipfilter and pf. 20041001: The following libraries had their version number bumped up: /lib/libm.so.2 -> libm.so.3 /lib/libreadline.so.4 -> libreadline.so.5 /usr/lib/libhistory.so.4 -> libhistory.so.5 /usr/lib/libopie.so.2 -> libopie.so.3 /usr/lib/libpcap.so.2 -> libpcap.so.3 FreeBSD 4.10 versions of these libraries will be added to the compat4x collection. If you expect to be able to run old 4.X executables you will need to remove the old versions of these libraries. However note that any 5.X executables you have built will stop working once you remove those old libraries. You should have all your ports/packages rebuilt before removing the old libraries. 20040928: If enabled, the default is now to run named in a chroot "sandbox." For users with existing configurations in /etc/namedb the migration should be simple. Upgrade your world as usual, then after installworld but before mergemaster do the following: If named is running: /etc/rc.d/named stop cd /etc mv namedb namedb.bak mkdir -p /var/named/etc/namedb cp -Rp namedb.bak/* /var/named/etc/namedb/ mergemaster (with your usual options) If using the generated localhost* files: cd /var/named/etc/namedb /bin/sh make-localhost rm -f localhost-v6.rev localhost.rev /etc/rc.d/syslogd restart /etc/rc.d/named start If you are using a custom configuration, or if you have customised the named_* variables in /etc/rc.conf[.local] then you may have to adjust the instructions accordingly. It is suggested that you carefully examine the new named variables in /etc/defaults/rc.conf and the options in /var/named/etc/namedb/named.conf to see if they might now be more suitable. 20040925: BIND 9 has been imported into the base, and is now fully functional. BIND 8 has now been removed. There are numerous differences between BIND 8 and 9, and users with critical named installations should read the migration documentation in /usr/share/doc/bind9/misc/migration. There is also a new instruction manual in /usr/share/doc/bind9/arm. The key differences that most users will experience are in how picky BIND 9 is about zone file format. If you are using named as a resolving (caching) name server, you will likely not have trouble. The following files are part of the old BIND 8 installation and should be removed: /usr/bin/dnskeygen /usr/bin/dnsquery /usr/libexec/named-xfer /usr/sbin/named.restart /usr/sbin/ndc The following files have moved to /usr/bin, and should be removed from their old locations: /usr/sbin/nslookup /usr/sbin/nsupdate 20040922: PFIL_HOOKS are a fixed part of the network stack now and do not need to be specified in the kernel configuration file anymore. Remove 'options PFIL_HOOKS' from your kernel config file. 20040914: The format of the pflogd(8) logfile "/var/log/pflog" has changed for architectures that have a 64 bit long type to make it compatible to the standard pcap format. In order to prevent corruption move away any old logfile before using a new pflogd(8). 20040913: debug.witness_* has been renamed to debug.witness.*. There are compatibility tunables left in for a few days. Update loader.conf as necessary. 20040907: The kernel and userland debugging options have been turned off by default. 20040902: The default configuration for the network stack has been changed such that it now runs without the Giant lock unless configured otherwise. If you experience network-related instability, you may wish to try setting "debug.mpsafenet=0" or compiling the kernel with "options NET_WITH_GIANT". Details on the netperf project may be found at: http://www.watson.org/~robert/freebsd/netperf/ Including the 20040828 announcement of configuration change details. 20040826: Netgraph changed its message format slightly to align the data portion well on 64 bit machines. Netgraph using utilities (e.g. ngctl, nghook, ppp, mpd, pppoed, bluetooth, ATM) should be recompiled when a new kernel is installed. 20040817: IPFW has been converted to use PFIL_HOOKS. This change is transparent to userland and preserves the ipfw ABI. The ipfw core packet inspection and filtering functions have not been changed, only how ipfw is invoked is different. Note that "option PFIL_HOOKS" is required to use IPFIREWALL compiled into the kernel or as KLD. 20040814: The RANDOM_IP_ID option has been replaced by the sysctl net.inet.ip.random_id. If you had RANDOM_IP_ID in your kernel then you may want to add "net.inet.ip.random_id=1" to /etc/sysctl.conf. 20040807: The size of 'struct ifnet' has changed due to the addition of the if_carp placeholder. All kernel modules implementing network interfaces must be recompiled as a result. 20040806: Module loading has been fixed. Some older installations will drop proper module_path initialization and modules will fail to load properly. If you have a line in /boot/loader.rc that says: "initialize drop", do (i386 only): cp /usr/src/sys/boot/i386/loader/loader.rc /boot/loader.rc chown root:wheel /boot/loader.rc chmod 444 /boot/loader.rc 20040802: making /dev/(null|zero) into a module proved to be too unpopular, so this bit has been revoked from the previous (20040801) entry. 20040801: The /dev/mem, /dev/io /dev/(null/zero) devices are now modules, so you may wish to add them to your kernel config file. See GENERIC for examples. 20040728: System compiler has been upgraded to GCC 3.4.2-pre. As with any major compiler upgrade, there are several issues to be aware of. GCC 3.4.x has broken C++ ABI compatibility with previous releases yet again and users will have to rebuild all their C++ programs with the new compiler. A new unit-at-a-time optimization mode, which is default in this compiler release, is more aggressive in removing unused static symbols. This is the likely cause of 'make buildworld' breakages with non-default CFLAGS where optimization level is set to -O2 or higher. With the upgrade of the system compiler, the kernel has been upgraded to match the new system compiler. This makes it impossible to build a new kernel with the old compiler. Upgrade your system via make buildworld and make kernel (see below) to fix this problem. 20040727: The size of 'struct ifnet' has changed due to the addition of the IFF_NEEDSGIANT flag (and what it implies). All kernel modules implementing network interfaces must be recompiled as a result. 20040716: The sound device drivers are renamed. `sound' is always required, while `snd_*' should be configured accordingly to your hardware. Refer to NOTES for the detail of the drivers. 20040710: __FreeBSD_version bumped to 502122. 20040710: The console initialization on Alpha has been reworked and is now identical to other platforms. This means that the hardcoding of the serial console and the debug port has been removed. As such, hints are now required for the sio(4) driver to become a console or debug port. The NO_SIO option has been decommissioned because of this. 20040710: A revamp of the debugging code in the kernel with some visible changes beyond just the debugging experience: o The DDB option is now specific to the DDB debugger backend and should not be used any more for conditional compilation of debugging code for when debugging is enabled. Use the KDB option for this. o The WITNESS_DDB, DDB_TRACE and DDB_UNATTENDED options have been renamed to WITNESS_KDB, KDB_TRACE and KDB_UNATTENDED respectively. This is in line with the first bullet. o The remote GDB support has been untangled from DDB and needs to be enabled separately now. Use the GDB option for this. o The GDB_REMOTE_CHAT option has been removed. Support for this homegrown feature is discontinued. The GDB remote protocol supports console output and it makes sense to use that. o The DDB_NOKLDSYM option has been removed. The DDB debugger now supports both direct symbol table lookups as well as KLD symbol lookups through the linker. 20040708: Bluetooth code has been marked as non-i386 specific. __FreeBSD_version has been bumped to 502121 to mark this change. 20040702: The native preemption has been added to the kernel scheduler. There is some report that the ULE scheduler was broken in some machines and we encourage users using the ULE scheduler either stick with a known good kernel, or temporarily switch to the 4BSD scheduler as a workaround. 20040630: The netgraph ABI version number has been incremented to indicate an incompatible change in the ABI. Old netgraph nodes will refuse to attach until recompiled. Netgraph now uses mbuf tags to move metadata and this commit removes its home-grown metadata facility. Nodes should just recompile, unless they use metadata, in which case the changes are simple; the file ng_ksocket.c serves as an example of such changes. This also broke i4b, although the compile problem has been papered over. 20040630: ACPI has been updated to disable known-bad BIOS revisions. A message will be printed on the console indicating that ACPI has been disabled automatically and that the user should use a newer BIOS, if possible. If you think ACPI does work on your system and want to override this (i.e., for testing), set hint.acpi.0.disabled="0" at the loader prompt. 20040623: pf was updated to OpenBSD-stable 3.5 and pflogd(8) is privilege separated now. It uses the newly created "_pflogd" user/group combination. If you plan to use pflogd(8) make sure to run mergemaster -p or install the "_pflogd" user and group manually. 20040622: Network interface cloning has been overhauled. This change will require a recompile of modules using cloning and modification of external ones to the new API. __FreeBSD_version has been bumped to 502119 to mark this change. Additionally, users creating stf(4) interfaces via "ifconfig stf" will need to update their scripts as this will create an interface named "stf" instead of "stf0" and ifconfig will not print "stf0" to stdout. 20040621: On 20040524, the /etc/rc.d/nsswitch script was modified to automatically create /etc/nsswitch.conf on startup if it did not already exist. Unfortunately, an error in the man page was carried over to the script, resulting in incorrect nsswitch settings. The simplest remedy is to remove both /etc/nsswitch.conf and /etc/host.conf; they will be recreated during the next reboot. 20040614: The return value of sema_timedwait(9) has been changed to make it consistent with cv_timedwait(9). Be sure to recompile the ips module and any third-party modules which call sema_timedwait. 20040613: ALTQ is now linked to the build. This breaks ABI for struct ifnet. Make sure to recompile modules and any userland that makes use of sizeof(struct ifnet). In order to get the altq headers in place please recompile and reinstall world. 20040607: Splitting kern_thread.c into 2 files (adding kern_kse.c) requires that you re-run config after updating your tree. 20040601: The MIDI drivers have been removed. Until the new module-friendly ones are merged, remove or comment out midi and seq from your kernel configuration. 20040423: Due to a new option in ipfw (versrcreach) the ipfw(8) command needs to be recompiled. Normal accept/reject rules without options are not affected but those with options may break until ipfw(8) is recompiled. 20040420: Due to changes in the callout ABI, kernels compiled after this date may be incompatible with kernel modules compiled prior to 20040406. 20040414: The PCI bus power state stuff has been turned on. If this causes problems for your system, please disable it using the tunable hw.pci.do_powerstate=0. 20040412: The bulk of the pci problems have been fixed, although the floppy drive is still broken. 20040410: A substantial update to the pci bus resource and power management have been committed. Expect a bumpy ride for a few days until the unanticipated problems have been resolved. 20040409: Due to changes in the the Yarrow initialization process, /dev/random needs to be fed before operations requiring temp files can succeed in single user mode. This includes running "make installworld". /dev/random may be fed by running "/etc/rc.d/initrandom start" or with 20040415 source by running "/etc/rc.d/preseedrandom". 20040322: The debug.mpsafenet tunable controls whether the kernel Giant lock is held across the lower levels of the network stack, and by default is turned off. In the few days following 20040322, the behavior of debug.mpsafenet will change such that this tunable controls Giant over all levels of the network stack. If you are currently setting debug.mpsafenet to 1, you should set it back to 0 (the default) again during the change-over. An additional note will be added to UPDATING when sufficient locking is merged to permit this to take place. 20040310: The FreeBSD/sparc64 platform is changing time_t from 32-bits to 64-bits. This is a very major incompatible change, so people using FreeBSD/sparc64 *must* read the UPDATING.64BTT file for detailed instructions on how to make this upgrade. People upgrading FreeBSD on other platforms can ignore this event. 20040308: The packet filter (pf) is now installed with the base system. Make sure to run mergemaster -p before installworld to create required user account ("proxy"). If you do not want to build pf with your system you can use the NO_PF knob in make.conf. Also note that pf requires "options PFIL_HOOKS" in the kernel. The pf system consists of the following three devices: device pf # required device pflog # optional device pfsync # optional 20040303: If you are having trouble with the libc_r -> libpthread transition (see the 20040130 entry), place the following lines at the top of /etc/libmap.conf: libc_r.so.5 libpthread.so.1 libc_r.so libpthread.so This will cause all programs and libraries linked against libc_r to use libpthread instead. 20040226: Some sshd configuration defaults have changed: protocol version 1 is no longer enabled by default, and password authentication is disabled by default if PAM is enabled (which it is by default). OpenSSH clients should not be affected by this; other clients may have to be reconfigured, upgraded or replaced. 20040225: The ABIs defined in and have been updated to support improved reentrancy. Multi-threaded programs that reference the "_res" or "h_errno" symbols may experience some problems if they are not recompiled. Single-threaded programs should remain unaffected. 20040225: routed has been updated in the base system from the vendor sources, routed v2.27, from rhyolite.com. This change means that for users who use RIP's MD5 authentication feature, FreeBSD -CURRENT's routed is now incompatible with previous versions of FreeBSD; however it is now compatible with implementations from Sun, Cisco and other vendors. 20040224: The tcpcb structure has changed and makes a recompile of libkvm and related userland network utilities necessary. 20040222: The cdevsw structure has changed in two externally visible ways. First, the sense of the D_GIANT flag has changed to D_NEEDSGIANT. Second, the d_version field must be filled in with D_VERSION. Drivers outside the tree will need to be updated. 20040207: The /etc/rc.d/ttys script has been removed. It is no longer necessary since devfs has been mandatory for some time. 20040130: libkse has been renamed back to libpthread and is now the default threads library. The gcc -pthread option has also been changed to link to libpthread instead of libc_r. For alpha and sparc64 machines, libkse is not renamed and links are installed so that libpthread points to libc_r. Until the ports system is updated to handle this change, it is recommended that folks install an /etc/libmap.conf(5) that maps libc_r to libpthread. If you have any binaries or libraries linked to libkse, then it is also recommended that you map libkse to libpthread. Anyone that is using nvidia supplied drivers and libraries should use a libmap.conf that maps libpthread to libc_r since their drivers/libraries do not work with libpthread. 20040125: ULE has entered into its probationary period as the default scheduler in GENERIC. For the average user, interactivity is reported to be better in many cases. On SMP machines ULE will be able to make more efficient use of the available parallel resources. If you are not running it now, please switch over, replacing the kernel option SCHED_4BSD with SCHED_ULE. 20040125: Move LongRun support out of identcpu.c, where it hardly belongs, into its own file and make it opt-in, not mandatory, depending on CPU_ENABLE_LONGRUN config(8) option. 20031213: src/lib/libc/gen/initgroups.c:1.8 now causes logins to fail if the login process is unable to successfully set the process credentials to include all groups defined for the user. The current kernel limit is 16 groups; administrators may wish to check that users do not have over 16 groups defined, or they will be unable to log in. 20031203: The ACPI module has been reactivated. It is no longer required to compile ACPI support into kernels statically. 20031112: The statfs structure has been updated with 64-bit fields to allow accurate reporting of multi-terabyte filesystem sizes. You should build world, then build and boot the new kernel BEFORE doing a `installworld' as the new kernel will know about binaries using the old statfs structure, but an old kernel will not know about the new system calls that support the new statfs structure. Note that the backwards compatibility is only present when the kernel is configured with the COMPAT_FREEBSD4 option. Since even /bin/sh will not run with a new kernel without said option you're pretty much dead in the water without it. Make sure you have COMPAT_FREEBSD4! Running an old kernel after a `make world' will cause programs such as `df' that do a statfs system call to fail with a bad system call. Marco Wertejuk also reports that cfsd (ports/security/cfs) needs to be recompiled after these changes are installed. ****************************DANGER******************************* DO NOT make installworld after the buildworld w/o building and installing a new kernel FIRST. You will be unable to build a new kernel otherwise on a system with new binaries and an old kernel. 20031112: Some netgraph string length constants have been changed. This change requires the netgraph kernel modules and all netgraph userland components to be in sync. Especially users who require netgraph to boot need to make sure to have world and kernel in sync before rebooting. 20031111: Hyperthreading logical CPU's are no longer probed by default when using the MP Table. If ACPI is being used, then logical CPUs will be probed if hyperthreading is enabled in the BIOS. If ACPI is not being used and hyperthreading is enabled in the BIOS, logical CPUs can be enabled by building a custom kernel with the option MPTABLE_FORCE_HTT enabled. 20031103: The i386 APIC_IO kernel option has been replaced by 'device apic'. The ACPI module has also been temporarily disabled, so ACPI must be statically compiled into your kernel using 'device acpi' if you wish to use the ACPI driver. 20031031: The API and ABI of struct ifnet have been changed by removing the if_name and if_unit members and replacing them with if_xname, if_dname, and if_dunit. All network drivers and most userland programs which include net/if_var.h must be updated and recompiled. __FreeBSD_version has been bumped to 501113 to reflect this change. 20030928: Changes to the cdevsw default functions have been made to remove the need to specify nullopen() and nullclose() explicitly. __FreeBSD_version bumped to 501110. 20030926: kiconv(3) has been added. mount_msdosfs(8), mount_ntfs(8) and mount_cd9660(8) need to be in sync with kernel. 20030925: Configuring a system to use IPFILTER now requires that PFIL_HOOKS also be explicitly configured. Previously this dependency was magically handled through some cruft in net/pfil.h; but that has been removed. Building a kernel with IPFILTER but not PFIL_HOOKS will fail with obtuse errors in ip_fil.c. 20030923: Fix a bug in arplookup(), whereby a hostile party on a locally attached network could exhaust kernel memory, and cause a system panic, by sending a flood of spoofed ARP requests. See FreeBSD-SA-03:14.arp. 20030915: A change to /etc/defaults/rc.conf now causes inetd to be started with `-C 60' if it is not overridden in /etc/rc.conf. This causes inetd to stop accepting connections from an IP address that exceeds the rate of 60 connections per minute. 20030829: The following rc.d scripts have been removed and should be deleted from your installation: atm2.sh atm3.sh devdb localdaemons network1 network2 network3. Depending on when you last updated world and used mergemaster(8) you may or may not have problems during the rc boot sequence. The simplest solution is an 'rm -rf /etc/rc.d/*' and then 'mergemaster -i'. The atm2.sh atm3.sh and devdb scripts were removed some time ago, so depending on when you installed -CURRENT these scripts may or may not exist on your system. 20030824: ATAng has been committed. You need to build world as sys/ata.h has changed, and userland atacontrol depends on it. If you use ATA SW raids you need "device ataraid" in your kernel config file, as it is no longer pulled in automatically. 20030819: The OFW_NEWPCI option has been turned on in the Sparc64 GENERIC kernel. Among other things, this changes the device enumeration to be closer to Solaris. Be aware that, this can even cause the machine to not boot without manual intervention before the fstab is adjusted. 20030728: All current USB and Firewire quirks in da(4) have been deprecated and will be removed for 5.2. If this causes failure for your umass(4) devices, enable "options DA_OLD_QUIRKS" in your kernel and send the output of "camcontrol inquiry da0" to scsi@freebsd.org so the quirk can be re-enabled. 20030724: Problems with entry 20030714 have been corrected and no known issues with /rescue and -j exist for host systems after this point in time. 20030722: FPU-less support has been removed from FreeBSD. Chances are you won't notice. 386+387 support should still work after this change, but it is now a minimum requirement for the i386 port that you have real FPU hardware. 20030714: Some people are having problems with changes related to /rescue. If you are building -j N, you will need to define NO_RESCUE. Others will need to define it if /rescue has issues with their environment. People should report those issues to current@. 20030711: gcc was upgraded to 3.3. You are advised to not build -DNOCLEAN across this point. Further, it might be a good idea to remove /usr/obj. 20030610: Remove deprecated locale names and transition period code for them, finishing switching to the new scheme. Check your LANG environment variable. 20030609: CCD has been changed to be a fully GEOMified class. Kernel and ccdconfig(8) needs to be in sync, this is particularly important to remember beforehand if your source tree is on a ccd device. Consider making a copy of the old ccdconfig into /boot/kernel.good or wherever you keep your backup kernel. 20030605: There was a small window in which sed(1) was broken. If you happen to have sed(1) installed during that window, which is evidenced by an inability to build world with the failure given below, you need to manually build and install sed(1) (and only sed(1)) before doing anything else. This is a one- time snafu. Typical failure mode: In file included from /usr/src/contrib/binutils/bfd/targets.c:1092: targmatch.h:7:1: null character(s) ignored targmatch.h:12:1: null character(s) ignored targmatch.h:16:1: null character(s) ignored : The window of "sed(1)-uction" is from Wed Jun 4 15:31:55 2003 UTC to Thu Jun 5 12:10:19 2003 UTC (from rev 1.30 to rev 1.31 of usr.bin/sed/process.c). 20030505: Kerberos 5 (Heimdal) is now built by default. Setting MAKE_KERBEROS5 no longer has any effect. If you do NOT want the "base" Kerberos 5, you need to set NO_KERBEROS. 20030502: groff has been updated. If you try to do a buildworld and get an infinite loop in troff, update to May 4th or newer. If you have a newer kernel than userland, you may need to set the OSRELDATE to 500110 in your environment before starting a buildworld. 20030501: The old rc system has been removed. Please report any problems to freebsd-rc@yahoogroups.com, and/or freebsd-current@freebsd.org. Your personal versions of these files will not be removed, so you can continue to use them. However, you should take great care when updating, especially when using mergemaster, since the compatibility code that utilizes these old scripts has also been removed. 20030423: A bug has been fixed in /dev/devctl which would cause devd to hang on boot, were it not for a workaround in devd. The work around in devd will be removed around 20030507. You have until then to upgrade your kernel before updating userland. In general, you should have a userland and kernel that's in sync with each other. However, given the effects of this bug (hang on boot when starting devd), some allowances are made. 20030329: Alphas with libc from between 20030312 and 20030329 exhibit floating point exceptions (FPEs), most notably in awk(1) while upgrading the system through a buildworld. So, to successfully upgrade your Alpha, you must either downgrade your libc.so to a pre-20030312 version, or update /usr/share/mk/bsd.cpu.mk to revision 1.26 which adds -mieee to CFLAGS, then forcibly rebuild and install libc: cd /usr/src/lib/libc && \ make cleandir && make obj && \ make -DNOMAN -DNOPROFILE all && \ make -DNOMAN -DNOPROFILE install 20030208: sendmail 8.12.7 has been imported. It has one important change for IPv6 users. The default submit.mc now uses '[127.0.0.1]' instead of 'localhost' meaning only IPv4 is used to connect to the MTA. Users on IPv6-only machines will need to edit /etc/mail/submit.mc appropriately. 20030128: NODEVFS option has been removed and DEVFS thereby made standard. This makes all references to MAKEDEV obsolete, and they should be removed when convenient. 20030126: The name of the device for the ofw console has changed, sparc64 users must run mergemaster to update their installed /etc/ttys. 20030125: The scheduler framework has grown a second scheduler and consequently you must specify one and only one scheduler in your kernel config. The cvs config files have been updated to use the old scheduler which may be selected via 'options SCHED_4BSD'. If you would like to try the new, much more experimental, scheduler please try 'options SCHED_ULE' and contribute to the arch@ discussion. 20030115: A new version of the wi driver has been imported into the tree. One now must have device wlan in the config file for it to operate properly. In addition, there have been some changes to how wi devices are configured for point to point links to bring it more in line with the former way of doing things, as well as compatibility with NetBSD. 20021222: For a period after the GCC 3.2.1 import (from 12/04 to 12/22), GCC used an incompatible form of ABI for returning structures and unions which FreeBSD's GCC maintainers were not aware of relative to previous versions of FreeBSD. We have gone back to the ABI for now, and any code compiled which is required to interoperate with other code (not built at the same time) returning structs or unions should be rebuilt. 20021216: A name change in /etc/netconfig has been reverted to stay compatible with suns TIRPC and also with NetBSD. You need to run mergemaster after make world. A new libc does still work with an outdated /etc/netconfig for some time, but you'll get a warning. This warning will be removed in 20030301. 20021202: The recent binutils upgrade marks a kernel flag day on sparc64: modules built with the old binutils will not work with new kernels and vice versa. Mismatches will result in panics. Make sure your kernel and modules are in sync. 20021029: The value of IPPROTO_DIVERT has changed. Make sure to keep your kernel, netstat, natd and any third-party DIVERT consumers in sync. 20021024: Old, compatibility slices have been removed in GEOM kernels. This means that you will have to update your /etc/fstab to not use disk devices of the form /dev/ad0a. Instead, you now must specify /dev/ad0s1a, or whatever slice your FreeBSD partition really is on. The old device names have gone away, so if you use them anywhere else, you must also adjust those uses. (This doesn't affect the disks formatted in the ``dangerously-dedicated'' mode.) 20021023: Alphas with kernels from between 20020830 and 20021023 and/or rtld (ld-elf.so.1) older than 20021023 may experience problems with groff while doing a buildworld (kernel: "out of memory", fixed in rev 1.129 of kern/imgact_elf.c; rtld: "too few PT_LOAD segments", fixed in rev 1.8 of libexec/rtld-elf/map_object.c). So, to successfully upgrade your Alpha, you must either upgrade your kernel and rtld first (which might be a bit tricky), or avoid running the bootstrapped groff during the "transitional" buildworld. To avoid running groff during the transitional upgrade run make buildworld with -DNOMAN, -DNO_SHAREDOCS, and -DNO_LPR. 20020831: gcc has been upgraded to 3.2. It is not all binary compatible with earlier versions of gcc for c++ programs. All c++ programs and libraries need to be recompiled. Also, if you encounter g++ issues, rm /usr/include/g++/* before doing an installworld to make sure that stale files are removed. 20020827: Our /etc/termcap now has all the entries from the XFree86 xterm almost unchanged. This means xterm now supports color by default. If you used TERM=xterm-color in the past you now should use TERM=xterm. (xterm-color will lead to benign warnings). 20020815: A "bug" in gcc(1) that was hiding warning in system headers was fixed. It's probably time to add -DNO_WERROR to your make line again. 20020729: COPY is being deprecated. The 20010530 change was reverted, as it causes far more pain than was expected, and to always compare before installing, please use INSTALL="install -C" again. The -C option is now silently ignored when used with the -d option. 20020702: Problems with libc_r clients like KDE and GNOME have been resolved. There are still some minor problems with some signals but the system is stable enough for general use again. SMP is less so than UP but each can successfully complete multiple buildworlds. Libkvm needs to be recompiled due to KSE. 20020701: Now would be a bad time to upgrade. Something in or near the KSE commit totally broke programs using libc_r like KDE and GNOME. 20020511: The k5su utility installed as part of Kerberos 5 is no longer installed with the set-user-ID bit set by default. Add ENABLE_SUID_K5SU=yes to /etc/make.conf to have it installed with the set-user-ID bit set. 20020510: Gcc 3.1 debugging format (cc -g) has changed from STABS to DWARF2. Unfortunately our native GDB (at version 4.18) does not understand the DWARF2 debugging format. Thus you must use `gcc -gstabs+' to generated debugging information for our native GDB. 20020510: Due to the way CVS works, it may not properly update src/contrib/gcc to the 3.1 sources. The easiest fix is to `rm -rf' src/contrib/gcc and then do a cvs update. 20020421: When exec'ing set[ug]id executables, the kernel now ensures that the stdio file descriptors (0..2) are open. See FreeBSD-SA-02:23.stdio. 20020404: New sendmail startup scripts have been installed to make it easier to use alternative MTAs with FreeBSD. Setting the rc.conf variable sendmail_enable to "NO" no longer prevents any sendmail daemons from starting. Instead, either set sendmail_enable to "NONE" or change mta_start_script to a script for starting an alternative MTA. Setting mta_start_script to "" will also prevent any MTA from being started at boot. 20020403: UCONSOLE is no longer a valid kernel option. 20020315: FreeBSD 5.0 DP-1 was basically branched today. 20020225: Warnings are now errors in the kernel. Unless you are a developer, you should add -DNO_WERROR to your make line. 20020217: sendmail 8.12.2 has been imported. The sendmail binary is no longer a set-user-ID root binary and the infrastructure to support command line mail submission has changed. Be sure to run mergemaster (especially for updating /etc/rc, /etc/defaults/rc.conf, and /etc/mail) and read /etc/mail/README for more details. Due to the import of sendmail 8.12.2, a new user and group are required in order for sendmail to run as a set-group-ID binary. A 'make installworld' will use the new user and group to set the owner and group of /var/spool/clientmqueue and will fail if the new user and group do not exist. The 'smmsp' user and group must be merged from src/etc/group and src/etc/master.passwd before using 'make installworld'. 'mergemaster -p' will do this. You may need to install mergemaster before this will work if you are updating from a very old version of current. The updating recipe has changed as of this date. 20020112: The preferred configuration method for PAM is now /etc/pam.d/ rather than /etc/pam.conf. If you have an unmodified pam.conf, just delete it after your next mergemaster run. If you have local modifications, you can use /usr/src/etc/pam.d/convert.pl to incorporate them into your /etc/pam.d. Please see the following url for more details: http://www.freebsd.org/cgi/mid.cgi?db=mid&id= 20011229: If anyone here is already using the new rc.conf(5) variable networkfs_types, please note that it has changed http://www.freebsd.org/cgi/mid.cgi?db=mid&id=<9744.1009655556@axl.seasidesoftware.co.za> 20011220: sys/i4b/driver/i4b_ispppsubr.c has been retired. This file started out its life in the ISDN4BSD project as an offspring from sys/net/if_spppsubr.c, which eventually got a life of its own. All the accumulated features and bug fixes of the i4b version have now been merged back into the base system's version now. The only user-visible change resulting from this is that i4b's sppp(4) interfaces are to be managed with spppcontrol(8) again, since ispppcontrol(8) has been retired as well. (There has never been rc file support for ispppcontrol in -current, but only in -stable. That will be reverted by the time the changes are MFCed.) 20011215: The fdc(4) driver has been updated and now automatically recognizes media in `standard' formats (like 1440 KB and 720 KB for a 3.5" high-density drive) when accessing the default device node (e. g. /dev/fd0). The old variety of floppy device nodes /dev/fd*.* is no longer present by default, devices can be created (in DEVFS) on demand. They will need to be customized then for `odd' densities using fdcontrol(8). 20011209: The bugs in procfs' debugging support code have been fixed, and truss(1) now works again. 20011207: Daily security checks have been split out to use the periodic(8) scripts. Some change in configuration may be necessary. Please see http://www.freebsd.org/cgi/mid.cgi?db=mid&id=<20011207155805.R8975@blossom.cjclark.org> for details. 20011204: sos added VCD/SVCD support to ata driver and that needs the kernel and burncd to be in sync. 20011203: The procfs pseudo-filesystem has now been converted to use the pseudofs framework. If you have 'options PROCFS' in your kernel config, you'll need to add 'options PSEUDOFS' if it's not there already. This change temporarily breaks truss(1); use ktrace(1) instead until the issue has been resolved. 20011202: A security hole in OpenSSH involving `UseLogin yes' has been patched. 20011126: You need to remove /usr/obj/.../usr.bin/tip before rebuilding after this date. You need to do this only once. 20011103: Most of the awk issues have been resolved. Some rough edges may be left, but for the most part things should be back to "normal." For CURRENT's usual definition of "normal." 20011030: Awk has been upgraded to the one true awk from bell labs. Expect choppy waves in the upgrade process. 20011030: The asr driver problem has been resolved. 20011027: Due to changes in other parts of the system, the asr driver now causes the system to panic on boot. Do not use it pending correction. Comment it out of any kernel config file that you try to use from this date forward. 20011025: When crossbuilding, use TARGET=xxx where you used to use MACHINE=xxx. You don't need to set TARGET_ARCH and TARGET, unless you are changing both of them. To cross build pc98 on an alpha, for example, you need to set TARGET=pc98 and TARGET_ARCH=i386. 20011001: The kernel interface that burncd depends on has changed. You must recompile both the kernel and userland applications at the same time. 20010929: When crossbuilding, please set TARGET_ARCH rather than MACHINE_ARCH to indicate the target. In the future, one will set TARGET_MACHINE where you set MACHINE now. At the moment, setting MACHINE alone for same MACHINE_ARCH machines works (eg, you can build pc98 on an i386 machine and vice versa). 20010927: Some weird problems result from using ACPI on some machines. To disable ACPI you can add hint.acpi.0.disabled="1" to /boot/loader.conf (or by putting set X=Y at the boot loader "ok" prompt). Alternatively, you can remove it from /boot/kernel/acpi.ko or use the MODULES_OVERRIDE function in your kernel config file and not list acpi in that list. 20010924: The buildworld has been fixed. You may need to install the 4.x compatibility libraries for some old binaries to work. Add COMPAT4X=true to your /etc/make.conf to get them installed on every installworld, or execute the following to get them installed only once: cd src/lib/compat/compat4x. make all install You will see ``__stdoutp undefined'' until you do this. 20010919: There's a bug in the world build process. The cross-tools are build with the NEW headers, but the OLD libc.a. This leads to all kinds of problems with the new libc. A temporary workaround is to add CFLAGS="-O -pipe -D_OLD_STDIO" before building world when upgrading from 4.x to current. This can be removed afterwards. A proper fix to the buildworld target is needed. 20010918: Peter has committed his new kthread nfs client/server code. NFS may be unstable after this date. 20010912: KSE has hit the tree. Lots of things are now different in the kernel. While a few problems were introduced in the initial commit, most of the major ones have been found and corrected. 20010901: In OLDCARD, CardBus bridges appear to be stable. The work arounds described in the 20010604 entry are now no longer necessary and will be ignored. Most insert/remove problems have been rectified around this date. 20010823: named now runs as user bind and group bind rather than as root. If named_enable is set to YES in /etc/rc.conf, ensure that user bind is available in /etc/passwd (using vipw(8)) and that group bind is available in /etc/group. Also make sure that user or group bind has read (and not write) permission for your name server configuration and that it has read and write permission for your slave zone files and directory. If you wish to continue to run named as root (a less secure alternative), add a line to /etc/rc.conf saying named_flags= 20010709: The PAM libraries have had an API upgrade that is beyond the ability of the shared library major number to handle. It is manifested by PAM-using ports dumping core. The solution is to rebuild those ports. 20010628: The kernel compile module has moved from src/sys/compile/FOO to src/sys/${MACHINE}/compile/FOO. 20010625: The pccard modem issue from 20010613 has been corrected. OLDCARD support is still a little weak in -current. slot 1 is known not to work on some TI based cardbus bridges. Some cardbus bridges do not properly detect insert/removal events. IRQ configuration needs more safety belts. 20010617: Softupdates problems have been corrected. 20010614: Peter ripped out the linkerset support. You must, as always, rerun config after you cvsup if you are using the traditional kernel building methods. 20010613: pccard modems may not work with current after 20010604 date. Some do, others result in panics. *MAKE*SURE* that you update your config and /etc/rc.conf ala the 20010604 entry, or you will have problems (this issue will be fixed, it just hasn't been yet). 20010613: SOFTUPDATES seem to be broken since the middle of May or so. Do not use them in current. You can disable softupdates on all mounted partitions, or remove SOFTUPDATES the kernel config file. 20010612: After Peter's commits to the hints code, people have been noticing that certain devices are attached (or try to) twice. This is due to having both static hints as well as a /boot/device.hints. To work around this issue, please use only one or the other mechanism until this bug is fixed. Please note that a feature of config is that if you have config file FOO and FOO.hints, it automatically adds FOO.hints to the hints.c file, whether you want it to or not. 20010610: Locale names have changed to match other systems better. 20010604: pccard support for pci cards has been committed. You must change your /etc/pccard.conf irq lines. It must match the irq used by pcic device. Interrupt storms may result if you fail to do this. Interrupt storms look a lot like a hang. You must also install a new pccardd, otherwise you will get an interrupt storm at card reset time (just after it tells you what it is). pccardd_flags="-I" is necessary for the time being. It tells pccardd not to ask the kernel if the interrupt is really free or not before using it. You can either change the /etc/pccard.conf irq lines to match pcic, or add "-i X" to the pccardd_flags. 20010530: INSTALL=install -C is being deprecated. If you want to do this, use COPY=-C instead. The former method will be supported for only a limited time. If you see install: warning: the -d and -C options may not be specified together in your makeworld, then you need to migrate towards using COPY=-C. 20010525: It appears that vm is now stable enough to use again. However, there may be other problems, so caution is still urged. alpha definitely is in bad shape. 20010521: Minor repo damage has happened. This may cause problems with cvsup of ports. If you get errors, please see http://www.FreeBSD.org/cgi/query-pr.cgi?pr=27495 at the bottom for details on a workaround. The error message is Updater failed: Cannot delete "/usr/ports/www/jakarta-tomcat/files": Directory not empty 20010520: Vm and/or swapping are busted on -current. Please be patient. 20010519: pccard has had much reorganizational work done to it over the past few days. Everything should still work, but if not, please contact imp@freebsd.org. 20010517: ata ioctl changed. Make sure to recompile both kernel and userland at the same time. 20010517: New ncurses imported. 20010512: DEVFS is now opt out, not opt in. Barring major problems, this will be the only way to go starting July 1. 20010504: OpenSSH has been updated to 2.9. Some defaults are different, including RhostsRSAAuthentication, which changes from yes to no. 20010502: Perl breakage in 20010501 was corrected at 14:18:33 PDT. 20010501: Building perl was broken at 02:25:25 PDT. 20010430: The bug in 20010429 was corrected at 07:35:37 PDT. It is safe to go back in the water. 20010429: A bad bug was committed at 04:48:42 PDT. Don't use kernels after this date, but before the correction date. 20010423: old fsck and new kernel interactions appear to have been fixed. 20010411: fsck and the kernel were changed to handle some optimizations to directory layout. This breaks backward compatibility. Update only if you understand that you must not use the old fsck with the new kernel ever. 20010330: fsck has changed the meaning of the pass column in /etc/fstab. Please see the cvs commit to fsck.8 or the fsck.8 man page for details. It is unclear if changes to /etc/fstab are necessary. 20010319: portmap had changed name to rpcbind for maximum POLA in your current world. /etc/hosts.{allow,deny} needs changes. nfs and other rpc based programs that rely on portmapper will not work without updates to /etc/hosts.{allow,deny} and /etc/netconfig. 20010315: ata subsystem changes. ATA_ENABLE_ATAPI_DMA, ATA_ENABLE_WC and ATA_ENABLE_TAGS are no longer kernel options. They have been replaced by tunables. See ata.4 for details. 20010312: The fxp driver was converted to use miibus. If you compile fxp into your kernel statically, you will need to add miibus. 20010312: The wi device now defaults to BSS (infrastructure) mode instead of ad-hoc. 20010310: /dev/urandom should be a symbolic link to /dev/random now. Users of current not using DEVFS need to run MAKEDEV std. ssh might not work if you don't. 20010303: The ed driver has been updated. It now allows mii attachments, which means that you must include the miibus in your kernel if you use the ed driver. 20010220: The problems with libc have been corrected. It is now mostly safe to go back into the water. 20010211: The size of FILE was changed. This breaks upgrading. If you must upgrade, be prepared for pain. It also breaks almost all binaries that you've compiled on -current. You are warned that before upgrading would be a good time to do a level 0 dump of your system. No, really, I mean it this time. To get to the new system, you'll need to use the following workaround. Hopefully this can be sorted out so that we don't have to move this to the updating section. To get around the installworld problem, do: # cd /usr/src/usr.bin/sed # make install # cd /usr/src # make installworld If that doesn't work, then try: # make -k installworld # make installworld 20010207: DEVFS is now the default. If you use vinum, make sure that you do not include devfs in your kernel as problems result. 20010205: FFS_ROOT and CD9660_ROOT have been removed or deprecated. Remove them from your config. 20010122: ****************************** WARNING ****************************** buildkernel has been changed slightly ****************************** WARNING ****************************** KERNCONF replaces the variable KERNEL for buildkernel. You should update your scripts and make.conf accordingly. 20010119: config has changed to allow DEV_FOO as a replacement for NFOO. This requires a new config to build correctly. 20010116: The kernel option I386_CPU is now mutually exclusive with the other cpu types. If you have an i386 system, be sure that it only had this line. Remove it for all other configurations. 20010110: Changes to the kernel require it and burncd be in sync. 20010102: Everyone who has hw.sndunit set to something in /etc/sysctl.conf, it is now hw.snd.unit. 20010101: ex and vi were broken by some changes to sys/queue.h. If you have a bad vi, you will see make buildworld fail with a core dump while building termcap. You can work around this problem by adding -k to your make buildworld. This will cause the build to complete and install a new vi. Once that's done, you can rebuild again without the -k to pick up anything that might have been ignored by the -k option. Others have suggested that you can just rebuild libc if your vi/ex is dynamically linked, but I've not received any reports of this working. 20001228: There have been some changes to libcrypt in -current. The libscrypt/libdescrypt symlink silliness is gone and the installed libcrypt is fully functional. Be aware of this. 20001218: Linksys Fast Ethernet PCCARD cards supported by the ed driver now require the addition of flag 0x80000 to their config line in pccard.conf(5). This flag is not optional. These Linksys cards will not be recognized without it. 20001205: Important new FreeBSD-version stuff: PAM support has been worked in, partially from the "Unix" OpenSSH version. This requires adding the following in pam.conf: sshd auth sufficient pam_skey.so sshd auth required pam_unix.so try_first_pass sshd session required pam_permit.so 20001031: cvs updated to 1.11. 20001020: The random device needs more entropy, so you need to make sure that you've run mergemaster to get a /etc/rc which will seed /dev/random. If you don't and the system hangs after ldconfig, then banging on the keyboard randomly until it unhangs is one workaround. 20001010: ****************************** WARNING ****************************** Sendmail has been updated. ****************************** WARNING ****************************** o mail.local(8) is no longer installed as a set-user-id binary. o sendmail(8) is now built with STARTTLS support unless NO_OPENSSL is set. o The default /etc/mail/sendmail.cf disables the SMTP EXPN and VRFY commands. o Now using sendmail's version of vacation(1). o The sendmail cf building tools (contrib/sendmail/cf) are installed in /usr/share/sendmail/cf. o sendmail.cw changed to local-host-names More details can be found at http://people.freebsd.org/~imp/UPDATING/sendmail-20001010 20001009: The ports tree's new layout is in place. Be sure to update your entire ports tree, or you will have problems. 20001006: The perl build procedure no longer installs miniperl, nor uses the installed miniperl. It is recommended that you delete /usr/bin/miniperl. 20001005: This weekend the ports tree will be updated to a new layout. It will be in an inconsistent state until noted in the UPDATING file, or with asami-san's message to the relevant mailing lists. With this new layout, you'll need to update the whole tree for anything to work. 20000928: There was a change in the passwd format. Need more information. 20000916: /boot/kernel/kernel.ko -> /boot/kernel/kernel change has taken place. Please update boot loader (not the boot blocks) at the same time as your kernel. 20000914: The new pmtimer device is necessary for laptops. Failure to include the device will cause suspended laptops losing time when they resume. Include device pmtimer in your config file and hint.pmtimer.0.at="isa" to your /boot/device.hints file. 20000911: The random device has been turned into a (pseudo-)device, rather than an option. The supplied kernel config files have been updated. You will need to do something similar in your own kernel config file. Remove: options RANDOMDEV Add: device random If you prefer to load the loadable module, you need to do nothing. 20000909: The random device module has been renamed from randomdev.ko to random.ko. You will need to edit your /boot/loader.conf to reflect this if you load this module at boot time. The line should read: random_load="YES" 20000907: The SMPNG commit has happened. It should work, but if it doesn't, fallback to the PRE_SMPNG CVS tag. There are likely to be a variety of minor issues. Please see 20000905 to make sure you don't have model loading problems which might at first blush appear related to SMP. 20000906: nsswitch has been imported from NetBSD. Among other things, this means that /etc/host.conf is no longer used. See nsswitch.conf(5) instead. Note that at boot time rc.network will attempt to produce a new nsswitch.conf file for you if you don't have one, and you have host.conf. 20000905: The ucred structure changed size. This breaks the interface that mountd uses. Trying to use an older mountd with a newer kernel guarantees a panic. This means that you need to use kernels newer than today only with matching mountd, but you needed to do that anyway with the boot loader changes. 20000905: The boot loader has been updated. The new default kernel is now /boot/kernel/kernel.ko. The new default module location is /boot/kernel. You *MUST* upgrade your boot loader and kernel at the same time. The easiest way to do this is to do the buildworld/buildkernel/ installkernel/installworld dance. Furthermore, you are urged to delete your old /modules directory before booting the new kernel, since kldload will find stale modules in that directory instead of finding them in the correct path, /boot/kernel. The most common complaint that this cures is that the linux module crashes your machine after the update. if [ ! -d /boot/kernel.old ]; then mv /modules.old /boot/kernel.old chflags noschg /kernel.old mv /kernel.old /boot/kernel.old/kernel.ko chflags schg /boot/kernel.old/kernel.ko fi 20000904: A new issue with the sendmail upgrade has come to light. /etc/aliases has moved to /etc/mail/aliases. Mergemaster will incorrectly install the default aliases in /etc/mail rather than move the old one from /etc. So you'll need to manually move the file, create a symbolic link, remove the old /etc/aliases.db and run newaliases. For safety sake, you should stop sendmail while doing this and run the upgrade when locally sourced email is not likely to be generated. 20000825: /boot/device.hints is now required for installkernel to succeed. You should copy GENERIC.hints for your architecture into /boot/device.hints. If and only if you compile hints into your kernel, then this file may be empty. Please note, if you have an empty or missing /boot/device.hints file and you neglected to compile hints into your kernel, no boot messages will appear after the boot loader tries to start the kernel. 20000821: If you do NOT have ``options RANDOMDEV'' in your kernel and you DO want the random device then add randomdev_load="YES" to /boot/loader.conf. 20000812: suidperl is now always built and installed on the system, but with permissions of 511. If you have applications that use this program, you are now required to add ENABLE_SUIDPERL=true to /etc/make.conf. If you forget to do this, chmod 4511 /usr/bin/suidperl will fix this until the next build. 20000812: sendmail has been updated from 8.9.3 to 8.11.0. Some of the more visible changes that may immediately affect your configuration include: - New default file locations from src/contrib/sendmail/cf/README - newaliases limited to root and trusted users - MSA port (587) turned on by default - New queue file naming system so can't go from 8.11 -> 8.9 - FEATURE(`rbl') renamed to FEATURE(`dnsbl') - FEATURE(`nullclient') is more full featured - FEATURE(`nouucp') requires an argument: `reject' or `nospecial' - mail.local FreeBSD-only -b option changed to -B - See src/contrib/sendmail/RELEASE_NOTES for more info 20000810: suidperl (aka sperl) is no longer build by default. You must specifically define BUILD_SUIDPERL to "true" for it to be build. Furthermore, we recommend that you remove /usr/bin/sperl* and /usr/bin/suidperl files from your system unless you have a specific use for it. 20000729: Networking defaults have been tightened. Anybody upgrading /etc/defaults/rc.conf needs to add the following lines to /etc/rc.conf if they want to have the same setup afterwards (unless the variables already are set, of course): # Enable network daemons for user convenience. inetd_enable="YES" portmap_enable="YES" sendmail_enable="YES" 20000728: If you have null_load="YES" in your /boot/loader.conf, you will need to change that to nullfs_load="YES". 20000728: The "installkernel" target has changed slightly. Now even if you override KERNEL e.g. 'make installkernel KERNEL=MYKERNEL' it will install the MYKERNEL file (built with the buildkernel target) as /kernel rather than /MYKERNEL. Those who have updated their /boot/loader.conf files to point to /MYKERNEL should remove that entry or perform manual rename of /kernel to /MYKERNEL. 20000711: If you use CVSUP or CTM to get CVS trees, AND you used to get the old crypto files from internat.freebsd.org AND you check out files from the CVS tree with the cvs command, please read http://people.freebsd.org/~imp/internat.txt for details on potential problems that you might have and how to get around them. If you are merely a mirror, or don't answer yes to each of the clauses above, you needn't worry. 20000711: /etc/security has been updated to print the inode number of setuid programs that have changed. You will see a large spike in the number of changed programs the first time when you run mergemaster to get a new /etc/security. 20000710: /dev/random now has good entropy collection (from the keyboard and sysmouse drivers). Please ensure that either `options RANDOMDEV' is present in your kernel config file or that `randomdev_load="YES"' is in your /boot/loader.conf. If you do not have the /dev/random driver, OpenSSL (and consequently lots of crypto tools (like SSH)) will fail with strange errors. (see below, 20000624). FreeBSD-current is safe again to run Crypto. 20000709: phk made the malloc default options AJ. This may slow things down and uncover other latent bugs in the code. If you need to run at full speed, you can disable this by doing the following: ln -s aj /etc/malloc.conf 20000706: libftpio's version was accidentally bumped a few days ago. This has been corrected. You may need to remove /usr/lib/libftpio.so.6 before doing your next buildworld/installworld pair. It certainly won't hurt to remove it before the update procedure. It will break fetch until a new one is built, but ftp can be used in the interim if needed. 20000705: The crypto packages have changed for the cvsup. This has been done in a backward compatible way, but the old packages will go away at some point in the future. Look at /usr/share/examples/cvsup for details. 20000704: With the new sys/modules/sound/drivers/*, you will need to set SYSDIR until you do an installworld after July 7th. 20000704: rc.shutdown and rc will now call the rc.d scripts with start or stop. This may cause some harmless warnings from older rc.d scripts that haven't been updated. 20000630: The libfetch based version of fetch has gone into the tree. Minor problems may result on some of the less popular sites, which should be reported to des@freebsd.org. 20000625: From approximately this date forward, one must have the crypto system installed in order to build the system and kernel. While not technically strictly true, one should treat it as required and grab the crypto bits. If you are grabbing CVS trees, src-all and cvs-crypto should be treated as if they were required. You should check with the latest collections to make sure that these haven't changed. 20000624: Mark Murray just committed the first parts of a cleanup of /dev/zero, et al. This is also cleaning up /dev/random. The entropy is disconnected, so DO NOT USE VERSIONS OF FREEBSD -CURRENT FROM THIS POINT to 2000710 for cryptographic services until Mark can merge in the fixes to this work in progress. openssh and openssl should not be used to generate keys from this date to the completion of the work. If you must operate at this reduced level of security, add ' options RANDOMDEV' to your kernel or modload the randomdev module. You may also need to copy a new MAKEDEV to /dev and recreate the random and urandom devices. 20000622: The license on the softupdates is now a standard 2 clause BSD license. You may need to remove your symbolic links that used to be required when updating. 20000621: Scott Flatman sent in a decent write-up on the config file update procedure. http://people.freebsd.org/~imp/config-upd.html NOTE: LINT is gone. It has been replaced with NOTES. NOTES isn't buildable. However, you can generate a LINT file: cd /sys//conf && make LINT 20000620: Binutils 2.10 have hit the tree, or will shortly. As soon as they do, the problem noted in 20000522 will be resolved and that workaround will no longer be required. 20000615: phk removed the compatibility creation of wd devices in the ad driver. If you haven't done so already, you must update your fstab, etc to use the ad devices instead of the wd devices. In addition, you'll need to update your boot blocks to a more modern version, if you haven't already done so. Modern here means 4.0 release or newer (although older releases may work). 20000612: Peter took an axe to config(8). Be sure that you read his mail on the topic before even thinking about updating. You will need to create a /boot/device.hints or add a hints directive to your config file to compile them in statically. The format of the config file has changed as well. Please see GENERIC or NEWCARD for examples of the new format. Indirectly, this also breaks USERCONFIG. Unless a newer entry says that it has been fixed, assume that must use the hints mechanism in the loader if you need to use a machine with very old ISA cards in it. 20000522: A new set of binutils went into the tree today. Anybody building a kernel after this point is advised that they need to rebuild their binutils (or better yet do a buildworld/installworld) before building a new kernel. Due to bugs in binutils, using malloc options (eg /etc/malloc.conf or MALLOC_OPTIONS env var) J will cause ld to dump core. It is recommended that you don't set this option until the problem is resolved. 20000513: The ethernet drivers were all updated to clean up the BPF handling. 20000510: The problems with boot blocks on the alphas have been corrected. This will require some care in updating alphas. A new libstand is requires for the boot blocks to build properly. 20000503: Recompile all kld modules. Proper version dependency info is now available. 20000502: Modules have been disconnected from the buildworld tree and connected to the kernel building instead. 20000427: You may need to build gperf cd /usr/src/gnu/usr.bin/gperf && make depend all install when upgrading from 4.0 -> current. The build system now uses an option only in -current. 20000417: The method that we brand ELF binaries has changed to be more acceptable to the binutils maintainers. You will need to rebrand your ELF binaries that aren't native. One problem binary is the Linux ldconfig. After your make world, but before you reboot, you'll need to issue: brandelf -t Linux /compat/linux/sbin/ldconfig if you have Linux compatibility enabled on your machine. 20000320: If you have really bad/marginal IDE drives, you may find they don't work well. Use pio mode instead. The easiest way to cope if you have a problem combination is to add: /sbin/sysctl hw.ata.ata_dma=0 to the start of /etc/rc.conf. 20000319: The ISA and PCI compatibility shims have been connected to the options COMPAT_OLDISA and COMPAT_OLDPCI. If you are using old style PCI or ISA drivers (i.e. tx, voxware, etc.) you must include the appropriate option in your kernel config. Drivers using the shims should be updated or they won't ship with 5.0-RELEASE, targeted for 2001. 20000318: We've entered the traditional post release dumping party. Large kernel changes are being committed and are in the works. It is important to keep the systems' klds and kernel in sync as kernel interfaces and structures are changing. Before reporting kernel panics, make sure that all modules that you are loading are up to date. 20000315: If you are upgrading from an older version of FreeBSD, you need to update your boot blocks as well. 'disklabel -B ad0' will do the trick. This isn't critical until you remove your wd device entries in /dev, at which point your system will not boot. 20000315: 4.0 RELEASE shipped. Please see the 4.0 UPDATING file for how to upgrade to 4.0 from 3.x. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. With a /usr/obj tree with a fresh buildworld, make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a 5.X system. Replace ${arch} with the architecture of your machine (e.g. "i386", "alpha", "amd64", "ia64", "pc98", "sparc64", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. [7] make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installkernel KERNCONF=YOUR_KERNEL_HERE [1] [3] /etc/rc.d/preseedrandom [10] mergemaster -p [5] make installworld mergemaster [4] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make installworld DESTDIR=${CURRENT_ROOT} make buildkernel KERNCONF=YOUR_KERNEL_HERE cp /usr/src/sys/${ARCH}/conf/GENERIC.hints \ ${CURRENT_ROOT}/boot/device.hints # as needed make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cd /usr/src/etc; make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from 4.x-stable to 5.x ---------------------------------------------- # 5.x uses more space than 4.x. Also, the location of kernel # modules has changed. If you are installing 5.x onto a 4.x # system, you'll need about 30MB of free disk space on your / # partition. If you have less than this, you may encounter difficult # to back out of problems with this procedure. If /tmp is on # the / partition, you may want to completely remove all its content # before upgrading, as this can be a common source of shortage of # space on /. [1] make buildworld [2] make buildkernel KERNCONF=YOUR_KERNEL_HERE [3] cp sys/${MACHINE}/conf/GENERIC.hints /boot/device.hints [4] make installkernel KERNCONF=YOUR_KERNEL_HERE cd sys/boot ; make STRIP= install [5] [6] [7] /usr/src/etc/rc.d/preseedrandom [8] mergemaster -p [9] rm -rf /usr/include/g++ rm -f /sbin/mountd make installworld [11] mergemaster -i [12] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of stable, things change fairly rarely. However, many changes from 4.x to 5.x are not reflected in this breif section. The defaults can also change, so please read ALL of the UPDATING entries. If you have any ports or packages installed on the system, please see /usr/ports/UPDATING. The following items are known to cause problems with the upgrade: (a) Kerberos upgrades are hard, and evidentally don't work. More on this later. (b) Old ports and packages can sometimes have subtle problems with newer libraries. portupgrade -f is recommended where possible after the upgrade. portupgrade -P will be faster since it uses packages. There are many known issues with running a mix of 4.x built packages and 5.x built packages, and this avoids them. (c) Perl based ports may also have the wrong version of perl hard coded into the headers. Spamassassin is one such port. (d) Startup scripts have significantly changed. Make sure that you verify that all expected daemons start. Modifications of /etc/rc* (except rc.conf) may cause problems. (e) Device names have changed and MAKEDEV is gone. (f) The boot loader defaults have changed, which may cause confusion. (g) If the build dies for some reason and you reboot, you'll be booting with new libraries, but old paths. You will need to tell ldconfig about them with ldconfig /lib /usr/lib /usr/local/lib (h) If the system appears to hang, try toggling the scroll lock. Many kvm switches use Scroll Lock to switch screens and on such systems a hang can often just be scroll lock active. (i) named now runs in a chroot unless you disable this. See named_chrootdir in /etc/defaults/rc.conf for details on restoring 4.x behavior. (j) Ports that use PAM on 4.x should be recompiled. This is especially true for sudo. know thy root password before the upgrade! [1] Before you upgrade, please make sure that you are not using compatibility slices. These are device names of the form, on i386 and other architectures that use MBR slicing, /dev/ad0a without the actual slice name. Chances are excellent that these will break. You generally must update these entries to use the post FreeBSD 2.x form of /dev/ad0s1a. [2] When checking out sources, you must include the -P flag to have cvs prune empty directories. Also, if CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. In case you would like to avoid installing new packages of everything, you might want to uncomment the "COMPAT4X= YES" entry, so that 4.x compatibility libraries are built which should allow you to continue using your existing software for a while. [3] In order to have a kernel that can run the 4.x or earlier binaries needed to do an installworld, you must include the COMPAT_FREEBSD4 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. If you have a.out binaries, then you need to add COMPAT_AOUT option too. If you are updating a system that's been updated from 2.x and/or 3.x in the past, enable both of these to be safe. [4] If you have legacy ISA devices, you may need to create your own device.hints to reflect your unique hardware configuration. [5] Since make installkernel also does a make heirarchy, PAM will stop working because /etc/pam.d's existance overrides /etc/pam.conf. If you want to login to your system after the install kernel, or if you have accidentally rebooted a remote system into multi-user, the existance of /etc/pam.d will cause remote logins to fail. It is safe to remove it at this step as it will be recreated later. [6] 4.x boot loader can be used to boot a 5.x system, however it is difficult to do that at best. If you wish to try, then you should interrupt the boot and at the ok prompt type: ok unload ok boot /boot/kernel/kernel If this fails to work, you must install a new boot loader as described here. [7] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [8] From the bootblocks, boot -s, and then do fsck -p mount -a -t ufs cd /usr/src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. For the 4.x -> 5.x upgrade, you will also see many messages about needing to recompile your userland. These are harmless and can be ignored while you proceed to the next step. [9] In order to create temporary files, /dev/random must be initialized by feeding data into it. /usr/src/etc/rc.d/preseedrandom takes care of this. [10] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. You may need to install a new mergemaster (cd src/usr.sbin/mergemaster && make install) after the buildworld before this step if you last updated from current before 20020224 or from -stable before 20020408. If you are installing not from /usr/src, don't forget the -m option. [11] If you are using a non-standard /usr/obj, you need to set an appropriate MAKEOBJDIRPREFIX before this step. [12] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. Also, if you are installing not from /usr/src, don't forget the -m option. Also note: since many libraries moved into /lib with 5.x, failure to run mergemaster will result in many shared libraries that should be there not being found. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -current. Not all things will be listed here, and it only starts on March 15, 2000. Updating files can found in previous releases if your system is older than this. Copyright information: Copyright 1998, 2002 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. If you find this document useful, and you want to, you may buy the author a beer. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: releng/5.5/gnu/usr.bin/gzip/gzip.h =================================================================== --- releng/5.5/gnu/usr.bin/gzip/gzip.h (revision 162446) +++ releng/5.5/gnu/usr.bin/gzip/gzip.h (revision 162447) @@ -1,318 +1,320 @@ /* gzip.h -- common declarations for all gzip modules * Copyright (C) 1992-1993 Jean-loup Gailly. * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. * * $FreeBSD$ */ #if defined(__STDC__) || defined(PROTO) # define OF(args) args #else # define OF(args) () #endif #ifdef __STDC__ typedef void *voidp; #else typedef char *voidp; #endif /* I don't like nested includes, but the string and io functions are used * too often */ #include #if !defined(NO_STRING_H) || defined(STDC_HEADERS) # include # if !defined(STDC_HEADERS) && !defined(NO_MEMORY_H) && !defined(__GNUC__) # include # endif # define memzero(s, n) memset ((voidp)(s), 0, (n)) #else # include # define strchr index # define strrchr rindex # define memcpy(d, s, n) bcopy((s), (d), (n)) # define memcmp(s1, s2, n) bcmp((s1), (s2), (n)) # define memzero(s, n) bzero((s), (n)) #endif #ifndef RETSIGTYPE # define RETSIGTYPE void #endif #define local static #include typedef uint8_t uch; typedef uint16_t ush; typedef uint32_t ulg; /* Return codes from gzip */ #define OK 0 #define ERROR 1 #define WARNING 2 /* Compression methods (see algorithm.doc) */ #define STORED 0 #define COMPRESSED 1 #define PACKED 2 #define LZHED 3 /* methods 4 to 7 reserved */ #define DEFLATED 8 #define MAX_METHODS 9 extern int method; /* compression method */ /* To save memory for 16 bit systems, some arrays are overlaid between * the various modules: * deflate: prev+head window d_buf l_buf outbuf * unlzw: tab_prefix tab_suffix stack inbuf outbuf * inflate: window inbuf * unpack: window inbuf prefix_len * unlzh: left+right window c_table inbuf c_len * For compression, input is done in window[]. For decompression, output * is done in window except for unlzw. */ #ifndef INBUFSIZ # ifdef SMALL_MEM # define INBUFSIZ 0x2000 /* input buffer size */ # else # define INBUFSIZ 0x8000 /* input buffer size */ # endif #endif #define INBUF_EXTRA 64 /* required by unlzw() */ #ifndef OUTBUFSIZ # ifdef SMALL_MEM # define OUTBUFSIZ 8192 /* output buffer size */ # else # define OUTBUFSIZ 16384 /* output buffer size */ # endif #endif #define OUTBUF_EXTRA 2048 /* required by unlzw() */ #ifndef DIST_BUFSIZE # ifdef SMALL_MEM # define DIST_BUFSIZE 0x2000 /* buffer for distances, see trees.c */ # else # define DIST_BUFSIZE 0x8000 /* buffer for distances, see trees.c */ # endif #endif #ifdef DYN_ALLOC # define EXTERN(type, array) extern type * near array # define DECLARE(type, array, size) type * near array # define ALLOC(type, array, size) { \ array = (type*)fcalloc((size_t)(((size)+1L)/2), 2*sizeof(type)); \ if (array == NULL) error("insufficient memory"); \ } # define FREE(array) {if (array != NULL) fcfree(array), array=NULL;} #else # define EXTERN(type, array) extern type array[] # define DECLARE(type, array, size) type array[size] # define ALLOC(type, array, size) # define FREE(array) #endif EXTERN(uch, inbuf); /* input buffer */ EXTERN(uch, outbuf); /* output buffer */ EXTERN(ush, d_buf); /* buffer for distances, see trees.c */ EXTERN(uch, window); /* Sliding window and suffix table (unlzw) */ #define tab_suffix window #ifndef MAXSEG_64K # define tab_prefix prev /* hash link (see deflate.c) */ # define head (prev+WSIZE) /* hash head (see deflate.c) */ EXTERN(ush, tab_prefix); /* prefix code (see unlzw.c) */ #else # define tab_prefix0 prev # define head tab_prefix1 EXTERN(ush, tab_prefix0); /* prefix for even codes */ EXTERN(ush, tab_prefix1); /* prefix for odd codes */ #endif extern unsigned insize; /* valid bytes in inbuf */ extern unsigned inptr; /* index of next byte to be processed in inbuf */ extern unsigned outcnt; /* bytes in output buffer */ extern long bytes_in; /* number of input bytes */ extern long bytes_out; /* number of output bytes */ extern long header_bytes;/* number of bytes in gzip header */ #define isize bytes_in /* for compatibility with old zip sources (to be cleaned) */ extern int ifd; /* input file descriptor */ extern int ofd; /* output file descriptor */ extern char ifname[]; /* input file name or "stdin" */ extern char ofname[]; /* output file name or "stdout" */ extern char *progname; /* program name */ extern long time_stamp; /* original time stamp (modification time) */ extern long ifile_size; /* input file size, -1 for devices (debug only) */ typedef int file_t; /* Do not use stdio */ #define NO_FILE (-1) /* in memory compression */ #define PACK_MAGIC "\037\036" /* Magic header for packed files */ #define GZIP_MAGIC "\037\213" /* Magic header for gzip files, 1F 8B */ #define OLD_GZIP_MAGIC "\037\236" /* Magic header for gzip 0.5 = freeze 1.x */ #define LZH_MAGIC "\037\240" /* Magic header for SCO LZH Compress files*/ #define PKZIP_MAGIC "\120\113\003\004" /* Magic header for pkzip files */ /* gzip flag byte */ #define ASCII_FLAG 0x01 /* bit 0 set: file probably ascii text */ #define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */ #define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */ #define ORIG_NAME 0x08 /* bit 3 set: original file name present */ #define COMMENT 0x10 /* bit 4 set: file comment present */ #define ENCRYPTED 0x20 /* bit 5 set: file is encrypted */ #define RESERVED 0xC0 /* bit 6,7: reserved */ /* internal file attribute */ #define UNKNOWN 0xffff #define BINARY 0 #define ASCII 1 #ifndef WSIZE # define WSIZE 0x8000 /* window size--must be a power of two, and */ #endif /* at least 32K for zip's deflate method */ #define MIN_MATCH 3 #define MAX_MATCH 258 /* The minimum and maximum match lengths */ #define MIN_LOOKAHEAD (MAX_MATCH+MIN_MATCH+1) /* Minimum amount of lookahead, except at the end of the input file. * See deflate.c for comments about the MIN_MATCH+1. */ #define MAX_DIST (WSIZE-MIN_LOOKAHEAD) /* In order to simplify the code, particularly on 16 bit machines, match * distances are limited to MAX_DIST instead of WSIZE. */ extern int decrypt; /* flag to turn on decryption */ extern int exit_code; /* program exit code */ extern int verbose; /* be verbose (-v) */ extern int quiet; /* be quiet (-q) */ extern int level; /* compression level */ extern int test; /* check .z file integrity */ extern int to_stdout; /* output to stdout (-c) */ extern int save_orig_name; /* set if original name must be saved */ +#define MIN(a,b) ((a) <= (b) ? (a) : (b)) + #define get_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(0)) #define try_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(1)) /* put_byte is used for the compressed output, put_ubyte for the * uncompressed output. However unlzw() uses window for its * suffix table instead of its output buffer, so it does not use put_ubyte * (to be cleaned up). */ #define put_byte(c) {outbuf[outcnt++]=(uch)(c); if (outcnt==OUTBUFSIZ)\ flush_outbuf();} #define put_ubyte(c) {window[outcnt++]=(uch)(c); if (outcnt==WSIZE)\ flush_window();} /* Output a 16 bit value, lsb first */ #define put_short(w) \ { if (outcnt < OUTBUFSIZ-2) { \ outbuf[outcnt++] = (uch) ((w) & 0xff); \ outbuf[outcnt++] = (uch) ((ush)(w) >> 8); \ } else { \ put_byte((uch)((w) & 0xff)); \ put_byte((uch)((ush)(w) >> 8)); \ } \ } /* Output a 32 bit value to the bit stream, lsb first */ #define put_long(n) { \ put_short((n) & 0xffff); \ put_short(((ulg)(n)) >> 16); \ } #define seekable() 0 /* force sequential output */ #define translate_eol 0 /* no option -a yet */ #define tolow(c) (isupper(c) ? (c)-'A'+'a' : (c)) /* force to lower case */ /* Macros for getting two-byte and four-byte header values */ #define SH(p) ((ush)(uch)((p)[0]) | ((ush)(uch)((p)[1]) << 8)) #define LG(p) ((ulg)(SH(p)) | ((ulg)(SH((p)+2)) << 16)) /* Diagnostic functions */ #ifdef DEBUG # define Assert(cond,msg) {if(!(cond)) error(msg);} # define Trace(x) fprintf x # define Tracev(x) {if (verbose) fprintf x ;} # define Tracevv(x) {if (verbose>1) fprintf x ;} # define Tracec(c,x) {if (verbose && (c)) fprintf x ;} # define Tracecv(c,x) {if (verbose>1 && (c)) fprintf x ;} #else # define Assert(cond,msg) # define Trace(x) # define Tracev(x) # define Tracevv(x) # define Tracec(c,x) # define Tracecv(c,x) #endif #define WARN(msg) {if (!quiet) fprintf msg ; \ if (exit_code == OK) exit_code = WARNING;} /* in zip.c: */ extern int zip OF((int in, int out)); extern int file_read OF((char *buf, unsigned size)); /* in unzip.c */ extern int unzip OF((int in, int out)); extern int check_zipfile OF((int in)); /* in unpack.c */ extern int unpack OF((int in, int out)); /* in unlzh.c */ extern int unlzh OF((int in, int out)); /* in gzip.c */ RETSIGTYPE abort_gzip OF((void)); /* in deflate.c */ void lm_init OF((int pack_level, ush *flags)); ulg deflate OF((void)); /* in trees.c */ void ct_init OF((ush *attr, int *method)); int ct_tally OF((int dist, int lc)); ulg flush_block OF((char *buf, ulg stored_len, int eof)); /* in bits.c */ void bi_init OF((file_t zipfile)); void send_bits OF((int value, int length)); unsigned bi_reverse OF((unsigned value, int length)); void bi_windup OF((void)); void copy_block OF((char *buf, unsigned len, int header)); extern int (*read_buf) OF((char *buf, unsigned size)); /* in util.c: */ extern int copy OF((int in, int out)); extern ulg updcrc OF((uch *s, unsigned n)); extern void clear_bufs OF((void)); extern int fill_inbuf OF((int eof_ok)); extern void flush_outbuf OF((void)); extern void flush_window OF((void)); extern void write_buf OF((int fd, voidp buf, unsigned cnt)); extern char *strlwr OF((char *s)); extern char *basename OF((char *fname)); extern void make_simple_name OF((char *name)); extern char *add_envopt OF((int *argcp, char ***argvp, char *env)); extern void error OF((char *m)); extern void warn OF((char *a, char *b)); extern void read_error OF((void)); extern void write_error OF((void)); extern void display_ratio OF((long num, long den, FILE *file)); extern voidp xmalloc OF((unsigned int size)); /* in inflate.c */ extern int inflate OF((void)); Index: releng/5.5/gnu/usr.bin/gzip/inflate.c =================================================================== --- releng/5.5/gnu/usr.bin/gzip/inflate.c (revision 162446) +++ releng/5.5/gnu/usr.bin/gzip/inflate.c (revision 162447) @@ -1,958 +1,958 @@ /* inflate.c -- Not copyrighted 1992 by Mark Adler version c10p1, 10 January 1993 */ /* You can do whatever you like with this source file, though I would prefer that if you modify it and redistribute it that you include comments to that effect with your name and the date. Thank you. [The history has been moved to the file ChangeLog.] */ /* Inflate deflated (PKZIP's method 8 compressed) data. The compression method searches for as much of the current string of bytes (up to a length of 258) in the previous 32K bytes. If it doesn't find any matches (of at least length 3), it codes the next byte. Otherwise, it codes the length of the matched string and its distance backwards from the current position. There is a single Huffman code that codes both single bytes (called "literals") and match lengths. A second Huffman code codes the distance information, which follows a length code. Each length or distance code actually represents a base value and a number of "extra" (sometimes zero) bits to get to add to the base value. At the end of each deflated block is a special end-of-block (EOB) literal/ length code. The decoding process is basically: get a literal/length code; if EOB then done; if a literal, emit the decoded byte; if a length then get the distance and emit the referred-to bytes from the sliding window of previously emitted data. There are (currently) three kinds of inflate blocks: stored, fixed, and dynamic. The compressor deals with some chunk of data at a time, and decides which method to use on a chunk-by-chunk basis. A chunk might typically be 32K or 64K. If the chunk is uncompressible, then the "stored" method is used. In this case, the bytes are simply stored as is, eight bits per byte, with none of the above coding. The bytes are preceded by a count, since there is no longer an EOB code. If the data is compressible, then either the fixed or dynamic methods are used. In the dynamic method, the compressed data is preceded by an encoding of the literal/length and distance Huffman codes that are to be used to decode this block. The representation is itself Huffman coded, and so is preceded by a description of that code. These code descriptions take up a little space, and so for small blocks, there is a predefined set of codes, called the fixed codes. The fixed method is used if the block codes up smaller that way (usually for quite small chunks), otherwise the dynamic method is used. In the latter case, the codes are customized to the probabilities in the current block, and so can code it much better than the pre-determined fixed codes. The Huffman codes themselves are decoded using a mutli-level table lookup, in order to maximize the speed of decoding plus the speed of building the decoding tables. See the comments below that precede the lbits and dbits tuning parameters. */ /* Notes beyond the 1.93a appnote.txt: 1. Distance pointers never point before the beginning of the output stream. 2. Distance pointers can point back across blocks, up to 32k away. 3. There is an implied maximum of 7 bits for the bit length table and 15 bits for the actual data. 4. If only one code exists, then it is encoded using one bit. (Zero would be more efficient, but perhaps a little confusing.) If two codes exist, they are coded using one bit each (0 and 1). 5. There is no way of sending zero distance codes--a dummy must be sent if there are none. (History: a pre 2.0 version of PKZIP would store blocks with no distance codes, but this was discovered to be too harsh a criterion.) Valid only for 1.93a. 2.04c does allow zero distance codes, which is sent as one code of zero bits in length. 6. There are up to 286 literal/length codes. Code 256 represents the end-of-block. Note however that the static length tree defines 288 codes just to fill out the Huffman codes. Codes 286 and 287 cannot be used though, since there is no length base or extra bits defined for them. Similarly, there are up to 30 distance codes. However, static trees define 32 codes (all 5 bits) to fill out the Huffman codes, but the last two had better not show up in the data. 7. Unzip can check dynamic Huffman blocks for complete code sets. The exception is that a single code would not be complete (see #4). 8. The five bits following the block type is really the number of literal codes sent minus 257. 9. Length codes 8,16,16 are interpreted as 13 length codes of 8 bits (1+6+6). Therefore, to output three times the length, you output three codes (1+1+1), whereas to output four times the same length, you only need two codes (1+3). Hmm. 10. In the tree reconstruction algorithm, Code = Code + Increment only if BitLength(i) is not zero. (Pretty obvious.) 11. Correction: 4 Bits: # of Bit Length codes - 4 (4 - 19) 12. Note: length code 284 can represent 227-258, but length code 285 really is 258. The last length deserves its own, short code since it gets used a lot in very redundant files. The length 258 is special since 258 - 3 (the min match length) is 255. 13. The literal/length and distance code bit lengths are read as a single stream of lengths. It is possible (and advantageous) for a repeat code (16, 17, or 18) to go across the boundary between the two sets of lengths. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #if defined(STDC_HEADERS) || !defined(NO_STDLIB_H) # include #endif #include "gzip.h" #define slide window /* Huffman code lookup table entry--this entry is four bytes for machines that have 16-bit pointers (e.g. PC's in the small or medium model). Valid extra bits are 0..13. e == 15 is EOB (end of block), e == 16 means that v is a literal, 16 < e < 32 means that v is a pointer to the next table, which codes e - 16 bits, and lastly e == 99 indicates an unused code. If a code with e == 99 is looked up, this implies an error in the data. */ struct huft { uch e; /* number of extra bits or operation */ uch b; /* number of bits in this code or subcode */ union { ush n; /* literal, length base, or distance base */ struct huft *t; /* pointer to next level of table */ } v; }; /* Function prototypes */ int huft_build OF((unsigned *, unsigned, unsigned, ush *, ush *, struct huft **, int *)); int huft_free OF((struct huft *)); int inflate_codes OF((struct huft *, struct huft *, int, int)); int inflate_stored OF((void)); int inflate_fixed OF((void)); int inflate_dynamic OF((void)); int inflate_block OF((int *)); int inflate OF((void)); /* The inflate algorithm uses a sliding 32K byte window on the uncompressed stream to find repeated byte strings. This is implemented here as a circular buffer. The index is updated simply by incrementing and then and'ing with 0x7fff (32K-1). */ /* It is left to other modules to supply the 32K area. It is assumed to be usable as if it were declared "uch slide[32768];" or as just "uch *slide;" and then malloc'ed in the latter case. The definition must be in unzip.h, included above. */ /* unsigned wp; current position in slide */ #define wp outcnt #define flush_output(w) (wp=(w),flush_window()) /* Tables for deflate from PKZIP's appnote.txt. */ static unsigned border[] = { /* Order of the bit length code lengths */ 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; static ush cplens[] = { /* Copy lengths for literal codes 257..285 */ 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0}; /* note: see note #13 above about the 258 in this list. */ static ush cplext[] = { /* Extra bits for literal codes 257..285 */ 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0, 99, 99}; /* 99==invalid */ static ush cpdist[] = { /* Copy offsets for distance codes 0..29 */ 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577}; static ush cpdext[] = { /* Extra bits for distance codes */ 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13}; /* Macros for inflate() bit peeking and grabbing. The usage is: NEEDBITS(j) x = b & mask_bits[j]; DUMPBITS(j) where NEEDBITS makes sure that b has at least j bits in it, and DUMPBITS removes the bits from b. The macros use the variable k for the number of bits in b. Normally, b and k are register variables for speed, and are initialized at the beginning of a routine that uses these macros from a global bit buffer and count. If we assume that EOB will be the longest code, then we will never ask for bits with NEEDBITS that are beyond the end of the stream. So, NEEDBITS should not read any more bytes than are needed to meet the request. Then no bytes need to be "returned" to the buffer at the end of the last block. However, this assumption is not true for fixed blocks--the EOB code is 7 bits, but the other literal/length codes can be 8 or 9 bits. (The EOB code is shorter than other codes because fixed blocks are generally short. So, while a block always has an EOB, many other literal/length codes have a significantly lower probability of showing up at all.) However, by making the first table have a lookup of seven bits, the EOB code will be found in that first lookup, and so will not require that too many bits be pulled from the stream. */ ulg bb; /* bit buffer */ unsigned bk; /* bits in bit buffer */ ush mask_bits[] = { 0x0000, 0x0001, 0x0003, 0x0007, 0x000f, 0x001f, 0x003f, 0x007f, 0x00ff, 0x01ff, 0x03ff, 0x07ff, 0x0fff, 0x1fff, 0x3fff, 0x7fff, 0xffff }; #ifdef CRYPT uch cc; # define NEXTBYTE() \ (decrypt ? (cc = get_byte(), zdecode(cc), cc) : get_byte()) #else # define NEXTBYTE() (uch)get_byte() #endif #define NEEDBITS(n) {while(k<(n)){b|=((ulg)NEXTBYTE())<>=(n);k-=(n);} /* Huffman code decoding is performed using a multi-level table lookup. The fastest way to decode is to simply build a lookup table whose size is determined by the longest code. However, the time it takes to build this table can also be a factor if the data being decoded is not very long. The most common codes are necessarily the shortest codes, so those codes dominate the decoding time, and hence the speed. The idea is you can have a shorter table that decodes the shorter, more probable codes, and then point to subsidiary tables for the longer codes. The time it costs to decode the longer codes is then traded against the time it takes to make longer tables. This results of this trade are in the variables lbits and dbits below. lbits is the number of bits the first level table for literal/ length codes can decode in one step, and dbits is the same thing for the distance codes. Subsequent tables are also less than or equal to those sizes. These values may be adjusted either when all of the codes are shorter than that, in which case the longest code length in bits is used, or when the shortest code is *longer* than the requested table size, in which case the length of the shortest code in bits is used. There are two different values for the two tables, since they code a different number of possibilities each. The literal/length table codes 286 possible values, or in a flat code, a little over eight bits. The distance table codes 30 possible values, or a little less than five bits, flat. The optimum values for speed end up being about one bit more than those, so lbits is 8+1 and dbits is 5+1. The optimum values may differ though from machine to machine, and possibly even between compilers. Your mileage may vary. */ int lbits = 9; /* bits in base literal/length lookup table */ int dbits = 6; /* bits in base distance lookup table */ /* If BMAX needs to be larger than 16, then h and x[] should be ulg. */ #define BMAX 16 /* maximum bit length of any code (16 for explode) */ #define N_MAX 288 /* maximum number of codes in any set */ unsigned hufts; /* track memory usage */ int huft_build(b, n, s, d, e, t, m) unsigned *b; /* code lengths in bits (all assumed <= BMAX) */ unsigned n; /* number of codes (assumed <= N_MAX) */ unsigned s; /* number of simple-valued codes (0..s-1) */ ush *d; /* list of base values for non-simple codes */ ush *e; /* list of extra bits for non-simple codes */ struct huft **t; /* result: starting table */ int *m; /* maximum lookup bits, returns actual */ /* Given a list of code lengths and a maximum table size, make a set of tables to decode that set of codes. Return zero on success, one if the given code set is incomplete (the tables are still built in this case), two if the input is invalid (all zero length codes or an oversubscribed set of lengths), and three if not enough memory. */ { unsigned a; /* counter for codes of length k */ unsigned c[BMAX+1]; /* bit length count table */ unsigned f; /* i repeats in table every f entries */ int g; /* maximum code length */ int h; /* table level */ register unsigned i; /* counter, current code */ register unsigned j; /* counter */ register int k; /* number of bits in current code */ int l; /* bits per table (returned in m) */ register unsigned *p; /* pointer into c[], b[], or v[] */ register struct huft *q; /* points to current table */ struct huft r; /* table entry for structure assignment */ struct huft *u[BMAX]; /* table stack */ unsigned v[N_MAX]; /* values in order of bit length */ register int w; /* bits before this table == (l * h) */ unsigned x[BMAX+1]; /* bit offsets, then code stack */ unsigned *xp; /* pointer into x */ int y; /* number of dummy codes added */ unsigned z; /* number of entries in current table */ /* Generate counts for each bit length */ memzero(c, sizeof(c)); p = b; i = n; do { Tracecv(*p, (stderr, (n-i >= ' ' && n-i <= '~' ? "%c %d\n" : "0x%x %d\n"), n-i, *p)); c[*p]++; /* assume all entries <= BMAX */ p++; /* Can't combine with above line (Solaris bug) */ } while (--i); if (c[0] == n) /* null input--all zero length codes */ { *t = (struct huft *)NULL; *m = 0; - return 0; + return 2; } /* Find minimum and maximum length, bound *m by those */ l = *m; for (j = 1; j <= BMAX; j++) if (c[j]) break; k = j; /* minimum code length */ if ((unsigned)l < j) l = j; for (i = BMAX; i; i--) if (c[i]) break; g = i; /* maximum code length */ if ((unsigned)l > i) l = i; *m = l; /* Adjust last length count to fill out codes, if needed */ for (y = 1 << j; j < i; j++, y <<= 1) if ((y -= c[j]) < 0) return 2; /* bad input: more codes than bits */ if ((y -= c[i]) < 0) return 2; c[i] += y; /* Generate starting offsets into the value table for each length */ x[1] = j = 0; p = c + 1; xp = x + 2; while (--i) { /* note that i == g from above */ *xp++ = (j += *p++); } /* Make a table of values in order of bit lengths */ p = b; i = 0; do { if ((j = *p++) != 0) v[x[j]++] = i; } while (++i < n); n = x[g]; /* set n to length of v */ /* Generate the Huffman codes and for each, make the table entries */ x[0] = i = 0; /* first Huffman code is zero */ p = v; /* grab values in bit order */ h = -1; /* no tables yet--level -1 */ w = -l; /* bits decoded == (l * h) */ u[0] = (struct huft *)NULL; /* just to keep compilers happy */ q = (struct huft *)NULL; /* ditto */ z = 0; /* ditto */ /* go through the bit lengths (k already is bits in shortest code) */ for (; k <= g; k++) { a = c[k]; while (a--) { /* here i is the Huffman code of length k bits for value *p */ /* make tables up to required level */ while (k > w + l) { h++; w += l; /* previous table always l bits */ /* compute minimum size table less than or equal to l bits */ z = (z = g - w) > (unsigned)l ? l : z; /* upper limit on table size */ if ((f = 1 << (j = k - w)) > a + 1) /* try a k-w bit table */ { /* too few codes for k-w bit table */ f -= a + 1; /* deduct codes from patterns left */ xp = c + k; if (j < z) while (++j < z) /* try smaller tables up to z bits */ { if ((f <<= 1) <= *++xp) break; /* enough codes to use up j bits */ f -= *xp; /* else deduct codes from patterns */ } } z = 1 << j; /* table entries for j-bit table */ /* allocate and link in new table */ if ((q = (struct huft *)malloc((z + 1)*sizeof(struct huft))) == (struct huft *)NULL) { if (h) huft_free(u[0]); return 3; /* not enough memory */ } hufts += z + 1; /* track memory usage */ *t = q + 1; /* link to list for huft_free() */ *(t = &(q->v.t)) = (struct huft *)NULL; u[h] = ++q; /* table starts after link */ /* connect to last table, if there is one */ if (h) { x[h] = i; /* save pattern for backing up */ r.b = (uch)l; /* bits to dump before this table */ r.e = (uch)(16 + j); /* bits in this table */ r.v.t = q; /* pointer to this table */ j = i >> (w - l); /* (get around Turbo C bug) */ u[h-1][j] = r; /* connect to last table */ } } /* set up table entry in r */ r.b = (uch)(k - w); if (p >= v + n) r.e = 99; /* out of values--invalid code */ else if (*p < s) { r.e = (uch)(*p < 256 ? 16 : 15); /* 256 is end-of-block code */ r.v.n = (ush)(*p); /* simple code is just the value */ p++; /* one compiler does not like *p++ */ } else { r.e = (uch)e[*p - s]; /* non-simple--look up in lists */ r.v.n = d[*p++ - s]; } /* fill code-like entries with r */ f = 1 << (k - w); for (j = i >> w; j < z; j += f) q[j] = r; /* backwards increment the k-bit code i */ for (j = 1 << (k - 1); i & j; j >>= 1) i ^= j; i ^= j; /* backup over finished tables */ while ((i & ((1 << w) - 1)) != x[h]) { h--; /* don't need to update q */ w -= l; } } } /* Return true (1) if we were given an incomplete table */ return y != 0 && g != 1; } int huft_free(t) struct huft *t; /* table to free */ /* Free the malloc'ed tables built by huft_build(), which makes a linked list of the tables it made, with the links in a dummy first entry of each table. */ { register struct huft *p, *q; /* Go through linked list, freeing from the malloced (t[-1]) address. */ p = t; while (p != (struct huft *)NULL) { q = (--p)->v.t; free((char*)p); p = q; } return 0; } int inflate_codes(tl, td, bl, bd) struct huft *tl, *td; /* literal/length and distance decoder tables */ int bl, bd; /* number of bits decoded by tl[] and td[] */ /* inflate (decompress) the codes in a deflated (compressed) block. Return an error code or zero if it all goes ok. */ { register unsigned e; /* table entry flag/number of extra bits */ unsigned n, d; /* length and index for copy */ unsigned w; /* current window position */ struct huft *t; /* pointer to table entry */ unsigned ml, md; /* masks for bl and bd bits */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* inflate the coded data */ ml = mask_bits[bl]; /* precompute masks for speed */ md = mask_bits[bd]; for (;;) /* do until end of block */ { NEEDBITS((unsigned)bl) if ((e = (t = tl + ((unsigned)b & ml))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) if (e == 16) /* then it's a literal */ { slide[w++] = (uch)t->v.n; Tracevv((stderr, "%c", slide[w-1])); if (w == WSIZE) { flush_output(w); w = 0; } } else /* it's an EOB or a length */ { /* exit if end of block */ if (e == 15) break; /* get length of block to copy */ NEEDBITS(e) n = t->v.n + ((unsigned)b & mask_bits[e]); DUMPBITS(e); /* decode distance of block to copy */ NEEDBITS((unsigned)bd) if ((e = (t = td + ((unsigned)b & md))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) NEEDBITS(e) d = w - t->v.n - ((unsigned)b & mask_bits[e]); DUMPBITS(e) Tracevv((stderr,"\\[%d,%d]", w-d, n)); /* do the copy */ do { n -= (e = (e = WSIZE - ((d &= WSIZE-1) > w ? d : w)) > n ? n : e); #if !defined(NOMEMCPY) && !defined(DEBUG) if (w - d >= e) /* (this test assumes unsigned comparison) */ { memcpy(slide + w, slide + d, e); w += e; d += e; } else /* do it slow to avoid memcpy() overlap */ #endif /* !NOMEMCPY */ do { slide[w++] = slide[d++]; Tracevv((stderr, "%c", slide[w-1])); } while (--e); if (w == WSIZE) { flush_output(w); w = 0; } } while (n); } } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; /* done */ return 0; } int inflate_stored() /* "decompress" an inflated type 0 (stored) block. */ { unsigned n; /* number of bytes in block */ unsigned w; /* current window position */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* go to byte boundary */ n = k & 7; DUMPBITS(n); /* get the length and its complement */ NEEDBITS(16) n = ((unsigned)b & 0xffff); DUMPBITS(16) NEEDBITS(16) if (n != (unsigned)((~b) & 0xffff)) return 1; /* error in compressed data */ DUMPBITS(16) /* read and output the compressed data */ while (n--) { NEEDBITS(8) slide[w++] = (uch)b; if (w == WSIZE) { flush_output(w); w = 0; } DUMPBITS(8) } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; return 0; } int inflate_fixed() /* decompress an inflated type 1 (fixed Huffman codes) block. We should either replace this with a custom decoder, or at least precompute the Huffman tables. */ { int i; /* temporary variable */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned l[288]; /* length list for huft_build */ /* set up literal table */ for (i = 0; i < 144; i++) l[i] = 8; for (; i < 256; i++) l[i] = 9; for (; i < 280; i++) l[i] = 7; for (; i < 288; i++) /* make a complete, but wrong code set */ l[i] = 8; bl = 7; if ((i = huft_build(l, 288, 257, cplens, cplext, &tl, &bl)) != 0) return i; /* set up distance table */ for (i = 0; i < 30; i++) /* make an incomplete code set */ l[i] = 5; bd = 5; if ((i = huft_build(l, 30, 0, cpdist, cpdext, &td, &bd)) > 1) { huft_free(tl); return i; } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_dynamic() /* decompress an inflated type 2 (dynamic Huffman codes) block. */ { int i; /* temporary variables */ unsigned j; unsigned l; /* last length */ unsigned m; /* mask for bit lengths table */ unsigned n; /* number of lengths to get */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned nb; /* number of bit length codes */ unsigned nl; /* number of literal/length codes */ unsigned nd; /* number of distance codes */ #ifdef PKZIP_BUG_WORKAROUND unsigned ll[288+32]; /* literal/length and distance code lengths */ #else unsigned ll[286+30]; /* literal/length and distance code lengths */ #endif register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in table lengths */ NEEDBITS(5) nl = 257 + ((unsigned)b & 0x1f); /* number of literal/length codes */ DUMPBITS(5) NEEDBITS(5) nd = 1 + ((unsigned)b & 0x1f); /* number of distance codes */ DUMPBITS(5) NEEDBITS(4) nb = 4 + ((unsigned)b & 0xf); /* number of bit length codes */ DUMPBITS(4) #ifdef PKZIP_BUG_WORKAROUND if (nl > 288 || nd > 32) #else if (nl > 286 || nd > 30) #endif return 1; /* bad lengths */ /* read in bit-length-code lengths */ for (j = 0; j < nb; j++) { NEEDBITS(3) ll[border[j]] = (unsigned)b & 7; DUMPBITS(3) } for (; j < 19; j++) ll[border[j]] = 0; /* build decoding table for trees--single level, 7 bit lookup */ bl = 7; if ((i = huft_build(ll, 19, 19, NULL, NULL, &tl, &bl)) != 0) { if (i == 1) huft_free(tl); return i; /* incomplete code set */ } if (tl == NULL) /* Grrrhhh */ return 2; /* read in literal and distance code lengths */ n = nl + nd; m = mask_bits[bl]; i = l = 0; while ((unsigned)i < n) { NEEDBITS((unsigned)bl) j = (td = tl + ((unsigned)b & m))->b; DUMPBITS(j) j = td->v.n; if (j < 16) /* length of code in bits (0..15) */ ll[i++] = l = j; /* save last length in l */ else if (j == 16) /* repeat last length 3 to 6 times */ { NEEDBITS(2) j = 3 + ((unsigned)b & 3); DUMPBITS(2) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = l; } else if (j == 17) /* 3 to 10 zero length codes */ { NEEDBITS(3) j = 3 + ((unsigned)b & 7); DUMPBITS(3) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } else /* j == 18: 11 to 138 zero length codes */ { NEEDBITS(7) j = 11 + ((unsigned)b & 0x7f); DUMPBITS(7) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } } /* free decoding table for trees */ huft_free(tl); /* restore the global bit buffer */ bb = b; bk = k; /* build the decoding tables for literal/length and distance codes */ bl = lbits; if ((i = huft_build(ll, nl, 257, cplens, cplext, &tl, &bl)) != 0) { if (i == 1) { fprintf(stderr, " incomplete literal tree\n"); huft_free(tl); } return i; /* incomplete code set */ } bd = dbits; if ((i = huft_build(ll + nl, nd, 0, cpdist, cpdext, &td, &bd)) != 0) { if (i == 1) { fprintf(stderr, " incomplete distance tree\n"); #ifdef PKZIP_BUG_WORKAROUND i = 0; } #else huft_free(td); } huft_free(tl); return i; /* incomplete code set */ #endif } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_block(e) int *e; /* last block flag */ /* decompress an inflated block */ { unsigned t; /* block type */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in last block bit */ NEEDBITS(1) *e = (int)b & 1; DUMPBITS(1) /* read in block type */ NEEDBITS(2) t = (unsigned)b & 3; DUMPBITS(2) /* restore the global bit buffer */ bb = b; bk = k; /* inflate that block type */ if (t == 2) return inflate_dynamic(); if (t == 0) return inflate_stored(); if (t == 1) return inflate_fixed(); /* bad block type */ return 2; } int inflate() /* decompress an inflated entry */ { int e; /* last block flag */ int r; /* result code */ unsigned h; /* maximum struct huft's malloc'ed */ /* initialize window, bit buffer */ wp = 0; bk = 0; bb = 0; /* decompress until the last block */ h = 0; do { hufts = 0; if ((r = inflate_block(&e)) != 0) return r; if (hufts > h) h = hufts; } while (!e); /* Undo too much lookahead. The next read will be byte aligned so we * can discard unused bits in the last meaningful byte. */ while (bk >= 8) { bk -= 8; inptr--; } /* flush out slide */ flush_output(wp); /* return success */ #ifdef DEBUG fprintf(stderr, "<%u> ", h); #endif /* DEBUG */ return 0; } Index: releng/5.5/gnu/usr.bin/gzip/unlzh.c =================================================================== --- releng/5.5/gnu/usr.bin/gzip/unlzh.c (revision 162446) +++ releng/5.5/gnu/usr.bin/gzip/unlzh.c (revision 162447) @@ -1,401 +1,405 @@ /* unlzh.c -- decompress files in SCO compress -H (LZH) format. * The code in this file is directly derived from the public domain 'ar002' * written by Haruhiko Okumura. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #include "gzip.h" #include "lzw.h" /* just for consistency checking */ /* decode.c */ local unsigned decode OF((unsigned count, uch buffer[])); local void decode_start OF((void)); /* huf.c */ local void huf_decode_start OF((void)); local unsigned decode_c OF((void)); local unsigned decode_p OF((void)); local void read_pt_len OF((int nn, int nbit, int i_special)); local void read_c_len OF((void)); /* io.c */ local void fillbuf OF((int n)); local unsigned getbits OF((int n)); local void init_getbits OF((void)); /* maketbl.c */ local void make_table OF((int nchar, uch bitlen[], int tablebits, ush table[])); #define DICBIT 13 /* 12(-lh4-) or 13(-lh5-) */ #define DICSIZ ((unsigned) 1 << DICBIT) #ifndef CHAR_BIT # define CHAR_BIT 8 #endif #ifndef UCHAR_MAX # define UCHAR_MAX 255 #endif #define BITBUFSIZ (CHAR_BIT * 2 * sizeof(char)) /* Do not use CHAR_BIT * sizeof(bitbuf), does not work on machines * for which short is not on 16 bits (Cray). */ /* encode.c and decode.c */ #define MAXMATCH 256 /* formerly F (not more than UCHAR_MAX + 1) */ #define THRESHOLD 3 /* choose optimal value */ /* huf.c */ #define NC (UCHAR_MAX + MAXMATCH + 2 - THRESHOLD) /* alphabet = {0, 1, 2, ..., NC - 1} */ #define CBIT 9 /* $\lfloor \log_2 NC \rfloor + 1$ */ #define CODE_BIT 16 /* codeword length */ #define NP (DICBIT + 1) #define NT (CODE_BIT + 3) #define PBIT 4 /* smallest integer such that (1U << PBIT) > NP */ #define TBIT 5 /* smallest integer such that (1U << TBIT) > NT */ #if NT > NP # define NPT NT #else # define NPT NP #endif /* local ush left[2 * NC - 1]; */ /* local ush right[2 * NC - 1]; */ #define left prev #define right head #if NC > (1<<(BITS-2)) error cannot overlay left+right and prev #endif /* local uch c_len[NC]; */ #define c_len outbuf #if NC > OUTBUFSIZ error cannot overlay c_len and outbuf #endif local uch pt_len[NPT]; local unsigned blocksize; local ush pt_table[256]; /* local ush c_table[4096]; */ #define c_table d_buf #if (DIST_BUFSIZE-1) < 4095 error cannot overlay c_table and d_buf #endif /*********************************************************** io.c -- input/output ***********************************************************/ local ush bitbuf; local unsigned subbitbuf; local int bitcount; local void fillbuf(n) /* Shift bitbuf n bits left, read n bits */ int n; { bitbuf <<= n; while (n > bitcount) { bitbuf |= subbitbuf << (n -= bitcount); subbitbuf = (unsigned)try_byte(); if ((int)subbitbuf == EOF) subbitbuf = 0; bitcount = CHAR_BIT; } bitbuf |= subbitbuf >> (bitcount -= n); } local unsigned getbits(n) int n; { unsigned x; x = bitbuf >> (BITBUFSIZ - n); fillbuf(n); return x; } local void init_getbits() { bitbuf = 0; subbitbuf = 0; bitcount = 0; fillbuf(BITBUFSIZ); } /*********************************************************** maketbl.c -- make table for decoding ***********************************************************/ local void make_table(nchar, bitlen, tablebits, table) int nchar; uch bitlen[]; int tablebits; ush table[]; { ush count[17], weight[17], start[18], *p; unsigned i, k, len, ch, jutbits, avail, nextcode, mask; for (i = 1; i <= 16; i++) count[i] = 0; - for (i = 0; i < (unsigned)nchar; i++) count[bitlen[i]]++; + for (i = 0; i < (unsigned)nchar; i++) { + if (bitlen[i] > 16) + error("Bad table (case a)\n"); + else count[bitlen[i]]++; + } start[1] = 0; for (i = 1; i <= 16; i++) start[i + 1] = start[i] + (count[i] << (16 - i)); - if ((start[17] & 0xffff) != 0) - error("Bad table\n"); + if ((start[17] & 0xffff) != 0 || tablebits > 16) /* 16 for weight below */ + error("Bad table (case b)\n"); jutbits = 16 - tablebits; for (i = 1; i <= (unsigned)tablebits; i++) { start[i] >>= jutbits; weight[i] = (unsigned) 1 << (tablebits - i); } while (i <= 16) { weight[i] = (unsigned) 1 << (16 - i); i++; } i = start[tablebits + 1] >> jutbits; if (i != 0) { - k = 1 << tablebits; - while (i != k) table[i++] = 0; + k = MIN(1 << tablebits, DIST_BUFSIZE); + while (i < k) table[i++] = 0; } avail = nchar; mask = (unsigned) 1 << (15 - tablebits); for (ch = 0; ch < (unsigned)nchar; ch++) { if ((len = bitlen[ch]) == 0) continue; - nextcode = start[len] + weight[len]; + nextcode = MIN(start[len] + weight[len], DIST_BUFSIZE); if (len <= (unsigned)tablebits) { for (i = start[len]; i < nextcode; i++) table[i] = ch; } else { k = start[len]; p = &table[k >> jutbits]; i = len - tablebits; while (i != 0) { if (*p == 0) { right[avail] = left[avail] = 0; *p = avail++; } if (k & mask) p = &right[*p]; else p = &left[*p]; k <<= 1; i--; } *p = ch; } start[len] = nextcode; } } /*********************************************************** huf.c -- static Huffman ***********************************************************/ local void read_pt_len(nn, nbit, i_special) int nn; int nbit; int i_special; { int i, c, n; unsigned mask; n = getbits(nbit); if (n == 0) { c = getbits(nbit); for (i = 0; i < nn; i++) pt_len[i] = 0; for (i = 0; i < 256; i++) pt_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NPT)) { c = bitbuf >> (BITBUFSIZ - 3); if (c == 7) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 3); while (mask & bitbuf) { mask >>= 1; c++; } } fillbuf((c < 7) ? 3 : c - 3); pt_len[i++] = c; if (i == i_special) { c = getbits(2); - while (--c >= 0) pt_len[i++] = 0; + while (--c >= 0 && i < NPT) pt_len[i++] = 0; } } while (i < nn) pt_len[i++] = 0; make_table(nn, pt_len, 8, pt_table); } } local void read_c_len() { int i, c, n; unsigned mask; n = getbits(CBIT); if (n == 0) { c = getbits(CBIT); for (i = 0; i < NC; i++) c_len[i] = 0; for (i = 0; i < 4096; i++) c_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NC)) { c = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (c >= NT) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) c = right[c]; else c = left [c]; mask >>= 1; - } while (c >= NT); + } while (c >= NT && (mask || c != left[c])); } fillbuf((int) pt_len[c]); if (c <= 2) { if (c == 0) c = 1; else if (c == 1) c = getbits(4) + 3; else c = getbits(CBIT) + 20; - while (--c >= 0) c_len[i++] = 0; + while (--c >= 0 && i < NC) c_len[i++] = 0; } else c_len[i++] = c - 2; } while (i < NC) c_len[i++] = 0; make_table(NC, c_len, 12, c_table); } } local unsigned decode_c() { unsigned j, mask; if (blocksize == 0) { blocksize = getbits(16); if (blocksize == 0) { return NC; /* end of file */ } read_pt_len(NT, TBIT, 3); read_c_len(); read_pt_len(NP, PBIT, -1); } blocksize--; j = c_table[bitbuf >> (BITBUFSIZ - 12)]; if (j >= NC) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 12); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NC); + } while (j >= NC && (mask || j != left[j])); } fillbuf((int) c_len[j]); return j; } local unsigned decode_p() { unsigned j, mask; j = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (j >= NP) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NP); + } while (j >= NP && (mask || j != left[j])); } fillbuf((int) pt_len[j]); if (j != 0) j = ((unsigned) 1 << (j - 1)) + getbits((int) (j - 1)); return j; } local void huf_decode_start() { init_getbits(); blocksize = 0; } /*********************************************************** decode.c ***********************************************************/ local int j; /* remaining bytes to copy */ local int done; /* set at end of input */ local void decode_start() { huf_decode_start(); j = 0; done = 0; } /* Decode the input and return the number of decoded bytes put in buffer */ local unsigned decode(count, buffer) unsigned count; uch buffer[]; /* The calling function must keep the number of bytes to be processed. This function decodes either 'count' bytes or 'DICSIZ' bytes, whichever is smaller, into the array 'buffer[]' of size 'DICSIZ' or more. Call decode_start() once for each new file before calling this function. */ { local unsigned i; unsigned r, c; r = 0; while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } for ( ; ; ) { c = decode_c(); if (c == NC) { done = 1; return r; } if (c <= UCHAR_MAX) { buffer[r] = c; - if (++r == count) return r; + if (++r >= count) return r; } else { j = c - (UCHAR_MAX + 1 - THRESHOLD); i = (r - decode_p() - 1) & (DICSIZ - 1); while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } } } } /* =========================================================================== * Unlzh in to out. Return OK or ERROR. */ int unlzh(in, out) int in; int out; { unsigned n; ifd = in; ofd = out; decode_start(); while (!done) { n = decode((unsigned) DICSIZ, window); if (!test && n > 0) { write_buf(out, (char*)window, n); } } return OK; } Index: releng/5.5/gnu/usr.bin/gzip/unpack.c =================================================================== --- releng/5.5/gnu/usr.bin/gzip/unpack.c (revision 162446) +++ releng/5.5/gnu/usr.bin/gzip/unpack.c (revision 162447) @@ -1,239 +1,238 @@ /* unpack.c -- decompress files in pack format. * Copyright (C) 1992-1993 Jean-loup Gailly * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include "tailor.h" #include "gzip.h" #include "crypt.h" -#define MIN(a,b) ((a) <= (b) ? (a) : (b)) /* The arguments must not have side effects. */ #define MAX_BITLEN 25 /* Maximum length of Huffman codes. (Minor modifications to the code * would be needed to support 32 bits codes, but pack never generates * more than 24 bits anyway.) */ #define LITERALS 256 /* Number of literals, excluding the End of Block (EOB) code */ #define MAX_PEEK 12 /* Maximum number of 'peek' bits used to optimize traversal of the * Huffman tree. */ local ulg orig_len; /* original uncompressed length */ local int max_len; /* maximum bit length of Huffman codes */ local uch literal[LITERALS]; /* The literal bytes present in the Huffman tree. The EOB code is not * represented. */ local int lit_base[MAX_BITLEN+1]; /* All literals of a given bit length are contiguous in literal[] and * have contiguous codes. literal[code+lit_base[len]] is the literal * for a code of len bits. */ local int leaves [MAX_BITLEN+1]; /* Number of leaves for each bit length */ local int parents[MAX_BITLEN+1]; /* Number of parents for each bit length */ local int peek_bits; /* Number of peek bits currently used */ /* local uch prefix_len[1 << MAX_PEEK]; */ #define prefix_len outbuf /* For each bit pattern b of peek_bits bits, prefix_len[b] is the length * of the Huffman code starting with a prefix of b (upper bits), or 0 * if all codes of prefix b have more than peek_bits bits. It is not * necessary to have a huge table (large MAX_PEEK) because most of the * codes encountered in the input stream are short codes (by construction). * So for most codes a single lookup will be necessary. */ #if (1< OUTBUFSIZ error cannot overlay prefix_len and outbuf #endif local ulg bitbuf; /* Bits are added on the low part of bitbuf and read from the high part. */ local int valid; /* number of valid bits in bitbuf */ /* all bits above the last valid bit are always zero */ /* Set code to the next 'bits' input bits without skipping them. code * must be the name of a simple variable and bits must not have side effects. * IN assertions: bits <= 25 (so that we still have room for an extra byte * when valid is only 24), and mask = (1<> (valid-(bits))) & (mask); \ } /* Skip the given number of bits (after having peeked at them): */ #define skip_bits(bits) (valid -= (bits)) #define clear_bitbuf() (valid = 0, bitbuf = 0) /* Local functions */ local void read_tree OF((void)); local void build_tree OF((void)); /* =========================================================================== * Read the Huffman tree. */ local void read_tree() { int len; /* bit length */ int base; /* base offset for a sequence of leaves */ int n; /* Read the original input size, MSB first */ orig_len = 0; for (n = 1; n <= 4; n++) orig_len = (orig_len << 8) | (ulg)get_byte(); max_len = (int)get_byte(); /* maximum bit length of Huffman codes */ if (max_len > MAX_BITLEN) { error("invalid compressed data -- Huffman code > 32 bits"); } /* Get the number of leaves at each bit length */ n = 0; for (len = 1; len <= max_len; len++) { leaves[len] = (int)get_byte(); n += leaves[len]; } if (n > LITERALS) { error("too many leaves in Huffman tree"); } Trace((stderr, "orig_len %ld, max_len %d, leaves %d\n", orig_len, max_len, n)); /* There are at least 2 and at most 256 leaves of length max_len. * (Pack arbitrarily rejects empty files and files consisting of * a single byte even repeated.) To fit the last leaf count in a * byte, it is offset by 2. However, the last literal is the EOB * code, and is not transmitted explicitly in the tree, so we must * adjust here by one only. */ leaves[max_len]++; /* Now read the leaves themselves */ base = 0; for (len = 1; len <= max_len; len++) { /* Remember where the literals of this length start in literal[] : */ lit_base[len] = base; /* And read the literals: */ - for (n = leaves[len]; n > 0; n--) { + for (n = leaves[len]; n > 0 && base < LITERALS; n--) { literal[base++] = (uch)get_byte(); } } leaves[max_len]++; /* Now include the EOB code in the Huffman tree */ } /* =========================================================================== * Build the Huffman tree and the prefix table. */ local void build_tree() { int nodes = 0; /* number of nodes (parents+leaves) at current bit length */ int len; /* current bit length */ uch *prefixp; /* pointer in prefix_len */ for (len = max_len; len >= 1; len--) { /* The number of parent nodes at this level is half the total * number of nodes at parent level: */ nodes >>= 1; parents[len] = nodes; /* Update lit_base by the appropriate bias to skip the parent nodes * (which are not represented in the literal array): */ lit_base[len] -= nodes; /* Restore nodes to be parents+leaves: */ nodes += leaves[len]; } /* Construct the prefix table, from shortest leaves to longest ones. * The shortest code is all ones, so we start at the end of the table. */ peek_bits = MIN(max_len, MAX_PEEK); prefixp = &prefix_len[1< prefix_len) *--prefixp = (uch)len; } /* The length of all other codes is unknown: */ while (prefixp > prefix_len) *--prefixp = 0; } /* =========================================================================== * Unpack in to out. This routine does not support the old pack format * with magic header \037\037. * * IN assertions: the buffer inbuf contains already the beginning of * the compressed data, from offsets inptr to insize-1 included. * The magic header has already been checked. The output buffer is cleared. */ int unpack(in, out) int in, out; /* input and output file descriptors */ { int len; /* Bit length of current code */ unsigned eob; /* End Of Block code */ register unsigned peek; /* lookahead bits */ unsigned peek_mask; /* Mask for peek_bits bits */ ifd = in; ofd = out; read_tree(); /* Read the Huffman tree */ build_tree(); /* Build the prefix table */ clear_bitbuf(); /* Initialize bit input */ peek_mask = (1< 0) { peek >>= peek_bits - len; /* discard the extra bits */ } else { /* Code of more than peek_bits bits, we must traverse the tree */ ulg mask = peek_mask; len = peek_bits; do { len++, mask = (mask<<1)+1; look_bits(peek, len, mask); } while (peek < (unsigned)parents[len]); /* loop as long as peek is a parent node */ } /* At this point, peek is the next complete code, of len bits */ if (peek == eob && len == max_len) break; /* end of file? */ put_ubyte(literal[peek+lit_base[len]]); Tracev((stderr,"%02d %04x %c\n", len, peek, literal[peek+lit_base[len]])); skip_bits(len); } /* for (;;) */ flush_window(); Trace((stderr, "bytes_out %ld\n", bytes_out)); if (orig_len != (ulg)bytes_out) { error("invalid compressed data--length error"); } return OK; } Index: releng/5.5/sys/conf/newvers.sh =================================================================== --- releng/5.5/sys/conf/newvers.sh (revision 162446) +++ releng/5.5/sys/conf/newvers.sh (revision 162447) @@ -1,96 +1,96 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="5.5" -BRANCH="RELEASE-p4" +BRANCH="RELEASE-p5" RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ $(dirname $0)/../sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/1992-$year/" \ -e 's/\[your name here\]\.* /The FreeBSD Project./' \ -e 's/\[your name\]\.*/The FreeBSD Project./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/*- * Copyright (c) 1992-$year The FreeBSD Project. * All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} t=`date` i=`${MAKE:-make} -V KERN_IDENT` cat << EOF > vers.c $COPYRIGHT char sccs[] = "@(#)${VERSION} #${v}: ${t}"; char version[] = "${VERSION} #${v}: ${t}\\n ${u}@${h}:${d}\\n"; char ostype[] = "${TYPE}"; char osrelease[] = "${RELEASE}"; int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF echo `expr ${v} + 1` > version Index: releng/6.0/UPDATING =================================================================== --- releng/6.0/UPDATING (revision 162446) +++ releng/6.0/UPDATING (revision 162447) @@ -1,478 +1,481 @@ Updating Information for FreeBSD STABLE users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. +20060919: p12 FreeBSD-SA-06:21.gzip + Correct multiple vulnerabilities in gzip(1). + 20060906: p11 FreeBSD-SA-06:19.openssl, FreeBSD-SA-06:20.bind Correct incorrect PKCS#1 v1.5 padding validation in crypto(3). [06:19] Correct multiple denial-of-service vulnerabilities in BIND related to SIG Query Processing and Excessive Recursive Queries. [06:20] 20060823: p10 FreeBSD-SA-06:18.ppp Correct buffer overflow in the handling of LCP options in ppp(4). 20060614: p9 FreeBSD-SA-06:17.sendmail Correct a bug in the handling of multipart messages by sendmail(8) which can allow a malformed message to crash a sendmail queue processing process. 20060531: p8 FreeBSD-SA-06:15.ypserv, FreeBSD-SA-06:16.smbfs Enable inadvertantly disabled "securenet" access controls in ypserv. [06:15] Correct a bug in the handling of backslash characters in smbfs which can allow an attacker to escape from a chroot(2). [06:16] 20060419: p7 FreeBSD-SA-06:14.fpu Correct a local information leakage bug affecting AMD FPUs. 20060322: p6 FreeBSD-SA-06:11.ipsec, FreeBSD-SA-06:12.opie, FreeBSD-SA-06:13.sendmail Add missing code needed for the detection of IPSec packet replays. [06:11] Correctly identify the user running opiepasswd(1) when the login name differs from the account name. [06:12] Modify timeout handling logic in sendmail(8) to correct a reported signal handling race condition. [06:13] 20060301: p5 FreeBSD-SA-06:10.nfs Correct a remote kernel panic when processing zero-length RPC records via TCP. 20060125: p4 FreeBSD-SA-06:06.kmem, FreeBSD-SA-06:07.pf Make sure buffers in if_bridge are fully initialized before copying them to userland. Correct a logic error which could allow too much data to be copied into userland. [06:06] Correct an error in pf handling of IP packet fragments which could result in a kernel panic. [06:07] 20060118: p3 FreeBSD-SA-06:05.80211 Correct a buffer overflow when scanning for 802.11 wireless networks which can be provoked by corrupt beacon or probe response frames. 20060111: p2 FreeBSD-SA-06:01.texindex, FreeBSD-SA-06:02.ee, FreeBSD-SA-06:03.cpio, FreeBSD-SA-06:04.ipfw Correct insecure temporary file usage in texindex. [06:01] Correct insecure temporary file usage in ee. [06:02] Correct a race condition when setting file permissions, sanitize file names by default, and fix a buffer overflow when handling files larger than 4GB in cpio. [06:03] Fix an error in the handling of IP fragments in ipfw which can cause a kernel panic. [06:04] 20051219: p1 FreeBSD-EN-05:04.nfs Correct a locking issue in nfs_lookup() where a call to vrele() might be made while holding the vnode mutex, which resulted in kernel panics under certain load patterns. 20051101: FreeBSD 6.0-RELEASE 20051011: FreeBSD-SA-05:21.openssl Correct a man-in-the-middle SSL version rollback vulnerability. 20051001: kern.polling.enable sysctl MIB is now deprecated. Use ifconfig(8) to turn polling(4) on your interfaces. 20050722: The ai_addrlen of a struct addrinfo was changed to a socklen_t to conform to POSIX-2001. This change broke an ABI compatibility on 64 bit architecture. You have to recompile userland programs that use getaddrinfo(3) on 64 bit architecture. 20050711: RELENG_6 branched here. 20050629: The pccard_ifconfig rc.conf variable has been removed and a new variable, ifconfig_DEFAULT has been introduced. Unlike pccard_ifconfig, ifconfig_DEFAULT applies to ALL interfaces that do not have ifconfig_ifn entries rather than just those in removable_interfaces. 20050616: Some previous versions of PAM have permitted the use of non-absolute paths in /etc/pam.conf or /etc/pam.d/* when referring to third party PAM modules in /usr/local/lib. A change has been made to require the use of absolute paths in order to avoid ambiguity and dependence on library path configuration, which may affect existing configurations. 20050610: Major changes to network interface API. All drivers must be recompiled. Drivers not in the base system will need to be updated to the new APIs. 20050609: Changes were made to kinfo_proc in sys/user.h. Please recompile userland, or commands like `fstat', `pkill', `ps', `top' and `w' will not behave correctly. The API and ABI for hwpmc(4) have changed with the addition of sampling support. Please recompile lib/libpmc(3) and usr.sbin/{pmcstat,pmccontrol}. 20050606: The OpenBSD dhclient was imported in place of the ISC dhclient and the network interface configuration scripts were updated accordingly. If you use DHCP to configure your interfaces, you must now run devd. Also, DNS updating was lost so you will need to find a workaround if you use this feature. 20050605: if_bridge was added to the tree. This has changed struct ifnet. Please recompile userland and all network related modules. 20050603: The n_net of a struct netent was changed to an uint32_t, and 1st argument of getnetbyaddr() was changed to an uint32_t, to conform to POSIX-2001. These changes broke an ABI compatibility on 64 bit architecture. With these changes, shlib major of libpcap was bumped. You have to recompile userland programs that use getnetbyaddr(3), getnetbyname(3), getnetent(3) and/or libpcap on 64 bit architecture. 20050528: Kernel parsing of extra options on '#!' first lines of shell scripts has changed. Lines with multiple options likely will fail after this date. For full details, please see http://people.freebsd.org/~gad/Updating-20050528.txt 20050503: The packet filter (pf) code has been updated to OpenBSD 3.7 Please note the changed anchor syntax and the fact that authpf(8) now needs a mounted fdescfs(5) to function. 20050415: The NO_MIXED_MODE kernel option has been removed from the i386 amd64 platforms as its use has been superceded by the new local APIC timer code. Any kernel config files containing this option should be updated. 20050227: The on-disk format of LC_CTYPE files was changed to be machine independent. Please make sure NOT to use NO_CLEAN buildworld when crossing this point. 20050225: The ifi_epoch member of struct if_data has been changed to contain the uptime at which the interface was created or the statistics zeroed rather then the wall clock time because wallclock time may go backwards. This should have no impact unless an snmp implementation is using this value (I know of none at this point.) 20050224: The acpi_perf and acpi_throttle drivers are now part of the acpi(4) main module. They are no longer built separately. 20050223: The layout of struct image_params has changed. You have to recompile all compatibility modules (linux, svr4, etc) for use with the new kernel. 20050223: The p4tcc driver has been merged into cpufreq(4). This makes "options CPU_ENABLE_TCC" obsolete. Please load cpufreq.ko or compile in "device cpufreq" to restore this functionality. 20050220: The responsibility of recomputing the file system summary of a SoftUpdates-enabled dirty volume has been transferred to the background fsck. A rebuild of fsck(8) utility is recommended if you have updated the kernel. To get the old behavior (recompute file system summary at mount time), you can set vfs.ffs.compute_summary_at_mount=1 before mounting the new volume. 20050206: The cpufreq import is complete. As part of this, the sysctls for acpi(4) throttling have been removed. The power_profile script has been updated, so you can use performance/economy_cpu_freq in rc.conf(5) to set AC on/offline cpu frequencies. 20050206: NG_VERSION has been increased. Recompiling kernel (or ng_socket.ko) requires recompiling libnetgraph and userland netgraph utilities. 20050114: Support for abbreviated forms of a number of ipfw options is now deprecated. Warnings are printed to stderr indicating the correct full form when a match occurs. Some abbreviations may be supported at a later date based on user feedback. To be considered for support, abbreviations must be in use prior to this commit and unlikely to be confused with current key words. 20041221: By a popular demand, a lot of NOFOO options were renamed to NO_FOO (see bsd.compat.mk for a full list). The old spellings are still supported, but will cause annoying warnings on stderr. Make sure you upgrade properly (see the COMMON ITEMS: section later in this file). 20041219: Auto-loading of ancillary wlan modules such as wlan_wep has been temporarily disabled; you need to statically configure the modules you need into your kernel or explicitly load them prior to use. Specifically, if you intend to use WEP encryption with an 802.11 device load/configure wlan_wep; if you want to use WPA with the ath driver load/configure wlan_tkip, wlan_ccmp, and wlan_xauth as required. 20041213: The behaviour of ppp(8) has changed slightly. If lqr is enabled (``enable lqr''), older versions would revert to LCP ECHO mode on negotiation failure. Now, ``enable echo'' is required for this behaviour. The ppp version number has been bumped to 3.4.2 to reflect the change. 20041201: The wlan support has been updated to split the crypto support into separate modules. For static WEP you must configure the wlan_wep module in your system or build and install the module in place where it can be loaded (the kernel will auto-load the module when a wep key is configured). 20041201: The ath driver has been updated to split the tx rate control algorithm into a separate module. You need to include either ath_rate_onoe or ath_rate_amrr when configuring the kernel. 20041116: Support for systems with an 80386 CPU has been removed. Please use FreeBSD 5.x or earlier on systems with an 80386. 20041110: We have had a hack which would mount the root filesystem R/W if the device were named 'md*'. As part of the vnode work I'm doing I have had to remove this hack. People building systems which use preloaded MD root filesystems may need to insert a "/sbin/mount -u -o rw /dev/md0 /" in their /etc/rc scripts. 20041104: FreeBSD 5.3 shipped here. 20041102: The size of struct tcpcb has changed again due to the removal of RFC1644 T/TCP. You have to recompile userland programs that read kmem for tcp sockets directly (netstat, sockstat, etc.) 20041022: The size of struct tcpcb has changed. You have to recompile userland programs that read kmem for tcp sockets directly (netstat, sockstat, etc.) 20041016: RELENG_5 branched here. For older entries, please see updating in the RELENG_5 branch. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. Due to several updates to the build infrastructure, source upgrades from versions prior to 5.3 no longer supported. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. With a /usr/obj tree with a fresh buildworld, make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To test a kernel once --------------------- If you just want to boot a kernel once (because you are not sure if it works, or if you want to boot a known bad kernel to provide debugging information) run make installkernel KERNCONF=YOUR_KERNEL_HERE KODIR=/boot/testkernel nextboot -k testkernel To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a 6.X system. Replace ${arch} with the architecture of your machine (e.g. "i386", "alpha", "amd64", "ia64", "pc98", "sparc64", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. make buildworld make kernel KERNCONF=YOUR_KERNEL_HERE [1] [3] mergemaster -p [5] make installworld mergemaster [4] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installworld DESTDIR=${CURRENT_ROOT} cd src/etc; make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from 5.x-stable or higher to 6.x-stable ----------------------------------------------------------- make buildworld [9] make kernel KERNCONF=YOUR_KERNEL_HERE [8] [1] [3] mergemaster -p [5] make installworld mergemaster -i [4] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. [5] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. [8] In order to have a kernel that can run the 5.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD5 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. Make sure that you merge any new devices from GENERIC since the last time you updated your kernel config file. [9] When checking out sources, you must include the -P flag to have cvs prune empty directories. If CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. MAKEOBJDIRPREFIX must be defined in an environment variable, and not on the command line, or in /etc/make.conf. buildworld will warn if it is improperly defined. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -STABLE. Not all things will be listed here, and it only starts on October 16, 2004. Updating files can found in previous releases if your system is older than this. Copyright information: Copyright 1998-2005 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. If you find this document useful, and you want to, you may buy the author a beer. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: releng/6.0/gnu/usr.bin/gzip/gzip.h =================================================================== --- releng/6.0/gnu/usr.bin/gzip/gzip.h (revision 162446) +++ releng/6.0/gnu/usr.bin/gzip/gzip.h (revision 162447) @@ -1,318 +1,320 @@ /* gzip.h -- common declarations for all gzip modules * Copyright (C) 1992-1993 Jean-loup Gailly. * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. * * $FreeBSD$ */ #if defined(__STDC__) || defined(PROTO) # define OF(args) args #else # define OF(args) () #endif #ifdef __STDC__ typedef void *voidp; #else typedef char *voidp; #endif /* I don't like nested includes, but the string and io functions are used * too often */ #include #if !defined(NO_STRING_H) || defined(STDC_HEADERS) # include # if !defined(STDC_HEADERS) && !defined(NO_MEMORY_H) && !defined(__GNUC__) # include # endif # define memzero(s, n) memset ((voidp)(s), 0, (n)) #else # include # define strchr index # define strrchr rindex # define memcpy(d, s, n) bcopy((s), (d), (n)) # define memcmp(s1, s2, n) bcmp((s1), (s2), (n)) # define memzero(s, n) bzero((s), (n)) #endif #ifndef RETSIGTYPE # define RETSIGTYPE void #endif #define local static #include typedef uint8_t uch; typedef uint16_t ush; typedef uint32_t ulg; /* Return codes from gzip */ #define OK 0 #define ERROR 1 #define WARNING 2 /* Compression methods (see algorithm.doc) */ #define STORED 0 #define COMPRESSED 1 #define PACKED 2 #define LZHED 3 /* methods 4 to 7 reserved */ #define DEFLATED 8 #define MAX_METHODS 9 extern int method; /* compression method */ /* To save memory for 16 bit systems, some arrays are overlaid between * the various modules: * deflate: prev+head window d_buf l_buf outbuf * unlzw: tab_prefix tab_suffix stack inbuf outbuf * inflate: window inbuf * unpack: window inbuf prefix_len * unlzh: left+right window c_table inbuf c_len * For compression, input is done in window[]. For decompression, output * is done in window except for unlzw. */ #ifndef INBUFSIZ # ifdef SMALL_MEM # define INBUFSIZ 0x2000 /* input buffer size */ # else # define INBUFSIZ 0x8000 /* input buffer size */ # endif #endif #define INBUF_EXTRA 64 /* required by unlzw() */ #ifndef OUTBUFSIZ # ifdef SMALL_MEM # define OUTBUFSIZ 8192 /* output buffer size */ # else # define OUTBUFSIZ 16384 /* output buffer size */ # endif #endif #define OUTBUF_EXTRA 2048 /* required by unlzw() */ #ifndef DIST_BUFSIZE # ifdef SMALL_MEM # define DIST_BUFSIZE 0x2000 /* buffer for distances, see trees.c */ # else # define DIST_BUFSIZE 0x8000 /* buffer for distances, see trees.c */ # endif #endif #ifdef DYN_ALLOC # define EXTERN(type, array) extern type * near array # define DECLARE(type, array, size) type * near array # define ALLOC(type, array, size) { \ array = (type*)fcalloc((size_t)(((size)+1L)/2), 2*sizeof(type)); \ if (array == NULL) error("insufficient memory"); \ } # define FREE(array) {if (array != NULL) fcfree(array), array=NULL;} #else # define EXTERN(type, array) extern type array[] # define DECLARE(type, array, size) type array[size] # define ALLOC(type, array, size) # define FREE(array) #endif EXTERN(uch, inbuf); /* input buffer */ EXTERN(uch, outbuf); /* output buffer */ EXTERN(ush, d_buf); /* buffer for distances, see trees.c */ EXTERN(uch, window); /* Sliding window and suffix table (unlzw) */ #define tab_suffix window #ifndef MAXSEG_64K # define tab_prefix prev /* hash link (see deflate.c) */ # define head (prev+WSIZE) /* hash head (see deflate.c) */ EXTERN(ush, tab_prefix); /* prefix code (see unlzw.c) */ #else # define tab_prefix0 prev # define head tab_prefix1 EXTERN(ush, tab_prefix0); /* prefix for even codes */ EXTERN(ush, tab_prefix1); /* prefix for odd codes */ #endif extern unsigned insize; /* valid bytes in inbuf */ extern unsigned inptr; /* index of next byte to be processed in inbuf */ extern unsigned outcnt; /* bytes in output buffer */ extern long bytes_in; /* number of input bytes */ extern long bytes_out; /* number of output bytes */ extern long header_bytes;/* number of bytes in gzip header */ #define isize bytes_in /* for compatibility with old zip sources (to be cleaned) */ extern int ifd; /* input file descriptor */ extern int ofd; /* output file descriptor */ extern char ifname[]; /* input file name or "stdin" */ extern char ofname[]; /* output file name or "stdout" */ extern char *progname; /* program name */ extern long time_stamp; /* original time stamp (modification time) */ extern long ifile_size; /* input file size, -1 for devices (debug only) */ typedef int file_t; /* Do not use stdio */ #define NO_FILE (-1) /* in memory compression */ #define PACK_MAGIC "\037\036" /* Magic header for packed files */ #define GZIP_MAGIC "\037\213" /* Magic header for gzip files, 1F 8B */ #define OLD_GZIP_MAGIC "\037\236" /* Magic header for gzip 0.5 = freeze 1.x */ #define LZH_MAGIC "\037\240" /* Magic header for SCO LZH Compress files*/ #define PKZIP_MAGIC "\120\113\003\004" /* Magic header for pkzip files */ /* gzip flag byte */ #define ASCII_FLAG 0x01 /* bit 0 set: file probably ascii text */ #define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */ #define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */ #define ORIG_NAME 0x08 /* bit 3 set: original file name present */ #define COMMENT 0x10 /* bit 4 set: file comment present */ #define ENCRYPTED 0x20 /* bit 5 set: file is encrypted */ #define RESERVED 0xC0 /* bit 6,7: reserved */ /* internal file attribute */ #define UNKNOWN 0xffff #define BINARY 0 #define ASCII 1 #ifndef WSIZE # define WSIZE 0x8000 /* window size--must be a power of two, and */ #endif /* at least 32K for zip's deflate method */ #define MIN_MATCH 3 #define MAX_MATCH 258 /* The minimum and maximum match lengths */ #define MIN_LOOKAHEAD (MAX_MATCH+MIN_MATCH+1) /* Minimum amount of lookahead, except at the end of the input file. * See deflate.c for comments about the MIN_MATCH+1. */ #define MAX_DIST (WSIZE-MIN_LOOKAHEAD) /* In order to simplify the code, particularly on 16 bit machines, match * distances are limited to MAX_DIST instead of WSIZE. */ extern int decrypt; /* flag to turn on decryption */ extern int exit_code; /* program exit code */ extern int verbose; /* be verbose (-v) */ extern int quiet; /* be quiet (-q) */ extern int level; /* compression level */ extern int test; /* check .z file integrity */ extern int to_stdout; /* output to stdout (-c) */ extern int save_orig_name; /* set if original name must be saved */ +#define MIN(a,b) ((a) <= (b) ? (a) : (b)) + #define get_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(0)) #define try_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(1)) /* put_byte is used for the compressed output, put_ubyte for the * uncompressed output. However unlzw() uses window for its * suffix table instead of its output buffer, so it does not use put_ubyte * (to be cleaned up). */ #define put_byte(c) {outbuf[outcnt++]=(uch)(c); if (outcnt==OUTBUFSIZ)\ flush_outbuf();} #define put_ubyte(c) {window[outcnt++]=(uch)(c); if (outcnt==WSIZE)\ flush_window();} /* Output a 16 bit value, lsb first */ #define put_short(w) \ { if (outcnt < OUTBUFSIZ-2) { \ outbuf[outcnt++] = (uch) ((w) & 0xff); \ outbuf[outcnt++] = (uch) ((ush)(w) >> 8); \ } else { \ put_byte((uch)((w) & 0xff)); \ put_byte((uch)((ush)(w) >> 8)); \ } \ } /* Output a 32 bit value to the bit stream, lsb first */ #define put_long(n) { \ put_short((n) & 0xffff); \ put_short(((ulg)(n)) >> 16); \ } #define seekable() 0 /* force sequential output */ #define translate_eol 0 /* no option -a yet */ #define tolow(c) (isupper(c) ? (c)-'A'+'a' : (c)) /* force to lower case */ /* Macros for getting two-byte and four-byte header values */ #define SH(p) ((ush)(uch)((p)[0]) | ((ush)(uch)((p)[1]) << 8)) #define LG(p) ((ulg)(SH(p)) | ((ulg)(SH((p)+2)) << 16)) /* Diagnostic functions */ #ifdef DEBUG # define Assert(cond,msg) {if(!(cond)) error(msg);} # define Trace(x) fprintf x # define Tracev(x) {if (verbose) fprintf x ;} # define Tracevv(x) {if (verbose>1) fprintf x ;} # define Tracec(c,x) {if (verbose && (c)) fprintf x ;} # define Tracecv(c,x) {if (verbose>1 && (c)) fprintf x ;} #else # define Assert(cond,msg) # define Trace(x) # define Tracev(x) # define Tracevv(x) # define Tracec(c,x) # define Tracecv(c,x) #endif #define WARN(msg) {if (!quiet) fprintf msg ; \ if (exit_code == OK) exit_code = WARNING;} /* in zip.c: */ extern int zip OF((int in, int out)); extern int file_read OF((char *buf, unsigned size)); /* in unzip.c */ extern int unzip OF((int in, int out)); extern int check_zipfile OF((int in)); /* in unpack.c */ extern int unpack OF((int in, int out)); /* in unlzh.c */ extern int unlzh OF((int in, int out)); /* in gzip.c */ RETSIGTYPE abort_gzip OF((void)); /* in deflate.c */ void lm_init OF((int pack_level, ush *flags)); ulg deflate OF((void)); /* in trees.c */ void ct_init OF((ush *attr, int *method)); int ct_tally OF((int dist, int lc)); ulg flush_block OF((char *buf, ulg stored_len, int eof)); /* in bits.c */ void bi_init OF((file_t zipfile)); void send_bits OF((int value, int length)); unsigned bi_reverse OF((unsigned value, int length)); void bi_windup OF((void)); void copy_block OF((char *buf, unsigned len, int header)); extern int (*read_buf) OF((char *buf, unsigned size)); /* in util.c: */ extern int copy OF((int in, int out)); extern ulg updcrc OF((uch *s, unsigned n)); extern void clear_bufs OF((void)); extern int fill_inbuf OF((int eof_ok)); extern void flush_outbuf OF((void)); extern void flush_window OF((void)); extern void write_buf OF((int fd, voidp buf, unsigned cnt)); extern char *strlwr OF((char *s)); extern char *basename OF((char *fname)); extern void make_simple_name OF((char *name)); extern char *add_envopt OF((int *argcp, char ***argvp, char *env)); extern void error OF((char *m)); extern void warn OF((char *a, char *b)); extern void read_error OF((void)); extern void write_error OF((void)); extern void display_ratio OF((long num, long den, FILE *file)); extern voidp xmalloc OF((unsigned int size)); /* in inflate.c */ extern int inflate OF((void)); Index: releng/6.0/gnu/usr.bin/gzip/inflate.c =================================================================== --- releng/6.0/gnu/usr.bin/gzip/inflate.c (revision 162446) +++ releng/6.0/gnu/usr.bin/gzip/inflate.c (revision 162447) @@ -1,958 +1,958 @@ /* inflate.c -- Not copyrighted 1992 by Mark Adler version c10p1, 10 January 1993 */ /* You can do whatever you like with this source file, though I would prefer that if you modify it and redistribute it that you include comments to that effect with your name and the date. Thank you. [The history has been moved to the file ChangeLog.] */ /* Inflate deflated (PKZIP's method 8 compressed) data. The compression method searches for as much of the current string of bytes (up to a length of 258) in the previous 32K bytes. If it doesn't find any matches (of at least length 3), it codes the next byte. Otherwise, it codes the length of the matched string and its distance backwards from the current position. There is a single Huffman code that codes both single bytes (called "literals") and match lengths. A second Huffman code codes the distance information, which follows a length code. Each length or distance code actually represents a base value and a number of "extra" (sometimes zero) bits to get to add to the base value. At the end of each deflated block is a special end-of-block (EOB) literal/ length code. The decoding process is basically: get a literal/length code; if EOB then done; if a literal, emit the decoded byte; if a length then get the distance and emit the referred-to bytes from the sliding window of previously emitted data. There are (currently) three kinds of inflate blocks: stored, fixed, and dynamic. The compressor deals with some chunk of data at a time, and decides which method to use on a chunk-by-chunk basis. A chunk might typically be 32K or 64K. If the chunk is uncompressible, then the "stored" method is used. In this case, the bytes are simply stored as is, eight bits per byte, with none of the above coding. The bytes are preceded by a count, since there is no longer an EOB code. If the data is compressible, then either the fixed or dynamic methods are used. In the dynamic method, the compressed data is preceded by an encoding of the literal/length and distance Huffman codes that are to be used to decode this block. The representation is itself Huffman coded, and so is preceded by a description of that code. These code descriptions take up a little space, and so for small blocks, there is a predefined set of codes, called the fixed codes. The fixed method is used if the block codes up smaller that way (usually for quite small chunks), otherwise the dynamic method is used. In the latter case, the codes are customized to the probabilities in the current block, and so can code it much better than the pre-determined fixed codes. The Huffman codes themselves are decoded using a mutli-level table lookup, in order to maximize the speed of decoding plus the speed of building the decoding tables. See the comments below that precede the lbits and dbits tuning parameters. */ /* Notes beyond the 1.93a appnote.txt: 1. Distance pointers never point before the beginning of the output stream. 2. Distance pointers can point back across blocks, up to 32k away. 3. There is an implied maximum of 7 bits for the bit length table and 15 bits for the actual data. 4. If only one code exists, then it is encoded using one bit. (Zero would be more efficient, but perhaps a little confusing.) If two codes exist, they are coded using one bit each (0 and 1). 5. There is no way of sending zero distance codes--a dummy must be sent if there are none. (History: a pre 2.0 version of PKZIP would store blocks with no distance codes, but this was discovered to be too harsh a criterion.) Valid only for 1.93a. 2.04c does allow zero distance codes, which is sent as one code of zero bits in length. 6. There are up to 286 literal/length codes. Code 256 represents the end-of-block. Note however that the static length tree defines 288 codes just to fill out the Huffman codes. Codes 286 and 287 cannot be used though, since there is no length base or extra bits defined for them. Similarly, there are up to 30 distance codes. However, static trees define 32 codes (all 5 bits) to fill out the Huffman codes, but the last two had better not show up in the data. 7. Unzip can check dynamic Huffman blocks for complete code sets. The exception is that a single code would not be complete (see #4). 8. The five bits following the block type is really the number of literal codes sent minus 257. 9. Length codes 8,16,16 are interpreted as 13 length codes of 8 bits (1+6+6). Therefore, to output three times the length, you output three codes (1+1+1), whereas to output four times the same length, you only need two codes (1+3). Hmm. 10. In the tree reconstruction algorithm, Code = Code + Increment only if BitLength(i) is not zero. (Pretty obvious.) 11. Correction: 4 Bits: # of Bit Length codes - 4 (4 - 19) 12. Note: length code 284 can represent 227-258, but length code 285 really is 258. The last length deserves its own, short code since it gets used a lot in very redundant files. The length 258 is special since 258 - 3 (the min match length) is 255. 13. The literal/length and distance code bit lengths are read as a single stream of lengths. It is possible (and advantageous) for a repeat code (16, 17, or 18) to go across the boundary between the two sets of lengths. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #if defined(STDC_HEADERS) || !defined(NO_STDLIB_H) # include #endif #include "gzip.h" #define slide window /* Huffman code lookup table entry--this entry is four bytes for machines that have 16-bit pointers (e.g. PC's in the small or medium model). Valid extra bits are 0..13. e == 15 is EOB (end of block), e == 16 means that v is a literal, 16 < e < 32 means that v is a pointer to the next table, which codes e - 16 bits, and lastly e == 99 indicates an unused code. If a code with e == 99 is looked up, this implies an error in the data. */ struct huft { uch e; /* number of extra bits or operation */ uch b; /* number of bits in this code or subcode */ union { ush n; /* literal, length base, or distance base */ struct huft *t; /* pointer to next level of table */ } v; }; /* Function prototypes */ int huft_build OF((unsigned *, unsigned, unsigned, ush *, ush *, struct huft **, int *)); int huft_free OF((struct huft *)); int inflate_codes OF((struct huft *, struct huft *, int, int)); int inflate_stored OF((void)); int inflate_fixed OF((void)); int inflate_dynamic OF((void)); int inflate_block OF((int *)); int inflate OF((void)); /* The inflate algorithm uses a sliding 32K byte window on the uncompressed stream to find repeated byte strings. This is implemented here as a circular buffer. The index is updated simply by incrementing and then and'ing with 0x7fff (32K-1). */ /* It is left to other modules to supply the 32K area. It is assumed to be usable as if it were declared "uch slide[32768];" or as just "uch *slide;" and then malloc'ed in the latter case. The definition must be in unzip.h, included above. */ /* unsigned wp; current position in slide */ #define wp outcnt #define flush_output(w) (wp=(w),flush_window()) /* Tables for deflate from PKZIP's appnote.txt. */ static unsigned border[] = { /* Order of the bit length code lengths */ 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; static ush cplens[] = { /* Copy lengths for literal codes 257..285 */ 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0}; /* note: see note #13 above about the 258 in this list. */ static ush cplext[] = { /* Extra bits for literal codes 257..285 */ 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0, 99, 99}; /* 99==invalid */ static ush cpdist[] = { /* Copy offsets for distance codes 0..29 */ 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577}; static ush cpdext[] = { /* Extra bits for distance codes */ 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13}; /* Macros for inflate() bit peeking and grabbing. The usage is: NEEDBITS(j) x = b & mask_bits[j]; DUMPBITS(j) where NEEDBITS makes sure that b has at least j bits in it, and DUMPBITS removes the bits from b. The macros use the variable k for the number of bits in b. Normally, b and k are register variables for speed, and are initialized at the beginning of a routine that uses these macros from a global bit buffer and count. If we assume that EOB will be the longest code, then we will never ask for bits with NEEDBITS that are beyond the end of the stream. So, NEEDBITS should not read any more bytes than are needed to meet the request. Then no bytes need to be "returned" to the buffer at the end of the last block. However, this assumption is not true for fixed blocks--the EOB code is 7 bits, but the other literal/length codes can be 8 or 9 bits. (The EOB code is shorter than other codes because fixed blocks are generally short. So, while a block always has an EOB, many other literal/length codes have a significantly lower probability of showing up at all.) However, by making the first table have a lookup of seven bits, the EOB code will be found in that first lookup, and so will not require that too many bits be pulled from the stream. */ ulg bb; /* bit buffer */ unsigned bk; /* bits in bit buffer */ ush mask_bits[] = { 0x0000, 0x0001, 0x0003, 0x0007, 0x000f, 0x001f, 0x003f, 0x007f, 0x00ff, 0x01ff, 0x03ff, 0x07ff, 0x0fff, 0x1fff, 0x3fff, 0x7fff, 0xffff }; #ifdef CRYPT uch cc; # define NEXTBYTE() \ (decrypt ? (cc = get_byte(), zdecode(cc), cc) : get_byte()) #else # define NEXTBYTE() (uch)get_byte() #endif #define NEEDBITS(n) {while(k<(n)){b|=((ulg)NEXTBYTE())<>=(n);k-=(n);} /* Huffman code decoding is performed using a multi-level table lookup. The fastest way to decode is to simply build a lookup table whose size is determined by the longest code. However, the time it takes to build this table can also be a factor if the data being decoded is not very long. The most common codes are necessarily the shortest codes, so those codes dominate the decoding time, and hence the speed. The idea is you can have a shorter table that decodes the shorter, more probable codes, and then point to subsidiary tables for the longer codes. The time it costs to decode the longer codes is then traded against the time it takes to make longer tables. This results of this trade are in the variables lbits and dbits below. lbits is the number of bits the first level table for literal/ length codes can decode in one step, and dbits is the same thing for the distance codes. Subsequent tables are also less than or equal to those sizes. These values may be adjusted either when all of the codes are shorter than that, in which case the longest code length in bits is used, or when the shortest code is *longer* than the requested table size, in which case the length of the shortest code in bits is used. There are two different values for the two tables, since they code a different number of possibilities each. The literal/length table codes 286 possible values, or in a flat code, a little over eight bits. The distance table codes 30 possible values, or a little less than five bits, flat. The optimum values for speed end up being about one bit more than those, so lbits is 8+1 and dbits is 5+1. The optimum values may differ though from machine to machine, and possibly even between compilers. Your mileage may vary. */ int lbits = 9; /* bits in base literal/length lookup table */ int dbits = 6; /* bits in base distance lookup table */ /* If BMAX needs to be larger than 16, then h and x[] should be ulg. */ #define BMAX 16 /* maximum bit length of any code (16 for explode) */ #define N_MAX 288 /* maximum number of codes in any set */ unsigned hufts; /* track memory usage */ int huft_build(b, n, s, d, e, t, m) unsigned *b; /* code lengths in bits (all assumed <= BMAX) */ unsigned n; /* number of codes (assumed <= N_MAX) */ unsigned s; /* number of simple-valued codes (0..s-1) */ ush *d; /* list of base values for non-simple codes */ ush *e; /* list of extra bits for non-simple codes */ struct huft **t; /* result: starting table */ int *m; /* maximum lookup bits, returns actual */ /* Given a list of code lengths and a maximum table size, make a set of tables to decode that set of codes. Return zero on success, one if the given code set is incomplete (the tables are still built in this case), two if the input is invalid (all zero length codes or an oversubscribed set of lengths), and three if not enough memory. */ { unsigned a; /* counter for codes of length k */ unsigned c[BMAX+1]; /* bit length count table */ unsigned f; /* i repeats in table every f entries */ int g; /* maximum code length */ int h; /* table level */ register unsigned i; /* counter, current code */ register unsigned j; /* counter */ register int k; /* number of bits in current code */ int l; /* bits per table (returned in m) */ register unsigned *p; /* pointer into c[], b[], or v[] */ register struct huft *q; /* points to current table */ struct huft r; /* table entry for structure assignment */ struct huft *u[BMAX]; /* table stack */ unsigned v[N_MAX]; /* values in order of bit length */ register int w; /* bits before this table == (l * h) */ unsigned x[BMAX+1]; /* bit offsets, then code stack */ unsigned *xp; /* pointer into x */ int y; /* number of dummy codes added */ unsigned z; /* number of entries in current table */ /* Generate counts for each bit length */ memzero(c, sizeof(c)); p = b; i = n; do { Tracecv(*p, (stderr, (n-i >= ' ' && n-i <= '~' ? "%c %d\n" : "0x%x %d\n"), n-i, *p)); c[*p]++; /* assume all entries <= BMAX */ p++; /* Can't combine with above line (Solaris bug) */ } while (--i); if (c[0] == n) /* null input--all zero length codes */ { *t = (struct huft *)NULL; *m = 0; - return 0; + return 2; } /* Find minimum and maximum length, bound *m by those */ l = *m; for (j = 1; j <= BMAX; j++) if (c[j]) break; k = j; /* minimum code length */ if ((unsigned)l < j) l = j; for (i = BMAX; i; i--) if (c[i]) break; g = i; /* maximum code length */ if ((unsigned)l > i) l = i; *m = l; /* Adjust last length count to fill out codes, if needed */ for (y = 1 << j; j < i; j++, y <<= 1) if ((y -= c[j]) < 0) return 2; /* bad input: more codes than bits */ if ((y -= c[i]) < 0) return 2; c[i] += y; /* Generate starting offsets into the value table for each length */ x[1] = j = 0; p = c + 1; xp = x + 2; while (--i) { /* note that i == g from above */ *xp++ = (j += *p++); } /* Make a table of values in order of bit lengths */ p = b; i = 0; do { if ((j = *p++) != 0) v[x[j]++] = i; } while (++i < n); n = x[g]; /* set n to length of v */ /* Generate the Huffman codes and for each, make the table entries */ x[0] = i = 0; /* first Huffman code is zero */ p = v; /* grab values in bit order */ h = -1; /* no tables yet--level -1 */ w = -l; /* bits decoded == (l * h) */ u[0] = (struct huft *)NULL; /* just to keep compilers happy */ q = (struct huft *)NULL; /* ditto */ z = 0; /* ditto */ /* go through the bit lengths (k already is bits in shortest code) */ for (; k <= g; k++) { a = c[k]; while (a--) { /* here i is the Huffman code of length k bits for value *p */ /* make tables up to required level */ while (k > w + l) { h++; w += l; /* previous table always l bits */ /* compute minimum size table less than or equal to l bits */ z = (z = g - w) > (unsigned)l ? l : z; /* upper limit on table size */ if ((f = 1 << (j = k - w)) > a + 1) /* try a k-w bit table */ { /* too few codes for k-w bit table */ f -= a + 1; /* deduct codes from patterns left */ xp = c + k; if (j < z) while (++j < z) /* try smaller tables up to z bits */ { if ((f <<= 1) <= *++xp) break; /* enough codes to use up j bits */ f -= *xp; /* else deduct codes from patterns */ } } z = 1 << j; /* table entries for j-bit table */ /* allocate and link in new table */ if ((q = (struct huft *)malloc((z + 1)*sizeof(struct huft))) == (struct huft *)NULL) { if (h) huft_free(u[0]); return 3; /* not enough memory */ } hufts += z + 1; /* track memory usage */ *t = q + 1; /* link to list for huft_free() */ *(t = &(q->v.t)) = (struct huft *)NULL; u[h] = ++q; /* table starts after link */ /* connect to last table, if there is one */ if (h) { x[h] = i; /* save pattern for backing up */ r.b = (uch)l; /* bits to dump before this table */ r.e = (uch)(16 + j); /* bits in this table */ r.v.t = q; /* pointer to this table */ j = i >> (w - l); /* (get around Turbo C bug) */ u[h-1][j] = r; /* connect to last table */ } } /* set up table entry in r */ r.b = (uch)(k - w); if (p >= v + n) r.e = 99; /* out of values--invalid code */ else if (*p < s) { r.e = (uch)(*p < 256 ? 16 : 15); /* 256 is end-of-block code */ r.v.n = (ush)(*p); /* simple code is just the value */ p++; /* one compiler does not like *p++ */ } else { r.e = (uch)e[*p - s]; /* non-simple--look up in lists */ r.v.n = d[*p++ - s]; } /* fill code-like entries with r */ f = 1 << (k - w); for (j = i >> w; j < z; j += f) q[j] = r; /* backwards increment the k-bit code i */ for (j = 1 << (k - 1); i & j; j >>= 1) i ^= j; i ^= j; /* backup over finished tables */ while ((i & ((1 << w) - 1)) != x[h]) { h--; /* don't need to update q */ w -= l; } } } /* Return true (1) if we were given an incomplete table */ return y != 0 && g != 1; } int huft_free(t) struct huft *t; /* table to free */ /* Free the malloc'ed tables built by huft_build(), which makes a linked list of the tables it made, with the links in a dummy first entry of each table. */ { register struct huft *p, *q; /* Go through linked list, freeing from the malloced (t[-1]) address. */ p = t; while (p != (struct huft *)NULL) { q = (--p)->v.t; free((char*)p); p = q; } return 0; } int inflate_codes(tl, td, bl, bd) struct huft *tl, *td; /* literal/length and distance decoder tables */ int bl, bd; /* number of bits decoded by tl[] and td[] */ /* inflate (decompress) the codes in a deflated (compressed) block. Return an error code or zero if it all goes ok. */ { register unsigned e; /* table entry flag/number of extra bits */ unsigned n, d; /* length and index for copy */ unsigned w; /* current window position */ struct huft *t; /* pointer to table entry */ unsigned ml, md; /* masks for bl and bd bits */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* inflate the coded data */ ml = mask_bits[bl]; /* precompute masks for speed */ md = mask_bits[bd]; for (;;) /* do until end of block */ { NEEDBITS((unsigned)bl) if ((e = (t = tl + ((unsigned)b & ml))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) if (e == 16) /* then it's a literal */ { slide[w++] = (uch)t->v.n; Tracevv((stderr, "%c", slide[w-1])); if (w == WSIZE) { flush_output(w); w = 0; } } else /* it's an EOB or a length */ { /* exit if end of block */ if (e == 15) break; /* get length of block to copy */ NEEDBITS(e) n = t->v.n + ((unsigned)b & mask_bits[e]); DUMPBITS(e); /* decode distance of block to copy */ NEEDBITS((unsigned)bd) if ((e = (t = td + ((unsigned)b & md))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) NEEDBITS(e) d = w - t->v.n - ((unsigned)b & mask_bits[e]); DUMPBITS(e) Tracevv((stderr,"\\[%d,%d]", w-d, n)); /* do the copy */ do { n -= (e = (e = WSIZE - ((d &= WSIZE-1) > w ? d : w)) > n ? n : e); #if !defined(NOMEMCPY) && !defined(DEBUG) if (w - d >= e) /* (this test assumes unsigned comparison) */ { memcpy(slide + w, slide + d, e); w += e; d += e; } else /* do it slow to avoid memcpy() overlap */ #endif /* !NOMEMCPY */ do { slide[w++] = slide[d++]; Tracevv((stderr, "%c", slide[w-1])); } while (--e); if (w == WSIZE) { flush_output(w); w = 0; } } while (n); } } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; /* done */ return 0; } int inflate_stored() /* "decompress" an inflated type 0 (stored) block. */ { unsigned n; /* number of bytes in block */ unsigned w; /* current window position */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* go to byte boundary */ n = k & 7; DUMPBITS(n); /* get the length and its complement */ NEEDBITS(16) n = ((unsigned)b & 0xffff); DUMPBITS(16) NEEDBITS(16) if (n != (unsigned)((~b) & 0xffff)) return 1; /* error in compressed data */ DUMPBITS(16) /* read and output the compressed data */ while (n--) { NEEDBITS(8) slide[w++] = (uch)b; if (w == WSIZE) { flush_output(w); w = 0; } DUMPBITS(8) } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; return 0; } int inflate_fixed() /* decompress an inflated type 1 (fixed Huffman codes) block. We should either replace this with a custom decoder, or at least precompute the Huffman tables. */ { int i; /* temporary variable */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned l[288]; /* length list for huft_build */ /* set up literal table */ for (i = 0; i < 144; i++) l[i] = 8; for (; i < 256; i++) l[i] = 9; for (; i < 280; i++) l[i] = 7; for (; i < 288; i++) /* make a complete, but wrong code set */ l[i] = 8; bl = 7; if ((i = huft_build(l, 288, 257, cplens, cplext, &tl, &bl)) != 0) return i; /* set up distance table */ for (i = 0; i < 30; i++) /* make an incomplete code set */ l[i] = 5; bd = 5; if ((i = huft_build(l, 30, 0, cpdist, cpdext, &td, &bd)) > 1) { huft_free(tl); return i; } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_dynamic() /* decompress an inflated type 2 (dynamic Huffman codes) block. */ { int i; /* temporary variables */ unsigned j; unsigned l; /* last length */ unsigned m; /* mask for bit lengths table */ unsigned n; /* number of lengths to get */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned nb; /* number of bit length codes */ unsigned nl; /* number of literal/length codes */ unsigned nd; /* number of distance codes */ #ifdef PKZIP_BUG_WORKAROUND unsigned ll[288+32]; /* literal/length and distance code lengths */ #else unsigned ll[286+30]; /* literal/length and distance code lengths */ #endif register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in table lengths */ NEEDBITS(5) nl = 257 + ((unsigned)b & 0x1f); /* number of literal/length codes */ DUMPBITS(5) NEEDBITS(5) nd = 1 + ((unsigned)b & 0x1f); /* number of distance codes */ DUMPBITS(5) NEEDBITS(4) nb = 4 + ((unsigned)b & 0xf); /* number of bit length codes */ DUMPBITS(4) #ifdef PKZIP_BUG_WORKAROUND if (nl > 288 || nd > 32) #else if (nl > 286 || nd > 30) #endif return 1; /* bad lengths */ /* read in bit-length-code lengths */ for (j = 0; j < nb; j++) { NEEDBITS(3) ll[border[j]] = (unsigned)b & 7; DUMPBITS(3) } for (; j < 19; j++) ll[border[j]] = 0; /* build decoding table for trees--single level, 7 bit lookup */ bl = 7; if ((i = huft_build(ll, 19, 19, NULL, NULL, &tl, &bl)) != 0) { if (i == 1) huft_free(tl); return i; /* incomplete code set */ } if (tl == NULL) /* Grrrhhh */ return 2; /* read in literal and distance code lengths */ n = nl + nd; m = mask_bits[bl]; i = l = 0; while ((unsigned)i < n) { NEEDBITS((unsigned)bl) j = (td = tl + ((unsigned)b & m))->b; DUMPBITS(j) j = td->v.n; if (j < 16) /* length of code in bits (0..15) */ ll[i++] = l = j; /* save last length in l */ else if (j == 16) /* repeat last length 3 to 6 times */ { NEEDBITS(2) j = 3 + ((unsigned)b & 3); DUMPBITS(2) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = l; } else if (j == 17) /* 3 to 10 zero length codes */ { NEEDBITS(3) j = 3 + ((unsigned)b & 7); DUMPBITS(3) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } else /* j == 18: 11 to 138 zero length codes */ { NEEDBITS(7) j = 11 + ((unsigned)b & 0x7f); DUMPBITS(7) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } } /* free decoding table for trees */ huft_free(tl); /* restore the global bit buffer */ bb = b; bk = k; /* build the decoding tables for literal/length and distance codes */ bl = lbits; if ((i = huft_build(ll, nl, 257, cplens, cplext, &tl, &bl)) != 0) { if (i == 1) { fprintf(stderr, " incomplete literal tree\n"); huft_free(tl); } return i; /* incomplete code set */ } bd = dbits; if ((i = huft_build(ll + nl, nd, 0, cpdist, cpdext, &td, &bd)) != 0) { if (i == 1) { fprintf(stderr, " incomplete distance tree\n"); #ifdef PKZIP_BUG_WORKAROUND i = 0; } #else huft_free(td); } huft_free(tl); return i; /* incomplete code set */ #endif } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_block(e) int *e; /* last block flag */ /* decompress an inflated block */ { unsigned t; /* block type */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in last block bit */ NEEDBITS(1) *e = (int)b & 1; DUMPBITS(1) /* read in block type */ NEEDBITS(2) t = (unsigned)b & 3; DUMPBITS(2) /* restore the global bit buffer */ bb = b; bk = k; /* inflate that block type */ if (t == 2) return inflate_dynamic(); if (t == 0) return inflate_stored(); if (t == 1) return inflate_fixed(); /* bad block type */ return 2; } int inflate() /* decompress an inflated entry */ { int e; /* last block flag */ int r; /* result code */ unsigned h; /* maximum struct huft's malloc'ed */ /* initialize window, bit buffer */ wp = 0; bk = 0; bb = 0; /* decompress until the last block */ h = 0; do { hufts = 0; if ((r = inflate_block(&e)) != 0) return r; if (hufts > h) h = hufts; } while (!e); /* Undo too much lookahead. The next read will be byte aligned so we * can discard unused bits in the last meaningful byte. */ while (bk >= 8) { bk -= 8; inptr--; } /* flush out slide */ flush_output(wp); /* return success */ #ifdef DEBUG fprintf(stderr, "<%u> ", h); #endif /* DEBUG */ return 0; } Index: releng/6.0/gnu/usr.bin/gzip/unlzh.c =================================================================== --- releng/6.0/gnu/usr.bin/gzip/unlzh.c (revision 162446) +++ releng/6.0/gnu/usr.bin/gzip/unlzh.c (revision 162447) @@ -1,401 +1,405 @@ /* unlzh.c -- decompress files in SCO compress -H (LZH) format. * The code in this file is directly derived from the public domain 'ar002' * written by Haruhiko Okumura. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #include "gzip.h" #include "lzw.h" /* just for consistency checking */ /* decode.c */ local unsigned decode OF((unsigned count, uch buffer[])); local void decode_start OF((void)); /* huf.c */ local void huf_decode_start OF((void)); local unsigned decode_c OF((void)); local unsigned decode_p OF((void)); local void read_pt_len OF((int nn, int nbit, int i_special)); local void read_c_len OF((void)); /* io.c */ local void fillbuf OF((int n)); local unsigned getbits OF((int n)); local void init_getbits OF((void)); /* maketbl.c */ local void make_table OF((int nchar, uch bitlen[], int tablebits, ush table[])); #define DICBIT 13 /* 12(-lh4-) or 13(-lh5-) */ #define DICSIZ ((unsigned) 1 << DICBIT) #ifndef CHAR_BIT # define CHAR_BIT 8 #endif #ifndef UCHAR_MAX # define UCHAR_MAX 255 #endif #define BITBUFSIZ (CHAR_BIT * 2 * sizeof(char)) /* Do not use CHAR_BIT * sizeof(bitbuf), does not work on machines * for which short is not on 16 bits (Cray). */ /* encode.c and decode.c */ #define MAXMATCH 256 /* formerly F (not more than UCHAR_MAX + 1) */ #define THRESHOLD 3 /* choose optimal value */ /* huf.c */ #define NC (UCHAR_MAX + MAXMATCH + 2 - THRESHOLD) /* alphabet = {0, 1, 2, ..., NC - 1} */ #define CBIT 9 /* $\lfloor \log_2 NC \rfloor + 1$ */ #define CODE_BIT 16 /* codeword length */ #define NP (DICBIT + 1) #define NT (CODE_BIT + 3) #define PBIT 4 /* smallest integer such that (1U << PBIT) > NP */ #define TBIT 5 /* smallest integer such that (1U << TBIT) > NT */ #if NT > NP # define NPT NT #else # define NPT NP #endif /* local ush left[2 * NC - 1]; */ /* local ush right[2 * NC - 1]; */ #define left prev #define right head #if NC > (1<<(BITS-2)) error cannot overlay left+right and prev #endif /* local uch c_len[NC]; */ #define c_len outbuf #if NC > OUTBUFSIZ error cannot overlay c_len and outbuf #endif local uch pt_len[NPT]; local unsigned blocksize; local ush pt_table[256]; /* local ush c_table[4096]; */ #define c_table d_buf #if (DIST_BUFSIZE-1) < 4095 error cannot overlay c_table and d_buf #endif /*********************************************************** io.c -- input/output ***********************************************************/ local ush bitbuf; local unsigned subbitbuf; local int bitcount; local void fillbuf(n) /* Shift bitbuf n bits left, read n bits */ int n; { bitbuf <<= n; while (n > bitcount) { bitbuf |= subbitbuf << (n -= bitcount); subbitbuf = (unsigned)try_byte(); if ((int)subbitbuf == EOF) subbitbuf = 0; bitcount = CHAR_BIT; } bitbuf |= subbitbuf >> (bitcount -= n); } local unsigned getbits(n) int n; { unsigned x; x = bitbuf >> (BITBUFSIZ - n); fillbuf(n); return x; } local void init_getbits() { bitbuf = 0; subbitbuf = 0; bitcount = 0; fillbuf(BITBUFSIZ); } /*********************************************************** maketbl.c -- make table for decoding ***********************************************************/ local void make_table(nchar, bitlen, tablebits, table) int nchar; uch bitlen[]; int tablebits; ush table[]; { ush count[17], weight[17], start[18], *p; unsigned i, k, len, ch, jutbits, avail, nextcode, mask; for (i = 1; i <= 16; i++) count[i] = 0; - for (i = 0; i < (unsigned)nchar; i++) count[bitlen[i]]++; + for (i = 0; i < (unsigned)nchar; i++) { + if (bitlen[i] > 16) + error("Bad table (case a)\n"); + else count[bitlen[i]]++; + } start[1] = 0; for (i = 1; i <= 16; i++) start[i + 1] = start[i] + (count[i] << (16 - i)); - if ((start[17] & 0xffff) != 0) - error("Bad table\n"); + if ((start[17] & 0xffff) != 0 || tablebits > 16) /* 16 for weight below */ + error("Bad table (case b)\n"); jutbits = 16 - tablebits; for (i = 1; i <= (unsigned)tablebits; i++) { start[i] >>= jutbits; weight[i] = (unsigned) 1 << (tablebits - i); } while (i <= 16) { weight[i] = (unsigned) 1 << (16 - i); i++; } i = start[tablebits + 1] >> jutbits; if (i != 0) { - k = 1 << tablebits; - while (i != k) table[i++] = 0; + k = MIN(1 << tablebits, DIST_BUFSIZE); + while (i < k) table[i++] = 0; } avail = nchar; mask = (unsigned) 1 << (15 - tablebits); for (ch = 0; ch < (unsigned)nchar; ch++) { if ((len = bitlen[ch]) == 0) continue; - nextcode = start[len] + weight[len]; + nextcode = MIN(start[len] + weight[len], DIST_BUFSIZE); if (len <= (unsigned)tablebits) { for (i = start[len]; i < nextcode; i++) table[i] = ch; } else { k = start[len]; p = &table[k >> jutbits]; i = len - tablebits; while (i != 0) { if (*p == 0) { right[avail] = left[avail] = 0; *p = avail++; } if (k & mask) p = &right[*p]; else p = &left[*p]; k <<= 1; i--; } *p = ch; } start[len] = nextcode; } } /*********************************************************** huf.c -- static Huffman ***********************************************************/ local void read_pt_len(nn, nbit, i_special) int nn; int nbit; int i_special; { int i, c, n; unsigned mask; n = getbits(nbit); if (n == 0) { c = getbits(nbit); for (i = 0; i < nn; i++) pt_len[i] = 0; for (i = 0; i < 256; i++) pt_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NPT)) { c = bitbuf >> (BITBUFSIZ - 3); if (c == 7) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 3); while (mask & bitbuf) { mask >>= 1; c++; } } fillbuf((c < 7) ? 3 : c - 3); pt_len[i++] = c; if (i == i_special) { c = getbits(2); - while (--c >= 0) pt_len[i++] = 0; + while (--c >= 0 && i < NPT) pt_len[i++] = 0; } } while (i < nn) pt_len[i++] = 0; make_table(nn, pt_len, 8, pt_table); } } local void read_c_len() { int i, c, n; unsigned mask; n = getbits(CBIT); if (n == 0) { c = getbits(CBIT); for (i = 0; i < NC; i++) c_len[i] = 0; for (i = 0; i < 4096; i++) c_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NC)) { c = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (c >= NT) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) c = right[c]; else c = left [c]; mask >>= 1; - } while (c >= NT); + } while (c >= NT && (mask || c != left[c])); } fillbuf((int) pt_len[c]); if (c <= 2) { if (c == 0) c = 1; else if (c == 1) c = getbits(4) + 3; else c = getbits(CBIT) + 20; - while (--c >= 0) c_len[i++] = 0; + while (--c >= 0 && i < NC) c_len[i++] = 0; } else c_len[i++] = c - 2; } while (i < NC) c_len[i++] = 0; make_table(NC, c_len, 12, c_table); } } local unsigned decode_c() { unsigned j, mask; if (blocksize == 0) { blocksize = getbits(16); if (blocksize == 0) { return NC; /* end of file */ } read_pt_len(NT, TBIT, 3); read_c_len(); read_pt_len(NP, PBIT, -1); } blocksize--; j = c_table[bitbuf >> (BITBUFSIZ - 12)]; if (j >= NC) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 12); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NC); + } while (j >= NC && (mask || j != left[j])); } fillbuf((int) c_len[j]); return j; } local unsigned decode_p() { unsigned j, mask; j = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (j >= NP) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NP); + } while (j >= NP && (mask || j != left[j])); } fillbuf((int) pt_len[j]); if (j != 0) j = ((unsigned) 1 << (j - 1)) + getbits((int) (j - 1)); return j; } local void huf_decode_start() { init_getbits(); blocksize = 0; } /*********************************************************** decode.c ***********************************************************/ local int j; /* remaining bytes to copy */ local int done; /* set at end of input */ local void decode_start() { huf_decode_start(); j = 0; done = 0; } /* Decode the input and return the number of decoded bytes put in buffer */ local unsigned decode(count, buffer) unsigned count; uch buffer[]; /* The calling function must keep the number of bytes to be processed. This function decodes either 'count' bytes or 'DICSIZ' bytes, whichever is smaller, into the array 'buffer[]' of size 'DICSIZ' or more. Call decode_start() once for each new file before calling this function. */ { local unsigned i; unsigned r, c; r = 0; while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } for ( ; ; ) { c = decode_c(); if (c == NC) { done = 1; return r; } if (c <= UCHAR_MAX) { buffer[r] = c; - if (++r == count) return r; + if (++r >= count) return r; } else { j = c - (UCHAR_MAX + 1 - THRESHOLD); i = (r - decode_p() - 1) & (DICSIZ - 1); while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } } } } /* =========================================================================== * Unlzh in to out. Return OK or ERROR. */ int unlzh(in, out) int in; int out; { unsigned n; ifd = in; ofd = out; decode_start(); while (!done) { n = decode((unsigned) DICSIZ, window); if (!test && n > 0) { write_buf(out, (char*)window, n); } } return OK; } Index: releng/6.0/gnu/usr.bin/gzip/unpack.c =================================================================== --- releng/6.0/gnu/usr.bin/gzip/unpack.c (revision 162446) +++ releng/6.0/gnu/usr.bin/gzip/unpack.c (revision 162447) @@ -1,239 +1,238 @@ /* unpack.c -- decompress files in pack format. * Copyright (C) 1992-1993 Jean-loup Gailly * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include "tailor.h" #include "gzip.h" #include "crypt.h" -#define MIN(a,b) ((a) <= (b) ? (a) : (b)) /* The arguments must not have side effects. */ #define MAX_BITLEN 25 /* Maximum length of Huffman codes. (Minor modifications to the code * would be needed to support 32 bits codes, but pack never generates * more than 24 bits anyway.) */ #define LITERALS 256 /* Number of literals, excluding the End of Block (EOB) code */ #define MAX_PEEK 12 /* Maximum number of 'peek' bits used to optimize traversal of the * Huffman tree. */ local ulg orig_len; /* original uncompressed length */ local int max_len; /* maximum bit length of Huffman codes */ local uch literal[LITERALS]; /* The literal bytes present in the Huffman tree. The EOB code is not * represented. */ local int lit_base[MAX_BITLEN+1]; /* All literals of a given bit length are contiguous in literal[] and * have contiguous codes. literal[code+lit_base[len]] is the literal * for a code of len bits. */ local int leaves [MAX_BITLEN+1]; /* Number of leaves for each bit length */ local int parents[MAX_BITLEN+1]; /* Number of parents for each bit length */ local int peek_bits; /* Number of peek bits currently used */ /* local uch prefix_len[1 << MAX_PEEK]; */ #define prefix_len outbuf /* For each bit pattern b of peek_bits bits, prefix_len[b] is the length * of the Huffman code starting with a prefix of b (upper bits), or 0 * if all codes of prefix b have more than peek_bits bits. It is not * necessary to have a huge table (large MAX_PEEK) because most of the * codes encountered in the input stream are short codes (by construction). * So for most codes a single lookup will be necessary. */ #if (1< OUTBUFSIZ error cannot overlay prefix_len and outbuf #endif local ulg bitbuf; /* Bits are added on the low part of bitbuf and read from the high part. */ local int valid; /* number of valid bits in bitbuf */ /* all bits above the last valid bit are always zero */ /* Set code to the next 'bits' input bits without skipping them. code * must be the name of a simple variable and bits must not have side effects. * IN assertions: bits <= 25 (so that we still have room for an extra byte * when valid is only 24), and mask = (1<> (valid-(bits))) & (mask); \ } /* Skip the given number of bits (after having peeked at them): */ #define skip_bits(bits) (valid -= (bits)) #define clear_bitbuf() (valid = 0, bitbuf = 0) /* Local functions */ local void read_tree OF((void)); local void build_tree OF((void)); /* =========================================================================== * Read the Huffman tree. */ local void read_tree() { int len; /* bit length */ int base; /* base offset for a sequence of leaves */ int n; /* Read the original input size, MSB first */ orig_len = 0; for (n = 1; n <= 4; n++) orig_len = (orig_len << 8) | (ulg)get_byte(); max_len = (int)get_byte(); /* maximum bit length of Huffman codes */ if (max_len > MAX_BITLEN) { error("invalid compressed data -- Huffman code > 32 bits"); } /* Get the number of leaves at each bit length */ n = 0; for (len = 1; len <= max_len; len++) { leaves[len] = (int)get_byte(); n += leaves[len]; } if (n > LITERALS) { error("too many leaves in Huffman tree"); } Trace((stderr, "orig_len %ld, max_len %d, leaves %d\n", orig_len, max_len, n)); /* There are at least 2 and at most 256 leaves of length max_len. * (Pack arbitrarily rejects empty files and files consisting of * a single byte even repeated.) To fit the last leaf count in a * byte, it is offset by 2. However, the last literal is the EOB * code, and is not transmitted explicitly in the tree, so we must * adjust here by one only. */ leaves[max_len]++; /* Now read the leaves themselves */ base = 0; for (len = 1; len <= max_len; len++) { /* Remember where the literals of this length start in literal[] : */ lit_base[len] = base; /* And read the literals: */ - for (n = leaves[len]; n > 0; n--) { + for (n = leaves[len]; n > 0 && base < LITERALS; n--) { literal[base++] = (uch)get_byte(); } } leaves[max_len]++; /* Now include the EOB code in the Huffman tree */ } /* =========================================================================== * Build the Huffman tree and the prefix table. */ local void build_tree() { int nodes = 0; /* number of nodes (parents+leaves) at current bit length */ int len; /* current bit length */ uch *prefixp; /* pointer in prefix_len */ for (len = max_len; len >= 1; len--) { /* The number of parent nodes at this level is half the total * number of nodes at parent level: */ nodes >>= 1; parents[len] = nodes; /* Update lit_base by the appropriate bias to skip the parent nodes * (which are not represented in the literal array): */ lit_base[len] -= nodes; /* Restore nodes to be parents+leaves: */ nodes += leaves[len]; } /* Construct the prefix table, from shortest leaves to longest ones. * The shortest code is all ones, so we start at the end of the table. */ peek_bits = MIN(max_len, MAX_PEEK); prefixp = &prefix_len[1< prefix_len) *--prefixp = (uch)len; } /* The length of all other codes is unknown: */ while (prefixp > prefix_len) *--prefixp = 0; } /* =========================================================================== * Unpack in to out. This routine does not support the old pack format * with magic header \037\037. * * IN assertions: the buffer inbuf contains already the beginning of * the compressed data, from offsets inptr to insize-1 included. * The magic header has already been checked. The output buffer is cleared. */ int unpack(in, out) int in, out; /* input and output file descriptors */ { int len; /* Bit length of current code */ unsigned eob; /* End Of Block code */ register unsigned peek; /* lookahead bits */ unsigned peek_mask; /* Mask for peek_bits bits */ ifd = in; ofd = out; read_tree(); /* Read the Huffman tree */ build_tree(); /* Build the prefix table */ clear_bitbuf(); /* Initialize bit input */ peek_mask = (1< 0) { peek >>= peek_bits - len; /* discard the extra bits */ } else { /* Code of more than peek_bits bits, we must traverse the tree */ ulg mask = peek_mask; len = peek_bits; do { len++, mask = (mask<<1)+1; look_bits(peek, len, mask); } while (peek < (unsigned)parents[len]); /* loop as long as peek is a parent node */ } /* At this point, peek is the next complete code, of len bits */ if (peek == eob && len == max_len) break; /* end of file? */ put_ubyte(literal[peek+lit_base[len]]); Tracev((stderr,"%02d %04x %c\n", len, peek, literal[peek+lit_base[len]])); skip_bits(len); } /* for (;;) */ flush_window(); Trace((stderr, "bytes_out %ld\n", bytes_out)); if (orig_len != (ulg)bytes_out) { error("invalid compressed data--length error"); } return OK; } Index: releng/6.0/sys/conf/newvers.sh =================================================================== --- releng/6.0/sys/conf/newvers.sh (revision 162446) +++ releng/6.0/sys/conf/newvers.sh (revision 162447) @@ -1,103 +1,103 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="6.0" -BRANCH="RELEASE-p11" +BRANCH="RELEASE-p12" if [ "X${BRANCH_OVERRIDE}" != "X" ]; then BRANCH=${BRANCH_OVERRIDE} fi RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ $(dirname $0)/../sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/1992-$year/" \ -e 's/\[your name here\]\.* /The FreeBSD Project./' \ -e 's/\[your name\]\.*/The FreeBSD Project./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/*- * Copyright (c) 1992-$year The FreeBSD Project. * All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} t=`date` i=`${MAKE:-make} -V KERN_IDENT` cat << EOF > vers.c $COPYRIGHT #define SCCSSTR "@(#)${VERSION} #${v}: ${t}" #define VERSTR "${VERSION} #${v}: ${t}\\n ${u}@${h}:${d}\\n" #define RELSTR "${RELEASE}" char sccs[sizeof(SCCSSTR) > 128 ? sizeof(SCCSSTR) : 128] = SCCSSTR; char version[sizeof(VERSTR) > 256 ? sizeof(VERSTR) : 256] = VERSTR; char ostype[] = "${TYPE}"; char osrelease[sizeof(RELSTR) > 32 ? sizeof(RELSTR) : 32] = RELSTR; int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF echo `expr ${v} + 1` > version Index: releng/6.1/UPDATING =================================================================== --- releng/6.1/UPDATING (revision 162446) +++ releng/6.1/UPDATING (revision 162447) @@ -1,525 +1,528 @@ Updating Information for FreeBSD STABLE users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. +20060919: p7 FreeBSD-SA-06:21.gzip + Correct multiple vulnerabilities in gzip(1). + 20060906: p6 FreeBSD-SA-06:19.openssl, FreeBSD-SA-06:20.bind Correct incorrect PKCS#1 v1.5 padding validation in crypto(3). [06:19] Correct multiple denial-of-service vulnerabilities in BIND related to SIG Query Processing and Excessive Recursive Queries. [06:20] 20060828: p5 FreeBSD-EN-06:02.net Correct several problems in the network stack, including a locally-triggered kernel panic, inaccuracy in reported memory allocation statistics, and problems on IPv6 point-to-point links. 20060823: p4 FreeBSD-SA-06:18.ppp Correct buffer overflow in the handling of LCP options in ppp(4). 20060707: p3 FreeBSD-EN-06:01.jail Correct bug in the startup scripts for jails. 20060614: p2 FreeBSD-SA-06:17.sendmail Correct a bug in the handling of multipart messages by sendmail(8) which can allow a malformed message to crash a sendmail queue processing process. 20060531: p1 FreeBSD-SA-06:15.ypserv, FreeBSD-SA-06:16.smbfs Enable inadvertantly disabled "securenet" access controls in ypserv. [06:15] Correct a bug in the handling of backslash characters in smbfs which can allow an attacker to escape from a chroot(2). [06:16] 20060506: FreeBSD 6.1-RELEASE 20060419: FreeBSD-SA-06:14.fpu Correct a local information leakage bug affecting AMD FPUs. 20060309: The support for Linux applications the 3dfx kernel module used to provide is in a separate module named 3dfx_linux now. The latter will pull in 3dfx.ko and linux.ko through its dependencies when loaded. 20060211: An IPv6 support of ipfw was enabled by default. If you don't want to filter an IPv6 by ipfw, please add following line into your ipfw rule: pass ip6 from any to any 20060210: Now most modules get their build-time options from the kernel configuration file. A few modules still have fixed options due to their non-conformant implementation, but they will be corrected eventually. You may need to review the options of the modules in use, explicitly specify the non-default options in the kernel configuration file, and rebuild the kernel and modules afterwards. 20060204: The 'audit' group was added to support the new auditing functionality in the base system. Be sure to follow the directions for updating, including the requirement to run mergemaster -p. 20060127: The i386 loader(8) now defaults to the serial speed set by the previous boot stage, if the comconsole is already in use. If you've changed BOOT_COMCONSOLE_SPEED in make.conf(5) and installed a new loader, but have not rebuilt and reinstalled the boot blocks, then your loader will leave the console at 9600 baud. Either install the new boot blocks, or set comconsole_speed in loader.conf(5). Note that the new boot blocks also support a -S flag described in boot(8). 20060122: /etc/rc.d/ppp-user has been renamed to /etc/rc.d/ppp. Its /etc/rc.conf.d configuration file has been `ppp' from the beginning, and hence there is no need to touch it. 20060113: si(4)'s device files now contain the unit number. Uses of {cua,tty}A[0-9a-f] should be replaced by {cua,tty}A0[0-9a-f]. 20051230: A lot of fixes and new features in the soundsystem. To get all benefits, you may want to recompile mplayer (if installed) after booting the new world. 20051222: Bug fixes to the trimdomain(3) function in libutil may result in slight changes to the host names appearing in log files under relatively rare circumstances. 20051220: Scripts in the local_startup directories (as defined in /etc/defaults/rc.conf) that have the new rc.d semantics will now be run as part of the base system rcorder. If there are errors or problems with one of these local scripts, it could cause boot problems. If you encounter such problems, boot in single user mode, remove that script from the */rc.d directory. Please report the problem to the port's maintainer, and the freebsd-ports@freebsd.org mailing list. 20051215: The setkey(8) utility was moved from /usr/sbin/setkey to /sbin/setkey. You may want to update scripts which depend on its location. 20051108: rp(4)'s device files now contain the unit number. Uses of {cua,tty}R[0-9a-f] should be replaced by {cua,tty}R0[0-9a-f]. 20051101: FreeBSD 6.0-RELEASE 20051001: kern.polling.enable sysctl MIB is now deprecated. Use ifconfig(8) to turn polling(4) on your interfaces. 20050722: The ai_addrlen of a struct addrinfo was changed to a socklen_t to conform to POSIX-2001. This change broke an ABI compatibility on 64 bit architecture. You have to recompile userland programs that use getaddrinfo(3) on 64 bit architecture. 20050711: RELENG_6 branched here. 20050629: The pccard_ifconfig rc.conf variable has been removed and a new variable, ifconfig_DEFAULT has been introduced. Unlike pccard_ifconfig, ifconfig_DEFAULT applies to ALL interfaces that do not have ifconfig_ifn entries rather than just those in removable_interfaces. 20050616: Some previous versions of PAM have permitted the use of non-absolute paths in /etc/pam.conf or /etc/pam.d/* when referring to third party PAM modules in /usr/local/lib. A change has been made to require the use of absolute paths in order to avoid ambiguity and dependence on library path configuration, which may affect existing configurations. 20050610: Major changes to network interface API. All drivers must be recompiled. Drivers not in the base system will need to be updated to the new APIs. 20050609: Changes were made to kinfo_proc in sys/user.h. Please recompile userland, or commands like `fstat', `pkill', `ps', `top' and `w' will not behave correctly. The API and ABI for hwpmc(4) have changed with the addition of sampling support. Please recompile lib/libpmc(3) and usr.sbin/{pmcstat,pmccontrol}. 20050606: The OpenBSD dhclient was imported in place of the ISC dhclient and the network interface configuration scripts were updated accordingly. If you use DHCP to configure your interfaces, you must now run devd. Also, DNS updating was lost so you will need to find a workaround if you use this feature. 20050605: if_bridge was added to the tree. This has changed struct ifnet. Please recompile userland and all network related modules. 20050603: The n_net of a struct netent was changed to an uint32_t, and 1st argument of getnetbyaddr() was changed to an uint32_t, to conform to POSIX-2001. These changes broke an ABI compatibility on 64 bit architecture. With these changes, shlib major of libpcap was bumped. You have to recompile userland programs that use getnetbyaddr(3), getnetbyname(3), getnetent(3) and/or libpcap on 64 bit architecture. 20050528: Kernel parsing of extra options on '#!' first lines of shell scripts has changed. Lines with multiple options likely will fail after this date. For full details, please see http://people.freebsd.org/~gad/Updating-20050528.txt 20050503: The packet filter (pf) code has been updated to OpenBSD 3.7 Please note the changed anchor syntax and the fact that authpf(8) now needs a mounted fdescfs(5) to function. 20050415: The NO_MIXED_MODE kernel option has been removed from the i386 amd64 platforms as its use has been superceded by the new local APIC timer code. Any kernel config files containing this option should be updated. 20050227: The on-disk format of LC_CTYPE files was changed to be machine independent. Please make sure NOT to use NO_CLEAN buildworld when crossing this point. Crossing this point also requires recompile or reinstall of all locale depended packages. 20050225: The ifi_epoch member of struct if_data has been changed to contain the uptime at which the interface was created or the statistics zeroed rather then the wall clock time because wallclock time may go backwards. This should have no impact unless an snmp implementation is using this value (I know of none at this point.) 20050224: The acpi_perf and acpi_throttle drivers are now part of the acpi(4) main module. They are no longer built separately. 20050223: The layout of struct image_params has changed. You have to recompile all compatibility modules (linux, svr4, etc) for use with the new kernel. 20050223: The p4tcc driver has been merged into cpufreq(4). This makes "options CPU_ENABLE_TCC" obsolete. Please load cpufreq.ko or compile in "device cpufreq" to restore this functionality. 20050220: The responsibility of recomputing the file system summary of a SoftUpdates-enabled dirty volume has been transferred to the background fsck. A rebuild of fsck(8) utility is recommended if you have updated the kernel. To get the old behavior (recompute file system summary at mount time), you can set vfs.ffs.compute_summary_at_mount=1 before mounting the new volume. 20050206: The cpufreq import is complete. As part of this, the sysctls for acpi(4) throttling have been removed. The power_profile script has been updated, so you can use performance/economy_cpu_freq in rc.conf(5) to set AC on/offline cpu frequencies. 20050206: NG_VERSION has been increased. Recompiling kernel (or ng_socket.ko) requires recompiling libnetgraph and userland netgraph utilities. 20050114: Support for abbreviated forms of a number of ipfw options is now deprecated. Warnings are printed to stderr indicating the correct full form when a match occurs. Some abbreviations may be supported at a later date based on user feedback. To be considered for support, abbreviations must be in use prior to this commit and unlikely to be confused with current key words. 20041221: By a popular demand, a lot of NOFOO options were renamed to NO_FOO (see bsd.compat.mk for a full list). The old spellings are still supported, but will cause annoying warnings on stderr. Make sure you upgrade properly (see the COMMON ITEMS: section later in this file). 20041219: Auto-loading of ancillary wlan modules such as wlan_wep has been temporarily disabled; you need to statically configure the modules you need into your kernel or explicitly load them prior to use. Specifically, if you intend to use WEP encryption with an 802.11 device load/configure wlan_wep; if you want to use WPA with the ath driver load/configure wlan_tkip, wlan_ccmp, and wlan_xauth as required. 20041213: The behaviour of ppp(8) has changed slightly. If lqr is enabled (``enable lqr''), older versions would revert to LCP ECHO mode on negotiation failure. Now, ``enable echo'' is required for this behaviour. The ppp version number has been bumped to 3.4.2 to reflect the change. 20041201: The wlan support has been updated to split the crypto support into separate modules. For static WEP you must configure the wlan_wep module in your system or build and install the module in place where it can be loaded (the kernel will auto-load the module when a wep key is configured). 20041201: The ath driver has been updated to split the tx rate control algorithm into a separate module. You need to include either ath_rate_onoe or ath_rate_amrr when configuring the kernel. 20041116: Support for systems with an 80386 CPU has been removed. Please use FreeBSD 5.x or earlier on systems with an 80386. 20041110: We have had a hack which would mount the root filesystem R/W if the device were named 'md*'. As part of the vnode work I'm doing I have had to remove this hack. People building systems which use preloaded MD root filesystems may need to insert a "/sbin/mount -u -o rw /dev/md0 /" in their /etc/rc scripts. 20041104: FreeBSD 5.3 shipped here. 20041102: The size of struct tcpcb has changed again due to the removal of RFC1644 T/TCP. You have to recompile userland programs that read kmem for tcp sockets directly (netstat, sockstat, etc.) 20041022: The size of struct tcpcb has changed. You have to recompile userland programs that read kmem for tcp sockets directly (netstat, sockstat, etc.) 20041016: RELENG_5 branched here. For older entries, please see updating in the RELENG_5 branch. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. Due to several updates to the build infrastructure, source upgrades from versions prior to 5.3 no longer supported. When upgrading from one major version to another it is generally best to upgrade to the latest code in the currently installed branch first, then do an upgrade to the new branch. This is the best-tested upgrade path, and has the highest probability of being successful. Please try this approach before reporting problems with a major version upgrade. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. With a /usr/obj tree with a fresh buildworld, make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To test a kernel once --------------------- If you just want to boot a kernel once (because you are not sure if it works, or if you want to boot a known bad kernel to provide debugging information) run make installkernel KERNCONF=YOUR_KERNEL_HERE KODIR=/boot/testkernel nextboot -k testkernel To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a 6.X system. Replace ${arch} with the architecture of your machine (e.g. "i386", "alpha", "amd64", "ia64", "pc98", "sparc64", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. make buildworld make kernel KERNCONF=YOUR_KERNEL_HERE [1] [3] mergemaster -p [5] make installworld make delete-old mergemaster [4] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installworld DESTDIR=${CURRENT_ROOT} cd src/etc; make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from 5.x-stable or higher to 6.x-stable ----------------------------------------------------------- make buildworld [9] make kernel KERNCONF=YOUR_KERNEL_HERE [8] [1] [3] mergemaster -p [5] make installworld make delete-old mergemaster -i [4] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. [5] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. [8] In order to have a kernel that can run the 5.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD5 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. A similar kernel option COMPAT_FREEBSD5 is required to run the 5.x binaries on more recent kernels. Make sure that you merge any new devices from GENERIC since the last time you updated your kernel config file. [9] When checking out sources, you must include the -P flag to have cvs prune empty directories. If CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. MAKEOBJDIRPREFIX must be defined in an environment variable, and not on the command line, or in /etc/make.conf. buildworld will warn if it is improperly defined. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -STABLE. Not all things will be listed here, and it only starts on October 16, 2004. Updating files can found in previous releases if your system is older than this. Copyright information: Copyright 1998-2005 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. If you find this document useful, and you want to, you may buy the author a beer. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: releng/6.1/gnu/usr.bin/gzip/gzip.h =================================================================== --- releng/6.1/gnu/usr.bin/gzip/gzip.h (revision 162446) +++ releng/6.1/gnu/usr.bin/gzip/gzip.h (revision 162447) @@ -1,318 +1,320 @@ /* gzip.h -- common declarations for all gzip modules * Copyright (C) 1992-1993 Jean-loup Gailly. * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. * * $FreeBSD$ */ #if defined(__STDC__) || defined(PROTO) # define OF(args) args #else # define OF(args) () #endif #ifdef __STDC__ typedef void *voidp; #else typedef char *voidp; #endif /* I don't like nested includes, but the string and io functions are used * too often */ #include #if !defined(NO_STRING_H) || defined(STDC_HEADERS) # include # if !defined(STDC_HEADERS) && !defined(NO_MEMORY_H) && !defined(__GNUC__) # include # endif # define memzero(s, n) memset ((voidp)(s), 0, (n)) #else # include # define strchr index # define strrchr rindex # define memcpy(d, s, n) bcopy((s), (d), (n)) # define memcmp(s1, s2, n) bcmp((s1), (s2), (n)) # define memzero(s, n) bzero((s), (n)) #endif #ifndef RETSIGTYPE # define RETSIGTYPE void #endif #define local static #include typedef uint8_t uch; typedef uint16_t ush; typedef uint32_t ulg; /* Return codes from gzip */ #define OK 0 #define ERROR 1 #define WARNING 2 /* Compression methods (see algorithm.doc) */ #define STORED 0 #define COMPRESSED 1 #define PACKED 2 #define LZHED 3 /* methods 4 to 7 reserved */ #define DEFLATED 8 #define MAX_METHODS 9 extern int method; /* compression method */ /* To save memory for 16 bit systems, some arrays are overlaid between * the various modules: * deflate: prev+head window d_buf l_buf outbuf * unlzw: tab_prefix tab_suffix stack inbuf outbuf * inflate: window inbuf * unpack: window inbuf prefix_len * unlzh: left+right window c_table inbuf c_len * For compression, input is done in window[]. For decompression, output * is done in window except for unlzw. */ #ifndef INBUFSIZ # ifdef SMALL_MEM # define INBUFSIZ 0x2000 /* input buffer size */ # else # define INBUFSIZ 0x8000 /* input buffer size */ # endif #endif #define INBUF_EXTRA 64 /* required by unlzw() */ #ifndef OUTBUFSIZ # ifdef SMALL_MEM # define OUTBUFSIZ 8192 /* output buffer size */ # else # define OUTBUFSIZ 16384 /* output buffer size */ # endif #endif #define OUTBUF_EXTRA 2048 /* required by unlzw() */ #ifndef DIST_BUFSIZE # ifdef SMALL_MEM # define DIST_BUFSIZE 0x2000 /* buffer for distances, see trees.c */ # else # define DIST_BUFSIZE 0x8000 /* buffer for distances, see trees.c */ # endif #endif #ifdef DYN_ALLOC # define EXTERN(type, array) extern type * near array # define DECLARE(type, array, size) type * near array # define ALLOC(type, array, size) { \ array = (type*)fcalloc((size_t)(((size)+1L)/2), 2*sizeof(type)); \ if (array == NULL) error("insufficient memory"); \ } # define FREE(array) {if (array != NULL) fcfree(array), array=NULL;} #else # define EXTERN(type, array) extern type array[] # define DECLARE(type, array, size) type array[size] # define ALLOC(type, array, size) # define FREE(array) #endif EXTERN(uch, inbuf); /* input buffer */ EXTERN(uch, outbuf); /* output buffer */ EXTERN(ush, d_buf); /* buffer for distances, see trees.c */ EXTERN(uch, window); /* Sliding window and suffix table (unlzw) */ #define tab_suffix window #ifndef MAXSEG_64K # define tab_prefix prev /* hash link (see deflate.c) */ # define head (prev+WSIZE) /* hash head (see deflate.c) */ EXTERN(ush, tab_prefix); /* prefix code (see unlzw.c) */ #else # define tab_prefix0 prev # define head tab_prefix1 EXTERN(ush, tab_prefix0); /* prefix for even codes */ EXTERN(ush, tab_prefix1); /* prefix for odd codes */ #endif extern unsigned insize; /* valid bytes in inbuf */ extern unsigned inptr; /* index of next byte to be processed in inbuf */ extern unsigned outcnt; /* bytes in output buffer */ extern long bytes_in; /* number of input bytes */ extern long bytes_out; /* number of output bytes */ extern long header_bytes;/* number of bytes in gzip header */ #define isize bytes_in /* for compatibility with old zip sources (to be cleaned) */ extern int ifd; /* input file descriptor */ extern int ofd; /* output file descriptor */ extern char ifname[]; /* input file name or "stdin" */ extern char ofname[]; /* output file name or "stdout" */ extern char *progname; /* program name */ extern long time_stamp; /* original time stamp (modification time) */ extern long ifile_size; /* input file size, -1 for devices (debug only) */ typedef int file_t; /* Do not use stdio */ #define NO_FILE (-1) /* in memory compression */ #define PACK_MAGIC "\037\036" /* Magic header for packed files */ #define GZIP_MAGIC "\037\213" /* Magic header for gzip files, 1F 8B */ #define OLD_GZIP_MAGIC "\037\236" /* Magic header for gzip 0.5 = freeze 1.x */ #define LZH_MAGIC "\037\240" /* Magic header for SCO LZH Compress files*/ #define PKZIP_MAGIC "\120\113\003\004" /* Magic header for pkzip files */ /* gzip flag byte */ #define ASCII_FLAG 0x01 /* bit 0 set: file probably ascii text */ #define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */ #define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */ #define ORIG_NAME 0x08 /* bit 3 set: original file name present */ #define COMMENT 0x10 /* bit 4 set: file comment present */ #define ENCRYPTED 0x20 /* bit 5 set: file is encrypted */ #define RESERVED 0xC0 /* bit 6,7: reserved */ /* internal file attribute */ #define UNKNOWN 0xffff #define BINARY 0 #define ASCII 1 #ifndef WSIZE # define WSIZE 0x8000 /* window size--must be a power of two, and */ #endif /* at least 32K for zip's deflate method */ #define MIN_MATCH 3 #define MAX_MATCH 258 /* The minimum and maximum match lengths */ #define MIN_LOOKAHEAD (MAX_MATCH+MIN_MATCH+1) /* Minimum amount of lookahead, except at the end of the input file. * See deflate.c for comments about the MIN_MATCH+1. */ #define MAX_DIST (WSIZE-MIN_LOOKAHEAD) /* In order to simplify the code, particularly on 16 bit machines, match * distances are limited to MAX_DIST instead of WSIZE. */ extern int decrypt; /* flag to turn on decryption */ extern int exit_code; /* program exit code */ extern int verbose; /* be verbose (-v) */ extern int quiet; /* be quiet (-q) */ extern int level; /* compression level */ extern int test; /* check .z file integrity */ extern int to_stdout; /* output to stdout (-c) */ extern int save_orig_name; /* set if original name must be saved */ +#define MIN(a,b) ((a) <= (b) ? (a) : (b)) + #define get_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(0)) #define try_byte() (inptr < insize ? inbuf[inptr++] : fill_inbuf(1)) /* put_byte is used for the compressed output, put_ubyte for the * uncompressed output. However unlzw() uses window for its * suffix table instead of its output buffer, so it does not use put_ubyte * (to be cleaned up). */ #define put_byte(c) {outbuf[outcnt++]=(uch)(c); if (outcnt==OUTBUFSIZ)\ flush_outbuf();} #define put_ubyte(c) {window[outcnt++]=(uch)(c); if (outcnt==WSIZE)\ flush_window();} /* Output a 16 bit value, lsb first */ #define put_short(w) \ { if (outcnt < OUTBUFSIZ-2) { \ outbuf[outcnt++] = (uch) ((w) & 0xff); \ outbuf[outcnt++] = (uch) ((ush)(w) >> 8); \ } else { \ put_byte((uch)((w) & 0xff)); \ put_byte((uch)((ush)(w) >> 8)); \ } \ } /* Output a 32 bit value to the bit stream, lsb first */ #define put_long(n) { \ put_short((n) & 0xffff); \ put_short(((ulg)(n)) >> 16); \ } #define seekable() 0 /* force sequential output */ #define translate_eol 0 /* no option -a yet */ #define tolow(c) (isupper(c) ? (c)-'A'+'a' : (c)) /* force to lower case */ /* Macros for getting two-byte and four-byte header values */ #define SH(p) ((ush)(uch)((p)[0]) | ((ush)(uch)((p)[1]) << 8)) #define LG(p) ((ulg)(SH(p)) | ((ulg)(SH((p)+2)) << 16)) /* Diagnostic functions */ #ifdef DEBUG # define Assert(cond,msg) {if(!(cond)) error(msg);} # define Trace(x) fprintf x # define Tracev(x) {if (verbose) fprintf x ;} # define Tracevv(x) {if (verbose>1) fprintf x ;} # define Tracec(c,x) {if (verbose && (c)) fprintf x ;} # define Tracecv(c,x) {if (verbose>1 && (c)) fprintf x ;} #else # define Assert(cond,msg) # define Trace(x) # define Tracev(x) # define Tracevv(x) # define Tracec(c,x) # define Tracecv(c,x) #endif #define WARN(msg) {if (!quiet) fprintf msg ; \ if (exit_code == OK) exit_code = WARNING;} /* in zip.c: */ extern int zip OF((int in, int out)); extern int file_read OF((char *buf, unsigned size)); /* in unzip.c */ extern int unzip OF((int in, int out)); extern int check_zipfile OF((int in)); /* in unpack.c */ extern int unpack OF((int in, int out)); /* in unlzh.c */ extern int unlzh OF((int in, int out)); /* in gzip.c */ RETSIGTYPE abort_gzip OF((void)); /* in deflate.c */ void lm_init OF((int pack_level, ush *flags)); ulg deflate OF((void)); /* in trees.c */ void ct_init OF((ush *attr, int *method)); int ct_tally OF((int dist, int lc)); ulg flush_block OF((char *buf, ulg stored_len, int eof)); /* in bits.c */ void bi_init OF((file_t zipfile)); void send_bits OF((int value, int length)); unsigned bi_reverse OF((unsigned value, int length)); void bi_windup OF((void)); void copy_block OF((char *buf, unsigned len, int header)); extern int (*read_buf) OF((char *buf, unsigned size)); /* in util.c: */ extern int copy OF((int in, int out)); extern ulg updcrc OF((uch *s, unsigned n)); extern void clear_bufs OF((void)); extern int fill_inbuf OF((int eof_ok)); extern void flush_outbuf OF((void)); extern void flush_window OF((void)); extern void write_buf OF((int fd, voidp buf, unsigned cnt)); extern char *strlwr OF((char *s)); extern char *basename OF((char *fname)); extern void make_simple_name OF((char *name)); extern char *add_envopt OF((int *argcp, char ***argvp, char *env)); extern void error OF((char *m)); extern void warn OF((char *a, char *b)); extern void read_error OF((void)); extern void write_error OF((void)); extern void display_ratio OF((long num, long den, FILE *file)); extern voidp xmalloc OF((unsigned int size)); /* in inflate.c */ extern int inflate OF((void)); Index: releng/6.1/gnu/usr.bin/gzip/inflate.c =================================================================== --- releng/6.1/gnu/usr.bin/gzip/inflate.c (revision 162446) +++ releng/6.1/gnu/usr.bin/gzip/inflate.c (revision 162447) @@ -1,958 +1,958 @@ /* inflate.c -- Not copyrighted 1992 by Mark Adler version c10p1, 10 January 1993 */ /* You can do whatever you like with this source file, though I would prefer that if you modify it and redistribute it that you include comments to that effect with your name and the date. Thank you. [The history has been moved to the file ChangeLog.] */ /* Inflate deflated (PKZIP's method 8 compressed) data. The compression method searches for as much of the current string of bytes (up to a length of 258) in the previous 32K bytes. If it doesn't find any matches (of at least length 3), it codes the next byte. Otherwise, it codes the length of the matched string and its distance backwards from the current position. There is a single Huffman code that codes both single bytes (called "literals") and match lengths. A second Huffman code codes the distance information, which follows a length code. Each length or distance code actually represents a base value and a number of "extra" (sometimes zero) bits to get to add to the base value. At the end of each deflated block is a special end-of-block (EOB) literal/ length code. The decoding process is basically: get a literal/length code; if EOB then done; if a literal, emit the decoded byte; if a length then get the distance and emit the referred-to bytes from the sliding window of previously emitted data. There are (currently) three kinds of inflate blocks: stored, fixed, and dynamic. The compressor deals with some chunk of data at a time, and decides which method to use on a chunk-by-chunk basis. A chunk might typically be 32K or 64K. If the chunk is uncompressible, then the "stored" method is used. In this case, the bytes are simply stored as is, eight bits per byte, with none of the above coding. The bytes are preceded by a count, since there is no longer an EOB code. If the data is compressible, then either the fixed or dynamic methods are used. In the dynamic method, the compressed data is preceded by an encoding of the literal/length and distance Huffman codes that are to be used to decode this block. The representation is itself Huffman coded, and so is preceded by a description of that code. These code descriptions take up a little space, and so for small blocks, there is a predefined set of codes, called the fixed codes. The fixed method is used if the block codes up smaller that way (usually for quite small chunks), otherwise the dynamic method is used. In the latter case, the codes are customized to the probabilities in the current block, and so can code it much better than the pre-determined fixed codes. The Huffman codes themselves are decoded using a mutli-level table lookup, in order to maximize the speed of decoding plus the speed of building the decoding tables. See the comments below that precede the lbits and dbits tuning parameters. */ /* Notes beyond the 1.93a appnote.txt: 1. Distance pointers never point before the beginning of the output stream. 2. Distance pointers can point back across blocks, up to 32k away. 3. There is an implied maximum of 7 bits for the bit length table and 15 bits for the actual data. 4. If only one code exists, then it is encoded using one bit. (Zero would be more efficient, but perhaps a little confusing.) If two codes exist, they are coded using one bit each (0 and 1). 5. There is no way of sending zero distance codes--a dummy must be sent if there are none. (History: a pre 2.0 version of PKZIP would store blocks with no distance codes, but this was discovered to be too harsh a criterion.) Valid only for 1.93a. 2.04c does allow zero distance codes, which is sent as one code of zero bits in length. 6. There are up to 286 literal/length codes. Code 256 represents the end-of-block. Note however that the static length tree defines 288 codes just to fill out the Huffman codes. Codes 286 and 287 cannot be used though, since there is no length base or extra bits defined for them. Similarly, there are up to 30 distance codes. However, static trees define 32 codes (all 5 bits) to fill out the Huffman codes, but the last two had better not show up in the data. 7. Unzip can check dynamic Huffman blocks for complete code sets. The exception is that a single code would not be complete (see #4). 8. The five bits following the block type is really the number of literal codes sent minus 257. 9. Length codes 8,16,16 are interpreted as 13 length codes of 8 bits (1+6+6). Therefore, to output three times the length, you output three codes (1+1+1), whereas to output four times the same length, you only need two codes (1+3). Hmm. 10. In the tree reconstruction algorithm, Code = Code + Increment only if BitLength(i) is not zero. (Pretty obvious.) 11. Correction: 4 Bits: # of Bit Length codes - 4 (4 - 19) 12. Note: length code 284 can represent 227-258, but length code 285 really is 258. The last length deserves its own, short code since it gets used a lot in very redundant files. The length 258 is special since 258 - 3 (the min match length) is 255. 13. The literal/length and distance code bit lengths are read as a single stream of lengths. It is possible (and advantageous) for a repeat code (16, 17, or 18) to go across the boundary between the two sets of lengths. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #if defined(STDC_HEADERS) || !defined(NO_STDLIB_H) # include #endif #include "gzip.h" #define slide window /* Huffman code lookup table entry--this entry is four bytes for machines that have 16-bit pointers (e.g. PC's in the small or medium model). Valid extra bits are 0..13. e == 15 is EOB (end of block), e == 16 means that v is a literal, 16 < e < 32 means that v is a pointer to the next table, which codes e - 16 bits, and lastly e == 99 indicates an unused code. If a code with e == 99 is looked up, this implies an error in the data. */ struct huft { uch e; /* number of extra bits or operation */ uch b; /* number of bits in this code or subcode */ union { ush n; /* literal, length base, or distance base */ struct huft *t; /* pointer to next level of table */ } v; }; /* Function prototypes */ int huft_build OF((unsigned *, unsigned, unsigned, ush *, ush *, struct huft **, int *)); int huft_free OF((struct huft *)); int inflate_codes OF((struct huft *, struct huft *, int, int)); int inflate_stored OF((void)); int inflate_fixed OF((void)); int inflate_dynamic OF((void)); int inflate_block OF((int *)); int inflate OF((void)); /* The inflate algorithm uses a sliding 32K byte window on the uncompressed stream to find repeated byte strings. This is implemented here as a circular buffer. The index is updated simply by incrementing and then and'ing with 0x7fff (32K-1). */ /* It is left to other modules to supply the 32K area. It is assumed to be usable as if it were declared "uch slide[32768];" or as just "uch *slide;" and then malloc'ed in the latter case. The definition must be in unzip.h, included above. */ /* unsigned wp; current position in slide */ #define wp outcnt #define flush_output(w) (wp=(w),flush_window()) /* Tables for deflate from PKZIP's appnote.txt. */ static unsigned border[] = { /* Order of the bit length code lengths */ 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15}; static ush cplens[] = { /* Copy lengths for literal codes 257..285 */ 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0}; /* note: see note #13 above about the 258 in this list. */ static ush cplext[] = { /* Extra bits for literal codes 257..285 */ 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0, 99, 99}; /* 99==invalid */ static ush cpdist[] = { /* Copy offsets for distance codes 0..29 */ 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577}; static ush cpdext[] = { /* Extra bits for distance codes */ 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13}; /* Macros for inflate() bit peeking and grabbing. The usage is: NEEDBITS(j) x = b & mask_bits[j]; DUMPBITS(j) where NEEDBITS makes sure that b has at least j bits in it, and DUMPBITS removes the bits from b. The macros use the variable k for the number of bits in b. Normally, b and k are register variables for speed, and are initialized at the beginning of a routine that uses these macros from a global bit buffer and count. If we assume that EOB will be the longest code, then we will never ask for bits with NEEDBITS that are beyond the end of the stream. So, NEEDBITS should not read any more bytes than are needed to meet the request. Then no bytes need to be "returned" to the buffer at the end of the last block. However, this assumption is not true for fixed blocks--the EOB code is 7 bits, but the other literal/length codes can be 8 or 9 bits. (The EOB code is shorter than other codes because fixed blocks are generally short. So, while a block always has an EOB, many other literal/length codes have a significantly lower probability of showing up at all.) However, by making the first table have a lookup of seven bits, the EOB code will be found in that first lookup, and so will not require that too many bits be pulled from the stream. */ ulg bb; /* bit buffer */ unsigned bk; /* bits in bit buffer */ ush mask_bits[] = { 0x0000, 0x0001, 0x0003, 0x0007, 0x000f, 0x001f, 0x003f, 0x007f, 0x00ff, 0x01ff, 0x03ff, 0x07ff, 0x0fff, 0x1fff, 0x3fff, 0x7fff, 0xffff }; #ifdef CRYPT uch cc; # define NEXTBYTE() \ (decrypt ? (cc = get_byte(), zdecode(cc), cc) : get_byte()) #else # define NEXTBYTE() (uch)get_byte() #endif #define NEEDBITS(n) {while(k<(n)){b|=((ulg)NEXTBYTE())<>=(n);k-=(n);} /* Huffman code decoding is performed using a multi-level table lookup. The fastest way to decode is to simply build a lookup table whose size is determined by the longest code. However, the time it takes to build this table can also be a factor if the data being decoded is not very long. The most common codes are necessarily the shortest codes, so those codes dominate the decoding time, and hence the speed. The idea is you can have a shorter table that decodes the shorter, more probable codes, and then point to subsidiary tables for the longer codes. The time it costs to decode the longer codes is then traded against the time it takes to make longer tables. This results of this trade are in the variables lbits and dbits below. lbits is the number of bits the first level table for literal/ length codes can decode in one step, and dbits is the same thing for the distance codes. Subsequent tables are also less than or equal to those sizes. These values may be adjusted either when all of the codes are shorter than that, in which case the longest code length in bits is used, or when the shortest code is *longer* than the requested table size, in which case the length of the shortest code in bits is used. There are two different values for the two tables, since they code a different number of possibilities each. The literal/length table codes 286 possible values, or in a flat code, a little over eight bits. The distance table codes 30 possible values, or a little less than five bits, flat. The optimum values for speed end up being about one bit more than those, so lbits is 8+1 and dbits is 5+1. The optimum values may differ though from machine to machine, and possibly even between compilers. Your mileage may vary. */ int lbits = 9; /* bits in base literal/length lookup table */ int dbits = 6; /* bits in base distance lookup table */ /* If BMAX needs to be larger than 16, then h and x[] should be ulg. */ #define BMAX 16 /* maximum bit length of any code (16 for explode) */ #define N_MAX 288 /* maximum number of codes in any set */ unsigned hufts; /* track memory usage */ int huft_build(b, n, s, d, e, t, m) unsigned *b; /* code lengths in bits (all assumed <= BMAX) */ unsigned n; /* number of codes (assumed <= N_MAX) */ unsigned s; /* number of simple-valued codes (0..s-1) */ ush *d; /* list of base values for non-simple codes */ ush *e; /* list of extra bits for non-simple codes */ struct huft **t; /* result: starting table */ int *m; /* maximum lookup bits, returns actual */ /* Given a list of code lengths and a maximum table size, make a set of tables to decode that set of codes. Return zero on success, one if the given code set is incomplete (the tables are still built in this case), two if the input is invalid (all zero length codes or an oversubscribed set of lengths), and three if not enough memory. */ { unsigned a; /* counter for codes of length k */ unsigned c[BMAX+1]; /* bit length count table */ unsigned f; /* i repeats in table every f entries */ int g; /* maximum code length */ int h; /* table level */ register unsigned i; /* counter, current code */ register unsigned j; /* counter */ register int k; /* number of bits in current code */ int l; /* bits per table (returned in m) */ register unsigned *p; /* pointer into c[], b[], or v[] */ register struct huft *q; /* points to current table */ struct huft r; /* table entry for structure assignment */ struct huft *u[BMAX]; /* table stack */ unsigned v[N_MAX]; /* values in order of bit length */ register int w; /* bits before this table == (l * h) */ unsigned x[BMAX+1]; /* bit offsets, then code stack */ unsigned *xp; /* pointer into x */ int y; /* number of dummy codes added */ unsigned z; /* number of entries in current table */ /* Generate counts for each bit length */ memzero(c, sizeof(c)); p = b; i = n; do { Tracecv(*p, (stderr, (n-i >= ' ' && n-i <= '~' ? "%c %d\n" : "0x%x %d\n"), n-i, *p)); c[*p]++; /* assume all entries <= BMAX */ p++; /* Can't combine with above line (Solaris bug) */ } while (--i); if (c[0] == n) /* null input--all zero length codes */ { *t = (struct huft *)NULL; *m = 0; - return 0; + return 2; } /* Find minimum and maximum length, bound *m by those */ l = *m; for (j = 1; j <= BMAX; j++) if (c[j]) break; k = j; /* minimum code length */ if ((unsigned)l < j) l = j; for (i = BMAX; i; i--) if (c[i]) break; g = i; /* maximum code length */ if ((unsigned)l > i) l = i; *m = l; /* Adjust last length count to fill out codes, if needed */ for (y = 1 << j; j < i; j++, y <<= 1) if ((y -= c[j]) < 0) return 2; /* bad input: more codes than bits */ if ((y -= c[i]) < 0) return 2; c[i] += y; /* Generate starting offsets into the value table for each length */ x[1] = j = 0; p = c + 1; xp = x + 2; while (--i) { /* note that i == g from above */ *xp++ = (j += *p++); } /* Make a table of values in order of bit lengths */ p = b; i = 0; do { if ((j = *p++) != 0) v[x[j]++] = i; } while (++i < n); n = x[g]; /* set n to length of v */ /* Generate the Huffman codes and for each, make the table entries */ x[0] = i = 0; /* first Huffman code is zero */ p = v; /* grab values in bit order */ h = -1; /* no tables yet--level -1 */ w = -l; /* bits decoded == (l * h) */ u[0] = (struct huft *)NULL; /* just to keep compilers happy */ q = (struct huft *)NULL; /* ditto */ z = 0; /* ditto */ /* go through the bit lengths (k already is bits in shortest code) */ for (; k <= g; k++) { a = c[k]; while (a--) { /* here i is the Huffman code of length k bits for value *p */ /* make tables up to required level */ while (k > w + l) { h++; w += l; /* previous table always l bits */ /* compute minimum size table less than or equal to l bits */ z = (z = g - w) > (unsigned)l ? l : z; /* upper limit on table size */ if ((f = 1 << (j = k - w)) > a + 1) /* try a k-w bit table */ { /* too few codes for k-w bit table */ f -= a + 1; /* deduct codes from patterns left */ xp = c + k; if (j < z) while (++j < z) /* try smaller tables up to z bits */ { if ((f <<= 1) <= *++xp) break; /* enough codes to use up j bits */ f -= *xp; /* else deduct codes from patterns */ } } z = 1 << j; /* table entries for j-bit table */ /* allocate and link in new table */ if ((q = (struct huft *)malloc((z + 1)*sizeof(struct huft))) == (struct huft *)NULL) { if (h) huft_free(u[0]); return 3; /* not enough memory */ } hufts += z + 1; /* track memory usage */ *t = q + 1; /* link to list for huft_free() */ *(t = &(q->v.t)) = (struct huft *)NULL; u[h] = ++q; /* table starts after link */ /* connect to last table, if there is one */ if (h) { x[h] = i; /* save pattern for backing up */ r.b = (uch)l; /* bits to dump before this table */ r.e = (uch)(16 + j); /* bits in this table */ r.v.t = q; /* pointer to this table */ j = i >> (w - l); /* (get around Turbo C bug) */ u[h-1][j] = r; /* connect to last table */ } } /* set up table entry in r */ r.b = (uch)(k - w); if (p >= v + n) r.e = 99; /* out of values--invalid code */ else if (*p < s) { r.e = (uch)(*p < 256 ? 16 : 15); /* 256 is end-of-block code */ r.v.n = (ush)(*p); /* simple code is just the value */ p++; /* one compiler does not like *p++ */ } else { r.e = (uch)e[*p - s]; /* non-simple--look up in lists */ r.v.n = d[*p++ - s]; } /* fill code-like entries with r */ f = 1 << (k - w); for (j = i >> w; j < z; j += f) q[j] = r; /* backwards increment the k-bit code i */ for (j = 1 << (k - 1); i & j; j >>= 1) i ^= j; i ^= j; /* backup over finished tables */ while ((i & ((1 << w) - 1)) != x[h]) { h--; /* don't need to update q */ w -= l; } } } /* Return true (1) if we were given an incomplete table */ return y != 0 && g != 1; } int huft_free(t) struct huft *t; /* table to free */ /* Free the malloc'ed tables built by huft_build(), which makes a linked list of the tables it made, with the links in a dummy first entry of each table. */ { register struct huft *p, *q; /* Go through linked list, freeing from the malloced (t[-1]) address. */ p = t; while (p != (struct huft *)NULL) { q = (--p)->v.t; free((char*)p); p = q; } return 0; } int inflate_codes(tl, td, bl, bd) struct huft *tl, *td; /* literal/length and distance decoder tables */ int bl, bd; /* number of bits decoded by tl[] and td[] */ /* inflate (decompress) the codes in a deflated (compressed) block. Return an error code or zero if it all goes ok. */ { register unsigned e; /* table entry flag/number of extra bits */ unsigned n, d; /* length and index for copy */ unsigned w; /* current window position */ struct huft *t; /* pointer to table entry */ unsigned ml, md; /* masks for bl and bd bits */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* inflate the coded data */ ml = mask_bits[bl]; /* precompute masks for speed */ md = mask_bits[bd]; for (;;) /* do until end of block */ { NEEDBITS((unsigned)bl) if ((e = (t = tl + ((unsigned)b & ml))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) if (e == 16) /* then it's a literal */ { slide[w++] = (uch)t->v.n; Tracevv((stderr, "%c", slide[w-1])); if (w == WSIZE) { flush_output(w); w = 0; } } else /* it's an EOB or a length */ { /* exit if end of block */ if (e == 15) break; /* get length of block to copy */ NEEDBITS(e) n = t->v.n + ((unsigned)b & mask_bits[e]); DUMPBITS(e); /* decode distance of block to copy */ NEEDBITS((unsigned)bd) if ((e = (t = td + ((unsigned)b & md))->e) > 16) do { if (e == 99) return 1; DUMPBITS(t->b) e -= 16; NEEDBITS(e) } while ((e = (t = t->v.t + ((unsigned)b & mask_bits[e]))->e) > 16); DUMPBITS(t->b) NEEDBITS(e) d = w - t->v.n - ((unsigned)b & mask_bits[e]); DUMPBITS(e) Tracevv((stderr,"\\[%d,%d]", w-d, n)); /* do the copy */ do { n -= (e = (e = WSIZE - ((d &= WSIZE-1) > w ? d : w)) > n ? n : e); #if !defined(NOMEMCPY) && !defined(DEBUG) if (w - d >= e) /* (this test assumes unsigned comparison) */ { memcpy(slide + w, slide + d, e); w += e; d += e; } else /* do it slow to avoid memcpy() overlap */ #endif /* !NOMEMCPY */ do { slide[w++] = slide[d++]; Tracevv((stderr, "%c", slide[w-1])); } while (--e); if (w == WSIZE) { flush_output(w); w = 0; } } while (n); } } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; /* done */ return 0; } int inflate_stored() /* "decompress" an inflated type 0 (stored) block. */ { unsigned n; /* number of bytes in block */ unsigned w; /* current window position */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local copies of globals */ b = bb; /* initialize bit buffer */ k = bk; w = wp; /* initialize window position */ /* go to byte boundary */ n = k & 7; DUMPBITS(n); /* get the length and its complement */ NEEDBITS(16) n = ((unsigned)b & 0xffff); DUMPBITS(16) NEEDBITS(16) if (n != (unsigned)((~b) & 0xffff)) return 1; /* error in compressed data */ DUMPBITS(16) /* read and output the compressed data */ while (n--) { NEEDBITS(8) slide[w++] = (uch)b; if (w == WSIZE) { flush_output(w); w = 0; } DUMPBITS(8) } /* restore the globals from the locals */ wp = w; /* restore global window pointer */ bb = b; /* restore global bit buffer */ bk = k; return 0; } int inflate_fixed() /* decompress an inflated type 1 (fixed Huffman codes) block. We should either replace this with a custom decoder, or at least precompute the Huffman tables. */ { int i; /* temporary variable */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned l[288]; /* length list for huft_build */ /* set up literal table */ for (i = 0; i < 144; i++) l[i] = 8; for (; i < 256; i++) l[i] = 9; for (; i < 280; i++) l[i] = 7; for (; i < 288; i++) /* make a complete, but wrong code set */ l[i] = 8; bl = 7; if ((i = huft_build(l, 288, 257, cplens, cplext, &tl, &bl)) != 0) return i; /* set up distance table */ for (i = 0; i < 30; i++) /* make an incomplete code set */ l[i] = 5; bd = 5; if ((i = huft_build(l, 30, 0, cpdist, cpdext, &td, &bd)) > 1) { huft_free(tl); return i; } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_dynamic() /* decompress an inflated type 2 (dynamic Huffman codes) block. */ { int i; /* temporary variables */ unsigned j; unsigned l; /* last length */ unsigned m; /* mask for bit lengths table */ unsigned n; /* number of lengths to get */ struct huft *tl; /* literal/length code table */ struct huft *td; /* distance code table */ int bl; /* lookup bits for tl */ int bd; /* lookup bits for td */ unsigned nb; /* number of bit length codes */ unsigned nl; /* number of literal/length codes */ unsigned nd; /* number of distance codes */ #ifdef PKZIP_BUG_WORKAROUND unsigned ll[288+32]; /* literal/length and distance code lengths */ #else unsigned ll[286+30]; /* literal/length and distance code lengths */ #endif register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in table lengths */ NEEDBITS(5) nl = 257 + ((unsigned)b & 0x1f); /* number of literal/length codes */ DUMPBITS(5) NEEDBITS(5) nd = 1 + ((unsigned)b & 0x1f); /* number of distance codes */ DUMPBITS(5) NEEDBITS(4) nb = 4 + ((unsigned)b & 0xf); /* number of bit length codes */ DUMPBITS(4) #ifdef PKZIP_BUG_WORKAROUND if (nl > 288 || nd > 32) #else if (nl > 286 || nd > 30) #endif return 1; /* bad lengths */ /* read in bit-length-code lengths */ for (j = 0; j < nb; j++) { NEEDBITS(3) ll[border[j]] = (unsigned)b & 7; DUMPBITS(3) } for (; j < 19; j++) ll[border[j]] = 0; /* build decoding table for trees--single level, 7 bit lookup */ bl = 7; if ((i = huft_build(ll, 19, 19, NULL, NULL, &tl, &bl)) != 0) { if (i == 1) huft_free(tl); return i; /* incomplete code set */ } if (tl == NULL) /* Grrrhhh */ return 2; /* read in literal and distance code lengths */ n = nl + nd; m = mask_bits[bl]; i = l = 0; while ((unsigned)i < n) { NEEDBITS((unsigned)bl) j = (td = tl + ((unsigned)b & m))->b; DUMPBITS(j) j = td->v.n; if (j < 16) /* length of code in bits (0..15) */ ll[i++] = l = j; /* save last length in l */ else if (j == 16) /* repeat last length 3 to 6 times */ { NEEDBITS(2) j = 3 + ((unsigned)b & 3); DUMPBITS(2) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = l; } else if (j == 17) /* 3 to 10 zero length codes */ { NEEDBITS(3) j = 3 + ((unsigned)b & 7); DUMPBITS(3) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } else /* j == 18: 11 to 138 zero length codes */ { NEEDBITS(7) j = 11 + ((unsigned)b & 0x7f); DUMPBITS(7) if ((unsigned)i + j > n) return 1; while (j--) ll[i++] = 0; l = 0; } } /* free decoding table for trees */ huft_free(tl); /* restore the global bit buffer */ bb = b; bk = k; /* build the decoding tables for literal/length and distance codes */ bl = lbits; if ((i = huft_build(ll, nl, 257, cplens, cplext, &tl, &bl)) != 0) { if (i == 1) { fprintf(stderr, " incomplete literal tree\n"); huft_free(tl); } return i; /* incomplete code set */ } bd = dbits; if ((i = huft_build(ll + nl, nd, 0, cpdist, cpdext, &td, &bd)) != 0) { if (i == 1) { fprintf(stderr, " incomplete distance tree\n"); #ifdef PKZIP_BUG_WORKAROUND i = 0; } #else huft_free(td); } huft_free(tl); return i; /* incomplete code set */ #endif } /* decompress until an end-of-block code */ if (inflate_codes(tl, td, bl, bd)) return 1; /* free the decoding tables, return */ huft_free(tl); huft_free(td); return 0; } int inflate_block(e) int *e; /* last block flag */ /* decompress an inflated block */ { unsigned t; /* block type */ register ulg b; /* bit buffer */ register unsigned k; /* number of bits in bit buffer */ /* make local bit buffer */ b = bb; k = bk; /* read in last block bit */ NEEDBITS(1) *e = (int)b & 1; DUMPBITS(1) /* read in block type */ NEEDBITS(2) t = (unsigned)b & 3; DUMPBITS(2) /* restore the global bit buffer */ bb = b; bk = k; /* inflate that block type */ if (t == 2) return inflate_dynamic(); if (t == 0) return inflate_stored(); if (t == 1) return inflate_fixed(); /* bad block type */ return 2; } int inflate() /* decompress an inflated entry */ { int e; /* last block flag */ int r; /* result code */ unsigned h; /* maximum struct huft's malloc'ed */ /* initialize window, bit buffer */ wp = 0; bk = 0; bb = 0; /* decompress until the last block */ h = 0; do { hufts = 0; if ((r = inflate_block(&e)) != 0) return r; if (hufts > h) h = hufts; } while (!e); /* Undo too much lookahead. The next read will be byte aligned so we * can discard unused bits in the last meaningful byte. */ while (bk >= 8) { bk -= 8; inptr--; } /* flush out slide */ flush_output(wp); /* return success */ #ifdef DEBUG fprintf(stderr, "<%u> ", h); #endif /* DEBUG */ return 0; } Index: releng/6.1/gnu/usr.bin/gzip/unlzh.c =================================================================== --- releng/6.1/gnu/usr.bin/gzip/unlzh.c (revision 162446) +++ releng/6.1/gnu/usr.bin/gzip/unlzh.c (revision 162447) @@ -1,401 +1,405 @@ /* unlzh.c -- decompress files in SCO compress -H (LZH) format. * The code in this file is directly derived from the public domain 'ar002' * written by Haruhiko Okumura. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include #include "tailor.h" #include "gzip.h" #include "lzw.h" /* just for consistency checking */ /* decode.c */ local unsigned decode OF((unsigned count, uch buffer[])); local void decode_start OF((void)); /* huf.c */ local void huf_decode_start OF((void)); local unsigned decode_c OF((void)); local unsigned decode_p OF((void)); local void read_pt_len OF((int nn, int nbit, int i_special)); local void read_c_len OF((void)); /* io.c */ local void fillbuf OF((int n)); local unsigned getbits OF((int n)); local void init_getbits OF((void)); /* maketbl.c */ local void make_table OF((int nchar, uch bitlen[], int tablebits, ush table[])); #define DICBIT 13 /* 12(-lh4-) or 13(-lh5-) */ #define DICSIZ ((unsigned) 1 << DICBIT) #ifndef CHAR_BIT # define CHAR_BIT 8 #endif #ifndef UCHAR_MAX # define UCHAR_MAX 255 #endif #define BITBUFSIZ (CHAR_BIT * 2 * sizeof(char)) /* Do not use CHAR_BIT * sizeof(bitbuf), does not work on machines * for which short is not on 16 bits (Cray). */ /* encode.c and decode.c */ #define MAXMATCH 256 /* formerly F (not more than UCHAR_MAX + 1) */ #define THRESHOLD 3 /* choose optimal value */ /* huf.c */ #define NC (UCHAR_MAX + MAXMATCH + 2 - THRESHOLD) /* alphabet = {0, 1, 2, ..., NC - 1} */ #define CBIT 9 /* $\lfloor \log_2 NC \rfloor + 1$ */ #define CODE_BIT 16 /* codeword length */ #define NP (DICBIT + 1) #define NT (CODE_BIT + 3) #define PBIT 4 /* smallest integer such that (1U << PBIT) > NP */ #define TBIT 5 /* smallest integer such that (1U << TBIT) > NT */ #if NT > NP # define NPT NT #else # define NPT NP #endif /* local ush left[2 * NC - 1]; */ /* local ush right[2 * NC - 1]; */ #define left prev #define right head #if NC > (1<<(BITS-2)) error cannot overlay left+right and prev #endif /* local uch c_len[NC]; */ #define c_len outbuf #if NC > OUTBUFSIZ error cannot overlay c_len and outbuf #endif local uch pt_len[NPT]; local unsigned blocksize; local ush pt_table[256]; /* local ush c_table[4096]; */ #define c_table d_buf #if (DIST_BUFSIZE-1) < 4095 error cannot overlay c_table and d_buf #endif /*********************************************************** io.c -- input/output ***********************************************************/ local ush bitbuf; local unsigned subbitbuf; local int bitcount; local void fillbuf(n) /* Shift bitbuf n bits left, read n bits */ int n; { bitbuf <<= n; while (n > bitcount) { bitbuf |= subbitbuf << (n -= bitcount); subbitbuf = (unsigned)try_byte(); if ((int)subbitbuf == EOF) subbitbuf = 0; bitcount = CHAR_BIT; } bitbuf |= subbitbuf >> (bitcount -= n); } local unsigned getbits(n) int n; { unsigned x; x = bitbuf >> (BITBUFSIZ - n); fillbuf(n); return x; } local void init_getbits() { bitbuf = 0; subbitbuf = 0; bitcount = 0; fillbuf(BITBUFSIZ); } /*********************************************************** maketbl.c -- make table for decoding ***********************************************************/ local void make_table(nchar, bitlen, tablebits, table) int nchar; uch bitlen[]; int tablebits; ush table[]; { ush count[17], weight[17], start[18], *p; unsigned i, k, len, ch, jutbits, avail, nextcode, mask; for (i = 1; i <= 16; i++) count[i] = 0; - for (i = 0; i < (unsigned)nchar; i++) count[bitlen[i]]++; + for (i = 0; i < (unsigned)nchar; i++) { + if (bitlen[i] > 16) + error("Bad table (case a)\n"); + else count[bitlen[i]]++; + } start[1] = 0; for (i = 1; i <= 16; i++) start[i + 1] = start[i] + (count[i] << (16 - i)); - if ((start[17] & 0xffff) != 0) - error("Bad table\n"); + if ((start[17] & 0xffff) != 0 || tablebits > 16) /* 16 for weight below */ + error("Bad table (case b)\n"); jutbits = 16 - tablebits; for (i = 1; i <= (unsigned)tablebits; i++) { start[i] >>= jutbits; weight[i] = (unsigned) 1 << (tablebits - i); } while (i <= 16) { weight[i] = (unsigned) 1 << (16 - i); i++; } i = start[tablebits + 1] >> jutbits; if (i != 0) { - k = 1 << tablebits; - while (i != k) table[i++] = 0; + k = MIN(1 << tablebits, DIST_BUFSIZE); + while (i < k) table[i++] = 0; } avail = nchar; mask = (unsigned) 1 << (15 - tablebits); for (ch = 0; ch < (unsigned)nchar; ch++) { if ((len = bitlen[ch]) == 0) continue; - nextcode = start[len] + weight[len]; + nextcode = MIN(start[len] + weight[len], DIST_BUFSIZE); if (len <= (unsigned)tablebits) { for (i = start[len]; i < nextcode; i++) table[i] = ch; } else { k = start[len]; p = &table[k >> jutbits]; i = len - tablebits; while (i != 0) { if (*p == 0) { right[avail] = left[avail] = 0; *p = avail++; } if (k & mask) p = &right[*p]; else p = &left[*p]; k <<= 1; i--; } *p = ch; } start[len] = nextcode; } } /*********************************************************** huf.c -- static Huffman ***********************************************************/ local void read_pt_len(nn, nbit, i_special) int nn; int nbit; int i_special; { int i, c, n; unsigned mask; n = getbits(nbit); if (n == 0) { c = getbits(nbit); for (i = 0; i < nn; i++) pt_len[i] = 0; for (i = 0; i < 256; i++) pt_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NPT)) { c = bitbuf >> (BITBUFSIZ - 3); if (c == 7) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 3); while (mask & bitbuf) { mask >>= 1; c++; } } fillbuf((c < 7) ? 3 : c - 3); pt_len[i++] = c; if (i == i_special) { c = getbits(2); - while (--c >= 0) pt_len[i++] = 0; + while (--c >= 0 && i < NPT) pt_len[i++] = 0; } } while (i < nn) pt_len[i++] = 0; make_table(nn, pt_len, 8, pt_table); } } local void read_c_len() { int i, c, n; unsigned mask; n = getbits(CBIT); if (n == 0) { c = getbits(CBIT); for (i = 0; i < NC; i++) c_len[i] = 0; for (i = 0; i < 4096; i++) c_table[i] = c; } else { i = 0; - while (i < n) { + while (i < MIN(n,NC)) { c = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (c >= NT) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) c = right[c]; else c = left [c]; mask >>= 1; - } while (c >= NT); + } while (c >= NT && (mask || c != left[c])); } fillbuf((int) pt_len[c]); if (c <= 2) { if (c == 0) c = 1; else if (c == 1) c = getbits(4) + 3; else c = getbits(CBIT) + 20; - while (--c >= 0) c_len[i++] = 0; + while (--c >= 0 && i < NC) c_len[i++] = 0; } else c_len[i++] = c - 2; } while (i < NC) c_len[i++] = 0; make_table(NC, c_len, 12, c_table); } } local unsigned decode_c() { unsigned j, mask; if (blocksize == 0) { blocksize = getbits(16); if (blocksize == 0) { return NC; /* end of file */ } read_pt_len(NT, TBIT, 3); read_c_len(); read_pt_len(NP, PBIT, -1); } blocksize--; j = c_table[bitbuf >> (BITBUFSIZ - 12)]; if (j >= NC) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 12); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NC); + } while (j >= NC && (mask || j != left[j])); } fillbuf((int) c_len[j]); return j; } local unsigned decode_p() { unsigned j, mask; j = pt_table[bitbuf >> (BITBUFSIZ - 8)]; if (j >= NP) { mask = (unsigned) 1 << (BITBUFSIZ - 1 - 8); do { if (bitbuf & mask) j = right[j]; else j = left [j]; mask >>= 1; - } while (j >= NP); + } while (j >= NP && (mask || j != left[j])); } fillbuf((int) pt_len[j]); if (j != 0) j = ((unsigned) 1 << (j - 1)) + getbits((int) (j - 1)); return j; } local void huf_decode_start() { init_getbits(); blocksize = 0; } /*********************************************************** decode.c ***********************************************************/ local int j; /* remaining bytes to copy */ local int done; /* set at end of input */ local void decode_start() { huf_decode_start(); j = 0; done = 0; } /* Decode the input and return the number of decoded bytes put in buffer */ local unsigned decode(count, buffer) unsigned count; uch buffer[]; /* The calling function must keep the number of bytes to be processed. This function decodes either 'count' bytes or 'DICSIZ' bytes, whichever is smaller, into the array 'buffer[]' of size 'DICSIZ' or more. Call decode_start() once for each new file before calling this function. */ { local unsigned i; unsigned r, c; r = 0; while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } for ( ; ; ) { c = decode_c(); if (c == NC) { done = 1; return r; } if (c <= UCHAR_MAX) { buffer[r] = c; - if (++r == count) return r; + if (++r >= count) return r; } else { j = c - (UCHAR_MAX + 1 - THRESHOLD); i = (r - decode_p() - 1) & (DICSIZ - 1); while (--j >= 0) { buffer[r] = buffer[i]; i = (i + 1) & (DICSIZ - 1); - if (++r == count) return r; + if (++r >= count) return r; } } } } /* =========================================================================== * Unlzh in to out. Return OK or ERROR. */ int unlzh(in, out) int in; int out; { unsigned n; ifd = in; ofd = out; decode_start(); while (!done) { n = decode((unsigned) DICSIZ, window); if (!test && n > 0) { write_buf(out, (char*)window, n); } } return OK; } Index: releng/6.1/gnu/usr.bin/gzip/unpack.c =================================================================== --- releng/6.1/gnu/usr.bin/gzip/unpack.c (revision 162446) +++ releng/6.1/gnu/usr.bin/gzip/unpack.c (revision 162447) @@ -1,239 +1,238 @@ /* unpack.c -- decompress files in pack format. * Copyright (C) 1992-1993 Jean-loup Gailly * This is free software; you can redistribute it and/or modify it under the * terms of the GNU General Public License, see the file COPYING. */ #ifdef RCSID static char rcsid[] = "$FreeBSD$"; #endif #include "tailor.h" #include "gzip.h" #include "crypt.h" -#define MIN(a,b) ((a) <= (b) ? (a) : (b)) /* The arguments must not have side effects. */ #define MAX_BITLEN 25 /* Maximum length of Huffman codes. (Minor modifications to the code * would be needed to support 32 bits codes, but pack never generates * more than 24 bits anyway.) */ #define LITERALS 256 /* Number of literals, excluding the End of Block (EOB) code */ #define MAX_PEEK 12 /* Maximum number of 'peek' bits used to optimize traversal of the * Huffman tree. */ local ulg orig_len; /* original uncompressed length */ local int max_len; /* maximum bit length of Huffman codes */ local uch literal[LITERALS]; /* The literal bytes present in the Huffman tree. The EOB code is not * represented. */ local int lit_base[MAX_BITLEN+1]; /* All literals of a given bit length are contiguous in literal[] and * have contiguous codes. literal[code+lit_base[len]] is the literal * for a code of len bits. */ local int leaves [MAX_BITLEN+1]; /* Number of leaves for each bit length */ local int parents[MAX_BITLEN+1]; /* Number of parents for each bit length */ local int peek_bits; /* Number of peek bits currently used */ /* local uch prefix_len[1 << MAX_PEEK]; */ #define prefix_len outbuf /* For each bit pattern b of peek_bits bits, prefix_len[b] is the length * of the Huffman code starting with a prefix of b (upper bits), or 0 * if all codes of prefix b have more than peek_bits bits. It is not * necessary to have a huge table (large MAX_PEEK) because most of the * codes encountered in the input stream are short codes (by construction). * So for most codes a single lookup will be necessary. */ #if (1< OUTBUFSIZ error cannot overlay prefix_len and outbuf #endif local ulg bitbuf; /* Bits are added on the low part of bitbuf and read from the high part. */ local int valid; /* number of valid bits in bitbuf */ /* all bits above the last valid bit are always zero */ /* Set code to the next 'bits' input bits without skipping them. code * must be the name of a simple variable and bits must not have side effects. * IN assertions: bits <= 25 (so that we still have room for an extra byte * when valid is only 24), and mask = (1<> (valid-(bits))) & (mask); \ } /* Skip the given number of bits (after having peeked at them): */ #define skip_bits(bits) (valid -= (bits)) #define clear_bitbuf() (valid = 0, bitbuf = 0) /* Local functions */ local void read_tree OF((void)); local void build_tree OF((void)); /* =========================================================================== * Read the Huffman tree. */ local void read_tree() { int len; /* bit length */ int base; /* base offset for a sequence of leaves */ int n; /* Read the original input size, MSB first */ orig_len = 0; for (n = 1; n <= 4; n++) orig_len = (orig_len << 8) | (ulg)get_byte(); max_len = (int)get_byte(); /* maximum bit length of Huffman codes */ if (max_len > MAX_BITLEN) { error("invalid compressed data -- Huffman code > 32 bits"); } /* Get the number of leaves at each bit length */ n = 0; for (len = 1; len <= max_len; len++) { leaves[len] = (int)get_byte(); n += leaves[len]; } if (n > LITERALS) { error("too many leaves in Huffman tree"); } Trace((stderr, "orig_len %ld, max_len %d, leaves %d\n", orig_len, max_len, n)); /* There are at least 2 and at most 256 leaves of length max_len. * (Pack arbitrarily rejects empty files and files consisting of * a single byte even repeated.) To fit the last leaf count in a * byte, it is offset by 2. However, the last literal is the EOB * code, and is not transmitted explicitly in the tree, so we must * adjust here by one only. */ leaves[max_len]++; /* Now read the leaves themselves */ base = 0; for (len = 1; len <= max_len; len++) { /* Remember where the literals of this length start in literal[] : */ lit_base[len] = base; /* And read the literals: */ - for (n = leaves[len]; n > 0; n--) { + for (n = leaves[len]; n > 0 && base < LITERALS; n--) { literal[base++] = (uch)get_byte(); } } leaves[max_len]++; /* Now include the EOB code in the Huffman tree */ } /* =========================================================================== * Build the Huffman tree and the prefix table. */ local void build_tree() { int nodes = 0; /* number of nodes (parents+leaves) at current bit length */ int len; /* current bit length */ uch *prefixp; /* pointer in prefix_len */ for (len = max_len; len >= 1; len--) { /* The number of parent nodes at this level is half the total * number of nodes at parent level: */ nodes >>= 1; parents[len] = nodes; /* Update lit_base by the appropriate bias to skip the parent nodes * (which are not represented in the literal array): */ lit_base[len] -= nodes; /* Restore nodes to be parents+leaves: */ nodes += leaves[len]; } /* Construct the prefix table, from shortest leaves to longest ones. * The shortest code is all ones, so we start at the end of the table. */ peek_bits = MIN(max_len, MAX_PEEK); prefixp = &prefix_len[1< prefix_len) *--prefixp = (uch)len; } /* The length of all other codes is unknown: */ while (prefixp > prefix_len) *--prefixp = 0; } /* =========================================================================== * Unpack in to out. This routine does not support the old pack format * with magic header \037\037. * * IN assertions: the buffer inbuf contains already the beginning of * the compressed data, from offsets inptr to insize-1 included. * The magic header has already been checked. The output buffer is cleared. */ int unpack(in, out) int in, out; /* input and output file descriptors */ { int len; /* Bit length of current code */ unsigned eob; /* End Of Block code */ register unsigned peek; /* lookahead bits */ unsigned peek_mask; /* Mask for peek_bits bits */ ifd = in; ofd = out; read_tree(); /* Read the Huffman tree */ build_tree(); /* Build the prefix table */ clear_bitbuf(); /* Initialize bit input */ peek_mask = (1< 0) { peek >>= peek_bits - len; /* discard the extra bits */ } else { /* Code of more than peek_bits bits, we must traverse the tree */ ulg mask = peek_mask; len = peek_bits; do { len++, mask = (mask<<1)+1; look_bits(peek, len, mask); } while (peek < (unsigned)parents[len]); /* loop as long as peek is a parent node */ } /* At this point, peek is the next complete code, of len bits */ if (peek == eob && len == max_len) break; /* end of file? */ put_ubyte(literal[peek+lit_base[len]]); Tracev((stderr,"%02d %04x %c\n", len, peek, literal[peek+lit_base[len]])); skip_bits(len); } /* for (;;) */ flush_window(); Trace((stderr, "bytes_out %ld\n", bytes_out)); if (orig_len != (ulg)bytes_out) { error("invalid compressed data--length error"); } return OK; } Index: releng/6.1/sys/conf/newvers.sh =================================================================== --- releng/6.1/sys/conf/newvers.sh (revision 162446) +++ releng/6.1/sys/conf/newvers.sh (revision 162447) @@ -1,103 +1,103 @@ #!/bin/sh - # # Copyright (c) 1984, 1986, 1990, 1993 # The Regents of the University of California. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # 4. Neither the name of the University nor the names of its contributors # may be used to endorse or promote products derived from this software # without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE # ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS # OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # @(#)newvers.sh 8.1 (Berkeley) 4/20/94 # $FreeBSD$ TYPE="FreeBSD" REVISION="6.1" -BRANCH="RELEASE-p6" +BRANCH="RELEASE-p7" if [ "X${BRANCH_OVERRIDE}" != "X" ]; then BRANCH=${BRANCH_OVERRIDE} fi RELEASE="${REVISION}-${BRANCH}" VERSION="${TYPE} ${RELEASE}" if [ "X${PARAMFILE}" != "X" ]; then RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ ${PARAMFILE}) else RELDATE=$(awk '/__FreeBSD_version.*propagated to newvers/ {print $3}' \ $(dirname $0)/../sys/param.h) fi b=share/examples/etc/bsd-style-copyright year=`date '+%Y'` # look for copyright template for bsd_copyright in ../$b ../../$b ../../../$b /usr/src/$b /usr/$b do if [ -r "$bsd_copyright" ]; then COPYRIGHT=`sed \ -e "s/\[year\]/1992-$year/" \ -e 's/\[your name here\]\.* /The FreeBSD Project./' \ -e 's/\[your name\]\.*/The FreeBSD Project./' \ -e '/\[id for your version control system, if any\]/d' \ $bsd_copyright` break fi done # no copyright found, use a dummy if [ X"$COPYRIGHT" = X ]; then COPYRIGHT="/*- * Copyright (c) 1992-$year The FreeBSD Project. * All rights reserved. * */" fi # add newline COPYRIGHT="$COPYRIGHT " LC_ALL=C; export LC_ALL if [ ! -r version ] then echo 0 > version fi touch version v=`cat version` u=${USER:-root} d=`pwd` h=${HOSTNAME:-`hostname`} t=`date` i=`${MAKE:-make} -V KERN_IDENT` cat << EOF > vers.c $COPYRIGHT #define SCCSSTR "@(#)${VERSION} #${v}: ${t}" #define VERSTR "${VERSION} #${v}: ${t}\\n ${u}@${h}:${d}\\n" #define RELSTR "${RELEASE}" char sccs[sizeof(SCCSSTR) > 128 ? sizeof(SCCSSTR) : 128] = SCCSSTR; char version[sizeof(VERSTR) > 256 ? sizeof(VERSTR) : 256] = VERSTR; char ostype[] = "${TYPE}"; char osrelease[sizeof(RELSTR) > 32 ? sizeof(RELSTR) : 32] = RELSTR; int osreldate = ${RELDATE}; char kern_ident[] = "${i}"; EOF echo `expr ${v} + 1` > version