Index: stable/10/UPDATING =================================================================== --- stable/10/UPDATING (revision 282274) +++ stable/10/UPDATING (revision 282275) @@ -1,2171 +1,2175 @@ Updating Information for FreeBSD current users This file is maintained and copyrighted by M. Warner Losh . See end of file for further details. For commonly done items, please see the COMMON ITEMS: section later in the file. These instructions assume that you basically know what you are doing. If not, then please consult the FreeBSD handbook: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before running portupgrade. NOTE: FreeBSD has switched from gcc to clang. If you have trouble bootstrapping from older versions of FreeBSD, try WITHOUT_CLANG to bootstrap to the tip of stable/10, and then rebuild without this option. The bootstrap process from older version of current is a bit fragile. +20150430: + The const qualifier has been removed from iconv(3) to comply with + POSIX. The ports tree is aware of this from r384038 onwards. + 20141215: At svn r275807, The default linux compat kernel ABI has been adjusted to 2.6.18 in support of the linux-c6 compat ports infrastructure update. If you wish to continue using the linux-f10 compat ports, add compat.linux.osrelease=2.6.16 to your local sysctl.conf. Users are encouraged to update their linux-compat packages to linux-c6 during their next update cycle. See ports/UPDATING 20141209 and 20141215 on migration to CentOS 6 ports. 20141205: pjdfstest has been integrated into kyua as an opt-in test suite. Please see share/doc/pjdfstest/README for a more details on how to execute it. 20141118: 10.1-RELEASE. 20140904: The ofwfb driver, used to provide a graphics console on PowerPC when using vt(4), no longer allows mmap() of all of physical memory. This will prevent Xorg on PowerPC with some ATI graphics cards from initializing properly unless x11-servers/xorg-server is updated to 1.12.4_8 or newer. 20140831: The libatf-c and libatf-c++ major versions were downgraded to 0 and 1 respectively to match the upstream numbers. They were out of sync because, when they were originally added to FreeBSD, the upstream versions were not respected. These libraries are private and not yet built by default, so renumbering them should be a non-issue. However, unclean source trees will yield broken test programs once the operator executes "make delete-old-libs" after a "make installworld". Additionally, the atf-sh binary was made private by moving it into /usr/libexec/. Already-built shell test programs will keep the path to the old binary so they will break after "make delete-old" is run. If you are using WITH_TESTS=yes (not the default), wipe the object tree and rebuild from scratch to prevent spurious test failures. This is only needed once: the misnumbered libraries and misplaced binaries have been added to OptionalObsoleteFiles.inc so they will be removed during a clean upgrade. 20140814: The ixgbe tunables now match their sysctl counterparts, for example: hw.ixgbe.enable_aim => hw.ix.enable_aim Anyone using ixgbe tunables should ensure they update /boot/loader.conf. 20140801: The NFSv4.1 server committed by r269398 changes the internal function call interfaces used between the NFS and krpc modules. As such, __FreeBSD_version was bumped. 20140729: The default unbound configuration has been modified to address issues with reverse lookups on networks that use private address ranges. If you use the local_unbound service, run "service local_unbound setup" as root to regenerate your configuration, then "service local_unbound reload" to load the new configuration. 20140717: It is no longer necessary to include the dwarf version in your DEBUG options in your kernel config file. The bug that required it to be placed in the config file has bene fixed. DEBUG should now just contain -g. The build system will automatically update things to do the right thing. 20140715: Several ABI breaking changes were merged to CTL and new iSCSI code. All CTL and iSCSI-related tools, such as ctladm, ctld, iscsid and iscsictl need to be rebuilt to work with a new kernel. 20140708: The WITHOUT_VT_SUPPORT kernel config knob has been renamed WITHOUT_VT. (The other _SUPPORT knobs have a consistent meaning which differs from the behaviour controlled by this knob.) 20140608: On i386 and amd64 systems, the onifconsole flag is now set by default in /etc/ttys for ttyu0. This causes ttyu0 to be automatically enabled as a login TTY if it is set in the bootloader as an active kernel console. No changes in behavior should result otherwise. To revert to the previous behavior, set ttyu0 to "off" in /etc/ttys. 20140512: Clang and llvm have been upgraded to 3.4.1 release. 20140321: Clang and llvm have been upgraded to 3.4 release. 20140306: If a Makefile in a tests/ directory was auto-generating a Kyuafile instead of providing an explicit one, this would prevent such Makefile from providing its own Kyuafile in the future during NO_CLEAN builds. This has been fixed in the Makefiles but manual intervention is needed to clean an objdir if you use NO_CLEAN: # find /usr/obj -name Kyuafile | xargs rm -f 20140303: OpenSSH will now ignore errors caused by kernel lacking of Capsicum capability mode support. Please note that enabling the feature in kernel is still highly recommended. 20140227: OpenSSH is now built with sandbox support, and will use sandbox as the default privilege separation method. This requires Capsicum capability mode support in kernel. 20140216: The nve(4) driver for NVIDIA nForce MCP Ethernet adapters has been deprecated and will not be part of FreeBSD 11.0 and later releases. If you use this driver, please consider switching to the nfe(4) driver instead. 20140120: 10.0-RELEASE. 20131216: The behavior of gss_pseudo_random() for the krb5 mechanism has changed, for applications requesting a longer random string than produced by the underlying enctype's pseudo-random() function. In particular, the random string produced from a session key of enctype aes256-cts-hmac-sha1-96 or aes256-cts-hmac-sha1-96 will be different at the 17th octet and later, after this change. The counter used in the PRF+ construction is now encoded as a big-endian integer in accordance with RFC 4402. __FreeBSD_version is bumped to 1000701. 20131108: The WITHOUT_ATF build knob has been removed and its functionality has been subsumed into the more generic WITHOUT_TESTS. If you were using the former to disable the build of the ATF libraries, you should change your settings to use the latter. 20131031: The default version of mtree is nmtree which is obtained from NetBSD. The output is generally the same, but may vary slightly. If you found you need identical output adding "-F freebsd9" to the command line should do the trick. For the time being, the old mtree is available as fmtree. 20131014: libbsdyml has been renamed to libyaml and moved to /usr/lib/private. This will break ports-mgmt/pkg. Rebuild the port, or upgrade to pkg 1.1.4_8 and verify bsdyml not linked in, before running "make delete-old-libs": # make -C /usr/ports/ports-mgmt/pkg build deinstall install clean or # pkg install pkg; ldd /usr/local/sbin/pkg | grep bsdyml 20131010: The rc.d/jail script has been updated to support jail(8) configuration file. The "jail__*" rc.conf(5) variables for per-jail configuration are automatically converted to /var/run/jail..conf before the jail(8) utility is invoked. This is transparently backward compatible. See below about some incompatibilities and rc.conf(5) manual page for more details. These variables are now deprecated in favor of jail(8) configuration file. One can use "rc.d/jail config " command to generate a jail(8) configuration file in /var/run/jail..conf without running the jail(8) utility. The default pathname of the configuration file is /etc/jail.conf and can be specified by using $jail_conf or $jail__conf variables. Please note that jail_devfs_ruleset accepts an integer at this moment. Please consider to rewrite the ruleset name with an integer. 20130930: BIND has been removed from the base system. If all you need is a local resolver, simply enable and start the local_unbound service instead. Otherwise, several versions of BIND are available in the ports tree. The dns/bind99 port is one example. With this change, nslookup(1) and dig(1) are no longer in the base system. Users should instead use host(1) and drill(1) which are in the base system. Alternatively, nslookup and dig can be obtained by installing the dns/bind-tools port. 20130916: With the addition of unbound(8), a new unbound user is now required during installworld. "mergemaster -p" can be used to add the user prior to installworld, as documented in the handbook. 20130911: OpenSSH is now built with DNSSEC support, and will by default silently trust signed SSHFP records. This can be controlled with the VerifyHostKeyDNS client configuration setting. DNSSEC support can be disabled entirely with the WITHOUT_LDNS option in src.conf. 20130906: The GNU Compiler Collection and C++ standard library (libstdc++) are no longer built by default on platforms where clang is the system compiler. You can enable them with the WITH_GCC and WITH_GNUCXX options in src.conf. 20130905: The PROCDESC kernel option is now part of the GENERIC kernel configuration and is required for the rwhod(8) to work. If you are using custom kernel configuration, you should include 'options PROCDESC'. 20130905: The API and ABI related to the Capsicum framework was modified in backward incompatible way. The userland libraries and programs have to be recompiled to work with the new kernel. This includes the following libraries and programs, but the whole buildworld is advised: libc, libprocstat, dhclient, tcpdump, hastd, hastctl, kdump, procstat, rwho, rwhod, uniq. 20130903: AES-NI intrinsic support has been added to gcc. The AES-NI module has been updated to use this support. A new gcc is required to build the aesni module on both i386 and amd64. 20130821: The PADLOCK_RNG and RDRAND_RNG kernel options are now devices. Thus "device padlock_rng" and "device rdrand_rng" should be used instead of "options PADLOCK_RNG" & "options RDRAND_RNG". 20130813: WITH_ICONV has been split into two feature sets. WITH_ICONV now enables just the iconv* functionality and is now on by default. WITH_LIBICONV_COMPAT enables the libiconv api and link time compatability. Set WITHOUT_ICONV to build the old way. If you have been using WITH_ICONV before, you will very likely need to turn on WITH_LIBICONV_COMPAT. 20130806: INVARIANTS option now enables DEBUG for code with OpenSolaris and Illumos origin, including ZFS. If you have INVARIANTS in your kernel configuration, then there is no need to set DEBUG or ZFS_DEBUG explicitly. DEBUG used to enable witness(9) tracking of OpenSolaris (mostly ZFS) locks if WITNESS option was set. Because that generated a lot of witness(9) reports and all of them were believed to be false positives, this is no longer done. New option OPENSOLARIS_WITNESS can be used to achieve the previous behavior. 20130806: Timer values in IPv6 data structures now use time_uptime instead of time_second. Although this is not a user-visible functional change, userland utilities which directly use them---ndp(8), rtadvd(8), and rtsold(8) in the base system---need to be updated to r253970 or later. 20130802: find -delete can now delete the pathnames given as arguments, instead of only files found below them or if the pathname did not contain any slashes. Formerly, the following error message would result: find: -delete: : relative path potentially not safe Deleting the pathnames given as arguments can be prevented without error messages using -mindepth 1 or by changing directory and passing "." as argument to find. This works in the old as well as the new version of find. 20130726: Behavior of devfs rules path matching has been changed. Pattern is now always matched against fully qualified devfs path and slash characters must be explicitly matched by slashes in pattern (FNM_PATHNAME). Rulesets involving devfs subdirectories must be reviewed. 20130716: The default ARM ABI has changed to the ARM EABI. The old ABI is incompatible with the ARM EABI and all programs and modules will need to be rebuilt to work with a new kernel. To keep using the old ABI ensure the WITHOUT_ARM_EABI knob is set. NOTE: Support for the old ABI will be removed in the future and users are advised to upgrade. 20130709: pkg_install has been disconnected from the build if you really need it you should add WITH_PKGTOOLS in your src.conf(5). 20130709: Most of network statistics structures were changed to be able keep 64-bits counters. Thus all tools, that work with networking statistics, must be rebuilt (netstat(1), bsnmpd(1), etc.) 20130629: Fix targets that run multiple make's to use && rather than ; so that subsequent steps depend on success of previous. NOTE: if building 'universe' with -j* on stable/8 or stable/9 it would be better to start the build using bmake, to avoid overloading the machine. 20130618: Fix a bug that allowed a tracing process (e.g. gdb) to write to a memory-mapped file in the traced process's address space even if neither the traced process nor the tracing process had write access to that file. 20130615: CVS has been removed from the base system. An exact copy of the code is available from the devel/cvs port. 20130613: Some people report the following error after the switch to bmake: make: illegal option -- J usage: make [-BPSXeiknpqrstv] [-C directory] [-D variable] ... *** [buildworld] Error code 2 this likely due to an old instance of make in ${MAKEPATH} (${MAKEOBJDIRPREFIX}${.CURDIR}/make.${MACHINE}) which src/Makefile will use that blindly, if it exists, so if you see the above error: rm -rf `make -V MAKEPATH` should resolve it. 20130516: Use bmake by default. Whereas before one could choose to build with bmake via -DWITH_BMAKE one must now use -DWITHOUT_BMAKE to use the old make. The goal is to remove these knobs for 10-RELEASE. It is worth noting that bmake (like gmake) treats the command line as the unit of failure, rather than statements within the command line. Thus '(cd some/where && dosomething)' is safer than 'cd some/where; dosomething'. The '()' allows consistent behavior in parallel build. 20130429: Fix a bug that allows NFS clients to issue READDIR on files. 20130426: The WITHOUT_IDEA option has been removed because the IDEA patent expired. 20130426: The sysctl which controls TRIM support under ZFS has been renamed from vfs.zfs.trim_disable -> vfs.zfs.trim.enabled and has been enabled by default. 20130425: The mergemaster command now uses the default MAKEOBJDIRPREFIX rather than creating it's own in the temporary directory in order allow access to bootstrapped versions of tools such as install and mtree. When upgrading from version of FreeBSD where the install command does not support -l, you will need to install a new mergemaster command if mergemaster -p is required. This can be accomplished with the command (cd src/usr.sbin/mergemaster && make install). 20130404: Legacy ATA stack, disabled and replaced by new CAM-based one since FreeBSD 9.0, completely removed from the sources. Kernel modules atadisk and atapi*, user-level tools atacontrol and burncd are removed. Kernel option `options ATA_CAM` is now permanently enabled and removed. 20130319: SOCK_CLOEXEC and SOCK_NONBLOCK flags have been added to socket(2) and socketpair(2). Software, in particular Kerberos, may automatically detect and use these during building. The resulting binaries will not work on older kernels. 20130308: CTL_DISABLE has also been added to the sparc64 GENERIC (for further information, see the respective 20130304 entry). 20130304: Recent commits to callout(9) changed the size of struct callout, so the KBI is probably heavily disturbed. Also, some functions in callout(9)/sleep(9)/sleepqueue(9)/condvar(9) KPIs were replaced by macros. Every kernel module using it won't load, so rebuild is requested. The ctl device has been re-enabled in GENERIC for i386 and amd64, but does not initialize by default (because of the new CTL_DISABLE option) to save memory. To re-enable it, remove the CTL_DISABLE option from the kernel config file or set kern.cam.ctl.disable=0 in /boot/loader.conf. 20130301: The ctl device has been disabled in GENERIC for i386 and amd64. This was done due to the extra memory being allocated at system initialisation time by the ctl driver which was only used if a CAM target device was created. This makes a FreeBSD system unusable on 128MB or less of RAM. 20130208: A new compression method (lz4) has been merged to -HEAD. Please refer to zpool-features(7) for more information. Please refer to the "ZFS notes" section of this file for information on upgrading boot ZFS pools. 20130129: A BSD-licensed patch(1) variant has been added and is installed as bsdpatch, being the GNU version the default patch. To inverse the logic and use the BSD-licensed one as default, while having the GNU version installed as gnupatch, rebuild and install world with the WITH_BSD_PATCH knob set. 20130121: Due to the use of the new -l option to install(1) during build and install, you must take care not to directly set the INSTALL make variable in your /etc/make.conf, /etc/src.conf, or on the command line. If you wish to use the -C flag for all installs you may be able to add INSTALL+=-C to /etc/make.conf or /etc/src.conf. 20130118: The install(1) option -M has changed meaning and now takes an argument that is a file or path to append logs to. In the unlikely event that -M was the last option on the command line and the command line contained at least two files and a target directory the first file will have logs appended to it. The -M option served little practical purpose in the last decade so its use is expected to be extremely rare. 20121223: After switching to Clang as the default compiler some users of ZFS on i386 systems started to experience stack overflow kernel panics. Please consider using 'options KSTACK_PAGES=4' in such configurations. 20121222: GEOM_LABEL now mangles label names read from file system metadata. Mangling affect labels containing spaces, non-printable characters, '%' or '"'. Device names in /etc/fstab and other places may need to be updated. 20121217: By default, only the 10 most recent kernel dumps will be saved. To restore the previous behaviour (no limit on the number of kernel dumps stored in the dump directory) add the following line to /etc/rc.conf: savecore_flags="" 20121201: With the addition of auditdistd(8), a new auditdistd user is now required during installworld. "mergemaster -p" can be used to add the user prior to installworld, as documented in the handbook. 20121117: The sin6_scope_id member variable in struct sockaddr_in6 is now filled by the kernel before passing the structure to the userland via sysctl or routing socket. This means the KAME-specific embedded scope id in sin6_addr.s6_addr[2] is always cleared in userland application. This behavior can be controlled by net.inet6.ip6.deembed_scopeid. __FreeBSD_version is bumped to 1000025. 20121105: On i386 and amd64 systems WITH_CLANG_IS_CC is now the default. This means that the world and kernel will be compiled with clang and that clang will be installed as /usr/bin/cc, /usr/bin/c++, and /usr/bin/cpp. To disable this behavior and revert to building with gcc, compile with WITHOUT_CLANG_IS_CC. Really old versions of current may need to bootstrap WITHOUT_CLANG first if the clang build fails (its compatibility window doesn't extend to the 9 stable branch point). 20121102: The IPFIREWALL_FORWARD kernel option has been removed. Its functionality now turned on by default. 20121023: The ZERO_COPY_SOCKET kernel option has been removed and split into SOCKET_SEND_COW and SOCKET_RECV_PFLIP. NB: SOCKET_SEND_COW uses the VM page based copy-on-write mechanism which is not safe and may result in kernel crashes. NB: The SOCKET_RECV_PFLIP mechanism is useless as no current driver supports disposeable external page sized mbuf storage. Proper replacements for both zero-copy mechanisms are under consideration and will eventually lead to complete removal of the two kernel options. 20121023: The IPv4 network stack has been converted to network byte order. The following modules need to be recompiled together with kernel: carp(4), divert(4), gif(4), siftr(4), gre(4), pf(4), ipfw(4), ng_ipfw(4), stf(4). 20121022: Support for non-MPSAFE filesystems was removed from VFS. The VFS_VERSION was bumped, all filesystem modules shall be recompiled. 20121018: All the non-MPSAFE filesystems have been disconnected from the build. The full list includes: codafs, hpfs, ntfs, nwfs, portalfs, smbfs, xfs. 20121016: The interface cloning API and ABI has changed. The following modules need to be recompiled together with kernel: ipfw(4), pfsync(4), pflog(4), usb(4), wlan(4), stf(4), vlan(4), disc(4), edsc(4), if_bridge(4), gif(4), tap(4), faith(4), epair(4), enc(4), tun(4), if_lagg(4), gre(4). 20121015: The sdhci driver was split in two parts: sdhci (generic SD Host Controller logic) and sdhci_pci (actual hardware driver). No kernel config modifications are required, but if you load sdhc as a module you must switch to sdhci_pci instead. 20121014: Import the FUSE kernel and userland support into base system. 20121013: The GNU sort(1) program has been removed since the BSD-licensed sort(1) has been the default for quite some time and no serious problems have been reported. The corresponding WITH_GNU_SORT knob has also gone. 20121006: The pfil(9) API/ABI for AF_INET family has been changed. Packet filtering modules: pf(4), ipfw(4), ipfilter(4) need to be recompiled with new kernel. 20121001: The net80211(4) ABI has been changed to allow for improved driver PS-POLL and power-save support. All wireless drivers need to be recompiled to work with the new kernel. 20120913: The random(4) support for the VIA hardware random number generator (`PADLOCK') is no longer enabled unconditionally. Add the padlock_rng device in the custom kernel config if needed. The GENERIC kernels on i386 and amd64 do include the device, so the change only affects the custom kernel configurations. 20120908: The pf(4) packet filter ABI has been changed. pfctl(8) and snmp_pf module need to be recompiled to work with new kernel. 20120828: A new ZFS feature flag "com.delphix:empty_bpobj" has been merged to -HEAD. Pools that have empty_bpobj in active state can not be imported read-write with ZFS implementations that do not support this feature. For more information read the zpool-features(5) manual page. 20120727: The sparc64 ZFS loader has been changed to no longer try to auto- detect ZFS providers based on diskN aliases but now requires these to be explicitly listed in the OFW boot-device environment variable. 20120712: The OpenSSL has been upgraded to 1.0.1c. Any binaries requiring libcrypto.so.6 or libssl.so.6 must be recompiled. Also, there are configuration changes. Make sure to merge /etc/ssl/openssl.cnf. 20120712: The following sysctls and tunables have been renamed for consistency with other variables: kern.cam.da.da_send_ordered -> kern.cam.da.send_ordered kern.cam.ada.ada_send_ordered -> kern.cam.ada.send_ordered 20120628: The sort utility has been replaced with BSD sort. For now, GNU sort is also available as "gnusort" or the default can be set back to GNU sort by setting WITH_GNU_SORT. In this case, BSD sort will be installed as "bsdsort". 20120611: A new version of ZFS (pool version 5000) has been merged to -HEAD. Starting with this version the old system of ZFS pool versioning is superseded by "feature flags". This concept enables forward compatibility against certain future changes in functionality of ZFS pools. The first read-only compatible "feature flag" for ZFS pools is named "com.delphix:async_destroy". For more information read the new zpool-features(5) manual page. Please refer to the "ZFS notes" section of this file for information on upgrading boot ZFS pools. 20120417: The malloc(3) implementation embedded in libc now uses sources imported as contrib/jemalloc. The most disruptive API change is to /etc/malloc.conf. If your system has an old-style /etc/malloc.conf, delete it prior to installworld, and optionally re-create it using the new format after rebooting. See malloc.conf(5) for details (specifically the TUNING section and the "opt.*" entries in the MALLCTL NAMESPACE section). 20120328: Big-endian MIPS TARGET_ARCH values no longer end in "eb". mips64eb is now spelled mips64. mipsn32eb is now spelled mipsn32. mipseb is now spelled mips. This is to aid compatibility with third-party software that expects this naming scheme in uname(3). Little-endian settings are unchanged. If you are updating a big-endian mips64 machine from before this change, you may need to set MACHINE_ARCH=mips64 in your environment before the new build system will recognize your machine. 20120306: Disable by default the option VFS_ALLOW_NONMPSAFE for all supported platforms. 20120229: Now unix domain sockets behave "as expected" on nullfs(5). Previously nullfs(5) did not pass through all behaviours to the underlying layer, as a result if we bound to a socket on the lower layer we could connect only to the lower path; if we bound to the upper layer we could connect only to the upper path. The new behavior is one can connect to both the lower and the upper paths regardless what layer path one binds to. 20120211: The getifaddrs upgrade path broken with 20111215 has been restored. If you have upgraded in between 20111215 and 20120209 you need to recompile libc again with your kernel. You still need to recompile world to be able to configure CARP but this restriction already comes from 20111215. 20120114: The set_rcvar() function has been removed from /etc/rc.subr. All base and ports rc.d scripts have been updated, so if you have a port installed with a script in /usr/local/etc/rc.d you can either hand-edit the rcvar= line, or reinstall the port. An easy way to handle the mass-update of /etc/rc.d: rm /etc/rc.d/* && mergemaster -i 20120109: panic(9) now stops other CPUs in the SMP systems, disables interrupts on the current CPU and prevents other threads from running. This behavior can be reverted using the kern.stop_scheduler_on_panic tunable/sysctl. The new behavior can be incompatible with kern.sync_on_panic. 20111215: The carp(4) facility has been changed significantly. Configuration of the CARP protocol via ifconfig(8) has changed, as well as format of CARP events submitted to devd(8) has changed. See manual pages for more information. The arpbalance feature of carp(4) is currently not supported anymore. Size of struct in_aliasreq, struct in6_aliasreq has changed. User utilities using SIOCAIFADDR, SIOCAIFADDR_IN6, e.g. ifconfig(8), need to be recompiled. 20111122: The acpi_wmi(4) status device /dev/wmistat has been renamed to /dev/wmistat0. 20111108: The option VFS_ALLOW_NONMPSAFE option has been added in order to explicitely support non-MPSAFE filesystems. It is on by default for all supported platform at this present time. 20111101: The broken amd(4) driver has been replaced with esp(4) in the amd64, i386 and pc98 GENERIC kernel configuration files. 20110930: sysinstall has been removed 20110923: The stable/9 branch created in subversion. This corresponds to the RELENG_9 branch in CVS. 20110913: This commit modifies vfs_register() so that it uses a hash calculation to set vfc_typenum, which is enabled by default. The first time a system is booted after this change, the vfc_typenum values will change for all file systems. The main effect of this is a change to the NFS server file handles for file systems that use vfc_typenum in their fsid, such as ZFS. It will, however, prevent vfc_typenum from changing when file systems are loaded in a different order for subsequent reboots. To disable this, you can set vfs.typenumhash=0 in /boot/loader.conf until you are ready to remount all NFS clients after a reboot. 20110828: Bump the shared library version numbers for libraries that do not use symbol versioning, have changed the ABI compared to stable/8 and which shared library version was not bumped. Done as part of 9.0-RELEASE cycle. 20110815: During the merge of Capsicum features, the fget(9) KPI was modified. This may require the rebuilding of out-of-tree device drivers -- issues have been reported specifically with the nVidia device driver. __FreeBSD_version is bumped to 900041. Also, there is a period between 20110811 and 20110814 where the special devices /dev/{stdin,stdout,stderr} did not work correctly. Building world from a kernel during that window may not work. 20110628: The packet filter (pf) code has been updated to OpenBSD 4.5. You need to update userland tools to be in sync with kernel. This update breaks backward compatibility with earlier pfsync(4) versions. Care must be taken when updating redundant firewall setups. 20110608: The following sysctls and tunables are retired on x86 platforms: machdep.hlt_cpus machdep.hlt_logical_cpus The following sysctl is retired: machdep.hyperthreading_allowed The sysctls were supposed to provide a way to dynamically offline and online selected CPUs on x86 platforms, but the implementation has not been reliable especially with SCHED_ULE scheduler. machdep.hyperthreading_allowed tunable is still available to ignore hyperthreading CPUs at OS level. Individual CPUs can be disabled using hint.lapic.X.disabled tunable, where X is an APIC ID of a CPU. Be advised, though, that disabling CPUs in non-uniform fashion will result in non-uniform topology and may lead to sub-optimal system performance with SCHED_ULE, which is a default scheduler. 20110607: cpumask_t type is retired and cpuset_t is used in order to describe a mask of CPUs. 20110531: Changes to ifconfig(8) for dynamic address family detection mandate that you are running a kernel of 20110525 or later. Make sure to follow the update procedure to boot a new kernel before installing world. 20110513: Support for sun4v architecture is officially dropped 20110503: Several KPI breaking changes have been committed to the mii(4) layer, the PHY drivers and consequently some Ethernet drivers using mii(4). This means that miibus.ko and the modules of the affected Ethernet drivers need to be recompiled. Note to kernel developers: Given that the OUI bit reversion problem was fixed as part of these changes all mii(4) commits related to OUIs, i.e. to sys/dev/mii/miidevs, PHY driver probing and vendor specific handling, no longer can be merged verbatim to stable/8 and previous branches. 20110430: Users of the Atheros AR71xx SoC code now need to add 'device ar71xx_pci' into their kernel configurations along with 'device pci'. 20110427: The default NFS client is now the new NFS client, so fstype "newnfs" is now "nfs" and the regular/old NFS client is now fstype "oldnfs". Although mounts via fstype "nfs" will usually work without userland changes, it is recommended that the mount(8) and mount_nfs(8) commands be rebuilt from sources and that a link to mount_nfs called mount_oldnfs be created. The new client is compiled into the kernel with "options NFSCL" and this is needed for diskless root file systems. The GENERIC kernel configs have been changed to use NFSCL and NFSD (the new server) instead of NFSCLIENT and NFSSERVER. To use the regular/old client, you can "mount -t oldnfs ...". For a diskless root file system, you must also include a line like: vfs.root.mountfrom="oldnfs:" in the boot/loader.conf on the root fs on the NFS server to make a diskless root fs use the old client. 20110424: The GENERIC kernels for all architectures now default to the new CAM-based ATA stack. It means that all legacy ATA drivers were removed and replaced by respective CAM drivers. If you are using ATA device names in /etc/fstab or other places, make sure to update them respectively (adX -> adaY, acdX -> cdY, afdX -> daY, astX -> saY, where 'Y's are the sequential numbers starting from zero for each type in order of detection, unless configured otherwise with tunables, see cam(4)). There will be symbolic links created in /dev/ to map old adX devices to the respective adaY. They should provide basic compatibility for file systems mounting in most cases, but they do not support old user-level APIs and do not have respective providers in GEOM. Consider using updated management tools with new device names. It is possible to load devices ahci, ata, siis and mvs as modules, but option ATA_CAM should remain in kernel configuration to make ata module work as CAM driver supporting legacy ATA controllers. Device ata still can be used in modular fashion (atacore + ...). Modules atadisk and atapi* are not used and won't affect operation in ATA_CAM mode. Note that to use CAM-based ATA kernel should include CAM devices scbus, pass, da (or explicitly ada), cd and optionally others. All of them are parts of the cam module. ataraid(4) functionality is now supported by the RAID GEOM class. To use it you can load geom_raid kernel module and use graid(8) tool for management. Instead of /dev/arX device names, use /dev/raid/rX. No kernel config options or code have been removed, so if a problem arises, please report it and optionally revert to the old ATA stack. In order to do it you can remove from the kernel config: options ATA_CAM device ahci device mvs device siis , and instead add back: device atadisk # ATA disk drives device ataraid # ATA RAID drives device atapicd # ATAPI CDROM drives device atapifd # ATAPI floppy drives device atapist # ATAPI tape drives 20110423: The default NFS server has been changed to the new server, which was referred to as the experimental server. If you need to switch back to the old NFS server, you must now put the "-o" option on both the mountd and nfsd commands. This can be done using the mountd_flags and nfs_server_flags rc.conf variables until an update to the rc scripts is committed, which is coming soon. 20110418: The GNU Objective-C runtime library (libobjc), and other Objective-C related components have been removed from the base system. If you require an Objective-C library, please use one of the available ports. 20110331: ath(4) has been split into bus- and device- modules. if_ath contains the HAL, the TX rate control and the network device code. if_ath_pci contains the PCI bus glue. For Atheros MIPS embedded systems, if_ath_ahb contains the AHB glue. Users need to load both if_ath_pci and if_ath in order to use ath on everything else. TO REPEAT: if_ath_ahb is not needed for normal users. Normal users only need to load if_ath and if_ath_pci for ath(4) operation. 20110314: As part of the replacement of sysinstall, the process of building release media has changed significantly. For details, please re-read release(7), which has been updated to reflect the new build process. 20110218: GNU binutils 2.17.50 (as of 2007-07-03) has been merged to -HEAD. This is the last available version under GPLv2. It brings a number of new features, such as support for newer x86 CPU's (with SSE-3, SSSE-3, SSE 4.1 and SSE 4.2), better support for powerpc64, a number of new directives, and lots of other small improvements. See the ChangeLog file in contrib/binutils for the full details. 20110218: IPsec's HMAC_SHA256-512 support has been fixed to be RFC4868 compliant, and will now use half of hash for authentication. This will break interoperability with all stacks (including all actual FreeBSD versions) who implement draft-ietf-ipsec-ciph-sha-256-00 (they use 96 bits of hash for authentication). The only workaround with such peers is to use another HMAC algorithm for IPsec ("phase 2") authentication. 20110207: Remove the uio_yield prototype and symbol. This function has been misnamed since it was introduced and should not be globally exposed with this name. The equivalent functionality is now available using kern_yield(curthread->td_user_pri). The function remains undocumented. 20110112: A SYSCTL_[ADD_]UQUAD was added for unsigned uint64_t pointers, symmetric with the existing SYSCTL_[ADD_]QUAD. Type checking for scalar sysctls is defined but disabled. Code that needs UQUAD to pass the type checking that must compile on older systems where the define is not present can check against __FreeBSD_version >= 900030. The system dialog(1) has been replaced with a new version previously in ports as devel/cdialog. dialog(1) is mostly command-line compatible with the previous version, but the libdialog associated with it has a largely incompatible API. As such, the original version of libdialog will be kept temporarily as libodialog, until its base system consumers are replaced or updated. Bump __FreeBSD_version to 900030. 20110103: If you are trying to run make universe on a -stable system, and you get the following warning: "Makefile", line 356: "Target architecture for i386/conf/GENERIC unknown. config(8) likely too old." or something similar to it, then you must upgrade your -stable system to 8.2-Release or newer (really, any time after r210146 7/15/2010 in stable/8) or build the config from the latest stable/8 branch and install it on your system. Prior to this date, building a current universe on 8-stable system from between 7/15/2010 and 1/2/2011 would result in a weird shell parsing error in the first kernel build phase. A new config on those old systems will fix that problem for older versions of -current. 20101228: The TCP stack has been modified to allow Khelp modules to interact with it via helper hook points and store per-connection data in the TCP control block. Bump __FreeBSD_version to 900029. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20101114: Generic IEEE 802.3 annex 31B full duplex flow control support has been added to mii(4) and bge(4), bce(4), msk(4), nfe(4) and stge(4) along with brgphy(4), e1000phy(4) as well as ip1000phy() have been converted to take advantage of it instead of using custom implementations. This means that these drivers now no longer unconditionally advertise support for flow control but only do so if flow control is a selected media option. This was implemented in the generic support that way in order to allow flow control to be switched on and off via ifconfig(8) with the PHY specific default to typically off in order to protect from unwanted effects. Consequently, if you used flow control with one of the above mentioned drivers you now need to explicitly enable it, for example via: ifconfig bge0 media auto mediaopt flowcontrol Along with the above mentioned changes generic support for setting 1000baseT master mode also has been added and brgphy(4), ciphy(4), e1000phy(4) as well as ip1000phy(4) have been converted to take advantage of it. This means that these drivers now no longer take the link0 parameter for selecting master mode but the master media option has to be used instead, for example like in the following: ifconfig bge0 media 1000baseT mediaopt full-duplex,master Selection of master mode now is also available with all other PHY drivers supporting 1000baseT. 20101111: The TCP stack has received a significant update to add support for modularised congestion control and generally improve the clarity of congestion control decisions. Bump __FreeBSD_version to 900025. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20101002: The man(1) utility has been replaced by a new version that no longer uses /etc/manpath.config. Please consult man.conf(5) for how to migrate local entries to the new format. 20100928: The copyright strings printed by login(1) and sshd(8) at the time of a new connection have been removed to follow other operating systems and upstream sshd. 20100915: A workaround for a fixed ld bug has been removed in kernel code, so make sure that your system ld is built from sources after revision 210245 from 2010-07-19 (r211583 if building head kernel on stable/8, r211584 for stable/7; both from 2010-08-21). A symptom of incorrect ld version is different addresses for set_pcpu section and __start_set_pcpu symbol in kernel and/or modules. 20100913: The $ipv6_prefer variable in rc.conf(5) has been split into $ip6addrctl_policy and $ipv6_activate_all_interfaces. The $ip6addrctl_policy is a variable to choose a pre-defined address selection policy set by ip6addrctl(8). A value "ipv4_prefer", "ipv6_prefer" or "AUTO" can be specified. The default is "AUTO". The $ipv6_activate_all_interfaces specifies whether IFDISABLED flag (see an entry of 20090926) is set on an interface with no corresponding $ifconfig_IF_ipv6 line. The default is "NO" for security reason. If you want IPv6 link-local address on all interfaces by default, set this to "YES". The old ipv6_prefer="YES" is equivalent to ipv6_activate_all_interfaces="YES" and ip6addrctl_policy="ipv6_prefer". 20100913: DTrace has grown support for userland tracing. Due to this, DTrace is now i386 and amd64 only. dtruss(1) is now installed by default on those systems and a new kernel module is needed for userland tracing: fasttrap. No changes to your kernel config file are necessary to enable userland tracing, but you might consider adding 'STRIP=' and 'CFLAGS+=-fno-omit-frame-pointer' to your make.conf if you want to have informative userland stack traces in DTrace (ustack). 20100725: The acpi_aiboost(4) driver has been removed in favor of the new aibs(4) driver. You should update your kernel configuration file. 20100722: BSD grep has been imported to the base system and it is built by default. It is completely BSD licensed, highly GNU-compatible, uses less memory than its GNU counterpart and has a small codebase. However, it is slower than its GNU counterpart, which is mostly noticeable for larger searches, for smaller ones it is measurable but not significant. The reason is complex, the most important factor is that we lack a modern and efficient regex library and GNU overcomes this by optimizing the searches internally. Future work on improving the regex performance is planned, for the meantime, users that need better performance, can build GNU grep instead by setting the WITH_GNU_GREP knob. 20100713: Due to the import of powerpc64 support, all existing powerpc kernel configuration files must be updated with a machine directive like this: machine powerpc powerpc In addition, an updated config(8) is required to build powerpc kernels after this change. 20100713: A new version of ZFS (version 15) has been merged to -HEAD. This version uses a python library for the following subcommands: zfs allow, zfs unallow, zfs groupspace, zfs userspace. For full functionality of these commands the following port must be installed: sysutils/py-zfs 20100429: 'vm_page's are now hashed by physical address to an array of mutexes. Currently this is only used to serialize access to hold_count. Over time the page queue mutex will be peeled away. This changes the size of pmap on every architecture. And requires all callers of vm_page_hold and vm_page_unhold to be updated. 20100402: WITH_CTF can now be specified in src.conf (not recommended, there are some problems with static executables), make.conf (would also affect ports which do not use GNU make and do not override the compile targets) or in the kernel config (via "makeoptions WITH_CTF=yes"). When WITH_CTF was specified there before this was silently ignored, so make sure that WITH_CTF is not used in places which could lead to unwanted behavior. 20100311: The kernel option COMPAT_IA32 has been replaced with COMPAT_FREEBSD32 to allow 32-bit compatibility on non-x86 platforms. All kernel configurations on amd64 and ia64 platforms using these options must be modified accordingly. 20100113: The utmp user accounting database has been replaced with utmpx, the user accounting interface standardized by POSIX. Unfortunately the semantics of utmp and utmpx don't match, making it practically impossible to support both interfaces. The user accounting database is used by tools like finger(1), last(1), talk(1), w(1) and ac(8). All applications in the base system use utmpx. This means only local binaries (e.g. from the ports tree) may still use these utmp database files. These applications must be rebuilt to make use of utmpx. After the system has been upgraded, it is safe to remove the old log files (/var/run/utmp, /var/log/lastlog and /var/log/wtmp*), assuming their contents is of no importance anymore. Old wtmp databases can only be used by last(1) and ac(8) after they have been converted to the new format using wtmpcvt(1). 20100108: Introduce the kernel thread "deadlock resolver" (which can be enabled via the DEADLKRES option, see NOTES for more details) and the sleepq_type() function for sleepqueues. 20091202: The rc.firewall and rc.firewall6 were unified, and rc.firewall6 and rc.d/ip6fw were removed. According to the removal of rc.d/ip6fw, ipv6_firewall_* rc variables are obsoleted. Instead, the following new rc variables are added to rc.d/ipfw: firewall_client_net_ipv6, firewall_simple_iif_ipv6, firewall_simple_inet_ipv6, firewall_simple_oif_ipv6, firewall_simple_onet_ipv6, firewall_trusted_ipv6 The meanings correspond to the relevant IPv4 variables. 20091125: 8.0-RELEASE. 20091113: The default terminal emulation for syscons(4) has been changed from cons25 to xterm on all platforms except pc98. This means that the /etc/ttys file needs to be updated to ensure correct operation of applications on the console. The terminal emulation style can be toggled per window by using vidcontrol(1)'s -T flag. The TEKEN_CONS25 kernel configuration options can be used to change the compile-time default back to cons25. To prevent graphical artifacts, make sure the TERM environment variable is set to match the terminal emulation that is being performed by syscons(4). 20091109: The layout of the structure ieee80211req_scan_result has changed. Applications that require wireless scan results (e.g. ifconfig(8)) from net80211 need to be recompiled. Applications such as wpa_supplicant(8) may require a full world build without using NO_CLEAN in order to get synchronized with the new structure. 20091025: The iwn(4) driver has been updated to support the 5000 and 5150 series. There's one kernel module for each firmware. Adding "device iwnfw" to the kernel configuration file means including all three firmware images inside the kernel. If you want to include just the one for your wireless card, use the devices iwn4965fw, iwn5000fw or iwn5150fw. 20090926: The rc.d/network_ipv6, IPv6 configuration script has been integrated into rc.d/netif. The changes are the following: 1. To use IPv6, simply define $ifconfig_IF_ipv6 like $ifconfig_IF for IPv4. For aliases, $ifconfig_IF_aliasN should be used. Note that both variables need the "inet6" keyword at the head. Do not set $ipv6_network_interfaces manually if you do not understand what you are doing. It is not needed in most cases. $ipv6_ifconfig_IF and $ipv6_ifconfig_IF_aliasN still work, but they are obsolete. 2. $ipv6_enable is obsolete. Use $ipv6_prefer and "inet6 accept_rtadv" keyword in ifconfig(8) instead. If you define $ipv6_enable=YES, it means $ipv6_prefer=YES and all configured interfaces have "inet6 accept_rtadv" in the $ifconfig_IF_ipv6. These are for backward compatibility. 3. A new variable $ipv6_prefer has been added. If NO, IPv6 functionality of interfaces with no corresponding $ifconfig_IF_ipv6 is disabled by using "inet6 ifdisabled" flag, and the default address selection policy of ip6addrctl(8) is the IPv4-preferred one (see rc.d/ip6addrctl for more details). Note that if you want to configure IPv6 functionality on the disabled interfaces after boot, first you need to clear the flag by using ifconfig(8) like: ifconfig em0 inet6 -ifdisabled If YES, the default address selection policy is set as IPv6-preferred. The default value of $ipv6_prefer is NO. 4. If your system need to receive Router Advertisement messages, define "inet6 accept_rtadv" in $ifconfig_IF_ipv6. The rc(8) scripts automatically invoke rtsol(8) when the interface becomes UP. The Router Advertisement messages are used for SLAAC (State-Less Address AutoConfiguration). 20090922: 802.11s D3.03 support was committed. This is incompatible with the previous code, which was based on D3.0. 20090912: A sysctl variable net.inet6.ip6.accept_rtadv now sets the default value of a per-interface flag ND6_IFF_ACCEPT_RTADV, not a global knob to control whether accepting Router Advertisement messages or not. Also, a per-interface flag ND6_IFF_AUTO_LINKLOCAL has been added and a sysctl variable net.inet6.ip6.auto_linklocal is its default value. The ifconfig(8) utility now supports these flags. 20090910: ZFS snapshots are now mounted with MNT_IGNORE flag. Use -v option for mount(8) and -a option for df(1) to see them. 20090825: The old tunable hw.bus.devctl_disable has been superseded by hw.bus.devctl_queue. hw.bus.devctl_disable=1 in loader.conf should be replaced by hw.bus.devctl_queue=0. The default for this new tunable is 1000. 20090813: Remove the option STOP_NMI. The default action is now to use NMI only for KDB via the newly introduced function stop_cpus_hard() and maintain stop_cpus() to just use a normal IPI_STOP on ia32 and amd64. 20090803: The stable/8 branch created in subversion. This corresponds to the RELENG_8 branch in CVS. 20090719: Bump the shared library version numbers for all libraries that do not use symbol versioning as part of the 8.0-RELEASE cycle. Bump __FreeBSD_version to 800105. 20090714: Due to changes in the implementation of virtual network stack support, all network-related kernel modules must be recompiled. As this change breaks the ABI, bump __FreeBSD_version to 800104. 20090713: The TOE interface to the TCP syncache has been modified to remove struct tcpopt () from the ABI of the network stack. The cxgb driver is the only TOE consumer affected by this change, and needs to be recompiled along with the kernel. As this change breaks the ABI, bump __FreeBSD_version to 800103. 20090712: Padding has been added to struct tcpcb, sackhint and tcpstat in to facilitate future MFCs and bug fixes whilst maintaining the ABI. However, this change breaks the ABI, so bump __FreeBSD_version to 800102. User space tools that rely on the size of any of these structs (e.g. sockstat) need to be recompiled. 20090630: The NFS_LEGACYRPC option has been removed along with the old kernel RPC implementation that this option selected. Kernel configurations may need to be adjusted. 20090629: The network interface device nodes at /dev/net/ have been removed. All ioctl operations can be performed the normal way using routing sockets. The kqueue functionality can generally be replaced with routing sockets. 20090628: The documentation from the FreeBSD Documentation Project (Handbook, FAQ, etc.) is now installed via packages by sysinstall(8) and under the /usr/local/share/doc/freebsd directory instead of /usr/share/doc. 20090624: The ABI of various structures related to the SYSV IPC API have been changed. As a result, the COMPAT_FREEBSD[456] and COMPAT_43 kernel options now all require COMPAT_FREEBSD7. Bump __FreeBSD_version to 800100. 20090622: Layout of struct vnet has changed as routing related variables were moved to their own Vimage module. Modules need to be recompiled. Bump __FreeBSD_version to 800099. 20090619: NGROUPS_MAX and NGROUPS have been increased from 16 to 1023 and 1024 respectively. As long as no more than 16 groups per process are used, no changes should be visible. When more than 16 groups are used, old binaries may fail if they call getgroups() or getgrouplist() with statically sized storage. Recompiling will work around this, but applications should be modified to use dynamically allocated storage for group arrays as POSIX.1-2008 does not cap an implementation's number of supported groups at NGROUPS_MAX+1 as previous versions did. NFS and portalfs mounts may also be affected as the list of groups is truncated to 16. Users of NFS who use more than 16 groups, should take care that negative group permissions are not used on the exported file systems as they will not be reliable unless a GSSAPI based authentication method is used. 20090616: The compiling option ADAPTIVE_LOCKMGRS has been introduced. This option compiles in the support for adaptive spinning for lockmgrs which want to enable it. The lockinit() function now accepts the flag LK_ADAPTIVE in order to make the lock object subject to adaptive spinning when both held in write and read mode. 20090613: The layout of the structure returned by IEEE80211_IOC_STA_INFO has changed. User applications that use this ioctl need to be rebuilt. 20090611: The layout of struct thread has changed. Kernel and modules need to be rebuilt. 20090608: The layout of structs ifnet, domain, protosw and vnet_net has changed. Kernel modules need to be rebuilt. Bump __FreeBSD_version to 800097. 20090602: window(1) has been removed from the base system. It can now be installed from ports. The port is called misc/window. 20090601: The way we are storing and accessing `routing table' entries has changed. Programs reading the FIB, like netstat, need to be re-compiled. 20090601: A new netisr implementation has been added for FreeBSD 8. Network file system modules, such as igmp, ipdivert, and others, should be rebuilt. Bump __FreeBSD_version to 800096. 20090530: Remove the tunable/sysctl debug.mpsafevfs as its initial purpose is no more valid. 20090530: Add VOP_ACCESSX(9). File system modules need to be rebuilt. Bump __FreeBSD_version to 800094. 20090529: Add mnt_xflag field to 'struct mount'. File system modules need to be rebuilt. Bump __FreeBSD_version to 800093. 20090528: The compiling option ADAPTIVE_SX has been retired while it has been introduced the option NO_ADAPTIVE_SX which handles the reversed logic. The KPI for sx_init_flags() changes as accepting flags: SX_ADAPTIVESPIN flag has been retired while the SX_NOADAPTIVE flag has been introduced in order to handle the reversed logic. Bump __FreeBSD_version to 800092. 20090527: Add support for hierarchical jails. Remove global securelevel. Bump __FreeBSD_version to 800091. 20090523: The layout of struct vnet_net has changed, therefore modules need to be rebuilt. Bump __FreeBSD_version to 800090. 20090523: The newly imported zic(8) produces a new format in the output. Please run tzsetup(8) to install the newly created data to /etc/localtime. 20090520: The sysctl tree for the usb stack has renamed from hw.usb2.* to hw.usb.* and is now consistent again with previous releases. 20090520: 802.11 monitor mode support was revised and driver api's were changed. Drivers dependent on net80211 now support DLT_IEEE802_11_RADIO instead of DLT_IEEE802_11. No user-visible data structures were changed but applications that use DLT_IEEE802_11 may require changes. Bump __FreeBSD_version to 800088. 20090430: The layout of the following structs has changed: sysctl_oid, socket, ifnet, inpcbinfo, tcpcb, syncache_head, vnet_inet, vnet_inet6 and vnet_ipfw. Most modules need to be rebuild or panics may be experienced. World rebuild is required for correctly checking networking state from userland. Bump __FreeBSD_version to 800085. 20090429: MLDv2 and Source-Specific Multicast (SSM) have been merged to the IPv6 stack. VIMAGE hooks are in but not yet used. The implementation of SSM within FreeBSD's IPv6 stack closely follows the IPv4 implementation. For kernel developers: * The most important changes are that the ip6_output() and ip6_input() paths no longer take the IN6_MULTI_LOCK, and this lock has been downgraded to a non-recursive mutex. * As with the changes to the IPv4 stack to support SSM, filtering of inbound multicast traffic must now be performed by transport protocols within the IPv6 stack. This does not apply to TCP and SCTP, however, it does apply to UDP in IPv6 and raw IPv6. * The KPIs used by IPv6 multicast are similar to those used by the IPv4 stack, with the following differences: * im6o_mc_filter() is analogous to imo_multicast_filter(). * The legacy KAME entry points in6_joingroup and in6_leavegroup() are shimmed to in6_mc_join() and in6_mc_leave() respectively. * IN6_LOOKUP_MULTI() has been deprecated and removed. * IPv6 relies on MLD for the DAD mechanism. KAME's internal KPIs for MLDv1 have an additional 'timer' argument which is used to jitter the initial membership report for the solicited-node multicast membership on-link. * This is not strictly needed for MLDv2, which already jitters its report transmissions. However, the 'timer' argument is preserved in case MLDv1 is active on the interface. * The KAME linked-list based IPv6 membership implementation has been refactored to use a vector similar to that used by the IPv4 stack. Code which maintains a list of its own multicast memberships internally, e.g. carp, has been updated to reflect the new semantics. * There is a known Lock Order Reversal (LOR) due to in6_setscope() acquiring the IF_AFDATA_LOCK and being called within ip6_output(). Whilst MLDv2 tries to avoid this otherwise benign LOR, it is an implementation constraint which needs to be addressed in HEAD. For application developers: * The changes are broadly similar to those made for the IPv4 stack. * The use of IPv4 and IPv6 multicast socket options on the same socket, using mapped addresses, HAS NOT been tested or supported. * There are a number of issues with the implementation of various IPv6 multicast APIs which need to be resolved in the API surface before the implementation is fully compatible with KAME userland use, and these are mostly to do with interface index treatment. * The literature available discusses the use of either the delta / ASM API with setsockopt(2)/getsockopt(2), or the full-state / ASM API using setsourcefilter(3)/getsourcefilter(3). For more information please refer to RFC 3768, 'Socket Interface Extensions for Multicast Source Filters'. * Applications which use the published RFC 3678 APIs should be fine. For systems administrators: * The mtest(8) utility has been refactored to support IPv6, in addition to IPv4. Interface addresses are no longer accepted as arguments, their names must be used instead. The utility will map the interface name to its first IPv4 address as returned by getifaddrs(3). * The ifmcstat(8) utility has also been updated to print the MLDv2 endpoint state and source filter lists via sysctl(3). * The net.inet6.ip6.mcast.loop sysctl may be tuned to 0 to disable loopback of IPv6 multicast datagrams by default; it defaults to 1 to preserve the existing behaviour. Disabling multicast loopback is recommended for optimal system performance. * The IPv6 MROUTING code has been changed to examine this sysctl instead of attempting to perform a group lookup before looping back forwarded datagrams. Bump __FreeBSD_version to 800084. 20090422: Implement low-level Bluetooth HCI API. Bump __FreeBSD_version to 800083. 20090419: The layout of struct malloc_type, used by modules to register new memory allocation types, has changed. Most modules will need to be rebuilt or panics may be experienced. Bump __FreeBSD_version to 800081. 20090415: Anticipate overflowing inp_flags - add inp_flags2. This changes most offsets in inpcb, so checking v4 connection state will require a world rebuild. Bump __FreeBSD_version to 800080. 20090415: Add an llentry to struct route and struct route_in6. Modules embedding a struct route will need to be recompiled. Bump __FreeBSD_version to 800079. 20090414: The size of rt_metrics_lite and by extension rtentry has changed. Networking administration apps will need to be recompiled. The route command now supports show as an alias for get, weighting of routes, sticky and nostick flags to alter the behavior of stateful load balancing. Bump __FreeBSD_version to 800078. 20090408: Do not use Giant for kbdmux(4) locking. This is wrong and apparently causing more problems than it solves. This will re-open the issue where interrupt handlers may race with kbdmux(4) in polling mode. Typical symptoms include (but not limited to) duplicated and/or missing characters when low level console functions (such as gets) are used while interrupts are enabled (for example geli password prompt, mountroot prompt etc.). Disabling kbdmux(4) may help. 20090407: The size of structs vnet_net, vnet_inet and vnet_ipfw has changed; kernel modules referencing any of the above need to be recompiled. Bump __FreeBSD_version to 800075. 20090320: GEOM_PART has become the default partition slicer for storage devices, replacing GEOM_MBR, GEOM_BSD, GEOM_PC98 and GEOM_GPT slicers. It introduces some changes: MSDOS/EBR: the devices created from MSDOS extended partition entries (EBR) can be named differently than with GEOM_MBR and are now symlinks to devices with offset-based names. fstabs may need to be modified. BSD: the "geometry does not match label" warning is harmless in most cases but it points to problems in file system misalignment with disk geometry. The "c" partition is now implicit, covers the whole top-level drive and cannot be (mis)used by users. General: Kernel dumps are now not allowed to be written to devices whose partition types indicate they are meant to be used for file systems (or, in case of MSDOS partitions, as something else than the "386BSD" type). Most of these changes date approximately from 200812. 20090319: The uscanner(4) driver has been removed from the kernel. This follows Linux removing theirs in 2.6 and making libusb the default interface (supported by sane). 20090319: The multicast forwarding code has been cleaned up. netstat(1) only relies on KVM now for printing bandwidth upcall meters. The IPv4 and IPv6 modules are split into ip_mroute_mod and ip6_mroute_mod respectively. The config(5) options for statically compiling this code remain the same, i.e. 'options MROUTING'. 20090315: Support for the IFF_NEEDSGIANT network interface flag has been removed, which means that non-MPSAFE network device drivers are no longer supported. In particular, if_ar, if_sr, and network device drivers from the old (legacy) USB stack can no longer be built or used. 20090313: POSIX.1 Native Language Support (NLS) has been enabled in libc and a bunch of new language catalog files have also been added. This means that some common libc messages are now localized and they depend on the LC_MESSAGES environmental variable. 20090313: The k8temp(4) driver has been renamed to amdtemp(4) since support for Family 10 and Family 11 CPU families was added. 20090309: IGMPv3 and Source-Specific Multicast (SSM) have been merged to the IPv4 stack. VIMAGE hooks are in but not yet used. For kernel developers, the most important changes are that the ip_output() and ip_input() paths no longer take the IN_MULTI_LOCK(), and this lock has been downgraded to a non-recursive mutex. Transport protocols (UDP, Raw IP) are now responsible for filtering inbound multicast traffic according to group membership and source filters. The imo_multicast_filter() KPI exists for this purpose. Transports which do not use multicast (SCTP, TCP) already reject multicast by default. Forwarding and receive performance may improve as a mutex acquisition is no longer needed in the ip_input() low-level input path. in_addmulti() and in_delmulti() are shimmed to new KPIs which exist to support SSM in-kernel. For application developers, it is recommended that loopback of multicast datagrams be disabled for best performance, as this will still cause the lock to be taken for each looped-back datagram transmission. The net.inet.ip.mcast.loop sysctl may be tuned to 0 to disable loopback by default; it defaults to 1 to preserve the existing behaviour. For systems administrators, to obtain best performance with multicast reception and multiple groups, it is always recommended that a card with a suitably precise hash filter is used. Hash collisions will still result in the lock being taken within the transport protocol input path to check group membership. If deploying FreeBSD in an environment with IGMP snooping switches, it is recommended that the net.inet.igmp.sendlocal sysctl remain enabled; this forces 224.0.0.0/24 group membership to be announced via IGMP. The size of 'struct igmpstat' has changed; netstat needs to be recompiled to reflect this. Bump __FreeBSD_version to 800070. 20090309: libusb20.so.1 is now installed as libusb.so.1 and the ports system updated to use it. This requires a buildworld/installworld in order to update the library and dependencies (usbconfig, etc). Its advisable to rebuild all ports which uses libusb. More specific directions are given in the ports collection UPDATING file. Any /etc/libmap.conf entries for libusb are no longer required and can be removed. 20090302: A workaround is committed to allow the creation of System V shared memory segment of size > 2 GB on the 64-bit architectures. Due to a limitation of the existing ABI, the shm_segsz member of the struct shmid_ds, returned by shmctl(IPC_STAT) call is wrong for large segments. Note that limits must be explicitly raised to allow such segments to be created. 20090301: The layout of struct ifnet has changed, requiring a rebuild of all network device driver modules. 20090227: The /dev handling for the new USB stack has changed, a buildworld/installworld is required for libusb20. 20090223: The new USB2 stack has now been permanently moved in and all kernel and module names reverted to their previous values (eg, usb, ehci, ohci, ums, ...). The old usb stack can be compiled in by prefixing the name with the letter 'o', the old usb modules have been removed. Updating entry 20090216 for xorg and 20090215 for libmap may still apply. 20090217: The rc.conf(5) option if_up_delay has been renamed to defaultroute_delay to better reflect its purpose. If you have customized this setting in /etc/rc.conf you need to update it to use the new name. 20090216: xorg 7.4 wants to configure its input devices via hald which does not yet work with USB2. If the keyboard/mouse does not work in xorg then add Option "AllowEmptyInput" "off" to your ServerLayout section. This will cause X to use the configured kbd and mouse sections from your xorg.conf. 20090215: The GENERIC kernels for all architectures now default to the new USB2 stack. No kernel config options or code have been removed so if a problem arises please report it and optionally revert to the old USB stack. If you are loading USB kernel modules or have a custom kernel that includes GENERIC then ensure that usb names are also changed over, eg uftdi -> usb2_serial_ftdi. Older programs linked against the ports libusb 0.1 need to be redirected to the new stack's libusb20. /etc/libmap.conf can be used for this: # Map old usb library to new one for usb2 stack libusb-0.1.so.8 libusb20.so.1 20090209: All USB ethernet devices now attach as interfaces under the name ueN (eg. ue0). This is to provide a predictable name as vendors often change usb chipsets in a product without notice. 20090203: The ichsmb(4) driver has been changed to require SMBus slave addresses be left-justified (xxxxxxx0b) rather than right-justified. All of the other SMBus controller drivers require left-justified slave addresses, so this change makes all the drivers provide the same interface. 20090201: INET6 statistics (struct ip6stat) was updated. netstat(1) needs to be recompiled. 20090119: NTFS has been removed from GENERIC kernel on amd64 to match GENERIC on i386. Should not cause any issues since mount_ntfs(8) will load ntfs.ko module automatically when NTFS support is actually needed, unless ntfs.ko is not installed or security level prohibits loading kernel modules. If either is the case, "options NTFS" has to be added into kernel config. 20090115: TCP Appropriate Byte Counting (RFC 3465) support added to kernel. New field in struct tcpcb breaks ABI, so bump __FreeBSD_version to 800061. User space tools that rely on the size of struct tcpcb in tcp_var.h (e.g. sockstat) need to be recompiled. 20081225: ng_tty(4) module updated to match the new TTY subsystem. Due to API change, user-level applications must be updated. New API support added to mpd5 CVS and expected to be present in next mpd5.3 release. 20081219: With __FreeBSD_version 800060 the makefs tool is part of the base system (it was a port). 20081216: The afdata and ifnet locks have been changed from mutexes to rwlocks, network modules will need to be re-compiled. 20081214: __FreeBSD_version 800059 incorporates the new arp-v2 rewrite. RTF_CLONING, RTF_LLINFO and RTF_WASCLONED flags are eliminated. The new code reduced struct rtentry{} by 16 bytes on 32-bit architecture and 40 bytes on 64-bit architecture. The userland applications "arp" and "ndp" have been updated accordingly. The output from "netstat -r" shows only routing entries and none of the L2 information. 20081130: __FreeBSD_version 800057 marks the switchover from the binary ath hal to source code. Users must add the line: options AH_SUPPORT_AR5416 to their kernel config files when specifying: device ath_hal The ath_hal module no longer exists; the code is now compiled together with the driver in the ath module. It is now possible to tailor chip support (i.e. reduce the set of chips and thereby the code size); consult ath_hal(4) for details. 20081121: __FreeBSD_version 800054 adds memory barriers to , new interfaces to ifnet to facilitate multiple hardware transmit queues for cards that support them, and a lock-less ring-buffer implementation to enable drivers to more efficiently manage queueing of packets. 20081117: A new version of ZFS (version 13) has been merged to -HEAD. This version has zpool attribute "listsnapshots" off by default, which means "zfs list" does not show snapshots, and is the same as Solaris behavior. 20081028: dummynet(4) ABI has changed. ipfw(8) needs to be recompiled. 20081009: The uhci, ohci, ehci and slhci USB Host controller drivers have been put into separate modules. If you load the usb module separately through loader.conf you will need to load the appropriate *hci module as well. E.g. for a UHCI-based USB 2.0 controller add the following to loader.conf: uhci_load="YES" ehci_load="YES" 20081009: The ABI used by the PMC toolset has changed. Please keep userland (libpmc(3)) and the kernel module (hwpmc(4)) in sync. 20081009: atapci kernel module now includes only generic PCI ATA driver. AHCI driver moved to ataahci kernel module. All vendor-specific code moved into separate kernel modules: ataacard, ataacerlabs, ataadaptec, ataamd, ataati, atacenatek, atacypress, atacyrix, atahighpoint, ataintel, ataite, atajmicron, atamarvell, atamicron, atanational, atanetcell, atanvidia, atapromise, ataserverworks, atasiliconimage, atasis, atavia 20080820: The TTY subsystem of the kernel has been replaced by a new implementation, which provides better scalability and an improved driver model. Most common drivers have been migrated to the new TTY subsystem, while others have not. The following drivers have not yet been ported to the new TTY layer: PCI/ISA: cy, digi, rc, rp, sio USB: ubser, ucycom Line disciplines: ng_h4, ng_tty, ppp, sl, snp Adding these drivers to your kernel configuration file shall cause compilation to fail. 20080818: ntpd has been upgraded to 4.2.4p5. 20080801: OpenSSH has been upgraded to 5.1p1. For many years, FreeBSD's version of OpenSSH preferred DSA over RSA for host and user authentication keys. With this upgrade, we've switched to the vendor's default of RSA over DSA. This may cause upgraded clients to warn about unknown host keys even for previously known hosts. Users should follow the usual procedure for verifying host keys before accepting the RSA key. This can be circumvented by setting the "HostKeyAlgorithms" option to "ssh-dss,ssh-rsa" in ~/.ssh/config or on the ssh command line. Please note that the sequence of keys offered for authentication has been changed as well. You may want to specify IdentityFile in a different order to revert this behavior. 20080713: The sio(4) driver has been removed from the i386 and amd64 kernel configuration files. This means uart(4) is now the default serial port driver on those platforms as well. To prevent collisions with the sio(4) driver, the uart(4) driver uses different names for its device nodes. This means the onboard serial port will now most likely be called "ttyu0" instead of "ttyd0". You may need to reconfigure applications to use the new device names. When using the serial port as a boot console, be sure to update /boot/device.hints and /etc/ttys before booting the new kernel. If you forget to do so, you can still manually specify the hints at the loader prompt: set hint.uart.0.at="isa" set hint.uart.0.port="0x3F8" set hint.uart.0.flags="0x10" set hint.uart.0.irq="4" boot -s 20080609: The gpt(8) utility has been removed. Use gpart(8) to partition disks instead. 20080603: The version that Linuxulator emulates was changed from 2.4.2 to 2.6.16. If you experience any problems with Linux binaries please try to set sysctl compat.linux.osrelease to 2.4.2 and if it fixes the problem contact emulation mailing list. 20080525: ISDN4BSD (I4B) was removed from the src tree. You may need to update a your kernel configuration and remove relevant entries. 20080509: I have checked in code to support multiple routing tables. See the man pages setfib(1) and setfib(2). This is a hopefully backwards compatible version, but to make use of it you need to compile your kernel with options ROUTETABLES=2 (or more up to 16). 20080420: The 802.11 wireless support was redone to enable multi-bss operation on devices that are capable. The underlying device is no longer used directly but instead wlanX devices are cloned with ifconfig. This requires changes to rc.conf files. For example, change: ifconfig_ath0="WPA DHCP" to wlans_ath0=wlan0 ifconfig_wlan0="WPA DHCP" see rc.conf(5) for more details. In addition, mergemaster of /etc/rc.d is highly recommended. Simultaneous update of userland and kernel wouldn't hurt either. As part of the multi-bss changes the wlan_scan_ap and wlan_scan_sta modules were merged into the base wlan module. All references to these modules (e.g. in kernel config files) must be removed. 20080408: psm(4) has gained write(2) support in native operation level. Arbitrary commands can be written to /dev/psm%d and status can be read back from it. Therefore, an application is responsible for status validation and error recovery. It is a no-op in other operation levels. 20080312: Support for KSE threading has been removed from the kernel. To run legacy applications linked against KSE libmap.conf may be used. The following libmap.conf may be used to ensure compatibility with any prior release: libpthread.so.1 libthr.so.1 libpthread.so.2 libthr.so.2 libkse.so.3 libthr.so.3 20080301: The layout of struct vmspace has changed. This affects libkvm and any executables that link against libkvm and use the kvm_getprocs() function. In particular, but not exclusively, it affects ps(1), fstat(1), pkill(1), systat(1), top(1) and w(1). The effects are minimal, but it's advisable to upgrade world nonetheless. 20080229: The latest em driver no longer has support in it for the 82575 adapter, this is now moved to the igb driver. The split was done to make new features that are incompatible with older hardware easier to do. 20080220: The new geom_lvm(4) geom class has been renamed to geom_linux_lvm(4), likewise the kernel option is now GEOM_LINUX_LVM. 20080211: The default NFS mount mode has changed from UDP to TCP for increased reliability. If you rely on (insecurely) NFS mounting across a firewall you may need to update your firewall rules. 20080208: Belatedly note the addition of m_collapse for compacting mbuf chains. 20080126: The fts(3) structures have been changed to use adequate integer types for their members and so to be able to cope with huge file trees. The old fts(3) ABI is preserved through symbol versioning in libc, so third-party binaries using fts(3) should still work, although they will not take advantage of the extended types. At the same time, some third-party software might fail to build after this change due to unportable assumptions made in its source code about fts(3) structure members. Such software should be fixed by its vendor or, in the worst case, in the ports tree. FreeBSD_version 800015 marks this change for the unlikely case that a portable fix is impossible. 20080123: To upgrade to -current after this date, you must be running FreeBSD not older than 6.0-RELEASE. Upgrading to -current from 5.x now requires a stop over at RELENG_6 or RELENG_7 systems. 20071128: The ADAPTIVE_GIANT kernel option has been retired because its functionality is the default now. 20071118: The AT keyboard emulation of sunkbd(4) has been turned on by default. In order to make the special symbols of the Sun keyboards driven by sunkbd(4) work under X these now have to be configured the same way as Sun USB keyboards driven by ukbd(4) (which also does AT keyboard emulation), f.e.: Option "XkbLayout" "us" Option "XkbRules" "xorg" Option "XkbSymbols" "pc(pc105)+sun_vndr/usb(sun_usb)+us" 20071024: It has been decided that it is desirable to provide ABI backwards compatibility to the FreeBSD 4/5/6 versions of the PCIOCGETCONF, PCIOCREAD and PCIOCWRITE IOCTLs, which was broken with the introduction of PCI domain support (see the 20070930 entry). Unfortunately, this required the ABI of PCIOCGETCONF to be broken again in order to be able to provide backwards compatibility to the old version of that IOCTL. Thus consumers of PCIOCGETCONF have to be recompiled again. As for prominent ports this affects neither pciutils nor xorg-server this time, the hal port needs to be rebuilt however. 20071020: The misnamed kthread_create() and friends have been renamed to kproc_create() etc. Many of the callers already used kproc_start().. I will return kthread_create() and friends in a while with implementations that actually create threads, not procs. Renaming corresponds with version 800002. 20071010: RELENG_7 branched. COMMON ITEMS: General Notes ------------- Avoid using make -j when upgrading. While generally safe, there are sometimes problems using -j to upgrade. If your upgrade fails with -j, please try again without -j. From time to time in the past there have been problems using -j with buildworld and/or installworld. This is especially true when upgrading between "distant" versions (eg one that cross a major release boundary or several minor releases, or when several months have passed on the -current branch). Sometimes, obscure build problems are the result of environment poisoning. This can happen because the make utility reads its environment when searching for values for global variables. To run your build attempts in an "environmental clean room", prefix all make commands with 'env -i '. See the env(1) manual page for more details. When upgrading from one major version to another it is generally best to upgrade to the latest code in the currently installed branch first, then do an upgrade to the new branch. This is the best-tested upgrade path, and has the highest probability of being successful. Please try this approach before reporting problems with a major version upgrade. When upgrading a live system, having a root shell around before installing anything can help undo problems. Not having a root shell around can lead to problems if pam has changed too much from your starting point to allow continued authentication after the upgrade. ZFS notes --------- When upgrading the boot ZFS pool to a new version, always follow these two steps: 1.) recompile and reinstall the ZFS boot loader and boot block (this is part of "make buildworld" and "make installworld") 2.) update the ZFS boot block on your boot drive The following example updates the ZFS boot block on the first partition (freebsd-boot) of a GPT partitioned drive ad0: "gpart bootcode -p /boot/gptzfsboot -i 1 ad0" Non-boot pools do not need these updates. To build a kernel ----------------- If you are updating from a prior version of FreeBSD (even one just a few days old), you should follow this procedure. It is the most failsafe as it uses a /usr/obj tree with a fresh mini-buildworld, make kernel-toolchain make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE To test a kernel once --------------------- If you just want to boot a kernel once (because you are not sure if it works, or if you want to boot a known bad kernel to provide debugging information) run make installkernel KERNCONF=YOUR_KERNEL_HERE KODIR=/boot/testkernel nextboot -k testkernel To just build a kernel when you know that it won't mess you up -------------------------------------------------------------- This assumes you are already running a CURRENT system. Replace ${arch} with the architecture of your machine (e.g. "i386", "arm", "amd64", "ia64", "pc98", "sparc64", "powerpc", "mips", etc). cd src/sys/${arch}/conf config KERNEL_NAME_HERE cd ../compile/KERNEL_NAME_HERE make depend make make install If this fails, go to the "To build a kernel" section. To rebuild everything and install it on the current system. ----------------------------------------------------------- # Note: sometimes if you are running current you gotta do more than # is listed here if you are upgrading from a really old current. make buildworld make kernel KERNCONF=YOUR_KERNEL_HERE [1] [3] mergemaster -p [5] make installworld mergemaster -i [4] make delete-old [6] To cross-install current onto a separate partition -------------------------------------------------- # In this approach we use a separate partition to hold # current's root, 'usr', and 'var' directories. A partition # holding "/", "/usr" and "/var" should be about 2GB in # size. make buildworld make buildkernel KERNCONF=YOUR_KERNEL_HERE make installworld DESTDIR=${CURRENT_ROOT} make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd make installkernel KERNCONF=YOUR_KERNEL_HERE DESTDIR=${CURRENT_ROOT} cp /etc/fstab ${CURRENT_ROOT}/etc/fstab # if newfs'd To upgrade in-place from stable to current ---------------------------------------------- make buildworld [9] make kernel KERNCONF=YOUR_KERNEL_HERE [8] [1] [3] mergemaster -p [5] make installworld mergemaster -i [4] make delete-old [6] Make sure that you've read the UPDATING file to understand the tweaks to various things you need. At this point in the life cycle of current, things change often and you are on your own to cope. The defaults can also change, so please read ALL of the UPDATING entries. Also, if you are tracking -current, you must be subscribed to freebsd-current@freebsd.org. Make sure that before you update your sources that you have read and understood all the recent messages there. If in doubt, please track -stable which has much fewer pitfalls. [1] If you have third party modules, such as vmware, you should disable them at this point so they don't crash your system on reboot. [3] From the bootblocks, boot -s, and then do fsck -p mount -u / mount -a cd src adjkerntz -i # if CMOS is wall time Also, when doing a major release upgrade, it is required that you boot into single user mode to do the installworld. [4] Note: This step is non-optional. Failure to do this step can result in a significant reduction in the functionality of the system. Attempting to do it by hand is not recommended and those that pursue this avenue should read this file carefully, as well as the archives of freebsd-current and freebsd-hackers mailing lists for potential gotchas. The -U option is also useful to consider. See mergemaster(8) for more information. [5] Usually this step is a noop. However, from time to time you may need to do this if you get unknown user in the following step. It never hurts to do it all the time. You may need to install a new mergemaster (cd src/usr.sbin/mergemaster && make install) after the buildworld before this step if you last updated from current before 20130425 or from -stable before 20130430. [6] This only deletes old files and directories. Old libraries can be deleted by "make delete-old-libs", but you have to make sure that no program is using those libraries anymore. [8] In order to have a kernel that can run the 4.x binaries needed to do an installworld, you must include the COMPAT_FREEBSD4 option in your kernel. Failure to do so may leave you with a system that is hard to boot to recover. A similar kernel option COMPAT_FREEBSD5 is required to run the 5.x binaries on more recent kernels. And so on for COMPAT_FREEBSD6 and COMPAT_FREEBSD7. Make sure that you merge any new devices from GENERIC since the last time you updated your kernel config file. [9] When checking out sources, you must include the -P flag to have cvs prune empty directories. If CPUTYPE is defined in your /etc/make.conf, make sure to use the "?=" instead of the "=" assignment operator, so that buildworld can override the CPUTYPE if it needs to. MAKEOBJDIRPREFIX must be defined in an environment variable, and not on the command line, or in /etc/make.conf. buildworld will warn if it is improperly defined. FORMAT: This file contains a list, in reverse chronological order, of major breakages in tracking -current. It is not guaranteed to be a complete list of such breakages, and only contains entries since October 10, 2007. If you need to see UPDATING entries from before that date, you will need to fetch an UPDATING file from an older FreeBSD release. Copyright information: Copyright 1998-2009 M. Warner Losh. All Rights Reserved. Redistribution, publication, translation and use, with or without modification, in full or in part, in any form or format of this document are permitted without further permission from the author. THIS DOCUMENT IS PROVIDED BY WARNER LOSH ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WARNER LOSH BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Contact Warner Losh if you have any questions about your use of this document. $FreeBSD$ Index: stable/10/bin/csh/config.h =================================================================== --- stable/10/bin/csh/config.h (revision 282274) +++ stable/10/bin/csh/config.h (revision 282275) @@ -1,277 +1,277 @@ /* $FreeBSD$ */ /* config.h. Generated from config.h.in by configure. */ /* config.h.in. Generated from configure.in by autoheader. */ /* Define to the type of elements in the array set by `getgroups'. Usually this is either `int' or `gid_t'. */ #define GETGROUPS_T gid_t /* Define to 1 if the `getpgrp' function requires zero arguments. */ #define GETPGRP_VOID 1 /* Define to 1 if you have the header file. */ /* #undef HAVE_AUTH_H */ /* Define to 1 if you have the header file. */ /* #undef HAVE_CRYPT_H */ /* Define to 1 if you have the declaration of `crypt', and to 0 if you don't. */ #define HAVE_DECL_CRYPT 1 /* Define to 1 if you have the declaration of `environ', and to 0 if you don't. */ #define HAVE_DECL_ENVIRON 0 /* Define to 1 if you have the declaration of `gethostname', and to 0 if you don't. */ #define HAVE_DECL_GETHOSTNAME 1 /* Define to 1 if you have the declaration of `getpgrp', and to 0 if you don't. */ #define HAVE_DECL_GETPGRP 1 /* Define to 1 if you have the header file, and it defines `DIR'. */ #define HAVE_DIRENT_H 1 /* Define to 1 if you have the `dup2' function. */ #define HAVE_DUP2 1 /* Define to 1 if you have the header file. */ /* #undef HAVE_FEATURES_H */ /* Define to 1 if you have the `getauthid' function. */ /* #undef HAVE_GETAUTHID */ /* Define to 1 if you have the `getcwd' function. */ #define HAVE_GETCWD 1 /* Define to 1 if you have the `gethostname' function. */ #define HAVE_GETHOSTNAME 1 /* Define to 1 if you have the `getpwent' function. */ #define HAVE_GETPWENT 1 /* Define to 1 if you have the `getutent' function. */ /* #undef HAVE_GETUTENT */ /* Define to 1 if you have the `getutxent' function. */ #define HAVE_GETUTXENT 1 /* Define if you have the iconv() function and it works. */ /* #undef HAVE_ICONV */ /* Define to 1 if you have the header file. */ #define HAVE_INTTYPES_H 1 /* Define to 1 if the system has the type `long long'. */ #define HAVE_LONG_LONG 1 /* Define to 1 if you have the `mallinfo' function. */ /* #undef HAVE_MALLINFO */ /* Define to 1 if mbrtowc and mbstate_t are properly declared. */ #define HAVE_MBRTOWC 1 /* Define to 1 if you have the `memmove' function. */ #define HAVE_MEMMOVE 1 /* Define to 1 if you have the header file. */ #define HAVE_MEMORY_H 1 /* Define to 1 if you have the `memset' function. */ #define HAVE_MEMSET 1 /* Define to 1 if you have the `mkstemp' function. */ #define HAVE_MKSTEMP 1 /* Define to 1 if you have the header file, and it defines `DIR'. */ /* #undef HAVE_NDIR_H */ /* Define to 1 if you have the `nice' function. */ #define HAVE_NICE 1 /* Define to 1 if you have the `nl_langinfo' function. */ #define HAVE_NL_LANGINFO 1 /* Define to 1 if you have the header file. */ #define HAVE_PATHS_H 1 /* Define to 1 if you have the `sbrk' function. */ #define HAVE_SBRK 1 /* Define to 1 if you have the `setpgid' function. */ #define HAVE_SETPGID 1 /* Define to 1 if you have the `setpriority' function. */ #define HAVE_SETPRIORITY 1 /* Define to 1 if you have the header file. */ /* #undef HAVE_SHADOW_H */ /* Define to 1 if you have the header file. */ #define HAVE_STDINT_H 1 /* Define to 1 if you have the header file. */ #define HAVE_STDLIB_H 1 /* Define to 1 if you have the `strcoll' function and it is properly defined. */ #define HAVE_STRCOLL 1 /* Define to 1 if you have the `strerror' function. */ #define HAVE_STRERROR 1 /* Define to 1 if you have the header file. */ #define HAVE_STRINGS_H 1 /* Define to 1 if you have the header file. */ #define HAVE_STRING_H 1 /* Define to 1 if you have the `strstr' function. */ #define HAVE_STRSTR 1 /* Define to 1 if `d_ino' is a member of `struct dirent'. */ #define HAVE_STRUCT_DIRENT_D_INO 1 /* Define to 1 if `ss_family' is a member of `struct sockaddr_storage'. */ #define HAVE_STRUCT_SOCKADDR_STORAGE_SS_FAMILY 1 /* Define to 1 if `ut_host' is a member of `struct utmpx'. */ #define HAVE_STRUCT_UTMPX_UT_HOST 1 /* Define to 1 if `ut_tv' is a member of `struct utmpx'. */ #define HAVE_STRUCT_UTMPX_UT_TV 1 /* Define to 1 if `ut_user' is a member of `struct utmpx'. */ #define HAVE_STRUCT_UTMPX_UT_USER 1 /* Define to 1 if `ut_xtime' is a member of `struct utmpx'. */ /* #undef HAVE_STRUCT_UTMPX_UT_XTIME */ /* Define to 1 if `ut_host' is a member of `struct utmp'. */ #define HAVE_STRUCT_UTMP_UT_HOST 1 /* Define to 1 if `ut_tv' is a member of `struct utmp'. */ #define HAVE_STRUCT_UTMP_UT_TV 1 /* Define to 1 if `ut_user' is a member of `struct utmp'. */ #define HAVE_STRUCT_UTMP_UT_USER 1 /* Define to 1 if `ut_xtime' is a member of `struct utmp'. */ /* #undef HAVE_STRUCT_UTMP_UT_XTIME */ /* Define to 1 if you have the `sysconf' function. */ #define HAVE_SYSCONF 1 /* Define to 1 if you have the header file, and it defines `DIR'. */ /* #undef HAVE_SYS_DIR_H */ /* Define to 1 if you have the header file, and it defines `DIR'. */ /* #undef HAVE_SYS_NDIR_H */ /* Define to 1 if you have the header file. */ #define HAVE_SYS_STAT_H 1 /* Define to 1 if you have the header file. */ #define HAVE_SYS_TYPES_H 1 /* Define to 1 if you have the header file. */ #define HAVE_UNISTD_H 1 /* Define to 1 if you have the header file. */ #define HAVE_UTMPX_H 1 /* Define to 1 if you have the header file. */ /* #undef HAVE_UTMP_H */ /* Define to 1 if you have the header file. */ #define HAVE_WCHAR_H 1 /* Define to 1 if you have the header file. */ #define HAVE_WCTYPE_H 1 /* Define to 1 if you have the `wcwidth' function. */ #define HAVE_WCWIDTH 1 /* Define as const if the declaration of iconv() needs const. */ -#define ICONV_CONST const +#define ICONV_CONST /* Support NLS. */ #define NLS 1 /* Support NLS catalogs. */ #define NLS_CATALOGS 1 /* Define to the address where bug reports for this package should be sent. */ #define PACKAGE_BUGREPORT "http://bugs.gw.com/" /* Define to the full name of this package. */ #define PACKAGE_NAME "tcsh" /* Define to the full name and version of this package. */ #define PACKAGE_STRING "tcsh 6.18.01" /* Define to the one symbol short name of this package. */ #define PACKAGE_TARNAME "tcsh" /* Define to the home page for this package. */ #define PACKAGE_URL "" /* Define to the version of this package. */ #define PACKAGE_VERSION "6.18.01" /* Define to 1 if the `setpgrp' function takes no argument. */ /* #undef SETPGRP_VOID */ /* The size of `wchar_t', as computed by sizeof. */ #define SIZEOF_WCHAR_T 4 /* Define to 1 if the `S_IS*' macros in do not work properly. */ /* #undef STAT_MACROS_BROKEN */ /* Define to 1 if you have the ANSI C header files. */ #define STDC_HEADERS 1 /* Define for Solaris 2.5.1 so the uint32_t typedef from , , or is not used. If the typedef were allowed, the #define below would cause a syntax error. */ /* #undef _UINT32_T */ /* Define to empty if `const' does not conform to ANSI C. */ /* #undef const */ /* Define to `int' if doesn't define. */ /* #undef gid_t */ /* Define to `int' if does not define. */ /* #undef mode_t */ /* Define to `unsigned int' if does not define. */ /* #undef size_t */ /* Define to `int' if neither nor define. */ /* #undef socklen_t */ /* Define to `int' not defined in . */ /* #undef ssize_t */ /* Define to `int' if doesn't define. */ /* #undef uid_t */ /* Define to the type of an unsigned integer type of width exactly 32 bits if such a type exists and the standard includes do not define it. */ /* #undef uint32_t */ /* Define to empty if the keyword `volatile' does not work. Warning: valid code using `volatile' can become incorrect without. Disable with care. */ /* #undef volatile */ #include "config_p.h" #include "config_f.h" /* Work around a vendor issue where config_f.h is #undef'ing this setting */ #define SYSMALLOC Index: stable/10/bin/csh/iconv_stub.h =================================================================== --- stable/10/bin/csh/iconv_stub.h (revision 282274) +++ stable/10/bin/csh/iconv_stub.h (revision 282275) @@ -1,44 +1,44 @@ /*- * Copyright (c) 2006 Hajimu UMEMOTO * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #ifndef _ICONV_H_ #define _ICONV_H_ typedef void *iconv_t; -typedef size_t dl_iconv_t(iconv_t, const char **, size_t *, char **, size_t *); +typedef size_t dl_iconv_t(iconv_t, char **, size_t *, char **, size_t *); typedef int dl_iconv_close_t(iconv_t); extern iconv_t dl_iconv_open(const char *, const char *); extern dl_iconv_t *dl_iconv; extern dl_iconv_close_t *dl_iconv_close; #define iconv_open dl_iconv_open #define iconv dl_iconv #define iconv_close dl_iconv_close #endif /* !_ICONV_H_ */ Index: stable/10/contrib/smbfs/include/netsmb/smb_lib.h =================================================================== --- stable/10/contrib/smbfs/include/netsmb/smb_lib.h (revision 282274) +++ stable/10/contrib/smbfs/include/netsmb/smb_lib.h (revision 282275) @@ -1,258 +1,258 @@ /* * Copyright (c) 2000-2001 Boris Popov * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Boris Popov. * 4. Neither the name of the author nor the names of any co-contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $Id: smb_lib.h,v 1.24 2001/12/20 15:19:43 bp Exp $ * $FreeBSD$ */ #ifndef _NETSMB_SMB_LIB_H_ #define _NETSMB_SMB_LIB_H_ #include #include #ifndef SMB_CFG_FILE #define SMB_CFG_FILE "/usr/local/etc/nsmb.conf" #endif #define STDPARAM_ARGS 'A':case 'B':case 'C':case 'E':case 'I': \ case 'L':case 'M': \ case 'N':case 'U':case 'R':case 'S':case 'T': \ case 'W':case 'O':case 'P' #define STDPARAM_OPT "A:BCE:I:L:M:NO:P:U:R:S:T:W:" /* * bits to indicate the source of error */ #define SMB_ERRTYPE_MASK 0xf0000 #define SMB_SYS_ERROR 0x00000 #define SMB_RAP_ERROR 0x10000 #define SMB_NB_ERROR 0x20000 #ifndef min #define min(a,b) (((a)<(b)) ? (a) : (b)) #endif #define getb(buf,ofs) (((const u_int8_t *)(buf))[ofs]) #define setb(buf,ofs,val) (((u_int8_t*)(buf))[ofs])=val #define getbw(buf,ofs) ((u_int16_t)(getb(buf,ofs))) #define getw(buf,ofs) (*((u_int16_t*)(&((u_int8_t*)(buf))[ofs]))) #define getdw(buf,ofs) (*((u_int32_t*)(&((u_int8_t*)(buf))[ofs]))) #if (BYTE_ORDER == LITTLE_ENDIAN) #define getwle(buf,ofs) (*((u_int16_t*)(&((u_int8_t*)(buf))[ofs]))) #define getdle(buf,ofs) (*((u_int32_t*)(&((u_int8_t*)(buf))[ofs]))) #define getwbe(buf,ofs) (ntohs(getwle(buf,ofs))) #define getdbe(buf,ofs) (ntohl(getdle(buf,ofs))) #define setwle(buf,ofs,val) getwle(buf,ofs)=val #define setwbe(buf,ofs,val) getwle(buf,ofs)=htons(val) #define setdle(buf,ofs,val) getdle(buf,ofs)=val #define setdbe(buf,ofs,val) getdle(buf,ofs)=htonl(val) #else /* (BYTE_ORDER == LITTLE_ENDIAN) */ #define getwbe(buf,ofs) (*((u_int16_t*)(&((u_int8_t*)(buf))[ofs]))) #define getdbe(buf,ofs) (*((u_int32_t*)(&((u_int8_t*)(buf))[ofs]))) #define getwle(buf,ofs) (bswap16(getwbe(buf,ofs))) #define getdle(buf,ofs) (bswap32(getdbe(buf,ofs))) #define setwbe(buf,ofs,val) getwbe(buf,ofs)=val #define setwle(buf,ofs,val) getwbe(buf,ofs)=bswap16(val) #define setdbe(buf,ofs,val) getdbe(buf,ofs)=val #define setdle(buf,ofs,val) getdbe(buf,ofs)=bswap32(val) #endif /* (BYTE_ORDER == LITTLE_ENDIAN) */ /* * SMB work context. Used to store all values which is necessary * to establish connection to an SMB server. */ struct smb_ctx { int ct_flags; /* SMBCF_ */ int ct_fd; /* handle of connection */ int ct_parsedlevel; int ct_minlevel; int ct_maxlevel; char * ct_srvaddr; /* hostname or IP address of server */ char ct_locname[SMB_MAXUSERNAMELEN + 1]; const char * ct_uncnext; struct nb_ctx * ct_nb; struct smbioc_ossn ct_ssn; struct smbioc_oshare ct_sh; long ct_smbtcpport; }; #define SMBCF_NOPWD 0x0001 /* don't ask for a password */ #define SMBCF_SRIGHTS 0x0002 /* share access rights was supplied */ #define SMBCF_LOCALE 0x0004 /* use current locale */ #define SMBCF_RESOLVED 0x8000 /* structure has been verified */ /* * request handling structures */ struct mbuf { int m_len; int m_maxlen; char * m_data; struct mbuf * m_next; }; struct mbdata { struct mbuf * mb_top; struct mbuf * mb_cur; char * mb_pos; int mb_count; }; #define M_ALIGNFACTOR (sizeof(long)) #define M_ALIGN(len) (((len) + M_ALIGNFACTOR - 1) & ~(M_ALIGNFACTOR - 1)) #define M_BASESIZE (sizeof(struct mbuf)) #define M_MINSIZE (256 - M_BASESIZE) #define M_TOP(m) ((char*)(m) + M_BASESIZE) #define mtod(m,t) ((t)(m)->m_data) #define M_TRAILINGSPACE(m) ((m)->m_maxlen - (m)->m_len) struct smb_rq { u_char rq_cmd; struct mbdata rq_rq; struct mbdata rq_rp; struct smb_ctx *rq_ctx; int rq_wcount; int rq_bcount; }; struct smb_bitname { u_int bn_bit; char *bn_name; }; extern struct rcfile *smb_rc; __BEGIN_DECLS struct sockaddr; int smb_lib_init(void); int smb_open_rcfile(void); void smb_error(const char *, int,...); char *smb_printb(char *, int, const struct smb_bitname *); void *smb_dumptree(void); /* * Context management */ int smb_ctx_init(struct smb_ctx *, int, char *[], int, int, int); void smb_ctx_done(struct smb_ctx *); int smb_ctx_parseunc(struct smb_ctx *, const char *, int, const char **); int smb_ctx_setcharset(struct smb_ctx *, const char *); int smb_ctx_setserver(struct smb_ctx *, const char *); int smb_ctx_setnbport(struct smb_ctx *, int); int smb_ctx_setsmbport(struct smb_ctx *, int); int smb_ctx_setuser(struct smb_ctx *, const char *); int smb_ctx_setshare(struct smb_ctx *, const char *, int); int smb_ctx_setscope(struct smb_ctx *, const char *); int smb_ctx_setworkgroup(struct smb_ctx *, const char *); int smb_ctx_setpassword(struct smb_ctx *, const char *); int smb_ctx_setsrvaddr(struct smb_ctx *, const char *); int smb_ctx_opt(struct smb_ctx *, int, const char *); int smb_ctx_lookup(struct smb_ctx *, int, int); int smb_ctx_login(struct smb_ctx *); int smb_ctx_readrc(struct smb_ctx *); int smb_ctx_resolve(struct smb_ctx *); int smb_ctx_setflags(struct smb_ctx *, int, int, int); -int smb_smb_open_print_file(struct smb_ctx *, int, int, const char *, smbfh*); +int smb_smb_open_print_file(struct smb_ctx *, int, int, char *, smbfh*); int smb_smb_close_print_file(struct smb_ctx *, smbfh); int smb_read(struct smb_ctx *, smbfh, off_t, size_t, char *); int smb_write(struct smb_ctx *, smbfh, off_t, size_t, const char *); #define smb_rq_getrequest(rqp) (&(rqp)->rq_rq) #define smb_rq_getreply(rqp) (&(rqp)->rq_rp) int smb_rq_init(struct smb_ctx *, u_char, size_t, struct smb_rq **); void smb_rq_done(struct smb_rq *); void smb_rq_wend(struct smb_rq *); int smb_rq_simple(struct smb_rq *); -int smb_rq_dmem(struct mbdata *, const char *, size_t); -int smb_rq_dstring(struct mbdata *, const char *); +int smb_rq_dmem(struct mbdata *, char *, size_t); +int smb_rq_dstring(struct mbdata *, char *); int smb_t2_request(struct smb_ctx *, int, int, const char *, int, void *, int, void *, int *, void *, int *, void *); char* smb_simplecrypt(char *dst, const char *src); int smb_simpledecrypt(char *dst, const char *src); int m_getm(struct mbuf *, size_t, struct mbuf **); int m_lineup(struct mbuf *, struct mbuf **); int mb_init(struct mbdata *, size_t); int mb_initm(struct mbdata *, struct mbuf *); int mb_done(struct mbdata *); int mb_fit(struct mbdata *mbp, size_t size, char **pp); int mb_put_uint8(struct mbdata *, u_int8_t); int mb_put_uint16be(struct mbdata *, u_int16_t); int mb_put_uint16le(struct mbdata *, u_int16_t); int mb_put_uint32be(struct mbdata *, u_int32_t); int mb_put_uint32le(struct mbdata *, u_int32_t); int mb_put_int64be(struct mbdata *, int64_t); int mb_put_int64le(struct mbdata *, int64_t); int mb_put_mem(struct mbdata *, const char *, size_t); int mb_put_pstring(struct mbdata *mbp, const char *s); int mb_put_mbuf(struct mbdata *, struct mbuf *); int mb_get_uint8(struct mbdata *, u_int8_t *); int mb_get_uint16(struct mbdata *, u_int16_t *); int mb_get_uint16le(struct mbdata *, u_int16_t *); int mb_get_uint16be(struct mbdata *, u_int16_t *); int mb_get_uint32(struct mbdata *, u_int32_t *); int mb_get_uint32be(struct mbdata *, u_int32_t *); int mb_get_uint32le(struct mbdata *, u_int32_t *); int mb_get_int64(struct mbdata *, int64_t *); int mb_get_int64be(struct mbdata *, int64_t *); int mb_get_int64le(struct mbdata *, int64_t *); int mb_get_mem(struct mbdata *, char *, size_t); extern u_char nls_lower[256], nls_upper[256]; int nls_setrecode(const char *, const char *); int nls_setlocale(const char *); -char* nls_str_toext(char *, const char *); -char* nls_str_toloc(char *, const char *); -void* nls_mem_toext(void *, const void *, int); -void* nls_mem_toloc(void *, const void *, int); +char* nls_str_toext(char *, char *); +char* nls_str_toloc(char *, char *); +void* nls_mem_toext(void *, void *, int); +void* nls_mem_toloc(void *, void *, int); char* nls_str_upper(char *, const char *); char* nls_str_lower(char *, const char *); __END_DECLS #endif /* _NETSMB_SMB_LIB_H_ */ Index: stable/10/contrib/smbfs/lib/smb/nls.c =================================================================== --- stable/10/contrib/smbfs/lib/smb/nls.c (revision 282274) +++ stable/10/contrib/smbfs/lib/smb/nls.c (revision 282275) @@ -1,223 +1,223 @@ /* * Copyright (c) 2000-2001, Boris Popov * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Boris Popov. * 4. Neither the name of the author nor the names of any co-contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $Id: nls.c,v 1.10 2002/07/22 08:33:59 bp Exp $ */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include #include #include #ifdef HAVE_ICONV #include #endif u_char nls_lower[256]; u_char nls_upper[256]; #ifdef HAVE_ICONV static iconv_t nls_toext, nls_toloc; #endif int nls_setlocale(const char *name) { int i; if (setlocale(LC_CTYPE, name) == NULL) { warnx("can't set locale '%s'\n", name); return EINVAL; } for (i = 0; i < 256; i++) { nls_lower[i] = tolower(i); nls_upper[i] = toupper(i); } return 0; } int nls_setrecode(const char *local, const char *external) { #ifdef HAVE_ICONV iconv_t icd; if (nls_toext) iconv_close(nls_toext); if (nls_toloc) iconv_close(nls_toloc); nls_toext = nls_toloc = (iconv_t)0; icd = iconv_open(external, local); if (icd == (iconv_t)-1) return errno; nls_toext = icd; icd = iconv_open(local, external); if (icd == (iconv_t)-1) { iconv_close(nls_toext); nls_toext = (iconv_t)0; return errno; } nls_toloc = icd; return 0; #else return ENOENT; #endif } char * -nls_str_toloc(char *dst, const char *src) +nls_str_toloc(char *dst, char *src) { #ifdef HAVE_ICONV char *p = dst; size_t inlen, outlen; if (nls_toloc == (iconv_t)0) return strcpy(dst, src); inlen = outlen = strlen(src); iconv(nls_toloc, NULL, NULL, &p, &outlen); while (iconv(nls_toloc, &src, &inlen, &p, &outlen) == -1) { *p++ = *src++; inlen--; outlen--; } *p = 0; return dst; #else return strcpy(dst, src); #endif } char * -nls_str_toext(char *dst, const char *src) +nls_str_toext(char *dst, char *src) { #ifdef HAVE_ICONV char *p = dst; size_t inlen, outlen; if (nls_toext == (iconv_t)0) return strcpy(dst, src); inlen = outlen = strlen(src); iconv(nls_toext, NULL, NULL, &p, &outlen); while (iconv(nls_toext, &src, &inlen, &p, &outlen) == -1) { *p++ = *src++; inlen--; outlen--; } *p = 0; return dst; #else return strcpy(dst, src); #endif } void * -nls_mem_toloc(void *dst, const void *src, int size) +nls_mem_toloc(void *dst, void *src, int size) { #ifdef HAVE_ICONV char *p = dst; - const char *s = src; + char *s = src; size_t inlen, outlen; if (size == 0) return NULL; if (nls_toloc == (iconv_t)0) return memcpy(dst, src, size); inlen = outlen = size; iconv(nls_toloc, NULL, NULL, &p, &outlen); while (iconv(nls_toloc, &s, &inlen, &p, &outlen) == -1) { *p++ = *s++; inlen--; outlen--; } return dst; #else return memcpy(dst, src, size); #endif } void * -nls_mem_toext(void *dst, const void *src, int size) +nls_mem_toext(void *dst, void *src, int size) { #ifdef HAVE_ICONV char *p = dst; - const char *s = src; + char *s = src; size_t inlen, outlen; if (size == 0) return NULL; if (nls_toext == (iconv_t)0) return memcpy(dst, src, size); inlen = outlen = size; iconv(nls_toext, NULL, NULL, &p, &outlen); while (iconv(nls_toext, &s, &inlen, &p, &outlen) == -1) { *p++ = *s++; inlen--; outlen--; } return dst; #else return memcpy(dst, src, size); #endif } char * nls_str_upper(char *dst, const char *src) { char *p = dst; while (*src) *dst++ = toupper(*src++); *dst = 0; return p; } char * nls_str_lower(char *dst, const char *src) { char *p = dst; while (*src) *dst++ = tolower(*src++); *dst = 0; return p; } Index: stable/10/contrib/smbfs/lib/smb/print.c =================================================================== --- stable/10/contrib/smbfs/lib/smb/print.c (revision 282274) +++ stable/10/contrib/smbfs/lib/smb/print.c (revision 282275) @@ -1,97 +1,97 @@ /* * Copyright (c) 2000, Boris Popov * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Boris Popov. * 4. Neither the name of the author nor the names of any co-contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $Id: print.c,v 1.4 2001/04/16 04:33:01 bp Exp $ */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include /*#include */ #include #include #include int smb_smb_open_print_file(struct smb_ctx *ctx, int setuplen, int mode, - const char *ident, smbfh *fhp) + char *ident, smbfh *fhp) { struct smb_rq *rqp; struct mbdata *mbp; int error; error = smb_rq_init(ctx, SMB_COM_OPEN_PRINT_FILE, 2, &rqp); if (error) return error; mbp = smb_rq_getrequest(rqp); mb_put_uint16le(mbp, setuplen); mb_put_uint16le(mbp, mode); smb_rq_wend(rqp); mb_put_uint8(mbp, SMB_DT_ASCII); smb_rq_dstring(mbp, ident); error = smb_rq_simple(rqp); if (!error) { mbp = smb_rq_getreply(rqp); mb_get_uint16(mbp, fhp); } smb_rq_done(rqp); return error; } int smb_smb_close_print_file(struct smb_ctx *ctx, smbfh fh) { struct smb_rq *rqp; struct mbdata *mbp; int error; error = smb_rq_init(ctx, SMB_COM_CLOSE_PRINT_FILE, 0, &rqp); if (error) return error; mbp = smb_rq_getrequest(rqp); mb_put_mem(mbp, (char*)&fh, 2); smb_rq_wend(rqp); error = smb_rq_simple(rqp); smb_rq_done(rqp); return error; } Index: stable/10/contrib/smbfs/lib/smb/rq.c =================================================================== --- stable/10/contrib/smbfs/lib/smb/rq.c (revision 282274) +++ stable/10/contrib/smbfs/lib/smb/rq.c (revision 282275) @@ -1,180 +1,180 @@ /* * Copyright (c) 2000, Boris Popov * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Boris Popov. * 4. Neither the name of the author nor the names of any co-contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $Id: rq.c,v 1.7 2001/04/16 04:33:01 bp Exp $ * $FreeBSD$ */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include int smb_rq_init(struct smb_ctx *ctx, u_char cmd, size_t rpbufsz, struct smb_rq **rqpp) { struct smb_rq *rqp; rqp = malloc(sizeof(*rqp)); if (rqp == NULL) return ENOMEM; bzero(rqp, sizeof(*rqp)); rqp->rq_cmd = cmd; rqp->rq_ctx = ctx; mb_init(&rqp->rq_rq, M_MINSIZE); mb_init(&rqp->rq_rp, rpbufsz); *rqpp = rqp; return 0; } void smb_rq_done(struct smb_rq *rqp) { mb_done(&rqp->rq_rp); mb_done(&rqp->rq_rq); free(rqp); } void smb_rq_wend(struct smb_rq *rqp) { if (rqp->rq_rq.mb_count & 1) smb_error("smbrq_wend: odd word count\n", 0); rqp->rq_wcount = rqp->rq_rq.mb_count / 2; rqp->rq_rq.mb_count = 0; } int -smb_rq_dmem(struct mbdata *mbp, const char *src, size_t size) +smb_rq_dmem(struct mbdata *mbp, char *src, size_t size) { struct mbuf *m; char * dst; int cplen, error; if (size == 0) return 0; m = mbp->mb_cur; if ((error = m_getm(m, size, &m)) != 0) return error; while (size > 0) { cplen = M_TRAILINGSPACE(m); if (cplen == 0) { m = m->m_next; continue; } if (cplen > (int)size) cplen = size; dst = mtod(m, char *) + m->m_len; nls_mem_toext(dst, src, cplen); size -= cplen; src += cplen; m->m_len += cplen; mbp->mb_count += cplen; } mbp->mb_pos = mtod(m, char *) + m->m_len; mbp->mb_cur = m; return 0; } int -smb_rq_dstring(struct mbdata *mbp, const char *s) +smb_rq_dstring(struct mbdata *mbp, char *s) { return smb_rq_dmem(mbp, s, strlen(s) + 1); } int smb_rq_simple(struct smb_rq *rqp) { struct smbioc_rq krq; struct mbdata *mbp; char *data; mbp = smb_rq_getrequest(rqp); m_lineup(mbp->mb_top, &mbp->mb_top); data = mtod(mbp->mb_top, char*); bzero(&krq, sizeof(krq)); krq.ioc_cmd = rqp->rq_cmd; krq.ioc_twc = rqp->rq_wcount; krq.ioc_twords = data; krq.ioc_tbc = mbp->mb_count; krq.ioc_tbytes = data + rqp->rq_wcount * 2; mbp = smb_rq_getreply(rqp); krq.ioc_rpbufsz = mbp->mb_top->m_maxlen; krq.ioc_rpbuf = mtod(mbp->mb_top, char *); if (ioctl(rqp->rq_ctx->ct_fd, SMBIOC_REQUEST, &krq) == -1) return errno; mbp->mb_top->m_len = krq.ioc_rwc * 2 + krq.ioc_rbc; rqp->rq_wcount = krq.ioc_rwc; rqp->rq_bcount = krq.ioc_rbc; return 0; } int smb_t2_request(struct smb_ctx *ctx, int setup, int setupcount, const char *name, int tparamcnt, void *tparam, int tdatacnt, void *tdata, int *rparamcnt, void *rparam, int *rdatacnt, void *rdata) { struct smbioc_t2rq krq; bzero(&krq, sizeof(krq)); krq.ioc_setup[0] = setup; krq.ioc_setupcnt = setupcount; krq.ioc_name = (char *)name; krq.ioc_tparamcnt = tparamcnt; krq.ioc_tparam = tparam; krq.ioc_tdatacnt = tdatacnt; krq.ioc_tdata = tdata; krq.ioc_rparamcnt = *rparamcnt; krq.ioc_rparam = rparam; krq.ioc_rdatacnt = *rdatacnt; krq.ioc_rdata = rdata; if (ioctl(ctx->ct_fd, SMBIOC_T2RQ, &krq) == -1) return errno; *rparamcnt = krq.ioc_rparamcnt; *rdatacnt = krq.ioc_rdatacnt; return 0; } Index: stable/10/include/iconv.h =================================================================== --- stable/10/include/iconv.h (revision 282274) +++ stable/10/include/iconv.h (revision 282275) @@ -1,133 +1,133 @@ /* $FreeBSD$ */ /* $NetBSD: iconv.h,v 1.6 2005/02/03 04:39:32 perry Exp $ */ /*- * Copyright (c) 2003 Citrus Project, * Copyright (c) 2009, 2010 Gabor Kovesdan * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #ifndef _ICONV_H_ #define _ICONV_H_ #include #include #include #include #include #ifdef __cplusplus typedef bool __iconv_bool; #elif __STDC_VERSION__ >= 199901L typedef _Bool __iconv_bool; #else typedef int __iconv_bool; #endif struct __tag_iconv_t; typedef struct __tag_iconv_t *iconv_t; __BEGIN_DECLS iconv_t iconv_open(const char *, const char *); -size_t iconv(iconv_t, const char ** __restrict, +size_t iconv(iconv_t, char ** __restrict, size_t * __restrict, char ** __restrict, size_t * __restrict); int iconv_close(iconv_t); /* * non-portable interfaces for iconv */ int __iconv_get_list(char ***, size_t *, __iconv_bool); void __iconv_free_list(char **, size_t); -size_t __iconv(iconv_t, const char **, size_t *, char **, +size_t __iconv(iconv_t, char **, size_t *, char **, size_t *, __uint32_t, size_t *); #define __ICONV_F_HIDE_INVALID 0x0001 /* * GNU interfaces for iconv */ typedef struct { void *spaceholder[64]; } iconv_allocation_t; int iconv_open_into(const char *, const char *, iconv_allocation_t *); void iconv_set_relocation_prefix(const char *, const char *); /* * iconvctl() request macros */ #define ICONV_TRIVIALP 0 #define ICONV_GET_TRANSLITERATE 1 #define ICONV_SET_TRANSLITERATE 2 #define ICONV_GET_DISCARD_ILSEQ 3 #define ICONV_SET_DISCARD_ILSEQ 4 #define ICONV_SET_HOOKS 5 #define ICONV_SET_FALLBACKS 6 #define ICONV_GET_ILSEQ_INVALID 128 #define ICONV_SET_ILSEQ_INVALID 129 typedef void (*iconv_unicode_char_hook) (unsigned int mbr, void *data); typedef void (*iconv_wide_char_hook) (wchar_t wc, void *data); struct iconv_hooks { iconv_unicode_char_hook uc_hook; iconv_wide_char_hook wc_hook; void *data; }; /* * Fallbacks aren't supported but type definitions are provided for * source compatibility. */ typedef void (*iconv_unicode_mb_to_uc_fallback) (const char*, size_t, void (*write_replacement) (const unsigned int *, size_t, void*), void*, void*); typedef void (*iconv_unicode_uc_to_mb_fallback) (unsigned int, void (*write_replacement) (const char *, size_t, void*), void*, void*); typedef void (*iconv_wchar_mb_to_wc_fallback) (const char*, size_t, void (*write_replacement) (const wchar_t *, size_t, void*), void*, void*); typedef void (*iconv_wchar_wc_to_mb_fallback) (wchar_t, void (*write_replacement) (const char *, size_t, void*), void*, void*); struct iconv_fallbacks { iconv_unicode_mb_to_uc_fallback mb_to_uc_fallback; iconv_unicode_uc_to_mb_fallback uc_to_mb_fallback; iconv_wchar_mb_to_wc_fallback mb_to_wc_fallback; iconv_wchar_wc_to_mb_fallback wc_to_mb_fallback; void *data; }; void iconvlist(int (*do_one) (unsigned int, const char * const *, void *), void *); const char *iconv_canonicalize(const char *); int iconvctl(iconv_t, int, void *); __END_DECLS #endif /* !_ICONV_H_ */ Index: stable/10/lib/libarchive/Makefile =================================================================== --- stable/10/lib/libarchive/Makefile (revision 282274) +++ stable/10/lib/libarchive/Makefile (revision 282275) @@ -1,420 +1,420 @@ # $FreeBSD$ .include LIBARCHIVEDIR= ${.CURDIR}/../../contrib/libarchive LIB= archive DPADD= ${LIBZ} LDADD= -lz DPADD+= ${LIBBZ2} LDADD+= -lbz2 CFLAGS+= -DHAVE_BZLIB_H=1 DPADD+= ${LIBLZMA} LDADD+= -llzma CFLAGS+= -DHAVE_LIBLZMA=1 -DHAVE_LZMA_H=1 DPADD+= ${LIBBSDXML} LDADD+= -lbsdxml # FreeBSD SHLIB_MAJOR value is managed as part of the FreeBSD system. # It has no real relation to the libarchive version number. SHLIB_MAJOR= 6 CFLAGS+= -DPLATFORM_CONFIG_H=\"${.CURDIR}/config_freebsd.h\" CFLAGS+= -I${.OBJDIR} .if ${MK_OPENSSL} != "no" CFLAGS+= -DWITH_OPENSSL DPADD+= ${LIBCRYPTO} LDADD+= -lcrypto .else DPADD+= ${LIBMD} LDADD+= -lmd .endif .if ${MK_ICONV} != "no" # TODO: This can be changed back to CFLAGS once iconv works correctly # with statically linked binaries. -SHARED_CFLAGS+= -DHAVE_ICONV=1 -DHAVE_ICONV_H=1 -DICONV_CONST=const +SHARED_CFLAGS+= -DHAVE_ICONV=1 -DHAVE_ICONV_H=1 -DICONV_CONST= .endif .if ${MACHINE_ARCH:Marm*} != "" || ${MACHINE_ARCH:Mmips*} != "" || \ ${MACHINE_ARCH:Msparc64*} != "" || ${MACHINE_ARCH:Mia64*} != "" NO_WCAST_ALIGN= yes .if ${MACHINE_ARCH:M*64*} == "" CFLAGS+= -DPPMD_32BIT .endif .endif NO_WCAST_ALIGN.clang= .PATH: ${LIBARCHIVEDIR}/libarchive # Headers to be installed in /usr/include INCS= archive.h archive_entry.h # Sources to be compiled. SRCS= archive_acl.c \ archive_check_magic.c \ archive_cmdline.c \ archive_crypto.c \ archive_entry.c \ archive_entry_copy_stat.c \ archive_entry_link_resolver.c \ archive_entry_sparse.c \ archive_entry_stat.c \ archive_entry_strmode.c \ archive_entry_xattr.c \ archive_getdate.c \ archive_match.c \ archive_options.c \ archive_pathmatch.c \ archive_ppmd7.c \ archive_rb.c \ archive_read.c \ archive_read_append_filter.c \ archive_read_data_into_fd.c \ archive_read_disk_entry_from_file.c \ archive_read_disk_posix.c \ archive_read_disk_set_standard_lookup.c \ archive_read_extract.c \ archive_read_open_fd.c \ archive_read_open_file.c \ archive_read_open_filename.c \ archive_read_open_memory.c \ archive_read_set_format.c \ archive_read_set_options.c \ archive_read_support_filter_all.c \ archive_read_support_filter_bzip2.c \ archive_read_support_filter_compress.c \ archive_read_support_filter_gzip.c \ archive_read_support_filter_grzip.c \ archive_read_support_filter_lrzip.c \ archive_read_support_filter_lzop.c \ archive_read_support_filter_none.c \ archive_read_support_filter_program.c \ archive_read_support_filter_rpm.c \ archive_read_support_filter_uu.c \ archive_read_support_filter_xz.c \ archive_read_support_format_7zip.c \ archive_read_support_format_all.c \ archive_read_support_format_ar.c \ archive_read_support_format_by_code.c \ archive_read_support_format_cab.c \ archive_read_support_format_cpio.c \ archive_read_support_format_empty.c \ archive_read_support_format_iso9660.c \ archive_read_support_format_lha.c \ archive_read_support_format_mtree.c \ archive_read_support_format_rar.c \ archive_read_support_format_raw.c \ archive_read_support_format_tar.c \ archive_read_support_format_xar.c \ archive_read_support_format_zip.c \ archive_string.c \ archive_string_sprintf.c \ archive_util.c \ archive_virtual.c \ archive_write.c \ archive_write_add_filter.c \ archive_write_disk_acl.c \ archive_write_disk_set_standard_lookup.c \ archive_write_disk_posix.c \ archive_write_open_fd.c \ archive_write_open_file.c \ archive_write_open_filename.c \ archive_write_open_memory.c \ archive_write_add_filter_b64encode.c \ archive_write_add_filter_by_name.c \ archive_write_add_filter_bzip2.c \ archive_write_add_filter_compress.c \ archive_write_add_filter_grzip.c \ archive_write_add_filter_gzip.c \ archive_write_add_filter_lrzip.c \ archive_write_add_filter_lzop.c \ archive_write_add_filter_none.c \ archive_write_add_filter_program.c \ archive_write_add_filter_uuencode.c \ archive_write_add_filter_xz.c \ archive_write_set_format.c \ archive_write_set_format_7zip.c \ archive_write_set_format_ar.c \ archive_write_set_format_by_name.c \ archive_write_set_format_cpio.c \ archive_write_set_format_cpio_newc.c \ archive_write_set_format_gnutar.c \ archive_write_set_format_iso9660.c \ archive_write_set_format_mtree.c \ archive_write_set_format_pax.c \ archive_write_set_format_shar.c \ archive_write_set_format_ustar.c \ archive_write_set_format_v7tar.c \ archive_write_set_format_xar.c \ archive_write_set_format_zip.c \ archive_write_set_options.c \ filter_fork_posix.c # Man pages to be installed. MAN= archive_entry.3 \ archive_entry_acl.3 \ archive_entry_linkify.3 \ archive_entry_paths.3 \ archive_entry_perms.3 \ archive_entry_stat.3 \ archive_entry_time.3 \ archive_read.3 \ archive_read_data.3 \ archive_read_disk.3 \ archive_read_extract.3 \ archive_read_filter.3 \ archive_read_format.3 \ archive_read_free.3 \ archive_read_header.3 \ archive_read_new.3 \ archive_read_open.3 \ archive_read_set_options.3 \ archive_util.3 \ archive_write.3 \ archive_write_blocksize.3 \ archive_write_data.3 \ archive_write_disk.3 \ archive_write_filter.3 \ archive_write_finish_entry.3 \ archive_write_format.3 \ archive_write_free.3 \ archive_write_header.3 \ archive_write_new.3 \ archive_write_open.3 \ archive_write_set_options.3 \ cpio.5 \ libarchive.3 \ libarchive_changes.3 \ libarchive_internals.3 \ libarchive-formats.5 \ tar.5 # Symlink the man pages under each function name. MLINKS+= archive_entry.3 archive_entry_clear.3 MLINKS+= archive_entry.3 archive_entry_clone.3 MLINKS+= archive_entry.3 archive_entry_free.3 MLINKS+= archive_entry.3 archive_entry_new.3 MLINKS+= archive_entry_acl.3 archive_entry_acl_add_entry.3 MLINKS+= archive_entry_acl.3 archive_entry_acl_add_entry_w.3 MLINKS+= archive_entry_acl.3 archive_entry_acl_clear.3 MLINKS+= archive_entry_acl.3 archive_entry_acl_count.3 MLINKS+= archive_entry_acl.3 archive_entry_acl_next.3 MLINKS+= archive_entry_acl.3 archive_entry_acl_next_w.3 MLINKS+= archive_entry_acl.3 archive_entry_acl_reset.3 MLINKS+= archive_entry_acl.3 archive_entry_acl_text_w.3 MLINKS+= archive_entry_linkify.3 archive_entry_linkresolver.3 MLINKS+= archive_entry_linkify.3 archive_entry_linkresolver_new.3 MLINKS+= archive_entry_linkify.3 archive_entry_linkresolver_set_strategy.3 MLINKS+= archive_entry_linkify.3 archive_entry_linkresolver_free.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_hardlink.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_hardlink_w.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_link.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_link_w.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_pathname.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_pathname_w.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_sourcepath.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_symlink.3 MLINKS+= archive_entry_paths.3 archive_entry_copy_symlink_w.3 MLINKS+= archive_entry_paths.3 archive_entry_hardlink.3 MLINKS+= archive_entry_paths.3 archive_entry_hardlink_w.3 MLINKS+= archive_entry_paths.3 archive_entry_pathname.3 MLINKS+= archive_entry_paths.3 archive_entry_pathname_w.3 MLINKS+= archive_entry_paths.3 archive_entry_set_hardlink.3 MLINKS+= archive_entry_paths.3 archive_entry_set_link.3 MLINKS+= archive_entry_paths.3 archive_entry_set_pathname.3 MLINKS+= archive_entry_paths.3 archive_entry_set_symlink.3 MLINKS+= archive_entry_paths.3 archive_entry_symlink.3 MLINKS+= archive_entry_paths.3 archive_entry_symlink_w.3 MLINKS+= archive_entry_paths.3 archive_entry_update_symlink_utf8.3 MLINKS+= archive_entry_paths.3 archive_entry_update_hardlink_utf8.3 MLINKS+= archive_entry_perms.3 archive_entry_copy_fflags_text.3 MLINKS+= archive_entry_perms.3 archive_entry_copy_fflags_text_w.3 MLINKS+= archive_entry_perms.3 archive_entry_copy_gname.3 MLINKS+= archive_entry_perms.3 archive_entry_copy_gname_w.3 MLINKS+= archive_entry_perms.3 archive_entry_copy_uname.3 MLINKS+= archive_entry_perms.3 archive_entry_copy_uname_w.3 MLINKS+= archive_entry_perms.3 archive_entry_fflags.3 MLINKS+= archive_entry_perms.3 archive_entry_fflags_text.3 MLINKS+= archive_entry_perms.3 archive_entry_gid.3 MLINKS+= archive_entry_perms.3 archive_entry_gname.3 MLINKS+= archive_entry_perms.3 archive_entry_gname_w.3 MLINKS+= archive_entry_perms.3 archive_entry_set_fflags.3 MLINKS+= archive_entry_perms.3 archive_entry_set_gid.3 MLINKS+= archive_entry_perms.3 archive_entry_set_gname.3 MLINKS+= archive_entry_perms.3 archive_entry_perm.3 MLINKS+= archive_entry_perms.3 archive_entry_set_perm.3 MLINKS+= archive_entry_perms.3 archive_entry_set_uid.3 MLINKS+= archive_entry_perms.3 archive_entry_set_uname.3 MLINKS+= archive_entry_perms.3 archive_entry_strmode.3 MLINKS+= archive_entry_perms.3 archive_entry_uid.3 MLINKS+= archive_entry_perms.3 archive_entry_uname.3 MLINKS+= archive_entry_perms.3 archive_entry_uname_w.3 MLINKS+= archive_entry_perms.3 archive_entry_update_gname_utf8.3 MLINKS+= archive_entry_perms.3 archive_entry_update_uname_utf8.3 MLINKS+= archive_entry_stat.3 archive_entry_copy_stat.3 MLINKS+= archive_entry_stat.3 archive_entry_dev.3 MLINKS+= archive_entry_stat.3 archive_entry_dev_is_set.3 MLINKS+= archive_entry_stat.3 archive_entry_devmajor.3 MLINKS+= archive_entry_stat.3 archive_entry_devminor.3 MLINKS+= archive_entry_stat.3 archive_entry_filetype.3 MLINKS+= archive_entry_stat.3 archive_entry_ino.3 MLINKS+= archive_entry_stat.3 archive_entry_ino64.3 MLINKS+= archive_entry_stat.3 archive_entry_ino_is_set.3 MLINKS+= archive_entry_stat.3 archive_entry_mode.3 MLINKS+= archive_entry_stat.3 archive_entry_nlink.3 MLINKS+= archive_entry_stat.3 archive_entry_rdev.3 MLINKS+= archive_entry_stat.3 archive_entry_rdevmajor.3 MLINKS+= archive_entry_stat.3 archive_entry_rdevminor.3 MLINKS+= archive_entry_stat.3 archive_entry_set_dev.3 MLINKS+= archive_entry_stat.3 archive_entry_set_devmajor.3 MLINKS+= archive_entry_stat.3 archive_entry_set_devminor.3 MLINKS+= archive_entry_stat.3 archive_entry_set_filetype.3 MLINKS+= archive_entry_stat.3 archive_entry_set_ino.3 MLINKS+= archive_entry_stat.3 archive_entry_set_ino64.3 MLINKS+= archive_entry_stat.3 archive_entry_set_mode.3 MLINKS+= archive_entry_stat.3 archive_entry_set_nlink.3 MLINKS+= archive_entry_stat.3 archive_entry_set_rdev.3 MLINKS+= archive_entry_stat.3 archive_entry_set_rdevmajor.3 MLINKS+= archive_entry_stat.3 archive_entry_set_rdevminor.3 MLINKS+= archive_entry_stat.3 archive_entry_set_size.3 MLINKS+= archive_entry_stat.3 archive_entry_size.3 MLINKS+= archive_entry_stat.3 archive_entry_size_is_set.3 MLINKS+= archive_entry_stat.3 archive_entry_unset_size.3 MLINKS+= archive_entry_time.3 archive_entry_atime.3 MLINKS+= archive_entry_time.3 archive_entry_atime_is_set.3 MLINKS+= archive_entry_time.3 archive_entry_atime_nsec.3 MLINKS+= archive_entry_time.3 archive_entry_birthtime.3 MLINKS+= archive_entry_time.3 archive_entry_birthtime_is_set.3 MLINKS+= archive_entry_time.3 archive_entry_birthtime_nsec.3 MLINKS+= archive_entry_time.3 archive_entry_ctime.3 MLINKS+= archive_entry_time.3 archive_entry_ctime_is_set.3 MLINKS+= archive_entry_time.3 archive_entry_ctime_nsec.3 MLINKS+= archive_entry_time.3 archive_entry_mtime.3 MLINKS+= archive_entry_time.3 archive_entry_mtime_is_set.3 MLINKS+= archive_entry_time.3 archive_entry_mtime_nsec.3 MLINKS+= archive_entry_time.3 archive_entry_set_atime.3 MLINKS+= archive_entry_time.3 archive_entry_set_birthtime.3 MLINKS+= archive_entry_time.3 archive_entry_set_ctime.3 MLINKS+= archive_entry_time.3 archive_entry_set_mtime.3 MLINKS+= archive_entry_time.3 archive_entry_unset_atime.3 MLINKS+= archive_entry_time.3 archive_entry_unset_birthtime.3 MLINKS+= archive_entry_time.3 archive_entry_unset_ctime.3 MLINKS+= archive_entry_time.3 archive_entry_unset_mtime.3 MLINKS+= archive_read_data.3 archive_read_data_block.3 MLINKS+= archive_read_data.3 archive_read_data_into_fd.3 MLINKS+= archive_read_data.3 archive_read_data_skip.3 MLINKS+= archive_read_header.3 archive_read_next_header.3 MLINKS+= archive_read_header.3 archive_read_next_header2.3 MLINKS+= archive_read_extract.3 archive_read_extract2.3 MLINKS+= archive_read_extract.3 archive_read_extract_set_progress_callback.3 MLINKS+= archive_read_extract.3 archive_read_extract_set_skip_file.3 MLINKS+= archive_read_open.3 archive_read_open2.3 MLINKS+= archive_read_open.3 archive_read_open_FILE.3 MLINKS+= archive_read_open.3 archive_read_open_fd.3 MLINKS+= archive_read_open.3 archive_read_open_file.3 MLINKS+= archive_read_open.3 archive_read_open_filename.3 MLINKS+= archive_read_open.3 archive_read_open_memory.3 MLINKS+= archive_read_free.3 archive_read_close.3 MLINKS+= archive_read_free.3 archive_read_finish.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_all.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_bzip2.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_compress.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_gzip.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_lzma.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_none.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_xz.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_program.3 MLINKS+= archive_read_filter.3 archive_read_support_filter_program_signature.3 MLINKS+= archive_read_format.3 archive_read_support_format_7zip.3 MLINKS+= archive_read_format.3 archive_read_support_format_all.3 MLINKS+= archive_read_format.3 archive_read_support_format_ar.3 MLINKS+= archive_read_format.3 archive_read_support_format_by_code.3 MLINKS+= archive_read_format.3 archive_read_support_format_cab.3 MLINKS+= archive_read_format.3 archive_read_support_format_cpio.3 MLINKS+= archive_read_format.3 archive_read_support_format_empty.3 MLINKS+= archive_read_format.3 archive_read_support_format_iso9660.3 MLINKS+= archive_read_format.3 archive_read_support_format_lha.3 MLINKS+= archive_read_format.3 archive_read_support_format_mtree.3 MLINKS+= archive_read_format.3 archive_read_support_format_rar.3 MLINKS+= archive_read_format.3 archive_read_support_format_raw.3 MLINKS+= archive_read_format.3 archive_read_support_format_tar.3 MLINKS+= archive_read_format.3 archive_read_support_format_xar.3 MLINKS+= archive_read_format.3 archive_read_support_format_zip.3 MLINKS+= archive_read_disk.3 archive_read_disk_entry_from_file.3 MLINKS+= archive_read_disk.3 archive_read_disk_gname.3 MLINKS+= archive_read_disk.3 archive_read_disk_new.3 MLINKS+= archive_read_disk.3 archive_read_disk_set_gname_lookup.3 MLINKS+= archive_read_disk.3 archive_read_disk_set_standard_lookup.3 MLINKS+= archive_read_disk.3 archive_read_disk_set_symlink_hybrid.3 MLINKS+= archive_read_disk.3 archive_read_disk_set_symlink_logical.3 MLINKS+= archive_read_disk.3 archive_read_disk_set_symlink_physical.3 MLINKS+= archive_read_disk.3 archive_read_disk_set_uname_lookup.3 MLINKS+= archive_read_disk.3 archive_read_disk_uname.3 MLINKS+= archive_read_set_options.3 archive_read_set_filter_option.3 MLINKS+= archive_read_set_options.3 archive_read_set_format_option.3 MLINKS+= archive_read_set_options.3 archive_read_set_option.3 MLINKS+= archive_util.3 archive_clear_error.3 MLINKS+= archive_util.3 archive_compression.3 MLINKS+= archive_util.3 archive_compression_name.3 MLINKS+= archive_util.3 archive_copy_error.3 MLINKS+= archive_util.3 archive_errno.3 MLINKS+= archive_util.3 archive_error_string.3 MLINKS+= archive_util.3 archive_file_count.3 MLINKS+= archive_util.3 archive_filter_code.3 MLINKS+= archive_util.3 archive_filter_count.3 MLINKS+= archive_util.3 archive_filter_name.3 MLINKS+= archive_util.3 archive_format.3 MLINKS+= archive_util.3 archive_format_name.3 MLINKS+= archive_util.3 archive_position.3 MLINKS+= archive_util.3 archive_set_error.3 MLINKS+= archive_write_blocksize.3 archive_write_get_bytes_in_last_block.3 MLINKS+= archive_write_blocksize.3 archive_write_get_bytes_per_block.3 MLINKS+= archive_write_blocksize.3 archive_write_set_bytes_in_last_block.3 MLINKS+= archive_write_blocksize.3 archive_write_set_bytes_per_block.3 MLINKS+= archive_write_disk.3 archive_write_data_block.3 MLINKS+= archive_write_disk.3 archive_write_disk_new.3 MLINKS+= archive_write_disk.3 archive_write_disk_set_group_lookup.3 MLINKS+= archive_write_disk.3 archive_write_disk_set_options.3 MLINKS+= archive_write_disk.3 archive_write_disk_set_skip_file.3 MLINKS+= archive_write_disk.3 archive_write_disk_set_standard_lookup.3 MLINKS+= archive_write_disk.3 archive_write_disk_set_user_lookup.3 MLINKS+= archive_write_filter.3 archive_write_add_filter_bzip2.3 MLINKS+= archive_write_filter.3 archive_write_add_filter_compress.3 MLINKS+= archive_write_filter.3 archive_write_add_filter_gzip.3 MLINKS+= archive_write_filter.3 archive_write_add_filter_lzip.3 MLINKS+= archive_write_filter.3 archive_write_add_filter_lzma.3 MLINKS+= archive_write_filter.3 archive_write_add_filter_none.3 MLINKS+= archive_write_filter.3 archive_write_add_filter_program.3 MLINKS+= archive_write_filter.3 archive_write_add_filter_xz.3 MLINKS+= archive_write_format.3 archive_write_set_format_cpio.3 MLINKS+= archive_write_format.3 archive_write_set_format_pax.3 MLINKS+= archive_write_format.3 archive_write_set_format_pax_restricted.3 MLINKS+= archive_write_format.3 archive_write_set_format_shar.3 MLINKS+= archive_write_format.3 archive_write_set_format_shar_dump.3 MLINKS+= archive_write_format.3 archive_write_set_format_ustar.3 MLINKS+= archive_write_free.3 archive_write_close.3 MLINKS+= archive_write_free.3 archive_write_fail.3 MLINKS+= archive_write_free.3 archive_write_finish.3 MLINKS+= archive_write_open.3 archive_write_open_FILE.3 MLINKS+= archive_write_open.3 archive_write_open_fd.3 MLINKS+= archive_write_open.3 archive_write_open_file.3 MLINKS+= archive_write_open.3 archive_write_open_filename.3 MLINKS+= archive_write_open.3 archive_write_open_memory.3 MLINKS+= archive_write_set_options.3 archive_write_set_filter_option.3 MLINKS+= archive_write_set_options.3 archive_write_set_format_option.3 MLINKS+= archive_write_set_options.3 archive_write_set_option.3 MLINKS+= libarchive.3 archive.3 .PHONY: check test clean-test check test: cd ${.CURDIR}/test && make obj && make test clean-test: cd ${.CURDIR}/test && make clean .include Index: stable/10/lib/libc/iconv/__iconv.c =================================================================== --- stable/10/lib/libc/iconv/__iconv.c (revision 282274) +++ stable/10/lib/libc/iconv/__iconv.c (revision 282275) @@ -1,38 +1,38 @@ /*- * Copyright (c) 2013 Peter Wemm * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #include #include #include "iconv-internal.h" size_t -__iconv(iconv_t a, const char **b, size_t *c, char **d, +__iconv(iconv_t a, char **b, size_t *c, char **d, size_t *e, __uint32_t f, size_t *g) { return __bsd___iconv(a, b, c, d, e, f, g); } Index: stable/10/lib/libc/iconv/bsd_iconv.c =================================================================== --- stable/10/lib/libc/iconv/bsd_iconv.c (revision 282274) +++ stable/10/lib/libc/iconv/bsd_iconv.c (revision 282275) @@ -1,319 +1,319 @@ /* $FreeBSD$ */ /* $NetBSD: iconv.c,v 1.11 2009/03/03 16:22:33 explorer Exp $ */ /*- * Copyright (c) 2003 Citrus Project, * Copyright (c) 2009, 2010 Gabor Kovesdan , * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_types.h" #include "citrus_module.h" #include "citrus_esdb.h" #include "citrus_hash.h" #include "citrus_iconv.h" #include "iconv-internal.h" #define ISBADF(_h_) (!(_h_) || (_h_) == (iconv_t)-1) static iconv_t __bsd___iconv_open(const char *out, const char *in, struct _citrus_iconv *handle) { const char *out_slashes; char *out_noslashes; int ret; /* * Remove anything following a //, as these are options (like * //ignore, //translate, etc) and we just don't handle them. * This is for compatibility with software that uses these * blindly. */ out_slashes = strstr(out, "//"); if (out_slashes != NULL) { out_noslashes = strndup(out, out_slashes - out); if (out_noslashes == NULL) { errno = ENOMEM; return ((iconv_t)-1); } ret = _citrus_iconv_open(&handle, in, out_noslashes); free(out_noslashes); } else { ret = _citrus_iconv_open(&handle, in, out); } if (ret) { errno = ret == ENOENT ? EINVAL : ret; return ((iconv_t)-1); } handle->cv_shared->ci_discard_ilseq = strcasestr(out, "//IGNORE"); handle->cv_shared->ci_ilseq_invalid = false; handle->cv_shared->ci_hooks = NULL; return ((iconv_t)(void *)handle); } iconv_t __bsd_iconv_open(const char *out, const char *in) { return (__bsd___iconv_open(out, in, NULL)); } int __bsd_iconv_open_into(const char *out, const char *in, iconv_allocation_t *ptr) { struct _citrus_iconv *handle; handle = (struct _citrus_iconv *)ptr; return ((__bsd___iconv_open(out, in, handle) == (iconv_t)-1) ? -1 : 0); } int __bsd_iconv_close(iconv_t handle) { if (ISBADF(handle)) { errno = EBADF; return (-1); } _citrus_iconv_close((struct _citrus_iconv *)(void *)handle); return (0); } size_t -__bsd_iconv(iconv_t handle, const char **in, size_t *szin, char **out, size_t *szout) +__bsd_iconv(iconv_t handle, char **in, size_t *szin, char **out, size_t *szout) { size_t ret; int err; if (ISBADF(handle)) { errno = EBADF; return ((size_t)-1); } err = _citrus_iconv_convert((struct _citrus_iconv *)(void *)handle, in, szin, out, szout, 0, &ret); if (err) { errno = err; ret = (size_t)-1; } return (ret); } size_t -__bsd___iconv(iconv_t handle, const char **in, size_t *szin, char **out, +__bsd___iconv(iconv_t handle, char **in, size_t *szin, char **out, size_t *szout, uint32_t flags, size_t *invalids) { size_t ret; int err; if (ISBADF(handle)) { errno = EBADF; return ((size_t)-1); } err = _citrus_iconv_convert((struct _citrus_iconv *)(void *)handle, in, szin, out, szout, flags, &ret); if (invalids) *invalids = ret; if (err) { errno = err; ret = (size_t)-1; } return (ret); } int __bsd___iconv_get_list(char ***rlist, size_t *rsz, bool sorted) { int ret; ret = _citrus_esdb_get_list(rlist, rsz, sorted); if (ret) { errno = ret; return (-1); } return (0); } void __bsd___iconv_free_list(char **list, size_t sz) { _citrus_esdb_free_list(list, sz); } /* * GNU-compatibile non-standard interfaces. */ static int qsort_helper(const void *first, const void *second) { const char * const *s1; const char * const *s2; s1 = first; s2 = second; return (strcmp(*s1, *s2)); } void __bsd_iconvlist(int (*do_one) (unsigned int, const char * const *, void *), void *data) { char **list, **names; const char * const *np; char *curitem, *curkey, *slashpos; size_t sz; unsigned int i, j; i = 0; if (__bsd___iconv_get_list(&list, &sz, true)) list = NULL; qsort((void *)list, sz, sizeof(char *), qsort_helper); while (i < sz) { j = 0; slashpos = strchr(list[i], '/'); curkey = (char *)malloc(slashpos - list[i] + 2); names = (char **)malloc(sz * sizeof(char *)); if ((curkey == NULL) || (names == NULL)) { __bsd___iconv_free_list(list, sz); return; } strlcpy(curkey, list[i], slashpos - list[i] + 1); names[j++] = curkey; for (; (i < sz) && (memcmp(curkey, list[i], strlen(curkey)) == 0); i++) { slashpos = strchr(list[i], '/'); curitem = (char *)malloc(strlen(slashpos) + 1); if (curitem == NULL) { __bsd___iconv_free_list(list, sz); return; } strlcpy(curitem, &slashpos[1], strlen(slashpos) + 1); if (strcmp(curkey, curitem) == 0) { continue; } names[j++] = curitem; } np = (const char * const *)names; do_one(j, np, data); free(names); } __bsd___iconv_free_list(list, sz); } __inline const char * __bsd_iconv_canonicalize(const char *name) { return (_citrus_iconv_canonicalize(name)); } int __bsd_iconvctl(iconv_t cd, int request, void *argument) { struct _citrus_iconv *cv; struct iconv_hooks *hooks; const char *convname; char src[PATH_MAX], *dst; int *i; cv = (struct _citrus_iconv *)(void *)cd; hooks = (struct iconv_hooks *)argument; i = (int *)argument; if (ISBADF(cd)) { errno = EBADF; return (-1); } switch (request) { case ICONV_TRIVIALP: convname = cv->cv_shared->ci_convname; dst = strchr(convname, '/'); strlcpy(src, convname, dst - convname + 1); dst++; if ((convname == NULL) || (src == NULL) || (dst == NULL)) return (-1); *i = strcmp(src, dst) == 0 ? 1 : 0; return (0); case ICONV_GET_TRANSLITERATE: *i = 1; return (0); case ICONV_SET_TRANSLITERATE: return ((*i == 1) ? 0 : -1); case ICONV_GET_DISCARD_ILSEQ: *i = cv->cv_shared->ci_discard_ilseq ? 1 : 0; return (0); case ICONV_SET_DISCARD_ILSEQ: cv->cv_shared->ci_discard_ilseq = *i; return (0); case ICONV_SET_HOOKS: cv->cv_shared->ci_hooks = hooks; return (0); case ICONV_SET_FALLBACKS: errno = EOPNOTSUPP; return (-1); case ICONV_GET_ILSEQ_INVALID: *i = cv->cv_shared->ci_ilseq_invalid ? 1 : 0; return (0); case ICONV_SET_ILSEQ_INVALID: cv->cv_shared->ci_ilseq_invalid = *i; return (0); default: errno = EINVAL; return (-1); } } void __bsd_iconv_set_relocation_prefix(const char *orig_prefix __unused, const char *curr_prefix __unused) { } Index: stable/10/lib/libc/iconv/citrus_iconv.h =================================================================== --- stable/10/lib/libc/iconv/citrus_iconv.h (revision 282274) +++ stable/10/lib/libc/iconv/citrus_iconv.h (revision 282275) @@ -1,64 +1,64 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_iconv.h,v 1.5 2008/02/09 14:56:20 junyoung Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #ifndef _CITRUS_ICONV_H_ #define _CITRUS_ICONV_H_ struct _citrus_iconv_shared; struct _citrus_iconv_ops; struct _citrus_iconv; __BEGIN_DECLS int _citrus_iconv_open(struct _citrus_iconv * __restrict * __restrict, const char * __restrict, const char * __restrict); void _citrus_iconv_close(struct _citrus_iconv *); const char *_citrus_iconv_canonicalize(const char *); __END_DECLS #include "citrus_iconv_local.h" #define _CITRUS_ICONV_F_HIDE_INVALID 0x0001 /* * _citrus_iconv_convert: * convert a string. */ static __inline int _citrus_iconv_convert(struct _citrus_iconv * __restrict cv, - const char * __restrict * __restrict in, size_t * __restrict inbytes, + char * __restrict * __restrict in, size_t * __restrict inbytes, char * __restrict * __restrict out, size_t * __restrict outbytes, uint32_t flags, size_t * __restrict nresults) { return (*cv->cv_shared->ci_ops->io_convert)(cv, in, inbytes, out, outbytes, flags, nresults); } #endif Index: stable/10/lib/libc/iconv/citrus_iconv_local.h =================================================================== --- stable/10/lib/libc/iconv/citrus_iconv_local.h (revision 282274) +++ stable/10/lib/libc/iconv/citrus_iconv_local.h (revision 282275) @@ -1,110 +1,110 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_iconv_local.h,v 1.3 2008/02/09 14:56:20 junyoung Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #ifndef _CITRUS_ICONV_LOCAL_H_ #define _CITRUS_ICONV_LOCAL_H_ #include #include #define _CITRUS_ICONV_GETOPS_FUNC_BASE(_n_) \ int _n_(struct _citrus_iconv_ops *) #define _CITRUS_ICONV_GETOPS_FUNC(_n_) \ _CITRUS_ICONV_GETOPS_FUNC_BASE(_citrus_##_n_##_iconv_getops) #define _CITRUS_ICONV_DECLS(_m_) \ static int _citrus_##_m_##_iconv_init_shared \ (struct _citrus_iconv_shared * __restrict, \ const char * __restrict, const char * __restrict); \ static void _citrus_##_m_##_iconv_uninit_shared \ (struct _citrus_iconv_shared *); \ static int _citrus_##_m_##_iconv_convert \ (struct _citrus_iconv * __restrict, \ - const char * __restrict * __restrict, \ + char * __restrict * __restrict, \ size_t * __restrict, \ char * __restrict * __restrict, \ size_t * __restrict outbytes, \ uint32_t, size_t * __restrict); \ static int _citrus_##_m_##_iconv_init_context \ (struct _citrus_iconv *); \ static void _citrus_##_m_##_iconv_uninit_context \ (struct _citrus_iconv *) #define _CITRUS_ICONV_DEF_OPS(_m_) \ extern struct _citrus_iconv_ops _citrus_##_m_##_iconv_ops; \ struct _citrus_iconv_ops _citrus_##_m_##_iconv_ops = { \ /* io_init_shared */ &_citrus_##_m_##_iconv_init_shared, \ /* io_uninit_shared */ &_citrus_##_m_##_iconv_uninit_shared, \ /* io_init_context */ &_citrus_##_m_##_iconv_init_context, \ /* io_uninit_context */ &_citrus_##_m_##_iconv_uninit_context, \ /* io_convert */ &_citrus_##_m_##_iconv_convert \ } typedef _CITRUS_ICONV_GETOPS_FUNC_BASE((*_citrus_iconv_getops_t)); typedef int (*_citrus_iconv_init_shared_t) (struct _citrus_iconv_shared * __restrict, const char * __restrict, const char * __restrict); typedef void (*_citrus_iconv_uninit_shared_t) (struct _citrus_iconv_shared *); typedef int (*_citrus_iconv_convert_t) (struct _citrus_iconv * __restrict, - const char *__restrict* __restrict, size_t * __restrict, + char *__restrict* __restrict, size_t * __restrict, char * __restrict * __restrict, size_t * __restrict, uint32_t, size_t * __restrict); typedef int (*_citrus_iconv_init_context_t)(struct _citrus_iconv *); typedef void (*_citrus_iconv_uninit_context_t)(struct _citrus_iconv *); struct _citrus_iconv_ops { _citrus_iconv_init_shared_t io_init_shared; _citrus_iconv_uninit_shared_t io_uninit_shared; _citrus_iconv_init_context_t io_init_context; _citrus_iconv_uninit_context_t io_uninit_context; _citrus_iconv_convert_t io_convert; }; struct _citrus_iconv_shared { struct _citrus_iconv_ops *ci_ops; void *ci_closure; _CITRUS_HASH_ENTRY(_citrus_iconv_shared) ci_hash_entry; TAILQ_ENTRY(_citrus_iconv_shared) ci_tailq_entry; _citrus_module_t ci_module; unsigned int ci_used_count; char *ci_convname; bool ci_discard_ilseq; struct iconv_hooks *ci_hooks; bool ci_ilseq_invalid; }; struct _citrus_iconv { struct _citrus_iconv_shared *cv_shared; void *cv_closure; }; #endif Index: stable/10/lib/libc/iconv/citrus_none.c =================================================================== --- stable/10/lib/libc/iconv/citrus_none.c (revision 282274) +++ stable/10/lib/libc/iconv/citrus_none.c (revision 282275) @@ -1,237 +1,237 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_none.c,v 1.18 2008/06/14 16:01:07 tnozaki Exp $ */ /*- * Copyright (c) 2002 Citrus Project, * Copyright (c) 2010 Gabor Kovesdan , * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_none.h" #include "citrus_stdenc.h" _CITRUS_STDENC_DECLS(NONE); _CITRUS_STDENC_DEF_OPS(NONE); struct _citrus_stdenc_traits _citrus_NONE_stdenc_traits = { 0, /* et_state_size */ 1, /* mb_cur_max */ }; static int _citrus_NONE_stdenc_init(struct _citrus_stdenc * __restrict ce, const void *var __unused, size_t lenvar __unused, struct _citrus_stdenc_traits * __restrict et) { et->et_state_size = 0; et->et_mb_cur_max = 1; ce->ce_closure = NULL; return (0); } static void _citrus_NONE_stdenc_uninit(struct _citrus_stdenc *ce __unused) { } static int _citrus_NONE_stdenc_init_state(struct _citrus_stdenc * __restrict ce __unused, void * __restrict ps __unused) { return (0); } static int _citrus_NONE_stdenc_mbtocs(struct _citrus_stdenc * __restrict ce __unused, - _csid_t *csid, _index_t *idx, const char **s, size_t n, + _csid_t *csid, _index_t *idx, char **s, size_t n, void *ps __unused, size_t *nresult, struct iconv_hooks *hooks) { if (n < 1) { *nresult = (size_t)-2; return (0); } *csid = 0; *idx = (_index_t)(unsigned char)*(*s)++; *nresult = *idx == 0 ? 0 : 1; if ((hooks != NULL) && (hooks->uc_hook != NULL)) hooks->uc_hook((unsigned int)*idx, hooks->data); return (0); } static int _citrus_NONE_stdenc_cstomb(struct _citrus_stdenc * __restrict ce __unused, char *s, size_t n, _csid_t csid, _index_t idx, void *ps __unused, size_t *nresult, struct iconv_hooks *hooks __unused) { if (csid == _CITRUS_CSID_INVALID) { *nresult = 0; return (0); } if (csid != 0) return (EILSEQ); if ((idx & 0x000000FF) == idx) { if (n < 1) { *nresult = (size_t)-1; return (E2BIG); } *s = (char)idx; *nresult = 1; } else if ((idx & 0x0000FFFF) == idx) { if (n < 2) { *nresult = (size_t)-1; return (E2BIG); } s[0] = (char)idx; /* XXX: might be endian dependent */ s[1] = (char)(idx >> 8); *nresult = 2; } else if ((idx & 0x00FFFFFF) == idx) { if (n < 3) { *nresult = (size_t)-1; return (E2BIG); } s[0] = (char)idx; /* XXX: might be endian dependent */ s[1] = (char)(idx >> 8); s[2] = (char)(idx >> 16); *nresult = 3; } else { if (n < 3) { *nresult = (size_t)-1; return (E2BIG); } s[0] = (char)idx; /* XXX: might be endian dependent */ s[1] = (char)(idx >> 8); s[2] = (char)(idx >> 16); s[3] = (char)(idx >> 24); *nresult = 4; } return (0); } static int _citrus_NONE_stdenc_mbtowc(struct _citrus_stdenc * __restrict ce __unused, - _wc_t * __restrict pwc, const char ** __restrict s, size_t n, + _wc_t * __restrict pwc, char ** __restrict s, size_t n, void * __restrict pspriv __unused, size_t * __restrict nresult, struct iconv_hooks *hooks) { if (s == NULL) { *nresult = 0; return (0); } if (n == 0) { *nresult = (size_t)-2; return (0); } if (pwc != NULL) *pwc = (_wc_t)(unsigned char) **s; *nresult = *s == '\0' ? 0 : 1; if ((hooks != NULL) && (hooks->wc_hook != NULL)) hooks->wc_hook(*pwc, hooks->data); return (0); } static int _citrus_NONE_stdenc_wctomb(struct _citrus_stdenc * __restrict ce __unused, char * __restrict s, size_t n, _wc_t wc, void * __restrict pspriv __unused, size_t * __restrict nresult, struct iconv_hooks *hooks __unused) { if ((wc & ~0xFFU) != 0) { *nresult = (size_t)-1; return (EILSEQ); } if (n == 0) { *nresult = (size_t)-1; return (E2BIG); } *nresult = 1; if (s != NULL && n > 0) *s = (char)wc; return (0); } static int _citrus_NONE_stdenc_put_state_reset(struct _citrus_stdenc * __restrict ce __unused, char * __restrict s __unused, size_t n __unused, void * __restrict pspriv __unused, size_t * __restrict nresult) { *nresult = 0; return (0); } static int _citrus_NONE_stdenc_get_state_desc(struct _stdenc * __restrict ce __unused, void * __restrict ps __unused, int id, struct _stdenc_state_desc * __restrict d) { int ret = 0; switch (id) { case _STDENC_SDID_GENERIC: d->u.generic.state = _STDENC_SDGEN_INITIAL; break; default: ret = EOPNOTSUPP; } return (ret); } Index: stable/10/lib/libc/iconv/citrus_stdenc.h =================================================================== --- stable/10/lib/libc/iconv/citrus_stdenc.h (revision 282274) +++ stable/10/lib/libc/iconv/citrus_stdenc.h (revision 282275) @@ -1,124 +1,124 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_stdenc.h,v 1.4 2005/10/29 18:02:04 tshiozak Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #ifndef _CITRUS_STDENC_H_ #define _CITRUS_STDENC_H_ struct _citrus_stdenc; struct _citrus_stdenc_ops; struct _citrus_stdenc_traits; #define _CITRUS_STDENC_SDID_GENERIC 0 struct _citrus_stdenc_state_desc { union { struct { int state; #define _CITRUS_STDENC_SDGEN_UNKNOWN 0 #define _CITRUS_STDENC_SDGEN_INITIAL 1 #define _CITRUS_STDENC_SDGEN_STABLE 2 #define _CITRUS_STDENC_SDGEN_INCOMPLETE_CHAR 3 #define _CITRUS_STDENC_SDGEN_INCOMPLETE_SHIFT 4 } generic; } u; }; #include "citrus_stdenc_local.h" __BEGIN_DECLS int _citrus_stdenc_open(struct _citrus_stdenc * __restrict * __restrict, char const * __restrict, const void * __restrict, size_t); void _citrus_stdenc_close(struct _citrus_stdenc *); __END_DECLS static __inline int _citrus_stdenc_init_state(struct _citrus_stdenc * __restrict ce, void * __restrict ps) { return ((*ce->ce_ops->eo_init_state)(ce, ps)); } static __inline int _citrus_stdenc_mbtocs(struct _citrus_stdenc * __restrict ce, _citrus_csid_t * __restrict csid, _citrus_index_t * __restrict idx, - const char ** __restrict s, size_t n, void * __restrict ps, + char ** __restrict s, size_t n, void * __restrict ps, size_t * __restrict nresult, struct iconv_hooks *hooks) { return ((*ce->ce_ops->eo_mbtocs)(ce, csid, idx, s, n, ps, nresult, hooks)); } static __inline int _citrus_stdenc_cstomb(struct _citrus_stdenc * __restrict ce, char * __restrict s, size_t n, _citrus_csid_t csid, _citrus_index_t idx, void * __restrict ps, size_t * __restrict nresult, struct iconv_hooks *hooks) { return ((*ce->ce_ops->eo_cstomb)(ce, s, n, csid, idx, ps, nresult, hooks)); } static __inline int _citrus_stdenc_wctomb(struct _citrus_stdenc * __restrict ce, char * __restrict s, size_t n, _citrus_wc_t wc, void * __restrict ps, size_t * __restrict nresult, struct iconv_hooks *hooks) { return ((*ce->ce_ops->eo_wctomb)(ce, s, n, wc, ps, nresult, hooks)); } static __inline int _citrus_stdenc_put_state_reset(struct _citrus_stdenc * __restrict ce, char * __restrict s, size_t n, void * __restrict ps, size_t * __restrict nresult) { return ((*ce->ce_ops->eo_put_state_reset)(ce, s, n, ps, nresult)); } static __inline size_t _citrus_stdenc_get_state_size(struct _citrus_stdenc *ce) { return (ce->ce_traits->et_state_size); } static __inline int _citrus_stdenc_get_state_desc(struct _citrus_stdenc * __restrict ce, void * __restrict ps, int id, struct _citrus_stdenc_state_desc * __restrict d) { return ((*ce->ce_ops->eo_get_state_desc)(ce, ps, id, d)); } #endif Index: stable/10/lib/libc/iconv/citrus_stdenc_local.h =================================================================== --- stable/10/lib/libc/iconv/citrus_stdenc_local.h (revision 282274) +++ stable/10/lib/libc/iconv/citrus_stdenc_local.h (revision 282275) @@ -1,162 +1,162 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_stdenc_local.h,v 1.4 2008/02/09 14:56:20 junyoung Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #ifndef _CITRUS_STDENC_LOCAL_H_ #define _CITRUS_STDENC_LOCAL_H_ #include #include "citrus_module.h" #define _CITRUS_STDENC_GETOPS_FUNC_BASE(n) \ int n(struct _citrus_stdenc_ops *, size_t) #define _CITRUS_STDENC_GETOPS_FUNC(_e_) \ _CITRUS_STDENC_GETOPS_FUNC_BASE(_citrus_##_e_##_stdenc_getops) typedef _CITRUS_STDENC_GETOPS_FUNC_BASE((*_citrus_stdenc_getops_t)); #define _CITRUS_STDENC_DECLS(_e_) \ static int _citrus_##_e_##_stdenc_init \ (struct _citrus_stdenc * __restrict, \ const void * __restrict, size_t, \ struct _citrus_stdenc_traits * __restrict); \ static void _citrus_##_e_##_stdenc_uninit(struct _citrus_stdenc *);\ static int _citrus_##_e_##_stdenc_init_state \ (struct _citrus_stdenc * __restrict, \ void * __restrict); \ static int _citrus_##_e_##_stdenc_mbtocs \ (struct _citrus_stdenc * __restrict, \ _citrus_csid_t * __restrict, \ _citrus_index_t * __restrict, \ - const char ** __restrict, size_t, \ + char ** __restrict, size_t, \ void * __restrict, size_t * __restrict, \ struct iconv_hooks *); \ static int _citrus_##_e_##_stdenc_cstomb \ (struct _citrus_stdenc * __restrict, \ char * __restrict, size_t, _citrus_csid_t, \ _citrus_index_t, void * __restrict, \ size_t * __restrict, struct iconv_hooks *); \ static int _citrus_##_e_##_stdenc_mbtowc \ (struct _citrus_stdenc * __restrict, \ _citrus_wc_t * __restrict, \ - const char ** __restrict, size_t, \ + char ** __restrict, size_t, \ void * __restrict, size_t * __restrict, \ struct iconv_hooks *); \ static int _citrus_##_e_##_stdenc_wctomb \ (struct _citrus_stdenc * __restrict, \ char * __restrict, size_t, _citrus_wc_t, \ void * __restrict, size_t * __restrict, \ struct iconv_hooks *); \ static int _citrus_##_e_##_stdenc_put_state_reset \ (struct _citrus_stdenc * __restrict, \ char * __restrict, size_t, void * __restrict, \ size_t * __restrict); \ static int _citrus_##_e_##_stdenc_get_state_desc \ (struct _citrus_stdenc * __restrict, \ void * __restrict, int, \ struct _citrus_stdenc_state_desc * __restrict) #define _CITRUS_STDENC_DEF_OPS(_e_) \ extern struct _citrus_stdenc_ops _citrus_##_e_##_stdenc_ops; \ struct _citrus_stdenc_ops _citrus_##_e_##_stdenc_ops = { \ /* eo_init */ &_citrus_##_e_##_stdenc_init, \ /* eo_uninit */ &_citrus_##_e_##_stdenc_uninit, \ /* eo_init_state */ &_citrus_##_e_##_stdenc_init_state, \ /* eo_mbtocs */ &_citrus_##_e_##_stdenc_mbtocs, \ /* eo_cstomb */ &_citrus_##_e_##_stdenc_cstomb, \ /* eo_mbtowc */ &_citrus_##_e_##_stdenc_mbtowc, \ /* eo_wctomb */ &_citrus_##_e_##_stdenc_wctomb, \ /* eo_put_state_reset */&_citrus_##_e_##_stdenc_put_state_reset,\ /* eo_get_state_desc */ &_citrus_##_e_##_stdenc_get_state_desc \ } typedef int (*_citrus_stdenc_init_t) (struct _citrus_stdenc * __reatrict, const void * __restrict , size_t, struct _citrus_stdenc_traits * __restrict); typedef void (*_citrus_stdenc_uninit_t)(struct _citrus_stdenc * __restrict); typedef int (*_citrus_stdenc_init_state_t) (struct _citrus_stdenc * __restrict, void * __restrict); typedef int (*_citrus_stdenc_mbtocs_t) (struct _citrus_stdenc * __restrict, _citrus_csid_t * __restrict, _citrus_index_t * __restrict, - const char ** __restrict, size_t, + char ** __restrict, size_t, void * __restrict, size_t * __restrict, struct iconv_hooks *); typedef int (*_citrus_stdenc_cstomb_t) (struct _citrus_stdenc *__restrict, char * __restrict, size_t, _citrus_csid_t, _citrus_index_t, void * __restrict, size_t * __restrict, struct iconv_hooks *); typedef int (*_citrus_stdenc_mbtowc_t) (struct _citrus_stdenc * __restrict, _citrus_wc_t * __restrict, - const char ** __restrict, size_t, + char ** __restrict, size_t, void * __restrict, size_t * __restrict, struct iconv_hooks *); typedef int (*_citrus_stdenc_wctomb_t) (struct _citrus_stdenc *__restrict, char * __restrict, size_t, _citrus_wc_t, void * __restrict, size_t * __restrict, struct iconv_hooks *); typedef int (*_citrus_stdenc_put_state_reset_t) (struct _citrus_stdenc *__restrict, char * __restrict, size_t, void * __restrict, size_t * __restrict); typedef int (*_citrus_stdenc_get_state_desc_t) (struct _citrus_stdenc * __restrict, void * __restrict, int, struct _citrus_stdenc_state_desc * __restrict); struct _citrus_stdenc_ops { _citrus_stdenc_init_t eo_init; _citrus_stdenc_uninit_t eo_uninit; _citrus_stdenc_init_state_t eo_init_state; _citrus_stdenc_mbtocs_t eo_mbtocs; _citrus_stdenc_cstomb_t eo_cstomb; _citrus_stdenc_mbtowc_t eo_mbtowc; _citrus_stdenc_wctomb_t eo_wctomb; _citrus_stdenc_put_state_reset_t eo_put_state_reset; /* version 0x00000002 */ _citrus_stdenc_get_state_desc_t eo_get_state_desc; }; struct _citrus_stdenc_traits { /* version 0x00000001 */ size_t et_state_size; size_t et_mb_cur_max; }; struct _citrus_stdenc { /* version 0x00000001 */ struct _citrus_stdenc_ops *ce_ops; void *ce_closure; _citrus_module_t ce_module; struct _citrus_stdenc_traits *ce_traits; }; #define _CITRUS_DEFAULT_STDENC_NAME "NONE" #endif Index: stable/10/lib/libc/iconv/citrus_stdenc_template.h =================================================================== --- stable/10/lib/libc/iconv/citrus_stdenc_template.h (revision 282274) +++ stable/10/lib/libc/iconv/citrus_stdenc_template.h (revision 282275) @@ -1,211 +1,211 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_stdenc_template.h,v 1.4 2008/02/09 14:56:20 junyoung Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include /* * CAUTION: THIS IS NOT STANDALONE FILE * * function templates of iconv standard encoding handler for each encodings. * */ /* * macros */ #undef _TO_EI #undef _CE_TO_EI #undef _TO_STATE #define _TO_EI(_cl_) ((_ENCODING_INFO*)(_cl_)) #define _CE_TO_EI(_ce_) (_TO_EI((_ce_)->ce_closure)) #define _TO_STATE(_ps_) ((_ENCODING_STATE*)(_ps_)) /* ---------------------------------------------------------------------- * templates for public functions */ int _FUNCNAME(stdenc_getops)(struct _citrus_stdenc_ops *ops, size_t lenops __unused) { memcpy(ops, &_FUNCNAME(stdenc_ops), sizeof(_FUNCNAME(stdenc_ops))); return (0); } static int _FUNCNAME(stdenc_init)(struct _citrus_stdenc * __restrict ce, const void * __restrict var, size_t lenvar, struct _citrus_stdenc_traits * __restrict et) { _ENCODING_INFO *ei; int ret; ei = NULL; if (sizeof(_ENCODING_INFO) > 0) { ei = calloc(1, sizeof(_ENCODING_INFO)); if (ei == NULL) return (errno); } ret = _FUNCNAME(encoding_module_init)(ei, var, lenvar); if (ret) { free((void *)ei); return (ret); } ce->ce_closure = ei; et->et_state_size = sizeof(_ENCODING_STATE); et->et_mb_cur_max = _ENCODING_MB_CUR_MAX(_CE_TO_EI(ce)); return (0); } static void _FUNCNAME(stdenc_uninit)(struct _citrus_stdenc * __restrict ce) { if (ce) { _FUNCNAME(encoding_module_uninit)(_CE_TO_EI(ce)); free(ce->ce_closure); } } static int _FUNCNAME(stdenc_init_state)(struct _citrus_stdenc * __restrict ce, void * __restrict ps) { _FUNCNAME(init_state)(_CE_TO_EI(ce), _TO_STATE(ps)); return (0); } static int _FUNCNAME(stdenc_mbtocs)(struct _citrus_stdenc * __restrict ce, _citrus_csid_t * __restrict csid, _citrus_index_t * __restrict idx, - const char ** __restrict s, size_t n, void * __restrict ps, + char ** __restrict s, size_t n, void * __restrict ps, size_t * __restrict nresult, struct iconv_hooks *hooks) { wchar_t wc; int ret; ret = _FUNCNAME(mbrtowc_priv)(_CE_TO_EI(ce), &wc, s, n, _TO_STATE(ps), nresult); if ((ret == 0) && *nresult != (size_t)-2) ret = _FUNCNAME(stdenc_wctocs)(_CE_TO_EI(ce), csid, idx, wc); if ((ret == 0) && (hooks != NULL) && (hooks->uc_hook != NULL)) hooks->uc_hook((unsigned int)*idx, hooks->data); return (ret); } static int _FUNCNAME(stdenc_cstomb)(struct _citrus_stdenc * __restrict ce, char * __restrict s, size_t n, _citrus_csid_t csid, _citrus_index_t idx, void * __restrict ps, size_t * __restrict nresult, struct iconv_hooks *hooks __unused) { wchar_t wc; int ret; wc = ret = 0; if (csid != _CITRUS_CSID_INVALID) ret = _FUNCNAME(stdenc_cstowc)(_CE_TO_EI(ce), &wc, csid, idx); if (ret == 0) ret = _FUNCNAME(wcrtomb_priv)(_CE_TO_EI(ce), s, n, wc, _TO_STATE(ps), nresult); return (ret); } static int _FUNCNAME(stdenc_mbtowc)(struct _citrus_stdenc * __restrict ce, - _citrus_wc_t * __restrict wc, const char ** __restrict s, size_t n, + _citrus_wc_t * __restrict wc, char ** __restrict s, size_t n, void * __restrict ps, size_t * __restrict nresult, struct iconv_hooks *hooks) { int ret; ret = _FUNCNAME(mbrtowc_priv)(_CE_TO_EI(ce), wc, s, n, _TO_STATE(ps), nresult); if ((ret == 0) && (hooks != NULL) && (hooks->wc_hook != NULL)) hooks->wc_hook(*wc, hooks->data); return (ret); } static int _FUNCNAME(stdenc_wctomb)(struct _citrus_stdenc * __restrict ce, char * __restrict s, size_t n, _citrus_wc_t wc, void * __restrict ps, size_t * __restrict nresult, struct iconv_hooks *hooks __unused) { int ret; ret = _FUNCNAME(wcrtomb_priv)(_CE_TO_EI(ce), s, n, wc, _TO_STATE(ps), nresult); return (ret); } static int _FUNCNAME(stdenc_put_state_reset)(struct _citrus_stdenc * __restrict ce __unused, char * __restrict s __unused, size_t n __unused, void * __restrict ps __unused, size_t * __restrict nresult) { #if _ENCODING_IS_STATE_DEPENDENT return ((_FUNCNAME(put_state_reset)(_CE_TO_EI(ce), s, n, _TO_STATE(ps), nresult))); #else *nresult = 0; return (0); #endif } static int _FUNCNAME(stdenc_get_state_desc)(struct _citrus_stdenc * __restrict ce, void * __restrict ps, int id, struct _citrus_stdenc_state_desc * __restrict d) { int ret; switch (id) { case _STDENC_SDID_GENERIC: ret = _FUNCNAME(stdenc_get_state_desc_generic)( _CE_TO_EI(ce), _TO_STATE(ps), &d->u.generic.state); break; default: ret = EOPNOTSUPP; } return (ret); } Index: stable/10/lib/libc/iconv/iconv-internal.h =================================================================== --- stable/10/lib/libc/iconv/iconv-internal.h (revision 282274) +++ stable/10/lib/libc/iconv/iconv-internal.h (revision 282275) @@ -1,45 +1,45 @@ /*- * Copyright (c) 2013 Peter Wemm * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * Interal prototypes for our back-end functions. */ -size_t __bsd___iconv(iconv_t, const char **, size_t *, char **, +size_t __bsd___iconv(iconv_t, char **, size_t *, char **, size_t *, __uint32_t, size_t *); void __bsd___iconv_free_list(char **, size_t); int __bsd___iconv_get_list(char ***, size_t *, __iconv_bool); -size_t __bsd_iconv(iconv_t, const char ** __restrict, +size_t __bsd_iconv(iconv_t, char ** __restrict, size_t * __restrict, char ** __restrict, size_t * __restrict); const char *__bsd_iconv_canonicalize(const char *); int __bsd_iconv_close(iconv_t); iconv_t __bsd_iconv_open(const char *, const char *); int __bsd_iconv_open_into(const char *, const char *, iconv_allocation_t *); void __bsd_iconv_set_relocation_prefix(const char *, const char *); int __bsd_iconvctl(iconv_t, int, void *); void __bsd_iconvlist(int (*) (unsigned int, const char * const *, void *), void *); Index: stable/10/lib/libc/iconv/iconv.3 =================================================================== --- stable/10/lib/libc/iconv/iconv.3 (revision 282274) +++ stable/10/lib/libc/iconv/iconv.3 (revision 282275) @@ -1,305 +1,305 @@ .\" $FreeBSD$ .\" $NetBSD: iconv.3,v 1.12 2004/08/02 13:38:21 tshiozak Exp $ .\" .\" Copyright (c) 2003 Citrus Project, .\" Copyright (c) 2009, 2010 Gabor Kovesdan , .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" 1. Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" 2. Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in the .\" documentation and/or other materials provided with the distribution. .\" .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" .Dd June 16, 2010 .Dt ICONV 3 .Os .Sh NAME .Nm iconv_open , .Nm iconv_open_into , .Nm iconv_close , .Nm iconv .Nd codeset conversion functions .Sh LIBRARY .Lb libc .Sh SYNOPSIS .In iconv.h .Ft iconv_t .Fn iconv_open "const char *dstname" "const char *srcname" .Ft int .Fn iconv_open_into "const char *dstname" "const char *srcname" "iconv_allocation_t *ptr" .Ft int .Fn iconv_close "iconv_t cd" .Ft size_t .Fn iconv "iconv_t cd" "char ** restrict src" "size_t * restrict srcleft" "char ** restrict dst" "size_t * restrict dstleft" .Ft size_t -.Fn __iconv "iconv_t cd" "const char ** restrict src" "size_t * restrict srcleft" "char ** restrict dst" "size_t * restrict dstleft" "uint32_t flags" "size_t invalids" +.Fn __iconv "iconv_t cd" "char ** restrict src" "size_t * restrict srcleft" "char ** restrict dst" "size_t * restrict dstleft" "uint32_t flags" "size_t * invalids" .Sh DESCRIPTION The .Fn iconv_open function opens a converter from the codeset .Fa srcname to the codeset .Fa dstname and returns its descriptor. The arguments .Fa srcname and .Fa dstname accept "" and "char", which refer to the current locale encoding. .Pp The .Fn iconv_open_into creates a conversion descriptor on a preallocated space. The .Ft iconv_allocation_t is used as a spaceholder type when allocating such space. The .Fa dstname and .Fa srcname arguments are the same as in the case of .Fn iconv_open . The .Fa ptr argument is a pointer of .Ft iconv_allocation_t to the preallocated space. .Pp The .Fn iconv_close function closes the specified converter .Fa cd . .Pp The .Fn iconv function converts the string in the buffer .Fa *src of length .Fa *srcleft bytes and stores the converted string in the buffer .Fa *dst of size .Fa *dstleft bytes. After calling .Fn iconv , the values pointed to by .Fa src , .Fa srcleft , .Fa dst , and .Fa dstleft are updated as follows: .Bl -tag -width 01234567 .It *src Pointer to the byte just after the last character fetched. .It *srcleft Number of remaining bytes in the source buffer. .It *dst Pointer to the byte just after the last character stored. .It *dstleft Number of remainder bytes in the destination buffer. .El .Pp If the string pointed to by .Fa *src contains a byte sequence which is not a valid character in the source codeset, the conversion stops just after the last successful conversion. If the output buffer is too small to store the converted character, the conversion also stops in the same way. In these cases, the values pointed to by .Fa src , .Fa srcleft , .Fa dst , and .Fa dstleft are updated to the state just after the last successful conversion. .Pp If the string pointed to by .Fa *src contains a character which is valid under the source codeset but can not be converted to the destination codeset, the character is replaced by an .Dq invalid character which depends on the destination codeset, e.g., .Sq \&? , and the conversion is continued. .Fn iconv returns the number of such .Dq invalid conversions . .Pp There are two special cases of .Fn iconv : .Bl -tag -width 0123 .It "src == NULL || *src == NULL" If the source and/or destination codesets are stateful, .Fn iconv places these into their initial state. .Pp If both .Fa dst and .Fa *dst are .No non- Ns Dv NULL , .Fn iconv stores the shift sequence for the destination switching to the initial state in the buffer pointed to by .Fa *dst . The buffer size is specified by the value pointed to by .Fa dstleft as above. .Fn iconv will fail if the buffer is too small to store the shift sequence. .Pp On the other hand, .Fa dst or .Fa *dst may be .Dv NULL . In this case, the shift sequence for the destination switching to the initial state is discarded. .Pp .El The .Fn __iconv function works just like .Fn iconv but if .Fn iconv fails, the invalid character count is lost there. This is a not bug rather a limitation of .St -p1003.1-2008 , so .Fn __iconv is provided as an alternative but non-standard interface. It also has a flags argument, where currently the following flags can be passed: .Bl -tag -width 0123 .It __ICONV_F_HIDE_INVALID Skip invalid characters, instead of returning with an error. .El .Sh RETURN VALUES Upon successful completion of .Fn iconv_open , it returns a conversion descriptor. Otherwise, .Fn iconv_open returns (iconv_t)\-1 and sets errno to indicate the error. .Pp Upon successful completion of .Fn iconv_open_into , it returns 0. Otherwise, .Fn iconv_open_into returns \-1, and sets errno to indicate the error. .Pp Upon successful completion of .Fn iconv_close , it returns 0. Otherwise, .Fn iconv_close returns \-1 and sets errno to indicate the error. .Pp Upon successful completion of .Fn iconv , it returns the number of .Dq invalid conversions. Otherwise, .Fn iconv returns (size_t)\-1 and sets errno to indicate the error. .Sh ERRORS The .Fn iconv_open function may cause an error in the following cases: .Bl -tag -width Er .It Bq Er ENOMEM Memory is exhausted. .It Bq Er EINVAL There is no converter specified by .Fa srcname and .Fa dstname . .El The .Fn iconv_open_into function may cause an error in the following cases: .Bl -tag -width Er .It Bq Er EINVAL There is no converter specified by .Fa srcname and .Fa dstname . .El .Pp The .Fn iconv_close function may cause an error in the following case: .Bl -tag -width Er .It Bq Er EBADF The conversion descriptor specified by .Fa cd is invalid. .El .Pp The .Fn iconv function may cause an error in the following cases: .Bl -tag -width Er .It Bq Er EBADF The conversion descriptor specified by .Fa cd is invalid. .It Bq Er EILSEQ The string pointed to by .Fa *src contains a byte sequence which does not describe a valid character of the source codeset. .It Bq Er E2BIG The output buffer pointed to by .Fa *dst is too small to store the result string. .It Bq Er EINVAL The string pointed to by .Fa *src terminates with an incomplete character or shift sequence. .El .Sh SEE ALSO .Xr iconv 1 , .Xr mkcsmapper 1 , .Xr mkesdb 1 .Sh STANDARDS The .Fn iconv_open , .Fn iconv_close , and .Fn iconv functions conform to .St -p1003.1-2008 . .Pp The .Fn iconv_open_into function is a GNU-specific extension and it is not part of any standard, thus its use may break portability. The .Fn __iconv function is an own extension and it is not part of any standard, thus its use may break portability. Index: stable/10/lib/libc/iconv/iconv.c =================================================================== --- stable/10/lib/libc/iconv/iconv.c (revision 282274) +++ stable/10/lib/libc/iconv/iconv.c (revision 282275) @@ -1,39 +1,39 @@ /*- * Copyright (c) 2013 Peter Wemm * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ #include #include #include "iconv-internal.h" size_t -iconv(iconv_t a, const char ** __restrict b, +iconv(iconv_t a, char ** __restrict b, size_t * __restrict c, char ** __restrict d, size_t * __restrict e) { return __bsd_iconv(a, b, c, d, e); } Index: stable/10/lib/libc/iconv/iconv_compat.c =================================================================== --- stable/10/lib/libc/iconv/iconv_compat.c (revision 282274) +++ stable/10/lib/libc/iconv/iconv_compat.c (revision 282275) @@ -1,121 +1,121 @@ /*- * Copyright (c) 2013 Peter Wemm * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * These are ABI implementations for when the raw iconv_* symbol * space was exposed via libc.so.7 in its early life. This is * a transition aide, these wrappers will not normally ever be * executed except via __sym_compat() references. */ #include #include #include "iconv-internal.h" size_t -__iconv_compat(iconv_t a, const char ** b, size_t * c, char ** d, +__iconv_compat(iconv_t a, char ** b, size_t * c, char ** d, size_t * e, __uint32_t f, size_t *g) { return __bsd___iconv(a, b, c, d, e, f, g); } void __iconv_free_list_compat(char ** a, size_t b) { __bsd___iconv_free_list(a, b); } int __iconv_get_list_compat(char ***a, size_t *b, __iconv_bool c) { return __bsd___iconv_get_list(a, b, c); } size_t -iconv_compat(iconv_t a, const char ** __restrict b, +iconv_compat(iconv_t a, char ** __restrict b, size_t * __restrict c, char ** __restrict d, size_t * __restrict e) { return __bsd_iconv(a, b, c, d, e); } const char * iconv_canonicalize_compat(const char *a) { return __bsd_iconv_canonicalize(a); } int iconv_close_compat(iconv_t a) { return __bsd_iconv_close(a); } iconv_t iconv_open_compat(const char *a, const char *b) { return __bsd_iconv_open(a, b); } int iconv_open_into_compat(const char *a, const char *b, iconv_allocation_t *c) { return __bsd_iconv_open_into(a, b, c); } void iconv_set_relocation_prefix_compat(const char *a, const char *b) { return __bsd_iconv_set_relocation_prefix(a, b); } int iconvctl_compat(iconv_t a, int b, void *c) { return __bsd_iconvctl(a, b, c); } void iconvlist_compat(int (*a) (unsigned int, const char * const *, void *), void *b) { return __bsd_iconvlist(a, b); } int _iconv_version_compat = 0x0108; /* Magic - not used */ __sym_compat(__iconv, __iconv_compat, FBSD_1.2); __sym_compat(__iconv_free_list, __iconv_free_list_compat, FBSD_1.2); __sym_compat(__iconv_get_list, __iconv_get_list_compat, FBSD_1.2); __sym_compat(_iconv_version, _iconv_version_compat, FBSD_1.3); __sym_compat(iconv, iconv_compat, FBSD_1.3); __sym_compat(iconv_canonicalize, iconv_canonicalize_compat, FBSD_1.2); __sym_compat(iconv_close, iconv_close_compat, FBSD_1.3); __sym_compat(iconv_open, iconv_open_compat, FBSD_1.3); __sym_compat(iconv_open_into, iconv_open_into_compat, FBSD_1.3); __sym_compat(iconv_set_relocation_prefix, iconv_set_relocation_prefix_compat, FBSD_1.3); __sym_compat(iconvctl, iconvctl_compat, FBSD_1.3); __sym_compat(iconvlist, iconvlist_compat, FBSD_1.3); Index: stable/10/lib/libc/locale/cXXrtomb_iconv.h =================================================================== --- stable/10/lib/libc/locale/cXXrtomb_iconv.h (revision 282274) +++ stable/10/lib/libc/locale/cXXrtomb_iconv.h (revision 282275) @@ -1,116 +1,115 @@ /*- * Copyright (c) 2013 Ed Schouten * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include "../iconv/citrus_hash.h" #include "../iconv/citrus_module.h" #include "../iconv/citrus_iconv.h" #include "xlocale_private.h" typedef struct { bool initialized; struct _citrus_iconv iconv; union { charXX_t widechar[SRCBUF_LEN]; char bytes[sizeof(charXX_t) * SRCBUF_LEN]; } srcbuf; size_t srcbuf_len; } _ConversionState; _Static_assert(sizeof(_ConversionState) <= sizeof(mbstate_t), "Size of _ConversionState must not exceed mbstate_t's size."); size_t cXXrtomb_l(char * __restrict s, charXX_t c, mbstate_t * __restrict ps, locale_t locale) { _ConversionState *cs; struct _citrus_iconv *handle; - const char *src; - char *dst; + char *src, *dst; size_t srcleft, dstleft, invlen; int err; FIX_LOCALE(locale); if (ps == NULL) ps = &locale->cXXrtomb; cs = (_ConversionState *)ps; handle = &cs->iconv; /* Reinitialize mbstate_t. */ if (s == NULL || !cs->initialized) { if (_citrus_iconv_open(&handle, UTF_XX_INTERNAL, nl_langinfo_l(CODESET, locale)) != 0) { cs->initialized = false; errno = EINVAL; return (-1); } handle->cv_shared->ci_discard_ilseq = true; handle->cv_shared->ci_hooks = NULL; cs->srcbuf_len = 0; cs->initialized = true; if (s == NULL) return (1); } assert(cs->srcbuf_len < sizeof(cs->srcbuf.widechar) / sizeof(charXX_t)); cs->srcbuf.widechar[cs->srcbuf_len++] = c; /* Perform conversion. */ src = cs->srcbuf.bytes; srcleft = cs->srcbuf_len * sizeof(charXX_t); dst = s; dstleft = MB_CUR_MAX_L(locale); err = _citrus_iconv_convert(handle, &src, &srcleft, &dst, &dstleft, 0, &invlen); /* Character is part of a surrogate pair. We need more input. */ if (err == EINVAL) return (0); cs->srcbuf_len = 0; /* Illegal sequence. */ if (dst == s) { errno = EILSEQ; return ((size_t)-1); } return (dst - s); } size_t cXXrtomb(char * __restrict s, charXX_t c, mbstate_t * __restrict ps) { return (cXXrtomb_l(s, c, ps, __get_locale())); } Index: stable/10/lib/libc/locale/mbrtocXX_iconv.h =================================================================== --- stable/10/lib/libc/locale/mbrtocXX_iconv.h (revision 282274) +++ stable/10/lib/libc/locale/mbrtocXX_iconv.h (revision 282275) @@ -1,159 +1,158 @@ /*- * Copyright (c) 2013 Ed Schouten * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include __FBSDID("$FreeBSD$"); #include #include #include #include #include #include #include #include "../iconv/citrus_hash.h" #include "../iconv/citrus_module.h" #include "../iconv/citrus_iconv.h" #include "xlocale_private.h" typedef struct { bool initialized; struct _citrus_iconv iconv; char srcbuf[MB_LEN_MAX]; size_t srcbuf_len; union { charXX_t widechar[DSTBUF_LEN]; char bytes[sizeof(charXX_t) * DSTBUF_LEN]; } dstbuf; size_t dstbuf_len; } _ConversionState; _Static_assert(sizeof(_ConversionState) <= sizeof(mbstate_t), "Size of _ConversionState must not exceed mbstate_t's size."); size_t mbrtocXX_l(charXX_t * __restrict pc, const char * __restrict s, size_t n, mbstate_t * __restrict ps, locale_t locale) { _ConversionState *cs; struct _citrus_iconv *handle; size_t i, retval; charXX_t retchar; FIX_LOCALE(locale); if (ps == NULL) ps = &locale->mbrtocXX; cs = (_ConversionState *)ps; handle = &cs->iconv; /* Reinitialize mbstate_t. */ if (s == NULL || !cs->initialized) { if (_citrus_iconv_open(&handle, nl_langinfo_l(CODESET, locale), UTF_XX_INTERNAL) != 0) { cs->initialized = false; errno = EINVAL; return (-1); } handle->cv_shared->ci_discard_ilseq = true; handle->cv_shared->ci_hooks = NULL; cs->srcbuf_len = cs->dstbuf_len = 0; cs->initialized = true; if (s == NULL) return (0); } /* See if we still have characters left from the previous invocation. */ if (cs->dstbuf_len > 0) { retval = (size_t)-3; goto return_char; } /* Fill up the read buffer as far as possible. */ if (n > sizeof(cs->srcbuf) - cs->srcbuf_len) n = sizeof(cs->srcbuf) - cs->srcbuf_len; memcpy(cs->srcbuf + cs->srcbuf_len, s, n); /* Convert as few characters to the dst buffer as possible. */ for (i = 0; ; i++) { - const char *src; - char *dst; + char *src, *dst; size_t srcleft, dstleft, invlen; int err; src = cs->srcbuf; srcleft = cs->srcbuf_len + n; dst = cs->dstbuf.bytes; dstleft = i * sizeof(charXX_t); assert(srcleft <= sizeof(cs->srcbuf) && dstleft <= sizeof(cs->dstbuf.bytes)); err = _citrus_iconv_convert(handle, &src, &srcleft, &dst, &dstleft, 0, &invlen); cs->dstbuf_len = (dst - cs->dstbuf.bytes) / sizeof(charXX_t); /* Got new character(s). Return the first. */ if (cs->dstbuf_len > 0) { assert(src - cs->srcbuf > cs->srcbuf_len); retval = src - cs->srcbuf - cs->srcbuf_len; cs->srcbuf_len = 0; goto return_char; } /* Increase dst buffer size, to obtain the surrogate pair. */ if (err == E2BIG) continue; /* Illegal sequence. */ if (invlen > 0) { cs->srcbuf_len = 0; errno = EILSEQ; return ((size_t)-1); } /* Save unprocessed remainder for the next invocation. */ memmove(cs->srcbuf, src, srcleft); cs->srcbuf_len = srcleft; return ((size_t)-2); } return_char: retchar = cs->dstbuf.widechar[0]; memmove(&cs->dstbuf.widechar[0], &cs->dstbuf.widechar[1], --cs->dstbuf_len * sizeof(charXX_t)); if (pc != NULL) *pc = retchar; if (retchar == 0) return (0); return (retval); } size_t mbrtocXX(charXX_t * __restrict pc, const char * __restrict s, size_t n, mbstate_t * __restrict ps) { return (mbrtocXX_l(pc, s, n, ps, __get_locale())); } Index: stable/10/lib/libiconv_modules/BIG5/citrus_big5.c =================================================================== --- stable/10/lib/libiconv_modules/BIG5/citrus_big5.c (revision 282274) +++ stable/10/lib/libiconv_modules/BIG5/citrus_big5.c (revision 282275) @@ -1,458 +1,458 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_big5.c,v 1.13 2011/05/23 14:53:46 joerg Exp $ */ /*- * Copyright (c)2002, 2006 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /*- * Copyright (c) 1993 * The Regents of the University of California. All rights reserved. * * This code is derived from software contributed to Berkeley by * Paul Borman at Krystal Technologies. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_prop.h" #include "citrus_types.h" #include "citrus_bcs.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_big5.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct { int chlen; char ch[2]; } _BIG5State; typedef struct _BIG5Exclude { TAILQ_ENTRY(_BIG5Exclude) entry; wint_t start; wint_t end; } _BIG5Exclude; typedef TAILQ_HEAD(_BIG5ExcludeList, _BIG5Exclude) _BIG5ExcludeList; typedef struct { _BIG5ExcludeList excludes; int cell[0x100]; } _BIG5EncodingInfo; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_BIG5_##m #define _ENCODING_INFO _BIG5EncodingInfo #define _ENCODING_STATE _BIG5State #define _ENCODING_MB_CUR_MAX(_ei_) 2 #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline void /*ARGSUSED*/ _citrus_BIG5_init_state(_BIG5EncodingInfo * __restrict ei __unused, _BIG5State * __restrict s) { memset(s, 0, sizeof(*s)); } #if 0 static __inline void /*ARGSUSED*/ _citrus_BIG5_pack_state(_BIG5EncodingInfo * __restrict ei __unused, void * __restrict pspriv, const _BIG5State * __restrict s) { memcpy(pspriv, (const void *)s, sizeof(*s)); } static __inline void /*ARGSUSED*/ _citrus_BIG5_unpack_state(_BIG5EncodingInfo * __restrict ei __unused, _BIG5State * __restrict s, const void * __restrict pspriv) { memcpy((void *)s, pspriv, sizeof(*s)); } #endif static __inline int _citrus_BIG5_check(_BIG5EncodingInfo *ei, unsigned int c) { return ((ei->cell[c & 0xFF] & 0x1) ? 2 : 1); } static __inline int _citrus_BIG5_check2(_BIG5EncodingInfo *ei, unsigned int c) { return ((ei->cell[c & 0xFF] & 0x2) ? 1 : 0); } static __inline int _citrus_BIG5_check_excludes(_BIG5EncodingInfo *ei, wint_t c) { _BIG5Exclude *exclude; TAILQ_FOREACH(exclude, &ei->excludes, entry) { if (c >= exclude->start && c <= exclude->end) return (EILSEQ); } return (0); } static int _citrus_BIG5_fill_rowcol(void * __restrict ctx, const char * __restrict s, uint64_t start, uint64_t end) { _BIG5EncodingInfo *ei; uint64_t n; int i; if (start > 0xFF || end > 0xFF) return (EINVAL); ei = (_BIG5EncodingInfo *)ctx; i = strcmp("row", s) ? 1 : 0; i = 1 << i; for (n = start; n <= end; ++n) ei->cell[n & 0xFF] |= i; return (0); } static int /*ARGSUSED*/ _citrus_BIG5_fill_excludes(void * __restrict ctx, const char * __restrict s __unused, uint64_t start, uint64_t end) { _BIG5EncodingInfo *ei; _BIG5Exclude *exclude; if (start > 0xFFFF || end > 0xFFFF) return (EINVAL); ei = (_BIG5EncodingInfo *)ctx; exclude = TAILQ_LAST(&ei->excludes, _BIG5ExcludeList); if (exclude != NULL && (wint_t)start <= exclude->end) return (EINVAL); exclude = (void *)malloc(sizeof(*exclude)); if (exclude == NULL) return (ENOMEM); exclude->start = (wint_t)start; exclude->end = (wint_t)end; TAILQ_INSERT_TAIL(&ei->excludes, exclude, entry); return (0); } static const _citrus_prop_hint_t root_hints[] = { _CITRUS_PROP_HINT_NUM("row", &_citrus_BIG5_fill_rowcol), _CITRUS_PROP_HINT_NUM("col", &_citrus_BIG5_fill_rowcol), _CITRUS_PROP_HINT_NUM("excludes", &_citrus_BIG5_fill_excludes), _CITRUS_PROP_HINT_END }; static void /*ARGSUSED*/ _citrus_BIG5_encoding_module_uninit(_BIG5EncodingInfo *ei) { _BIG5Exclude *exclude; while ((exclude = TAILQ_FIRST(&ei->excludes)) != NULL) { TAILQ_REMOVE(&ei->excludes, exclude, entry); free(exclude); } } static int /*ARGSUSED*/ _citrus_BIG5_encoding_module_init(_BIG5EncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { const char *s; int err; memset((void *)ei, 0, sizeof(*ei)); TAILQ_INIT(&ei->excludes); if (lenvar > 0 && var != NULL) { s = _bcs_skip_ws_len((const char *)var, &lenvar); if (lenvar > 0 && *s != '\0') { err = _citrus_prop_parse_variable( root_hints, (void *)ei, s, lenvar); if (err == 0) return (0); _citrus_BIG5_encoding_module_uninit(ei); memset((void *)ei, 0, sizeof(*ei)); TAILQ_INIT(&ei->excludes); } } /* fallback Big5-1984, for backward compatibility. */ _citrus_BIG5_fill_rowcol(ei, "row", 0xA1, 0xFE); _citrus_BIG5_fill_rowcol(ei, "col", 0x40, 0x7E); _citrus_BIG5_fill_rowcol(ei, "col", 0xA1, 0xFE); return (0); } static int /*ARGSUSED*/ _citrus_BIG5_mbrtowc_priv(_BIG5EncodingInfo * __restrict ei, wchar_t * __restrict pwc, - const char ** __restrict s, size_t n, + char ** __restrict s, size_t n, _BIG5State * __restrict psenc, size_t * __restrict nresult) { wchar_t wchar; - const char *s0; + char *s0; int c, chlenbak; s0 = *s; if (s0 == NULL) { _citrus_BIG5_init_state(ei, psenc); *nresult = 0; return (0); } chlenbak = psenc->chlen; /* make sure we have the first byte in the buffer */ switch (psenc->chlen) { case 0: if (n < 1) goto restart; psenc->ch[0] = *s0++; psenc->chlen = 1; n--; break; case 1: break; default: /* illegal state */ goto ilseq; } c = _citrus_BIG5_check(ei, psenc->ch[0] & 0xff); if (c == 0) goto ilseq; while (psenc->chlen < c) { if (n < 1) { goto restart; } psenc->ch[psenc->chlen] = *s0++; psenc->chlen++; n--; } switch (c) { case 1: wchar = psenc->ch[0] & 0xff; break; case 2: if (!_citrus_BIG5_check2(ei, psenc->ch[1] & 0xff)) goto ilseq; wchar = ((psenc->ch[0] & 0xff) << 8) | (psenc->ch[1] & 0xff); break; default: /* illegal state */ goto ilseq; } if (_citrus_BIG5_check_excludes(ei, (wint_t)wchar) != 0) goto ilseq; *s = s0; psenc->chlen = 0; if (pwc) *pwc = wchar; *nresult = wchar ? c - chlenbak : 0; return (0); ilseq: psenc->chlen = 0; *nresult = (size_t)-1; return (EILSEQ); restart: *s = s0; *nresult = (size_t)-2; return (0); } static int /*ARGSUSED*/ _citrus_BIG5_wcrtomb_priv(_BIG5EncodingInfo * __restrict ei, char * __restrict s, size_t n, wchar_t wc, _BIG5State * __restrict psenc __unused, size_t * __restrict nresult) { size_t l; int ret; /* check invalid sequence */ if (wc & ~0xffff || _citrus_BIG5_check_excludes(ei, (wint_t)wc) != 0) { ret = EILSEQ; goto err; } if (wc & 0x8000) { if (_citrus_BIG5_check(ei, (wc >> 8) & 0xff) != 2 || !_citrus_BIG5_check2(ei, wc & 0xff)) { ret = EILSEQ; goto err; } l = 2; } else { if (wc & ~0xff || !_citrus_BIG5_check(ei, wc & 0xff)) { ret = EILSEQ; goto err; } l = 1; } if (n < l) { /* bound check failure */ ret = E2BIG; goto err; } if (l == 2) { s[0] = (wc >> 8) & 0xff; s[1] = wc & 0xff; } else s[0] = wc & 0xff; *nresult = l; return (0); err: *nresult = (size_t)-1; return (ret); } static __inline int /*ARGSUSED*/ _citrus_BIG5_stdenc_wctocs(_BIG5EncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { *csid = (wc < 0x100) ? 0 : 1; *idx = (_index_t)wc; return (0); } static __inline int /*ARGSUSED*/ _citrus_BIG5_stdenc_cstowc(_BIG5EncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { switch (csid) { case 0: case 1: *wc = (wchar_t)idx; break; default: return (EILSEQ); } return (0); } static __inline int /*ARGSUSED*/ _citrus_BIG5_stdenc_get_state_desc_generic(_BIG5EncodingInfo * __restrict ei __unused, _BIG5State * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(BIG5); _CITRUS_STDENC_DEF_OPS(BIG5); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/DECHanyu/citrus_dechanyu.c =================================================================== --- stable/10/lib/libiconv_modules/DECHanyu/citrus_dechanyu.c (revision 282274) +++ stable/10/lib/libiconv_modules/DECHanyu/citrus_dechanyu.c (revision 282275) @@ -1,394 +1,394 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_dechanyu.c,v 1.4 2011/11/19 18:20:13 tnozaki Exp $ */ /*- * Copyright (c)2007 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_bcs.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_dechanyu.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct { size_t chlen; char ch[4]; } _DECHanyuState; typedef struct { int dummy; } _DECHanyuEncodingInfo; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.__CONCAT(s_,_func_) #define _FUNCNAME(m) __CONCAT(_citrus_DECHanyu_,m) #define _ENCODING_INFO _DECHanyuEncodingInfo #define _ENCODING_STATE _DECHanyuState #define _ENCODING_MB_CUR_MAX(_ei_) 4 #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline void /*ARGSUSED*/ _citrus_DECHanyu_init_state(_DECHanyuEncodingInfo * __restrict ei __unused, _DECHanyuState * __restrict psenc) { psenc->chlen = 0; } #if 0 static __inline void /*ARGSUSED*/ _citrus_DECHanyu_pack_state(_DECHanyuEncodingInfo * __restrict ei __unused, void * __restrict pspriv, const _DECHanyuState * __restrict psenc) { memcpy(pspriv, (const void *)psenc, sizeof(*psenc)); } static __inline void /*ARGSUSED*/ _citrus_DECHanyu_unpack_state(_DECHanyuEncodingInfo * __restrict ei __unused, _DECHanyuState * __restrict psenc, const void * __restrict pspriv) { memcpy((void *)psenc, pspriv, sizeof(*psenc)); } #endif static void /*ARGSUSED*/ _citrus_DECHanyu_encoding_module_uninit(_DECHanyuEncodingInfo *ei __unused) { /* ei may be null */ } static int /*ARGSUSED*/ _citrus_DECHanyu_encoding_module_init(_DECHanyuEncodingInfo * __restrict ei __unused, const void * __restrict var __unused, size_t lenvar __unused) { /* ei may be null */ return (0); } static __inline bool is_singlebyte(int c) { return (c <= 0x7F); } static __inline bool is_leadbyte(int c) { return (c >= 0xA1 && c <= 0xFE); } static __inline bool is_trailbyte(int c) { c &= ~0x80; return (c >= 0x21 && c <= 0x7E); } static __inline bool is_hanyu1(int c) { return (c == 0xC2); } static __inline bool is_hanyu2(int c) { return (c == 0xCB); } #define HANYUBIT 0xC2CB0000 static __inline bool is_94charset(int c) { return (c >= 0x21 && c <= 0x7E); } static int /*ARGSUSED*/ _citrus_DECHanyu_mbrtowc_priv(_DECHanyuEncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, size_t n, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _DECHanyuState * __restrict psenc, size_t * __restrict nresult) { - const char *s0; + char *s0; wchar_t wc; int ch; if (*s == NULL) { _citrus_DECHanyu_init_state(ei, psenc); *nresult = _ENCODING_IS_STATE_DEPENDENT; return (0); } s0 = *s; wc = (wchar_t)0; switch (psenc->chlen) { case 0: if (n-- < 1) goto restart; ch = *s0++ & 0xFF; if (is_singlebyte(ch)) { if (pwc != NULL) *pwc = (wchar_t)ch; *nresult = (size_t)((ch == 0) ? 0 : 1); *s = s0; return (0); } if (!is_leadbyte(ch)) goto ilseq; psenc->ch[psenc->chlen++] = ch; break; case 1: ch = psenc->ch[0] & 0xFF; if (!is_leadbyte(ch)) return (EINVAL); break; case 2: case 3: ch = psenc->ch[0] & 0xFF; if (is_hanyu1(ch)) { ch = psenc->ch[1] & 0xFF; if (is_hanyu2(ch)) { wc |= (wchar_t)HANYUBIT; break; } } /*FALLTHROUGH*/ default: return (EINVAL); } switch (psenc->chlen) { case 1: if (is_hanyu1(ch)) { if (n-- < 1) goto restart; ch = *s0++ & 0xFF; if (!is_hanyu2(ch)) goto ilseq; psenc->ch[psenc->chlen++] = ch; wc |= (wchar_t)HANYUBIT; if (n-- < 1) goto restart; ch = *s0++ & 0xFF; if (!is_leadbyte(ch)) goto ilseq; psenc->ch[psenc->chlen++] = ch; } break; case 2: if (n-- < 1) goto restart; ch = *s0++ & 0xFF; if (!is_leadbyte(ch)) goto ilseq; psenc->ch[psenc->chlen++] = ch; break; case 3: ch = psenc->ch[2] & 0xFF; if (!is_leadbyte(ch)) return (EINVAL); } if (n-- < 1) goto restart; wc |= (wchar_t)(ch << 8); ch = *s0++ & 0xFF; if (!is_trailbyte(ch)) goto ilseq; wc |= (wchar_t)ch; if (pwc != NULL) *pwc = wc; *nresult = (size_t)(s0 - *s); *s = s0; psenc->chlen = 0; return (0); restart: *nresult = (size_t)-2; *s = s0; return (0); ilseq: *nresult = (size_t)-1; return (EILSEQ); } static int /*ARGSUSED*/ _citrus_DECHanyu_wcrtomb_priv(_DECHanyuEncodingInfo * __restrict ei __unused, char * __restrict s, size_t n, wchar_t wc, _DECHanyuState * __restrict psenc, size_t * __restrict nresult) { int ch; if (psenc->chlen != 0) return (EINVAL); /* XXX: assume wchar_t as int */ if ((uint32_t)wc <= 0x7F) { ch = wc & 0xFF; } else { if ((uint32_t)wc > 0xFFFF) { if ((wc & ~0xFFFF) != (wchar_t)HANYUBIT) goto ilseq; psenc->ch[psenc->chlen++] = (wc >> 24) & 0xFF; psenc->ch[psenc->chlen++] = (wc >> 16) & 0xFF; wc &= 0xFFFF; } ch = (wc >> 8) & 0xFF; if (!is_leadbyte(ch)) goto ilseq; psenc->ch[psenc->chlen++] = ch; ch = wc & 0xFF; if (!is_trailbyte(ch)) goto ilseq; } psenc->ch[psenc->chlen++] = ch; if (n < psenc->chlen) { *nresult = (size_t)-1; return (E2BIG); } memcpy(s, psenc->ch, psenc->chlen); *nresult = psenc->chlen; psenc->chlen = 0; return (0); ilseq: *nresult = (size_t)-1; return (EILSEQ); } static __inline int /*ARGSUSED*/ _citrus_DECHanyu_stdenc_wctocs(_DECHanyuEncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { wchar_t mask; int plane; plane = 0; mask = 0x7F; /* XXX: assume wchar_t as int */ if ((uint32_t)wc > 0x7F) { if ((uint32_t)wc > 0xFFFF) { if ((wc & ~0xFFFF) != (wchar_t)HANYUBIT) return (EILSEQ); plane += 2; } if (!is_leadbyte((wc >> 8) & 0xFF) || !is_trailbyte(wc & 0xFF)) return (EILSEQ); plane += (wc & 0x80) ? 1 : 2; mask |= 0x7F00; } *csid = plane; *idx = (_index_t)(wc & mask); return (0); } static __inline int /*ARGSUSED*/ _citrus_DECHanyu_stdenc_cstowc(_DECHanyuEncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { if (csid == 0) { if (idx > 0x7F) return (EILSEQ); } else if (csid <= 4) { if (!is_94charset(idx >> 8)) return (EILSEQ); if (!is_94charset(idx & 0xFF)) return (EILSEQ); if (csid % 2) idx |= 0x80; idx |= 0x8000; if (csid > 2) idx |= HANYUBIT; } else return (EILSEQ); *wc = (wchar_t)idx; return (0); } static __inline int /*ARGSUSED*/ _citrus_DECHanyu_stdenc_get_state_desc_generic( _DECHanyuEncodingInfo * __restrict ei __unused, _DECHanyuState * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(DECHanyu); _CITRUS_STDENC_DEF_OPS(DECHanyu); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/EUC/citrus_euc.c =================================================================== --- stable/10/lib/libiconv_modules/EUC/citrus_euc.c (revision 282274) +++ stable/10/lib/libiconv_modules/EUC/citrus_euc.c (revision 282275) @@ -1,387 +1,387 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_euc.c,v 1.14 2009/01/11 02:46:24 christos Exp $ */ /*- * Copyright (c)2002 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /*- * Copyright (c) 1993 * The Regents of the University of California. All rights reserved. * * This code is derived from software contributed to Berkeley by * Paul Borman at Krystal Technologies. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_bcs.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_euc.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct { int chlen; char ch[3]; } _EUCState; typedef struct { wchar_t bits[4]; wchar_t mask; unsigned count[4]; unsigned mb_cur_max; } _EUCEncodingInfo; #define _SS2 0x008e #define _SS3 0x008f #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_EUC_##m #define _ENCODING_INFO _EUCEncodingInfo #define _ENCODING_STATE _EUCState #define _ENCODING_MB_CUR_MAX(_ei_) (_ei_)->mb_cur_max #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline int _citrus_EUC_cs(unsigned int c) { c &= 0xff; return ((c & 0x80) ? c == _SS3 ? 3 : c == _SS2 ? 2 : 1 : 0); } static __inline int _citrus_EUC_parse_variable(_EUCEncodingInfo *ei, const void *var, size_t lenvar __unused) { char *e; const char *v; int x; /* parse variable string */ if (!var) return (EFTYPE); v = (const char *)var; while (*v == ' ' || *v == '\t') ++v; ei->mb_cur_max = 1; for (x = 0; x < 4; ++x) { ei->count[x] = (int)_bcs_strtol(v, (char **)&e, 0); if (v == e || !(v = e) || ei->count[x] < 1 || ei->count[x] > 4) { return (EFTYPE); } if (ei->mb_cur_max < ei->count[x]) ei->mb_cur_max = ei->count[x]; while (*v == ' ' || *v == '\t') ++v; ei->bits[x] = (int)_bcs_strtol(v, (char **)&e, 0); if (v == e || !(v = e)) { return (EFTYPE); } while (*v == ' ' || *v == '\t') ++v; } ei->mask = (int)_bcs_strtol(v, (char **)&e, 0); if (v == e || !(v = e)) { return (EFTYPE); } return (0); } static __inline void /*ARGSUSED*/ _citrus_EUC_init_state(_EUCEncodingInfo *ei __unused, _EUCState *s) { memset(s, 0, sizeof(*s)); } #if 0 static __inline void /*ARGSUSED*/ _citrus_EUC_pack_state(_EUCEncodingInfo *ei __unused, void *pspriv, const _EUCState *s) { memcpy(pspriv, (const void *)s, sizeof(*s)); } static __inline void /*ARGSUSED*/ _citrus_EUC_unpack_state(_EUCEncodingInfo *ei __unused, _EUCState *s, const void *pspriv) { memcpy((void *)s, pspriv, sizeof(*s)); } #endif static int -_citrus_EUC_mbrtowc_priv(_EUCEncodingInfo *ei, wchar_t *pwc, const char **s, +_citrus_EUC_mbrtowc_priv(_EUCEncodingInfo *ei, wchar_t *pwc, char **s, size_t n, _EUCState *psenc, size_t *nresult) { wchar_t wchar; int c, chlenbak, cs, len; - const char *s0, *s1 = NULL; + char *s0, *s1 = NULL; s0 = *s; if (s0 == NULL) { _citrus_EUC_init_state(ei, psenc); *nresult = 0; /* state independent */ return (0); } chlenbak = psenc->chlen; /* make sure we have the first byte in the buffer */ switch (psenc->chlen) { case 0: if (n < 1) goto restart; psenc->ch[0] = *s0++; psenc->chlen = 1; n--; break; case 1: case 2: break; default: /* illgeal state */ goto encoding_error; } c = ei->count[cs = _citrus_EUC_cs(psenc->ch[0] & 0xff)]; if (c == 0) goto encoding_error; while (psenc->chlen < c) { if (n < 1) goto restart; psenc->ch[psenc->chlen] = *s0++; psenc->chlen++; n--; } *s = s0; switch (cs) { case 3: case 2: /* skip SS2/SS3 */ len = c - 1; s1 = &psenc->ch[1]; break; case 1: case 0: len = c; s1 = &psenc->ch[0]; break; default: goto encoding_error; } wchar = 0; while (len-- > 0) wchar = (wchar << 8) | (*s1++ & 0xff); wchar = (wchar & ~ei->mask) | ei->bits[cs]; psenc->chlen = 0; if (pwc) *pwc = wchar; *nresult = wchar ? (size_t)(c - chlenbak) : 0; return (0); encoding_error: psenc->chlen = 0; *nresult = (size_t)-1; return (EILSEQ); restart: *nresult = (size_t)-2; *s = s0; return (0); } static int _citrus_EUC_wcrtomb_priv(_EUCEncodingInfo *ei, char *s, size_t n, wchar_t wc, _EUCState *psenc __unused, size_t *nresult) { wchar_t m, nm; unsigned int cs; int ret; short i; m = wc & ei->mask; nm = wc & ~m; for (cs = 0; cs < sizeof(ei->count) / sizeof(ei->count[0]); cs++) if (m == ei->bits[cs]) break; /* fallback case - not sure if it is necessary */ if (cs == sizeof(ei->count) / sizeof(ei->count[0])) cs = 1; i = ei->count[cs]; if (n < (unsigned)i) { ret = E2BIG; goto err; } m = (cs) ? 0x80 : 0x00; switch (cs) { case 2: *s++ = _SS2; i--; break; case 3: *s++ = _SS3; i--; break; } while (i-- > 0) *s++ = ((nm >> (i << 3)) & 0xff) | m; *nresult = (size_t)ei->count[cs]; return (0); err: *nresult = (size_t)-1; return (ret); } static __inline int /*ARGSUSED*/ _citrus_EUC_stdenc_wctocs(_EUCEncodingInfo * __restrict ei, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { wchar_t m, nm; m = wc & ei->mask; nm = wc & ~m; *csid = (_citrus_csid_t)m; *idx = (_citrus_index_t)nm; return (0); } static __inline int /*ARGSUSED*/ _citrus_EUC_stdenc_cstowc(_EUCEncodingInfo * __restrict ei, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { if ((csid & ~ei->mask) != 0 || (idx & ei->mask) != 0) return (EINVAL); *wc = (wchar_t)csid | (wchar_t)idx; return (0); } static __inline int /*ARGSUSED*/ _citrus_EUC_stdenc_get_state_desc_generic(_EUCEncodingInfo * __restrict ei __unused, _EUCState * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } static int /*ARGSUSED*/ _citrus_EUC_encoding_module_init(_EUCEncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { return (_citrus_EUC_parse_variable(ei, var, lenvar)); } static void /*ARGSUSED*/ _citrus_EUC_encoding_module_uninit(_EUCEncodingInfo * __restrict ei __unused) { } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(EUC); _CITRUS_STDENC_DEF_OPS(EUC); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/EUCTW/citrus_euctw.c =================================================================== --- stable/10/lib/libiconv_modules/EUCTW/citrus_euctw.c (revision 282274) +++ stable/10/lib/libiconv_modules/EUCTW/citrus_euctw.c (revision 282275) @@ -1,379 +1,379 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_euctw.c,v 1.11 2008/06/14 16:01:07 tnozaki Exp $ */ /*- * Copyright (c)2002 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /*- * Copyright (c)1999 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $Citrus: xpg4dl/FreeBSD/lib/libc/locale/euctw.c,v 1.13 2001/06/21 01:51:44 yamt Exp $ */ #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_euctw.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct { int chlen; char ch[4]; } _EUCTWState; typedef struct { int dummy; } _EUCTWEncodingInfo; #define _SS2 0x008e #define _SS3 0x008f #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_EUCTW_##m #define _ENCODING_INFO _EUCTWEncodingInfo #define _ENCODING_STATE _EUCTWState #define _ENCODING_MB_CUR_MAX(_ei_) 4 #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline int _citrus_EUCTW_cs(unsigned int c) { c &= 0xff; return ((c & 0x80) ? (c == _SS2 ? 2 : 1) : 0); } static __inline int _citrus_EUCTW_count(int cs) { switch (cs) { case 0: /*FALLTHROUGH*/ case 1: /*FALLTHROUGH*/ case 2: return (1 << cs); case 3: abort(); /*NOTREACHED*/ } return (0); } static __inline void /*ARGSUSED*/ _citrus_EUCTW_init_state(_EUCTWEncodingInfo * __restrict ei __unused, _EUCTWState * __restrict s) { memset(s, 0, sizeof(*s)); } #if 0 static __inline void /*ARGSUSED*/ _citrus_EUCTW_pack_state(_EUCTWEncodingInfo * __restrict ei __unused, void * __restrict pspriv, const _EUCTWState * __restrict s) { memcpy(pspriv, (const void *)s, sizeof(*s)); } static __inline void /*ARGSUSED*/ _citrus_EUCTW_unpack_state(_EUCTWEncodingInfo * __restrict ei __unused, _EUCTWState * __restrict s, const void * __restrict pspriv) { memcpy((void *)s, pspriv, sizeof(*s)); } #endif static int /*ARGSUSED*/ _citrus_EUCTW_encoding_module_init(_EUCTWEncodingInfo * __restrict ei, const void * __restrict var __unused, size_t lenvar __unused) { memset((void *)ei, 0, sizeof(*ei)); return (0); } static void /*ARGSUSED*/ _citrus_EUCTW_encoding_module_uninit(_EUCTWEncodingInfo *ei __unused) { } static int _citrus_EUCTW_mbrtowc_priv(_EUCTWEncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _EUCTWState * __restrict psenc, size_t * __restrict nresult) { - const char *s0; + char *s0; wchar_t wchar; int c, chlenbak, cs; s0 = *s; if (s0 == NULL) { _citrus_EUCTW_init_state(ei, psenc); *nresult = 0; /* state independent */ return (0); } chlenbak = psenc->chlen; /* make sure we have the first byte in the buffer */ switch (psenc->chlen) { case 0: if (n < 1) goto restart; psenc->ch[0] = *s0++; psenc->chlen = 1; n--; break; case 1: case 2: break; default: /* illgeal state */ goto ilseq; } c = _citrus_EUCTW_count(cs = _citrus_EUCTW_cs(psenc->ch[0] & 0xff)); if (c == 0) goto ilseq; while (psenc->chlen < c) { if (n < 1) goto ilseq; psenc->ch[psenc->chlen] = *s0++; psenc->chlen++; n--; } wchar = 0; switch (cs) { case 0: if (psenc->ch[0] & 0x80) goto ilseq; wchar = psenc->ch[0] & 0xff; break; case 1: if (!(psenc->ch[0] & 0x80) || !(psenc->ch[1] & 0x80)) goto ilseq; wchar = ((psenc->ch[0] & 0xff) << 8) | (psenc->ch[1] & 0xff); wchar |= 'G' << 24; break; case 2: if ((unsigned char)psenc->ch[1] < 0xa1 || 0xa7 < (unsigned char)psenc->ch[1]) goto ilseq; if (!(psenc->ch[2] & 0x80) || !(psenc->ch[3] & 0x80)) goto ilseq; wchar = ((psenc->ch[2] & 0xff) << 8) | (psenc->ch[3] & 0xff); wchar |= ('G' + psenc->ch[1] - 0xa1) << 24; break; default: goto ilseq; } *s = s0; psenc->chlen = 0; if (pwc) *pwc = wchar; *nresult = wchar ? c - chlenbak : 0; return (0); ilseq: psenc->chlen = 0; *nresult = (size_t)-1; return (EILSEQ); restart: *s = s0; *nresult = (size_t)-1; return (0); } static int _citrus_EUCTW_wcrtomb_priv(_EUCTWEncodingInfo * __restrict ei __unused, char * __restrict s, size_t n, wchar_t wc, _EUCTWState * __restrict psenc __unused, size_t * __restrict nresult) { wchar_t cs, v; int clen, i, ret; size_t len; cs = wc & 0x7f000080; clen = 1; if (wc & 0x00007f00) clen = 2; if ((wc & 0x007f0000) && !(wc & 0x00800000)) clen = 3; if (clen == 1 && cs == 0x00000000) { /* ASCII */ len = 1; if (n < len) { ret = E2BIG; goto err; } v = wc & 0x0000007f; } else if (clen == 2 && cs == ('G' << 24)) { /* CNS-11643-1 */ len = 2; if (n < len) { ret = E2BIG; goto err; } v = wc & 0x00007f7f; v |= 0x00008080; } else if (clen == 2 && 'H' <= (cs >> 24) && (cs >> 24) <= 'M') { /* CNS-11643-[2-7] */ len = 4; if (n < len) { ret = E2BIG; goto err; } *s++ = _SS2; *s++ = (cs >> 24) - 'H' + 0xa2; v = wc & 0x00007f7f; v |= 0x00008080; } else { ret = EILSEQ; goto err; } i = clen; while (i-- > 0) *s++ = (v >> (i << 3)) & 0xff; *nresult = len; return (0); err: *nresult = (size_t)-1; return (ret); } static __inline int /*ARGSUSED*/ _citrus_EUCTW_stdenc_wctocs(_EUCTWEncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { *csid = (_csid_t)(wc >> 24) & 0xFF; *idx = (_index_t)(wc & 0x7F7F); return (0); } static __inline int /*ARGSUSED*/ _citrus_EUCTW_stdenc_cstowc(_EUCTWEncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { if (csid == 0) { if ((idx & ~0x7F) != 0) return (EINVAL); *wc = (wchar_t)idx; } else { if (csid < 'G' || csid > 'M' || (idx & ~0x7F7F) != 0) return (EINVAL); *wc = (wchar_t)idx | ((wchar_t)csid << 24); } return (0); } static __inline int /*ARGSUSED*/ _citrus_EUCTW_stdenc_get_state_desc_generic(_EUCTWEncodingInfo * __restrict ei __unused, _EUCTWState * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(EUCTW); _CITRUS_STDENC_DEF_OPS(EUCTW); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/GBK2K/citrus_gbk2k.c =================================================================== --- stable/10/lib/libiconv_modules/GBK2K/citrus_gbk2k.c (revision 282274) +++ stable/10/lib/libiconv_modules/GBK2K/citrus_gbk2k.c (revision 282275) @@ -1,419 +1,419 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_gbk2k.c,v 1.7 2008/06/14 16:01:07 tnozaki Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_bcs.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_gbk2k.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct _GBK2KState { int chlen; char ch[4]; } _GBK2KState; typedef struct { int mb_cur_max; } _GBK2KEncodingInfo; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_GBK2K_##m #define _ENCODING_INFO _GBK2KEncodingInfo #define _ENCODING_STATE _GBK2KState #define _ENCODING_MB_CUR_MAX(_ei_) (_ei_)->mb_cur_max #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline void /*ARGSUSED*/ _citrus_GBK2K_init_state(_GBK2KEncodingInfo * __restrict ei __unused, _GBK2KState * __restrict s) { memset(s, 0, sizeof(*s)); } #if 0 static __inline void /*ARGSUSED*/ _citrus_GBK2K_pack_state(_GBK2KEncodingInfo * __restrict ei __unused, void * __restrict pspriv, const _GBK2KState * __restrict s) { memcpy(pspriv, (const void *)s, sizeof(*s)); } static __inline void /*ARGSUSED*/ _citrus_GBK2K_unpack_state(_GBK2KEncodingInfo * __restrict ei __unused, _GBK2KState * __restrict s, const void * __restrict pspriv) { memcpy((void *)s, pspriv, sizeof(*s)); } #endif static __inline bool _mb_singlebyte(int c) { return ((c & 0xff) <= 0x7f); } static __inline bool _mb_leadbyte(int c) { c &= 0xff; return (0x81 <= c && c <= 0xfe); } static __inline bool _mb_trailbyte(int c) { c &= 0xff; return ((0x40 <= c && c <= 0x7e) || (0x80 <= c && c <= 0xfe)); } static __inline bool _mb_surrogate(int c) { c &= 0xff; return (0x30 <= c && c <= 0x39); } static __inline int _mb_count(wchar_t v) { uint32_t c; c = (uint32_t)v; /* XXX */ if (!(c & 0xffffff00)) return (1); if (!(c & 0xffff0000)) return (2); return (4); } #define _PSENC (psenc->ch[psenc->chlen - 1]) #define _PUSH_PSENC(c) (psenc->ch[psenc->chlen++] = (c)) static int _citrus_GBK2K_mbrtowc_priv(_GBK2KEncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, size_t n, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _GBK2KState * __restrict psenc, size_t * __restrict nresult) { - const char *s0, *s1; + char *s0, *s1; wchar_t wc; int chlenbak, len; s0 = *s; if (s0 == NULL) { /* _citrus_GBK2K_init_state(ei, psenc); */ psenc->chlen = 0; *nresult = 0; return (0); } chlenbak = psenc->chlen; switch (psenc->chlen) { case 3: if (!_mb_leadbyte (_PSENC)) goto invalid; /* FALLTHROUGH */ case 2: if (!_mb_surrogate(_PSENC) || _mb_trailbyte(_PSENC)) goto invalid; /* FALLTHROUGH */ case 1: if (!_mb_leadbyte (_PSENC)) goto invalid; /* FALLTHOROUGH */ case 0: break; default: goto invalid; } for (;;) { if (n-- < 1) goto restart; _PUSH_PSENC(*s0++); switch (psenc->chlen) { case 1: if (_mb_singlebyte(_PSENC)) goto convert; if (_mb_leadbyte (_PSENC)) continue; goto ilseq; case 2: if (_mb_trailbyte (_PSENC)) goto convert; if (ei->mb_cur_max == 4 && _mb_surrogate (_PSENC)) continue; goto ilseq; case 3: if (_mb_leadbyte (_PSENC)) continue; goto ilseq; case 4: if (_mb_surrogate (_PSENC)) goto convert; goto ilseq; } } convert: len = psenc->chlen; s1 = &psenc->ch[0]; wc = 0; while (len-- > 0) wc = (wc << 8) | (*s1++ & 0xff); if (pwc != NULL) *pwc = wc; *s = s0; *nresult = (wc == 0) ? 0 : psenc->chlen - chlenbak; /* _citrus_GBK2K_init_state(ei, psenc); */ psenc->chlen = 0; return (0); restart: *s = s0; *nresult = (size_t)-2; return (0); invalid: return (EINVAL); ilseq: *nresult = (size_t)-1; return (EILSEQ); } static int _citrus_GBK2K_wcrtomb_priv(_GBK2KEncodingInfo * __restrict ei, char * __restrict s, size_t n, wchar_t wc, _GBK2KState * __restrict psenc, size_t * __restrict nresult) { size_t len; int ret; if (psenc->chlen != 0) { ret = EINVAL; goto err; } len = _mb_count(wc); if (n < len) { ret = E2BIG; goto err; } switch (len) { case 1: if (!_mb_singlebyte(_PUSH_PSENC(wc ))) { ret = EILSEQ; goto err; } break; case 2: if (!_mb_leadbyte (_PUSH_PSENC(wc >> 8)) || !_mb_trailbyte (_PUSH_PSENC(wc))) { ret = EILSEQ; goto err; } break; case 4: if (ei->mb_cur_max != 4 || !_mb_leadbyte (_PUSH_PSENC(wc >> 24)) || !_mb_surrogate (_PUSH_PSENC(wc >> 16)) || !_mb_leadbyte (_PUSH_PSENC(wc >> 8)) || !_mb_surrogate (_PUSH_PSENC(wc))) { ret = EILSEQ; goto err; } break; } memcpy(s, psenc->ch, psenc->chlen); *nresult = psenc->chlen; /* _citrus_GBK2K_init_state(ei, psenc); */ psenc->chlen = 0; return (0); err: *nresult = (size_t)-1; return (ret); } static __inline int /*ARGSUSED*/ _citrus_GBK2K_stdenc_wctocs(_GBK2KEncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { uint8_t ch, cl; if ((uint32_t)wc < 0x80) { /* ISO646 */ *csid = 0; *idx = (_index_t)wc; } else if ((uint32_t)wc >= 0x10000) { /* GBKUCS : XXX */ *csid = 3; *idx = (_index_t)wc; } else { ch = (uint8_t)(wc >> 8); cl = (uint8_t)wc; if (ch >= 0xA1 && cl >= 0xA1) { /* EUC G1 */ *csid = 1; *idx = (_index_t)wc & 0x7F7FU; } else { /* extended area (0x8140-) */ *csid = 2; *idx = (_index_t)wc; } } return (0); } static __inline int /*ARGSUSED*/ _citrus_GBK2K_stdenc_cstowc(_GBK2KEncodingInfo * __restrict ei, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { switch (csid) { case 0: /* ISO646 */ *wc = (wchar_t)idx; break; case 1: /* EUC G1 */ *wc = (wchar_t)idx | 0x8080U; break; case 2: /* extended area */ *wc = (wchar_t)idx; break; case 3: /* GBKUCS : XXX */ if (ei->mb_cur_max != 4) return (EINVAL); *wc = (wchar_t)idx; break; default: return (EILSEQ); } return (0); } static __inline int /*ARGSUSED*/ _citrus_GBK2K_stdenc_get_state_desc_generic(_GBK2KEncodingInfo * __restrict ei __unused, _GBK2KState * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } static int /*ARGSUSED*/ _citrus_GBK2K_encoding_module_init(_GBK2KEncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { const char *p; p = var; memset((void *)ei, 0, sizeof(*ei)); ei->mb_cur_max = 4; while (lenvar > 0) { switch (_bcs_tolower(*p)) { case '2': MATCH("2byte", ei->mb_cur_max = 2); break; } p++; lenvar--; } return (0); } static void /*ARGSUSED*/ _citrus_GBK2K_encoding_module_uninit(_GBK2KEncodingInfo *ei __unused) { } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(GBK2K); _CITRUS_STDENC_DEF_OPS(GBK2K); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/HZ/citrus_hz.c =================================================================== --- stable/10/lib/libiconv_modules/HZ/citrus_hz.c (revision 282274) +++ stable/10/lib/libiconv_modules/HZ/citrus_hz.c (revision 282275) @@ -1,648 +1,648 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_hz.c,v 1.2 2008/06/14 16:01:07 tnozaki Exp $ */ /*- * Copyright (c)2004, 2006 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_bcs.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_hz.h" #include "citrus_prop.h" /* * wchar_t mapping: * * CTRL/ASCII 00000000 00000000 00000000 gxxxxxxx * GB2312 00000000 00000000 0xxxxxxx gxxxxxxx * 94/96*n (~M) 0mmmmmmm 0xxxxxxx 0xxxxxxx gxxxxxxx */ #define ESCAPE_CHAR '~' typedef enum { CTRL = 0, ASCII = 1, GB2312 = 2, CS94 = 3, CS96 = 4 } charset_t; typedef struct { int start; int end; int width; } range_t; static const range_t ranges[] = { #define RANGE(start, end) { start, end, (end - start) + 1 } /* CTRL */ RANGE(0x00, 0x1F), /* ASCII */ RANGE(0x20, 0x7F), /* GB2312 */ RANGE(0x21, 0x7E), /* CS94 */ RANGE(0x21, 0x7E), /* CS96 */ RANGE(0x20, 0x7F), #undef RANGE }; typedef struct escape_t escape_t; typedef struct { charset_t charset; escape_t *escape; ssize_t length; #define ROWCOL_MAX 3 } graphic_t; typedef TAILQ_HEAD(escape_list, escape_t) escape_list; struct escape_t { TAILQ_ENTRY(escape_t) entry; escape_list *set; graphic_t *left; graphic_t *right; int ch; }; #define GL(escape) ((escape)->left) #define GR(escape) ((escape)->right) #define SET(escape) ((escape)->set) #define ESC(escape) ((escape)->ch) #define INIT(escape) (TAILQ_FIRST(SET(escape))) static __inline escape_t * find_escape(escape_list *set, int ch) { escape_t *escape; TAILQ_FOREACH(escape, set, entry) { if (ESC(escape) == ch) break; } return (escape); } typedef struct { escape_list e0; escape_list e1; graphic_t *ascii; graphic_t *gb2312; } _HZEncodingInfo; #define E0SET(ei) (&(ei)->e0) #define E1SET(ei) (&(ei)->e1) #define INIT0(ei) (TAILQ_FIRST(E0SET(ei))) #define INIT1(ei) (TAILQ_FIRST(E1SET(ei))) typedef struct { escape_t *inuse; int chlen; char ch[ROWCOL_MAX]; } _HZState; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_HZ_##m #define _ENCODING_INFO _HZEncodingInfo #define _ENCODING_STATE _HZState #define _ENCODING_MB_CUR_MAX(_ei_) MB_LEN_MAX #define _ENCODING_IS_STATE_DEPENDENT 1 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) ((_ps_)->inuse == NULL) static __inline void _citrus_HZ_init_state(_HZEncodingInfo * __restrict ei, _HZState * __restrict psenc) { psenc->chlen = 0; psenc->inuse = INIT0(ei); } #if 0 static __inline void /*ARGSUSED*/ _citrus_HZ_pack_state(_HZEncodingInfo * __restrict ei __unused, void *__restrict pspriv, const _HZState * __restrict psenc) { memcpy(pspriv, (const void *)psenc, sizeof(*psenc)); } static __inline void /*ARGSUSED*/ _citrus_HZ_unpack_state(_HZEncodingInfo * __restrict ei __unused, _HZState * __restrict psenc, const void * __restrict pspriv) { memcpy((void *)psenc, pspriv, sizeof(*psenc)); } #endif static int _citrus_HZ_mbrtowc_priv(_HZEncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, size_t n, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _HZState * __restrict psenc, size_t * __restrict nresult) { escape_t *candidate, *init; graphic_t *graphic; const range_t *range; - const char *s0; + char *s0; wchar_t wc; int bit, ch, head, len, tail; if (*s == NULL) { _citrus_HZ_init_state(ei, psenc); *nresult = 1; return (0); } s0 = *s; if (psenc->chlen < 0 || psenc->inuse == NULL) return (EINVAL); wc = (wchar_t)0; bit = head = tail = 0; graphic = NULL; for (len = 0; len <= MB_LEN_MAX;) { if (psenc->chlen == tail) { if (n-- < 1) { *s = s0; *nresult = (size_t)-2; return (0); } psenc->ch[psenc->chlen++] = *s0++; ++len; } ch = (unsigned char)psenc->ch[tail++]; if (tail == 1) { if ((ch & ~0x80) <= 0x1F) { if (psenc->inuse != INIT0(ei)) break; wc = (wchar_t)ch; goto done; } if (ch & 0x80) { graphic = GR(psenc->inuse); bit = 0x80; ch &= ~0x80; } else { graphic = GL(psenc->inuse); if (ch == ESCAPE_CHAR) continue; bit = 0x0; } if (graphic == NULL) break; } else if (tail == 2 && psenc->ch[0] == ESCAPE_CHAR) { if (tail < psenc->chlen) return (EINVAL); if (ch == ESCAPE_CHAR) { ++head; } else if (ch == '\n') { if (psenc->inuse != INIT0(ei)) break; tail = psenc->chlen = 0; continue; } else { candidate = NULL; init = INIT0(ei); if (psenc->inuse == init) { init = INIT1(ei); } else if (INIT(psenc->inuse) == init) { if (ESC(init) != ch) break; candidate = init; } if (candidate == NULL) { candidate = find_escape( SET(psenc->inuse), ch); if (candidate == NULL) { if (init == NULL || ESC(init) != ch) break; candidate = init; } } psenc->inuse = candidate; tail = psenc->chlen = 0; continue; } } else if (ch & 0x80) { if (graphic != GR(psenc->inuse)) break; ch &= ~0x80; } else { if (graphic != GL(psenc->inuse)) break; } range = &ranges[(size_t)graphic->charset]; if (range->start > ch || range->end < ch) break; wc <<= 8; wc |= ch; if (graphic->length == (tail - head)) { if (graphic->charset > GB2312) bit |= ESC(psenc->inuse) << 24; wc |= bit; goto done; } } *nresult = (size_t)-1; return (EILSEQ); done: if (tail < psenc->chlen) return (EINVAL); *s = s0; if (pwc != NULL) *pwc = wc; psenc->chlen = 0; *nresult = (wc == 0) ? 0 : len; return (0); } static int _citrus_HZ_wcrtomb_priv(_HZEncodingInfo * __restrict ei, char * __restrict s, size_t n, wchar_t wc, _HZState * __restrict psenc, size_t * __restrict nresult) { escape_t *candidate, *init; graphic_t *graphic; const range_t *range; size_t len; int bit, ch; if (psenc->chlen != 0 || psenc->inuse == NULL) return (EINVAL); if (wc & 0x80) { bit = 0x80; wc &= ~0x80; } else { bit = 0x0; } if ((uint32_t)wc <= 0x1F) { candidate = INIT0(ei); graphic = (bit == 0) ? candidate->left : candidate->right; if (graphic == NULL) goto ilseq; range = &ranges[(size_t)CTRL]; len = 1; } else if ((uint32_t)wc <= 0x7F) { graphic = ei->ascii; if (graphic == NULL) goto ilseq; candidate = graphic->escape; range = &ranges[(size_t)graphic->charset]; len = graphic->length; } else if ((uint32_t)wc <= 0x7F7F) { graphic = ei->gb2312; if (graphic == NULL) goto ilseq; candidate = graphic->escape; range = &ranges[(size_t)graphic->charset]; len = graphic->length; } else { ch = (wc >> 24) & 0xFF; candidate = find_escape(E0SET(ei), ch); if (candidate == NULL) { candidate = find_escape(E1SET(ei), ch); if (candidate == NULL) goto ilseq; } wc &= ~0xFF000000; graphic = (bit == 0) ? candidate->left : candidate->right; if (graphic == NULL) goto ilseq; range = &ranges[(size_t)graphic->charset]; len = graphic->length; } if (psenc->inuse != candidate) { init = INIT0(ei); if (SET(psenc->inuse) == SET(candidate)) { if (INIT(psenc->inuse) != init || psenc->inuse == init || candidate == init) init = NULL; } else if (candidate == (init = INIT(candidate))) { init = NULL; } if (init != NULL) { if (n < 2) return (E2BIG); n -= 2; psenc->ch[psenc->chlen++] = ESCAPE_CHAR; psenc->ch[psenc->chlen++] = ESC(init); } if (n < 2) return (E2BIG); n -= 2; psenc->ch[psenc->chlen++] = ESCAPE_CHAR; psenc->ch[psenc->chlen++] = ESC(candidate); psenc->inuse = candidate; } if (n < len) return (E2BIG); while (len-- > 0) { ch = (wc >> (len * 8)) & 0xFF; if (range->start > ch || range->end < ch) goto ilseq; psenc->ch[psenc->chlen++] = ch | bit; } memcpy(s, psenc->ch, psenc->chlen); *nresult = psenc->chlen; psenc->chlen = 0; return (0); ilseq: *nresult = (size_t)-1; return (EILSEQ); } static __inline int _citrus_HZ_put_state_reset(_HZEncodingInfo * __restrict ei, char * __restrict s, size_t n, _HZState * __restrict psenc, size_t * __restrict nresult) { escape_t *candidate; if (psenc->chlen != 0 || psenc->inuse == NULL) return (EINVAL); candidate = INIT0(ei); if (psenc->inuse != candidate) { if (n < 2) return (E2BIG); n -= 2; psenc->ch[psenc->chlen++] = ESCAPE_CHAR; psenc->ch[psenc->chlen++] = ESC(candidate); } if (n < 1) return (E2BIG); if (psenc->chlen > 0) memcpy(s, psenc->ch, psenc->chlen); *nresult = psenc->chlen; _citrus_HZ_init_state(ei, psenc); return (0); } static __inline int _citrus_HZ_stdenc_get_state_desc_generic(_HZEncodingInfo * __restrict ei, _HZState * __restrict psenc, int * __restrict rstate) { if (psenc->chlen < 0 || psenc->inuse == NULL) return (EINVAL); *rstate = (psenc->chlen == 0) ? ((psenc->inuse == INIT0(ei)) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_STABLE) : ((psenc->ch[0] == ESCAPE_CHAR) ? _STDENC_SDGEN_INCOMPLETE_SHIFT : _STDENC_SDGEN_INCOMPLETE_CHAR); return (0); } static __inline int /*ARGSUSED*/ _citrus_HZ_stdenc_wctocs(_HZEncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { int bit; if (wc & 0x80) { bit = 0x80; wc &= ~0x80; } else bit = 0x0; if ((uint32_t)wc <= 0x7F) { *csid = (_csid_t)bit; *idx = (_index_t)wc; } else if ((uint32_t)wc <= 0x7F7F) { *csid = (_csid_t)(bit | 0x8000); *idx = (_index_t)wc; } else { *csid = (_index_t)(wc & ~0x00FFFF7F); *idx = (_csid_t)(wc & 0x00FFFF7F); } return (0); } static __inline int /*ARGSUSED*/ _citrus_HZ_stdenc_cstowc(_HZEncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { *wc = (wchar_t)idx; switch (csid) { case 0x80: case 0x8080: *wc |= (wchar_t)0x80; /*FALLTHROUGH*/ case 0x0: case 0x8000: break; default: *wc |= (wchar_t)csid; } return (0); } static void _citrus_HZ_encoding_module_uninit(_HZEncodingInfo *ei) { escape_t *escape; while ((escape = TAILQ_FIRST(E0SET(ei))) != NULL) { TAILQ_REMOVE(E0SET(ei), escape, entry); free(GL(escape)); free(GR(escape)); free(escape); } while ((escape = TAILQ_FIRST(E1SET(ei))) != NULL) { TAILQ_REMOVE(E1SET(ei), escape, entry); free(GL(escape)); free(GR(escape)); free(escape); } } static int _citrus_HZ_parse_char(void *context, const char *name __unused, const char *s) { escape_t *escape; void **p; p = (void **)context; escape = (escape_t *)p[0]; if (escape->ch != '\0') return (EINVAL); escape->ch = *s++; if (escape->ch == ESCAPE_CHAR || *s != '\0') return (EINVAL); return (0); } static int _citrus_HZ_parse_graphic(void *context, const char *name, const char *s) { _HZEncodingInfo *ei; escape_t *escape; graphic_t *graphic; void **p; p = (void **)context; escape = (escape_t *)p[0]; ei = (_HZEncodingInfo *)p[1]; graphic = calloc(1, sizeof(*graphic)); if (graphic == NULL) return (ENOMEM); if (strcmp("GL", name) == 0) { if (GL(escape) != NULL) goto release; GL(escape) = graphic; } else if (strcmp("GR", name) == 0) { if (GR(escape) != NULL) goto release; GR(escape) = graphic; } else { release: free(graphic); return (EINVAL); } graphic->escape = escape; if (_bcs_strncasecmp("ASCII", s, 5) == 0) { if (s[5] != '\0') return (EINVAL); graphic->charset = ASCII; graphic->length = 1; ei->ascii = graphic; return (0); } else if (_bcs_strncasecmp("GB2312", s, 6) == 0) { if (s[6] != '\0') return (EINVAL); graphic->charset = GB2312; graphic->length = 2; ei->gb2312 = graphic; return (0); } else if (strncmp("94*", s, 3) == 0) graphic->charset = CS94; else if (strncmp("96*", s, 3) == 0) graphic->charset = CS96; else return (EINVAL); s += 3; switch(*s) { case '1': case '2': case '3': graphic->length = (size_t)(*s - '0'); if (*++s == '\0') break; /*FALLTHROUGH*/ default: return (EINVAL); } return (0); } static const _citrus_prop_hint_t escape_hints[] = { _CITRUS_PROP_HINT_STR("CH", &_citrus_HZ_parse_char), _CITRUS_PROP_HINT_STR("GL", &_citrus_HZ_parse_graphic), _CITRUS_PROP_HINT_STR("GR", &_citrus_HZ_parse_graphic), _CITRUS_PROP_HINT_END }; static int _citrus_HZ_parse_escape(void *context, const char *name, const char *s) { _HZEncodingInfo *ei; escape_t *escape; void *p[2]; ei = (_HZEncodingInfo *)context; escape = calloc(1, sizeof(*escape)); if (escape == NULL) return (EINVAL); if (strcmp("0", name) == 0) { escape->set = E0SET(ei); TAILQ_INSERT_TAIL(E0SET(ei), escape, entry); } else if (strcmp("1", name) == 0) { escape->set = E1SET(ei); TAILQ_INSERT_TAIL(E1SET(ei), escape, entry); } else { free(escape); return (EINVAL); } p[0] = (void *)escape; p[1] = (void *)ei; return (_citrus_prop_parse_variable( escape_hints, (void *)&p[0], s, strlen(s))); } static const _citrus_prop_hint_t root_hints[] = { _CITRUS_PROP_HINT_STR("0", &_citrus_HZ_parse_escape), _CITRUS_PROP_HINT_STR("1", &_citrus_HZ_parse_escape), _CITRUS_PROP_HINT_END }; static int _citrus_HZ_encoding_module_init(_HZEncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { int errnum; memset(ei, 0, sizeof(*ei)); TAILQ_INIT(E0SET(ei)); TAILQ_INIT(E1SET(ei)); errnum = _citrus_prop_parse_variable( root_hints, (void *)ei, var, lenvar); if (errnum != 0) _citrus_HZ_encoding_module_uninit(ei); return (errnum); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(HZ); _CITRUS_STDENC_DEF_OPS(HZ); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/ISO2022/citrus_iso2022.c =================================================================== --- stable/10/lib/libiconv_modules/ISO2022/citrus_iso2022.c (revision 282274) +++ stable/10/lib/libiconv_modules/ISO2022/citrus_iso2022.c (revision 282275) @@ -1,1291 +1,1291 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_iso2022.c,v 1.20 2010/12/07 22:01:45 joerg Exp $ */ /*- * Copyright (c)1999, 2002 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $Citrus: xpg4dl/FreeBSD/lib/libc/locale/iso2022.c,v 1.23 2001/06/21 01:51:44 yamt Exp $ */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_iso2022.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ /* * wchar_t mappings: * ASCII (ESC ( B) 00000000 00000000 00000000 0xxxxxxx * iso-8859-1 (ESC , A) 00000000 00000000 00000000 1xxxxxxx * 94 charset (ESC ( F) 0fffffff 00000000 00000000 0xxxxxxx * 94 charset (ESC ( M F) 0fffffff 1mmmmmmm 00000000 0xxxxxxx * 96 charset (ESC , F) 0fffffff 00000000 00000000 1xxxxxxx * 96 charset (ESC , M F) 0fffffff 1mmmmmmm 00000000 1xxxxxxx * 94x94 charset (ESC $ ( F) 0fffffff 00000000 0xxxxxxx 0xxxxxxx * 96x96 charset (ESC $ , F) 0fffffff 00000000 0xxxxxxx 1xxxxxxx * 94x94 charset (ESC & V ESC $ ( F) * 0fffffff 1vvvvvvv 0xxxxxxx 0xxxxxxx * 94x94x94 charset (ESC $ ( F) 0fffffff 0xxxxxxx 0xxxxxxx 0xxxxxxx * 96x96x96 charset (ESC $ , F) 0fffffff 0xxxxxxx 0xxxxxxx 1xxxxxxx * reserved for UCS4 co-existence (UCS4 is 31bit encoding thanks to mohta bit) * 1xxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx */ #define CS94 (0U) #define CS96 (1U) #define CS94MULTI (2U) #define CS96MULTI (3U) typedef struct { unsigned char type; unsigned char final; unsigned char interm; unsigned char vers; } _ISO2022Charset; static const _ISO2022Charset ascii = { CS94, 'B', '\0', '\0' }; static const _ISO2022Charset iso88591 = { CS96, 'A', '\0', '\0' }; typedef struct { _ISO2022Charset g[4]; /* need 3 bits to hold -1, 0, ..., 3 */ int gl:3, gr:3, singlegl:3, singlegr:3; char ch[7]; /* longest escape sequence (ESC & V ESC $ ( F) */ size_t chlen; int flags; #define _ISO2022STATE_FLAG_INITIALIZED 1 } _ISO2022State; typedef struct { _ISO2022Charset *recommend[4]; size_t recommendsize[4]; _ISO2022Charset initg[4]; int maxcharset; int flags; #define F_8BIT 0x0001 #define F_NOOLD 0x0002 #define F_SI 0x0010 /*0F*/ #define F_SO 0x0020 /*0E*/ #define F_LS0 0x0010 /*0F*/ #define F_LS1 0x0020 /*0E*/ #define F_LS2 0x0040 /*ESC n*/ #define F_LS3 0x0080 /*ESC o*/ #define F_LS1R 0x0100 /*ESC ~*/ #define F_LS2R 0x0200 /*ESC }*/ #define F_LS3R 0x0400 /*ESC |*/ #define F_SS2 0x0800 /*ESC N*/ #define F_SS3 0x1000 /*ESC O*/ #define F_SS2R 0x2000 /*8E*/ #define F_SS3R 0x4000 /*8F*/ } _ISO2022EncodingInfo; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_ISO2022_##m #define _ENCODING_INFO _ISO2022EncodingInfo #define _ENCODING_STATE _ISO2022State #define _ENCODING_MB_CUR_MAX(_ei_) MB_LEN_MAX #define _ENCODING_IS_STATE_DEPENDENT 1 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) \ (!((_ps_)->flags & _ISO2022STATE_FLAG_INITIALIZED)) #define _ISO2022INVALID (wchar_t)-1 static __inline bool isc0(__uint8_t x) { return ((x & 0x1f) == x); } static __inline bool isc1(__uint8_t x) { return (0x80 <= x && x <= 0x9f); } static __inline bool iscntl(__uint8_t x) { return (isc0(x) || isc1(x) || x == 0x7f); } static __inline bool is94(__uint8_t x) { return (0x21 <= x && x <= 0x7e); } static __inline bool is96(__uint8_t x) { return (0x20 <= x && x <= 0x7f); } static __inline bool isecma(__uint8_t x) { return (0x30 <= x && x <= 0x7f); } static __inline bool isinterm(__uint8_t x) { return (0x20 <= x && x <= 0x2f); } static __inline bool isthree(__uint8_t x) { return (0x60 <= x && x <= 0x6f); } static __inline int getcs(const char * __restrict p, _ISO2022Charset * __restrict cs) { if (!strncmp(p, "94$", 3) && p[3] && !p[4]) { cs->final = (unsigned char)(p[3] & 0xff); cs->interm = '\0'; cs->vers = '\0'; cs->type = CS94MULTI; } else if (!strncmp(p, "96$", 3) && p[3] && !p[4]) { cs->final = (unsigned char)(p[3] & 0xff); cs->interm = '\0'; cs->vers = '\0'; cs->type = CS96MULTI; } else if (!strncmp(p, "94", 2) && p[2] && !p[3]) { cs->final = (unsigned char)(p[2] & 0xff); cs->interm = '\0'; cs->vers = '\0'; cs->type = CS94; } else if (!strncmp(p, "96", 2) && p[2] && !p[3]) { cs->final = (unsigned char )(p[2] & 0xff); cs->interm = '\0'; cs->vers = '\0'; cs->type = CS96; } else return (1); return (0); } #define _NOTMATCH 0 #define _MATCH 1 #define _PARSEFAIL 2 static __inline int get_recommend(_ISO2022EncodingInfo * __restrict ei, const char * __restrict token) { _ISO2022Charset cs, *p; int i; if (!strchr("0123", token[0]) || token[1] != '=') return (_NOTMATCH); if (getcs(&token[2], &cs) == 0) ; else if (!strcmp(&token[2], "94")) { cs.final = (unsigned char)(token[4]); cs.interm = '\0'; cs.vers = '\0'; cs.type = CS94; } else if (!strcmp(&token[2], "96")) { cs.final = (unsigned char)(token[4]); cs.interm = '\0'; cs.vers = '\0'; cs.type = CS96; } else if (!strcmp(&token[2], "94$")) { cs.final = (unsigned char)(token[5]); cs.interm = '\0'; cs.vers = '\0'; cs.type = CS94MULTI; } else if (!strcmp(&token[2], "96$")) { cs.final = (unsigned char)(token[5]); cs.interm = '\0'; cs.vers = '\0'; cs.type = CS96MULTI; } else return (_PARSEFAIL); i = token[0] - '0'; if (!ei->recommend[i]) ei->recommend[i] = malloc(sizeof(_ISO2022Charset)); else { p = realloc(ei->recommend[i], sizeof(_ISO2022Charset) * (ei->recommendsize[i] + 1)); if (!p) return (_PARSEFAIL); ei->recommend[i] = p; } if (!ei->recommend[i]) return (_PARSEFAIL); ei->recommendsize[i]++; (ei->recommend[i] + (ei->recommendsize[i] - 1))->final = cs.final; (ei->recommend[i] + (ei->recommendsize[i] - 1))->interm = cs.interm; (ei->recommend[i] + (ei->recommendsize[i] - 1))->vers = cs.vers; (ei->recommend[i] + (ei->recommendsize[i] - 1))->type = cs.type; return (_MATCH); } static __inline int get_initg(_ISO2022EncodingInfo * __restrict ei, const char * __restrict token) { _ISO2022Charset cs; if (strncmp("INIT", &token[0], 4) || !strchr("0123", token[4]) || token[5] != '=') return (_NOTMATCH); if (getcs(&token[6], &cs) != 0) return (_PARSEFAIL); ei->initg[token[4] - '0'].type = cs.type; ei->initg[token[4] - '0'].final = cs.final; ei->initg[token[4] - '0'].interm = cs.interm; ei->initg[token[4] - '0'].vers = cs.vers; return (_MATCH); } static __inline int get_max(_ISO2022EncodingInfo * __restrict ei, const char * __restrict token) { if (!strcmp(token, "MAX1")) ei->maxcharset = 1; else if (!strcmp(token, "MAX2")) ei->maxcharset = 2; else if (!strcmp(token, "MAX3")) ei->maxcharset = 3; else return (_NOTMATCH); return (_MATCH); } static __inline int get_flags(_ISO2022EncodingInfo * __restrict ei, const char * __restrict token) { static struct { const char *tag; int flag; } const tags[] = { { "DUMMY", 0 }, { "8BIT", F_8BIT }, { "NOOLD", F_NOOLD }, { "SI", F_SI }, { "SO", F_SO }, { "LS0", F_LS0 }, { "LS1", F_LS1 }, { "LS2", F_LS2 }, { "LS3", F_LS3 }, { "LS1R", F_LS1R }, { "LS2R", F_LS2R }, { "LS3R", F_LS3R }, { "SS2", F_SS2 }, { "SS3", F_SS3 }, { "SS2R", F_SS2R }, { "SS3R", F_SS3R }, { NULL, 0 } }; int i; for (i = 0; tags[i].tag; i++) if (!strcmp(token, tags[i].tag)) { ei->flags |= tags[i].flag; return (_MATCH); } return (_NOTMATCH); } static __inline int _citrus_ISO2022_parse_variable(_ISO2022EncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar __unused) { char const *e, *v; char buf[20]; size_t len; int i, ret; /* * parse VARIABLE section. */ if (!var) return (EFTYPE); v = (const char *) var; /* initialize structure */ ei->maxcharset = 0; for (i = 0; i < 4; i++) { ei->recommend[i] = NULL; ei->recommendsize[i] = 0; } ei->flags = 0; while (*v) { while (*v == ' ' || *v == '\t') ++v; /* find the token */ e = v; while (*e && *e != ' ' && *e != '\t') ++e; len = e - v; if (len == 0) break; if (len >= sizeof(buf)) goto parsefail; snprintf(buf, sizeof(buf), "%.*s", (int)len, v); if ((ret = get_recommend(ei, buf)) != _NOTMATCH) ; else if ((ret = get_initg(ei, buf)) != _NOTMATCH) ; else if ((ret = get_max(ei, buf)) != _NOTMATCH) ; else if ((ret = get_flags(ei, buf)) != _NOTMATCH) ; else ret = _PARSEFAIL; if (ret == _PARSEFAIL) goto parsefail; v = e; } return (0); parsefail: free(ei->recommend[0]); free(ei->recommend[1]); free(ei->recommend[2]); free(ei->recommend[3]); return (EFTYPE); } static __inline void /*ARGSUSED*/ _citrus_ISO2022_init_state(_ISO2022EncodingInfo * __restrict ei, _ISO2022State * __restrict s) { int i; memset(s, 0, sizeof(*s)); s->gl = 0; s->gr = (ei->flags & F_8BIT) ? 1 : -1; for (i = 0; i < 4; i++) if (ei->initg[i].final) { s->g[i].type = ei->initg[i].type; s->g[i].final = ei->initg[i].final; s->g[i].interm = ei->initg[i].interm; } s->singlegl = s->singlegr = -1; s->flags |= _ISO2022STATE_FLAG_INITIALIZED; } #if 0 static __inline void /*ARGSUSED*/ _citrus_ISO2022_pack_state(_ISO2022EncodingInfo * __restrict ei __unused, void * __restrict pspriv, const _ISO2022State * __restrict s) { memcpy(pspriv, (const void *)s, sizeof(*s)); } static __inline void /*ARGSUSED*/ _citrus_ISO2022_unpack_state(_ISO2022EncodingInfo * __restrict ei __unused, _ISO2022State * __restrict s, const void * __restrict pspriv) { memcpy((void *)s, pspriv, sizeof(*s)); } #endif static int /*ARGSUSED*/ _citrus_ISO2022_encoding_module_init(_ISO2022EncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { return (_citrus_ISO2022_parse_variable(ei, var, lenvar)); } static void /*ARGSUSED*/ _citrus_ISO2022_encoding_module_uninit(_ISO2022EncodingInfo *ei __unused) { } #define ESC '\033' #define ECMA -1 #define INTERM -2 #define OECMA -3 static const struct seqtable { int type; int csoff; int finaloff; int intermoff; int versoff; int len; int chars[10]; } seqtable[] = { /* G0 94MULTI special */ { CS94MULTI, -1, 2, -1, -1, 3, { ESC, '$', OECMA }, }, /* G0 94MULTI special with version identification */ { CS94MULTI, -1, 5, -1, 2, 6, { ESC, '&', ECMA, ESC, '$', OECMA }, }, /* G? 94 */ { CS94, 1, 2, -1, -1, 3, { ESC, CS94, ECMA, }, }, /* G? 94 with 2nd intermediate char */ { CS94, 1, 3, 2, -1, 4, { ESC, CS94, INTERM, ECMA, }, }, /* G? 96 */ { CS96, 1, 2, -1, -1, 3, { ESC, CS96, ECMA, }, }, /* G? 96 with 2nd intermediate char */ { CS96, 1, 3, 2, -1, 4, { ESC, CS96, INTERM, ECMA, }, }, /* G? 94MULTI */ { CS94MULTI, 2, 3, -1, -1, 4, { ESC, '$', CS94, ECMA, }, }, /* G? 96MULTI */ { CS96MULTI, 2, 3, -1, -1, 4, { ESC, '$', CS96, ECMA, }, }, /* G? 94MULTI with version specification */ { CS94MULTI, 5, 6, -1, 2, 7, { ESC, '&', ECMA, ESC, '$', CS94, ECMA, }, }, /* LS2/3 */ { -1, -1, -1, -1, -1, 2, { ESC, 'n', }, }, { -1, -1, -1, -1, -1, 2, { ESC, 'o', }, }, /* LS1/2/3R */ { -1, -1, -1, -1, -1, 2, { ESC, '~', }, }, { -1, -1, -1, -1, -1, 2, { ESC, /*{*/ '}', }, }, { -1, -1, -1, -1, -1, 2, { ESC, '|', }, }, /* SS2/3 */ { -1, -1, -1, -1, -1, 2, { ESC, 'N', }, }, { -1, -1, -1, -1, -1, 2, { ESC, 'O', }, }, /* end of records */ // { 0, } { 0, 0, 0, 0, 0, 0, { ESC, 0, }, } }; static int seqmatch(const char * __restrict s, size_t n, const struct seqtable * __restrict sp) { const int *p; p = sp->chars; while ((size_t)(p - sp->chars) < n && p - sp->chars < sp->len) { switch (*p) { case ECMA: if (!isecma(*s)) goto terminate; break; case OECMA: if (*s && strchr("@AB", *s)) break; else goto terminate; case INTERM: if (!isinterm(*s)) goto terminate; break; case CS94: if (*s && strchr("()*+", *s)) break; else goto terminate; case CS96: if (*s && strchr(",-./", *s)) break; else goto terminate; default: if (*s != *p) goto terminate; break; } p++; s++; } terminate: return (p - sp->chars); } static wchar_t _ISO2022_sgetwchar(_ISO2022EncodingInfo * __restrict ei __unused, - const char * __restrict string, size_t n, const char ** __restrict result, + char * __restrict string, size_t n, char ** __restrict result, _ISO2022State * __restrict psenc) { const struct seqtable *sp; wchar_t wchar = 0; int i, cur, nmatch; while (1) { /* SI/SO */ if (1 <= n && string[0] == '\017') { psenc->gl = 0; string++; n--; continue; } if (1 <= n && string[0] == '\016') { psenc->gl = 1; string++; n--; continue; } /* SS2/3R */ if (1 <= n && string[0] && strchr("\217\216", string[0])) { psenc->singlegl = psenc->singlegr = (string[0] - '\216') + 2; string++; n--; continue; } /* eat the letter if this is not ESC */ if (1 <= n && string[0] != '\033') break; /* look for a perfect match from escape sequences */ for (sp = &seqtable[0]; sp->len; sp++) { nmatch = seqmatch(string, n, sp); if (sp->len == nmatch && n >= (size_t)(sp->len)) break; } if (!sp->len) goto notseq; if (sp->type != -1) { if (sp->csoff == -1) i = 0; else { switch (sp->type) { case CS94: case CS94MULTI: i = string[sp->csoff] - '('; break; case CS96: case CS96MULTI: i = string[sp->csoff] - ','; break; default: return (_ISO2022INVALID); } } psenc->g[i].type = sp->type; psenc->g[i].final = '\0'; psenc->g[i].interm = '\0'; psenc->g[i].vers = '\0'; /* sp->finaloff must not be -1 */ if (sp->finaloff != -1) psenc->g[i].final = string[sp->finaloff]; if (sp->intermoff != -1) psenc->g[i].interm = string[sp->intermoff]; if (sp->versoff != -1) psenc->g[i].vers = string[sp->versoff]; string += sp->len; n -= sp->len; continue; } /* LS2/3 */ if (2 <= n && string[0] == '\033' && string[1] && strchr("no", string[1])) { psenc->gl = string[1] - 'n' + 2; string += 2; n -= 2; continue; } /* LS1/2/3R */ /* XXX: { for vi showmatch */ if (2 <= n && string[0] == '\033' && string[1] && strchr("~}|", string[1])) { psenc->gr = 3 - (string[1] - '|'); string += 2; n -= 2; continue; } /* SS2/3 */ if (2 <= n && string[0] == '\033' && string[1] && strchr("NO", string[1])) { psenc->singlegl = (string[1] - 'N') + 2; string += 2; n -= 2; continue; } notseq: /* * if we've got an unknown escape sequence, eat the ESC at the * head. otherwise, wait till full escape sequence comes. */ for (sp = &seqtable[0]; sp->len; sp++) { nmatch = seqmatch(string, n, sp); if (!nmatch) continue; /* * if we are in the middle of escape sequence, * we still need to wait for more characters to come */ if (n < (size_t)(sp->len)) { if ((size_t)(nmatch) == n) { if (result) *result = string; return (_ISO2022INVALID); } } else { if (nmatch == sp->len) { /* this case should not happen */ goto eat; } } } break; } eat: /* no letter to eat */ if (n < 1) { if (result) *result = string; return (_ISO2022INVALID); } /* normal chars. always eat C0/C1 as is. */ if (iscntl(*string & 0xff)) cur = -1; else if (*string & 0x80) cur = (psenc->singlegr == -1) ? psenc->gr : psenc->singlegr; else cur = (psenc->singlegl == -1) ? psenc->gl : psenc->singlegl; if (cur == -1) { asis: wchar = *string++ & 0xff; if (result) *result = string; /* reset single shift state */ psenc->singlegr = psenc->singlegl = -1; return (wchar); } /* length error check */ switch (psenc->g[cur].type) { case CS94MULTI: case CS96MULTI: if (!isthree(psenc->g[cur].final)) { if (2 <= n && (string[0] & 0x80) == (string[1] & 0x80)) break; } else { if (3 <= n && (string[0] & 0x80) == (string[1] & 0x80) && (string[0] & 0x80) == (string[2] & 0x80)) break; } /* we still need to wait for more characters to come */ if (result) *result = string; return (_ISO2022INVALID); case CS94: case CS96: if (1 <= n) break; /* we still need to wait for more characters to come */ if (result) *result = string; return (_ISO2022INVALID); } /* range check */ switch (psenc->g[cur].type) { case CS94: if (!(is94(string[0] & 0x7f))) goto asis; case CS96: if (!(is96(string[0] & 0x7f))) goto asis; break; case CS94MULTI: if (!(is94(string[0] & 0x7f) && is94(string[1] & 0x7f))) goto asis; break; case CS96MULTI: if (!(is96(string[0] & 0x7f) && is96(string[1] & 0x7f))) goto asis; break; } /* extract the character. */ switch (psenc->g[cur].type) { case CS94: /* special case for ASCII. */ if (psenc->g[cur].final == 'B' && !psenc->g[cur].interm) { wchar = *string++; wchar &= 0x7f; break; } wchar = psenc->g[cur].final; wchar = (wchar << 8); wchar |= (psenc->g[cur].interm ? (0x80 | psenc->g[cur].interm) : 0); wchar = (wchar << 8); wchar = (wchar << 8) | (*string++ & 0x7f); break; case CS96: /* special case for ISO-8859-1. */ if (psenc->g[cur].final == 'A' && !psenc->g[cur].interm) { wchar = *string++; wchar &= 0x7f; wchar |= 0x80; break; } wchar = psenc->g[cur].final; wchar = (wchar << 8); wchar |= (psenc->g[cur].interm ? (0x80 | psenc->g[cur].interm) : 0); wchar = (wchar << 8); wchar = (wchar << 8) | (*string++ & 0x7f); wchar |= 0x80; break; case CS94MULTI: case CS96MULTI: wchar = psenc->g[cur].final; wchar = (wchar << 8); if (isthree(psenc->g[cur].final)) wchar |= (*string++ & 0x7f); wchar = (wchar << 8) | (*string++ & 0x7f); wchar = (wchar << 8) | (*string++ & 0x7f); if (psenc->g[cur].type == CS96MULTI) wchar |= 0x80; break; } if (result) *result = string; /* reset single shift state */ psenc->singlegr = psenc->singlegl = -1; return (wchar); } static int _citrus_ISO2022_mbrtowc_priv(_ISO2022EncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _ISO2022State * __restrict psenc, size_t * __restrict nresult) { - const char *p, *result, *s0; + char *p, *result, *s0; wchar_t wchar; int c, chlenbak; if (*s == NULL) { _citrus_ISO2022_init_state(ei, psenc); *nresult = _ENCODING_IS_STATE_DEPENDENT; return (0); } s0 = *s; c = 0; chlenbak = psenc->chlen; /* * if we have something in buffer, use that. * otherwise, skip here */ if (psenc->chlen > sizeof(psenc->ch)) { /* illgeal state */ _citrus_ISO2022_init_state(ei, psenc); goto encoding_error; } if (psenc->chlen == 0) goto emptybuf; /* buffer is not empty */ p = psenc->ch; while (psenc->chlen < sizeof(psenc->ch)) { if (n > 0) { psenc->ch[psenc->chlen++] = *s0++; n--; } wchar = _ISO2022_sgetwchar(ei, p, psenc->chlen - (p-psenc->ch), &result, psenc); c += result - p; if (wchar != _ISO2022INVALID) { if (psenc->chlen > (size_t)c) memmove(psenc->ch, result, psenc->chlen - c); if (psenc->chlen < (size_t)c) psenc->chlen = 0; else psenc->chlen -= c; goto output; } if (n == 0) { if ((size_t)(result - p) == psenc->chlen) /* complete shift sequence. */ psenc->chlen = 0; goto restart; } p = result; } /* escape sequence too long? */ goto encoding_error; emptybuf: wchar = _ISO2022_sgetwchar(ei, s0, n, &result, psenc); if (wchar != _ISO2022INVALID) { c += result - s0; psenc->chlen = 0; s0 = result; goto output; } if (result > s0) { c += (result - s0); n -= (result - s0); s0 = result; if (n > 0) goto emptybuf; /* complete shift sequence. */ goto restart; } n += c; if (n < sizeof(psenc->ch)) { memcpy(psenc->ch, s0 - c, n); psenc->chlen = n; s0 = result; goto restart; } /* escape sequence too long? */ encoding_error: psenc->chlen = 0; *nresult = (size_t)-1; return (EILSEQ); output: *s = s0; if (pwc) *pwc = wchar; *nresult = wchar ? c - chlenbak : 0; return (0); restart: *s = s0; *nresult = (size_t)-2; return (0); } static int recommendation(_ISO2022EncodingInfo * __restrict ei, _ISO2022Charset * __restrict cs) { _ISO2022Charset *recommend; size_t j; int i; /* first, try a exact match. */ for (i = 0; i < 4; i++) { recommend = ei->recommend[i]; for (j = 0; j < ei->recommendsize[i]; j++) { if (cs->type != recommend[j].type) continue; if (cs->final != recommend[j].final) continue; if (cs->interm != recommend[j].interm) continue; return (i); } } /* then, try a wildcard match over final char. */ for (i = 0; i < 4; i++) { recommend = ei->recommend[i]; for (j = 0; j < ei->recommendsize[i]; j++) { if (cs->type != recommend[j].type) continue; if (cs->final && (cs->final != recommend[j].final)) continue; if (cs->interm && (cs->interm != recommend[j].interm)) continue; return (i); } } /* there's no recommendation. make a guess. */ if (ei->maxcharset == 0) { return (0); } else { switch (cs->type) { case CS94: case CS94MULTI: return (0); case CS96: case CS96MULTI: return (1); } } return (0); } static int _ISO2022_sputwchar(_ISO2022EncodingInfo * __restrict ei, wchar_t wc, char * __restrict string, size_t n, char ** __restrict result, _ISO2022State * __restrict psenc, size_t * __restrict nresult) { _ISO2022Charset cs; char *p; char tmp[MB_LEN_MAX]; size_t len; int bit8, i = 0, target; unsigned char mask; if (isc0(wc & 0xff)) { /* go back to INIT0 or ASCII on control chars */ cs = ei->initg[0].final ? ei->initg[0] : ascii; } else if (isc1(wc & 0xff)) { /* go back to INIT1 or ISO-8859-1 on control chars */ cs = ei->initg[1].final ? ei->initg[1] : iso88591; } else if (!(wc & ~0xff)) { if (wc & 0x80) { /* special treatment for ISO-8859-1 */ cs = iso88591; } else { /* special treatment for ASCII */ cs = ascii; } } else { cs.final = (wc >> 24) & 0x7f; if ((wc >> 16) & 0x80) cs.interm = (wc >> 16) & 0x7f; else cs.interm = '\0'; if (wc & 0x80) cs.type = (wc & 0x00007f00) ? CS96MULTI : CS96; else cs.type = (wc & 0x00007f00) ? CS94MULTI : CS94; } target = recommendation(ei, &cs); p = tmp; bit8 = ei->flags & F_8BIT; /* designate the charset onto the target plane(G0/1/2/3). */ if (psenc->g[target].type == cs.type && psenc->g[target].final == cs.final && psenc->g[target].interm == cs.interm) goto planeok; *p++ = '\033'; if (cs.type == CS94MULTI || cs.type == CS96MULTI) *p++ = '$'; if (target == 0 && cs.type == CS94MULTI && strchr("@AB", cs.final) && !cs.interm && !(ei->flags & F_NOOLD)) ; else if (cs.type == CS94 || cs.type == CS94MULTI) *p++ = "()*+"[target]; else *p++ = ",-./"[target]; if (cs.interm) *p++ = cs.interm; *p++ = cs.final; psenc->g[target].type = cs.type; psenc->g[target].final = cs.final; psenc->g[target].interm = cs.interm; planeok: /* invoke the plane onto GL or GR. */ if (psenc->gl == target) goto sideok; if (bit8 && psenc->gr == target) goto sideok; if (target == 0 && (ei->flags & F_LS0)) { *p++ = '\017'; psenc->gl = 0; } else if (target == 1 && (ei->flags & F_LS1)) { *p++ = '\016'; psenc->gl = 1; } else if (target == 2 && (ei->flags & F_LS2)) { *p++ = '\033'; *p++ = 'n'; psenc->gl = 2; } else if (target == 3 && (ei->flags & F_LS3)) { *p++ = '\033'; *p++ = 'o'; psenc->gl = 3; } else if (bit8 && target == 1 && (ei->flags & F_LS1R)) { *p++ = '\033'; *p++ = '~'; psenc->gr = 1; } else if (bit8 && target == 2 && (ei->flags & F_LS2R)) { *p++ = '\033'; /*{*/ *p++ = '}'; psenc->gr = 2; } else if (bit8 && target == 3 && (ei->flags & F_LS3R)) { *p++ = '\033'; *p++ = '|'; psenc->gr = 3; } else if (target == 2 && (ei->flags & F_SS2)) { *p++ = '\033'; *p++ = 'N'; psenc->singlegl = 2; } else if (target == 3 && (ei->flags & F_SS3)) { *p++ = '\033'; *p++ = 'O'; psenc->singlegl = 3; } else if (bit8 && target == 2 && (ei->flags & F_SS2R)) { *p++ = '\216'; *p++ = 'N'; psenc->singlegl = psenc->singlegr = 2; } else if (bit8 && target == 3 && (ei->flags & F_SS3R)) { *p++ = '\217'; *p++ = 'O'; psenc->singlegl = psenc->singlegr = 3; } else goto ilseq; sideok: if (psenc->singlegl == target) mask = 0x00; else if (psenc->singlegr == target) mask = 0x80; else if (psenc->gl == target) mask = 0x00; else if ((ei->flags & F_8BIT) && psenc->gr == target) mask = 0x80; else goto ilseq; switch (cs.type) { case CS94: case CS96: i = 1; break; case CS94MULTI: case CS96MULTI: i = !iscntl(wc & 0xff) ? (isthree(cs.final) ? 3 : 2) : 1; break; } while (i-- > 0) *p++ = ((wc >> (i << 3)) & 0x7f) | mask; /* reset single shift state */ psenc->singlegl = psenc->singlegr = -1; len = (size_t)(p - tmp); if (n < len) { if (result) *result = (char *)0; *nresult = (size_t)-1; return (E2BIG); } if (result) *result = string + len; memcpy(string, tmp, len); *nresult = len; return (0); ilseq: *nresult = (size_t)-1; return (EILSEQ); } static int _citrus_ISO2022_put_state_reset(_ISO2022EncodingInfo * __restrict ei, char * __restrict s, size_t n, _ISO2022State * __restrict psenc, size_t * __restrict nresult) { char *result; char buf[MB_LEN_MAX]; size_t len; int ret; /* XXX state will be modified after this operation... */ ret = _ISO2022_sputwchar(ei, L'\0', buf, sizeof(buf), &result, psenc, &len); if (ret) { *nresult = len; return (ret); } if (sizeof(buf) < len || n < len-1) { /* XXX should recover state? */ *nresult = (size_t)-1; return (E2BIG); } memcpy(s, buf, len - 1); *nresult = len - 1; return (0); } static int _citrus_ISO2022_wcrtomb_priv(_ISO2022EncodingInfo * __restrict ei, char * __restrict s, size_t n, wchar_t wc, _ISO2022State * __restrict psenc, size_t * __restrict nresult) { char *result; char buf[MB_LEN_MAX]; size_t len; int ret; /* XXX state will be modified after this operation... */ ret = _ISO2022_sputwchar(ei, wc, buf, sizeof(buf), &result, psenc, &len); if (ret) { *nresult = len; return (ret); } if (sizeof(buf) < len || n < len) { /* XXX should recover state? */ *nresult = (size_t)-1; return (E2BIG); } memcpy(s, buf, len); *nresult = len; return (0); } static __inline int /*ARGSUSED*/ _citrus_ISO2022_stdenc_wctocs(_ISO2022EncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { wchar_t m, nm; m = wc & 0x7FFF8080; nm = wc & 0x007F7F7F; if (m & 0x00800000) nm &= 0x00007F7F; else m &= 0x7F008080; if (nm & 0x007F0000) { /* ^3 mark */ m |= 0x007F0000; } else if (nm & 0x00007F00) { /* ^2 mark */ m |= 0x00007F00; } *csid = (_csid_t)m; *idx = (_index_t)nm; return (0); } static __inline int /*ARGSUSED*/ _citrus_ISO2022_stdenc_cstowc(_ISO2022EncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { *wc = (wchar_t)(csid & 0x7F808080) | (wchar_t)idx; return (0); } static __inline int /*ARGSUSED*/ _citrus_ISO2022_stdenc_get_state_desc_generic(_ISO2022EncodingInfo * __restrict ei __unused, _ISO2022State * __restrict psenc, int * __restrict rstate) { if (psenc->chlen == 0) { /* XXX: it should distinguish initial and stable. */ *rstate = _STDENC_SDGEN_STABLE; } else *rstate = (psenc->ch[0] == '\033') ? _STDENC_SDGEN_INCOMPLETE_SHIFT : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(ISO2022); _CITRUS_STDENC_DEF_OPS(ISO2022); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/JOHAB/citrus_johab.c =================================================================== --- stable/10/lib/libiconv_modules/JOHAB/citrus_johab.c (revision 282274) +++ stable/10/lib/libiconv_modules/JOHAB/citrus_johab.c (revision 282275) @@ -1,335 +1,335 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_johab.c,v 1.4 2008/06/14 16:01:07 tnozaki Exp $ */ /*- * Copyright (c)2006 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_bcs.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_johab.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct { int chlen; char ch[2]; } _JOHABState; typedef struct { int dummy; } _JOHABEncodingInfo; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_JOHAB_##m #define _ENCODING_INFO _JOHABEncodingInfo #define _ENCODING_STATE _JOHABState #define _ENCODING_MB_CUR_MAX(_ei_) 2 #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline void /*ARGSUSED*/ _citrus_JOHAB_init_state(_JOHABEncodingInfo * __restrict ei __unused, _JOHABState * __restrict psenc) { psenc->chlen = 0; } #if 0 static __inline void /*ARGSUSED*/ _citrus_JOHAB_pack_state(_JOHABEncodingInfo * __restrict ei __unused, void * __restrict pspriv, const _JOHABState * __restrict psenc) { memcpy(pspriv, (const void *)psenc, sizeof(*psenc)); } static __inline void /*ARGSUSED*/ _citrus_JOHAB_unpack_state(_JOHABEncodingInfo * __restrict ei __unused, _JOHABState * __restrict psenc, const void * __restrict pspriv) { memcpy((void *)psenc, pspriv, sizeof(*psenc)); } #endif static void /*ARGSUSED*/ _citrus_JOHAB_encoding_module_uninit(_JOHABEncodingInfo *ei __unused) { /* ei may be null */ } static int /*ARGSUSED*/ _citrus_JOHAB_encoding_module_init(_JOHABEncodingInfo * __restrict ei __unused, const void * __restrict var __unused, size_t lenvar __unused) { /* ei may be null */ return (0); } static __inline bool ishangul(int l, int t) { return ((l >= 0x84 && l <= 0xD3) && ((t >= 0x41 && t <= 0x7E) || (t >= 0x81 && t <= 0xFE))); } static __inline bool isuda(int l, int t) { return ((l == 0xD8) && ((t >= 0x31 && t <= 0x7E) || (t >= 0x91 && t <= 0xFE))); } static __inline bool ishanja(int l, int t) { return (((l >= 0xD9 && l <= 0xDE) || (l >= 0xE0 && l <= 0xF9)) && ((t >= 0x31 && t <= 0x7E) || (t >= 0x91 && t <= 0xFE))); } static int /*ARGSUSED*/ _citrus_JOHAB_mbrtowc_priv(_JOHABEncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, size_t n, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _JOHABState * __restrict psenc, size_t * __restrict nresult) { - const char *s0; + char *s0; int l, t; if (*s == NULL) { _citrus_JOHAB_init_state(ei, psenc); *nresult = _ENCODING_IS_STATE_DEPENDENT; return (0); } s0 = *s; switch (psenc->chlen) { case 0: if (n-- < 1) goto restart; l = *s0++ & 0xFF; if (l <= 0x7F) { if (pwc != NULL) *pwc = (wchar_t)l; *nresult = (l == 0) ? 0 : 1; *s = s0; return (0); } psenc->ch[psenc->chlen++] = l; break; case 1: l = psenc->ch[0] & 0xFF; break; default: return (EINVAL); } if (n-- < 1) { restart: *nresult = (size_t)-2; *s = s0; return (0); } t = *s0++ & 0xFF; if (!ishangul(l, t) && !isuda(l, t) && !ishanja(l, t)) { *nresult = (size_t)-1; return (EILSEQ); } if (pwc != NULL) *pwc = (wchar_t)(l << 8 | t); *nresult = s0 - *s; *s = s0; psenc->chlen = 0; return (0); } static int /*ARGSUSED*/ _citrus_JOHAB_wcrtomb_priv(_JOHABEncodingInfo * __restrict ei __unused, char * __restrict s, size_t n, wchar_t wc, _JOHABState * __restrict psenc, size_t * __restrict nresult) { int l, t; if (psenc->chlen != 0) return (EINVAL); /* XXX assume wchar_t as int */ if ((uint32_t)wc <= 0x7F) { if (n < 1) goto e2big; *s = wc & 0xFF; *nresult = 1; } else if ((uint32_t)wc <= 0xFFFF) { if (n < 2) { e2big: *nresult = (size_t)-1; return (E2BIG); } l = (wc >> 8) & 0xFF; t = wc & 0xFF; if (!ishangul(l, t) && !isuda(l, t) && !ishanja(l, t)) goto ilseq; *s++ = l; *s = t; *nresult = 2; } else { ilseq: *nresult = (size_t)-1; return (EILSEQ); } return (0); } static __inline int /*ARGSUSED*/ _citrus_JOHAB_stdenc_wctocs(_JOHABEncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { int m, l, linear, t; /* XXX assume wchar_t as int */ if ((uint32_t)wc <= 0x7F) { *idx = (_index_t)wc; *csid = 0; } else if ((uint32_t)wc <= 0xFFFF) { l = (wc >> 8) & 0xFF; t = wc & 0xFF; if (ishangul(l, t) || isuda(l, t)) { *idx = (_index_t)wc; *csid = 1; } else { if (l >= 0xD9 && l <= 0xDE) { linear = l - 0xD9; m = 0x21; } else if (l >= 0xE0 && l <= 0xF9) { linear = l - 0xE0; m = 0x4A; } else return (EILSEQ); linear *= 188; if (t >= 0x31 && t <= 0x7E) linear += t - 0x31; else if (t >= 0x91 && t <= 0xFE) linear += t - 0x43; else return (EILSEQ); l = (linear / 94) + m; t = (linear % 94) + 0x21; *idx = (_index_t)((l << 8) | t); *csid = 2; } } else return (EILSEQ); return (0); } static __inline int /*ARGSUSED*/ _citrus_JOHAB_stdenc_cstowc(_JOHABEncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { int m, n, l, linear, t; switch (csid) { case 0: case 1: *wc = (wchar_t)idx; break; case 2: if (idx >= 0x2121 && idx <= 0x2C71) { m = 0xD9; n = 0x21; } else if (idx >= 0x4A21 && idx <= 0x7D7E) { m = 0xE0; n = 0x4A; } else return (EILSEQ); l = ((idx >> 8) & 0xFF) - n; t = (idx & 0xFF) - 0x21; linear = (l * 94) + t; l = (linear / 188) + m; t = linear % 188; t += (t <= 0x4D) ? 0x31 : 0x43; break; default: return (EILSEQ); } return (0); } static __inline int /*ARGSUSED*/ _citrus_JOHAB_stdenc_get_state_desc_generic(_JOHABEncodingInfo * __restrict ei __unused, _JOHABState * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(JOHAB); _CITRUS_STDENC_DEF_OPS(JOHAB); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/MSKanji/citrus_mskanji.c =================================================================== --- stable/10/lib/libiconv_modules/MSKanji/citrus_mskanji.c (revision 282274) +++ stable/10/lib/libiconv_modules/MSKanji/citrus_mskanji.c (revision 282275) @@ -1,475 +1,475 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_mskanji.c,v 1.13 2008/06/14 16:01:08 tnozaki Exp $ */ /*- * Copyright (c)2002 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /* * ja_JP.SJIS locale table for BSD4.4/rune * version 1.0 * (C) Sin'ichiro MIYATANI / Phase One, Inc * May 12, 1995 * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. All advertising materials mentioning features or use of this software * must display the following acknowledgement: * This product includes software developed by Phase One, Inc. * 4. The name of Phase One, Inc. may be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_bcs.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_mskanji.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct _MSKanjiState { int chlen; char ch[2]; } _MSKanjiState; typedef struct { int mode; #define MODE_JIS2004 1 } _MSKanjiEncodingInfo; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_MSKanji_##m #define _ENCODING_INFO _MSKanjiEncodingInfo #define _ENCODING_STATE _MSKanjiState #define _ENCODING_MB_CUR_MAX(_ei_) 2 #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static bool _mskanji1(int c) { return ((c >= 0x81 && c <= 0x9f) || (c >= 0xe0 && c <= 0xfc)); } static bool _mskanji2(int c) { return ((c >= 0x40 && c <= 0x7e) || (c >= 0x80 && c <= 0xfc)); } static __inline void /*ARGSUSED*/ _citrus_MSKanji_init_state(_MSKanjiEncodingInfo * __restrict ei __unused, _MSKanjiState * __restrict s) { s->chlen = 0; } #if 0 static __inline void /*ARGSUSED*/ _citrus_MSKanji_pack_state(_MSKanjiEncodingInfo * __restrict ei __unused, void * __restrict pspriv, const _MSKanjiState * __restrict s) { memcpy(pspriv, (const void *)s, sizeof(*s)); } static __inline void /*ARGSUSED*/ _citrus_MSKanji_unpack_state(_MSKanjiEncodingInfo * __restrict ei __unused, _MSKanjiState * __restrict s, const void * __restrict pspriv) { memcpy((void *)s, pspriv, sizeof(*s)); } #endif static int /*ARGSUSED*/ _citrus_MSKanji_mbrtowc_priv(_MSKanjiEncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, size_t n, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _MSKanjiState * __restrict psenc, size_t * __restrict nresult) { - const char *s0; + char *s0; wchar_t wchar; int chlenbak, len; s0 = *s; if (s0 == NULL) { _citrus_MSKanji_init_state(ei, psenc); *nresult = 0; /* state independent */ return (0); } chlenbak = psenc->chlen; /* make sure we have the first byte in the buffer */ switch (psenc->chlen) { case 0: if (n < 1) goto restart; psenc->ch[0] = *s0++; psenc->chlen = 1; n--; break; case 1: break; default: /* illegal state */ goto encoding_error; } len = _mskanji1(psenc->ch[0] & 0xff) ? 2 : 1; while (psenc->chlen < len) { if (n < 1) goto restart; psenc->ch[psenc->chlen] = *s0++; psenc->chlen++; n--; } *s = s0; switch (len) { case 1: wchar = psenc->ch[0] & 0xff; break; case 2: if (!_mskanji2(psenc->ch[1] & 0xff)) goto encoding_error; wchar = ((psenc->ch[0] & 0xff) << 8) | (psenc->ch[1] & 0xff); break; default: /* illegal state */ goto encoding_error; } psenc->chlen = 0; if (pwc) *pwc = wchar; *nresult = wchar ? len - chlenbak : 0; return (0); encoding_error: psenc->chlen = 0; *nresult = (size_t)-1; return (EILSEQ); restart: *nresult = (size_t)-2; *s = s0; return (0); } static int _citrus_MSKanji_wcrtomb_priv(_MSKanjiEncodingInfo * __restrict ei __unused, char * __restrict s, size_t n, wchar_t wc, _MSKanjiState * __restrict psenc __unused, size_t * __restrict nresult) { int ret; /* check invalid sequence */ if (wc & ~0xffff) { ret = EILSEQ; goto err; } if (wc & 0xff00) { if (n < 2) { ret = E2BIG; goto err; } s[0] = (wc >> 8) & 0xff; s[1] = wc & 0xff; if (!_mskanji1(s[0] & 0xff) || !_mskanji2(s[1] & 0xff)) { ret = EILSEQ; goto err; } *nresult = 2; return (0); } else { if (n < 1) { ret = E2BIG; goto err; } s[0] = wc & 0xff; if (_mskanji1(s[0] & 0xff)) { ret = EILSEQ; goto err; } *nresult = 1; return (0); } err: *nresult = (size_t)-1; return (ret); } static __inline int /*ARGSUSED*/ _citrus_MSKanji_stdenc_wctocs(_MSKanjiEncodingInfo * __restrict ei, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { _index_t col, row; int offset; if ((_wc_t)wc < 0x80) { /* ISO-646 */ *csid = 0; *idx = (_index_t)wc; } else if ((_wc_t)wc < 0x100) { /* KANA */ *csid = 1; *idx = (_index_t)wc & 0x7F; } else { /* Kanji (containing Gaiji zone) */ /* * 94^2 zone (contains a part of Gaiji (0xED40 - 0xEEFC)): * 0x8140 - 0x817E -> 0x2121 - 0x215F * 0x8180 - 0x819E -> 0x2160 - 0x217E * 0x819F - 0x81FC -> 0x2221 - 0x227E * * 0x8240 - 0x827E -> 0x2321 - 0x235F * ... * 0x9F9F - 0x9FFc -> 0x5E21 - 0x5E7E * * 0xE040 - 0xE07E -> 0x5F21 - 0x5F5F * ... * 0xEF9F - 0xEFFC -> 0x7E21 - 0x7E7E * * extended Gaiji zone: * 0xF040 - 0xFCFC * * JIS X0213-plane2: * 0xF040 - 0xF09E -> 0x2121 - 0x217E * 0xF140 - 0xF19E -> 0x2321 - 0x237E * ... * 0xF240 - 0xF29E -> 0x2521 - 0x257E * * 0xF09F - 0xF0FC -> 0x2821 - 0x287E * 0xF29F - 0xF2FC -> 0x2C21 - 0x2C7E * ... * 0xF44F - 0xF49E -> 0x2F21 - 0x2F7E * * 0xF49F - 0xF4FC -> 0x6E21 - 0x6E7E * ... * 0xFC9F - 0xFCFC -> 0x7E21 - 0x7E7E */ row = ((_wc_t)wc >> 8) & 0xFF; col = (_wc_t)wc & 0xFF; if (!_mskanji1(row) || !_mskanji2(col)) return (EILSEQ); if ((ei->mode & MODE_JIS2004) == 0 || row < 0xF0) { *csid = 2; offset = 0x81; } else { *csid = 3; if ((_wc_t)wc <= 0xF49E) { offset = (_wc_t)wc >= 0xF29F || ((_wc_t)wc >= 0xF09F && (_wc_t)wc <= 0xF0FC) ? 0xED : 0xF0; } else offset = 0xCE; } row -= offset; if (row >= 0x5F) row -= 0x40; row = row * 2 + 0x21; col -= 0x1F; if (col >= 0x61) col -= 1; if (col > 0x7E) { row += 1; col -= 0x5E; } *idx = ((_index_t)row << 8) | col; } return (0); } static __inline int /*ARGSUSED*/ _citrus_MSKanji_stdenc_cstowc(_MSKanjiEncodingInfo * __restrict ei, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { uint32_t col, row; int offset; switch (csid) { case 0: /* ISO-646 */ if (idx >= 0x80) return (EILSEQ); *wc = (wchar_t)idx; break; case 1: /* kana */ if (idx >= 0x80) return (EILSEQ); *wc = (wchar_t)idx + 0x80; break; case 3: if ((ei->mode & MODE_JIS2004) == 0) return (EILSEQ); /*FALLTHROUGH*/ case 2: /* kanji */ row = (idx >> 8); if (row < 0x21) return (EILSEQ); if (csid == 3) { if (row <= 0x2F) offset = (row == 0x22 || row >= 0x26) ? 0xED : 0xF0; else if (row >= 0x4D && row <= 0x7E) offset = 0xCE; else return (EILSEQ); } else { if (row > 0x97) return (EILSEQ); offset = (row < 0x5F) ? 0x81 : 0xC1; } col = idx & 0xFF; if (col < 0x21 || col > 0x7E) return (EILSEQ); row -= 0x21; col -= 0x21; if ((row & 1) == 0) { col += 0x40; if (col >= 0x7F) col += 1; } else col += 0x9F; row = row / 2 + offset; *wc = ((wchar_t)row << 8) | col; break; default: return (EILSEQ); } return (0); } static __inline int /*ARGSUSED*/ _citrus_MSKanji_stdenc_get_state_desc_generic(_MSKanjiEncodingInfo * __restrict ei __unused, _MSKanjiState * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } static int /*ARGSUSED*/ _citrus_MSKanji_encoding_module_init(_MSKanjiEncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { const char *p; p = var; memset((void *)ei, 0, sizeof(*ei)); while (lenvar > 0) { switch (_bcs_toupper(*p)) { case 'J': MATCH(JIS2004, ei->mode |= MODE_JIS2004); break; } ++p; --lenvar; } return (0); } static void _citrus_MSKanji_encoding_module_uninit(_MSKanjiEncodingInfo *ei __unused) { } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(MSKanji); _CITRUS_STDENC_DEF_OPS(MSKanji); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/UES/citrus_ues.c =================================================================== --- stable/10/lib/libiconv_modules/UES/citrus_ues.c (revision 282274) +++ stable/10/lib/libiconv_modules/UES/citrus_ues.c (revision 282275) @@ -1,414 +1,414 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_ues.c,v 1.3 2012/02/12 13:51:29 wiz Exp $ */ /*- * Copyright (c)2006 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_bcs.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_ues.h" typedef struct { size_t mb_cur_max; int mode; #define MODE_C99 1 } _UESEncodingInfo; typedef struct { int chlen; char ch[12]; } _UESState; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_UES_##m #define _ENCODING_INFO _UESEncodingInfo #define _ENCODING_STATE _UESState #define _ENCODING_MB_CUR_MAX(_ei_) (_ei_)->mb_cur_max #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline void /*ARGSUSED*/ _citrus_UES_init_state(_UESEncodingInfo * __restrict ei __unused, _UESState * __restrict psenc) { psenc->chlen = 0; } #if 0 static __inline void /*ARGSUSED*/ _citrus_UES_pack_state(_UESEncodingInfo * __restrict ei __unused, void *__restrict pspriv, const _UESState * __restrict psenc) { memcpy(pspriv, (const void *)psenc, sizeof(*psenc)); } static __inline void /*ARGSUSED*/ _citrus_UES_unpack_state(_UESEncodingInfo * __restrict ei __unused, _UESState * __restrict psenc, const void * __restrict pspriv) { memcpy((void *)psenc, pspriv, sizeof(*psenc)); } #endif static __inline int to_int(int ch) { if (ch >= '0' && ch <= '9') return (ch - '0'); else if (ch >= 'A' && ch <= 'F') return ((ch - 'A') + 10); else if (ch >= 'a' && ch <= 'f') return ((ch - 'a') + 10); return (-1); } #define ESCAPE '\\' #define UCS2_ESC 'u' #define UCS4_ESC 'U' #define UCS2_BIT 16 #define UCS4_BIT 32 #define BMP_MAX UINT32_C(0xFFFF) #define UCS2_MAX UINT32_C(0x10FFFF) #define UCS4_MAX UINT32_C(0x7FFFFFFF) static const char *xdig = "0123456789abcdef"; static __inline int to_str(char *s, wchar_t wc, int bit) { char *p; p = s; *p++ = ESCAPE; switch (bit) { case UCS2_BIT: *p++ = UCS2_ESC; break; case UCS4_BIT: *p++ = UCS4_ESC; break; default: abort(); } do { *p++ = xdig[(wc >> (bit -= 4)) & 0xF]; } while (bit > 0); return (p - s); } static __inline bool is_hi_surrogate(wchar_t wc) { return (wc >= 0xD800 && wc <= 0xDBFF); } static __inline bool is_lo_surrogate(wchar_t wc) { return (wc >= 0xDC00 && wc <= 0xDFFF); } static __inline wchar_t surrogate_to_ucs(wchar_t hi, wchar_t lo) { hi -= 0xD800; lo -= 0xDC00; return ((hi << 10 | lo) + 0x10000); } static __inline void ucs_to_surrogate(wchar_t wc, wchar_t * __restrict hi, wchar_t * __restrict lo) { wc -= 0x10000; *hi = (wc >> 10) + 0xD800; *lo = (wc & 0x3FF) + 0xDC00; } static __inline bool is_basic(wchar_t wc) { return ((uint32_t)wc <= 0x9F && wc != 0x24 && wc != 0x40 && wc != 0x60); } static int _citrus_UES_mbrtowc_priv(_UESEncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, size_t n, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _UESState * __restrict psenc, size_t * __restrict nresult) { - const char *s0; + char *s0; int ch, head, num, tail; wchar_t hi, wc; if (*s == NULL) { _citrus_UES_init_state(ei, psenc); *nresult = 0; return (0); } s0 = *s; hi = (wchar_t)0; tail = 0; surrogate: wc = (wchar_t)0; head = tail; if (psenc->chlen == head) { if (n-- < 1) goto restart; psenc->ch[psenc->chlen++] = *s0++; } ch = (unsigned char)psenc->ch[head++]; if (ch == ESCAPE) { if (psenc->chlen == head) { if (n-- < 1) goto restart; psenc->ch[psenc->chlen++] = *s0++; } switch (psenc->ch[head]) { case UCS2_ESC: tail += 6; break; case UCS4_ESC: if (ei->mode & MODE_C99) { tail = 10; break; } /*FALLTHROUGH*/ default: tail = 0; } ++head; } for (; head < tail; ++head) { if (psenc->chlen == head) { if (n-- < 1) { restart: *s = s0; *nresult = (size_t)-2; return (0); } psenc->ch[psenc->chlen++] = *s0++; } num = to_int((int)(unsigned char)psenc->ch[head]); if (num < 0) { tail = 0; break; } wc = (wc << 4) | num; } head = 0; switch (tail) { case 0: break; case 6: if (hi != (wchar_t)0) break; if ((ei->mode & MODE_C99) == 0) { if (is_hi_surrogate(wc) != 0) { hi = wc; goto surrogate; } if ((uint32_t)wc <= 0x7F /* XXX */ || is_lo_surrogate(wc) != 0) break; goto done; } /*FALLTHROUGH*/ case 10: if (is_basic(wc) == 0 && (uint32_t)wc <= UCS4_MAX && is_hi_surrogate(wc) == 0 && is_lo_surrogate(wc) == 0) goto done; *nresult = (size_t)-1; return (EILSEQ); case 12: if (is_lo_surrogate(wc) == 0) break; wc = surrogate_to_ucs(hi, wc); goto done; } ch = (unsigned char)psenc->ch[0]; head = psenc->chlen; if (--head > 0) memmove(&psenc->ch[0], &psenc->ch[1], head); wc = (wchar_t)ch; done: psenc->chlen = head; if (pwc != NULL) *pwc = wc; *nresult = (size_t)((wc == 0) ? 0 : (s0 - *s)); *s = s0; return (0); } static int _citrus_UES_wcrtomb_priv(_UESEncodingInfo * __restrict ei, char * __restrict s, size_t n, wchar_t wc, _UESState * __restrict psenc, size_t * __restrict nresult) { wchar_t hi, lo; if (psenc->chlen != 0) return (EINVAL); if ((ei->mode & MODE_C99) ? is_basic(wc) : (uint32_t)wc <= 0x7F) { if (n-- < 1) goto e2big; psenc->ch[psenc->chlen++] = (char)wc; } else if ((uint32_t)wc <= BMP_MAX) { if (n < 6) goto e2big; psenc->chlen = to_str(&psenc->ch[0], wc, UCS2_BIT); } else if ((ei->mode & MODE_C99) == 0 && (uint32_t)wc <= UCS2_MAX) { if (n < 12) goto e2big; ucs_to_surrogate(wc, &hi, &lo); psenc->chlen += to_str(&psenc->ch[0], hi, UCS2_BIT); psenc->chlen += to_str(&psenc->ch[6], lo, UCS2_BIT); } else if ((ei->mode & MODE_C99) && (uint32_t)wc <= UCS4_MAX) { if (n < 10) goto e2big; psenc->chlen = to_str(&psenc->ch[0], wc, UCS4_BIT); } else { *nresult = (size_t)-1; return (EILSEQ); } memcpy(s, psenc->ch, psenc->chlen); *nresult = psenc->chlen; psenc->chlen = 0; return (0); e2big: *nresult = (size_t)-1; return (E2BIG); } /*ARGSUSED*/ static int _citrus_UES_stdenc_wctocs(_UESEncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { *csid = 0; *idx = (_index_t)wc; return (0); } static __inline int /*ARGSUSED*/ _citrus_UES_stdenc_cstowc(_UESEncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { if (csid != 0) return (EILSEQ); *wc = (wchar_t)idx; return (0); } static __inline int /*ARGSUSED*/ _citrus_UES_stdenc_get_state_desc_generic(_UESEncodingInfo * __restrict ei __unused, _UESState * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } static void /*ARGSUSED*/ _citrus_UES_encoding_module_uninit(_UESEncodingInfo *ei __unused) { /* ei seems to be unused */ } static int /*ARGSUSED*/ _citrus_UES_encoding_module_init(_UESEncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { const char *p; p = var; memset((void *)ei, 0, sizeof(*ei)); while (lenvar > 0) { switch (_bcs_toupper(*p)) { case 'C': MATCH(C99, ei->mode |= MODE_C99); break; } ++p; --lenvar; } ei->mb_cur_max = (ei->mode & MODE_C99) ? 10 : 12; return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(UES); _CITRUS_STDENC_DEF_OPS(UES); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/UTF1632/citrus_utf1632.c =================================================================== --- stable/10/lib/libiconv_modules/UTF1632/citrus_utf1632.c (revision 282274) +++ stable/10/lib/libiconv_modules/UTF1632/citrus_utf1632.c (revision 282275) @@ -1,453 +1,453 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_utf1632.c,v 1.9 2008/06/14 16:01:08 tnozaki Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_bcs.h" #include "citrus_utf1632.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct { int chlen; int current_endian; uint8_t ch[4]; } _UTF1632State; #define _ENDIAN_UNKNOWN 0 #define _ENDIAN_BIG 1 #define _ENDIAN_LITTLE 2 #if BYTE_ORDER == BIG_ENDIAN #define _ENDIAN_INTERNAL _ENDIAN_BIG #define _ENDIAN_SWAPPED _ENDIAN_LITTLE #else #define _ENDIAN_INTERNAL _ENDIAN_LITTLE #define _ENDIAN_SWAPPED _ENDIAN_BIG #endif #define _MODE_UTF32 0x00000001U #define _MODE_FORCE_ENDIAN 0x00000002U typedef struct { int preffered_endian; unsigned int cur_max; uint32_t mode; } _UTF1632EncodingInfo; #define _FUNCNAME(m) _citrus_UTF1632_##m #define _ENCODING_INFO _UTF1632EncodingInfo #define _ENCODING_STATE _UTF1632State #define _ENCODING_MB_CUR_MAX(_ei_) ((_ei_)->cur_max) #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline void /*ARGSUSED*/ _citrus_UTF1632_init_state(_UTF1632EncodingInfo *ei __unused, _UTF1632State *s) { memset(s, 0, sizeof(*s)); } static int _citrus_UTF1632_mbrtowc_priv(_UTF1632EncodingInfo *ei, wchar_t *pwc, - const char **s, size_t n, _UTF1632State *psenc, size_t *nresult) + char **s, size_t n, _UTF1632State *psenc, size_t *nresult) { - const char *s0; + char *s0; size_t result; wchar_t wc = L'\0'; int chlenbak, endian, needlen; s0 = *s; if (s0 == NULL) { _citrus_UTF1632_init_state(ei, psenc); *nresult = 0; /* state independent */ return (0); } result = 0; chlenbak = psenc->chlen; refetch: needlen = ((ei->mode & _MODE_UTF32) != 0 || chlenbak >= 2) ? 4 : 2; while (chlenbak < needlen) { if (n == 0) goto restart; psenc->ch[chlenbak++] = *s0++; n--; result++; } /* judge endian marker */ if ((ei->mode & _MODE_UTF32) == 0) { /* UTF16 */ if (psenc->ch[0] == 0xFE && psenc->ch[1] == 0xFF) { psenc->current_endian = _ENDIAN_BIG; chlenbak = 0; goto refetch; } else if (psenc->ch[0] == 0xFF && psenc->ch[1] == 0xFE) { psenc->current_endian = _ENDIAN_LITTLE; chlenbak = 0; goto refetch; } } else { /* UTF32 */ if (psenc->ch[0] == 0x00 && psenc->ch[1] == 0x00 && psenc->ch[2] == 0xFE && psenc->ch[3] == 0xFF) { psenc->current_endian = _ENDIAN_BIG; chlenbak = 0; goto refetch; } else if (psenc->ch[0] == 0xFF && psenc->ch[1] == 0xFE && psenc->ch[2] == 0x00 && psenc->ch[3] == 0x00) { psenc->current_endian = _ENDIAN_LITTLE; chlenbak = 0; goto refetch; } } endian = ((ei->mode & _MODE_FORCE_ENDIAN) != 0 || psenc->current_endian == _ENDIAN_UNKNOWN) ? ei->preffered_endian : psenc->current_endian; /* get wc */ if ((ei->mode & _MODE_UTF32) == 0) { /* UTF16 */ if (needlen == 2) { switch (endian) { case _ENDIAN_LITTLE: wc = (psenc->ch[0] | ((wchar_t)psenc->ch[1] << 8)); break; case _ENDIAN_BIG: wc = (psenc->ch[1] | ((wchar_t)psenc->ch[0] << 8)); break; default: goto ilseq; } if (wc >= 0xD800 && wc <= 0xDBFF) { /* surrogate high */ needlen = 4; goto refetch; } } else { /* surrogate low */ wc -= 0xD800; /* wc : surrogate high (see above) */ wc <<= 10; switch (endian) { case _ENDIAN_LITTLE: if (psenc->ch[3] < 0xDC || psenc->ch[3] > 0xDF) goto ilseq; wc |= psenc->ch[2]; wc |= (wchar_t)(psenc->ch[3] & 3) << 8; break; case _ENDIAN_BIG: if (psenc->ch[2]<0xDC || psenc->ch[2]>0xDF) goto ilseq; wc |= psenc->ch[3]; wc |= (wchar_t)(psenc->ch[2] & 3) << 8; break; default: goto ilseq; } wc += 0x10000; } } else { /* UTF32 */ switch (endian) { case _ENDIAN_LITTLE: wc = (psenc->ch[0] | ((wchar_t)psenc->ch[1] << 8) | ((wchar_t)psenc->ch[2] << 16) | ((wchar_t)psenc->ch[3] << 24)); break; case _ENDIAN_BIG: wc = (psenc->ch[3] | ((wchar_t)psenc->ch[2] << 8) | ((wchar_t)psenc->ch[1] << 16) | ((wchar_t)psenc->ch[0] << 24)); break; default: goto ilseq; } if (wc >= 0xD800 && wc <= 0xDFFF) goto ilseq; } *pwc = wc; psenc->chlen = 0; *nresult = result; *s = s0; return (0); ilseq: *nresult = (size_t)-1; psenc->chlen = 0; return (EILSEQ); restart: *nresult = (size_t)-2; psenc->chlen = chlenbak; *s = s0; return (0); } static int _citrus_UTF1632_wcrtomb_priv(_UTF1632EncodingInfo *ei, char *s, size_t n, wchar_t wc, _UTF1632State *psenc, size_t *nresult) { wchar_t wc2; static const char _bom[4] = { 0x00, 0x00, 0xFE, 0xFF, }; const char *bom = &_bom[0]; size_t cnt; cnt = (size_t)0; if (psenc->current_endian == _ENDIAN_UNKNOWN) { if ((ei->mode & _MODE_FORCE_ENDIAN) == 0) { if (ei->mode & _MODE_UTF32) cnt = 4; else { cnt = 2; bom += 2; } if (n < cnt) goto e2big; memcpy(s, bom, cnt); s += cnt, n -= cnt; } psenc->current_endian = ei->preffered_endian; } wc2 = 0; if ((ei->mode & _MODE_UTF32)==0) { /* UTF16 */ if (wc > 0xFFFF) { /* surrogate */ if (wc > 0x10FFFF) goto ilseq; if (n < 4) goto e2big; cnt += 4; wc -= 0x10000; wc2 = (wc & 0x3FF) | 0xDC00; wc = (wc>>10) | 0xD800; } else { if (n < 2) goto e2big; cnt += 2; } surrogate: switch (psenc->current_endian) { case _ENDIAN_BIG: s[1] = wc; s[0] = (wc >>= 8); break; case _ENDIAN_LITTLE: s[0] = wc; s[1] = (wc >>= 8); break; } if (wc2 != 0) { wc = wc2; wc2 = 0; s += 2; goto surrogate; } } else { /* UTF32 */ if (wc >= 0xD800 && wc <= 0xDFFF) goto ilseq; if (n < 4) goto e2big; cnt += 4; switch (psenc->current_endian) { case _ENDIAN_BIG: s[3] = wc; s[2] = (wc >>= 8); s[1] = (wc >>= 8); s[0] = (wc >>= 8); break; case _ENDIAN_LITTLE: s[0] = wc; s[1] = (wc >>= 8); s[2] = (wc >>= 8); s[3] = (wc >>= 8); break; } } *nresult = cnt; return (0); ilseq: *nresult = (size_t)-1; return (EILSEQ); e2big: *nresult = (size_t)-1; return (E2BIG); } static void parse_variable(_UTF1632EncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { const char *p; p = var; while (lenvar > 0) { switch (*p) { case 'B': case 'b': MATCH(big, ei->preffered_endian = _ENDIAN_BIG); break; case 'L': case 'l': MATCH(little, ei->preffered_endian = _ENDIAN_LITTLE); break; case 'i': case 'I': MATCH(internal, ei->preffered_endian = _ENDIAN_INTERNAL); break; case 's': case 'S': MATCH(swapped, ei->preffered_endian = _ENDIAN_SWAPPED); break; case 'F': case 'f': MATCH(force, ei->mode |= _MODE_FORCE_ENDIAN); break; case 'U': case 'u': MATCH(utf32, ei->mode |= _MODE_UTF32); break; } p++; lenvar--; } } static int /*ARGSUSED*/ _citrus_UTF1632_encoding_module_init(_UTF1632EncodingInfo * __restrict ei, const void * __restrict var, size_t lenvar) { memset((void *)ei, 0, sizeof(*ei)); parse_variable(ei, var, lenvar); ei->cur_max = ((ei->mode&_MODE_UTF32) == 0) ? 6 : 8; /* 6: endian + surrogate */ /* 8: endian + normal */ if (ei->preffered_endian == _ENDIAN_UNKNOWN) { ei->preffered_endian = _ENDIAN_BIG; } return (0); } static void /*ARGSUSED*/ _citrus_UTF1632_encoding_module_uninit(_UTF1632EncodingInfo *ei __unused) { } static __inline int /*ARGSUSED*/ _citrus_UTF1632_stdenc_wctocs(_UTF1632EncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, _wc_t wc) { *csid = 0; *idx = (_index_t)wc; return (0); } static __inline int /*ARGSUSED*/ _citrus_UTF1632_stdenc_cstowc(_UTF1632EncodingInfo * __restrict ei __unused, _wc_t * __restrict wc, _csid_t csid, _index_t idx) { if (csid != 0) return (EILSEQ); *wc = (_wc_t)idx; return (0); } static __inline int /*ARGSUSED*/ _citrus_UTF1632_stdenc_get_state_desc_generic(_UTF1632EncodingInfo * __restrict ei __unused, _UTF1632State * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(UTF1632); _CITRUS_STDENC_DEF_OPS(UTF1632); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/UTF7/citrus_utf7.c =================================================================== --- stable/10/lib/libiconv_modules/UTF7/citrus_utf7.c (revision 282274) +++ stable/10/lib/libiconv_modules/UTF7/citrus_utf7.c (revision 282275) @@ -1,502 +1,502 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_utf7.c,v 1.5 2006/08/23 12:57:24 tnozaki Exp $ */ /*- * Copyright (c)2004, 2005 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_utf7.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ #define EI_MASK UINT16_C(0xff) #define EI_DIRECT UINT16_C(0x100) #define EI_OPTION UINT16_C(0x200) #define EI_SPACE UINT16_C(0x400) typedef struct { uint16_t cell[0x80]; } _UTF7EncodingInfo; typedef struct { unsigned int mode: 1, /* whether base64 mode */ bits: 4, /* need to hold 0 - 15 */ cache: 22, /* 22 = BASE64_BIT + UTF16_BIT */ surrogate: 1; /* whether surrogate pair or not */ int chlen; char ch[4]; /* BASE64_IN, 3 * 6 = 18, most closed to UTF16_BIT */ } _UTF7State; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_UTF7_##m #define _ENCODING_INFO _UTF7EncodingInfo #define _ENCODING_STATE _UTF7State #define _ENCODING_MB_CUR_MAX(_ei_) 4 #define _ENCODING_IS_STATE_DEPENDENT 1 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline void /*ARGSUSED*/ _citrus_UTF7_init_state(_UTF7EncodingInfo * __restrict ei __unused, _UTF7State * __restrict s) { memset((void *)s, 0, sizeof(*s)); } #if 0 static __inline void /*ARGSUSED*/ _citrus_UTF7_pack_state(_UTF7EncodingInfo * __restrict ei __unused, void *__restrict pspriv, const _UTF7State * __restrict s) { memcpy(pspriv, (const void *)s, sizeof(*s)); } static __inline void /*ARGSUSED*/ _citrus_UTF7_unpack_state(_UTF7EncodingInfo * __restrict ei __unused, _UTF7State * __restrict s, const void * __restrict pspriv) { memcpy((void *)s, pspriv, sizeof(*s)); } #endif static const char base64[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" "abcdefghijklmnopqrstuvwxyz" "0123456789+/"; static const char direct[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" "abcdefghijklmnopqrstuvwxyz" "0123456789'(),-./:?"; static const char option[] = "!\"#$%&*;<=>@[]^_`{|}"; static const char spaces[] = " \t\r\n"; #define BASE64_BIT 6 #define UTF16_BIT 16 #define BASE64_MAX 0x3f #define UTF16_MAX UINT16_C(0xffff) #define UTF32_MAX UINT32_C(0x10ffff) #define BASE64_IN '+' #define BASE64_OUT '-' #define SHIFT7BIT(c) ((c) >> 7) #define ISSPECIAL(c) ((c) == '\0' || (c) == BASE64_IN) #define FINDLEN(ei, c) \ (SHIFT7BIT((c)) ? -1 : (((ei)->cell[(c)] & EI_MASK) - 1)) #define ISDIRECT(ei, c) (!SHIFT7BIT((c)) && (ISSPECIAL((c)) || \ ei->cell[(c)] & (EI_DIRECT | EI_OPTION | EI_SPACE))) #define ISSAFE(ei, c) (!SHIFT7BIT((c)) && (ISSPECIAL((c)) || \ (c < 0x80 && ei->cell[(c)] & (EI_DIRECT | EI_SPACE)))) /* surrogate pair */ #define SRG_BASE UINT32_C(0x10000) #define HISRG_MIN UINT16_C(0xd800) #define HISRG_MAX UINT16_C(0xdbff) #define LOSRG_MIN UINT16_C(0xdc00) #define LOSRG_MAX UINT16_C(0xdfff) static int _citrus_UTF7_mbtoutf16(_UTF7EncodingInfo * __restrict ei, - uint16_t * __restrict u16, const char ** __restrict s, size_t n, + uint16_t * __restrict u16, char ** __restrict s, size_t n, _UTF7State * __restrict psenc, size_t * __restrict nresult) { _UTF7State sv; - const char *s0; + char *s0; int done, i, len; s0 = *s; sv = *psenc; for (i = 0, done = 0; done == 0; i++) { if (i == psenc->chlen) { if (n-- < 1) { *nresult = (size_t)-2; *s = s0; sv.chlen = psenc->chlen; memcpy(sv.ch, psenc->ch, sizeof(sv.ch)); *psenc = sv; return (0); } psenc->ch[psenc->chlen++] = *s0++; } if (SHIFT7BIT((int)psenc->ch[i])) goto ilseq; if (!psenc->mode) { if (psenc->bits > 0 || psenc->cache > 0) return (EINVAL); if (psenc->ch[i] == BASE64_IN) psenc->mode = 1; else { if (!ISDIRECT(ei, (int)psenc->ch[i])) goto ilseq; *u16 = (uint16_t)psenc->ch[i]; done = 1; continue; } } else { if (psenc->ch[i] == BASE64_OUT && psenc->cache == 0) { psenc->mode = 0; *u16 = (uint16_t)BASE64_IN; done = 1; continue; } len = FINDLEN(ei, (int)psenc->ch[i]); if (len < 0) { if (psenc->bits >= BASE64_BIT) return (EINVAL); psenc->mode = 0; psenc->bits = psenc->cache = 0; if (psenc->ch[i] != BASE64_OUT) { if (!ISDIRECT(ei, (int)psenc->ch[i])) goto ilseq; *u16 = (uint16_t)psenc->ch[i]; done = 1; } else { psenc->chlen--; i--; } } else { psenc->cache = (psenc->cache << BASE64_BIT) | len; switch (psenc->bits) { case 0: case 2: case 4: case 6: case 8: psenc->bits += BASE64_BIT; break; case 10: case 12: case 14: psenc->bits -= (UTF16_BIT - BASE64_BIT); *u16 = (psenc->cache >> psenc->bits) & UTF16_MAX; done = 1; break; default: return (EINVAL); } } } } if (psenc->chlen > i) return (EINVAL); psenc->chlen = 0; *nresult = (size_t)((*u16 == 0) ? 0 : s0 - *s); *s = s0; return (0); ilseq: *nresult = (size_t)-1; return (EILSEQ); } static int _citrus_UTF7_mbrtowc_priv(_UTF7EncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, size_t n, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _UTF7State * __restrict psenc, size_t * __restrict nresult) { uint32_t u32; uint16_t hi, lo; size_t nr, siz; int err; if (*s == NULL) { _citrus_UTF7_init_state(ei, psenc); *nresult = (size_t)_ENCODING_IS_STATE_DEPENDENT; return (0); } if (psenc->surrogate) { hi = (psenc->cache >> psenc->bits) & UTF16_MAX; if (hi < HISRG_MIN || hi > HISRG_MAX) return (EINVAL); siz = 0; } else { err = _citrus_UTF7_mbtoutf16(ei, &hi, s, n, psenc, &nr); if (nr == (size_t)-1 || nr == (size_t)-2) { *nresult = nr; return (err); } if (err != 0) return (err); n -= nr; siz = nr; if (hi < HISRG_MIN || hi > HISRG_MAX) { u32 = (uint32_t)hi; goto done; } psenc->surrogate = 1; } err = _citrus_UTF7_mbtoutf16(ei, &lo, s, n, psenc, &nr); if (nr == (size_t)-1 || nr == (size_t)-2) { *nresult = nr; return (err); } if (err != 0) return (err); hi -= HISRG_MIN; lo -= LOSRG_MIN; u32 = (hi << 10 | lo) + SRG_BASE; siz += nr; done: if (pwc != NULL) *pwc = (wchar_t)u32; if (u32 == (uint32_t)0) { *nresult = (size_t)0; _citrus_UTF7_init_state(ei, psenc); } else { *nresult = siz; psenc->surrogate = 0; } return (err); } static int _citrus_UTF7_utf16tomb(_UTF7EncodingInfo * __restrict ei, char * __restrict s, size_t n __unused, uint16_t u16, _UTF7State * __restrict psenc, size_t * __restrict nresult) { int bits, i; if (psenc->chlen != 0 || psenc->bits > BASE64_BIT) return (EINVAL); if (ISSAFE(ei, u16)) { if (psenc->mode) { if (psenc->bits > 0) { bits = BASE64_BIT - psenc->bits; i = (psenc->cache << bits) & BASE64_MAX; psenc->ch[psenc->chlen++] = base64[i]; psenc->bits = psenc->cache = 0; } if (u16 == BASE64_OUT || FINDLEN(ei, u16) >= 0) psenc->ch[psenc->chlen++] = BASE64_OUT; psenc->mode = 0; } if (psenc->bits != 0) return (EINVAL); psenc->ch[psenc->chlen++] = (char)u16; if (u16 == BASE64_IN) psenc->ch[psenc->chlen++] = BASE64_OUT; } else { if (!psenc->mode) { if (psenc->bits > 0) return (EINVAL); psenc->ch[psenc->chlen++] = BASE64_IN; psenc->mode = 1; } psenc->cache = (psenc->cache << UTF16_BIT) | u16; bits = UTF16_BIT + psenc->bits; psenc->bits = bits % BASE64_BIT; while ((bits -= BASE64_BIT) >= 0) { i = (psenc->cache >> bits) & BASE64_MAX; psenc->ch[psenc->chlen++] = base64[i]; } } memcpy(s, psenc->ch, psenc->chlen); *nresult = psenc->chlen; psenc->chlen = 0; return (0); } static int _citrus_UTF7_wcrtomb_priv(_UTF7EncodingInfo * __restrict ei, char * __restrict s, size_t n, wchar_t wchar, _UTF7State * __restrict psenc, size_t * __restrict nresult) { uint32_t u32; uint16_t u16[2]; int err, i, len; size_t nr, siz; u32 = (uint32_t)wchar; if (u32 <= UTF16_MAX) { u16[0] = (uint16_t)u32; len = 1; } else if (u32 <= UTF32_MAX) { u32 -= SRG_BASE; u16[0] = (u32 >> 10) + HISRG_MIN; u16[1] = ((uint16_t)(u32 & UINT32_C(0x3ff))) + LOSRG_MIN; len = 2; } else { *nresult = (size_t)-1; return (EILSEQ); } siz = 0; for (i = 0; i < len; ++i) { err = _citrus_UTF7_utf16tomb(ei, s, n, u16[i], psenc, &nr); if (err != 0) return (err); /* XXX: state has been modified */ s += nr; n -= nr; siz += nr; } *nresult = siz; return (0); } static int /* ARGSUSED */ _citrus_UTF7_put_state_reset(_UTF7EncodingInfo * __restrict ei __unused, char * __restrict s, size_t n, _UTF7State * __restrict psenc, size_t * __restrict nresult) { int bits, pos; if (psenc->chlen != 0 || psenc->bits > BASE64_BIT || psenc->surrogate) return (EINVAL); if (psenc->mode) { if (psenc->bits > 0) { if (n-- < 1) return (E2BIG); bits = BASE64_BIT - psenc->bits; pos = (psenc->cache << bits) & BASE64_MAX; psenc->ch[psenc->chlen++] = base64[pos]; psenc->ch[psenc->chlen++] = BASE64_OUT; psenc->bits = psenc->cache = 0; } psenc->mode = 0; } if (psenc->bits != 0) return (EINVAL); if (n-- < 1) return (E2BIG); *nresult = (size_t)psenc->chlen; if (psenc->chlen > 0) { memcpy(s, psenc->ch, psenc->chlen); psenc->chlen = 0; } return (0); } static __inline int /*ARGSUSED*/ _citrus_UTF7_stdenc_wctocs(_UTF7EncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { *csid = 0; *idx = (_index_t)wc; return (0); } static __inline int /*ARGSUSED*/ _citrus_UTF7_stdenc_cstowc(_UTF7EncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { if (csid != 0) return (EILSEQ); *wc = (wchar_t)idx; return (0); } static __inline int /*ARGSUSED*/ _citrus_UTF7_stdenc_get_state_desc_generic(_UTF7EncodingInfo * __restrict ei __unused, _UTF7State * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } static void /*ARGSUSED*/ _citrus_UTF7_encoding_module_uninit(_UTF7EncodingInfo *ei __unused) { /* ei seems to be unused */ } static int /*ARGSUSED*/ _citrus_UTF7_encoding_module_init(_UTF7EncodingInfo * __restrict ei, const void * __restrict var __unused, size_t lenvar __unused) { const char *s; memset(ei, 0, sizeof(*ei)); #define FILL(str, flag) \ do { \ for (s = str; *s != '\0'; s++) \ ei->cell[*s & 0x7f] |= flag; \ } while (/*CONSTCOND*/0) FILL(base64, (s - base64) + 1); FILL(direct, EI_DIRECT); FILL(option, EI_OPTION); FILL(spaces, EI_SPACE); return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(UTF7); _CITRUS_STDENC_DEF_OPS(UTF7); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/UTF8/citrus_utf8.c =================================================================== --- stable/10/lib/libiconv_modules/UTF8/citrus_utf8.c (revision 282274) +++ stable/10/lib/libiconv_modules/UTF8/citrus_utf8.c (revision 282275) @@ -1,351 +1,351 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_utf8.c,v 1.17 2008/06/14 16:01:08 tnozaki Exp $ */ /*- * Copyright (c)2002 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ /*- * Copyright (c) 1993 * The Regents of the University of California. All rights reserved. * * This code is derived from software contributed to Berkeley by * Paul Borman at Krystal Technologies. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_utf8.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ static uint8_t _UTF8_count_array[256] = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 00 - 0F */ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 10 - 1F */ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 20 - 2F */ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 30 - 3F */ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 40 - 4F */ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 50 - 5F */ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 60 - 6F */ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, /* 70 - 7F */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 80 - 8F */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 90 - 9F */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* A0 - AF */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* B0 - BF */ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, /* C0 - CF */ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, /* D0 - DF */ 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, /* E0 - EF */ 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0 /* F0 - FF */ }; static uint8_t const *_UTF8_count = _UTF8_count_array; static const uint32_t _UTF8_range[] = { 0, /*dummy*/ 0x00000000, 0x00000080, 0x00000800, 0x00010000, 0x00200000, 0x04000000, 0x80000000, }; typedef struct { int chlen; char ch[6]; } _UTF8State; typedef void *_UTF8EncodingInfo; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_ei_, _func_) (_ei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_UTF8_##m #define _ENCODING_INFO _UTF8EncodingInfo #define _ENCODING_STATE _UTF8State #define _ENCODING_MB_CUR_MAX(_ei_) 6 #define _ENCODING_IS_STATE_DEPENDENT 0 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static size_t _UTF8_findlen(wchar_t v) { size_t i; uint32_t c; c = (uint32_t)v; /*XXX*/ for (i = 1; i < sizeof(_UTF8_range) / sizeof(_UTF8_range[0]) - 1; i++) if (c >= _UTF8_range[i] && c < _UTF8_range[i + 1]) return (i); return (-1); /*out of range*/ } static __inline bool _UTF8_surrogate(wchar_t wc) { return (wc >= 0xd800 && wc <= 0xdfff); } static __inline void /*ARGSUSED*/ _citrus_UTF8_init_state(_UTF8EncodingInfo *ei __unused, _UTF8State *s) { s->chlen = 0; } #if 0 static __inline void /*ARGSUSED*/ _citrus_UTF8_pack_state(_UTF8EncodingInfo *ei __unused, void *pspriv, const _UTF8State *s) { memcpy(pspriv, (const void *)s, sizeof(*s)); } static __inline void /*ARGSUSED*/ _citrus_UTF8_unpack_state(_UTF8EncodingInfo *ei __unused, _UTF8State *s, const void *pspriv) { memcpy((void *)s, pspriv, sizeof(*s)); } #endif static int -_citrus_UTF8_mbrtowc_priv(_UTF8EncodingInfo *ei, wchar_t *pwc, const char **s, +_citrus_UTF8_mbrtowc_priv(_UTF8EncodingInfo *ei, wchar_t *pwc, char **s, size_t n, _UTF8State *psenc, size_t *nresult) { - const char *s0; + char *s0; wchar_t wchar; int i; uint8_t c; s0 = *s; if (s0 == NULL) { _citrus_UTF8_init_state(ei, psenc); *nresult = 0; /* state independent */ return (0); } /* make sure we have the first byte in the buffer */ if (psenc->chlen == 0) { if (n-- < 1) goto restart; psenc->ch[psenc->chlen++] = *s0++; } c = _UTF8_count[psenc->ch[0] & 0xff]; if (c < 1 || c < psenc->chlen) goto ilseq; if (c == 1) wchar = psenc->ch[0] & 0xff; else { while (psenc->chlen < c) { if (n-- < 1) goto restart; psenc->ch[psenc->chlen++] = *s0++; } wchar = psenc->ch[0] & (0x7f >> c); for (i = 1; i < c; i++) { if ((psenc->ch[i] & 0xc0) != 0x80) goto ilseq; wchar <<= 6; wchar |= (psenc->ch[i] & 0x3f); } if (_UTF8_surrogate(wchar) || _UTF8_findlen(wchar) != c) goto ilseq; } if (pwc != NULL) *pwc = wchar; *nresult = (wchar == 0) ? 0 : s0 - *s; *s = s0; psenc->chlen = 0; return (0); ilseq: *nresult = (size_t)-1; return (EILSEQ); restart: *s = s0; *nresult = (size_t)-2; return (0); } static int _citrus_UTF8_wcrtomb_priv(_UTF8EncodingInfo *ei __unused, char *s, size_t n, wchar_t wc, _UTF8State *psenc __unused, size_t *nresult) { wchar_t c; size_t cnt; int i, ret; if (_UTF8_surrogate(wc)) { ret = EILSEQ; goto err; } cnt = _UTF8_findlen(wc); if (cnt <= 0 || cnt > 6) { /* invalid UCS4 value */ ret = EILSEQ; goto err; } if (n < cnt) { /* bound check failure */ ret = E2BIG; goto err; } c = wc; if (s) { for (i = cnt - 1; i > 0; i--) { s[i] = 0x80 | (c & 0x3f); c >>= 6; } s[0] = c; if (cnt == 1) s[0] &= 0x7f; else { s[0] &= (0x7f >> cnt); s[0] |= ((0xff00 >> cnt) & 0xff); } } *nresult = (size_t)cnt; return (0); err: *nresult = (size_t)-1; return (ret); } static __inline int /*ARGSUSED*/ _citrus_UTF8_stdenc_wctocs(_UTF8EncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { *csid = 0; *idx = (_citrus_index_t)wc; return (0); } static __inline int /*ARGSUSED*/ _citrus_UTF8_stdenc_cstowc(_UTF8EncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { if (csid != 0) return (EILSEQ); *wc = (wchar_t)idx; return (0); } static __inline int /*ARGSUSED*/ _citrus_UTF8_stdenc_get_state_desc_generic(_UTF8EncodingInfo * __restrict ei __unused, _UTF8State * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } static int /*ARGSUSED*/ _citrus_UTF8_encoding_module_init(_UTF8EncodingInfo * __restrict ei __unused, const void * __restrict var __unused, size_t lenvar __unused) { return (0); } static void /*ARGSUSED*/ _citrus_UTF8_encoding_module_uninit(_UTF8EncodingInfo *ei __unused) { } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(UTF8); _CITRUS_STDENC_DEF_OPS(UTF8); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/VIQR/citrus_viqr.c =================================================================== --- stable/10/lib/libiconv_modules/VIQR/citrus_viqr.c (revision 282274) +++ stable/10/lib/libiconv_modules/VIQR/citrus_viqr.c (revision 282275) @@ -1,498 +1,498 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_viqr.c,v 1.5 2011/11/19 18:20:13 tnozaki Exp $ */ /*- * Copyright (c)2006 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_bcs.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_viqr.h" #define ESCAPE '\\' /* * this table generated from RFC 1456. */ static const char *mnemonic_rfc1456[0x100] = { NULL , NULL , "A(?", NULL , NULL , "A(~", "A^~", NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , "Y?" , NULL , NULL , NULL , NULL , "Y~" , NULL , NULL , NULL , NULL , "Y." , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , "A." , "A('", "A(`", "A(.", "A^'", "A^`", "A^?", "A^.", "E~" , "E." , "E^'", "E^`", "E^?", "E^~", "E^.", "O^'", "O^`", "O^?", "O^~", "O^.", "O+.", "O+'", "O+`", "O+?", "I." , "O?" , "O." , "I?" , "U?" , "U~" , "U." , "Y`" , "O~" , "a('", "a(`", "a(.", "a^'", "a^`", "a^?", "a^.", "e~" , "e." , "e^'", "e^`", "e^?", "e^~", "e^.", "o^'", "o^`", "o^?", "o^~", "O+~", "O+" , "o^.", "o+`", "o+?", "i." , "U+.", "U+'", "U+`", "U+?", "o+" , "o+'", "U+" , "A`" , "A'" , "A^" , "A~" , "A?" , "A(" , "a(?", "a(~", "E`" , "E'" , "E^" , "E?" , "I`" , "I'" , "I~" , "y`" , "DD" , "u+'", "O`" , "O'" , "O^" , "a." , "y?" , "u+`", "u+?", "U`" , "U'" , "y~" , "y." , "Y'" , "o+~", "u+" , "a`" , "a'" , "a^" , "a~" , "a?" , "a(" , "u+~", "a^~", "e`" , "e'" , "e^" , "e?" , "i`" , "i'" , "i~" , "i?" , "dd" , "u+.", "o`" , "o'" , "o^" , "o~" , "o?" , "o." , "u." , "u`" , "u'" , "u~" , "u?" , "y'" , "o+.", "U+~", }; typedef struct { const char *name; wchar_t value; } mnemonic_def_t; static const mnemonic_def_t mnemonic_ext[] = { /* add extra mnemonic here (should be sorted by wchar_t order). */ }; static const size_t mnemonic_ext_size = sizeof(mnemonic_ext) / sizeof(mnemonic_def_t); static const char * mnemonic_ext_find(wchar_t wc, const mnemonic_def_t *head, size_t n) { const mnemonic_def_t *mid; for (; n > 0; n >>= 1) { mid = head + (n >> 1); if (mid->value == wc) return (mid->name); else if (mid->value < wc) { head = mid + 1; --n; } } return (NULL); } struct mnemonic_t; typedef TAILQ_HEAD(mnemonic_list_t, mnemonic_t) mnemonic_list_t; typedef struct mnemonic_t { TAILQ_ENTRY(mnemonic_t) entry; struct mnemonic_t *parent; mnemonic_list_t child; wchar_t value; int ascii; } mnemonic_t; static mnemonic_t * mnemonic_list_find(mnemonic_list_t *ml, int ch) { mnemonic_t *m; TAILQ_FOREACH(m, ml, entry) { if (m->ascii == ch) return (m); } return (NULL); } static mnemonic_t * mnemonic_create(mnemonic_t *parent, int ascii, wchar_t value) { mnemonic_t *m; m = malloc(sizeof(*m)); if (m != NULL) { m->parent = parent; m->ascii = ascii; m->value = value; TAILQ_INIT(&m->child); } return (m); } static int mnemonic_append_child(mnemonic_t *m, const char *s, wchar_t value, wchar_t invalid) { mnemonic_t *m0; int ch; ch = (unsigned char)*s++; if (ch == '\0') return (EINVAL); m0 = mnemonic_list_find(&m->child, ch); if (m0 == NULL) { m0 = mnemonic_create(m, ch, (wchar_t)ch); if (m0 == NULL) return (ENOMEM); TAILQ_INSERT_TAIL(&m->child, m0, entry); } m = m0; for (m0 = NULL; (ch = (unsigned char)*s) != '\0'; ++s) { m0 = mnemonic_list_find(&m->child, ch); if (m0 == NULL) { m0 = mnemonic_create(m, ch, invalid); if (m0 == NULL) return (ENOMEM); TAILQ_INSERT_TAIL(&m->child, m0, entry); } m = m0; } if (m0 == NULL) return (EINVAL); m0->value = value; return (0); } static void mnemonic_destroy(mnemonic_t *m) { mnemonic_t *m0; TAILQ_FOREACH(m0, &m->child, entry) mnemonic_destroy(m0); free(m); } typedef struct { mnemonic_t *mroot; wchar_t invalid; size_t mb_cur_max; } _VIQREncodingInfo; typedef struct { int chlen; char ch[MB_LEN_MAX]; } _VIQRState; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_VIQR_##m #define _ENCODING_INFO _VIQREncodingInfo #define _ENCODING_STATE _VIQRState #define _ENCODING_MB_CUR_MAX(_ei_) (_ei_)->mb_cur_max #define _ENCODING_IS_STATE_DEPENDENT 1 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) 0 static __inline void /*ARGSUSED*/ _citrus_VIQR_init_state(_VIQREncodingInfo * __restrict ei __unused, _VIQRState * __restrict psenc) { psenc->chlen = 0; } #if 0 static __inline void /*ARGSUSED*/ _citrus_VIQR_pack_state(_VIQREncodingInfo * __restrict ei __unused, void *__restrict pspriv, const _VIQRState * __restrict psenc) { memcpy(pspriv, (const void *)psenc, sizeof(*psenc)); } static __inline void /*ARGSUSED*/ _citrus_VIQR_unpack_state(_VIQREncodingInfo * __restrict ei __unused, _VIQRState * __restrict psenc, const void * __restrict pspriv) { memcpy((void *)psenc, pspriv, sizeof(*psenc)); } #endif static int _citrus_VIQR_mbrtowc_priv(_VIQREncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char ** __restrict s, size_t n, + wchar_t * __restrict pwc, char ** __restrict s, size_t n, _VIQRState * __restrict psenc, size_t * __restrict nresult) { mnemonic_t *m, *m0; - const char *s0; + char *s0; wchar_t wc; ssize_t i; int ch, escape; if (*s == NULL) { _citrus_VIQR_init_state(ei, psenc); *nresult = (size_t)_ENCODING_IS_STATE_DEPENDENT; return (0); } s0 = *s; i = 0; m = ei->mroot; for (escape = 0;;) { if (psenc->chlen == i) { if (n-- < 1) { *s = s0; *nresult = (size_t)-2; return (0); } psenc->ch[psenc->chlen++] = *s0++; } ch = (unsigned char)psenc->ch[i++]; if (ch == ESCAPE) { if (m != ei->mroot) break; escape = 1; continue; } if (escape != 0) break; m0 = mnemonic_list_find(&m->child, ch); if (m0 == NULL) break; m = m0; } while (m != ei->mroot) { --i; if (m->value != ei->invalid) break; m = m->parent; } if (ch == ESCAPE && m != ei->mroot) ++i; psenc->chlen -= i; memmove(&psenc->ch[0], &psenc->ch[i], psenc->chlen); wc = (m == ei->mroot) ? (wchar_t)ch : m->value; if (pwc != NULL) *pwc = wc; *nresult = (size_t)(wc == 0 ? 0 : s0 - *s); *s = s0; return (0); } static int _citrus_VIQR_wcrtomb_priv(_VIQREncodingInfo * __restrict ei, char * __restrict s, size_t n, wchar_t wc, _VIQRState * __restrict psenc, size_t * __restrict nresult) { mnemonic_t *m; const char *p; int ch = 0; switch (psenc->chlen) { case 0: case 1: break; default: return (EINVAL); } m = NULL; if ((uint32_t)wc <= 0xFF) { p = mnemonic_rfc1456[wc & 0xFF]; if (p != NULL) goto mnemonic_found; if (n-- < 1) goto e2big; ch = (unsigned int)wc; m = ei->mroot; if (psenc->chlen > 0) { m = mnemonic_list_find(&m->child, psenc->ch[0]); if (m == NULL) return (EINVAL); psenc->ch[0] = ESCAPE; } if (mnemonic_list_find(&m->child, ch) == NULL) { psenc->chlen = 0; m = NULL; } psenc->ch[psenc->chlen++] = ch; } else { p = mnemonic_ext_find(wc, &mnemonic_ext[0], mnemonic_ext_size); if (p == NULL) { *nresult = (size_t)-1; return (EILSEQ); } else { mnemonic_found: psenc->chlen = 0; while (*p != '\0') { if (n-- < 1) goto e2big; psenc->ch[psenc->chlen++] = *p++; } } } memcpy(s, psenc->ch, psenc->chlen); *nresult = psenc->chlen; if (m == ei->mroot) { psenc->ch[0] = ch; psenc->chlen = 1; } else psenc->chlen = 0; return (0); e2big: *nresult = (size_t)-1; return (E2BIG); } static int /* ARGSUSED */ _citrus_VIQR_put_state_reset(_VIQREncodingInfo * __restrict ei __unused, char * __restrict s __unused, size_t n __unused, _VIQRState * __restrict psenc, size_t * __restrict nresult) { switch (psenc->chlen) { case 0: case 1: break; default: return (EINVAL); } *nresult = 0; psenc->chlen = 0; return (0); } static __inline int /*ARGSUSED*/ _citrus_VIQR_stdenc_wctocs(_VIQREncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { *csid = 0; *idx = (_index_t)wc; return (0); } static __inline int /*ARGSUSED*/ _citrus_VIQR_stdenc_cstowc(_VIQREncodingInfo * __restrict ei __unused, wchar_t * __restrict pwc, _csid_t csid, _index_t idx) { if (csid != 0) return (EILSEQ); *pwc = (wchar_t)idx; return (0); } static void _citrus_VIQR_encoding_module_uninit(_VIQREncodingInfo *ei) { mnemonic_destroy(ei->mroot); } static int /*ARGSUSED*/ _citrus_VIQR_encoding_module_init(_VIQREncodingInfo * __restrict ei, const void * __restrict var __unused, size_t lenvar __unused) { const char *s; size_t i, n; int errnum; ei->mb_cur_max = 1; ei->invalid = (wchar_t)-1; ei->mroot = mnemonic_create(NULL, '\0', ei->invalid); if (ei->mroot == NULL) return (ENOMEM); for (i = 0; i < sizeof(mnemonic_rfc1456) / sizeof(const char *); ++i) { s = mnemonic_rfc1456[i]; if (s == NULL) continue; n = strlen(s); if (ei->mb_cur_max < n) ei->mb_cur_max = n; errnum = mnemonic_append_child(ei->mroot, s, (wchar_t)i, ei->invalid); if (errnum != 0) { _citrus_VIQR_encoding_module_uninit(ei); return (errnum); } } /* a + 1 < b + 1 here to silence gcc warning about unsigned < 0. */ for (i = 0; i + 1 < mnemonic_ext_size + 1; ++i) { const mnemonic_def_t *p; p = &mnemonic_ext[i]; n = strlen(p->name); if (ei->mb_cur_max < n) ei->mb_cur_max = n; errnum = mnemonic_append_child(ei->mroot, p->name, p->value, ei->invalid); if (errnum != 0) { _citrus_VIQR_encoding_module_uninit(ei); return (errnum); } } return (0); } static __inline int /*ARGSUSED*/ _citrus_VIQR_stdenc_get_state_desc_generic(_VIQREncodingInfo * __restrict ei __unused, _VIQRState * __restrict psenc, int * __restrict rstate) { *rstate = (psenc->chlen == 0) ? _STDENC_SDGEN_INITIAL : _STDENC_SDGEN_INCOMPLETE_CHAR; return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(VIQR); _CITRUS_STDENC_DEF_OPS(VIQR); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/ZW/citrus_zw.c =================================================================== --- stable/10/lib/libiconv_modules/ZW/citrus_zw.c (revision 282274) +++ stable/10/lib/libiconv_modules/ZW/citrus_zw.c (revision 282275) @@ -1,457 +1,457 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_zw.c,v 1.4 2008/06/14 16:01:08 tnozaki Exp $ */ /*- * Copyright (c)2004, 2006 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * */ #include #include #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_stdenc.h" #include "citrus_zw.h" /* ---------------------------------------------------------------------- * private stuffs used by templates */ typedef struct { int dummy; } _ZWEncodingInfo; typedef enum { NONE, AMBIGIOUS, ASCII, GB2312 } _ZWCharset; typedef struct { _ZWCharset charset; int chlen; char ch[4]; } _ZWState; #define _CEI_TO_EI(_cei_) (&(_cei_)->ei) #define _CEI_TO_STATE(_cei_, _func_) (_cei_)->states.s_##_func_ #define _FUNCNAME(m) _citrus_ZW_##m #define _ENCODING_INFO _ZWEncodingInfo #define _ENCODING_STATE _ZWState #define _ENCODING_MB_CUR_MAX(_ei_) MB_LEN_MAX #define _ENCODING_IS_STATE_DEPENDENT 1 #define _STATE_NEEDS_EXPLICIT_INIT(_ps_) ((_ps_)->charset != NONE) static __inline void /*ARGSUSED*/ _citrus_ZW_init_state(_ZWEncodingInfo * __restrict ei __unused, _ZWState * __restrict psenc) { psenc->chlen = 0; psenc->charset = NONE; } #if 0 static __inline void /*ARGSUSED*/ _citrus_ZW_pack_state(_ZWEncodingInfo * __restrict ei __unused, void *__restrict pspriv, const _ZWState * __restrict psenc) { memcpy(pspriv, (const void *)psenc, sizeof(*psenc)); } static __inline void /*ARGSUSED*/ _citrus_ZW_unpack_state(_ZWEncodingInfo * __restrict ei __unused, _ZWState * __restrict psenc, const void * __restrict pspriv) { memcpy((void *)psenc, pspriv, sizeof(*psenc)); } #endif static int _citrus_ZW_mbrtowc_priv(_ZWEncodingInfo * __restrict ei, - wchar_t * __restrict pwc, const char **__restrict s, size_t n, + wchar_t * __restrict pwc, char **__restrict s, size_t n, _ZWState * __restrict psenc, size_t * __restrict nresult) { - const char *s0; + char *s0; wchar_t wc; int ch, len; if (*s == NULL) { _citrus_ZW_init_state(ei, psenc); *nresult = (size_t)_ENCODING_IS_STATE_DEPENDENT; return (0); } s0 = *s; len = 0; #define STORE \ do { \ if (n-- < 1) { \ *nresult = (size_t)-2; \ *s = s0; \ return (0); \ } \ ch = (unsigned char)*s0++; \ if (len++ > MB_LEN_MAX || ch > 0x7F)\ goto ilseq; \ psenc->ch[psenc->chlen++] = ch; \ } while (/*CONSTCOND*/0) loop: switch (psenc->charset) { case ASCII: switch (psenc->chlen) { case 0: STORE; switch (psenc->ch[0]) { case '\0': case '\n': psenc->charset = NONE; } /*FALLTHROUGH*/ case 1: break; default: return (EINVAL); } ch = (unsigned char)psenc->ch[0]; if (ch > 0x7F) goto ilseq; wc = (wchar_t)ch; psenc->chlen = 0; break; case NONE: if (psenc->chlen != 0) return (EINVAL); STORE; ch = (unsigned char)psenc->ch[0]; if (ch != 'z') { if (ch != '\n' && ch != '\0') psenc->charset = ASCII; wc = (wchar_t)ch; psenc->chlen = 0; break; } psenc->charset = AMBIGIOUS; psenc->chlen = 0; /* FALLTHROUGH */ case AMBIGIOUS: if (psenc->chlen != 0) return (EINVAL); STORE; if (psenc->ch[0] != 'W') { psenc->charset = ASCII; wc = L'z'; break; } psenc->charset = GB2312; psenc->chlen = 0; /* FALLTHROUGH */ case GB2312: switch (psenc->chlen) { case 0: STORE; ch = (unsigned char)psenc->ch[0]; if (ch == '\0') { psenc->charset = NONE; wc = (wchar_t)ch; psenc->chlen = 0; break; } else if (ch == '\n') { psenc->charset = NONE; psenc->chlen = 0; goto loop; } /*FALLTHROUGH*/ case 1: STORE; if (psenc->ch[0] == ' ') { ch = (unsigned char)psenc->ch[1]; wc = (wchar_t)ch; psenc->chlen = 0; break; } else if (psenc->ch[0] == '#') { ch = (unsigned char)psenc->ch[1]; if (ch == '\n') { psenc->charset = NONE; wc = (wchar_t)ch; psenc->chlen = 0; break; } else if (ch == ' ') { wc = (wchar_t)ch; psenc->chlen = 0; break; } } ch = (unsigned char)psenc->ch[0]; if (ch < 0x21 || ch > 0x7E) goto ilseq; wc = (wchar_t)(ch << 8); ch = (unsigned char)psenc->ch[1]; if (ch < 0x21 || ch > 0x7E) { ilseq: *nresult = (size_t)-1; return (EILSEQ); } wc |= (wchar_t)ch; psenc->chlen = 0; break; default: return (EINVAL); } break; default: return (EINVAL); } if (pwc != NULL) *pwc = wc; *nresult = (size_t)(wc == 0 ? 0 : len); *s = s0; return (0); } static int /*ARGSUSED*/ _citrus_ZW_wcrtomb_priv(_ZWEncodingInfo * __restrict ei __unused, char *__restrict s, size_t n, wchar_t wc, _ZWState * __restrict psenc, size_t * __restrict nresult) { int ch; if (psenc->chlen != 0) return (EINVAL); if ((uint32_t)wc <= 0x7F) { ch = (unsigned char)wc; switch (psenc->charset) { case NONE: if (ch == '\0' || ch == '\n') psenc->ch[psenc->chlen++] = ch; else { if (n < 4) return (E2BIG); n -= 4; psenc->ch[psenc->chlen++] = 'z'; psenc->ch[psenc->chlen++] = 'W'; psenc->ch[psenc->chlen++] = ' '; psenc->ch[psenc->chlen++] = ch; psenc->charset = GB2312; } break; case GB2312: if (n < 2) return (E2BIG); n -= 2; if (ch == '\0') { psenc->ch[psenc->chlen++] = '\n'; psenc->ch[psenc->chlen++] = '\0'; psenc->charset = NONE; } else if (ch == '\n') { psenc->ch[psenc->chlen++] = '#'; psenc->ch[psenc->chlen++] = '\n'; psenc->charset = NONE; } else { psenc->ch[psenc->chlen++] = ' '; psenc->ch[psenc->chlen++] = ch; } break; default: return (EINVAL); } } else if ((uint32_t)wc <= 0x7E7E) { switch (psenc->charset) { case NONE: if (n < 2) return (E2BIG); n -= 2; psenc->ch[psenc->chlen++] = 'z'; psenc->ch[psenc->chlen++] = 'W'; psenc->charset = GB2312; /* FALLTHROUGH*/ case GB2312: if (n < 2) return (E2BIG); n -= 2; ch = (wc >> 8) & 0xFF; if (ch < 0x21 || ch > 0x7E) goto ilseq; psenc->ch[psenc->chlen++] = ch; ch = wc & 0xFF; if (ch < 0x21 || ch > 0x7E) goto ilseq; psenc->ch[psenc->chlen++] = ch; break; default: return (EINVAL); } } else { ilseq: *nresult = (size_t)-1; return (EILSEQ); } memcpy(s, psenc->ch, psenc->chlen); *nresult = psenc->chlen; psenc->chlen = 0; return (0); } static int /*ARGSUSED*/ _citrus_ZW_put_state_reset(_ZWEncodingInfo * __restrict ei __unused, char * __restrict s, size_t n, _ZWState * __restrict psenc, size_t * __restrict nresult) { if (psenc->chlen != 0) return (EINVAL); switch (psenc->charset) { case GB2312: if (n-- < 1) return (E2BIG); psenc->ch[psenc->chlen++] = '\n'; psenc->charset = NONE; /*FALLTHROUGH*/ case NONE: *nresult = psenc->chlen; if (psenc->chlen > 0) { memcpy(s, psenc->ch, psenc->chlen); psenc->chlen = 0; } break; default: return (EINVAL); } return (0); } static __inline int /*ARGSUSED*/ _citrus_ZW_stdenc_get_state_desc_generic(_ZWEncodingInfo * __restrict ei __unused, _ZWState * __restrict psenc, int * __restrict rstate) { switch (psenc->charset) { case NONE: if (psenc->chlen != 0) return (EINVAL); *rstate = _STDENC_SDGEN_INITIAL; break; case AMBIGIOUS: if (psenc->chlen != 0) return (EINVAL); *rstate = _STDENC_SDGEN_INCOMPLETE_SHIFT; break; case ASCII: case GB2312: switch (psenc->chlen) { case 0: *rstate = _STDENC_SDGEN_STABLE; break; case 1: *rstate = (psenc->ch[0] == '#') ? _STDENC_SDGEN_INCOMPLETE_SHIFT : _STDENC_SDGEN_INCOMPLETE_CHAR; break; default: return (EINVAL); } break; default: return (EINVAL); } return (0); } static __inline int /*ARGSUSED*/ _citrus_ZW_stdenc_wctocs(_ZWEncodingInfo * __restrict ei __unused, _csid_t * __restrict csid, _index_t * __restrict idx, wchar_t wc) { *csid = (_csid_t)(wc <= (wchar_t)0x7FU) ? 0 : 1; *idx = (_index_t)wc; return (0); } static __inline int /*ARGSUSED*/ _citrus_ZW_stdenc_cstowc(_ZWEncodingInfo * __restrict ei __unused, wchar_t * __restrict wc, _csid_t csid, _index_t idx) { switch (csid) { case 0: case 1: break; default: return (EINVAL); } *wc = (wchar_t)idx; return (0); } static void /*ARGSUSED*/ _citrus_ZW_encoding_module_uninit(_ZWEncodingInfo *ei __unused) { } static int /*ARGSUSED*/ _citrus_ZW_encoding_module_init(_ZWEncodingInfo * __restrict ei __unused, const void *__restrict var __unused, size_t lenvar __unused) { return (0); } /* ---------------------------------------------------------------------- * public interface for stdenc */ _CITRUS_STDENC_DECLS(ZW); _CITRUS_STDENC_DEF_OPS(ZW); #include "citrus_stdenc_template.h" Index: stable/10/lib/libiconv_modules/iconv_none/citrus_iconv_none.c =================================================================== --- stable/10/lib/libiconv_modules/iconv_none/citrus_iconv_none.c (revision 282274) +++ stable/10/lib/libiconv_modules/iconv_none/citrus_iconv_none.c (revision 282275) @@ -1,127 +1,127 @@ /* $FreeBSD$ */ /* $NetBSD: citrus_iconv_none.c,v 1.3 2011/05/23 14:45:44 joerg Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include "citrus_types.h" #include "citrus_module.h" #include "citrus_hash.h" #include "citrus_iconv.h" #include "citrus_iconv_none.h" /* ---------------------------------------------------------------------- */ _CITRUS_ICONV_DECLS(iconv_none); _CITRUS_ICONV_DEF_OPS(iconv_none); /* ---------------------------------------------------------------------- */ int _citrus_iconv_none_iconv_getops(struct _citrus_iconv_ops *ops) { memcpy(ops, &_citrus_iconv_none_iconv_ops, sizeof(_citrus_iconv_none_iconv_ops)); return (0); } static int /*ARGSUSED*/ _citrus_iconv_none_iconv_init_shared( struct _citrus_iconv_shared * __restrict ci, const char * __restrict in __unused, const char * __restrict out __unused) { ci->ci_closure = NULL; return (0); } static void /*ARGSUSED*/ _citrus_iconv_none_iconv_uninit_shared(struct _citrus_iconv_shared *ci __unused) { } static int /*ARGSUSED*/ _citrus_iconv_none_iconv_init_context(struct _citrus_iconv *cv) { cv->cv_closure = NULL; return (0); } static void /*ARGSUSED*/ _citrus_iconv_none_iconv_uninit_context(struct _citrus_iconv *cv __unused) { } static int /*ARGSUSED*/ _citrus_iconv_none_iconv_convert(struct _citrus_iconv * __restrict ci __unused, - const char * __restrict * __restrict in, size_t * __restrict inbytes, + char * __restrict * __restrict in, size_t * __restrict inbytes, char * __restrict * __restrict out, size_t * __restrict outbytes, uint32_t flags __unused, size_t * __restrict invalids) { size_t len; int e2big; if ((in == NULL) || (out == NULL) || (inbytes == NULL)) return (0); if ((*in == NULL) || (*out == NULL) || (*inbytes == 0) || (*outbytes == 0)) return (0); len = *inbytes; e2big = 0; if (*outbytes #include #include #include #include #include #include #include #include #include #include "citrus_namespace.h" #include "citrus_types.h" #include "citrus_module.h" #include "citrus_region.h" #include "citrus_mmap.h" #include "citrus_hash.h" #include "citrus_iconv.h" #include "citrus_stdenc.h" #include "citrus_mapper.h" #include "citrus_csmapper.h" #include "citrus_memstream.h" #include "citrus_iconv_std.h" #include "citrus_esdb.h" /* ---------------------------------------------------------------------- */ _CITRUS_ICONV_DECLS(iconv_std); _CITRUS_ICONV_DEF_OPS(iconv_std); /* ---------------------------------------------------------------------- */ int _citrus_iconv_std_iconv_getops(struct _citrus_iconv_ops *ops) { memcpy(ops, &_citrus_iconv_std_iconv_ops, sizeof(_citrus_iconv_std_iconv_ops)); return (0); } /* ---------------------------------------------------------------------- */ /* * convenience routines for stdenc. */ static __inline void save_encoding_state(struct _citrus_iconv_std_encoding *se) { if (se->se_ps) memcpy(se->se_pssaved, se->se_ps, _stdenc_get_state_size(se->se_handle)); } static __inline void restore_encoding_state(struct _citrus_iconv_std_encoding *se) { if (se->se_ps) memcpy(se->se_ps, se->se_pssaved, _stdenc_get_state_size(se->se_handle)); } static __inline void init_encoding_state(struct _citrus_iconv_std_encoding *se) { if (se->se_ps) _stdenc_init_state(se->se_handle, se->se_ps); } static __inline int mbtocsx(struct _citrus_iconv_std_encoding *se, - _csid_t *csid, _index_t *idx, const char **s, size_t n, size_t *nresult, + _csid_t *csid, _index_t *idx, char **s, size_t n, size_t *nresult, struct iconv_hooks *hooks) { return (_stdenc_mbtocs(se->se_handle, csid, idx, s, n, se->se_ps, nresult, hooks)); } static __inline int cstombx(struct _citrus_iconv_std_encoding *se, char *s, size_t n, _csid_t csid, _index_t idx, size_t *nresult, struct iconv_hooks *hooks) { return (_stdenc_cstomb(se->se_handle, s, n, csid, idx, se->se_ps, nresult, hooks)); } static __inline int wctombx(struct _citrus_iconv_std_encoding *se, char *s, size_t n, _wc_t wc, size_t *nresult, struct iconv_hooks *hooks) { return (_stdenc_wctomb(se->se_handle, s, n, wc, se->se_ps, nresult, hooks)); } static __inline int put_state_resetx(struct _citrus_iconv_std_encoding *se, char *s, size_t n, size_t *nresult) { return (_stdenc_put_state_reset(se->se_handle, s, n, se->se_ps, nresult)); } static __inline int get_state_desc_gen(struct _citrus_iconv_std_encoding *se, int *rstate) { struct _stdenc_state_desc ssd; int ret; ret = _stdenc_get_state_desc(se->se_handle, se->se_ps, _STDENC_SDID_GENERIC, &ssd); if (!ret) *rstate = ssd.u.generic.state; return (ret); } /* * init encoding context */ static int init_encoding(struct _citrus_iconv_std_encoding *se, struct _stdenc *cs, void *ps1, void *ps2) { int ret = -1; se->se_handle = cs; se->se_ps = ps1; se->se_pssaved = ps2; if (se->se_ps) ret = _stdenc_init_state(cs, se->se_ps); if (!ret && se->se_pssaved) ret = _stdenc_init_state(cs, se->se_pssaved); return (ret); } static int open_csmapper(struct _csmapper **rcm, const char *src, const char *dst, unsigned long *rnorm) { struct _csmapper *cm; int ret; ret = _csmapper_open(&cm, src, dst, 0, rnorm); if (ret) return (ret); if (_csmapper_get_src_max(cm) != 1 || _csmapper_get_dst_max(cm) != 1 || _csmapper_get_state_size(cm) != 0) { _csmapper_close(cm); return (EINVAL); } *rcm = cm; return (0); } static void close_dsts(struct _citrus_iconv_std_dst_list *dl) { struct _citrus_iconv_std_dst *sd; while ((sd = TAILQ_FIRST(dl)) != NULL) { TAILQ_REMOVE(dl, sd, sd_entry); _csmapper_close(sd->sd_mapper); free(sd); } } static int open_dsts(struct _citrus_iconv_std_dst_list *dl, const struct _esdb_charset *ec, const struct _esdb *dbdst) { struct _citrus_iconv_std_dst *sd, *sdtmp; unsigned long norm; int i, ret; sd = malloc(sizeof(*sd)); if (sd == NULL) return (errno); for (i = 0; i < dbdst->db_num_charsets; i++) { ret = open_csmapper(&sd->sd_mapper, ec->ec_csname, dbdst->db_charsets[i].ec_csname, &norm); if (ret == 0) { sd->sd_csid = dbdst->db_charsets[i].ec_csid; sd->sd_norm = norm; /* insert this mapper by sorted order. */ TAILQ_FOREACH(sdtmp, dl, sd_entry) { if (sdtmp->sd_norm > norm) { TAILQ_INSERT_BEFORE(sdtmp, sd, sd_entry); sd = NULL; break; } } if (sd) TAILQ_INSERT_TAIL(dl, sd, sd_entry); sd = malloc(sizeof(*sd)); if (sd == NULL) { ret = errno; close_dsts(dl); return (ret); } } else if (ret != ENOENT) { close_dsts(dl); free(sd); return (ret); } } free(sd); return (0); } static void close_srcs(struct _citrus_iconv_std_src_list *sl) { struct _citrus_iconv_std_src *ss; while ((ss = TAILQ_FIRST(sl)) != NULL) { TAILQ_REMOVE(sl, ss, ss_entry); close_dsts(&ss->ss_dsts); free(ss); } } static int open_srcs(struct _citrus_iconv_std_src_list *sl, const struct _esdb *dbsrc, const struct _esdb *dbdst) { struct _citrus_iconv_std_src *ss; int count = 0, i, ret; ss = malloc(sizeof(*ss)); if (ss == NULL) return (errno); TAILQ_INIT(&ss->ss_dsts); for (i = 0; i < dbsrc->db_num_charsets; i++) { ret = open_dsts(&ss->ss_dsts, &dbsrc->db_charsets[i], dbdst); if (ret) goto err; if (!TAILQ_EMPTY(&ss->ss_dsts)) { ss->ss_csid = dbsrc->db_charsets[i].ec_csid; TAILQ_INSERT_TAIL(sl, ss, ss_entry); ss = malloc(sizeof(*ss)); if (ss == NULL) { ret = errno; goto err; } count++; TAILQ_INIT(&ss->ss_dsts); } } free(ss); return (count ? 0 : ENOENT); err: free(ss); close_srcs(sl); return (ret); } /* do convert a character */ #define E_NO_CORRESPONDING_CHAR ENOENT /* XXX */ static int /*ARGSUSED*/ do_conv(const struct _citrus_iconv_std_shared *is, _csid_t *csid, _index_t *idx) { struct _citrus_iconv_std_dst *sd; struct _citrus_iconv_std_src *ss; _index_t tmpidx; int ret; TAILQ_FOREACH(ss, &is->is_srcs, ss_entry) { if (ss->ss_csid == *csid) { TAILQ_FOREACH(sd, &ss->ss_dsts, sd_entry) { ret = _csmapper_convert(sd->sd_mapper, &tmpidx, *idx, NULL); switch (ret) { case _MAPPER_CONVERT_SUCCESS: *csid = sd->sd_csid; *idx = tmpidx; return (0); case _MAPPER_CONVERT_NONIDENTICAL: break; case _MAPPER_CONVERT_SRC_MORE: /*FALLTHROUGH*/ case _MAPPER_CONVERT_DST_MORE: /*FALLTHROUGH*/ case _MAPPER_CONVERT_ILSEQ: return (EILSEQ); case _MAPPER_CONVERT_FATAL: return (EINVAL); } } break; } } return (E_NO_CORRESPONDING_CHAR); } /* ---------------------------------------------------------------------- */ static int /*ARGSUSED*/ _citrus_iconv_std_iconv_init_shared(struct _citrus_iconv_shared *ci, const char * __restrict src, const char * __restrict dst) { struct _citrus_esdb esdbdst, esdbsrc; struct _citrus_iconv_std_shared *is; int ret; is = malloc(sizeof(*is)); if (is == NULL) { ret = errno; goto err0; } ret = _citrus_esdb_open(&esdbsrc, src); if (ret) goto err1; ret = _citrus_esdb_open(&esdbdst, dst); if (ret) goto err2; ret = _stdenc_open(&is->is_src_encoding, esdbsrc.db_encname, esdbsrc.db_variable, esdbsrc.db_len_variable); if (ret) goto err3; ret = _stdenc_open(&is->is_dst_encoding, esdbdst.db_encname, esdbdst.db_variable, esdbdst.db_len_variable); if (ret) goto err4; is->is_use_invalid = esdbdst.db_use_invalid; is->is_invalid = esdbdst.db_invalid; TAILQ_INIT(&is->is_srcs); ret = open_srcs(&is->is_srcs, &esdbsrc, &esdbdst); if (ret) goto err5; _esdb_close(&esdbsrc); _esdb_close(&esdbdst); ci->ci_closure = is; return (0); err5: _stdenc_close(is->is_dst_encoding); err4: _stdenc_close(is->is_src_encoding); err3: _esdb_close(&esdbdst); err2: _esdb_close(&esdbsrc); err1: free(is); err0: return (ret); } static void _citrus_iconv_std_iconv_uninit_shared(struct _citrus_iconv_shared *ci) { struct _citrus_iconv_std_shared *is = ci->ci_closure; if (is == NULL) return; _stdenc_close(is->is_src_encoding); _stdenc_close(is->is_dst_encoding); close_srcs(&is->is_srcs); free(is); } static int _citrus_iconv_std_iconv_init_context(struct _citrus_iconv *cv) { const struct _citrus_iconv_std_shared *is = cv->cv_shared->ci_closure; struct _citrus_iconv_std_context *sc; char *ptr; size_t sz, szpsdst, szpssrc; szpssrc = _stdenc_get_state_size(is->is_src_encoding); szpsdst = _stdenc_get_state_size(is->is_dst_encoding); sz = (szpssrc + szpsdst)*2 + sizeof(struct _citrus_iconv_std_context); sc = malloc(sz); if (sc == NULL) return (errno); ptr = (char *)&sc[1]; if (szpssrc > 0) init_encoding(&sc->sc_src_encoding, is->is_src_encoding, ptr, ptr+szpssrc); else init_encoding(&sc->sc_src_encoding, is->is_src_encoding, NULL, NULL); ptr += szpssrc*2; if (szpsdst > 0) init_encoding(&sc->sc_dst_encoding, is->is_dst_encoding, ptr, ptr+szpsdst); else init_encoding(&sc->sc_dst_encoding, is->is_dst_encoding, NULL, NULL); cv->cv_closure = (void *)sc; return (0); } static void _citrus_iconv_std_iconv_uninit_context(struct _citrus_iconv *cv) { free(cv->cv_closure); } static int _citrus_iconv_std_iconv_convert(struct _citrus_iconv * __restrict cv, - const char * __restrict * __restrict in, size_t * __restrict inbytes, + char * __restrict * __restrict in, size_t * __restrict inbytes, char * __restrict * __restrict out, size_t * __restrict outbytes, uint32_t flags, size_t * __restrict invalids) { const struct _citrus_iconv_std_shared *is = cv->cv_shared->ci_closure; struct _citrus_iconv_std_context *sc = cv->cv_closure; _csid_t csid; _index_t idx; - const char *tmpin; + char *tmpin; size_t inval, szrin, szrout; int ret, state = 0; inval = 0; if (in == NULL || *in == NULL) { /* special cases */ if (out != NULL && *out != NULL) { /* init output state and store the shift sequence */ save_encoding_state(&sc->sc_src_encoding); save_encoding_state(&sc->sc_dst_encoding); szrout = 0; ret = put_state_resetx(&sc->sc_dst_encoding, *out, *outbytes, &szrout); if (ret) goto err; if (szrout == (size_t)-2) { /* too small to store the character */ ret = EINVAL; goto err; } *out += szrout; *outbytes -= szrout; } else /* otherwise, discard the shift sequence */ init_encoding_state(&sc->sc_dst_encoding); init_encoding_state(&sc->sc_src_encoding); *invalids = 0; return (0); } /* normal case */ for (;;) { if (*inbytes == 0) { ret = get_state_desc_gen(&sc->sc_src_encoding, &state); if (state == _STDENC_SDGEN_INITIAL || state == _STDENC_SDGEN_STABLE) break; } /* save the encoding states for the error recovery */ save_encoding_state(&sc->sc_src_encoding); save_encoding_state(&sc->sc_dst_encoding); /* mb -> csid/index */ tmpin = *in; szrin = szrout = 0; ret = mbtocsx(&sc->sc_src_encoding, &csid, &idx, &tmpin, *inbytes, &szrin, cv->cv_shared->ci_hooks); if (ret) goto err; if (szrin == (size_t)-2) { /* incompleted character */ ret = get_state_desc_gen(&sc->sc_src_encoding, &state); if (ret) { ret = EINVAL; goto err; } switch (state) { case _STDENC_SDGEN_INITIAL: case _STDENC_SDGEN_STABLE: /* fetch shift sequences only. */ goto next; } ret = EINVAL; goto err; } /* convert the character */ ret = do_conv(is, &csid, &idx); if (ret) { if (ret == E_NO_CORRESPONDING_CHAR) { inval++; szrout = 0; if ((((flags & _CITRUS_ICONV_F_HIDE_INVALID) == 0) && !cv->cv_shared->ci_discard_ilseq) && is->is_use_invalid) { ret = wctombx(&sc->sc_dst_encoding, *out, *outbytes, is->is_invalid, &szrout, cv->cv_shared->ci_hooks); if (ret) goto err; } goto next; } else goto err; } /* csid/index -> mb */ ret = cstombx(&sc->sc_dst_encoding, *out, *outbytes, csid, idx, &szrout, cv->cv_shared->ci_hooks); if (ret) goto err; next: *inbytes -= tmpin-*in; /* szrin is insufficient on \0. */ *in = tmpin; *outbytes -= szrout; *out += szrout; } *invalids = inval; return (0); err: restore_encoding_state(&sc->sc_src_encoding); restore_encoding_state(&sc->sc_dst_encoding); *invalids = inval; return (ret); } Index: stable/10/lib/libkiconv/xlat16_iconv.c =================================================================== --- stable/10/lib/libkiconv/xlat16_iconv.c (revision 282274) +++ stable/10/lib/libkiconv/xlat16_iconv.c (revision 282275) @@ -1,465 +1,464 @@ /*- * Copyright (c) 2003, 2005 Ryuichiro Imura * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * $FreeBSD$ */ /* * kiconv(3) requires shared linked, and reduce module size * when statically linked. */ #ifdef PIC #include #include #include #include #include #include #include #include #include #include #include #include #include "quirks.h" struct xlat16_table { uint32_t * idx[0x200]; void * data; size_t size; }; static struct xlat16_table kiconv_xlat16_open(const char *, const char *, int); static int chklocale(int, const char *); #ifdef ICONV_DLOPEN typedef void *iconv_t; static int my_iconv_init(void); static iconv_t (*my_iconv_open)(const char *, const char *); -static size_t (*my_iconv)(iconv_t, const char **, size_t *, char **, size_t *); +static size_t (*my_iconv)(iconv_t, char **, size_t *, char **, size_t *); static int (*my_iconv_close)(iconv_t); #else #include #define my_iconv_init() 0 #define my_iconv_open iconv_open #define my_iconv iconv #define my_iconv_close iconv_close #endif -static size_t my_iconv_char(iconv_t, const u_char **, size_t *, u_char **, size_t *); +static size_t my_iconv_char(iconv_t, u_char **, size_t *, u_char **, size_t *); int kiconv_add_xlat16_cspair(const char *tocode, const char *fromcode, int flag) { int error; size_t idxsize; struct xlat16_table xt; void *data; char *p; const char unicode[] = ENCODING_UNICODE; if ((flag & KICONV_WCTYPE) == 0 && strcmp(unicode, tocode) != 0 && strcmp(unicode, fromcode) != 0 && kiconv_lookupconv(unicode) == 0) { error = kiconv_add_xlat16_cspair(unicode, fromcode, flag); if (error) return (-1); error = kiconv_add_xlat16_cspair(tocode, unicode, flag); return (error); } if (kiconv_lookupcs(tocode, fromcode) == 0) return (0); if (flag & KICONV_WCTYPE) xt = kiconv_xlat16_open(fromcode, fromcode, flag); else xt = kiconv_xlat16_open(tocode, fromcode, flag); if (xt.size == 0) return (-1); idxsize = sizeof(xt.idx); if ((idxsize + xt.size) > ICONV_CSMAXDATALEN) { errno = E2BIG; return (-1); } if ((data = malloc(idxsize + xt.size)) != NULL) { p = data; memcpy(p, xt.idx, idxsize); p += idxsize; memcpy(p, xt.data, xt.size); error = kiconv_add_xlat16_table(tocode, fromcode, data, (int)(idxsize + xt.size)); return (error); } return (-1); } int kiconv_add_xlat16_cspairs(const char *foreigncode, const char *localcode) { int error, locale; error = kiconv_add_xlat16_cspair(foreigncode, localcode, KICONV_FROM_LOWER | KICONV_FROM_UPPER); if (error) return (error); error = kiconv_add_xlat16_cspair(localcode, foreigncode, KICONV_LOWER | KICONV_UPPER); if (error) return (error); locale = chklocale(LC_CTYPE, localcode); if (locale == 0) { error = kiconv_add_xlat16_cspair(KICONV_WCTYPE_NAME, localcode, KICONV_WCTYPE); if (error) return (error); } return (0); } static struct xlat16_table kiconv_xlat16_open(const char *tocode, const char *fromcode, int lcase) { u_char src[3], dst[4], *srcp, *dstp, ud, ld; int us, ls, ret; uint16_t c; uint32_t table[0x80]; size_t inbytesleft, outbytesleft, pre_q_size, post_q_size; struct xlat16_table xt; struct quirk_replace_list *pre_q_list, *post_q_list; iconv_t cd; char *p; xt.data = NULL; xt.size = 0; src[2] = '\0'; dst[3] = '\0'; ret = my_iconv_init(); if (ret) return (xt); cd = my_iconv_open(search_quirk(tocode, fromcode, &pre_q_list, &pre_q_size), search_quirk(fromcode, tocode, &post_q_list, &post_q_size)); if (cd == (iconv_t) (-1)) return (xt); if ((xt.data = malloc(0x200 * 0x80 * sizeof(uint32_t))) == NULL) return (xt); p = xt.data; for (ls = 0 ; ls < 0x200 ; ls++) { xt.idx[ls] = NULL; for (us = 0 ; us < 0x80 ; us++) { srcp = src; dstp = dst; inbytesleft = 2; outbytesleft = 3; bzero(dst, outbytesleft); c = ((ls & 0x100 ? us | 0x80 : us) << 8) | (u_char)ls; if (lcase & KICONV_WCTYPE) { if ((c & 0xff) == 0) c >>= 8; if (iswupper(c)) { c = towlower(c); if ((c & 0xff00) == 0) c <<= 8; table[us] = c | XLAT16_HAS_LOWER_CASE; } else if (iswlower(c)) { c = towupper(c); if ((c & 0xff00) == 0) c <<= 8; table[us] = c | XLAT16_HAS_UPPER_CASE; } else table[us] = 0; /* * store not NULL */ if (table[us]) xt.idx[ls] = table; continue; } c = quirk_vendor2unix(c, pre_q_list, pre_q_size); src[0] = (u_char)(c >> 8); src[1] = (u_char)c; - ret = my_iconv_char(cd, (const u_char **)&srcp, - &inbytesleft, &dstp, &outbytesleft); + ret = my_iconv_char(cd, &srcp, &inbytesleft, + &dstp, &outbytesleft); if (ret == -1) { table[us] = 0; continue; } ud = (u_char)dst[0]; ld = (u_char)dst[1]; switch(outbytesleft) { case 0: #ifdef XLAT16_ACCEPT_3BYTE_CHR table[us] = (ud << 8) | ld; table[us] |= (u_char)dst[2] << 16; table[us] |= XLAT16_IS_3BYTE_CHR; #else table[us] = 0; continue; #endif break; case 1: table[us] = quirk_unix2vendor((ud << 8) | ld, post_q_list, post_q_size); if ((table[us] >> 8) == 0) table[us] |= XLAT16_ACCEPT_NULL_OUT; break; case 2: table[us] = ud; if (lcase & KICONV_LOWER && ud != tolower(ud)) { table[us] |= (u_char)tolower(ud) << 16; table[us] |= XLAT16_HAS_LOWER_CASE; } if (lcase & KICONV_UPPER && ud != toupper(ud)) { table[us] |= (u_char)toupper(ud) << 16; table[us] |= XLAT16_HAS_UPPER_CASE; } break; } switch(inbytesleft) { case 0: if ((ls & 0xff) == 0) table[us] |= XLAT16_ACCEPT_NULL_IN; break; case 1: c = ls > 0xff ? us | 0x80 : us; if (lcase & KICONV_FROM_LOWER && c != tolower(c)) { table[us] |= (u_char)tolower(c) << 16; table[us] |= XLAT16_HAS_FROM_LOWER_CASE; } if (lcase & KICONV_FROM_UPPER && c != toupper(c)) { table[us] |= (u_char)toupper(c) << 16; table[us] |= XLAT16_HAS_FROM_UPPER_CASE; } break; } if (table[us] == 0) continue; /* * store not NULL */ xt.idx[ls] = table; } if (xt.idx[ls]) { memcpy(p, table, sizeof(table)); p += sizeof(table); } } my_iconv_close(cd); xt.size = p - (char *)xt.data; xt.data = realloc(xt.data, xt.size); return (xt); } static int chklocale(int category, const char *code) { char *p; int error = -1; p = strchr(setlocale(category, NULL), '.'); if (p++) { error = strcasecmp(code, p); if (error) { /* XXX - can't avoid calling quirk here... */ error = strcasecmp(code, kiconv_quirkcs(p, KICONV_VENDOR_MICSFT)); } } return (error); } #ifdef ICONV_DLOPEN static int my_iconv_init(void) { void *iconv_lib; iconv_lib = dlopen("libiconv.so", RTLD_LAZY | RTLD_GLOBAL); if (iconv_lib == NULL) { warn("Unable to load iconv library: %s\n", dlerror()); errno = ENOENT; return (-1); } my_iconv_open = dlsym(iconv_lib, "iconv_open"); my_iconv = dlsym(iconv_lib, "iconv"); my_iconv_close = dlsym(iconv_lib, "iconv_close"); return (0); } #endif static size_t -my_iconv_char(iconv_t cd, const u_char **ibuf, size_t * ilen, u_char **obuf, +my_iconv_char(iconv_t cd, u_char **ibuf, size_t * ilen, u_char **obuf, size_t * olen) { - const u_char *sp; - u_char *dp, ilocal[3], olocal[3]; + u_char *sp, *dp, ilocal[3], olocal[3]; u_char c1, c2; int ret; size_t ir, or; sp = *ibuf; dp = *obuf; ir = *ilen; bzero(*obuf, *olen); - ret = my_iconv(cd, (const char **)&sp, ilen, (char **)&dp, olen); + ret = my_iconv(cd, (char **)&sp, ilen, (char **)&dp, olen); c1 = (*obuf)[0]; c2 = (*obuf)[1]; if (ret == -1) { if (*ilen == ir - 1 && (*ibuf)[1] == '\0' && (c1 || c2)) return (0); else return (-1); } /* * We must judge if inbuf is a single byte char or double byte char. * Here, to judge, try first byte(*sp) conversion and compare. */ ir = 1; or = 3; bzero(olocal, or); memcpy(ilocal, *ibuf, sizeof(ilocal)); sp = ilocal; dp = olocal; - if ((my_iconv(cd,(const char **)&sp, &ir, (char **)&dp, &or)) != -1) { + if ((my_iconv(cd,(char **)&sp, &ir, (char **)&dp, &or)) != -1) { if (olocal[0] != c1) return (ret); if (olocal[1] == c2 && (*ibuf)[1] == '\0') { /* * inbuf is a single byte char */ *ilen = 1; *olen = or; return (ret); } switch(or) { case 0: case 1: if (olocal[1] == c2) { /* * inbuf is a single byte char, * so return false here. */ return (-1); } else { /* * inbuf is a double byte char */ return (ret); } break; case 2: /* * should compare second byte of inbuf */ break; } } else { /* * inbuf clould not be splitted, so inbuf is * a double byte char. */ return (ret); } /* * try second byte(*(sp+1)) conversion, and compare */ ir = 1; or = 3; bzero(olocal, or); sp = ilocal + 1; dp = olocal; - if ((my_iconv(cd,(const char **)&sp, &ir, (char **)&dp, &or)) != -1) { + if ((my_iconv(cd,(char **)&sp, &ir, (char **)&dp, &or)) != -1) { if (olocal[0] == c2) /* * inbuf is a single byte char */ return (-1); } return (ret); } #else /* statically linked */ #include #include #include int kiconv_add_xlat16_cspair(const char *tocode __unused, const char *fromcode __unused, int flag __unused) { errno = EINVAL; return (-1); } int kiconv_add_xlat16_cspairs(const char *tocode __unused, const char *fromcode __unused) { errno = EINVAL; return (-1); } #endif /* PIC */ Index: stable/10/sys/sys/param.h =================================================================== --- stable/10/sys/sys/param.h (revision 282274) +++ stable/10/sys/sys/param.h (revision 282275) @@ -1,347 +1,347 @@ /*- * Copyright (c) 1982, 1986, 1989, 1993 * The Regents of the University of California. All rights reserved. * (c) UNIX System Laboratories, Inc. * All or some portions of this file are derived from material licensed * to the University of California by American Telephone and Telegraph * Co. or Unix System Laboratories, Inc. and are reproduced herein with * the permission of UNIX System Laboratories, Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 4. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)param.h 8.3 (Berkeley) 4/4/95 * $FreeBSD$ */ #ifndef _SYS_PARAM_H_ #define _SYS_PARAM_H_ #include #define BSD 199506 /* System version (year & month). */ #define BSD4_3 1 #define BSD4_4 1 /* * __FreeBSD_version numbers are documented in the Porter's Handbook. * If you bump the version for any reason, you should update the documentation * there. * Currently this lives here in the doc/ repository: * * head/en_US.ISO8859-1/books/porters-handbook/book.xml * * scheme is: Rxx * 'R' is in the range 0 to 4 if this is a release branch or * x.0-CURRENT before RELENG_*_0 is created, otherwise 'R' is * in the range 5 to 9. */ #undef __FreeBSD_version -#define __FreeBSD_version 1001513 /* Master, propagated to newvers */ +#define __FreeBSD_version 1001514 /* Master, propagated to newvers */ /* * __FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD, * which by definition is always true on FreeBSD. This macro is also defined * on other systems that use the kernel of FreeBSD, such as GNU/kFreeBSD. * * It is tempting to use this macro in userland code when we want to enable * kernel-specific routines, and in fact it's fine to do this in code that * is part of FreeBSD itself. However, be aware that as presence of this * macro is still not widespread (e.g. older FreeBSD versions, 3rd party * compilers, etc), it is STRONGLY DISCOURAGED to check for this macro in * external applications without also checking for __FreeBSD__ as an * alternative. */ #undef __FreeBSD_kernel__ #define __FreeBSD_kernel__ #ifdef _KERNEL #define P_OSREL_SIGWAIT 700000 #define P_OSREL_SIGSEGV 700004 #define P_OSREL_MAP_ANON 800104 #define P_OSREL_MAJOR(x) ((x) / 100000) #endif #ifndef LOCORE #include #endif /* * Machine-independent constants (some used in following include files). * Redefined constants are from POSIX 1003.1 limits file. * * MAXCOMLEN should be >= sizeof(ac_comm) (see ) */ #include #define MAXCOMLEN 19 /* max command name remembered */ #define MAXINTERP PATH_MAX /* max interpreter file name length */ #define MAXLOGNAME 33 /* max login name length (incl. NUL) */ #define MAXUPRC CHILD_MAX /* max simultaneous processes */ #define NCARGS ARG_MAX /* max bytes for an exec function */ #define NGROUPS (NGROUPS_MAX+1) /* max number groups */ #define NOFILE OPEN_MAX /* max open files per process */ #define NOGROUP 65535 /* marker for empty group set member */ #define MAXHOSTNAMELEN 256 /* max hostname size */ #define SPECNAMELEN 63 /* max length of devicename */ /* More types and definitions used throughout the kernel. */ #ifdef _KERNEL #include #include #ifndef LOCORE #include #include #endif #ifndef FALSE #define FALSE 0 #endif #ifndef TRUE #define TRUE 1 #endif #endif #ifndef _KERNEL /* Signals. */ #include #endif /* Machine type dependent parameters. */ #include #ifndef _KERNEL #include #endif #ifndef DEV_BSHIFT #define DEV_BSHIFT 9 /* log2(DEV_BSIZE) */ #endif #define DEV_BSIZE (1<>PAGE_SHIFT) #endif /* * btodb() is messy and perhaps slow because `bytes' may be an off_t. We * want to shift an unsigned type to avoid sign extension and we don't * want to widen `bytes' unnecessarily. Assume that the result fits in * a daddr_t. */ #ifndef btodb #define btodb(bytes) /* calculates (bytes / DEV_BSIZE) */ \ (sizeof (bytes) > sizeof(long) \ ? (daddr_t)((unsigned long long)(bytes) >> DEV_BSHIFT) \ : (daddr_t)((unsigned long)(bytes) >> DEV_BSHIFT)) #endif #ifndef dbtob #define dbtob(db) /* calculates (db * DEV_BSIZE) */ \ ((off_t)(db) << DEV_BSHIFT) #endif #define PRIMASK 0x0ff #define PCATCH 0x100 /* OR'd with pri for tsleep to check signals */ #define PDROP 0x200 /* OR'd with pri to stop re-entry of interlock mutex */ #define NZERO 0 /* default "nice" */ #define NBBY 8 /* number of bits in a byte */ #define NBPW sizeof(int) /* number of bytes per word (integer) */ #define CMASK 022 /* default file mask: S_IWGRP|S_IWOTH */ #define NODEV (dev_t)(-1) /* non-existent device */ /* * File system parameters and macros. * * MAXBSIZE - Filesystems are made out of blocks of at most MAXBSIZE bytes * per block. MAXBSIZE may be made larger without effecting * any existing filesystems as long as it does not exceed MAXPHYS, * and may be made smaller at the risk of not being able to use * filesystems which require a block size exceeding MAXBSIZE. * * BKVASIZE - Nominal buffer space per buffer, in bytes. BKVASIZE is the * minimum KVM memory reservation the kernel is willing to make. * Filesystems can of course request smaller chunks. Actual * backing memory uses a chunk size of a page (PAGE_SIZE). * * If you make BKVASIZE too small you risk seriously fragmenting * the buffer KVM map which may slow things down a bit. If you * make it too big the kernel will not be able to optimally use * the KVM memory reserved for the buffer cache and will wind * up with too-few buffers. * * The default is 16384, roughly 2x the block size used by a * normal UFS filesystem. */ #define MAXBSIZE 65536 /* must be power of 2 */ #define BKVASIZE 16384 /* must be power of 2 */ #define BKVAMASK (BKVASIZE-1) /* * MAXPATHLEN defines the longest permissible path length after expanding * symbolic links. It is used to allocate a temporary buffer from the buffer * pool in which to do the name expansion, hence should be a power of two, * and must be less than or equal to MAXBSIZE. MAXSYMLINKS defines the * maximum number of symbolic links that may be expanded in a path name. * It should be set high enough to allow all legitimate uses, but halt * infinite loops reasonably quickly. */ #define MAXPATHLEN PATH_MAX #define MAXSYMLINKS 32 /* Bit map related macros. */ #define setbit(a,i) (((unsigned char *)(a))[(i)/NBBY] |= 1<<((i)%NBBY)) #define clrbit(a,i) (((unsigned char *)(a))[(i)/NBBY] &= ~(1<<((i)%NBBY))) #define isset(a,i) \ (((const unsigned char *)(a))[(i)/NBBY] & (1<<((i)%NBBY))) #define isclr(a,i) \ ((((const unsigned char *)(a))[(i)/NBBY] & (1<<((i)%NBBY))) == 0) /* Macros for counting and rounding. */ #ifndef howmany #define howmany(x, y) (((x)+((y)-1))/(y)) #endif #define nitems(x) (sizeof((x)) / sizeof((x)[0])) #define rounddown(x, y) (((x)/(y))*(y)) #define rounddown2(x, y) ((x)&(~((y)-1))) /* if y is power of two */ #define roundup(x, y) ((((x)+((y)-1))/(y))*(y)) /* to any y */ #define roundup2(x, y) (((x)+((y)-1))&(~((y)-1))) /* if y is powers of two */ #define powerof2(x) ((((x)-1)&(x))==0) /* Macros for min/max. */ #define MIN(a,b) (((a)<(b))?(a):(b)) #define MAX(a,b) (((a)>(b))?(a):(b)) #ifdef _KERNEL /* * Basic byte order function prototypes for non-inline functions. */ #ifndef LOCORE #ifndef _BYTEORDER_PROTOTYPED #define _BYTEORDER_PROTOTYPED __BEGIN_DECLS __uint32_t htonl(__uint32_t); __uint16_t htons(__uint16_t); __uint32_t ntohl(__uint32_t); __uint16_t ntohs(__uint16_t); __END_DECLS #endif #endif #ifndef lint #ifndef _BYTEORDER_FUNC_DEFINED #define _BYTEORDER_FUNC_DEFINED #define htonl(x) __htonl(x) #define htons(x) __htons(x) #define ntohl(x) __ntohl(x) #define ntohs(x) __ntohs(x) #endif /* !_BYTEORDER_FUNC_DEFINED */ #endif /* lint */ #endif /* _KERNEL */ /* * Scale factor for scaled integers used to count %cpu time and load avgs. * * The number of CPU `tick's that map to a unique `%age' can be expressed * by the formula (1 / (2 ^ (FSHIFT - 11))). The maximum load average that * can be calculated (assuming 32 bits) can be closely approximated using * the formula (2 ^ (2 * (16 - FSHIFT))) for (FSHIFT < 15). * * For the scheduler to maintain a 1:1 mapping of CPU `tick' to `%age', * FSHIFT must be at least 11; this gives us a maximum load avg of ~1024. */ #define FSHIFT 11 /* bits to right of fixed binary point */ #define FSCALE (1<> (PAGE_SHIFT - DEV_BSHIFT)) #define ctodb(db) /* calculates pages to devblks */ \ ((db) << (PAGE_SHIFT - DEV_BSHIFT)) /* * Old spelling of __containerof(). */ #define member2struct(s, m, x) \ ((struct s *)(void *)((char *)(x) - offsetof(struct s, m))) /* * Access a variable length array that has been declared as a fixed * length array. */ #define __PAST_END(array, offset) (((__typeof__(*(array)) *)(array))[offset]) #endif /* _SYS_PARAM_H_ */ Index: stable/10/usr.bin/iconv/iconv.c =================================================================== --- stable/10/usr.bin/iconv/iconv.c (revision 282274) +++ stable/10/usr.bin/iconv/iconv.c (revision 282275) @@ -1,220 +1,219 @@ /* $FreeBSD$ */ /* $NetBSD: iconv.c,v 1.16 2009/02/20 15:28:21 yamt Exp $ */ /*- * Copyright (c)2003 Citrus Project, * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include #include #include #include static int do_conv(FILE *, const char *, const char *, bool, bool); static int do_list(unsigned int, const char * const *, void *); static void usage(void) __dead2; static const struct option long_options[] = { {"from-code", required_argument, NULL, 'f'}, {"list", no_argument, NULL, 'l'}, {"silent", no_argument, NULL, 's'}, {"to-code", required_argument, NULL, 't'}, {NULL, no_argument, NULL, 0} }; static void usage(void) { (void)fprintf(stderr, "Usage:\t%1$s [-cs] -f -t [file ...]\n" "\t%1$s -f [-cs] [-t ] [file ...]\n" "\t%1$s -t [-cs] [-f ] [file ...]\n" "\t%1$s -l\n", getprogname()); exit(1); } #define INBUFSIZE 1024 #define OUTBUFSIZE (INBUFSIZE * 2) static int do_conv(FILE *fp, const char *from, const char *to, bool silent, bool hide_invalid) { iconv_t cd; - char inbuf[INBUFSIZE], outbuf[OUTBUFSIZE], *out; + char inbuf[INBUFSIZE], outbuf[OUTBUFSIZE], *in, *out; unsigned long long invalids; - const char *in; size_t inbytes, outbytes, ret; if ((cd = iconv_open(to, from)) == (iconv_t)-1) err(EXIT_FAILURE, "iconv_open(%s, %s)", to, from); if (hide_invalid) { int arg = 1; if (iconvctl(cd, ICONV_SET_DISCARD_ILSEQ, (void *)&arg) == -1) err(EXIT_FAILURE, NULL); } invalids = 0; while ((inbytes = fread(inbuf, 1, INBUFSIZE, fp)) > 0) { in = inbuf; while (inbytes > 0) { size_t inval; out = outbuf; outbytes = OUTBUFSIZE; ret = __iconv(cd, &in, &inbytes, &out, &outbytes, 0, &inval); invalids += inval; if (outbytes < OUTBUFSIZE) (void)fwrite(outbuf, 1, OUTBUFSIZE - outbytes, stdout); if (ret == (size_t)-1 && errno != E2BIG) { if (errno != EINVAL || in == inbuf) err(EXIT_FAILURE, "iconv()"); /* incomplete input character */ (void)memmove(inbuf, in, inbytes); ret = fread(inbuf + inbytes, 1, INBUFSIZE - inbytes, fp); if (ret == 0) { fflush(stdout); if (feof(fp)) errx(EXIT_FAILURE, "unexpected end of file; " "the last character is " "incomplete."); else err(EXIT_FAILURE, "fread()"); } in = inbuf; inbytes += ret; } } } /* reset the shift state of the output buffer */ outbytes = OUTBUFSIZE; out = outbuf; ret = iconv(cd, NULL, NULL, &out, &outbytes); if (ret == (size_t)-1) err(EXIT_FAILURE, "iconv()"); if (outbytes < OUTBUFSIZE) (void)fwrite(outbuf, 1, OUTBUFSIZE - outbytes, stdout); if (invalids > 0 && !silent) warnx("warning: invalid characters: %llu", invalids); iconv_close(cd); return (invalids > 0); } static int do_list(unsigned int n, const char * const *list, void *data __unused) { unsigned int i; for(i = 0; i < n; i++) { printf("%s", list[i]); if (i < n - 1) printf(" "); } printf("\n"); return (1); } int main(int argc, char **argv) { FILE *fp; char *opt_f, *opt_t; int ch, i, res; bool opt_c = false, opt_s = false; opt_f = opt_t = strdup(""); setlocale(LC_ALL, ""); setprogname(argv[0]); while ((ch = getopt_long(argc, argv, "csLlf:t:", long_options, NULL)) != -1) { switch (ch) { case 'c': opt_c = true; break; case 's': opt_s = true; break; case 'l': /* list */ if (opt_s || opt_c || strcmp(opt_f, "") != 0 || strcmp(opt_t, "") != 0) { warnx("-l is not allowed with other flags."); usage(); } iconvlist(do_list, NULL); return (EXIT_SUCCESS); case 'f': /* from */ if (optarg != NULL) opt_f = strdup(optarg); break; case 't': /* to */ if (optarg != NULL) opt_t = strdup(optarg); break; default: usage(); } } argc -= optind; argv += optind; if ((strcmp(opt_f, "") == 0) && (strcmp(opt_t, "") == 0)) usage(); if (argc == 0) res = do_conv(stdin, opt_f, opt_t, opt_s, opt_c); else { res = 0; for (i = 0; i < argc; i++) { fp = (strcmp(argv[i], "-") != 0) ? fopen(argv[i], "r") : stdin; if (fp == NULL) err(EXIT_FAILURE, "Cannot open `%s'", argv[i]); res |= do_conv(fp, opt_f, opt_t, opt_s, opt_c); (void)fclose(fp); } } return (res == 0 ? EXIT_SUCCESS : EXIT_FAILURE); } Index: stable/10 =================================================================== --- stable/10 (revision 282274) +++ stable/10 (revision 282275) Property changes on: stable/10 ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head:r275805,281550,281591