diff --git a/en_US.ISO8859-1/books/arch-handbook/Makefile b/en_US.ISO8859-1/books/arch-handbook/Makefile index 371551ab9e..acf0c62b67 100644 --- a/en_US.ISO8859-1/books/arch-handbook/Makefile +++ b/en_US.ISO8859-1/books/arch-handbook/Makefile @@ -1,35 +1,38 @@ # -# $FreeBSD: doc/en_US.ISO_8859-1/books/developers-handbook/Makefile,v 1.2 2001/05/02 01:53:13 murray Exp $ +# $FreeBSD: doc/en_US.ISO_8859-1/books/developers-handbook/Makefile,v 1.3 2001/05/11 10:27:00 murray Exp $ # # Build the FreeBSD Developers' Handbook. # MAINTAINER=asmodai@FreeBSD.org DOC?= book FORMATS?= html-split INSTALL_COMPRESSED?= gz INSTALL_ONLY_COMPRESSED?= # # SRCS lists the individual SGML files that make up the document. Changes # to any of these files will force a rebuild # # SGML content SRCS= book.sgml SRCS+= tools/chapter.sgml SRCS+= secure/chapter.sgml SRCS+= locking/chapter.sgml SRCS+= isa/chapter.sgml SRCS+= pci/chapter.sgml SRCS+= usb/chapter.sgml SRCS+= scsi/chapter.sgml SRCS+= x86/chapter.sgml +SRCS+= vm/chapter.sgml +SRCS+= dma/chapter.sgml +SRCS+= ipv6/chapter.sgml # Entities DOC_PREFIX?= ${.CURDIR}/../../.. .include "${DOC_PREFIX}/share/mk/doc.project.mk" diff --git a/en_US.ISO8859-1/books/arch-handbook/book.sgml b/en_US.ISO8859-1/books/arch-handbook/book.sgml index a1a485a73d..10879f994e 100644 --- a/en_US.ISO8859-1/books/arch-handbook/book.sgml +++ b/en_US.ISO8859-1/books/arch-handbook/book.sgml @@ -1,517 +1,518 @@ %bookinfo; %man; %chapters; %authors; %mailing-lists; ]> FreeBSD Developers' Handbook The FreeBSD Documentation Project August 2000 2000 2001 The FreeBSD Documentation Project &bookinfo.legalnotice; Welcome to the Developers' Handbook. This manual is a work in progress and is the work of many individuals. Many sections do not yet exist and some of those that do exist need to be updated. If you are interested in helping with this project, send email to the &a.doc;. The latest version of this document is always available from the FreeBSD World Wide Web server. It may also be downloaded in a variety of formats and compression options from the FreeBSD FTP server or one of the numerous mirror sites. Introduction Developing on FreeBSD This will need to discuss FreeBSD as a development platform, the vision of BSD, architectural overview, layout of /usr/src, history, etc. Thank you for considering FreeBSD as your development platform! We hope it will not let you down. The BSD Vision Architectural Overview The Layout of /usr/src The complete source code to FreeBSD is available from our public CVS repository. The source code is normally installed in /usr/src which contains the following subdirectories. Directory Description bin/ Source for files in /bin contrib/ Source for files from contributed software. crypto/ DES source etc/ Source for files in /etc games/ Source for files in /usr/games gnu/ Utilities covered by the GNU Public License include/ Source for files in /usr/include kerberosIV/ Source for Kerbereros version IV kerberos5/ Source for Kerbereros version 5 lib/ Source for files in /usr/lib libexec/ Source for files in /usr/libexec release/ Files required to produce a FreeBSD release sbin/ Source for files in /sbin secure/ FreeSec sources share/ Source for files in /sbin sys/ Kernel source files tools/ Tools used for maintenance and testing of FreeBSD usr.bin/ Source for files in /usr/bin usr.sbin/ Source for files in /usr/sbin Basics &chap.tools; &chap.secure; Kernel History of the Unix Kernel Some history of the Unix/BSD kernel, system calls, how do processes work, blocking, scheduling, threads (kernel), context switching, signals, interrupts, modules, etc. &chap.locking; - Memory and Virtual Memory + Memory Management - - Virtual Memory + &chap.vm; + &chap.dma; - VM, paging, swapping, allocating memory, testing for - memory leaks, mmap, vnodes, etc. + - - I/O System UFS UFS, FFS, Ext2FS, JFS, inodes, buffer cache, labeling, locking, metadata, soft-updates, LFS, portalfs, procfs, vnodes, memory sharing, memory objects, TLBs, caching Interprocess Communication Signals Signals, pipes, semaphores, message queues, shared memory, ports, sockets, doors Networking Sockets Sockets, bpf, IP, TCP, UDP, ICMP, OSI, bridging, firewalling, NAT, switching, etc + + &chap.ipv6; + Network Filesystems AFS AFS, NFS, SANs etc] Terminal Handling Syscons Syscons, tty, PCVT, serial console, screen savers, etc Sound OSS OSS, waveforms, etc Device Drivers &chap.driverbasics; &chap.isa; &chap.pci; &chap.scsi; &chap.usb; NewBus This chapter will talk about the FreeBSD NewBus architecture. Architectures &chap.x86; Alpha Talk about the architectural specifics of FreeBSD/alpha. Explanation of allignment errors, how to fix, how to ignore. Example assembly language code for FreeBSD/alpha. IA-64 Talk about the architectural specifics of FreeBSD/ia64. Debugging Truss various descriptions on how to debug certain aspects of the system using truss, ktrace, gdb, kgdb, etc Compatibility Layers Linux Linux, SVR4, etc Appendices Dave A Patterson John L Hennessy 1998Morgan Kaufmann Publishers, Inc. 1-55860-428-6 Morgan Kaufmann Publishers, Inc. Computer Organization and Design The Hardware / Software Interface 1-2 W. Richard Stevens 1993Addison Wesley Longman, Inc. 0-201-56317-7 Addison Wesley Longman, Inc. Advanced Programming in the Unix Environment 1-2 Marshall Kirk McKusick Keith Bostic Michael J Karels John S Quarterman 1996Addison-Wesley Publishing Company, Inc. 0-201-54979-4 Addison-Wesley Publishing Company, Inc. The Design and Implementation of the 4.4 BSD Operating System 1-2 Aleph One Phrack 49; "Smashing the Stack for Fun and Profit" Chrispin Cowan Calton Pu Dave Maier StackGuard; Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks Todd Miller Theo de Raadt strlcpy and strlcat -- consistent, safe string copy and concatenation. diff --git a/en_US.ISO8859-1/books/arch-handbook/chapters.ent b/en_US.ISO8859-1/books/arch-handbook/chapters.ent index 6e2d0c3f6f..044535d282 100644 --- a/en_US.ISO8859-1/books/arch-handbook/chapters.ent +++ b/en_US.ISO8859-1/books/arch-handbook/chapters.ent @@ -1,61 +1,61 @@ - - + + - + diff --git a/en_US.ISO8859-1/books/arch-handbook/vm/chapter.sgml b/en_US.ISO8859-1/books/arch-handbook/vm/chapter.sgml new file mode 100644 index 0000000000..4710973d5f --- /dev/null +++ b/en_US.ISO8859-1/books/arch-handbook/vm/chapter.sgml @@ -0,0 +1,255 @@ + + + + Virtual Memory System + + + The FreeBSD VM System + + Contributed by &a.dillon;. 6 Feb 1999 + + + Management of physical + memory—<literal>vm_page_t</literal> + + Physical memory is managed on a page-by-page basis through the + vm_page_t structure. Pages of physical memory are + categorized through the placement of their respective + vm_page_t structures on one of several paging + queues. + + A page can be in a wired, active, inactive, cache, or free state. + Except for the wired state, the page is typically placed in a doubly + link list queue representing the state that it is in. Wired pages + are not placed on any queue. + + FreeBSD implements a more involved paging queue for cached and + free pages in order to implement page coloring. Each of these states + involves multiple queues arranged according to the size of the + processor's L1 and L2 caches. When a new page needs to be allocated, + FreeBSD attempts to obtain one that is reasonably well aligned from + the point of view of the L1 and L2 caches relative to the VM object + the page is being allocated for. + + Additionally, a page may be held with a reference count or locked + with a busy count. The VM system also implements an ultimate + locked state for a page using the PG_BUSY bit in the page's + flags. + + In general terms, each of the paging queues operates in a LRU + fashion. A page is typically placed in a wired or active state + initially. When wired, the page is usually associated with a page + table somewhere. The VM system ages the page by scanning pages in a + more active paging queue (LRU) in order to move them to a less-active + paging queue. Pages that get moved into the cache are still + associated with a VM object but are candidates for immediate reuse. + Pages in the free queue are truly free. FreeBSD attempts to minimize + the number of pages in the free queue, but a certain minimum number of + truly free pages must be maintained in order to accommodate page + allocation at interrupt time. + + If a process attempts to access a page that does not exist in its + page table but does exist in one of the paging queues ( such as the + inactive or cache queues), a relatively inexpensive page reactivation + fault occurs which causes the page to be reactivated. If the page + does not exist in system memory at all, the process must block while + the page is brought in from disk. + + FreeBSD dynamically tunes its paging queues and attempts to + maintain reasonable ratios of pages in the various queues as well as + attempts to maintain a reasonable breakdown of clean v.s. dirty pages. + The amount of rebalancing that occurs depends on the system's memory + load. This rebalancing is implemented by the pageout daemon and + involves laundering dirty pages (syncing them with their backing + store), noticing when pages are activity referenced (resetting their + position in the LRU queues or moving them between queues), migrating + pages between queues when the queues are out of balance, and so forth. + FreeBSD's VM system is willing to take a reasonable number of + reactivation page faults to determine how active or how idle a page + actually is. This leads to better decisions being made as to when to + launder or swap-out a page. + + + + The unified buffer + cache—<literal>vm_object_t</literal> + + FreeBSD implements the idea of a generic VM object. + VM objects can be associated with backing store of various + types—unbacked, swap-backed, physical device-backed, or + file-backed storage. Since the filesystem uses the same VM objects to + manage in-core data relating to files, the result is a unified buffer + cache. + + VM objects can be shadowed. That is, they + can be stacked on top of each other. For example, you might have a + swap-backed VM object stacked on top of a file-backed VM object in + order to implement a MAP_PRIVATE mmap()ing. This stacking is also + used to implement various sharing properties, including, + copy-on-write, for forked address spaces. + + It should be noted that a vm_page_t can only be + associated with one VM object at a time. The VM object shadowing + implements the perceived sharing of the same page across multiple + instances. + + + + Filesystem I/O—<literal>struct buf</literal> + + vnode-backed VM objects, such as file-backed objects, generally + need to maintain their own clean/dirty info independent from the VM + system's idea of clean/dirty. For example, when the VM system decides + to synchronize a physical page to its backing store, the VM system + needs to mark the page clean before the page is actually written to + its backing s tore. Additionally, filesystems need to be able to map + portions of a file or file metadata into KVM in order to operate on + it. + + The entities used to manage this are known as filesystem buffers, + struct buf's, and also known as + bp's. When a filesystem needs to operate on a + portion of a VM object, it typically maps part of the object into a + struct buf and the maps the pages in the struct buf into KVM. In the + same manner, disk I/O is typically issued by mapping portions of + objects into buffer structures and then issuing the I/O on the buffer + structures. The underlying vm_page_t's are typically busied for the + duration of the I/O. Filesystem buffers also have their own notion of + being busy, which is useful to filesystem driver code which would + rather operate on filesystem buffers instead of hard VM pages. + + FreeBSD reserves a limited amount of KVM to hold mappings from + struct bufs, but it should be made clear that this KVM is used solely + to hold mappings and does not limit the ability to cache data. + Physical data caching is strictly a function of + vm_page_t's, not filesystem buffers. However, + since filesystem buffers are used placehold I/O, they do inherently + limit the amount of concurrent I/O possible. As there are usually a + few thousand filesystem buffers available, this is not usually a + problem. + + + + Mapping Page Tables - vm_map_t, vm_entry_t + + FreeBSD separates the physical page table topology from the VM + system. All hard per-process page tables can be reconstructed on the + fly and are usually considered throwaway. Special page tables such as + those managing KVM are typically permanently preallocated. These page + tables are not throwaway. + + FreeBSD associates portions of vm_objects with address ranges in + virtual memory through vm_map_t and + vm_entry_t structures. Page tables are directly + synthesized from the + vm_map_t/vm_entry_t/ + vm_object_t hierarchy. Remember when I mentioned + that physical pages are only directly associated with a + vm_object. Well, that isn't quite true. + vm_page_t's are also linked into page tables that + they are actively associated with. One vm_page_t + can be linked into several pmaps, as page tables + are called. However, the hierarchical association holds so all + references to the same page in the same object reference the same + vm_page_t and thus give us buffer cache unification + across the board. + + + + KVM Memory Mapping + + FreeBSD uses KVM to hold various kernel structures. The single + largest entity held in KVM is the filesystem buffer cache. That is, + mappings relating to struct buf entities. + + Unlike Linux, FreeBSD does NOT map all of physical memory into + KVM. This means that FreeBSD can handle memory configurations up to + 4G on 32 bit platforms. In fact, if the mmu were capable of it, + FreeBSD could theoretically handle memory configurations up to 8TB on + a 32 bit platform. However, since most 32 bit platforms are only + capable of mapping 4GB of ram, this is a moot point. + + KVM is managed through several mechanisms. The main mechanism + used to manage KVM is the zone allocator. The + zone allocator takes a chunk of KVM and splits it up into + constant-sized blocks of memory in order to allocate a specific type + of structure. You can use vmstat -m to get an + overview of current KVM utilization broken down by zone. + + + + Tuning the FreeBSD VM system + + A concerted effort has been made to make the FreeBSD kernel + dynamically tune itself. Typically you do not need to mess with + anything beyond the maxusers and + NMBCLUSTERS kernel config options. That is, kernel + compilation options specified in (typically) + /usr/src/sys/i386/conf/CONFIG_FILE. + A description of all available kernel configuration options can be + found in /usr/src/sys/i386/conf/LINT. + + In a large system configuration you may wish to increase + maxusers. Values typically range from 10 to 128. + Note that raising maxusers too high can cause the + system to overflow available KVM resulting in unpredictable operation. + It is better to leave maxusers at some reasonable number and add other + options, such as NMBCLUSTERS, to increase specific + resources. + + If your system is going to use the network heavily, you may want + to increase NMBCLUSTERS. Typical values range from + 1024 to 4096. + + The NBUF parameter is also traditionally used + to scale the system. This parameter determines the amount of KVA the + system can use to map filesystem buffers for I/O. Note that this + parameter has nothing whatsoever to do with the unified buffer cache! + This parameter is dynamically tuned in 3.0-CURRENT and later kernels + and should generally not be adjusted manually. We recommend that you + not try to specify an NBUF + parameter. Let the system pick it. Too small a value can result in + extremely inefficient filesystem operation while too large a value can + starve the page queues by causing too many pages to become wired + down. + + By default, FreeBSD kernels are not optimized. You can set + debugging and optimization flags with the + makeoptions directive in the kernel configuration. + Note that you should not use unless you can + accommodate the large (typically 7 MB+) kernels that result. + + makeoptions DEBUG="-g" +makeoptions COPTFLAGS="-O -pipe" + + Sysctl provides a way to tune kernel parameters at run-time. You + typically do not need to mess with any of the sysctl variables, + especially the VM related ones. + + Run time VM and system tuning is relatively straightforward. + First, use softupdates on your UFS/FFS filesystems whenever possible. + /usr/src/contrib/sys/softupdates/README contains + instructions (and restrictions) on how to configure it up. + + Second, configure sufficient swap. You should have a swap + partition configured on each physical disk, up to four, even on your + work disks. You should have at least 2x the swap space + as you have main memory, and possibly even more if you do not have a + lot of memory. You should also size your swap partition based on the + maximum memory configuration you ever intend to put on the machine so + you do not have to repartition your disks later on. If you want to be + able to accommodate a crash dump, your first swap partition must be at + least as large as main memory and /var/crash must + have sufficient free space to hold the dump. + + NFS-based swap is perfectly acceptable on -4.x or later systems, + but you must be aware that the NFS server will take the brunt of the + paging load. + + + + diff --git a/en_US.ISO8859-1/books/developers-handbook/Makefile b/en_US.ISO8859-1/books/developers-handbook/Makefile index 371551ab9e..acf0c62b67 100644 --- a/en_US.ISO8859-1/books/developers-handbook/Makefile +++ b/en_US.ISO8859-1/books/developers-handbook/Makefile @@ -1,35 +1,38 @@ # -# $FreeBSD: doc/en_US.ISO_8859-1/books/developers-handbook/Makefile,v 1.2 2001/05/02 01:53:13 murray Exp $ +# $FreeBSD: doc/en_US.ISO_8859-1/books/developers-handbook/Makefile,v 1.3 2001/05/11 10:27:00 murray Exp $ # # Build the FreeBSD Developers' Handbook. # MAINTAINER=asmodai@FreeBSD.org DOC?= book FORMATS?= html-split INSTALL_COMPRESSED?= gz INSTALL_ONLY_COMPRESSED?= # # SRCS lists the individual SGML files that make up the document. Changes # to any of these files will force a rebuild # # SGML content SRCS= book.sgml SRCS+= tools/chapter.sgml SRCS+= secure/chapter.sgml SRCS+= locking/chapter.sgml SRCS+= isa/chapter.sgml SRCS+= pci/chapter.sgml SRCS+= usb/chapter.sgml SRCS+= scsi/chapter.sgml SRCS+= x86/chapter.sgml +SRCS+= vm/chapter.sgml +SRCS+= dma/chapter.sgml +SRCS+= ipv6/chapter.sgml # Entities DOC_PREFIX?= ${.CURDIR}/../../.. .include "${DOC_PREFIX}/share/mk/doc.project.mk" diff --git a/en_US.ISO8859-1/books/developers-handbook/book.sgml b/en_US.ISO8859-1/books/developers-handbook/book.sgml index a1a485a73d..10879f994e 100644 --- a/en_US.ISO8859-1/books/developers-handbook/book.sgml +++ b/en_US.ISO8859-1/books/developers-handbook/book.sgml @@ -1,517 +1,518 @@ %bookinfo; %man; %chapters; %authors; %mailing-lists; ]> FreeBSD Developers' Handbook The FreeBSD Documentation Project August 2000 2000 2001 The FreeBSD Documentation Project &bookinfo.legalnotice; Welcome to the Developers' Handbook. This manual is a work in progress and is the work of many individuals. Many sections do not yet exist and some of those that do exist need to be updated. If you are interested in helping with this project, send email to the &a.doc;. The latest version of this document is always available from the FreeBSD World Wide Web server. It may also be downloaded in a variety of formats and compression options from the FreeBSD FTP server or one of the numerous mirror sites. Introduction Developing on FreeBSD This will need to discuss FreeBSD as a development platform, the vision of BSD, architectural overview, layout of /usr/src, history, etc. Thank you for considering FreeBSD as your development platform! We hope it will not let you down. The BSD Vision Architectural Overview The Layout of /usr/src The complete source code to FreeBSD is available from our public CVS repository. The source code is normally installed in /usr/src which contains the following subdirectories. Directory Description bin/ Source for files in /bin contrib/ Source for files from contributed software. crypto/ DES source etc/ Source for files in /etc games/ Source for files in /usr/games gnu/ Utilities covered by the GNU Public License include/ Source for files in /usr/include kerberosIV/ Source for Kerbereros version IV kerberos5/ Source for Kerbereros version 5 lib/ Source for files in /usr/lib libexec/ Source for files in /usr/libexec release/ Files required to produce a FreeBSD release sbin/ Source for files in /sbin secure/ FreeSec sources share/ Source for files in /sbin sys/ Kernel source files tools/ Tools used for maintenance and testing of FreeBSD usr.bin/ Source for files in /usr/bin usr.sbin/ Source for files in /usr/sbin Basics &chap.tools; &chap.secure; Kernel History of the Unix Kernel Some history of the Unix/BSD kernel, system calls, how do processes work, blocking, scheduling, threads (kernel), context switching, signals, interrupts, modules, etc. &chap.locking; - Memory and Virtual Memory + Memory Management - - Virtual Memory + &chap.vm; + &chap.dma; - VM, paging, swapping, allocating memory, testing for - memory leaks, mmap, vnodes, etc. + - - I/O System UFS UFS, FFS, Ext2FS, JFS, inodes, buffer cache, labeling, locking, metadata, soft-updates, LFS, portalfs, procfs, vnodes, memory sharing, memory objects, TLBs, caching Interprocess Communication Signals Signals, pipes, semaphores, message queues, shared memory, ports, sockets, doors Networking Sockets Sockets, bpf, IP, TCP, UDP, ICMP, OSI, bridging, firewalling, NAT, switching, etc + + &chap.ipv6; + Network Filesystems AFS AFS, NFS, SANs etc] Terminal Handling Syscons Syscons, tty, PCVT, serial console, screen savers, etc Sound OSS OSS, waveforms, etc Device Drivers &chap.driverbasics; &chap.isa; &chap.pci; &chap.scsi; &chap.usb; NewBus This chapter will talk about the FreeBSD NewBus architecture. Architectures &chap.x86; Alpha Talk about the architectural specifics of FreeBSD/alpha. Explanation of allignment errors, how to fix, how to ignore. Example assembly language code for FreeBSD/alpha. IA-64 Talk about the architectural specifics of FreeBSD/ia64. Debugging Truss various descriptions on how to debug certain aspects of the system using truss, ktrace, gdb, kgdb, etc Compatibility Layers Linux Linux, SVR4, etc Appendices Dave A Patterson John L Hennessy 1998Morgan Kaufmann Publishers, Inc. 1-55860-428-6 Morgan Kaufmann Publishers, Inc. Computer Organization and Design The Hardware / Software Interface 1-2 W. Richard Stevens 1993Addison Wesley Longman, Inc. 0-201-56317-7 Addison Wesley Longman, Inc. Advanced Programming in the Unix Environment 1-2 Marshall Kirk McKusick Keith Bostic Michael J Karels John S Quarterman 1996Addison-Wesley Publishing Company, Inc. 0-201-54979-4 Addison-Wesley Publishing Company, Inc. The Design and Implementation of the 4.4 BSD Operating System 1-2 Aleph One Phrack 49; "Smashing the Stack for Fun and Profit" Chrispin Cowan Calton Pu Dave Maier StackGuard; Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks Todd Miller Theo de Raadt strlcpy and strlcat -- consistent, safe string copy and concatenation. diff --git a/en_US.ISO8859-1/books/developers-handbook/chapters.ent b/en_US.ISO8859-1/books/developers-handbook/chapters.ent index 6e2d0c3f6f..044535d282 100644 --- a/en_US.ISO8859-1/books/developers-handbook/chapters.ent +++ b/en_US.ISO8859-1/books/developers-handbook/chapters.ent @@ -1,61 +1,61 @@ - - + + - + diff --git a/en_US.ISO8859-1/books/developers-handbook/dma/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/dma/chapter.sgml new file mode 100644 index 0000000000..7bbb7d973d --- /dev/null +++ b/en_US.ISO8859-1/books/developers-handbook/dma/chapter.sgml @@ -0,0 +1,1326 @@ + + + + DMA + + + DMA: What it is and How it Works + + Copyright © 1995,1997 &a.uhclem;, All Rights + Reserved. 10 December 1996. Last Update 8 October + 1997. + + Direct Memory Access (DMA) is a method of allowing data to be moved + from one location to another in a computer without intervention from the + central processor (CPU). + + The way that the DMA function is implemented varies between computer + architectures, so this discussion will limit itself to the + implementation and workings of the DMA subsystem on the IBM Personal + Computer (PC), the IBM PC/AT and all of its successors and + clones. + + The PC DMA subsystem is based on the Intel 8237 DMA controller. The + 8237 contains four DMA channels that can be programmed independently and + any one of the channels may be active at any moment. These channels are + numbered 0, 1, 2 and 3. Starting with the PC/AT, IBM added a second + 8237 chip, and numbered those channels 4, 5, 6 and 7. + + The original DMA controller (0, 1, 2 and 3) moves one byte in each + transfer. The second DMA controller (4, 5, 6, and 7) moves 16-bits from + two adjacent memory locations in each transfer, with the first byte + always coming from an even-numbered address. The two controllers are + identical components and the difference in transfer size is caused by + the way the second controller is wired into the system. + + The 8237 has two electrical signals for each channel, named DRQ and + -DACK. There are additional signals with the names HRQ (Hold Request), + HLDA (Hold Acknowledge), -EOP (End of Process), and the bus control + signals -MEMR (Memory Read), -MEMW (Memory Write), -IOR (I/O Read), and + -IOW (I/O Write). + + The 8237 DMA is known as a fly-by DMA controller. + This means that the data being moved from one location to another does + not pass through the DMA chip and is not stored in the DMA chip. + Subsequently, the DMA can only transfer data between an I/O port and a + memory address, but not between two I/O ports or two memory + locations. + + + The 8237 does allow two channels to be connected together to allow + memory-to-memory DMA operations in a non-fly-by mode, + but nobody in the PC industry uses this scarce resource this way since + it is faster to move data between memory locations using the + CPU. + + + In the PC architecture, each DMA channel is normally activated only + when the hardware that uses a given DMA channel requests a transfer by + asserting the DRQ line for that channel. + + + A Sample DMA transfer + + Here is an example of the steps that occur to cause and perform a + DMA transfer. In this example, the floppy disk controller (FDC) has + just read a byte from a diskette and wants the DMA to place it in + memory at location 0x00123456. The process begins by the FDC + asserting the DRQ2 signal (the DRQ line for DMA channel 2) to alert + the DMA controller. + + The DMA controller will note that the DRQ2 signal is asserted. The + DMA controller will then make sure that DMA channel 2 has been + programmed and is unmasked (enabled). The DMA controller also makes + sure that none of the other DMA channels are active or want to be + active and have a higher priority. Once these checks are complete, + the DMA asks the CPU to release the bus so that the DMA may use the + bus. The DMA requests the bus by asserting the HRQ signal which goes + to the CPU. + + The CPU detects the HRQ signal, and will complete executing the + current instruction. Once the processor has reached a state where it + can release the bus, it will. Now all of the signals normally + generated by the CPU (-MEMR, -MEMW, -IOR, -IOW and a few others) are + placed in a tri-stated condition (neither high or low) and then the + CPU asserts the HLDA signal which tells the DMA controller that it is + now in charge of the bus. + + Depending on the processor, the CPU may be able to execute a few + additional instructions now that it no longer has the bus, but the CPU + will eventually have to wait when it reaches an instruction that must + read something from memory that is not in the internal processor cache + or pipeline. + + Now that the DMA is in charge, the DMA activates its + -MEMR, -MEMW, -IOR, -IOW output signals, and the address outputs from + the DMA are set to 0x3456, which will be used to direct the byte that + is about to transferred to a specific memory location. + + The DMA will then let the device that requested the DMA transfer + know that the transfer is commencing. This is done by asserting the + -DACK signal, or in the case of the floppy disk controller, -DACK2 is + asserted. + + The floppy disk controller is now responsible for placing the byte + to be transferred on the bus Data lines. Unless the floppy controller + needs more time to get the data byte on the bus (and if the peripheral + does need more time it alerts the DMA via the READY signal), the DMA + will wait one DMA clock, and then de-assert the -MEMW and -IOR signals + so that the memory will latch and store the byte that was on the bus, + and the FDC will know that the byte has been transferred. + + Since the DMA cycle only transfers a single byte at a time, the + FDC now drops the DRQ2 signal, so the DMA knows that it is no longer + needed. The DMA will de-assert the -DACK2 signal, so that the FDC + knows it must stop placing data on the bus. + + The DMA will now check to see if any of the other DMA channels + have any work to do. If none of the channels have their DRQ lines + asserted, the DMA controller has completed its work and will now + tri-state the -MEMR, -MEMW, -IOR, -IOW and address signals. + + Finally, the DMA will de-assert the HRQ signal. The CPU sees + this, and de-asserts the HOLDA signal. Now the CPU activates its + -MEMR, -MEMW, -IOR, -IOW and address lines, and it resumes executing + instructions and accessing main memory and the peripherals. + + For a typical floppy disk sector, the above process is repeated + 512 times, once for each byte. Each time a byte is transferred, the + address register in the DMA is incremented and the counter in the DMA + that shows how many bytes are to be transferred is decremented. + + When the counter reaches zero, the DMA asserts the EOP signal, + which indicates that the counter has reached zero and no more data + will be transferred until the DMA controller is reprogrammed by the + CPU. This event is also called the Terminal Count (TC). There is only + one EOP signal, and since only DMA channel can be active at any + instant, the DMA channel that is currently active must be the DMA + channel that just completed its task. + + If a peripheral wants to generate an interrupt when the transfer + of a buffer is complete, it can test for its -DACKn signal and the EOP + signal both being asserted at the same time. When that happens, it + means the DMA will not transfer any more information for that + peripheral without intervention by the CPU. The peripheral can then + assert one of the interrupt signals to get the processors' attention. + In the PC architecture, the DMA chip itself is not capable of + generating an interrupt. The peripheral and its associated hardware + is responsible for generating any interrupt that occurs. + Subsequently, it is possible to have a peripheral that uses DMA but + does not use interrupts. + + It is important to understand that although the CPU always + releases the bus to the DMA when the DMA makes the request, this + action is invisible to both applications and the operating systems, + except for slight changes in the amount of time the processor takes to + execute instructions when the DMA is active. Subsequently, the + processor must poll the peripheral, poll the registers in the DMA + chip, or receive an interrupt from the peripheral to know for certain + when a DMA transfer has completed. + + + + DMA Page Registers and 16Meg address space limitations + + You may have noticed earlier that instead of the DMA setting the + address lines to 0x00123456 as we said earlier, the DMA only set + 0x3456. The reason for this takes a bit of explaining. + + When the original IBM PC was designed, IBM elected to use both DMA + and interrupt controller chips that were designed for use with the + 8085, an 8-bit processor with an address space of 16 bits (64K). + Since the IBM PC supported more than 64K of memory, something had to + be done to allow the DMA to read or write memory locations above the + 64K mark. What IBM did to solve this problem was to add an external + data latch for each DMA channel that holds the upper bits of the + address to be read to or written from. Whenever a DMA channel is + active, the contents of that latch are written to the address bus and + kept there until the DMA operation for the channel ends. IBM called + these latches Page Registers. + + So for our example above, the DMA would put the 0x3456 part of the + address on the bus, and the Page Register for DMA channel 2 would put + 0x0012xxxx on the bus. Together, these two values form the complete + address in memory that is to be accessed. + + Because the Page Register latch is independent of the DMA chip, + the area of memory to be read or written must not span a 64K physical + boundary. For example, if the DMA accesses memory location 0xffff, + after that transfer the DMA will then increment the address register + and the DMA will access the next byte at location 0x0000, not 0x10000. + The results of letting this happen are probably not intended. + + + Physical 64K boundaries should not be confused + with 8086-mode 64K Segments, which are created by + mathematically adding a segment register with an offset register. + Page Registers have no address overlap and are mathematically OR-ed + together. + + + To further complicate matters, the external DMA address latches on + the PC/AT hold only eight bits, so that gives us 8+16=24 bits, which + means that the DMA can only point at memory locations between 0 and + 16Meg. For newer computers that allow more than 16Meg of memory, the + standard PC-compatible DMA cannot access memory locations above + 16Meg. + + To get around this restriction, operating systems will reserve a + RAM buffer in an area below 16Meg that also does not span a physical + 64K boundary. Then the DMA will be programmed to transfer data from + the peripheral and into that buffer. Once the DMA has moved the data + into this buffer, the operating system will then copy the data from + the buffer to the address where the data is really supposed to be + stored. + + When writing data from an address above 16Meg to a DMA-based + peripheral, the data must be first copied from where it resides into a + buffer located below 16Meg, and then the DMA can copy the data from + the buffer to the hardware. In FreeBSD, these reserved buffers are + called Bounce Buffers. In the MS-DOS world, they are + sometimes called Smart Buffers. + + + A new implementation of the 8237, called the 82374, allows 16 + bits of page register to be specified, allows access to the entire + 32 bit address space, without the use of bounce buffers. + + + + + DMA Operational Modes and Settings + + The 8237 DMA can be operated in several modes. The main ones + are: + + + + Single + + + A single byte (or word) is transferred. The DMA must + release and re-acquire the bus for each additional byte. This is + commonly-used by devices that cannot transfer the entire block + of data immediately. The peripheral will request the DMA each + time it is ready for another transfer. + + The standard PC-compatible floppy disk controller (NEC 765) + only has a one-byte buffer, so it uses this mode. + + + + + Block/Demand + + + Once the DMA acquires the system bus, an entire block of + data is transferred, up to a maximum of 64K. If the peripheral + needs additional time, it can assert the READY signal to suspend + the transfer briefly. READY should not be used excessively, and + for slow peripheral transfers, the Single Transfer Mode should + be used instead. + + The difference between Block and Demand is that once a Block + transfer is started, it runs until the transfer count reaches + zero. DRQ only needs to be asserted until -DACK is asserted. + Demand Mode will transfer one more bytes until DRQ is + de-asserted, at which point the DMA suspends the transfer and + releases the bus back to the CPU. When DRQ is asserted later, + the transfer resumes where it was suspended. + + Older hard disk controllers used Demand Mode until CPU + speeds increased to the point that it was more efficient to + transfer the data using the CPU, particularly if the memory + locations used in the transfer were above the 16Meg mark. + + + + + Cascade + + + This mechanism allows a DMA channel to request the bus, but + then the attached peripheral device is responsible for placing + the addressing information on the bus instead of the DMA. This + is also used to implement a technique known as Bus + Mastering. + + When a DMA channel in Cascade Mode receives control of the + bus, the DMA does not place addresses and I/O control signals on + the bus like the DMA normally does when it is active. Instead, + the DMA only asserts the -DACK signal for the active DMA + channel. + + At this point it is up to the peripheral connected to that + DMA channel to provide address and bus control signals. The + peripheral has complete control over the system bus, and can do + reads and/or writes to any address below 16Meg. When the + peripheral is finished with the bus, it de-asserts the DRQ line, + and the DMA controller can then return control to the CPU or to + some other DMA channel. + + Cascade Mode can be used to chain multiple DMA controllers + together, and this is exactly what DMA Channel 4 is used for in + the PC architecture. When a peripheral requests the bus on DMA + channels 0, 1, 2 or 3, the slave DMA controller asserts HLDREQ, + but this wire is actually connected to DRQ4 on the primary DMA + controller instead of to the CPU. The primary DMA controller, + thinking it has work to do on Channel 4, requests the bus from + the CPU using HLDREQ signal. Once the CPU grants the bus to the + primary DMA controller, -DACK4 is asserted, and that wire is + actually connected to the HLDA signal on the slave DMA + controller. The slave DMA controller then transfers data for + the DMA channel that requested it (0, 1, 2 or 3), or the slave + DMA may grant the bus to a peripheral that wants to perform its + own bus-mastering, such as a SCSI controller. + + Because of this wiring arrangement, only DMA channels 0, 1, + 2, 3, 5, 6 and 7 are usable with peripherals on PC/AT + systems. + + + DMA channel 0 was reserved for refresh operations in early + IBM PC computers, but is generally available for use by + peripherals in modern systems. + + + When a peripheral is performing Bus Mastering, it is + important that the peripheral transmit data to or from memory + constantly while it holds the system bus. If the peripheral + cannot do this, it must release the bus frequently so that the + system can perform refresh operations on main memory. + + The Dynamic RAM used in all PCs for main memory must be + accessed frequently to keep the bits stored in the components + charged. Dynamic RAM essentially consists of + millions of capacitors with each one holding one bit of data. + These capacitors are charged with power to represent a + 1 or drained to represent a + 0. Because all capacitors leak, power must + be added at regular intervals to keep the 1 + values intact. The RAM chips actually handle the task of + pumping power back into all of the appropriate locations in RAM, + but they must be told when to do it by the rest of the computer + so that the refresh activity won't interfere with the computer + wanting to access RAM normally. If the computer is unable to + refresh memory, the contents of memory will become corrupted in + just a few milliseconds. + + Since memory read and write cycles count as + refresh cycles (a dynamic RAM refresh cycle is actually an + incomplete memory read cycle), as long as the peripheral + controller continues reading or writing data to sequential + memory locations, that action will refresh all of memory. + + Bus-mastering is found in some SCSI host interfaces and + other high-performance peripheral controllers. + + + + + Autoinitialize + + + This mode causes the DMA to perform Byte, Block or Demand + transfers, but when the DMA transfer counter reaches zero, the + counter and address are set back to where they were when the DMA + channel was originally programmed. This means that as long as + the peripheral requests transfers, they will be granted. It is + up to the CPU to move new data into the fixed buffer ahead of + where the DMA is about to transfer it when doing output + operations, and read new data out of the buffer behind where the + DMA is writing when doing input operations. + + This technique is frequently used on audio devices that have + small or no hardware sample buffers. There is + additional CPU overhead to manage this circular + buffer, but in some cases this may be the only way to eliminate + the latency that occurs when the DMA counter reaches zero and + the DMA stops transfers until it is reprogrammed. + + + + + + + Programming the DMA + + The DMA channel that is to be programmed should always be + masked before loading any settings. This is because the + hardware might unexpectedly assert the DRQ for that channel, and the + DMA might respond, even though not all of the parameters have been + loaded or updated. + + Once masked, the host must specify the direction of the transfer + (memory-to-I/O or I/O-to-memory), what mode of DMA operation is to be + used for the transfer (Single, Block, Demand, Cascade, etc), and + finally the address and length of the transfer are loaded. The length + that is loaded is one less than the amount you expect the DMA to + transfer. The LSB and MSB of the address and length are written to + the same 8-bit I/O port, so another port must be written to first to + guarantee that the DMA accepts the first byte as the LSB and the + second byte as the MSB of the length and address. + + Then, be sure to update the Page Register, which is external to + the DMA and is accessed through a different set of I/O ports. + + Once all the settings are ready, the DMA channel can be un-masked. + That DMA channel is now considered to be armed, and will + respond when the DRQ line for that channel is asserted. + + Refer to a hardware data book for precise programming details for + the 8237. You will also need to refer to the I/O port map for the PC + system, which describes where the DMA and Page Register ports are + located. A complete port map table is located below. + + + + DMA Port Map + + All systems based on the IBM-PC and PC/AT have the DMA hardware + located at the same I/O ports. The complete list is provided below. + Ports assigned to DMA Controller #2 are undefined on non-AT + designs. + + + 0x00–0x1f DMA Controller #1 (Channels 0, 1, 2 and + 3) + + DMA Address and Count Registers + + + + + + 0x00 + write + Channel 0 starting address + + + + 0x00 + read + Channel 0 current address + + + + 0x01 + write + Channel 0 starting word count + + + + 0x01 + read + Channel 0 remaining word count + + + + 0x02 + write + Channel 1 starting address + + + + 0x02 + read + Channel 1 current address + + + + 0x03 + write + Channel 1 starting word count + + + + 0x03 + read + Channel 1 remaining word count + + + + 0x04 + write + Channel 2 starting address + + + + 0x04 + read + Channel 2 current address + + + + 0x05 + write + Channel 2 starting word count + + + + 0x05 + read + Channel 2 remaining word count + + + + 0x06 + write + Channel 3 starting address + + + + 0x06 + read + Channel 3 current address + + + + 0x07 + write + Channel 3 starting word count + + + + 0x07 + read + Channel 3 remaining word count + + + + + + DMA Command Registers + + + + + + 0x08 + write + Command Register + + + + 0x08 + read + Status Register + + + + 0x09 + write + Request Register + + + + 0x09 + read + - + + + + 0x0a + write + Single Mask Register Bit + + + + 0x0a + read + - + + + + 0x0b + write + Mode Register + + + + 0x0b + read + - + + + + 0x0c + write + Clear LSB/MSB Flip-Flop + + + + 0x0c + read + - + + + + 0x0d + write + Master Clear/Reset + + + + 0x0d + read + Temporary Register (not available on newer + versions) + + + 0x0e + write + Clear Mask Register + + + + 0x0e + read + - + + + + 0x0f + write + Write All Mask Register Bits + + + + 0x0f + read + Read All Mask Register Bits (only in Intel + 82374) + + + + + + + + 0xc0–0xdf DMA Controller #2 (Channels 4, 5, 6 and + 7) + + DMA Address and Count Registers + + + + + + 0xc0 + write + Channel 4 starting address + + + + 0xc0 + read + Channel 4 current address + + + + 0xc2 + write + Channel 4 starting word count + + + + 0xc2 + read + Channel 4 remaining word count + + + + 0xc4 + write + Channel 5 starting address + + + + 0xc4 + read + Channel 5 current address + + + + 0xc6 + write + Channel 5 starting word count + + + + 0xc6 + read + Channel 5 remaining word count + + + + 0xc8 + write + Channel 6 starting address + + + + 0xc8 + read + Channel 6 current address + + + + 0xca + write + Channel 6 starting word count + + + + 0xca + read + Channel 6 remaining word count + + + + 0xcc + write + Channel 7 starting address + + + + 0xcc + read + Channel 7 current address + + + + 0xce + write + Channel 7 starting word count + + + + 0xce + read + Channel 7 remaining word count + + + + + + DMA Command Registers + + + + + + 0xd0 + write + Command Register + + + + 0xd0 + read + Status Register + + + + 0xd2 + write + Request Register + + + + 0xd2 + read + - + + + + 0xd4 + write + Single Mask Register Bit + + + + 0xd4 + read + - + + + + 0xd6 + write + Mode Register + + + + 0xd6 + read + - + + + + 0xd8 + write + Clear LSB/MSB Flip-Flop + + + + 0xd8 + read + - + + + + + + 0xda + write + Master Clear/Reset + + + + 0xda + read + Temporary Register (not present in Intel + 82374) + + + + 0xdc + write + Clear Mask Register + + + + 0xdc + read + - + + + + 0xde + write + Write All Mask Register Bits + + + + 0xdf + read + Read All Mask Register Bits (only in Intel + 82374) + + + + + + + + 0x80–0x9f DMA Page Registers + + + + + + 0x87 + r/w + Channel 0 Low byte (23-16) page Register + + + + 0x83 + r/w + Channel 1 Low byte (23-16) page Register + + + + 0x81 + r/w + Channel 2 Low byte (23-16) page Register + + + + 0x82 + r/w + Channel 3 Low byte (23-16) page Register + + + + 0x8b + r/w + Channel 5 Low byte (23-16) page Register + + + + 0x89 + r/w + Channel 6 Low byte (23-16) page Register + + + + 0x8a + r/w + Channel 7 Low byte (23-16) page Register + + + + 0x8f + r/w + Low byte page Refresh + + + + + + + + 0x400–0x4ff 82374 Enhanced DMA Registers + + The Intel 82374 EISA System Component (ESC) was introduced in + early 1996 and includes a DMA controller that provides a superset of + 8237 functionality as well as other PC-compatible core peripheral + components in a single package. This chip is targeted at both EISA + and PCI platforms, and provides modern DMA features like + scatter-gather, ring buffers as well as direct access by the system + DMA to all 32 bits of address space. + + If these features are used, code should also be included to + provide similar functionality in the previous 16 years worth of + PC-compatible computers. For compatibility reasons, some of the + 82374 registers must be programmed after + programming the traditional 8237 registers for each transfer. + Writing to a traditional 8237 register forces the contents of some + of the 82374 enhanced registers to zero to provide backward software + compatibility. + + + + + + 0x401 + r/w + Channel 0 High byte (bits 23-16) word count + + + + 0x403 + r/w + Channel 1 High byte (bits 23-16) word count + + + + 0x405 + r/w + Channel 2 High byte (bits 23-16) word count + + + + 0x407 + r/w + Channel 3 High byte (bits 23-16) word count + + + + 0x4c6 + r/w + Channel 5 High byte (bits 23-16) word count + + + + 0x4ca + r/w + Channel 6 High byte (bits 23-16) word count + + + + 0x4ce + r/w + Channel 7 High byte (bits 23-16) word count + + + + 0x487 + r/w + Channel 0 High byte (bits 31-24) page Register + + + + 0x483 + r/w + Channel 1 High byte (bits 31-24) page Register + + + + 0x481 + r/w + Channel 2 High byte (bits 31-24) page Register + + + + 0x482 + r/w + Channel 3 High byte (bits 31-24) page Register + + + + 0x48b + r/w + Channel 5 High byte (bits 31-24) page Register + + + + 0x489 + r/w + Channel 6 High byte (bits 31-24) page Register + + + + 0x48a + r/w + Channel 6 High byte (bits 31-24) page Register + + + + 0x48f + r/w + High byte page Refresh + + + + 0x4e0 + r/w + Channel 0 Stop Register (bits 7-2) + + + + 0x4e1 + r/w + Channel 0 Stop Register (bits 15-8) + + + + 0x4e2 + r/w + Channel 0 Stop Register (bits 23-16) + + + + 0x4e4 + r/w + Channel 1 Stop Register (bits 7-2) + + + + 0x4e5 + r/w + Channel 1 Stop Register (bits 15-8) + + + + 0x4e6 + r/w + Channel 1 Stop Register (bits 23-16) + + + + 0x4e8 + r/w + Channel 2 Stop Register (bits 7-2) + + + + 0x4e9 + r/w + Channel 2 Stop Register (bits 15-8) + + + + 0x4ea + r/w + Channel 2 Stop Register (bits 23-16) + + + + 0x4ec + r/w + Channel 3 Stop Register (bits 7-2) + + + + 0x4ed + r/w + Channel 3 Stop Register (bits 15-8) + + + + 0x4ee + r/w + Channel 3 Stop Register (bits 23-16) + + + + 0x4f4 + r/w + Channel 5 Stop Register (bits 7-2) + + + + 0x4f5 + r/w + Channel 5 Stop Register (bits 15-8) + + + + 0x4f6 + r/w + Channel 5 Stop Register (bits 23-16) + + + + 0x4f8 + r/w + Channel 6 Stop Register (bits 7-2) + + + + 0x4f9 + r/w + Channel 6 Stop Register (bits 15-8) + + + + 0x4fa + r/w + Channel 6 Stop Register (bits 23-16) + + + + 0x4fc + r/w + Channel 7 Stop Register (bits 7-2) + + + + 0x4fd + r/w + Channel 7 Stop Register (bits 15-8) + + + + 0x4fe + r/w + Channel 7 Stop Register (bits 23-16) + + + + 0x40a + write + Channels 0-3 Chaining Mode Register + + + + 0x40a + read + Channel Interrupt Status Register + + + + 0x4d4 + write + Channels 4-7 Chaining Mode Register + + + + 0x4d4 + read + Chaining Mode Status + + + + 0x40c + read + Chain Buffer Expiration Control Register + + + + 0x410 + write + Channel 0 Scatter-Gather Command Register + + + + 0x411 + write + Channel 1 Scatter-Gather Command Register + + + + 0x412 + write + Channel 2 Scatter-Gather Command Register + + + + 0x413 + write + Channel 3 Scatter-Gather Command Register + + + + 0x415 + write + Channel 5 Scatter-Gather Command Register + + + + 0x416 + write + Channel 6 Scatter-Gather Command Register + + + + 0x417 + write + Channel 7 Scatter-Gather Command Register + + + + 0x418 + read + Channel 0 Scatter-Gather Status Register + + + + 0x419 + read + Channel 1 Scatter-Gather Status Register + + + + 0x41a + read + Channel 2 Scatter-Gather Status Register + + + + 0x41b + read + Channel 3 Scatter-Gather Status Register + + + + 0x41d + read + Channel 5 Scatter-Gather Status Register + + + + 0x41e + read + Channel 5 Scatter-Gather Status Register + + + + 0x41f + read + Channel 7 Scatter-Gather Status Register + + + + 0x420-0x423 + r/w + Channel 0 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x424-0x427 + r/w + Channel 1 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x428-0x42b + r/w + Channel 2 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x42c-0x42f + r/w + Channel 3 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x434-0x437 + r/w + Channel 5 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x438-0x43b + r/w + Channel 6 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x43c-0x43f + r/w + Channel 7 Scatter-Gather Descriptor Table Pointer + Register + + + + + + + + + \ No newline at end of file diff --git a/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.sgml new file mode 100644 index 0000000000..e6028b6cb3 --- /dev/null +++ b/en_US.ISO8859-1/books/developers-handbook/ipv6/chapter.sgml @@ -0,0 +1,1603 @@ + + + + IPv6 Internals + + + IPv6/IPsec Implementation + + Contributed by &a.shin;, 5 March + 2000. + + This section should explain IPv6 and IPsec related implementation + internals. These functionalities are derived from KAME project + + + IPv6 + + + Conformance + + The IPv6 related functions conforms, or tries to conform to + the latest set of IPv6 specifications. For future reference we list + some of the relevant documents below (NOTE: this + is not a complete list - this is too hard to maintain...). + + For details please refer to specific chapter in the document, + RFCs, manpages, or comments in the source code. + + Conformance tests have been performed on the KAME STABLE kit + at TAHI project. Results can be viewed at http://www.tahi.org/report/KAME/ + . We also attended Univ. of New Hampshire IOL tests (http://www.iol.unh.edu/) in the + past, with our past snapshots. + + + + RFC1639: FTP Operation Over Big Address Records + (FOOBAR) + + + RFC2428 is preferred over RFC1639. FTP clients will + first try RFC2428, then RFC1639 if failed. + + + + + + RFC1886: DNS Extensions to support IPv6 + + + + RFC1933: Transition Mechanisms for IPv6 Hosts and + Routers + + + IPv4 compatible address is not supported. + + + automatic tunneling (described in 4.3 of this RFC) is not + supported. + + + &man.gif.4; interface implements IPv[46]-over-IPv[46] + tunnel in a generic way, and it covers "configured tunnel" + described in the spec. See 23.5.1.5 + in this document for details. + + + + + + RFC1981: Path MTU Discovery for IPv6 + + + + RFC2080: RIPng for IPv6 + + + usr.sbin/route6d support this. + + + + + + RFC2292: Advanced Sockets API for IPv6 + + + For supported library functions/kernel APIs, see + sys/netinet6/ADVAPI. + + + + + + RFC2362: Protocol Independent Multicast-Sparse + Mode (PIM-SM) + + + RFC2362 defines packet formats for PIM-SM. + draft-ietf-pim-ipv6-01.txt is + written based on this. + + + + + + RFC2373: IPv6 Addressing Architecture + + + supports node required addresses, and conforms to + the scope requirement. + + + + + + RFC2374: An IPv6 Aggregatable Global Unicast Address + Format + + + supports 64-bit length of Interface ID. + + + + + + RFC2375: IPv6 Multicast Address Assignments + + + Userland applications use the well-known addresses + assigned in the RFC. + + + + + + RFC2428: FTP Extensions for IPv6 and NATs + + + RFC2428 is preferred over RFC1639. FTP clients will + first try RFC2428, then RFC1639 if failed. + + + + + + RFC2460: IPv6 specification + + + + RFC2461: Neighbor discovery for IPv6 + + + See 23.5.1.2 + in this document for details. + + + + + + RFC2462: IPv6 Stateless Address Autoconfiguration + + + See 23.5.1.4 in this + document for details. + + + + + + RFC2463: ICMPv6 for IPv6 specification + + + See 23.5.1.9 in this + document for details. + + + + + + RFC2464: Transmission of IPv6 Packets over Ethernet + Networks + + + + RFC2465: MIB for IPv6: Textual Conventions and General + Group + + + Necessary statistics are gathered by the kernel. Actual + IPv6 MIB support is provided as a patchkit for ucd-snmp. + + + + + + RFC2466: MIB for IPv6: ICMPv6 group + + + Necessary statistics are gathered by the kernel. Actual + IPv6 MIB support is provided as patchkit for ucd-snmp. + + + + + + RFC2467: Transmission of IPv6 Packets over FDDI + Networks + + + + RFC2497: Transmission of IPv6 packet over ARCnet + Networks + + + + RFC2553: Basic Socket Interface Extensions for IPv6 + + + IPv4 mapped address (3.7) and special behavior of IPv6 + wildcard bind socket (3.8) are supported. See 23.5.1.12 + in this document for details. + + + + + + RFC2675: IPv6 Jumbograms + + + See 23.5.1.7 in + this document for details. + + + + + + RFC2710: Multicast Listener Discovery for IPv6 + + + + RFC2711: IPv6 router alert option + + + + draft-ietf-ipngwg-router-renum-08: Router + renumbering for IPv6 + + + + draft-ietf-ipngwg-icmp-namelookups-02: + IPv6 Name Lookups Through ICMP + + + + draft-ietf-ipngwg-icmp-name-lookups-03: + IPv6 Name Lookups Through ICMP + + + + draft-ietf-pim-ipv6-01.txt: + PIM for IPv6 + + + &man.pim6dd.8; implements dense mode. &man.pim6sd.8; + implements sparse mode. + + + + + + draft-itojun-ipv6-tcp-to-anycast-00: + Disconnecting TCP connection toward IPv6 anycast address + + + + draft-yamamoto-wideipv6-comm-model-00 + + + + See 23.5.1.6 in this + document for details. + + + + + + draft-ietf-ipngwg-scopedaddr-format-00.txt + : An Extension of Format for IPv6 Scoped + Addresses + + + + + + Neighbor Discovery + + Neighbor Discovery is fairly stable. Currently Address + Resolution, Duplicated Address Detection, and Neighbor Unreachability + Detection are supported. In the near future we will be adding Proxy + Neighbor Advertisement support in the kernel and Unsolicited Neighbor + Advertisement transmission command as admin tool. + + If DAD fails, the address will be marked "duplicated" and + message will be generated to syslog (and usually to console). The + "duplicated" mark can be checked with &man.ifconfig.8;. It is + administrators' responsibility to check for and recover from DAD + failures. The behavior should be improved in the near future. + + Some of the network driver loops multicast packets back to itself, + even if instructed not to do so (especially in promiscuous mode). + In such cases DAD may fail, because DAD engine sees inbound NS packet + (actually from the node itself) and considers it as a sign of duplicate. + You may want to look at #if condition marked "heuristics" in + sys/netinet6/nd6_nbr.c:nd6_dad_timer() as workaround (note that the code + fragment in "heuristics" section is not spec conformant). + + Neighbor Discovery specification (RFC2461) does not talk about + neighbor cache handling in the following cases: + + + + when there was no neighbor cache entry, node + received unsolicited RS/NS/NA/redirect packet without + link-layer address + + + neighbor cache handling on medium without link-layer + address (we need a neighbor cache entry for IsRouter bit) + + + + For first case, we implemented workaround based on discussions + on IETF ipngwg mailing list. For more details, see the comments in + the source code and email thread started from (IPng 7155), dated + Feb 6 1999. + + IPv6 on-link determination rule (RFC2461) is quite different + from assumptions in BSD network code. At this moment, no on-link + determination rule is supported where default router list is empty + (RFC2461, section 5.2, last sentence in 2nd paragraph - note that + the spec misuse the word "host" and "node" in several places in + the section). + + To avoid possible DoS attacks and infinite loops, only 10 + options on ND packet is accepted now. Therefore, if you have 20 + prefix options attached to RA, only the first 10 prefixes will be + recognized. If this troubles you, please ask it on FREEBSD-CURRENT + mailing list and/or modify nd6_maxndopt in + sys/netinet6/nd6.c. If there are high demands + we may provide sysctl knob for the variable. + + + + Scope Index + + IPv6 uses scoped addresses. Therefore, it is very important to + specify scope index (interface index for link-local address, or + site index for site-local address) with an IPv6 address. Without + scope index, scoped IPv6 address is ambiguous to the kernel, and + kernel will not be able to determine the outbound interface for a + packet. + + Ordinary userland applications should use advanced API + (RFC2292) to specify scope index, or interface index. For similar + purpose, sin6_scope_id member in sockaddr_in6 structure is defined + in RFC2553. However, the semantics for sin6_scope_id is rather vague. + If you care about portability of your application, we suggest you to + use advanced API rather than sin6_scope_id. + + In the kernel, an interface index for link-local scoped address is + embedded into 2nd 16bit-word (3rd and 4th byte) in IPv6 address. For + example, you may see something like: + + + + fe80:1::200:f8ff:fe01:6317 + + + in the routing table and interface address structure (struct + in6_ifaddr). The address above is a link-local unicast address + which belongs to a network interface whose interface identifier is 1. + The embedded index enables us to identify IPv6 link local + addresses over multiple interfaces effectively and with only a + little code change. + + Routing daemons and configuration programs, like &man.route6d.8; + and &man.ifconfig.8;, will need to manipulate the "embedded" scope + index. These programs use routing sockets and ioctls (like + SIOCGIFADDR_IN6) and the kernel API will return IPv6 addresses with + 2nd 16bit-word filled in. The APIs are for manipulating kernel + internal structure. Programs that use these APIs have to be prepared + about differences in kernels anyway. + + When you specify scoped address to the command line, NEVER write + the embedded form (such as ff02:1::1 or fe80:2::fedc). This is not + supposed to work. Always use standard form, like ff02::1 or + fe80::fedc, with command line option for specifying interface (like + ping6 -I ne0 ff02::1). In general, if a command + does not have command line option to specify outgoing interface, that + command is not ready to accept scoped address. This may seem to be + opposite from IPv6's premise to support "dentist office" situation. + We believe that specifications need some improvements for this. + + Some of the userland tools support extended numeric IPv6 syntax, + as documented in + draft-ietf-ipngwg-scopedaddr-format-00.txt. You + can specify outgoing link, by using name of the outgoing interface + like "fe80::1%ne0". This way you will be able to specify link-local + scoped address without much trouble. + + To use this extension in your program, you'll need to use + &man.getaddrinfo.3;, and &man.getnameinfo.3; with NI_WITHSCOPEID. + The implementation currently assumes 1-to-1 relationship between a + link and an interface, which is stronger than what specs say. + + + + Plug and Play + + Most of the IPv6 stateless address autoconfiguration is implemented + in the kernel. Neighbor Discovery functions are implemented in the + kernel as a whole. Router Advertisement (RA) input for hosts is + implemented in the kernel. Router Solicitation (RS) output for + endhosts, RS input for routers, and RA output for routers are + implemented in the userland. + + + Assignment of link-local, and special addresses + + IPv6 link-local address is generated from IEEE802 address + (ethernet MAC address). Each of interface is assigned an IPv6 + link-local address automatically, when the interface becomes up + (IFF_UP). Also, direct route for the link-local address is added + to routing table. + + Here is an output of netstat command: + + +Internet6: +Destination Gateway Flags Netif Expire +fe80:1::%ed0/64 link#1 UC ed0 +fe80:2::%ep0/64 link#2 UC ep0 + + + Interfaces that has no IEEE802 address (pseudo interfaces + like tunnel interfaces, or ppp interfaces) will borrow IEEE802 + address from other interfaces, such as ethernet interfaces, + whenever possible. If there is no IEEE802 hardware attached, + last-resort pseudorandom value, which is from MD5(hostname), will + be used as source of link-local address. If it is not suitable + for your usage, you will need to configure the link-local address + manually. + + If an interface is not capable of handling IPv6 (such as + lack of multicast support), link-local address will not be + assigned to that interface. See section 2 for details. + + Each interface joins the solicited multicast address and the + link-local all-nodes multicast addresses (e.g. fe80::1:ff01:6317 + and ff02::1, respectively, on the link the interface is attached). + In addition to a link-local address, the loopback address (::1) + will be assigned to the loopback interface. Also, ::1/128 and + ff01::/32 are automatically added to routing table, and loopback + interface joins node-local multicast group ff01::1. + + + + Stateless address autoconfiguration on hosts + + In IPv6 specification, nodes are separated into two categories: + routers and hosts. Routers + forward packets addressed to others, hosts does not forward the + packets. net.inet6.ip6.forwarding defines whether this node is + router or host (router if it is 1, host if it is 0). + + When a host hears Router Advertisement from the router, a host + may autoconfigure itself by stateless address autoconfiguration. + This behavior can be controlled by net.inet6.ip6.accept_rtadv (host + autoconfigures itself if it is set to 1). By autoconfiguration, + network address prefix for the receiving interface (usually global + address prefix) is added. Default route is also configured. + Routers periodically generate Router Advertisement packets. To + request an adjacent router to generate RA packet, a host can + transmit Router Solicitation. To generate a RS packet at any time, + use the rtsol command. &man.rtsold.8; daemon is + also available. &man.rtsold.8; generates Router Solicitation whenever + necessary, and it works great for nomadic usage (notebooks/laptops). + If one wishes to ignore Router Advertisements, use sysctl to set + net.inet6.ip6.accept_rtadv to 0. + + To generate Router Advertisement from a router, use the + &man.rtadvd.8 daemon. + + Note that, IPv6 specification assumes the following items, and + nonconforming cases are left unspecified: + + + + Only hosts will listen to router advertisements + + + Hosts have single network interface (except loopback) + + + + Therefore, this is unwise to enable net.inet6.ip6.accept_rtadv + on routers, or multi-interface host. A misconfigured node can + behave strange (nonconforming configuration allowed for those who + would like to do some experiments). + + To summarize the sysctl knob: + + + accept_rtadv forwarding role of the node + --- --- --- + 0 0 host (to be manually configured) + 0 1 router + 1 0 autoconfigured host + (spec assumes that host has single + interface only, autoconfigured host + with multiple interface is + out-of-scope) + 1 1 invalid, or experimental + (out-of-scope of spec) + + + RFC2462 has validation rule against incoming RA prefix + information option, in 5.5.3 (e). This is to protect hosts from + malicious (or misconfigured) routers that advertise very short + prefix lifetime. There was an update from Jim Bound to ipngwg + mailing list (look for "(ipng 6712)" in the archive) and it is + implemented Jim's update. + + See 23.5.1.2 in + the document for relationship between DAD and + autoconfiguration. + + + + + Generic tunnel interface + + GIF (Generic InterFace) is a pseudo interface for configured + tunnel. Details are described in &man.gif.4;. Currently + + + + v6 in v6 + + + v6 in v4 + + + v4 in v6 + + + v4 in v4 + + + + are available. Use &man.gifconfig.8; to assign physical (outer) + source and destination address to gif interfaces. Configuration that + uses same address family for inner and outer IP header (v4 in v4, or + v6 in v6) is dangerous. It is very easy to configure interfaces and + routing tables to perform infinite level of tunneling. + Please be warned. + + gif can be configured to be ECN-friendly. See 23.5.4.5 for ECN-friendliness of + tunnels, and &man.gif.4; for how to configure. + + If you would like to configure an IPv4-in-IPv6 tunnel with gif + interface, read &man.gif.4; carefully. You will need to + remove IPv6 link-local address automatically assigned to the gif + interface. + + + + Source Address Selection + + Current source selection rule is scope oriented (there are some + exceptions - see below). For a given destination, a source IPv6 + address is selected by the following rule: + + + + If the source address is explicitly specified by + the user (e.g. via the advanced API), the specified address + is used. + + + + If there is an address assigned to the outgoing + interface (which is usually determined by looking up the + routing table) that has the same scope as the destination + address, the address is used. + + This is the most typical case. + + + + If there is no address that satisfies the above + condition, choose a global address assigned to one of + the interfaces on the sending node. + + + + If there is no address that satisfies the above condition, + and destination address is site local scope, choose a site local + address assigned to one of the interfaces on the sending node. + + + + + If there is no address that satisfies the above condition, + choose the address associated with the routing table entry for the + destination. This is the last resort, which may cause scope + violation. + + + + For instance, ::1 is selected for ff01::1, + fe80:1::200:f8ff:fe01:6317 for fe80:1::2a0:24ff:feab:839b (note + that embedded interface index - described in 23.5.1.3 - helps us + choose the right source address. Those embedded indices will not + be on the wire). If the outgoing interface has multiple address for + the scope, a source is selected longest match basis (rule 3). Suppose + 3ffe:501:808:1:200:f8ff:fe01:6317 and 3ffe:2001:9:124:200:f8ff:fe01:6317 + are given to the outgoing interface. 3ffe:501:808:1:200:f8ff:fe01:6317 + is chosen as the source for the destination 3ffe:501:800::1. + + Note that the above rule is not documented in the IPv6 spec. + It is considered "up to implementation" item. There are some cases + where we do not use the above rule. One example is connected TCP + session, and we use the address kept in tcb as the source. Another + example is source address for Neighbor Advertisement. Under the spec + (RFC2461 7.2.2) NA's source should be the target address of the + corresponding NS's target. In this case we follow the spec rather + than the above longest-match rule. + + For new connections (when rule 1 does not apply), deprecated + addresses (addresses with preferred lifetime = 0) will not be chosen + as source address if other choices are available. If no other choices + are available, deprecated address will be used as a last resort. If + there are multiple choice of deprecated addresses, the above scope + rule will be used to choose from those deprecated addresses. If you + would like to prohibit the use of deprecated address for some reason, + configure net.inet6.ip6.use_deprecated to 0. The issue related to + deprecated address is described in RFC2462 5.5.4 (NOTE: there is + some debate underway in IETF ipngwg on how to use "deprecated" + address). + + + + Jumbo Payload + + The Jumbo Payload hop-by-hop option is implemented and can + be used to send IPv6 packets with payloads longer than 65,535 octets. + But currently no physical interface whose MTU is more than 65,535 is + supported, so such payloads can be seen only on the loopback + interface (i.e. lo0). + + If you want to try jumbo payloads, you first have to reconfigure + the kernel so that the MTU of the loopback interface is more than + 65,535 bytes; add the following to the kernel configuration file: + + + options "LARGE_LOMTU" #To test jumbo payload + + + and recompile the new kernel. + + Then you can test jumbo payloads by the &man.ping6.8; command + with -b and -s options. The -b option must be specified to enlarge + the size of the socket buffer and the -s option specifies the length + of the packet, which should be more than 65,535. For example, + type as follows: + + + &prompt.user; ping6 -b 70000 -s 68000 ::1 + + + The IPv6 specification requires that the Jumbo Payload option + must not be used in a packet that carries a fragment header. If + this condition is broken, an ICMPv6 Parameter Problem message must + be sent to the sender. specification is followed, but you cannot + usually see an ICMPv6 error caused by this requirement. + + When an IPv6 packet is received, the frame length is checked and + compared to the length specified in the payload length field of the + IPv6 header or in the value of the Jumbo Payload option, if any. If + the former is shorter than the latter, the packet is discarded and + statistics are incremented. You can see the statistics as output of + &man.netstat.8; command with `-s -p ip6' option: + + + &prompt.user; netstat -s -p ip6 + ip6: + (snip) + 1 with data size < data length + + + So, kernel does not send an ICMPv6 error unless the erroneous + packet is an actual Jumbo Payload, that is, its packet size is more + than 65,535 bytes. As described above, currently no physical interface + with such a huge MTU is supported, so it rarely returns an + ICMPv6 error. + + TCP/UDP over jumbogram is not supported at this moment. This + is because we have no medium (other than loopback) to test this. + Contact us if you need this. + + IPsec does not work on jumbograms. This is due to some + specification twists in supporting AH with jumbograms (AH header + size influences payload length, and this makes it real hard to + authenticate inbound packet with jumbo payload option as well as AH). + + + There are fundamental issues in *BSD support for jumbograms. + We would like to address those, but we need more time to finalize + these. To name a few: + + + + mbuf pkthdr.len field is typed as "int" in 4.4BSD, so + it will not hold jumbogram with len > 2G on 32bit architecture + CPUs. If we would like to support jumbogram properly, the field + must be expanded to hold 4G + IPv6 header + link-layer header. + Therefore, it must be expanded to at least int64_t + (u_int32_t is NOT enough). + + + + We mistakingly use "int" to hold packet length in many + places. We need to convert them into larger integral type. + It needs a great care, as we may experience overflow during + packet length computation. + + + + We mistakingly check for ip6_plen field of IPv6 header + for packet payload length in various places. We should be + checking mbuf pkthdr.len instead. ip6_input() will perform + sanity check on jumbo payload option on input, and we can + safely use mbuf pkthdr.len afterwards. + + + + TCP code needs a careful update in bunch of places, of + course. + + + + + + Loop prevention in header processing + + IPv6 specification allows arbitrary number of extension headers + to be placed onto packets. If we implement IPv6 packet processing + code in the way BSD IPv4 code is implemented, kernel stack may + overflow due to long function call chain. sys/netinet6 code + is carefully designed to avoid kernel stack overflow. Because of + this, sys/netinet6 code defines its own protocol switch + structure, as "struct ip6protosw" (see + netinet6/ip6protosw.h). There is no such + update to IPv4 part (sys/netinet) for compatibility, but small + change is added to its pr_input() prototype. So "struct ipprotosw" + is also defined. Because of this, if you receive IPsec-over-IPv4 + packet with massive number of IPsec headers, kernel stack may blow + up. IPsec-over-IPv6 is okay. (Off-course, for those all IPsec + headers to be processed, each such IPsec header must pass each + IPsec check. So an anonymous attacker won't be able to do such an + attack.) + + + + ICMPv6 + + After RFC2463 was published, IETF ipngwg has decided to + disallow ICMPv6 error packet against ICMPv6 redirect, to prevent + ICMPv6 storm on a network medium. This is already implemented + into the kernel. + + + + Applications + + For userland programming, we support IPv6 socket API as + specified in RFC2553, RFC2292 and upcoming internet drafts. + + TCP/UDP over IPv6 is available and quite stable. You can + enjoy &man.telnet.1;, &man.ftp.1;, &man.rlogin.1;, &man.rsh.1;, + &man.ssh.1, etc. These applications are protocol independent. + That is, they automatically chooses IPv4 or IPv6 according to DNS. + + + + + Kernel Internals + + While ip_forward() calls ip_output(), ip6_forward() directly + calls if_output() since routers must not divide IPv6 packets into + fragments. + + ICMPv6 should contain the original packet as long as possible + up to 1280. UDP6/IP6 port unreach, for instance, should contain + all extension headers and the *unchanged* UDP6 and IP6 headers. + So, all IP6 functions except TCP never convert network byte + order into host byte order, to save the original packet. + + tcp_input(), udp6_input() and icmp6_input() can't assume that + IP6 header is preceding the transport headers due to extension + headers. So, in6_cksum() was implemented to handle packets whose IP6 + header and transport header is not continuous. TCP/IP6 nor UDP6/IP6 + header structure don't exist for checksum calculation. + + To process IP6 header, extension headers and transport headers + easily, network drivers are now required to store packets in one + internal mbuf or one or more external mbufs. A typical old driver + prepares two internal mbufs for 96 - 204 bytes data, however, now + such packet data is stored in one external mbuf. + + netstat -s -p ip6 tells you whether or not + your driver conforms such requirement. In the following example, + "cce0" violates the requirement. (For more information, refer to + Section 2.) + + + Mbuf statistics: + 317 one mbuf + two or more mbuf:: + lo0 = 8 + cce0 = 10 + 3282 one ext mbuf + 0 two or more ext mbuf + + + Each input function calls IP6_EXTHDR_CHECK in the beginning to + check if the region between IP6 and its header is continuous. + IP6_EXTHDR_CHECK calls m_pullup() only if the mbuf has M_LOOP flag, + that is, the packet comes from the loopback interface. m_pullup() + is never called for packets coming from physical network interfaces. + + + Both IP and IP6 reassemble functions never call m_pullup(). + + + + IPv4 mapped address and IPv6 wildcard socket + + RFC2553 describes IPv4 mapped address (3.7) and special behavior + of IPv6 wildcard bind socket (3.8). The spec allows you to: + + + Accept IPv4 connections by AF_INET6 wildcard bind + socket. + + + Transmit IPv4 packet over AF_INET6 socket by using + special form of the address like ::ffff:10.1.1.1. + + + + but the spec itself is very complicated and does not specify + how the socket layer should behave. Here we call the former one + "listening side" and the latter one "initiating side", for + reference purposes. + + You can perform wildcard bind on both of the address families, + on the same port. + + The following table show the behavior of FreeBSD 4.x. + + + listening side initiating side + (AF_INET6 wildcard (connection to ::ffff:10.1.1.1) + socket gets IPv4 conn.) + --- --- +FreeBSD 4.x configurable supported + default: enabled + + + The following sections will give you more details, and how you can + configure the behavior. + + Comments on listening side: + + It looks that RFC2553 talks too little on wildcard bind issue, + especially on the port space issue, failure mode and relationship + between AF_INET/INET6 wildcard bind. There can be several separate + interpretation for this RFC which conform to it but behaves differently. + So, to implement portable application you should assume nothing + about the behavior in the kernel. Using &man.getaddrinfo.3; is the + safest way. Port number space and wildcard bind issues were discussed + in detail on ipv6imp mailing list, in mid March 1999 and it looks + that there's no concrete consensus (means, up to implementers). + You may want to check the mailing list archives. + + If a server application would like to accept IPv4 and IPv6 + connections, there will be two alternatives. + + One is using AF_INET and AF_INET6 socket (you'll need two + sockets). Use &man.getaddrinfo.3; with AI_PASSIVE into ai_flags, + and &man.socket.2; and &man.bind.2; to all the addresses returned. + By opening multiple sockets, you can accept connections onto the + socket with proper address family. IPv4 connections will be + accepted by AF_INET socket, and IPv6 connections will be accepted + by AF_INET6 socket. + + Another way is using one AF_INET6 wildcard bind socket. Use + &man.getaddrinfo.3; with AI_PASSIVE into ai_flags and with + AF_INET6 into ai_family, and set the 1st argument hostname to + NULL. And &man.socket.2; and &man.bind.2; to the address returned. + (should be IPv6 unspecified addr). You can accept either of IPv4 + and IPv6 packet via this one socket. + + To support only IPv6 traffic on AF_INET6 wildcard binded socket + portably, always check the peer address when a connection is made + toward AF_INET6 listening socket. If the address is IPv4 mapped + address, you may want to reject the connection. You can check the + condition by using IN6_IS_ADDR_V4MAPPED() macro. + + To resolve this issue more easily, there is system dependent + &man.setsockopt.2; option, IPV6_BINDV6ONLY, used like below. + + + int on; + + setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY, + (char *)&on, sizeof (on)) < 0)); + + + When this call succeed, then this socket only receive IPv6 + packets. + + Comments on initiating side: + + Advise to application implementers: to implement a portable + IPv6 application (which works on multiple IPv6 kernels), we believe + that the following is the key to the success: + + + + NEVER hardcode AF_INET nor AF_INET6. + + + + Use &man.getaddrinfo.3; and &man.getnameinfo.3; + throughout the system. Never use gethostby*(), getaddrby*(), + inet_*() or getipnodeby*(). (To update existing applications + to be IPv6 aware easily, sometime getipnodeby*() will be + useful. But if possible, try to rewrite the code to use + &man.getaddrinfo.3; and &man.getnameinfo.3;.) + + + + If you would like to connect to destination, use + &man.getaddrinfo.3; and try all the destination returned, + like &man.telnet.1; does. + + + + Some of the IPv6 stack is shipped with buggy + &man.getaddrinfo.3;. Ship a minimal working version with + your application and use that as last resort. + + + + If you would like to use AF_INET6 socket for both IPv4 and + IPv6 outgoing connection, you will need to use &man.getipnodebyname.3;. + When you would like to update your existing application to be IPv6 + aware with minimal effort, this approach might be chosen. But please + note that it is a temporal solution, because &man.getipnodebyname.3; + itself is not recommended as it does not handle scoped IPv6 addresses + at all. For IPv6 name resolution, &man.getaddrinfo.3; is the + preferred API. So you should rewrite your application to use + &man.getaddrinfo.3;, when you get the time to do it. + + When writing applications that make outgoing connections, + story goes much simpler if you treat AF_INET and AF_INET6 as totally + separate address family. {set,get}sockopt issue goes simpler, + DNS issue will be made simpler. We do not recommend you to rely + upon IPv4 mapped address. + + + unified tcp and inpcb code + + FreeBSD 4.x uses shared tcp code between IPv4 and IPv6 + (from sys/netinet/tcp*) and separate udp4/6 code. It uses + unified inpcb structure. + + The platform can be configured to support IPv4 mapped address. + Kernel configuration is summarized as follows: + + + + By default, AF_INET6 socket will grab IPv4 + connections in certain condition, and can initiate + connection to IPv4 destination embedded in IPv4 mapped + IPv6 address. + + + + You can disable it on entire system with sysctl like + below. + + + sysctl -w net.inet6.ip6.mapped_addr=0 + + + + + + + listening side + + Each socket can be configured to support special AF_INET6 + wildcard bind (enabled by default). You can disable it on + each socket basis with &man.setsockopt.2; like below. + + + int on; + + setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY, + (char *)&on, sizeof (on)) < 0)); + + + Wildcard AF_INET6 socket grabs IPv4 connection if and only + if the following conditions are satisfied: + + + + there's no AF_INET socket that matches the IPv4 + connection + + + + the AF_INET6 socket is configured to accept IPv4 + traffic, i.e. getsockopt(IPV6_BINDV6ONLY) returns 0. + + + + There's no problem with open/close ordering. + + + + initiating side + + FreeBSD 4.x supports outgoing connection to IPv4 mapped + address (::ffff:10.1.1.1), if the node is configured to support + IPv4 mapped address. + + + + + + sockaddr_storage + + When RFC2553 was about to be finalized, there was discussion on + how struct sockaddr_storage members are named. One proposal is to + prepend "__" to the members (like "__ss_len") as they should not be + touched. The other proposal was that don't prepend it (like "ss_len") + as we need to touch those members directly. There was no clear + consensus on it. + + As a result, RFC2553 defines struct sockaddr_storage as + follows: + + + struct sockaddr_storage { + u_char __ss_len; /* address length */ + u_char __ss_family; /* address family */ + /* and bunch of padding */ + }; + + + On the contrary, XNET draft defines as follows: + + + struct sockaddr_storage { + u_char ss_len; /* address length */ + u_char ss_family; /* address family */ + /* and bunch of padding */ + }; + + + In December 1999, it was agreed that RFC2553bis should pick + the latter (XNET) definition. + + Current implementation conforms to XNET definition, based on + RFC2553bis discussion. + + If you look at multiple IPv6 implementations, you will be able + to see both definitions. As an userland programmer, the most + portable way of dealing with it is to: + + + + ensure ss_family and/or ss_len are available on the + platform, by using GNU autoconf, + + + + have -Dss_family=__ss_family to unify all occurrences + (including header file) into __ss_family, or + + + + never touch __ss_family. cast to sockaddr * and use sa_family + like: + + + struct sockaddr_storage ss; + family = ((struct sockaddr *)&ss)->sa_family + + + + + + + + + Network Drivers + + Now following two items are required to be supported by standard + drivers: + + + + mbuf clustering requirement. In this stable release, we + changed MINCLSIZE into MHLEN+1 for all the operating systems + in order to make all the drivers behave as we expect. + + + + multicast. If &man.ifmcstat.8; yields no multicast group for + a interface, that interface has to be patched. + + + + If any of the driver don't support the requirements, then + the driver can't be used for IPv6 and/or IPsec communication. If + you find any problem with your card using IPv6/IPsec, then, please + report it to freebsd-bugs@FreeBSD.org. + + (NOTE: In the past we required all PCMCIA drivers to have a + call to in6_ifattach(). We have no such requirement any more) + + + + Translator + + We categorize IPv4/IPv6 translator into 4 types: + + + + Translator A --- It is used in the early + stage of transition to make it possible to establish a + connection from an IPv6 host in an IPv6 island to an IPv4 host + in the IPv4 ocean. + + + + Translator B --- It is used in the early + stage of transition to make it possible to establish a connection + from an IPv4 host in the IPv4 ocean to an IPv6 host in an + IPv6 island. + + + + Translator C --- It is used in the late + stage of transition to make it possible to establish a + connection from an IPv4 host in an IPv4 island to an IPv6 host + in the IPv6 ocean. + + + + Translator D --- It is used in the late + stage of transition to make it possible to establish a + connection from an IPv6 host in the IPv6 ocean to an IPv4 host + in an IPv4 island. + + + + TCP relay translator for category A is supported. This is called + "FAITH". We also provide IP header translator for category A. + (The latter is not yet put into FreeBSD 4.x yet.) + + + FAITH TCP relay translator + + FAITH system uses TCP relay daemon called &man.faithd.8; helped + by the kernel. FAITH will reserve an IPv6 address prefix, and relay + TCP connection toward that prefix to IPv4 destination. + + For example, if the reserved IPv6 prefix is + 3ffe:0501:0200:ffff::, and the IPv6 destination for TCP connection + is 3ffe:0501:0200:ffff::163.221.202.12, the connection will be + relayed toward IPv4 destination 163.221.202.12. + + + destination IPv4 node (163.221.202.12) + ^ + | IPv4 tcp toward 163.221.202.12 + FAITH-relay dual stack node + ^ + | IPv6 TCP toward 3ffe:0501:0200:ffff::163.221.202.12 + source IPv6 node + + + &man.faithd.8; must be invoked on FAITH-relay dual stack + node. + + For more details, consult + src/usr.sbin/faithd/README + + + + + IPsec + + IPsec is mainly organized by three components. + + + + Policy Management + + + + Key Management + + + + AH and ESP handling + + + + + Policy Management + + The kernel implements experimental policy management code. + There are two way to manage security policy. One is to configure + per-socket policy using &man.setsockopt.2;. In this cases, policy + configuration is described in &man.ipsec.set.policy.3;. The other + is to configure kernel packet filter-based policy using PF_KEY + interface, via &man.setkey.8;. + + The policy entry is not re-ordered with its + indexes, so the order of entry when you add is very significant. + + + + Key Management + + The key management code implemented in this kit (sys/netkey) + is a home-brew PFKEY v2 implementation. This conforms to RFC2367. + + + The home-brew IKE daemon, "racoon" is included in the + kit (kame/kame/racoon). Basically you'll need to run racoon as + daemon, then setup a policy to require keys (like + ping -P 'out ipsec esp/transport//use'). + The kernel will contact racoon daemon as necessary to exchange + keys. + + + + AH and ESP handling + + IPsec module is implemented as "hooks" to the standard IPv4/IPv6 + processing. When sending a packet, ip{,6}_output() checks if ESP/AH + processing is required by checking if a matching SPD (Security + Policy Database) is found. If ESP/AH is needed, + {esp,ah}{4,6}_output() will be called and mbuf will be updated + accordingly. When a packet is received, {esp,ah}4_input() will be + called based on protocol number, i.e. (*inetsw[proto])(). + {esp,ah}4_input() will decrypt/check authenticity of the packet, + and strips off daisy-chained header and padding for ESP/AH. It is + safe to strip off the ESP/AH header on packet reception, since we + will never use the received packet in "as is" form. + + By using ESP/AH, TCP4/6 effective data segment size will be + affected by extra daisy-chained headers inserted by ESP/AH. Our + code takes care of the case. + + Basic crypto functions can be found in directory "sys/crypto". + ESP/AH transform are listed in {esp,ah}_core.c with wrapper functions. + If you wish to add some algorithm, add wrapper function in + {esp,ah}_core.c, and add your crypto algorithm code into + sys/crypto. + + Tunnel mode is partially supported in this release, with the + following restrictions: + + + + IPsec tunnel is not combined with GIF generic tunneling + interface. It needs a great care because we may create an + infinite loop between ip_output() and tunnelifp->if_output(). + Opinion varies if it is better to unify them, or not. + + + + MTU and Don't Fragment bit (IPv4) considerations need more + checking, but basically works fine. + + + + Authentication model for AH tunnel must be revisited. + We'll need to improve the policy management engine, + eventually. + + + + + + Conformance to RFCs and IDs + + The IPsec code in the kernel conforms (or, tries to conform) + to the following standards: + + "old IPsec" specification documented in + rfc182[5-9].txt + + "new IPsec" specification documented in + rfc240[1-6].txt, + rfc241[01].txt, rfc2451.txt + and draft-mcdonald-simple-ipsec-api-01.txt + (draft expired, but you can take from + ftp://ftp.kame.net/pub/internet-drafts/). + (NOTE: IKE specifications, rfc241[7-9].txt are + implemented in userland, as "racoon" IKE daemon) + + Currently supported algorithms are: + + + old IPsec AH + + + null crypto checksum (no document, just for + debugging) + + + keyed MD5 with 128bit crypto checksum + (rfc1828.txt) + + + keyed SHA1 with 128bit crypto checksum + (no document) + + + HMAC MD5 with 128bit crypto checksum + (rfc2085.txt) + + + HMAC SHA1 with 128bit crypto checksum + (no document) + + + + + + old IPsec ESP + + + null encryption (no document, similar to + rfc2410.txt) + + + DES-CBC mode (rfc1829.txt) + + + + + + new IPsec AH + + + null crypto checksum (no document, + just for debugging) + + + keyed MD5 with 96bit crypto checksum + (no document) + + + keyed SHA1 with 96bit crypto checksum + (no document) + + + HMAC MD5 with 96bit crypto checksum + (rfc2403.txt) + + + HMAC SHA1 with 96bit crypto checksum + (rfc2404.txt) + + + + + + new IPsec ESP + + + null encryption + (rfc2410.txt) + + + DES-CBC with derived IV + (draft-ietf-ipsec-ciph-des-derived-01.txt, + draft expired) + + + DES-CBC with explicit IV + (rfc2405.txt) + + + 3DES-CBC with explicit IV + (rfc2451.txt) + + + BLOWFISH CBC + (rfc2451.txt) + + + CAST128 CBC + (rfc2451.txt) + + + RC5 CBC + (rfc2451.txt) + + + each of the above can be combined with: + + + ESP authentication with HMAC-MD5(96bit) + + + ESP authentication with HMAC-SHA1(96bit) + + + + + + + + The following algorithms are NOT supported: + + + + old IPsec AH + + + + HMAC MD5 with 128bit crypto checksum + 64bit + replay prevention (rfc2085.txt) + + + keyed SHA1 with 160bit crypto checksum + 32bit padding + (rfc1852.txt) + + + + + + + IPsec (in kernel) and IKE (in userland as "racoon") has been + tested at several interoperability test events, and it is known to + interoperate with many other implementations well. Also, current + IPsec implementation as quite wide coverage for IPsec crypto + algorithms documented in RFC (we cover algorithms without intellectual + property issues only). + + + + ECN consideration on IPsec tunnels + + ECN-friendly IPsec tunnel is supported as described in + draft-ipsec-ecn-00.txt. + + Normal IPsec tunnel is described in RFC2401. On encapsulation, + IPv4 TOS field (or, IPv6 traffic class field) will be copied from inner + IP header to outer IP header. On decapsulation outer IP header + will be simply dropped. The decapsulation rule is not compatible + with ECN, since ECN bit on the outer IP TOS/traffic class field will be + lost. + + To make IPsec tunnel ECN-friendly, we should modify encapsulation + and decapsulation procedure. This is described in + http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt, + chapter 3. + + IPsec tunnel implementation can give you three behaviors, by + setting net.inet.ipsec.ecn (or net.inet6.ipsec6.ecn) to some + value: + + + + RFC2401: no consideration for ECN (sysctl value -1) + + + ECN forbidden (sysctl value 0) + + + ECN allowed (sysctl value 1) + + + + Note that the behavior is configurable in per-node manner, + not per-SA manner (draft-ipsec-ecn-00 wants per-SA configuration, + but it looks too much for me). + + The behavior is summarized as follows (see source code for + more detail): + + + + encapsulate decapsulate + --- --- +RFC2401 copy all TOS bits drop TOS bits on outer + from inner to outer. (use inner TOS bits as is) + +ECN forbidden copy TOS bits except for ECN drop TOS bits on outer + (masked with 0xfc) from inner (use inner TOS bits as is) + to outer. set ECN bits to 0. + +ECN allowed copy TOS bits except for ECN use inner TOS bits with some + CE (masked with 0xfe) from change. if outer ECN CE bit + inner to outer. is 1, enable ECN CE bit on + set ECN CE bit to 0. the inner. + + + + General strategy for configuration is as follows: + + + if both IPsec tunnel endpoint are capable of ECN-friendly + behavior, you'd better configure both end to "ECN allowed" + (sysctl value 1). + + + if the other end is very strict about TOS bit, use "RFC2401" + (sysctl value -1). + + + in other cases, use "ECN forbidden" (sysctl value 0). + + + + The default behavior is "ECN forbidden" (sysctl value 0). + + For more information, please refer to: + + + http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt, + RFC2481 (Explicit Congestion Notification), + src/sys/netinet6/{ah,esp}_input.c + + (Thanks goes to Kenjiro Cho kjc@csl.sony.co.jp + for detailed analysis) + + + + Interoperability + + Here are (some of) platforms that KAME code have tested + IPsec/IKE interoperability in the past. Note that both ends may + have modified their implementation, so use the following list just + for reference purposes. + + Altiga, Ashley-laurent (vpcom.com), Data Fellows (F-Secure), + Ericsson ACC, FreeS/WAN, HITACHI, IBM AIX, IIJ, Intel, + Microsoft WinNT, NIST (linux IPsec + plutoplus), Netscreen, OpenBSD, + RedCreek, Routerware, SSH, Secure Computing, Soliton, Toshiba, + VPNet, Yamaha RT100i + + + + + + diff --git a/en_US.ISO8859-1/books/developers-handbook/vm/chapter.sgml b/en_US.ISO8859-1/books/developers-handbook/vm/chapter.sgml new file mode 100644 index 0000000000..4710973d5f --- /dev/null +++ b/en_US.ISO8859-1/books/developers-handbook/vm/chapter.sgml @@ -0,0 +1,255 @@ + + + + Virtual Memory System + + + The FreeBSD VM System + + Contributed by &a.dillon;. 6 Feb 1999 + + + Management of physical + memory—<literal>vm_page_t</literal> + + Physical memory is managed on a page-by-page basis through the + vm_page_t structure. Pages of physical memory are + categorized through the placement of their respective + vm_page_t structures on one of several paging + queues. + + A page can be in a wired, active, inactive, cache, or free state. + Except for the wired state, the page is typically placed in a doubly + link list queue representing the state that it is in. Wired pages + are not placed on any queue. + + FreeBSD implements a more involved paging queue for cached and + free pages in order to implement page coloring. Each of these states + involves multiple queues arranged according to the size of the + processor's L1 and L2 caches. When a new page needs to be allocated, + FreeBSD attempts to obtain one that is reasonably well aligned from + the point of view of the L1 and L2 caches relative to the VM object + the page is being allocated for. + + Additionally, a page may be held with a reference count or locked + with a busy count. The VM system also implements an ultimate + locked state for a page using the PG_BUSY bit in the page's + flags. + + In general terms, each of the paging queues operates in a LRU + fashion. A page is typically placed in a wired or active state + initially. When wired, the page is usually associated with a page + table somewhere. The VM system ages the page by scanning pages in a + more active paging queue (LRU) in order to move them to a less-active + paging queue. Pages that get moved into the cache are still + associated with a VM object but are candidates for immediate reuse. + Pages in the free queue are truly free. FreeBSD attempts to minimize + the number of pages in the free queue, but a certain minimum number of + truly free pages must be maintained in order to accommodate page + allocation at interrupt time. + + If a process attempts to access a page that does not exist in its + page table but does exist in one of the paging queues ( such as the + inactive or cache queues), a relatively inexpensive page reactivation + fault occurs which causes the page to be reactivated. If the page + does not exist in system memory at all, the process must block while + the page is brought in from disk. + + FreeBSD dynamically tunes its paging queues and attempts to + maintain reasonable ratios of pages in the various queues as well as + attempts to maintain a reasonable breakdown of clean v.s. dirty pages. + The amount of rebalancing that occurs depends on the system's memory + load. This rebalancing is implemented by the pageout daemon and + involves laundering dirty pages (syncing them with their backing + store), noticing when pages are activity referenced (resetting their + position in the LRU queues or moving them between queues), migrating + pages between queues when the queues are out of balance, and so forth. + FreeBSD's VM system is willing to take a reasonable number of + reactivation page faults to determine how active or how idle a page + actually is. This leads to better decisions being made as to when to + launder or swap-out a page. + + + + The unified buffer + cache—<literal>vm_object_t</literal> + + FreeBSD implements the idea of a generic VM object. + VM objects can be associated with backing store of various + types—unbacked, swap-backed, physical device-backed, or + file-backed storage. Since the filesystem uses the same VM objects to + manage in-core data relating to files, the result is a unified buffer + cache. + + VM objects can be shadowed. That is, they + can be stacked on top of each other. For example, you might have a + swap-backed VM object stacked on top of a file-backed VM object in + order to implement a MAP_PRIVATE mmap()ing. This stacking is also + used to implement various sharing properties, including, + copy-on-write, for forked address spaces. + + It should be noted that a vm_page_t can only be + associated with one VM object at a time. The VM object shadowing + implements the perceived sharing of the same page across multiple + instances. + + + + Filesystem I/O—<literal>struct buf</literal> + + vnode-backed VM objects, such as file-backed objects, generally + need to maintain their own clean/dirty info independent from the VM + system's idea of clean/dirty. For example, when the VM system decides + to synchronize a physical page to its backing store, the VM system + needs to mark the page clean before the page is actually written to + its backing s tore. Additionally, filesystems need to be able to map + portions of a file or file metadata into KVM in order to operate on + it. + + The entities used to manage this are known as filesystem buffers, + struct buf's, and also known as + bp's. When a filesystem needs to operate on a + portion of a VM object, it typically maps part of the object into a + struct buf and the maps the pages in the struct buf into KVM. In the + same manner, disk I/O is typically issued by mapping portions of + objects into buffer structures and then issuing the I/O on the buffer + structures. The underlying vm_page_t's are typically busied for the + duration of the I/O. Filesystem buffers also have their own notion of + being busy, which is useful to filesystem driver code which would + rather operate on filesystem buffers instead of hard VM pages. + + FreeBSD reserves a limited amount of KVM to hold mappings from + struct bufs, but it should be made clear that this KVM is used solely + to hold mappings and does not limit the ability to cache data. + Physical data caching is strictly a function of + vm_page_t's, not filesystem buffers. However, + since filesystem buffers are used placehold I/O, they do inherently + limit the amount of concurrent I/O possible. As there are usually a + few thousand filesystem buffers available, this is not usually a + problem. + + + + Mapping Page Tables - vm_map_t, vm_entry_t + + FreeBSD separates the physical page table topology from the VM + system. All hard per-process page tables can be reconstructed on the + fly and are usually considered throwaway. Special page tables such as + those managing KVM are typically permanently preallocated. These page + tables are not throwaway. + + FreeBSD associates portions of vm_objects with address ranges in + virtual memory through vm_map_t and + vm_entry_t structures. Page tables are directly + synthesized from the + vm_map_t/vm_entry_t/ + vm_object_t hierarchy. Remember when I mentioned + that physical pages are only directly associated with a + vm_object. Well, that isn't quite true. + vm_page_t's are also linked into page tables that + they are actively associated with. One vm_page_t + can be linked into several pmaps, as page tables + are called. However, the hierarchical association holds so all + references to the same page in the same object reference the same + vm_page_t and thus give us buffer cache unification + across the board. + + + + KVM Memory Mapping + + FreeBSD uses KVM to hold various kernel structures. The single + largest entity held in KVM is the filesystem buffer cache. That is, + mappings relating to struct buf entities. + + Unlike Linux, FreeBSD does NOT map all of physical memory into + KVM. This means that FreeBSD can handle memory configurations up to + 4G on 32 bit platforms. In fact, if the mmu were capable of it, + FreeBSD could theoretically handle memory configurations up to 8TB on + a 32 bit platform. However, since most 32 bit platforms are only + capable of mapping 4GB of ram, this is a moot point. + + KVM is managed through several mechanisms. The main mechanism + used to manage KVM is the zone allocator. The + zone allocator takes a chunk of KVM and splits it up into + constant-sized blocks of memory in order to allocate a specific type + of structure. You can use vmstat -m to get an + overview of current KVM utilization broken down by zone. + + + + Tuning the FreeBSD VM system + + A concerted effort has been made to make the FreeBSD kernel + dynamically tune itself. Typically you do not need to mess with + anything beyond the maxusers and + NMBCLUSTERS kernel config options. That is, kernel + compilation options specified in (typically) + /usr/src/sys/i386/conf/CONFIG_FILE. + A description of all available kernel configuration options can be + found in /usr/src/sys/i386/conf/LINT. + + In a large system configuration you may wish to increase + maxusers. Values typically range from 10 to 128. + Note that raising maxusers too high can cause the + system to overflow available KVM resulting in unpredictable operation. + It is better to leave maxusers at some reasonable number and add other + options, such as NMBCLUSTERS, to increase specific + resources. + + If your system is going to use the network heavily, you may want + to increase NMBCLUSTERS. Typical values range from + 1024 to 4096. + + The NBUF parameter is also traditionally used + to scale the system. This parameter determines the amount of KVA the + system can use to map filesystem buffers for I/O. Note that this + parameter has nothing whatsoever to do with the unified buffer cache! + This parameter is dynamically tuned in 3.0-CURRENT and later kernels + and should generally not be adjusted manually. We recommend that you + not try to specify an NBUF + parameter. Let the system pick it. Too small a value can result in + extremely inefficient filesystem operation while too large a value can + starve the page queues by causing too many pages to become wired + down. + + By default, FreeBSD kernels are not optimized. You can set + debugging and optimization flags with the + makeoptions directive in the kernel configuration. + Note that you should not use unless you can + accommodate the large (typically 7 MB+) kernels that result. + + makeoptions DEBUG="-g" +makeoptions COPTFLAGS="-O -pipe" + + Sysctl provides a way to tune kernel parameters at run-time. You + typically do not need to mess with any of the sysctl variables, + especially the VM related ones. + + Run time VM and system tuning is relatively straightforward. + First, use softupdates on your UFS/FFS filesystems whenever possible. + /usr/src/contrib/sys/softupdates/README contains + instructions (and restrictions) on how to configure it up. + + Second, configure sufficient swap. You should have a swap + partition configured on each physical disk, up to four, even on your + work disks. You should have at least 2x the swap space + as you have main memory, and possibly even more if you do not have a + lot of memory. You should also size your swap partition based on the + maximum memory configuration you ever intend to put on the machine so + you do not have to repartition your disks later on. If you want to be + able to accommodate a crash dump, your first swap partition must be at + least as large as main memory and /var/crash must + have sufficient free space to hold the dump. + + NFS-based swap is perfectly acceptable on -4.x or later systems, + but you must be aware that the NFS server will take the brunt of the + paging load. + + + + diff --git a/en_US.ISO_8859-1/books/developers-handbook/Makefile b/en_US.ISO_8859-1/books/developers-handbook/Makefile index 371551ab9e..acf0c62b67 100644 --- a/en_US.ISO_8859-1/books/developers-handbook/Makefile +++ b/en_US.ISO_8859-1/books/developers-handbook/Makefile @@ -1,35 +1,38 @@ # -# $FreeBSD: doc/en_US.ISO_8859-1/books/developers-handbook/Makefile,v 1.2 2001/05/02 01:53:13 murray Exp $ +# $FreeBSD: doc/en_US.ISO_8859-1/books/developers-handbook/Makefile,v 1.3 2001/05/11 10:27:00 murray Exp $ # # Build the FreeBSD Developers' Handbook. # MAINTAINER=asmodai@FreeBSD.org DOC?= book FORMATS?= html-split INSTALL_COMPRESSED?= gz INSTALL_ONLY_COMPRESSED?= # # SRCS lists the individual SGML files that make up the document. Changes # to any of these files will force a rebuild # # SGML content SRCS= book.sgml SRCS+= tools/chapter.sgml SRCS+= secure/chapter.sgml SRCS+= locking/chapter.sgml SRCS+= isa/chapter.sgml SRCS+= pci/chapter.sgml SRCS+= usb/chapter.sgml SRCS+= scsi/chapter.sgml SRCS+= x86/chapter.sgml +SRCS+= vm/chapter.sgml +SRCS+= dma/chapter.sgml +SRCS+= ipv6/chapter.sgml # Entities DOC_PREFIX?= ${.CURDIR}/../../.. .include "${DOC_PREFIX}/share/mk/doc.project.mk" diff --git a/en_US.ISO_8859-1/books/developers-handbook/book.sgml b/en_US.ISO_8859-1/books/developers-handbook/book.sgml index a1a485a73d..10879f994e 100644 --- a/en_US.ISO_8859-1/books/developers-handbook/book.sgml +++ b/en_US.ISO_8859-1/books/developers-handbook/book.sgml @@ -1,517 +1,518 @@ %bookinfo; %man; %chapters; %authors; %mailing-lists; ]> FreeBSD Developers' Handbook The FreeBSD Documentation Project August 2000 2000 2001 The FreeBSD Documentation Project &bookinfo.legalnotice; Welcome to the Developers' Handbook. This manual is a work in progress and is the work of many individuals. Many sections do not yet exist and some of those that do exist need to be updated. If you are interested in helping with this project, send email to the &a.doc;. The latest version of this document is always available from the FreeBSD World Wide Web server. It may also be downloaded in a variety of formats and compression options from the FreeBSD FTP server or one of the numerous mirror sites. Introduction Developing on FreeBSD This will need to discuss FreeBSD as a development platform, the vision of BSD, architectural overview, layout of /usr/src, history, etc. Thank you for considering FreeBSD as your development platform! We hope it will not let you down. The BSD Vision Architectural Overview The Layout of /usr/src The complete source code to FreeBSD is available from our public CVS repository. The source code is normally installed in /usr/src which contains the following subdirectories. Directory Description bin/ Source for files in /bin contrib/ Source for files from contributed software. crypto/ DES source etc/ Source for files in /etc games/ Source for files in /usr/games gnu/ Utilities covered by the GNU Public License include/ Source for files in /usr/include kerberosIV/ Source for Kerbereros version IV kerberos5/ Source for Kerbereros version 5 lib/ Source for files in /usr/lib libexec/ Source for files in /usr/libexec release/ Files required to produce a FreeBSD release sbin/ Source for files in /sbin secure/ FreeSec sources share/ Source for files in /sbin sys/ Kernel source files tools/ Tools used for maintenance and testing of FreeBSD usr.bin/ Source for files in /usr/bin usr.sbin/ Source for files in /usr/sbin Basics &chap.tools; &chap.secure; Kernel History of the Unix Kernel Some history of the Unix/BSD kernel, system calls, how do processes work, blocking, scheduling, threads (kernel), context switching, signals, interrupts, modules, etc. &chap.locking; - Memory and Virtual Memory + Memory Management - - Virtual Memory + &chap.vm; + &chap.dma; - VM, paging, swapping, allocating memory, testing for - memory leaks, mmap, vnodes, etc. + - - I/O System UFS UFS, FFS, Ext2FS, JFS, inodes, buffer cache, labeling, locking, metadata, soft-updates, LFS, portalfs, procfs, vnodes, memory sharing, memory objects, TLBs, caching Interprocess Communication Signals Signals, pipes, semaphores, message queues, shared memory, ports, sockets, doors Networking Sockets Sockets, bpf, IP, TCP, UDP, ICMP, OSI, bridging, firewalling, NAT, switching, etc + + &chap.ipv6; + Network Filesystems AFS AFS, NFS, SANs etc] Terminal Handling Syscons Syscons, tty, PCVT, serial console, screen savers, etc Sound OSS OSS, waveforms, etc Device Drivers &chap.driverbasics; &chap.isa; &chap.pci; &chap.scsi; &chap.usb; NewBus This chapter will talk about the FreeBSD NewBus architecture. Architectures &chap.x86; Alpha Talk about the architectural specifics of FreeBSD/alpha. Explanation of allignment errors, how to fix, how to ignore. Example assembly language code for FreeBSD/alpha. IA-64 Talk about the architectural specifics of FreeBSD/ia64. Debugging Truss various descriptions on how to debug certain aspects of the system using truss, ktrace, gdb, kgdb, etc Compatibility Layers Linux Linux, SVR4, etc Appendices Dave A Patterson John L Hennessy 1998Morgan Kaufmann Publishers, Inc. 1-55860-428-6 Morgan Kaufmann Publishers, Inc. Computer Organization and Design The Hardware / Software Interface 1-2 W. Richard Stevens 1993Addison Wesley Longman, Inc. 0-201-56317-7 Addison Wesley Longman, Inc. Advanced Programming in the Unix Environment 1-2 Marshall Kirk McKusick Keith Bostic Michael J Karels John S Quarterman 1996Addison-Wesley Publishing Company, Inc. 0-201-54979-4 Addison-Wesley Publishing Company, Inc. The Design and Implementation of the 4.4 BSD Operating System 1-2 Aleph One Phrack 49; "Smashing the Stack for Fun and Profit" Chrispin Cowan Calton Pu Dave Maier StackGuard; Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks Todd Miller Theo de Raadt strlcpy and strlcat -- consistent, safe string copy and concatenation. diff --git a/en_US.ISO_8859-1/books/developers-handbook/chapters.ent b/en_US.ISO_8859-1/books/developers-handbook/chapters.ent index 6e2d0c3f6f..044535d282 100644 --- a/en_US.ISO_8859-1/books/developers-handbook/chapters.ent +++ b/en_US.ISO_8859-1/books/developers-handbook/chapters.ent @@ -1,61 +1,61 @@ - - + + - + diff --git a/en_US.ISO_8859-1/books/developers-handbook/dma/chapter.sgml b/en_US.ISO_8859-1/books/developers-handbook/dma/chapter.sgml new file mode 100644 index 0000000000..7bbb7d973d --- /dev/null +++ b/en_US.ISO_8859-1/books/developers-handbook/dma/chapter.sgml @@ -0,0 +1,1326 @@ + + + + DMA + + + DMA: What it is and How it Works + + Copyright © 1995,1997 &a.uhclem;, All Rights + Reserved. 10 December 1996. Last Update 8 October + 1997. + + Direct Memory Access (DMA) is a method of allowing data to be moved + from one location to another in a computer without intervention from the + central processor (CPU). + + The way that the DMA function is implemented varies between computer + architectures, so this discussion will limit itself to the + implementation and workings of the DMA subsystem on the IBM Personal + Computer (PC), the IBM PC/AT and all of its successors and + clones. + + The PC DMA subsystem is based on the Intel 8237 DMA controller. The + 8237 contains four DMA channels that can be programmed independently and + any one of the channels may be active at any moment. These channels are + numbered 0, 1, 2 and 3. Starting with the PC/AT, IBM added a second + 8237 chip, and numbered those channels 4, 5, 6 and 7. + + The original DMA controller (0, 1, 2 and 3) moves one byte in each + transfer. The second DMA controller (4, 5, 6, and 7) moves 16-bits from + two adjacent memory locations in each transfer, with the first byte + always coming from an even-numbered address. The two controllers are + identical components and the difference in transfer size is caused by + the way the second controller is wired into the system. + + The 8237 has two electrical signals for each channel, named DRQ and + -DACK. There are additional signals with the names HRQ (Hold Request), + HLDA (Hold Acknowledge), -EOP (End of Process), and the bus control + signals -MEMR (Memory Read), -MEMW (Memory Write), -IOR (I/O Read), and + -IOW (I/O Write). + + The 8237 DMA is known as a fly-by DMA controller. + This means that the data being moved from one location to another does + not pass through the DMA chip and is not stored in the DMA chip. + Subsequently, the DMA can only transfer data between an I/O port and a + memory address, but not between two I/O ports or two memory + locations. + + + The 8237 does allow two channels to be connected together to allow + memory-to-memory DMA operations in a non-fly-by mode, + but nobody in the PC industry uses this scarce resource this way since + it is faster to move data between memory locations using the + CPU. + + + In the PC architecture, each DMA channel is normally activated only + when the hardware that uses a given DMA channel requests a transfer by + asserting the DRQ line for that channel. + + + A Sample DMA transfer + + Here is an example of the steps that occur to cause and perform a + DMA transfer. In this example, the floppy disk controller (FDC) has + just read a byte from a diskette and wants the DMA to place it in + memory at location 0x00123456. The process begins by the FDC + asserting the DRQ2 signal (the DRQ line for DMA channel 2) to alert + the DMA controller. + + The DMA controller will note that the DRQ2 signal is asserted. The + DMA controller will then make sure that DMA channel 2 has been + programmed and is unmasked (enabled). The DMA controller also makes + sure that none of the other DMA channels are active or want to be + active and have a higher priority. Once these checks are complete, + the DMA asks the CPU to release the bus so that the DMA may use the + bus. The DMA requests the bus by asserting the HRQ signal which goes + to the CPU. + + The CPU detects the HRQ signal, and will complete executing the + current instruction. Once the processor has reached a state where it + can release the bus, it will. Now all of the signals normally + generated by the CPU (-MEMR, -MEMW, -IOR, -IOW and a few others) are + placed in a tri-stated condition (neither high or low) and then the + CPU asserts the HLDA signal which tells the DMA controller that it is + now in charge of the bus. + + Depending on the processor, the CPU may be able to execute a few + additional instructions now that it no longer has the bus, but the CPU + will eventually have to wait when it reaches an instruction that must + read something from memory that is not in the internal processor cache + or pipeline. + + Now that the DMA is in charge, the DMA activates its + -MEMR, -MEMW, -IOR, -IOW output signals, and the address outputs from + the DMA are set to 0x3456, which will be used to direct the byte that + is about to transferred to a specific memory location. + + The DMA will then let the device that requested the DMA transfer + know that the transfer is commencing. This is done by asserting the + -DACK signal, or in the case of the floppy disk controller, -DACK2 is + asserted. + + The floppy disk controller is now responsible for placing the byte + to be transferred on the bus Data lines. Unless the floppy controller + needs more time to get the data byte on the bus (and if the peripheral + does need more time it alerts the DMA via the READY signal), the DMA + will wait one DMA clock, and then de-assert the -MEMW and -IOR signals + so that the memory will latch and store the byte that was on the bus, + and the FDC will know that the byte has been transferred. + + Since the DMA cycle only transfers a single byte at a time, the + FDC now drops the DRQ2 signal, so the DMA knows that it is no longer + needed. The DMA will de-assert the -DACK2 signal, so that the FDC + knows it must stop placing data on the bus. + + The DMA will now check to see if any of the other DMA channels + have any work to do. If none of the channels have their DRQ lines + asserted, the DMA controller has completed its work and will now + tri-state the -MEMR, -MEMW, -IOR, -IOW and address signals. + + Finally, the DMA will de-assert the HRQ signal. The CPU sees + this, and de-asserts the HOLDA signal. Now the CPU activates its + -MEMR, -MEMW, -IOR, -IOW and address lines, and it resumes executing + instructions and accessing main memory and the peripherals. + + For a typical floppy disk sector, the above process is repeated + 512 times, once for each byte. Each time a byte is transferred, the + address register in the DMA is incremented and the counter in the DMA + that shows how many bytes are to be transferred is decremented. + + When the counter reaches zero, the DMA asserts the EOP signal, + which indicates that the counter has reached zero and no more data + will be transferred until the DMA controller is reprogrammed by the + CPU. This event is also called the Terminal Count (TC). There is only + one EOP signal, and since only DMA channel can be active at any + instant, the DMA channel that is currently active must be the DMA + channel that just completed its task. + + If a peripheral wants to generate an interrupt when the transfer + of a buffer is complete, it can test for its -DACKn signal and the EOP + signal both being asserted at the same time. When that happens, it + means the DMA will not transfer any more information for that + peripheral without intervention by the CPU. The peripheral can then + assert one of the interrupt signals to get the processors' attention. + In the PC architecture, the DMA chip itself is not capable of + generating an interrupt. The peripheral and its associated hardware + is responsible for generating any interrupt that occurs. + Subsequently, it is possible to have a peripheral that uses DMA but + does not use interrupts. + + It is important to understand that although the CPU always + releases the bus to the DMA when the DMA makes the request, this + action is invisible to both applications and the operating systems, + except for slight changes in the amount of time the processor takes to + execute instructions when the DMA is active. Subsequently, the + processor must poll the peripheral, poll the registers in the DMA + chip, or receive an interrupt from the peripheral to know for certain + when a DMA transfer has completed. + + + + DMA Page Registers and 16Meg address space limitations + + You may have noticed earlier that instead of the DMA setting the + address lines to 0x00123456 as we said earlier, the DMA only set + 0x3456. The reason for this takes a bit of explaining. + + When the original IBM PC was designed, IBM elected to use both DMA + and interrupt controller chips that were designed for use with the + 8085, an 8-bit processor with an address space of 16 bits (64K). + Since the IBM PC supported more than 64K of memory, something had to + be done to allow the DMA to read or write memory locations above the + 64K mark. What IBM did to solve this problem was to add an external + data latch for each DMA channel that holds the upper bits of the + address to be read to or written from. Whenever a DMA channel is + active, the contents of that latch are written to the address bus and + kept there until the DMA operation for the channel ends. IBM called + these latches Page Registers. + + So for our example above, the DMA would put the 0x3456 part of the + address on the bus, and the Page Register for DMA channel 2 would put + 0x0012xxxx on the bus. Together, these two values form the complete + address in memory that is to be accessed. + + Because the Page Register latch is independent of the DMA chip, + the area of memory to be read or written must not span a 64K physical + boundary. For example, if the DMA accesses memory location 0xffff, + after that transfer the DMA will then increment the address register + and the DMA will access the next byte at location 0x0000, not 0x10000. + The results of letting this happen are probably not intended. + + + Physical 64K boundaries should not be confused + with 8086-mode 64K Segments, which are created by + mathematically adding a segment register with an offset register. + Page Registers have no address overlap and are mathematically OR-ed + together. + + + To further complicate matters, the external DMA address latches on + the PC/AT hold only eight bits, so that gives us 8+16=24 bits, which + means that the DMA can only point at memory locations between 0 and + 16Meg. For newer computers that allow more than 16Meg of memory, the + standard PC-compatible DMA cannot access memory locations above + 16Meg. + + To get around this restriction, operating systems will reserve a + RAM buffer in an area below 16Meg that also does not span a physical + 64K boundary. Then the DMA will be programmed to transfer data from + the peripheral and into that buffer. Once the DMA has moved the data + into this buffer, the operating system will then copy the data from + the buffer to the address where the data is really supposed to be + stored. + + When writing data from an address above 16Meg to a DMA-based + peripheral, the data must be first copied from where it resides into a + buffer located below 16Meg, and then the DMA can copy the data from + the buffer to the hardware. In FreeBSD, these reserved buffers are + called Bounce Buffers. In the MS-DOS world, they are + sometimes called Smart Buffers. + + + A new implementation of the 8237, called the 82374, allows 16 + bits of page register to be specified, allows access to the entire + 32 bit address space, without the use of bounce buffers. + + + + + DMA Operational Modes and Settings + + The 8237 DMA can be operated in several modes. The main ones + are: + + + + Single + + + A single byte (or word) is transferred. The DMA must + release and re-acquire the bus for each additional byte. This is + commonly-used by devices that cannot transfer the entire block + of data immediately. The peripheral will request the DMA each + time it is ready for another transfer. + + The standard PC-compatible floppy disk controller (NEC 765) + only has a one-byte buffer, so it uses this mode. + + + + + Block/Demand + + + Once the DMA acquires the system bus, an entire block of + data is transferred, up to a maximum of 64K. If the peripheral + needs additional time, it can assert the READY signal to suspend + the transfer briefly. READY should not be used excessively, and + for slow peripheral transfers, the Single Transfer Mode should + be used instead. + + The difference between Block and Demand is that once a Block + transfer is started, it runs until the transfer count reaches + zero. DRQ only needs to be asserted until -DACK is asserted. + Demand Mode will transfer one more bytes until DRQ is + de-asserted, at which point the DMA suspends the transfer and + releases the bus back to the CPU. When DRQ is asserted later, + the transfer resumes where it was suspended. + + Older hard disk controllers used Demand Mode until CPU + speeds increased to the point that it was more efficient to + transfer the data using the CPU, particularly if the memory + locations used in the transfer were above the 16Meg mark. + + + + + Cascade + + + This mechanism allows a DMA channel to request the bus, but + then the attached peripheral device is responsible for placing + the addressing information on the bus instead of the DMA. This + is also used to implement a technique known as Bus + Mastering. + + When a DMA channel in Cascade Mode receives control of the + bus, the DMA does not place addresses and I/O control signals on + the bus like the DMA normally does when it is active. Instead, + the DMA only asserts the -DACK signal for the active DMA + channel. + + At this point it is up to the peripheral connected to that + DMA channel to provide address and bus control signals. The + peripheral has complete control over the system bus, and can do + reads and/or writes to any address below 16Meg. When the + peripheral is finished with the bus, it de-asserts the DRQ line, + and the DMA controller can then return control to the CPU or to + some other DMA channel. + + Cascade Mode can be used to chain multiple DMA controllers + together, and this is exactly what DMA Channel 4 is used for in + the PC architecture. When a peripheral requests the bus on DMA + channels 0, 1, 2 or 3, the slave DMA controller asserts HLDREQ, + but this wire is actually connected to DRQ4 on the primary DMA + controller instead of to the CPU. The primary DMA controller, + thinking it has work to do on Channel 4, requests the bus from + the CPU using HLDREQ signal. Once the CPU grants the bus to the + primary DMA controller, -DACK4 is asserted, and that wire is + actually connected to the HLDA signal on the slave DMA + controller. The slave DMA controller then transfers data for + the DMA channel that requested it (0, 1, 2 or 3), or the slave + DMA may grant the bus to a peripheral that wants to perform its + own bus-mastering, such as a SCSI controller. + + Because of this wiring arrangement, only DMA channels 0, 1, + 2, 3, 5, 6 and 7 are usable with peripherals on PC/AT + systems. + + + DMA channel 0 was reserved for refresh operations in early + IBM PC computers, but is generally available for use by + peripherals in modern systems. + + + When a peripheral is performing Bus Mastering, it is + important that the peripheral transmit data to or from memory + constantly while it holds the system bus. If the peripheral + cannot do this, it must release the bus frequently so that the + system can perform refresh operations on main memory. + + The Dynamic RAM used in all PCs for main memory must be + accessed frequently to keep the bits stored in the components + charged. Dynamic RAM essentially consists of + millions of capacitors with each one holding one bit of data. + These capacitors are charged with power to represent a + 1 or drained to represent a + 0. Because all capacitors leak, power must + be added at regular intervals to keep the 1 + values intact. The RAM chips actually handle the task of + pumping power back into all of the appropriate locations in RAM, + but they must be told when to do it by the rest of the computer + so that the refresh activity won't interfere with the computer + wanting to access RAM normally. If the computer is unable to + refresh memory, the contents of memory will become corrupted in + just a few milliseconds. + + Since memory read and write cycles count as + refresh cycles (a dynamic RAM refresh cycle is actually an + incomplete memory read cycle), as long as the peripheral + controller continues reading or writing data to sequential + memory locations, that action will refresh all of memory. + + Bus-mastering is found in some SCSI host interfaces and + other high-performance peripheral controllers. + + + + + Autoinitialize + + + This mode causes the DMA to perform Byte, Block or Demand + transfers, but when the DMA transfer counter reaches zero, the + counter and address are set back to where they were when the DMA + channel was originally programmed. This means that as long as + the peripheral requests transfers, they will be granted. It is + up to the CPU to move new data into the fixed buffer ahead of + where the DMA is about to transfer it when doing output + operations, and read new data out of the buffer behind where the + DMA is writing when doing input operations. + + This technique is frequently used on audio devices that have + small or no hardware sample buffers. There is + additional CPU overhead to manage this circular + buffer, but in some cases this may be the only way to eliminate + the latency that occurs when the DMA counter reaches zero and + the DMA stops transfers until it is reprogrammed. + + + + + + + Programming the DMA + + The DMA channel that is to be programmed should always be + masked before loading any settings. This is because the + hardware might unexpectedly assert the DRQ for that channel, and the + DMA might respond, even though not all of the parameters have been + loaded or updated. + + Once masked, the host must specify the direction of the transfer + (memory-to-I/O or I/O-to-memory), what mode of DMA operation is to be + used for the transfer (Single, Block, Demand, Cascade, etc), and + finally the address and length of the transfer are loaded. The length + that is loaded is one less than the amount you expect the DMA to + transfer. The LSB and MSB of the address and length are written to + the same 8-bit I/O port, so another port must be written to first to + guarantee that the DMA accepts the first byte as the LSB and the + second byte as the MSB of the length and address. + + Then, be sure to update the Page Register, which is external to + the DMA and is accessed through a different set of I/O ports. + + Once all the settings are ready, the DMA channel can be un-masked. + That DMA channel is now considered to be armed, and will + respond when the DRQ line for that channel is asserted. + + Refer to a hardware data book for precise programming details for + the 8237. You will also need to refer to the I/O port map for the PC + system, which describes where the DMA and Page Register ports are + located. A complete port map table is located below. + + + + DMA Port Map + + All systems based on the IBM-PC and PC/AT have the DMA hardware + located at the same I/O ports. The complete list is provided below. + Ports assigned to DMA Controller #2 are undefined on non-AT + designs. + + + 0x00–0x1f DMA Controller #1 (Channels 0, 1, 2 and + 3) + + DMA Address and Count Registers + + + + + + 0x00 + write + Channel 0 starting address + + + + 0x00 + read + Channel 0 current address + + + + 0x01 + write + Channel 0 starting word count + + + + 0x01 + read + Channel 0 remaining word count + + + + 0x02 + write + Channel 1 starting address + + + + 0x02 + read + Channel 1 current address + + + + 0x03 + write + Channel 1 starting word count + + + + 0x03 + read + Channel 1 remaining word count + + + + 0x04 + write + Channel 2 starting address + + + + 0x04 + read + Channel 2 current address + + + + 0x05 + write + Channel 2 starting word count + + + + 0x05 + read + Channel 2 remaining word count + + + + 0x06 + write + Channel 3 starting address + + + + 0x06 + read + Channel 3 current address + + + + 0x07 + write + Channel 3 starting word count + + + + 0x07 + read + Channel 3 remaining word count + + + + + + DMA Command Registers + + + + + + 0x08 + write + Command Register + + + + 0x08 + read + Status Register + + + + 0x09 + write + Request Register + + + + 0x09 + read + - + + + + 0x0a + write + Single Mask Register Bit + + + + 0x0a + read + - + + + + 0x0b + write + Mode Register + + + + 0x0b + read + - + + + + 0x0c + write + Clear LSB/MSB Flip-Flop + + + + 0x0c + read + - + + + + 0x0d + write + Master Clear/Reset + + + + 0x0d + read + Temporary Register (not available on newer + versions) + + + 0x0e + write + Clear Mask Register + + + + 0x0e + read + - + + + + 0x0f + write + Write All Mask Register Bits + + + + 0x0f + read + Read All Mask Register Bits (only in Intel + 82374) + + + + + + + + 0xc0–0xdf DMA Controller #2 (Channels 4, 5, 6 and + 7) + + DMA Address and Count Registers + + + + + + 0xc0 + write + Channel 4 starting address + + + + 0xc0 + read + Channel 4 current address + + + + 0xc2 + write + Channel 4 starting word count + + + + 0xc2 + read + Channel 4 remaining word count + + + + 0xc4 + write + Channel 5 starting address + + + + 0xc4 + read + Channel 5 current address + + + + 0xc6 + write + Channel 5 starting word count + + + + 0xc6 + read + Channel 5 remaining word count + + + + 0xc8 + write + Channel 6 starting address + + + + 0xc8 + read + Channel 6 current address + + + + 0xca + write + Channel 6 starting word count + + + + 0xca + read + Channel 6 remaining word count + + + + 0xcc + write + Channel 7 starting address + + + + 0xcc + read + Channel 7 current address + + + + 0xce + write + Channel 7 starting word count + + + + 0xce + read + Channel 7 remaining word count + + + + + + DMA Command Registers + + + + + + 0xd0 + write + Command Register + + + + 0xd0 + read + Status Register + + + + 0xd2 + write + Request Register + + + + 0xd2 + read + - + + + + 0xd4 + write + Single Mask Register Bit + + + + 0xd4 + read + - + + + + 0xd6 + write + Mode Register + + + + 0xd6 + read + - + + + + 0xd8 + write + Clear LSB/MSB Flip-Flop + + + + 0xd8 + read + - + + + + + + 0xda + write + Master Clear/Reset + + + + 0xda + read + Temporary Register (not present in Intel + 82374) + + + + 0xdc + write + Clear Mask Register + + + + 0xdc + read + - + + + + 0xde + write + Write All Mask Register Bits + + + + 0xdf + read + Read All Mask Register Bits (only in Intel + 82374) + + + + + + + + 0x80–0x9f DMA Page Registers + + + + + + 0x87 + r/w + Channel 0 Low byte (23-16) page Register + + + + 0x83 + r/w + Channel 1 Low byte (23-16) page Register + + + + 0x81 + r/w + Channel 2 Low byte (23-16) page Register + + + + 0x82 + r/w + Channel 3 Low byte (23-16) page Register + + + + 0x8b + r/w + Channel 5 Low byte (23-16) page Register + + + + 0x89 + r/w + Channel 6 Low byte (23-16) page Register + + + + 0x8a + r/w + Channel 7 Low byte (23-16) page Register + + + + 0x8f + r/w + Low byte page Refresh + + + + + + + + 0x400–0x4ff 82374 Enhanced DMA Registers + + The Intel 82374 EISA System Component (ESC) was introduced in + early 1996 and includes a DMA controller that provides a superset of + 8237 functionality as well as other PC-compatible core peripheral + components in a single package. This chip is targeted at both EISA + and PCI platforms, and provides modern DMA features like + scatter-gather, ring buffers as well as direct access by the system + DMA to all 32 bits of address space. + + If these features are used, code should also be included to + provide similar functionality in the previous 16 years worth of + PC-compatible computers. For compatibility reasons, some of the + 82374 registers must be programmed after + programming the traditional 8237 registers for each transfer. + Writing to a traditional 8237 register forces the contents of some + of the 82374 enhanced registers to zero to provide backward software + compatibility. + + + + + + 0x401 + r/w + Channel 0 High byte (bits 23-16) word count + + + + 0x403 + r/w + Channel 1 High byte (bits 23-16) word count + + + + 0x405 + r/w + Channel 2 High byte (bits 23-16) word count + + + + 0x407 + r/w + Channel 3 High byte (bits 23-16) word count + + + + 0x4c6 + r/w + Channel 5 High byte (bits 23-16) word count + + + + 0x4ca + r/w + Channel 6 High byte (bits 23-16) word count + + + + 0x4ce + r/w + Channel 7 High byte (bits 23-16) word count + + + + 0x487 + r/w + Channel 0 High byte (bits 31-24) page Register + + + + 0x483 + r/w + Channel 1 High byte (bits 31-24) page Register + + + + 0x481 + r/w + Channel 2 High byte (bits 31-24) page Register + + + + 0x482 + r/w + Channel 3 High byte (bits 31-24) page Register + + + + 0x48b + r/w + Channel 5 High byte (bits 31-24) page Register + + + + 0x489 + r/w + Channel 6 High byte (bits 31-24) page Register + + + + 0x48a + r/w + Channel 6 High byte (bits 31-24) page Register + + + + 0x48f + r/w + High byte page Refresh + + + + 0x4e0 + r/w + Channel 0 Stop Register (bits 7-2) + + + + 0x4e1 + r/w + Channel 0 Stop Register (bits 15-8) + + + + 0x4e2 + r/w + Channel 0 Stop Register (bits 23-16) + + + + 0x4e4 + r/w + Channel 1 Stop Register (bits 7-2) + + + + 0x4e5 + r/w + Channel 1 Stop Register (bits 15-8) + + + + 0x4e6 + r/w + Channel 1 Stop Register (bits 23-16) + + + + 0x4e8 + r/w + Channel 2 Stop Register (bits 7-2) + + + + 0x4e9 + r/w + Channel 2 Stop Register (bits 15-8) + + + + 0x4ea + r/w + Channel 2 Stop Register (bits 23-16) + + + + 0x4ec + r/w + Channel 3 Stop Register (bits 7-2) + + + + 0x4ed + r/w + Channel 3 Stop Register (bits 15-8) + + + + 0x4ee + r/w + Channel 3 Stop Register (bits 23-16) + + + + 0x4f4 + r/w + Channel 5 Stop Register (bits 7-2) + + + + 0x4f5 + r/w + Channel 5 Stop Register (bits 15-8) + + + + 0x4f6 + r/w + Channel 5 Stop Register (bits 23-16) + + + + 0x4f8 + r/w + Channel 6 Stop Register (bits 7-2) + + + + 0x4f9 + r/w + Channel 6 Stop Register (bits 15-8) + + + + 0x4fa + r/w + Channel 6 Stop Register (bits 23-16) + + + + 0x4fc + r/w + Channel 7 Stop Register (bits 7-2) + + + + 0x4fd + r/w + Channel 7 Stop Register (bits 15-8) + + + + 0x4fe + r/w + Channel 7 Stop Register (bits 23-16) + + + + 0x40a + write + Channels 0-3 Chaining Mode Register + + + + 0x40a + read + Channel Interrupt Status Register + + + + 0x4d4 + write + Channels 4-7 Chaining Mode Register + + + + 0x4d4 + read + Chaining Mode Status + + + + 0x40c + read + Chain Buffer Expiration Control Register + + + + 0x410 + write + Channel 0 Scatter-Gather Command Register + + + + 0x411 + write + Channel 1 Scatter-Gather Command Register + + + + 0x412 + write + Channel 2 Scatter-Gather Command Register + + + + 0x413 + write + Channel 3 Scatter-Gather Command Register + + + + 0x415 + write + Channel 5 Scatter-Gather Command Register + + + + 0x416 + write + Channel 6 Scatter-Gather Command Register + + + + 0x417 + write + Channel 7 Scatter-Gather Command Register + + + + 0x418 + read + Channel 0 Scatter-Gather Status Register + + + + 0x419 + read + Channel 1 Scatter-Gather Status Register + + + + 0x41a + read + Channel 2 Scatter-Gather Status Register + + + + 0x41b + read + Channel 3 Scatter-Gather Status Register + + + + 0x41d + read + Channel 5 Scatter-Gather Status Register + + + + 0x41e + read + Channel 5 Scatter-Gather Status Register + + + + 0x41f + read + Channel 7 Scatter-Gather Status Register + + + + 0x420-0x423 + r/w + Channel 0 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x424-0x427 + r/w + Channel 1 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x428-0x42b + r/w + Channel 2 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x42c-0x42f + r/w + Channel 3 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x434-0x437 + r/w + Channel 5 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x438-0x43b + r/w + Channel 6 Scatter-Gather Descriptor Table Pointer + Register + + + + 0x43c-0x43f + r/w + Channel 7 Scatter-Gather Descriptor Table Pointer + Register + + + + + + + + + \ No newline at end of file diff --git a/en_US.ISO_8859-1/books/developers-handbook/ipv6/chapter.sgml b/en_US.ISO_8859-1/books/developers-handbook/ipv6/chapter.sgml new file mode 100644 index 0000000000..e6028b6cb3 --- /dev/null +++ b/en_US.ISO_8859-1/books/developers-handbook/ipv6/chapter.sgml @@ -0,0 +1,1603 @@ + + + + IPv6 Internals + + + IPv6/IPsec Implementation + + Contributed by &a.shin;, 5 March + 2000. + + This section should explain IPv6 and IPsec related implementation + internals. These functionalities are derived from KAME project + + + IPv6 + + + Conformance + + The IPv6 related functions conforms, or tries to conform to + the latest set of IPv6 specifications. For future reference we list + some of the relevant documents below (NOTE: this + is not a complete list - this is too hard to maintain...). + + For details please refer to specific chapter in the document, + RFCs, manpages, or comments in the source code. + + Conformance tests have been performed on the KAME STABLE kit + at TAHI project. Results can be viewed at http://www.tahi.org/report/KAME/ + . We also attended Univ. of New Hampshire IOL tests (http://www.iol.unh.edu/) in the + past, with our past snapshots. + + + + RFC1639: FTP Operation Over Big Address Records + (FOOBAR) + + + RFC2428 is preferred over RFC1639. FTP clients will + first try RFC2428, then RFC1639 if failed. + + + + + + RFC1886: DNS Extensions to support IPv6 + + + + RFC1933: Transition Mechanisms for IPv6 Hosts and + Routers + + + IPv4 compatible address is not supported. + + + automatic tunneling (described in 4.3 of this RFC) is not + supported. + + + &man.gif.4; interface implements IPv[46]-over-IPv[46] + tunnel in a generic way, and it covers "configured tunnel" + described in the spec. See 23.5.1.5 + in this document for details. + + + + + + RFC1981: Path MTU Discovery for IPv6 + + + + RFC2080: RIPng for IPv6 + + + usr.sbin/route6d support this. + + + + + + RFC2292: Advanced Sockets API for IPv6 + + + For supported library functions/kernel APIs, see + sys/netinet6/ADVAPI. + + + + + + RFC2362: Protocol Independent Multicast-Sparse + Mode (PIM-SM) + + + RFC2362 defines packet formats for PIM-SM. + draft-ietf-pim-ipv6-01.txt is + written based on this. + + + + + + RFC2373: IPv6 Addressing Architecture + + + supports node required addresses, and conforms to + the scope requirement. + + + + + + RFC2374: An IPv6 Aggregatable Global Unicast Address + Format + + + supports 64-bit length of Interface ID. + + + + + + RFC2375: IPv6 Multicast Address Assignments + + + Userland applications use the well-known addresses + assigned in the RFC. + + + + + + RFC2428: FTP Extensions for IPv6 and NATs + + + RFC2428 is preferred over RFC1639. FTP clients will + first try RFC2428, then RFC1639 if failed. + + + + + + RFC2460: IPv6 specification + + + + RFC2461: Neighbor discovery for IPv6 + + + See 23.5.1.2 + in this document for details. + + + + + + RFC2462: IPv6 Stateless Address Autoconfiguration + + + See 23.5.1.4 in this + document for details. + + + + + + RFC2463: ICMPv6 for IPv6 specification + + + See 23.5.1.9 in this + document for details. + + + + + + RFC2464: Transmission of IPv6 Packets over Ethernet + Networks + + + + RFC2465: MIB for IPv6: Textual Conventions and General + Group + + + Necessary statistics are gathered by the kernel. Actual + IPv6 MIB support is provided as a patchkit for ucd-snmp. + + + + + + RFC2466: MIB for IPv6: ICMPv6 group + + + Necessary statistics are gathered by the kernel. Actual + IPv6 MIB support is provided as patchkit for ucd-snmp. + + + + + + RFC2467: Transmission of IPv6 Packets over FDDI + Networks + + + + RFC2497: Transmission of IPv6 packet over ARCnet + Networks + + + + RFC2553: Basic Socket Interface Extensions for IPv6 + + + IPv4 mapped address (3.7) and special behavior of IPv6 + wildcard bind socket (3.8) are supported. See 23.5.1.12 + in this document for details. + + + + + + RFC2675: IPv6 Jumbograms + + + See 23.5.1.7 in + this document for details. + + + + + + RFC2710: Multicast Listener Discovery for IPv6 + + + + RFC2711: IPv6 router alert option + + + + draft-ietf-ipngwg-router-renum-08: Router + renumbering for IPv6 + + + + draft-ietf-ipngwg-icmp-namelookups-02: + IPv6 Name Lookups Through ICMP + + + + draft-ietf-ipngwg-icmp-name-lookups-03: + IPv6 Name Lookups Through ICMP + + + + draft-ietf-pim-ipv6-01.txt: + PIM for IPv6 + + + &man.pim6dd.8; implements dense mode. &man.pim6sd.8; + implements sparse mode. + + + + + + draft-itojun-ipv6-tcp-to-anycast-00: + Disconnecting TCP connection toward IPv6 anycast address + + + + draft-yamamoto-wideipv6-comm-model-00 + + + + See 23.5.1.6 in this + document for details. + + + + + + draft-ietf-ipngwg-scopedaddr-format-00.txt + : An Extension of Format for IPv6 Scoped + Addresses + + + + + + Neighbor Discovery + + Neighbor Discovery is fairly stable. Currently Address + Resolution, Duplicated Address Detection, and Neighbor Unreachability + Detection are supported. In the near future we will be adding Proxy + Neighbor Advertisement support in the kernel and Unsolicited Neighbor + Advertisement transmission command as admin tool. + + If DAD fails, the address will be marked "duplicated" and + message will be generated to syslog (and usually to console). The + "duplicated" mark can be checked with &man.ifconfig.8;. It is + administrators' responsibility to check for and recover from DAD + failures. The behavior should be improved in the near future. + + Some of the network driver loops multicast packets back to itself, + even if instructed not to do so (especially in promiscuous mode). + In such cases DAD may fail, because DAD engine sees inbound NS packet + (actually from the node itself) and considers it as a sign of duplicate. + You may want to look at #if condition marked "heuristics" in + sys/netinet6/nd6_nbr.c:nd6_dad_timer() as workaround (note that the code + fragment in "heuristics" section is not spec conformant). + + Neighbor Discovery specification (RFC2461) does not talk about + neighbor cache handling in the following cases: + + + + when there was no neighbor cache entry, node + received unsolicited RS/NS/NA/redirect packet without + link-layer address + + + neighbor cache handling on medium without link-layer + address (we need a neighbor cache entry for IsRouter bit) + + + + For first case, we implemented workaround based on discussions + on IETF ipngwg mailing list. For more details, see the comments in + the source code and email thread started from (IPng 7155), dated + Feb 6 1999. + + IPv6 on-link determination rule (RFC2461) is quite different + from assumptions in BSD network code. At this moment, no on-link + determination rule is supported where default router list is empty + (RFC2461, section 5.2, last sentence in 2nd paragraph - note that + the spec misuse the word "host" and "node" in several places in + the section). + + To avoid possible DoS attacks and infinite loops, only 10 + options on ND packet is accepted now. Therefore, if you have 20 + prefix options attached to RA, only the first 10 prefixes will be + recognized. If this troubles you, please ask it on FREEBSD-CURRENT + mailing list and/or modify nd6_maxndopt in + sys/netinet6/nd6.c. If there are high demands + we may provide sysctl knob for the variable. + + + + Scope Index + + IPv6 uses scoped addresses. Therefore, it is very important to + specify scope index (interface index for link-local address, or + site index for site-local address) with an IPv6 address. Without + scope index, scoped IPv6 address is ambiguous to the kernel, and + kernel will not be able to determine the outbound interface for a + packet. + + Ordinary userland applications should use advanced API + (RFC2292) to specify scope index, or interface index. For similar + purpose, sin6_scope_id member in sockaddr_in6 structure is defined + in RFC2553. However, the semantics for sin6_scope_id is rather vague. + If you care about portability of your application, we suggest you to + use advanced API rather than sin6_scope_id. + + In the kernel, an interface index for link-local scoped address is + embedded into 2nd 16bit-word (3rd and 4th byte) in IPv6 address. For + example, you may see something like: + + + + fe80:1::200:f8ff:fe01:6317 + + + in the routing table and interface address structure (struct + in6_ifaddr). The address above is a link-local unicast address + which belongs to a network interface whose interface identifier is 1. + The embedded index enables us to identify IPv6 link local + addresses over multiple interfaces effectively and with only a + little code change. + + Routing daemons and configuration programs, like &man.route6d.8; + and &man.ifconfig.8;, will need to manipulate the "embedded" scope + index. These programs use routing sockets and ioctls (like + SIOCGIFADDR_IN6) and the kernel API will return IPv6 addresses with + 2nd 16bit-word filled in. The APIs are for manipulating kernel + internal structure. Programs that use these APIs have to be prepared + about differences in kernels anyway. + + When you specify scoped address to the command line, NEVER write + the embedded form (such as ff02:1::1 or fe80:2::fedc). This is not + supposed to work. Always use standard form, like ff02::1 or + fe80::fedc, with command line option for specifying interface (like + ping6 -I ne0 ff02::1). In general, if a command + does not have command line option to specify outgoing interface, that + command is not ready to accept scoped address. This may seem to be + opposite from IPv6's premise to support "dentist office" situation. + We believe that specifications need some improvements for this. + + Some of the userland tools support extended numeric IPv6 syntax, + as documented in + draft-ietf-ipngwg-scopedaddr-format-00.txt. You + can specify outgoing link, by using name of the outgoing interface + like "fe80::1%ne0". This way you will be able to specify link-local + scoped address without much trouble. + + To use this extension in your program, you'll need to use + &man.getaddrinfo.3;, and &man.getnameinfo.3; with NI_WITHSCOPEID. + The implementation currently assumes 1-to-1 relationship between a + link and an interface, which is stronger than what specs say. + + + + Plug and Play + + Most of the IPv6 stateless address autoconfiguration is implemented + in the kernel. Neighbor Discovery functions are implemented in the + kernel as a whole. Router Advertisement (RA) input for hosts is + implemented in the kernel. Router Solicitation (RS) output for + endhosts, RS input for routers, and RA output for routers are + implemented in the userland. + + + Assignment of link-local, and special addresses + + IPv6 link-local address is generated from IEEE802 address + (ethernet MAC address). Each of interface is assigned an IPv6 + link-local address automatically, when the interface becomes up + (IFF_UP). Also, direct route for the link-local address is added + to routing table. + + Here is an output of netstat command: + + +Internet6: +Destination Gateway Flags Netif Expire +fe80:1::%ed0/64 link#1 UC ed0 +fe80:2::%ep0/64 link#2 UC ep0 + + + Interfaces that has no IEEE802 address (pseudo interfaces + like tunnel interfaces, or ppp interfaces) will borrow IEEE802 + address from other interfaces, such as ethernet interfaces, + whenever possible. If there is no IEEE802 hardware attached, + last-resort pseudorandom value, which is from MD5(hostname), will + be used as source of link-local address. If it is not suitable + for your usage, you will need to configure the link-local address + manually. + + If an interface is not capable of handling IPv6 (such as + lack of multicast support), link-local address will not be + assigned to that interface. See section 2 for details. + + Each interface joins the solicited multicast address and the + link-local all-nodes multicast addresses (e.g. fe80::1:ff01:6317 + and ff02::1, respectively, on the link the interface is attached). + In addition to a link-local address, the loopback address (::1) + will be assigned to the loopback interface. Also, ::1/128 and + ff01::/32 are automatically added to routing table, and loopback + interface joins node-local multicast group ff01::1. + + + + Stateless address autoconfiguration on hosts + + In IPv6 specification, nodes are separated into two categories: + routers and hosts. Routers + forward packets addressed to others, hosts does not forward the + packets. net.inet6.ip6.forwarding defines whether this node is + router or host (router if it is 1, host if it is 0). + + When a host hears Router Advertisement from the router, a host + may autoconfigure itself by stateless address autoconfiguration. + This behavior can be controlled by net.inet6.ip6.accept_rtadv (host + autoconfigures itself if it is set to 1). By autoconfiguration, + network address prefix for the receiving interface (usually global + address prefix) is added. Default route is also configured. + Routers periodically generate Router Advertisement packets. To + request an adjacent router to generate RA packet, a host can + transmit Router Solicitation. To generate a RS packet at any time, + use the rtsol command. &man.rtsold.8; daemon is + also available. &man.rtsold.8; generates Router Solicitation whenever + necessary, and it works great for nomadic usage (notebooks/laptops). + If one wishes to ignore Router Advertisements, use sysctl to set + net.inet6.ip6.accept_rtadv to 0. + + To generate Router Advertisement from a router, use the + &man.rtadvd.8 daemon. + + Note that, IPv6 specification assumes the following items, and + nonconforming cases are left unspecified: + + + + Only hosts will listen to router advertisements + + + Hosts have single network interface (except loopback) + + + + Therefore, this is unwise to enable net.inet6.ip6.accept_rtadv + on routers, or multi-interface host. A misconfigured node can + behave strange (nonconforming configuration allowed for those who + would like to do some experiments). + + To summarize the sysctl knob: + + + accept_rtadv forwarding role of the node + --- --- --- + 0 0 host (to be manually configured) + 0 1 router + 1 0 autoconfigured host + (spec assumes that host has single + interface only, autoconfigured host + with multiple interface is + out-of-scope) + 1 1 invalid, or experimental + (out-of-scope of spec) + + + RFC2462 has validation rule against incoming RA prefix + information option, in 5.5.3 (e). This is to protect hosts from + malicious (or misconfigured) routers that advertise very short + prefix lifetime. There was an update from Jim Bound to ipngwg + mailing list (look for "(ipng 6712)" in the archive) and it is + implemented Jim's update. + + See 23.5.1.2 in + the document for relationship between DAD and + autoconfiguration. + + + + + Generic tunnel interface + + GIF (Generic InterFace) is a pseudo interface for configured + tunnel. Details are described in &man.gif.4;. Currently + + + + v6 in v6 + + + v6 in v4 + + + v4 in v6 + + + v4 in v4 + + + + are available. Use &man.gifconfig.8; to assign physical (outer) + source and destination address to gif interfaces. Configuration that + uses same address family for inner and outer IP header (v4 in v4, or + v6 in v6) is dangerous. It is very easy to configure interfaces and + routing tables to perform infinite level of tunneling. + Please be warned. + + gif can be configured to be ECN-friendly. See 23.5.4.5 for ECN-friendliness of + tunnels, and &man.gif.4; for how to configure. + + If you would like to configure an IPv4-in-IPv6 tunnel with gif + interface, read &man.gif.4; carefully. You will need to + remove IPv6 link-local address automatically assigned to the gif + interface. + + + + Source Address Selection + + Current source selection rule is scope oriented (there are some + exceptions - see below). For a given destination, a source IPv6 + address is selected by the following rule: + + + + If the source address is explicitly specified by + the user (e.g. via the advanced API), the specified address + is used. + + + + If there is an address assigned to the outgoing + interface (which is usually determined by looking up the + routing table) that has the same scope as the destination + address, the address is used. + + This is the most typical case. + + + + If there is no address that satisfies the above + condition, choose a global address assigned to one of + the interfaces on the sending node. + + + + If there is no address that satisfies the above condition, + and destination address is site local scope, choose a site local + address assigned to one of the interfaces on the sending node. + + + + + If there is no address that satisfies the above condition, + choose the address associated with the routing table entry for the + destination. This is the last resort, which may cause scope + violation. + + + + For instance, ::1 is selected for ff01::1, + fe80:1::200:f8ff:fe01:6317 for fe80:1::2a0:24ff:feab:839b (note + that embedded interface index - described in 23.5.1.3 - helps us + choose the right source address. Those embedded indices will not + be on the wire). If the outgoing interface has multiple address for + the scope, a source is selected longest match basis (rule 3). Suppose + 3ffe:501:808:1:200:f8ff:fe01:6317 and 3ffe:2001:9:124:200:f8ff:fe01:6317 + are given to the outgoing interface. 3ffe:501:808:1:200:f8ff:fe01:6317 + is chosen as the source for the destination 3ffe:501:800::1. + + Note that the above rule is not documented in the IPv6 spec. + It is considered "up to implementation" item. There are some cases + where we do not use the above rule. One example is connected TCP + session, and we use the address kept in tcb as the source. Another + example is source address for Neighbor Advertisement. Under the spec + (RFC2461 7.2.2) NA's source should be the target address of the + corresponding NS's target. In this case we follow the spec rather + than the above longest-match rule. + + For new connections (when rule 1 does not apply), deprecated + addresses (addresses with preferred lifetime = 0) will not be chosen + as source address if other choices are available. If no other choices + are available, deprecated address will be used as a last resort. If + there are multiple choice of deprecated addresses, the above scope + rule will be used to choose from those deprecated addresses. If you + would like to prohibit the use of deprecated address for some reason, + configure net.inet6.ip6.use_deprecated to 0. The issue related to + deprecated address is described in RFC2462 5.5.4 (NOTE: there is + some debate underway in IETF ipngwg on how to use "deprecated" + address). + + + + Jumbo Payload + + The Jumbo Payload hop-by-hop option is implemented and can + be used to send IPv6 packets with payloads longer than 65,535 octets. + But currently no physical interface whose MTU is more than 65,535 is + supported, so such payloads can be seen only on the loopback + interface (i.e. lo0). + + If you want to try jumbo payloads, you first have to reconfigure + the kernel so that the MTU of the loopback interface is more than + 65,535 bytes; add the following to the kernel configuration file: + + + options "LARGE_LOMTU" #To test jumbo payload + + + and recompile the new kernel. + + Then you can test jumbo payloads by the &man.ping6.8; command + with -b and -s options. The -b option must be specified to enlarge + the size of the socket buffer and the -s option specifies the length + of the packet, which should be more than 65,535. For example, + type as follows: + + + &prompt.user; ping6 -b 70000 -s 68000 ::1 + + + The IPv6 specification requires that the Jumbo Payload option + must not be used in a packet that carries a fragment header. If + this condition is broken, an ICMPv6 Parameter Problem message must + be sent to the sender. specification is followed, but you cannot + usually see an ICMPv6 error caused by this requirement. + + When an IPv6 packet is received, the frame length is checked and + compared to the length specified in the payload length field of the + IPv6 header or in the value of the Jumbo Payload option, if any. If + the former is shorter than the latter, the packet is discarded and + statistics are incremented. You can see the statistics as output of + &man.netstat.8; command with `-s -p ip6' option: + + + &prompt.user; netstat -s -p ip6 + ip6: + (snip) + 1 with data size < data length + + + So, kernel does not send an ICMPv6 error unless the erroneous + packet is an actual Jumbo Payload, that is, its packet size is more + than 65,535 bytes. As described above, currently no physical interface + with such a huge MTU is supported, so it rarely returns an + ICMPv6 error. + + TCP/UDP over jumbogram is not supported at this moment. This + is because we have no medium (other than loopback) to test this. + Contact us if you need this. + + IPsec does not work on jumbograms. This is due to some + specification twists in supporting AH with jumbograms (AH header + size influences payload length, and this makes it real hard to + authenticate inbound packet with jumbo payload option as well as AH). + + + There are fundamental issues in *BSD support for jumbograms. + We would like to address those, but we need more time to finalize + these. To name a few: + + + + mbuf pkthdr.len field is typed as "int" in 4.4BSD, so + it will not hold jumbogram with len > 2G on 32bit architecture + CPUs. If we would like to support jumbogram properly, the field + must be expanded to hold 4G + IPv6 header + link-layer header. + Therefore, it must be expanded to at least int64_t + (u_int32_t is NOT enough). + + + + We mistakingly use "int" to hold packet length in many + places. We need to convert them into larger integral type. + It needs a great care, as we may experience overflow during + packet length computation. + + + + We mistakingly check for ip6_plen field of IPv6 header + for packet payload length in various places. We should be + checking mbuf pkthdr.len instead. ip6_input() will perform + sanity check on jumbo payload option on input, and we can + safely use mbuf pkthdr.len afterwards. + + + + TCP code needs a careful update in bunch of places, of + course. + + + + + + Loop prevention in header processing + + IPv6 specification allows arbitrary number of extension headers + to be placed onto packets. If we implement IPv6 packet processing + code in the way BSD IPv4 code is implemented, kernel stack may + overflow due to long function call chain. sys/netinet6 code + is carefully designed to avoid kernel stack overflow. Because of + this, sys/netinet6 code defines its own protocol switch + structure, as "struct ip6protosw" (see + netinet6/ip6protosw.h). There is no such + update to IPv4 part (sys/netinet) for compatibility, but small + change is added to its pr_input() prototype. So "struct ipprotosw" + is also defined. Because of this, if you receive IPsec-over-IPv4 + packet with massive number of IPsec headers, kernel stack may blow + up. IPsec-over-IPv6 is okay. (Off-course, for those all IPsec + headers to be processed, each such IPsec header must pass each + IPsec check. So an anonymous attacker won't be able to do such an + attack.) + + + + ICMPv6 + + After RFC2463 was published, IETF ipngwg has decided to + disallow ICMPv6 error packet against ICMPv6 redirect, to prevent + ICMPv6 storm on a network medium. This is already implemented + into the kernel. + + + + Applications + + For userland programming, we support IPv6 socket API as + specified in RFC2553, RFC2292 and upcoming internet drafts. + + TCP/UDP over IPv6 is available and quite stable. You can + enjoy &man.telnet.1;, &man.ftp.1;, &man.rlogin.1;, &man.rsh.1;, + &man.ssh.1, etc. These applications are protocol independent. + That is, they automatically chooses IPv4 or IPv6 according to DNS. + + + + + Kernel Internals + + While ip_forward() calls ip_output(), ip6_forward() directly + calls if_output() since routers must not divide IPv6 packets into + fragments. + + ICMPv6 should contain the original packet as long as possible + up to 1280. UDP6/IP6 port unreach, for instance, should contain + all extension headers and the *unchanged* UDP6 and IP6 headers. + So, all IP6 functions except TCP never convert network byte + order into host byte order, to save the original packet. + + tcp_input(), udp6_input() and icmp6_input() can't assume that + IP6 header is preceding the transport headers due to extension + headers. So, in6_cksum() was implemented to handle packets whose IP6 + header and transport header is not continuous. TCP/IP6 nor UDP6/IP6 + header structure don't exist for checksum calculation. + + To process IP6 header, extension headers and transport headers + easily, network drivers are now required to store packets in one + internal mbuf or one or more external mbufs. A typical old driver + prepares two internal mbufs for 96 - 204 bytes data, however, now + such packet data is stored in one external mbuf. + + netstat -s -p ip6 tells you whether or not + your driver conforms such requirement. In the following example, + "cce0" violates the requirement. (For more information, refer to + Section 2.) + + + Mbuf statistics: + 317 one mbuf + two or more mbuf:: + lo0 = 8 + cce0 = 10 + 3282 one ext mbuf + 0 two or more ext mbuf + + + Each input function calls IP6_EXTHDR_CHECK in the beginning to + check if the region between IP6 and its header is continuous. + IP6_EXTHDR_CHECK calls m_pullup() only if the mbuf has M_LOOP flag, + that is, the packet comes from the loopback interface. m_pullup() + is never called for packets coming from physical network interfaces. + + + Both IP and IP6 reassemble functions never call m_pullup(). + + + + IPv4 mapped address and IPv6 wildcard socket + + RFC2553 describes IPv4 mapped address (3.7) and special behavior + of IPv6 wildcard bind socket (3.8). The spec allows you to: + + + Accept IPv4 connections by AF_INET6 wildcard bind + socket. + + + Transmit IPv4 packet over AF_INET6 socket by using + special form of the address like ::ffff:10.1.1.1. + + + + but the spec itself is very complicated and does not specify + how the socket layer should behave. Here we call the former one + "listening side" and the latter one "initiating side", for + reference purposes. + + You can perform wildcard bind on both of the address families, + on the same port. + + The following table show the behavior of FreeBSD 4.x. + + + listening side initiating side + (AF_INET6 wildcard (connection to ::ffff:10.1.1.1) + socket gets IPv4 conn.) + --- --- +FreeBSD 4.x configurable supported + default: enabled + + + The following sections will give you more details, and how you can + configure the behavior. + + Comments on listening side: + + It looks that RFC2553 talks too little on wildcard bind issue, + especially on the port space issue, failure mode and relationship + between AF_INET/INET6 wildcard bind. There can be several separate + interpretation for this RFC which conform to it but behaves differently. + So, to implement portable application you should assume nothing + about the behavior in the kernel. Using &man.getaddrinfo.3; is the + safest way. Port number space and wildcard bind issues were discussed + in detail on ipv6imp mailing list, in mid March 1999 and it looks + that there's no concrete consensus (means, up to implementers). + You may want to check the mailing list archives. + + If a server application would like to accept IPv4 and IPv6 + connections, there will be two alternatives. + + One is using AF_INET and AF_INET6 socket (you'll need two + sockets). Use &man.getaddrinfo.3; with AI_PASSIVE into ai_flags, + and &man.socket.2; and &man.bind.2; to all the addresses returned. + By opening multiple sockets, you can accept connections onto the + socket with proper address family. IPv4 connections will be + accepted by AF_INET socket, and IPv6 connections will be accepted + by AF_INET6 socket. + + Another way is using one AF_INET6 wildcard bind socket. Use + &man.getaddrinfo.3; with AI_PASSIVE into ai_flags and with + AF_INET6 into ai_family, and set the 1st argument hostname to + NULL. And &man.socket.2; and &man.bind.2; to the address returned. + (should be IPv6 unspecified addr). You can accept either of IPv4 + and IPv6 packet via this one socket. + + To support only IPv6 traffic on AF_INET6 wildcard binded socket + portably, always check the peer address when a connection is made + toward AF_INET6 listening socket. If the address is IPv4 mapped + address, you may want to reject the connection. You can check the + condition by using IN6_IS_ADDR_V4MAPPED() macro. + + To resolve this issue more easily, there is system dependent + &man.setsockopt.2; option, IPV6_BINDV6ONLY, used like below. + + + int on; + + setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY, + (char *)&on, sizeof (on)) < 0)); + + + When this call succeed, then this socket only receive IPv6 + packets. + + Comments on initiating side: + + Advise to application implementers: to implement a portable + IPv6 application (which works on multiple IPv6 kernels), we believe + that the following is the key to the success: + + + + NEVER hardcode AF_INET nor AF_INET6. + + + + Use &man.getaddrinfo.3; and &man.getnameinfo.3; + throughout the system. Never use gethostby*(), getaddrby*(), + inet_*() or getipnodeby*(). (To update existing applications + to be IPv6 aware easily, sometime getipnodeby*() will be + useful. But if possible, try to rewrite the code to use + &man.getaddrinfo.3; and &man.getnameinfo.3;.) + + + + If you would like to connect to destination, use + &man.getaddrinfo.3; and try all the destination returned, + like &man.telnet.1; does. + + + + Some of the IPv6 stack is shipped with buggy + &man.getaddrinfo.3;. Ship a minimal working version with + your application and use that as last resort. + + + + If you would like to use AF_INET6 socket for both IPv4 and + IPv6 outgoing connection, you will need to use &man.getipnodebyname.3;. + When you would like to update your existing application to be IPv6 + aware with minimal effort, this approach might be chosen. But please + note that it is a temporal solution, because &man.getipnodebyname.3; + itself is not recommended as it does not handle scoped IPv6 addresses + at all. For IPv6 name resolution, &man.getaddrinfo.3; is the + preferred API. So you should rewrite your application to use + &man.getaddrinfo.3;, when you get the time to do it. + + When writing applications that make outgoing connections, + story goes much simpler if you treat AF_INET and AF_INET6 as totally + separate address family. {set,get}sockopt issue goes simpler, + DNS issue will be made simpler. We do not recommend you to rely + upon IPv4 mapped address. + + + unified tcp and inpcb code + + FreeBSD 4.x uses shared tcp code between IPv4 and IPv6 + (from sys/netinet/tcp*) and separate udp4/6 code. It uses + unified inpcb structure. + + The platform can be configured to support IPv4 mapped address. + Kernel configuration is summarized as follows: + + + + By default, AF_INET6 socket will grab IPv4 + connections in certain condition, and can initiate + connection to IPv4 destination embedded in IPv4 mapped + IPv6 address. + + + + You can disable it on entire system with sysctl like + below. + + + sysctl -w net.inet6.ip6.mapped_addr=0 + + + + + + + listening side + + Each socket can be configured to support special AF_INET6 + wildcard bind (enabled by default). You can disable it on + each socket basis with &man.setsockopt.2; like below. + + + int on; + + setsockopt(s, IPPROTO_IPV6, IPV6_BINDV6ONLY, + (char *)&on, sizeof (on)) < 0)); + + + Wildcard AF_INET6 socket grabs IPv4 connection if and only + if the following conditions are satisfied: + + + + there's no AF_INET socket that matches the IPv4 + connection + + + + the AF_INET6 socket is configured to accept IPv4 + traffic, i.e. getsockopt(IPV6_BINDV6ONLY) returns 0. + + + + There's no problem with open/close ordering. + + + + initiating side + + FreeBSD 4.x supports outgoing connection to IPv4 mapped + address (::ffff:10.1.1.1), if the node is configured to support + IPv4 mapped address. + + + + + + sockaddr_storage + + When RFC2553 was about to be finalized, there was discussion on + how struct sockaddr_storage members are named. One proposal is to + prepend "__" to the members (like "__ss_len") as they should not be + touched. The other proposal was that don't prepend it (like "ss_len") + as we need to touch those members directly. There was no clear + consensus on it. + + As a result, RFC2553 defines struct sockaddr_storage as + follows: + + + struct sockaddr_storage { + u_char __ss_len; /* address length */ + u_char __ss_family; /* address family */ + /* and bunch of padding */ + }; + + + On the contrary, XNET draft defines as follows: + + + struct sockaddr_storage { + u_char ss_len; /* address length */ + u_char ss_family; /* address family */ + /* and bunch of padding */ + }; + + + In December 1999, it was agreed that RFC2553bis should pick + the latter (XNET) definition. + + Current implementation conforms to XNET definition, based on + RFC2553bis discussion. + + If you look at multiple IPv6 implementations, you will be able + to see both definitions. As an userland programmer, the most + portable way of dealing with it is to: + + + + ensure ss_family and/or ss_len are available on the + platform, by using GNU autoconf, + + + + have -Dss_family=__ss_family to unify all occurrences + (including header file) into __ss_family, or + + + + never touch __ss_family. cast to sockaddr * and use sa_family + like: + + + struct sockaddr_storage ss; + family = ((struct sockaddr *)&ss)->sa_family + + + + + + + + + Network Drivers + + Now following two items are required to be supported by standard + drivers: + + + + mbuf clustering requirement. In this stable release, we + changed MINCLSIZE into MHLEN+1 for all the operating systems + in order to make all the drivers behave as we expect. + + + + multicast. If &man.ifmcstat.8; yields no multicast group for + a interface, that interface has to be patched. + + + + If any of the driver don't support the requirements, then + the driver can't be used for IPv6 and/or IPsec communication. If + you find any problem with your card using IPv6/IPsec, then, please + report it to freebsd-bugs@FreeBSD.org. + + (NOTE: In the past we required all PCMCIA drivers to have a + call to in6_ifattach(). We have no such requirement any more) + + + + Translator + + We categorize IPv4/IPv6 translator into 4 types: + + + + Translator A --- It is used in the early + stage of transition to make it possible to establish a + connection from an IPv6 host in an IPv6 island to an IPv4 host + in the IPv4 ocean. + + + + Translator B --- It is used in the early + stage of transition to make it possible to establish a connection + from an IPv4 host in the IPv4 ocean to an IPv6 host in an + IPv6 island. + + + + Translator C --- It is used in the late + stage of transition to make it possible to establish a + connection from an IPv4 host in an IPv4 island to an IPv6 host + in the IPv6 ocean. + + + + Translator D --- It is used in the late + stage of transition to make it possible to establish a + connection from an IPv6 host in the IPv6 ocean to an IPv4 host + in an IPv4 island. + + + + TCP relay translator for category A is supported. This is called + "FAITH". We also provide IP header translator for category A. + (The latter is not yet put into FreeBSD 4.x yet.) + + + FAITH TCP relay translator + + FAITH system uses TCP relay daemon called &man.faithd.8; helped + by the kernel. FAITH will reserve an IPv6 address prefix, and relay + TCP connection toward that prefix to IPv4 destination. + + For example, if the reserved IPv6 prefix is + 3ffe:0501:0200:ffff::, and the IPv6 destination for TCP connection + is 3ffe:0501:0200:ffff::163.221.202.12, the connection will be + relayed toward IPv4 destination 163.221.202.12. + + + destination IPv4 node (163.221.202.12) + ^ + | IPv4 tcp toward 163.221.202.12 + FAITH-relay dual stack node + ^ + | IPv6 TCP toward 3ffe:0501:0200:ffff::163.221.202.12 + source IPv6 node + + + &man.faithd.8; must be invoked on FAITH-relay dual stack + node. + + For more details, consult + src/usr.sbin/faithd/README + + + + + IPsec + + IPsec is mainly organized by three components. + + + + Policy Management + + + + Key Management + + + + AH and ESP handling + + + + + Policy Management + + The kernel implements experimental policy management code. + There are two way to manage security policy. One is to configure + per-socket policy using &man.setsockopt.2;. In this cases, policy + configuration is described in &man.ipsec.set.policy.3;. The other + is to configure kernel packet filter-based policy using PF_KEY + interface, via &man.setkey.8;. + + The policy entry is not re-ordered with its + indexes, so the order of entry when you add is very significant. + + + + Key Management + + The key management code implemented in this kit (sys/netkey) + is a home-brew PFKEY v2 implementation. This conforms to RFC2367. + + + The home-brew IKE daemon, "racoon" is included in the + kit (kame/kame/racoon). Basically you'll need to run racoon as + daemon, then setup a policy to require keys (like + ping -P 'out ipsec esp/transport//use'). + The kernel will contact racoon daemon as necessary to exchange + keys. + + + + AH and ESP handling + + IPsec module is implemented as "hooks" to the standard IPv4/IPv6 + processing. When sending a packet, ip{,6}_output() checks if ESP/AH + processing is required by checking if a matching SPD (Security + Policy Database) is found. If ESP/AH is needed, + {esp,ah}{4,6}_output() will be called and mbuf will be updated + accordingly. When a packet is received, {esp,ah}4_input() will be + called based on protocol number, i.e. (*inetsw[proto])(). + {esp,ah}4_input() will decrypt/check authenticity of the packet, + and strips off daisy-chained header and padding for ESP/AH. It is + safe to strip off the ESP/AH header on packet reception, since we + will never use the received packet in "as is" form. + + By using ESP/AH, TCP4/6 effective data segment size will be + affected by extra daisy-chained headers inserted by ESP/AH. Our + code takes care of the case. + + Basic crypto functions can be found in directory "sys/crypto". + ESP/AH transform are listed in {esp,ah}_core.c with wrapper functions. + If you wish to add some algorithm, add wrapper function in + {esp,ah}_core.c, and add your crypto algorithm code into + sys/crypto. + + Tunnel mode is partially supported in this release, with the + following restrictions: + + + + IPsec tunnel is not combined with GIF generic tunneling + interface. It needs a great care because we may create an + infinite loop between ip_output() and tunnelifp->if_output(). + Opinion varies if it is better to unify them, or not. + + + + MTU and Don't Fragment bit (IPv4) considerations need more + checking, but basically works fine. + + + + Authentication model for AH tunnel must be revisited. + We'll need to improve the policy management engine, + eventually. + + + + + + Conformance to RFCs and IDs + + The IPsec code in the kernel conforms (or, tries to conform) + to the following standards: + + "old IPsec" specification documented in + rfc182[5-9].txt + + "new IPsec" specification documented in + rfc240[1-6].txt, + rfc241[01].txt, rfc2451.txt + and draft-mcdonald-simple-ipsec-api-01.txt + (draft expired, but you can take from + ftp://ftp.kame.net/pub/internet-drafts/). + (NOTE: IKE specifications, rfc241[7-9].txt are + implemented in userland, as "racoon" IKE daemon) + + Currently supported algorithms are: + + + old IPsec AH + + + null crypto checksum (no document, just for + debugging) + + + keyed MD5 with 128bit crypto checksum + (rfc1828.txt) + + + keyed SHA1 with 128bit crypto checksum + (no document) + + + HMAC MD5 with 128bit crypto checksum + (rfc2085.txt) + + + HMAC SHA1 with 128bit crypto checksum + (no document) + + + + + + old IPsec ESP + + + null encryption (no document, similar to + rfc2410.txt) + + + DES-CBC mode (rfc1829.txt) + + + + + + new IPsec AH + + + null crypto checksum (no document, + just for debugging) + + + keyed MD5 with 96bit crypto checksum + (no document) + + + keyed SHA1 with 96bit crypto checksum + (no document) + + + HMAC MD5 with 96bit crypto checksum + (rfc2403.txt) + + + HMAC SHA1 with 96bit crypto checksum + (rfc2404.txt) + + + + + + new IPsec ESP + + + null encryption + (rfc2410.txt) + + + DES-CBC with derived IV + (draft-ietf-ipsec-ciph-des-derived-01.txt, + draft expired) + + + DES-CBC with explicit IV + (rfc2405.txt) + + + 3DES-CBC with explicit IV + (rfc2451.txt) + + + BLOWFISH CBC + (rfc2451.txt) + + + CAST128 CBC + (rfc2451.txt) + + + RC5 CBC + (rfc2451.txt) + + + each of the above can be combined with: + + + ESP authentication with HMAC-MD5(96bit) + + + ESP authentication with HMAC-SHA1(96bit) + + + + + + + + The following algorithms are NOT supported: + + + + old IPsec AH + + + + HMAC MD5 with 128bit crypto checksum + 64bit + replay prevention (rfc2085.txt) + + + keyed SHA1 with 160bit crypto checksum + 32bit padding + (rfc1852.txt) + + + + + + + IPsec (in kernel) and IKE (in userland as "racoon") has been + tested at several interoperability test events, and it is known to + interoperate with many other implementations well. Also, current + IPsec implementation as quite wide coverage for IPsec crypto + algorithms documented in RFC (we cover algorithms without intellectual + property issues only). + + + + ECN consideration on IPsec tunnels + + ECN-friendly IPsec tunnel is supported as described in + draft-ipsec-ecn-00.txt. + + Normal IPsec tunnel is described in RFC2401. On encapsulation, + IPv4 TOS field (or, IPv6 traffic class field) will be copied from inner + IP header to outer IP header. On decapsulation outer IP header + will be simply dropped. The decapsulation rule is not compatible + with ECN, since ECN bit on the outer IP TOS/traffic class field will be + lost. + + To make IPsec tunnel ECN-friendly, we should modify encapsulation + and decapsulation procedure. This is described in + http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt, + chapter 3. + + IPsec tunnel implementation can give you three behaviors, by + setting net.inet.ipsec.ecn (or net.inet6.ipsec6.ecn) to some + value: + + + + RFC2401: no consideration for ECN (sysctl value -1) + + + ECN forbidden (sysctl value 0) + + + ECN allowed (sysctl value 1) + + + + Note that the behavior is configurable in per-node manner, + not per-SA manner (draft-ipsec-ecn-00 wants per-SA configuration, + but it looks too much for me). + + The behavior is summarized as follows (see source code for + more detail): + + + + encapsulate decapsulate + --- --- +RFC2401 copy all TOS bits drop TOS bits on outer + from inner to outer. (use inner TOS bits as is) + +ECN forbidden copy TOS bits except for ECN drop TOS bits on outer + (masked with 0xfc) from inner (use inner TOS bits as is) + to outer. set ECN bits to 0. + +ECN allowed copy TOS bits except for ECN use inner TOS bits with some + CE (masked with 0xfe) from change. if outer ECN CE bit + inner to outer. is 1, enable ECN CE bit on + set ECN CE bit to 0. the inner. + + + + General strategy for configuration is as follows: + + + if both IPsec tunnel endpoint are capable of ECN-friendly + behavior, you'd better configure both end to "ECN allowed" + (sysctl value 1). + + + if the other end is very strict about TOS bit, use "RFC2401" + (sysctl value -1). + + + in other cases, use "ECN forbidden" (sysctl value 0). + + + + The default behavior is "ECN forbidden" (sysctl value 0). + + For more information, please refer to: + + + http://www.aciri.org/floyd/papers/draft-ipsec-ecn-00.txt, + RFC2481 (Explicit Congestion Notification), + src/sys/netinet6/{ah,esp}_input.c + + (Thanks goes to Kenjiro Cho kjc@csl.sony.co.jp + for detailed analysis) + + + + Interoperability + + Here are (some of) platforms that KAME code have tested + IPsec/IKE interoperability in the past. Note that both ends may + have modified their implementation, so use the following list just + for reference purposes. + + Altiga, Ashley-laurent (vpcom.com), Data Fellows (F-Secure), + Ericsson ACC, FreeS/WAN, HITACHI, IBM AIX, IIJ, Intel, + Microsoft WinNT, NIST (linux IPsec + plutoplus), Netscreen, OpenBSD, + RedCreek, Routerware, SSH, Secure Computing, Soliton, Toshiba, + VPNet, Yamaha RT100i + + + + + + diff --git a/en_US.ISO_8859-1/books/developers-handbook/vm/chapter.sgml b/en_US.ISO_8859-1/books/developers-handbook/vm/chapter.sgml new file mode 100644 index 0000000000..4710973d5f --- /dev/null +++ b/en_US.ISO_8859-1/books/developers-handbook/vm/chapter.sgml @@ -0,0 +1,255 @@ + + + + Virtual Memory System + + + The FreeBSD VM System + + Contributed by &a.dillon;. 6 Feb 1999 + + + Management of physical + memory—<literal>vm_page_t</literal> + + Physical memory is managed on a page-by-page basis through the + vm_page_t structure. Pages of physical memory are + categorized through the placement of their respective + vm_page_t structures on one of several paging + queues. + + A page can be in a wired, active, inactive, cache, or free state. + Except for the wired state, the page is typically placed in a doubly + link list queue representing the state that it is in. Wired pages + are not placed on any queue. + + FreeBSD implements a more involved paging queue for cached and + free pages in order to implement page coloring. Each of these states + involves multiple queues arranged according to the size of the + processor's L1 and L2 caches. When a new page needs to be allocated, + FreeBSD attempts to obtain one that is reasonably well aligned from + the point of view of the L1 and L2 caches relative to the VM object + the page is being allocated for. + + Additionally, a page may be held with a reference count or locked + with a busy count. The VM system also implements an ultimate + locked state for a page using the PG_BUSY bit in the page's + flags. + + In general terms, each of the paging queues operates in a LRU + fashion. A page is typically placed in a wired or active state + initially. When wired, the page is usually associated with a page + table somewhere. The VM system ages the page by scanning pages in a + more active paging queue (LRU) in order to move them to a less-active + paging queue. Pages that get moved into the cache are still + associated with a VM object but are candidates for immediate reuse. + Pages in the free queue are truly free. FreeBSD attempts to minimize + the number of pages in the free queue, but a certain minimum number of + truly free pages must be maintained in order to accommodate page + allocation at interrupt time. + + If a process attempts to access a page that does not exist in its + page table but does exist in one of the paging queues ( such as the + inactive or cache queues), a relatively inexpensive page reactivation + fault occurs which causes the page to be reactivated. If the page + does not exist in system memory at all, the process must block while + the page is brought in from disk. + + FreeBSD dynamically tunes its paging queues and attempts to + maintain reasonable ratios of pages in the various queues as well as + attempts to maintain a reasonable breakdown of clean v.s. dirty pages. + The amount of rebalancing that occurs depends on the system's memory + load. This rebalancing is implemented by the pageout daemon and + involves laundering dirty pages (syncing them with their backing + store), noticing when pages are activity referenced (resetting their + position in the LRU queues or moving them between queues), migrating + pages between queues when the queues are out of balance, and so forth. + FreeBSD's VM system is willing to take a reasonable number of + reactivation page faults to determine how active or how idle a page + actually is. This leads to better decisions being made as to when to + launder or swap-out a page. + + + + The unified buffer + cache—<literal>vm_object_t</literal> + + FreeBSD implements the idea of a generic VM object. + VM objects can be associated with backing store of various + types—unbacked, swap-backed, physical device-backed, or + file-backed storage. Since the filesystem uses the same VM objects to + manage in-core data relating to files, the result is a unified buffer + cache. + + VM objects can be shadowed. That is, they + can be stacked on top of each other. For example, you might have a + swap-backed VM object stacked on top of a file-backed VM object in + order to implement a MAP_PRIVATE mmap()ing. This stacking is also + used to implement various sharing properties, including, + copy-on-write, for forked address spaces. + + It should be noted that a vm_page_t can only be + associated with one VM object at a time. The VM object shadowing + implements the perceived sharing of the same page across multiple + instances. + + + + Filesystem I/O—<literal>struct buf</literal> + + vnode-backed VM objects, such as file-backed objects, generally + need to maintain their own clean/dirty info independent from the VM + system's idea of clean/dirty. For example, when the VM system decides + to synchronize a physical page to its backing store, the VM system + needs to mark the page clean before the page is actually written to + its backing s tore. Additionally, filesystems need to be able to map + portions of a file or file metadata into KVM in order to operate on + it. + + The entities used to manage this are known as filesystem buffers, + struct buf's, and also known as + bp's. When a filesystem needs to operate on a + portion of a VM object, it typically maps part of the object into a + struct buf and the maps the pages in the struct buf into KVM. In the + same manner, disk I/O is typically issued by mapping portions of + objects into buffer structures and then issuing the I/O on the buffer + structures. The underlying vm_page_t's are typically busied for the + duration of the I/O. Filesystem buffers also have their own notion of + being busy, which is useful to filesystem driver code which would + rather operate on filesystem buffers instead of hard VM pages. + + FreeBSD reserves a limited amount of KVM to hold mappings from + struct bufs, but it should be made clear that this KVM is used solely + to hold mappings and does not limit the ability to cache data. + Physical data caching is strictly a function of + vm_page_t's, not filesystem buffers. However, + since filesystem buffers are used placehold I/O, they do inherently + limit the amount of concurrent I/O possible. As there are usually a + few thousand filesystem buffers available, this is not usually a + problem. + + + + Mapping Page Tables - vm_map_t, vm_entry_t + + FreeBSD separates the physical page table topology from the VM + system. All hard per-process page tables can be reconstructed on the + fly and are usually considered throwaway. Special page tables such as + those managing KVM are typically permanently preallocated. These page + tables are not throwaway. + + FreeBSD associates portions of vm_objects with address ranges in + virtual memory through vm_map_t and + vm_entry_t structures. Page tables are directly + synthesized from the + vm_map_t/vm_entry_t/ + vm_object_t hierarchy. Remember when I mentioned + that physical pages are only directly associated with a + vm_object. Well, that isn't quite true. + vm_page_t's are also linked into page tables that + they are actively associated with. One vm_page_t + can be linked into several pmaps, as page tables + are called. However, the hierarchical association holds so all + references to the same page in the same object reference the same + vm_page_t and thus give us buffer cache unification + across the board. + + + + KVM Memory Mapping + + FreeBSD uses KVM to hold various kernel structures. The single + largest entity held in KVM is the filesystem buffer cache. That is, + mappings relating to struct buf entities. + + Unlike Linux, FreeBSD does NOT map all of physical memory into + KVM. This means that FreeBSD can handle memory configurations up to + 4G on 32 bit platforms. In fact, if the mmu were capable of it, + FreeBSD could theoretically handle memory configurations up to 8TB on + a 32 bit platform. However, since most 32 bit platforms are only + capable of mapping 4GB of ram, this is a moot point. + + KVM is managed through several mechanisms. The main mechanism + used to manage KVM is the zone allocator. The + zone allocator takes a chunk of KVM and splits it up into + constant-sized blocks of memory in order to allocate a specific type + of structure. You can use vmstat -m to get an + overview of current KVM utilization broken down by zone. + + + + Tuning the FreeBSD VM system + + A concerted effort has been made to make the FreeBSD kernel + dynamically tune itself. Typically you do not need to mess with + anything beyond the maxusers and + NMBCLUSTERS kernel config options. That is, kernel + compilation options specified in (typically) + /usr/src/sys/i386/conf/CONFIG_FILE. + A description of all available kernel configuration options can be + found in /usr/src/sys/i386/conf/LINT. + + In a large system configuration you may wish to increase + maxusers. Values typically range from 10 to 128. + Note that raising maxusers too high can cause the + system to overflow available KVM resulting in unpredictable operation. + It is better to leave maxusers at some reasonable number and add other + options, such as NMBCLUSTERS, to increase specific + resources. + + If your system is going to use the network heavily, you may want + to increase NMBCLUSTERS. Typical values range from + 1024 to 4096. + + The NBUF parameter is also traditionally used + to scale the system. This parameter determines the amount of KVA the + system can use to map filesystem buffers for I/O. Note that this + parameter has nothing whatsoever to do with the unified buffer cache! + This parameter is dynamically tuned in 3.0-CURRENT and later kernels + and should generally not be adjusted manually. We recommend that you + not try to specify an NBUF + parameter. Let the system pick it. Too small a value can result in + extremely inefficient filesystem operation while too large a value can + starve the page queues by causing too many pages to become wired + down. + + By default, FreeBSD kernels are not optimized. You can set + debugging and optimization flags with the + makeoptions directive in the kernel configuration. + Note that you should not use unless you can + accommodate the large (typically 7 MB+) kernels that result. + + makeoptions DEBUG="-g" +makeoptions COPTFLAGS="-O -pipe" + + Sysctl provides a way to tune kernel parameters at run-time. You + typically do not need to mess with any of the sysctl variables, + especially the VM related ones. + + Run time VM and system tuning is relatively straightforward. + First, use softupdates on your UFS/FFS filesystems whenever possible. + /usr/src/contrib/sys/softupdates/README contains + instructions (and restrictions) on how to configure it up. + + Second, configure sufficient swap. You should have a swap + partition configured on each physical disk, up to four, even on your + work disks. You should have at least 2x the swap space + as you have main memory, and possibly even more if you do not have a + lot of memory. You should also size your swap partition based on the + maximum memory configuration you ever intend to put on the machine so + you do not have to repartition your disks later on. If you want to be + able to accommodate a crash dump, your first swap partition must be at + least as large as main memory and /var/crash must + have sufficient free space to hold the dump. + + NFS-based swap is perfectly acceptable on -4.x or later systems, + but you must be aware that the NFS server will take the brunt of the + paging load. + + + +